Odd file corruption, StableBit says no problem with the drive. Any explanation?

[ben98923]

I shot some photos, put the SD card into my PC, copied the images to my main drive (SSD), then because they were important family photos and I was feeling paranoid I binary diffed the source and the copy. One file was different (of the ~320 files, ~1.7 GB). I then viewed the files in a hex diff viewer, and I attach that image.

I've seen corruption occur before but never like this. There are maybe 40 bytes total that have changed, but they aren't adjacent, though they are in the same region of the file (each change has gaps of 15-40 bytes).

I recopied the file from the SD and the new copy matched the SD copy, as it should.

But, I'm just stumped and now super paranoid, I just diffed copied files out of an abundance of caution, not because I actually expected this sort of problem to happen. And certainly not a problem like this, maybe a truncated file or something.  How could this be?  

I scanned all my drives (not SD) with StableBit and it says all are fine.  But how can I ever trust a file write (or read) again knowing this happened?  Anyone have a comforting explanation?

Thanks!

[Christopher (Drashna)]

I have a number of answers here, but nothing good. Meaning they won't necessarily comfort you. 

 

First, StableBit Scanner doesn't check the disk integrity. 

The surface scan reads each sector on the drive, to see if it's readable.  If that has issues, then we flag it.  If it reads fine, then it's "okay".   

The file system scan we run is essentially just a "chkdsk x:" pass (no /f, no /r, etc). 

 

This means that we are not able to detect corruption in the disk, as we're not checking for it.

 

 

So if the data changed or got corrupted, then the issue isn't with readability. 

 

 

 

 

That said, the most likely place that the corruption occurred is either in memory (the most common), or on the SD card itself.  

Low quality or old cards may do this, as they may not be storing the data properly.

Additionally, the file system in use may be a factor here.  

 

Additionally, have you tried comparing the file repeatedly (eg, open the file, close it, unmount the SD card, and repeat)?  

Namely to see if the data is coming back consistently.  If it's not, the SD card's controller may be buggy, and silently corrupting data. 

[ben98923]

I have a number of answers here, but nothing good. Meaning they won't necessarily comfort you. 

 

First, StableBit Scanner doesn't check the disk integrity. 

The surface scan reads each sector on the drive, to see if it's readable.  If that has issues, then we flag it.  If it reads fine, then it's "okay".   

The file system scan we run is essentially just a "chkdsk x:" pass (no /f, no /r, etc). 

 

This means that we are not able to detect corruption in the disk, as we're not checking for it.

 

 

So if the data changed or got corrupted, then the issue isn't with readability. 

 

 

 

 

That said, the most likely place that the corruption occurred is either in memory (the most common), or on the SD card itself.  

Low quality or old cards may do this, as they may not be storing the data properly.

Additionally, the file system in use may be a factor here.  

 

Additionally, have you tried comparing the file repeatedly (eg, open the file, close it, unmount the SD card, and repeat)?  

Namely to see if the data is coming back consistently.  If it's not, the SD card's controller may be buggy, and silently corrupting data. 

 

I feel like an idiot, all these years I thought chkdsk /r read/overwrote/read each block to see not just that it could be read from but that it could be written to and subsequently held the value expected.

 

I'm totally stumped on this one.  I let the computer run overnight doing the built in Windows memory test (choosing all the various write tests) and it came up fine.  So, presumably it's not memory.  Sadly I over-wrote the local file and the SD card files soon after the event so I can't go back and check that.  I was hoping JPEGs had some sort of CRC or something but they don't, so I can't tell which copy was the bit-perfect version.

 

On a possibly related, possibly not, note...  StableBit Scanner just reported a reallocated sector on that drive (a 1TB mSATA Samsung 850 EVO), it's the second one in 3-6 months (don't recall exactly, perception of time being funny that way).  The other time I rightly or wrongly didn't think much of it since it followed a hard reset after lockup and I imagined maybe that contributed.  But this time nothing proceeded it.  If I've got 2 reallocated sectors is there any reason to worry?  or suspect a problem that could be playing a role in the file corruption issue?

 

Thanks Chris!

[Christopher (Drashna)]

No, it doesn't.  It checks certain data structures to see if there is damage. It also checks the NTFS logs to see if there are any sectors that are marked as "problematic" or bad. 

 

the old SCANDISK utility did check everything, but it's incredibly outdated and no longer included. 

 

However, the CHKDSK "/r" and "/b" flags are much, much more thorough, and close to this behavior.

 

Additionally a full (non-quick) format writes zeros to the entire partition, which is a good way to check readability. 

 

 

 

As for the reallocated sectors, on a spinning disk, I'd say be wary, But on an SSD, it may be fine.  As long as the value isn't rapidly increasing, there shouldn't be anything to worry about. 

 

And while it *may* be a cause of corruption, it's highly unlikely. Generally, the reallocated sector means that the system was able to handle the issue properly.  If you're seeing ... any value of "uncorrectable sector count"  (other than zero), then that could (would) be a likely cause.   

Generally, you'll see a number of unreadable/damaged sectors (as reported by the surface scan) equal to that value, and you may see it climb while a surface scan occurs.