silent bit rot protection

[Andreas W]

Hi

I am starting building out my home storage system and learning as I go.
Below is my current thinking/question

I will use drivepool and with all pool/files duplicated.
I will have additional backups (cloud and local)
I feel ok against drive-failures, against entire system failing etc.

My question is - how do I protect against silent bitrot
I know drivepool via integrated scanner will find blocks that are bad.
But how do I protect against silent bit flip.
I know its very rare.
But still - would be nice to know how to protect against it

Question does drivepool secure against "silent bitrot"
Lets say a file fileA is stored in diskpool and dubplicated on two drives and one of the drives a bit flips, changes value from say 1 to 0. At this point in time I will have
* disk1: fileA (correct file)
* disk2: fileA (corrupt file)

Question: Does diskpool have a "diskpool check" command that checks all files (all duplicates) and detects corrupted files?
Question: Next read of file1 - will it be the fileA from disk1 or from disk2? or?
Question: Will diskpool detect that fileA is now different on disk1 vs disk2?

Essentially
* will diskpool detect a underlying bit-root like this?
* will it in addition to detect it be able to fix it?

If not - sounds like a great value-add feature - ex
everytime diskpool writes a file it also calculates and stores a checksum of the file
* option to enable: write verification: after write - read both duplicates and verify checksum

dispool check command 
* reads all files (all duplicates) and verifies that checksum is as expected

diskpool file read
* option to enable: read verification: everytime diskpool reads a file - verifies that read file checksum is as expected
* option to enable: read verification all duplicated: everytime diskpool reads a file - reads all duplicates and verifies that read files checksum are as expected

 

/Andreas

[Shane]

Hi Andreas,

My question is - how do I protect against silent bitrot

The two main risks for silent bitrot are "while in RAM" and "sitting on disk".

For the former, ECC RAM for your PC (and making sure your PC will actually support and use ECC RAM in the first place).

For the latter, you can rely on hardware (e.g. RAID 5/6 cards, NAS boxes like Synology) and/or software (e.g. for Windows, md5deep for checking, SnapRaid or PAR2/MultiPar for checking and repairs; for linux there's also better filesystems like ZFS or bcachefs; or linux-based operating systems like Unraid which simplify the use of those filesystems, etc).

For use with StableBit DrivePool, my personal recommendation would be ECC RAM + StableBit Scanner + either (SnapRAID) or (md5deep +optionally PAR2/MultiPar). And if you have very large pools consider whether to use hardware RAID 5/6 instead of SnapRAID et al.

Plus, of course, backups. Backups should go without saying. 

Question does drivepool secure against "silent bitrot"

Not actively. Duplication may "protect" against bitrot in the sense that you can compare the duplicates yourself - with x2 duplication you would need to eyeball inspect or rely on checksums you'd made with other tools, while with x3 duplication or more you could also decide the bad one based on the "odd one out" - but DrivePool does not have any functions that actively assist with this.

Question: Does diskpool have a "diskpool check" command that checks all files (all duplicates) and detects corrupted files?

It does not.

If not - sounds like a great value-add feature - ex
everytime diskpool writes a file it also calculates and stores a checksum of the file

I would like to see a feature along those lines too.

* option to enable: read verification: everytime diskpool reads a file - verifies that read file checksum is as expected

You'd see a very noticeable performance hit; regular (e.g. monthly) background scans would I think be better.

[Andreas W]

Hi

Thanks for answer!
ECC ram - good idea - will make sure to use such.

Current installation will be windows (later down the road might switch to linux and zfz or similar)

For now I will go with ECC ram + drivepool + stablebit scanner + snapraid/md5deep (think I will start with trying this one: https://github.com/marcopaganini/bitrot + a bit of scripting to have it run weekly on all drives)


Updated Drivepool feature wish

* at every write, calculate and write checksum
* have a check/scan command that can be run daily/weekly/monthly uses checksums and verifies data integrity

/Andreas