What do I do with RAID?

[dashbarron]

I was using DrivePool because I couldn't get a RAID on an old server.  Now I have a fancy server with hot swappable bays and a raid controller.

What is the best practice to use RAID with stablebit? Should I just ignore the RAID setup and just Drivepool and all drives not in a RAID? Do I put them in the RAID as...mirrors? Or if money is no object do I just make my hardware raids and ignore Stablebit?

[Shane]

The big advantage of hardware RAID (2+) is parity-based duplication and healing (remember to use scheduled scrubbing!), and complete transparency to your file system, and RAID 5 in particular can give great performance (especially for reads).

The big disadvantage of hardware RAID (2+) is that if more drives die than you've got parity for, you lose the entire array instead of just the failed drives.

The big advantage of DrivePool is you don't have to care about keeping all your drives the same size and type, you can remove drives as you please without having to rebuild the whole bloody array from scratch, adding a drive doesn't require (background) parity recomputation, you can freely mix SSD and HDD and use the former as cache, you can even add non-local drives via CloudDrive.

The big disadvantage of DrivePool is that if any bitrot happens it has no self-healing capability.

Late edit/add: one other disadvantage of DrivePool is that it can't create a file larger than the free space of its largest drive (since it doesn't use striping), which is something to keep in mind if you plan to be working with extremely large files.

So if money was no object and I had a big honking bay of drives, no particulars withstanding I'd build multiple small sets of RAID 5 (three to five disks to an array) and pool them with DrivePool (duplication x2) to get the best of both worlds while minimising the downsides of each.

One drive dies? No problem, pool's still writable, thanks RAID 5!
An array dies? No problem, pool's still readable, thanks DrivePool!
File bitrot happens? No problem, file's healable, thanks RAID 5!

[dashbarron]

How do you prevent bitrot, or what can you do to circumvent it?

I used Drivepool for years and loved it, but I dont honestly know if I would have known if something went wrong unless I went to open my stuff and it was gone/inaccessible. If I go Drivepool route, I need to pull everything out of RAIDS first?

 

[Shane]

Preventing: Do regular (monthly? YMMV) full reads of drives (so that their own firmware can find/refresh iffy blocks) if you don't already have something doing that for you (e.g. RAID scheduled scrubbing or StableBit Scanner or the like). Don't use SSDs as long-term offline storage, it might work but they're meant to be powered on at least semi-regularly to keep the data charge fresh (at least until someone invents a better SSD). Use ECC RAM (and a mainboard+cpu that actually supports it, which can be a pain). Don't overclock.

Circumventing: Use some form of parity-based checking, e.g. RAID 5 disk array or a file system that does parity (e.g. zfs for linux) or utility software such as SnapRAID or MultiPar etc. Keep backups!

TLDR: best (IMO) is use ECC RAM on a mainboard+cpu that supports it, use RAID5 arrays with a monthly scheduled scrub, have backups.

Since I'm firmly not in the "money no object" category, however, I mostly just rely on duplication (3x for critical docs), backups, Scanner and MultiPar (I keep meaning to use SnapRAID...).

"If I go Drivepool route, I need to pull everything out of RAIDS first?"

No. DrivePool can add a hardware RAID array to the pool as if it was a single disk (because that's how arrays present themselves to the OS), so you don't need to pull everything out of an existing array if you're happy to just add the array to the pool.