Using DrivePool's Duplication, Drive Usage Limiter & Ordered File Placement for a 3-2-1 Backup Strategy

[Ariolander]

Right now I have 5 identical drives 4 are used for DrivePool and 1 is used for Parity via SnapRaid, While SnapRaid is "good enough" for media storage and things that I don't REALLY care about / can always re-download I do want to protect my personal / work documents with a proper 3-2-1 backup strategy, I am wondering if using Drive Pool's built in Duplication and Ordered File Placement balancer are what I need to do this job versus specialized backup software.

As I am new to DrivePool and the balancers can sometimes be confusing I am wondering if someone can sanity check my Balancers and double check my 3-2-1 strategy is working as intended and I am not mistakenly overlooking some files or mis-configuring my balancers.

Drive Usage Limiter = I have my normal 4x 8TB HDD Drive Pool set to accept both Duplicated AND Unduplicated data.
In my front panel 2.5" hot-swap bay I have a smaller 1x 500GB SSD set to accept Duplicated data only.

Ordered File Placement = "Prioritize the placement of duplicated files" is checked. Unduplicated is left unchecked.
"Control new file placement and move the existing files into the pre-defined order" is selected.
The top priority on the Duplicated tab is the 1x SSD.

Every other balancer except StableBit Scanner are disabled as not to interfere with SnapRaid.

My /homes/ folder where myself and my family keep our personal files along with anything else deemed to be important is set to be Duplicated.

From there I target the SSD's hidden DrivePool folder using my rclone to my NAS and my cloud provider's cloud sync software.

With the above balancers enabled does that mean at least 1 copy of every important / flagged to be Duplicated will always be on the SSD?
If true does that mean the other copy of the file will also be on one of the mechanical HDDs and thus also protected by SnapRaid?
Does it matter that the SSD (500GB) is significantly smaller than the mechanical HDDs (8TB) in the pool if I expect only a minority of files to be flagged to be Duplicated? IE the balancer won't try to skip the SSD because it is 75% full vs the HDDs which are 25% full.

When I setup Rsync to my NAS to add a 2nd machine does that mean I only need to target DrivePool's hidden folder on my SSD to make sure all my important stuff is on the 2nd machine? Same for the cloud, if I back everything up to a cloud provider can I just  target DrivePool's hidden folder on my SSD rather than the individual folders with important files?

[Shane]

Yes. Still, it's always a good idea to initially (and occasionally) test that any redundancy/backup method is actually doing what it's meant to do, so I do recommend using Everything or similar to see if your files are indeed going where they're supposed to when they're supposed to. I know from personal experience that discovering "oh, the backups I thought I had stopped working a month ago" is not a good feeling! 

[Shane]

#1 "With the above balancers enabled does that mean at least 1 copy of every important / flagged to be Duplicated will always be on the SSD?"

Not always, because if you ever have more duplicated files than can fit on the SSD then some of your duplicated files will end up with their duplicates only on the HDDs instead.

#2 "If true does that mean the other copy of the file will also be on one of the mechanical HDDs and thus also protected by SnapRaid?

Yes. I would suggest you make sure that real-time duplication is enabled if you want immediate redundancy.

#3 "Does it matter that the SSD (500GB) is significantly smaller than the mechanical HDDs (8TB) in the pool if I expect only a minority of files to be flagged to be Duplicated? IE the balancer won't try to skip the SSD because it is 75% full vs the HDDs which are 25% full."

Both the Drive Usage Limiter and Ordered File Placement balancers have sliders to control the percentage of use at which they will skip the drive(s).

Note that there are utilities (e.g. Everything by Voidtools) which can let you see in real-time where your files are, so you could use such to monitor how DrivePool is behaving in response to your settings.

#4 "When I setup Rsync to my NAS to add a 2nd machine does that mean I only need to target DrivePool's hidden folder on my SSD to make sure all my important stuff is on the 2nd machine? Same for the cloud, if I back everything up to a cloud provider can I just  target DrivePool's hidden folder on my SSD rather than the individual folders with important files?"

You could do that, yes. Just keep in mind the answer to #1 - if your SSD "overflows" then targeting it directly rather than the pool would miss any excess files.

The 3-2-1 backup strategy (3 copies, 2 storage types, 1 offsite) presumes that the copies are independent - i.e. that altering or deleting any copy will not affect the others (or at least not immediately). DrivePool's duplicates are not independent - if you alter or delete a file in the pool, all duplicates are simultaneously altered or deleted.

DrivePool's duplication feature technically provides Redundancy (the ability of your files to remain in a usable state despite the loss of a storage device) not Backup (the ability of your files to be recreated as they existed at a previous point in time).

If you are backing up your pool to both a NAS and a cloud provider on a regular basis then that would be where your 3-2-1 backup strategy would be achieved.

[Ariolander]

Excellent! So the basics at least of my strategy make sense and should work fine as long as the  #1 Ordered File Placement duplicate-only drive doesn't overflow?

Since my OneDrive storage limit is 1TB anyways I am thinking of upgrading the SSD to 1 TB to match. Worse comes to worse if I ever outgrow the 1TB I will switch to a HDD and rethink my cloud provider, but OneDrive is free as part of my Office365 subscription so I am using that for now since it doesn't cost anything extra.

My NAS is my old Synology I outgrew that I am planning to sync to. I will probably enable snapshots so I can rollback files on the NAS if I want to add file versioning. If I have more money, maybe down the road, I can add a 2nd remote Synology as a HyperBackup endpoint so I stop counting my working files as one of the "3" in 3-2-1.

[Christopher (Drashna)]

I agree with shane in that untested backup/redundancy is just catastrophic data loss in disguise/waiting. "trust but verify".

Also, knowing how to restore/repair things rather than having to desperately figure that out ***when*** (not if) things fail is always a good idea (and not just because panicking can lead to even more data loss, but that's definitely a factor too!)