Balancing as a Backup

[amateurphotographer]

Hi

Just installed a further 2 x 6tb drives into my home server so now running with 4 x 6tb.

My current system is setup as a 2 drive pool that has a duplication factor of 2 so that 2 copies are stored one on each hard drive.  So currently no physical backup, all files are duplicated as the system is set to "balance immediatelty".

The question I have is what is the difference between the following, or am I thinking it right that there is no difference.

1. Set the balance to be every day at say 01:00
2. Installing some backup software or sync software such as EaseUS EverySync to sync from the pool to a backup everyday at 01:00

I am just thinking that Stablebit is tried and true and works like a charm so why go hunting and buying other software if I can configure Stablebit to work in the same way.

Is it possible to balance on drive pool with another or do I need to get creative with file placement etc?

Thanks

Paul

[Umfriend]

One difference is that if you accidentally delete a file, it will be gone. DP is _not_ a backup and you're setting yourself up for trouble when trying to use it as such.

[amateurphotographer]

Thanks, am confused.

If I save a file to the pool using the above example then I only have the single pool copy until 1am when it is copies to a backup drive.  if i set the balance to be at 1am only then surely its the same.

Later on

If I delete the file from the pool during the dayon then it remains on the backup drive until the pool balances at 1am the next day.

[Jaga]

You should consider using Drivepool's Hierarchical Pooling feature.

Let's call your existing data pool Pool 1.  Make another identical pool using the two new drives you have, we'll call that Pool 2.  Disable all mirroring on both pools so you only have a duplication factor of 1.

Then make a third pool called Pool Mirror, which consists of Pools 1 and 2.  Enable full mirroring on Pool Mirror, and you have full redundancy.  Drivepool will auto-balance between your old pool and your new one.

 

Additionally, Umfriend is correct - despite being a mirror of live data, pools are not backups where you instance the data and can go back to get old copies (or deleted ones) later.  For that I might recommend something like Snapraid.  You'd want to purchase another disk, set it up as Snapraid's parity, and then sync on a regular basis.  If you lost a file/folder/drive you could easily use the parity information Snapraid made to restore what you needed - back to the last date of syncing.

[Umfriend]

On 28/04/2018 at 11:22 AM, amateurphotographer said:

Thanks, am confused.

If I save a file to the pool using the above example then I only have the single pool copy until 1am when it is copies to a backup drive.  if i set the balance to be at 1am only then surely its the same.

Later on

If I delete the file from the pool during the dayon then it remains on the backup drive until the pool balances at 1am the next day.

No. If you have x2 duplication and you save a file then it will be duplicated instantly. If you delete it, both duplicates are deleted instantly.

Balancing is the procedure by which the usage of the pooled HDDs is made to comply with the balancing rules. As an example. DP writes, by default, to the HDD with the most space available. If you have two HDDs with equal space available and data gets deleted from one of these then balancing would fill it up again.

A simple advice: Do not assume DP has anything to do with backups. It does not. It helps you increasing uptime and easing recovery in case of a HDD failure (assuming you have at least x2 duplication and not 2 HDD failures at the same time or during recovery) and allows for a BFVHDD (Bif Fing Virtual HDD) but that is it.

For backups, that is copies of files which protect you against HDD failure, system failure, fire and theft etc., look for real backup solutions. If, for instance, you are running a server OS, use Server Backup for instance (and rotate backup HDDs offsite).

[Julius]

Umfriend and amateurphotographer, I don't know what your ages are, but I'm 52 and been using computers to store data for about 36 years now, and let me just throw this in here;

The times when I 'accidentally' deleted a file or files and lost them that way can be counted on one hand. Especially since we have the RECYCLE BIN or Trash can that problem has disappeared for me, if I even ever accidentally do hit delete (in TotalCommander or DxO Photolab or some other program) and confirm its deletion, because the confirmation is still not automatic for me. So, I don't know about you guys, but that barely ever happens, if at all, in life.

What *did* happen a lot more in the past 36 years is storage media suddenly deciding to crap out on me , floppies or drives becoming entirely or partly inaccessible. I've even lost 16 GB of stored data when an at the time very expensive Intel SLC SSD decided to stop working a few years back, it went from 1 to 0. No software out there could get it back, not even SpinRite, probably a charged spark borked the controller chip. Luckily it was mostly an OS installed boot-drive, but even so, it warned me about not trusting on the high MTBF of SSDs too much.

What makes backup policies suck the most for me is the fact that they're almost always dated or too old, not current or not current enough. HDDs or SSDs breaking down almost always happens at moments you least hoped they would, and in ways you never expect. This, for me, is where DrivePool comes in. It has already saved me from losing data several times by having a direct dual or tripple written down copy of what I was working on. These days I have big data passing through my storage media, and it's really comforting to know I can access the functional left-over storage whenever one or more stop working. RAID storage methods for redundancy are horribly overrated, in my experience. Oh the times I've tried to restore data from broken RAID-arrays probably outnumbers those for just broken disks, and the time it took, good grief, incomparable to having StableBit DrivePool create standalone copies.

So, backup or not, for me backup policies only exist in order to have data off-site and/or offline.

[Christopher (Drashna)]

On 4/27/2018 at 10:13 AM, amateurphotographer said:

I am just thinking that Stablebit is tried and true and works like a charm so why go hunting and buying other software if I can configure Stablebit to work in the same way.

StableBit DrivePool is redundancy, not backup, as others have mentioned. 

THat means that if something gets deleted, modified or even corrupted, you're SOL.  

Backup is for versioning, really.  Storing files that you've deleted, as well as different versions of the file. 

So, really, it's good to have both, but for very different reasons. 

4 hours ago, Julius said:

The times when I 'accidentally' deleted a file or files and lost them that way can be counted on one hand. Especially since we have the RECYCLE BIN or Trash can that problem has disappeared for me, if I even ever accidentally do hit delete (in TotalCommander or DxO Photolab or some other program) and confirm its deletion, because the confirmation is still not automatic for me. So, I don't know about you guys, but that barely ever happens, if at all, in life.

I've done it a LOT.   especially, as I access everything over the network. And in this case, there is no recycle bin.  Everything gets deleted. Period. 

Also, as I mentioned above, it's not just about deletion, but modification, as well.  

And data recovery can take hours or days, depending.

 

But even if it doesn't happen often, it's absolutely worth making the distinction, so that somebody doesn't get bitten by that distinction. 

 

 

[Julius]

3 hours ago, Christopher (Drashna) said:

Also, as I mentioned above, it's not just about deletion, but modification, as well.  

And data recovery can take hours or days, depending.

 

That can go two ways:

Say you make auto backups every hour, so the backup can potentially be 59 minutes old if you've got a bad day ;-)

You're working on something, you save the last bit of work, you work on it some more, then a disk stops working. Phew, you still have the other disk in the pool, where it did write your last work to. A backup would be useless, because it misses all that last work. Plus, restoring from the pool dir on that disk in the pool is a LOT easier than from any backup solution, most likely.

[Umfriend]

Julius, I am 48 and have been using tapes and disks to store data since 1982 (Tandy TRS-80 Model III).

Yes, DP may save you from HDD failure due to its duplication but that is about the only fail-safe it provides (same as RAID and I agree on the recovery issues with RAID. DP is, to my mind, far easier at the expense of additional space (compared to RAID 5/6)). Yes, a good backup policy can be cumbersome to implement and, especially, adhere to. Nevertheless, modifications, deletions over the network (as Christopher said, no recycle bin then), fire and theft etc. are not covered by DP. It may be that it is safe enough for you but I would never advise that DP is good as a backup.

In my case, I run WHS2011 (will move to WSE someday), with DP x2. I also run a daily Server Backup which is scheduled and needs no manual intervention (and I could set it up to backup every hour if I wanted). The Backup HDDs are rotated offsite on a weekly basis. Can I still lose data? Sure. But at least I can restore *most* at all times.

An alternative, of course, mught be the cloud. For example, DropBox retains all versions if you have a professional subscription (including deleted files) and it is offsite at the same time. However, I do not like the cloud personally.

Edit: Ransomware! I forgot to mention ransomware.

[Julius]

On 5/3/2018 at 11:28 AM, Umfriend said:

fire and theft etc. are not covered by DP. It may be that it is safe enough for you but I would never advise that DP is good as a backup.

In my case, I run WHS2011 (will move to WSE someday), with DP x2. I also run a daily Server Backup which is scheduled and needs no manual intervention (and I could set it up to backup every hour if I wanted). The Backup HDDs are rotated offsite on a weekly basis. Can I still lose data? Sure. But at least I can restore *most* at all times.

An alternative, of course, mught be the cloud. For example, DropBox retains all versions if you have a professional subscription (including deleted files) and it is offsite at the same time. However, I do not like the cloud personally.

Edit: Ransomware! I forgot to mention ransomware.

Like I already wrote; For me backup policies only exist in order to have data off-site and/or offline. I use my own nextcloud cluster servers, syncthing and rsync for that, in several different sync to backup setups.