File change limit?

[bissquitt]

I know its not a perfect defense, but usually I have all of my pooled drives unmapped with just the pool given a letter. I know when DP loses a disk, it goes into read-only mode until its resolved.

Would it be feasible to do this if the pool hits a threshold of modified files in a short timeframe? Im thinking ransomware mitigation mostly. Not fool proof, but I doubt most ransomware can access a non-mapped disk on a remote server provided the server stays clean.

[Christopher (Drashna)]

It may be possible, but it's not something that I'd recommend.  Windows does this a lot (thumbnails, for instance), so this could cause issues.   Also, if you're writing  files, this could be triggering too, depending on how the software works. 

[bissquitt]

I feel like with some thought it could be implemented well. Possibly some excluded extensions like those thumbnails or excluded folders or ignore new files (only apply to modify). You could probably even piggyback off of the windows ACL. The files I care about are not written very often, and while I might not be the majority, I'm probably at least not the minority in that the vast majority of my data is mostly archival. I would gladly take the inconvenience of having to disable something to make significant changes. Just an idea for an optional feature since lots of people can get scared of ransomware. Not something that I'm actively pushing for.

[Thronic]

On 11/14/2018 at 1:43 AM, bissquitt said:

I know when DP loses a disk, it goes into read-only mode until its resolved.

Would it be feasible to do this if the pool hits a threshold of modified files in a short timeframe? Im thinking ransomware mitigation mostly. 

This is a horrible idea. First of all there would always be SOME damage done and there's lots of situations where you would write to a large amount of files at once. Second, ultimately backups are the only real data protection. DrivePool offers redundancy, which is not a backup. Look into 3-2-1.

That said, you can ransomware protect your most important files by simply scheduling a separate copy of them locally on the server to a non-shared folder. I'd look into Veeam free agent for both GNU/Linux and Windows. It works great to local and network destinations, supports incremental backups and individual file browsing and recovery. I personally also use rclone to cloud for offsite copies.

[bissquitt]

 

28 minutes ago, Thronic said:

This is a horrible idea. First of all there would always be SOME damage done and there's lots of situations where you would write to a large amount of files at once. Second, ultimately backups are the only real data protection. DrivePool offers redundancy, which is not a backup. Look into 3-2-1.

That said, you can ransomware protect your most important files by simply scheduling a separate copy of them locally on the server to a non-shared folder. I'd look into Veeam free agent for both GNU/Linux and Windows. It works great to local and network destinations, supports incremental backups and individual file browsing and recovery. I personally also use rclone to cloud for offsite copies.

I am well aware of 3-2-1 and everything you said. I never intended for this to replace backups. I really don't understand why people get so bent out of shape in regards to something that is helpful even if it is not foolproof. I don't here people arguing against fire doors in buildings because it only slows down and limits the damage of the fire. No, we need to completely remove all oxygen from the building to prevent the fire in the first place, and have a complete second copy of building in another country.

Sometimes you have a backup solution, but would rather only be forced to download or sneakernet 10tb of data rather than the full 100tb server. Mitigation is important. 

Also, I pretty clearly stated it should be optional and able to be turned off when you need to write a lot of files. Ironic that thats pretty much the exact opposite argument that you made above. "I'll put my valuables in this safe, but I refuse to lock it in case I want to put something else in the safe sometime later"

So in some cases you want 0% protection, and in others you want 100% protection, but anywhere in between is unacceptable? Also, is it really that hard to just not enable it?

[Thronic]

1 hour ago, bissquitt said:

we need to completely remove all oxygen from the building to prevent the fire in the first place

For rooms without people this is actually a very good concept.

 

1 hour ago, bissquitt said:

... and have a complete second copy of building in another country.

Wouldn't be bad at all, in reality all the valuable data in one building to another is already being done in major enterprise computing by time-based geo replication e.g. every 5 min.

 

1 hour ago, bissquitt said:

I don't here people arguing against fire doors in buildings because it only slows down and limits the damage of the fire.

I compare that analogy more towards SMART and internal drive ECC. Like you say, mitigation is good even if it's not the entire solution.

 

1 hour ago, bissquitt said:

Sometimes you have a backup solution, but would rather only be forced to download or sneakernet 10tb of data rather than the full 100tb server.

A pure system backup is perhaps 20-60GB max. The rest can be backup up in single files or through an atomic/browsable solution. I think this specific situation would be better served by you researching backup options that would make you happy rather than having Stablebit create something half-assed. I can't believe Alex would want to implementment only partial protection if ever moving in that direction, and behavioral/heuristic algorithms can quicly become a fickle beast to deal with.

 

1 hour ago, bissquitt said:

I pretty clearly stated it should be optional and able to be turned off when you need to write a lot of files. Ironic that thats pretty much the exact opposite argument that you made above. "I'll put my valuables in this safe, but I refuse to lock it in case I want to put something else in the safe sometime later"

Think about it. If you only have a few files you want to protect, than your own solution won't even work. If you have many, your solution still includes accepting SOME damage but not be overrun by it, which is a horrible way to protect anything and a lot of users would rage against loosing anything if it was even an option that had the word "protection" in it. Why have that fire-door when you can easily protect it entirely with a scheduled separate copy and have no performance or usability penalties. Having to manually consider how much an application will modify your files (turning it on/off) is advanced/expert level computing. And at that level there are much better options, simple, obvious ones, to protect your data. 

 

2 hours ago, bissquitt said:

So in some cases you want 0% protection, and in others you want 100% protection, but anywhere in between is unacceptable? Also, is it really that hard to just not enable it?

Yeah I personally don't see why anyone would want something like this when 100% is so easy to achieve, and more often than not the stance is that you either accept loss or you don't. DP is loved over RAID because you'll rarely loose everything due to individual drives and their own intact FS, but this is a forward development from typical RAID where you would loose everything. What you're proposing is a backwards approach to rarely loose everything, from loosing nothing if you just bother to do basic separate copies in any of the multiple ways available even with native tools.

 

If anything, I would rather prefer an option to have some files being copied to a read-only location for a retention of X days or something that would be a 100% protection with easier implementation. The very LAST thing I would like is to have DP try being intelligent and decide what's malware and not during normal operations. I don't want it to be its job at all, I just want GOOD pooling with GOOD integrity. I don't want anything non-related to even remotely possibly affect that. I'm a fan of tools that do mainly one job, and do it well instead of trying to adhoc anything and everything that comes to mind, especially partial solutions to data protection.