Jump to content
  • 0

DrivePool - Recycle Bin - Undelete Functionality (Grayhole Trash Equivalent)?


vfsrecycle_kid

Question

Hi folks,

 

I'm in the process of setting up a DIY NAS. I plan to use SnapRAID. Due to the nature of the RAID, deletes that are left unsynced can be problematic.

 

See http://wiki.flexraid.com/2011/10/18/understanding-the-limitations-of-snapshot-raid/ for more information.

 

Originally I had the following idea:

 

1. Pool SnapRAID content drives together (excluding parity drive)

2. On delete, move to pools recycle bin

3. Weekly schedule, run a TRUE delete on the recycle bin of the pool

4. SnapRAID sync changes.

 

This would give minimal time between being out of sync and being in sync.

 

Someone once brought up my exact issue on your old discussion board.

 

http://forum.covecube.com/discussion/1076/undelete-files-deleted-via-smb/p1

 

However, for "next time", a number of people have recommend running other Undelete utils that create their own "recycle bin" of sorts to help prevent this sort of issue.

 

I am not clear on what that comment means with the issue.

 

However i'm left with the following issues.

 

1. SnapRAID + DrivePool might mean deletes are immediate (rather than moved to a pooled recycle bin, moves do not compromise parity)

2. FlexRAID offers essentially what I'm trying to do with drive pool (a pooled drive)....however I lose features I want out of DrivePool (SSD Landing Drive, Folder Duplication).

 

Hopefully some input from you guys could tell me if DrivePool offers this feature, or a potential alternative solution in mitigating the "delete problem" mentioned above.

 

I will be accessing the drive via SMB.

 

Greyhole (a linux "equivalent" of DrivePool does this by a "Trash" feature. Instead of permanent delete, it moves there. Then eventually you run a permanent delete command or delete it from the Trash folder).

https://github.com/gboudreau/Greyhole/wiki/AboutTrash

 

Additionally in linux, there's vfs_recycle which allows you to move instead of delete on a delete request. Unfortunately that doesn't exist in Windows Samba sharing.

 

Thanks folks for your help! 

 

edit: One annoying but potential compromise is to create a "DELETE" folder on the pool. Whenever I want to delete files, move it to that folder instead. However that requires educating users accessing the share that when they want to delete something they need to move it to a special directory. But you can't disable DELETE in NTFS without disabling MOVE (Move is just a COPY+DELETE)...so it'd really be a case where I have to trust that my users will delete files in the "move" way

Link to comment
Share on other sites

16 answers to this question

Recommended Posts

  • 0

Well, if you're using the StableBit DrivePool pool locally, when you delete files, it will use the "normal" delete methods. This means that it will send it to the Windows Recycle Bit as normal.  You can force delete to bypass this method, as per normal.

 

However, the issue is when using network shares.The default action is to permanently delete. This is how Windows works, and not a behavior specific to StableBit DrivePool.  Under very specific circumstances, you can "recycle" files instead of deleting them on network shares. I believe (I'm not sure, and would have to verify) that this should occur when you make files available offline (but that's not feasible for a pooled shares).

But this will definitely work to that end:

  1. Map a network drive to the network share you want to use. Make sure that the drive is re-connected on logon. If you don't know how to do this, search Google.
  2. Browse to C:\users\<user name>.
  3. Right-click on one of the folders in this location (I chose saved games) and click properties.
  4. Select the Location tab.
  5. Click Move, browse to to root of the drive you mapped in step 1, and click Select Folder.
  6. Click Ok and click yes in the dialogue box that appears.
  7. Repeat these same steps for all users on the computer.

 

 

We could add an intermediary step here, so that files deleted over the network would go to a temp storage/recycler. WHich I think is what you're asking for. 

However, implementing this would be tricky at best. This would require intercepting the network deletes (and differentiating them), and then storing them. And then we'd need a way to display these deleted files. 
 

 

 

As for utilities to do this, there used to be a few, but I'm no longer able to find them.

THey would monitor the network shares, and try to move files that were to be deleted. Given that they don't exist anymore, I suspect that they didn't work too well. :(

Link to comment
Share on other sites

  • 0

We could add an intermediary step here, so that files deleted over the network would go to a temp storage/recycler. WHich I think is what you're asking for. 

However, implementing this would be tricky at best. This would require intercepting the network deletes (and differentiating them), and then storing them. And then we'd need a way to display these deleted files. 

 

First of all thank you for your quick reply. That's awesome!

 

I'm intrigued by your solution. I'm curious what that actually accomplishes. Before I ask about that I'll re-explain what I want.

 

When deleting (which as you said is a permanent-delete) via SMB. I'd like files moved instead to a pooled "Recycle Bin". Then technically all that's needed to be done is when a permanent delete is desired, delete from the recycle bin. This is exactly how Greyhole does it.

I think Greyhole does differentiate between a normal delete vs a SMB delete, which is what makes this work for them. However that is linux and does not apply to us unfortunately.

 

--

 

So having said that, doesn't seem what I want is in DrivePool unless there was an option that added something along the lines of "Intercept delete request with a Move to Recycle Bin, unless you're deleting from Recycle Bin, in which case permanently delete".

 

--

 

Back to your "workaround" steps. What exactly does this accomplish. So we set a random "User" folder via the Move Dialog in Properties to the networked pool. What does your set of instructions allow me to do now?

 

I haven't setup anything on my end hence I cannot confirm what that does.

 

Thanks again!

 

Apologies for the double post. I'm connected to the forum via mobile device and I cannot find the edit button.

 

When I say the files are moved to a recycle bin I mean on the machine doing the pool. Files shouldn't be copied to the local machines recycle bin when I delete from SMB. I doubt that can be interpreted from what I said but just want to make sure I mentioned this case.

 

Its really just a case of

When I delete from the pool, move it to a recycle-bin-like directory.

Link to comment
Share on other sites

  • 0

When deleting (which as you said is a permanent-delete) via SMB. I'd like files moved instead to a pooled "Recycle Bin". Then technically all that's needed to be done is when a permanent delete is desired, delete from the recycle bin. This is exactly how Greyhole does it.

I think Greyhole does differentiate between a normal delete vs a SMB delete, which is what makes this work for them. However that is linux and does not apply to us unfortunately.

Greyhole is taking advantage of the build in linux stuff for this. At least, that is what it appears to be doing, from a quick read.

 

We have a way to differentiate local access from network access, in the "Network IO Boost" feature. But the issue is that eats up additional CPU cycles and memory, so it may adversely affect performance, ESPECIALLY on lower end hardware (and is why it's disabled by default). 

Maybe we can hook into that to provide the functionality, (i've submitted the request to Alex (the developer), for review). 

Feature request here: https://stablebit.com/Admin/IssueAnalysis/14560

 

 

Back to your "workaround" steps. What exactly does this accomplish. So we set a random "User" folder via the Move Dialog in Properties to the networked pool. What does your set of instructions allow me to do now?

 

I haven't setup anything on my end hence I cannot confirm what that does.

 

Thanks again!

When you map the user folders to a network share, Windows is very nice here.... It creates a recycle bin ON THE SHARES for these folders. This means that if you delete files from the user folders (and only the user folders), then it will be moved to the recycle bin folder on the shares.  

 

I can absolutely confirm this behavior. And this is also set up by the "Folder Redirection" group policy on Server Essentials. And definitely works on a standard user account, if done manually.

 

 

Again, this would only work for the User folders, but ... you could map your folders to main shares, such as \\SERVER\Pictures, \\SERVER\Videos, \\SERVER\Documents, etc. 

 

This would get the behavior you're looking for, actually. But this would be for the 'local machine'. It's still stored on the network, but it would only show up in the recycle bin for any system that has the folders mapped this way.

Link to comment
Share on other sites

  • 0

Once again thank you for the prompt reply. Really liking the support you guys are giving. I'm a dev myself so the technical insight is appreciated.

 

Having said that. I now understand what your workaround is suggesting. However this doesn't actually apply to my usecase. 

 

To further expand here's the current (proposed) network structure:

 

* NAS - Windows Server, SnapRAID + DrivePool

* PC1 - Windows

* PC2 - Windows

* PCn - Windows/Android (Various via WiFi)

 

I'm not concerned with a PC->NAS recycle bin (though I can appreciate that I may have been vague in my description, I'm still trying to figure out how to appropriately describe my setup).

 

The only concern with this is PC1 and PC2 will be connecting to the NAS via SMB (the DrivePool will be set to Shared on the network) as a mapped network drive. They will be interacting with the device as if it was local. With the sole exception that only permanent deletes exist in SMB world, wherein lies the issue.

 

--

 

The workaround I mentioned before is simply instead of deleting via SMB, create a folder in the DrivePool (lets call it "Trash" to mimic Greyhole's name). We move it to the Trash directory (which is just a normal folder, no special designation from the OS).

 

Then, when I run my weekly SnapRAID scripts (which is run locally on the NAS): I can run something like

del X:\Trash\*.* /s
snapraid.exe sync
pause

# Simplified the snapraid stuff just for simplicity of the post
# Where X is the DrivePool letter.

--

 

Having said all of that. It does seem the end goal is something that is not currently offered in DrivePool. But it does offer 9/10ths of what I would like, and I cannot find another solution to this problem.

 

In which case if the feature was to come into affect as a selectable option in the future for DrivePool, I think it would be fantastic. Snapshot based "RAID" adds a lot of value to a home redundancy solution, but it has some drawbacks.

 

I do appreciate the that the feature is not widely request and do not hold any expectation for it to be implemented.

 

If I've misinterpreted anything you've said @Drashna, do let me know :).

Thanks!

Link to comment
Share on other sites

  • 0

The "user folders" thing is a method that works "now", though, it's certainly not ideal.

 

 

And what you've mentioned, I've specifically created a feature request for:

https://stablebit.com/Admin/IssueAnalysis/14560

 

 

when files are deleted from the pool, on the network, it would be sent to a recycle bin type temp folder first. And then could be deleted later

That's not the entire bit of what I've posted, but the important bits. So we will definitely look into it.

Link to comment
Share on other sites

  • 0

Gotcha. Thank you for all your info on the matter.

The link/issue you referenced seems to display no real information for outside users (I see is it as a bunch of empty form fields).

 

J4uHYB1.png

 

But good to know it is officially tracked. I'll be keeping a close eye on this issue :).

 

Thanks for all the quick responses. I see DrivePool in my future (parts for the NAS will arrive in the coming weeks!)

Link to comment
Share on other sites

  • 0

Yeah, I don't usually post publically on these. Sometimes there is personal information in them, and sometimes, there is a lot of "extra" information so I can properly convey the idea to Alex (the developer). 

 

Once he's reviewed it, he'll add a public comment and change it to "feature request". 

You can see these and more here: 

http://wiki.covecube.com/Development_Status

 

 

And I'm glad to hear it. :)

 

Though don't forget to check out StableBit Scanner as well (as there is some communication between the two).

Link to comment
Share on other sites

  • 0

Gotcha. Thank you for all your info on the matter.

The link/issue you referenced seems to display no real information for outside users (I see is it as a bunch of empty form fields).

 

 

But good to know it is officially tracked. I'll be keeping a close eye on this issue :).

 

Thanks for all the quick responses. I see DrivePool in my future (parts for the NAS will arrive in the coming weeks!)

Alex has replied to the request.

And no, it definitely won't be simple (as I hoped it may be).

 

Implementing this feature would be a lot of work. I won't say that we won't do it, but it's not a priority at this time. 

Link to comment
Share on other sites

  • 0

Just noticed. Thanks for the update.

 

Alex is not wrong in that it requires intercepting a delete with a move (unless we're deleting from the designated "recycle bin").

 

I appreciate the technical POV on the issue and glad at least it didn't take a while to find out what I need to do.

 

I might just have to compile http://liquesce.codeplex.com/ from source to accomplish what I'd like. THey use CBFS and I assume I'd just intercept the Delete on specific conditions and handle accordingly.

 

However I've used Liquece in the past and I don't think it's as stable as DrivePool. I had a few minor issues working with it in the past.

 

As well as I'd lose a couple nice-to-have features that DrivePool offers.

 

Either way I'm glad I have all this new knowledge! If it ever does get implemented I'm sure I'll latch onto it :).

 

Thanks

Link to comment
Share on other sites

  • 0

Liquesce would probably work. However, the last time I tried using it .... it wasn't very stable. 

 

Also, as for the delete command, .... and for some of the conversation that you didn't see... any program that installs a file system filter could intercept the delete command and move it instead. 

Adding a file system filter is an option that we've discussed internally for one of our "future products". So it's possible that we could add this functionality at a future date. It wouldn't be immediately, because we're gearing up for a big release soon (and we're a very small company). But after that, I'll bug Alex about it.

 

Though... another option is to remove or outright "Deny" the "delete" permissions for the network share, so that you cannot delete files over the network. 

http://helpdeskgeek.com/how-to/prevent-file-deletion-and-file-renaming-in-windows/

Link to comment
Share on other sites

  • 0

Though... another option is to remove or outright "Deny" the "delete" permissions for the network share, so that you cannot delete files over the network. 

http://helpdeskgeek.com/how-to/prevent-file-deletion-and-file-renaming-in-windows/

 

Definitely considered this one already. Originally thought I could disable DELETE but keep MOVE (that way you'd just "Move" to the 'Recycle Bin' and I'd have a script clean up the folder). But obviously I quickly found out a move is simply a NTFS Copy+Delete. So had to nix that.

 

DrivePool + simply telling our users to manually move when they want to delete is probably going to be the solution. Sure we might have some accidental times where they'll be a delete. But hopefully the drives won't die during that period!

 

When you say a file system filter (I've been in talks with CBFS to get a single use license), do you mean a system similar to your balancer plugins, so something exposed for someone like me to sneak in a "delete override" plugin type functionality? Interesting stuff!

 

Thanks

Link to comment
Share on other sites

  • 0

Yup, unfortunately moves are just copy then delete commands. :(

 

 

 

As for file system filters: 

File system filters ... well, sit "on top" of the file system, and ... filter any and all requests made to the files system.

This is actually how antivirus programs create the real time scanners. They're just file system filters.

This is also how the UAC works, some disk encryption programs (diskcryptor, in specific), and even how the Data Deduplication feature in Server 2012R2 works (it stores the duplicate data in the system volume info folder, and "rebuilds" the data when you access it, by intercepting "in transit" and merging the parts back together).

ALso, the UAC virtualization that can occur happens this way (luafv). 

 

Specifically, a command is sent to the disk (look up info, access, copy, delete, etc). The file system filter intercepts this command and determines what to do with it. Then it passes it on to the actual disk. 

 

So, it's conceivable that to have/create a file system filter that will look for file deletes, check the source (network or local) and then actually CHANGE the command (to a move or ... cancel it altogether) at the time of the command. 

 

During the XP era, there used to be half a dozen utilities that actually did this. But I've been looking and can't really find one. I'll keep looking, but I can't promise anything.

 

This is also why antivirus programs can cause performance issues... and even stability issues.... filters are incredibly powerful, and need to work "just right". 

Link to comment
Share on other sites

  • 0

creating the filter? easy. Making sure it works without corrupting the files, damaging the file system, or causing unknown issues.... that's the hard part. :)

I'll have to give it a shot anyways haha. There's some nice abstractions people offer, wonder if they'll offer non-commercial single use license as a gesture of good will, haha!

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...