How to remove empty folders from drives that doesn't keep any data?

[Ac3M]

I'm curious how to do this as my data is constantly being shifted between drives thanks to certain apps that update stuff, and I'm left with a crap-ton of empty folders that sometimes takes forever to move (Among these battalions of empty folders are some that still contains data) in some cases when there's something wrong with a drive whether it's glitching, bad connection or some file system error and it's at 100 % usage and to then have it copying/moving some 20k empty folders isn't smooth, especially if you're having issues with BSOD and such. Sometimes it freezes and doesn't move for minutes, sometimes even hours.

 

I'd like DrivePool to remove these if they're no longer in use - How do I make this happen?

Didn't find any topic (I looked) that matches this, so sorry if I've created a duplicate question.

[Shane]

It also occurs to me that some users may have the opposite issue: where they use apps that rely on the presence of folders that may be empty at times for whatever reason, so they wouldn't want those folders being automatically removed by DrivePool.

So any "cleanup empty folders" function implemented by DrivePool would have to ensure that it only removes "excess" instances of such folders within the poolparts, the same as it does with files. How tricky would it be to extend DrivePool's duplication-checking to check for excess instances of empty folders in addition to excess instances of files (e.g. currently, if duplication is set to x3 and there are x4 instances of a file, DrivePool will remove the 4th instance of that file - can DrivePool be made to apply that to empty folders too)?

[Shane]

It's come up a few times over the years. This post mentions reasons why DP might leave empty (and "empty") folders, while this post mentions a really neat trick with robocopy that can be used to clean them up if you decide you want them gone anyway - note that if you use it but you've got any empty folders you actually want to keep you'll have to exclude them somehow (e.g. via the /XD switch). Hope that helps.

[zzwindsor]

I've been using Czkawka. Select individual disks (not the pool) in "Folder to Search" and then run "Empty Directories" and delete them all.

My workflow has been:

- Run Drivepool balancing by space used

- Stop Drivepool service

- Run Czkawka Empty Directories

- Run SnapRaid Sync/Touch/Scrub

- Start Drivepool service

It's been working pretty well. You can also create a .bat to make it even easier.

[marquis6461]

I use Remove Empty Directories from this website https://www.jonasjohn.de/red.htm

You can use the simple method with the GUI.  I just select the PoolPart for each of the drives.  

Scan it and delete.  Since I only use the Main Drive letter I must assign a temporary Drive letter to each partition.

I use it because sometimes the directory has sensitive information in the label.  Once a month is good enough.

[Ac3M]

I've been busy with other things so sorry for the late reply.

I appreciate the replies with suggestions, but I think that perhaps the developers should, I dunno, do something about it?

It can literary take hours to remove an "empty" drive just because of this, and failing drives can be very time sensitive. I don't know if these empty folders would impede moving files, but if it does, it's horrible.

Seeing as how there are several options for various functions for the app itself, not to mention the plugins, I fail to see why there can't be something that cleans up empty folders just as well, setting or plugin.

I see this as a sort of cancer. If I plug in a set of new drives, DP goes to town to re-balance the pool, meaning it'll move stuff between old and new, and once again, as evident, leave empty folders behind...

[Shane]

You could request it as a feature via the contact form? @Christopher (Drashna) perhaps there could be an option to schedule it alongside the regular pool consistency checks, and/or as a dpcmd command which could be run manually or scheduled (e.g. via Task Scheduler) by those for whom it's a significant issue?

[Christopher (Drashna)]

Well, the biggest issue is that the folders are likely not actually empty.  There are some hidden files that are in use by StableBit DrivePool.  For instance, we use Alternate Data Streams to set the duplication settings.  These don't show up normally, even with enabling the "show protected system files" option.

 

[Ac3M]

Sorry, been otherwise occupied lately, and past week I've been battling an infection with fever and stuff.

I concur with @Shane here, that there should be a clean-up-function available.

But, even with the disks removed from the pool with the folder structure intact, when connected to another non-pooled computer still shows empty folders.

Sure, I don't, to my knowledge, have any app that reads hidden files so I can't confirm if they're still existing.

But from what I've seen, when I have removed a such disk from the pool, as far as I could see by looking at the folders and file structure (To check if there's anything missing on the pool), it's all there.

I realize that when it's running in the pool, and it does it thing (sometimes I see long filenames.copytemp in apps like DC and in the control panel for DP, which normally aren't seen, so this makes sense, but for small folders like I describe below - Not so sure. I think there are kinks to work out there)

I sort of feel that the app should be able to sense, and make a note of if it, if it needs to create a folder, so that it can then remove it after it's done with whatever operation it's running.

It took me over a week and a half to remove 5 x 2TB disks from the pool (At about 80-85% full). Most extraction went as expected, but it seemed to halt when it comes to deep folder hierarchies (Possibly too long names) without erroring out.
I replaced the HDDs with SSDs, so at least it should go a lot quicker in the future.

I ended up having to run RED (Thanks @marquis6461 for the tip) o delete these (and also the ones that RED was unable to delete I had to remove manually), and some of the disks I had to move data from the pooled disk, first up one level to a new folder, then from there back to the pool.

 

I also think that DrivePool should keep track of disks selected for removal. Some of the times during this removal process, some failed, and started filling back up again during the night which was very annoying.

Another thing that it forgot, was that I set the SSD optimizer for a smaller SSD that I have in the pool, as a cache-drive (selected SSD and not Archive).

It defaulted back to setting it to Archived and filled up that disk. I had this problem a while back with another Windows-installation that was screwy (aren't they all..?) and I thought it was Window's fault at the time, but I'm not so sure. It still has regular data on it, and I have some stuff running at the moment so I can't reboot (It's a server), but something is preventing it from cleaning up.

Some file locks, permission denied (Some folders do have NTFS set to nothing, so that they won't be removed - And it seems it has copied those).

Hopefully, the next reboot will take care of that.