Jump to content
  • 0

Manualy delete duplicates?


silk

Question

I have a bad drive that i had to force remove from my pool (I was out of town for a couple weeks and came home to this). I did data recovery and now have a mirror of the drive in a folder. I want to delete those duplicated files on the mirror before I move the data back to my pool.

For example, this was easy in Drivebender, I just search for all the $<duplicate folder name>$ and then delete them. How do I do this with Stablebit?

Thanks!

Edit: And also, I saw a post over on Reddit saying Scanner and Drivepool can be configured to work together so that once a disk is marked as damaged by Scanner Drviepool will move the files off the drive. I have 2 questions on this:

1) Is there a guide somewhere on how to set this up, would've saved me a headache here? I couldn't find it in the wiki, but I may not have good search terms.

2) I was hit by the bug where Scanner was marking drives as damaged when they were not

(this bug: https://community.covecube.com/index.php?/topic/4272-file-system-damaged-yet-there-are-no-errors-in-chkdsk-or-windows-chkdsk-i-even-ran-chkdsk-t-f-r-for-a-week/#comment-29307).

This seemingly got fixed a month or so later but was the problem identified with a permanent fix? I would hate to run into this problem again after I setup the auto drive removal.

Thanks!

Link to comment
Share on other sites

6 answers to this question

Recommended Posts

  • 0

StableBit DrivePool doesn't have a master and subordinate/duplicate copy. Both are equally viable, and treated as such.  This is very different from how Drive Bender handles things.5

As for being documented, not really, but sort of. Eg: https://wiki.covecube.com/StableBit_DrivePool_Knowledge_base#How_To.27s

 

That said, if the data is in the same relative path under the PoolPart folders, they're considered duplicates.  Changing the duplication settings, or even remeasuring can kick off a duplication pass that will automatically prune the duplicates, as needed. 

Also, the "dpcmd" utility has an option to disable duplication for the entire pool, recursively. However, that kicks off a duplication pass that actually manages the files. 

19 hours ago, silk said:

1) Is there a guide somewhere on how to set this up, would've saved me a headache here? I couldn't find it in the wiki, but I may not have good search terms.

Just have both products installed. That's it.  You can fine tune settings in StableBit DrivePool, in the balancing settings, as the "StableBit Scanner" balancer is one of the 5 preinstalled balancer plugins. 

19 hours ago, silk said:

2) I was hit by the bug where Scanner was marking drives as damaged when they were not

(this bug: https://community.covecube.com/index.php?/topic/4272-file-system-damaged-yet-there-are-no-errors-in-chkdsk-or-windows-chkdsk-i-even-ran-chkdsk-t-f-r-for-a-week/#comment-29307).

This seemingly got fixed a month or so later but was the problem identified with a permanent fix? I would hate to run into this problem again after I setup the auto drive removal.


That should be fixed now.  Though, the file system scan won't trigger the drive evacuation.  And yeah, that fix was shipping in the 2.5.5 version, and the latest stable release is 2.5.6, so this definitely shouldn't be an issue anymore.  (we haven't seen it in a while). 

 

Link to comment
Share on other sites

  • 0

I've been using duplicate commander for manual de-dupe for a few years, and it's worked great.

If you suspect you suffered data loss / corruption during the failure, however, you might want to use something to actually hash your files.

Comparing NTFS entries like modified date/time and file-size alone won't catch corruption, until you actually try to open the files.

Link to comment
Share on other sites

  • 0

I am not sure if we are speaking of the same thing, probably due to my ignorance of Stablebit. I am not looking to dedupe all the data on the failed disk, which is what it *looks* (and I may be very wrong) like duplicate commander is for?

 

What I am looking for is how does Stablebit drivepool physically store duplicated data on the disk. So say I have a folder ..\XYZfiles and I set it to be duplicated in Drivepool. XYZFiles has ..\XYZfiles\morefiles\hello.txt. Where is Drivepool physcially storing the copy of .\XYZfiles\morefiles\hello.txt? What is the path of the duplicated file?

Link to comment
Share on other sites

  • 0

Inside each physical disk that's part of the pool, exists a hidden folder named with a unique identification ID.

 

Inside these folders is the same folder structure as the pool, itself.

 

Your duplicated files / folders would simply be on the appropriate number of disks. They're not actually denoted as duplicates in any way.

 

If files are now duplicated (that shouldn't) be, it may be enough to simply re-check duplication.

Link to comment
Share on other sites

  • 0
Quote

exists a hidden folder named with a unique identification ID.

Thanks but is this documented anywhere? Can you be specific? Like can you give me an example of what that looks like? Where is this folder placed? Is it in the same directory as the folder it is duplicated? Is it in a parallel structure separated from the main poolpart folder tree?

Here is a thread on how Drivebender does it: https://community.division-m.com/topic/816-drivebender-file-structure-for-duplicated-filesfolders/. I can't find any information on how Stablebit does it. It shouldn't be this hard to find it documented somewhere, yet I simply cannot find this information. Even here you have only described it in very vague terms. Is this secret information nobody knows or so obvious that I am blind for not seeing it?

Link to comment
Share on other sites

  • 0
11 hours ago, Christopher (Drashna) said:

StableBit DrivePool doesn't have a master and subordinate/duplicate copy. Both are equally viable, and treated as such.  This is very different from how Drive Bender handles things.

Thanks Christopher! So it is my ignorance of Stablebit that is the root of my confusion. Just so I am clear, as I understand now Stablebit uses no parallel or alternative folder structure at all to store duplicated files. It just keeps copies of the duplicated files on other drives, in the same original folder structure, and uses software to keep from listing duplicated files and having file collisions in the drivepool?

 

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...