failed drive removed from pool but files missing

[Shooter3k]

Hello,

I have drivepool doing "pool file duplication" to make 2x of the files. 

A drive failed so I removed it from the pool. Before having it reduplicate everything, I needed to move files off the drives to make room (so that duplication had enough room to complete) but I realized a lot of the directories contained 0 files after I moved them out of the pool. When I use other software to try and recover the bad hard drive, I'm finding the missing files on the bad drive. 

What happened to the duplicated copies from the pool? Was I supposed to let duplication try to finish first? Something doesn't seem right here...

[Christopher (Drashna)]

Depending on the exact chain of events, this may be normal.   When balancing or changing duplication, empty folders are not pruned.  So if the files were on that disk but moved off, you will likely see empty folders.   As for the recovery tools seeing the files, that may be normal. Files deleted from a disk, even when done by the balancing/duplication engine will still leave the data on the drive until those sections are needed.  This is actually how recovery software is able to get that data.  

So if duplication was enabled, it soulds like you should be fine.  However, you can recover the data, and remeasure the pool.  It will check the duplication level against the number of copies, and adjust things accordingly (eg reduplicate data, remove extra copies, etc). 

[Shooter3k]

1 hour ago, Christopher (Drashna) said:

Depending on the exact chain of events, this may be normal.   When balancing or changing duplication, empty folders are not pruned.  So if the files were on that disk but moved off, you will likely see empty folders.   As for the recovery tools seeing the files, that may be normal. Files deleted from a disk, even when done by the balancing/duplication engine will still leave the data on the drive until those sections are needed.  This is actually how recovery software is able to get that data.  

So if duplication was enabled, it soulds like you should be fine.  However, you can recover the data, and remeasure the pool.  It will check the duplication level against the number of copies, and adjust things accordingly (eg reduplicate data, remove extra copies, etc). 

it appears that the duplicated data just vanished for some strange reason...

This is roughly what happened: 

1. pool was in good status as far as I know (but I guess maybe it wasn't?)

2. drive failed

3. drive reported missing

4. I removed the drive from the pool

5. I think it tried to check or duplicate the contents but I cancelled it

6. I moved a folder off the pool to make room 

7. the folder I moved off the pool was only 1/2 of the data 

8. I found the missing data on the bad drive I replaced 

----------

So, it's almost as if because the pool got in a wacky state, it didn't give me all the data it should've when I moved the data out of the pool?  Very strange. 

[Shane]

"1. pool was in good status as far as I know (but I guess maybe it wasn't?)"

If it wasn't, this is something that should've been picked up the consistency checks that DrivePool does in the background. Agreed, very strange.

You can have DrivePool show you the expected duplication levels of all folders in the pool and the actual space used via the GUI -> Manage Pool -> File Protection -> Folder Duplication... it will show an expandable color-coded folder tree of your pool, and beneath that it will begin calculating the actual size used before duplication (pale blue) and size used by duplication (dark blue); for a consistent pool with entirely x2 duplication those two sizes should finish roughly equal (I say roughly because system folders have their own special duplication levels which may throw the figure off a little).

Note that if you see a plus sign after a folder's duplication level it means one or more sub-folders have a different duplication level (including possibly no duplication; "plus" here doesn't mean "or more", it means "or otherwise").

You can force DrivePool to recheck/repair duplication via the GUI -> Cog icon (top right) -> Troubleshooting -> Recheck duplication...