Jump to content
  • 0

To What Degree do DrivePool and Scanner Work Together?


santacruzskim

Question

I recently had Scanner flag a disk as containing "unreadable" sectors.  I went into the UI and ran the file scan utility to identify which files, if any, had been damaged by the 48 bad sectors Scanner had identified.  Turns out all 48 sectors were part of the same (1) ~1.5GB video file, which had become corrupted.

 

As Scanner spent the following hours scrubbing all over the platters of this fairly new WD RED spinner in an attempt to recover the data, it dawned on me that my injured file was part of a redundant pool, courtesy of DrivePool.  Meaning, a perfectly good copy of the file was sitting 1 disk over.

 

SO...

  1. Is Scanner not aware of this file?
  2. What is the best way to handle this manually if the file cannot be recovered?  Should I manually delete the file and let DrivePool figure out the discrepancy and re-duplicate the file onto a healthy set of sectors on another drive in the pool?  Should I overwrite the bad file with the good one???

IN A PERFECT WORLD, I WOULD LOVE TO SEE...

  1. Scanner identifies the bad sectors, checks to see if any files were damaged, and presents that information to the user. (currently i was alerted to possible issues, manually started a scan, was told there may be damaged files, manually started a file scan, then I was presented with the list of damaged files).
  2. At this point, the user can take action with a list of options which, in one way or another, allow the user to:
    1. Flag the sectors-in-question as bad so no future data is written to them (remapped).
    2. Automatically (with user authority) create a new copy of the damaged file(s) using a healthy copy found in the same pool.
    3. Attempt to recover the damaged file (with a warning that this could be a very lengthy operation)

Thanks for your ears and some really great software.  Would love to see what the developers and community think about this as I'm sure its been discussed before, but couldn't find anything relevant in the forums.

Link to comment
Share on other sites

8 answers to this question

Recommended Posts

  • 0

StableBit Scanner isn't aware of the DrivePool pool itself, nor the contents of the other disks.

 

This is because it was developed first, and Drivepool later.

 

However, this is something that has come up, and is something that we would like to implement.  There is a pending feature request for this already.   (eg, check to see if this is a pooled file, and then check the pool to see if it's duplicated, and then handle based on that, eg delete and reduplicate)

 

As for flagging sectors as bad ... that's the trick.  The only way to force this is to write the sector. That is an inherently destructive act, preventing data recovery from properly working. 

 

 

That said, what you've described above, that's pretty much what we'd love to do.  however, that's a significant change to StableBit Scanner, which is why it hasn't happened yet

Link to comment
Share on other sites

  • 0

Got it. thanks for the follow-up Drashna.  I think this wraps back around to, given these limitations, what do i do now?

 

In a backup scenario, i'd just pull the good file from the backup, confirm its good, delete the corrupted file, and copy over the healthy one from the backup.  In this case we're dealing with live, or scheduled (though my pools duplicate live), file duplication and I don't want to irritate or confuse that system.  Should I...

  1. find out which drive in the pool has the healthy copy and hunt that file down. (I'm guessing this is a manual task of looking in the folder hierarchy where the file should be located on each drive in the pool until you find it).
  2. then, copy the healthy file to a location outside of the pool
  3. then, access the file in the pool through the volume created by DP representing the pool and delete the file
  4. then, copy the good copy of the file back into the pool and let drivepool re-duplicate it

Thanks Drashna.  I sing the praises of both products regularly (and will continue to do so), but I never thought about this part of the puzzle until I was forced to and I'm finding it less than intuitive.

Link to comment
Share on other sites

  • 0

If you're using 2.2.0.693 or higher, there is a "dpcmd" to list this information:
http://community.covecube.com/index.php?/topic/1587-check-pool-fileparts/
This can be used to identify which drives a specific file is located on. 

 

As to replace the file, you can replace the file in the "poolPart" folder, without issue.  However, you will need to remeasure the pool, if you do this.  

 

(if you're using the latest beta, you can run "dpcmd remeasure-pool x:" to remeasure the pool. 

Link to comment
Share on other sites

  • 0

Hey there! Long time fan based on the CloudDrive software and I've just recently decided to build a massive file server that replicates using Windows Server DFS Replication (so that the file shares will stay online even if one of the servers crashes or has to go down for maintenance).  I was struggling finding something that could allow me to safely "pool" a bunch of commodity SATA HDDs on each server and after nearly 2 days of research, it finally clicked: DrivePool!

 

But there was only one issue remaining. Due to the sheer number of HDDs that I'll have in the servers, and since I'll be using cheap (non-enterprise) disks, bad blocks will be expected. I had hoped StableBit Scanner would be the answer, but based on what you've told us above, this is not a feature; yet.  Any ETA?  Having a feature that would automatically mark bad blocks and rebuild data from the duplication that DrivePool provides would truly put StableBit into a class of its own (trust me, I searched the web high and low for something like this and it doesn't exist; Storage Space Direct comes close, but you cant use that for general file shares and there's a lot of complexity involved; I really like that DrivePool is simply storing the files direct to the disk, without any special formatting or placeholder image files / virtual disk files).

 

Anyways, just wanted to +1 this and give you my support. Based on the skill and experience you obviously have, I'm sure it won't be too difficult to implement. Thanks!

Link to comment
Share on other sites

  • 0

Well, just a heads up, DFS Replication may not work when targeting the pool.  Specifically, it relies on Quota Support and/or USN Journalling, neither which is supported on the pool properly, ATM. 

 

 

That said, StableBit DrivePool will clear out a disk that StableBit Scanner does detect as "problematic" (eg, unreadable sectors). 

 

And from the sounds of it, you want a step beyond that, where it actively avoids reading from a disk that has problems, and/or repair the problems. 

This is something that has been discussed at length internally.   And it is something that we'd love to do. It's not not easy to implement.   But it *is* something we'd like to, and I have and will be pushing it. 

Link to comment
Share on other sites

  • 0

No.

 

The StableBit Scanner balancer in StableBit DrivePool is responsible for anything that the Pool does in this case. 

The default behavior is to evacuate the contents of the drive in question.  Meaning that the data is balanced off of the pooled disk. 

 

However, the drive is NOT removed from the pool automatically.  The balancer isn't set up to do this, especially as some data may not be movable.

 

That said, the balancer does set up a "real time placement limiter" on the drive so that no new data can be added to this disk.  That way, new data is not at risk. 

Link to comment
Share on other sites

  • 0

Resurrecting an old thread here -- but I think this functionality has been implemented now, right?

I was wondering if the DrivePool + Scanner integration works for SAS drives. I realize that Scanner doesn't support SAS, but it does seem to be able to scan the drives fine...but I'm not sure if it actually works with DrivePool together (haven't gotten to testing it -- thought it would be easier to simply ask.)

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...