Jump to content
Covecube Inc.
  • Announcements

    • Christopher (Drashna)

      Login issues   11/07/17

      If you have issues with logging in, make sure you use your display name and not the "username" or email.  Or head here for more info.   http://community.covecube.com/index.php?/topic/3252-login-issues/  
    • Christopher (Drashna)

      Getting Help   11/07/17

      If you're experiencing problems with the software, the best way to get ahold of us is to head to https://stablebit.com/Contact, especially if this is a licensing issue.    Issues submitted there are checked first, and handled more aggressively. So, especially if the problem is urgent, please head over there first. 
  • 0
santacruzskim

To What Degree do DrivePool and Scanner Work Together?

Question

I recently had Scanner flag a disk as containing "unreadable" sectors.  I went into the UI and ran the file scan utility to identify which files, if any, had been damaged by the 48 bad sectors Scanner had identified.  Turns out all 48 sectors were part of the same (1) ~1.5GB video file, which had become corrupted.

 

As Scanner spent the following hours scrubbing all over the platters of this fairly new WD RED spinner in an attempt to recover the data, it dawned on me that my injured file was part of a redundant pool, courtesy of DrivePool.  Meaning, a perfectly good copy of the file was sitting 1 disk over.

 

SO...

  1. Is Scanner not aware of this file?
  2. What is the best way to handle this manually if the file cannot be recovered?  Should I manually delete the file and let DrivePool figure out the discrepancy and re-duplicate the file onto a healthy set of sectors on another drive in the pool?  Should I overwrite the bad file with the good one???

IN A PERFECT WORLD, I WOULD LOVE TO SEE...

  1. Scanner identifies the bad sectors, checks to see if any files were damaged, and presents that information to the user. (currently i was alerted to possible issues, manually started a scan, was told there may be damaged files, manually started a file scan, then I was presented with the list of damaged files).
  2. At this point, the user can take action with a list of options which, in one way or another, allow the user to:
    1. Flag the sectors-in-question as bad so no future data is written to them (remapped).
    2. Automatically (with user authority) create a new copy of the damaged file(s) using a healthy copy found in the same pool.
    3. Attempt to recover the damaged file (with a warning that this could be a very lengthy operation)

Thanks for your ears and some really great software.  Would love to see what the developers and community think about this as I'm sure its been discussed before, but couldn't find anything relevant in the forums.

Share this post


Link to post
Share on other sites

7 answers to this question

Recommended Posts

  • 0

StableBit Scanner isn't aware of the DrivePool pool itself, nor the contents of the other disks.

 

This is because it was developed first, and Drivepool later.

 

However, this is something that has come up, and is something that we would like to implement.  There is a pending feature request for this already.   (eg, check to see if this is a pooled file, and then check the pool to see if it's duplicated, and then handle based on that, eg delete and reduplicate)

 

As for flagging sectors as bad ... that's the trick.  The only way to force this is to write the sector. That is an inherently destructive act, preventing data recovery from properly working. 

 

 

That said, what you've described above, that's pretty much what we'd love to do.  however, that's a significant change to StableBit Scanner, which is why it hasn't happened yet

Share this post


Link to post
Share on other sites
  • 0

Got it. thanks for the follow-up Drashna.  I think this wraps back around to, given these limitations, what do i do now?

 

In a backup scenario, i'd just pull the good file from the backup, confirm its good, delete the corrupted file, and copy over the healthy one from the backup.  In this case we're dealing with live, or scheduled (though my pools duplicate live), file duplication and I don't want to irritate or confuse that system.  Should I...

  1. find out which drive in the pool has the healthy copy and hunt that file down. (I'm guessing this is a manual task of looking in the folder hierarchy where the file should be located on each drive in the pool until you find it).
  2. then, copy the healthy file to a location outside of the pool
  3. then, access the file in the pool through the volume created by DP representing the pool and delete the file
  4. then, copy the good copy of the file back into the pool and let drivepool re-duplicate it

Thanks Drashna.  I sing the praises of both products regularly (and will continue to do so), but I never thought about this part of the puzzle until I was forced to and I'm finding it less than intuitive.

Share this post


Link to post
Share on other sites
  • 0

If you're using 2.2.0.693 or higher, there is a "dpcmd" to list this information:
http://community.covecube.com/index.php?/topic/1587-check-pool-fileparts/
This can be used to identify which drives a specific file is located on. 

 

As to replace the file, you can replace the file in the "poolPart" folder, without issue.  However, you will need to remeasure the pool, if you do this.  

 

(if you're using the latest beta, you can run "dpcmd remeasure-pool x:" to remeasure the pool. 

Share this post


Link to post
Share on other sites
  • 0

Hey there! Long time fan based on the CloudDrive software and I've just recently decided to build a massive file server that replicates using Windows Server DFS Replication (so that the file shares will stay online even if one of the servers crashes or has to go down for maintenance).  I was struggling finding something that could allow me to safely "pool" a bunch of commodity SATA HDDs on each server and after nearly 2 days of research, it finally clicked: DrivePool!

 

But there was only one issue remaining. Due to the sheer number of HDDs that I'll have in the servers, and since I'll be using cheap (non-enterprise) disks, bad blocks will be expected. I had hoped StableBit Scanner would be the answer, but based on what you've told us above, this is not a feature; yet.  Any ETA?  Having a feature that would automatically mark bad blocks and rebuild data from the duplication that DrivePool provides would truly put StableBit into a class of its own (trust me, I searched the web high and low for something like this and it doesn't exist; Storage Space Direct comes close, but you cant use that for general file shares and there's a lot of complexity involved; I really like that DrivePool is simply storing the files direct to the disk, without any special formatting or placeholder image files / virtual disk files).

 

Anyways, just wanted to +1 this and give you my support. Based on the skill and experience you obviously have, I'm sure it won't be too difficult to implement. Thanks!

Share this post


Link to post
Share on other sites
  • 0

Well, just a heads up, DFS Replication may not work when targeting the pool.  Specifically, it relies on Quota Support and/or USN Journalling, neither which is supported on the pool properly, ATM. 

 

 

That said, StableBit DrivePool will clear out a disk that StableBit Scanner does detect as "problematic" (eg, unreadable sectors). 

 

And from the sounds of it, you want a step beyond that, where it actively avoids reading from a disk that has problems, and/or repair the problems. 

This is something that has been discussed at length internally.   And it is something that we'd love to do. It's not not easy to implement.   But it *is* something we'd like to, and I have and will be pushing it. 

Share this post


Link to post
Share on other sites
  • 0

No.

 

The StableBit Scanner balancer in StableBit DrivePool is responsible for anything that the Pool does in this case. 

The default behavior is to evacuate the contents of the drive in question.  Meaning that the data is balanced off of the pooled disk. 

 

However, the drive is NOT removed from the pool automatically.  The balancer isn't set up to do this, especially as some data may not be movable.

 

That said, the balancer does set up a "real time placement limiter" on the drive so that no new data can be added to this disk.  That way, new data is not at risk. 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


×