Duplication Warnings and IO Error

[Kelby]

I am using DrivePool on a WHS 2011 server.  Today I had a drive that went missing, and I put it back in and then it worked fine.  However, since then, I'm getting duplication warnings and Drive Pool reports duplication at 50%.  When I click on the details, numerous files could not be duplicated, and each says "The request could not be performed because of an I/O device error."  If I click "duplicate now," it tries to duplicate, then gives me the same warnings. 

 

I am also using Drive Scanner on the server, and SMART shows healthy for each drive.  I am running a scan on the drives now, but so far the scan doesn't show any problems.

 

How do correct the duplication warnings and the I/O problems that my pool seems to be having?

[Christopher (Drashna)]

I'd recommend running a "chkdsk /r" scan on the drive. This is a deep scan, and may find file system errors that StableBit Scanner may not.  However, this can take hours, and the disk will be marked as "missing" during the duration of the scan.

[Kelby]

Thanks.  It looks like I may have a bad cable or SATA port or something to the hard drive that went missing.  The scanner showed it as a very bad drive, but when I removed the drive and tested on another computer, the drive works fine.  For now, I had to forcibly remove it from the pool to try to get these problems resolved.

 

So, now, the pool is in heavy duplication mode based on the removal of the one hard drive.  That's fine, but it seems to be running into a problem.  This afternoon, it showed that it was 80.2% rebuilt.  Several hours later, it still showed 80.2%.  I rebooted the server and the duplication worked its way up to 75.6%, and now it's stuck at 75.6% percent.  Any idea why duplication would get stuck like that, and how do I resolve it?

[Kelby]

Here's the update.  Shortly before I went to bed last night, it got back up to 80.2%.  This morning, it's at 83.4%.  The drive I removed was a 4TB drive.  Is it just moving slowly because of the size of the removed drive, or does it sound like something else is a problem?

[Christopher (Drashna)]

Thanks.  It looks like I may have a bad cable or SATA port or something to the hard drive that went missing.  The scanner showed it as a very bad drive, but when I removed the drive and tested on another computer, the drive works fine.  For now, I had to forcibly remove it from the pool to try to get these problems resolved.

 

So, now, the pool is in heavy duplication mode based on the removal of the one hard drive.  That's fine, but it seems to be running into a problem.  This afternoon, it showed that it was 80.2% rebuilt.  Several hours later, it still showed 80.2%.  I rebooted the server and the duplication worked its way up to 75.6%, and now it's stuck at 75.6% percent.  Any idea why duplication would get stuck like that, and how do I resolve it?

A bad data cable could definitely cause a lot of issues. Including what you've described. 

 

The Burst test in StableBit Scanner is actually a good way to test this. Since the burst test reads the same data over and over, the drive keeps it in its cache, and is able to read it VERY fast.  And if there are issues reading it, it's not because of the disk, but because of something between teh disk and the system. (controller card on the drive, data cable, loose connection, bad controller card, bad PCI/PCIe/PCI-X port, etc).

 

Here's the update.  Shortly before I went to bed last night, it got back up to 80.2%.  This morning, it's at 83.4%.  The drive I removed was a 4TB drive.  Is it just moving slowly because of the size of the removed drive, or does it sound like something else is a problem?

 

Depending on the files, yes, it can take a long while,  The general average I've  found is about 4 hours per TB for a normal copy. It may take longer if the system is having issues reading from the disk, though (as the case appears to be here), and may take longer if there are a lot of small files, or if the pool being used while removing the disk. 

[Kelby]

Thanks.  Tonight, the duplication is now at 86.7%.  So it has progressed a grand total of 6.7% in the last 24 hours. 

 

The only drive that had any problems is the drive I removed.  The rest of the drives are functioning fine.  So I think that means the reason for the slow duplication is not a function of the bad drive, which was removed before the current (slow) duplication efforts.

 

The total pool size is good sized -- about 21TB, including duplication.  But the duplication should have already been fine other than having to re-duplicate the 4TB drive I removed, right?  So it should only have to duplicate a maximum of 4 TB?  Or could it be that because I removed a drive, it is re-doing all of the prior duplication? 

 

Either way, does 6.7% progress during a 24 hour period indicate some kind of problem?  If so, what should I do?  Scanner shows no health problems on the remaining drives.

[Christopher (Drashna)]

If it's making progress, then it should be okay.

 

And I wasn't sure if you'd removed the problem drive or not. I just wanted to make sure.

 

As for the slow progress, all of our tasks run at a Background IO priority. This means that they may take longer than if you copied the files yourself, but it should never adversely affect the performance of the pool.  This priority can be boosted. Version 2.1.1.561 and up support this from within the UI. Just click on the ">>" icon at the bottom of the screen.  This will increase the priority.

 

As for the duplication, it has to recheck the entire pool. Once it does that, it needs to copy the files over to the appropriate disks. 

 

But yes, that does seem to be on the slow side, Are the disks rather full already? If so, that may impact the performance.