Jump to content
  • 0

Error - Files not constistant across pooled drives


Desani

Question

I have three cloud drives that are all pooled together with DrivePool. I have it set to duplicate the data across all three cloud drives for 3x duplication for redundancy.

 

Drive 1: 100 TB Amazon Cloud Drive

Drive 2: 100 TB Gdrive

Drive 3: 100 TB Gdrive

 

I went to access a media file on the pooled drive and I was unable to play the file. The file appeared corrupt. At first I thought that it may have gotten corrupted in transit for the initial transfer. Then I checked other files in the same show and other seasons in the same show, all tv eposides for the one show exhibit the same corruption and refuse to play even copying the file locally from the cloud.

 

I manually went into each drive in the pool and found the file and downloaded it. What I found was, the file was corrupt and would not play on both GDrives volumes but the file was working properly off of the ACD volume.

 

I believe when I added the show I only had 2 cloud drives, 1 ACD and 1 GDrive. When I added the 3rd drive I think it replicated from the 1 GDrive that already had the error with the file and thus duplicating the error to the second GDrive.

 

My question is, how is the file in an inconsistent state across the pool? Shouldn't the file be an exact copy on each volume? I tested removing one of the episodes on both GDrives and it proceeded to mirror the file and it is now working as expected and plays without issue. I would like to be able to tell if there are more shows like this and correct the issue before it becomes unrecoverable. Drivepool should be able to see if the files are in an inconstant state and perhaps even prompt me for which version I would like to keep and mirror to the other drives.

 

I have left the rest of the show in an inconstant state so that I can assist with troubleshooting how to track down and fix the issue.

 

OS: Windows Server 2012 R2 x64

CloudDrive Version: 10.0.0.842

DrivePool Version: 2.2.0.740

Link to comment
Share on other sites

3 answers to this question

Recommended Posts

  • 0

Did StableBit CloudDrive indicate that it was running recovery on your drives? 

If so, there was an issue where the data was getting corrupted by Google (it's a long explanation).  The newest version attempts to automatically repair that data, but it may not be 100%. 

 

In this case, if your drives did get repaired, it *could* cause this. 

 

 

That said, this issue is no longer an issue.  We've made changes that will absolutely make sure that this CANNOT happen in the future. 

 

 

 

 

 

That said, StableBit DrivePool does write identical files, and ties to make sure of this.

If/when there is an issue, it will be flagged in the UI.  However, this relies on accessing the file, in most cases.  It will check the file modify date and if that doesn't match, then it will run a CRC check on the files.  

 

However, given the above issue, the dates may have been fine, so it didn't flag the file for a crc hash. 

 

 


 

This should not occur in the future, but if it does, then please let me know, right away. 

Link to comment
Share on other sites

  • 0

Hi Christopher,

 

I believe when I was testing different versions of the BETA it is possible that the google drive had to run a recovery. I know there was an issue a few times were it prevented the server from rebooting and I had to manually shut it down, thus causing a recovery to run on the drive after the server was powered back on.

 

The date modified had stayed consistent and that it why it did not flag for a CRC check on the file. I did a scan of all files modified in the time frame for error file and found a few more files that had an issue but it leads me to believe this was all related to a single event a last year in which there potentially may have run a drive recovery.

 

Thanks for the update. I will restore the files from the working drive and continue off as normal.

 

Is there currently a way of running a CRC check on the files on the cloud drive without having to re-download the file to the local cache? I would like to run a check  on all the files, just to make sure there isn't anything I am missing.

Link to comment
Share on other sites

  • 0

Is there currently a way of running a CRC check on the files on the cloud drive without having to re-download the file to the local cache? I would like to run a check  on all the files, just to make sure there isn't anything I am missing.

 

Not really... the integrity checks are done as part of the download process for the data.  The checksum hashes are actually stored as part of the chunks of data themselves... so to get to them, you need to download the data.

 

If you want to force this, then running a surface scan on the CloudDrive disk would be a simple way to trigger this, as it will read ALL of the data.  

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...