Jump to content

  • Log in with Twitter Log in with Windows Live Log In with Google      Sign In   
  • Create Account

Photo

Files starting to go corrupt, Cloud Drive throws no errors


  • Please log in to reply
53 replies to this topic

#1 steffenmand

steffenmand

    Advanced Member

  • Members
  • PipPipPip
  • 296 posts

Posted 22 January 2017 - 10:26 AM

Recently some of my drives have started to make pretty much all files corrupt on my drives.

 

At first i thought, OK, BETA errors may happen, but now this is starting to happen on multiple drives.

 

Stablebit Cloud Drive seems to prefetch fine and throws no errors, but what arrives is just corrupt data.

 

The data in question have all been lying at rest, so we have not uploaded new corrupt data on top

 

This is the second drive this has happened to,

 

The funny part is, if i try to copy one of the corrupt files from the cloud drive to my own disc, it shows copy speeds of 500+ MB/s and finishes in a few seconds as if it didnt even have to try and download stuff first. In the technical overview i can see prefetch showing "Transfer Completed Successfully" with 0 kbps and 0% completion (attached a snippet of one so you can see)

 

lSh7nMx.jpg

 

Something is seriously wrong here and it seems that it somehow can avoid trying to download the actual chunk and instead just fill in some empty data.

 

I have verified that the chunks do exist on the google drive and i am able to download it myself

 

Been talking to other users who have experienced the same thing!


  • Ginoliggime and Antoineki like this

#2 triadcool

triadcool

    Advanced Member

  • Members
  • PipPipPip
  • 87 posts

Posted 22 January 2017 - 04:28 PM

This is the exact issue that I have with 3 of my drives totaling around 48TB of data.  I have two Google Drive accounts and this affected all of the drives on one of the Google Drive accounts.  The 3 drives on my other Google account are fine at the moment.

 

 

This issue affected all 3 of the drives at the same time while on version .723.  I upgraded to the latest version 1.0.0.802 with no change in behavior.

 

 

D5D97JK.png

jPYVmdA.png

it5nko2.png



#3 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 8,208 posts
  • LocationSan Diego, CA, USA

Posted 22 January 2017 - 08:12 PM

That's definitely a serious issue. You've posted this in a ticket already, and I've responded there first. 

 

 
 
Could you do both of these: 
 
 
And if this drive is not encrypted and doesn't contain any sensitive info, would it be possible for you to share the drive's content with us? 
 
 
 
The first log should see what's going on on the driver level, and the web logs should allow us to see the data being transferred to and from Google's servers. 
 
 
And is this only happening with drives that were affected by the double CloudDrive folder issues? Or are you seeing this on drives created before or since? 
 
 
 
And, if possible, reference "425755" if you open a ticket or upload 

Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.


#4 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 8,208 posts
  • LocationSan Diego, CA, USA

Posted 22 January 2017 - 08:45 PM

Okay, could you confirm that the Chunk actually exists on the provider? 

 

In the techncial details windows, the "Chunk" column lists the ID.  That should match a file on the provider 

 

Specifically in \StableBit CloudDrive\Cloudpart-xxxxx\xxxxx-CONTENT\xxxxxx-chunk-###

 

If this file doesn't exist on the Cloud Provider, then we assume the file is NULL (all zeros), as this is usually a safe assumption. 

 

 

However, if the file is supposed to exist, and doesn't, then it would absolutely cause this issue.  

 

 

 

 

Unfortunately, I suspect that this is exactly the case. 

 

Worse, I know that Google Drive had an outage on the 17th... if they lost files, it could cause this, as well. 


Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.


#5 steffenmand

steffenmand

    Advanced Member

  • Members
  • PipPipPip
  • 296 posts

Posted 22 January 2017 - 09:30 PM

Haven't created new drives since the double folder issue, but got 4 other drives which still works.

 

Chunks exists on the provider and I am able to download them through the Web interface!

 

First drive I got this issue on was on the 8th and the second was the 19th so days before and after the outage!

 

It just seems like it skips the download and fills in empty data



#6 triadcool

triadcool

    Advanced Member

  • Members
  • PipPipPip
  • 87 posts

Posted 22 January 2017 - 09:31 PM

I can confirm that the chunks still exist on Google Drive.  This issue came up well before the Google outage on the 17th for me.  I was never affected by the double CloudDrive folder issue.



#7 steffenmand

steffenmand

    Advanced Member

  • Members
  • PipPipPip
  • 296 posts

Posted 24 January 2017 - 12:09 PM

I updated the ticket with all the desired info :)



#8 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 8,208 posts
  • LocationSan Diego, CA, USA

Posted 25 January 2017 - 12:03 AM

Thank you guys. 

 

And it definitely looks like there may be something going on here, so this may actually be a bug. 

 

I've updated Alex, and messaged him directly about it, and I'll let you know when he has a response. 


  • steffenmand likes this

Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.


#9 CalvinNL

CalvinNL

    Newbie

  • Members
  • Pip
  • 9 posts

Posted 01 February 2017 - 08:03 AM

I am having similar issues. For some reason, a folder on my drive containing 2TB of data has gone corrupt. The build-in Windows disk check tool didn't do anything to solve the problem. After updating to .819 the drive appeared as RAW in Disk Management in Windows and is no longer accessible from Windows Explorer. 

 

What's happening :(



#10 Choruptian

Choruptian

    Member

  • Members
  • PipPip
  • 27 posts

Posted 01 February 2017 - 12:35 PM

I am having similar issues. For some reason, a folder on my drive containing 2TB of data has gone corrupt. The build-in Windows disk check tool didn't do anything to solve the problem. After updating to .819 the drive appeared as RAW in Disk Management in Windows and is no longer accessible from Windows Explorer. 

 

What's happening :(

 

Same issue here.



#11 Akame

Akame

    Member

  • Members
  • PipPip
  • 20 posts

Posted 01 February 2017 - 01:19 PM

Same here, updated to 819, got some corrupt folders that wouldn't open so ran chkdsk /r and then it went RAW. 

 

My cloud usage went from 11.2 TB to 375 GB at the UI of the program.

 

I can confirm the chunks are still at Google Drive provider but it seems it lost track of them. 

I ran testdisk http://www.cgsecurit...g/wiki/TestDiskwhich allows you to see files even though it's RAW.

 

When you start testdisk.exe press Create for log file

Pick the physicaldrive which has your clouddrive size. 

Pick EFI GPT, then go to advanced and press down button to press on the other bigger size and do LIST, when I do list I only see 375 GB of information which is worrying. 



#12 steffenmand

steffenmand

    Advanced Member

  • Members
  • PipPipPip
  • 296 posts

Posted 01 February 2017 - 04:48 PM

I suggest you all open a ticket and reference "425755"  which is my ticket. 

 

We are all having the same issue :) It is simply the chunk database which is not getting all the id's of the chunks and then instead putting in empty data. 



#13 Akame

Akame

    Member

  • Members
  • PipPip
  • 20 posts

Posted 01 February 2017 - 05:14 PM

I suggest you all open a ticket and reference "425755"  which is my ticket. 

 

We are all having the same issue :) It is simply the chunk database which is not getting all the id's of the chunks and then instead putting in empty data. 

I think our data is lost due to the way MFT works if it is MFT it should be fairly hard to restore. 



#14 Gbyrd

Gbyrd

    Advanced Member

  • Members
  • PipPipPip
  • 72 posts

Posted 01 February 2017 - 05:18 PM

Same here, updated to 819, got some corrupt folders that wouldn't open so ran chkdsk /r and then it went RAW. 

 

My cloud usage went from 11.2 TB to 375 GB at the UI of the program.

 

I can confirm the chunks are still at Google Drive provider but it seems it lost track of them. 

I ran testdisk http://www.cgsecurit...g/wiki/TestDiskwhich allows you to see files even though it's RAW.

 

When you start testdisk.exe press Create for log file

Pick the physicaldrive which has your clouddrive size. 

Pick EFI GPT, then go to advanced and press down button to press on the other bigger size and do LIST, when I do list I only see 375 GB of information which is worrying. 

 

But if you're using it like that it is only gonna see what is physically in the cache drive. It won't show you what is in your cloud. So i don't think this is accurate.



#15 steffenmand

steffenmand

    Advanced Member

  • Members
  • PipPipPip
  • 296 posts

Posted 01 February 2017 - 06:33 PM

I think our data is lost due to the way MFT works if it is MFT it should be fairly hard to restore. 

 

Well as far as i can see, then the system knows what chunks to use. They are simply just not linked to the proper id on the provider. Thats why we were seeing tons of chunks completing successfully with 0%.

I think the big problem with data would arise if the problem was that it lost reference to what chunk numbers refers to which file


  • Akame likes this

#16 Akame

Akame

    Member

  • Members
  • PipPip
  • 20 posts

Posted 01 February 2017 - 10:07 PM

If you enter some folders does it say corrupt? As the 819 update may not be related

#17 steffenmand

steffenmand

    Advanced Member

  • Members
  • PipPipPip
  • 296 posts

Posted 01 February 2017 - 10:17 PM

For me I can see everything fine - it's just when I download files that they get filled with empty data instead of the correct chunk data - all because it lost the reference to the chunk on cloud drive



#18 KiLLeRRaT

KiLLeRRaT

    Member

  • Members
  • PipPip
  • 23 posts

Posted 02 February 2017 - 01:01 AM

When trying to enter some folders for me it says that it's corrupt.  Also getting corruption errors when I try to robocopy stuff from my cloud drive across to my machine.



#19 Akame

Akame

    Member

  • Members
  • PipPip
  • 20 posts

Posted 02 February 2017 - 09:42 AM

I can say I've managed to restore my stuff, so apparently the new update or maybe two updates backwards added something called authorative chunk ids so there is only a metadata folder and nothing in chunkid folder. So I used the google business restore which allows you to restore permanently deleted items, data-chunkIdStorage is where the chunks where saved before it migrated over, luckily I had one from 31 the day the migration and the corruption started. Cache is at 0 B but afraid of a reboot at this point as moving pc made it unallocated. Moving files works and opening. Adding as well I think but I put it to read-only.

 

1: Updating to 819 made some folders corrupt as I could not access them.

2. I ran chkdsk and it detected errors

3. I ran chkdsk /r and it deleted index and fixed errors so cloud usage showed 6 GB at this point. 

4. After several troubleshooting steps involving contacting google support they notified me if you go to google suite admin center and go to user you can press at three dot menu and press store, but you can only specify date and not what files to restore so you may end up with multiple chunks. As probably most know API deletes on CloudDrive deletes a file permanently which rendered our files unable to restore without going to google suite admin center, I think most has unlimited plan.

5. After restoring my fourth chunkid storage file from 31st it migrated into Metadata and as we know API does not manage versions when stablebit is editing them which proved to be a problem if I wanted whole drive restored, but when I was on my fifth file it managed to mount but still corruption errors. And the random file I tried CloudDrive deleted the chunkid file for the migration process and my clouddrive was 12.1 TB, even though some chunkid messed with my CONTENT folder and I may have multiple chunks that are different and I believe I may have corruption due to the steps involved I took the risk. 

 

At this point was my drive no longer unallocated and I was able to access any file and folder, calculated i may have around 80 MB corruption, so I transferred over the most important stuff and transferring less important now and it seems stable, remind me not to update versions as soon as it releases. 

 

I have been up all night and pretty tired writing this, admin can edit this to make it more readable if they want. But with that said StableBit CloudDrive is a good product which is in beta, I've had no issues with DrivePool. Also you may wanna wait on @Drashna before you do what I did due to the consequences involved. 



#20 KiLLeRRaT

KiLLeRRaT

    Member

  • Members
  • PipPip
  • 23 posts

Posted 02 February 2017 - 09:57 AM

I can say I've managed to restore my stuff, so apparently the new update or maybe two updates backwards added something called authorative chunk ids so there is only a metadata folder and nothing in chunkid folder. So I used the google business restore which allows you to restore permanently deleted items, data-chunkIdStorage is where the chunks where saved before it migrated over, luckily I had one from 31 the day the migration and the corruption started. Cache is at 0 B but afraid of a reboot at this point as moving pc made it unallocated. Moving files works and opening. Adding as well I think but I put it to read-only.

 

1: Updating to 819 made some folders corrupt as I could not access them.

2. I ran chkdsk and it detected errors

3. I ran chkdsk /r and it deleted index and fixed errors so cloud usage showed 6 GB at this point. 

4. After several troubleshooting steps involving contacting google support they notified me if you go to google suite admin center and go to user you can press at three dot menu and press store, but you can only specify date and not what files to restore so you may end up with multiple chunks. As probably most know API deletes on CloudDrive deletes a file permanently which rendered our files unable to restore without going to google suite admin center, I think most has unlimited plan.

5. After restoring my fourth chunkid storage file from 31st it migrated into Metadata and as we know API does not manage versions when stablebit is editing them which proved to be a problem if I wanted whole drive restored, but when I was on my fifth file it managed to mount but still corruption errors. And the random file I tried CloudDrive deleted the chunkid file for the migration process and my clouddrive was 12.1 TB, even though some chunkid messed with my CONTENT folder and I may have multiple chunks that are different and I believe I may have corruption due to the steps involved I took the risk. 

 

At this point was my drive no longer unallocated and I was able to access any file and folder, calculated i may have around 80 MB corruption, so I transferred over the most important stuff and transferring less important now and it seems stable, remind me not to update versions as soon as it releases. 

 

I have been up all night and pretty tired writing this, admin can edit this to make it more readable if they want. But with that said StableBit CloudDrive is a good product which is in beta, I've had no issues with DrivePool. Also you may wanna wait on @Drashna before you do what I did due to the consequences involved. 

 

 

That sounds pretty painful! Let's hope they can give us a quick easy fix for this via an update!






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users