Jump to content
  • 0

Files starting to go corrupt, Cloud Drive throws no errors


steffenmand

Question

Recently some of my drives have started to make pretty much all files corrupt on my drives.

 

At first i thought, OK, BETA errors may happen, but now this is starting to happen on multiple drives.

 

Stablebit Cloud Drive seems to prefetch fine and throws no errors, but what arrives is just corrupt data.

 

The data in question have all been lying at rest, so we have not uploaded new corrupt data on top

 

This is the second drive this has happened to,

 

The funny part is, if i try to copy one of the corrupt files from the cloud drive to my own disc, it shows copy speeds of 500+ MB/s and finishes in a few seconds as if it didnt even have to try and download stuff first. In the technical overview i can see prefetch showing "Transfer Completed Successfully" with 0 kbps and 0% completion (attached a snippet of one so you can see)

 

lSh7nMx.jpg

 

Something is seriously wrong here and it seems that it somehow can avoid trying to download the actual chunk and instead just fill in some empty data.

 

I have verified that the chunks do exist on the google drive and i am able to download it myself

 

Been talking to other users who have experienced the same thing!

Link to comment
Share on other sites

Recommended Posts

  • 0

This is the exact issue that I have with 3 of my drives totaling around 48TB of data.  I have two Google Drive accounts and this affected all of the drives on one of the Google Drive accounts.  The 3 drives on my other Google account are fine at the moment.

 

 

This issue affected all 3 of the drives at the same time while on version .723.  I upgraded to the latest version 1.0.0.802 with no change in behavior.

 

 

D5D97JK.png

jPYVmdA.png

it5nko2.png

Link to comment
Share on other sites

  • 0

That's definitely a serious issue. You've posted this in a ticket already, and I've responded there first. 

 

 
 
Could you do both of these: 
 
 
And if this drive is not encrypted and doesn't contain any sensitive info, would it be possible for you to share the drive's content with us? 
 
 
 
The first log should see what's going on on the driver level, and the web logs should allow us to see the data being transferred to and from Google's servers. 
 
 
And is this only happening with drives that were affected by the double CloudDrive folder issues? Or are you seeing this on drives created before or since? 
 
 
 
And, if possible, reference "425755" if you open a ticket or upload 
Link to comment
Share on other sites

  • 0

Okay, could you confirm that the Chunk actually exists on the provider? 

 

In the techncial details windows, the "Chunk" column lists the ID.  That should match a file on the provider 

 

Specifically in \StableBit CloudDrive\Cloudpart-xxxxx\xxxxx-CONTENT\xxxxxx-chunk-###

 

If this file doesn't exist on the Cloud Provider, then we assume the file is NULL (all zeros), as this is usually a safe assumption. 

 

 

However, if the file is supposed to exist, and doesn't, then it would absolutely cause this issue.  

 

 

 

 

Unfortunately, I suspect that this is exactly the case. 

 

Worse, I know that Google Drive had an outage on the 17th... if they lost files, it could cause this, as well. 

Link to comment
Share on other sites

  • 0

Haven't created new drives since the double folder issue, but got 4 other drives which still works.

 

Chunks exists on the provider and I am able to download them through the Web interface!

 

First drive I got this issue on was on the 8th and the second was the 19th so days before and after the outage!

 

It just seems like it skips the download and fills in empty data

Link to comment
Share on other sites

  • 0

I am having similar issues. For some reason, a folder on my drive containing 2TB of data has gone corrupt. The build-in Windows disk check tool didn't do anything to solve the problem. After updating to .819 the drive appeared as RAW in Disk Management in Windows and is no longer accessible from Windows Explorer. 

 

What's happening :(

Link to comment
Share on other sites

  • 0

I am having similar issues. For some reason, a folder on my drive containing 2TB of data has gone corrupt. The build-in Windows disk check tool didn't do anything to solve the problem. After updating to .819 the drive appeared as RAW in Disk Management in Windows and is no longer accessible from Windows Explorer. 

 

What's happening :(

 

Same issue here.

Link to comment
Share on other sites

  • 0

Same here, updated to 819, got some corrupt folders that wouldn't open so ran chkdsk /r and then it went RAW. 

 

My cloud usage went from 11.2 TB to 375 GB at the UI of the program.

 

I can confirm the chunks are still at Google Drive provider but it seems it lost track of them. 

I ran testdisk http://www.cgsecurity.org/wiki/TestDiskwhich allows you to see files even though it's RAW.

 

When you start testdisk.exe press Create for log file

Pick the physicaldrive which has your clouddrive size. 

Pick EFI GPT, then go to advanced and press down button to press on the other bigger size and do LIST, when I do list I only see 375 GB of information which is worrying. 

Link to comment
Share on other sites

  • 0

I suggest you all open a ticket and reference "425755"  which is my ticket. 

 

We are all having the same issue :) It is simply the chunk database which is not getting all the id's of the chunks and then instead putting in empty data. 

I think our data is lost due to the way MFT works if it is MFT it should be fairly hard to restore. 

Link to comment
Share on other sites

  • 0

Same here, updated to 819, got some corrupt folders that wouldn't open so ran chkdsk /r and then it went RAW. 

 

My cloud usage went from 11.2 TB to 375 GB at the UI of the program.

 

I can confirm the chunks are still at Google Drive provider but it seems it lost track of them. 

I ran testdisk http://www.cgsecurity.org/wiki/TestDiskwhich allows you to see files even though it's RAW.

 

When you start testdisk.exe press Create for log file

Pick the physicaldrive which has your clouddrive size. 

Pick EFI GPT, then go to advanced and press down button to press on the other bigger size and do LIST, when I do list I only see 375 GB of information which is worrying. 

 

But if you're using it like that it is only gonna see what is physically in the cache drive. It won't show you what is in your cloud. So i don't think this is accurate.

Link to comment
Share on other sites

  • 0

I think our data is lost due to the way MFT works if it is MFT it should be fairly hard to restore. 

 

Well as far as i can see, then the system knows what chunks to use. They are simply just not linked to the proper id on the provider. Thats why we were seeing tons of chunks completing successfully with 0%.

I think the big problem with data would arise if the problem was that it lost reference to what chunk numbers refers to which file

Link to comment
Share on other sites

  • 0

I can say I've managed to restore my stuff, so apparently the new update or maybe two updates backwards added something called authorative chunk ids so there is only a metadata folder and nothing in chunkid folder. So I used the google business restore which allows you to restore permanently deleted items, data-chunkIdStorage is where the chunks where saved before it migrated over, luckily I had one from 31 the day the migration and the corruption started. Cache is at 0 B but afraid of a reboot at this point as moving pc made it unallocated. Moving files works and opening. Adding as well I think but I put it to read-only.

 

1: Updating to 819 made some folders corrupt as I could not access them.

2. I ran chkdsk and it detected errors

3. I ran chkdsk /r and it deleted index and fixed errors so cloud usage showed 6 GB at this point. 

4. After several troubleshooting steps involving contacting google support they notified me if you go to google suite admin center and go to user you can press at three dot menu and press store, but you can only specify date and not what files to restore so you may end up with multiple chunks. As probably most know API deletes on CloudDrive deletes a file permanently which rendered our files unable to restore without going to google suite admin center, I think most has unlimited plan.

5. After restoring my fourth chunkid storage file from 31st it migrated into Metadata and as we know API does not manage versions when stablebit is editing them which proved to be a problem if I wanted whole drive restored, but when I was on my fifth file it managed to mount but still corruption errors. And the random file I tried CloudDrive deleted the chunkid file for the migration process and my clouddrive was 12.1 TB, even though some chunkid messed with my CONTENT folder and I may have multiple chunks that are different and I believe I may have corruption due to the steps involved I took the risk. 

 

At this point was my drive no longer unallocated and I was able to access any file and folder, calculated i may have around 80 MB corruption, so I transferred over the most important stuff and transferring less important now and it seems stable, remind me not to update versions as soon as it releases. 

 

I have been up all night and pretty tired writing this, admin can edit this to make it more readable if they want. But with that said StableBit CloudDrive is a good product which is in beta, I've had no issues with DrivePool. Also you may wanna wait on @Drashna before you do what I did due to the consequences involved. 

Link to comment
Share on other sites

  • 0

I can say I've managed to restore my stuff, so apparently the new update or maybe two updates backwards added something called authorative chunk ids so there is only a metadata folder and nothing in chunkid folder. So I used the google business restore which allows you to restore permanently deleted items, data-chunkIdStorage is where the chunks where saved before it migrated over, luckily I had one from 31 the day the migration and the corruption started. Cache is at 0 B but afraid of a reboot at this point as moving pc made it unallocated. Moving files works and opening. Adding as well I think but I put it to read-only.

 

1: Updating to 819 made some folders corrupt as I could not access them.

2. I ran chkdsk and it detected errors

3. I ran chkdsk /r and it deleted index and fixed errors so cloud usage showed 6 GB at this point. 

4. After several troubleshooting steps involving contacting google support they notified me if you go to google suite admin center and go to user you can press at three dot menu and press store, but you can only specify date and not what files to restore so you may end up with multiple chunks. As probably most know API deletes on CloudDrive deletes a file permanently which rendered our files unable to restore without going to google suite admin center, I think most has unlimited plan.

5. After restoring my fourth chunkid storage file from 31st it migrated into Metadata and as we know API does not manage versions when stablebit is editing them which proved to be a problem if I wanted whole drive restored, but when I was on my fifth file it managed to mount but still corruption errors. And the random file I tried CloudDrive deleted the chunkid file for the migration process and my clouddrive was 12.1 TB, even though some chunkid messed with my CONTENT folder and I may have multiple chunks that are different and I believe I may have corruption due to the steps involved I took the risk. 

 

At this point was my drive no longer unallocated and I was able to access any file and folder, calculated i may have around 80 MB corruption, so I transferred over the most important stuff and transferring less important now and it seems stable, remind me not to update versions as soon as it releases. 

 

I have been up all night and pretty tired writing this, admin can edit this to make it more readable if they want. But with that said StableBit CloudDrive is a good product which is in beta, I've had no issues with DrivePool. Also you may wanna wait on @Drashna before you do what I did due to the consequences involved. 

 

 

That sounds pretty painful! Let's hope they can give us a quick easy fix for this via an update!

Link to comment
Share on other sites

  • 0

Fixed it for all drives which had not gone RAW.

 

The one the went RAW seemed to have multiple chunks on the main chunks. So i just tried deleting the new versions (took a backup) and started reindexing all over to see if that would fix the issues and let it mount properly. I guess one of those chunks hold the MFT data and if it uploaded a new empty file, it would ofc. not be able to mount, because the fix loads the newest version == empty file.

 

Will report back if that fixes the issue with the RAW drive :)

Link to comment
Share on other sites

  • 0

Fixed it for all drives which had not gone RAW.

 

The one the went RAW seemed to have multiple chunks on the main chunks. So i just tried deleting the new versions (took a backup) and started reindexing all over to see if that would fix the issues and let it mount properly. I guess one of those chunks hold the MFT data and if it uploaded a new empty file, it would ofc. not be able to mount, because the fix loads the newest version == empty file.

 

Will report back if that fixes the issue with the RAW drive :)

 

Deleting all the newest chunks when we had multiple chunks with the same key worked. drive now mounts properly! Before trying this make sure to backup the chunks so you can fix it again!

 

if you already indexed just restart the indexing procedure by resetting the settings and the database under troubleshooting -> reset settings as your index currently will be pointing to the wrong chunk where we had multiple

Link to comment
Share on other sites

  • 0

Deleting all the newest chunks when we had multiple chunks with the same key worked. drive now mounts properly! Before trying this make sure to backup the chunks so you can fix it again!

 

if you already indexed just restart the indexing procedure by resetting the settings and the database under troubleshooting -> reset settings as your index currently will be pointing to the wrong chunk where we had multiple

 

Just curious my drive mounted but is this any issue? As I've had no issues so bit worried if deleting them will destroy my drive. They must of came back from my restore process when I fixed the chunkIDs and raw drive. 

 

CloudDrive.Service.exe Warning 0 [ChunkIdSQLiteStorage:36] Multiple files (2) found for chunk 577910. Choosing first one. 2017-02-04 01:52:20Z 9897885844
CloudDrive.Service.exe Warning 0 [ChunkIdSQLiteStorage:25] Multiple files (2) found for chunk 137. Choosing first one. 2017-02-04 01:53:09Z 10038302772
CloudDrive.Service.exe Warning 0 [ChunkIdSQLiteStorage:41] Multiple files (2) found for chunk 137. Choosing first one. 2017-02-04 01:53:30Z 10098630653
CloudDrive.Service.exe Warning 0 [ChunkIdSQLiteStorage:33] Multiple files (2) found for chunk 133. Choosing first one. 2017-02-04 01:54:14Z 10221480929
CloudDrive.Service.exe Warning 0 [ChunkIdSQLiteStorage:33] Multiple files (2) found for chunk 133. Choosing first one. 2017-02-04 01:54:18Z 10234209721
CloudDrive.Service.exe Warning 0 [ChunkIdSQLiteStorage:41] Multiple files (2) found for chunk 219021. Choosing first one. 2017-02-04 01:54:19Z 10237928233
CloudDrive.Service.exe Warning 0 [ChunkIdSQLiteStorage:33] Multiple files (2) found for chunk 6. Choosing first one. 2017-02-04 01:54:23Z 10247725571
CloudDrive.Service.exe Warning 0 [ChunkIdSQLiteStorage:48] Multiple files (2) found for chunk 137. Choosing first one. 2017-02-04 01:54:23Z 10247844483
CloudDrive.Service.exe Warning 0 [ChunkIdSQLiteStorage:46] Multiple files (2) found for chunk 160. Choosing first one. 2017-02-04 01:54:23Z 10247887275
CloudDrive.Service.exe Warning 0 [ChunkIdSQLiteStorage:27] Multiple files (2) found for chunk 577910. Choosing first one. 2017-02-04 01:54:23Z 10247909988
CloudDrive.Service.exe Warning 0 [ChunkIdSQLiteStorage:52] Multiple files (2) found for chunk 133. Choosing first one. 2017-02-04 01:54:23Z 10248987571
CloudDrive.Service.exe Warning 0 [ChunkIdSQLiteStorage:33] Multiple files (2) found for chunk 6. Choosing first one. 2017-02-04 01:54:25Z 10254499991
CloudDrive.Service.exe Warning 0 [ChunkIdSQLiteStorage:41] Multiple files (2) found for chunk 219021. Choosing first one. 2017-02-04 01:54:27Z 10259745493
CloudDrive.Service.exe Warning 0 [ChunkIdSQLiteStorage:48] Multiple files (2) found for chunk 137. Choosing first one. 2017-02-04 01:54:28Z 10261123716
CloudDrive.Service.exe Warning 0 [ChunkIdSQLiteStorage:46] Multiple files (2) found for chunk 160. Choosing first one. 2017-02-04 01:54:28Z 10262252306
CloudDrive.Service.exe Warning 0 [ChunkIdSQLiteStorage:27] Multiple files (2) found for chunk 577910. Choosing first one. 2017-02-04 01:54:29Z 10266083381
CloudDrive.Service.exe Warning 0 [ChunkIdSQLiteStorage:3] Multiple files (2) found for chunk 45398. Choosing first one. 2017-02-04 01:56:09Z 10549383981
CloudDrive.Service.exe Warning 0 [ChunkIdSQLiteStorage:24] Multiple files (2) found for chunk 45398. Choosing first one. 2017-02-04 01:56:30Z 10609982457
CloudDrive.Service.exe Warning 0 [ChunkIdSQLiteStorage:39] Multiple files (2) found for chunk 45398. Choosing first one. 2017-02-04 01:56:39Z 10633779392
Link to comment
Share on other sites

  • 0

As long as you backup chunks it should be possible to revert back by uploading then again and reindexing :-)

 

I chose to go for the older ones, as i knew ihad not done any changes since the updates where they were made. so i trusted the old data more than the new, as the new potentially could have faulty data - this was also why my drive went RAW as i had multiple chunks on the chunk which contained the MFT data - the new one was corrupt, the old just fine

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...