Jump to content
Covecube Inc.
  • 0
WBadminUser

Upload Verification vs Just a checksum check

Question

I absolutely love this product just which i had a better back-end then Amazon cloud drive.

My question is about the option of Upload Verification. This actually pulls the block down to verify it? with this option not checked i assume it still does a checksum check on the data. 

 

Thanks for the clarification.

 

 

Share this post


Link to post
Share on other sites

3 answers to this question

Recommended Posts

  • 0

Never mind i found the tool tips that explain it, and it works like i thought it did. 

That checkbox to ignore checksums is pretty scary though.

Generally, it's not needed though. Most providers are very good about getting the data properly uploaded, or erroring out if there was any issues.

 

However, some providers (such as Amazon Cloud Drive, especially) do run into errors. This is to ensure that they're uploading the data correctly *and* that it's actually there.

 

 

That said, yes, it does pull the data back down once it's uploaded, to make sure that it's been properly uploaded and that the blocks of data are intact.  Once it has, it moves onto the next chunk... otherwise, it re-uploads the file.

Share this post


Link to post
Share on other sites
  • 0

I absolutely love this product just which i had a better back-end then Amazon cloud drive.

My question is about the option of Upload Verification. This actually pulls the block down to verify it? with this option not checked i assume it still does a checksum check on the data. 

 

Thanks for the clarification.

 

StableBit CloudDrive performs a checksum check after downloading every chunk, if and only if, the Use checksum verification checkbox is checked when you first create the drive.

Hvhq6Vf.png

It's always enabled by default for cloud providers, and not available as an option for the Local Disk and File Share providers.

 

The difference between using upload verification vs. simply just checksum verification is that, checksum verification is performed much later. That is, it's performed when you need to download that chunk from the provider as a result of a local read request from the local cloud drive.

 

While upload verification is performed at upload time. It uploads a chunk of data to the cloud and then downloads it to make sure that what was downloaded is the same data that was uploaded. And here's the key difference, if it's not, then that chunk upload will be retried. If multiple retries fail, you will be notified in the UI and uploading will halt for 1 minute after which more retries will follow.

 

Upload verification protects your data from the scenario where the cloud provider receives your data and then saves it incorrectly, or if the data was corrupted in transit. Upload verification will detect this scenario and correct it, while checksum verification will detect it much later, and not correct it.

 

How common is this? From my testing, and remember this is still a 1.0 BETA, I've already seen this happen. And specifically I've seen it happen with Amazon Cloud Drive, where an uploaded file would not download at all. I've confirmed this with the Amazon Cloud Drive web interface, so it definitely was not a bug in our software. The file got into some kind of corrupted state where its metadata was there, but its contents were gone.

 

So for non enterprise-grade providers, I really recommend that upload verification be enabled.

 

What is the point of disabling checksum verification at download time?

 

If you run into the scenario where your data is saved incorrectly to the cloud, and you didn't have upload verification enabled. When you try to read that data back, you will get a checksum verification error. This is exactly how a physical hard drive works, and this is similar to having a "bad sector". Your hard drive will never allow corrupt data to be read back, it will retry over and over again in hopes that the original data can be recovered. At this point your system may lock up or freeze temporarily because it can't read or write to the drive in a timely manner.

 

This is the exact behavior that StableBit CloudDrive will emulate, except that the data being read back is now coming from the cloud. If for some reason you want to break out of this "loop", and allow the corrupt cloud data to be read back, you can toggle Ignore checkum on, and that corrupt data will be read back.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

×
×
  • Create New...