Jump to content
  • 0

Stablebit "overupload" data


Freppa

Question

This is a topic I think have been discussed before but is still an issue on my setup.

If I upload 1GB of data, it takes three to five times as long time as if I was uploading directly utilizing the same speed.

I looked at the "technical details" and noticed something peculiar - progress on some threads where over 100% in completion. Why would that be?

post-3277-0-64363500-1486790463_thumb.gif

Link to comment
Share on other sites

11 answers to this question

Recommended Posts

  • 0

You have upload verification enabled, correct?

 

If so, that may be specifically why this is occurring.

 

If not, it may be that we're getting error codes from the provider, and have to retry the chunk.

 

Either way, I've already flagged this for Alex (the Developer).  I know we just talked about this sort of thing, but I can't remember exactly for the life of me. 

https://stablebit.com/Admin/IssueAnalysis/27360

Link to comment
Share on other sites

  • 0

Upload verification is completely different than checksumming.

 

The checksum verfiication makes sure that when you download the data, that it is intact, and not modified/corrupted. 

 

Upload verification makes sure that the data was uploaded properly (eg not corrupted in transit, and that the provider ACTUALLY received it). This is done by immediately re-downloading the file and comparing it to the cache. 

 

 

 

That said:

 

 

 

The percentage is calculated as a ratio of how much data is read from the local cache / how much data needs to be uploaded.
 
So for example, if 20 MB is read and we're uploading a 100 MB chunk, then we're at 20%.

 

However, this shouldn't really be happening for the Amazon CloudDrive provider. 

 

If this is continuing to happen, let me know. 

Link to comment
Share on other sites

  • 0

If it does continue to do that, could you enable both trace logging and web logging: 

http://wiki.covecube.com/StableBit_CloudDrive_Drive_Tracing

 

http://wiki.covecube.com/StableBit_CloudDrive_Web_Logs

 

And try to minimize the logs. The less that is going on, the easier it is for us to go through everything (these are both quite detailed logs).

Link to comment
Share on other sites

  • 0

Okay, this is an issue with the Amazon Cloud Drive API specifically. And this is actually ... entirely normal. 

 

The data is being sent with the request.  That means that the data is sent before we get a response.  So if/when we get a throttling or "HTTP 429" response, the data is essentially "wasted" because it's ignored.  We then have to retry, again and again until it works.

 

Because of this, we show the 100+% percentage because we have resent the data REPEATEDLY at this point. 

 

 

So this isn't really a bug, but ... essentially poorly written API ... both on their and our parts.  
There also, isn't a great way to handle this .... Alex has an idea but basically it's sending two requests per upload, one to make sure that we can, and then a second to actually upload.   

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...