Freppa Posted February 11, 2017 Share Posted February 11, 2017 This is a topic I think have been discussed before but is still an issue on my setup. If I upload 1GB of data, it takes three to five times as long time as if I was uploading directly utilizing the same speed. I looked at the "technical details" and noticed something peculiar - progress on some threads where over 100% in completion. Why would that be? Ginoliggime and KiaraEvirm 2 Quote Link to comment Share on other sites More sharing options...
0 Christopher (Drashna) Posted February 11, 2017 Share Posted February 11, 2017 You have upload verification enabled, correct? If so, that may be specifically why this is occurring. If not, it may be that we're getting error codes from the provider, and have to retry the chunk. Either way, I've already flagged this for Alex (the Developer). I know we just talked about this sort of thing, but I can't remember exactly for the life of me. https://stablebit.com/Admin/IssueAnalysis/27360 Quote Link to comment Share on other sites More sharing options...
0 Freppa Posted February 12, 2017 Author Share Posted February 12, 2017 Upload verification is on, yes. However isn't that normally handled on checksum? Keep up the great work! (And let us know if we can help on putting pressure on Amazon Cloud Services 😀) Quote Link to comment Share on other sites More sharing options...
0 Christopher (Drashna) Posted February 12, 2017 Share Posted February 12, 2017 Upload verification is completely different than checksumming. The checksum verfiication makes sure that when you download the data, that it is intact, and not modified/corrupted. Upload verification makes sure that the data was uploaded properly (eg not corrupted in transit, and that the provider ACTUALLY received it). This is done by immediately re-downloading the file and comparing it to the cache. That said: The percentage is calculated as a ratio of how much data is read from the local cache / how much data needs to be uploaded. So for example, if 20 MB is read and we're uploading a 100 MB chunk, then we're at 20%. However, this shouldn't really be happening for the Amazon CloudDrive provider. If this is continuing to happen, let me know. Quote Link to comment Share on other sites More sharing options...
0 Freppa Posted February 12, 2017 Author Share Posted February 12, 2017 Ok, I'll correct myself: Upload verification is ​not​ on. I'll run some tests on uploading with newest release to see if it continues over 100%. I'll get back to you on that one. Quote Link to comment Share on other sites More sharing options...
0 Freppa Posted February 12, 2017 Author Share Posted February 12, 2017 Still the same issue. Small file, 15mb, maxed 100mbit line, took 4 times as long. Quote Link to comment Share on other sites More sharing options...
0 Christopher (Drashna) Posted February 12, 2017 Share Posted February 12, 2017 If it does continue to do that, could you enable both trace logging and web logging: http://wiki.covecube.com/StableBit_CloudDrive_Drive_Tracing http://wiki.covecube.com/StableBit_CloudDrive_Web_Logs And try to minimize the logs. The less that is going on, the easier it is for us to go through everything (these are both quite detailed logs). Quote Link to comment Share on other sites More sharing options...
0 Freppa Posted February 12, 2017 Author Share Posted February 12, 2017 Drashna, I have uploaded the two files to dropbox for your viewing pleasure Quote Link to comment Share on other sites More sharing options...
0 Christopher (Drashna) Posted February 13, 2017 Share Posted February 13, 2017 Thanks! I've flagged them for Alex, so he can take a look at them. Quote Link to comment Share on other sites More sharing options...
0 Christopher (Drashna) Posted February 14, 2017 Share Posted February 14, 2017 Okay, this is an issue with the Amazon Cloud Drive API specifically. And this is actually ... entirely normal. The data is being sent with the request. That means that the data is sent before we get a response. So if/when we get a throttling or "HTTP 429" response, the data is essentially "wasted" because it's ignored. We then have to retry, again and again until it works. Because of this, we show the 100+% percentage because we have resent the data REPEATEDLY at this point. So this isn't really a bug, but ... essentially poorly written API ... both on their and our parts. There also, isn't a great way to handle this .... Alex has an idea but basically it's sending two requests per upload, one to make sure that we can, and then a second to actually upload. Quote Link to comment Share on other sites More sharing options...
0 Freppa Posted February 14, 2017 Author Share Posted February 14, 2017 Interesting. It would be interesting to understand how ACD thought when developing such a solution. I must say, if Google Drive Unlimited wasn't so expensive to procure, I would probably switch at this point. Quote Link to comment Share on other sites More sharing options...
0 Christopher (Drashna) Posted February 14, 2017 Share Posted February 14, 2017 Well, the issue is how to handle it. IIRC, Google will actually close the connection in this case, rather than complete and force it to retry. Not really sure which is better from a technical standpoint. Quote Link to comment Share on other sites More sharing options...
Question
Freppa
This is a topic I think have been discussed before but is still an issue on my setup.
If I upload 1GB of data, it takes three to five times as long time as if I was uploading directly utilizing the same speed.
I looked at the "technical details" and noticed something peculiar - progress on some threads where over 100% in completion. Why would that be?
Link to comment
Share on other sites
11 answers to this question
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.