Question about chunk upload

[steffenmand]

Hi,

 

I noticed that after a write thread finishes the upload verification downloads the content to check if its correct. however while it downloads no new upload threads are started - shouldnt it start uploading the next chunk, while it downloads and checks the former one? Guess you are using some sort of queue system and that the file could just be added to the queue if it needs to be reuploaded.

 

Any specific reason why it runs like this? could imagine a little more speed if it can continue upload meanwhile

[WBadminUser]

I tested on a 100/100 connection with 50 MB chunks and i see long pauses between threads even when upload verification is turned off.

I would be interested to know whats going on also.

 

I will add that it is on ACD with 2 threads.

[Christopher (Drashna)]

This depends on the number of threads.  Because Amazon Cloud Drive is limited on the number of threads... it means that you'll see this pretty often, as it has to wait to verify that file (download and check) until it can move onto the next chunk.

[WBadminUser]

Just seems like an inefficient way. If we are allowed two upload threads why are these threads not uploading all the time? I have "Upload Verification" turn off so what is the the hold up?

Here is a screenshot of what i am seeing.

 

This is with 2 upload threads and 4 download.

[Christopher (Drashna)]

We'd need to have logs to see exactly why. 

 

But there are a lot of moving parts here. It may be the connection has to be properly closed and re-opened. Or it could be a response timing issue with Amazon. or it could be something else.

 

 

And if you're using Amazon Cloud Drive, we HIGHLY recommend having upload verification enabled. Frequently, chunks are uploaded that don't come back "right", or at least right away. The upload verification makes sure that the chunks are properly uploaded.