Poor performance Google Drive (not rate limit)

[john doe]

removed

[Christopher (Drashna)]

If pausing the connection and then restarting it fixes the issue, then the issue is likely a throttling issue.  More than likely, we're getting throttling requests for Google and respecting it. 

 

And if this is the case, try reducing the number of threads that Google Drive is using.  And if possible, try using larger chunk sizes when creating the drive, and see if that helps. 

[john doe]

removed

[john doe]

removed

[wid_sbdp]

Is the API issue being worked at all with Google yet? Last I read it's "something we're going to address with them." I'm super pro-StableBit, but I'm honestly about to give up and just go to massive amounts of unencrypted drives pooled together and use something like rclone and risk the ToS bans because it would be 100000x quicker to reupload everything at 500 MB/s than it would encrypted at 11 MB/s :/

[Christopher (Drashna)]

Is the API issue being worked at all with Google yet? Last I read it's "something we're going to address with them." I'm super pro-StableBit, but I'm honestly about to give up and just go to massive amounts of unencrypted drives pooled together and use something like rclone and risk the ToS bans because it would be 100000x quicker to reupload everything at 500 MB/s than it would encrypted at 11 MB/s :/

 

In the latest version (770), we've implemented an API pool, so that may help with performance. 

 

You will need to re-authorize the drives to use this pool.  This should be found under "Disk Options" for each drive. 

 

If you have done this and you're still seeing performance issues, let me know.

[john doe]

removed

[Christopher (Drashna)]

Yes I noticed very bad performance (not even close to the BW usage you see in my video) with the old keys (roughly around 20~60 mbps) so I updated on the 20th to the 770 build. Initial speeds was OK for a few hours (5-6), around 200~280 mbps with 100% I/O usage.

 

Yesterday speeds went back to the same as with the old api keys (40-80 mbps) and has been steady since then. I/O usage on the HDD is now at 30~50% so I'm sure it isn't that.

 

If Alex want's I'd let him remote controll the PC doing the upload, if he want's to debug.

 

PS: yes, I did re-authorize.

 

 

Could you grab the logs from the system? 

 

And do you see a "turtle" icon next to the bandwidth usage? 

 

If so, then the slower speed may be because you're being throttled.  And in this case, reducing the number of threads may help.

[thnz]

In the latest version (770), we've implemented an API pool, so that may help with performance. 

 

Hopefully Google doesn't take exception to that.

[jimbb]

I experience the same issue as well and on the latest version (770).

 

I can upload between 300-600 mbps on every fresh reboot, and after a while, it drops to 50-150 mbps and later dropped to 0-50 mbps. (On a 1Gbps line)

 

The weird thing is that on every fresh restart, I can go back to 300-600 mbps for a while until it drops again. From the look of it, it seems more like a CloudDrive issue rather than Google.

 

Tried changing the thread settings etc and nothing seems to help.

 

Already reported to support and waiting for a resolution

[john doe]

removed

[raidz]

I am seeing the exact same issues as these two guys. I uploaded logs about a week ago so hopefully Alex can give this some time soon.

[jimbb]

Exactly the same issue for me (just lower bandwidth  ), I'll see if I can find some logs when I get home (OS logs or CloudDrive?)

 

I am seeing the exact same issues as these two guys. I uploaded logs about a week ago so hopefully Alex can give this some time soon.

 

 

They came back to me this morning saying it might be Google drive who is limiting the API request frequent. However, I disagree as I have set the thread number to 1 on both upload and download and the speed drop is still significant comparing between fresh reboot (50-100mbps per thread) vs after 2-3 hours of run time (0-15 mbps per thread).

 

 They have advised me to tune up the chunk size as well which I created a new drive to test the theory, but still having the same issue over time

 

Will keep you guys as we go along. 

 

[raidz]

My Chunk size is at 100mb, Minimum download at 10mb, Download threads at 12 and Upload Threads at 6. No matter how many threads I set (more or less) it is still slow as molasses (on a 1gbit/1gbit line).  Download speeds have decreased as well due to delay in thread response time. This started happening to me after .726 and has not been different since. 

 

I do believe this might has to do with Google limiting API request but also think it is something they changed in the software that is triggering it unless Google just made a big change to how they handle requests.

[Christopher (Drashna)]

Hopefully Google doesn't take exception to that.

 

Hopefully.

 

But we're not round-robining them API per request.  The re-authorization process picks one "at random", and the drive uses that key for the entire time it's mounted. 

 

If there is an issue with this behavior, we'll deal with at that time (or before, because yeah, I'm not a fan of it either)

 

 

 

 

They came back to me this morning saying it might be Google drive who is limiting the API request frequent. However, I disagree as I have set the thread number to 1 on both upload and download and the speed drop is still significant comparing between fresh reboot (50-100mbps per thread) vs after 2-3 hours of run time (0-15 mbps per thread).

 

 They have advised me to tune up the chunk size as well which I created a new drive to test the theory, but still having the same issue over time

 

Will keep you guys as we go along. 

 

Checking the text logs can verify that.  If you're seeing a lot of throttling requests, then that's what going on.

 

Otherwise, logging the traffic with FiddlerCap (http://www.telerik.com/fiddler/fiddlercap), usin the "decrypt https traffic" option may help us to identify the issue.

[john doe]

removed

[steffenmand]

It's the http response time that seems to increase over time - also happens here and nothing in the logs. A fresh reboot makes it all fine again! Takes some days for it to get really bad here though

[thnz]

Perhaps the exponential backoff isn't resetting itself or something.

[Christopher (Drashna)]

Hmmm,  on an offchance, have you ran the network auto-tuning command? 

 

Specifically, try running the following from an administrative command line, and reboot.

 

netsh int tcp set global autotuninglevel=highlyrestricted

[john doe]

removed

[Gbyrd]

I have not been able to get Google drive working well. The download is always limited to about 3-4 Mbps. Before it use to be much higher, seems like the more people using this app the performance takes a heavy hit because of it. Anyway to get it resolved?

[Christopher (Drashna)]

I have not been able to get Google drive working well. The download is always limited to about 3-4 Mbps. Before it use to be much higher, seems like the more people using this app the performance takes a heavy hit because of it. Anyway to get it resolved?

The latest versions ( 1.0.0.770+ ) use a pool of API keys.  If you've installed that version (ideally the 1.0.0.777) build and have re-authorized the drive, it should be using a random key.  This should significantly reduce the "rate limit exceeded" issue, if not eliminate it entirely (a main cause of the performance issues). 

 

Additionally, try re-attaching the drive, and increase the "minimum download size". 

Otherwise, try creating a new driver and use larger chunk sizes, as well. 

[john doe]

removed

[wid_sbdp]

I don't think it's CD specifically, I think it's the interaction with ACD and Google. I made a new Azure account (with the free $200 in credit you get when you make one) and built a storage instance, added that to CD and uploaded some stuff. I was getting very fast 700mbit transfers to it. I think ACD and GCD are just not "as open" as the respective enterprise services are in terms of speed and capability (number of connections, etc). I mean it would make sense to limit those on Amazon's and Google's side as they are cheaper alternatives to paying for an enterprise-level tier.

 

Which, is fine by me. Stability + 150mbit uploads to ACD for $5/mo beats out Stability + 700mbit uploads for $200-300+ (and into the thousands and tens of thousands if you're crazy and have 200TB on your drive).

[Gbyrd]

The latest versions ( 1.0.0.770+ ) use a pool of API keys.  If you've installed that version (ideally the 1.0.0.777) build and have re-authorized the drive, it should be using a random key.  This should significantly reduce the "rate limit exceeded" issue, if not eliminate it entirely (a main cause of the performance issues). 

 

Additionally, try re-attaching the drive, and increase the "minimum download size". 

Otherwise, try creating a new driver and use larger chunk sizes, as well. 

 

After reattaching it works perfectly again. I didn't change the minimum download size however its still set to off.  After trying it again it seems to be downloading at a lower 15 mbps, ive seen it reach up to 50mbps. My line is 1gbps down. After the prefetch it works well, but the initial part is a bit tough.

[raidz]

This issue is pretty much making cd unusable after a few hours for me at this point. Restarting every few hours is a bad solution.

 

Any progress being made towards a fix on this Christopher?