How CloudDrive handles "User rate limit exceeded"

[jsbzoh6]

I have a very large library I'm trying to upload to Google Drive (~100TB worth). I obviously hit the 750GB limit pretty quickly. My issue is how CloudDrive handles this limit once it happens.. which is not handling it at all. It will continue to bombard Google with ~500mbps uploads until the limit lifts and it can successfully upload again. The pause button doesn't seem to work so I'm finding myself either drastically lowering the throttle or killing the service until later in the day (if I remember to turn it back on). Is there a setting or a way for CloudDrive to keep uploading until it hits the limit, then pause or stop for a certain amount of time? I know it isn't smart enough to know when the limit is lifted, but even a periodic "test" file is better than hammering Google (and my network) with near gigabit speeds into a brick wall.

[Christopher (Drashna)]

For any throttling request (like this), we use exponential backoff, so it may hammer at first, but after a while, it should quiet down. 

That said, limiting the upload bandwidth to 75mbps should avoid hitting this at all. 

[jsbzoh6]

Is there a hardcoded time for it to back off? I've unpaused and started monitoring since I read your reply and it's still going 100% (I have it throttled to 100mbps) on both the CloudDrive UI as well as on my actual network traffic logs.

[srcrist]

The exponential backoff is hardcoded. If you check your logs you can see how long it pauses every time it receives a throttling request. It will continue trying periodically, but that period gets longer and longer while you're at the threshold. But, like Christopher said, dropping your upload to 75mbps from 100mbps will prevent you from hitting the 750gb/day limit altogether. You're only getting those messages because you're allowing it to exceed the limit for no reason. 

In any case, there isn't a lot of harm in letting it keep trying. CloudDrive will not exceed Google's limits and will not result in an API ban no matter what. 

[jsbzoh6]

My concern is less Googles end and more my network end with a constant 100mbps (or 75mbps after I've throttled it down) stream going out. I just looked through today's logs and it looks like the exponential backoff may not be working/configured properly. I'm showing identical log entries every second which coincide with my traffic logs.

[srcrist]

You'll likely need to adjust the filtering level to see the exponential back-off messages. Change your ApiHttp tracing level to information and you'll see messages like this "3:10:38.5: Information: 0 : [ApiHttp:98] Server is throttling us, waiting 1,928ms and retrying." But there is no way that the exponential backoffs aren't working for you. If they stopped working, you'd quickly hit google's API "ban" and no longer be able to submit API requests for 24 hours. Even if they *weren't* working, though, it would not be a constant 100mbps stream or 75mbps. Google rejects the API requests once you hit your threshold, so you can't actually transfer any data no matter what CloudDrive wants to do. Aside from the data required to make the API request, which is minimal, there would be no data transferred. It just isn't a technical possibility. 

[Christopher (Drashna)]

21 hours ago, jsbzoh6 said:

. I'm showing identical log entries every second which coincide with my traffic logs.

This is handled per thread, actually.  So yeah, you may see a lot of these before it starts increasing. Especially if you have a high thread count.