Rate Limit Exceeded

[IanN]

I keep getting "Error: Rate Limit Exceeded", just wondering what it means and if I need to do something to reduce the amount of errors I get?

 

[steffenmand]

I think it means you are reaching the max api calls per second and that the thread got cancelled. it will just be retried again

[Christopher (Drashna)]

I think it means you are reaching the max api calls per second and that the thread got cancelled. it will just be retried again

This. 

 

 

I keep getting "Error: Rate Limit Exceeded", just wondering what it means and if I need to do something to reduce the amount of errors I get?

 

What version of Stablebit CloudDrive are you using? 

What provider is this with? 

Have you made any changes (such as to the number of threads, prefetching option, etc)?

[IanN]

Thanks for the replies!

 

Stablebit version is 682

GDrive - 2 Drives (being written to most of the time)

Download/Upload threads = 3

Prefetch trigger = 1mb

Prefetch forward = 100mb

Prefetch time window = 3600 seconds

[Christopher (Drashna)]

Okay, those should all be "fine", then. 

 

Was the drive created with an older version of StableBit CloudDrive (such as 1.0.0.463)? Or otherwise using larger chunk size (eg 100MB chunk, for instance)? 

 

In fact, getting the drive geometry may help here. Mouse over the size in the  UI and it will list the chunk size and the validation size. 

Post that here.  

[IanN]

Hi,

 

Apologies for the delay. The issue is getting worse - I leave it overnight and it hits about 32k error messages..

 

I can't remember which version I created it with - is there anyway to tell?

 

Chunk size: 10mb

Validation 1mb

 

 

[Christopher (Drashna)]

Hi,

 

Apologies for the delay. The issue is getting worse - I leave it overnight and it hits about 32k error messages..

 

I can't remember which version I created it with - is there anyway to tell?

 

Chunk size: 10mb

Validation 1mb

 

 

Could you grab logs of this happening? 

 

http://wiki.covecube.com/StableBit_CloudDrive_Drive_Tracing

[smdotio]

I am experiencing the same issue - a massive load of 'rate limit exceeded' - logs uploaded today.

 

Version: 722

Download/Upload threads = 10

Prefetch trigger = 1mb

Prefetch forward = 40mb

Prefetch time window = 3600 seconds

Chunk size: 10mb

Validation 1mb

[Christopher (Drashna)]

For Google Drive, this may be "our end",  There are limits on our end, and it sounds like this is what is being hit.  

 

Alex is aware of this, and should be increasing this (if he hasn't already).  

 

If it continues to happen on Google Drive, let me know. 

If it's happening on other providers, also, let me know.

[IanN]

Thanks for the reply.

 

Where did you want me to upload the logs? Or are you sure its your end now? It happens permanently on Google Drive.. Is there anyway to stop it? What is the "rate"? Is it speed or data?

 

Thanks

[Christopher (Drashna)]

Sorry, any logs like this should be grabbed and uploaded via this link:

http://wiki.covecube.com/StableBit_CloudDrive_Drive_Tracing

[anonymouspi]

I'm having the same issue several thousand rate limit exceeded Google drive errors. Have limited my upload and download threads to a single thread, but the issue is still occurring. I will post Drive tracing logs in a minute.

[IanN]

Won't even let me reconnect the drive now, states Rate Limit Exceeded....

[KiLLeRRaT]

I'm also getting a lot of Rate Limited Exceeded exceptions... Google Drive provider.

[triadcool]

^ Same here.  It it occurs no matter how many threads I have set.

[KiLLeRRaT]

^ Same here.  It it occurs no matter how many threads I have set.

 

I only have 1 up and 1 down thread set too.

[IanN]

Pretty unusable in its current state - does anyone know if it is only Google Drive that has the issue? Is it worth trying Amazon or is this a bug for all providers?

[smdotio]

From Christopher's reply, quoted below. I think the rate limits we're hitting is the application as a whole reaching it's google dev cap.

 

It's not us - they need to increase it on their end.

 

For Google Drive, this may be "our end",  There are limits on our end, and it sounds like this is what is being hit.  

 

Alex is aware of this, and should be increasing this (if he hasn't already).  

 

If it continues to happen on Google Drive, let me know. 

If it's happening on other providers, also, let me know.

[Brian]

Hello I can not attach to Google Drive at all at this time using 463 because I get Rate Limit exceeded errors. :-( 

 

Is there a fix coming?

[triadcool]

Hopefully a fix is being worked on right now.  I'm on version .719 and my drive keeps dismounting because of this.

[smdotio]

I can't even mount my drives now either.

[Utnes]

Same issue for me, can't mount drives, and If I'm lucky it's a real struggle to upload anything.

[steffenmand]

Took me 5 reboots and 3 hours to mount my drives When i finally got them mounted i had 83 red errors in the top)

 

Even after they are mounted i'm getting Rate Limit Exceeded like every 20 sec

[Christopher (Drashna)]

I'm sorry that you guys have been have issues. 

 

I've reproduce the issue ... accidently (creating a new drive), and am getting the errors right away (not show stopping, but still). 

 

Alex is aware of the issue, and should be looking into the issue tonight or tomorrow. 

[smdotio]

Thanks for the update Christopher.

 

I managed to re-attach my drives, but still many many rate limit exceeded warnings.

[Christopher (Drashna)]

Since I am probably going to post this a lot: 

http://community.covecube.com/index.php?/topic/2228-google-drive-rate-limits-and-threadingexp-backoff/

 

Basically, download 1.0.0.725 for now. It should fix/help the issue. 

 

But we either need to overhaul the provider, or appeal to google for a rate limit increase.

There is a "hard" limit to the number of APIs made "per app", and we're hitting that. Either that limit needs to be increased, we need to optimize the code to reduce the number of API calls, or both.