Extremely poor download performance with Google Drive

[CalvinNL]

Hi there,

 

I have been using StableBit CloudDrive for around a month on a dedicated server.

The upload speed is always between 0 and 200 Mbit/s per drive with an average of 150 Mbit/s.

Since build 1.0.0.753 BETA the download speed maxes out on 10 Mbit/s which results in stuttering videos with PLEX and extremely slow copying.

 

The number of threads doesn't seem to matter, I tried 2, but I also tried 5, 10 and 20. 

I also tried another Google account, but that also doesn't help.

 

I tried both a drive block size of 10 and 20MB, but it really seems to be a bug with the software, because changing settings doesn't help a bit, the download speed does not improve.

 

- Calvin

[Kadmium]

.

[andy012345]

While it's writing in 10 or 20mb chunks the service is currently bugged and only reading in 1mb chunks. This causes a bunch of issues from a large number of API hits and slow download speeds as a large proportion of the time is making the connection and requesting the data (I have a 1gbps line server currently reading google drive at 1.68mbps).

 

This really needs fixed. I would also like to see a prefetch option for fast lines where the prefetch brings the whole chunk into cache regardless of only part of it being needed.

 

(as I wrote this post the drive errored out and dismounted )

[steffenmand]

While it's writing in 10 or 20mb chunks the service is currently bugged and only reading in 1mb chunks. This causes a bunch of issues from a large number of API hits and slow download speeds as a large proportion of the time is making the connection and requesting the data (I have a 1gbps line server currently reading google drive at 1.68mbps).

 

This really needs fixed. I would also like to see a prefetch option for fast lines where the prefetch brings the whole chunk into cache regardless of only part of it being needed.

 

(as I wrote this post the drive errored out and dismounted )

It is not bugged. It is because you made the drive with an old version of CloudDrive. Unfortunately the increase prefetch amount was only added in later versions, so the drives need to be remade to utilize this feature. I've read 20 MB chunks with prefetching with no issue at all since version .463 or something close. Recreate the drive using the newest version found here:

http://dl.covecube.com/CloudDriveWindows/beta/download/

[andy0123456]

Me again (can't remember password for other account, is saved on other PC).

 

Here is a drive created on 763 and being used on 763 setup with 20mb chunks. Note all reads at 1mb and all writes are 20mb.

 

http://i.imgur.com/XbCktRC.png

 

And here is the current throughput on the UI

 

http://i.imgur.com/ruobEkh.png

 

463 was actually better because the reads were in 2mb chunks.

[steffenmand]

Me again (can't remember password for other account, is saved on other PC).

 

Here is a drive created on 763 and being used on 763 setup with 20mb chunks. Note all reads at 1mb and all writes are 20mb.

 

http://i.imgur.com/XbCktRC.png

 

And here is the current throughput on the UI

 

http://i.imgur.com/ruobEkh.png

 

463 was actually better because the 

i think you forgot to change the minimal download while creating /attaching. You cant change it before the stored chunk size is increased. you most likely left it on 1 MB by mistake as it is the default. Its one of the last calues you can change in the bottom - but remember to change it last :-) The minimal download works just fine, you just set it up wrong

[andy0123456]

Edit: you are correct, "minimal download" appears to just be "download", there's nothing minimal about it, it will never download anything larger. Also with this setting on "off" all reads are 1mb.

[andy012345]

In this case I would recommend as a workaround that min download be set to chunk size until the 1mb reads are fixed to reduce API hits and throttling.

[nightwing]

is it possible to change this setting after the drive has been setup or do I need to lose all my data to change how much the app will download?

[steffenmand]

is it possible to change this setting after the drive has been setup or do I need to lose all my data to change how much the app will download?

 

It is only possible on drives made after it was implemented. If the was made after it was implemented, then you can simply detach and attach again to be able to set the settings

[Christopher (Drashna)]

The 770 build introduces a pool of API keys for us to use, to help prevent hitting the "rate limits" for the account. This should help with performance on Google Drive.

 

http://dl.covecube.com/CloudDriveWindows/beta/download/

 

 

You'll need to reauthorize the drives, which can be done from "Disk Options" to take advantage of the pool of API keys. 

[Shiz]

I have a Google Drive drive I created on .777.  I copied a 26gb, 30mbps movie over to it to test the ability to stream it and find the best prefetching settings.  Everything worked great throughout all my testing.  Then I moved roughly 10gb of mp3s over.  That process has never completed.  It got stuck at about 8gb to upload and was showing 0 threads/bps with some user rate limit exceeded errors in the logs.  I re-authorized and re-booted and it made it about another 2gb. Rebooted last night before going to bed. It didn't make any headway.  It's stuck at 6.61gb to upload, 0 threads/bps.  2 user rate limit exceeded errors in the logs.  This is not usable this way.  Is there anything I can do?

[Christopher (Drashna)]

I have a Google Drive drive I created on .777.  I copied a 26gb, 30mbps movie over to it to test the ability to stream it and find the best prefetching settings.  Everything worked great throughout all my testing.  Then I moved roughly 10gb of mp3s over.  That process has never completed.  It got stuck at about 8gb to upload and was showing 0 threads/bps with some user rate limit exceeded errors in the logs.  I re-authorized and re-booted and it made it about another 2gb. Rebooted last night before going to bed. It didn't make any headway.  It's stuck at 6.61gb to upload, 0 threads/bps.  2 user rate limit exceeded errors in the logs.  This is not usable this way.  Is there anything I can do?

 

 

That's ... very odd.

 

Could you grab logs form the system?

 

http://wiki.covecube.com/StableBit_CloudDrive_Drive_Tracing

And

http://wiki.covecube.com/StableBit_CloudDrive_Web_Logs

 

And open a ticket for this at https://stablebit.com/Contact ?

[Shiz]

Opened the ticket, I hope I was able to get what you needed.  Tough to get things going around a reboot.  Also, in the contact form when you multi-select files to attach, it attaches all the files but only lists the first one in the UI.. Explains why I have two copies of one file attached to that ticket.

 

Thanks for your help.

[Christopher (Drashna)]

Thank you for opening the ticket! 

 

It looks like this may have been an issue with the Chunk ID system, and would have affected larger drives dispropotionately. 

 

Please grab the latest build, and see if that helps: 

http://dl.covecube.com/CloudDriveWindows/beta/download/StableBit.CloudDrive_1.0.0.782_x64_BETA.exe

[bnbg]

I have a Google Drive drive I created on .777.  I copied a 26gb, 30mbps movie over to it to test the ability to stream it and find the best prefetching settings.  Everything worked great throughout all my testing.  Then I moved roughly 10gb of mp3s over.  That process has never completed.  It got stuck at about 8gb to upload and was showing 0 threads/bps with some user rate limit exceeded errors in the logs.  I re-authorized and re-booted and it made it about another 2gb. Rebooted last night before going to bed. It didn't make any headway.  It's stuck at 6.61gb to upload, 0 threads/bps.  2 user rate limit exceeded errors in the logs.  This is not usable this way.  Is there anything I can do?

 

Hi Shiz, would you be able to share your prefetching settings please? I am currently having great trouble with high bitrate movies!

[loganwolf]

I believe I'm experiencing this very same issue.  Drive was created on 802.

[Dimtar]

Just wanted to chime in and say I feel I am having the same issue. 

On my home connection I can saturate upload speed 40mbps, but download is 4-2mbps. The same can be said of a Windows VPS with Vultr. Uploads of 100mbps and downoad of 2-4mbps at best. Both drives created on .802 in ReFS format.