Jump to content
Google Drive is Discontinued (FAQ added)

Joel Kåberg

Members
 View Profile  See their activity

  • Posts

    15

  • Joined

  • Last visited

  • Days Won

    3

 Content Type 

Posts posted by Joel Kåberg

    Poor performance Google Drive (not rate limit)

    in Providers

    Posted

    I've tried my setup again, this time with ACD which seems to upload at full speed (now 24h later and still uploading at full speed).

     

    Perhaps CD is doing to many requests, which then forces Google Drive to add an delay to each request?

     

    I'd like to note that rclone does not have this issue (I've already uploaded +/- 20TB using rclone to Google Drive and the avarage speed of that upload was 290mbps, which is my max upload).

    Poor performance Google Drive (not rate limit)

    in Providers

    Posted

    The latest versions ( 1.0.0.770+ ) use a pool of API keys.  If you've installed that version (ideally the 1.0.0.777) build and have re-authorized the drive, it should be using a random key.  This should significantly reduce the "rate limit exceeded" issue, if not eliminate it entirely (a main cause of the performance issues). 

     

    Additionally, try re-attaching the drive, and increase the "minimum download size". 

    Otherwise, try creating a new driver and use larger chunk sizes, as well. 

     

    As you know I've already tried all this, and I'm on the very latest BETA build. The issue essentially boils down to the response time which increases over time, this really kills the performance of CD.

    Poor performance Google Drive (not rate limit)

    in Providers

    Posted

    I experience the same issue as well and on the latest version (770).

     

    I can upload between 300-600 mbps on every fresh reboot, and after a while, it drops to 50-150 mbps and later dropped to 0-50 mbps. (On a 1Gbps line)

     

    The weird thing is that on every fresh restart, I can go back to 300-600 mbps for a while until it drops again. From the look of it, it seems more like a CloudDrive issue rather than Google.

     

    Tried changing the thread settings etc and nothing seems to help.

     

    Already reported to support and waiting for a resolution :)

     

    Exactly the same issue for me (just lower bandwidth  :D), I'll see if I can find some logs when I get home (OS logs or CloudDrive?)

    Poor performance Google Drive (not rate limit)

    in Providers

    Posted

    Yes I noticed very bad performance (not even close to the BW usage you see in my video) with the old keys (roughly around 20~60 mbps) so I updated on the 20th to the 770 build. Initial speeds was OK for a few hours (5-6), around 200~280 mbps with 100% I/O usage.

     

    Yesterday speeds went back to the same as with the old api keys (40-80 mbps) and has been steady since then. I/O usage on the HDD is now at 30~50% so I'm sure it isn't that.

     

    If Alex want's I'd let him remote controll the PC doing the upload, if he want's to debug.

    PS: yes, I did re-authorize.

    Poor performance Google Drive (not rate limit)

    in Providers

    Posted

    Just to make my point, here's a video: https://eth0.im/upload/2016-11-17_23-19-10.mp4

     

    At 1 min I pause the transfer (from an SMB share) to the CloudDrive (while CD is still uploading to Google Drive)

    At 2 min I resume the transfer from SMB to CD

    At 3 min I show you system specs

     

    Pay attention to the upload speed. Normally it's steady above 300 mbps (while SMB transfer is paused) and occasionally hits 400 mbps, just had a bad run here.

     

    So while transfering from SMB to CD I see an 30-40% performance loss uploading to Google Drive, and I suspect/guess this is due to CD beeing dependent on the OS drive to split up chunks etc(?). So my suggestion is to offload I/O heavy actions into RAM (atleast make it optional), which should speed things up nicely.

    Poor performance Google Drive (not rate limit)

    in Providers

    Posted

    Not pausing the connection, pausing the transfer to the (local) CloudDrive (while CloudDrive tries to upload)

     

    What happends is I try to copy stuff to the local CloudDrive, CloudDrive then encrypts and uploads to Google Drive. When all this is happeing at the same time I see disk utilization hitting 100% and uploads really seem's to suffer from it as explained above

     

    Seem's the way CloudDrive handles disk IO is really killing it performance wise. I've now moved to rclone while this gets sorted (which caches in ram instead of on disk which i assume CloudDrive is doing as ram is hardly used by CloudDrive)

    Poor performance Google Drive (not rate limit)

    in Providers

    Posted

    I'm seeing poor upload performance with GD, it goes somewhere between 0 to 150 mbps (on average 50-70mbps) but very rarely above that.

    Here's my settings https://goo.gl/iXdlWc | https://goo.gl/H5T8G7

     

    An regular Speedtest https://goo.gl/iRjC8o

     

    Looking over the logs I can't see any Rate limit or similar, just a bunch of Write chunk

     

     

    14:48:50.5: Information: 0 : [GoogleDriveIoProvider:39] [W] Writing chunk 37,743.

    14:48:51.1: Information: 0 : [GoogleDriveIoProvider:29] [W] Writing chunk 37,744.
    14:48:51.9: Information: 0 : [GoogleDriveIoProvider:64] [W] Writing chunk 37,745.
    14:48:53.3: Information: 0 : [GoogleDriveIoProvider:14] [W] Writing chunk 37,746.
    14:48:54.8: Information: 0 : [GoogleDriveIoProvider:45] [W] Writing chunk 37,747.
    14:48:57.1: Information: 0 : [GoogleDriveIoProvider:41] [W] Writing chunk 37,748.

     

    My workflow

    Ubuntu server with samba (main storage for now)->1gbit lan->Windows 10 PC with clouddrive->google drive

     

    So I take it this is something related to my ISP or something between me and Google?

     

    I'm investigateing now if my data gets uploaded to EU or US datacenters (I'm located in the EU) Uploaded to EU/Amsterdam

     

    For comparision I can saturate my bandwith with the same Google Drive using rlcone, so this must be a CloudDrive issue.

     

    One thing that I've noticed is when I pause the transfer to the CloudDrive (and Clouddrive just sits uploading) I'm seeing a lot better bandwith usage (nearly 100%). I've now tried several (SSD) disk on different PC but they all suffer the same fate.

    So I think I got this down to IO issues, it seems better without encryption but I'd love haveing that. Task manager gives a pretty good hint as to the disk beeing excessevly used (I'm assuming due to the cache in CloudDrive). Perhaps this need some attention, also an RAM option would be nice here (for us with plenty of it). I tried creating a RAMdrive to see if that helped (use as a cache drive) but CloudDrive won't recognize it

    Local cache disk filling up despite "Fixed" cache size

    in General

    Posted

    I'm also seeing the exact same issue, pausing the local file transfer to the CloudDrive mount point will resume upload but once you resume the local copy it will pause in CloudDrive

     

    I hope Alex gets around to it soon!

    EDIT: Sorry for the noise. Using the latest beta now (756) with fixed cache size. Seem to do the trick here (you guys should update the frontpage links :-))

    Can I mount a trail drive with an licensed one?

    in General

    Posted

    I'm about to migrate lots of data from an drive to another, and to minimize the wait time I will use an VPS for the initial transfer (better bandwith)

     

    But when I have transfered my data onto the CloudDrive (Google Drive), can I then later mount the same drive using a licensed CloudDrive?

     

    I'm thinking of using a trial on the VPS, and my license at home.

×
×
  • Create New...