Jump to content

MandalorePatriot

Members
  • Posts

    8
  • Joined

  • Last visited

Reputation Activity

  1. Thanks
    MandalorePatriot reacted to srcrist in Warning from GDrive (Plex)   
    To my knowledge, Google does not throttle bandwidth at all, no. But they do have the upload limit of 750GB/day, which means that a large number of upload threads is relatively pointless if you're constantly uploading large amounts of data. It's pretty easy to hit 75mbps or so with only 2 or 3 upload threads, and anything more than that will exceed Google's upload limit anyway. If you *know* that you're uploading less than 750GB that day anyway, though, you could theoretically get several hundred mbps performance out of 10 threads. So it's sort of situational.
    Many of us do use servers with 1gbps synchronous pipes, in any case, so there is a performance benefit to more threads...at least in the short term. 
    But, ultimately, I'm mostly just interested in understanding the technical details from Christopher so that I can experiment and tweak. I just feel like I have a fundamental misunderstanding of how the API limits work. 
  2. Thanks
    MandalorePatriot reacted to Christopher (Drashna) in Warning from GDrive (Plex)   
    I'm not sure?  But the number of threads is set by our program. Mostly, it's just the number of open/active connections. 
    Also, given how uploading is handled, the upload threshold may help prevent this from being an issue. But you can reduce the upload threads, if you want. 
    Parallel connections.  For stuff like prefetching, it makes a different.  Or if you have a lot of random access on the drives...
    But otherwise, they do have the daily upload limit, and they will throttle for other reasons (eg, DOS/DDoS protection)
  3. Like
    MandalorePatriot got a reaction from Christopher (Drashna) in CloudDrive Cache - SSD vs HDD   
    For a homelab use, I can't really see reading and writing affecting the SSDs that much. I have an SSD that is being used for firewall/IPS logging and it's been in use every day for the past few years. No SMART errors and expected life is still at 99%. I can't really see more usage in a homelab than that.
    In an enterprise environment, sure, lots of big databases and constant access/changes/etc.
    I have a spare 500GB SSD I will be using for the CloudDrive and downloader cache. Thanks for the responses again everyone! -MandalorePatriot
  4. Thanks
    MandalorePatriot reacted to srcrist in Cache Drive, and the limits of Transferable data?   
    It won't really limit your ability to upload larger amounts of data, it just throttles writes to the drive when the cache drive fills up. So if you have 150GB of local disk space on the cache drive, but you copy 200GB of data to it, the first roughly 145GB of data will copy at essentially full speed, as if you're just copying from one local drive to another, and then it will throttle the drive writes so that the last 55GB of data will slowly copy to the CloudDrive drive as chunks are uploaded from your local cache to the cloud provider. 
    Long story short: it isn't a problem unless high speeds are a concern. As long as you're fine copying data at roughly the speed of your upload, it will work fine no matter how much data you're writing to the CloudDrive drive. 
  5. Like
    MandalorePatriot reacted to srcrist in Warning from GDrive (Plex)   
    That's just a warning. You thread count is a bit too high, and you're probably getting throttled. Google only allows around 15 simultaneous threads at a time. Try dropping your upload threads to 5 and keeping your download threads where they are. That warning will probably go away.
    Ultimately, though, even temporary network hiccups can occasionally cause those warnings. So it might also be nothing. It's only something to worry about if it happens regularly and frequently. 
  6. Thanks
    MandalorePatriot reacted to Christopher (Drashna) in Warning from GDrive (Plex)   
    Thread count is fine. We really haven't seen issues with 10.  
    However, the settings you have set WILL cause bottlenecking and issues. 
    Download threads: 10
    Upload threads: 10
    Minimum download size: 20MB
    Prefetch trigger: 5MB
    Prefetch forward: 150 MB
    Prefetch time windows: 30 seconds
     
    The Prefetch forward should be roughly 75% of download threads x minimum download size.   If you can set a higher minimum size, then you can increase the forward.  
     
  7. Like
    MandalorePatriot reacted to srcrist in Warning from GDrive (Plex)   
    Out of curiosity, does Google set different limits for the upload and download threads in the API? I've always assumed that since I see throttling around 12-15 threads in one direction, that the total number of threads in both directions needed to be less than that. Are you saying it should be fine with 10 in each direction even though 20 in one direction would get throttled?
  8. Thanks
    MandalorePatriot reacted to Christopher (Drashna) in CloudDrive Cache - SSD vs HDD   
    That depends ENTIRELY on your use case.  It's not a question that others can really answer. 
     
    But if performance is important, then the SSD is going to be the better choice for you. 
    But if you're accessing a lot of data (reading and writing), then a hard drive may be a better option.
  9. Like
    MandalorePatriot got a reaction from srcrist in CloudDrive Cache - SSD vs HDD   
    Thank you, I really appreciate your quick and informative answer!
  10. Thanks
    MandalorePatriot reacted to srcrist in CloudDrive Cache - SSD vs HDD   
    SSD. Disk usage for the cache, particularly with a larger drive, can be heavy. I always suggest an SSD cache drive. You'll definitely notice a significant impact. Aside from upload space, most drives don't need or generally benefit from a cache larger than 50-100GB or so. You'll definitely get diminishing returns with anything larger than that. So speed is far more important than size. 
×
×
  • Create New...