Jump to content
Covecube Inc.

thnz

Members
  • Content Count

    139
  • Joined

  • Last visited

  • Days Won

    7

Everything posted by thnz

  1. Will Windows eventually force kill services if they take too long to close gracefully? If so, does the Shutdown Service explicitly avoid this happening?
  2. thnz

    unusual activity

    Plex does 'extensive media analysis' during the maintenance window (I think its like 1am-6am, or thereabouts by default) which involves reading through entire media files. It will be very download heavy if its doing it on remote files. I'm not sure how often it does it per file - I assume its a one off per file, though it might get redone again. Regardless, it can be disabled by unchecking 'Perform extensive media analysis during maintenance' under 'Scheduled Tasks'. There's not much of a downside in having it disabled. Bit more info here, and what benefits it gives here.
  3. It was reported both in CloudDrive and in the system event logs, so I'm now assuming the system was actually crashing while shutting down, and CloudDrive was doing what it was supposed to by recovering. Unfortunately CCleaner has since removed any dump files, so I'm not able to investigate any further at the moment. As tends to happen with these things, I've been unable to reproduce it since. I've excluded system memory dumps from CCleaner in the future, so if/when it happens in the future, I should have more to work with - it could well just be a wobbly driver or something.
  4. Windows 10 x64, and CloudDrive beta 850. Looking over the system event logs, it did mention an unexpected shutdown when It was supposedly shut down safely, so there may well be something else going on preventing a clean shut down. I'll have a closer look later on.
  5. Done. It only took a single clean shutdown to trigger it.
  6. Unfortunately I've been seeing 'unsafe shutdown' messages occasionally again after apparently clean machine restarts.
  7. I see the latest beta version (.818) contains the following change: Hopefully that fixes this issue, and I can now go back to a much larger cache size without the risk of it going into recovery so often after a restart.
  8. thnz

    Web Proxy Issue

    When troubleshooting CloudDrive, I found I had to edit the CloudDrive.Service.exe.config file in order to force a specific proxy.
  9. thnz

    CloudDrive problem

    Alternatively, you can manually increase IoManager_DefaultMaximumReadAggregation in the service config file - by default its set to 1MB. This will combine consecutive 1MB downloads into larger requests (ie. while prefetching), while leaving smaller reads at 1MB. That way you can get the benefit of fewer API calls/larger downloads, without suffering the extra overhead of increasing the minimum download size on smaller. Here's some clarification on what it does: https://stablebit.com/Admin/IssueAnalysis/27309
  10. I've found manually increasing IoManager_DefaultMaximumReadAggregation via the config file and then restarting the service helps work around this. I'm not entirely sure what it does as there's only minimal documentation in the changelog regarding it, so I'm not entirely sure what the full consequences in altering it are. By default its set to 1048576. My observations are that by increasing it to a value >10MB, the pre-fetcher would then download up to the drive's chunk size (10MB) per pre-fetch download request, rather than splitting it into lots of smaller 1MB requests. I guess whether or not this a good thing depends on use case. As a 10MB chunk downloads in less than a second for me (after the delay/throttling response), splitting the 10MB request up into separate threads isn't really necessary, and I'd rather use less API calls.
  11. thnz

    Drive keeps dismounting

    I'm happy with the minimum download size as is and don't think it should be changed. It's perfect for smaller reads. HOWEVER, for larger reads, ie. when the pre-fetcher kicks in, it doesn't increase that download size at all, making (by default) 10x the number of API calls than is needed. A 10MB pre-fetch (with default drive settings) could be one or two API calls rather than 10 as is what currently happens. So its more the way in which the pre-fetcher doesn't increase the download size when contiguous parts of the same chunk are downloaded, rather than the minimum download size itself.
  12. thnz

    Drive keeps dismounting

    Increasing the minimum download sizes improves it, but then the opposite happens and you download much more than is needed for smaller reads. In that case there must be a bug, as this isn't happening with the pre-fetcher - it always breaks it up into MINIMUM_DOWNLOAD_SIZE chunks, even when multiple contiguous blocks are read. 10MB of prefetching will result in 10x1MB downloads, rather than a single 10MB download (or an 8MB + 2MB etc). Fixing it could cut down on API requests by 90% with default chunk sizes.
  13. Any updates on this? Optimizing this could have a significant impact on speeds. If we're going to pre-fetch 50MB it'll be much more efficient to do so using 5x10MB downloads rather than 50x1MB ones - its a potential 90% saving in API requests on drives with default settings.
  14. thnz

    Drive keeps dismounting

    Might want to follow this thread to see if/when its fixed. But yeah, that 3GB download being done in 1MB chunks is crazy inefficient. 3000 I/O requests when 300 would do (assuming the default 10MB chunk size) - thats 90% less API calls. FWIW you only need to reattach a drive to change the minimum download size - you don't need to delete and recreate it.
  15. If you mean transfer threads, rclone transfers 4 files at a time in parallel by default.
  16. Perhaps the exponential backoff isn't resetting itself or something.
  17. Hopefully Google doesn't take exception to that.
  18. Doing that might have unforeseen consequences. Google throttles by application for a reason, and bypassing that could potentially make their service worse for everyone. Amazon made their API invite only shortly after CloudDrive allowed custom profiles (not saying that the two were necessarily related), making it now much harder for other applications to integrate Amazon Drive support as a result.
  19. Not sure if you guys are aware, or if this will even be of benefit, but if you want to further optimize Google Drive, you can batch API requests together - with some limitations: https://developers.google.com/drive/v2/web/batch
  20. thnz

    Excessive memory usage

    Just noticed that the machine running CloudDrive was unresponsive - the CloudDrive service was apparently using upwards of 8GB of memory and it was thrashing the page file. It's running .744 and was uploading at the time, I had previously copied across several GB to the cloud drive, but am unsure if that had finished before the memory explosion. Not sure if any of this will be relevant, but AV is Nod32, and the machine is also running Crashplan for backups. Only a single GoogleDrive is attached.
  21. Is this the local file ID database, or is there something else in the pipeline too?
  22. thnz

    Rate Limit Exceeded

    Amazon made their API invite only shortly after they did this with Amazon Drive.
  23. Isn't there a per-user requests/100s quota you can set? Perhaps you could turn it down in the meantime to keep things functional for as many users as possible - until you can increase the apps total daily limit.
  24. I had mostly been testing with ACD (with a custom dev limited security profile), and have only recently been trying Google Drive instead - speeds are far superior on the latter. Google traffic seems to go through Sydney, so its also a lot closer than Amazon (I'm in NZ - Amazon data goes to the US). Granted its more expensive ($10 vs $5 /mo), but its probably worth it for the speed and reliability - files on Amazon seem to have a very small chance to randomly disappear altogether. NZ is making great progress in net speeds - so long as youre not rural - Gigabit speeds are now available in a lot of places - you can get uncapped/unthrottled/unshaped 1000/500 for $130NZD ($95USD)/mo.
  25. Weird that it was denied - it looks similar to Arq (encrypted backups with deduplication etc), which is even listed in their app gallery page.
×
×
  • Create New...