pedges
-
Posts
13 -
Joined
-
Last visited
Posts posted by pedges
-
-
23 hours ago, steffenmand said:
The more threads, the more it downloads at the same time, thus faster speed. However of course you should not do more threads than your CPU can handle - as well there can be an upper limit from the cloud provider for max api calls.
I use 14 threads
Is there an obvious way to tell how many threads my CPU can handle? I'm up to 10 threads and 100MB minimum download and it's still taking forever to download even smaller amounts of data.
This is the server I use: https://www.ebay.com/itm/Dell-Precision-T5600-16-Core-2-20GHz-E5-2660-16GB-RAM-2x-500GB-HDD/182388025949?hash=item2a772c6a5d:g:B4YAAOSw3Z9asP2S
-
5 minutes ago, steffenmand said:
Did you do big chunks when creating the drive ?
Also remember to increase the thread count in I/O operations.
I increased download threads to 4 (it was at 2) and minimum download size to 40MB.
Should I increase those two even further, or how do I know the limit to which I can increase them to?
-
1 minute ago, steffenmand said:
Did you do big chunks when creating the drive ?
Also remember to increase the thread count in I/O operations.
I think I set it to the largest chunk size when I created the drive. And I'll try increase the thread count and see if that has any impact at all.
-
I have about 5TB worth of data in a CloudDrive (via Google Drive) that I'm trying to copy to a physical hard drive and it is taking forever to do so.
I have a gigabit internet connection, so I don't think that is affecting CloudDrive's ability to download data to be able to move it.
Is there a faster way to do this that I'm not aware of? So far it's been running ~12 hours and has only been able to move ~30GB to the physical hard drive.
-
1 minute ago, srcrist said:
Now that's an interesting possibility. Maybe? Sure. Maybe. You'd want to detach the CloudDrive first, probably. It might be worth a shot. The Google outage was March 13th, so a date before that would be your best shot.
I have two Cloud Drives saved to that Google account, and one is working. Is there any way to differentiate between the two drives and restore only data for the drive that is experiencing issues?
-
2 minutes ago, srcrist said:
Yes...with the caveat that it didn't prevent the google corruption even on multiple accounts that happened last month. The problem appears to be that Google rolled back data to an older version of some of the files. This is obviously fine for the actual file data itself, since that doesn't really change. But the chunks containing the filesystem data DO change. Often. So everybody's file systems were corrupted. If you mirror the pool to another pool that is on another account, and google has a similar issue, both pools will both be being modified basically simultaneously, and both pools would be corrupted if they did another rollback. It would actually be better to mirror it to an entirely different provider, or to mirror it locally.
Got it.
Just a thought - I have the ability to restore the data in my Google Drive account to a certain point in time. Could I use that functionality to restore the data on my drive to point where it can be used?
-
Just now, srcrist said:
You can just go ahead and use recuva. It's going to scan the drive sector by sector, so it doesn't matter if the file system is screwed.
Interesting. When I tried to select the drive using Recuva, it told me I needed to format it. After formatting it, Recuva crashes when I try to run it.
-
On 3/16/2019 at 2:43 AM, srcrist said:
You pool each set of volumes together to create a pooled volume, and then you add the pooled volumes to a pool themselves and enable duplication on that pool. That will mirror each set of volumes with each other.
Would doing this ensure that all data on the drives is backed up?
-
51 minutes ago, srcrist said:
I'm afraid I don't have good news for you...
I did all of the research I could, and, as far as I could tell, that just means the drive is borked. That error would usually indicate a failing hard drive, but that's obviously not the case here. It's just unrecoverable corruption.
The data on the drive is probably recoverable with recuva. I could recover mine that way, at least. Ultimately, though, I didn't have anything irreplaceable on my drive, so I just opted to wipe it and start over rather than go through and rename everything. Any files recovered will have arbitrary names. That data on the drive should be fine, though, even though the file system is trashed--if you have anything important.
Do I format it first, and then move forward with Recuva, or use Recuva while it is still formatted as raw?
-
8 hours ago, jonesc said:
Chris has suggested this in the past:
1. Take the drive offline in disk management
2. Turn off all pinning in the CloudDrive UI
3. Clear the local cache, and wait until it's down to 0 bytes (literally empty)
4. Bring the cloud drive back online from disk management
You may need to repeat step #3 a couple of times to get this to work.
Specifically, this clears out the cache and forces the software to redownload the data from the cloud, including the file system data.
Setting the drive as offline ensures that drive data is not updated, and that any open files are forcibly closed (eg, without saving).
If that doesn't help, then CHKDSK or data recovery may be needed.Okay. I've tried Chris's suggestion and that didn't work. I also ran chkdsk and am receiving the error "Insufficient disk space to insert the index entry."
This comes up after a few lines of saying "Inserting an index entry into index $0 of file 19."
I should note that the drive I'm trying this on is a 10TB drive and has ~5TB of free space (less than half full).
-
I have two drives set up to work with CloudDrive and Google Drive that store some media.
I'm not sure what happened, but when I checked them this morning (they were fine last night), Windows told me I need to format them before I can use them (in disk management they show as raw disks). I'm not sure if Windows updated overnight and something crashed, but I'm now afraid that if I format them, I'll lose all the data I have on them and have to redownload all of it from Google Drive (it's a couple TB worth of data).
Is there any way to format these without losing the data?
Up until this morning, this setup has been working flawlessly for me for months, so I'm disappointed that this is happening to me now.
-
Edit: I let it run overnight and now it is up and running! Sorry for the false alarm.
I'm running Cloud Drive on Windows 10 and after an unexpected shutdown (lost power), I can't get the drives back up and running. It just pops up and says "Starting Services" with an increasing amount of time.
I've tried uninstalling and reinstalling, and the only UI I get is that window. I don't have an option to reset or anything.
Am I going to lose all the data on my drives, and how can I prevent this in the future?
Thank you.
Is there any way to speed up downloads from my CloudDrive?
in General
Posted
For what it's worth, I've been using CloudDrive to store my Plex media for the past year and the default settings have always worked well for me.
The reason I posted this question is because I'd like to move all my data out of and away from CloudDrive. After the GDrive corrupt data issue a few weeks ago, I lost a ton of media on one of my drives and no longer trust it being in the cloud, so I'm trying to do a bulk download to physical hard drives.
The default settings do not seem catered to performing bulk downloads, which is why we've been messing with all of the I/O options to figure out what would.
From what I've found so far, the fastest combination of settings seems to be 1MB trigger/20MB forward/30 second time window for the prefetch settings, with 14 download threads and a 20MB minimum download size. I'm fairly certain I maxed out the chunk size when creating the drive, but I really can't remember.
It's still not blazing fast, but it is much faster than the default settings I was using just two days ago.