General FAQ?

[BuddyRich]

I currently am a user of drivepool and am now looking at cloud storage as a backup local.

 

Right now i use snapraid to make a parity drive for my pool. i do not use drivepools duplication.

 

If i also wanted to backup my pool to cloud drive, is there any integration to do so via drivepool? if not, since cloud drive acts as a virtual drive, could i use any software that syncs changes between drives to back up everything initially and then deltas of changed items?

 

In addition to that if i refoldered something does cloudrive bring it down and reup it, or can it do those operations online?

[17thspartan]

Why not add the CloudDrive to Drivepool, and then use Drivepool's built in duplication to make your CloudDrive a mirror for the rest of the pool? Just make sure to turn off anything in regards to file/drive balancing, and limiting drive usage in order to avoid constant uploading.

 

http://community.covecube.com/index.php?/topic/1226-how-can-i-use-stablebit-drivepool-with-stablebit-clouddrive/

 

As for the refoldering, I'm not sure how it works with Drivepool and CloudDrive together (I didn't use drivepool long enough to find out), but if you just mean renaming a folder, or cutting and pasting, then it doesn't need to download and re-upload. I had to do that earlier today, cut and pasted 40gb of files from one folder to the other in my CloudDrive drive and had no issues. Copy and paste operations on the other hand will induce a download and re-upload.

[BuddyRich]

Thanks for the link, that was my question more or less.

 

I knew it acted as virtual drive, but I didn't know if it was pool eligible, etc.  So turning on file duplication and setting the cloud drive as the only eligible target for duplicated files, will force files to essentially sync, and then only new stuff that is added after that, giving me deltas...

 

How kludgey is Amazon Cloud Drive support?  It will probably be my goto as I will be duplicating an 18TB pool, which may or may not take an entire month to upload to Amazon.  Would Google Drive be better in that regard, its the only other unlimited alternative thats reasonably priced.

 

My final question, since it emulates a drive and encypts stuff with AES256, essentialy writing slabs in 100MB, how easy is it to recover if things go south?  Obviously logging into Amazon Cloud I would just see a bunch of 100MB files, could I somehow recombine them locally if downloaded manually, say if CloudDrive's API access got cut off?

[17thspartan]

Google would be the better choice. Due to difficulties with folks at Amazon, Stablebit CloudDrive is currently limited to 20mbps upload speed to Amazon, and that's part of why it's labeled as an experimental provider. And Amazon tends to throttle a bit more aggressively if you use too many upload or download threads. As far as I know, Google has no such limit when it comes to speeds, but they might have a limit on how many up/down threads you can add.

 

As far as downloading all your clouddrive files, and using them locally; this is something I've been wondering myself lately after reading this thread, which follows a similar line of thought (transferring from cloud to cloud, instead of downloading to local hard drives).

 

http://community.covecube.com/index.php?/topic/2472-backup-clone-of-a-google-drive-account/

 

I created a CloudDrive on my local hard drive to see if there were any differences between CloudDrive on the cloud (like on Google servers) vs the ones on local hard drives. It does appear that they are structured differently, so I'm not sure if they're compatible.

 

If it doesn't work, it's certainly something that should be requested as a feature, because that's something I'd be interested in having as well.

[BuddyRich]

So Ive set this up...  right now for testing purposes.  My connection is only 100/10 so speed is not paramount.

 

I got a trial Amazon Cloud Drive account.

 

Used .631 to setup the drive since ACD is no longer supported.

 

Upgraded to .802 and they added the drive to my drivepool.

 

Used disk limiter balancer in drivepool and set my CD to duplicate only and set my other drives to unduplicated only.

 

 

I think its all OK.

 

If I access something in that duplicated folder from my pool drive, which version will get used?  The original (local copy) or the duplicated version on the cloud drive?  Could it be either or will it always use the original?  Id rather not download (stream) from the cloud drive if I can help it.

[17thspartan]

I'm actually not sure which it would try to read files from. Hopefully someone else can answer your question, because when I used Drivepool, I used it to make a pool of all my CloudDrives. And you don't need to worry about using 631 anymore, .802 is able to create drives on Amazon as well. So you can just use the latest version of CloudDrive and it should work just fine with Amazon. You just need to be sure to go into the settings and enable experimental providers.

 

One thing to note about using Amazon is that there's an issue where the chunks being uploaded to Amazon get retried over and over, sometimes leading one chunk to be uploaded 4 times before it works. On the one hand, it's better to have a chunk get retried over and over until it actually makes it into the cloud (last thing you want is for your data not to be there when you want it), on the other hand it can use a lot of bandwidth and time. For example, I tried uploading 2gb a while back and it ended up using about 4-5gb worth of bandwidth to get it to the Amazon cloud. I still use Amazon from time to time with Stablebit CloudDrive, so it's not a dealbreaker, just thought you should be aware of this.

[BuddyRich]

I didn't realise ACD was again "unofficially" supported.  I thought they removed it unless you had your own API key?

 

I did notice the multiple retries though in my limited testing.

 

What are good settings to use for ACD?  Both in initial drive setup and the IO section?

 

I had used 10MB chunks as it was the default but that seemed small.  My ACD drive will be 19TB total, though currently I'll only duplicate ~9TB of files to it. I still have space on the server.

 

And because of all of those retries, I assume I should keep the verify setting on.  Seems wasteful as it basically downloads it, or does it only download a hash to check?

 

Would there be a benefit to creating a new drive in the newest version?  My settings seem to be limited to 50/10 and 4/2 threads but Ive seen other people say 10/10 is good for ACD...

[17thspartan]

They had removed it, but it seems it's since been fixed.

 

I'm not sure about the best settings for creating a drive, but the only advice I have is limiting the number of download/upload threads. I'm not sure how many threads you can have (10/10 might be fine), but I do know that once it hits the 15-20 range, there will be some severe throttling.

 

I usually leave the upload verification on, but I believe it downloads the chunk to check it against the uploaded one. I don't know if Amazon is any different, but Stablebit CloudDrive isn't able to run any kind of checksum in the cloud for most cloud providers, so the only option to verify that it was properly uploaded is to download the entire chunk and then run a checksum.

 

I don't know of any benefits to making a drive in the newest version. All my drives were made using 802 or newer, but if you want you can always check the changelog to see if there's anything there. http://dl.covecube.com/CloudDriveWindows/beta/download/changes.txt

[Christopher (Drashna)]

Yeah, Amazon Cloud Drive isn't officially supported.  And we didn't really "fix" it, but we did fix a number of performance and other related issues that did significantly improve the stability and performance of Amazon Cloud Drive. 

 

And the ACD provider is limited to 50mbps down, 20mbps up, and 4 download threads and 2 upload threads.   

This is the only provider that is limited, but this is because the provider is stuck on "development" status, and the more people using it, the worse things get, so the limitations where necessary 

[BuddyRich]

My 10 Mbps upload speed is not going to break limits anyway.  However from my testing it didn't seem like it was respecting the limits anyway.  My was set to 4/2 threads but it was going upto 10 on upload sometimes, and I was quite often getting the turtle even though I throttled myself to 8 Mbps up, well low of the 20Mbps max of Amazon.  Maybe I was just misreading what the GUI was telling me though?  Also in the web interface I did have about 2GB of slabs but the graph was showing 0 residing on the cloud and it was all in my cache.  Though with a new disk created in 802 things seem to be better.

 

Can you answer the question about what "copy" of a file will be used if a cloud drive is added to a drivepool as a duplication-only target?  Using Drivepools file protection would be handy to eventually get deltas pushed automatically but I want to ensure that my local applications read the "local/original" copy and not stream it from the cloud drive if at all possible.

[Christopher (Drashna)]

The turtle icon means that you're being throttled.  TOo many requests in a short period could cause this, Or if you're running multiple drives on the same provider, as well. 

 

 

 

Can you answer the question about what "copy" of a file will be used if a cloud drive is added to a drivepool as a duplication-only target?  Using Drivepools file protection would be handy to eventually get deltas pushed automatically but I want to ensure that my local applications read the "local/original" copy and not stream it from the cloud drive if at all possible.

 

I missed that, sorry. 

 

If you se the CloudDrive to be the only eligible drive for duplication, it means that one copy will absolutely end up there. Period.  But it will degrade the pool condition, may make it want to balance data more often (depending on your balancing settings). 

 

As for reading the data, this depends on the Read Striping feature.  it is very likely that it will cause it to read from the local drives rather than the CloudDrive disk.

 

But we do have a feature request to FORCE it to not read from the CloudDrive disks, if possible.  This would likely be exactly what you're after.