[Ridiculous questions on CloudDrive]

12 hours ago, Alex said:

1. By default, no. But you can easily enable checksumming by checking Verify chunk integrity in the Create a New Cloud Drive window (under Advanced Settings).

1a. For unencrypted drives, chunk verification works by applying a checksum at the end of every 1 MB block of data inside every chunk. When that data is downloaded, the checksum is verified to make sure that the data was not corrupted. If the drive is encrypted, an HMAC is used in order to ensure that the data has not been modified maliciously (the exact algorithm is described in the manual here https://stablebit.com/Support/CloudDrive/Manual?Section=Encryption#Chunk Integrity Verification).

1b. No, ECC correction is not supported / applied by StableBit CloudDrive.

2. It's perfectly fine. The chunk data can be safely stored on a StableBit DrivePool pool.

3. Yep, you should absolutely be able to do that, however, it looks like you've found a bug here with your particular setup. But no worries, I'm already working on a fix.

One issue has to do with how cloud drives are unmounted when you reboot or shutdown your computer. Right now, there is no notion of drive unmount priority. But because of the way you set up your cloud drives (cloud drive -> pool -> cloud drive), we need to make sure that the root cloud drive gets unmounted first before unmounting the cloud drive that is part of the pool. So essentially, each cloud drive needs to have an unmount priority, and that priority needs to be based on how those drives are organized in the overall storage hierarchy.

Like I said, I'm already working on a fix and that should be ready within a day or so. It'll take a few days after that to run the driver through the Microsoft approval process. A fix should be ready sometime next week.

Bug report: https://stablebit.com/Admin/IssueAnalysis/27859

Watch the bug report for progress updates and you can always download the latest development builds here:
http://wiki.covecube.com/Downloads
http://dl.covecube.com/CloudDriveWindows/beta/download/

As far as being ridiculous, I don't think so  I think what you want to do should be possible. In one setup, I use a cloud drive to power a Hyper-V VM server, storing the cache on a 500GB SSD and backing that with a 30+ TB hybrid (cloud / local) pool of spinning drives using the Local Disk provider.

Thanks for the answers and support, Alex. I understand now that the integrity verification is for revealing corruption/tampering at the source but correction is a non-goal. 

One more for you then:

1c. If a CloudDrive fails verification in a DrivePool, does DrivePool treat the drive as corrupted and pull data from an available duplicated source?

Your explanation makes sense. I was wondering how nested dependencies were supposed to work during loading. I also saw the unmounting service and I think you answered why it exists.

I have another 3 weeks left in my CloudDrive trial, so hopefully I can try the fix out. My college recently gave all alumni "unlimited" GDrive space and I'd like to use whatever my network can muster. Unfortunately during the whole CloudDrive on DrivePool fiasco I lost a drive to the frequent power cycling and hard shutdowns. It now only initializes properly at an angle (???).

That Hyper-V setup sounds just as ridiculous--it's assuring to know I'm in good company. Thanks again for all your (and Chris's) hard work! DrivePool and Scanner have been great to me over the years.

[Ridiculous questions on CloudDrive]

Update: I tried using the local disk provider stored on a DrivePool. I transferred a 300MB file over and then it crashed, as in ALL drives appeared disconnected from the DrivePool and CloudDrive, and no drives--not even the boot SSD--appeared in the Task Manager. Upon restarting, neither the DrivePool nor CloudDrive service started successfully and I cannot access Disk Management.

One restart later, after uninstalling CloudDrive everything seems OK.

PSA: CLOUD DRIVE ON DRIVEPOOL: DO NOT TRY THIS! Edit: Tracking available via https://stablebit.com/Admin/IssueAnalysis/27859

[Ridiculous questions on CloudDrive]

1. If I create a CloudDrive using the local disk provider, is there any sort of chunk checksumming that takes place?
1a. Does the chunk verification work on local drives?
1b. Does chunk verification apply any ECC?

2. Is it bad practice to store a local disk CloudDrive on a DrivePool?

3. Is it bad practice (aside from being slow) to store a local disk CloudDrive on a DrivePool that has a Google Drive CloudDrive as part of the DrivePool drives? ie: a CloudDrive that relies on DrivePool which relies on another CloudDrive.

[Can Drive Bender and DrivePool coexist on same machine?]

I've had no issues when running Drive Bender and DrivePool on the same machine, so long as they use different drives. In fact, it's how I migrated to DrivePool.

[What does DrivePool do on a duplicated file read error?]

I've read Drashna's post here: http://community.covecube.com/index.php?/topic/2596-drivepool-and-refs/&do=findComment&comment=17810

 

However I have a few questions about ReFS support, and DrivePool behavior in general:

 

1) If Integrity Streams are enabled on a DrivePooled ReFS partition and corruption occurs, doesn't the kernel emit an error when checksums don't match?

2) As I understand it, DrivePool automatically chooses the least busy disk to support read striping. Suppose an error occurs reading a file. Regardless of the underlying filesystem, would DrivePool automatically switch to another disk to attempt to read the same file?

3) Does DrivePool attempt to rewrite a good copy of a file that is corrupt?

 

Thanks!

[What does "unusable for duplication" mean?]

Ok, so here's what happened:

 

I had 3x ~250GB disks in a pool. One was dying, so I removed it. This left me with the 3.05GB of unusable for duplication.

 

I replaced that with a 480GB disk. It then read there were 17GB of "unusable for duplication."

 

So, freaking out, I did a balancing pass, and it went back down to about 3GB afterwards. I suppose there's some math that might leave 3GB of unduplicatable space, but given the slight variances in disk capacities (~240GB/~250GB/~480GB), I'll accept it.

 

No data lost and everything else appears to be in order. Thanks again, Drashna!

[What does "unusable for duplication" mean?]

I recently had to remove a drive from a pool when I realized it was failing during a duplication check. The drive was emptied OK, but I was left with 700MB of "unusable for duplication" even after repairing duplication and remeasuring the pool.

 

Then I decided to move a 3GB file off of the pool, and now it's saying it has 3.05GB of "unusable for duplication." But 700MB + 3GB != 3.05 GB...

 

I'm lost. What is going on? How can I see what is unusable for duplication? Did I lose data? How do I fix this?

 

EDIT: I also have 17.9GB marked "Other" but even after mounting each component drive as its own drive letter, no other files/folders are present than the hidden pool folder.

[Questions on DrivePool behavior]

Thanks so much everyone, especially Drashna!

 

I decided to bite after seeing how robust Stablebit's stuff seems to be, and how good the community is here. I've gone from DriveBender to Storage Spaces and now to DrivePool all for the sake of being able to handle dying drives. I think Stablebit Scanner is exactly what I've needed in conjunction with pooling this whole time.

[Questions on DrivePool behavior]

Thanks for the reply, seems like I might have to get both if I decide to bite.

 

Anyone know about #1?

[Questions on DrivePool behavior]

I've been stalking out a good replacement for Windows Storage Spaces and am interested in DrivePool's behavior as an alternative pooling solution. Since it's file-based and not block-based though, I was wondering:

 

1) Suppose I have large, changing files like an Outlook .pst or virtual hard drives stored on the pool with duplication enabled. What happens when the file is modified/appended to? Does the entire file need to be duplicated again? ie: does it get copied again in its entirety to other drives or does it do something fancy like rsync's rolling hash strategy to calculate differences and update each file incrementally?

I'm trying to avoid saving gigabytes of information every time I use a .vhd or when Crashplan backs up, since Crashplan relies on many 4GB files.

 

2) What happens when a drive begins to fail for some reason and an error occurs while reading a certain file? Does DrivePool pull the file from another drive?

I ask because I've experienced corruption from a SMART failure on a drive recently which made reading certain files from a pool error out. This should have been solved by pulling the file from a different drive in the pool but it didn't.

 

Thanks!