Jump to content
  • 0

What happens when the cache drive fails?


darkly

Question

I've been running my drive with a cache on my RAID so far, but due to the amount of i/o from other apps causing a bottleneck from time to time, I've been considering getting a dedicated SSD to use as a cache drive. My only concern with moving it off my RAID is, what happens when the cache drive fails? The way I see it, there's a few different scenarios here. For instance, there could only be downloaded read cache stored on the cache at the time of failure, there could be an actively uploading write cache, or there could be an upload queue with uploads currently paused. There could be other cases I haven't considered too. What I'm interested is what would happen if the drive failed in any of these scenarios. What protections does CloudDrive have to prevent or limit data loss/corruption in the cloud? 

Link to comment
Share on other sites

12 answers to this question

Recommended Posts

  • 0

Well, you'd be able to access the data still, but it will be all over the place.   

So, yes, you'd want/need both products installed.

As for assigning the drive letters, maybe.  If you'd removed them, then it "shouldn't" reassign drive letters on the new system, but Windows can be weird about it.  

On 5/5/2018 at 6:24 AM, darkly said:

does DrivePool see the existing DrivePool data in each partition and know that they belong to a pool?

Yes.  That's specifically how it automatically repools the data.

On 5/5/2018 at 6:24 AM, darkly said:

Basically, will the existing pool structure automatically get recognized and implemented?

Yup, exactly. 

Link to comment
Share on other sites

  • 0

That depends. But generally "bad things happen". 

If you have any "To upload" data, then that data is lost, and you may end up with a corrupted drive... :(
In this case, mount the drive on a new system and run CHKDSK on the drive to find and fix any problems/corruption. 

If there was no "to upload" data, then you're fine, and you should be able to just reattach the drive, albeit using the "force attach" option. 

 

As for limit or prevent, there isn't really anything, unfortunately.  Having StableBit Scanner installed may/should give you a heads up, if the drive is having issues.  Then you can detach the drive before you run into actual issues.

Link to comment
Share on other sites

  • 0

Sounds like a good reason to make the Cache SSD a RAID 1 mirror - two smaller but fast SSDs with some redundancy to protect the Pool data.  Would also help with write speeds to the Pool on concurrent transfers, with enough Pool drives present.

That makes me wonder - could you also use a RAMdrive in place of the SSD cache drive, to act as the write buffer for DrivePool?  If it's just picking a logical volume as the cache volume..

Link to comment
Share on other sites

  • 0

So I went and did some testing, answered my own question.  With the right RAMdrive software (I used SoftPerfect RAMDisk 3.4.8 which is free) and Hard Drive Emulation turned on, it's very easy to add a RAMdrive into the Pool and set it up as the SSD drive for for SSD Optimizer Plugin.

The only drawback, is that the Pool Balancer doesn't immediately (on-the-fly) move files off that SSD to the larger Archive drives, so that newer files coming into the Pool have a place to land.  Basically the SSD Optimizer allows any drive used for the SSD Drive in the Pool, to count as a totally valid Pool drive and hold on to files as long as it cares to.  That may be by design, but I didn't find DP treats files on that SSD cache drive with any sense of true urgency - even with 100% immediate balancing turned on, it took an entire minute (?) for it to check and start moving files off the RAMdrive I had in place for it.

So from that perspective, I don't think I'd recommend a RAMdrive in place of the SSD cache drive, due to volatility and SD's non-urgency.  Might be better off just buying a larger SSD and using it as designed.  Or something like Primocache with a write-only cache set on the underlying DP volumes, which I might try next for write performance testing.

If you get to reading this Christopher - have you and the rest of the team done any testing with Primocache, to be able to say if it's compatible with the rest of DP?

Link to comment
Share on other sites

  • 0
4 hours ago, Christopher (Drashna) said:

That depends. But generally "bad things happen". 

If you have any "To upload" data, then that data is lost, and you may end up with a corrupted drive... :(
In this case, mount the drive on a new system and run CHKDSK on the drive to find and fix any problems/corruption. 

So right now, my cloud drive has 8 partitions, each are mounted to directory, then all 8 are pooled together in Drive Pool and mounted to another directory, then that pool is in another pool by itself. Only that highest level pool is assigned a drive letter and I access all the data through that only. How would I go about properly mounting this on another system if I needed to? 

4 hours ago, Christopher (Drashna) said:

If there was no "to upload" data, then you're fine, and you should be able to just reattach the drive, albeit using the "force attach" option.

Does the amount of upload data that was queued at the time of drive failure affect the likelihood of issues/corruption? 

4 hours ago, Christopher (Drashna) said:

As for limit or prevent, there isn't really anything, unfortunately.  Having StableBit Scanner installed may/should give you a heads up, if the drive is having issues.  Then you can detach the drive before you run into actual issues.

Is there any possibility of adding levels of protection in the future? I don't know the inner workings of CloudDrive very well, but off the top of my head, would something like writing empty placeholder files onto Google first or something of the sort reduce the risk of damage? Or maybe there's something else you can do entirely? I wouldn't know where to start. 

3 hours ago, Jaga said:

Sounds like a good reason to make the Cache SSD a RAID 1 mirror - two smaller but fast SSDs with some redundancy to protect the Pool data.  Would also help with write speeds to the Pool on concurrent transfers, with enough Pool drives present. 

Yeah I probably need to do this. The only problem for me is that I'm regularly working with around a terabyte at a time so I was hoping to use a 1tb SSD. Two of those is getting $$$$$

Link to comment
Share on other sites

  • 0
8 minutes ago, darkly said:

Yeah I probably need to do this. The only problem for me is that I'm regularly working with around a terabyte at a time so I was hoping to use a 1tb SSD. Two of those is getting $$$$$

That sure would get expensive quickly, especially for SSDs of that size.  Depending on the type of data you were placing on the Pool, NTFS volume compression might help a little.  I wouldn't count on it however.  For now, probably just stick with 1 SSD and set balancing to happen immediately.  1 minute when you have that much space and other local volumes to copy off to isn't too long.  You couldn't saturate network bandwidth enough to overflow with immediate balancing.

I will still do some write-cache testing with Primocache if I can get around to it.  The drawback there is I can only manage about a 26 gig write-only cache in the software, so stuff like large backups would spill over and defeat the purpose without a highly aggressive write policy.  Smaller files and media files would still work fine.

Link to comment
Share on other sites

  • 0
1 hour ago, Jaga said:

That sure would get expensive quickly, especially for SSDs of that size.  Depending on the type of data you were placing on the Pool, NTFS volume compression might help a little.  I wouldn't count on it however.  For now, probably just stick with 1 SSD and set balancing to happen immediately.  1 minute when you have that much space and other local volumes to copy off to isn't too long.  You couldn't saturate network bandwidth enough to overflow with immediate balancing.

I will still do some write-cache testing with Primocache if I can get around to it.  The drawback there is I can only manage about a 26 gig write-only cache in the software, so stuff like large backups would spill over and defeat the purpose without a highly aggressive write policy.  Smaller files and media files would still work fine.

Not sure what balancing you're talking about. I'm only DrivePool with the partitions of my cloud drive (the configuration I described 2 posts above). The SSD would only be used as a cache for the letter mounted drive (top level pool)

Link to comment
Share on other sites

  • 0
18 minutes ago, darkly said:

Not sure what balancing you're talking about. I'm only DrivePool with the partitions of my cloud drive (the configuration I described 2 posts above). The SSD would only be used as a cache for the letter mounted drive (top level pool)

That's how the SSD cache works - DP "balances" the stuff off of that drive, onto your Pool drives, on a schedule according to what you set in the balancing area of DP.  It's a slight misnomer, but that's how the cache plugin works with DP.

Anything 'caught' on the cache SSD if it failed, would be lost.  So a more aggressive balancing schedule in DP (to move from cache SSD to your cloud partitions) would protect better against drive issues.

Link to comment
Share on other sites

  • 0
36 minutes ago, Jaga said:

That's how the SSD cache works - DP "balances" the stuff off of that drive, onto your Pool drives, on a schedule according to what you set in the balancing area of DP.  It's a slight misnomer, but that's how the cache plugin works with DP.

Anything 'caught' on the cache SSD if it failed, would be lost.  So a more aggressive balancing schedule in DP (to move from cache SSD to your cloud partitions) would protect better against drive issues.

Not sure why you keep going back to DrivePool balancing. This is the CloudDrive forum after all. And as I've repeatedly stated above, I'm not talking about a DrivePool cache. I'm talking about CloudDrive's cache. As I've said, again, the only way DrivePool is being used is to pool together the multiple partitions of the CloudDrive into one drive. 

Link to comment
Share on other sites

  • 0
On 5/3/2018 at 12:02 PM, Jaga said:

Sounds like a good reason to make the Cache SSD a RAID 1 mirror - two smaller but fast SSDs with some redundancy to protect the Pool data.  Would also help with write speeds to the Pool on concurrent transfers, with enough Pool drives present.

That makes me wonder - could you also use a RAMdrive in place of the SSD cache drive, to act as the write buffer for DrivePool?  If it's just picking a logical volume as the cache volume..

Yup, to all of that.  But I highly recommend against RAM drives, because they're small and very prone to data loss. 

21 hours ago, darkly said:

So right now, my cloud drive has 8 partitions, each are mounted to directory, then all 8 are pooled together in Drive Pool and mounted to another directory, then that pool is in another pool by itself. Only that highest level pool is assigned a drive letter and I access all the data through that only. How would I go about properly mounting this on another system if I needed to? 

If you move all of the drives (virtual or physical), then it should "just work". 
Otherwise, you may need to re-mount the folder paths.   The rest should "just work" after connecting them to the new system. 

 

21 hours ago, darkly said:

Does the amount of upload data that was queued at the time of drive failure affect the likelihood of issues/corruption? 

No.  Well, it can.  

It's the "physical location" of that data on the drive that matters. The physical data structures that make a difference.  Namely, the partition information and file allocation table ("MFT").   

But the more data that you have to upload, the more likely that this is some of that data. 

A CHKDSK pass can fix/recover a lot of this type of damage/corruption, so it's always worth running in this case. 

21 hours ago, darkly said:

Is there any possibility of adding levels of protection in the future? I don't know the inner workings of CloudDrive very well, but off the top of my head, would something like writing empty placeholder files onto Google first or something of the sort reduce the risk of damage? Or maybe there's something else you can do entirely? I wouldn't know where to start.

Not really.  The issue is that the data is upload or not.  

That said, if the system crashes, but the cache data is intact, you could attach that drive to the new system, and StableBit CloudDrive will actually pick it up.  You'd need to "reauthorize" the drive, but it would continue to upload the data.... as if nothing happened. 

For this reason, I personally recommend using a non-system drive, to make recovery simpler. 

21 hours ago, darkly said:

Yeah I probably need to do this. The only problem for me is that I'm regularly working with around a terabyte at a time so I was hoping to use a 1tb SSD. Two of those is getting $$$$$

Ouch, yeah, I understand that.  Though I don't have a good answer, really. 

Well, other than to have StableBit Scanner installed on the system, and hope it catches any issues before the drive dies (internal betas support NVMe health info, now) 

Link to comment
Share on other sites

  • 0
16 hours ago, Christopher (Drashna) said:

If you move all of the drives (virtual or physical), then it should "just work". 
Otherwise, you may need to re-mount the folder paths.   The rest should "just work" after connecting them to the new system. 

Here's what I'm trying to figure out. So I only have one actual "cloud drive", but it's split into multiple partitions. Those are pooled together in DrivePool, and the resulting pool is pooled as well (on its own for now (future proofing)) in yet another pool. So obviously, I'd need both CloudDrive and DrivePool in the second system to properly mount the drive. I'm confused at how this would happen. First I mount the cloud drive. I'd assume that my system would try to automatically assign drive letters to each partition? I then have to manually remap each partition to a directory instead, right? But either before or after I do that, does DrivePool see the existing DrivePool data in each partition and know that they belong to a pool? Or do I have to add each partition to a pool again? This is the part that I'm not sure about. Basically, will the existing pool structure automatically get recognized and implemented? Because without the pool, the data on each partition is useless.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...