Why is duplication inefficient?

[wid_sbdp]

Say I have drive A. There is 5TB in the cloud and 100GB cached locally. I add drive B and turn on file duplication with 2 copies (one on each drive). Why does Drivepool immediately start to download data from the cloud (slow) to duplicate to drive B? Wouldn't it make more sense initially to duplicate anything that already exists in cache (locally) on drive A? 

[wid_sbdp]

This is totally in the wrong forum lol. Hmm

[Christopher (Drashna)]

lol, wow, you really screwed the pooch on this one, didn't you?

I mean, it would have been fine in the DrivePool or CloudDrive forum, but you posted it in the Scanner forum! 

 

Kidding aside:

 

Honestly, it's because StableBit DrivePool is completely ignorant of StableBit CloudDrive.  It treats all drives like they're physically there, with no special handling for StableBit CloudDrive. 

 

But yes, this would be a good idea to implement.  The problem is that CloudDrive is completely file agnostic. It doesn't track or maintain file contents in the cache.  Meaning that we may only have partial files cached (such as in the case of large files), and we have no way of actually knowing what files are in the cache.

 

However, since I believe that Alex wanted to implement a "file pinning" option for CloudDrive... once that is implemented, then we should have the ability to track files in the cache.  From there, it should "trivial" to implement in DrivePool's balancing and duplication engine (eg, query for locally cached files, and bump those to the top of the processing list). 

 

https://stablebit.com/Admin/IssueAnalysis/27265

[wid_sbdp]

Thanks for moving this haha. 

 

Sounds like a great idea, look forward to seeing that. I realize a lot of people use DrivePool for physical storage but it would definitely be nice to have some behind-the-scenes cooperation between CloudDrive and DrivePool, especially with the rising popularity of using cheap unlimited cloud storage to host files.

[Christopher (Drashna)]

Welcome.

 

And yeah, it would be a nice optimization for both products.