Jump to content
Covecube Inc.
  • 0
JulesTop

Pools, Cloud Drive, and Duplication (Optimal configuration question)

Question

Hi,

 

This has likely been asked before, but I didn't find an answer with a quick scan of the forums.

I have multiple HDDs pooled together making up a HDDPool. I also have two Cloud Drives making up a Cloud Pool. I combined both to make a Hybrid Storage Pool.

Now, this is setup with zero duplication but I recently had a drive failure and I am also worried about potential cloud account closure or limitations in the future.

So I have a third cloud drive that I would like to use for duplication... but ideally all three of my cloud drives would be used for duplication... So... do I add teh 3rd cloud drive to the Hybrid storage pool as a third drive and set it as duplication of storage, or is there a way I can add it to the cloud pool and get my cloud pool to share the load of the duplication for all the sub drives in the system. Essentially, I would like to be able to use the three cloud drives in conjunction to get better overall upload rate 3x 75Mbps instead of 2x 75Mbps and 1x 75Mbps for duplication.

 

Thanks!

Share this post


Link to post
Share on other sites

10 answers to this question

Recommended Posts

  • 0

OK, I came up with a solution to my own problem which will likely be the best of both worlds.

Setup my Cloud Pool to Pool Duplicate, also setup my HDD Pool to Pool Duplicate.

Then use both the pools in the storage pool with no duplication as normal.

Share this post


Link to post
Share on other sites
  • 0
On 5/8/2019 at 1:41 PM, JulesTop said:

OK, I came up with a solution to my own problem which will likely be the best of both worlds.

Setup my Cloud Pool to Pool Duplicate, also setup my HDD Pool to Pool Duplicate.

Then use both the pools in the storage pool with no duplication as normal.

I'm actually looking to do something very similar.  In my case I have a pool of 16x physical drives.  Only about 10% free across the pool.  No duplication.

I was hoping to create a couple of CloudDrives (using BackBlaze) that would duplicate the pool of physical drives (basically creating an offsite copy that can be used for restoring)

If I understand you correctly, you created a pool of physical drives with duplication (so all data is duplicated within that data pool).  You then created a cloud pool with duplication.  Then you created a pool of the two pools (physical drives and cloud drives).  Under normal usage, wouldn't Drivepool copy half of the data into the cloud pool to normalize.  Then you have half your data on-site with a backup (duplication), and half your data off-site with backup.

I think the best situation would be to have a physical pool with duplication, AND a cloud pool duplicating the entire physical pool.  That would essentially give you an on-site AND off-site backup.

Share this post


Link to post
Share on other sites
  • 0

The issue with duplicating in a hybrid pool (1 HDD-Pool combined with 1 Cloud-Pool) is that you won't be able to duplicate past the size of your HDD pool (or whichever disk is smaller). It tries to duplicate in the event of a drive failure, but in the hybrid pool, where your two drives are an HDD-Pool and a Cloud-Pool, it will take the smaller of the two drives and duplicate to the other. It won't be able to duplicate beyond that.

The reason I did mine by setting up duplication in the HDD pool via it's sub drives (in your case, the 10 HDDs would be setup with 2x Pool duplication to protect against 1 drive failure) and setup duplication in the cloud pool separately is because I want to have an ever growing drive (so all overflow goes to the cloud drives) while still protecting against any individual drive failure.

I also use the file order placement plugin (downloadable plugin) to prioritize placing files on the HDD-Pool and on top of that I place file placement rules on the folders I want to keep locally... the rest gets uploaded to the cloud.

So, the idea is to protect against any single drive failure while still having unlimited storage available on the PC.

Share this post


Link to post
Share on other sites
  • 0

So, my existing DrivePool is about 60TB.  I create a CouldDrive of size 250TB, then create 5x 50TB volumes, then create a Pool of the CloudDrives for a CloudPool of 250TB.  If I then create a HybridPool (DrivePool + CloudPool) with duplication then it should have no problems duplicating the 60TB I have.

If I add drives to the DrivePool does is automatically adjust the HybridPool, or does it need to be recreated?

Share this post


Link to post
Share on other sites
  • 0

Ok, so I added another physical drive to my pool (aka DrivePool), and got it cleaned up.

image.thumb.png.24de6ddbe6c751d88a74de77a15c0a00.png

I created a 250TB CloudDrive on BackBlaze and split it into 50TB volumes and created a DrivePool from the volumes (aka CloudPool).

image.thumb.png.70fec3270908e703a301e0db7452b429.png

 

I then created a pool from the DrivePool and CloudPool (aka HybridPool)

image.thumb.png.e741f6bc3ebcf93b417e4f2775cd4267.png

 

I downloaded the Balancing Plug-In and set it up as follows.  My understanding is that with these settings my Unduplicated files will fill up the DrivePool first.  And if I never let them fill up then they will all stay on the DrivePool.  The Duplicated files will fill the CloudDrive first.  So when I enable File Duplication everything will be duplicated onto the CloudDrive, and as long as the CloudDrive is larger than the DrivePool and has free space, the duplicated files will stay there.

 

Are my assumptions and set up correct?  Or am I completely clueless and should just go to bed?

 

I disabled the Drive Usage Limiter after taking the screens.

image.thumb.png.0cd816f73939eea816a08bf31270280d.png

 

image.thumb.png.060962561c7e70d0d8d8796d55b638d6.png

 

image.thumb.png.7cd9b169e367a7474d3821de3b696731.png

 

Share this post


Link to post
Share on other sites
  • 0

This setup looks good to me. Yes, all your duplication will go to the CloudPool, from what I see.

This setup will limit the size of your hybrid pool (with duplication fully enabled) to the size of your smallest sub pool, in this case the DrivePool.

Share this post


Link to post
Share on other sites
  • 0
29 minutes ago, JulesTop said:

This setup looks good to me. Yes, all your duplication will go to the CloudPool, from what I see.

This setup will limit the size of your hybrid pool (with duplication fully enabled) to the size of your smallest sub pool, in this case the DrivePool.

Which is totally fine :D

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

×
×
  • Create New...