Jump to content
  • 0

CloudDrive: Pool into DP or separate?


Edward

Question

Looking for a clean strategy for backing up vital data to CloudDrive please. 

Up to now I've been using Crashplan for backing up vital data but want to migrate away from them (long story).  Data in Crashplan comprises about 3tb.

So far I have a DP (12tb with full x2 duplication) and a CD (2tb) using several OneDrive drives which I can readily enlarge/add to. 

My thinking, for simplicity, I would just synchronise my vital data (i.e. not all data) directly to the CD (one way synch with a synch/backup program). But it occurs to me that perhaps a better solution is to add my CD to the existing DP and do some form of folder duplication/balancing of vital data to the CD drives from the DP drives. Is this more optimal?  Another alternative is to create a new DP and then include the existing DP and CD.  Not sure what added benefit this would give me, but asking anyway. 

Data in the CD does not need to be duplicated within the CD (as by definition it will be stored locally and duplicated there and anyway I assume Microsoft's Onedrives are robust).  However I need an easy way of including/excluding folders that get uploaded to the CD. 

 

Link to comment
Share on other sites

14 answers to this question

Recommended Posts

  • 0

It sounds like you'd want a nested pool setup. That is, you'll want a pool that contains your existing pool and the CloudDrive (or a pool of cloud drives, if necessary), and then you can enable duplication between the pool and the drive, or the pool and pool of drives (depending on your needs). That will duplicate your entire existing pool to the cloud. 

Link to comment
Share on other sites

  • 0

Thanks again srcrist.  I'll have a look at nested pools and see what they provide. 

To be clear I don't want to have a copy of all my local data in the cloud, only a subset of data, important data that is critical to survive a local disaster (fire, theft etc) .   Maybe in time I will mirror all local data in the cloud, but for now cloud expense counts against that. 

Link to comment
Share on other sites

  • 0

You can specify what specific folders and files you want to duplicate with the duplication and balancing options in drivepool. But a nested drivepool setup is still what you'd want to use. 

You want a pool that contains your existing pool and the cloud storage, and then configure the duplication at that level. 

Link to comment
Share on other sites

  • 0

Well I gave this a go and hit a brick wall as the parent pool, containing my local and cloud pools, do not 'see' the folders/files in the local pool.

Specifically I following the walk through examples mentioned here:

https://blog.covecube.com/2017/09/stablebit-drivepool-2-2-0-847-beta/

When I try do folder duplication (or the relevant balancing) all I get are the hidden folders and not any of the folders within my LocalPool.  This is contrary to what I expected in the walk through examples mentioned in the above link.  Here is what I'm getting:

image.thumb.png.78d627923a21ea3d11572d9f0c5acea7.png

This is what my LocalPool looks like.  (NB, the sizes mentioned are incorrect, DP was still counting when I grabbed the screenshot).

image.thumb.png.e79a7686386613ef576bce16fe79ec64.png

 

Clearly I have done something wrong but for the life of me don't see what.

Any ideas please?

cheers

Edward

 

 

Link to comment
Share on other sites

  • 0

Once you've created the nested pool, you'll need to move all of the existing data into the poolpart hidden folder within the outer poolpart hidden folder before it will be accessible from the pool. It's the same process that you need to complete if you simply added a drive to a non-nested pool that already had data on it. If you want the data to be accessible within the pool, you'll have to move the data into the pool structure. Right now you should have drives with a hidden poolpart folder and all of the other data on the drive within your subpool. You need to take all of that other data and simply move it within the hidden folder. See this older thread for a similar situation: https://community.covecube.com/index.php?/topic/4040-data-now-showing-in-hierarchical-pool/&sortby=date

 

Link to comment
Share on other sites

  • 0

Thanks again for your valuable input srcrist.  

Hmm, yes after I posted I did imagine I would need to move data around and likely it would involve poolparts.  I certainly did see various poolparts being created.  I'll need to drill down and make sure I do it correctly as I would hate to move things around and inadvertently delete things. 

In the community post you mentioned there is a link to some additional help (see here) which I will also digest. 

I'm also wondering if after I move folders/data around the relevant shares will move as well? Also there are various apps that point to specific locations for their data (e.g music files for Roon and video files for Plex), but I guess I may just have to re-define them.

Link to comment
Share on other sites

  • 0

You want all of your applications now pointing at the hybrid pool. Once the data is correctly moved, the data will appear identically to how it appeared before you nested the pool. The structure of the underlying pool is, as always, transparent at the file system level to applications. Your sub pool(s) do not even need drive letters/mount points, FYI. You can simply give the hybrid pool the localpool mount point. Which, in your case, appears to be P:

To move your data, here is the process:

So let's say you have a hybrid pool (O:), consisting of a localpool (P:) which contains drives D:, E:, F:, and G: and a cloudpool (M:) containing a single cloud drive (we'll just say H:). Right now, if you look at your actual drive file systems you'll have a poolpart folder containing another poolpart folder. That is, D:, E:, F:, G:, and H: all have a hidden poolpart folder in root containing a second poolpart folder. All of the data on each drive that you want to be accessible to the pool needs to be moved into the second poolpart folder on that drive.

So, right now, for example, you probably have G:\Poolpart-XXXX\Poolpart-YYYY\ and G:\Poolpart-XXXX\<a bunch of other stuff> within your poolpart folder on that drive. All of the <a bunch of other stuff> simply needs to be cut and pasted to move it to the Poolpart-YYYY folder instead of Poolpart-XXXX. It will then be accessible at O:, with an identical structure to how it is presently accessible via P:. Note that Poolpart-XXXX represents localpool (P:) and Poolpart-YYYY represents hybridpool (O:), in this example. Each level of nesting actually represents one pool level above the previous. Thus, the master pool of any given hierarchy will be contained in the deepest nested poolpart folder.

You will repeat this movement process for each individual logical volume you are including in the pool. That is, E:\Poolpart-XXXX\Poolpart-YYYY, D:\Poolpart-XXXX\Poolpart-YYYY, etc, etc. Just move everything on the drive to the corresponding poolpart-YYYY folder on the same drive. Then restart the service and remeasure the hybrid pool and it will all be within the pool. 

Link to comment
Share on other sites

  • 0

Hi srcrist

I took yesterday (Dec 25th) off, but now coming back to this.  Thanks for your patience. :)

So conceptually I think I have this.  Stop the drivepool service, move (within each pooled drive) all folders/files I have at the same level as the first poolpart folder into the child poolpart folder, restart drivepool service.   Change drive letters etc.

Small issue however, one of the pooled drives (drive F: in image below) does not show the child poolpart folder. Am confused. :unsure:

Any ideas? 

Cheers

image.png.f08dbfc3ea060ba096fe26803c77e252.png

Link to comment
Share on other sites

  • 0

Doublecheck that it's correctly added to the subpool. I can't see that it is from your screenshots, and that folder should be created as soon as it's added to the pool. If it does look like it's correctly added, and the folder still does not exist, I would just remove it and re-add it to the subpool and see if that causes it to be created. Beyond that, you'd have to open an actual support ticket, because I'm not sure why it wouldn't be created when the drive is added. 

Link to comment
Share on other sites

  • 0

Well I deleted the parent pool (which I had called HybridPool) which contained LocalPool which itself did contain 4 real HDDs all of which contains the relevant poolparts for LocalPool.

I then rebooted and created HybridPool again and only added LocalPool (skipping CloudPool for now).  Now the relevant child poolparts (created by the creation of HybridPool) are only seen in 2 of the 4 real HDDs.  :unsure:

So something screwy happening here.  I'll ask Christopher via a ticket for his assistance. I had hoped to get this wrapped up now, but will have to wait for Christopher's timelime (I know he is super busy). 

By the way where do users here get such massive cloud storage? For example Google charges over $200pm for 10tb which is way above what I would pay. 

Link to comment
Share on other sites

  • 0
22 hours ago, Edward said:

By the way where do users here get such massive cloud storage? For example Google charges over $200pm for 10tb which is way above what I would pay. 

Some people are lucky enough to get unlimited drives through their work or school, and some people use gsuite accounts which have unlimited with more than 5 users on the domain, or 1TB with less than 5...but Google doesn't actually enforce that limit, as far as I know. 

Link to comment
Share on other sites

  • 0

Hi @vrcsix

I'm still struggling to reach a conclusion on this sadly. I opened a ticket regarding the child poolparts not appearing on the underlying HDDs and Christopher (at support) came back and said:

______

Quote

 

For the child poolpart, that may not appear on all of the poolparts.  The best way to handle this may be to seed the child pool, and then seed the hybrid pool, rather than to try to move the data directly into the nested poolpart folder. 
 
This may be more work, but it should avoid the issue that you're running into. 

 

_____
 
However, before I could start 'seeding', I ran into a 'permissions' error state which Christopher helped me out by pointing out the following link:
 
 
I've now done that and still only see partial poolparts so I will need to do the 'seeding' mentioned but it is not clear how.  So I've gone back to support asking for details on how the seeding should take place.
 
However I'm 8 hours ahead of support so the turnaround times become protracted.  :(
 
I'll update this thread when I can.
 
E

 

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...