Jump to content
  • 0

Cloud Drive + G Suite = Backup disk


Bigsease30

Question

Hello All,

I have been using Drivepool for a long time now and love it. I am now looking to further protect my files so I purchased Cloud drive and G Suite Business today. I currently have 34.6TB of total space on my server with 8.71TB free. Duplication is currently turned on using (2X). I have connected my Google Drive to the cloud drive and created a 50TB drive. Assuming that this maybe to much now after figuring out that I am only actually using 13TB of space if I were to remove duplication from my current setup. I configured the download to use 10 threads and the upload to use 5 with an upload limit of 70mbps to avoid being banned.  To further add fuel to the fire, most of my data is movies and TV shows that currently are being utilized from Plex media player.

What I would like to achieve is to have a local copy of my files and have them dupe to my cloud drive. I would like for Plex to only use the local files and not attempt to re-download any files. Doing this would allow me to reclaim my duplicated space on my local server. I have tried to figure it out myself but have been struggling the last few hours. I don't want to accidentally delete my pool and lose everything. Any help would be appreciated. I have included screen shots of my drive pool and Cloud Drive. Please let me know if you need additional information to assist me.

Thanks in advance.

drivepool.PNG

clouddrive.PNG

Link to comment
Share on other sites

Recommended Posts

  • 0

Hey Srcrist, after several more reboots of the system and a little time, the Cloud Drive ended up showing as an available add after all. I have since decided to hold off on adding the Cloud drive until I complete the section below. 

On to my next issue. I disabled duplication and manually backups up my important files in-case of a mistake. I removed a single drive from the pool and reformatted it from ReFs to NTFS for compatibility and other personal reasons. When I add it back to the original pool, the stats on the left hand side still say ReFs but if I create a new pool with this drive, it shows as NTFS. I have since replicated this across 6 of my 14 drives so far and counting to get them all over to NTFS.

My question is this, If all of the drives are formatted and added back to the original pool, will the status update to NTFS if there are no ReFs drives left? I would rather not have to create a new pool and then move the data over and then have to link everything back to Z:\ manually. If it stays in the same pool, I will not have to do any extra work minus the drive removal, format and then re-add.

Thanks

drivepool.png

Link to comment
Share on other sites

  • 0

That's actually a really interesting question, and one I don't know the answer to. I have a pretty strong aversion to ReFS (had a large volume go RAW during a windows update once), so I haven't really used it enough to tell you. If I had to guess, yes. Once all of the drives are NTFS, I would imagine DrivePool would report the pool itself as NTFS. But, honestly, even if it *didn't*, you could always move the data out of poolpart folders on the drives and simply recreate the pool and move the data back--and then it would be NTFS for sure. So there isn't really a problem with continuing this path. That process would only take literal minutes, since Windows simply marks the data as moved without actually initiating a transfer. Moving data in and out of a poolpart folder on the same drive is a matter of seconds, generally speaking. 

One thing I want to make clear, though, is that we would still need a nested pool setup to accomplish the end goal. This is because the point of the nested pool isn't to provide some sort of compatibility with ReFS, it's to enable us to treat the entire pool of disks as a single volume to duplicate to another location. There simply isn't any way for us to add a CloudDrive volume to the same pool with all of the local drives, and tell DrivePool to place an additional copy of *all* of the data on the CloudDrive unless we treat the pool as a single volume to mirror. 

The good news is that hierarchical pooling is an in-built feature in DrivePool already. When you nest the pools, you'll see it make adjustments in the UI, signalling some operational adjustments that will ensure that performance isn't diminished. We aren't needlessly complicating the system, or making it do something it isn't *supposed* to do, this is just how the duplication logic of the pool needs to work in order to accomplish your goals. 

Link to comment
Share on other sites

  • 0
On 2/25/2019 at 3:37 PM, Bigsease30 said:

My question is this, If all of the drives are formatted and added back to the original pool, will the status update to NTFS if there are no ReFs drives left? I would rather not have to create a new pool and then move the data over and then have to link everything back to Z:\ manually. If it stays in the same pool, I will not have to do any extra work minus the drive removal, format and then re-add.

 

Yes.

 

And by default, you can't mix NTFS and ReFS drives, unless you enable an advanced option.

 

Also, StableBit DrivePool should be smart enough to not read data from the cloud, if it's available locally.  The only exception is directory listings.

Quote

* [D] [Issue #27655] When a file is opened, each open context is given a priority. Read striping will always work over the highest 
                     available priority class. 
                     Priority classes from lowest to highest are:
                        - CloudDrive, CloudPool, LocalExternalDrive, NonCloudPool, StandardDrive

 

Link to comment
Share on other sites

  • 0

Ok Friends,

The work on the hard drives is complete and data is back as it should be. The drivepool now identifies as NTFS.

I have created a second pool adding both the original pool as well as the google drive. I just cant figure out how to set the correct settings to allow the pool to duplicate to the Google Drive.

NewError.PNG

Capture1.PNG

Capture2.PNG

Capture3.PNG

Capture4.PNG

Link to comment
Share on other sites

  • 0

BOTH the pool and the CloudDrive volume have to be allowed to accept *duplicated* data in the Drive Usage Limiter. If you set it so one can only hold duplicated data and one can only hold unduplicated data, the logic won't allow it to duplicate the content from one to the other. Make sense? So your pool should be able to hold BOTH, while your CloudDrive should only be able to accept duplicated data. Then it should replicate correctly. All NEW data will go to your Z: Pool, and the duplication settings will slowly copy that data to the CloudDrive over time. 

If duplicated data is disabled from a drive, it cannot store either the *original* copy NOR the second copy of a duplication. 

Link to comment
Share on other sites

  • 0
13 hours ago, srcrist said:

Sure thing. Two other things to note:

1) You'll also need to MOVE all of the existing data to the PoolPart folder created on Z: when you made the new pool. Otherwise it will not show up in the new pool for duplication to CloudDrive. 

2) The old pool no longer needs a drive letter. Move your Z: label to the NEW pool now, and applications will access the data via that. After you've moved the data to the new PoolPart folder, of course. 

OK, moved all of the data over to the new pool, changed the drive letter back to Z:/. Changed the option on the pool to allow duplication on both new drives. This seemed to work well, data was uploading fine, until it gave me an error. It told me that I was being throttled because my C:\ was full. Sure enough, it created a hidden folder called CloudPart.XYXYX that was consuming 50GB of space maxing out the drive.

I checked the "Cache"Settings and it was set to 5GB expandable. I have since changed it to Fixed and space appears to be coming back very slowly. What is the appropriate setting for this to not consume my entire C: drive?

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...