Google drive+ drive pool+ Cloud drive read only mode possible?

[Sassan]

I currently have a setup using Google Workspace, StableBit DrivePool, and Cloud Drive.

I recently received notification from Google stating that I will no longer be permitted to exceed their allotted storage limit of 5TB. However, I am currently utilizing over 50TB of storage.

In just one month, my service with Google will be transitioned to read-only mode. Considering my familiarity with DrivePool, I am concerned about the potential for drive corruption if I continue using the service until I find a suitable alternative.

I need help to determine if my drive would be at risk of getting corrupted during this transitional period?

[Shane]

CloudDrive is pretty good at avoiding corruption from interruptions, although we did just have someone with a problem that a cloud drive became inaccessible (not corrupted, just unable to be mounted) on Dropbox when that provider set the content read-only (due to an expired account) which has apparently since been fixed in a new CloudDrive update so...

First, make sure you have the latest version of CloudDrive. As noted, this apparently fixes an issue where a cloud drive that becomes read-only due to an action on the provider's end can still be mounted by CloudDrive. @Christopher (Drashna) can you confirm whether my understanding is accurate?

Make sure you complete any existing pending / in-progress uploads to your cloud drive from the local cache well before the transition date; as soon as that's done you  should change the cloud drive to a read-only state. That way there's no risk of anything bad happening from google flicking the switch while CloudDrive is in the middle of writing to the cloud drive.

Note, while the manual also states that cloud drives that are part of a DrivePool pool cannot be set read-only (this way). So if your cloud drive is being used by DrivePool you are going to need to remove the cloud drive from the drive pool before you can change the cloud drive to read-only; if you don't and your cloud drive becomes read-only anyway (due to Google) then your DrivePool pool will be unable to write to the cloud drive and will have trouble operating (I don't think that itself would cause corruption; I'd expect you'd just be unable to perform tasks that expect the cloud drive to be writable - which given Windows loves to write to disk could be problematic).

If you want to remove the cloud drive from drive pool but you are concerned that the amount of data that would need to be moved locally is too much to fit locally and/or complete transferring normally in the month remaining, it is possible to manually split (so to speak) a drive pool; you could then manually complete moving the google-stored read-only pool content into your local-stored working pool afterwards.

TLDR: 1. update to latest CloudDrive version. 2. remove the cloud drive from the drive pool (split the pool manually if necessary to do so in time). 3. complete any pending/current uploads from the local cache to the cloud drive. 4. change the cloud drive to read-only in CloudDrive itself. 5. manually complete moving pool content from google if you had to split it in step 2.

[Christopher (Drashna)]

It depends on how the API handles it, TBH.  As long as the data is accessible, that's the important part.  However, I do believe that the account has to be writable, as some data is written to the provider when the drive is mounted (at the very least, for the "attach"/lock file for the drive.    But as shane mentioned, there was a fix related to this in the beta version.

However, we've added Google drive to the converter tool, so that you can download the contents, and convert to the local disk provider.  This is also in the beta version.

Specifically, you'd need to run "CloudDrive.Convert GoogleDriveToLocalDisk" to kick this off.

 

 

[Sassan]

I'm planning to back up my files using NAS. I installed the beta version of CloudDrive, and I need to create another cloud drive before duplicating the files. Is my understanding correct? I appreciate your expertise.

[Shane]

Yes, assuming access to the NAS is via shares (e.g. \\nas1\share1) you'd add the share(s) you intend to store the new cloud drive on via the File Share provider in CloudDrive and then create the new cloud drive(s) on it. Note that if you plan to create a single cloud drive initially smaller than 16TB and make it larger than 16TB later, you'll need to change the default values it initially uses (cluster size and/or file system) to be able to do so (and if you do run into that issue with an existing cloud drive, you can just create additional cloud drives and use DrivePool to pool them unless you actually need a single "physical" volume to be larger than 16TB for some reason).

[Sassan]

When I try to remove a pool, it's taking forever and I don't think it will take just couple of minutes. Is this going to take days for each separate pool?

[Shane]

When you remove a drive from a pool, (normally) the data in that drive is removed to the rest of the pool first, which is limited by the speed of the connection. If you're using duplication, you can make it quicker by ticking the "Duplicate files later" checkbox at the beginning of the removal process.

Edit: If you've got multiple drives in the cloud that are all part of the same pool, it could become even slower if it's moving data from the drive  in the cloud that is being removed to the other drives that are in the cloud. To avoid that you'd need to set a balancing rule that would ensure the data was moved to only your local drives.

... When you say "remove a pool", do you literally mean "remove a pool" or do you mean to say "remove a drive from a pool"?

 

[Sassan]

It's to remove the cloud drives from the stablebit drivepool. I decided to remove the balancing from it and not sure if that was a good idea.

[Shane]

Ah, okay, you've got multiple google drives in the pool; with default settings that makes the default removal method non-viable because normally DrivePool will remove files from the drive being removed to the remaining drive(s) with the most free space - i.e. most of the time it'll be going to be your other google drives... that you also want to remove. So we'll need to use a different method.

Are the "Google Drive #" disks each on separate accounts each with a 5TB limit or all on one account with one 5TB limit?

Is the "Cloud Media" the NAS or is it also a google drive?

How fast is your average download speed from google drive directly? via CloudDrive? Is your speed high enough to empty or copy the google drives before google switches them to read-only?

If it's fast enough (allow a margin of error), you could add your NAS to the pool and then use the Drive Usage Limiter balancer to empty the google drives into it.

If it's not fast enough, I'd recommend switching your cloud drives to read-only now (by manually "breaking" - this is reversible - the pool if necessary) and then begin copying the cloud drives / pool parts to your NAS.

If you'd prefer to set up a chat (e.g. discord/teams/zoom/whatever) instead of going back and forth day(s) at a time feel free to Message me (rest the cursor over my user icon, should show an option to send a Message) with your preferred chat details/time(zone)s.