Using DrivePool + SSD Optimizer Plugin as an upload buffer for CloudDrive

[RG9400]

I've been thinking about a new setup, and I wanted to float an idea to see if it works (pros/cons). Basically, right now, I have my clouddrive mounted to a single Optane SSD (C:). This works great with speeds, but the drive is limited to 1TB, and with slow upload, this drive is full almost always, so it's hard to copy new data over. For this purpose, I have a bunch of HDDs that I added to a DrivePool pool along with the CloudDrive, and I copy over new data to those pools before manually moving it to the PoolPart folder in C: when the most recent data is uploaded. It is manual and cumbersome.

Could I do something like this instead? I create a pool of my HDDs, then add that to the CloudDrive pool. I set up the CloudDrive pool to download new data to the HDD pool, and then use the plugin to move data from the HDD pool to the CloudDrives? This way, my pool sees all the data, but the underlying locations are being managed automatically. 

If this scenario is feasible, I do have a few questions.

1. If I have a file on the HDD portion, and I do a "move" via my Pool, will it remain on the HDD portion? Will it remain on the same disk it was downloaded to initially?

2. Can I control the plugin so it moves to CloudDrives based on the cache drive space? The CloudDrives will always look empty/have tons of space but my underlying cache may not. I am concerned that the plugin will constantly be trying to move data to the cache drive, resulting in it being full 100% of the time and slowing writes down.

[srcrist]

I'm having a little trouble parsing exactly what your overall goal is here, but, if I'm reading you correctly, I should note that the CloudDrive cache is a single consolidated file, and storing it on a DrivePool pool wouldn't really provide any benefit at all over simply storing it on a single drive. You can do so using the advanced settings for CloudDrive, but it really doesn't add any benefit. It still wouldn't be able to scale any larger than a single drive, and it can't be duplicated to increase performance.

Unfortunately, a slow upload is really just a hard limitation on the functionality of CloudDrive. A larger cache drive is a bandaid, but, in the long term, there really isn't any way to add data faster than your upload can send it to your provider.

[RG9400]

Yeah, I knew the cache drive limitation which is unfortunate. I was actually thinking to do it the other way around. Basically the CloudDrive remains on a single SSD cache, and then I add a local hard drive to a pool with the CloudDrive. I set DrivePool to download to the local HDD first, and then use the SSD Optimizer or some other balancing mechanism to move files from the HDD to the CloudDrive. In this manner, the CloudDrive cache remains on a single SSD, but I have an upload buffer via the HDDs.

I am not sure if the above is feasible. I think the biggest concern would be how to move the files from local to CloudDrive given that the pool will not see the space of the underlying cache.

EDIT: It does not seem like my pool with CloudDrives in it allows the balancing plugins to actually work. The options seem disabled, though I felt the SSD Optimizer Plugin was close to what I wanted in theory where the HDD acted as an "SSD" and the CloudDrive acted as the Archive. 

[RG9400]

I did some reorganizing, and I basically created a pool of pools. The pool contains an HDD pool (a bunch of local hard disks) and a CloudDrive pool (a bunch of CloudDrive partitions). Now the optimizer plugin seems to be available, and I can properly set the HDD pool as the SSD, and the CloudDrive pool as the Archive. This seems fairly close to what I want, but I still don't know how the balancing would work.

1. Does it try to move *all* files off the SSD into the archive in the first balancing run? Does it run for a set amount of time, or once it starts, it will try to move everything? Reason is that I have a ton of data on the HDD pool that cannot move to the SSD pool right away. I only see a way to immediately balance or to balance at a set time every day, maybe indicating there is no way to move the files over time

2. Are the files that are being moved inaccessible during this time? Can this create issues if applications like Plex are running?

3. If the balancing is running, and the files being moved are too large in size for my CloudDrive cache, I assume writes will be slowed, and the whole balancing task will still run as and when data is uploaded from the CloudDrive. If, during this time, I add a file, will it get placed on the HDD pool since that is still the SSD? Basically, could I theoretically be running balancing 24/7 where files are being added to the HDD pool as others are being moved to the CloudDrive pool? The concern would be that those files may be inaccessible while being moved, and I guess my cache drive would be perpetually be at 99% capacity with slowed writes and under heavy wear.