Jump to content
  • 1

Use DrivePool as a cache for a CloudDrive


Neaox

Question

Hi there,

Is it possible to use a DrivePool made up an SSD (1TB) and an HDD (10TB) as the cache drive for a CloudDrive?

I would love to use the SSD Optimizer plugin with DrivePool to create a fast and large cache drive. This would give the best of both worlds, newer and more frequently used files would be stored on the SSD, older files would slowly be moved to the HDD, you would only need to download from the CloudDrive if you are accessing an older file that is not in the cache...

Is this possible?

Link to comment
Share on other sites

8 answers to this question

Recommended Posts

  • 0

It's possible, but provides no benefit over simply using the SSD as the cache drive. The cache is stored as a single file, and will not be split among multiple drives in your pool. Just use the SSD as your cache drive, and the cached content will always be at SSD speeds. 

Link to comment
Share on other sites

  • 0

Adding on to this:

My Pool Consists of x3 8TB HDDs, and x2 500GB SSDs. I currently have 2 TB "Free", but can't define the pool as a cache drive and have to dedicated another SSD I have dedicated as cache for the role, but its 500GB of storage limits the speed at which I can upload files.

For example if I'm uploading a bunch of files, it has to wait to clear out (upload) the SSD and then write more files which are effectively throttled. I guess I'm going to have to clear at least a good 4-6 TB of data, and assign a drive letter to one of the mechanical drives so I can set it as a cache drive for efficient uploads.

Link to comment
Share on other sites

  • 0
9 hours ago, Bowsa said:

Adding on to this:

My Pool Consists of x3 8TB HDDs, and x2 500GB SSDs. I currently have 2 TB "Free", but can't define the pool as a cache drive and have to dedicated another SSD I have dedicated as cache for the role, but its 500GB of storage limits the speed at which I can upload files.

For example if I'm uploading a bunch of files, it has to wait to clear out (upload) the SSD and then write more files which are effectively throttled. I guess I'm going to have to clear at least a good 4-6 TB of data, and assign a drive letter to one of the mechanical drives so I can set it as a cache drive for efficient uploads.

Just remember your hard upload max of 750 GB /day at GDrive. So if upload is your concern, you won't be able to do more on a single account pr. day

Link to comment
Share on other sites

  • 0
6 hours ago, steffenmand said:

Just remember your hard upload max of 750 GB /day at GDrive. So if upload is your concern, you won't be able to do more on a single account pr. day

Fortunately, I'm not subject to this upload max of 750 GB a day...so the point still stands

Link to comment
Share on other sites

  • 0

You'll need to use the CreateDrive_AllowCacheOnDrivePool setting in the advanced settings to enable this functionality. See this wiki page for more information. The cache being full will not limit your upload, only your copy speed. It will throttle transfers to the speed of your upstream bandwidth, so it should make effectively no difference, aside from the fact that you won't be able to copy new data into the drive faster. That data will still upload at the same rate either way. 

Link to comment
Share on other sites

  • 0
2 hours ago, srcrist said:

You'll need to use the CreateDrive_AllowCacheOnDrivePool setting in the advanced settings to enable this functionality. See this wiki page for more information. The cache being full will not limit your upload, only your copy speed. It will throttle transfers to the speed of your upstream bandwidth, so it should make effectively no difference, aside from the fact that you won't be able to copy new data into the drive faster. That data will still upload at the same rate either way. 

It does make a difference, CloudDrive doesn't start uploading at full speed until a certain amount of data is copied/cached, so having to go through the process for every 400GB isn't ideal.

Link to comment
Share on other sites

  • 0
5 hours ago, Bowsa said:

It does make a difference, CloudDrive doesn't start uploading at full speed until a certain amount of data is copied/cached, so having to go through the process for every 400GB isn't ideal.

That simply isn't true. Are you sure that you aren't running into some sort of I/O issue? CloudDrive will upload without restriction as soon as your upload threshold is met in the performance settings. Leave "Background I/O" enabled to ensure that writes are prioritized over reads in Windows' I/O and see if that fixes your problem. Or try disabling it if it's already enabled and see if other processes are simply getting in the way. 

I know this, btw, because I transferred 90TB from one drive to another, and my cache was full for months. So I know from experience that the cache being full does not throttle upstream performance. 

Link to comment
Share on other sites

  • 0
On 5/1/2019 at 5:05 PM, srcrist said:

It's possible, but provides no benefit over simply using the SSD as the cache drive. The cache is stored as a single file, and will not be split among multiple drives in your pool. Just use the SSD as your cache drive, and the cached content will always be at SSD speeds. 

Is this still true? I'm looking at the CloudPart folder in one of my cache drives and I'm seeing over 5000 files in that directory.

Edit: Does this apply to data you have queued to upload as well? For instance could I not use a pool of a small SSD and large HDD as a cache to increase the total amount of data I can "transfer" to the CloudDrive and leave it to upload in the background?

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...