Jump to content
  • 0

CloudDrive consuming local drives


Edward

Question

I had not fully appreciated that for all CloudDrive allocations it consumes the same amount of space on my local drives on a 1-2-1 basis.  Specifically I have ~2.2tb of CloudDrives (over 3 allocations) and I also have 3 instances of CloudPart folders on 1 local drive.  This has consumed the majority of of my spare local disk space.  In my case the CloudParts are on my largest HDD which is now nearly full.

 

Is this correct or is something wrong with my set up?

 

If it is correct behaviour then CloudDrive is quite a bit more costly due to the requirement to provision local storage.  :(

 

I would have thought that these mirroring requirements could be done via some sort of efficient database handling of metadata or some such handling.  I really do hope my set up is in error.  :)

 

Cheers

Edward

Link to comment
Share on other sites

7 answers to this question

Recommended Posts

  • 0

Well, there are a couple of things here.... 

 

 

The default cache type is "expandable".  This means that as you write to the drive, it will expand past the limit specified, if needed.  And can/will consume most of the available disk space, if needed. 

There are a few other cache types as well, that hard limit the amount of space that the cache can use. And will throttle writes to the drive, to prevent it from exceeding that limit.  (aka, as the cache gets full, the drive will get slower to write to).

 

 

That said, the cache uses "sparse files".  These have a "size" and "size on disk".  The second being the actual size that the data takes, while the "size" is what it's reported as (which would be the CloudDrive disk size, IIRC).  

So make sure you're checking the "size on disk" and not the reported "size". 

Link to comment
Share on other sites

  • 0

Hi Chris.  Thanks for responding.

 

I'm not getting the full picture here.   I get the difference between 'size' and 'size on disk'.  However the numbers are not consistent.  I have 3 onedrive 'disks' (H, I and J) all within one CloudDrive Pool (Z). Currently I have nothing in the pool.

 

When I look directly at H, I or J at the 'size on disks' the numbers are low (~200mb each).  The size of the PoolParts on these drives are zero.  Looking at the poolparts on the real physical drive (d:) the numbers are inconsistent.  Two of the poolparts show 1gb used and the other 280gb used. 

 

All three of these Onedrives were set exactly the same in the CloudDrive app - namely with 1gb local cache and expandable drives albeit max sizes set at different values.

 

Within the DrivePool app I'm seeing ~700gb of 'other' and I can't get anywhere near that number by summing the values I see as being either 'size on disk' or 'used' (fully recognise that there may be other stuff like metadata in 'other').  Having a way of drilling down into 'other' would help in the analysis. The app is reporting a value so must know what it comprises.

 

Also it would be helpful for my understanding that once data is uploaded to the relevant CloudDrive at that point the local cache is deleted?  Or is it the case that all data in a CloudDrive has to be mirrored physically at all times on a local physical drive? 

 

kr

E

Link to comment
Share on other sites

  • 0

Would you mind posting a screenshot of what you mean?

 

 

 

And specifically, for the "other" data:

http://community.covecube.com/index.php?/topic/37-faq-unduplicated-vs-duplicated-vs-other-vs-unusable/

 

It could be shadow copies, or other hidden data, or data that isn't pooled. 

Link to comment
Share on other sites

  • 0

Would you mind posting a screenshot of what you mean?

 

 

 

And specifically, for the "other" data:

http://community.covecube.com/index.php?/topic/37-faq-unduplicated-vs-duplicated-vs-other-vs-unusable/

 

It could be shadow copies, or other hidden data, or data that isn't pooled. 

 

Thanks Christoper 

 

Since my last post I've been trying to sort things out.  Specifically I moved about 500gb of duplicated data out of the pool to an external USB drive. This had two positive and one negative effects. The positives are that the free space in the pool increased by 2x (as expected) and 'other' decreased by about 400gb (not expected). The negative is that I have 500gb unprotected data.

 

The current view of the main pool is as per screen shot below (I did not grab a screenshot of what it was before).  What I'm not getting is why 'other' has reduced to such an extent (still significant but far better than ~700gb it was before).  How is Drivepool calculating 'other' and do you have a utility that allows me to drill into the detail?

 

DrivePoolCapture.JPG

Link to comment
Share on other sites

  • 0

For other data: 

http://community.covecube.com/index.php?/topic/37-faq-unduplicated-vs-duplicated-vs-other-vs-unusable/

 

And there isn't a good way to tell

 

However, for the unduplicated data, there are a few ways to get to that. 

Check the folder duplciation options. If you see a "+" next to the number in the tree, then something isn't inheriting the duplication setting. 

 

Otherwise, you can use the "dpcmd check-pool-fileparts" to see what files are affected and where they are located (set the duplication level to "1" when running the command).

(in fact, the last command in the example is what you want)

dpcmd check-pool-fileparts "P:\" 3 true 1

 

This will show you all of the unduplicated data on the pool.  

Link to comment
Share on other sites

  • 0

Thanks Chris

 

I downloaded the dpcmd utility from here, however the paramater check-pool-fileparts does not seem to recognised.  But In any event my unduplicated data is explicitly input by me in a partial endeavour of reducing disk usage.

 

Thanks again for the link to Shane's and Alex's post about 'other' - all fully understood.  Small question though if I may.  How/where does DrivePool get the reported value?

 

thanks/regards

Edward

Link to comment
Share on other sites

  • 0

the dpcmd utility is included in any 2.X version of the software. You shouldn't need to download it, unless you're using the 1.X version. 

 

 

As for the reported value, DrivePool indexes all of the data.  The "measurement pass" reads the size/size on disk of all of the files on the pool.  This is why it can take a while, or some disks take longer than others. 

 

The data is stored in the settings store, and is updated as data is modified through the pool.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...