Numbers don't add up

[Andy T]

I'm just in the process of switching from DriveBender to DrivePool. As of this this screenshot I had added:

• 1 x 1tb hard disk
• 1 x 8tb hard disk
• 1 x 256gb SSD

So theoretically the most data that I can store with duplication is 1.2tb.

So how did DrivePool manage to duplicate 1.9tb? And what is the 54gb of "Other"? These 3 disks were formatted before being joined to DrivePool. I know that Windows may put some things in System Volume Info behind the scenes, even if the disk doesn't have a drive letter... but 54.5gb?

[Shane]

Duplication measurement is the total used space; e.g. if you had a 1MB file duplicated across drive A and drive B, the Duplicated measurement would be 2MB.

For finding the Other, I suggest a program like JAM TreeSize which (when run elevated) should be able to accurately report the consumption of a drive including system folders.

[Andy T]

Thanks Shane. Once you point it out thats pretty obvious actually!

I still can't account for Other. Even using TreeSize there is nothing else on the disks. I am using 64k clusters but it feels unnaturally high to be the wasted space/in-efficiency of 64k for small files, given that most of the files are multi-gb.

[Shane]

Here's a FAQ on what Other etc is, with a followup post where Alex goes into more detail on Other; in your case I'd suggest it's the metadata, directory entries and slack space.

For whatever vague comparison it's worth, my 39.1TB pool with 4KB clusters has 18.0GB of Other. Note that (64/4)x(8.4/39.1)x18 ≈ 62GB, so by those factors your pool appears to have proportionately less "Other" than mine... and as a fraction of total capacity your Other is consuming less than 0.65% of your pool. I wouldn't worry about it! 

[Andy T]

Thanks for all the info Shane.

I'm trying to force the only small files, which exist in a single folder to go onto a pair of drives formatted with 4k clusters, but I'm battling with the Balancer vs File Placement rules settings.

I'm now up to 71tb of data and 469gb of Other!

[Christopher (Drashna)]

That's a lot of "Other" data.  Enough, in fact, that I suspect that it's being used by Previous Versions, aka Volume Shadowcopy Service (VSS).  Disabling this feature on the drives in the pool may fix this.  (and this data is located in the system+hidden folder "System Volume Information").

You can quickly get to this by running "systempropertiesprotection.exe" on the system.  Turn off the protection for the pooled drives, with the "configure" button  for each drive.  You can also delete the "restore points" from there. 

[Andy T]

It's not VSS.

I'm exporting a list of every file and it's size to Excel so I can do some maths to see how much spaces is wasted due to the 64k block size. I have a File Placement rule that *.nfo files (which are mostly the files < 64kb) are all on 2 drives which are formatted for 4k clusters.

Hopefully using Excel I'll be able to clarify the root cause.

[Andy T]

So in case anyone else comes here I have reduced it down to 628gb of "Other" on 110tb of disk.

I've determined (using a blank disk) that the root cause is the checksums for ReFS metadata. See https://learn.microsoft.com/en-us/windows-server/storage/refs/refs-overview

 

I can reduce it using NTFS, but I prefer ReFS for integrity. I know that is a controversial topic but I've made my choice, I'm happy with it and over many years of using ReFS with DriveBender I haven't had a problem.

 

Big thanks to Shane and Christopher for offering suggestions.