Jump to content
  • 0

DrivePool + Deduplication issue


tomm1ed

Question

StableBit DrivePool DrivePool 2.2.0.701 Beta running on Windows 2012 R2 Standard with 2 Pools. 

 

One pool containing 4 harddisk partitions (on 4 separate harddisks) which works perfectly.

One pool containing 2 SSD partitions (on 2 seperate SSDs) with (VDI) deduplication enabled, Bypass fill system drivers is disabled.

 

Pool is configured to mirror one specific directory containing VHDx files which deduplicates quiet nicely:

FreeSpace    SavedSpace   OptimizedFiles     InPolicyFiles      Volume

---------    ----------   --------------     -------------      ------

190.59 GB    166.58 GB    9                  9                  E:

120.26 GB    166.38 GB    9                  9                  F:

 

Since the moment that {SavedSpace of E: and F:} > {FreeSpace of E: and F:} DrivePool no longer shows Statistics. It says 'Statistics are incomplete' and I am never able to rebalance anything because of this.

full.png

 

Both Read Striping and duplication is working so it seems it is just a cosmetical error.

I actually think I understand what is happening here, probably DrivePool does not expect there to be more data on the drive than can physically fit.

Is there any way to fix this issue?

Link to comment
Share on other sites

4 answers to this question

Recommended Posts

  • 0

The issue with measuring should be fixed now.
 
However, the space reported is a problem.  This is because of how Deduplication works.  We had partial size reported, but .... 
 
There isn't a good way to handle deduplication here. Period.  Most of the data is stored outside of the pool folder strucuture, in the "System Volume Information" folder specifically.   So it's not actually pooled. 
 
Additionally, beacuse of how NTFS reports the data .... it's far from ideal. 
 
 

I fixed the dedup measurement issue in .699, but here's the problem: 
6KHUsfSZ8LWBYL7yPjT7wxhAsRr6Q.jpg
Deduplicated files don't report how much disk space they are actually taking up, so that's what the measurement pass sees. But it's not crashing anymore.
Technically speaking, that is actually "accurate", because any files that are 100% deduplicated are actually moved into System Volume Information, and only the reparse point exists on the pool part.
But this is misleading. If you place 2 identical files on the pool part, *both* of them will report 0 bytes used on the disk. This happens even though, technically, both files take up the space of what one files should take up (due to deduplication). But the MS filter reports the size as 0 for both files. So the size on disk statistics are actually incorrect.

 

 

So, this is a known issue, but there really isn't a good way to properly handle this.  It's partially a cosmetic issue, but it's partially a technical issue. And one we aren't really sure how to handle. 

 

For now, just keep in mind that deduplication will do this, but it should work fine otherwise. 

Link to comment
Share on other sites

  • 0

The issue with measuring should be fixed now.

 

However, the space reported is a problem.  This is because of how Deduplication works.  We had partial size reported, but .... 

 

There isn't a good way to handle deduplication here. Period.  Most of the data is stored outside of the pool folder strucuture, in the "System Volume Information" folder specifically.   So it's not actually pooled. 

 

Additionally, beacuse of how NTFS reports the data .... it's far from ideal. 

 

 

 

 

So, this is a known issue, but there really isn't a good way to properly handle this.  It's partially a cosmetic issue, but it's partially a technical issue. And one we aren't really sure how to handle. 

 

For now, just keep in mind that deduplication will do this, but it should work fine otherwise. 

 

Logged back into my server and with 708 I am again getting statistics:

full.png

 

Great work Alex!

Link to comment
Share on other sites

  • 0

Glad to hear it. 

 

And yeah, Alex has been hammering away at some of the bigger issues with StableBit DrivePool, including the measuring issues.  We've ran into some weird behavior. 

 

But glad to see that it's working (mostly) properly for you. 

 

It looks like you're entire pool isn't duplicated though, but you're not using "pool file duplication", so that should be fine. 

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...