Server 2012 R2 Deduplication and DrivePool

[Kayos]

This is Kayot, due to some weirdness I can't seem to log into my real account. The recovery email won't show up either (I checked spam) That aside;

 

Does Server 2012 R2 Deduplication break DrivePool?

 

I know it has to be per drive, which is fine since I use a SnapRaid. I was reading from posts in May on the old forum that DrivePool couldn't access deduped files. Since I'm about to Dedup my Archive I wanted to make sure I wasn't about to nuke DrivePool.

[XAlpha]

With build 692 I seem to be having issues with dedup - it's set up on a couple of the larger disks, and direct drive access through Windows is working fine.  However, DP will not successfully finish measuring the pool.  It will measure all the drives, except the drives with dedup enabled.

[Christopher (Drashna)]

Try 695, as there are a few improvements. 

 

If it continues, let me know.

[XAlpha]

The issue also seems to be with file names greater than 255 characters... Win10  is using the pool for file histories, and some of those directory structures get pretty long.  That also caused the measurements to hang.  I'm waiting for the pool to "heal" & rebalance after removing the long file/directory names, along with re-populating the dedup'd drives.  That will probably take a week or so... so stay tuned.

[Christopher (Drashna)]

The issue also seems to be with file names greater than 255 characters... Win10  is using the pool for file histories, and some of those directory structures get pretty long.  That also caused the measurements to hang.  I'm waiting for the pool to "heal" & rebalance after removing the long file/directory names, along with re-populating the dedup'd drives.  That will probably take a week or so... so stay tuned.

 

This should be fixed in the 684 build, as Alex ran into this while testing. 

684
* [D] Added FILE_SYNCHRONOUS_IO_NONALERT to directory opens. These are used to enumerate folders when reading the duplication count,
      processing file pattern limits, and processing reparse points.
* Enabling duplication on file names > around 200 characters would not work. File names on NTFS are limited to 255 characters, 
  but because we were generating the temporary file name used for background duplication based on the existing file name, it could 
  potentially exceed that limit. This will never happen now. File names on the pool can now be at their maximum 255 character limit.

[XAlpha]

Measured and rebalancing with 701, but the GUI has some glitches.  And for whatever reason, when I reboot the server, any volumes without drive letters lose their labels and won't show up again in the pool until I assign a drive letter and restart the DP service.

[Christopher (Drashna)]

Measured and rebalancing with 701, but the GUI has some glitches.  And for whatever reason, when I reboot the server, any volumes without drive letters lose their labels and won't show up again in the pool until I assign a drive letter and restart the DP service.

Try the 703 build, and see if this issue remains.

[emraith]

I have a question about deduplication with drivepool. If I am understanding everything correctly, I should setup dedup on each individual drive within the pool? So by doing this, it is only going to detect duplicates within the same disk? If I have various files that are actually duplicates and they are split among the 5 disks in my pool, it isn't going to see these as duplicates?

[Christopher (Drashna)]

Correct.  

 

Specifically, Deduplication works on the block level, but since there are no blocks on the pool... it won't work. 

 

And yeah, you're not going to get as much savings from it, because it's going to deduplicate the data per drive, and not across the entire pool.