Jump to content
Covecube Inc.
  • 0
TheBigMan

Data Dedupe Role on WinSrv 2019 not working

Question

Hi,

I recently bought and started using DrivePool on my home media server.  I am running a fresh install of Windows Server 2019 Datacenter.  Since I installed DrivePool, I am no longer able to use the Deduplication role, as the service won't start.  Removing DrivePool and rebooting allows the Deduplication service to start again successfully.  To be clear, I am not using Deduplication in conjunction with any drives that are part of my pool, but it is used on another volume to provide massive space savings on File History and even image backups of my PCs.  So, not having Deduplication working is a pretty big issue!

The Dedupe service seems to be failing with the following information in the Event Log:

===

Data Deduplication error: Unexpected error.

Operation:

   Starting File Server Deduplication Service.

Error-specific details:

   Error: DeviceIoControl (FSCTL_GET_NTFS_VOLUME_DATA), 0x80070001, Incorrect function.

===

Has anyone else seen this issue?  Could one of the developers let me know if this is a known problem?  This does *not* happen on Windows Server 2016, so I assume something has changed in Windows Server 2019 such that the Dedupe service is looking for something from the pool drive that DrivePool isn't responding correctly on.

I'm using v 2.2.3.950 BETA as this specifically calls out Windows Server 2019 compatibility.  If there's a different version to try, please let me know!

Thanks,

- TheBigMan

 

Updated 07/05/2019 with specific error information from the Event Log

Share this post


Link to post
Share on other sites

16 answers to this question

Recommended Posts

  • 0

All,

Working with Microsoft Support on this too...  burned a support incident because I care *that* much :-)

I am looking at the Covecube Disk Enumerator in Device Manager;  I assume this is the driver referenced in the release notes for the v2.2.3.950 beta.  In those notes, it mentions that the driver was updated... see below:

2.2.3.950
---------
* Rebuilt with 17763.134 WDK. Tested and signed for Windows Server 2019.

However, the driver I see on my system is dated 9/29/2017 and is version 2.0.61.0.  As the version of the WDK mentioned in the release notes was the v1809 build released late in 2018, I am assuming that I don't have the updated driver installed (it's signing date should be late 2018 at the earliest)?

My best guess about all this is that the Dedupe service is being a bit stricter in 2019 with it's enumeration of volumes compared with the prior release (2016 a.k.a. 1607), and specifically with the data it's expecting to receive back from the FSCTL_GET_NTFS_VOLUME_DATA call.  I am absolutely NOT a developer, so I apologize in advance if I'm misstating something here.

The thinking here is that we should start with the Covecube driver updated with the latest WDK, which is what you've already gone and done in the latest beta...  but it's not showing up for some reason.

Again, any help you can provide is much appreciated.  Hoping that @Christopher (Drashna) can assist!

Thanks,

- TheBigMan

 

Share this post


Link to post
Share on other sites
  • 0

This is a known issue.  And this only affects Windows Server 2019.  It does not affect any other server version. 

I actually just brought this up to Alex, since it completely breaks deduplication. 

That said, we don't actually support dedup on the pooled drives.   But since this affects more than just the pool.... 

 

Hopefully, Alex will be able to look at this issue soon, and be able to fix it. 

Share this post


Link to post
Share on other sites
  • 0

Thanks @Christopher (Drashna)!  Really appreciate the team looking at this.  I have been very impressed with the responsiveness of your team.

I saw you have a defect tracking system... is there a bug open for this that I can subscribe to or track so that I can get notified on progress?  Does Alex have a rough order of magnitude on getting this fixed (weeks, months, quarters, etc.)?  Just trying to set my expectations accordingly.

I still have a case open with Microsoft tech support.  It's been a frustrating experience as they've essentially been stating that *no* changes were made to the Dedupe role in Server 2019 other than the inclusion of ReFS support.  So obviously there were changes.  I'm trying to get the dev team to at least be able to explain what they changed in the enumeration code that broke all this, i.e. what they're looking for that breaks your code.  Because while I appreciate that ultimately you (Covecube) will need to change your code to adapt to the new environment, Microsoft did not document or even recognize that they'd made a change that would break your product.  If you could let me know whether I should continue pursuing this, I'd be most appreciative.  My guess is that your dev team know what the issue is, so if I should just drop it with Microsoft, I can do so.

BTW - if it helps at all, the *exact* same error occurs with Drive Bender, which is the product I used to use.  So Microsoft's change breaks more than Drive Pool.

I have been very happy with the move to your product suite!  Very well put together and the reliability seems extremely high.  And I say that as a storage/data management professional who works for one of the largest storage/data management companies in the industry...

Thanks,

- TheBigMan

 

Share this post


Link to post
Share on other sites
  • 0

Actually, this is fixed now, and we're just waiting on driver signing to complete. 

* [D] [Issue #28108] Added support for FSCTL_GET_NTFS_VOLUME_DATA.

The issue for this is: https://stablebit.com/Admin/IssueAnalysis/28108

But once you see the installer pop up, you should be able to install it:
http://dl.covecube.com/DrivePoolWindows/beta/download/?C=N;O=D

 

Share this post


Link to post
Share on other sites
  • 0

Was just about to raise this as an issue, as some troubleshooting had led me to believe the Drivepool was involved. I've updated to the 1008 beta, but it appears the similar needs doing for ReFS:
 

[hyper-v1]: PS C:\Users\stuart.admin\Documents> (Get-Item 'C:\Program Files\StableBit\DrivePool\DrivePool.Service.exe').VersionInfo

ProductVersion   FileVersion      FileName
--------------   -----------      --------
2.2.3.1008       2.2.3.1008       C:\Program Files\StableBit\DrivePool\DrivePool.Service.exe

[hyper-v1]: PS C:\Users\stuart.admin\Documents> Get-WinEvent -LogName "Microsoft-Windows-Deduplication/Operational" -MaxEvents 1 | fl


TimeCreated  : 01/08/2019 08:46:59
ProviderName : Microsoft-Windows-Deduplication
Id           : 4105
Message      : Data Deduplication error: Unexpected error.


               Operation:
                  Starting File Server Deduplication Service.

               Error-specific details:
                  Error: DeviceIoControl (FSCTL_GET_REFS_VOLUME_DATA), 0x80070001, Incorrect function.

 

Share this post


Link to post
Share on other sites
  • 0

I downloaded v2.2.3.1012 yesterday, as this issue appears to have been fixed under 28178. Unfortunately I am still unable to use data deduplication:

[hyper-v1]: PS C:\Users\stuart.admin\Documents> Enable-DedupVolume -Volume V -UsageType HyperV
Enable-DedupVolume : MSFT_DedupVolume.Volume='V' - HRESULT 0x80070057, The parameter is incorrect.
    + CategoryInfo          : InvalidArgument: (MSFT_DedupVolume:ROOT/Microsoft/...SFT_DedupVolume) [Enable-DedupVolume], CimException
    + FullyQualifiedErrorId : HRESULT 0x80070057,Enable-DedupVolume


[hyper-v1]: PS C:\Users\stuart.admin\Documents> (Get-Item 'C:\Program Files\StableBit\DrivePool\DrivePool.Service.exe').VersionInfo

ProductVersion   FileVersion      FileName
--------------   -----------      --------
2.2.3.1012       2.2.3.1012       C:\Program Files\StableBit\DrivePool\DrivePool.Service.exe

However, I am no longer getting any errors in the deduplication operational log with the new version:

[hyper-v1]: PS C:\Users\stuart.admin\Documents> get-date

22 August 2019 18:31:44


[hyper-v1]: PS C:\Users\stuart.admin\Documents> Enable-DedupVolume -Volume V -UsageType HyperV
Enable-DedupVolume : MSFT_DedupVolume.Volume='V' - HRESULT 0x80070057, The parameter is incorrect.
    + CategoryInfo          : InvalidArgument: (MSFT_DedupVolume:ROOT/Microsoft/...SFT_DedupVolume) [Enable-DedupVolume], CimException
    + FullyQualifiedErrorId : HRESULT 0x80070057,Enable-DedupVolume

[hyper-v1]: PS C:\Users\stuart.admin\Documents> Get-WinEvent -LogName "Microsoft-Windows-Deduplication/Operational" -MaxEvents 1 | fl


TimeCreated  : 21/08/2019 18:45:01
ProviderName : Microsoft-Windows-Deduplication
Id           : 4105
Message      : Data Deduplication error: Unexpected error.


               Operation:
                  Starting File Server Deduplication Service.

               Error-specific details:
                  Error: DeviceIoControl (FSCTL_GET_REFS_VOLUME_DATA), 0x80070001, Incorrect function.

So it appears something may still be broken. For reference, I can sucessfully reproduce this in a Server 2019 Datacentre VM. I tried a fresh install of 1012 in a new Server 2019 VM, and I get similar results.

Share this post


Link to post
Share on other sites
  • 0

@stuart475898,

By way of another data point, I am not seeing this on my setup, and I have successfully got Dedupe setup on a volume.  Note however that this volume has nothing to do with my Drivepool drives, and I don't use ReFS.  This is WS2019 DC fully patched.

- TheBigMan

Share this post


Link to post
Share on other sites
  • 0

I just installed StableBit DrivePool on a fresh install of Windows Server 2019, utilizing SAS drives in a separate shelf, but I'm still getting this error.  When I look at my version it says Version 2.2.2.934.  I see people mentioning 2.2.3.x in this list, did I not get the most recent version when I downloaded it?  When I try to force the update check, I never get any notification (so I assume that means I'm on the most recent version...) 

Log Name:      Microsoft-Windows-Deduplication/Operational
Source:        Microsoft-Windows-Deduplication
Date:          10/6/2019 8:32:08 AM
Event ID:      4105
Task Category: None
Level:         Error
Keywords:      
User:          SYSTEM
Computer:      XXXXX
Description:
Data Deduplication error: Unexpected error.


Operation:
   Starting File Server Deduplication Service.

Error-specific details:
   Error: DeviceIoControl (FSCTL_GET_NTFS_VOLUME_DATA), 0x80070001, Incorrect function.

Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-Windows-Deduplication" Guid="{f9fe3908-44b8-48d9-9a32-5a763ff5ed79}" />
    <EventID>4105</EventID>
    <Version>0</Version>
    <Level>2</Level>
    <Task>0</Task>
    <Opcode>0</Opcode>
    <Keywords>0x8000000000000000</Keywords>
    <TimeCreated SystemTime="2019-10-06T15:32:08.296713500Z" />
    <EventRecordID>13</EventRecordID>
    <Correlation />
    <Execution ProcessID="13720" ThreadID="2576" />
    <Channel>Microsoft-Windows-Deduplication/Operational</Channel>
    <Computer>PhatServer</Computer>
    <Security UserID="S-1-5-18" />
  </System>
  <EventData>
    <Data Name="Context">

Operation:
   Starting File Server Deduplication Service.

Error-specific details:
   Error: DeviceIoControl (FSCTL_GET_NTFS_VOLUME_DATA), 0x80070001, Incorrect function.
</Data>
    <Data Name="DebugInfo">Code: SRMVOLMC.00002080; Call: SRMVOLMC.00002021; CMD: C:\Windows\system32\svchost -k ddpsvc   ; User: Name: NT AUTHORITY\SYSTEM, SID:S-1-5-18 </Data>
  </EventData>
</Event>

Thanks for any assistance here!

Share this post


Link to post
Share on other sites
  • 0

Oh!  Thanks for that Stuart!  Is there any idea how soon it'll show up for the stable version?  I don't need it immediately, though sooner rather than later would be beneficial...  If I had a rough time frame I could decide if I wanted to try the beta version or just wait.

 

Much appreciated!

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

×
×
  • Create New...