Jump to content
  • 0

Any update on REFS?


thepregnantgod

Question

Recommended Posts

  • 0

So update for all, it seems that the REFS driver uses a lot of memory when moving data onto a REFS drive.

 

I've tested it - and unless you're saying it's hardware specific (i.e. my specific ASUS motherboard or my mouse driver...), on a clean install of Win10 x64, with 96gb of DDR4 and a beta install of Drivepool.

 

When I move 3-4TB of data onto a REFS drive the mem usage climbs all the way up to 96gb.  It resets when I reboot.  And stays steady unless I start moving data to/from a REFS drive.

 

So those folks not seeing memleaks, I suspect it's because they're not moving terabytes of data back/forth from/to REFS drives.

Link to comment
Share on other sites

  • 0

See, that's just the thing, I've moved TB of data around and didn't see massive memory usage.   And by TBs, I mean when I reformatted my entire pool, one/two disks at a time, moving +100TB of data around. And my server stays up for weeks, and I add a LOT of data, especially in batches. 

 

So, this doesn't appear to affect every system, which is why I said it's "system specific".  There are probably a bunch of factors that contribute to the issue, which is probably why Microsoft hasn't issued a hotfix/patch to fix this bug yet. 

Link to comment
Share on other sites

  • 0

Exactly.  And it's this type of system specific issue that is hard to troubleshoot.

 

And being on both ends of this, it's absolutely frustrating.   While I know I haven't experienced it, I know that you and others have. It's definitely an issue.  And ... there isn't anything we really can do about it. 

Link to comment
Share on other sites

  • 0

From my works perspective (Windows 2016, Veeam, 90+TB, Moved to 64k ReFS format, Add 64GB Ram) MS has fixed the issue as long as you use 64k ReFS format, fit as much RAM as possible, and add a registry key or two.

 

https://support.microsoft.com/en-us/help/4016173/fix-heavy-memory-usage-in-refs-on-windows-server-2016-and-windows-10

 

We are using Option 1 registry key only.

Link to comment
Share on other sites

  • 0

For those with Mem Leak issues you may be interested in this Util I wrote - Info over here - https://yabb.jriver.com/interact/index.php/topic,112264.0.html

 

 

 

 

I've had issues on and off over the years with something eating memory till the "In Use" memory approaches Total Installed memory.... and I've noticed it starting again.  Tracking these down can be a PITA and while a reboot fixes it (temporarily) it can be inconvenient if you are doing something.  

I really should rebuild this PC and get off the Insider Previews but for now....Here is a very simple AutoHotKey script that I wrote awhile ago to test the memory when trying to track the leak.  One great side affect is that (for me) it actually also releases this lost memory.  All it does is increase a variables memory allocation and when the app closes the "In Use" memory is now greatly reduced.

Attached is the AHK Script and a compiled 64-Bit Version.
Link to comment
Share on other sites

  • 0

From my works perspective (Windows 2016, Veeam, 90+TB, Moved to 64k ReFS format, Add 64GB Ram) MS has fixed the issue as long as you use 64k ReFS format, fit as much RAM as possible, and add a registry key or two.

 

https://support.microsoft.com/en-us/help/4016173/fix-heavy-memory-usage-in-refs-on-windows-server-2016-and-windows-10

 

We are using Option 1 registry key only.

Thanks - I'll also give this a go.  I'm not running out of memory to the point that Windows halt just very heavy mem usage after a day

Link to comment
Share on other sites

  • 0

I am not 100% but I am pretty sure that ReFS metadata is the problem, this does not show up in task manager as used memory. Look at the animated pink metadata map half way down this page https://forums.veeam.com/veeam-backup-replication-f2/refs-4k-horror-story-t40629-375.html  The MS registry settings relate flushing this data. Use rammap to get an idea of what is actually using your memory.

Link to comment
Share on other sites

  • 0

I've got some big issues on my new Build with the MetaFile.  Even immediately after a cold boot I'm at 95% Memory Usage and the MetaFile has eaten all my memory and just grows!  I've tried some of the recommendations including:

- Disabling "Storage Services"

- Adding to the Registry : Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem\RefsEnableLargeWorkingSetTrim = 1

 

My New Build is:

- Windows 10 Pro - 1709 (16299.19)

- i7-8700K

- ASUS ROG Maximus X Hero MB (latest BIOS 0505)
- Corsair H110i CPU Cooler
- G.Skill Trident Z 32GB (4x8GB) PC4-25600 (3200MHz) DDR4, 16-18-18-38-2N, 1.35v, Quad Channel Kit
- Samsung 512GB SSD, 960 PRO Series, m.2 (PCIE), Read up to 3500MB/s, Write up to 2100MB/s, Type 2280

- Drive Pool 2.2.0.871_x64_Beta pooling 7 x 8TB HDD Refs 3.2 64Bit Clusters (mix of Seagates and HDS)

 

Any suggestions?

Link to comment
Share on other sites

  • 0

So I managed to flush the MetaFile back to a few 100K using my mem util and rebooting (several times).  DP is now "checking" all the files and as it is doing that the MetaFile is growing steadily.  At the Checking 20% mark I've lost 6GB to the MetaFile so far.....and at:

- 25%, 8.5GB

- 50%, 24GB

- it then seemed to "stopped" growing when it hits 26GB!

Link to comment
Share on other sites

  • 0

Can not install the KB - Get a "The update is not applicable to your computer".  I'm guessing this is because "Cumulative Update for Windows 10 Version 1607 for x64 based Systems (KB4013429)" is earlier than the Falls Creator Update version that I had.

Link to comment
Share on other sites

  • 0

No Difference with also adding registry setting #3.  As soon as DP access the pool, Metafile grows.  At present I have 20GB in the Metafile and 6GB in the Mapped File. 

 

Both of these figures however are included in the "Standby" Memory (not active memory) so (from what I understand) can be used as needed by other apps (eg I can flush it down using my mem test app).  If so - does it matter if the Metafile grows to consume all available RAM?  I've been having instability probs with my new build but it may be a co-incidence....

Link to comment
Share on other sites

  • 0

Okay, in that case, could you open a ticket at https://stablebit.com/Contact.

 

And do this: 

http://wiki.covecube.com/StableBit_DrivePool_System_Freeze

 

Make sure that the memory leak is apparent before doing this. 

This will BSOD the system, as that is how it gets the memory dump. 

Link to comment
Share on other sites

  • 0

If it's the ticket/dump that I think it is.... then unfortunately, not really.

 

At least, it's not our code.  It's specifically the "memory mapped files" that are eating up all of the memory.  So, this would definitely be ReFS' fault.  And not a lot we can do.

 

Try limiting balancing, as that could help. 

Otherwise, these disks were formatted on Windows 10, right?

If so, they're probably using the 4k cluster size, and that's contributing to the issue.  Reformatting with the 64k cluster size may help... but that would be tedious....

Link to comment
Share on other sites

  • 0

Too early to call it solved, but I seemed to have fixed the instability by changing my XMP RAM on the new build from G.Skill to Corsair.  So far DP seems to be working well (before it would not load the GUI or the GUI would freeze once the memory had all been allocated).  I also reinstalled DP (this took many attempts of installing / uninstalling various beta versions with windows complaining it could not find the correct MSI - but I got it sorted).  Currently running on 852 instead of 871.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...