thepregnantgod Posted August 30, 2017 Share Posted August 30, 2017 Anyone find out if the REFS driver still causes a Memleak? Quote Link to comment Share on other sites More sharing options...
0 jmone Posted August 31, 2017 Share Posted August 31, 2017 I'm not seeing any mem leak on Win 10 or Win Server 2012R2 Essentials Quote Link to comment Share on other sites More sharing options...
0 Christopher (Drashna) Posted August 31, 2017 Share Posted August 31, 2017 It's very system specific. I'm not seeing it on my system, and I have 20+ ReFS formatted drives. And I've not seen any indications in patches for a fix. So it may be a while Quote Link to comment Share on other sites More sharing options...
0 thepregnantgod Posted September 1, 2017 Author Share Posted September 1, 2017 So update for all, it seems that the REFS driver uses a lot of memory when moving data onto a REFS drive. I've tested it - and unless you're saying it's hardware specific (i.e. my specific ASUS motherboard or my mouse driver...), on a clean install of Win10 x64, with 96gb of DDR4 and a beta install of Drivepool. When I move 3-4TB of data onto a REFS drive the mem usage climbs all the way up to 96gb. It resets when I reboot. And stays steady unless I start moving data to/from a REFS drive. So those folks not seeing memleaks, I suspect it's because they're not moving terabytes of data back/forth from/to REFS drives. Quote Link to comment Share on other sites More sharing options...
0 Christopher (Drashna) Posted September 1, 2017 Share Posted September 1, 2017 See, that's just the thing, I've moved TB of data around and didn't see massive memory usage. And by TBs, I mean when I reformatted my entire pool, one/two disks at a time, moving +100TB of data around. And my server stays up for weeks, and I add a LOT of data, especially in batches. So, this doesn't appear to affect every system, which is why I said it's "system specific". There are probably a bunch of factors that contribute to the issue, which is probably why Microsoft hasn't issued a hotfix/patch to fix this bug yet. Quote Link to comment Share on other sites More sharing options...
0 jmone Posted September 1, 2017 Share Posted September 1, 2017 Same for me. Reformatted my 50TB Win10 ReFS pool to V3.2 one drive at a time and moved the data around just fine. Quote Link to comment Share on other sites More sharing options...
0 Christopher (Drashna) Posted September 2, 2017 Share Posted September 2, 2017 Exactly. And it's this type of system specific issue that is hard to troubleshoot. And being on both ends of this, it's absolutely frustrating. While I know I haven't experienced it, I know that you and others have. It's definitely an issue. And ... there isn't anything we really can do about it. Quote Link to comment Share on other sites More sharing options...
0 browned Posted September 4, 2017 Share Posted September 4, 2017 From my works perspective (Windows 2016, Veeam, 90+TB, Moved to 64k ReFS format, Add 64GB Ram) MS has fixed the issue as long as you use 64k ReFS format, fit as much RAM as possible, and add a registry key or two. https://support.microsoft.com/en-us/help/4016173/fix-heavy-memory-usage-in-refs-on-windows-server-2016-and-windows-10 We are using Option 1 registry key only. jmone 1 Quote Link to comment Share on other sites More sharing options...
0 jmone Posted September 14, 2017 Share Posted September 14, 2017 For those with Mem Leak issues you may be interested in this Util I wrote - Info over here - https://yabb.jriver.com/interact/index.php/topic,112264.0.html I've had issues on and off over the years with something eating memory till the "In Use" memory approaches Total Installed memory.... and I've noticed it starting again. Tracking these down can be a PITA and while a reboot fixes it (temporarily) it can be inconvenient if you are doing something. I really should rebuild this PC and get off the Insider Previews but for now....Here is a very simple AutoHotKey script that I wrote awhile ago to test the memory when trying to track the leak. One great side affect is that (for me) it actually also releases this lost memory. All it does is increase a variables memory allocation and when the app closes the "In Use" memory is now greatly reduced.Attached is the AHK Script and a compiled 64-Bit Version. Quote Link to comment Share on other sites More sharing options...
0 jmone Posted September 14, 2017 Share Posted September 14, 2017 From my works perspective (Windows 2016, Veeam, 90+TB, Moved to 64k ReFS format, Add 64GB Ram) MS has fixed the issue as long as you use 64k ReFS format, fit as much RAM as possible, and add a registry key or two. https://support.microsoft.com/en-us/help/4016173/fix-heavy-memory-usage-in-refs-on-windows-server-2016-and-windows-10 We are using Option 1 registry key only. Thanks - I'll also give this a go. I'm not running out of memory to the point that Windows halt just very heavy mem usage after a day Christopher (Drashna) 1 Quote Link to comment Share on other sites More sharing options...
0 browned Posted September 15, 2017 Share Posted September 15, 2017 I am not 100% but I am pretty sure that ReFS metadata is the problem, this does not show up in task manager as used memory. Look at the animated pink metadata map half way down this page https://forums.veeam.com/veeam-backup-replication-f2/refs-4k-horror-story-t40629-375.html The MS registry settings relate flushing this data. Use rammap to get an idea of what is actually using your memory. Christopher (Drashna) 1 Quote Link to comment Share on other sites More sharing options...
0 jmone Posted October 26, 2017 Share Posted October 26, 2017 I've got some big issues on my new Build with the MetaFile. Even immediately after a cold boot I'm at 95% Memory Usage and the MetaFile has eaten all my memory and just grows! I've tried some of the recommendations including: - Disabling "Storage Services" - Adding to the Registry : Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem\RefsEnableLargeWorkingSetTrim = 1 My New Build is: - Windows 10 Pro - 1709 (16299.19) - i7-8700K - ASUS ROG Maximus X Hero MB (latest BIOS 0505)- Corsair H110i CPU Cooler- G.Skill Trident Z 32GB (4x8GB) PC4-25600 (3200MHz) DDR4, 16-18-18-38-2N, 1.35v, Quad Channel Kit- Samsung 512GB SSD, 960 PRO Series, m.2 (PCIE), Read up to 3500MB/s, Write up to 2100MB/s, Type 2280 - Drive Pool 2.2.0.871_x64_Beta pooling 7 x 8TB HDD Refs 3.2 64Bit Clusters (mix of Seagates and HDS) Any suggestions? Quote Link to comment Share on other sites More sharing options...
0 jmone Posted October 26, 2017 Share Posted October 26, 2017 So I managed to flush the MetaFile back to a few 100K using my mem util and rebooting (several times). DP is now "checking" all the files and as it is doing that the MetaFile is growing steadily. At the Checking 20% mark I've lost 6GB to the MetaFile so far.....and at: - 25%, 8.5GB - 50%, 24GB - it then seemed to "stopped" growing when it hits 26GB! Christopher (Drashna) 1 Quote Link to comment Share on other sites More sharing options...
0 Christopher (Drashna) Posted October 26, 2017 Share Posted October 26, 2017 Make sure that you have this update installed: http://www.catalog.update.microsoft.com/Search.aspx?q=4013429 And try setting options 1 and 2: https://support.microsoft.com/en-us/help/4016173/fix-heavy-memory-usage-in-refs-on-windows-server-2016-and-windows-10 Quote Link to comment Share on other sites More sharing options...
0 browned Posted October 26, 2017 Share Posted October 26, 2017 I believe every time an reFS system is booted the file system is checked with some scheduled tasks. Might need to disable them as some others have, but you have to think if MS set them up and enabled them they are important and should be run at some stage. Quote Link to comment Share on other sites More sharing options...
0 jmone Posted October 27, 2017 Share Posted October 27, 2017 Make sure that you have this update installed: http://www.catalog.update.microsoft.com/Search.aspx?q=4013429 And try setting options 1 and 2: https://support.microsoft.com/en-us/help/4016173/fix-heavy-memory-usage-in-refs-on-windows-server-2016-and-windows-10 Can not install the KB - Get a "The update is not applicable to your computer". I'm guessing this is because "Cumulative Update for Windows 10 Version 1607 for x64 based Systems (KB4013429)" is earlier than the Falls Creator Update version that I had. Quote Link to comment Share on other sites More sharing options...
0 Christopher (Drashna) Posted October 27, 2017 Share Posted October 27, 2017 Ah yes, if you have CU or FCU, then you should have this hotfix installed already. However, you'd still need to make the registry tweaks. Quote Link to comment Share on other sites More sharing options...
0 jmone Posted October 27, 2017 Share Posted October 27, 2017 I've tried 1 and 2 (with a setting of 32). No difference - 26GB in Metafile used. Will try 3 once DP has finished it current "Checking" phase. Quote Link to comment Share on other sites More sharing options...
0 jmone Posted October 28, 2017 Share Posted October 28, 2017 No Difference with also adding registry setting #3. As soon as DP access the pool, Metafile grows. At present I have 20GB in the Metafile and 6GB in the Mapped File. Both of these figures however are included in the "Standby" Memory (not active memory) so (from what I understand) can be used as needed by other apps (eg I can flush it down using my mem test app). If so - does it matter if the Metafile grows to consume all available RAM? I've been having instability probs with my new build but it may be a co-incidence.... Quote Link to comment Share on other sites More sharing options...
0 Christopher (Drashna) Posted October 28, 2017 Share Posted October 28, 2017 Okay, in that case, could you open a ticket at https://stablebit.com/Contact. And do this: http://wiki.covecube.com/StableBit_DrivePool_System_Freeze Make sure that the memory leak is apparent before doing this. This will BSOD the system, as that is how it gets the memory dump. Quote Link to comment Share on other sites More sharing options...
0 jmone Posted October 29, 2017 Share Posted October 29, 2017 Job logged and Dump File sent. Here is a pic of my RAMMap. Quote Link to comment Share on other sites More sharing options...
0 jmone Posted October 31, 2017 Share Posted October 31, 2017 I'm really hoping you find something as my write speed to the pool is appalling (ends up going to 1KB/Sec after a time unless I reboot).... read speeds are OK. Quote Link to comment Share on other sites More sharing options...
0 Christopher (Drashna) Posted October 31, 2017 Share Posted October 31, 2017 If it's the ticket/dump that I think it is.... then unfortunately, not really. At least, it's not our code. It's specifically the "memory mapped files" that are eating up all of the memory. So, this would definitely be ReFS' fault. And not a lot we can do. Try limiting balancing, as that could help. Otherwise, these disks were formatted on Windows 10, right? If so, they're probably using the 4k cluster size, and that's contributing to the issue. Reformatting with the 64k cluster size may help... but that would be tedious.... Quote Link to comment Share on other sites More sharing options...
0 jmone Posted October 31, 2017 Share Posted October 31, 2017 I've already migrated all the drives over to ReFS V3 / 64Bit / Check On. I'll have to have a think on what to do and why my new build is different to the old one. Quote Link to comment Share on other sites More sharing options...
0 jmone Posted November 1, 2017 Share Posted November 1, 2017 ... also sometimes it the "Mapped File" that grows Quote Link to comment Share on other sites More sharing options...
0 jmone Posted November 3, 2017 Share Posted November 3, 2017 Too early to call it solved, but I seemed to have fixed the instability by changing my XMP RAM on the new build from G.Skill to Corsair. So far DP seems to be working well (before it would not load the GUI or the GUI would freeze once the memory had all been allocated). I also reinstalled DP (this took many attempts of installing / uninstalling various beta versions with windows complaining it could not find the correct MSI - but I got it sorted). Currently running on 852 instead of 871. Quote Link to comment Share on other sites More sharing options...
Question
thepregnantgod
Anyone find out if the REFS driver still causes a Memleak?
Link to comment
Share on other sites
26 answers to this question
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.