Jump to content
  • 0

REFS causes memleak?


thepregnantgod

Question

Drashna, as you know I've been struggling with the memleak issue so much that I considered switching to Storage Spaces.  

 

However, that product is quite confusing so I asked on the datahoarder subreddit and a user (that sounds knowledgeable) suggested that REFS might be the leak.

 

And...this did start happening on/about the same time I converted to REFS.

 

I'm going to remove each REFS drive individually and then readd as NTFS drives and report back to see if that's the issue.

 

Here's the link.

 

https://www.reddit.com/r/DataHoarder/comments/6nlzwr/how_to_setup_up_windows_10_storage_spaces_118tb/

 

 

Link to comment
Share on other sites

16 answers to this question

Recommended Posts

  • 0

Another link for reading.  To lee1978 (what OS and cluster size are you using)?

 

https://www.reddit.com/r/DataHoarder/comments/6f00re/storage_spaces_memory_leak_refs_file_system/

 

Here it's REFS leaking in Storage spaces.  Guess MS recognized it then patched it but some claim patch didn't work.

 

Another poster (fan of Drivepool) was seeing the same thing I am but identified it as subsystem of storage spaces?

Link to comment
Share on other sites

  • 0

Well, to be honest, I've been having some memory related issues, and it may be ReFS related, as well.   Not as bad as what you're reporting, but much more inconsistent.  

 

So it may very well be that ReFS is having issues still.  It is a very new file system, so it's not entirely unexpected. 

 

 

 

That said, I saw the post on reddit before checking here, actually. :)

 

If you do wish to migrate away for storage spaces, grab the latest version. There has been a recent overhaul to the removal code, and that may make things easier. 

 

 

Additionally, using the "Drive usage limiter" balancer may help, but it may also trigger the issue to occur, as well.   if you have a backup of the pool, then just "nuking it" and reformatting as NTFS would be the best solution.  

 

Otherwise, slowly converting it should work (albeit, with difficulties).  Just don't forget to do this:

http://wiki.covecube.com/StableBit_DrivePool_2.x_Advanced_Settings

Set "CoveFs_AllowMixedFilesystems" to "True", to allow mixing NTFS and ReFS in a pool. 

Link to comment
Share on other sites

  • 0

Drashna, just to keep you up to date, I'm at work now, remoting into my system.  I stopped and disabled the "storage service" service and I'm 5 hours into removing a drive from the pool and have kept steady at 25gb RAM (I started it at 25gb RAM).

 

Meaning, that could be one variable in my issue.

Link to comment
Share on other sites

  • 0

My drives are 64k clusters under server 2012 es r2 I also have a storage space pool which is an exact copy of my drivepool both pools are 40tb and everything is running fine My test rig is running server 2016 and I have been swapping my storage spaces pool between both systems with no problems. My server also has 16gb ram installed.

 

Software installed:

 

Server 2012 r2

Drivepool

Scanner

Ihomeserver

Plex

Crashplan

iTunes

Lights out

Link to comment
Share on other sites

  • 0

Drashna, sorry you guys are taking heat on reddit.  That was not my intention.

 

I simply want it fixed as a few family members use my Plex server (driven off of drivepool) and my system is maxing mem every 3-4hrs.  Which often means their movie gets paused until I can reboot server.

 

I tried the steps to cause a memdump (even with a pagefile) and it did not work.

 

Hence the slow process of removing all REFS drives.  

 

Disabling the storage service seemingly worked but...alas...it didn't.  I'm sitting at 63gb of RAM used right now.

Link to comment
Share on other sites

  • 0

Have you run any updates since this started Microsoft updates are Notorious For breaking stuff my pool's are only 40tb the links provided seem to point at larger pools so I think jmone has around the same as you so he might be better to try help. failing that why not try a fresh install of the OS

Link to comment
Share on other sites

  • 0

My Pools are 50TB as well, and they have been rock solid for 18mnths.... but I was having all sorts of memory leak (back in my DriveBender days).  I never really could track it down, but it was either DriveBender, ReFS.sys , LSI Driver, or the Dual Intel NIC card (in teaming mode).  Memory would just disappear when the pool was being used.  I posted all over the place at the time but as new versions of software, drivers and Windows Updates were released it just stopped.  Sorry to be so vague.

Link to comment
Share on other sites

  • 0

I believe the pregnant god had migrated from drivebender to drivepool I wonder if there any files left over from drivebender that could be causing the problem my setup was a fresh install and like you has been stable for well over 3 years now not even a blip I don't let windows auto update anymore I personally check each update and make sure I can rollback if they cause issues windows auto update is the devil in disguise

Link to comment
Share on other sites

  • 0

Drashna, sorry you guys are taking heat on reddit.  That was not my intention.

 

I simply want it fixed as a few family members use my Plex server (driven off of drivepool) and my system is maxing mem every 3-4hrs.  Which often means their movie gets paused until I can reboot server.

 

I tried the steps to cause a memdump (even with a pagefile) and it did not work.

 

Hence the slow process of removing all REFS drives.  

 

Disabling the storage service seemingly worked but...alas...it didn't.  I'm sitting at 63gb of RAM used right now.

I wouldn't really call that heat.  There are some good points... but ... not really "heat", IMO.  

 

(well, rather, we get asked the whole "are you still around" question periodically.  Because, yeah, it's been 2+ years since a release of DrivePool.......

 

And very, very odd on the memory dump.  

 

This may be related.

 

https://forums.veeam.com/veeam-backup-replication-f2/refs-4k-horror-story-t40629.html

 

There are a couple of fixes for 2016 Server and ReFS in there, also some registry settings. Sorry 30+ pages to read through.

I'm going to have to do some reading then.... 

 

My Pools are 50TB as well, and they have been rock solid for 18mnths.... but I was having all sorts of memory leak (back in my DriveBender days).  I never really could track it down, but it was either DriveBender, ReFS.sys , LSI Driver, or the Dual Intel NIC card (in teaming mode).  Memory would just disappear when the pool was being used.  I posted all over the place at the time but as new versions of software, drivers and Windows Updates were released it just stopped.  Sorry to be so vague.

Yeah, memory leaks can be complicated things to track down. Especially if you're not well versed in how to read them, or have access to the code for them. 

 

 

 

I believe the pregnant god had migrated from drivebender to drivepool I wonder if there any files left over from drivebender that could be causing the problem my setup was a fresh install and like you has been stable for well over 3 years now not even a blip I don't let windows auto update anymore I personally check each update and make sure I can rollback if they cause issues windows auto update is the devil in disguise

 

Nope.  If Drive Bender isn't installed anymore, then it wouldn't be doing anything. 

 

The file layout, and contents won't matter or cause any issues for our software. Nor would it cause issues with the file system. 

 

Basically, once the software is removed, there is no way it could continue to cause issues .... unless the driver wasn't fully removed (not common). 

Link to comment
Share on other sites

  • 0

Just a followup, Drashna, I noticed that Windows removed REFS from the optional formats.

 

I suspected it was your program that was causing my memleak into a softlock.  But, it was actually REFS when do a lot of balancing file/moving.  

 

For other readers, I have 34 drives in a single pool, duplicated.  I was in the process of removing a drive, formatting it in REFS and then readding it.  I did this about 13 times before I realized I was softlocking due to memory exhaustion.  (It's usually an unstable overclock).

 

I finally removed all the REFS drives and now maintain steady despite a lot of file movement/balancing around 16gb used total.

 

My money is that REFS was the culprit.

Link to comment
Share on other sites

  • 0

Yeah, we are pretty sure that this is the case, as well.

 

Though, as for ReFS being removed, that may have multiple reasons for it .... Such as the Windows 10 Pro for Workstations SKU that does include ReFS support....

 

Though, since you definitely weren't the only one seeing this (either the ReFS issue, nor the DrivePool+ReFS issue).... 

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...