Jump to content

  • Log in with Twitter Log in with Windows Live Log In with Google      Sign In   
  • Create Account

Photo

REFS causes memleak?


  • Please log in to reply
14 replies to this topic

#1 thepregnantgod

thepregnantgod

    Advanced Member

  • Members
  • PipPipPip
  • 118 posts

Posted 16 July 2017 - 02:39 PM

Drashna, as you know I've been struggling with the memleak issue so much that I considered switching to Storage Spaces.  

 

However, that product is quite confusing so I asked on the datahoarder subreddit and a user (that sounds knowledgeable) suggested that REFS might be the leak.

 

And...this did start happening on/about the same time I converted to REFS.

 

I'm going to remove each REFS drive individually and then readd as NTFS drives and report back to see if that's the issue.

 

Here's the link.

 

https://www.reddit.c...e_spaces_118tb/

 

 


  • Antoineki likes this

#2 lee1978

lee1978

    Advanced Member

  • Members
  • PipPipPip
  • 199 posts

Posted 16 July 2017 - 03:03 PM

Hi my whole pool is refs and I don't have any memory leaks not to say it isn't the cause of yours but every bit of information helps

#3 thepregnantgod

thepregnantgod

    Advanced Member

  • Members
  • PipPipPip
  • 118 posts

Posted 16 July 2017 - 04:13 PM

Another link for reading.  To lee1978 (what OS and cluster size are you using)?

 

https://www.reddit.c...fs_file_system/

 

Here it's REFS leaking in Storage spaces.  Guess MS recognized it then patched it but some claim patch didn't work.

 

Another poster (fan of Drivepool) was seeing the same thing I am but identified it as subsystem of storage spaces?



#4 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 7,806 posts
  • LocationSan Diego, CA, USA

Posted 16 July 2017 - 04:54 PM

Well, to be honest, I've been having some memory related issues, and it may be ReFS related, as well.   Not as bad as what you're reporting, but much more inconsistent.  

 

So it may very well be that ReFS is having issues still.  It is a very new file system, so it's not entirely unexpected. 

 

 

 

That said, I saw the post on reddit before checking here, actually. :)

 

If you do wish to migrate away for storage spaces, grab the latest version. There has been a recent overhaul to the removal code, and that may make things easier. 

 

 

Additionally, using the "Drive usage limiter" balancer may help, but it may also trigger the issue to occur, as well.   if you have a backup of the pool, then just "nuking it" and reformatting as NTFS would be the best solution.  

 

Otherwise, slowly converting it should work (albeit, with difficulties).  Just don't forget to do this:

http://wiki.covecube...vanced_Settings

Set "CoveFs_AllowMixedFilesystems" to "True", to allow mixing NTFS and ReFS in a pool. 


Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.


#5 thepregnantgod

thepregnantgod

    Advanced Member

  • Members
  • PipPipPip
  • 118 posts

Posted 16 July 2017 - 05:49 PM

Drashna, just to keep you up to date, I'm at work now, remoting into my system.  I stopped and disabled the "storage service" service and I'm 5 hours into removing a drive from the pool and have kept steady at 25gb RAM (I started it at 25gb RAM).

 

Meaning, that could be one variable in my issue.



#6 lee1978

lee1978

    Advanced Member

  • Members
  • PipPipPip
  • 199 posts

Posted 16 July 2017 - 07:18 PM

My drives are 64k clusters under server 2012 es r2 I also have a storage space pool which is an exact copy of my drivepool both pools are 40tb and everything is running fine My test rig is running server 2016 and I have been swapping my storage spaces pool between both systems with no problems. My server also has 16gb ram installed.

Software installed:

Server 2012 r2
Drivepool
Scanner
Ihomeserver
Plex
Crashplan
iTunes
Lights out

#7 jmone

jmone

    Advanced Member

  • Members
  • PipPipPip
  • 104 posts

Posted 16 July 2017 - 08:27 PM

No leak here either on either Pools (Win10 and WS2012R2 Essentials) and I'm hammering the Win10 pool as I convert disks one at a time from REFS V1.2 to REFS V3.2



#8 thepregnantgod

thepregnantgod

    Advanced Member

  • Members
  • PipPipPip
  • 118 posts

Posted 17 July 2017 - 02:38 AM

Drashna, sorry you guys are taking heat on reddit.  That was not my intention.

 

I simply want it fixed as a few family members use my Plex server (driven off of drivepool) and my system is maxing mem every 3-4hrs.  Which often means their movie gets paused until I can reboot server.

 

I tried the steps to cause a memdump (even with a pagefile) and it did not work.

 

Hence the slow process of removing all REFS drives.  

 

Disabling the storage service seemingly worked but...alas...it didn't.  I'm sitting at 63gb of RAM used right now.



#9 browned

browned

    Member

  • Members
  • PipPip
  • 18 posts

Posted 17 July 2017 - 03:21 AM

This may be related.

 

https://forums.veeam...ory-t40629.html

 

There are a couple of fixes for 2016 Server and ReFS in there, also some registry settings. Sorry 30+ pages to read through.



#10 lee1978

lee1978

    Advanced Member

  • Members
  • PipPipPip
  • 199 posts

Posted 17 July 2017 - 06:10 AM

Have you run any updates since this started Microsoft updates are Notorious For breaking stuff my pool's are only 40tb the links provided seem to point at larger pools so I think jmone has around the same as you so he might be better to try help. failing that why not try a fresh install of the OS

#11 jmone

jmone

    Advanced Member

  • Members
  • PipPipPip
  • 104 posts

Posted 17 July 2017 - 07:38 AM

My Pools are 50TB as well, and they have been rock solid for 18mnths.... but I was having all sorts of memory leak (back in my DriveBender days).  I never really could track it down, but it was either DriveBender, ReFS.sys , LSI Driver, or the Dual Intel NIC card (in teaming mode).  Memory would just disappear when the pool was being used.  I posted all over the place at the time but as new versions of software, drivers and Windows Updates were released it just stopped.  Sorry to be so vague.



#12 jmone

jmone

    Advanced Member

  • Members
  • PipPipPip
  • 104 posts

Posted 17 July 2017 - 07:53 AM

Here is the link I posted on the Intel PRO/1000PT leaking memory.  Got rid of the card and the leak at the time.



#13 lee1978

lee1978

    Advanced Member

  • Members
  • PipPipPip
  • 199 posts

Posted 17 July 2017 - 08:52 AM

I believe the pregnant god had migrated from drivebender to drivepool I wonder if there any files left over from drivebender that could be causing the problem my setup was a fresh install and like you has been stable for well over 3 years now not even a blip I don't let windows auto update anymore I personally check each update and make sure I can rollback if they cause issues windows auto update is the devil in disguise

#14 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 7,806 posts
  • LocationSan Diego, CA, USA

Posted 17 July 2017 - 07:08 PM

Drashna, sorry you guys are taking heat on reddit.  That was not my intention.

 

I simply want it fixed as a few family members use my Plex server (driven off of drivepool) and my system is maxing mem every 3-4hrs.  Which often means their movie gets paused until I can reboot server.

 

I tried the steps to cause a memdump (even with a pagefile) and it did not work.

 

Hence the slow process of removing all REFS drives.  

 

Disabling the storage service seemingly worked but...alas...it didn't.  I'm sitting at 63gb of RAM used right now.

I wouldn't really call that heat.  There are some good points... but ... not really "heat", IMO.  

 

(well, rather, we get asked the whole "are you still around" question periodically.  Because, yeah, it's been 2+ years since a release of DrivePool.......

 

And very, very odd on the memory dump.  

 

This may be related.

 

https://forums.veeam...ory-t40629.html

 

There are a couple of fixes for 2016 Server and ReFS in there, also some registry settings. Sorry 30+ pages to read through.

I'm going to have to do some reading then.... 

 

My Pools are 50TB as well, and they have been rock solid for 18mnths.... but I was having all sorts of memory leak (back in my DriveBender days).  I never really could track it down, but it was either DriveBender, ReFS.sys , LSI Driver, or the Dual Intel NIC card (in teaming mode).  Memory would just disappear when the pool was being used.  I posted all over the place at the time but as new versions of software, drivers and Windows Updates were released it just stopped.  Sorry to be so vague.

Yeah, memory leaks can be complicated things to track down. Especially if you're not well versed in how to read them, or have access to the code for them. 

 

 

 

I believe the pregnant god had migrated from drivebender to drivepool I wonder if there any files left over from drivebender that could be causing the problem my setup was a fresh install and like you has been stable for well over 3 years now not even a blip I don't let windows auto update anymore I personally check each update and make sure I can rollback if they cause issues windows auto update is the devil in disguise

 

Nope.  If Drive Bender isn't installed anymore, then it wouldn't be doing anything. 

 

The file layout, and contents won't matter or cause any issues for our software. Nor would it cause issues with the file system. 

 

Basically, once the software is removed, there is no way it could continue to cause issues .... unless the driver wasn't fully removed (not common). 


Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.


#15 Christopher (Drashna)

Christopher (Drashna)

    Customer and Technical Support

  • Administrators
  • 7,806 posts
  • LocationSan Diego, CA, USA

Posted 17 July 2017 - 07:30 PM

Also, for the memory dump failing, this may be because the page file is too small.  

 

Bevause of the memory leak, you may need a 60+ GB page file..... which is shitty.....


Christopher Courtney

aka "Drashna"

Microsoft MVP for Windows Home Server 2009-2012

Lead Moderator for We Got Served

Moderator for Home Server Show

 

This is my server

 

Lots of "Other" data on your pool? Read about what it is here.





0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users