Memory Leak in 2.2.0.906?

[MStark]

Hi,

I'm taking a stab in the dark.  I recently noticed that one of my server's will run out of memory in the course of a day or so -- the user processes don't account for the memory consumption.  Looking at the system memory, a vast majority of the memory is being used by the tag "COFS".  I am not certain, but I believe that this started occurring shortly after I updated to the latest version of DrivePool on the server -- does the tag COFS correspond to the drive pool software?  If so, I will take the next steps to provide any information required to help debug the issue.

Memory:16759896K Avail:  862048K  PageFlts:271809   InRam Krnl: 7660K P:1567224K
 Commit:19431332K Limit:24624216K Peak:23519472K            Pool N:11019488K P:4192236K
 System pool information
 

Tag  Type     Allocs            Frees            Diff       Bytes                  Per Alloc

COFS Nonp  589885353 (110005) 515819158 (90272) 74066195  9480643840 (    2585504)         128
 FSel Paged  73998876 (19694)      2174 (   0) 73996702  3551841696 (     945312)          48
 EtwB Nonp       2682 (   0)      1670 (   0)     1012   105332768 (          0)      104083

[MStark]

Upon some further research, I think this tag definitely corresponds to DrivePool.  Tracking the TAG through the drivers folder yields the file:

C:\Windows\system32\drivers\covefs.sys

This is the only system file which contains the string "COFS" in it (and it is found numerous times).

The version of the file found on my system is 2.0.345.0 and is signed by both CoveCube and "Microsoft Hardware Compatibility Publisher".

[Christopher (Drashna)]

Yes, the tag corresponds with the CoveFS driver. 

As for a memory leak, I'm not sure about that.  We've had people occasionally (like once every six months or more) reporting something like this.

However, this is a pretty complicated issue.  It may be something as simple as a shitload of files being opened on the pool.  Each open file has to de tracked, and that takes up resources. But this should be rather efficient. 

Also, other drivers (especially malware protection and other disk utilities) can cause issues too. 

 

That said, if you could grab a memory dump of the system when you're seeing this and submit it, it may help us to identifying what is going on. 
http://wiki.covecube.com/StableBit_DrivePool_System_Freeze

[MStark]

Thanks for the reply.  You would have to know the context for the server, but I can confidently say that very few files are open at any given time, and I can also rule out malware scanning as a cause.

I played around with this some more last night, after I realized I have another server configured almost identically (save for this one being older) which was not exhibiting this problem.  I tried doing the same operations on both servers and noticed that quite a bit more functionality was wrong than just the memory consumption (couldn't restart the DrivePool service, couldn't change some configuration, couldn't reset configuration, etc.).  This led me to try and completely nuke the DrivePool installation on the symptomatic machine (even uninstall didn't successfully remove the service).  Afterwards I reinstalled it, and as of this morning, it is running at the lowest memory consumption since I updated the version a couple weeks ago.  This was 12 hours after the last reboot, where the machine has continuously been exhausting the RAM on the machine in about 24 hours.

I'll continue to monitor, but assuming this trend holds then I would assume that the problem was due to some setting migration or incomplete version upgrade at some point along the way.  I had previously tried the repair option earlier this week and that was not effective, but you could determine that almost immediately after reboot.

I'll collect the dump for you if the problem reoccurs over the weekend.

Thanks,
Mike

[Christopher (Drashna)]

That's very odd! 

If this is the case, again, it may be very helpful to run the StableBit Troubleshooter before resetting the settings or uninstalling the software. 
http://wiki.covecube.com/StableBit_Troubleshooter

 

Hopefully, though, that was the issue, and it continues to run well. 

[cechk01]

I saw a similar issue where the drivepool service was using 1.5G of ram.

[Christopher (Drashna)]

4 hours ago, cechk01 said:

I saw a similar issue where the drivepool service was using 1.5G of ram.

That's a very (completely) different issue.  

Was the software balancing or duplicating at the time?