Jump to content
  • 0

Bypass NTFS Filters + Disable Buffer Flushing


B00ze

Question

Good day.

Sorry to flood the forum with questions.

My first questions is about BypassNtfsFilters. The WIKI says that this is True by default, but it is in fact (nowadays) False by default. Have you found lots of programs that NEED this to be True? Because I'm thinking of getting a good write-back cache program, and it uses a filter to cache stuff, which means I would not be able to bypass. What about A/V? If we bypass the NTFS filters, will that not disable protection on those drives?

My second question is about Window's "Disable Buffer Flushing" option. The internet is full of different opinions on this. Like I said above, I want to improve write-caching. Anyone know if that option in Windows will do this? See, I come from having tested an Intel built-in RAID, and to enable write-back caching we need to disable "Buffer Flushing." The difference is huge; the Intel RAID has horrible write performance without the write-back cache, so I am kinda sensitive to this subject right now.

Thank you all.

Link to comment
Share on other sites

14 answers to this question

Recommended Posts

  • 0

Well, I finally found a good explanation of that Buffer Flushing setting: 
https://blogs.msdn.microsoft.com/oldnewthing/20100909-00/?p=12913/

And I ran a real quick robocopy test; disabling Buffer Flushing saved ~10 seconds on a ~180 seconds copy, not exactly a superstellar improvement (I guess it is within the margin of error, meaning it might not have improved anything at all.)

Regards,

Link to comment
Share on other sites

  • 0
14 hours ago, B00ze said:

My first questions is about BypassNtfsFilters. The WIKI says that this is True by default, but it is in fact (nowadays) False by default. Have you found lots of programs that NEED this to be True? Because I'm thinking of getting a good write-back cache program, and it uses a filter to cache stuff, which means I would not be able to bypass. What about A/V? If we bypass the NTFS filters, will that not disable protection on those drives?

Hmmm, I don't remember when we changed that.  It should be enabled by default. 

Some AV programs need it disabled.  When accessing the file on the pool, it attemps to scan it.. but then when we go get the actual file, it sees the same file, and blocks it from being accessed until the first is completed. but the first won't complete, because it's a "proxy request" to the second. 
This is pretty rare, but ... that it's happened at all...

But it's also a performance hit to have the filters check each time, especially when AV is concerned. 

The only time it really should be disabled, is when using stuff like data deduplication which uses a file system filter to splice data back together. Or some encryption software. 

14 hours ago, B00ze said:

My second question is about Window's "Disable Buffer Flushing" option. The internet is full of different opinions on this. Like I said above, I want to improve write-caching. Anyone know if that option in Windows will do this? See, I come from having tested an Intel built-in RAID, and to enable write-back caching we need to disable "Buffer Flushing." The difference is huge; the Intel RAID has horrible write performance without the write-back cache, so I am kinda sensitive to this subject right now.

Honestly, this doesn't matter unless your disk is SAS.  Because otherwise, your disk is probably lying. 

Link to comment
Share on other sites

  • 0

Good day.

BypassNtfsFilters is False by default @ least in the version I installed, which is 2.2.0.890 (beta). I'll leave it disabled, I'm going to download to the pool and I need those files scanned by the Anti-Virus as they are opened/closed. I'll come back if it becomes a problem.

  "CoveFs_BypassNtfsFilters": {
    "Default": false,
    "Override": null }

As for that Buffer Flushing thing, yeah, the link I posted above has plenty of comments from people saying hard drives are lying anyway.

Best Regards,

Link to comment
Share on other sites

  • 0

Yeah, I've checked with Alex (the dev).  This was changed at around 750, for compatibility reasons with some reparse point code/handling. 

If you don't see any issues, then definitely leave it, as is.

16 hours ago, B00ze said:

As for that Buffer Flushing thing, yeah, the link I posted above has plenty of comments from people saying hard drives are lying anyway.

Yup, and even SQLite has commented about this. 

https://www.sqlite.org/howtocorrupt.html

Sections 3 and 4 cover this in some detail. 

Link to comment
Share on other sites

  • 0

Hi Christopher.

Quote

Yeah, I've checked with Alex (the dev).  This was changed at around 750, for compatibility reasons with some reparse point code/handling.  If you don't see any issues, then definitely leave it, as is.

Indeed, one of the reasons I went with DrivePool is the awesome support for reparse points (too bad Plex uses hardlinks). Plus I do want stuff scanned by the A/V on the pool, so unless I run into race conditions I'll definitively leave this as is! Thanks for checking!

Quote

Yup, and even SQLite has commented about this.

Yeah, I'll disable that Buffer-Flushing thing on the pool, I will gain milliseconds (maybe) and it doesn't really matter anyway! Even BSODs won't crash the drive because of this (they WILL crash the drive because Windows has write-cache, but not because of buffer flushing) so why not, I got a big UPS anyway.

Best Regards,

Edited by B00ze
EDIT: Leave AS-IS, i.e. Disabled...
Link to comment
Share on other sites

  • 0

Thanks! 

And yeah, I don't know why Plex uses the file system to hard link stuff, rather than it's own database to index and reuse files.  It would make things easier, for sure. 

And yeah, as long as you have a good UPS... :) 
Just make sure you test the batteries periodically ("patiently awaiting replacement batteries for my UPS). 

Link to comment
Share on other sites

  • 0

I went with an APC BR1500; I only have the one PC so I do not need an industrial one like you lol. And APC batteries are easy to get on Amazon. APC's aren't that good, and their software is really bad, but because they are popular you can find batteries everywhere.

Link to comment
Share on other sites

  • 0

Well, mine's not that much bigger that that, actually. However, mine's Active PFC rated (true sine wave, not stepped/approximated/simulated), since that's what my system needed. 

But if you want industrial, I really want a line interactive UPS, like the CyberPower PR1500LCDRTXL2U.  But those are CRAZY expensive. :)

Link to comment
Share on other sites

  • 0

Ah man! I didn't know about Active PFC until now! And of course my relatively cheap UPS uses simulated sine waves. So I had to search for EVGA and UPS and Active PFC. According to a FAQ on the EVGA website, my PSU should be fine with my UPS, but I will have to do tests to make sure. I should've talked to you BEFORE getting the UPS, lol. As for line interactive, I thought they would all be like that by now. According to this, my Back-UPS "Pro" should be Line-Interactive: http://www.apc.com/us/en/faqs/FA157448/

Regards,

Link to comment
Share on other sites

  • 0

Yeah, Haswell is when that stuff changed, IIRC.  The real issue is when it switches off of wall power onto battery power. Especially if you have a high load, it can overload the battery, and turn off (I've heard that worse can happen, but no stories to back that up). 

And yeah, feel free to ask us about server related purchases, any time!  That's what the "offtopic" is for, too. :)
And yeah, I've done a good amount of research into UPS' because ... I have a giant server rack. 

And yeah, most are.  I think I meant the double conversion, which is super nice. But only if you're a power hungry fiend. (we have solar, because of AC and my servers... so yeah......)

Link to comment
Share on other sites

  • 0

Alright, tonight was UPS testing night. As far as Active PFC and simulated sine waves, no issues, but I can only pull 200W off the UPS (I currently do not have a graphics card). Unfortunately, not everything works. If I set the computer to sleep at say 80% and to shutdown at 70%, it goes to sleep correctly but then it wakes-up immediately, and stays turned on until it hits the critical level. Event log is not helping, it always says the wake-up source is unknown (even if you wake with the mouse). This works fine with a laptop, not so well with a UPS and a desktop. Maybe it works better in Windows 10? At least it does shut down correctly at the critical level (when I power it on again it records that it shutdown unexpectedly in the event log; hopefully it flushes its buffers correctly when it shuts down!) Anyway, thanks for the Active PFC info!

Link to comment
Share on other sites

  • 0

Hi Chistopher.

Nope, LastWake is useless, here is what it prints: Wake History Count - 1, Wake History [0],  Wake Source Count - 0.

And I got ALL the wake-up methods disabled in the NIC, so that's not it, I think maybe it's not supported on desktops, it's some bug in Windows, something like it wakes-up immediately so it can watch for the critical % coming-up. I just use the critical action, set sleep to do nothing.

Regards,

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...