Jump to content
  • 0

Drivepool and ReFS


Mike_B

Question

Not sure if this has been asked before, so forgive the repeat...

 

I've been using Drivepool for years now with NTFS as the filesystem. I'm going to be building a new storage server with Windows Server 2016 for my home, and like to reevaluate how I do things from time to time...

 

I'm curious if there are any advantages to using ReFS as the filesystem for my pooled drives over NTFS. Are Is it a matter of preference, or are there performance pros and cons to consider (disk I/O)? I've heard that ReFS has the ability to recover from data issues (bad blocks), which is why I'm asking the question.

Link to comment
Share on other sites

6 answers to this question

Recommended Posts

  • 0

Hi I have been using REFS for ages now and have not suffered any file corruption or bit rot since I swapped. the only downfall is to get the self heal you have to use storage spaces some of us were hoping Alex might be able to get it to work with drivepool but he's been Working on cloud drive for so long now it's well on the back burner. On a side note I run an identical pool with storage space and it works just as well as drivepool with the added benefit if it works self heal.

Link to comment
Share on other sites

  • 0

Are you saying you've been using ReFS with Drivepool, and you haven't seen any type of problems or performance hits vs NTFS?

 

Also, I've looked at Storage Spaces on Server 2016, and it seems to be somewhat limiting with how volumes are handled vs Drivepool. Drivepool appears to be much more flexible with available storage space when using different sized disks vs a mirroring configuration in Storage Spaces.

Link to comment
Share on other sites

  • 0

Hi yes being using Refs with drivepool for well over a year now and not had any problems whatsoever even tho drivepool doesn't officially support Refs yet. I haven't noticed any performance issues but since everybody's machines are different how yours will perform is an unknown

Link to comment
Share on other sites

  • 0

I posted a reply to Mike, but let me repost my answer here.

 

 


First is that ReFS only allows the use of "64kb clusters" (allocation unit size).  The nice thing about this is that it means less fragmentation of the underlying data, as more data is stored sequentially (as opposed to the default "4kb" cluster size for NTFS).  
 
This reduces fragmentation, and can significantly increase performance for large files (10-20MB/s difference, in my experience) 
The downside is that if you have a lot of small files, or files that don't fit neatly into this size, you may end up with a lot of wasted disk space.  This is what most refer to as "slack space". it's that left over space from that 10kb file, where the other 54kbs are not used.  Those clusters can't be shared by files, so if it's not filled, it's left empty.
This can add up on the disk, and DrivePool shows this as "Other" space (along with other reasons).  
On my personal pool, that's between 20-50GB of space (adding up to 600GB over 25 disks). 
 
 
Another benefit is ReFS is that it's a "Copy on Write" or "Allocate on Write" file system. That means that when you modify a file, it doesn't modify the data on the disk, it creates a new copy of the file, and then modifies that. That way, if power loss or the like occurs, the worst case scenario is that the file is reverted back to it's previous state. 
 
This alone makes the file system much, much more resilient to corruption or damage. 
 
 
Additionally, the "self healing" aspect... you're right, this only occurs on storage Spaces (mirrored or parity configurations only).  
 
However, this is a two part process, and that first part still works.  Specifically, ReFS uses "integrity streams" when writing data. This means that a checksum block is written for each block of data.  This is created when writing data, and read back when accessing the data.
 
This is very useful, in that allows you to verify that the disks' content is intact.  No worrying about media degradation or bit flips (but in all honesty, your disk is doing this as well, so it's just doubling up on protection). 
 
The difference here, is that when Storage Spaces is backing ReFS, if it detects integrity errors, it will check the other disks in the Storage Spaces pool and see if it has a good set of that data. If it does, it will automatically rebuild the data. 
That's specifically what happens when the self healing occurs. 
 
 
 
The downside to using ReFS here is that namely that StableBit Scanner doesn't work with ReFS file systems.  The surface scan works just fine, but it's unable to perform file system checks, and if there is damage, we cannot recover files. 
We would need to be a custom ReFS parser, so we can read the data from the file system. This requires reverse engineering the file system though, and is not a small task. 
 
 
The other downside, is that there is definitely more IO that is occurring on the ReFS drives because of the copy on write, and the integrity streams.  However, in my experience, this isn't significant, when using the disks outside of Storage Spaces. 
 
The final downside is that DrivePool treats these drives as normal NTFS volumes, rather than ReFS drives. This mostly means that we are not checking the integrity streams, at all.  This is something that we would like to implement, and ... well, effectively handle in a way similar to what Storage Spaces does.  But this requires research and testing, which is a significant time investment, as well. 
But hopefully, in the future, we will expose the drive fully as ReFS and support the file system in full. 
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...