[What does "unusable for duplication" mean?]

Good points, I think I have my answers now.  I did some more reading as well and I have enabled file integrity on each of my drives within my pool. Any drives I add from now on out I will format with file integrity enabled from the start.

 

Thanks for the walk through!!

 

Best wishes,

Scott

[What does "unusable for duplication" mean?]

Drashna,

If I do want to enable file integrity streaming I would need Storage Spaces, correct? Then I would run this command in PS Set-FileIntegrity H:\ -Enable

Since I don't want to enable storage spaces (I would rather you guys worked out a StableBit version and save some space. Pretty Please!!) will I gain any value from enabling file integrity streaming?

So I guess my basic question is this, will ReFS by itself (no commands ran other than a standard Quick Format as ReFS) stop bitrot?

[What does "unusable for duplication" mean?]

Thanks again, might have to change the title of this discussion!

So, taking your points in order

On 3/30/2018 at 1:18 PM, Christopher (Drashna) said:

If "bad things happen", then there is no (cheap) way to recover your data. Very few data recovery tools support ReFS yet.  So if you needed to run recovery, you're SOL (either having no options, or paying an obscene amount of money to get it recovered). 

Well, I will enable pool duplication as you do. In fact I have started it back up as I continue to move files over..

On 3/30/2018 at 1:18 PM, Christopher (Drashna) said:

If there is file system corruption, there is no equivalent to CHKDSK (this shouldn't be as much of an issue but it can still happen)

Now this is irritating as I am can get a little OCD but I think I can get my head wrapped around it, especially if I can have file integrity. NO BIT ROT!!

On 3/30/2018 at 1:18 PM, Christopher (Drashna) said:

There is also a performance hit for using ReFS.  Not just the integrity streams... but because it's a COW file system (copy/allocate on write).  So it's not going to be as fast as NTFS, in most cases.

I have read that but usually my write speed is not so important as my read speed, except for moving files over from old drives! LOL Other than that not so worried.

On 3/30/2018 at 1:18 PM, Christopher (Drashna) said:

And integrity streams can take up a significant portion of the disk, so you end up losing space.

True but drive are cheap right? LOL!!  I am slowly upgrading all my drives to a standard of 4TB and specifically WD Red drives.

PROs ..

On 3/30/2018 at 1:18 PM, Christopher (Drashna) said:

However, because it's a COW file system, it's much less prone to data loss or corruption do to power loss (less likely to see "raw" partitions, for instance), since the data isn't allocated until the write is complete (IIRC).

I remember reading about this years ago, perhaps about ZFS?? Not sure but when I heard that ReFS would have it for Windows I was almost sold then and there. You are right, very attractive!

On 3/30/2018 at 1:18 PM, Christopher (Drashna) said:

Additionally, there is the file system integrity thing. However, this is NOT enabled for normal files, by default. It's only enabled for metadata, normally.  You'd have to use the "Set-FileIntegrity" command after formatting to enable this.  

This is a point I want to know more about .... 

On 3/30/2018 at 1:18 PM, Christopher (Drashna) said:

Larger partition sizes supported out of box

Absolutely, once I have standardized with 4TB then I will start upgrading those. Nice to have room to grow!

On 3/30/2018 at 1:18 PM, Christopher (Drashna) said:

Integrity checking

Okay, so you mention the "file system integrity thing" but say it is not enabled by default and you follow that up with a PS Command. Then your last line mentions "Integrity Checking". I need a break down on these two and the differences as I thought this ability was enabled by default...

Is there a link you can send me that I can read about, with the pros and cons???  I am assuming with file integrity checks there is more space lost ...

My irritation has been that I will save a file and then a month or more later I will go to open it and it won't. I will go to play a video and I will be told that the system is unable to render that file! Bitrot drives me crazy! I have, over time, upgraded my server and am now running server memory (E.C.C.) and I 'think' I have seen fewer errors and lost files but ...

Do you think "file system integrity" is worth it or is "integrity checking" enough?

Thanks Drashna.

 

 

 

 

[What does "unusable for duplication" mean?]

yep, that's about how long it is taking, the folders with larger files move a bit faster but the folders with smaller files takes a bit longer.

 

I know I am dragging this conversation out but I have one more subject; ReFS.

 

From all that I have read ReFS is better for data storage, even without Windows Storage Spaces. Is that correct from you guys point of view?  I have already started the move, obviously, and no one popped up and said "Hold on, that's not a good idea!" so I hope the answer is "Yes, ReFS is better for data than NTFS, especially if you use StableBits DrivePool!"

I have had bitrot happen to me too many times and I just got tired of it, I am hoping that ReFS will put a stop to it.

And are you all still working on adding more benefits to using ReFS to drivepool? Can we look forward to a Drivepool with Storage Spaces flavor added?

 

Your thoughts?

[What does "unusable for duplication" mean?]

Thanks,

 

I was wondering about the free space on the new pool.  Since my original question, I formatted all the drives on the new ReFS pool with a cluster size of 4096 and with a 2nd 4TB drive added all my "problems" went away. I was getting worried for a bit there but thanks for the comment "But if you're not using duplication, you can ignore it, as it won't affect your free space. " I feel a bit better, I will re-enable duplication once I am sure I have enough disks to support my new pool. 

Wow, it takes a while to move 24TB of data!!

 

Thanks again everyone!

 

 

[What does "unusable for duplication" mean?]

Drashna,

I know this is an old thread but ..... would cluster size have anything to do with "unusable for duplication 637GB"?

 

I pulled a 4TB WD RED from an existing NTFS pool on my win10pro workstation, formatted it ReFS (I can't remember what the cluster setting was, 64K or 4096 bytes) and started a new pool with it. I disabled duplication on both pools. Then I started moving files over to new ReFS pool. This created enough space to remove a 2TB drive from old pool. I formatted that drive ReFS and set cluster to 64K.

Then I removed my 2 SSDs from old pool, formatted them to ReFS (64k) and added to new pool.

I am now sitting at 637GB of unusable space for duplication. I haven't even turned duplication on yet as I have another 24 TB to move to new pool!

I installed drivepool version 2.2.0.906 when I first saw this problem and both pools are on the same system. I have included captures of old and new pool.

PS. Love Drivepool, can't wait for you guys to fully support ReFS!! 

Thanks for a great product!