Looking at options - Windows Server 2019 (Parity/RAID-5)

[Nathan]

So I have built a new Plex box out of a combo of spare parts and some new drives, the box is running Windows Server 2019. What has occurred over the past week has been a journey into discovering how much of a mess RAID support is in the Windows world right now.

So what I originally had wanted was 3X - 4 TB Ironwolves in RAID-5 with a single 120 GB SSD as write cache. I have a 10 TB Barracuda Pro as backup drive for that array. My original plan was to use Intel IRST to manage this. Well it turns out Intel IRST only supports drives on the Intel mobo controller, it won't support other PCIe SATA controllers. So no go there.

Next up, RAID-5 with Windows Disk Manager. So this works but I can't easily expand the RAID with new drives and there is no way to add SSD cache.

Okay, lets try Windows Storage Spaces. This does work, but for a home user this is kind of a half baked mess and the parity support is beyond awful. It actually has fantastic read speed but the write speed is just ridiculous. Since the Storage Spaces GUI shoehorns you into a very specific type of setup the only way to actually setup a storage pool with the configuration I wanted was to build the entire thing in Powershell which involved a lot of trial and error with a bit of head banging. It was possible though.

 

So I keep getting recommended to use Drivepool. My issue with Drivepool is it means I lose parity (which is not the end of the world) but worse I lose the benefit of RAID-5 combined drive read speed (this is a deal breaker). So my question comes down to is there anyway for Drivepool to incorporate RAID-5 /parity arrays from Windows Disk Manager or Windows Storage Spaces into the pools?

 

Thanks

 

[Umfriend]

AFAIK, DP can use Raid-arrays as disks in a Pool. DP does not, however, provide RAID-functionality itself (other than duplication which is sort of a file-based Raid-1).

[Nathan]

6 hours ago, Umfriend said:

AFAIK, DP can use Raid-arrays as disks in a Pool. DP does not, however, provide RAID-functionality itself (other than duplication which is sort of a file-based Raid-1).

Yah, I just tested it. It wouldn't pickup a Windows Disk Manager RAID-5. But it did surprisingly pick up the Storage Spaces Virtual Disk.

While certainly an interesting piece of software, sort of a RAID-light type of solution Drivepool does not really bring anything to the table that I don't already have being done. Given I have a parity RAID and have a directory mirroring tool already installed (just mirrors any changes in my media directory to the backup drive).

I think I was more looking for something that could replace Storage Spaces and be a good RAID type disk manager but that is not really what Drivepool is.

If the devs are interested in throwing in a parity drive tool down the road it would definitely be worth another look.

[Umfriend]

It's been asked and I got the impression they won't. Apparently, keeping parity has quite a performance impact? The nice thing about DP is that you can mix and match I guess, no requirements on any size differences between disks and recovery is fast a disk failes (at x2 duplication).

[Nathan]

5 hours ago, Umfriend said:

It's been asked and I got the impression they won't. Apparently, keeping parity has quite a performance impact? The nice thing about DP is that you can mix and match I guess, no requirements on any size differences between disks and recovery is fast a disk failes (at x2 duplication).

Parity has a negative write performance impact. Its definitely noticeable, but when implemented properly not a huge issue and speeds are still very usable. The big perk is parity read speeds are excellent. Its an ideal solution for something like a home media library.

Parity is not a backup though, I would describe RAID-5 as a space efficient, fault tolerant RAID configuration. You still absolutely still require a backup with RAID-5.

What DP is doing is closer to file system mirroring sort of in the realm of thin volume RAID-1 or RAID-10 in the RAID world. I would suspect RAID-10 would have better performance though.

What DP does bring to the table is a very simple and easy to use UI and more flexibility with on the fly file system configuration changes.

[Umfriend]

I agree Raid (or DP) is not a backup.

What read speeds does your media library need? I've never heard of spinners not being able to feed a stream.

Yeah, I'd say Raid-10 (or 5) offers better read speeds but at the cost of inflexible disk configurations and less flexible recovery mechanics.

[Nathan]

Right now I have settled on Storage Spaces, its definitely not ideal but its the best solution I could find right now. I'm using an PCIE SATA card for the Ironwolf spinning drives which is limiting speeds a bit as well, but I'm stuck with this config unless I want to change my motherboard or do a full system upgrade.

I'm getting sustained 200-300 MB/s write while things go to the pools SSD cache drive (connected to a motherboard SATA 3), then its drops to 30-40 MB/s when SSD cache runs out which is horrible but apparently normal for parity storage on Storage Spaces.

For read speed I'm getting about 300-350 MB/s off the Storage Spaces virtual disk (pulling from the spinning drives) which is probably hitting the cap of my PCIE SATA card.

All the Plex transcode cache feed into a 500 GB SSD (motherboard SATA 3) which has read/write speeds in 400-500 MB/s range.

In this config with my current CPU limits I can probably run 5X - 1080p transcodes simultaneously and who knows how many 480-720p ones without any issues. There are currently 3-4 active streamers using the system so the math works out well for the use case.

If this was a NAS/file server instead of a media server I would absolutely be using RAID 10. I actually have a QNAP setup in RAID 10 specifically for that, though Storage Spaces also has its own version of RAID 10 as well.

[Umfriend]

That's fine. What I just don;t understand is that even 4K should have a bitrate of less than 40Mbit/s so I don't see how this helps. I wonder whether a single HDD wouldn't be able to provide that (unless in transcoding there are writes to the HDD as well but there Raid-5 would be penalizing methinks.

[Nathan]

34 minutes ago, Umfriend said:

That's fine. What I just don;t understand is that even 4K should have a bitrate of less than 40Mbit/s so I don't see how this helps. I wonder whether a single HDD wouldn't be able to provide that (unless in transcoding there are writes to the HDD as well but there Raid-5 would be penalizing methinks.

My understanding is you want about 10 MB/s per 1080p stream, 4K is somewhere between 15-20 MB/s per stream. Note that is the minimum amount to maintain a stable stream with overhead without having a constant buffering issue. Then you also need to allow additional bandwidth for other tasks and interruptions (videos being copied on or off the drives in the background or a media scanning happening). I don't use 4k given the CPU requirements of transcoding those, plus there is only one 4k TV in the entire house right now. 

For a single user you are fine with a single spinning drive.

It would start getting dicey with 4 people streaming at once, especially if the box has other tasks going on. At 6 or more people streaming you can just forget it on a single spinning disk. You have to keep in mind that most of the time people are going to be watching different things so that is 4 different files being streamed off the drive at once.

Yah, while its possible to use a spinning disk its highly recommend to use an SSD for transcoding. It drastically drops the buffering time.

[Umfriend]

I am probably most likely completely wrong but my understanding is the minimum requirements are around 10Mb/s for 1080p as a minimum. But 10MB/s seems a bit much.