Read Striping not working at all?

[danielb]

Hi

I have 1x3TB WD RED + 1x 4TB WD RED, and with read striping enabled, drive pool insists on reading only from the slightly faster 4TB HDD. I'm copying a 6GB file to a RAMDisk, so that is not a bottleneck. I get around 160MB/S with striping enable/disabled, that's the max speed of the 4TB drive. During the transfer i can see for brief moments the 3TB hdd showing up in the performance tab, but it goes away almost instantly!

I understand i wont get the total sum of read speed of both HDD, but i expected a improvement, even if only 10MB/s or something, at this time i don't see ANY improvement.

OS Windows server 2016

Folder duplication is enabled 2x on the file i was testing.

[Umfriend]

As far as I can tell, read striping only works when you are reading LOTS of files, not a single large file. If, for instance, I browse through a folder with lots of pictures, then the thumbnails come up way quicker with read striping enabled. I *think* this is because DP is file based and opens I/O to individual files located on an individual disk. If many files are to be read concurrently, then it may initiate some I/O on one and some on the other HDD.

[Jaga]

I was all set to agree with you Umfriend (actually had it typed in already), until I re-read the docs about read striping.  Large sequential I/O uses block-based read striping, so it should have kicked in for danielb.

@danielb - have you benchmarked each drive to see if there's much of a difference?  It's possible that one actually is performing better than the other for some reason, and Drivepool picks up on that.

 

Quote

StableBit DrivePool utilizes a number of read striping algorithms, depending on the situation.

For large sequential I/O, such as large file copying, read striping will utilize a block based algorithm, maximizing the use of each disk and minimizing disk context switches.

For random non-sequential I/O read striping always sends the request to the disk with the least outstanding requests. Because seek times can be high in this scenario, StableBit DrivePool tries to switch disk contexts often.

For slow non-concurrent I/O read striping passively measures the speed of each disk and dynamically switches the the fastest disk.

 

What it sounds like Drivepool is doing in this case, is assigning different priority classes to the two different drives, which puts it in a situation where it would never read stripe across the two.

[danielb]

Thanks for both of your replies.

Yes the 3TB is slightly slower, since its older, it gets around ~110MB/s.

I was expecting it to work since they say so in this page: http://blog.covecube.com/2013/05/stablebit-drivepool-2-0-0-256-beta-performance-ui/ they even show it working..( http://blog.covecube.com/wp-content/uploads/2013/05/read_duplicated.png )

I didn't really tested this before, since a single HDD would hit the 1Gb NIC max speed, but now that i got a 10Gb NIC, i was hoping to get a bit more performance out of this.

EDIT: Just tested, copying 3 different files, all of them are duplicated 2x. Drivepool seems to insists in using only the 4TB drive?! (see screenshot, i think i showed all relevant stuff)

 

[Umfriend]

1 hour ago, Jaga said:

I was all set to agree with you Umfriend (actually had it typed in already), until I re-read the docs about read striping.  Large sequential I/O uses block-based read striping, so it should have kicked in for danielb.

 

Yes. But I have never ever been able to get this to work.

For instance, if it would work, then even if you had two HDDs where one would read at 100MB/s and the other at 20MB/s, then a bulk reader should be able to profit from using both.

[danielb]

Well i tried a bunch of random files, and out of nowhere it showed it was read striping, but wasn't able to get it to work again!

This feature seems to be a lot temperamental! It would be great if there was a settings to force it on all the time.

[Jaga]

2 hours ago, Umfriend said:

Yes. But I have never ever been able to get this to work.

For instance, if it would work, then even if you had two HDDs where one would read at 100MB/s and the other at 20MB/s, then a bulk reader should be able to profit from using both.

Except for the priority classes that Drivepool puts drives into.  i.e. an external USB will always be in a different priority class from an internal SATA drive, no matter what (per the DP docs).  I'm not entirely sure if that gives a benefit when you have super fast external drives (mine are almost as fast as my internal spinners due to their new USB connections), but it's part of the algorithm that Alex put into DP.

 

1 hour ago, danielb said:

Well i tried a bunch of random files, and out of nowhere it showed it was read striping, but wasn't able to get it to work again!

This feature seems to be a lot temperamental! It would be great if there was a settings to force it on all the time.

Which is interesting.  Makes you wonder if the UI is lying to you about block striping in it's UI, or if block striping needs something else to fully kick in on large files.  Might have to wait for @Christopher (Drashna) or @Alex to chime in and let us know.

[danielb]

Still having the "problem". I was really hoping to get this to work... Btw this is on a fresh install of Windows Server 2016.

[Christopher (Drashna)]

Could you mind opening a ticket at https://stablebit.com/Contact ?

[danielb]

Ticket opened.

[Christopher (Drashna)]

Thanks! 

[postcd]

Was there any resolution/progress on that ticket?

[Christopher (Drashna)]

I believe this was resolved:
* [D] [Issue #28076] Real-time file placement limits were not always being respected when real-time duplication was disabled.
http://dl.covecube.com/DrivePoolWindows/beta/download/StableBit.DrivePool_2.2.3.978_x64_BETA.exe

But I'm not 100% sure this was that issue. 

 

[danielb]

Not sure if that is the same issue. Do you want me to test it anyway?

[Christopher (Drashna)]

Yes, if you would. 

If that doesn't help, then could you open a ticket at https://stablebit.com/Contact?

[danielb]

I already have a ticket open for some time now.. 1909971

[danielb]

Just installed the latest beta. Same result.

[Christopher (Drashna)]

Okay.  Thank you for confirming.  And I've bumped the ticket.