Jump to content
  • 0

Read Striping Slows Things Down. A LOT.


davidfowler2000

Question

I'm using DrivePool on my home server. It's a new machine to me, new install of Windows, latest version of DrivePool. DrivePool is running on the trial license until I copy the data off the old server and transfer the license. I first noticed this problem transferring data from the old server (latest DrivePool) to the new one via the network and when I tried transferring data from the new server to my PC I got the same thing.

I'm using 2 x 1Gb network connections to get 2Gb via SMB multichannel. Transferring a single large file from the server runs at about 45MB/sec and transferring lots of smaller files like photographs runs at about 10MB/sec in bursts of 15 - 20 files with 2 or 3 seconds of inactivity between each burst.

Turning read striping off gets me 110MB/sec for a large file and 60 - 70MB/sec for lots of little files.

Transfers from my PC to the server run at about 220MB/sec until the SSD fills up.

System is:
Dell PowerEdge T320
Xeon E5-2407
24GB DDR3 ECC RAM
Windows Server 2016 Essentials
2 x 2TB Toshiba hot swap SAS HDDs
1 x 256GB Plextor NVMe PCI-Ex SSD (Boot drive. 2 partitions - one for system, one pooled)

Link to comment
Share on other sites

10 answers to this question

Recommended Posts

  • 0

Going to try to help dig at the problem until Christopher shows up..

What's your Pool architecture on both servers?  You may already know - read striping only helps when you have Pool/folder/file duplication on separate physical drives on the server you copy files from, though I'm unsure why it would decrease performance as much as you're seeing even without duplication.

Have you disabled SMB multichannel on your PC and the old server, then re-tried read striping on/off copy tests (server->PC) to see if the mutlichannel is playing a part in the problem?

Going to assume that your PC -> new server transfer (@220MB/s) is a PC->Pool copy, not a Pool->Pool  or Pool->PC copy (where you see the issues).

Are both copies of DrivePool release channel and not beta channel?

Are you seeing any read striping bar color changes in the performance UI (scroll down to Read Striping) of DrivePool when you have striping enabled on the source server during a copy to the PC?

Link to comment
Share on other sites

  • 0

Hi Jaga.

The files I am reading from the server are in a duplicated folder. The current set up is 2 SAS HDDs in a hot swap backplane and a SSD on a PCI-Ex card with the SSD Optimizer plugin so the data should be coming from the two HDDs.

With read striping enabled, the Read Striping bar goes light blue and dark blue and you can see both drives being used in the activity panel. Mostly it's dark blue and very occasionally there's a little bit of light green.

I disabled SMB multichannel on the server by unplugging one of the network cables. It made no difference. The old server doesn't have SMB multichannel.

The 220MB/sec is from the SSD on my PC to the server pool with the SSD optimizer.

 

On the old server it's 4 HDDs and 2 SSDs which will eventually make their way to the new server. Duplication enabled with some folders duplictaed 3 or 4 times.

Both copies of DrivePool are release channel (downloaded from the main website). I tried an older version on the new server and it made no difference.

Link to comment
Share on other sites

  • 0

Okay cool.  So we know it's not the SMB multichannel, especially if the old one never had that and you still get the striping slowdown there.

It is odd however that with duplication enabled on the pool(s), that the read striping performance bar changes between dark/light blue, when it *should* be green under ideal conditions and read striping is actively using all drives.

If both servers are delivering reduced speeds with read striping on the hardware is less likely of a culprit.  And it isn't the network layer.  It points more squarely at either configuration, or DrivePool's current install package on both.

Let's see what Christopher/Alex have on it - it's starting to get above my DP pay grade.  :) 

Link to comment
Share on other sites

  • 0

Yes. Putting files on to the server / pool is fine as I'm using a SSD to buffer incoming files. It's getting duplicated files back off the pool that is slow. I have since discovered unduplicated files are read normally whether read striping is on or off.

I have been experimenting with local file transfers from the server's pool to it's own C:\ drive.
Read striping enabled does not seem to slow down single large file transfers but it will only use one drive so the maximum transfer rate is the maximum of whatever disk it is coming from. Read Striping bar is 100% dark blue. (My interpretation of read striping is that it will transfer faster than the maximum drive speed as it can read from multiple sources but I am quickly figuring out that read striping is more for better access to lots of files at once.)
Multiple small files are slowed down slightly. Photographs transferring at ~72MB/sec without striping, ~55MB/Sec with it enabled. Multiple drives being used. Read Striping Bar all 3 colours but mostly dark blue.

Network transfers from the server to my PC:
Read striping enabled causes single large files to transfer erratically. When the bar is 100% dark blue, it's only using one drive and transfers at 110MB/sec (network maximum) but when it starts going light blue up to 50% and using multiple drives, it slows right down to 45MB/sec.
I have tried swapping out the Dell PERC H310 raid card for a HP H220 host bus card to no appreciable effect.

So it seems to be there's a bottleneck appearing somewhere when the system is trying to use more than one hard drive at a time. I'll see if there's any firmware and / or driver updates for the PERC card and SAS backplane. Is there anything in Windows itself that may be causing this?

Edit: There was an Intel chipset update. It improved single file transfers over the network although the transfer speeds are still erratic and slow when using multiple disks. But overall a bit better than before as it's not using multiple disks as much. Multiple small file copies with read striping enabled are now down to 10MB/sec using multiple drives or 80 - 90MB/sec with read striping disabled.

Further experiments:
I duplicated a folder with big files 3 times so it would go on to the SSD as well. Copying from that is fine with read striping enabled but it seems to be because it's mostly using the SSD. HDDs sometimes get used but doesn't cause any slow down. Remember the SSD is a M.2 drive on a PCI-Ex adapter card so it's not using the same interface as the HDDs.

Link to comment
Share on other sites

  • 0
1 hour ago, davidfowler2000 said:

Further experiments:
I duplicated a folder with big files 3 times so it would go on to the SSD as well. Copying from that is fine with read striping enabled but it seems to be because it's mostly using the SSD. HDDs sometimes get used but doesn't cause any slow down. Remember the SSD is a M.2 drive on a PCI-Ex adapter card so it's not using the same interface as the HDDs.

Can't rely on that though, since it's part of the SSD Optimizer which *will* clean out that SSD and move files to the regular pool drives.  Something else is going on here.

Link to comment
Share on other sites

  • 0

Read Striping should only affect the pool that data is being read from.  It shouldn't affect writes, at all. 

 

Either way, could you open a ticket at https://stablebit.com/Contact?

And enable file system logging on both systems, and reproduce the issue? 
http://wiki.covecube.com/StableBit_DrivePool_2.x_Log_Collection

And then run the StableBit Troubleshooter on both systems? 
http://wiki.covecube.com/StableBit_Troubleshooter

Link to comment
Share on other sites

  • 0

Hi Christopher.

I cannot really test this with the old server as it's been mostly emptied in order to move the drives in to the new server. It's not a major concern as it's getting decommisioned anyways.

I'll enable logging on the new server and do transfers of large files and many smaller files with and without read striping enabled. I'll do it from the server to the PC over the network with SMB multichannel disabled (one network connection plugged) and two network connections to each machine. I'll use the same group of files and restart both machines between each test so there's nothing sitting in any caches.

Link to comment
Share on other sites

  • 0

@nagetech Try the latest beta.  We did find some issues with read striping:
http://dl.covecube.com/DrivePoolWindows/beta/download/StableBit.DrivePool_2.2.3.978_x64_BETA.exe

 

Also, we added a testing tool for Read Striping:
http://wiki.covecube.com/Downloads#Other_Tools

It's the File Read Simulator program. 

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...