Poor Pool Performance - Ideas?

[bzowk]

Hey Guys -

 

A couple of years ago, I started building my pool and have only added to it since.  Currently, it's 32tb in total size which is comprised of 11 disks (one of which is an SSD for caching.)  DrivePool has worked well for me and even though experiencing an issue, is much better than what I started with - DriveBender.

 

My partition setup is like this:

- C:\ - SSD - 256gb - OS & App Installs

- D:\ - Drive Pool - 32tb - Storage (Mostly media)

- E:\ - Hybrid - 1tb - Game Installations & VMs

 

It seems as the pool has grown, it's performance has decreased.  Initially, I thought it may be the OS so have tried complete OS reinstalls (not just restoring from backup) which didn't help.  At the time, I had all of my disks plugged into the motherboard or one of two SATA expansion cards.  After research and posting, I purchased a RocketRAID 2720 SAS card which had the ability to add 8 new SATA ports via 2 SAS->SATA fan out cables to bear some of the load.  This seems to improve reliability and a bit of performance, but still not close to what the pool would be like compared to a single HDD.  I do plan on reinstalling the OS again soon - but only because I do this 2-3 times a year anyways to keep things running at top notch.

 

Issue Details

Besides just slugish speed overall much of the time, most issues seem to be when I am reading files from a large folder, it takes a while - much longer than when I copied the folder to a similar single drive a while back.  Also, when browsing through it in Explorer, sometimes it takes a while to display contents or if I delete files / folders, I must manually refresh to see the changes.  The pool doesn't not contain the OS, paging, or any application installations.  It's mostly used for storage of media, other various files, and is the download destination for many scripts.  

 

Individual Drive Benchmark

To determine if the performance may be due to hardware or software, I set out today to run some tests.  Using HD Tune Pro (5.60), I ran benchmarks on each disk in the system plus an external USB 3.0 for comparison.  I only performed the "Read" benchmark as the "Write" one required wiping the disk.  If there's a different type of test you suggest I run, please let me know.  I had planned on using a different app, but was only able to select by partition - not disk - which obviously was an issue.  I then recorded the results plus data gathered from StableBit Scanner + AIDA64 into a spreadsheet.  The spreadsheet also includes a summary of my system specs as well as a couple of notes.

 

Below is a screenshot of the report created today.  If interested or easier to view, attached are copies in XLSX & PDF formats.  I had to zip up the XLSX because the forum wouldn't allow me to attach it otherwise.

 

Any ideas or suggestions?  Thanks!!!

Generalized Disk Benchmark Report.zip

Generalized Disk Benchmark Report.pdf

[Christopher (Drashna)]

You've posted this as a ticket, as well. But I'll repeat here. 

 

 

As for the slowdown, that is definitely unusual. Though, this definitely can be influenced by the hardware and software in use. 

 

For instance, anything that uses file system filters can adversely affect the performance of any disk (not just the pool). By default, we bypass these filters for accessing the pooled disks (to boost performance and for compatibility reasons). But we don't bypass them for the main pool.  

Antivirus, backup, defrag and other disk tools use these filters to intercept and in some cases, alter the disk access. This may well be the cause of the performance issues you are seeing. 

 

Could you post the results of this: http://wiki.covecube.com/StableBit_DrivePool_Q2159701

This lists all of the file system filters installed on the system, and may help point out where the issue is.

 

 

 

Also, is the pool duplicated (or using high levels of duplication for the folders in question, or maybe, no duplication)? 

 

And the issue, is it with enumerating the contents of the folders? Or just copying/accessing files from them? 

Though, it does sound like both. 

 

 

And also, could you try the latest beta build? 

http://dl.covecube.com/DrivePoolWindows/beta/download/StableBit.DrivePool_2.2.0.663_x64_BETA.exe

There are a large list of fixes and optimizations that have been done, and a number of them may specifically address the issues you are having with performance. 

 

 

Additionally, try toggling the "Read Striping" feature (Pool Options -> Performance), to see if that helps. 

 

 

And if the new version doesn't help, and toggling the Read Striping feature doesn't help, then try enabling file system logging, and attempt to reproduce the issues.  Stop the logging immediately after reproducing the issue (as this is a very detailed log, so the shorter it is, the easier it is (aka, it's a smaller haystack for us).