Write Performance Question (multi pool vs. file placement)

[donnatronious]

Happy Drivepool user here.  Have 40TB raw.  Currently Drivepool is my home "production".

 

Drivepool is about to become backup for Freenas.  I"m looking for ways to increase the throughput moving data from Freenas to Drivepool across 10GbE.

 

Consider these two hypothetical scenarios. Will either of these allow two copy jobs in parallel to write at the speed of two disks?  Assuming a single disk will sequential write at 150MB, I'm looking to write at 300MB.

 

SCENARIO 1 - single pool with two folders

------------------------------------------------------------------------------

Pool Z with 4 disks

File placement options set for /ISO folder to use disk 1,2

File placement options set for /MKV folder to use disk 3,4.

 

 

SCENARIO 2 - two pools each having 1 folder

------------------------------------------------------------------------------

Pool Y with 2 disks storing /ISO

Pool Z with 2 disks storing /MKV

[Christopher (Drashna)]

Well, either option should work well, actually.  

 

And if the copy program you are using supports multi-threaded copies, then that should work better.

 

 

However, you may want to check out the SSD Optimizer balancer plugin.

https://stablebit.com/DrivePool/Plugins

 

However, that said, it will have to balance the data off of the "SSD" drives, which may be the speed/performance bottleneck.

(it writes to the SSDs, and then the files get balanced off)

 

That said, you don't have to use SSDs. You can use any fast drive. 

[donnatronious]

Hmm, I didn't expect you to say that scenario 1 would write at the speed of 2 disks...

[Christopher (Drashna)]

Hmm, I didn't expect you to say that scenario 1 would write at the speed of 2 disks...

 

well, it may, but it may not.  It really depends on what exactly happens.

 

But "for the most part", StableBit DrivePool passes on the IO to the underlying disks.  So if you're writing two files, they may be written to the two different disks.  And in your scenario, it may be that four different files will be written to four different disks. 

 

This isn't a guarantee, as the default placement strategy is to write to the disk with the most available free space. So depending on the disks, it may not get the best case scenario, and may write the files to the same disk.

 

But this really applies to both scenarios, really.