How much Parallelism?

[donnatronious]

Lets say I have 4 drives connected through SAS to the pool and I have no duplication turned on.  The files in the pool are all 40GB ISO files.

 

Now I add 4 more identical drives to the pool via SAS.  ( I know drivepool doesn't care how they are connected)

 

Now I turn on 2x duplication.

 

Will Stablebit drivepool copy 1 file at a time?

 

Maybe it ends up doing 4 copies at the same time because it has 4 disks to read from and 4 to write to?

[donnatronious]

so much lonesome on this question...

[Christopher (Drashna)]

Sorry, the forums don't get checked as often. But they do get checked several times a week.  

 

This depends on what you're using to copy the files.  StableBit DrivePool doesn't do any trickery to increase this. 

 

Specifically, the Explorer copy is single threaded, so it will only copy a single file at a time.   At least "per job" (and by job, I mean each "section" in the "transfer" window). 

There are multithreaded copy programs (such as robocopy) that can perform multithreaded copies, that copy multiple files at a time, and StableBit DrivePool can handle that, without issue. 

 

Additionally, if you have real time duplication enabled (default), then when a file is copied to the pool, all copies of that file will be written to the destination drives in parallel.  This is specifically to prevent issues with "locked" (open) files, so they are properly duplicated without any problems. 

 

 

That said, if you perform 4 copies to the pool, yes, it can handle that without a problem.  However, depending on the disk configuration, this may end up slowing down the process, if it's writing to the same disks, at the same time. 

[donnatronious]

I appreciate the response, however, there are no copies whatsoever going on in the question.

[Christopher (Drashna)]

If you mean copying from the pool, then.... 

 

Then we do have some performance optimizations here. Specifically "Read Striping" which will read from both disks, or will read from only one of the disks, if one of them is busy/active. 

 

http://stablebit.com/Support/DrivePool/2.X/Manual?Section=Performance%20Options

[thepregnantgod]

Drashna, am I reading this correctly that if I install Robocopy or say Terracopy (since both are Windows Explorer type add-ins) then Drivepool will use their accelerated features to increase copy speed?

[Christopher (Drashna)]

Drashna, am I reading this correctly that if I install Robocopy or say Terracopy (since both are Windows Explorer type add-ins) then Drivepool will use their accelerated features to increase copy speed?

 

*can*.  But yes. 

 

I do emphasize "can", because it absolutely depends on what files are being accessed, and where they reside (what disks).  Also, duplication can/will affect this, as well. 

 

Unfortunately, these programs are not "DrivePool" aware, so they go through the list linearly, rather than performing any sort of optimization.   Because of this, it may not actually help, but may hinder the copy, as it would on a normal disk.

[Christopher (Drashna)]

And before you ask.... IIRC, a program could query the kernel driver for the actual location of the files (much like how dpcmd works), and "know" what disks are being used.... so it could potentially optimize the performance. 

 

I've already added a request to either build something that can do this, or at least document how to, for other developers. 

https://stablebit.com/Admin/IssueAnalysis/27450