Only adding files to single drive in pool

[phhowe17]

Empty pool of 10TB and 14 TB drive.  Copying files to the pool exclusively uses the 14 TB drive.  Doing a measure does not trigger the ability to rebalance.  Drive pool v23.10.1661.  Balancers are set to default. Pool status line is cyan.

Source files are another pool.  Using a multi-thread file copy tool.

Small drive

     Total Files Listed:
             754 File(s)        12,005,032,379 bytes
             450 Dir(s)    9,987,804,217,344 bytes free

Large drive

     Total Files Listed:
           412413 File(s)   4,009,345,041,775 bytes
           345027 Dir(s)  9,987,096,260,608 bytes free

 

[Shane]

(skip to the TLDR at the bottom if you just want a specific solution)

Volume Equalization is for any physical disk that has been partitioned into multiple volumes that are part of the pool, while the Disk Space Equalizer is a scheduled balancer (if you rest the mouse cursor over a balancer the tool-tip mentions "can set file placement limits" only if it's able to act in real-time) so unfortunately neither would help here.

(Note that DrivePool preferring to write to the drive with the most free space is because it's impossible for a file system to guarantee knowing the size of a new file, the software's design prefers reliability over performance when it has to choose).

That said, it's not impossible - just a little fiddly - to get DrivePool to write separate files to separate drives in the pool at the same time. First you need to have separate copy operations happening at the same time: if you use Windows Explorer to highlight two folders and drag them to the pool Explorer will only copy the contents to the pool one file at a time, whereas if you drag each folder separately to the pool drive Explorer copies them concurrently.

Second - this is the fiddly part - you need to either (1) have close enough amounts of free space on the pool's drives so that DrivePool picks a different drive in the pool for each operation as each drive's free space drops from receiving incoming files or (2) set File Placement rules such that DrivePool is forced to use different drives (e.g. setting a rule that folders/files starting with A go on disk 1 and a rule that folders/files starting with B go on disk 2).

TLDR ~~~> In your scenario, you could create two folders on the destination pool, e.g. "Source 1" and "Source 2" then use File Placement to allocate different disks in the pool to those folders; then you copy from one source into Source 1 and from the other source into Source 2. Once you've got everything into the pool you can turn off those rules or move the content into other folders on the pool, and let the pool balance overnight or whatever.

[Shane]

DrivePool's default behaviour is to write to whichever drive* has the most free space; given two empty drives, one large and one small, it will write to the large until the free space is less than the small.

Presuming that you started with an empty pool on empty drives, and the stats in your post are the end result, notice that the bytes free on the two drives are very close.

*(if you have real-time duplication enabled, drives plural)

[phhowe17]

Have the Volume equalization enabled. I figured that would balance writes across drives.  I'm trying to improve performance during multiple file writes to the pool.  I'm copying about 4 TB from two different source drives.  I sort of gave up and just did the copies directly to the underlying pool part folders.

Would disk space equalizer plug in set to "equalize by space used" help?

[Christopher (Drashna)]

On 11/15/2024 at 1:04 PM, phhowe17 said:

Have the Volume equalization enabled. I figured that would balance writes across drives.  I'm trying to improve performance during multiple file writes to the pool.  I'm copying about 4 TB from two different source drives.  I sort of gave up and just did the copies directly to the underlying pool part folders.

Would disk space equalizer plug in set to "equalize by space used" help?

Volume equalization works between volumes on the same disk.  You want "Drive Space Equalizer" specifically.   It should be disabled by default. 

[Shane]

Hi Chris, I was under the impression that the Disk Space Equalizer balancer is not capable of real-time file placement (since it's not mentioned in its tooltip)?

 

[Christopher (Drashna)]

Correct, it doesn't effect realtime placement, unfortunately.