Q: How fast does DrivePool re-measure your entire (USB) pool? A: ~21 TB/hour

[gtaus]

FYI, I currently have a 71.2 TB DrivePool with 18 HDDs and 1 SSD. I just replaced an older HDD with a newer HDD and DrivePool went into a complete Re-Measuring task of all drives. Just for info, I wrote down some stats to see how "fast" it would take for DrivePool to completely remeasure my pool, with nothing else running on the server.

My pool is 71.2 TB, of which 66.8 TB was data and 3.07 TB was free at the start. I know that does not add up exactly, but it's close enough for this example. All my HDDs are connected via USB 3.0 and the 224 GB SSD is attached to the motherboard. After tracking the "measuring" stats for 3 hours 9 mins, I found DrivePool was able to measure 66.8 TB over 3 Hours 9 mins for ~21.2 TB per hour read on the re-measure task.

As I said, I had no other operations going on my DrivePool server at that time of the re-measure task. But, I think you still have both full read/write access to the DrivePool files when a Measuring task is running. If not, maybe someone will correct me. The only time I have seen DrivePool go into "Read-only" mode is when there is "Disk Missing" from the pool and you get a notification. 

My setup: I have an older, ~8 years old, HP desktop computer with an AMD A4-5300B CPU running at 3.40 GHz and it has 8 GB DDR3 RAM. Nothing to brag about. But it works great with DrivePool.

If you have a newer computer with a faster CPU, you might get better results than I did. If your HDDs are connected directly to the motherboard, then I suspect you would have much better results. If so, please post your results for comparison.

 

[Spider99]

For me a remeasure is about 80-90mins for 45TB with a similar number of disks - the make up of the files on the disk will affect this greatly - lots of small files = slow

i recently got rid of 500k of images - which accounted for a lot of the time above - but have not remeasured since

with the HDD DP disks connected to a 9211

the speed is mainly affected by the files sizes/number, controller and HDD speed - cpu is only a very minor player in this

 

[gtaus]

@Spider99, thanks for the reply. You are probably right in that the CPU has less affect on the transfer/read of files than the make up of the files on those drives (I had lots of small files), or the controller used (I use USB 3.0), and the HDD speed (I have a mix of 5600rpm and 7200rpm). I guess I was thinking that maybe a multi core processor would allow more reads at the same time, and then the re-measure task would take less time.

I am in the process of removing directories full of small .mid and .kar files I have collected over the years. I am dumping them off to archive HDDs to be stored in the closet. After I get those files off of DrivePool, it will be interesting to see if there is a substantial improvement in time it takes to measure the pool. I can already see that it takes "forever" to move those small files one by one to a backup drive. So, once those small files are gone from DrivePool, maybe I'll run a retest on the Measure task to see if actually makes a significant improvement.

[Spider99]

the usb controller will be the bottleneck for 7200 rpm drives especially if they are newer models

i tend to shuck my usb drives and a couple of WD Blacks i got recently can do 240MB/s plus once they have room to breath 

[Spider99]

@gtaus remeasure time now less than 30 mins for 45TB of data

[gtaus]

4 hours ago, Spider99 said:

@gtaus remeasure time now less than 30 mins for 45TB of data

Yeah, that's a lot faster than what I am getting. I just offloaded about 2TB of really small files that I had in DrivePool. I might get a chance to re-measure the entire pool sometime this weekend to see if that made a difference. My USB 3.0 HDDs have a write speed of about 80-100 MB/s, so I am thinking that is my bottleneck. Fortunately, one can still use DrivePool in full read/write while the re-measure task is running, so although speed is always important, I could live with letting the task run overnight.