Reasonable time to evacuate a drive?

[fleggett1]

Is it common for an almost-full 16 TB drive evacuation to take several days?  I started the process this past Tuesday and it's not going to complete until tomorrow (Friday).  The source and target are bog-standard 6 Gb/s drives (WD and Seagate).  I haven't done the math, but the time it's taking just seems a little excessive.  All drives are in an external Sabrent enclosure.

[Shane]

While a SATA cable plugged directly into a SATA controller can in theory support a 6 Gb/s transfer rate (before overhead), the transfer rate on a mechanical drive isn't likely to come even close - e.g. a 16TB WD Red Pro's ideal sequential read rate benches a little over 2 Gb/s - and that can drop way down if the drive is being asked to pull many small files non-sequentially (as little as 0.1 Gb/s or less in certain scenarios). If you've an external enclosure doing SATA to USB, the USB connection can be a bottleneck as well.

That said, DrivePool is sluggish in evacuating drives; AFAICT it methodically removes each file before starting the next with no queueing optimisation, so it taking noticeably longer than it would take for you to do a bulk move of the contents yourself wouldn't surprise me.

[fleggett1]

Well, I'm using a Sabrent external enclosure - https://sabrent.com/products/ds-uctb - so I dunno what sort of performance hit I'm running into versus if the drives were plugged directly into a motherboard.  But if you think several days to evacuate a large drive is reasonable, that's fine, I just wanted to make sure I wasn't seeing a big problem with my setup.

[Shane]

Likely very little difference versus plugged directly into the motherboard.

It is possible to evacuate/move files much faster if you don't mind stopping the DrivePool service while you manually move them yourself (from hidden poolpart to hidden poolpart) then restarting the service and requesting a re-measure. Basically a tradeoff between speed and comfort, if that makes sense.

[gtaus]

On 8/13/2023 at 11:01 PM, Shane said:

It is possible to evacuate/move files much faster if you don't mind stopping the DrivePool service while you manually move them yourself (from hidden poolpart to hidden poolpart) then restarting the service and requesting a re-measure. Basically a tradeoff between speed and comfort, if that makes sense.

 

That's the info I was just looking for, how to speed up removing a drive from DrivePool. Currently, I am removing a 4TB drive to replace it with an 8TB drive. I logged the progress of the task and it took 3 hours to move the first 5.5% of the drive. At that rate, it will take about 54 hours to complete the removal task. Good thing I started the removal process 3 days before the new HDD is to be delivered.

Any idea on how much overhead DrivePool adds to the process in the drive removal process? With larger and larger HDD's, this is becoming more of an issue for me. I like to keep my DrivePool up and running while I remove a drive, so I plan for a good 3 days to remove a 4TB drive. But it would be nice to know how much longer DrivePool takes to remove a drive compared to stopping DrivePool and manually moving them yourself.

[gtaus]

On 8/13/2023 at 8:52 PM, fleggett1 said:

Well, I'm using a Sabrent external enclosure - https://sabrent.com/products/ds-uctb -

That looks like a real nice 10-bay enclosure. When I started out with DrivePool, I had two 4-bay enclosures and a handful of individual HDD enclosures.

Unfortunately, after a few years, one of my 4-bay enclosures died so I lost access to all 4 drives. It was less expensive for me to replace the 4-bay enclosure with 4 separate HDD enclosures, so that is what I did.

Currently, I see the 10-bay enclosure sells for about $540 on Amazon. That's $54 per slot. Again, I am comparing that price to individual HDD enclosures at about $20 per case. I know the multi bay enclosures have many advantages, but I wish they could bring down the price. Also, for me, I don't think I would ever lose 10 individual HDD enclosures all at one time whereas I know that if the board in the enclosure goes down, I would lose access to all those drives.

Do you think your 10-bay enclosure is moving files among the drives faster than what you could see if you had separate HDDs in individual cases? From your post, I don't see any great speed removing the drive within DrivePool itself. Well, frankly, DrivePool is not nearly as fast as my old RAID system or Wndows Storage Spaces, but DrivePool just works better, and I have had no catastrophic data failures as I did in the other systems.

I am currently removing a 4TB HDD from my 23 HDD DrivePool Media Server. I calculate it will take about 54 hours to complete the task within DrivePool. That's a long time, and with my newer 8TB drives, I suppose it will take almost a week to remove one of those.

I think it would be great if DrivePool had some kind of way to estimate how long it would take to complete the remove drive task. I see my remove drive task is clocking at about a 40 MB/s rate but I think the HDDs can transfer over 80 MB/s on their own. And I have an older computer with a USB 3.0 system. I'm sure the bottleneck in my system is the USB 3.0. So, just curious on what transfer speed you see on your 10-bay enclosure as you remove your drive. Thanks.

[Shane]

2 hours ago, gtaus said:

Any idea on how much overhead DrivePool adds to the process in the drive removal process? With larger and larger HDD's, this is becoming more of an issue for me. I like to keep my DrivePool up and running while I remove a drive, so I plan for a good 3 days to remove a 4TB drive. But it would be nice to know how much longer DrivePool takes to remove a drive compared to stopping DrivePool and manually moving them yourself.

I know the overhead's more noticeable when moving smaller files, but I haven't kept any hard numbers sorry. I also try to always keep at least one spare port (internal or external) available so that I can just plug in a new drive and let DrivePool take care of the rest, and everything at least x2 duplicated (and backed up) so if I'm in a hurry for some reason (e.g. a drive failing in a way that panics the OS) I can simply just ditch the culprit and let DrivePool handle re-duplication.

I'm shifting things around at the moment but I've a mostly-full 4TB drive (internal SATA) I could test removing both normally and manually afterwards if you're interested (not that such a test would be rigorous, since pool content varies, but the offer's there). Do you use duplication?

[Christopher (Drashna)]

There isn't a set amount of time, because tasks like balancing, duplication, etc run as a background priority.   This means that normal usage will trump these tasks.   
Additionally, it has the normal file move/copy issue, estimates can jump radically.   A bunch of small files take a lot more time than a few large files, because it's updating the file system much more frequently.  And for hard drives, this means that the read/write heads are jumping back and forth, frequently.  

But 6-12 hours per TB is a decent estimate for removal. 

[gtaus]

On 9/20/2023 at 3:42 AM, Shane said:

I'm shifting things around at the moment but I've a mostly-full 4TB drive (internal SATA) I could test removing both normally and manually afterwards if you're interested (not that such a test would be rigorous, since pool content varies, but the offer's there). Do you use duplication?

I don't use duplication on my Home Media Center storage DrivePool. I have all the files backup up on HDDs in the closest. So, if I lose an HDD in DrivePool, I could reload them from the backups. Nothing critical if I lose a drive.

As far as time required for removing a drive, I initially estimated 54 hours to remove my drive after the progress in the first 3 hours. However, my transfer rates improved over time and the remove drive task completed last night at about 25 hours total run time. I suspect it's a matter of what files are being transferred. Small files will take considerably longer to transfer than larger files. I can see the transfer rate drop way down on those small files, but build up speed on larger files.

Thanks for the offer, but you don't have to perform any comparison tests for me. I am just hoping that DrivePool will someday include an "estimated time to complete" for the remove drive task. The % progress in DrivePool is OK, but it really does not give you any idea how long it will take to complete. 

[gtaus]

On 9/20/2023 at 12:25 PM, Christopher (Drashna) said:

There isn't a set amount of time, because tasks like balancing, duplication, etc run as a background priority.   This means that normal usage will trump these tasks.   
Additionally, it has the normal file move/copy issue, estimates can jump radically.   A bunch of small files take a lot more time than a few large files, because it's updating the file system much more frequently.  And for hard drives, this means that the read/write heads are jumping back and forth, frequently.  

But 6-12 hours per TB is a decent estimate for removal. 

 

Yep, it took 25 hours to remove my 4TB HDD. As I stated, I could see the transfer rates going up and down in speed depending on the size of the files.

Although speed is important, I have stayed with DrivePool for years because the program just works so well and does not crash like other systems I had including RAID and Windows Storage Spaces.

[fleggett1]

gtaus, the Sabrent is a really nice enclosure.  Looks to be all-metal and would undoubtedly kill someone if dropped on their head, even empty.  Doesn't even need drive sleds, which a lot of server-style enclosures force onto people, like my old Norco (now Secdin).  On the expensive side, but the build quality and it only needing the one USB-C port are terrific selling points.

Unfortunately, I have no idea regarding drive speeds, as I haven't run any tests.  I would guess that it's just as quick as if the drives were directly connected to the motherboard.  I've streamed several near-100 GB UHDs and haven't run into any stuttering, which is all I wanted.

It sounds like DP is, to put it charitably, "slow" when it comes to drive evacuations.  I'm not sure a faster USB connection would help in this regard, as DP seems to have its own game plan when shuffling stuff around.

If you have any other questions about the Sabrent, feel free to ask.