pest Posted December 30, 2023 Share Posted December 30, 2023 I've tried to read up on balancing limitation but I am either still missing something or my setup is not configured/working correctly. NAS Configuration: Folder duplication on the entire drive pool (including temp). NAS Hardware: 2x 500GB NVME WD Black drives (temp SSD optimizer storage with rules to auto balance) 6x WD RED's varying (3x10TB, 1x6TB, 2x4TB) I'm able to transfer from workstation to NAS > 1000MB/s over 10gbe to the NVME temp. When the NAS has MULTIPLE files to balance, it only migrates (and maxes out) 1 HDD at a time (see attached). My attachment is transferring ~10 files of ~40GB from my temp drives to the spinning rust archives. At least 4 out of 5 HDD have open space to transfer to based on the 90% fill rule. Currently it moves from 1 NVME to 1 HDD at a time, maxing out the HDD speed for each drive. Shouldn't this hit AT LEAST 2 HDD (1 for each NVME) or better yet max with multiple HDD per NVME given available multiple file transfers. In this example, my 2 NVME drives can saturate 5 HDD throughput of 10 files of 40GB at the same time. What am I missing? Does the single thread limitation still apply for multiple files from multiple drives to multiple sources (lol) ? Quote Link to comment Share on other sites More sharing options...
0 Shane Posted December 30, 2023 Share Posted December 30, 2023 I believe the short answer is that single-file-at-a-time balancing was by far the simplest and safest to code*, so guaranteed reliability (and more time to code other things) was chosen over raw speed. *In my very limited and outdated experience, multi-threaded operations are fantastic when you can just tick a checkbox in a compiler that can safely (hah) optimise it all for you, and a complex pile of risk conditions when you have to write it yourself. Quote Link to comment Share on other sites More sharing options...
0 pest Posted December 30, 2023 Author Share Posted December 30, 2023 Yea I couldn't find if it was truly only single file or single file to a single drive, where in my case there are multiple files and multiple drives to be utilized. Not sure how many others use multiple temp drives so maybe I'm a fringe case. Quote Link to comment Share on other sites More sharing options...
Question
pest
I've tried to read up on balancing limitation but I am either still missing something or my setup is not configured/working correctly.
NAS Configuration:
Folder duplication on the entire drive pool (including temp).
NAS Hardware:
2x 500GB NVME WD Black drives (temp SSD optimizer storage with rules to auto balance)
6x WD RED's varying (3x10TB, 1x6TB, 2x4TB)
I'm able to transfer from workstation to NAS > 1000MB/s over 10gbe to the NVME temp.
When the NAS has MULTIPLE files to balance, it only migrates (and maxes out) 1 HDD at a time (see attached).
My attachment is transferring ~10 files of ~40GB from my temp drives to the spinning rust archives.
At least 4 out of 5 HDD have open space to transfer to based on the 90% fill rule.
Currently it moves from 1 NVME to 1 HDD at a time, maxing out the HDD speed for each drive.
Shouldn't this hit AT LEAST 2 HDD (1 for each NVME) or better yet max with multiple HDD per NVME given available multiple file transfers.
In this example, my 2 NVME drives can saturate 5 HDD throughput of 10 files of 40GB at the same time.
What am I missing? Does the single thread limitation still apply for multiple files from multiple drives to multiple sources (lol) ?
Link to comment
Share on other sites
2 answers to this question
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.