Can I double LLM GGUF files reading speed using two SSDs?

[MaZeNsMz]

Hi there,

I am wondering if this app is able to double the GGUF files speed if they are duplicated over two NVMEs.

 

I tested a 60GB GGUF file, duplicated on two NVMEs. When I started performing inferences, I see the app actually reading only from one of the NVMEs. The Read strip option is on, and the balance is also performed (60GB on each disk)

Looking forward your kind reply.

[Shane]

Hi, I'd suspect either of:

• the program isn't requesting the data faster than just one of your NVME drives can supply it
• or the program is requesting data in a way that isn't triggering the striping algorithm.

See https://stablebit.com/Support/DrivePool/2.X/Manual?Section=Performance Options for more details on what triggers read striping.

You could test the feature by placing the 60GB file on a pool of slow disks (e.g. two SATA HDD or two USB HDD), with 2x duplication and read-striping enabled, then:

1. Copying the file from the slow pool to a NVME target and observing the source reads (DrivePool should perform read-striping from both HDDs because each should be far slower than the NVME SSD target). This should confirm that striping is working as designed.
2. Presuming #1 passes, performing your inferences using the file on the slow pool and observing the source reads. If it stripes then you have confirmed the bottleneck is the app. If it doesn't stripe then either the app's requests aren't enough to even saturate a slow drive or something strange is going on.

[Christopher (Drashna)]

Just a heads up, read striping doesn't guarantee that it will read from the different drives.