Tarh

Sort of solved it; since the file placement rules are a priority list, stuff closer to the bottom gets ignored. After removing and readding and moving several rules to the top, after about a dozen of balancing passes, it finnally moved all the files. It still for some reason at first only moved to files to the cache and the next balancing put them to their place but whatever. It seems that after like a year or so stuff always gets wonky with DrivePool and I have to reset settings to get it working flawlessly again but resetting is a major PITA.

Hey, I have 12 drives, 10 archive HDDs, 1 SSD cache landing drive and 1 SSD archive for millions of tiny files for fast access. I have a few dozen file placement rules set for various file extensions (eg. .MKV) so that tiny files are always on the SSD archive and big files like videos are never on that drive. It worked fine for a while but now it doesn't and the archive SSD is being filled with media files and the HDDs with tiny files. The only plugin enabled is SSD Optimizer with 98% fill options and each file placement rule is set to "never allow files to be placed on any other disks" which is being violated. The drives are between 92%-95% full but even if I get a drive to like 85% and balancing after re-measuring still doesn't respect the file placement rules and doesn't move anything. The files are not locked. "Allow balacing plug-ins to force immediate balancing"; "File placement rules respect real-time file placement limits set by the balancing plug-ins"; "Balancing plug-ins respect file placement rules" are all enabled. But it all worked flawlessly in the past, not sure what changed, other than the drives being more filled but now it doesn't work.

If some disks disconnect DrivePool will not re-add them automatically. It needs a restart of the service or restart of the PC. I have some HDD dockers that can go down for like a minute in case of an outage as they are not UPS protected and DrivePool will say the disks are missing indefinitely and there's no apparent way to get them reconnected to the pool.

I guess this can be closed...

Update: Every time I started a rebalance, it started moving stuff onto the landing drive as I said above. Even with clear extension rules going against that. But, After emptying the drive by manually moving everything to archive drives then re-measuring and then re-checking (which somehow goes twice from 0-100%), rebalancing (which took a long time and did nothing) now it works as intended, but seems much slower but I can get behind that. Rebalancing also takes like 15-20 minutes until it hits around ~50%, and until then disk IO is literally 0% and only after that point it actually start moving files. Either way, now it's working and respecting the rules.

Hey, so I replaced a HDD and I have a landing SSD that gets filled and DrivePool did a great job of pushing files on that drive onto archive spinning rust with each balancing. But ever since the replacing of one of the HDDs, balancing takes a long time but there's zero file movement (as per Scanner, drive activity is 0). I've seen several old post about the same issue. There's literally zero warnings or errors in the logs (only "[Rebalance] Cannot balance. Immediate re-balance throttled." "[Rebalance] Not enabling background I/O priority" Information), it's just does nothing for some reason. In fact it moved some files from archives to the SSD that goes against every rule but then it stopped?! I did an update, reinstall, several remeasuring and rebalancing and even after 1-2 hours no files were moved at all! I really don't want to reset the settings as I have several dozens of extension-related rules (tiny files like pics are stored on a different SSD for overall faster access, and big fileson 10x archive drives and we're talking about 2M files). Before balancing it says that the "unduplicated target for rebalancing is -474GB" on the landing SSD and the archive drives each say positive ~40-48GB as it always did, yet balancing doesn't actually empty the landing SSD drive, it just does nothing for an hour or so. Nothing runs that keeps these files open. I'm really lost and annoyed here as I have to manually move files so that they're protected by SnapRAID.

Christian helped me suggesting the use of file placement rules based on file extensions for small files. Works flawlessly for my usecase. Thanks!

Hey, so I have a 10 disk HDD pool with 2 parity and SnapRAID and I'm considering adding an SSD and wondering if I can set DrivePool up somehow so that it'd put all tiny files the pool recieves to the SSD? Like everything under <100KB going to the SSD and the rest to the HDDs and obviously don't fill the SSD with big files. I have tons of small files that are a pain to work with on HDDs but I don't really want to manually put small files to the SSD. If it can't be done as is, would a plugin be able to do this? Basically if a file is smaller than a set amount it would go to the specific drive (if it has enough space) and the rest of the files going to the least filled disk as is the default case.

Hi! So playing with the SSD caching and a torrent client, I realized that for some reason file preallocation is always done at around 32MB/s on a SATA SSD. Which is much faster even on a HDD. No other software interference, raid controller, cable quality, trim, wear or any issues exist and write caching is of course enabled. I know because if I manually start moving the same files it does it with multiple hundreds per seconds a megabyte - as it should - during this very slow prealloc. So there's no hardware bottleneck. And no, we're not talking about thousands of tiny files, they are all 1GB+ files a piece. Now, I'm not 100% sure it's DrivePool's fault, but then again, even after many hours of various tasks and benchmarks I can't reproduce any slowdown on the SSD. And since qBittorrent is preallocing files much faster on a HDD, I'd hazard a bet that it's not its fault. I've adde an SSD cache to be faster, yet it's slower than my bandwidth so something is very wrong here. https://i.imgur.com/57CPrI7.png So what is going on, why is this so unbearably slow? Edit: I just learned the "increase priority" fast-forward icon while balancing (well hidden) today, so is it possible prealloc is being limited by some option here?

I know this has been discussed a few times and I more or less understand the issues and limitations, but I was wondering if it would be possible to give the balancer a bit of a leeway with the free space based balancing logic. What I mean is by default the balancer is putting files on the disk with the most free space. Great. If the next file in the queue would go to another drive (because the actively copied file makes the first target disk less free than another disk) we now have two files being copied and so on. Sounds fast and great, except this is very rare so the SSD is almost always only feeding and is limited by one HDD - which we're trying to avoid as it sucks, even without SMR drives. Like an NVMe SSD cache writing at like 100MB/s is painful. The drive fills faster than it offloads. So, what if we could give the balancer an optional threshold to be more flexible or in another words less strict with only pushing files to the least filled drive? Surely then it could push files to multiple drives at once in a default setup. For this theoritical threshold, let's say if we used the value 5%, then basically the balancer would start copying to the least filled drive as it does now, then it would look at the rest of the drives as if they had 5% more free space (respecting fill limits) therefore it could use them to push files simultanously. Obviously only copying one file to one HDD at a time (better performance, less fragments, as it is now). With a theoritical secondary option we could limit how many threads it could do at once, or just a built in one thread per one HDD (which is how it already seem to work). This way, given enough SSD bandwidth, the SSD could feed each drives at once until it's empty most of the time instead of when very rarely the stars align. I don't believe this would be overly complex (like multithreading) and I think it would make many people's lives better who use SSD caches. Unless I miss some major issue.

Hi, so I'm using DrivePool with SnapRAID and I added a few drives a few months ago and plan on adding more in the future. My small issue is that new drives are getting filled with all the new files which makes the combined performance of those files worse than if they were all across the drives. Imagine having 8 drives filled at 20%, and I add 2 more drives. So now the new files are only on the 2 drives until they reach 20%. I'd much rather have new files on all 10 of them roughly evenly and I can do some manual balancing safely if needed. So how can I do this, this being basically a round-robin sort of file placement across all disks regardless of available capacity?

Yes, I've tried that. Both Direct I/O and Scanner fails to see Samsung NVMe Health.

Hi, I know it's been asked before but I'm still wondering if we can expect NVMe "smart" support in the future for these drives. In Direct I/O everything is a red cross except the "SSD" part with every option, combination of options and specific methods. The devices in question are listed as "Disk # - NVMe Samsung SSD 960 SCSI Disk Device" Using Scanner version 2.5.4.3216 and Samsung NVMe Controller 3.0.0.1802 driver. CrystalDiskInfo and some other software can get the data.

It wasn't a Drivepool issue. Sorry, my bad.

I just did, thank you!