I am sorry for this post, but it needs to be said.
I was so exited when I found DrivePool and Scanner about 2 years ago, that I bought the whole "package" right then and there. I think I even own a licence for CloudDrive, though I never tried it
I've been keeping myself from posting a rant for a looong time now. But after reading this thread, it's just too much...
Long story short - my setup is 50+ drives in one pool, with a million plus files. A lot of points made below stem from this - having lots of drives with a shitload of files.
Scattered all over files
1. DrivePool, for some ungodly reason, scatters files willy-nilly, like a shot from the shotgun (I know there are reason(s), I read something by Alex somewhere, but I don't think, they're good enough to warrant this situation). At least it's by default. And actually that's a "preferred" type of operation, as I understand. Why there isn't an option to keep files together in the same folder structure, is beyond me. For example how unRAID does it. Further-more unRAID could get away with scattering files wherever, because it uses parity for protection, not duplication. Drivepool, on the other hand - cannot get away with this (more on that later).
IMHO, this is the biggest flaw in DrivePool, so it bears repeating. No folders splitting. Or rather a really good system to setup the folders splitting. Again - look at unRAID and do what they're doing. i.e.: splitting of TV Shows when copying to the pool. Now - all the shows have most of the episodes scattered on all 50+ drives. Stupid.
2. Simple real life example (my home server). At the "height" of my excitement with DrivePool, and by that I mean the days just after I found/bough/installed it on my server, I put all my spinners (on my server) in a pool. That's 50+ drives in a single pool. I loved the convenience of the pool and navigating/finding everything easily. I have setup 2x duplication on some folders, not all the of pool (not all files need protecting). Anyway, some weeks later my first drive died, while using DrivePool. Fuck. Not because drive died - no - because the first thing I obviously needed to know was - what files have I lost. I came to the forums, and quickly found, that even though DrivePool's half of functionality is file security (by a way duplication) - it does not keep track of files on the pool, in any shape or form. Fakme. Now that was stupid, I thought to myself. Especially, if you gonna throw my files around the disks, where I have no idea where they actually are - how the fuck can you not keep track of them, so you could tell me, what I've lost if a drive dies? And up to this day, I have no idea what I've actually lost, just that it was ~3TB of stuff. Love it
After that happened, I had a thought and stopped using duplication. Actually, of course, the first thing I've done was to setup auto-scanning of the DrivePool pool every 6 hours with WhereIsIt, so the next time a drive dies, I would at least be able to know what the hell I've lost. Once again - I need a third party tool to keep track of my files. I'm sorry, but this is inexcusable.
Please, do not mention the "Ordered File Placement" plug-in. Of course, drives need to be filled evenly, not one at a time. For speed, for wear and tear, what have you. Just simply, don't scatter files - keep them together.
3. A shitload of empty folders in the drives are driving me nuts. With this scattering of files there are probably more freaking empty folders on the drives than actual files. Dafuk.
4. At the very least - implement file and folder access in the balancer development (IIRC, now you can't), so I could write my own implementation of keeping files and folder on the same disks, if you don't want to... Because this madness with file placement is making the program unusable. Reasons - below.
5. I have been using DrivePool for the last 1,5, at least - just because I am too lazy to disband the pool and then manually restore the order in all that mess that DrivePool created. But I'm finishing up writing a little program, that will do it for me, so fingers crossed... I might get back some sanity in my file system, soon. Anyone know of a tool which would help in this? Really lazy to code now...
6. I've been eagerly waiting for grouping of drives, to finally make duplication usable. Yes, I said it. Because as it is now - duplication is unusable. Not even close. Anyone having 2x duplication and lots of drives and lots of files is actually bound to loose a shitload of files in case more than 1 drive fails at the same time. And with quality of drives nowadays - good luck... And if you don't believe me, think about it - I'll describe my reasoning in the following points:
Like I mentioned before, after loosing my first drive in the pool, I disabled the duplication. The reason was simple - I did some thinking. I have 50+ drives, probably a million plus files. DrivePool scatters files to as many drives as it can, whenever it can. If 2 drives die - how many files would actually have both copies on those drives (1 copy on each drive with 2x duplication). The answer is not zero. 100% - some files would get lost. And if someone offers me to do 3x duplication, then can I haz some of your money, plz... It's been too long for me to remember how to actually do the probability math, but simple logic dictates - that duplication would not help me at all to keep all my files safe, in case 2 drives go tits up at the same time. Basically, the more drives you have, the more files you have - the more fucked you are with 2x duplication. And remember... on top of that you will have no fucking idea, what files you have just lost... enjoy...
7. Now I see the drive groups arrived. And it's still does not fix the above problem. Am I understanding this correctly - you can make same old-style pools which can have duplication settings inside them and separately from one another. And then you pool those pools together? So they do not duplicate from one pool to another pool? Why the hell not? Isn't this, like, the first and only thought process, how to do this type of drives grouping, with the duplication in mind? What the hell, seriously... However many drives you have - split them in the middle by the age (how old they are) and that's your 2 groups. This way, all the oldest drives are duplicated into the all the newest drives. And you are sure, that each file has at least one copy in a new drive. Isn't that obvious? Want more than 2x duplication - split into more groups, but again, use common sense and group them drives by age. And duplicate between groups!
8. And anyway, duplication of 100TB+ or more of media files, seriously? Would love to be that rich, so it wouldn't hurt the wallet. How about implementing a SnapRAID over the pool. With cache/landing disk (or landing disks group) that would be easy. To keep all the files always secured:
files get copied to duplicated landing disks.
Then, in the background they get copied to the snapraid protected pool drives.
Snapshot gets updated.
Now that files are properly secured on the pool by snapraid - they can be deleted from the landing disks.
Result - no wasted money and drives on a stupid duplication and files are protected way way better with multiple parities, if you want, and checksumming against bitrot and some-such. If it's impossible for you to use another free program like that in your paid software - give as an option to do it ourselves, by keeping the fucking files together. Because, I would've done things this way ages ago, if not for the stupid shit DrivePool does whenever it wants.
9. Balancing. Don't get me started... Stop this fucking Apple-like magic, where program does whatever it wants, and I have no clue what the hell is going on. Instead, show it to me. Because magic-like operation is nice, sometimes, but not here. And it must work perfectly 100% of the time. If not - I'm an idiot who doesn't know what the hell is going on with his own files on his own computer. And DrivePool is far from 100% perfect. And I'm not expecting it to be. It is a mirracle, as it is, with only 1 dev working on it.
10. This bears repeating. Show me what the fuck DP is doing and especially - why. Not the same single meaningless phrase on the GUI - "moving files because whatever". But actually, moving these files from this drive to this drive, because of this balancer, or these file placement rules changes, etc...
11. Add an option to simulate balancing and let me choose what or if I wanna do it.
12. Add option to force re-balancing. Better still, to force re-balancing on individual balancer.
13. For the love of god - give me an option to manually control (start/stop) the balancing. Let me select which drives I want to balance, with which balancer(s), so when I fire up the balancer, it would not go and do some shit I do not care for and waste time and my patience. When I change the file placement rules, I probably always want to balance something specific. Just... let me.
14. It would be nice, to able to prioritize balancing actions. Like a queue. This would be very helpful on big re-balancing jobs. That's, if we ever get control of balancing.
15. SSD Optimizer title for a balancer is confusing. Change it to Landing Disk or Cache Drive or something similar. Because that's was it does, basically.
16. The minimum free space should be set in one and only one place . Multiple balancers have it - why!? Also, there is percent and size based settings at the same time - again - why? In short, leave the free space settings only in Prevent Drive Overfill and that's it, only with ability to set free space per drive basis, if we want. Drives now come in 1-10TB, most commonly, which is a huge difference.
17. Filter (file/folder placement rules) also by size. eg: maybe I would want all the metadata for movies and tv shows to be placed on the ssd, so the spinners would contain only big video files. Less fragmentation. Speed accessing metadata. That'd be neat.
18. OK, let's move on onto another awful thing. The GUI. Talk about style over substance. Should I drop a screenshot of the DrivePool GUI in full-screen on a 21:9 ultra-wide screen monitor, or is it easy enough for anyone to actually imagine, how ridiculously unpractical it is, without even seeing it? Starting with having a program in 3440x1440 resolution and still not being able to see even half of my drives in a list. Stupid wasted desktop real estate galore. I have never ever seen anything like this before, ever.
19. The drives list is also wasted space. And of course - there's no way to see all your drives at once. Have too many drives - scroll through this one listview, never-mind there's plenty of space all around for a different view displaying more/all the drives in a wrap panel.
20. Disk performance view at the bottom right - you will see activity for only 5 files - and that's it. That's all you get. What the hell? I seriously wonder, how this decision got made. The main reason the program was created, was to manage files - but if you want to know anything about what's going on with those files - I'll show you only 5. Fuck the rest. You have 10 transfers... more...? Does not matter - 5, I say. Better look at that huge pie-chart, pretty, isn't it?...
21. Want to change some settings for the pool? Here's a universal icon for settings at the top on the right, nice and pretty intuitive... oh, wait... shit. That is not the options - what the hell? Oh here's options below that huge pancake in the middle. Oh, it's not a button to open the Settings - it's a menu, with sub-menus? Oh, I see, some menus are check-boxes and some actually open the settings window. What? There are multiple separate settings windows for all kinds of things.. Couldn't all of them be in one settings window, as per usual? Oh, it would be too easy, intuitive and too simple to use, you say? OK, then...
22. I could go on like this for quite a while longer, but fuck it. There's more important things. During probably almost 2 years of using DrivePool and Scanner - I can confidently say, that 8 out 10 times I tried to control DP or Scanner on my server from my main rig - I could not. The trial on main rig expired ages ago. The drop-down menu selection to connect to server - there's no selection but the local rig. 8/10 times there's no server option. I can't connect and control the actual DP and Scanner that I use. So now I have 2 programs installed in my main rig for no fucking reason. Beautiful.
23. I have a shit load of drives. I want to keep some folder on a single drive. I go into settings > File Placement > and either Folders or Rules, does not matter. Then I need to un-check all my drives, but one. One by fucking one. Where are the "Check All" or "Un-Check All" buttons? Which is a standard pretty much everywhere nowadays. But here - you wish, just click away... Stop using that stupid fucking Telerik (I gather it's Telerik, from the looks of it). Take Caliburn.micro and MahApps and use listviews which support all keyboard shortcuts of selecting out of the box. Done.
24. Let's say I am doing the file placement setup. Why the hell do you have 2 separate tabs for Folder and Rules? If you set the rules in Folder tab, they show up in the rules tab, which may have some other wildcard based rules as well. Is it for sorting? Because, you know - you could put it all in one tab...
25. And as always with DrivePool - the drives list in File Placement rules is in one column - scroll away... How about WrapPanel, at least here?
26. Folder tab at least shows how much of that folder is on what drive. Now why it scans for files every single time you select the folder in the list - nobody knows, I bet. Every single fucking time - pick two folders and click away between them - every time same shit - calculating. Calculating fucking what? How many minutes it would take to code one fucking button like - Scan - which would scan the whole pool once and then keep that info until you done with the settings? Fuck me. Off course here's that fucking problem, that DP does not keep the list of files and where they are on which drives. And it cannot display it to me, when it need to... Ashamed of the fucking mess it has made, probably...
27. So, because spreading the files all over the place is simply fucking stupid, I want to actually manage my file placement. I have a list of folders on my pool on the left, I have a list of drives on the right. OK. I select one of the folders from the pool on the left, and... fucking calculating... and... I get another fucking list of drives down below. Because... reasons. Now, if I want to check where actually are my files, on what disks, compared to the settings - I need to dart my tired eyes back and forth between two lists. Now, both lists have different font sizes and font weight (maybe even font family, who the fuck knows at this point), which makes it so much harder to "quickly match up". Because... reasons. Why the fuck would you not use the same list of drives on the right side, that you already have? How much easier would be to actually see straight away, which drives have been checked on/off and which actually have files from that folder in them? Elementary, my dear Watson. Of course, because it's DrivePool we are talking about - I need to mention straight away, that when I say - show the status of the files in the drive list on the right - I mean really fucking show it! Not some ridiculously small progress bar, like in the main window, where you could never see anything, if you let's say have 10GB of files on a 10TB drive. That shit would be indiscernible. Well, in this case, maybe the size column would help to fix this, at least. You could make it that it would be easy to spot if drives have any files from selected folder - change the drive item background color, or name color, whatever. Big green dot, if the drive has any files from that folder whatsoever. Anything to make it intuitive and easy to use.
28. Can I at least sort that second drive list down below? Nope. Sorted only by size taken. Imagine 50+ drives and finding some drive by name to see if it's even on the fucking list!
29. Wanna see the folder size - need to switch between a list and a pie-chart (thosetiny buttons at the top of folders list). Would it be so hard to show folder size by the folder name in the list up top? And whats with that pie-chart selection? It shows me the same size 2 times. One with a round blob, one with a tiny line. Am I missing something here?
30. Filter (file/folder placement rules) also on disks. ie: I select a disk and see/set which folders can go onto that disk. Now, it is a pain to unselect every disk but one for every folder that I want to filter on. Especially, if I want it this way, I need to remember to go into settings and set filters on every new folder I create in the root of the pool in the future. This filtering on disks could be an additional (opposite) way to filter and set the rules.
31. Give us WebUI and let us control DP and others from the browser from wherever we want! It's not middle ages. And it's not that kind of program or the UI (especially as it is now) that would warrant you using WPF, and most importantly forcing me install 2 programs just to control them on another pc. That is whack.
I have 1 license of all 3 programs - DrivePool, Scanner and CloudDrive. But both DrivePool and Scanner are running on my home server, which I actually want and do control from my main rig. Now, of course, I need to install both DrivePool and Scanner on my main rig, as-well. Just to control them on the server. Fuck that...
32. WPF would be warranted for a program, which needs a hugely fast update times. All of the web UIs I use (and they are numerous) are faster than this WPF GUI now. Seriously. The GUIs now for both DP and SCanner are fucking slow as snails.
33. Constant Not Responding when opening GUIs. This embarrassing, dude... Async.
34. Give us full featured (and I mean full featured - every single command and data point) json Web API and see the community blow your socks of with their imagination and awesome web UIs. You're a 1 dev team. Use your head and delegate. With unRAID waking from the development sleep with these ground-breaking new features and FreeNAS catching up to unraid lately in the last versions (even though it's still a mess, with coral, etc...), if DrivePool wants to stay alive and relevant - make an API and let people make your program "great again"... lol But seriously... not a joke, just do it.
Please, don't suggest that there is that dpcmd thing. It's capabilities are weak. And the output format is crazy (why would you not make it easily consumable)?
35. Or if you hell bent on WPF - still, with full API, we can make a kick ass WPF UI. MahApps and Caliburn.Micro - both free and awesome - fuck that stupid Telerik.
36. Would be nice to have an option to see the log in the UI. With good filtering.
37. At least give us as simple text log files. So I could tail them when I wanna know whats going on in DrivePool real time.
38. Probably a bug. Setting the priority of the running balancer or duplication is not displayed properly in the UI, if using multiple UIs on multiple computers. And priority resets to normal, it seems, if you close/re-open the UI which is totally annoying.
39. The last stable version was, when exactly? All I know it was before 07-Jul-2014, judging by the betas numbers in the list. And it's even buggy enough, that I never used it, and started by using beta straight away. Not by my own whim, mind you, by Chris'es recommendation. I know, there was CloudDrive to release. But 2 years? That's how software dies.
40. I'm not sure, if this crap still not solved - but you cannot save your settings with the pool, or export/import them. At least that was the case, last I checked. That is seriously messed up. If this is solved already, sorry... my bad. But I do not care to look for it, as it is especially hard to look for something that may not exist. I tried finding the info on dpcmd for about 20 minutes, when I was writing about it - could not find it.
41. And... Scanner. I'm too tired already to go through and remember every single shit that's bugging me about scanner. But most of what's UI-wise and stuff - applies here too. Scanners UI is crap. Once again - only a single view with a single lit. Oh, and could the drives list items be even bigger, so I could see even less number of drives at the same time and would need to scroll more?..
42. Allow setting concurrent scans per case. In my case every backplane has a separate link, and could use 4 links on the LSI HBA, 1 for each backplane.
43. Or better still - allow creating scanning groups. Most people probably use HBAs or RAID cards. Let us control the scanning settings, so we could achieve the optimum performance. Stop your fucking magic I-know-better and I do-it-auto-magically, which does not work properly, like most of the time.
44. UI does not collapse the list of drives. Actually it collapses the list and immediately expands it back again. wtf?
45. Yesterday I tried to open Scanner UI, and it took more than 2 minutes to open. It always takes long, way longer than it should (for what it is), but 2 minutes - fuck that shit. I'm asking you again, would you, please, gives us an API and let us make an awesome UI? Pretty fucking please... Because this shit is embarrassing (2+ minutes). Or at least Async that shit. Just for good measure - fuck Telerik and it's shit controls, again.
All, but the UI stuff, which could anyone argue is more of the preference kinda thing, IMHO is essential features, which methinks DrivePool must have, being the software that it is, as in managing your files on your file system. It needs to stop being magic that does not work perfectly, and instead - gives us control and choices. On the other hand, when it comes to UI, I stand by my opinions. Give me full featured API and I'll take care of my problems myself. That's all I ask Or again, wake up and let the community write your UI and balancers and what-have-you, for you...
Phew... This one's got away from me
This started as a reply to the pinned thread about pool groups, but I quickly realized, that the frustration started pouring a bit too much and I had to post it as a new thread.
Sorry for all the swearing. It's because I love this company and the software. It is 100% true. I would love to be able to make a living as a one man software developer. And I have already said it in this post and in numerous others - imho, Alex is awesome, to have built these things alone. And I do not forget Chris, you, my friend are an exceptional person too. The job you 2 have done and keep doing is an inspiration. But from this love, and the want for your software to be better for you and us, your customers, this frustration had to be let out.
Please, take it as a feature request and not an attack.
I promise, I will cool off, return and edit all the fucks to something more cultured. Although, I don't want to take out all the emotion from the rant completely, and English is not my native tongue, so we'll see how that goes
P.S. If, whoever accused me of loving to hear my own voice as a reason for my long post last time, gonna come back with the same shit - I most definitely do not. I hate it. It takes a shitload of time for me. Like I said - English is not my strong suite. And I make typos in like every second word, and need to keep editing endlessly. Lastly, I - maybe - write 1 rant like this a year. Not 1/year here - 1 a year total. Maybe less. And only with good reason. Check my freakin post count, dude.