Jump to content
  • 0

I have a couple of problems...


cryodream

Question

I am sorry for this post, but it needs to be said.

I was so exited when I found DrivePool and Scanner about 2 years ago, that I bought the whole "package" right then and there. I think I even own a licence for CloudDrive, though I never tried it  :unsure:

 

I've been keeping myself from posting a rant for a looong time now. But after reading this thread, it's just too much...

Long story short - my setup is 50+ drives in one pool, with a million plus files. A lot of points made below stem from this - having lots of drives with a shitload of files.

 

 

Scattered all over files

  •  1. DrivePool, for some ungodly reason, scatters files willy-nilly, like a shot from the shotgun (I know there are reason(s), I read something by Alex somewhere, but I don't think, they're good enough to warrant this situation). At least it's by default. And actually that's a "preferred" type of operation, as I understand. Why there isn't an option to keep files together in the same folder structure, is beyond me. For example how unRAID does it. Further-more unRAID could get away with scattering files wherever, because it uses parity for protection, not duplication. Drivepool, on the other hand - cannot get away with this (more on that later).

    • IMHO, this is the biggest flaw in DrivePool, so it bears repeating. No folders splitting. Or rather a really good system to setup the folders splitting. Again - look at unRAID and do what they're doing. i.e.: splitting of TV Shows when copying to the pool. Now - all the shows have most of the episodes scattered on all 50+ drives. Stupid.

  • 2. Simple real life example (my home server). At the "height" of my excitement with DrivePool, and by that I mean the days just after I found/bough/installed it on my server, I put all my spinners (on my server) in a pool. That's 50+ drives in a single pool. I loved the convenience of the pool and navigating/finding everything easily. I have setup 2x duplication on some folders, not all the of pool (not all files need protecting). Anyway, some weeks later my first drive died, while using DrivePool. Fuck. Not because drive died - no - because the first thing I obviously needed to know was - what files have I lost. I came to the forums, and quickly found, that even though DrivePool's half of functionality is file security (by a way duplication) - it does not keep track of files on the pool, in any shape or form. Fakme. Now that was stupid, I thought to myself. Especially, if you gonna throw my files around the disks, where I have no idea where they actually are - how the fuck can you not keep track of them, so you could tell me, what I've lost if a drive dies? And up to this day, I have no idea what I've actually lost, just that it was ~3TB of stuff. Love it  :angry:

    • After that happened, I had a thought and stopped using duplication. Actually, of course, the first thing I've done was to setup auto-scanning of the DrivePool pool every 6 hours with WhereIsIt, so the next time a drive dies, I would at least be able to know what the hell I've lost. Once again - I need a third party tool to keep track of my files. I'm sorry, but this is inexcusable.
    • Please, do not mention the "Ordered File Placement" plug-in. Of course, drives need to be filled evenly, not one at a time. For speed, for wear and tear, what have you. Just simply, don't scatter files - keep them together.

  • 3. A shitload of empty folders in the drives are driving me nuts. With this scattering of files there are probably more freaking empty folders on the drives than actual files. Dafuk.
  • 4. At the very least - implement file and folder access in the balancer development (IIRC, now you can't), so I could write my own implementation of keeping files and folder on the same disks, if you don't want to...  Because this madness with file placement is making the program unusable. Reasons - below.
  • 5. I have been using DrivePool for the last 1,5, at least - just because I am too lazy to disband the pool and then manually restore the order in all that mess that DrivePool created. But I'm finishing up writing a little program, that will do it for me, so fingers crossed... I might get back some sanity in my file system, soon. Anyone know of a tool which would help in this? Really lazy to code now...
 

Duplication

  • 6. I've been eagerly waiting for grouping of drives, to finally make duplication usable. Yes, I said it. Because as it is now - duplication is unusable. Not even close. Anyone having 2x duplication and lots of drives and lots of files is actually bound to loose a shitload of files in case more than 1 drive fails at the same time. And with quality of drives nowadays - good luck... And if you don't believe me, think about it - I'll describe my reasoning in the following points:

    • Like I mentioned before, after loosing my first drive in the pool, I disabled the duplication. The reason was simple - I did some thinking. I have 50+ drives, probably a million plus files. DrivePool scatters files to as many drives as it can, whenever it can. If 2 drives die - how many files would actually have both copies on those drives (1 copy on each drive with 2x duplication). The answer is not zero. 100% - some files would get lost. And if someone offers me to do 3x duplication, then can I haz some of your money, plz... It's been too long for me to remember how to actually do the probability math, but simple logic dictates - that duplication would not help me at all to keep all my files safe, in case 2 drives go tits up at the same time. Basically, the more drives you have, the more files you have - the more fucked you are with 2x duplication. And remember... on top of that you will have no fucking idea, what files you have just lost... enjoy...

  • 7. Now I see the drive groups arrived. And it's still does not fix the above problem. Am I understanding this correctly - you can make same old-style pools which can have duplication settings inside them and separately from one another. And then you pool those pools together? So they do not duplicate from one pool to another pool? Why the hell not? Isn't this, like, the first and only thought process, how to do this type of drives grouping, with the duplication in mind? What the hell, seriously... However many drives you have - split them in the middle by the age (how old they are) and that's your 2 groups. This way, all the oldest drives are duplicated into the all the newest drives. And you are sure, that each file has at least one copy in a new drive. Isn't that obvious? Want more than 2x duplication - split into more groups, but again, use common sense and group them drives by age. And duplicate between groups!
  • 8. And anyway, duplication of 100TB+ or more of media files, seriously? Would love to be that rich, so it wouldn't hurt the wallet. How about implementing a SnapRAID over the pool. With cache/landing disk (or landing disks group) that would be easy. To keep all the files always secured:

    • files get copied to duplicated landing disks.
    • Then, in the background they get copied to the snapraid protected pool drives.
    •  Snapshot gets updated.
    • Now that files are properly secured on the pool by snapraid - they can be deleted from the landing disks.
    • Result - no wasted money and drives on a stupid duplication and files are protected way way better with multiple parities, if you want, and checksumming against bitrot and some-such. If it's impossible for you to use another free program like that in your paid software - give as an option to do it ourselves, by keeping the fucking files together. Because, I would've done things this way ages ago, if not for the stupid shit DrivePool does whenever it wants.

 

Balancing

  • 9. Balancing. Don't get me started... Stop this fucking Apple-like magic, where program does whatever it wants, and I have no clue what the hell is going on. Instead, show it to me. Because magic-like operation is nice, sometimes, but not here. And it must work perfectly 100% of the time. If not - I'm an idiot who doesn't know what the hell is going on with his own files on his own computer. And DrivePool is far from 100% perfect. And I'm not expecting it to be. It is a mirracle, as it is, with only 1 dev working on it.
  • 10. This bears repeating. Show me what the fuck DP is doing and especially - why. Not the same single meaningless phrase on the GUI - "moving files because whatever". But actually, moving these files from this drive to this drive, because of this balancer, or these file placement rules changes, etc...
  • 11. Add an option to simulate balancing and let me choose what or if I wanna do it.
  • 12. Add option to force re-balancing. Better still, to force re-balancing on individual balancer.
  • 13. For the love of god - give me an  option to manually control (start/stop) the balancing. Let me select which drives I want to balance, with which balancer(s), so when I fire up the balancer, it would not go and do some shit I do not care for and waste time and my patience. When I change the file placement rules, I probably always want to balance something specific. Just... let me.
  • 14. It would be nice, to able to prioritize balancing actions. Like a queue. This would be very helpful on big re-balancing jobs. That's, if  we ever get control of balancing.
 

Settings

  • 15. SSD Optimizer title for a balancer is confusing. Change it to Landing Disk or Cache Drive or something similar. Because that's was it does, basically.
  • 16. The minimum free space should be set in one and only one place . Multiple balancers have it - why!? Also, there is percent and size based settings at the same time - again - why? In short, leave the free space settings only in Prevent Drive Overfill and that's it, only with ability to set free space per drive basis, if we want. Drives now come in 1-10TB, most commonly, which is a huge difference.
  • 17. Filter (file/folder placement rules) also by size. eg: maybe I would want all the metadata for movies and tv shows to be placed on the ssd, so the spinners would contain only big video files. Less fragmentation. Speed accessing metadata. That'd be neat.
 

 

UI

  • 18. OK, let's move on onto another awful thing. The GUI. Talk about style over substance. Should I drop a screenshot of the DrivePool GUI in full-screen on a 21:9 ultra-wide screen monitor, or is it easy enough for anyone to actually imagine, how ridiculously unpractical it is, without even seeing it? Starting with having a program in 3440x1440 resolution and still not being able to see even half of my drives in a list. Stupid wasted desktop real estate galore. I have never ever seen anything like this before, ever.
  • 19. The drives list is also wasted space. And of course - there's no way to see all your drives at once. Have too many drives - scroll through this one listview, never-mind there's plenty of space all around for a different view displaying more/all the drives in a wrap panel.
  • 20. Disk performance view at the bottom right - you will see activity for only 5 files - and that's it. That's all you get. What the hell? I seriously wonder, how this decision got made. The main reason the program was created, was to manage files - but if you want to know anything about what's going on with those files - I'll show you only 5. Fuck the rest. You have 10 transfers... more...? Does not matter - 5, I say. Better look at that huge pie-chart, pretty, isn't it?...
  • 21. Want to change some settings for the pool? Here's a universal icon for settings at the top on the right, nice and pretty intuitive... oh, wait... shit. That is not the options - what the hell? Oh here's options below that huge pancake in the middle. Oh, it's not a button to open the Settings - it's a menu, with sub-menus? Oh, I see, some menus are check-boxes and some actually open the settings window. What? There are multiple separate settings windows for all kinds of things.. Couldn't all of them be in one settings window, as per usual?  Oh, it would be too easy, intuitive and too simple to use, you say? OK, then...
  • 22. I could go on like this for quite a while longer, but fuck it. There's more important things. During probably almost 2 years of using DrivePool and Scanner - I can confidently say, that 8 out 10 times I tried to control DP or Scanner on my server from my main rig - I could not. The trial on main rig expired ages ago. The drop-down menu selection to connect to server - there's no selection  but the local rig. 8/10 times there's no server option. I can't connect and control the actual DP and Scanner that I use. So now I have 2 programs installed in my main rig for no fucking reason. Beautiful.
  • 23. I have a shit load of drives. I want to keep some folder on a single drive. I go into settings > File Placement > and either Folders or Rules, does not matter. Then I need to un-check all my drives, but one. One by fucking one. Where are the "Check All" or "Un-Check All" buttons? Which is a standard pretty much everywhere nowadays. But here - you wish, just click away... Stop using that stupid fucking Telerik (I gather it's Telerik, from the looks of it). Take Caliburn.micro and MahApps and use listviews which support all keyboard shortcuts of selecting out of the box. Done.
  • 24. Let's say I am doing the file placement setup. Why the hell do you have 2 separate tabs for Folder and Rules? If you set the rules in Folder tab, they show up in the rules tab, which may have some other wildcard based rules as well. Is it for sorting? Because, you know - you could put it all in one tab...
  • 25. And as always with DrivePool - the drives list in File Placement rules is in one column - scroll away... How about WrapPanel, at least here?
  • 26. Folder tab at least shows how much of that folder is on what drive. Now why it scans for files every single time you select the folder in the list - nobody knows, I bet. Every single fucking time - pick two folders and click away between them - every time same shit - calculating. Calculating fucking what? How many minutes it would take to code one fucking button like - Scan - which would scan the whole pool once and then keep that info until you done with the settings? Fuck me. Off course here's that fucking problem, that DP does not keep the list of files and where they are on which drives. And it cannot display it to me, when it need to... Ashamed of the fucking mess it has made, probably...
  • 27. So, because spreading the files all over the place is simply fucking stupid, I want to actually manage my file placement. I have a list of folders on my pool on the left, I have a list of drives on the right. OK. I select one of the folders from the pool on the left, and... fucking calculating... and... I get another fucking list of drives down below. Because... reasons. Now, if I want to check where actually are my files, on what disks, compared to the settings - I need to dart my tired eyes back and forth between two lists. Now, both lists have different font sizes and font weight (maybe even font family, who the fuck knows at this point), which makes it so much harder to "quickly match up". Because... reasons. Why the fuck would you not use the same list of drives on the right side, that you already have? How much easier would be to actually see straight away, which drives have been checked on/off and which actually have files from that folder in them? Elementary, my dear Watson. Of course, because it's DrivePool we are talking about - I need to mention straight away, that when I say - show the status of the files in the drive list on the right - I mean really fucking show it! Not some ridiculously small progress bar, like in the main window, where you could never see anything, if you let's say have 10GB of files on a 10TB drive. That shit would be indiscernible. Well, in this case, maybe the size column would help to fix this, at least. You could make it that it would be easy to spot if drives have any files from selected folder - change the drive item background color, or name color, whatever. Big green dot, if the drive has any files from that folder whatsoever. Anything to make it intuitive and easy to use.
  • 28. Can I at least sort that second drive list down below? Nope. Sorted only by size taken. Imagine 50+ drives and finding some drive by name to see if it's even on the fucking list!
  • 29. Wanna see the folder size - need to switch between a list and a pie-chart (thosetiny buttons at the top of folders list). Would it be so hard to show folder size by the folder name in the list up top? And whats with that pie-chart selection? It shows me the same size 2 times. One with a round blob, one with a tiny line. Am I missing something here?
  • 30. Filter (file/folder placement rules) also on disks. ie: I select a disk and see/set which folders can go onto that disk.  Now, it is a pain to unselect every disk but one for every folder that I want to filter on. Especially, if I want it this way, I need to remember to go into settings and set filters on every new folder  I create in the root of the pool in the future. This filtering on disks could be an additional (opposite) way to  filter and set the rules.
  • 31. Give us WebUI and let us control DP and others from the browser from wherever we want! It's not middle ages. And it's not that kind of program or the UI (especially as it is now) that would warrant you using WPF, and most importantly forcing me install 2 programs just to control them on another pc. That is whack.

    • I have 1 license of all 3 programs - DrivePool, Scanner and CloudDrive. But both DrivePool and Scanner are running on my home server, which I actually want and do control from my main rig. Now, of course, I need to install both DrivePool and Scanner on my main rig, as-well. Just to control them on the server. Fuck that...

  • 32. WPF would be warranted for a program, which needs a hugely fast update times. All of the web UIs I use (and they are numerous) are faster than this WPF GUI now. Seriously. The GUIs now for both DP and SCanner are fucking slow as snails.
  • 33. Constant Not Responding when opening GUIs. This embarrassing, dude... Async.
  • 34. Give us full featured (and I mean full featured - every single command and data point) json Web API and see the community blow your socks of with their imagination and awesome web UIs. You're a 1 dev team. Use your head and delegate. With unRAID waking from the development sleep with these ground-breaking new features and FreeNAS catching up to unraid lately in the last versions (even though it's still a mess, with coral, etc...), if DrivePool wants to stay alive and relevant - make an API and let people make your program "great again"... lol But seriously... not a joke, just do it.
  • Please, don't suggest that there is that dpcmd thing. It's capabilities are weak. And the output format is crazy (why would you not make it easily consumable)?
  • 35. Or if you hell bent on WPF - still, with full API, we can make a kick ass WPF UI. MahApps and Caliburn.Micro - both free and awesome - fuck that stupid Telerik.
  • 36. Would be nice to have an option to see the log in the UI. With good filtering.
 

Other

  • 37. At least give us as simple text log files. So I could tail them when I wanna know whats going on in DrivePool real time.
  • 38. Probably a bug. Setting the priority of the running balancer or duplication is not displayed properly in the UI, if using multiple UIs on multiple computers. And priority resets to normal, it seems, if you close/re-open the UI which is totally annoying.
  • 39. The last stable version was, when exactly? All I know it was before 07-Jul-2014, judging by the betas numbers in the list. And it's even buggy enough, that I never used it, and started by using beta straight away. Not by my own whim, mind you, by Chris'es recommendation. I know, there was CloudDrive to release. But 2 years? That's how software dies.
  • 40. I'm not sure, if this crap still not solved - but you cannot save your settings with the pool, or export/import them. At least that was the case, last I checked. That is seriously messed up. If this is solved already, sorry... my bad. But I do not care to look for it, as it is especially hard to look for something that may not exist. I tried finding the info on dpcmd for about 20 minutes, when I was writing about it - could not find it.
 

 

Scanner

  • 41. And... Scanner. I'm too tired already to go through and remember every single shit that's bugging me about scanner. But most of what's UI-wise and stuff - applies here too. Scanners UI is crap. Once again - only a single view with a single lit. Oh, and could the drives list items be even bigger, so I could see even less number of drives at the same time and would need to scroll more?..
  • 42. Allow setting concurrent scans per case. In my case every backplane has a separate link, and could use 4 links on the LSI HBA, 1 for each backplane.
  • 43. Or better still - allow creating scanning groups. Most people probably use HBAs or RAID cards. Let us control the scanning settings, so we could achieve the optimum performance. Stop your fucking magic I-know-better and I do-it-auto-magically, which does not work properly, like most of the time.
  • 44. UI does not collapse the list of drives. Actually it collapses the list and immediately expands it back again. wtf?
  • 45. Yesterday I tried to open Scanner UI, and it took more than 2 minutes to open. It always takes long, way longer than it should (for what it is), but 2 minutes - fuck that shit. I'm asking you again, would you, please, gives us an API and let us make an awesome UI? Pretty fucking please... Because this shit is embarrassing (2+ minutes). Or at least Async that shit. Just for good measure - fuck Telerik and it's shit controls, again.
 

Afterthoughts

All, but the UI stuff, which could anyone argue is more of the preference kinda thing, IMHO is essential features, which methinks DrivePool must have, being the software that it is, as in managing your files on your file system. It needs to stop being magic that does not work perfectly, and instead - gives us control and choices. On the other hand, when it comes to UI, I stand by my opinions. Give me full featured API and I'll take care of my problems myself. That's all I ask :) Or again, wake up and let the community write your UI and balancers and what-have-you, for you...

 

Phew... This one's got away from me :)

This started as a reply to the pinned thread about pool groups, but I quickly realized, that the frustration started pouring a bit too much and I had to post it as a new thread.

Sorry for all the swearing. It's because I love this company and the software. It is 100% true. I would love to be able to make a living as a one man software developer. And I have already said it in this post and in numerous others - imho, Alex is awesome, to have built these things alone. And I do not forget Chris, you, my friend are an exceptional person too. The job you 2 have done and keep doing is an inspiration. But from this love, and the want for your software to be better for you and us, your customers, this frustration had to be let out.

Please, take it as a feature request and not an attack.

I promise, I will cool off, return and edit all the fucks to something more cultured. Although, I don't want to take out all the emotion from the rant completely, and English is not my native tongue, so we'll see how that goes :)

 

P.S. If, whoever accused me of loving to hear my own voice as a reason for my long post last time, gonna come back with the same shit - I most definitely do not. I hate it. It takes a shitload of time for me. Like I said - English is not my strong suite. And I make typos in like every second word, and need to keep editing endlessly. Lastly, I - maybe - write 1 rant like this a year. Not 1/year here - 1 a year total. Maybe less. And only with good reason. Check my freakin post count, dude.

 

Peace.

Link to comment
Share on other sites

5 answers to this question

Recommended Posts

  • 0

​I agree that folder level splitting is a must-have. I'd really like to use folder split level 2 for media files.

​Fortunately for me I can live with the Ordered File placement plug-in for now.

​

The folder/file catalog should be built in, I agree. Thanks for the suggestion with WhereIsIt. I don't plan on using Duplication so I need to know which folders are on which drive in order to be able to rip the DVD/Blu-rays again when a hard drive fails.

​

​I've only been using DrivePool since yesterday and these two points jumped out to me. I also agree on most of your other points but they haven't bothered me yet.

​

Link to comment
Share on other sites

  • 0

@Wiidesire

Do not use, or at least do not think of buying WhereIsIt. I am using it, because I have bought it like 10 years ago. It was, maybe even still is the best at cataloging files and folders on drives. But it has serious problems.

  • It has been, basically abandoned. Even here it shows up as discontinued: Where Is It? - AlternativeTo.net
  • The program also has a single dev, but not like guys here. That one is a real asshole. Check out comments on AlternativeTo (the link above). and I, myself can attest to that - I had a displeasure of a support email exchange with him. Fuck that guy.
  • The program never has before or most definitely never will have any community forums, or any other kind of support, except email with that idiot. Which you better avoid.
  • I think, that's enough reasons to stay away...

I myself am gonna try out this: Free Disk Catalog Software for Windows - WinCatalog 2017

There was never a good alternative to WhereIsIt before, but this seems promising. I need something "evolving" for this kind of stuff. This one is also paid, if you want all the features. but it looks good and not dead.

 

Well, check it out yourself. My job was to warn you.

Good luck.

Link to comment
Share on other sites

  • 0

Hi you have some valid points most which have already been raised there is an extensive list of feature upgrades which a lot of people have recommended and been waiting for upto 2 years and more. Like yourself I love the software but it has become dated Alex is doing a great job but I do think he has missed a few great opportunities to make his software a hot topic but that's the problem with being the only developer I myself have also looked at other alternatives and have been using storage spaces alongside drivepool for a longtime now I am now at the point of once I upgrade to server 2016 I will probably use storage spaces. Scanner like you say is very slow to the point it leaves you waiting for the program is not responding message to pop up and it gives from time to time false drive error messages to this day I have 2 drives I can't let scanner scan as it shows them as bad but they are in fact fine.

 

These forums used to be buzzing with comments and banter but I think the lack of development has produced a lack of interest in the products which is very sad.

Link to comment
Share on other sites

  • 0

Wow, you're not making this easy for us, are you? 

And to be blunt, you're coming off as VERY hostile, especially the further I get into the post.  Most of this information is actually documented in our manual, can be found rather easily on our forums, or (as here) asked.

 

http://stablebit.com/Support/DrivePool/2.X/Manual

 

http://community.covecube.com/index.php?/forum/20-nuts-bolts/

 

http://wiki.covecube.com/StableBit_DrivePool_2.x_Advanced_Settings

 

Regardless, let me address each and every one of these questions/concerns/feedback. 

 

Well, here we go.

 

  1. Yes, correct.  There are a number of reasons for this, such as disk load.  However, if you're not happy with that, then please grab the "Ordered File Placement" balancer plugin.  This will fill up one (or more, if using duplication) disks at a time, filling them sequentially (in an order that you can specify). 
    https://stablebit.com/DrivePool/Plugins

    This balancer will also help prevent folder splitting, but it isn't a guarantee.  But this is a topic that has come up a LOT, and we've discussed it a lot internally.  I have been pushing Alex (the Developer) for this functionality, as it IS highly desirable for specific setups. 
     
  2. For the file locations, ... yeah.  NTFS keeps track of this and generally does a great job. Except when it doesn't.   If duplication is enabled, then this isn't even an issue, as you just "fix" the drive (replace it, format it, etc), and you're set.   But otherwise, yeah, it can be a PITA.  That said, we do have plans on addressing this issue in the future. 

    Part of the reason for this is that maintaining a database is "expensive" ... and redundant (NTFS *is* a database).  And there are other issues/complications with this. 
     
  3. The empty folders may not be.  And no, the software doesn't remove "empty folders" for a number of reasons.  Such as duplication information being "tagged" on the folder (in the form of alternate data streams).   Additionally, from a programmatic standpoint, there is no reason to delete a folder here. The amount of saved space is so small, that it's not worth the hassle that it can cause. 
     
  4. See above (#1).   That said, doing so would require a complete rewrite from the ground up.   And introduce a lot more "knobs and dials" that may not be necessary for 99.9999% of people.   But implementing something to keep folders together is a lot simpler, and would not necessarily require a complete rewrite of the code. 
     
  5.  If you do want to use the pool, and make it "orderly" to do so,  you can use the File Placement rules, and/or the Ordered File Placement/Drive Usage Limiter" to clump up/group up the files, to make this less "nightmarish". 
     
  6. I'm not entirely sure what you're getting at here. 
    As for the "unusable for duplication" space, this is a complicated calculation, based on the number of disks, the size of the disks, the amount of used and free space on the disks, etc.  The "Duplication Space Optimizer" balancer will attempt to rebalance the data in such a way to optimize this and reduce the "unusable" space.  

    However, the balancing is done at a low IO priority, as to prevent performance issues with the pool.  You can temporarily boost the priority by clicking the ">>" button by the condition bar at the bottom, or there is an advanced config file that can be used to permanently boost the priority. 

    As for multiple drives failing at the same time, yes, you can end up in the same situation.  As for "drive quality these days" ....  We are firmly "post flood manufacturing".  So failure rates have dropped significantly.  Many sources can confirm this.  But if you have proof otherwise, please do link it here. 
    That said, the StableBit Scanner balancer can help with this by evacuating the contents of problem drives, hopefully before the disk fails.   So that should minimize the impact.  
     
  7. See above. 
     
  8. Here you go:
    http://community.covecube.com/index.php?/topic/52-faq-parity-and-duplication-and-drivepool/

    And at this point, there is absolutely no plans on introducing parity into the software.  Parity by it's nature is expensive, in terms of resource cost.  Especially if you're performing realtime parity protection.    

    And yes, duplication is more expensive.  But to be blunt, this author put the issue succinctly. : 
    http://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-not-raidz/

    Otherwise, you can absolutely do what you want with the StableBit DrivePool, the SSD Optimizer, and SnapRAID.  
     
  9. Just because it doesn't fit your needs isn't a reason to get nasty.   For a vast majority of people, the balancing works just fine for their needs.    

    That said, it is worth mentioning that the log files actually go over SPECIFICALLY why files are being moved, and what balancer triggered it.

    That said, we do plan on improving the feedback for StableBit DrivePool, to make things like this more transparent.

    Additionally, in most cases, balancing isn't going to happen unless you have a balancer installed.  The built in balancers are generally "edge case" options, that deal with very specific situations. 

    In fact, you may want/need to read this: 
    http://community.covecube.com/index.php?/topic/2954-understanding-file-placement/&do=findComment&comment=20223
     
  10. See previous.
     
  11. Not a bad idea.  
    https://stablebit.com/Admin/IssueAnalysis/27546
     
  12. Exists already.  WIll pop up next to the "Pool condition bar" when the placement is "out of whack". 
    http://stablebit.com/Support/DrivePool/2.X/Manual?Section=Pool%20Organization%20Bar

    http://stablebit.com/Support/DrivePool/2.X/Manual?Section=Balancing

     
  13. Can be controlled by the balancing settings, as above.

    As for "what drives to balance", .... That really doesn't make sense given the context of how the pool works, and how the balancing works.   
    And again, for the most part, most of the balancers are not going to be doing much.  They're going to sit there until the pool hits specific configurations.  
     
  14. It can.   The order of the balancers  are listed in the UI is the order of priority. 
    As for IO priority, you can.  When balancing, there is a ">>" button that you can click to temporarily boost the priority.  

    Additionally, the advanced config file allows you to permanently increase this, if you want. 

    As for queuing, again, set the order in the balancer's tab.   Otherwise, this becomes complicated and problematic.  meaning ... we try to make sure that there is no situation where data is balanced back and forth continuously.    Because that's a great way to cause issues. 
     
  15. Terminology could be better.  But yes, the "SSD" drive is a write cache. 
    https://stablebit.com/Admin/IssueAnalysis/27547
     
  16. Because multiple balancers are checking for different things.   
    However, a "master free space limit" or some such may not be a bad idea, assuming it's not incredibly complex to implement. 

    However, certain balancers may not be able to use this, even if we do implement it, just because of how they work. 
    https://stablebit.com/Admin/IssueAnalysis/27548
     
  17. Just thinking about that .... would be incredibly complex.  However, I've bugged it as a feature request, just in case. 
    https://stablebit.com/Admin/IssueAnalysis/27549

    That said, you can use File Placement rules to do this.  Such as "\*.nfo", "\*.jpg", "\*.xml", etc to force placement of these files onto a specific drive.
    http://stablebit.com/Support/DrivePool/2.X/Manual?Section=File%20Placement
     
  18. The real irony, is that most of the feedback that we get is "simple and elegant".  
    But bugged.  
    https://stablebit.com/Admin/IssueAnalysis/27550
     
  19. See above.
    Bugged.

    that said, there are some additional elements, such as the balancing targets and the real time placement limiters that do show up on the bar. 
    http://stablebit.com/Support/DrivePool/2.X/Manual?Section=Disks%20List
     
  20. Overtly hostile.  
    That said, adding more and more files here can (does) cause a performance hit. I'm pretty sure that this is explicitly why there is a limit.  However, bugged. 
     
  21. No response here.  Design aesthetics are really a "per person" thing. 

    However, we do have a planned "overhaul" of the UI, so this may change. 
     
  22. The remote control not showing all devices on the network is a known issue, but not a software specific issue.  It's a network issue.  However, we have always supported manually adding peers to the remote control.  And IIRC, we do plan on significantly improving the remote control functionality, to make this easier.
    http://wiki.covecube.com/StableBit_DrivePool_2.x_Advanced_Settings#RemoteControl.xml

     
  23. We don't use Telerik in StableBit DrivePool.  At all. period.  StableBit Scanner does, and you can even see the "Telerik" dll in the application directory.   And we don't use it because ... telerik sucks.  And we've ran into that with StableBit Scanner, but Alex hasn't wanted to design the UI for Scanner from the ground up, yet. 

    That said, the select/deselect option should be there. Bugged. 
    https://stablebit.com/Admin/IssueAnalysis/27552
     
  24. Folders are for easy creation. Rules are for more complex creations.  This is actually covered in the manual.
    http://stablebit.com/Support/DrivePool/2.X/Manual?Section=File%20Placement
     
  25. Already bugged (#23)
     
  26. Overtly hostile.  Skimmed, so if I missed something, you know why.

    That said, the measured data is stored, and updated.  The UI element you're talking about is updated in realtime, so yeah, it checks.    I don't remember the reason for this (but there was).  
     
  27. Overtly hostile opinion.  

    Also see above. 
     
  28. You mean the drive list in File Placement Rules? 
     
  29.  StableBit DrivePool isn't meant to be an indexing program.  If we did, it would be more conducive to this, I'm sure.
    That said, is this just bitching, or was there a specific complaint here that I could address? 
     
  30. You covered this already, above.  
     
  31. Planned, actually. 
     
  32. ..... I don't even know where to start. You're accusing us of using Telerik on one hand, and then WPF on the other....

     
  33. Pretty sure it is.   .....
     
  34. Sure?   We should do exactly the same thing as FreeNAS and unRAID because they are.  
    I'm not really sure why there is a need for this.  Also, it's an additional point of failure here.   But it's already requested above. 
     
  35. Again, we're not using a single Telerik control in StableBit DrivePool.    
     
  36. Not sure how useful it would really be. 
     
  37. C:\ProgramData\StableBit DrivePool\Service\Logs\Service
     
  38. Probably an info update issue. 
    What version of StableBit DrivePool are you using? 
     
  39. This has not only been addressed, but it's the top most pinned thread in this sub-forum.
    http://community.covecube.com/index.php?/topic/2620-is-drivepool-abandoned-software/

    We're a small team, and StableBit CloudDrive took a lot of our focus, and took a LOT longer than expected.   We do have plans on preventing this from happening in the future. 

    But unless you're holding out on a time machine, not much we can do about that now.  That said we are actively going through issues and do plan on having a stable release in the near future.  
    Also, if you do have a time machine, awesome!  (go back and post some of the stuff 2 years ago, rather than letting these issues pile up and NOT contacting us for 2+ years). 
     
  40. It's a pending feature request. 
     
  41. We plan on overhauling the StableBit Scanner UI, IIRC.  And address a lot of the issues it has. 
     
  42. It does this per controller.  So a single USB enclosure would be recognized as as that "single case".   
     
  43. We do already do the scanning "Per controller".  
     
  44. See #41.  It's a known issue, and one we plan on addressing. 
     
  45. Are you on the latest beta build? If not, a lot of that has been addressed/fixed. 

    If not, then .... see #41.  
     

 

 

 

That said, I double checked.... you have not ONCE contacted us in the 2+ years that these issues have apparently stewed.  Not once.  

 

Some of these are absolutely legitimate complaints. And we would have gladly addressed them.  Either in private (via https://stablebit.com/Contact) or here, publically.   

 

We

want to help. But we CANNOT if you don't let us.  We may not know of an issue, if you never bother telling us.   Which really appears to be the case.  

 

 

You can here to rant at us, without ... well, doing any research without contacting us.  Because this is pretty clearly a rant.  Not a legitimate grievance. 

 

 

Even still, we want to help you out, and address many (most) of these issues.  We really do.  

 

 

 

But if you're this upset with us, I'm not sure how well that can progress.  it's next to impossible to help an overtly hostile person.  

 

And absolutely worst case, open a ticket and request a refund. We'll do it.  We'd rather have you happy, by issuing a refund, than responding angrily, or by being assholes.    Because that's not cool. At all. 

 

 

 

Regards

Christopher Courtney.

Director of Customer Relations for Covecube Inc. 

Link to comment
Share on other sites

  • 0

Just to reply to some of these concerns (without any of the drama of the original post):

 

Scattered files

  • By default, StableBit DrivePool places new files on the volume with the most free disk space (queried at the time of file creation). It's as simple as that.
  • Grouping files "together" has been requested a bunch, but I personally, don't see a need for it. There will always be a need to split across folders even with a best-effort "grouping" algorithm. The only foolproof way to protect yourself from file loss is file duplication. But I will be looking into the possibility of implementing this in the future.
  • Why is there no database that keeps track of your files? It would degrade performance, introduce complexity, and be another point of failure. The file system is already a database that keeps track of your files, and a very good one. What you're asking for is a backup database of your files (i.e. an indexing engine). StableBit DrivePool is a simple high performance disk pooling application, and it was specifically designed to not require a database.
  • As for empty folders in pool parts, yes StableBit DrivePool doesn't delete empty folders if they don't affect the functionality of the pool. The downside of this is some extra file system metadata on your pool parts. The upside is that, when placing a new file on that pool part, the folder structure doesn't have to be recreated if it's already there. The cons don't justify the extra work in my opinion, and that's why it's implemented like that.
  • Per-file balancing is excluded by design. The #1 requirement of the balancing engine is scalability, which implies no periodic balancing.

I could be wrong, but just an observation, it appears that you may want to use StableBit DrivePool exclusively as a means of organizing the data in your pool parts (and not as a pooling engine). That's perhaps why empty folders, no exact per-file balancing rules, and databases of missing files bother you. The pool parts are designed to be "invisible" in normal use, and all file access is supposed to be happening through the pool. The main scenario for accessing pool parts is to perform manual file recovery, not much more.

 

Duplication

  • x2 duplication protects your files from at most 1 drive failure at a time. x3 can handle 2 drive failures concurrently.
  • You can have 2 groups of drives and duplicate everything among those 2 groups with hierarchical pooling. Simply set up a pool of your "old" drives, and then a pool of your "new" drives. Then add both pools to another pool, and enable x2 pool duplication on the parent pool.
  • That's really how StableBit DrivePool works. If you want something different, that's not what this is.

Balancing

  • The architecture of the balancing engine is not to enable "Apple-like" magic (although, sometimes, apple-like magic is pretty cool). The architecture was based on answering the question "How can you design a balancing engine that scales very well, and that doesn't require periodic balancing?".

    First you would need to keep track of file sizes on the pool parts, so that requires either an indexing engine (which was the first idea), but better yet, if you had full control of the file system, you could measure file sizes in real-time very quickly. You then need to take that concept and combine it with some kind of balancing rules. The result is what you see in StableBit DrivePool.
  • The rest of the points here are either already there or don't make any sense.
  • One area that I do agree with you on is improving the transparency of what each balancer is doing. We do already have the balancing markers on the horizontal UI's Disks list, but it would be nice to see what each balancer is trying to do and how they're interacting.

Settings

  • Setting filters by size is not possible. Filters are implemented as real-time (kernel-based) new file placement rules. At the time of file creation, the file size is not known. Implementing a periodic balancing pass is not ideal due to performance (as we've seen in Drive Extender IMO).

UI

  • UI is subjective. The UI is intended to be minimalistic.
  • Performance is meant to give you a quick snapshot into what your pool is doing, not to be a comprehensive UI like the Resource Monitor.
  • There are 2 types of settings / actions. Global application settings and per-pool settings / actions. That's why you see 2 menus. Per-pool settings are different for each pool.
  • "Check-all", "Uncheck-all". Yeah, good idea. It should probably be there. Will add it.
  • As for wrap panels for lists of drives in file placement, I think that would be harder to scan through in this case.
  • In some places in the UI, selecting a folder will start a size calculation of that folder in the background. The size calculation will never interrupt another ongoing task, it's automatically abortable, and it's asynchronous. It doesn't block or affect the usage of the UI.
  • Specifically in file placement, the chart at the bottom left is meant to give you a quick snapshot of how the files in the selected folder are organized. It's sorted by size to give you a quick at-a-glance view of the top 5 drives that contain those files, or how duplicated vs. non-duplicated space is used. So yes, there's definitely a reason for why it's like that.
  • The size of the buttons in the Balancing window reflects traditional WinForms (or Win32) design. It's a bit different than the rest of the program, and that's intentional. The balancing window was meant to be a little more advanced, and I feel Win32 is a little better at expressing more complex UIs, while WPF is good at minimalistic design.
  • I would not want to build an application that only has a web UI. A desktop application has to first and foremost have a desktop user interface.
  • As for an API, I feel that if we have a comprehensive API then we could certainly make a StableBit DrivePool SDK available for integration into 3rd party solutions. Right now, that's not on the roadmap, but it's certainly an interesting idea.
  • I think that the performance of the UIs for StableBit DrivePool and CloudDrive are very good right now. You keep referring to Telerik, but that's simply not true. Both of those products don't use Telerik. By far, it's using mostly standard Microsoft WPF controls, along with some custom controls that were written from scratch.Very sparingly it does use some other 3rd party controls where it makes sense.
  • All StableBit UIs are entirely asynchronous and multithreaded. That was one of the core design principles.
  • The StableBit Scanner UI is a bit sluggish, yes I do agree, especially when displaying many drives. I do plan on addressing that for the next release.

Updates

  • Yes, it has been a while since the last stable release of StableBit DrivePool (and Scanner). But as you can see, there has been a steady stream of updates since that time: http://dl.covecube.com/DrivePoolWindows/beta/download/
  • We are working with an issue oriented approach, in order to ensure a more stable final release:
    • Customer reports potential issue to stablebit.com/Contact (or publicly here on the forum).
    • Technical support works with customer and software development to determine if the issue is a bug.
    • Bugs are entered into the system with a priority.
    • Once all bugs are resolved for a given product, a public BETA is pushed out to everyone.
    • Meanwhile, as bugs are resolved, internal builds are made available to the community who wants to test them and report more issues (see dev wiki http://wiki.covecube.com/Product_Information).
    • Simple really. And as you can see, there has been a relatively steady stream of builds for StableBit DrivePool.

When StableBit CloudDrive was released, there were 0 open bugs for it. Not to say that it was perfect, but just to give everyone an idea of how this works. Up until recently, StableBit DrivePool had a number of open bugs, they are now all resolved (as of yesterday actually). There are still some open issues (not bugs) through, when those are complete, we'll have our next public BETA. I don't want to add anymore features to StableBit DrivePool at this point, but just concentrate on getting a stable release out.

 

I'm not going to comment on whether I think your post is offensive and egotistic. But I did skip most of the "Why the hell do you...", "Why the fuck would you...",  "I have no clue what the hell is going on" parts.

Link to comment
Share on other sites

Guest
This topic is now closed to further replies.
×
×
  • Create New...