Jump to content
Covecube Inc.

Andy2

Members
  • Content Count

    9
  • Joined

  • Last visited

  1. Andy2

    DrivePool Black Hole

    It's not that the pool needed remeasuring, the files are gone. I don't think it was a bug, the issue is that I moved the files from the Drive Pool to the PoolPart folder on a specific drive, rather than going from one PoolPart folder to another with the DrivePool service disabled. They were "moved" via Explorer but disappeared; hence the "black hole" in the title. And I think the file placement issue itself probably was caused by having automatic balancing disabled. I had been running balancing manually recently, because I was trying to figure out an issue where it would just keep running balancing over and over again without ever getting out of the red. So far, I've been able to restore about 750GB of that data, but there is about 40GB or so of archived websites that no longer exist, which wasn't being backed up due to a Junction not being followed (lesson learned: don't rely on automatic backup validation). The rest of that ~3TB should be able to be reacquired from the source.
  2. Andy2

    DrivePool Black Hole

    So, I guess this is on me, but I wanted to share a bad experience with DrivePool. I've been using DrivePool for a number of years, and though it generally works well, I find the balancing tools to be very frustrating if you want files to be placed on a specific drive, and drives to be written to in order of their performance. Recently, I added a pair of 4TB SSDs to the pool, and wanted to move all of my music and photos exclusively to those drives. Well, DrivePool's balancer is as dumb as rocks, and no matter how many times I set the file placement, disabled all other balancers, or even reset the DrivePool settings, I kept getting a message that DrivePool was "unable to move 3.5TB" in the file placement tab (to an empty drive with enough space) and those files were being left where they were. So I did what I have done in the past, and assigned letters to my drives to move the files from the pool to that drive manually. The next day, since it was a lot of small files and DrivePool had placed them on my slowest SMR drives, I do a re-measure of the pool… to find that it now has a lot of extra free space. I guess that I forgot to disable the DrivePool service before moving these files, or I moved them from the pool rather than the individual drives, and they've now ceased to exist. Now, my family photos should be backed up in two separate locations, and I have many binders full of CDs as my "backup" for that media (I am not looking forward to that project), but I also had data which I considered to be "not important enough for external backups" and instead relied on DrivePool's folder duplication feature to store several copies so that they could survive several disk failures. Well, that doesn't help when you moved those files off the pool, does it? I'm not really expecting anything from this, just frustrated and venting.
  3. I have duplication enabled for some folders, but not the ones I was using in this test. I was moving files from my boot drive (NVMe SSD) to the drive pool, not modifying files.
  4. I'm running the 2.2.3.938 beta (latest version) and am experiencing issues with the SSD Optimizer Plugin. The only balancer enabled in DrivePool is the SSD optimizer. I have a 512GB SSD configured as an "SSD" in the plug-in, with all other drives configured as "archive". File placement settings are configured so that: "File placement rules respect real-time file placement limits set by the balancing plug-ins" is enabled, and "Balancing plug-ins respect file placement rules" is disabled. Based on these descriptions, that should force all files to be written to the SSD before being moved off the SSD by rebalancing. Instead, files are still following the placement rules that are set, and being written to slow HDDs - sometimes even my lowest priority "archive" disks (SMR HDDs) - rather than all writes happening on the SSD. I have tried other combinations of settings but nothing seems to work other than adding the SSD to the placement rules, or removing the rules for that directory. Then the writes immediately start happening on the SSD. But I don't want the files to live on the SSD, and folders which have file placement rules are done for a specific reason; e.g. files where faster access is required are placed on my fastest HDDs, while files that access speeds don't matter are placed on slower drives.
  5. Yes, it's only 2GB since I'm using multiple free accounts as another redundant backup for some important documents. I thought it must be file system overheads, though Windows doesn't report it (still reported having lots of free space on each disk), and the amount seemed excessive - as though duplication may not have been working correctly with ReFS formatted drives. DrivePool doesn't appear to support exFAT; the disks are not listed when formatted using it. Having formatted the disks as NTFS and copied the same data to the new pool, the overhead only appears to be 80MB now rather than 1.1GB.
  6. I use CloudDrive as part of my backup solution, and want to expand that to using multiple cloud services for redundant backups. To do this, I created three 2GB ReFS Cloud Drives (maximum that any one service I have supports) and then pooled them together with DrivePool, setting duplication to 3. The issue I am running into is that I can only store ~900MB of data (~45% of the total capacity) on this pool as the remaining 55% is filled with "other" data. I know that 2GB is not a lot of storage to begin with, but it's sufficient to store everything that I need, while 900MB is not. I understand that there are file system overheads, but if I mount the drives individually, I'm still only seeing ~900MB disk space used on each drive. Is there anything which can be done about this?
  7. Sorry if it was not clear. When I set Archive drives to be filled up to 300GB free space: It also marks my SSD drives to be emptied so that they have ~300GB free space: Note that the rebalancing target (blue arrow) is set to ~177GB on the larger SSD, and is set to completely empty the smaller SSD, since it only has 112GB available. That is despite the file placement limit (orange arrow) being set to 97%. In effect, none of the files that I want to be kept on the SSDs for quick access are stored on those drives when this option is enabled.
  8. Thank you, this seems to have helped - though it has not fixed the problem. I'm somewhat concerned that these settings will also prevent the StableBit Scanner plugin from working if necessary. There are at least a couple of issues which I think are caused by the way that the SSD Optimizer works though: There is an option to "Fill archive drives up to __%". As you might expect, this scales with the capacity of the drive, so the larger the drive is, the more free space it allows. There is an sub-option for this, which is "Or this much free space __GB" So if I want to ensure that there is ~300GB free space on all drives, my assumption is that I would set this to 97% (3% of a 10TB drive = 300GB) or 300GB Free Space, since 3% of a 4TB drive only allows for 120GB free space. However it seems that these are processed in the wrong order. The percentage takes priority over the specified amount of free space and it wants to fill all the archive drives up to 97% in the rebalancing pass. For it to work correctly, at least based on the re-balancing target indicators, I need to set the archive drives to only be filled up to 1% and then set a free space limit of 300GB. Additionally, when I enable the "Or this much free space 300GB" option, it marks both of my SSDs (120GB and 512GB) for being completely emptied.
  9. I have been using Drive Pool for several years now, and it has been working just fine with a pool of similar-speed hard drives, set to keep the same amount of free space available across all of them, and certain directories set to be duplicated across some or all drives. However I've moved everything to a new system which can hold more drives, and so I've added a pair of archive drives and a pair of SSDs to the pool, which is causing me grief. The pool consists of: 120GB SSD 512GB SSD 4x 4TB HDDs 2x 9TB HDDs 2x 8TB Archive Drives I can never get Drive Pool to keep files where I want them to be, and it seems like I'm fundamentally misunderstanding how the plug-in and file placement rules interact. I've gone as far as disabling all other plug-ins, and it still doesn't work like I'd expect. I have the two SSDs set up as "SSD" in the plug-in, and everything else set as an "Archive" drive. Ordered placement is as listed above, with the fastest drives at the top and the slower ones at the bottom of the list. SSDs should be filled up to 95%, 'archive' drives up to 98% or 200GB free space (97% would be >200GB on the 9TB disks). In the file placement rules, I have the "Apps" and "Documents" folders (which total ~15GB) explicitly set to be stored on the SSDs, though "Documents" is also set to be duplicated across all 10 drives. I have "\Video\Current Projects\" set to be stored on the regular HDDs, and "\Video\Archived Projects\" set to be stored on the Archive HDDs. The pool is constantly rebalancing, and I'll come back to it a day later to find that some of the drives are 30% or more fragmented. If I mount any of the drives, I can see that the files are scattered all over the place, not following the placement rules I have set at all. The SSDs are nearly always empty, other than a small amount of duplicated data (which is different for each drive, somehow). Currently, three of the HDDs have <100GB free space, despite the balancing rules being set to try and keep ~200GB free, and two of the other drives each having almost 1TB free. Access to anything stored on the 8TB Archive drives is painfully slow now, with them often stalling for 10-20 seconds at a time, since they're SMR drives with bad fragmentation. File Placement reports that it was unable to move 9TB of data because a suitable destination could not be found. There is currently 4TB of free space on the pool (I plan to add another drive shortly) and none of the balancing / duplication rules should be preventing them from being followed. The directories are smaller than the total space offered by the drives the rules point them at, even factoring in duplication. The largest files are ~100GB, though the vast majority are <40GB. I spent some time over the weekend manually placing files as they should be, but three days later, it's back to how it was before. I was also having problems with new files not being written to the SSDs at all, and prioritizing the slowest Archive drives for some reason (because they had the most free space?) but after disabling everything except the SSD Optimizer and StableBit Scanner plug-ins, that now seems to be working correctly. I realize that this is a long and potentially confusing post, but any help would be appreciated.
×
×
  • Create New...