Jump to content
  • 0

Trying to Understand the SSD Optimizer Plug-in


Andy2

Question

I have been using Drive Pool for several years now, and it has been working just fine with a pool of similar-speed hard drives, set to keep the same amount of free space available across all of them, and certain directories set to be duplicated across some or all drives.

However I've moved everything to a new system which can hold more drives, and so I've added a pair of archive drives and a pair of SSDs to the pool, which is causing me grief.

The pool consists of:

  • 120GB SSD
  • 512GB SSD
  • 4x 4TB HDDs
  • 2x 9TB HDDs
  • 2x 8TB Archive Drives

I can never get Drive Pool to keep files where I want them to be, and it seems like I'm fundamentally misunderstanding how the plug-in and file placement rules interact.

I've gone as far as disabling all other plug-ins, and it still doesn't work like I'd expect.

 

I have the two SSDs set up as "SSD" in the plug-in, and everything else set as an "Archive" drive. Ordered placement is as listed above, with the fastest drives at the top and the slower ones at the bottom of the list.

SSDs should be filled up to 95%, 'archive' drives up to 98% or 200GB free space (97% would be >200GB on the 9TB disks).

In the file placement rules, I have the "Apps" and "Documents" folders (which total ~15GB) explicitly set to be stored on the SSDs, though "Documents" is also set to be duplicated across all 10 drives.

I have "\Video\Current Projects\" set to be stored on the regular HDDs, and "\Video\Archived Projects\" set to be stored on the Archive HDDs.

 

The pool is constantly rebalancing, and I'll come back to it a day later to find that some of the drives are 30% or more fragmented. If I mount any of the drives, I can see that the files are scattered all over the place, not following the placement rules I have set at all.

The SSDs are nearly always empty, other than a small amount of duplicated data (which is different for each drive, somehow).

Currently, three of the HDDs have <100GB free space, despite the balancing rules being set to try and keep ~200GB free, and two of the other drives each having almost 1TB free.

Access to anything stored on the 8TB Archive drives is painfully slow now, with them often stalling for 10-20 seconds at a time, since they're SMR drives with bad fragmentation.

File Placement reports that it was unable to move 9TB of data because a suitable destination could not be found. There is currently 4TB of free space on the pool (I plan to add another drive shortly) and none of the balancing / duplication rules should be preventing them from being followed. The directories are smaller than the total space offered by the drives the rules point them at, even factoring in duplication. The largest files are ~100GB, though the vast majority are <40GB.

I spent some time over the weekend manually placing files as they should be, but three days later, it's back to how it was before.

 

I was also having problems with new files not being written to the SSDs at all, and prioritizing the slowest Archive drives for some reason (because they had the most free space?) but after disabling everything except the SSD Optimizer and StableBit Scanner plug-ins, that now seems to be working correctly.

I realize that this is a long and potentially confusing post, but any help would be appreciated.

Link to comment
Share on other sites

5 answers to this question

Recommended Posts

  • 0
11 hours ago, Andy2 said:

SSDs should be filled up to 95%, 'archive' drives up to 98% or 200GB free space (97% would be >200GB on the 9TB disks).

Yes and no.  They can be filled up to that amount.  Any more, and it will start placing files on other drives.

Also, at each balancing pass (as defined by your settings), it will attempt to completely empty the "SSD" drives. 

11 hours ago, Andy2 said:

I have "\Video\Current Projects\" set to be stored on the regular HDDs, and "\Video\Archived Projects\" set to be stored on the Archive HDDs.

The pool is constantly rebalancing, and I'll come back to it a day later to find that some of the drives are 30% or more fragmented. If I mount any of the drives, I can see that the files are scattered all over the place, not following the placement rules I have set at all.

Make sure that you're including the SSD in the rules.  

 

And if you're using the "Ordered File Placement" option, then you need to do this:

  • UNCHECK "File placement rules respect real-time file placement limits set by the balancing plug-ins."
  • CHECK "Balancing plug-ins respect file placement rules."
  • UNCHECK "Unless the drive is being emptied."
11 hours ago, Andy2 said:

Currently, three of the HDDs have <100GB free space, despite the balancing rules being set to try and keep ~200GB free, and two of the other drives each having almost 1TB free.

Make sure that the "prevent Drive Overfill" balancer is disabled.  This may be the culprit here. 

11 hours ago, Andy2 said:

I was also having problems with new files not being written to the SSDs at all, and prioritizing the slowest Archive drives for some reason (because they had the most free space?) but after disabling everything except the SSD Optimizer and StableBit Scanner plug-ins, that now seems to be working correctly.

As above. Make sure that the file placement rules include the SSDs.  Otherwise, it will override the real time placement limits, and do this.  

Link to comment
Share on other sites

  • 0

Thank you, this seems to have helped - though it has not fixed the problem.

I'm somewhat concerned that these settings will also prevent the StableBit Scanner plugin from working if necessary.

There are at least a couple of issues which I think are caused by the way that the SSD Optimizer works though:

 

There is an option to "Fill archive drives up to __%".

As you might expect, this scales with the capacity of the drive, so the larger the drive is, the more free space it allows.

 

There is an sub-option for this, which is "Or this much free space __GB"

So if I want to ensure that there is ~300GB free space on all drives, my assumption is that I would set this to 97% (3% of a 10TB drive = 300GB) or 300GB Free Space, since 3% of a 4TB drive only allows for 120GB free space.

However it seems that these are processed in the wrong order. The percentage takes priority over the specified amount of free space and it wants to fill all the archive drives up to 97% in the rebalancing pass. For it to work correctly, at least based on the re-balancing target indicators, I need to set the archive drives to only be filled up to 1% and then set a free space limit of 300GB.

 

Additionally, when I enable the "Or this much free space 300GB" option, it marks both of my SSDs (120GB and 512GB) for being completely emptied.

Link to comment
Share on other sites

  • 0
On 2/8/2018 at 6:15 AM, Andy2 said:

I'm somewhat concerned that these settings will also prevent the StableBit Scanner plugin from working if necessary.

Yes, it will interfere with StableBit Scanner evacuating the drive, to a degree.  
It will attempt to "empty" the drive, but won't violate the File Placement rules.  Which may be a problem to which there isn't a good solution to (or at least one that isn't immensely complex). 

On 2/8/2018 at 6:15 AM, Andy2 said:

However it seems that these are processed in the wrong order. The percentage takes priority over the specified amount of free space and it wants to fill all the archive drives up to 97% in the rebalancing pass. For it to work correctly, at least based on the re-balancing target indicators, I need to set the archive drives to only be filled up to 1% and then set a free space limit of 300GB.

I'll have to follow yp with Alex (The Developer on this one.

 

On 2/8/2018 at 6:15 AM, Andy2 said:

Additionally, when I enable the "Or this much free space 300GB" option, it marks both of my SSDs (120GB and 512GB) for being completely emptied.

I'm not sure what you mean exactly here. 

 

Link to comment
Share on other sites

  • 0
21 hours ago, Christopher (Drashna) said:

I'm not sure what you mean exactly here. 

Sorry if it was not clear.

When I set Archive drives to be filled up to 300GB free space:

archive300gb.png.fc9e3ca489d8cfb016bce97006481bdb.png

It also marks my SSD drives to be emptied so that they have ~300GB free space:

rebalancing.png.3bca4822768f3a454f97d642a6def479.png

Note that the rebalancing target (blue arrow) is set to ~177GB on the larger SSD, and is set to completely empty the smaller SSD, since it only has 112GB available.

That is despite the file placement limit (orange arrow) being set to 97%.

In effect, none of the files that I want to be kept on the SSDs for quick access are stored on those drives when this option is enabled.

Link to comment
Share on other sites

  • 0

Well, the 1% limit would be an issue here, I think. And is what is causing the issues.  You'd want this at 90+% percent normally. 

From the looks of it. it wants to clear out Disk 9, but it may be limited on how much it can do so, because of the other disks.  1% means that next to nothing can be stored on the drive, but the "or this much free space" may change how that is handled. 

If you could, grab the logs from the system:
http://wiki.covecube.com/StableBit_Troubleshooter

And use 3414 as the Contact/Request ID. 

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...