Archive Optimizer functionality

[Salty]

Hi all,

 

I've been trying out the Archive Optimizer on my Drive Pool installation. I find that after a couple of hours the Feeder disk (SSD) becomes full. The pool remains in a 100% healthy state even though the bar graph beside the Feeder disk indicates that it has exceeded the limit.

 

 

I guess the first thing is understanding what I expected the Archive Optimizer to do and if this actually aligns with its implementation. I would have expected that with Balancing settings configured to allow for immediate balancing, any data on the SSD Feeder disk would be scheduled for archive to the other disks in the pool.

 

 

Furthermore, whilst unduplicated or duplicated data remains on the Feeder, the pool's health would be less than 100%. My pool's health remains at 100%.

 

I do have some extra data on the drive outside of the Pool - Windows indexing database etc.

 

I've tried a remeasure and trace logging. The only log entry is: "[Rebalance] Cannot balance. Requested critical only."

 

Am I expecting this to do something that's not actually intended?

 

Any configuration settings changes anyone can recommend that they find works well?

 

Cheers,

Salty.

 

Edit: Running version 2.0.0.345 Beta, Server 2012.

[sspell]

I think you need two ssd feeder disk. One would work but two is optimal for duplicate folders. You really not getting as much a benefit from one ssd drive with file duplicates.

[Christopher (Drashna)]

Yes, if you are using duplication, then you absolutely need two feeder disks.

 

Please set up two feeder disks, and then see if the Archive Optimizer is exhibiting the same results.

 

As for the "[Rebalance] Cannot balance. Requested critical only." message, that just means that the Service believes that the pool doesn't need balancing at this time.

[Salty]

I moved all of the files that were stuck on the feeder disks to non-pool disks. Added a second feeder disk and did a full remeasure. Copied the files back into the pool and they correctly stored onto the feeder disks. It looks like some of the files were then moved from the feeders to archive but there's still about 45GB that remain on the feeders. Some files were destined for duplicated folders some were not.

 

I tried another remeasure after this and the pool health returns to 100%.

 

Not really sure what else I can try, any suggestions?

[hopester]

Try moving the automatic balancing trigger slider to 100%. There might be an issue with the logic not working on the 1GB data limit you have set. Worth a try.

[Salty]

Unfortunately playing with the slider hasn't helped - i've logged a support ticket and will see what we can shake loose.

[sspell]

I don't know if I'm understanding this correctly or not but you feeders should contain data to access faster than the archive disk so your feeder disk should always have data on them. More than likely the last data you accessed from your drive pool. Or having fast ssd feeder drives would be of no use...

[Salty]

Would definitely like clarification if that's the expected behaviour of this plugin.

 

My thoughts on this are: In the event I start writing a 40GB backup file to the pool, this should be initially written to the feeder disks (if the file is an unduplicated file, it will be written to a single feeder disk). If there is existing data on the feeder disks, its going to take longer if the existing data has to be written to a magnetic archive disk prior to accepting the new 40GB data file.

 

My understanding is that whilst data remains on the feeder disks, the pool's health should remain lower than 100%. During the next balancing schedule, that data should be archived off to the designated archive disks. For me, the pool stays at 100% and the data is never archived. This has the undesired effect of once the feeder disks are full, all new data is written directly to archive disks.

 

Hopefully we can get to the bottom of this

[sspell]

well the feeder disk work both ways fetching and transmitting data to the archive disk. So the last data accessed should be on your feeders for fast retrieval otherwise it reverts to the slower archive disk. I'm thinking of the feeder disk as working as a cache. Maybe the guru from covecube could clarify this but that is my understanding of feeder disk.

[Shane]

What should happen is this:

• Files written to the pool land on the feeder disk(s); at least two feeder disks are required if duplication is enabled.
• The files remain on the feeder disk(s) and the pool condition is reduced below 100% until the next scheduled balancing run. Incidentally, this does indeed mean the feeder disk(s) act like a cache for recent files.
• The balancing run moves any files it finds on the feeder disk(s) to the archive disk(s) and the pool condition should return to 100%.

If it's not happening like this, it's not working as intended, and like Salty says hopefully we can fix it.

 

For whatever it may be worth, the "notes" for the Archive Optimizer plugin do suggest it should be used by itself (i.e., try turning off any other balancers you might be using, in case they are conflicting).

[Salty]

Ok cool, Shane's description is how I think it should work, I'm just not seeing the balancing taking place.

 

Two feeder disks - Check

Archive Balancer Plugin only - Check

 

If I had to take a stab at this I would start by looking at how the drive ratios are calculated and the subsequent file move actions that occur based on the ratios. Particularly when non-pooled data is present on the feeder disks. I have some system data like Windows indexes that are on these feeder drives. A few other services like MySQL. All of these sit outside the PoolPart hidden folder, they are not pooled files but the total size for a rebalance may be affected. I'm not trying to tell people how to do their job here - this is outstanding work just offering friendly suggestions

 

Ps. I tried to reflect the assemblies but didn't get very far with most of the core logic obfuscated

 

Cheers,

Matt.