Archive drive - keeps getting filled then emptied

[Spider99]

I have 12 disks in a pool Drive 2 (14_Blue) has strange behaviour in that it either has no files on it with the two arrow markers at position zero (left side) of the bar or it gets filled up as you would expect - then on the next balance it gets emptied.

 

 

 

The drive is not new to the pool - one of the original disks to create the pool

balancing rules all look ok with the disk in the middle of all lists

1) Order file placement - position 5 out of 12

2) Drive usage limiter - position 2

 

I have no file placement rules set

Minimal duplication (couple of folders as a test) as filling up the pool at the moment

 

Only exception is the SSD Optimiser which has it second to last - Question - why is this list not in order like every other list of drives????????

 

 

 

 

List of Plugins and order

 

 

Its almost (but not quite) acting like a SSD drive - although its set as an archive drive - its the same size as 13_Blue-16_Blue - on the same controller for all archive drives

 

As far as i can see its should be acting like an archive drive

 

Any ideas????????????

 

 

[Spider99]

Well having turned on more duplication the 14_blue disk was filled up to a highish percentage with duplicated files and now they are being moved off????

 

 

[Christopher (Drashna)]

Too many balancer options enabled. 

 

Specifically, SSD Optimizer and Ordered File Placement balancers WILL cause issues with each other. 

 

You cannot use both, unfortunately.  However, the SSD Optimizer supports the "ordered File placement" options, if you need that.

 

 

So if you're using the SSD Optimizer, try disabling all of the balancers first, From there, try enabling one balancer at a time to see if it works properly. 

 

 

 

As for the disk arrows:

http://stablebit.com/Support/DrivePool/2.X/Manual?Section=Disks%20List

 

They're the target for balancing.  Eg, that's where the software wants to rebalance the data to.   

Red arrows are the "real time limiters", meaning that this is what is computed for "free space" on the drive. 

[Spider99]

this is still happening

 

i have disabled most of the plugins

 

Using

1. SSD Opt - have ordered the drives the same for dup and undup manually i.e. 1-24

2. Scanner

3. Disk space equal

4. Prevent overfill

 

No file placement rules

 

Have drives being filled up to max (10GB left) others marked as Zero space to be used been like this for a few days - then overnight get filled up to 80-90%

 

Majority are "correct" in having approx 50% fill limit and seem to be stable - as you can monitor the actual file placement easily difficult to tell.

 

DP is balancing a large pool 40+TB with 12 new drives added about a week ago - took a few days to do the duplication and has been at the balance for a few days now (would have been much quicker to do it myself i think!)

 

Any ideas as i am coming to the opinion this is a logic bug you only see with large volumes of data

[Christopher (Drashna)]

If you have the SSD Optimizer and the Disk Space Equalizer balancers enabled, ... well, that's likely the cause of the issue here. 

 

Try disabling *all* of the balancers but the SSD Optimizer and the StableBit Scanner balancers.  See if that works properly. 

 

If that does, then try adding additional providers, until the issue returns.

However, if only those two balancers are enabled (SSD Optimizer and the StableBit Scanner) and it's still causing this issue, let me know. 

[Spider99]

Disabled all made no difference just have ssd Opt running

 

still have drives being emptied and other set to store no files - using the blur triangles as a guide

 

i stopped the balance and it restarted and carried on as before minimal if any change

 

any other ideas

[Spider99]

Current pool view

 

 

 

 

 

 

 

Disk 19 is being emptied it was full yesterday now approx half looking like its going back to zero

 

Disk 9 and 10 set to zero

 

1 full

and 

2 5% full

 

does not make any sense to me

 

pool is approx 48% full - so most if not all disks should be approx that full also?

 

 

[Spider99]

update

 

no real change apart from drive 19 is now approx 20% full

[Christopher (Drashna)]

So disabling all of the other balancers isn't helping with the fill and empty behavior?

[Spider99]

nope its stayed the same for a few days now

 

had just ssd plugin for a while - then turned scanner back on - no change - turned on drive space equal - no change

 

dp is happy the drives are balanced although have three drives empty and one full to the brim

 

images above are still close to what i see now - differences are ssd's empty and drive 19 is also empty along with 9 and 10

 

i changed the max fill level to try and kick dp into balancing up disk 1 but it just moved 90GB off to drive 2.

 

I have kicked off another manual balance to see if it will do anything - i will report back

 

any ideas on what to try??

[Spider99]

Manual balance did not change anything

[Spider99]

Have pushed 250GB plus at the pool tonight - its still balancing but no change in profile

 

three archive drives empty

[Spider99]

calling @Christopher any ideas poll still lopsided and the balance is not changing over time?

[Christopher (Drashna)]

Sorry, I've been not feeling 100%, and haven't been checking the forums regularly. 

 

And because I'm still not feeling 100% here ... 

 

All of the other balancers besides the SSD Optimizer are disabled.  No file placement rules exist, 

And are you're using the Ordered File Placement option  in the SSD Optimizer?

 

But yeah, I think something is going on in the SSD Optimizer balancer, and it's acting weird. 

I'm having weird issues on my server, as well.... But honestly, I tend to discount a lot of issues because i"m used to weird things occurring on my server.... 

[Spider99]

@Christopher

 

thanks for coming back and i hope you feel better soon

 

Yes the placement rules are set in ssd optimiser - disks are set with order placement 01 to 24 in both duplicated and unduplicated

 

What i just noticed is that the SSD Opt "drives" list which has always been random has the "empty" drives as the last three in the list - this might account for the no files - as 19 has just been allowed to have a few gb after copying another 0.5tb to the pool - 9 and 10 were also given some when i deliberately over whelmed the cache - but they were then emptied again and before the ssd'd??? 

 

What changed in 734 over the previous versions - did the rules get changed at all?

 

Pool is set to balance ever 4 hours - not that things are changing

[Christopher (Drashna)]

The measurement code got overhauled.  But ironically, a lot of it was rolling back to the 2.1.1.561 code.  

 

And there was a change to the SSD Optimizer balancer as well, allowing you to specify MB, GB, etc for the "other t his much disk space" for filling (setting it to a lower number is fantastic, so you can almost totally fill a drive, so you don't have 100+GB free on each drive, which is ~2TB of space for me).

 

There is definitely something going on here, as I'm definitely experiencing similar behavior. 

 

Alex should get to this and other DrivePool issues very soon, and since I have "accidently replicated" the issue, it should be relatively easy to fix. 

 

 

 

(and you can always see the full list of changes here:)

http://dl.covecube.com/DrivePoolWindows/beta/download/changes.txt

[Spider99]

ok cool if its not just me then i will wait as its not a major issue just annoying

[Spider99]

what i found today was setting the drive space equaliser to % rather than free space remaining has it appears solved the imbalance

 

the predicted space used on most drives is now close to 50% - the three which were not being filled or only to a small amount are now due to be filled to 50 %

 

have a couple of odd ones but much better balance

 

it still has to actually do it - approx 28% done at present

 

thing i still dont like is the balance is prioritising the archive drives over the ssd cache - which is the wrong way around - as this should be emptied first! 

 

Especially as have had an ssd go futt on me - no warning - nothing from scanner - just not there after a reboot - it did have some files on it but not sure what yet as was experimenting with immediate duplication off - typical - nothing lost as it was stuff i had downloaded off the net so i can get it again

 

will pull that drive and test it tomorrow when balance done.

[Christopher (Drashna)]

Well, sorry for the delay and missing it...

 

The SSD Optimizer balancer is a highly specialized balancer, and it can (will) have issues running with any other balancers, especially any of the other Balancer Plugins (Ordered File Placement, and Disk Space Equalizer), as these drastically change how data is handled, as well. 

[Spider99]

well with time the disks have a much better spread of data - none of them are consistent in amount - but broadly they are "even" ish

 

the ssd came back to life on a reboot and has stayed up since - odd

[Christopher (Drashna)]

well with time the disks have a much better spread of data - none of them are consistent in amount - but broadly they are "even" ish

 

the ssd came back to life on a reboot and has stayed up since - odd

 

Well, that's good to hear, at least.

 

As for the SSD, if it does it again, I'd recommend RMAing it regardless.  i've had similar happen and then found issues with the drive.