Jump to content
Covecube Inc.
  • 0
habe

Problems with Ordered File Placement plug-in

Question

I recently changed over from FlexRaid (due to ongoing issues) to using Scanner + Drivepool + SnapRaid. I am running Windows 7 64bit OS. I liked the way that FlexRaid pooled my drives but, had many issues on the RAID side. Since all of my data on my drives is very static; movies, TV shows, music, etc., I have no problem filling my data-only drives up to 99%. All of my data drives are 3TB so that leaves me with appx 27.9GB free per drive.

 

I liked how FlexRaid would use all drives space, filling each to appx 99% capacity before moving to the next drive which is why I want to run the Ordered File Placement plug-in.

 

Switching over from Flex, my drives were pretty balanced out at close to 99% except for the last drive which was only up to about about 50%. I set up my Drivepool, turned off all balancing, disabled all plug-ins, installed the OFP plug-in and changed the settings for the OFP plug-in.

 

Initially, all drives were about 27 - 50GB free space. I tried moving some some files to the pool but the files were always being placed on the drive with the most space which is only about half full (Drive F for example had 48GB free so should have been able to take a 1.6GB file). No matter what, all files went to the last drive with the max free space until, I changed the settings under "New Drives" & "Overflow" on the rules page. Once I did that, it seemed to start working correctly (odd thing however is changes I make to that page don't seem to stay selected). However, I added some new TV shows (all about 1.6GB) and they were placed on the top drive (drive F:) and that drive has been taken down to only 8GB free which violates my setting of 99%! The red pointer now has moved to the "K" drive which I assume is next to receive files that it is capable of holding which would be coorect since drives "G - J" all should be maxed out around 27.5GB free.......

 

Any ideas?

 

thanks,

 

habe

post-1935-0-90725300-1428940337_thumb.jpg

post-1935-0-62850500-1428940339_thumb.jpg

post-1935-0-52185400-1428940340_thumb.jpg

post-1935-0-35955400-1428940341_thumb.jpg

post-1935-0-41281600-1428940342_thumb.jpg

post-1935-0-05550100-1428940343_thumb.jpg

Share this post


Link to post
Share on other sites

13 answers to this question

Recommended Posts

  • 0

Well, if balancing is disabled, then it's going to default to the standard method, which is to place the files on teh disk with the most free space.

 

Enable balancing, and make sure the Ordered File Placement balancer is set to only affect new files. 

 

 

 

As for the red arrows, these represent the "real time placement limiters" meaning that new files will only be placed up to these limits. Once you hit that, it will place files on another drive.

 

 

And as for the files being placed on teh wrong disk, did you have a file placement rule limiting the placement of the file?

If you did, then the settings on the "General" balancing tab will control how this is handled.

Share this post


Link to post
Share on other sites
  • 0

Christopher,

 

When you say enable balancing, you mean select "Balance Immediately" under "Automatic Balancing"?

 

I guess I don't completely understand what settings control balancing. For me, I don't want Drivepool to perform ANY balancing on my pool drives other than control file placement and fill drives up to my 99% setting thereby utilizing max space. (It looked to me that I had all settings where they should be to accomplish that)

 

Right now, I have 28.4GB free on my top drive which is F. My balancing max is set at 99% per drive which would leave my drives appx 27.9GB free. Whether I select "Balance Immediately" or leave it unselected and, with all of the current settings you can see, if I try and copy a 20GB to my pool drive, Drivepool wants to place it on that F drive and take it down to only 8GB free. If I didn't have it set properly as you say and Drivepool was defaulting to it's standard method, it should be putting the 20GB file on the L drive which, it is not doing either.......

 

Confused.....

 

habe

Share this post


Link to post
Share on other sites
  • 0

Yes. 
 
As for balancing, if you don't want stuff moved around at all... check the specific balancers.
The simplest way would be to disable ALL of the balancers but the Ordered File Placement Balancer plugin. 
 
I say simpler, because the issue with the "Prevent Drive Overfill" balancer may be what was causing issues for you in the past (it sounds like it may have been the case).
 
Disabling all the but the one you want means that only IT will be used. And since it's the Ordered File Placement balancer, it will ensure that it's settings are the ONLY relavient ones, outside of the File Placement Rules.
 
 
And speaking of which.
http://dl.covecube.com/DrivePoolBalancingPlugins/OrderedFilePlacement/Notes.txt

By default, this plug-in will override any folder based or pattern based file placement rules. In order for folder based
or pattern based rules to take priority over this plug-in you can change the balancing settings on the "Settings" tab
as follows:

  • UNCHECK "File placement rules respect real-time file placement limits set by the balancing plug-ins."
  • CHECK "Balancing plug-ins respect file placement rules."
  • UNCHECK "Unless the drive is being emptied."

Make sure that you set this, if you're using both the Ordered File Placement balancer and file placement rules.

 

Otherwise, ... well, you may notice issues. Which may be the case here. And sorry for not picking up on that earlier, as well.

Share this post


Link to post
Share on other sites
  • 0

Just tried all of what you posted, checked all settings again. All balancers except OFP remain disabled and I don't have any file placement rules.

 

Initially, I saw the same behavior. OFP was still trying to place a 16GB test file on my F drive which is already close to my 99% full rule at 28.5GB free. My K drive has 48.xGB free so, it should be the one getting the 16GB test file.....

 

I reset the OFP balancer settings and started over, settings same as before. Now, no matter what, ALL files (even a 1.6GB file) are going to L drive which has the largest amount of free space (around 50% free) despite the OFP balancer being active.

 

I keep getting very inconsistent results with the OFP plug-in. Unfortunately, OFP is the method that I want my drive space utilized with no other drive balancing (due to using SnapRaid).

 

Thanks for your help,

 

habe

Share this post


Link to post
Share on other sites
  • 0

Christopher,

 

Is there anything else I can send you to try and get to the bottom of this issue? I played around with it for hours, trying different settings combinations, and it still doesn't work for me in the way that it is supposed to work.........

 

Is there a way to uninstall plug-ins once they've been installed? I'd like to uninstall the OFP plug-in and re-install it to see if that makes a difference. 

 

Request: Could someone who has the Ordered File Placement plug-in working correctly post some screen shots of their settings? I would greatly appreciate it.... For me, OFP is either placing files on drives that violate the "do not fill drives above xx%" limiter setting or, it puts everything on the drive with the most free space (which is Drivepools default method).

 

thanks in advance,

 

habe

Share this post


Link to post
Share on other sites
  • 0

Yes, actually, could you get the log files from the system:

http://wiki.covecube.com/StableBit_DrivePool_2.x_Log_Collection

Enable that, and add some files. Try to note the time when it violates the balancing stuff.

 

 

Reinstalling the plugin shouldn't make any difference here. It's basically just a dll that we load, and influences/controls the balancing engine. You can go ahead and reinstall, but it shouldn't matter.

Share this post


Link to post
Share on other sites
  • 0
Christopher (Drashna), on 15 Apr 2015 - 10:21 PM, said:Christopher (Drashna), on 15 Apr 2015 - 10:21 PM, said:Christopher (Drashna), on 15 Apr 2015 - 10:21 PM, said:

Yes. 

 

As for balancing, if you don't want stuff moved around at all... check the specific balancers.

The simplest way would be to disable ALL of the balancers but the Ordered File Placement Balancer plugin. 

 

I say simpler, because the issue with the "Prevent Drive Overfill" balancer may be what was causing issues for you in the past (it sounds like it may have been the case).

 

Disabling all the but the one you want means that only IT will be used. And since it's the Ordered File Placement balancer, it will ensure that it's settings are the ONLY relavient ones, outside of the File Placement Rules.

 

 

And speaking of which.

http://dl.covecube.com/DrivePoolBalancingPlugins/OrderedFilePlacement/Notes.txt

Make sure that you set this, if you're using both the Ordered File Placement balancer and file placement rules.

 

Otherwise, ... well, you may notice issues. Which may be the case here. And sorry for not picking up on that earlier, as well.

Ha! Thanks was having a problem with using snapraid and the ordered file placement plugin respecting file placement rules solved it in a jiffy thanks. Trying snapraid because my limited server filling up with dup files. I have yet to try a restore with snapraid still trying to get things in order and this really helps out. I also haven't overfilled a drive yet to reproduce habe's question I'm sure it won't be long before that happens. I can set the limiter down to see if I can reproduce the problem here if you all would like me to try. I will be doing a snapraid sync soon so let me know. 

Share this post


Link to post
Share on other sites
  • 0

We are writing about snapraid here a good question I have least to me. Christopher or anyone with knowledge do you think the .covefs file should be synced with each drive in a snapraid backup? It is a hidden system file and therefore ignored by snapraid the pool part file is synced even though it is hidden. Only system files are not. Can change that if it is needed. If it is written with every new disk install to drive pool then I think it is not needed when the pool data is restored and can use that default .covefs. The pool part file is unique and can be changed in the snapraid .conf file when restoring a new disk. What to do with this file .covefs is the question?  Thanks

Share this post


Link to post
Share on other sites
  • 0

Here's a few things I done with snapraid and drive pool getting them to work nice with each other.

1. Only allowed one snapraid.content file on the pool. Seems even thou drive pool does not write the file sees it as a duplicate and keeps trying to balance it if out of sync. 

other snapraid.content files go on parity disk, c drive, or any other spare disk not in pool with proper pointers.

2. Used the poolPart id as the root directory for disk.

3. For failed disk replace with new drive or partition of same or larger size and drive letter of failed disk

4. Install new disk to drive pool and copy the new poolPart id to snapraid.conf file of the replaced disk. Note: you have to right click the hidden poolPart id selecting properties to be able to copy the file name from the dialog box.

5. run the fix command.

I'm using the ordered file placement, scanner with only warning no movement of files except overheat don't know if that is good not tested, Drive Limiter unchecked duplication don't know if needed hasn't hurt, for balancer settings as stated by Christopher above. Did a 2tb restore by shutting down and removed the disk from server bay. Then restarted and removed the missing disk from drive pool very fast. Installed a 4tb disk and changed the drive letter to match the remove drive installed into pool. Edited the snapraid.conf file appropriately with new poolPart id. Run the fix and everything in pool where it should be all's well. Only thing the files showed up as grey other files till I did a re-measure nice green bar the pool is synced great.

Share this post


Link to post
Share on other sites
  • 0

The ".covefs" folder is mostly used for reparse point information. If you're not using any on the pool, then the folder doesn't need to be backed up.

 

However, if you're using reparse points, then you need the contents of this folder (which is hard coded to x3 duplication, IIRC).

 

 

 

1. Only allowed one snapraid.content file on the pool. Seems even thou drive pool does not write the file sees it as a duplicate and keeps trying to balance it if out of sync.  other snapraid.content files go on parity disk, c drive, or any other spare disk not in pool with proper pointers.

 

If the file is in the pool (either the drive, or in the folder structure), then yes, it will be balanced.

You could use the File Placement Rules to lock the file's location to a specific drive, though.

 

 

Also, if it helps, the PoolPart folders don't need to be hidden (I think SnapRAID did have an issue with that... but i could be wrong).  We hide them for "convenience", so that people don't mess with them. But it's not required to be hidden.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

×
×
  • Create New...