Jump to content
  • 0

Understanding file placement


stalfos

Question

Hi,

I am trialing Drivepool currently and am trying to get a handle on how it places files.

 

For example in my initial testing (without changing defaults) I encountered many situations like this:

E:\movies\Big Buck Bunny\Big Buck Bunny.avi

F:\movies\Big Buck Bunny\Big Buck Bunny.srt

(the subtitles for a given movie were placed on a different drive to the movie itself, which will surely cause two spinups to play the movie)

 

I should also say I intend to use it in combination with snapraid, so any post-placement moving/balancing is unwanted.

 

The userguide explains balancing as  "the act of reshuffling your existing pooled files among the physical disks that are part of the pool."

Does this mean balacing is entirely separate to initial file placement?

For use with snapraid I should untick automatic balancing and also untick all the balancing plugins?

 

With balancing plugins disabled, what determines where new files go?

Link to comment
Share on other sites

9 answers to this question

Recommended Posts

  • 0

First, thank you for your interest in our product(s)! 

 

 

The default file placement strategy is to place files on the drive(s) with the most available free space (measured absolutely, rather than based on percentage). This happens regardless of the balancing status.   In fact, it's the balancers themselves that can (will) change the placement strategy of new files. 

 

For what you want, that isn't ideal.... and before I get to the solution: 

 

 

I should also say I intend to use it in combination with snapraid, so any post-placement moving/balancing is unwanted.

 

The userguide explains balancing as  "the act of reshuffling your existing pooled files among the physical disks that are part of the pool."

Does this mean balacing is entirely separate to initial file placement?

For use with snapraid I should untick automatic balancing and also untick all the balancing plugins?

 

With balancing plugins disabled, what determines where new files go?

 

The first issue here is that there is a misconception about how the balancing engine works (or more specifically, with how frequently or aggressive it is).

 

For the most part, the balancing engine DOES NOT move files around.   For a new, empty pool, balancing will rarely, if ever move files around.  Partly because, it will proactively control where files are placed in the first place. 

 

That said, each balancer does have exceptions here.  But just so you understand how and why each balancer works and when it would actually move files, let me enumerate each one and give a brief description of them. 

 

  • StableBit Scanner (the balancer). 
    This balancer only works if you have StableBit Scanner installed on the same system.  By default, it is only configured to move contents off of a disk if "damaged sectors" (aka "Unreadable sectors") are detected during the surface scan.  This is done in an attempt to prevent data loss from file corruption.  
    Optionally, you can do this for SMART warnings as well. And to avoid usage if the drive has "overheated". 
    • If you're using SnapRAID, then it may be worth turning this balancer off, as it isn't really needed
  • Volume Equalization.
    This only affects drives that are using multiple volumes/partitions on the same physical disk.  It will equalize the usage, and help prevent duplicates from residing on the same physical disk. 
    • Chances are that this balancer will never do anything on your system. 
  • Drive Usage Limiter
    This balancer controls what type of data (duplicated or unduplicated) can reside on a disk.  For the most part, most people won't need this.  
    We recommend using it for drive removal or "special configurations" (eg, my gaming system uses it to store only duplicated data, aka games, on the SSD, and store all unduplicated data on the hard drive)
    • Unless configured manually, this balancer will not move data around. 
  • Prevent Drive Overfill
    This balancer specifically will move files around, and will do so only if the drive is 90+% full by default. This can be configured, based on your needs. However, this will only move files out of the drive until the drive is 85% filled. 
    • This is one of the balancers that is likely to move data.  But this will only happen on very full pool.
      This can be disabled, but may lead to situations where the drives are too full.
  • Duplication Space Optimizer.
    This balancer's sole job is to rebalance the data in such a way that removes the "Unusable for duplication" space on the pool.   
    • If you're not using duplication at all, you can absolutely disable this balancer

 

 

 

So, for the most part, there is no real reason to disable balancing.  Yes, I understand that it can cause issues for SnapRAID. But depending on the system, it is very unlikely to.  And the benefits you gain by disabling it may be outweighed by the what the balancers do.  

 

Especially because of the balancer plugins. 

 

 

Specifically, you may want to look at the "Ordered File Placement" balancer plugin.  This specifically fills up one drive at a time.  Once the pool fills up the disk to the preset threshold, it will move onto the next disk. 

 

This may help keep the contents of a specific folders together.  Meaning that it may help keep the SRT file in the same folder as the AVI file. Or at least, better about it than the default placement strategy.  This won't guarantee the folder placement, but significantly increases the odds.

 

That said, you can use file placement rules to help with this.  Either to micromanage placement, or ... you can set up a SSD dedicated for metadata like this, so that all of the SRT and other files end up on the SSD.  That way, the access is fast and the power consumption is minimal. 

Link to comment
Share on other sites

  • 0

I know what the OP is trying to achieve and it is something that always bugged me as well. Is there any way to submit a feature request to Alex for this specific type of placement aka folder split level.

 

Example: if the folder split level was to only split the top folder let's call it TV Shows, and TV series you put in there could go on any disk. However once a TV Show folder was created such as "NCIS", all subsequent folders or files would be placed within the NCIS folder on that same disk. This would eliminate my, the OP's, and I assume a bunch of other users frustration.

 

For my use case it was a pain when let's say I added a new season of TV Show "X". New files would go on whatever disk based on my balancer settings. When in reality it would make a lot more sense to keep episodes of the same series on a specific disk with the rest of the episodes from that series.

 

The only option we currently have is to go into the hidden pool parts folder on that specific disk and move those files manually which is kinda counter productive. Just my $0.02

 

 

Sent from my iPhone using Tapatalk

Link to comment
Share on other sites

  • 0

First of all thank you for the detailed reply, I have a better understanding of it now.

 

For now I will use the Ordered File Placement plugin. It is a little bit cruder than what I was hoping for but it will suffice. It would be nice to see a 'high water' type allocation method.

 

I also agree with Cyberskulls request - folder split level would be a nice feature which helps to keep related files on the same disk.

Link to comment
Share on other sites

  • 0

You are very welcome.  

 

 

As for the "split folder" stuff, I've talked with Alex about this recently, so it's definitely on his mind. And that it keeps on coming up... it's something that I do bring up to him often (because you guys want it). 

 

I can't make any promises, but I've tossed a few ideas Alex's way.  Maybe we'll get this in a new version. Though, I can't promise anything. 

Link to comment
Share on other sites

  • 0

You are very welcome.

 

 

As for the "split folder" stuff, I've talked with Alex about this recently, so it's definitely on his mind. And that it keeps on coming up... it's something that I do bring up to him often (because you guys want it).

 

I can't make any promises, but I've tossed a few ideas Alex's way. Maybe we'll get this in a new version. Though, I can't promise anything.

This would be great. You know I'm on another platform right now but I do want to get back to using using Drivepool. It was just a pain in the butt and way too time intensive to keep going into the hidden folders trying to track down what Drive a TV series was on in order to add new episodes in the same location. So I needed a less time intensive solution.

 

So tell Alex to get on it :) I wanna use my DP licenses again!

 

 

Sent from my iPhone using Tapatalk

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...