Jump to content
Covecube Inc.
  • 0

Is it possible to craft file placement rules such that files in folders stay grouped together?


Question

I saw that this was requested as early as 4 years ago and im not sure if it ever got traction. As you would expect I'm asking this question coming from a place of media storage. I intend on having SnapRaid underneath DrivePool to handle my fault tolerance, but Im looking to place files intelligently to minimally spin up drives when files are accessed.

This is accomplished with MergerFS in linux by specifying epmfs as the default policy (of all drives where the path exists, choose drive with most available free space).

Drive1: A:\Media\Movies\TheFirstMovie (2020)\
Drive2: B:\Media\Movies

If I wrote a metadata file (poster, srt subtitles, etc) belonging to TheFirstMovie (2020), it would write it to Drive 1. This is ideal for serving media as like-files are similarly requested. If I wrote a new movie TheSecondMovie (2021), since both Drive1 and Drive2 have the path \Media\Movies, it will choose the drive with the most free space (lets say its Drive2), creating B:\Media\Movies\TheSecondMovie (2021) on Drive2. This accomplishes balancing while minimizing the chances of spinning up the entire array of disks when a single media entry is accessed.

I've looked through the documentation and forums and couldn't quite get a clear answer on if this was possible in the current DrivePool version. I could see static / wildcarded pathing support for directing files to drives, but that's only a half solution as that would force us to track specific movies or tv series and make entries for them in the config. This logic seems easy to create in an automated fashion.

Is it possible?

Link to post
Share on other sites

3 answers to this question

Recommended Posts

  • 0

I can see the advantages. It's not an available option in the current DrivePool version, though I'd imagine it should be possible to add as (or to) a balancer plugin.

How does mergerfs/epmfs handle the situation where the file being written doesn't fit in the remaining space of the chosen drive (because you can't guarantee in advance the size of an incoming file)?

Link to post
Share on other sites
  • 0

I believe the default is to just throw a write error (out of space), but I imagine there's an alternative way to handle it. More here: https://github.com/trapexit/mergerfs#policy-descriptions

"If all branches are filtered an error will be returned. Typically EROFS (read-only filesystem) or ENOSPC (no space left on device) depending on the most recent reason for filtering a branch. ENOENT will be returned if no elegible branch is found."

I may look into writing a balancer plugin if its possible with the current framework. Thanks for the response! I hope it does get implemented. If I were to use the current version of Drivepool, whats the closest I can get to spinning up the least amount of drives? Ordered File Placement filling up one drive at a time?

Link to post
Share on other sites
  • 0

Yes, it'd be the Ordered File Placement balancer (or cjmanca's All In One balancer, which is what I'm currently using instead of multiple others).

My programming ability (such as it is) would be enough that I could write a brute-force shell script to organise the pool as needed on a manual/scheduled basis, but an actual balancer plugin would obviously be far superior.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...