Jump to content
  • 0

how to force that a folder content is always copied on only one physical drive, at a predifined depth of folder hierarchy ?


iomgui

Question

Hello everyone

 

Ok here is what i would like to do, lets say my structure on the pool is like this 

folder A / sub Ai / Sub Sub Ai,j 

folder B / sub Bi / Sub Sub Bi,j

etc

 

At some point in folder hierarchy, lets say starting at the 3rd subfolder (for exemplae sub sub A i,j).

What i would like is a way when everytime i transfer a folder that is at 3rd or more deeper in foilder structure, that this folder content is always put on the same drive, as long as there is enough space. I do not care on which drive it is put, just that it is maintained on only one drive.

Also, anytime i m adding files to that folder, thoses files should follow the same rule and be put on the same drive as the drive of the original folder.

I m looking for a rule that could be set once and would never require me to a destination drive.

I m also wondering if such a rule could rebalance an existing pool rearranging every files so that SubSubA etc are only on one physical drive, even though its not the case actually because of what driveopool did so far.

 

Many thanks in advance 

Link to comment
Share on other sites

8 answers to this question

Recommended Posts

  • 0

Thanks,

But i dont see how for incoming folders that would be created on the 3rd levels. 

For existing folders i can select, one by one, to balance them on only one drive. But i m looking for a way to do this automaticly so i dont have to add a rule for any 3rd level folder that would be created and probably with a content already dispatched on different physical drives.

Unless i missed something

Link to comment
Share on other sites

  • 0

A quick test with a pool of two drives shows that Christopher's suggestion has merit.

For example if you create a File Placement Rule of pattern "\*\*\a*\*" -> drive K:\ then any new file whose path matches \a*\ at the 3rd folder level will be put on K:\ drive. You will need to balance the pool (manually or automatically) to move existing files as well.

So you would use "\*\*\*\*" to match ALL files that are three or more folders deep; one "\*" for each folder level and a final "\*" for all content within it. E.g. it would match P:\level1\level2\level3\somefile and it would match P:\level1\level2\level3\somefolder and it would also match any files, folders and subfolders within P:\level1\level2\level3\somefolder.

Link to comment
Share on other sites

  • 0

I understand, but it wont do what i need :

for each new folder created on the level where no disk separation should be done, it will not be possible to automaticly create this folder and its content on one and only one aleatory disk, choice of alleatory following other balance rules. (it would work if the directory where its created in is already assigned a physical drive, but its not the case when you create it at the level where the choice of physical drive should be allowed).

Because the simpliest solution would then be to only allow to fill one disk with existing plugins, withouth the need to create rules at directory level, and then switch after a few months to another physical disk (without any plugin rebalancing depending on disk equilibrium). By simpliest solution, i mean the one that would require the minimum number of manual intervention on my side to change balancing rules or create directory rules. Though this simpliest solution has the huge disadvantage to not share the risk of disk failure by balkancing the amount of data between disks.

 

I dont see any way to automate it so far, and i found a post in the meantime saying it was not possible with actual pluggins. I can link it here if its authorized and usefull

Link to comment
Share on other sites

  • 0

This one illustrates it as usefull to store movies with limiting the number of disk spining, and keeping balance between disks.

I share the same need, with also the need that when one disk fail, i accept to loose some folders, but it would be a problem to loose part of all my folders ;) because of the way i archive things, the amount of work to rebuild the data without duplication would be HUGE (find the files missing and rescanning them not by batch but by picking one or 2 files in any physical folder cold storage from the office, for example).

I know i could duplicate or use snapraid, but even with snapraid for example, with 1 parity disk, you still have to consider a 2 disk failure. And in the case of a 2 disk failure i need to minimize the effort of rebuilding the data. But its very similar to the movie collection explained in this post.

https://community.covecube.com/index.php?/topic/5889-is-it-possible-to-craft-file-placement-rules-such-that-files-in-folders-stay-grouped-together/

Link to comment
Share on other sites

  • 0
20 hours ago, iomgui said:

but it would be a problem to loose part of all my folders

I recently ran into this problem myself. I use DrivePool as my home media center. I don't have duplication set on most of my folders because I have backups on HDDs in storage in my closet. I figured if I lost a drive, then I would just pull out the backups and copy the files back into the pool. However, for example, I discovered that any Album folder in DrivePool might have tracks in that folder spread out on 2 or more drives. So instead of losing 1000 complete album folders on that drive, I found out that I lost more like 2000+ partial album folders that had to be restored from backups.

For some reason, I thought DrivePool would write complete folders to any drive, but it appears that is not the case. I have started adding 10% .par2 recovery files to my folders, which will verify the files, repair some missing or damaged files, or tell me that I needed more recovery files and it is time to get out the backup from the closet.

Is it possible to tell DrivePool to write complete folders to any drive and not split the files over 2 or more drives? I would think File Order Placement would help, but there seems to be no guarantee that changes made later to files in a folder would end up on the original drive.

Link to comment
Share on other sites

  • 0

The only easy solution i found (easy = not requiring that you intevrene each time you add a folder) is to set balancers so that they fill one disk to 90% (for example) before starting to fill another disk.

then you can either  :

- do nothing only 1 folder could have files on 2 disks when it changes disk to the next one, when one disk reached max capacity as defined by you

- or you could every now and then (every 3 months for example), change the rule to fill another disk even if you have reached only 30 or 40% for example. That way you will have 0 folder with files on more than 1 drive

 

But i agree, such a balancer option would be nice, but mostly concerns people without parity disks or duplicate (managed by drivepool). If you have a backup that is not at the same place (only strategy to prevent complete lost in case of fire or flooding, or burglary), then you can see how usefull it would be

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...