Jump to content
  • 0

DrivePool Folder Fragmentation


PhotonJunkie

Question

I recently set up WHS 2011 with DrivePool to store media files. Other than an early glitch when all my shared folders went missing from the WHS Dashboard and had to be "Recreated", DrivePool has been great. Blu-ray titles occupy most of the drive space and there isn't enough room to do duplication in the pool. I'm concerned from what I'm reading here that any given title will have its folder contents irretrievably scattered across the 8 physical disks in my pool. If that is the case, then I take it that a partial fix is the "Ordered File Placement" plug-in, but that there isn't any sure way to keep the contents of any given DVD or BD title entirely in its original folder?

 

On a related question, If I add a populated drive to the pool, can I then "move" all the existing folders on that drive into the pool without them being rewritten and fragmented? 

Link to comment
Share on other sites

9 answers to this question

Recommended Posts

  • 0

Question is, why would you care if all the files from a particular BR were spread over different physical disks? The point of DrivePool is you no longer need to worry/care where files are placed across physical disks. If you want to bring them back together again just move them from the pool drive to a normal standalone drive (your C drive or whatever) and they'll all be together again on one physical drive, not that it makes any difference.

Link to comment
Share on other sites

  • 0

I would imagine because he said that there isnt enough room to do any duplication

 

so while he wants pooling, he wants individual folders to stay together

 

so if a drive dies.. he will only loose whatever was in the folder on that dead drive (which is not ideal.....)

 

rather than simply losing a fraction of the files in every single folder (which would be a huge pain in the ass)

Link to comment
Share on other sites

  • 0

I recently set up WHS 2011 with DrivePool to store media files. Other than an early glitch when all my shared folders went missing from the WHS Dashboard and had to be "Recreated", DrivePool has been great. Blu-ray titles occupy most of the drive space and there isn't enough room to do duplication in the pool. I'm concerned from what I'm reading here that any given title will have its folder contents irretrievably scattered across the 8 physical disks in my pool. If that is the case, then I take it that a partial fix is the "Ordered File Placement" plug-in, but that there isn't any sure way to keep the contents of any given DVD or BD title entirely in its original folder?

 

On a related question, If I add a populated drive to the pool, can I then "move" all the existing folders on that drive into the pool without them being rewritten and fragmented? 

The "File Placement Rules" feature can do this, actually. 

It may require extensive micromanagement, depending on how your pool is setup, but the UI should allow you to easily do this.

http://stablebit.com/Support/DrivePool/2.X/Manual?Section=File%20Placement

 

As for fragmentation, you can defragment the disks in the pool, without any issues.

 

 

 

Question is, why would you care if all the files from a particular BR were spread over different physical disks? The point of DrivePool is you no longer need to worry/care where files are placed across physical disks. If you want to bring them back together again just move them from the pool drive to a normal standalone drive (your C drive or whatever) and they'll all be together again on one physical drive, not that it makes any difference.

So that is a drive does fail, you can be certain that you have all of the contents of a particular folder (or none at all). 

 

 

I've been there (not blue ray rips, but re-encoded files), and it sucks.  It may be worth investing in enough drives to duplicate everything.

Link to comment
Share on other sites

  • 0

Thanks Christopher. I will check out the File Placement option. My DrivePool is wonderfully convenient but it isn't my primary backup so it won't be a complete disaster when a pooled drive suddenly fails. However, the scattered folder contents will break many or even most of the BD titles that took most of a week to feed in. A ripped Blu-ray title folder can have 10s of sub-folders and hundreds of individual files with names like 00023.m2ts. The only practical repair for a broken DVD/BD media pool would be to rebuild it from external sources.

 

I am running the StableBit Scanner on the pool so I expect that to provides a bit of insurance.

 

I'm still wondering how to add a fully populated drive to the pool.  Can I somehow "move" all the existing folders on that drive into the pool without them being rewritten and their contents potentially scattered?

 

It's good to know you can do a routine defrag on a drive in the pool without breaking the pool but that isn't the fragmentation I was worried about. 

Link to comment
Share on other sites

  • 0

I would imagine because he said that there isnt enough room to do any duplication

 

so while he wants pooling, he wants individual folders to stay together

 

so if a drive dies.. he will only loose whatever was in the folder on that dead drive (which is not ideal.....)

 

rather than simply losing a fraction of the files in every single folder (which would be a huge pain in the ass)

Exactly right! I'm a little surprised that keeping a folder's contents together isn't DrivePool's default behavior. Sooner or later bad things happen to all forms of storage media, and software design that makes recovery unnecessarily harder should be avoided. As for drive space, I would have to upgrade to all 8 TB drives to do full duplication and still fit in the box and I'm waiting a year of two to see how reliable these shingled drives turn out to be (I have 1 in the pool). In any case, data duplication on a single platform is not a serious backup method - I've seen a catastrophic power supply failure fry practically every component on a couple of PCs. 

Link to comment
Share on other sites

  • 0

That's why you should also run something like Snapraid so if you lose a drive, you use the parity drive to restore all the drive's contents to a replacement drive. Then it doesn't matter where files are stored on physical drives.

Link to comment
Share on other sites

  • 0

@PhotonJunkie,

 

You could seed the pool, and then set up a rule, right away.

http://wiki.covecube.com/StableBit_DrivePool_Q4142489

Don't reset the settings, just remeasure the pool.

That should get what you want, I think.

 

As for the fragmentation, I wasn't sure what you were getting at, and I wanted to make sure I covered all bases. 

You were referring to fragmenting the folders, right?

 

 

As for the drive loss, that's always unpleasant. However, one could argue that having the entire contents of a folder on a specific drive, that you're more likely to lose EVERYTHING from that folder, rather than just partial contents. 

And that's part of why we recommend duplicating everything, if it's important at all. 

This way, if a disk fails or starts to go bad, you can remove it right away and just reduplicate the files after the disk is removed (skipping the duplicate files during removal).

 

And Parity is definitely another solution, but not one that I'm found of. You're exchanging raw capacity at the expense of additional (significant) CPU usage and additional load on the drives.  With duplication, you can access ALL of the files taht are duplicated when a disk fails/goes missing, and only have to copy them over to another disk to rebuild (and the StableBit DrivePool service handles this automatically once you've removed the disk), whereas parity ... the files may not be accessible until you "decompress" (oversimplification) them from the parity data.

However, this is a well worn debate (at least to me), so if you have a preferred method, by all means, use it.

Link to comment
Share on other sites

  • 0

@PhotonJunkie,

 

You could seed the pool, and then set up a rule, right away.

http://wiki.covecube.com/StableBit_DrivePool_Q4142489

Don't reset the settings, just remeasure the pool.

That should get what you want, I think.

 

As for the fragmentation, I wasn't sure what you were getting at, and I wanted to make sure I covered all bases. 

You were referring to fragmenting the folders, right?

 

 

As for the drive loss, that's always unpleasant. However, one could argue that having the entire contents of a folder on a specific drive, that you're more likely to lose EVERYTHING from that folder, rather than just partial contents. 

And that's part of why we recommend duplicating everything, if it's important at all. 

This way, if a disk fails or starts to go bad, you can remove it right away and just reduplicate the files after the disk is removed (skipping the duplicate files during removal).

 

And Parity is definitely another solution, but not one that I'm found of. You're exchanging raw capacity at the expense of additional (significant) CPU usage and additional load on the drives.  With duplication, you can access ALL of the files taht are duplicated when a disk fails/goes missing, and only have to copy them over to another disk to rebuild (and the StableBit DrivePool service handles this automatically once you've removed the disk), whereas parity ... the files may not be accessible until you "decompress" (oversimplification) them from the parity data.

However, this is a well worn debate (at least to me), so if you have a preferred method, by all means, use it.

Thanks for the good news on how to feed a populated disk into the DrivePool. That process looks about as simple as I hoped.

Link to comment
Share on other sites

  • 0

Thanks for the good news on how to feed a populated disk into the DrivePool. That process looks about as simple as I hoped.

Yup. :)

 

The big thing here is that you need to stop the service before starting. Otherwise, it will continue to duplicate and balance, "as necessary".  The other, is that once you've finished, you absolutely must re-measure the pool! This rebuilds the information that is used for balancing and rechecks the duplication status. 

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...