daveyboy37

Doh! I figured it out. I didn't realize that there was no active "Equalizer" balancer installed by default. Therefore, I disabled all other balancers, installed the 'Disk Space Equalizer' and now it is balancing as it should (I think!). I will update you once the move is complete.

For a number of reasons. 
 
By default, Windows wants to use 4k clusters. This means that each allocation unit for files is this size. If you ahve a 10MB file, it's broken up into 4k chunks and placed on the disk.  The flip side is that each file uses 4k period. That includes folders, which have very little data.  
 
Additionally, even if the file (or the remaining chunk) uses a couple of bytes, it will take up the entire allocation unit. 
 
 
Now, why is this important? Well, a 64kb chunk is 16x the size of the 4kb one.  
This means that more data is guaranteed to be sequential.  This is of one main benefit: Fragmentation. 
 
Sinch each chunk is larger, that means that you're less likely to see fragmentation, Additionally, since more of that data is in once section of the disk, it means that there is less seeking being performanced on the disk, and may increase performance.
 
THis is the same reason some people recommend that you defragment your disk. 
 
 
 
The flip side, as I mentioned, that if you have a lot of small files, you're losing disk space. A cluster or allocation unit can only be used by one file. 
For an OS disk, or one with a lot of very small files, this can cause an issue, as you may run out of allocation units before you run out of disk space.   
However, for storage, a majority of the files are going to be much larger, and will benefit from the allocation unit size increase. 
 
 
 
 
So what does this equate to for you? Well, on my Seagate Archive drives, I saw fewer performance drop offs, and I saw about a 20MB/s speed increase on reads. 

Oh wow Thank you.
 
I had totally forgotten about the ability to manually type in rules. That's definitely what I'm looking for.  Because my movie collection is much larger than the single 8TB hard drive I had hoped for example as a starting point to just have movies starting with  #-H
 
So in effect all movies from "2 Fast 2 Furious" through to "Fury" Not necessarily an all in one rule but perhaps something like (in my case)
\SERVERFOLDERS\COUCHPOTATO MOVIES\A*
\SERVERFOLDERS\COUCHPOTATO MOVIES\B*
Would this work and are those rules correct for movies beginning with A & B?
 
As for deleting stuff it never ever happens (I have some amazingly bad movies in my collection)
The nearest I get to deleting is when SickRage replaces a season of web-downloads with a Blu-Ray release that It finds or I  purchase.
This is the very reason that shows will not go on the archive drives at least until a show is ended/cancelled and the quality is at optimum. This way the archive drives will hopefully be "write once" which as you imply are their intended purpose.
 
I cannot believe I forgot about the ability to write rules.... I tend to see a feature in programs think to myself "I will never use it" and then forget it exists.​
 
Anyway thanks for the help as always Chris... If you could confirm (or correct) the rules above It would be appreciated.

Well, ideally, you want our products to just sit there and do their thing.  
So I'm glad to hear that StableBit DrivePool has been working quite well, in that regards!

I also do the fly by the seat of my pants method..Everything ctitical is backed up three fold or more... If its a few TB of movies then c'est la vie.
This approach may change though when the 8TB drives come down in price a little more.
 
As for DrivePool itself... well (as another Plex user), the best endorsement I can give is that I have been using it since the very early beta stages. 
As someone who used to be extremely active on here I just don't have the time any more. BUT I don't need to be because it just works!!!
Im well over two years of not having touched my server (windows updates aside) in fact I actually feel guilty that I guess I just take DrivePool for granted. It sits there silently in the notification area doing its stuff. 

Alright guys, starting with build 511 file placement rules will no longer include added drives by default. There is now a new option to enable that on a rule by rule basis.
 
I've also added the ability to rearrange folder based rules, as long as you don't break folder depth rules. See attached image to illustrate what this looks like now.
 
Download: http://dl.covecube.com/DrivePoolWindows/beta/download/
 
Edit: Multiselect implemented in build 512.

Another question on File Placement Rules!!
 
I set all my folders to the specified drives with File Placement Rules. All went well and everything is on the drives that I want it to be on.
However I added a new drive earlier and Drivepool is set to allow anything and everything to go on it. So If my Music is set to be only on Mount point 4 adding another drive (Mount Pont 8) will automatically allow music to be placed on Mount Point 8 after going to all the trouble of making sure that the music was only on Mount Point 4.
Multiply this by about 20 server folders and its a pita.
 
Can I suggest that the default for new drives is that all the Folder Placement checkboxes are unticked and not ticked. I know adding a drive is not an every day event but I still think the default should be to allow nothing on the drive until you allow it. Had I not realised in advance and then hit balance all my neat and tidy music would have again been scattered over 2 drives.

Thank you to both yourself and Drashna for the explanation. I guess an end user with a very stable system doesnt see all the bugs that need fixing in the background.
 
 Will test out the new release as soon as i get home from work.
 
Oh and "product 3" !!!   Awesome news!

I fixed this by deleting the contents of C:\ProgramData\StableBit DrivePool and rebooting. Balancing completed now and all disks displaying correctly.

Thanks!
 
Migration worked like a charm!
 
Wished every software would be so terriffic
 
Keep up the good work!

I tried it a MONTH ago. Locally, it works great (minus a hiccup every once in a while), but outside my network..... buffers every 5 minutes or so. Or just starts dropping frames like crazy. 
And I have decent upload (4mbps), so it shouldn't be an issue.

@daveyboy37: Very, very well put!
 
@Umfriend:
And to each their own. But that is why we have a list of "extra" balancers, the framework for balancing, API for developing your own (if you're inclined), etc. If you're noticing a theme, "choice" is a big consideration for DrivePool, and we hope that it is appreciated.
 
And yes, Server 2012 allows for large backups. Specifically, the backup engine has always (well, the "Windows Backup" feature, not ntbackup) used VHDs for storing the backups. If you look it up, VHD's were limited to 2TB size, until Server 2012, which introduced the VHDX format which allows for a 64TiB per VHDX file.

Ok, so as mentioned in another thread I'm currently replacing all my 2TB drives with 4TB ones. 
I connected Two new 4TB drives to the server and added them to the pool. I then told Drivepool to remove 3 of the 2TB drives. (It's pretty sweet that Drivepool lets you queue up this as I'm currently working nights, so all happening while I either work or sleep.)
 
So the first drive evacuated fine and the last one is doing its stuff now. However the second drive seemed to go ok and removed itself from the pool. Mousing over the now removed drive in (Server2012 R2 Essentials) Drivepool shows it as still having 1.18TB of used space. Worse still it's now giving me permission denied messages so I cannot actually check if the drive is empty.
 
Meanwhile in Windows Explorer the relevant mount point for the drive says there is only 184MB of data used. So probably empty after all.
 
So how do i get permissions for the drive to check this. I think I'm correct in thinking that Drivepool Utilities will not reset permissions unless the drive is part of the pool? 
If Windows Explorer is correct, is the false reporting in Drivepool a bug? 
Worse case scenario.... I know roughly how much data was on the three drives that I'm removing and can work on the basis that if there is roughly that amount of data on the new drives it went ok and its just a false alarm.
 
Thanks in advance.
Dave
 
Edit:
 
Cancel all that, bored at work I stopped the last drive migrating and rebooted the server. I have permissions back for the removed drives and they are indeed empty which Drivepool now correctly reports.   
 
But this then leads to something else...  
 
I have now decided that i really do not want two pools... What's the best method for getting rid of the second pool?
I'm guessing simply moving the server folders from Pool B to Pool A , via the dashboard ?   But Pool B is all my duplicated data. Will this complicate things? 
 
 
 
 .

Glad to hear it.
 
And yeah, I was going to recommend that file. As I know it works (the dropbox link is to my account)

Since Drivepool becomes read only when you have a missing disk I don't believe you can add a new disk like you want to.
 
But you can still stop drivepool service and swop in your new 2TB drive (if new initialize and format via windows) and copy the content of the poolpart.xxxxxxxxxxx folder on 500GB to the new drive.  Shutdown server remove new drive readd old, start server then force remove 500GB drive (might be able to just remove 500GB drive and leave new 2TB when reinstalling old 2TB drive and just remove missing disk from dashboard).  Then add new 2TB disk. Move content into new poolpart.xxxxxxxxxxxxxxxxxxx on new 2TB.
 
DP really has nothing to do with a copy from disk to disk in this situation.  Just ensure the service is not running during this process.

I'm not sure what you mean by "over provisioning" here. 
(Specifically because all SSD over provisioning is usually done on the firmware level, and invisible to end users or even the system).
 
And yes, you could resize the partitions, and create a dedicated partition for a feeder disk.
 
However, I'd recommend a dedicated drive for the feeder disks. Why? To prevent sharing the IO with the system. As well, as prevent the system disk from getting "thrashed". It will receive a LOT of writes (depending on what you're doing), and may adversely affect the drive's life (SSD's life is very write dependent, though newer drives definitely last much longer). 
And if you're going to be using duplication, then ideally, you will want two disks.
 
 
As for if this "will be worth the trouble", are you doing a lot of writes? Such as a database on the Pool (like the Client Computer Backup database), or copying/moving files to the pool often? If speed isn't an issue, then this isn't worth it.
 
As for the disadvantages, I covered that already, above: wear and tear on the drives. Also, you'll want to dedicate a drive or two to this role. Which may or may not be pratical depending on your system.

Again this depends on organisation and on an individuals system and how they use it. I personally would never let the drives get to the position they would overflow. In my case Server 2012r2 and from past experience WHS2011 gives plenty of notice of such issues and thoughtful initial setup of the existing balancers will simply take care of the rest. The server backup on the operating systems mentioned are a pita to initially setup. You cannot just select which folders to back up. You have to browse to each individual serverpoolpart folder on each drive and select the relevant folders. With a 2 or 3 drive system that's no big deal but with 10-15 drives that's pretty time consuming. If music is scattered over a dozen drives and you miss selecting it on one of the drives backup will fail or at least be incomplete. Then it's back to all those drives to see what was missed. Mutiply that by the 10 or 15 different server folders and its a nightmare.  If my music is on two drives, then that's  two drives i need to check that it's being backed up properly.  
 
Having replaced the numerous 2TB drives with the 4TB ones I personally will have plenty of scope to add another 4TB drive as and when needed. The folder placement will simply be another option for those who feel they need it. 
 
Don't get me wrong. I am currently using two pools just to make sure all my flac files are duplicated but to be honest for me at least it just negates the benefit of having a pool in the first place. 
 
Anyway i'm glad you are starting to realise why limiting folder placement will be a massive benefit to some (if not all) of us. In years to come as your system grows you may also come to realise just how much of a benefit. Though hopefully you will never get as OCD as Drashna and myself.
 
Finally with regard to the drive failing, I actually see limiting placement as a good thing. My personal plan is to have all my "must not lose because i probably cannot replace data" duplicated over two drives. If one of those drives fails. My full data is still on the other drive. A much better scenario than a random drive failing and losing a couple of tracks off most if not every single album.

Yes it is.
 
Right now, Alex is busy trying to add Reparse Point support before adding this.

I've been thinking about how I can better communicate what is in store for each of our products, there are 3 now and another one in the works. Starting today I'll be setting up topics in each forum that I'll be updating on a regular basis. Each post will maintain what the future holds for each product.
 
I try to keep a lot of the development driven by user feedback, but most of that feedback doesn't happen in the public forum (but usually in tech support tickets). I'd just like to take this opportunity to highlight the direction that each product is heading in, a kind of roadmap.
 
I'll be setting up those posts today so look for them popping up soon in each respective forum.

This is something that I have hoped for, for a very long time. I really have never been keen on having things scattered around the pool. Especially with music... 10 album tracks scattered over 5 or 6 drives.. But then I'm probably a bit O.C.D.
 
I'm sure for people who have various devices streaming to different rooms this must be a good thing. Knowing that all the Disney films are all on one hard drive for the rug rats, and the teenage daughter can watch her twilight knowing its on a separate drive so no risk of intensive I/O and so on. And yes i know that this could be achieved by organising multiple pools. However when you get to 13 drives and around 22TB of data creating new pools seems like a hassle. 
 
First thought is that this would eliminate the need.
Second thought is that once implemented the folder placement would to my mind then simplify the operation of creating separate pools and may actually lead me to do it, instead of just thinking about it 
 
I'm all for it!!!
 
.

This is something that I have hoped for, for a very long time. I really have never been keen on having things scattered around the pool. Especially with music... 10 album tracks scattered over 5 or 6 drives.. But then I'm probably a bit O.C.D.
 
I'm sure for people who have various devices streaming to different rooms this must be a good thing. Knowing that all the Disney films are all on one hard drive for the rug rats, and the teenage daughter can watch her twilight knowing its on a separate drive so no risk of intensive I/O and so on. And yes i know that this could be achieved by organising multiple pools. However when you get to 13 drives and around 22TB of data creating new pools seems like a hassle. 
 
First thought is that this would eliminate the need.
Second thought is that once implemented the folder placement would to my mind then simplify the operation of creating separate pools and may actually lead me to do it, instead of just thinking about it 
 
I'm all for it!!!
 
.