Christopher (Drashna)

btw i wanna thank u christopher for the xeon 1246v3 opinion for my server ive had no major issues at all and its still running flawless 24/7....

it has 32 gb gskill trident (4 x8gb sticks)

Virtual Volumes View (http://vvvapp.sourceforge.net/) is what i use to achieve this.
It's not perfect but if you have all drives mounted to a Pool-Folder like Chris does,
you can simply index that folder and the output might be a bit more navigatable than simple textfiles.

Honestly, if you have 2 SSD drives.... what I'd recommend doing is setting one as the system drive (not RAIDed), and use the other for the Steam library. That way, you're splitting the IO between the systems.  It may not be strictly necessary due to the SSD's speed and IOPS (I/O Operations Per Second).   But it it may, depending on the load, and at the very least, if you have to wipe the system, your steam library is intact. 
 
 
Additionally, you don't have to worry about degraded performance due to a lack of properly TRIM support. 
 
And you could absolutely pool the secondary disk for the steam library. It's not a bad idea, especially if you have a large library. 

Well that is "easy" so far, similar to the other thread on how to migrate all I did was:
- Install DrivePool and the Plugins
- Created a DrivePool using the Same Disks as DriveBender
- Added a drive Letter to each HDD
- Stopped the DriveBender and Stablebit DrivePool Services in services.msc
- On each HDD, moved the folders with content from the DriveBender to the DrivePool folders (eg not Recycle Bin, Vol Info etc), this only took a few minutes as it is a move not a copy process
- Rebooted
- Removed Drive Letters for the physical HDD
- Changed the Drive Letter and Name for the "Old / Now Empty" Drive Bender pool
- Changed the Drive Letter and Name for the "New / Full" Stablebit DrivePool to what I was using before
- Rebooted
 
...and not only is all my "stuff" where it should be, but the Windows Permissions, and UNC Shares were automatically re-established.
 
To test:
- Ran a FreeFileSync compare over this new Pool vs my Backup Pool (Drive Bender) and all looks good (I got one sub folder with a permission issue but it may have been as I was doing the compare while also changing other things.  2nd compare was fine).
- I use JRiver Media Center and did a missing File check (both on a Local and a Networked PC) = All good
 
So far Pretty Impressed.  Lets see how it goes over a week or so (I've Left DriveBender installed and working, just with no content in the folders). 
 
Thanks
Nathan

Well, thank you for your interest!
 
And that's certainly a lot of questions.
 
- Is ReFS supported on both Win10 and 2012R2
ReFS is only supported on Server OS's officially (this is as per Microsoft, and requires a hack to support in client OS's, aka Windows 10). That said, we do have preliminary support for ReFS, and it appears to work fine. We need to do more testing to ensure complete compatibility with it. 
 
- Are the 8TB Seagate's OK to use
As for the 8TB Seagate Archive drives (I'm assuming that's what you're referring to... as there are absolutely no issue with the enterprise drives), see this thread:
http://community.covecube.com/index.php?/topic/1625-do-i-start-buying-8tb-archive-drives-or-not/
The TL;DR of it is, yes, they're fine to use (I have 5, with a 6th on the way), but write performance isn't great and you may want to use a dedicated "cache" drive and the SSD Optimizer (doesn't have to be a SSD)
 
 
- Any issues with the mix of SATA/USB Controllers I've listed
Absolutely none. The only issue is that USB isn't all that reliable, and I personally recommend against using it for long term storage. eSATA is better. 
That said, as for the cheap SATA cards... it depends on the chipset. SIL chipset based cards are horrible and shouldn't be used. 
 
 
- OK to continue to use Hard Disk Sentinel for SMART monitoring
Absolutely okay. However, we don't include integration with HDS, only for StableBit Scanner. But you can run both concurrently, without any issues.
 
- I prefer Cascade over Balance for spreading the content across the drives (eg fill up one drive, move to the next) That's absolutely fine. You will need to download and install the "Ordered File Placement" balancer to do this though. https://stablebit.com/DrivePool/Plugins   Once you've installed and enabled it, you should be fine.   - OK to use "Lights Out" on my 2012R2 to keep it asleep most of the time Absolutely. And even better, we support the "running file" option for LightsOut, so that it creates a file when running tasks, so that LightsOut doesn't put the system to sleep.    http://wiki.covecube.com/StableBit_Scanner_Advanced_Settings Scanner_RunningFile  (this is done in the UI, so check the "Advanced Settings" section for details)   http://wiki.covecube.com/StableBit_DrivePool_2.x_Advanced_Settings DrivePool_RunningFile    If you have any issues with this, use a 3rd party text editor such as Notepad++, or let me know (the text config file for DrivePool can be finicky)   - How fast is the Pool (eg I have teamed Dual Gigabit  - 2gbps) it's at least as fast as the underlying disks.  Since it's a file based solution, we can't do the same level of performance optimization that block based solutions (like RAID).  That said, we do have a number of performance optimizations that may help.  Including "Read Striping" (complicated), and Network IO Boost (prioritizes network traffic over local disk access, at the expense of CPU). And here is a link talking about the features: http://stablebit.com/Support/DrivePool/2.X/Manual?Section=Performance%20Options     - Any specific notes or issues serving media files (like with windows storage spaces and DVD stuctures) Don't store the Plex database on the pool (it uses hardlinks, which are not supported). Aside from that, none that I'm aware of.    - Any specific notes or issues with assigning permissions using Homegroup or UNC Shares Nope. In regards to permissions, it's just like a normal disk. No special considerations (aside form the Network IO Boost, for prioritization).   Though, I recommend using normal shares (using "fsmgmt.msc" to create and manage the shares) over homegroups.          Additionally, you may want to check out this thread:  http://community.covecube.com/index.php?/topic/511-migrate-from-drive-bender-to-drive-pool/   It's a user created thread, but it outlines how to migrate from Drive Bender to StableBit DrivePool without having to spend hours and hours on moving files around

No, we haven't had a chance to really look into it.  
Alex is the one that does this (he's the developer, and i'm the technical support and customer service guy). 
 
Though, I'll bring it up to alex.

@4Frame,
 
Yes, that is absolutely correct. If you check the notes link, the 3rd bullet point states this. Specifically:

Well, thanks!
 
 
And that is definitely understandable. The balancing system has significantly grown and improved with time, and does a lot more that it was initially designed to. 
 
 
 
As for your settings, those do look good.
The file placement settings (at the bottom of the general page) are set to what they need to be to ensure they work with the file placement rules (check the notes for the balancer for details):
http://dl.covecube.com/DrivePoolBalancingPlugins/OrderedFilePlacement/Notes.txt
 
 
However, I would recommend setting the ratio slider higher, and maybe enable the checkbox under it. This may help make the balancing a bit more aggressive.  That may be a good thing for your setup. 
 
This basically deals with the first question. The Ratio determines when the balancing is triggered. As files get "out of place", it lowers the ratio. Once it hits the threshold, it will trigger a balancing pass. The higher the threshold is set, the more aggressively the system will balance.
Additionally, enabling the "or this amount to be moved" option is a good idea if you have a very large pool, as a 100GBs may be too small to trigger the balancing aggressively, as your pool grows larger.
 
 
 
As for Question 2:
Yes, that's precisely why. The default placement strategy is to place files on the disk with the most (absolutely) free space. If there are multiple or no suitable drives, this is what the software defaults to. 
 
 
Question 3:
This is controlled by the "Real time placement limiters", as set up by the Ordered File Placement (or other) balancers. 
In theory, the file placement rules should trump this, if you have the "file placement rules respect ...." option disabled (which you do).  If it's not sending the files directly to that drive, that may be an issue.  
 
 
Question 4:
Not really. This is normal. Unfortunately, there isn't a good way to really handle this. 
Partially, because we store the duplication status of the pool in the folder structure (as super hidden file system objects, called alternate data streams), so we need at least part of the folder structure to be on multiple disks if duplication is enabled.
However, it really shouldn't cause any sort of harm. 

If it's called the "File Placement Limiter", then you're using an older version of StableBit DrivePool, and you may want to update.  We changed the name for it a while ago, to avoid confusion with the "File Placement Rules" feature we added.
 
If you're using the 1.3 version, then you're fine.
 
 
 
Either way, by unchecking the "duplicated" option for all of the other drives, you're telling the system that the 8TB drive is the ONLY valid target for duplication. Which... well, defeats the point. 
We don't differentiate between "original" and "copy" in the software, so if it's showing "duplication", that means that the files have duplication enabled for them, and reside on more than one disk. Not that it's the copy.
 
If you want to ensure that a copy resides on this disk, then you're setup is correct. However, if you don't care where the files end up, but you only want duplicated data on the 8TB drive, then what you want to do is leave all of the other drives alone, and just uncheck "unduplicated" data on the 8TB drive. This will make sure that no unduplicated data ends up on this disk.
 
 
As for the write penalties, I'm assuming you're using a Seagate Archive drive, with SMR. 
if so, you should really check out this thread:
http://community.covecube.com/index.php?/topic/1625-do-i-start-buying-8tb-archive-drives-or-not/
 
 
 
As for corruption, this is a touchy subject for a lot of people.
I'm guessing you're referring to "random bit flips" here (commonly called bit rot, incorrectly). If so...  these are statistically unlikely to happen that ... well, it's literally a white whale... and your drive is designed to detect and correct them at the physical and firmware level (invisibility to the OS).  The chances that you'll experience this sort of issue is pretty much nil.
 
If you mean media degradation due to physical defects and age... then that is something different. And StableBit DrivePool does do a number of checks for that.  Specifically, we check the file modify time when accessing the data. If it doesn't match, then we grab a checksum of the files. if that matches, we update the time. Otherwise, we flag the file and notify the user that there is a file part mismatch that needs to be resolved.
Additionally, changing the duplication status, or remeasuring the pool will trigger this for all files in the pool. 
 
Also, I apologize for the shameless self promotion here, but that's exactly what StableBit Scanner SPECIFICALLY addresses.  By default, StableBit Scanner performs a surface scan of all attached disks.  This is a sector by sector scan of the drive, where it attempts to read the data on the drive.  Any unreadable sectors are flagged in the system.  StableBit Scanner will then prompt you to run a file scan to see if we can identify any affected files (sometimes, these can end up in free space but we don't know until we check).  Additionally, if both products are installed on the same system, StableBit DrivePool will evacuate the contents of ANY disk with unreadable sectors as marked by StableBit Scanner. This is in attempt to prevent any potential corruption of more data (as this sort of issue tends to only get worse).

Int the newest beta builds, we include command line auditing tools, so you could set up a script to dump the information on a schedule (using Windows' Task Scheduler).
 
It requires 2.2.0.659 or higher, though.
http://dl.covecube.com/DrivePoolWindows/beta/download/
 
And details about the new commands here:
http://community.covecube.com/index.php?/topic/1587-check-pool-fileparts/
 
Specifically, the "4" detail level includes EVERYTHING, including the location of the files. 
 
 
 
This will actually work, if you do it for each pooled disk, and make sure that you can enumerate the hidden "PoolPart" folder on the drive. 
 
But you'd need to do this for each and every drive.

First, I'm sorry to hear about the drive failing, as that is never a pleasant experience. 
 
 
As for the email, if you removed the missing/failed disk, it may have completed, or it may have corrupted the settings.
 
The newer versions (beta version) handles issues like this better and may fix the issue outright.
 
Otherwise, resetting the settings should fix this issue (but will wipe out your balancing settings).
http://wiki.covecube.com/StableBit_DrivePool_Q2299585B

Unfortunately, we don't really have a "merge pools" option right now. It's something that has been requested, and we may add in the future, though.
 
However, for now, the quickest way to merge the pools?
 
 
Please read, and reread this section. If you're not 100% certain about the process, we can set up a remote support session to assist you with this.  If you would prefer that we help, then please contact me at "https://stablebit.com/Contact",reference the thread and let us know that you'd like to set up a remote support session. 
 
That said, here's how to do it, basically. 
 
Move the disks over.This will create two pools... Stop the StableBit DrivePool Service. (run "net stop drivepoolservice") Enable "Show Hidden Files" for the system. "Unseed" one of the pools (do this with the pool with the least number of disks). By this, I mean open the "PoolPart.GUID" folder (where "GUID" is a random sequence of letters and numbers).  You should see a folder and file layout that mirrors the pool (or part of it at least). Move the contents of the Poolpart.GUID folder out of this folder.This moves the contents out of the pool effectively, as the software uses these folders as the actual contents Once the PoolPart folder is empty, delete the folder. Repeat this for each of disks in the one pool Start the "StableBit DrivePool Serivce".  (run "net start drivepoolservice")It should cause the pool in question to "disappear", as there are no disks in that pool anymore. Add the disks to the existing pool. Stop the StableBit DrivePool Service again.  Put the contents of the disks into the newly created "PoolPart.GUID" folders Start the StableBit DrivePool Service again. Remeasure the pool (Pool options -> remeasure).

Correct me if i am wrong but if your first goal ist to make the data available again, you should be able to just put the 3 drives in server 2 and a second pool would be recognized.
This way, you wouldn't have to copy data over the network.

I would say yes. I can not find a "normal" 6TB HDD that is cheaper than these 8TB HDDs and they read like crazy (for spinners, certainly at 5.9Krpm). And if you write about 1 movie a day (assuming they are less than say 25GB) then you won't even suffer a write-penalty). Or per hour. Or per ten minutes I would think. And if you have a number of them in a Pool and don't use file-placement rules it gets even better still I would speculate.

I just did the same thing (completely different hardware; but, the same steps).
 
Yes.  It will find and have the actual full Drivepool all setup and active.  As long as the drives are "Online": using the built-in Disk Management application, you can check that they are online.
 
I recommend these steps though: Get the hardware setup first before installing the software.
 
- Deactive license, shut down old machine.
- Install HDDs in new machine, do NOT install DrivePool!!!  Hold off installing any software.
- Go to Disk Management and verify the drives are all "Online."  If they are not, right click on them and select "Online."
- Then install DrivePool, DriveScanner, etc.
 
These are the steps I took and it worked perfectly.
 
The reason why I suggestion this is you can only bring a single HDD "online" at a time.  Even though I didn't try it; I would not want DrivePool complaining that I only have 1 HDD out of my pool.  It may go wacky or whatever.
 
So, I decided to get the hardware setup first before installing the software.
 
Note: DrivePool will try go through a bit "Checking" phase.  After it is done, it will start to Balance with the DEFAULT settings.  If you are like me, you may have very custom settings setup in your Balancing.  Get that done first and don't let it start balancing.  The initial checking that happens after installing DrivePool took several hours; so, it gave me plenty of time to get things "Balanced" correctly according to my settings.
 
Scanner will run with defaults.  Mine was set to 3 AM - 6 Am for the work period (I was up until 4 AM working on mine, and wondered why throughput lowered dramatically - it was Scanner that started at 3 AM).
 
As far as virtual machines and DrivePool/Scanner, be warned that SMART access does not pass through Hyper-V to the VMs.  I originally tried to get DrivePool/Scanner all setup on VMs under QEMU (a good native hypervisor under Linux) and W2K12R2 using Hyper-V.  But times I ran into SMART data not being passed, which ticked me off.  It's a Windows/Linux thing.
 
So, I advise setting up the DrivePool / Scanner on the HOST.  If that's Windows 10, that's fine.  That will give you a single Drive Pool drive.  I name mine "A:\", as a throwback to the floppy disk days (if you don't know Floppy Disks, then enjoy your youth!).
 
Once you have that setup, I advise setting up a "Virtual Network Adapter" in Hyper-V for "Internal" only <- if that's possible under Windows 10 (I am using Windows Server 2012 R2, which allows me to do "Internal" network adapters).  Internal creates a private 10 Gbps network that all VMs can use, as well as talking to the host - you ahve to setup the proper IPs and Mask.  The advantage is there is no physical "NIC' used, so you get some great speed without interfering with remote access to the physical machine.
 
With that, setup a "Share" only for your internal private network (mine is 10.16.210.X).  The share will be for your Drivepool drive, open to all anonymous access... And there inlies yet another issue with Windows, true "anonymous" access to a shares is very difficult with so many hoops to jump through at many different security levels.  
 
Anyways, use a Windows Share from your Windows 10 box to your "internal" networking for all of your VMs you have running.

I've been testing CloudDrive for this purpose, and it seems to work well. Created a local-source cloud drive, then it can be added to a pool in DrivePool. Not sure about performance vs BitLocker, as I only use it to back up documents and photos.

solved.
by resetting the settings in the troubleshoot menu. With thnx to Christopher for the solution
Thnx

Usually I just leave DrivePool to do its thing, but tonight I checked in with its control panel on my WHS2011 home server for the first time in ages, and was confused (and somewhat alarmed) to see 101GB of unduplicated files. I use file duplication, with a fairly anal set of rules to force certain folders onto certain pairs of disks (Docs & Pics are duplicated always on disks 1 & 2; Music & Videos always on disks 3 & 4). This is so I can mix HDD brands for added protection.
 
I was utterly baffled by what seemed like an 'undupe-able' 101GB of files, and tried rebalancing, remeasuring, checking the rules etc. I logged in here to ask a question & post some screen shots.
 
Fortunately before I had time to make a prat of myself I noticed something I'd missed in the 'file placement' tab: I'd been so busy checking & rechecking my folder rules, that I'd totally overlooked the 'Recycle Bin' line. My Recycle Bin isn't duplicated in the same way that my 'real' folders are (why bother?).
 
Checked the Bin's properties - it contained exactly 101GB of stuff I'd thrown away. Emptied the Bin, and all my DrivePool stats reverted to normal (equal amount of files on the appropriate disks, everything duped). The moral of this story: if you have inexplicable duping stats weirdness, check the Bin, and empty it. I'm posting this as confession is good for the soul; as potential help for others; but mainly so that when I've forgotten it in a year's time and I have the same problem again, I might find the answer on this forum from my past self...

Specifically, scales better than Amazon's able to accommodate for (because they're not throttling on the server side properly).  

Well, as it took 2 weeks and several emails originally ..... the answer is "I don't know". Sorry.
 
We've had much better communication with Amazon lately (eg, they're actually replying).
 
In fact, right now, the emails are about the exact API usage and clearing up some functionality, between Alex and Amazon.  So, things are looking good.   
 
 If you check the first post, Alex has updated it with the latest responses.

Well, as it took 2 weeks and several emails originally ..... the answer is "I don't know". Sorry.
 
We've had much better communication with Amazon lately (eg, they're actually replying).
 
In fact, right now, the emails are about the exact API usage and clearing up some functionality, between Alex and Amazon.  So, things are looking good.   
 
 If you check the first post, Alex has updated it with the latest responses.

About to add my next 4tb Seagate NAS drive to the server probably tomorrow. Got in on a good deal from NewEgg on Cyber Monday for $119. Should arrive tomorrow. Good timing to because I've only got like 2GB space left!
 
So I'm looking at unmanaged switches for my next big buy. I've done a bit of deep thinking and I think I've narrowed it down to two possibilities.
 
http://www.newegg.com/Product/Productcompare.aspx?Submit=ENE&N=100158106%20600052100%20600052097&IsNodeId=1&Description=unmanaged%20gigabit%20switch&bop=And&CompareItemList=30%7C33-122-058%5E33-122-058%2C33-150-156%5E33-150-156&percm=33-122-058%3A%24%24%24%24%24%24%24%3B33-150-156%3A%24%24%24%24%24%24%24
 
Hopefully that "compare" link will work. Essentially they're the same thing. 16 ports, Gigabit, unmanaged, lifetime warranties, same cost (with the NetGear's discount). The Cisco has some kind of mini port that I have no use for at this time being but might be useful in the future, and QoS which might be useful in a home server environment for streaming videos. The NetGear brand I'm more familiar with, matches the brand of my router, and has over a hundred positive reviews versus Cisco's less than 10.
 
So what do you guys think?

I will have a go at it this weekend and come back with the results

I upgraded to 422 this morning and it seems to be working. Thanks.

#5 I didn't mean to use them at the same time - I would try each of them, one at a time to see which one is most stable .