Christopher (Drashna)

Small update : I did replace the original heatsink with a bigger and 100% copper one.
I did fixed it with Arctic Sylver Thermal Adhesive, so it's permanent but much more efficient than thermal paste in this case.
Then I did bended the heatsink pins one by one.
The result is an impressive 20°C temp drop from 45-50°C to 27-30°C.
 
I plan to do the same for the top M1115.
 
 

 
 
 
 
 
 

 
 
 
 

 
 
 
 

I'm a long time FlexRAID user, and while the protection appeared to be there the reality is that it never really worked for me.
 
I will have a pool with 14 drives (10x 3TB, 2x 4TB and 2x 5TB drives).  The majority of my data is media (about 18TB) and not practical to duplicate it all.  I have another 2-3TB of personal files, work files, pics etc that I would like to duplicate.
I will also be running the Scanner software.  I understand that if I want to be fully protected I'd need to spend a ton of money on Enterprise drives, hardware RAID, etc, etc, etc.  I'm looking for settings/configuration (for both Drivepool and Scanner) that will offer me good protection/prevention.
 
I hooked up some old drives to my existing FlexRAID server and installed Drivepool and Scanner to play with it.  I need to play with Drivepool a bit more to understand how it works... I had one drive with data on it, and it added the free space to the pool, and there was a drive with just the data on it that remained.  I moved the data, but it didn't remove the now empty drive.  It scanned the drives, and one was a complete mess (but still worked), so it warned me it was going to die.  Another has always had heat issues, and it gave me a warning about the temp.  That's great, and as long as I have 5TB free it should copy files off the drive to minimize the risk of data loss.
 
Any suggestions for actual setup/configuration?  Tips for a newb?
 
Thanks!
 
 

As I read it, you have 48TB capacity and 21TB of data. I would duplicate all as I assume you do not make backups and do want to be at least somewhat protected. Duplication <> backup but at least you are likely not to lose anything in case of a single HDD failure. It will leave about 6TB free (although, due to duplication it would only be about 3TB net).
 
I would consider adding one or two (or four) 8TB Seagate Archive HDDs. They are cheap per TB and read like crazy. There is a write-performance penalty but, frankly, I do not see that affecting your system as it would hit only when you have a lot of writes and I assume most of your data is actually static. It will leave you with a lot of room to move data to from current HDDs that start to fail.
 
The is a File Placement add-in that helps you to ensure that media files that belong to each other or not scattered over many HDDs (you could get it to have groups of files to be written to two HDDs only) but I do not use it and can not actually help with that.
 
I am usnure what you mean exactly by "I had one drive with data on it, and it added the free space to the pool, and there was a drive with just the data on it that remained.  I moved the data, but it didn't remove the now empty drive."

So, basically no difference.  Do you have multiple SSDs?  If so, ... well, to properly test... you'd need 7+ that can run at about 500MB/s (for the sequential).  6 and change to saturate one connection, and then a seventh or more to exceed.  If you have that many and can test them all at the same time (running a surface scan or chkdsk or a full format), that would be the real test.  (otherwise, it's closer to 30+ HDDS).

You can find all builds here:
http://dl.covecube.com/CloudDriveWindows/beta/download/
 
However, these should be relatively stable:
http://dl.covecube.com/CloudDriveWindows/beta/download/StableBit.CloudDrive_1.0.0.395_x64_BETA.exe
http://dl.covecube.com/CloudDriveWindows/beta/download/StableBit.CloudDrive_1.0.0.395_x86_BETA.exe

my 2 WD red 6 TB and my 1TB 2.5 inch have been running strong 24/7 since I rebuilt my xeon

The cache size is the target size that should be used. 
 
The upload cache may exceed this, and fill the drive.
 
We do plan on putting a max cap on this in future versions. 

The benefit to me with server OS is the stability and automation.  It's very easy to setup for automatic computer backups, and the use of storage space is a breeze.  The only reason I didn't stay with WHS 2011 was because after finally getting everything installed, the OS started corrupting my system files and registry.  Not sure what caused it, probably a Windows Update or something, but I was essentially locked out of modifying anything.  Whenever I would try to access the registry, it would give me an error saying the registry was in read-only mode because I was connected remotely.  Of course the first thing I then tried was connect a monitor/keyboard/mouse directly to the server, and it STILL gave me that error.  I couldn't perform any administrative jobs either, like system restore, to try and fix the problem.  The only solution was a complete reformat and reinstall, which I was unable to do because of the current condition of my drive/folder structures.  This was before I got DrivePool.  If I formatted my OS drive, I would lose what Microsoft called "Tombstones" that stores the location of the files.  Lose those files, and you're basically SOL on getting your files off any of the pool drives.  My ultimate solution was to go for Windows Server 2012 R2 Essentials, which according to Wikipedia (link) was the next upgrade after WHS 2011.  I've had to reinstall it twice because of my misunderstanding of how the system works, but now that it's running it's great.
 
Something else you might consider, though I have no experience with, is something like FreeNAS.  From what I understand, it'll do what you're looking for.  The OS can be installed on a tiny little USB drive, and then all your drives are pooled.  You access the interface for configuring through a built in website, and then access the storage pool through a standard Explorer linked network share.  The downside I think is that there is no built in backup solutions.  The only reason I didn't go this route is because I'm not good with Linux.  But my new understanding is that you don't even SEE Linux.  The OS installs and sets everything up automatically.  The only downside you might experience would be drivers for your controller (which is also the problem you CURRENTLY have).  If you're still in the experimentation phase of locking down an OS, I'd at least recommend giving it a try.  I for one would love to know how you get on with it if you do, since it sounds a LOT easier to set and have running.

yes, please! We bought a licence from you immediately after release and are waiting for Google Drive (for work) support since then.
(and for Amazon Cloud Drive Unlimited arriving in Europe - but as it's not that reliable that's maybe not the best solution).
 
In fact Google gives you >5TB even with one user     ... pssst. 

Hi you could just buy yourself some rackmount bars off eBay go to your local hardware shop and get them to cut some 22mm mdf to your custome sizes screw the bars on there you go I built mine like that cost me about £30 I needed a case at a specific size for where I wanted it stored.

I have installed W7 64 Pro and have not had a single crash! I looked in the event viewer and I am still getting some errors relating to the 3ware driver BUT it has not crashed and it has been 3 days now. Does this mean that I could maybe build my server around W7 and not have any problems? What do you think?

Bissquitt,
It's a custom built rack out of 3/4 plywood and audio equipment Raxxess rails. If you want the specs I could look for them but it was made to house my APC UPS being the deepest and enough room below for cables and a less deep top shelf to help with airflow. Additionally I mounted caster wheels to roll it in and out of my office closet.

I had the same issue with BT Sync.  That's why I love Stablebit...can duplicate across vdisks.

I had issues when I last used BitTorrent Sync (When it was in beta) and ended up losing some files and was also looking for a way to use a friend as backup. We settled on AllwaySync, which works very well and does not impact our files at all. It does cost a little, but it is well worth it in my opinion.

okay well thats grate news ill see if i can buy it next month.

Please, definitely do check, and let me know what you find.
 
Also, a good way to max out the bus speed... use StableBit Scanner's Burst test option (right click on a drive).  This runs the drive a higher speed than the disk is capable, but maxes out the bus.  It's great for testing for connectivity issues, but doing this on multiple drives may be a good way to test the controller link.
 
 
And Expander Cards make for excellent cable management. 

http://www.lsi.com/products/storagecomponents/Pages/LSISAS9201-16i.aspx
 
Driver version: 2.0.60.2
Firmware version: 16.00.00.00
BIOS Version: 08.00.00.00_68.51.34.17_09000000
 
Using specific method: ScsiPassThrough48

 
Using specific method: AtaPassThrough

Thanks a lot : it instantly worked 
For now I am still testing SSD pluggin on unduplicated files. I am waiting for a third 240gb SSD to arrived (Sandisk PRO, not the slowest SSD  ...) to start testing with x3 duplication.
From what I can see for now, it would work very well!

The controller should handling 64 drives, or it was 256. It depends on the controller firmware, really. And I'm not sure.
http://www.avagotech.com/products/server-storage/raid-controllers/megaraid-sas-9240-8i#specifications
http://www.avagotech.com/products/server-storage/host-bus-adapters/sas-9211-8i#specifications
 
And you can chain Expander cards, IIRC. 
 
The 34-36 limit is for the number of drives you can scan at the same time, assuming 120MB/s throughput. This is based on the PCI Express 2.0 8 lane (8x) connection.  This would be an issue for RAID arrays, but because DrivePool isn't going to be using every drive at the same time, it shouldn't be an issue for your Pool.
 
 
 
And I'm very sorry to hear that you're feeling sick. I hope you feel well soon!

This depends on the balancing settings. 
 
Try setting the balancing to occur immediately, and disable the "but not more often than" option. Also, set the ratio slider on the main tab to "100 %" and setting the "or needs to move this much" to "1GB". 
 
This should help it be more aggressive about moving data out. 
And this should definitely work, as that's exactly what I'm doing on my system, and it's constantly moving data off of the 2x128GB SSDs I have.

lol, I definitely understand that!
 
 
As for the auction, that's not a bad price at all. As you've said, it's much more expensive for a new card (and more so with cables, at around $350-400, in fact).  So, it's not a bad purchase at all!
But I definitely understand what you mean.  Just keep in mind that this is enterprise grade tech, and is designed for heavy loads and 24/7 operation. That's why there is such a premium on the price.

Please do!  
 
The 153 event is a "retried" error.  This may be harmless (it's rather normal, actually), and if you're only seeing it during boot, it may just be a timing issue with the drive controller.
 
Check the disk number it mentions with Disk Management (run "diskmgmt.msc") and check which drive it is.  

Yep. Sounds right. I will offer at $90.
 
Since I have a very cramped build : 9 HDs + 4 SSDs inside a litle mid tower, the expander powered with a Molex would offer better cable management than an other M1115.

The topic of adding RAID-style parity to DrivePool was raised several times on the old forum. I've posted this FAQ because (1) this is a new forum and (2) a new user asked about adding folder-level parity, which - to mangle a phrase - is the same fish but a different kettle.   Since folks have varying levels of familiarity with parity I'm going to divide this post into three sections: (1) how parity works and the difference between parity and duplication, (2) the difference between drive-level and folder-level parity, (3) the TLDR conclusion for parity in DrivePool. I intend to update the post if anything changes or needs clarification (or if someone points out any mistakes I've made).   Disclaimer: I do not work for Covecube/Stablebit. These are my own comments. You don't know me from Jack. No warranty, express or implied, in this or any other universe.   Part 1: how parity works and the difference between parity and duplication   Duplication is fast. Every file gets simultaneously written to multiple disks, so as long as all of those disks don't die the file is still there, and by splitting reads amongst the copies you can load files faster. But to fully protect against a given number of disks dying, you need that many times number of disks. That doesn't just add up fast, it multiplies fast.   Parity relies on the ability to generate one or more "blocks" of a series of reversible checksums equal to the size of the largest protected "block" of content. If you want to protect three disks, each parity block requires its own disk as big as the biggest of those three disks. For every N parity blocks you have, any N data blocks can be recovered if they are corrupted or destroyed. Have twelve data disks and want to be protected against any two of them dying simultaneously? You'll only need two parity disks.   Sounds great, right? Right. But there are tradeoffs.   Whenever the content of any data block is altered, the corresponding checksums must be recalculated within the parity block, and if the content of any data block is corrupted or lost, the corresponding checksums must be combined with the remaining data blocks to rebuild the data. While duplication requires more disks, parity requires more time.   In a xN duplication system, you xN your disks, for each file it simultaneously writes the same data to N disks, but so long as p<=N disks die, where 'p' depends on which disks died, you replace the bad disk(s) and keep going - all of your data is immediately available. The drawback is the geometric increase in required disks and the risk of the wrong N disks dying simultaneously (e.g. if you have x2 duplication, if two disks die simultaneously and one happens to be a disk that was storing duplicates of the first disk's files, those are gone for good).   In a +N parity system, you add +N disks, for each file it writes the data to one disk and calculates the parity checksums which it then writes to N disks, but if any N disks die, you replace the bad disk(s) and wait while the computer recalculates and rebuilds the lost data - some of your data might still be available, but no data can be changed until it's finished (because parity needs to use the data on the good disks for its calculations).    (sidenote: "snapshot"-style parity systems attempt to reduce the time cost by risking a reduction in the amount of recoverable data; the more dynamic your content, the more you risk not being able to recover)   Part 2: the difference between drive-level and folder-level parity   Drive-level parity, aside from the math and the effort of writing the software, can be straightforward enough for the end user: you dedicate N drives to parity that are as big as the biggest drive in your data array. If this sounds good to you, some folks (e.g. fellow forum moderator Saitoh183) use DrivePool and the FlexRAID parity module together for this sort of thing. It apparently works very well.   (I'll note here that drive-level parity has two major implementation methods: striped and dedicated. In the dedicated method described above, parity and content are on separate disks, with the advantages of simplicity and readability and the disadvantage of increased wear on the parity disks and the risk that entails. In the striped method, each disk in the array contains both data and parity blocks; this spreads the wear evenly across all disks but makes the disks unreadable on other systems that don't have compatible parity software installed. There are ways to hybridise the two, but it's even more time and effort).   Folder-level parity is... more complicated. Your parity block has to be as big as the biggest folder in your data array. Move a file into that folder, and your folder is now bigger than your parity block - oops. This is a solvable problem, but 'solvable' does not mean 'easy', sort of how building a skyscraper is not the same as building a two-storey home. For what it's worth, FlexRAID's parity module is (at the time of my writing this post) $39.95 and that's drive-level parity.   Conclusion: the TLDR for parity in DrivePool  
As I see it, DrivePool's "architectural imperative" is "elegant, friendly, reliable". This means not saddling the end-user with technical details or vast arrays of options. You pop in disks, tell the pool, done; a disk dies, swap it for a new one, tell the pool, done; a dead disk doesn't bring everything to a halt and size doesn't matter, done.   My impression (again, I don't speak for Covecube/Stablebit) is that parity appears to be in the category of "it would be nice to have for some users but practically it'd be a whole new subsystem, so unless a rabbit gets pulled out of a hat we're not going to see it any time soon and it might end up as a separate product even then (so that folks who just want pooling don't have to pay twice+ as much)".

Thank you guys from grabbing the info. 
 
I've submitted the issue and info to Alex (the developer), and he'll take a look at it soon (I suspect it maybe a simple issue).
https://stablebit.com/Admin/IssueAnalysis/20806
 
However, please do install the beta version (2.5.2.3103). There are a few fixes in it in regards to how SMART data is handled, and it may fix the issues you are seeing (specifically). 
 
 
 
Yeah, sorry to not specify, but the ID is for all the devices in the system. It grabs the SMART data and other information about the disks and submits them to BitFlock.