Christopher (Drashna)

Thank you for the sentiment.
 
What I was thinking at the time, and I mentioned this to Christopher... At the heart of the problem is that our product scales... it scales really well.

If you're not familiar with dpcmd.exe, it's the command line interface to StableBit DrivePool's low level file system and was originally designed for troubleshooting the pool. It's a standalone EXE that's included with every installation of StableBit DrivePool 2.X and is available from the command line.
 
If you have StableBit DrivePool 2.X installed, go ahead and open up the Command Prompt with administrative access (hold Ctrl + Shift from the Start menu), and type in dpcmd to get some usage information.
 
Previously, I didn't recommend that people mess with this command because it wasn't really meant for public consumption. But the latest internal build of StableBit DrivePool, 2.2.0.659, includes a completely rewritten dpcmd.exe which now has some more useful functions for more advanced users of StableBit DrivePool, and I'd like to talk about some of these here.
 
Let's start with the new check-pool-fileparts command.
 
This command can be used to:
Check the duplication consistency of every file on the pool and show you any inconsistencies. Report any inconsistencies found to StableBit DrivePool for corrective actions. Generate detailed audit logs, including the exact locations where each file part is stored of each file on the pool. Now let's see how this all works. The new dpcmd.exe includes detailed usage notes and examples for some of the more complicated commands like this one.
 
To get help on this command type: dpcmd check-pool-fileparts
 
Here's what you will get:
 
dpcmd - StableBit DrivePool command line interface Version 2.2.0.659 The command 'check-pool-fileparts' requires at least 1 parameters. Usage: dpcmd check-pool-fileparts [parameter1 [parameter2 ...]] Command: check-pool-fileparts - Checks the file parts stored on the pool for consistency. Parameters: poolPath - A path to a directory or a file on the pool. detailLevel - Detail level to output (0 to 4). (optional) isRecursive - Is this a recursive listing? (TRUE / false) (optional) Detail levels: 0 - Summary 1 - Also show directory duplication status 2 - Also show inconsistent file duplication details, if any (default) 3 - Also show all file duplication details 4 - Also show all file part details Examples: - Perform a duplication check over the entire pool, show any inconsistencies, and inform StableBit DrivePool >dpcmd check-pool-fileparts P:\ - Perform a full duplication check and output all file details to a log file >dpcmd check-pool-fileparts P:\ 3 > Check-Pool-FileParts.log - Perform a full duplication check and just show a summary >dpcmd check-pool-fileparts P:\ 0 - Perform a check on a specific directory and its sub-directories >dpcmd check-pool-fileparts P:\MyFolder - Perform a check on a specific directory and NOT its sub-directories >dpcmd check-pool-fileparts "P:\MyFolder\Specific Folder To Check" 2 false - Perform a check on one specific file >dpcmd check-pool-fileparts "P:\MyFolder\File To Check.exe" The above help text includes some concrete examples on how to use this commands for various scenarios. To perform a basic check of an entire pool and get a summary back, you would simply type:
dpcmd check-pool-fileparts P:\
 
This will scan your entire pool and make sure that the correct number of file parts exist for each file. At the end of the scan you will get a summary:
Scanning...   ! Error: Can't get duplication information for '\\?\p:\System Volume Information\storageconfiguration.xml'. Access is denied   Summary:   Directories: 3,758   Files: 47,507 3.71 TB (4,077,933,565,417   File parts: 48,240 3.83 TB (4,214,331,221,046     * Inconsistent directories: 0   * Inconsistent files: 0   * Missing file parts: 0 0 B (0     ! Error reading directories: 0   ! Error reading files: 1 Any inconsistent files will be reported here, and any scan errors will be as well. For example, in this case I can't scan the System Volume Information folder because as an Administrator, I don't have the proper access to do that (LOCAL SYSTEM does).
 
Another great use for this command is actually something that has been requested often, and that is the ability to generate audit logs. People want to be absolutely sure that each file on their pool is properly duplicated, and they want to know exactly where it's stored. This is where the maximum detail level of this command comes in handy:
dpcmd check-pool-fileparts P:\ 4
 
This will show you how many copies are stored of each file on your pool, and where they're stored.
 
The output looks something like this:
Detail level: File Parts   Listing types:     + Directory   - File   -> File part   * Inconsistent duplication   ! Error   Listing format:     [{0}/{1} IM] {2}     {0} - The number of file parts that were found for this file / directory.     {1} - The expected duplication count for this file / directory.     I   - This directory is inheriting its duplication count from its parent.     M   - At least one sub-directory may have a different duplication count.     {2} - The name and size of this file / directory.   ... + [3x/2x] p:\Media -> \Device\HarddiskVolume2\PoolPart.5823dcd3-485d-47bf-8cfa-4bc09ffca40e\Media [Device 0] -> \Device\HarddiskVolume3\PoolPart.6a76681a-3600-4af1-b877-a31815b868c8\Media [Device 0] -> \Device\HarddiskVolume8\PoolPart.d1033a47-69ef-453a-9fb4-337ec00b1451\Media [Device 2] - [2x/2x] p:\Media\commandN Episode 123.mov (80.3 MB - 84,178,119 -> \Device\HarddiskVolume2\PoolPart.5823dcd3-485d-47bf-8cfa-4bc09ffca40e\Media\commandN Episode 123.mov [Device 0] -> \Device\HarddiskVolume8\PoolPart.d1033a47-69ef-453a-9fb4-337ec00b1451\Media\commandN Episode 123.mov [Device 2] - [2x/2x] p:\Media\commandN Episode 124.mov (80.3 MB - 84,178,119 -> \Device\HarddiskVolume2\PoolPart.5823dcd3-485d-47bf-8cfa-4bc09ffca40e\Media\commandN Episode 124.mov [Device 0] -> \Device\HarddiskVolume8\PoolPart.d1033a47-69ef-453a-9fb4-337ec00b1451\Media\commandN Episode 124.mov [Device 2] ... The listing format and listing types are explained at the top, and then for each folder and file on the pool, a record like the above one is generated.
 
Of course like any command output, it could always be piped into a log file like so:
dpcmd check-pool-fileparts P:\ 4 > check-pool-fileparts.log
 
I'm sure with a bit of scripting, people will be able to generate daily audit logs of their pool
 
Now this is essentially the first version of this command, so if you have an idea on how to improve it, please let us know.
 
Also, check out set-duplication-recursive. It lets you set the duplication count on multiple folders at once using a file pattern rule (or a regular expression). It's pretty cool.
 
That's all for now.

Christopher, thank you for answers.
 
Very happy to hear, that I can safely re-shuffle the drives.
 
Yep, I have my drives mounted to the folders on one of the SSDs (D:\Drives\...).
 
And thanks a lot for reminding me to use the case and bay names in the disk settings in the Scanner. That'll make this way easier. Awesome.
 
Thanks again.

Well, I got some news that will make some of you guys happy.
 
"StableBit FileSafe".
It's in the planning stage right now. What does that mean? No code for it yet. Especially as we will have to consider exactly how to integrate it into our other products. And what features it will have. Etc.
But it's definitely something we want to release.

getting corsair carbide series spec 01 and crucial MX200  250gb ssd for my gaming rig... (currently amd phenom 2 945 with 4x4 (16gb) gskill ddr 3 1600 and geforce 640gt

I know, right?!
 
But apparently, it's a completely different team that doesn't interact with the S3 team (at least, that's my guess, and what it appears like).
 
In fact, I'm not sure why they didn't just use the S3 infrastructure. ....

Doesn't matter what they are. If they won't communicate with us and let us know what the limits *should* be, then we have to guess and HOPE they won't demote us to development status again.  Getting official word is better. But they'd still have to re-promote us to production status again..... 
 
 
But to Alex's point: we're more than willing to work with them. But they need to respond to their emails.  Communication only works if it's going in both directions. 
 
That .... or we EECB them (and that's .... well, not a good option). 

I'm pretty sure the limits are quite high. I'm getting around 33 mb/s with netdrive (which is far inferior to Stablebit Cloud Drive). I think the issue must be around the max amount of threads and therefore the amount of api calls due to each thread making calls.
 
Thank god it was more people than me who showed an interest in getting Stablebit up and running which hopefully will lead to them starting to reply!
 
All in all, your product is awesome

Especially for C:\ (OS) you might want to consider performing actual backups one way or the other. DrivePool (had it supported it), dynamic disks, RAID do not protect you against deletions, failed updates/installations and whatnot. Actually, this goes for all data.

Considering the huge list of bug fixes between the Release version (2.1.1.561) and the Public Beta (2.2.0.651), this doesn't surprise me. 
 
I know there was some locking and performance fixes in there as well.
 
 
So if the beta is working much better, I'm very glad to hear that.
If you're still having issues or have any other issues, do let us know (and what OS you're using specifically) 

Final update: after using "File Placement Limiter" (Above we were calling it something different) there was a progress bar and arrows showing the activity, which was nice, and that took a while, but not too crazy long considering the amount of data. After the data was moved off the drive as requested from file placement limiter plug in, I selected the drive for removal and that took maybe 5 minutes, Once the drive was not being used, the pool displayed the available space after which I shut down the server, removed the 2tb drive and installed a new 4tb drive in it's place, after reboot, I chose to use the RAW drive without assigning a drive letter so the pool would just use the entire drive. Done deal, more space, nothing missing! Thanks for the help!

Saw the newest update includes Google Drive!!! Will do some testing with it, very excited to see this! Keep up the good work!

Fxied

Yes, after using Stablebit Cloudrive with Amazon Cloud Drive then amazon causing a bunch of issues because of "excessive" api usage I have now paid for a Google Apps for Work account for it's unlimited storage.  As soon as a beta is released that includes support for it I will be buying Stablebit Clouddrive.  Keep up the hard work.

Been almost 2 weeks now without an update.  So I thought I'd real quick announce that I have ordered my HP 24-port SAS Expander!
http://www.ebay.com/itm/New-HP-6Gbps-SAS-Expender-Card-24-Port-SAS-PCI-E-Expander-Board-468405-001-/171532956108?hash=item27f02931cc:g:aQUAAOSwdsFUPgAb
I tried to haggle a bit with the seller.  I originally bookmarked his sale when it was at $210, it's now at $160.  In fact, when I started my haggle, he had it set at $110.  I only managed to get him down to $100 even, plus $8 shipping.  Still, it beats +$200, and it's brand new.  I could have gotten it cheaper, in the $80 range, but those other sales are different models running at 3G speed instead of 6G.  Now I need to buy me some more SAS cables!

When I setup my drive, I wanted to use a Striped SSD Dynamic disk as the cache drive, however in the dropdown the stripe drive was not listed.
Any idea why it's not there and how I can get it to be listed?
 
Thanks.  Great product so far.  Looking forward to Google Drive!

Well, thank you for your interest in our product!
 
 
And when duplicating data, it depends.  It does want to place files on the disk(s) with the most absolute free space. However, it does follow the balancing settings, as well. So if you're using the Disk Space Equalizer, it should follow it's settings, as well.
 
If you don't have the balancer installed yet, then it wouldn't use the 2TB drives until the 8TB drives are ~6TBs filled and the 4TB drives are ~2TBs filled.  Only once all the drives have the same amount of space, would it start using the 2TB drives.  
(that's barring "special circumstances" such as higher than 2x duplication).

I agree. I've been testing a lot of different alternatives to CloudDrive and its still the most stable app I've used.

Also please be aware that this stragegy alone does not protect you against an unnoticed accidental deletion or corruption of files (either from user error, malware, software bug etc).  Any deletion or corruption will be propogated to all 3 copies virtually instantly.
 
I'm designing a strategy at the moment and am trying to take this into account.  I'm thinking it is better to have a backup process using backup software that can restore state to a given time.  Back this up to a local drive (either in the pool or seperate) and then mirror it offsite.  I'm thinking of using Backup For Workgroups for this as it has the mirroring built in.

So far I have:
 
Supermicro X10SLH-F-O
Xeon E3-1276v3
16GB Crucial UDIMM (Another 16GB will be here on Friday)
LSI SAS 9207-8i
120GB OCZ Vertex3 SSD for OS
2x 5TB Toshiba (PH3500U-1|72)
 
I'll use that to upgrade my existing setup.
 
If my boss buys my existing server, I'll pick up a Norco 4224, and probably 4x 8TB Seagate Archive Drives
 

"And glad to hear that you got the airflow fixed and the drives running cooler! "
 
Biggest thing was that I had the cover off, and it made a HUGE difference when I put it back on.  Makes sense though, with the cover on, the air is forced through the front where the drives are.  I also closed the vents on the bays where I didn't have drives.
 
If I upgrade this, or rebuild using a 4224 I will be getting the 120mm fan wall.

My CURRENT setup:
 
Norco RPC-4220
Supermicro X7SPA-H-D525 Mobo
D525 (1.80Ghz) CPU
4GB Mushkin SO-DIMM DDR3 1333
Corsair TX650 PSU
LSI SAS 9201-16i
60GB OCZ Vertex2 SSD for OS
5x 3TB Barracuda XT (ST3300651AS)
6x 3TB WD Red (WD30EFRX-68A)
1x 2TB Landing Drive for Torrents
 
I'll either be upgrading to building a new server in the next two weeks...  Can't wait!
 

Thanks.
Yes, after 10 years with watercooling cases (some were dual Xeon), I am now only doing air cooled stuff and even if its more challenging regarding temperatures, its way cheaper...
A good controled air flow with positive air pressure can cool a lot of setup, with near to zero maintenance.
 
Now that I use DrivePool, I would be able to keep the same case and if I need more data room, only change 4TB drives as price drop with 8TB, one by one...
72TB in a small mid tower with all HD under 35°C 

Sorry, I have "only" 4 SSDs 
I was just curious if dual link would increase single SSD read/write but it doesn't.

Some picts of my quick but functional SAS expander mounting 
 
A very efficient 350W TFX PSU :
 

 
 
 
 
 
 
SAS mount   :
 

 
 
 
 

 
 
 
 
 
Mounted in the case.
The setup isn't complete yet (but working). When completed it would have 1x 240GB SSD + 3x240Gb caching SSDs, 6x4TB  + 3x5TB HDs : virtualy 39TB with SSD write caching x3.
On this picture I was testing dual link between the M1115 and Intel SAS expander :
 
 

 
 
 
 
Single link :
      I need some more work on cable management... But cooling is now perfectly tunned with SpeedFan that controls 5 case fans. HDs are between 29°C and 34°C and both SAS card's CPU below 45°C (thanks to a case fan puching air between both SAS cards).