Christopher (Drashna)

You are very welcome. Alex did a good amount of research on how to implement the encryption feature, and we hope that it is secure enough that even the most security conscious users would be fine with using it. And without impacting performance too much. If you have any more questions, don't hesitate to ask.

@PhotonJunkie, You could seed the pool, and then set up a rule, right away. http://wiki.covecube.com/StableBit_DrivePool_Q4142489 Don't reset the settings, just remeasure the pool. That should get what you want, I think. As for the fragmentation, I wasn't sure what you were getting at, and I wanted to make sure I covered all bases. You were referring to fragmenting the folders, right? As for the drive loss, that's always unpleasant. However, one could argue that having the entire contents of a folder on a specific drive, that you're more likely to lose EVERYTHING from that folder, rather than just partial contents. And that's part of why we recommend duplicating everything, if it's important at all. This way, if a disk fails or starts to go bad, you can remove it right away and just reduplicate the files after the disk is removed (skipping the duplicate files during removal). And Parity is definitely another solution, but not one that I'm found of. You're exchanging raw capacity at the expense of additional (significant) CPU usage and additional load on the drives. With duplication, you can access ALL of the files taht are duplicated when a disk fails/goes missing, and only have to copy them over to another disk to rebuild (and the StableBit DrivePool service handles this automatically once you've removed the disk), whereas parity ... the files may not be accessible until you "decompress" (oversimplification) them from the parity data. However, this is a well worn debate (at least to me), so if you have a preferred method, by all means, use it.

Duplicated is the total amount of space used by duplicated files. When this is done, this will be double the amount of data you ahve. If you're duplciating everything, it will be 20.4TBs of duplicated data. From the looks of it, and assuming a few things, your pool should this: Duplicated = 13.4TB Unduplicated = 3.62TB Other = 8.15GB This means that you have 6.7TBs of data duplicated, (x2 since you have two copies) and 3.62TBs of unduplicated data. That's 10.32TBs of data, which is pretty spot on from what you've said. As for the other data, if it's in GBs, then that's fine (and fairly decent for that size pool) As for the progress, that's it's current progress on the current duplication pass (if you rebooted, or restarted the service, or changed settings, it will reset the progress, so it may not be completely indicative of the overall progress since starting). If you need any clarification or have any more questions, don't hesitate to ask.

It's definitely possible, but we chose not to enable this option, as these are generally "black box tests" and not every drive supports the tests (even if it says it does). So to prevent issues, we just don't enable the option. However, in most cases, a surface scan from StableBit Scanner should pick up most (if not all) of the same issues.

Well, the Expander cards ... don't have to be plugged in, in most cases. The Intel SAS Expander card I picked up can be powered by the PCIe bus, or by molex, so you can mount the card anywhere in the case, actually. That may be worth looking into. But yes, they're about the same price as the controller cards themselves. But for an organization neat freak (like myself), they're very nice. As for the Name, this is pulled from the device itself (via WMI, but it is what is reported by the OS). Check Device Management ("devmgmt.msc") and compare the entries there. It should be the same, actually. This also depends HEAVILY on the controller card, and how it passes through the information. Some include just the name, but some cards alter the name (for instance, my LSI card, which is a IBM ServeRAID M1015, adds "ATA" to the beginning of the disk's name, and "SCIS Device" to the end of it. And you can use the Disk Settings to customize the disks names, if needed (right click on the disk).

Well, they're expensive for a few reasons. They're solid metal usually (not as "flimsy" as some cases), they usually include redundant power supplies (which are expensive), and they usually use some sort of SAS chipset or expander (which are expensive). Also, they usually support chaining drives to the enclosures, so you can get a LOT of drives connected to one system. But yeah, very, very expensive. And either way, very impressive!

The issue with that, is that somebody else controls and maintains it. And it's completely out of your hands. And at least in the USA, providers/hosts are not required to inform customers that they have been issued a warrant or given access to your data/system to gov't agencies. So a major point of StableBit DrivePool, and StableBit CloudDrive is to mitigate privacy issues, like that, by having much more control over the data.

Thank you for uploading. What OS are you using specifically, and the architecture of it (32 bit vs 64 bit)?

I'm glad to hear it!

Could you grab the logs from the system? http://wiki.covecube.com/StableBit_DrivePool_2.x_Log_Collection Otherwise, try using the "Disk Usage Limiter" balancer to clear the drive in question. Uncheck both options for the drive, and let it sit. And yes, in both cases, drive removal may be slow, as it's removing all of the data from the drive in question. If you don't have a lot of duplicated data, then it will take longer.

Very nice. And speed doesn't matter for "spinners". SATA II gets up to 380MB/s per drive, IIRC. So more than fast enough for slow spinners. And very nice. I'm nowhere near your raw storage yet, but I'm working my way up (14 drives: 2x 8TB Seagate archive, 4x Seagate NAS, 8x WD Reds). And I don't have enough stuff to fill more than my current amount of space, and it's already duplicated. And if you haven't already, look into SAS Expander cards. You can get up to 512 drives connected to that ONE controller, IIRC (the 3Wire).

I've been using MetaBrowser for a while, actually. And I've seen this issue, I believe it's related to deleting the folders during the rename. And if it is, the issue is fixed in the internal beta builds. http://dl.covecube.com/DrivePoolWindows/beta/download/StableBit.DrivePool_2.2.0.625_x64_BETA.exe http://dl.covecube.com/DrivePoolWindows/beta/download/StableBit.DrivePool_2.2.0.625_x86_BETA.exe Try downloading the one for your system, and see if that fixes the issue. If it doesn't, then enable file system logging and reproduce the issue. http://wiki.covecube.com/StableBit_DrivePool_2.x_Log_Collection This is the issue that I think is the issue, specifically: [D] [issue #13443] When a folder delete fails because of files in one pool part folder, roll back the delete on any other pool parts.

Thanks. So it's definitely quota related, then. I'll bump the issue then, and bug Alex (the developer) about it.

The "File Placement Rules" feature can do this, actually. It may require extensive micromanagement, depending on how your pool is setup, but the UI should allow you to easily do this. http://stablebit.com/Support/DrivePool/2.X/Manual?Section=File%20Placement As for fragmentation, you can defragment the disks in the pool, without any issues. So that is a drive does fail, you can be certain that you have all of the contents of a particular folder (or none at all). I've been there (not blue ray rips, but re-encoded files), and it sucks. It may be worth investing in enough drives to duplicate everything.

We don't support "EFS" on the pool. This is what it's complaining about. Do you have any files that are locally encrypted on the disk? If so, that's what it's talking about. Specifically, it may notice that we don't support EFS and is warning you.

Very nice! And I can definitely appreciate how heavy that is. I have a Norco RPC-4220 that's half filled... and it's heavy as heck. And that's with the 3ware 24 port card? Also, if you need more space, somebody linked this to me recently: http://www.supermicro.com/products/chassis/4U/946/SC946ED-R2KJBOD.cfm Maybe you might need one of them soon (I know ... I do ... )

Yeah, by default, we use background IO, which means it's slower but won't interfere with normal usage. Disabling that or bumping the priority will definitely copy faster. As for multithreaded, yeah, we are definitely running the duplication task as a single threaded process. I can make a good guess why. The placement of the files are important. They need to not violate the balancing settings and the file placement rules (if any), and the copies have to be on different physical disks. By making it multi threaded, it will mean that there is more work to be done in the backend to track where the files are ending up, and where new ones should be copied to. This would definitely make the code handling duplication a lot more complicated. However, it sounds like Alex would definitely like to implement this in the future.

Very nice! The second card is "only" SATA II, from the looks of it, but for most HDDs, that should be more than fine.

Honestly, I'm not sure about that. I'll ask Alex (the developer). However, the priority can be boosted UI, which helps with copy speeds. And there is an advanced setting to disable "background IO" for duplication: http://wiki.covecube.com/StableBit_DrivePool_2.x_Advanced_Settings set "FileDuplication_BackgroundIO" to "False" and restart the "StableBit DrivePool Service" (run "services.msc" to do so, or reboot the system).

Ah. Odd. Something like this? http://serverfault.com/questions/396722/your-system-administrator-does-not-allow-the-user-of-saved-credentials-to-log-on

I've definitely not noticed this. However, could you do a few things? http://wiki.covecube.com/StableBit_DrivePool_2.x_Error_Reports http://wiki.covecube.com/StableBit_DrivePool_Service_Memory_Dump http://wiki.covecube.com/StableBit_DrivePool_UI_Memory_Dump Once you've done that, let us know.

To clarify, you have both of those option enabled, and it is still causing the disks to wake?

Glad to hear that it finished measuring. If it continues to have issues measuring, then let us know.

That's .... odd. Well, glad you found the fix. Try running "mstsc" on the system. From there, click on the "more options" option (bottom, left corner). This should expose a lot more settings, including a checkbox that says "Allow me to save credentials". That should be what you want.

No additional hardware is required here (eg, you don't need a TPM module). As for the encryption: StableBit CloudDrive uses low level kernel-based encryption for maximum performance and for full round trip encryption support. Industry standard AES-256 CBC is used to encrypt every bit of your data. The Operating System's core services (Cryptography API: Next Generation) are used for all hashing and encryption functions in the kernel (FIPS 140-2 compliant). This, and a number of other features are mentioned here: https://stablebit.com/CloudDrive/Features As for accessing the CloudDrive, you can only access it from one system at a time. If you try to "attach" it to another system, it will give you a warning that it's already in use. You can go ahead and forcible attach it on the new system, and it should detach it on the other system. However, it will be encrypted, and require you to put in the passphrase or hash key for the CloudDrive before you can access the data.