Jump to content

Henrik

Members
  • Posts

    6
  • Joined

  • Last visited

  • Days Won

    1

Reputation Activity

  1. Like
    Henrik reacted to Alex in Server Backup and duplication question   
    If you are looking for a way to backup only the "master" copy of a file and not the duplicated part, then that is not possible.
     
    DrivePool has no notion of a master copy vs. a backup copy. Each copy of the same file is exactly identical, and the copies can reside on any pool part.
     
    If you want to backup your duplicated pooled files without backing them up twice, you will need to use a backup solution that does deduplication (E.g. the client computer backup engine part of the Windows Home Server 2011 / Windows Server 2012 Essentials). Alternatively you can use a file based backup system to backup directly from the pool (such as SyncBackSE)
  2. Like
    Henrik reacted to Alex in Questions regarding hard drive spindown/standby   
    This is actually a fairly complicated topic.
     
    Let's start by talking about how normal standby works without the StableBit Scanner getting involved.
     
    Windows "Put the Disk to Sleep" Feature
     
    Normally, Windows will monitor the disk for activity and if there is no disk activity for some preset amount of time it will put the disk to "sleep" by flushing all of the data in the cache to the disk and sending a special standby command to it. At the same time, it will remember that the disk is asleep in case any other application asks.
     
    Shortly after the disk goes to sleep, the StableBit Scanner will indicate the fact that the disk is asleep in the Power column. Normally, the Scanner gets the power status of the disk by querying Windows and not the disk.
     
    It does not query the disk directly for the power state because Windows considers this power query disk activity and wakes up the disk as a result.
     
    Now, things get a bit more complicated if you want to include the on-disk power management in this picture.
     
    Disks can optionally support these features, which can put them to sleep without Windows knowing it:
    Advanced power management. Standby timer Advanced Power Management
     
    This is a technology that implements power consumption profiles. For instance, if you don't care about performance but want maximum power savings, then you can tell your disk just that. Simply set the Advanced Power Management to Minimum Power Consumption. Or you can do the exact opposite by setting it to Maximum Performance (which guarantees no standby).
     
    With Advanced Power Management you don't concern yourself with "sleep timeouts", like in Windows. You simply state your intent and the disk will adjust various parameters, including the standby time, according to your setting.
     
    The implementation of Advanced Power Management is completely up to the manufacturer of the drive, and there are no specifications that explicitly state what each power mode does. This entire feature may not even be supported, depending on the disk model.
     
    Standby Timer
     
    The Standby timer is more widely supported because it is an older feature. You simply specify after how much disk inactivity you would like the disk to be put to sleep. This is similar to how things work in Windows, except that the low power mode will be initiated by the disk firmware itself.
     
    Again, the implementation of this is up to the manufacturer of the drive.
     
    StableBit Scanner "Put into Standby"
     
    In the StableBit Scanner, you can right click on a disk and put it into standby mode. What this does is send a power down command to the disk. This type of power down is equivalent to what Advanced Power Management or the Standby Timer would do.
     
    More importantly, when a disk is powered down in this way, Windows will not be aware that the disk is in a low power state, and will continue to report that the disk is still powered up. This is not an issue because the disk will simply spin up the next time that Windows tries to access it.
     
    But this leaves the StableBit Scanner with a dilemma. If we can't query the disk for the power state directly, how do we report the true power state of the disk? What the StableBit Scanner implements is a power state in which it's not sure whether the disk is in standby or active, and this is what you were seeing.
     
    Forcing the StableBit Scanner to Query the Power Mode from the Disk
     
    If you want to use on-disk power management exclusively, and you don't care about Windows putting your disks to sleep, you can instruct the StableBit Scanner to query the power mode directly from the disk.
     

     
    When this is enabled, you will no longer see the standby or active message, but Windows will never try to put that disk to sleep. That's why this is off by default.
     
    SMART
     
    And just to make things even more complicated, sometimes a disk will wake up when it's queried for SMART data.
     
    To this end the StableBit Scanner implements some more settings to deal with this:
     

     
    I hope that this clears things up.
  3. Like
    Henrik got a reaction from Christopher (Drashna) in Duplication count greater than expected   
    It's only the directory entries that occur in many duplicates and not the files within them.
     
    Example setup:
     
    +Pool of 3 drives.
    +Folder X has a duplication count of 2x and contains 5 files. (10 files in total across 3 drives)
     
    If the folder is then balanced across the 3 drives Folder X will exist in three copies because there will be files belonging to that folder stored on all drives
    (For example drive A with 3 files, Drive B with 4 files and Drive C with 3 files)
     
    So getting the duplication count on folders is a bit misleading. You should look at individual files instead.
     
    Hope this clears it out
  4. Like
    Henrik got a reaction from Shane in Duplication count greater than expected   
    It's only the directory entries that occur in many duplicates and not the files within them.
     
    Example setup:
     
    +Pool of 3 drives.
    +Folder X has a duplication count of 2x and contains 5 files. (10 files in total across 3 drives)
     
    If the folder is then balanced across the 3 drives Folder X will exist in three copies because there will be files belonging to that folder stored on all drives
    (For example drive A with 3 files, Drive B with 4 files and Drive C with 3 files)
     
    So getting the duplication count on folders is a bit misleading. You should look at individual files instead.
     
    Hope this clears it out
  5. Like
    Henrik reacted to Shane in FAQ - Parity and Duplication and DrivePool   
    The topic of adding RAID-style parity to DrivePool was raised several times on the old forum. I've posted this FAQ because (1) this is a new forum and (2) a new user asked about adding folder-level parity, which - to mangle a phrase - is the same fish but a different kettle.   Since folks have varying levels of familiarity with parity I'm going to divide this post into three sections: (1) how parity works and the difference between parity and duplication, (2) the difference between drive-level and folder-level parity, (3) the TLDR conclusion for parity in DrivePool. I intend to update the post if anything changes or needs clarification (or if someone points out any mistakes I've made).   Disclaimer: I do not work for Covecube/Stablebit. These are my own comments. You don't know me from Jack. No warranty, express or implied, in this or any other universe.   Part 1: how parity works and the difference between parity and duplication   Duplication is fast. Every file gets simultaneously written to multiple disks, so as long as all of those disks don't die the file is still there, and by splitting reads amongst the copies you can load files faster. But to fully protect against a given number of disks dying, you need that many times number of disks. That doesn't just add up fast, it multiplies fast.   Parity relies on the ability to generate one or more "blocks" of a series of reversible checksums equal to the size of the largest protected "block" of content. If you want to protect three disks, each parity block requires its own disk as big as the biggest of those three disks. For every N parity blocks you have, any N data blocks can be recovered if they are corrupted or destroyed. Have twelve data disks and want to be protected against any two of them dying simultaneously? You'll only need two parity disks.   Sounds great, right? Right. But there are tradeoffs.   Whenever the content of any data block is altered, the corresponding checksums must be recalculated within the parity block, and if the content of any data block is corrupted or lost, the corresponding checksums must be combined with the remaining data blocks to rebuild the data. While duplication requires more disks, parity requires more time.   In a xN duplication system, you xN your disks, for each file it simultaneously writes the same data to N disks, but so long as p<=N disks die, where 'p' depends on which disks died, you replace the bad disk(s) and keep going - all of your data is immediately available. The drawback is the geometric increase in required disks and the risk of the wrong N disks dying simultaneously (e.g. if you have x2 duplication, if two disks die simultaneously and one happens to be a disk that was storing duplicates of the first disk's files, those are gone for good).   In a +N parity system, you add +N disks, for each file it writes the data to one disk and calculates the parity checksums which it then writes to N disks, but if any N disks die, you replace the bad disk(s) and wait while the computer recalculates and rebuilds the lost data - some of your data might still be available, but no data can be changed until it's finished (because parity needs to use the data on the good disks for its calculations).    (sidenote: "snapshot"-style parity systems attempt to reduce the time cost by risking a reduction in the amount of recoverable data; the more dynamic your content, the more you risk not being able to recover)   Part 2: the difference between drive-level and folder-level parity   Drive-level parity, aside from the math and the effort of writing the software, can be straightforward enough for the end user: you dedicate N drives to parity that are as big as the biggest drive in your data array. If this sounds good to you, some folks (e.g. fellow forum moderator Saitoh183) use DrivePool and the FlexRAID parity module together for this sort of thing. It apparently works very well.   (I'll note here that drive-level parity has two major implementation methods: striped and dedicated. In the dedicated method described above, parity and content are on separate disks, with the advantages of simplicity and readability and the disadvantage of increased wear on the parity disks and the risk that entails. In the striped method, each disk in the array contains both data and parity blocks; this spreads the wear evenly across all disks but makes the disks unreadable on other systems that don't have compatible parity software installed. There are ways to hybridise the two, but it's even more time and effort).   Folder-level parity is... more complicated. Your parity block has to be as big as the biggest folder in your data array. Move a file into that folder, and your folder is now bigger than your parity block - oops. This is a solvable problem, but 'solvable' does not mean 'easy', sort of how building a skyscraper is not the same as building a two-storey home. For what it's worth, FlexRAID's parity module is (at the time of my writing this post) $39.95 and that's drive-level parity.   Conclusion: the TLDR for parity in DrivePool  
    As I see it, DrivePool's "architectural imperative" is "elegant, friendly, reliable". This means not saddling the end-user with technical details or vast arrays of options. You pop in disks, tell the pool, done; a disk dies, swap it for a new one, tell the pool, done; a dead disk doesn't bring everything to a halt and size doesn't matter, done.   My impression (again, I don't speak for Covecube/Stablebit) is that parity appears to be in the category of "it would be nice to have for some users but practically it'd be a whole new subsystem, so unless a rabbit gets pulled out of a hat we're not going to see it any time soon and it might end up as a separate product even then (so that folks who just want pooling don't have to pay twice+ as much)".
×
×
  • Create New...