Jump to content

Christopher (Drashna)

Administrators
  • Posts

    11804
  • Joined

  • Last visited

  • Days Won

    394

Reputation Activity

  1. Like
    Christopher (Drashna) reacted to Hakuren in Is SBDP a good fit for me?   
    Not to brag or anything, but I'm that mystery 'someone on OCN'.
     
    I'm glad you went with my suggestion. I'm pretty certain you won't regret it. SBDP is excellent piece of software.
  2. Like
    Christopher (Drashna) got a reaction from Blindsay in Is SBDP a good fit for me?   
    StableBit DrivePool stores the files on the individual disks themselves, in hidden "PoolPart" folders.  So, yes, if a single disk fails, you only lose the contents of that disk.  However, if you're using duplication, only unduplicated data on that disk is lost, and the software will check and reduplicate data as needed after you remove the problem disk.  
    Additionally, since each disk is handled separately, and the pool drive is an emulated disk, there is no size limitation here. You can continue to use 4K cluster (allocation units) on the drives without any problems.  In fact, I was doing so until recently (well past 50TBs in fact). 
     
    Unfortunately, no, there isn't a list of the data on the drive. We use NTFS to look up what is on the disk, rather than relying on an external database, for a number of reasons.  We do have a thread on this (as well, as potential plans on changing that, in some regards):
    http://community.covecube.com/index.php?/topic/746-drive-must-have-died-is-there-a-way-to-tell-what-was-on-it/
     
    You should definitely check it out, as a number of people list ways (or software) they use to inventory their systems. 
     
    But to be honest, the easiest way is to enable duplication on the entire pool, or maybe use something like SnapRAID to add snapshot parity. 
     
    This one is trickier. Having a backup is absolutely a good idea. But redundancy is good too. But redundancy isn't a backup. Generally, it is good to have both.   However, the reason that I bring this up is that if you do have slower drives in the pool, and duplication enabled, the Read Striping feature will actually attempt to read from the faster or less busy disk, to optimize performance. 
     
    Otherwise, as for backing up, some tools won't work on the pool. Because the pool doesn't support VSS, tools like Acronis don't support it properly, and you'd have to backup the pooled drives rather than the pool. 
    However, using syncing tools (such as BTSync, Allways Sync, Free File Sync, or the like) you can copy the files to a backup location without a problem. 
     
    If you need any more clarification or have any additional questions, don't hesitate to ask.
  3. Like
    Christopher (Drashna) got a reaction from steffenmand in Multiple reads on the same offset   
    Honestly, I'm not sure here.  (this is definitely one of the much more technical aspects of the software, and I'd rather have a right answer for you, than a guess). 
     
    I've flagged the question for Alex (and it's marked as critical, so he should get to it soon).
    https://stablebit.com/Admin/IssueAnalysis/25952
     
    However, I suspect that this is normal and may be related to something the software is doing. But again, since I'm not sure ....
     
     
     
     
    As for the partial chunk size, I'll push it again.  
     
     
    Though, Alex has been running consistency checks on all of the providers, and has been looking into a potential corruption issue that Modplan has ran into. (any corruption issues always get top priority, because of their nature). 
  4. Like
    Christopher (Drashna) got a reaction from Talyrius in Checksum Mismatch....What do I do?   
    Just a heads up, there was an issue with the checksum code, that is resolved in the latest internal builds. 
     
  5. Like
    Christopher (Drashna) got a reaction from rimmon1971 in Win7 -> Win10 and using of NTFS folders mount option for underlying drives *and* Drivepool volume   
    Yup, exactly. 
     
    Though, I'd recommend mounting the pooled disks to a folder (such as folder under "c:\DrivePool", so you can keep the root folder "clean").  And this way, you just have the C:\ drive and the pool drive. listed in This PC/My Computer/etc.  Part of the reason for this is if you need to move files around, it's handles it better in regards to use feedback if it's not to a folder path "on the same disk".
  6. Like
    Christopher (Drashna) reacted to mh272 in Data on drive but not in pool   
    Thanks to both of you for the help and the additional links.  Very helpful.
  7. Like
    Christopher (Drashna) reacted to grimpr in Drivepool and Scanner under Hyper-V / Server 2012.   
    Thanks, i'll think i just use Windows 10 and only run Drivepool /w Scanner on it, everything else will be run in virtual machines with the included Hyper-V, its the better way to do this for me, no need to mess up with complex solutions.
  8. Like
    Christopher (Drashna) got a reaction from grimpr in Drivepool and Scanner under Hyper-V / Server 2012.   
    Depends on the exact setup.
     
    If you're passing the disks for StableBit DrivePool to a VM, there should be absolutely no issues with doing so.
     
    However, StableBit Scanner won't be able to pick up the SMART data for the passed through disks (the HyperV drivers don't pass this info along).
    You can see more about that here:
    http://community.covecube.com/index.php?/topic/29-how-to-getting-smart-data-from-hyper-v/
     
    Aside from that, you shouldn't have any issues. Though, deactivate the license prior to moving things around. 
     
     
     
     
    Oh, and WIndows 10 Pro includes HyperV
  9. Like
    Christopher (Drashna) got a reaction from modplan in Amazon Cloud Drive - Why is it not supported?   
    The problem is that we are. and have been.  The problem is server stability issues, as well as trying to pull teeth to get actual limitations that we should be using. They're not documenting these at all, so we're doing this blindly. 
     
    Also, because we are a very small company, we don't really have the resources to go around. So our top priority pretty much as to be issues and improvements that benefit all (or many) of the providers, rather than focusing on a single provider. 
     
    To be blunt, if Amazon Cloud Drive documented their API limits and guidelines and documented the max bandwidth per client allowed, or .... well, actually created a service that PROPERLY implemented throttling (both for APIs per time period, and for connection speed aka MB/s), then there would be a much smaller issue here, and it would be much easier to implement.
     
    Because the issue why we have "temporarily abandoned" Amazon Cloud Drive, because the point we were at was in a very bad development cycle that Amazon mandates. Specifically:
    Release a build and send it to Amazon Amazon analyzes it's usage Amazon tells us that we're making too many API calls, using too many threads, uploading too fast, or making more API calls than expected and tells us we need to change that Fix it to fit within Amazon's black box limits Repeat That's what we were doing, and it was affecting the development status of the entire product. Because from our standpoint, it really looks like Amazon doesn't have their shit together. At least in regards to their Cloud Drive service.
     
     
    And yes, I know it's very popular, because it's about the cheapest, unlimited provider. In fact, I do have a paid subscription for it, so I definitely have a vested interest in it working properly. 
     
     
     
     
     
    Even *if* we do get approved for production status (again), there will be a hard limit on the bandwidth (otherwise Amazon *will* demote us again, most likely), and unless we are able to get it working reliably and stably, it will be stuck as an experimental provider. 
     
    And we do plan on pursuing this, but once we have a stable build and .. well, more time to devote solely to it.
  10. Like
    Christopher (Drashna) got a reaction from Mirabis in SSD Optimizer - Idiot's Guide   
    Not a problem. It's a bit more complicated, especially as it basically requires the use of the File Placement rules:
    Install the SSD OPtimizer, enable it, and configure it. Set File Placement rules for the documents, pictures, etc, selecting every disk except for the SSD drive(s).  On the Settings (main) page for the balancing, uncheck the "File placement rules respect real-time file placement limits set by the balancing plug-ins".  This tells the software to respect the file placement rules, even when they violate the real time placement limits set by the SSD Optimizer. 
     
     
    If you want the File Placement Rules to keep the files on the SSD drive, regardless:
    Install the SSD OPtimizer, enable it, and configure it. Set File Placement rules for the folders/files in question etc, selecting the SSD drive(s) (and any others).  On the Settings (main) page for the balancing, uncheck the "Unless the drive is being emptied".  Normally, the SSD Optimizer balancer attempts to "empty" the SSD/cache drives, clearing them out. Normally, the balancers will respect the file placement rules *except* when the drives are being emptied. Meaning it will not respect the file placement rules properly.  Unchecking this means that the file placement rule will *always* be respected, even if a balancer (including the StableBit Scanner balancer) wants to empty out the drive.      These rules are not mutually exclusive, meaning that if you want to do both, just set the rules and settings accordingly, and you can get both options. 
  11. Like
    Christopher (Drashna) reacted to JscoLP in Plans for ReFS support?   
    I would really like to see ReFS fully supported (2012 R2 version), and wouldn't mind helping you test it out if you could provide some way to get you helpful data from my environment. I installed the latest beta (2.2.0.665) and created a pool, files went on ok... but not sure how I can be more helpful.
  12. Like
    Christopher (Drashna) reacted to grimpr in GPT vs MBR   
    Nice to know, i have 2 drives as GPT in my pool and the rest MBR, Drivepool initialized only those two and the rest stayed as they were, now with the new information from you and other threads i'm in the process of emptying one by one the drives and reformatting them again to GPT,64k cluster size.
  13. Like
    Christopher (Drashna) got a reaction from grimpr in GPT vs MBR   
    IIRC, StableBit DrivePool initializes new disks as GPT by default. So if that matters... 
     
    And yeah, the 64k cluster/allocation unit size is worth it. Especially with Seagate Archive drives!
  14. Like
    Christopher (Drashna) got a reaction from grimpr in GPT vs MBR   
    Yes. 
     
    GPT supports addressing more than 2TBs of space without the need for advanced formatting, as well as native support for advanced formatting.  Additionally GPT supports very large disk capacities specifically (in the ZB/billions of TB range).
     
    Additionally, GPT drives provide redundancy, in the form of writing the the GPT header and partition table at the beginning and end of the disk (this helps prevent the "RAW" issue, along with other issues). 
     
     
    Aside from these, the only desernable difference is the boot drive. BIOS based systems (or legacy boot modes) require MBR disks, while UEFI usually require GPT disks. 
     
    For storage though, there is no noticeable difference for "daily use". Other that stated limitations of MBR.
  15. Like
    Christopher (Drashna) got a reaction from grimpr in ECC ram for a Drivepool server.   
    As long as it's USB3, and a very fast USB drive (eg, a certified Windows To Go drive), then there shouldn't be any issues with doing so. 
     
    I actually tested this out (sort of) recently. Using a USB3 enclosure and a SSD for Windows To Go, and it works fine in that config. 
  16. Like
    Christopher (Drashna) got a reaction from Shane in Change allocation unit size of all drives in DrivePool: Best way?   
    LOL.  I just recently did this on my own pool, actually. 
     
     
    There isn't a good way. You need to format the drives to do this (or use a 3rd party tool, and hope it doesn't corrupt your disk).
     
    The simplest way is to use the balancing system.  Use the "Disk Space Limiter" balancer to clear out one or more disks at a time. Once it's done that (it make take a few hours or longer), remove the disk from the pool and reformat it. Re-add it and repeat until you've cycled out ALL of the disks.
     
     
    Specifically, the reason that I mention the Disk Space Limiter balancer is that it runs in the background and doesn't set the pool to be read only. And is mostly automated.
  17. Like
    Christopher (Drashna) got a reaction from grimpr in Change allocation unit size of all drives in DrivePool: Best way?   
    For a number of reasons. 
     
    By default, Windows wants to use 4k clusters. This means that each allocation unit for files is this size. If you ahve a 10MB file, it's broken up into 4k chunks and placed on the disk.  The flip side is that each file uses 4k period. That includes folders, which have very little data.  
     
    Additionally, even if the file (or the remaining chunk) uses a couple of bytes, it will take up the entire allocation unit. 
     
     
    Now, why is this important? Well, a 64kb chunk is 16x the size of the 4kb one.  
    This means that more data is guaranteed to be sequential.  This is of one main benefit: Fragmentation. 
     
    Sinch each chunk is larger, that means that you're less likely to see fragmentation, Additionally, since more of that data is in once section of the disk, it means that there is less seeking being performanced on the disk, and may increase performance.
     
    THis is the same reason some people recommend that you defragment your disk. 
     
     
     
    The flip side, as I mentioned, that if you have a lot of small files, you're losing disk space. A cluster or allocation unit can only be used by one file. 
    For an OS disk, or one with a lot of very small files, this can cause an issue, as you may run out of allocation units before you run out of disk space.   
    However, for storage, a majority of the files are going to be much larger, and will benefit from the allocation unit size increase. 
     
     
     
     
    So what does this equate to for you? Well, on my Seagate Archive drives, I saw fewer performance drop offs, and I saw about a 20MB/s speed increase on reads. 
  18. Like
    Christopher (Drashna) got a reaction from taflemer in CloudDrive Signature   
    That's very odd.
     
    The SHA-1 hash should be "EF287BAE442E906CD7694FE595C920171964F816" and the MD5 hash should be "F406036DA6581D5782979098193EDB6F"
     
    You can verify this with stuff such as HashTab: http://implbits.com/products/hashtab/
     
    However, chances are that this is an issue with your root certificates. 
    You should be able to fix that by doing this:
    http://wiki.covecube.com/StableBit_DrivePool_Q6246716
  19. Like
    Christopher (Drashna) got a reaction from daveyboy37 in Change allocation unit size of all drives in DrivePool: Best way?   
    For a number of reasons. 
     
    By default, Windows wants to use 4k clusters. This means that each allocation unit for files is this size. If you ahve a 10MB file, it's broken up into 4k chunks and placed on the disk.  The flip side is that each file uses 4k period. That includes folders, which have very little data.  
     
    Additionally, even if the file (or the remaining chunk) uses a couple of bytes, it will take up the entire allocation unit. 
     
     
    Now, why is this important? Well, a 64kb chunk is 16x the size of the 4kb one.  
    This means that more data is guaranteed to be sequential.  This is of one main benefit: Fragmentation. 
     
    Sinch each chunk is larger, that means that you're less likely to see fragmentation, Additionally, since more of that data is in once section of the disk, it means that there is less seeking being performanced on the disk, and may increase performance.
     
    THis is the same reason some people recommend that you defragment your disk. 
     
     
     
    The flip side, as I mentioned, that if you have a lot of small files, you're losing disk space. A cluster or allocation unit can only be used by one file. 
    For an OS disk, or one with a lot of very small files, this can cause an issue, as you may run out of allocation units before you run out of disk space.   
    However, for storage, a majority of the files are going to be much larger, and will benefit from the allocation unit size increase. 
     
     
     
     
    So what does this equate to for you? Well, on my Seagate Archive drives, I saw fewer performance drop offs, and I saw about a 20MB/s speed increase on reads. 
  20. Like
    Christopher (Drashna) got a reaction from grimpr in ECC ram for a Drivepool server.   
    ECC is not required for StableBit DrivePool, at all. In fact, I'm pretty sure that a vast majority of our users are not using ECC memory, and it's tested on systems without ECC memory. 
     
     
    However, because NTFS does use the memory extensively for caching, so there is definitely a benefit to use ECC memory.  If you have a choice and there isn't a vast difference in price, paying for ECC memory is worth the money. 
     
     
    Additionally, ZFS has some pretty high memory requirements too (IIRC, the recommended is 1GB per TB of storage). StableBit DrivePool can operate on systems with 512MB of memory without an issue (though, we do recommend a lot more than that, as Windows may be very slow with low memory)
  21. Like
    Christopher (Drashna) got a reaction from steffenmand in +1 for GoogleDrive for Work support   
    Well, thank you for the kind words! And we're glad that you like StableBit CloudDrive!
     
    As for the erratic speeds, this may be related the partial read stuff. Since we break up the larger chunks into smaller pieces and read just the partial chunks, this can impact performance in some cases (hence the desire for larger partial chunk sizes). 
    As for the IO errors, these may be normal, as they happen from time to time. More with some providers than others. But any error is automatically retried. 
     
     
     
     
     
    These two are related, so grouping.
     
    No, there isn't any way to pause or throttle the bandwidth properly right now. It's on the to-do list, but probably won't be until after the stable release is out. (including maybe scheduling).
     
    However, you can "pause" it by setting the upload and/or download threads to "0". (Disk Options -> Performance -> IO Performance).  This essentially pauses the transfer. 
     
     
     
     
    Well, missing chunks is a bad thing, in general. Since we're essentially storing the raw disk data on the drive, this would be like a piece of a physical disk "dying" and would cause data loss.
     
    However, like a physical disk, running a "CHKDSK" pass should "fix" a lot of disk issues. 
    Additionally, we do a lot to make sure that nothing happens to the data, at least from our end.  In fact, one of those options is "upload verification" (where we download the file, make sure it uploaded and uploaded properly, and re-upload it if something went wrong).  Some providers have this enabled by default (such as ... Amazon Cloud Drive, as it's necessary for this provider), but it's an option for any provider. 
     
    However, for each chunk (and the partial chunks), we do include checksums, to help detect errors. 
     
    But really, you should check out these links:
    http://community.covecube.com/index.php?/topic/1610-how-the-stablebit-clouddrive-cache-works/
    http://community.covecube.com/index.php?/topic/1622-large-chunks-and-the-io-manager/
     
    These talk about the cache manager, and some of the details that you want to know (rather than me retyping it all here, and not doing as good of a job). 
     
    As for redundancy, no, there isn't any, but yes, you could use StableBit DrivePool for that. But just keep in mind, if you're using multiple drives from your google account, this will reduce the overall throughput to the provider. Though, from the sounds of it, that may not be an issue. 
     
     
     
    If checksumming is enabled, then yes, there is verification. This is enabled by default on most providers IIRC.
     
    As for partial chunk support for larger chunks, every MB of data has a checksum section, specifically to ensure that the data is intact. 
  22. Like
    Christopher (Drashna) reacted to jonww74 in Smart check not accessible - Proliant Gen8 Microserver   
    I appreciate that you have your reasons on what to support or not, it's just a shame that all my main drives don't get the monitoring I was expecting.
     
    Regarding the disk display, in this case there is only one physical disk per array so identifying and displaying wouldn't be an issue.
     
    Thanks for the response though, I'm glad to see a support forum actually supporting
  23. Like
    Christopher (Drashna) reacted to jerblack in Setup Same CloudDrive on multiple Computers?   
    I am currently sharing video content through Plex to multiple remote users by just having my Plex server index the content that is stored in my CloudDrive hosted on Google Drive. I have gigabit fiber with no caps though and unlimited space on my Google Drive, so It's easier for me to host Plex for remote access that way. I found that I need to jack up to prefetch settings quite a bit to get acceptable performance. (Trigger:1, Forward: 40, Window: 3600), but after making that change I have no problem streaming from my CloudDrive. I have the bandwidth to support that though.
     
    If you are bandwidth constrained, then you can try using Plex Cloud Sync (needs Plex Pass). Then you can store your content in your CloudDrive, and then sync it to Cloud Sync (this means two copies of all your content in the cloud). When your remote user streams from Cloud Sync, they are streaming directly from your cloud provider (Google Drive, ACD) bypassing your Plex server and it's limited bandwidth entirely. You still need to use your upload bandwidth to sync the content up to Cloud Sync and your CloudDrive initially, but once it's there, it's there and all streams will come from the cloud.
     
    Some people also use SeedBoxes (a 3rd-party hosted VPS) to host Plex along with their torrents, but you would need a Windows-based host to make that work with CloudDrive. Check out r/seedboxes on Reddit for more info.
  24. Like
    Christopher (Drashna) reacted to steffenmand in +1 for GoogleDrive for Work support   
    I'm still pushing for the update that gives increased partial reads! - Hopefully you will get it running before you go out of beta  - As it will require a new drive, i can't really upload stuff before then!
     
    A lot of great fixes these days though, so props for that!
  25. Like
    Christopher (Drashna) got a reaction from steffenmand in Setup Same CloudDrive on multiple Computers?   
    I think the Op doesn't want to use his upload speed to do this. 
     
    If you upload the content to a CloudDrive, you can detach (unmount) it from the local system and then mount it on the other system. That works fine. But as for sharing it between two systems? Not really.
     
    However, Plex does allow for multiple users (via plex accounts, IIRC). But it does consume your upstream bandwidth. 
     
     
    NetDrive does have a solution that allows you to access from multiple systems, but it's not a full drive, and it's not encrypted (IIRC).
    THe difference is that StableBit CloudDrive is using the provider as a raw disk backend, basically. Meaning it stores raw drive data. And it caches new content locally before uploading it.  Because of this, there isn't a good solution to "share" (read) the drive in multiple locations, as the content may not be uploaded yet. This would cause corruption, in any scenario. 
     
    Maybe we could (later) add a "mount as read only" option, so that you can mount an unused disk as read only, and add a flag to it so it cannot be mounted as writable anywhere else. 
    This would allow you to do what you want, but it render the drive as un-updatable. Meaning no new content. 
×
×
  • Create New...