Jump to content

Alex

Administrators
  • Posts

    253
  • Joined

  • Last visited

  • Days Won

    49

Posts posted by Alex

  1. Are there any plans for DrivePool to support TrueCrypt? Are there any drive pooling products that support encryption? Does DrivePool support any other encryption tools?

     

    I would like to second this wish :)

     

    (I must admit I didn't use the trial... Instead I bought it right away since my old Flexraid didn't know this "problem" I thought drivepool would be able to do this and stumbled upon this Problem now)

     

    I've looked into this pretty extensively actually and got pretty far as far as adding TrueCrypt support. Unfortunately I ran into kind of a big technical issue. TrueCrypt volumes do not reside on emulated physical disks. So for example, if you look at our DrivePool virtual volume in disk management you will see that it has an emulated virtual disk backing it. That's because DrivePool actually emulates a SCSI disk and lets the Windows Plug and Play system mount it as usual.

     

    It turns out that TrueCrypt doesn't do this. It "hacks" its volume into existence, but there is no disk backing it. This presents a problem for DrivePool because it was written to work with physical disks (or even virtual disks, but there must be a disk there). Whenever a new pool part arrives in the system, DrivePool will query that pool part for some disk information including the physical storage unit index. This is critical to how real-time duplication functions because one or more pool parts can reside on the same physical disk (e.g. on multiple partitions). TrueCrypt doesn't know how to answer to these queries.

     

    So the bottom line is that DrivePool's pool part management code needs to understand how to deal with volumes that don't reside on physical disk and I plan to implement this in the forthcoming 2.1 BETA.

  2. Monty,

     

    I've done some testing with the SiL 3114 using the latest drivers and RAID firmware available here: http://www.siliconimage.com/support/

     

    I've posted my standard StableBit Scanner compatibility report here:

    http://community.covecube.com/index.php?/topic/237-sil3114-sata-controller-chip/

     

    For me, the card was displaying the correct serial number on all ports, but SMART data was only available on the first port. I couldn't get the BASE firmware flashed. The Windows and DOS flash utility both failed.

     

    Do you know what firmware you're using, and make sure that you're on the latest driver.

  3. Overall

    • Supports Identify: Yes *
    • Supports SMART: Yes *

    * When using the RAID firmware and driver, but only on the first port.
     



    RAID Firmware

    StableBit Scanner - Direct I/O

    • Methods: SmartIoCtl
       
    • Identify: Yes *
    • SDD: Yes
       
    • SMART Status: Yes *
    • SMART Attributes: Yes *
    • SMART Thresholds: Yes *
    • SMART Error Log: No
       
    • Power Mode: No

    * Data is only valid on the first port of the controller (bug in controller driver). Workaround implemented in StableBit Scanner 2.4.0.2914+.
     
    StableBit Scanner - WMI

    • SMART Status: No
    • SMART Attributes: No
    • SMART Thresholds: No
    • SMART Error Log: No
       
    • Power Mode: No

    Performance

    • Burst: 80.7 MB/s

    Tested on Windows 8 x64



    BASE Firmware

     

    Unable to Flash. All attempts failed.

  4. Does Scanner iterate through the methods until it finds one that is suitable to read the drives or is there a setting where I can tell it to use ScsiPassThrough48 explicitly?

     

    In particular, the Scanner iterates through the "safe" methods on first disk detection and from then on uses the method that works. For USB enclosures it uses BitFlock to get a "hint method" based on the hardware ID of the USB enclosure.

     

    SAT is not on the safe list because it was reported to crash some controllers in the past. You can enable the Unsafe Direct I/O method to always iterate through all of the non-USB methods.

  5. Saiyan,

     

    The StableBit Scanner uses your disk model to decipher your SMART data, in that process, it obtains SMART interpretation data which includes a flag indicating whether your drive is a SSD and whether it has a lifetime indicator that we know how to read.

     

    If you could submit your SMART data to BitFlock (see attached) then I can take a look at it and see if I could add that lifetime meter to your SMART screen.

     

    And as Drashna said, the SSD flag does not control whether we write to a drive during a surface scan. We simply never do.

     

    Thanks,

    bitflock_analysis.png

  6. About the error reporting, unfortunately BitFlock is a bit out of date in this regard. Automatic online reporting did not work out as intended. The main issue is that some people have repetitive errors (such as WMI or COM errors and the such) that can clog the automatic reporting system fairly quickly (at you can see).

     

    I'll try to update BitFlock within the next couple of days to stop doing that, and I'll also rebuild it with the latest code. Let me know if the issue continues after that and we can look into it further.

  7. 1/

    My mistake; - The system has three Sil3114 SATA cards each of is displaying wrong serial numbers.  (ie each card is reporting the same serial number for each of the four drives connected to it but the serial number shown is different from card to card). 

     

    The PnP info from cards the is: Ven 1095, Dev 3114, SubSys 3114 & Rev 02.

     

    2/

    You're right!  The machine name is showing in the body of each email.  However, can drive idenitifcation info be added to the notifications?

    1. Hmm, I'm looking at the code and it's checking for VID_1095 and whether the driver name starts with Si. I just checked my inventory and I actually have this card in-house. I'll plug it in tomorrow and try to reproduce this issue.

       

    2. A notifications overhaul is on the way. There are a number of things that people have been requesting, including what you've suggested. Other popular requests are to better control overheating emails to prevent getting too many of them at a time, SMS support, naming multiple sites support. All of this is coming to the StableBit Scanner 2.5.

       

    3. Thank you for the compliment.  :)

       

      Both product are running on WPF and .NET 4.0. The StableBit Scanner 2.X uses mostly Telerik controls (a 3rd party provider) StableBit DrivePool 2.X uses all custom controls written here (and some standard built-in WPF controls), and all of this running on a custom interprocess communications system to synchronize the UI with the Service.

  8. Yep,

     

    StableBit DrivePool's background duplication makes a note of any errors error and continues on. The errors are shown to you at the end of the entire duplication pass, but the UI for conflict resolution is really not that fancy. It's basically an automatic option or none. I'm sure that we can improve this in the future, but my thinking is that duplication conflicts should really be rare. I can't think of a situation where you would encounter duplication conflicts on a regular basis (at least in my experience).

     

    Also, with real-time duplication enabled, background duplication should only happen when you make an administrative change, or disconnect a disk.

  9. Ok, I've gotten a response (yes, I am a bit shocked).

     

    This is what they said: "if you would like to inquiry smart status of each drive, you can use our api to send inquiry commands to specific drives. api is public which you can find in the support CD and our web site, the download page."

     

    Based on what they're saying, it sounds like they have a proprietary and public API for querying the SMART data of individual drives. If so, this is great news. It means that we can certainly implement SMART support for their products.

     

    While I don't have the support CD, I will check the web site's download page. If anyone can supply the said support CD, that might help a bit.

  10. Well, this is going to be as off topic as it gets and fairly personal :)

     

    One of my personal love affairs is sci fi short stories, and I love originality. I'm not a big fan of the Sci Fi that's on TV today, but I loved Rod Serling's work on the Twilight Zone. Today, some of the best stuff that I've read online resides here: http://qntm.org/fiction

     

    This site is fully programmed by one Perl programmer (named Sam) and features some amazing and mind boggling stuff. I love what he's doing and would like to support him.

     

    The stories are absolutely free and the ones that I loved in particular are:

    But all of his stuff is just amazing, especially the ongoing Ra series.

  11. ...

     

    Perhaps you guys can contact Areca Technology (areca.com.tw) for assistance.

     

    It's not a high priority item on my list yet but it would be nice to see Areca cards supported since I purchase a license for Scanner a few days ago.. :)

     

    Thanks.

     

    If none of the current Direct I/O methods work then they might be using a proprietary protocol. I'll try contacting them, and see if I can get any technical info on this.

  12. Reparse points (or links) are a very simple concept to imagine, but it's deceptive.

     

    I've tried implementing 3 different approaches to handle reparse points and have run into brick walls each time.

     

    I think that we will have reparse point support but not in the 2.0 RELEASE FINAL. I have it planned for a 2.1 BETA.

     

    Regards,.

  13. I've looked at the dump, and it's clear that the Virtual Disk Service has stopped responding. VDS is a core component of Windows that enumerates information about all of the disks in the system.

     

    In particular the following call is stuck: http://msdn.microsoft.com/en-us/library/windows/desktop/aa383021(v=vs.85).aspx

     

    As a result, everything else eventually locks up as well. I'm not sure what I can do about this in my code. I can probably at some complexity put the whole update code in a new thread and then kill the entire thread if the VDS calls don't come back within some reasonable amount of time, but this is really counterproductive because nothing will be able to enumerate new disks in the system, which would make the Scanner quite useless.

     

    Here's a little test utility that I wrote some time ago to enumerate disks using VDS. It will simply enumerate all the disks and print out the enumeration data to the screen. It will also listed for disk update events and print them to the console as they come in. You may want to try running it when the service locks up to confirm that it is indeed VDS. The utility should lock up as well.

     

    Download: http://dl.covecube.com/VdsTest/VdsTest.exe

  14. Scanner is taking > 36 hours to scan a single disk.

    Is this normative?  An indication of lots of disk damage?  what gives

     

    As the developer I should mention something important here. The disk scan speed is really not our top priority. While we have a very fast surface scanner (as is advertised in the Features tab), it is not always running at full speed.

     

    Because we're running in a multitasking environment, our top priority is not to interfere with other applications using the disk.

     

    You  can override this behavior under the Throttling tab in  Scanner Settings and have faster scan times at the expense of system slowdown, but I don't recommend it.

     

    In short, the StableBit Scanner is tuned towards not interfering with any existing disk access more than getting the absolute fastest scan time possible.

  15. Hi Saiyan, I'm the developer.

     

    The Scanner never writes to SSDs while performing a surface scan and therefore does not in any way impact the lifespan of the SSD.

     

    However, SSDs do benefit from full disk surface scans, just like spinning hard drive, in that the surface scan will bring the drive's attention to any latent sectors that may become unreadable in the future. The Scanner's disk surface scan will force your SSD to remap the damaged sectors before the data becomes unreadable.

     

    In short, there is no negative side effect to running the Scanner on SSDs, but there is a positive one.

     

    Please let me know if you need more information.

  16. What does concern me somewhat is the remnants of the Client Computer Backups folder that were left on the "removed" drive. If these files are missing from the pool, or if I re-copy them to the pool, whether they are already extant there or not, will I hose my backups? I'm in no big hurry, as the build I was going to put the now-spare 1TB drive into will be a Christmas gift.

     

    JMarsh,

     

    After a successful drive removal, DrivePool does not leave any of your files only in the removed PoolPart and not on the pool. As you've mentioned this would mean that something like a backup database would become useless or perhaps corrupt. In fact, while a drive removal is taking place, DrivePool makes your entire pool read-only just to safeguard your data from something like that happening.

     

    But there are a few cases where your files will remain on the PoolPart:

    • If there is a mismatch in the file times or the file sizes between a file in a PoolPart and the same file on the pool. For example, if a duplicated file is in a conflicted state, we don't want to "merge" it, so we keep both parts around, one on the pool and one on the PoolPart being removed.
    • If the file is currently in use on the pool part being removed. Normally, this is taken care of by the restart manager API. But if the Windows restart manager is not functioning properly then you can end up with some files left behind.
    • If a folder is in use, it will not be deleted from the PoolPart being removed. This is because the restart manager does not handle in-use folders. This is typical if you're removing a pool part that has a shared folder in it. In this case, you would end up with an empty folder in a removed pool part.

    The actual reason for any files getting left behind is written to the log file. The logs are available in C:\ProgramData\StableBit DrivePool\Service\Logs\Service. If you could send me the logs, I'll take a look at whether what you're seeing is normal or something that needs to be fixed.

     

    Ultimately I would like to get rid of all of the above cases, because it's very confusing and a bit unsettling to see files left in a PoolPart folder after drive removal.

     

    There is one exception to what I've said above. If you've specified the removal option "Force damaged drive removal (unreadable files will be left on the disk)", then any unreadable files will be left on the pool part, because that's what you've asked for specifically.

  17. Shane,

     

    Very interesting.

     

    I was thinking of doing it like this:

    • Internally the system would associate a standard path pattern with a set of pool parts.
    • For example:
      • \Sketches\2012\* ->Disk 0,1,2
      • \Sketches\2013\* -> Disk 2,3,4

    And you would set up an unlimited amount of these patterns to configure any kind of folder placement strategy that you want. In addition, every rule would have a maximum fill limit (e.g. 90%), so that if you had to copy more files into a given folder than the set of disks (that store files for that folder) can contain, the rule would be violated and your "overflowing" files would be placed onto other disks. This is very similar to how our existing balancing system works.

     

    Hmm... I'll think about if your suggestion can be implemented with my patterns scheme.

  18. Alex, Thanks for the response. Can it be setup to do a Direct I/O request instead of using WMI? I notice that there's a setting in the Scanner.Service.config file. In the meantime I will do a drive swap to freeze the UI and send you a dump of the Scanner service.

     

    Yes, for some data.

     

    You can set Smart_NoWmi to True (service .config setting) to never use WMI for SMART data. But the Scanner always uses WMI to enumerate the list of available disks.

     

    I see your Scanner service dump and I'll take a look at it now to see why it's locked up.

  19. Yes, the bandwidth to upload/download from the cloud is seldom mentioned, the promoters seem to want the public to forget about the cost of that. Example, tablet users with "free" storage space on the web, using cellular connection with limited bandwidth.

    The company I work at is looking at updating our SAN storage, what cost us $600K seven years ago cost less than $300K with greatly increased bandwidth and much better software [tiered] with SSDs on the front.

    So yes, I agree storage is making great strides, but the bottleneck will be for the foreseeable future the connection from home to remote storage. Around dinner time in my neighborhood eveything slows to a crawl as the drones connect to netflix.

     

    Secure cloud storage has been foremost on my mind. I've been thinking about different options that would offer practical and affordable cloud storage. There will be more coming from Covecube regarding cloud storage stay tuned, the wheels are already in motion.

  20. Tayub,

     

    Normally the submission should take a few minutes and no one outside of Covecube will be able to see your SMART data. I'm not sure why it took a few days.

     

    I typically go through the "unreconciled" SMART data every month or so in order to improve our SMART interpretation capability.

     

    Regards,

  21. Because it's kind of wordy.... you mean to be able to control what files go to which disk in the pool, basically, correct?

    If so, I know that I would definitely be interested, I'm sure there are others that definitely would be as well.

     

    I can be a bit wordy by my very nature. But yes, that's exactly what I'm talking about, controlling which files go onto which pool part.

     

    And, in the future, a pool part may not necessarily represent a single local physical disk, which would make this even more interesting :)

×
×
  • Create New...