Alex

I've looked at the dump, and it's clear that the Virtual Disk Service has stopped responding. VDS is a core component of Windows that enumerates information about all of the disks in the system. In particular the following call is stuck: http://msdn.microsoft.com/en-us/library/windows/desktop/aa383021(v=vs.85).aspx As a result, everything else eventually locks up as well. I'm not sure what I can do about this in my code. I can probably at some complexity put the whole update code in a new thread and then kill the entire thread if the VDS calls don't come back within some reasonable amount of time, but this is really counterproductive because nothing will be able to enumerate new disks in the system, which would make the Scanner quite useless. Here's a little test utility that I wrote some time ago to enumerate disks using VDS. It will simply enumerate all the disks and print out the enumeration data to the screen. It will also listed for disk update events and print them to the console as they come in. You may want to try running it when the service locks up to confirm that it is indeed VDS. The utility should lock up as well. Download: http://dl.covecube.com/VdsTest/VdsTest.exe

As the developer I should mention something important here. The disk scan speed is really not our top priority. While we have a very fast surface scanner (as is advertised in the Features tab), it is not always running at full speed. Because we're running in a multitasking environment, our top priority is not to interfere with other applications using the disk. You can override this behavior under the Throttling tab in Scanner Settings and have faster scan times at the expense of system slowdown, but I don't recommend it. In short, the StableBit Scanner is tuned towards not interfering with any existing disk access more than getting the absolute fastest scan time possible.

Hi Saiyan, I'm the developer. The Scanner never writes to SSDs while performing a surface scan and therefore does not in any way impact the lifespan of the SSD. However, SSDs do benefit from full disk surface scans, just like spinning hard drive, in that the surface scan will bring the drive's attention to any latent sectors that may become unreadable in the future. The Scanner's disk surface scan will force your SSD to remap the damaged sectors before the data becomes unreadable. In short, there is no negative side effect to running the Scanner on SSDs, but there is a positive one. Please let me know if you need more information.

JMarsh, After a successful drive removal, DrivePool does not leave any of your files only in the removed PoolPart and not on the pool. As you've mentioned this would mean that something like a backup database would become useless or perhaps corrupt. In fact, while a drive removal is taking place, DrivePool makes your entire pool read-only just to safeguard your data from something like that happening. But there are a few cases where your files will remain on the PoolPart: If there is a mismatch in the file times or the file sizes between a file in a PoolPart and the same file on the pool. For example, if a duplicated file is in a conflicted state, we don't want to "merge" it, so we keep both parts around, one on the pool and one on the PoolPart being removed. If the file is currently in use on the pool part being removed. Normally, this is taken care of by the restart manager API. But if the Windows restart manager is not functioning properly then you can end up with some files left behind. If a folder is in use, it will not be deleted from the PoolPart being removed. This is because the restart manager does not handle in-use folders. This is typical if you're removing a pool part that has a shared folder in it. In this case, you would end up with an empty folder in a removed pool part. The actual reason for any files getting left behind is written to the log file. The logs are available in C:\ProgramData\StableBit DrivePool\Service\Logs\Service. If you could send me the logs, I'll take a look at whether what you're seeing is normal or something that needs to be fixed. Ultimately I would like to get rid of all of the above cases, because it's very confusing and a bit unsettling to see files left in a PoolPart folder after drive removal. There is one exception to what I've said above. If you've specified the removal option "Force damaged drive removal (unreadable files will be left on the disk)", then any unreadable files will be left on the pool part, because that's what you've asked for specifically.

Oh no... It seems that you've taken a 64-bit dump of a 32-bit process (the Scanner is a 32-bit process). Unfortunately I can't open such a dump, it's simply not possible. You will need to launch a 32-bit task manager in order to take a dump of the Scanner service: C:\Windows\SysWOW64\taskmgr.exe I'm sorry about that (sigh). I wish we could make this process simpler.

Yes, for some data. You can set Smart_NoWmi to True (service .config setting) to never use WMI for SMART data. But the Scanner always uses WMI to enumerate the list of available disks. I see your Scanner service dump and I'll take a look at it now to see why it's locked up.

Tayub, Normally the submission should take a few minutes and no one outside of Covecube will be able to see your SMART data. I'm not sure why it took a few days. I typically go through the "unreconciled" SMART data every month or so in order to improve our SMART interpretation capability. Regards,

Just to follow up on this, I've done extensive troubleshooting on this issue with Kihim in a support ticket. It seems like this is caused by the system getting bogged down at certain times leading to a periodic slowdown in the UI. At least that's what I can see from the Scanner's built-in UI performance profiler (accessed by pressing P in a BETA build). I don't see any Scanner UI task running away with the CPU. I've also set up a local test rig with similar specs, 26 disks and a 2Ghz 8-core AMD server, and have run the Scanner UI for hours looking for any slowdowns and I couldn't find any. But I think the bottom line here is that the Scanner uses WPF to render its UI, and WPF does require some more CPU resources than your typical Win32 application. I think that that's really the core issue here. So in the future I would like to offer a "lite" version of the UI for situations where CPU resources are scarce. I imagine that there will be a simple drop down that will let you switch between "standard" and "lite" versions of the UI, letting you run whichever one you want.

Kihim, I've looked at the DrivePool dumps and it's pretty clear that the DrivePool service is waiting for a WMI query to return. The query is intended to synchronize disk metadata between the StableBit Scanner and StableBit DrivePool. It looks like this: Management scope: root\StableBit\Scanner SELECT * FROM Disks WHERE DeviceIndex = "..." I would need to see a memory dump of the Scanner.Service.exe when it is in this locked up state, in order to understand why it has locked up. Perhaps it's for the same reason, a stuck WMI query. The Scanner does query WMI whenever it detects a new disk, which would make sense in your case, since you're swapping disks.

Pal, The problem was probably corrected by chkdsk before the Scanner could get to it. The problem here is that if the Scanner detects damage, and then you (or some other app) runs chkdsk to repair the problem, the Scanner is not notified. This is something that I'm working on improving, although I'm not sure how it can be fixed. In any case, there is no harm in what you did. The Scanner simply ran chkdsk and chkdsk told it that all is ok. As far as chkdsk logs, no, the Scanner doesn't keep a log of that anywhere. But this is not a bad idea. I've added this to the todo list and it will be implemented in the future.

As far as WSS is concerned (WHS 2011, WSS 2012 and R2), 2.0 doesn't fully replace 1.X. We will continue to rev 1.X for time being. In fact, there should be a new version of 1.X coming up in the next few weeks. In short, for now 1.X is targeted at WHS 2011 users (and similar). So what doesn't 2.X support? In particular 2.X doesn't support the following for WSS (which 1.X does): No Dashboard notifications. No WSSX installer. It doesn't understand server backup disks and OS partitions and will allow you to add them to the pool. So in short, 1.X was deeply optimized for the WSS platform, while 2.X was optimized for the standard Windows platform. Yes, with time 2.X will become a 100% replacement for 1.X, but for now we will maintain both versions. ---- As far as what's next for StableBit DrivePool in general, well it's something really exciting, but I can't really say any more than that

You're right, the temperature in DrivePool was not respecting the C/F setting in the Scanner. This has already been fixed in the latest builds of DrivePool 2.X but it requires a new build of the Scanner which have not been built yet. The next build of the Scanner will have the necessary code to make this work. So in short, this will resolve itself with some time and a newer build.

JSox, Hi, I'm the developer. The temperature history is something that the drive maintains itself. Not all drives maintain this history so you won't see it for every drive. This is part of the "SCT" standard and the StableBit Scanner is able to parse that information and display it in a graph. In addition, we can tell the drive to change its sampling interval. regards,

I've been debating how to handle automatic updates while we're in BETA. There are 2 issues I think: Too many updates are annoying. (E.g. Flash) Because this is a BETA there is the potential for a build to have some issues, and pushing that out to everyone might not be such a good idea. So I've come up with a compromise. The BETA automatic updates will not push out every build, but builds that have accumulated some major changes since the last update. Also, I don't want to push the automatic update as soon as the build is out because I want to give our users the chance to submit feedback in case there are problems with that build. Once we go into release final, then every final build will be pushed out at the same time that it's published.

I've looked at the dump that was referencing this thread and this has already been fixed and the fix will be available in the next public build.

Joe, I'll try to add the ability to limit automatic scanning to days of the week or days of the month in a future build. Thanks for the suggestion,

We can set up a remote support session on this if you'd like. I'll log into your server and take a look at the problem. I'll private message you the details in a second.

Good to know that this is resolved. Build 320 resolves a bug that was (unfortunately) introduced in 312 which I think was responsible for this and some other issues that people have been reporting.

If you are looking for a way to backup only the "master" copy of a file and not the duplicated part, then that is not possible. DrivePool has no notion of a master copy vs. a backup copy. Each copy of the same file is exactly identical, and the copies can reside on any pool part. If you want to backup your duplicated pooled files without backing them up twice, you will need to use a backup solution that does deduplication (E.g. the client computer backup engine part of the Windows Home Server 2011 / Windows Server 2012 Essentials). Alternatively you can use a file based backup system to backup directly from the pool (such as SyncBackSE)

JazzMan, Simply put, it is much harder than I initially thought to make a reliable implementation of symbolic links. As far as I can tell, we can make a reliable implementation in the future, it's just going to take some time. Look for this in a future build. Regards,

Thanks for testing it out. My initial implementation in build 281 above was based on the premise that we can reuse the reparse functionality that's already present in NTFS. I've been reading up some more on exactly how this is supposed to work and playing around with some different approaches, it looks like the entire concept of reusing NTFS for this is not going to work. So don't use build 281 I'm going to take the current reparse implementation out and rewrite it from scratch using a different approach. Using this new approach, reparse points (or symbolic links) will appear as regular files or directories on the underlying NTFS volume, but will work like reparse points on the pool. This will also eliminate the burden of accounting for reparse points when duplicating or rebalancing, since they will be regular files on the underlying NTFS volume. Look for that in a future build. I don't think that it will make it into the next build because that one is concentrating on updating the locking and caching model, which is a big change as it is.

Guys, I've added preliminary support for symbolic links (reparse points). It still needs more testing, but here is the internal build with this feature: http://dl.covecube.com/DrivePoolWindows/beta/download/StableBit.DrivePool_2.0.0.281_x64_BETA.exe http://dl.covecube.com/DrivePoolWindows/beta/download/StableBit.DrivePool_2.0.0.281_x86_BETA.exe

Because there is no way to query the disk for its current standby settings, the checkbox defaults to an indeterminate state. You can only set standby. I would suggest that if you would like to set different standby times on a per-disk basis then use the firmware, but if you just want a single standby time for all the disks then use Windows.

This is actually a fairly complicated topic. Let's start by talking about how normal standby works without the StableBit Scanner getting involved. Windows "Put the Disk to Sleep" Feature Normally, Windows will monitor the disk for activity and if there is no disk activity for some preset amount of time it will put the disk to "sleep" by flushing all of the data in the cache to the disk and sending a special standby command to it. At the same time, it will remember that the disk is asleep in case any other application asks. Shortly after the disk goes to sleep, the StableBit Scanner will indicate the fact that the disk is asleep in the Power column. Normally, the Scanner gets the power status of the disk by querying Windows and not the disk. It does not query the disk directly for the power state because Windows considers this power query disk activity and wakes up the disk as a result. Now, things get a bit more complicated if you want to include the on-disk power management in this picture. Disks can optionally support these features, which can put them to sleep without Windows knowing it: Advanced power management. Standby timer Advanced Power Management This is a technology that implements power consumption profiles. For instance, if you don't care about performance but want maximum power savings, then you can tell your disk just that. Simply set the Advanced Power Management to Minimum Power Consumption. Or you can do the exact opposite by setting it to Maximum Performance (which guarantees no standby). With Advanced Power Management you don't concern yourself with "sleep timeouts", like in Windows. You simply state your intent and the disk will adjust various parameters, including the standby time, according to your setting. The implementation of Advanced Power Management is completely up to the manufacturer of the drive, and there are no specifications that explicitly state what each power mode does. This entire feature may not even be supported, depending on the disk model. Standby Timer The Standby timer is more widely supported because it is an older feature. You simply specify after how much disk inactivity you would like the disk to be put to sleep. This is similar to how things work in Windows, except that the low power mode will be initiated by the disk firmware itself. Again, the implementation of this is up to the manufacturer of the drive. StableBit Scanner "Put into Standby" In the StableBit Scanner, you can right click on a disk and put it into standby mode. What this does is send a power down command to the disk. This type of power down is equivalent to what Advanced Power Management or the Standby Timer would do. More importantly, when a disk is powered down in this way, Windows will not be aware that the disk is in a low power state, and will continue to report that the disk is still powered up. This is not an issue because the disk will simply spin up the next time that Windows tries to access it. But this leaves the StableBit Scanner with a dilemma. If we can't query the disk for the power state directly, how do we report the true power state of the disk? What the StableBit Scanner implements is a power state in which it's not sure whether the disk is in standby or active, and this is what you were seeing. Forcing the StableBit Scanner to Query the Power Mode from the Disk If you want to use on-disk power management exclusively, and you don't care about Windows putting your disks to sleep, you can instruct the StableBit Scanner to query the power mode directly from the disk. When this is enabled, you will no longer see the standby or active message, but Windows will never try to put that disk to sleep. That's why this is off by default. SMART And just to make things even more complicated, sometimes a disk will wake up when it's queried for SMART data. To this end the StableBit Scanner implements some more settings to deal with this: I hope that this clears things up.

Parity has some advantages and disadvantages. The short answer is that it doesn't make sense to put parity into the current version of DrivePool. We would need a new product that is a block based pooling solution. This block based DrivePool would have some advantages and disadvantages over the current file based DrivePool. The main disadvantage would be that your files would no longer be stored as standard NTFS files on the physical disk. Overall I feel that there are a lot of other well established block based pooling solutions out there and I'm not sure that we would bring anything unique to the table. So I'm not in favor of doing a "me too" product. In general I'd like to create products that are unique in some way.