ikon

Yay, the scan of the new 6TB HGST is complete — no errors found. Back to removing the 4TB from the OffSite pool. It took a while, but the plans are back on track

That sounds like a really long time for a software program. I have to clarify about the backups. These are pretty much all data copying CMD files that my father set up to run something called Robocopy. Any data that's placed in the MainStorage area of the server is copied to a NearLine external enclosure, then to an OffSite enclosure, then just the Documents folder is copied to a couple of pooled drives that he labelled MyDocumentsReserve. I have no idea why it's called that (but I have no reason to change it). There are 2 sets of hard drives that rotate through the OffSite enclosure. Somehow his routine figures out which set of OffSite drives are currently in the enclosure. My father rotated them every day. I'm not that religious I do it about once a week, or when there have been significant changes. The final piece is Task Scheduler. I had to learn how this works in order to set up the new Windows 10 Server. I'll be honest; it's a bit of a beast to wrap my brain around. The concept I get; the way they've implemented it, not as much. I figured out enough to get new tasks set up to do the backups, so yeah....

I'm thinking Mr. Gibson is taking too long. And I do like that Scanner works while Windows is running, and works in the background. I now have it set to scan between 5am and 10am. The backups my father set up run between midnight and 5am, although I will have to check if they really need that much time. If not, I may be able to give more time to Scanner. So, yeah, push on with that update

Today I decided to temporarily abort the drive removal. I recall my father telling me he never put a new hard drive into service without first doing a surface scan on it. He used to use spinrite, but I remember him complaining something about spinrite not working on larger drives. In any case, I decided to do the next best thing: I'm now using Scanner to do a surface scan of the new 6TB drive. It does have about 1.9TB of data already, but there's still a ton to put on it before the 4TB drive can be removed. Anyway, it's 4% done and no problems so far.

Late this afternoon the 6TB HGST hard drive I ordered on sale from NewEgg arrived. I put it into my OffSite Lian-Li enclosure, added it to the OffSite A pool, and am in the process of removing the 4TB drive that was part of the pool. It's going slowly, but seems to be working. The 4TB drive that's coming out will then replace a 2 or 3 TB drive in the NearLine pool, giving both pools another terabyte or 2 of room. Hopefully, with Christmas sales coming up, I will be able to get another couple of 6TB drives that I can use to replace the 4x2TB drives that make up most of the MainStorage pool (that's the one that's showing the sector errors). That, in turn, will free up another 4TB drive that can be used to expand the OffSite B pool. When it's all done, every pool should have no less that 1TB of free space, and most should have 2TB. That will last me a while.

Well, I ran chkdsk /r /scan /perf again, and this time it got to a Stage 5. I didn't see it do that last time. Anyway, apparently Stage 4 checks only sectors that are currently being used by files and Stage 5 checks the ones that are currently unused. Stage 5 found 3 more areas (again, remember, this is over 4 drives, not just 1). It said it was flagging the areas for later repair. After it finally finished I ran chkdsk /spotfix. This took overnight to complete (well, I got tired of waiting for it to finish so I went to bed). It did complete though. I then went to Scanner and tried a Surface Scan. It sill said there are bad sectors and flagged the drive as Damaged. So, I went into the options for that drive and told it to Not do Surface Scans. At least, that way, I can still use the drive without Scanner preventing me from writing files. To be honest, despite what Scanner says, I haven't noticed any issues with the drive. And, since chkdsk has done its own scan of all the sectors, I'm hoping I'm fairly safe. And there's always the fact that I have quite a lot of backup. Between them all, I figure I'm not too bad off. Oh, and I did something today that I think is pretty geeky: I set up OpenVPN on my (Dad's) Untangle box, something even he hadn't done. I had some missteps, and had to get help from the Untangle forums, but I was able connect up my Surface tablet to my phone's Internet Sharing and then use Windows Explorer to get to the files on my new Windows 10 server. And, thanks to an article by a guy named pcdoc, I found out what Full Tunnel means and got it to work. Now, when I'm away from home, I can surf the net as safely as if I was still at home. Pretty cool. Sorry, I know this is off topic: I hope that's OK.

I agree. My father told me years ago that he, "has 4 to 5 copies of everything". He also said a number of people on the Homeservershow forums asked him to post his CMD files so they could implement something similar for themselves. Right now I'm reading Terry Walsh's book, Building a Windows 10 Home Server. It covers something called File History, so I'm going to look at that. What's a VM? Oh wait, I just looked it up... Virtual Machine, is that right? If so, it's kinda weird, running a computer inside a computer, like something ominous from science fiction. Anyway, I just ran chkdsk /spotfix and it says it fixed the 2 problems it found on the drive(s). I'm running chkdsk /r /scan /perf on it again, to see if it finds any more.

I get what you're saying, I think. However, maybe because of the drive, or maybe because it's in a pool, every time Scanner does a surface scan of this one drive, it finds these same bad sectors and lists the drive as Damaged. It doesn't seem to be triggering the drive to reallocate the sectors. One problem, now that I think of it (and which I just figured out), may be that the drive is actually 4x2TB drives configured as RAID 0 on a 3ware card. I didn't set it up this way; my father did. But, maybe, that's making it extra hard for the firmware to be triggered. I wonder if it's possible to have the drives connected to the 3ware card without having them in any kind of raid (I don't have enough SATA ports on the mother board otherwise). Maybe they would show up to Windows, and Scanner, as individual drives? Maybe? Right now, I'm running chkdsk /r /scan /perf on the drive (it only took me nearly all day to figure out that these seem to be the best options to use, I hope). Anyhoo, it got through Stages 1 thru 3 with zero issues. It's now doing "Stage 4: Looking for bad clusters in user file data". So far, it's found 2 files it doesn't like, and they're now "queued for offline repair". I'm hoping that the repair will truly trigger the drive(s) to reallocate the bad sectors. Got my fingers crossed. One of the files it found is a Vista update ISO so...... who cares The other is a family photo from 2011. I'm not sure which one, but I should have backups, thanks to my father's VERY robust backup strategy (thanks Dad).

I don't really think I would want to try to re-evaluate suspect sectors. I would rather just have them marked bad and move on. There's no doubt that, over time, all drives will get worse. I don't need to know much about drives to say that. All I need to know is that all magnetic materials eventually lose their power. Anyone who's had a magnet laying around for several years should notice that it isn't quite as powerful after a few years as it was in the beginning. When we're talking about electroplated disk platters, where the thickness of the magnetic material will be very thin, it seems pretty obvious, at least to me, that weak spots will develop over time. That said, I think a drive could still last for quite a few years after the first weak spots appear. I think drives would last even longer if the information on the platters is reinforced every so often. Nothing helps a magnet last longer than having its magnetism reinforced from time to time. Can Scanner rewrite the sectors it reads? Is there a mode wherein it does that? If not, it might be a nice feature to add. In the meantime, if Scanner doesn't do rewrites, does anyone know of a Windows app that does?

So, it's been quite a day. I've read more techie stuff today than I can remember doing in a long time. I was trying to find out what I could do about the 8 bad sectors that Scanner found in my main storage pool. Now, I need to mention that those 8 sectors are spread out over 4x2TB drives, so it's really only 2 sectors per drive (on average obviously; it is possible all 8 are on just 1 drive of the 4, but anyway...). I needed to find a way of locking out, or reallocating, or whatever, those 8 sectors, so I could continue to use the pool. So my hunt began.... In one forum I read about "badblocks". Apparently, this is a program that runs in Linux. The post also mentioned a Knoppix Live CD. I had to look up what that means. Basically, it sounds to me like it's an operating system that doesn't have to be installed; you can just carry it around and use it on different computers. So, I downloaded a version of Knoppix, after spending almost an hour trying to figure out which one to get. Man, do Linux people make it hard to understand anything: they make it all so complicated. Anyway, I was able to create a CD of Knoppix and boot up my father's old test computer with it. Then, I had to try to find out how to use badblocks. Well, that sure wasn't easy. Actually, I never did get it running, but more about that in a minute. What I found out is that you have to know nitty gritty details about a hard drive before you can run badblocks on it. Turns out there's a desktop System Info app I can run to get basic info about drives, so I ran it. By the way, I noticed some really odd behaviours with the desktop. When you move a window, it distorts and wavers like it's being deformed. When you close a window, it breaks apart like a tiled wall being blown up. And there was other stuff. I couldn't help wondering if it was programmed by 12 year olds who thought gimmicks like these are somehow 'cool' (they're not). I don't know if these 'features' can be changed or turned off, but the fact that they seem to be the default actions says tons about their creators, and not in a good way. These guys probably think Rocky Horror is a current release movie OK, so the System Info app still didn't give me the blocksize of the drive Apparently, I have to run yet another command in order to find that info. While searching that command (I honestly forget what it's called) I discovered that what badblocks really does is find bad sectors on a drive and make note of them in a file that the system can then use to avoid using those sectors. Sooooo, badblocks is of no use to me anyway, cause I won't be running Knoppix. There's 3 hours I won't get back... Back to the drawing board. Wow, this is so much fun Courtney has mentioned chkdsk in a couple of posts. I hadn't seen it being run since the early 90s, when my father and I would use DOS computers (he set them up thank goodness) so we could play Duke Nukem. He would run chkdsk sometimes on those old machines, but he also said it was only minimally useful (that was my cue to erase any knowledge of it from my mind). Turns out, the chkdsk he used to run, and the one that comes with Windows 10, are very different beasts. The new one, from what I understand of the stuff I read, is way more powerful, and effective, than that old one. No wonder Courtney mentioned it (thanks, by the way). Not only that, but it's enormously easier to understand and use than those Knoppix programs. So, after figuring out how to run a command window in administrative mode (a little obscure, but not nearly so much as Linux), I ran chkdsk with a few of the parameters I found in a Technet article about chkdsk (it was a great article; straightforward and pretty easy to understand, even for me). As I understand it, it will map out the bad sectors in the Windows file system, much like badblocks does in Linux, but it will be useful to me because it's in Windows. I can also see now why Scanner doesn't (yet) have the ability to map out bad sectors: it appears that Windows itself is doing that work. Now I'm wondering if what Scanner should do is find bad sectors and then pass them over to chkdsk for resolution. I read about a "/spotfix" option for chkdsk that can 'fix' in seconds bad sectors that are in some kind of list or file. If Scanner could create, or update, that list or file...... Of course, it would only work for Windows 8 and 10 systems, but that should be most systems, right?

If I'm understanding correctly, and that's a big if, the way you would get bad sectors reallocated is to, repeatedly, try to write to the bad sector, eventually causing the drive's firmware to go, "oh, I see that's a bad sector.... here, I'll give you a good one to use instead". Is that about it?

OK, this is an area I actually do know something about, probably because it's not really an area of tech. For about 5 years, I managed a facility that manufactured circuit boards; not the finished boards with components, but the bare boards, ready to have components added. This industry really isn't techie. It's about machining and electroplating. The raw material, fiber glass cloth covered with epoxy resin and copper sheeting on one or both sides, has to be drilled, punched, routed, and sheared to get it to the final shape. In the case of multi-layer boards, the epoxy-cloth layers are alternated with copper much like a layer cake. This allows for circuits through the middle of the board. To get the circuits, the boards are masked in the pattern of the desired circuits, then electroplated with copper and solder. If the design calls for it, connection fingers are stripped of solder, then plated with nickel and gold. I'm sure you've all seen them on add-in cards for PCs. The extra copper sheeting is etched off the boards, leaving copper and solder circuits on top of bare epoxy resin. Here's the real point: electroplating isn't an entirely exact 'science'. Variations in the flow of current throughout the electroplating solution in the tanks leads to variations in the deposition of material on the target (in my case the targets were circuit boards). These days, all hard disk platters are electroplated. They, too, suffer from slight variations in the deposition of, in this case, magnetic material. These variations, over time, can lead to one area of a platter becoming non-responsive to magnetic impulses while the rest of the platter responds just fine. In other words, it's possible for one of more areas of a platter to become 'unreadable', or unable to take or hold a magnetic 'charge', while the vast majority of the platter remains fully functional. If you get a few bad sectors, I don't think it's necessarily indicative of a failing drive. I think it's more indicative of variations in the plated material revealing themselves. That said, if a drive starts exhibiting many bad sectors, especially if there seems to be a cascade of them, then it could be a situation where nearly all the magnetic material on the platter surface has grown 'tired', and is no longer able to take and hold magnetic charges like it used to. Time to send that drive to platter heaven As for the remapping feature, I really look forward to that. I would like to humbly suggest that the user be given 4 choices: try to recover data from bad sectors; just remap bad sectors; don't try to recover; try to recover data from bad sectors, but only remap the unrecoverable ones; try to recover data from bad sectors, then remap both the recoverable and unrecoverable ones. I would love to see these choices available in both manual and background modes. In other words, as a 'popup' window when interacting with the Scanner console? and as an option in the Scanner Settings (so they apply when the user isn't interacting with the console). Oh, and thanks for the tips on Burst Mode. I will definitely give them a try, especially on the 2 Lian-Li enclosures.

Thanks for that tip. My two Lian-Li EX-503 enclosures do indeed have activity LEDs. I tried using the Ping test to identify a drive, but it seemed to do nothing at all. My one disappointment with Scanner so far is that it won't lock out bad sectors. I have to admit, I find that very baffling. It seems like Scanner is already doing all the heavy lifting in finding and identifying bad sectors (and relocating recovered files?). Is it really that hard to lock them out? If it did that, my feeling, from a newbie point of view, is that Scanner could become much more usable, and popular, with the non-techie crowd. Imagine being able to tell people, "Hey, we've got this awesome little program that will sit in the backgound on your computers and keep your hard drive(s) in good shape. If it ever finds a problem it can't fix, it will notify you by email, so you can take action before you lose all your data". Basically, it would be like putting disk/file checking on auto pilot: non-techies LOVE things on auto pilot. By the way, if Scanner won't lock out bad sectors, how is someone supposed to do it? You mentioned something about the drive itself doing it. How does that work? And can Scanner be used to trigger it? UPDATE: ooh, I can edit my posts...nice. And, the Burst test works great (once I figured out I have to click Start (and Stop if I don't want the test to go on forever )

Yeah, not being able to Remove drives has happened to me at least twice. It so happens that it has always happened to pools located in my two Lian-Li EX-503 5-drive eSATA/USB3 enclosures. These units use a JMicron controller. I wonder if that's somehow relevant. As for the poolpart folders, indeed, I always do a Shift-Delete to ensure they're gone. I'm no guru, but I am learning, bit by bit.

I would like to humbly request that the known issues post in Nuts and Bolts be updated. For example, which is the latest stable Beta release? Thx.

I've just started using Scanner on two 5-bay eSATA/USB3 drive enclosures I inherited. They're great, but it can be a little hard to match up the drive model numbers displayed by Scanner with the drive letters and names in Windows Explorer. Fortunately, I happened to notice that it's possible to find out letter used for a particular drive by checking the Drive Details for that drive (the drive letter is shown a few lines down from the top of the window). Then, by checking the drive's label in Windows Explorer, it's possible to go into Drive Settings in Scanner and change the drive's name. I add the drive's label and drive letter just before the model number that's the default name Scanner assigns to each drive. So, if I have 4 drives that are part of my MainStorage pool, I might change the names in Disk Settings to something like: MainStorage E: (WDC WD10EAVS-00D7B1) MainStorage F: (WDC WD10EAVS-00D7B1) MainStorage G: (WDC WD10EADS-00L5B1) MainStorage H: (WDC WD10EADS-00L5B1) This helps keep all the MainStorage drives together in the list, sorted by drive letter. I guess my next chore is to put a physical label on each drive, so I can quickly pick out one when needed. ps. I'm so impressed with Scanner I just bought a 2nd copy. Now I have it running on my server and desktop (does it ever run fast on my desktop's SSD boot drive ) Oh, and the 2nd copy was HALF PRICE! Awesome!

OK, first, just in case any one here knows of Ikon, let me say, I'm not him. Ikon was my father. I say 'was' because he passed away in 2012. I inherited all his tech, including his forum and social media accounts, software licences, etc. Anyway, I am not the techie he was but, for the past 5 years, I've been learning. His notes, emails, forum posts, and just things we talked about over the years have been invaluable in getting me more up to speed. That said, in the past couple of days I've picked up a couple of things about DrivePool that I thought might be helpful to another newbie at some point. I had a 3-drive pool set up, and it was working fine until Scanner found some unreadable sectors on 1 of the drives and labelled it as Damaged. I tried Removing the drive from the pool, but it wouldn't work. The pool's whole page was kind of grayed out: I couldn't do anything on it. I shut down the computer, removed the 'damaged' drive, then booted up. DrivePool said the drive was missing (yup, it was; good of you to notice), but I still couldn't Remove it. I moved all of the data folders from the PoolPartGUID folder on each of the 3 drives to the root of each drive. Then I deleted the PoolPartGUID folders.... all except for one that simply would not delete. I got an error every time. This particular PoolPartGUID folder had a .covefs folder in it. It always seems to be that specific PoolPartGUID folder that won't delete. I finally got around this issue by getting Properties on the PoolPartGUID folder, going to Security, then Advanced, and giving Ownership of the folder to my account. After messing around with it a few times I was able to delete the folder. However, deleting the folder didn't completely solve the problem. Even though none of the drives had a PoolPartGUID folder on it any longer, DrivePool refused to list the drives in the list of non-pooled drives. Somehow, it thought the drives were still part of a pool, even though it no longer would display a page for that pool. I could have gotten around the issue by use Disk Manager to delete & recreate the partitions from all the drives, but I had almost 4TB of data on them. That's a lot of recopying. It was a Catch-22. Thanks to this forum, and Courtney (who is the most awesome tech support rep I've ever dealt with), I found out about the Reset All Settings feature, although it took me a couple of minutes to find it (it's in the Troubleshooting section in the tiny gear icon in the top right area of the DrivePool window). After resetting the settings, the 3 drives showed up in the list of non-pooled drives, and I was able to build a new pool with them. Many thanks to the posters in this forum, and many, many, many thanks to Courtney. Stablebit/CoveCube is very lucky to have him.

I've disabled Network I/O Boost again. As I said, it hasn't really seemed to make much of a difference, but we'll see. I will try the logging feature, probably tomorrow, after the server has had a chance to settle down. re: the memory change: I have 4GB of RAM. Would that qualify do you think?

Environment 4x2TB WD RE4 HDD on a 3ware 9650SE-4LPML card (no RAID arrays) Intel Pro/1000 CT NIC Windows Home Server 2011 DrivePool v2.1.0.552 BETA (Read Striping and Network I/O Boost enabled) I had these same 4 drives attached to the same 3ware card, on the same motherboard in a RAID5 setup previously. In that setup, access over the network showed no particular delay or lag before presenting a list of files & folders in a Shared Folder. Since installing DrivePool, I've noticed a delay or lag that can sometimes be as long as 10 to 30 seconds before the directory list appears. The effect is exactly what I would expect if DrivePool had to re-enumerate the directory list from scratch before sending it to the client computer. There are 3 things that make it particularly seem like it's an enumeration issue: once I do get a list of files & directories at the top level of a Shared Folder, if I click on a level 2 directory I get another delay, exactly as if that directory is being enumerated; the length of time it takes to bring up a list of files and directories seems to vary with the number files and sub-directories contained in the directory; once I have drilled down on a directory, subsequent accesses to that directory are very quick - back up to what I had before I installed DrivePool. However, if I leave the system alone for a few hours, there will again be a delay or lag in accessing the Shared Folders. In addition, there are some other things I've noticed: if I access the directories in a Remote Desktop session on the server itself there doesn't seem to be any appreciable delay or lag - it only appears on the networked clients; there doesn't seem to be a lot of difference having Read Striping or Network I/O Boost Enabled. I think Network I/O Boost gave a little performance boost, but it's hard to be certain. So I'm wondering if there's anything I can do to get rid of the delay or lag other than getting rid of that pool and going back to a RAID array.

It's WHS2011 with DP 2.0.0.420. Yeah, the drives are still present individually. I can access them via Windows Explorer. But, often, I can't access the DP drive letter from Windows Explorer that incorporates those drives. Unfortunately, this is my OffSite backup, so I can't do as much experimenting as I would normally do, so I think trying to duplicate the problem is probably a non-starter. I may be able to do something along this line with different disks; but that's the rub, they would be different disks. Given all the issues using DP to pool these drives, I've wiped them out again by Cleaning them with Diskpart and set my EX-503 enclosure to Span the disks together. So, they are no longer pooled using DP. I'm doing yet another RoboCopy of the data over from my NearLine storage to the OffSite to see if I run into any of the same issues. If not, I'm going to have to assume there is some issue with DP.

I did try that. It worked up to a point. I, too, had issues with the $recyclebin and System Volume Information folders. Rather than mess around with it, I went into Disk Mangler and deleted the volumes from all 3 drives in the pool, rebooted, then recreated the pool. All went well except I now have an issue I've never had with DP (admittedly, I'm a DP noob): on the newly created (recreated??) pool, I'm getting disks dropping out -- DP is saying the Disk is missing. I'm trying to track down what's going on. Also, when I try to copy a file to the recreated pool I get an error that there isn't enough disk space. Getting the Properties of the drive using Windows Explorer shows over 2TB of space available. DP Manager also shows tons of space available.

As the question implies, I would like to know if the process of creating a drive pool can be easily reversed. DP does a great job of letting you create a pool and add drives to it. What if I just want to reverse that process? I know I can remove drives from a pool, but I don't think that's the same thing. If I remove a drive, DP has to go through all kinds of rigmarole to move the data to the drives remaining in the pool. What about just moving the files & folders to where they were on the same drive before the pool was created and deleting the DP related files & folders? I would think this could be done a lot more quickly than removing a drive. Possible? Does deactivating a licence do this? Alternatively, is there a way to simply delete a pool? What about shutting down Stablebit DrivePool Service and reformatting the drives that are in the pool? Maybe I could just move the files in the hidden folder up to the root and delete the pool related files manually?