gtaus

Yep, it took 25 hours to remove my 4TB HDD. As I stated, I could see the transfer rates going up and down in speed depending on the size of the files. Although speed is important, I have stayed with DrivePool for years because the program just works so well and does not crash like other systems I had including RAID and Windows Storage Spaces.

I don't use duplication on my Home Media Center storage DrivePool. I have all the files backup up on HDDs in the closest. So, if I lose an HDD in DrivePool, I could reload them from the backups. Nothing critical if I lose a drive. As far as time required for removing a drive, I initially estimated 54 hours to remove my drive after the progress in the first 3 hours. However, my transfer rates improved over time and the remove drive task completed last night at about 25 hours total run time. I suspect it's a matter of what files are being transferred. Small files will take considerably longer to transfer than larger files. I can see the transfer rate drop way down on those small files, but build up speed on larger files. Thanks for the offer, but you don't have to perform any comparison tests for me. I am just hoping that DrivePool will someday include an "estimated time to complete" for the remove drive task. The % progress in DrivePool is OK, but it really does not give you any idea how long it will take to complete.

That looks like a real nice 10-bay enclosure. When I started out with DrivePool, I had two 4-bay enclosures and a handful of individual HDD enclosures. Unfortunately, after a few years, one of my 4-bay enclosures died so I lost access to all 4 drives. It was less expensive for me to replace the 4-bay enclosure with 4 separate HDD enclosures, so that is what I did. Currently, I see the 10-bay enclosure sells for about $540 on Amazon. That's $54 per slot. Again, I am comparing that price to individual HDD enclosures at about $20 per case. I know the multi bay enclosures have many advantages, but I wish they could bring down the price. Also, for me, I don't think I would ever lose 10 individual HDD enclosures all at one time whereas I know that if the board in the enclosure goes down, I would lose access to all those drives. Do you think your 10-bay enclosure is moving files among the drives faster than what you could see if you had separate HDDs in individual cases? From your post, I don't see any great speed removing the drive within DrivePool itself. Well, frankly, DrivePool is not nearly as fast as my old RAID system or Wndows Storage Spaces, but DrivePool just works better, and I have had no catastrophic data failures as I did in the other systems. I am currently removing a 4TB HDD from my 23 HDD DrivePool Media Server. I calculate it will take about 54 hours to complete the task within DrivePool. That's a long time, and with my newer 8TB drives, I suppose it will take almost a week to remove one of those. I think it would be great if DrivePool had some kind of way to estimate how long it would take to complete the remove drive task. I see my remove drive task is clocking at about a 40 MB/s rate but I think the HDDs can transfer over 80 MB/s on their own. And I have an older computer with a USB 3.0 system. I'm sure the bottleneck in my system is the USB 3.0. So, just curious on what transfer speed you see on your 10-bay enclosure as you remove your drive. Thanks.

That's the info I was just looking for, how to speed up removing a drive from DrivePool. Currently, I am removing a 4TB drive to replace it with an 8TB drive. I logged the progress of the task and it took 3 hours to move the first 5.5% of the drive. At that rate, it will take about 54 hours to complete the removal task. Good thing I started the removal process 3 days before the new HDD is to be delivered. Any idea on how much overhead DrivePool adds to the process in the drive removal process? With larger and larger HDD's, this is becoming more of an issue for me. I like to keep my DrivePool up and running while I remove a drive, so I plan for a good 3 days to remove a 4TB drive. But it would be nice to know how much longer DrivePool takes to remove a drive compared to stopping DrivePool and manually moving them yourself.

Well, that's not what the people at PrimoCache told me. They made it clear to me that PrimoCache will not cache any network transfers because it uses a different protocol for network file transfers. I also don't see what caching benefit you would achieve using PrimoCache on the server. The bottleneck for speed on network transfers is usually the limited bandwidth speed of the network itself. My home network, for sure. Are there home networks that have data transfer rates higher than a write speed of a SSD? That's the only case where I could see a speed advantage to using RAM as cache with PrimoCache. Otherwise, you might as well write directly to a SSD/HDD over the network. It has been about a year since I last looked at PrimoCache for network transfers, and at that time it was not possible. Maybe they have made some progress on their end because I bet a lot of people would love to speed up the network file transfers, if possible. Caching to an SSD seems like a safer option. File writes would be verified before completing, and would be permanent on the SSD, as opposed to using RAM as cache and the possibility of a power loss with data in volatile RAM being lost forever. Windows 10 will cache some file transfer data in RAM natively. If you had lots of RAM on your computer, then your Windows 10 data file cache would be higher. PrimoCache just allows you to use more RAM as primary cache, and an SSD as secondary cache, to speed up your transfers within your computer. If you find that PrimoCache works to speed up your transfers, either on the server or client end, then that would be great. I can't even test it out anymore because my trial period ended about a year ago and PrimoCache doesn't forget your 30 day trial period. I'm not going to buy a license to try out software again that did not work for me in the past. Anyways, I hope you find a solution to your needs.

I have stopped the DrivePool services, too, but like you, I reboot when starting everything back up. I don't know if there is a shortcut. And by that I mean, I have 23 USB HDDs (82TB) and it takes a few hours to reboot DrivePool, check all the drives, check duplication, etc... before all those housekeeping services are complete. I typically only schedule a reboot for overnight housekeeping, if at all possible. DrivePool is a great system for my home media server, but it's just not lightning fast on reboots and checking itself. Of course, if I only had a few HDDs or few TBs of data to validate, then the process would go much faster.

I think if you modify a file, and then save it, it will save it first on the SDD cache and then flush it to Archive drives later depending on your settings. That's exactly what I wanted to do with PrimoCache on my system. Either running on the server or the client computer. But, alas, it does not work. I contacted PrimoCache support and they verified to me that transferring files over any network, in any direction, is not supported by PrimoCache. They said the transfer protocol over a network is different and PrimoCache does not (yet) support that functionality. Too bad. I would have gladly bought a copy if it did.

I don't use SnapRAID, but there is pretty good video on YouTube at:

I have never really figured out the File Placement system. So, I'm just signing on here to see if anyone has a good response. I understand how DrivePool might split up a movie folder on 2 or more drives, depending on how many files were in that folder. I see that when I upload an album folder and the tracks of the album might be on any number of drives in the pool. If your goal is to have all the files in the movie folder on one drive, then I think there is an option in DrivePool to completely fill up one drive before it starts writing to the next drive. Also, I think you can designate your DrivePool "Movie" folder to only be on certain drive(s) in File Placement. One of those options might get you closer to your goal.

I think I understand what you are doing. You are partioning an HDD into 2 or more volumes and DrivePool does not recognize the new partitions. From what I understand, and I could be wrong here, DrivePool reads the serial number (not the Drive letter) of the HDD and adds a hidden PoolPart folder to that drive. Problem is, if you add additional partitions to that drive, it still only has 1 serial number for that HDD. I ran into this issue thinking I could partition a SSD into 2 volumes and have 2 frontend caches for Duplication files. But no, it will not work that way because DrivePool only allows one hidden PoolPart folder per physical disk.

You can easily hide the DrivePool drives from File Explorer by simply going into Disk Management and removing the Drive letter of that/those drives. DrivePool does not need a drive letter for the HDDs in the pool. I have 23 USB HDDs, without Drive letters, in my DrivePool, but the only thing showing in File Explore is the combined DrivePool volume letter. BONUS IDEA: Instead of assigning Drive letters to your DrivePool drives, you can mount them in a folder on your C: drive. You can still access each and every drive mounted in DrivePool that way in the mount folder and see what it contains, but you won't be clogging up File Explorer with a bunch of drives in the main list. BONUS BONUS IDEA: If you use a disk cataloging program to organize your files and/or conduct searches, the cataloging program will read and catalog each drive mounted in the special folder. That way, if you had a HDD fail in your DrivePool, you can easily go into your cataloging program and find out what was on that specific drive. For my DrivePool home media server without Duplication, it would allow me to see what files were lost on that drive and if I want to reload them from a backup disk in my storage closet.

I am trying to wrap my head around your idea. DrivePool works differently than RAID, but I think I can offer some suggestions. Since DrivePool writes full files to one disk at a time, not packets spread over multiple disks, would not standard duplication in DrivePool give you the same result of having a copy of your data on two separate disks? Essentially, Raid 1+0 is Mirroring with Striping combined. Mirroring for data redundancy, and Striping for speed in a packet writing system. I think you could do something similar in DrivePool using Duplication (Mirroring) on either your entire DrivePool, or just selected important folders, and using 2 SSD's, one each as frontend cache for each duplicate file (Striping) for increased speed. I was able to achieve RAID-like speed on my DrivePool when I added an SSD frontend cache (for WRITES). I also set my DrivePool SSD cache high enough that I can also use it as a very fast READ/WRITE cache for temporary files. I set my SSD cache not to flush until it has 100GB. That's more than I need on any of my temporary working directories. Another big plus with using DrivePool is that you don't have to add additional HDDs in pairs. I recently built a DrivePool with 3 end-of-life HDDs that I cannot trust on my main DrivePool. I set that entire end-of-life DrivePool to 2X Duplication, In theory, if/when any one drive fails, I'll still have another copy of my files on one of the other 2 remaining HDDs. If there is enough capacity on the remaining 2 HDDs on that DrivePool, it will duplicate the files as needed that were on the failed third HDD. I would never consider using those end-of-life HDDs in a RAID system, but DrivePool lets me get more time out of those old drives. That saves me money and was worth the cost of DrivePool itself. Another consideration is that a typical RAID system, with packet writing across disks, leaves you with nothing if a HDD fails - unless you build in duplication like your RAID 1+0 setup. You can't read a failed RAID HDD. DrivePool, on the other hand, can often retrieve many, if not most, files off a failing HDD because it contains the entire file, not mere packets. I have been able to retrieve all but maybe 2 or 3 files off a 4TB HDD that crapped out on me. Unless the HDD is totally dead, you have a decent chance of retrieving files off that failing HDD. Bottom line: To replicate the features of RAID 1+0, use Duplication on your DrivePool for redundancy and add 2 SSDs as frontend cache, one SSD for each duplicate file to be written for RAID-like speed. When you want to expand your setup, in DrivePool you can add one disk at a time instead of having to add disks in pairs.

I now have 23 USB HDDs in my DrivePool. I don't remember at what point I started having problems with DrivePool reconnecting to the USB HDDs, but it's not just DrivePool. Windows Disk Management doesn't connect with all the drives if I try to reconnect all of them at the same time. I am thinking it has something to do with the USB and identifying too many drives at one time. My solution was to buy 7 port USB Hubs with individual power switches for each port. So, I start my computer, and then load each USB drive, one at a time, making sure it shows connection in the DrivePool GUI before I move on to the next USB HDD. Obviously, I hate rebooting the server. But at least I can now get DrivePool to reconnect to the HDDs, albeit one drive at a time. On my system, it takes about 30 seconds to turn on and reconnect to each HDD. Not a very fast process but is much better than before when half my USB HDDs never reconnected. I have one internal SSD as a frontend cache for my DrivePool. When I reboot my computer, it is the only drive found by DrivePool and all the other USB HDDs are missing until I turn them on and load them - one by one - into DrivePool. In your case, if you shut down all the USB HDDs but still had one small SSD/HDD online, you should be able to reconnect all the other USB HDDs without rebooting. I do that all the time with my DrivePool when I swap in/out USB HDDs. Since I always have that one SSD online and DrivePool is always up and running, I can disconnect and reconnect the other USB HDDs without rebooting the computer. Of course, if you have any HDDs listed as missing, DrivePool goes into a READ Only mode and you will not get full functionality READ/WRITE until all HDDs are reconnected. There might be a way to start/stop the DrivePool services without rebooting, but I have not figured it out.

Sounds like the Duplication process is stuck. Have you tried stopping the Duplication process, and then restarting it? If I determine that DrivePool is frozen, sometimes I'll just shut down the computer/server and restart it again. In general, I find the DrivePool does a pretty good job at duplicating the folders in the background, but, like you said, it's not a very fast process. Have you tried opening up Task Manager and checking to see if there is any disk activity? The progress bar in DrivePool does not offer much information on the activity going on in the background. In the past, when I have had to swap out/in a number of HDDs all at the same time, I shut off all duplication, swapped out the HDDs, and then reset duplication for the folders I wanted. Only suggest that if you can survive without having all those files backed up all the time. But it allowed me to complete the swaps, get everything back up and running, and then do the duplication in the background at its own pace. I wonder if the minimal info on the DrivePool task bar that really does not show any activity happening just makes a person think that everything is taking a very long time to complete. I know that transferring large volumes of data from one HDD to another with Windows Explorer can also take forever, but at least it shows you an activity box and the transfer speed. If someone has a true method of speeding up duplication, I, too, would love to hear it.

From what I understand, Balancing only moves files your SSD front end cache into the Archive drives. That's all I see happening. If you are talking about Equalizing data on all the Archive disks, I have never known DrivePool to do that by will. For Equalize, DrivePool writes new data to HDDs that have more capacity. As DrivePool writes new data to the HDDs, it will more or less equalize the data among the pool. But i have never seen it rob one drive to free up space and then move that data to another HDD that has more free space. If anyone is able to get DrivePool to equalize HDDs on demand, I would like to know how.

Well, I never got a response on this issue of losing Drive names and progress on scanning of the drives after a reboot, but I have been reading this topic from posts from a few years ago and maybe it's just a continuing bug in the software. At any rate, I had a number of other issues with Scanner on my 20 USB HDD (82TB) DrivePool volume and ended up uninstalling Scanner after 3 weeks. There are so many things to like about Scanner, and it worked just fine on my smaller 3 USB HDD (10TB) DrivePool volume, but I got massive slowdowns and hanging on my 20 USB HDD (82TB) DrivePool volume to the point where I was unable to use DrivePool as my home media server. I uninstalled Scanner last night, rebooted my server, and everything is back to working normally again.

Uninstalled Scanner last night. Restarted my server. DrivePool is back to working normally, again, at normal speeds and completing all the Measuring, Checking, and Balancing tasks in a matter of a few hours - which is what is takes on my 20 USB HDDs (82TB). I really liked the way Scanner worked on my smaller (3 HDD/10 TB) DrivePool volume, but for some reason all I got were problems with my 20 USB HDD DrivePool volume. Thanks to Stablebit for allowing us a 30 day trial period to see if the program works for our setup. I ended up using Scanner for 3 weeks and had a lot of issues with the program that I was not able to resolve. But it was good to give it a try.

Yeah, I don't know. I have 20 USB HDDs (82TB) in my DrivePool and since I installed Scanner a few weeks ago, I am experiencing a number of issues slowing down my system and hanging. They seem to be related to Scanner and I have not been able to figure out why. So, unfortunately, I am to the point where I will be uninstalling Scanner to see if I can get my DrivePool running again, like it used to run. I have a smaller DrivePool volume with only 3 USB HDDs (10TB) and Scanner seems to run fine on that volume. Maybe the number of USB HDDs in a pool affects the performance of Scanner? Anyway, DrivePool is my home media server and I have not been able to access my files because Scanner appears to be slowing down, or completely hanging, my DrivePool. At least, I know I never had these issues before I installed Scanner. So, I will be uninstalling Scanner to see if I can return DrivePool to its former state.

I installed the trial version of Stablebit Scanner about 3 weeks ago, and my DrivePool volume with 20 USB HDDs (82TB) has basically gone done to a crawl on Measuring and is at a standstill with Balancing. I exited Scanner to see if everything in DrivePool would return to it's previous state, but it still remained extremely slowed down. Then, I noticed that although I had exited Scanner, there were still 3 active tasks for Scanner running in my Task Manager. So, I killed those Scanner tasks and my DrivePool ran better/faster. I have another DrivePool volume with only 3 USB HDDs (10TB) and Scanner seemed to work fine on that volume. Maybe the number of USB HDDs in a pool is a factor with Scanner slowing everything down so much? Anyway, currently my DrivePool with Scanner GUI not running, but with those 3 Scanner tasks showing up in Task Manager, my system got stuck on Balancing at 0.1% for the past several hours. I really don't know what to do other than to uninstall Scanner, and then restart my computer to see if it has cleaned out all those Scanner tasks in the Task Manager. With my 20 USB HDDs (82TB) in DrivePool, I was used to having to wait a couple of hours for Measuring and then maybe another hour for Balancing and Checking Duplication. Since I installed Scanner, those same jobs will run for days if they ever complete, but most often I have seen my computer just hang in the process requiring reboot of the server.

FWIW, if anybody is still interested in this topic, I tried PrimoCache with DrivePool on my home media server, and the transfer speeds did indeed show significant improvement within that computer. However, most of my workflow is from transferring files from client computers into my host server. In that case, PrimoCache has NO effect at all. PrimoCache does not work with network drives, which is how my client computers see the DrivePool volume on my server. So, transferring files from my client computers to DrivePool on my server has absolutely no improvement in transfer rate with PrimoCache. I have found my better option is to use the SSD Optimizer as a frontend cache for DrivePool, which allows me to transfer my files from a client computer into the SSD frontend of DrivePool on the server. Then the SSD will rebalance the files as needed on to the archive HDDs in the background. Of course, file transfers are still limited by your network speed, but having the SSD frontend on DrivePool pick up the files is usually much faster than writing to the archive HDDs directly over the network. I contacted PrimoCache support about this issue, and they verified that PrimoCache does not work with network drives. Since network transfers is the bottleneck in my system, I ended up not purchasing PrimoCache although I do think it's a very good solution for other setups.

I have noticed that some of my HDDs are running hotter than others. The status showing on those drives often say "Throttling..." which I assume will slow down the scan on that drive to prevent it running even hotter. Anything different showing up on the status of the one HDD that is running so much slower than the other HDDs?

I have used CHKDSK /R in the past. It works on some HDDs and can fix some problems. What I am saying is the HD Sentinel was able to fix a failing HDD which CHKDSK /R could not. I don't pretend to know how HD Sentinel fixes the HDD, but it worked for me on some disks that CHKDSK could not fix. Also, HD Sentinel has a number of different ways to fix a HDD. Anyways, if I fix a HDD with HD Sentinel, I now move it to a second DrivePool, with Duplication set on the entire pool with other End-of-Life HDDs. In theory, I should be able to lose a HDD in that pool and still have all my data somewhere else in that pool. Just trying to get a little more mileage out of my old drives that I no longer trust 100%.

I have successfully used HD Sentinel to repair some damaged HDDs. I don't know the exact way it works, but evidently HD Sentinel reads and writes each sector, and if it finds a damaged area, HD Sentinel reallocates that damaged area to a non use mode. It's like a low level format in MS, but seems to work better. You have to purchase a license for HD Sentinel if you want to attempt any HDD repairs, but I can vouch that I saved a few HDDs that way and they were put back into service in DrivePool for many months. Having said that, if I have a drive that needs to be repaired, then I try to replace that drive as fast as I can because that drive will more than likely start to show more problems. Recently, I created a new DrivePool with my repaired and end-of-life HDDs using the Duplication feature on the entire pool. If I lose one of those HDDs in that DrivePool volume, it really won't matter because there is always a copy. In the meantime, my new drives are on my main DrivePool volume where I have my home media files that are unduplicated. Anyway, using duplication on a separate DrivePool volume with end-of-life drives should give me another year or so of use on those drives before they die. DrivePool saves me money and extends the useful life of those old, not so reliable, out of warranty, drives.

FWIW, I had similar problems when I was running MS Storage Spaces. The only way I was able to recover from that read-only mode issue was to create a new volume, transfer all the data to that new volume, delete the old volume. If MS Storage Spaces is telling you which drive is causing the problem, then I would remove that drive from the pool, low level format the HDD in a different computer, and if it cleared up the bad sectors warning, then I would put it back into the MS Storage Spaces pool. I remember a couple of times that I was able to transfer all my files, except maybe 2 or 3 corrupt files, and that was what causing all the problems and putting my MS Storage Spaces volume into read-only mode. Because MS Storage Spaces uses a stripping method, a bad sector on drive can affect the entire volume. DrivePool is much better, IMHO, in handling errors like that. A corrupt file on a HDD in DrivePool only affects that specific file on that specific HDD. Makes data recovery in DrivePool much easier.

I see. Because we are in a DrivePool forum, I immediately read into your info that it was a DrivePool volume that was named Storage Spaces. I ran MS Storage Spaces for about ~7 years on my server before I hit my third major data loss after a single HDD failure in a ~30 HDD pool of drive somewhere about 120TB. I had Parity and Duplication on different MS Storage Spaces volumes, but a single HDD failure wiped out lots of my data and I was not able to recover from it. At the end of my experience with MS Storage Spaces, I was spending most of my time working in Powershell trying to get my system back up and running. I finally gave up using MS Storage Spaces and bought a license for DrivePool, and have never looked back. I have had a few HDDs fail, over time, but in most cases I have been able to transfer the data off the failing HDD before complete failure. In any case, if a HDD fails in DrivePool, you only lose the data on that one failed HDD, not the entire pool as in the MS Storage Spaces volumes. More to your point, I am not sure that StableBit Scanner is the program you need to monitor your individual HDDs in the MS Storage Spaces volume. Since Scanner picked up the entire MS Storage Spaces as a single volume, it is not able to see the individual HDDs in the pool and I cannot imagine it would not be able to read the S.M.A.R.T. data on each drive that it uses to monitor their status. In fact, from the picture you posted, Scanner is already warning you what it cannot do. How long have you been using Scanner on your MS Storage Space volumes? And, has it been working in the past to scan and report problems for you? From what I remember using MS Storage Spaces, none of my programs were able to see the individual HDDs in the MS Storage Spaces pool of drives. MS locks out any prying eyes from seeing what any individual drive is doing. You might want to download the free version of HD Sentinel and see if that program is able to look into the individual HDDs in your MS Storage Spaces pool. I find HD Sentinel has some advantages over StableBit Scanner, and Scanner has other advantages over HD Sentinel. The free version of HD Sentinel usually lets you test drives to determine their health, but if you want to perform any fixes, then you need to purchase a license. I bought a license for HD Sentinel and have "repaired" a few drives, restoring them to health, and have used those drives for another year or so. Anyway, HD Sentinel is free to download and maybe that would be able to check your MS Storage Spaces volume for bad sectors. If not, you have not invested any money into it.