gtaus

Whenever I have problems with duplication, I go to the Settings Cog>Troubleshooting>Recheck Duplication and let DrivePool try to figure it out. Honestly, if there are duplication problems with DrivePool (like after removal of a failed HDD), it takes me a couple times running the Recheck Duplication and Balancing tasks. Last time that happened to me, it literally took a few days for DrivePool to clean itself up, but I have 80TB in my DrivePool. To be fair to DrivePool, it did fix itself given time. I only have a few folders set for duplication in my DrivePool, so out of my 80TB pool, only about 20TB are duplicated. Also, DrivePool duplication is good for some things, but it does not ensure that your files are actually intact and complete. It is possible to have a corrupted file/folder and DrivePool is happy to duplicate the corruption. If there is a mismatch between the original and the copy, DrivePool cannot tell you which file/folder is true and which may have been corrupted. For example, my DrivePool is mainly used as my home media storage. If I have an album folder with 15 tracks, and one or two tracks gets deleted or corrupted, DrivePool cannot tell me if the original directory is complete, if the duplicate directory is complete, or if neither copy is complete. Because of this, I now add 10% .par2 files to my folders for verification and possible rebuild. With the .par2 files, I can quickly determine if the folder is complete, if any missing or corrupted files can be rebuilt from the .par2 files in that folder, or if I have to take out my backup HDDs from the closet to rebuild the corrupted data in DrivePool. Unfortunately, DrivePool duplication does not ensure that your data has not been corrupted. For this reason, I don't consider DrivePool duplication in any way a backup solution. It lacks the ability to verify if the original or the duplicate copy is complete and intact and cannot resolve mismatches between copies. In theory, from what I understand, duplication is mainly good for rebuilding your DrivePool if you have a HDD failure and the bad drive is a complete loss. Then, DrivePool will still have a copy of the files on other drives in DrivePool and can rebuild the failed data. That may be a great option, and of course I said I do use duplication, but DrivePool duplication still lacks any ability to verify if the files are complete and uncorrupted. For that reason, I have gone to using those .par2 files for file verification. My backup HDDs are stored in my closet. It's not the best solution to my backup needs, but it is the best I have found for me at this point. In a more perfect world, DrivePool would have the ability to duplicate folders for faster pool recovery, and there would also be some way to verify and rebuild lost data like the .par2 files. In your case, if you have good backups of your data, I might consider turning off duplication in DrivePool, Rebalancing the pool and/or forcing a Recheck Duplication to clean up the data, and then turning duplication back on for the folders/pool as you want. But before I did that, I think I would contact the programmer directly for support and ask him for his recommendation(s). DrivePool is a great program and data recovery is much better than other methods I have used such as RAID systems and Windows Storage Spaces. But I do run into errors like you are experiencing and I cannot always understand the corrective action to take. Mostly, I have found that DrivePool is able to correct itself with its various troubleshooting tasks, but it might take a long time on a large pool.

Shortly after I switched over to DrivePool, I had a number of old pool drives that started to die on me. I was getting those random corrupt files and then my drive would just die. The drive monitoring programs, like Stablebit Scanner, or Hard Disk Sentinel which I use, only provide so much protection. I replaced all those failing drives, and so far, my problems have gone away. However, some of the lessons I learned during that period may be of use to you. First of all, for as good as DrivePool is, it does not tell you if the file has been corrupted. My first clue to corrupt files is when they failed to read off the drive. Too late by then, which I imagine is what you are dealing with also. Second, although you can set any or all directories to be duplicated, DrivePool itself does not verify if any of the copies are not corrupted. In theory, you could have a corrupted file and DrivePool will just duplicate it. Who needs a copy of a corrupted file? From what I understand, if DrivePool sees a difference in the copies, it will ask you if you want to delete the older versioin. But that may, or may not be, the corrupted file. I know some people use SNAPRaid to provide parity checks on their DrivePool, and that appears to be a step forward in data recovery. But again, if your original file was incomplete or corrupt, you would only be restoring the bad files from the parity checks in SNAPRaid. Considering all of this, I ended up using QuickPar and MultiPar to create 10% par files to verify and recover my directories. It is an added step and takes time, but the advantage is that I am able to verify if my directory files are all intact and complete, and if not, with 10% recovery files, I am able to recover from most problems. If I had any critical files to protect, I would maybe use 50% or more par recovery files. But I mainly use DrivePool as my home media storage center and my files are backed up offline and not very critical if I should lose them in DrivePool. There is an option to create an index only file with MultiPar and QuickPar, and that is only good for verifying if the files are still intact. But the index file has no recovery blocks. QuickPar and MultiPar are good for verification and recovery on a directory level, like maybe to verify all the tracks of the music album are intact, but I have not been able to figure out how/if I could use these programs on a drive or pool level basis. So, it's not my ideal solution. But if works better for me than the other options I have tried.

Well, I did not take it that way, so nothing to apologize for from my perspective. Even though the full format routine might take awhile, I think you will be better off for the effort. From my experience, data loss can sometimes take a lot longer to recover than a few days. Data corruption on a pool can sometimes go undetected for a long time, and repair might be very difficult. So I always take the time upfront for a complete full format with new drives before I add them to my DrivePool. I ended up getting Hard Disk Sentinel for monitoring my drives, but, in general, I also think these monitoring programs mainly read drives and look for problems that way. Having said that, I did buy the Pro version of Hard Disk Sentinel and it has testing programs that read and write to the drive. Sometimes you are able to put a "failed" drive back into service after you run a program to "fix" the hard drive. In that case, I think it hides weak sectors from being used and resets the drive less those failed sectors. I was able to put one drive back into use after it passed that test and recovery routine, so I figure Hard Disk Sentinel already paid for itself on that one drive recovery. Most of my drives that go bad are not able to be recovered. I guess it depends on what is going bad with the drive and I won't pretend to know more than that. If I get a report that a drive is starting to fail, I just replace it if the Hard Disk Sentinel testing routines fail the drive.

There are many options to the balancing settings. It took me a while to figure out what works best for me. Once I got to my happy place, I have not changed it. Sounds like you dialed it in faster than I did. Just wanted to mention that when I was changing DrivePool settings, sometimes I would get unexpected results. I found that if I restarted my computer, DrivePool would come up fresh and things worked as expected.

OK. Hope you get your answer on the forum. If not, maybe a support ticket....

Hello SCB, welcome to the forum. If your HDD has never been formatted, it is still recommended to do a full format on a new drive. That way the OS will check the HDD for any bad sectors and mark them not to be used. A quick format should be good enough for a HDD that was formerly fully formatted. The quick format does not check for bad sectors, so if you have a new drive with only a quick format to save time, you could end up sending data to bad sectors. Bottom line, IMHO, it is far better to use the full format on a new drive even though it may take days. It would be far worse to find corrupted files on the HDD at a later date and sometimes those errors can cause the drive to crash. Just a friendly suggestion, it would be better to open a new question/thread on the forum for your question(s) then to tag onto an old thread that ended 6 years ago. Even if the question has been asked before, the solution could have dramatically evolved over the years. Again, welcome to the forum and hope to hear more from you in the future.

I had a similar problem on my computers. Everything was working fine with the remote control for months on my computers. I upgraded DrivePool on my main server (with license), and then none of the remote controls (without license) worked on my other computers. In my case, I was able to update DrivePool on my remote computers and once all computers had the same DrivePool version numbers, they worked again. If you have additional plug-ins in use, you need to also install them on your remote computer(s).

If Scanner is suggesting the drive might fail, then I would try to move that data off and get that drive offline as fast as possible. Once offline, then I would suggest running some diagnostics on the empty drive. You already have Scanner, but I would also suggest downloading the free version of Hard Disk Sentinel for a second opinion. In my experience, once these monitoring programs detect something they don't like, it is best practice to assume imminent drive failure. Only once did my programs detect a false positive on a drive. I removed all data from it and took it offline. Fortunately, I was able to put that drive back into service after running a number of diagnostics on it to verify it was still good. All my other drives failed within a few days from first detection of any problems, and one drive just failed without warning. There are just so many attributes that these programs monitor that I don't really understand why "similar" reading on drives might indicate pending failure on one drive but not on the other. And I have also seen all Green attributes on a drive that is reported as pending failure. So there might be attributes way down at a much lower level than we see that are causing concern.

I also use DrivePool as my media storage center. It does not matter if your movies are in a temporary cache or moved to their archive drive, DrivePool treats the files the same. I set up a 224GB SSD as a front end cache on my DrivePool and set the cache to hold 100GB of files before it balances. Most of my movie files are 12GB or less, and unless I am transferring lots of files to DrivePool, I can go days between balancing. If your DrivePool is constantly re-balancing, maybe you want to check the Manage Pool > Balancing > Settings > Automatic balancing - Triggers > Or if at least this much data needs to be moved "100" GB in my case. You can manually set whatever trigger limit works best for you. A little trial and error and you will figure it out. I just let DrivePool send the cache files to whatever pool drive it wants (defaults to the drive with most free space). However, there are other plugins and options to modify this behavior. For example, the Ordered File Placement plugin will send files to one drive until full and then move on to the next drive. You just tell DrivePool the order of the drives to use. That has advantages that some people may prefer. As to settings that best suit your needs, I can only tell you that what I thought were my best DrivePool settings 6 months ago are not the same as what I am using today. I suspect that 6 months from now my best settings may have evolved in another direction. It really does not matter because with DrivePool you have many options to tweak it in various ways to better suit your particular needs.

Stablebit scanner, from what I understand, mainly detects problems and notifies you. If you are lucky, it will move data off that drive to the other pool drives before it completely fails. But is is not a data recovery program. I use Hard Disk Sentinel which is similar. They have a free version for just monitoring the drives, and a Pro (paid) version which fixes some HDD problems - but not all. HDS is not a data recovery program either. All HDDs will eventually fail, so I have gotten into the habit of adding 10% .par2 files to my offline data. Check out the free MultiPar if that sounds interesting to you. The .par2 files allows me to verify if the folder data is still intact and possibly recover a damaged file. I also have a data recovery program. Sometimes it will recover a file but the recovered file itself is still damaged anyway. So I started using the .par2 files for both verification and recovery of files. Works better for me.

I have the latest updates. All my graphs are working. Have you tried the first steps such as cold starting your computer, or uninstalling and then reinstalling DrivePool? If none of that works, I'd suggest sending a message to the programmer via the contact form. Again, I have the latest DrivePool updates installed and everything is working fine for me.

I used to run a Windows Storage Spaces server for about 7 years. The last 2 years, as my pool kept larger and larger, I had more and more problems with Storage Spaces. I spent a long time considering other options including FreeNAS. I talked to people who were running, or used to use, FreeNAS and learned that FreeNAS has problems like Storage Spaces when the pool gets large. At that time, I was just over 80TB on the pool and having significant problems with Storage Spaces that I did not have when the pool was much smaller. The people I talked to about FreeNAS told me similar stories, it worked fine to a point and then when the pool got larger, they started having significant problems. In fact, the guys I talked to had already given up on FreeNAS and moved on to other options. I moved on to DrivePool and my experience has been much better. I am now over 80TB on my DrivePool server and, so far, have not seen the problems I experienced with Storage Spaces. There are some things I miss about the "promise" of Storage Spaces, but in real life, the performance of Storage Spaces falls short. My friends running FreeNAS told me the same story with using FreeNAS. I am not claiming that DrivePool is perfect, but it just seems to work better for me. After adding a SSD to DrivePool as a front end cache, I now get write speeds that exceeded my Storage Spaces setup. If you chose to duplicate some folders in DrivePool, then you have the option of using Read Striping and that can almost double your read speed in some scenarios. However, I chose DrivePool over other options not because it was faster, but rather because it just worked better for me. When a pool drive fails in DrivePool, you only lose the data on that one drive, not the entire pool (as happened to me in Storage Spaces). If you have duplication set on either the entire pool or just certain folders, you can rebuild the pool from the duplicated data. Also, when I have had HDDs fail, sometimes most of the data on that drive is still available and can be transferred back to the pool. In one instance, I had only 2 or 3 corrupt files on a 3TB HDD that was failing. I was able to move all good files off the drive before it finally, totally, failed.

FYI, I just ordered another renewed 4TB HDD with 5 year warranty from Amazon for the sale price of $59.99. I was able to select GoHardDrive as the seller. Interestingly, GoHardDrive lists that same drive on their website as not in stock, so maybe they have some contract with Amazon at the moment. They sell a different renewed HDD on their website, but it comes with a 3 year warranty, not the 5 year warranty as if bought through Amazon. Just curious on how other people make their purchasing decisions on HDDs. I used to just buy the least expensive drive in terms of $/TBs. For example, a 3TB drive for $45.00 would be $15/TB. My renewed 4Tb at $60 would also be $60/4TB or $15/TB. However, if you factor in the warranty periods, then you have a better look at lifetime value. For example, the $45.00/3TB HDD/3 year warranty comes out to be $5/TB/year for warrantied storage. The renewed 4TB HDD is $60.00/4TB HDD/5 year warranty which comes out to be $3/TB/year, which in my book is a better value. Over the years, I have lost so many HDDs that I only count them as viable drives for their warranty period. Most drives last that long, but I have had several HDDs die before the warranty period expired. So now I only consider their warranty period in my purchasing decisions. If the drive lasts longer, I am happy. If the drive only lasts as long as the warranty period, I figure I got my money's worth out of the drive.

Total agreement. When I upgrade my HDDs, I often buy 2 or 3 of the same models on sale at the same time. Mfgr model name is almost useless to me, and trying to find a HDD by serial number is insane. IF these programs would allow the option to identify and list the HDD by Drive Name, I could quickly pull that HDD and then verify the S/N of the drive. Yes, I understand why these programs use the unique S/N of the drive as the ultimate identifier, but they would be more people friendly if we could list the HDDs by Drive Name. Again, my Hard Disk Sentinel orders the drives by Disk Number, which can change on every reboot of the computer. I sent off a support request about this issue to see if there is any way HDS can order and list the drives by Drive Name. That would be my preferred option, but if not yet possible, it's not the end of the world. I hope they would consider my request for a future update.

Nicely done. Looks like Stablebit Scanner has more sorting options than Hard Disk Sentinel.

I have been looking into using the features of SNAPraid with DrivePool. From what I understand, SNAPraid only provides parity files for reconstruction of the pool after a drive loss. It is not a backup program, per se.

DrivePool is a great option for providing duplication on either the pool level or down to the folder level. However, DrivePool duplication is mainly used for drive loss failure on the pool itself, so you can rebuild the pool from the loss of a drive. It is not meant to be a backup system. I backup my files to external USB drives which I store in my closet. If you are looking at 6TB storage, you can get an external 6TB (or larger) USB drive and backup the files there. That is about the simplest way to make an offline backup. If these files represent present or future revenue, then I would have more than one backup drive. One backup drive might be stored on location, but the second backup drive should be located offsite - in the rare case of fire, for example. Some people use cloud backup, but the transfer time to the cloud might be really slow. I personally don't use any cloud backup service, but I do have a few free accounts for small files and limited file size. @Shane has mentioned he uses a backup system that offers incremental backups. I cannot remember the program he uses, but maybe a shout out to him will help you. IF I was just starting my backup system, and had 6TB to backup, I might go with the system @Shane uses. But my DrivePool is currently about 80TB and I have lots of additional data stored offline. So I have not found my perfect backup system.

Interesting. I use Hard Disk Sentinel, and it displays the Drive Name on the status window. That is good, but it still orders the list by Disk Number as they attach to Windows on boot. I would prefer that the drives were listed by Drive Name, as I have it listed in DrivePool, but so far I have not figured out how to make Hard Disk Sentinel order the drives by name. It's not a big deal, as I only really check Hard Disk Sentinel only if I get an alert. I might check the status of the drives in DrivePool multiple times per day, and there the drives are order by Drive Name. But, yes, using Drive Names is a much better option for organizing pool drives, IMHO.

I tried to buy my Amazon renewed drives from the seller GoHardDrive. I have purchased drives from GoHardDrive for years and their customer support has always been first rate. I feel confident they would back their warranty.

OK. That helps. Thanks.

Thanks to both for the responses. I am not using any File Placement rules. I only mentioned Album folders as an example to see if DrivePool would gather up all the files in a folder that might currently reside on two or more drives and place them on one drive in the re-balance. I am only looking for a general order priority for the drives, and not sending specific folders to specific drives. As to Question 2) Any thoughts on where the Ordered File Placement plugin should be listed on the priority list? Again, thanks.

@Christopher (Drashna), yes, I know that my DrivePool is working as it should. And if there are any corrupt files found, I am glad that DrivePool does a complete re-check. I was trying to compare my satisfaction with DrivePool in contrast to my frustrations I had with Windows Storage Spaces. Appreciate your feedback. Thanks.

I have decided to try the Ordered File Placement balancer plugin. I currently have 18 USB 3.0 HDDs in my pool, and want them to fill up in order such as DP01, DP02, DP03, etc... I will be changing the default option to "Control new file placement and move the existing files into the pre-defined order" instead of "Only control new file placement." I understand that this option may require more than one balancing pass to complete. Question 1) Will this consolidate all files in folders that may currently be stored on 2 or more pool drives? For example, if I have a music album folder, currently the tracks in that folder may be spread across 2 or more pool drives. Will the Ordered File Placement plugin grab the entire folder and write all those files spread across the drives and move them into one drive? Question 2) After installing the Ordered File Placement balancer plugin, it is currently sitting at the bottom (# 7) of my Balancers list. Where should it be placed in the list for best effect? Current listing as follows: 1. SSD Optimizer 2. Stablebit Scanner 3. Volume Equalization 4. Drive Usage Limiter 5. Prevent Drive Overfill 6. Duplication Space Optimizer 7. Ordered File Placement I am thinking it makes sense to move the (7. Ordered File Placement) just under (2. Stablebit Scanner) and bump everything else down. Any help appreciated. Thanks.

Fortunately, no. I have 18 USB drives and was very happy when I discovered that DrivePool did not require Drive Letters to work. I have auto update on Windows 10, so I imagine I have the latest updates. Everything is working fine here. Although I do not use Drive Letter assignments, I have all my pool drives named. Since I am so clever, I named them DP01, DP02, DP03, etc... and they sit on a shelf in that order, too. I also labeled the drives with a small tag. Makes it easy for me to pull a drive if it has any problems. I hope you find out what happened to your DrivePool and why you now need Drive Letters assigned. I'll be reading to see what develops. Hope you get a fix soon.

I recently ran into this problem myself. I use DrivePool as my home media center. I don't have duplication set on most of my folders because I have backups on HDDs in storage in my closet. I figured if I lost a drive, then I would just pull out the backups and copy the files back into the pool. However, for example, I discovered that any Album folder in DrivePool might have tracks in that folder spread out on 2 or more drives. So instead of losing 1000 complete album folders on that drive, I found out that I lost more like 2000+ partial album folders that had to be restored from backups. For some reason, I thought DrivePool would write complete folders to any drive, but it appears that is not the case. I have started adding 10% .par2 recovery files to my folders, which will verify the files, repair some missing or damaged files, or tell me that I needed more recovery files and it is time to get out the backup from the closet. Is it possible to tell DrivePool to write complete folders to any drive and not split the files over 2 or more drives? I would think File Order Placement would help, but there seems to be no guarantee that changes made later to files in a folder would end up on the original drive.