Jump to content

gtaus

Members
  • Posts

    241
  • Joined

  • Last visited

  • Days Won

    15

gtaus last won the day on November 15 2021

gtaus had the most liked content!

Recent Profile Visitors

1110 profile views

gtaus's Achievements

  1. When I remove a drive from DrivePool, for example a 4TB HDD, it can be a lot longer than 18 hours. Add to that, a file duplication after the removal, and you would be looking at many, many hours for the task to complete. Having said that, I know DrivePool is not the fastest solution for file transfers, but it seems to get the job done in its own time. In my case, the process takes place in the background and I just let it run because I'm usually not in any great hurry. I almost always have DrivePool remove the drive and duplicate as needed in the removal process. I suspect, that DrivePool might offload all your files on your remove drive but you might still have some duplicate files on the drive when it is done. But, I have very few folders designated to duplicate. To check that your files set for duplication were redistributed properly on DrivePool, you can go to Settings>Troubleshooting>Recheck duplication and that should report results for you. Again, that process might take some time. I have 80TB+ on my DrivePool, almost all pool drives are slow USB archive drives, and I have learned to be patient. Fortunately, DrivePool seems to handle things better without my interference and I have seen some processes take a few days to complete.
  2. I see. your SSD's are used as archive drives, just very fast archive drives. I have not used File Placement in my DrivePool, so I don't have any personal experience to offer. If adding your new 8TB drive to the systems cleans up the files being dumped on your SSD's, then I hope you update your results so we all can learn.
  3. I am not support, just another user. However, I like your approach Method2 for Backup. If SSD1 fails, you could remove it from DrivePool and replace it with a new SSD. Then, you run a compare folders of your backup drive and DrivePool in any number of File Explorer programs (Free Commander is what I use), and select it to write any missing files/folders. To me, that is a simple approach, and it lets DrivePool write the files to your SSD's per your DrivePool settings. OK, I am a big fan of DrivePool, however, there are some shortcomings I have become aware of in using the program for the past ~2+ years. First of all, file duplication is not a smart duplication in that if your source files are damaged, then your duplicate copy will also have the damaged files. I have been looking for some kind of automatic checksum program that verifies all files are present and in good working order. The only thing I have found, so far, is using Multipar (freeware) and its .par2 files for verification and recovery, but that is a manual process. For my media files, like an album with multiple file tracks, or an audiobook with multiple file chapters, I typically set my Redundancy threshold at 10%. That allows me to verify all contents of the folder are present, and if some files are lost or damage, it can recover some files. Of course, a higher Redundancy threshold would recover more missing or damaged files. Using DrivePool, I have discovered that a media folder might be spread out over 2 or more drives. Without some means of verifying all files are present in the folder, you could end up being a song or two short of the full album. Ask me how I know! I had a HDD fail in DrivePool and I discovered lots of albums missing a few tracks. Would DrivePool file duplication save the day? Maybe, if none of the files were damaged and duplicated as damaged. But with Multipar, I can easily verify all files are still in the folder and/or recover some lost or damaged files. At 10% redundancy, I can verify my files and recover/repair most of my lost or missing files if needed. DrivePool file duplication at 100% redundancy cannot verify all files are in your folder and/or recover lost or damaged files in the same manner. If you have read thus far, let me say that DrivePool works great for me, but I have separate HDD backups of all my important files sitting in my closet. Additionally, I use Multipar with .par2 files to verify my data. If any or your DrivePool data gets damaged, you might never know and happily backup damaged files or incomplete folders. Multipar works great on media sized folders of less than a few GBs, but it would take forever to create .par2 files for my entire DrivePool. So, unfortunately, I have to create .par2 files on a per folder basis. If you know of anything that creates checksums and verification/recover options like Multipar, but on a large data set like a DrivePool virtual drive, please let me know.
  4. EDIT: Deleted by user.
  5. I have had to deal with a similar problem before. If you know which PoolPart is old and not being used, move all files from the old PoolPart into Drivepool using Explorer. Then delete the old PoolPart. Do this on both HDDs that have duplicate PoolParts. Then delete the old PoolParts. When you remeasure, you should have all your files in DrivePool and your "other" data should be back to nil (or close to it). It might be faster to just move all data from the old PoolPart to the new PoolPart on the same HDD, and then remeasure, but I felt more confident in letting DrivePool "fix" itself by moving the files from the old PoolParts into DrivePool. That way, DrivePool will put the files on the HDDs that make most sense for your settings (balancing, etc..).
  6. I have found that DrivePool balancing and duplication often have problems when I hit that 90% threshold. I have seen my SSD cache get kicked offline as well. Fortunately, adding more HDD storage and/or removing unused files corrects the problems on my system. I have to manually recheck my SSD to tell DrivePool that it should be used as an SSD cache and not an archive disk. But it seems to work again without any problems. If you are happy with your current custom SSD cache settings, I would write them down because it seems to me that I had reenter my settings. After many months of not looking at DrivePool (it just works), I had forgotten my custom SSD cache settings and had to play around with it again until I got it back to where it works best for my system. If you use the default settings, then it might not be an issue.
  7. Answering my own question. After having basically given up on the DrivePool remote app on my replacement laptop for the past 4 weeks, I decided again to uninstall the app and reinstall it with a fresh copy a few days ago. For whatever reason, this time the app worked and connected to my DrivePool on my home server. It's all a mystery to me as to why it did not work before, but it's working now.
  8. If you are only transferring that single 1 TB file once in a blue moon, it might be easiest to temporarily turn off the SSD Optimizer until that file transfer is complete. Is it possible to have your backup program split the file into smaller parts for easier transfers? You can also just transfer files directly into a DrivePool HDD and DrivePool should pick it up without any problem. I have done that in a few cases and it worked fine for me. Of course, that circumvents your SSD and probably will be a much slower transfer. But at least you would not have to turn off the SSD Optimizer for all your other file transfers.
  9. I really like DrivePool, but I will admit that bad things have happened to me when my pool reaches its max storage over 95% like you mentioned. Most frequently, I get Duplication Inconsistent errors and I don't know how to fix them - other than adding more storage and/or removing data from the pool. Rebalancing does not help much if your HDDs are already over 95%. As a rule of thumb, IMHO, DrivePool should never be considered a backup solution. So if you lose a pool drive, and lose data on the pool, you should have already had a backup offline stored somewhere else for recovery. In my case, I use DrivePool as a media storage server and if I lose a pool drive, and some or all of that data on the failed drive, I might not even bother to recovery the loss with my backups. Once I watch a movie, for example, I don't really care if it stays on my DrivePool or not. So I would probably not restore a copy from my backup copy. I try to keep as much free space on DrivePool as my largest drive in the pool. That way, if a drive starts showing signs of failure, then DrivePool (via Scanner) has a chance of offloading the failing drive into the pool before it completely dies. I have had some drives report a problem and I was able to offload the drive into the pool over a day or two. That's when things go good. Other times, a failing drive may die within hours, and there is not time to fully transfer all files off the drive. And, of course, I have had some drives just up and die without any warning.... DrivePool offers duplication, but to be honest, I have never really understood the method it duplicates the files. Worse yet, if a file becomes corrupt, DrivePool cannot tell you which is the good copy and which is the corrupted copy. From what I understand, DrivePool at best can only tell you that one copy is older than the other. DrivePool itself cannot repair any corrupt files or tell you which copy is good. Over a period of time, I use DrivePool Duplication less and less. What I do for my media files is use a free program called MultiPar which creates .par2 files for verification and recovery. I have MultiPar set to create 10% recovery files, which is usually enough to repair a track or two on a music album folder. In DrivePool, any album folder, for example, may have music tracks on more than one pool drive. If you lose a HDD, you might lose a track or more on a failing HDD. With the .par2 verification, you can tell within a few moments if your data set is all there, if the data can be recovered from your .par2 recovery files, or if you have to pull out that backup copy from the closet like I mention earlier. Anyway, the verification and recovery features of MultiPar have saved me lots of time and effort after a HDD failure. Here is a link for the free MultiPar program, if you are interested. As for connecting your failing drive and see it spinning up.... well, I guess if you have already given up the idea that you will get anything else off the drive, you could just let the program run either until the program completes whatever it is doing, or the drive dies and becomes unreadable. When I first got into DrivePool, I had some HDD issues with older drives and decided to buy Hard Disk Sentinel after trying the free version. The free version will monitor your drives and alert you to possible failures, and if also offers a bunch of free diagnostic tools, but the paid version has a feature that allows you to "fix" certain HDD issues and continue to use that drive. I was able to "save" a couple of my HDDs using Hard Disk Sentinel's paid version, so I figure I got my money's worth out of the program with my first HDD "save." A couple other drives died hard and there was no saving them. Even though I was able to "save" a couple of my drives with Hard Disk Sentinel, I still replaced them as soon as possible. In my experience, a failing drive will continue to fail until it crashes completely. But I was able to buy time using Hard Disk Sentinel and did not lose data on those "saved" drives before I replaced them. Finally, the Service Log can be found by clicking on the upper right icon Settings>Troubleshooting>Service Log... Hope that helps.
  10. gtaus

    File Placement Issues.

    Sounds like a free space issue to me, too. It may have something to do with your settings. For example, under Balancing>Balancers>SSD Optimizer there is a section where you set the "Fill archive drives up to:" and my DrivePool is set at 90%. But maybe your limit is much lower. On a larger HDD, it is possible to have lots of free space left on the drive but maybe your set limit is preventing use of that drive(s). I'd check all those limit settings to look for anything that stands out. There is also another option to check off below that "Fill archive drives up to:" called "Or this much free space" which I have set on 100 GB free per drive. I am running mostly 4TB HDDs on my DrivePool, so 90% limit would mean that 360GB would not be used per 4TB drive. So I set the optional free space setting to 100GB per HDD. I suspect this means that DrivePool will first fill up all my HDDs to 90%, and then continue to add data to the drives until it reaches that 100GB free limit. DrivePool works great for me, but I have noticed that when I get to my free space limits on the drives, unexpected things happen. Well, probably unexpected to me, anyway. Adding another drive to DrivePool usually solves these free space problems as DrivePool will rebalance itself and usually correct those free space issues. Another thing I occasionally do is to move data off to offline backup drives and/or delete data that I no longer want. You might try and temporarily move some data off of DrivePool, have DrivePool rebalance itself and clear the errors, and then move the data back on to DrivePool. Most of my errors are related to Duplication Inconsistency and those errors usually show up when DrivePool reaches its set storage limits. Adding another Archive drive to DrivePool usually solves most of my problems. I have used many different drive pool systems (RAID, Storage Spaces, DrivePool) and they almost always want you to add another drive when free space hits the pool limits. All these drive pool systems seem to work better when they have lots of free space to use. So I hope you find the cause of this error message and can correct your issue.
  11. I ran into a similar situation recently, only my SSD ended up with 2 different PoolParts!? What I did was to manually move my data off the older PoolPart on that SSD to the newer PoolPart on that same SSD, and DrivePool seems to be OK with that solution. I then deleted the older PoolPart folder on my SSD. Everything seems to running OK, but I won't swear that was the best method to clean up the old PoolPart. But it is similar to your situation, so I hope this helps.
  12. I don't use the File Order Placement on my DrivePool, but as I understand it, you can order your drives in the priority they should be used. If you designated your 12TB Seagate Exos as the last drive to be used in the pool, that should significantly reduce the writes to that drive as long as your other drives with higher priority have free space. At the same time, you could also manually transfer files to/from your 12TB Seagate drive when you want. If you have files on the 12TB Seagate that you constantly need to read, then I would manually move them to another drive. If you have lots of frequently used files to move off the 12TB Seagate, then you might consider to "Remove" the drive from DrivePool, have DrivePool empty its data from your 12TB drive, and then add the 12TB Seagate back in again to the pool as the lowest priority drive in the pool. Then you could manually move rarely used files to the empty 12TB Seagate and basically have the 12TB drive there as storage. I think that would work. I'll be reading this thread to see if there are other ideas.
  13. If I understand you correctly, you want to know what advantage adding an SSD to DrivePool v2.2.5 would add to your setup. In your case, since all your pool drives are already SSDs, I don't think you would see any advantage to designating an SSD as a front end cache because all of your SSD drives probably have about the same read/write speed. In my setup, I have 19 USB 3.0 HDDs as Archive Drives (much cheaper, but also much slower, than SSDs) and then I added a single ~250GB SSD as my front end cache for DrivePool. When I write a file to DrivePool, it first goes to my SSD (very fast) and latter when DrivePool "rebalances" it moves the files off the SSD to the much slower Archive USB 3.0 HDDs in the background. In my case, the SSD front end cache significantly sped up my DrivePool writes. If the file(s) is still in the SSD cache when I read it, it will also speed up the read process. However, by the time I want to read any of my files in DrivePool, they are almost always flushed to the Archive drives by then so my read speeds are limited by the transfer rate of my USB 3.0 HDDs. I use my DrivePool primarily as a home media storage server, so the speed of my read rates are not critical. The SSD front end cache allows to me quickly transfer files to DrivePool and then move on to some other tasks. If, at some point, you decided to add non-SSDs to your DrivePool as Archive drives - to save money, maybe - then you would see an advantage to designating one or more of your SSDs as a front end cache for DrivePool. That way, if you write a file to DrivePool, it will always go to the SDD front end cache first at SSD speed. If not, DrivePool will attempt to balance the pool drives and maybe write directly to your Archive drive(s) at their non-SSD rated speed - which would be much slower than your SSD drives. I only mention this because my DrivePool storage continued to expand beyond my initial setup and I bought non-SSDs to save money.
  14. I have DrivePool running on my home media server computer. It normally does not have a monitor attached. So I access and control DrivePool from my other computers using the DrivePool Remote Control app features. One of my laptop computers recently died and I had to replace it. When I installed the DrivePool app, initially it would not accept my license. A quick email to DrivePool Support solved that problem and now the app is running on my new laptop. However, it does not see I have DrivePool already running on my network. On the "Manage a different computer running StableBit DrivePool" banner, the only option I have is the new laptop itself. It does not list my server with DrivePool running on it. The new laptop is running Windows 10 Home, whereas my other computers all have Windows 10 Pro. I don't know if that makes any difference. I have checked and made sure that the Stablebit DrivePool program has Private permission checked to go through the Firewall by Defender - the same setting on all my other computers. My DrivePool on the server is showing up in the network share as Drive J: and I can transfer files without any problems. I just can't get the DrivePool Remote Control app to work on the new laptop. Any thoughts or suggestions welcomed. Thanks.
  15. I have never experienced your problem of files getting corrupted when moved off the cache drive to the archive drives in the pool. I have never used the DrivePool Verify After Copy feature because I have not had the issues you are reporting. I don't know how DrivePool would respond if a verification failed, but I suspect it will alert you and will probably leave the file on the cache drive and try again the next time it balances. But, I really don't know as I have never used that feature. I once had a problem with a USB 3.0 HDD caddy and transferring files into/from DrivePool. Files were getting corrupted in that transfer and I solved that issue by plugging the caddy into a USB 2.0 port and slowing all the transfers down. For some reason, my corrupt file transfer issues went away with USB 2.0. Maybe the caddy had a buggy USB 3.0 engine? I would still encourage you to use something like MultiPar with verification files to narrow down your issue. If you had a test folder(s) with files and .par2 files, you could verify the folders first and then transfer those files to your DrivePool cache. You could immediately run a verification on the files in cache to see if they transferred without corruption into DrivePool. Then after the cache flushes to the archive drives, run the verification on the folder again using the .par2 files. In that way, you could verify the files on your initial device or computer, in the DrivePool cache, and in DrivePool archives. Also, you might consider trying a different interface cable on your archive drive to see if that makes a difference. Cables can go buggy and cause intermittent corruption. Just enough to drive you crazy because you probably can't verify it was a bad cable unless a different cable solves your issue. Good luck. I hope you find the solution to your corrupted files issue.
×
×
  • Create New...