Jump to content

gtaus

Members
  • Posts

    227
  • Joined

  • Last visited

  • Days Won

    13

Everything posted by gtaus

  1. I have never experienced your problem of files getting corrupted when moved off the cache drive to the archive drives in the pool. I have never used the DrivePool Verify After Copy feature because I have not had the issues you are reporting. I don't know how DrivePool would respond if a verification failed, but I suspect it will alert you and will probably leave the file on the cache drive and try again the next time it balances. But, I really don't know as I have never used that feature. I once had a problem with a USB 3.0 HDD caddy and transferring files into/from DrivePool. Files were getting corrupted in that transfer and I solved that issue by plugging the caddy into a USB 2.0 port and slowing all the transfers down. For some reason, my corrupt file transfer issues went away with USB 2.0. Maybe the caddy had a buggy USB 3.0 engine? I would still encourage you to use something like MultiPar with verification files to narrow down your issue. If you had a test folder(s) with files and .par2 files, you could verify the folders first and then transfer those files to your DrivePool cache. You could immediately run a verification on the files in cache to see if they transferred without corruption into DrivePool. Then after the cache flushes to the archive drives, run the verification on the folder again using the .par2 files. In that way, you could verify the files on your initial device or computer, in the DrivePool cache, and in DrivePool archives. Also, you might consider trying a different interface cable on your archive drive to see if that makes a difference. Cables can go buggy and cause intermittent corruption. Just enough to drive you crazy because you probably can't verify it was a bad cable unless a different cable solves your issue. Good luck. I hope you find the solution to your corrupted files issue.
  2. I have not experienced your specific problems, but I have a few things I might try. You might want to turn on the verify after copy option. Goto Settings Cog>Troublshooting>Verify After Copy. This feature is normally turned off because it will slow down your system. However, in your case with suspected bad copies, it might be worth it to check out that feature. I use the free program MultiPar to create .par2 files for file verification and rebuild. In your case, you would tag the movie file, or folder, and create .par2 files. When you run MultiPar on the .par2 file, it will verify if the original file is intact or if it has been damaged. If damaged, it will attempt to repair the file(s) with the .par2 files you created. You can adjust the % of .par2 files for rebuild anywhere from 0% (index only) to 100% complete blocks for total rebuild. I have my MultiPar set to 10% .par2 files which is enough to verify the file(s) in the folder and will rebuild some damaged and lost files. Of course, I would recommend a good offline backup plan. If your files become corrupt, and MultiPar cannot rebuild the damaged files with the .par2 files you created, then you can pull the backup files and reload them. I really don't understand why DrivePool would be corrupting your files when it balances the pool, but maybe some of these ideas will help you narrow down the cause of your problem.
  3. My DrivePool SSD as a front end cache has been working just great for about 1 year. However, I recently experienced something with my computer that reset my DrivePool settings, removed the SSD cache, and spent weeks telling me that Duplication was inconsistent. Since DrivePool was in an error state, I attempt to use "Troubleshooting" and recheck duplication. Never fixed itself. Thinking my DrivePool was reaching its max capacity, I added another 4TB drive to the pool and within couple days DrivePool was able to fix the Duplication inconsistent situation. So I added my SSD drive back into the pool, as an SSD, and reset the settings to where I think they were set before I had the problems. Unfortunately, my SSD cache no longer flushes to the archive disks as expected. Before, I had the SSD set to flush at 100 GB and had no problems. Furthermore, if I wanted to force a flush of the SSD cache, I would just tell DrivePool to rebalance itself and it would flush the SSD cache to the archive disks. Now, DrivePool will write to the SSD cache, but it does not flush the data to the archive drives when it reaches the 100 GB limit, and when I attempt to force a manual flush of the SSD it will not work. So my SSD eventually reaches a point where it is almost full and then new data will be written to any drive on the pool with the most free space. Something got broke and I don't know how to fix it. For the time being, I have to manually "remove" the SSD to flush the cache and then add the SSD back to the pool. Although this is a viable work around, it is not a solution to my problem. A SSD cache that will not flush itself automatically is not really a cache. Any suggestions would be much appreciated.
  4. gtaus

    Blank UI - Drivepool

    I have not had your problem. However, when I do have any problems with the DrivePool UI on my server, I usually just restart the server and the problem(s) seem to go away. If that does not work, I might suggest uninstalling/reinstalling DrivePool with the latest updates. Failing that, I would suggest contacting support and open a ticket. Like many issues, you might not know if this a Windows OS issue or something to do with the DrivePool UI itself. FWIW, I have experienced unexpected results on my programs after Windows completes an update. But usually a reboot solves these temporary issues. Hope you get your problem fixed.
  5. If you want to protect your data, then you really need a good backup plan. IMHO, no online pool system - RAID, DrivePool, etc... - is adequate to protect your data. But that is just based on years of having these online systems fail and data not backed up being lost. As to your specific concerns: 1. RAID 1 - takes a lot of HDD to mirror your data. Does everything in your pool need to be duplicated? If not, then RAID 1 may be too costly as your pool size grows. 2. DrivePool w/ Duplication - I only have a few directories selected for duplication in my DrivePool. Duplication in DrivePool is not designed to be a backup solution. It is there if you have a HDD failure and you could probably recover the pool faster from the duplicate copies. However, if you have some kind of file corruption, DrivePool cannot tell you if the original or the duplicate copy, or both have been corrupted. 3. DrivePool w/o Duplication + Scanner - Scanner might be able to save your data is the HDD has signs of pending doom and there is time enough to evacuate that drive. However, in my experience, with these newer larger TB HDDs, you might not have enough time to evacuate the drive before it fails hard and loses all data. I had a couple HDDs that were starting to fail and they were able to be evacuated without data loss. Hurray! However, I had a couple other drives that alerted and died within hours with no where near enough time to evacuate the drives. Lost data on those drives. Good thing I had a backup plan and archive HDDs stored elsewhere. Other options to consider.... 4. DrivePool with SNAPRaid - there are a number of people using DrivePool for their main data and SNAPRaid to offer recovery options. I personally don't use SNAPRaid because I have very little data on my DrivePool media storage center that needs that extra protection. But maybe your situation is different. 5. DrivePool with MultiPar - This is my method. I use Multipar to create 10% .par2 files for folder verification and rebuild. For my applications, 10% .par2 files is enough for me to verify if the files are still all there and intact, or if some of the files are missing or damaged, then there is probably enough .par2 file blocks to rebuild the damaged file(s). You can set the .par2 file setting all the way up to 100%, but then you would probably be better off with DrivePool duplication or RAID 1 options. I prefer to use MultiPar on my album folders, because DrivePool will send your album tracks to any number of HDDs in your pool. If you lose a track or two from a HDD failure, then MultiPar will fail verification and attempt to rebuild the missing files. Since I have all my data backed up on HDDs I keep in the closet, I don't really worry too much about lost data and if there are not enough .par2 blocks to rebuild the folder, then I'll just pull out the archive HDD and reload that album if I still want it. DrivePool itself cannot tell you if the folder is intact with all associated files, and if the data was damaged somewhere, then your DrivePool duplicate copy might be damaged as well. MultiPar will tell you if the files are verified or not, and you can decided what to do after that. Anyway, good luck on setting up your system.
  6. Don't have an answer to your question, but just noticed that my DrivePool balancer settings were automagically reset and my SSD was no longer working as my front end cache. So, I am trying to figure out what caused DrivePool to lose its settings. I have spent the past few days trying to tell DrivePool that my SSD is not an archive drive, but it still treats it the same as all my archive drives. I don't know why my settings were changed, but I'm having a problem getting DrivePool back to where it belongs.
  7. That's a nice looking chassis you have. 45 HDD's - nirvana! But let me say that I really don't know much about your type of setup because I have never used one. What has me most confused is that your card is for a RAID system. Do you have your controller card and chassis to run in a RAID system, or are your drives listed separately in file explorer? AFAI, DrivePool was designed to work with regular NTFS drives in a non-RAID environment. I have about 19 HDDs connected via USB 3.0 on my system, and it works fine. But I do not know how DrivePool would talk to a RAID controller. So I might be totally off base, but since nobody else has chimed in on your concern, I would be looking at RAID card and maybe that is slowing down your transfer speeds. FWIW, my USB 3.0 drives can transfer data about 120 MBs from my SSD to the archive drives. All my USB 3.0 drives are standalone drives, but I don't have drive letters assigned to the pool drives because DrivePool does not require drive letters. I hope someone can help you. DrivePool is a great software program, but I suspect you might have a hardware issue somewhere with your setup.
  8. My MediaSonic 4-disk ProBox bit the dust last week. So 4 HDDs in my DrivePool were reported as missing. For better or worse, my DrivePool is now up to 19 HDDs (via USB 3.0) and sitting at about 80TB. I was able to remove some HDDs from USB enclosures I had in storage and finally got those 4 missing HDDs from the pool back online. However, DrivePool is not liking the new pool setup. After the missing disks were put back online, DrivePool measured the pool, which seemed to go OK. But then it did a duplication check and erred out on that task. So I forced a trouble shooting duplication check and for the past 2 days, DrivePool has been Measuring... Duplicating... Calculating... etc. With an 80TB DrivePool, it appears that a task is working on the animated status bar flashing, but I have absolutely no idea what % of the task is complete, or if the task is hung up and not progressing. It can be flashing the exact same task on the status bar for hours and hours, overnight, and into the next day. I have just been letting it run on it's own, because in the past I have seen that DrivePool does eventually seem to correct itself - it just might take a very long time. And with an 80TB DrivePool, I am seeing that it is taking over 48 hours and still counting. My suggestion is to add a percentage complete display to the task bar so a person can verify the program/task is still working and not hung up. FWIW, I know all my data is still safely on the HDDs that were in the ProBox. I really don't care if DrivePool takes a long time to correct itself (with an 80TB DrivePool) as long as the task is eventually completed. I just want to be sure that everything is still moving forward and not hung up in a death loop.
  9. @Rob Manderson, Thanks for the offer. Attempted to send you a PM with some info.
  10. My ~10 year old MediaSonic ProBox died today. Will no longer recognize the USB interface. So my DrivePool is now down hard because I have 4 drives offline. At first I was just going to replace the old MediaSonic ProBox with a new one. However, current price on the MediaSonic ProBox is now close to $200. When I bought my ProBox, it was $50. OK, sticker shock. But this leads me to question other options. Individual USB enclosures are around $25, so about $100 for 4 enclosures to replace the ProBox. Have also looked at dual docking stations for about $36 each, or $72 for my 4 drives. Are there other options I have not considered? Don't know if anyone here has some recent advice on enclosure purchases, but would appreciate any help in deciding how to move forward. Thanks.
  11. Follow up: I moved my CineRaid USB Docking Station to a USB 2.0 port instead of a USB 3.0 port, changed the TeraCopy default from asynchronistic mode to OFF, and checked the default copy/move command to verify. Since then, I have had no further problems. I suspect the main benefit was moving the USB Docking Station to USB 2.0 and just slowing down everything. But that is just a guess. At any rate, I no longer have the problems I was complaining about in this post.
  12. I don't think DrivePool would like to see 2 hidden directories with the same directory names. If you took the HDDs offline, I think you could simply cut and paste the whole hidden folder within the drive containing 6TB of information, and then only connect the new drives back into DrivePool. IIRC, DrivePool only checks for the hidden PoolPart directories and does not care what the HDD serial number or name is. But you don't want 2 HDDs with the same hidden directory names. +1, that is how I would go about this transfer. Then I would "remove" the old drives using the remove command in DrivePool and let DrivePool clean itself up. That has happened to me, too. So I have one computer without DrivePool on it and that is how I perform any and all "offline" tasks with DrivePool HDDs.
  13. I'd like to have a 256PB disk, too! If upgrading to 2.3.0.1244 Beta does the magic, I think I'll upgrade now! More seriously, I have not seen this issue with v2.2.5.1237 Stable, so let's just hope it's a beta issue that can be worked out before final release.
  14. What is the size our backup system file? Is it possible to write the backup file to your system HDD first, and then transfer it to DrivePool? I have never seen the Status_Wait_1 error you mentioned, but I don't use the "old" Windows Backup System in Windows 10 (Labeled as Windows 7 Backup) that you mentioned. I have used some other backup software, and it worked fine on DrivePool. Assuming you have at least one pool drive that can accept the complete system backup file, I don't know why the process is timing out on you. But I have never tried to write a file that takes an hour or two to complete.
  15. I don't have an answer. I have never seen this error. I'm signing on to see if anyone else knows what is going on.
  16. In theory, you should be able to run chkdsk on your drives without negatively affecting DrivePool. In fact, I have seen DrivePool recommend running chkdsk on HDDs if it detects some problems. In most cases, chkdsk will find and repair problems without any problems. Having said that, I once ran chkdsk on a DrivePool HDD and chkdsk wiped out the directory on that HDD, so it was basically an empty disk after it ran its task. So those contents were gone from everything, including DrivePool. My lesson learned was that if I needed to run chkdsk on a DrivePool HDD, I will now empty the HDD to the pool using the remove command in DrivePool, and then run chkdsk on the HDD, fix any repairs, and then add that HDD back to DrivePool. This, of course, takes a lot longer to accomplish because the remove command in DrivePool can take hours and hours on a large drive. However, it takes me even longer to replace the contrents of a HDD if chkdsk blows its mind and essentially reformats the drive during the chkdsk task. When you tell Windows to automatically fix your HDD with chkdsk, you don't have any more control over what it does at that point. I don't know if my answer is of any help, but I hope you heed the warning and don't experience data loss like I did. Chkdsk is a Windows task and has nothing to do with DrivePool. If chkdsk runs without any problems, all your data will still be there for DrivePool.
  17. I had a similar goal, but use a different method than what you suggest trying to do with DrivePool HDDs. I added some numbers to your requirements for discussion. First of all, I don't think DrivePool is capable of performing #1-#4 like you want. [1] As soon as you remove a drive from DrivePool, the pool goes into a missing disk routine and the pool becomes read-only. [2] DrivePool will scatter files over various HDDs in the pool. So, if you have an album of 15 tracks, you might have 10 tracks on HDD01 and maybe 5 tracks on HDD03. DrivePool does not split a file, but it will split folders. There are some plugins that, in theory, that will write all files to one HDD until full, then move on to the next HDD in the ordered list. But there are a number of overrides that can happen and I would not completely trust a plugin to keep folders complete and not split files in a folder to other HDDs in the pool. [3] Yes, if you remove a HDD from the pool, the files on that drive will be removed, but only so far as DrivePool reporting that the HDD is missing and locking down the pool into a read-only mode. I don't think you want that. [4] DrivePool does not split a file. So any files written to a HDD will be the complete file. But that does not mean that your folder(s) will be complete as the files may be split over any number of HDDs in the pool. As to [5], you can take any DrivePool HDD and attach it to any other computer and still be able to read the files in the hidden PoolPart directory. Since DrivePool writes complete files to the HDD, the files on that HDD are complete and can be read by any computer. However, the folders might not be complete, as mentioned before, so you might only have 10 tracks of the 15 tracks of the complete album folder. But, those 10 tracks would be complete. Again, I don't think this is the solution you are looking for. MY METHOD: Here is how I manage my DrivePool data. I have my DrivePool data up and running all the time. When I want to backup or archive any data for offline storage, I pop a HDD into a disk caddy and transfer files from DrivePool to the HDD in the disk caddy using TeraCopy with verification. In that way, DrivePool will read and consolidate all the data from the pool and transfer the complete files and folders to the HDD in the disk caddy. When the HDD in the disk caddy gets full, I label it for storage and put it in my closet. This method ensures [1] the pool is accessible all the time. [2] DrivePool will consolidate and transfer all selected files/folders to the HDD in the disk caddy. [3] If you choose to move the files from DrivePool to the HDD in the disk caddy, then those files will no longer be in the pool freeing up space in DrivePool for other data. [4] I use Teracopy to verify all archive moves. So DrivePool will write the file to the HDD in the caddy, and if the newly written file passes the verification checksum, then, and only then, is the file removed from DrivePool. [5] The HDD in the disk caddy can be removed and placed in any other of my computer systems and it reads just fine as a standard NTFS file formatted HDD. I think my method achieves your main goal without the problems inherent in removing HDDs from DrivePool. I currently buy bare HDDs which is why I use the hard disk caddy. In the past, I have also used USB HDDs for backup/archive purposes in the same manner. I know my method works for me, but I suspect your proposed approach to removing and adding drives to DrivePool will not work as you want.
  18. Whenever I have problems with duplication, I go to the Settings Cog>Troubleshooting>Recheck Duplication and let DrivePool try to figure it out. Honestly, if there are duplication problems with DrivePool (like after removal of a failed HDD), it takes me a couple times running the Recheck Duplication and Balancing tasks. Last time that happened to me, it literally took a few days for DrivePool to clean itself up, but I have 80TB in my DrivePool. To be fair to DrivePool, it did fix itself given time. I only have a few folders set for duplication in my DrivePool, so out of my 80TB pool, only about 20TB are duplicated. Also, DrivePool duplication is good for some things, but it does not ensure that your files are actually intact and complete. It is possible to have a corrupted file/folder and DrivePool is happy to duplicate the corruption. If there is a mismatch between the original and the copy, DrivePool cannot tell you which file/folder is true and which may have been corrupted. For example, my DrivePool is mainly used as my home media storage. If I have an album folder with 15 tracks, and one or two tracks gets deleted or corrupted, DrivePool cannot tell me if the original directory is complete, if the duplicate directory is complete, or if neither copy is complete. Because of this, I now add 10% .par2 files to my folders for verification and possible rebuild. With the .par2 files, I can quickly determine if the folder is complete, if any missing or corrupted files can be rebuilt from the .par2 files in that folder, or if I have to take out my backup HDDs from the closet to rebuild the corrupted data in DrivePool. Unfortunately, DrivePool duplication does not ensure that your data has not been corrupted. For this reason, I don't consider DrivePool duplication in any way a backup solution. It lacks the ability to verify if the original or the duplicate copy is complete and intact and cannot resolve mismatches between copies. In theory, from what I understand, duplication is mainly good for rebuilding your DrivePool if you have a HDD failure and the bad drive is a complete loss. Then, DrivePool will still have a copy of the files on other drives in DrivePool and can rebuild the failed data. That may be a great option, and of course I said I do use duplication, but DrivePool duplication still lacks any ability to verify if the files are complete and uncorrupted. For that reason, I have gone to using those .par2 files for file verification. My backup HDDs are stored in my closet. It's not the best solution to my backup needs, but it is the best I have found for me at this point. In a more perfect world, DrivePool would have the ability to duplicate folders for faster pool recovery, and there would also be some way to verify and rebuild lost data like the .par2 files. In your case, if you have good backups of your data, I might consider turning off duplication in DrivePool, Rebalancing the pool and/or forcing a Recheck Duplication to clean up the data, and then turning duplication back on for the folders/pool as you want. But before I did that, I think I would contact the programmer directly for support and ask him for his recommendation(s). DrivePool is a great program and data recovery is much better than other methods I have used such as RAID systems and Windows Storage Spaces. But I do run into errors like you are experiencing and I cannot always understand the corrective action to take. Mostly, I have found that DrivePool is able to correct itself with its various troubleshooting tasks, but it might take a long time on a large pool.
  19. Shortly after I switched over to DrivePool, I had a number of old pool drives that started to die on me. I was getting those random corrupt files and then my drive would just die. The drive monitoring programs, like Stablebit Scanner, or Hard Disk Sentinel which I use, only provide so much protection. I replaced all those failing drives, and so far, my problems have gone away. However, some of the lessons I learned during that period may be of use to you. First of all, for as good as DrivePool is, it does not tell you if the file has been corrupted. My first clue to corrupt files is when they failed to read off the drive. Too late by then, which I imagine is what you are dealing with also. Second, although you can set any or all directories to be duplicated, DrivePool itself does not verify if any of the copies are not corrupted. In theory, you could have a corrupted file and DrivePool will just duplicate it. Who needs a copy of a corrupted file? From what I understand, if DrivePool sees a difference in the copies, it will ask you if you want to delete the older versioin. But that may, or may not be, the corrupted file. I know some people use SNAPRaid to provide parity checks on their DrivePool, and that appears to be a step forward in data recovery. But again, if your original file was incomplete or corrupt, you would only be restoring the bad files from the parity checks in SNAPRaid. Considering all of this, I ended up using QuickPar and MultiPar to create 10% par files to verify and recover my directories. It is an added step and takes time, but the advantage is that I am able to verify if my directory files are all intact and complete, and if not, with 10% recovery files, I am able to recover from most problems. If I had any critical files to protect, I would maybe use 50% or more par recovery files. But I mainly use DrivePool as my home media storage center and my files are backed up offline and not very critical if I should lose them in DrivePool. There is an option to create an index only file with MultiPar and QuickPar, and that is only good for verifying if the files are still intact. But the index file has no recovery blocks. QuickPar and MultiPar are good for verification and recovery on a directory level, like maybe to verify all the tracks of the music album are intact, but I have not been able to figure out how/if I could use these programs on a drive or pool level basis. So, it's not my ideal solution. But if works better for me than the other options I have tried.
  20. Well, I did not take it that way, so nothing to apologize for from my perspective. Even though the full format routine might take awhile, I think you will be better off for the effort. From my experience, data loss can sometimes take a lot longer to recover than a few days. Data corruption on a pool can sometimes go undetected for a long time, and repair might be very difficult. So I always take the time upfront for a complete full format with new drives before I add them to my DrivePool. I ended up getting Hard Disk Sentinel for monitoring my drives, but, in general, I also think these monitoring programs mainly read drives and look for problems that way. Having said that, I did buy the Pro version of Hard Disk Sentinel and it has testing programs that read and write to the drive. Sometimes you are able to put a "failed" drive back into service after you run a program to "fix" the hard drive. In that case, I think it hides weak sectors from being used and resets the drive less those failed sectors. I was able to put one drive back into use after it passed that test and recovery routine, so I figure Hard Disk Sentinel already paid for itself on that one drive recovery. Most of my drives that go bad are not able to be recovered. I guess it depends on what is going bad with the drive and I won't pretend to know more than that. If I get a report that a drive is starting to fail, I just replace it if the Hard Disk Sentinel testing routines fail the drive.
  21. There are many options to the balancing settings. It took me a while to figure out what works best for me. Once I got to my happy place, I have not changed it. Sounds like you dialed it in faster than I did. Just wanted to mention that when I was changing DrivePool settings, sometimes I would get unexpected results. I found that if I restarted my computer, DrivePool would come up fresh and things worked as expected.
  22. OK. Hope you get your answer on the forum. If not, maybe a support ticket....
  23. Hello SCB, welcome to the forum. If your HDD has never been formatted, it is still recommended to do a full format on a new drive. That way the OS will check the HDD for any bad sectors and mark them not to be used. A quick format should be good enough for a HDD that was formerly fully formatted. The quick format does not check for bad sectors, so if you have a new drive with only a quick format to save time, you could end up sending data to bad sectors. Bottom line, IMHO, it is far better to use the full format on a new drive even though it may take days. It would be far worse to find corrupted files on the HDD at a later date and sometimes those errors can cause the drive to crash. Just a friendly suggestion, it would be better to open a new question/thread on the forum for your question(s) then to tag onto an old thread that ended 6 years ago. Even if the question has been asked before, the solution could have dramatically evolved over the years. Again, welcome to the forum and hope to hear more from you in the future.
  24. I had a similar problem on my computers. Everything was working fine with the remote control for months on my computers. I upgraded DrivePool on my main server (with license), and then none of the remote controls (without license) worked on my other computers. In my case, I was able to update DrivePool on my remote computers and once all computers had the same DrivePool version numbers, they worked again. If you have additional plug-ins in use, you need to also install them on your remote computer(s).
  25. If Scanner is suggesting the drive might fail, then I would try to move that data off and get that drive offline as fast as possible. Once offline, then I would suggest running some diagnostics on the empty drive. You already have Scanner, but I would also suggest downloading the free version of Hard Disk Sentinel for a second opinion. In my experience, once these monitoring programs detect something they don't like, it is best practice to assume imminent drive failure. Only once did my programs detect a false positive on a drive. I removed all data from it and took it offline. Fortunately, I was able to put that drive back into service after running a number of diagnostics on it to verify it was still good. All my other drives failed within a few days from first detection of any problems, and one drive just failed without warning. There are just so many attributes that these programs monitor that I don't really understand why "similar" reading on drives might indicate pending failure on one drive but not on the other. And I have also seen all Green attributes on a drive that is reported as pending failure. So there might be attributes way down at a much lower level than we see that are causing concern.
×
×
  • Create New...