Jump to content

gtaus

Members
  • Posts

    285
  • Joined

  • Last visited

  • Days Won

    20

Everything posted by gtaus

  1. In theory, cloud based storage seems to be a great idea. In practice, I have only experienced extremely slow upload times and have never considered it a viable option for mass storage or backup, like for my home media server with ~60TB of data sitting in DrivePool. Worse yet, my internet provider throttles back on connections to such cloud based services unless you pay for their extremely high business connection accounts. So the only data I backup to the cloud is relatively small financial and document files that I might want to share across mobile devices. I expect that any solution to removing your cloud data I might offer may not be fast enough. But there are some options I'll mention just the same. When you click remove drive in the DrivePool GUI, one of the options is "Duplicate files later (faster drive removal)" which might reduce the amount of files DrivePool will have to offload. If I understand that option, DrivePool will skip over the cloud files that it has a copy of locally. Then, after all non duplicated data is removed from your CloudDrive (slow drive), it will recheck your DrivePool duplication requirements on your folders on your local drives and perform that task in the background. If you are worried about data loss during the slow removal of your CloudDrive files, I think there is yet another option to consider. Let's say you have a duplicated 2X Movie folder that you want to protect the data as you remove the CloudDrive. Maybe 1X copy is on your local (fast) drives and the other 1X copy is on your cloud based storage = 2X copies. What I would suggest, is that you create a local copy of your 2X data on a separate 1X temp folder in DrivePool - if you have enough room. DrivePool can use Read Striping on 2X duplicated folders to increase read speeds and will write the data to your local drives - assuming you have already told DrivePool to remove the CloudDrive which should block it from any data being written back to that drive. Copying your 2X folders to a separate 1X folder should be a lot faster than offloading data from the Cloud, and if things went bad with the Cloud offload, then you still have a backup of all your 2X data in your local 1X temp backup folder. Once you have told DrivePool to remove your (slow) CloudDrive, it should block any data from being written back to it. At that point, if you copy your 2X folder to a separate 1X temp folder, it could use Read Striping to make a copy of the data to your local drives using the fastest drives for duplicating the data. One of the Balancers is the Drive Usage Limiter, which allows you to designate each drive for either duplicated or unduplicated data, neither, or both. I have not yet switched over to the All-In-One Balancer, but I think the concept is similar. That might present you with additional options as you move/duplicate data around. In any case, your CloudDrive is probably showing up as only 1 drive in DrivePool, and therefore should not have more than 1 copy of your 2X files on your CloudDrive. When you copy data from a 2X folder with Read Striping, DrivePool seems to pull data off the fastest drive possible and uses less data from the slower drive. If your cloud data transfer rate is really slow, then DrivePool will be pulling almost all the data from your local faster drives. I have been using DrivePool for less than 1 year, but the more I learn about using it the more I like it. There are many options available with DrivePool that I did not have using hardware RAID controllers or Windows Storage Spaces. The biggest selling point for me was data integrity and recovery in DrivePool. I just had too many problems with Storage Spaces and my life is much better with DrivePool. Whatever software you decide to go with, I hope you find what works best for you.
  2. I don't know why you saw so much memory usage on your first install of DrivePool. I have never seen anything like that. Is it possible that DrivePool was performing some intensive task at that moment? I am glad that Christopher was able to get you back on track and that things are going much better (normal) now. A long yellow bar usually means DrivePool activity or an error state - like duplication needs to be re-checked. The green bar usually means everything is OK with DrivePool and does not need any attention. If you have a green bar, then I don't understand why you would be looking for a cancellation option. Is there some task running in the background? Your screenshot of DrivePool looks all good to me. Speaking of background tasks, I have suggested that if would be nice to have an option to select that more information be displayed of what is going on in the background tasks running in DrivePool. Many of those tasks and status on them seems to be hidden, maybe for a good reason. Most of the time, I'm just happy to let DrivePool do its thing in the background and I don't need to see the inner workings. But, it would be good to have an option to get more info and status on those tasks for those few times I do care to understand what is going on behind the curtain. Under normal circumstances, DrivePool pretty much takes care of itself. However, I have noticed a few times that it appeared to me that a task like balancing might not be behaving properly. Don't know if it was a DrivePool issue, or a Windows issue, but I rebooted the computer and everything went back to "normal." That's pretty much my first step to take if I notice something fishy going on with my computer. Again, I don't know if it's a DrivePool issue, or a Windows issue. As you know, Windows can lock itself up for unknown reasons and a fresh reboot seems to correct the OS. And that certainly has nothing to do with DrivePool. I would also suggest keeping an informal log (note on piece of paper) of what was wrong and how you corrected it. If you see a pattern of misbehavior, you could share that with Christopher and maybe he could look for a bug to correct. One thing I appreciate with DrivePool is that the programmer(s) are looking for ways to improve the software and they issue new releases.
  3. From what I have read, many people use PrimoCache without any problems. If you have lots of RAM, it just works better. PrimoCache lets you designate how much, if any, RAM and/or SSD you want to set aside for the cache/buffer. The free trial period is long enough to see if it works for you. I only run my own home systems, so third party apps is not as issue as it might be at the office. I am surprised to hear that DrivePool is taking up so much memory. I just checked my server's Task Manager and my DrivePool is reported as using only 40MB of memory. I have 8GB RAM, so no issue for me on my 6+ years old HP desktop running Windows 10. If you don't have StableBit Scanner, which will report real serial numbers, you can download the free version of Hard Disk Sentinel and that will also report real serial numbers. I don't know if that will help you locate and identify your drives in your enclosures, but if you pull a drive and put it into a USB disk caddy, you would be able to identify each drive that way. I can't remember exactly how Storage Spaces lists the drives, but somewhere along the line I got smarter when I moved to DrivePool and the simpe naming of the drives as DP01, DP02, DP03 works better for me. Don't forget to put a matching tag on the drive/case itself. I ran Storage Spaces for ~7 years and when it works, it is great. But when it fails, it can be epic. I had 3 catastrophic failures with Storage Spaces. On my last failure, I had 26 pool drives, Storage Spaces was set up for 2 drive failure, but I had only 1 HDD fail and it crashed my entire pool. OK, my home media server is not vital to life and I had HDDs as backup in the closet. But I just lost confidence in Storage Space's promise to rebuild itself after a HDD failure. 3 strikes and I was out with Storage Spaces. I moved to DrivePool after Storage Spaces. The read/write speed of DrivePool is limited to the speed of a single drive whereas Storage Spaces can read/write packets from multiple drives at once. So, at first, Storage Spaces appeared to be faster than DrivePool. However, I added a SSD as front end cache to DrivePool and now I get faster write performance on DrivePool than I ever got with Storage Spaces. If you have 2X duplication on your DrivePool folder, it can take advantage of read striping to increase read speeds. Occassionally, I will set up a 2X TEMP folder just to take advantage of Read Striping for some temp folders of working files. Since I have moved to DrivePool, I have had 3 HDDs fail. Those have been older drives I moved over from my Storage Spaces days and they were at the end of their life cycle. On 2 of those drives, I recovered almost all my data except for a few corrupt files. That made recovery really easy. On the other drive, DrivePool recommended that I run Scandisk on the drive because there was a problem reading the directory. So I did, but Scandisk (with repair) wiped out the entire directory and I then lost access to all the data on the drive. DON'T run scandisk with repair on a HDD until after you have transferred as much data off of it as possible. At any rate, DrivePool stores complete files on the HDD and not just packets of files like Storage Spaces. So, under many failures, you can still recover almost all your data off a DrivePool drive. I am still relatively new to DrivePool (less than a year), so I am reading posts here all the time trying to learn more. I try to give back if I am able. @Shane , a moderator, has been very helpful on many of my questions. Responses are always appreciated.
  4. @TPham You simply click the mouse and drag the cursor over the text you want to copy. The text should turn to blue background. When you get to the end of your selection, release the mouse and a popup box should appear "Quote selection". Click on the popup box and it will transfer that quote to your response box. Unlike Windows File Manager, you don't need a drive letter for DrivePool to see and list the drive. I just added another HDD to DrivePool this morning, without a Drive Letter, and all I had to do was simply click the add button on the Non-Pooled drives list on the DrivePool GUI. I currently have 18 HDDs and 1 SSD in my DrivePool. So I wanted to remove all drive letters from my pool drives and free those Drive Letters for flash drives and/or other devices I might want to temporarily add to my system. If you ever need to work with any drive in Windows, you can go into Disk Management and add a Drive Letter to any pool drive you want. DrivePool does not care if you add/remove a Drive Letter to the drive, it identifies the drive by the hidden PoolPart folder it writes to the drive itself. That is one way of organizing your duplication folders that I have also considered. It certainly would clean up the root directory to only 3 folders. I really noticed a performance boost in DrivePool when I added a front end SSD. DrivePool is limited to the speed of the drive it is writing data to at that time. If it is writing to a SSD cache, then it's very fast. When I transfer large amounts of data, I am glad I have that SSD front end cache. DrivePool can use a SSD as a front end cache for writes. That works good enough for me. If you are a power user and need more speed in your server, I would suggest you take a look at PrimoCache. PrimoCache can use both your system RAM and your SSD as cache/buffers for reads/writes. I believe PrimoCache still offers a free trial period to use their software to see if it works for you. Years ago I tried PrimoCache, but I only had 4GB RAM and no SSD on my system. Writing/reading from RAM was lightning fast with PrimoCache, but with limited RAM, the buffer filled up in no time with large data transfers (>3GB at that time) and dropped back down to the slow write speed of my archive HDD. So it was not worth it for me at the time to buy the program. Your situation is different with 16GB RAM and 2 SSDs. I'd recommend checking out the program at least for the free trial period. I use DrivePool primarily as my home media storage server, so using the SSD as a front end cache is enough for me. Even without using the SSD, DrivePool was just fine for my needs and I had no complaints. The SSD just caches the writes faster on large data transfers. Sometimes that comes in handy. Yeah, I had a hard time trying to figure out what drive/serial number I was working with in Storage Spaces. If you buy a few of the same HDD models, only the serial number may be different, which was a real pain for me. Anyway, when I set up DrivePool, I just named my HDDs DP01, DP02, DP03, etc... I placed them in physical order on my shelf and I also tagged each drive/case with the drive name. The drives I have in my ProBox enclosures are also in physical order, so I know exactly which drive is in the slot. My life was easier when I got away from using serial numbers to identify the pool drives. I don't have any magic way to identify your HDDs with serial numbers in your enclosures. Sometimes the software will have a feature to "identify" your drive which might flash a light on the drive slot if your enclosure has that feature. Mine did not. If you have a USB disk caddy, you could pull your enclosure drives and identify them in the caddy. I'd label the drives at that time, if needed, before returning them to the enclosure. But, like I said, with DrivePool, I just tagged and named the drives in the physical order that they are placed in the enclosure and/or on the shelf. Anything to make like simpler works better for me.
  5. Thanks for the response, @Christopher (Drashna). I lsearched the tool online, and it says that the command line tools are built into DrivePool v2x, but I cannot find the tool. How do I access the DrivePool command line tools? I can't find it on the DrivePool GUI nor in the C:\Program Files\StableBit\DrivePool directory. Thanks.
  6. If setting the drive volume label to match the drive's serial number works for you, then stay with it. My approach is different and works better for me. First of all, I currently have 18 USB HDDs in my DrivePool. I don't use Drive Letters on any of the pool drives. There are just too many drives in the pool and DrivePool does not need Drive Letters anyway. I preferred to clean up my Windows File Explorer listing, so the Drive Letters had to go. What I do is just name the drives in logical order as they sit on the shelf. Being not too creative, my pool drives are labeled as DP01, DP02, DP03, etc.... I also put a label on each drive case. I have DrivePool GUI sort the pool list by name for easy reading. If there is any problem with a HDD, I immediately know which drive is affected. I have lots of unduplicated home media files in my DrivePool. But I also have a few folders that I want 2X duplication. Not only do I find the duplication options better in DrivePool than Storage Spaces, but the net result is that I am saving lots of money by not duplicating my entire pool when it is not required for about 85% of my stored media files. Also, I have had a couple HDD failures in the past month, and I have been able to recover almost all my data off the drives. In the meantime, DrivePool was still serving up all my other files like nothing happened. When my Storage Spaces crashed, it could takes weeks to rebuild. I don't miss Storage Spaces....
  7. Thanks for the response @Christopher (Drashna). Is "dp-cmd"'s "ignore-poolpart" command a command line prompt? If so, is there a section in the DrivePool manual, or elsewhere, that explains how to perform this task. It sounds like a much more sophisticated method to eject a trouble drive than actually pulling it out of the rack.
  8. That's the trouble with computers. It can always be a bad something that is not detected. I am not too concerned that 4+ year old HDDs are dying, I kind of expect they have reached the end of their life cycle, and certainly beyond their warranty period. So far, it's only my older HDDs that have failed. If I had a bad cable or controller, I would think that would affect my new drives as well. It has not. If anything, I suspect my external USB drives are running too hot. They have passive cooling and it might not be enough. I am running Hard Disk Sentinel and I see that my external USB drives are always hotter than my drives in the MediaSonic ProBoxes. Of course, the MediaSonic ProBoxes have a cooling fan so they should not overheat.
  9. This is a very new plugin. The guy who wrote the plugin started a thread here a number of weeks ago and has been working on his "All In One" balancer plugin. I know he recently submitted a minor update to fix a small bug, but I hear it's working good. I have been having some HDD failures and subsequently am swapping out dead drives with new drives. Once I get my DrivePool back up and running "normally", I will be trying out that plugin, too. It sounds like a better option than having 3 or 4 balancer plugins fighting each other.
  10. @Shane, for your info and consideration... UPDATE: I just finished manually moving off all good data of this drive. I am happy to report that all but 12 small corrupt files were able to be saved. It took me ~36 hours to remove the data manually, but in the end, almost everything was saved. After all the uncorrupted files were moved from the HDD (leaving only 12 small corrupt files), I tried once again to remove the drive within the DrivePool GUI, checking all 3 boxes to remove the files and leave the damaged files on the drive. Again, DrivePool was unable to remove the drive and errored out. It appears to me that DrivePool errors out the first time it hits a corrupt file and stops the remove task altogether. I don't understand what difference it makes to mark those checkboxes to remove a damaged drive and leave the damaged file on the drive. It did not work in my case. Moving on... Clicking on the error details in the DrivePool GUI, I got the message once again to run chkdsk on the drive to repair the problem(s). This time I knew that I only had those 12 corrupt files left on the drive, so I first ran chkdsk without repair. It reported that there were errors on the drive and I needed to run a repair. So I ran the scandisk with repair, Windows fixed the drive, but deleted all the remaining folders and files on the drive. It also deleted the PoolPart directory. I mention this because, when DrivePool GUI errored out on the remove disk task, it told me to run the scandisk repair. IF I would have run scandisk with repair when the HDD was full of files, and the handful of corrupt files, I think it would have wiped out my entire drive like it did last time with the other drive I had that recently failed. SUGGESTION: I suggest that if DrivePool cannot remove a drive from within DrivePool's GUI, then the error details info box should state something like "Remove files manually first and then attempt a scandisk repair." I was going to test out my question #2... ...but the scandisk repair deleted all folders and files, including the PoolPart on that drive, so it was no longer in the pool and I could not test to see if DrivePool would have written new files to a drive that was marked for removal and in an error state.
  11. Yes, I noticed that you have 3 10TB drives. I don't know if 50% is some magic number for balancing new drives. I just wanted to comment that when I add a new drive if starts to fill up part way, but stops far short of "perfect" balancing the drives in the pool. Maybe the developers will comment on what we should expect on this balancing issue when we add a new drive. As I said, DrivePool balances the pool drives about as good as I saw with Storage Spaces. Not perfect, but good enough for me. Over time, they seem to level out. Well, again, that's an issue for the programmers of DrivePool. I don't know how they set DrivePool to balance the drives when you initially add a new drive. It is my understanding that DrivePool defaults to adding files to the drive with most free space reported at the time. In that manner, your drives should be in balance over time. That is what I see on my DrivePool, and it works for me. I have read, and re-read, the Stablebit DrivePool manual on this issue, but I find the information rather general in nature and it does not appear to dig down as far as you want. But maybe give that a look and something might pop up for you.
  12. FWIW, I have seen the same thing on my DrivePool when I add a drive. DrivePool will "balance" the pool, filling up the new drive to a certain point, and then stop. My pool drives never really show up as perfectly balanced, either. I had assumed this was because I have a mix of 3TB to 8TB drives and maybe DrivePool was using some magic formula to balance them, doing the best it could with drives of various sizes. Over time, my new drive will populate first and eventually the drives in the pool look fairly well balanced. It is certainly not as "balanced" as my hardware RAID system, but about as "balanced" as Windows Storage Spaces drives when I was using that system. Some people prefer not to have their pool drives "balanced" at all. There is a system for file order placement, where DrivePool will fill up the oldest drive first, and then move on to the next drive in the list. I have not used that method of file order "balancing" on DrivePool, but that system offers some advantages that I am considering might be a better option for me and my data. DrivePool also offers file and folder placement rules to send assigned files/folders to designated drives. Which, for some people, I am sure is a great option. There is an option in DrivePool to manually initiate a re-balance task. I would be curious to know, if you manually initiated a re-balance task, if your drives would look more balanced upon completion. Since all 3 of your drives are the same size, that would be an interesting test. There is always the option to manually move a chunk of data from PoolPart on drives 1 and 2 to PoolPart drive 3 with less data. That would work for the unduplicated data. I don't think you want to touch your duplicated data unless you plan on having DrivePool recheck duplication after you move the files around. But you might be able to force a balancing on your own terms that way. For me, I'm OK with the way DrivePool handles the balancing. Over time, the drives will all look pretty much the same.
  13. Thanks, @Shane for the answer to #3. Just wanted to be sure before I messed up things.
  14. I am currently having similar issues with removing drives from DrivePool related to corrupt files and/or directory issues. DrivePool is erroring out on the removal of the drive and stops. If DrivePool tells you to run chcdsk on the drive, I would suggest you DO NOT follow that suggestion. I ran chkdsk on a failing HDD last week and chkdsk corrupted the whole directory and I lost access to all data on that drive. At present, I am vacating another problem drive manually using TeraCopy. TeraCopy is better than Windows File Manager because when it hits a corrupt file, TeraCopy will skip over that bad file and continue to move all the other files. MS File Manager, when it hits the first problem file, will stop everything and not continue to run the task until you clear the error box. In other words, you have to babysit Windows File Manager through the whole process whereas TeraCopy will complete the task and then give you an error report of the problem files. With high capacity TB drives, my current file transfer estimated time was about ~12 hours. With TeraCopy, I can just let TeraCopy run overnight knowing that it will complete the task and give me an error report on the corrupt files it could not move the next morning. TeraCopy has an option to filter the files to those that it has skipped, or failed. In my currently running transfer, I filtered the report output to show my failed file transfers and it lists the complete file directory path of the bad file and the description of the error along with an error code. That should give you some idea of the bad directory you are looking for. Good luck.
  15. Last week I had a 5TB HDD failure and tried to remove the disk via DrivePool's GUI. It was unsuccessful. I did get an error message to run chkdsk on the HDD to correct the problem. Unfortunately, chkdsk corrupted the directory and I was left with no data on that drive. Total loss. Today, I have another 6TB HDD that is misbehaving according to DrivePool. So, once again, I tried to remove the HDD using the DrivePool GUI. Again, it was unsuccessful and gave me an error message to run chkdsk on the HDD. NO WAY am I going to fall for that trap again. So I am currently moving data off that drive manually to other drives using TeraCopy. What I have discovered is that there are a few corrupt files on that HDD that cannot be moved. At least with TeraCopy, it will automatically skip over those corrupt files and continue to move the rest of the files on the list - and then give you an error report of the failed files at the end of the task. So I don't have to babysit the transfer of files over the next ~10 hours. Questions: 1) Why is DrivePool unable to remove the drive when I have all 3 boxes checked to remove the drive and just to leave the failed files on the HDD? It appears to me that the remove task is erroring out when it hits the first corrupt file and does not try anymore. 2) If I have a HDD checked for removal, and it errors out, does DrivePool lock out that drive from the pool and not allow any new files to be written to it? I have ordered a couple new drives to replace these drives, but can I continue to use DrivePool without worrying about data being written to the drive marked for removal? 3) Since I am unable to remove the drive from within DrivePool's GUI, can I physically pull that drive out of the pool, then when DrivePool lists it as a missing disk, can I remove it then? After that, am I able to manually transfer files off that pulled disk back into the pool as is, or do I have to rename the PoolPart directory on that drive so DrivePool does not see it as part of the pool again? I don't believe there is anything physically wrong with this current 6TB HDD. It reports at 100% Health, 100% Performance, and No error reports in SMART. Maybe related to the 5TB HDD failure of last week, for some reason, I got a few corrupt files thrown on this 6TB HDD. My intention if to vacate all data on this drive, run a few integrity tests on it for good measure, and assuming it passes, I will reformat the drive and put it back into DrivePool. Let me end with saying something positive about DrivePool. Although I am having a few issues with a failed drive and now some corrupt files on (what appears to be) a good drive, with DrivePool I am still able to manually vacate the files off the drive, leaving only a few corrupt files on the drive that are causing me a problem. When I had problems with my old RAID and Storage Spaces pools, it was game over and I lost everything. This is another case that confirms my decision to move to DrivePool, and that is when you do have a HDD problem, chances are good with DrivePool that you might be able to minimize your loss and recover almost all your data.
  16. Yes, DrivePool, like many of my Windows programs, sometimes hangs and requires a reboot. Most of the time DrivePool works without any problems, but I have run into some circumstances where DrivePool misbehaves and does not correct itself until after a reboot. I ran Windows Storage Spaces for ~7 years, and the small problems I occasionally experience with DrivePool are nothing compared to the problems I had with Storage Spaces trying to manage the same size pool (currently 70TB). IF I had any real complaint about DrivePool, it would be that it really keeps you in the dark on background tasks it performs. I personally would like more status info displayed in the DrviePool GUI in those cases, because background tasks might have a really low priority and it may look like nothing is happening. Well, it may be happening but not very fast. Or, maybe the task got hung up and needs a reboot. Sometimes you can go into Task Manager and check for disk activity there. I have done that on occasion to verify activity was really going on in the background tasks. It would be nice to have the option of seeing some of that background activity on the DrivePool GUI. Most of the time I don't care as long as it gets done. Sometimes I want to verify work is actually going on. FWIW, I had the same idea as you when I was testing out DrivePool. I did a couple disk removals, but they worked just fine on my system, so I started off with a positive experience.
  17. I have become really cynical on any system that has not been verified after failure(s). Like I said, my Windows Storage Spaces had 26 HDDs in the pool and was setup to recover from a 2 drive failure. I had 1 small HDD fail and it took down my entire Storage Spaces. Since Storage Spaces uses striping, all data was lost. That was the third time Storage Spaces let me down in that respect, and then I moved to DrivePool. I have had much better results with DrivePool. Yes, I have suffered 2 HDDs failures (but not at the same time) in DrivePool. The first time I was able to offload all but 2 corrupt files from a 5TB drive. The second drive failure resulted from me following MS Windows' recommendation to run a chkdsk on a failing drive - that resulted in a corrupted directory on that 5TB drive and I lost all access to the data on that drive. If I would have just offloaded the data from the failing drive, I would have probably saved almost all my data on that drive too. But for some reason, chkdsk corrupted the directory and it was game over. I only have a few folders set to 2X duplication, but the files were indeed safe after my HDD failures. I don't quite understand the way DrivePool knows where the good copy of the file is, but the system seems to work. All I know is that 2X duplication means DrivePool will write the file(s) on 2 separate drives, ditto for 3X on 2 different drives, etc... The more I use DrivePool and understand how it works, the more I like it. I decided to get Hard Disk Sentinel instead of Stablebit Scanner because Hard Disk Sentinel has HDD repair features in the paid version. I figured if I can "fix" one HDD with that software, then it more than paid for itself. I am happy to state that I had 7 failed HDDs in a box that had failed over the years in Storage Spaces, but I was able to "fix" 2 of the 3TB drives with Hard Disk Sentinel and put them back into service. So the program has already paid for itself. My latest 5TB HDD failure is sitting in my USB caddy and Hard Disk Sentinel is running a "Reinitialize disk surface" program on it to see if it can renew the drive. It is a very in depth testing program, testing every single block, and is currently estimated completion time of 50 hours, but I have already been running it for the past 47 hours. So maybe later in the morning I will get a final report if the drive can be put back into service. So far, all blocks have passed the test, but the graph does show some portions of the drive are slower than others. I think Hard Disk Sentinel "magically" tells the drive not to use weak or failing sectors and that is how it is able to renew some of the drives (but not all). The YouTube video I watched did mention that the parity drive had to be the largest drive. I guess that would not be a problem. However, if you only need 1 parity disk to cover the first 4 data disks, and then only 1 more disk for the next 7 data disks, I may look into SnapRAID some more. I currently have 17 disks in my pool, so I guess that would require 3 parity disks, right? I was thinking back to Storage Spaces where I would need 9 parity disks. I don't want to do that, but DrivePool with SnapRaid (using only 3 parity disks) might be a good option. Thanks.
  18. I watched a 3-year old YouTube video on DrivePool + SnapRAID, but there he was using drive letters for SnapRAID. I know DrivePool creates PoolPart directories, does SnapRAID create its own separate PoolPart? In SnapRAID, what is the ratio between data drives and parity drives? For example, in Windows Storage Spaces, it would take 1 parity drive to 2 data drives of the same size. In other words, 33% overhead loss for parity. That is, of course, better than 100% overhead for a mirrored array. One big advantage of DrivePool is that you can just assign certain important folders for duplication. In my case, that amounts to less than 5% overhead. Have you ever had to recover from a loss of a pool HDD using SnapRAID? From the YouTube video, it looked like a very long rebuild time for his small pool. My DrivePool is currently 70TB and I am wondering if it would be faster to rebuild from SnapRAID or my backup HDDs stored on a shelf in my closet. Same here. I have 2X duplication on some financial and personal data folders that I would hope to rebuild directly in DrivePool. But I also keep backups of those (relatively) small folders on a Cloud backup. Most of my DrivePool is just a 1X home media storage, but I can rebuild lost files from my offline HDD backups. With the large data HDDs we use these days, a loss of a single HDD can get you into trouble fast. In theory, SMART should give us some warning before a HDD fails, but in my experience, 80% of my drives have a sudden death with no real warning. So I am trying to develop a recovery plan that would work for me for then next time I have a pool HDD failure. Still considering all options.
  19. In the past, I have used hardware RAID arrays, then I moved to Windows Storage Spaces (software ~RAID), but now I use DrivePool for my home media storage. Hardware arrays can be very fast and you can get great I/O performance. Windows Storage Spaces uses striping and, depending on the pool type you setup, you can get fast I/O performance. However, if you have drive failures in either hardware RAID, or Windows Storage Spaces, data recovery can be next to impossible because they strip their data over many drives in the pool. Despite having 2 drive failure set on my Storage Spaces, I lost 1 HDD and it took down my entire Storage Spaces pool of 26 drives - lost everything. DrivePool is limited to the read/write speed of the HDDs you use. If you don't have duplication on your pool/folder, then you cannot take advantage of read striping. However, I recently added a SSD to the front end of my DrivePool and set it to cache 100GB of data. So now, I can use my fast SSD cache for DrivePool writes, and as a temp directory for read/writes for certain programs I use. I am now getting faster I/O performance than I ever got on my older RAID and Storage Spaces systems. If you really want to step up to higher performance, then I would suggest a product called Primo Cache which can use both system RAM and SSD as cache for your system. I think they still offer a free trial period. If I was running a server with lots of I/O requests, I would certainly consider purchasing Primo Cache. However, just adding a SSD to the front end of my DrivePool more than meets my needs for my media home storage. No, DrivePool does not create more than 1 copy of any file unless you tell it to duplicate the pool or that folder. However, the power of DrivePool is that you can designate which folder(s) you want duplicated and you don't have to duplicate the entire pool. So if you have certain folders that you will need higher I/O performance, you can just set them to 2X or higher duplication and take advantage of Read Striping. (I don't know if DrivePool would take advantage of 3X or 4X duplication for read striping from 3 or 4 drives.) In my case, if I need to use a higher performance I/O cache for working programs, I will flush my 100GB SSD cache in DrivePool and then direct my temp directory back to DrivePool. 100GB of SSD cache is more than enough for me, however, Primo Cache would be even faster as it can use system RAM as well as your SSD. I find DrivePool more reliable than my older hardware RAID or Windows Storage Spaces solutions. Data recovery from a drive failure is better with DrivePool. Adding a SSD to the front end of my DrivePool makes it faster than I ever got with my other systems. I only add 2X duplication on certain folders I don't want to have to rebuild from backups, but most of my media storage DrivePool is only at 1X and that saves me lots of money on drives. DrivePool just works better for me.
  20. I ran Windows Storage Spaces for ~7 years before I moved to DrivePool. IIRC, Windows locks out all access to the pool drives in Storage Spaces, and as you stated, the best you get is one large drive displayed with most programs. At that time, I don't think I was able to get any disk monitoring programs to work with my Storage Spaces pool of drives. If StableBit Scanner does not work well with Storage Spaces, you might want to download the free version of Hard Disk Sentinel to see if that program can detect the individual drives in the Storage Spaces pool. I know Hard Disk Sentinel detects all my pool drives in DrivePool and none of them have drive letters assigned. But, that may not be the same as Storage Spaces which takes total control over the drives because of the packet writing feature of Storage Spaces. It's worth a try and won't cost you anything but a little bit of time to see if it works. Good luck.
  21. Right, I was re-reading the File Placement rules for folders in the DrivePool online manual, and I think might work better for me if I go that route. Thanks for explaining that to me. I looked up FreeFileSync after you mentioned it, but I did not realize that it had the ability for versioning as you explained. I can see where that would really be a nice feature for some uses of DrivePool. In my case, DrivePool is primarily just my media server. Also, I am not yet at a point where I want to build another 70TB+ pool to backup my current 70TB DrivePool. I could see FreeFileSync as a backup solution for the small part of my DrivePool that is not media files. I can now see what you mean by using DrivePool's duplication in case of drive failure, but using FreeFileSync as a backup solution for human error. If I accidently deleted a duplicated file in DrivePool, I would be short on luck trying to get it back without some kind of versioning option like that in FreeFileSync. I'll be looking at that more closely. Thanks.
  22. I agree. I see shortcomings in my current backup methodology and as future HDDs increase in size, my current method of backups is totally inadequate. That would certainly be better than my current backup system. I have been rethinking my DrivePool balancing and have been considering moving to the ordered file placement. For some reason, I thought DrivePool would move an entire folder to one drive, but what I find is that files from a folder could be written to a number of different drives. I guess that makes certain sense from a balancing point of view, but it makes it harder to recover from a loss. OK. I think I understand your backup strategy with FreeFileSync. What advantage do you get from having a second server with a 1:1 backup via FreeFileSync that you could not get by just setting your DrivePool to 2X duplication from the start? Do you keep your second server online all the time, or do you only turn it on for your 30 day updates? Yeah, that's what I grew up with too. Those backup tapes kept you busy, but if you had to actually recover from a loss, it was almost hit or miss and success was not guaranteed. Fortunately, I have all my "important" financial and personal files duplicated 2X or 3X in DrivePool and on the cloud, and my original media files are stored on HDDs in the closet. So, I have not lost anything per se. It's just that this HDD failure has shown me I need a better way to plan for future problems. As always, thank you @Shane for your helpful responses.
  23. UPDATE: I have figured out how to use the command line prompts with WinCatalog 2020 and run nightly updates on DrivePool using Windows Task Scheduler. That will update my virtual DrivePool J: volume every night, but it does not tell me what file is in which specific HDD. If I need to go into the detail level, I would have to turn on the Drive Letters of the pool drives and also update them every night. I don't think WinCatalog 2020 can scan a drive without a Drive Letter assigned, but I'm still looking into that possibility. Assuming I might only lose one HDD, at least running an update report on WinCatalog 2020 on the entire DrivePool J: volume would tell me which files had been deleted from the HDD failure. That gets me closer to a recovery plan.
  24. FWIW, I have the same Windows 10 and update version you reported. I have 17 HDDs and 1 SSD in my DrivePool. None of them have Drive Letters and everything is working fine on my DrivePool system. IIRC, DrivePool does not even use Drive Letters, so I don't understand why turning on the Drive Letters makes any difference. I did not move my DrivePool from another OS. My DrivePool was initially created in Windows 10 and has always been on the same computer. Now that you are running DrivePool on a Windows 10 machine, have you considered just rebuilding your pool from the ground up in Win10? I know I really like not having to use Drive Letters for my 18 pool drives.
×
×
  • Create New...