Jump to content

gtaus

Members
  • Posts

    285
  • Joined

  • Last visited

  • Days Won

    20

gtaus last won the day on July 23 2023

gtaus had the most liked content!

Recent Profile Visitors

1819 profile views

gtaus's Achievements

  1. Yep, it took 25 hours to remove my 4TB HDD. As I stated, I could see the transfer rates going up and down in speed depending on the size of the files. Although speed is important, I have stayed with DrivePool for years because the program just works so well and does not crash like other systems I had including RAID and Windows Storage Spaces.
  2. I don't use duplication on my Home Media Center storage DrivePool. I have all the files backup up on HDDs in the closest. So, if I lose an HDD in DrivePool, I could reload them from the backups. Nothing critical if I lose a drive. As far as time required for removing a drive, I initially estimated 54 hours to remove my drive after the progress in the first 3 hours. However, my transfer rates improved over time and the remove drive task completed last night at about 25 hours total run time. I suspect it's a matter of what files are being transferred. Small files will take considerably longer to transfer than larger files. I can see the transfer rate drop way down on those small files, but build up speed on larger files. Thanks for the offer, but you don't have to perform any comparison tests for me. I am just hoping that DrivePool will someday include an "estimated time to complete" for the remove drive task. The % progress in DrivePool is OK, but it really does not give you any idea how long it will take to complete.
  3. That looks like a real nice 10-bay enclosure. When I started out with DrivePool, I had two 4-bay enclosures and a handful of individual HDD enclosures. Unfortunately, after a few years, one of my 4-bay enclosures died so I lost access to all 4 drives. It was less expensive for me to replace the 4-bay enclosure with 4 separate HDD enclosures, so that is what I did. Currently, I see the 10-bay enclosure sells for about $540 on Amazon. That's $54 per slot. Again, I am comparing that price to individual HDD enclosures at about $20 per case. I know the multi bay enclosures have many advantages, but I wish they could bring down the price. Also, for me, I don't think I would ever lose 10 individual HDD enclosures all at one time whereas I know that if the board in the enclosure goes down, I would lose access to all those drives. Do you think your 10-bay enclosure is moving files among the drives faster than what you could see if you had separate HDDs in individual cases? From your post, I don't see any great speed removing the drive within DrivePool itself. Well, frankly, DrivePool is not nearly as fast as my old RAID system or Wndows Storage Spaces, but DrivePool just works better, and I have had no catastrophic data failures as I did in the other systems. I am currently removing a 4TB HDD from my 23 HDD DrivePool Media Server. I calculate it will take about 54 hours to complete the task within DrivePool. That's a long time, and with my newer 8TB drives, I suppose it will take almost a week to remove one of those. I think it would be great if DrivePool had some kind of way to estimate how long it would take to complete the remove drive task. I see my remove drive task is clocking at about a 40 MB/s rate but I think the HDDs can transfer over 80 MB/s on their own. And I have an older computer with a USB 3.0 system. I'm sure the bottleneck in my system is the USB 3.0. So, just curious on what transfer speed you see on your 10-bay enclosure as you remove your drive. Thanks.
  4. That's the info I was just looking for, how to speed up removing a drive from DrivePool. Currently, I am removing a 4TB drive to replace it with an 8TB drive. I logged the progress of the task and it took 3 hours to move the first 5.5% of the drive. At that rate, it will take about 54 hours to complete the removal task. Good thing I started the removal process 3 days before the new HDD is to be delivered. Any idea on how much overhead DrivePool adds to the process in the drive removal process? With larger and larger HDD's, this is becoming more of an issue for me. I like to keep my DrivePool up and running while I remove a drive, so I plan for a good 3 days to remove a 4TB drive. But it would be nice to know how much longer DrivePool takes to remove a drive compared to stopping DrivePool and manually moving them yourself.
  5. gtaus

    SSD Cache

    Well, that's not what the people at PrimoCache told me. They made it clear to me that PrimoCache will not cache any network transfers because it uses a different protocol for network file transfers. I also don't see what caching benefit you would achieve using PrimoCache on the server. The bottleneck for speed on network transfers is usually the limited bandwidth speed of the network itself. My home network, for sure. Are there home networks that have data transfer rates higher than a write speed of a SSD? That's the only case where I could see a speed advantage to using RAM as cache with PrimoCache. Otherwise, you might as well write directly to a SSD/HDD over the network. It has been about a year since I last looked at PrimoCache for network transfers, and at that time it was not possible. Maybe they have made some progress on their end because I bet a lot of people would love to speed up the network file transfers, if possible. Caching to an SSD seems like a safer option. File writes would be verified before completing, and would be permanent on the SSD, as opposed to using RAM as cache and the possibility of a power loss with data in volatile RAM being lost forever. Windows 10 will cache some file transfer data in RAM natively. If you had lots of RAM on your computer, then your Windows 10 data file cache would be higher. PrimoCache just allows you to use more RAM as primary cache, and an SSD as secondary cache, to speed up your transfers within your computer. If you find that PrimoCache works to speed up your transfers, either on the server or client end, then that would be great. I can't even test it out anymore because my trial period ended about a year ago and PrimoCache doesn't forget your 30 day trial period. I'm not going to buy a license to try out software again that did not work for me in the past. Anyways, I hope you find a solution to your needs.
  6. I have stopped the DrivePool services, too, but like you, I reboot when starting everything back up. I don't know if there is a shortcut. And by that I mean, I have 23 USB HDDs (82TB) and it takes a few hours to reboot DrivePool, check all the drives, check duplication, etc... before all those housekeeping services are complete. I typically only schedule a reboot for overnight housekeeping, if at all possible. DrivePool is a great system for my home media server, but it's just not lightning fast on reboots and checking itself. Of course, if I only had a few HDDs or few TBs of data to validate, then the process would go much faster.
  7. gtaus

    SSD Cache

    I think if you modify a file, and then save it, it will save it first on the SDD cache and then flush it to Archive drives later depending on your settings. That's exactly what I wanted to do with PrimoCache on my system. Either running on the server or the client computer. But, alas, it does not work. I contacted PrimoCache support and they verified to me that transferring files over any network, in any direction, is not supported by PrimoCache. They said the transfer protocol over a network is different and PrimoCache does not (yet) support that functionality. Too bad. I would have gladly bought a copy if it did.
  8. I don't use SnapRAID, but there is pretty good video on YouTube at:
  9. I have never really figured out the File Placement system. So, I'm just signing on here to see if anyone has a good response. I understand how DrivePool might split up a movie folder on 2 or more drives, depending on how many files were in that folder. I see that when I upload an album folder and the tracks of the album might be on any number of drives in the pool. If your goal is to have all the files in the movie folder on one drive, then I think there is an option in DrivePool to completely fill up one drive before it starts writing to the next drive. Also, I think you can designate your DrivePool "Movie" folder to only be on certain drive(s) in File Placement. One of those options might get you closer to your goal.
  10. I think I understand what you are doing. You are partioning an HDD into 2 or more volumes and DrivePool does not recognize the new partitions. From what I understand, and I could be wrong here, DrivePool reads the serial number (not the Drive letter) of the HDD and adds a hidden PoolPart folder to that drive. Problem is, if you add additional partitions to that drive, it still only has 1 serial number for that HDD. I ran into this issue thinking I could partition a SSD into 2 volumes and have 2 frontend caches for Duplication files. But no, it will not work that way because DrivePool only allows one hidden PoolPart folder per physical disk.
  11. You can easily hide the DrivePool drives from File Explorer by simply going into Disk Management and removing the Drive letter of that/those drives. DrivePool does not need a drive letter for the HDDs in the pool. I have 23 USB HDDs, without Drive letters, in my DrivePool, but the only thing showing in File Explore is the combined DrivePool volume letter. BONUS IDEA: Instead of assigning Drive letters to your DrivePool drives, you can mount them in a folder on your C: drive. You can still access each and every drive mounted in DrivePool that way in the mount folder and see what it contains, but you won't be clogging up File Explorer with a bunch of drives in the main list. BONUS BONUS IDEA: If you use a disk cataloging program to organize your files and/or conduct searches, the cataloging program will read and catalog each drive mounted in the special folder. That way, if you had a HDD fail in your DrivePool, you can easily go into your cataloging program and find out what was on that specific drive. For my DrivePool home media server without Duplication, it would allow me to see what files were lost on that drive and if I want to reload them from a backup disk in my storage closet.
  12. gtaus

    Simulating Raid 10

    I am trying to wrap my head around your idea. DrivePool works differently than RAID, but I think I can offer some suggestions. Since DrivePool writes full files to one disk at a time, not packets spread over multiple disks, would not standard duplication in DrivePool give you the same result of having a copy of your data on two separate disks? Essentially, Raid 1+0 is Mirroring with Striping combined. Mirroring for data redundancy, and Striping for speed in a packet writing system. I think you could do something similar in DrivePool using Duplication (Mirroring) on either your entire DrivePool, or just selected important folders, and using 2 SSD's, one each as frontend cache for each duplicate file (Striping) for increased speed. I was able to achieve RAID-like speed on my DrivePool when I added an SSD frontend cache (for WRITES). I also set my DrivePool SSD cache high enough that I can also use it as a very fast READ/WRITE cache for temporary files. I set my SSD cache not to flush until it has 100GB. That's more than I need on any of my temporary working directories. Another big plus with using DrivePool is that you don't have to add additional HDDs in pairs. I recently built a DrivePool with 3 end-of-life HDDs that I cannot trust on my main DrivePool. I set that entire end-of-life DrivePool to 2X Duplication, In theory, if/when any one drive fails, I'll still have another copy of my files on one of the other 2 remaining HDDs. If there is enough capacity on the remaining 2 HDDs on that DrivePool, it will duplicate the files as needed that were on the failed third HDD. I would never consider using those end-of-life HDDs in a RAID system, but DrivePool lets me get more time out of those old drives. That saves me money and was worth the cost of DrivePool itself. Another consideration is that a typical RAID system, with packet writing across disks, leaves you with nothing if a HDD fails - unless you build in duplication like your RAID 1+0 setup. You can't read a failed RAID HDD. DrivePool, on the other hand, can often retrieve many, if not most, files off a failing HDD because it contains the entire file, not mere packets. I have been able to retrieve all but maybe 2 or 3 files off a 4TB HDD that crapped out on me. Unless the HDD is totally dead, you have a decent chance of retrieving files off that failing HDD. Bottom line: To replicate the features of RAID 1+0, use Duplication on your DrivePool for redundancy and add 2 SSDs as frontend cache, one SSD for each duplicate file to be written for RAID-like speed. When you want to expand your setup, in DrivePool you can add one disk at a time instead of having to add disks in pairs.
  13. I now have 23 USB HDDs in my DrivePool. I don't remember at what point I started having problems with DrivePool reconnecting to the USB HDDs, but it's not just DrivePool. Windows Disk Management doesn't connect with all the drives if I try to reconnect all of them at the same time. I am thinking it has something to do with the USB and identifying too many drives at one time. My solution was to buy 7 port USB Hubs with individual power switches for each port. So, I start my computer, and then load each USB drive, one at a time, making sure it shows connection in the DrivePool GUI before I move on to the next USB HDD. Obviously, I hate rebooting the server. But at least I can now get DrivePool to reconnect to the HDDs, albeit one drive at a time. On my system, it takes about 30 seconds to turn on and reconnect to each HDD. Not a very fast process but is much better than before when half my USB HDDs never reconnected. I have one internal SSD as a frontend cache for my DrivePool. When I reboot my computer, it is the only drive found by DrivePool and all the other USB HDDs are missing until I turn them on and load them - one by one - into DrivePool. In your case, if you shut down all the USB HDDs but still had one small SSD/HDD online, you should be able to reconnect all the other USB HDDs without rebooting. I do that all the time with my DrivePool when I swap in/out USB HDDs. Since I always have that one SSD online and DrivePool is always up and running, I can disconnect and reconnect the other USB HDDs without rebooting the computer. Of course, if you have any HDDs listed as missing, DrivePool goes into a READ Only mode and you will not get full functionality READ/WRITE until all HDDs are reconnected. There might be a way to start/stop the DrivePool services without rebooting, but I have not figured it out.
  14. Sounds like the Duplication process is stuck. Have you tried stopping the Duplication process, and then restarting it? If I determine that DrivePool is frozen, sometimes I'll just shut down the computer/server and restart it again. In general, I find the DrivePool does a pretty good job at duplicating the folders in the background, but, like you said, it's not a very fast process. Have you tried opening up Task Manager and checking to see if there is any disk activity? The progress bar in DrivePool does not offer much information on the activity going on in the background. In the past, when I have had to swap out/in a number of HDDs all at the same time, I shut off all duplication, swapped out the HDDs, and then reset duplication for the folders I wanted. Only suggest that if you can survive without having all those files backed up all the time. But it allowed me to complete the swaps, get everything back up and running, and then do the duplication in the background at its own pace. I wonder if the minimal info on the DrivePool task bar that really does not show any activity happening just makes a person think that everything is taking a very long time to complete. I know that transferring large volumes of data from one HDD to another with Windows Explorer can also take forever, but at least it shows you an activity box and the transfer speed. If someone has a true method of speeding up duplication, I, too, would love to hear it.
  15. From what I understand, Balancing only moves files your SSD front end cache into the Archive drives. That's all I see happening. If you are talking about Equalizing data on all the Archive disks, I have never known DrivePool to do that by will. For Equalize, DrivePool writes new data to HDDs that have more capacity. As DrivePool writes new data to the HDDs, it will more or less equalize the data among the pool. But i have never seen it rob one drive to free up space and then move that data to another HDD that has more free space. If anyone is able to get DrivePool to equalize HDDs on demand, I would like to know how.
×
×
  • Create New...