gtaus

I usually have my DrivePool server running 24/7. However, if I reboot my system on a normal shutdown and restart, DrivePool does not have to perform the long, complete, re-measure of the pool. If I have a missing disk, or if there was some problem affecting the drives before the reboot, then DrivePool does re-measure everything. On my 76 TB pool, with 18 USB 3.0 drives, that can take about 2 hours to complete. However, I can still use my DrivePool home media server and the re-measure task does not prevent me from normal use of most of my files. I am all for anything that improves the performance of DrivePool, of course, but at the moment I am just happy that DrivePool will re-check the complete pool if it suspects any changes to file integrity. If there was some way that DrivePool could quickly check a drive for "unauthorized" changes, that would help speed up the pool re-measure task by skipping over those drives that have not changed. For example, if I have one of my 18 USB HDDs go offline, for whatever reason, DrivePool will re-measure the entire pool even though the other 17 drives have not been changed. I don't know if a running hash code could be saved for each drive, and then just compare drive hash codes after reboots/reattaching missing drives for changes, skipping over drives where hash codes match and only re-measure drives where hash codes do not match. I don't want to complain too much, however, because I came from a Windows Storage Spaces environment where a missing disk could permanently corrupt the entire pool - despite having 1 or 2 disk failure enabled. I have had at least 3 HDDs fail in my DrivePool in the past year, and never once did I lose my entire pool. At this point, I am more than happy to let DrivePool re-check my pool for a couple hours as compared to trying to manually recover my Storage Spaces for weeks, and weeks, and weeks, before giving up everything as a total loss.

With DrivePool, you can use any drive, including your SSD, as both a system drive and a pool drive. When you say that your SSD is a shared drive into which you copy folders from a networked PC, are you saying that you are using your SSD as a system drive? If that is the case, then DrivePool would not automatically move files from the SSD into the DrivePool pool of drives. In order to use your SSD as a "Landing Zone" for DrivePool, you need to add the SSD to your DrivePool. DrivePool will add the hidden PoolPart directory to the SSD and make the SSD part of the pool. Then you need to go into the DrivePool SSD Optimizer and designate your SSD drive as "SSD" and not "Archive." Note: you can designate any drive as "SSD" for purposes of using that drive as a "Landing Zone" or what I call a DrivePool write cache. Once you have added your SSD to DrivePool, and designated it as "SSD" in the SSD Optimizer, then it should fall under your DrivePool directory - in my case, it's J:\ drive. Then you copy files from your network to DrivePool (for example, J:\ drive) and the files should be first written to the SSD as your DrivePool write cache. Note: You do not copy the files directly to the SSD, which is possible, but instead you have to copy the files to DrivePool. In the balancing settings, you can set the trigger point at which you want the SSD cache to flush out and write all the data to the DrivePool archive drives. In my case, I have a 224 GB SSD and I set the trigger point to flush at 100 GB. So, I can see my SSD start to fill up on transfers by watching the DrivePool GUI under the Disks panel. When the SSD reaches my trigger point at 100 GB, then it will start the flush, or Re-Balance, to the DrivePool archive drives. When you get everything set up properly, the SSD write cache, or "Landing Zone" has the potential to really speed up your writes to DrivePool because the SSD is just so much faster than the typical archive HDDs used in the pool. But it also depends on your particular system and how you use DrivePool.

Good to hear. DrivePool has advantages over other pool systems, and once you get it tweaked to what you need, it does not need much intervention afterwards.

Well, I only have one SSD and what I see happening is that one copy goes into the SSD cache and another copy goes directly into the archive HDD pool. I can watch this process play out in the Performance>Disk Performance window. The copy being sent to the SSD cache finishes first, as expected, and the second copy going to the archive HDD finishes up a little bit later - obviously depending on the file transfer size. I have never seen any problems with DrivePool using only one SSD when writing to a duplicate folder. It works for me.

I only have 1 SSD in my DrivePool. If I write a file to a folder with duplication, I see one copy going into the SSD cache and the other copy going into the archive HDD pool. In the pool case, DrivePool defaults to sending the second file to the archive HDD that has the most free space. I have Read Striping turned on DrivePool. If I read a file from a folder with duplication set, what I see is that DrivePool will read from both the SSD cache (if the file is still in the cache) and also read the copy in the archive HDD. For the most part, DrivePool pulls the file data from the faster SSD cache and very little from the archive HDD. So, the archive HDD does not slow down your file transfer in that case. If both copies of the duplicated file are residing in the archive pool HDDs, then DrivePool seems to pull from both drives about equal, almost doubling the speed of a single HDD. You can see this take place if you expand the Performance arrow in DrivePool and watch the Disk Performance window for the transfers. I know with one SSD cache that DrivePool will pull the file data as fast as it can from the SSD. I don't know if it would pull file data from both SSDs with Read Striping and double the speed of your single SSD. I say this because there are also many other bottlenecks on a computer system and you might run up against a problem with bandwidth on a file transfer. In theory, if your SSD's are both rated at 480 MB/s, will DrivePool Read Striping pull the file data down at 960 MB/s, or will you see some bottleneck related to bandwidth at that point? I don't know because I only have one SSD, and most of my data is transferred over a home network ethernet which tops out at about 120 MB/s. From what I have read, if you use Folder Duplication and want to maximize your speed, then you would set both your SSDs to the main pool and designate them as SSD in the SSD Optimizer. I don't use File Placement rules, so I don't know the exact answer. I still think you might be better off just using a portion of the SSD for direct file storage placement if your goal is to keep the files on that specific SSD for faster access. DrivePool allows you to use any drive as both a normal system drive and as a pool drive for DrivePool. I also know that without File Placement rules set on my DrivePool, that it might move files around on the drives in the pool when it does maintenance for balancing optimization, for example. Again, what I see on my DrivePool with only one SSD is that DrivePool with Read Striping reads data as fast as it can from both drives and sends the data out. Most of the data therefore is pulled from the SSD and very little from my HDD. I don't see the HDD slowing down the SSD in any way in my transfers. I am trying to be very careful to state what I see on my system with DrivePool, because your setup may be completely different from mine. I use DrivePool as my media storage server and most of my file transfers take place over my home network. If I transfer a media file from a USB 2.0 HDD, via my laptop computer in the kitchen, over a 1 Gbit ethernet, into my DrivePool home media server with 1 SSD cache and 18 archive drives attached via USB 3.0, then my top speed on that transfer is about 30 MB/s, which is all the faster my laptop can pull data from a USB 2.0 HDD. Obviously, my DrivePool SSD cache is much faster than my transfer rate in that scenario. If I transfer that same media file from the system HDD in my server computer, I reach a top speed of about 120 MB/s which is the read speed of the system HDD. If I transfer that same media file from the SSD cache in DrivePool and make another copy to DrivePool (back into cache), then it will copy the file at about 480 MB/s which is the speed of my SSD. So, you can see, that I might get transfer rates anywhere between 30 MB/s and 480 MB/s depending on the slowest device used. In none of those cases on my system is DrivePool itself slowing down anything.

I don't use the File Placement rules. So I cannot help you there from experience. I do not completely understand what you are asking. If you have both your SSDs designated as "SSD" in the SSD Optimizer panel, then I would suspect that both SSDs are being flushed when it hits your trigger point. In the DrivePool SSD Optimizer, you can designate your SSD as either to be treated as a SSD or Archive drive. If you have your second SSD designated as an Archive drive, then it should not flush the data. Also, in case you did not know, you can designate an ordinary HDD as a SSD drive for the purpose of the SSD Optimizer. It will not make that HDD any faster, but DrivePool will treat that HDD as a SSD drive for the purpose of write caching. As to duplication, here is what I see happening on my system. I only have one SSD for write cache, so if I write to a folder with duplication - and if I have real time duplication set - I see one copy going to my SSD cache and a second copy going directly to an Archive drive. If I had two SSDs for write cache, I think DrivePool would cache a copy of the file on each SSD and then flush that data later when it hits the trigger point. I hope that explanation helps. I suspect we may have a communication problem due to the language differences as I see you are running a German version of DrivePool. But I hope we can work out an answer for you. Also, if you are not getting the answers you need from me, maybe someone like the moderator @Shane might be helpful. He knows a lot about DrivePool and has answered many of my questions. There is also a contact form for support from the DrivePool team/programmers. I have been using DrivePool for less than 1 year, and it works great for me, but I don't pretend to know everything about the program.

I don't think the SSD Optimizer works that way. When it hits the trigger point and flushes data, it appears to flush all the data in the cache. There is nothing left in the cache after the flush. I think there are some SSDs with learning software that will keep your most used programs/data on the SSD. Those are typically system SSDs. The DrivePool SSD Optimizer is not designed for keeping your most used files on the SSD. However, DrivePool can use your SSD as both a pool drive and a system drive. You could simply install the programs and/or put those files you use most often directly on the SSD and not as part of DrivePool. For example, my DrivePool SSD is drive Z:. I can install, store, or locate any files I want on my Z: drive as normal, or I can also use that SSD in DrivePool which adds the hidden DrivePool PoolPart directory on my Z: drive. One of the big advantages to DrivePool over other systems such as RAID or Windows Storage Spaces is that you are able to add and use an existing drive to DrivePool and, at the same time, also use that drive normally on your system. My older RAID and Storage Spaces took full control over my drives and those pool drives could not be used for anything else.

Welcome to the forum. As to stealing this topic.... on the top of this thread it states that the issue was solved by @Christopher (Drashna) way back in 2016. Given that DrivePool has been updated many times since then, if you have a similar issue as this old thread, then it might be better to start a new thread with your concerns. There was just another recent thread on the SSD Optimizer. You might also find that discussion helpful. When I was first setting up my SSD cache, I noticed that sometimes it would not flush properly until I rebooted the computer. After making changes to the SSD cache settings, it appeared to hang on the flushing. Through trial and error, I discovered everything with the SSD cache flushing to archive drives worked as expected after a reboot. I have not changed my SSD cache settings for some time now, and it flushes as expected without any problems. If rebooting your computer after you change your SSD cache settings does not work for you, I'd suggest starting a new thread because this thread is already marked as solved. I think you would get more responses that way. Take care.

I have no problems with the DrivePool SSD Optimizer. I have set my 228 GB SSD to cache 100 GB before it triggers a flush to the archive drives. I see this is a very old thread, and this topic was just discussed in a new post/thread. You might want to look at that. There are some helpful people on the DrivePool forums and you might get your answer here. If not, I'd suggest contacting Customer Support directly. DrivePool continues to be updated and I have always got a response from Customer Support when needed.

Because my main computer drive is an older 7200 RPM HDD, the SSD Optimizer plugin for DrivePool is a really nice option for me. I think you too will see the advantage to adding a SSD write cache to your DrivePool, but if your main drive on your computer is already a SSD, there is probably not much benefit to you by directing your temp working files to DrivePool. At least you seem to have a good idea now of what might work best for you. Take care.

Go to Manage Pool>Balancing>Settings>Automatic balancing - Triggers>check the box "Or if at least this much data needs to be moved:">set the trigger point to 100 GB. Mind you, I have a 228 GB SSD, so 100 GB is just under half usage. From what I have read, you want to keep your usage under 80% of the SSD or you risk wearing out the SSD prematurely as it will continue to use the same sectors over and over at the end of its available space as the drive fills up. I have some programs that require temp working directories, like for video editing. I just made a Temp directory in my DrivePool and tell the programs to use that DrivePool J:\Temp directory. Nothing more complicated that that.

Yes, I do have Gigabit ethernet. But my maximum transfer speed over the gigabit ethernet never approaches the theoretical limit of the Gigabit design. I just stated my real world transfer rates that I see on my system. Again, if you have a USB HDD attached to a laptop computer transferring data to your DrivePool server, then the bottleneck is probably the USB HDD on that laptop. So many things slow down your system. Point is, DrivePool itself has never been my bottleneck. In DrivePool, I set my SSD cache not to flush until it reaches 100 GB. All files written to DrivePool will first go to the SSD cache, and then when it hits that 100 GB limit, it will start to flush the data to the archive HDDs in DrivePool. For my temp folders directed to DrivePool, it will use that 100 GB to both read and write as the data is still on the SSD and has not flushed yet to the archive HDDs. Since my server's main drive is a standard 7200 rpm HDD (about 100 MB/s), the SSD I set up for use in DrivePool is much faster at about 480 MB/s.

The SSD write cache did increase speed on my system, but it depends on my file transfer scenario. If I am transferring files directly from my server's drive into DrivePool on that same server, I see my maximum speed of the SSD (about 480 MB/s). If I am transferring files over my home network ethernet (about 30 MB/s), or wifi (? MB/s), or from an attached USB HDD (about 80 MB/s), then even my DrivePool archive HDDs are fast enough to keep up with the transfer. I have set aside 100 GB of my 228 GB SSD as my DrivePool write cache and also use that as a cache for my working temp folders. In that way, the 100 GB SSD cache not only works for DrivePool writes, but also as my system cache for both reads/writes for my temp working folders. In those cases, the SSD cache read/writes at about 480 MB/s whereas just my DrivePool archive HDDs would max out around 80-100 MB/s. So, yes, the SSD can significantly increase your speeds, but it all depends on your specific setup.

DrivePool has a feature called Read Striping, which you can turn on. If the folder is duplicated, DrivePool will read from both drives and that can speed up the read task. If you expand the performance tab, you can see both drives that DrivePool is reading from and the speed at which it is reading. You can almost double the read speed in some transfers. What I see on my system is that DrivePool will read from both my SSD and an archive HDD at the same time, but since the SSD is just so much faster, most of the read data comes from the SSD. I don't know if DrivePool is specifically programmed to recognize the faster drive and use that pipeline first, but in effect that is what I see happening on my system. My read speed, with Read Striping turned on, in those cases is almost exactly the same as the read speed on my SSD. FYI, I only have 1 SSD, so I do not know if DrivePool would Read Strip both of your SSDs and almost double your read speed from the SSDs. Also, in my case, the bottleneck on my transfers is not the DrivePool read speeds, it is the speed of which I can transfer the data over my home network ethernet, the wifi, or to maybe a destination USB HDD. In most of my cases, transfers on my system are slowed down by lots of things but DrivePool is not one of them.

@Shane, thanks for response. I have searched the internet for a solution to setting a size limit to a Windows folder, but have yet to find anything that works. Some people have suggested that third party software exists to add this function, but nobody even offers a name of a product. So far my searching has come up empty. But thanks for the feedback on that issue. Yes, I already catalog my drives so I know what was on the backup if the drive should fail. I can replace those files from the existing copies in DrivePool, but, like I said, that will take some time. I'm just trying to think of a better way to handle these backups in case I ever lose a backup drive again while in the process of moving files to it. Until I find a better solution, I guess I'll just make a folder in DrivePool to match the backup HDD. After all the files are successfully copied and verified, then I will delete that folder from DrivePool. It would have been nice to be able to size limit the folder to exactly the size limit of the backup HDD, but I'll just have to check on the folder size manually. BTW, I contacted CineRAID, the maker of my USB Docking Station, and they suggested it might be a bad cable, or I might have to use a different USB connection on the computer. Unfortunately, there is no way to test either suggestion other than just replacing the cable and/or moving the docking station to a different USB connection and hoping for the best. What's worse for me is that this problem has only come up 3X in the last 4 months, so it might be one of those random events that you can never quite isolate as to the core problem. It did, however, expose a shortcoming in my backup plan and I need to rethink my steps in that process.

That is what I expected, but glad you spelled it out. That is also what I have come to believe. DrivePool makes a copy of the file(s), but really doesn't know which, if either, copy is good. For this reason, I have started creating .par2 files on folders before I archive them. That way, the .par2 file will tell me if the file is still intact and complete. I create about 10% .par2 files, which also allows the program to "heal" most files that may have become corrupt. I don't pretend to know all the magic that goes into the .par2 program, but the .par2 files are an easy way to ensure that the saved files have not been damaged. @Shane, Thanks for the responses and the explanations.

My backup plan has been to move files to HDDs that I store in my closet. Normally, that has worked. Unfortunately, the other day I was moving media files to a 3TB HDD backup drive and the backup drive's directory became corrupted and unreadable. Fortunately, I still have a copy of those files in DrivePool, but I will have to manually determine which exact files were lost on that specific backup drive. Pain in the butt, but it can be done given time. I am now thinking that a better option for me would be to create a 3TB pool/sub pool/or folder in DrivePool that I could copy files to over time and then dump on a backup HDD when the pool space is full. After the files have been copied over and verified, I would delete them from DrivePool. Is it possible to exact size limit a pool/sub pool/or folder like that in DrivePool? If not, is there any third party software that can perform this same task? I know some other operating systems allow you to size limit a folder, but Windows does not have that feature itself that I am aware of. Any help appreciated.

+1, it a question that I have asked before but never received any response. I would also like to know how DrivePool recovers from a drive loss and how it rebuilds itself from a drive failure. Nobody wants duplicates of date time stamp newer 0kb files. How does DrivePool know which is the good copy and how does it repair itself?

Yes, but when I copy files to the backup HDD, I use TeraCopy with the verification feature turned on. Yesterday I was copying a chunk of files to a backup HDD and TeraCopy said the files failed to verify. For some reason, my backup HDD directory became corrupted and unreadable. I suspect it might have something to do with my CineRAID docking station that I use for the backup HDDs, so I sent off a support request to them about the issue. Anyway, TeraCopy gave me a head's up that the files failed to verify and that allows me to go back and look at the problem. I use a file manager called SuperCat to catalog my discs, drives, etc... It's an older program (last updated for Windows 7) and has not been updated in many years, but it still works. Recently, I have tried WinCatalog 2020 but have had issues with it. WinCatalog has many more features than SuperCat, but it did not work very well for me. So I would not recommend it at this point. I am still looking for a file manager replacement to SuperCat. My DrivePool is sitting at 74 TB, and I have more than that on my backups. I don't know if my backup strategy is the best method, but in general I can search SuperCat to find any files I need to restore from my backups. Unfortunately, I don't know what to do if my backup HDD fails if I don't also have that data on DrivePool to make another copy for backups. In general, I could live with the loss of media files in the rare circumstance that a storage drive might fail. But I would like a better option for my backups just the same.

Duplication is a strategy to protect against drive failure in a pool. That should not be considered the same as a backup plan where you have the data stored safely away somewhere else. Duplication and backups serve different purposes. One of the great advantages of DrivePool is that you have folder level duplication. RAID systems and Windows Storage Spaces only offer pool duplication of the entire pool. I use my DrivePool as my home media storage server. All my media files are backed up on other HDDs that are stored in my closet. I don't have any need to set 2X duplication on those media files in DrivePool because a loss of files from a HDD failure in the pool could be rebuilt from a backup, if I wanted. About 95% of my home media data on my DrivePool is stored on backup HDDs, so I chose not to duplicate those folders in DrivePool. I do have some folders that I have marked for 2X duplication, but not very many. The end result is that DrivePool allows me to duplicate only those folders I want and therefore I save lots of money by not having to duplicate the entire pool. I came from a RAID background, then used Windows Storage Spaces for many years, but now use DrivePool. At first, I was more concerned about duplicating the pool in case of a HDD failure. Over time, I realized that my limited money was better spent on a good backup system instead of trying to duplicate all my data within the pool. DrivePool offers you many more options than other systems, but we each have to decide where to spend our money to protect our data.

Yes, it works just fine. But like you said, sometimes the File Explorer window does not refresh very fast and I will force a refresh with F5, or just change the focus to another directory and then come back to the original.

What kind of small files are you working with? Are they small program files, or some kind of temp working files? If they are small temp working files, you could maybe set the program's temp directory to your SSD. I have tons of small .mid .kar, .gp4, .idx, and drum beat files from older programs I used back in the day. I would not say they are hard to work with on my HDDs, but they waste lots of space on the drive given the large allocation block size of the formatted drive and the tiny file sitting in each block. There are programs that would allow you to search and find all your small files, and move them to a specific drive or folder, but I don't know of any programs that do that automatically. Maybe you could explain more on why you think you need those small files stored on your SSD? I only want to use my SSD for files that I am currently using, for temp directories, or for files that I use so often that it makes sense to have them stored on the SSD instead of an archive HDD. But that has nothing to do with the file size itself.

The auto refresh of Windows File Explorer has been a long standing issue with Windows. There are a number of fixes for this issue posted on the internet, but every version of Windows 10 seems to have a different solution. I personally just live with inconvenience because the next Windows update could "un-fix" the problem you thought you had solved. I don't have enough time in my day to manually apply "fixes" to Windows that might be wiped out next week on an update. So, if I need to refresh File Explorer, I just hit F5 and it will refresh. I also use a third party File Manager and it has a refresh icon I click on and the directories are refreshed. In short, I doubt your problems have anything to do with DrivePool as this is a known issue with Windows itself. But yes, I feel your pain as I go through this issue all the time on my computers. If you find a real fix to this issue, or if someone else has the solution, then we would all be happy.

Is the drive properly initialized and formatted? I don't know if DrivePool can add a raw drive or not. I have not seen this error on my setup.

I have not moved to 4K yet, but the streaming requirements of my 1080p movies is less than 1 MB/s. Any drive in my pool can keep up with that demand. What is your streaming requirements for 4K media? Anyway, if you really want to maximize your cache speed, you could try out PrimoCache and use your system RAM as Level 1 cache/buffer. That would be many times faster than even a SSD. They offer a free 30 day trial period to see if it works for your setup.