JasonC

I realize this is probably a asking a bit much, but is there any kind of rolling activity log option for DrivePool? I ask because sometimes I question my sanity, like files go missing. But they aren't things I've touched often, so I have no idea when they went away, or if I did it or what. Case in point: I told Plex to rescan a folder that contains unusual variants of movies. I generally don't ever touch anything in here once it's in there. The re-scan suddenly tells me a file is gone. I go look...yep it's gone (but the folder is still there). But it's not something I would have deleted. I don't know where it went, or how long it's been gone. I'll scan my FS and hope it's an accidental drag and drop, but I'm not hopeful. Since I have Plex scan my folders regularly, it had to be a recent removal. So...it'd be nice if I could just have a high level log tracking operations like that (log delete operations). Thanks!

Anyone ever seen any odd with a file named like this: "C:\ProgramData\Microsoft\Windows Defender\Support\MpWppTracing-20190202-183619-00000003-ffffffff.bin" Odd defined as conitnual writing for extended periods at fairly high speeds (relative to any other Windows instasll) Obviously it's some sort of Defender related file, but on most of my machines not much happens with that file (1-2KB/sec of writes). On my machine running DrivePool, I'll go through phases where it's writing to that file continually at 300-400KB/sec, sometimes more. It's only a 4K file so that seems a little excessive, and it's particularly annoying because I have All Defender related things turned off on this machine. Obviously something Defender related is still doing something though. Anyway, since this machine doesn't do anything but run DrivePool for me, I thought I'd check here. Thanks!

Well, so new wrench in the works, but maybe a new avenue to explore. I haven't yet changed the tcp auto-tune, but partly because I've gotten sick of Windows 10 reboots, so I've been migrating a lot of my things over to Linux. I'm still seeing the pauses in file operations, and I've had the index turned off for a long time so I know it's not that, now. On the Linux side, I think I've got my mount points correctly set to not do any caching, presumably similar to how I've got the Windows client set, but I'm still seeing some odd behavior when I've got the identical tasks happening, but via Linux/samba mounts, but otherwise the same software on the Linux side. I did just notice something though...I think whenever I am getting those weird pauses, I get this in the app log on the machine running DrivePool: I've just started investigating it, and haven't proven a correlation to myself beyond a couple data points, but the early things I've found make me feel like this could be related to the pause behaviors I'm seeing on the clients. So two things: - if you have run into that EseDiskFlushConsistency and can tell me if that's something I should look into addressing or it's not anything that impacts DrivePool things... - If you have any suggestions with regard to configure a Linux/Samba mount of the a share backed by DrivePool (I have cache=none set, I'm not 100% sure if that's the correct/equivalent to Windows SMB caching settings, but it looks it from the docs) that would be great too. Thanks!

Thanks, I'll give a look at the other options you've listed, I'll want to read up the potential impacts of changing the autotune on tcp before I put that in place. I've been running without the indexing service enabled for some time now, and I still see the weird pause. In particular, I see it at the end of file operations, usually when I'm moving folders and files to other places. I get this long pause at 99% like there is some cleanup or ack that the client is waiting for before it marks the operation complete. Long being defined here as 5 seconds to say...30? I don't time it, but it's just a file move between folders on the same target typically, so I don't know what it could be doing. I'll try and cap a video of it sometime just to show what I see.

Well, I wouldn't say exactly, I'm not getting any errors. It just is slower than I expect, or has a pause. It does succeed though. Which is why I hadn't disabled the indexing service, because I actually use the remote search capability. So it'll be a little unfortunate to lose indexed searches, if that is it. But, for science, I'll turn off the indexer, and see if it addresses those strange hangs

I've always assumed this was DrivePool, but I'm double checking/asking if there are any mitigations. I've noticed with a couple of fairly common operations, I'll run into pauses on the Drivepool disk that I don't see typically. Primarily I noticed this when I create a new folder, rename a folder, or a delete a lot of things. When creating or deleting, there is a several second pause. Same with a rename. It's somewhat annoying because I'll often get an error because of the pause, because I'll create a new, rename it, and try and enter it. But the pause between the create and rename means my system tried to enter "New Folder" not the name I just renamed it to. With deleting, it'll stick at 99%, sometimes for quite sometime. I'm guessing it's something to do with DP updating it's indexes, before telling the OS the operations are complete, but if there's a way to improve performance of this, I'd be interested in knowing what it is. Or especially if it's not supposed to do that. Thanks!

I tried switching...it seems that Rename-Item is much less capable in handling the square brackets then Move-Item is. I get errors I don't get when moving the file to a new name. I think I found my problem. It also turns out the Move-Item doesn't like it very much if you throw it a rename command with replace that doesn't meet the replace criteria. It appears that if Get-ChildItem passes such an item, it throws an error as though the file were locked, like it thinks that's the reason it can't successfully rename. Anyway, looks like it may have been a bit of a red herring. Thanks!

I can go ahead and collect some logs. I'm pretty sure I can make it happen on demand. This is actually the second or third time I've seen it. I first saw it last week, which was when I first tried running my scripts. I'll can generate a test setup and run my script against it. It could very well be too what Spider99 pointed out, I don't know if Move-Item has more overhead that would be avoided if I was calling Rename-Item instead. It was sort of a mental map failing from *nix days where you just used mv to rename things.

Having owned a few of these (a 4 and 8 bay), I'd say to get whatever ASMedia card they recommend. I used a 4-bay one with whatever eSata was on my mobo, and the SATA port replication was a little wonky until I picked up the card MediaSonic recommended. They've been rock solid after that.

Just a wild guess off the top of my head as a possible thing...check on the host where your physical disk activity is happening. You mentioned your using virtual disks, are your virtual disk files consuming the majority of the space on the physical disks, or do you have it sliced up so you are presenting multiple disks to DrivePool from a single disk? Using virtual disks means that you're hiding the real physical layout from DrivePool. DrivePool thinks of each disk as a spindle though. It won't know any better. So if you have 3 virtual disks that reside on a single physical disk, DrivePool may be sending more disk operations to the physical disk then it realizes, because it thinks it has 3 different disks. So 3x the operations but still 1 physical disk servicing requests. So effectively you'd be choking yourself out by throwing way more iops at the physical disk. DrivePool could be throwing parallel operations at a single spindle. Check the perfmon counters for the physical disks hosting your drive pool, I'd look at queue depths in particular, and probably read and write operations. I run a VM as well to host my pool, but I am doing passthrough of the pooldisks, so DrivePool does have a real 1-to-1 mapping of spindles and pool disks.

Didn't occur to me to switch it. I went with that little used "reuse code" strategy pulling some code that I wrote for a related process that moves files while renaming them. This guy actually feeds the file mover, the square brackets were causing my Powershell script to choke though. So, it just occurred to me as I was logging into the forum and saw the other thread, the Windows search indexer could be the problem. I'll have to try suspending the Search Indexer and then running the operation.

I have a tiny powershell script I'm using to replace square brackets with parenthesis on folder names(it seems that while you can do them, square brackets are technically not supposed to be in file/folder names in NTFS anymore). Anyway, I've noticed when I run this guy, it seems like at first, it will be humming along smoothly, and then after some point, I start getting IOException errors that items are in use by another process. I've looked extensively, and I can't find anything holding a lock though. This happens regardless if it's a network share or local to the pool(I'm usually starting on a network share). What's more odd, it seems like the change actually does go through. Regardless, it was a little bit disconcerting, I thought I'd see if this is a DrivePool thing or what. I've never run into anything like it with just a more typical ntfs setup. the error is I start getting is: Or more specifically: The most recent run I did was about 200 foldes, so it doesn't even seem like that much load. Is this DrivePool causing this, do you think? Thanks!

A possible additional data point. I took my original 4-bay MediaSonic unit, and plugged it into another machine. I was using it to migrate data to a new disk (from a 3 to a 8TB disk). I connected via USB3, and I was unable to complete the encryption/format stage of the new disk. I ended up moving it to a standalone USB 3 drive dock(slightly newer, with UAS support) and it has been rock solid there. I am starting to lean towards having all the disks connected through whatever internal USB hubs these things use is just not particularly stable.

So I'm looking to replace CrashPlan, like a good many people. I've been seeing various things that I can't really tell which, if any of the common ones work successfully with DrivePool. It'd be running all local, Windows 10, so DrivePool, and the backup app. I'm currently looking at CrashPlan and BackBlaze, but info on any of the common guys would be useful. Thanks!

Update: I've switched back to eSATA. I was noticing what I think are sporadic time-outs occurring on the disks connected through USB storage. I suspect that this is a result of USB having some trouble under high load. The thing is, I'm also not sure this isn't a fault of my setup, as I'm not 100% certain what happens as far as drivers and such when you pass-through a USB attached disk to a VM, so it's entirely possible that I'm not getting the benefits of UASP. Windows makes it somewhat hard to tell what is happening at that point, in the chain of how things are connected and attached and communicating. I'm curious enough that I may see if I can get an answer out of Microsoft or if someone here knows for certain, I'd be interested in hearing the answer.

Well, I've migrated the 2 disks I had in my 4 bay over to the new StarTech. I forgot I re-organized my disk layout a while ago, so I only had 2 SSDs in the 4 bay, so I can't yet speak to heat. I don't recall how the StarTech is setup, I'll say this for the MediaSonic's, I thought they had excellent cooling in them. But I also have my equipment in the basement where the ambiant temperature is always fairly cool. Now, the sorta good news, the SSDs are holding virtual disks for VMs, so they are very sensitive to disconnects. It's only been about a day, but no issues so far, and the performance seems good. I know what you are saying here, but I've had a different experience, the drivers have become mature, there aren't updates because there's nothing to update. They are stable and performant enough, that is, you are held up by disk speed or fundamental bus speed of SATA at this point, there's nothing left to optimize. I'm not sure what issues you have, I'm running Sil3132 based SATA controllers on Windows Server 2016 with no issues connecting eSATA with port multipliers. I've gone pretty good runs without reboots, it's more been external events (power failure) or Windows patching that's made me reboot. I had almost done what you did, if port multipliers didn't pan out, I was looking at going to SAS, but boy was it expensive at the time.

I'll try to remember to follow up. I can say right now, the MediaSonic 8-bay as an eSATA device has been perfectly fine. The bigger challenge when I was setting it up was making sure I had pieces that supported everything properly. eSATA with JBOD external boxes early on had really similar challenges to USB 3, and at the time I first got these things going, most on-board SATA chip sets didn't support port multipliers. The add in eSATA card I originally was using, early drivers and versions of SIL3132 chipsets I think were wonky. I ended up just buying the card MediaSonic recommended, which had the same SIL3132 chipset in my first card, but has been stable and reliable compared to my first card. That's all a long way of saying, if you are going to connect it via eSATA, the 8-bay MediaSonic has worked very well for me as long as all the pieces in your setup support it (i.e. port multiplier with your SATA chipset is reliable). It was in USB3 I had problems with it, but I've never gone back and re-tried it, so I can't say that USB3 attached now wouldn't work perfectly fine. I also can't say for certain if the USB problems I had then were drivers, USB chipset on my system, USB support in the device or OS (I've upgraded my server OS and hardware since then). But I can say, as I noted earlier, that it doesn't have UASP, so I am less inclined to connect it that way now. It's only been a couple days of course, but the StarTech has worked great since I installed it, and I plan tonight to migrate the disks from my 4-bay to the StarTech enclosure, so that should really help me test the USB connection out.

I have the MediaSonic 8-bay you mention, and the earlier 4-bay version of it. I initially tried USB 3.0 connections with it, but found the connection a bit unstable (random disconnect about once a week), but it's solid under eSATA. I do not know, but kind of suspect that the USB instability was partly me being a pretty early adopter in USB 3 land, and drivers and possibly USB hardware being less stable then. Another caveat with these enclosures, they do not support UASP, so you don't get optimal performance on USB 3. I believe driver and hardware support of USB 3 to be much more mature and reliable now then it was when I first tried 4-5 years ago. I just purchased another 8-bay unit, but this time the StarTech equivalent (S358BU33ERM), otherwise mostly the same. It has a bit less manual options (fan speed, connection) but I don't really find that to be an issue. I'm going to give it a try USB 3 first, partly because this unit DOES support UASP (One of the main reasons I switched from the MediaSonic) so I should get better performance. So far it seems very reliable and performant, but if I encounter any hiccups I will just move it to eSATA as well.

So the balance setting changes does seem to do what I want, so my observations appear correct. Related question, with regard to the balance settings and migration from SSD with the SSD Optimizer. I'm using the balance "not more often" setting with a 10 minute interval. 2 questions related to that: 1) does that mean the balancer kicks off every 10 minutes regardless, or does it check if other parameters are set, and run based on those (for instance, if a ratio value is set) 2) Is the balancer going to be very basic and just move everything at that time from my SSD, or is it based on the age of the items being moved? Ideally, I'd like it to be age based, the idea being that files that are recent in use(so say, they last couple of minutes if I am running it every 10) I don't want to move yet, since they are more likely to be waiting for additional processing. I can set the interval higher, but that's a fairly crude approach and really doesn't get me what I want, just reduces the frequency that it would move files that are recent in-use/likely to be used again soon. Thanks!

I was watching DrivePool operations recently and I suspect the aggressive balance immediately settings are actually probably hurting overall performance of file operations that frequently happen for me. Specifically, merge/decode of multi-part files. I have an SSD in the pool with the SSD Optimizer, and I had thought there was a little delay before the balancer tried moving files off, but watching the operations, it looks like this actually happens much quicker than I thought. The net result being that I think temp files are getting pushed to spindle disks and decode ops are happening from there instead of on SSD. I'm going to enable the "not more often" settings and just swag a value, and see if that improves performance. I'm more looking for commentary on if my reasoning is correct with my observations of what I'm seeing in the DrivePool UI. It would explain why some of those things always seemed to take a little longer then I thought they should against an SSD(since they weren't against SSD!). Thanks!

So, just a mildly interesting follow up.... I recently migrated my Drivepool off Windows Server and onto Windows 10 (CrashPlan exiting the home user market motivated it). Anyway, it's now residing on a Win10 VM with the same configuration as the server machine had, but that little pause I was seeing before when I'd browse to it...I don't see anymore. It's possible moving to the latest version impacted that as well though. In fact, overall, without any objective measurements, it just feels a little faster/more responsive. (I keep having a "that was quick" reaction as I'm navigating the share).

I'll give updating the software a go, but I imagine it's the disk listing that is the cause. I guessed it was probably something like that but I figured I'd ask in case there were settings I could adjust. I'm able to throw memory/cpu/SSD storage resources at it if it would make it go away, but it's not so bad that I can't live with it either. I just figured it wouldn't hurt to ask. I suppose I could double the core count and see if that helps, but I don't think I've hardly ever seen it peg as it is, so I just kinda wrote that off. Thanks!

I've checked for performance problems and nothing jumped out as problematic. Disks don't power down, server is generally idle. It's dual 6 core xeons with 48GB memory, although I only have 2 cores and 8GB assigned to the VM. It's not cpu ready at the host or memory pressure in the guest as far as I can tell, but generally I'm normally not hurting for CPU or memory.

It's a minor thing, but something I've noticed since I switched over to DrivePool, when I first hit the file server, there's always a pause partway through displaying the share contents. It shows about the first 15 or 20 folders, pauses for a second or two, and then continues. I also see similar behavior in browsing at least some folders. I haven't been carefully tracking it, but I think it's probably folders with more stuff in them, I never had this pause when I was using StorageSpaces, and haven't ever seen it with other things, so I'm guessing it's something to do with DrivePool. I suspect maybe it's building/presenting the virtualized filesystem? Anyway, is there anything I can do to mitigate that? On the one hand, it's a small thing, it really doesn't affect me a whole lot. On the other hand, it is a thing I constantly run into, so it's one of those minor irritations that constantly occurs. I always get just a little bit paranoid about it because under other circumstances, it feels like behavior you get when a disk is starting to fail, i.e. retries go through (I've checked, disks are all good). Thanks!

Have you checked your event logs for any warnings being thrown? Something doesn't make sense to me though, you said if you copy directly to the disk, bypassing using DrivePool, you still get slow speeds, yet you still think DrivePool is the issue? Have you just uninstalled DrivePool and then seen if you still have slowness to those disks? There should be no real harm to doing this, and re-adding your disks to a pool after you re-install, other then if you had significant amounts of file shares on the aggregate disk. Copying directly to the pool disks would bypass any DrivePool involvement, so this seems to me to exclude DrivePool as the problem and suggests either cabling, controller, or disk problems. You can even go the extra step of temporarily stopping your drivepool service and seeing if you still have the slow performance directly to disk. How old are your disks? Have you re-seated your cables/checked them? Did you do the copy test directly to each disk in the pool, i.e. do they all exhibit slowness? You have significant external USB disks, if you swap an internal disk and an external disk, does slowness on the previously internal disk go-away and does the disk that was USB now appear slow? If you pop task manager and resource monitor, and do a large copy, do you see anything unusual happening (excessively high CPU, or in resource monitor, processes that are touching the files you are copying)? Boot safe mode and copy some files around between pool disks, still show performance problems?