JasonC

Anyone ever seen any odd with a file named like this: "C:\ProgramData\Microsoft\Windows Defender\Support\MpWppTracing-20190202-183619-00000003-ffffffff.bin" Odd defined as conitnual writing for extended periods at fairly high speeds (relative to any other Windows instasll) Obviously it's some sort of Defender related file, but on most of my machines not much happens with that file (1-2KB/sec of writes). On my machine running DrivePool, I'll go through phases where it's writing to that file continually at 300-400KB/sec, sometimes more. It's only a 4K file so that seems a little excessive, and it's particularly annoying because I have All Defender related things turned off on this machine. Obviously something Defender related is still doing something though. Anyway, since this machine doesn't do anything but run DrivePool for me, I thought I'd check here. Thanks!

Well, so new wrench in the works, but maybe a new avenue to explore. I haven't yet changed the tcp auto-tune, but partly because I've gotten sick of Windows 10 reboots, so I've been migrating a lot of my things over to Linux. I'm still seeing the pauses in file operations, and I've had the index turned off for a long time so I know it's not that, now. On the Linux side, I think I've got my mount points correctly set to not do any caching, presumably similar to how I've got the Windows client set, but I'm still seeing some odd behavior when I've got the identical tasks happening, but via Linux/samba mounts, but otherwise the same software on the Linux side. I did just notice something though...I think whenever I am getting those weird pauses, I get this in the app log on the machine running DrivePool: I've just started investigating it, and haven't proven a correlation to myself beyond a couple data points, but the early things I've found make me feel like this could be related to the pause behaviors I'm seeing on the clients. So two things: - if you have run into that EseDiskFlushConsistency and can tell me if that's something I should look into addressing or it's not anything that impacts DrivePool things... - If you have any suggestions with regard to configure a Linux/Samba mount of the a share backed by DrivePool (I have cache=none set, I'm not 100% sure if that's the correct/equivalent to Windows SMB caching settings, but it looks it from the docs) that would be great too. Thanks!

Thanks, I'll give a look at the other options you've listed, I'll want to read up the potential impacts of changing the autotune on tcp before I put that in place. I've been running without the indexing service enabled for some time now, and I still see the weird pause. In particular, I see it at the end of file operations, usually when I'm moving folders and files to other places. I get this long pause at 99% like there is some cleanup or ack that the client is waiting for before it marks the operation complete. Long being defined here as 5 seconds to say...30? I don't time it, but it's just a file move between folders on the same target typically, so I don't know what it could be doing. I'll try and cap a video of it sometime just to show what I see.

Well, I wouldn't say exactly, I'm not getting any errors. It just is slower than I expect, or has a pause. It does succeed though. Which is why I hadn't disabled the indexing service, because I actually use the remote search capability. So it'll be a little unfortunate to lose indexed searches, if that is it. But, for science, I'll turn off the indexer, and see if it addresses those strange hangs

I've always assumed this was DrivePool, but I'm double checking/asking if there are any mitigations. I've noticed with a couple of fairly common operations, I'll run into pauses on the Drivepool disk that I don't see typically. Primarily I noticed this when I create a new folder, rename a folder, or a delete a lot of things. When creating or deleting, there is a several second pause. Same with a rename. It's somewhat annoying because I'll often get an error because of the pause, because I'll create a new, rename it, and try and enter it. But the pause between the create and rename means my system tried to enter "New Folder" not the name I just renamed it to. With deleting, it'll stick at 99%, sometimes for quite sometime. I'm guessing it's something to do with DP updating it's indexes, before telling the OS the operations are complete, but if there's a way to improve performance of this, I'd be interested in knowing what it is. Or especially if it's not supposed to do that. Thanks!

I tried switching...it seems that Rename-Item is much less capable in handling the square brackets then Move-Item is. I get errors I don't get when moving the file to a new name. I think I found my problem. It also turns out the Move-Item doesn't like it very much if you throw it a rename command with replace that doesn't meet the replace criteria. It appears that if Get-ChildItem passes such an item, it throws an error as though the file were locked, like it thinks that's the reason it can't successfully rename. Anyway, looks like it may have been a bit of a red herring. Thanks!

I can go ahead and collect some logs. I'm pretty sure I can make it happen on demand. This is actually the second or third time I've seen it. I first saw it last week, which was when I first tried running my scripts. I'll can generate a test setup and run my script against it. It could very well be too what Spider99 pointed out, I don't know if Move-Item has more overhead that would be avoided if I was calling Rename-Item instead. It was sort of a mental map failing from *nix days where you just used mv to rename things.

Having owned a few of these (a 4 and 8 bay), I'd say to get whatever ASMedia card they recommend. I used a 4-bay one with whatever eSata was on my mobo, and the SATA port replication was a little wonky until I picked up the card MediaSonic recommended. They've been rock solid after that.

Just a wild guess off the top of my head as a possible thing...check on the host where your physical disk activity is happening. You mentioned your using virtual disks, are your virtual disk files consuming the majority of the space on the physical disks, or do you have it sliced up so you are presenting multiple disks to DrivePool from a single disk? Using virtual disks means that you're hiding the real physical layout from DrivePool. DrivePool thinks of each disk as a spindle though. It won't know any better. So if you have 3 virtual disks that reside on a single physical disk, DrivePool may be sending more disk operations to the physical disk then it realizes, because it thinks it has 3 different disks. So 3x the operations but still 1 physical disk servicing requests. So effectively you'd be choking yourself out by throwing way more iops at the physical disk. DrivePool could be throwing parallel operations at a single spindle. Check the perfmon counters for the physical disks hosting your drive pool, I'd look at queue depths in particular, and probably read and write operations. I run a VM as well to host my pool, but I am doing passthrough of the pooldisks, so DrivePool does have a real 1-to-1 mapping of spindles and pool disks.

Didn't occur to me to switch it. I went with that little used "reuse code" strategy pulling some code that I wrote for a related process that moves files while renaming them. This guy actually feeds the file mover, the square brackets were causing my Powershell script to choke though. So, it just occurred to me as I was logging into the forum and saw the other thread, the Windows search indexer could be the problem. I'll have to try suspending the Search Indexer and then running the operation.

I have a tiny powershell script I'm using to replace square brackets with parenthesis on folder names(it seems that while you can do them, square brackets are technically not supposed to be in file/folder names in NTFS anymore). Anyway, I've noticed when I run this guy, it seems like at first, it will be humming along smoothly, and then after some point, I start getting IOException errors that items are in use by another process. I've looked extensively, and I can't find anything holding a lock though. This happens regardless if it's a network share or local to the pool(I'm usually starting on a network share). What's more odd, it seems like the change actually does go through. Regardless, it was a little bit disconcerting, I thought I'd see if this is a DrivePool thing or what. I've never run into anything like it with just a more typical ntfs setup. the error is I start getting is: Or more specifically: The most recent run I did was about 200 foldes, so it doesn't even seem like that much load. Is this DrivePool causing this, do you think? Thanks!

A possible additional data point. I took my original 4-bay MediaSonic unit, and plugged it into another machine. I was using it to migrate data to a new disk (from a 3 to a 8TB disk). I connected via USB3, and I was unable to complete the encryption/format stage of the new disk. I ended up moving it to a standalone USB 3 drive dock(slightly newer, with UAS support) and it has been rock solid there. I am starting to lean towards having all the disks connected through whatever internal USB hubs these things use is just not particularly stable.

So I'm looking to replace CrashPlan, like a good many people. I've been seeing various things that I can't really tell which, if any of the common ones work successfully with DrivePool. It'd be running all local, Windows 10, so DrivePool, and the backup app. I'm currently looking at CrashPlan and BackBlaze, but info on any of the common guys would be useful. Thanks!

Update: I've switched back to eSATA. I was noticing what I think are sporadic time-outs occurring on the disks connected through USB storage. I suspect that this is a result of USB having some trouble under high load. The thing is, I'm also not sure this isn't a fault of my setup, as I'm not 100% certain what happens as far as drivers and such when you pass-through a USB attached disk to a VM, so it's entirely possible that I'm not getting the benefits of UASP. Windows makes it somewhat hard to tell what is happening at that point, in the chain of how things are connected and attached and communicating. I'm curious enough that I may see if I can get an answer out of Microsoft or if someone here knows for certain, I'd be interested in hearing the answer.

Well, I've migrated the 2 disks I had in my 4 bay over to the new StarTech. I forgot I re-organized my disk layout a while ago, so I only had 2 SSDs in the 4 bay, so I can't yet speak to heat. I don't recall how the StarTech is setup, I'll say this for the MediaSonic's, I thought they had excellent cooling in them. But I also have my equipment in the basement where the ambiant temperature is always fairly cool. Now, the sorta good news, the SSDs are holding virtual disks for VMs, so they are very sensitive to disconnects. It's only been about a day, but no issues so far, and the performance seems good. I know what you are saying here, but I've had a different experience, the drivers have become mature, there aren't updates because there's nothing to update. They are stable and performant enough, that is, you are held up by disk speed or fundamental bus speed of SATA at this point, there's nothing left to optimize. I'm not sure what issues you have, I'm running Sil3132 based SATA controllers on Windows Server 2016 with no issues connecting eSATA with port multipliers. I've gone pretty good runs without reboots, it's more been external events (power failure) or Windows patching that's made me reboot. I had almost done what you did, if port multipliers didn't pan out, I was looking at going to SAS, but boy was it expensive at the time.