Jump to content
  • 0

Can't copy large file to pool over the network (not a space issue)


jp7

Question

Hey all,

I have an unusual problem that I can't figure out, and am wondering if I am just overlooking something in settings or if there's something else going on.

My pool is on a Win 10 PC and has 7.76 TB free. Each individual disk in the pool has at least 1 TB free. What's been happening is that if I try to copy over a single file that is larger than 10-12 GB to the pool over the network, it fails with the message "insufficient system resources exist." Usually that means disk space or RAM, but I have enough space on all my disks and RAM is fine according to system monitors on both source and destination machines.

  • I can copy a bunch of smaller files that total up to that size into the pool over the network.
  • I can copy a single 12GB file to the pool from a local USB on the Win 10 PC.
  • I can copy a single 12GB file directly into the poolpart folder on a single drive in the pool over the network (e.g. not going to the pool drive letter). [This makes me think it's the pool at issue, not the network.]
  • It's only when copying a single large file to the pool over the network that it fails.

It's been doing this for about a month and I have no idea why. Even with file duplication, there should be plenty of space. All I can think of is that there's some setting on the balancers/file placement that's causing it. I've looked there and nothing jumped out at me, but I'm also not sure what I'd be looking for.

I've attached an image of my drives in Explorer and the DrivePool main dashboard window.

It's difficult to use the pool for backup destinations and such with this issue. Does anyone have any ideas about what might be going on and how I could fix it?

Thanks!

image.thumb.png.f1a42ae1faa57347c2a2189b290c9df5.png

Link to comment
Share on other sites

17 answers to this question

Recommended Posts

  • 0
51 minutes ago, Umfriend said:

The only thing I can think of is Manage Pool -> Performance -> Network I/O Boost (even though it says it relates to read requests, not write requests).

Good idea - I tried that (both with it on and off) and still the same error. It fails more quickly with I/O boost on, which makes sense, I guess!

Link to comment
Share on other sites

  • 0

Another thing I have is that if I copy files from a client to the Server of the network, it matters whether I access the Server through Explorer -> Network or through a mapped network drive. The latter sometimes fails but I am pretty sure it has to do with some sort of permission (SQL Server backup files I can not copy through the mapped network drive) and I get a different message anyway. So, completely OT.

So basically, I have no clue. I hope someone else here as an idea on how to diagnose and/or fix. I would have a look at Event Viewer on both the client as the server. Not optimistic but I'd look.

Link to comment
Share on other sites

  • 0

If it is not already present, try adding PoolUsageMaximum (REG_DWORD, Decimal, 60) to HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Memory Management in the registry.

If it is already present but higher than 60, record its value and try changing it to 60.

In both cases a restart will be required.

Link to comment
Share on other sites

  • 0
18 hours ago, Shane said:

If it is not already present, try adding PoolUsageMaximum (REG_DWORD, Decimal, 60) to HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Memory Management in the registry.

If it is already present but higher than 60, record its value and try changing it to 60.

In both cases a restart will be required.

Thanks - I tried this setting at 60 and at 40 (which I read sometimes works elsewhere on the internet) and neither made a difference, unfortunately. Still the same error.

Link to comment
Share on other sites

  • 0
20 minutes ago, Shane said:

Perhaps you could work around the problem with a third-party copier (e.g. UltraCopier, FastCopy or Copy Handler) or file manager (e.g. Double Commander), as some use their own copying code (to varying degrees) rather than rely on Windows?

I tried TeraCopy, and it failed silently, which was disappointing. Let me try one of your suggestions and see how that works.

Link to comment
Share on other sites

  • 0

Other registry changes to try (separately or together):

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management

PagedPoolSize (REG_DWORD, Hexadecimal) to ffffffff (that's eight 'f' characters)

DisablePagedPoolHint (REG_DWORD, Decimal) to 1

These changes should be reverted if they don't help.

You could also try resetting security permissions on the pool (yes, that sounds weird but yes, it may fix it).

Link to comment
Share on other sites

  • 0

Hi Shane,

Thanks - I tried those changes and the icacls permissions reset on the pool drive. I was hopeful, but the same error still occurs.

I also looked at the Service Log in DrivePool's troubleshooting menu, but didn't see anything listed as an error.

However, a couple other interesting data points:

  • I was apparently able to copy my 13 GB test file from a Mac to the pool over the network. Oddly, the Mac would not show the file in Finder after the transfer - it seemed to think it didn't work and the file wasn't there. However, looking in the pool directory on the target showed that the file was transferred successfully.
  • When I went to the target machine to check the pool directory, I got a message that the recycle bin on the pool was corrupt and needed to be emptied. I did that, and then tried a transfer from my Win 7 PC (the one I normally use as a source machine) to the pool; still didn't work.
    • Normally I'd say it's got to be some quirk of the Win 7 machine that's causing the problem, but going back to the 3rd bullet in my original post, it still seems like it's got to be something about the pool. Otherwise I don't see how I can successfully copy the same file to the same target PC from the same Win 7 PC as long it doesn't go to the pool, but I'm willing to be proven wrong.
Link to comment
Share on other sites

  • 0
42 minutes ago, Umfriend said:

There must be some sort of log file showing a bit more info. Small chance you'll learn anything but you might want to take a look at Event Viewer, both on the client as at the server.

I have checked event viewer on both, but there's nothing useful - when copying the test file and it fails, Event Viewer has no warnings, and when my backup software runs and fails, it does generate an entry, but it's the same as the error message: "insufficient system resources." Wish error reporting had more details!

I think DrivePool does have more logs, but from what I gather they aren't human-readable and I'd have to file a ticket, which I may end up doing in the next week or so if there's no progress. I'm hoping to solve it on the forums first if possible before resorting to that.

Link to comment
Share on other sites

  • 0

Hmm. What happens if you tick Bypass File System Filters under Manage Pool -> Performance?

Have you changed or updated your antivirus software, if any? (note: antivirus software may continue to interact at the driver level even when supposedly "off" according to the user interface; you may need to uninstall the antivirus software completely for it to be actually off).

Link to comment
Share on other sites

  • 0

It got a little farther than it normally does with Bypass File System Filters checked, but still failed with the same error.

I haven't changed or updated AV software on either machine - destination PC uses built-in Windows Defender and the source PC has been running the same version of Norton for a while now (I know, Norton is not great, but I got it for free).

Link to comment
Share on other sites

  • 0

1. Do you encounter the problem when you copy from a local disk on the pool computer to the pool via the pool's network share (e.g. "\\poolcomputer\poolshare\") instead of via the pool's drive letter (e.g. "P:\")?

2. Is there any difference between copying over the network from another computer to the pool via a mapped network drive (e.g. "P:\") versus the UNC share (e.g. "\\poolcomputer\poolshare\")?

3. On the source/target computers, has there been a network driver update (or any other update) around or shortly before the time this started happening? (check the Windows Update history)

 

Link to comment
Share on other sites

  • 0
On 2/27/2021 at 10:12 PM, Shane said:

1. Do you encounter the problem when you copy from a local disk on the pool computer to the pool via the pool's network share (e.g. "\\poolcomputer\poolshare\") instead of via the pool's drive letter (e.g. "P:\")?

2. Is there any difference between copying over the network from another computer to the pool via a mapped network drive (e.g. "P:\") versus the UNC share (e.g. "\\poolcomputer\poolshare\")?

3. On the source/target computers, has there been a network driver update (or any other update) around or shortly before the time this started happening? (check the Windows Update history)

 

Hi Shane,

Hope you had a good weekend. I tried the suggested actions and here's where I ended up:

1. No - local disk copying on the pool PC works fine for both drive letter and UNC path.

2. No - it still fails over the network if I map a drive to the destination.

3. Checked and there have been no network driver updates on either machine in the last few months.

I did notice something else today - although I can copy smaller files over the network without it failing as quickly, I tried copying about 10,000 small files that total up to 100GB, and eventually it failed with the same error as I get on large files after having only copied about 2000 files over. Again, it wouldn't be the disk space that's the issue. I did notice that I could click "try again" and the copy seemed to complete. Of course, with a large file or a scheduled backup that wouldn't be an option, so it isn't much of a help.

I tried the same thing over the network but to an individual drive in the pool instead of the pool share, and that worked, although it seemed to be much slower than copying to the pool.

I don't know what it is between the network and the pool - is it possible that the copying is happening too fast for Drivepool and the system interprets it as a network or disk error? Seems unlikely or others would have noticed it.

Really strange...

Edited by jp7
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...