Issue When Working With Folders

[GavinCampbell]

I have a strange problem and don't have any clue as to where to start. 

 

I'm running the latest version of drivepool (2.1.1.561) on Windows Server 2012 Essentials R2 with 4 WD Red drives and a SSD drive for the cache.

 

When working with folders with a number of subfolders and files (via a network share or directly on the box) the folders seem to get locked and I am not able to delete them.  It's almost like the pool is very slow to catch up.  Sometimes I may reboot and they may unlock and let me delete them but sometimes I have to actually go to each drive and delete it directly from the pool parts. 

 

It only happens when I try to delete the top level folder.  If I manually go through every sub directory and delete the files before deleting the folder they are in it works.  But this is not practical.

 

To help eliminate other programs that may be interfereing I started up the server with nothing running on it just to ensure it is still happening and it is.

 

Any ideas as to where I can start troubleshooting this?  Logs? Monitoring?  Anything?

[Christopher (Drashna)]

Okay, I just wanted to make sure. And no, a reboot shouldn't be necessary here.

[SolidSol]

Excellent.  So far it looks like it is working but I will report back in a few days once I have a chance to really try.

 

As for the eSATA issue.  I don't think its an issue with DrivePool.  It was doing this when I was testing other products as well.  I was able to troubleshoot and reduce the frequency of it going offline by disabling the link state power management.  But when the enclosure was under really heavy load (like when scanner was running) it would still drop off randomly.  I just switched over to usb3 and everything was working great so figured it was either the enclosure or the card.  I was using the silicon image card for esata.  From what I read that had issues.

Hi

It seems that I have the same problem. When do you plan to make the official release?

[Christopher (Drashna)]

Hi

It seems that I have the same problem. When do you plan to make the official release?

If you mean the build that fixes the deleting folders issue:

Not right away, we have a major change we want to include before pushing a new release build (as well a lot of testing with it).

However, the built builds are available here:

http://dl.covecube.com/DrivePoolWindows/beta/download/

 

The 2.2.0.586 build is the newest build as of day. It includes this fix.

Please download that version for your system and see if that fixes the issue. If so, let us know.

[SolidSol]

If you mean the build that fixes the deleting folders issue:

Not right away, we have a major change we want to include before pushing a new release build (as well a lot of testing with it).

However, the built builds are available here:

http://dl.covecube.com/DrivePoolWindows/beta/download/

 

The 2.2.0.586 build is the newest build as of day. It includes this fix.

Please download that version for your system and see if that fixes the issue. If so, let us know.

What about stability of the build 2.2.0.586? Is it covered by tests? Any known issues?

I just have a big disk pool and don't want to loose any information.

Thanks.

[Christopher (Drashna)]

What about stability of the build 2.2.0.586? Is it covered by tests? Any known issues?

I just have a big disk pool and don't want to loose any information.

Thanks.

Our beta process is pretty simple.

 

If we find a bug or somebody else does, we figure out what is causing it, write a patch, test the specific issue and then push the new installer.

These are not well tested. However, we have a number of users (myself included) that run these "nightlies" on their systems in production. So we do get feedback about them. And for the most part, they are pretty stable.

That said, I'm using the 2.2.0.586 build on my production server (where all my media resides, and my work VMs are), and it's been very stable.

 

However, the release versions receive a lot more stress testing, to make sure that they are stable.

 

But in either case, we are generally very quick about fixing serious issues (like this).

[GavinCampbell]

You can use Marvell chipsets as well, but ASMedia is the most reliable we've seen.

 

 

Just upgraded my setup to the Mediasonic esata card with the chipset you recommended. 

 

So far smooth.  Pretty sure it will stay that way. 

[Christopher (Drashna)]

I'm glad to hear it!

 

If you're using the ASMedia card, then make sure you install the drivers for it.

Windows will identify and install generic drives, but it doesn't support port multiplication. You need the actual ASMedia drivers for that.

[GavinCampbell]

I'm glad to hear it!

 

If you're using the ASMedia card, then make sure you install the drivers for it.

Windows will identify and install generic drives, but it doesn't support port multiplication. You need the actual ASMedia drivers for that.

 

Yup.  Mediasonic had the drivers for it.  It port multiplication wouldn't work until it was installed.

 

Working fine so far on Server 2012 R2 Essentials.

[Christopher (Drashna)]

Okay, I just wanted to make sure.

 

And yeah, it definitely does.

And I'm glad to hear it.

[Thronic]

Actually, it looks like it pointed out a rare, race condition when deleting folders with contents. 

 

Could you go into more specifics about that? I just want to understand it more. 

 

Is it the Windows API crashing with some kind of DrivePool filter when taking care of deletion, causing (in my best way to explain it) "multiple access attempts" in a bad sequence? 

[Christopher (Drashna)]

It's entirely caused by DrivePool, and the way we handle deleting folders IIRC.

Specifically, this is the bug in question:

[D] [issue #13443] When a folder delete fails because of files in one pool part folder, roll back the delete on 

                     any other pool parts.

 

Just to forewarn you, I'm not going to explain this quite right, because this is a VERY low level file system issue. I talked with Alex directly about this, so I don't have the advantage of a nice issue tracker conversation to fall back onto to give a very detailed, accurate and correct explanation here. 

 

Specifically, we are marking them as deleted, failing to delete a folder and leaving them in "deleted" state. But this only happens when you have empty folders on other pool parts, IIRC.

When deleting a file, it adds a "deleted" attribute to it before actually deleting it. We were setting that when deleting the files (which is normal), but we were not unsetting it when the delete failed (which is not normal). So when you try to access the folder afterwards, it would end up in an unusable state because the file system sees that the folder is in the process of being deleted. 

 

The patch corrects this by better handling the delete process, and if necessary, rolling back the changes (removing the attribute) if needed.

 

 

 

 

If needed, I'll flag the thread for Alex, and he can see about giving a better explanation here.