Jump to content
Covecube Inc.
  • 0

Files are not sorted alphabetically when using the command prompt


Question

Normally, Windows sorts files alphabetically when enumerating a directory from the command prompt.  I have several tools and batch files I have created over the years to do various batch operations to all the files in a folder.  Many of these tools rely on the fact that the operating system enumerates the folders alphabetically.  With DrivePool, this is not the case.   When enumerating a folder on a DrivePool volume, if the files in that folder span multiple drives, each individual drives files will be sorted alphabetically, but it will start over as it goes from drive to drive.

 

Is this a bug or a limitation of DrivePool?

Link to post
Share on other sites

4 answers to this question

Recommended Posts

  • 0

IIRC, this is unfortunately an issue with how DrivePool enumerates the content of the folder, and displays it.

 

Most utilities will reorder it properly, but some programs/utilities have issues doing so.

 

If you don't mind me asking, what tool/command are you using that is having this issue?

 

And I've flagged this for Alex (the developer).

(https://stablebit.com/Admin/IssueAnalysis/15571)

Link to post
Share on other sites
  • 0

Alex has posted a rather detailed explanation of the issue. I'm not going to paraphrase it.

 

https://stablebit.com/Admin/IssueAnalysis/15571

 

In the kernel, there is actually no requirement that the the directory listing has to be in any particular order. That's actually the job of the application making the call.
 
Because NTFS directory index entries are stored in a binary tree, they always happen to be listed in alphabetical order. So it would be nice if we could do that as well, but it's really not ideal because of the performance overhead it would introduce.
 
We would have to pre-buffer the entire directory listing, then sort it, and finally return parts of it to the caller upon request.
 
Instead we use a "fast directory listing" algorithm, and it works somewhat like this:
  • Each disk in the pool is queried for a list of directories in parallel.
  • As each disk starts to responds with blocks of data, we buffer that data internally and once we have enough of it, we respond to the caller.
  • The caller may ask us for the "next" block of entries, and usually we can instantly come back with more from our internal buffer.
This all happens in the kernel and so it's really fast. It is however probably one of the most complicated pieces of code in DrivePool, and is really at the heart of how it performs its pooling magic.
 
Now you may say, this is all theoretical and today's computers are fast enough, why not just pre-buffer everything? Well, we had one case were someone was storing hundreds of thousands of files in a folder on the pool and even this method of listing directories was excessively slow. I had to re-tune the data structures used in the listing algorithm in order to support that kind of scenario.

 

 

tl;dr:

Basically, NTFS usually stores it in alphabetical order. But because we're combining the list, it may screw up that order.

We could fix this, but it could severely impact performance, especially with folders with a LOT of content in them (or on a slower system).

Link to post
Share on other sites
  • 0

because it's been requested repeated, Alex has decided to implement this feature.  However, this is an optional feature, that must be enabled in the Advanced Settings:

Set "CoveFs_SortedDirectoryQueries" to "True" and reboot the system.
 
This will significantly impact directory enumeration, however.
You can download the beta build for this here:
 
Regards
Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...