Jump to content
  • 0

Directory\File Queries


RandomEvents

Question

I'm curious if (and how if so) DrivePool caches directory and file information instead of looking at the disks?

For context I have a 24 HDD drive pool (>210TB) that has x2 duplication enabled for the entire set. Used as a media server to store video. The OS (Windows Server 2019) and any programs are on SSDs (SATA or NVMe) and the system has 8GB of RAM (supposed to be 16GB, error I need to investigate) and SB Scanner. I've always been hesitant to move audio, books, and comics over to the pool because of potentially how long it would take to scan such a large number of smaller files across the pool. Right now those files live on a mirrored Storage Pool (SSDs) on my other server (everything is reachable with SMB and DFS).

If the file and directory info is cached in memory or a db that's great and worth testing, if not, is there anything I can do as a user to increase performance?

Link to comment
Share on other sites

2 answers to this question

Recommended Posts

  • 1

DP does not have a cache of files etc

it provides the virtual disk to windows and it takes over from there

having more memory will improve things to a degree as 8gb is very much on the low side for a server with potentially millions of files in a pool of that size

the performance will depend on the cpu used, memory and the speed of the disks used

i have a similar setup with 31 disks 7 ssds and 24 HDD and the speed is generally fine

my media server (Emby) can scan the disks in a couple of minutes when checking for new additions etc - the initial scan is slow as it has to process all the files and can take a few hours when doing a new setup - but after that its not an issue

i have had over 500k of photos in emby and after the initial scan it was not an issue for the media server to find and access files

i think you are worrying about something that is not going to affect a well specified server - as windows 2019 is designed to be a file server as well as all the other stuff it can do. Yes if you had all SSD's things would be snappier and reading lots of small files (always slow) would be quicker but its not practical unless you are Elon Musk and have money to burn. 

Give it a try - you can always move them back if performance is not good enough etc -i think you will find it will be fine

Link to comment
Share on other sites

  • 0

Thanks, after reading your reply, I reread the description on the DrivePool page and realize how they handle queries (queries multiple drives at once and presents a combined list). Going through the advanced settings wiki, upping the CoveFs_AsyncFindFilesThreads parameter might give me a bit of a boost.

 

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...