Jump to content
Covecube Inc.
  • 0
Firefly

Problem with hdd standby

Question

Hello,

 

I have a problem with the standby (actually it's the spinup) of my drives.

 

Initial setting: All drives are in standby (= spinned down)

 

Accessing share -> Spinup of ALL drives.

You don't even have to browse through your share, opening it (e.g. mount network drive) is enough to make the drives spinup.

Why do all drives spin up?

I mean, ok, it's logical that they spin up so you get the directory etc, but why isn't there an internal drivepool cache storing the directory structure (folders, size etc)?

With a cache like this there would be no need to spin up all drives.

 

All clients already have a cache like this. Once you accessed your share (drives spun/span (?) up) everything is stored in this cache. Once you access your share a second time no drives spin up (when in standby).

So we need a server-side cache from which the clients can retrieve the data to build their own cache.

 

In my case it's very annoying: I set up a feeder disk to store all my incoming data (in a specific folder which is only allowed to store data on the feeder disk) but when I access this new data all drives do a spinup which makes the feeder this pretty useless to me.

 

Can you please make a server-side cache? Or something similiar, I don't know ^^

Share this post


Link to post
Share on other sites

20 answers to this question

Recommended Posts

  • 0

No, therei s no internal caching for StableBit DrivePool. We maintain no database. That is why it may spin up the drives like this. 

 

However, Windows does cache disk access, which would "trickle down" to us. 

If you have a large amount of memory available on the system, then you may want to consider increasing the memory that the Windows uses for caching:

 

The command is: "fsutil behavior set memoryusage 2" (run from an elevated command prompt, and then reboot).

Details about it here: http://technet.microsoft.com/en-us/library/cc785435.aspx

Share this post


Link to post
Share on other sites
  • 0

Thanks for that command, it seems to work. :)

 

//Edit:

Okay, I did some more testing and it clearly does not work.

Maybe it's time for an internal cache/database?

 

Well I could use reparse points on my OS drive (which is always running) so that nothing would spin up. Unfortunately they don't work properly (see other thread)

 

//Edit2:

Okay I have no idea, sometimes it works and sometimes it doesn't. :huh:

Share this post


Link to post
Share on other sites
  • 0

Yeah, that's.... basically expected. It really depends on what the kernel wants to keep cached. And unfortunately there isn't any other settings to use even more memory.

 

As for an internal cache/database.... the issue with that is how do you keep it in sync? You start to introduce a whole lot of issues. Including some of the serious issues that Drive Extender from WHSv1 had... You also start needing a lot more resources and writing to the disk a lot more. And any database can adversely affect performance. 

We've considered it, but this is the best method, in our opinion. Also, NTFS *is* a database, of sorts. For files.

 

 

But specifically, why it works sometimes and not others is that it the kernel (I believe) keeps a certain amount of IO activity (that includes LISTING files) in memory. And it "dumps" this out of memory as it feels necessary. the "memoryusage 2" command tells windows that it should use more memory than default. This means that it's more likely to keep this stuff in memory and not cause disk access.

Share this post


Link to post
Share on other sites
  • 0

The cache could be created initially on system startup (when all drives spin up anyway) and could get synced when files change. That's it, isn't it?

Share this post


Link to post
Share on other sites
  • 0

hehehehe....  I laugh because I tease Alex every time he says something "should be simple". Because it NEVER turns out to be simple. Ever. (well maybe sometimes, but usually not).

 

In ten minutes of usage, the driver can get hundreds of thousands of IO requests (this is a guess, TBH, and Alex can post more realistic stats, if inclined). We would have to track each and every single one. And then, we would have to filter through these, figure out which are relevant and which are "junk". And then we have to make sure that none of these requests error out, are canceled or are reversed (which happens, such as in the case of a failed rename/copy/move).  And then update the database/cache, on each successful change.

 

You can see how this could cause a serious impact on performance. Both on disk performance but also on memory/processor usage.

 

And this is basically what the NTFS driver does already. But it "flushes" the cache to the disk periodically (and by periodically, this delay is measured in milliseconds or less). In fact, that's what that "memory usage" setting increases. But to a limited degree.

 

So instead of reinventing the wheel from the ground up... we just hook into it, basically, and let 20+ years of development handle that for us.

 

 

 

 

 

At least, I am pretty sure that is what the "situation" is. Alex is DEFINITELY the expert here. I'm just relaying the situation and information as best as I understand it. He's the one that has written the driver and everything. But I've "pinged" him, so he can either tear apart what I've said, confirm it, or at least provide his point of view.

Share this post


Link to post
Share on other sites
  • 0

I'm trying to clearly define the problem here. As far as I understand, when you access any share on the pool (from a remote client computer?), all of the drives spin up?

 

As far as having a server side cache, we already do, and a client side cache in the form of oplocks (this means that the server isn't even contacted over the network for some I/O operations).

Share this post


Link to post
Share on other sites
  • 0

Yes exactly, all drives spin up. e.g. when I mount a network drive or access it via win explorer (from a remote computer).

 

Could it be caused by Lightsout? I configured Lightsout to monitor my shares for activity.

Can't check it right now cause I'm not at home till sunday.

Share this post


Link to post
Share on other sites
  • 0

Depending on how it's monitoring them, then yes, it definitely could.

 

Though, the best way to find out would be to disable it and see if it changes behavior.

 

Also, is anything else accessing the the disks? At all?

Including StableBit Scanner? If you do have StableBit Scanner installed, throttle the SMART queries, as that *may* help. 

But anything that queries the drives in the pool or the Pool itself could cause that behavior.

 

 

Also, by "mounting a network drive", do you mean that you map it to a drive letter? 

If so, I vaguely remember that this can cause access to the shares. I can't find confirmation nor relevant links, ATM though.

Share this post


Link to post
Share on other sites
  • 0

I'll deactivate the specific feature in Lightsout as soon as I have (physical) access.

 

No, I don't have StableBit Scanner installed, so that's not the cause of the problem.

 

And yes, I assign a drive letter. But the drives spin up via win explorer (\\Address), too. So mapping it to a drive letter itself is not the problem.

Share this post


Link to post
Share on other sites
  • 0

Maybe I've found the problem.

I manually assigned drive letters to my pool drives to browse through them via windows explorer.

In total I have 11 folders in my (single) share but only 4 of them exist on my feeder disk. So the system has to spin up other drives to retrieve them (names, attributes etc.).

 

Is there a way to create them on the feeder disk? Can I do it manually or would this cause problems?

 

//Edit:

Okay, Lightsout is definitely not the causing the problem, I just deactivated the share-monitor-feature but the drives still spin up. So the above is my best guess.

Share this post


Link to post
Share on other sites
  • 0

I found a way to trigger the folder creation of drivepool: I copied a small text file to every missing folder and now I have all folders on my feeder disk.

 

I need to do more testing but it seems to work so far, all drives stay in standby and only the feeder disk is active - with an empty cache (after a reboot).

 

//Edit:

-.-

Most of the time it works but sometimes it doesn't... This doesn't make any sense. With an empty cache (after reboot) it worked but now with a filled cache (I guess) it's not working.

Share this post


Link to post
Share on other sites
  • 0

Uploaded it, #4693230.

I only included the newest logs and files.

In the .log file there's not much to see. Maybe in the other files but I don't know cause I can't read those.

 

//Edit:

Wow, now I'm officially confused.

The drives only spin up if I hover (with my mouse) over a folder. They don't spin up if I only click on my feeder-only-folder.

So I guess windows queries something when I hover over a folder and wants to retrieve something, maybe the size? If it's the size it would make sense. Though the folders exist on the feeder disk, the files in them don't.

Share this post


Link to post
Share on other sites
  • 0

When it hovers over the folder, it reads the folder property, and I believe it also checks for the thumbnail file. I suspect that this last bit may be the key here.

Have you tried disabling folder preview on the system?

Share this post


Link to post
Share on other sites
  • 0

No, only on my client, but this shouldn't be a problem.

As far as I know the thumbnails are only created if you access the folders and I'm not doing that on my server machine.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

×
×
  • Create New...