Sleeping drives & Pool access

[Da_Teach]

I've tried looking for it, but couldn't really find anything.

 

I've got a pool of 8x3TB drives (well, 10x3TB but 2 of those are parity with flexraid snapshot raid) and those drives go to sleep after 10 minutes of not being used.

 

Before trying Drivepool I was using FlexRAID and even though the developer said 'It only wakes up  the drive that has the file', my personal tests have shown this is not the case. It would spin up drive 1 to x until it found the required file.

 

So my question is, how does DrivePool handle this?

 

If I access a file will it spin up all drives, or does it know on which drive a file resides and just access that drive?

 

[p3x-749]

Interesting question.

I think that I have indirect proof, that each disk is spun-up individually and only when access is required.

 

As I am running DrivePool in my TV-Recorder, where the recording SW creates a thumbnail for every media-file/recording,

I can see an interesting effect when trying to access the files.

When browsing the repository via Web-UI, the Meta-Data for each recording is fetched from a database on the system disk.

The thumbnail is stored alongside with the recording file on the Pool and this is fetched during the browsing action as well.

 

However, on a first attempt only the metadata is shown..and only after repeating the query the thumbnail comes up too.

This must be due to the pool-disk spinning-up.

Now, after the first thumbnail *is* shown, other thumbnails (where the files reside on other disks) still show that behavior (whilst others from the same pool-disk display immediately at first attempt now).

If the whole pool had been spun-up in the first attempt, this effect  would not be observed.

Hence I conclude, that each disk is getting spun-up individually.

[Alex]

So my question is, how does DrivePool handle this?

 

• For unduplicated files, StableBit DrivePool forwards all file I/O to the individual disk that the file is on.
• If the file exists on more than one disks (i.e. is duplicated), then:
  • All file modification requests (such as writing to the file, setting its attributes, etc...) go to all of the disks that the file is on.
  • Read requests will either go to the first disk that the file is on (when read striping is disabled), or to one of the disks that the file is on, as determined by the read striping algorithm.
• A directory listing operation works by querying all of the disks in parallel. This will force NTFS to read the MFT directory indexes on all of the disks where the directory being listed exists. These indexes can be cached and can theoretically be served entirely from RAM.
• Opening a file is similar to a directory listing. This will tell NTFS to query its directory indexes on all of the disks part of that pool in order to locate the disks that contain that file. This also is done in parallel and can be served by the system cache.

So in short, StableBit DrivePool may spin up disks in many circumstances, I haven't really done testing to see how often this is.

[Da_Teach]

• A directory listing operation works by querying all of the disks in parallel. This will force NTFS to read the MFT directory indexes on all of the disks where the directory being listed exists. These indexes can be cached and can theoretically be served entirely from RAM.
• Opening a file is similar to a directory listing. This will tell NTFS to query its directory indexes on all of the disks part of that pool in order to locate the disks that contain that file. This also is done in parallel and can be served by the system cache.

 

So the caching is done by the system not DrivePool on the underlying drive?

 

Is there anyway to increase the cache size or do tools exist to help cache the directory structure?

[Christopher (Drashna)]

Yes.

 

As for increasing the file system cache size... It's not really supported, by us or Microsoft, but it's rather simple to do so:

http://technet.microsoft.com/en-us/library/cc785435.aspx

"memoryusage" section.. 

[Da_Teach]

So far I'm liking DrivePool more than the pooling feature of FlexRaid. The FlexRaid pool somehow caused stutters when playing video's, DrivePool doesn't seem to suffer from that at all.

 

I've set memoryusage to 2, not really sure if it's does anything though. At least I haven't noticed a change yet.

[Christopher (Drashna)]

Da_Teach,

 

You probably won't notice a change. It's all done very low level.  At best, you may notice less queries to the disks, and faster repeated access of files. That and maybe the "SYSTEM" process using more memory. But that would be about it.

 

 

As for DrivePool, all the IO operations (including reading) are passed onto the disks more or less directly (duplication+read striping does some extra stuff), so you should see performance on par with regular disk throughput. We try to stay as hands off of ... well everything as we can. And we're glad that you are definitely appreciating everything that has gone into making DrivePool.

[philbio]

Digging up an old thread here, I have Drivepool 2.2.0.651 and Scanner 2.5.1.3062, I have 5 drives in an un-duplicated pool and none of the drives ever spin down, I have windows power mangement enabled, I can spin a drive down manually via Scanner but the drive immediately powers back up, windows resource monitor shows no processes accessing the disks. If I stop the Drivepool service the disks stay down,so it suggests its a Drivepool issue, I am also running the Ordered File Placement plugin,  what am I missing?

[Christopher (Drashna)]

Digging up an old thread here, I have Drivepool 2.2.0.651 and Scanner 2.5.1.3062, I have 5 drives in an un-duplicated pool and none of the drives ever spin down, I have windows power mangement enabled, I can spin a drive down manually via Scanner but the drive immediately powers back up, windows resource monitor shows no processes accessing the disks. If I stop the Drivepool service the disks stay down,so it suggests its a Drivepool issue, I am also running the Ordered File Placement plugin,  what am I missing?

 

 

In StableBit Scanner:

Click "settings" and then "Scanner settings".

Open the "SMART" tab, and check the "throtlle queries" option. 

http://stablebit.com/Support/Scanner/2.X/Manual?Section=SMART

 

 

For StableBit DrivePool:

http://wiki.covecube.com/StableBit_DrivePool_2.x_Advanced_Settings

Set "BitLocker_PoolPartUnlockDetect" to "False" and reboot.

[philbio]

Thanks for the info, I've already trawled the forum and implemented both those changes with no success?

[Christopher (Drashna)]

To clarify, this only happens if the StableBit DrivePool Service is running?

If you stop the service, it no longer happens?

[philbio]

Hi Chris, yes that's right, it's got me at a complete loss, resource monitor shows nothing access the disks and when the stablebit drivepool service is stopped the disks power down after the 15 minute setting via windows HDD power options, no strange event log entries in windows? The disks aren't accessed too often so the power down ability is the preferred option, appreciate any further suggestions.

[Christopher (Drashna)]

Okay, thank you for confirming for me.

 

Have you opened a ticket at https://stablebit.com/Contactalready?

If not, it's not necessary, but if you have, let me know the ticket number. 

[philbio]

Hi Chris, no I haven't opened a ticket, thought it might have been some simple config setting I was missing, thanks for raising a ticket on my behalf.

[Christopher (Drashna)]

Well, I haven't really opened a ticket, but we do have one other user experiencing the same issue.  Logs and the like have alreayd been collected and the issue flagged for Alex.

 

But more logs are usually helpful

[Crack]

I also got this issue: I noticed small reads every 5 seconds on two drives that aren't part of any pool, and disabling "StableBit DrivePool Service" caused this to stop. After changing "BitLocker_PoolPartUnlockDetect" to "False" these reads disappeared permanently.

[philbio]

Hi Crack, great to hear you resolved your issue with the BitLocker_PoolPartUnlockDetect change, unfortunately this change didn't have any impact on the issue that I'm experiencing.