Jump to content
  • 0

File placement misbehaviour


HPLovecraft

Question

Hi there - I've been running DrivePool & Scanner successfully since new year, and today I had one of my server HDDs fail. No big deal, as my files were all duplicated in Drivepool.

 

I have quite a strict regime in where I allow my server folders to live: server Documents & Pictures are duplicated on 2 HDDs; server Music & Videos are duplicated on the other 2 HDDs. I won't bore you with why - this just works for me.

 

The drive that failed is one of the pair that stores my Documents & Pictures. I removed it from the pool and have ordered a replacement. Whilst I'm waiting for it to arrive, I've added an unused spare partition to the pool, so that Documents & Pictures are still duplicated. But something's changed - a new Document does not get duplicated on the correct disk any more, and it's driving me mad. Here's my file placement rules:

 

DrivePool_270715_zpskf1lm9ua.jpg

 

So on my PC, I create a new document & drop it into the server's Documents folder. On the server it appears on disk G:\ as expected, but it doesn't appear on disk J:\, as per my rules. Instead, a new Documents folder with the file in it is created on disk I:\.

 

I simply can't understand this, and it could really mess up my backup regime. At the moment I daren't put files on the server. The only thing that I can think is different, is that J:\ & I:\ are two partitions on the same physical HDD. Is DrivePool unable to cope with this - should I be using different physical disks with files placement rules?

 

In a few days I'll have a new 4th physical disk to slot in, so maybe it'll be OK, but this has got me a bit worried... If anyone can give any guidance or suggestions, I'd be very grateful. Thanks!

Link to comment
Share on other sites

9 answers to this question

Recommended Posts

  • 0

I can only assume what I was experiencing last week was DrivePool making an executive decision when my Seagates were going bad / were removed from the Pool: the program's desire to maintain duplication was considered a higher priority than obeying my file placement rules..? I'm not complaining - duplication should come first.

 

So normality has been restored (touch wood).

 

The only thing that I still don't understand is the weird way that DrivePool handled the partition that I added temporarily to the Pool, in order to help take the strain during the downtime.

 

@Christopher: does DrivePool consider two partitions on the same physical HDD to be separate disks? Or is there something about partitions that's 'special' in DrivePool? Thanks!

 

Duplication is the priority. The code regarding it tries to respect the balancing setttings (including file placement rules), but will disregard them if it's not able to find a valid drive to place the files.

This is intential, as duplication is literally the most important feature here. As it should be.

 

 

 

As for the drives, Yes, StableBit DrivePool is "aware" of the physical disks.  It will ACTIVELY avoid placing duplicated data on the same physical disk, if you have two partitions on the same drive added to the pool.  This way, you don't lose data because we "did something stupid" basically (what's the point of duplicating the data if both copies are on the same disk!!). 

This is also why we don't support Dynamic Disks. It makes the detection significantly more complex, and would adversely affect performance.  That's coupled with the wierd and random issues that Dynamic disks have, especially when moving to other systems (the "import foreign disks" thing can fail, and I've had it happen to me .... it's not fun, as you've basically just lost your data because of this). 

 

 

 

And hopefully, everything stays stable and sane! And it definitely sounds like the failing drives were the issue.

Link to comment
Share on other sites

  • 0

Hi Christopher - thanks for your reply. Don't know how I missed that "Never allow..." button. I don't know if it was clicked in my original setup - when I seeded the newly-added partition last night, as part of the procedure the guide said to reset DrivePool - and of course I lost all my settings. Once I get it working properly again I'll screengrab all the settings for future reference. It would be great if it were possible to seed a new drive without that step.

 

Anyway, I checked the "Never allow files to be placed on any other disks" button, and now the interface looks like this:

 

DrivePool_280715_zpsyjrqahhd.jpg

 

Despite reading the user guide I still don't really understand what that 'Pool Organisation' bar means. I have 4 x drives, and I'm deliberately forcing Drivepool to treat them as 2 mirrored pairs, so maybe that's the reading I should expect (?).

 

Anyway, DrivePool immediately started duplicating files again, same as last night: it seems determined to force a copy of my Documents folder onto drive I:\, though I can't understand why - it contradicts the placement rules and the "Never allow..." setting. I'm going to have to abort it - it's a lot of files and I definitely don't want them on disk I:\.

 

None of the disks are even close to being 90% full - I don't think it's an overflow problem. Can DrivePool tell the difference between two lettered partitions on the same disk? Could that be my problem? Drive J:\ (where I want the files) and drive I:\ (where DrivePool is duplicating them) are both partitions on the same physical HDD.

 

regards, Simon

Link to comment
Share on other sites

  • 0

NEWSFLASH! This just in...

 

Whilst I was typing my previous post, DrivePool finished the duplication of my Documents folder to the (incorrect) drive I:\. But it hasn't copied the files: it has simply created a version of the Documents folder structure using empty folders. Not all the folders are present, but the biggest three folders appear to be replicated in their structure of nested subfolders, all the way down.

 

However, the original problem remains: if I create a new document and drop it into my server's Documents folder, DrivePool puts its second copy on the wrong drive (I:\ when it should be J:\).

 

I also tried it dropping a new file into my Pictures folder - DrivePool immediately created a Pictures folder on drive I:\ (where none should be) and put the second copy of the file in it (not on drive J:\ where it should have gone to).

 

Also, something else new has happened - these indicators have appeared on my disk list, I've never seen these before:

 

DrivePool_280715_2_zps6xfcdmgi.jpg

 

I've read & re-read the entry in the User Guide without really understanding it - these arrows are: "...showing you where the balancing system wants that particular bar to be.". Does this mean that the balancing system "wants" to empty drive F:\, and distribute the files around the other 3 drives? 

Link to comment
Share on other sites

  • 0

Ooooo... things just took a turn for the worst, and it looks like I now have bigger fish to fry.

 

Last night a second drive died, with no warning, So now both of my 4-year old 2TB Seagates have died within 48 hours! That seems unlucky, even by my standards. Both of my 6-month old Hitachis are still OK in Scanner (though one needed a file system repair).

 

I installed DrivePool, Scanner & the two Hitachis at new year, and am now feeling smug at the way I paired the HDDs (a Seagate always duplicated on a Hitachi, hedging my bets on brand reliability).

 

I already have a new Toshiba HDD on order to replace the first failed Seagate, looks like I need another...

 

I remain baffled on the original file placement problem though... the placing of new Documents & Pictures files on the wrong disk was not a result of this second failing Seagate being evacuated - as that disk had never been permitted as a destination for the files in the first place. Meanwhile, DrivePool has created even more nests of empty Documents folders on the 'wrong' drive.

 

I'm still open to suggestions, though I'm inclined to put this problem on a back burner until the server is repopulated with 4 working drives again...

 

Has anyone ever had two drives go on them like this..? It seems incredibly unlikely, I'm thinking it's either:

  • Something in my server is faulty & fried the 2 Seagates in quick succession. Is that possible? What could do that?
  • or...
  • Both Seagates were Friday-afternoon lemons from the moment they were made. I bought them at the same time from Amazon - for all I know they were manufactured within minutes of each other. Is that a stupid theory?

Opinions / advice would be welcome, as always!

Link to comment
Share on other sites

  • 0
SNIP

 

Has anyone ever had two drives go on them like this..? It seems incredibly unlikely, I'm thinking it's either:

  • Something in my server is faulty & fried the 2 Seagates in quick succession. Is that possible? What could do that?
  • or...
  • Both Seagates were Friday-afternoon lemons from the moment they were made. I bought them at the same time from Amazon - for all I know they were manufactured within minutes of each other. Is that a stupid theory?

Opinions / advice would be welcome, as always!

 

I have had 3 drives fail in a 2 week span, 2 of those three were ~4 days apart. All drives by the same manufacture.  Now my set up consists of 48TB so I have many drives so my failure % may not be all that "unlucky". 

 

I alway duplicate important info at least 4x's...

Link to comment
Share on other sites

  • 0

 

I have had 3 drives fail in a 2 week span, 2 of those three were ~4 days apart. All drives by the same manufacture.

 

Ah well, at least I'm in good company!  :D

 

This has been a trying week, but it's confirmed the wisdom of switching over to DrivePool & Scanner with 2 additional disks at new year. My previous configuration was just the 2 Seagates in a Raid1 array. The first Seagate would have gone bad, and I'd have ordered a new HDD as I did this week with the intention of installing it and rebuilding the array. But before it was delivered, the second Seagate would have gone bad too - ouch!

 

Don't get me wrong, I keep backups on- and off-site, but it would have been a tedious experience. DrivePool is way more flexible and friendly for my needs (even if it baffles me sometimes..!).

Link to comment
Share on other sites

  • 0

Well, I'm very sorry to hear about the additional disk dying! :(

 

 

Has anyone ever had two drives go on them like this..? It seems incredibly unlikely, I'm thinking it's either:

  • Something in my server is faulty & fried the 2 Seagates in quick succession. Is that possible? What could do that?
  • or...
  • Both Seagates were Friday-afternoon lemons from the moment they were made. I bought them at the same time from Amazon - for all I know they were manufactured within minutes of each other. Is that a stupid theory?

Opinions / advice would be welcome, as always!

As for the second disk dying, if both were bought at the same time, then it's likely that both were from the same batch.  And if both were from the same batch, then it is very likely that they had the same defects, as this is normal in the fabrication process of pretty much anything that is done electronically or with automated machines. 

 

If both were put into the pool at the same time, and have been being used the entire time, then it's very likely that there was some batch defect.  However, I'm very sorry that this happened regardless.

 

 

 

And it may be possible that the issues you were seeing were related to the the drive's rather sudden demise. There may have been underlying issues that didn't show up in SMART (and if you didn't have a recent surface scan). 


As for the indicators on the disk...

 

The red arrow underneath it is the "real time placement limiter". This denotes the limitation of new files to be placed on the disk. In your case, none. 

 

The blue arrows indicate the target for rebalancing (eg, files will be balanced until they hit that mark specifically.

 

Given what you've described, I would suspect that the reason your seeing this is because one of the drives showed up as damaged in Scanner. That would cause this behavior. 

 

NEWSFLASH! This just in...

 

Whilst I was typing my previous post, DrivePool finished the duplication of my Documents folder to the (incorrect) drive I:\. But it hasn't copied the files: it has simply created a version of the Documents folder structure using empty folders. Not all the folders are present, but the biggest three folders appear to be replicated in their structure of nested subfolders, all the way down.

 

However, the original problem remains: if I create a new document and drop it into my server's Documents folder, DrivePool puts its second copy on the wrong drive (I:\ when it should be J:\).

 

I also tried it dropping a new file into my Pictures folder - DrivePool immediately created a Pictures folder on drive I:\ (where none should be) and put the second copy of the file in it (not on drive J:\ where it should have gone to).

 

Also, something else new has happened - these indicators have appeared on my disk list, I've never seen these before:

 

DrivePool_280715_2_zps6xfcdmgi.jpg

 

I've read & re-read the entry in the User Guide without really understanding it - these arrows are: "...showing you where the balancing system wants that particular bar to be.". Does this mean that the balancing system "wants" to empty drive F:\, and distribute the files around the other 3 drives? 

Link to comment
Share on other sites

  • 0

Just to (hopefully) draw a line under this thread...

 

This week I installed 2 nice new HDDs (Toshibas) to replace my 2 x failed Seagates. I let Scanner look at them before I bothered seeding them with files & adding them to the Pool.

 

I'm now back to my usual storage solution:

Documents, Pictures on one Hitachi + Toshiba pairing

Music, Video on the other Hitachi + Toshiba pairing

 

Now I have 4 x disks in place, paired up nicely, my file placement weirdness seems to have gone away. Looks like DrivePool is once again respecting my file placement rules, and is no longer creating strange structures of inappropriate empty folders on the wrong HDDs.

 

I can only assume what I was experiencing last week was DrivePool making an executive decision when my Seagates were going bad / were removed from the Pool: the program's desire to maintain duplication was considered a higher priority than obeying my file placement rules..? I'm not complaining - duplication should come first.

 

So normality has been restored (touch wood).

 

The only thing that I still don't understand is the weird way that DrivePool handled the partition that I added temporarily to the Pool, in order to help take the strain during the downtime.

 

@Christopher: does DrivePool consider two partitions on the same physical HDD to be separate disks? Or is there something about partitions that's 'special' in DrivePool? Thanks!

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...