Jump to content
Covecube Inc.
  • 0
ars3nic

Duplication disappeared?

Question

Hey all,

I've been using DrivePool for a year or two now with no issues.  Recently I added an 8TB HDD and moved a bunch more data into the pool, to where I was at about 15TB of free space (out of 30TB RAW).  Most of my data has 2x duplication, with a few folders set to 3x.  Everything was fine until about an hour ago, when I added about 1000 small files (totaling just 15MB) to the pool that were being processed by an application and then automatically moved to a different folder in the pool.  About halfway through them, DrivePool suddenly 'lost' all of its duplication, notified me about balancing issues, and the bar at the bottom was yellow and had a warning.

Here's a screenshot -- immediately prior to this, just over half of that pie chart was the normal light blue color and I had ~15TB duplicated and ~15TB free.  DrivePool is currently reduplicating and rebalancing, and quick math suggests that all my data might still be there.....but what the hell happened?

Lzc9Ls0.png

Share this post


Link to post
Share on other sites

6 answers to this question

Recommended Posts

  • 0

Things I'd do to try and get a better grasp of what's on the pool now:

  • Check your settings 3x duplicated folders, to ensure it still has those correctly.  The UI you posted still has 2x pool duplication shown in the lower left corner.
  • Open Drivepool, click the gear icon in the upper right, go to Troubleshooting, choose "Recheck duplication.."
  • Once that's done, do a full re-measure on the pool under Manage Pool.
  • Do some quick math on your data sizes, see if it looks like it's all still there.

Did you have real-time duplication set on the pool, or was it set to delayed?  The activity in combination with delayed duplication could throw off the measurements I'd think (add a file to the pool for DP to duplicate, but before it can you move it again).  I tend to keep real-time duplication turned on, so I'm certain that stuff happens when submitted the first time.

The application you use that adds files to a fully-duplicated pool and then "automatically moves them to another folder" might be messing with Drivepool's duplication schedule, throwing the measurements off (or the display) and confusing it.  Hard to know without seeing screenshots each step of the way.  

 

Share this post


Link to post
Share on other sites
  • 0
31 minutes ago, Jaga said:

Things I'd do to try and get a better grasp of what's on the pool now:

Thanks for your reply!

  • The folders I had set to 3x are still 3x....the contents are only a few GB, so it's not something that would cause many TB of difference by getting reset.
  • Recheck/remeasure results in the same calculations.
  • Quick math suggests that once duplicated it will be back to the size (or very close to it) that it was before, so I think I may be okay there.

Yes, real-time duplication is and has been enabled.  That's a good point about moving so many files so quickly possibly messing something up -- in hindsight I shouldn't have used a folder in the pool for 'staging' all of those files (definitely wasn't necessary), and should have just let the application copy them into the pool as they're processed.

Share this post


Link to post
Share on other sites
  • 0
1 hour ago, ars3nic said:

Yes, real-time duplication is and has been enabled.  That's a good point about moving so many files so quickly possibly messing something up -- in hindsight I shouldn't have used a folder in the pool for 'staging' all of those files (definitely wasn't necessary), and should have just let the application copy them into the pool as they're processed.

Yep - I usually stage files to be delivered to the pool on a separate volume, or in a temp folder.  But glad to hear you're in the green!

Share this post


Link to post
Share on other sites
  • 0

The duplication information is stored on the pool, But it should handle disk issues, like this.

That said, if you could, run the StableBit Troubleshooter on the system in question, and use 3830 as the Contact ID request.
http://wiki.covecube.com/StableBit_Troubleshooter

Unfortunately, I don't think that the logs will have caught the error, but it's worth trying still. 

Share this post


Link to post
Share on other sites
  • 0
2 hours ago, Christopher (Drashna) said:

The duplication information is stored on the pool, But it should handle disk issues, like this.

That said, if you could, run the StableBit Troubleshooter on the system in question, and use 3830 as the Contact ID request.
http://wiki.covecube.com/StableBit_Troubleshooter

Unfortunately, I don't think that the logs will have caught the error, but it's worth trying still. 

Thanks Christopher!  It's submitting right now.  Pool is mostly rebuilt by now.

Share this post


Link to post
Share on other sites
  • 0

I do see that your "Linux ISO" folder was "unduplicated".  In that it found that the files were duplicated, and that the duplication should have been disabled for that folder. 

If that folder was supposed to be duplicated as well, then I'm not seeing a cause of the issue here.  

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

×
×
  • Create New...