Jump to content
  • 0

Best Practice for manually moving files around on drives within a Pool


pclausen

Question

I was wondering what the best practices are for moving files around between drives within a pool.  I have a number of drives that are giving me SMART warnings, and I want to move the contents of those drives onto other healthy drives on the same pool that have plenty of available space.

 

Is this the recommended procedure:

 

1. Stop DrivePool Service

2. Move the content from within the DrivePool hidden folder from the source disk to the target disk as seen in the below screenshot:

 

diskcopy1.JPG

 

3. Start DrivePool Service

4. Remove empty drive(s) from DrivePool via GUI

 

Thanks!

 

Peter

Link to comment
Share on other sites

19 answers to this question

Recommended Posts

  • 0

Thanks!  I neglected to mention that I'm also running SnapRaid, so I disabled the Scanner plug-in.  Perhaps I can run all 3 together?  I'll have to look into that.  Definitely would be cool to automate moving data off disks that are failing.

Link to comment
Share on other sites

  • 0

Hi scanner does just that if the settings are

Correct upon a falling drive it will start to evacuate data providing you have enough space on other drive, good practice would be to leave enough free space to cover the largest drive you have installed. I have mine set to only remove non duplicated data this to me is the quickest and best option then drivepool will start to reduplicate the files once you remove the failing drive.

Link to comment
Share on other sites

  • 0

Thanks!  I neglected to mention that I'm also running SnapRaid, so I disabled the Scanner plug-in.  Perhaps I can run all 3 together?  I'll have to look into that.  Definitely would be cool to automate moving data off disks that are failing.

 

I have Flexraid  - DP and Scanner all working in harmonie, so i dont know why SnapRaid would be a problem

Link to comment
Share on other sites

  • 0

I was wondering what the best practices are for moving files around between drives within a pool.  I have a number of drives that are giving me SMART warnings, and I want to move the contents of those drives onto other healthy drives on the same pool that have plenty of available space.

 

Is this the recommended procedure:

 

1. Stop DrivePool Service

2. Move the content from within the DrivePool hidden folder from the source disk to the target disk as seen in the below screenshot:

3. Start DrivePool Service

4. Remove empty drive(s) from DrivePool via GUI

 

Thanks!

 

Peter

No, this is not.

 

There are two recommended ways to do this.

Both methods use the balancer feature.

 

  1. Find the StableBit Scanner balancer (Pool Options -> balancing -> balancers tab).  There is are two options for SMART data. "move unduplicated data off of drives with SMART Errors" and "Move duplicated data off of drives with SMART Errors". 

    Check one or both of these options. This will cause the pool to move the data off of ANY disk with any SMART errors (this includes LCC or temperature errors, so that's why it's off by default).

  2. Find the "Disk Usage Limiter 'balancer. Uncheck the "duplicated" and/or "unduplicated" options for the disks in question. This will automatically balance the data off of the disk.

 

 

Hi scanner does just that if the settings are

Correct upon a falling drive it will start to evacuate data providing you have enough space on other drive, good practice would be to leave enough free space to cover the largest drive you have installed. I have mine set to only remove non duplicated data this to me is the quickest and best option then drivepool will start to reduplicate the files once you remove the failing drive.

This is good advice as well. That way, if you are having issues with a disk, the unduplicated data, which is "vulnerable" from a disk failure will be moved out of problem disks.

 

Also, this way, you can remote the disk and use the "duplicate data later" option, that way you can remove it immediately, after the unduplicated has been moved off.

Link to comment
Share on other sites

  • 0

Appreciate the feedback.  Since I have 10 disks with SMART errors, most of which are rather innocent (spin retry count), I didn't want to automatically move data off all those disks to other disks.  That said, the disks with warnings are all Seagate 1TB Enterprise drives that are, on average, over 5 years old.  Some of them are coming up on their 7 year anniversary.

 

So what I did was to remove the ones with severe errors (reallocated sector counts in the 1000s) from the pool by simply clicking the "Remove" button from the gui.  This worked beautifully and automatically spread the content to a few of the other disks in the pool with the most free space.  Keep in mind that I got 44 drives in my pool, a mix of 1, 2 and 4TB drives with a total of 71TB with 24TB free.

 

So for the drives with the spin retry count SMART errors, I chose to "Ignore Warnings" until new ones show up.  So with that done, if I was to enable one of the 2 options you mention above, would DrivePool begin moving data off those drives immediately, or would it wait until an additional SMART error occurred?

 

Thanks!

 

Peter

Link to comment
Share on other sites

  • 0

So for the drives with the spin retry count SMART errors, I chose to "Ignore Warnings" until new ones show up.  So with that done, if I was to enable one of the 2 options you mention above, would DrivePool begin moving data off those drives immediately, or would it wait until an additional SMART error occurred?

If you ignore the warning in StableBit Scanner, then DrivePool won't move the data off. It is specifically pulling the SMART status from Scanner.

 

So if the drive shows up as "Healthy", it won't move data off. But as soon as it shows "SMART Warnings" (or similar), it would start moving the data off... according the balancing settings

 

So "ignoring" the settings until the a value increases is pretty much what you want, along with the SMART Warnings in the balancer settings.

 

 

As for the SMART errors... 

Reallocated Sector Count is a fairly bad one. It may not be critical, unless it's steadily growing, as this possibly denotes physical issues with the disk. But a lot of sectors can and most likely WILL degree performance of the drive, as the sectors have been remapped, meaning the read head takes longer when it hits these "patches".

Spin Retry Count is a really bad one. As in, RMA or replace the drive, soon. It means that the disk was unable to properly spin up the disk to speed. Which means that there is definitely a mechanical issue with the disk, and ... could eventually cause the spindle to seize up, rendering the drive ... into a paperweight.

 

Depending on what other SMART errors you're seeing ... well, let me know and I'll let you know how severe they may be.

 

 

 

 

 

Also, you should definitely check to see which of your drives are in warranty. This is good information, and we provide at least an indicator. Additionally, the "Disk Details" lists the standard manufacturer warranty period. However, this period is usually from the manufacture date, not the sale date.

Link to comment
Share on other sites

  • 0

Appreciate the response.  As stated, I got a total of 48 drives (4 are 4TB parity drives and then 44 data disks 1, 2 and 4TB in size).  All the ones with errors are my original 1TB Seagate ST31000340NS Barracuda ES.2 Enterprise drives.  I purchased 24 of these originally and over the years I was able to replace about 10 of them under warranty.  The replacement units only came with a 1 year warranty unfortunately.  I still have 19 of these running in my pool  Here are the SMART errors on them:

 

smarterrorsummary.JPG

 

As you can see, only 1 of the 19 has no SMART errors.  I removed the one from the pool with over 2000 reallocated sectors as indicated above.

 

I do have enough space left in the pool to migrate the data off the other 18.  Some of them are actually mostly empty right now.  So do you think I should just go ahead and do that now, or do you think they may have another 6 years left in them? :)

 

Btw, my drives never spin down to idle since I moved to my current HBA controllers, and it has probably been 6 months since the server was last powered down.  Plenty of reboots, but no power downs, so I don't think the 18 drives with spin retry counts have had to try to spin up for 6 months.

Link to comment
Share on other sites

  • 0

I think I'm going to go ahead and remove most of those other drives with the SMART errors from the pool.

 

Based on my experience with removing one so far, I have a concern.  Even with all balancing turned off, as soon as I remove the first drive from the pool via the GUI, DrivePool is going to begin moving the data on it to the other drives in the pool with the most free space.  This is normally a good thing, but in my case, some of those are disks that I also want to remove from the pool.  So how to I prevent data being moved to any of those drives?

Link to comment
Share on other sites

  • 0

Install one or a few 8TB Seagate Archive HDDs? Cheap per TB. If you'd consider that then I advise to google a bit (storagereview.com and Christpher and I have written a bit on our experience on this forum). You would then remove large HDDs, the 4TB ones, first so that you can write 6TB to those 8TB drives before data would be allocated to the older 2TB drives.

 

I *think* there is also a balancer plugin through which you can cap the amount of data written to individual HDDs. If you'd put them at current usage then data to be moved should be moved to new HDDs (provided they are there and have space).

Link to comment
Share on other sites

  • 0

I'll look into those 8TB archive HDs and the balancer plugin that lets you cap per individual HDDs.    I went a different router morning I went ahead and "un-ignored" the SMART warning messages, then I enabled the Scanner Plugin, and allowed immediate balancing.  Well, nothing happened after like a hour.  I then told scanner to start scanning the drives, and it just hung.  So I decided to do a reboot.  The scanner service never started.  Tried restarting the scanner service.  No error, but when I started the scanner UI, it told me the service wasn't running and recommended doing a reboot.  I went through this dance 3 times and then finally decided to go back to Windows 8.1 64 bit rather than do a rollback.  I'm a big fan of clean installs.

 

So I backed up the drivepool, scanner and snapraid dirs and various other things.  I then proceeded with a clean install, wiping the C drive completely.  It has a huge pain just to get started since I had 50 drives connected and the installer had to scan them all before I even was able to tell the installer where to install.

 

Anyway, I'm just about done applying all the patches and also got all 48 drives mapped to folders matching exactly what I had before (c:\raid\data01, etc).  I forgot to deactivate my drivepool and scanner licenses, so we'll see how that goes what I do fresh installs of those to apps.  So after doing clean installs of those apps, should I copy any config files over from my backups, or am I better off just setting everything up fresh via the GUI?

 

If I setup everything fresh, will the current drivepool hidden folders in which all my data resides still work?  (see screen cap in first post for an example)

 

I'm thinking that before installing drivepool, I might just manually move the folder/files off those failing 1TB drives.  This way there won't be any balancing for drivepool to do for the time being.

 

Thanks

 

Peter

Link to comment
Share on other sites

  • 0

I *think* there is also a balancer plugin through which you can cap the amount of data written to individual HDDs. If you'd put them at current usage then data to be moved should be moved to new HDDs (provided they are there and have space).

Yes, in fact. The "Disk Usage Limiter" can do this. Open the settings for it, and you can uncheck both "duplicated" and "unduplicated" for the disks in question.

 

This will not only start moving data off of these disks (which is great for prepping for removal), but it will actively PREVENT new files from being moved to the disks (depending on the exact balancer settings).

 

 

As for Scanner, could you do this:

http://wiki.covecube.com/StableBit_Scanner_for_Windows_Error_Reports

But grab the entire folder, and not just the "Error Reports" folder?

 

After doing that, resetting the settings may fix the issue with Scanner (and with the balancing).

http://wiki.covecube.com/StableBit_Scanner_Q4200749

 

 

As for rescanning... you can mark the drives as "good". Click on the "+" next to a disk (or double click on it). There should be four buttons on the left side. Click on the forth one, and select the "Mark all unchecked as good" (or to that affect). 

But yes, this can be tedious for 40 something drives.

 

 

 

And yeah, I have two of the Seagate 8TB drives now, and plan on picking up more as the budget allows. They're fantastic, and the speed I get on them is just amazing as well. (not SSD good, but better than the NAS drives I have).

Link to comment
Share on other sites

  • 0

I'm now using the "Disk Usage Limiter" plugin and it is working great. Found one little issue with it that does not appear to affect functionality, but just looks a bit odd in the UI.

 

I'm removing 3 drives from my pool. So I went into Disk Usage Limiter and unchecked both boxes for the 3 drives. I'm not in the process of removing the 2nd disk (the first one has completed). If you look at the below screen cap, you can see that I have both boxes unchecked for both of the remaining drives. But for some reason, that upper little triangle did not move all the way to the left in a UI as indicated by the green circle below for the last drive. It did move both triangles all the way to the left for both the 1st and 2nd (as seen by the red circle below) disks.

 

drivepoolbalance.PNG

 

It doesn't affect anything function wise as far as I can tell, so perhaps it is just a cosmetic bug. Minimizing the window and restoring it doesn't change anything.

Link to comment
Share on other sites

  • 0

It's not a cosmetic bug, but we should explain it better....

 

See the "Try not to fill a drive above X" slider? 

Set that to 95%. That should fix what you're seeing. 

 

 

As for hte arrows, the blue is the "Target for balancing" and the red arrow is the real time placement limiter (files can only be placed on the drive up to that limit). 

Link to comment
Share on other sites

  • 0

Thanks.  That explains it.  So I was successful in removing all but 22 data drives from my pool, consolidate those 22 data drives along with 2 parity drives into a single 24 bay chassis.  Next I removed a couple of 2TB drives, one at a time, and replaced them with 4TB drives.  Next I went into Disk Manager and relabeled the drives and also changed their mount points to match.  DrivePool picked up on the changes within 20 seconds or so, but Scanner still hasn't several hours later.  I stopped Scanner and restarted, but it still didn't pick up on the change.

 

Here's a screenshot showing the correct labels and paths in DrivePool and the old ones in Scanner:

 

scannernorefresh.PNG

 

Also, I hot swapped the drives and a couple of times Scanner crashed when doing this.  I submitted an on-line error report the 2nd time.

 

Once the balancing completes, I'll do a full reboot and hopefully Scanner will pick up the new disk IDs.

Link to comment
Share on other sites

  • 0

Restarting the service should cause scanner to forcibly update the information.

 

Did you remove the old mount points?
If not, then you may want to do that.

 

As for the crashing part, could you grab the error reports from the system?

http://wiki.covecube.com/StableBit_Scanner_for_Windows_Error_Reports

 

 

If rebooting the system doesn't work, then please try the beta build:

http://dl.covecube.com/ScannerWindows/beta/download/StableBit.Scanner_2.5.2.3095_BETA.exe

Link to comment
Share on other sites

  • 0

Restarting the service did do the trick.  I misspoke before when I said I restarted the service as I actually just clicked the X in the upper right hand corner and then launched the app again.  I now realize that in essence just minimized the app

 

I sent the error reports as requested.

 

Thanks

Link to comment
Share on other sites

  • 0

Yup, basically.

 

The UI just provides a user interface. Everything is done by the service, itself.

 

However, that it's not updating properly is an issue. If you see this happening again, before restarting the service, get a memory dump of it:

http://wiki.covecube.com/StableBit_Scanner_Freeze

And maybe one afterwards to compare it.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...