Jump to content
  • 0

Migrating pool to new machine: sanity check


Edward

Question

Hi

 

Appreciate if someone could give me feedback that the process I'm about to follow will work ok.

 

I'm currently moving my server duties from a machine running WHS11 to a new machine running win10 (I don't need domain controllers, client backups etc etc yet).

 

Old WHS11 machine has 4 drives all drivepooled and mapped as P:  Covecube scanner also running.

 

steps to take:

 

1. Install stable releases of drivepool and scanner on W10. Enable 30 day licences.

2. On WHS11 machine deactivate licences for Drivepool and Scanner

3. Transplant 4 drives to new W10 machine.

4. Add the 4 drives to drivepool.

5. Somehow, automagically, drivepool will recognise the old drivepool settings from the WHS11 machine and recreate same drivepool (as P:) ???

6. Check for consistencies/anomalies, correct as required.

7. Run scanner adding drives as required. Not sure if scanner will have to recheck all drives (currently being reported as healthy under WHS11). I assume it will have to.

8. Enable licences etc.

9. Be happy!

 

Is that it?

 

As a follow on I have installed Hyper-V within the W10 setup. That is working just fine. Question: Will the various VMs I create be able to see the pool as one large pool as well as read/write/delete?

 

many thanks

Edward

Link to comment
Share on other sites

16 answers to this question

Recommended Posts

  • 0

I just did the same thing (completely different hardware; but, the same steps).

 

Yes.  It will find and have the actual full Drivepool all setup and active.  As long as the drives are "Online": using the built-in Disk Management application, you can check that they are online.

 

I recommend these steps though: Get the hardware setup first before installing the software.

 

- Deactive license, shut down old machine.

- Install HDDs in new machine, do NOT install DrivePool!!!  Hold off installing any software.

- Go to Disk Management and verify the drives are all "Online."  If they are not, right click on them and select "Online."

- Then install DrivePool, DriveScanner, etc.

 

These are the steps I took and it worked perfectly.

 

The reason why I suggestion this is you can only bring a single HDD "online" at a time.  Even though I didn't try it; I would not want DrivePool complaining that I only have 1 HDD out of my pool.  It may go wacky or whatever.

 

So, I decided to get the hardware setup first before installing the software.

 

Note: DrivePool will try go through a bit "Checking" phase.  After it is done, it will start to Balance with the DEFAULT settings.  If you are like me, you may have very custom settings setup in your Balancing.  Get that done first and don't let it start balancing.  The initial checking that happens after installing DrivePool took several hours; so, it gave me plenty of time to get things "Balanced" correctly according to my settings.

 

Scanner will run with defaults.  Mine was set to 3 AM - 6 Am for the work period (I was up until 4 AM working on mine, and wondered why throughput lowered dramatically - it was Scanner that started at 3 AM).

 

As far as virtual machines and DrivePool/Scanner, be warned that SMART access does not pass through Hyper-V to the VMs.  I originally tried to get DrivePool/Scanner all setup on VMs under QEMU (a good native hypervisor under Linux) and W2K12R2 using Hyper-V.  But times I ran into SMART data not being passed, which ticked me off.  It's a Windows/Linux thing.

 

So, I advise setting up the DrivePool / Scanner on the HOST.  If that's Windows 10, that's fine.  That will give you a single Drive Pool drive.  I name mine "A:\", as a throwback to the floppy disk days (if you don't know Floppy Disks, then enjoy your youth!).

 

Once you have that setup, I advise setting up a "Virtual Network Adapter" in Hyper-V for "Internal" only <- if that's possible under Windows 10 (I am using Windows Server 2012 R2, which allows me to do "Internal" network adapters).  Internal creates a private 10 Gbps network that all VMs can use, as well as talking to the host - you ahve to setup the proper IPs and Mask.  The advantage is there is no physical "NIC' used, so you get some great speed without interfering with remote access to the physical machine.

 

With that, setup a "Share" only for your internal private network (mine is 10.16.210.X).  The share will be for your Drivepool drive, open to all anonymous access... And there inlies yet another issue with Windows, true "anonymous" access to a shares is very difficult with so many hoops to jump through at many different security levels.  

 

Anyways, use a Windows Share from your Windows 10 box to your "internal" networking for all of your VMs you have running.

Link to comment
Share on other sites

  • 0

Wow eduncan911, what a comprehensive and well thought out reply.  Really appreciate that. :)

 

I need to go through your various points and if ok with you come back later on some of the more subsidiary points.

 

Good idea of installing hardware first and then software for the reason you mention.  I had already installed Drivepool and Scanner, however I will uninstall and then work from there.

 

Re A: drive.  That made me smile.  I started life off with 5 1/4 inch floppies and MSDos 2.xx. Upgrading to 3.xx was a massive step! I use to access the internet long before html.  God this ages me! 

 

Reading around the Covecube forums did reveal that running drivepool and/or scanner in VMs could be problematic so, yes, I will run them on the host. In fact due to the possible VM problems of running drivepool and scanner in a VM i discarded the idea of running exsi as the bare metal host.

 

I need to think through your idea of only running an 'internal' nic in the VM setup and then 'sharing' the drivepool. Not sure what the advantages of that are beyond the 10gb apparent connection you mention. But even then I'm confused as to how such a bump in speed would come about given that, internally on the one machine the limit is to the sata data bandwidth and externally (one local machine to another) limited to a gigabit connection (am running cat6 ethernet here). But yes hyper-v allows three virtual NIC settings - external, internal and private.

 

thx again for comprehensive response.

 

E

Link to comment
Share on other sites

  • 0

Using an Internal virtual network adapter offloads the network traffic from your External-physical NICs.  

 

I previously ran into issues streaming an MKV (say, from drive 11) and my NZB process (running on another machine) copying a recent download directly to the pool over the network (to, say, disk 21).  They were different disks, but, the same NIC.  I got stuttering in the movies...

 

New setup uses the two onboard Intel i210 NICs in a Team-1 for "common" network traffic.  I purposely installed an Intel i350-T4 NIC, with all four NICs setup in a single Team-2 dedicated solely to "Remote Streaming Access."  IOW, if i want to copy a bunch of movies/TV shows off the server, I go through the IP address I ahve assigned to the Team-2.

 

The problem now is the bottleneck of my cheap switches throughout the house - often times there's only 1 path for data to flow.  

 

But, i have assured that each WiFi Access Point (the vast majority of streaming is purely over WiFi in my household) has a dedicated path to the server.  That's 4 WiFi routers connected to each port (with a switch inbetween as needed).  Zero issues now streaming in my household (up to 7 devices I've seen streaming from the server at once).

 

Well, my desktop uses a NIC connection... I cheat there.  :)

Link to comment
Share on other sites

  • 0

Actually, I would recommend against using A:\ or B:\.  We've found that Windows will access these drive letters and cause activity on the pool, waking up the drives.  It's not something that can be worked around apparently, as it's some hard coded behavior deep in Windows (it expects floppy drives for these letters). 

 

 

 

As for virtualization:  HyperV does not pass on SMART data for any passed through disks.  This is incredibly disappointing, but is entirely a driver/hypervisor issue.  ESX can, but only if pass along the entire controller, IIRC. 

 

 

As for performance, if you're passing through disks, there will be some performance hit, but it should be barely noticeable (if at all). 

 

 

 

As for working: Gigabit limits you to 125MB/s.  10gig limits to 1.25GB/s.  

However, depending on the disks in use, you may not even hit the limits of the gigabit network. 

Link to comment
Share on other sites

  • 0

@eduncan911.  Thanks for that.  NIC teaming is way above my experience level, but for sure I'm keen to learn (for all the apparent speed advances and flexibility).  For now however I think will get my new server (Lenovo TS140) up and running with the 4 drives and then move forward from there. As recommended I will first uninstall covecube stuff and install once all drives are in place.  But a quick question if I may? You say you are connecting/streaming only via wifi but yet you have a 4 port ethernet NIC on your host. Don't understand. How does that all work?

 

@Chris (Drashna). I'm hoping that I will simply stick with P: as my main pool and it will automagically come to that once I adapt the already pooled discs. Having SMART data only at the host level will be fine. After all the whole purpose of scanner is risk mitigation only so it being reported only at host will be sufficient. However I'm hoping that the drivepool will be visible at the VM levels together with ability to read/write directly to the pool from a VM. Recent testing on my gb network was showing sustained throughput of about 105 mb/s which is not too shabby (using ssd to ssd as end points).  However this is quite variable in real life situations (mix of machines, slow drives, poor cat6 termination etc).

 

E

Link to comment
Share on other sites

  • 0

The Intel i350-T4 NIC has 4 ports on it.  Each port is connected to a dedicated switch (and each switch has a connection to itself).  The switches are cheap built-in router switches and a few 5 port and 8 port cheap Linksys/belkin switches.  All are Gigabit switches.

 

Each Wifi router I have either acts as those dedicated switches, or is connected directly to one of the other switches.  This ensures the lowest number of routes from the NIC, through a switch, to the Wifi router.

 

@Chris (Drashna): Humm, no drive A:\ eh?  That may explain the odd "why are the drives spun up?" issues I've had with my previous install.  Holy ****.

 

Well, as I haven't completed my setup, I guess I can still change it in time... 

Link to comment
Share on other sites

  • 0

@Chris (Drashna): Humm, no drive A:\ eh?  That may explain the odd "why are the drives spun up?" issues I've had with my previous install.  Holy ****.

 

Well, as I haven't completed my setup, I guess I can still change it in time... 

Yeah, absolutely.

 

If it makes you feel better, we didn't actually discover that until somebody had a similar "waking up all the time" issue with their drive. Alex actually eventually tracked it down.   But some low level code expects A:\ and B:\ to be floppies, and as such, constantly queries the drives to see if there is actually a disk in them. 

 

It's stupid, and a PITA, if you're not expecting it.  

 

What does work better is using a letter much farther in the alphabet (S:\ is my preference, for "storage" :) ), and then mounting all the pooled drives to folder paths. That way, there is only the system, the pool, and maybe other "special purpose" drives listed in the computer section.

Link to comment
Share on other sites

  • 0

Just a quick update from my side. Good and annoying news.

 

Transplanted 4 storage drives from old machine to new machine, ensured all drives online, reinstalled drivepool and old pool seen immediately.  Drivepool went away measuring things.  New pool shown as H: but, in Disk Management, changed that to P: 

 

However on opening folders within the pool I can't open the various folders, get error of "You don't currently have permission to access this folder. Click continue to permanently get access to this folder."

 

Clicking 'continue, simply results in a long wait with nothing apparently going on. Slowly folder permissions coming through. Tried advanced security settings and found I'm not listed as having permission (even though the account is an Administrator). Added my account and that enumerated the folder tree.  However still had to claim permissions when navigating the folder tree. All a massive time waste!  Claiming permissions via the Pool also has set up permissions on underlying data disks (D: through G:).

 

Scanner installed just fine.  Now all q'ed for full scan.  Currently undergoing 'temperature equalization' - whatever that means.

 

E

Link to comment
Share on other sites

  • 0

As for the permissions entries, depending on what exactly happened, that may be normal.   If you used user accounts for the permission entries and reinstalled or switched hardware (which is what you've indicated has happened), that would cause the issue you're seeing, as the new accounts will not be the same as the old accounts (even if they're using the same name and password).

 

Resetting the permissions on the pool may be a good idea (eg, take ownership of the root folder, and "replace ownership on child items", and change the permission on the root folder, and again "replace ... child entries"). 

 

 

 

As for StableBit Scanner, by default, the software will stop scanning a drive if it becomes warmer than the other drives. This is to prevent the creation of "hot spots" in your system. 

YOu can disable or change this option in the "Heat" section of the Scanner settings (click on "Settings" in the toolbar, click on "Scanner settings", and open the "Heat" tab).

http://stablebit.com/Support/Scanner/2.X/Manual?Section=Heat

Link to comment
Share on other sites

  • 0

Thanks Chris.  Permissions now seem all ok. Just wish there was an easier way of doing this.

 

Pool organization has now settled down. All data now reported as duplicated with no wastage. Pleasantly surprised how fast that took.

 

Scanner is now starting scanning disks. OS disk on an SSD reported as healthy. First drive in q already completed at ~25% on a 1.82tb drive.  Interestingly scanner is incorrectly reporting the name on this drive.  Actual drive is a Samsung but is being reported as a Crucial SSD.  (I wish I had a 1.82tb SSD drive!!). Is there anything I can do to force Scanner to correctly report name?

 

Now need to propagate to my various client machines the new server details (mainly JRiver media server, but also remote desktops). Also now need to re-adapt my crashplan cloud backup (always fearful of that in case I do something worng and delete my cloud backup which will take weeks to upload again).

 

E

Link to comment
Share on other sites

  • 0

As for the StableBit Scanner issue, I'm not entirely sure why it would do that, But on the "SMART details" page of one of the drives (doesn't matter which), check the "submit to bitflock" option. Give it a few minutes and post the ID it generates here.

 

Aside from that, install the internal beta build, as it may fix the name detection issue.

WHS2011: http://dl.covecube.com/ScannerWhs2/beta/download/StableBit.Scanner_2.5.2.3116_BETA.wssx

Windows:   http://dl.covecube.com/ScannerWindows/beta/download/StableBit.Scanner_2.5.2.3116_BETA.exe

 

That, or right click on the disk, select "Disk Settings", and you can edit the name of the drive manually.

Link to comment
Share on other sites

  • 0

As for file permissions, I reset all permissions and force it onto all sub-files and folders like memory muscle.  It's just a default action one has to make when moving NTFS volumes from one machine to another.  And annoying to users that aren't familiar with the NTFS side of things.

 

A necessary evil - and I should have mentioned that in my previous posts.  Sorry about that.

 

Good to hear it went through.

Link to comment
Share on other sites

  • 0

As for file permissions, I reset all permissions and force it onto all sub-files and folders like memory muscle.  It's just a default action one has to make when moving NTFS volumes from one machine to another.  And annoying to users that aren't familiar with the NTFS side of things.

 

 

Yup, exactly this.  And like a normal NTFS volume, migrating StableBit DrivePool to a new system/install has the exact same issues. 

 

And that's why a lot of people recommend using the built in groups for permissions when possible, and only use created accounts when absolutely necessary.  That's because the built in groups use the same "SID" (Security IDentifier) between all installations.  Less to change, less to create issues.

Link to comment
Share on other sites

  • 0

I have a similar issue with a twist.  The posts in this thread were very helpful in troubleshooting to this point.  Thanks for that.


 


I have two NAS servers and an offline backup server.  All three are running Drivepool and Scanner.  The issue is with the backup server.  It has ten SATA drives running on a LSI SAS31061E 16 port card.  This setup has been running well for the past 18 months.  I start up the server; check to see if there are any pending OS updates; update is needed; do a partial backup that freshens changes files and adds any new ones; let the server run 24 hours to catch any other OS updates (Windows 10 has lots); shut the server down until the next time.


 


On the most recent backup cycle, Drivepool informed me that there was a missing disk.  I have had this with the two NAS server.  A couple of times I had SATA cables come loose.  I would re-seat or replace with a new cable.  Problem solved.  Drives have failed; I replaced with new drives; and Drivepool did the rest.  No problem.


 


Sorry in advance for this detail.  I have a spreadsheet with all my drives.  Each has a unique code that I put on the drive/drive tray.  The spreadsheet has the physical position of the drive; model number, serial number, date of manufacture, date put into service, date and reason for removal.  I also have this code in the volume label and disk description.  So, each drive is known in Windows Explorer, Drivepool, and Scanner.  Megaraid software does not allow label editing, however, with the list I can verify port easily.


 


So, when Drivepool informed me that a drive was missing, identifying the offending drive was easy.  I went through the routine, checked the cable; no good; swapped the drive to another port; no good.  In Megaraid, when I removed the drive it would show 9 drives.  Add the drive back in 10 showed.  Scanner always showed only 9.  Drivepool continued to say a drive is missing.  I think this was operator error on my part.  I Removed the drive in Drivepool.  Now it is gone, but the files are locked in read only mode.


 


This is where I need help.


 


Hardware wise, when I plug the drive in Megaraid shows 10 drives.  After a few rounds of this, I finally checked each drive in Megaraid and my list and found that the offending drive was showing up with a duplicate of another drive serial number.  OK, the drive must have a I/O board problem.  I disconnected all the drives except the offending one.  Restarted the server.  No drives.  I am stumped.  I can accept the drive is bad and move on.  I don’t know how to get Drivepool out of read only mode.


 


Now, why this thread.  The server I have been using has a couple issues that I am reasonably sure are not causing a problem with the drives or any of the drive software.  I have a replacement server ready to go.  Both servers are DL360 G5s (I know, antiques, but these work for me).  I can’t move to the replacement until I have a path to get there.  One option is to wipe the drive array and start over.  However, to copy the full set of files would take days.  I am hoping to avoid that.


 


I have a variation on the rebuild theme.  The drive card will run 16 drives.  So I could buy a couple of new drives leave these out of the pool and copy locked drivepool files drive to drive.  The total drivepool is 10TB, with 4.9TB filled.  Hopefully this could cutdown on the rebuild time.  Any suggestions would be appreciated.

Link to comment
Share on other sites

  • 0

Well, for the drives that keep dropping, is the same drive that keeps having issues or different drives every time?

 

If it is the same drive, try swapping ports/locations/whatever.  If the issue persists with the drive, replace it. It may be failing already, in a non-mechanical fashion.  It the port has issues ... replace the cable/backplane/etc.

 

 

If it's different drives, it could be your power supply (either not enough power, or unstable), or it could be the controller card .... or motherboard,  etc.   

 

 

 

StableBit Scanner's burst test may be able to help out with this, as it's a good stress tester.

 

As for the card itself, with a SAS Expander card, you can get a LOT more than just 16...  The card itself supports 256-512 drives, I believe.... so ... there is that.

 

As for moving things over, are the drives RAIDed together? Or passed through to the OS?  Or?

Link to comment
Share on other sites

  • 0

Thanks again.  Reading the original thread and posting my particular issue and your responses have helped on next steps.

 

Same drive - There is one drive that has failed. I tried different ports and power plugs.. The drive spins up and get warm (not hot).  Since the I/O board on the drive appears to have failed, I understand why Scanner would not pick up the problem.  I though about buying a replacement board to salvage the drive.  The cost is about the same a used drive.  The manufacture date on this drive is 1/7/2010.  So, a proper drilling and burial seams the best at this point.

 

Swapping - I did and could trace from Hardware drive to Megaraid, Scanner, DP, and Windows Explorer.  With the remaining 9 drives running, scanner and DP are OK.  W.E. still shows the drives in read only mode.  I was probably too hasty on removing the drive from the pool.  Maybe if I had recognized the failure mode of the drive, I could have replaced the I/O board and let DP remove the drive normally.  When I have done a removal in the past it took some time for DP to move the files around. I this case, as soon a I hit Remove, the drive was gone from DP.

 

Certainly makes the case for file duplication.  The SATA card is running JBOD.  DP does the pooling.  I made some major changes in the file structure, so will need to copy new backup files anyway.  Will grab a couple new drives to give me space.  I have a strategy to offline the machines so the house network still functions.  The regular refresh and add does not put much stress on the network and can be done online.

 

I appreciate to pointer to the Burst Test.  Before I start loading files, I will give the drives a good shaking.  Many of the drives are the same vintage as the one that died.

 

One more question - If I uninstall DP and Scanner on the new server, will the drives drop the read only mode so I can format? 

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...