ESXi Hosting of DrivePool (and Scanner)

[eduncan911]

I've very happily been running W2K12 and Stablebit Drivepool/Scanner for years.  I do run 3 to 5 Hyper-V VMs at times for various activities.

 

I'm upgrading, and I am also rebuilding those VMs (moving mostly to Linux).  But, Stablebit is 100% Windows.  So, I need to keep a Windows machine around.

 

I'd like to use ESXi hypervisor on this new build; but, I have a few advanced concerns about running Stablebit Drivepool/scanner in a VM and passing the disks through.

 

• HBA is an LSA9211-8i flashed to IT mode, passing SMART.  The SAS/SATA backplane is an LSI Expander, that scales up to the 24 drive bays I'll have.  I have verified SMART passes all the way through the Expander, through the HBA, to the OS (with the right drivers that is).
• I must have HDD spin down functional from the Windows VM w/StableBit DrivePool/Scanner.  Currently I just have it set in the Power Options and it works (with the hacked HBA driver).
• I plan on running up to 4 Nvidia/AMD GPUs off and on at times for various CUDA and number crunching (FAH, crypto/cracking, Tor, etc).  Plan on passing through these to a Linux OS of choice.
• Some of the VMs IO operations are Disk I/O Intensive.  They will have dedicated onboard PCH drives for those tasks, not connected to the HBA.
• Plex Media Server on the same VM as DrivePool (because of the files/library change detection)
  • Note, PMS will need as many vCPUs as possible and will get the highest priority CPU access over all other VMs.  Stablebit gets to come along with on the priority ride.
• One VM will be a dedicated NZBs VM (copies 10s of GBs per day over to the Windows VM w/DrivePool on it - most likely over a network IP (loopback I hope)).

 

Does Drivepool/Scanner live happily in an ESXi Windows VM with passthrough disks?  

 

Do I pass the HBA device through to the Windows VM, in order to get all the HDDs and SMART to show up?  DrivePool will be the only thing using the HBA device.  Or, do I pass each individual drive through, along with disabling that option I saw in ESXi to get SMART to pass through?

 

 

 

 

 

 

 

[Christopher (Drashna)]

StableBit DrivePool should have no issue with how the drives are passed through, as long as they're passed through. 

 

StableBit Scanner is a bit more finicky, about that.  You may need to do this:

http://community.covecube.com/index.php?/topic/131-how-to-get-smart-data-passed-on-from-esxi-51-host/

Also, the DirectIO Test tool is a good way to test if Scanner can get SMART data:

http://community.covecube.com/index.php?/topic/36-how-to-contribute-test-data/

 

 

Also, you may want to throttle SMART queries in StableBit Scanner to help idle the disks.

[eduncan911]

Thanks for those links!  Found the first one already and that is what convienced me that I can break from the Windows Hyper-V host and go ESXi.

 

Been really wanting to setup a home Cuda lab for years in Linux but Hyper-V doesn't pass GPUs through.  And the better-half would kill me if I was to run more than 1 server...  

[Christopher (Drashna)]

Well, HyperV does support RemoteFX, but that's ... a bit more complicated, IIRC.

[eduncan911]

Yeah, read up on RemoteFX... Naaaahhhhhh

[Christopher (Drashna)]

Yeah, read up on RemoteFX... Naaaahhhhhh

lol. Remote FX is rather simple to set up. The sticky part may be the licenses required for it (I'm not sure, actually).

Just install a compatible video card (most any anymore) and enable the GPU in HyperV settings.

[eduncan911]

Isn't RemoteFX vGPUs?  Not dedicated?  

[Christopher (Drashna)]

Yes, it is.

But I do believe it supports 2 physical GPUs. 

[eduncan911]

I have 4 Titans to install (at some point, when I am done gaming on a few of them)...  

 

 

EDIT: Actually a mix of AMD 280x and Titans to start.  I have some Tor projects I want to do some speed comparisons with (280x was far more efficient in the past over my .NET CUDA code, but with recent Golang CUDA apis I want to give it another shot).

[Christopher (Drashna)]

Ah, okay, the ESX would probably be the better choice (though, I think Remote FX supports more than that... that's just what' Microsoft's tested)

[otispresley]

I think you have to be careful whichever way you choose.  With RemoteFX, the default license is only 120 days, last I checked, and then you have to purchase a license.  With ESXi, if you want real 3D support, then you need to do DirectPath I/O (PCI Pass-through), and not all hardware works correctly when you do that.  Definitely do some research and make sure that others have had success with your graphics hardware; you may end up opting for different hardware to make it work.  Getting 3D graphics support in a VM is still pretty difficult at this point in time without the right hardware.  XenServer supports sharing of a graphics card between multiple VM's, but only with very specific hardware.

 

XenServer management tools are not great or fully featured when you want to do things like NIC bonding.  Hyper-V remote management can be a pain to get set up but is fine afterward; once again, you have to go to the CLI for NIC bonding and updating firewall rules (at least initially).  You are able to perform most all management tasks for ESXi from the management GUI.

 

There is also KVM, which is a kernel hypervisor (type 2).  With this, your VMs use your actual hardware rather than virtualizing it.  There are both benefits and drawbacks to this approach, but KVM has become very popular and is the default hypervisor for OpenStack.

 

Honestly, you may have better luck with 3D graphics in desktop software, like VMware Workstation or VirtualBox.

[Christopher (Drashna)]

Hyper-V remote management can be a pain to get set up but is fine afterward; once again, you have to go to the CLI for NIC bonding and updating firewall rules (at least initially).

Well, at least, if you're not domain joining the HyperV system (and have GPO's in place to enable remote management already).

For me, all I had to do was join it to the domain, and install the RSAT tools on my client.

 

 

 

With RemoteFX, the default license is only 120 days, last I checked, and then you have to purchase a license.  .

It's not RemoteFX specifically, but the Terminal Services (remote desktop services) licensing that is the issue. And yeah, you need CALs after that. Though I think it's 180 days for the trial/eval period, but I'm not sure.