vSphere and Multipathing iSCSI

This is just a quick reference to create a multiplathing iSCSI setup…

Create two virtual kernel switches, one called “iSCSI-1” and the other called “iSCSI-2” (and so on if you have more nics)

Then per kernel portgroup ensure that only one of the nics is active. For the “iSCSI-1” portgroup configure it to override the virtual switch settings and move nic 0 to active and nic 1 to unused. For the “iSCSI-2” portgroup configure it to override the virtual switch settings and move nic 1 to active and nic 0 to unused.

Now you have to run some esxcli commands to gel things together…. alt-f1 on the esxi console and type “unsupported” followed by your root password. The following is the command i have to run to get both my portgroups to work together on the iscsi hba….  (you can check your vmk number from the networking config screen)

esxcli swiscsi nic add -n vmk1 -d vmhba33
esxcli swiscsi nic add -n vmk2 -d vmhba33

For each target then change the path selection method to “round-robin”.

Go back to storage adapters and click “rescan”

If you want all future iSCSI targets to automatically use round-robin you must also run the following from commandline…   (this is for our HP Lefthand, your “storage array type” may be different. Its listed under your target details). Basically sets round robin as a default for this type of array. In general you should do this first before presenting any LUNs etc, else you may have to bounce your box.

esxcli nmp satp setdefaultpsp –satp VMW_SATP_DEFAULT_AA –psp VMW_PSP_RR

Update : Hardware iSCSI nics

The process for hardware iSCSI initiators is similar to above, but you assign a single kernel port per nic. To find which nic belongs to which iSCSI initiator you must run this command from the CLI;

esxcli swiscsi vmnic list -d vmhba#

vmhba# is the name of the iSCSI adapter.

create usb install for esx / esxi

First format your usb / flash drive with FAT. Then copy all the contents of the installer .iso onto the drive.

Next delete isolinux.bin and rename isolinux.cfg to syslinux.cfg

Edit syslinux.cfg and append “usb” to the line starting with “append” … i.e. similar to this

append vmkboot.gz — vmkernel.gz — sys.vgz — cim.vgz — ienviron.tgz — image.tgz — install.tgz usb

Next grab the latest syslinux.exe (zip for windows) from here and run syslinux -s –ma <driveletter>:

Done.

boot esx from usb drive

Its quite easy to boot esx from a usb device or flash card..

First download the .iso, then browse with winrar or similar. See if you can find the “big”.dd file this is the file we need.

i.e. the ESXi 4 file is called VMware-VMvisor-big-208167-x86_64.dd

Next grab WinImage and “restore image to physical harddrive” choose the .dd file the select your usb / flash drive.

Done.

vMA – Using HP Power Protector agent to shutdown virtual hosts

Goal was to create a vMA which would send a shutdown signal to all the virtual hosts it knows about. I’ve assumed that ESX and management agents that live within the service console will eventually be phased out.

The steps…

  1. Download and install the vMA from vmware – fire up the vm and setup the basic networking required
  2. Create trusts within the vMA to each of the vmware hosts you wish to manage (sudo vifp addserver <server>)
  3. Install HP Power Protector (linux agent) – you could substitute for your UPS client software
  4. Modify the shutdown script (may differ per vendor) to shutdown the esxboxs- “SDScript” (hostops.pl)

The 1st step is easy enough. Find the download at vmware and install (Deploy OVF Template…)

Once up and running you need to create the trust between the vMA and your ESX hosts. Logon using vi-admin and your password. then;

sudo vifp addserver esxbox1 (you will be prompted for the hosts root password)

Do this for each box you wish the vMA to manage.

Once you have installed your UPS agent into the vMA (linux client should work without issue) the next step is to modify the shutdown script to do the work. Within the script you will need the following in it…

vifpinit esxbox1

This sets the context of vMA to this host. Note: the server should have been added as one of the managed

/usr/lib/vmware-vcli/apps/host/hostops.pl –target_host esxbox1 –operation enter_maintenance –url https:///sdk/vimService.wsdl

Note: you need to use the actual name of the host and not its IP. You can get the exact name of the

The HP power protector agent script is located at /usr/local/DevMan/SDScript

If you want to ensure all your guests shutdown cleanly, enable “shutdown virtual machines on host shutdown”. Also note that if you shutdown the host that the vMA is running on it will kill the script. So shutdown the host that the vMA is running on last (remove vm from DRS)

Update (08/04/2010): I have found the above vMA to be quite fiddly. And have had much better luck with the PowerCLI code found on this page : http://www.virtu-al.net/2010/01/06/powercli-shutdown-your-virtual-infrastructure/

I have made some slight mods but essentaillly…

$VCENTER = "vcenter"
Connect-VIServer $VCENTER

# Get All the ESX Hosts
$ESXSRV = Get-VMHost

# For each of the VMs on the ESX hosts (excluding virtual center box)
Foreach ($VM in ($ESXSRV | Get-VM)){
    # Shutdown the guest cleanly
    if ($VM -match $VCENTER){}
    else {$VM | Shutdown-VMGuest -Confirm:$false}
}

# Set the amount of time to wait before assuming the remaining powered on guests are stuck

$WaitTime = 120 #Seconds

do {
    # Wait for the VMs to be Shutdown cleanly
    sleep 1.0
    $WaitTime = $WaitTime - 1
    $numvms = ($ESXSRV | Get-VM | Where { $_.PowerState -eq "poweredOn" }).Count
    Write "Waiting for shutdown of $numvms VMs or $WaitTime seconds"
   
    } until ((@($ESXSRV | Get-VM | Where { $_.PowerState -eq "poweredOn" }).Count) -eq 0 -or $WaitTime -eq 0)

# Shutdown the ESX Hosts - and remaining virtual center box (if virtual)
$ESXSRV | Foreach {Get-View $_.ID} | Foreach {$_.ShutdownHost_Task($TRUE)}

Write-Host "Shutdown Complete"

# If virtual center box is physical and still alive it will need to be shutdown...

Write-Host "Shutting down virtual center"
shutdown -s -f -t 1