Wan Acceleration Appliances

There are some really good products out there in the market at the moment…

Exinda – low cost, really good QOS / prioitisation and visibility. Falls short on acceleration (at the moment)
Riverbed – One of the key players in the acceleration market. Optimized algorithms for specific traffic.
Silver Peak – Relatively new, work well as a capture all acceleration device with QOS / prioritisation included.

One to watch – traffic squeezer. I’m always a fan of open source getting into new technologies, and this project seems to be a good start. Its still early days but I’m hopeful this will take off and become a contender in the future ahead.

A few of the above vendors actually have a virtual version of their appliance. Depending on your current investment in virtual infrastructure this may or may not be a good idea. I’m still a fan of dedicated hardware, but in some cases the redundancy provided by a virtual infrastructure can out way the potential performance gains.

Optimising Windows 7 Images for use in VDI

Cool tool for optimizing win7 VDI – http://blogs.technet.com/b/deploymentguys/archive/2010/08/31/optimising-windows-7-images-for-use-in-vdi.aspx

The beta of this tool is available from http://www.autoitscript.com/files/tools/VDIOptimizer.zip but please remember The tool is provided “AS IS” with no warranties, confers no rights, and is not supported by the authors or Microsoft Corporation.

vmware : IDE to SCSI

I’ve found that vmware converter (this may be fixed in newer verions) creates vmware guests with an IDE controller. There can be performance issues if you choose to remain with this particular controller… Best bet is to change it to one of the various vmware SCSI controllers…

Depending on which windows operating system you are running depends on which controller you use….  http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1006621

Guest Operating System
Adapter Type
Windows 2003, 2008, Vista
Windows NT, 2000, XP



You can easily change the type of the virtual controller for a given disk.
Lets have a look at an example.

# Disk DescriptorFile

# Extent description
RW 4193792 FLAT “diskname-f001.vmdk” 0
RW 2097664 FLAT “diskname-f002.vmdk” 0
# The Disk Data Base
ddb.adapterType = “ide”
ddb.virtualHWVersion = “3”
ddb.geometry.cylinders = “6241”
ddb.geometry.heads = “16”
ddb.geometry.sectors = “63”

The disk above uses a virtual ide-controller.

ddb.adapterType = “buslogic” This entry converts the disk into a SCSI-disk with BusLogic Controller

ddb.adapterType = “lsilogic”   This entry converts the disk into a SCSI-disk with LSILogic Controller
ddb.adapterType = “ide”   This entry converts the disk into a IDE-disk with Intel-IDE Controller

This changes the harddisk – but doesn’t change the controller itself.

ide0.present = “TRUE”
ide1.present = “TRUE”
scsi0.virtualDev = “lsilogic”
scsi0.virtualDev = “buslogic”
scsi1.virtualDev = “lsilogic”
scsi1.virtualDev = “buslogic”

Use entries like this in your *.vmx file. By the way, you can have LSI-logic and BUS-logic controllers in one VM.

Think twice before you make changes like this with a boot-disk.

Bluescreen 07b – mass-storage driver:
Activate the apropriate driver in the registry: intelide.sys or vmscsi.sys or symmpi.sys – you may have to add files as well.

If you get the above issue on a w2k8 box you might be able to enable the LSI_SAS driver before you convert the machine to SCSI controller.

  1. Boot machine with IDE controller
  2. Take a snapshot (for failback)
  3. Regedit and find the following key \\HKLM\SYSTEM\ControlSet001\Services\LSI_SAS
  4. Change the “Start” dword from 4 to 0
  5. Shutdown the machine
  6. Remove all the virtual disks (do not delete the disks, just remove them)
  7. Create copies of each .vmdk file (cp) (for failback)
  8. Edit the .vmdk file for each disk (vi)
  9. Change the “adaptertype” to “lsilogic” (if w2k8)
  10. Re-add existing disks (this should also bring in a LSI SAS controller)
  11. Boot the machine

Black screen with cursor blinking in the topleft of the screen:
Write a new partition boot-sector.

vmware – finding orphaned files

Use following script for finding orphaned files in your vmware env…

# Purpose : List all orphaned vmdk on all datastores in all VC's
# Version: 1.1
# Author : HJA van Bokhoven
# Modifications: LucD

$arrayVC = @(“server”)
$report = @()

foreach ($strVC in $arrayVC)
Connect-VIServer $strVC
$arrUsedDisks = Get-View -ViewType VirtualMachine | % {$_.Layout} | % {$_.Disk} | % {$_.DiskFile}
$arrDS = Get-Datastore | Sort-Object -property Name
foreach ($strDatastore in $arrDS)
Write-Host $strDatastore.Name
$ds = Get-Datastore -Name $strDatastore.Name | % {Get-View $_.Id}
$fileQueryFlags = New-Object VMware.Vim.FileQueryFlags
$fileQueryFlags.FileSize = $true
$fileQueryFlags.FileType = $true
$fileQueryFlags.Modification = $true
$searchSpec = New-Object VMware.Vim.HostDatastoreBrowserSearchSpec
$searchSpec.details = $fileQueryFlags
$searchSpec.matchPattern = "*.vmdk"
$searchSpec.sortFoldersFirst = $true
$dsBrowser = Get-View $ds.browser
$rootPath = "["+$ds.summary.Name+"]"

#Workaround for vSphere 4 fileOwner bug
if ($dsBrowser.Client.Version -eq "Vim4") {
$searchSpec = [VMware.Vim.VIConvert]::ToVim4($searchSpec)
$searchSpec.details.fileOwnerSpecified = $true
$dsBrowserMoRef = [VMware.Vim.VIConvert]::ToVim4($dsBrowser.MoRef);
$searchTaskMoRef = $dsBrowser.Client.VimService.SearchDatastoreSubFolders_Task($dsBrowserMoRef, $rootPath, $searchSpec)
$searchResult = [VMware.Vim.VIConvert]::ToVim($dsBrowser.WaitForTask([VMware.Vim.VIConvert]::ToVim($searchTaskMoRef)))
} else {
$searchResult = $dsBrowser.SearchDatastoreSubFolders($rootPath, $searchSpec)

foreach ($folder in $searchResult)
foreach ($fileResult in $folder.File)
if ($fileResult.Path)
if (-not ($arrUsedDisks -contains ($folder.FolderPath + $fileResult.Path))){
$row = "" | Select DS, Path, File, Size, ModDate, Host
$row.DS = $strDatastore.Name
$row.Path = $folder.FolderPath
$row.File = $fileResult.Path
$row.Size = $fileResult.FileSize
$row.ModDate = $fileResult.Modification
$row.Host = (Get-View $ds.Host[0].Key).Name
$report += $row
# Disconnect session from VC
disconnect-viserver -confirm:$false

$report | Export-Csv "C:\VMDK-orphaned.csv" -noTypeInformation

vmware and NLB (unicast vs multicast)

Thought I’d get a bit of clarification around this one. VMWare states –

VMware recommends that you use multicast mode, because unicast mode forces the physical switches on the LAN to broadcast all Network Load Balancing traffic to every machine on the LAN.

There are some issues which come to the surface if you are using unicast. Where possible we should always try to use a Multicast NLB as Unicast can cause some complications around vSwitch configurations. (requirement to change switch notifications to “No” – and seemingly break port id teaming). Detail below;


• All members of the NLB cluster must be running on the same ESX host.
• All members of the NLB cluster must be connected to the single portgroup on the virtual switch
• VMotion for unicast NLB virtual machines is not supported (unless you want to migrate ALL NLB members to a different ESX host)
• The Security Policy Forged Transmit on the Portgroup is set to Accept
• The transmission of RARP Packet is prevented on the Portgroup/Virtual Switch as explained in the later part of the article.

Here is a bit about each type… (from vmware docs)


Multicast mode allows communication among hosts because it adds a Layer 2 multicast address to the cluster instead of changing the cluster. Communication among hosts is possible because the hosts retain their original unique media access control (MAC) addresses and already have unique media access control (MAC) addresses and already have unique dedicated IP addresses. However, the address resolution protocol (ARP) reply that is sent by a host in the cluster (in response to an ARP request) maps the cluster’s unicast IP address to its multicast MAC address.

Some routers do not support the resolution of unicast IP addresses to multicast MAC addresses, and they discard the ARP reply. As a result, an administrator must add a static ARP entry in the router, mapping the cluster IP address to its MAC address.

  • Can be single nic
  • Add static ARP to default gateway


Unicast mode works seamlessly with all routers and Layer 2 switches. However, this mode induces switch flooding, a condition in which all switch ports are flooded with Network Load Balancing
traffic, even ports to which servers not involved in Network Load Balancing are attached. To communicate among hosts, you must have a second virtual adapter for each host.

Normally, switched environments avoid port flooding when a switch learns the MAC addresses of the hosts that are sending network traffic through it. The Network Load Balancing cluster masks the cluster’s MAC address for all outgoing traffic to prevent the switch from learning the MAC address.

On an ESX host, the VMkernel sends a reverse address resolution protocol (RARP) packet each time
certain actions occur—for example, when a virtual machine is powered on, when there is a teaming failover, or when certain VMotion operations occur. The RARP packet gives physical switches the MAC
address of the virtual machine involved in the action. In a Network Load Balancing cluster environment, after a Network Load Balancing node is powered on, the notification in the RARP packet exposes the MAC address of the cluster NIC. As a result, switches might begin to send all inbound traffic destined for the Network Load Balancing cluster through one switch port to a single node of the cluster.

Because the virtual switch operates with complete data about the underlying MAC addresses of the virtual NICs inside each virtual machine, it always correctly forwards packets containing a MAC address
matching that of a running virtual machine. As a result of this behavior, the virtual switch does not forward traffic destined for the Network Load Balancing MAC address outside the virtual environment
into the physical network, because it is able to forward it to a local virtual machine.

  • Requires 2 nics if you want host to host communication.

vmware – cpu ready (what to look for) and performance

keeping things simple – below are the numbers to look for when your investigating cpu scheduling issues withing a vmware environment…

Look for the cpu “ready” counter which is a percentage of time that the virtual machine was ready, but could not get scheduled to run on the physical CPU. CPU ready time is dependent on the number of virtual machines on the host and their CPU loads.

These are the peak / average figures to watch for in vmware performance graphs. Anything above 5% cpu ready time is worth fixing. (Note : the sampling rate is different depending on which view you are looking at)

Realtime : 1000 x 20 (20 second sample) x 5% = 1000

Daily : 1000 x 300 (5 min sample) x 5% = 15000
Weekly : 1000 x 1800 (30 min sample) x 5% = 90000
Monthly : 1000 x 7200 (2hrs sample) x 5% = 360000

Note : above is 1vCPU only, see this reference for some more detail and some nice charts!


EMC Celerra Optimizations for VMware on NFS

go here http://blog.scottlowe.org/2010/01/31/emc-celerra-optimizations-for-vmware-on-nfs/

Turn on the uncached write mechanism for NFS file systems used as VMware datastores. This can have a significant performance improvement for VMDKs on NFS but isn’t the default setting. From the Control Station, you can use this command to turn on the uncached write mechanism:
server_mount <data mover name> -option <options>,uncached <file system name> <mount point>
Be sure to review pages 99 through 101 of the VMware on Celerra best practices document for more information on the uncached write mechanism and any considerations for its use.

vmware and load balancing NFS trunks

Straight from the post below, this is the best way (currently) to load balance your NFS datastores…  No MPIO magic here unfortunately.


Basically you can setup IP alias on the NFS side and then setup multiple connections (using a unique IP) per datastore  on each ESX host. This works well if you are using a team of nics running IP-hash load balancing…

Static EtherChannel.

My setup is as follows:

ESXi 4.0 U1, Cisco 3750 Switches, and NetApp NFS on the storage side.

I have a total of 8 nics. I divided the nics into 3 groups.

2 nics on vSwitch0 for Mgmt & vMotion
3 nics on vSwitch1 for VM’s (Multiple port groups (3 VLANS))
3 nics on vSwitch2 for IP Storage (Mostly NFS, a little iSCSI)
(One vSwitch3 I also have a VM port group for iSCSI access from within the VM)

Since I have 3 nics on my IPStorage port group I needed a way to be able to utilize all three nics and not have the server just use one for ingress and egress traffic. This was done by:

Setting up a static EtherChannel on the cisco switch (Port Channel).
Configuring the cisco switch to IP Hash
Configure the vSwitch to “Route based on IP Hash” as well.

The next part is to create multiple datastores on the NFS device. Each of my NFS datastores is about 500GB in size. Reason for this is that my larger luns are iSCSI and are access directly from the VM using the MS iSCSI initiator from the VM itself.
My NetApp NAS has an address of, let say, So all my data stores are accessible by utilizing the address of “\\\NFS-Store#”. This will not be useful as the esx box and the cisco switch will always use the same nic/port to access the nas device. This is due to the algorithm (IP HASH) to decide what link it’ll go over. So to resolve the issue, I added IP aliases to the NFS box. NetApp allows to have multiple Ip addresses pointing to the same NFS export, I suspect EMC would do the same. So, I added 2 aliases 51 & 52. Now my NFS datastores are accessible by using Ip address,.51, & .52.

So I went ahead and added the datastores to the ESX box using the multiple IP addresses:

Datastore1 = \\\NFS-Store1
Datastore2 = \\\NFS-Store2
Datastore3 = \\\NFS-Store3

If you have more datastores it’ll just repeat: Datastore4 = \\\NFS-Store4 and so on…

Since having multiple datastores and address to each, the 3 nics on the ESX box dedicated to IP Storage get utilized. It does not aggregate the bandwidth but it does use all three to send and recieve packets. So the fastest speed you will get is 1Gbit, theoretically, each way for traffic but, it is better than trying to cram all the traffic over 1 nic.

I also enabled Jumbo Frames on the vSwitch as well as the vmNic for IP-Stroage. (need the best performance!)
I should mention that your NFS storage device should have EtherChannel setup on it as well. Otherwise, you’ll be on the same boat just on the other end of it.

Hope it helps!

Larry B.

I should mention that you should not use different addresses to access the same NFS share (datastore). It is not supported and may cause you issues.