HP Gen 8 servers and networking issues – TG3 driver

There is a bug in the tg3 driver on the ESXi hosts (1gbit broadcom cards in the new hosts). If the network card is put under load and netqueue is enabled it will sometimes decide to drop all traffic. Essentially i’ve disabled netqueue and the problems have gone away…. as per this vm kb :

http://kb.vmware.com/kb/2035701

The isues will present themselves as log entires like so;

2012-11-19T18:58:52.137Z cpu17:4155)<6>tg3 : vmnic8: RX NetQ allocated on 1
2012-11-19T18:58:52.138Z cpu17:4155)<6>tg3 : vmnic8: NetQ set RX Filter: 1 [00:50:56:71:46:87 0]
2012-11-19T18:58:52.138Z cpu17:4155)<6>tg3 : vmnic7: RX NetQ allocated on 1
2012-11-19T18:58:52.138Z cpu17:4155)<6>tg3 : vmnic7: NetQ set RX Filter: 1 [00:50:56:71:46:87 0]
2012-11-19T18:59:12.139Z cpu21:4155)<6>tg3 : vmnic4: NetQ remove RX filter: 1
2012-11-19T18:59:12.139Z cpu21:4155)<6>tg3 : vmnic4: Free NetQ RX Queue: 1
2012-11-19T18:59:22.137Z cpu24:4155)<6>tg3 : vmnic4: RX NetQ allocated on 1
2012-11-19T18:59:22.138Z cpu24:4155)<6>tg3 : vmnic4: NetQ set RX Filter: 1 [00:50:56:71:46:87 0]
2012-11-19T18:59:42.138Z cpu21:4155)<6>tg3 : vmnic7: NetQ remove RX filter: 1
2012-11-19T18:59:42.138Z cpu21:4155)<6>tg3 : vmnic7: Free NetQ RX Queue: 1
2012-11-19T18:59:42.140Z cpu21:4155)<6>tg3 : vmnic4: NetQ remove RX filter: 1
2012-11-19T18:59:42.140Z cpu21:4155)<6>tg3 : vmnic4: Free NetQ RX Queue: 1
2012-11-19T19:00:02.139Z cpu28:4155)<6>tg3 : vmnic8: NetQ remove RX filter: 1

hyper-v to esx conversion

V2V cold clone process for SBS / exchange / sql / DC’s etc…..    (works in vSphere)

Steps to convert from hyper-v guest to vmware guest

  • Remove hyper-v integrated services while hyper-v guest is running. (if possible)
  • Note down NIC networking details first
  • Clean shutdown hyper-v guest
  • Use StarWind Converter (free) to convert VHD to “dynamically growing VMware” image and use “IDE” as type. Note “SCSI” does NOT work if you have not got the LSI / BUS drivers in the source image, just use the default options – dynamic vmware and IDE.
  • Create VMware guest shell (virtual machine equivalent to hyper-v specs but without the disks) – remove all hard drives, iscsi controller should also disappear.
  • Upload the VMDK to same data store as virtual machine and attach to the virtual guest in the right order. They should be detected as “IDE”
  • Start new vmware guest
  • If you could not remove integrated services (i.e. hot clone), then you MUST disable hyper-v services immediately.
  • Install vmware tools – reboot
  • Remove hidden NIC and other orphaned devices, and reconfigure network card as per original.
  • Windows may need to be re-activated – do this.

Note : converting a machine to use an IDE controller will limit its performance!

Jumbo Frames on your vSphere ESXi box

Continuing on from this https://sigtar.com/2010/02/04/vsphere-and-multipathing-iscsi/

you may want to implement jumbo frames to your iSCSI backend…

Enable jumbo frames on your iSCSI target and switches then complete the following on the ESXi hosts (iSCSI initiators)…

This to list your current switch details and port group names…

esxcfg-vswitch -l

The following to allow jumbo frames onto your vswtich (insert your vSwich in following)

esxcfg-vswtich -m 9000 vSwitch0

Then create your kernel port groups that you want to use for iscsi, my commands looked like this… (repeat for each iscsi kernel port you have) — note : DO NOT ADD ANY VMKERNEL(s) from the GUI, instead use the steps below

esxcfg-vswitch -A iSCSI vSwitch0

If you are using a tagged vlan you will also need to add the vlan tag to the above port group

esxcfg-vswitch -v 192 -p iSCSI vSwitch0

esxcfg-vmknic -a iSCSI -i 10.0.0.101 -n 255.255.0.0 -m 9000

To confirm you have set the MTU (frame size) correctely, run the following….

esxcfg-vmknic -l

you should see your iSCSI kernel ports with a MTU of 9000 all going well. Confirm connectivity using

vmkping –s 9000 10.0.0.10

create usb install for esx / esxi

First format your usb / flash drive with FAT. Then copy all the contents of the installer .iso onto the drive.

Next delete isolinux.bin and rename isolinux.cfg to syslinux.cfg

Edit syslinux.cfg and append “usb” to the line starting with “append” … i.e. similar to this

append vmkboot.gz — vmkernel.gz — sys.vgz — cim.vgz — ienviron.tgz — image.tgz — install.tgz usb

Next grab the latest syslinux.exe (zip for windows) from here and run syslinux -s –ma <driveletter>:

Done.

boot esx from usb drive

Its quite easy to boot esx from a usb device or flash card..

First download the .iso, then browse with winrar or similar. See if you can find the “big”.dd file this is the file we need.

i.e. the ESXi 4 file is called VMware-VMvisor-big-208167-x86_64.dd

Next grab WinImage and “restore image to physical harddrive” choose the .dd file the select your usb / flash drive.

Done.