HP Gen 8 servers and networking issues – TG3 driver

There is a bug in the tg3 driver on the ESXi hosts (1gbit broadcom cards in the new hosts). If the network card is put under load and netqueue is enabled it will sometimes decide to drop all traffic. Essentially i’ve disabled netqueue and the problems have gone away…. as per this vm kb :

http://kb.vmware.com/kb/2035701

The isues will present themselves as log entires like so;

2012-11-19T18:58:52.137Z cpu17:4155)<6>tg3 : vmnic8: RX NetQ allocated on 1
2012-11-19T18:58:52.138Z cpu17:4155)<6>tg3 : vmnic8: NetQ set RX Filter: 1 [00:50:56:71:46:87 0]
2012-11-19T18:58:52.138Z cpu17:4155)<6>tg3 : vmnic7: RX NetQ allocated on 1
2012-11-19T18:58:52.138Z cpu17:4155)<6>tg3 : vmnic7: NetQ set RX Filter: 1 [00:50:56:71:46:87 0]
2012-11-19T18:59:12.139Z cpu21:4155)<6>tg3 : vmnic4: NetQ remove RX filter: 1
2012-11-19T18:59:12.139Z cpu21:4155)<6>tg3 : vmnic4: Free NetQ RX Queue: 1
2012-11-19T18:59:22.137Z cpu24:4155)<6>tg3 : vmnic4: RX NetQ allocated on 1
2012-11-19T18:59:22.138Z cpu24:4155)<6>tg3 : vmnic4: NetQ set RX Filter: 1 [00:50:56:71:46:87 0]
2012-11-19T18:59:42.138Z cpu21:4155)<6>tg3 : vmnic7: NetQ remove RX filter: 1
2012-11-19T18:59:42.138Z cpu21:4155)<6>tg3 : vmnic7: Free NetQ RX Queue: 1
2012-11-19T18:59:42.140Z cpu21:4155)<6>tg3 : vmnic4: NetQ remove RX filter: 1
2012-11-19T18:59:42.140Z cpu21:4155)<6>tg3 : vmnic4: Free NetQ RX Queue: 1
2012-11-19T19:00:02.139Z cpu28:4155)<6>tg3 : vmnic8: NetQ remove RX filter: 1

hyper-v to esx conversion

V2V cold clone process for SBS / exchange / sql / DC’s etc…..    (works in vSphere)

Steps to convert from hyper-v guest to vmware guest

  • Remove hyper-v integrated services while hyper-v guest is running. (if possible)
  • Note down NIC networking details first
  • Clean shutdown hyper-v guest
  • Use StarWind Converter (free) to convert VHD to “dynamically growing VMware” image and use “IDE” as type. Note “SCSI” does NOT work if you have not got the LSI / BUS drivers in the source image, just use the default options – dynamic vmware and IDE.
  • Create VMware guest shell (virtual machine equivalent to hyper-v specs but without the disks) – remove all hard drives, iscsi controller should also disappear.
  • Upload the VMDK to same data store as virtual machine and attach to the virtual guest in the right order. They should be detected as “IDE”
  • Start new vmware guest
  • If you could not remove integrated services (i.e. hot clone), then you MUST disable hyper-v services immediately.
  • Install vmware tools – reboot
  • Remove hidden NIC and other orphaned devices, and reconfigure network card as per original.
  • Windows may need to be re-activated – do this.

Note : converting a machine to use an IDE controller will limit its performance!

Jumbo Frames on your vSphere ESXi box

Continuing on from this http://sigtar.com/2010/02/04/vsphere-and-multipathing-iscsi/

you may want to implement jumbo frames to your iSCSI backend…

Enable jumbo frames on your iSCSI target and switches then complete the following on the ESXi hosts (iSCSI initiators)…

This to list your current switch details and port group names…

esxcfg-vswitch -l

The following to allow jumbo frames onto your vswtich (insert your vSwich in following)

esxcfg-vswtich -m 9000 vSwitch0

Then create your kernel port groups that you want to use for iscsi, my commands looked like this… (repeat for each iscsi kernel port you have) — note : DO NOT ADD ANY VMKERNEL(s) from the GUI, instead use the steps below

esxcfg-vswitch -A iSCSI vSwitch0

If you are using a tagged vlan you will also need to add the vlan tag to the above port group

esxcfg-vswitch -v 192 -p iSCSI vSwitch0

esxcfg-vmknic -a iSCSI -i 10.0.0.101 -n 255.255.0.0 -m 9000

To confirm you have set the MTU (frame size) correctely, run the following….

esxcfg-vmknic -l

you should see your iSCSI kernel ports with a MTU of 9000 all going well. Confirm connectivity using

vmkping –s 9000 10.0.0.10

create usb install for esx / esxi

First format your usb / flash drive with FAT. Then copy all the contents of the installer .iso onto the drive.

Next delete isolinux.bin and rename isolinux.cfg to syslinux.cfg

Edit syslinux.cfg and append “usb” to the line starting with “append” … i.e. similar to this

append vmkboot.gz — vmkernel.gz — sys.vgz — cim.vgz — ienviron.tgz — image.tgz — install.tgz usb

Next grab the latest syslinux.exe (zip for windows) from here and run syslinux -s –ma <driveletter>:

Done.

boot esx from usb drive

Its quite easy to boot esx from a usb device or flash card..

First download the .iso, then browse with winrar or similar. See if you can find the “big”.dd file this is the file we need.

i.e. the ESXi 4 file is called VMware-VMvisor-big-208167-x86_64.dd

Next grab WinImage and “restore image to physical harddrive” choose the .dd file the select your usb / flash drive.

Done.

Using W2k3 R2 / W2k8 server as a NFS share for vmware

This is something i do in the lab so all of my vm’s are able to access iso’s etc (very handy for quick builds). Quite handy doing it through windows since its easiey enough to setup a windows network share to the same location and update various files via that.

This site has some good clear instructions : http://vmetc.com/2008/02/19/create-a-nfs-share-for-vm-iso-files-with-windows-2003-server-r2/

  1. On the Windows 2003 Server make sure “Microsoft Services for NFS” in installed. If not you need to add it under Add/Remove Programs, Windows
    Components, Other Network File and Print Services
  2. Next go to folder you want to share and right-click on it and select Properties
  3. Click on the NFS Sharing tab and select “Share this Folder”
  4. Enter a Share Name, check “Anonymous Access”
  5. In VirtualCenter, select your ESX server and click the “Configuration” tab and then select “Storage”
  6. Click on “Add Storage” and select “Network File System” as the storage type
  7. Enter the Windows Server name, the folder (share) name and a descriptive Datastore Name
  8. Done. Now you can map CD iso’s to your various vm’s.

 

This is similar in windows 2008 — screenshots of settings below…