I’ve been working through a SRM setup and have been looking at ways to optimize the amount of traffic that is sent over the WAN. The first obvious move is to move your vmware swap files off the replicated LUNS.
Another way is to reduce the sync window. i.e. how often is yoru replication technology trying to keep the source and destination in sync? — Increasing this window can sometimes help you out. But that all depends on your delta’s.
For example – In the case of a windows page file on an active server (SQL etc) it could “potentially” change the whole file within an hour. If your replication was set to every hour and the page file was 4gb then you’d be sending at least 4gb every hour. Changing the sync on your replicated LUN to 8hrs instead would mean you’d only send the 4gb of “delta” (i.e. blocks that have changed since original snap)
Problem is that you would typically want to sync your virtual machines on a more frequent schedule than 8hrs. So this is where you need to move your windows page files onto a separate LUN (also replicated), but on a larger sync window (perhaps only once if your servers are static).
Check this link for a great script to monitor changes made to a virtual machine via the vmware CBT API. This is perfect for finding culprit machines that are generating a lot of replication traffic if you are replicating over a WAN.
http://www.vmguru.com/index.php/articles-mainmenu-62/scripting/105-using-powershell-to-track-block-change-sizes-over-time
If you have some disks on a virtual machine you don’t want this script to capture then just set them as independent disks (so no snapshots can take place). This is handy if you have your windows page file on a separate disk that you don’t want to be measured as a part of the CBT changes.
Some very nice examples by Gene on how to setup vyatta as a router on this page;
http://www.sonoracomm.com/support/19-inet-support/233-vyatta-cable
Masquerade Natting;
set service nat rule 10 source address 192.168.77.0/24
set service nat rule 10 outbound-interface eth0
set service nat rule 10 type masquerade
set service nat rule 20 source address 10.200.88.0/24
set service nat rule 20 outbound-interface eth0
set service nat rule 20 type masquerade
Essentially every thing leaving from these two networks will be natted via the eth0 interface. i.e. they will be using the same IP as eth0 for their requests.
Static Nat Route (pinhole);
set service nat rule 2 inbound-interface eth0
set service nat rule 2 protocol tcp
set service nat rule 2 type destination
set service nat rule 2 description “Web Server1″
set service nat rule 2 destination address 192.168.109.254
set service nat rule 2 destination port 81
set service nat rule 2 inside-address address 10.200.88.45
set service nat rule 2 inside-address port 80
This rule takes everything that hits port 81 on 192.168.109.254 and sends it through to 10.200.88.45 (which is another routeable network) — in this case a directly attached network on eth1.
type the following to check your default routes…
netstat -rn
Continuing on from this http://sigtar.com/2010/02/04/vsphere-and-multipathing-iscsi/
you may want to implement jumbo frames to your iSCSI backend…
Enable jumbo frames on your iSCSI target and switches then complete the following on the ESXi hosts (iSCSI initiators)…
This to list your current switch details and port group names…
esxcfg-vswitch -l
The follownig to allow jumbo frames onto your vswtich (insert your vSwich in following)
esxcfg-vswtich -m 9000 vSwitch0
Then delete and recreate your kernel port groups that you setup for iscsi, my commands looked like this… (repeat for each iscsi kernel port you have)
esxcfg-vmknic -d iSCSI-1
esxcfg-vmknic -i 10.0.0.101 -n 255.255.0.0 -m 9000 iSCSI-1
to confirm you have set the MTU (frame size) correctely, run the following….
esxcfg-vmknic -l
you should see your iSCSI kernel ports with a MTU of 9000 all going well.