http://www.sonoracomm.com/support/19-inet-support/233-vyatta-cable

Masquerade Natting;

set service nat rule 10 source address 192.168.77.0/24
set service nat rule 10 outbound-interface eth0
set service nat rule 10 type masquerade

set service nat rule 20 source address 10.200.88.0/24
set service nat rule 20 outbound-interface eth0
set service nat rule 20 type masquerade

Essentially every thing leaving from these two networks will be natted via the eth0 interface. i.e. they will be using the same IP as eth0 for their requests.

Static Nat Route (pinhole);

set service nat rule 2 inbound-interface eth0
set service nat rule 2 protocol tcp
set service nat rule 2 type destination
set service nat rule 2 description “Web Server1″
set service nat rule 2 destination address 192.168.109.254
set service nat rule 2 destination port 81
set service nat rule 2 inside-address address 10.200.88.45
set service nat rule 2 inside-address port 80

This rule takes everything that hits port 81 on 192.168.109.254 and sends it through to 10.200.88.45 (which is another routeable network) — in this case a directly attached network on eth1.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

netstat -rn

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

you may want to implement jumbo frames to your iSCSI backend…

Enable jumbo frames on your iSCSI target and switches then complete the following on the ESXi hosts (iSCSI initiators)…

This to list your current switch details and port group names…

esxcfg-vswitch -l

The follownig to allow jumbo frames onto your vswtich (insert your vSwich in following)

esxcfg-vswtich -m 9000 vSwitch0

Then delete and recreate your kernel port groups that you setup for iscsi, my commands looked like this… (repeat for each iscsi kernel port you have)

esxcfg-vmknic -d iSCSI-1

esxcfg-vmknic -i 10.0.0.101 -n 255.255.0.0 -m 9000 iSCSI-1

to confirm you have set the MTU (frame size) correctely, run the following….

esxcfg-vmknic -l

you should see your iSCSI kernel ports with a MTU of 9000 all going well.

Related posts:

1. vSphere and Multipathing iSCSI This is just a quick reference to create a multiplathing...
2. create usb install for esx / esxi First format your usb / flash drive with FAT. Then...
3. ESX – network utilization One of the best articles i have found on this...

Related posts brought to you by Yet Another Related Posts Plugin.

Create two virtual kernel switches, one called “iSCSI-1″ and the other called “iSCSI-2″ (and so on if you have more nics)

Then per kernel portgroup ensure that only one of the nics is active. For the “iSCSI-1″ portgroup configure it to override the virtual switch settings and move nic 0 to active and nic 1 to unused. For the “iSCSI-2″ portgroup configure it to override the virtual switch settings and move nic 1 to active and nic 0 to unused.

Now you have to run some esxcli commands to gel things together…. alt-f1 on the esxi console and type “unsupported” followed by your root password. The following is the command i have to run to get both my portgroups to work together on the iscsi hba….  (you can check your vmk number from the networking config screen)

esxcli swiscsi nic add -n vmk1 -d vmhba33
esxcli swiscsi nic add -n vmk2 -d vmhba33

For each target then change the path selection method to “round-robin”.

Go back to storage adapters and click “rescan”

If you want all future iSCSI targets to automatically use round-robin you must also run the following from commandline…   (this is for our HP Lefthand, your “storage array type” may be different. Its listed under your target details). Basically sets round robin as a default for this type of array. In general you should do this first before presenting any LUNs etc, else you may have to bounce your box.

esxcli nmp satp setdefaultpsp –satp VMW_SATP_DEFAULT_AA –psp VMW_PSP_RR

Related posts:

1. Jumbo Frames on your vSphere ESXi box Continuing on from this http://sigtar.com/2010/02/04/vsphere-and-multipathing-iscsi/ you may want to implement...
2. OpenSolaris – iSCSI Want iSCSI in opensolaris? Grab SUNWiscsitgt via package manager. enable...
3. vmware – measuring iscsi write performance I picked this trick up off vmware support. If you’ve...

Related posts brought to you by Yet Another Related Posts Plugin.

Grab SUNWiscsitgt via package manager.

enable the server via svcadm;

svcadm enable iscsitgt

create your zfs iscsi pool;  (this command will limit iscsi drive to 500GB in size)

zfs create -V 500G tank/iscsi

set isci on via zfs command;

zfs set shareiscsi=on tank/iscsi

check that target is up and running;

iscsitadm list target -v

Done. Should be able to connect via ip from another machine. I have not covered CHAP or any client side configuration. Assumed isolated LAN.

Related posts:

1. vSphere and Multipathing iSCSI This is just a quick reference to create a multiplathing...
2. Jumbo Frames on your vSphere ESXi box Continuing on from this http://sigtar.com/2010/02/04/vsphere-and-multipathing-iscsi/ you may want to implement...
3. vmware – measuring iscsi write performance I picked this trick up off vmware support. If you’ve...

Related posts brought to you by Yet Another Related Posts Plugin.

If you want your box “network browseable” you’ll need to load nmbd.

You can find a really good write up on modifiying the SMF for samba to autostart the required service here : http://wikis.sun.com/display/BigAdmin/Enabling+Browsing+with+Samba+in+Solaris+10+Update+4

import this file samba.xml

run these commands;

# svcadm disable samba
# svccfg import samba.xml
# svccfg delete svc:/network/samba:default
# svcadm refresh samba:smbd
# svcadm refresh samba:nmbd
# svcadm enable samba:smbd
# svcadm enable samba:nmbd

Related posts:

1. OpenSolaris – Samba server Time to share your newly created ZFS volume via samba...
2. opensolaris – network teaming Otherwise known as trunking or link aggregation. I believe it...
3. OpenSolaris – iSCSI Want iSCSI in opensolaris? Grab SUNWiscsitgt via package manager. enable...

Related posts brought to you by Yet Another Related Posts Plugin.

There is some additional information here on setting up an etherchannel on the cisco side : http://blog.scottlowe.org/2006/12/04/esx-server-nic-teaming-and-vlan-trunking/

This can be handy if you need a single VM to use both physical nics in a load-balanced manner – both outbound and inbound. Of course its not really that simple though. This will really only add a benefit if the VM is communicating to multiple destinations (using ip hash – a single destination from a single VM with one IP will always be limited to the same physical nic).

switch(config)#int port-channel 1
switch(config-if)#description NIC team for ESX server
switch(config-if)#int gi0/1
switch(config-if)#channel-group 1 mode on
switch(config-if)#int gi0/2
switch(config-if)#channel-group 1 mode on

As per the article ensure you are using the same etherchannel method. The first command shows your current load-blance method, the 2nd command changes it to ip hash.

show etherchannel load-balance
port-channel load-balance src-dst-ip

Another solution is to use multiple iSCSI paths. This is newly supported within vSphere, see this post on setting up multiple paths : http://goingvirtual.wordpress.com/2009/07/17/vsphere-4-0-with-software-iscsi-and-2-paths/

Here is another good article on iSCSI within vSphere : http://www.delltechcenter.com/page/A+“Multivendor+Post”+on+using+iSCSI+with+VMware+vSphere

Some important points on using EMC Clariion with vSphere : http://virtualgeek.typepad.com/virtual_geek/2009/08/important-note-for-all-emc-clariion-customers-using-iscsi-and-vsphere.html

Related posts:

1. vSphere and Multipathing iSCSI This is just a quick reference to create a multiplathing...
2. Jumbo Frames on your vSphere ESXi box Continuing on from this http://sigtar.com/2010/02/04/vsphere-and-multipathing-iscsi/ you may want to implement...
3. OpenSolaris – iSCSI Want iSCSI in opensolaris? Grab SUNWiscsitgt via package manager. enable...

Related posts brought to you by Yet Another Related Posts Plugin.

There is alot of talk about ZFS compression being “faster” than a non-compressed pool due to less physical data being pulled off the drives. This of course depends on the system powering ZFS, but i wanted to run some tests specifically on latency. Throughput is fine in some situations, but latency is a killer when it comes to lots of small reads and writes (in the case of hosting virtual machines)

I recently completed some basic tests focusing on the differences in latency when ZFS compression (lzjb) is enabled or disabled. IOMeter was my tool of choice and i hit my ZFS box via a mapped drive.

I’m not concerned with the actual figures, but the difference between the figures

I have run the test multiple times (to eliminate caching as a factor) and can validate that compression (on my system anyhow) increases latency

Basic Results from a “All in one” test suite… (similar results across all my tests)

ZFS uncompressed:

IOps : 2376.68
Read MBps : 15.14
Write MBps : 15.36
Average Response Time : 0.42
Average Read Response Time : 0.42
Average Write Response Time : 0.43
Average Transaction Time : 0.42

ZFS compressed: (lzjb)

IOps : 1901.82
Read MBps : 12.09
Write MBps : 12.28
Average Response Time : 0.53
Average Read Response Time : 0.44
Average Write Response Time : 0.61
Average Transaction Time : 0.53

As you can see from the results, the AWRT especially is much higher due to compression. I wouldn’t recommend using zfs compression where latency is a large factor (virtual machines)

Note: Under all the tests performed the CPU (dual core) on the zfs box was never 100% – eliminating that as a bottleneck.

Related posts:

1. iometer testing The following is some notes i have around the parameters...
2. vMA – Using HP Power Protector agent to shutdown virtual hosts Goal was to create a vMA which would send a...
3. vmware – measuring iscsi write performance I picked this trick up off vmware support. If you’ve...

Related posts brought to you by Yet Another Related Posts Plugin.

Official docs on the process here… http://docs.sun.com/app/docs/doc/819-6990/gdysn?a=view and some good bits here http://blogs.sun.com/nickyv/entry/link_aggregation_jumpstart_post_install

dladm (data link admin) is the tool for the job. List the links you currently have…

dladm show-link

First shut down the links you are currently using..  (you will have to do this on the console)

ifconfig e1000g1 unplumb

Now join the two nics into one aggregate connection via….

dladm create-aggr -l e1000g1 -l rge0 aggr1

then bring up the new aggregate link

ifconfig aggr1 plumb IP-address up

Show link

dladm show-aggr

(Optional) Make the IP configuration of the link aggregation persist across reboots.

1. Create the /etc/hostname file for the aggregation’s interface.

   If the aggregation contains IPv4 addresses, the corresponding hostname file is/etc/hostname.aggr1. For IPv6–based link aggregations, the corresponding hostname file is/etc/hostname6.aggr1.

2. Type the IPv4 or IPv6 address of the link aggregation into the file.

3. Perform a reconfiguration boot.

I have teamed an intel nic (e1000g) and a (rge) together without any issues…  the rge drive by itself had issues, but i have not come across them again since i trunked both interfaces together. Perhaps the e1000g takes the load while the other nic dies off..

Updated : 4/08/2009

To test the throughput / load balancing run these commands (in two terminal sesssions);

dladm show-link -s -i 5 rge0

dladm show-link -s -i 5 e1000g1

It will return the packets going over each nic. Copy some files back and forth over the interface and watch the numbers. RBYTES and OBYTES are the fields to watch (received and out bytes)

Related posts:

1. Opensolaris – Samba and Network browsing If you’ve setup samba (rather than the builtin cifs/smb) chances...
2. ESX – network utilization One of the best articles i have found on this...

Related posts brought to you by Yet Another Related Posts Plugin.

Type this command to check to see what http ports are currently allowed;  (remove filter to show all rules)

semanage port -l|grep http

To add another port type the following (with the port you wish to add etc);

semanage port -a -t http_port_t -p tcp 81

If you want to disable selinux completely then go into /etc/selinux/config and set selinux=disabled. Save then reboot.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.