One of the best articles i have found on this subject is here : http://blog.scottlowe.org/2008/07/16/understanding-nic-utilization-in-vmware-esx/
There is some additional information here on setting up an etherchannel on the cisco side : http://blog.scottlowe.org/2006/12/04/esx-server-nic-teaming-and-vlan-trunking/
This can be handy if you need a single VM to use both physical nics in a load-balanced manner – both outbound and inbound. Of course its not really that simple though. This will really only add a benefit if the VM is communicating to multiple destinations (using ip hash – a single destination from a single VM with one IP will always be limited to the same physical nic).
switch(config)#int port-channel 1
switch(config-if)#description NIC team for ESX server
switch(config-if)#int gi0/1
switch(config-if)#channel-group 1 mode on
switch(config-if)#int gi0/2
switch(config-if)#channel-group 1 mode on
As per the article ensure you are using the same etherchannel method. The first command shows your current load-blance method, the 2nd command changes it to ip hash.
show etherchannel load-balance
port-channel load-balance src-dst-ip
Another solution is to use multiple iSCSI paths. This is newly supported within vSphere, see this post on setting up multiple paths : http://goingvirtual.wordpress.com/2009/07/17/vsphere-4-0-with-software-iscsi-and-2-paths/
Here is another good article on iSCSI within vSphere : http://www.delltechcenter.com/page/A+“Multivendor+Post”+on+using+iSCSI+with+VMware+vSphere
Some important points on using EMC Clariion with vSphere : http://virtualgeek.typepad.com/virtual_geek/2009/08/important-note-for-all-emc-clariion-customers-using-iscsi-and-vsphere.html