Want iSCSI in opensolaris?
Grab SUNWiscsitgt via package manager.
enable the server via svcadm;
svcadm enable iscsitgt
create your zfs iscsi pool; (this command will limit iscsi drive to 500GB in size)
zfs create -V 500G tank/iscsi
set isci on via zfs command;
zfs set shareiscsi=on tank/iscsi
check that target is up and running;
iscsitadm list target -v
Done. Should be able to connect via ip from another machine. I have not covered CHAP or any client side configuration. Assumed isolated LAN.
If you want to give zfs a go and also want a dedicated file server this is the solution.
Check it out here;
http://www.freenas.org/
I’m still using opensolaris though as i like running a few virtualbox machines on the same box.
Recently i hit what i thought was a huge disaster with my ZFS array. Essentially i was unable to import my zpool without causing the kernel to panic and reboot. Still unsure of the exact reason, but it didn’t seem to be due to a hardware fault. (zpool import showed all disks as ONLINE)
When i tried to import with zpool import -f tank the machine would lockup and reboot (panic).
The kernel panic; (key line)
> genunix: [ID 361072 kern.notice] zfs: freeing free segment (offset=3540185931776 size=22528)
Nothing i could do would fix it… tried both of these options in the system file with no success;
set zfs:zfs_recover=1
set aok=1
After a quick email from a Sun Engineer, the zdb command line that fixed it;
zdb -e -bcsvL <poolname>
zdb is a read only diagnostic tool, but seemed to read through the sectors that had the corrupt data and fix things?? (not sure how a read only tool does that) – the run took well over 15hrs.
Updated: 20/10/2009
Apparently if you have set zfs:zfs_recover=1 in your system file the zdb command will operate in a different manner fixing the issues it encounters.
Remember to run a zpool scrub <poolname> if you are lucky enough to get it back online.
Time for me to add some more sata ports to my opensolaris build. I’ve been using the SI 3114 PCI cards (4x sata) up until now without any issue, but they are limited by the bandwidth on the PCI slot. Time to upgrade and boost my performance.
At the moment i’m looking at grabbing one of these UIO cards;
http://www.supermicro.com/products/accessories/addon/AOC-USAS-L8i.cfm
From what iv’e been reading these cards will work fine in a PCI-e slot (8x / 16x) after a bit of modding and display the drives straight to opensolaris without any additional drivers etc. (same chipset used in various sun servers)
The backplate on a UIO card is essentially on backwards, when you remove the backplate and put the card into the PCI-e slot all the components will appear on the other side to normal. It is possible if you have a spare PCI-e backplate to attach to this card (just unscrew the current backplate and replace).
And the required mini SAS to SATA cables from extreme deal;
http://www.dealextreme.com/details.dx/sku.18023
Done.
Updated : 02/09/2009
Put this card in and bingo no problems. Had to export and re-import the zpool as it had problems with the drives being on a different controller? (hadn’t seen that before), but after that everything was working very well as expected. Cool!
Since im using ZFS as storage via NFS for my some of my vmware environments i need to ensure that latency on my disk is reduced where ever possible.
There is alot of talk about ZFS compression being “faster” than a non-compressed pool due to less physical data being pulled off the drives. This of course depends on the system powering ZFS, but i wanted to run some tests specifically on latency. Throughput is fine in some situations, but latency is a killer when it comes to lots of small reads and writes (in the case of hosting virtual machines)
I recently completed some basic tests focusing on the differences in latency when ZFS compression (lzjb) is enabled or disabled. IOMeter was my tool of choice and i hit my ZFS box via a mapped drive.
I’m not concerned with the actual figures, but the difference between the figures
I have run the test multiple times (to eliminate caching as a factor) and can validate that compression (on my system anyhow) increases latency
Basic Results from a “All in one” test suite… (similar results across all my tests)
ZFS uncompressed:
IOps : 2376.68
Read MBps : 15.14
Write MBps : 15.36
Average Response Time : 0.42
Average Read Response Time : 0.42
Average Write Response Time : 0.43
Average Transaction Time : 0.42
ZFS compressed: (lzjb)
IOps : 1901.82
Read MBps : 12.09
Write MBps : 12.28
Average Response Time : 0.53
Average Read Response Time : 0.44
Average Write Response Time : 0.61
Average Transaction Time : 0.53
As you can see from the results, the AWRT especially is much higher due to compression. I wouldn’t recommend using zfs compression where latency is a large factor (virtual machines)
Note: Under all the tests performed the CPU (dual core) on the zfs box was never 100% – eliminating that as a bottleneck.