Daz’s bits and bobs

I’ve been working through a SRM setup and have been looking at ways to optimize the amount of traffic that is sent over the WAN. The first obvious move is to move your vmware swap files off the replicated LUNS.

Another way is to reduce the sync window. i.e. how often is yoru replication technology trying to keep the source and destination in sync? — Increasing this window can sometimes help you out. But that all depends on your delta’s.

For example – In the case of a windows page file on an active server (SQL etc) it could “potentially” change the whole file within an hour. If your replication was set to every hour and the page file was 4gb then you’d be sending at least 4gb every hour. Changing the sync on your replicated LUN to 8hrs instead would mean you’d only send the 4gb of “delta” (i.e. blocks that have changed since original snap)

Problem is that you would typically want to sync your virtual machines on a more frequent schedule than 8hrs. So this is where you need to move your windows page files onto a separate LUN (also replicated), but on a larger sync window (perhaps only once if your servers are static).

I hit this as a problem recently. One of my disks died in my raidz so i ran down to the store and grabbed me a replacement WD10EARS (Western Digital 1Tb Green) drive.

BUT…

The one thing the store didn’t mention to me is the new 4K cluster sizing on the drive. I guess they assume most people run windows (though the issues are also present in XP). See these posts…

http://blog.temeletry.co.uk/2010/05/wd-green-wd10ears/

Unfortunately they really don’t work as well as you’d like in a server 

• They come with a 5 second head spin down setting that causes them to park their heads if they have been left idle for more than 5 seconds. As it takes a second or two to spin back up this can result in a very laggy experience during interactive sessions.
• They do not have NCQ or any form of command queing/optimisation. This means that (on FreeBSD at least) you are stuck in the LOOK elevator. In particular this was noticed when doing sequential read & write (think dump|restore tar|untar etc) and interactive tasks simultaneously
• They really suck with FreeBSD and ZFS…

http://community.wdc.com/t5/Desktop/Poor-performace-in-OpenSolaris-with-4K-sector-drive-WD10EARS-in/m-p/21132

While the other 512-byte sector HDDs were reading/writing at 30MB/s sustained, this EARS model did not exceeded the 1MB/s barrier.

I know for sure that this is related to the 512-byte sector firmware emulation, because the disk works perfectly well if I partition it in a 4k-sector alignment.

The thing is that even in that way, using it in a ZFS RAIDZ configuration the performance is very poor because RAIDZ uses a dynamic stripe size.

The bottom line here is that folks like me, that use different versions of Unix, need the firmware to present the disk as a 4K-sector disk to unleash the full potential of the technology. The OS is already prepared to support that sector size, no need for emulation here.

http://opensolaris.org/jive/thread.jspa?threadID=125702

Some preliminary testing that I have done…the WD20EARS (2TB advanced format drives) actually presents emulated 512byte sectors to the host o/s.

The drive documentation indicates that jumpers 7-8 should be enabled if the o/s does not support advanced format drives – the drive still present 512 bytes sectors.

I have attempted to raise a support ticket querying this, and how one can disable 512byte sector emulation in the drive (perhaps through a firmware upgrade) but I have not received any response to date.

Hopefully is enough people raise support tickets, WD may release firmware that allows the drive to natively present 4k blocks. Other doco indicates several other jumper combinations – all do not seem to make the drive present 4k byte blocks.

Perhaps someone internal to sun that has a relationship with WD may be able to shed some light on this? It would be fantastic to find out that I was just doing something wrong -> then I can get the drives to be seen on 32bit systems (ie – our embedded kit for osol, velitium)

Tested using b133 (64bit intel).

Try to avoid the green drives in ZFS for now. Remember to do your research before you buy a bunch of disks. I was caught off guard by this small change (works fine in win7 etc) which kills performance in ZFS. Ouch.

Errr, I cant find the CIFS service in the 134 build.

I know it was renamed to as per below but still cant see it anywhere?

> system/file-system/smb (was SUNWsmbfs*)
> service/file-system/smb (was SUNWsmbs*)

Anyone have any ideas?

pkg search *smb gets me these… but i cant install them.

require depend service/file-system/smb@0.5.11-0.134 pkg:/redistributable@0.1-0.134
require depend service/file-system/smb@0.5.11-0.134 pkg:/storage/storage-server@0.1-0.134
require depend service/file-system/smb@0.5.11-0.134 pkg:/system/security/kerberos-5@0.5.11-0.134
require depend service/file-system/smb@0.5.11-0.134 pkg:/storage/storage-nas@0.1-0.134
require depend system/file-system/smb@0.5.11-0.134 pkg:/redistributable@0.1-0.134
require depend system/file-system/smb@0.5.11-0.134 pkg:/storage/storage-server@0.1-0.134
require depend system/file-system/smb@0.5.11-0.134 pkg:/slim_install@0.1-0.134

pkg install system/file-system/smb

No updates necessary for this image.

pkg install service/file-system/smb

Creating Plan
pkg: The following pattern(s) did not match any packages in the current catalog.
Try relaxing the pattern, refreshing and/or examining the catalogs:
service/file-system/smb

Update 23/05/2010
Problem was due to errors within package manager — see this post

Cool. zfs as of version 21 has deduplication built in. And thats the good dedup – synchronous dedup. i.e. deduped on the fly!

How easy is it to turn on? – very!

Once you have upgraded your zpool to 21 or above you can run the following command at the pool level and deduplication will be over all your data from that point onwards.

zpool set dedup=on tank

Done

Note : Watch your performance, it will drop like a rock if you do not have enough ram for your dedup tables. Do some tests after enabling this feature.

http://hub.opensolaris.org/bin/view/Community+Group+zfs/dedup

The following is some notes i have around the parameters i usually use during testing. There is a good document at ixbt labs on the various tests and settings….

For a 1GB test set the sectors to : 2048000

Typical benchmarks; (which you will need to add)

128K 100% Read – Sequential

128K 100% Write – Sequential

Remember to bump the workers up to 5 or so. You can set the mixes across all the workers if you select the PC object.

http://ixbtlabs.com/articles/hddide2k1feb/iometer.html