Storage on sigtar.com

unraid smart check - dead WD green drive

Thu, 30 Oct 2014 00:00:00 +1300

errors on unraid GUI - sometimes its a loose cable, sometimes its an issue with the drive.

Run this command to check smart status

smartctl -a -d ata /dev/sda or if you are using a newer SATA controller smartctl -a -A /dev/sda

http://lime-technology.com/wiki/index.php/Troubleshooting

unfortunately in my case, looks like drive is pretty much dead… not too bad for a drive almost 5 years old.

its pretty typical of a WD green drive in its default config to die in this type of environment, no plans to replace it with a similar type drive. You can see below the incredibly high LCC count which indicates the drive header has parked this many times over its life. This is probably part of the problem - there is a tool you can run (check this vid, link for WDIDDLE3 also in comments - http://www.youtube.com/watch?v=J2eYyRI_F98) which disables the intellipark feature of the green drive. I never disabled the park timeout before this drive died (which defaults to 8 seconds!) – note: i have disabled it completely on my other green drives.

windows 2012 - dedup

Wed, 07 Aug 2013 00:00:00 +1200

If you have some archive data that you would like to copy onto a win 2012 server, but dont know how to estimate the end volume size (if VM etc). Start small - about 1/2 of the total capacity of the drive. Robocopy the data onto the drive until nearly full, manually run dedupe, wait until complete, continue robocopy, repeat and rinse. There will be occasions where you will run out of disk space at that point its time to extend your disk.

Equallogic - 6.0 Feature List

Sun, 28 Jul 2013 00:00:00 +1200

http://www.dell.com/learn/us/en/04/campaigns/equallogic-software

Features introduced in release 6.0:

• Synchronous Replication • Snapshot Space Borrowing • Volume Unmap • Volume Undelete • Support for SED drives and IPsec

Features you have come to rely on:

• Data Center Bridging • Automatic load balancing • VMware® Thin Provisioning awareness • Auditing of administrative actions • Microsoft® Active Directory integration • Peer storage architecture • Rapid provisioning • Storage pools • Roles-based administration • Thin clones • SAN Data Copy Offload • SAN Zero Offload and Scalable Lock Management for VMware • Intelligent tiering of data across models that combine solid-state drive (SSD) and hard drives • Integration with the VMware® vStorage APIs • Nondisruptive array upgrades • Nondisruptive array removal • Online data and volume movement • Automatic RAID placement • Thin provisioning • Instant volume restore • Multivolume, writable snapshots • Snapshot and point-in-time replication scheduler • Volume cloning • Volume consistency sets • Auto-start disk monitoring system • Email home • Enclosure monitoring system • Performance monitoring

zfs share for Time machine backups

Thu, 11 Jul 2013 00:00:00 +1200

I’ve tried using a share for time machine before, but due to the nature of how time machine works it eventually consumed all the spare space that is available on that particular share.

bring on zfs quotas… (my drive on my mini mac is 100GB, so 200GB should be enough for a few variations)

zfs set quota=200G unprotected/timemachine

This adds a artificial limit to your zfs filesystem, making sure that time machine does not consume more than 200 gigabytes of space.

Why is UNRAID cool for home?

Sat, 18 May 2013 00:00:00 +1200

I’ve played with many storage technologies at home, ZFS being one of my favs when it comes to performance. But i’ve been looking for something that suits a typical home environment where power usage and capacity is usually more important than performance. Thats where UNRAID has come in…

UNRAID give me these advantages;

Different sized disks in a single pool (only requires largest disk as parity)
Files are distributed over all the disks - so even if you lost more than a single drive you still still have some of your data. Note : with parity drive you can handle a single drive failing without loosing anything, sorta like RAID4 - non distributed parity.
Power usage, since the files are stored on specific disks not all the disks need to power on to give you your file.
Runs on a USB stick - no large operating system required.
Crashplan module can be installed to provide backup options.

http://lime-technology.com/

short stroking - v2

Mon, 28 Nov 2011 00:00:00 +1300

I’ve run through this exercise again - my old findings here

4 x 500 WD Black drives – in one big RAID 0 (128K blocks)

and again only using 500GB of capacity (roughly 1/4 of each of the drives)

P4000 / Lefthand - v9.5 feature list

Mon, 10 Oct 2011 00:00:00 +1300

The 9.5 release is a comprehensive release that includes enhancements in the following areas:

Application managed snapshots for VMware ESX Server and Microsoft Hyper-V Server
SAN Status Home Page — the SAN Status home page is the new home page for the CMC and presents key information for all management groups in the storage environment in a single view.
Zero to VSA automated installation and deployment — Single installer can install VSA, FOM and CMC. Go from “zero” to a fully functional virtual SAN in just a few easy steps. Scriptable install tool allows ROBO customers to script install once and deploy VSA quickly at multiple remote locations.
Peer Motion data mobility with cluster swap — Cluster swap allows users to replace or upgrade P4000 clusters. Edit cluster menu guides decisions about adding, removing, or exchanging storage systems. Easily migrate data online between tiers.
Remote Copy performance improvements — Improvements to remote copy times, the time required to scan volumes, ability to support more frequent snapshots, reduction of RPOs for DR solutions.
Storage Replication Adapter for Vsphere Site Recovery Manager 5.x — Supports automated failback and is included on the HP P4000 Application Integration Solution Pack
Best Practice configuration wizards — Configuration wizards guide towards highest availability configurations: add Failover Manager for 2–node configuration, configure disk RAID 6 for single-node configurations, recommends volume replication when expanding clusters to more than one storage system

allow root ssh login solaris 11 express

Wed, 27 Jul 2011 00:00:00 +1200

Change SSHD to allow root login;

vi /etc/ssh/sshd_config PermitRootLogin = yes

Allow remote logon for root user, comment out the following line;

vi /etc/default/login **

#CONSOLE =/dev/login

rolemod -K type=normal root

solaris - storage server

Mon, 11 Jul 2011 00:00:00 +1200

http://p2v.it/2010/04/15/a-next-generation-multiprotocol-storage-for-your-homelab-on-a-budget-part-1/

Setup iSCSI target using comstar – pkg install -v storage-server pkg install pkg:/SUNWiscsit svcadm enable stmf svcadm enable svc:/network/iscsi/target:default itadm create-target reboot – -r

Create the iSCSI store — zfs create -V 100G DataPool/TestDatastore1 sbdadm create-lu /dev/zvol/rdsk/DataPool/TestDatastore1 stmfadm create-hg ESX4-group stmfadm add-hg-member -g ESX4-group wwn.2100001b329711bd wwn.2101001b32b711bd stmfadm add-view -h ESX4-group -n 0 600144F03EBEC50000004BA86E460001 stmfadm list-lu -v

enable TRIM in windows 7

Sun, 19 Jun 2011 00:00:00 +1200

This command will confirm if TRIM has been auto-enabled by windows 7

fsutil behavior query disabledeletenotify

DisableDeleteNotify = 1 (Windows TRIM commands are disabled) DisableDeleteNotify = 0 (Windows TRIM commands are enabled)

If you would like to force it on;

fsutil behavior set disabledeletenotify 0

email alerts on zpool failure

Thu, 02 Jun 2011 00:00:00 +1200

simple but functional script to check if your zpools are healthy and send email if they are not…

http://blog.qcnetwork.com/2011/03/monitoring-zpool-with-email-alert.html

P4000 / Lefthand and Windows DSM (MPIO)

Wed, 04 May 2011 00:00:00 +1200

Steps to setup MPIO round robin and DSM with P4000 / Lefthand nodes

This example is windows 2008 with 2 x 1gbit nics

Install HP DSM driver onto windows 2008 (this should also install MPIO feature on windows 2008)
Setup 2 x IP’s on windows host - storage subnet (used with MPIO)
1. Confirm you can ping target from both (enable one at a time - ensure there is no routing on your other nics)
Provision storage and allow server read / write access to windows server (via initiator name)
Open iSCSI on windows 2008
Put in IP of target in discovery tab
On first tab confirm that iSCSI drive is presented to host
Click “connect” - check both auto-connect and use MPIO
1. click advanced - chose MS iSCSI initiator, choose first IP and target IP. Click o.k. / o.k
2. click advanced - chose MS iSCSI initiator, choose second IP and target IP. Click o.k. / o.k
3. Repeat above per additional Nic
Confirm via “devices” that there is x (as many targets as nics) targets per disk
1. Within devices choose MPIO - change from vendor specific to “round robin”
  1. Note : read / write access to LUN is required when using “round robin” MPIO opposed to the default “vendor specific” which works with read only access. Else you will get an error - “not supported”
2. Repeat above per “device”
Confirm on Lefthand / P4000 CMC that the LUN has one connection per initiator Nic, and that each connection also has its DSM children (visible in CMC if working).

Confirm that the connection are as expected…

zfs : accidentally adding cache drive to raidz zpool

Wed, 04 May 2011 00:00:00 +1200

http://forums.freebsd.org/showthread.php?t=23127

Unfortunately if you have accidentally added a single drive into your raidz pool at the top-level there is no way to just remove the non redundant disk. Your pool is now dependant on this disk.

If you want your pool to be just raidz vdevs, then you will need to backup your data, destroy your pool, create a new pool, and restore your data.

There is no current way to remove a top-level vdev from a pool.

vmware - finding orphaned files

Fri, 21 Jan 2011 00:00:00 +1300

Use following script for finding orphaned files in your vmware env…

# # Purpose : List all orphaned vmdk on all datastores in all VC's # Version: 1.1 # Author : HJA van Bokhoven # Modifications: LucD

$arrayVC = @(“server”) $report = @()

foreach ($strVC in $arrayVC) { Connect-VIServer $strVC $arrUsedDisks = Get-View -ViewType VirtualMachine | % {$_.Layout} | % {$_.Disk} | % {$_.DiskFile} $arrDS = Get-Datastore | Sort-Object -property Name foreach ($strDatastore in $arrDS) { Write-Host $strDatastore.Name $ds = Get-Datastore -Name $strDatastore.Name | % {Get-View $_.Id} $fileQueryFlags = New-Object VMware.Vim.FileQueryFlags $fileQueryFlags.FileSize = $true $fileQueryFlags.FileType = $true $fileQueryFlags.Modification = $true $searchSpec = New-Object VMware.Vim.HostDatastoreBrowserSearchSpec $searchSpec.details = $fileQueryFlags $searchSpec.matchPattern = "*.vmdk" $searchSpec.sortFoldersFirst = $true $dsBrowser = Get-View $ds.browser $rootPath = "["+$ds.summary.Name+"]"

Vmware & HP MSA

Fri, 21 Jan 2011 00:00:00 +1300

Here is a great guide on multipathing and iSCSI MSA

http://jpaul.me/?tag=vmware-multipathing

Celerra - Unable to create, refresh, or delete checkpoint of a replicated destination file system

Tue, 09 Nov 2010 00:00:00 +1300

I’ve been stuck with this error recently on a Celerra manager device : DpRequest_Max_VS_SuspendedCheckpointsReached

This can sometimes happen when a filesystem is held during a replication task. To check what filesystems are currently being replcated run the following command;

nas_replicate -info -all

Either wait until the replication has completed - check status. Or temporary stop the replication task (stop it from the source side if possible).

EMC Celerra Optimizations for VMware on NFS

Tue, 12 Oct 2010 00:00:00 +1300

go here http://blog.scottlowe.org/2010/01/31/emc-celerra-optimizations-for-vmware-on-nfs/

Turn on the uncached write mechanism for NFS file systems used as VMware datastores. This can have a significant performance improvement for VMDKs on NFS but isn’t the default setting. From the Control Station, you can use this command to turn on the uncached write mechanism: server_mount <data mover name> -option <options>,**uncached** <file system name> <mount point> Be sure to review pages 99 through 101 of the VMware on Celerra best practices document for more information on the uncached write mechanism and any considerations for its use.

vmware and load balancing NFS trunks

Wed, 22 Sep 2010 00:00:00 +1200

Straight from the post below, this is the best way (currently) to load balance your NFS datastores… No MPIO magic here unfortunately.

http://communities.vmware.com/message/1466595#1466595

Basically you can setup IP alias on the NFS side and then setup multiple connections (using a unique IP) per datastore on each ESX host. This works well if you are using a team of nics running IP-hash load balancing…

Static EtherChannel.

My setup is as follows:

ESXi 4.0 U1, Cisco 3750 Switches, and NetApp NFS on the storage side.

I have a total of 8 nics. I divided the nics into 3 groups.

2 nics on vSwitch0 for Mgmt & vMotion 3 nics on vSwitch1 for VM’s (Multiple port groups (3 VLANS)) 3 nics on vSwitch2 for IP Storage (Mostly NFS, a little iSCSI) (One vSwitch3 I also have a VM port group for iSCSI access from within the VM)

Since I have 3 nics on my IPStorage port group I needed a way to be able to utilize all three nics and not have the server just use one for ingress and egress traffic. This was done by:

Setting up a static EtherChannel on the cisco switch (Port Channel). Configuring the cisco switch to IP Hash Configure the vSwitch to “Route based on IP Hash” as well.

The next part is to create multiple datastores on the NFS device. Each of my NFS datastores is about 500GB in size. Reason for this is that my larger luns are iSCSI and are access directly from the VM using the MS iSCSI initiator from the VM itself. My NetApp NAS has an address of, let say, 192.168.1.50. So all my data stores are accessible by utilizing the address of “\\192.168.1.50\NFS-Store#”. This will not be useful as the esx box and the cisco switch will always use the same nic/port to access the nas device. This is due to the algorithm (IP HASH) to decide what link it’ll go over. So to resolve the issue, I added IP aliases to the NFS box. NetApp allows to have multiple Ip addresses pointing to the same NFS export, I suspect EMC would do the same. So, I added 2 aliases 51 & 52. Now my NFS datastores are accessible by using Ip address 192.168.1.50,.51, & .52.

So I went ahead and added the datastores to the ESX box using the multiple IP addresses:

Datastore1 = \\192.168.1.50\NFS-Store1 Datastore2 = \\192.168.1.51\NFS-Store2 Datastore3 = \\192.168.1.52\NFS-Store3

If you have more datastores it’ll just repeat: Datastore4 = \\192.168.1.50\NFS-Store4 and so on…

Since having multiple datastores and address to each, the 3 nics on the ESX box dedicated to IP Storage get utilized. It does not aggregate the bandwidth but it does use all three to send and recieve packets. So the fastest speed you will get is 1Gbit, theoretically, each way for traffic but, it is better than trying to cram all the traffic over 1 nic.

I also enabled Jumbo Frames on the vSwitch as well as the vmNic for IP-Stroage. (need the best performance!) I should mention that your NFS storage device should have EtherChannel setup on it as well. Otherwise, you’ll be on the same boat just on the other end of it.

Hope it helps!

Larry B.

I should mention that you should not use different addresses to access the same NFS share (datastore). It is not supported and may cause you issues.

vmware srm - replicating over a wan - optimizations

Tue, 17 Aug 2010 00:00:00 +1200

I’ve been working through a SRM setup and have been looking at ways to optimize the amount of traffic that is sent over the WAN. The first obvious move is to move your vmware swap files off the replicated LUNS.

Another way is to reduce the sync window. i.e. how often is yoru replication technology trying to keep the source and destination in sync? – Increasing this window can sometimes help you out. But that all depends on your delta’s.

4k sector hard drives and zfs

Sun, 06 Jun 2010 00:00:00 +1200

I hit this as a problem recently. One of my disks died in my raidz so i ran down to the store and grabbed me a replacement WD10EARS (Western Digital 1Tb Green) drive.

BUT…

The one thing the store didn’t mention to me is the new 4K cluster sizing on the drive. I guess they assume most people run windows (though the issues are also present in XP). See these posts…

opensolaris v134 - CIFS has gone walkies

Sun, 11 Apr 2010 00:00:00 +1200

Errr, I cant find the CIFS service in the 134 build.

I know it was renamed to as per below but still cant see it anywhere?

> system/file-system/smb (was SUNWsmbfs*) > service/file-system/smb (was SUNWsmbs*)

Anyone have any ideas?

pkg search *smb gets me these… but i cant install them.

require depend service/file-system/smb@0.5.11-0.134 pkg:/redistributable@0.1-0.134 require depend service/file-system/smb@0.5.11-0.134 pkg:/storage/storage-server@0.1-0.134 require depend service/file-system/smb@0.5.11-0.134 pkg:/system/security/kerberos-5@0.5.11-0.134 require depend service/file-system/smb@0.5.11-0.134 pkg:/storage/storage-nas@0.1-0.134 require depend system/file-system/smb@0.5.11-0.134 pkg:/redistributable@0.1-0.134 require depend system/file-system/smb@0.5.11-0.134 pkg:/storage/storage-server@0.1-0.134 require depend system/file-system/smb@0.5.11-0.134 pkg:/slim_install@0.1-0.134

zfs - now has dedup!

Mon, 05 Apr 2010 00:00:00 +1200

Cool. zfs as of version 21 has deduplication built in. And thats the good dedup - synchronous dedup. i.e. deduped on the fly!

How easy is it to turn on? - very!

Once you have upgraded your zpool to 21 or above you can run the following command at the pool level and deduplication will be over all your data from that point onwards.

zpool set dedup=on tank

Done

Note : Watch your performance, it will drop like a rock if you do not have enough ram for your dedup tables. Do some tests after enabling this feature.

iometer testing

Fri, 26 Mar 2010 00:00:00 +1300

The following is some notes i have around the parameters i usually use during testing. There is a good document at ixbt labs on the various tests and settings….

For a 1GB test set the sectors to : 2048000

Typical benchmarks; (which you will need to add)

128K 100% Read - Sequential

128K 100% Write - Sequential

Remember to bump the workers up to 5 or so. You can set the mixes across all the workers if you select the PC object.

Jumbo Frames on your vSphere ESXi box

Tue, 02 Mar 2010 00:00:00 +1300

Continuing on from this /2010/02/04/vsphere-and-multipathing-iscsi/

you may want to implement jumbo frames to your iSCSI backend…

Enable jumbo frames on your iSCSI target and switches then complete the following on the ESXi hosts (iSCSI initiators)…

This to list your current switch details and port group names…

esxcfg-vswitch -l

The following to allow jumbo frames onto your vswtich (insert your vSwich in following)

esxcfg-vswtich -m 9000 vSwitch0

Then create your kernel port groups that you want to use for iscsi, my commands looked like this… (repeat for each iscsi kernel port you have) – note : DO NOT ADD ANY VMKERNEL(s) from the GUI, instead use the steps below

vSphere and Multipathing iSCSI

Thu, 04 Feb 2010 00:00:00 +1300

This is just a quick reference to create a multiplathing iSCSI setup…

Create two virtual kernel switches, one called “iSCSI-1” and the other called “iSCSI-2” (and so on if you have more nics)

Then per kernel portgroup ensure that only one of the nics is active. For the “iSCSI-1” portgroup configure it to override the virtual switch settings and move nic 0 to active and nic 1 to unused. For the “iSCSI-2” portgroup configure it to override the virtual switch settings and move nic 1 to active and nic 0 to unused.

HDD short stroking - is it worth it?

Mon, 21 Dec 2009 00:00:00 +1300

I’ve got some old 250Gb drives that are starting to show their age. I’ve currently got them setup in a 3x RAID 0 config which presents about 750Gb of space.

I’ve got everything on a single partition (meh, i’m lazy). I’ve done various speed tests in the current setup (with all space allocated), but i thought i’d re-image onto a short stroke partition.

I only use about 150Gb of space on my main machine (most of my data is on another box), so i’m going to try creating a 200Gb partition to test if this provides any kind of performance boost.

OpenSolaris - iSCSI

Fri, 13 Nov 2009 00:00:00 +1300

Want iSCSI in opensolaris?

Grab SUNWiscsitgt via package manager.

enable the server via svcadm;

svcadm enable iscsitgt

create your zfs iscsi pool; (this command will limit iscsi drive to 500GB in size)

zfs create -V 500G tank/iscsi

set isci on via zfs command;

zfs set shareiscsi=on tank/iscsi

check that target is up and running;

iscsitadm list target -v

Done. Should be able to connect via ip from another machine. I have not covered CHAP or any client side configuration. Assumed isolated LAN.

FreeNAS - zfs in version 0.7

Wed, 28 Oct 2009 00:00:00 +1300

If you want to give zfs a go and also want a dedicated file server this is the solution.

Check it out here;

http://www.freenas.org/

I’m still using opensolaris though as i like running a few virtualbox machines on the same box.

Opensolaris - ZFS recovery after kernel panic

Mon, 19 Oct 2009 00:00:00 +1300

Recently i hit what i thought was a huge disaster with my ZFS array. Essentially i was unable to import my zpool without causing the kernel to panic and reboot. Still unsure of the exact reason, but it didn’t seem to be due to a hardware fault. (zpool import showed all disks as ONLINE)

When i tried to import with zpool import -f tank the machine would lockup and reboot (panic).

zfs compression and latency

Wed, 19 Aug 2009 00:00:00 +1200

Since im using ZFS as storage via NFS for my some of my vmware environments i need to ensure that latency on my disk is reduced where ever possible.

There is alot of talk about ZFS compression being “faster” than a non-compressed pool due to less physical data being pulled off the drives. This of course depends on the system powering ZFS, but i wanted to run some tests specifically on latency. Throughput is fine in some situations, but latency is a killer when it comes to lots of small reads and writes (in the case of hosting virtual machines)

opensolaris - smbd issues?

Mon, 27 Jul 2009 00:00:00 +1200

Hmm… i’ve been having problems since the 2009.06 (snv_111b) update with cifs.

Cant pin it exactly as it could be “load” related… hmmm.

found this ? http://opensolaris.org/jive/thread.jspa?threadID=107681 this also may be a clue.. http://opensolaris.org/jive/thread.jspa?threadID=92472&tstart=75

imapd ? might have to go back to 2008.11

You might get better performance if you enable oplocks but there are known issues with it but you can do it just to see if you see any difference:

svccfg -s smb/server setprop smbd/oplock_enable=boolean: true

Opensolaris : Citrix XenServer / ESX - Hooking into ZFS

Wed, 22 Jul 2009 00:00:00 +1200

To share your zfs pool via NFS (that works with Citrix Xen / ESX) to a host called “esxhost”;

zfs set sharenfs=rw,nosuid,root=esxhost tank/nfs

Note : You MUST have a resolvable name from the opensolaris box. i.e. you should be able to ping it. I have tried with ip’s only and it will fail. I have edited the /etc/hosts file to include the following line for my config;

opensolaris / zfs - whitebox build

Sun, 19 Jul 2009 00:00:00 +1200

I’ve built a little server for home use, but it pales in comparison to this beast… This type of setup would be perfect for a lab / test environment that requires lots of fast and reliable disk. SCSI drives are fading out, SATA can perform if its setup right. When you look at the price of the entire build you wonder why corporations continue to spend the big bucks on the big storage names.

zfs - playing with various configs

Sat, 18 Jul 2009 00:00:00 +1200

If you dont have the disks available to build up a zpool and have a play with zfs you can actually just use files created with the mkfile command… The commands are exactly the same.

mkfile 64m disk1

mkfile 64m disk2

mkfile 64m disk3

mkfile 10m disk4

mkfile 100m disk5

mkfile 100m disk6

Now you can create a zpool using the above files… (i’m using raidz for this setup)

zpool create test raidz /fullpath/disk1 /fullpath/disk2 /fullpath/disk3