Daz – Page 36 – sigtar.com

Windows / Linux – File Migration Tools

Posted on February 21, 2009 by Daz — No Comments ↓

In Windows i use robocopy…

This is the syntax to copy from one location to another… (i like how robocopy uses UNC paths)

robocopy \\192.168.9.70\videos \\192.168.9.101\videos /e /r:2 /w:10 /sec
robocopy \\192.168.9.70\software \\192.168.9.101\software /e /r:2 /w:10 /sec

Note: i always copy instead of the move. Run multiple times before switch over to shorten time for final sync. Always use the /SEC to copy over the NTFS permissions. I have had problems with the non X010 version of robocopy not copying NTFS permissions across. Check NTFS permissions before removing the source files. You can update files NTFS permissions with /SECFIX switch if required.

/e moves all directories including empty ones.

In Linux i use rsync…

i typically mount a smb share on a windows host…. (you must have cifs client on linux first)

mount -t cifs -n //192.168.9.70/backup /mnt/backup -o username=user,password=mypassword

then use rsync for the copy work…

rsync -rcav /etc/ /mnt/backup/etc/

if your after linux to linux copying then rsync is also the best way to do it (via ssh), logon to your destination box via ssh and run this from your distination diretory..

rsync -ave ssh 192.168.9.10:/var/lib/mysql .

Linux – routing examples

Posted on February 20, 2009 by Daz — No Comments ↓

set ip_forward to 1 to enable routing between nics.
type ‘route‘ to insure both networks are routed as expected (connected route added with nic)

Setup a script with the following (CHMOD 0755 the script so it executes)

iptables -F (clears previous iptables stuff)
iptables -P INPUT DROP (will set default policy to DROP all INPUT packets – Incoming to local NICs)
iptables -P OUTPUT DROP (will set default policy to DROP all OUTPUT packets – Outgoing to local NICs)
iptables -P FORWARD DROP (will set default policy to DROP all FORWARDed packets – Routed via local NICs)

allow rules;
iptables -A FORWARD -s 172.23.23.1 -d 172.23.23.2 -p tcp –dport 80 -j ACCEPT (append forward rules – self explanatory)

Add more as required.

Want to use NAT outbound?

wlan0 => external network – route to Internet
eth0 => internal network

#setup masquerading
iptables -P INPUT ACCEPT
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -A FORWARD -m state –state ESTABLISHED,RELATED -j ACCEPT
iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE

A very handy tool and the current best way i’ve found to test firewall rules is to fire up tcpdump. Check man for details but here is an example of monitoring a specific destination port and host on a specific NIC;

tcpdump -i eth1 dst port 22 and dst host 192.168.4.34

If you can see the traffic leaving the interface outbound it has made it past the firewall, if not then you need to have a look at your firewall rules or check your routing table. Type route to see current routing table

Done

Windows – Port forwarding

Posted on February 19, 2009 by Daz — No Comments ↓

Just wanted to note here that there is a tool for windows to ‘port forward’ Its a command MS ships that enables IPv6 to work on IPv4, but can be used as a port redirector…. so as long as you have IPv6 support on your box, you can do this.

syntax as follows;

netsh interface portproxy add v4tov4 listenport=443 connectaddress=targetmachine connectport=3389

so the machine above listens on 443 and redirects traffic to 3389 on targetmachine.

Be warned though, NETSH is insanely powerful. And if you make a mistake good luck at cleaning up the mess…. Feeling confident now?

zfs – checking your zpool throughput

Posted on February 18, 2009 by Daz — No Comments ↓

This is quite a good diagnostic for checking your disk throughput. Try copying data to and from your zpool while your running this command on the host…

zpool iostat -v unprotected 2

capacity     operations    bandwidth
pool         used  avail   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
unprotected  1.39T   668G     18      7  1.35M   161K
c7d0       696G   403M      1      2  55.1K  21.3K
c9d0       584G   112G      8      2   631K  69.3K
c7d1       141G   555G      8      2   697K  70.0K
----------  -----  -----  -----  -----  -----  -----

The above command will keep displaying the above output every 2 seconds (average during that time). I’ve used it a few times to ensure that all disks are being used (in write operations) where needed. Of course read op’s may not be typically across all disks as it will depend where the data is…

As you can see in the output from my “unprotected” zpool, my disk “c7d0” is near full so less write operations will be on this disk. In my scenario most of my reads also come from this disk, this was due me copying most of the data into this zpool when there was only this single disk.

I’ve heard rumor of a zfs feature in future that will re-balance the data across all the disks (unsure if its live or on a set schedule)

Another way to show some disk throughput figures is to run the iostat command like so…

iostat -exn 10

extended device statistics

device r/s w/s kr/s kw/s wait actv svc_t %w %b

cmdk17 1.0 0.0 71.5 0.0 0.0 0.0 10.9 0 1

cmdk18 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0

cmdk19 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0

cmdk20 0.8 0.0 33.5 0.0 0.0 0.0 13.5 0 1

cmdk21 0.4 0.0 0.5 0.0 0.0 0.0 15.5 0 1

cmdk22 0.8 0.0 66.3 0.0 0.0 0.0 9.0 0 1

cmdk23 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0

cmdk24 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0

extended device statistics —- errors —

                     extended device statistics       ---- errors --- 


r/s    w/s   kr/s   kw/s wait actv wsvc_t asvc_t  %w  %b s/w h/w trn tot device
0.0    0.0    0.0    0.0  0.0  0.0    0.0    0.0   0   0   0   0   0   0 c11d1
0.0    7.7    0.0   25.8  0.0  0.0    2.3    4.9   0   3   0   0   0   0 c8d0
0.0   17.6    0.0  238.0  0.0  0.0    0.0    0.3   0   0   0   0   0   0 c9d0
0.0    1.0    0.0    0.8  0.0  0.0    0.0    0.3   0   0   0   0   0   0 c7t0d0
0.0    1.0    0.0    0.8  0.0  0.0    0.0    0.2   0   0   0   0   0   0 c7t2d0
0.0    1.0    0.0    0.8  0.0  0.0    0.0    0.3   0   0   0   0   0   0 c7t3d0
0.7   21.1   29.9  315.0  0.0  0.0    0.0    1.1   0   1   0   0   0   0 c7t4d0
0.7   20.9   29.8  314.9  0.0  0.0    0.0    1.7   0   2   0   0   0   0 c7t5d0
0.8   21.0   34.1  315.0  0.0  0.0    0.0    1.2   0   1   0   0   0   0 c7t6d0
0.5   20.8   21.3  314.8  0.0  0.0    0.0    1.1   0   1   0   0   0   0 c7t7d0

This should show you all your disks and update on a 5 second interval. Copying data back and forth to your drives will show various stats.

OpenSolaris cifs/smb server – configuring ACL’s on shares – Part 2

Posted on February 15, 2009 by Daz — No Comments ↓

I have done most of the explaining in another post on how the permissions work with a zfs cifs share – link here.

In this post I’m going to show an example of how to add multiple groups / users to a specific folder. Which then recursively adds its ACL”s to any child objects. Its essentially like the other commands i used, but with an extra line per user or group you want to configure;

chmod -R A=\

owner@:full_set:d:allow,\

owner@:full_set:f:allow,\

user:lisa:full_set:d:allow,\

user:lisa:full_set:f:allow,\

everyone@:rxaARWcs:d:allow,\

everyone@:raARWcs:f:allow \

/protected/

The above command gives the owner and the user “lisa” full permissions, the everybody else group has read only access. If you want to add a group then just use group: instead of user: at the start.

This gives full access to myself and list, but deny’s access to everyone else;

chmod -R A=\

owner@:full_set:d:allow,\

owner@:full_set:f:allow,\

user:daz:full_set:d:allow,\

user:daz:full_set:f:allow,\

user:lisa:full_set:d:allow,\

user:lisa:full_set:f:allow,\

everyone@:full_set:d:deny,\

everyone@:full_set:f:deny \

/protected/photos/

Done.