Opensolaris : SMF and Squid

Decided to create a SMF manifest for squid so that it autostarts on boot.

Command to start squid : /usr/squid/sbin/squid 

Command to stop squid : /usr/squid/sbin/squid -k shutdown 

This site has a good starter on SMF creation : http://blogs.sun.com/observatory/entry/mouting_windows_shares_at_startup

I’ve taken a copy of the /var/svc/manifest/network/smb/server.xml file for my squid manifest and placed it at /var/svc/manifest/network/squid/squid.xml

Import the XML using this command; 

svccfg import /var/svc/manifest/network/squid/squid.xml 

Update: i’ve stopped this work as i missed the fact that squid already comes with a SMF manifest! :)  No idea how i missed that one. You can enable via ; 

svcadm enable svc:/network/http:squid

This is the manifest for the above service; 

Continue reading Opensolaris : SMF and Squid

Opensolaris – configuring squid as a transparent proxy

I used to have a virtual machine that acted as a transparent proxy, but as of late thought i’d move squid onto my host box (opensolaris). This way hopefully increasing the amount of free ram available to the host while also taking advantage of ZFS etc…

So this is a very brief guide on how to configure your opensolaris 2008.11 box as a transparent proxy. Essentially meaning if you set the solaris box as the default gateway all web traffic will be cached as it goes through it before it goes onto your dls router or outside connection.

At this point i’m assuming that your opensolaris box can access the web without issue. The config i have loaded on my box points DNS and the default route straight to my dsl router. If you have having problems getting to the web it may be the DNS problem discussed here : https://sigtar.com/2009/02/11/opensolaris-manual-network-config-issue/

First step is to enable IP forwarding;

svcadm enable ipv4-forwarding

Then install the squid package though package manager – search for “squid” and install

Configure squid via /etc/squid/squid.conf file. Find the http_port setting and add “transparent” to the required line like so…

http_port 3128 transparent

Find the network acl and add your subnet to the required line. My subnet is 192.168.4.0/24 so it should look like this. (note: you can have multiple subnets so i have two in the following example). You may need to uncomment the line…

acl our_networks src 192.168.4.0/24 192.168.2.0/24

http_access allow our_networks

Update : i had some problems with web traffic slowing down and stalling eventually over time so i have made an additional update to the /etc/squid/squid.conf file…

httpd_accel_no_pmtu_disc on

Run /usr/squid/sbin/squid -z to create the swap directories.

The default disk cache size in squid as of writing is 100MB, i usually increase mine to 30GB since disk is cheap). I also move my disk cache to a faster disk array and also increase the maximum cached object size to 1000 MB – but probably isn’t required for the small load that i would generate. Here are some of the additional changes i make (optional);

refresh_pattern . 0 80% 10080 reload-into-ims
maximum_object_size 1000 MB

I have made another post that details increasing your hit ratio here : https://sigtar.com/2009/06/10/squid-optimizing-cache-hits/

Run /usr/squid/sbin/squid

Check for any issues in the cache log located at /var/squid/logs/cache.log

You should test your proxy at the default port of 3128 first. i.e. setup the proxy details in Internet Explorer / FireFox to point to your solaris box ipaddress:3128

Next you need to redirect traffic hitting the solaris box on port 80 to 3128 (the transparent port forwarding) like so…

First enable the ipfilter service…    (no iptables in solaris)

svcadm enable ipfilter

create a file called /etc/ipf/ipnat.conf, and place this rule in it;  (this file should be auto loaded at startup)

rdr e1000g1 0.0.0.0/0 port 80 -> 127.0.0.1 port 3128

This redirects my nic (e1000g1) any network port 80 to the local loopback port 3128. To load this rule type;  (clears current rules and loads file)

ipnat -Cf /etc/ipf/ipnat.conf

to check if its working and loaded type ipnat -l

Browse some websites from your web client, and check the squid logs that all http requests are hitting the cache. default log location is /var/squid/logs/access.log

Set your DHCP to point to your solaris box as the default gateway. Done. In most cases your dsl router if it has DHCP will not allow you to change your default gateway. If this is the case you will need to setup your own DHCP server and define the opensolaris box as the default gateway. The DHCP scope can point the clients to the dsl router for DNS, only the default gateway needs to be changed.

Set squid to autostart at boot via SMF manifest;

svcadm enable svc:/network/http:squid

Done

squid – your transparent proxy friend

Still my favourite light and fast web proxy is Squid. Its very easy to setup and get running on almost all flavours of linux. You can find it here http://www.squid-cache.org/ but will probably find it in your local package manager…

This is for more my own reference… as i don’t think too many people have the need for a transparent proxy. You only need one NIC configured on your VM / machine as its most probably on the same subnet as your dsl etc…

1. Setup squid working as a proxy first!

Set the default gateway on the NIC to your router. I also added a default route 0.0.0.0 sub 0.0.0.0 to the router also just in case the default route didn’t work. But seemed to!

Setup DNS! remember you should be able to resolve a name to an IP without issues (can cause most of the problems) Point your default DNS to your router (most have built in DNS forwarders), else you can just use your ISP’s DNS’s

2. Change the squid.config to have the necessary changes to act as a transparent proxy;

nano /etc/squid/squid.conf

http_port 3128 transparent

3. Put this into the startup script; (usually in rc.d under etc) rc.local

iptables -t nat -A PREROUTING -i eth0 -p tcp –dport 80 -j REDIRECT –to-port 3128

This redirects standard tcp requests to 3128 (squid’s port)… transparent to user

4. Ensure that IP forwarding is ENABLED;

change ip_forward from 0 to 1 (usually a txt file somewhere with 0 in it), do a search and you will find it… most probably in the ipv4 directory. This way all 443 (https and other apps) can forward to the web without issue…

echo 1 > /proc/sys/net/ipv4/ip_forward

Should be it…. else good luck!

Update: I have done an updated post about configuring open solaris as a transparent proxy here : https://sigtar.com/2009/04/22/opensolaris-configuring-squid-as-transparent-proxy/

Router on a stick – Adding VLANs to a cisco router

Telnet to your cisco router, enable then go into config t. Passwords may be required depending on how it has been configured…

en

config t

Ensure that ip routing has been enabled…

ip routing

To create the VLAN adapter you’ll need to run the following commands to create a “router on a stick”…   Note: the .2 after the interface name creates a virtual adapter on the interface.

interface FastEthernet1/1.2
description “VM Host Network”
encapsulation dot1Q 2
ip address 172.25.2.1 255.255.255.0

The following is another example of a VLAN (110) on a router. This one has the ip helper-address (DHCP server) specified for this range.

interface FastEthernet1/1.110
description “Ak IDN”
encapsulation dot1Q 110
ip address 172.25.20.1 255.255.255.0
ip helper-address 172.25.20.15

That is the router side of things setup. Remember to save your changes via…

copy run start

Now you’ll need to add the VLAN to your switches. If you have VTP in your environment its as simple as adding the extra vlan onto your “server” vtp switch. Telnet (or null cable) to your “server” switch and go into “enable” mode, then go into the vlan database and add the required vlans as per above…

en

vlan database

vlan 2 name “VM Host Network”

vlan 110 name “Ak IDN”

VTP will propagate the vlans out to your client switches. Done.

Windows / Linux – File Migration Tools

In Windows i use robocopy

This is the syntax to copy from one location to another…  (i like how robocopy uses UNC paths)

robocopy \\192.168.9.70\videos \\192.168.9.101\videos /e /r:2 /w:10 /sec
robocopy \\192.168.9.70\software \\192.168.9.101\software /e /r:2 /w:10 /sec

Note: i always copy instead of the move. Run multiple times before switch over to shorten time for final sync. Always use the /SEC to copy over the NTFS permissions. I have had problems with the non X010 version of robocopy not copying NTFS permissions across. Check NTFS permissions before removing the source files. You can update files NTFS permissions with /SECFIX switch if required.

/e moves all directories including empty ones.

In Linux i use rsync

i typically mount a smb share on a windows host…. (you must have cifs client on linux first)

mount -t cifs -n //192.168.9.70/backup /mnt/backup -o username=user,password=mypassword

then use rsync for the copy work…

rsync -rcav /etc/ /mnt/backup/etc/

if your after linux to linux copying then rsync is also the best way to do it (via ssh), logon to your destination box via ssh and run this from your distination diretory..

rsync -ave ssh 192.168.9.10:/var/lib/mysql .