Squid

Just built a pfsense router on  a pc engine to replace my aging 7390 fritzbox…. Very very awesome and fast!

Perfect for UFB (ultra fast broadband) here in New Zealand - FTTH has arrived :)

I’m currently running these services on it (2 x 1GHZ cpu, 4GB RAM, 16GB msata SSD);

• dhcpd - DHCP server
• miniupnpd - UPnP server
• ntpd - NTP server
• squid - Transparent Proxy & Reverse Proxy
• snort - IDS (Intrusion Detection System) / IPS (Intrusion Prevention System)
• ssd - SSH server
• unbound - DNS Server
• ipsec - IPsec VPN (site to site VPN)
• openvpn - Open VPN (client VPN)

Check these links for some great advise…

I had the above issue after a squid upgrade and after changing from http_port 3128 transparent to http_port 3128 intercept.

Add “http_port 8080” line to squid.config to avoid this message, if you are not already using that port.

The changes in security require that a separate port be setup for forwarding proxy requests

Previously to take down this wordpress site all you needed to do was hold down F5 for about 20 seconds then the site would take about 5 mins to recover.

There were a few factors causing this and quite a few different methods to solving the problem. Wordpress itself is run on php / apache, apache has an evasive mod which can block certain IPs depending on the defined abusive behavior (typical DDOS attack). Since i like to run a few websites behind a single IP i looked at fixing the issue closer to the perimeter….

Great guide here - http://www.classhelper.org/articles/reverse-proxy-server-squid-debian/installing-squid-proxy-server.shtml

I’ve modified slightly for Ubuntu, but almost identical.

apt-get install squid cp /etc/squid3/squid.conf /etc/squid3/squid.bak rm /etc/squid3/squid.conf nano /etc/squid/squid.conf

http_port 80 defaultsite=www.yoursite.com vhost forwarded_for on refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern . 0 20% 4320 cache_peer 192.168.0.13 parent 80 0 no-query no-digest originserver name=web1 acl sites_web1 dstdomain www.yoursite.com yoursite.com cache_peer_access web1 allow sites_web1 cache_peer 192.168.0.14 parent 80 0 no-query no-digest originserver name=web2 acl sites_web2 dstdomain www.anothersite.com anothersite.com cache_peer_access web2 allow sites_web2 acl all src 0.0.0.0/0.0.0.0 acl manager proto cache_object acl localhost src 127.0.0.1/255.255.255.255 acl to_localhost dst 127.0.0.0/8 acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 443 # https acl Safe_ports port 1025-65535 # unregistered ports acl CONNECT method CONNECT http_access allow sites_web1 http_access allow sites_web2 http_access allow manager all http_access allow manager http_access deny !Safe_ports http_access deny CONNECT !SSL_ports #http_access deny all access_log /var/log/squid3/access.log cache_mgr [email protected]

I’m using the following crontab script to renice my squid process….

Put following in script and  call via cron (i’m doing every 10 mins)

*/10 * * * * /home/user/renice.sh

(renice.sh)

renice -15 $(ps -C squid3 -o pid=) renice -15 $(ps -C unlinkd -o pid=)

The first place to look for increasing your hit ratio is the refresh_pattern parameter within the squid.conf file.

Note: the following applies to squid 3.0 and higher only….

I have found a really good page here http://linux.com/archive/feature/153221 that explains some good setups if you with to increase your cache hits…

refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i \.(gif|png|jpg|jpeg|ico)$ 10080 90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private refresh_pattern -i \.(iso|avi|wav|mp3|mp4|mpeg|swf|flv|x-flv)$ 43200 90% 432000 override-expire ignore-no-cache ignore-no-store ignore-private refresh_pattern -i \.(deb|rpm|exe|zip|tar|tgz|ram|rar|bin|ppt|doc|tiff)$ 10080 90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private refresh_pattern -i \.index.(html|htm)$ 0 40% 10080 refresh_pattern -i \.(html|htm|css|js)$ 1440 40% 40320 refresh_pattern . 0 40% 40320

I used to have a virtual machine that acted as a transparent proxy, but as of late thought i’d move squid onto my host box (opensolaris). This way hopefully increasing the amount of free ram available to the host while also taking advantage of ZFS etc…

So this is a very brief guide on how to configure your opensolaris 2008.11 box as a transparent proxy. Essentially meaning if you set the solaris box as the default gateway all web traffic will be cached as it goes through it before it goes onto your dls router or outside connection.

Still my favourite light and fast web proxy is Squid. Its very easy to setup and get running on almost all flavours of linux. You can find it here http://www.squid-cache.org/ but will probably find it in your local package manager…

This is for more my own reference… as i don’t think too many people have the need for a transparent proxy. You only need one NIC configured on your VM / machine as its most probably on the same subnet as your dsl etc…