pc engine – pfsense as router / firewall

download (1)Just built a pfsense router on  a pc engine to replace my aging 7390 fritzbox…. Very very awesome and fast!

Perfect for UFB (ultra fast broadband) here in New Zealand – FTTH has arrived :)

 

I’m currently running these services on it (2 x 1GHZ cpu, 4GB RAM, 16GB msata SSD);

  • dhcpd – DHCP server
  • miniupnpd – UPnP server
  • ntpd – NTP server
  • squid – Transparent Proxy & Reverse Proxy
  • snort – IDS (Intrusion Detection System) / IPS (Intrusion Prevention System)
  • ssd – SSH server
  • unbound – DNS Server
  • ipsec – IPsec VPN (site to site VPN)
  • openvpn – Open VPN (client VPN)

Check these links for some great advise…

https://mateh.id.au/2014/09/build-awesome-apu-based-pfsense-router/

http://www.smallnetbuilder.com/other/security/security-howto/31406-build-your-own-ids-firewall-with-pfsense

http://homeservershow.com/building-your-own-super-router-with-pfsense-and-untangle.html

Grab hardware here – http://www.pcengines.ch/apu.htm

IMG_5305

ERROR: No forward-proxy ports configured

I had the above issue after a squid upgrade and after changing from http_port 3128 transparent to http_port 3128 intercept.

Add “http_port 8080” line to squid.config to avoid this message, if you are not already using that port.

The changes in security require that a separate port be setup for forwarding proxy requests

nginx – setup as reverse proxy

 

Previously to take down this wordpress site all you needed to do was hold down F5 for about 20 seconds then the site would take about 5 mins to recover.

There were a few factors causing this and quite a few different methods to solving the problem. WordPress itself is run on php / apache, apache has an evasive mod which can block certain IPs depending on the defined abusive behavior (typical DDOS attack). Since i like to run a few websites behind a single IP i looked at fixing the issue closer to the perimeter….

Enter nginx (engine x) as a reverse proxy, the site now typically caches all content and serves it straight out of memory. No longer does mysql / apache kill itself under high load on the backend…

nginx

You will need to create the nginx directories if they dont already exist. Check /var/log/nginx/error.log (default ubuntu) if any issues starting the service.

sudo aptitude install nginx
sudo service nginx start

The following added to http {}
(located in /etc/nginx/nginx.conf)


log_format cache '***$time_local '
'$remote_addr '
'$upstream_cache_status '
'Cache-Control: $upstream_http_cache_control '
'Expires: $upstream_http_expires '
'"$request" ($status) ';
access_log /var/log/nginx/access.log cache;
error_log /var/log/nginx/error.log;
server_names_hash_bucket_size 64;
proxy_cache_path /var/www/nginx_cache levels=1:2
keys_zone=one:10m
max_size=1g inactive=30m;
proxy_temp_path /var/www/nginx_temp;

the following added to location / {}
(located in /etc/nginx/sites-enabled/default)


proxy_pass http://sigtar;
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
proxy_redirect off;
proxy_buffering on;
proxy_buffer_size 16k;
proxy_buffers 32 16k;
proxy_cache one;
proxy_cache_valid 200 302 304 10m;
proxy_cache_valid 301 1h;
proxy_cache_valid any 1m;
client_body_buffer_size 128k;
proxy_busy_buffers_size 64k;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass_header Set-Cookie;

Note if you have problems with wordpress redirect issues… check this post;

http://tommcfarlin.com/resolving-the-wordpress-multisite-redirect-loop/

Also confirm you have this line in http {}

server_names_hash_bucket_size 64;

Very basic load test, user load time is reasonably consistent as user count increases.
http://loadimpact.com/load-test/sigtar.com-ad07b7870a75c854a935752b0a032c53

load test

squid – reverse proxy

Great guide here – http://www.classhelper.org/articles/reverse-proxy-server-squid-debian/installing-squid-proxy-server.shtml

I’ve modified slightly for Ubuntu, but almost identical.

apt-get install squid
cp /etc/squid3/squid.conf /etc/squid3/squid.bak
rm /etc/squid3/squid.conf
nano /etc/squid/squid.conf

http_port 80 defaultsite=www.yoursite.com vhost
forwarded_for on
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern . 0 20% 4320
cache_peer 192.168.0.13 parent 80 0 no-query no-digest originserver name=web1
acl sites_web1 dstdomain www.yoursite.com yoursite.com
cache_peer_access web1 allow sites_web1
cache_peer 192.168.0.14 parent 80 0 no-query no-digest originserver name=web2
acl sites_web2 dstdomain www.anothersite.com anothersite.com
cache_peer_access web2 allow sites_web2
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 443 # https
acl Safe_ports port 1025-65535 # unregistered ports
acl CONNECT method CONNECT
http_access allow sites_web1
http_access allow sites_web2
http_access allow manager all
http_access allow manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
#http_access deny all
access_log /var/log/squid3/access.log
cache_mgr support@yoursite.com

 

Modify your hosts file to include web1 and web2

nano /etc/hosts

192.168.0.13 web1
192.168.0.14 web2

Another good article about benefits of reverse proxy

http://2bits.com/articles/increasing-drupals-speed-squid-caching-reverse-proxy.html

Squid – optimizing cache hits

The first place to look for increasing your hit ratio is the refresh_pattern parameter within the squid.conf file.

Note: the following applies to squid 3.0 and higher only….

I have found a really good page here http://linux.com/archive/feature/153221 that explains some good setups if you with to increase your cache hits…

refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i \.(gif|png|jpg|jpeg|ico)$ 10080 90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private
refresh_pattern -i \.(iso|avi|wav|mp3|mp4|mpeg|swf|flv|x-flv)$ 43200 90% 432000 override-expire ignore-no-cache ignore-no-store ignore-private
refresh_pattern -i \.(deb|rpm|exe|zip|tar|tgz|ram|rar|bin|ppt|doc|tiff)$ 10080 90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private
refresh_pattern -i \.index.(html|htm)$ 0 40% 10080
refresh_pattern -i \.(html|htm|css|js)$ 1440 40% 40320
refresh_pattern . 0 40% 40320

This is an example of a site that you may wish to heavily cache…

refresh_pattern -i youtube.com/.* 10080 90% 43200

http://wiki.squid-cache.org/ConfigExamples/DynamicContent/YouTube

http://wiki.squid-cache.org/ConfigExamples/DynamicContent/YouTube

# REMOVE these lines from squid.conf

acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY

 

# Break HTTP standard for flash videos. Keep them in cache even if asked not to.
refresh_pattern -i \.flv$ 10080 90% 999999 ignore-no-cache override-expire ignore-private

# Apparently youtube.com use 'Range' requests
# - not seen, but presumably when a video is stopped for a long while then resumed, (or fast-forwarded).
# - convert range requests into a full-file request, so squid can cache it
# NP: BUT slows down their _first_ load time.
quick_abort_min -1 KB

# Also videos are LARGE; make sure you aren't killing them as 'too big to save'
# - squid defaults to 4MB, which is too small for videos and even some sound files
maximum_object_size 4 GB

# Let the clients favorite video site through with full caching
# - they can come from any of a number of youtube.com subdomains.
# - this is NOT ideal, the 'merging' of identical content is really needed here
acl youtube dstdomain .youtube.com
cache allow youtube

# kept to demonstrate that the refresh_patterns involved above go before this.
# You may be missing the CGI pattern, it will need to be added if so.
refresh_pattern -i (/cgi-bin/|\?)   0   0%      0
refresh_pattern .                   0   0%   4320

 


Opensolaris – configuring squid as a transparent proxy

I used to have a virtual machine that acted as a transparent proxy, but as of late thought i’d move squid onto my host box (opensolaris). This way hopefully increasing the amount of free ram available to the host while also taking advantage of ZFS etc…

So this is a very brief guide on how to configure your opensolaris 2008.11 box as a transparent proxy. Essentially meaning if you set the solaris box as the default gateway all web traffic will be cached as it goes through it before it goes onto your dls router or outside connection.

At this point i’m assuming that your opensolaris box can access the web without issue. The config i have loaded on my box points DNS and the default route straight to my dsl router. If you have having problems getting to the web it may be the DNS problem discussed here : http://sigtar.com/2009/02/11/opensolaris-manual-network-config-issue/

First step is to enable IP forwarding;

svcadm enable ipv4-forwarding

Then install the squid package though package manager – search for “squid” and install

Configure squid via /etc/squid/squid.conf file. Find the http_port setting and add “transparent” to the required line like so…

http_port 3128 transparent

Find the network acl and add your subnet to the required line. My subnet is 192.168.4.0/24 so it should look like this. (note: you can have multiple subnets so i have two in the following example). You may need to uncomment the line…

acl our_networks src 192.168.4.0/24 192.168.2.0/24

http_access allow our_networks

Update : i had some problems with web traffic slowing down and stalling eventually over time so i have made an additional update to the /etc/squid/squid.conf file…

httpd_accel_no_pmtu_disc on

Run /usr/squid/sbin/squid -z to create the swap directories.

The default disk cache size in squid as of writing is 100MB, i usually increase mine to 30GB since disk is cheap). I also move my disk cache to a faster disk array and also increase the maximum cached object size to 1000 MB – but probably isn’t required for the small load that i would generate. Here are some of the additional changes i make (optional);

refresh_pattern . 0 80% 10080 reload-into-ims
maximum_object_size 1000 MB

I have made another post that details increasing your hit ratio here : http://sigtar.com/2009/06/10/squid-optimizing-cache-hits/

Run /usr/squid/sbin/squid

Check for any issues in the cache log located at /var/squid/logs/cache.log

You should test your proxy at the default port of 3128 first. i.e. setup the proxy details in Internet Explorer / FireFox to point to your solaris box ipaddress:3128

Next you need to redirect traffic hitting the solaris box on port 80 to 3128 (the transparent port forwarding) like so…

First enable the ipfilter service…    (no iptables in solaris)

svcadm enable ipfilter

create a file called /etc/ipf/ipnat.conf, and place this rule in it;  (this file should be auto loaded at startup)

rdr e1000g1 0.0.0.0/0 port 80 -> 127.0.0.1 port 3128

This redirects my nic (e1000g1) any network port 80 to the local loopback port 3128. To load this rule type;  (clears current rules and loads file)

ipnat -Cf /etc/ipf/ipnat.conf

to check if its working and loaded type ipnat -l

Browse some websites from your web client, and check the squid logs that all http requests are hitting the cache. default log location is /var/squid/logs/access.log

Set your DHCP to point to your solaris box as the default gateway. Done. In most cases your dsl router if it has DHCP will not allow you to change your default gateway. If this is the case you will need to setup your own DHCP server and define the opensolaris box as the default gateway. The DHCP scope can point the clients to the dsl router for DNS, only the default gateway needs to be changed.

Set squid to autostart at boot via SMF manifest;

svcadm enable svc:/network/http:squid

Done

squid – your transparent proxy friend

Still my favourite light and fast web proxy is Squid. Its very easy to setup and get running on almost all flavours of linux. You can find it here http://www.squid-cache.org/ but will probably find it in your local package manager…

This is for more my own reference… as i don’t think too many people have the need for a transparent proxy. You only need one NIC configured on your VM / machine as its most probably on the same subnet as your dsl etc…

1. Setup squid working as a proxy first!

Set the default gateway on the NIC to your router. I also added a default route 0.0.0.0 sub 0.0.0.0 to the router also just in case the default route didn’t work. But seemed to!

Setup DNS! remember you should be able to resolve a name to an IP without issues (can cause most of the problems) Point your default DNS to your router (most have built in DNS forwarders), else you can just use your ISP’s DNS’s

2. Change the squid.config to have the necessary changes to act as a transparent proxy;

nano /etc/squid/squid.conf

http_port 3128 transparent

3. Put this into the startup script; (usually in rc.d under etc) rc.local

iptables -t nat -A PREROUTING -i eth0 -p tcp –dport 80 -j REDIRECT –to-port 3128

This redirects standard tcp requests to 3128 (squid’s port)… transparent to user

4. Ensure that IP forwarding is ENABLED;

change ip_forward from 0 to 1 (usually a txt file somewhere with 0 in it), do a search and you will find it… most probably in the ipv4 directory. This way all 443 (https and other apps) can forward to the web without issue…

echo 1 > /proc/sys/net/ipv4/ip_forward

Should be it…. else good luck!

Update: I have done an updated post about configuring open solaris as a transparent proxy here : http://sigtar.com/2009/04/22/opensolaris-configuring-squid-as-transparent-proxy/