pc engine – pfsense as router / firewall

download (1)Just built a pfsense router on  a pc engine to replace my aging 7390 fritzbox…. Very very awesome and fast!

Perfect for UFB (ultra fast broadband) here in New Zealand – FTTH has arrived :)

 

I’m currently running these services on it (2 x 1GHZ cpu, 4GB RAM, 16GB msata SSD);

  • dhcpd – DHCP server
  • miniupnpd – UPnP server
  • ntpd – NTP server
  • squid – Transparent Proxy & Reverse Proxy
  • snort – IDS (Intrusion Detection System) / IPS (Intrusion Prevention System)
  • ssd – SSH server
  • unbound – DNS Server
  • ipsec – IPsec VPN (site to site VPN)
  • openvpn – Open VPN (client VPN)

Check these links for some great advise…

https://mateh.id.au/2014/09/build-awesome-apu-based-pfsense-router/

http://www.smallnetbuilder.com/other/security/security-howto/31406-build-your-own-ids-firewall-with-pfsense

http://homeservershow.com/building-your-own-super-router-with-pfsense-and-untangle.html

Grab hardware here – http://www.pcengines.ch/apu.htm

IMG_5305

ERROR: No forward-proxy ports configured

I had the above issue after a squid upgrade and after changing from http_port 3128 transparent to http_port 3128 intercept.

Add “http_port 8080” line to squid.config to avoid this message, if you are not already using that port.

The changes in security require that a separate port be setup for forwarding proxy requests

nginx – and gzip

i only have vdsl at home and host my website via it… the following graphs shows when nginx started serving faster than the vdsl connection could handle (which is about 9Mbits)

ngnix with cache enabled, gzip enabled and set to 6

http://loadimpact.com/load-test/sigtar.com-6a246200e28f354c64582cf2c0b6ce46

ngnix with cache enabled, gzip enabled and set to 9 (maximum)

http://loadimpact.com/load-test/sigtar.com-1422b3e7250c4e3acb7868177f2b2056

ngnix with cache disabled, gzip enabled and set to 9 (maximum)

http://loadimpact.com/load-test/sigtar.com-19561e7422e24f35fb557c662d20beac

 

nginx – setup as reverse proxy

 

Previously to take down this wordpress site all you needed to do was hold down F5 for about 20 seconds then the site would take about 5 mins to recover.

There were a few factors causing this and quite a few different methods to solving the problem. WordPress itself is run on php / apache, apache has an evasive mod which can block certain IPs depending on the defined abusive behavior (typical DDOS attack). Since i like to run a few websites behind a single IP i looked at fixing the issue closer to the perimeter….

Enter nginx (engine x) as a reverse proxy, the site now typically caches all content and serves it straight out of memory. No longer does mysql / apache kill itself under high load on the backend…

nginx

You will need to create the nginx directories if they dont already exist. Check /var/log/nginx/error.log (default ubuntu) if any issues starting the service.

sudo aptitude install nginx
sudo service nginx start

The following added to http {}
(located in /etc/nginx/nginx.conf)


log_format cache '***$time_local '
'$remote_addr '
'$upstream_cache_status '
'Cache-Control: $upstream_http_cache_control '
'Expires: $upstream_http_expires '
'"$request" ($status) ';
access_log /var/log/nginx/access.log cache;
error_log /var/log/nginx/error.log;
server_names_hash_bucket_size 64;
proxy_cache_path /var/www/nginx_cache levels=1:2
keys_zone=one:10m
max_size=1g inactive=30m;
proxy_temp_path /var/www/nginx_temp;

the following added to location / {}
(located in /etc/nginx/sites-enabled/default)


proxy_pass http://sigtar;
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
proxy_redirect off;
proxy_buffering on;
proxy_buffer_size 16k;
proxy_buffers 32 16k;
proxy_cache one;
proxy_cache_valid 200 302 304 10m;
proxy_cache_valid 301 1h;
proxy_cache_valid any 1m;
client_body_buffer_size 128k;
proxy_busy_buffers_size 64k;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass_header Set-Cookie;

Note if you have problems with wordpress redirect issues… check this post;

http://tommcfarlin.com/resolving-the-wordpress-multisite-redirect-loop/

Also confirm you have this line in http {}

server_names_hash_bucket_size 64;

Very basic load test, user load time is reasonably consistent as user count increases.
http://loadimpact.com/load-test/sigtar.com-ad07b7870a75c854a935752b0a032c53

load test

use nmap to port scan / find free IPS

I use angry IP scanner in windows, linux has a good util for port scanning an ip range called nmap;

Grab nmap from usual repositories

sudo nmap -sP 192.168.9.0/24 (will show hosts up and resolve mac addresses to vendors)

sudo nmap -v -sT 192.168.9.0/24 (will show hosts up and the various open ports)

sudo nmap -v -sT 192.168.9.104 (will show particular host and the various open ports)

Starting Nmap 6.00 ( http://nmap.org ) at 2013-07-12 11:48 NZST
Initiating ARP Ping Scan at 11:48
Scanning 192.168.9.104 [1 port]
Completed ARP Ping Scan at 11:48, 0.00s elapsed (1 total hosts)
Initiating Parallel DNS resolution of 1 host. at 11:48
Completed Parallel DNS resolution of 1 host. at 11:48, 0.00s elapsed
Initiating Connect Scan at 11:48
Scanning win2012-veeam.fritz.box (192.168.9.104) [1000 ports]
Discovered open port 111/tcp on 192.168.9.104
Discovered open port 445/tcp on 192.168.9.104
Discovered open port 3389/tcp on 192.168.9.104
Discovered open port 135/tcp on 192.168.9.104
Discovered open port 139/tcp on 192.168.9.104
Discovered open port 49155/tcp on 192.168.9.104
Discovered open port 1063/tcp on 192.168.9.104
Discovered open port 2049/tcp on 192.168.9.104
Discovered open port 3260/tcp on 192.168.9.104
Completed Connect Scan at 11:48, 4.21s elapsed (1000 total ports)
Nmap scan report for win2012-veeam.fritz.box (192.168.9.104)
Host is up (0.00059s latency).
Not shown: 991 filtered ports
PORT STATE SERVICE
111/tcp open rpcbind
135/tcp open msrpc
139/tcp open netbios-ssn
445/tcp open microsoft-ds
1063/tcp open kyoceranetdev
2049/tcp open nfs
3260/tcp open iscsi
3389/tcp open ms-wbt-server
49155/tcp open unknown
MAC Address: 00:50:56:9C:68:FA (VMware)

Read data files from: /usr/bin/../share/nmap
Nmap done: 1 IP address (1 host up) scanned in 4.25 seconds
Raw packets sent: 1 (28B) | Rcvd: 1 (28B)

nsswitch.conf SMF config

# svccfg
svc:> select dns/client
svc:/network/dns/client> setprop config/search = astring: \
(“us.company.com” “eu.company.com” “companya.com” “companyb.com” “company.com” )
svc:/network/dns/client> setprop config/nameserver = net_address: \
( 138.2.202.15 138.2.202.25 )
svc:/network/dns/client> select dns/client:default
svc:/network/dns/client:default> refresh
svc:/network/dns/client:default> validate
svc:/network/dns/client:default> select name-service/switch
svc:/system/name-service/switch> setprop config/host = astring: “files dns”
svc:/system/name-service/switch> select system/name-service/switch:default
svc:/system/name-service/switch:default> refresh
svc:/system/name-service/switch:default> validate
svc:/system/name-service/switch:default>
# svcadm enable dns/client
# svcadm refresh name-service/switch
# grep host /etc/nsswitch.conf
hosts: files dns
# cat /etc/resolv.conf
#
# opyright (c) 2011, Oracle and/or its affiliates. All rights reserved.
#

#
# _AUTOGENERATED_FROM_SMF_V1_
#
# WARNING: THIS FILE GENERATED FROM SMF DATA.
# DO NOT EDIT THIS FILE. EDITS WILL BE LOST.
# See resolv.conf(4) for details.

search us.company.com eu.company.com companya.com companyb.com company.com
nameserver 138.2.202.15
nameserver 138.2.202.25

zfs share for Time machine backups

I’ve tried using a share for time machine before, but due to the nature of how time machine works it eventually consumed all the spare space that is available on that particular share.

bring on zfs quotas… (my drive on my mini mac is 100GB, so 200GB should be enough for a few variations)

zfs set quota=200G unprotected/timemachine

This adds a artificial limit to your zfs filesystem, making sure that time machine does not consume more than 200 gigabytes of space.

NFS is your best bet, i’ve created a NFS share like so…

zfs create unprotected/timemachine

zfs set sharenfs=on unprotected/timemachine



HP Gen 8 servers and networking issues – TG3 driver

There is a bug in the tg3 driver on the ESXi hosts (1gbit broadcom cards in the new hosts). If the network card is put under load and netqueue is enabled it will sometimes decide to drop all traffic. Essentially i’ve disabled netqueue and the problems have gone away…. as per this vm kb :

http://kb.vmware.com/kb/2035701

The isues will present themselves as log entires like so;

2012-11-19T18:58:52.137Z cpu17:4155)<6>tg3 : vmnic8: RX NetQ allocated on 1
2012-11-19T18:58:52.138Z cpu17:4155)<6>tg3 : vmnic8: NetQ set RX Filter: 1 [00:50:56:71:46:87 0]
2012-11-19T18:58:52.138Z cpu17:4155)<6>tg3 : vmnic7: RX NetQ allocated on 1
2012-11-19T18:58:52.138Z cpu17:4155)<6>tg3 : vmnic7: NetQ set RX Filter: 1 [00:50:56:71:46:87 0]
2012-11-19T18:59:12.139Z cpu21:4155)<6>tg3 : vmnic4: NetQ remove RX filter: 1
2012-11-19T18:59:12.139Z cpu21:4155)<6>tg3 : vmnic4: Free NetQ RX Queue: 1
2012-11-19T18:59:22.137Z cpu24:4155)<6>tg3 : vmnic4: RX NetQ allocated on 1
2012-11-19T18:59:22.138Z cpu24:4155)<6>tg3 : vmnic4: NetQ set RX Filter: 1 [00:50:56:71:46:87 0]
2012-11-19T18:59:42.138Z cpu21:4155)<6>tg3 : vmnic7: NetQ remove RX filter: 1
2012-11-19T18:59:42.138Z cpu21:4155)<6>tg3 : vmnic7: Free NetQ RX Queue: 1
2012-11-19T18:59:42.140Z cpu21:4155)<6>tg3 : vmnic4: NetQ remove RX filter: 1
2012-11-19T18:59:42.140Z cpu21:4155)<6>tg3 : vmnic4: Free NetQ RX Queue: 1
2012-11-19T19:00:02.139Z cpu28:4155)<6>tg3 : vmnic8: NetQ remove RX filter: 1

vmware – copy networking port groups to another host

You might be able to do this via “host profiles” but if you do not have the licensing for it, this is the alternative. Very handy if you have over 50 or so port groups. It can be re-run to add to additional hosts as needed.

Install powercli, run the following to obtain your current list of virtual port groups off existing host;

Get-VirtualPortGroup -VirtualSwitch vSwitch0 -VMHost esx-01

Grab the output and place into CSV file or copy straght into Excel…

Following the formatting of the following “esx_switching-generic.csv” import the required data from aboves output. Note : the top line is the headers and should always be at the top of the CSV file.

Type,HostName,SwitchName,NIC,PortGroupName,VLAN,IP,Subnet,KernelGW
Portgroup,esx-02,vSwitch0,,Test_Network1,510,,,
Portgroup,esx-02,vSwitch0,,Test_Network2,511,,,

 

Place this file in a location that is called by this script —

#This script is designed to allow you to configure switches on multiple hosts by
#importing information from a prepopulated .csv file. vMotion switch created based
#on Mike Laverick's posting http://www.rtfm-ed.co.uk/?p=1514
#!!!!!!!Values passed for Type are Case sensitive since comparisons are being made.!!!!!!!

$getinfo = Import-Csv "D:\esx_switching-generic.csv" #need to input locatin of .CSV file

#Connect-VIServer -Server #Need to input appropriate vCenter Server

$getinfo | % {
$Type = $_.Type #!!!! Case Sensitive !!!!!!
$gethost = Get-VMHost -Name $_.HostName
$SwitchName = $_.SwitchName
$PortGroup = $_.PortGroupName
$Nic = $_.NIC
$VLAN = $_.VLAN
$IP = $_.IP
$Subnet = $_.Subnet
$kernelGW = $_.KernelGW

If ($Type -eq "Switch") {
$gethost | New-VirtualSwitch -Name $SwitchName -Nic $Nic
}

#Gets Switch object based on the value for SwitchName (required for several cmd-lets that do not accept Strings)
#'If' statement is used since a vMotion type does not already have a switch configured which will throw up an error.
If ($Type -ne "vMotion") {
$getswitch = Get-VirtualSwitch -VMHost $gethost -Name $SwitchName
}

#Add additional NIC to vSwitch to create a Team
If ($Type -eq "Team"){
$getswitch | Set-VirtualSwitch -Nic $Nic
}

#Add Portgroup to existing switch with VLAN
IF ($Type -eq "Portgroup") {
$getswitch | New-VirtualPortGroup $PortGroup -VLanId $VLAN
}

#Creates vMotion switch and configures vmkernel gateway (located under DNS and Routing in configuration tab)
IF ($Type -eq "vMotion") {

$newvswitch = New-VirtualSwitch -VMHost $gethost -Name $SwitchName -Nic $Nic
$vmotion = New-VirtualPortGroup -VirtualSwitch $newvswitch -Name $PortGroup
New-VMHostNetworkAdapter -VMHost $gethost -PortGroup $PortGroup -VirtualSwitch $newvswitch -IP $IP -SubnetMask $subnet -VMotionEnabled: $true

$vmhostnetwork = get-vmhostnetwork $gethost
set-vmhostnetwork -network $vmhostnetwork -vmkernelgateway $kernelGW
}
}

When Is A Trunk Not A Trunk?

Very good article here – http://networkingnerd.net/2011/02/02/when-is-a-trunk-not-a-trunk/

Particuly like setting of Native vLAN and tagged networks over an HP trunk;

So, if HP refers to an uplink carrying multiple VLANs are a tagged port, then does HP have a “trunk”? In fact they do. In HPvania, a trunk is a logical construct that aggregates multiple ports into one logical link. For those of you that might be out there scratching your heads about this one, this means that when you “trunk” a group of ports on an HP switch, you are creating one LACP link from up to four individual ports. This kind of configuration should look like this:

Switch(config)#trunk 19-24
Switch(config)#trk1
Switch(config-trk)#lacp
Switch(config-trk)#vlan 1
Swtich(config-vlan)#untagged trk1
Swtich(config-vlan)#vlan 10
Swtich(config-vlan)#tagged trk1
Swtich(config-vlan)#vlan 99
Swtich(config-vlan)#tagged trk1

Those of you that are fans of irony will appreciate that the above config sets up this LACP port aggregation to pass multiple VLANs to another switch. In other words, we are configuring a Cisco “trunk” on top of an HP “trunk”.