ipsec on a small LAN

Most tutorials cover implementing IPsec as a VPN solution between two sites, but what if you want to secure communications on one site representing a not atypical home LAN with a couple of machines: some wireless, some Linux, some Android, some Windows, and a dual ipv6/ipv4 stack (courtesy of Hurricane Electric’s free TunnelBroker service)?  And to make life more difficult, there are some devices that don’t support IPsec at all.  Obviously, you need at least two IPsec capable machines for this to make much sense.

Does it work, and how well?


Well I run a weird mix of Debian, Ubuntu, and Raspbian machines at home for different purposes, but as they all have a common root the procedure is the same on each.  First, we need to install the user space tools (and I’m using Racoon for automatic key exchange).

sudo apt-get install racoon ipsec-tools

and choose “Direct” configuration for racoon when asked.  Using pre-shared keys is a pain in the ‘arris so I’m going to use RSA keys for the moment.  Therefore, we need to create RSA keys for each of the hosts involved and compile a file of their public keys.  So on each host, run

sudo plainrsa-gen -b 4096 -f /etc/racoon/certs/hostname.key

Note that on Ubuntu hosts the ‘certs’ subdirectory will need to be created first.

If you look at the contents of the hostname.key file you will see the public key on the top line, which is preceded with a hash.  Create a file called public.keys and copy the public key of each host (minus the hash) into it so that it looks something like this

 : PUB 0sAQPIdX8L5BBW88YMUOKLhMckJbY830F[...snip...]mUHeDybGbI2NexGgLxJ2Q==
 : PUB 0sAQO9WobR9V57+BqEszLKMCxhlL/W6eR[...snip...]HqAG6PLyuxThiwpyjBhHw==
 : PUB 0sAQPXx0+BaYoeAjfzUKzTW4JC1ML8BFO[...snip...]DUVvxuw25lmHDP9RrkASw==
 : PUB 0sAQPJQwrx4O/VElaqACZ+jL01X3MFU0b[...snip...]HBrV949zZIr3Il+AwwN/w==
 : PUB 0sAQPcdFpQMOsN/+SKrGRupZxy8Mhg95h[...snip...]XSIUlG6PtDOSbAr6vmP0Q==

Note that there should be a space before ‘ : PUB’.  Now copy this public.keys file to the /etc/racoon/certs directory of every host so that each host has access to the public key of every other host.

Next up edit the /etc/racoon/racoon.conf file to look like below.

log notify;
path certificate "/etc/racoon/certs";

remote anonymous {
        exchange_mode main,aggressive;
        lifetime time 12 hour;
        certificate_type plain_rsa "hostname.key";
        peers_certfile plain_rsa "public.keys";
        proposal {
                encryption_algorithm aes;
                hash_algorithm sha1;
                authentication_method rsasig;
                dh_group modp1024;
        generate_policy off;

sainfo anonymous {
        pfs_group modp768;
        encryption_algorithm 3des;
        authentication_algorithm hmac_md5;
        compression_algorithm deflate;

Here we have set rsasig to be the initial key authentication method for all hosts, and the encryption mechanisms for all hosts.  Now, restart racoon and we are nearly there.

sudo service racoon restart

If you are running a firewall then you must remember to punch a hole in it for racoon something like

sudo ufw allow to any port 500 proto udp from any

Having set up the host keys, and configured racoon to automatically exchange keys and set up the Security Associations, all that remains is to set the Security Policies.  Edit the /etc/ipsec-tool.conf script to look like below, replacing your ip ranges as appropriate.

#!/usr/sbin/setkey -f

# NOTE: Do not use this file if you use racoon with racoon-tool
# utility. racoon-tool will setup SAs and SPDs automatically using
# /etc/racoon/racoon-tool.conf configuration.

## Flush the SAD and SPD

#ipv4 policies
spdadd any -P out ipsec
spdadd any -P in ipsec

#ipv6 policies
spdadd 2001:470:xxxx:xxxx::/64 2001:470:xxxx:xxxx::/64 any -P out ipsec
spdadd 2001:470:xxxx:xxxx::/64 2001:470:xxxx:xxxx::/64 any -P in ipsec

The policies shown here for both ipv4 and ipv6 are that any ip-packet to and from a host should be both encrypted (ESP) and authenticated (AH) if possible.  You can up the policy from ‘use‘ to ‘require‘ but you may run into problems with non-IPsec hardware such as ADSL routers etc. which will be rejected by the policy.

Lastly, put the policies into effect

sudo /etc/ipsec-tool.conf

Testing it out

Start a tcpdump session on a terminal on one of the IPsec enabled hosts

sudo tcpdump ah

and then run a ping to or from another

ping -c 1

and you should get an output similar to

10:53:13.582305 IP > AH(spi=0x0a0f7d17,seq=0x22): ESP(spi=0x0ba5fd41,seq=0x22), length 100
10:53:13.583514 IP > AH(spi=0x04b896f3,seq=0x20): ESP(spi=0x0b30198e,seq=0x20), length 100

showing that both AH and ESP are working in both directions.


So it takes just a few minutes for Linux hosts to start talking to each other with IPsec, although it strikes me as complete overkill for a home LAN.  But at least I can do it if I want to.

RSA keys are a bit less of a pain that pre-shared keys when you have a handful of hosts, but would be very difficult to maintain on a larger network.  And Windows does not support them.  Pre-shared keys would be fine for a tunnel.

In the next installment, I shall move to a CA scheme instead of RSA, and hook up the Windows client.