The FreeBSD Diary

Leon Dang wrote this article. Mike Meyer reviewed it for me. Thanks to both of them.

I am using CableNet access with Australian telco Optus - which uses Excite@Home to handle their cable network. I don't know whether our network would be the same as that in the US, but I'm writing this article to give you guys a starting point for your own broadband connection.

My home network consists of a bunch of boxes NATed through a FreeBSD boxen. The Optus network requires me to obtain an IP address through DHCP. I am not allowed to run any servers, which is fine by me.

In this article, I'll describe how you could configure DHCP, setup a 'faking' identd server, some ipfilter/ipnat sample configuration, and BIND as the single name server for your internal network. I don't like to write too much and prefer consiceness, so you won't be fed any background information - that's really up to the individual man pages and docs. All the necessary applications are part of the FreeBSD distribution, and I'll assume that you've installed them on your box. I'll also assume that only once you finished configuring the box with these instructions do you reconnect it back to the cable modem :-)

DHCP setup is fairly straight forward. You simply need dhclient(8) which is a little application that triggers a shell script to do change the IP address of the network interface. dhclient reads in /etc/dhclient.conf for its configuration. Mine basically looks like:

#
#       See ``man 5 dhclient.conf'' for details.
#
interface "ext0" {
        send host-name "MYHOSTNAME";
        supersede domain-name "MYDOMAIN.box xxx.optushome.com.au";
        prepend domain-name-servers 192.168.1.1;
        request subnet-mask, broadcast-address, time-offset, routers,
                domain-name, domain-name-servers;
        require subnet-mask;
        media "media autoselect";
}

The values that you will need to change are:

1. ext0 to be the name of the network interface that you are using.
2. MYHOSTNAME to be the name of your machine as provided by the ISP. For Optus users, this is CAxxxxxxx. If your ISP configures a Windows machine, simply go to Control Panel ->Networking and look at the second tab.
3. MYDOMAIN.box to be whatever your internal network domain name to be.
4. xxx.optushome.com.au is my ISP provided domain name for the box.
   You might note that MYDOMAIN.box and xxx.optushome.com.au are used as domain suffixes to any address you lookup. They're basically search order in /etc/resolv.conf
5. 192.168.1.1 to the IP address of the box you are configuring. You can remove this line if you won't be running a name server internally.

My crummy de0 network card seems to hang if it doesn't get flushed for some reason, but when I reinitialize it with dhclient, things become fine. So I run a shell script that forces dhclient to obtain a new address every day. It's also kinda useful when you want to tell the modem that the gateway's alive, with the added precaution that you might get an IP address change from the ISP (dhclient should obtain it's lease life-time and know when to renew the IP address through negotiation with the ISP. If you want to check for yourself, view /var/db/dhclient.leases).

I set the shell script to run at 3 am, at which point I'll probably be dead asleep anyways so my machine would be inactive. To do this, I create a dhclient.sh script:

#!/bin/sh
#
# dhclient.sh
#
killall dhclient
sleep 1
dhclient

(save dhclient.sh in /root and chmod u+x dhclient.sh). I then place an entry in /etc/crontab for dhclient.sh:

# crontab entry for dhclient - reset DHCP every day
0   3   *   *   *   root   /root/dhclient.sh

That's all there is to DHCP!

I don't use IRC a lot, but every now and then I do and a number of servers I connect to require ident approval. Now providing a real identity to the world is a not particularly a good idea, especially with the amount of visibility to get from IRC. That's why I have one that produces fake idents.

[Mike mentions that using the -g flag on inetd accomplishes the same thing:

-g      Instead of returning the user's name to the ident requester, re-
        port a username made up of random alphanumeric characters, e.g.
        ``c0c993''.  The -g flag overrides not only the user names, but
        also any .fakeid or .noident files.

]

I like C and it's my primary language, so I'll give you the source code in C also. Basically, I ripped a couple of functions from the identd distribution (by Peter Eriksson <pen@lysator.liu.se>) - so I'd like to give him the credits for the source. I basically plugged in a randomizer, and here is:

/*
 * Random identd generator. Runs under inetd.
 *
 */
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <ctype.h>
#include <sys/stat.h>
#include <errno.h>
#include <netdb.h>
#include <signal.h>
#include <fcntl.h>
#include <unistd.h>

#include <sys/types.h>
#include <sys/param.h>
#include <sys/ioctl.h>
#include <sys/socket.h>
#include <sys/file.h>
#include <sys/time.h>
#include <sys/wait.h>

#include <netinet/in.h>
#include <arpa/inet.h>

#include <syslog.h>

#define USE_TIME_SEED 0
#define IO_TIMEOUT    30

extern int errno;

/*
 * Returns 0 on timeout, -1 on error, #bytes read on success.
 */
ssize_t timed_read(int fd, void *buf, size_t siz, time_t timeout)
{
        int error, tot = 0, i, r;
        char *p = buf;
        fd_set readfds;
        struct timeval tv, start, after, duration, tmp;

        tv.tv_sec = timeout;
        tv.tv_usec = 0;

        while (1) {
                FD_ZERO(&readfds);
                FD_SET(fd, &readfds);

                gettimeofday(&start, NULL);
                if ((error = select(fd + 1, &readfds, 0, 0, &tv)) <= 0)
                        return error;
                r = read(fd, p, siz - tot);
                if (r == -1 || r == 0)
                        return (r);
                for (i = 0; i < r; i++)
                        if (p[i] == '\r' || p[i] == '\n') {
                                tot += r;
                                return (tot);
                        }
                gettimeofday(&after, NULL);
                timersub(&start, &after, &duration);
                timersub(&tv, &duration, &tmp);
                tv = tmp;
                if (tv.tv_sec < 0 || !timerisset(&tv))
                        return (tot);
                tot += r;
                p += r;
        }
}

/*
 * Returns 0 on timeout, -1 on error, #bytes read on success.
 */
ssize_t timed_write(int fd, const void *buf, size_t siz, time_t timeout)
{
        int error;
        fd_set writeds;
        struct timeval tv;

        FD_ZERO(&writeds);
        FD_SET(fd, &writeds);

        tv.tv_sec = timeout;
        tv.tv_usec = 0;

        if ((error = select(fd + 1, 0, &writeds, 0, &tv)) <= 0)
                return error;
        return(write(fd, buf, siz));
}

int main()
{
	int i;
	char c;
	static char buf[4096];
	static char buf2[4096];
	static char randstr[12];
	struct sockaddr_in sin;
	struct timeval tv;
	int len;
	int sock;

	len = sizeof(sin);
	sock = accept(0, (struct sockaddr *)&sin, &len);
	if (sock < 0) {
		syslog(LOG_ERR, "accept failed: %s", strerror(errno));
		perror("identd: accept()");
		exit(1);
	}
	syslog(LOG_NOTICE, "connection from: %s", inet_ntoa(sin.sin_addr));

	memset(buf, 0, sizeof(buf));
	if ((i = timed_read(sock, buf, sizeof(buf)-1, IO_TIMEOUT)) <= 0) {
		perror("identd: read()");
		i = snprintf(buf, sizeof(buf), "%d, %d : ERROR : UNKNOWN-ERROR\r\n",
		             0, 0);
		timed_write(sock, buf, i, IO_TIMEOUT);
		return 0;
	}
	buf[i] = '\0';
	for (i = 0; buf[i]; ++i) {
		if (buf[i] == '\r' || buf [i] == '\n') {
			buf[i] = '\0';
			break;
		}
	}

#if USE_TIME_SEED
	srand(time(NULL));
#else
	/*
	 * Base the seed on the request just in case we are asked
	 * for the same details... (be cheeky)
	 *
	 */
	i = 0;
	for (c = 0; c < 30; c++) {
		i ^= buf[c]; /* Perform XORs... */
	}
	srand(i);
#endif
	for (i = 0; i < 5; i++) {
		randstr[i] = (rand() % 26) + 'a';
	}
	for (; i < 7; i++) {
		randstr[i] = (rand() % 10) + '0';
	}
	randstr[i] = '\0';
	i = snprintf(buf2, sizeof(buf2), "%s : USERID : UNIX : %s\r\n",
	             buf, randstr);
	timed_write(sock, buf2, i, IO_TIMEOUT);
	return 0;
}

Compile the code and place the executable into /usr/local/sbin:

$ cc identd.c -o identd
$ cp ident /usr/local/sbin

Modify /etc/inetd.conf with an entry for the identd server. This will cause inetd(8) to run identd whenever there is a connection to the ident port

# inetd.conf entry for identd
identd    stream    tcp    wait    root   /usr/local/sbin/identd    identd

Send a SIGHUP signal to inetd to tell it to reread the configuration file:

$ killall -SIGHUP inetd

Okay, we're done with identd, lets move on...

The reason I set up a local name server is that run a local network and prefer to know things by name than IP addresses. It also helps with the Winblows environment that my family is using. I basically setup BIND to handle all local name requests and forward any others to my ISP's name server.

Depending on how you set up BIND, you may have configuration files located in various places. My default configuration is located in /etc/namedb/named.conf. I place all other BIND stuff in /var/named. Below is the /etc/namedb/named.conf configuration file that I use:

[Mike noted that he has had bad luck with forwarders. He doesn't use that clause at all.]

#
# BIND Configuration File (/etc/namedb/named.conf)
#

acl "internal" {
   { 192.168.0.0/16; };
};

options {
   directory "/var/named";
   dump-file "/var/named/named_dump.db";
   datasize 2m;
   statistics-file "/var/named/named.stats";
   allow-query { internal; };
   listen-on { 192.168.1.1; 127.0.0.1; };

   #
   # forwarders used to send non-localnet dns queries to ISP's dns server
   # Change these values to whatever your ISP's DNS servers are.
   forwarders {
      203.123.123.211;
      203.123.123.212;
   };

}:

zone "mydomain.box" in {
   type master;
   file "db.mydomain";
};

zone "168.192.in-addr.arpa" in {
   type master;
   file "db.192.168";
};

zone "0.0.127.in-addr.arpa" in {
   type master;
   file "localhost.rev";
};

zone "." in {
   type hint;
   file "db.cache";
};

[Note from Dan: I've often

The "internal" acl is for my local network only, since I don't want anyone else to browse my DNS. You can change the 192.168.0.0/16 network mask to whatever your's is.

The options section is for global BIND options. Most the the fields are obvious. datasize sets the process data size for BIND (you can remove this line if you want).

The important section to note is forwarders. This basically tells bind to send non-local requests to the ISP's name servers. Change them to what your ISP has provide to you.

I use the forwarders since I don't want to query root name servers as it can get pretty slow. The remaining zone sections are configurations for local names and root db.cache. The configuration files for each (as specified by the file keyword) are stored in /var/named

To generate the localhost.rev file, do the following:

$ cd /etc/namedb
$ sh make-localhost
$ cp localhost.rev /var/named

My reverse name lookup /var/named/db.192.168 file looks like this:

$TTL 604800
168.192.in-addr.arpa. IN SOA gateway.mydomain.box. root.mydomain.box. (
        200103021       ; serial
        10800   ; refresh after 3 hours
        3600    ; retry after 1 hour
        604800  ; expire after 1 week
        86400 ) ; minim ttl of 1 day

;Name servers
168.192.in-addr.arpa.       IN NS    gateway.mydomain.box.

;Reverse name lookups
1.1.168.192.in-addr.arpa.   IN PTR   gateway.mydomain.box.

2.1.168.192.in-addr.arpa.   IN PTR   boxer.mydomain.box.

Note that the IP addresses are placed in reverse order and that all IP names contain a '.' at the end of them. I won't go into the details, but just remember to place them there otherwise you'll be scratching your head wondering what was going on.

Next, /var/named/db.mydomain, which contains forward lookup entries for my localnet:

$TTL 604800
@ 864000 IN SOA gateway.mydomain.box. root.mydomain.box. (
        200103021       ; serial
        10800   ; refresh after 3 hours
        3600    ; retry after 1 hour
        604800  ; expire after 1 week
        86400 ) ; minimum ttl of 1 day

@       IN A  192.168.1.1
@       IN NS gateway.mydomain.box.

mydomain.box.            IN NS  gateway.mydomain.box.
;
; Name to address mappings
;
localhost.mydomain.box.  IN A   127.0.0.1
;
; gateway
;
gateway.mydomain.box.    IN A   192.168.1.1
;
; all other entries
;
boxer.mydomain.box.      IN A   192.168.1.2
;
; Aliases - just an example
;
;gateway2.mydomain.box.  IN CNAME gateway.mydomain.box.

Once you've created these files, you can start named and perform a name lookup. To start named every time your system starts up, edit /etc/rc.conf and place an entry for named_enable="YES"

For testing, I modified the /etc/resolv.conf with:

#
# Contents of /etc/resolv.conf
#
search mydomain.box
nameserver 192.168.1.1

To check that your name server works, use nslookup:

$ named
$ nslookup boxer

BIND should then give you the appropriate IP response. If you don't get a response, the most likely culprit would be an invalid named configuration. Check that named starts up ok; it should produce a log file during startup so skim that file for details.

If everything is A-OK, then you're off to a good start. We now move on to securing the box before connecting it to the cable modem.

ipfilter (ipf(8)) is a great firewalling tool. If incorrectly setup it does some weird things to you. After using it for over a year, I still am trying to get the hang of things because every now and then I see a new problem popping up from prior configurations.

I'm going to give a preliminary configuration sample that you can expand on. Please refer to the ipfilter documentation and how-to's for extra information that I cannot provide myself in this article. I must say that the rules I provide here cannot be proven to be totally correct as there are so many possible variations around, it's to be considered a starting point only.

I once used ipfw(8) for diallup. It was great for that purpose, however I haven't touched it for a very long time and have moved on to using ipfilter instead. For those who desire to use ipfw, simply ignore this section - I reckon you would know what you're doing with ipfw :-)

Ok, to use ipfilter, you might need to recompile the kernel. You can actually use ipfilter in conjunction with ipfw - which is weird since there'll be extra bottlenecks and I don't recommend you doing so.

The kernel configuration entries that you'll need are (not sure if you actually need IPDIVERT... I'm doing this off the top of my head almost):

options IPFILTER
options IPDIVERT

You can remove options IPFIREWALL if you want. After configuring the kernel, compile it and install it.

Edit /etc/rc.conf and add the following lines:

ipfilter_enable="YES"
ipfilter_rules="/etc/ipf.rules"
ipnat_enable="YES"
ipnat_rules="/etc/ipf.rules"

The following is a sample of my ipfilter rules file /etc/ipf.rules:

#
# ipf.rules
#

###
### NOTE: in0 is internal network interface, ext0 is connected to cable modem.
###       Change the to the appropriate interface names
###


#############################################################################
#
# Allow localhost full access
#
pass  in  quick on lo0 all
pass  out quick on lo0 all

#############################################################################
#
# Allow everything on our localnet
#
pass in  quick on in0 all
pass out quick on in0 all

#
# Block access from Internet, but we can access the Internet from here
#
block in log on ext0 all

#############################################################################
#
# Pass all outgoing traffic
#
pass out quick on ext0 proto tcp  all flags S/SA keep frags keep state
pass out quick on ext0 proto udp  all keep state
pass out quick on ext0 proto icmp all keep state

#############################################################################
#
# Control what comes in from the Internet
#

#
# Nicely return error messages to those inaccessible ports.
#
block return-rst in log on ext0 proto tcp all flags S/SA

#
# Return errors for icmp and udp
#
block return-icmp-as-dest(port-unr) in log on ext0 proto udp all
block return-icmp-as-dest(port-unr) in log on ext0 proto icmp all

#
# Pass incoming DHCP messages
#
# (!!! change 203.123.123.12 to the cable modem's IP address. You can
#      get it's IP address by doing a tcpdump or ipmon on ext0
#      and look for entries that contain tcp/udp port 67)
#
pass in log quick on ext0 proto udp from 203.123.123.12/32 port = 67 to any port = 68 keep state

#
# Block 'fake' addresses
#
block in quick on ext0 from 192.168.0.0/16 to any
block in quick on ext0 from 172.16.0.0/16  to any
block in quick on ext0 from 10.0.0.0/8     to any
block in quick on ext0 from 127.0.0.0/8    to any

#
# Enable identd server access (for IRC). (run a fake identd server if enabled)
#
pass in log quick on ext0 proto tcp all port = 113 flags S/SA keep state

#
# Allow only a subset of ICMP requests
#
# icmp echo reply
pass in quick on ext0 proto icmp all icmp-type 0

# destination unreachable
pass in quick on ext0 proto icmp all icmp-type 3

# source quench
pass in quick on ext0 proto icmp all icmp-type 4

# redirect message
pass in quick on ext0 proto icmp all icmp-type 5

# icmp echo request
pass in quick on ext0 proto icmp all icmp-type 8

# time exceeded
pass in quick on ext0 proto icmp all icmp-type 11

# bad header
pass in quick on ext0 proto icmp all icmp-type 12

# block all other icmp's
block in log quick on ext0 proto icmp all

#
# > ipf automatically blocks here
#
# end of ipf.rules

You might like to remove some ICMP types from above to protect your network further.

Now for your localnet to access the Internet, edit the file /etc/ipnat.rules:

#
# ipnat rules
#
# !!! change ext0 to whatever your external interface (connecting to the
#     cable modem) is
#

map ext0 192.168.0.0/16 -> 0/32 portmap tcp/udp 1025:65000
map ext0 192.168.0.0/16 -> 0/32

I've found that ipnat doesn't clear it's list until a connection is disconnected or a timeout occurs. Sometimes, however, ipnat can have really long lists. What I basically do is force ipnat to flush it's list every 12 hours, which can be bad if I was in the stage of downloading something that cannot be recovered (ftp/http continue). To do this, I add an entry to /etc/crontab (though you might like to flush it manually if you want):

#
# crontab: Flush NAT tables every 12 hours
#
0       */12    *       *       *       root    /sbin/ipnat -F

Once all this is setup, you can plug your box to the cable modem and reboot. You should now have a fully working localnet gateway to the Internet.

As a sidenote, you might like to change your ipfilter timeouts. You can get their current values by doing:

$ sysctl net | grep ipf

And to change the values, do:

$ sysctl -w variable=n

where variable is the sysctl variable, and n is the value to assign.

Below is a script that does network accounting for you. It basically sums up the total number of bytes received from the cable modem for the period since it last ran for.

#!/bin/sh
# systats.sh - grab the number of bytes received on the external adapter

PATH=/bin:/sbin:/usr/bin:/usr/sbin; export PATH

#
# change these two parameters to whatever you have
#
NETSTAT_IF=ext0
STAT_FILE=/root/cable_stats.txt

# MAX_UINT taken from printf "%u\n" 0xffffffff in /usr/include/machine/limits.h
MAX_UINT=4294967295

OUT_BYTES=`netstat -b -I ${NETSTAT_IF} | head -2 | tail -1 | awk '{ print $7; }'`

LAST_OBYTES=0

if [ -f ${STAT_FILE} ]; then
        LAST_OBYTES=`tail -1 ${STAT_FILE} | awk '{ print $2; }'`
fi

TOTAL_OBYTES=`echo "printf(\"%u\n\", ${OUT_BYTES} - ${LAST_OBYTES})" | perl`

LAST_GT_OBYTES=`echo "printf(\"%d\n\", ${LAST_OBYTES} > ${OUT_BYTES})" | perl`
if [ ${LAST_GT_OBYTES} != 0  ]
then
        # Last obytes is greater than current, the system must
        # have rotated the byte counter
        BYTES_TO_MAX=`echo "printf(\"%u\n\", {MAX_UINT} - ${LAST_OBYTES})" |perl`
        TOTAL_OBYTES=`echo "printf(\"%u\n\", ${BYTES_TO_MAX} + ${OUT_BYTES})" | perl`
fi

echo `date '+%C%y/%m/%d:%H:%M'`' '${OUT_BYTES}' '${TOTAL_OBYTES} >> ${STAT_FILE}

The script outputs three columns: time, current netstat counter, and the total number of bytes since the last time the script was run. I use the script to monitor my network usage by including it in /etc/crontab to be run every mid-night . I hope that it'll be useful to you also.

So far I've produced pretty raw material. I was under an assumption that you have good knowledge of Unix before you delved into the material. I hope that this stuff is useful to you and please do make corrections if you find that I did something wrong or could improve upon.

Before I leave you alone, I have found that the ipfilter rules that I've provided you with seems to have a problem in that it sometimes blocks ACKs+someother flagged packets (run ipmon to see which packets are blocked and you'll know what I'm talking about). If you know how to correct the rules so that this doesn't happen, I'll be glad to know.

- Leon