The FreeBSD Diary

This article shows you how I created a jail for the OSW website. It runs in a jail on the same system as this website. I originally did this install back in November 2003 and the notes from that session form the basis of this article. I have need to recreate the jail now as we recently had an HDD failure.

NOTE: This article applies to FreeBSD 4. I have since written about jail on FreeBSD 5, and it appears to apply to FreeBSD 6.x just as well. I recommend read that article over this one.

A jail is useful for many purposes. In my case, I wanted to give the OSW project a place to run their websites, mailing lists, etc, but at the same time keep them isolated from the rest of the machine. In short, it gives them a virtual machine, and it gives me peace of mind knowing that I have less to worry about with respect to the rest of the machine.

The main document for creating a jail is man jail. I followed the instructions listed under Setting up a Jail Directory Tree. I followed those instructions to create the jail.

Of note, I did not do this:

• I left sendmail (actually, postfix) running. I just changed it so that it did not listen on all IP addresses. This allowed the jail to run its own mail server.
• I did not specify the portmap_enable="NO" directive as that is default setting (see /etc/defaults/rc.conf).

Most daemons will listen to whatever IP addresses are available to them. After starting your jail, if you try to ssh to it, you will not get into it. You'll be in the host environment instead. To get into the jail environment via ssh, you need to:

• Tell the host environment sshd not to listen to the jail's IP address
• run sshd in the jail

Host environment sshd

ListenAddress 192.168.0.100

kill -HUP `cat /var/run/sshd.pid`

$ telnet 192.168.0.155 22
Trying 192.168.0.155...
telnet: connect to address 192.168.0.155: Connection refused
telnet: Unable to connect to remote host

Jail environment sshd

To start sshd in the jail environment, add the following line to /etc/rc.conf:

sshd_enable="YES"

From man jail, to start a jail, issue this command:

[root@mtwenty:/home/dan] # jail /usr/jails/192.168.0.155 osw.example.org 192.168.0.155 /bin/sh
$

$ sh /etc/rc
Loading configuration files.
mdmfs: mdconfig (attach) exited with error code 1
cd: can't cd to ppp
rm: utmp: Permission denied
cp: utmp: Permission denied
/etc/rc.d/cleanvar: cannot create /var/run/clean_var: Permission denied
undef.unixathome.org
Setting hostname: undef.unixathome.org.
Generating nsswitch.conf.
eval: cannot create /etc/nsswitch.conf: Permission denied
Generating host.conf.
eval: cannot open /etc/nsswitch.conf: No such file or directory
ln: /dev/log: Operation not permitted
eval: cannot create /var/run/syslogd.sockets: Permission denied
Starting syslogd.
syslogd: child pid 97899 exited with return code 1
ELF ldconfig path: /lib /usr/lib /usr/lib/compat
ldconfig: mkstemp(/var/run/ld-elf.so.hints.EO1FRT): Permission denied
a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout
ldconfig: /var/run/ld.so.hints.QnuzP1cZaF: Permission denied
Starting local daemons:.
Updating motd ... /etc/motd is not writable, update failed.
/etc/rc: WARNING: Setting entropy source to blocking mode.
====================================================
Type a full screenful of random junk to unblock
it and remember to finish with . This will
timeout in 300 seconds, but waiting for
the timeout without typing junk may make the
entropy source deliver predictable output.

Just hit for fast+insecure startup.
====================================================
kern.random.sys.seeded: 1
lkajdflkjadsflkjsdfalk; voiusfady 098125 09okjcv lkhq234ou 8g09fuzohj adjf
You don't exist, go away!
You don't exist, go away!
You don't exist, go away!
sendmail_submit: /etc/mail/aliases.db not present, generating
Permission denied (real uid not trusted)
asendmail_clientmqueue: /etc/mail/aliases.db not present, generating
Permission denied (real uid not trusted)
dStarting cron.
cron: can't open or create /var/run/cron.pid: Permission denied
Local package initialization:.
/etc/rc.d/msgs: cannot create /var/msgs/bounds: Permission denied
l
Sun Sep 11 21:05:35 UTC 2005
$

For the most part, this looks exactly like a normal startup. A few things to note

• /etc/fstab - if you see this, you didn't do as man jail said. You need to create an empty /etc/fstab file.
• adjkerntz - Not sure about this. I know you should comment out the /etc/crontab entry for adjkerntz within your jail environment.
• sendmail - My host environment does not use sendmail (i.e. I have "NO_SENDMAIL= true" in /etc/make.conf). I think this will go away once I install Postfix within the jail.
• net.inet.tcp.always_keepalive - I have no idea why this occurs

I found two interesting tools for starting and stopping the jails: sysutils/jailer and sysutils/jailutils. sysutils/jailer is installed in the jail environment. sysutils/jailutils should be installed in the host environment. Using those two tools, I created this start/stop script:

#!/bin/sh

case "$1" in
start)
        jail /usr/jail/192.168.0.155 osw.example.org 192.168.0.155 /usr/local/sbin/jailer > /dev/null && echo -n ' jail 66.154.97.254'
        ;;
stop)
        /usr/local/sbin/jails | /usr/bin/xargs /usr/local/sbin/killjail > /dev/null && echo -n ' jail'
        ;;
*)
        echo "Usage: `basename $0` {start|stop}" >&2
        y;;
esac

exit 0

This is a very limited script. It doesn't check that a jail is already running before starting it. That would be a nice addition. If you want to add it, I look forward to your patch.

In addition, you might want to add this to the host environment's /etc/sysctl.conf

jail.set_hostname_allowed=0

Under 5.*, this variable has a slightly different name.