Building Diskless Clients with FreeBSD 5.2

by Mikhail Zakharov
09/30/2004

Having prepared the FreeBSD 5.2 Netboot server, we can begin to configure our diskless workstations. Let's prepare the kernel and add it to the two-part filesystem (accessible for reading only and accessible for reading and writing) and finally create rc-scripts.

Kernel

As the previous article explained, pxeboot expects the NFS-resource at 192.168.1.2/diskless_ro to be the root filesystem, so it'll try to find the kernel there. While booting the diskless client using pxeboot, we could use the GENERIC kernel. GENERIC has many drivers, but there are no useful BOOTP options for us. Instead, let's compile the DISKLESS kernel to put on the client filesystem.

Compile the new kernel as usual, changing GENERIC and removing everything unnecessary. Be sure to keep the following lines, though:

options    NFS_ROOT
options NFSCLIENT

It's a very good thing to add the following options; they aren't in the GENERIC kernel, but you can certainly do without them:

options    BOOTP
options BOOTP_NFSROOT
options BOOTP_COMPAT

These options will cause an additional dialogue with the DHCP server to rediscover the diskless client's IP address. Otherwise, the server cannot send additional useful information over BOOTP such as the hostname. As the handbook mentions, you can also add the options BOOTP_NFSV3 and BOOTP_WIRED_TO if you like.

I've tested this kernel successfully on several diskless stations.

Directory tree

The only difference between the diskless workstation and any other computer is that the diskless client has its filesystems on the NFS resource. That's why when it boots successfully, it's necessary to re-create a similar directory tree with only a few changes. Having chosen the directories /diskless_ro and /diskless_rw on the server for this task, we'll place all the client files there.

/diskless_rw

First prepare directory_rw and put it aside. It's necessary only at the final stage of diskless client booting.

The directory /diskless_rw holds individual filesystems for all clients. Having set up the NFS server, these directories follow the form /diskless_rw/workstation IP address/rw-file-system. For example, we've created the following directories for the test diskless client:

server# mkdir /diskless_rw/192.168.1.101
server# mkdir /diskless_rw/192.168.1.101/etc
server# mkdir /diskless_rw/192.168.1.101/var

We'll mount the server directories /diskless_rw/workstation IP address/etc and /diskless_rw/workstation IP address/var as /etc and /var on the diskless client. This naming scheme allows us to give each diskless station its individual directories for reading and writing.

It's a good time to create more directories in /var and /etc for later use. Run:

server# mkdir /diskless_rw/192.168.1.101/etc/pam.d
server# mkdir /diskless_rw/192.168.1.101/etc/X11
server# mkdir /diskless_rw/192.168.1.101/var/home
server# mkdir /diskless_rw/192.168.1.101/var/log
server# mkdir /diskless_rw/192.168.1.101/var/run
server# mkdir /diskless_rw/192.168.1.101/var/tmp

The directory trees for /etc and /var on the test diskless workstation will be:

/diskless_rw/192.168.1.101
/diskless_rw/192.168.1.101/etc
/diskless_rw/192.168.1.101/etc/pam.d
/diskless_rw/192.168.1.101/etc/X11
/diskless_rw/192.168.1.101/var
/diskless_rw/192.168.1.101/var/home
/diskless_rw/192.168.1.101/var/log
/diskless_rw/192.168.1.101/var/tmp
/diskless_rw/192.168.1.101/var/run

Unfortunately, the code for passing the kernel information about the swapfile through BOOTP no longer exists in FreeBSD. Instead, place each client's swapfile within its filesystem:

dd if=/dev/zero of=diskless_rw/192.168.1.101/swap bs=1k
count=32000

Repeat this operation on every diskless workstation. We'll speak later about the file contents of all these directories as well as how to ensure that /etc and /var work properly.

/diskless_ro

The directory diskless_ro will host the diskless station's root filesystem. It will contain the directories bin, boot, dev, etc, lib, libexec, mnt, sbin, usr, and var.

The directories bin, lib, libexec, and sbin contain the main FreeBSD libraries and programs. We'll take them from the server filesystem without modifying them:

server# cp -r /bin /lib /libexec /sbin /diskless_ro

Create the directory usr to mount the directory /usr from the server later:

server# mkdir /diskless_ro/usr

Now we must prepare the diskless station /boot directory and put the kernel there. We'll make it by copying the boot directory of the server and there copying the compiled DISKLESS kernel:

server# cp -r /boot /diskless_ro
server# cp /sys/i386/compile/DISKLESS/kernel /diskless_ro/boot/kernel

When I used this configuration I had problems booting some workstations. I solved this on some computers by removing boot4th and creating my own loader.rc. If you are ready to do the same, then use:

server# cd /diskless_ro/boot
server# rm *.4th

Without boot4th there's no automatic loading of device.hints. You have two ways to fix the problem:

Compile device.hints into the kernel. See hints GENERIC.hints in the GENERIC kernel example.
Use loader.rc instead of device.hints.

Old equipment isn't very reliable, and sometimes it takes too long to configure. To be more flexible configuring the kernel, I used the second approach on the test station.

As the values in device hints are common kernel variables, it's possible to include them as set variable="value" into the loader.rc file. An example loader.rc will look like:

set hint.fdc.0.at="isa"
...
boot /boot/kernel/kernel

The last line of loader.rc specifies which kernel to load.

I think it's more efficient to compile loader.rc this way and then remove spare kernel variables:

server# cd /diskless_ro/boot
server# awk '{print "set "$1}' device.hints > loader.rc
server# echo "boot /boot/kernel/kernel" >> loader.rc

I used this loader.rc.

Now we don't need device.hints on the client filesystem any more and can safely remove it. The defaults directory containing loader.conf is no longer useful either, so execute the following:

server# cd /diskless_ro/boot
server# rm -r defaults device.hints

The ACPI power-management module loads by default, but it may be useful to disable it; sometimes diskless stations fail to boot with ACPI enabled. It can also happen the other way around, when diskless clients can't boot without ACPI (see acpi(4)). To disable ACPI, set the kernel variable hint.acpi.0.disabled="1".

In device.hints, it's:

hint.acpi.0.disabled="1"

In loader.rc, it's:

set hint.acpi.0.disabled="1"

That's all for the directory /boot.

The dev directory mounts the devfs filesystem, which contains all the device files of the FreeBSD system. Without this directory, the diskless station will hang while loading the init process without giving an error! So:

server# mkdir /diskless_ro/dev

Let's also take care of the free directory var to mount /diskless_rw/var from the server 192.168.1.2:

server# mkdir /diskless_ro/var

In order not to create separate filesystems for the client's directories home and tmp, make soft links from /home to /var/home and /tmp to /var/tmp, which will ensure access to the directories for reading and writing:

server# cd /diskless_ro
server# ln -s /var/tmp .
server# ln -s /var/home .

Then create and populate the etc directory. As it lives in the /diskless_ro and all of the clients share it in read-only mode, it's important to make its contents universal and compact. It will have only a few required files. Let's take some of them (services, netconfig, and login.conf) from the server filesystem:

server# mkdir /diskless_ro/etc
server# cp /etc/services /etc/netconfig /etc/login.conf /diskless_ro/etc

We'll play a trick to give each diskless station its own individual configuration. Considering that the init process runs /etc/rc, we'll mount a filesystem from diskless_rw over diskless_ro/etc. To this effect we'll create our own etc/rc in diskless_ro/etc:

#!/bin/sh

PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/X11R6/bin; export PATH

boot_ip=`kenv boot.netif.ip`
mount -t nfs 192.168.1.2:/diskless_rw/${boot_ip}/etc /etc
mount -t nfs 192.168.1.2:/diskless_rw/${boot_ip}/var /var
swapon /var/swap
rm -rf /var/tmp/*; rm -rf /var/tmp/.*;

. /etc/rc2

exit 0

This simple script executes the following actions:

Sets the environment variable PATH to specify the path to the executable files.
Defines the IP address received during booting and places it into the boot_ip variable.
Uses the value of boot_ip (the client IP address) to mount the NFS filesystems /etc and /var from the server 192.168.1.2. At this point, the directories /etc and /var, previously accessible in read-only mode, become accessible both for reading and writing. In this case, though, they have different files for different stations.
Sets /var/swap as the swapfile.
Clears the directory /var/tmp and, in consequence, /tmp because of the soft links created above.
Finally, runs the second rc-script (rc2), which continues booting the system. Because the filesystem mounted in the directory /etc is original for every diskless client, the script rc2 as well as all the other files of the directory /etc can differ. At this point we can start configuring each diskless client by writing different commands in their rc2 files.