From owner-freebsd-current@FreeBSD.ORG Sun Jun 19 15:07:23 2005 Return-Path: X-Original-To: current@FreeBSD.org Delivered-To: freebsd-current@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4DC0816A41C for ; Sun, 19 Jun 2005 15:07:23 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [204.156.12.53]) by mx1.FreeBSD.org (Postfix) with ESMTP id F218C43D53 for ; Sun, 19 Jun 2005 15:07:22 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by cyrus.watson.org (Postfix) with ESMTP id 07A2746BD9 for ; Sun, 19 Jun 2005 11:07:22 -0400 (EDT) Date: Sun, 19 Jun 2005 16:10:00 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: current@FreeBSD.org Message-ID: <20050619155228.Y6413@fledge.watson.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Subject: Summary: experiences with NanoBSD, successes and nits on a Soekris 4801 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Jun 2005 15:07:23 -0000 While in San Francisco a week or two ago, Sam Leffler kindly handed me a Soekris net4801 system. After borrowing the necessary power parts from Mark Murray here in Cambridge, and picking up a slightly overpriced flash disk from a local electronics store, I decided I'd experiment some with NanoBSD and see where it got me. In particular, I wanted to configure a DHCP/NAT/firewall box, and figured that this should be relatively straight forward, as it's likely one of the most common uses of embedded hardware of this type. I general, I was quite pleased with the experience. NanoBSD is fairly straight forward to configre and adapt. I ran into a number of nits, several related to the recent dhclient changes that have not fit well into the NanoBSD environment. I thought I'd provide a brief summary of what worked, and what didn't, on the way to having it up and running. I should point out that I'm now reaching the rest of the world through the box, and was pleased with how easy the NanoBSD work made it to trivially assemble flash images -- really a boon for this sort of thing. Kudos also to the Soekris guys -- it's a pleasure to work with a box that's so well tuned for running an open source OS in an embedded network environment. Nits are identified with an XXX in the left of the text. These are opportunities for people with interests in this space to investigate what went wrong :-). - First, I configured src/tools/tools/nanobsd/make.conf. - Primarily, this was to configure a kernel that did not have debugging enabled, disable SMP, and so on. I also enabled WITH_GEOM_VOL and GEOM_VOL in the kernel. See below for more on this. - I deleted NO_KLDLOAD and the NANOBSD_PRUNE of usr/share/misc. I'm too much of a vi guy to go without termcap entries. - I added NET4801 to the CUSTOMIZE= list. - I enabled CUSTOMIZE_ARGS=${.CURDIR}/localfiles. - I created a src/tools/tools/nanobsd/etc directory in which to place my local configuration files. In particular, rc.conf, ipfw.conf, and dhcpd.conf. - I created an rc.conf file. There are three if_sis interfaces in the net4801 system -- I used the first interface for an external network port, and the second and third interfaces for two different internal networks (one wired, one wireless). Due to the nature of the network environment I run in, it was necessary for me to configure a specific ethernet address as part of the ifconfig_sis0 line. I was pleased that it's now possible to combine directives with DHCP for ifconfig -- I was displeased that this only sort of worked. First nit: XXX As I posted last night, the ifconfig_sis0="ether XXX DHCP" line worked fine for launching dhclient and running ifconfig first, but it caused natd to fail to match the interface as requiring -dynamic, requiring me me to manually configure -dynamic for natd in rc.conf. - I downloaded the ISC dhcpd package to use for a DHCP server, and added a packages directive to localfiles to cause it to be installed on the image. Next nit. XXX It appears package installs involving user/group additions don't work with NanoBSD. Specifically, when the package is added, it looks like it tries to add the user to the build system's password/group files, and never end up in the DESTDIR. As a result, DHCPd couldn't switch to the dhcpd user. I added rc.conf lines to force DHCPd to use the existing _dhcp user, rather than modifying the src/etc files. Obviously, not pleased that my build system /etc files were modified either. - I configured dhcpd.conf and configured localfiles to install it in the right place. On boot, I run into the following as yet unresolved nits: XXX As I posted last night, dhclient now exits if there is no link, relying on devd to restart it. Unfortunately, devd relies on C++, and NanoBSD disables CXX, meaning that once dhclient detects a down link on a NanoBSD box, that link will never have IP configured on it again. This is a big problem. XXX Every boot, I get savecore warnings: Loading configuration files. kenv: unable to get dumpdev No suitable dump device was found. ... /etc/rc: WARNING: Dump device does not exist. Savecore not run. I've not yet tracked down why that is, but suspect a missing directive, no swap, or the like. This isn't worth an upper-case WARNING. XXX I get a sendmail warning every boot: sendmail: execing /usr/libexec/sendmail/sendmail: No such file or directory This is likely vi.recover running and trying to do something with sendmail. Unhelpful, given no sendmail is present. Maybe a better failure mode is desirable here? XXX I liked that it seemed easy to do incremental rebuilds with NanoBSD, but found I had to do a fair amount of manually deleting intermediate targets to get it to work. One thing I found that really didn't work was that if you install any third party packages, you have to blow away the DESTDIR install tree or when you re-build, the packages conflict and abort the image creation. XXX The following FAILURE message from ATA seems a bit extreme, perhaps the word warning would be more appropriate: ad1: FAILURE - SETFEATURES ENABLE RCACHE status=51 error=4 error=4