Chapter 23. Linux Mode

Table of Contents
23.1. How to Install the Linux Mode
23.2. How to Install Mathematica on FreeBSD
23.3. How does the Linux mode work?

Contributed by Brian N. Handy and Rich Murphey

23.1. How to Install the Linux Mode

Linux binary compatibility in FreeBSD has reached a point where it is possible to run a large fraction of Linux binaries in both a.out and ELF format. The Linux compatibility in the 2.1-STABLE branch is capable of running Linux DOOM and Mathematica; the version present in 3.3-RELEASE is vastly more capable and runs all these as well as Oracle8, WordPerfect, StarOffice, Acrobat, Quake, Abuse, IDL, and netrek for Linux and a whole host of other programs.

There are some Linux-specific operating system features that are not supported on FreeBSD. Linux binaries will not work on FreeBSD if they overly use the Linux /proc filesystem (which is different from the optional FreeBSD /proc filesystem) or i386-specific calls, such as enabling virtual 8086 mode.

Depending on which version of FreeBSD you are running, how you get Linux mode up will vary somewhat:

23.1.1. Installing Linux Mode in 3.0-RELEASE and later

It is no longer necessary to specify options LINUX or options COMPAT_LINUX. Linux binary compatibility is done with an KLD object (``Kernel LoaDable object'') so it can be installed on the fly without having to reboot. You will need the following things in your startup files, however:

  1. In /etc/rc.conf, you need the following line:

        linux_enable=YES
    
  2. This, in turn, triggers the following action in /etc/rc.i386:

        # Start the Linux binary compatibility if requested.
        if [ "X${linux_enable}" = X"YES" ]; then echo -n '
                linux';               linux > /dev/null 2>&1
        fi
    

If you want to verify that the KLD is loaded, kldstat will do that:

    % kldstat
    Id Refs Address    Size     Name
     1    2 0xc0100000 16bdb8   kernel
     7    1 0xc24db000 d000     linux.ko

If for some reason you do not want to or cannot load the Linux KLD, then statically link the binary compatibility in the kernel by adding

    options  LINUX
to your kernel config file. Then run config and install the new kernel as described in the kernel configuration section.

23.1.2. Installing Linux Mode in 2.2.2-RELEASE and later 2.2.x versions

It is no longer necessary to specify options LINUX or options COMPAT_LINUX. Linux binary compatibility is done with an LKM (``Loadable Kernel Module'') so it can be installed on the fly without having to reboot. You will need the following things in your startup files, however:

  1. In /etc/rc.conf, you need the following line:

        linux_enable=YES
    
  2. This, in turn, triggers the following action in /etc/rc.i386:

        # Start the Linux binary emulation if requested.
        if [ "X${linux_enable}" = X"YES" ]; then echo -n '
                linux';               linux > /dev/null 2>&1
        fi
    

If you want to verify that the LKM is running, modstat will do that:

    % modstat
    Type     Id Off Loadaddr Size Info     Rev Module Name
    EXEC      0   4 f09e6000 001c f09ec010   1 linux_mod

However, there have been reports that this fails on some 2.2-RELEASE and later systems. If for some reason you cannot load the Linux LKM, then statically link the Linux compatibility in the kernel by adding

    options  LINUX
to your kernel config file. Then run config and install the new kernel as described in the kernel configuration section.

23.1.3. Installing Linux Mode in 2.1-STABLE

The GENERIC kernel in 2.1-STABLE is not configured for Linux compatibility so you must reconfigure your kernel for it. There are two ways to do this: 1. linking the binary compatibility statically in the kernel itself and 2. configuring your kernel to dynamically load the Linux loadable kernel module (LKM).

To enable Linux binary compatibility, add the following to your configuration file (c.f. /sys/i386/conf/LINT):

    options COMPAT_LINUX

If you want to run doom or other applications that need shared memory, also add the following.

    options SYSVSHM

The Linux system calls require 4.3BSD system call compatibility. So make sure you have the following.

    options "COMPAT_43"

If you prefer to statically link the binary compatibility in the kernel rather than use the loadable kernel module (LKM), then add

    options  LINUX

Then run config and install the new kernel as described in the kernel configuration section.

If you decide to use the LKM you must also install the loadable module. A mismatch of versions between the kernel and loadable module can cause the kernel to crash, so the safest thing to do is to reinstall the LKM when you install the kernel.

    # cd /usr/src/lkm/linux
    # make all install

Once you have installed the kernel and the LKM, you can invoke linux as root to load the LKM.

    # linux
    Linux emulator installed
    Module loaded as ID 0

To see whether the LKM is loaded, run modstat.

    % modstat
    Type     Id Off Loadaddr Size Info     Rev
    Module Name EXEC      0   3 f0baf000 0018 f0bb4000   1 linux_emulator

You can cause the LKM to be loaded when the system boots in either of two ways. In FreeBSD 2.2.1-RELEASE and 2.1-STABLE enable it in /etc/sysconfig

    linux=YES
by changing it from NO to YES. FreeBSD 2.1 RELEASE and earlier do not have such a line and on those you will need to edit /etc/rc.local to add the following line.

    linux

23.1.4. Installing Linux Runtime Libraries

23.1.4.1. Installing using the linux_base port

Most Linux applications use shared libraries, so you are still not done until you install the shared libraries. It is possible to do this by hand, however, it is vastly simpler to just grab the linux_base port:

    # cd /usr/ports/emulators/linux_base
    # make all install

and you should have working Linux binary compatibility. Legend (and the mail archives :-) seems to hold that Linux mode works best with Linux binaries linked against the ZMAGIC libraries; QMAGIC libraries (such as those used in Slackware V2.0) may tend to give the Linux mode heartburn. Also, expect some programs to complain about incorrect minor versions of the system libraries. In general, however, this does not seem to be a problem.

23.1.4.2. Installing libraries manually

If you do not have the ``ports'' distribution, you can install the libraries by hand instead. You will need the Linux shared libraries that the program depends on and the runtime linker. Also, you will need to create a "shadow root" directory, /compat/linux, for Linux libraries on your FreeBSD system. Any shared libraries opened by Linux programs run under FreeBSD will look in this tree first. So, if a Linux program loads, for example, /lib/libc.so, FreeBSD will first try to open /compat/linux/lib/libc.so, and if that does not exist then it will try /lib/libc.so. Shared libraries should be installed in the shadow tree /compat/linux/lib rather than the paths that the Linux ld.so reports.

FreeBSD-2.2-RELEASE and later works slightly differently with respect to /compat/linux: all files, not just libraries, are searched for from the ``shadow root'' /compat/linux.

Generally, you will need to look for the shared libraries that Linux binaries depend on only the first few times that you install a Linux program on your FreeBSD system. After a while, you will have a sufficient set of Linux shared libraries on your system to be able to run newly imported Linux binaries without any extra work.

23.1.4.3. How to install additional shared libraries

What if you install the linux_base port and your application still complains about missing shared libraries? How do you know which shared libraries Linux binaries need, and where to get them? Basically, there are 2 possibilities (when following these instructions: you will need to be root on your FreeBSD system to do the necessary installation steps).

If you have access to a Linux system, see what shared libraries the application needs, and copy them to your FreeBSD system. Example: you have just ftp'ed the Linux binary of Doom. Put it on the Linux system you have access to, and check which shared libraries it needs by running ldd linuxxdoom:

    % ldd linuxxdoom
    libXt.so.3 (DLL Jump 3.1) => /usr/X11/lib/libXt.so.3.1.0
    libX11.so.3 (DLL Jump 3.1) => /usr/X11/lib/libX11.so.3.1.0
    libc.so.4 (DLL Jump 4.5pl26) => /lib/libc.so.4.6.29

You would need to get all the files from the last column, and put them under /compat/linux, with the names in the first column as symbolic links pointing to them. This means you eventually have these files on your FreeBSD system:

    /compat/linux/usr/X11/lib/libXt.so.3.1.0
    /compat/linux/usr/X11/lib/libXt.so.3 -> libXt.so.3.1.0
    /compat/linux/usr/X11/lib/libX11.so.3.1.0
    /compat/linux/usr/X11/lib/libX11.so.3 -> libX11.so.3.1.0
    /compat/linux/lib/libc.so.4.6.29 /compat/linux/lib/libc.so.4 -> libc.so.4.6.29

Note: Note that if you already have a Linux shared library with a matching major revision number to the first column of the ldd output, you will not need to copy the file named in the last column to your system, the one you already have should work. It is advisable to copy the shared library anyway if it is a newer version, though. You can remove the old one, as long as you make the symbolic link point to the new one. So, if you have these libraries on your system:

    /compat/linux/lib/libc.so.4.6.27
    /compat/linux/lib/libc.so.4 -> libc.so.4.6.27

and you find a new binary that claims to require a later version according to the output of ldd:

    libc.so.4 (DLL Jump 4.5pl26) -> libc.so.4.6.29

If it is only one or two versions out of date in the in the trailing digit then do not worry about copying /lib/libc.so.4.6.29 too, because the program should work fine with the slightly older version. However, if you like you can decide to replace the libc.so anyway, and that should leave you with:

    /compat/linux/lib/libc.so.4.6.29
    /compat/linux/lib/libc.so.4 -> libc.so.4.6.29

Note: The symbolic link mechanism is only needed for Linux binaries. The FreeBSD runtime linker takes care of looking for matching major revision numbers itself and you do not need to worry about it.

23.1.4.4. Configuring the ld.so --- for FreeBSD 2.2-RELEASE and later

This section applies only to FreeBSD 2.2-RELEASE and later. Those running 2.1-STABLE should skip this section.

Finally, if you run FreeBSD 2.2-RELEASE you must make sure that you have the Linux runtime linker and its config files on your system. You should copy these files from the Linux system to their appropriate place on your FreeBSD system (to the /compat/linux tree):

    /compat/linux/lib/ld.so
    /compat/linux/etc/ld.so.config

If you do not have access to a Linux system, you should get the extra files you need from various ftp sites. Information on where to look for the various files is appended below. For now, let us assume you know where to get the files.

Retrieve the following files (all from the same ftp site to avoid any version mismatches), and install them under /compat/linux (i.e. /foo/bar is installed as /compat/linux/foo/bar):

    /sbin/ldconfig
    /usr/bin/ldd
    /lib/libc.so.x.y.z
    /lib/ld.so

ldconfig and ldd do not necessarily need to be under /compat/linux; you can install them elsewhere in the system too. Just make sure they do not conflict with their FreeBSD counterparts. A good idea would be to install them in /usr/local/bin as ldconfig-linux and ldd-linux.

Create the file /compat/linux/etc/ld.so.conf, containing the directories in which the Linux runtime linker should look for shared libraries. It is a plain text file, containing a directory name on each line. /lib and /usr/lib are standard, you could add the following:

    /usr/X11/lib
    /usr/local/lib

When a Linux binary opens a library such as /lib/libc.so the Linux ABI support maps the name to /compat/linux/lib/libc.so internally. All Linux libraries should be installed under /compat/linux (e.g. /compat/linux/lib/libc.so, /compat/linux/usr/X11/lib/libX11.so, etc.) in order for the Linux ABI loader to find them.

Those running FreeBSD 2.2-RELEASE should run the Linux ldconfig program.

    # cd /compat/linux/lib
    # /compat/linux/sbin/ldconfig

ldconfig is statically linked, so it does not need any shared libraries to run. It creates the file /compat/linux/etc/ld.so.cache which contains the names of all the shared libraries and should be rerun to recreate this file whenever you install additional shared libraries.

On 2.1-STABLE do not install /compat/linux/etc/ld.so.cache or run ldconfig; in 2.1-STABLE the syscalls are implemented differently and ldconfig is not needed or used.

You should now be set up for Linux binaries which only need a shared libc. You can test this by running the Linux ldd on itself. Supposing that you have it installed as ldd-linux, it should produce something like:

    # ldd-linux `which ldd-linux`
    libc.so.4 (DLL Jump 4.5pl26) => /lib/libc.so.4.6.29

This being done, you are ready to install new Linux binaries. Whenever you install a new Linux program, you should check if it needs shared libraries, and if so, whether you have them installed in the /compat/linux tree. To do this, you run the Linux version ldd on the new program, and watch its output. ldd (see also the manual page for ldd(1)) will print a list of shared libraries that the program depends on, in the form majorname (jumpversion) => fullname.

If it prints not found instead of fullname it means that you need an extra library. The library needed is shown in majorname and will be of the form libXXXX.so.N. You will need to find a libXXXX.so.N.mm on a Linux ftp site, and install it on your system. The XXXX (name) and N (major revision number) should match; the minor number(s) mm are less important, though it is advised to take the most recent version.

23.1.5. Installing Linux ELF binaries

ELF binaries sometimes require an extra step of ``branding''. If you attempt to run an unbranded ELF binary, you will get an error message like the following;

    % ./my-linux-elf-binary
    ELF binary type not known
    Abort

To help the FreeBSD kernel distinguish between a FreeBSD ELF binary from a Linux binary, use the brandelf(1) utility.

    % brandelf -t Linux my-linux-elf-binary

The GNU toolchain now places the appropriate branding information into ELF binaries automatically, so you should be needing to do this step increasingly rarely in future.

23.1.6. Configuring the host name resolver

If DNS does not work or you get the messages

    resolv+: "bind" is an invalid keyword resolv+:
    "hosts" is an invalid keyword
then you need to configure a /compat/linux/etc/host.conf file containing:
    order hosts, bind
    multi on
where the order here specifies that /etc/hosts is searched first and DNS is searched second. When /compat/linux/etc/host.conf is not installed linux applications find FreeBSD's /etc/host.conf and complain about the incompatible FreeBSD syntax. You should remove bind if you have not configured a name-server using the /etc/resolv.conf file.

Lastly, those who run 2.1-STABLE need to set an the RESOLV_HOST_CONF environment variable so that applications will know how to search the host tables. If you run FreeBSD 2.2-RELEASE or later, you can skip this. For the /bin/csh shell use:

    % setenv RESOLV_HOST_CONF /compat/linux/etc/host.conf

For /bin/sh use:

    % RESOLV_HOST_CONF=/compat/linux/etc/host.conf; export RESOLV_HOST_CONF

23.1.7. Finding the necessary files

Note: The information below is valid as of the time this document was written, but certain details such as names of ftp sites, directories and distribution names may have changed by the time you read this.

Linux is distributed by several groups that make their own set of binaries that they distribute. Each distribution has its own name, like ``Slackware'' or ``Yggdrasil''. The distributions are available on a lot of ftp sites. Sometimes the files are unpacked, and you can get the individual files you need, but mostly they are stored in distribution sets, usually consisting of subdirectories with gzipped tar files in them. The primary ftp sites for the distributions are:

  1. sunsite.unc.edu:/pub/Linux/distributions

  2. tsx-11.mit.edu:/pub/linux/distributions

Some European mirrors:

  1. ftp.luth.se:/pub/linux/distributions

  2. ftp.demon.co.uk:/pub/unix/linux

  3. src.doc.ic.ac.uk:/packages/linux/distributions

For simplicity, let us concentrate on Slackware here. This distribution consists of a number of subdirectories, containing separate packages. Normally, they are controlled by an install program, but you can retrieve files ``by hand'' too. First of all, you will need to look in the contents subdirectory of the distribution. You will find a lot of small text files here describing the contents of the separate packages. The fastest way to look something up is to retrieve all the files in the contents subdirectory, and grep through them for the file you need. Here is an example of a list of files that you might need, and in which contents-file you will find it by grepping through them:

Library Package
ld.so ldso
ldconfig ldso
ldd ldso
libc.so.4 shlibs
libX11.so.6.0 xf_lib
libXt.so.6.0 xf_lib
libX11.so.3 oldlibs
libXt.so.3 oldlibs

So, in this case, you will need the packages ldso, shlibs, xf_lib and oldlibs. In each of the contents-files for these packages, look for a line saying PACKAGE LOCATION, it will tell you on which ``disk'' the package is, in our case it will tell us in which subdirectory we need to look. For our example, we would find the following locations:

Package Location
ldso diska2
shlibs diska2
oldlibs diskx6
xf_lib diskx9

The locations called ``diskXX'' refer to the slakware/XX subdirectories of the distribution, others may be found in the contrib subdirectory. In this case, we could now retrieve the packages we need by retrieving the following files (relative to the root of the Slackware distribution tree):

  • slakware/a2/ldso.tgz

  • slakware/a2/shlibs.tgz

  • slakware/x6/oldlibs.tgz

  • slakware/x9/xf_lib.tgz

Extract the files from these gzipped tarfiles in your /compat/linux directory (possibly omitting or afterwards removing files you do not need), and you are done.

See also: ftp://ftp.FreeBSD.org/pub/FreeBSD/2.0.5-RELEASE/xperimnt/linux-emu/README and /usr/src/sys/i386/ibcs2/README.iBCS2