Date: Mon, 15 Jun 2015 11:23:00 -0600 From: Warner Losh <imp@bsdimp.com> To: Adrian Chadd <adrian@freebsd.org> Cc: "freebsd-embedded@freebsd.org" <embedded@freebsd.org>, "freebsd-mips@freebsd.org" <freebsd-mips@freebsd.org> Subject: Re: Starting to cross-compile non-base software for MIPS - what it's like Message-ID: <B29CEB72-367C-4295-BD93-2A5C417D0553@bsdimp.com> In-Reply-To: <CAJ-VmonSXYdLP_VpGh%2B3N_WfGwDLqWGcuPvOCf4VkOd0pJ5QWg@mail.gmail.com> References: <CAJ-VmonSXYdLP_VpGh%2B3N_WfGwDLqWGcuPvOCf4VkOd0pJ5QWg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_2B777C26-1D06-4FE9-96C4-978352D20094
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=utf-8
> On Jun 14, 2015, at 1:13 PM, Adrian Chadd <adrian@freebsd.org> wrote:
>=20
> Hi!
>=20
> I've been hacking at cross compiling things on FreeBSD for a while,
> and I have made some progress. This email is less of a "how you can do
> it" and more of a "these are the roadblocks that prevent it from
> working out of the box."
>=20
> I'm using gcc-4.9.2 from ports - there's a cross compiler for
> mips/mips64, and there's mips-binutils/mips64-binutils. I'm using
> those to compile world and kernel. The kernel works; world requires
> some patching to compile. I'll write a post when I have a complete
> system booting/running from gcc-4.9.2.
>=20
> My test application is dropbear. Ie, I'm cross-compiling dropbear for
> mips from freebsd/i386.
>=20
> * There are some issues in libgcc - I have a patch that I've emailed
> out for review. I'll do some more testing and commit it next week when
> I know it works.
> * There are some environment variables that need to be set before I
> compile dropbear. Here's what it looks like:
>=20
> CPU_ARGS=3D"-march=3Dmips32 -msoft-float -Wa,-msoft-float"
> =
INCS=3D"-I/home/adrian/work/freebsd/head-embedded-2/root/mips/usr/include"=
>=20
> export CROSS_COMPILE=3Dmips-portbld-freebsd11.0
> export CC=3D${CROSS_COMPILE}-gcc
> export CXX=3D${CROSS_COMPILE}-g++
> #export LD=3D${CROSS_COMPILE}-ld
> #export AR=3D${CROSS_COMPILE}-ar
> #export RANLIB=3D${CROSS_COMPILE}-ranlib
> #export STRIP=3D${CROSS_COMPILE}-strip
> export LD=3Dmips-freebsd-l
> export AR=3Dmips-freebsd-ar
> export RANLIB=3Dmips-freebsd-ranlib
> export STRIP=3Dmips-freebsd-strip
> export SYSROOT=3D/home/adrian/work/freebsd/head-embedded-2/root/mips/
> export CFLAGS=3D"--sysroot=3D$SYSROOT ${CPU_ARGS} ${INCS} -O"
> export CXXFLAGS=3D"--sysroot=3D$SYSROOT ${CPU_ARGS} ${INCS} -O"
> export CPPFLAGS=3D"--sysroot=3D$SYSROOT ${CPU_ARGS} ${INCS} -O"
> export LDFLAGS=3D--sysroot=3D$SYSROOT
>=20
> # now run:
> # /configure --host=3Dmips-portbld-freebsd11.0
> # gmake
>=20
> ... then I hit a problem with how we populate links in /usr/lib and
> such. TL;DR - the symlinks we use are not relative paths, they're
> absolute paths - and this makes cross building using sysroot /
> explicit library paths fail to work right. Sigh. In particular, I
> needed to manually undo the libgcc symlink for it to compile.
> compilation complained about other libraries, but the resulting
> binaries did run.
That=E2=80=99s one of about 6 reasons that what we produce isn=E2=80=99t =
a
sysroot. We need to create a sysroot target that=E2=80=99s very close
to the xdev targets with all the compiler bits stripped out. We=E2=80=99re=
close though. I have about 1/2 of the patch needed to remove
xdev entirely and replace it with sysroot.
You shouldn=E2=80=99t need the INCS args if you have a proper sysroot.
And the CPU_ARGS is an indication we=E2=80=99re building our port
incorrectly. This is the same feedback I gave on the kernel
make changes you suggested.
> * yes, I tested the dropbear/dbclient binaries in a mips qemu VM (not
> qemu-mips-user, but a full mips MALTA VM.) Both worked fine.
>=20
> * the sysroot setup didn't correctly set the include path stuff right
> - I need to chase that down. Ie, it was still searching in
> /usr/include/ rather than ${SYSROOT}/usr/include/ . The library code
> obeyed the library searching, except for that hardlink.
That sounds like a bug in the default path of our gcc port.
> So, the very basics of "how can we cross compile things to run inside
> a freebsd-mips machine" work fine. It's amusing and annoying that the
> sysroot thing popped up. I do wonder if this class of problem keeps
> popping up when people do builds from one freebsd version to another;
> but I currently have no clear idea how to automate detecting this.
For one version to another we have a very controlled and constrained
environment. Inside of that, there are enough people building in =
purposely
broken environments to detect regressions. Once you involve configure
and foreign software things become rather unconstrained...
> On the plus side, it did work.
I=E2=80=99ve had hacks like this for =E2=80=9Csimple=E2=80=9D arts for =
years. It=E2=80=99s why I wrote
xdev in the first place. I had it working in the FreeBSD 6 time frame
for cross building some ports. But once you get past a certain
class of very well behaved ports, it goes to hell in a hurry.
> So:
>=20
> * I'll try to wrangle juli/bsdimp to review the libgcc change I have
> so userland compiles using gcc-4.9 with warnings disabled.
I=E2=80=99ll be happy to take a look.
> * I'll try to wrangle juli/bsdimp to review the kernel changes needed
> to compile up the kernel with gcc-4.9.
I=E2=80=99ve already reviewed that one. gcc isn=E2=80=99t being compiled =
correctly
and that need to be fixed. This is the forth time I=E2=80=99ve given =
this
feedback.
> * I'd like some help / comments on what we can do about the absolute
> symlinks used for various things inside an installed root. Ideally
> we'd have an option that would let us populate only relative links -
> that'd instantly make this problem go away.
All installworld-ish targets should use relative path symbolic links.
The fact they are using absolute paths is a bug. In the FreeBSD 6
time frame, I fought this battle at work and it wasn=E2=80=99t hard to =
fix. I
no longer have access to those sources, and I honestly can=E2=80=99t
recall how I fixed it (it may have been in a post-processing
step).
> * I'll go hit up bapt and some netbsd friends for some help with
> expected sysroot behaviour on gcc/binutils. It's possible there are
> still bugs with the dynamic sysroot (ie, when you don't compile your
> toolchain with an explicit sysroot to something other than '/') that
> need addressing.
bapt and I have been talking about this for a while. We=E2=80=99ve both =
been
busy doing other things so haven=E2=80=99t focused on it other than in =
off
moments.
> Then I'll worry about starting to convert a few ports to be native
> cross building.
You should talk to bapt about the rather extensive work he=E2=80=99s =
done
in this area, which is why I=E2=80=99ve looked at transitioning the xdev
support to just sysroot support.
Warner
--Apple-Mail=_2B777C26-1D06-4FE9-96C4-978352D20094
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=signature.asc
Content-Type: application/pgp-signature;
name=signature.asc
Content-Description: Message signed with OpenPGP using GPGMail
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - https://gpgtools.org
iQIcBAEBCgAGBQJVfwn1AAoJEGwc0Sh9sBEAMI0P/j12ZOpHoOHhBkn0crUdLTR8
FRskASw84nMM5/h0vOkVlaBVrWxj/pRSbl7ysUhMXWC0CHx3RK5tKqrDby0A7TFj
UurUZE1kmD7oUacX/tAMlnZuL5di2jopmqntprY83HvZoUsXsoeoDhTPxOXjW5KB
c7UhbCZPCZkxXMNWUFIAPs/DI5HfpfxQxZmtq3HyLPuQOlzQt+Q24nocQzRXxT5b
PcKSd0pN+F/Km6CiPcZzdg0oGefuwLZ+oOMrJH/igiqR94WmYypOvksb/E+H5mEa
6kpGDJuzrBueVy/Rx0TRK+MhlsGcM3zFtYmpmU9H06smR9RaYizka2NLqA38gop5
gvoZvELK0050+cjUBFuoS2hM8e258mkphfr+HKAsittZtT1HX7A8OA0W3AID6Vi6
xYvGisMllvHqN9cnGj3pNnbXRqeYx6RAJylmJnnAvGjbTw4W8C6XcD/07kzBxO3M
H5qgTnn/STF0TwwlA4o6JxX0XF1mKXtGP8pvBMHun77yfQ1tBTuLEUHrCuK7Uar4
UbPpu0BgOapEhDsVZUK+MiMcUy88n1kxRoBkgzBQXsQ9JMdd2pnOAyQh+xL5tHQg
GFg2UeyPRok1EkaJJLGTEXlwp6CT9xq0FufC+b3JBM46qZQLvio9LgxM97nLUDCZ
YSSgptmDj+RtNJeJdXHS
=J58A
-----END PGP SIGNATURE-----
--Apple-Mail=_2B777C26-1D06-4FE9-96C4-978352D20094--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?B29CEB72-367C-4295-BD93-2A5C417D0553>