Skip site navigation (1)Skip section navigation (2)
Date:      Sat, 30 Aug 2014 00:20:46 -0700 (PDT)
From:      Don Lewis <truckman@FreeBSD.org>
To:        gjb@FreeBSD.org
Cc:        stable@FreeBSD.org, ian@FreeBSD.org
Subject:   Re: building an 8.4-STABLE i386 poudriere jail on an 10.0-STABLE amd64 host
Message-ID:  <201408300720.s7U7KkDk073975@gw.catspoiler.org>
In-Reply-To: <20140830024255.GL1200@hub.FreeBSD.org>

next in thread | previous in thread | raw e-mail | index | archive | help
On 29 Aug, Glen Barber wrote:
> On Fri, Aug 29, 2014 at 05:43:18PM -0700, Don Lewis wrote:
>> I simplified things a bit and am now just doing normal crossbuilds.  I'm
>> still seeing core dumps, even on amd64 -> amd64 crossbuilds.  I'm not
>> seeing a reason, though ...
>> 
>> [gdb backtrace]
>> 
>> [...]
>> 
>> Looks like all the pointers are OK, so why the SIGBUS?
>> 
> 
> Pardon any incoherency in what follows, because it is mostly based off
> smashed-together recollection from a text file of notes with the 8.x
> (and prior) cross builds.
> 
> The TL;DR is, there's probably not much information I can provide that
> will be useful to you in debugging this.  Sorry.  :(
> 
> (And Warner, if you're listening, I know your stance on the topic, so
> no need to re-enlighten me.) :-)
> 
> While it may sounds like I'm going in a different direction than what
> you're try to solve, there actually is a reason for it, and a possible
> solution for your case.
> 
> The issues you are running into are 80% of the reason that after 8.4 was
> released, there were no 8.4-STABLE snapshots produced.  The other 20% is
> split up with 10% because of how the release builds for 8.x and earlier
> work, and 10% because it was not reasonable to implement "special" code
> in the scripts I use for a single branch.  If this was a few years ago,
> and 7.x (and possibly 6.x) was still supported, I may have considered
> it.  That said, when I say "8.x" below, I really mean "8.x and prior
> branches."
> 
> My primary issue with how the 8.x release builds worked is that it would
> use chroot(8) by default, specifically:
> 
>     1) buildworld
>     2) installworld DESTDIR=/blah
>     3) chroot /blah make -C /usr/src buildworld buildkernel
>     4) chroot /blah make -C /usr/src/release release

I did a lot of release builds in the days of 4.x and that sounds
familiar ...

> The 9.x and later release builds do not do this.  You specify the target
> directory for the resulting build, and a bunch of magic happens, and
> when it's done, you have all the things to install FreeBSD.  The major
> benefit to this is that it does *not* use chroot(8) by default.
> 
> The 8.4-RELEASE was built on a machine running 9-STABLE, so make(1),
> gcc(1), and so on were not that far diverged for there to be any real
> issues (such as what you're seeing).

Interestingly enough, make and clang don't seem to be an issue so far as
I can tell.  I do have the fmake port installed because poudriere told
me that it was necessary, but if I just cd to the directory containing
the source for 8-STABLE, "make buildworld" mostly seems to work.  The
incompatibility of the patch -b option was the first problem that I ran
into, and that just required a few individual Makefile edits.  The
second problem was the lex upgrade during sometime after 10 was
branched, I worked around that by making lex a bootstrap tool using the
existing knob.  Yacc was already a bootstrap tool.

The final problem that is vexing me is that the cross-tool version of
lint.  The source for lint is basically unchanged between 8-STABLE and
10-STABLE.  The only significant change that I noticed was a one line
patch   to lint1/tree.c (and lint1 is the SIGBUS victim) documented
here: <https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=152549>; and
fixed in r224702.  I applied that patch and saw no change.

If I run the cross version of lint with the -B option to get it to use
the host version of lint1, then things work.  It's just the cross
version of lint1 that is broken.  Both the host and cross versions of
lint1 are compiled with clang, from nearly identical source code, though
the cross version was built with a parser and lexer gernerated using the
bootstrap versions of lex and yacc.  What's really maddening is that I
don't see anything wrong if I point gdb at the core file.  The next
thing I'm going to try is to try to run lint1 under gdb and single step
it.  That's tricky because lint1 is run from lint and I need to figure
out how to set up its arguments and input file.

Oh, and I discovered that neither "truss -fa" nor ktrace output anything
for exec*() calls.
 
> The snapshot builder in use now, at that time was running 10-CURRENT.
> Several "hacks" were in place, such as WITH_GNUCXX (which somehow turned
> into 4 or more src.conf(5) entries by now), which would allow building
> 8.x on 10-CURRENT to somewhat work.  That did not last long, though.

Oh yeah ... I stumbled across that problem as well.  I think I only
needed WITH_GCC and WITH_GNUCXX.

> I think it was around when bmake became the default in -CURRENT that
> I stopped spending time trying to build 8.x snapshots -- at that point,
> the time spent debugging the build failures and updating the scripts to
> "fix" the build became increasingly less beneficial.
> 
> Now, having said all that, I have a possible solution that, quite
> honestly, may or may not work (I have no idea at this point).
> 
> You can grab the 8.4-RELEASE distribution set from FTP, and extract it
> into a scratch directory, check out the stable/8/ tree into the scratch
> directory usr/src/, then do:
> 
>     # chroot /scratch make -C /usr/src buildworld [...]
> 
> That would at least provide you with the binary bits you want for your
> package builds.

I've actually got a couple of 8-STABLE boxes here already.  They are
just underpowered for mass building packages, which is why I'm not just
running "poudriere bulk" on one of them.  The other missing piece is
going from there to a working poudriere jail while not using the "jail
-c" command.

Hmn ... maybe I could build the jail on one of my 8.4-STABLE machines
and just copy the jail bits over ...

> I *think* this is somewhat what the cluster package builders do, but
> then again, I do not know off-hand if packages for 8.x are build for
> pkg(8) - they may be only pkg_install(1).

I think they are building pkg(8) packages, but they just use
8.4-RELEASE, plus security updates I imagine.  I've been able to do
that here for testing individual package builds on 8.x.

> Again, sorry that I do not provide a "just do $this" kind of magic wand
> in this case, especially since I've been in the same painful situation
> you are in.  But I hope at least some of this information is useful to
> you.
> 
> Glen
> 




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201408300720.s7U7KkDk073975>