From owner-freebsd-current@FreeBSD.ORG Mon Jan 25 22:33:18 2010 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C3A591065697 for ; Mon, 25 Jan 2010 22:33:18 +0000 (UTC) (envelope-from christof.schulze@gmx.com) Received: from mailout-eu.gmx.com (mailout-eu.gmx.com [213.165.64.42]) by mx1.freebsd.org (Postfix) with SMTP id 3BB9A8FC15 for ; Mon, 25 Jan 2010 22:33:18 +0000 (UTC) Received: (qmail invoked by alias); 25 Jan 2010 22:33:16 -0000 Received: from e181212122.adsl.alicedsl.de (EHLO klausdieter0815.dyndns.org) [85.181.212.122] by mail.gmx.com (mp-eu003) with SMTP; 25 Jan 2010 23:33:16 +0100 X-Authenticated: #56306756 X-Provags-ID: V01U2FsdGVkX1+6Avka+2K6zh5fExberuoH67eFqf6m8jzQ8vN729 4NMiyA7geGP3fC Received: by myhost.mydomain.de (Postfix, from userid 1001) id 6A6BC768A; Mon, 25 Jan 2010 23:33:11 +0100 (CET) From: Christof Schulze To: freebsd-current@freebsd.org Date: Mon, 25 Jan 2010 23:32:58 +0100 User-Agent: KMail/1.12.4 (FreeBSD/8.0-STABLE; KDE/4.3.4; amd64; ; ) References: <201001231233.18832.naylor.b.david@gmail.com> <201001252039.16220.naylor.b.david@gmail.com> In-Reply-To: <201001252039.16220.naylor.b.david@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart8493944.lMmWUNsqzN"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <201001252333.08045.christof.schulze@gmx.com> X-Y-GMX-Trusted: 0 X-FuHaFi: 0.5 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: "tinderbox" using stacked unionfs X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jan 2010 22:33:18 -0000 --nextPart8493944.lMmWUNsqzN Content-Type: multipart/mixed; boundary="Boundary-01=_hwhXLcMfEjAV8bv" Content-Transfer-Encoding: 7bit --Boundary-01=_hwhXLcMfEjAV8bv Content-Type: Text/Plain; charset="iso-8859-6" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Hi David, I like your unionfs idea. Noticing the tinderbox limitation I moved on my z= fs=20 based system to a jail I set up using a self written script. It is currentl= y=20 customized to my environment but still might prove useful for quick setup o= f=20 jails. it handles=20 * fs creation=20 * putting the proper data on them * routing setup (since I use a laptop and my active nic changes from cable- bound to wireless) Since this both contributes to the topic of building software in a separate= =20 environment, I share it here.=20 Regards Christof Am Montag 25 Januar 2010 19:39:11 schrieb David Naylor: > On Saturday 23 January 2010 12:33:14 David Naylor wrote: > > Hi, > > > > I have been experimenting with using unionfs to build ports in a > > "tinderbox" environment. This avoids having to remove and extract fil= es > > for the build of each port and it allows the discovery of > > installed/modified files by the port. > > > > Please see attached for a (updated) shell script that handles all the > > "heavy lifting" of building ports. Of importance is LOCALBASE and > > BUILDDIR. If you want to override LOCALBASE please use `env` as the > > script needs to know about it. BUILDDIR (/usr/build by default) is > > where the script stores everything (including PKG_DBDIR). >=20 > Please see attached for an updated script. This no longer uses `sort -u` > but removed duplicates while maintaining dependency order. (See below) >=20 > > # env LOCALBASE=3D/tmp/local BUILDDIR=3D/tmp/build ./ports-union-builde= r.sh > > > > Will install x11/xorg without affecting already installed systems. > > > > CURRENT STATUS: > > - *** Currently kernel stack size is too small and the above will > > trigger a stack overflow. Recompiling a kernel with ``options > > KSTACK_PAGES=3D32'' will alleviate that problem. >=20 > In building xorg there were at least 207 stacked unionfs (206 ro, 1 rw, a= ll > noatime). >=20 > > - Currently there is a build problem that affects eggdbus/polkit > > (possibly others) thus preventing x11/xorg from being built. I will > > investigate the cause (help welcome). >=20 > This is due to not mounting the dependencies in the correct order. If > dependency 'a' modified file from dependency 'b' then mounting in order > 'a', 'b' will result in those changes being lost as the original files > from 'b' will supersede 'a'. The dependencies need to be mounted 'b', > 'a'. >=20 > This has been fixed although may cause a problem if multiple "independent" > ports modify the same file. This is a detectable problem. >=20 > > - I highly recommend running this in a chroot > > - NO WARRANTY, SLIPPERY WHEN WET, EATS CHILDREN. >=20 > - I am experiencing process freeze (anything involved in the directories > that are unionfs). Any way that I can figure out the problem? I can run > a kernel will full debug capability. > - mtree does not behave well with unionfs and consumed a fair amount of > resources: > # /usr/sbin/mtree -U -f /usr/ports/Templates/BSD.local.dist -d -e -p > /usr/local/ > took a long time (many minutes) to complete. >=20 > > Since the script doesn't complete a full build I am unable to compare t= he > > build speeds (thus the overhead of unionfs) but here are some partial > > results (with FORCE_MAKE_JOBS and quad core): >=20 > The script is now able to complete building but not at once due to process > freezing. >=20 > Please help with debugging the process freezing. (There is a LOR I have > reported for unionfs: kern/141950) >=20 > Regards >=20 > David >=20 =2D-=20 () ascii ribbon campaign - against html e-mail=20 /\ www.asciiribbon.org - against proprietary attachments --Boundary-01=_hwhXLcMfEjAV8bv-- --nextPart8493944.lMmWUNsqzN Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.13 (FreeBSD) iEYEABECAAYFAkteHCIACgkQpZfyPAmdZJnTqgCglEYg/Eb5a8aVJ6SBfs0PPN1O 8ToAoK7AxTAsBVMEMxVrD4nJ8wkmF6Kt =uDzN -----END PGP SIGNATURE----- --nextPart8493944.lMmWUNsqzN--