From owner-cvs-all Sat Apr 27 7: 5:25 2002 Delivered-To: cvs-all@freebsd.org Received: from whale.sunbay.crimea.ua (whale.sunbay.crimea.ua [212.110.138.65]) by hub.freebsd.org (Postfix) with ESMTP id 2423037B417; Sat, 27 Apr 2002 07:04:58 -0700 (PDT) Received: (from ru@localhost) by whale.sunbay.crimea.ua (8.11.6/8.11.2) id g3RE4iK40120; Sat, 27 Apr 2002 17:04:44 +0300 (EEST) (envelope-from ru) Date: Sat, 27 Apr 2002 17:04:43 +0300 From: Ruslan Ermilov To: John Baldwin Cc: Marcel Moolenaar , cvs-committers@FreeBSD.org, cvs-all@FreeBSD.org, Makoto Matsushita Subject: Re: cvs commit: src Makefile Makefile.inc1 src/etc Makefile src/ Message-ID: <20020427140443.GA35685@sunbay.com> References: <20020427033118.GA583@athlon.pn.xcllnt.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="9amGYk9869ThD9tj" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.3.27i Sender: owner-cvs-all@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --9amGYk9869ThD9tj Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Apr 27, 2002 at 12:23:32AM -0400, John Baldwin wrote: >=20 > On 27-Apr-2002 Marcel Moolenaar wrote: > > On Fri, Apr 26, 2002 at 11:10:22PM -0400, John Baldwin wrote: > >>=20 > >> distribute stuff already uses what is in /usr/obj, so it would just si= mply > >> involve adding an extra buildworld after the world. I would actually = prefer > >> that as it still preserves the "clean room" atmosphere that a release = is > >> supposed to have. > >=20 > > How would this work for a cross-release? The bits in /usr/obj may not > > be for the architecture you're building the release on. >=20 > The cross-release kind of would work like this: >=20 > 1. You start off with matching src/ and obj/ like we do now. > 2. You install that obj/ into the chroot. > 3. You do a non-cross make world in the chroot like we do now. > 4. What we do next is do all the make distribute stuff using > what is in /usr/obj in the chroot. >=20 > We do step 3 to try to get "clean room" binaries to use in stage 4. >=20 > Now for a cross-release we need to make sure the binaries in /usr/obj > in the chroot are cross-built binaries. Ruslan's current approach is > to do this by make step 3 be a cross-buildworld instead of a full world. > This means, then, that any tools for the new world you need for 4 > need to be built as build-tools or cross-tools or the like. What I'm > suggesting is instead to insert a step at 3.5 to do a cross-build world > and then we still have the right tools installed and don't have to worry > about using the right build/cross tools for the release scripts. >=20 After so much noise, let me tell you the truth now. :-) The old version worked like this (the numbering does not resemble the release.X steps): 1. Install the same world that you're currently running into a chroot environment. 2. Check out fresh sources in the chroot's /usr/src and chroot into there. (The next steps are done in the chroot'ed environment.) 3. Build world from /usr/src and install it under root. This has the following disadvantages: o Installworld from step 1 should install the world _compatible_ with the world built in this step, otherwise the new world may not be able to run with the current kernel. That means that you should run a relatively fresh (ideally, the same /usr/src stuff as in chroot) world before rolling a release. That's why I wrote that my patch obviates the need to have an up-to- date world. o You can't install and run the world for a different arch. 4. Distribute the same world under /R/stage/trees, and make additional distributions like krb4, krb5 and fix non-crypto stuff (FIXCRYPTO). (The old version fixed non-crypto stuff later, for no apparent reason.) 5. Rebuild GENERIC and additional kernels in the new world environment. 6. Build crunched binaries, boot and fixit. 7. Create distribution tarballs using some sample tools like sed(1) and tar(1). 8. Build mfsroot.flp: copy some misc. stuff from the either the / (e.g. /sbin/dhclient-script) or /usr/src (etc/services), set up the "boot" crunched binary from step 6, copy the necessary device nodes, docs, boot blocks, etc., unnecessarily re-make _all_ the GENERIC kernel's modules we already built in step 5, and install a subset of them under mfsfd/stand/modules, and finally create a floppy image with this stuff with the doFS.sh script. 9. Create the BOOTMFS kernel config, compile the BOOTMFS kernel, and install it in two incarnations, small (kern.flp) and big (boot.flp) that also has a copy of mfsroot.flp. 10. Create a fixit.flp using the "fixit" crunched binary we built in step 6 and copy some stuff from the "base" dist. 11. Set up FTP and CD-ROM distributions. That's how it works now: 1. Install the same world that you're currently running into a chroot environment. Note: this world doesn't need to be very fresh, it should only be compatible with the currently running kernel, for obvious reasons. 2. Check out fresh sources in the chroot's /usr/src and chroot into there. (The next steps are done in the chroot'ed environment.) 3. Build world from /usr/src but _do not_ install it under root. This has the following advantages: o Installworld from step 1 _does not_ need to be compatible with the world built in this step. o We may build world for a different arch (no knobs are provided in this milestone commit). 4. Distribute the world we built in step 3 under /R/stage/trees, and make additional distributions like krb4, krb5 and fix non-crypto stuff (FIXCRYPTO). Note the new way how additional distributions are built: because we do not install the new world (it may be for a different arch or be incompatible with the currently running kernel) they are built the same way as the main world, as the sub-world (thanks to the SUBDIR_OVERRIDE knob) using the right cross-tools (compiler, binutils, etc.), libraries and headers from ${WORLDTMP}. 5. Rebuild GENERIC and additional kernels using the "buildkernel" Makefile.inc1 target which uses the right cross-tools. 6. Build crunched binaries, boot and fixit, using the right cross-tools, libraries and headers. This is done by importing the WMAKEENV idea from ../Makefile.inc1. Note: we may need to bootstrap crunchgen(1) to do it. (Just to avoid any possible ambiguities, bootstrapping means "rebuild crunchgen(1) from /usr/src in the running (host) environment, using the /usr/bin/cc compiler, /usr/lib libraries, and /usr/include headers, and install them under root. But so far (even for 4.x doing the 5.0 release) this is not necessary. Anyway, I plan to fix it so that we eliminate any possible problems if someone (joe@?) adds some nice new stuff to crunchgen(1). :-) 7. Create distribution tarballs using some sample tools like sed(1) and tar(1). This step was not even touched by my changes because it does not depend on the new world's binaries (unless we introduce some incompatibilities in sed(1) and tar(1), in which case we'd need to bootstrap them too the same way as we (will) bootstrap crunchgen(1). 8. Build mfsroot.flp: copy some misc. stuff from the "base" dist (compare with the old version) or /usr/src, set up the "boot" crunched binary from step 6, copy the necessary device nodes, docs, boot blocks (from "base" dist), etc., install the subset of the GENERIC kernel's modules we already built in step 5, and finally create a floppy image with this stuff with the doFS.sh script, making sure to use the boot blocks from the "base" dist (remember: we are aimed at different arch support). 9. Create the BOOTMFS kernel config, compile the BOOTMFS kernel with Makefile.inc1:buildkernel, using the right cross-tools, and install it in two incarnations, small (kern.flp) and big (boot.flp) that also has a copy of mfsroot.flp. (See how doMFSKERN was fixed to achieve this.) I have some minor optimizations planned for this step too like avoiding the unnecessary "buildkernel" step on the second invokation. 10. Create a fixit.flp using the "fixit" crunched binary we built in step 6 and copy some stuff from the "base" dist. This stage wasn't modified. 11. Set up FTP and CD-ROM distributions. These stages were not modified either. I have intentionally excluded the doc.1 and ports.1 steps from this discussion, because they seem to be not affected by these changes. You'll probably not be surprised now that the milestone #2 commit that will bring us to the level of supporting cross-arch "make release"s will be a really minor patch for release/Makefile that will: 1. Borrow the TARGET_ARCH and TARGET definitions from Makefile.inc1. 2. Rename MACHINE_ARCH -> TARGET_ARCH and MACHINE -> TARGET all around release/Makefile. 3. Ensure world-related targets are passed the TARGET_ARCH and TARGET. [4. There will probably some other fixups that I missed in the milestone #1 sweep.] Cheers, --=20 Ruslan Ermilov Sysadmin and DBA, ru@sunbay.com Sunbay Software AG, ru@FreeBSD.org FreeBSD committer, +380.652.512.251 Simferopol, Ukraine http://www.FreeBSD.org The Power To Serve http://www.oracle.com Enabling The Information Age --9amGYk9869ThD9tj Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (FreeBSD) Comment: For info see http://www.gnupg.org iD8DBQE8yq/7Ukv4P6juNwoRAoJ9AJ44TjlROn8H3EAiVZBXPreVPMT1IACePOp9 qRARZvMUtUiGP81QeanvPes= =88MI -----END PGP SIGNATURE----- --9amGYk9869ThD9tj-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe cvs-all" in the body of the message