Date: Fri, 6 Nov 1998 14:57:48 -0800 (PST) From: asami@FreeBSD.ORG (Satoshi Asami) To: jkh@time.cdrom.com Cc: mike@smith.net.au, mark@grondar.za, sos@FreeBSD.ORG, ports@FreeBSD.ORG Subject: Re: Who built XFree86 with Kerberos? Message-ID: <199811062257.OAA10011@silvia.hip.berkeley.edu> In-Reply-To: <15699.909845800@time.cdrom.com> (jkh@time.cdrom.com)
next in thread | previous in thread | raw e-mail | index | archive | help
* > very hard to detect on the building machine. The scheduling, which * > didn't leave him any time to put the ELF packages up for testing * > before the release, is at fault. * * You can say that again. Despite having plenty of warning, and we're * talking several month's worth here, the task was apparently dumped at * the very last moment on Justin and Steve. I know that Steve certainly * wasn't very happy about how this was done and it certainly took Mike * and I by surprise as well - there was some even doubt, for awhile * there, as to whether 3.0 would ship with a packages collection at all. You are right about the splitting-up part. I was going to do it myself, but couldn't quite do it and had to ask help from Steve in the last moment. I am really grateful (and sorry) about that. As for the package building itself, I think you got the wrong impression. It was the idea from the very beginning that Justin was going to build 3.0 packages. He's been the one that's been building packages-current throughout its lifetime. (In fact, I have never built packages-current since it split from packages-stable.) * My suggestion would be that we either: a) go to some sort of * automation scheme which [re]builds packages on an ongoing basis and, * at release time, I just take a snapshot of the collection or b) We try * appointing a ports/packages releasemeister who's responsible for doing * in ports what I do for src each time a release rolls around. This * person could be paid since it's important to Walnut Creek CDROM * that packages and distfiles be rendered into ISO images periodically, * especially now that we have this "toolkit" CD for FreeBSD in addition * to the usual 4 CD sets. (a) is not going to work. The ports collection is just not uniform enough (like src) that it requires a lot of hand-tweaking to build. The problem is that for most committers, the ports collection is his ports and some other ports it depends on. When you try to build the entire thing, all hell breaks loose. See all the MANUAL_PACKAGE_BUILD and LOOP_OPTIONs stuff? That's your chewing gum and duct tape right there. Also, you can't just take a "snapshot" and call it a release. Unless that automated builds sets FOR_CDROM, you'll have various bits that can't be put on a CDROM on the ftp site. There is also an issue of dependencies. Unless you freeze the tree during the build, you can get wrong dependency info in the packages due to dependencies getting updated after the package is built. And the build will take a long time (about 4 days unless you try to speed it up by parallelizing it by hand), if you don't update the tree during the build you'll end up with a bunch of stale packages. Besides, nobody around here has enough disk space to build the entire package tree from the top. I don't even know how much space it takes now, but my guess is about 15GB. (Subdirectories like lang and japanese get to 1GB during build.) You'll have to do a "make package clean" in each of the subdirectories individually to avoid filling up the disks. As for (b), that is exactly what I've been doing (with Steve and Justin's help), minus the pay. If you give me a couple weeks' advance notice, I can build the entire package tree for any CDROM easily. (I do it every couple of weeks anyway---it's just a matter of doing it at the right time.) As I see, the problem has always been the lack of communication between you and me. There were a couple of times that I had a wrong idea about the release date (once when I sent you a mail asking me and you replied with a wrong date, and I didn't notice that it was different from what you were saying in other places, and another time when I was in the plane from Japan to here when you announced the final freeze) and had to pull all-nighters trying to build the packages, but generally it has been pretty smooth. (Oh, and there was one time when I was on a trip during the actual release, and you lost the mail I sent to you prior to leaving telling you where to take the stuff from.... :) Maybe we can set up a "release-engineers" alias or something that we (you, I, msmith, and whoever else that's involved in that particular release) use to communicate? * [-current culled; too many mailing lists again] Ok.... * With every previous release, you see, I've never known just how long * it was going to take to get packages for the CD images, and it has * indeed been a delaying factor for several of them. To be fair, I've * also not yelped too loudly about it since I've always been able to put * such delays to good use in applying last-minute tweaks to the * installation as the first serious flood of installation reports * started coming in. Still, there are more FreeBSD products now and Sorry, but I do not remember any releases in which the packages not being ready was the delaying factor. Exactly which one are you talking about? * this kind of slippage in installation or package bits is probably a * rapidly receeding luxury of the past, to say nothing of the day when * the packages collection was actually small enough to be handled * without significant automation. The problem here is that it is actually too big to be handled without significant manual work. ;) * package builds, I think it could work. Heck, I'll even go drive * shopping tomorrow just so that bento and paddock can have enough * storage to make prototyping something like this easier. Satoshi's * been asking for awhile anyway, and another promised upgrade didn't * happen when somebody else grabbed the drives I'd hoped to allocate to * it (grrr). Here's an idea. The only way automation will work is if you start building every package with an empty /usr/local. If you can get me enough diskspace, I'll promise to work on a scheme in which for each port, a script will set up a chroot directory and build the package in there. Actually we probably need multiple machines too, since it will take a very long time (all dependencies will always be built from scratch too) to do it this way. How about a 8-machine cluster with 18GB of disks each? :) Satoshi To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-ports" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199811062257.OAA10011>