Date: Wed, 11 Nov 1998 14:38:51 -0800 (PST) From: asami@FreeBSD.ORG (Satoshi Asami) To: sprice@hiwaay.net Cc: ports@FreeBSD.ORG, msmith@FreeBSD.ORG Subject: Re: auto ports build script Message-ID: <199811112238.OAA16776@silvia.hip.berkeley.edu> In-Reply-To: <Pine.OSF.4.02.9811110801400.15738-100000@fly.HiWAAY.net> (message from Steve Price on Wed, 11 Nov 1998 08:45:28 -0600 (CST))
next in thread | previous in thread | raw e-mail | index | archive | help
* Yes, 2) make things complicated but we shouldn't ignore it. * With Mike's promise to find the resources we need, we shouldn't * have a problem addressing 2) in a separate build tree. I'm not saying it should be ignored, I'm just contending that it's an entirely different problem. :) * Ah, it is amazing what the dawn of a new day can do to the * fog in one's head. I'm finally beginning to see what you are * getting at. The really cool part is that the script you Good thing we agree with each other. :) * already have does what you describe without all the pkg_add * business. Given a fresh chroot'd environment the algorithm goes * something like this: * * for each line in INDEX file * save dependencies for this port * for each port * build an inverse dependency list, aka a required by list * for each port * if this port doesn't depend on any other port then * recurse(port) * * And recurse() looks like this: * * if this port has been visited before then * return * set visited to 1 * for each port this one depends on * recurse(port) * install, package this port * for each port that requires this port * recurse(port) * deinstall this port Oh, I see. * All this in 157 lines of perl code that should be pretty much * immune to changes to bsd.port*.mk. You also get a nifty * exclude feature to boot - like excluding XFree86-3.3.2 and * all the ports that depend on it. :) That's a nice feature. * Now I see said the blind man. All I have to do is copy it out * of the chroot'd environment. The only difference between what * you are describing is when this is done. If I really understand * your proposed method, you build up, copy out the good stuff, and * break down the chroot'd env for each port. What I gave you was * something that built up the tree once, simulated the one dependency * tree approach in between, and then copied everything at the very * end. This latter approach should be magnitudes faster, take * the same amount of total space (maybe a little more in a few * corner cases), and accomplishes the same thing. :) I'm trying to come up with something that is easily parallelizable. It's not clear to me how you can do that in perl, although I think we can use your script (modified) to take care of a single port in the parallelized world. Another is that I truly believe the only way to build correct packages is to always start with an empty /usr/local. One thing I haven't mentioned yet in this discussion is the existence of ports that configure themselves and include support for all sorts of cruft they find. You see all the gunk in the tree that says ".if defined(PACKAGE_BUILDING) CONFIGURE_ARGS=--disable-foobar"? That's because when we build packages on a heavily populated machine, there are stuff that aren't on the porter's machine and we end up shipping a binary with dependencies which are not covered by @pkgdep. The only way to find this, right now, is to have an annoyed package user report to us. Since this problem is not reproducible without a specific set of conditions, it is very hard to diagnose, requires a hack to work around, and there always is a danger that it will slip through whenever someone makes a last-minute update before releases. Another nice thing the empty /usr/local approach will let us do is to add a check to verify completeness of pkg/PLIST. Just pkg_delete all the stuff in /var/db/pkg and if anything is still under /usr/local, the port in question is probably missing some entries. :) * This is exactly what my little perl script does, except it doesn't * just print it out. It does the real work during the post-order * tranversal. I see. I told you it's easy. :) * I'll play with this a little more later on today. Pretty impressive * if this does in one long line of awk what took me 157 lines of perl. ;) Well, it does only print it out so far, but I think this is the approach we should take. Building stuff according to dependencies is exactly what make is designed for. In fact, I think our final goal should be to use pmake to parallelize this between machines. That way, we can guarantee that a dependency is built before a port that's requiring it, so we can always use packages. By the way, does anyone know where I can find most up-to-date sources of pmake? I know there's some stuff in /usr/src/usr.bin/make, but the "pmake tutorial" doesn't seem to agree with the manpage, and adding "-DREMOTE" to CFLAGS causes the build to fail. Also, the tutorial says nothing about how to specify remote machines, etc. * Yes, walking the whole tree once, updating, and starting over * was what I was proposing too. Ok. * This would be nice. Changing one line in the lps (little perl * script :) would make the whole thing much faster and cleaner * if this were implemented. However, it shouldn't really matter if all the dependencies are built first. It will use packages most of the time anyway. (Also, aren't there RUN_DEPENDS that truly are required for installing, not only running? USE_PERL5 is one of them....) Satoshi To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-ports" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199811112238.OAA16776>