From owner-freebsd-arch Sun Jul 7 2:18:15 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2960037B400 for ; Sun, 7 Jul 2002 02:18:11 -0700 (PDT) Received: from aleph.net.uniovi.es (aleph.net.uniovi.es [156.35.11.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1A10243E31 for ; Sun, 7 Jul 2002 02:18:10 -0700 (PDT) (envelope-from sobrado@string1.ciencias.uniovi.es) Received: from CONVERSION-DAEMON.aleph.net.uniovi.es by aleph.net.uniovi.es (PMDF V6.1 #39514) id <01KJT83TSSF400GK96@aleph.net.uniovi.es> for arch@freebsd.org; Sun, 07 Jul 2002 11:18:06 +0200 (MET) Received: from string1.ciencias.uniovi.es (string1.ciencias.uniovi.es [156.35.97.40]) by aleph.net.uniovi.es (PMDF V6.1 #39514) with ESMTP id <01KJT83T0OVU00FZQ2@aleph.net.uniovi.es>; Sun, 07 Jul 2002 11:18:05 +0200 (MET) Received: from string1.ciencias.uniovi.es (sobrado@localhost [127.0.0.1]) by string1.ciencias.uniovi.es (8.11.1/8.11.1) with ESMTP id g679I4E06788; Sun, 07 Jul 2002 11:18:05 +0200 Date: Sun, 07 Jul 2002 11:18:04 +0200 From: Igor Sobrado Subject: Re: software in /usr/contrib In-reply-to: Message from Terry Lambert "of Sat, 06 Jul 2002 15:16:34 PDT." <3D276C42.67C25814@mindspring.com> To: Terry Lambert Cc: Igor Sobrado , Dag-Erling Smorgrav , arch@freebsd.org Message-id: <200207070918.g679I4E06788@string1.ciencias.uniovi.es> MIME-version: 1.0 X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > I believe when he says "in a strong evolution", he actually means > "undergoing active developement". > Yes, of course! It is exactly what I wanted to say. My English skills are not as good as they should be. All that software is in a very active development with too frequently changes. And it can be a problem for FreeBSD as it is now for Solaris. > The problem he appears to be trying to solve is that these things > tend to change very quickly, without any FreeBSD control over > their size, shape, or direction. > Exactly! That is the problem. An additional problem with all those things that change very quickly is that, sometimes, there are not only stable and development releases, but also some different branches. I need a modified Tcl/Tk (OTcl from MIT) to build the ISI/Berkeley Network Simulator. Same happens with all the video-conference at MICE project. That software requires another modified release of that software. It should be nice to provide a release of that software under active development as a part of FreeBSD. But, IMHO, it should not be in /usr/bin. There is not a problem hard-coding the path to those programs in scripts and binaries when required, but users should have a chance to install their own customized versions of that software, or even another release (either more recent or more ancient) if they need it. > This was the initial justification for the removal of Perl in the > based system, and the seed of the ongoing dicussion entitled > "Removing perl in make world". > I am new to the *BSD world (I come from the Solaris, HP-UX and IRIX world). I did not know that discussion about Perl. By the way, I have checked that my fresh installation of FreeBSD 4.6-RELEASE includes Perl (an old Perl, not the latest stable release) in /usr/bin. What was wrong with my installation? I have not added it from the ports. And I use /usr/local (in the BSD flavors of Unix) to install all the optional software. > Igor is saying that there are other tools which are obtained from > thir parties which should also atl least be moved to "controib", > if not banished from the system to a port/package. > Well... "banish" a tool to a port seems dangerous! Moving Perl (that is included with 4.6-RELEASE), bzip2, gzip, and other tools to /usr/contrib/bin seems reasonable. In any case, it is possible to provide a link to those binaries in /usr/bin. Some tools (and compression tools are good examples) can be useful for the operating system (ports probably require those tools). And it is possible to write administrative scripts in Perl (Solaris has some of those scripts now), that can be *required* for the operating system to run. There is nothing wrong in that approach, but users should have a chance to install their own tools. Other software (Info-ZIP) supports a lot of extensions not provided either because there are of no use for most users (like the ability to manage OpenVMS versions ";x" in files) or because there are some license restrictions on the algorithms. But some users need those fully functional [un]zip releases too. > For some tools, this makes sense. For other tools, this is very > dangerous... and here's why: > > FreeBSD has been steadily giving away control of base system > components to third parties. Any time it takes a BSD verion > of a program, such as "tar", and replaces it with a GNU or > other vendor's equivalent, even if it's done for good reason, > then it sets up a situation like the one Igor is concerned > about. > > I think that he's right for some tools; they simply don't belong > in the base system (e.g. "bzip2" is not in specified by POSIX or > the Sinugle UNIX Specification). But for other things, where > there is not a BSD equivalent, or where the BSD equivalent has > been intentionally abandoned, it doesn't make sense. > Agreed! I *never* wanted, we say, Gnu tar moved to /usr/contrib even if it is maintained by third parties. It is very dangerous to move something like tar or cc out of /usr/bin even providing a link to those binaries. I was speaking only about changing some software that other Unix vendors are providing in /usr/contrib to that place. Both, BSDi (now Wind River) and Hewlett-Packard (now Compaq) have the same set of packages in (/usr/contrib), with very small differences: bzip2, gzip, Perl, Tcl/Tk, Expect or (why not?) nmh. But I was not speaking about moving Gnu tar or gcc (I think that at least gcc will be very difficult to replace with a native BSD C compiler!). Those programs are a part of any Unix distribution. > I don't claim to have a litmus test that would let you pick in > every case; if this idea goes forward, it should be with caution. > I agree with you, Terry. All changes should be adopted with caution. But we have a reference to follow in the BSD/OS and HP-UX operating systems. Thanks a lot for your feedback. You have explained some very important points, like what software can (and cannot) be moved, and what was the idea behind that proposal. Cheers, Igor. -- Igor Sobrado, UK34436 - sobrado@acm.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 3: 9:12 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DCC4E37B400; Sun, 7 Jul 2002 03:09:08 -0700 (PDT) Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id 701A843E09; Sun, 7 Jul 2002 03:09:08 -0700 (PDT) (envelope-from des@ofug.org) Received: by flood.ping.uio.no (Postfix, from userid 2602) id 9FCD6534B; Sun, 7 Jul 2002 12:09:06 +0200 (CEST) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Wes Peters Cc: Dan Moschuk , arch@freebsd.org Subject: Re: Package system flaws? References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> From: Dag-Erling Smorgrav Date: 07 Jul 2002 12:09:05 +0200 In-Reply-To: <3D27A296.D58FB4B4@softweyr.com> Message-ID: Lines: 25 User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.2 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Wes Peters writes: > Dan Moschuk wrote: > > Where can we improve? > In all of the above areas, plus all the ones we haven't addressed yet. *grin* One thing we could improve a lot is the package file format. We currently use gzipped tarballs, which have to be completely unpacked before processing can begin. One improvement we can make is to use an archive format such as zip that allows us to extract individual files quickly, so we can extract the metadata and check dependencies etc. without unpacking the entire package. This would save both time and space (the current system can fail if the temp directory doesn't have room for the unpacked package, even if the installation directory has room to spare). A further improvement is to use a custom archive format that always stores the metadata at the beginning of the archive so we can install packages from the net without having to download and store them locally first (zip isn't suitable for this as it stores the content directory at the end of the archive, and the files within the archive can appear in any order). DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 4:20:18 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5340737B400; Sun, 7 Jul 2002 04:20:08 -0700 (PDT) Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id 83EF643E31; Sun, 7 Jul 2002 04:20:07 -0700 (PDT) (envelope-from des@ofug.org) Received: by flood.ping.uio.no (Postfix, from userid 2602) id 86D8F534A; Sun, 7 Jul 2002 13:20:05 +0200 (CEST) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: arch@freebsd.org Cc: obrien@freebsd.org Subject: -Werror removed from bsd.sys.mk From: Dag-Erling Smorgrav Date: 07 Jul 2002 13:20:04 +0200 Message-ID: Lines: 71 User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.2 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Why was -Werror removed from bsd.sys.mk without any prior discussion, and with a log message that seems deliberately misleading? David, if you really hate -Werror, couldn't you just have added NO_WERROR to your make.conf instead of taking this valuable tool away from us? des@des /usr/src/share/mk% lcvs log -r1.9 bsd.sys.mk RCS file: /home/ncvs/src/share/mk/bsd.sys.mk,v Working file: bsd.sys.mk head: 1.9 branch: locks: strict access list: symbolic names: RELENG_4_6_0_RELEASE: 1.3.2.4 RELENG_4_6: 1.3.2.4.0.2 RELENG_4_6_BP: 1.3.2.4 RELENG_4_5_0_RELEASE: 1.3.2.3 RELENG_4_5: 1.3.2.3.0.4 RELENG_4_5_BP: 1.3.2.3 RELENG_4_4_0_RELEASE: 1.3.2.3 RELENG_4_4: 1.3.2.3.0.2 RELENG_4_4_BP: 1.3.2.3 RELENG_4: 1.3.0.2 keyword substitution: kv total revisions: 14; selected revisions: 1 description: ---------------------------- revision 1.9 date: 2002/05/19 18:24:00; author: obrien; state: Exp; lines: +1 -7 Tweak the WARNS levels a tad. ============================================================================= des@des /usr/src/share/mk% lcvs diff -r1.8 -r1.9 bsd.sys.mk Index: bsd.sys.mk =================================================================== RCS file: /home/ncvs/src/share/mk/bsd.sys.mk,v retrieving revision 1.8 retrieving revision 1.9 diff -u -r1.8 -r1.9 --- bsd.sys.mk 10 May 2002 01:58:16 -0000 1.8 +++ bsd.sys.mk 19 May 2002 18:24:00 -0000 1.9 @@ -1,4 +1,4 @@ -# $FreeBSD: src/share/mk/bsd.sys.mk,v 1.8 2002/05/10 01:58:16 obrien Exp $ +# $FreeBSD: src/share/mk/bsd.sys.mk,v 1.9 2002/05/19 18:24:00 obrien Exp $ # # This file contains common settings used for building FreeBSD # sources. @@ -11,9 +11,6 @@ .if !defined(NO_WARNS) . if defined(WARNS) . if ${WARNS} > 0 -. if !defined(NO_WERROR) -CFLAGS += -Werror -. endif . endif . if ${WARNS} > 1 CFLAGS += -Wall -Wno-format-y2k @@ -45,9 +42,6 @@ . if ${WFORMAT} > 0 #CFLAGS += -Wformat-nonliteral -Wformat-security -Wno-format-extra-args CFLAGS += -Wformat=2 -Wno-format-extra-args -. if !defined(NO_WERROR) -CFLAGS += -Werror -. endif . endif . endif .endif DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 8:35: 4 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4874437B400 for ; Sun, 7 Jul 2002 08:35:02 -0700 (PDT) Received: from scoobysnax.jaded.net (d141-7-230.home.cgocable.net [24.141.7.230]) by mx1.FreeBSD.org (Postfix) with ESMTP id A9D7C43E09 for ; Sun, 7 Jul 2002 08:35:01 -0700 (PDT) (envelope-from dan@scoobysnax.jaded.net) Received: from scoobysnax.jaded.net (localhost [127.0.0.1]) by scoobysnax.jaded.net (8.12.3/8.12.3) with ESMTP id g67FZ2ow001110; Sun, 7 Jul 2002 11:35:02 -0400 (EDT) (envelope-from dan@scoobysnax.jaded.net) Received: (from dan@localhost) by scoobysnax.jaded.net (8.12.3/8.12.3/Submit) id g67FYvg0001106; Sun, 7 Jul 2002 11:34:57 -0400 (EDT) Date: Sun, 7 Jul 2002 11:34:57 -0400 From: Dan Moschuk To: Dag-Erling Smorgrav Cc: Wes Peters , arch@FreeBSD.ORG Subject: Re: Package system flaws? Message-ID: <20020707153457.GA1086@scoobysnax.jaded.net> References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG | > Dan Moschuk wrote: | > > Where can we improve? | > In all of the above areas, plus all the ones we haven't addressed yet. | | *grin* | | One thing we could improve a lot is the package file format. We | currently use gzipped tarballs, which have to be completely unpacked | before processing can begin. One improvement we can make is to use an | archive format such as zip that allows us to extract individual files | quickly, so we can extract the metadata and check dependencies | etc. without unpacking the entire package. This would save both time | and space (the current system can fail if the temp directory doesn't | have room for the unpacked package, even if the installation directory | has room to spare). A further improvement is to use a custom archive | format that always stores the metadata at the beginning of the archive | so we can install packages from the net without having to download and | store them locally first (zip isn't suitable for this as it stores the | content directory at the end of the archive, and the files within the | archive can appear in any order). I don't think using an archive format like zip would be a step in the right direction. If the package file format were to be redesigned, I would vote for a custom header prepended to a bziped tarball. Cheers, -Dan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 8:54:40 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D7C3737B400; Sun, 7 Jul 2002 08:54:36 -0700 (PDT) Received: from thuvia.demon.co.uk (thuvia.demon.co.uk [193.237.34.248]) by mx1.FreeBSD.org (Postfix) with ESMTP id A432E43E09; Sun, 7 Jul 2002 08:54:32 -0700 (PDT) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (dotar.thuvia.org [10.0.0.4]) by phaidor.thuvia.org (8.11.6/8.11.6) with ESMTP id g67FkWb05963; Sun, 7 Jul 2002 16:46:32 +0100 (BST) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (localhost [IPv6:::1]) by dotar.thuvia.org (8.12.3/8.12.3) with ESMTP id g67FkVKC018924; Sun, 7 Jul 2002 16:46:31 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.3/8.12.3/Submit) id g67FkVIL018923; Sun, 7 Jul 2002 16:46:31 +0100 (BST) Date: Sun, 7 Jul 2002 16:46:31 +0100 (BST) From: Mark Valentine Message-Id: <200207071546.g67FkVIL018923@dotar.thuvia.org> In-Reply-To: Dag-Erling Smorgrav's message of Jul 7, 10:14am X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: des@ofug.org (Dag-Erling Smorgrav), Wes Peters Subject: Re: Package system flaws? Cc: Dan Moschuk , arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: des@ofug.org (Dag-Erling Smorgrav) > Date: Sun 7 Jul, 2002 > Subject: Re: Package system flaws? > One thing we could improve a lot is the package file format. We > currently use gzipped tarballs, which have to be completely unpacked > before processing can begin. One improvement we can make is to use an > archive format such as zip that allows us to extract individual files > quickly, so we can extract the metadata and check dependencies > etc. without unpacking the entire package. My idea for this is to use a hybrid format. The package itself would be an uncompressed archive in any available simple format. Non-sequential access to the members of the archive is non-critical for local storage where seeks are "cheap" (so tar archives would be acceptable for that purpose, or for when the entire archive will be transferred, e.g. from an ftp server), but compressing the entire archive would be a significant efficiency loss due to having to uncompress before seeking. For package installs over slow data streams, however, sequential access to the archive is important. A _directory_ makes a good direct-access archive (whether stored locally or via ftp/web/whatever server). If that's unpalatable, then a server- side CGI/PHP script could serve up member elements from a sequential- access archive. In any case the tools should accept a directory name as a package archive. However, the archive members are _not_ the individual files comprising the package. Instead, the package is split into logical sub-packages (there may be only one, that's fine), such as the core run-time package, language sets, development headers and libraries, end-user documentation. Each sub-package is stored as a compressed archive (.tar.bz2 or whatever). The _package_ archive comprises the package metadata files (optionally compressed - useful for the larger files such as +CONTENTS) and the sub-package(s), each as a package archive member. The idea here is to allow individual access to those parts of the package that are needed individually, whilst gaining the benefits of compressing major portions of the package as collections of files (to which individual access is not generally required). While the division of an existing port into sub-packages requires effort, it's necessary only to gain the benefits of optionally-installable sub- packages. For existing ports, and for the majority of ports which will never likely be sub-package, pkg_create simple creates the package archive using the existing metadata files and a single compressed archive of the packages files. Examples 1: simple package, not sub-packaged. $ ls /var/spool/pkg/foo-x.y +COMMENT +DESC +INSTALL +CONTENTS +DISPLAY base.tgz Example 2: package with optional development and documentation components $ ls /var/spool/pkg/bar-m.n +COMMENT +DESC +INSTALL devel.tgz +CONTENTS +DISPLAY base.tgz doc.tgz The package install tools would allow the user to select between optional sub-packages. The base sub-package may be marked as "required". If there are only "required" sub-packages, or the user elects to install "all" or "only required" sub-packages, no dialog is necessary. And so on.. Cheers, Mark. -- Mark Valentine, Thuvia Labs "Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses "We're kind of stupid that way." *munch* *munch* and endorses FreeBSD -- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 9: 0:11 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8809437B406; Sun, 7 Jul 2002 09:00:05 -0700 (PDT) Received: from spqr.osg.gov.bc.ca (spqr.osg.gov.bc.ca [142.32.102.24]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1530F43E4A; Sun, 7 Jul 2002 09:00:05 -0700 (PDT) (envelope-from Cy.Schubert@osg.gov.bc.ca) Received: from passer.osg.gov.bc.ca (passer.osg.gov.bc.ca [142.32.110.29]) by spqr.osg.gov.bc.ca (Postfix) with ESMTP id 8F8129EF16; Sun, 7 Jul 2002 09:00:04 -0700 (PDT) Received: from cwsys.cwsent.com (cwsys2 [10.1.2.1]) by passer.osg.gov.bc.ca (8.12.5/8.12.3) with ESMTP id g67G03OX023822; Sun, 7 Jul 2002 09:00:04 -0700 (PDT) (envelope-from cy@cwsent.com) Received: from cwsys (localhost [127.0.0.1]) by cwsys.cwsent.com (8.12.5/8.12.3) with ESMTP id g67G03fP014193; Sun, 7 Jul 2002 09:00:03 -0700 (PDT) (envelope-from cy@cwsys.cwsent.com) Message-Id: <200207071600.g67G03fP014193@cwsys.cwsent.com> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 Reply-To: Cy Schubert - CITS Open Systems Group From: Cy Schubert - CITS Open Systems Group X-os: FreeBSD X-Sender: cy@cwsent.com To: Dan Moschuk Cc: arch@FreeBSD.ORG Subject: Re: Package system flaws? In-Reply-To: Message from Dan Moschuk of "Sat, 06 Jul 2002 18:05:11 EDT." <20020706220511.GA88651@scoobysnax.jaded.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 07 Jul 2002 09:00:03 -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20020706220511.GA88651@scoobysnax.jaded.net>, Dan Moschuk writes: > > I've been doing some thinking lately about our ports system versus what > other systems have adopted and was curious as to what people think on > the subject? What does FreeBSD do well? Where can we improve? How does it > rate against the umpteen Linux flavours? Comparing our package system with others, such as RH's RPM. Sun's pkg*, and DEC's... er Compaq's... er HP's setld, the FreeBSD pkg_* utilities are one of the better package applications in the industry. Actually I like our package system better than Suns. However, I have been spoiled by 19 years of using IBM's SMP/E (and it's predecessor SMP) for the MVS (mainframe) operating system prior to my 10 years in the UNIX world. Unfortunately I don't think we have enough people on the project to build the Cadillac (some might argue monstrosity), with all of its bells, whistles, prereq's, coreq's, functions, dependent functions, PTF's, APARFIXES, USERMODS, zones, recovery, and other handy features that IBM has built for MVS over the past 40 years. Interestingly IBM has not ported SMP/E to AIX, probably because it would take them an ice age to port and SMP/E isn't the easiest application to understand -- most people hated it while the few of us who fully understood it loved it. Excluding the IBM "anomaly," I think we probably have the best package system in the market. Can we improve our package system? Yes. Can I think of any? Well, if we want to keep it simple, I cannot think of any enhancements that would improve our lives, except for possibly a GUI front end. -- Cheers, Phone: 250-387-8437 Cy Schubert Fax: 250-387-5766 Team Leader, Sun/Alpha Team Email: Cy.Schubert@osg.gov.bc.ca Open Systems Group, CITS Ministry of Management Services Province of BC FreeBSD UNIX: cy@FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 11:48:26 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A4F9737B400 for ; Sun, 7 Jul 2002 11:48:25 -0700 (PDT) Received: from dragon.nuxi.com (trang.nuxi.com [66.92.13.169]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2368043E09 for ; Sun, 7 Jul 2002 11:48:25 -0700 (PDT) (envelope-from obrien@NUXI.com) Received: from dragon.nuxi.com (obrien@localhost [127.0.0.1]) by dragon.nuxi.com (8.12.5/8.12.2) with ESMTP id g67ImOoi089049; Sun, 7 Jul 2002 11:48:24 -0700 (PDT) (envelope-from obrien@dragon.nuxi.com) Received: (from obrien@localhost) by dragon.nuxi.com (8.12.5/8.12.5/Submit) id g67ImOMh089048; Sun, 7 Jul 2002 11:48:24 -0700 (PDT) Date: Sun, 7 Jul 2002 11:48:24 -0700 From: "David O'Brien" To: Dag-Erling Smorgrav Cc: arch@freebsd.org Subject: Re: -Werror removed from bsd.sys.mk Message-ID: <20020707184824.GB58914@dragon.nuxi.com> Reply-To: obrien@freebsd.org Mail-Followup-To: David O'Brien , Dag-Erling Smorgrav , arch@freebsd.org References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4i X-Operating-System: FreeBSD 5.0-CURRENT Organization: The NUXI BSD group X-Pgp-Rsa-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Rsa-Keyid: 1024/34F9F9D5 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, Jul 07, 2002 at 01:20:04PM +0200, Dag-Erling Smorgrav wrote: > Why was -Werror removed from bsd.sys.mk without any prior discussion, Reinstated. There was too much breakage due to GCC 3.1 turmoil. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 12:19:55 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5170537B400; Sun, 7 Jul 2002 12:19:53 -0700 (PDT) Received: from softweyr.com (softweyr.com [65.88.244.127]) by mx1.FreeBSD.org (Postfix) with ESMTP id C2AEF43E09; Sun, 7 Jul 2002 12:19:52 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from homer.softweyr.com ([204.68.178.39] helo=softweyr.com) by softweyr.com with esmtp (Exim 3.35 #1) id 17RHZF-000DDU-00; Sun, 07 Jul 2002 13:19:33 -0600 Message-ID: <3D289529.95C1EC8A@softweyr.com> Date: Sun, 07 Jul 2002 12:23:21 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Dag-Erling Smorgrav Cc: Dan Moschuk , arch@freebsd.org Subject: Re: Package system flaws? References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Dag-Erling Smorgrav wrote: > > Wes Peters writes: > > Dan Moschuk wrote: > > > Where can we improve? > > In all of the above areas, plus all the ones we haven't addressed yet. > > *grin* > > One thing we could improve a lot is the package file format. We > currently use gzipped tarballs, which have to be completely unpacked > before processing can begin. One improvement we can make is to use an > archive format such as zip that allows us to extract individual files > quickly, so we can extract the metadata and check dependencies > etc. without unpacking the entire package. I'd rather go one step further and separate the metadata from the files completely. I'm thinking an XML-ish format for the metadata, with blobs of files stored in whatever format makes the most sense inter- spersed. Zip format makes a lot of sense and is available in library format, but I could easily tolerate tar or cpio, as well as a variety of compression formats. The metadata should not be compressed for speed of package processing. This would also allow us to separate the files in the package into distinct pieces, and apply metadata to the collections of files. Picture, for instance, a package that installs an X application. We have executable files that need to go into $PKG_BIN, perhaps a library or two that needs to in $PKG_LIB, an application resource file that needs to go into $X_APPRESDIR and some fonts that need to be installed in $X_FONTDIR. We provide defaults for each of these, let the user override them as necessary. > This would save both time > and space (the current system can fail if the temp directory doesn't > have room for the unpacked package, even if the installation directory > has room to spare). A further improvement is to use a custom archive > format that always stores the metadata at the beginning of the archive > so we can install packages from the net without having to download and > store them locally first (zip isn't suitable for this as it stores the > content directory at the end of the archive, and the files within the > archive can appear in any order). Yes, an excellent idea. With a format as I've proposed, it would be possible to specify the file collections as URLs rather than encoded archives. You could download the entire package collection in about the same amount of space as the current ports collection. Of course this means pkg_add would have to be linked with libfetch, too. ;^) -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 13:26: 8 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7453837B400; Sun, 7 Jul 2002 13:26:06 -0700 (PDT) Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id CCF8543E09; Sun, 7 Jul 2002 13:26:05 -0700 (PDT) (envelope-from des@ofug.org) Received: by flood.ping.uio.no (Postfix, from userid 2602) id D0309534A; Sun, 7 Jul 2002 22:26:03 +0200 (CEST) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Wes Peters Cc: Dan Moschuk , arch@freebsd.org Subject: Re: Package system flaws? References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> <3D289529.95C1EC8A@softweyr.com> From: Dag-Erling Smorgrav Date: 07 Jul 2002 22:26:02 +0200 In-Reply-To: <3D289529.95C1EC8A@softweyr.com> Message-ID: Lines: 9 User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.2 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Wes Peters writes: > Of course > this means pkg_add would have to be linked with libfetch, too. ;^) It already is. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 13:51:13 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 324D037B400; Sun, 7 Jul 2002 13:51:10 -0700 (PDT) Received: from mailsrv.otenet.gr (mailsrv.otenet.gr [195.170.0.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id D35BF43E3B; Sun, 7 Jul 2002 13:51:03 -0700 (PDT) (envelope-from keramida@ceid.upatras.gr) Received: from shadow.otenet.gr (shadow.otenet.gr [195.170.0.7]) by mailsrv.otenet.gr (8.12.4/8.12.4) with ESMTP id g67Kp1FD008957; Sun, 7 Jul 2002 23:51:01 +0300 (EEST) Received: from hades.hell.gr (patr530-b142.otenet.gr [212.205.244.150]) by shadow.otenet.gr (8.12.4/8.12.4) with ESMTP id g67KokHE008099; Sun, 7 Jul 2002 23:50:47 +0300 (EEST) Received: from hades.hell.gr (hades [127.0.0.1]) by hades.hell.gr (8.12.5/8.12.5) with ESMTP id g67KojS9015766; Sun, 7 Jul 2002 23:50:45 +0300 (EEST) (envelope-from keramida@ceid.upatras.gr) Received: (from charon@localhost) by hades.hell.gr (8.12.5/8.12.5/Submit) id g67KoiFL015756; Sun, 7 Jul 2002 23:50:44 +0300 (EEST) (envelope-from keramida@ceid.upatras.gr) Date: Sun, 7 Jul 2002 23:50:43 +0300 From: Giorgos Keramidas To: Dag-Erling Smorgrav Cc: Wes Peters , Dan Moschuk , arch@FreeBSD.ORG Subject: Re: Package system flaws? Message-ID: <20020707205043.GA2778@hades.hell.gr> References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD 5.0-CURRENT i386 X-PGP-Fingerprint: C1EB 0653 DB8B A557 3829 00F9 D60F 941A 3186 03B6 X-Phone: +30-944-116520 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 2002-07-07 12:09 +0000, Dag-Erling Smorgrav wrote: > Wes Peters writes: > > Dan Moschuk wrote: > > > Where can we improve? > > In all of the above areas, plus all the ones we haven't addressed yet. > > *grin* > > One thing we could improve a lot is the package file format. We > currently use gzipped tarballs, which have to be completely unpacked > before processing can begin. One improvement we can make is to use an > archive format such as zip that allows us to extract individual files > quickly, so we can extract the metadata and check dependencies > etc. without unpacking the entire package. Err, that can be done with tar too. Although one might argue that reading the entire archive (instead of a small TOC at the beginning or end of the archive) is a bit clumsier... % cat archive | ( cd /tmp; tar xzvf - PKG_METADATA ) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 13:55:41 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9395E37B405; Sun, 7 Jul 2002 13:55:35 -0700 (PDT) Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id CFB2A43E54; Sun, 7 Jul 2002 13:55:34 -0700 (PDT) (envelope-from des@ofug.org) Received: by flood.ping.uio.no (Postfix, from userid 2602) id 07441534A; Sun, 7 Jul 2002 22:55:32 +0200 (CEST) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Giorgos Keramidas Cc: Wes Peters , Dan Moschuk , arch@FreeBSD.ORG Subject: Re: Package system flaws? References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> <20020707205043.GA2778@hades.hell.gr> From: Dag-Erling Smorgrav Date: 07 Jul 2002 22:55:31 +0200 In-Reply-To: <20020707205043.GA2778@hades.hell.gr> Message-ID: Lines: 29 User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.2 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Giorgos Keramidas writes: > On 2002-07-07 12:09 +0000, Dag-Erling Smorgrav wrote: > > Wes Peters writes: > > > Dan Moschuk wrote: > > > > Where can we improve? > > > In all of the above areas, plus all the ones we haven't addressed yet. > > > > *grin* > > > > One thing we could improve a lot is the package file format. We > > currently use gzipped tarballs, which have to be completely unpacked > > before processing can begin. One improvement we can make is to use an > > archive format such as zip that allows us to extract individual files > > quickly, so we can extract the metadata and check dependencies > > etc. without unpacking the entire package. > > Err, that can be done with tar too. Although one might argue that > reading the entire archive (instead of a small TOC at the beginning or > end of the archive) is a bit clumsier... That's precisely what I want to avoid. With tar, you always have to uncompress the entire package to find even just one file (unless you use the GNU-specific --fast-read option, in which case it stops once it finds the file you asked for) DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 14:15:43 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5B8C737B405; Sun, 7 Jul 2002 14:15:36 -0700 (PDT) Received: from 12-234-90-219.client.attbi.com (12-234-90-219.client.attbi.com [12.234.90.219]) by mx1.FreeBSD.org (Postfix) with ESMTP id EC59843E54; Sun, 7 Jul 2002 14:15:35 -0700 (PDT) (envelope-from DougB@FreeBSD.org) Received: from FreeBSD.org (master.gorean.org [10.0.0.2]) by 12-234-90-219.client.attbi.com (8.12.3/8.12.3) with ESMTP id g67LFUBu087846; Sun, 7 Jul 2002 14:15:35 -0700 (PDT) (envelope-from DougB@FreeBSD.org) Message-ID: <3D28AF72.8F14DC40@FreeBSD.org> Date: Sun, 07 Jul 2002 14:15:30 -0700 From: Doug Barton Organization: Triborough Bridge & Tunnel Authority X-Mailer: Mozilla 4.79 [en] (X11; U; FreeBSD 4.5-RELEASE i386) X-Accept-Language: en MIME-Version: 1.0 To: Dan Moschuk Cc: arch@FreeBSD.org Subject: Re: Package system flaws? References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> <20020707153457.GA1086@scoobysnax.jaded.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Dan Moschuk wrote: > I don't think using an archive format like zip would be a step in the > right direction. If the package file format were to be redesigned, I would > vote for a custom header prepended to a bziped tarball. What is it about working on ports that makes people automatically gravitate to the most hideous, complicated solutions? Why not just create a tarball with the stuff to install, create the metadata file (in whatever format), then create a tarball with the two together? Then, step one is to unroll the tarball, read the metadata, grab dependencies if needed, then proceed to do the rest of the install. Before you start saying, "THAT WON'T WORK," please understand that I've already used this exact technique in 3 different projects, including windows, OS/2, and unix platforms. Doug To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 14:28:56 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8FDBC37B401; Sun, 7 Jul 2002 14:28:53 -0700 (PDT) Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id 049E843E65; Sun, 7 Jul 2002 14:28:51 -0700 (PDT) (envelope-from des@ofug.org) Received: by flood.ping.uio.no (Postfix, from userid 2602) id 1FDC6534A; Sun, 7 Jul 2002 23:28:50 +0200 (CEST) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Doug Barton Cc: Dan Moschuk , arch@FreeBSD.org Subject: Re: Package system flaws? References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> <20020707153457.GA1086@scoobysnax.jaded.net> <3D28AF72.8F14DC40@FreeBSD.org> From: Dag-Erling Smorgrav Date: 07 Jul 2002 23:28:49 +0200 In-Reply-To: <3D28AF72.8F14DC40@FreeBSD.org> Message-ID: Lines: 15 User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.2 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Doug Barton writes: > What is it about working on ports that makes people automatically > gravitate to the most hideous, complicated solutions? Why not just > create a tarball with the stuff to install, create the metadata file (in > whatever format), then create a tarball with the two together? Then, > step one is to unroll the tarball, read the metadata, grab dependencies > if needed, then proceed to do the rest of the install. We want to be able to install a package from a non-rewindable source without storing a temporary copy on disk. This means the metadata must without fail be at the very beginning of the package. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 15: 2:35 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CFAFD37B400; Sun, 7 Jul 2002 15:02:29 -0700 (PDT) Received: from spqr.osg.gov.bc.ca (spqr.osg.gov.bc.ca [142.32.102.24]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4862C43E31; Sun, 7 Jul 2002 15:02:29 -0700 (PDT) (envelope-from Cy.Schubert@osg.gov.bc.ca) Received: from passer.osg.gov.bc.ca (passer.osg.gov.bc.ca [142.32.110.29]) by spqr.osg.gov.bc.ca (Postfix) with ESMTP id 0AA729EF16; Sun, 7 Jul 2002 15:02:29 -0700 (PDT) Received: from cwsys.cwsent.com (cwsys2 [10.1.2.1]) by passer.osg.gov.bc.ca (8.12.5/8.12.3) with ESMTP id g67M2KOX026307; Sun, 7 Jul 2002 15:02:20 -0700 (PDT) (envelope-from cy@cwsent.com) Received: from cwsys (localhost [127.0.0.1]) by cwsys.cwsent.com (8.12.5/8.12.3) with ESMTP id g67M2IfP015705; Sun, 7 Jul 2002 15:02:19 -0700 (PDT) (envelope-from cy@cwsys.cwsent.com) Message-Id: <200207072202.g67M2IfP015705@cwsys.cwsent.com> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 Reply-To: Cy Schubert - CITS Open Systems Group From: Cy Schubert - CITS Open Systems Group X-os: FreeBSD X-Sender: cy@cwsent.com To: Wes Peters Cc: Dag-Erling Smorgrav , Dan Moschuk , arch@FreeBSD.ORG Subject: Re: Package system flaws? In-Reply-To: Message from Wes Peters of "Sun, 07 Jul 2002 12:23:21 PDT." <3D289529.95C1EC8A@softweyr.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 07 Jul 2002 15:02:18 -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <3D289529.95C1EC8A@softweyr.com>, Wes Peters writes: > Dag-Erling Smorgrav wrote: > > > > Wes Peters writes: > > > Dan Moschuk wrote: > > > > Where can we improve? > > > In all of the above areas, plus all the ones we haven't addressed yet. > > > > *grin* > > > > One thing we could improve a lot is the package file format. We > > currently use gzipped tarballs, which have to be completely unpacked > > before processing can begin. One improvement we can make is to use an > > archive format such as zip that allows us to extract individual files > > quickly, so we can extract the metadata and check dependencies > > etc. without unpacking the entire package. > > I'd rather go one step further and separate the metadata from the files > completely. I'm thinking an XML-ish format for the metadata, with > blobs of files stored in whatever format makes the most sense inter- > spersed. Zip format makes a lot of sense and is available in library > format, but I could easily tolerate tar or cpio, as well as a variety > of compression formats. The metadata should not be compressed for > speed of package processing. This seems like a reasonable approach. I don't think that the format of the files matters that much. The zip file format has a couple of advantages. First, as you mention, it's available in library format. Secondly, vendors like Sun are using this format to distribute patches. I think this format will quickly become the UNIX standard, superseding tar. Whether metadata is compressed or not is immaterial. With the speed of processors and the price of disk these days, I don't think it matters either way, except for possibly network congestion. I suspect that metadata would not be the lions share of the data anyhow. > > This would also allow us to separate the files in the package into > distinct pieces, and apply metadata to the collections of files. > Picture, for instance, a package that installs an X application. We > have executable files that need to go into $PKG_BIN, perhaps a library > or two that needs to in $PKG_LIB, an application resource file that > needs to go into $X_APPRESDIR and some fonts that need to be installed > in $X_FONTDIR. We provide defaults for each of these, let the user > override them as necessary. So far so good. I would also include the possibility of source being distributed with a package. In my previous life as an MVS systems programmer, packages (in MVS they're called functions) may be delivered in source form, binary form, or both. For example, JES/2 (Job Entry Subsystem/2), one of the MVS spooling subsystems, was distributed in source and object form. When JES would be installed, sources and binaries were installed by the same package (in different locations of course). I as the systems programmer for a site could apply fixes (PTF's and APARFIXES) or apply my own modifications to the code (USERMODS) to the source. SMP/E would automatically invoke the assembler to rebuild any affected binaries. Alternatively I could apply fixes or modifications in binary form (using an application called superzap -- IMASPZAP). (Pretty scary that I can remember this stuff after not having looked at it for 10 years.) When upgrading SMP/E would flag my USERMODS that were being backed out in order to accommodate the upgrade. This way I as the systems programmer for a site wouldn't have to remember a lot of details about my system, SMP/E did all that for me. I personally used SMP/E to update system config files too. That saved me from having to write a lot of stuff down because it was already recorded in my target zone (database). Dates and times and all that good stuff were recored. I don't think we need anything as elaborate as this. IMO not many FreeBSD users make custom modifications to their systems. Those of us who do, myself included, probably need our heads examined (or in my case I should make a case to have them applied to the source tree). However in the MVS world, I have never worked at a site that did not customize the source or apply binary patches in any way. Different requirements and different culture I guess. All of this helped make SMP/E the monstrosity it became. It was great for techno-wizards but for the average MVS guy, I've been told it was too much. I think we need to find the happy medium between functionality and scaryness. Much of this stuff is pretty useless here, but if you want to use any of these ideas, be my guest. > > > This would save both time > > and space (the current system can fail if the temp directory doesn't > > have room for the unpacked package, even if the installation directory > > has room to spare). A further improvement is to use a custom archive > > format that always stores the metadata at the beginning of the archive > > so we can install packages from the net without having to download and > > store them locally first (zip isn't suitable for this as it stores the > > content directory at the end of the archive, and the files within the > > archive can appear in any order). > > Yes, an excellent idea. With a format as I've proposed, it would be > possible to specify the file collections as URLs rather than encoded > archives. You could download the entire package collection in about > the same amount of space as the current ports collection. Of course > this means pkg_add would have to be linked with libfetch, too. ;^) This too is a good idea. However I personally like to keep local copy of package distribution files locally just in case they're needed for DRP recovery. In a DRP situation one may not want to install the latest version of a package, because the only focus should be to get the application(s) working again, not upgrade the system, and there is no guarantee that a certain version of a package will be there days, months, or even years from now. Keeping one's own copy is the only way to guarantee that. The option to fetch and store the files that comprise a package might be worthwhile looking at. -- Cheers, Phone: 250-387-8437 Cy Schubert Fax: 250-387-5766 Team Leader, Sun/Alpha Team Email: Cy.Schubert@osg.gov.bc.ca Open Systems Group, CITS Ministry of Management Services Province of BC FreeBSD UNIX: cy@FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 15: 4:37 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0197537B400 for ; Sun, 7 Jul 2002 15:04:36 -0700 (PDT) Received: from khavrinen.lcs.mit.edu (khavrinen.lcs.mit.edu [18.24.4.193]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6D4E743E31 for ; Sun, 7 Jul 2002 15:04:35 -0700 (PDT) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: from khavrinen.lcs.mit.edu (localhost [IPv6:::1]) by khavrinen.lcs.mit.edu (8.12.3/8.12.3) with ESMTP id g67M4WRC027053; Sun, 7 Jul 2002 18:04:32 -0400 (EDT) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: (from wollman@localhost) by khavrinen.lcs.mit.edu (8.12.3/8.12.3/Submit) id g67M4V1F027052; Sun, 7 Jul 2002 18:04:31 -0400 (EDT) (envelope-from wollman) Date: Sun, 7 Jul 2002 18:04:31 -0400 (EDT) From: Garrett Wollman Message-Id: <200207072204.g67M4V1F027052@khavrinen.lcs.mit.edu> To: wes@softweyr.com Subject: Re: Package system flaws? In-Reply-To: <3D289529.95C1EC8A@softweyr.com> References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> Organization: MIT Laboratory for Computer Science Cc: arch@FreeBSD.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In article <3D289529.95C1EC8A@softweyr.com> Wes wrote: >spersed. Zip format makes a lot of sense and is available in library >format, but I could easily tolerate tar or cpio, as well as a variety >of compression formats. The metadata should not be compressed for >speed of package processing. `pax' format provides for a convenient extension-header mechanism that would make this relatively easy to accomplish without inventing a new archive format. -GAWollman -- Garrett A. Wollman | [G]enes make enzymes, and enzymes control the rates of wollman@lcs.mit.edu | chemical processes. Genes do not make ``novelty- Opinions not those of| seeking'' or any other complex and overt behavior. MIT, LCS, CRS, or NSA| - Stephen Jay Gould (1941-2002) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 15:21:24 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AECD737B400; Sun, 7 Jul 2002 15:21:21 -0700 (PDT) Received: from goose.mail.pas.earthlink.net (goose.mail.pas.earthlink.net [207.217.120.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0FCA643E31; Sun, 7 Jul 2002 15:21:21 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0316.cvx40-bradley.dialup.earthlink.net ([216.244.43.61] helo=mindspring.com) by goose.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17RKP3-0007Zf-00; Sun, 07 Jul 2002 15:21:14 -0700 Message-ID: <3D28BEAC.4A5BA84@mindspring.com> Date: Sun, 07 Jul 2002 15:20:28 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Dag-Erling Smorgrav Cc: Doug Barton , Dan Moschuk , arch@FreeBSD.org Subject: Re: Package system flaws? References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> <20020707153457.GA1086@scoobysnax.jaded.net> <3D28AF72.8F14DC40@FreeBSD.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Dag-Erling Smorgrav wrote: > Doug Barton writes: > > What is it about working on ports that makes people automatically > > gravitate to the most hideous, complicated solutions? Why not just > > create a tarball with the stuff to install, create the metadata file (in > > whatever format), then create a tarball with the two together? Then, > > step one is to unroll the tarball, read the metadata, grab dependencies > > if needed, then proceed to do the rest of the install. > > We want to be able to install a package from a non-rewindable source > without storing a temporary copy on disk. This means the metadata > must without fail be at the very beginning of the package. I think you just want lookahead, actually. HTTP and FTP are both rewindable; the expense is relative to the server, but FTP supports "reget" and HTTP supports range requests. Putting the metadata up front resloves the issue of downloading data you don't want, and also resolves the problem of being able to collect all the metadata at the start, so that you can make accurate time estimates, and the user can make a time-cost-basis decision on whether or not to go ahead. Modern transfer protocols argue in general for a new archive format that collects the metadata at the start of the archive to permit such decisions to be made. From a general perspective, archives need to be able to be nested recursively, and to be extracted recursively. Obviously, this is a control mechanism enabling requirement: full extraction as a default is necessary for "bootstrap", and selective extraction is necessary as a component/function/customization selection method. Also from a general perspective, it needs to be possible to blindly extract archives for "bootstrap" purposes: a bind extraction of an archive from a current directory of "/" needs to result in not only installation of the package contents in the correct default location, but also in correct registration of those contents into the system, minimally for one archive... but ideally for "n" archives... until a more complete tools system has been bootstrapped. If you guys all want to think about something, think first about the issues surrounding bootstrapping, and second, about the issues that surround update and failure recovery, since they are the most critical and least well supported by the current tools. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 15:30:26 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6EC7237B401; Sun, 7 Jul 2002 15:30:25 -0700 (PDT) Received: from 12-234-90-219.client.attbi.com (12-234-90-219.client.attbi.com [12.234.90.219]) by mx1.FreeBSD.org (Postfix) with ESMTP id A490243E58; Sun, 7 Jul 2002 15:30:24 -0700 (PDT) (envelope-from DougB@FreeBSD.org) Received: from Master.gorean.org (master.gorean.org [10.0.0.2]) by 12-234-90-219.client.attbi.com (8.12.3/8.12.3) with ESMTP id g67MUIBu088510; Sun, 7 Jul 2002 15:30:18 -0700 (PDT) (envelope-from DougB@FreeBSD.org) Received: from localhost (doug@localhost) by Master.gorean.org (8.12.5/8.12.5/Submit) with ESMTP id g67MRda0001595; Sun, 7 Jul 2002 15:27:39 -0700 (PDT) X-Authentication-Warning: Master.gorean.org: doug owned process doing -bs Date: Sun, 7 Jul 2002 15:27:39 -0700 (PDT) From: Doug Barton To: Dag-Erling Smorgrav Cc: Dan Moschuk , Subject: Re: Package system flaws? In-Reply-To: <3D28BEAC.4A5BA84@mindspring.com> Message-ID: <20020707152651.V679-100000@master.gorean.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, 7 Jul 2002, Terry Lambert wrote: > > We want to be able to install a package from a non-rewindable source > > without storing a temporary copy on disk. This means the metadata > > must without fail be at the very beginning of the package. Ok, then what about storing the meta data as a seperate file? Why do they have to be in the same package? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 15:30:49 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B9CF937B400 for ; Sun, 7 Jul 2002 15:30:47 -0700 (PDT) Received: from mail.inka.de (quechua.inka.de [212.227.14.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6B22B43E52 for ; Sun, 7 Jul 2002 15:30:46 -0700 (PDT) (envelope-from mailnull@mips.inka.de) Received: from kemoauc.mips.inka.de (uucp@) by mail.inka.de with local-bsmtp id 17RKYG-00026S-00; Mon, 8 Jul 2002 00:30:44 +0200 Received: from kemoauc.mips.inka.de (localhost [127.0.0.1]) by kemoauc.mips.inka.de (8.12.5/8.12.5) with ESMTP id g67Lk0nM063551 for ; Sun, 7 Jul 2002 23:46:00 +0200 (CEST) (envelope-from mailnull@localhost.mips.inka.de) Received: (from mailnull@localhost) by kemoauc.mips.inka.de (8.12.5/8.12.5/Submit) id g67Lk0IZ063550 for freebsd-arch@freebsd.org; Sun, 7 Jul 2002 23:46:00 +0200 (CEST) From: naddy@mips.inka.de (Christian Weisgerber) Subject: Re: Package system flaws? Date: Sun, 7 Jul 2002 21:45:59 +0000 (UTC) Message-ID: References: <20020706220511.GA88651@scoobysnax.jaded.net> Originator: naddy@mips.inka.de (Christian Weisgerber) To: freebsd-arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Dan Moschuk wrote: > I've been doing some thinking lately about our ports system versus what > other systems have adopted and was curious as to what people think on > the subject? What does FreeBSD do well? Where can we improve? How does it > rate against the umpteen Linux flavours? Compared to OpenBSD, the FreeBSD ports system lacks FAKE, FLAVORS, and MULTI_PACKAGES. regress would be nice, too. Compared to Debian and RPM the ability to do comprehensive updates from packages is missing. Install 500 ports. Wait a year for revisions, library versions, and even dependencies to change all over. Now let's upgrade all installed ports from a current set of binary packages. That sort of thing. -- Christian "naddy" Weisgerber naddy@mips.inka.de To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 15:36:27 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E17C637B401; Sun, 7 Jul 2002 15:36:25 -0700 (PDT) Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id D6F0543E54; Sun, 7 Jul 2002 15:36:21 -0700 (PDT) (envelope-from des@ofug.org) Received: by flood.ping.uio.no (Postfix, from userid 2602) id 1308B534A; Mon, 8 Jul 2002 00:36:20 +0200 (CEST) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Doug Barton Cc: Dan Moschuk , Subject: Re: Package system flaws? References: <20020707152651.V679-100000@master.gorean.org> From: Dag-Erling Smorgrav Date: 08 Jul 2002 00:36:19 +0200 In-Reply-To: <20020707152651.V679-100000@master.gorean.org> Message-ID: Lines: 10 User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.2 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Doug Barton writes: > Ok, then what about storing the meta data as a seperate file? Why do they > have to be in the same package? How do you ensure the metadata match the data if they're in separate files? DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 16: 0:21 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A3CA137B401; Sun, 7 Jul 2002 16:00:18 -0700 (PDT) Received: from 12-234-90-219.client.attbi.com (12-234-90-219.client.attbi.com [12.234.90.219]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1242543E4A; Sun, 7 Jul 2002 16:00:18 -0700 (PDT) (envelope-from DougB@FreeBSD.org) Received: from Master.gorean.org (master.gorean.org [10.0.0.2]) by 12-234-90-219.client.attbi.com (8.12.3/8.12.3) with ESMTP id g67N0HBu088741; Sun, 7 Jul 2002 16:00:17 -0700 (PDT) (envelope-from DougB@FreeBSD.org) Received: from localhost (doug@localhost) by Master.gorean.org (8.12.5/8.12.5/Submit) with ESMTP id g67Md2Fh001627; Sun, 7 Jul 2002 15:39:02 -0700 (PDT) X-Authentication-Warning: Master.gorean.org: doug owned process doing -bs Date: Sun, 7 Jul 2002 15:39:02 -0700 (PDT) From: Doug Barton To: Dag-Erling Smorgrav Cc: Dan Moschuk , Subject: Re: Package system flaws? In-Reply-To: Message-ID: <20020707153812.D679-100000@master.gorean.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 8 Jul 2002, Dag-Erling Smorgrav wrote: > Doug Barton writes: > > Ok, then what about storing the meta data as a seperate file? Why do they > > have to be in the same package? > > How do you ensure the metadata match the data if they're in separate > files? coolport-1.0-package.tgz coolport-1.0-meta.gz c'mon... use a little creativity. :) -- "We have known freedom's price. We have shown freedom's power. And in this great conflict, ... we will see freedom's victory." - George W. Bush, President of the United States State of the Union, January 28, 2002 Do YOU Yahoo!? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 17:43:11 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3C9C837B400; Sun, 7 Jul 2002 17:43:08 -0700 (PDT) Received: from jkh-gw.queasyweasel.com (adsl-64-173-3-158.dsl.sntc01.pacbell.net [64.173.3.158]) by mx1.FreeBSD.org (Postfix) with ESMTP id A36F143E42; Sun, 7 Jul 2002 17:43:07 -0700 (PDT) (envelope-from jkh@queasyweasel.com) Received: from adsl-64-173-15-99.dsl.sntc01.pacbell.net (jkh@mango.freebsd.com [64.173.15.99]) by jkh-gw.queasyweasel.com (8.12.3/8.12.3) with ESMTP id g680fAvm038889; Sun, 7 Jul 2002 17:41:12 -0700 (PDT) (envelope-from jkh@queasyweasel.com) Date: Sun, 7 Jul 2002 17:41:45 -0700 Subject: Re: Package system flaws? Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v482) Cc: Giorgos Keramidas , Wes Peters , Dan Moschuk , arch@FreeBSD.ORG To: Dag-Erling Smorgrav From: Jordan K Hubbard In-Reply-To: Message-Id: <7AAA514D-920B-11D6-AACD-0003938C7B7E@queasyweasel.com> Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.482) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sunday, July 7, 2002, at 01:55 PM, Dag-Erling Smorgrav wrote: > That's precisely what I want to avoid. With tar, you always have to > uncompress the entire package to find even just one file (unless you > use the GNU-specific --fast-read option, in which case it stops once > it finds the file you asked for) Actually, it's the FreeBSD-specific --fast-read option. I added it so that I could extract the +CONTENTS file without having tar go all the way through the tarball looking for additional +CONTENTS files. I'm actually with Wes anyway - zip would be a fine format on account of the fact that it: 1) Allows random access AND compression 2) Has comment fields for each archive member and for the zip file itself which can be [ab]used for storing checksums, PGP signatures, etc. 3) Can be programmatically accessed using some of the libzip libraries available so you aren't always at the mercy of being at arm's length with a command like tar (which is nice when you're trying to write a package-extracting front-end which is graphical and also has little progress meters and such which advance as each item is extracted from the archive).. But then again, I also promised myself I'd stay out of this discussion so I'll shut up now and continue to watch all of you debate this. :) -- Jordan K. Hubbard Engineering Manager, BSD technology group Apple Computer To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 19: 0:52 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A5CEC37B400; Sun, 7 Jul 2002 19:00:47 -0700 (PDT) Received: from mail.rpi.edu (mail.rpi.edu [128.113.22.40]) by mx1.FreeBSD.org (Postfix) with ESMTP id EDE1743E31; Sun, 7 Jul 2002 19:00:46 -0700 (PDT) (envelope-from drosih@rpi.edu) Received: from [128.113.24.47] (gilead.netel.rpi.edu [128.113.24.47]) by mail.rpi.edu (8.12.1/8.12.1) with ESMTP id g6820jiI066802; Sun, 7 Jul 2002 22:00:45 -0400 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: <3D27A296.D58FB4B4@softweyr.com> References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> Date: Sun, 7 Jul 2002 22:00:43 -0400 To: Wes Peters , Dan Moschuk From: Garance A Drosihn Subject: Re: Package system flaws? Cc: arch@FreeBSD.ORG Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Scanned-By: MIMEDefang 2.3 (www dot roaringpenguin dot com slash mimedefang) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG At 7:08 PM -0700 7/6/02, Wes Peters wrote: >Dan Moschuk wrote: > > I'm not sure that this classifies as an architectual discussion > > (for now) so if you feel its appropriate please feel free to > > redirect to ports@. > >I'd certainly like to see it evolve into an architectural >discussion, even if all it accomplishes is to layout the work >that needs to be done and provide some sort of road map of what >the next step or two might be. There is probably room for several different architectural discussions. One is whether we use tar vs some other archive format to hold the data and the metadata for each port. I agree that is an important topic, but I have no strong opinions on it. Whatever is decided upon by others would probably work fine for me. I do find the ports collection one of the most useful things in the freebsd community, but the more comfortable you get with depending on it, the more you see little cracks where various problems crop up -- over and over again. - - - - Separate from that, one thing I think we need is some mechanism which makes certain that the generated package-list for a port is exactly correct. I install about 70-80 ports, and I know several of them install files which are not included in their package-list. Try running find /usr/local /usr/X11R6 -type f -print0 | \ xargs -0 /usr/local/sbin/pkg_which -v | fgrep '?' Whatever we do for this, it should be something which is simple enough to use that it does not add work for the ports maintainers. Conceptually what I would like to do is have ports *built* for /usr/local (just as they are now), but then when it comes time to do the make-install, the make-install would do something like: (chroot /usr/local/TEMP ; make install) and then copy all files from /usr/local/TEMP/usr/local to the real /usr/local, based on the package-list for the port. It obviously wouldn't be quite *that* simple to do, but the idea is that you could be certain that the package list included every file that was actually installed. It would also be easy to see every file which SHOULD have been installed, because they would all be still sitting in the /usr/local/TEMP/usr/local directories. I believe that openbsd has something like this, but I do not know the details. I have also thought that maybe it could be done with Kirk's filesystem snapshots, where you'd do a snapshot before building a port, and another one afterwards, and find out what files it changed. This would be a rather heavy-handed way to approach the problem, but it avoids some of the technical issues that come up with the chroot approach. - - - - Portupgrade is a great step forward in rebuilding ports, but it also has it's limitations. It has an idea of dependences between ports, but it only knows about the dependences as of the last time the port was built. If progAA was built a month ago, and at that time it only depended on progBB, then 'portupgrade -Rr -n progAA' will only consider progAA and progBB. If the *new* version of progAA (the one you will build) now depends on progCC, you will not find out about that until portupgrade does the 'make', and it turns around to build progCC. This also shows up if you have NOCLEANDEPENDS=true in your /etc/make.conf file. Portupgrade will do a: make clean ; make ; make install; make clean for every port that it builts -- but it only knows to do that for the ports that it *already* knows about. If you ask it to install progAA, that depends on progBB and progCC, then those 'make clean's are only done for progAA. The work-directories for progBB and progCC are not cleaned out before they are made, and they are not cleaned out after they are installed. - - - - Portupgrade also looses track of what dependencies are, as various things get rebuilt. I just pkg_deinstall'ed a number of ports, and pkg_deinstall missed a good number of package dependencies. By that I mean it quite happy removed some port, even though there were other ports still installed which in fact depended on that port. [that was fine for me in this case, but it was still a little disappointing...] That's a problem when things get new versions. It gets even more confusing when ports are renamed, or entirely removed. - - - - The current ports mechanism has the idea "this port depends on file XYZZY, and if that file does not exist then you should first build port PLOUGH". It also needs the idea that "this port depends on XYZZY, but it must the XYZZY built by XFree86-4, and not the one built by XFree86-3". - - - - As we move into more platforms (sparc64, ia-64, ppc), then I expect that will add even more to the work of the ports collection. - - - - In addition to the above issues, there is also the issue of how large and unwieldy the ports collection is, and how it's "one single giant collection" (as far as something like the INDEX is concerned, for instance). It would be nice to rearrange it a bit, such that I can easily clip off all foreign-language ports that I know I don't want. Right now I can kinda do that by changing my cvsup file, and changing my portupgrade-config file, and editing /usr/ports/Makefile, and... >I'm more interested in the binary package side personally, a >holdover from a previous professional involvement in this area. >I have a number of ideas for things that could be improved in >relatively small projects if someone wants to discuss those >with me. I have had a number of ideas, but no time to really pursue them. I think we try to stuff too much information into the name of a port, and we try to do too much to shoehorn all ports-processing into standard makefile variables and standard make-cmd processing. [in the background, the sound of people screaming is heard...] And lastly, one of the problems with our ports and package system is that it has grown so large without addressing some of these issues. Any attempt to switch to a significantly different mechanism would have to include a plan to EASE INTO the new approach, probably over months of time. There is no way we are going to convert 7,000 ports to any new mechanism, on multiple HW platforms, on a wide-range of freebsd-versions, in some single one-week cut-over period. -- Garance Alistair Drosehn = gad@gilead.netel.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 19: 1:38 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 451FF37B400; Sun, 7 Jul 2002 19:01:36 -0700 (PDT) Received: from thuvia.demon.co.uk (thuvia.demon.co.uk [193.237.34.248]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4F96443E31; Sun, 7 Jul 2002 19:01:34 -0700 (PDT) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (dotar.thuvia.org [10.0.0.4]) by phaidor.thuvia.org (8.11.6/8.11.6) with ESMTP id g681xlb08143; Mon, 8 Jul 2002 02:59:47 +0100 (BST) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (localhost [IPv6:::1]) by dotar.thuvia.org (8.12.3/8.12.3) with ESMTP id g681xkKC040305; Mon, 8 Jul 2002 02:59:46 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.3/8.12.3/Submit) id g681xkTX040304; Mon, 8 Jul 2002 02:59:46 +0100 (BST) Date: Mon, 8 Jul 2002 02:59:46 +0100 (BST) From: Mark Valentine Message-Id: <200207080159.g681xkTX040304@dotar.thuvia.org> In-Reply-To: Jordan K Hubbard's message of Jul 8, 12:44am X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: jkh@queasyweasel.com (Jordan K Hubbard), Dag-Erling Smorgrav Subject: Re: Package system flaws? Cc: Giorgos Keramidas , Wes Peters , Dan Moschuk , arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: jkh@queasyweasel.com (Jordan K Hubbard) > Date: Mon 8 Jul, 2002 > Subject: Re: Package system flaws? > Actually, it's the FreeBSD-specific --fast-read option. I added it so > that I could extract the +CONTENTS file without having tar go all the > way through the tarball looking for additional +CONTENTS files. Thanks for the clarification. It would be nice if the package format didn't rely on a specific network client functionality for performance (thought you can mitigate this with a CGI/PHP script on the server in some environments). Of course the format designer can still try to say he's only trying to solve the network package install problem for FreeBSD, or that the entry barrier for efficient implementation is this specific functionality, but that limits the usefulness of the design a little... > I'm actually with Wes anyway - zip would be a fine format on account of > the fact that it: > > 1) Allows random access AND compression At the expense of having to seek to the end first? What about access to the metadata over a slow data stream, like you might have got with tar --fast-read? Cheers, Mark. -- Mark Valentine, Thuvia Labs "Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses "We're kind of stupid that way." *munch* *munch* and endorses FreeBSD -- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 20: 1:59 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5C22237B400; Sun, 7 Jul 2002 20:01:56 -0700 (PDT) Received: from jkh-gw.queasyweasel.com (adsl-64-173-3-158.dsl.sntc01.pacbell.net [64.173.3.158]) by mx1.FreeBSD.org (Postfix) with ESMTP id A0C5F43E58; Sun, 7 Jul 2002 20:01:55 -0700 (PDT) (envelope-from jkh@queasyweasel.com) Received: from adsl-64-173-15-99.dsl.sntc01.pacbell.net (jkh@mango.freebsd.com [64.173.15.99]) by jkh-gw.queasyweasel.com (8.12.3/8.12.3) with ESMTP id g6831dvm039071; Sun, 7 Jul 2002 20:01:39 -0700 (PDT) (envelope-from jkh@queasyweasel.com) Date: Sun, 7 Jul 2002 20:02:13 -0700 Subject: Re: Package system flaws? Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v482) Cc: Dag-Erling Smorgrav , Giorgos Keramidas , Wes Peters , Dan Moschuk , arch@freebsd.org To: Mark Valentine From: Jordan K Hubbard In-Reply-To: <200207080159.g681xkTX040304@dotar.thuvia.org> Message-Id: <1A55D91B-921F-11D6-AACD-0003938C7B7E@queasyweasel.com> Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.482) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sunday, July 7, 2002, at 06:59 PM, Mark Valentine wrote: >> 1) Allows random access AND compression > > At the expense of having to seek to the end first? What about access > to the > metadata over a slow data stream, like you might have got with tar > --fast-read? Having to seek to the end is, indeed, one of the major draw-backs of zip. I have no idea why the originators, in their infinite wisdom, put it there. I also don't much like the red hat approach of concocting a totally new archive format, however, even if its an agglomeration of an existing archive format. Perhaps Garrett's pax recommendation has more merit than we thought, unless people have other candidates to put forward. - Jordan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 20:35: 9 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C5D1D37B400; Sun, 7 Jul 2002 20:35:07 -0700 (PDT) Received: from thuvia.demon.co.uk (thuvia.demon.co.uk [193.237.34.248]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0F64E43E42; Sun, 7 Jul 2002 20:35:06 -0700 (PDT) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (dotar.thuvia.org [10.0.0.4]) by phaidor.thuvia.org (8.11.6/8.11.6) with ESMTP id g683YRb08850; Mon, 8 Jul 2002 04:34:27 +0100 (BST) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (localhost [IPv6:::1]) by dotar.thuvia.org (8.12.3/8.12.3) with ESMTP id g683YQKC042548; Mon, 8 Jul 2002 04:34:26 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.3/8.12.3/Submit) id g683YQsg042547; Mon, 8 Jul 2002 04:34:26 +0100 (BST) Date: Mon, 8 Jul 2002 04:34:26 +0100 (BST) From: Mark Valentine Message-Id: <200207080334.g683YQsg042547@dotar.thuvia.org> In-Reply-To: Jordan K Hubbard's message of Jul 7, 8:02pm X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: Jordan K Hubbard Subject: Re: Package system flaws? Cc: Dag-Erling Smorgrav , Giorgos Keramidas , Wes Peters , Dan Moschuk , arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: Jordan K Hubbard > Date: Sun 7 Jul, 2002 > Subject: Re: Package system flaws? > Perhaps Garrett's pax recommendation has more merit > than we thought, unless people have other candidates to put forward. It made me sit up and think, but you almost certainly have to extend pax(1) to realise the extensions (correct me if I'm wrong), and I rather like a format I can script with existing utilities. Cheers, Mark. -- Mark Valentine, Thuvia Labs "Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses "We're kind of stupid that way." *munch* *munch* and endorses FreeBSD -- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 20:55:16 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B4E5D37B400; Sun, 7 Jul 2002 20:55:13 -0700 (PDT) Received: from scaup.mail.pas.earthlink.net (scaup.mail.pas.earthlink.net [207.217.120.49]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6724743E3B; Sun, 7 Jul 2002 20:55:13 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0280.cvx21-bradley.dialup.earthlink.net ([209.179.193.25] helo=mindspring.com) by scaup.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17RPcB-000279-00; Sun, 07 Jul 2002 23:55:07 -0400 Message-ID: <3D290CEF.7181BE71@mindspring.com> Date: Sun, 07 Jul 2002 20:54:23 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Doug Barton Cc: Dag-Erling Smorgrav , Dan Moschuk , arch@FreeBSD.org Subject: Re: Package system flaws? References: <20020707152651.V679-100000@master.gorean.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Doug Barton wrote: > On Sun, 7 Jul 2002, Terry Lambert wrote: > > > We want to be able to install a package from a non-rewindable source > > > without storing a temporary copy on disk. This means the metadata > > > must without fail be at the very beginning of the package. > > Ok, then what about storing the meta data as a seperate file? Why do they > have to be in the same package? For the same reason the fact that root owns /etc/master.passwd is not in a seperate file? -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 20:56:19 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4CCE137B400 for ; Sun, 7 Jul 2002 20:56:12 -0700 (PDT) Received: from khavrinen.lcs.mit.edu (khavrinen.lcs.mit.edu [18.24.4.193]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6B4DB43E4A for ; Sun, 7 Jul 2002 20:56:12 -0700 (PDT) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: from khavrinen.lcs.mit.edu (localhost [IPv6:::1]) by khavrinen.lcs.mit.edu (8.12.3/8.12.3) with ESMTP id g683uBRC028227; Sun, 7 Jul 2002 23:56:11 -0400 (EDT) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: (from wollman@localhost) by khavrinen.lcs.mit.edu (8.12.3/8.12.3/Submit) id g683uA0v028226; Sun, 7 Jul 2002 23:56:10 -0400 (EDT) (envelope-from wollman) Date: Sun, 7 Jul 2002 23:56:10 -0400 (EDT) From: Garrett Wollman Message-Id: <200207080356.g683uA0v028226@khavrinen.lcs.mit.edu> To: drosih@rpi.edu Subject: Re: Package system flaws? X-Newsgroups: mit.lcs.mail.freebsd-arch In-Reply-To: References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> Organization: MIT Laboratory for Computer Science Cc: arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In article Garance writes: >Separate from that, one thing I think we need is some mechanism >which makes certain that the generated package-list for a port >is exactly correct. The ports-build cluster does that, but only for packages. The problem stems mainly from the fact that, when you build a port from source, you're not starting from a clean environment (unlike bento). Many configure scripts will then glom onto whatever random software you happen to have installed (``Gee! You seem to have emacs installed! Wouldn't you like to play with my Emacs Lisp files?'') and builds or installs extra stuff that doesn't make it into the official package. Since bento never notices it, maintainers are not frequently motivated to deal with the problem. (I've run into this recently working on FreeRADIUS, which I've had to instruct to build only static executables, with no loadable modules, in order to deal cleanly with this behavior.) The ports system really does not have a good way of dealing with conditional dependencies (``I can use one of these if you have it''). For many packages, there are simply too many possible combinations for the maintainer to effectively test all of them. It gets even worse when some packages depend on specific options of other packages (viz., GNOME needs libxml+Python but KDE and docproj do not). There really isn't any good mechanism of supporting this -- there are already too many ports which differ from another based only on the setting of some option or other. It would be nice to have a mechanism like the following (since we're all expressing our wishlists here): - Each port contains an enumeration of the possible variants and on-off options. Some options might not make sense with all possible variants, so the listing of options is per-variant. Some options might not be compatible with each other, so there needs to be a way to handle that as well. (The distinction I'm making here is that a `variant' changes the behavior of the resulting program or library, in such a way as other programs or libraries using it might depend upon, whereas an `option' simply causes additional files to be installed, which might or might not add functionality.) - Each variant must have a human-friendly short name and comment, including the default variant. Likewise options, except that the default state (which must be `off') does not have a name. The whole thing can be referred to as: $PORTNAME${variant+"-$variant"}${options+"+$(echo $options | tr ' ' '+')"} - Whether building from source or installing a package, users are required to explicitly select the variant of their choice. If no variant is selected, the user gets a list of possibilities and is invited to select one (and any options that might apply). pkg_info options and make targets are provided for automatically displaying this list without attempting to install. - The package build cluster does combinatorial expansion on each port, and builds provisional packages for all variants and permissible option combinations. (Maintainers are encouraged not to go overboard on this!) (Here's the important new part, which would require a significantly enhanced package format....) - The package build cluster, having constructed all of the possible combinations of options and variants, and having made provisional packages of each one, runs a unification process across all of the provisional packages to create a single ``fat'' package with metainformation indicating which versions of which files are present for which variant and option combinations. This mechanism could also be used to create multi-architecture packages, but in most cases the executables take up the lion's share of the space so doing this makes little sense. The functionality should be available, though, in case third-party package creators wish to do so. Possible Makefile syntax might look something like this: VARIANTS= generic foo bar COMMENT_generic= "Whatever pkg-plist used to say" OPTIONS_generic= ipv6 COMMENT_foo= "Use the foo graphical user interface" OPTIONS_foo= ipv6 quux:!ipv6 COMMENT_bar= "Use the bar graphical user interface" COMMENT_ipv6= "Include support for IP version 6" COMMENT_quux= "Build a loadable completion module for the quux shell" .if "${VARIANT}" = "foo" || "${VARIANT}" = "generic" .if "${OPTIONS:M/ipv6/}" != "" PATCH_FILES+= ${PORTNAME}-ipv6-patch .endif .endif .if "${VARIANT}" = "foo" CONFIGURE_ARGS+=--with-libfoo LIB_DEPENDS+= foo.23:${PORTSDIR}/x11/foo-lib .if "${OPTIONS:M/quux/}" != "" CONFIGURE_ARGS+=--enable-quux-completion BUILD_DEPENDS+= ${PREFIX}/include/quux/complete.h:${PORTSDIR}/shells/quux::generic+dso # depend specifically on the `dso' option of the `generic' variant of quux .endif .endif .if "${VARIANT}" = "bar" CONFIGURE_ARGS+=--with-libbar LIB_DEPENDS+= bar.42:${PORTSDIR}/x11/bar-base::42 # depend on variant `42' of the bar-base package .endif OK, so that's a lot of work, and I'm not exactly volunteering. But it would be very nice to have. -GAWollman -- Garrett A. Wollman | [G]enes make enzymes, and enzymes control the rates of wollman@lcs.mit.edu | chemical processes. Genes do not make ``novelty- Opinions not those of| seeking'' or any other complex and overt behavior. MIT, LCS, CRS, or NSA| - Stephen Jay Gould (1941-2002) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 21: 6: 3 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5F30137B400 for ; Sun, 7 Jul 2002 21:06:01 -0700 (PDT) Received: from khavrinen.lcs.mit.edu (khavrinen.lcs.mit.edu [18.24.4.193]) by mx1.FreeBSD.org (Postfix) with ESMTP id C734443E31 for ; Sun, 7 Jul 2002 21:06:00 -0700 (PDT) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: from khavrinen.lcs.mit.edu (localhost [IPv6:::1]) by khavrinen.lcs.mit.edu (8.12.3/8.12.3) with ESMTP id g68460RC028328; Mon, 8 Jul 2002 00:06:00 -0400 (EDT) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: (from wollman@localhost) by khavrinen.lcs.mit.edu (8.12.3/8.12.3/Submit) id g6845xCf028327; Mon, 8 Jul 2002 00:05:59 -0400 (EDT) (envelope-from wollman) Date: Mon, 8 Jul 2002 00:05:59 -0400 (EDT) From: Garrett Wollman Message-Id: <200207080405.g6845xCf028327@khavrinen.lcs.mit.edu> To: jkh@queasyweasel.com Subject: Re: Package system flaws? X-Newsgroups: mit.lcs.mail.freebsd-arch In-Reply-To: <1A55D91B-921F-11D6-AACD-0003938C7B7E@queasyweasel.com> References: <200207080159.g681xkTX040304@dotar.thuvia.org> Organization: MIT Laboratory for Computer Science Cc: arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In article <1A55D91B-921F-11D6-AACD-0003938C7B7E@queasyweasel.com> Jordan writes: [what is that, a GUID in that Message-ID field?!] >Having to seek to the end is, indeed, one of the major draw-backs of >zip. I have no idea why the originators, in their infinite wisdom, put >it there. I do. (Was I the last person to still be using an IBM PC when this happened back in 1988?) Recall that Phil Katz was under legal pressure to create something that was as entirely unlike System Enhancement Associates' ARC as possible. ARC used a `distributed directory' model (stolen from tar) because that made it trivial to append to an archive -- just overwrite the ``end of archive'' block at the end of the file with new data, and eventually a new ``end of archive'' block. Katz wanted to preserve this ability without using the `distributed directory' model, because floppy disks were and are unspeakably slow, and even seeking through an archive to list all the files was painful compared to reading a single sector with all the directory information in it. If the directory is at the beginning, it can never be expanded without making a copy of the entire archive (painfully inconvenient for a 320K archive on a 360K floppy). So, he put the directory at the end, where it was easy to find (in a maximum of two seeks) and could be overwritten when extending the archive, just like the old `end of archive' header in ARC format. -GAWollman -- Garrett A. Wollman | [G]enes make enzymes, and enzymes control the rates of wollman@lcs.mit.edu | chemical processes. Genes do not make ``novelty- Opinions not those of| seeking'' or any other complex and overt behavior. MIT, LCS, CRS, or NSA| - Stephen Jay Gould (1941-2002) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jul 7 21:10:17 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B8E5637B400 for ; Sun, 7 Jul 2002 21:10:15 -0700 (PDT) Received: from khavrinen.lcs.mit.edu (khavrinen.lcs.mit.edu [18.24.4.193]) by mx1.FreeBSD.org (Postfix) with ESMTP id E569A43E42 for ; Sun, 7 Jul 2002 21:10:14 -0700 (PDT) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: from khavrinen.lcs.mit.edu (localhost [IPv6:::1]) by khavrinen.lcs.mit.edu (8.12.3/8.12.3) with ESMTP id g684AERC028360; Mon, 8 Jul 2002 00:10:14 -0400 (EDT) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: (from wollman@localhost) by khavrinen.lcs.mit.edu (8.12.3/8.12.3/Submit) id g684ADZt028359; Mon, 8 Jul 2002 00:10:13 -0400 (EDT) (envelope-from wollman) Date: Mon, 8 Jul 2002 00:10:13 -0400 (EDT) From: Garrett Wollman Message-Id: <200207080410.g684ADZt028359@khavrinen.lcs.mit.edu> To: mark@thuvia.demon.co.uk Subject: Re: Package system flaws? In-Reply-To: <200207080334.g683YQsg042547@dotar.thuvia.org> Organization: MIT Laboratory for Computer Science Cc: arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In article <200207080334.g683YQsg042547@dotar.thuvia.org> you write: >It made me sit up and think, but you almost certainly have to extend pax(1) >to realise the extensions (correct me if I'm wrong), and I rather like a >format I can script with existing utilities. Not at all. You certainly *could* do it that way, but whether it's worth it to do so depends greatly on what it is that you want to do (that you have declined to share) with the archives other than installing them. Using the pax headers to store package metainformation is no more difficult to deal with than using the comment field in a ZIP archive's directory to do the same thing. If you're just examining the package, you can use a purpose-built tool to get the metainformation out, and if you don't need that, you don't need the tool either. The pax specification provides that unrecognized extension headers are to be ignored. -GAWollman -- Garrett A. Wollman | [G]enes make enzymes, and enzymes control the rates of wollman@lcs.mit.edu | chemical processes. Genes do not make ``novelty- Opinions not those of| seeking'' or any other complex and overt behavior. MIT, LCS, CRS, or NSA| - Stephen Jay Gould (1941-2002) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 0: 0:26 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0ECEB37B400 for ; Mon, 8 Jul 2002 00:00:22 -0700 (PDT) Received: from 12-234-90-219.client.attbi.com (12-234-90-219.client.attbi.com [12.234.90.219]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7429C43E31 for ; Mon, 8 Jul 2002 00:00:21 -0700 (PDT) (envelope-from DougB@FreeBSD.org) Received: from Master.gorean.org (master.gorean.org [10.0.0.2]) by 12-234-90-219.client.attbi.com (8.12.3/8.12.3) with ESMTP id g6870JBu093034; Mon, 8 Jul 2002 00:00:19 -0700 (PDT) (envelope-from DougB@FreeBSD.org) Received: from localhost (doug@localhost) by Master.gorean.org (8.12.5/8.12.5/Submit) with ESMTP id g686l3WR002298; Sun, 7 Jul 2002 23:47:04 -0700 (PDT) X-Authentication-Warning: Master.gorean.org: doug owned process doing -bs Date: Sun, 7 Jul 2002 23:47:03 -0700 (PDT) From: Doug Barton To: Garrett Wollman Cc: mark@thuvia.demon.co.uk, Subject: Re: Package system flaws? In-Reply-To: <200207080410.g684ADZt028359@khavrinen.lcs.mit.edu> Message-ID: <20020707234138.X2247-100000@master.gorean.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG As much as I love zip, it's probably not appropriate for compressing the binary packages we actually distribute. I did a quick test with zip vs. bzip2, both on max compression, and the bzip'ed tarball was 40% smaller than the .zip file. Not only is bandwidth an issue for our users, but there is currently a problem just pushing the packages from the cluster out to ftp-master. Not to mention pushing them around to the mirrors. One of the reasons I suggested the "compress the binaries, then compress the metadata plus binary tarball" method is that it is more space efficient. I think that we should start asking the questions about how we can end up with the smallest packages, then go backwards from there to decide how to beat that format into doing what we want. Doug To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 0: 0:34 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F0C2A37B405; Mon, 8 Jul 2002 00:00:25 -0700 (PDT) Received: from 12-234-90-219.client.attbi.com (12-234-90-219.client.attbi.com [12.234.90.219]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9148C43E31; Mon, 8 Jul 2002 00:00:25 -0700 (PDT) (envelope-from DougB@FreeBSD.org) Received: from Master.gorean.org (master.gorean.org [10.0.0.2]) by 12-234-90-219.client.attbi.com (8.12.3/8.12.3) with ESMTP id g6870JBw093034; Mon, 8 Jul 2002 00:00:24 -0700 (PDT) (envelope-from DougB@FreeBSD.org) Received: from localhost (doug@localhost) by Master.gorean.org (8.12.5/8.12.5/Submit) with ESMTP id g686ZY20002289; Sun, 7 Jul 2002 23:35:35 -0700 (PDT) X-Authentication-Warning: Master.gorean.org: doug owned process doing -bs Date: Sun, 7 Jul 2002 23:35:34 -0700 (PDT) From: Doug Barton To: Terry Lambert Cc: Dag-Erling Smorgrav , Dan Moschuk , Subject: Re: Package system flaws? In-Reply-To: <3D290CEF.7181BE71@mindspring.com> Message-ID: <20020707233525.U2247-100000@master.gorean.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, 7 Jul 2002, Terry Lambert wrote: > Doug Barton wrote: > > On Sun, 7 Jul 2002, Terry Lambert wrote: > > > > We want to be able to install a package from a non-rewindable source > > > > without storing a temporary copy on disk. This means the metadata > > > > must without fail be at the very beginning of the package. > > > > Ok, then what about storing the meta data as a seperate file? Why do they > > have to be in the same package? > > For the same reason the fact that root owns /etc/master.passwd is > not in a seperate file? ENOPARSE To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 0:14:24 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 863DA37B400; Mon, 8 Jul 2002 00:14:22 -0700 (PDT) Received: from thuvia.demon.co.uk (thuvia.demon.co.uk [193.237.34.248]) by mx1.FreeBSD.org (Postfix) with ESMTP id AE55043E31; Mon, 8 Jul 2002 00:14:20 -0700 (PDT) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (dotar.thuvia.org [10.0.0.4]) by phaidor.thuvia.org (8.11.6/8.11.6) with ESMTP id g687EIb09462; Mon, 8 Jul 2002 08:14:18 +0100 (BST) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (localhost [IPv6:::1]) by dotar.thuvia.org (8.12.3/8.12.3) with ESMTP id g687EHKC047235; Mon, 8 Jul 2002 08:14:17 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.3/8.12.3/Submit) id g687EH6t047234; Mon, 8 Jul 2002 08:14:17 +0100 (BST) Date: Mon, 8 Jul 2002 08:14:17 +0100 (BST) From: Mark Valentine Message-Id: <200207080714.g687EH6t047234@dotar.thuvia.org> In-Reply-To: Doug Barton's message of Jul 7, 11:47pm X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: Doug Barton , Garrett Wollman Subject: Re: Package system flaws? Cc: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: Doug Barton > Date: Sun 7 Jul, 2002 > Subject: Re: Package system flaws? > One of the reasons I suggested the "compress the binaries, then compress > the metadata plus binary tarball" method is that it is more space > efficient. Compressing the "metadata + binary tarball" just lost you the ability to access the metadata without uncompressing the whole caboodle. It isn't _all_ about space (but space helps a lot). See my earlier suggestion which was basically "compress the binaries, then _archive_ the metadata plus binary tarball". Cheers, Mark. -- Mark Valentine, Thuvia Labs "Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses "We're kind of stupid that way." *munch* *munch* and endorses FreeBSD -- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 1: 0:35 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 893F737B40A for ; Mon, 8 Jul 2002 01:00:27 -0700 (PDT) Received: from 12-234-90-219.client.attbi.com (12-234-90-219.client.attbi.com [12.234.90.219]) by mx1.FreeBSD.org (Postfix) with ESMTP id D2F2643E52 for ; Mon, 8 Jul 2002 01:00:24 -0700 (PDT) (envelope-from DougB@FreeBSD.org) Received: from Master.gorean.org (master.gorean.org [10.0.0.2]) by 12-234-90-219.client.attbi.com (8.12.3/8.12.3) with ESMTP id g6880IC0094671; Mon, 8 Jul 2002 01:00:20 -0700 (PDT) (envelope-from DougB@FreeBSD.org) Received: from localhost (doug@localhost) by Master.gorean.org (8.12.5/8.12.5/Submit) with ESMTP id g687WK11002402; Mon, 8 Jul 2002 00:32:21 -0700 (PDT) X-Authentication-Warning: Master.gorean.org: doug owned process doing -bs Date: Mon, 8 Jul 2002 00:32:20 -0700 (PDT) From: Doug Barton To: Mark Valentine Cc: Garrett Wollman , Subject: Re: Package system flaws? In-Reply-To: <200207080714.g687EH6t047234@dotar.thuvia.org> Message-ID: <20020708001921.C2247-100000@master.gorean.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, 8 Jul 2002, Mark Valentine wrote: > > From: Doug Barton > > Date: Sun 7 Jul, 2002 > > Subject: Re: Package system flaws? > > > One of the reasons I suggested the "compress the binaries, then compress > > the metadata plus binary tarball" method is that it is more space > > efficient. > > Compressing the "metadata + binary tarball" just lost you the ability to > access the metadata without uncompressing the whole caboodle. Well, if people are dead set on having both things in the same package (I still think two seperate files is a cleaner solution) then as long as the binaries are compressed efficiently (using tar + bzip or some such) then we can use a less efficient, though more friendly alternative to compress the metadata + binary bit. Assuming that we use the most efficient means possible to compress the binaries (and by binaries I of course mean "what the package is designed to install") then it's not going to compress further, no matter what method we use to squish the two together. > It isn't _all_ about space (but space helps a lot). Don't underestimate this factor. Pushing the packages out to ftp-master from bento is already a limiting factor on how often we can update the package set. We also have to take foreign mirrors, and users who pay for every byte they download into account. There is also another reason to consider seperating the binary tarball and the metadata that I haven't mentioned yet. And actually, now that I think of it more it's another good reason to seperate the two things into different files. If I have package A that depends on package B, under the current system if we up the version of package B, we have to re-roll package A altogether just to update the dependency data, even though nothing about the binary has, or needs to change. By seperating the metadata and the binaries we can just update the metadata with the new dependency and push just that. > See my earlier suggestion which was basically "compress the binaries, > then _archive_ the metadata plus binary tarball". Sorry to say I skimmed that too rapidly. It sounds like we agree roughly on this point, although I'd love for us to use a non-GNU tool for this (sorry, blatant prejudice there). Doug -- "We have known freedom's price. We have shown freedom's power. And in this great conflict, ... we will see freedom's victory." - George W. Bush, President of the United States State of the Union, January 28, 2002 Do YOU Yahoo!? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 2:30:19 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5317D37B407; Mon, 8 Jul 2002 02:30:11 -0700 (PDT) Received: from avocet.mail.pas.earthlink.net (avocet.mail.pas.earthlink.net [207.217.120.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id CADE643E4A; Mon, 8 Jul 2002 02:30:10 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0110.cvx40-bradley.dialup.earthlink.net ([216.244.42.110] helo=mindspring.com) by avocet.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17RUqP-0002pO-00; Mon, 08 Jul 2002 05:30:10 -0400 Message-ID: <3D295B77.E62ED5FC@mindspring.com> Date: Mon, 08 Jul 2002 02:29:27 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Doug Barton Cc: Dag-Erling Smorgrav , Dan Moschuk , arch@FreeBSD.org Subject: Re: Package system flaws? References: <20020707233525.U2247-100000@master.gorean.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Doug Barton wrote: > On Sun, 7 Jul 2002, Terry Lambert wrote: > > Doug Barton wrote: > > > On Sun, 7 Jul 2002, Terry Lambert wrote: > > > > > We want to be able to install a package from a non-rewindable source > > > > > without storing a temporary copy on disk. This means the metadata > > > > > must without fail be at the very beginning of the package. > > > > > > Ok, then what about storing the meta data as a seperate file? Why do they > > > have to be in the same package? > > > > For the same reason the fact that root owns /etc/master.passwd is > > not in a seperate file? > > ENOPARSE Metadata is metadata. If it's good to put metadata in a file seperate from the data it describes, then the judgement of "goodness" is universal. We store the metadata for individual files in most file systems in a shared structure which contains both the references to the data and to the metadata (exceptions are FS's which do not support doing this intrinsically, e.g. MSDOSFS, or FS's which have to store it seperately in order to achieve minimal POSIX semantics, e.g. Udo Walter's "UMSDOSFS" under Linux). We do this because it makes the data and metadata non-severable, it relieves us of having to consider synchronization issues which would otherwise arise, and we do it because it's convenient in terms of speed of operations involving both, and convenient in terms of locality of reference. It also gets rid of the implied graph edge for locking of data and metadata, which can lead to an undetectable deadly embrace deadlock . All of these arguments apply equally well to bundling package metadata with package data: conceptually, that metadata is no different than file ownership, flags, or permissions. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 7:31:20 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ADE4B37B400; Mon, 8 Jul 2002 07:31:17 -0700 (PDT) Received: from thuvia.demon.co.uk (thuvia.demon.co.uk [193.237.34.248]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1948F43E54; Mon, 8 Jul 2002 07:31:12 -0700 (PDT) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (dotar.thuvia.org [10.0.0.4]) by phaidor.thuvia.org (8.11.6/8.11.6) with ESMTP id g68EUmb10797; Mon, 8 Jul 2002 15:30:48 +0100 (BST) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (localhost [IPv6:::1]) by dotar.thuvia.org (8.12.3/8.12.3) with ESMTP id g68EUlKC062948; Mon, 8 Jul 2002 15:30:47 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.3/8.12.3/Submit) id g68EUgoZ062947; Mon, 8 Jul 2002 15:30:42 +0100 (BST) Date: Mon, 8 Jul 2002 15:30:42 +0100 (BST) From: Mark Valentine Message-Id: <200207081430.g68EUgoZ062947@dotar.thuvia.org> In-Reply-To: Garance A Drosihn's message of Jul 8, 2:04am X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: drosih@rpi.edu (Garance A Drosihn), Wes Peters , Dan Moschuk Subject: Re: Package system flaws? Cc: arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: drosih@rpi.edu (Garance A Drosihn) > Date: Mon 8 Jul, 2002 > Subject: Re: Package system flaws? > I do find the ports collection one of the most useful things in > the freebsd community, but the more comfortable you get with > depending on it, the more you see little cracks where various > problems crop up -- over and over again. Indeed the system is very useful already. But here are my main gripes anyway... 1. Packages install to /usr/local by default; however, they do not (and cannot) follow my long-standing (since 4.2BSD) cross-platform administrative policies for /usr/local. In fact, just attempting a package install on those systems generally screws up the permissions on /usr/local fatally. The next generation system should install to /usr/pkg or /opt or whatever. (Policy differences here: my /usr/local is generally administered by group "staff", not user "root"; I put in a bit of effort to generally _not_ require the co-existence of multiple versions of a package - a lot of the ports patches are there to accomplish just this, renaming libraries and data directories - I don't want it). 2. I'd like to see improved support for building/installing packages as non-root. Installing as root should only be necessary if local policy requires it, or if there are set[ug]id components to install (or similar permission requirements). Even in the latter case it would be nice if the install created a simple script to run as root to fix up the permissions. I especially don't like having to build ports as root (I consider bsd.port.mk unauditable, and I know a thing or two about make(1)). 3. There's no way to have pkg_add -r use a local package cache so that only the first install needs to fetch the package _and its dependencies_ over the Internet (see Cygwin's setup.exe for an example of how this can be done reasonably well). I went to add this feature and my stumbling block was that pkg_add doesn't know the versioned name of the package archive to store (but only for the primary package, the dependencies are fine). That's to say that "pkg_add -r XFree86" should fetch and store (if necessary) "XFree86-4.2.0_1,1.tgz" and its dependencies; alternatively, "pkg_add -r XFree86-4.2.0_1,1.tgz" should do the right thing. Currently, to install a package on multiple systems, I have to jump through hoops to get the package and its dependencies into my local package area. > Separate from that, one thing I think we need is some mechanism > which makes certain that the generated package-list for a port > is exactly correct. I like your idea for creating packages from a chroot'd area, though I'd probably prefer this to be an optional testing feature than the default method (but yes, nobody would use it then...). Cheers, Mark. -- Mark Valentine, Thuvia Labs "Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses "We're kind of stupid that way." *munch* *munch* and endorses FreeBSD -- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 7:38:26 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2819337B400 for ; Mon, 8 Jul 2002 07:38:25 -0700 (PDT) Received: from mx02.egartech.com (aloha.egartech.com [62.118.81.133]) by mx1.FreeBSD.org (Postfix) with SMTP id 2F94A43E31 for ; Mon, 8 Jul 2002 07:38:23 -0700 (PDT) (envelope-from temik@egartech.com) Received: (qmail 69896 invoked by uid 85); 8 Jul 2002 14:40:37 -0000 Received: from temik@egartech.com by mx02.egartech.com with qmail-scanner-1.03 (. Clean. Processed in 0.374209 secs); 08 Jul 2002 14:40:37 -0000 Received: from unknown (HELO turtle.egar.egartech.com) (192.168.8.4) by 0 with SMTP; 8 Jul 2002 14:40:36 -0000 Subject: RE: Package system flaws? MIME-Version: 1.0 Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: quoted-printable Date: Mon, 8 Jul 2002 18:38:19 +0400 content-class: urn:content-classes:message X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Message-ID: <5235EF9BAE6B7F4CB3735789EEF73B296E6146@turtle.egar.egartech.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Package system flaws? Thread-Index: AcImK9WdHmjp3HCXTVqVZ99rdi35jwAYPi0g From: "Artem Tepponen" To: "Jordan K Hubbard" Cc: "Dag-Erling Smorgrav" , "Giorgos Keramidas" , "Wes Peters" , "Dan Moschuk" , Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > >> 1) Allows random access AND compression > > > > At the expense of having to seek to the end first? What=20 > > about access to the metadata over a slow data stream, > > like you might have got with tar --fast-read? >=20 > Having to seek to the end is, indeed, one of the major draw-backs of=20 > zip. I have no idea why the originators, in their infinite=20 > wisdom, put it there. They optimized for adding files to huge archive. Artem Tepponen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 7:41:15 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 506E637B400 for ; Mon, 8 Jul 2002 07:41:10 -0700 (PDT) Received: from thuvia.demon.co.uk (thuvia.demon.co.uk [193.237.34.248]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9CEE943E42 for ; Mon, 8 Jul 2002 07:41:08 -0700 (PDT) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (dotar.thuvia.org [10.0.0.4]) by phaidor.thuvia.org (8.11.6/8.11.6) with ESMTP id g68Ef4b10860; Mon, 8 Jul 2002 15:41:04 +0100 (BST) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (localhost [IPv6:::1]) by dotar.thuvia.org (8.12.3/8.12.3) with ESMTP id g68Ef4KC063248; Mon, 8 Jul 2002 15:41:04 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.3/8.12.3/Submit) id g68Ef3fr063247; Mon, 8 Jul 2002 15:41:03 +0100 (BST) Date: Mon, 8 Jul 2002 15:41:03 +0100 (BST) From: Mark Valentine Message-Id: <200207081441.g68Ef3fr063247@dotar.thuvia.org> In-Reply-To: Garrett Wollman's message of Jul 8, 12:10am X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: Garrett Wollman Subject: Re: Package system flaws? Cc: arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: Garrett Wollman > Date: Mon 8 Jul, 2002 > Subject: Re: Package system flaws? > In article <200207080334.g683YQsg042547@dotar.thuvia.org> you write: > > >It made me sit up and think, but you almost certainly have to extend pax(1) > >to realise the extensions (correct me if I'm wrong), and I rather like a > >format I can script with existing utilities. > > Not at all. You certainly *could* do it that way, but whether it's > worth it to do so depends greatly on what it is that you want to do > (that you have declined to share) with the archives other than > installing them. Using the pax headers to store package > metainformation is no more difficult to deal with than using the > comment field in a ZIP archive's directory to do the same thing. If > you're just examining the package, you can use a purpose-built tool to > get the metainformation out, and if you don't need that, you don't > need the tool either. Now I'm confused. I'd assumed you were suggesting using pax(1) extensions to provide some means of direct access to the archive members, not for storing metadata... What advantage is there in storing the metadata as extended pax(1) headers instead of as the first file(s) in the archive? And how does this help solve the problem of direct access to metadata while keeping the package's files compressed? Cheers, Mark. -- Mark Valentine, Thuvia Labs "Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses "We're kind of stupid that way." *munch* *munch* and endorses FreeBSD -- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 7:55:45 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 323DC37B400; Mon, 8 Jul 2002 07:55:42 -0700 (PDT) Received: from thuvia.demon.co.uk (thuvia.demon.co.uk [193.237.34.248]) by mx1.FreeBSD.org (Postfix) with ESMTP id 119FB43E3B; Mon, 8 Jul 2002 07:55:40 -0700 (PDT) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (dotar.thuvia.org [10.0.0.4]) by phaidor.thuvia.org (8.11.6/8.11.6) with ESMTP id g68Etcb10956; Mon, 8 Jul 2002 15:55:38 +0100 (BST) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (localhost [IPv6:::1]) by dotar.thuvia.org (8.12.3/8.12.3) with ESMTP id g68EtcKC063765; Mon, 8 Jul 2002 15:55:38 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.3/8.12.3/Submit) id g68Etclk063764; Mon, 8 Jul 2002 15:55:38 +0100 (BST) Date: Mon, 8 Jul 2002 15:55:38 +0100 (BST) From: Mark Valentine Message-Id: <200207081455.g68Etclk063764@dotar.thuvia.org> In-Reply-To: Doug Barton's message of Jul 8, 12:32am X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: Doug Barton Subject: Re: Package system flaws? Cc: Garrett Wollman , Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: Doug Barton > Date: Mon 8 Jul, 2002 > Subject: Re: Package system flaws? > On Mon, 8 Jul 2002, Mark Valentine wrote: > > Compressing the "metadata + binary tarball" just lost you the ability to > > access the metadata without uncompressing the whole caboodle. > > Well, if people are dead set on having both things in the same package (I > still think two seperate files is a cleaner solution) My earlier suggestion actually said just about the same thing, with the _option_ of storing the two (or more) parts in an uncompressed archive instead of a directory, for ease of handling. In light of Wes' comments on storing the metadata, I'd modify my examples as follows. Example 1: simple package, not sub-packaged. $ ls /var/spool/pkg/foo-x.y base.bz2 package.xml Example 2: package with optional development and documentation components $ ls /var/spool/pkg/bar-m.n base.bz2 devel.bz2 doc.bz2 package.xml (In an archive, of course, package.xml would be the first member.) > then as long as the > binaries are compressed efficiently (using tar + bzip or some such) then > we can use a less efficient, though more friendly alternative to compress > the metadata + binary bit. Bingo! > > It isn't _all_ about space (but space helps a lot). > > Don't underestimate this factor. I should have said a _lot_. ;-) > There is also another reason to consider seperating the binary tarball and > the metadata that I haven't mentioned yet. And actually, now that I think > of it more it's another good reason to seperate the two things into > different files. If I have package A that depends on package B, under the > current system if we up the version of package B, we have to re-roll > package A altogether just to update the dependency data, even though > nothing about the binary has, or needs to change. By seperating the > metadata and the binaries we can just update the metadata with the new > dependency and push just that. Indeed. > > See my earlier suggestion which was basically "compress the binaries, > > then _archive_ the metadata plus binary tarball". > > Sorry to say I skimmed that too rapidly. It sounds like we agree roughly > on this point, although I'd love for us to use a non-GNU tool for this > (sorry, blatant prejudice there). Ideally you use POSIX archive formats and any tool which implements them. Cheers, Mark. -- Mark Valentine, Thuvia Labs "Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses "We're kind of stupid that way." *munch* *munch* and endorses FreeBSD -- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 8:18:41 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 969CC37B401 for ; Mon, 8 Jul 2002 08:18:39 -0700 (PDT) Received: from mx02.egartech.com (aloha.egartech.com [62.118.81.133]) by mx1.FreeBSD.org (Postfix) with SMTP id 907A743E54 for ; Mon, 8 Jul 2002 08:18:36 -0700 (PDT) (envelope-from temik@egartech.com) Received: (qmail 71741 invoked by uid 85); 8 Jul 2002 15:20:50 -0000 Received: from temik@egartech.com by mx02.egartech.com with qmail-scanner-1.03 (. Clean. Processed in 0.426798 secs); 08 Jul 2002 15:20:50 -0000 Received: from unknown (HELO turtle.egar.egartech.com) (192.168.8.4) by 0 with SMTP; 8 Jul 2002 15:20:49 -0000 Subject: RE: Package system flaws? MIME-Version: 1.0 Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: quoted-printable Date: Mon, 8 Jul 2002 19:18:31 +0400 content-class: urn:content-classes:message X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Message-ID: <5235EF9BAE6B7F4CB3735789EEF73B2907416F@turtle.egar.egartech.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Package system flaws? Thread-Index: AcImTTIMdGyC6cuKQZ6M8QNDUqQPlwARNurQ From: "Artem Tepponen" To: "Doug Barton" Cc: "Garrett Wollman" , , Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: Doug Barton [mailto:DougB@FreeBSD.org] > Sent: Monday, July 08, 2002 10:47 AM >=20 > I think that we should start asking the questions about how=20 > we can end up with the smallest packages, then go backwards > from there to decide how to beat that format into doing what we want. You will achieve best compression with tar+(bzip2|gzip), cause format that compresses individual pieces and makes archive usually loses to make archive and compress to one file kind of format. Exceptions are very rare. The only problem is how to handle metadata piece that have to have at least two properties: 1. It should be available without whole thing being decompressed. 2. It should be available asap in case of slow link. Artem Tepponen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 8:19:23 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D5BD037B400 for ; Mon, 8 Jul 2002 08:19:21 -0700 (PDT) Received: from thuvia.demon.co.uk (thuvia.demon.co.uk [193.237.34.248]) by mx1.FreeBSD.org (Postfix) with ESMTP id 40BC043E3B for ; Mon, 8 Jul 2002 08:19:20 -0700 (PDT) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (dotar.thuvia.org [10.0.0.4]) by phaidor.thuvia.org (8.11.6/8.11.6) with ESMTP id g68FJCb11045; Mon, 8 Jul 2002 16:19:13 +0100 (BST) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (localhost [IPv6:::1]) by dotar.thuvia.org (8.12.3/8.12.3) with ESMTP id g68FJCKC064278; Mon, 8 Jul 2002 16:19:12 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.3/8.12.3/Submit) id g68FJBXJ064277; Mon, 8 Jul 2002 16:19:11 +0100 (BST) Date: Mon, 8 Jul 2002 16:19:11 +0100 (BST) From: Mark Valentine Message-Id: <200207081519.g68FJBXJ064277@dotar.thuvia.org> In-Reply-To: Garrett Wollman's message of Jul 8, 4:04am X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: wollman@lcs.mit.edu (Garrett Wollman), drosih@rpi.edu Subject: Re: Package system flaws? Cc: arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: wollman@lcs.mit.edu (Garrett Wollman) > Date: Mon 8 Jul, 2002 > Subject: Re: Package system flaws? > It would be nice to have a mechanism like the following (since we're > all expressing our wishlists here): > > - Each port contains an enumeration of the possible variants and > on-off options. Some options might not make sense with all possible > variants, so the listing of options is per-variant. Some options > might not be compatible with each other, so there needs to be a way > to handle that as well. > > (The distinction I'm making here is that a `variant' changes the > behavior of the resulting program or library, in such a way as other > programs or libraries using it might depend upon, whereas an `option' > simply causes additional files to be installed, which might or might > not add functionality.) I like this taxonomy. Cheers, Mark. -- Mark Valentine, Thuvia Labs "Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses "We're kind of stupid that way." *munch* *munch* and endorses FreeBSD -- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 8:47:30 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 857A537B400 for ; Mon, 8 Jul 2002 08:47:27 -0700 (PDT) Received: from rover.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6288143E09 for ; Mon, 8 Jul 2002 08:47:26 -0700 (PDT) (envelope-from imp@village.org) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.11.3/8.11.3) with ESMTP id g68FlJY67464; Mon, 8 Jul 2002 09:47:20 -0600 (MDT) (envelope-from imp@village.org) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.11.6/8.11.6) with ESMTP id g68FlIG98049; Mon, 8 Jul 2002 09:47:19 -0600 (MDT) (envelope-from imp@village.org) Date: Mon, 08 Jul 2002 09:46:52 -0600 (MDT) Message-Id: <20020708.094652.21114246.imp@village.org> To: naddy@mips.inka.de Cc: freebsd-arch@FreeBSD.ORG Subject: Re: Package system flaws? From: "M. Warner Losh" In-Reply-To: References: <20020706220511.GA88651@scoobysnax.jaded.net> X-Mailer: Mew version 2.1 on Emacs 21.1 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message: naddy@mips.inka.de (Christian Weisgerber) writes: : Compared to Debian and RPM the ability to do comprehensive updates : from packages is missing. Install 500 ports. Wait a year for : revisions, library versions, and even dependencies to change all : over. Now let's upgrade all installed ports from a current set of : binary packages. That sort of thing. portupgrade does his very thing. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 8:58:24 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9DB2937B400; Mon, 8 Jul 2002 08:58:19 -0700 (PDT) Received: from scoobysnax.jaded.net (d141-7-230.home.cgocable.net [24.141.7.230]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4AF5543E09; Mon, 8 Jul 2002 08:58:13 -0700 (PDT) (envelope-from dan@scoobysnax.jaded.net) Received: from scoobysnax.jaded.net (localhost [127.0.0.1]) by scoobysnax.jaded.net (8.12.3/8.12.3) with ESMTP id g68Fgt2g003325; Mon, 8 Jul 2002 11:42:55 -0400 (EDT) (envelope-from dan@scoobysnax.jaded.net) Received: (from dan@localhost) by scoobysnax.jaded.net (8.12.3/8.12.3/Submit) id g68FgtIY003324; Mon, 8 Jul 2002 11:42:55 -0400 (EDT) Date: Mon, 8 Jul 2002 11:42:55 -0400 From: Dan Moschuk To: Artem Tepponen Cc: Doug Barton , Garrett Wollman , mark@thuvia.demon.co.uk, arch@FreeBSD.ORG Subject: Re: Package system flaws? Message-ID: <20020708154255.GC304@scoobysnax.jaded.net> References: <5235EF9BAE6B7F4CB3735789EEF73B2907416F@turtle.egar.egartech.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5235EF9BAE6B7F4CB3735789EEF73B2907416F@turtle.egar.egartech.com> User-Agent: Mutt/1.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG | > I think that we should start asking the questions about how | > we can end up with the smallest packages, then go backwards | > from there to decide how to beat that format into doing what we want. | | You will achieve best compression with tar+(bzip2|gzip), | cause format that compresses individual pieces and makes | archive usually loses to make archive and compress to one | file kind of format. Exceptions are very rare. | | The only problem is how to handle metadata piece that | have to have at least two properties: | 1. It should be available without whole thing being decompressed. | 2. It should be available asap in case of slow link. I think the meta data should be available uncompressed prepended to the .tar.bz2 archive. Zipping would also work, however as someone earlier pointed out zip compression is sub-optimal compared to bzip (or gzip for that matter). Considering the sheer amount (and size) of some of our packages I don't think that this is a minor issue. I'll be preparing a summary of this whole thread later on today. -Dan -- That poverty is no disaster is understood by everyone who has not yet succumbed to the madness of greed and luxury that turns everything topsy-turvy. -- Seneca To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 9: 1:44 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 05A0837B405 for ; Mon, 8 Jul 2002 09:01:43 -0700 (PDT) Received: from mx02.egartech.com (aloha.egartech.com [62.118.81.133]) by mx1.FreeBSD.org (Postfix) with SMTP id 40DB343E3B for ; Mon, 8 Jul 2002 09:01:39 -0700 (PDT) (envelope-from temik@egartech.com) Received: (qmail 73365 invoked by uid 85); 8 Jul 2002 16:03:52 -0000 Received: from temik@egartech.com by mx02.egartech.com with qmail-scanner-1.03 (. Clean. Processed in 0.411249 secs); 08 Jul 2002 16:03:52 -0000 Received: from unknown (HELO turtle.egar.egartech.com) (192.168.8.4) by 0 with SMTP; 8 Jul 2002 16:03:52 -0000 Subject: RE: Package system flaws? MIME-Version: 1.0 Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: quoted-printable Date: Mon, 8 Jul 2002 20:01:35 +0400 content-class: urn:content-classes:message X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Message-ID: <5235EF9BAE6B7F4CB3735789EEF73B29074170@turtle.egar.egartech.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Package system flaws? Thread-Index: AcImliUo6eE46+MKTqmvlAIMDJ+sVQAANCrg From: "Artem Tepponen" To: "Dan Moschuk" Cc: "Doug Barton" , "Garrett Wollman" , , Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: Dan Moschuk [mailto:dan@FreeBSD.ORG] > Sent: Monday, July 08, 2002 7:43 PM > | The only problem is how to handle metadata piece that > | have to have at least two properties: > | 1. It should be available without whole thing being decompressed. > | 2. It should be available asap in case of slow link. >=20 > I think the meta data should be available uncompressed=20 > prepended to the .tar.bz2 archive. Zipping would also work, > however as someone earlier pointed out zip compression > is sub-optimal compared to bzip (or gzip for that matter). > Considering the sheer amount (and size) of some of our > packages I don't think that this is a minor issue. I guess that two files solution has some nice properties like not having to update the whole shebang only when dependencies are changed. It just needs a secure way to ensure that two files are paired (MD5?). But it loses convenience and beauty of single file scheme. Artem Tepponen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 10:12:48 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DE59037B400; Mon, 8 Jul 2002 10:12:43 -0700 (PDT) Received: from hotmail.com (f229.law14.hotmail.com [64.4.21.229]) by mx1.FreeBSD.org (Postfix) with ESMTP id A010343E52; Mon, 8 Jul 2002 10:12:43 -0700 (PDT) (envelope-from xx_jayrod_x@hotmail.com) Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Mon, 8 Jul 2002 10:12:43 -0700 Received: from 212.68.199.163 by lw14fd.law14.hotmail.msn.com with HTTP; Mon, 08 Jul 2002 17:12:43 GMT X-Originating-IP: [212.68.199.163] From: "Jefferson Harlough" To: dan@FreeBSD.ORG, temik@egartech.com Cc: DougB@FreeBSD.ORG, wollman@lcs.mit.edu, mark@thuvia.demon.co.uk, arch@FreeBSD.ORG Subject: Re: Package system flaws? Date: Mon, 08 Jul 2002 17:12:43 +0000 Mime-Version: 1.0 Content-Type: text/plain; format=flowed Message-ID: X-OriginalArrivalTime: 08 Jul 2002 17:12:43.0611 (UTC) FILETIME=[AC3E32B0:01C226A2] Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG >I think the meta data should be available uncompressed prepended to the >.tar.bz2 archive. Zipping would also work, however as someone earlier >pointed out zip compression is sub-optimal compared to bzip (or gzip for >that matter). Considering the sheer amount (and size) of some of our >packages I don't think that this is a minor issue. Would it be possible to replace the current compression algorithm(s) used in zip with the bzip2 compression algorithms? Would that allow for a greater degree of compression while still offering the benefits of the zip archive format? Package size is quite an issue to me personally. I do not yet have a broadband connection, and I much prefer to download archives compressed with the bzip2 software as they usually smaller than archives compressed with gzip or compress. I'd have to ask that package size is considered as a very important part of the decision, and hopefully a system that allows for a high degree of compression will be used in the end. Sincerely, Jefferson _________________________________________________________________ Send and receive Hotmail on your mobile device: http://mobile.msn.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 11:24:50 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 10B5937B400; Mon, 8 Jul 2002 11:24:47 -0700 (PDT) Received: from mail2.panix.com (mail2.panix.com [166.84.1.73]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9BB4A43E42; Mon, 8 Jul 2002 11:24:46 -0700 (PDT) (envelope-from rsi@panix.com) Received: from panix1.panix.com (panix1.panix.com [166.84.1.1]) by mail2.panix.com (Postfix) with ESMTP id 107C5917F; Mon, 8 Jul 2002 14:24:46 -0400 (EDT) Received: (from rsi@localhost) by panix1.panix.com (8.11.3nb1/8.8.8/PanixN1.0) id g68IOko07696; Mon, 8 Jul 2002 14:24:46 -0400 (EDT) Message-Id: <200207081824.g68IOko07696@panix1.panix.com> X-Authentication-Warning: panix1.panix.com: rsi set sender to rsi@panix.com using -f To: "Artem Tepponen" Cc: "Doug Barton" , "Garrett Wollman" , , Subject: Re: Package system flaws? References: <5235EF9BAE6B7F4CB3735789EEF73B2907416F@turtle.egar.egartech.com> From: Rajappa Iyer Date: 08 Jul 2002 14:24:46 -0400 Reply-To: rsi@panix.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG "Artem Tepponen" writes: > The only problem is how to handle metadata piece that > have to have at least two properties: > 1. It should be available without whole thing being decompressed. > 2. It should be available asap in case of slow link. It seems to me that the solution is staring us right in the face. Currently, the ports mechanism already separates data and metadata. IMHO, the best solution is to extend the ports mechanism so that it can be told to fetch and install binaries instead of building it locally. IOW, perhaps one additional file per port and some additional rules in bsd.port.mk. This seems to me to be the cleanest solution. Thanks, Rajappa -- a.k.a. Rajappa Iyer. They also surf who stand in the waves. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 12:12:20 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 69DA637B400 for ; Mon, 8 Jul 2002 12:12:19 -0700 (PDT) Received: from khavrinen.lcs.mit.edu (khavrinen.lcs.mit.edu [18.24.4.193]) by mx1.FreeBSD.org (Postfix) with ESMTP id C32C243E3B for ; Mon, 8 Jul 2002 12:12:18 -0700 (PDT) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: from khavrinen.lcs.mit.edu (localhost [IPv6:::1]) by khavrinen.lcs.mit.edu (8.12.3/8.12.3) with ESMTP id g68JCHRC032746; Mon, 8 Jul 2002 15:12:17 -0400 (EDT) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: (from wollman@localhost) by khavrinen.lcs.mit.edu (8.12.3/8.12.3/Submit) id g68JCHMi032743; Mon, 8 Jul 2002 15:12:17 -0400 (EDT) (envelope-from wollman) Date: Mon, 8 Jul 2002 15:12:17 -0400 (EDT) From: Garrett Wollman Message-Id: <200207081912.g68JCHMi032743@khavrinen.lcs.mit.edu> To: Mark Valentine Cc: arch@freebsd.org Subject: Re: Package system flaws? In-Reply-To: <200207081441.g68Ef3fr063247@dotar.thuvia.org> References: <200207081441.g68Ef3fr063247@dotar.thuvia.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG < said: > What advantage is there in storing the metadata as extended pax(1) headers > instead of as the first file(s) in the archive? For one, representing files specific to a variant or option. You could also have one that meant `the contents of this member file are compressed', or one that meant `the contents of this member file are another archive'. Perhaps both at the same time. -GAWollman To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 12:54:36 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DEC4A37B40D for ; Mon, 8 Jul 2002 12:54:32 -0700 (PDT) Received: from thuvia.demon.co.uk (thuvia.demon.co.uk [193.237.34.248]) by mx1.FreeBSD.org (Postfix) with ESMTP id C3B2F43E31 for ; Mon, 8 Jul 2002 12:54:30 -0700 (PDT) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (dotar.thuvia.org [10.0.0.4]) by phaidor.thuvia.org (8.11.6/8.11.6) with ESMTP id g68Jk5b12323; Mon, 8 Jul 2002 20:46:05 +0100 (BST) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (localhost [IPv6:::1]) by dotar.thuvia.org (8.12.3/8.12.3) with ESMTP id g68Jk5KC071232; Mon, 8 Jul 2002 20:46:05 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.3/8.12.3/Submit) id g68Jk5NB071231; Mon, 8 Jul 2002 20:46:05 +0100 (BST) Date: Mon, 8 Jul 2002 20:46:05 +0100 (BST) From: Mark Valentine Message-Id: <200207081946.g68Jk5NB071231@dotar.thuvia.org> In-Reply-To: Garrett Wollman's message of Jul 8, 3:12pm X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: Garrett Wollman Subject: Re: Package system flaws? Cc: arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: Garrett Wollman > Date: Mon 8 Jul, 2002 > Subject: Re: Package system flaws? > < said: > > > What advantage is there in storing the metadata as extended pax(1) headers > > instead of as the first file(s) in the archive? > > For one, representing files specific to a variant or option. > > You could also have one that meant `the contents of this member file are > compressed', or one that meant `the contents of this member file are > another archive'. Perhaps both at the same time. As much as I hate to rely on it in general, for specific applications filename conventions can be very useful. What do your extensions do that a suffix of -gtk.tar.gz in the archive TOC doesn't? Cheers, Mark. -- Mark Valentine, Thuvia Labs "Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses "We're kind of stupid that way." *munch* *munch* and endorses FreeBSD -- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 13:12:58 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B705637B400 for ; Mon, 8 Jul 2002 13:12:56 -0700 (PDT) Received: from khavrinen.lcs.mit.edu (khavrinen.lcs.mit.edu [18.24.4.193]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1C34A43E52 for ; Mon, 8 Jul 2002 13:12:56 -0700 (PDT) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: from khavrinen.lcs.mit.edu (localhost [IPv6:::1]) by khavrinen.lcs.mit.edu (8.12.3/8.12.3) with ESMTP id g68KCtRC033313; Mon, 8 Jul 2002 16:12:55 -0400 (EDT) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: (from wollman@localhost) by khavrinen.lcs.mit.edu (8.12.3/8.12.3/Submit) id g68KCsda033307; Mon, 8 Jul 2002 16:12:54 -0400 (EDT) (envelope-from wollman) Date: Mon, 8 Jul 2002 16:12:54 -0400 (EDT) From: Garrett Wollman Message-Id: <200207082012.g68KCsda033307@khavrinen.lcs.mit.edu> To: Mark Valentine Cc: arch@freebsd.org Subject: Re: Package system flaws? In-Reply-To: <200207081946.g68Jk5NB071231@dotar.thuvia.org> References: <200207081946.g68Jk5NB071231@dotar.thuvia.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG < said: > As much as I hate to rely on it in general, for specific applications > filename conventions can be very useful. What do your extensions do > that a suffix of -gtk.tar.gz in the archive TOC doesn't? Avoids encoding kluges like that into archives and archivers. -GAWollman To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 13:48:36 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CEB3637B400 for ; Mon, 8 Jul 2002 13:48:33 -0700 (PDT) Received: from thuvia.demon.co.uk (thuvia.demon.co.uk [193.237.34.248]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1511443E3B for ; Mon, 8 Jul 2002 13:48:32 -0700 (PDT) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (dotar.thuvia.org [10.0.0.4]) by phaidor.thuvia.org (8.11.6/8.11.6) with ESMTP id g68KmRb12742; Mon, 8 Jul 2002 21:48:27 +0100 (BST) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (localhost [IPv6:::1]) by dotar.thuvia.org (8.12.3/8.12.3) with ESMTP id g68KmRKC073046; Mon, 8 Jul 2002 21:48:27 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.3/8.12.3/Submit) id g68KmQL2073045; Mon, 8 Jul 2002 21:48:26 +0100 (BST) Date: Mon, 8 Jul 2002 21:48:26 +0100 (BST) From: Mark Valentine Message-Id: <200207082048.g68KmQL2073045@dotar.thuvia.org> In-Reply-To: Garrett Wollman's message of Jul 8, 4:12pm X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: Garrett Wollman Subject: Re: Package system flaws? Cc: arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: Garrett Wollman > Date: Mon 8 Jul, 2002 > Subject: Re: Package system flaws? > < said: > > > As much as I hate to rely on it in general, for specific applications > > filename conventions can be very useful. What do your extensions do > > that a suffix of -gtk.tar.gz in the archive TOC doesn't? > > Avoids encoding kluges like that into archives and archivers. OK, I thought you might have some compelling technical reason up your sleeve (ugh, did I just use that word?). This is an innocuous kludge. We're only encoding this (common!) convention here for specific application purposes. That makes scripting simple, and avoids having to modify utilities to understand extensions. Seems a reasonable compromise to me (and I'm usually labelled an uncompromising idealist...). Cheers, Mark. -- Mark Valentine, Thuvia Labs "Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses "We're kind of stupid that way." *munch* *munch* and endorses FreeBSD -- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 14:20:23 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5EBE337B42C for ; Mon, 8 Jul 2002 14:20:14 -0700 (PDT) Received: from swan.mail.pas.earthlink.net (swan.mail.pas.earthlink.net [207.217.120.123]) by mx1.FreeBSD.org (Postfix) with ESMTP id 90F0D43E09 for ; Mon, 8 Jul 2002 14:20:13 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0159.cvx21-bradley.dialup.earthlink.net ([209.179.192.159] helo=mindspring.com) by swan.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17RfvV-0001Rh-00; Mon, 08 Jul 2002 17:20:10 -0400 Message-ID: <3D2A01DF.EF997A91@mindspring.com> Date: Mon, 08 Jul 2002 14:19:27 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Garrett Wollman Cc: Mark Valentine , arch@freebsd.org Subject: Re: Package system flaws? References: <200207081441.g68Ef3fr063247@dotar.thuvia.org> <200207081912.g68JCHMi032743@khavrinen.lcs.mit.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Garrett Wollman wrote: > < said: > > What advantage is there in storing the metadata as extended pax(1) headers > > instead of as the first file(s) in the archive? > > For one, representing files specific to a variant or option. > > You could also have one that meant `the contents of this member file are > compressed', or one that meant `the contents of this member file are > another archive'. Perhaps both at the same time. THis is very important. Specifically, another of the main problems, besides no "metadata first" priority, of the tgz format is that the index should be uncompressed while the contents are compressed. Historically, the whole archive, metadata and all, was compressed in order to support seperate uncompress and tar. But it makes just as much sense to support seperate tar and uncompress -- i.e. compress the individual files, rather than compressing all files plus the metadata. Most compression will work better on individual files rather than random file contents because of dictionary locality, anyway. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 14:23: 6 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7388B37B405 for ; Mon, 8 Jul 2002 14:23:01 -0700 (PDT) Received: from swan.mail.pas.earthlink.net (swan.mail.pas.earthlink.net [207.217.120.123]) by mx1.FreeBSD.org (Postfix) with ESMTP id 34C5743E09 for ; Mon, 8 Jul 2002 14:23:01 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0159.cvx21-bradley.dialup.earthlink.net ([209.179.192.159] helo=mindspring.com) by swan.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17RfyD-0005NX-00; Mon, 08 Jul 2002 17:22:58 -0400 Message-ID: <3D2A0285.559EF2E@mindspring.com> Date: Mon, 08 Jul 2002 14:22:13 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Mark Valentine Cc: Garrett Wollman , arch@freebsd.org Subject: Re: Package system flaws? References: <200207081946.g68Jk5NB071231@dotar.thuvia.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Mark Valentine wrote: > > From: Garrett Wollman > > You could also have one that meant `the contents of this member file are > > compressed', or one that meant `the contents of this member file are > > another archive'. Perhaps both at the same time. > > As much as I hate to rely on it in general, for specific applications > filename conventions can be very useful. What do your extensions do > that a suffix of -gtk.tar.gz in the archive TOC doesn't? Access to metadata in archives with compressed contents without the need to decompress everything. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 14:27: 6 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1CC1437B400 for ; Mon, 8 Jul 2002 14:27:05 -0700 (PDT) Received: from thuvia.demon.co.uk (thuvia.demon.co.uk [193.237.34.248]) by mx1.FreeBSD.org (Postfix) with ESMTP id 419F643E4A for ; Mon, 8 Jul 2002 14:27:02 -0700 (PDT) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (dotar.thuvia.org [10.0.0.4]) by phaidor.thuvia.org (8.11.6/8.11.6) with ESMTP id g68LQsb12956; Mon, 8 Jul 2002 22:26:55 +0100 (BST) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (localhost [IPv6:::1]) by dotar.thuvia.org (8.12.3/8.12.3) with ESMTP id g68LQsKC074174; Mon, 8 Jul 2002 22:26:54 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.3/8.12.3/Submit) id g68LQsCc074173; Mon, 8 Jul 2002 22:26:54 +0100 (BST) Date: Mon, 8 Jul 2002 22:26:54 +0100 (BST) From: Mark Valentine Message-Id: <200207082126.g68LQsCc074173@dotar.thuvia.org> In-Reply-To: Terry Lambert's message of Jul 8, 2:19pm X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: Terry Lambert , Garrett Wollman Subject: Re: Package system flaws? Cc: arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: Terry Lambert > Date: Mon 8 Jul, 2002 > Subject: Re: Package system flaws? > Most compression will work better on individual files rather than random > file contents because of dictionary locality, anyway. If you're saying what I think you are, then this is very contrary to what we've been led to believe to date. Cheers, Mark. -- Mark Valentine, Thuvia Labs "Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses "We're kind of stupid that way." *munch* *munch* and endorses FreeBSD -- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 14:48:29 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A75EA37B400 for ; Mon, 8 Jul 2002 14:48:27 -0700 (PDT) Received: from flamingo.mail.pas.earthlink.net (flamingo.mail.pas.earthlink.net [207.217.120.232]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3AD3643E31 for ; Mon, 8 Jul 2002 14:48:27 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0159.cvx21-bradley.dialup.earthlink.net ([209.179.192.159] helo=mindspring.com) by flamingo.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17RgMh-0001DR-00; Mon, 08 Jul 2002 17:48:15 -0400 Message-ID: <3D2A0872.FE692237@mindspring.com> Date: Mon, 08 Jul 2002 14:47:30 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Mark Valentine Cc: Garrett Wollman , arch@freebsd.org Subject: Re: Package system flaws? References: <200207082126.g68LQsCc074173@dotar.thuvia.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Mark Valentine wrote: > > From: Terry Lambert > > Most compression will work better on individual files rather than random > > file contents because of dictionary locality, anyway. > > If you're saying what I think you are, then this is very contrary to what > we've been led to believe to date. ??? The "Welch" part that Unisys's Terry Welch added to Lempel-Ziv-Welch algorithm used by the UNIX "compress" utility was a calculation of a data domain specific dictionary, and a reset of the dictionary every N K in order to hadle file content locality (e.g. text vs. object files vs. other). In other words, the reason LZW is *so* much better than LZ is that it knows about dictionary locality. If you know this going in, then your locality is going to change any time you toggle between data and metadata, and since there is great benefit to being able to access metadata even if content is compressed, then the answer seems obvious... -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 15:15:43 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 230D737B400 for ; Mon, 8 Jul 2002 15:15:40 -0700 (PDT) Received: from mrout2.yahoo.com (mrout2.yahoo.com [216.145.54.172]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4F1D743E4A for ; Mon, 8 Jul 2002 15:15:39 -0700 (PDT) (envelope-from DougB@FreeBSD.org) Received: from zoot.corp.yahoo.com (zoot.corp.yahoo.com [216.145.52.89]) by mrout2.yahoo.com (8.11.6/8.11.6/y.out) with ESMTP id g68MFCR22064; Mon, 8 Jul 2002 15:15:12 -0700 (PDT) Received: from localhost (dougb@localhost) by zoot.corp.yahoo.com (8.12.5/8.12.5/Submit) with ESMTP id g68MFCw5084723; Mon, 8 Jul 2002 15:15:12 -0700 (PDT) Date: Mon, 8 Jul 2002 15:15:12 -0700 (PDT) From: Doug Barton To: Terry Lambert Cc: arch@FreeBSD.org Subject: Re: Package system flaws? In-Reply-To: <3D295B77.E62ED5FC@mindspring.com> Message-ID: <20020708150549.D84324-100000@zoot.corp.yahoo.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, 8 Jul 2002, Terry Lambert wrote: > Metadata is metadata. For sufficiently broad definitions of metadata, ok. > If it's good to put metadata in a file seperate from the data it > describes, then the judgement of "goodness" is universal. This I disagree with. Insert obligatory quote about "foolish consistency." > We do this because it makes the data and metadata non-severable, See a later post by me on this thread where I clearly state severability as a specific, and desirable goal for this application. > it relieves us of having to consider synchronization issues which > would otherwise arise, and we do it because it's convenient in > terms of speed of operations involving both, and convenient in > terms of locality of reference. In this case, I think that the costs of synchronization are very small, and easily addressable by existing tools. Whereas, the benefits of severability are very great, and easily offset the costs of both not keeping them together, and the cost of actually performing and maintaining the severance. > It also gets rid of the implied graph edge for locking of data and > metadata, which can lead to an undetectable deadly embrace deadlock . I agree that this is a factor we need to keep an eye on. However, I think that the problem space is sufficiently small that we'll easily pass the 90/10 rule, and may even get as high as 95/5 with little additional effort. > All of these arguments apply equally well to bundling package > metadata with package data: conceptually, that metadata is no > different than file ownership, flags, or permissions. And here is where we would need to actually define which metadata we're considering splitting out where. I think dependancy data is a slam dunk for being split out into a seperate file, both because it's already been identified as something we need to know before we download the whole binary package, and because it is rather likely to change independent of the actual binaries themselves. I'm studiously trying to avoid focusing on implementation details because I'd like to spend some more time discussing the problem space, but if it'll help us understand the problems better, I could describe what I have in mind in a little more detail.... Doug -- "We have known freedom's price. We have shown freedom's power. And in this great conflict, ... we will see freedom's victory." - George W. Bush, President of the United States State of the Union, January 28, 2002 Do YOU Yahoo!? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 15:37:42 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6FAD537B405; Mon, 8 Jul 2002 15:37:32 -0700 (PDT) Received: from softweyr.com (softweyr.com [65.88.244.127]) by mx1.FreeBSD.org (Postfix) with ESMTP id 77F9243E42; Mon, 8 Jul 2002 15:37:31 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from nextgig-9.customer.nethere.net ([209.132.102.169] helo=softweyr.com) by softweyr.com with esmtp (Exim 3.35 #1) id 17Rh8H-0000ky-00; Mon, 08 Jul 2002 16:37:25 -0600 Message-ID: <3D2A151B.4261B51@softweyr.com> Date: Mon, 08 Jul 2002 15:41:31 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Mark Valentine Cc: Doug Barton , Garrett Wollman , arch@freebsd.org Subject: Re: Package system flaws? References: <200207081455.g68Etclk063764@dotar.thuvia.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Mark Valentine wrote: > > > From: Doug Barton > > Date: Mon 8 Jul, 2002 > > Subject: Re: Package system flaws? > > > On Mon, 8 Jul 2002, Mark Valentine wrote: > > > Compressing the "metadata + binary tarball" just lost you the ability to > > > access the metadata without uncompressing the whole caboodle. > > > > Well, if people are dead set on having both things in the same package (I > > still think two seperate files is a cleaner solution) > > My earlier suggestion actually said just about the same thing, with the > _option_ of storing the two (or more) parts in an uncompressed archive > instead of a directory, for ease of handling. > > In light of Wes' comments on storing the metadata, I'd modify my examples > as follows. > > Example 1: simple package, not sub-packaged. > > $ ls /var/spool/pkg/foo-x.y > base.bz2 package.xml Ick. Why not have the XML include the base.bz2 file in whatever encoding (including direct binary) we deem appropriate? If the XML descrbing the binary blob includes the length of the blob, you can skip it with a single seek and continue reading the XML metadata. If you want truly minimal package sizes, specify the blob(s) as external URLs rather than encoding them. I don't like the multiple file thing at all. The metadata is there to glue the files together. > Example 2: package with optional development and documentation components > > $ ls /var/spool/pkg/bar-m.n > base.bz2 devel.bz2 doc.bz2 package.xml > > (In an archive, of course, package.xml would be the first member.) Or better yet, wrapped around the others. Come to think if it, it would be a simple transformation to "convert" a package from external references to a full binary, with something like a pkg_fetch command. It would read a package with external URLs for the filesets, fetch them, and re-write the package with the blobs encased. That would let you sleep overnight, then install in the next morning/evening when you have the full package and won't be killed by an interrupted download. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 15:49:56 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1425C37B400 for ; Mon, 8 Jul 2002 15:49:55 -0700 (PDT) Received: from scoobysnax.jaded.net (d141-7-230.home.cgocable.net [24.141.7.230]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3848B43E31 for ; Mon, 8 Jul 2002 15:49:54 -0700 (PDT) (envelope-from dan@scoobysnax.jaded.net) Received: from scoobysnax.jaded.net (localhost [127.0.0.1]) by scoobysnax.jaded.net (8.12.3/8.12.3) with ESMTP id g68Mnq2g005725 for ; Mon, 8 Jul 2002 18:49:52 -0400 (EDT) (envelope-from dan@scoobysnax.jaded.net) Received: (from dan@localhost) by scoobysnax.jaded.net (8.12.3/8.12.3/Submit) id g68MnqJV005724 for arch@freebsd.org; Mon, 8 Jul 2002 18:49:52 -0400 (EDT) Date: Mon, 8 Jul 2002 18:49:52 -0400 From: Dan Moschuk To: arch@freebsd.org Subject: Summary of package system flaws Message-ID: <20020708224952.GA5381@scoobysnax.jaded.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG There's been a lot of opinions and ideas expressed on this list over the last twenty four hours that seems to have side-stepped into numerous discussions. However, rather than tossing out ideas that solve problem X but do nothing about problem Y (or create problem Z) I think it would be beneficial to first decide on what functionality we are looking for in a package system. Then we can begin discussing how to best implement them. Rather than fork off another web of discussions, please email me privately what you would like to see a new package system do, and I will summarize everything and follow up tomorrow afternoon. Once a "wishlist" has been decided on, the architectural bikeshed can begin. :) Cheers! -Dan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 15:54:26 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BCF7537B400; Mon, 8 Jul 2002 15:54:24 -0700 (PDT) Received: from softweyr.com (softweyr.com [65.88.244.127]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3DB8F43E31; Mon, 8 Jul 2002 15:54:24 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from nextgig-5.customer.nethere.net ([209.132.102.165] helo=softweyr.com) by softweyr.com with esmtp (Exim 3.35 #1) id 17RhOg-0000mZ-00; Mon, 08 Jul 2002 16:54:23 -0600 Message-ID: <3D2A191C.3103D0FB@softweyr.com> Date: Mon, 08 Jul 2002 15:58:36 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Doug Barton Cc: Dag-Erling Smorgrav , Dan Moschuk , arch@FreeBSD.org Subject: Re: Package system flaws? References: <20020707152651.V679-100000@master.gorean.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Doug Barton wrote: > > On Sun, 7 Jul 2002, Terry Lambert wrote: > > > > We want to be able to install a package from a non-rewindable source > > > without storing a temporary copy on disk. This means the metadata > > > must without fail be at the very beginning of the package. > > Ok, then what about storing the meta data as a seperate file? Why do they > have to be in the same package? So you can (md5, sign) them together and know that they "apply" to each other. If you're sensible about the encoding, the metadata doesn't have to be at the beginning of the media. Consider a simple package that has a collection of binaries and two language sets; you have to install one or both language sets. You put the binaries in first, because everyone has to install those. Then you put one of the language sets, then the other. In the encoding, you specify the size of the fileset blob following, so if I'm going to skip the first set and install the second, I know how many bytes to skip over. On a tape or disk, you can seek forward this many bytes, on a socket stream you have to just read and skip that many bytes. If you're just looking at the package gleaning metadata, you skip over all 3 blobs; pkg_info would do this. The actual encoding of the blobs is inconsequential, you could use 3 different encodings to get the maximum compression on each if you wished -- as long as pkg_add supports all of those encodings. The logical firsts are PROBABLY tar (or cpio) and gzip/bzip2. Features of the encoding format like headers don't matter since we're not going to put the metadata into the encodings. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 16: 4:51 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 45F8D37B400; Mon, 8 Jul 2002 16:04:48 -0700 (PDT) Received: from softweyr.com (softweyr.com [65.88.244.127]) by mx1.FreeBSD.org (Postfix) with ESMTP id A3EDE43E3B; Mon, 8 Jul 2002 16:04:47 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from nextgig-5.customer.nethere.net ([209.132.102.165] helo=softweyr.com) by softweyr.com with esmtp (Exim 3.35 #1) id 17RhYa-0000nF-00; Mon, 08 Jul 2002 17:04:36 -0600 Message-ID: <3D2A1B77.AF0678FC@softweyr.com> Date: Mon, 08 Jul 2002 16:08:39 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Terry Lambert Cc: Dag-Erling Smorgrav , Doug Barton , Dan Moschuk , arch@FreeBSD.org Subject: Re: Package system flaws? References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> <20020707153457.GA1086@scoobysnax.jaded.net> <3D28AF72.8F14DC40@FreeBSD.org> <3D28BEAC.4A5BA84@mindspring.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Terry Lambert wrote: > > Putting the metadata up front resloves the issue of downloading > data you don't want, and also resolves the problem of being able > to collect all the metadata at the start, so that you can make > accurate time estimates, and the user can make a time-cost-basis > decision on whether or not to go ahead. > > Modern transfer protocols argue in general for a new archive format > that collects the metadata at the start of the archive to permit > such decisions to be made. > > From a general perspective, archives need to be able to be nested > recursively, and to be extracted recursively. Obviously, this is a > control mechanism enabling requirement: full extraction as a > default is necessary for "bootstrap", and selective extraction is > necessary as a component/function/customization selection method. The more you write, the more I see XML as being terribly appropriate for the package metadata. Consider the nesting problem; XML handles this with complete grace if you write it into the DTD that a package can contain a package. > Also from a general perspective, it needs to be possible to blindly > extract archives for "bootstrap" purposes: a bind extraction of an > archive from a current directory of "/" needs to result in not only > installation of the package contents in the correct default location, > but also in correct registration of those contents into the system, > minimally for one archive... but ideally for "n" archives... until a > more complete tools system has been bootstrapped. > > If you guys all want to think about something, think first about the > issues surrounding bootstrapping, and second, about the issues that > surround update and failure recovery, since they are the most critical > and least well supported by the current tools. Please give more detail about failure recovery, this is an important point and I'm not certain you mean what I mean when I think of pkg_add failure recovery. I generally worry about the atomicity of the operation; I either want all of the package I've asked for or none of it. As far as the bootstrapping issue, I'd like to see, at a minimum, pkg_add depend ONLY on libraries that are a "normal" part of the FreeBSD system. If this means we have to add a library or two to the base system, we'll make that argument when it is needed. I know all the arguments for something like pkg_add to use other executable system tools to remain in sync with them, but I'd rather see such tools (like tar) written as programs that call the same library as pkg_add. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 16: 8: 6 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7C3E137B400 for ; Mon, 8 Jul 2002 16:08:01 -0700 (PDT) Received: from softweyr.com (softweyr.com [65.88.244.127]) by mx1.FreeBSD.org (Postfix) with ESMTP id DD3ED43E42 for ; Mon, 8 Jul 2002 16:08:00 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from nextgig-3.customer.nethere.net ([209.132.102.163] helo=softweyr.com) by softweyr.com with esmtp (Exim 3.35 #1) id 17Rhbn-0000nQ-00; Mon, 08 Jul 2002 17:07:55 -0600 Message-ID: <3D2A1C47.82E52933@softweyr.com> Date: Mon, 08 Jul 2002 16:12:07 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Garrett Wollman Cc: arch@FreeBSD.org Subject: Re: Package system flaws? References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> <200207072204.g67M4V1F027052@khavrinen.lcs.mit.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Garrett Wollman wrote: > > In article <3D289529.95C1EC8A@softweyr.com> Wes wrote: > > >spersed. Zip format makes a lot of sense and is available in library > >format, but I could easily tolerate tar or cpio, as well as a variety > >of compression formats. The metadata should not be compressed for > >speed of package processing. > > `pax' format provides for a convenient extension-header mechanism that > would make this relatively easy to accomplish without inventing a new > archive format. One of these days I'm going to tear pax apart, make it pretty (and hopefully a bit faster), and give it back to the world as the gift it has needed for years. Turn most of it into a library in the process, too, so the code can be shared with tools like we're discussing. Unless I can talk YOU into doing it instead, Garrett? ;^) -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 17:47: 1 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BD0EE37B401 for ; Mon, 8 Jul 2002 17:46:57 -0700 (PDT) Received: from mail.speakeasy.net (mail16.speakeasy.net [216.254.0.216]) by mx1.FreeBSD.org (Postfix) with ESMTP id 506AE43E52 for ; Mon, 8 Jul 2002 17:46:57 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Received: (qmail 16828 invoked from network); 9 Jul 2002 00:46:54 -0000 Received: from unknown (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) by mail16.speakeasy.net (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 9 Jul 2002 00:46:54 -0000 Received: from laptop.baldwin.cx (laptop.baldwin.cx [192.168.0.4]) by server.baldwin.cx (8.11.6/8.11.6) with ESMTP id g690kjf06522; Mon, 8 Jul 2002 20:46:45 -0400 (EDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.5.2 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20020708224952.GA5381@scoobysnax.jaded.net> Date: Mon, 08 Jul 2002 20:46:56 -0400 (EDT) From: John Baldwin To: Dan Moschuk Subject: RE: Summary of package system flaws (for the 100th time) Cc: arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 08-Jul-2002 Dan Moschuk wrote: > > There's been a lot of opinions and ideas expressed on this list over the last > twenty four hours that seems to have side-stepped into numerous discussions. > > However, rather than tossing out ideas that solve problem X but do nothing > about problem Y (or create problem Z) I think it would be beneficial to first > decide on what functionality we are looking for in a package system. > Then we can begin discussing how to best implement them. > > Rather than fork off another web of discussions, please email me privately > what you would like to see a new package system do, and I will summarize > everything and follow up tomorrow afternoon. Once a "wishlist" has been > decided on, the architectural bikeshed can begin. :) *sigh* Has anyone in this discussion looked at the libh package stuff or are you all just going to sit down and hash out all the same arguments for the bazillionth time? -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 18:38:44 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 24B7C37B400; Mon, 8 Jul 2002 18:38:39 -0700 (PDT) Received: from avocet.mail.pas.earthlink.net (avocet.mail.pas.earthlink.net [207.217.120.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id C37D043E42; Mon, 8 Jul 2002 18:38:38 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0460.cvx22-bradley.dialup.earthlink.net ([209.179.199.205] helo=mindspring.com) by avocet.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17Rjxd-00057N-00; Mon, 08 Jul 2002 21:38:37 -0400 Message-ID: <3D2A3E73.8A6BFA16@mindspring.com> Date: Mon, 08 Jul 2002 18:37:55 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Doug Barton Cc: arch@FreeBSD.org Subject: Re: Package system flaws? References: <20020708150549.D84324-100000@zoot.corp.yahoo.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Doug Barton wrote: > > If it's good to put metadata in a file seperate from the data it > > describes, then the judgement of "goodness" is universal. > > This I disagree with. Insert obligatory quote about "foolish consistency." > > > We do this because it makes the data and metadata non-severable, > > See a later post by me on this thread where I clearly state severability > as a specific, and desirable goal for this application. Insert obligatory quote about "1 0wn y00r b0x". 8-). I don't understand your severability argument. Or maybe I just disagree with it at such a fundamental philosophical level that it just seems like I don't understand it. > In this case, I think that the costs of synchronization are very small, > and easily addressable by existing tools. Whereas, the benefits of > severability are very great, and easily offset the costs of both not > keeping them together, and the cost of actually performing and maintaining > the severance. I give: What are the benefits of having to downlaod multiple files, rather than a single file, via HTTP protocol? > > It also gets rid of the implied graph edge for locking of data and > > metadata, which can lead to an undetectable deadly embrace deadlock . > > I agree that this is a factor we need to keep an eye on. However, I think > that the problem space is sufficiently small that we'll easily pass the > 90/10 rule, and may even get as high as 95/5 with little additional > effort. The current code partially solves the problem. If 90/10 is OK, then the problem is solved, and we can quit discussing it, as 90% of the direct needs of people are satisfied already, I think. > > All of these arguments apply equally well to bundling package > > metadata with package data: conceptually, that metadata is no > > different than file ownership, flags, or permissions. > > And here is where we would need to actually define which metadata we're > considering splitting out where. I think dependancy data is a slam dunk > for being split out into a seperate file, both because it's already been > identified as something we need to know before we download the whole > binary package, and because it is rather likely to change independent of > the actual binaries themselves. OK, you need to clarify what you mean by "file". Do you mean "archive content element", or do you really mean *file", as in "down load this archive *file* and download this metadata *file* seperately, and hope nothing gets lost or outdated". > I'm studiously trying to avoid focusing on implementation details because > I'd like to spend some more time discussing the problem space, but if > it'll help us understand the problems better, I could describe what I have > in mind in a little more detail.... OK. But be aware that we may have different ideas of the problem space... -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jul 8 19:10:13 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9EA8937B400; Mon, 8 Jul 2002 19:10:05 -0700 (PDT) Received: from albatross.prod.itd.earthlink.net (albatross.mail.pas.earthlink.net [207.217.120.120]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0E19843E31; Mon, 8 Jul 2002 19:10:05 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0460.cvx22-bradley.dialup.earthlink.net ([209.179.199.205] helo=mindspring.com) by albatross.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 17RkRi-0000G6-00; Mon, 08 Jul 2002 22:09:43 -0400 Message-ID: <3D2A45BC.75556D69@mindspring.com> Date: Mon, 08 Jul 2002 19:09:00 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Wes Peters Cc: Dag-Erling Smorgrav , Doug Barton , Dan Moschuk , arch@FreeBSD.org Subject: Re: Package system flaws? References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> <20020707153457.GA1086@scoobysnax.jaded.net> <3D28AF72.8F14DC40@FreeBSD.org> <3D28BEAC.4A5BA84@mindspring.com> <3D2A1B77.AF0678FC@softweyr.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Wes Peters wrote: > The more you write, the more I see XML as being terribly appropriate > for the package metadata. Consider the nesting problem; XML handles > this with complete grace if you write it into the DTD that a package > can contain a package. XML would be good, if you didn't have to have 90M of crap in order to parse it. Whatever deals with this has to fit on a floppy with enough to get a temporary system up an running for installation of a permanent system. > Please give more detail about failure recovery, this is an important > point and I'm not certain you mean what I mean when I think of pkg_add > failure recovery. I generally worry about the atomicity of the > operation; I either want all of the package I've asked for or none of > it. That too. 8-). I'm talking you delete one random system file, and you want to get the system back to the state it was before you screwed up, without reinstalling everything. RPM has this facility where it can tell you what files are missing. I'm also talking about someone installing a "r00tk1t" on your machine, and you wanting to be able to recover everything that was a distribution binary back onto the system. At the same time, you want to identify everything that represents added files, so that there isn't anything left behind. And, once again, you want to do it without reinstalling everything. > As far as the bootstrapping issue, I'd like to see, at a minimum, pkg_add > depend ONLY on libraries that are a "normal" part of the FreeBSD system. > If this means we have to add a library or two to the base system, we'll > make that argument when it is needed. I know all the arguments for > something like pkg_add to use other executable system tools to remain > in sync with them, but I'd rather see such tools (like tar) written as > programs that call the same library as pkg_add. I'd like to see everything that's currently in "distribution sets" be placed in aggregate packages. I'd also like to see the ablity to "cd / ; tar xvzf base" (or any appropriate archive extraction command that it ends up using) to install a base system -- *with the base system components registered into the system as individual packages, so that e.g. "/bin/ls" can be incrementally updated at a later point in time. I'd like to be able to have a cron job that fetch'es down a list of security advisory "secure component version information" data automatically at 2 AM, and then send me an email after using the packaging tools to identify which components -- if any -- have outstanding security issues. Depending on my settings, that email will either advise me what I need to think about... or it will tell me what it updated automatically, overnight, if I've selected the "autoupdate" option. I'd like to be able to install a base system via CDROM, and then *deinstall* UUCP and OpenSSH and sendmail and bind and whatever else I want to deinstall, as if they were packages that had been installed into the system. *THEN* I'd like to be able to reinstall everything I deinstalled, and end up with exactly the same system I started with in the first place. I'd like to be able to make a decision based on expected download time as to whether or not to continue to install a package or not; specifically, I'd like to know that the 24K game "bobo" is going to want to pull down all of KDE, which is going to pull down Qt, which is going to pull down "kitchen sink" and take up 18TB of disk, and I'd like to know it _to the byte_ because the tools did an HTTP range fetch to pull down the metadata parts of each of the packages in the full dependency list so that it can tell me without guestimating, and without lying to me. I'd like to know up front how much disk space I'm going to need in /usr and / for the role I intend the machine to fill. I'd like "PicoBSD" to differ from "FreeBSD" only in terms if what is installed by default, and not have to build a different CDROM or other distribution in order to get it. I'd like to be able to say "Here's the URL of the system independent configuration data for machine A; please make machines B, D, and E look like that, but not C; make C look like R, instead. I'll be back after lunch to judge your work". I'd like to have the questions I'm going to have to answer for everything I'm going to install known *up front*, so I can answer them *up front*, and then go do useful work instead of babysitting a box and hitting "return" on a million "Please look at me, I'm lonely!" dialog boxes. In short, I don't want much... 8-). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 0:19:28 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 60E1637B400 for ; Tue, 9 Jul 2002 00:19:23 -0700 (PDT) Received: from mx02.egartech.com (aloha.egartech.com [62.118.81.133]) by mx1.FreeBSD.org (Postfix) with SMTP id D6C0443E4A for ; Tue, 9 Jul 2002 00:19:18 -0700 (PDT) (envelope-from temik@egartech.com) Received: (qmail 95876 invoked by uid 85); 9 Jul 2002 07:19:12 -0000 Received: from temik@egartech.com by mx02.egartech.com with qmail-scanner-1.03 (. Clean. Processed in 0.328881 secs); 09 Jul 2002 07:19:12 -0000 Received: from unknown (HELO turtle.egar.egartech.com) (192.168.8.4) by 0 with SMTP; 9 Jul 2002 07:19:11 -0000 Subject: RE: Package system flaws? MIME-Version: 1.0 Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: quoted-printable Date: Tue, 9 Jul 2002 11:19:13 +0400 content-class: urn:content-classes:message X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Message-ID: <5235EF9BAE6B7F4CB3735789EEF73B296E614A@turtle.egar.egartech.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Package system flaws? Thread-Index: AcImrL37j0amgPO6RRS2RDo3X+OV7AAbCyrw From: "Artem Tepponen" To: Cc: "Doug Barton" , "Garrett Wollman" , , Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: Rajappa Iyer [mailto:rsi@panix.com] > Sent: Monday, July 08, 2002 10:25 PM > > The only problem is how to handle metadata piece that > > have to have at least two properties: > > 1. It should be available without whole thing being decompressed. > > 2. It should be available asap in case of slow link. >=20 > It seems to me that the solution is staring us right in the face. > Currently, the ports mechanism already separates data and metadata. > IMHO, the best solution is to extend the ports mechanism so that it > can be told to fetch and install binaries instead of building it > locally. IOW, perhaps one additional file per port and some > additional rules in bsd.port.mk. >=20 > This seems to me to be the cleanest solution. Unfortunately this doesn't solve third party packages problem well. Note: ports do have freebsd maintainers but packages may not (well, there should be possibility for standalone thirdparty packages). Artem Tepponen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 0:31:18 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C3B2A37B405 for ; Tue, 9 Jul 2002 00:31:14 -0700 (PDT) Received: from mx02.egartech.com (aloha.egartech.com [62.118.81.133]) by mx1.FreeBSD.org (Postfix) with SMTP id 157DD43E52 for ; Tue, 9 Jul 2002 00:31:13 -0700 (PDT) (envelope-from temik@egartech.com) Received: (qmail 96453 invoked by uid 85); 9 Jul 2002 07:31:09 -0000 Received: from temik@egartech.com by mx02.egartech.com with qmail-scanner-1.03 (. Clean. Processed in 0.32657 secs); 09 Jul 2002 07:31:09 -0000 Received: from unknown (HELO turtle.egar.egartech.com) (192.168.8.4) by 0 with SMTP; 9 Jul 2002 07:31:08 -0000 Subject: RE: Package system flaws? MIME-Version: 1.0 Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: quoted-printable Date: Tue, 9 Jul 2002 11:31:10 +0400 content-class: urn:content-classes:message X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Message-ID: <5235EF9BAE6B7F4CB3735789EEF73B29074171@turtle.egar.egartech.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Package system flaws? Thread-Index: AcImoq4qegn4HgCERkm+0pSgDV+EGwAdkFuA From: "Artem Tepponen" To: "Jefferson Harlough" , Cc: , , , Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: Jefferson Harlough [mailto:xx_jayrod_x@hotmail.com] > Sent: Monday, July 08, 2002 9:13 PM > Would it be possible to replace the current compression=20 > algorithm(s) used in zip with the bzip2 compression algorithms? > Would that allow for a greater degree of compression while > still offering the benefits of the zip archive format? Yes, but it won't be zip. Major benefit of tar+bzip2 (or tar.gz) comes from the fact that compression algorithm (can) see all the files together and can use dictionary between files as opposed to per file dictionary in rar. Eslimates lie around 10-15% archive size reduction. > I'd have to ask that package size is considered as a very=20 > important part of the decision, and hopefully a system that=20 > allows for a high degree of compression will be used in the end. Sitting on a modem link I definitely agree. Artem Tepponen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 0:33:13 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CBD8A37B400 for ; Tue, 9 Jul 2002 00:33:11 -0700 (PDT) Received: from dragon.nuxi.com (trang.nuxi.com [66.92.13.169]) by mx1.FreeBSD.org (Postfix) with ESMTP id 679E243E58 for ; Tue, 9 Jul 2002 00:33:11 -0700 (PDT) (envelope-from obrien@NUXI.com) Received: from dragon.nuxi.com (obrien@localhost [127.0.0.1]) by dragon.nuxi.com (8.12.5/8.12.2) with ESMTP id g697XAoi096611; Tue, 9 Jul 2002 00:33:10 -0700 (PDT) (envelope-from obrien@dragon.nuxi.com) Received: (from obrien@localhost) by dragon.nuxi.com (8.12.5/8.12.5/Submit) id g697X91b096610; Tue, 9 Jul 2002 00:33:09 -0700 (PDT) Date: Tue, 9 Jul 2002 00:33:09 -0700 From: "David O'Brien" To: "M. Warner Losh" Cc: naddy@mips.inka.de, freebsd-arch@FreeBSD.ORG Subject: Re: Package system flaws? Message-ID: <20020709073309.GB96335@dragon.nuxi.com> Reply-To: freebsd-arch@FreeBSD.ORG References: <20020706220511.GA88651@scoobysnax.jaded.net> <20020708.094652.21114246.imp@village.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020708.094652.21114246.imp@village.org> User-Agent: Mutt/1.4i X-Operating-System: FreeBSD 5.0-CURRENT Organization: The NUXI BSD Group X-Pgp-Rsa-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Rsa-Keyid: 1024/34F9F9D5 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, Jul 08, 2002 at 09:46:52AM -0600, M. Warner Losh wrote: > In message: > naddy@mips.inka.de (Christian Weisgerber) writes: > : Compared to Debian and RPM the ability to do comprehensive updates > : from packages is missing. Install 500 ports. Wait a year for > : revisions, library versions, and even dependencies to change all > : over. Now let's upgrade all installed ports from a current set of > : binary packages. That sort of thing. > > portupgrade does his very thing. Only from /usr/ports, building each port -- it will not use precompiled packages. :-( To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 1: 9:28 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9738C37B401; Tue, 9 Jul 2002 01:09:24 -0700 (PDT) Received: from harrier.mail.pas.earthlink.net (harrier.mail.pas.earthlink.net [207.217.120.12]) by mx1.FreeBSD.org (Postfix) with ESMTP id D844443E75; Tue, 9 Jul 2002 01:09:17 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0079.cvx40-bradley.dialup.earthlink.net ([216.244.42.79] helo=mindspring.com) by harrier.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17Rq3Z-0003fC-00; Tue, 09 Jul 2002 04:09:10 -0400 Message-ID: <3D2A99F6.D4B1999E@mindspring.com> Date: Tue, 09 Jul 2002 01:08:22 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Artem Tepponen Cc: rsi@panix.com, Doug Barton , Garrett Wollman , mark@thuvia.demon.co.uk, arch@FreeBSD.org Subject: Re: Package system flaws? References: <5235EF9BAE6B7F4CB3735789EEF73B296E614A@turtle.egar.egartech.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Artem Tepponen wrote: > Unfortunately this doesn't solve third party packages problem well. > Note: ports do have freebsd maintainers but packages may not (well, > there should be possibility for standalone thirdparty packages). It doesn't solve the third party binary only kernel components problem, either. 8-). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 1:29:36 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CC16D37B400 for ; Tue, 9 Jul 2002 01:29:31 -0700 (PDT) Received: from infinitive.futureperfectcorporation.com (infinitive.futureperfectcorporation.com [196.25.137.68]) by mx1.FreeBSD.org (Postfix) with SMTP id 68B6D43E42 for ; Tue, 9 Jul 2002 01:29:21 -0700 (PDT) (envelope-from nbm@gerund.futureperfectcorporation.com) Received: (qmail 76210 invoked by uid 0); 9 Jul 2002 08:27:46 -0000 Received: from gerund.futureperfectcorporation.com (196.25.137.65) by infinitive.futureperfectcorporation.com with SMTP; 9 Jul 2002 08:27:46 -0000 Received: (qmail 60514 invoked by uid 1001); 9 Jul 2002 08:27:53 -0000 Date: Tue, 9 Jul 2002 10:27:53 +0200 From: Neil Blakey-Milner To: freebsd-arch@FreeBSD.ORG Cc: "M. Warner Losh" , naddy@mips.inka.de Subject: Re: Package system flaws? Message-ID: <20020709082753.GA60250@mithrandr.moria.org> References: <20020706220511.GA88651@scoobysnax.jaded.net> <20020708.094652.21114246.imp@village.org> <20020709073309.GB96335@dragon.nuxi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020709073309.GB96335@dragon.nuxi.com> User-Agent: Mutt/1.3.27i Organization: iTouch Labs X-Operating-System: FreeBSD 4.3-RELEASE i386 X-URL: http://mithrandr.moria.org/nbm/ Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue 2002-07-09 (00:33), David O'Brien wrote: > On Mon, Jul 08, 2002 at 09:46:52AM -0600, M. Warner Losh wrote: > > In message: > > naddy@mips.inka.de (Christian Weisgerber) writes: > > : Compared to Debian and RPM the ability to do comprehensive updates > > : from packages is missing. Install 500 ports. Wait a year for > > : revisions, library versions, and even dependencies to change all > > : over. Now let's upgrade all installed ports from a current set of > > : binary packages. That sort of thing. > > > > portupgrade does his very thing. > > Only from /usr/ports, building each port -- it will not use precompiled > packages. :-( Yeah it will - using -P, and only using packages using -PP. (Not saying it's perfect, or that it'll work in the given example, especially with regards to upgrading libraries without upgrading all packages that depend on it that may not be in the "current set of binary packages.) Thus, going back to Warner's "portupgrade does this very thing", I'm afraid it doesn't. Christian's example would require the ability to use version ranges (I've previously used the term "relative versioning") to express dependencies. And the dpkg (and rpm tools to a lesser degree) are based a bit more on dependency graph theory, and thus can tell you whether the path you're going to travel upgrading a certain package and all packages that depend on that, and the packages that depend on that and so forth, will actually end up successfully fulfilling all the requirements, and what, if any, the problem areas are. Of course, that totally sucks in some cases (cf. rpm dependency hell). And apt uses package lists over the Internet to achieve lots of its dependency solving ability, but it works for the most part with package CDs without Internet connectivity. A lot better than FreeBSD held up when I was trying 4.3->4.4 using only the packages on the CD, but that may have been a problem with my use of portupgrade. And portupgrade has grown up a lot in the mean-time. Neil -- Neil Blakey-Milner nbm@mithrandr.moria.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 1:30: 8 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0E1C837B400 for ; Tue, 9 Jul 2002 01:30:04 -0700 (PDT) Received: from mx02.egartech.com (aloha.egartech.com [62.118.81.133]) by mx1.FreeBSD.org (Postfix) with SMTP id 3BCC243E58 for ; Tue, 9 Jul 2002 01:30:02 -0700 (PDT) (envelope-from temik@egartech.com) Received: (qmail 677 invoked by uid 85); 9 Jul 2002 08:29:57 -0000 Received: from temik@egartech.com by mx02.egartech.com with qmail-scanner-1.03 (. Clean. Processed in 0.334595 secs); 09 Jul 2002 08:29:57 -0000 Received: from unknown (HELO turtle.egar.egartech.com) (192.168.8.4) by 0 with SMTP; 9 Jul 2002 08:29:56 -0000 Subject: RE: Package system flaws? MIME-Version: 1.0 Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: quoted-printable Date: Tue, 9 Jul 2002 12:29:58 +0400 content-class: urn:content-classes:message X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Message-ID: <5235EF9BAE6B7F4CB3735789EEF73B296E614C@turtle.egar.egartech.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Package system flaws? Thread-Index: AcInH+n1P8Op5mWeRhCNzo/CJaxAPAAAqG2w From: "Artem Tepponen" To: "Terry Lambert" Cc: , "Doug Barton" , "Garrett Wollman" , , Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: Terry Lambert [mailto:tlambert2@mindspring.com] > Sent: Tuesday, July 09, 2002 12:08 PM > Artem Tepponen wrote: > > Unfortunately this doesn't solve third party packages problem well. > > Note: ports do have freebsd maintainers but packages may not (well, > > there should be possibility for standalone thirdparty packages). >=20 > It doesn't solve the third party binary only kernel components > problem, either. 8-). Hmm, I guess this problem won't be solved by packages system only 8-) Stable binary APIs is not a strong side of open source projects.... Artem Tepponen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 1:59:32 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AC0F837B400 for ; Tue, 9 Jul 2002 01:59:29 -0700 (PDT) Received: from mx02.egartech.com (aloha.egartech.com [62.118.81.133]) by mx1.FreeBSD.org (Postfix) with SMTP id 69F6043E09 for ; Tue, 9 Jul 2002 01:59:28 -0700 (PDT) (envelope-from temik@egartech.com) Received: (qmail 2179 invoked by uid 85); 9 Jul 2002 08:59:23 -0000 Received: from temik@egartech.com by mx02.egartech.com with qmail-scanner-1.03 (. Clean. Processed in 0.377058 secs); 09 Jul 2002 08:59:23 -0000 Received: from unknown (HELO turtle.egar.egartech.com) (192.168.8.4) by 0 with SMTP; 9 Jul 2002 08:59:22 -0000 Subject: RE: Package system flaws? MIME-Version: 1.0 Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: quoted-printable Date: Tue, 9 Jul 2002 12:59:25 +0400 content-class: urn:content-classes:message X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Message-ID: <5235EF9BAE6B7F4CB3735789EEF73B29074172@turtle.egar.egartech.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Package system flaws? Thread-Index: AcImxVGsqyhQhNA3Tm2+znWNY6gkbQAXbWiA From: "Artem Tepponen" To: "Terry Lambert" , "Garrett Wollman" Cc: "Mark Valentine" , Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: Terry Lambert [mailto:tlambert2@mindspring.com] > Sent: Tuesday, July 09, 2002 1:19 AM > Specifically, another of the main problems, besides no "metadata > first" priority, of the tgz format is that the index should be > uncompressed while the contents are compressed. Historically, the > whole archive, metadata and all, was compressed in order to support > seperate uncompress and tar. But it makes just as much sense to > support seperate tar and uncompress -- i.e. compress the individual > files, rather than compressing all files plus the metadata. Most > compression will work better on individual files rather than random > file contents because of dictionary locality, anyway. No, Terry. Dictionary locality works in a different way. gzipped tar will almost always win vs. tarred gzipped files. 10-15% from memory. Just a quick check: -rw-r--r-- 1 temik develops 29020160 Jul 9 12:36 gcc-3.0.1.gz.tar -rw-r--r-- 1 temik develops 13821669 Jul 9 12:41 = gcc-3.0.1.tar.bz2 -rw-r--r-- 1 temik develops 18054324 Sep 24 2001 gcc-3.0.1.tar.gz -rw-r--r-- 1 temik develops 22746511 Jul 9 12:52 gcc-3.0.1.zip Oops. I was wrong. >35% is a big difference. And bzip adds another 24%. But for binaries difference between gzip vs. bzip2 will be smaller. This is quite simple check but the picture will remain the same for pretty any kind of data and hope that's enough to choose single tar.somez + header. Will header be combined or in a different file is another question. Artem Tepponen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 2:15:20 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E6A0137B400; Tue, 9 Jul 2002 02:15:16 -0700 (PDT) Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id 519D443E52; Tue, 9 Jul 2002 02:15:16 -0700 (PDT) (envelope-from des@ofug.org) Received: by flood.ping.uio.no (Postfix, from userid 2602) id 41E10534A; Tue, 9 Jul 2002 11:15:08 +0200 (CEST) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Wes Peters Cc: Doug Barton , Dan Moschuk , arch@FreeBSD.org Subject: Re: Package system flaws? References: <20020707152651.V679-100000@master.gorean.org> <3D2A191C.3103D0FB@softweyr.com> From: Dag-Erling Smorgrav Date: 09 Jul 2002 11:15:08 +0200 In-Reply-To: <3D2A191C.3103D0FB@softweyr.com> Message-ID: Lines: 9 User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.2 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Wes Peters writes: > If you're sensible about the encoding, the metadata doesn't have to be > at the beginning of the media. Yes, it does, to allow installation from non-rewindable streams. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 4:35:56 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A75EB37B400 for ; Tue, 9 Jul 2002 04:35:52 -0700 (PDT) Received: from alix.lpt.ens.fr (bluerondo.lpt.ens.fr [129.199.122.217]) by mx1.FreeBSD.org (Postfix) with SMTP id 5F63D43E52 for ; Tue, 9 Jul 2002 04:35:51 -0700 (PDT) (envelope-from rsidd@lpt.ens.fr) Received: (qmail 322 invoked by uid 1001); 9 Jul 2002 11:35:49 -0000 Date: Tue, 9 Jul 2002 13:35:49 +0200 From: Rahul Siddharthan To: arch@freebsd.org Subject: Cleaning old packages (was: Package system flaws?) Message-ID: <20020709113549.GA249@lpt.ens.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.3.27i X-Operating-System: FreeBSD 4.6-PRERELEASE i386 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I'm not subscribed to this list but I happened on this discussion in the archives today. One mild peeve I've had about FreeBSD's ports system is the difficulty of deleting an old package which has been overwritten by a new package. Suppose you had installed foo-1.3.1, and then another port you were installing overwrote this with foo-1.3.2. Now both are registered in /var/db/pkg but you can't pkg_delete foo-1.3.1 because it will remove files with common names from foo-1.3.2. I've been playing with gentoo linux a bit recently and it has a command to take care of this. The idea seems simple enough: it looks at the mtimes, and makes sure that it is never deleting a file which belongs to the most recently installed version. So what about doing something like this: 1. look at the mtimes of all the /var/db/pkg/foo-* 2. pick the second-most-recent of these (say, if you have foo-1.3.0, 1.3.1 and 1.3.2 installed in that order, take the mtime of foo-1.3.1) 3. Delete all files listed in +CONTENTS for foo-1.3.0 and foo-1.3.1 which are older than this. (Since the package registration is the last step, I assume that all files belonging to foo-1.3.1 will be older than the files in /var/db/pkg/foo-1.3.1?) Would this work, or is there some obvious problem here? It seems to me that this is totally "safe" in that it will never delete a file belonging to the most recent version (unless your clock was wrong when it was installed). It may leave some older files intact if they've been touched in the meantime. There are other features I like about gentoo (I'll list them if anyone is interested) but I think they would require major modifications to the ports system (some of them are supported by portupgrade, though). As it stands, I think the ports system is in some ways more intuitive -- it's easier to do the configure, build etc yourself and supply your own options on the way if you want to. - Rahul To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 6:43:47 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B152037B400; Tue, 9 Jul 2002 06:43:45 -0700 (PDT) Received: from spqr.osg.gov.bc.ca (spqr.osg.gov.bc.ca [142.32.102.24]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4829943E31; Tue, 9 Jul 2002 06:43:45 -0700 (PDT) (envelope-from Cy.Schubert@osg.gov.bc.ca) Received: from passer.osg.gov.bc.ca (passer.osg.gov.bc.ca [142.32.110.29]) by spqr.osg.gov.bc.ca (Postfix) with ESMTP id 1AA699EF18; Tue, 9 Jul 2002 06:43:45 -0700 (PDT) Received: from cwsys.cwsent.com (cwsys2 [10.1.2.1]) by passer.osg.gov.bc.ca (8.12.5/8.12.3) with ESMTP id g69DhPOX041167; Tue, 9 Jul 2002 06:43:25 -0700 (PDT) (envelope-from cy@cwsent.com) Received: from cwsys (localhost [127.0.0.1]) by cwsys.cwsent.com (8.12.5/8.12.3) with ESMTP id g69DhOfP080432; Tue, 9 Jul 2002 06:43:24 -0700 (PDT) (envelope-from cy@cwsys.cwsent.com) Message-Id: <200207091343.g69DhOfP080432@cwsys.cwsent.com> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 Reply-To: Cy Schubert - CITS Open Systems Group From: Cy Schubert - CITS Open Systems Group X-os: FreeBSD X-Sender: cy@cwsent.com To: Dag-Erling Smorgrav Cc: Wes Peters , Doug Barton , Dan Moschuk , arch@FreeBSD.ORG Subject: Re: Package system flaws? In-Reply-To: Message from Dag-Erling Smorgrav of "09 Jul 2002 11:15:08 +0200." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 09 Jul 2002 06:43:24 -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message , Dag-Erling Smorgrav writes: > Wes Peters writes: > > If you're sensible about the encoding, the metadata doesn't have to be > > at the beginning of the media. > > Yes, it does, to allow installation from non-rewindable streams. IMO, this is extremely important. -- Cheers, Phone: 250-387-8437 Cy Schubert Fax: 250-387-5766 Team Leader, Sun/Alpha Team Email: Cy.Schubert@osg.gov.bc.ca Open Systems Group, CITS Ministry of Management Services Province of BC FreeBSD UNIX: cy@FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 7:50:10 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2A71737B401 for ; Tue, 9 Jul 2002 07:50:00 -0700 (PDT) Received: from infinitive.futureperfectcorporation.com (infinitive.futureperfectcorporation.com [196.25.137.68]) by mx1.FreeBSD.org (Postfix) with SMTP id AB4C543E52 for ; Tue, 9 Jul 2002 07:49:53 -0700 (PDT) (envelope-from nbm@gerund.futureperfectcorporation.com) Received: (qmail 84994 invoked by uid 0); 9 Jul 2002 14:49:29 -0000 Received: from gerund.futureperfectcorporation.com (196.25.137.65) by infinitive.futureperfectcorporation.com with SMTP; 9 Jul 2002 14:49:29 -0000 Received: (qmail 62431 invoked by uid 1001); 9 Jul 2002 14:50:14 -0000 Date: Tue, 9 Jul 2002 16:50:14 +0200 From: Neil Blakey-Milner To: Cy Schubert - CITS Open Systems Group Cc: arch@FreeBSD.ORG Subject: Learning from Debian (Was: Re: Package system flaws?) Message-ID: <20020709145013.GA61758@mithrandr.moria.org> References: <200207091343.g69DhOfP080432@cwsys.cwsent.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200207091343.g69DhOfP080432@cwsys.cwsent.com> User-Agent: Mutt/1.3.27i Organization: iTouch Labs X-Operating-System: FreeBSD 4.3-RELEASE i386 X-URL: http://mithrandr.moria.org/nbm/ Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG [ CC's trimmed - I hope nobody minds (they won't get my apology anyway if they do, I suppose) ] On Tue 2002-07-09 (06:43), Cy Schubert - CITS Open Systems Group wrote: > In message , Dag-Erling Smorgrav > writes: > > Wes Peters writes: > > > If you're sensible about the encoding, the metadata doesn't have to be > > > at the beginning of the media. > > > > Yes, it does, to allow installation from non-rewindable streams. > > IMO, this is extremely important. The Debian dpkg format uses 'ar' to put together three files - debian-binary, control.tar.gz, and data.tar.gz. debian-binary is checked so that it is known that it is at least working with a debian package. The file itself contains a version number ('2.0' from a randomly chosen one). control.tar.gz contains a few files. preinst and postinst resemble our +INSTALL files, and the same with prerm and postrm and our +DEINSTALL. conffiles contains the configuration files in the package (to protect them), and control contains most of the metadata: Package: dpkg Version: 1.10.2 Section: base Priority: required Architecture: i386 Essential: yes Pre-Depends: dselect, libc6 (>= 2.2.4-4) Conflicts: sysvinit (<< 2.82-1), dpkg-iasearch, dpkg-static, dpkg-dev (<< 1.9) Replaces: dpkg-doc-ja, dpkg-static, manpages-de (<= 0.4-3) Installed-Size: 3272 Origin: debian Maintainer: Dpkg Development Bugs: debbugs://bugs.debian.org Description: Package maintenance system for Debian data.tar.gz contains a root prefixed tar file with all the files: drwxr-xr-x root/root 0 Jul 5 05:26 2002 ./ drwxr-xr-x root/root 0 Jul 5 05:26 2002 ./usr/ drwxr-xr-x root/root 0 Jul 5 05:26 2002 ./usr/share/ drwxr-xr-x root/root 0 Jul 5 05:26 2002 ./usr/share/doc/ The debian "metadata" file used by apt and similar upgrade programs contains a summary of the "control" file for all the packages contained there, thus including all depends, conflicts, install size, and so forth, with relative versions if necessary. Are there any disadvantages with the way the Debian people have done it? And what can we learn from them? Neil -- Neil Blakey-Milner nbm@mithrandr.moria.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 8:57: 4 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7C7E237B43E for ; Tue, 9 Jul 2002 08:57:00 -0700 (PDT) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by mx1.FreeBSD.org (Postfix) with ESMTP id D345A43E31 for ; Tue, 9 Jul 2002 08:56:59 -0700 (PDT) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.12.5/8.12.5) id g69FnDd1082871; Tue, 9 Jul 2002 10:49:13 -0500 (CDT) (envelope-from dan) Date: Tue, 9 Jul 2002 10:49:13 -0500 From: Dan Nelson To: Rahul Siddharthan Cc: arch@FreeBSD.ORG Subject: Re: Cleaning old packages (was: Package system flaws?) Message-ID: <20020709154912.GA20718@dan.emsphone.com> References: <20020709113549.GA249@lpt.ens.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020709113549.GA249@lpt.ens.fr> X-OS: FreeBSD 5.0-CURRENT X-message-flag: Outlook Error User-Agent: Mutt/1.5.1i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In the last episode (Jul 09), Rahul Siddharthan said: > I'm not subscribed to this list but I happened on this discussion in > the archives today. > > One mild peeve I've had about FreeBSD's ports system is the > difficulty of deleting an old package which has been overwritten by a > new package. Suppose you had installed foo-1.3.1, and then another > port you were installing overwrote this with foo-1.3.2. Now both are > registered in /var/db/pkg but you can't pkg_delete foo-1.3.1 because > it will remove files with common names from foo-1.3.2. If you use portupgrade to install ports and packages, it will deinstall the old port before installing the new one. -- Dan Nelson dnelson@allantgroup.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 9:20: 2 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6ED1F37B400 for ; Tue, 9 Jul 2002 09:19:58 -0700 (PDT) Received: from alix.lpt.ens.fr (bluerondo.lpt.ens.fr [129.199.122.217]) by mx1.FreeBSD.org (Postfix) with SMTP id 308DB43E4A for ; Tue, 9 Jul 2002 09:19:56 -0700 (PDT) (envelope-from rsidd@lpt.ens.fr) Received: (qmail 69866 invoked by uid 1001); 9 Jul 2002 16:19:53 -0000 Date: Tue, 9 Jul 2002 18:19:53 +0200 From: Rahul Siddharthan To: Dan Nelson Cc: arch@FreeBSD.ORG Subject: Re: Cleaning old packages (was: Package system flaws?) Message-ID: <20020709161953.GA69779@lpt.ens.fr> References: <20020709113549.GA249@lpt.ens.fr> <20020709154912.GA20718@dan.emsphone.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020709154912.GA20718@dan.emsphone.com> User-Agent: Mutt/1.3.27i X-Operating-System: FreeBSD 4.6-PRERELEASE i386 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Dan Nelson said on Jul 9, 2002 at 10:49:13: > > new package. Suppose you had installed foo-1.3.1, and then another > > port you were installing overwrote this with foo-1.3.2. Now both are > > registered in /var/db/pkg but you can't pkg_delete foo-1.3.1 because > > it will remove files with common names from foo-1.3.2. > > If you use portupgrade to install ports and packages, it will deinstall > the old port before installing the new one. Since you're the third person telling me this (the first two were off-list) let me clarify -- I know about portupgrade. Yes, if I use only portupgrade and nothing else, I won't have the problem. But using portupgrade all the time is totally unrealistic: (1) it has problems with binary packages, and (2) I don't like rebuilding every dependency which has had the most minor of changes every time I install a new port. Besides, portupgrade is not part of the base system, it's an add-on, so in my opinion saying "use portupgrade" is not an answer. If you use the base system tools you *will* run into the problem of duplicate installed packages, rather quickly. I was initially impressed with portupgrade, but after using it a while I realized it solves problems which I don't have and not the ones I have. Mainly, I want dependency tracking (which the ports system supplies already) but I don't want to upgrade old packages automatically unless it's essential or I specifically ask for it. As others have pointed out, what's needed is using version ranges to express dependencies. Portupgrade (pkgdb -F) also has a "fix" for duplicate packages but it doesn't actually remove any redundant files -- it tacks all the +CONTENTS to the most recently installed package and removes the entries of the rest. I think one can do better. Gentoo Linux's (http://www.gentoo.org) "portage" system has a lot of nice ideas worth stealing, in my opinion (in particular it supports version ranges, but other things too). At some point during the next month I was thinking of implementing some gentoo-like features on freebsd for my own use (but since I'm not an expert C programmer I'll probably end up using python or something like that.) - Rahul To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 9:40: 5 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 79A6537B401 for ; Tue, 9 Jul 2002 09:40:01 -0700 (PDT) Received: from felix.automagic.org (felix.automagic.org [204.152.186.101]) by mx1.FreeBSD.org (Postfix) with SMTP id 98C0743E31 for ; Tue, 9 Jul 2002 09:40:00 -0700 (PDT) (envelope-from jabley@automagic.org) Received: (qmail 10625 invoked by uid 0); 9 Jul 2002 16:40:00 -0000 Received: from localhost.automagic.org (HELO hyperion) (127.0.0.1) by localhost.automagic.org with SMTP; 9 Jul 2002 16:40:00 -0000 Date: Tue, 9 Jul 2002 12:40:02 -0400 Subject: Re: Package system flaws? Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v482) Cc: Wes Peters , Dag-Erling Smorgrav , Doug Barton , Dan Moschuk , arch@FreeBSD.org To: Terry Lambert From: Joe Abley In-Reply-To: <3D2A45BC.75556D69@mindspring.com> Message-Id: <83FD8E1F-935A-11D6-B320-00039312C852@automagic.org> Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.482) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Monday, July 8, 2002, at 10:09 , Terry Lambert wrote: > Wes Peters wrote: >> The more you write, the more I see XML as being terribly appropriate >> for the package metadata. Consider the nesting problem; XML handles >> this with complete grace if you write it into the DTD that a package >> can contain a package. > > XML would be good, if you didn't have to have 90M of crap in order > to parse it. felix# ls -al /usr/local/lib/libxml.a -rw-r--r-- 1 root wheel 399096 Jul 9 09:39 /usr/local/lib/libxml.a felix# To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 9:47:37 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 13DF837B400 for ; Tue, 9 Jul 2002 09:47:35 -0700 (PDT) Received: from mail.speakeasy.net (mail14.speakeasy.net [216.254.0.214]) by mx1.FreeBSD.org (Postfix) with ESMTP id 60E3F43E4A for ; Tue, 9 Jul 2002 09:47:34 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Received: (qmail 17553 invoked from network); 9 Jul 2002 16:47:30 -0000 Received: from unknown (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) by mail14.speakeasy.net (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 9 Jul 2002 16:47:30 -0000 Received: from laptop.baldwin.cx (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.11.6/8.11.6) with ESMTP id g69GlEf09402; Tue, 9 Jul 2002 12:47:15 -0400 (EDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.5.2 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20020709161953.GA69779@lpt.ens.fr> Date: Tue, 09 Jul 2002 12:47:17 -0400 (EDT) From: John Baldwin To: Rahul Siddharthan Subject: Re: Cleaning old packages (was: Package system flaws?) Cc: arch@FreeBSD.ORG, Dan Nelson Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 09-Jul-2002 Rahul Siddharthan wrote: > Dan Nelson said on Jul 9, 2002 at 10:49:13: >> > new package. Suppose you had installed foo-1.3.1, and then another >> > port you were installing overwrote this with foo-1.3.2. Now both are >> > registered in /var/db/pkg but you can't pkg_delete foo-1.3.1 because >> > it will remove files with common names from foo-1.3.2. >> >> If you use portupgrade to install ports and packages, it will deinstall >> the old port before installing the new one. > > Since you're the third person telling me this (the first two were > off-list) let me clarify -- I know about portupgrade. Yes, if I use > only portupgrade and nothing else, I won't have the problem. But > using portupgrade all the time is totally unrealistic: (1) it has > problems with binary packages, and (2) I don't like rebuilding every > dependency which has had the most minor of changes every time I > install a new port. Besides, portupgrade is not part of the base > system, it's an add-on, so in my opinion saying "use portupgrade" is > not an answer. If you use the base system tools you *will* run into > the problem of duplicate installed packages, rather quickly. > > I was initially impressed with portupgrade, but after using it a while > I realized it solves problems which I don't have and not the ones I > have. Mainly, I want dependency tracking (which the ports system > supplies already) but I don't want to upgrade old packages > automatically unless it's essential or I specifically ask for it. As > others have pointed out, what's needed is using version ranges to > express dependencies. Or rather, to depend on abstract functions provided by other packages. I.e., package A might depend on the 'foo' functionality which is provided by package B. Thus, package A is not tied to B's specific version. If you have a new API then you can append a version to 'foo'. For example, you might have 'qt1', 'qt2', 'gtk12', etc. "functions". RPM does this and I think that the new /etc/rc.d stuff in current uses a similar scheme. RPM also supports the notion of a package having a function that conflicts with other functions. The idea though is that dependencies aren't strictly on a package, but on functoinality provided by that package. This does mean more work when trying to figure out how to fulfill a needed dependency, however. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 9:57:29 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 24A1A37B400 for ; Tue, 9 Jul 2002 09:57:25 -0700 (PDT) Received: from obsecurity.dyndns.org (adsl-63-207-60-128.dsl.lsan03.pacbell.net [63.207.60.128]) by mx1.FreeBSD.org (Postfix) with ESMTP id 864BB43E09 for ; Tue, 9 Jul 2002 09:57:24 -0700 (PDT) (envelope-from kris@obsecurity.org) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id DAD5366E48; Tue, 9 Jul 2002 09:57:23 -0700 (PDT) Date: Tue, 9 Jul 2002 09:57:23 -0700 From: Kris Kennaway To: Sam Leffler Cc: freebsd-arch@freebsd.org Subject: Re: status of hardware crypto support Message-ID: <20020709165723.GA84297@xor.obsecurity.org> References: <05c801c222d2$ad797550$52557f42@errno.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="opJtzjQTFsWo+cga" Content-Disposition: inline In-Reply-To: <05c801c222d2$ad797550$52557f42@errno.com> User-Agent: Mutt/1.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --opJtzjQTFsWo+cga Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Wed, Jul 03, 2002 at 01:46:16PM -0700, Sam Leffler wrote: > This is a short note about the status of my work to port openbsd's support > for hardware crypto devices to freebsd. [...] This sounds great, thanks for doing it! Kris --opJtzjQTFsWo+cga Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.7 (FreeBSD) iD8DBQE9KxXyWry0BWjoQKURAvruAJ4vdbKSS6oPIbzSxiiYwrxRVYBguQCgnLd+ 4KwfN+hs6q+R6r+5ivDEjBs= =TfWm -----END PGP SIGNATURE----- --opJtzjQTFsWo+cga-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 10:14:24 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DE39C37B406 for ; Tue, 9 Jul 2002 10:14:20 -0700 (PDT) Received: from alix.lpt.ens.fr (bluerondo.lpt.ens.fr [129.199.122.217]) by mx1.FreeBSD.org (Postfix) with SMTP id AF58143E67 for ; Tue, 9 Jul 2002 10:14:19 -0700 (PDT) (envelope-from rsidd@lpt.ens.fr) Received: (qmail 70306 invoked by uid 1001); 9 Jul 2002 17:14:17 -0000 Date: Tue, 9 Jul 2002 19:14:17 +0200 From: Rahul Siddharthan To: John Baldwin Cc: arch@FreeBSD.ORG, Dan Nelson Subject: Re: Cleaning old packages (was: Package system flaws?) Message-ID: <20020709171417.GA69932@lpt.ens.fr> References: <20020709161953.GA69779@lpt.ens.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.3.27i X-Operating-System: FreeBSD 4.6-PRERELEASE i386 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG John Baldwin said on Jul 9, 2002 at 12:47:17: > > Or rather, to depend on abstract functions provided by other packages. > I.e., package A might depend on the 'foo' functionality which is provided > by package B. Thus, package A is not tied to B's specific version. If > you have a new API then you can append a version to 'foo'. For example, > you might have 'qt1', 'qt2', 'gtk12', etc. "functions". RPM does this > and I think that the new /etc/rc.d stuff in current uses a similar scheme. > RPM also supports the notion of a package having a function that conflicts > with other functions. That seems rather ambitious, and too drastic a change, to me. What I'd like to see is probably more like, the gfoo port needs gtk+ 1.2.6 or above, but not gtk+ 2.0 and above (incompatible) and not gtk+ 1.2.5 or below (buggy). There should be some way to specify this in the makefile of the port, so that any port-management program like portupgrade can make use of the information. Right now, dependencies are typically specified by files (binaries or shared libraries) which are searched for before proceeding. This is better than a "hard" version number requirement and works a lot of the time, but not always. I'll have a stab at writing something in some scripting language. I'm planning to try and implement some of the following features, for a start, which I think can be done without any modification to the existing ports tree/system: (1) Upgrade dependencies only if essential, and then too only after user confirmation, perhaps listing the possible affected packages (to avoid things like the libpng chaos some time ago). Perhaps allow "pinning" to a certain version number, so that it will never be upgraded unless you manually "un-pin" it. (2) When upgrading dependencies, provide a way to safely remove old versions of duplicate packages, as I described in my earlier mail. (3) Automatically generate the +CONTENTS file by first doing a "fake" install in a temporary directory (assuming the port honours $PREFIX), then moving the contents to their final destination (a la gentoo). If your temporary location is on the same filesystem as the final one, it won't even take additional disk space, and very little additional time. Is there any obvious drawback with doing this? If anyone is interested, the gentoo portage system is described here: http://www.gentoo.org/doc/portage-manual.html - Rahul To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 11:50:21 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 80B0437B400 for ; Tue, 9 Jul 2002 11:50:18 -0700 (PDT) Received: from dragon.nuxi.com (trang.nuxi.com [66.92.13.169]) by mx1.FreeBSD.org (Postfix) with ESMTP id A4B6543E4A for ; Tue, 9 Jul 2002 11:50:17 -0700 (PDT) (envelope-from obrien@NUXI.com) Received: from dragon.nuxi.com (obrien@localhost [127.0.0.1]) by dragon.nuxi.com (8.12.5/8.12.2) with ESMTP id g69IoHoi027649; Tue, 9 Jul 2002 11:50:17 -0700 (PDT) (envelope-from obrien@dragon.nuxi.com) Received: (from obrien@localhost) by dragon.nuxi.com (8.12.5/8.12.5/Submit) id g69IoAmZ027648; Tue, 9 Jul 2002 11:50:10 -0700 (PDT) Date: Tue, 9 Jul 2002 11:50:10 -0700 From: "David O'Brien" To: Neil Blakey-Milner Cc: freebsd-arch@FreeBSD.ORG, "M. Warner Losh" , naddy@mips.inka.de Subject: Re: Package system flaws? Message-ID: <20020709185010.GA26983@dragon.nuxi.com> Reply-To: freebsd-arch@FreeBSD.ORG References: <20020706220511.GA88651@scoobysnax.jaded.net> <20020708.094652.21114246.imp@village.org> <20020709073309.GB96335@dragon.nuxi.com> <20020709082753.GA60250@mithrandr.moria.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020709082753.GA60250@mithrandr.moria.org> User-Agent: Mutt/1.4i X-Operating-System: FreeBSD 5.0-CURRENT Organization: The NUXI BSD Group X-Pgp-Rsa-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Rsa-Keyid: 1024/34F9F9D5 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, Jul 09, 2002 at 10:27:53AM +0200, Neil Blakey-Milner wrote: > > > portupgrade does his very thing. > > > > Only from /usr/ports, building each port -- it will not use precompiled > > packages. :-( > > Yeah it will - using -P, and only using packages using -PP. (Not saying > it's perfect, or that it'll work in the given example, especially with > regards to upgrading libraries without upgrading all packages that > depend on it that may not be in the "current set of binary packages.) I should have said more -- it requires the packages to be local. This means if I want to update something I practically have to download all ?5? GB of packages from ftp.freebsd.org so that what ever package portupgrade might want will be available to it. Portupgrade really needs to grow the functionality of `pkg_add -r'. Someone demoed the Debian package system to me at USENIX and it appears it does have this functionality. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 11:53: 5 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8926737B400 for ; Tue, 9 Jul 2002 11:53:03 -0700 (PDT) Received: from squall.waterspout.com (squall.waterspout.com [208.13.56.12]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3D06943E09 for ; Tue, 9 Jul 2002 11:53:03 -0700 (PDT) (envelope-from will@csociety.org) Received: by squall.waterspout.com (Postfix, from userid 1050) id AF6EE9B34; Tue, 9 Jul 2002 13:53:02 -0500 (EST) Date: Tue, 9 Jul 2002 13:53:02 -0500 From: Will Andrews To: freebsd-arch@FreeBSD.ORG Subject: Re: Package system flaws? Message-ID: <20020709185302.GL56853@squall.waterspout.com> Mail-Followup-To: freebsd-arch@FreeBSD.ORG References: <20020706220511.GA88651@scoobysnax.jaded.net> <20020708.094652.21114246.imp@village.org> <20020709073309.GB96335@dragon.nuxi.com> <20020709082753.GA60250@mithrandr.moria.org> <20020709185010.GA26983@dragon.nuxi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020709185010.GA26983@dragon.nuxi.com> User-Agent: Mutt/1.3.26i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, Jul 09, 2002 at 11:50:10AM -0700, David O'Brien wrote: > I should have said more -- it requires the packages to be local. This > means if I want to update something I practically have to download all ?5? > GB of packages from ftp.freebsd.org so that what ever package portupgrade > might want will be available to it. Portupgrade really needs to grow the > functionality of `pkg_add -r'. Umm.. it does support it. It even does a better job than pkg_add on this front, in that it's configurable. Regards, -- wca To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 13:10:20 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1E11337B400; Tue, 9 Jul 2002 13:10:13 -0700 (PDT) Received: from mta07-svc.ntlworld.com (mta07-svc.ntlworld.com [62.253.162.47]) by mx1.FreeBSD.org (Postfix) with ESMTP id 16A0B43E31; Tue, 9 Jul 2002 13:10:12 -0700 (PDT) (envelope-from simonm@sm.dnsalias.com) Received: from sm.dnsalias.com ([213.107.104.1]) by mta07-svc.ntlworld.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020709201010.FJUR19225.mta07-svc.ntlworld.com@sm.dnsalias.com>; Tue, 9 Jul 2002 21:10:10 +0100 Received: from sm.dnsalias.com (localhost [127.0.0.1]) by sm.dnsalias.com (8.12.4/8.12.4) with ESMTP id g69KA9pU013609; Tue, 9 Jul 2002 21:10:09 +0100 (BST) (envelope-from simonm@sm.dnsalias.com) Received: (from simonm@localhost) by sm.dnsalias.com (8.12.4/8.12.4/Submit) id g69KA7Ul013606; Tue, 9 Jul 2002 21:10:07 +0100 (BST) To: Rahul Siddharthan Cc: John Baldwin , arch@freebsd.org, Dan Nelson Subject: Re: Cleaning old packages (was: Package system flaws?) References: <20020709161953.GA69779@lpt.ens.fr> <20020709171417.GA69932@lpt.ens.fr> From: Simon Marlow Date: 09 Jul 2002 21:10:07 +0100 In-Reply-To: <20020709171417.GA69932@lpt.ens.fr> Message-ID: Lines: 44 User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.1 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Rahul Siddharthan writes: > That seems rather ambitious, and too drastic a change, to me. What > I'd like to see is probably more like, the gfoo port needs gtk+ 1.2.6 > or above, but not gtk+ 2.0 and above (incompatible) and not gtk+ 1.2.5 > or below (buggy). There should be some way to specify this in the > makefile of the port, so that any port-management program like > portupgrade can make use of the information. A port can't know which versions of a library it will be compatible with ahead of time. Only the port itself knows which older versions of itself are compatible with the current version, so the information about whether an upgrade is safe or not should reside in the port which is being upgraded. This why the suggestion of using capability names or APIs to specify the functionality is a better way, but I think it's perhaps a level of abstraction too far. I've always been annoyed by the way the ports system spams older versions of ports with newer ones. It isn't safe to do in general, so IMO the ports system should at the least just refuse to continue if it finds it needs to do this. Then if the specification for a port had some way to say that it was safe to upgrade certain versions to the new version, it could go ahead (deleting the old one first, of course). So here's my suggestions: - Allow a port to specify a range of versions which are safe to upgrade to the current version without user intervention. - The ports system should *not* automatically install a new version of a port over an old one. If the port says it is safe to do so, then remove the old port and install the new one. Allowing dependencies to specify version ranges would help reduce the number of upgrades required, but wouldn't otherwise improve matters. Also it requires updates to N ports rather than 1 for each dependency, so is far more error prone. (disclaimer: I'm lazy and haven't looked at libh, so I apologise if this is old news) Cheers, Simon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 14:10:23 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 775C937B400 for ; Tue, 9 Jul 2002 14:10:21 -0700 (PDT) Received: from postfix2-1.free.fr (postfix2-1.free.fr [213.228.0.9]) by mx1.FreeBSD.org (Postfix) with ESMTP id EAF8243E64 for ; Tue, 9 Jul 2002 14:10:20 -0700 (PDT) (envelope-from rsidd@lpt.ens.fr) Received: from bluerondo.a.la.turk (nas-cbv-6-62-147-150-98.dial.proxad.net [62.147.150.98]) by postfix2-1.free.fr (Postfix) with ESMTP id 53FA03CA for ; Tue, 9 Jul 2002 23:10:19 +0200 (CEST) Received: (qmail 280 invoked by uid 1001); 9 Jul 2002 21:09:12 -0000 Date: Tue, 9 Jul 2002 23:09:12 +0200 From: Rahul Siddharthan To: Simon Marlow Cc: John Baldwin , arch@freebsd.org, Dan Nelson Subject: Re: Cleaning old packages (was: Package system flaws?) Message-ID: <20020709210911.GA248@lpt.ens.fr> References: <20020709161953.GA69779@lpt.ens.fr> <20020709171417.GA69932@lpt.ens.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.3.27i X-Operating-System: FreeBSD 4.6-PRERELEASE i386 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Simon Marlow said on Jul 9, 2002 at 21:10:07: > Rahul Siddharthan writes: > > > That seems rather ambitious, and too drastic a change, to me. What > > I'd like to see is probably more like, the gfoo port needs gtk+ 1.2.6 > > or above, but not gtk+ 2.0 and above (incompatible) and not gtk+ 1.2.5 > > or below (buggy). There should be some way to specify this in the > > makefile of the port, so that any port-management program like > > portupgrade can make use of the information. > > A port can't know which versions of a library it will be compatible > with ahead of time. Only the port itself knows which older versions > of itself are compatible with the current version, so the information > about whether an upgrade is safe or not should reside in the port > which is being upgraded. OK, these are two different problems I think. My problem is, is it *necessary* to upgrade the gtk+ port (from the point of view of gfoo); and your problem is, is it *safe* to do so. In principle, any of the four possibilities could exist; to take libpng as an example: neither safe nor necessary (this was true of a very large number of ports which depended on libpng, during the 1.0 -> 1.2 transition) safe but not necessary (usually, upgrading through minor versions, like libpng 1.2.2 -> 1.2.3) necessary but not safe (a port which depends on libpng 1.2.x and won't work with libpng 1.0.x) both safe and necessary (a bugfix in the earlier version which doesn't affect compatibility otherwise) So I think both the port, and the dependency, should carry some sort of versioning information. The port should say which versions of the dependency it's compatible with, and the dependency should say which earlier versions are safe to upgrade from. - Rahul To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 15:18:19 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5BCC137B400; Tue, 9 Jul 2002 15:18:15 -0700 (PDT) Received: from snipe.mail.pas.earthlink.net (snipe.mail.pas.earthlink.net [207.217.120.62]) by mx1.FreeBSD.org (Postfix) with ESMTP id 82C0A43E31; Tue, 9 Jul 2002 15:18:14 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0275.cvx22-bradley.dialup.earthlink.net ([209.179.199.20] helo=mindspring.com) by snipe.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17S3J1-0004nH-00; Tue, 09 Jul 2002 18:18:00 -0400 Message-ID: <3D2B60EB.3CAB559B@mindspring.com> Date: Tue, 09 Jul 2002 15:17:15 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Artem Tepponen Cc: rsi@panix.com, Doug Barton , Garrett Wollman , mark@thuvia.demon.co.uk, arch@FreeBSD.org Subject: Re: Package system flaws? References: <5235EF9BAE6B7F4CB3735789EEF73B296E614C@turtle.egar.egartech.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Artem Tepponen wrote: > > > Unfortunately this doesn't solve third party packages problem well. > > > Note: ports do have freebsd maintainers but packages may not (well, > > > there should be possibility for standalone thirdparty packages). > > > > It doesn't solve the third party binary only kernel components > > problem, either. 8-). > > Hmm, I guess this problem won't be solved by packages system only 8-) > Stable binary APIs is not a strong side of open source projects.... That's a problem, too. But the problem I was thinking of is the one of dependencies on a binary object for which there is no source. It's precisely analogous to the precompiled package issue, particularly if the package you are wanting to install is source with binary components for the kernel, and you want to subsequently statically link it into the kernel. The Smiley is because it's not a problem that I think is intended by FreeBSD to be solved with a package system ... though it could be. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 15:30:40 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7EF1237B400; Tue, 9 Jul 2002 15:30:36 -0700 (PDT) Received: from mail3.panix.com (mail3.panix.com [166.84.1.74]) by mx1.FreeBSD.org (Postfix) with ESMTP id E193B43E3B; Tue, 9 Jul 2002 15:30:35 -0700 (PDT) (envelope-from rsi@panix.com) Received: from panix1.panix.com (panix1.panix.com [166.84.1.1]) by mail3.panix.com (Postfix) with ESMTP id 25E2F981D5; Tue, 9 Jul 2002 18:30:35 -0400 (EDT) Received: (from rsi@localhost) by panix1.panix.com (8.11.3nb1/8.8.8/PanixN1.0) id g69MUZu18917; Tue, 9 Jul 2002 18:30:35 -0400 (EDT) Message-Id: <200207092230.g69MUZu18917@panix1.panix.com> X-Authentication-Warning: panix1.panix.com: rsi set sender to rsi@panix.com using -f To: "Artem Tepponen" Cc: "Doug Barton" , "Garrett Wollman" , , Subject: Re: Package system flaws? References: <5235EF9BAE6B7F4CB3735789EEF73B296E614A@turtle.egar.egartech.com> From: Rajappa Iyer Date: 09 Jul 2002 18:30:35 -0400 Reply-To: rsi@panix.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG "Artem Tepponen" writes: > > From: Rajappa Iyer [mailto:rsi@panix.com] > > It seems to me that the solution is staring us right in the face. > > Currently, the ports mechanism already separates data and metadata. > > IMHO, the best solution is to extend the ports mechanism so that it > > can be told to fetch and install binaries instead of building it > > locally. IOW, perhaps one additional file per port and some > > additional rules in bsd.port.mk. > > > > This seems to me to be the cleanest solution. > Unfortunately this doesn't solve third party packages problem well. > Note: ports do have freebsd maintainers but packages may not (well, > there should be possibility for standalone thirdparty packages). Well, recall that we do have ports that install third-party binaries (e.g. netscape). I agree that someone has to create a port entry for the package, but then someone has to create the package too. The only problem with the ports approach is that you have to have the ports tree locally installed. But this is not materially different from, say, Debian's apt approach which maintains a local copy of the database of all available packages from a particular source. I'm going to think about this concept a little more and see if I can't come up with something. rsi -- a.k.a. Rajappa Iyer. They also surf who stand in the waves. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 15:33:18 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B260B37B401; Tue, 9 Jul 2002 15:33:16 -0700 (PDT) Received: from mail3.panix.com (mail3.panix.com [166.84.1.74]) by mx1.FreeBSD.org (Postfix) with ESMTP id 211E543E64; Tue, 9 Jul 2002 15:33:16 -0700 (PDT) (envelope-from rsi@panix.com) Received: from panix1.panix.com (panix1.panix.com [166.84.1.1]) by mail3.panix.com (Postfix) with ESMTP id 8C5A9982EE; Tue, 9 Jul 2002 18:33:15 -0400 (EDT) Received: (from rsi@localhost) by panix1.panix.com (8.11.3nb1/8.8.8/PanixN1.0) id g69MXFp19435; Tue, 9 Jul 2002 18:33:15 -0400 (EDT) Message-Id: <200207092233.g69MXFp19435@panix1.panix.com> X-Authentication-Warning: panix1.panix.com: rsi set sender to rsi@panix.com using -f To: "Artem Tepponen" Cc: "Doug Barton" , "Garrett Wollman" , , Subject: Re: Package system flaws? References: <5235EF9BAE6B7F4CB3735789EEF73B296E614A@turtle.egar.egartech.com> <200207092230.g69MUZu18917@panix1.panix.com> From: Rajappa Iyer Date: 09 Jul 2002 18:33:15 -0400 Reply-To: rsi@panix.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Rajappa Iyer writes: > Well, recall that we do have ports that install third-party binaries > (e.g. netscape). I agree that someone has to create a port entry for > the package, but then someone has to create the package too. Whoops, what I meant to write was: someone has to create the metadata for the package anyway. -- a.k.a. Rajappa Iyer. They also surf who stand in the waves. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 15:38:20 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DE90C37B400 for ; Tue, 9 Jul 2002 15:38:17 -0700 (PDT) Received: from hawk.mail.pas.earthlink.net (hawk.mail.pas.earthlink.net [207.217.120.22]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0C74D43E4A for ; Tue, 9 Jul 2002 15:38:17 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0275.cvx22-bradley.dialup.earthlink.net ([209.179.199.20] helo=mindspring.com) by hawk.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17S3cX-0001Wv-00; Tue, 09 Jul 2002 18:38:10 -0400 Message-ID: <3D2B65A3.ABB92114@mindspring.com> Date: Tue, 09 Jul 2002 15:37:23 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Artem Tepponen Cc: Garrett Wollman , Mark Valentine , arch@freebsd.org Subject: Re: Package system flaws? References: <5235EF9BAE6B7F4CB3735789EEF73B29074172@turtle.egar.egartech.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Artem Tepponen wrote: > No, Terry. Dictionary locality works in a different way. > gzipped tar will almost always win vs. tarred gzipped files. > 10-15% from memory. Just a quick check: > > -rw-r--r-- 1 temik develops 29020160 Jul 9 12:36 gcc-3.0.1.gz.tar > -rw-r--r-- 1 temik develops 13821669 Jul 9 12:41 gcc-3.0.1.tar.bz2 > -rw-r--r-- 1 temik develops 18054324 Sep 24 2001 gcc-3.0.1.tar.gz > -rw-r--r-- 1 temik develops 22746511 Jul 9 12:52 gcc-3.0.1.zip > > Oops. I was wrong. >35% is a big difference. And bzip adds another 24%. > But for binaries difference between gzip vs. bzip2 will be smaller. > > This is quite simple check but the picture will remain the same > for pretty any kind of data and hope that's enough to choose > single tar.somez + header. > > Will header be combined or in a different file is another question. 1) "Most compression", not "all compression". 2) LZW resets the dictionary every 12K. This is the patented process that Terry Welch of Unisys introduced. So your argument is only valid for a lot of small files who size is well under 12K, which have similar contents. 3) I believe gzip and bzip were both written to get out from under the Unisys patent, and therefore do not compress as well as they could compress, even though Unisys has granted blanket royalty free use for certain applications which fall into this category. 4) Nothing in my statement precludes maintaining the dictionary as a spanning set over a number of small files, per #2, while at the same time leaving the index uncompressed. 5) Yes, I would expect that an uncompressed index would take more room than a compressed index. 6) For most modern communications media, (including broad-band where a modulator/demodulator pair is used... e.g. cable modem) the modems involved include their own compression; usually a form of trellis encoding. As a side note: compression of compressed data is useless, and usually, in fact, counter-productive. All of the format arguments I've been making are predicated on non-CDROM distribution over some medium which is two orders of magnitude or more slower than a local CDROM... and which, by their very nature of having hardware compression, tend to not benefit at all from compression anyway. But even if your argument were totally valid, then the compression you seek is going to come from the link level compression on top of the data being transferred anyway. NB: The Unisys patent expires on Dec 10th of this year, in any case, so the only reason bzip/gzip wouldn't support using it after that is religious. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 15:41:13 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6C82E37B400; Tue, 9 Jul 2002 15:41:11 -0700 (PDT) Received: from hawk.mail.pas.earthlink.net (hawk.mail.pas.earthlink.net [207.217.120.22]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1A31643E3B; Tue, 9 Jul 2002 15:41:11 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0275.cvx22-bradley.dialup.earthlink.net ([209.179.199.20] helo=mindspring.com) by hawk.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17S3fE-0005ha-00; Tue, 09 Jul 2002 18:40:57 -0400 Message-ID: <3D2B664B.64079A1F@mindspring.com> Date: Tue, 09 Jul 2002 15:40:11 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Dag-Erling Smorgrav Cc: Wes Peters , Doug Barton , Dan Moschuk , arch@FreeBSD.org Subject: Re: Package system flaws? References: <20020707152651.V679-100000@master.gorean.org> <3D2A191C.3103D0FB@softweyr.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Dag-Erling Smorgrav wrote: > Wes Peters writes: > > If you're sensible about the encoding, the metadata doesn't have to be > > at the beginning of the media. > > Yes, it does, to allow installation from non-rewindable streams. And to allow partial downloading of the package via HTTP GET range requests, in order to obtain the metadata for e.g. "KDE", without having to download the entire 18TB it would take up over a 28K modem line. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 15:55:25 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5825C37B400; Tue, 9 Jul 2002 15:55:23 -0700 (PDT) Received: from thuvia.demon.co.uk (thuvia.demon.co.uk [193.237.34.248]) by mx1.FreeBSD.org (Postfix) with ESMTP id 662B143E09; Tue, 9 Jul 2002 15:55:21 -0700 (PDT) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (dotar.thuvia.org [10.0.0.4]) by phaidor.thuvia.org (8.11.6/8.11.6) with ESMTP id g69Mt4b19446; Tue, 9 Jul 2002 23:55:05 +0100 (BST) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (localhost [IPv6:::1]) by dotar.thuvia.org (8.12.3/8.12.3) with ESMTP id g69Mt4KC011432; Tue, 9 Jul 2002 23:55:04 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.3/8.12.3/Submit) id g69Mt3n8011431; Tue, 9 Jul 2002 23:55:03 +0100 (BST) Date: Tue, 9 Jul 2002 23:55:03 +0100 (BST) From: Mark Valentine Message-Id: <200207092255.g69Mt3n8011431@dotar.thuvia.org> In-Reply-To: Wes Peters's message of Jul 8, 3:41pm X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: Wes Peters Subject: Re: Package system flaws? Cc: Doug Barton , Garrett Wollman , arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: Wes Peters > Date: Mon 8 Jul, 2002 > Subject: Re: Package system flaws? > Mark Valentine wrote: > > Example 1: simple package, not sub-packaged. > > > > $ ls /var/spool/pkg/foo-x.y > > base.bz2 package.xml > > Ick. Why not have the XML include the base.bz2 file in whatever encoding > (including direct binary) we deem appropriate? That requires special tools to extract base.bz2 conveniently. > If you want truly > minimal package sizes, specify the blob(s) as external URLs rather than > encoding them. That's effectively identical to the behaviour implicit in my proposal. > Come to think if it, it would be a simple transformation to "convert" a > package from external references to a full binary, with something like > a pkg_fetch command. It would read a package with external URLs for > the filesets, fetch them, and re-write the package with the blobs > encased. tar(1) is a simpler conversion. ;-) Cheers, Mark. -- Mark Valentine, Thuvia Labs "Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses "We're kind of stupid that way." *munch* *munch* and endorses FreeBSD -- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 16:17:38 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EEC8137B400; Tue, 9 Jul 2002 16:17:36 -0700 (PDT) Received: from falcon.mail.pas.earthlink.net (falcon.mail.pas.earthlink.net [207.217.120.74]) by mx1.FreeBSD.org (Postfix) with ESMTP id 89A7543E09; Tue, 9 Jul 2002 16:17:36 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0275.cvx22-bradley.dialup.earthlink.net ([209.179.199.20] helo=mindspring.com) by falcon.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17S4EU-0000yF-00; Tue, 09 Jul 2002 16:17:23 -0700 Message-ID: <3D2B6ED6.66FAEF5B@mindspring.com> Date: Tue, 09 Jul 2002 16:16:38 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Joe Abley Cc: Wes Peters , Dag-Erling Smorgrav , Doug Barton , Dan Moschuk , arch@FreeBSD.org Subject: Re: Package system flaws? References: <83FD8E1F-935A-11D6-B320-00039312C852@automagic.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Joe Abley wrote: > On Monday, July 8, 2002, at 10:09 , Terry Lambert wrote: > > Wes Peters wrote: > >> The more you write, the more I see XML as being terribly appropriate > >> for the package metadata. Consider the nesting problem; XML handles > >> this with complete grace if you write it into the DTD that a package > >> can contain a package. > > > > XML would be good, if you didn't have to have 90M of crap in order > > to parse it. > > felix# ls -al /usr/local/lib/libxml.a > -rw-r--r-- 1 root wheel 399096 Jul 9 09:39 /usr/local/lib/libxml.a > felix# There's 1/3 of a floppy disc gone... the cut down "vi" is what, 27K? -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 16:28: 0 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 74DA537B400; Tue, 9 Jul 2002 16:27:57 -0700 (PDT) Received: from falcon.mail.pas.earthlink.net (falcon.mail.pas.earthlink.net [207.217.120.74]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0B9DC43E09; Tue, 9 Jul 2002 16:27:57 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0275.cvx22-bradley.dialup.earthlink.net ([209.179.199.20] helo=mindspring.com) by falcon.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17S4OX-0007OA-00; Tue, 09 Jul 2002 16:27:46 -0700 Message-ID: <3D2B7144.AA93AAB1@mindspring.com> Date: Tue, 09 Jul 2002 16:27:01 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Rahul Siddharthan Cc: Simon Marlow , John Baldwin , arch@freebsd.org, Dan Nelson Subject: Re: Cleaning old packages (was: Package system flaws?) References: <20020709161953.GA69779@lpt.ens.fr> <20020709171417.GA69932@lpt.ens.fr> <20020709210911.GA248@lpt.ens.fr> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Rahul Siddharthan wrote: > So I think both the port, and the dependency, should carry some sort > of versioning information. The port should say which versions of the > dependency it's compatible with, and the dependency should say which > earlier versions are safe to upgrade from. We used to put these things called "minor version numbers" on shared libraries, back in the pre-ELF days. They were removed when ELF came in. The reason given was $REASON ...oops... sorry... looks like that variable still doesn't expand. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 16:31:15 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 46E4337B400 for ; Tue, 9 Jul 2002 16:31:13 -0700 (PDT) Received: from gnuppy.monkey.org (wsip68-15-8-100.sd.sd.cox.net [68.15.8.100]) by mx1.FreeBSD.org (Postfix) with ESMTP id EC24443E09 for ; Tue, 9 Jul 2002 16:31:12 -0700 (PDT) (envelope-from billh@gnuppy.monkey.org) Received: from billh by gnuppy.monkey.org with local (Exim 3.35 #1 (Debian)) id 17S4Rr-0000V7-00; Tue, 09 Jul 2002 16:31:11 -0700 Date: Tue, 9 Jul 2002 16:31:11 -0700 To: freebsd-arch@FreeBSD.ORG Cc: Neil Blakey-Milner , "M. Warner Losh" , naddy@mips.inka.de, Bill Huey Subject: Re: Package system flaws? Message-ID: <20020709233111.GA1651@gnuppy.monkey.org> References: <20020706220511.GA88651@scoobysnax.jaded.net> <20020708.094652.21114246.imp@village.org> <20020709073309.GB96335@dragon.nuxi.com> <20020709082753.GA60250@mithrandr.moria.org> <20020709185010.GA26983@dragon.nuxi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020709185010.GA26983@dragon.nuxi.com> User-Agent: Mutt/1.4i From: Bill Huey Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, Jul 09, 2002 at 11:50:10AM -0700, David O'Brien wrote: > Someone demoed the Debian package system to me at USENIX and it > appears > it does have this functionality. The Debian system is pretty hard to beat in just about all areas. Their apt-get facility with the commands "update" and "dist-upgrade" automatically resolves dependencies, downloads the packages and installs them (with pre/post config script) in one swoop. It's DB driven, so you can do stuff like query what file a package belongs to, etc... The package quality is very good and well maintained. It's for that reason that I still use Debian/Linux as my workstation. bill To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 16:36: 6 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F0C2137B400 for ; Tue, 9 Jul 2002 16:36:04 -0700 (PDT) Received: from gnuppy.monkey.org (wsip68-15-8-100.sd.sd.cox.net [68.15.8.100]) by mx1.FreeBSD.org (Postfix) with ESMTP id A0F1843E4A for ; Tue, 9 Jul 2002 16:36:04 -0700 (PDT) (envelope-from billh@gnuppy.monkey.org) Received: from billh by gnuppy.monkey.org with local (Exim 3.35 #1 (Debian)) id 17S4Wa-0000Vq-00; Tue, 09 Jul 2002 16:36:04 -0700 Date: Tue, 9 Jul 2002 16:36:04 -0700 To: freebsd-arch@FreeBSD.ORG Cc: Neil Blakey-Milner , "M. Warner Losh" , naddy@mips.inka.de, Bill Huey Subject: Re: Package system flaws? Message-ID: <20020709233604.GB1651@gnuppy.monkey.org> References: <20020706220511.GA88651@scoobysnax.jaded.net> <20020708.094652.21114246.imp@village.org> <20020709073309.GB96335@dragon.nuxi.com> <20020709082753.GA60250@mithrandr.moria.org> <20020709185010.GA26983@dragon.nuxi.com> <20020709233111.GA1651@gnuppy.monkey.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020709233111.GA1651@gnuppy.monkey.org> User-Agent: Mutt/1.4i From: Bill Huey Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, Jul 09, 2002 at 04:31:11PM -0700, Bill Huey wrote: > It's DB driven, so you can do stuff like query what file a package > belongs to, etc... The package quality is very good and well maintained. > It's for that reason that I still use Debian/Linux as my workstation. I meant you can query which "package a specific file" belongs to, sorry. There's also a pretty solid error recovery schema if you accidently have problem upgrading it in that you can restart the process and it'll continue from that point on... It's definitely worth checking out if you haven't seen it before. OS X's Fink project is Debian based and does very well for many that use it in that community folks. bill To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 16:40:27 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A090A37B400 for ; Tue, 9 Jul 2002 16:40:23 -0700 (PDT) Received: from albatross.prod.itd.earthlink.net (albatross.mail.pas.earthlink.net [207.217.120.120]) by mx1.FreeBSD.org (Postfix) with ESMTP id C434943E3B for ; Tue, 9 Jul 2002 16:40:22 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0275.cvx22-bradley.dialup.earthlink.net ([209.179.199.20] helo=mindspring.com) by albatross.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 17S4ah-00068E-00; Tue, 09 Jul 2002 16:40:19 -0700 Message-ID: <3D2B7434.366B9679@mindspring.com> Date: Tue, 09 Jul 2002 16:39:32 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Bill Huey Cc: freebsd-arch@FreeBSD.ORG, Neil Blakey-Milner , "M. Warner Losh" , naddy@mips.inka.de Subject: Re: Package system flaws? References: <20020706220511.GA88651@scoobysnax.jaded.net> <20020708.094652.21114246.imp@village.org> <20020709073309.GB96335@dragon.nuxi.com> <20020709082753.GA60250@mithrandr.moria.org> <20020709185010.GA26983@dragon.nuxi.com> <20020709233111.GA1651@gnuppy.monkey.org> <20020709233604.GB1651@gnuppy.monkey.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Bill Huey wrote: > I meant you can query which "package a specific file" belongs to, sorry. RPM can do this too. The permutations of what you should be able to do, and what having that ability lets you build on top of it, are ...interesting. I like the idea of an "unrootkit". -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 17: 8: 9 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 762C337B400 for ; Tue, 9 Jul 2002 17:08:07 -0700 (PDT) Received: from gnuppy.monkey.org (wsip68-15-8-100.sd.sd.cox.net [68.15.8.100]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1F2F743E42 for ; Tue, 9 Jul 2002 17:08:07 -0700 (PDT) (envelope-from billh@gnuppy.monkey.org) Received: from billh by gnuppy.monkey.org with local (Exim 3.35 #1 (Debian)) id 17S4jF-0000aJ-00; Tue, 09 Jul 2002 16:49:09 -0700 Date: Tue, 9 Jul 2002 16:49:03 -0700 To: Terry Lambert Cc: freebsd-arch@FreeBSD.ORG, Neil Blakey-Milner , "M. Warner Losh" , naddy@mips.inka.de, Bill Huey Subject: Re: Package system flaws? Message-ID: <20020709234903.GA2020@gnuppy.monkey.org> References: <20020706220511.GA88651@scoobysnax.jaded.net> <20020708.094652.21114246.imp@village.org> <20020709073309.GB96335@dragon.nuxi.com> <20020709082753.GA60250@mithrandr.moria.org> <20020709185010.GA26983@dragon.nuxi.com> <20020709233111.GA1651@gnuppy.monkey.org> <20020709233604.GB1651@gnuppy.monkey.org> <3D2B7434.366B9679@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3D2B7434.366B9679@mindspring.com> User-Agent: Mutt/1.4i From: Bill Huey Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, Jul 09, 2002 at 04:39:32PM -0700, Terry Lambert wrote: > RPM can do this too. > > The permutations of what you should be able to do, and what having > that ability lets you build on top of it, are ...interesting. I > like the idea of an "unrootkit". > > -- Terry dpkg does that and is much more sophisticated about package management in general. I can't really explain it in a manner that justifies its positive attributes, but I can definitely say that after using both RPM and dpkg based system that dpkg is vastly superior to it mainly because of dependency resolution. A sophisticated binary packaging system becomes important as dependencies get more complicated and since packages get out of sync. A solid system disiplines that problem into something much more automatic. I've done something close to your unroot kit pretty easily with dpkg on a damage machine without too much creative effort. This assumes that I understand you correctly. It's definitely worth check out if you haven't seen it before and makes most packaging systems (and definitely pkg_*) seem like they're in the dark ages. bill To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 17:17:20 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EBE0237B405 for ; Tue, 9 Jul 2002 17:17:18 -0700 (PDT) Received: from felix.automagic.org (felix.automagic.org [204.152.186.101]) by mx1.FreeBSD.org (Postfix) with SMTP id 2AEE643E5E for ; Tue, 9 Jul 2002 17:17:18 -0700 (PDT) (envelope-from jabley@felix.automagic.org) Received: (qmail 13067 invoked by uid 1000); 10 Jul 2002 00:17:18 -0000 Date: Tue, 9 Jul 2002 17:17:18 -0700 From: Joe Abley To: Terry Lambert Cc: Wes Peters , Dag-Erling Smorgrav , Doug Barton , Dan Moschuk , arch@FreeBSD.org Subject: Re: Package system flaws? Message-ID: <20020710001717.GB13050@felix.automagic.org> References: <83FD8E1F-935A-11D6-B320-00039312C852@automagic.org> <3D2B6ED6.66FAEF5B@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3D2B6ED6.66FAEF5B@mindspring.com> User-Agent: Mutt/1.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, Jul 09, 2002 at 04:16:38PM -0700, Terry Lambert wrote: > Joe Abley wrote: > > On Monday, July 8, 2002, at 10:09 , Terry Lambert wrote: > > > Wes Peters wrote: > > >> The more you write, the more I see XML as being terribly appropriate > > >> for the package metadata. Consider the nesting problem; XML handles > > >> this with complete grace if you write it into the DTD that a package > > >> can contain a package. > > > > > > XML would be good, if you didn't have to have 90M of crap in order > > > to parse it. > > > > felix# ls -al /usr/local/lib/libxml.a > > -rw-r--r-- 1 root wheel 399096 Jul 9 09:39 /usr/local/lib/libxml.a > > felix# > > There's 1/3 of a floppy disc gone... the cut down "vi" is what, 27K? Hey! You said XML would be good if the parser came in under 90M! :) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 19:18:43 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EFA5737B400 for ; Tue, 9 Jul 2002 19:18:42 -0700 (PDT) Received: from islab.kaist.ac.kr (islab.kaist.ac.kr [143.248.138.225]) by mx1.FreeBSD.org (Postfix) with ESMTP id E566F43E42 for ; Tue, 9 Jul 2002 19:18:41 -0700 (PDT) (envelope-from hgkang@islab.kaist.ac.kr) Received: from islabpc14 (islabpc14.kaist.ac.kr [143.248.138.244]) by islab.kaist.ac.kr (8.9.3/8.9.3) with SMTP id LAA17838 for ; Wed, 10 Jul 2002 11:21:04 +0900 (KST) From: =?ks_c_5601-1987?B?sK3I7LHZ?= To: Subject: unsubscribe Date: Wed, 10 Jul 2002 11:20:02 +0900 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="ks_c_5601-1987" Content-Transfer-Encoding: base64 X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG YXV0aCA0NTNmMDc0MyB1bnN1YnNjcmliZSBmcmVlYnNkLWFyY2ggaGdrYW5nQGlzbGFiLmthaXN0 LmFjLmty To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 19:30:38 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 98A9037B400; Tue, 9 Jul 2002 19:30:35 -0700 (PDT) Received: from gull.mail.pas.earthlink.net (gull.mail.pas.earthlink.net [207.217.120.84]) by mx1.FreeBSD.org (Postfix) with ESMTP id 43C8243E09; Tue, 9 Jul 2002 19:30:35 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0551.cvx21-bradley.dialup.earthlink.net ([209.179.194.41] helo=mindspring.com) by gull.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17S7FE-0004Vc-00; Tue, 09 Jul 2002 19:30:20 -0700 Message-ID: <3D2B9C11.94DF373F@mindspring.com> Date: Tue, 09 Jul 2002 19:29:37 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Joe Abley Cc: Wes Peters , Dag-Erling Smorgrav , Doug Barton , Dan Moschuk , arch@FreeBSD.org Subject: Re: Package system flaws? References: <83FD8E1F-935A-11D6-B320-00039312C852@automagic.org> <3D2B6ED6.66FAEF5B@mindspring.com> <20020710001717.GB13050@felix.automagic.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Joe Abley wrote: > > There's 1/3 of a floppy disc gone... the cut down "vi" is what, 27K? > > Hey! You said XML would be good if the parser came in under 90M! > > :) I have a 10 user dramatic license. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 20:15: 4 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F1F1137B400; Tue, 9 Jul 2002 20:15:02 -0700 (PDT) Received: from InterJet.dellroad.org (adsl-63-194-81-26.dsl.snfc21.pacbell.net [63.194.81.26]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6989B43E42; Tue, 9 Jul 2002 20:15:02 -0700 (PDT) (envelope-from archie@dellroad.org) Received: from arch20m.dellroad.org (arch20m.dellroad.org [10.1.1.20]) by InterJet.dellroad.org (8.9.1a/8.9.1) with ESMTP id UAA63731; Tue, 9 Jul 2002 20:11:07 -0700 (PDT) Received: (from archie@localhost) by arch20m.dellroad.org (8.11.6/8.11.6) id g6A3AZB23117; Tue, 9 Jul 2002 20:10:35 -0700 (PDT) (envelope-from archie) From: Archie Cobbs Message-Id: <200207100310.g6A3AZB23117@arch20m.dellroad.org> Subject: Re: Package system flaws? In-Reply-To: <20020707153457.GA1086@scoobysnax.jaded.net> "from Dan Moschuk at Jul 7, 2002 11:34:57 am" To: Dan Moschuk Date: Tue, 9 Jul 2002 20:10:35 -0700 (PDT) Cc: Dag-Erling Smorgrav , Wes Peters , arch@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL88 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Dan Moschuk writes: > I don't think using an archive format like zip would be a step in the > right direction. If the package file format were to be redesigned, I would > vote for a custom header prepended to a bziped tarball. tar has a limitation which I've encountered: suppose you have a port that installs a man page with lots of references (i.e., hard linked files with different names with a single underlying file). Then in tar format, you get the same file copied N times. If we used cpio instead (for example) then it "knows" how to handle hard links. So I'd say cpio is better than tar, though something else altogether might be better than both. -Archie __________________________________________________________________________ Archie Cobbs * Packet Design * http://www.packetdesign.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 20:32: 2 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 45C2C37B400; Tue, 9 Jul 2002 20:31:57 -0700 (PDT) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by mx1.FreeBSD.org (Postfix) with ESMTP id B766743E58; Tue, 9 Jul 2002 20:31:56 -0700 (PDT) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.12.5/8.12.5) id g6A3Vsoi023788; Tue, 9 Jul 2002 22:31:54 -0500 (CDT) (envelope-from dan) Date: Tue, 9 Jul 2002 22:31:54 -0500 From: Dan Nelson To: Archie Cobbs Cc: Dan Moschuk , Dag-Erling Smorgrav , Wes Peters , arch@FreeBSD.ORG Subject: Re: Package system flaws? Message-ID: <20020710033154.GD8625@dan.emsphone.com> References: <20020707153457.GA1086@scoobysnax.jaded.net> <200207100310.g6A3AZB23117@arch20m.dellroad.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200207100310.g6A3AZB23117@arch20m.dellroad.org> X-OS: FreeBSD 5.0-CURRENT X-message-flag: Outlook Error User-Agent: Mutt/1.5.1i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In the last episode (Jul 09), Archie Cobbs said: > Dan Moschuk writes: > > I don't think using an archive format like zip would be a step in > > the right direction. If the package file format were to be > > redesigned, I would vote for a custom header prepended to a bziped > > tarball. > > tar has a limitation which I've encountered: suppose you have a port > that installs a man page with lots of references (i.e., hard linked > files with different names with a single underlying file). Then in > tar format, you get the same file copied N times. If we used cpio > instead (for example) then it "knows" how to handle hard links. Tar handles hardlinks just fine. -- Dan Nelson dnelson@allantgroup.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 21:10:31 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B771237B400; Tue, 9 Jul 2002 21:10:26 -0700 (PDT) Received: from jkh-gw.queasyweasel.com (adsl-64-173-3-158.dsl.sntc01.pacbell.net [64.173.3.158]) by mx1.FreeBSD.org (Postfix) with ESMTP id E46BD43E58; Tue, 9 Jul 2002 21:10:25 -0700 (PDT) (envelope-from jkh@queasyweasel.com) Received: from adsl-64-173-15-99.dsl.sntc01.pacbell.net (jkh@mango.freebsd.com [64.173.15.99]) by jkh-gw.queasyweasel.com (8.12.3/8.12.3) with ESMTP id g6A4A1vm043889; Tue, 9 Jul 2002 21:10:02 -0700 (PDT) (envelope-from jkh@queasyweasel.com) Date: Tue, 9 Jul 2002 21:10:42 -0700 Subject: Re: Package system flaws? Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v482) Cc: Archie Cobbs , Dan Moschuk , Dag-Erling Smorgrav , Wes Peters , arch@FreeBSD.ORG To: Dan Nelson From: Jordan K Hubbard In-Reply-To: <20020710033154.GD8625@dan.emsphone.com> Message-Id: Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.482) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Oh dear, why do I find myself so unable to resist this thread? :) I agree with whomever it was that said that the "right" way to conduct this exercise would be to first define the objectives and then pick whichever package format (or invent one) which fills all those goals. We've already seen considerable debate over random access capabilities and being able to get at the "dictionary" portion of a package either by sticking it at the front or putting it in another file, so I won't go into that all over again, but I will bring up something that nobody seems to have touched on yet: Fat packages. I'm borrowing a bit of Macintosh nomenclature there (though I'm sure Terry will come along and correct me by pointing out that IBM was the first to introduce "Fat binaries" with VM/CMS or something :) but I'm sure people get the idea. If you're distributing an Emacs or TeX package which weighs in at some hefty percentage of the New York phone book in size, and with KDE and Gnome one doesn't even have to look to Emacs anymore for a good example of a "really big, honkin' package", you naturally want to save on disk space if at all possible both to minimize load on the archives and make those poor Australian users with their 9600 baud Telstra link to the US happy. Compression is certainly a good start, but when you start distributing those packages for 3 or 4 different architectures (as FreeBSD is definitely not far away from doing) you also would like to not distribute 3 or 4 different copies of the same architecture-neutral bits if you can possibly help it. That's where the idea of munging attributes into the dictionary namespace starts making more and more sense, and not just for representing different architectures but also thinks like "experimental vs stable" variants, "mix-ins" (like all the various versions of Apache which have various bits of compiled-in smarts) and what have you. If you introduce the concept of install-time attributes, some of which may be implicit (like architecture) and some of which may be explicit (like "give me the experimental version please"), you conceivably end up with mangled pathnames within the package which are demangled on the way out, C++-style. This allows you to have, say, just one copy of all the Emacs lisp files and documentation but 3 or 4 different "bin/emacs" files which don't collide internally and are properly selected for on extraction. Anyway, wish-list items like this are why it's a good idea to define the goals first and the package format second. :-) P.S. I also gree with jhb's assertion that some folks really need to take a good look at libh since it takes a number of things like this into account, including all the "occludes, obsoletes, upgrades, ..." attributes that people were recently demanding as package metadata. -- Jordan K. Hubbard Engineering Manager, BSD technology group Apple Computer To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 21:15: 8 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3431E37B400; Tue, 9 Jul 2002 21:15:05 -0700 (PDT) Received: from InterJet.dellroad.org (adsl-63-194-81-26.dsl.snfc21.pacbell.net [63.194.81.26]) by mx1.FreeBSD.org (Postfix) with ESMTP id A5AA943E09; Tue, 9 Jul 2002 21:15:04 -0700 (PDT) (envelope-from archie@dellroad.org) Received: from arch20m.dellroad.org (arch20m.dellroad.org [10.1.1.20]) by InterJet.dellroad.org (8.9.1a/8.9.1) with ESMTP id UAA63956; Tue, 9 Jul 2002 20:58:37 -0700 (PDT) Received: (from archie@localhost) by arch20m.dellroad.org (8.11.6/8.11.6) id g6A3w4D25814; Tue, 9 Jul 2002 20:58:04 -0700 (PDT) (envelope-from archie) From: Archie Cobbs Message-Id: <200207100358.g6A3w4D25814@arch20m.dellroad.org> Subject: Re: Package system flaws? In-Reply-To: <20020710033154.GD8625@dan.emsphone.com> "from Dan Nelson at Jul 9, 2002 10:31:54 pm" To: Dan Nelson Date: Tue, 9 Jul 2002 20:58:04 -0700 (PDT) Cc: Archie Cobbs , Dan Moschuk , Dag-Erling Smorgrav , Wes Peters , arch@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL88 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Dan Nelson writes: > > tar has a limitation which I've encountered: suppose you have a port > > that installs a man page with lots of references (i.e., hard linked > > files with different names with a single underlying file). Then in > > tar format, you get the same file copied N times. If we used cpio > > instead (for example) then it "knows" how to handle hard links. > > Tar handles hardlinks just fine. Oops, you're right... don't know how I got that idea in my head. OK me shut up now :-) -Archie __________________________________________________________________________ Archie Cobbs * Packet Design * http://www.packetdesign.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 23:23:23 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B73FD37B400; Tue, 9 Jul 2002 23:23:21 -0700 (PDT) Received: from pintail.mail.pas.earthlink.net (pintail.mail.pas.earthlink.net [207.217.120.122]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4E72843E31; Tue, 9 Jul 2002 23:23:21 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0241.cvx22-bradley.dialup.earthlink.net ([209.179.198.241] helo=mindspring.com) by pintail.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17SAse-0000Id-00; Tue, 09 Jul 2002 23:23:17 -0700 Message-ID: <3D2BD2A7.F77DDB93@mindspring.com> Date: Tue, 09 Jul 2002 23:22:31 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Archie Cobbs Cc: Dan Nelson , Dan Moschuk , Dag-Erling Smorgrav , Wes Peters , arch@FreeBSD.ORG Subject: Re: Package system flaws? References: <200207100358.g6A3w4D25814@arch20m.dellroad.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Archie Cobbs wrote: > Dan Nelson writes: > > > tar has a limitation which I've encountered: suppose you have a port > > > that installs a man page with lots of references (i.e., hard linked > > > files with different names with a single underlying file). Then in > > > tar format, you get the same file copied N times. If we used cpio > > > instead (for example) then it "knows" how to handle hard links. > > > > Tar handles hardlinks just fine. > > Oops, you're right... don't know how I got that idea in my head. > OK me shut up now :-) It's a GNU tar thing; real tar acts the way you say, and also has problems with holey files, etc.. We used to use cpio for nearly everything back in the old days at Century Software (mid-late 80's). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 23:36:40 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CE59237B400 for ; Tue, 9 Jul 2002 23:36:38 -0700 (PDT) Received: from mail.nsu.ru (mx.nsu.ru [193.124.215.71]) by mx1.FreeBSD.org (Postfix) with ESMTP id 40E4B43E42 for ; Tue, 9 Jul 2002 23:36:35 -0700 (PDT) (envelope-from danfe@regency.nsu.ru) Received: from drweb by mail.nsu.ru with drweb-scanned (Exim 3.20 #1) id 17SB4x-0004QE-00; Wed, 10 Jul 2002 13:35:59 +0700 Received: from regency.nsu.ru ([193.124.210.26]) by mail.nsu.ru with esmtp (Exim 3.20 #1) id 17SB4x-0004Q2-00; Wed, 10 Jul 2002 13:35:59 +0700 Received: from regency.nsu.ru (localhost [127.0.0.1]) by regency.nsu.ru (8.12.4/8.12.4) with ESMTP id g6A6aaLF041443; Wed, 10 Jul 2002 13:36:36 +0700 (NOVST) (envelope-from danfe@regency.nsu.ru) Received: (from danfe@localhost) by regency.nsu.ru (8.12.4/8.12.4/Submit) id g6A6aYkt041368; Wed, 10 Jul 2002 13:36:34 +0700 (NOVST) Date: Wed, 10 Jul 2002 13:36:33 +0700 From: Alexey Dokuchaev To: Bill Huey Cc: freebsd-arch@freebsd.org, Neil Blakey-Milner , "M. Warner Losh" , naddy@mips.inka.de Subject: Re: Package system flaws? Message-ID: <20020710133633.B35244@regency.nsu.ru> References: <20020706220511.GA88651@scoobysnax.jaded.net> <20020708.094652.21114246.imp@village.org> <20020709073309.GB96335@dragon.nuxi.com> <20020709082753.GA60250@mithrandr.moria.org> <20020709185010.GA26983@dragon.nuxi.com> <20020709233111.GA1651@gnuppy.monkey.org> <20020709233604.GB1651@gnuppy.monkey.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <20020709233604.GB1651@gnuppy.monkey.org>; from billh@gnuppy.monkey.org on Tue, Jul 09, 2002 at 04:36:04PM -0700 X-Envelope-To: billh@gnuppy.monkey.org, freebsd-arch@freebsd.org, nbm@mithrandr.moria.org, imp@village.org, naddy@mips.inka.de Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, Jul 09, 2002 at 04:36:04PM -0700, Bill Huey wrote: > On Tue, Jul 09, 2002 at 04:31:11PM -0700, Bill Huey wrote: > > It's DB driven, so you can do stuff like query what file a package > > belongs to, etc... The package quality is very good and well maintained. > > It's for that reason that I still use Debian/Linux as my workstation. > > I meant you can query which "package a specific file" belongs to, sorry. > > There's also a pretty solid error recovery schema if you accidently > have problem upgrading it in that you can restart the process and > it'll continue from that point on... It's definitely worth checking > out if you haven't seen it before. > > OS X's Fink project is Debian based and does very well for many that > use it in that community folks. Indeed, lots of Linux distros out there had acquired Debian packaging system for its robustness, convenience, and reliability. Heck, Mandrake was RPM based until recently, same does apply to SuSE and Russian ALT Linux AFAIK. People tend to drop RPM in favor of apt-based package repository and tools, which is, at least, a good indication of that Debian packaging system is something quite worth looking into. Just my $.02 ./danfe To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 23:47:18 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6C5B537B400; Tue, 9 Jul 2002 23:47:16 -0700 (PDT) Received: from mail.nsu.ru (mx.nsu.ru [193.124.215.71]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9A8D143E09; Tue, 9 Jul 2002 23:47:14 -0700 (PDT) (envelope-from danfe@regency.nsu.ru) Received: from drweb by mail.nsu.ru with drweb-scanned (Exim 3.20 #1) id 17SBFa-0005c0-00; Wed, 10 Jul 2002 13:46:58 +0700 Received: from regency.nsu.ru ([193.124.210.26]) by mail.nsu.ru with esmtp (Exim 3.20 #1) id 17SBFZ-0005bo-00; Wed, 10 Jul 2002 13:46:57 +0700 Received: from regency.nsu.ru (localhost [127.0.0.1]) by regency.nsu.ru (8.12.4/8.12.4) with ESMTP id g6A6lZLF046834; Wed, 10 Jul 2002 13:47:35 +0700 (NOVST) (envelope-from danfe@regency.nsu.ru) Received: (from danfe@localhost) by regency.nsu.ru (8.12.4/8.12.4/Submit) id g6A6lZkD046788; Wed, 10 Jul 2002 13:47:35 +0700 (NOVST) Date: Wed, 10 Jul 2002 13:47:34 +0700 From: Alexey Dokuchaev To: Archie Cobbs Cc: Dan Moschuk , Dag-Erling Smorgrav , Wes Peters , arch@freebsd.org Subject: Re: Package system flaws? Message-ID: <20020710134734.C35244@regency.nsu.ru> References: <20020707153457.GA1086@scoobysnax.jaded.net> <200207100310.g6A3AZB23117@arch20m.dellroad.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <200207100310.g6A3AZB23117@arch20m.dellroad.org>; from archie@dellroad.org on Tue, Jul 09, 2002 at 08:10:35PM -0700 X-Envelope-To: archie@dellroad.org, dan@freebsd.org, des@ofug.org, wes@softweyr.com, arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, Jul 09, 2002 at 08:10:35PM -0700, Archie Cobbs wrote: > Dan Moschuk writes: > > I don't think using an archive format like zip would be a step in the > > right direction. If the package file format were to be redesigned, I would > > vote for a custom header prepended to a bziped tarball. > > tar has a limitation which I've encountered: suppose you have a port > that installs a man page with lots of references (i.e., hard linked > files with different names with a single underlying file). Then in > tar format, you get the same file copied N times. If we used cpio > instead (for example) then it "knows" how to handle hard links. Uhm, I'm afraid you are wrong on this one: [danfe@regency:ttypb] /tmp/tar> echo "123123" > foo [danfe@regency:ttypb] /tmp/tar> ln foo bar [danfe@regency:ttypb] /tmp/tar> ls -i 43 bar 43 foo [danfe@regency:ttypb] /tmp/tar> tar cvf ../foobar.tar . ./ foo bar [danfe@regency:ttypb] /tmp/tar> rm * [danfe@regency:ttypb] /tmp/tar> ls -i [danfe@regency:ttypb] /tmp/tar> tar xvf ../foobar.tar ./ foo bar [danfe@regency:ttypb] /tmp/tar> ls -i 48 bar 48 foo [danfe@regency:ttypb] /tmp/tar> ./danfe To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jul 9 23:54: 1 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ACCED37B400 for ; Tue, 9 Jul 2002 23:53:59 -0700 (PDT) Received: from gw.catspoiler.org (217-ip-163.nccn.net [209.79.217.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0B86543E54 for ; Tue, 9 Jul 2002 23:53:59 -0700 (PDT) (envelope-from dl-freebsd@catspoiler.org) Received: from mousie.catspoiler.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.12.5/8.12.5) with ESMTP id g6A6rjwr006212; Tue, 9 Jul 2002 23:53:49 -0700 (PDT) (envelope-from dl-freebsd@catspoiler.org) Message-Id: <200207100653.g6A6rjwr006212@gw.catspoiler.org> Date: Tue, 9 Jul 2002 23:53:45 -0700 (PDT) From: Don Lewis Subject: Re: Package system flaws? To: tlambert2@mindspring.com Cc: temik@egartech.com, wollman@lcs.mit.edu, mark@thuvia.demon.co.uk, arch@FreeBSD.ORG In-Reply-To: <3D2B65A3.ABB92114@mindspring.com> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 9 Jul, Terry Lambert wrote: > Artem Tepponen wrote: >> No, Terry. Dictionary locality works in a different way. >> gzipped tar will almost always win vs. tarred gzipped files. >> 10-15% from memory. Just a quick check: >> >> -rw-r--r-- 1 temik develops 29020160 Jul 9 12:36 gcc-3.0.1.gz.tar >> -rw-r--r-- 1 temik develops 13821669 Jul 9 12:41 gcc-3.0.1.tar.bz2 >> -rw-r--r-- 1 temik develops 18054324 Sep 24 2001 gcc-3.0.1.tar.gz >> -rw-r--r-- 1 temik develops 22746511 Jul 9 12:52 gcc-3.0.1.zip >> >> Oops. I was wrong. >35% is a big difference. And bzip adds another 24%. >> But for binaries difference between gzip vs. bzip2 will be smaller. >> >> This is quite simple check but the picture will remain the same >> for pretty any kind of data and hope that's enough to choose >> single tar.somez + header. >> >> Will header be combined or in a different file is another question. > > 1) "Most compression", not "all compression". > > 2) LZW resets the dictionary every 12K. This is the patented > process that Terry Welch of Unisys introduced. So your > argument is only valid for a lot of small files who size > is well under 12K, which have similar contents. > > 3) I believe gzip and bzip were both written to get out from > under the Unisys patent, and therefore do not compress as > well as they could compress, even though Unisys has granted > blanket royalty free use for certain applications which fall > into this category. In the comparisons I've done between gzip(1) and compress(1), gzip has always gotten better compression that compress, though gzip runs slower. When I've tuned the compression level knob on gzip to get similar compression levels, it runs faster than compress. The algorithm.doc file in the gzip distribution seems to indicate that gzip resets its dictionary when it decides it would be advantageous to do so. I've read somewhere a long time ago that the compression results would be better if it used arithmetic encoding on its output instead of Huffman encoding, but I believe that IBM has the patent on arithmetic encoding. > NB: The Unisys patent expires on Dec 10th of this year, in any case, > so the only reason bzip/gzip wouldn't support using it after that is > religious. I was just thinking the other day that the patent expiration date should be approaching. I believe that the area most impacted by this patent in recent years is the creation of .gif files. At least that's what has gotten all the press. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 0: 0:44 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C531C37B409 for ; Wed, 10 Jul 2002 00:00:26 -0700 (PDT) Received: from 12-234-90-219.client.attbi.com (12-234-90-219.client.attbi.com [12.234.90.219]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3246143E54 for ; Wed, 10 Jul 2002 00:00:26 -0700 (PDT) (envelope-from DougB@FreeBSD.org) Received: from Master.gorean.org (master.gorean.org [10.0.0.2]) by 12-234-90-219.client.attbi.com (8.12.3/8.12.3) with ESMTP id g6A70LC6029457; Wed, 10 Jul 2002 00:00:25 -0700 (PDT) (envelope-from DougB@FreeBSD.org) Received: from localhost (doug@localhost) by Master.gorean.org (8.12.5/8.12.5/Submit) with ESMTP id g6A6XWHA006105; Tue, 9 Jul 2002 23:33:33 -0700 (PDT) X-Authentication-Warning: Master.gorean.org: doug owned process doing -bs Date: Tue, 9 Jul 2002 23:33:32 -0700 (PDT) From: Doug Barton To: Terry Lambert Cc: arch@FreeBSD.org Subject: Re: Package system flaws? In-Reply-To: <3D2A3E73.8A6BFA16@mindspring.com> Message-ID: <20020709232917.X5990-100000@master.gorean.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, 8 Jul 2002, Terry Lambert wrote: > Doug Barton wrote: > > > If it's good to put metadata in a file seperate from the data it > > > describes, then the judgement of "goodness" is universal. > > > > This I disagree with. Insert obligatory quote about "foolish consistency." > > > > > We do this because it makes the data and metadata non-severable, > > > > See a later post by me on this thread where I clearly state severability > > as a specific, and desirable goal for this application. > > Insert obligatory quote about "1 0wn y00r b0x". 8-). Blah... it would be very easy to make sure that the right two things are matched up. Also, if there are no instructions for what to do in the metadata file, there is no danger. > I give: What are the benefits of having to downlaod multiple > files, rather than a single file, via HTTP protocol? Well, for one we avoid all the ugly arguments about why we need to encode the metadata into archive/compressor/tarball/blahblah format X. Two, we allow users to see and view the files without having to "download" anything. Three, we can upgrade metadata seperately from the binaries when one changes but the other doesn't need to. Those are off the top of my head. > But be aware that we may have different ideas of the problem space... That's why I think we should still be discussing THAT, instead of possible solutions. One of the problems with this group is that everyone is eager to be the one who fixes things.... -- "We have known freedom's price. We have shown freedom's power. And in this great conflict, ... we will see freedom's victory." - George W. Bush, President of the United States State of the Union, January 28, 2002 Do YOU Yahoo!? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 0: 0:53 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D9F3637B43A; Wed, 10 Jul 2002 00:00:29 -0700 (PDT) Received: from 12-234-90-219.client.attbi.com (12-234-90-219.client.attbi.com [12.234.90.219]) by mx1.FreeBSD.org (Postfix) with ESMTP id 58BE843E09; Wed, 10 Jul 2002 00:00:28 -0700 (PDT) (envelope-from DougB@FreeBSD.org) Received: from Master.gorean.org (master.gorean.org [10.0.0.2]) by 12-234-90-219.client.attbi.com (8.12.3/8.12.3) with ESMTP id g6A70LCA029457; Wed, 10 Jul 2002 00:00:27 -0700 (PDT) (envelope-from DougB@FreeBSD.org) Received: from localhost (doug@localhost) by Master.gorean.org (8.12.5/8.12.5/Submit) with ESMTP id g6A6akJZ006111; Tue, 9 Jul 2002 23:36:49 -0700 (PDT) X-Authentication-Warning: Master.gorean.org: doug owned process doing -bs Date: Tue, 9 Jul 2002 23:36:46 -0700 (PDT) From: Doug Barton To: Wes Peters Cc: Dag-Erling Smorgrav , Dan Moschuk , Subject: Re: Package system flaws? In-Reply-To: <3D2A191C.3103D0FB@softweyr.com> Message-ID: <20020709233503.I5990-100000@master.gorean.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, 8 Jul 2002, Wes Peters wrote: > Doug Barton wrote: > > > > On Sun, 7 Jul 2002, Terry Lambert wrote: > > > > > > We want to be able to install a package from a non-rewindable source > > > > without storing a temporary copy on disk. This means the metadata > > > > must without fail be at the very beginning of the package. > > > > Ok, then what about storing the meta data as a seperate file? Why do they > > have to be in the same package? > > So you can (md5, sign) them together and know that they "apply" to each > other. If we limit the metadata properly (for my definition of "proper") there won't be anything that needs to be verified. The metadata file should of course include the md5 of the package(s) it applies to directly. -- "We have known freedom's price. We have shown freedom's power. And in this great conflict, ... we will see freedom's victory." - George W. Bush, President of the United States State of the Union, January 28, 2002 Do YOU Yahoo!? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 0: 2:40 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ADDF937B400; Wed, 10 Jul 2002 00:02:33 -0700 (PDT) Received: from swan.mail.pas.earthlink.net (swan.mail.pas.earthlink.net [207.217.120.123]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1F0E843E42; Wed, 10 Jul 2002 00:02:33 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0241.cvx22-bradley.dialup.earthlink.net ([209.179.198.241] helo=mindspring.com) by swan.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17SBUW-00021d-00; Wed, 10 Jul 2002 00:02:25 -0700 Message-ID: <3D2BDBD5.CA71A2C1@mindspring.com> Date: Wed, 10 Jul 2002 00:01:41 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Jordan K Hubbard Cc: Dan Nelson , Archie Cobbs , Dan Moschuk , Dag-Erling Smorgrav , Wes Peters , arch@FreeBSD.ORG Subject: Re: Package system flaws? References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Jordan K Hubbard wrote: > Oh dear, why do I find myself so unable to resist this thread? :) Occupational hazard? 8-). > I'm borrowing a bit of Macintosh nomenclature there (though I'm sure > Terry will come along and correct me by pointing out that IBM was the > first to introduce "Fat binaries" with VM/CMS or something :) but I'm > sure people get the idea. If you're distributing an Emacs or TeX > package which weighs in at some hefty percentage of the New York phone > book in size, and with KDE and Gnome one doesn't even have to look to > Emacs anymore for a good example of a "really big, honkin' package", you > naturally want to save on disk space if at all possible both to minimize > load on the archives and make those poor Australian users with their > 9600 baud Telstra link to the US happy. Compression is certainly a > good start, but when you start distributing those packages for 3 or 4 > different architectures (as FreeBSD is definitely not far away from > doing) you also would like to not distribute 3 or 4 different copies of > the same architecture-neutral bits if you can possibly help it. That's > where the idea of munging attributes into the dictionary namespace > starts making more and more sense, and not just for representing > different architectures but also thinks like "experimental vs stable" > variants, "mix-ins" (like all the various versions of Apache which have > various bits of compiled-in smarts) and what have you. If you introduce > the concept of install-time attributes, some of which may be implicit > (like architecture) and some of which may be explicit (like "give me the > experimental version please"), you conceivably end up with mangled > pathnames within the package which are demangled on the way out, > C++-style. This allows you to have, say, just one copy of all the Emacs > lisp files and documentation but 3 or 4 different "bin/emacs" files > which don't collide internally and are properly selected for on > extraction. Apple has dealt with this for the 68K vs. PPC binaries by stuffing the binaries into the same package, as a unit, and putting them in different "code resource" in the resource fork of the same file, while sharing the "data fork" between the code. The historical canonical answer is "ANDF" -- Architecture Neutral Distribution Format. In ANDF, one would compute the quad tree for a compiled program (for example), but not do the code generation. Instead, one would store the quad tree, and generate the actual code, at install time. ANDF is actually a brilliant idea, since the resulting program is not source code. It also doesn't have the problem of the Apple approach, which is bloating the file size by the unused portion(s) of the code contained therein. Unfortunately, creation of intermediate languages is against the policy of the FSF, due to it's ability to weaken the effect of the GPL on programs. This is the same reason the data dictionary work that was discussed on -advocacy and -chat recently is frowned upon by RMS, and why he posted what he did about sweeping the research under the rug to avoid exposure. At this point in time, ANDF is not an option because of license politics having nothing to do with available technology. I would like to think that a pure copy of the Apple approach was, likewise, not an option. Specifically -- and it looks like this is what you are actually advocating here -- extracting the portions of the applicable binaries at installation time, rather than installing the whole thing, is probably the most correct approach. It means that you only end up with what you are worried about on your disk. It also has the (dubious, in some people's eyes) attribute of making it impossible to take an installed package and recreate the package as a distribution -- you can't go to any machine with the software installed on it, and simply plug in you iPirate or whatever, and suck the package down to ensure redistributability to any target platform supported by the original distribution. The one real drawback with this approach is linear media. That is, if I have a DVD distribution, it's not a problem. But if I have a 28K modem... it is. A CDROM distribution is mildy problematic, since putting i386, PPC, SPARC, Alpha, and IA64 (and S/360? 8-)) binaries all in the same package could require a lot of CDROMs. But that's still less of a problem then linear media, like a TCP connection stream or a magnetic tape. There are two workarounds for this... assuming you can get the metadata early, which means that it's at a known location in every file, which pretty much dictates the front of the file. The first is that the FTP protocol supports the concept of a "REGET", and does not check that the receiver has the most recent version of the file ... it only cares about the byte offset, AND it supports interruption of transfers. The second is that the HTTP 1.1 protocol support HTTP GET with a range argument for a byte range for a server object. Both of these options effectively support random acces of known locations within a file, at a known offset. Not all FTP and HTTP servers support these facilities; enough do that it might be OK to rely upon them. HTTP is particularly attractive, due to firewall issues at large companies, where other protocols are blocked. A corollary to the use of random access at a known offset from metadata gathered from a known offset is that per-file MD5 (or other cryptographic) signatures can not be applied to the file as a whole. Magnetic tape may not have to worry about this... since you are going to have to read everything to byte-count for offset based extents anyway... but other other media where the intervening bytes are not traversed *is* a problem for those of us without corporate network connectivity. Even a cable modem or DSL could be come onerous for a large package (your EMACS example would probably strain a T1 user's patience ;^)). > Anyway, wish-list items like this are why it's a good idea to define the > goals first and the package format second. :-) Yes. > P.S. I also gree with jhb's assertion that some folks really need to > take a good look at libh since it takes a number of things like this > into account, including all the "occludes, obsoletes, upgrades, ..." > attributes that people were recently demanding as package metadata. The current system uses libh. We would like to (or *I* would like to 8^)) exceed the capabilities of the current system. The thing that strikes me about libh is that there is a lot of human effort involved in the dependency tracking mechanism, if it's going to be possible to perform some of the relationship tracking that some of the posters have already requested. In particular, I think that there is not sufficient differentiation between "necessary" and "sufficient". Part of the problem there is that the ports system has always been based on the idea that most of the things in it get built by the user as needed, so no matter what, it's always "sufficient". Moving to a system that can support binary-only implies that the implicit guarantees that were there are no longer there, and you need to consider that "just rebuild" will not be an option. I don't mean to call the demands that were made incomplete or naieve, but... well, yes I *do* mean to call them naieve, or at *least* call them incomplete. 8-). Finally, I think that people often confuse design with implementation, and that just because a system is *capable* of solving a particular problem, that the initial implementation would have to be delayed until it *actually solves the problem*. I have a really big wishlish, and adding everyone's wishlist together yields a *huge* wishlist. It's clear that implementing everything isn't possible at the present time. But that doesn't mean that a design should not take this into account as potential futuere work. Even if something ends up falling on the floor (I rather expect almost *everything* will *have to*, and that the first revision will only solve one or two fundamental issues, like the "packaging the base system" problem), whatever the final design is, it needs to not *preclude* solving certain problems in the future, without having to reinvent the framework yet again. I guess we should start a seperate thread specifically for a "wishlist"; no offense to the person who volunteered to collect and summarize this, but I'd like to see the information captured without any editorializing by a single person; I think it will be more useful as raw data. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 0:25:40 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3E9D437B400 for ; Wed, 10 Jul 2002 00:25:36 -0700 (PDT) Received: from falcon.mail.pas.earthlink.net (falcon.mail.pas.earthlink.net [207.217.120.74]) by mx1.FreeBSD.org (Postfix) with ESMTP id A3B4D43E09 for ; Wed, 10 Jul 2002 00:25:35 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0241.cvx22-bradley.dialup.earthlink.net ([209.179.198.241] helo=mindspring.com) by falcon.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17SBqv-0001Eu-00 for arch@freebsd.org; Wed, 10 Jul 2002 00:25:34 -0700 Message-ID: <3D2BE142.E25CA9BC@mindspring.com> Date: Wed, 10 Jul 2002 00:24:50 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: arch@freebsd.org Subject: Package system wishlist Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG So, following Jordan's advice, what's on everyone's wishlist? Terry's Wishlist: o I don't want to download unnecessary information over a slow link, because PacBell won't put a DSLM on the end of a fiber optic cable in my area in order to service the 3000 apartments full of Oracle and other big company engineers whose companies would automatically pay them a nice monthly fee. o I want to know how long it's going to take to download all the package and package dependencies for what I've asked to be installed o I want to answer all the questions at once, in a marathon session, and then have everything "just work" afterwards (front loading user configuration) o I want to know that an individual part is good, if I can download parts of a package (I don't care if this means that the modern FTP/HTTP range approaches are used, or if things are stored in sperate files) o I'd like there to be one thing to download, so that when I get outside an area with cheap broadband communications, I can still get the work done o I want to have a distinction between "necessary" and "sufficient", so that if I have pbm 1.2.3 installed, I don't end up with pbm 1.6.9 installed, too, if 1.2.3 would have been sufficient o I'd like "one click install" of packages from a web site or a set of websites. This implies: o Cryptographic signatures o A different file extension that's not already supported by a browser or Apache o Modifications to the default FreeBSD Apache to set content transfer encoding for a binary file type o Modifications of the default "MIME Types" file on FreeBSD to reference an installation tool for the selected extension name o A tool to do the installing o I want the option of having pretty progress bars that are actually meaningful (e.g. "Time remaining: 00:22:31") o I want to have a command that can tell me everything that didn't come out of a package (this includes things that did come out of a package, but were replaced by some cracker or by mistake, etc.) o I want "blind packages"; these are packages that are depended upon by another package and not explicitly installed by a user. When all the packages that depend on it are removed (dependency count goes from 1->0), I want the system to not_remove/automatically_remove/offer_to_remove the blind package o I want to be able to remove system components, like "sendmail" and "OpenSSH". o Eventually, I want to be able to not install system components in the first place (e.g. it should be possible to do a "PicoBSD" with the standard tools). o I want to know how much disk space I have, vs. how much it's going to take to install something, so I can decide whether I really want it or not. o I want the "installed components" list to be accurate after an initial system install ("/bin/ls" is a comonent). o I want the "installed components" list to be accurate after a "make installworld". o I want an "Add/Remove" software icon in the KDE control panel, just like Windows has. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 0:27:13 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 855DB37B400 for ; Wed, 10 Jul 2002 00:27:10 -0700 (PDT) Received: from gw.catspoiler.org (217-ip-163.nccn.net [209.79.217.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id D609B43E4A for ; Wed, 10 Jul 2002 00:27:09 -0700 (PDT) (envelope-from dl-freebsd@catspoiler.org) Received: from mousie.catspoiler.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.12.5/8.12.5) with ESMTP id g6A7Quwr006313; Wed, 10 Jul 2002 00:27:00 -0700 (PDT) (envelope-from dl-freebsd@catspoiler.org) Message-Id: <200207100727.g6A7Quwr006313@gw.catspoiler.org> Date: Wed, 10 Jul 2002 00:26:56 -0700 (PDT) From: Don Lewis Subject: Re: Package system flaws? To: tlambert2@mindspring.com Cc: archie@dellroad.org, dnelson@allantgroup.com, dan@FreeBSD.ORG, des@ofug.org, wes@softweyr.com, arch@FreeBSD.ORG In-Reply-To: <3D2BD2A7.F77DDB93@mindspring.com> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 9 Jul, Terry Lambert wrote: > Archie Cobbs wrote: >> Dan Nelson writes: >> > > tar has a limitation which I've encountered: suppose you have a port >> > > that installs a man page with lots of references (i.e., hard linked >> > > files with different names with a single underlying file). Then in >> > > tar format, you get the same file copied N times. If we used cpio >> > > instead (for example) then it "knows" how to handle hard links. >> > >> > Tar handles hardlinks just fine. >> >> Oops, you're right... don't know how I got that idea in my head. >> OK me shut up now :-) > > It's a GNU tar thing; real tar acts the way you say, and also has > problems with holey files, etc.. We used to use cpio for nearly > everything back in the old days at Century Software (mid-late 80's). tar has handled hard links properly for a long time. My copy of the 7th Edition manual documents tar's "l" option: tells tar to complain if it cannot resolve all of the links to the files dumped. If this is not specified, no error messages are printed. While this doesn't directly indicate that tar does the restores one copy of the file and all the links, it certainly suggests this possibility. The 1985 edition of the Masscomp version of the Unix Programmers Manual documents the format of the tar(5) header. #define NAMSIZ 100 struct header { char name[NAMSIZ]; char mode[8]; char uid[8]; char gid[8]; char size[8]; char mtime[12]; char chksum[8]; char linkflag; char linkname[NAMSIZ]; } In the description is says: _Name_ is a null-terminated string. ... _Size_ is the size of the file in bytes. Links and symbolic links are dump with this field specified as zero. ... _Linkflag_ is ASCII '0' if the file is "normal" or a special file, ASCII '1' if it is an hard link, and ASCII '2' if it is a symbolic link. The name linked-to, if any, is in _linkname_, with a trailing null. ... The first time a given i-node number is dumped, it is dumped as a regular file. The second and subsequent times, it is dumped as a link instead. Upon retrieval, if a link entry is retrieved, but not the file it was linked to, an error message is printed and the tape must be manually rescanned to retrieve the linked-to file. Back in those days I prefered tar's interface to cpio's, but I had to use cpio a lot because it had larger filename length limits. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 0:37:53 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8D19637B400 for ; Wed, 10 Jul 2002 00:37:52 -0700 (PDT) Received: from 12-234-90-219.client.attbi.com (12-234-90-219.client.attbi.com [12.234.90.219]) by mx1.FreeBSD.org (Postfix) with ESMTP id 21A7B43E54 for ; Wed, 10 Jul 2002 00:37:52 -0700 (PDT) (envelope-from DougB@FreeBSD.org) Received: from FreeBSD.org (master.gorean.org [10.0.0.2]) by 12-234-90-219.client.attbi.com (8.12.3/8.12.3) with ESMTP id g6A7bpBu029863; Wed, 10 Jul 2002 00:37:51 -0700 (PDT) (envelope-from DougB@FreeBSD.org) Message-ID: <3D2BE44F.1D6094C0@FreeBSD.org> Date: Wed, 10 Jul 2002 00:37:51 -0700 From: Doug Barton Organization: Triborough Bridge & Tunnel Authority X-Mailer: Mozilla 4.79 [en] (X11; U; FreeBSD 4.5-RELEASE i386) X-Accept-Language: en MIME-Version: 1.0 To: Jordan K Hubbard Cc: arch@FreeBSD.org Subject: Re: Package system flaws? References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Jordan K Hubbard wrote: > but when you start distributing those packages for 3 or 4 > different architectures (as FreeBSD is definitely not far away from > doing) you also would like to not distribute 3 or 4 different copies of > the same architecture-neutral bits if you can possibly help it. That's another argument in favor of having the metadata like dependencies in completely seperate files from the binaries. Doug To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 1:15:54 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5984B37B400 for ; Wed, 10 Jul 2002 01:15:48 -0700 (PDT) Received: from mail.nsu.ru (mx.nsu.ru [193.124.215.71]) by mx1.FreeBSD.org (Postfix) with ESMTP id B23B743E09 for ; Wed, 10 Jul 2002 01:15:43 -0700 (PDT) (envelope-from danfe@regency.nsu.ru) Received: from drweb by mail.nsu.ru with drweb-scanned (Exim 3.20 #1) id 17SCco-0007R9-00; Wed, 10 Jul 2002 15:15:02 +0700 Received: from regency.nsu.ru ([193.124.210.26]) by mail.nsu.ru with esmtp (Exim 3.20 #1) id 17SCco-0007Qw-00; Wed, 10 Jul 2002 15:15:02 +0700 Received: from regency.nsu.ru (localhost [127.0.0.1]) by regency.nsu.ru (8.12.4/8.12.4) with ESMTP id g6A8ExnP041955; Wed, 10 Jul 2002 15:14:59 +0700 (NOVST) (envelope-from danfe@regency.nsu.ru) Received: (from danfe@localhost) by regency.nsu.ru (8.12.4/8.12.4/Submit) id g6A8Ex2R041876; Wed, 10 Jul 2002 15:14:59 +0700 (NOVST) Date: Wed, 10 Jul 2002 15:14:59 +0700 From: Alexey Dokuchaev To: Terry Lambert Cc: arch@freebsd.org Subject: Re: Package system wishlist Message-ID: <20020710151459.A20701@regency.nsu.ru> References: <3D2BE142.E25CA9BC@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <3D2BE142.E25CA9BC@mindspring.com>; from tlambert2@mindspring.com on Wed, Jul 10, 2002 at 12:24:50AM -0700 X-Envelope-To: tlambert2@mindspring.com, arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Jul 10, 2002 at 12:24:50AM -0700, Terry Lambert wrote: > So, following Jordan's advice, what's on everyone's wishlist? > > Terry's Wishlist: > > o I don't want to download unnecessary information over a slow > link, because PacBell won't put a DSLM on the end of a fiber > optic cable in my area in order to service the 3000 apartments > full of Oracle and other big company engineers whose companies > would automatically pay them a nice monthly fee. Agreed. > > o I want to know how long it's going to take to download all > the package and package dependencies for what I've asked to > be installed This is kinda nice to have, but far not essential IMHO. > > o I want to answer all the questions at once, in a marathon > session, and then have everything "just work" afterwards > (front loading user configuration) True. I'll also add: o I don't want to less through Makefile in order to find out what possible options (-DWITH[OUT]_SOMETHING) a port might accept. The idea of using /usr/bin/dialog, like used in net/samba and graphics/mplayer-skins ports, seems neat to me. > > o I want to know that an individual part is good, if I can > download parts of a package (I don't care if this means > that the modern FTP/HTTP range approaches are used, or if > things are stored in sperate files) I'll probably won't care much. > > o I'd like there to be one thing to download, so that when > I get outside an area with cheap broadband communications, > I can still get the work done Sounds useful (at least for some of us). > > o I want to have a distinction between "necessary" and > "sufficient", so that if I have pbm 1.2.3 installed, I > don't end up with pbm 1.6.9 installed, too, if 1.2.3 > would have been sufficient Very, very true. I was really annoyed when devel/gettext-old changed their libintl suffix from .1 to 2. After patching some Makefiles, I gave up and ended up installing second one on top of another, followed by `pkgdb -F'. Gee! That's definitely something that I do not want to happen again. > > o I'd like "one click install" of packages from a web site > or a set of websites. This implies: > > o Cryptographic signatures > o A different file extension that's not already > supported by a browser or Apache > o Modifications to the default FreeBSD Apache to > set content transfer encoding for a binary file > type > o Modifications of the default "MIME Types" file > on FreeBSD to reference an installation tool for > the selected extension name > o A tool to do the installing Oh well, I don't care much. Moreover, making new ports/package system TOO smart might get really annoying (not to mention potential bugginess) for end-users/admins. > > o I want the option of having pretty progress bars that are > actually meaningful (e.g. "Time remaining: 00:22:31") That's like bells and whistles already. 8-) > > o I want to have a command that can tell me everything that > didn't come out of a package (this includes things that did > come out of a package, but were replaced by some cracker or > by mistake, etc.) Sounds OK. > > o I want "blind packages"; these are packages that are > depended upon by another package and not explicitly installed > by a user. When all the packages that depend on it are > removed (dependency count goes from 1->0), I want the system > to not_remove/automatically_remove/offer_to_remove the blind > package Sounds nice. "Offer-to-remove" seems to be the most reasonable. > > o I want to be able to remove system components, like "sendmail" > and "OpenSSH". Actually, I like the idea of "base system", so it does not belong to any port or package. And I never had anything against having sendmail, OpenSSH, or whatever in the base. Well, maybe with one exception of yp* tools. :-) :-) > > o Eventually, I want to be able to not install system components > in the first place (e.g. it should be possible to do a "PicoBSD" > with the standard tools). Well, maybe. > > o I want to know how much disk space I have, vs. how much it's > going to take to install something, so I can decide whether I > really want it or not. Seems not so useful, but it might be just me. ;-) > > o I want the "installed components" list to be accurate after an > initial system install ("/bin/ls" is a comonent). > > o I want the "installed components" list to be accurate after a > "make installworld". While on these, I'd like to mention that it would be really nice if, after installing the world, I could be sure that any file that was removed from installation (that is, it would not appear in a clean DESTDIR), could be removed from original (/) as well (since it's a left-over from previous world). > > o I want an "Add/Remove" software icon in the KDE control panel, > just like Windows has. I assume you are kidding us on this one, Terry, right?? ./danfe To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 3:14:46 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 069FF37B400; Wed, 10 Jul 2002 03:14:45 -0700 (PDT) Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6972943E54; Wed, 10 Jul 2002 03:14:44 -0700 (PDT) (envelope-from des@ofug.org) Received: by flood.ping.uio.no (Postfix, from userid 2602) id 7D93D534A; Wed, 10 Jul 2002 12:14:43 +0200 (CEST) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Nik Clayton Cc: Wes Peters , Dan Moschuk , arch@freebsd.org Subject: Re: Package system flaws? References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> <20020707145116.GC5610@canyon.nothing-going-on.org> From: Dag-Erling Smorgrav Date: 10 Jul 2002 12:14:42 +0200 In-Reply-To: <20020707145116.GC5610@canyon.nothing-going-on.org> Message-ID: Lines: 8 User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.2 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Nik Clayton writes: > gzipped UFS filesystem images that can be mounted on a vn/md device. . . That would be the ideal solution... DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 3:18:46 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3858737B401 for ; Wed, 10 Jul 2002 03:18:42 -0700 (PDT) Received: from mailout09.sul.t-online.com (mailout09.sul.t-online.com [194.25.134.84]) by mx1.FreeBSD.org (Postfix) with ESMTP id C520243E31 for ; Wed, 10 Jul 2002 03:18:37 -0700 (PDT) (envelope-from nicolas@dauerreden.de) Received: from fwd04.sul.t-online.de by mailout09.sul.t-online.com with smtp id 17SEHc-000787-0C; Wed, 10 Jul 2002 12:01:16 +0200 Received: from pc5.abc (520067998749-0001@[217.233.115.221]) by fmrl04.sul.t-online.com with esmtp id 17SEHT-1lXmxkC; Wed, 10 Jul 2002 12:01:07 +0200 Received: from pc5.abc (localhost.abc [127.0.0.1]) by pc5.abc (8.12.3/8.12.3) with ESMTP id g6AA16XP079075 for ; Wed, 10 Jul 2002 12:01:06 +0200 (CEST) (envelope-from nicolas@pc5.abc) Received: (from nicolas@localhost) by pc5.abc (8.12.3/8.12.3/Submit) id g6AA165I079074 for arch@FreeBSD.ORG; Wed, 10 Jul 2002 12:01:06 +0200 (CEST) Date: Wed, 10 Jul 2002 12:01:06 +0200 From: Nicolas Rachinsky To: arch@FreeBSD.ORG Subject: Re: Package system wishlist Message-ID: <20020710100106.GJ42073@narr.dauerreden.de> Mail-Followup-To: arch@FreeBSD.ORG References: <3D2BE142.E25CA9BC@mindspring.com> <20020710151459.A20701@regency.nsu.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020710151459.A20701@regency.nsu.ru> User-Agent: Mutt/1.3.28i X-Powered-by: FreeBSD X-Homepage: http://www.rachinsky.de X-PGP-Keyid: C11ABC0E X-PGP-Fingerprint: 19DB 8392 8FE0 814A 7362 EEBD A53B 526A C11A BC0E X-PGP-Key: http://www.rachinsky.de/nicolas/nicolas_rachinsky.asc X-Sender: 520067998749-0001@t-dialin.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * Alexey Dokuchaev [2002-07-10 15:14 +0700]: > On Wed, Jul 10, 2002 at 12:24:50AM -0700, Terry Lambert wrote: > > o I want to answer all the questions at once, in a marathon > > session, and then have everything "just work" afterwards > > (front loading user configuration) > > True. I'll also add: > > o I don't want to less through Makefile in order to find out what > possible options (-DWITH[OUT]_SOMETHING) a port might accept. > The idea of using /usr/bin/dialog, like used in net/samba and > graphics/mplayer-skins ports, seems neat to me. But it should be possible to install (with any set of options) without having to use /usr/bin/dialog. This does not seem to be the case with samba (at least the last time I did it, I was unable to find ist). > > o I want the "installed components" list to be accurate after an > > initial system install ("/bin/ls" is a comonent). > > > > o I want the "installed components" list to be accurate after a > > "make installworld". > > While on these, I'd like to mention that it would be really nice if, > after installing the world, I could be sure that any file that was > removed from installation (that is, it would not appear in a clean > DESTDIR), could be removed from original (/) as well (since it's a > left-over from previous world). It would be nice if, after pkg_delete -f and reinstall of a package, the dependency lists can be complete. Nicolas To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 3:38:21 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 99DB437B401; Wed, 10 Jul 2002 03:38:18 -0700 (PDT) Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2710643E3B; Wed, 10 Jul 2002 03:38:18 -0700 (PDT) (envelope-from des@ofug.org) Received: by flood.ping.uio.no (Postfix, from userid 2602) id 48BC9534B; Wed, 10 Jul 2002 12:38:16 +0200 (CEST) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: rsi@panix.com Cc: "Artem Tepponen" , "Doug Barton" , "Garrett Wollman" , , Subject: Re: Package system flaws? References: <5235EF9BAE6B7F4CB3735789EEF73B296E614A@turtle.egar.egartech.com> <200207092230.g69MUZu18917@panix1.panix.com> From: Dag-Erling Smorgrav Date: 10 Jul 2002 12:38:15 +0200 In-Reply-To: <200207092230.g69MUZu18917@panix1.panix.com> Message-ID: Lines: 10 User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.2 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Rajappa Iyer writes: > The only problem with the ports approach is that you have to have the > ports tree locally installed. # pkg_add -r porteasy # man porteasy DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 4:54:18 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3A04737B400 for ; Wed, 10 Jul 2002 04:54:15 -0700 (PDT) Received: from spielberg.vip.uk.com (spielberg.vip.uk.com [194.176.218.9]) by mx1.FreeBSD.org (Postfix) with ESMTP id A611343E4A for ; Wed, 10 Jul 2002 04:54:10 -0700 (PDT) (envelope-from mtm98@linuxdriven.net) Received: from modem-95-46-60-62.vip.uk.com ([62.60.46.95] helo=uriel.fakedomain.net) by spielberg.vip.uk.com with esmtp (Exim 3.22 #1) id 17SFQZ-0002Fq-00 for arch@freebsd.org; Wed, 10 Jul 2002 12:14:39 +0100 Received: from uriel.fakedomain.net (mtm98@localhost [127.0.0.1]) by uriel.fakedomain.net (8.12.5/8.12.3) with ESMTP id g6ABrftx014108 for ; Wed, 10 Jul 2002 12:53:53 +0100 (BST) (envelope-from mtm98@uriel.fakedomain.net) Received: (from mtm98@localhost) by uriel.fakedomain.net (8.12.5/8.12.3/Submit) id g6ABrRT5014107 for arch@freebsd.org; Wed, 10 Jul 2002 12:53:27 +0100 (BST) (envelope-from mtm98) Date: Wed, 10 Jul 2002 12:53:26 +0100 From: Michael McGoldrick To: arch@freebsd.org Subject: Re: Package system flaws? Message-ID: <20020710115326.GA8843@uriel.fakedomain.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG As a dialup user with limited space, having the package system tell me how much it needed to download in advance would be very handy, as would automatic 'regetting' if my connection goes down. As regards ports 'plists', the way the Stow program does things is interesting. It simply builds each port under it's own directory, then symlinks it in from the relevant directories under $PREFIX. -- Michael McGoldrick: mmcgoldrick@linuxdriven.net To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 7:19:48 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B554937B400 for ; Wed, 10 Jul 2002 07:19:46 -0700 (PDT) Received: from scoobysnax.jaded.net (d141-7-230.home.cgocable.net [24.141.7.230]) by mx1.FreeBSD.org (Postfix) with ESMTP id 53C9F43E54 for ; Wed, 10 Jul 2002 07:19:45 -0700 (PDT) (envelope-from dan@scoobysnax.jaded.net) Received: from scoobysnax.jaded.net (localhost [127.0.0.1]) by scoobysnax.jaded.net (8.12.3/8.12.3) with ESMTP id g6AEJiub001541 for ; Wed, 10 Jul 2002 10:19:44 -0400 (EDT) (envelope-from dan@scoobysnax.jaded.net) Received: (from dan@localhost) by scoobysnax.jaded.net (8.12.3/8.12.3/Submit) id g6AEJheD001540 for arch@freebsd.org; Wed, 10 Jul 2002 10:19:43 -0400 (EDT) Date: Wed, 10 Jul 2002 10:19:43 -0400 From: Dan Moschuk To: arch@freebsd.org Subject: Package format proposal upcoming Message-ID: <20020710141943.GA1157@scoobysnax.jaded.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Thanks to everyone for offering up their ideas. There is certainly no shortage of opinion around here. :-) After I wade through the concoction of package format wishlists and implementation ideas, I'll be drafting up a proposal for people to berate^W comment on. Bearing work obligations, assorted other real life obligations and disasters like tornados, earthquakes, volcanic eruptions and godzilla coming out of the water to stomp on the power plant (who built this city anyway?) I hope to have something by the end of the month. Cheers, -Dan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 7:29:57 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0620237B400 for ; Wed, 10 Jul 2002 07:29:56 -0700 (PDT) Received: from scoobysnax.jaded.net (d141-7-230.home.cgocable.net [24.141.7.230]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7419943E52 for ; Wed, 10 Jul 2002 07:29:55 -0700 (PDT) (envelope-from dan@scoobysnax.jaded.net) Received: from scoobysnax.jaded.net (localhost [127.0.0.1]) by scoobysnax.jaded.net (8.12.3/8.12.3) with ESMTP id g6AETsub001617 for ; Wed, 10 Jul 2002 10:29:54 -0400 (EDT) (envelope-from dan@scoobysnax.jaded.net) Received: (from dan@localhost) by scoobysnax.jaded.net (8.12.3/8.12.3/Submit) id g6AETsKG001616 for arch@FreeBSD.org; Wed, 10 Jul 2002 10:29:54 -0400 (EDT) Date: Wed, 10 Jul 2002 10:29:54 -0400 From: Dan Moschuk To: arch@FreeBSD.org Subject: Re: Package format proposal upcoming Message-ID: <20020710142954.GB1157@scoobysnax.jaded.net> References: <20020710141943.GA1157@scoobysnax.jaded.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020710141943.GA1157@scoobysnax.jaded.net> User-Agent: Mutt/1.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG | Thanks to everyone for offering up their ideas. There is certainly no | shortage of opinion around here. :-) | | After I wade through the concoction of package format wishlists and | implementation ideas, I'll be drafting up a proposal for people to berate^W | comment on. Bearing work obligations, assorted other real life | obligations and disasters like tornados, earthquakes, volcanic eruptions | and godzilla coming out of the water to stomp on the power plant (who | built this city anyway?) I hope to have something by the end of the month. On a side note, is there documentation into the libh package format anywhere? No sense reinventing the wheel IF it can do everything that people seem to want. And on a side note to that side note, libh needs a new name. How about libsheep? ;) -Dan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 7:57:39 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A8F6737B400; Wed, 10 Jul 2002 07:57:35 -0700 (PDT) Received: from prism.flugsvamp.com (66-191-112-47.mad.wi.charter.com [66.191.112.47]) by mx1.FreeBSD.org (Postfix) with ESMTP id CB74E43E09; Wed, 10 Jul 2002 07:57:34 -0700 (PDT) (envelope-from jlemon@flugsvamp.com) Received: (from jlemon@localhost) by prism.flugsvamp.com (8.11.6/8.11.6) id g6AEtjW11923; Wed, 10 Jul 2002 09:55:45 -0500 (CDT) (envelope-from jlemon) Date: Wed, 10 Jul 2002 09:55:45 -0500 From: Jonathan Lemon To: Jordan K Hubbard Cc: Dan Nelson , Archie Cobbs , Dan Moschuk , Dag-Erling Smorgrav , Wes Peters , arch@FreeBSD.ORG Subject: Re: Package system flaws? Message-ID: <20020710095545.C65393@prism.flugsvamp.com> References: <20020710033154.GD8625@dan.emsphone.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0pre2i In-Reply-To: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, Jul 09, 2002 at 09:10:42PM -0700, Jordan K Hubbard wrote: > We've already seen considerable debate over random access capabilities > and being able to get at the "dictionary" portion of a package either by > sticking it at the front or putting it in another file, so I won't go > into that all over again, but I will bring up something that nobody > seems to have touched on yet: Fat packages. Why does this remind me of the pain I had to go through where I had to download a .pkg to my netcom account, and write my own 'thinner' program to strip out the unwanted architecture cruft before actually downloading the new .pkg over a 9600b link to my NeXTSTEP/i386 machine? -- Jonathan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 7:59:31 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D79F937B400 for ; Wed, 10 Jul 2002 07:59:28 -0700 (PDT) Received: from spqr.osg.gov.bc.ca (spqr.osg.gov.bc.ca [142.32.102.24]) by mx1.FreeBSD.org (Postfix) with ESMTP id 69E9D43E09 for ; Wed, 10 Jul 2002 07:59:28 -0700 (PDT) (envelope-from Cy.Schubert@osg.gov.bc.ca) Received: from passer.osg.gov.bc.ca (passer.osg.gov.bc.ca [142.32.110.29]) by spqr.osg.gov.bc.ca (Postfix) with ESMTP id 21F099F133; Wed, 10 Jul 2002 07:59:28 -0700 (PDT) Received: from cwsys.cwsent.com (cwsys2 [10.1.2.1]) by passer.osg.gov.bc.ca (8.12.5/8.12.3) with ESMTP id g6AExROX051627; Wed, 10 Jul 2002 07:59:27 -0700 (PDT) (envelope-from cy@cwsent.com) Received: from cwsys (localhost [127.0.0.1]) by cwsys.cwsent.com (8.12.5/8.12.3) with ESMTP id g6AExQfP034695; Wed, 10 Jul 2002 07:59:26 -0700 (PDT) (envelope-from cy@cwsys.cwsent.com) Message-Id: <200207101459.g6AExQfP034695@cwsys.cwsent.com> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 Reply-To: Cy Schubert - CITS Open Systems Group From: Cy Schubert - CITS Open Systems Group X-os: FreeBSD X-Sender: cy@cwsent.com To: Terry Lambert Cc: arch@FreeBSD.ORG Subject: Re: Package system wishlist In-Reply-To: Message from Terry Lambert of "Wed, 10 Jul 2002 00:24:50 PDT." <3D2BE142.E25CA9BC@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 10 Jul 2002 07:59:26 -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <3D2BE142.E25CA9BC@mindspring.com>, Terry Lambert writes: > So, following Jordan's advice, what's on everyone's wishlist? > > Terry's Wishlist: [...] + Cy's Wishlist: o Optional installation of sources. RH's SRPM's is a very poor example of this. A better example would be what IBM does to install JES/2 on their MVS system, e.g. an OpenSSH package might contain source in addition to binaries. The sources would be installed in /usr/src while the binaries would be installed in /usr/bin, sbin.... o Files replaced by a package backed up in case of package removal o Check option: Tell me what it will do without doing it o Group option: Install prerequisites o Groupextend option: Install postrequisites, e.g. dependent packages and patches o Ability to install my own packages on top of packages and patches, I like to call them USERMODS. o The package system should be independent of the compression tool used. In the future new compression algorithms and tools will be developed. The package system should be flexible enough to not care how its files are compressed or packaged. o The ability to export and import the package database (currently to clone systems, I rsync /usr/local, /usr/X11R6, and /var/db/pkg to a new system I am installing, this saves many hours of work). > o I want to be able to remove system components, like "sendmail" > and "OpenSSH". Ideally everything should install as a package, however that would create a lot of extra work for us developers. I have yet to think of a painless way to do this. -- Cheers, Phone: 250-387-8437 Cy Schubert Fax: 250-387-5766 Team Leader, Sun/Alpha Team Email: Cy.Schubert@osg.gov.bc.ca Open Systems Group, CITS Ministry of Management Services Province of BC FreeBSD UNIX: cy@FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 8:33:42 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CC49837B401 for ; Wed, 10 Jul 2002 08:33:35 -0700 (PDT) Received: from mail.nsu.ru (mx.nsu.ru [193.124.215.71]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1F29543E52 for ; Wed, 10 Jul 2002 08:33:29 -0700 (PDT) (envelope-from danfe@regency.nsu.ru) Received: from drweb by mail.nsu.ru with drweb-scanned (Exim 3.20 #1) id 17SJSm-0000A7-00; Wed, 10 Jul 2002 22:33:08 +0700 Received: from regency.nsu.ru ([193.124.210.26]) by mail.nsu.ru with esmtp (Exim 3.20 #1) id 17SJSm-00009v-00; Wed, 10 Jul 2002 22:33:08 +0700 Received: from regency.nsu.ru (localhost [127.0.0.1]) by regency.nsu.ru (8.12.4/8.12.4) with ESMTP id g6AFXAnP075337; Wed, 10 Jul 2002 22:33:10 +0700 (NOVST) (envelope-from danfe@regency.nsu.ru) Received: (from danfe@localhost) by regency.nsu.ru (8.12.4/8.12.4/Submit) id g6AFX927075312; Wed, 10 Jul 2002 22:33:09 +0700 (NOVST) Date: Wed, 10 Jul 2002 22:33:09 +0700 From: Alexey Dokuchaev To: Cy Schubert - CITS Open Systems Group Cc: Terry Lambert , arch@freebsd.org Subject: Re: Package system wishlist Message-ID: <20020710223309.A69788@regency.nsu.ru> References: <200207101459.g6AExQfP034695@cwsys.cwsent.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <200207101459.g6AExQfP034695@cwsys.cwsent.com>; from Cy.Schubert@uumail.gov.bc.ca on Wed, Jul 10, 2002 at 07:59:26AM -0700 X-Envelope-To: Cy.Schubert@uumail.gov.bc.ca, tlambert2@mindspring.com, arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Jul 10, 2002 at 07:59:26AM -0700, Cy Schubert - CITS Open Systems Group wrote: > > o Optional installation of sources. RH's SRPM's is a very poor > example of this. A better example would be what IBM does to > install JES/2 on their MVS system, e.g. an OpenSSH package might > contain source in addition to binaries. The sources would be > installed in /usr/src while the binaries would be installed > in /usr/bin, sbin.... Err, you mean sources of base or what? Otherwise, installing anything other than base in /usr/{bin,sbin,src} seems kinda, uhm, wrong, not to say more. > > o Files replaced by a package backed up in case of package removal This is probably a good point, but IMHO it is better to make sure (at least try to) that such thing does not [regularly?] happen. > > o Check option: Tell me what it will do without doing it > > o Group option: Install prerequisites > > o Groupextend option: Install postrequisites, e.g. dependent > packages and patches > > o Ability to install my own packages on top of packages and > patches, I like to call them USERMODS. This is also something that currently is not implemented (considered? designed?) very well. Ideally, ports should go in /usr/pkg or /opt, while local user's bits should go in /usr/local. > > o The package system should be independent of the compression tool > used. In the future new compression algorithms and tools will > be developed. The package system should be flexible enough to > not care how its files are compressed or packaged. I believe it's quite easy to implement a some kind of wrapper to provide this functionality. > > o The ability to export and import the package database (currently > to clone systems, I rsync /usr/local, /usr/X11R6, and /var/db/pkg > to a new system I am installing, this saves many hours of work). Sounds good to me. > > > o I want to be able to remove system components, like "sendmail" > > and "OpenSSH". > > Ideally everything should install as a package, however that would Currently, I cannot agree with this. I had enough head ache in the past dealing with packages of "compatibility symlinks", man pages, and so on, which seems overly ridiculous to me. I don't consider this worthwhile. Generally, I prefer base as monolithic collection of bits. ./danfe To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 8:51:33 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E311C37B400 for ; Wed, 10 Jul 2002 08:51:29 -0700 (PDT) Received: from infinitive.futureperfectcorporation.com (infinitive.futureperfectcorporation.com [196.25.137.68]) by mx1.FreeBSD.org (Postfix) with SMTP id 32FC943E52 for ; Wed, 10 Jul 2002 08:51:26 -0700 (PDT) (envelope-from nbm@gerund.futureperfectcorporation.com) Received: (qmail 20610 invoked by uid 0); 10 Jul 2002 15:51:21 -0000 Received: from gerund.futureperfectcorporation.com (196.25.137.65) by infinitive.futureperfectcorporation.com with SMTP; 10 Jul 2002 15:51:21 -0000 Received: (qmail 70267 invoked by uid 1001); 10 Jul 2002 15:52:06 -0000 Date: Wed, 10 Jul 2002 17:52:06 +0200 From: Neil Blakey-Milner To: Cy Schubert - CITS Open Systems Group Cc: arch@FreeBSD.ORG Subject: Re: Package system wishlist Message-ID: <20020710155206.GA70184@mithrandr.moria.org> References: <3D2BE142.E25CA9BC@mindspring.com> <200207101459.g6AExQfP034695@cwsys.cwsent.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200207101459.g6AExQfP034695@cwsys.cwsent.com> User-Agent: Mutt/1.3.27i Organization: iTouch Labs X-Operating-System: FreeBSD 4.3-RELEASE i386 X-URL: http://mithrandr.moria.org/nbm/ Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed 2002-07-10 (07:59), Cy Schubert - CITS Open Systems Group wrote: > In message <3D2BE142.E25CA9BC@mindspring.com>, Terry Lambert writes: > > So, following Jordan's advice, what's on everyone's wishlist? > > > > Terry's Wishlist: > [...] > > + Cy's Wishlist: > > o Optional installation of sources. RH's SRPM's is a very poor > example of this. A better example would be what IBM does to > install JES/2 on their MVS system, e.g. an OpenSSH package might > contain source in addition to binaries. The sources would be > installed in /usr/src while the binaries would be installed > in /usr/bin, sbin.... Not in the same file, please. We also need to think about people who want to use conventional tools to move packages about. > o The package system should be independent of the compression tool > used. In the future new compression algorithms and tools will > be developed. The package system should be flexible enough to > not care how its files are compressed or packaged. Debian's dpkg is designed that way - dpkg-deb is the tool that understands Debian packages, while dpkg does the actual package management logic. Neil -- Neil Blakey-Milner nbm@mithrandr.moria.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 9:46:43 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 14DD637B400 for ; Wed, 10 Jul 2002 09:46:41 -0700 (PDT) Received: from w250.z064001178.sjc-ca.dsl.cnc.net (adsl-66.218.45.239.dslextreme.com [66.218.45.239]) by mx1.FreeBSD.org (Postfix) with SMTP id 6B43843E4A for ; Wed, 10 Jul 2002 09:46:40 -0700 (PDT) (envelope-from jos@catnook.com) Received: (qmail 43551 invoked by uid 1000); 10 Jul 2002 16:47:01 -0000 Date: Wed, 10 Jul 2002 09:46:39 -0701 From: Jos Backus To: arch@FreeBSD.ORG Subject: Re: Package system flaws? Message-ID: <20020710164701.GA32702@lizzy.catnook.com> Reply-To: jos@catnook.com Mail-Followup-To: arch@FreeBSD.ORG References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> <20020707145116.GC5610@canyon.nothing-going-on.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.1i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Jul 10, 2002 at 12:14:42PM +0200, Dag-Erling Smorgrav wrote: > Nik Clayton writes: > > gzipped UFS filesystem images that can be mounted on a vn/md device. . . > > That would be the ideal solution... Isn't this something OS X does already? -- Jos Backus _/ _/_/_/ Santa Clara, CA _/ _/ _/ _/ _/_/_/ _/ _/ _/ _/ jos@catnook.com _/_/ _/_/_/ require 'std/disclaimer' To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 10: 0:54 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 620E837B401 for ; Wed, 10 Jul 2002 10:00:49 -0700 (PDT) Received: from mail.rpi.edu (mail.rpi.edu [128.113.22.40]) by mx1.FreeBSD.org (Postfix) with ESMTP id 176FE43E4A for ; Wed, 10 Jul 2002 10:00:49 -0700 (PDT) (envelope-from drosih@rpi.edu) Received: from [128.113.24.47] (gilead.netel.rpi.edu [128.113.24.47]) by mail.rpi.edu (8.12.1/8.12.1) with ESMTP id g6AH0eB2157868; Wed, 10 Jul 2002 13:00:40 -0400 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: <20020710095545.C65393@prism.flugsvamp.com> References: <20020710033154.GD8625@dan.emsphone.com> <20020710095545.C65393@prism.flugsvamp.com> Date: Wed, 10 Jul 2002 13:00:39 -0400 To: Jonathan Lemon , Jordan K Hubbard From: Garance A Drosihn Subject: Re: Package system flaws? Cc: arch@FreeBSD.ORG Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Scanned-By: MIMEDefang 2.3 (www dot roaringpenguin dot com slash mimedefang) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG At 9:55 AM -0500 7/10/02, Jonathan Lemon wrote: >On Tue, Jul 09, 2002 at 09:10:42PM -0700, Jordan K Hubbard wrote: > > ..., but I will bring up something that nobody seems to have > > touched on yet: Fat packages. > >Why does this remind me of the pain I had to go through where I >had to download a .pkg to my netcom account, and write my own >'thinner' program to strip out the unwanted architecture cruft >before actually downloading the new .pkg over a 9600b link to >my NeXTSTEP/i386 machine? You didn't have to do that with *my* NeXTSTEP packages... :-) I always provided multiple files. Four single-architecture packages, and one fat package. If you needed to download two separate architectures, the four-architecture package was about the same size as two single-architecture packages (iirc). -- Garance Alistair Drosehn = gad@gilead.netel.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 11:30:19 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4807937B400; Wed, 10 Jul 2002 11:30:03 -0700 (PDT) Received: from InterJet.dellroad.org (adsl-63-194-81-26.dsl.snfc21.pacbell.net [63.194.81.26]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7D79643E3B; Wed, 10 Jul 2002 11:30:02 -0700 (PDT) (envelope-from archie@dellroad.org) Received: from arch20m.dellroad.org (arch20m.dellroad.org [10.1.1.20]) by InterJet.dellroad.org (8.9.1a/8.9.1) with ESMTP id LAA68895; Wed, 10 Jul 2002 11:15:46 -0700 (PDT) Received: (from archie@localhost) by arch20m.dellroad.org (8.11.6/8.11.6) id g6AIFDm28655; Wed, 10 Jul 2002 11:15:13 -0700 (PDT) (envelope-from archie) From: Archie Cobbs Message-Id: <200207101815.g6AIFDm28655@arch20m.dellroad.org> Subject: Re: Timeout and SMP race In-Reply-To: "from John Baldwin at Jul 10, 2002 01:28:09 pm" To: John Baldwin Date: Wed, 10 Jul 2002 11:15:13 -0700 (PDT) Cc: davidx@viasoft.com.cn, freebsd-arch@FreeBSD.org, julian@elischer.org X-Mailer: ELM [version 2.4ME+ PL88 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG [ NOTE: I'm moving this discussion to freebsd-arch@freebsd.org ] John Baldwin writes: > > What do you think of the idea of letting the timer code (optionally) > > handle all the locking and race conditions? > > I'm not sure it can in a clean fashion since of the few cases I've known > so far each client needs a customized solution. I am open to ideas though. > I'm also open to some redesign of how callouts work to begin with (maybe > using other threads than the one running softclock() to actually execute > callout handlers, etc.). FWIW, here is an API I've used before. This handles all race conditions and the 'other thread' question. struct timer_event; /* opaque structure */ typedef struct timer_event *timer_handle_t; /* caller's timer "handle" */ typedef void timer_func_t(void *arg); /* timeout function type */ /* flags for timer_start() */ #define TIMER_RECURRING 0x0001 /* timer is recurring */ #define TIMER_OWN_THREAD 0x0002 /* handle in separate thread */ extern int timer_start(timer_handle_t *handlep, mutex_t *mutexp, timer_func_t tfunc, void *arg, u_int delay, int flags); extern void timer_cancel(timer_handle_t *handlep); extern int timer_remaining(timer_handle_t handle, u_int *delayp); static inline int timer_isrunning(timer_handle_t handle) { return (handle != NULL); } Semantics: 1. The caller supplies a pointer to the 'handle', which must initially be NULL. The handle != NULL if and only if the timer is running. 2. timer_cancel() guarantees that tfunc() will not be called subsequently 3. *handlep is set to NULL by timer_cancel() and by the timer expiring. So when *handlep is NULL when tfunc() is invoked (unless TIMER_RECURRING). 4. Calling timer_start() or timer_stop() from within tfunc() is OK. 5. If TIMER_RECURRING, timer started again before calling tfunc() 6. If TIMER_OWN_THREAD, timer runs in a newly created thread (rather than the timer service thread), which means that tfunc() may sleep or be canceled. If tfunc() sleeps or the thread is canceled but TIMER_OWN_THREAD was not set -> panic. 7. If mutexp != NULL, *mutexp is acquired before calling tfunc() and released after it returns. Items 1, and 2 are guaranteed only if mutexp != NULL and the caller acquires *mutexp before any calls to timer_start() or timer_cancel() (you would normally be doing this anyway). Errors: - timer_start() returns EBUSY if *handlep != NULL - timer_remaining() returns ESRCH if handle != NULL The model is: you have some object that has an associated lock and one or more associated timers. The object is to be locked whenever you muck with it (including when you start, stop, or handle a timer): struct foobar { struct lock mutex; timer_handle_t timer1; timer_handle_t timer2; ... }; Then all calls to the timer_* routines are "well behaved" and the timeout thread caling tfunc() never races with any other thread that may be stopping or starting the timer, or destroying the object. E.g., to destroy the object, the following suffices: void foobar_destroy(struct foobar *f) { mutex_lock(&f->mutex); timer_cancel(&f->timer1); timer_cancel(&f->timer2); mutex_unlock(&f->mutex); free(f); } The only remaining complexity for the caller is that if you have any TIMER_OWN_THREAD handlers which unlock the object (e.g., in order to go to sleep), then you need to reference count the object and have a FOOBAR_INVALID flag. If you are working under a different model then this API may not be appropriate, but at least in my multi-threading experience this model is very typical. -Archie __________________________________________________________________________ Archie Cobbs * Packet Design * http://www.packetdesign.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 12:56:23 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E11FB37B400; Wed, 10 Jul 2002 12:56:19 -0700 (PDT) Received: from falcon.mail.pas.earthlink.net (falcon.mail.pas.earthlink.net [207.217.120.74]) by mx1.FreeBSD.org (Postfix) with ESMTP id 897D443ED8; Wed, 10 Jul 2002 12:55:58 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0451.cvx22-bradley.dialup.earthlink.net ([209.179.199.196] helo=mindspring.com) by falcon.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17SNYh-0004Gb-00; Wed, 10 Jul 2002 12:55:31 -0700 Message-ID: <3D2C9107.C11F9F09@mindspring.com> Date: Wed, 10 Jul 2002 12:54:47 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Dag-Erling Smorgrav Cc: Nik Clayton , Wes Peters , Dan Moschuk , arch@freebsd.org Subject: Re: Package system flaws? References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> <20020707145116.GC5610@canyon.nothing-going-on.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Dag-Erling Smorgrav wrote: > Nik Clayton writes: > > gzipped UFS filesystem images that can be mounted on a vn/md device. . . > > That would be the ideal solution... But how would you implement it, given that compression is not size deterministic? -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 12:58: 6 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9131137B409 for ; Wed, 10 Jul 2002 12:57:58 -0700 (PDT) Received: from mail3.ksc.th.com (mail3.ksc.th.com [203.155.0.234]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9712843E5E for ; Wed, 10 Jul 2002 12:57:51 -0700 (PDT) (envelope-from easytoberich01@yahoo.com) Received: from ksc.th.com ([203.156.15.36]) by mail3.ksc.th.com (8.12.1/8.12.0) with SMTP id g6AJjxHa022592 for ; Thu, 11 Jul 2002 02:57:48 +0700 Message-Id: <200207101957.g6AJjxHa022592@mail3.ksc.th.com> Date: Thu, 11 Jul 2002 02:59:54 To: freebsd-arch@FreeBSD.org From: easytoberich01@yahoo.com (international e-business) Subject: ÊÓËÃѺ¼Ùé·Õèµéͧ¡ÒÃâÍ¡ÒÊ㹡ÒÃà»ÅÕè¹á»Å§ªÕÇÔµ Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG !!!!! Part-Time Job!! ÊÓËÃѺ¹Ñ¡àÃÕ¹ ¹Ñ¡ÈÖ¡ÉÒ áÅмÙé·Ó§Ò¹»ÃÐ¨Ó ¤Ø³µéͧ¡ÒçҹẺ¹ÕéºéÒ§äËÁ…?? -§Ò¹ parttime ·Ó§Ò¹·ÕèºéÒ¹ä´é ¶éҤسãªé Internet à»ç¹ -·Ó§Ò¹à¾Õ§ÇѹÅÐ 2-3 ªÁ. -ÃÒÂä´é 5,000 – 15,000 ºÒ· ¶éҤسà»ç¹¤¹Ë¹Ö觷Õè·Ó§Ò¹»ÃШÓËÃ×ÍÂѧäÁèÁÕ§Ò¹·Ó ¹Ñ¡ÈÖ¡ÉÒ·Õè¡ÓÅѧÈÖ¡ÉÒÍÂÙè ¼ÙéÇèÒ§§Ò¹ ËÃ×ͼÙé·ÕèÂѧ¾ÍÁÕàÇÅÒÇèÒ§¨Ò¡§Ò¹»ÃÐ¨Ó ÁդسÊÁºÑµÔàº×éͧµé¹´Ñ§¹Õé 1. ÁÕ·Ñȹ¤µÔ·Õè´Õ 2. ¾ÃéÍÁ·Õè¨ÐàÃÕ¹ÃÙé à¹×èͧ¨Ò¡à»ç¹ÃкºãËÁè¨Ö§µéͧãËéÁÕ¡ÒÃͺÃÁãËéµÒÁ¤ÇÒÁàËÁÒÐÊÁ 3. µéͧ¡Ò÷Õè¨Ð·Ó§Ò¹ÍÂèÒ§¨ÃÔ§¨Ñ§ ÍÂÒ¡·Õè¨Ðà»ÅÕ蹰ҹзҧ¡ÒÃà§Ô¹¢Í§µ¹àͧ áÅÐÍÂÒ¡ÁÕÃÒÂä´é¨Ò¡¡Ò÷ӧҹµÃ§¹Õé¨ÃÔ§æ ·Ø¡ÍÂèÒ§à»ç¹ä»ä´é ã¹ http://www.geocities.com/getchances2000/ ÍÂèÒ !…………….. à»ç¹á¤èà¾Õ§¤¹·Õè¹Ñè§ÃÍâÍ¡ÒÊ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 13:35:45 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4D22437B406 for ; Wed, 10 Jul 2002 13:35:43 -0700 (PDT) Received: from harrier.mail.pas.earthlink.net (harrier.mail.pas.earthlink.net [207.217.120.12]) by mx1.FreeBSD.org (Postfix) with ESMTP id 57E2943E3B for ; Wed, 10 Jul 2002 13:35:42 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0451.cvx22-bradley.dialup.earthlink.net ([209.179.199.196] helo=mindspring.com) by harrier.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17SOBC-0004C1-00; Wed, 10 Jul 2002 13:35:18 -0700 Message-ID: <3D2C9A5C.B5701103@mindspring.com> Date: Wed, 10 Jul 2002 13:34:36 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Alexey Dokuchaev Cc: Cy Schubert - CITS Open Systems Group , arch@freebsd.org Subject: Re: Package system wishlist References: <200207101459.g6AExQfP034695@cwsys.cwsent.com> <20020710223309.A69788@regency.nsu.ru> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Are you sure you don't work for Microsoft or Linux? ;^). The way "brain-storming" works is to capture all ideas, whether you agree with them or not. After you capture all the ideas, you then switch from "brain-storming" to "collation and analysis". So it's important to not comment on other people's ideas right now, but to offer your own. Alexey Dokuchaev wrote: > > > o I want to be able to remove system components, like "sendmail" > > > and "OpenSSH". > > > > Ideally everything should install as a package, however that would > > Currently, I cannot agree with this. I had enough head ache in the past > dealing with packages of "compatibility symlinks", man pages, and so on, > which seems overly ridiculous to me. I don't consider this worthwhile. > Generally, I prefer base as monolithic collection of bits. It is a prerequisite for: o Ability to do binary upgrades of the base system in order to automatically (e.g. via cron) obtain, and optionally install, security and other fixes. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 13:53:24 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8ADBF37B400; Wed, 10 Jul 2002 13:53:21 -0700 (PDT) Received: from swan.mail.pas.earthlink.net (swan.mail.pas.earthlink.net [207.217.120.123]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3AD4F43E4A; Wed, 10 Jul 2002 13:53:21 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0451.cvx22-bradley.dialup.earthlink.net ([209.179.199.196] helo=mindspring.com) by swan.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17SOST-00017u-00; Wed, 10 Jul 2002 13:53:10 -0700 Message-ID: <3D2C9E8A.8D7657E7@mindspring.com> Date: Wed, 10 Jul 2002 13:52:26 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Archie Cobbs Cc: John Baldwin , davidx@viasoft.com.cn, freebsd-arch@FreeBSD.org, julian@elischer.org Subject: Re: Timeout and SMP race References: <200207101815.g6AIFDm28655@arch20m.dellroad.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Archie Cobbs wrote: > [ NOTE: I'm moving this discussion to freebsd-arch@freebsd.org ] > > FWIW, here is an API I've used before. This handles all race > conditions and the 'other thread' question. I like this timer API. You need to add a timer_modify_interval (or something similar), so that intervals can be changed without having to destroy and recreate the timer, though, to avoid another potential race condition. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 14: 5:20 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7FFF037B401 for ; Wed, 10 Jul 2002 14:05:18 -0700 (PDT) Received: from postfix2-1.free.fr (postfix2-1.free.fr [213.228.0.9]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5506143E64 for ; Wed, 10 Jul 2002 14:05:17 -0700 (PDT) (envelope-from rsidd@lpt.ens.fr) Received: from bluerondo.a.la.turk (nas-cbv-4-62-147-143-212.dial.proxad.net [62.147.143.212]) by postfix2-1.free.fr (Postfix) with ESMTP id 9C42715 for ; Wed, 10 Jul 2002 23:05:14 +0200 (CEST) Received: (qmail 775 invoked by uid 1001); 10 Jul 2002 21:05:09 -0000 Date: Wed, 10 Jul 2002 23:05:09 +0200 From: Rahul Siddharthan To: Terry Lambert Cc: Alexey Dokuchaev , Cy Schubert - CITS Open Systems Group , arch@freebsd.org Subject: Re: Package system wishlist Message-ID: <20020710210509.GA686@lpt.ens.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3D2C9A5C.B5701103@mindspring.com> User-Agent: Mutt/1.3.27i X-Operating-System: FreeBSD 4.6-PRERELEASE i386 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Terry Lambert wrote: > > > Ideally everything should install as a package, however that would > > > > Currently, I cannot agree with this. I had enough head ache in the > > past dealing with packages of "compatibility symlinks", man pages, > > and so on, which seems overly ridiculous to me. I don't consider > > this worthwhile. Generally, I prefer base as monolithic collection > > of bits. > > It is a prerequisite for: > > o Ability to do binary upgrades of the base system in order to > automatically (e.g. via cron) obtain, and optionally install, > security and other fixes. For people who are running -release, what about having an executable shell script, which contains uuencoded patched binaries and, when executed, unpacks them and installs them to the proper locations (like the shell-script "installers" provided by some commercial software vendors), overwriting the old binaries? For people who're running -stable, well, I suppose they don't mind a make world. But such a shell archive may still work. The full bells-and-whistles of a package/ports system are needed for clean uninstalling and dependency tracking. For security fixes in the base system, it seems to me, it's overkill. - Rahul To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 14:20:19 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7F9F937B400; Wed, 10 Jul 2002 14:20:13 -0700 (PDT) Received: from rwcrmhc52.attbi.com (rwcrmhc52.attbi.com [216.148.227.88]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1946743E09; Wed, 10 Jul 2002 14:20:13 -0700 (PDT) (envelope-from julian@elischer.org) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc52.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020710212012.BEAG8262.rwcrmhc52.attbi.com@InterJet.elischer.org>; Wed, 10 Jul 2002 21:20:12 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id OAA42297; Wed, 10 Jul 2002 14:03:49 -0700 (PDT) Date: Wed, 10 Jul 2002 14:03:47 -0700 (PDT) From: Julian Elischer To: Archie Cobbs Cc: John Baldwin , davidx@viasoft.com.cn, freebsd-arch@FreeBSD.org Subject: Re: Timeout and SMP race In-Reply-To: <200207101815.g6AIFDm28655@arch20m.dellroad.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 10 Jul 2002, Archie Cobbs wrote: > > [ NOTE: I'm moving this discussion to freebsd-arch@freebsd.org ] > 1. The caller supplies a pointer to the 'handle', which must initially > be NULL. The handle != NULL if and only if the timer is running. The pointer is NULL or the pointee (tm) is NULL? Why would you have to send a pointer if it must be NULL? SO assuming that the handle is what is NULL, what IS it when it is not NULL?? An integer? a pointer? a magic number? I'm assuming it is an 'int' that makes some sense only to the callout system. > 2. timer_cancel() guarantees that tfunc() will not be called subsequently how? By use of a mutex? If the cancel holds the mutex when the timeout tries to get it, how do you ensure that the timeout gets scheduled and notices, before the timeout handle is re-used? > 3. *handlep is set to NULL by timer_cancel() and by the timer expiring. > So when *handlep is NULL when tfunc() is invoked (unless > TIMER_RECURRING). > 4. Calling timer_start() or timer_stop() from within tfunc() is OK. > 5. If TIMER_RECURRING, timer started again before calling tfunc() > 6. If TIMER_OWN_THREAD, timer runs in a newly created thread (rather > than the timer service thread), which means that tfunc() may sleep > or be canceled. If tfunc() sleeps or the thread is canceled but > TIMER_OWN_THREAD was not set -> panic. > 7. If mutexp != NULL, *mutexp is acquired before calling tfunc() and > released after it returns. > > Items 1, and 2 are guaranteed only if mutexp != NULL and the caller > acquires *mutexp before any calls to timer_start() or timer_cancel() > (you would normally be doing this anyway). What if the timeout handle was a structure that included the mutex? > > Errors: > > - timer_start() returns EBUSY if *handlep != NULL > - timer_remaining() returns ESRCH if handle != NULL > > The model is: you have some object that has an associated lock and > one or more associated timers. The object is to be locked whenever > you muck with it (including when you start, stop, or handle a timer): > > struct foobar { > struct lock mutex; > timer_handle_t timer1; > timer_handle_t timer2; > ... > }; > > Then all calls to the timer_* routines are "well behaved" and the > timeout thread caling tfunc() never races with any other thread > that may be stopping or starting the timer, or destroying the object. > E.g., to destroy the object, the following suffices: > > void > foobar_destroy(struct foobar *f) > { > mutex_lock(&f->mutex); > timer_cancel(&f->timer1); > timer_cancel(&f->timer2); > mutex_unlock(&f->mutex); > free(f); > } > > The only remaining complexity for the caller is that if you have > any TIMER_OWN_THREAD handlers which unlock the object (e.g., in order > to go to sleep), then you need to reference count the object and > have a FOOBAR_INVALID flag. > > If you are working under a different model then this API may not > be appropriate, but at least in my multi-threading experience this > model is very typical. > > -Archie > > __________________________________________________________________________ > Archie Cobbs * Packet Design * http://www.packetdesign.com > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-current" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 14:21: 0 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5C8CA37B400; Wed, 10 Jul 2002 14:20:59 -0700 (PDT) Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id E66B243E09; Wed, 10 Jul 2002 14:20:58 -0700 (PDT) (envelope-from des@ofug.org) Received: by flood.ping.uio.no (Postfix, from userid 2602) id 718BC534B; Wed, 10 Jul 2002 23:20:57 +0200 (CEST) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Dan Moschuk Cc: arch@FreeBSD.org Subject: Re: Package format proposal upcoming References: <20020710141943.GA1157@scoobysnax.jaded.net> <20020710142954.GB1157@scoobysnax.jaded.net> From: Dag-Erling Smorgrav Date: 10 Jul 2002 23:20:56 +0200 In-Reply-To: <20020710142954.GB1157@scoobysnax.jaded.net> Message-ID: Lines: 11 User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.2 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Dan Moschuk writes: > And on a side note to that side note, libh needs a new name. How about > libsheep? ;) No, liboveewe (-lovewe) *duck, run* DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 14:22: 0 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2C55137B400 for ; Wed, 10 Jul 2002 14:21:57 -0700 (PDT) Received: from goose.mail.pas.earthlink.net (goose.mail.pas.earthlink.net [207.217.120.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id A9F2843E54 for ; Wed, 10 Jul 2002 14:21:56 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0451.cvx22-bradley.dialup.earthlink.net ([209.179.199.196] helo=mindspring.com) by goose.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17SOu0-0007AQ-00; Wed, 10 Jul 2002 14:21:37 -0700 Message-ID: <3D2CA535.EC11BDA1@mindspring.com> Date: Wed, 10 Jul 2002 14:20:53 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Rahul Siddharthan Cc: Alexey Dokuchaev , Cy Schubert - CITS Open Systems Group , arch@freebsd.org Subject: Re: Package system wishlist References: <20020710210509.GA686@lpt.ens.fr> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Rahul Siddharthan wrote: > > It is a prerequisite for: > > > > o Ability to do binary upgrades of the base system in order to > > automatically (e.g. via cron) obtain, and optionally install, > > security and other fixes. > > For people who are running -release, what about having an executable > shell script, which contains uuencoded patched binaries and, when > executed, unpacks them and installs them to the proper locations (like > the shell-script "installers" provided by some commercial software > vendors), overwriting the old binaries? > > For people who're running -stable, well, I suppose they don't mind a > make world. But such a shell archive may still work. > > The full bells-and-whistles of a package/ports system are needed for > clean uninstalling and dependency tracking. For security fixes in the > base system, it seems to me, it's overkill. o I would like to be able to run a cron job that fetches a file of path names to files that are part of my current release, and known to have had security problems, and corresponding MD5 hashes of the fixed files, to compare to, and issue a security report and/or automatically add security patches to the system. o I would like to be able to redefine any release from being "Release X" to "Release X plus all relevent security patches" or "Release X plus all relevent security and performance patches", as a site local option. This is mostly an issue for an installed system with poor upgrade prospects, but a long life expectancy, e.g. for RackSpace.com or a similar situation. The combinatorics for a large number of patches which accumulate slowly over time end up making this problematic. I can re-donate my "patchkit" code, but that means serializing security updates through a human being, and applying them all in order, even if one update completely overwrites the contents of another (i.e. "download 4M of obsolete binaries" ... "download 4M again"). 8-). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 14:37: 7 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 295E637B400 for ; Wed, 10 Jul 2002 14:37:03 -0700 (PDT) Received: from postfix1-2.free.fr (postfix1-2.free.fr [213.228.0.130]) by mx1.FreeBSD.org (Postfix) with ESMTP id 74BC243E31 for ; Wed, 10 Jul 2002 14:37:02 -0700 (PDT) (envelope-from rsidd@lpt.ens.fr) Received: from bluerondo.a.la.turk (nas-cbv-4-62-147-140-37.dial.proxad.net [62.147.140.37]) by postfix1-2.free.fr (Postfix) with ESMTP id 2C6C3AB44B for ; Wed, 10 Jul 2002 23:37:00 +0200 (CEST) Received: (qmail 913 invoked by uid 1001); 10 Jul 2002 21:36:19 -0000 Date: Wed, 10 Jul 2002 23:36:19 +0200 From: Rahul Siddharthan To: Terry Lambert Cc: Alexey Dokuchaev , Cy Schubert - CITS Open Systems Group , arch@freebsd.org Subject: Re: Package system wishlist Message-ID: <20020710213619.GA882@lpt.ens.fr> References: <20020710210509.GA686@lpt.ens.fr> <3D2CA535.EC11BDA1@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3D2CA535.EC11BDA1@mindspring.com> User-Agent: Mutt/1.3.27i X-Operating-System: FreeBSD 4.6-PRERELEASE i386 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Terry Lambert wrote: > Rahul Siddharthan wrote: > > > It is a prerequisite for: > > > > > > o Ability to do binary upgrades of the base system in order to > > > automatically (e.g. via cron) obtain, and optionally install, > > > security and other fixes. > > > > For people who are running -release, what about having an executable > > shell script, which contains uuencoded patched binaries and, when > > executed, unpacks them and installs them to the proper locations (like > > the shell-script "installers" provided by some commercial software > > vendors), overwriting the old binaries? > > o I would like to be able to run a cron job that fetches a > file of path names to files that are part of my current > release, and known to have had security problems, and > corresponding MD5 hashes of the fixed files, to compare > to, and issue a security report and/or automatically add > security patches to the system. I still don't see why you need a packaging system for that. Why not 1. Issue each security alert in two pieces, consisting of a list of affected files and a binary-upgrade shell script; 2. Download the list, and if you're affected, download the shell-script and run it. The shell script could have an MD5 hash as a whole, issued by the FreeBSD project; this would be the complete "binary patch" for the problem. > o I would like to be able to redefine any release from being > "Release X" to "Release X plus all relevent security patches" > or "Release X plus all relevent security and performance > patches", as a site local option. > > This is mostly an issue for an installed system with poor upgrade > prospects, but a long life expectancy, e.g. for RackSpace.com or > a similar situation. > > The combinatorics for a large number of patches which accumulate > slowly over time end up making this problematic. True with source patches, but not with binary upgrades, as far as I can see. If you install the most recent updates in any category, you should be safe. If you're doing this via a cron job, you'll anyway be installing each upgrade immediately as it comes out. Since the fixes for a -release will not involve major upgrades of system components (eg openssh 2.9 -> 3.4), only minor patches, even if you miss an update somewhere it shouldn't affect the compatibility of the next update. If you skipped update 1 and applied update 2 for openssh, the worst that can happen is that you missed a security fix which came with update 1 (and if you're lucky, the relevant files got overwritten by update 2 anyway). Or am I missing something here? - Rahul To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 14:44:40 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DD99D37B406 for ; Wed, 10 Jul 2002 14:44:38 -0700 (PDT) Received: from herbelot.dyndns.org (d108.dhcp212-198-26.noos.fr [212.198.26.108]) by mx1.FreeBSD.org (Postfix) with ESMTP id E7F8C43E42 for ; Wed, 10 Jul 2002 14:44:36 -0700 (PDT) (envelope-from thierry@herbelot.com) Received: from herbelot.com (tulipe.herbelot.nom [192.168.2.5]) by herbelot.dyndns.org (8.9.3/8.9.3) with ESMTP id XAA19874 for ; Wed, 10 Jul 2002 23:44:34 +0200 (CEST) (envelope-from thierry@herbelot.com) Message-ID: <3D2CAAC1.5CF66C96@herbelot.com> Date: Wed, 10 Jul 2002 23:44:33 +0200 From: Thierry Herbelot X-Mailer: Mozilla 4.79 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 Cc: arch@FreeBSD.ORG Subject: Listening to users [was Re: Package system wishlist] References: <3D2BE142.E25CA9BC@mindspring.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG [Snip the long arguments over which new format for the packages] there is a interesting post by an ex-user of desktop Linux, which can help when designing the new package system (it will have to be used by mere mortals, not only by the elite posting on -arch) (shamelessly cut&pasted from /.) we still have a long way to go before even computer-aware people (that is, even people formely developping on SGI or Sun machines) can feel at ease installing and using FreeBSD. TfH To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 14:45: 7 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 73C2637B400; Wed, 10 Jul 2002 14:45:05 -0700 (PDT) Received: from InterJet.dellroad.org (adsl-63-194-81-26.dsl.snfc21.pacbell.net [63.194.81.26]) by mx1.FreeBSD.org (Postfix) with ESMTP id DA18843E52; Wed, 10 Jul 2002 14:45:04 -0700 (PDT) (envelope-from archie@dellroad.org) Received: from arch20m.dellroad.org (arch20m.dellroad.org [10.1.1.20]) by InterJet.dellroad.org (8.9.1a/8.9.1) with ESMTP id OAA69955; Wed, 10 Jul 2002 14:28:20 -0700 (PDT) Received: (from archie@localhost) by arch20m.dellroad.org (8.11.6/8.11.6) id g6ALRl329601; Wed, 10 Jul 2002 14:27:47 -0700 (PDT) (envelope-from archie) From: Archie Cobbs Message-Id: <200207102127.g6ALRl329601@arch20m.dellroad.org> Subject: Re: Timeout and SMP race In-Reply-To: <3D2C9E8A.8D7657E7@mindspring.com> "from Terry Lambert at Jul 10, 2002 01:52:26 pm" To: Terry Lambert Date: Wed, 10 Jul 2002 14:27:47 -0700 (PDT) Cc: Archie Cobbs , John Baldwin , davidx@viasoft.com.cn, freebsd-arch@FreeBSD.org, julian@elischer.org X-Mailer: ELM [version 2.4ME+ PL88 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Terry Lambert writes: > > FWIW, here is an API I've used before. This handles all race > > conditions and the 'other thread' question. > > I like this timer API. > > You need to add a timer_modify_interval (or something similar), so > that intervals can be changed without having to destroy and recreate > the timer, though, to avoid another potential race condition. There's no race condition there. You can just do this to 'reset' a timer: timer_cancel(&foo->timer); timer_start(&foo->timer, ...); Your timer_modify_interval() would just be an optimization of this. I forgot to mention that timer_cancel() does nothing if *handlep == NULL. -Archie __________________________________________________________________________ Archie Cobbs * Packet Design * http://www.packetdesign.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 15: 4:33 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 69FDD37B400 for ; Wed, 10 Jul 2002 15:04:31 -0700 (PDT) Received: from mail3.panix.com (mail3.panix.com [166.84.1.74]) by mx1.FreeBSD.org (Postfix) with ESMTP id CC19643E31 for ; Wed, 10 Jul 2002 15:04:30 -0700 (PDT) (envelope-from rsi@panix.com) Received: from panix1.panix.com (panix1.panix.com [166.84.1.1]) by mail3.panix.com (Postfix) with ESMTP id 2C796981FF; Wed, 10 Jul 2002 18:04:30 -0400 (EDT) Received: (from rsi@localhost) by panix1.panix.com (8.11.3nb1/8.8.8/PanixN1.0) id g6AM4UD11370; Wed, 10 Jul 2002 18:04:30 -0400 (EDT) Message-Id: <200207102204.g6AM4UD11370@panix1.panix.com> X-Authentication-Warning: panix1.panix.com: rsi set sender to rsi@panix.com using -f To: Thierry Herbelot Cc: arch@FreeBSD.ORG Subject: Re: Listening to users [was Re: Package system wishlist] References: <3D2BE142.E25CA9BC@mindspring.com> <3D2CAAC1.5CF66C96@herbelot.com> From: Rajappa Iyer Date: 10 Jul 2002 18:04:29 -0400 Reply-To: rsi@panix.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Thierry Herbelot writes: > [Snip the long arguments over which new format for the packages] > > there is a interesting post by an ex-user of desktop Linux, which can > help when designing the new package system (it will have to be used by > mere mortals, not only by the elite posting on -arch) > > > (shamelessly cut&pasted from /.) > > we still have a long way to go before even computer-aware people (that > is, even people formely developping on SGI or Sun machines) can feel at > ease installing and using FreeBSD. Note that almost all of his frustration was due to the multiple distributions problem in Linux; a problem we do not, happily, share with Linux. Also note that FreeBSD, by and large, does not have nearly as many incompatible changes to the device driver interface. Another Linux problem which is a non-problem in the FreeBSD world. As far as probing hardware is concerned, by and large, I've not needed to use anything but the GENERIC kernel for a long, long time now. The whole setting up of CD-RW devices as a pseudo-scsi device on Linux is a kludge. In other words, I have to say that the complaint above does not in any way support your contention that FreeBSD is hard to install and use. rsi -- a.k.a. Rajappa Iyer. They also surf who stand in the waves. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 15:15: 6 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4A1BA37B400; Wed, 10 Jul 2002 15:15:03 -0700 (PDT) Received: from InterJet.dellroad.org (adsl-63-194-81-26.dsl.snfc21.pacbell.net [63.194.81.26]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8179A43E3B; Wed, 10 Jul 2002 15:15:02 -0700 (PDT) (envelope-from archie@dellroad.org) Received: from arch20m.dellroad.org (arch20m.dellroad.org [10.1.1.20]) by InterJet.dellroad.org (8.9.1a/8.9.1) with ESMTP id OAA70143; Wed, 10 Jul 2002 14:59:36 -0700 (PDT) Received: (from archie@localhost) by arch20m.dellroad.org (8.11.6/8.11.6) id g6ALx3b29709; Wed, 10 Jul 2002 14:59:03 -0700 (PDT) (envelope-from archie) From: Archie Cobbs Message-Id: <200207102159.g6ALx3b29709@arch20m.dellroad.org> Subject: Re: Timeout and SMP race In-Reply-To: "from Julian Elischer at Jul 10, 2002 02:03:47 pm" To: Julian Elischer Date: Wed, 10 Jul 2002 14:59:03 -0700 (PDT) Cc: Archie Cobbs , John Baldwin , davidx@viasoft.com.cn, freebsd-arch@FreeBSD.org X-Mailer: ELM [version 2.4ME+ PL88 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Julian Elischer writes: > > 1. The caller supplies a pointer to the 'handle', which must initially > > be NULL. The handle != NULL if and only if the timer is running. > > The pointer is NULL or the pointee (tm) is NULL? Why would you have to > send a pointer if it must be NULL? SO assuming that the handle is what is > NULL, what IS it when it is not NULL?? An integer? a pointer? a magic > number? I'm assuming it is an 'int' that makes some sense only to the > callout system. The handle is a pointer, but a pointer to something opaque to the caller. All the caller can do with it is see if it's NULL or not. And the non-NULL'ness of the pointer always reflects whether the timer is 'ticking' or not. Secretly we know that the handle points to a 'struct timer_info' which is defined by, and managed in, the timer code. Hence the public header file only contains this: struct timer_info; /* declared but not defined */ typedef struct timer_info *timer_handle_t; > > 2. timer_cancel() guarantees that tfunc() will not be called subsequently > > how? By use of a mutex? If the cancel holds the mutex when the timeout > tries to get it, how do you ensure that the timeout gets scheduled and > notices, before the timeout handle is re-used? You are already holding the mutex when timer_cancel() is called. So the timer can only try to fire afterwards. But timer_cancel() sets a flag in the 'struct timer_info' marking it as invalid so the timeout code knows to just free it. timer_cancel() also sets *handlep (the caller's reference) to NULL before returning but does not necessarily free the 'struct timer_info' right away. Note a 'struct timer_info' is never reused, only the handle gets reused. On the next use, the handle would point to a new 'struct timer_info'. > What if the timeout handle was a structure that included the mutex? You want the mutex that the timer code locks to be the same one that all your other code locks to protect the object. So the caller should tell the timer code which mutex to use, not the other way around. Or maybe I'm misunderstanding your question. -Archie __________________________________________________________________________ Archie Cobbs * Packet Design * http://www.packetdesign.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 15:17:31 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0DDA937B400; Wed, 10 Jul 2002 15:17:27 -0700 (PDT) Received: from prism.flugsvamp.com (66-191-112-47.mad.wi.charter.com [66.191.112.47]) by mx1.FreeBSD.org (Postfix) with ESMTP id E0B2043E5E; Wed, 10 Jul 2002 15:17:25 -0700 (PDT) (envelope-from jlemon@flugsvamp.com) Received: (from jlemon@localhost) by prism.flugsvamp.com (8.11.6/8.11.6) id g6AMFqC26935; Wed, 10 Jul 2002 17:15:52 -0500 (CDT) (envelope-from jlemon) Date: Wed, 10 Jul 2002 17:15:52 -0500 From: Jonathan Lemon To: Archie Cobbs Cc: John Baldwin , davidx@viasoft.com.cn, freebsd-arch@FreeBSD.ORG, julian@elischer.org Subject: Re: Timeout and SMP race Message-ID: <20020710171552.F65393@prism.flugsvamp.com> References: <200207101815.g6AIFDm28655@arch20m.dellroad.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0pre2i In-Reply-To: <200207101815.g6AIFDm28655@arch20m.dellroad.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Jul 10, 2002 at 11:15:13AM -0700, Archie Cobbs wrote: > > [ NOTE: I'm moving this discussion to freebsd-arch@freebsd.org ] > > John Baldwin writes: > > > What do you think of the idea of letting the timer code (optionally) > > > handle all the locking and race conditions? > > > > I'm not sure it can in a clean fashion since of the few cases I've known > > so far each client needs a customized solution. I am open to ideas though. > > I'm also open to some redesign of how callouts work to begin with (maybe > > using other threads than the one running softclock() to actually execute > > callout handlers, etc.). > > FWIW, here is an API I've used before. This handles all race > conditions and the 'other thread' question. > > struct timer_event; /* opaque structure */ > > typedef struct timer_event *timer_handle_t; /* caller's timer "handle" */ > > typedef void timer_func_t(void *arg); /* timeout function type */ > > /* flags for timer_start() */ > #define TIMER_RECURRING 0x0001 /* timer is recurring */ > #define TIMER_OWN_THREAD 0x0002 /* handle in separate thread */ > > extern int timer_start(timer_handle_t *handlep, mutex_t *mutexp, > timer_func_t tfunc, void *arg, u_int delay, > int flags); > extern void timer_cancel(timer_handle_t *handlep); > extern int timer_remaining(timer_handle_t handle, u_int *delayp); It seems to me that we can achieve the same functionality without changing the callout API too much. You mention that the mutex (if supplied) will be acquired before calling tfunc. This means that it has to be stored somewhere, presumably in the callout structure itself. The callout() consumers typically allocate their own storage, so perhaps we should add an (optional) pointer to a mutex to the callout structure, where the mutex is obtained/released before the callout function is made. -- Jonathan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 15:38:46 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C9E2037B400; Wed, 10 Jul 2002 15:38:43 -0700 (PDT) Received: from snipe.mail.pas.earthlink.net (snipe.mail.pas.earthlink.net [207.217.120.62]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6222543E31; Wed, 10 Jul 2002 15:38:43 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0451.cvx22-bradley.dialup.earthlink.net ([209.179.199.196] helo=mindspring.com) by snipe.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17SQ6Z-000114-00; Wed, 10 Jul 2002 15:38:39 -0700 Message-ID: <3D2CB73E.670D80C8@mindspring.com> Date: Wed, 10 Jul 2002 15:37:50 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Archie Cobbs Cc: John Baldwin , davidx@viasoft.com.cn, freebsd-arch@FreeBSD.org, julian@elischer.org Subject: Re: Timeout and SMP race References: <200207102127.g6ALRl329601@arch20m.dellroad.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Archie Cobbs wrote: > Terry Lambert writes: > > > FWIW, here is an API I've used before. This handles all race > > > conditions and the 'other thread' question. > > > > I like this timer API. > > > > You need to add a timer_modify_interval (or something similar), so > > that intervals can be changed without having to destroy and recreate > > the timer, though, to avoid another potential race condition. > > There's no race condition there. You can just do this to 'reset' a timer: > > timer_cancel(&foo->timer); > timer_start(&foo->timer, ...); > > Your timer_modify_interval() would just be an optimization of this. > > I forgot to mention that timer_cancel() does nothing if *handlep == NULL. This depends on whether you mutex is local to the timer only, or if it has other uses. There is also an order of operation race, if you can be preempted between the cancel and the start. 8-). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 15:40:13 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CCDF037B401 for ; Wed, 10 Jul 2002 15:40:05 -0700 (PDT) Received: from postfix1-2.free.fr (postfix1-2.free.fr [213.228.0.130]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5ACE143E5E for ; Wed, 10 Jul 2002 15:40:05 -0700 (PDT) (envelope-from rsidd@lpt.ens.fr) Received: from bluerondo.a.la.turk (nas-cbv-11-62-147-118-15.dial.proxad.net [62.147.118.15]) by postfix1-2.free.fr (Postfix) with ESMTP id DDB07AB2A5 for ; Thu, 11 Jul 2002 00:40:02 +0200 (CEST) Received: (qmail 1381 invoked by uid 1001); 10 Jul 2002 22:40:00 -0000 Date: Thu, 11 Jul 2002 00:40:00 +0200 From: Rahul Siddharthan To: Rajappa Iyer Cc: Thierry Herbelot , arch@freebsd.org Subject: Re: Listening to users [was Re: Package system wishlist] Message-ID: <20020710224000.GA1331@lpt.ens.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200207102204.g6AM4UD11370@panix1.panix.com> User-Agent: Mutt/1.3.27i X-Operating-System: FreeBSD 4.6-PRERELEASE i386 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Maybe this isn't terribly on-topic, but anyway: Rajappa Iyer wrote: > > ; > > Note that almost all of his frustration was due to the multiple > distributions problem in Linux No - he notes that but didn't have a problem with that. His problem was that *none* of the distributions were suitable. But almost all of them would have been friendlier than FreeBSD. > In other words, I have to say that the complaint above does not in any > way support your contention that FreeBSD is hard to install and use. Well, let's go through the major points of that article again: XFree86 -- FreeBSD uses it too, but the linuxen have better configurators, and possibly better default font setups, these days. So FreeBSD is actually worse off. Drivers -- FreeBSD hardware support is, if anything, worse than Linux (eg, cardbus; a lot of multimedia hardware; winmodems; etc). And I haven't encountered so many third-party binary device drivers for FreeBSD, much less drivers that work across different kernel versions (as he wants for linux). Software distribution -- well, that's what this whole thread has been about, hasn't it? I like FreeBSD ports, but I've met enough people who think rpm is "easier" that I'm not going to argue with them. If you read what he's asking for, you'll see that FreeBSD doesn't provide it. Support -- Well, linux does have some commercial distributors who'll support you for a fee. FreeBSD doesn't. The FreeBSD mailing lists are great, but his complaint of RTFM-type "elitism" is equally true of FreeBSD. In short, everything he says, and more, applies to FreeBSD. Pretending otherwise won't help anyone. And generalizing your own experience (you find it easy, so everyone must find it easy) is exactly the elitism he doesn't like. - Rahul To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 15:42:49 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0EE7337B400 for ; Wed, 10 Jul 2002 15:42:48 -0700 (PDT) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id CC92D43E4A for ; Wed, 10 Jul 2002 15:42:47 -0700 (PDT) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id 9506CAE162; Wed, 10 Jul 2002 15:42:47 -0700 (PDT) Date: Wed, 10 Jul 2002 15:42:47 -0700 From: Alfred Perlstein To: Rahul Siddharthan Cc: Rajappa Iyer , Thierry Herbelot , arch@freebsd.org Subject: Re: Listening to users [was Re: Package system wishlist] Message-ID: <20020710224247.GN97638@elvis.mu.org> References: <200207102204.g6AM4UD11370@panix1.panix.com> <20020710224000.GA1331@lpt.ens.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020710224000.GA1331@lpt.ens.fr> User-Agent: Mutt/1.3.27i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * Rahul Siddharthan [020710 15:40] wrote: > Maybe this isn't terribly on-topic, but anyway: You're right, it isn't, can you all please keep this stuff off of -arch. thank you, -Alfred To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 15:54:10 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1ACB937B400 for ; Wed, 10 Jul 2002 15:54:07 -0700 (PDT) Received: from avocet.mail.pas.earthlink.net (avocet.mail.pas.earthlink.net [207.217.120.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 95D5043E31 for ; Wed, 10 Jul 2002 15:54:06 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0451.cvx22-bradley.dialup.earthlink.net ([209.179.199.196] helo=mindspring.com) by avocet.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17SQL9-0000PX-00; Wed, 10 Jul 2002 15:53:44 -0700 Message-ID: <3D2CBAC4.6AC3CAC9@mindspring.com> Date: Wed, 10 Jul 2002 15:52:52 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Rahul Siddharthan Cc: Alexey Dokuchaev , Cy Schubert - CITS Open Systems Group , arch@freebsd.org Subject: Re: Package system wishlist References: <20020710210509.GA686@lpt.ens.fr> <3D2CA535.EC11BDA1@mindspring.com> <20020710213619.GA882@lpt.ens.fr> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Rahul Siddharthan wrote: > > o I would like to be able to run a cron job that fetches a > > file of path names to files that are part of my current > > release, and known to have had security problems, and > > corresponding MD5 hashes of the fixed files, to compare > > to, and issue a security report and/or automatically add > > security patches to the system. > > I still don't see why you need a packaging system for that. Why not > > 1. Issue each security alert in two pieces, consisting of a list of > affected files and a binary-upgrade shell script; > 2. Download the list, and if you're affected, download the > shell-script and run it. The shell script could have an MD5 hash > as a whole, issued by the FreeBSD project; this would be the > complete "binary patch" for the problem. o Order of application o Number of "base systems" (hint: try and make your suggested system work today, with all FreeBSD release starting with 4.0 and going up through 4.6, while supporting upgrade from 4.2 to 4.4). > > o I would like to be able to redefine any release from being > > "Release X" to "Release X plus all relevent security patches" > > or "Release X plus all relevent security and performance > > patches", as a site local option. > > > > This is mostly an issue for an installed system with poor upgrade > > prospects, but a long life expectancy, e.g. for RackSpace.com or > > a similar situation. > > > > The combinatorics for a large number of patches which accumulate > > slowly over time end up making this problematic. > > True with source patches, but not with binary upgrades, as far as I > can see. If you install the most recent updates in any category, you > should be safe. "The most recent updates" are only appropriate if I am keeping my base OS version updated as well. I don't want to have to do that. I want to install a version that works and never update it ever again, except for security patches, until I have to face the 2038 UNIX 32 bit seconds rollover problem. We Fear Change(tm). > If you're doing this via a cron job, you'll anyway be > installing each upgrade immediately as it comes out. NOT UPGRADE. I DON'T WANT TO UPGRADE. I ONLY WANT TO PATCH. Upgrading destroys my use model. > Since the fixes for a -release will not involve major upgrades of > system components (eg openssh 2.9 -> 3.4), only minor patches, even > if you miss an update somewhere it shouldn't affect the compatibility > of the next update. If you skipped update 1 and applied update 2 for > openssh, the worst that can happen is that you missed a security fix > which came with update 1 (and if you're lucky, the relevant files got > overwritten by update 2 anyway). > > Or am I missing something here? I am refusing to go from 4.x to 5.x; or from 4.4 to 4.6. Whichever. The important part is my refusal to change for the sake of change. The worst that can happen is that some moron decided that there needed to be a break-out for ssh in /etc/pam.conf, and the fact that I missed a single update in the middle somewhere when this decision was made means a plane trip to the colocation facility in Dallas to add the "ssh" lines into the file manually, following a security update, after ssh starts ignoring the "login" lines that it used to pay attention to, instead of looking for the "ssh" lines, and falling back to the "login" lines if the "ssh" lines aren't there. Ala the change from FreeBSD 4.2 to 4.3. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 16: 7:16 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9115E37B400 for ; Wed, 10 Jul 2002 16:07:14 -0700 (PDT) Received: from postfix2-2.free.fr (postfix2-2.free.fr [213.228.0.140]) by mx1.FreeBSD.org (Postfix) with ESMTP id 19E3443E31 for ; Wed, 10 Jul 2002 16:07:14 -0700 (PDT) (envelope-from rsidd@lpt.ens.fr) Received: from bluerondo.a.la.turk (nas-cbv-6-62-147-149-102.dial.proxad.net [62.147.149.102]) by postfix2-2.free.fr (Postfix) with ESMTP id 6346D5F9CC for ; Thu, 11 Jul 2002 01:07:12 +0200 (CEST) Received: (qmail 1560 invoked by uid 1001); 10 Jul 2002 23:07:10 -0000 Date: Thu, 11 Jul 2002 01:07:10 +0200 From: Rahul Siddharthan To: Terry Lambert Cc: Alexey Dokuchaev , Cy Schubert - CITS Open Systems Group , chat@freebsd.org Subject: Re: Package system wishlist Message-ID: <20020710230709.GA1512@lpt.ens.fr> References: <20020710210509.GA686@lpt.ens.fr> <3D2CA535.EC11BDA1@mindspring.com> <20020710213619.GA882@lpt.ens.fr> <3D2CBAC4.6AC3CAC9@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3D2CBAC4.6AC3CAC9@mindspring.com> User-Agent: Mutt/1.3.27i X-Operating-System: FreeBSD 4.6-PRERELEASE i386 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG [moved to -chat] Terry Lambert wrote: > "The most recent updates" are only appropriate if I am keeping my > base OS version updated as well. I don't want to have to do that. > I want to install a version that works and never update it ever > again, except for security patches, until I have to face the 2038 > UNIX 32 bit seconds rollover problem. > > We Fear Change(tm). [snip] > I am refusing to go from 4.x to 5.x; or from 4.4 to 4.6. Whichever. > The important part is my refusal to change for the sake of change. Then you'd have to convince the FreeBSD security people to issue fixes for all branches, back to 1.0 or earlier, no? (Equivalently, to support version 4.4 until the year 2038.) I still don't see your argument for packaging components of the base system. A binary from FreeBSD 7.3 in 2006 is unlikely to work on your FreeBSD 4.4 system. Microsoft did support Windows 95 for a long time, but they have a rather less aggressive release schedule. - Rahul To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 16: 7:43 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8255F37B401 for ; Wed, 10 Jul 2002 16:07:30 -0700 (PDT) Received: from mail3.panix.com (mail3.panix.com [166.84.1.74]) by mx1.FreeBSD.org (Postfix) with ESMTP id D8C8243E54 for ; Wed, 10 Jul 2002 16:07:28 -0700 (PDT) (envelope-from rsi@panix.com) Received: from panix1.panix.com (panix1.panix.com [166.84.1.1]) by mail3.panix.com (Postfix) with ESMTP id 75FD2981A2; Wed, 10 Jul 2002 19:07:28 -0400 (EDT) Received: (from rsi@localhost) by panix1.panix.com (8.11.3nb1/8.8.8/PanixN1.0) id g6AN7SV22593; Wed, 10 Jul 2002 19:07:28 -0400 (EDT) Message-Id: <200207102307.g6AN7SV22593@panix1.panix.com> X-Authentication-Warning: panix1.panix.com: rsi set sender to rsi@panix.com using -f To: Rahul Siddharthan Cc: Thierry Herbelot , arch@freebsd.org Subject: Re: Listening to users [was Re: Package system wishlist] References: <20020710224000.GA1331@lpt.ens.fr> From: Rajappa Iyer Date: 10 Jul 2002 19:07:28 -0400 Reply-To: rsi@panix.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Rahul Siddharthan writes: > Maybe this isn't terribly on-topic, but anyway: > > Rajappa Iyer wrote: > > > ; > > > > Note that almost all of his frustration was due to the multiple > > distributions problem in Linux > > No - he notes that but didn't have a problem with that. His problem > was that *none* of the distributions were suitable. But almost all of > them would have been friendlier than FreeBSD. Huh? Read the article again. Then read what I wrote here again. He's complaining about the fact that third-party drivers do not work across different kernels. Since different distributions use different (minor) versions of the kernel, the binary compatability problem he has is squarely attributable to the multiple distribution problem in Linux. > XFree86 -- FreeBSD uses it too, but the linuxen have better > configurators, and possibly better default font setups, these days. > So FreeBSD is actually worse off. I don't know if any Linux distribution has a different font setup. I've only tried to install True Type fonts in Debian and it was not at all trivial. It's a whole lot easier to install the webfonts port on FreeBSD. I haven't tried this with other Linux distributions. > Drivers -- FreeBSD hardware support is, if anything, worse than Linux > (eg, cardbus; a lot of multimedia hardware; winmodems; etc). And I > haven't encountered so many third-party binary device drivers for > FreeBSD, much less drivers that work across different kernel versions > (as he wants for linux). There aren't many, but e.g. OSS drivers worked across several kernel versions. Certainly across minor revisions. As a matter of policy, FreeBSD takes far more care to maintain backward compatibility. Whereas Linus has explicitly stated that he will not be held hostage to maintaining compatibility. > Software distribution -- well, that's what this whole thread has been > about, hasn't it? I like FreeBSD ports, but I've met enough people > who think rpm is "easier" that I'm not going to argue with them. If > you read what he's asking for, you'll see that FreeBSD doesn't provide > it. Actually, it does address one of his objections. He wants a separation of system and applications; which is precisely the way FreeBSD operates. He wouldn't like the ports system because it rebuilds everything from source, but then the package system does take care of that objection. Yes, the package installer isn't as slick as Windows, but it's hardly unusable. > In short, everything he says, and more, applies to FreeBSD. > Pretending otherwise won't help anyone. And generalizing your own > experience (you find it easy, so everyone must find it easy) is > exactly the elitism he doesn't like. Stop being patronizing. The fact of the matter is, he pointed out one example of hardware that Linux failed to recognize and configure, namely a CD-RW drive. On Linux, using this hardware involves a kludge. On FreeBSD this kludge is not required. One can, ultimately, only speak from one's experience. This does not mean one does not listen to others' experiences. But the objections have to be more specific in order to address them. An over-broad comment like "your hardware support sucks" is simply not something that can be meaningfully addressed. rsi -- a.k.a. Rajappa Iyer. They also surf who stand in the waves. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 16:47:25 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 56B9437B400 for ; Wed, 10 Jul 2002 16:47:22 -0700 (PDT) Received: from avocet.mail.pas.earthlink.net (avocet.mail.pas.earthlink.net [207.217.120.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id BC95F43E09 for ; Wed, 10 Jul 2002 16:47:21 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0451.cvx22-bradley.dialup.earthlink.net ([209.179.199.196] helo=mindspring.com) by avocet.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17SRAz-0007ZT-00; Wed, 10 Jul 2002 16:47:17 -0700 Message-ID: <3D2CC741.D8E7FBAC@mindspring.com> Date: Wed, 10 Jul 2002 16:46:09 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: rsi@panix.com Cc: Rahul Siddharthan , Thierry Herbelot , arch@freebsd.org Subject: Re: Listening to users [was Re: Package system wishlist] References: <20020710224000.GA1331@lpt.ens.fr> <200207102307.g6AN7SV22593@panix1.panix.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Please limit yourself to architectural issues, and move discussions of articles recently seen on slashdot to -chat. The original posting of the article reference was (maybe) defensible. The subsequent argument about its meaning is not. Thanks, -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 17:45:12 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 063A237B401; Wed, 10 Jul 2002 17:45:10 -0700 (PDT) Received: from InterJet.dellroad.org (adsl-63-194-81-26.dsl.snfc21.pacbell.net [63.194.81.26]) by mx1.FreeBSD.org (Postfix) with ESMTP id 98D9B43E72; Wed, 10 Jul 2002 17:45:05 -0700 (PDT) (envelope-from archie@dellroad.org) Received: from arch20m.dellroad.org (arch20m.dellroad.org [10.1.1.20]) by InterJet.dellroad.org (8.9.1a/8.9.1) with ESMTP id RAA71066; Wed, 10 Jul 2002 17:25:52 -0700 (PDT) Received: (from archie@localhost) by arch20m.dellroad.org (8.11.6/8.11.6) id g6B0PHA30341; Wed, 10 Jul 2002 17:25:17 -0700 (PDT) (envelope-from archie) From: Archie Cobbs Message-Id: <200207110025.g6B0PHA30341@arch20m.dellroad.org> Subject: Re: Timeout and SMP race In-Reply-To: <20020710171552.F65393@prism.flugsvamp.com> "from Jonathan Lemon at Jul 10, 2002 05:15:52 pm" To: Jonathan Lemon Date: Wed, 10 Jul 2002 17:25:17 -0700 (PDT) Cc: Archie Cobbs , John Baldwin , davidx@viasoft.com.cn, freebsd-arch@FreeBSD.ORG, julian@elischer.org X-Mailer: ELM [version 2.4ME+ PL88 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Jonathan Lemon writes: > > extern int timer_start(timer_handle_t *handlep, mutex_t *mutexp, > > timer_func_t tfunc, void *arg, u_int delay, > > int flags); > > extern void timer_cancel(timer_handle_t *handlep); > > extern int timer_remaining(timer_handle_t handle, u_int *delayp); > > It seems to me that we can achieve the same functionality without > changing the callout API too much. You mention that the mutex (if supplied) > will be acquired before calling tfunc. This means that it has to be > stored somewhere, presumably in the callout structure itself. > > The callout() consumers typically allocate their own storage, so perhaps > we should add an (optional) pointer to a mutex to the callout structure, > where the mutex is obtained/released before the callout function is made. Yep, that would work too.. essentially it's the same thing. If you're doing that, why not just store the mutex itself in the callout structure, rather than a pointer to it? I guess if you did that then you would then need some kind of flag that says whether to use it or not. Or.. maybe there would be some way for the timer code to tell if the mutex has been initialized or not, and use this to decide whether to use it or not? -Archie __________________________________________________________________________ Archie Cobbs * Packet Design * http://www.packetdesign.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 18: 0:19 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F369B37B400; Wed, 10 Jul 2002 18:00:11 -0700 (PDT) Received: from InterJet.dellroad.org (adsl-63-194-81-26.dsl.snfc21.pacbell.net [63.194.81.26]) by mx1.FreeBSD.org (Postfix) with ESMTP id AC9E943E4A; Wed, 10 Jul 2002 18:00:10 -0700 (PDT) (envelope-from archie@dellroad.org) Received: from arch20m.dellroad.org (arch20m.dellroad.org [10.1.1.20]) by InterJet.dellroad.org (8.9.1a/8.9.1) with ESMTP id RAA71144; Wed, 10 Jul 2002 17:42:00 -0700 (PDT) Received: (from archie@localhost) by arch20m.dellroad.org (8.11.6/8.11.6) id g6B0fRV30395; Wed, 10 Jul 2002 17:41:27 -0700 (PDT) (envelope-from archie) From: Archie Cobbs Message-Id: <200207110041.g6B0fRV30395@arch20m.dellroad.org> Subject: Re: Timeout and SMP race In-Reply-To: <3D2CB73E.670D80C8@mindspring.com> "from Terry Lambert at Jul 10, 2002 03:37:50 pm" To: Terry Lambert Date: Wed, 10 Jul 2002 17:41:27 -0700 (PDT) Cc: Archie Cobbs , John Baldwin , davidx@viasoft.com.cn, freebsd-arch@FreeBSD.org, julian@elischer.org X-Mailer: ELM [version 2.4ME+ PL88 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Terry Lambert writes: > > > You need to add a timer_modify_interval (or something similar), so > > > that intervals can be changed without having to destroy and recreate > > > the timer, though, to avoid another potential race condition. > > > > There's no race condition there. You can just do this to 'reset' a timer: > > > > timer_cancel(&foo->timer); > > timer_start(&foo->timer, ...); > > > > Your timer_modify_interval() would just be an optimization of this. > > > > I forgot to mention that timer_cancel() does nothing if *handlep == NULL. > > This depends on whether you mutex is local to the timer only, or > if it has other uses. Under the model in mind, the mutex would protect the caller's object. > There is also an order of operation race, if you can be preempted > between the cancel and the start. That wouldn't be possible because you've acquired the mutex already. I should have shown that more explicitly: mutex_lock(&foo->mutex); ... /* Reset the timer */ timer_cancel(&foo->timer); /* ok if not running */ timer_start(&foo->timer, ...); ... mutex_unlock(&foo->mutex); Also, the timer library of course will need it's own mutex to protect the timer queues, etc. An invariant is that the timer mutex is always acquired after the caller's mutex to avoid deadlock. This requires minor trickery in the timeout routine. -Archie __________________________________________________________________________ Archie Cobbs * Packet Design * http://www.packetdesign.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 18: 0:30 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A011737B400; Wed, 10 Jul 2002 18:00:23 -0700 (PDT) Received: from sccrmhc02.attbi.com (sccrmhc02.attbi.com [204.127.202.62]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8E5BA43E58; Wed, 10 Jul 2002 18:00:22 -0700 (PDT) (envelope-from julian@elischer.org) Received: from InterJet.elischer.org ([12.232.206.8]) by sccrmhc02.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020711010021.TGOS6023.sccrmhc02.attbi.com@InterJet.elischer.org>; Thu, 11 Jul 2002 01:00:21 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id RAA43259; Wed, 10 Jul 2002 17:53:33 -0700 (PDT) Date: Wed, 10 Jul 2002 17:53:30 -0700 (PDT) From: Julian Elischer To: Archie Cobbs Cc: Jonathan Lemon , John Baldwin , davidx@viasoft.com.cn, freebsd-arch@FreeBSD.ORG Subject: Re: Timeout and SMP race In-Reply-To: <200207110025.g6B0PHA30341@arch20m.dellroad.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 10 Jul 2002, Archie Cobbs wrote: > Jonathan Lemon writes: > > > extern int timer_start(timer_handle_t *handlep, mutex_t *mutexp, > > > timer_func_t tfunc, void *arg, u_int delay, > > > int flags); > > > extern void timer_cancel(timer_handle_t *handlep); > > > extern int timer_remaining(timer_handle_t handle, u_int *delayp); > > > > It seems to me that we can achieve the same functionality without > > changing the callout API too much. You mention that the mutex (if supplied) > > will be acquired before calling tfunc. This means that it has to be > > stored somewhere, presumably in the callout structure itself. > > > > The callout() consumers typically allocate their own storage, so perhaps > > we should add an (optional) pointer to a mutex to the callout structure, > > where the mutex is obtained/released before the callout function is made. > > Yep, that would work too.. essentially it's the same thing. > > If you're doing that, why not just store the mutex itself in the > callout structure, rather than a pointer to it? I guess if you did > that then you would then need some kind of flag that says whether > to use it or not. Or.. maybe there would be some way for the timer > code to tell if the mutex has been initialized or not, and use this > to decide whether to use it or not? Some timeouts don't need cancelability. (we'll not go here into whether having a separate timeout for each TCP session was a good or bad thing) In that case they shouldn't have to pay the price of carrying around a mutex.. unless we can say that there are almost no such cases.. On the other hand how many timeouts canyou do at one time? I'd say N where N is the number of processors. what is the hit from having just ONE (1) mutex. to cover all timeout operations. Surely all we need is for a certain amount of atomicity.. You do not want more than one processor at a time doing timeouts, so if you play your cards right, the same mutex could be used to make sure that removals didn;t clash with executions. > > -Archie > > __________________________________________________________________________ > Archie Cobbs * Packet Design * http://www.packetdesign.com > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 18:21: 7 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8662237B400 for ; Wed, 10 Jul 2002 18:21:03 -0700 (PDT) Received: from smtp.noos.fr (claudel.noos.net [212.198.2.83]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8F89043E31 for ; Wed, 10 Jul 2002 18:21:02 -0700 (PDT) (envelope-from root@gits.dyndns.org) Received: (qmail 27078623 invoked by uid 0); 11 Jul 2002 01:21:01 -0000 Received: from unknown (HELO gits.gits.dyndns.org) ([212.198.229.153]) (envelope-sender ) by 212.198.2.83 (qmail-ldap-1.03) with SMTP for ; 11 Jul 2002 01:21:01 -0000 Received: from gits.gits.dyndns.org (e2epfxapzitoueop@localhost [127.0.0.1]) by gits.gits.dyndns.org (8.12.5/8.12.5) with ESMTP id g6B1KCTb083194; Thu, 11 Jul 2002 03:21:00 +0200 (CEST) (envelope-from root@gits.dyndns.org) Received: (from root@localhost) by gits.gits.dyndns.org (8.12.5/8.12.5/Submit) id g6B0puvo082817; Thu, 11 Jul 2002 02:51:56 +0200 (CEST) (envelope-from root) Date: Thu, 11 Jul 2002 02:51:56 +0200 From: Cyrille Lefevre To: Christian Weisgerber , freebsd-arch@freebsd.org Subject: Re: Package system flaws? Message-ID: <20020711005155.GC82744@gits.dyndns.org> Mail-Followup-To: Cyrille Lefevre , Christian Weisgerber , freebsd-arch@freebsd.org References: <20020706220511.GA88651@scoobysnax.jaded.net> <20020708024404.GC83084@gits.dyndns.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020708024404.GC83084@gits.dyndns.org> User-Agent: Mutt/1.3.99i Organization: ACME X-Face: V|+c;4!|B?E%BE^{E6);aI.[< List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG message resent due to mta misconfigation. On Mon, Jul 08, 2002 at 04:44:04AM +0200, Cyrille Lefevre wrote: > On Sun, Jul 07, 2002 at 09:45:59PM +0000, Christian Weisgerber wrote: > > Dan Moschuk wrote: > > > > > I've been doing some thinking lately about our ports system versus what > > > other systems have adopted and was curious as to what people think on > > > the subject? What does FreeBSD do well? Where can we improve? How does it > > > rate against the umpteen Linux flavours? > > > > Compared to OpenBSD, the FreeBSD ports system lacks FAKE, FLAVORS, > > and MULTI_PACKAGES. regress would be nice, too. > > it seems they also support multi-level ports such as mutt/{stable,snapshot}. > > Compared to NetBSD, the FreeBSD ports system lacks non-static version > dependencies (DEPENDS) and CONFLICTS. also, what's that buildlink stuff ? Cyrille. -- Cyrille Lefevre mailto:cyrille.lefevre@laposte.net To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 18:21:36 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B93D037B405 for ; Wed, 10 Jul 2002 18:21:30 -0700 (PDT) Received: from smtp.noos.fr (aragon.noos.net [212.198.2.75]) by mx1.FreeBSD.org (Postfix) with ESMTP id 01FEF43E65 for ; Wed, 10 Jul 2002 18:21:29 -0700 (PDT) (envelope-from root@gits.dyndns.org) Received: (qmail 4504061 invoked by uid 0); 11 Jul 2002 01:21:27 -0000 Received: from unknown (HELO gits.gits.dyndns.org) ([212.198.229.153]) (envelope-sender ) by 212.198.2.75 (qmail-ldap-1.03) with SMTP for ; 11 Jul 2002 01:21:27 -0000 Received: from gits.gits.dyndns.org (e2epfxapzitoueop@localhost [127.0.0.1]) by gits.gits.dyndns.org (8.12.5/8.12.5) with ESMTP id g6B1KCTh083194; Thu, 11 Jul 2002 03:21:27 +0200 (CEST) (envelope-from root@gits.dyndns.org) Received: (from root@localhost) by gits.gits.dyndns.org (8.12.5/8.12.5/Submit) id g6B0ql9n082846; Thu, 11 Jul 2002 02:52:47 +0200 (CEST) (envelope-from root) Date: Thu, 11 Jul 2002 02:52:47 +0200 From: Cyrille Lefevre To: Rahul Siddharthan Cc: John Baldwin , arch@freebsd.org, Dan Nelson Subject: Re: Cleaning old packages (was: Package system flaws?) Message-ID: <20020711005247.GE82744@gits.dyndns.org> References: <20020709161953.GA69779@lpt.ens.fr> <20020709171417.GA69932@lpt.ens.fr> <20020709231820.GA49510@gits.dyndns.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020709231820.GA49510@gits.dyndns.org> User-Agent: Mutt/1.3.99i Organization: ACME X-Face: V|+c;4!|B?E%BE^{E6);aI.[< List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG message resent due to mta misconfigation. On Wed, Jul 10, 2002 at 01:18:20AM +0200, Cyrille Lefevre wrote: > On Tue, Jul 09, 2002 at 07:14:17PM +0200, Rahul Siddharthan wrote: > [snip] > > That seems rather ambitious, and too drastic a change, to me. What > > I'd like to see is probably more like, the gfoo port needs gtk+ 1.2.6 > > or above, but not gtk+ 2.0 and above (incompatible) and not gtk+ 1.2.5 > > or below (buggy). There should be some way to specify this in the > > makefile of the port, so that any port-management program like > > portupgrade can make use of the information. > > take a look at NetBSD pkg (aka ports) system, they have this kind > of version handling. > > > (3) Automatically generate the +CONTENTS file by first doing a "fake" > > install in a temporary directory (assuming the port honours > > $PREFIX), then moving the contents to their final destination (a la > > gentoo). If your temporary location is on the same filesystem as > > the final one, it won't even take additional disk space, and very > > little additional time. Is there any obvious drawback with doing > > this? > > that's what OpenBSD port system has... Cyrille. -- Cyrille Lefevre mailto:cyrille.lefevre@laposte.net To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 18:24:20 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E982337B400 for ; Wed, 10 Jul 2002 18:24:14 -0700 (PDT) Received: from smtp.noos.fr (claudel.noos.net [212.198.2.83]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7CC9C43E42 for ; Wed, 10 Jul 2002 18:24:13 -0700 (PDT) (envelope-from root@gits.dyndns.org) Received: (qmail 27014052 invoked by uid 0); 11 Jul 2002 01:24:12 -0000 Received: from unknown (HELO gits.gits.dyndns.org) ([212.198.229.153]) (envelope-sender ) by 212.198.2.83 (qmail-ldap-1.03) with SMTP for ; 11 Jul 2002 01:24:12 -0000 Received: from gits.gits.dyndns.org (e2epfxapzitoueop@localhost [127.0.0.1]) by gits.gits.dyndns.org (8.12.5/8.12.5) with ESMTP id g6B1KCWx083194; Thu, 11 Jul 2002 03:24:11 +0200 (CEST) (envelope-from root@gits.dyndns.org) Received: (from root@localhost) by gits.gits.dyndns.org (8.12.5/8.12.5/Submit) id g6B0pSLm082803; Thu, 11 Jul 2002 02:51:28 +0200 (CEST) (envelope-from root) Date: Thu, 11 Jul 2002 02:51:27 +0200 From: Cyrille Lefevre To: Mark Valentine Cc: Dag-Erling Smorgrav , Wes Peters , Dan Moschuk , arch@freebsd.org Subject: Re: Package system flaws? Message-ID: <20020711005127.GB82744@gits.dyndns.org> References: <200207071546.g67FkVIL018923@dotar.thuvia.org> <20020708020907.GB83084@gits.dyndns.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020708020907.GB83084@gits.dyndns.org> User-Agent: Mutt/1.3.99i Organization: ACME X-Face: V|+c;4!|B?E%BE^{E6);aI.[< List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG message resent due to mta misconfigation. On Mon, Jul 08, 2002 at 04:09:07AM +0200, Cyrille Lefevre wrote: > On Sun, Jul 07, 2002 at 04:46:31PM +0100, Mark Valentine wrote: > [snip] > > However, the archive members are _not_ the individual files comprising > > the package. Instead, the package is split into logical sub-packages > > (there may be only one, that's fine), such as the core run-time package, > > language sets, development headers and libraries, end-user documentation. > > Each sub-package is stored as a compressed archive (.tar.bz2 or whatever). > > The _package_ archive comprises the package metadata files (optionally > > compressed - useful for the larger files such as +CONTENTS) and the > > sub-package(s), each as a package archive member. > > that's exactly what HP-UX packages (depots) are w/ using another level. > > BUNDLE.PRODUCT.FILESET > > a bundle may contain one or more products which may contain one or more > filesets. usually, depot are just in the form of PRODUCT.FILESET where > PRODUCT is something like `tar' and FILESETS are something like `core', > `man', `lib', `shlib', `doc', etc. w/ dependencies between them. > > > The idea here is to allow individual access to those parts of the package > > that are needed individually, whilst gaining the benefits of compressing > > major portions of the package as collections of files (to which individual > > access is not generally required). > > > > While the division of an existing port into sub-packages requires effort, > > it's necessary only to gain the benefits of optionally-installable sub- > > packages. > > > > For existing ports, and for the majority of ports which will never likely be > > sub-package, pkg_create simple creates the package archive using the existing > > metadata files and a single compressed archive of the packages files. > [snip] > > IMHO, it would be better to provide each sub-package as it's own rather > to put them all in a single archive. so, you have more granularity on > what to download. if I just want tar-core and tar-man, I don't have to > fill my disk w/ tar-doc, etc. > > > The package install tools would allow the user to select between optional > > sub-packages. The base sub-package may be marked as "required". If there > > are only "required" sub-packages, or the user elects to install "all" or > > "only required" sub-packages, no dialog is necessary. And so on.. > > last year, I propose to port the SVR4 pkg commands to FreeBSD but no > one was interrested. so, I give up. Cyrille. -- Cyrille Lefevre mailto:cyrille.lefevre@laposte.net To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 19:18:19 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D212D37B400 for ; Wed, 10 Jul 2002 19:18:15 -0700 (PDT) Received: from softweyr.com (softweyr.com [65.88.244.127]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2B68243E3B for ; Wed, 10 Jul 2002 19:18:15 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from nextgig-3.customer.nethere.net ([209.132.102.163] helo=softweyr.com) by softweyr.com with esmtp (Exim 3.35 #1) id 17STWq-000O1M-00; Wed, 10 Jul 2002 20:18:00 -0600 Message-ID: <3D2CEBCE.55DC3C6D@softweyr.com> Date: Wed, 10 Jul 2002 19:22:06 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Cy Schubert - CITS Open Systems Group Cc: Terry Lambert , arch@FreeBSD.ORG Subject: Re: Package system wishlist References: <200207101459.g6AExQfP034695@cwsys.cwsent.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Cy Schubert - CITS Open Systems Group wrote: > > In message <3D2BE142.E25CA9BC@mindspring.com>, Terry Lambert writes: > > So, following Jordan's advice, what's on everyone's wishlist? > > > > Terry's Wishlist: > [...] > > + Cy's Wishlist: > > o Optional installation of sources. RH's SRPM's is a very poor > example of this. A better example would be what IBM does to > install JES/2 on their MVS system, e.g. an OpenSSH package might > contain source in addition to binaries. The sources would be > installed in /usr/src while the binaries would be installed > in /usr/bin, sbin.... Yes! My mythical XML metadata format, with or without external "filesets", would handle this with aplomb. The source set would be included in the metadata and you could skip it or install it as with any other fileset. Come to think of it, you could include the ports Makefile and patches as well. Hmm, that bears some thinking about. Most of what is in a "port" right now is metadata too. > o Files replaced by a package backed up in case of package removal I'm not sure what you mean here. Be able to create a backup script of the files related to a package for backing up? Be able to restore only missing files from a package? Both seem like good ideas... > o Check option: Tell me what it will do without doing it > > o Group option: Install prerequisites Wouldn't you want this to be the default, perhaps with an option to abort if they're not "readily available"??? > o Groupextend option: Install postrequisites, e.g. dependent > packages and patches In other words, roll portupgrade into the system. > o Ability to install my own packages on top of packages and > patches, I like to call them USERMODS. Your own packages or your changes to a standard package? I can see the value, but how to do it doesn't leap immediately to mind. > o The package system should be independent of the compression tool > used. In the future new compression algorithms and tools will > be developed. The package system should be flexible enough to > not care how its files are compressed or packaged. Ditto for archive formats, encoding formats, etc. We should probably specify one of each as a bare minimum, choosing from those that are available in library format, reasonably licensed, and have acceptable performance (for some definition of acceptable). > o The ability to export and import the package database (currently > to clone systems, I rsync /usr/local, /usr/X11R6, and /var/db/pkg > to a new system I am installing, this saves many hours of work). Yes, perhaps even the ability to capture a currently installed package and turn it back into a package file. That'd be way cool for duplicating packages with local customizations. > > o I want to be able to remove system components, like "sendmail" > > and "OpenSSH". > > Ideally everything should install as a package, however that would > create a lot of extra work for us developers. I have yet to think of a > painless way to do this. Yeah, Debian has certainly showed us how NOT to do it. "Which version of /bin/cat do you want?" -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 19:36:40 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4E6E337B400 for ; Wed, 10 Jul 2002 19:36:38 -0700 (PDT) Received: from goose.mail.pas.earthlink.net (goose.mail.pas.earthlink.net [207.217.120.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id D37B543E09 for ; Wed, 10 Jul 2002 19:36:37 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0317.cvx22-bradley.dialup.earthlink.net ([209.179.199.62] helo=mindspring.com) by goose.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17SToM-0001nf-00; Wed, 10 Jul 2002 19:36:06 -0700 Message-ID: <3D2CEE9C.D6EF2C0E@mindspring.com> Date: Wed, 10 Jul 2002 19:34:04 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Wes Peters Cc: Cy Schubert - CITS Open Systems Group , arch@FreeBSD.ORG Subject: Re: Package system wishlist References: <200207101459.g6AExQfP034695@cwsys.cwsent.com> <3D2CEBCE.55DC3C6D@softweyr.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Wes Peters wrote: > > o Files replaced by a package backed up in case of package removal > > I'm not sure what you mean here. Be able to create a backup script of > the files related to a package for backing up? Be able to restore only > missing files from a package? Both seem like good ideas... o For the upgrade which is actually a downgrade, to kill it, e.g.: o Install "bob". o Install of "bob" overwrites "libc". o Deinstall "bob". o Old "libc" is restored. > > o Group option: Install prerequisites > > Wouldn't you want this to be the default, perhaps with an option to > abort if they're not "readily available"??? o Package Q that imports A, B, and C and exports X, Y, and Z, where Y does not depend on A, B, or C. Package R depends on Q::R, rather than the whole of Q. My first impulse would be: "Break up package Q, rather than hacking up package R to not pull in Q's dependencies". On the other hand, this is a legitimate thing to do, if you want something to work, and don't care how, so making it impossible to do would probably be bad. > > o Groupextend option: Install postrequisites, e.g. dependent > > packages and patches > > In other words, roll portupgrade into the system. Yes. In case it wasn't clear: o No more "portupgrade" o No more "mergemaster" > > o The ability to export and import the package database (currently > > to clone systems, I rsync /usr/local, /usr/X11R6, and /var/db/pkg > > to a new system I am installing, this saves many hours of work). > > Yes, perhaps even the ability to capture a currently installed package > and turn it back into a package file. That'd be way cool for duplicating > packages with local customizations. o Ability to capture anything that didn't come in from a package, +/- specified files, plus specified package dependencies, as a package (e.g. "Terry's mail server configuration", "Terry's DNS server configuration", etc.. 8-)). (this one would be kind of cool... bento, et. al. would not need to be set up to do all the builds in a jail to be able to distinguish package components). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 23:46:38 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7282F37B400; Wed, 10 Jul 2002 23:46:35 -0700 (PDT) Received: from softweyr.com (softweyr.com [65.88.244.127]) by mx1.FreeBSD.org (Postfix) with ESMTP id AF4E743E4A; Wed, 10 Jul 2002 23:46:34 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from nextgig-2.customer.nethere.net ([209.132.102.162] helo=softweyr.com) by softweyr.com with esmtp (Exim 3.35 #1) id 17SXiU-000O8N-00; Thu, 11 Jul 2002 00:46:18 -0600 Message-ID: <3D2D2ABB.EA5889A0@softweyr.com> Date: Wed, 10 Jul 2002 23:50:35 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Cy Schubert - CITS Open Systems Group Cc: Dag-Erling Smorgrav , Doug Barton , Dan Moschuk , arch@FreeBSD.ORG Subject: Re: Package system flaws? References: <200207091343.g69DhOfP080432@cwsys.cwsent.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Cy Schubert - CITS Open Systems Group wrote: > > In message , Dag-Erling Smorgrav > writes: > > Wes Peters writes: > > > If you're sensible about the encoding, the metadata doesn't have to be > > > at the beginning of the media. > > > > Yes, it does, to allow installation from non-rewindable streams. No, it doesn't, you just have to make sure that the metadata for each chunk gets downloaded in time to decide if you want to skip over the chunk or not. The idea I like the best is to have the filesets (picture a .tar.gz or a .zip here) be external references on the fileserver; the XML contains all the metadata and URLs for the filesets. As you fetch the XML, or after you've fetched all of the XML, you fetch the filesets you're not skipping. Once you have the filesets on local storage, you can rewrite the URL references or convert them to in-line encoding, leaving the ones that have been skipped in the original URL encoding. For instance: Now assume you specified you want to install only the en_UK language files. pkg_add would leave the en_US and fr_FR as external references, download the binaries, man pages, and en_UK filesets, and convert those three into local file references OR directly encode them into the package file. > IMO, this is extremely important. Yup, you just have to be sensible about the encoding. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jul 10 23:51: 1 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 97C4737B400; Wed, 10 Jul 2002 23:50:59 -0700 (PDT) Received: from softweyr.com (softweyr.com [65.88.244.127]) by mx1.FreeBSD.org (Postfix) with ESMTP id E1D0243E4A; Wed, 10 Jul 2002 23:50:58 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from nextgig-9.customer.nethere.net ([209.132.102.169] helo=softweyr.com) by softweyr.com with esmtp (Exim 3.35 #1) id 17SXms-000O8W-00; Thu, 11 Jul 2002 00:50:51 -0600 Message-ID: <3D2D2BC9.7C3D1F82@softweyr.com> Date: Wed, 10 Jul 2002 23:55:05 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Mark Valentine Cc: Doug Barton , Garrett Wollman , arch@freebsd.org Subject: Re: Package system flaws? References: <200207092255.g69Mt3n8011431@dotar.thuvia.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Mark Valentine wrote: > > > From: Wes Peters > > Date: Mon 8 Jul, 2002 > > Subject: Re: Package system flaws? > > > Mark Valentine wrote: > > > Example 1: simple package, not sub-packaged. > > > > > > $ ls /var/spool/pkg/foo-x.y > > > base.bz2 package.xml > > > > Ick. Why not have the XML include the base.bz2 file in whatever encoding > > (including direct binary) we deem appropriate? > > That requires special tools to extract base.bz2 conveniently. Uh, yeah, that tool is "pkg_add". > > If you want truly > > minimal package sizes, specify the blob(s) as external URLs rather than > > encoding them. > > That's effectively identical to the behaviour implicit in my proposal. > > > Come to think if it, it would be a simple transformation to "convert" a > > package from external references to a full binary, with something like > > a pkg_fetch command. It would read a package with external URLs for > > the filesets, fetch them, and re-write the package with the blobs > > encased. > > tar(1) is a simpler conversion. ;-) Tar sucks for trying to extract the metadata only. Give it up dude, tar just ain't where it's at for this. tar is fine for balling up groups of files, let's leave it at that. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jul 11 0:33:16 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6321537B400 for ; Thu, 11 Jul 2002 00:33:12 -0700 (PDT) Received: from postfix3-2.free.fr (postfix3-2.free.fr [213.228.0.169]) by mx1.FreeBSD.org (Postfix) with ESMTP id AA56C43E09 for ; Thu, 11 Jul 2002 00:33:11 -0700 (PDT) (envelope-from rsidd@lpt.ens.fr) Received: from bluerondo.a.la.turk (nas-cbv-11-62-147-120-195.dial.proxad.net [62.147.120.195]) by postfix3-2.free.fr (Postfix) with ESMTP id CBB6718119 for ; Thu, 11 Jul 2002 09:33:09 +0200 (CEST) Received: (qmail 521 invoked by uid 1001); 11 Jul 2002 07:31:05 -0000 Date: Thu, 11 Jul 2002 09:31:05 +0200 From: Rahul Siddharthan To: Cyrille Lefevre Cc: John Baldwin , arch@freebsd.org, Dan Nelson Subject: Other BSD's (was Re: Cleaning old packages (was: Package system flaws?)) Message-ID: <20020711073105.GB264@lpt.ens.fr> References: <20020709161953.GA69779@lpt.ens.fr> <20020709171417.GA69932@lpt.ens.fr> <20020709231820.GA49510@gits.dyndns.org> <20020711005247.GE82744@gits.dyndns.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020711005247.GE82744@gits.dyndns.org> User-Agent: Mutt/1.3.27i X-Operating-System: FreeBSD 4.6-PRERELEASE i386 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Cyrille Lefevre said on Jul 11, 2002 at 02:52:47: > > That seems rather ambitious, and too drastic a change, to me. What > > I'd like to see is probably more like, the gfoo port needs gtk+ 1.2.6 > > or above, but not gtk+ 2.0 and above (incompatible) and not gtk+ 1.2.5 > > or below (buggy). There should be some way to specify this in the > > makefile of the port, so that any port-management program like > > portupgrade can make use of the information. > > take a look at NetBSD pkg (aka ports) system, they have this kind > of version handling. > > > (3) Automatically generate the +CONTENTS file by first doing a "fake" > > install in a temporary directory (assuming the port honours > > $PREFIX), then moving the contents to their final destination (a la > > gentoo). If your temporary location is on the same filesystem as > > the final one, it won't even take additional disk space, and very > > little additional time. Is there any obvious drawback with doing > > this? > > that's what OpenBSD port system has... Hm, then, why not steal those ideas? It shouldn't be too hard and would solve a few problems. Some other things I like about gentoo are 1. it supports multiple versions of the same port in the tree, and in the case of the latest-and-greatest versions which have potential problems, installing it can be "blocked" by the maintainer in a package.mask file, but it's still available for those who want to try it (they have to edit package.mask first). 2. you can do a "pretend" install, which will tell you which dependencies and version numbers would be installed (ignoring the ones which are already installed on your system), but not do anything. 3. You can fetch all the source tarballs (including dependencies, but excluding what's already installed) in one shot, before building anything. Are there such features in the other BSDs? In principle I think (1) could be done with differently named makefiles for different versions, and (2) could be implemented within the existing ports system. DES pointed out that (3) can be done by his porteasy. On that topic, with all the talk of ambitious new ports systems, does it make sense to go the route of, eg, openpackages (http://www.openpackages.org)? Perhaps they've already thought about many of these issues, and if one wants cross-architecture across FreeBSD, maybe it makes sense to go cross-BSD as well. The project seems to be dead or asleep (last webpage update 31 July 2001). - Rahul To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jul 11 1:33:27 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6CF6737B400 for ; Thu, 11 Jul 2002 01:33:23 -0700 (PDT) Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id ABF8F43E65 for ; Thu, 11 Jul 2002 01:33:22 -0700 (PDT) (envelope-from des@ofug.org) Received: by flood.ping.uio.no (Postfix, from userid 2602) id F00D8534A; Thu, 11 Jul 2002 10:33:20 +0200 (CEST) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: rsi@panix.com Cc: Rahul Siddharthan , Thierry Herbelot , arch@freebsd.org Subject: Re: Listening to users [was Re: Package system wishlist] References: <20020710224000.GA1331@lpt.ens.fr> <200207102307.g6AN7SV22593@panix1.panix.com> From: Dag-Erling Smorgrav Date: 11 Jul 2002 10:33:20 +0200 In-Reply-To: <200207102307.g6AN7SV22593@panix1.panix.com> Message-ID: Lines: 24 User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.2 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Rajappa Iyer writes: > Rahul Siddharthan writes: > > XFree86 -- FreeBSD uses it too, but the linuxen have better > > configurators, and possibly better default font setups, these days. > > So FreeBSD is actually worse off. > I don't know if any Linux distribution has a different font setup. Most of them set up a font server by default, which gives better and smoother scaling. On FreeBSD, the font server has to be set up manually, which requires figuring out the undocumented "magic" font path (unix/:7100) if you want to use a Unix socket instead of a TCP socket (recommended for security reasons, though our docs don't tell you that, just like they don't tell you how to stop X from listening for TCP connections, or even that it would be a good idea to do so), then editing /etc/X11/XF86Config and writing a startup script. Worse, the new "bits-and-pieces" XFree86-4 port does not even install default configuration files in /etc/X11 any more, so you have to locate a sample xfs configuration, customize it, and figure out where to put it (/etc/X11/fs/config). DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jul 11 1:34:46 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8FF6837B400; Thu, 11 Jul 2002 01:34:44 -0700 (PDT) Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id 12DFE43E31; Thu, 11 Jul 2002 01:34:44 -0700 (PDT) (envelope-from des@ofug.org) Received: by flood.ping.uio.no (Postfix, from userid 2602) id 02172534B; Thu, 11 Jul 2002 10:34:42 +0200 (CEST) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Rahul Siddharthan Cc: Cyrille Lefevre , John Baldwin , arch@freebsd.org, Dan Nelson Subject: Re: Other BSD's (was Re: Cleaning old packages (was: Package system flaws?)) References: <20020709161953.GA69779@lpt.ens.fr> <20020709171417.GA69932@lpt.ens.fr> <20020709231820.GA49510@gits.dyndns.org> <20020711005247.GE82744@gits.dyndns.org> <20020711073105.GB264@lpt.ens.fr> From: Dag-Erling Smorgrav Date: 11 Jul 2002 10:34:42 +0200 In-Reply-To: <20020711073105.GB264@lpt.ens.fr> Message-ID: Lines: 17 User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.2 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Rahul Siddharthan writes: > 2. you can do a "pretend" install, which will tell you which > dependencies and version numbers would be installed (ignoring > the ones which are already installed on your system), but not do > anything. # porteasy -le > > 3. You can fetch all the source tarballs (including dependencies, but > excluding what's already installed) in one shot, before building > anything. # porteasy -f DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jul 11 3: 5:55 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4BE1337B401 for ; Thu, 11 Jul 2002 03:05:52 -0700 (PDT) Received: from mail.nsu.ru (mx.nsu.ru [193.124.215.71]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2922543E31 for ; Thu, 11 Jul 2002 03:05:50 -0700 (PDT) (envelope-from danfe@regency.nsu.ru) Received: from drweb by mail.nsu.ru with drweb-scanned (Exim 3.20 #1) id 17Saoh-00039W-00; Thu, 11 Jul 2002 17:04:55 +0700 Received: from regency.nsu.ru ([193.124.210.26]) by mail.nsu.ru with esmtp (Exim 3.20 #1) id 17Saog-00039C-00; Thu, 11 Jul 2002 17:04:54 +0700 Received: from regency.nsu.ru (localhost [127.0.0.1]) by regency.nsu.ru (8.12.4/8.12.4) with ESMTP id g6BA45nP049154; Thu, 11 Jul 2002 17:04:05 +0700 (NOVST) (envelope-from danfe@regency.nsu.ru) Received: (from danfe@localhost) by regency.nsu.ru (8.12.4/8.12.4/Submit) id g6BA3P0L049079; Thu, 11 Jul 2002 17:03:25 +0700 (NOVST) Date: Thu, 11 Jul 2002 17:03:25 +0700 From: Alexey Dokuchaev To: Terry Lambert Cc: Cy Schubert - CITS Open Systems Group , arch@freebsd.org Subject: Re: Package system wishlist Message-ID: <20020711170325.A44656@regency.nsu.ru> References: <200207101459.g6AExQfP034695@cwsys.cwsent.com> <20020710223309.A69788@regency.nsu.ru> <3D2C9A5C.B5701103@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <3D2C9A5C.B5701103@mindspring.com>; from tlambert2@mindspring.com on Wed, Jul 10, 2002 at 01:34:36PM -0700 X-Envelope-To: tlambert2@mindspring.com, Cy.Schubert@uumail.gov.bc.ca, arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Jul 10, 2002 at 01:34:36PM -0700, Terry Lambert wrote: > Are you sure you don't work for Microsoft or Linux? ;^). Hell no, I despise both! Why, did I sound that way? 8-) > > The way "brain-storming" works is to capture all ideas, whether > you agree with them or not. After you capture all the ideas, > you then switch from "brain-storming" to "collation and analysis". > > So it's important to not comment on other people's ideas right > now, but to offer your own. OK, I'm sorry, I was just hoping that my comments would induce others to come up with more ideas, thus generating some sort of chain reaction, as to further extend your (definitely good) method of "brain-storming". ./danfe To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jul 11 3: 6:14 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C431137B400 for ; Thu, 11 Jul 2002 03:06:06 -0700 (PDT) Received: from postfix2-2.free.fr (postfix2-2.free.fr [213.228.0.140]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1FA0743E42 for ; Thu, 11 Jul 2002 03:06:06 -0700 (PDT) (envelope-from rsidd@lpt.ens.fr) Received: from bluerondo.a.la.turk (nas-cbv-2-62-147-133-252.dial.proxad.net [62.147.133.252]) by postfix2-2.free.fr (Postfix) with ESMTP id E39175F964 for ; Thu, 11 Jul 2002 12:06:03 +0200 (CEST) Received: (qmail 2420 invoked by uid 1001); 11 Jul 2002 10:04:20 -0000 Date: Thu, 11 Jul 2002 12:04:20 +0200 From: Rahul Siddharthan To: Dag-Erling Smorgrav Cc: rsi@panix.com, Thierry Herbelot , doc@freebsd.org Subject: Re: Listening to users [was Re: Package system wishlist] Message-ID: <20020711100420.GB2212@lpt.ens.fr> References: <20020710224000.GA1331@lpt.ens.fr> <200207102307.g6AN7SV22593@panix1.panix.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.3.27i X-Operating-System: FreeBSD 4.6-PRERELEASE i386 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG [followups to -doc] Dag-Erling Smorgrav said on Jul 11, 2002 at 10:33:20: > Rajappa Iyer writes: > > Rahul Siddharthan writes: > > > XFree86 -- FreeBSD uses it too, but the linuxen have better > > > configurators, and possibly better default font setups, these days. > > > So FreeBSD is actually worse off. > > I don't know if any Linux distribution has a different font setup. > > Most of them set up a font server by default, which gives better and > smoother scaling. Can you use a fontserver with Xft (needed for anti-aliased fonts)? If so, what are the required entries in /etc/X11/XftConfig? > On FreeBSD, the font server has to be set up > manually, which requires figuring out the undocumented "magic" font > path (unix/:7100) if you want to use a Unix socket instead of a TCP > socket (recommended for security reasons, though our docs don't tell > you that, just like they don't tell you how to stop X from listening > for TCP connections, or even that it would be a good idea to do so), > then editing /etc/X11/XF86Config and writing a startup script. I could write a few words about it for the handbook, once I figure out how it plays with xft. However, there seems to be no problem running a font server at the same time as using kde with xft (with explicit font path entries in XftConfig). Much of the xft stuff in the font section of the handbook was written by me, and it's high time some of it was updated. The unix socket stuff should really be in the manpage too, as supplied by the XFree86 project, is there any reason it isn't? > Worse, the new "bits-and-pieces" XFree86-4 port does not even install > default configuration files in /etc/X11 any more, so you have to > locate a sample xfs configuration, customize it, and figure out where > to put it (/etc/X11/fs/config). What do the port maintainers say? I didn't know this, since I have a pre-4.2.0 cvs version of XFree86, which works fine so I didn't bother upgrading. - Rahul To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jul 11 7: 8:28 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2CA7C37B406 for ; Thu, 11 Jul 2002 07:08:24 -0700 (PDT) Received: from hotmail.com (vic-dial-196-31-177-29.mweb.co.za [196.31.177.29]) by mx1.FreeBSD.org (Postfix) with SMTP id 49D9A43E9B for ; Thu, 11 Jul 2002 07:07:41 -0700 (PDT) (envelope-from mikedube64@hotmail.com) From: "MIKENA DDUBELA" To: Subject: IN NEED OF HELP Mime-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Date: Thu, 11 Jul 2002 15:51:43 +0200 Reply-To: "MIKENA DDUBELA" Content-Transfer-Encoding: 8bit Message-Id: <20020711140741.49D9A43E9B@mx1.FreeBSD.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG FAX: 27-731485293. ATTN: THE MANAGING DIRECTOR/C. E. O. TEL:27-825087950. DEAR SIR/MADAM, BUSINESS TRANSACTION. MY NAME IS MIKENA DUBELA THE SON OF DR MOSES DUBE, ONE OF THE GREAT FARMERS IN ZIMBABWE. MY FATHER HAPPENS TO BE ONE OF THE OPPONENTS OF THE PRESIDENT OF ZIMBABWE [ROBERT MUGABE] IN HIS POLITICAL AMBITIONS TO REMAIN IN POWER FOREVER, BECAUSE OF THIS, MY FATHER'S FARM WAS BURNT DOWN AND THE WHOLE ASSETS WAS LOOTED WHILE MY FATHER LOST HIS LIFE. WHEN WE DISCOVERED THAT OUR LIVES WAS NO LONGER SAFE IN ZIMBABWE, MY MOTHER AND I DECIDED TO FLEE TO SOUTH AFRICA WITH THE MONEY MY FATHER HID AWAY IN MY MOTHER'S HOUSE. THIS MONEY, WHICH IS OUR ONLY HOPE OF SURVIVING IS US$16.5M [SIXTEEN MILLION FIVE HUNDRED THOUSAND UNITED STATES DOLLARS ]. THIS MONEY IS NOW DEPOSITED IN A FINANCE & SECURITY COMPANY AS A BOX OF VALUABLE CONTAINING JEWELERIES, TO AVOID SABOTAGE. NOW, MY REFUGEE STATUS IN SOUTH AFRICA DOES NOT ALLOW ME TO HAVE A BANK ACCOUNT OR EMBARK ON ANY INVESTMENT AND AGAIN, THE CLOSENESS OF ZIMBABWE TO SOUTH AFRICA DOES NOT MAKE US FEEL VERY SAFE. I WANT YOUR HELP TO ASSIST ME MOVE THIS MONEY OUT OF SOUTH AFRICA TO YOUR FOREIGN ACCOUNTSO THAT I WILL COME OVER THERE TO YOUR COUNTRYTO INVEST THE MONEY. I, SURELY WILL REWARD YOUR EFFORT WITH 25% OF THE TOTAL AMOUNTWHILE 5% WILL COVER THE EXPENSES THAT MAY ARISE IN THE PROCESS OF THE TRANSFER OF THE MONEY. THOUGH, THERE IS NO RISK IN THIS BUSINESS BUT MAKE SURE THAT YOU DON'T DISCUSS THIS TRANSACTION WITH ANYONE. BEST REGARDS, MIKENA DUBELA. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jul 11 8: 1: 7 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A5DE237B43B; Thu, 11 Jul 2002 08:00:55 -0700 (PDT) Received: from prism.flugsvamp.com (66-191-112-47.mad.wi.charter.com [66.191.112.47]) by mx1.FreeBSD.org (Postfix) with ESMTP id 893EE43E31; Thu, 11 Jul 2002 08:00:53 -0700 (PDT) (envelope-from jlemon@flugsvamp.com) Received: (from jlemon@localhost) by prism.flugsvamp.com (8.11.6/8.11.6) id g6BExDa58451; Thu, 11 Jul 2002 09:59:13 -0500 (CDT) (envelope-from jlemon) Date: Thu, 11 Jul 2002 09:59:13 -0500 From: Jonathan Lemon To: Archie Cobbs Cc: Jonathan Lemon , John Baldwin , davidx@viasoft.com.cn, freebsd-arch@FreeBSD.ORG, julian@elischer.org Subject: Re: Timeout and SMP race Message-ID: <20020711095913.H65393@prism.flugsvamp.com> References: <20020710171552.F65393@prism.flugsvamp.com> <200207110025.g6B0PHA30341@arch20m.dellroad.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0pre2i In-Reply-To: <200207110025.g6B0PHA30341@arch20m.dellroad.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Jul 10, 2002 at 05:25:17PM -0700, Archie Cobbs wrote: > Jonathan Lemon writes: > > > extern int timer_start(timer_handle_t *handlep, mutex_t *mutexp, > > > timer_func_t tfunc, void *arg, u_int delay, > > > int flags); > > > extern void timer_cancel(timer_handle_t *handlep); > > > extern int timer_remaining(timer_handle_t handle, u_int *delayp); > > > > It seems to me that we can achieve the same functionality without > > changing the callout API too much. You mention that the mutex (if supplied) > > will be acquired before calling tfunc. This means that it has to be > > stored somewhere, presumably in the callout structure itself. > > > > The callout() consumers typically allocate their own storage, so perhaps > > we should add an (optional) pointer to a mutex to the callout structure, > > where the mutex is obtained/released before the callout function is made. > > Yep, that would work too.. essentially it's the same thing. > > If you're doing that, why not just store the mutex itself in the > callout structure, rather than a pointer to it? I guess if you did > that then you would then need some kind of flag that says whether > to use it or not. Or.. maybe there would be some way for the timer > code to tell if the mutex has been initialized or not, and use this > to decide whether to use it or not? Well, the other day, I was thinking we have this current API: callout_init(struct callout *c, int mpsafe) where 'mpsafe' indicates whether the callout code should grab the Giant lock or not. It wouldn't seem too much of a stretch to change this to: callout_init(struct callout *c, struct mtx *m) (perhaps redundant, since *m will be stored in the callout structure). Now, this removes the special cases dealing with 'Giant' lock in the softclock, since those callers who need the Giant lock would be able to specify it themselves. For TCP timeouts, all of the timeouts would point to the lock on the structure: /* cut down pseudocode */ struct inp_tp { struct mtx inp_mtx; struct callout inp_tp_rexmt; struct callout inp_tp_persist; struct callout inp_tp_keep; struct callout inp_tp_delack; } callout_init(&inp->inp_tp_rexmt, &inp->inp_mtx); callout_init(&inp->inp_tp_persist, &inp->inp_mtx); callout_init(&inp->inp_tp_keep, &inp->inp_mtx); callout_init(&inp->inp_tp_delack, &inp->inp_mtx); So TCP code, which grabs the inp_mtx for the duration, would be protected against the timer, and no additional locking is needed. The timer code would grab the appropriate lock before calling the timeout. I wouldn't want an individual mutex in each timer, as by storing a pointer, I can obtain/protect the appropriate resource, which extends further than just the callout structure. -- Jonathan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jul 11 9:58: 4 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C9CE637B400; Thu, 11 Jul 2002 09:57:59 -0700 (PDT) Received: from thuvia.demon.co.uk (thuvia.demon.co.uk [193.237.34.248]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1208543E67; Thu, 11 Jul 2002 09:57:58 -0700 (PDT) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (dotar.thuvia.org [10.0.0.4]) by phaidor.thuvia.org (8.11.6/8.11.6) with ESMTP id g6BGvhb29937; Thu, 11 Jul 2002 17:57:43 +0100 (BST) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (localhost [IPv6:::1]) by dotar.thuvia.org (8.12.3/8.12.3) with ESMTP id g6BGvgKC071736; Thu, 11 Jul 2002 17:57:42 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.3/8.12.3/Submit) id g6BGvg7B071735; Thu, 11 Jul 2002 17:57:42 +0100 (BST) Date: Thu, 11 Jul 2002 17:57:42 +0100 (BST) From: Mark Valentine Message-Id: <200207111657.g6BGvg7B071735@dotar.thuvia.org> In-Reply-To: Cyrille Lefevre's message of Jul 11, 2:51am X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: Cyrille Lefevre Subject: Re: Package system flaws? Cc: Dag-Erling Smorgrav , Wes Peters , Dan Moschuk , arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: Cyrille Lefevre > Date: Thu 11 Jul, 2002 > Subject: Re: Package system flaws? > > IMHO, it would be better to provide each sub-package as it's own rather > > to put them all in a single archive. so, you have more granularity on > > what to download. if I just want tar-core and tar-man, I don't have to > > fill my disk w/ tar-doc, etc. My examples did show the parts stored seperately, but mentioned bindling them in a single archive as an option. Cheers, Mark. -- Mark Valentine, Thuvia Labs "Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses "We're kind of stupid that way." *munch* *munch* and endorses FreeBSD -- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jul 11 10:15:22 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 43F1237B400; Thu, 11 Jul 2002 10:15:18 -0700 (PDT) Received: from thuvia.demon.co.uk (thuvia.demon.co.uk [193.237.34.248]) by mx1.FreeBSD.org (Postfix) with ESMTP id 507C543E65; Thu, 11 Jul 2002 10:15:16 -0700 (PDT) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (dotar.thuvia.org [10.0.0.4]) by phaidor.thuvia.org (8.11.6/8.11.6) with ESMTP id g6BHF6b30035; Thu, 11 Jul 2002 18:15:07 +0100 (BST) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (localhost [IPv6:::1]) by dotar.thuvia.org (8.12.3/8.12.3) with ESMTP id g6BHF6KC072301; Thu, 11 Jul 2002 18:15:06 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.3/8.12.3/Submit) id g6BHF6nb072300; Thu, 11 Jul 2002 18:15:06 +0100 (BST) Date: Thu, 11 Jul 2002 18:15:06 +0100 (BST) From: Mark Valentine Message-Id: <200207111715.g6BHF6nb072300@dotar.thuvia.org> In-Reply-To: Wes Peters's message of Jul 10, 11:55pm X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: Wes Peters Subject: Re: Package system flaws? Cc: Doug Barton , Garrett Wollman , arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: Wes Peters > Date: Wed 10 Jul, 2002 > Subject: Re: Package system flaws? > > > Ick. Why not have the XML include the base.bz2 file in whatever encoding > > > (including direct binary) we deem appropriate? > > > > That requires special tools to extract base.bz2 conveniently. > > Uh, yeah, that tool is "pkg_add". It has side effects, though it may be the case that it has sufficient options to turn them all off. > Tar sucks for trying to extract the metadata only. I only used tar as one particular bundling example, but it seems to serve the purpose with: tar --extract --to-stdout --fast-read --file foo-x.y.tar package.xml > tar is fine for balling up groups of files, let's leave it at that. That's all I was using for. Note that my updated examples showed a scheme identical to the variant of your proposal which used external file references, which I like better. The bundled variant was simply intended to make transport of packages more convenient, at the expense of convenient direct access to the files. Cheers, Mark. -- Mark Valentine, Thuvia Labs "Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses "We're kind of stupid that way." *munch* *munch* and endorses FreeBSD -- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jul 11 10:42: 3 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6D02137B400; Thu, 11 Jul 2002 10:41:58 -0700 (PDT) Received: from thuvia.demon.co.uk (thuvia.demon.co.uk [193.237.34.248]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0434D43E3B; Thu, 11 Jul 2002 10:41:56 -0700 (PDT) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (dotar.thuvia.org [10.0.0.4]) by phaidor.thuvia.org (8.11.6/8.11.6) with ESMTP id g6BHffb30164; Thu, 11 Jul 2002 18:41:41 +0100 (BST) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (localhost [IPv6:::1]) by dotar.thuvia.org (8.12.3/8.12.3) with ESMTP id g6BHffKC073048; Thu, 11 Jul 2002 18:41:41 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.3/8.12.3/Submit) id g6BHffn8073047; Thu, 11 Jul 2002 18:41:41 +0100 (BST) Date: Thu, 11 Jul 2002 18:41:41 +0100 (BST) From: Mark Valentine Message-Id: <200207111741.g6BHffn8073047@dotar.thuvia.org> In-Reply-To: Wes Peters's message of Jul 11, 6:53am X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: wes@softweyr.com (Wes Peters), Cy Schubert - CITS Open Systems Group Subject: Re: Package system flaws? Cc: Dag-Erling Smorgrav , Doug Barton , Dan Moschuk , arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: wes@softweyr.com (Wes Peters) > Date: Thu 11 Jul, 2002 > Subject: Re: Package system flaws? > The idea I like the best is to have the filesets (picture a .tar.gz > or a .zip here) be external references on the fileserver; the XML > contains all the metadata and URLs for the filesets. As you fetch > the XML, or after you've fetched all of the XML, you fetch the > filesets you're not skipping. Once you have the filesets on local > storage, you can rewrite the URL references or convert them to > in-line encoding, leaving the ones that have been skipped in the > original URL encoding. This is equivalent to my proposal, except that I don't see much value in converting to the in-line encoding, and my method specifies a directory name (or any suitable simple archive name) containing a package.xml and compressed file sets, whereas you presumably expect the XML file to be the primary referenced object. > For instance: > > > > > > > > My equivalent would be: or: name=cat bin=bin.tar.gz man=man.tar.gz lang[en_US]=en_US.tar.gz lang[en_UK]=en_UK.tar.gz lang[fr_FR]=fr_FR.tar.gz according to taste. > Now assume you specified you want to install only the en_UK language > files. pkg_add would leave the en_US and fr_FR as external references, > download the binaries, man pages, and en_UK filesets, and convert those > three into local file references OR directly encode them into the > package file. Same except that I don't need the last step, though I could optionally bundle up for archiving purposes. Cheers, Mark. -- Mark Valentine, Thuvia Labs "Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses "We're kind of stupid that way." *munch* *munch* and endorses FreeBSD -- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jul 11 11:29: 9 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B7D3037B400 for ; Thu, 11 Jul 2002 11:29:07 -0700 (PDT) Received: from smtp.noos.fr (camus.noos.net [212.198.2.70]) by mx1.FreeBSD.org (Postfix) with ESMTP id F192E43E52 for ; Thu, 11 Jul 2002 11:29:05 -0700 (PDT) (envelope-from root@gits.dyndns.org) Received: (qmail 40112287 invoked by uid 0); 11 Jul 2002 18:29:04 -0000 Received: from unknown (HELO gits.gits.dyndns.org) ([212.198.229.153]) (envelope-sender ) by 212.198.2.70 (qmail-ldap-1.03) with SMTP for ; 11 Jul 2002 18:29:04 -0000 Received: from gits.gits.dyndns.org (5a714h5s0kbmio49@localhost [127.0.0.1]) by gits.gits.dyndns.org (8.12.5/8.12.5) with ESMTP id g6BIT4TL021151; Thu, 11 Jul 2002 20:29:04 +0200 (CEST) (envelope-from root@gits.dyndns.org) Received: (from root@localhost) by gits.gits.dyndns.org (8.12.5/8.12.5/Submit) id g6BIT3dm021150; Thu, 11 Jul 2002 20:29:03 +0200 (CEST) (envelope-from root) Date: Thu, 11 Jul 2002 20:29:03 +0200 From: Cyrille Lefevre To: Rahul Siddharthan Cc: John Baldwin , arch@freebsd.org, Dan Nelson Subject: Re: Other BSD's (was Re: Cleaning old packages (was: Package system flaws?)) Message-ID: <20020711182903.GD12246@gits.dyndns.org> References: <20020709161953.GA69779@lpt.ens.fr> <20020709171417.GA69932@lpt.ens.fr> <20020709231820.GA49510@gits.dyndns.org> <20020711005247.GE82744@gits.dyndns.org> <20020711073105.GB264@lpt.ens.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020711073105.GB264@lpt.ens.fr> User-Agent: Mutt/1.3.99i Organization: ACME X-Face: V|+c;4!|B?E%BE^{E6);aI.[< List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, Jul 11, 2002 at 09:31:05AM +0200, Rahul Siddharthan wrote: [snip] > Hm, then, why not steal those ideas? It shouldn't be too hard and don't know, try to search the -arch ml ? > would solve a few problems. > > Some other things I like about gentoo are > > 2. you can do a "pretend" install, which will tell you which > dependencies and version numbers would be installed (ignoring > the ones which are already installed on your system), but not do > anything. pkg_add -n ? (not tested) > On that topic, with all the talk of ambitious new ports systems, does > it make sense to go the route of, eg, openpackages > (http://www.openpackages.org)? Perhaps they've already thought about > many of these issues, and if one wants cross-architecture across > FreeBSD, maybe it makes sense to go cross-BSD as well. The project > seems to be dead or asleep (last webpage update 31 July 2001). as well as the ml which sleep since may 2002. Cyrille. -- Cyrille Lefevre mailto:cyrille.lefevre@laposte.net To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jul 11 12: 9:46 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 91B3437B400 for ; Thu, 11 Jul 2002 12:09:44 -0700 (PDT) Received: from dragon.nuxi.com (trang.nuxi.com [66.92.13.169]) by mx1.FreeBSD.org (Postfix) with ESMTP id C197843E31 for ; Thu, 11 Jul 2002 12:09:43 -0700 (PDT) (envelope-from obrien@NUXI.com) Received: from dragon.nuxi.com (obrien@localhost [127.0.0.1]) by dragon.nuxi.com (8.12.5/8.12.2) with ESMTP id g6BJ9foi098322; Thu, 11 Jul 2002 12:09:41 -0700 (PDT) (envelope-from obrien@dragon.nuxi.com) Received: (from obrien@localhost) by dragon.nuxi.com (8.12.5/8.12.5/Submit) id g6BJ9cOg098321; Thu, 11 Jul 2002 12:09:38 -0700 (PDT) Date: Thu, 11 Jul 2002 12:09:38 -0700 From: "David O'Brien" To: Dag-Erling Smorgrav Cc: rsi@panix.com, Rahul Siddharthan , Thierry Herbelot , arch@freebsd.org Subject: Re: Listening to users [was Re: Package system wishlist] Message-ID: <20020711190938.GF97950@dragon.nuxi.com> Reply-To: obrien@freebsd.org Mail-Followup-To: David O'Brien , Dag-Erling Smorgrav , rsi@panix.com, Rahul Siddharthan , Thierry Herbelot , arch@freebsd.org References: <20020710224000.GA1331@lpt.ens.fr> <200207102307.g6AN7SV22593@panix1.panix.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4i X-Operating-System: FreeBSD 5.0-CURRENT Organization: The NUXI BSD Group X-Pgp-Rsa-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Rsa-Keyid: 1024/34F9F9D5 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, Jul 11, 2002 at 10:33:20AM +0200, Dag-Erling Smorgrav wrote: > Worse, the new "bits-and-pieces" XFree86-4 port does not even install > default configuration files in /etc/X11 any more, so you have to > locate a sample xfs configuration, customize it, and figure out where > to put it (/etc/X11/fs/config). I think we need to replace the XFree86-4 maintainer. I had to make patches for the X server to configure properly on FreeBSD. The existing maintainer doesn't seem to have the time or interest in polishing XF-4 for FreeBSD. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jul 11 13:37:15 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9BFF737B400; Thu, 11 Jul 2002 13:37:13 -0700 (PDT) Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2981A43E42; Thu, 11 Jul 2002 13:37:13 -0700 (PDT) (envelope-from des@ofug.org) Received: by flood.ping.uio.no (Postfix, from userid 2602) id 8590B534A; Thu, 11 Jul 2002 22:37:11 +0200 (CEST) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: obrien@freebsd.org Cc: rsi@panix.com, Rahul Siddharthan , Thierry Herbelot , arch@freebsd.org Subject: Re: Listening to users [was Re: Package system wishlist] References: <20020710224000.GA1331@lpt.ens.fr> <200207102307.g6AN7SV22593@panix1.panix.com> <20020711190938.GF97950@dragon.nuxi.com> From: Dag-Erling Smorgrav Date: 11 Jul 2002 22:37:10 +0200 In-Reply-To: <20020711190938.GF97950@dragon.nuxi.com> Message-ID: Lines: 9 User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.2 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG "David O'Brien" writes: > I think we need to replace the XFree86-4 maintainer. We already have a candidate (anholt) and a patch that fixes a number of problems (http://people.freebsd.org/~anholt/files/x420diff-1). DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jul 11 15: 0:51 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9001C37B40F; Thu, 11 Jul 2002 15:00:41 -0700 (PDT) Received: from softweyr.com (softweyr.com [65.88.244.127]) by mx1.FreeBSD.org (Postfix) with ESMTP id DA49643E09; Thu, 11 Jul 2002 15:00:40 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from nextgig-8.customer.nethere.net ([209.132.102.168] helo=softweyr.com) by softweyr.com with esmtp (Exim 3.35 #1) id 17SlzF-000ObJ-00; Thu, 11 Jul 2002 16:00:33 -0600 Message-ID: <3D2E0100.C8958F07@softweyr.com> Date: Thu, 11 Jul 2002 15:04:48 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Archie Cobbs Cc: Dan Moschuk , Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: Package system flaws? References: <200207100310.g6A3AZB23117@arch20m.dellroad.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Archie Cobbs wrote: > > Dan Moschuk writes: > > I don't think using an archive format like zip would be a step in the > > right direction. If the package file format were to be redesigned, I would > > vote for a custom header prepended to a bziped tarball. > > tar has a limitation which I've encountered: suppose you have a port > that installs a man page with lots of references (i.e., hard linked > files with different names with a single underlying file). Then in > tar format, you get the same file copied N times. If we used cpio > instead (for example) then it "knows" how to handle hard links. > > So I'd say cpio is better than tar, though something else altogether > might be better than both. Tar and cpio both have anachronisms due to being "tape backup" utilities. I'd hope we could come up with some simple archive format that essentially encodes the inode data and file data and is smart about unfilled blocks. Make it into a library so it can be used by this AND OTHER utilities easily. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jul 11 15:26: 1 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C438337B400; Thu, 11 Jul 2002 15:25:08 -0700 (PDT) Received: from softweyr.com (softweyr.com [65.88.244.127]) by mx1.FreeBSD.org (Postfix) with ESMTP id DC26643E3B; Thu, 11 Jul 2002 15:25:07 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from nextgig-4.customer.nethere.net ([209.132.102.164] helo=softweyr.com) by softweyr.com with esmtp (Exim 3.35 #1) id 17SmMY-000OcY-00; Thu, 11 Jul 2002 16:24:39 -0600 Message-ID: <3D2E069E.1298B4B8@softweyr.com> Date: Thu, 11 Jul 2002 15:28:46 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Terry Lambert Cc: Jordan K Hubbard , Dan Nelson , Archie Cobbs , Dan Moschuk , Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: Package system flaws? References: <3D2BDBD5.CA71A2C1@mindspring.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Terry Lambert wrote: > > Jordan K Hubbard wrote: > > Oh dear, why do I find myself so unable to resist this thread? :) > > Occupational hazard? 8-). > > > I'm borrowing a bit of Macintosh nomenclature there (though I'm sure > > Terry will come along and correct me by pointing out that IBM was the > > first to introduce "Fat binaries" with VM/CMS or something :) but I'm > > sure people get the idea. If you're distributing an Emacs or TeX > > package which weighs in at some hefty percentage of the New York phone > > book in size, and with KDE and Gnome one doesn't even have to look to > > Emacs anymore for a good example of a "really big, honkin' package", you > > naturally want to save on disk space if at all possible both to minimize > > load on the archives and make those poor Australian users with their > > 9600 baud Telstra link to the US happy. Compression is certainly a > > good start, but when you start distributing those packages for 3 or 4 > > different architectures (as FreeBSD is definitely not far away from > > doing) you also would like to not distribute 3 or 4 different copies of > > the same architecture-neutral bits if you can possibly help it. That's > > where the idea of munging attributes into the dictionary namespace > > starts making more and more sense, and not just for representing > > different architectures but also thinks like "experimental vs stable" > > variants, "mix-ins" (like all the various versions of Apache which have > > various bits of compiled-in smarts) and what have you. If you introduce > > the concept of install-time attributes, some of which may be implicit > > (like architecture) and some of which may be explicit (like "give me the > > experimental version please"), you conceivably end up with mangled > > pathnames within the package which are demangled on the way out, > > C++-style. This allows you to have, say, just one copy of all the Emacs > > lisp files and documentation but 3 or 4 different "bin/emacs" files > > which don't collide internally and are properly selected for on > > extraction. > > Apple has dealt with this for the 68K vs. PPC binaries by stuffing > the binaries into the same package, as a unit, and putting them in > different "code resource" in the resource fork of the same file, > while sharing the "data fork" between the code. > > The historical canonical answer is "ANDF" -- Architecture Neutral > Distribution Format. > > In ANDF, one would compute the quad tree for a compiled program (for > example), but not do the code generation. Instead, one would store > the quad tree, and generate the actual code, at install time. > > ANDF is actually a brilliant idea, since the resulting program is not > source code. It also doesn't have the problem of the Apple approach, > which is bloating the file size by the unused portion(s) of the code > contained therein. > > Unfortunately, creation of intermediate languages is against the > policy of the FSF, due to it's ability to weaken the effect of the > GPL on programs. This is the same reason the data dictionary work > that was discussed on -advocacy and -chat recently is frowned upon > by RMS, and why he posted what he did about sweeping the research > under the rug to avoid exposure. > > At this point in time, ANDF is not an option because of license > politics having nothing to do with available technology. In point of fact, The Gnu Compiler suite *does* have an intermediate language that with some work could be grown into an ANDF language, and yet Mr. Stallman will not allow it so long as he has any control over GCC at all. But you knew that, didn't you? > I would like to think that a pure copy of the Apple approach was, > likewise, not an option. Specifically -- and it looks like this is > what you are actually advocating here -- extracting the portions of > the applicable binaries at installation time, rather than installing > the whole thing, is probably the most correct approach. It means > that you only end up with what you are worried about on your disk. It would also be nice if you can avoid DOWNLOADING the parts you're not going to install. > It also has the (dubious, in some people's eyes) attribute of making > it impossible to take an installed package and recreate the package > as a distribution -- you can't go to any machine with the software > installed on it, and simply plug in you iPirate or whatever, and > suck the package down to ensure redistributability to any target > platform supported by the original distribution. > > The one real drawback with this approach is linear media. That is, > if I have a DVD distribution, it's not a problem. But if I have a > 28K modem... it is. A CDROM distribution is mildy problematic, > since putting i386, PPC, SPARC, Alpha, and IA64 (and S/360? 8-)) > binaries all in the same package could require a lot of CDROMs. But > that's still less of a problem then linear media, like a TCP connection > stream or a magnetic tape. Right. Linear media introduces so many limitations on what you can do I think it's a valid point to discuss whether we want to support it or not. I just checked with my favorite screwdriver shop in SLC; the smallest disk drive they sell now is 40GB. Downloading the components onto local storage BEFORE you install isn't (or shouldn't be) an issue, and has other benefits as well. > There are two workarounds for this... assuming you can get the metadata > early, which means that it's at a known location in every file, which > pretty much dictates the front of the file. The first is that the FTP > protocol supports the concept of a "REGET", and does not check that the > receiver has the most recent version of the file ... it only cares about > the byte offset, AND it supports interruption of transfers. The second > is that the HTTP 1.1 protocol support HTTP GET with a range argument for > a byte range for a server object. Both of these options effectively > support random acces of known locations within a file, at a known offset. > > Not all FTP and HTTP servers support these facilities; enough do that it > might be OK to rely upon them. HTTP is particularly attractive, due to > firewall issues at large companies, where other protocols are blocked. I'm coming around to the idea of storing the blobs of file data as separate files, at least on the server. The program that fetches them onto the local system can interweave the filesets and metadata as needed, and this format can easily be placed on a CD or DVD as well. If the metadata includes a signature for the fileblob, we can be assured it is the correct blob of data. > > Anyway, wish-list items like this are why it's a good idea to define the > > goals first and the package format second. :-) > > Yes. I hope somebody is keeping track of all this. > > P.S. I also gree with jhb's assertion that some folks really need to > > take a good look at libh since it takes a number of things like this > > into account, including all the "occludes, obsoletes, upgrades, ..." > > attributes that people were recently demanding as package metadata. > > The current system uses libh. It does? What part of "the current system?" I know the pkg_ tools don't. > We would like to (or *I* would like to 8^)) > exceed the capabilities of the current system. Yes. > The thing that strikes me about libh is that there is a lot of human > effort involved in the dependency tracking mechanism, if it's going to > be possible to perform some of the relationship tracking that some of > the posters have already requested. > > In particular, I think that there is not sufficient differentiation > between "necessary" and "sufficient". > > Part of the problem there is that the ports system has always been > based on the idea that most of the things in it get built by the > user as needed, so no matter what, it's always "sufficient". Moving > to a system that can support binary-only implies that the implicit > guarantees that were there are no longer there, and you need to > consider that "just rebuild" will not be an option. Right. On problem in particular comes from changing the PREFIX of a package. You can easily do this in the package tools, but how do you change the BINARIES to realize they've been installed under another PREFIX? This leads us back to the idea of system-level environment variables, like VMS logicals. You could almost do this on UNIX now, by putting such settings into init's environment, if you could get the application developers to write their code (especially libraries) appropriately. > I don't mean to call the demands that were made incomplete or naieve, > but... well, yes I *do* mean to call them naieve, or at *least* call > them incomplete. 8-). > > Finally, I think that people often confuse design with implementation, > and that just because a system is *capable* of solving a particular > problem, that the initial implementation would have to be delayed > until it *actually solves the problem*. I have a really big wishlish, > and adding everyone's wishlist together yields a *huge* wishlist. It's > clear that implementing everything isn't possible at the present time. > But that doesn't mean that a design should not take this into account > as potential futuere work. This is precisely the problem OpenPackages fell into. > I guess we should start a seperate thread specifically for a "wishlist"; > no offense to the person who volunteered to collect and summarize this, > but I'd like to see the information captured without any editorializing > by a single person; I think it will be more useful as raw data. Ah, I can see value in that. Include my bit about system-wide environment variables too, then. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jul 11 17:56:33 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9233137B411; Thu, 11 Jul 2002 17:56:11 -0700 (PDT) Received: from thuvia.demon.co.uk (thuvia.demon.co.uk [193.237.34.248]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7FFF643E42; Thu, 11 Jul 2002 17:56:09 -0700 (PDT) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (dotar.thuvia.org [10.0.0.4]) by phaidor.thuvia.org (8.11.6/8.11.6) with ESMTP id g6C0tdb32184; Fri, 12 Jul 2002 01:55:39 +0100 (BST) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (localhost [IPv6:::1]) by dotar.thuvia.org (8.12.3/8.12.3) with ESMTP id g6C0tcKC084566; Fri, 12 Jul 2002 01:55:38 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.3/8.12.3/Submit) id g6C0tbpQ084565; Fri, 12 Jul 2002 01:55:37 +0100 (BST) Date: Fri, 12 Jul 2002 01:55:37 +0100 (BST) From: Mark Valentine Message-Id: <200207120055.g6C0tbpQ084565@dotar.thuvia.org> In-Reply-To: Wes Peters's message of Jul 11, 10:33pm X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: wes@softweyr.com (Wes Peters), Terry Lambert Subject: Re: Package system flaws? Cc: Jordan K Hubbard , Dan Nelson , Archie Cobbs , Dan Moschuk , Dag-Erling Smorgrav , arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: wes@softweyr.com (Wes Peters) > Date: Thu 11 Jul, 2002 > Subject: Re: Package system flaws? > On problem in particular comes from changing the PREFIX of a > package. You can easily do this in the package tools, but how do you > change the BINARIES to realize they've been installed under another > PREFIX? How about providing a packaging API call (and utility) for the packages to grab their install root from a record in /var/db/pkg? Something like pkg_getroot(3) and pkg_config(1). There probably isn't much less intrusive way to make a package's root configurable at install time. If you didn't want the package to have to look in /var/db/pkg, have the package binaries hold a maximum-length padded string variable (marked similarly to what(1) strings), and provide a tool to edit the binary (and scripts) at install time. I think environment variables are too fragile for this purpose. > You could almost do this on UNIX now, by putting such settings into > init's environment, if you could get the application developers to > write their code (especially libraries) appropriately. This doesn't sound like it would be easy to allow different packages to be installed with different roots. Cheers, Mark. -- Mark Valentine, Thuvia Labs "Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses "We're kind of stupid that way." *munch* *munch* and endorses FreeBSD -- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jul 11 18: 9:22 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7F0D337B400; Thu, 11 Jul 2002 18:09:20 -0700 (PDT) Received: from softweyr.com (softweyr.com [65.88.244.127]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0264143E6E; Thu, 11 Jul 2002 18:09:20 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from nextgig-9.customer.nethere.net ([209.132.102.169] helo=softweyr.com) by softweyr.com with esmtp (Exim 3.35 #1) id 17Sovo-000OgZ-00; Thu, 11 Jul 2002 19:09:12 -0600 Message-ID: <3D2E2D33.D09B8F16@softweyr.com> Date: Thu, 11 Jul 2002 18:13:23 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Doug Barton Cc: Dag-Erling Smorgrav , Dan Moschuk , arch@FreeBSD.org Subject: Re: Package system flaws? References: <20020709233503.I5990-100000@master.gorean.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Doug Barton wrote: > > On Mon, 8 Jul 2002, Wes Peters wrote: > > > Doug Barton wrote: > > > > > > On Sun, 7 Jul 2002, Terry Lambert wrote: > > > > > > > > We want to be able to install a package from a non-rewindable source > > > > > without storing a temporary copy on disk. This means the metadata > > > > > must without fail be at the very beginning of the package. > > > > > > Ok, then what about storing the meta data as a seperate file? Why do they > > > have to be in the same package? > > > > So you can (md5, sign) them together and know that they "apply" to each > > other. > > If we limit the metadata properly (for my definition of "proper") there > won't be anything that needs to be verified. The metadata file should of > course include the md5 of the package(s) it applies to directly. I'm coming around to your viewpoint. We can write the metadata to have external references to binary filesets, and even to the source, patches, and Makefiles that make up the "port." Filesets can be signed so we can verify they are the correct contents and weren't tampered with in transmission. This would allow us to update the metadata files in the same way ports are updated now, using CVSup, keeping the packages system up to date. Only the filesets that have actually changed need be downloaded to upgrade to a newer package. This would also allow the user to skip filesets she doesn't need, such as support for languages not comprehended or binaries for platforms not present. This would even allow OpenBSD users to skip the installation of man pages, so long as the package creator is clever enough to package the man pages in a separate fileset. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jul 11 19:16: 5 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D83E737B400; Thu, 11 Jul 2002 19:16:03 -0700 (PDT) Received: from mailout02.sul.t-online.com (mailout02.sul.t-online.com [194.25.134.17]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1432F43E7B; Thu, 11 Jul 2002 19:16:03 -0700 (PDT) (envelope-from corecode@corecode.ath.cx) Received: from fwd08.sul.t-online.de by mailout02.sul.t-online.com with smtp id 17SpyK-0000CK-08; Fri, 12 Jul 2002 04:15:52 +0200 Received: from spirit.zuhause.stoert.net (320050403952-0001@[217.224.167.144]) by fmrl08.sul.t-online.com with esmtp id 17SpyI-22IwmeC; Fri, 12 Jul 2002 04:15:50 +0200 Received: from terrorfish.uni.stoert.net (terrorfish.uni.stoert.net [10.150.180.178]) by spirit.zuhause.stoert.net (8.11.6/8.11.6) with ESMTP id g6C2FnQ06046; Fri, 12 Jul 2002 04:15:49 +0200 (CEST) (envelope-from corecode@corecode.ath.cx) Received: from terrorfish.uni.stoert.net (localhost [127.0.0.1]) by terrorfish.uni.stoert.net (8.12.5/8.12.4) with ESMTP id g6C2EmBA078243; Fri, 12 Jul 2002 04:14:48 +0200 (CEST) (envelope-from corecode@terrorfish.uni.stoert.net) Received: (from corecode@localhost) by terrorfish.uni.stoert.net (8.12.5/8.12.4/Submit) id g6C2Ed7H078242; Fri, 12 Jul 2002 04:14:39 +0200 (CEST) Date: Fri, 12 Jul 2002 04:14:36 +0200 From: "Simon 'corecode' Schubert" To: Wes Peters Cc: DougB@FreeBSD.ORG, des@ofug.org, dan@FreeBSD.ORG, arch@FreeBSD.ORG Subject: Re: Package system flaws? Message-Id: <20020712041436.5bfb634a.corecode@corecode.ath.cx> In-Reply-To: <3D2E2D33.D09B8F16@softweyr.com> References: <20020709233503.I5990-100000@master.gorean.org> <3D2E2D33.D09B8F16@softweyr.com> X-Mailer: Sylpheed version 0.7.8claws (GTK+ 1.2.10; i386-portbld-freebsd5.0) Mime-Version: 1.0 Content-Type: multipart/signed; protocol="application/pgp-signature"; boundary="=.cd(khredbSGs6:" X-Sender: 320050403952-0001@t-dialin.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --=.cd(khredbSGs6: Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit On Thu, 11 Jul 2002 18:13:23 -0700 Wes Peters wrote: > I'm coming around to your viewpoint. We can write the metadata to have > external references to binary filesets, and even to the source, patches, > and Makefiles that make up the "port." Filesets can be signed so we can > verify they are the correct contents and weren't tampered with in > transmission. > > This would allow us to update the metadata files in the same way ports > are updated now, using CVSup, keeping the packages system up to date. and while we're at it, change the ports system to use something like that so that my (our) inodes are not all being eaten by /usr/ports ;] this would actually be a good thing [tm] to marry both systems. one metadata tree for both ports (source, customized build) and packages (binary). just ``echo USE_PACKAGES=YES >> /etc/make.conf'' or whatever. cheers simon -- /"\ http://corecode.ath.cx/#donate \ / \ ASCII Ribbon Campaign / \ Against HTML Mail and News --=.cd(khredbSGs6: Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.7 (FreeBSD) iD8DBQE9LjuPr5S+dk6z85oRAk+fAKCVlpx6EMRpEgXkyOr5mY4/+21KhQCgrkH0 LB/LBGHEre72LidgSHME/rI= =8etX -----END PGP SIGNATURE----- --=.cd(khredbSGs6:-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jul 11 19:41:35 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 92FBC37B400; Thu, 11 Jul 2002 19:41:33 -0700 (PDT) Received: from mail.rpi.edu (mail.rpi.edu [128.113.22.40]) by mx1.FreeBSD.org (Postfix) with ESMTP id DE54C43E64; Thu, 11 Jul 2002 19:41:32 -0700 (PDT) (envelope-from drosih@rpi.edu) Received: from [128.113.24.47] (gilead.netel.rpi.edu [128.113.24.47]) by mail.rpi.edu (8.12.1/8.12.1) with ESMTP id g6C2fUcA196866; Thu, 11 Jul 2002 22:41:31 -0400 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> Date: Thu, 11 Jul 2002 22:41:30 -0400 To: Wes Peters , Dan Moschuk From: Garance A Drosihn Subject: Re: Package system flaws? Cc: arch@FreeBSD.ORG Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Scanned-By: MIMEDefang 2.3 (www dot roaringpenguin dot com slash mimedefang) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG At 10:00 PM -0400 7/7/02, Garance A Drosihn wrote: >I think we try to stuff too much information into the name of a >port, and we try to do too much to shoehorn all ports-processing >into standard makefile variables and standard make-cmd processing. To explain this a bit more, we sometimes get into a problem when portAA needs portBB, and you: cd /usr/ports/*/portAA make -> make sees it needs to make portBB -> it does a cd /usr/ports/*/portBB -> and does a 'make' there, but it still has a whatever make variables had been set for portAA, which you might *not* want to have set when making portBB. I know I've hit this, but I can't remember the specifics, and I know I have not hit it often. Using portupgrade also probably reduces the chance of this happening. If I had more spare time, what I'd like to try my hand at is to redo how all the port interactions are described in the Makefile. Instead of doing that with makefiles and make-variables, do it as makefile comments, and then have a separate program (not make itself) figure out what other ports should be made based on that information in the comments in the makefile. Make would then run that program as the first step towards making the port, and run that program again as the first step towards 'make install' of the port. The way I described that it might sound a little hair-brained, but I think it could be an improvement if done right. -- Garance Alistair Drosehn = gad@gilead.netel.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jul 11 19:51:39 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3740537B400; Thu, 11 Jul 2002 19:51:37 -0700 (PDT) Received: from mail.rpi.edu (mail.rpi.edu [128.113.22.40]) by mx1.FreeBSD.org (Postfix) with ESMTP id AAEAC43E3B; Thu, 11 Jul 2002 19:51:36 -0700 (PDT) (envelope-from drosih@rpi.edu) Received: from [128.113.24.47] (gilead.netel.rpi.edu [128.113.24.47]) by mail.rpi.edu (8.12.1/8.12.1) with ESMTP id g6C2pYcA059730; Thu, 11 Jul 2002 22:51:34 -0400 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> Date: Thu, 11 Jul 2002 22:51:33 -0400 To: Wes Peters , Dan Moschuk From: Garance A Drosihn Subject: Re: Package system flaws? Cc: arch@FreeBSD.ORG Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Scanned-By: MIMEDefang 2.3 (www dot roaringpenguin dot com slash mimedefang) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG [as Garance continues to talk to himself...] At 10:41 PM -0400 7/11/02, Garance A Drosihn wrote: >To explain this a bit more, we sometimes get into a problem when >portAA needs portBB, and you: > > cd /usr/ports/*/portAA > make > -> make sees it needs to make portBB > -> it does a cd /usr/ports/*/portBB > -> and does a 'make' there, but it still has a whatever > make variables had been set for portAA, which you might > *not* want to have set when making portBB. > >I know I've hit this, but I can't remember the specifics, and I >know I have not hit it often. It might be that this was self-inflicted, in the sense that I may have added settings on the make command I typed in, eg: DOJIGGY=YES make or make BLAH=DISABLE because I did want those settings when making portAA, but I wasn't expecting portBB to be built with the same settings. -- Garance Alistair Drosehn = gad@gilead.netel.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jul 11 21:58: 5 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 19BF937B400 for ; Thu, 11 Jul 2002 21:58:00 -0700 (PDT) Received: from spqr.osg.gov.bc.ca (spqr.osg.gov.bc.ca [142.32.102.24]) by mx1.FreeBSD.org (Postfix) with ESMTP id 83B6B43E31 for ; Thu, 11 Jul 2002 21:57:59 -0700 (PDT) (envelope-from Cy.Schubert@osg.gov.bc.ca) Received: from passer.osg.gov.bc.ca (passer.osg.gov.bc.ca [142.32.110.29]) by spqr.osg.gov.bc.ca (Postfix) with ESMTP id 3D1789F171; Thu, 11 Jul 2002 21:57:59 -0700 (PDT) Received: from cwsys.cwsent.com (cwsys2 [10.1.2.1]) by passer.osg.gov.bc.ca (8.12.5/8.12.3) with ESMTP id g6C4vvOX067587; Thu, 11 Jul 2002 21:57:57 -0700 (PDT) (envelope-from cy@cwsent.com) Received: from cwsys (localhost [127.0.0.1]) by cwsys.cwsent.com (8.12.5/8.12.3) with ESMTP id g6C4vux4006072; Thu, 11 Jul 2002 21:57:56 -0700 (PDT) (envelope-from cy@cwsys.cwsent.com) Message-Id: <200207120457.g6C4vux4006072@cwsys.cwsent.com> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 Reply-To: Cy Schubert - CITS Open Systems Group From: Cy Schubert - CITS Open Systems Group X-os: FreeBSD X-Sender: cy@cwsent.com To: Wes Peters Cc: Cy Schubert - CITS Open Systems Group , Terry Lambert , arch@FreeBSD.ORG Subject: Re: Package system wishlist In-Reply-To: Message from Wes Peters of "Wed, 10 Jul 2002 19:22:06 PDT." <3D2CEBCE.55DC3C6D@softweyr.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 11 Jul 2002 21:57:56 -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Sorry for the late reply, management course. In message <3D2CEBCE.55DC3C6D@softweyr.com>, Wes Peters writes: > Cy Schubert - CITS Open Systems Group wrote: > > > > In message <3D2BE142.E25CA9BC@mindspring.com>, Terry Lambert writes: > > > So, following Jordan's advice, what's on everyone's wishlist? > > > > > > Terry's Wishlist: > > [...] > > > > + Cy's Wishlist: > > > > o Optional installation of sources. RH's SRPM's is a very poor > > example of this. A better example would be what IBM does to > > install JES/2 on their MVS system, e.g. an OpenSSH package might > > contain source in addition to binaries. The sources would be > > installed in /usr/src while the binaries would be installed > > in /usr/bin, sbin.... > > Yes! My mythical XML metadata format, with or without external "filesets", > would handle this with aplomb. The source set would be included in the > metadata and you could skip it or install it as with any other fileset. > Come to think of it, you could include the ports Makefile and patches as > well. Hmm, that bears some thinking about. Most of what is in a "port" > right now is metadata too. IBM used UCL. XML is better. > > > o Files replaced by a package backed up in case of package removal > > I'm not sure what you mean here. Be able to create a backup script of > the files related to a package for backing up? Be able to restore only > missing files from a package? Both seem like good ideas... If for example openssh-overwrite-base-3.4p1 is installed, the old binaries are saved (backed up) before the package is installed. If I pkg_delete openssh-overwrite-base-3.4p1, the old ssh files are restored (reappear). > > > o Check option: Tell me what it will do without doing it > > > > o Group option: Install prerequisites > > Wouldn't you want this to be the default, perhaps with an option to > abort if they're not "readily available"??? You're right. Then there should be an option to just install the selected packages and nothing else. This would allow for "creative" problem solving. > > > o Groupextend option: Install postrequisites, e.g. dependent > > packages and patches > > In other words, roll portupgrade into the system. Yes. > > > o Ability to install my own packages on top of packages and > > patches, I like to call them USERMODS. > > Your own packages or your changes to a standard package? I can see the > value, but how to do it doesn't leap immediately to mind. This increases the complexity of the proposed package system. This was mentioned as a possible ideal. I doubt this feature would be used much. Please use it as you see fit. > > > o The package system should be independent of the compression tool > > used. In the future new compression algorithms and tools will > > be developed. The package system should be flexible enough to > > not care how its files are compressed or packaged. > > Ditto for archive formats, encoding formats, etc. We should probably > specify one of each as a bare minimum, choosing from those that are > available in library format, reasonably licensed, and have acceptable > performance (for some definition of acceptable). > > > o The ability to export and import the package database (currently > > to clone systems, I rsync /usr/local, /usr/X11R6, and /var/db/pkg > > to a new system I am installing, this saves many hours of work). > > Yes, perhaps even the ability to capture a currently installed package > and turn it back into a package file. That'd be way cool for duplicating > packages with local customizations. > > > > o I want to be able to remove system components, like "sendmail" > > > and "OpenSSH". > > > > Ideally everything should install as a package, however that would > > create a lot of extra work for us developers. I have yet to think of a > > painless way to do this. > > Yeah, Debian has certainly showed us how NOT to do it. "Which version of > /bin/cat do you want?" Exactly. This had its usefulness in the mainframe world where decisions made years ago would cost millions of dollars to undo. OTOH, choosing a SYSV init v.s. a BSD init might be nice (just an example, no flames please). Ultimately striking the proper balance is our goal. Please pick and choose any ideas as you see fit. -- Cheers, Phone: 250-387-8437 Cy Schubert Fax: 250-387-5766 Team Leader, Sun/Alpha Team Email: Cy.Schubert@osg.gov.bc.ca Open Systems Group, CITS Ministry of Management Services Province of BC FreeBSD UNIX: cy@FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jul 11 22:15:34 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2AA5F37B400; Thu, 11 Jul 2002 22:15:30 -0700 (PDT) Received: from InterJet.dellroad.org (adsl-63-194-81-26.dsl.snfc21.pacbell.net [63.194.81.26]) by mx1.FreeBSD.org (Postfix) with ESMTP id 757A143E54; Thu, 11 Jul 2002 22:15:29 -0700 (PDT) (envelope-from archie@dellroad.org) Received: from arch20m.dellroad.org (arch20m.dellroad.org [10.1.1.20]) by InterJet.dellroad.org (8.9.1a/8.9.1) with ESMTP id WAA80888; Thu, 11 Jul 2002 22:05:32 -0700 (PDT) Received: (from archie@localhost) by arch20m.dellroad.org (8.11.6/8.11.6) id g6C54m736085; Thu, 11 Jul 2002 22:04:48 -0700 (PDT) (envelope-from archie) From: Archie Cobbs Message-Id: <200207120504.g6C54m736085@arch20m.dellroad.org> Subject: Re: Timeout and SMP race In-Reply-To: <200207101815.g6AIFDm28655@arch20m.dellroad.org> "from Archie Cobbs at Jul 10, 2002 11:15:13 am" To: Archie Cobbs Date: Thu, 11 Jul 2002 22:04:48 -0700 (PDT) Cc: John Baldwin , davidx@viasoft.com.cn, freebsd-arch@FreeBSD.ORG, julian@elischer.org X-Mailer: ELM [version 2.4ME+ PL88 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Archie Cobbs writes: > FWIW, here is an API I've used before. This handles all race > conditions and the 'other thread' question. I should also mention that if we were to implement this API it may be useful to generalize it to handle arbitrary events rather than just timeout events (this is how I've implemented it in our application). For example, 'user defined' events could be triggered from any thread simply by calling a trigger function. This would be like an 'event' version of tsleep(): instead of sleeping for the event, a callback function (handler) that you provide would be called when the event occurs. The key is that you get the same benefits mentioned before, i.e., your mutex would already be held when your handler is invoked, there's no race between the handler and cancelling the event, etc. Event-based programming is more efficient than threads when you don't require a stack to store your state while waiting for the event. Not sure how many instances there are in the kernel that could benefit from this however. -Archie __________________________________________________________________________ Archie Cobbs * Packet Design * http://www.packetdesign.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jul 11 23: 7:36 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 06A7237B400; Thu, 11 Jul 2002 23:07:35 -0700 (PDT) Received: from rover.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id B61D343E42; Thu, 11 Jul 2002 23:07:33 -0700 (PDT) (envelope-from imp@bsdimp.com) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.11.3/8.11.3) with ESMTP id g6C67QY91668; Fri, 12 Jul 2002 00:07:26 -0600 (MDT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.11.6/8.11.6) with ESMTP id g6C67PG28382; Fri, 12 Jul 2002 00:07:25 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Fri, 12 Jul 2002 00:07:11 -0600 (MDT) Message-Id: <20020712.000711.40763963.imp@bsdimp.com> To: archie@dellroad.org Cc: jhb@FreeBSD.ORG, davidx@viasoft.com.cn, freebsd-arch@FreeBSD.ORG, julian@elischer.org Subject: Re: Timeout and SMP race From: "M. Warner Losh" In-Reply-To: <200207120504.g6C54m736085@arch20m.dellroad.org> References: <200207101815.g6AIFDm28655@arch20m.dellroad.org> <200207120504.g6C54m736085@arch20m.dellroad.org> X-Mailer: Mew version 2.1 on Emacs 21.1 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message: <200207120504.g6C54m736085@arch20m.dellroad.org> Archie Cobbs writes: : Event-based programming is more efficient than threads when you : don't require a stack to store your state while waiting for the : event. Not sure how many instances there are in the kernel that : could benefit from this however. pccard/cardbus bridges could benefit. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 0:20:13 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 579FE37B401; Fri, 12 Jul 2002 00:20:01 -0700 (PDT) Received: from softweyr.com (softweyr.com [65.88.244.127]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4A3DC43E4A; Fri, 12 Jul 2002 00:20:00 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from nextgig-3.customer.nethere.net ([209.132.102.163] helo=softweyr.com) by softweyr.com with esmtp (Exim 3.35 #1) id 17Sui5-000Opp-00; Fri, 12 Jul 2002 01:19:25 -0600 Message-ID: <3D2E83F7.6EDA681C@softweyr.com> Date: Fri, 12 Jul 2002 00:23:35 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Mark Valentine Cc: Cy Schubert - CITS Open Systems Group , Dag-Erling Smorgrav , Doug Barton , Dan Moschuk , arch@freebsd.org Subject: Re: Package system flaws? References: <200207111741.g6BHffn8073047@dotar.thuvia.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Mark Valentine wrote: > > > From: wes@softweyr.com (Wes Peters) > > Date: Thu 11 Jul, 2002 > > Subject: Re: Package system flaws? > > > The idea I like the best is to have the filesets (picture a .tar.gz > > or a .zip here) be external references on the fileserver; the XML > > contains all the metadata and URLs for the filesets. As you fetch > > the XML, or after you've fetched all of the XML, you fetch the > > filesets you're not skipping. Once you have the filesets on local > > storage, you can rewrite the URL references or convert them to > > in-line encoding, leaving the ones that have been skipped in the > > original URL encoding. > > This is equivalent to my proposal, except that I don't see much value > in converting to the in-line encoding, and my method specifies a directory > name (or any suitable simple archive name) containing a package.xml and > compressed file sets, whereas you presumably expect the XML file to be the > primary referenced object. Yes. You see, you could distribute and update the XML files via CVSup, the way we do with the ports bits now. > > For instance: > > > > > > > > > > > > > > > > > > My equivalent would be: > > > > > > > > > > or: > > name=cat > bin=bin.tar.gz > man=man.tar.gz > lang[en_US]=en_US.tar.gz > lang[en_UK]=en_UK.tar.gz > lang[fr_FR]=fr_FR.tar.gz > > according to taste. > > > Now assume you specified you want to install only the en_UK language > > files. pkg_add would leave the en_US and fr_FR as external references, > > download the binaries, man pages, and en_UK filesets, and convert those > > three into local file references OR directly encode them into the > > package file. > > Same except that I don't need the last step, though I could optionally > bundle up for archiving purposes. Not storing them in-line on the local system means you have to have a "standard" place to store the filesets, or tell the program where to put them (or where they already are) every time you run it. If you pick a standard location, somebody somewhere someday is going to run it out of space and bitch. Converting the files to inline on disk-like media (i.e. local hard drive, on a CD-ROM, etc) makes the packages much easier to copy around. But that is certainly an optional step. The default place to stick the filesets downloaded, IMHO, is the directory where the XML file is located. No surprises, easy to find a filesystem that has space available, etc. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 0:35:52 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3804037B400; Fri, 12 Jul 2002 00:35:46 -0700 (PDT) Received: from softweyr.com (softweyr.com [65.88.244.127]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9337843E4A; Fri, 12 Jul 2002 00:35:42 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from nextgig-6.customer.nethere.net ([209.132.102.166] helo=softweyr.com) by softweyr.com with esmtp (Exim 3.35 #1) id 17SuxR-000OqZ-00; Fri, 12 Jul 2002 01:35:17 -0600 Message-ID: <3D2E87AD.76DC449D@softweyr.com> Date: Fri, 12 Jul 2002 00:39:25 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Mark Valentine Cc: Terry Lambert , Jordan K Hubbard , Dan Nelson , Archie Cobbs , Dan Moschuk , Dag-Erling Smorgrav , arch@freebsd.org Subject: Re: Package system flaws? References: <200207120055.g6C0tbpQ084565@dotar.thuvia.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Mark Valentine wrote: > > > From: wes@softweyr.com (Wes Peters) > > Date: Thu 11 Jul, 2002 > > Subject: Re: Package system flaws? > > > On problem in particular comes from changing the PREFIX of a > > package. You can easily do this in the package tools, but how do you > > change the BINARIES to realize they've been installed under another > > PREFIX? > > How about providing a packaging API call (and utility) for the packages > to grab their install root from a record in /var/db/pkg? Something > like pkg_getroot(3) and pkg_config(1). I was thinking we could replace a lot of the mish-mash in /var/db/pkg with a munged up version of the metadata file that includes all the real file locations, translations for various meta directory names, etc. Providing a simply library function to translate PREFIX for "my package" or a named package would be ever so sensible. > There probably isn't much less intrusive way to make a package's root > configurable at install time. It's a good idea, if we can just get the application developers to use it. Actually you might be able to do it by forcing autoconf to code in a meta-name for each the directory prefixes and linking against an overridden version of open(2) that properly translates the meta-name. But that's icky. And the next thing you know, somebody's gonna want that function to allow the meta-name to be expanded to a path, etc. etc. Augh! > If you didn't want the package to have to look in /var/db/pkg, have > the package binaries hold a maximum-length padded string variable (marked > similarly to what(1) strings), and provide a tool to edit the binary (and > scripts) at install time. Similar to the above, substituting scary editing of binaries for scary library linking. ;^) > I think environment variables are too fragile for this purpose. > > > You could almost do this on UNIX now, by putting such settings into > > init's environment, if you could get the application developers to > > write their code (especially libraries) appropriately. > > This doesn't sound like it would be easy to allow different packages to > be installed with different roots. You're right. VMS has a system table, a per-group table, and a per-user table to allow overrides. They teeter between being neat features and be overkill. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 0:57:56 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3ABCD37B400 for ; Fri, 12 Jul 2002 00:57:52 -0700 (PDT) Received: from softweyr.com (softweyr.com [65.88.244.127]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3FE7743E42 for ; Fri, 12 Jul 2002 00:57:46 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from nextgig-8.customer.nethere.net ([209.132.102.168] helo=softweyr.com) by softweyr.com with esmtp (Exim 3.35 #1) id 17SvJ4-000OrH-00; Fri, 12 Jul 2002 01:57:38 -0600 Message-ID: <3D2E8CED.9A6B30D3@softweyr.com> Date: Fri, 12 Jul 2002 01:01:49 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Cy Schubert - CITS Open Systems Group Cc: Terry Lambert , arch@FreeBSD.ORG Subject: Re: Package system wishlist References: <200207120457.g6C4vux4006072@cwsys.cwsent.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Cy Schubert - CITS Open Systems Group wrote: > > Sorry for the late reply, management course. > > In message <3D2CEBCE.55DC3C6D@softweyr.com>, Wes Peters writes: > > Cy Schubert - CITS Open Systems Group wrote: > > > > > > + Cy's Wishlist: > > > > > > o Optional installation of sources. RH's SRPM's is a very poor > > > example of this. A better example would be what IBM does to > > > install JES/2 on their MVS system, e.g. an OpenSSH package might > > > contain source in addition to binaries. The sources would be > > > installed in /usr/src while the binaries would be installed > > > in /usr/bin, sbin.... > > > > Yes! My mythical XML metadata format, with or without external "filesets", > > would handle this with aplomb. The source set would be included in the > > metadata and you could skip it or install it as with any other fileset. > > Come to think of it, you could include the ports Makefile and patches as > > well. Hmm, that bears some thinking about. Most of what is in a "port" > > right now is metadata too. > > IBM used UCL. XML is better. > > > > > > o Files replaced by a package backed up in case of package removal > > > > I'm not sure what you mean here. Be able to create a backup script of > > the files related to a package for backing up? Be able to restore only > > missing files from a package? Both seem like good ideas... > > If for example openssh-overwrite-base-3.4p1 is installed, the old > binaries are saved (backed up) before the package is installed. If I > pkg_delete openssh-overwrite-base-3.4p1, the old ssh files are restored > (reappear). Oh, OK, now I see. Yes, of course. ;^) > > > o Check option: Tell me what it will do without doing it > > > > > > o Group option: Install prerequisites > > > > Wouldn't you want this to be the default, perhaps with an option to > > abort if they're not "readily available"??? > > You're right. Then there should be an option to just install the > selected packages and nothing else. This would allow for "creative" > problem solving. And for just jamming Gimp onto the system when you know it'll do what you want without lib{greatestgraphicsformatnobodyuses}.so. > > > o Groupextend option: Install postrequisites, e.g. dependent > > > packages and patches > > > > In other words, roll portupgrade into the system. > > Yes. This (and mergemaster as Terry pointed out) need to be done anyhow. > > > o Ability to install my own packages on top of packages and > > > patches, I like to call them USERMODS. > > > > Your own packages or your changes to a standard package? I can see the > > value, but how to do it doesn't leap immediately to mind. > > This increases the complexity of the proposed package system. This was > mentioned as a possible ideal. I doubt this feature would be used > much. Please use it as you see fit. Something to think about for the future. Terry mentioned being able to re-encapsulate edited configuration files, etc. For example, a package that installs a conf.example file should have the real conf file associated with it in some way, too. > > > Ideally everything should install as a package, however that would > > > create a lot of extra work for us developers. I have yet to think of a > > > painless way to do this. > > > > Yeah, Debian has certainly showed us how NOT to do it. "Which version of > > /bin/cat do you want?" > > Exactly. This had its usefulness in the mainframe world where > decisions made years ago would cost millions of dollars to undo. OTOH, > choosing a SYSV init v.s. a BSD init might be nice (just an example, no > flames please). Ultimately striking the proper balance is our goal. > Please pick and choose any ideas as you see fit. With some nice, canned configurations that do a good approximation of working out of the box. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 1: 2:14 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 85E1D37B400 for ; Fri, 12 Jul 2002 01:02:11 -0700 (PDT) Received: from softweyr.com (softweyr.com [65.88.244.127]) by mx1.FreeBSD.org (Postfix) with ESMTP id 124C443E54 for ; Fri, 12 Jul 2002 01:02:11 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from nextgig-6.customer.nethere.net ([209.132.102.166] helo=softweyr.com) by softweyr.com with esmtp (Exim 3.35 #1) id 17Sv85-000Oqu-00; Fri, 12 Jul 2002 01:46:17 -0600 Message-ID: <3D2E8A43.AF04381@softweyr.com> Date: Fri, 12 Jul 2002 00:50:27 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Thierry Herbelot Cc: arch@FreeBSD.ORG Subject: Re: Listening to users [was Re: Package system wishlist] References: <3D2BE142.E25CA9BC@mindspring.com> <3D2CAAC1.5CF66C96@herbelot.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Thierry Herbelot wrote: > > [Snip the long arguments over which new format for the packages] > > there is a interesting post by an ex-user of desktop Linux, which can > help when designing the new package system (it will have to be used by > mere mortals, not only by the elite posting on -arch) > > > (shamelessly cut&pasted from /.) > > we still have a long way to go before even computer-aware people (that > is, even people formely developping on SGI or Sun machines) can feel at > ease installing and using FreeBSD. Really? Believe it or not, I've formerly (and currently) developed on SGI and Sun machines, along with IBM, HP, DEC, Motorola, Intel, NCR, Unisys, Sequent, CCI, Arete/Arix, and even Radio Shack UNIX-ish machines, and I strongly prefer FreeBSD to any of them. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 2:27:28 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9D22237B400; Fri, 12 Jul 2002 02:27:24 -0700 (PDT) Received: from rina.r.dl.itc.u-tokyo.ac.jp (rina.r.dl.itc.u-tokyo.ac.jp [133.11.199.247]) by mx1.FreeBSD.org (Postfix) with ESMTP id 09EAC43E5E; Fri, 12 Jul 2002 02:27:20 -0700 (PDT) (envelope-from tanimura@r.dl.itc.u-tokyo.ac.jp) Received: from rina.r.dl.itc.u-tokyo.ac.jp (localhost [127.0.0.1]) by rina.r.dl.itc.u-tokyo.ac.jp (8.12.5/3.7W-rina.r-Nankai-Koya) with ESMTP id g6C9RFUl052728 ; Fri, 12 Jul 2002 18:27:16 +0900 (JST) Message-Id: <200207120927.g6C9RFUl052728@rina.r.dl.itc.u-tokyo.ac.jp> Date: Fri, 12 Jul 2002 18:27:14 +0900 From: Seigo Tanimura To: Terry Lambert Cc: John Baldwin , Seigo Tanimura , arch@FreeBSD.org Subject: Re: multiple threads for interrupts In-Reply-To: <3D1CC378.5377CDB9@mindspring.com> References: <3D1CC378.5377CDB9@mindspring.com> User-Agent: Wanderlust/2.8.1 (Something) SEMI/1.14.3 (Ushinoya) FLIM/1.14.3 (=?ISO-8859-1?Q?Unebigory=F2mae?=) APEL/10.3 MULE XEmacs/21.1 (patch 14) (Cuyahoga Valley) (i386--freebsd) Organization: Digital Library Research Division, Information Techinology Centre, The University of Tokyo MIME-Version: 1.0 (generated by SEMI 1.14.3 - "Ushinoya") Content-Type: text/plain; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, 28 Jun 2002 13:13:44 -0700, Terry Lambert said: >> FWIW, Solaris doesn't use multiple >> threads for a shared interrupt, instead, when an ithread is awakened, it >> is bound to the CPU receiving the interrupt until it finishes. If you >> don't have a copy of Solaris Internals I would recommend getting a >> copy. :) tlambert2> It's a good book, event hough Solaris 9 changes again. I'll tlambert2> also note that they have "invented" the buffer cache seperate tlambert2> from the VM (again), to get around the lack of working set tlambert2> quotas, so some things in that book need to be taken with a tlambert2> grain of salt... I ordered one two days ago, it should be a great help to understand the source of Solaris 8 I have. -- Seigo Tanimura To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 3: 6:58 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 52C2137B400 for ; Fri, 12 Jul 2002 03:06:57 -0700 (PDT) Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id 810A143E31 for ; Fri, 12 Jul 2002 03:06:56 -0700 (PDT) (envelope-from des@ofug.org) Received: by flood.ping.uio.no (Postfix, from userid 2602) id F3D08534A; Fri, 12 Jul 2002 12:06:53 +0200 (CEST) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Wes Peters Cc: Thierry Herbelot , arch@FreeBSD.ORG Subject: Re: Listening to users [was Re: Package system wishlist] References: <3D2BE142.E25CA9BC@mindspring.com> <3D2CAAC1.5CF66C96@herbelot.com> <3D2E8A43.AF04381@softweyr.com> From: Dag-Erling Smorgrav Date: 12 Jul 2002 12:06:52 +0200 In-Reply-To: <3D2E8A43.AF04381@softweyr.com> Message-ID: Lines: 21 User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.2 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Wes Peters writes: > Really? Believe it or not, I've formerly (and currently) developed on > SGI and Sun machines, along with IBM, HP, DEC, Motorola, Intel, NCR, > Unisys, Sequent, CCI, Arete/Arix, and even Radio Shack UNIX-ish > machines, and I strongly prefer FreeBSD to any of them. Agreed. I've used SGI, Sun and Digital systems for both fun and profit in the past, and will choose FreeBSD over them any day of the week. It's a far more comfortable development environment, and locating, installing and configuring any third-party tools I might need is far easier than on any commercial Unix I know of. Anyone who thinks IRIX, Solaris or Tru64 have better hardware support than FreeBSD is losing sight of the fact that they are designed to run on a very limited and tightly controlled range of hardware, while FreeBSD will run (with a little effort) on practically anything you throw at it, within reason. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 3:28:18 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9710437B400 for ; Fri, 12 Jul 2002 03:28:16 -0700 (PDT) Received: from dire.bris.ac.uk (dire.bris.ac.uk [137.222.10.60]) by mx1.FreeBSD.org (Postfix) with ESMTP id D475343E64 for ; Fri, 12 Jul 2002 03:28:15 -0700 (PDT) (envelope-from Jan.Grant@bristol.ac.uk) Received: from mail.ilrt.bris.ac.uk by dire.bris.ac.uk with SMTP-PRIV with ESMTP; Fri, 12 Jul 2002 11:28:06 +0100 Received: from cmjg (helo=localhost) by mail.ilrt.bris.ac.uk with local-esmtp (Exim 3.16 #1) id 17Sxc1-0003JD-00; Fri, 12 Jul 2002 11:25:21 +0100 Date: Fri, 12 Jul 2002 11:25:20 +0100 (BST) From: Jan Grant X-X-Sender: cmjg@mail.ilrt.bris.ac.uk To: Wes Peters Cc: Cy Schubert - CITS Open Systems Group , Terry Lambert , arch Subject: Re: Package system wishlist In-Reply-To: <3D2E8CED.9A6B30D3@softweyr.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, 12 Jul 2002, Wes Peters wrote: > With some nice, canned configurations that do a good approximation of > working out of the box. Seconded: sample configurations and real, user-modified configurations and I don't want a package upgrade to "accidentally" kill my older config of a dependency that got upgraded in the process, and so on. God, I can't parse that. "Do the right thing with configuration files". Also, avoid the use of pre/post scripts as much as possible: declarative metadata to describe external dependencies that a package has (90% of the time these will probably just be "this group" or "that user") so that I can configure local policy for creating daemon users, getting rid of them when no installed packages still require them, etc. -- jan grant, ILRT, University of Bristol. http://www.ilrt.bris.ac.uk/ Tel +44(0)117 9287088 Fax +44 (0)117 9287112 RFC822 jan.grant@bris.ac.uk Talk is cheap: free, as in beer. As in Real Ale, not that Budweiser rubbish. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 5:14:18 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A543537B400 for ; Fri, 12 Jul 2002 05:14:16 -0700 (PDT) Received: from tchpc01.tcd.ie (tchpc.tcd.ie [134.226.10.78]) by mx1.FreeBSD.org (Postfix) with ESMTP id F110543E4A for ; Fri, 12 Jul 2002 05:14:11 -0700 (PDT) (envelope-from bobb+freebsd-arch@redbrick.dcu.ie) Received: from flipflop.tchpc.tcd.ie (hpc04.iss.tcd.ie [134.226.10.47]) by tchpc01.tcd.ie (Postfix) with ESMTP id B9C5C33F7; Fri, 12 Jul 2002 13:14:10 +0100 (IST) Received: by flipflop.tchpc.tcd.ie (Postfix, from userid 1001) id 2C621195; Fri, 12 Jul 2002 13:14:28 +0100 (IST) Date: Fri, 12 Jul 2002 13:14:27 +0100 From: Robert bobb Crosbie To: Garance A Drosihn Cc: arch@FreeBSD.ORG Subject: Re: Package system flaws? Message-ID: <20020712121427.GD3678@lummux.tchpc.tcd.ie> References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4i Organization: bobb Industries Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Garance A Drosihn hath declared on Thursday the 11 day of July 2002 :-: > > cd /usr/ports/*/portAA > > make > > -> make sees it needs to make portBB > > -> it does a cd /usr/ports/*/portBB > > -> and does a 'make' there, but it still has a whatever > > make variables had been set for portAA, which you might > > *not* want to have set when making portBB. > because I did want those settings when making portAA, but I > wasn't expecting portBB to be built with the same settings. This reminds me, for ports it would be rather handy if each installed port could save the make options that you passed to when it was built so then when you build it again it can recall and use those options. Example: I built apache2 with ``WITH_SUEXEC=yes'', then after the chunking thing I did a ``portupgrade apache'', apache no longer works, scratched head for a while until I rememberd how I buile it origionally. Of course configurable as some people wouldn't always want the same options when they rebuilt ports, and I'm sure it would also cause a number of problems. - bobb To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 5:43:13 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 432EF37B400 for ; Fri, 12 Jul 2002 05:43:08 -0700 (PDT) Received: from mailout07.sul.t-online.com (mailout07.sul.t-online.com [194.25.134.83]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1F7FE43E65 for ; Fri, 12 Jul 2002 05:43:07 -0700 (PDT) (envelope-from corecode@corecode.ath.cx) Received: from fwd08.sul.t-online.de by mailout07.sul.t-online.com with smtp id 17SzlG-00018D-0A; Fri, 12 Jul 2002 14:43:02 +0200 Received: from spirit.zuhause.stoert.net (320050403952-0001@[217.224.167.144]) by fmrl08.sul.t-online.com with esmtp id 17Szl6-13JfG4C; Fri, 12 Jul 2002 14:42:52 +0200 Received: from terrorfish.uni.stoert.net (terrorfish.uni.stoert.net [10.150.180.178]) by spirit.zuhause.stoert.net (8.11.6/8.11.6) with ESMTP id g6CCgpQ14832; Fri, 12 Jul 2002 14:42:51 +0200 (CEST) (envelope-from corecode@corecode.ath.cx) Received: from terrorfish.uni.stoert.net (localhost [127.0.0.1]) by terrorfish.uni.stoert.net (8.12.5/8.12.4) with ESMTP id g6CCfo1v000961; Fri, 12 Jul 2002 14:41:50 +0200 (CEST) (envelope-from corecode@terrorfish.uni.stoert.net) Received: (from corecode@localhost) by terrorfish.uni.stoert.net (8.12.5/8.12.4/Submit) id g6CCfnrH000960; Fri, 12 Jul 2002 14:41:49 +0200 (CEST) Date: Fri, 12 Jul 2002 14:41:44 +0200 From: "Simon 'corecode' Schubert" To: Robert bobb Crosbie Cc: drosih@rpi.edu, arch@FreeBSD.ORG Subject: Re: Package system flaws? Message-Id: <20020712144144.797cb676.corecode@corecode.ath.cx> In-Reply-To: <20020712121427.GD3678@lummux.tchpc.tcd.ie> References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> <20020712121427.GD3678@lummux.tchpc.tcd.ie> X-Mailer: Sylpheed version 0.7.8claws (GTK+ 1.2.10; i386-portbld-freebsd5.0) Mime-Version: 1.0 Content-Type: multipart/signed; protocol="application/pgp-signature"; boundary="=.QUuintHq3xl6FW" X-Sender: 320050403952-0001@t-dialin.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --=.QUuintHq3xl6FW Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit On Fri, 12 Jul 2002 13:14:27 +0100 Robert bobb Crosbie wrote: > This reminds me, for ports it would be rather handy if each installed > port could save the make options that you passed to when it was built > so then when you build it again it can recall and use those options. good point. i've wanted to work on this for nearly a year now but never found the time to do so. > Of course configurable as some people wouldn't always want the > same options when they rebuilt ports, and I'm sure it would > also cause a number of problems. OPTIONSREVISION++ or OPTIONS_ALWAYSASK=yes plus, when upgrading via package the installer could try and find a suitable customized package (ghostscript should do A4 for me etc). i think i'll look into this next month. perhaps there is a way to implement stuff for the current ports system at least. cheers simon -- /"\ http://corecode.ath.cx/#donate \ / \ ASCII Ribbon Campaign / \ Against HTML Mail and News --=.QUuintHq3xl6FW Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.7 (FreeBSD) iD8DBQE9Ls6Nr5S+dk6z85oRAra+AJ4ppgKFsYTFuNmoKqoFVzITt0mswwCeIVek uPO8Kgy+iDD89mJ3nCkQhhw= =EqlY -----END PGP SIGNATURE----- --=.QUuintHq3xl6FW-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 7:42:58 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1B07737B400; Fri, 12 Jul 2002 07:42:56 -0700 (PDT) Received: from thuvia.demon.co.uk (thuvia.demon.co.uk [193.237.34.248]) by mx1.FreeBSD.org (Postfix) with ESMTP id A07E543E5E; Fri, 12 Jul 2002 07:42:53 -0700 (PDT) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (dotar.thuvia.org [10.0.0.4]) by phaidor.thuvia.org (8.11.6/8.11.6) with ESMTP id g6CEgbb35150; Fri, 12 Jul 2002 15:42:38 +0100 (BST) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (localhost [IPv6:::1]) by dotar.thuvia.org (8.12.3/8.12.3) with ESMTP id g6CEgbKC002992; Fri, 12 Jul 2002 15:42:37 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.3/8.12.3/Submit) id g6CEgZGc002989; Fri, 12 Jul 2002 15:42:35 +0100 (BST) Date: Fri, 12 Jul 2002 15:42:35 +0100 (BST) From: Mark Valentine Message-Id: <200207121442.g6CEgZGc002989@dotar.thuvia.org> In-Reply-To: Wes Peters's message of Jul 12, 12:23am X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: Wes Peters Subject: Re: Package system flaws? Cc: Cy Schubert - CITS Open Systems Group , Dag-Erling Smorgrav , Doug Barton , Dan Moschuk , arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: Wes Peters > Date: Fri 12 Jul, 2002 > Subject: Re: Package system flaws? > The default place to stick the filesets downloaded, IMHO, is the > directory where the XML file is located. No surprises, easy to find > a filesystem that has space available, etc. I hadn't actually considered any other option besides that. In my scheme it's the directory name that you key off, and the metadata is in a fixed name file, e.g. $ ls /var/spool/pkg/bar-m.n base.bz2 devel.bz2 doc.bz2 package.xml $ pkg_add /var/spool/pkg/bar-m.n Alternatively, using a "bundling transformation": $ (cd /var/spool/pkg/bar-m.n; tar cf ../bar-m.n.tar *.xml *.bz2) $ pkg_add /var/spool/pkg/bar-m.n.tar Am I right in saying that you have something like the following scheme in mind? $ ls /var/spool/pkg bar-m.n-base.bz2 bar-m.n-doc.bz2 foo-x.y-base.bz2 bar-m.n-devel.bz2 bar-m.n.xml foo-x.y.xml In that case my "bundling transformation" becomes: $ (cd /var/spool/pkg; tar cf bar-m.n.tar bar-m.n.xml bar-m.n-*.bz2) My scheme eats more inodes, but I prefer the shorter file names. However, I'm happy with whichever ends up being ultimately more useful. Cheers, Mark. -- Mark Valentine, Thuvia Labs "Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses "We're kind of stupid that way." *munch* *munch* and endorses FreeBSD -- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 8: 3:11 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 38A8437B400; Fri, 12 Jul 2002 08:03:08 -0700 (PDT) Received: from thuvia.demon.co.uk (thuvia.demon.co.uk [193.237.34.248]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0684E43E6A; Fri, 12 Jul 2002 08:03:06 -0700 (PDT) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (dotar.thuvia.org [10.0.0.4]) by phaidor.thuvia.org (8.11.6/8.11.6) with ESMTP id g6CF2gb35244; Fri, 12 Jul 2002 16:02:43 +0100 (BST) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (localhost [IPv6:::1]) by dotar.thuvia.org (8.12.3/8.12.3) with ESMTP id g6CF2gKC003506; Fri, 12 Jul 2002 16:02:42 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.3/8.12.3/Submit) id g6CF2eTL003505; Fri, 12 Jul 2002 16:02:40 +0100 (BST) Date: Fri, 12 Jul 2002 16:02:40 +0100 (BST) From: Mark Valentine Message-Id: <200207121502.g6CF2eTL003505@dotar.thuvia.org> In-Reply-To: Wes Peters's message of Jul 12, 12:39am X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: Wes Peters Subject: Re: Package system flaws? Cc: Terry Lambert , Jordan K Hubbard , Dan Nelson , Archie Cobbs , Dan Moschuk , Dag-Erling Smorgrav , arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: Wes Peters > Date: Fri 12 Jul, 2002 > Subject: Re: Package system flaws? > I was thinking we could replace a lot of the mish-mash in /var/db/pkg > with a munged up version of the metadata file that includes all the > real file locations, translations for various meta directory names, > etc. Certainly. > > There probably isn't much less intrusive way to make a package's root > > configurable at install time. > > It's a good idea, if we can just get the application developers to use it. In real life I think it'll be the ports maintainers who will do this, in much the same way as it's usually their job to modify ports to allow multiple versions to co-exist (which is not even universal policy for all FreeBSD users, which is why many developers don't do this). Unless of course you can get the whole world to adopt a common packaging API for their operating systems so it becomes worthwhile for portable application developers... In any case it'll take years to be able to rely on the feature, but at least the mechanism is there for the ports which matter to somebody. > Actually you might be able to do it by forcing autoconf to code in a > meta-name for each the directory prefixes and linking against an overridden > version of open(2) that properly translates the meta-name. But that's > icky. Ick! > > If you didn't want the package to have to look in /var/db/pkg, have > > the package binaries hold a maximum-length padded string variable (marked > > similarly to what(1) strings), and provide a tool to edit the binary (and > > scripts) at install time. > > Similar to the above, substituting scary editing of binaries for > scary library linking. ;^) Mine was a simpler low-tech icky-ness. ;-) > > I think environment variables are too fragile for this purpose. > > > > > You could almost do this on UNIX now, by putting such settings into > > > init's environment, if you could get the application developers to > > > write their code (especially libraries) appropriately. > > > > This doesn't sound like it would be easy to allow different packages to > > be installed with different roots. > > You're right. VMS has a system table, a per-group table, and a per-user > table to allow overrides. They teeter between being neat features and > be overkill. It's underkill for our purposes if it doesn't have them per-package... Cheers, Mark. -- Mark Valentine, Thuvia Labs "Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses "We're kind of stupid that way." *munch* *munch* and endorses FreeBSD -- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 8:39:46 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 134B137B400 for ; Fri, 12 Jul 2002 08:39:43 -0700 (PDT) Received: from smtp.netcologne.de (smtp.netcologne.de [194.8.194.112]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0A42C43E6E for ; Fri, 12 Jul 2002 08:39:42 -0700 (PDT) (envelope-from tmseck-lists@netcologne.de) Received: from localhost (xdsl-213-168-116-252.netcologne.de [213.168.116.252]) by smtp.netcologne.de (8.12.2/8.12.2) with ESMTP id g6CFdaUt000341 for ; Fri, 12 Jul 2002 17:39:36 +0200 (MEST) Received: (qmail 808 invoked by uid 1001); 12 Jul 2002 14:48:54 -0000 Date: Fri, 12 Jul 2002 16:48:54 +0200 From: Thomas Seck To: freebsd-arch@freebsd.org Subject: Re: Package system flaws? Message-ID: <20020712144854.GA756@laurel.tmseck.homedns.org> References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> <20020712121427.GD3678@lummux.tchpc.tcd.ie> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020712121427.GD3678@lummux.tchpc.tcd.ie> User-Agent: Mutt/1.4i Organization: private site in Germany X-PGP-KeyID: DF46EE05 X-PGP-Fingerprint: A38F AE66 6B11 6EB9 5D1A B67D 2444 2FE1 DF46 EE05 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * Robert bobb Crosbie (bobb+freebsd-arch@redbrick.dcu.ie): > I built apache2 with ``WITH_SUEXEC=yes'', then after the chunking thing > I did a ``portupgrade apache'', apache no longer works, scratched head > for a while until I rememberd how I buile it origionally. portupgrade can handle this. /usr/local/etc/pkgtools.conf.sample tells you how to do it. -- Thomas Seck To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 10:55:17 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3DF4B37B400 for ; Fri, 12 Jul 2002 10:55:13 -0700 (PDT) Received: from herbelot.dyndns.org (d108.dhcp212-198-26.noos.fr [212.198.26.108]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5B25B43E5E for ; Fri, 12 Jul 2002 10:55:11 -0700 (PDT) (envelope-from thierry@herbelot.com) Received: from herbelot.com (tulipe.herbelot.nom [192.168.2.5]) by herbelot.dyndns.org (8.9.3/8.9.3) with ESMTP id TAA23397; Fri, 12 Jul 2002 19:54:45 +0200 (CEST) (envelope-from thierry@herbelot.com) Message-ID: <3D2F17E5.5ED27749@herbelot.com> Date: Fri, 12 Jul 2002 19:54:45 +0200 From: Thierry Herbelot X-Mailer: Mozilla 4.79 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Wes Peters Cc: arch@FreeBSD.ORG Subject: Re: Listening to users [was Re: Package system wishlist] References: <3D2BE142.E25CA9BC@mindspring.com> <3D2CAAC1.5CF66C96@herbelot.com> <3D2E8A43.AF04381@softweyr.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Wes Peters wrote: > > Thierry Herbelot wrote: > > > > [Snip the long arguments over which new format for the packages] > > > > there is a interesting post by an ex-user of desktop Linux, which can > > help when designing the new package system (it will have to be used by > > mere mortals, not only by the elite posting on -arch) > > > > > > (shamelessly cut&pasted from /.) > > > > we still have a long way to go before even computer-aware people (that > > is, even people formely developping on SGI or Sun machines) can feel at > > ease installing and using FreeBSD. ^^^^^^^^^^ > > Really? Believe it or not, I've formerly (and currently) developed on > SGI and Sun machines, along with IBM, HP, DEC, Motorola, Intel, NCR, > Unisys, Sequent, CCI, Arete/Arix, and even Radio Shack UNIX-ish > machines, and I strongly prefer FreeBSD to any of them. I was not clear enough : I too wouldn't any day exchange FreeBSD for any commercial Unix version (FreeBSD is really a hacker's delight : everything is accessible, and there are people to explain its mysteries !) FreeBSD is also a "place" where ordinary people learn quite a lot of what is generally hidden elsewhere. What I was stressing is that people coming from the world of "packaged" "easy-to-install" operating systems (even Solaris holds your hand for the installation, and the limited hardware scope allows a full driver coverage, for sure) are a bit afraid when (e.g.) presented with 3 choices for the XFree config, with the first two are buggy. TfH To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 12:55: 5 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BA8FF37B400 for ; Fri, 12 Jul 2002 12:55:03 -0700 (PDT) Received: from mail.rpi.edu (mail.rpi.edu [128.113.22.40]) by mx1.FreeBSD.org (Postfix) with ESMTP id 06B7D43E4A for ; Fri, 12 Jul 2002 12:55:03 -0700 (PDT) (envelope-from drosih@rpi.edu) Received: from [128.113.24.47] (gilead.netel.rpi.edu [128.113.24.47]) by mail.rpi.edu (8.12.1/8.12.1) with ESMTP id g6CJstFm172566; Fri, 12 Jul 2002 15:54:55 -0400 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: <20020712121427.GD3678@lummux.tchpc.tcd.ie> References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> <20020712121427.GD3678@lummux.tchpc.tcd.ie> Date: Fri, 12 Jul 2002 15:54:54 -0400 To: Robert bobb Crosbie From: Garance A Drosihn Subject: Re: Package system flaws? Cc: arch@FreeBSD.ORG Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Scanned-By: MIMEDefang 2.3 (www dot roaringpenguin dot com slash mimedefang) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG At 1:14 PM +0100 7/12/02, Robert bobb Crosbie wrote: >This reminds me, for ports it would be rather handy if each >installed port could save the make options that you passed >to when it was built so then when you build it again it can >recall and use those options. If you use portupgrade/portinstall for all your builds, then you can put such settings in the config file for portupgrade. Check the files under /usr/local/etc/pkgtools.conf* If you remember to put any settings in that pkgtools.conf file, then they will be remembered when you rebuild the port. -- Garance Alistair Drosehn = gad@gilead.netel.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 13:11:59 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 987BD37B400 for ; Fri, 12 Jul 2002 13:11:57 -0700 (PDT) Received: from mgr4.xmission.com (mgr4.xmission.com [198.60.22.204]) by mx1.FreeBSD.org (Postfix) with ESMTP id 33DC643E31 for ; Fri, 12 Jul 2002 13:11:53 -0700 (PDT) (envelope-from glewis@misty.eyesbeyond.com) Received: from mail by mgr4.xmission.com with spam-scanned (Exim 3.35 #1) id 17T6lc-0003Uk-00; Fri, 12 Jul 2002 14:11:52 -0600 Received: from [207.135.128.145] (helo=misty.eyesbeyond.com) by mgr4.xmission.com with esmtp (Exim 3.35 #1) id 17T6la-0003UK-00; Fri, 12 Jul 2002 14:11:52 -0600 Received: (from glewis@localhost) by misty.eyesbeyond.com (8.11.6/8.11.6) id g6CKBf926316; Sat, 13 Jul 2002 05:41:41 +0930 (CST) (envelope-from glewis) Date: Sat, 13 Jul 2002 05:41:41 +0930 From: Greg Lewis To: Thomas Seck Cc: freebsd-arch@FreeBSD.ORG Subject: Re: Package system flaws? Message-ID: <20020713054141.A26277@misty.eyesbeyond.com> References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> <20020712121427.GD3678@lummux.tchpc.tcd.ie> <20020712144854.GA756@laurel.tmseck.homedns.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20020712144854.GA756@laurel.tmseck.homedns.org>; from tmseck-lists@netcologne.de on Fri, Jul 12, 2002 at 04:48:54PM +0200 X-Spam-Status: No, hits=-3.5 required=8.0 tests=IN_REP_TO,SUBJ_ENDS_IN_Q_MARK version=2.31 X-Spam-Level: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, Jul 12, 2002 at 04:48:54PM +0200, Thomas Seck wrote: > * Robert bobb Crosbie (bobb+freebsd-arch@redbrick.dcu.ie): > > > I built apache2 with ``WITH_SUEXEC=yes'', then after the chunking thing > > I did a ``portupgrade apache'', apache no longer works, scratched head > > for a while until I rememberd how I buile it origionally. > > portupgrade can handle this. /usr/local/etc/pkgtools.conf.sample tells > you how to do it. Thats both of the problems: (a) portupgrade isn't part of the standard package system. (b) I have to do it. I already told the packaging system what I wanted when I built the port, I shouldn't have to tell it again. -- Greg Lewis Email : glewis@eyesbeyond.com Eyes Beyond Web : http://www.eyesbeyond.com Information Technology FreeBSD : glewis@FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 16:25:15 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 59EDF37B401; Fri, 12 Jul 2002 16:25:03 -0700 (PDT) Received: from thuvia.demon.co.uk (thuvia.demon.co.uk [193.237.34.248]) by mx1.FreeBSD.org (Postfix) with ESMTP id 48D6E43E4A; Fri, 12 Jul 2002 16:25:00 -0700 (PDT) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (dotar.thuvia.org [10.0.0.4]) by phaidor.thuvia.org (8.11.6/8.11.6) with ESMTP id g6CNOTb37380; Sat, 13 Jul 2002 00:24:30 +0100 (BST) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (localhost [IPv6:::1]) by dotar.thuvia.org (8.12.3/8.12.3) with ESMTP id g6CNOTKC020673; Sat, 13 Jul 2002 00:24:29 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.3/8.12.3/Submit) id g6CNOQ7L020672; Sat, 13 Jul 2002 00:24:26 +0100 (BST) Date: Sat, 13 Jul 2002 00:24:26 +0100 (BST) From: Mark Valentine Message-Id: <200207122324.g6CNOQ7L020672@dotar.thuvia.org> In-Reply-To: Terry Lambert's message of Jul 12, 3:19pm X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: Terry Lambert Subject: Re: Package system flaws? Cc: Wes Peters , Jordan K Hubbard , Dan Nelson , Archie Cobbs , Dan Moschuk , Dag-Erling Smorgrav , arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: Terry Lambert > Date: Fri 12 Jul, 2002 > Subject: Re: Package system flaws? > At this point, the best approach is probably a registry. It already exists for our purposes: /var/db/pkg. One improvement to its existing structure would be to move all the FreeBSD package metadata under a freebsd.org sub-directory, and have all FreeBSD ports register under this vendor name space, allowing other vendors' packages to co-exist more easily using the same base system tools. (Using a seperate pkg db area would acceptable, but only if the location could be specified within the package metadata; i.e. pkg_add /cdrom/mypkg.tgz should work for third party packages without having to specify additional options.) I'm speaking as a vendor who intends to ship binary packages (albeit supplied with full source; it's just easier for customers who don't [yet] need to customise); some will be proprietary, others will be versions of software for which FreeBSD already has ports; currently I just intend to prepend an almost unique prefix to the package name (and of course install it somewhere unlikely to clash with another vendor's packages). Being guaranteed the /var/db/pkg/thuvia.co.uk/* (or /var/db/pkg/uk/co/thuvia/*, whatever) name space would be better, of course, as would be being able to re-target the installation (but I can of course resort to my icky hack of editing the binaries in a post-install script if this becomes an issue). > While it's theoretically possible to edit this data in place, > assuming the path is not assembled from components at runtime, > in practice, you are limited to a path of equal or smaller size. My location strings would be padded out to a reasonable maximum size. > A lesser alternative is to have a "/var/opt" or some other > location which is guaranteed to be a symlink to the real path > location... and is never, itself, a real directory. Well, we could already create such a symlink, even on a per-package basis, as (for example) /var/db/pkg/foo-x.y/location... > In an ideal world, you would probably want to put each package > and all dependent files, including the shared libraries it > cares about, into its own directory. This was really Windows > original problem. I'd rather put some trust in our shared library versioning and package dependency schemes, and actually share the shared libraries where it makes sense... However, you're already free to use your own policy here. > A number of researchers have suggested that this problem could > be solved by causing packages themselves to be in directories > that were exported as their own mini FS's. This "live image" > could be mounted read-only off a remote server, or locally off > a CDROM, without introducing further issues. See QNX 6 for an implementation which actually looks like it might be functional! (These QNX folks are smart.) The package installs in its own private area, and registers with the package file system so that its component files magically appear in all the expected places in /usr/bin and so on. > There are really three problems with this: > > 1) Windows is limited to 26(32) mount points: one per drive > letter; Not that I care a whit about whether our packaging scheme would work with Windoze, but I vaguely recall hearing news of them fixing that a few years back... > I personally don't know if I'm ready to fight the "Not Invented Here" > war that trying to implement a registry entails. There are plenty of lightweight UNIX-like mechanisms to solve all the problems they tried to solve with that mess. And you don't have to solve them all using the same one... Cheers, Mark. -- Mark Valentine, Thuvia Labs "Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses "We're kind of stupid that way." *munch* *munch* and endorses FreeBSD -- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 16:41:56 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5DAF337B400 for ; Fri, 12 Jul 2002 16:41:53 -0700 (PDT) Received: from w250.z064001178.sjc-ca.dsl.cnc.net (adsl-66.218.45.239.dslextreme.com [66.218.45.239]) by mx1.FreeBSD.org (Postfix) with SMTP id D1D6243E5E for ; Fri, 12 Jul 2002 16:41:52 -0700 (PDT) (envelope-from jos@catnook.com) Received: (qmail 26659 invoked by uid 1000); 12 Jul 2002 23:42:14 -0000 Date: Fri, 12 Jul 2002 16:41:52 -0701 From: Jos Backus To: arch@FreeBSD.ORG Subject: Re: Package system flaws? Message-ID: <20020712234214.GC32583@lizzy.catnook.com> Reply-To: jos@catnook.com Mail-Followup-To: arch@FreeBSD.ORG References: <200207122324.g6CNOQ7L020672@dotar.thuvia.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200207122324.g6CNOQ7L020672@dotar.thuvia.org> User-Agent: Mutt/1.5.1i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, Jul 13, 2002 at 12:24:26AM +0100, Mark Valentine wrote: > Being guaranteed the /var/db/pkg/thuvia.co.uk/* (or > /var/db/pkg/uk/co/thuvia/*, whatever) name space would be better, of course, > as would be being able to re-target the installation (but I can of course > resort to my icky hack of editing the binaries in a post-install script if > this becomes an issue). Also, fyi, see the ``Delegations'' section on http://cr.yp.to/slashpackage/names.html -- Jos Backus _/ _/_/_/ Santa Clara, CA _/ _/ _/ _/ _/_/_/ _/ _/ _/ _/ jos@catnook.com _/_/ _/_/_/ require 'std/disclaimer' To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 18:16:58 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 73EF837B400; Fri, 12 Jul 2002 18:16:52 -0700 (PDT) Received: from flamingo.mail.pas.earthlink.net (flamingo.mail.pas.earthlink.net [207.217.120.232]) by mx1.FreeBSD.org (Postfix) with ESMTP id D609643E58; Fri, 12 Jul 2002 18:16:51 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0223.cvx40-bradley.dialup.earthlink.net ([216.244.42.223] helo=mindspring.com) by flamingo.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17TBWS-0007CW-00; Fri, 12 Jul 2002 18:16:33 -0700 Message-ID: <3D2F7F1A.D7C1DBE4@mindspring.com> Date: Fri, 12 Jul 2002 18:15:06 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Mark Valentine Cc: Wes Peters , Jordan K Hubbard , Dan Nelson , Archie Cobbs , Dan Moschuk , Dag-Erling Smorgrav , arch@freebsd.org Subject: Re: Package system flaws? References: <200207122324.g6CNOQ7L020672@dotar.thuvia.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Mark Valentine wrote: > > From: Terry Lambert > > At this point, the best approach is probably a registry. > > It already exists for our purposes: /var/db/pkg. If you can tell me how to do a quick indexed lookup by key for something like "path to configuration file", which wanted to be in "/etc", was comipled to be in "/usr/local/etc", and which the user wants to install in "/var/opt/etc", I'd be grateful. 8-). > > While it's theoretically possible to edit this data in place, > > assuming the path is not assembled from components at runtime, > > in practice, you are limited to a path of equal or smaller size. > > My location strings would be padded out to a reasonable maximum size. With respect, I'm not worried about *your* location strings, I'm worried about the location strings that end up in binaries that result from "configure" resulting from "configure.in" from "autoconf", etc.. If we were just talking about *your* location strings, then I could be pretty sure that whatever API the OS presented, you would use it. > > A lesser alternative is to have a "/var/opt" or some other > > location which is guaranteed to be a symlink to the real path > > location... and is never, itself, a real directory. > > Well, we could already create such a symlink, even on a per-package basis, > as (for example) /var/db/pkg/foo-x.y/location... It's really ugly. And, as I pointed out, "lesser". 8-). > > In an ideal world, you would probably want to put each package > > and all dependent files, including the shared libraries it > > cares about, into its own directory. This was really Windows > > original problem. > > I'd rather put some trust in our shared library versioning and package > dependency schemes, and actually share the shared libraries where it > makes sense... > > However, you're already free to use your own policy here. Sharing of libraries is possible via factoring of code. The DLL scheme used by Windows is particularly amenable to the approach I had outlined, since all libraries are generally accessed via compiled in stub libraries that use the OpenLibrary() to explicitly gain access to the stubbed interfaces, in the case of shared libraries. Windows also lacks the ability for major versions to exist simultaneously. FreeBSD, as of the ELF conversion, lacks the ability for minor versions to exist simultaneously. In theory, this is OK. In practice, for shared libraries from third party vendors (e.g. "libintl" from GNU getopt), minor version numbering is a necessity, as more recent "minor" versions would *remove* code used by programs linked with older versions. Basically, you can't rely on third party vendors to do versioning right, you have to either do it for them, or, if you don't have their source code available so you can do that, you have to containerize their use of code, so that coexistance is possible, even if it's normally undesirable. For dependent shared libraries, we are effectively talking about the ability to provide a module shared by various programs, but with an explicit dependency. To deal with the versioning, you would need to be *able* to have different versions, and then *choose to not use them*. Effectively, this means symlinks to the real thing for use as a reference count. It also means that the "real thing" goes into a known location that is different from the known location owned by the vendor. This is effectively the "module problem" mentioned previously. This is compounded by the fact that modules optional at compile time are, if present at compile time, mandatory at runtime. This is one of the fundamental flaws in the "autoconf/automake" paradigm that no one ever seems to recognize publically. The FreeBSD ports system package build for release recognizes this by insisting on a "pristine" (jailed) build environment per package, with the "mandatory optional components" explicitly spelled out by the port maintainer in the port metadata (e.g. you can build without OpenSSL, but OpenSSL is required). Going in that direction works... but then you end up with a port per combinatoric feature, e.g.: /usr/ports/www/apache-jserv /usr/ports/www/apache13 /usr/ports/www/apache13+ipv6 /usr/ports/www/apache13-fp /usr/ports/www/apache13-modssl /usr/ports/www/apache13-ssl (Note: in this particular example, the combinatorics are *not* worked out -- and therefore, some combinations are simply not supported). In practice: yes, it would be nice to be able to dictate a universal module API to software vendors, but in the real world, FreeBSD does not wield monopolistic power in the marketplace. If anything, given a choice, most vendors would make things run on Linux, and to hell with other platforms, even if "they did things right". > > A number of researchers have suggested that this problem could > > be solved by causing packages themselves to be in directories > > that were exported as their own mini FS's. This "live image" > > could be mounted read-only off a remote server, or locally off > > a CDROM, without introducing further issues. > > See QNX 6 for an implementation which actually looks like it might be > functional! (These QNX folks are smart.) The package installs in its > own private area, and registers with the package file system so that > its component files magically appear in all the expected places in > /usr/bin and so on. I was thinking in particular of the ability to create FS "views" on data sets; however, looking at the QNX facility, that's another way of doing it successfully. I think the point is to learn that it's possible to do, instead of believing it isn't. It seems that most people have been arguing that it isn't possible: any example to the contrary is welcome. 8-). > > There are really three problems with this: > > > > 1) Windows is limited to 26(32) mount points: one per drive > > letter; > > Not that I care a whit about whether our packaging scheme would work with > Windoze, but I vaguely recall hearing news of them fixing that a few years > back... The point was why it was not widely deployed in the standard example OS ("If it's so good, why doesn't Microsoft do it?"). 8-). The fix they have is the IFSMgr fix I described previously, where you are still limited on sources of exported FS's by the number of drive letters. Don't adopt MS's limitations as your own, unless it's on purpose. > > I personally don't know if I'm ready to fight the "Not Invented Here" > > war that trying to implement a registry entails. > > There are plenty of lightweight UNIX-like mechanisms to solve all the > problems they tried to solve with that mess. And you don't have to > solve them all using the same one... If you don't solve them with a centralized mechanism, then you have failed to create a Schelling point. At some time in the future -- usually sooner, rather than later -- you will find that by doing it piecemeal, you have created a problem situation. Specifically, if you choose to solve the problem in package "B" by writing FreeBSD specific code for handling the issue using a particular approach, and then someone takes it upon themselves to do the same for dependency "A", with a different approach, then you are pretty much screwed. The issue has to be solved "bottom up"; even if you could do this by picking the various packages on which other packages depend, but which depend on no others, you haven't solved the problem for the base system, and you haven't solved the problem when someone adds a new dependency. It's like the Issac Asimov story about the Mars City Museum robbery, where Mars is a dry planet, and it just so happened that the place they chose to put the date-line ended up with a city on it. The thieves find the museum opening at 9AM *Monday*, while they are hanging from ropes from a skylight with a bypassed security system, on a day they think is *Sunday*. When you hit the interference graph, you are going to find impedence mismatches. Moire patterns happen for a reason. 8-). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 18:20:20 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4131037B406; Fri, 12 Jul 2002 18:20:09 -0700 (PDT) Received: from soulshock.mail.pas.earthlink.net (soulshock.mail.pas.earthlink.net [207.217.120.130]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7D0D343E75; Fri, 12 Jul 2002 18:20:08 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from hawk.mail.pas.earthlink.net (hawk.mail.pas.earthlink.net [207.217.120.22]) by soulshock.mail.pas.earthlink.net (8.11.6+Sun/8.11.6) with ESMTP id g6CMLLI08846; Fri, 12 Jul 2002 15:21:21 -0700 (PDT) Received: from pool0049.cvx22-bradley.dialup.earthlink.net ([209.179.198.49] helo=mindspring.com) by hawk.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17T8m7-000604-00; Fri, 12 Jul 2002 15:20:31 -0700 Message-ID: <3D2F5602.9039F5DE@mindspring.com> Date: Fri, 12 Jul 2002 15:19:46 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Mark Valentine Cc: Wes Peters , Jordan K Hubbard , Dan Nelson , Archie Cobbs , Dan Moschuk , Dag-Erling Smorgrav , arch@freebsd.org Subject: Re: Package system flaws? References: <200207120055.g6C0tbpQ084565@dotar.thuvia.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Mark Valentine wrote: > wes@softweyr.com (Wes Peters) > > On problem in particular comes from changing the PREFIX of a > > package. You can easily do this in the package tools, but how do you > > change the BINARIES to realize they've been installed under another > > PREFIX? > > How about providing a packaging API call (and utility) for the packages > to grab their install root from a record in /var/db/pkg? Something > like pkg_getroot(3) and pkg_config(1). > > There probably isn't much less intrusive way to make a package's root > configurable at install time. > > If you didn't want the package to have to look in /var/db/pkg, have > the package binaries hold a maximum-length padded string variable (marked > similarly to what(1) strings), and provide a tool to edit the binary (and > scripts) at install time. > > I think environment variables are too fragile for this purpose. At this point, the best approach is probably a registry. The problem that you are trying to solve with this is that there is implied knowledge of a fixed path which is compiled into data stored in the binary. While it's theoretically possible to edit this data in place, assuming the path is not assembled from components at runtime, in practice, you are limited to a path of equal or smaller size. A lesser alternative is to have a "/var/opt" or some other location which is guaranteed to be a symlink to the real path location... and is never, itself, a real directory. This still would not address the ability to locate packages in more than one installation target location at a time, when you have multiple packages on a system: you are still limited to one directory hierarchy. In an ideal world, you would probably want to put each package and all dependent files, including the shared libraries it cares about, into its own directory. This was really Windows original problem. A number of researchers have suggested that this problem could be solved by causing packages themselves to be in directories that were exported as their own mini FS's. This "live image" could be mounted read-only off a remote server, or locally off a CDROM, without introducing further issues. Likewise, it would permit control of licensing by providing a usecount control point that could be monitored asn adminstered centrally, with little additional effort. The main benefits to this are that you install an application once, it never interferes with the operation of another applicaiton, and it's location is a Schelling point which can be identified by other applications easily, if they depend on it. A secondary benefit is that, "if you need powerpoint for a presentation, you can mount it from the local public network and use a local license instance, rather than carrying it around with you". That's probably not a "benefit", if your job is selling as many "powerpoint" licenses as humanly possible. 8-). There are really three problems with this: 1) Windows is limited to 26(32) mount points: one per drive letter; so it is difficult to make this work in Windows, though it can be done... either through unification of the applicaiton namespace on a local network through peer-to-peer implementation, or through a single FS that hooks IFSMgr to turn directories themselves into mount points in the Windows FS namespace, rather than relying on the Windows "drive letter" mechanism, whose only benefit is "it's already supported". 2) Applications vendors don't write applications this way, so needing them to be written this way implies a constraint on porting. 3) Cross-references are unidirectional along a graph edge; the implications of this may not be obvious at first, but to break it down into simple terms: it's hard to add plug-ins to applications that already exist, without those applications knowing about the possibility of a given plug-in at the time they are written, without writing the plug-in to a directory owned by the original application -- thereby violating the model; e.g.: C:\Program Files\Netscape\Communicator\Program\Plugins Once again, some of this can be resolved... with a Schelling point that is location invariant ... or within which, locations are themselves invariant: once again... a registry. I personally don't know if I'm ready to fight the "Not Invented Here" war that trying to implement a registry entails. As an idea that came from Windows, there is a gut-level reaction among many UNIX people that tries to pretend that engineers employed by Microsoft (or, sometimes, even RedHat) could not possibly have a good idea on their own, or solve a problem in a way that was not already thought of by themselves. Maybe we can table the "change PREFIX on binary during install" issue while we slog through the rest of the mess, and then come back to it when people aren't looking? 8-). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 18:28:13 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9C5E737B400 for ; Fri, 12 Jul 2002 18:28:11 -0700 (PDT) Received: from smtp.netcologne.de (smtp.netcologne.de [194.8.194.112]) by mx1.FreeBSD.org (Postfix) with ESMTP id 885AB43E5E for ; Fri, 12 Jul 2002 18:28:10 -0700 (PDT) (envelope-from tmseck-lists@netcologne.de) Received: from localhost (xdsl-213-168-116-217.netcologne.de [213.168.116.217]) by smtp.netcologne.de (8.12.2/8.12.2) with ESMTP id g6D1S5Ut028470 for ; Sat, 13 Jul 2002 03:28:05 +0200 (MEST) Received: (qmail 1440 invoked by uid 1001); 13 Jul 2002 01:17:50 -0000 Date: Sat, 13 Jul 2002 03:17:50 +0200 From: Thomas Seck To: freebsd-arch@freebsd.org Subject: Re: Package system flaws? Message-ID: <20020713011750.GA755@laurel.tmseck.homedns.org> References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> <20020712121427.GD3678@lummux.tchpc.tcd.ie> <20020712144854.GA756@laurel.tmseck.homedns.org> <20020713054141.A26277@misty.eyesbeyond.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020713054141.A26277@misty.eyesbeyond.com> User-Agent: Mutt/1.4i Organization: private site in Germany X-PGP-KeyID: DF46EE05 X-PGP-Fingerprint: A38F AE66 6B11 6EB9 5D1A B67D 2444 2FE1 DF46 EE05 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * Greg Lewis (glewis@eyesbeyond.com): > On Fri, Jul 12, 2002 at 04:48:54PM +0200, Thomas Seck wrote: > > * Robert bobb Crosbie (bobb+freebsd-arch@redbrick.dcu.ie): > > > > > I built apache2 with ``WITH_SUEXEC=yes'', then after the chunking thing > > > I did a ``portupgrade apache'', apache no longer works, scratched head > > > for a while until I rememberd how I buile it origionally. > > > > portupgrade can handle this. /usr/local/etc/pkgtools.conf.sample tells > > you how to do it. > > Thats both of the problems: > > (a) portupgrade isn't part of the standard package system. > (b) I have to do it. I already told the packaging system what I wanted > when I built the port, I shouldn't have to tell it again. The person I was replying to explicitly claimed that he wedged his apache update with portupgrade. Had he bothered to read the sample configuration or the documentation, he would have quickly learned how to avoid this. But I agree with you. A tool like portupgrade should not exist (and it should not have been written in Ruby - yet another dependency to track). The inability of the present ports and package system to maintain a consistent package database is apalling. The way dependencies are recorded is plain broken. In my opinion when specify a dependency, the minimum working requirement should be specified (e.g. 'gmake 3.79.1, do not care about VERSION or REVISION'). The package installer should decide at installation time via a package db lookup whether a locally installed package fulfills the minimum requirement and record the dependencies according to what was actually found on the system. When a depending package is not present or outdated, a recursive update should be done. The same applies for the ports system. At present, the dependencies in the package db reflect the state of the ports tree at build time but not the state of packages present on the system. -- Thomas Seck To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 18:34:18 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A740737B400 for ; Fri, 12 Jul 2002 18:34:16 -0700 (PDT) Received: from swan.mail.pas.earthlink.net (swan.mail.pas.earthlink.net [207.217.120.123]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4275243E31 for ; Fri, 12 Jul 2002 18:34:16 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0223.cvx40-bradley.dialup.earthlink.net ([216.244.42.223] helo=mindspring.com) by swan.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17TBnS-0005fu-00; Fri, 12 Jul 2002 18:34:07 -0700 Message-ID: <3D2F8331.465C6ED7@mindspring.com> Date: Fri, 12 Jul 2002 18:32:33 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: jos@catnook.com Cc: arch@FreeBSD.ORG Subject: Re: Package system flaws? References: <200207122324.g6CNOQ7L020672@dotar.thuvia.org> <20020712234214.GC32583@lizzy.catnook.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Jos Backus wrote: > On Sat, Jul 13, 2002 at 12:24:26AM +0100, Mark Valentine wrote: > > Being guaranteed the /var/db/pkg/thuvia.co.uk/* (or > > /var/db/pkg/uk/co/thuvia/*, whatever) name space would be better, of course, > > as would be being able to re-target the installation (but I can of course > > resort to my icky hack of editing the binaries in a post-install script if > > this becomes an issue). > > Also, fyi, see the ``Delegations'' section on > > http://cr.yp.to/slashpackage/names.html He is cheating. He is serializing name space allocations through an individual person, in order to avoid collisions. This is basically the approach I tongue-in-cheek suggested when I jokingly offered the use of the original 386BSD "patchkit" software. Serialization through a human CVS repository is a bad idea; we already have a namespace allocation system (DNS), in which internal namespace delegation after you get past the individually assigned domain name, is up to you. The Java and Perl communities have used this with great success, e.g.: ${ROOT}/org/freebsd/system/bin/ls -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 19:13:28 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9999D37B400 for ; Fri, 12 Jul 2002 19:13:26 -0700 (PDT) Received: from c001.snv.cp.net (h000.c001.snv.cp.net [209.228.32.114]) by mx1.FreeBSD.org (Postfix) with SMTP id 03F6843E64 for ; Fri, 12 Jul 2002 19:13:26 -0700 (PDT) (envelope-from kutulu@kutulu.org) Received: (cpmta 18160 invoked from network); 12 Jul 2002 19:13:24 -0700 Received: from 68.50.102.254 (HELO KutuluWare) by smtp.register-admin.com (209.228.32.114) with SMTP; 12 Jul 2002 19:13:24 -0700 X-Sent: 13 Jul 2002 02:13:24 GMT Message-ID: <002401c22a12$57659c10$fe663244@KutuluWare> From: "Kutulu" To: References: <200207120055.g6C0tbpQ084565@dotar.thuvia.org> <3D2F5602.9039F5DE@mindspring.com> Subject: Re: Package system flaws? Date: Fri, 12 Jul 2002 22:09:37 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG ----- Original Message ----- From: "Terry Lambert" Sent: Friday, July 12, 2002 6:19 PM > At this point, the best approach is probably a registry. > > I personally don't know if I'm ready to fight the "Not Invented Here" > war that trying to implement a registry entails. As an idea that > came from Windows, there is a gut-level reaction among many UNIX > people that tries to pretend that engineers employed by Microsoft > (or, sometimes, even RedHat) could not possibly have a good idea on > their own, or solve a problem in a way that was not already thought > of by themselves. While I agree in general with this logic (that just because it's Microsoft's idea doesn't automatically make it bad), even THEY eventually concluded that the registry was a really bad way to manage a centralized software "package" database (COM components), and are moving towards a more directory-based, localized package management system with XP/.NET. I'm not yet fully versed on the whole .NET Manifest structure but it basically uses an XML file, located alongside the binaries I beleive, to describe everything Windows needs to know about an installed application/component/ whatever. Of course, history would indicate they'll discover this is a really bad idea by 2004 and can it, so who knows. --K To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 19:35:55 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C292237B400 for ; Fri, 12 Jul 2002 19:35:53 -0700 (PDT) Received: from thuvia.demon.co.uk (thuvia.demon.co.uk [193.237.34.248]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5252A43E4A for ; Fri, 12 Jul 2002 19:35:52 -0700 (PDT) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (dotar.thuvia.org [10.0.0.4]) by phaidor.thuvia.org (8.11.6/8.11.6) with ESMTP id g6D2Zhb38333; Sat, 13 Jul 2002 03:35:44 +0100 (BST) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (localhost [IPv6:::1]) by dotar.thuvia.org (8.12.3/8.12.3) with ESMTP id g6D2ZhKC024902; Sat, 13 Jul 2002 03:35:43 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.3/8.12.3/Submit) id g6D2ZhLq024901; Sat, 13 Jul 2002 03:35:43 +0100 (BST) Date: Sat, 13 Jul 2002 03:35:43 +0100 (BST) From: Mark Valentine Message-Id: <200207130235.g6D2ZhLq024901@dotar.thuvia.org> In-Reply-To: Terry Lambert's message of Jul 13, 1:43am X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: tlambert2@mindspring.com (Terry Lambert), jos@catnook.com Subject: Re: Package system flaws? Cc: arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: tlambert2@mindspring.com (Terry Lambert) > Date: Sat 13 Jul, 2002 > Subject: Re: Package system flaws? > Jos Backus wrote: > > Also, fyi, see the ``Delegations'' section on > > > > http://cr.yp.to/slashpackage/names.html > > He is cheating. He is serializing name space allocations through > an individual person, in order to avoid collisions. Um, you didn't read down to the ``Delegations'' section... Cheers, Mark. -- Mark Valentine, Thuvia Labs "Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses "We're kind of stupid that way." *munch* *munch* and endorses FreeBSD -- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 20:10:38 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2C7BF37B400 for ; Fri, 12 Jul 2002 20:10:36 -0700 (PDT) Received: from flamingo.mail.pas.earthlink.net (flamingo.mail.pas.earthlink.net [207.217.120.232]) by mx1.FreeBSD.org (Postfix) with ESMTP id B42E543E58 for ; Fri, 12 Jul 2002 20:10:35 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0343.cvx22-bradley.dialup.earthlink.net ([209.179.199.88] helo=mindspring.com) by flamingo.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17TDIm-0002bS-00; Fri, 12 Jul 2002 20:10:33 -0700 Message-ID: <3D2F99FC.61337E12@mindspring.com> Date: Fri, 12 Jul 2002 20:09:48 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Kutulu Cc: arch@freebsd.org Subject: Re: Package system flaws? References: <200207120055.g6C0tbpQ084565@dotar.thuvia.org> <3D2F5602.9039F5DE@mindspring.com> <002401c22a12$57659c10$fe663244@KutuluWare> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Kutulu wrote: > While I agree in general with this logic (that just because it's Microsoft's > idea doesn't automatically make it bad), even THEY eventually concluded that > the registry was a really bad way to manage a centralized software "package" > database (COM components), and are moving towards a more directory-based, > localized package management system with XP/.NET. That's not really what they concluded, if we want to be honest about things. A hierarchical data store is a hierarchical data store. Doesn't matter if it's an FS, a registry, an LDAP directory, a DNS server, or FreeBSD's use of "_" as a path component seperator in /etc/rc.conf ...hierarchical is hierarchical. The choice of implementation technology is totally seperate from that of structure; it only becomes an issue when you need to look data up. Finding one tag in one of a thousand files is, to be blunt, computationally Very Hard, which was one of the arguments originally put forward against the NetBSD RC system, before it was understood that configuration data remained centralized, and that one could assemble and cache a single script instance. The COM component registration model is actually a good model, since the issue is to ensure uniqueness within the namespace; it fails under ongoing maintenance because it picked uniquifiers that were not invariant between maintainers of a particular set of code, or even the same maintainer, over time: the MAC ID of the ethernet car in the workstation used to perform the component definition. COM failed because the variant part failed to group derivative works, as they had hoped it would. It also failed because of the method of interface versioning they selected, but moving to files doesn't fix the lack of major/minor numbers in the file naming conventions (the Visual C RunTime Library puts the version number in the file name, *without establishing a convention*, in order to get around this same problem). In other words, their failures were not attributable to their use of a hierarchical data model (structure), but to their selection of hierarchy identifiers (selection of identifier can be stretched to fit under "implementation technology", even if I'd argue that it's a third, and therefore totally irrelevant, factor). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 20:18: 5 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5A69137B400 for ; Fri, 12 Jul 2002 20:18:03 -0700 (PDT) Received: from flamingo.mail.pas.earthlink.net (flamingo.mail.pas.earthlink.net [207.217.120.232]) by mx1.FreeBSD.org (Postfix) with ESMTP id E2C7D43E67 for ; Fri, 12 Jul 2002 20:18:02 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0343.cvx22-bradley.dialup.earthlink.net ([209.179.199.88] helo=mindspring.com) by flamingo.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17TDPw-0002E8-00; Fri, 12 Jul 2002 20:17:57 -0700 Message-ID: <3D2F9BB8.6C2F0D34@mindspring.com> Date: Fri, 12 Jul 2002 20:17:12 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Mark Valentine Cc: jos@catnook.com, arch@freebsd.org Subject: Re: Package system flaws? References: <200207130235.g6D2ZhLq024901@dotar.thuvia.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Mark Valentine wrote: > tlambert2@mindspring.com (Terry Lambert) > > Jos Backus wrote: > > > Also, fyi, see the ``Delegations'' section on > > > > > > http://cr.yp.to/slashpackage/names.html > > > > He is cheating. He is serializing name space allocations through > > an individual person, in order to avoid collisions. > > Um, you didn't read down to the ``Delegations'' section... I read it. It is not enforced. There is an escape mechanism through human serialization, and, given his organization, you would have to be insane not to take advantage of it. The problem is still one of back references for modules that attempt to add functionality to existing software at runtime, rather than at compile time, and compile time presence notification without explicit modification of the packages being notified (e.g. autoconf "finding" packages on the developers machine at compile time that are not explicitly listed as dependencies for run-time). Finding and using e.g. OpenSSL at runtime is OK; requiring it at runtime when it was optional at compile time, when it happened to be present, is not. The Perl and Java people resolve this by having packages register themselves into the hierarchy. Admittedly, they have an advantage, in that they have a module registration standard mechinism for "class factories" for specific implementation clases for abstract base classes... but that was kind of the point. 8-). You don't *have* to be an interpreter to do this, but it's very easy if you are. In a shell script, the same thing can be accomplished with "ls" and a "test -x" (ugly, but workable). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 21:12:41 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0C44E37B400 for ; Fri, 12 Jul 2002 21:12:39 -0700 (PDT) Received: from ussenterprise.ufp.org (ussenterprise.ufp.org [208.185.30.210]) by mx1.FreeBSD.org (Postfix) with ESMTP id 79F6543E67 for ; Fri, 12 Jul 2002 21:12:38 -0700 (PDT) (envelope-from bicknell@ussenterprise.ufp.org) Received: (from bicknell@localhost) by ussenterprise.ufp.org (8.11.1/8.11.1) id g6D3lPV48178; Fri, 12 Jul 2002 23:47:25 -0400 (EDT) (envelope-from bicknell) Date: Fri, 12 Jul 2002 23:47:25 -0400 From: Leo Bicknell To: freebsd-arch@freebsd.org Cc: louie@TransSys.COM, listsub@rambo.simx.org, tlambert2@mindspring.com, leifn@neland.dk Subject: Mail subsystem defaults, adding authentication. Message-ID: <20020713034725.GB47677@ussenterprise.ufp.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Organization: United Federation of Planets Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG [Those CC:ed, you send me mail in a freebsd-hackers discussion late last year.] [I am not on freebsd-arch, please keep me in CC's.] Short form: I believe it is important for FreeBSD's default install to either make it easy, or default to a setup that allows SMTP AUTH against the password file. To do this we need to include a SASL library. As such, I would like a SASL library in the base distribution. Long form: FreeBSD already supports STMP over SSL in the default install (if the user creates keys). There is a port for imap-ssl that works quite well, plus there is no imap in the default install. To provide secure, e-mail (sending and receiving) you need a download protocol that is encrypted (imap-ssl), and a sending protocol (SMTP AUTH) that prevents relaying. Since SMTP AUTH does not require non-cleartext passwords, doing SMTP AUTH over SSL is a good idea. SMTP AUTH checks a userid and password. To my knowledge, sendmail only supports using SASL to perform this function. We have SASL libraries in a port. If they are installed, a few flags can be changed so "make world" builds a sendmail that has SASL support. If the SASL port were installed as part of the base system, these flags could be the default. Since all major e-mail clients support IMAP/SSL and SMTP AUTH (usually over SSL, if IMAP/SSL is supported) this seems the right combination for a fully secure, non-open relay configuration. So, I would like comments on the following issues: 1) Is it desirable to provide a default install for which SMTP AUTH against the password file works? 2) If yes to #1, is including the cyrus-sasl port in the base distribution the best way to get a SASL library? [Included in this is license issues, code quality issues, etc.] If it is not the best, is there a better choice? At the end of the day, I think we are close to the right thing, which means imap-ssl is easy to install from a port (and easy to keep separate from the base system). SMTP over SSL seems to already be in the base system. The only thing preventing a user from running a "secure" SMTP/IMAP server from a base install, is the lack of SMTP AUTH. AFAIK, SASL is the only way to get that working, and cyrus-sasl is the best (technically) library available. At this time, I don't want to nit-pick on details, that can be done off list. What I'm really after is if there is support for making SMTP AUTH work in the base install, and if my _general_ outline is the right way to go about it. There are more appropriate places for the details. -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/ Read TMBG List - tmbg-list-request@tmbg.org, www.tmbg.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 21:14:48 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 65C4437B400 for ; Fri, 12 Jul 2002 21:14:45 -0700 (PDT) Received: from thuvia.demon.co.uk (thuvia.demon.co.uk [193.237.34.248]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4F44543E64 for ; Fri, 12 Jul 2002 21:14:43 -0700 (PDT) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (dotar.thuvia.org [10.0.0.4]) by phaidor.thuvia.org (8.11.6/8.11.6) with ESMTP id g6D4EXb38939; Sat, 13 Jul 2002 05:14:34 +0100 (BST) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (localhost [IPv6:::1]) by dotar.thuvia.org (8.12.3/8.12.3) with ESMTP id g6D4EXKC029386; Sat, 13 Jul 2002 05:14:33 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.3/8.12.3/Submit) id g6D4EXo6029385; Sat, 13 Jul 2002 05:14:33 +0100 (BST) Date: Sat, 13 Jul 2002 05:14:33 +0100 (BST) From: Mark Valentine Message-Id: <200207130414.g6D4EXo6029385@dotar.thuvia.org> In-Reply-To: Terry Lambert's message of Jul 12, 8:17pm X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: Terry Lambert Subject: Re: Package system flaws? Cc: jos@catnook.com, arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: Terry Lambert > Date: Fri 12 Jul, 2002 > Subject: Re: Package system flaws? > Mark Valentine wrote: > > Um, you didn't read down to the ``Delegations'' section... > > I read it. It is not enforced. There is an escape mechanism > through human serialization, and, given his organization, you > would have to be insane not to take advantage of it. That's an escape mechanism? Sorry, but giving a few units of currency to my local friendly domain registrar is much, much easier... I think Jos was pointing out just one of many places where there exists a convention of mapping vendor subsets of a "registry" name space onto domain registrations. Yes, Java classes are a more significant example. > The problem is still one of back references for modules that > attempt to add functionality to existing software at runtime, > rather than at compile time, and compile time presence > notification without explicit modification of the packages > being notified (e.g. autoconf "finding" packages on the > developers machine at compile time that are not explicitly > listed as dependencies for run-time). I'm very familar with the necessary structure of dependency graphs in component software, but give me till after my sleep period to figure out if that's relevant to any of the points I was trying to make... But please don't bring such design disasters as autoconf in to muddle the picture... (Hmm, it may not be autoconf's design at fault, but how it's used in every configure script. Without having used it in earnest, I'm given the impression it might be possible to use only autoconf rules which don't "guess".) > The Perl and Java people resolve this by having packages > register themselves into the hierarchy. Admittedly, they > have an advantage, in that they have a module registration > standard mechinism for "class factories" for specific > implementation clases for abstract base classes... but that > was kind of the point. 8-). You don't *have* to be an > interpreter to do this, but it's very easy if you are. Giving preferential power to scripts over worrying about the API was always a Unix strong point. ;-) > In a shell script, the same thing can be accomplished with "ls" > and a "test -x" (ugly, but workable). Ugly? After you just mentioned Perl and Java? I must have misread how long you've been with us, Terry. ;-) Cheers, Mark. -- Mark Valentine, Thuvia Labs "Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses "We're kind of stupid that way." *munch* *munch* and endorses FreeBSD -- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 21:43:21 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1319437B400 for ; Fri, 12 Jul 2002 21:43:19 -0700 (PDT) Received: from gull.mail.pas.earthlink.net (gull.mail.pas.earthlink.net [207.217.120.84]) by mx1.FreeBSD.org (Postfix) with ESMTP id 75CF143E6D for ; Fri, 12 Jul 2002 21:43:18 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0343.cvx22-bradley.dialup.earthlink.net ([209.179.199.88] helo=mindspring.com) by gull.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17TEkR-0001xr-00; Fri, 12 Jul 2002 21:43:11 -0700 Message-ID: <3D2FAFB2.E2E9CF36@mindspring.com> Date: Fri, 12 Jul 2002 21:42:26 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Leo Bicknell Cc: freebsd-arch@freebsd.org, louie@TransSys.COM, listsub@rambo.simx.org, leifn@neland.dk Subject: Re: Mail subsystem defaults, adding authentication. References: <20020713034725.GB47677@ussenterprise.ufp.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Leo Bicknell wrote: > So, I would like comments on the following issues: > > 1) Is it desirable to provide a default install for which SMTP AUTH > against the password file works? Yes. But it's not possible without destorying the ablity of the default install to run over port 25. The contents of the crypt-encrypted password are one-way hashed with an externally unrecoverable salt. The net effect of this is that you can not use crypt-based passwords unless they are encrypted for comparison on the server -- which means passing them over the wire as plaintext. You *force* the use of SSL, if this is enabled. > 2) If yes to #1, is including the cyrus-sasl port in the base > distribution the best way to get a SASL library? [Included > in this is license issues, code quality issues, etc.] If it > is not the best, is there a better choice? You would have to use SMTP over SSL, *NOT* "STARTTLS", and then enforce its use if "SMTP AUTH" is to be used, to make sending passwords in the clear acceptable due to the secure link. You are almost better off simply using SMTP over SSL, and permitting connections only to certificated clients, at which point you can just sign the client certificates and be done with it, without using the "SMTP AUTH" approach at all. An alternate approach would be to use the crypted passwords, with the salt being passed as apart of the SASL dialog, so the crypted password could be passed. This would be less painful, overall, for the server (clients outnumber servers by a large margin), but... it would require client modification, and the definition of an "x-crypt" authentication type (and/or a full RFC process to define it without the "-x"). The OpenLDAP list archives are quire extensive on this subject, as are the Cyrus list archives, and the Sendmail list archives, where there has been a similar desire to use the standard UNIX authentication mechanism with SASL. PAM has the same problem, if it's any consolation. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jul 12 21:57: 9 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EAB9937B400 for ; Fri, 12 Jul 2002 21:57:06 -0700 (PDT) Received: from ussenterprise.ufp.org (ussenterprise.ufp.org [208.185.30.210]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4BD2443E31 for ; Fri, 12 Jul 2002 21:57:06 -0700 (PDT) (envelope-from bicknell@ussenterprise.ufp.org) Received: (from bicknell@localhost) by ussenterprise.ufp.org (8.11.1/8.11.1) id g6D4v4949537; Sat, 13 Jul 2002 00:57:05 -0400 (EDT) (envelope-from bicknell) Date: Sat, 13 Jul 2002 00:57:04 -0400 From: Leo Bicknell To: Terry Lambert Cc: freebsd-arch@freebsd.org, louie@TransSys.COM, listsub@rambo.simx.org, leifn@neland.dk Subject: Re: Mail subsystem defaults, adding authentication. Message-ID: <20020713045704.GA49379@ussenterprise.ufp.org> References: <20020713034725.GB47677@ussenterprise.ufp.org> <3D2FAFB2.E2E9CF36@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3D2FAFB2.E2E9CF36@mindspring.com> Organization: United Federation of Planets Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In a message written on Fri, Jul 12, 2002 at 09:42:26PM -0700, Terry Lambert wrote: > You are almost better off simply using SMTP over SSL, and > permitting connections only to certificated clients, at which > point you can just sign the client certificates and be done > with it, without using the "SMTP AUTH" approach at all. I want to address this specifically. I have personally been involved with a half dozen situations where SMTP AUTH against the password file was desired. In most, certificates were also investigated. They were rejected for one or more of the following reasons: 1) There was no certificate management system already in place, and creating one just for e-mail was "too expensive". 2) Management wanted centrally revokable credentials, which generally means kerberos (although there are other methods for login access). While hooking that to certificates is possible, it is not the default in anything, so it's additional work on top of 1. 3) Configuring trusted credentials in the client software was non-trival for "regular" end users. 4) Users are used to authenticating with a "password", and preserving that model is a good thing. I don't suggest that any or all of them are good ideas, but simply that from my point of view everyone insists on passwords as the (only) mechanism. Today, most people I know are using ssh to port foward 25 + 110/143/220. ssh uses passwords, "localhost" can be trusted. This works. I want to make this go away, in the default install if at all possible. imap/ssl solves the mail download issue, so the only issue I see is how to you securely send mail, without compromising passwords, using protocols supported today by the majority of e-mail clients. SMTP AUTH, requiring SSL as I outlined before, is the only solution I have ever found. -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/ Read TMBG List - tmbg-list-request@tmbg.org, www.tmbg.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jul 13 0: 5:32 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C853437B400 for ; Sat, 13 Jul 2002 00:05:30 -0700 (PDT) Received: from w250.z064001178.sjc-ca.dsl.cnc.net (adsl-66.218.45.239.dslextreme.com [66.218.45.239]) by mx1.FreeBSD.org (Postfix) with SMTP id 5ABBA43E3B for ; Sat, 13 Jul 2002 00:05:30 -0700 (PDT) (envelope-from jos@catnook.com) Received: (qmail 3576 invoked by uid 1000); 13 Jul 2002 07:05:51 -0000 Date: Sat, 13 Jul 2002 00:05:29 -0700 From: Jos Backus To: arch@freebsd.org Subject: Re: Package system flaws? Message-ID: <20020713070551.GA931@lizzy.catnook.com> Reply-To: jos@catnook.com Mail-Followup-To: arch@freebsd.org References: <200207130414.g6D4EXo6029385@dotar.thuvia.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200207130414.g6D4EXo6029385@dotar.thuvia.org> User-Agent: Mutt/1.5.1i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, Jul 13, 2002 at 05:14:33AM +0100, Mark Valentine wrote: > I think Jos was pointing out just one of many places where there > exists a convention of mapping vendor subsets of a "registry" name > space onto domain registrations. Yes, Java classes are a more > significant example. Indeed; thanks, Mark, for expressing this more clearly. -- Jos Backus _/ _/_/_/ Santa Clara, CA _/ _/ _/ _/ _/_/_/ _/ _/ _/ _/ jos@catnook.com _/_/ _/_/_/ require 'std/disclaimer' To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jul 13 4:33:26 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4C1F137B400 for ; Sat, 13 Jul 2002 04:33:23 -0700 (PDT) Received: from avocet.mail.pas.earthlink.net (avocet.mail.pas.earthlink.net [207.217.120.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id E034843E3B for ; Sat, 13 Jul 2002 04:33:22 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0019.cvx22-bradley.dialup.earthlink.net ([209.179.198.19] helo=mindspring.com) by avocet.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17TL9M-0001Cx-00; Sat, 13 Jul 2002 04:33:20 -0700 Message-ID: <3D300FD4.7479A8E5@mindspring.com> Date: Sat, 13 Jul 2002 04:32:36 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Leo Bicknell Cc: freebsd-arch@freebsd.org, louie@TransSys.COM, listsub@rambo.simx.org, leifn@neland.dk Subject: Re: Mail subsystem defaults, adding authentication. References: <20020713034725.GB47677@ussenterprise.ufp.org> <3D2FAFB2.E2E9CF36@mindspring.com> <20020713045704.GA49379@ussenterprise.ufp.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Leo Bicknell wrote: > Terry Lambert wrote: > > You are almost better off simply using SMTP over SSL, and > > permitting connections only to certificated clients, at which > > point you can just sign the client certificates and be done > > with it, without using the "SMTP AUTH" approach at all. > > I want to address this specifically. > > I have personally been involved with a half dozen situations where > SMTP AUTH against the password file was desired. This isn't really a FreeBSD issue, except for the inclusion of the SASL library in the base system, which is not currently the case. The license on Cyrus SASL is right (BSD license) for it to be included int he base system (one of the things you are asking) and for sendmail to be configured to use it, and linked with it by default (the other thing you are asking). I personally approve of including it, but it *is* "bloat". But integration into the UNIX password mechanism is not really possible at this time. People can want it, but they aren't going to get it, because UNIX passwords are not stored as plaintext, as they are when you use the "saslpasswd" program, and sendmail only advertises SMTP AUTH KERBEROS and CRAM-MD5 mechanisms (per RFC-2222), and not "PLAIN" if saslpasswd(8) has been run, and the saslpasswd file exists. Note that RFC-2595 does not apply to SMTP, and it doesn't apply to what it does apply to, without TLS (SSL). > using protocols supported today by the majority of e-mail > clients. SMTP AUTH, requiring SSL as I outlined before, is the only > solution I have ever found. You need to submit your patches for this to the sendmail people. Without modification, sendmail does not enforce use of SSL for permitting advertisement of SMTP AUTH, and therefore addition of a pseudo-RFC-2595 "PLAIN" or "EXTERNAL X-UNIX" mechansim can not reasonably be added to FreeBSD so that it's operational by default. You might as well argue for rsh or telnet being reenabled by default. The STARTTLS SMTP command doesn't work, because it is issued after the EHLO, which solicits the capabilities list that exposes the SMTP AUTH. The only method that works, therefore, is to use an SSL connection -- SMTPS... port 465, instead of port 25). You can see the order of operation problem, I hope? The normal commercial practice (assuming you aren't also using SASL for IMAP4 server authentication, in which case, the password database is shared, but seperate from the UNIX password database) is to trap UNIX password authentication to the POP3 server, and, on successful authentication, create a corresponding SASL password file entry using the pop3 daemon's knowledge of the unencrypted shared secret (either from it's own database, if APOP is used, or the over the wire value, if USER/PASS is used). In other words, it's something you do with the plaintext in hand, no matter how you look at it. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jul 13 6:20:54 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F16C237B401; Sat, 13 Jul 2002 06:20:51 -0700 (PDT) Received: from nothing-going-on.demon.co.uk (pc-62-31-42-140-hy.blueyonder.co.uk [62.31.42.140]) by mx1.FreeBSD.org (Postfix) with ESMTP id 99F9343E5E; Sat, 13 Jul 2002 06:20:50 -0700 (PDT) (envelope-from nik@nothing-going-on.demon.co.uk) Received: (from nik@localhost) by nothing-going-on.demon.co.uk (8.11.3/8.11.3) id g6CKMWm00372; Fri, 12 Jul 2002 21:22:32 +0100 (BST) (envelope-from nik) Date: Fri, 12 Jul 2002 21:22:31 +0100 From: Nik Clayton To: Rahul Siddharthan Cc: Cyrille Lefevre , John Baldwin , arch@freebsd.org, Dan Nelson Subject: Re: Other BSD's (was Re: Cleaning old packages (was: Package system flaws?)) Message-ID: <20020712202231.GA333@canyon.nothing-going-on.org> References: <20020709161953.GA69779@lpt.ens.fr> <20020709171417.GA69932@lpt.ens.fr> <20020709231820.GA49510@gits.dyndns.org> <20020711005247.GE82744@gits.dyndns.org> <20020711073105.GB264@lpt.ens.fr> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="dDRMvlgZJXvWKvBx" Content-Disposition: inline In-Reply-To: <20020711073105.GB264@lpt.ens.fr> User-Agent: Mutt/1.4i Organization: FreeBSD Project Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --dDRMvlgZJXvWKvBx Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jul 11, 2002 at 09:31:05AM +0200, Rahul Siddharthan wrote: > 3. You can fetch all the source tarballs (including dependencies, but > excluding what's already installed) in one shot, before building > anything. =20 make fetch-recursive Yes, this stuff needs to be better documented. N --=20 FreeBSD: The Power to Serve http://www.freebsd.org/ (__) FreeBSD Documentation Project http://www.freebsd.org/docproj/ \\\'',) \/ \= ^ --- 15B8 3FFC DDB4 34B0 AA5F 94B7 93A8 0764 2C37 E375 --- .\._/= _) --dDRMvlgZJXvWKvBx Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.7 (FreeBSD) iD8DBQE9LzqHk6gHZCw343URAtDNAJ44oF3sYXRk1muNFFa47RCMFoOtRgCeJMRo TW/x0Qtd7+k22uZJKzsZ8Qs= =l/0P -----END PGP SIGNATURE----- --dDRMvlgZJXvWKvBx-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jul 13 6:21: 5 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5AA5337B43D; Sat, 13 Jul 2002 06:21:01 -0700 (PDT) Received: from nothing-going-on.demon.co.uk (pc-62-31-42-140-hy.blueyonder.co.uk [62.31.42.140]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2731743E3B; Sat, 13 Jul 2002 06:20:57 -0700 (PDT) (envelope-from nik@nothing-going-on.demon.co.uk) Received: (from nik@localhost) by nothing-going-on.demon.co.uk (8.11.3/8.11.3) id g67EpG609278; Sun, 7 Jul 2002 15:51:16 +0100 (BST) (envelope-from nik) Date: Sun, 7 Jul 2002 15:51:16 +0100 From: Nik Clayton To: Dag-Erling Smorgrav Cc: Wes Peters , Dan Moschuk , arch@freebsd.org Subject: Re: Package system flaws? Message-ID: <20020707145116.GC5610@canyon.nothing-going-on.org> References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="jy6Sn24JjFx/iggw" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4i Organization: FreeBSD Project Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --jy6Sn24JjFx/iggw Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Jul 07, 2002 at 12:09:05PM +0200, Dag-Erling Smorgrav wrote: > One thing we could improve a lot is the package file format. We > currently use gzipped tarballs, which have to be completely unpacked > before processing can begin. One improvement we can make is to use an > archive format such as zip that allows us to extract individual files > quickly, so we can extract the metadata and check dependencies > etc. without unpacking the entire package. =20 gzipped UFS filesystem images that can be mounted on a vn/md device. . . N --=20 FreeBSD: The Power to Serve http://www.freebsd.org/ (__) FreeBSD Documentation Project http://www.freebsd.org/docproj/ \\\'',) \/ \= ^ --- 15B8 3FFC DDB4 34B0 AA5F 94B7 93A8 0764 2C37 E375 --- .\._/= _) --jy6Sn24JjFx/iggw Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.7 (FreeBSD) iD8DBQE9KFVjk6gHZCw343URAqnRAJ9z9byoMumCrwhwsnpg5TG2Ma4gsQCfVKCW 5fu863HFmnEUKeKf2Xx/gLM= =Ft1z -----END PGP SIGNATURE----- --jy6Sn24JjFx/iggw-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jul 13 6:26:19 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6375737B400 for ; Sat, 13 Jul 2002 06:26:18 -0700 (PDT) Received: from ussenterprise.ufp.org (ussenterprise.ufp.org [208.185.30.210]) by mx1.FreeBSD.org (Postfix) with ESMTP id D071F43E3B for ; Sat, 13 Jul 2002 06:26:17 -0700 (PDT) (envelope-from bicknell@ussenterprise.ufp.org) Received: (from bicknell@localhost) by ussenterprise.ufp.org (8.11.1/8.11.1) id g6DDQGX59328; Sat, 13 Jul 2002 09:26:16 -0400 (EDT) (envelope-from bicknell) Date: Sat, 13 Jul 2002 09:26:16 -0400 From: Leo Bicknell To: Terry Lambert Cc: freebsd-arch@freebsd.org, louie@TransSys.COM, listsub@rambo.simx.org, leifn@neland.dk Subject: Re: Mail subsystem defaults, adding authentication. Message-ID: <20020713132616.GB58979@ussenterprise.ufp.org> References: <20020713034725.GB47677@ussenterprise.ufp.org> <3D2FAFB2.E2E9CF36@mindspring.com> <20020713045704.GA49379@ussenterprise.ufp.org> <3D300FD4.7479A8E5@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3D300FD4.7479A8E5@mindspring.com> Organization: United Federation of Planets Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In a message written on Sat, Jul 13, 2002 at 04:32:36AM -0700, Terry Lambert wrote: > This isn't really a FreeBSD issue, except for the inclusion of > the SASL library in the base system, which is not currently the > case. Correct. > The STARTTLS SMTP command doesn't work, because it is issued > after the EHLO, which solicits the capabilities list that exposes > the SMTP AUTH. The only method that works, therefore, is to use > an SSL connection -- SMTPS... port 465, instead of port 25). You > can see the order of operation problem, I hope? I was more thinking SMTPS; that is the SMTPS server would allow SMTP AUTH, and the SMTP (25) server would not. I understand you can't configure sendmail to do this today. That could be patched though, and an interim hack would to just run two copies of sendmail with slightly different configs. No? -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/ Read TMBG List - tmbg-list-request@tmbg.org, www.tmbg.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jul 13 6:57: 4 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CB3A737B400; Sat, 13 Jul 2002 06:57:02 -0700 (PDT) Received: from falcon.mail.pas.earthlink.net (falcon.mail.pas.earthlink.net [207.217.120.74]) by mx1.FreeBSD.org (Postfix) with ESMTP id 40E3043E9B; Sat, 13 Jul 2002 06:57:02 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0019.cvx22-bradley.dialup.earthlink.net ([209.179.198.19] helo=mindspring.com) by falcon.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17TNOJ-0005k4-00; Sat, 13 Jul 2002 06:56:56 -0700 Message-ID: <3D30316F.DCFBB407@mindspring.com> Date: Sat, 13 Jul 2002 06:55:59 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Nik Clayton Cc: Dag-Erling Smorgrav , Wes Peters , Dan Moschuk , arch@freebsd.org Subject: Re: Package system flaws? References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> <20020707145116.GC5610@canyon.nothing-going-on.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Nik Clayton wrote: > On Sun, Jul 07, 2002 at 12:09:05PM +0200, Dag-Erling Smorgrav wrote: > > One thing we could improve a lot is the package file format. We > > currently use gzipped tarballs, which have to be completely unpacked > > before processing can begin. One improvement we can make is to use an > > archive format such as zip that allows us to extract individual files > > quickly, so we can extract the metadata and check dependencies > > etc. without unpacking the entire package. > > gzipped UFS filesystem images that can be mounted on a vn/md device. . . The "md" devices have a nasty lockup which was recently run down because it originally looked like a tuning problem. See the -hackers list for details. It's basically a race with a race to root, and happens if two images are in the same directory, as they would be if you were installing package images from the /usr/ports/distfiles directory. Basically, it's possible to deadlock because the device is locked and the device the device is on is locked, and they aren't in the same deadlock avoidance space. The proper fix is probably to add a VFSOP to get an RLE encoded list of raw sectors for a file configured as an md device backing object, and then use that to translate sector requests into sector requests on the underlying device, rather than running the whole thing through the file system a second time. You would need to lock the file before you started doing this, so that it couldn't be extended accidently. That would avoid the deadlock. It would also be really useful for swapping on a file, FWIW. You could even make it work with sparse files, this way... -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jul 13 7:18:27 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D2B8C37B400 for ; Sat, 13 Jul 2002 07:18:24 -0700 (PDT) Received: from postfix2-2.free.fr (postfix2-2.free.fr [213.228.0.140]) by mx1.FreeBSD.org (Postfix) with ESMTP id F351243E42 for ; Sat, 13 Jul 2002 07:18:23 -0700 (PDT) (envelope-from rsidd@lpt.ens.fr) Received: from bluerondo.a.la.turk (nas-cbv-4-62-147-143-29.dial.proxad.net [62.147.143.29]) by postfix2-2.free.fr (Postfix) with ESMTP id AE38B5F705 for ; Sat, 13 Jul 2002 16:18:22 +0200 (CEST) Received: (qmail 442 invoked by uid 1001); 13 Jul 2002 14:18:19 -0000 Date: Sat, 13 Jul 2002 16:18:19 +0200 From: Rahul Siddharthan To: Nik Clayton Cc: Cyrille Lefevre , John Baldwin , arch@freebsd.org, Dan Nelson Subject: Re: Other BSD's (was Re: Cleaning old packages (was: Package system flaws?)) Message-ID: <20020713141819.GB268@lpt.ens.fr> References: <20020709161953.GA69779@lpt.ens.fr> <20020709171417.GA69932@lpt.ens.fr> <20020709231820.GA49510@gits.dyndns.org> <20020711005247.GE82744@gits.dyndns.org> <20020711073105.GB264@lpt.ens.fr> <20020712202231.GA333@canyon.nothing-going-on.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020712202231.GA333@canyon.nothing-going-on.org> User-Agent: Mutt/1.3.27i X-Operating-System: FreeBSD 4.6-PRERELEASE i386 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Nik Clayton said on Jul 12, 2002 at 21:22:31: > > 3. You can fetch all the source tarballs (including dependencies, but > > excluding what's already installed) in one shot, before building > > anything. > > make fetch-recursive Deja vu on -chat... The key phrase in what I wrote was "excluding what's already installed." fetch-recursive fetches everything which is a waste of time and bandwidth. But as DES pointed out, porteasy can do what I wanted (though it still prints the results in alphabetical order, I prefer Gentoo's output which is in order of what will be installed). - Rahul To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jul 13 10:57:27 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 839A637B400 for ; Sat, 13 Jul 2002 10:57:26 -0700 (PDT) Received: from zardoc.esmtp.org (adsl-63-195-85-27.dsl.snfc21.pacbell.net [63.195.85.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id F202E43E42 for ; Sat, 13 Jul 2002 10:57:25 -0700 (PDT) (envelope-from ca@zardoc.esmtp.org) Received: from zardoc.esmtp.org (localhost [127.0.0.1]) by zardoc.esmtp.org (8.12.5/8.12.4) with ESMTP id g6DHtSgf019798 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO); Sat, 13 Jul 2002 10:55:28 -0700 (PDT) Received: (from ca@localhost) by zardoc.esmtp.org (8.12.5/8.12.3/Submit) id g6DHtSkT018906; Sat, 13 Jul 2002 10:55:28 -0700 (PDT) Date: Sat, 13 Jul 2002 10:55:28 -0700 From: Claus Assmann To: Leo Bicknell Cc: freebsd-arch@FreeBSD.ORG Subject: Re: Mail subsystem defaults, adding authentication. Message-ID: <20020713105528.A24650@zardoc.esmtp.org> Reply-To: freebsd-arch@FreeBSD.ORG Mail-Followup-To: freebsd-arch@FreeBSD.ORG References: <20020713034725.GB47677@ussenterprise.ufp.org> <3D2FAFB2.E2E9CF36@mindspring.com> <20020713045704.GA49379@ussenterprise.ufp.org> <3D300FD4.7479A8E5@mindspring.com> <20020713132616.GB58979@ussenterprise.ufp.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii In-Reply-To: <20020713132616.GB58979@ussenterprise.ufp.org>; from bicknell@ufp.org on Sat, Jul 13, 2002 at 09:26:16AM -0400 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, Jul 13, 2002, Leo Bicknell wrote: > I was more thinking SMTPS; that is the SMTPS server would allow > SMTP AUTH, and the SMTP (25) server would not. I understand you > can't configure sendmail to do this today. That could be > patched though, and an interim hack would to just run two copies > of sendmail with slightly different configs. No? AuthOptions ... Example: O AuthOptions=p,y would disallow ANONYMOUS as AUTH mechanism and would allow PLAIN only if a security layer (e.g., provided by STARTTLS) is already active. .... PS: you may want to check out _FFR_SMTP_SSL To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jul 13 14:56:21 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3902637B400; Sat, 13 Jul 2002 14:56:19 -0700 (PDT) Received: from thuvia.demon.co.uk (thuvia.demon.co.uk [193.237.34.248]) by mx1.FreeBSD.org (Postfix) with ESMTP id 20C3C43E58; Sat, 13 Jul 2002 14:56:17 -0700 (PDT) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (dotar.thuvia.org [10.0.0.4]) by phaidor.thuvia.org (8.11.6/8.11.6) with ESMTP id g6DLtob41341; Sat, 13 Jul 2002 22:55:50 +0100 (BST) (envelope-from mark@thuvia.demon.co.uk) Received: from dotar.thuvia.org (localhost [IPv6:::1]) by dotar.thuvia.org (8.12.3/8.12.3) with ESMTP id g6DLtnKC049906; Sat, 13 Jul 2002 22:55:49 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.3/8.12.3/Submit) id g6DLtjfQ049903; Sat, 13 Jul 2002 22:55:45 +0100 (BST) Date: Sat, 13 Jul 2002 22:55:45 +0100 (BST) From: Mark Valentine Message-Id: <200207132155.g6DLtjfQ049903@dotar.thuvia.org> In-Reply-To: Terry Lambert's message of Jul 12, 6:15pm X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: Terry Lambert Subject: Re: Package system flaws? Cc: Wes Peters , Jordan K Hubbard , Dan Nelson , Archie Cobbs , Dan Moschuk , Dag-Erling Smorgrav , arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > From: Terry Lambert > Date: Fri 12 Jul, 2002 > Subject: Re: Package system flaws? > Mark Valentine wrote: > > > From: Terry Lambert > > > At this point, the best approach is probably a registry. > > > > It already exists for our purposes: /var/db/pkg. > > If you can tell me how to do a quick indexed lookup by key for > something like "path to configuration file", which wanted to > be in "/etc", was comipled to be in "/usr/local/etc", and which > the user wants to install in "/var/opt/etc", I'd be grateful. > 8-). A slight refinement on "grep @cwd /var/db/pkg/$1/+CONTENTS" is good enough for me. (The new metadata format will specifically cater for this lookup, and I've suggested there be an API call for it.) If performance is a serious issue, we can create a .db file. > > > While it's theoretically possible to edit this data in place, > > > assuming the path is not assembled from components at runtime, > > > in practice, you are limited to a path of equal or smaller size. > > > > My location strings would be padded out to a reasonable maximum size. > > With respect, I'm not worried about *your* location strings, > I'm worried about the location strings that end up in binaries > that result from "configure" resulting from "configure.in" from > "autoconf", etc.. > > If we were just talking about *your* location strings, then I > could be pretty sure that whatever API the OS presented, you > would use it. Most applications need modification to allow install-time relocation, end of story. Our job is to make this as easy as possible. > > > A lesser alternative is to have a "/var/opt" or some other > > > location which is guaranteed to be a symlink to the real path > > > location... and is never, itself, a real directory. > > > > Well, we could already create such a symlink, even on a per-package basis, > > as (for example) /var/db/pkg/foo-x.y/location... > > It's really ugly. And, as I pointed out, "lesser". 8-). I was simply pointing out what's possible now. Cheers, Mark. -- Mark Valentine, Thuvia Labs "Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses "We're kind of stupid that way." *munch* *munch* and endorses FreeBSD -- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jul 13 17:14:40 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2BDCD37B400 for ; Sat, 13 Jul 2002 17:14:39 -0700 (PDT) Received: from horsey.gshapiro.net (horsey.gshapiro.net [209.220.147.178]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4983543E4A for ; Sat, 13 Jul 2002 17:14:38 -0700 (PDT) (envelope-from gshapiro@gshapiro.net) Received: from monkeyboy.gshapiro.net (root@[IPv6:2001:218:1e1f:40:260:1dff:fef0:e51f]) by horsey.gshapiro.net (8.12.5.Beta0/8.12.5.Beta0) with ESMTP id g6DNgFdN085698 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=OK); Sat, 13 Jul 2002 16:42:17 -0700 (PDT) Received: from monkeyboy.gshapiro.net (gshapiro@localhost [127.0.0.1]) by monkeyboy.gshapiro.net (8.12.5/8.12.5) with ESMTP id g6DNgCuo009681 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO); Sat, 13 Jul 2002 16:42:13 -0700 (PDT) (envelope-from gshapiro@monkeyboy.gshapiro.net) Received: (from gshapiro@localhost) by monkeyboy.gshapiro.net (8.12.5/8.12.5/Submit) id g6DNgBhv009678; Sat, 13 Jul 2002 16:42:11 -0700 (PDT) (envelope-from gshapiro) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15664.47827.844708.151118@monkeyboy.gshapiro.net> Date: Sat, 13 Jul 2002 16:42:11 -0700 From: Gregory Neil Shapiro To: Terry Lambert Cc: freebsd-arch@FreeBSD.ORG Subject: Re: Mail subsystem defaults, adding authentication. In-Reply-To: <3D300FD4.7479A8E5@mindspring.com> References: <20020713034725.GB47677@ussenterprise.ufp.org> <3D2FAFB2.E2E9CF36@mindspring.com> <20020713045704.GA49379@ussenterprise.ufp.org> <3D300FD4.7479A8E5@mindspring.com> X-Mailer: VM 7.03 under 21.1 (patch 14) "Cuyahoga Valley" XEmacs Lucid Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG tlambert2> You need to submit your patches for this to the sendmail people. tlambert2> Without modification, sendmail does not enforce use of SSL for tlambert2> permitting advertisement of SMTP AUTH, and therefore addition of tlambert2> a pseudo-RFC-2595 "PLAIN" or "EXTERNAL X-UNIX" mechansim can not tlambert2> reasonably be added to FreeBSD so that it's operational by default. tlambert2> The STARTTLS SMTP command doesn't work, because it is issued tlambert2> after the EHLO, which solicits the capabilities list that exposes tlambert2> the SMTP AUTH. The only method that works, therefore, is to use tlambert2> an SSL connection -- SMTPS... port 465, instead of port 25). You tlambert2> can see the order of operation problem, I hope? You need to go back and read the RFC's/documentation. First, you can limit the AUTH mechanisms offered based on whether STARTTLS was used or not. Second, after a successful STARTTLS negotiation, a new EHLO is done and a new set of AUTH mechanisms is given. You can (and should) use STARTTLS with SMTP AUTH PLAIN/LOGIN and do not (and should not) use SMTP over SSL as it is non-standard. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jul 13 17:25:53 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 378C037B400; Sat, 13 Jul 2002 17:25:51 -0700 (PDT) Received: from flamingo.mail.pas.earthlink.net (flamingo.mail.pas.earthlink.net [207.217.120.232]) by mx1.FreeBSD.org (Postfix) with ESMTP id B709B43E65; Sat, 13 Jul 2002 17:25:50 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0002.cvx40-bradley.dialup.earthlink.net ([216.244.42.2] helo=mindspring.com) by flamingo.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17TXCq-0005jm-00; Sat, 13 Jul 2002 17:25:44 -0700 Message-ID: <3D30C4DA.22A255A8@mindspring.com> Date: Sat, 13 Jul 2002 17:24:59 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Gregory Neil Shapiro Cc: freebsd-arch@FreeBSD.ORG Subject: Re: Mail subsystem defaults, adding authentication. References: <20020713034725.GB47677@ussenterprise.ufp.org> <3D2FAFB2.E2E9CF36@mindspring.com> <20020713045704.GA49379@ussenterprise.ufp.org> <3D300FD4.7479A8E5@mindspring.com> <15664.47827.844708.151118@monkeyboy.gshapiro.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Gregory Neil Shapiro wrote: > You need to go back and read the RFC's/documentation. > > First, you can limit the AUTH mechanisms offered based on whether STARTTLS > was used or not. Second, after a successful STARTTLS negotiation, a new > EHLO is done and a new set of AUTH mechanisms is given. I see it now. My bad. I was looking at 2246 and 2595, when I should have been looking at 3207. As you say, the race is addresses by the state reset requirement. > You can (and should) use STARTTLS with SMTP AUTH PLAIN/LOGIN and do not > (and should not) use SMTP over SSL as it is non-standard. IMO, this is broken. Here's why: Implementation of SSL in the kernel is a foregone conclusion. It is a matter of "when", not "if", due to work like that of Sam Leffler's recent porting of the OpenBSD crypto hardware interface framework to FreeBSD. Basically, asking for conversion of a socket from one type to another is not something that will necessarily be supportable. The whole "STARTTLS" thing was introduced to kludge around the lack of IPSEC support in IPv4. Even if you argue that it's an issue for IPv4 because IPSEC bloats the hell out of IPv4 even when it's not being used, IPv6 requires implementation of IPSEC for it to be called an IPv6 implementation. This means that the days of transport crypto decisions like this one, and the code to implement it, living in user space are numbered, no matter what. I know the sendmail folks don't like SMTP over SSL, but... there is an IANA assigned number in /etc/services for it, which makes it about as standard as it can be; I don't think SSL RFC policy requires a per protocol SSL usage RFC for SSL to be used (that wouldn't make sense, in terms of promoting the adoption of SSL). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jul 13 18:46: 5 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 23E6337B400 for ; Sat, 13 Jul 2002 18:46:02 -0700 (PDT) Received: from ussenterprise.ufp.org (ussenterprise.ufp.org [208.185.30.210]) by mx1.FreeBSD.org (Postfix) with ESMTP id 76CB843E4A for ; Sat, 13 Jul 2002 18:46:01 -0700 (PDT) (envelope-from bicknell@ussenterprise.ufp.org) Received: (from bicknell@localhost) by ussenterprise.ufp.org (8.11.1/8.11.1) id g6E1k0171060 for freebsd-arch@FreeBSD.ORG; Sat, 13 Jul 2002 21:46:00 -0400 (EDT) (envelope-from bicknell) Date: Sat, 13 Jul 2002 21:46:00 -0400 From: Leo Bicknell To: freebsd-arch@FreeBSD.ORG Subject: Re: Mail subsystem defaults, adding authentication. Message-ID: <20020714014600.GA70961@ussenterprise.ufp.org> References: <20020713034725.GB47677@ussenterprise.ufp.org> <3D2FAFB2.E2E9CF36@mindspring.com> <20020713045704.GA49379@ussenterprise.ufp.org> <3D300FD4.7479A8E5@mindspring.com> <20020713132616.GB58979@ussenterprise.ufp.org> <20020713105528.A24650@zardoc.esmtp.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020713105528.A24650@zardoc.esmtp.org> Organization: United Federation of Planets Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In a message written on Sat, Jul 13, 2002 at 10:55:28AM -0700, Claus Assmann wrote: > AuthOptions > ... > Example: > > O AuthOptions=p,y > > would disallow ANONYMOUS as AUTH mechanism > and would allow PLAIN only if a security > layer (e.g., provided by STARTTLS) is > already active. .... Thanks. I found a document on the authoptions earlier, but it confused me more than it enlightened me. This, plus Greg's mail makes a lot more things clear. Tomorrow I'll write up a better summary with this new info. At the end of the day it looks like if we add cyrus-sasl, which is BSD licensed then the default behavior will be unchanged, but it will be possible through a combination of rc.conf options, running saslpasswd, and/or running ssl key generation tools to do auth on a non-encrypted session using challenge response (against sasl passwords), or do auth against the password file (or any PAM method) over an ssl session. Thus we could make it as simple as 'sendmail_auth="unix"' (or pam, or whatever) for an admin to allow end clients to starttls, auth, and securely send e-mail all with their existing credential. That is exactly what I want to promote. Hopefully people will agree, and we can get to the code details (which actually seem really simple). -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/ Read TMBG List - tmbg-list-request@tmbg.org, www.tmbg.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jul 13 21:22:23 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 36E8B37B400 for ; Sat, 13 Jul 2002 21:22:21 -0700 (PDT) Received: from w250.z064001178.sjc-ca.dsl.cnc.net (adsl-66.218.45.239.dslextreme.com [66.218.45.239]) by mx1.FreeBSD.org (Postfix) with SMTP id 29E5543E31 for ; Sat, 13 Jul 2002 21:22:19 -0700 (PDT) (envelope-from jos@catnook.com) Received: (qmail 6537 invoked by uid 1000); 14 Jul 2002 04:22:37 -0000 Date: Sat, 13 Jul 2002 21:22:15 -0700 From: Jos Backus To: freebsd-arch@FreeBSD.ORG Subject: Re: Package system flaws? Message-ID: <20020714042237.GD931@lizzy.catnook.com> Reply-To: jos@catnook.com Mail-Followup-To: freebsd-arch@FreeBSD.ORG References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> <20020712121427.GD3678@lummux.tchpc.tcd.ie> <20020712144854.GA756@laurel.tmseck.homedns.org> <20020713054141.A26277@misty.eyesbeyond.com> <20020713011750.GA755@laurel.tmseck.homedns.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020713011750.GA755@laurel.tmseck.homedns.org> User-Agent: Mutt/1.5.1i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, Jul 13, 2002 at 03:17:50AM +0200, Thomas Seck wrote: > A tool like portupgrade should not exist (and it should not have been > written in Ruby - yet another dependency to track). Indeed. We should try hard to keep powerful, easy to use, general-purpose, high-level scripting languages out of the base system. After all, we have C/ awk/sed/sh etc., and anything that can be done with those scripting languages can also be done with these traditional tools. -- Jos Backus _/ _/_/_/ Santa Clara, CA _/ _/ _/ _/ _/_/_/ _/ _/ _/ _/ jos@catnook.com _/_/ _/_/_/ require 'std/disclaimer' To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jul 13 21:26:27 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4AC5737B400 for ; Sat, 13 Jul 2002 21:26:25 -0700 (PDT) Received: from squall.waterspout.com (squall.waterspout.com [208.13.56.12]) by mx1.FreeBSD.org (Postfix) with ESMTP id C617743E31 for ; Sat, 13 Jul 2002 21:26:24 -0700 (PDT) (envelope-from will@csociety.org) Received: by squall.waterspout.com (Postfix, from userid 1050) id 24CB79B32; Sat, 13 Jul 2002 23:26:24 -0500 (EST) Date: Sat, 13 Jul 2002 23:26:24 -0500 From: Will Andrews To: freebsd-arch@FreeBSD.ORG Subject: Re: Package system flaws? Message-ID: <20020714042623.GB95460@squall.waterspout.com> Mail-Followup-To: freebsd-arch@FreeBSD.ORG References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> <20020712121427.GD3678@lummux.tchpc.tcd.ie> <20020712144854.GA756@laurel.tmseck.homedns.org> <20020713054141.A26277@misty.eyesbeyond.com> <20020713011750.GA755@laurel.tmseck.homedns.org> <20020714042237.GD931@lizzy.catnook.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020714042237.GD931@lizzy.catnook.com> User-Agent: Mutt/1.3.26i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, Jul 13, 2002 at 09:22:15PM -0700, Jos Backus wrote: > Indeed. We should try hard to keep powerful, easy to use, general-purpose, > high-level scripting languages out of the base system. After all, we have C/ > awk/sed/sh etc., and anything that can be done with those scripting languages > can also be done with these traditional tools. The people who do the work should do it the best way they can. Or, with a volunteer twist, the most enjoyable way they can. Regards, -- wca To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message