From owner-freebsd-arch Sun Jul 7 12:19:55 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5170537B400; Sun, 7 Jul 2002 12:19:53 -0700 (PDT) Received: from softweyr.com (softweyr.com [65.88.244.127]) by mx1.FreeBSD.org (Postfix) with ESMTP id C2AEF43E09; Sun, 7 Jul 2002 12:19:52 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from homer.softweyr.com ([204.68.178.39] helo=softweyr.com) by softweyr.com with esmtp (Exim 3.35 #1) id 17RHZF-000DDU-00; Sun, 07 Jul 2002 13:19:33 -0600 Message-ID: <3D289529.95C1EC8A@softweyr.com> Date: Sun, 07 Jul 2002 12:23:21 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Dag-Erling Smorgrav Cc: Dan Moschuk , arch@freebsd.org Subject: Re: Package system flaws? References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Dag-Erling Smorgrav wrote: > > Wes Peters writes: > > Dan Moschuk wrote: > > > Where can we improve? > > In all of the above areas, plus all the ones we haven't addressed yet. > > *grin* > > One thing we could improve a lot is the package file format. We > currently use gzipped tarballs, which have to be completely unpacked > before processing can begin. One improvement we can make is to use an > archive format such as zip that allows us to extract individual files > quickly, so we can extract the metadata and check dependencies > etc. without unpacking the entire package. I'd rather go one step further and separate the metadata from the files completely. I'm thinking an XML-ish format for the metadata, with blobs of files stored in whatever format makes the most sense inter- spersed. Zip format makes a lot of sense and is available in library format, but I could easily tolerate tar or cpio, as well as a variety of compression formats. The metadata should not be compressed for speed of package processing. This would also allow us to separate the files in the package into distinct pieces, and apply metadata to the collections of files. Picture, for instance, a package that installs an X application. We have executable files that need to go into $PKG_BIN, perhaps a library or two that needs to in $PKG_LIB, an application resource file that needs to go into $X_APPRESDIR and some fonts that need to be installed in $X_FONTDIR. We provide defaults for each of these, let the user override them as necessary. > This would save both time > and space (the current system can fail if the temp directory doesn't > have room for the unpacked package, even if the installation directory > has room to spare). A further improvement is to use a custom archive > format that always stores the metadata at the beginning of the archive > so we can install packages from the net without having to download and > store them locally first (zip isn't suitable for this as it stores the > content directory at the end of the archive, and the files within the > archive can appear in any order). Yes, an excellent idea. With a format as I've proposed, it would be possible to specify the file collections as URLs rather than encoded archives. You could download the entire package collection in about the same amount of space as the current ports collection. Of course this means pkg_add would have to be linked with libfetch, too. ;^) -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message