Date: Fri, 12 Jul 2002 18:15:06 -0700 From: Terry Lambert <tlambert2@mindspring.com> To: Mark Valentine <mark@thuvia.demon.co.uk> Cc: Wes Peters <wes@softweyr.com>, Jordan K Hubbard <jkh@queasyweasel.com>, Dan Nelson <dnelson@allantgroup.com>, Archie Cobbs <archie@dellroad.org>, Dan Moschuk <dan@freebsd.org>, Dag-Erling Smorgrav <des@ofug.org>, arch@freebsd.org Subject: Re: Package system flaws? Message-ID: <3D2F7F1A.D7C1DBE4@mindspring.com> References: <200207122324.g6CNOQ7L020672@dotar.thuvia.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Mark Valentine wrote:
> > From: Terry Lambert <tlambert2@mindspring.com>
> > At this point, the best approach is probably a registry.
>
> It already exists for our purposes: /var/db/pkg.
If you can tell me how to do a quick indexed lookup by key for
something like "path to configuration file", which wanted to
be in "/etc", was comipled to be in "/usr/local/etc", and which
the user wants to install in "/var/opt/etc", I'd be grateful.
8-).
> > While it's theoretically possible to edit this data in place,
> > assuming the path is not assembled from components at runtime,
> > in practice, you are limited to a path of equal or smaller size.
>
> My location strings would be padded out to a reasonable maximum size.
With respect, I'm not worried about *your* location strings,
I'm worried about the location strings that end up in binaries
that result from "configure" resulting from "configure.in" from
"autoconf", etc..
If we were just talking about *your* location strings, then I
could be pretty sure that whatever API the OS presented, you
would use it.
> > A lesser alternative is to have a "/var/opt" or some other
> > location which is guaranteed to be a symlink to the real path
> > location... and is never, itself, a real directory.
>
> Well, we could already create such a symlink, even on a per-package basis,
> as (for example) /var/db/pkg/foo-x.y/location...
It's really ugly. And, as I pointed out, "lesser". 8-).
> > In an ideal world, you would probably want to put each package
> > and all dependent files, including the shared libraries it
> > cares about, into its own directory. This was really Windows
> > original problem.
>
> I'd rather put some trust in our shared library versioning and package
> dependency schemes, and actually share the shared libraries where it
> makes sense...
>
> However, you're already free to use your own policy here.
Sharing of libraries is possible via factoring of code. The DLL
scheme used by Windows is particularly amenable to the approach
I had outlined, since all libraries are generally accessed via
compiled in stub libraries that use the OpenLibrary() to
explicitly gain access to the stubbed interfaces, in the case of
shared libraries. Windows also lacks the ability for major versions
to exist simultaneously. FreeBSD, as of the ELF conversion, lacks
the ability for minor versions to exist simultaneously. In theory,
this is OK. In practice, for shared libraries from third party
vendors (e.g. "libintl" from GNU getopt), minor version numbering
is a necessity, as more recent "minor" versions would *remove*
code used by programs linked with older versions. Basically, you
can't rely on third party vendors to do versioning right, you have
to either do it for them, or, if you don't have their source code
available so you can do that, you have to containerize their use
of code, so that coexistance is possible, even if it's normally
undesirable.
For dependent shared libraries, we are effectively talking about
the ability to provide a module shared by various programs, but
with an explicit dependency. To deal with the versioning, you
would need to be *able* to have different versions, and then
*choose to not use them*. Effectively, this means symlinks to
the real thing for use as a reference count. It also means that
the "real thing" goes into a known location that is different
from the known location owned by the vendor. This is effectively
the "module problem" mentioned previously.
This is compounded by the fact that modules optional at compile
time are, if present at compile time, mandatory at runtime. This
is one of the fundamental flaws in the "autoconf/automake" paradigm
that no one ever seems to recognize publically. The FreeBSD ports
system package build for release recognizes this by insisting on a
"pristine" (jailed) build environment per package, with the "mandatory
optional components" explicitly spelled out by the port maintainer in
the port metadata (e.g. you can build without OpenSSL, but OpenSSL is
required).
Going in that direction works... but then you end up with a port per
combinatoric feature, e.g.:
/usr/ports/www/apache-jserv
/usr/ports/www/apache13
/usr/ports/www/apache13+ipv6
/usr/ports/www/apache13-fp
/usr/ports/www/apache13-modssl
/usr/ports/www/apache13-ssl
(Note: in this particular example, the combinatorics are *not*
worked out -- and therefore, some combinations are simply not
supported).
In practice: yes, it would be nice to be able to dictate a universal
module API to software vendors, but in the real world, FreeBSD does
not wield monopolistic power in the marketplace. If anything, given
a choice, most vendors would make things run on Linux, and to hell
with other platforms, even if "they did things right".
> > A number of researchers have suggested that this problem could
> > be solved by causing packages themselves to be in directories
> > that were exported as their own mini FS's. This "live image"
> > could be mounted read-only off a remote server, or locally off
> > a CDROM, without introducing further issues.
>
> See QNX 6 for an implementation which actually looks like it might be
> functional! (These QNX folks are smart.) The package installs in its
> own private area, and registers with the package file system so that
> its component files magically appear in all the expected places in
> /usr/bin and so on.
I was thinking in particular of the ability to create FS "views"
on data sets; however, looking at the QNX facility, that's another
way of doing it successfully. I think the point is to learn that
it's possible to do, instead of believing it isn't. It seems that
most people have been arguing that it isn't possible: any example
to the contrary is welcome. 8-).
> > There are really three problems with this:
> >
> > 1) Windows is limited to 26(32) mount points: one per drive
> > letter;
>
> Not that I care a whit about whether our packaging scheme would work with
> Windoze, but I vaguely recall hearing news of them fixing that a few years
> back...
The point was why it was not widely deployed in the standard
example OS ("If it's so good, why doesn't Microsoft do it?").
8-). The fix they have is the IFSMgr fix I described previously,
where you are still limited on sources of exported FS's by the
number of drive letters. Don't adopt MS's limitations as your
own, unless it's on purpose.
> > I personally don't know if I'm ready to fight the "Not Invented Here"
> > war that trying to implement a registry entails.
>
> There are plenty of lightweight UNIX-like mechanisms to solve all the
> problems they tried to solve with that mess. And you don't have to
> solve them all using the same one...
If you don't solve them with a centralized mechanism, then you
have failed to create a Schelling point. At some time in the
future -- usually sooner, rather than later -- you will find
that by doing it piecemeal, you have created a problem situation.
Specifically, if you choose to solve the problem in package "B"
by writing FreeBSD specific code for handling the issue using a
particular approach, and then someone takes it upon themselves
to do the same for dependency "A", with a different approach,
then you are pretty much screwed. The issue has to be solved
"bottom up"; even if you could do this by picking the various
packages on which other packages depend, but which depend on no
others, you haven't solved the problem for the base system, and
you haven't solved the problem when someone adds a new dependency.
It's like the Issac Asimov story about the Mars City Museum robbery,
where Mars is a dry planet, and it just so happened that the place
they chose to put the date-line ended up with a city on it. The
thieves find the museum opening at 9AM *Monday*, while they are
hanging from ropes from a skylight with a bypassed security system,
on a day they think is *Sunday*. When you hit the interference
graph, you are going to find impedence mismatches. Moire patterns
happen for a reason. 8-).
-- Terry
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3D2F7F1A.D7C1DBE4>