Date: Tue, 7 Aug 2007 22:43:14 +0200 From: Alexander Leidinger <Alexander@Leidinger.net> To: Kris Kennaway <kris@obsecurity.org> Cc: Henrik Brix Andersen <henrik@brixandersen.dk>, Michael Nottebrock <lofi@freebsd.org>, cvs-all@freebsd.org, ports-committers@freebsd.org, Pav Lucistnik <pav@freebsd.org>, cvs-ports@freebsd.org Subject: Re: cvs commit: ports/Mk bsd.port.mk Message-ID: <20070807224314.3a4fb8c3@deskjail> In-Reply-To: <20070807175856.GA56120@rot26.obsecurity.org> References: <20070802181715.46yikycm8gc8g8kk@webmail.leidinger.net> <20070803125410.GB1062@tirith.brixandersen.dk> <200708032144.57558.lofi@freebsd.org> <20070803204215.GA68620@rot26.obsecurity.org> <20070806074318.q9mw6ulngg00gwsw@webmail.leidinger.net> <20070806065634.GA31676@rot26.obsecurity.org> <20070806113855.0fcq213io0www04k@webmail.leidinger.net> <46B7072E.8070307@freebsd.org> <20070806173808.GA39444@rot26.obsecurity.org> <20070807153744.rxsoo6p0ogoowsw4@webmail.leidinger.net> <20070807175856.GA56120@rot26.obsecurity.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Quoting Kris Kennaway <kris@obsecurity.org> (Tue, 7 Aug 2007 13:58:56 -0400): > On Tue, Aug 07, 2007 at 03:37:44PM +0200, Alexander Leidinger wrote: > > Quoting Kris Kennaway <kris@obsecurity.org> (from Mon, 6 Aug 2007 > > 13:38:08 -0400): > > >dependencies a better way is to use LIB_ or RUN_DEPENDS and track the > > >direct vs inherited dependencies differently in the package database. > > > > Why do you want to differentiate between implicit and explicit > > dependencies? We don't need to list the implicit ones, they are > > resolved implicitly already even without listing them in the package. > > Having to compute them recursively is inefficient. It's more Not that inefficient anymore. The new "make actual-package-depends" is really fast compared to what we had before (it just greps the dependencies out of the installed +CONTENTS files). > efficient to keep the full transitive list for each port and update > that information at the time when one of the ports changes. For which scenarios do we need the full list? > > Let's step back for a moment... > > > > ATM we record the dependencies and the dependencies of the > > dependencies and so on in a package. I call this implicit dependency > > recording. > > > > With explicit dependency recording I associate to only record the > > dependencies in the package, which are listed in LIB_ or RUN_DEPENDS > > of the corresponding port for the package. > > > > What I propose and what is implemented with the feature switch which > > triggered this discussion is: to do the explicit dependency recording > > instead of the implicit dependency recording. The ports collection > > resolves the dependencies recursively so we don't need to record the > > implicit dependencies. They are superflous in +CONTENTS (similar for > > the corresponding entries in +REQUIRED_BY). > > > > This changes > > - the amount (less) of packages listed in INDEX for a given port. > > - the amount (less) of dependencies listed in +CONTENTS for a given > > port/package. > > - the amount (less) of ports listed in +REQUIRED_BY. > > This is part of the problem: you're trading space for time when we're > doing a lot of work to *reduce* the time complexity of ports. At the I know that we try to reduce the time complexity. I'm one of the people which helped to reduce it. > least we'd need to (separately) maintain both the explicit and the > full set of implicit dependencies for each port for efficiency > reasons. This is an implementation detail though. An implementation detail I want to understand... (therefore the question above) > > When we _would_ switch _now_ it does _not_ change > > - the dependency tree for a port. > > - the amount of installed ports when installing x11/gnome2 or x11/kde > > or whatever (as they will still be resolved and installed > > implicitly). > > > > It allows to have those nice development/update/build properties I > > talked about in a previous post. > > Well sort of. Sometimes you can get away with only updating the ports > that explicitly depend on a changed port, but often you have to > transitively update everything that depends on it. There is no way to > reliably tell which situation you're in, so the safe option remains to > always recurse. Yes, and it is the responsibility of the tool which is used to update to compute the list of ports to update. For the downward recursive list you do what the current actual-package-depends target does to get the complete list. And even with the gnome2 port it is fast. For the upward recursive list you just need to take the initial +REQUIRED_BY and recursively add the +REQUIRED_BY of the contained ports. It's not as fast as just having to take one +REQUIRED_BY, but it should be still very fast, as you need to calculate this only once in the beginning of the update and as it only includes simple operations (it is similar to what the actual-package-depends target does). I agree that this (supposed to be) rare case where you need to do a full rebuild of the dependencies would need the full list. What other scenarios are there which need the full list? > > To get those nice properties, we have > > to extend the LIB_ and RUN_DEPENDS of our ports with all those > > dependencies, which the binaries of the port in question reference but > > have not listed already in the Makefile. Not all ports in the ports > > collection fail to have all dependencies, but those which fail them > > are most of the time not easy (gnome/kde/... are a huge number of > > ports and are the prominent ports which would need such a treatment). > > > > When we switch to explicit dependency recording without adding the > > explicit LIB_ and RUN_DEPENDS > > - we gain the benefit of using less space (on disk/CD, via network > > transfer). > > - we will not break existing ports. > > - we don't gain the build/update/development benefits. > > > > > > Can someone please point me to a strong technical reason not to switch > > to explicit dependency recording? If not, are there strong technical > > reasons to not record all the required dependencies in the ports which > > are missing them after switching to explicit dependency recording > > (note: please see my other mail where I inlined scripts to > > automatically find suitable LIB_DEPENDS lines for the common case)? > > I think I missed this. It might help with another (unrelated) case, > where port builds autodetect optional components that happen to be > installed, and then link to them without the port registering a > dependency. Yes, the objdump part can be a base for such a check. > Probably these scripts should be added to Tools/. Please have a look at them first. They are a quick shot and maybe need a little bit of refinement (which I'm willing to do) before they should enter the tree (I'm specially thinking about the second script...). Bye, Alexander. -- The "think positive" leader tends to listen to his subordinate's premonitions only during the postmortems. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20070807224314.3a4fb8c3>