Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 23 Jul 2010 09:46:15 +0000
From:      "b. f." <bf1783@googlemail.com>
To:        Fbsd8 <fbsd8@a1poweruser.com>
Cc:        freebsd-questions@freebsd.org
Subject:   Re: ports INDEX file
Message-ID:  <AANLkTi=q%2B_oq7uGFJTU4%2BLUR84bPg6D8-Q9JoCW1eDZm@mail.gmail.com>
In-Reply-To: <4C495958.5000106@a1poweruser.com>
References:  <AANLkTinC%2B4-QhC8MBRRoe4qMpEzubBTcJWF4M3pya866@mail.gmail.com> <4C495958.5000106@a1poweruser.com>

next in thread | previous in thread | raw e-mail | index | archive | help
On 7/23/10, Fbsd8 <fbsd8@a1poweruser.com> wrote:
> b. f. wrote:
>>> Benjamin Lee wrote:
>>>> On 07/22/2010 06:20 PM, Fbsd8 wrote:
...
>>
> Well first thanks for the info you provided though it was all negative.

I think that you were misinterpreting what I wrote if you think that
it was all negative.

> I will explain what my goal is.
>
> First though, I have verified that the /usr/ports/INDEX-8 file can be
> gotten without the using cvs or cvsup.
>
> fetch -m "http://www.freebsd.org/ports/INDEX-8.bz2" does if fact work
> and the data on the file is as of 3 hours ago. So that indicates its
> being kept current. The -m means that if the date of the remote file is
> NOT newer then the local one, the download is bypassed>

Yes, as long as the server supports this, and doesn't unnecessarily
change the mtime of the file.  Minus some variable expansions, this is
basically what the fetchindex target does. If you don't care about
hard-coding version numbers, etc., then you might as well not invoke
make at all, and just do what you're doing, because it's faster.
...
> compile php5 from the port. But to short cut the compile process, I
> pre-install all of php5's dependents as packages. And of course I had to
> figure out who they all were by hand the first time and built a script
> that automates the whole procedure. I use cvsup at NEW RELEASE time to
> populate the empty ports tree with ports-base. Then I use cvsup to
> checkout the php5 make files and them "make install" and everything
> comes together just fine.

You may be interested in using ports-mgmt/portmaster ( a shell script
with minimal dependencies), which can do something similar to what you
are trying to do with the --index-only, -P/-PP, and --packages-build
flags.

> Now the Freebsd method of the 22,000 individual ports each with 3 to 5
> files is a method which has out lived its usefulness. TAKE NOTE: NO

There is no doubt that the increasing size of the tree, and the fact
that some parts of the build infrastructure don't scale well, have
created some challenges.  But I hardly think that it has outlived its
usefulness.

> FLAME WAR INTENDED. I just think a option should exist for us who don't
> follow the bleeding edge. Sure to some people that big ports tree is no

Well, the bleeding edge versus snapshot issue is a bit different from
the debate about the size and modularity of the ports tree.

> big deal, but I bet they don't do backups. That ports tree directory is
> a large resource hog if you lift the blinders and look at the big picture.

I guess it depends upon the constraints that you are operating under.
But fetching a new index is going to take about as much network
traffic as an update of the ports tree with csup.

> So since I have a method all ready working as I explained above, I am
> collecting information on the elements needed to write a shell script
> port application based on the method already described. Figure I will
> use cvsup to populate the port-base and checkout just the "parent" port
> make files. Read the INDEX file to automate finding the parent port
> dependents and reading the /var/db/pkg to skip an dependents all ready
> installed and then launch pkg_add to install the dependents and on any
> package failures cycle back and use cvsup to also checkout its make
> files, before issuing the "make install" on the parent.

Just bear in mind that the default INDEX contains the dependencies for
ports built with default options.  Changing the options may result in
different dependencies.  Consider using portmaster.

> Along this same line of thought,
> Another area I have problems with is why don't the port make system go
> and checkout any dependent ports missing make files instead of halting
> like it does now.

The ports system wasn't designed to meet your objectives.  Delegating
authority to perform bursts of unsupervised network activity at
unpredictable intervals would probably be considered a problem by many
users.  And some tasks require the entirety of the tree to be present.

>
> When installing a package it will auto install all of it dependents.
>

There is interest in work with "fat packages" to do something like you describe:

"Complete (a.k.a. Fat) packages

Suggested Summer of Code 2010 project idea

Technical contact: Brooks Davis

When bootstrapping systems it would be useful to be able to create a
single package file that contains one or more packages and all the
required dependent packages. This is conceptually similar to, but
different from PC-BSD's PBI package format. PBI's contain a private
copy of all dependencies, fat packages would contain each individual
package and once installed it would be as though each package was
individually installed in the usual manner.

This project would consist of additions to the pkg_tools to support
creation and installation of a new package file format and to ports to
build these packages.

Requirements:

Strong knowledge of C code.
A basic understanding of the inner workings of the ports tree."



Also, there is some ongoing work in a slightly different direction:

http://wiki.freebsd.org/IvanVoras/pkg_patch

PC-BSD has such a system, but it is very inefficient in it's use of
disk space, etc., so I don't think it would appeal to you.


b.



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AANLkTi=q%2B_oq7uGFJTU4%2BLUR84bPg6D8-Q9JoCW1eDZm>