Date: Mon, 14 Apr 1997 19:57:11 +0200 (MET DST) From: cracauer@wavehh.hanse.de (Martin Cracauer) To: paul@originat.demon.co.uk Cc: jfieber@indiana.edu, www@freebsd.org Subject: Re: Comments on new search page.. Message-ID: <9704141757.AA05465@wavehh.hanse.de> In-Reply-To: <573estplq4.fsf@tees.elsevier.co.uk> from "Paul Richards" at Apr 14, 97 06:17:55 pm
next in thread | previous in thread | raw e-mail | index | archive | help
[full quote, my comments at end]
> John Fieber <jfieber@indiana.edu> writes:
>
> > Before raising the bar, we should survey the mirrors and figure
> > out how far the bar can be raised before too many drop out. For
> > example, with the mailing list archives approaching 700
> > megabytes, I'm sure mirroring those would kill off almost all!
>
> This is certainly a consideration but if we work this out carefully we
> should be able to modularise what is local and what is remote. The
> first step though will have to be a self contained distribution that
> can build a web site. With a local config file and a build system it
> will be possible to build a site that has some local parts and some
> remote parts and local admins can tailor the config to suit the amount
> of resources they have.
>
> > like. To do this, the web source needs to be more self
> > contained. Currently, as Paul Richards recently reminded me,
> > building it depends on thing neither in FreeBSD or the web tree.
> > It also depends on having access to a cvs repository to get at
> > the handbook, faq and (currently) the index file from the ports
> > collection.
>
> I've finally got my life to the point where I'm hacking again (getting
> my Phd and meeting my future wife has put me out of the picture for
> quite some time but I've finally collected all my computers together
> into an office area at home and started playing again :-)) and this
> looks like my field of work at the moment (and it's what I'm signed up
> to do) so....
>
> At work we have something like the following
>
> /tool/{bin,lib,src}
>
> The tools area is where the tools for the web site live. These are
> things like sgmlnorm, perl and so on. This is just like /usr/local but
> is tailored to the web site i.e. if you want to build a new web site
> you distribute the /tool area. We use it as a distributed tool server
> i.e. /tool is an NFS export to the build server and internal web server
> (there's an external web server as well outside the firewall which
> obviously has a copy rather than an NFS mount).
>
> /product/FreeBSD (project root)
> /build/
> bin
> lib
> /content/
> bin
> lib
> data
> htdocs ( these two are were the
> cgi-bin ( server points
> /data
>
> The idea is that the product is totally self contained and that it
> builds the site into a single area (content) which is also
> self-contained and can be exported to the web server which may be a
> different machine to the build server. ./data is where the raw data
> lives. The build process does two things, it processes the raw data in
> some way and installs the processed data either into content/htdocs or
> content/data depending on what the output is and it also builds the
> executables into ./content/{cgi-bin,bin.lib}.
>
> cgi-bin contains only those parts of the code that are called from
> URLs. Any libs etc that are included, or other executables that are run
> are put into content/{bin,lib}. This is for extra security.
> ./content/data is where non-html online data is stored, e.g. databases,
> or other data that must be further processed on the fly.
>
> It would take some effort to move to this but it would solve a lot of
> problems. If the build process has the right knobs then the amount of
> data required/processed can be site dependant and the "built" site
> will either have local or remote links to particular elements
> depending on the configuration e.g. some sites will have the whole lot
> (Freefall), some will have everything except the mail archive with a
> link to Freefall's instead etc.
>
> The master site can also distribute different "built" components as
> well since the whole thing is self-contained both at build time and
> delivery time. The raw data area would contain links or copies to
> sources for cvs, gnats, ports etc and different parts of the build
> procedure would do whatever's best for that data source to create
> content/data (maybe nothing more than a copy but it's best to do as
> much processing as possible at build time rather than at delivery
> time).
>
> As a first step we need a self-contained build environment so I can
> actually build a FreeBSD site without installing a lot of bits by
> hand. I suggest we create a www/tool area in cvs and put John's SP
> port into it as a starting point and see where we go from there.
A few comments:
1) We should not copy tools that are in src/ to www/ as well. CVS
doesn't support a concept like symbolic links, so we can't have both
without a maintainace nightmare and we can't have just www/ , because
the normal FreeBSD src/ build should install those tols as before.
This doesn't mean we should have no programs in www/ at all. Programs
that are needed by the HTML build process, but that should not be
parts of a normally installed FreeBSD system goes into www/. Programs
for all FreeBSD installations go into src/. A third category are
ports, but I think we can require having the newest version of a port
installed normally on a WWW mirror (if it build on older releases,
that is :-)
2) I think your setup require one installation procedure more than
needed. /product/FreeBSD/build is not needed.
I like my idea of fully relative build more (obviously :-) and to have
just an install target that puts the WWW things (but not tools to
build these) where the httpd expects it.
The only problem I see is with libraries needed by tools. If we have a
C program that links to a non-standard library, we need to set up an
additional Makefile target so that it is linked to ../lib/whatever.a
instead of /usr/lib/whatever.a. Maybe it is acceptable to do this by
the standard FreeBSD build process as well.
Same applies for perl modules, but those need to be current in the
base system anyway when CGI scripts are to be supported. I'm not a
perl expert, can one change the place perl looks for modules in by
environment variables?
Martin
--
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Martin Cracauer <cracauer@wavehh.hanse.de>
http://cracauer.cons.org
Fax +49 40 522 85 36
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9704141757.AA05465>