From owner-freebsd-www Mon Apr 14 10:18:41 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id KAA29966 for www-outgoing; Mon, 14 Apr 1997 10:18:41 -0700 (PDT) Received: from pillar.elsevier.co.uk (root@pillar.elsevier.co.uk [193.131.222.35]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id KAA29960 for ; Mon, 14 Apr 1997 10:18:36 -0700 (PDT) Received: from snowdon.elsevier.co.uk (snowdon.elsevier.co.uk [193.131.197.164]) by pillar.elsevier.co.uk (8.8.5/8.8.5) with ESMTP id SAA00381 for ; Mon, 14 Apr 1997 18:15:43 +0100 (BST) Received: from cadair.elsevier.co.uk by snowdon.elsevier.co.uk with SMTP (PP); Mon, 14 Apr 1997 18:18:06 +0100 Received: from tees.elsevier.co.uk (tees.elsevier.co.uk [193.131.197.60]) by cadair.elsevier.co.uk (8.8.5/8.8.5) with ESMTP id SAA23574; Mon, 14 Apr 1997 18:17:53 +0100 (BST) Received: (from dpr@localhost) by tees.elsevier.co.uk (8.8.5/8.8.5) id SAA02024; Mon, 14 Apr 1997 18:17:56 +0100 (BST) To: John Fieber Cc: www@freebsd.org Reply-To: paul@originat.demon.co.uk Subject: Re: Comments on new search page.. References: From: Paul Richards Date: 14 Apr 1997 18:17:55 +0100 In-Reply-To: John Fieber's message of Mon, 14 Apr 1997 09:10:52 -0500 (EST) Message-ID: <573estplq4.fsf@tees.elsevier.co.uk> Lines: 97 X-Mailer: Gnus v5.4.37/Emacs 19.30 Sender: owner-freebsd-www@freebsd.org X-Loop: FreeBSD.org Precedence: bulk John Fieber writes: > Before raising the bar, we should survey the mirrors and figure > out how far the bar can be raised before too many drop out. For > example, with the mailing list archives approaching 700 > megabytes, I'm sure mirroring those would kill off almost all! This is certainly a consideration but if we work this out carefully we should be able to modularise what is local and what is remote. The first step though will have to be a self contained distribution that can build a web site. With a local config file and a build system it will be possible to build a site that has some local parts and some remote parts and local admins can tailor the config to suit the amount of resources they have. > like. To do this, the web source needs to be more self > contained. Currently, as Paul Richards recently reminded me, > building it depends on thing neither in FreeBSD or the web tree. > It also depends on having access to a cvs repository to get at > the handbook, faq and (currently) the index file from the ports > collection. I've finally got my life to the point where I'm hacking again (getting my Phd and meeting my future wife has put me out of the picture for quite some time but I've finally collected all my computers together into an office area at home and started playing again :-)) and this looks like my field of work at the moment (and it's what I'm signed up to do) so.... At work we have something like the following /tool/{bin,lib,src} The tools area is where the tools for the web site live. These are things like sgmlnorm, perl and so on. This is just like /usr/local but is tailored to the web site i.e. if you want to build a new web site you distribute the /tool area. We use it as a distributed tool server i.e. /tool is an NFS export to the build server and internal web server (there's an external web server as well outside the firewall which obviously has a copy rather than an NFS mount). /product/FreeBSD (project root) /build/ bin lib /content/ bin lib data htdocs ( these two are were the cgi-bin ( server points /data The idea is that the product is totally self contained and that it builds the site into a single area (content) which is also self-contained and can be exported to the web server which may be a different machine to the build server. ./data is where the raw data lives. The build process does two things, it processes the raw data in some way and installs the processed data either into content/htdocs or content/data depending on what the output is and it also builds the executables into ./content/{cgi-bin,bin.lib}. cgi-bin contains only those parts of the code that are called from URLs. Any libs etc that are included, or other executables that are run are put into content/{bin,lib}. This is for extra security. ./content/data is where non-html online data is stored, e.g. databases, or other data that must be further processed on the fly. It would take some effort to move to this but it would solve a lot of problems. If the build process has the right knobs then the amount of data required/processed can be site dependant and the "built" site will either have local or remote links to particular elements depending on the configuration e.g. some sites will have the whole lot (Freefall), some will have everything except the mail archive with a link to Freefall's instead etc. The master site can also distribute different "built" components as well since the whole thing is self-contained both at build time and delivery time. The raw data area would contain links or copies to sources for cvs, gnats, ports etc and different parts of the build procedure would do whatever's best for that data source to create content/data (maybe nothing more than a copy but it's best to do as much processing as possible at build time rather than at delivery time). As a first step we need a self-contained build environment so I can actually build a FreeBSD site without installing a lot of bits by hand. I suggest we create a www/tool area in cvs and put John's SP port into it as a starting point and see where we go from there. This isn't a complete document of how this would work but it's probably enough to start the discussion :-) -- Dr Paul Richards. [p.richards@elsevier.co.uk] Originative Solutions Ltd. [paul@originat.demon.co.uk] Phone: 0370 462071 (Mobile), +44 (0)1865 843155 (Elsevier)