Date: Thu, 22 Feb 2001 22:05:50 +1100 (EST) From: Enno Davids <enno.davids@metva.com.au> To: leif@neland.dk Cc: freebsd-isp@FreeBSD.ORG Subject: Re: Mirrorred webservers: Updating, logging. Message-ID: <200102221105.WAA16826@metva.com.au> In-Reply-To: <00e201c09c62$fbd45aa0$0e00a8c0@neland.dk> from Leif Neland at "Feb 22, 1 01:04:10 am"
next in thread | previous in thread | raw e-mail | index | archive | help
| There exist different solutions, however, I have not seen any mentioning on how to update the sites; the customers shouldn't have to update two sites; it should work transparently. We use a ataging server. (Or it could be a virtual host). The customer installs their content to the staging host, tests it in place and when they're happy they go to some PHP pages which kick off rsync/ssh to all the relevant hosts (currently 2, about to become 6). | Another thing is logging. How do I produce a statistic with eg webalyzer when two or more servers are producing their own logfile? We use the CustomLog directive to produce Common Log Format with an extra field at the front which is YYYMMDDHHMMSS. This then allows us to merge/sort the logs of a bunch of servers together to make one unifed log which we pass through cut to discard the first field leaving CLF for webalizer etc. | For updating, I think I should assign one server in the cluster the role as master, and restrict the users to only be able to update that server, be it by frontpage, ftp or whatever. That would work. Being able to go the other way provides cheap recovery too. | Would running rdist or something like that on several GB's of webpages be possible? rsync is more bandwidth efficient as it only copies changed files and then if set that way, only changed portions of changed files (i.e. if a few blocks in the middle are the noly difference, that can be all it sends). That then means you seldom send Gb of data as your customers seldom replace that much of the site in one go. | Having the customer pressing a button on a cgi to start replicating would be forgotten too often, I think. In our environment we have the QA step on the staging host. Content doesn't get into production without approval from the content owner and that approval comes in the form of using the PHP content push page(s). | Regarding logging, I think we should have a separate statistic-server. Each server would generate its own logfile, and once every night transfer the logfile to the stat-server. The statserver would then merge the logfiles from all the mirrors, and feed the combined file to webalizer. And we use the staging host to do this too in fact. Few content owners are logged in at midnight! :) | Webalizer doesn't like the time in the logfile to go backwards, so the logfiles need to be combined in time-order; they can't be processed sequentially. Later versions cope by simply attributing all traffic to the most recent time they've seen, causing spikes in the graphs. THings like click trail analyses in Webtrends react poorly to out of order data too though. You need to sort it. | Comments, please... | | Leif Enno. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200102221105.WAA16826>