Date: 13 Mar 98 19:48:17 +0100 From: leifn@image.dk (Leif Neland) To: freebsd-questions@FreeBSD.ORG Subject: Squid: Proxying for fun and profit Message-ID: <690_9803132215@swimsuit.swimsuit.roskildebc.dk> References: <XFMail.980313072037.dburr@POBoxes.com>
next in thread | previous in thread | raw e-mail | index | archive | help
At 13 Mar 98 16:20:37 Donald Burr wrote regarding Squid: Proxying for fun and profit DB> Well, I just set up my first Web proxy server, using squid (in DB> ports). Wow. Now I have all of my various machines in here DB> going through the web proxy on my FreeBSD box instead of going DB> out and trying to fetch the page themselves. Certainly speeds DB> things up since a lot of the users access the same pages over DB> and over again (the FreeBSD site is quite popular :) ). DB> DB> Now I'd like to try and do something more fancy, and perhaps DB> someone out there can help me out. DB> DB> I would like to automatically keep the most current pages of DB> various web sites online. For example, a lot of people access DB> http://www.freebsd.org (FreeBSD) and http://www.linux.org DB> (Linux). I'd like my proxy server to automatically go through DB> these entire sites, and fetch all pages, and keep checking DB> periodically to see if any pages here get updated. (Sort of DB> like what a "web spider" type of indexing bot like AltaVista DB> does) Try doing a lynx -traversal www.freebsd.org Remember to remove the traverse.dat (can't remember the exact name, might be wrong) lynx produces, as it won't revisit pages already in that file. DB> The catch, though, is that I don't want this automatic fetching DB> to cross site boundaries. It'll work like you want it. Leif Neland leifn@image.dk --- |Fidonet: Leif Neland 2:234/49 |Internet: leifn@image.dk To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?690_9803132215>