From owner-freebsd-questions Fri Mar 13 14:13:58 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id OAA26971 for freebsd-questions-outgoing; Fri, 13 Mar 1998 14:13:58 -0800 (PST) (envelope-from owner-freebsd-questions@FreeBSD.ORG) Received: from mail.netcetera.dk (root@sleipner.netcetera.dk [194.192.207.2]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id OAA26903 for ; Fri, 13 Mar 1998 14:13:48 -0800 (PST) (envelope-from leifn@image.dk) Received: (from uucp@localhost) by mail.netcetera.dk (8.8.8/8.8.8) with UUCP id XAA30325 for freebsd-questions@freebsd.org; Fri, 13 Mar 1998 23:12:38 +0100 Received: by swimsuit.swimsuit.roskildebc.dk (0.99.970109) id AA04226; 13 Mar 98 22:15:41 +0100 From: leifn@image.dk (Leif Neland) Date: 13 Mar 98 19:48:17 +0100 Subject: Squid: Proxying for fun and profit Message-ID: <690_9803132215@swimsuit.swimsuit.roskildebc.dk> References: Organization: Fidonet: Swimsuit Safari. Go for it. To: freebsd-questions@FreeBSD.ORG Sender: owner-freebsd-questions@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG At 13 Mar 98 16:20:37 Donald Burr wrote regarding Squid: Proxying for fun and profit DB> Well, I just set up my first Web proxy server, using squid (in DB> ports). Wow. Now I have all of my various machines in here DB> going through the web proxy on my FreeBSD box instead of going DB> out and trying to fetch the page themselves. Certainly speeds DB> things up since a lot of the users access the same pages over DB> and over again (the FreeBSD site is quite popular :) ). DB> DB> Now I'd like to try and do something more fancy, and perhaps DB> someone out there can help me out. DB> DB> I would like to automatically keep the most current pages of DB> various web sites online. For example, a lot of people access DB> http://www.freebsd.org (FreeBSD) and http://www.linux.org DB> (Linux). I'd like my proxy server to automatically go through DB> these entire sites, and fetch all pages, and keep checking DB> periodically to see if any pages here get updated. (Sort of DB> like what a "web spider" type of indexing bot like AltaVista DB> does) Try doing a lynx -traversal www.freebsd.org Remember to remove the traverse.dat (can't remember the exact name, might be wrong) lynx produces, as it won't revisit pages already in that file. DB> The catch, though, is that I don't want this automatic fetching DB> to cross site boundaries. It'll work like you want it. Leif Neland leifn@image.dk --- |Fidonet: Leif Neland 2:234/49 |Internet: leifn@image.dk To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message