From owner-freebsd-questions@FreeBSD.ORG Sat Sep 17 04:26:19 2005 Return-Path: X-Original-To: freebsd-questions@FreeBSD.ORG Delivered-To: freebsd-questions@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 84CF916A41F for ; Sat, 17 Sep 2005 04:26:19 +0000 (GMT) (envelope-from kline@tao.thought.org) Received: from tao.thought.org (dsl231-043-140.sea1.dsl.speakeasy.net [216.231.43.140]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0CD9543D45 for ; Sat, 17 Sep 2005 04:26:16 +0000 (GMT) (envelope-from kline@tao.thought.org) Received: from tao.thought.org (localhost [127.0.0.1]) by tao.thought.org (8.13.1/8.13.1) with ESMTP id j8H4QFb2070376 for ; Fri, 16 Sep 2005 21:26:15 -0700 (PDT) (envelope-from kline@tao.thought.org) Received: (from kline@localhost) by tao.thought.org (8.13.1/8.13.1/Submit) id j8H4QFUV070375 for freebsd-questions@FreeBSD.ORG; Fri, 16 Sep 2005 21:26:15 -0700 (PDT) (envelope-from kline) Date: Fri, 16 Sep 2005 21:26:15 -0700 From: Gary Kline To: FreeBSD Mailing List Message-ID: <20050917042615.GA70296@thought.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.1i X-Organization: Thought Unlimited. Public service Unix since 1986. X-Of_Interest: Observing 19 years of service to the Unix community Cc: Subject: is there a way to auto-gather web data? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Sep 2005 04:26:19 -0000 Hi people, No, I'm not trying to "steal" this info--it has been in the public domain for cnturies. But on "the.public.foobar.com" there is an "/cgi/author.html?id=hallaj", let's say. In order to gleen the sayings from "hallaj" I have to click several times before the paragraphs are displayed. wget fails on this one. I'd like to scarf up all their sayings automagically.... Anybody know how to automate my clicking? There are at least 150 authors, each with N blurbs. :-X ...Talk about carpal-tunnel syndrome. I've seen what spambots can do to my guestbooks, so this may be a __black art__. tia, y'all, gary -- Gary Kline kline@thought.org www.thought.org Public service Unix