From owner-freebsd-database  Mon Mar 30 08:03:43 1998
Return-Path: <owner-freebsd-database@FreeBSD.ORG>
Received: (from majordom@localhost)
          by hub.freebsd.org (8.8.8/8.8.8) id IAA24310
          for freebsd-database-outgoing; Mon, 30 Mar 1998 08:03:43 -0800 (PST)
          (envelope-from owner-freebsd-database@FreeBSD.ORG)
Received: from fallout.campusview.indiana.edu (fallout.campusview.indiana.edu [149.159.1.1])
          by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id IAA24300
          for <freebsd-database@FreeBSD.ORG>; Mon, 30 Mar 1998 08:03:41 -0800 (PST)
          (envelope-from jfieber@indiana.edu)
Received: from localhost (jfieber@localhost)
	by fallout.campusview.indiana.edu (8.8.8/8.8.7) with SMTP id LAA07196;
	Mon, 30 Mar 1998 11:03:39 -0500 (EST)
Date: Mon, 30 Mar 1998 11:03:39 -0500 (EST)
From: John Fieber <jfieber@indiana.edu>
To: nik@iii.co.uk
cc: freebsd-database@FreeBSD.ORG
Subject: Re: Mailing list search interface
In-Reply-To: <19980330164024.47510@iii.co.uk>
Message-ID: <Pine.BSF.3.96.980330104904.485V-100000@fallout.campusview.indiana.edu>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: owner-freebsd-database@FreeBSD.ORG
Precedence: bulk

[excessive CC list removed]

On Mon, 30 Mar 1998 nik@iii.co.uk wrote:

> On Mon, Mar 30, 1998 at 09:48:45AM -0500, John Fieber wrote:
> > > The output of 'time -l' on the conversion process was:
> > >
> > >       626.11 real       438.83 user        93.13 sys
> > 
> > On what sort of hardware?
> 
> 200 Mhz PPro w/64MB of RAM and 256MB of swap.

So, in the same ballpark of hub.  Hub has more RAM, but I limit
the RAM consumption of waisindex to around 25-30MB because there
is a lot of other stuff going on on the machine that I don't want
to interfere with.

> > By quick back-of-an-envelope calculations, this is slower than
> > the current indexing scheme on hub by at least a factor of 10.
> 
> The time above was for creation of the HTML archives and for indexing,
> not just indexing alone.

Ah, but the thread index creation is inseparable from the
creation of the HTML archives, yet the HTML creation is a
complete waste of time and disk space.  It is far more efficient
to generate the HTML on the fly because only a tiny fraction of
the messages will ever be viewed.  Contrast with Wolfram's thread
scheme which just builds a message-id based index for threads. 

> Are those survey results available online somewhere?

No, I'll have to dig a bit and they are probably not in a very
useful form.  I'll have to fire up SPSS and generate some
reports...

-john


To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-database" in the body of the message