From owner-freebsd-isp  Mon Nov 11 06:25:51 1996
Return-Path: owner-isp
Received: (from root@localhost)
          by freefall.freebsd.org (8.7.5/8.7.3) id GAA16307
          for isp-outgoing; Mon, 11 Nov 1996 06:25:51 -0800 (PST)
Received: from brasil.moneng.mei.com (brasil.moneng.mei.com [151.186.109.160])
          by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id GAA16285;
          Mon, 11 Nov 1996 06:25:45 -0800 (PST)
Received: (from jgreco@localhost) by brasil.moneng.mei.com (8.7.Beta.1/8.7.Beta.1) id IAA19231; Mon, 11 Nov 1996 08:25:10 -0600
From: Joe Greco <jgreco@brasil.moneng.mei.com>
Message-Id: <199611111425.IAA19231@brasil.moneng.mei.com>
Subject: Re: Best mount options, tunefs for newsserver
To: hgoldste@bbs.mpcs.com
Date: Mon, 11 Nov 1996 08:25:09 -0600 (CST)
Cc: freebsd-isp@freebsd.org, freebsd-fs@freebsd.org, dg@root.com,
        michaelv@MindBender.serv.net
In-Reply-To: <199611111259.HAA31462@bbs.mpcs.com> from "Howard Goldstein" at Nov 11, 96 07:59:20 am
X-Mailer: ELM [version 2.4 PL24]
Content-Type: text
Sender: owner-isp@freebsd.org
X-Loop: FreeBSD.org
Precedence: bulk

> Would be neat to have a new class of filesystem, call it "expfs"
> (expendable filesystem) for low-value high-volume stuff like news
> articles.  Could have default async writes, noatime, preferred time
> optimization, anything else that would lend itself to this sort of thing.

Oh, boatloads :-)

One big win in general would be an extension to FFS to allow some sort of
sorted or hashed directory, which would be useful for directories with lots
of files (not just news spool, think about large mail spools, etc).

News in particular could stand to have its very own type of filesystem,
since it has lots of things that can be optimized for.

Consider changing the news directory behavior on a news spool.  You have two
types of data, subdirectories and articles.  In general isdigit(element[0])
is a good first guess at which type of directory entry it is, so now
consider the following:

A portion of the "directory" is reserved for subdirectories.  Since it is
not often updated with new entries, it is considered to be acceptable to
maintain it as a sorted list, or some other method (hashing perhaps) to
allow rapid binary tree style lookups.

A portion of the "directory" is reserved for news articles.  Usenet news
articles have fascinating properties: they are written in numerically
ascending order, and tend to be erased in the same fashion.  Adding a new
file need be only as complex as knowing the offset of the last entry
written and checking against it to make sure that the new entry is higher
(if not, one would need to shuffle the directory to "make it right").
Removal simply zeroes the entry; empty directory blocks can be freed.

As a further twist, the "article" entries are stored in native integer
format as opposed to ASCII "string" format, allowing comparisons to happen
even more rapidly.

Now we have mechanisms in place for extremely fast lookup operations and
write operations.

If only I had the hours in the day to learn how to write my own file
systems...

FFS is excellent as a general purpose file system, but it lacks the ability
to take advantage of these types of things.

... JG