From owner-freebsd-hackers Tue Mar 11 14:00:52 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id OAA13133 for hackers-outgoing; Tue, 11 Mar 1997 14:00:52 -0800 (PST) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.50]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id OAA13128 for ; Tue, 11 Mar 1997 14:00:49 -0800 (PST) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id OAA26149; Tue, 11 Mar 1997 14:49:28 -0700 From: Terry Lambert Message-Id: <199703112149.OAA26149@phaeton.artisoft.com> Subject: Re: freebsd as a news server? To: jgreco@solaria.sol.net (Joe Greco) Date: Tue, 11 Mar 1997 14:49:28 -0700 (MST) Cc: terry@lambert.org, scrappy@hub.org, marc@bowtie.nl, neal@pernet.net, hackers@freebsd.org In-Reply-To: <199703111837.MAA29432@solaria.sol.net> from "Joe Greco" at Mar 11, 97 12:37:36 pm X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-hackers@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > Any time you have a list that isn't stored as directory data, you have > > an index. > > Terry, what ARE you babbling about? > > A list that is stored as directory data certainly IS an index, and INN > really does not use anything much more complicated than the FS structure, > flat files, and a big hash table _anywhere_ in the code. If you wish to > insist on calling _those_ indexes and databases, fine, but your original > babble still makes no sense even in that context. I think you are mixing threads here: Index is Index is Index is Index is in file in file in directory in directory mount -async !mount -async mount -async !mount -async Index safe from crash corruption? NO NO NO YES I am making an exception for directory data in this discussion so I can talk about *only* data that may imply state that *is* subject to corruption on crash, regardless of sync vs. async mount. For two stage commit using sync/fsync/O_WRITESYNC, I can say: Index safe from crash corruption? NO YES NO YES In no -async case is an index safe. > What little chance there is for desynchronization is generally worked > around in various manners. A great example is the news overview > "database"... if it is missing, broken, or doesn't contain the > information, the data is generated on the fly. Unless the information was locally generated, in which case it is not possible to recover it. If I post to a host running mounted -async, and that host does not guaranteed to put the article on disk before it responds to me, and crashes after the response before the article has been committed to stable storage, then my article is lost. This is different thatn if it is just a read/mirroring server, since all articales which did not locally originate are recoverable, per your example above. The special case of locally generated articles is not. > In the commonly distributed version of INN, the current spool IS the final > authority as far as which articles exist and which don't. If you choose > to crash the system, clri a bunch of articles and then fsck the file > system, INN will take the spool's current state as being authoritative. > > If you wipe out the overview caches, it will generate that data from > the actual article files. Neither case is a problem. The second case is a "delay from crash to service availability" problem. Whether it is a real problem or not depends on the type of availability guarantees you make to your customers, since the recovery process does not happen in zero time. > That's why I don't really CARE if I lose bunches of files when I mount > my disks -o async. It is a system that is relatively impervious to > various sorts of {Terry-imagined,real} damage, Because the data is recoverable from other sources, and because your service availability requirements are scoped that downtime following a crash permits recovery to occur. Yes. Which all goes to say that, as I said before, it's subjective based on your application, how you implement the application, and service availability guarantees which are in place. Your setup works for you, but your parameters are sufficiently narrowly scoped as to make it a non-problem. For you. Regards, Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers.