From owner-freebsd-hackers@FreeBSD.ORG Tue May 13 16:03:07 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DD3FE1065670; Tue, 13 May 2008 16:03:07 +0000 (UTC) (envelope-from lidl@rox.fddx.com) Received: from rox.fddx.com (rox.fddx.com [71.241.230.59]) by mx1.freebsd.org (Postfix) with ESMTP id 842898FC1E; Tue, 13 May 2008 16:03:07 +0000 (UTC) (envelope-from lidl@rox.fddx.com) Received: from rox.fddx.com (localhost.pix.net [127.0.0.1]) by rox.fddx.com (8.13.8+Sun/8.13.8) with ESMTP id m4DFfdZN029027; Tue, 13 May 2008 11:41:39 -0400 (EDT) Received: (from lidl@localhost) by rox.fddx.com (8.13.8+Sun/8.13.8/Submit) id m4DFfd4r029026; Tue, 13 May 2008 11:41:39 -0400 (EDT) Date: Tue, 13 May 2008 11:41:39 -0400 From: "Kurt J. Lidl" To: Jeremy Chadwick Message-ID: <20080513154137.GA28842@pix.net> Mail-Followup-To: Jeremy Chadwick , Anthony Pankov , freebsd-hackers@freebsd.org References: <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <15336578.20080512123806@mail.ru> <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> <1663320218.20080512223531@mail.ru> <20080512152430.3720683e@mbook.local> <2117635718.20080513154406@mail.ru> <20080513121452.GA70860@eos.sc1.parodius.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20080513121452.GA70860@eos.sc1.parodius.com> User-Agent: Mutt/1.5.16 (2007-06-09) X-Virus-Scanned: ClamAV version 0.91.2, clamav-milter version 0.91.2 on rox.fddx.com X-Virus-Status: Clean Cc: freebsd-hackers@freebsd.org, Anthony Pankov Subject: Re: BDB corrupt X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 May 2008 16:03:08 -0000 On Tue, May 13, 2008 at 05:14:52AM -0700, Jeremy Chadwick wrote: > On Tue, May 13, 2008 at 03:44:06PM +0400, Anthony Pankov wrote: > > If concurrency is the only problem then: > > 1. ?an data corruption be avoided? Or this is impossible? > > 2. How? > > Use Sleepycat/Oracle DB instead? The libc DB1.x, despite being > "mature", really should be deprecated in some manner. This catapults back into the arena of "stuff that isn't in the base system". Not to mention I'm not sure that the Oracle BDB license would allow bundling in the OS as a binary. I doubt it, but that's a different bikeshed to paint :-) > I'm sure there are others I've forgotten, but the only thing I can think > of in the base system which relies on DB1.x is sendmail (which IMHO > should really be removed from the base system and replaced with a small > standalone mailer -- but that's been discussed in a previous thread in > the past). Even "simple" ports like postgrey pull in db41, even though > they could technically "work fine" with DB1.x. Well, on a 7.0 machine, it looks like the following: /etc/mail/aliases.db /etc/{s,}pwd.db /usr/share/misc/{termcap,vgrindefs}.db And of course, nvi uses the db code's recno interface to do all its work. > > If all BDB readers would use O_SHLOCK and all writers O_EXLOCK is it > > guarantee for data integrity? > > The corruption I've seen in the past results in DB operations failing > for no particular reason ("what do you mean those are all the records? > No! I just inserted a bunch more!"). It turns out some of the data in > the actual .db file is corrupt -- even when locking is used everywhere. > It's as if the code has some weird bug where it'll write bogus data to > the file for some reason. There are known problems with certain keys corrupting the DB 1.8x series code. In fact, the "release" of the 1.86 was an attempt to solve this problem when the KerberosV people at MIT found a repeatable key insert sequence that would corrupt things. (Or at least that's what I remember, it was a long time ago, and I might have the details wrong.) -Kurt