From owner-freebsd-hackers@FreeBSD.ORG Tue May 13 16:03:07 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4532B106566C for ; Tue, 13 May 2008 16:03:07 +0000 (UTC) (envelope-from lidl@rox.fddx.com) Received: from rox.fddx.com (rox.fddx.com [71.241.230.59]) by mx1.freebsd.org (Postfix) with ESMTP id DA0F88FC19 for ; Tue, 13 May 2008 16:03:06 +0000 (UTC) (envelope-from lidl@rox.fddx.com) Received: from rox.fddx.com (localhost.pix.net [127.0.0.1]) by rox.fddx.com (8.13.8+Sun/8.13.8) with ESMTP id m4DFmfeA029152; Tue, 13 May 2008 11:48:41 -0400 (EDT) Received: (from lidl@localhost) by rox.fddx.com (8.13.8+Sun/8.13.8/Submit) id m4DFmf0V029151; Tue, 13 May 2008 11:48:41 -0400 (EDT) Date: Tue, 13 May 2008 11:48:41 -0400 From: "Kurt J. Lidl" To: Anthony Pankov Message-ID: <20080513154840.GB28842@pix.net> Mail-Followup-To: Anthony Pankov , freebsd-hackers@freebsd.org References: <9FC19AC2-DAD8-418C-8B9C-F129DEC58CEF@gmail.com> <15336578.20080512123806@mail.ru> <200805121153.00809.jonathan+freebsd-hackers@hst.org.za> <1663320218.20080512223531@mail.ru> <20080512152430.3720683e@mbook.local> <2117635718.20080513154406@mail.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2117635718.20080513154406@mail.ru> User-Agent: Mutt/1.5.16 (2007-06-09) X-Virus-Scanned: ClamAV version 0.91.2, clamav-milter version 0.91.2 on rox.fddx.com X-Virus-Status: Clean Cc: freebsd-hackers@freebsd.org Subject: Re: BDB corrupt X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 May 2008 16:03:07 -0000 On Tue, May 13, 2008 at 03:44:06PM +0400, Anthony Pankov wrote: > My requirements is > 1. there is no need for SQL > 2. processes are sharing db file in concurrent mode > 3. reading/writing = 60%/40% > > With BDB > clause 1 - satisfied > clause 3 - satisfied (databases of relatively small items that are > changed infrequently). > > Is there a problem with concurrent access? And, most important, is > this the ONLY problem? (I still don't understand because of blurred > mention of "data corruption".) > > If concurrency is the only problem then: > 1. Can data corruption be avoided? Or this is impossible? > 2. How? > > If all BDB readers would use O_SHLOCK and all writers O_EXLOCK is it > guarantee for data integrity? This is probably good enough, if your writers also do a fflush() before releasing the lock. If the writing process ever dies horribly, you can still have database corruption. You could turn the thing that does the actual I/O into the database into daemon that fullfils read/write requests from a unix domain socket. In fact, there is a sample application that does exactly that for the newer Sleepycat BDB code. It could probably be adapted or re-written for the 1.8x series code. If it were me, I'd just punt on using the 1.8x code, drop straight into using the BDB 4.6 codebase, and use their transactioning code. It's mature, robust, etc, etc, for a much large set of operations than the 1.8x code ever was. But there's still that licensing issue. -Kurt