From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 6 08:46:04 2013 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 80C1E339; Wed, 6 Nov 2013 08:46:04 +0000 (UTC) (envelope-from atte.peltomaki@iki.fi) Received: from filtteri1.pp.htv.fi (filtteri1.pp.htv.fi [213.243.153.184]) by mx1.freebsd.org (Postfix) with ESMTP id 1F09C2BB2; Wed, 6 Nov 2013 08:46:03 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by filtteri1.pp.htv.fi (Postfix) with ESMTP id F399721BC49; Wed, 6 Nov 2013 10:45:55 +0200 (EET) X-Virus-Scanned: Debian amavisd-new at pp.htv.fi Received: from smtp6.welho.com ([213.243.153.40]) by localhost (filtteri1.pp.htv.fi [213.243.153.184]) (amavisd-new, port 10024) with ESMTP id V8VN4P3WDBWW; Wed, 6 Nov 2013 10:45:51 +0200 (EET) Received: from kameli.org (cs181030192.pp.htv.fi [82.181.30.192]) by smtp6.welho.com (Postfix) with SMTP id E7ABE5BC003; Wed, 6 Nov 2013 10:45:50 +0200 (EET) Received: by kameli.org (sSMTP sendmail emulation); Wed, 06 Nov 2013 10:45:50 +0200 From: "Atte Peltomaki" Date: Wed, 6 Nov 2013 10:45:50 +0200 To: Baptiste Daroussin , hackers@FreeBSD.org Subject: Re: Importing netbsd cdb Message-ID: <20131106084550.GA21489@ass.pp.htv.fi> References: <20131027232628.GB74512@ithaqua.etoilebsd.net> <20131105220225.GF37879@caravan.chchile.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20131105220225.GF37879@caravan.chchile.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2013 08:46:04 -0000 On Tue, Nov 05, 2013 at 11:02:26PM +0100, Jeremie Le Hen wrote: > On Mon, Oct 28, 2013 at 12:26:28AM +0100, Baptiste Daroussin wrote: > > > > Here is a code that imports the cdbrw from netbsd into a new lib/libcdbrw > > library, the read part is also added to libc but not exposed. > > > > As an example of using that library I also got the service_mkdb patches from > > netbsd that makes it by default emit a cdb database and add a switch to allow it > > to still create the old .db database format. > > > > in the libc, getservent has been modified to first try to read the .cdb files > > and fallback on reading the old .db file. (I'm not sure if it is worth keeping > > reading the old db format.) > > > > http://people.freebsd.org/~bapt/cdbrw.diff > > > > The plan after that is to get pw_util(3) directly generating a cdb file for > > pwd.db and spwd.db and modifiy pwd_mkdb(8) so that by default it uses the API > > from pw_util(3) and have a switch to fallback on creating in bdb format. > > > > I also plan to do the same for cap_mkdb(1) and getcap(3). > > > > With cdb querying is way faster (I don't have number but I can get some it > > needed) > > the size of the db is also way smaller: > > 64K /var/db/services.cdb > > 2,1M /var/db/services.db > > > > Any objection? > > What are the benefits of this, beside generating a much smaller > database? Do you know about benchmarks that compare it against the > current database engine used? It would be unfortunate to notice a > regression afterward :). At least randomly googled third-party benchmarks suggest that CDB is vastly superior to BDB: http://www.dmo.ca/blog/benchmarking-hash-databases-on-large-data/ -- Atte Peltomäki atte.peltomaki@iki.fi <> http://kameli.org "Your effort to remain what you are is what limits you"