From owner-freebsd-arch@FreeBSD.ORG Mon May 7 20:48:55 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9666F106566C for ; Mon, 7 May 2012 20:48:55 +0000 (UTC) (envelope-from peter@wemm.org) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 09EBA8FC0A for ; Mon, 7 May 2012 20:48:54 +0000 (UTC) Received: by yhgm50 with SMTP id m50so5506142yhg.13 for ; Mon, 07 May 2012 13:48:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wemm.org; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=4jC95xvR4DdO2m2wDm4xsXujzCJ7TQmcdVepQ0/gX2A=; b=KbQ90JNBteLpPgFuxyr/ZwYN50yCC2F5FK06/C6JNKqNZrw9TE8fgYGrDv2R676u+d nCHXkmVeYeFQAA5Hr9zN5MG1HOwXeVEiPNbv2LKfNO94nfGrMpVtF3N46niTlVzDszPY 3v+v+PYVxiW6i9C62RQGbRd3gajMZATZTZUDU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=4jC95xvR4DdO2m2wDm4xsXujzCJ7TQmcdVepQ0/gX2A=; b=ENPknVSgv0smmTbYG08BStqUfmV4dqmaLwJ+wIZYVnwO1rWdm9yEcyhRzNcK7LF5Wi yl/wXGjdEiccXQoFcczoJqIvJFhbIKQElhO6zGb41inpm0e25SrOq2TyVyaHubGAAgW3 PKnVd0eGKWd6SW2HQ5eXOFvtofH0Um84kKvBmZF9TTVZM3JjXu50Cn6ZG8IvAwBztDN4 yQnN+5GuhDQ392puwQsOYvHuayO0FH4fQJVmsmeecUGc7XVK50YLInw5L+O0v3r6sZeI yVxZkVLQgztE0TM3yY//GwX+qvloNOTAOl1rcwC03V/6SmIl69sn+zGjRFKQ0vXhB0CJ sBrg== MIME-Version: 1.0 Received: by 10.50.179.38 with SMTP id dd6mr9418758igc.9.1336423734217; Mon, 07 May 2012 13:48:54 -0700 (PDT) Received: by 10.231.243.4 with HTTP; Mon, 7 May 2012 13:48:54 -0700 (PDT) In-Reply-To: <20120502114115.GG31034@azathoth.lan> References: <20120502114115.GG31034@azathoth.lan> Date: Mon, 7 May 2012 13:48:54 -0700 Message-ID: From: Peter Wemm To: Baptiste Daroussin Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQmsg3Poik+620yQ590wXd5Q5CjPZ16s5rHDLWybxwhT1JE0VDb71nDDFgB5leu93nccaxz3 Cc: freebsd-arch@freebsd.org Subject: Re: Switching /etc/*.db from bdb to tinycdb X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 May 2012 20:48:55 -0000 On Wed, May 2, 2012 at 4:41 AM, Baptiste Daroussin wrote: > Hi, > > I plan to work on switching all the /etc/*.db files from bdb 1.85 to cdb (using > the tinycdb implementation) > > Reason for doing that: > 1/ we use the *.db files as a constant database on recreating them using > *_mkdb(8) commands > 2/ bdb 1.85 database are endianness dependant which can cause problem if I want > pkgng to be able to create users when cross installing packages (like adding > packages in a mips eb chroot environment on an amd64 host) > 3/ the tinycdb API is clean and easy to use (I find it easier and cleaner to > deal with it) > 4/ tinycdb is public domain so not problem for us to use it. > > On the side effects, I'm not sure this is relevant but tinycdb is also faster to > read then bdb 1.85. > > How do I plan to do it. > > I plan to import the read part or tinycdb into libc and the write part into > libutil (to avoid cluttering libc). > > Why importing to libc? to allow all the get*ent to parse the /etc/*.db files. > > How to implment it? slowly with backward compatibility: > for each kind of files: try to open with cdb, if fail fall back to bdb. > > I plan to rewrite all the *_mkdb for make them use cdb. > > Link for tinycdb: > http://www.corpit.ru/mjt/tinycdb.html > > Any opinion, ideas, advices on this? Please keep in mind that old binaries need to keep working. Old libc doesn't have a text file parser in it - the only thing it knows is how to read the *.db files, so they can never go away or be made incompatible. eg: there are people "supposedly" running a libc.so.4 based oracle. Even FreeBSD-2.1.5 a.out binaries still understand the *.db files that we use (assuming you dig up the a.out stuff to make them run). Please keep backwards compatability in mind. Even if that means moving db/* to a standalone shim/converter utility or have some other way to generate/update the "old" *.db files. This has far bigger impact than things like the utmp/utx change. Also keep in mind the symbol scope stuff may not necessarily have the same visibility goals as everyything else. eg: libc.so.7 based binaries from an old machine should still "see" the new implementation - things like getpwnam() ABI won't be changing. You *do want* an ancient libc.so.7 binary that was compiled on 7.0-release to correctly use the new implementation when its run on a 10.0 machine. Please take a lot of care with a change like this. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV "All of this is for nothing if we don't go to the stars" - JMS/B5 "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell