From owner-freebsd-arch@FreeBSD.ORG Mon Apr 28 05:06:40 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D9B4E37B401 for ; Mon, 28 Apr 2003 05:06:40 -0700 (PDT) Received: from gw.nectar.cc (gw.nectar.cc [208.42.49.153]) by mx1.FreeBSD.org (Postfix) with ESMTP id 05E2343F75 for ; Mon, 28 Apr 2003 05:06:40 -0700 (PDT) (envelope-from nectar@celabo.org) Received: from madman.celabo.org (madman.celabo.org [10.0.1.111]) (using TLSv1 with cipher EDH-RSA-DES-CBC3-SHA (168/168 bits)) (Client CN "madman.celabo.org", Issuer "celabo.org CA" (verified OK)) by gw.nectar.cc (Postfix) with ESMTP id 6DE68E; Mon, 28 Apr 2003 07:06:39 -0500 (CDT) Received: by madman.celabo.org (Postfix, from userid 1001) id BA07878C66; Mon, 28 Apr 2003 07:06:38 -0500 (CDT) Date: Mon, 28 Apr 2003 07:06:38 -0500 From: "Jacques A. Vidrine" To: Poul-Henning Kamp Message-ID: <20030428120638.GA3289@madman.celabo.org> References: <31398.1049876261@critter.freebsd.dk> <20030409122620.GC19391@madman.celabo.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030409122620.GC19391@madman.celabo.org> X-Url: http://www.celabo.org/ User-Agent: Mutt/1.5.3i-ja.1 cc: arch@freebsd.org Subject: Re: endianess of /etc/pwd.db X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2003 12:06:41 -0000 On Wed, Apr 09, 2003 at 07:26:20AM -0500, Jacques A. Vidrine wrote: > On Wed, Apr 09, 2003 at 10:17:41AM +0200, Poul-Henning Kamp wrote: > > > > Kris ran into this problem: copying a /etc/pwd.db from one endianess > > to another gave him really weird uid/gid numbers. > > > > The DB code itself is endianess-agnostic, so the first warning one > > gets is the weird UID/GID. > > > > Should we make the endianess of this file explicit to prevent this > > pit-fall for our users ? The cost would be less than epsilon. > > I am working in this area recently, and the exact same thought > occurred to me. I think we should do it. I would add a file format > version flag, the absence of which indicates the current MD format, > and adjust getpwent/pwd_mkdb accordingly. Are there any other tools > that would need to be touched? In case you didn't otherwise notice, this is done. The `.db' files now have versioned entries. It is necessary for the `old MD formatted' and `new MI formatted' entries to co-exist within the .db files so that old binaries still work. A `version key' within the database indicates to consumers what version they should use for lookups. The key is "\xFF" "VERSION" (8 bytes) and the value is a single byte representation of the integer version. The `old MD formatted' version is version 3; the `new MI formatted' version is version 4. All entries have historically had a single-byte tag to seperate the namespace for lookups by name, by user-ID, and so forth. The tags have had values that happened to be ASCII-encoded digits (i.e. '1' -> '\x31', '2' -> 'x32', ... ). These tags are now interpreted to contain the version in the upper four bits, and the namespace in the lower four bits. (Now you see why I picked version 3 for old entries.) For example, the keys for the entry `nectar:*:1001:...' on i386 would be "\x31" "nectar" MD (version 3) by name "\x32" "\xe9\x03\0\0" MD (version 3) by user-ID "\x41" "nectar" MI (version 4) by name "\x42" "\0\0\x03\xe9" MI (version 4) by user-ID Version 4 (MI formatted) entries have all integers encoded as 32-bit values in network byte order, but are otherwise the same as the old MD formatted entries. libc will read version 4 entries if the version key is present, or else attempt to read version 3 entries. pwd_mkdb creates both version 3 and version 4 entries. Thus, you may safely move your versioned '.db' files between architectures (as long as the DB format itself doesn't change!), and binaries using a new libc will return the right results for getpwent() et. al. Cheers, -- Jacques Vidrine . NTT/Verio SME . FreeBSD UNIX . Heimdal nectar@celabo.org . jvidrine@verio.net . nectar@freebsd.org . nectar@kth.se