From owner-freebsd-ports@FreeBSD.ORG Fri Aug 20 12:43:24 2010 Return-Path: Delivered-To: freebsd-ports@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EDDBD1065673; Fri, 20 Aug 2010 12:43:23 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 7E5F38FC0A; Fri, 20 Aug 2010 12:43:22 +0000 (UTC) Received: by wwb31 with SMTP id 31so2178712wwb.31 for ; Fri, 20 Aug 2010 05:43:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:date:from:to:cc :subject:message-id:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=iUPQ5LCvFLu6h+F08Curkwo3r4ZblaNqR2n9u2Z3gJY=; b=VfhB2XvQrJJ/ECz7krEUBVclDj9v6+4EpmgBJV7fdAsHkZQ/5kv8qTCM/KPTi0mef5 R1no+wKnJB9YNnaLxUBQB+2RJ7Svn/gw46gnVBJHOwuoHdOBREd1jwkK4q+dfPyQV/2S KJhCUEcDhKT5YmhDRWXAXMr6YsEOeBXok48NU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=CsMesXOxNj5bOKVwzrOZFoAeXRnqacizaahXi8JfH7OTxwQ6SmUnMBCx6aEKkoASm4 JHfHjg1dNlexUCsUjUwkxFRdKaB6VzGHQ/Co/EYSDy5yc9eMMDIRCzYokmxndoMrZJ39 EBJkDAmCPDhthdQRMn9xqd2GAWTxPl7sMbN2A= Received: by 10.227.155.20 with SMTP id q20mr1177544wbw.98.1282308201458; Fri, 20 Aug 2010 05:43:21 -0700 (PDT) Received: from azathoth.lan (stc92-3-82-245-249-89.fbx.proxad.net [82.245.249.89]) by mx.google.com with ESMTPS id i14sm433857wbe.6.2010.08.20.05.43.19 (version=TLSv1/SSLv3 cipher=RC4-MD5); Fri, 20 Aug 2010 05:43:20 -0700 (PDT) Sender: Baptiste Daroussin Date: Fri, 20 Aug 2010 14:43:32 +0200 From: Bapt To: Julien Laffaye Message-ID: <20100820124332.GB1786@azathoth.lan> References: <20100819143830.GJ35140@azathoth.lan> <4C6DA0FA.7080203@DataIX.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ZoaI/ZTpAVc4A5k6" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Florent Thoumie , David Forsythe , Garrett Cooper , Tim Kientzle , Ivan Voras , freebsd-ports@freebsd.org Subject: Re: what next for the pkg_install rewrite X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Aug 2010 12:43:24 -0000 --ZoaI/ZTpAVc4A5k6 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Aug 20, 2010 at 02:09:59PM +0200, Julien Laffaye wrote: > On Fri, Aug 20, 2010 at 12:00 PM, Ivan Voras wrote: >=20 > > > > And... both ideas are completely wrong. SQLite can be imported as a > > single C file + header, which you must agree is practically the > > optimum, and its license is "public domain" which is, if anything, > > "freer" than BSDL and eminently compatible with it. > > > > And if we don't want to build a sqlite.so which can conflict with the > version from ports, we can statically link libpkg to it. Incognito. >=20 >=20 > > BDB in base offer exactly one (1) single "feature", if you can call it > > that: storing and retrieving key-value binary blobs. It has no > > practical concurrency support, it's locking is laughable and upgrading > > data stored as binary blobs is even more nightmarish than maintaining > > the current plist format (and it will lead to similar uglyness soon - > > rather than upgrading the data piece called "x", I'm sure developers > > will introduce new keys called "x_extdata", "x_moredata" etc etc). > > >=20 > If we are going key-value storage, I think that TinyCDB is worth a look. >=20 I also think tinycdb is a good candidate: very simple, clean and fast and And as most of the access are readonly, the fact that is a constant databas= e is not a problem, I already tested tinycdb in my freebsd port of pkgin (an apt= like for pkgsrc) and result in terms of performance and simplicity were clearly in f= avor of tinycdb. but if we go to SQLite it is not a problem for me. [...] > > > > As for backward compatibility: basically it can be done in two ways: > > 1) build a one-time converter that will read the present plist > > database and output a new database or 2) build a compatibility layer > > which would support both the old and the new format at the same time, > > so people upgrading will continue to use their old data. I consider > > the latter wasteful in terms of developer resources and because > > bit-rot will eventually make support for the old format decrepit. > > >=20 > Agreed, a one time converter seems to be the solution. Especially if the > transition occur between two major FreeBSD version (say, 9 - > 10). >=20 I think that for the /var/db/pkg we should make a converter for a one time transition.=20 But tools like pkg_add or pkg_info should be able to works with both new fo= rmat and old one (at least through a wrapper) >=20 > > > > > 2. XML is a bad idea. Great in theory, wonderful in my browser, but a > > > bloated plaintext file with a lot of complexity. I would prefer a > > > database solution that could be dumped to a simple text file format if > > > needed. > > > > XML, at least in my proposal, would not be used for the system package > > database, but would replace (either in part or all with a single file) > > today's "+" files in the packages, together with other purposes where > > some metadata transfer is required. > > > > That said, I would also be happy with other self-described formats > > like JSON. XML is the front runner because it's more standard > > (industry standard, whether someone likes this fact or not!) and there > > is already a parser in base. > > >=20 > The pro of XML here is that we have a parser in base, The con is its > verbosity. But anyway the manifest is not meant to be human readable. > Nevertheless, I prefer JSON, because it's less complex than XML and we can > have a parser/emiter in a single 500SLOC .c file. > And it looks sexier than XML, eh : http://pastebin.com/8hzPSSJC +1 for JSON, XML would have been interesting if we add a validating parser = in base which would have help us to verify the package integrity, but expat is= n't a good validating parser. writting a json parser/emiter is easy, and there al= ready exists tons of BSDL friendly license JSON parser. For exemple we could impo= rt yajl (BSDL) into base or use cJSON (MIT) which consist in one simple C + he= ader file regards, Bapt --ZoaI/ZTpAVc4A5k6 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (FreeBSD) iEYEARECAAYFAkxueHQACgkQ8kTtMUmk6Ew+VQCgh73hO1u/p0wWKe4z3bMsoYfe gXYAmQH/2EAmWP78LbAZsI62vaEzRg9J =x+rx -----END PGP SIGNATURE----- --ZoaI/ZTpAVc4A5k6--