From owner-freebsd-ports@FreeBSD.ORG Fri Aug 20 01:56:20 2010 Return-Path: Delivered-To: freebsd-ports@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 08F5A10656A3 for ; Fri, 20 Aug 2010 01:56:20 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 805D98FC12 for ; Fri, 20 Aug 2010 01:56:19 +0000 (UTC) Received: by bwz20 with SMTP id 20so2643974bwz.13 for ; Thu, 19 Aug 2010 18:56:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=y9hedTpywI0XNB4Kg1puMqWYmEIBsbLmw8nW/uVXNXU=; b=T/FCMH19RpjLRTWvpk7RZW28I525TprtSJHz22sJPiMLp8dxQKGRjO/9mHdmPHmun8 Ts7ESXgHuSrZ/KEJrZ5c1N9pWPN95XuAm6iAhvptmJiXBZX7XJo5A2fqBQWd50YNfGJz jI6/xMrhk1mgf86Gz0nD7AjliQpVGuveK0sKc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=Ev44q4rgx9YXgbZCYZ/RIQxa4WsdIdZOcovVfYRrvhvu9QJRmbDWyAvr1KKTGstAi0 k3D9N2bD65jCSpu+U814rbu2wGVQj7TNmmRh4yawGUMg7J0gH1FJzaN7ranWWC14Y3WM u4BPoYjVexEdcd74z7TU7V1AxkCifDAZ0gKP0= MIME-Version: 1.0 Received: by 10.204.49.205 with SMTP id w13mr498528bkf.28.1282267565876; Thu, 19 Aug 2010 18:26:05 -0700 (PDT) Sender: yanegomi@gmail.com Received: by 10.204.82.6 with HTTP; Thu, 19 Aug 2010 18:26:05 -0700 (PDT) In-Reply-To: References: <20100819143830.GJ35140@azathoth.lan> <4C6DA0FA.7080203@DataIX.net> Date: Thu, 19 Aug 2010 18:26:05 -0700 X-Google-Sender-Auth: p6wE2SPMuBZmirkzNKsrvTFgBH0 Message-ID: From: Garrett Cooper To: Ivan Voras Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: bapt@freebsd.org, Florent Thoumie , Julien Laffaye , David Forsythe , Tim Kientzle , freebsd-ports@freebsd.org Subject: Re: what next for the pkg_install rewrite X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Aug 2010 01:56:20 -0000 On Thu, Aug 19, 2010 at 3:10 PM, Ivan Voras wrote: > On 19/08/2010, jhell wrote: >> =A0 =A0 =A0 Adding to this I would like to see a central database create= d for >> packages that have been removed like in Slackware Linux. They keep a >> file in /var/log/preserved_packages with a flat text format with the >> file name looking like: >> >> ${PORTNAME}-${PORTVERSION}${PORTREVISION}-`date +%Y%m%d%H%M%S` Please no. We need another ad hoc text format like we need holes in our hea= d. > Ah yes, you reminded me of this other thing: I would also suggest > getting rid of text files carrying rich information in ad-hoc formats > :) > > I'm not saying XML should be the only choice, but it *is* well > supported - expat is even in base as libbsdxml. > > While suggesting nebulous things I know will be hard to pass near a > lot of people: sqlite is *the* choice for any record-based file > databases today. The single most important thing I'll promote with it > is its transaction capabilities and ACID - these would get much use if > parallel operations (upgrades / installs) are to be supported. There > are a ton of other reasons too. > > I started writing this a long time ago but abandoned it because of > strong opposition: http://wiki.freebsd.org/PortsUsingSQLite - maybe it > would help at this time. There are a lot of ideas going around, but unless the current patches get polished up and portmgr (points in flz's general direction) actually accept ideas, ideas are nothing more than ideas and will essentially be vaporware. That has been the chokepoint for patches I've added to the PR database. Also, if it breaks backwards compatibility used everywhere over the map and is actually functionality that's used in ports and src packages without equivalent functionality, it's a non-starter. If any work needs to be done, we should be getting _rid_ of features which aren't in use, or can be easily replaced with equivalent methods. Stuff I've identified over the past year that can be worked on are: =3D=3D=3D=3D=3D=3D=3D=3D pkg_install =3D=3D=3D=3D=3D=3D=3D=3D Locking: - We need concurrent package installation. Packing list: @dirrmtry (needs to be replaced with scripting infrastructure called via +INSTALL/+DEINSTALL). See @exec/@unexec (similar problem). @exec/@unexec (needs to be replaced with scripting infrastructure using +INSTALL/+DEINSTALL). @group/@mode/@owner (needs to be replaced with scripting infrastructure). @srcdir really shouldn't be in a plist (someone can script the call easily with pkg_create). At the end of the day, it would be really nice if there was one small file with the package metadata, and the other file was checksums via an mtree file. mtree files will solve a lot of the problems with the ad hoc plist `format'. @noinst should be checked to see whether or not it's widely used in ports. It might be a good idea to implement, but I'm not confident in that statement. Dependencies: - We should be using reverse dependencies, not forward dependencies. There's less value in forward dependencies, because I can uninstall things, and I don't have an accurate idea of what the current state is on the system. Reverse dependencies actually tell us _what_ we depend on, and should be expressed in terms of origins, not package versions. Whenever the version for the origins change, the package should be bumped. This would (IMO) reduce a lot of headaches with packages. INDEX*: - We should depend on INDEX* for package data, and it should be distributed with the base system, not ports, and ports should build off of this data. *libpkg*!!: - I emphasize this, because it's extremely important... David/Florent are already doing work to make this a reality, BUT we shouldn't be duplicating their work. It would be nice if the work being done by both Florent and David was more transparent, could be tasked out to various individuals, etc... but that's not how it is today. =3D=3D=3D=3D=3D=3D=3D=3D Ports =3D=3D=3D=3D=3D=3D=3D=3D bsd.*.mk: - Needs to use pkg_add / pkg_create out of the box instead of installing things directly into /. This would avoid a lot of issues with corrupted installs, etc. - [Nice to have] should be trimmed wherever possible. There's a lot of cruft that actually could be moved elsewhere to reduce the amount of work in terms of parsing and evaluating statements for everyday ports users. In general: - Ports need to be fixed so that BUILD_DEPENDS and RUN_DEPENDS don't end up in the package dependency list. We suffer from this in areas, Redhat suffers from this, etc, and it's not an easy task to do. However, if done properly, this will 1) speed up package building, 2) package installation, 3) reduce installed package count, etc. Just for the sake of not repeating past discussions: 1. SQLite was killed before because of complexity and because it was needs another package in base that isn't BSD licensed. That's why everyone in the know has been pushing for BDB 1.8x (in base). People suggested that to me back when I was trying to get off the ground in GSoC 2006, they did it to you before Ivan, and I'm sure they've done it before in the past. We need to implement things in terms of BDB first, but make the APIs generic enough so that it _could_ be extended to SQLite if people chose to do the work later on. 2. XML is a bad idea. Great in theory, wonderful in my browser, but a bloated plaintext file with a lot of complexity. I would prefer a database solution that could be dumped to a simple text file format if needed. Again (and I can't overemphasize this): no matter what we do, say, etc, unless we have buy-in from portmgr beforehand on anything here, the work will never see a ports/src CVS/svn repo, etc. This includes work done for past GSoCs, and current GSoCs. Thanks, -Garrett