From owner-freebsd-ports@FreeBSD.ORG Fri Nov 18 13:34:17 2005 Return-Path: X-Original-To: freebsd-ports@freebsd.org Delivered-To: freebsd-ports@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B4FC616A41F; Fri, 18 Nov 2005 13:34:17 +0000 (GMT) (envelope-from vd@datamax.bg) Received: from jengal.datamax.bg (jengal.datamax.bg [82.103.104.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 07AB943D46; Fri, 18 Nov 2005 13:34:16 +0000 (GMT) (envelope-from vd@datamax.bg) Received: from qlovarnika.bg.datamax (qlovarnika.bg.datamax [192.168.10.2]) by jengal.datamax.bg (Postfix) with SMTP id B25B3B854; Fri, 18 Nov 2005 15:35:33 +0200 (EET) Received: (nullmailer pid 24469 invoked by uid 1002); Fri, 18 Nov 2005 13:34:21 -0000 Date: Fri, 18 Nov 2005 15:34:21 +0200 From: Vasil Dimov To: Pav Lucistnik Message-ID: <20051118133421.GA17545@qlovarnika.bg.datamax> References: <200511171540.jAHFeJSL017500@freefall.freebsd.org> <200511170807.46482.ringworm01@gmail.com> <1132244794.79514.28.camel@localhost> <200511170832.06601.ringworm01@gmail.com> <1132245985.79514.32.camel@localhost> <20051117193353.GA53615@qlovarnika.bg.datamax> <1132257478.79514.47.camel@localhost> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ZPt4rx8FFjLCG7dd" Content-Disposition: inline In-Reply-To: <1132257478.79514.47.camel@localhost> X-OS: FreeBSD 6.0-STABLE User-Agent: Mutt/1.5.11 Cc: freebsd-ports@freebsd.org Subject: Re: ports/89164: [PATCH] /var/db/pkg/{portname}/+CONTENTS files sometimes contain wrong data X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vd@datamax.bg List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Nov 2005 13:34:17 -0000 --ZPt4rx8FFjLCG7dd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Migrate this from ports-bugs@ to ports@ to see if something good comes out from my patch. On Thu, Nov 17, 2005 at 08:57:57PM +0100, Pav Lucistnik wrote: > > > Yes, there is a long standing problem and we're aware of it. Sadly, no > > > workable solution was submitted so far (at least I haven't found any = PR > > > filed against it). > > >=20 > >=20 > > Hmmz, what about the following: > >=20 > > --- bsd.port.mk_dep.diff begins here --- > > --- bsd.port.mk.orig Thu Nov 17 21:04:11 2005 > > +++ bsd.port.mk.dep Thu Nov 17 21:18:21 2005 > > @@ -4638,7 +4638,7 @@ > > dir=3D$$(${REALPATH} $$dir); \ > > if [ -d $$dir ]; then \ > > if (${ECHO_CMD} $$checked | ${GREP} -qwv "$$dir"); then \ > > - childout=3D$$(cd $$dir; ${MAKE} CHILD_DEPENDS=3Dyes PARENT_CHECKED= =3D"$$checked" package-depends-list); \ > > + childout=3D$$(cd $$dir ; pkgnm=3D`${MAKE} -V PKGNAME` ; if [ -d ${= PKG_DBDIR}/$$pkgnm ] ; then for p in $$pkgnm `${PKG_INFO} -qr ${PKG_DBDIR}/= $$pkgnm |${CUT} -f 2 -d ' '` ; do porigin=3D`pkg_info -qo /var/db/pkg/$$p` = ; ${ECHO_CMD} "$$p ${PORTSDIR}/$$porigin $$porigin" ; done ; else ${MAKE} C= HILD_DEPENDS=3Dyes PARENT_CHECKED=3D"$$checked" package-depends-list; fi); \ > > set -- $$childout; \ > > childdir=3D""; \ > > while [ $$\# !=3D 0 ]; do \ > > @@ -4651,7 +4651,7 @@ > > else \ > > ${ECHO_MSG} "${PKGNAME}: \"$$dir\" non-existent -- dependency list = incomplete" >&2; \ > > fi; \ > > - done > > + done | ${SORT} -u > > =20 > > # Print out package names. > > =20 > > --- bsd.port.mk_dep.diff ends here --- > >=20 > > The idea is to check if the port is installed and get necessary > > info with `pkg_info -r', instead of `make package-depends-list' > >=20 > > Looks quite ugly, some part of the sh code can be extracted > > into separate make variable or at least that 380+ bytes line can be > > broken down into shorter lines. >=20 > Have you runtested this? I'm not sure, this is the code which creates > the list which is later stored in /var/db/pkg, right? So you can't read > it before it's written. >=20 Ok, I should have explained more, let me do it now: Below I mean bsd.port.mk 1.518 Firstly let's describe the problem so it's clear what we are trying to solve. We shall use an example: 1. databases/db4 is installed 2. databases/p5-BerkeleyDB is installed with WITH_BDB_VER=3D4, that's why the installed version of p5-BerkeleyDB depends on db4, instead of the default db3. That information is kept in /var/db/pkg, not in /usr/ports. # pkg_info -qr /var/db/pkg/p5-BerkeleyDB-0.27 @pkgdep perl-5.8.7 @pkgdep db4-4.0.14_1,1 # # cd /usr/ports/databases/p5-BerkeleyDB # make package-depends db3-3.3.11_2,1:databases/db3 perl-5.8.7:lang/perl5.8 # You can see the problem creeping behind the corner. Then we install something that depends on p5-BerkeleyDB, say amavisd-new. What we get is that db3 is recorded as dependency for amavisd-new, although it is not even installed on the system. This is the problem. Let's investigate how we can solve it. Why does it happen? Obviously because dependency information (for amavisd-new) is retrieved from /usr/ports, which does not contain any tracks for our previous deed - p5-BerkeleyDB installation with WITH_BDB_VER=3D4 being set. Then we focus on line 2052 in bsd.port.mk and more precisely on the -P argument - that's where dependencies are fed to pkg_create, using ``${MAKE} package-depends''. So we decide to alter/fix ``package-depends'' target in such a way that it looks for dependencies information in /var/db/pkg if the port is installed and falls back to /usr/ports otherwise. Then we go to line 4658 and then to 4622 and investigate what's happening: skip some stuff and focus on line 4641, that's where dependencies are calculated with ``${MAKE} ... package-depends-list''. Because we want to touch as little as possible we choose the following algo: if port is installed then look at /var/db/pkg with pkg_info -r else fallback to /usr/ports with make ... package-depends-list This turns into get package_name if package_name is installed then for package itself and each dependency returned by pkg_info -r do print stuff just like package-depends-list does, being transparent to the outside world else do it in the old way - make ... package-depends-list This turns into (variable names should be improved): pkgnm=3D`make -V PKGNAME` if [ -d /var/db/pkg/$pkgnm ] ; then for p in $pkgnm `pkg_info -qr /var/db/pkg/$pkgnm |cut -f 2 -d ' '` ; do porigin=3D`pkg_info -qo /var/db/pkg/$p` echo "$p /usr/ports/$porigin $porigin" # just like package-depends-list done else make ... package-depends-list This is it. Let's test it: # ls -d /var/db/pkg/db* /var/db/pkg/p5-BerkeleyDB-0.27 /var/db/pkg/db4-4.0.14_1,1 /var/db/pkg/p5-BerkeleyDB-0.27 # # cd /usr/ports/security/amavisd-new # make package-depends |sort > /tmp/old (apply the patch to bsd.port.mk) # make package-depends |sort > /tmp/new # diff -u /tmp/old /tmp/new --- /tmp/old Fri Nov 18 15:21:15 2005 +++ /tmp/new Fri Nov 18 15:22:36 2005 @@ -1,6 +1,6 @@ arc-5.21n:archivers/arc cabextract-1.1:archivers/cabextract -db3-3.3.11_2,1:databases/db3 +db4-4.0.14_1,1:databases/db4 freeze-2.5_1:archivers/freeze lha-1.14i_6:archivers/lha lzo2-2.02:archivers/lzo2 # Seems like it does the right thing. --=20 Vasil Dimov --ZPt4rx8FFjLCG7dd Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- iD8DBQFDfdhcFw6SP/bBpCARAuAjAKCUjm1CxspE8k2a/+nNKHs1Z0qrzgCfZmTA zlzlUBtM/V1aBZKP90jl3K4= =e5IF -----END PGP SIGNATURE----- --ZPt4rx8FFjLCG7dd--