Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 18 Nov 2005 15:34:21 +0200
From:      Vasil Dimov <vd@datamax.bg>
To:        Pav Lucistnik <pav@FreeBSD.org>
Cc:        freebsd-ports@freebsd.org
Subject:   Re: ports/89164: [PATCH] /var/db/pkg/{portname}/+CONTENTS files sometimes contain wrong data
Message-ID:  <20051118133421.GA17545@qlovarnika.bg.datamax>
In-Reply-To: <1132257478.79514.47.camel@localhost>
References:  <200511171540.jAHFeJSL017500@freefall.freebsd.org> <200511170807.46482.ringworm01@gmail.com> <1132244794.79514.28.camel@localhost> <200511170832.06601.ringworm01@gmail.com> <1132245985.79514.32.camel@localhost> <20051117193353.GA53615@qlovarnika.bg.datamax> <1132257478.79514.47.camel@localhost>

next in thread | previous in thread | raw e-mail | index | archive | help

--ZPt4rx8FFjLCG7dd
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Migrate this from ports-bugs@ to ports@ to see if something good comes
out from my patch.

On Thu, Nov 17, 2005 at 08:57:57PM +0100, Pav Lucistnik wrote:
> > > Yes, there is a long standing problem and we're aware of it. Sadly, no
> > > workable solution was submitted so far (at least I haven't found any =
PR
> > > filed against it).
> > >=20
> >=20
> > Hmmz, what about the following:
> >=20
> > --- bsd.port.mk_dep.diff begins here ---
> > --- bsd.port.mk.orig	Thu Nov 17 21:04:11 2005
> > +++ bsd.port.mk.dep	Thu Nov 17 21:18:21 2005
> > @@ -4638,7 +4638,7 @@
> >  		dir=3D$$(${REALPATH} $$dir); \
> >  		if [ -d $$dir ]; then \
> >  			if (${ECHO_CMD} $$checked | ${GREP} -qwv "$$dir"); then \
> > -				childout=3D$$(cd $$dir; ${MAKE} CHILD_DEPENDS=3Dyes PARENT_CHECKED=
=3D"$$checked" package-depends-list); \
> > +				childout=3D$$(cd $$dir ; pkgnm=3D`${MAKE} -V PKGNAME` ; if [ -d ${=
PKG_DBDIR}/$$pkgnm ] ; then for p in $$pkgnm `${PKG_INFO} -qr ${PKG_DBDIR}/=
$$pkgnm |${CUT} -f 2 -d ' '` ; do porigin=3D`pkg_info -qo /var/db/pkg/$$p` =
; ${ECHO_CMD} "$$p ${PORTSDIR}/$$porigin $$porigin" ; done ; else ${MAKE} C=
HILD_DEPENDS=3Dyes PARENT_CHECKED=3D"$$checked" package-depends-list; fi); \
> >  				set -- $$childout; \
> >  				childdir=3D""; \
> >  				while [ $$\# !=3D 0 ]; do \
> > @@ -4651,7 +4651,7 @@
> >  		else \
> >  			${ECHO_MSG} "${PKGNAME}: \"$$dir\" non-existent -- dependency list =
incomplete" >&2; \
> >  		fi; \
> > -	done
> > +	done | ${SORT} -u
> > =20
> >  # Print out package names.
> > =20
> > --- bsd.port.mk_dep.diff ends here ---
> >=20
> > The idea is to check if the port is installed and get necessary
> > info with `pkg_info -r', instead of `make package-depends-list'
> >=20
> > Looks quite ugly, some part of the sh code can be extracted
> > into separate make variable or at least that 380+ bytes line can be
> > broken down into shorter lines.
>=20
> Have you runtested this? I'm not sure, this is the code which creates
> the list which is later stored in /var/db/pkg, right? So you can't read
> it before it's written.
>=20

Ok, I should have explained more, let me do it now:

Below I mean bsd.port.mk 1.518

Firstly let's describe the problem so it's clear what we are trying to
solve. We shall use an example:
1. databases/db4 is installed
2. databases/p5-BerkeleyDB is installed with WITH_BDB_VER=3D4, that's why
the installed version of p5-BerkeleyDB depends on db4, instead of the
default db3. That information is kept in /var/db/pkg, not in /usr/ports.

# pkg_info -qr /var/db/pkg/p5-BerkeleyDB-0.27
@pkgdep perl-5.8.7
@pkgdep db4-4.0.14_1,1
#

# cd /usr/ports/databases/p5-BerkeleyDB
# make package-depends
db3-3.3.11_2,1:databases/db3
perl-5.8.7:lang/perl5.8
#

You can see the problem creeping behind the corner.

Then we install something that depends on p5-BerkeleyDB, say amavisd-new.
What we get is that db3 is recorded as dependency for amavisd-new,
although it is not even installed on the system. This is the problem.

Let's investigate how we can solve it.

Why does it happen? Obviously because dependency information
(for amavisd-new) is retrieved from /usr/ports, which does not contain
any tracks for our previous deed - p5-BerkeleyDB installation
with WITH_BDB_VER=3D4 being set.

Then we focus on line 2052 in bsd.port.mk and more precisely on the -P
argument - that's where dependencies are fed to pkg_create, using
``${MAKE} package-depends''.

So we decide to alter/fix ``package-depends'' target in such a way that
it looks for dependencies information in /var/db/pkg if the port is
installed and falls back to /usr/ports otherwise.

Then we go to line 4658 and then to 4622 and investigate what's
happening: skip some stuff and focus on line 4641, that's where
dependencies are calculated with ``${MAKE} ... package-depends-list''.

Because we want to touch as little as possible we choose the following
algo:

if port is installed then
	look at /var/db/pkg with pkg_info -r
else
	fallback to /usr/ports with make ... package-depends-list

This turns into

get package_name
if package_name is installed then
	for package itself and each dependency returned by pkg_info -r do
		print stuff just like package-depends-list does, being
		transparent to the outside world
else
	do it in the old way - make ... package-depends-list

This turns into (variable names should be improved):

pkgnm=3D`make -V PKGNAME`
if [ -d /var/db/pkg/$pkgnm ] ; then
	for p in $pkgnm `pkg_info -qr /var/db/pkg/$pkgnm |cut -f 2 -d ' '` ; do
		porigin=3D`pkg_info -qo /var/db/pkg/$p`
		echo "$p /usr/ports/$porigin $porigin"  # just like package-depends-list
	done
else
	make ... package-depends-list

This is it. Let's test it:

# ls -d /var/db/pkg/db* /var/db/pkg/p5-BerkeleyDB-0.27
/var/db/pkg/db4-4.0.14_1,1
/var/db/pkg/p5-BerkeleyDB-0.27
#

# cd /usr/ports/security/amavisd-new
# make package-depends |sort > /tmp/old
(apply the patch to bsd.port.mk)
# make package-depends |sort > /tmp/new
# diff -u /tmp/old /tmp/new
--- /tmp/old	Fri Nov 18 15:21:15 2005
+++ /tmp/new	Fri Nov 18 15:22:36 2005
@@ -1,6 +1,6 @@
 arc-5.21n:archivers/arc
 cabextract-1.1:archivers/cabextract
-db3-3.3.11_2,1:databases/db3
+db4-4.0.14_1,1:databases/db4
 freeze-2.5_1:archivers/freeze
 lha-1.14i_6:archivers/lha
 lzo2-2.02:archivers/lzo2
#

Seems like it does the right thing.

--=20
Vasil Dimov

--ZPt4rx8FFjLCG7dd
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----

iD8DBQFDfdhcFw6SP/bBpCARAuAjAKCUjm1CxspE8k2a/+nNKHs1Z0qrzgCfZmTA
zlzlUBtM/V1aBZKP90jl3K4=
=e5IF
-----END PGP SIGNATURE-----

--ZPt4rx8FFjLCG7dd--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20051118133421.GA17545>