Date: Tue, 19 Mar 2013 18:10:01 GMT From: Jeremy Messenger <mezz.freebsd@gmail.com> To: gnome@FreeBSD.org Subject: Re: ports/175276: [patch] devel/py-gobject OPTIONSFILE eval order problem Message-ID: <201303191810.r2JIA1d3072127@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
The following reply was made to PR ports/175276; it has been noted by GNATS.
From: Jeremy Messenger <mezz.freebsd@gmail.com>
To: Koop Mast <kwm@rainbow-runner.nl>
Cc: "John W. O'Brien" <john@saltant.com>, gnome@freebsd.org,
FreeBSD Ports Mailing List <ports@freebsd.org>, Baptiste Daroussin <bapt@freebsd.org>, bug-followup@freebsd.org
Subject: Re: ports/175276: [patch] devel/py-gobject OPTIONSFILE eval order problem
Date: Tue, 19 Mar 2013 13:02:33 -0500
On Tue, Mar 19, 2013 at 12:50 PM, Koop Mast <kwm@rainbow-runner.nl> wrote:
> On 19-3-2013 17:56, Jeremy Messenger wrote:
>>
>> Sorry, this is way too long to read. I will just skip the read and
>> post my suggest of solution to this problem in the top of your email.
>> I think the OPTIONS needs to change from ${UNIQUENAME} to
>> ${PKGORIGIN:S/\//_/}. It will be looked like
>> "${PORT_DBDIR}/cat_port/options". Here's example:
>>
>> In bsd.options.mk:
>> -----------------------------------
>> [...]
>>
>> OPTIONSFILE?= ${PORT_DBDIR}/${PKGORIGIN:S/\//_/}/options
>> -----------------------------------
>>
>> Then add compatible in somewhere like this:
>> -----------------------------------
>> .if exist (${PORT_DBDIR}/${UNIQUENAME}/options)
>> @${MV} ${PORT_DBDIR}/${UNIQUENAME}
>> ${PORT_DBDIR}/${PKGORIGIN:S/\//_/}
>> .endif
>> -----------------------------------
>>
>> Then teach the portmaster about if the port has been moved to the
>> different category or renamed (by read MOVED) then change the
>> ${PORT_DBDIR}/${PKGORIGIN:S/\//_/}.
>>
>> What do anyone think of my suggest solution? I haven't test anything
>> at all, which it's just what I have in my mind right now.
>
>
> I was thinking of just axing the option completely and moving libffi to
> lib_depend. glib20 already depends on libffi, so we also could get away with
> removing it completely. But that doesn't resolve that this problem might
> appear in other ports.
My suggest was for solve an issue in the OPTIONS, so that way all
other ports will be fixed. The py-gobject isn't only a port that have
this issue as it's a bug in the OPTIONS.
> -Koop
>
>
>>
>> On Sat, Feb 23, 2013 at 11:30 AM, John W. O'Brien <john@saltant.com>
>> wrote:
>>>
>>> The following reply was made to PR ports/175276; it has been noted by
>>> GNATS.
>>>
>>> From: "John W. O'Brien" <john@saltant.com>
>>> To: bug-followup@FreeBSD.org, jhein@symmetricom.com
>>> Cc: freebsd-python@freebsd.org
>>> Subject: Re: ports/175276: [patch] devel/py-gobject OPTIONSFILE eval
>>> order
>>> problem
>>> Date: Sat, 23 Feb 2013 12:23:35 -0500
>>>
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA1
>>>
>>> 1. Should this be assigned to freebsd-python@?
>>>
>>> I realize that freebsd-gnome@ is the maintainer, but the root cause
>>> lies with the way Python ports use PKGNAMEPREFIX, and this is not the
>>> only affected port.
>>>
>>>
>>> 2. Allow me to elaborate on the originator's description, for those
>>> interested in the analysis.
>>>
>>> The common use of
>>>
>>> PKGNAMEPREFIX=${PYTHON_PKGNAMEPREFIX}
>>>
>>> depends on lazy evaluation, because the right-hand side is not defined
>>> until the "pre" section of bsd.python.mk. Relatively early on in
>>> bsd.port.mk, we get a default definition for UNIQUENAME based on
>>> PKGNAMEPREFIX, unless LATEST_LINK is already defined, which doesn't
>>> ordinarily happen until the "post" section of bsd.port.mk. Shortly
>>> after that, between the "options" section and the "pre" section of
>>> bsd.port.mk, we include bsd.options.mk which provides a default
>>> definition of OPTIONSFILE, based on UNIQUENAME. At that point in
>>> bsd.options.mk, we haven't yet included bsd.python.mk, so
>>> PYTHON_PKGNAMEPREFIX is undefined. That means that when make reads the
>>> saved options (inside the first pass through bsd.options.mk) thereby
>>> triggering evaluation of OPTIONSFILE, it is as if we hadn't set
>>> PKGNAMEPREFIX at all.
>>>
>>> As the originator points out, the do-config target, where make
>>> performs the work of writing saved options, re-evaluates OPTIONSFILE
>>> after bsd.python.mk sets PYTHON_PKGNAMEPREFIX, because do-config is
>>> defined in the "post" section of bsd.port.mk.
>>>
>>>
>>> 3. What ports are affected?
>>>
>>> Any port that sets PKGNAMEPREFIX equal to a make variable that is not
>>> defined until the "pre" section or later, and fails to work-around the
>>> staggered evaluation by defining one of UNIQUENAME, LATEST_LINK, or
>>> OPTIONSFILE, is broken. It turns out that Python ports are
>>> disproportionately affected, but mainly because Python ports are heavy
>>> users of PKGNAMEPREFIX. The other PKGNAMEPREFIXs are:
>>>
>>> % egrep "^[A-Z_]+_PKGNAMEPREFIX" /usr/ports/Mk/* -h
>>> APACHE_PKGNAMEPREFIX= ap${APACHE_VERSION}-
>>> PYTHON_PKGNAMEPREFIX?= py*-
>>> LUA_PKGNAMEPREFIX?= lua${LUA_VER_STR}-
>>> PYTHON_PKGNAMEPREFIX= py${PYTHON_SUFFIX}-
>>> RUBY_PKGNAMEPREFIX?= ruby${RUBY_SUFFIX}-
>>>
>>> But the distribution among these is heavily skewed toward Python.
>>>
>>> % find /usr/ports -depth 3 -type f -name Makefile \
>>> | xargs egrep "^OPTIONS_DEFINE" -l \
>>> | xargs egrep "^(OPTIONSFILE|UNIQUENAME|LATEST_LINK)" -L \
>>> | xargs egrep '^PKGNAMEPREFIX=.*\$' -h \
>>> | sed -e "s/[ ]//g" \
>>> | sort | uniq -c | sort -n
>>> 1 PKGNAMEPREFIX=${APACHE_PKGNAMEPREFIX}
>>> 1 PKGNAMEPREFIX=${DMPKGNAMEPREFIX}
>>> 1 PKGNAMEPREFIX=${DN3DPKGNAMEPREFIX}
>>> 1 PKGNAMEPREFIX=${TGTARCH}-${TGTABI}-
>>> 1 PKGNAMEPREFIX=php${PHP_VER}-
>>> 2 PKGNAMEPREFIX=${LANG_PKGNAME}-
>>> 22 PKGNAMEPREFIX=${PYTHON_PKGNAMEPREFIX}
>>>
>>> (That's supposed to be a tab and a space in the sed command, by the
>>> way.)
>>>
>>> So, let's focus on the 22 ports at the end.
>>>
>>> % find /usr/ports -depth 3 -type f -name Makefile \
>>> | xargs egrep "^OPTIONS_DEFINE" -l \
>>> | xargs egrep "^(OPTIONSFILE|UNIQUENAME|LATEST_LINK)" -L \
>>> | xargs egrep '^PKGNAMEPREFIX=.*PYTHON' -l \
>>> | cut -d/ -f4-5 | sort
>>> astro/py-RO
>>> audio/py-karaoke
>>> audio/py-pyaudio
>>> databases/py-sqlkit
>>> devel/py-bison
>>> devel/py-gobject
>>> devel/py-hgsubversion
>>> dns/ldns
>>> graphics/py-PyX
>>> graphics/py-gdal
>>> mail/py-spf
>>> math/py-sympy
>>> net/py-medusa
>>> security/arm
>>> security/py-volatility
>>> security/py-yara-editor
>>> www/py-django_compressor
>>> www/py-imdbpy
>>> www/py-qp
>>> www/py-qpy
>>> www/py-rhodecode
>>> www/py-satchmo
>>>
>>> I've checked every one of these by running
>>>
>>> make config-conditional
>>>
>>> twice in a row, and every one of them gave me a dialog the second
>>> time, which implies that they are reading and writing saved options in
>>> different places.
>>>
>>>
>>> 4. How can we fix this?
>>>
>>> As I see it, there are at least the following alternatives.
>>>
>>> A) Require each maintainer to choose and implement their preferred
>>> work-around, defining one or more of UNIQUENAME, LATEST_LINK, or
>>> OPTIONSFILE. This is what most affected ports do already, and what the
>>> originator is proposing for this port. 100 ports use
>>> PYTHON_PKGNAMEPREFIX and OPTIONS_DEFINE. 74 of those set OPTIONSFILE,
>>> 3 set LATEST_LINK, and 1 sets UNIQUENAME.
>>>
>>> In this case we should update the documentation in bsd.python.mk and
>>> the Porter's Handbook to make the requirement clear, and consider
>>> implementing a validation check somewhere in /usr/ports/Mk and/or
>>> portlint.
>>>
>>> B) Cause part or all of the "pre" section of bsd.python.mk to be
>>> processed earlier in bsd.port.mk, so that PYTHON_PKGNAMEPREFIX is
>>> defined by the time we hit bsd.options.mk and need OPTIONSFILE for
>>> reading. This would require additional analysis and testing to prevent
>>> collateral breakage, and it would mean that bsd.python.mk becomes a
>>> special case.
>>>
>>> I've skimmed the portion of bsd.python.mk prior to the definition of
>>> PYTHON_PKGNAMEPREFIX, and nothing major jumps out. If there is
>>> interest, I would be glad to prepare a patch at which to throw darts.
>>>
>>> C) Redefine OPTIONSFILE inside bsd.python.mk upon detecting that it
>>> changes after defining PYTHON_PKGNAMEPREFIX, so that OPTIONSFILE is
>>> the same when reading and writing saved options. I think we could do
>>> this without affecting bsd.port.mk, or the ports that have already
>>> implemented a workaround. It would mean that the default behavior when
>>> using PYTHON_PKGNAMEPREFIX is to put saved options in
>>> ${PORT_DBDIR}/${_PNP}${PORTNAME}/options, where _PNP is any literal
>>> used along with PYTHON_PKGNAMEPREFIX in the port's Makefile, which
>>> some might consider a POLA violation. On the other hand, doing one
>>> thing consistently that works is better than doing something that
>>> breaks your port unexpectedly.
>>>
>>> The big problem with this alternative is that PORTNAME by itself is
>>> nowhere near unique enough to avoid conflict with other ports, and
>>> would pretty much require bubbling up the definition of
>>> PYTHON_PKGNAMEPREFIX from bsd.python.mk to each affected port.
>>>
>>>
>>> 5. What is the best alternative?
>>>
>>> I find B very attractive because it frees maintainers from defining an
>>> extra variable if they don't want (i.e. good defaults), but still
>>> allows them to do so if they do want (i.e. good mechanism). It may be
>>> difficult, hackish, and error-prone though.
>>>
>>> Option A would be easiest, and least traumatic both to individual
>>> ports and to the ports infrastructure itself. For this reason alone, A
>>> is probably the right choice for now.
>>>
>>> Sadly, C may be a complete non-starter due to the uniqueness problem,
>>> but I wouldn't rule it out completely as a long-term follow-up to A.
>>> The way I see it working out is in three phases. Phase one is to
>>> implement option A but also invite maintainers to replace
>>>
>>> PKGNAMEPREFIX= ${PYTHON_PKGNAMEPREFIX}
>>>
>>> with
>>>
>>> PKGNAMEPREFIX= py${PYTHON_SUFFIX}-
>>>
>>> Phase two is to implement option C. Phase three is to invite
>>> maintainers to remove the option A work-around if they like the
>>> then-default behavior.
>>>
>>>
>>> 6. Conclusion.
>>>
>>> I invite commentary and criticism, especially on the potential
>>> resolutions I proposed. When we reach consensus, I will set about
>>> preparing some patches, if need be, and seeking the help of a friendly
>>> committer.
>>>
>>> Thank you for your kind indulgence.
>>>
>>> Cheers,
>>> John
>>> -----BEGIN PGP SIGNATURE-----
>>> Version: GnuPG v1.4.11 (GNU/Linux)
>>> Comment: Using GnuPG with undefined - http://www.enigmail.net/
>>>
>>> iQEcBAEBAgAGBQJRKPsXAAoJEEdKvTwaez9w6yEIALFz+xrYLMdR1AhcPE2jEBd6
>>> uR4dOZye8PQFTHbvhA/t20NFTroalr2kXF49+PTqR6kCFes+vNgjIlWUdKsIngYk
>>> y5x32f60Bd/TtqPo6M2aeOE/M322U6cIH5jJhh3EBTEpm+Upd9enIetxR0NpjTnP
>>> G+6yf8e7P4oBaYGSk01i3pah00OR2YeC87rtcEdgs1sM94PjxbXZGcuA+K9UbgVQ
>>> 2WB8Z4IvrD3d2UqRnC8TRq1/bZyiPSHKNeMFBRJZ4gFe/wr9G0txDnH1LTy/q0Gq
>>> kVHvdbApLYytMX/VmMMgDRnbzGS/kDMvIED8dJnwWf9pMLmzsi0pcVX/vH0m1Vw=
>>> =q6eG
>>> -----END PGP SIGNATURE-----
>>> _______________________________________________
>>> freebsd-gnome@freebsd.org mailing list
>>> http://lists.freebsd.org/mailman/listinfo/freebsd-gnome
>>> To unsubscribe, send any mail to "freebsd-gnome-unsubscribe@freebsd.org"
>>
>>
>>
>
--
mezz.freebsd@gmail.com - mezz@FreeBSD.org
FreeBSD GNOME Team
http://www.FreeBSD.org/gnome/ - gnome@FreeBSD.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201303191810.r2JIA1d3072127>