From owner-freebsd-python@FreeBSD.ORG Fri Aug 23 19:19:51 2013 Return-Path: Delivered-To: freebsd-python@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 8F9E2ED5; Fri, 23 Aug 2013 19:19:51 +0000 (UTC) (envelope-from mva@FreeBSD.org) Received: from smtprelay02.ispgateway.de (smtprelay02.ispgateway.de [80.67.18.14]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 23A1C2D71; Fri, 23 Aug 2013 19:19:50 +0000 (UTC) Received: from [89.183.11.41] (helo=localhost) by smtprelay02.ispgateway.de with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.68) (envelope-from ) id 1VCwZm-0008Pf-8A; Fri, 23 Aug 2013 20:59:14 +0200 Date: Fri, 23 Aug 2013 20:59:13 +0200 From: Marcus von Appen To: freebsd-python@freebsd.org Subject: Bringing the python ports into shape Message-ID: <20130823185913.GA1266@medusa.sysfault.org> Mail-Followup-To: freebsd-python@freebsd.org, kwm@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="YiEDa0DAkWCtVeE4" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-Df-Sender: MTEyNTc0Mg== Cc: kwm@FreeBSD.org X-BeenThere: freebsd-python@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Marcus von Appen List-Id: FreeBSD-specific Python issues List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Aug 2013 19:19:51 -0000 --YiEDa0DAkWCtVeE4 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Dear all, the meta-ports are available and we are ready to move on to bring the python ports into shape for different python versions and (later on) different python implementations. There are a couple of issues (in order to be done) within the ports tree to sort out before moving on: - bsd.python.mk private variables are used in port Makefiles This especially relates to the usage of _PYTHON_VERSION within several ports and needs to be changed to PYTHON_VERSION. This is a cleanup task that should not have any impact on the ports. - USE_PYTHON vs. PYTHON_VERSION. There is no explicit need for a port to set PYTHON_VERSION to enforce a certain python version. If a port needs this, it should use USE_PYTHON= instead. PYTHON_VERSION shall still exist as readonly variable, but not be reassigned by port Makefiles. This is a cleanup task, which should lead to a more clear bsd.python.mk variable usage within the ports. - fix ports not to use `python` as command at build and in some cases run-time (see below for the problems with that) - Introduce support USE_PYTHON=2|3 If a port does not care about a specific Python minor release, it should pull in the meta ports wherever possible. This is only a cosmetic behaviour to avoid cross-dependency problems with ports asking for different python versions (see below for the problems behind the scenes) - enhance USE_PYTHON=yes to pull in lang/python and rely on whatever the user specified as default python version. As before, a cosmetic change to get rid of the current workaround that always pulls in lang/python. "The problems with that" / "The problems behind the scenes" Some ports ask for `python` at build or run-time. This is plainly wrong for about 99 percent of the ports and only worked in the past, because at least one python port installed itself as bin/python and noone changed the default python version with using a lot of ports at the same time. It is wrong insofar as it relies on the assumption that each port using `python` works with the current version that identifies itself as `python`. Let's assume to have a Python C extension installed (e.g. one of the million py-XXX ports installing .so files) for the current default version 2.7. The .so module links against libpython2.7.so, stuff is installed in lib/python2.7/site-packages, scripts however use #!/usr/bin/env python and rely on the .so module. Changing the default python to a different python version will cause the script to be unable to work anymore, for obvious ABI compatibility reasons. Thus, every port installing stuff into site-packages or linking against the python libraries _must_ use the explicit python version for scripts, binaries and whatnot. Some ports install scripts without actually using Python itself (e.g. x11-toolkits/gtk20). For those, it must be ensured that the scripts work for different python versions without modification or they have to be changed to use either a meta port python version (python2 or python3) or an explicit python version. Fixing ports using `python` is not as hard as it sounds, in my opinion. Thanks to Uses/shebangfix.mk and the possibility to override python_OLD_CMD and python_CMD in there, we can fix a huge set of ports simply by adding SHEBANG_FILES= to the port Makefile and python_CMD= ${PYTHON_VERSION} (or ${PYTHON_CMD}) to bsd.python.mk. This will leave us with a small set of ports that need a post-patch target or something similar to be fixed. The first two cleanup task(s) will start as soon as possible and hopefully will be done in a week or two. I'm CC'ing kwm@ within this mail, since he has a vivid interest on that topic (for getting gnome3 into the tree). Please keep him CC'd within this thread until he tells us not to. Cheers Marcus --YiEDa0DAkWCtVeE4 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.21 (FreeBSD) iEYEARECAAYFAlIXsQEACgkQi68/ErJnpkepPQCfWrXkTwsUc8eoAchI3YVMFw+P eAUAnRnKVbnBQFw4hqFyPMP8o6WDgEQj =nS7l -----END PGP SIGNATURE----- --YiEDa0DAkWCtVeE4--