From owner-freebsd-python@FreeBSD.ORG  Tue Jul 30 09:47:53 2013
Return-Path: <owner-freebsd-python@FreeBSD.ORG>
Delivered-To: freebsd-python@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTP id 1BF2DF70;
 Tue, 30 Jul 2013 09:47:53 +0000 (UTC) (envelope-from mva@freebsd.org)
Received: from smtprelay05.ispgateway.de (smtprelay05.ispgateway.de
 [80.67.31.97])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id 491A4223C;
 Tue, 30 Jul 2013 09:47:52 +0000 (UTC)
Received: from [80.67.16.121] (helo=webmailfront01.ispgateway.de)
 by smtprelay05.ispgateway.de with esmtpsa (TLSv1:AES256-SHA:256)
 (Exim 4.68) (envelope-from <mva@freebsd.org>)
 id 1V46Wt-000604-G0; Tue, 30 Jul 2013 11:47:43 +0200
Received: from his1.his.de (his1.his.de [192.124.237.237]) by webmail.df.eu
 (Horde Framework) with HTTP; Tue, 30 Jul 2013 11:47:43 +0200
Date: Tue, 30 Jul 2013 11:47:43 +0200
Message-ID: <20130730114743.Horde.XOo8rQeBW_7C8Wx-seaAuw1@webmail.df.eu>
From: Marcus von Appen <mva@freebsd.org>
To: David Demelier <demelier.david@gmail.com>
Subject: Re: Dealing with 2.7 and 3.3 installations
References: <CAO+PfDdfdhfnyTi0sJzvbJA0GLnOB=3dNme0gn6uhnenzUcjaQ@mail.gmail.com>
 <20130730095456.Horde.XlxQMdj5YU1hIxf6WAYZOQ2@webmail.df.eu>
 <CAO+PfDdpOw8FONKaze0_CMYET7eHrfWPzZd1-K0FwBVvnJ8jpg@mail.gmail.com>
In-Reply-To: <CAO+PfDdpOw8FONKaze0_CMYET7eHrfWPzZd1-K0FwBVvnJ8jpg@mail.gmail.com>
User-Agent: Internet Messaging Program (IMP) H5 (6.0.4)
Content-Type: text/plain; charset=UTF-8; format=flowed; DelSp=Yes
MIME-Version: 1.0
Content-Disposition: inline
X-Df-Sender: ZnJlZWJzZEBzeXNmYXVsdC5vcmc=
Cc: Daniel Braniss <danny@cs.huji.ac.il>, bapt@freebsd.org, koobs@freebsd.org,
 freebsd-python@freebsd.org
X-BeenThere: freebsd-python@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
Reply-To: mva@freebsd.org
List-Id: FreeBSD-specific Python issues <freebsd-python.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-python>,
 <mailto:freebsd-python-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-python>
List-Post: <mailto:freebsd-python@freebsd.org>
List-Help: <mailto:freebsd-python-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-python>,
 <mailto:freebsd-python-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 30 Jul 2013 09:47:53 -0000

David Demelier <demelier.david@gmail.com>:

> 2013/7/30 Marcus von Appen <mva@freebsd.org>:
>> David, Daniel,
>>
>> just to take this to python@ and to clarify, what problems are still open
>> for you:
>>
>> - bapt confirmed that the parallel installation of python itself from binary
>> packages should work
>>   without any problem
>> - Installation from the ports tree should work without problems
>> - Installation of python modules from the ports tree basically works without
>> problems (given that
>>   you set the correct make knobs), BUT can create tons of issues, when it
>> comes to deinstallation,
>>   port dependency detection, etc.
>>
>> As per my understanding, you both mentioned issues with both, python itself
>> and 3rd party packages.
>> As said above, python itself should not be any problem. If you ran into any
>> issue, please let python@
>> know about it (ideally with a reproducible step-by-step description) or
>> create a PR.
>>
>> Regarding the 3rd party modules, I am not sure about the pkg status. Maybe
>> bapt can say a word or
>> two about whether packages are (planned to or currently) built for different
>> python versions or not.
>> David, Daniel: if you are currently running into trouble with the 3rd party
>> modules, please
>> let us know, what your requirements are, so we can discuss different
>> solutions top be applied to the
>> ports tree and, where applicable, improved support for the pkg builds.
>>
>> Thanks and cheers
>> Marcus
>>
>
> The problem is not directly related to python. Let me explain with
> details how we bulk build for both python 2.7 and 3.3 using the
> poudriere infrastructure.
>
> As you know, to build devel/py-* for python 2.7 and for 3.3, you will
> need to call make install and set PYTHON_DEFAULT_VERSION to the
> correct one. For us, if I plan to build all ports as packages, the
> poudriere script will only build devel/py-foobar for the
> PYTHON_DEFAULT_VERSION set, which is still 2.7.
>
> To permit other users to use both 2.7 and 3.3 modules, we need to
> create two jails which each build devel/py-foobar one with
> PYTHON_DEFAULT_VERSION set to python 2.7 and the other one to
> python3.3.
>
> And, because these modules requires lang/python, each of these jail
> will build the corresponding interpreter and including the symlink to
> /usr/local/bin/python.
>
> So we end with the following:
>
> [Jail python 2.7]
> lang/python27 built with /usr/local/bin/python
>
> py27-foobar
> py27-otherport
> ...
>
> [Jail python 3.3]
> lang/python33 built with /usr/local/bin/python too
>
> py33-foobar
> py33-otherport
>
> And then, thanks to the support of multiple repositories with pkgng.
> We are able to install py33-foobar and py27-foobar. But, the python
> 3.3 interpreter will not be installed because it also wants to install
> the symlink.
>
> Bapt successfully installed both interpreter because they were
> compiled in the same jail, thus, only one of the both have the
> IF_DEFAULT conditional evaluated.
>
> Now I agree that the problem is not related to python itself and more
> to pkgng. But if we install the /usr/local/bin/python symlink *only*
> if it does not currently exist, we will not being confused of from
> which repository should I install python 3.3.
>
> That's why I proposed to avoid the use of that conditional IF_DEFAULT
> and use a test-symlink before install so that even if python 3.3 is
> built within more than one repositories, it will never conflicts with
> other interpreters.

Understood. We need to make sure, that ports will pull in lang/pythonXX
as BUILD/RUN dependency rather than lang/python (if that's the case at the
moment - that needs to be checked).

However, many modules or applications requiring python ask for "python" as
interpreter, ignoring a specific version, so we might need to pull in
lang/python as RUN dependency for several applications in some way.

I would not want to patch hundreds of ports to be explicit on the python
interpreter, but that should not be a big problem at all.
Applications that work with either python version, do not care about what
/usr/local/bin/python links to.
Applications that care about the python version, will have to set
USE_PYTHON=XX explicitly (if they did not so far, that's an error).

As per my understanding (coming back to your poudriere issue), getting rid
of the IF_DEFAULT magic and moving everything of relevance into lang/python
should solve nearly all of the current problems.

In my opinion, such a change should also be safe in regards to the ports
infrastructure and compatibility, so it would be a good place to start sorting
out those issues.

Cheers
Marcus