From owner-freebsd-python@FreeBSD.ORG Tue Jul 30 09:47:53 2013 Return-Path: <owner-freebsd-python@FreeBSD.ORG> Delivered-To: freebsd-python@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 1BF2DF70; Tue, 30 Jul 2013 09:47:53 +0000 (UTC) (envelope-from mva@freebsd.org) Received: from smtprelay05.ispgateway.de (smtprelay05.ispgateway.de [80.67.31.97]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 491A4223C; Tue, 30 Jul 2013 09:47:52 +0000 (UTC) Received: from [80.67.16.121] (helo=webmailfront01.ispgateway.de) by smtprelay05.ispgateway.de with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.68) (envelope-from <mva@freebsd.org>) id 1V46Wt-000604-G0; Tue, 30 Jul 2013 11:47:43 +0200 Received: from his1.his.de (his1.his.de [192.124.237.237]) by webmail.df.eu (Horde Framework) with HTTP; Tue, 30 Jul 2013 11:47:43 +0200 Date: Tue, 30 Jul 2013 11:47:43 +0200 Message-ID: <20130730114743.Horde.XOo8rQeBW_7C8Wx-seaAuw1@webmail.df.eu> From: Marcus von Appen <mva@freebsd.org> To: David Demelier <demelier.david@gmail.com> Subject: Re: Dealing with 2.7 and 3.3 installations References: <CAO+PfDdfdhfnyTi0sJzvbJA0GLnOB=3dNme0gn6uhnenzUcjaQ@mail.gmail.com> <20130730095456.Horde.XlxQMdj5YU1hIxf6WAYZOQ2@webmail.df.eu> <CAO+PfDdpOw8FONKaze0_CMYET7eHrfWPzZd1-K0FwBVvnJ8jpg@mail.gmail.com> In-Reply-To: <CAO+PfDdpOw8FONKaze0_CMYET7eHrfWPzZd1-K0FwBVvnJ8jpg@mail.gmail.com> User-Agent: Internet Messaging Program (IMP) H5 (6.0.4) Content-Type: text/plain; charset=UTF-8; format=flowed; DelSp=Yes MIME-Version: 1.0 Content-Disposition: inline X-Df-Sender: ZnJlZWJzZEBzeXNmYXVsdC5vcmc= Cc: Daniel Braniss <danny@cs.huji.ac.il>, bapt@freebsd.org, koobs@freebsd.org, freebsd-python@freebsd.org X-BeenThere: freebsd-python@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: mva@freebsd.org List-Id: FreeBSD-specific Python issues <freebsd-python.freebsd.org> List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-python>, <mailto:freebsd-python-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-python> List-Post: <mailto:freebsd-python@freebsd.org> List-Help: <mailto:freebsd-python-request@freebsd.org?subject=help> List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-python>, <mailto:freebsd-python-request@freebsd.org?subject=subscribe> X-List-Received-Date: Tue, 30 Jul 2013 09:47:53 -0000 David Demelier <demelier.david@gmail.com>: > 2013/7/30 Marcus von Appen <mva@freebsd.org>: >> David, Daniel, >> >> just to take this to python@ and to clarify, what problems are still open >> for you: >> >> - bapt confirmed that the parallel installation of python itself from binary >> packages should work >> without any problem >> - Installation from the ports tree should work without problems >> - Installation of python modules from the ports tree basically works without >> problems (given that >> you set the correct make knobs), BUT can create tons of issues, when it >> comes to deinstallation, >> port dependency detection, etc. >> >> As per my understanding, you both mentioned issues with both, python itself >> and 3rd party packages. >> As said above, python itself should not be any problem. If you ran into any >> issue, please let python@ >> know about it (ideally with a reproducible step-by-step description) or >> create a PR. >> >> Regarding the 3rd party modules, I am not sure about the pkg status. Maybe >> bapt can say a word or >> two about whether packages are (planned to or currently) built for different >> python versions or not. >> David, Daniel: if you are currently running into trouble with the 3rd party >> modules, please >> let us know, what your requirements are, so we can discuss different >> solutions top be applied to the >> ports tree and, where applicable, improved support for the pkg builds. >> >> Thanks and cheers >> Marcus >> > > The problem is not directly related to python. Let me explain with > details how we bulk build for both python 2.7 and 3.3 using the > poudriere infrastructure. > > As you know, to build devel/py-* for python 2.7 and for 3.3, you will > need to call make install and set PYTHON_DEFAULT_VERSION to the > correct one. For us, if I plan to build all ports as packages, the > poudriere script will only build devel/py-foobar for the > PYTHON_DEFAULT_VERSION set, which is still 2.7. > > To permit other users to use both 2.7 and 3.3 modules, we need to > create two jails which each build devel/py-foobar one with > PYTHON_DEFAULT_VERSION set to python 2.7 and the other one to > python3.3. > > And, because these modules requires lang/python, each of these jail > will build the corresponding interpreter and including the symlink to > /usr/local/bin/python. > > So we end with the following: > > [Jail python 2.7] > lang/python27 built with /usr/local/bin/python > > py27-foobar > py27-otherport > ... > > [Jail python 3.3] > lang/python33 built with /usr/local/bin/python too > > py33-foobar > py33-otherport > > And then, thanks to the support of multiple repositories with pkgng. > We are able to install py33-foobar and py27-foobar. But, the python > 3.3 interpreter will not be installed because it also wants to install > the symlink. > > Bapt successfully installed both interpreter because they were > compiled in the same jail, thus, only one of the both have the > IF_DEFAULT conditional evaluated. > > Now I agree that the problem is not related to python itself and more > to pkgng. But if we install the /usr/local/bin/python symlink *only* > if it does not currently exist, we will not being confused of from > which repository should I install python 3.3. > > That's why I proposed to avoid the use of that conditional IF_DEFAULT > and use a test-symlink before install so that even if python 3.3 is > built within more than one repositories, it will never conflicts with > other interpreters. Understood. We need to make sure, that ports will pull in lang/pythonXX as BUILD/RUN dependency rather than lang/python (if that's the case at the moment - that needs to be checked). However, many modules or applications requiring python ask for "python" as interpreter, ignoring a specific version, so we might need to pull in lang/python as RUN dependency for several applications in some way. I would not want to patch hundreds of ports to be explicit on the python interpreter, but that should not be a big problem at all. Applications that work with either python version, do not care about what /usr/local/bin/python links to. Applications that care about the python version, will have to set USE_PYTHON=XX explicitly (if they did not so far, that's an error). As per my understanding (coming back to your poudriere issue), getting rid of the IF_DEFAULT magic and moving everything of relevance into lang/python should solve nearly all of the current problems. In my opinion, such a change should also be safe in regards to the ports infrastructure and compatibility, so it would be a good place to start sorting out those issues. Cheers Marcus