Date: Sun, 18 May 2014 12:30:23 +0800 From: Irjohn Junus <i.junus@gmail.com> To: "John W. O'Brien" <john@saltant.com> Cc: python@freebsd.org, Jimmy Olgeni <olgeni@freebsd.org> Subject: Re: math/py-numpy cannot satisfy libgfortran dependency on libgcc_s/GCC_4.6.0 [WAS: Re: py27-pandas-0.13.1_1 conflict with py27-MySQLdb-1.2.3_4] Message-ID: <CALXkR%2B-6_2k4pD7-Ln6e-PbgFTKriRXSrqOpd=UVbwppi=dpvg@mail.gmail.com> In-Reply-To: <53718C14.1040807@saltant.com> References: <CALXkR%2B8ivP3m2aJN%2Bm88Sz%2B4BodAxnYioBvgUotFZf_-TVPLgg@mail.gmail.com> <536C40B0.2060705@saltant.com> <CALXkR%2B_r4kG8CHO8R2MOsGHK4JDeEMa28MN4rzZ1hAtd%2BzAu4g@mail.gmail.com> <53718C14.1040807@saltant.com>
next in thread | previous in thread | raw e-mail | index | archive | help
John, Thank you for looking at this. Apologies for my late reply it's been crazy busy at work lately. Here's answers to your questions: 1. readelf -d _umath_linalg.so | grep RPATH 0x000000000000000f (RPATH) Library rpath: [/usr/local/lib/gcc47] readelf -d lapack_lite.so | grep RPATH 0x000000000000000f (RPATH) Library rpath: [/usr/local/lib/gcc47] 2. cat /usr/local/lib/python2.7/site-packages/numpy/distutils/site.cfg [DEFAULT] lapack_type=atlas library_dirs = /usr/lib:/usr/local/lib:/usr/local/lib/gcc47/gcc/x86_64-portbld-freebsd9.1/4.7.3/../../../ include_dirs = /usr/include:/usr/local/include:/usr/local/include/suitesparse src_dirs = /usr/local/src # search static libraries (.a) in preference to shared ones (.so) search_static_first = 0 [atlas] library_dirs = /usr/local/lib:/usr/local/lib/gcc47/gcc/x86_64-portbld-freebsd9.1/4.7.3/../../../ atlas_libs = lapack, blas 3. cat /etc/make.conf # added by use.perl 2013-03-25 18:12:09 PERL_VERSION=5.12.4 #WITH_MPM=worker 4. I use portmaster exclusively, to install: portmaster --packages-build --delete-build-only port-name to upgrade: portmaster --dwv port-name Thanks, Irjohn On Tue, May 13, 2014 at 11:05 AM, John W. O'Brien <john@saltant.com> wrote: > On 5/8/14 11:33 PM, Irjohn Junus wrote: > > Hi John, > > > > No worries and yes I'm still having the problem. > > > > I remember seeing this message: > > ===>>> pkg-message for gcc-4.7.3_1 > > To ensure binaries built with this toolchain find appropriate versions > > of the necessary run-time libraries, you may want to link using > > > > -Wl,-rpath=/usr/local/lib/gcc47 > > > > after upgrading gcc port a few days ago, which is described in your link > > [0]. Do I issue this command as root? Need advice here, thanks. > > > > Regards, > > Irjohn > > > > > > On Fri, May 9, 2014 at 10:42 AM, John W. O'Brien <john@saltant.com > > <mailto:john@saltant.com>> wrote: > > > > On 4/18/14 8:45 AM, Irjohn Junus wrote: > > > Dear All, > > > > > > I'm running 9.1-RELEASE and today just after freebsd-update to p11 > and > > > updating ports to the latest found out that pandas won't import > after > > > MySQLdb is imported first. Strangely, it will import no problem if > > > MySQLdb is imported after: > > > > Hi Irjohn, > > > > Sorry for the long delay. Are you still having this problem? > > > > >>>> import MySQLdb > > >>>> import pandas as pd > > > /lib/libgcc_s.so.1: version GCC_4.6.0 required by > > > /usr/local/lib/gcc47/libgfortran.so.3 not found > > [...] > > > Traceback (most recent call last): > > [...] > > > File > > "/usr/local/lib/python2.7/site-packages/numpy/linalg/linalg.py", > > > line 29, in <module> > > > from numpy.linalg import lapack_lite, _umath_linalg > > > ImportError: /lib/libgcc_s.so.1: version GCC_4.6.0 required by > > > /usr/local/lib/gcc47/libgfortran.so.3 not found > > > > It looks like this thread [0], these commits [1] [2], and this PR [3] > > are related. > > > > [0] > > > https://lists.freebsd.org/pipermail/freebsd-toolchain/2014-April/001149.html > > > > [1] https://svnweb.freebsd.org/ports?view=revision&revision=351167 > > > > [2] https://svnweb.freebsd.org/ports?view=revision&revision=351159 > > > > [3] https://www.freebsd.org/cgi/query-pr.cgi?pr=185902 > > Irjohn, > > This is just a little out of my depth, so I'm going to talk through it > out loud while I investigate. Presumably somebody more knowledgeable > than I will step in to correct me if I get something wrong. > > First off, there are a few open PRs ([0], [1]) on math/py-numpy, both of > which seem to have been triggered when ports started using GCC 4.7 by > default [2]. > > What I know from reading gcc(1) is that "-Wl,<option>" passes <option> > to the linker, and from ld(1) is that "-rpath=<path>" embeds <path> in > the output file so that the runtime linker knows where to look for > shared objects. It makes sense why this would help resolve the traceback > you reported because one thing (linalg.py + ??? + libgfortran.so.3) is > looking for another thing (libgcc_s.so.1), finds it, yet remains > unsatisfied. Setting the rpath in the right place in the former will > help it find the right one of the latter. > > I dug a little more to learn how to find out what rpath is embedded in a > given file. It turns out that readelf(1) with the "-d" option to show an > ELF file's dynamic section is just the ticket. Here is some output from > my system. > > $ cd /usr/local/lib/gcc47 > $ readelf -d libgfortran.so.3 | grep RPATH > 0x000000000000000f (RPATH) Library rpath: > [/usr/local/lib/gcc47] > $ cd /usr/local/lib/python2.7/site-packages/numpy/linalg > $ readelf -d {_umath_linalg.so,lapack_lite.so} | grep RPATH > 0x000000000000000f (RPATH) Library rpath: > [/usr/local/lib/gcc47] > 0x000000000000000f (RPATH) Library rpath: > [/usr/local/lib/gcc47] > > In fact, this tool will also tell you which shared objects a file needs. > > $ readelf -d {_umath_linalg.so,lapack_lite.so} | grep libgfortran > 0x0000000000000001 (NEEDED) Shared library: > [libgfortran.so.3] > 0x0000000000000001 (NEEDED) Shared library: > [libgfortran.so.3] > > So this line of python > > >>> from numpy.linalg import lapack_lite, _umath_linalg > > tries to load two binary shared objects, both of which require libgfortran. > > Moving right along. If everything goes as it should, how is numpy > supposed to learn the correct rpath? Time to dive into > math/py-numpy/Makefile and the ports machinery. > > In math/py-numpy, I find "USES=fortran" and some fishy business in the > pre-configure target involving GCCLIBDIR. The former is pretty > interesting because it pulls in /usr/ports/Mk/Uses/fortran.mk to set all > kinds of useful things, including > > FFLAGS+= -Wl,-rpath=${LOCALBASE}/lib/gcc${_GCC_VER} > FCFLAGS+= -Wl,-rpath=${LOCALBASE}/lib/gcc${_GCC_VER} > LDFLAGS+= -Wl,-rpath=${LOCALBASE}/lib/gcc${_GCC_VER} \ > -L${LOCALBASE}/lib/gcc${_GCC_VER} -B${LOCALBASE}/bin > > It also sets FC=gfortran${_GCC_VER}; a fact I can use to further probe > the GCCLIBDIR funny business in the numpy port. During the pre-configure > target, ports determines GCCLIBDIR like this: > > $ gfortran47 -print-file-name=libgfortran.so \ > | sed -e s/libgfortran.so// > /usr/local/lib/gcc47/gcc/x86_64-portbld-freebsd9.2/4.7.3/../../../ > > Which simplifies, of course, to /usr/local/lib/gcc47. So far, so good. > > I've built math/py-numpy on my machine, and will begin to examine the > build logs [3] for clues about what to dig into next (e.g. differences > between my setting CC=clang et al, and not). In the mean time... > > Questions to you: > > 1. What is the rpath stored in your copy of those numpy files? That > is, what is the output from > > cd /usr/local/lib/python2.7/site-packages/numpy/linalg > readelf -d _umath_linalg.so | grep RPATH > readelf -d lapack_lite.so | grep RPATH > > ? > > 2. What is the contents of > /usr/local/lib/python2.7/site-packages/numpy/distutils/site.cfg on your > machine? > > 2. What is the contents of /etc/make.conf on your machine? > > 3. How to you install and upgrade your ports? Build from source by > hand? Build from source with the help of portmaster, poudriere, etc? > Install binary packages with pkg_* or ports-mgmt/pkg (pkgng)? > > One thing that still puzzles me is what databases/py-MySQLdb would have > to do with it. > > [0] ports/188114: math/py-numpy is broken on 9.2-STABLE > http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/188114 > [1] ports/188327: math/py-numpy: Numpy is broken > http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/188327 > [2] https://svnweb.freebsd.org/ports?view=revision&revision=347809 > [3] > > https://pkg.saltant.net/poudriere/bulk/92amd64-current/2014-05-12_21h56m01s/logs/py27-numpy-1.8.0_1,1.log > > Regards, > John > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CALXkR%2B-6_2k4pD7-Ln6e-PbgFTKriRXSrqOpd=UVbwppi=dpvg>