Date: Mon, 12 May 2014 23:05:56 -0400 From: "John W. O'Brien" <john@saltant.com> To: Irjohn Junus <i.junus@gmail.com> Cc: python@freebsd.org, Jimmy Olgeni <olgeni@freebsd.org> Subject: math/py-numpy cannot satisfy libgfortran dependency on libgcc_s/GCC_4.6.0 [WAS: Re: py27-pandas-0.13.1_1 conflict with py27-MySQLdb-1.2.3_4] Message-ID: <53718C14.1040807@saltant.com> In-Reply-To: <CALXkR%2B_r4kG8CHO8R2MOsGHK4JDeEMa28MN4rzZ1hAtd%2BzAu4g@mail.gmail.com> References: <CALXkR%2B8ivP3m2aJN%2Bm88Sz%2B4BodAxnYioBvgUotFZf_-TVPLgg@mail.gmail.com> <536C40B0.2060705@saltant.com> <CALXkR%2B_r4kG8CHO8R2MOsGHK4JDeEMa28MN4rzZ1hAtd%2BzAu4g@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --OJHnpXUV1METTqspoUOTie7LGatUvageN Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 5/8/14 11:33 PM, Irjohn Junus wrote: > Hi John, >=20 > No worries and yes I'm still having the problem. >=20 > I remember seeing this message: > =3D=3D=3D>>> pkg-message for gcc-4.7.3_1 > To ensure binaries built with this toolchain find appropriate versions > of the necessary run-time libraries, you may want to link using >=20 > -Wl,-rpath=3D/usr/local/lib/gcc47 >=20 > after upgrading gcc port a few days ago, which is described in your lin= k > [0]. Do I issue this command as root? Need advice here, thanks. >=20 > Regards, > Irjohn >=20 >=20 > On Fri, May 9, 2014 at 10:42 AM, John W. O'Brien <john@saltant.com > <mailto:john@saltant.com>> wrote: >=20 > On 4/18/14 8:45 AM, Irjohn Junus wrote: > > Dear All, > > > > I'm running 9.1-RELEASE and today just after freebsd-update to p1= 1 and > > updating ports to the latest found out that pandas won't import a= fter > > MySQLdb is imported first. Strangely, it will import no problem i= f > > MySQLdb is imported after: >=20 > Hi Irjohn, >=20 > Sorry for the long delay. Are you still having this problem? >=20 > >>>> import MySQLdb > >>>> import pandas as pd > > /lib/libgcc_s.so.1: version GCC_4.6.0 required by > > /usr/local/lib/gcc47/libgfortran.so.3 not found > [...] > > Traceback (most recent call last): > [...] > > File > "/usr/local/lib/python2.7/site-packages/numpy/linalg/linalg.py", > > line 29, in <module> > > from numpy.linalg import lapack_lite, _umath_linalg > > ImportError: /lib/libgcc_s.so.1: version GCC_4.6.0 required by > > /usr/local/lib/gcc47/libgfortran.so.3 not found >=20 > It looks like this thread [0], these commits [1] [2], and this PR [= 3] > are related. >=20 > [0] > https://lists.freebsd.org/pipermail/freebsd-toolchain/2014-April/00= 1149.html >=20 > [1] https://svnweb.freebsd.org/ports?view=3Drevision&revision=3D351= 167 >=20 > [2] https://svnweb.freebsd.org/ports?view=3Drevision&revision=3D351= 159 >=20 > [3] https://www.freebsd.org/cgi/query-pr.cgi?pr=3D185902 Irjohn, This is just a little out of my depth, so I'm going to talk through it out loud while I investigate. Presumably somebody more knowledgeable than I will step in to correct me if I get something wrong. First off, there are a few open PRs ([0], [1]) on math/py-numpy, both of which seem to have been triggered when ports started using GCC 4.7 by default [2]. What I know from reading gcc(1) is that "-Wl,<option>" passes <option> to the linker, and from ld(1) is that "-rpath=3D<path>" embeds <path> in the output file so that the runtime linker knows where to look for shared objects. It makes sense why this would help resolve the traceback you reported because one thing (linalg.py + ??? + libgfortran.so.3) is looking for another thing (libgcc_s.so.1), finds it, yet remains unsatisfied. Setting the rpath in the right place in the former will help it find the right one of the latter. I dug a little more to learn how to find out what rpath is embedded in a given file. It turns out that readelf(1) with the "-d" option to show an ELF file's dynamic section is just the ticket. Here is some output from my system. $ cd /usr/local/lib/gcc47 $ readelf -d libgfortran.so.3 | grep RPATH 0x000000000000000f (RPATH) Library rpath: [/usr/local/lib/gcc47] $ cd /usr/local/lib/python2.7/site-packages/numpy/linalg $ readelf -d {_umath_linalg.so,lapack_lite.so} | grep RPATH 0x000000000000000f (RPATH) Library rpath: [/usr/local/lib/gcc47] 0x000000000000000f (RPATH) Library rpath: [/usr/local/lib/gcc47] In fact, this tool will also tell you which shared objects a file needs. $ readelf -d {_umath_linalg.so,lapack_lite.so} | grep libgfortran 0x0000000000000001 (NEEDED) Shared library: [libgfortran.so.3] 0x0000000000000001 (NEEDED) Shared library: [libgfortran.so.3] So this line of python >>> from numpy.linalg import lapack_lite, _umath_linalg tries to load two binary shared objects, both of which require libgfortra= n. Moving right along. If everything goes as it should, how is numpy supposed to learn the correct rpath? Time to dive into math/py-numpy/Makefile and the ports machinery. In math/py-numpy, I find "USES=3Dfortran" and some fishy business in the pre-configure target involving GCCLIBDIR. The former is pretty interesting because it pulls in /usr/ports/Mk/Uses/fortran.mk to set all kinds of useful things, including FFLAGS+=3D -Wl,-rpath=3D${LOCALBASE}/lib/gcc${_GCC_VER} FCFLAGS+=3D -Wl,-rpath=3D${LOCALBASE}/lib/gcc${_GCC_VER} LDFLAGS+=3D -Wl,-rpath=3D${LOCALBASE}/lib/gcc${_GCC_VER} \ -L${LOCALBASE}/lib/gcc${_GCC_VER} -B${LOCALBASE}/bin It also sets FC=3Dgfortran${_GCC_VER}; a fact I can use to further probe the GCCLIBDIR funny business in the numpy port. During the pre-configure target, ports determines GCCLIBDIR like this: $ gfortran47 -print-file-name=3Dlibgfortran.so \ | sed -e s/libgfortran.so// /usr/local/lib/gcc47/gcc/x86_64-portbld-freebsd9.2/4.7.3/../../../ Which simplifies, of course, to /usr/local/lib/gcc47. So far, so good. I've built math/py-numpy on my machine, and will begin to examine the build logs [3] for clues about what to dig into next (e.g. differences between my setting CC=3Dclang et al, and not). In the mean time... Questions to you: 1. What is the rpath stored in your copy of those numpy files? That is, what is the output from cd /usr/local/lib/python2.7/site-packages/numpy/linalg readelf -d _umath_linalg.so | grep RPATH readelf -d lapack_lite.so | grep RPATH ? 2. What is the contents of /usr/local/lib/python2.7/site-packages/numpy/distutils/site.cfg on your machine? 2. What is the contents of /etc/make.conf on your machine? 3. How to you install and upgrade your ports? Build from source by hand? Build from source with the help of portmaster, poudriere, etc? Install binary packages with pkg_* or ports-mgmt/pkg (pkgng)? One thing that still puzzles me is what databases/py-MySQLdb would have to do with it. [0] ports/188114: math/py-numpy is broken on 9.2-STABLE http://www.freebsd.org/cgi/query-pr.cgi?pr=3Dports/188114 [1] ports/188327: math/py-numpy: Numpy is broken http://www.freebsd.org/cgi/query-pr.cgi?pr=3Dports/188327 [2] https://svnweb.freebsd.org/ports?view=3Drevision&revision=3D347809 [3] https://pkg.saltant.net/poudriere/bulk/92amd64-current/2014-05-12_21h56m0= 1s/logs/py27-numpy-1.8.0_1,1.log Regards, John --OJHnpXUV1METTqspoUOTie7LGatUvageN Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) iQEcBAEBCgAGBQJTcYwYAAoJEORay8JGGICYo00IANjAEmXtv/dMIJrZTwjBV2QZ f8cWuUpII18DIFgo94cjSi6Ak3sxqwiVvuBdXnL0CeoNwbHmTQdG5oIHN97TziZx cf3qEN58UKdg2JyZ0anuJ+wMR2/EMhdlw4XVKrsSScamgtXtWHa5Xn8BgutWWOGP 2zHaH22kMp1pLGQgvVROO1cSZkBmmIK7JEfwTktSC9w8LxfPSxjLiG2nt2MHlVsS xfiSX2tu/bIlxA2QorAjz5rXX7bWj5TJYANCEkJjLryy/1aFnmMkSpKrgcg9wP8E TjSRY3wdnUAqJh6XIMI9qsZIJWrPmBYEpyg3gc5GWcQe1PoIwQOE60KiWr1fUxw= =VSnp -----END PGP SIGNATURE----- --OJHnpXUV1METTqspoUOTie7LGatUvageN--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?53718C14.1040807>