From owner-freebsd-questions@FreeBSD.ORG Tue Nov 10 07:13:21 2009 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 93A3B106566C for ; Tue, 10 Nov 2009 07:13:21 +0000 (UTC) (envelope-from maho.nakata@gmail.com) Received: from mail-yw0-f178.google.com (mail-yw0-f178.google.com [209.85.211.178]) by mx1.freebsd.org (Postfix) with ESMTP id 434568FC12 for ; Tue, 10 Nov 2009 07:13:20 +0000 (UTC) Received: by ywh8 with SMTP id 8so3443194ywh.3 for ; Mon, 09 Nov 2009 23:13:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:date:message-id:to:cc :subject:from:in-reply-to:references:x-mailer:mime-version :content-type:content-transfer-encoding; bh=jGgivHuMPIX7d9gST18/zaEeu+FfyWFSwf0XlLFcAfo=; b=eJOWDPYLdsJnhJdBsWwc5AL74OytCWjyIIDPIB5o1/WyS1xpuYyftWa+OriojlWJRy biOFwUeVSj9o3Vc613BKA/gR8DPljvBZO56O350Di3RCmFD8zMtIpHEMRT7P9Tgj2bXN r4gxdjZ05hJhThsTmTmnV7s5ZKCx8AGgVInwg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:date:message-id:to:cc:subject:from:in-reply-to:references :x-mailer:mime-version:content-type:content-transfer-encoding; b=S5gkzmQTKW1qeQIy7Y+nwSiItCryBPId6nP1hyDDP9qM+hmtpWSKHwdF3Iobo83u5B kQwIXRM1VuGhz3eJOJEpQ4iHAcdLDXr0peyxrnDC0DWqOop0Oxs5PIo4d4yqCwJev0tX br28AzkldcWaJL8pDpOXs+9eC8c1kKmLRlK1E= Received: by 10.91.54.8 with SMTP id g8mr16342233agk.83.1257835282239; Mon, 09 Nov 2009 22:41:22 -0800 (PST) Received: from localhost (rikad42.riken.jp [134.160.214.42]) by mx.google.com with ESMTPS id 23sm232228yxe.36.2009.11.09.22.41.19 (version=SSLv3 cipher=RC4-MD5); Mon, 09 Nov 2009 22:41:21 -0800 (PST) Sender: Maho NAKATA Date: Tue, 10 Nov 2009 15:41:05 +0900 (JST) Message-Id: <20091110.154105.769051484062721846.chat95@mac.com> To: bennett@cs.niu.edu From: Maho NAKATA In-Reply-To: <200911092107.nA9L7tHX029304@mp.cs.niu.edu> References: <200911092107.nA9L7tHX029304@mp.cs.niu.edu> X-Mailer: Mew version 6.2 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: bf1783@googlemail.com, freebsd-questions@freebsd.org Subject: Re: math/py-numpy vs. math/atlas-devel X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Nov 2009 07:13:21 -0000 Hi all, I'm willing to apply patches to atlas ports if available. I talked with Goto Kazushige (author of GotoBLAS), he told me that L2 cache handling on FreeBSD is very bad. It may be a reason why ATLAS build is so fragile. On Linux machines, it takes only 20 min or so. Thanks for good discussions! Best, Nakata Maho From: Scott Bennett Subject: Re: math/py-numpy vs. math/atlas-devel Date: Mon, 09 Nov 2009 15:07:55 -0600 (CST) > On Mon, 9 Nov 2009 20:26:15 +0000 "b. f." > wrote: >>On 11/9/09, Scott Bennett wrote: >>> On Sun, 08 Nov 2009 23:59:29 -0800 Doug Barton >>> wrote: >> >>... >> >>> Anyway, the math/py-numpy port now proceeds to build without bothering >>> with math/atlas. It quickly goes astray when it doesn't recognize that any >>> of gfortran4[345] has been installed--I guess there no longer is a FORTRAN >>> compiler included in the base system--and instead tries to use lang/g95, >>> which has also been installed. Of course, this won't work because the ATLAS >>> library needs to have been compiled with the same compiler as the programs >>> that use it. >>> >>> ===> Configuring for py26-numpy-1.3.0_2,1 >>> Running from numpy source directory. >>> [39mF2PY Version 2 [0m >>> [39mblas_opt_info: [0m >>> [39mblas_mkl_info: [0m >>> [39m libraries mkl,vml,guide not found in /usr/lib [0m >>> [39m libraries mkl,vml,guide not found in /usr/local/lib [0m >>> [39m libraries mkl,vml,guide not found in >>> /usr/local/lib/gcc44/gcc/i386-portbld-freebsd7.2/4.4.3/../../../ [0m >>> [39m NOT AVAILABLE [0m >>> [39m [0m >>> [39matlas_blas_threads_info: [0m >>> [39mSetting PTATLAS=ATLAS [0m >>> [39mSetting PTATLAS=ATLAS [0m >>> [39mSetting PTATLAS=ATLAS [0m >>> [39m FOUND: [0m >>> [39m libraries = ['alapack_r', 'f77blas_r', 'cblas_r', 'atlas_r'] [0m >>> [39m library_dirs = ['/usr/local/lib'] [0m >>> [39m language = c [0m >>> [39m include_dirs = ['/usr/local/include'] [0m >>> [39m [0m >>> /usr/ports/math/py-numpy/work/numpy-1.3.0/numpy/distutils/command/config.py:361: >>> DeprecationWarning: >>> +++++++++++++++++++++++++++++++++++++++++++++++++ >>> Usage of get_output is deprecated: please do not >>> use it anymore, and avoid configuration checks >>> involving running executable on the target machine. >>> +++++++++++++++++++++++++++++++++++++++++++++++++ >>> >>> DeprecationWarning) >>> [39mcustomize GnuFCompiler [0m >>> [32mFound executable /usr/local/bin/gfortran44 [0m >>> [31mgnu: no Fortran 90 compiler found [0m >>> [31mgnu: no Fortran 90 compiler found [0m >>> [39mcustomize Gnu95FCompiler [0m >>> [39mcustomize Gnu95FCompiler [0m >>> [39mcustomize Gnu95FCompiler using config [0m >>> compiling '_configtest.c': >>> >>> /* This file is generated from numpy/distutils/system_info.py */ >>> void ATL_buildinfo(void); >>> int main(void) { >>> ATL_buildinfo(); >>> return 0; >>> } >>> [39mC compiler: gcc44 -DNDEBUG -O2 -fno-strict-aliasing -pipe >>> -march=prescott -D__wchar_t=wchar_t -DTHREAD_STACK_SIZE=0x100000 -O2 >>> -fno-strict-aliasing -pipe -march=prescott -Wl,-rpath=/usr/local/lib/gcc44 >>> -fPIC >>> [0m >>> [39mcompile options: '-c' [0m >>> [39mgcc44: _configtest.c [0m >>> [39mgcc44 _configtest.o -L/usr/local/lib -lalapack_r -lf77blas_r -lcblas_r >>> -latlas_r -o _configtest [0m >>> /usr/bin/ld: _configtest: hidden symbol `__powisf2' in >>> /usr/local/lib/gcc44/gcc/i386-portbld-freebsd7.2/4.4.3/libgcc.a(_powisf2.o) >>> is referenced by DSO >>> collect2: ld returned 1 exit status >>> /usr/bin/ld: _configtest: hidden symbol `__powisf2' in >>> /usr/local/lib/gcc44/gcc/i386-portbld-freebsd7.2/4.4.3/libgcc.a(_powisf2.o) >>> is referenced by DSO >>> collect2: ld returned 1 exit status >>> [39mfailure. [0m >>> [39mremoving: _configtest.c _configtest.o [0m >>> [39mStatus: 255 [0m >>> [39mOutput: [0m >>> [39m FOUND: [0m >>> [39m libraries = ['alapack_r', 'f77blas_r', 'cblas_r', 'atlas_r'] [0m >>> [39m library_dirs = ['/usr/local/lib'] [0m >>> [39m language = c [0m >>> [39m define_macros = [('NO_ATLAS_INFO', 2)] [0m >>> [39m include_dirs = ['/usr/local/include'] [0m >>> [39m [0m >>> [39mlapack_opt_info: [0m >>> [39mlapack_mkl_info: [0m >>> [39mmkl_info: [0m >>> [39m libraries mkl,vml,guide not found in /usr/lib [0m >>> [39m libraries mkl,vml,guide not found in /usr/local/lib [0m >>> [39m libraries mkl,vml,guide not found in >>> /usr/local/lib/gcc44/gcc/i386-portbld-freebsd7.2/4.4.3/../../../ [0m >>> [39m NOT AVAILABLE [0m >>> [39m [0m >>> [39m NOT AVAILABLE [0m >>> [39m [0m >>> [39matlas_threads_info: [0m >>> [39mSetting PTATLAS=ATLAS [0m >>> [39m libraries lapack_atlas not found in /usr/local/lib [0m >>> [39mnumpy.distutils.system_info.atlas_threads_info [0m >>> [39mSetting PTATLAS=ATLAS [0m >>> /usr/ports/math/py-numpy/work/numpy-1.3.0/numpy/distutils/system_info.py:999: >>> UserWarning: >>> ********************************************************************* >>> Lapack library (from ATLAS) is probably incomplete: >>> size of /usr/local/lib/libalapack_r.so is 3832k (expected >4000k) >>> >>> Follow the instructions in the KNOWN PROBLEMS section of the file >>> numpy/INSTALL.txt. >>> ********************************************************************* >>> >>> warnings.warn(message) >>> >>> >>> The above sequence gets more or less repeated several more times, and >>> after much compiling, running of python26, and other activities, the >>> process runs aground as follows. >>> >>> >>> [39mcreating build/temp.freebsd-7.2-STABLE-i386-2.6/numpy/fft [0m >>> [39mcompile options: '-Inumpy/core/include >>> -Ibuild/src.freebsd-7.2-STABLE-i386-2.6/numpy/core/include/numpy >>> -Inumpy/core/src -Inumpy/core/include -I/usr/local/include/python2.6 -c' [0m >>> [39mgcc44: numpy/fft/fftpack_litemodule.c [0m >>> [39mgcc44: numpy/fft/fftpack.c [0m >>> [39mcc -shared -pthread -O2 -fno-strict-aliasing -pipe -march=prescott >>> -Wl,-rpath=/usr/local/lib/gcc44 >>> build/temp.freebsd-7.2-STABLE-i386-2.6/numpy/fft/fftpack_litemodule.o >>> build/temp.freebsd-7.2-STABLE-i386-2.6/numpy/fft/fftpack.o >>> -Lbuild/temp.freebsd-7.2-STABLE-i386-2.6 -o >>> build/lib.freebsd-7.2-STABLE-i386-2.6/numpy/fft/fftpack_lite.so [0m >>> [39mbuilding 'numpy.linalg.lapack_lite' extension [0m >>> [39mcompiling C sources [0m >>> [39mC compiler: gcc44 -DNDEBUG -O2 -fno-strict-aliasing -pipe >>> -march=prescott -D__wchar_t=wchar_t -DTHREAD_STACK_SIZE=0x100000 -O2 >>> -fno-strict-aliasing -pipe -march=prescott -Wl,-rpath=/usr/local/lib/gcc44 >>> -fPIC >>> [0m >>> [39mcreating build/temp.freebsd-7.2-STABLE-i386-2.6/numpy/linalg [0m >>> [39mcompile options: '-DATLAS_INFO="\"3.9.11\"" -I/usr/local/include >>> -Inumpy/core/include >>> -Ibuild/src.freebsd-7.2-STABLE-i386-2.6/numpy/core/include/numpy >>> -Inumpy/core/src -Inumpy/core/include -I/usr/local/include/python2.6 -c' [0m >>> [39mgcc44: numpy/linalg/lapack_litemodule.c [0m >>> [39mgcc44: numpy/linalg/python_xerbla.c [0m >>> [39mcc -shared -pthread -O2 -fno-strict-aliasing -pipe -march=prescott >>> -Wl,-rpath=/usr/local/lib/gcc44 >>> build/temp.freebsd-7.2-STABLE-i386-2.6/numpy/linalg/lapack_litemodule.o >>> build/temp.freebsd-7.2-STABLE-i386-2.6/numpy/linalg/python_xerbla.o >>> -L/usr/local/lib -Lbuild/temp.freebsd-7.2-STABLE-i386-2.6 -lalapack_r >>> -lalapack_r -lf77blas_r -lcblas_r -latlas_r -lgfortran -lm -lpthread -o >>> build/lib.freebsd-7.2-STABLE-i386-2.6/numpy/linalg/lapack_lite.so [0m >>> /usr/bin/ld: cannot find -lgfortran >>> /usr/bin/ld: cannot find -lgfortran >>> error: Command "cc -shared -pthread -O2 -fno-strict-aliasing -pipe >>> -march=prescott -Wl,-rpath=/usr/local/lib/gcc44 >>> build/temp.freebsd-7.2-STABLE-i386-2.6/numpy/linalg/lapack_litemodule.o >>> build/temp.freebsd-7.2-STABLE-i386-2.6/numpy/linalg/python_xerbla.o >>> -L/usr/local/lib -Lbuild/temp.freebsd-7.2-STABLE-i386-2.6 -lalapack_r >>> -lalapack_r -lf77blas_r -lcblas_r -latlas_r -lgfortran -lm -lpthread -o >>> build/lib.freebsd-7.2-STABLE-i386-2.6/numpy/linalg/lapack_lite.so" failed >>> with exit status 1 >>> *** Error code 1 >>> >>> Stop in /usr/ports/math/py-numpy. >>> >>> ===>>> make failed for math/py-numpy >>> ===>>> Aborting update >>> >>> I looked in /usr/local/lib and didn't spot libraries by name for >>> gfortran[345], so I assume they have either been renamed or included in >>> with the corresponding gcc libraries. What is the best way to proceed? >>> Will simply deleting the instances of "-lgfortran" do the job? Or do I >>> need to specify the library explicitly? >>> >> >>The port is broken WITH_ATLAS=yes on your system: it reverts to using >>the base system C compiler, cc, when it should be using gcc44. You >>trimmed off some relevant parts, so it is difficult to tell at a >>glance whether this is because something is wrong with your system, or >>with the port itself, although the port has a history of problems >>with this non-default option. If I have some time later, I may play >>with it, but at the moment, I don't have atlas installed. You should > > Okay, but be advised that the ATLAS library takes around six hours > to build on a 3.4 GHz Prescott with little else active on the machine > at the time. (ATLAS should be built on a very quiet machine, so that > its timing tests will be fairly accurate. The build is likely to die > if several runs of a timing test yield a variance greater than what the > build procedure wants.) > >>send a full build transcript to me and the math/py-numpy maintainer >>via private email. > > Will do, but probably not till tonight sometime. >> >>The libraries in question should be in /usr/local/lib/gcc44. When >>searching for registered shared libraries, it is easier to do >>something like: >> >>ldconfig -vr | grep -ie fortran > > 1046:-lgfortran.3 => /usr/local/lib/gcc43/libgfortran.so.3 > 1056:-lgfortran.3 => /usr/local/lib/gcc44/libgfortran.so.3 > 1068:-lgfortran.3 => /usr/local/lib/gcc45/libgfortran.so.3 >> >>and for static libraries or unregistered shared libraries, locate(1) >>and find(1) are your friends. >> > Well, locate(1) *ought* to be, you're right. However, although it > worked fine on FreeBSD 5, I've never succeeded in getting FreeBSD 6 or 7 > to build the locate database properly. It would be nice, but I've already > spent too many hours on too many occasions trying to track down the trouble, > so I just use find(1). > find(1) turns up the following. > > /usr/local/lib/gcc43/gcc/i386-portbld-freebsd7.2/4.3.5/libgfortranbegin.la > /usr/local/lib/gcc43/gcc/i386-portbld-freebsd7.2/4.3.5/libgfortranbegin.a > /usr/local/lib/gcc43/libgfortran.so.3 > /usr/local/lib/gcc43/libgfortran.so > /usr/local/lib/gcc43/libgfortran.a > /usr/local/lib/gcc44/gcc/i386-portbld-freebsd7.2/4.4.3/libgfortranbegin.la > /usr/local/lib/gcc44/gcc/i386-portbld-freebsd7.2/4.4.3/libgfortranbegin.a > /usr/local/lib/gcc44/libgfortran.so.3 > /usr/local/lib/gcc44/libgfortran.so > /usr/local/lib/gcc44/libgfortran.a > /usr/local/lib/gcc45/gcc/i386-portbld-freebsd7.2/4.5.0/libgfortranbegin.la > /usr/local/lib/gcc45/gcc/i386-portbld-freebsd7.2/4.5.0/libgfortranbegin.a > /usr/local/lib/gcc45/libgfortran.so.3 > /usr/local/lib/gcc45/libgfortran.so > /usr/local/lib/gcc45/libgfortran.a > > I'll get the rest of the stuff off to you later tonight. Thank you > very much for looking at all of this. I wonder if I'm the only person > trying to install science/gnudatalanguage. There have been enough > obstacles to getting that done that I have to wonder how the maintainer > managed to test it. > > > Scott Bennett, Comm. ASMELG, CFIAG > ********************************************************************** > * Internet: bennett at cs.niu.edu * > *--------------------------------------------------------------------* > * "A well regulated and disciplined militia, is at all times a good * > * objection to the introduction of that bane of all free governments * > * -- a standing army." * > * -- Gov. John Hancock, New York Journal, 28 January 1790 * > ********************************************************************** >