Date: Fri, 29 Aug 2014 14:45:55 -0500 From: Bryan Drewery <bdrewery@FreeBSD.org> To: Konstantin Belousov <kostikbel@gmail.com> Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org Subject: Re: svn commit: r270803 - head/libexec/rtld-elf Message-ID: <5400D873.70707@FreeBSD.org> In-Reply-To: <20140829172311.GP2737@kib.kiev.ua> References: <201408291044.s7TAiwmI077897@svn.freebsd.org> <540094DB.5050307@FreeBSD.org> <20140829172311.GP2737@kib.kiev.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --Xw3mDU01mQ6XERfTtjnuEU2AticK7S1We Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 8/29/2014 12:23 PM, Konstantin Belousov wrote: > On Fri, Aug 29, 2014 at 09:57:31AM -0500, Bryan Drewery wrote: >> On 8/29/2014 5:44 AM, Konstantin Belousov wrote: >>> Author: kib >>> Date: Fri Aug 29 10:44:58 2014 >>> New Revision: 270803 >>> URL: http://svnweb.freebsd.org/changeset/base/270803 >>> >>> Log: >>> Document the whole settings needed to build a debug version of rtld= =2E >>> =20 >>> Sponsored by: The FreeBSD Foundation >>> MFC after: 3 days >>> >>> Modified: >>> head/libexec/rtld-elf/Makefile >>> >>> Modified: head/libexec/rtld-elf/Makefile >>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D >>> --- head/libexec/rtld-elf/Makefile Fri Aug 29 10:43:56 2014 (r270802)= >>> +++ head/libexec/rtld-elf/Makefile Fri Aug 29 10:44:58 2014 (r270803)= >>> @@ -1,5 +1,9 @@ >>> # $FreeBSD$ >>> =20 >>> +# Use the following command to build local debug version of dynamic >>> +# linker: >>> +# make DEBUG_FLAGS=3D-g DEBUG=3D-DDEBUG MK_TESTS=3Dno all >>> + >>> .include <src.opts.mk> >>> MK_SSP=3D no >>> =20 >>> >> >> How difficult would it be to allow DEBUG to be set during runtime like= >> GNU's can with LD_DEBUG? I have found GNU's LD_DEBUG to be very useful= >> for userland debugging, especially when using dlopen(3). >> >> We have LD_DEBUG environment variable but it only prints if built with= >> -DDEBUG. >> >> Is there a concern about performance by enabling this based only on >> environment? >=20 > I am sure that nobody evaluated the performance consequences of > unconditionally compiling the debugging stuff in. I am sure that the > ld-elf.so.1 size will increase. >=20 > The reason why nobody cares to evaluate and enable this stuff by defaul= t > is that the debugging output is very ad-hoc. I found it almost useless > in both the content and amount of data it outputs. The reason why I > enable it for my work on ld-elf is that I insert my own dbg() calls for= > debugging (and usually remove them before the commit since they add eve= n > more verbosity useless for general public consumption). Ah. >=20 > I find the combination of the ELF dumping tools like readelf and > objdump, together with gdb (-g) and custom dbg() statement (slighly > glorified printf debugging) most adequate combination to debug rtld. >=20 > After the long preamble. What use do you have for LD_DEBUG ? If it is > possible to formalize and tailor the debugging output for real users > needs, I am all for making it available unconditionally from LD_DEBUG > knob. The significants part of the current dbg() statements would be > removed or require more agressive settings to become active. >=20 My only uses so far have been observing which file is loaded for a library with dlopen(3) and where symbols are resolved from. Seeing where symbols are resolved from is useful for both dlopen(3)/dlsym(3) usage and LD_PRELOAD. I've written an application that "optionally" allows using some libraries such as libtcl to be used without requiring the library at build time or startup. At build time the library's headers are used to generate wrapper functions for every symbol needed. The library is not linked in. When the binary is ran it starts up fine if libtcl is missing. At runtime if TCL is enabled then it will attempt to dlopen(libtcl.so.VER) using the VER its symbols was compiled against. If it cannot find it then the feature is disabled. It uses dlsym(3) to load all symbols into a table and the wrapper functions use that table via hash table lookups. This all pretty convoluted and only done to avoid requiring a library at startup. The application is distributed pre-compiled for specific OS releases and not intended to be compiled manually. Anyway some examples from GNU's rtld: > # env LD_DEBUG=3Dhelp ./app > Valid options for the LD_DEBUG environment variable are: >=20 > libs display library search paths > reloc display relocation processing > files display progress for input file > symbols display symbol table processing > bindings display information about symbol binding > versions display version dependencies > all all previous options combined > statistics display relocation statistics > unused determined unused DSOs > help display this help message and exit >=20 > To direct the debugging output into a file instead of standard output > a filename can be specified using the LD_DEBUG_OUTPUT environment varia= ble. With libs: > # env LD_DEBUG=3Dlibs ./app > ... > 9328: find library=3Dlibtcl.so [0]; searching > 9328: search cache=3D/etc/ld.so.cache > 9328: search path=3D/lib64/tls/x86_64:/lib64/tls:/lib64/x86_= 64:/lib64:/usr/lib64/tls/x86_64:/usr/lib64/tls:/usr/lib64/x86_64:/usr/lib= 64 (system search path) > 9328: trying file=3D/lib64/tls/x86_64/libtcl.so > 9328: trying file=3D/lib64/tls/libtcl.so > 9328: trying file=3D/lib64/x86_64/libtcl.so > 9328: trying file=3D/lib64/libtcl.so > 9328: trying file=3D/usr/lib64/tls/x86_64/libtcl.so > 9328: trying file=3D/usr/lib64/tls/libtcl.so > 9328: trying file=3D/usr/lib64/x86_64/libtcl.so > 9328: trying file=3D/usr/lib64/libtcl.so > 9328: > 9328: find library=3Dlibpthread.so.0 [0]; searching > 9328: search path=3D/usr/lib64 (system search path) > 9328: trying file=3D/usr/lib64/libpthread.so.0 > 9328: search cache=3D/etc/ld.so.cache > 9328: trying file=3D/lib64/libpthread.so.0 > 9328: > 9328: > 9328: calling init: /lib64/libpthread.so.0 > 9328: > 9328: > 9328: calling init: /usr/lib64/libtcl.so > 9328: > 9328: find library=3Dlibnss_compat.so.2 [0]; searching > 9328: search cache=3D/etc/ld.so.cache > 9328: trying file=3D/lib64/libnss_compat.so.2 > 9328: > 9328: find library=3Dlibnsl.so.1 [0]; searching > 9328: search cache=3D/etc/ld.so.cache > 9328: trying file=3D/lib64/libnsl.so.1 > 9328: > 9328: > 9328: calling init: /lib64/libnsl.so.1 > 9328: > 9328: > 9328: calling init: /lib64/libnss_compat.so.2 > 9328: > 9328: find library=3Dlibnss_nis.so.2 [0]; searching > 9328: search cache=3D/etc/ld.so.cache > 9328: trying file=3D/lib64/libnss_nis.so.2 > 9328: > 9328: find library=3Dlibnss_files.so.2 [0]; searching > 9328: search cache=3D/etc/ld.so.cache > 9328: trying file=3D/lib64/libnss_files.so.2 > 9328: > 9328: > 9328: calling init: /lib64/libnss_files.so.2 > 9328: > 9328: > 9328: calling init: /lib64/libnss_nis.so.2 > ^C > 9328: calling fini: /usr/lib64/libtcl.so [0] > 9328: > 9328: > 9328: calling fini: /lib64/libm.so.6 [0] > 9328: > 9328: > 9328: calling fini: /lib64/libdl.so.2 [0] > 9328: > 9328: > 9328: calling fini: /lib64/libpthread.so.0 [0] > 9328: > 9328: > 9328: calling fini: /lib64/libnss_compat.so.2 [0] > 9328: > 9328: > 9328: calling fini: /lib64/libnss_nis.so.2 [0] > 9328: > 9328: > 9328: calling fini: /lib64/libnsl.so.1 [0] > 9328: > 9328: > 9328: calling fini: /lib64/libnss_files.so.2 [0] > 9328: > 9328: > 9328: calling fini: /lib64/libc.so.6 [0] With symbols: > # LD_DEBUG=3Dsymbols ./app > ... > 9355: symbol=3DTcl_SetObjResult; lookup in file=3D/lib64/lib= dl.so.2 [0] > 9355: symbol=3DTcl_SetObjResult; lookup in file=3D/usr/lib/g= cc/x86_64-pc-linux-gnu/4.8.2/libstdc++.so.6 [0] > 9355: symbol=3DTcl_SetObjResult; lookup in file=3D/lib64/lib= m.so.6 [0] > 9355: symbol=3DTcl_SetObjResult; lookup in file=3D/usr/lib/g= cc/x86_64-pc-linux-gnu/4.8.2/libgcc_s.so.1 [0] > 9355: symbol=3DTcl_SetObjResult; lookup in file=3D/lib64/lib= c.so.6 [0] > 9355: symbol=3DTcl_SetObjResult; lookup in file=3D/lib64/ld-= linux-x86-64.so.2 [0] > 9355: symbol=3DTcl_SetObjResult; lookup in file=3D/usr/lib64= /libcrypto.so.1.0.0 [0] > 9355: symbol=3DTcl_SetObjResult; lookup in file=3D/lib64/lib= z.so.1 [0] > 9355: symbol=3DTcl_SetObjResult; lookup in file=3D/usr/lib64= /libtcl.so [0] > 9355: symbol=3DTcl_GetObjResult; lookup in file=3D./netplay = [0] > 9355: symbol=3DTcl_GetObjResult; lookup in file=3D/lib64/lib= dl.so.2 [0] > 9355: symbol=3DTcl_GetObjResult; lookup in file=3D/usr/lib/g= cc/x86_64-pc-linux-gnu/4.8.2/libstdc++.so.6 [0] > 9355: symbol=3DTcl_GetObjResult; lookup in file=3D/lib64/lib= m.so.6 [0] > 9355: symbol=3DTcl_GetObjResult; lookup in file=3D/usr/lib/g= cc/x86_64-pc-linux-gnu/4.8.2/libgcc_s.so.1 [0] > 9355: symbol=3DTcl_GetObjResult; lookup in file=3D/lib64/lib= c.so.6 [0] > 9355: symbol=3DTcl_GetObjResult; lookup in file=3D/lib64/ld-= linux-x86-64.so.2 [0] > 9355: symbol=3DTcl_GetObjResult; lookup in file=3D/usr/lib64= /libcrypto.so.1.0.0 [0] > 9355: symbol=3DTcl_GetObjResult; lookup in file=3D/lib64/lib= z.so.1 [0] > 9355: symbol=3DTcl_GetObjResult; lookup in file=3D/usr/lib64= /libtcl.so [0] I would love to work on adding such a feature to ours, especially if it can assist with debugging rtld itself. I would think with some predict_false() checks we could avoid much of the performance penalties. As a side note I think I may have found an rtld issue (on 8.4) with -32 handling. Being a production machine I don't want to attempt installing a DEBUG rtld to figure out the problem. Having LD_DEBUG would have been useful on this as well. The issue I ran into with -32 was from converting a i386 system to amd64. I have hundreds of out-of-ports binaries that need to be manually recompiled. To allow delaying this task for a bit I copied all of the libraries from /usr/local/lib from the old i386 system to /usr/local/lib32/compat/system and added all of the paths in there to ldconfig: > [~] # ldconfig -32 -r|head -n3 > /var/run/ld-elf32.so.hints: > search directories: /usr/lib32:/usr/local/lib32/compat/system:/= usr/local/lib32/compat/system/mysql:/usr/local/lib32/compat/system/gnutls= 3:/usr/local/lib32/compat/system/pth:/usr/local/lib32/compat/system/qt4:/= usr/local/lib32/compat/system/tdom0.8.3:/usr/local/lib32/compat/system/gc= c46:/usr/local/lib32/compat/system/gcc47:/usr/local/lib32/compat/system/g= cc48:/usr/local/lib32/compat/system/gcc49:/usr/local/lib32/compat/system/= event2:/usr/local/lib32/compat/pkg:/usr/local/lib32/compat It worked for 90% of the system but then I ran into some libraries that refused to find their own dependencies. The issue seemingly is that dependencies for libraries that are also in the -32 path are not looked u= p. For example: > [~] # ldd -a /usr/local/lib32/compat/system/libssl.so > /usr/local/lib32/compat/system/libssl.so: > libcrypto.so.8 =3D> not found (0x0) > libthr.so.3 =3D> /usr/lib32/libthr.so.3 (0x310fb000) > libc.so.7 =3D> /usr/lib32/libc.so.7 (0x29165000) > /usr/lib32/libthr.so.3: > libc.so.7 =3D> /usr/lib32/libc.so.7 (0x29165000) > [~] # ldconfig -32 -r|grep libcrypto.so.8 > 105:-lcrypto.8 =3D> /usr/local/lib32/compat/system/libcrypto.so= =2E8 > [~] # readelf -a /usr/local/lib32/compat/system/libssl.so|egrep "(path|= NEEDED)" > 0x00000001 (NEEDED) Shared library: [libcrypto.so.= 8] > 0x00000001 (NEEDED) Shared library: [libthr.so.3] > 0x00000001 (NEEDED) Shared library: [libc.so.7] > 0x0000000f (RPATH) Library rpath: [/usr/local/lib= ] My guess is it is the RPATH. I am not sure without a simple debug ability= =2E --=20 Regards, Bryan Drewery --Xw3mDU01mQ6XERfTtjnuEU2AticK7S1We Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (MingW32) iQEcBAEBAgAGBQJUANhzAAoJEDXXcbtuRpfPmf4H/1gcuAe+7veKWq+jBEBmvxmF qOvhRdoUuLqJ6p+i12EECaOKX8ocoLn+6WutZRbYFJ+/TzKRxyflvmEwRC7JAPkY U2AbmrFmuk8T8MzPHdK26z5A1i4hiqfmlFJ0sDTWUW2pl0JjK7DuyezwLaQtLEO4 pkghihI7KmLFF4AZ1WrYdqEFhH4aYvOwT9vvBKcOD/mZhn8IM9/HMpzHbmXNEQCe qHdCNyu5dVJd/CXvpQ75Sf950P1mzoxvJFATBVh1mcDa1DAAVTOpVzrycSrpvVmb TBF51VLjMpbUVR5T2qhZSsJPgZfObg47XKteRPIUQ4ft8MnxtIffTn5YE5VOdAc= =QlIM -----END PGP SIGNATURE----- --Xw3mDU01mQ6XERfTtjnuEU2AticK7S1We--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5400D873.70707>