Date: Tue, 25 Mar 2014 00:13:02 +0200 From: Minas Dasygenis <mdasyg@ieee.org> To: dteske@FreeBSD.org Cc: freebsd-questions@freebsd.org Subject: Re: An admin script to check missing dependencies Message-ID: <20140324221302.GQ36346@bigb5.homeftp.net> In-Reply-To: <00cd01cf42d2$9e5b9080$db12b180$@FreeBSD.org> References: <20140318155612.GA84624@bigb5.homeftp.net> <00c401cf42cc$ef01ade0$cd0509a0$@FreeBSD.org> <00cd01cf42d2$9e5b9080$db12b180$@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Mar 18, 2014 at 10:50:56AM -0700, dteske@FreeBSD.org wrote: > > > > -----Original Message----- > > From: dteske@FreeBSD.org [mailto:dteske@FreeBSD.org] > > Sent: Tuesday, March 18, 2014 10:10 AM > > To: 'Minas Dasygenis'; freebsd-questions@freebsd.org > > Cc: dteske@FreeBSD.org > > Subject: RE: An admin script to check missing dependencies > > > > > > > > > -----Original Message----- > > > From: Minas Dasygenis [mailto:mdasyg@ieee.org] > > > Sent: Tuesday, March 18, 2014 8:56 AM > > > To: freebsd-questions@freebsd.org > > > Subject: An admin script to check missing dependencies > > > > > > > > > > > > Greetings, > > > > > > While I administer a number of FreeBSD Servers, I have faced the > > > problem > > of > > > an application failing to start, due to a missing dynamic library > > dependency. > > > Furthermore, if this application impacts the reputation of the server, > > then it > > > is mandatory to fix it as soon as possible. > > > > > > The problem originates when I update a port [using portmaster], which > > > installs a newer version of a library [e.g. removes the library.so.10 > > > and > > installs > > > library.so.11]. In such cases, applications that demand the old > > > library > > stop > > > working. Usually an "ln -s library.so.11 library.so.10" > > > fixes the problem, but this means that an early detection is required. > > > > > > To help myself [and other freebsd administrators], I have created a > > > script that I execute it daily. This script checks all files on the > > > system. If it > > finds a > > > missing dependency it reports it to the user, as well as a possible > > > fix if > > this is > > > available. > > > > > > This script is available at my home page at the miscellaneous section: > > > > > > http://arch.icte.uowm.gr/mdasyg/misc/check_requisite_library_files.sh > > > > > > > > > > Why not use "ldd -f%p\\n $file" instead of reading the normal ldd output? > > -- > > Devin > > > > > I am executing it on every FreeBSD server and whenever a missing > > > library > > is > > > found I am notified via email. > > > > > > Feel free to contribute any improvements via email and I will update it. > > > > [Devin Teske] > > Since you're running this on a large number of files, speed is important. > The following statement is going to cause a large slow-down: > > status=`file $i | cut -f2 -d":" | grep -v text | grep -v "link " | grep -v > "\.a:" | grep shared` > > Slow-down because you the shell has to fork-exec 7 times in that > single statement. If you replace that with the following, you can > reduce that to 1 fork and 1 fork-exec: > > # Don't process ".a" files > if [ "$i" = "${i%.a}" ]; then > status=$( file "$i" ) > case "${status#*:}" in *text*|*"link "*) status= ;; esac > fi > > While this is only one slow-down, there's quite a few. Here's another... > > missing=`ldd $i | grep = | grep -v /` > [...] > ldd $i | grep = | grep -v / | awk '{print $1}' | while read j > do > > That would be better written as: > > missing=$( ldd -f'%p\n' "$i" | awk '/=/&&$0!~"/"{print $1}' ) > [...] > echo "$missing"| while read j; do > > But there are also major mistakes... for example... > > status=`file $i | cut -f2 -d":" | grep -v text | grep -v "link " | grep -v > "\.a:" | grep shared` > if [ "$?" -eq 0 ] ;then > > Did you know that the return status of such a command (the > status=`...` command) will always be that of the last element > in the pipe-chain (grep shared)? > > You should change the: > > if [ "$?" -eq 0 ]; then > > to instead: > > if [ "$status" ]; then > > NB: Which is short-hand for: if [ -n "$status" ]; then > > Here's another mistake... > > output=`ldd $i 2>/dev/null | grep = | grep -v / | wc -l` > if [ "$output" -ne 0 ]; then > > While sh(1) appears to allow this, it is not correct to quote > $output here because "wc -l" has prepended whitespace > to the number returned on stdout. > > Also, it makes no sense to calculate $output as a number > of missing libraries and THEN immediately after re-calculate > the same exact list. Better to just calculate missing and then > test to see if the list of missing items is NULL... > > # Don't process ".a" files > if [ "$i" = "${i%.a}" ]; then > status=$( file "$i" ) > case "${status#*:}" in *text*|*"link "*) status= ;; esac > fi > [ "$status" ] || continue > # Still here? this is a file that uses shared libraries > missing=$( ldd -f'%p\n' "$i" | awk '/=/&&$0!~"/"{print $1}' ) > if [ "$missing" ]; then > output_on_screen=1 > echo " " > echo "WARNING: Missing library(-ies) for $i" > echo " --------------------------------------" > echo "$missing" | while read j > do > [...] > > But there are other performance issues... > > missingfile=`basename $j` > > Is better written as: > > missingfile="${j##*/}" > > and > > similarbase=`dirname $similarfile` > > Is better written as: > > similarbase="${similarfile%/*}" > > and > > base=`echo $missingfile | cut -f 1 -d" " | cut -f 1-2 -d.` > > Is better written as: > > base="${missingfile%%[$IFS]*}" > base="${base%.so*}.so" > > and > > similarfile=`locate $base | grep -v lib32 | grep -v compat | head > -1` > > Is better written as: > > similarfile=$( locate "$base" | awk '!/(lib32|compat)/{print;exit}' > ) > > and it seems silly to re-run locate to create $similarfilecount... why not > just... > > similarfiles=$( locate "$base" | awk '!/(lib32|compat)/{print}' ) > similarfile="${similarfiles%%$NL*}" # Where NL is defined as > follows: > NL=" > " # END-QUOTE (NL is a literal newline) > if [ "$similarfile" ]; then > similarbase="${similarfile%/*}" > [ -f "$similarfile" ] && echo \ > "Maybe1: ln -s $similarfile > $similarbase/$missinglib" > similarfilecount=$( echo "$similarfiles" | awk 'END{print > NR}' ) > if [ $similarfilecount -gt 2 ]; then > similarfile="${similarfiles##*$NL}" > similarbase="${similarfile##*/}" > [ -f "$similarfile" ] && echo \ > "Maybe2: ln -s $similarfile > $similarbase/$missinglib" > fi > > -- > Cheers, > Devin > > P.S. Not mentioning that the script doesn't protect against filenames > containing either whitespace or single-quotes or double-quotes, etc. etc. > > _____________ > The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. Thank you very much for your suggestions. I have included them at my new version of the script, together with a check to locate void links [if any]. I know that the correct thing is to rebuild every package that depends on a library, but sometimes this means that in a system with 550 packages too many packages will need to be rebuilt. If the maintenance time is too sort, then a symbolic link will help a lot. Finally, I didn't know about the libmap functionality. It seems that it may solve these dependencies. Of course this means that the file /etc/libmap.conf should be edited appropriately every time there is a problem. Seems a clean and good way to do it. I will check it out and I will modify my script to also suggest to the user to either insert a line on the libmap or do the ln -s MD
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20140324221302.GQ36346>