From owner-freebsd-questions@FreeBSD.ORG Tue Mar 18 17:51:09 2014 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5A0EBAB2; Tue, 18 Mar 2014 17:51:09 +0000 (UTC) Received: from mx1.fisglobal.com (mx1.fisglobal.com [199.200.24.190]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 242EE65D; Tue, 18 Mar 2014 17:51:08 +0000 (UTC) Received: from smarthost.fisglobal.com ([10.132.206.191]) by ltcfislmsgpa02.fnfis.com (8.14.5/8.14.5) with ESMTP id s2IHp72A031425 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Tue, 18 Mar 2014 12:51:08 -0500 Received: from THEMADHATTER (10.242.181.54) by smarthost.fisglobal.com (10.132.206.191) with Microsoft SMTP Server id 14.3.174.1; Tue, 18 Mar 2014 12:51:06 -0500 From: Sender: Devin Teske To: "'Minas Dasygenis'" , References: <20140318155612.GA84624@bigb5.homeftp.net> <00c401cf42cc$ef01ade0$cd0509a0$@FreeBSD.org> In-Reply-To: <00c401cf42cc$ef01ade0$cd0509a0$@FreeBSD.org> Subject: RE: An admin script to check missing dependencies Date: Tue, 18 Mar 2014 10:50:56 -0700 Message-ID: <00cd01cf42d2$9e5b9080$db12b180$@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 15.0 Thread-Index: AQJA7qPdsnzqasj+r/GNVCWb6JtaQwHcZyAFmfTZ8YA= Content-Language: en-us X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.11.87, 1.0.14, 0.0.0000 definitions=2014-03-18_07:2014-03-18,2014-03-18,1970-01-01 signatures=0 Cc: dteske@FreeBSD.org X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 17:51:09 -0000 > -----Original Message----- > From: dteske@FreeBSD.org [mailto:dteske@FreeBSD.org] > Sent: Tuesday, March 18, 2014 10:10 AM > To: 'Minas Dasygenis'; freebsd-questions@freebsd.org > Cc: dteske@FreeBSD.org > Subject: RE: An admin script to check missing dependencies > > > > > -----Original Message----- > > From: Minas Dasygenis [mailto:mdasyg@ieee.org] > > Sent: Tuesday, March 18, 2014 8:56 AM > > To: freebsd-questions@freebsd.org > > Subject: An admin script to check missing dependencies > > > > > > > > Greetings, > > > > While I administer a number of FreeBSD Servers, I have faced the > > problem > of > > an application failing to start, due to a missing dynamic library > dependency. > > Furthermore, if this application impacts the reputation of the server, > then it > > is mandatory to fix it as soon as possible. > > > > The problem originates when I update a port [using portmaster], which > > installs a newer version of a library [e.g. removes the library.so.10 > > and > installs > > library.so.11]. In such cases, applications that demand the old > > library > stop > > working. Usually an "ln -s library.so.11 library.so.10" > > fixes the problem, but this means that an early detection is required. > > > > To help myself [and other freebsd administrators], I have created a > > script that I execute it daily. This script checks all files on the > > system. If it > finds a > > missing dependency it reports it to the user, as well as a possible > > fix if > this is > > available. > > > > This script is available at my home page at the miscellaneous section: > > > > http://arch.icte.uowm.gr/mdasyg/misc/check_requisite_library_files.sh > > > > > > Why not use "ldd -f%p\\n $file" instead of reading the normal ldd output? > -- > Devin > > > I am executing it on every FreeBSD server and whenever a missing > > library > is > > found I am notified via email. > > > > Feel free to contribute any improvements via email and I will update it. > > [Devin Teske] Since you're running this on a large number of files, speed is important. The following statement is going to cause a large slow-down: status=`file $i | cut -f2 -d":" | grep -v text | grep -v "link " | grep -v "\.a:" | grep shared` Slow-down because you the shell has to fork-exec 7 times in that single statement. If you replace that with the following, you can reduce that to 1 fork and 1 fork-exec: # Don't process ".a" files if [ "$i" = "${i%.a}" ]; then status=$( file "$i" ) case "${status#*:}" in *text*|*"link "*) status= ;; esac fi While this is only one slow-down, there's quite a few. Here's another... missing=`ldd $i | grep = | grep -v /` [...] ldd $i | grep = | grep -v / | awk '{print $1}' | while read j do That would be better written as: missing=$( ldd -f'%p\n' "$i" | awk '/=/&&$0!~"/"{print $1}' ) [...] echo "$missing"| while read j; do But there are also major mistakes... for example... status=`file $i | cut -f2 -d":" | grep -v text | grep -v "link " | grep -v "\.a:" | grep shared` if [ "$?" -eq 0 ] ;then Did you know that the return status of such a command (the status=`...` command) will always be that of the last element in the pipe-chain (grep shared)? You should change the: if [ "$?" -eq 0 ]; then to instead: if [ "$status" ]; then NB: Which is short-hand for: if [ -n "$status" ]; then Here's another mistake... output=`ldd $i 2>/dev/null | grep = | grep -v / | wc -l` if [ "$output" -ne 0 ]; then While sh(1) appears to allow this, it is not correct to quote $output here because "wc -l" has prepended whitespace to the number returned on stdout. Also, it makes no sense to calculate $output as a number of missing libraries and THEN immediately after re-calculate the same exact list. Better to just calculate missing and then test to see if the list of missing items is NULL... # Don't process ".a" files if [ "$i" = "${i%.a}" ]; then status=$( file "$i" ) case "${status#*:}" in *text*|*"link "*) status= ;; esac fi [ "$status" ] || continue # Still here? this is a file that uses shared libraries missing=$( ldd -f'%p\n' "$i" | awk '/=/&&$0!~"/"{print $1}' ) if [ "$missing" ]; then output_on_screen=1 echo " " echo "WARNING: Missing library(-ies) for $i" echo " --------------------------------------" echo "$missing" | while read j do [...] But there are other performance issues... missingfile=`basename $j` Is better written as: missingfile="${j##*/}" and similarbase=`dirname $similarfile` Is better written as: similarbase="${similarfile%/*}" and base=`echo $missingfile | cut -f 1 -d" " | cut -f 1-2 -d.` Is better written as: base="${missingfile%%[$IFS]*}" base="${base%.so*}.so" and similarfile=`locate $base | grep -v lib32 | grep -v compat | head -1` Is better written as: similarfile=$( locate "$base" | awk '!/(lib32|compat)/{print;exit}' ) and it seems silly to re-run locate to create $similarfilecount... why not just... similarfiles=$( locate "$base" | awk '!/(lib32|compat)/{print}' ) similarfile="${similarfiles%%$NL*}" # Where NL is defined as follows: NL=" " # END-QUOTE (NL is a literal newline) if [ "$similarfile" ]; then similarbase="${similarfile%/*}" [ -f "$similarfile" ] && echo \ "Maybe1: ln -s $similarfile $similarbase/$missinglib" similarfilecount=$( echo "$similarfiles" | awk 'END{print NR}' ) if [ $similarfilecount -gt 2 ]; then similarfile="${similarfiles##*$NL}" similarbase="${similarfile##*/}" [ -f "$similarfile" ] && echo \ "Maybe2: ln -s $similarfile $similarbase/$missinglib" fi -- Cheers, Devin P.S. Not mentioning that the script doesn't protect against filenames containing either whitespace or single-quotes or double-quotes, etc. etc. _____________ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you.