From owner-freebsd-ia64@FreeBSD.ORG Thu Mar 18 19:26:27 2010 Return-Path: Delivered-To: freebsd-ia64@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4925B106564A for ; Thu, 18 Mar 2010 19:26:27 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-px0-f200.google.com (mail-px0-f200.google.com [209.85.216.200]) by mx1.freebsd.org (Postfix) with ESMTP id 1B2D38FC18 for ; Thu, 18 Mar 2010 19:26:26 +0000 (UTC) Received: by pxi38 with SMTP id 38so1522231pxi.27 for ; Thu, 18 Mar 2010 12:26:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=ERW7BWabPP3KPR2dvErA6G/KfmWa8ljxgC5Hiu2+y9c=; b=BxlcGXGkkn6UjRGdVKEPIEGnK8JU17+HD5g8rZE9Boed04KXpWFvaJAcMfVH9aSci0 MKJTkijSG6jQ/3IECQI/wlHvx/Lip0oPuP7+tmRgfuHBuFJIm9MMd6wWS3FMAi9dBpth XDlLjJh+DHv2dutzlDIVO8k3/3EBVbIOsJuYY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=Z0Eb8D713FrMecisTabot4++WQIiWIKZKbpaONgDFHuOye4Xw5ebtMuWyWjgAgbUgg SO7DJox2r3mXQX9rdO/Xq+Eu+a2jBLOxklk58zgXr1nUwqkecF5xpQ/46IHsiApPhgYd BQfk6d79vg+eVXeupqTw9gFkalMV+cMx33I1c= MIME-Version: 1.0 Received: by 10.142.249.29 with SMTP id w29mr125249wfh.163.1268938774569; Thu, 18 Mar 2010 11:59:34 -0700 (PDT) In-Reply-To: <20100318155113.GE1552@mech-cluster241.men.bris.ac.uk> References: <20100317163230.GJ87732@mech-cluster241.men.bris.ac.uk> <20100318155113.GE1552@mech-cluster241.men.bris.ac.uk> Date: Thu, 18 Mar 2010 11:59:34 -0700 Message-ID: <7d6fde3d1003181159t3eb1a665ge07f5673cf096e67@mail.gmail.com> From: Garrett Cooper To: Anton Shterenlikht Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: jhell , FreeBSD Current , freebsd-ia64@freebsd.org Subject: Re: ldd leaves the machine unresponsive X-BeenThere: freebsd-ia64@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting FreeBSD to the IA-64 List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Mar 2010 19:26:27 -0000 On Thu, Mar 18, 2010 at 8:51 AM, Anton Shterenlikht w= rote: > On Thu, Mar 18, 2010 at 11:29:36AM -0400, jhell wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> >> >> On Wed, 17 Mar 2010 12:32, Anton Shterenlikht wrote: >> In Message-Id: <20100317163230.GJ87732@mech-cluster241.men.bris.ac.uk> >> >> > Just updated to ia64 r205248 >> > >> > If my problem is due to my mis-configuration, >> > I apologise in advance. >> > >> > I run this shell script after each upgrade >> > and 'make delete-old-libs' to check >> > if any shared objects need to be rebuilt: >> > >> > >> > >> > #!/bin/sh >> > >> > for file in `find /bin /sbin /usr/bin /usr/sbin /usr/lib /usr/libexec = /usr/local -name "*"` >> > do >> > =A0 =A0 =A0 =A0echo $file >> > =A0 =A0 =A0 =A0ldd $file >> /root/ldd_results 2> /dev/zero >> > done >> > >> > >> > >> >> This will probably do closer to what you actually would want to look for= . >> >> Writing to /dev/zero ... I don't know never tried it since /dev/null is >> usually the standard place to throw trash. >> >> #!/bin/sh >> for file in `find /*bin /usr/*bin /usr/lib* /usr/local/*bin -type f` do >> =A0 =A0 =A0 echo $file >> =A0 =A0 =A0 ldd $file >>/root/ldd_results 2>/dev/null >> done >> >> The problem with your script is that it finds most files that it can not >> or is not useful to run ldd on and leaves you junk in return. >> >> It might be more useful if you searched for dynamically linked ELF >> binaries to run ldd against like the following. >> >> =3D=3D=3D Script starts here =3D=3D=3D >> #!/bin/sh >> >> SEARCHPATH=3D"/*bin /usr/*bin /usr/lib* /usr/local/*bin" >> >> trap 'exit 1' 2 >> >> check_libs() { >> for spath in $SEARCHPATH; do >> =A0 =A0 =A0 =A0 =A0for ifelf in `find $spath -type f`; do >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ldd `file $ifelf | grep dynamically |= cut -f1 -d:` >> =A0 =A0 =A0 =A0 =A0done >> done >> } >> >> check_libs 2>/dev/null >> =3D=3D=3D Script ends here =3D=3D=3D >> >> The above will find all type ELF * that are dynamically linked within th= e >> SEARCHPATH variable and run ldd on them and print the results to stdout. >> >> Obviously since you are going to have thousands of files being questione= d, >> stdout is not going to be useful. >> >> So with the about stated: >> save the script to: checklibs.sh >> run with: "sh checklibs.sh >/root/checklibs_output" >> or: "script /root/checklibs_output checklibs.sh" >> >> > After the upgrade to r205248, the script >> > freezes at seemingly random points. >> > >> >> Unneeded disk usage & execution. >> >> > I can still ssh to the machine (using keys), i.e. >> > I see the welcome message, but cannot get to the console prompt. >> >> Of course... to many open files or processes in wait. SSH already has th= e >> information it needs loaded into memory, that's why you can get sort-of-= in >> >> ZFS file-system perhaps ? >> >> > >> > On the serial console I cannot get the prompt >> > after entering the root password. >> > >> >> See above. >> >> > I have top(1) running interactively in another window. >> > The sh process is in "getblk" state, and ignores kill -9. >> > But there's no ldd process. >> > >> > And shutdown requests are also ignored: >> > >> > # shutdown -r now >> > Shutdown NOW! >> > shutdown: [pid 8019] >> > # >> > and nothing happens after that >> > >> > So I have to do a cold reset via MP. >> > >> > On ia64 r204322, this script causes no problems. >> > >> > Please advise >> > >> >> The above edited script should help to limit disk usage and too many ope= n >> processes that causes your machine to bog down like that. This script do= es >> have its limitations and there is one bug in it... Ill let you figure ou= t >> how to get rid of that bug but it really does not effect the intended >> output so I left it alone and sent error output to fd/2. >> >> The limitations you'll find is how many files that ldd(1) or file(1) can >> handle at one time. But if you specify specific paths like already in >> SEARCHPATH then you will most likely never see this unless the files in >> /*bin grow to be over max number of files that file(1) or ldd(1) can >> handle at one time. Shortly said... use direct paths or short globs like >> above. >> >> > many thanks >> > anton >> > >> >> A final note you might want to just install sysutils/libchk and run that= . >> >> Standard Disclaimer: NONE OF THIS CONTAINED HEREIN "THIS MESSAGE" EXCUSE= S >> ANY OF THE UNEXPLAINED DISK LOCKING THAT IS GOING ON AND THE INFORMATION >> FOR WHICH IT MAY CONTAIN BECOMING UNAVAILABLE AT ANY POINT IN TIME DURIN= G >> THE ORIGINAL RUN OF THE FIRST SCRIPT OR THE SECOND SCRIPT THAT WAS POSTE= D >> EITHER AS A ATTACHMENT OR IN-LINE. >> >> ;) JK! >> >> Good Luck. > > many thanks, this is very helpful > > I don't seem to have this lockup anymore. > Don't know what was happening. I've run > it now several times on 3 different ia64 > current (different revisions) boxes, with > disks of different speed, and can't reproduce. > My script was very crude, of course. > I'll try sysutils/libchk FWIW I've been seeing some performance issues with iir(4) and mfi(4) backed UFS2 with softupdate filesystems on my new machine with some other drivers loaded on my system [a PCI based em(4) card and nvidia-driver enabled card -- which uses GIANT locking still]. Machine is Core i7 on an ASUS W6T Professional MB, 12GB RAM, with debug symbols, ddb, kgdb, anti-reslock contention manager, (no witness) etc. I don't have much other than that to provide at this time, but it might help to see if and when there's an overlap in the drivers noted here. Thanks, -Garrett