From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 24 18:17:54 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E12DA106564A for ; Wed, 24 Feb 2010 18:17:53 +0000 (UTC) (envelope-from psteele@maxiscale.com) Received: from server505.appriver.com (server505e.appriver.com [98.129.35.9]) by mx1.freebsd.org (Postfix) with ESMTP id A77C28FC0A for ; Wed, 24 Feb 2010 18:17:53 +0000 (UTC) X-Policy: GLOBAL - maxiscale.com X-Primary: psteele@maxiscale.com X-Note: This Email was scanned by AppRiver SecureTide X-ALLOW: psteele@maxiscale.com ALLOWED X-Virus-Scan: V- X-Note: Spam Tests Failed: X-Country-Path: UNITED STATES->UNITED STATES->UNITED STATES X-Note-Sending-IP: 98.129.23.45 X-Note-Reverse-DNS: ht03.exg5.exghost.com X-Note-WHTLIST: psteele@maxiscale.com X-Note: User Rule Hits: X-Note: Global Rule Hits: G173 G174 G175 G176 G180 G181 G192 G279 X-Note: Encrypt Rule Hits: X-Note: Mail Class: ALLOWEDSENDER X-Note: Headers Injected Received: from [98.129.23.45] (HELO HT03.exg5.exghost.com) by server505.appriver.com (CommuniGate Pro SMTP 5.3.2) with ESMTPS id 25472348 for freebsd-hackers@freebsd.org; Wed, 24 Feb 2010 12:17:52 -0600 Received: from mbx03.exg5.exghost.com ([169.254.1.200]) by HT03.exg5.exghost.com ([10.242.228.75]) with mapi; Wed, 24 Feb 2010 12:17:51 -0600 From: Peter Steele To: "freebsd-hackers@freebsd.org" Date: Wed, 24 Feb 2010 12:17:50 -0600 Thread-Topic: ntpd hangs under FBSD 8 Thread-Index: Acqz1wAQdYoZ2dInQG2e7o15W4OUdwAAMdUwAGMe0qA= Message-ID: <7B9397B189EB6E46A5EE7B4C8A4BB7CB39E95389@MBX03.exg5.exghost.com> References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D5C73@MBX03.exg5.exghost.com> <20100220113349.GA22800@kiwi.sharlinx.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60B7@MBX03.exg5.exghost.com> <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60DD@MBX03.exg5.exghost.com> In-Reply-To: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D60DD@MBX03.exg5.exghost.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: RE: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 18:17:54 -0000 >You're going to need a debug version of libc, too. gdb won't be able to f= ind a backtrace out of a libc function without it. What's the proper way to build a debug version of libc and the other librar= ies? I tried this: export CFLAGS=3D"-O0" make buildworld make installworld DESTDIR=3D/mydir and then copied libc.so.7 from /mydir/lib to the /lib dir on my target syst= em. I also replaced the ntpd binary with the debug version. I can see that = -O0 is being used in the various "cc" commands that are generated, but libc= still doesn't seem to be built properly. When I attach to a hung ntpd proc= ess, I get this: # gdb /usr/sbin/ntpd -p 2113 GNU gdb 6.1.1 [FreeBSD] ... Attaching to program: /usr/sbin/ntpd, process 2113 Reading symbols from /lib/libm.so.5...(no debugging symbols found)...done. ... Reading symbols from /lib/libc.so.7...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /lib/libthr.so.3...(no debugging symbols found)...done= . ... [Switching to Thread 8012041c0 (LWP 100283)] 0x0000000800dbeddc in select () from /lib/libc.so.7 (gdb) bt #0 0x0000000800dbeddc in select () from /lib/libc.so.7 #1 0x00000000004335de in ntpdmain () #2 0x000000000043310b in main () So I'm getting some symbols from ntpd but I still can't see into select(). = It hangs in there forever so that's where I need to drill down further. How= do I get libc built with full debug symbols? In other testing I've narrowed the problem down to some kind of Python issu= e. If I run the Python code at the end of this email where "ntpd -g -q" is = launched as part of a Python thread class, the command hangs (the code assu= mes that ntpd is not already running). If I run the same ntpd command in a = normal function (e.g. main) no hang occurs. I've tried subcommand.Popen and= os.spawnv to run ntpd and these calls behave exactly the same way--when ca= lled from a thread the ntpd process hangs but it works fine when called fro= m outside of a thread. This is a breakdown of course of our larger project = into a simple test app. In our real code we cannot so easily eliminate this= thread wrapper. The same code BTW works fine on our FreeBSD 7 boxes, the main difference be= ing we are running an older version of Python on those boxes (2.5.1 instead= of 2.6.2). I tried installing the same 2.5.1 package on a FBSD 8 box and t= hat solved the problem. Curiously a slightly newer FBSD 7 version of Python= , 2.5.5, causes the same hang to occur. So only Python 2.5.1 built under Fr= eeBSD 7 works to get around this issue with ntpd on FreeBSD 8. That means o= ne potential solution is to downgrade to this 2.5.1, but we have other libr= aries targeted to work with Python 2.6 and we don't really want to downgrad= e all these associated libraries. If anyone has any clues at all as to what is causing this issue, I'd apprec= iate the feedback. Here's the code that reproduces this behavior. #! /usr/bin/env python import os import threading class RunProc(threading.Thread): def __init__(self, cmd): threading.Thread.__init__(self) self.cmd =3D cmd def run(self): os.system(self.cmd) def main(): RunProc("/usr/sbin/ntpd -g -q").start() if __name__ =3D=3D "__main__": main()