From owner-freebsd-hackers@FreeBSD.ORG Wed Aug 24 06:45:28 2005 Return-Path: X-Original-To: freebsd-hackers@FreeBSD.org Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5B22016A41F; Wed, 24 Aug 2005 06:45:28 +0000 (GMT) (envelope-from imp@bsdimp.com) Received: from harmony.village.org (vc4-2-0-87.dsl.netrack.net [199.45.160.85]) by mx1.FreeBSD.org (Postfix) with ESMTP id ECD8D43D45; Wed, 24 Aug 2005 06:45:27 +0000 (GMT) (envelope-from imp@bsdimp.com) Received: from localhost (localhost.village.org [127.0.0.1]) by harmony.village.org (8.13.3/8.13.3) with ESMTP id j7O6ijlb080424; Wed, 24 Aug 2005 00:44:45 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Wed, 24 Aug 2005 00:44:45 -0600 (MDT) Message-Id: <20050824.004445.74670884.imp@bsdimp.com> To: lists@nbux.com From: Warner Losh In-Reply-To: <430C042E.70009@nbux.com> References: <430C042E.70009@nbux.com> X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.0 (harmony.village.org [127.0.0.1]); Wed, 24 Aug 2005 00:44:46 -0600 (MDT) Cc: deischen@FreeBSD.org, freebsd-hackers@FreeBSD.org Subject: Re: nagios and freebsd threads issue : help please ... X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2005 06:45:28 -0000 Thanks for forwarding the reply. First, I'll note that the response contains highly charged, inflamitory language. I've tried my best to ignore that language and focus on the actual issues. Second, this posting demonstrates a fundamental confusion between thread-safe and async-safe. That is the root of the problem in the communication. Thread-safe functions are a dime a dozen and relatively easy to write. async-safe functions are very rare and much harder to do useful things with. I've tried to explain the difference below using fgets() as an example of the difficulties. > fgets() must also be async-safe, since it's passed its storage-buffer > from the calling function. It can contain races if several threads (or > programs for that matter) tries to read FIFO's at the same time or are > trying to store things to the same piece of memory, but that's neither > new, strange or in any way non-obvious. Obviously, fgets() relies on > lower-level IO code which must be thread-safe (read() in this case) on > account of them being syscalls inside multitasking kernels. fgets need not be async-safe, but it does need to be thread-safe. When one fork after pthread_create, one may only call async-safe functions. The weaker requirements of thread safety can be shown to not necessarily be async safe. If two different threads call fgets(), mutexes will keep one thread from running if the other is in the middle of changing the FILE * internal state. However, if that thread is interrupted by the scheduler with the mutex held, and fork() is called, then the new copy of the address space will still have that mutex held. Any attempt by this new process, with its own address space, to acquire the lock is doomed to failure. Since the parent and child execute in different address spaces, there is no way for a thread that does not exist in the child to unlock the locked mutex. Normally this happens like so: Thread A Thread B fgets(fp, b1, 10); lock fp's mutex copy 5 available bytes into b1 fgets(fp, b2, 10) try lock fp's mutex unlock fp's mutex return attempt to lock finishes b2 can be updated unlock mutex. However, in the fork case: Thread A Thread B fgets(fp, b1, 10); lock fp's mutex copy 5 available bytes into b1 fork() fgets(fp, b2, 10) try lock fp's mutex At this point B', the only thread in the child, will never be able to grab this lock because A exists only in the parent and the parent/child have independent address spaces. While the above example is not what nagios is doing, it illustrates the point. There are some functions that necessarily touch global state. These functions need to coordinate that touching of state. If one of the is interrupted with locks held, then all bets are off of a program forks and the threads holding those locks can never unlock them. > >> The list of async-signal-safe functions > >> are here: http://www.opengroup.org/onlinepubs/009695399/nframe.html > >> The restriction on fork() is here (20th bullet down): > >> http://www.opengroup.org/onlinepubs/009695399/nframe.html > > Both of those links point to the same document, which is just the > frameset for the navigation-frames. > > For async-safe functions, this is the proper url; > http://www.opengroup.org/onlinepubs/009695399/functions/xsh_chap02_09.html#tag_02_09_01 This reference is for thread-safe functions. You are confusing thread-safe and async-safe. The correct url for async-safe is http://www.opengroup.org/onlinepubs/009695399/functions/xsh_chap02_04.html#tag_02_04_03 >> The following table defines a set of functions that shall be either >> reentrant or non-interruptible by signals and shall be >> async-signal-safe. Therefore applications may invoke them, without >> restriction, from signal-catching functions: >> Notice that this list is very short, and there are many functions that one would think should be on here, but in fact aren't. > For the fork() specification, the doc is here; > http://www.opengroup.org/onlinepubs/009695399/functions/fork.html ... > "A process shall be created with a single thread. If a multi-threaded > process calls fork(), the new process shall contain a replica of the > calling thread and its entire address space, possibly including the > states of mutexes and other resources. Consequently, to avoid errors, > the child process may only execute async-signal-safe operations until > such time as one of the exec functions is called. Notice here it says specifically 'async-sngial-safe operations' not 'thread-safe' operations. The standard explicitly calls attention to the difficulties and differences between these two types of functions. > This is funny, because nagios apparently runs properly on Linux, HPUX, > Solaris, Irix, AIX and Tru64. To me that seems to indicate that Nagios > is very portable indeed and that the BSD fellows somehow botched it. I > might be wrong, but... Just because it works doesn't make it standards conforming. Maybe there's some simple extension that can be implemented to help the situation. However, the inflamitory language gets very much in the way of having a technical discussion. Warner