From owner-freebsd-bugs Wed Aug 12 01:50:06 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id BAA22167 for freebsd-bugs-outgoing; Wed, 12 Aug 1998 01:50:06 -0700 (PDT) (envelope-from owner-freebsd-bugs@FreeBSD.ORG) Received: from freefall.freebsd.org (freefall.FreeBSD.ORG [204.216.27.21]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id BAA22138 for ; Wed, 12 Aug 1998 01:50:03 -0700 (PDT) (envelope-from gnats@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.8.8/8.8.5) id BAA25552; Wed, 12 Aug 1998 01:50:00 -0700 (PDT) Received: from tyree.iii.co.uk (tyree.iii.co.uk [195.89.149.230]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id BAA21470 for ; Wed, 12 Aug 1998 01:44:42 -0700 (PDT) (envelope-from nik@iii.co.uk) Received: from carrig.strand.iii.co.uk (carrig.strand.iii.co.uk [192.168.7.25]) by tyree.iii.co.uk (8.8.8/8.8.8) with ESMTP id JAA00902 for ; Wed, 12 Aug 1998 09:44:17 +0100 (BST) Received: (from nik@localhost) by carrig.strand.iii.co.uk (8.8.8/8.8.7) id JAA13453; Wed, 12 Aug 1998 09:43:33 +0100 (BST) Message-Id: <199808120843.JAA13453@carrig.strand.iii.co.uk> Date: Wed, 12 Aug 1998 09:43:33 +0100 (BST) From: Nik Clayton Reply-To: nik@iii.co.uk To: FreeBSD-gnats-submit@FreeBSD.ORG X-Send-Pr-Version: 3.2 Subject: kern/7586: Threads bug in libc_r preventing MySQL from shutting down Sender: owner-freebsd-bugs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >Number: 7586 >Category: kern >Synopsis: [PATCH] Threads bug in libc_r preventing MySQL from shutting down >Confidential: no >Severity: serious >Priority: high >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Wed Aug 12 01:50:00 PDT 1998 >Last-Modified: >Originator: Nik Clayton >Organization: Dazed and confused >Release: FreeBSD 2.2.6-STABLE i386 >Environment: There's a thread in the -stable mailing list at the moment about a problem with threads in -current that prevents MySQL from shutting down. I don't understand the problem. However, Michael Gile in message-id <199808112043.NAA00879@hub.freebsd.org> has just reposted a message that was sent by the MySQL developer that contains a description of the problem and patches. I see no reason why this message won't also sit in someone's mail queue and slowly rot, so I'm submitting it as a PR so that at least it gets stuck in the system, 'officially' :-) THIS PR INCLUDES PATCHES! However, I'm not in a position to test them (or even understand them much). >Description: Date: Tue, 11 Aug 1998 16:19:23 -0400 To: Mike Smith From: "Michael R. Gile" Subject: Re: threads bug [and mysql] (large) At 03:44 PM 8/10/98 -0700, Mike Smith wrote: >How-To-Repeat: >Fix: >Audit-Trail: >Unformatted: >> At 12:29 PM 8/10/98 -0700, Tom wrote: >> > >> >On Sun, 9 Aug 1998, Michael R. Gile wrote: >> > >> >> There appears to be a recurring bug in libc_r that is causing >> >> Mysql not to respond to a shutdown. The author claims that >> >> he helped the development group fix this bug some time ago, >> >> but that it has recurred. Any suggestions on where to find this >> >> documentation would be helpful. >> > >> > I don't believe this bug was ever fixed. You can look at the cvslogs to >> >see all changes to libc_r. >> >> i tend to agree, but monty insists that he mailed the necessary fixes >> to the freebsd team months ago. > >To whom? The correct thing do to would be to file a PR, and then use >the PR when referring to this. It wouldn't be surprising if the >patches are rotting in someone's inbox where nobody else can get at >them. >-- i have attached all the relevant documentation that the author had in his sentmail box. I wouldn't normally post such a large message, but i wanted to make sure that this stuff made it into the mail archives. If anyone else wants to help me try to sort through it and find what the current signal problem with mysql & freebsd, then let me know. -mike ============== ----------- To: jb@cimlogic.com.au cc: pauls@locust.etext.org Subject: Freebsd 3.0 threads Reply-To: monty@analytikerna.se Hi. Sorry to mail you directly, but I got your name from pauls@locust.etext.org and he said that you usually answer questions about the FreeBSD pthread implementation. I am the author of MySQL, a 'almost free' SQL implementation. MySQL uses Posix threads and is currently running with native threads on Solaris, Linux (with Linuxthreads), AIX, IRIX and OSF1. I also maintain much updated version of MIT threads and with this MySQL is known to run on SunOS, FreeBSD 2.0 and BSD 2.#. This MIT thread version is distrubted with the MySQL source distribution. I am now trying to help pauls out on FreeBSD 3.0 but I have now got into some trouble and I thought that maybe you can help me on the right track. 1) There isn't a sched.h file. pthread.h have prototypes for some functions that uses sched_param but it's a little problem to use them now :) 2) I looked into /usr/src/lib/libc_r/uthread/pthread_priv.h and found the following: ---- struct sched_param { int prio; /* Should be named sched_priority */ void *no_data; }; enum schedparam_policy { SCHED_RR, SCHED_IO, SCHED_FIFO, SCHED_OTHER }; ---- Shouldn't this be in a sched.h file ? It would also be nice if 'prio' would be renamed 'sched_priority'. I think this is the standard slot name (at least all other pthread implementation uses this) 3) The following functions are not defined: pthread_setprio(), pthread_attr_setprio(),pthread_attr_setscope() When do you think these will be available? 4) I have now fixed workarounds for the above cases. All code compiles and links clean but when run a test program, pthread_create doesn't start a thread. Should threads work at all in FreeBSD 3.0 ? Yours, Monty ************************************ To: pauls@locust.etext.org cc: jb@cimlogic.com.au Subject: Info about mysql porting and fixed bugs in the libc_r library Reply-To: monty@analytikerna.se John Birrell hasn't answered my yet about my problems with the pthread library (I assume he is very busy like the rest of us). Because I couldn't get -lpthread to work I compiled the source version with -g and some things started to work. I can only assume that the libc_r that you have installed isn't the newest one or then something has gone wrong in your installation. I have now started getting some things to work. Here follows a list of bugs I have fixed in the pthread library. I really hope that John will have time to at least examine them and hopefully add some of them to the next distribution. I haven't yet get everything to work, so there will be more patches. Yours, Monty ------------- Bugs found in the Freebsd 3.0 pthread implementation: ---------------------------------------- - sigsuspend should replace the callers mask to the given mask, not add the given mask to the old mask. The signals should also be handled and not as with SIGWAIT where the signal function is ignored. Fix: uthread_sigsuspend.c: /* Save the current sigmal mask: */ oset = _thread_run->sigmask; ! /* Replace the caller's mask with the current one: */ ! _thread_run->sigmask= *set; /* Patched by Monty */ /* Wait for a signal: */ ! _thread_kern_sched_state(PS_SIGSUSPEND, __FILE__, __LINE__); uthread_kern.c: case PS_FDR_WAIT: case PS_FDW_WAIT: case PS_SLEEP_WAIT: + case PS_SIGSUSPEND: And of course add a definition for PS_SIGSUSPEND. ---------------------------------------- - Signals that are not waited for with sigwait() should be handled before they are returned. Fix: - Add a separate 'sigwait_waiting' mask in the thread structure and handle the signal in uthread_kern.c for SIGWAIT if the signal isn't in the sigwait_waiting mask. ---------------------------------------- - It almost impossible to debug threads because all signals are masked in the threads library. Fix: uthread_kern.c, line 53 - static sigset_t sig_to_block = 0xffffffff; + static sigset_t sig_to_block = ~((1 << 5) | (1 << 6) | (1 << 4)); ---------------------------------------- The sigwait function is in the library but it isn't exported. Fix: Add uthread_sigwait.c to libc_r/uthread/Makefile.inc Fix: Add uthread_sigwait.c to libc_r/uthread/Makefile.inc ---------------------------------------- sigwait returns 'random' value: Fix: uthread_sigwait.c, line 42: - int ret; + int ret=EINTR; ---------------------------------------- *********************************' >From monty Sun Aug 10 23:52:21 1997 Date: Sun, 10 Aug 97 23:52:21 EET DST From: To: Paul Southworth Subject: Re: Info about mysql porting and fixed bugs in the libc_r library In-Reply-To: References: <199708101719.UAA21091@monty.pp.sci.fi> Reply-To: monty@analytikerna.se >>>>> "Paul" == Paul Southworth writes: Paul> My mail sent to " jb@cimlogic.com.au" has been bouncing -- timed out. Paul> Perhaps there is something wrong with that site. I took the questions you Paul> asked him and sent them to the freebsd-hackers mailing list. Paul> I can send your patches there as well if that's OK with you. Here is the 'newest' compleate patch that appears to work: (At least sigwait() now works according to Posix and MySQL really neads this): Please mail a copy of this to the freebsd-hackers list. This mails ends with a compleat patch that fixes everything except the pthread.h problem with struct sched_param, but I was able to handle this with configure. Yours, Monty ------- Bugs found in Freebsd's pthread implementation: ---------------------------------------- - sigsuspend should replace the callers mask to the given mask, not add the given mask to the old mask. The signals should also be handled and not as with SIGWAIT where the signal function is ignored. Fix: uthread_sigsuspend.c: /* Save the current sigmal mask: */ oset = _thread_run->sigmask; ! /* Replace the caller's mask with the current one: */ ! _thread_run->sigmask= *set; /* Patched by Monty */ /* Wait for a signal: */ ! _thread_kern_sched_state(PS_SIGSUSPEND, __FILE__, __LINE__); uthread_kern.c: case PS_FDR_WAIT: case PS_FDW_WAIT: case PS_SLEEP_WAIT: + case PS_SIGSUSPEND: And of course add a definition for PS_SIGSUSPEND. ---------------------------------------- - It almost impossible to debug threads because all signals are masked in the threads library. Fix: uthread_kern.c, line 53 - static sigset_t sig_to_block = 0xffffffff; + static sigset_t sig_to_block = ~((1 << 5) | (1 << 6) | (1 << 4)); ---------------------------------------- - The sigwait function is in the library but it isn't exported. - A signal to a thread in sigwait doesn't set the thread 'runnable'. This is because the arguments to sigwait() is put to sigmask which doesn't work because the signal must be sent to _thread_signal(). - Signals that are not waited for with sigwait() should be handled before they are returned. - sigwait returns 'random' value: Fix: Add uthread_sigwait.c to libc_r/uthread/Makefile.inc Change sigwait to: In pthread_privat.h PS_DEAD, ! PS_STATE_MAX, + PS_SIGSUSPEND }; ---- /* * Current signal mask and array of pending signals. */ ! sigset_t sigmask,sigwaitmask; int sigpend[NSIG]; uthread_kern.c: Change: /* Waiting on a signal: */ case PS_SIGWAIT: /* Change the state of the thread to run: */ PTHREAD_NEW_STATE(pthread,PS_RUNNING); /* Return the signal number: */ pthread->signo = sig; /* Flag the signal as dealt with: */ done = 1; break; } to /* Waiting on a signal: */ case PS_SIGWAIT: /* Change the state of the thread to run: */ PTHREAD_NEW_STATE(pthread,PS_RUNNING); if (sigismember(&pthread->sigwaitmask, sig)) { /* Return the signal number: */ pthread->signo = sig; /* Flag the signal as dealt with: */ done = 1; } break; ---------------------------------------- Because one needs to access the struct sched_param slots this must be defined when one includes pthread.h. One should also change the slot name from 'prior' to 'sched_priority' according to Posix. Fix: Move definition of structs sched_param from pthread_private.h to pthread.h. Change: struct sched_param { int prio; void *no_data; }; to struct sched_param { int sched_priority; void *no_data; }; Empty definition of the schedparam functions would also be nice.. ******************************************************************************* Complete patch of the above with some small additions for sigsuspend: diff of Makefile.inc in /usr/src/lib/libc_r/uthread/ and . 85a86 > uthread_sigwait.c \ diff of pthread_private.h in /usr/src/lib/libc_r/uthread/ and . 243c243,244 < PS_STATE_MAX --- > PS_STATE_MAX, > PS_SIGSUSPEND 344c345 < sigset_t sigmask; --- > sigset_t sigmask,sigwaitmask; diff of uthread_kern.c in /usr/src/lib/libc_r/uthread/ and . 53c53,54 < static sigset_t sig_to_block = 0xffffffff; --- > /* Changed by monty for easy debugging of pthreads with gdb */ > static sigset_t sig_to_block = ~((1 << 5) | (1 << 6) | (1 << 4)); 893a895 > case PS_SIGSUSPEND: 909a912,913 > if (sigismember(&pthread->sigwaitmask, sig)) > { 914a919 > } 1100a1106 > case PS_SIGSUSPEND: 1449a1456 > case PS_SIGSUSPEND: diff of uthread_sigsuspend.c in /usr/src/lib/libc_r/uthread/ and . 50,51c50,51 < /* Combine the caller's mask with the current one: */ < _thread_run->sigmask |= *set; --- > /* Replace the caller's mask with the current one: */ > _thread_run->sigmask= *set;/* Patched by Monty */ 54c54 < _thread_kern_sched_state(PS_SIGWAIT, __FILE__, __LINE__); --- > _thread_kern_sched_state(PS_SIGSUSPEND, __FILE__, __LINE__); diff of uthread_sigwait.c in /usr/src/lib/libc_r/uthread/ and . 38a39,40 > /* This version of sigwait is modifed by monty@tcx.se to work properly */ > 42c44 < int ret; --- > int ret=0; 49,51d50 < /* Save the current sigmal mask: */ < oset = _thread_run->sigmask; < 53c52,56 < _thread_run->sigmask |= *set; --- > _thread_run->sigmask &= ~*set; > _thread_run->sigwaitmask = *set; > > /* Clear for easy error check */ > _thread_run->signo=0; 62a66,68 > > /* Reset the sigwait mask for debugging */ > _thread_run->sigwaitmask = 0; **************************************************************** From: John Birrell To: edmond@shaman.lycaeum.org (Andrew N. Edmond) Cc: jb@cimlogic.com.au, monty@tcx.se, ross@flyingcroc.com Subject: Re: is libc_r broken in 980520-SNAP? Date: Wed, 17 Jun 1998 15:22:39 +1000 (EST) Andrew N. Edmond wrote: > We are specifically looking for something for monty to be able to put in > his INSTALL-SOURCE doc on how to compile MySQL 3.22.1-alpha and beyond on > FreeBSD 3.0-current past march or so... (including optimizing it with > egcc -O6). Monty also has assembler code for i386 platforms for the > string functions that we are trying to compile in here as well... > (--enable-assembler) > > ... you'd be willing to help with this? MySQL was one of the packages mentioned when people complained about performance differences between libc_r and MIT pthreads. An attempt has been made to address those issues, but nobody has commented on any performance change. Since I'm also working on FreeBSD/Alpha, I'm choosing packages to checkout with that too. MySQL might be worth trying provided that the tests are good enough to indicate if the package actually works. Is the FreeBSD i386 asm code for the string functions no good? ***************************' From: John Birrell To: tom@uniserve.com (Tom) Cc: jb@cimlogic.com.au, edmond@shaman.lycaeum.org, stable@FreeBSD.ORG, monty@tcx.se Subject: Re: is libc_r broken in 980520-SNAP (and stable?)? Date: Thu, 18 Jun 1998 07:55:55 +1000 (EST) Tom wrote: > One thing that confuses me is that how does libc_r from 2.2.6-stable and > -current compare these days? They used to be identical, but now that > -current is getting kernel support for threads and alpha stuff, I assume > that they must be quite different now. Primarily, is libc_r from > 2.2.6-stable just as usable as libc_r from current? Highwind Software > (www.highwind.com) has a beta version of their news software that it is > statically linked to a libc_r with fixes they say they got from you. > Would this be a -stable libc_r or a -current libc_r that they are using? Highwind are only using 2.2.6 with libc_r built from the RELENG_2_2 branch. The -current libc_r differs substantially from the -stable implementation because it supports just the POSIX style of signal handling where there are one set of signal handlers declared for the threaded process, not one set per thread. This allows -current to avoid blocking signals as part of it's scheduler. This is what drains performance for some applications. The version of libc_r in the RELENG_2_2 contains fixes for all bugs that people have reported. In Highwind's case, they have code that did things like read/write with zero bytes; and write to a read only file. These caused libc_r to hang waiting on a file descriptor to become ready when it never would. Most software doesn't do this sort of thing. It seems to come from C++ code. 8-) Kernel threads in 3.0 actually have nothing to do with libc_r. They require a libpthread and a thread-aware libc. The -current libc_r implementation works on FreeBSD/Alpha. ************************** From: John Birrell To: tom@uniserve.com (Tom) Cc: jb@cimlogic.com.au, edmond@shaman.lycaeum.org, stable@FreeBSD.ORG, monty@tcx.se Subject: Re: is libc_r broken in 980520-SNAP (and stable?)? Date: Thu, 18 Jun 1998 08:27:37 +1000 (EST) Tom wrote: > Are you aware of the problems with MySQL? I have 2.2.6-stable of May > 21, and a signal/alarm test program (thr_alarm) hangs. No, I wasn't aware of that. > Also, how should -stable threaded applications be compiled and linked? > You mentioned before that "-pthread" should be used rather than linking > with libc_r? Does that apply to -stable too? I think -stable thread apps > need to be compiled with "-D_THREAD_SAFE" and linked with "-lc_r" but I'm > not sure anymore. -stable thread apps need to be compiled with -D_THREAD_SAFE to get a thread-safe errno. -current does not require that. Linking both -current and -stable uses -pthread to get the right libraries because people don't seem to be able to use -nostdlib. I'm going to try building MySQL on -current this morning. -- John Birrell - jb@cimlogic.com.au; jb@freebsd.org http://www.cimlogic.com.au/ *******************************************' After this I haven't got anything from 'jb@cimlogic.com.au'. I suspect that the new alarm handling that he mentions in the second last mail has breaked something. Regards, Monty ====================================================== Michael Gile gilem@wsg.net President (518)435-0682 Web Services Group http://www.wsg.net/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-bugs" in the body of the message