From owner-freebsd-arch Thu Sep 26 8: 8:49 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5DD1C37B47B for ; Thu, 26 Sep 2002 08:08:44 -0700 (PDT) Received: from mail.speakeasy.net (mail12.speakeasy.net [216.254.0.212]) by mx1.FreeBSD.org (Postfix) with ESMTP id A964043E86 for ; Thu, 26 Sep 2002 08:08:43 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Received: (qmail 26402 invoked from network); 26 Sep 2002 15:08:43 -0000 Received: from unknown (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) by mail12.speakeasy.net (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 26 Sep 2002 15:08:43 -0000 Received: from laptop.baldwin.cx (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.5/8.12.5) with ESMTP id g8QF8fBv091921; Thu, 26 Sep 2002 11:08:41 -0400 (EDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.5.2 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200209260943.g8Q9hURP072845@grimreaper.grondar.org> Date: Thu, 26 Sep 2002 11:08:43 -0400 (EDT) From: John Baldwin To: Mark Murray Subject: RE: [patch] module-with-thread exit routine. Cc: arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 26-Sep-2002 Mark Murray wrote: > Hi > > If an unloadable module has a thread (like random.ko has), then > unloading the module is problematic because of the race between > the module termination and the unload. > > The thread must "die" in a part of the code that will not be > unloaded. > > The random device has its own code to do this, but I think this > is better put with the kthread stuff. > > Attached is a patch to the kthread code. I've been running this > for at least three months. > > Comments? You don't need it. Look at the crash module I have. It has a thread and I committed an extra wakeup() inside of exit1() for kthread's a while ago to handle just this case. Here's an excerpt from the crash module code: In the kthread's main loop (it usually gets events via a sysctl from userland): static void crash_thread(void *arg) { int ev; while (1) { mtx_lock(&event_mtx); while ((ev = event) == 0) cv_wait(&event_cv, &event_mtx); event = 0; mtx_unlock(&event_mtx); ... switch (ev) { case -1: mtx_lock(&Giant); kthread_exit(0); break; ... } } } Here's the unload() function that the module calls for MOD_UNLOAD: static int unload(void *arg) { mtx_lock(&event_mtx); event = -1; cv_signal(&event_cv); msleep(kthread, &event_mtx, PWAIT, "crshun", 0); mtx_unlock(&event_mtx); mtx_destroy(&event_mtx); cv_destroy(&event_cv); ... return (0); } The actual wakeup in exit1 is from: revision 1.129 date: 2001/06/27 06:15:44; author: jhb; state: Exp; lines: +13 -16 ... - When a kthread exits, do a wakeup() on its proc pointers. This can be used by kernel modules that have kthreads and want to ensure they have safely exited before completely the MOD_UNLOAD event. and is at: /* * If this is a kthread, then wakeup anyone waiting for it to exit. */ if (p->p_flag & P_KTHREAD) wakeup(p); PROC_UNLOCK(p); The trick is to avoid missing the wakeup. The way I do it above is to make sure and sleep on the kthread address before and drop event_mtx at the same time so that I am asleep before the kthread gets to run. Since the wakeup() is under the proc lock another valid way that might be easier to accomplish for some kthreads would be something like this: static int unload(void *arg) { mtx_lock(&event_mtx); event = -1; cv_signal(&event_mtx); PROC_LOCK(kthread); mtx_unlock(&event_mtx); msleep(kthread, &kthread->p_mtx, PWAIT | PDROP, "crshun", 0); ... } Well, except that you need to use PDROP in that case since a kthread might have already been harvested by init() and not have a proc mutex around any longer to release when you get woken up. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message