From owner-freebsd-arch@FreeBSD.ORG Wed Jul 14 00:23:01 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1F81516A4CE for ; Wed, 14 Jul 2004 00:23:01 +0000 (GMT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8DF5643D1F for ; Wed, 14 Jul 2004 00:23:00 +0000 (GMT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.11/8.12.11) with ESMTP id i6E0MhpD070751; Tue, 13 Jul 2004 20:22:44 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i6E0MhA2070747; Tue, 13 Jul 2004 20:22:43 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Tue, 13 Jul 2004 20:22:43 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: "M. Warner Losh" In-Reply-To: <20040713.181419.82427717.imp@bsdimp.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org cc: phk@phk.freebsd.dk Subject: Re: newbus integration of MOD_QUIESCE X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jul 2004 00:23:01 -0000 On Tue, 13 Jul 2004, M. Warner Losh wrote: > : MOD_STRONGUNLOAD - Unload even though you're in use. Detach the driver, > : deadfs the file system, wither the geom, sever the sockets, etc. May > : cause disruption, but may also veto, depending on the subsystem, > : especially if the subsytem has no way to notify its consumers of > : impending doom. Can be vetoed, but try harder before vetoing. Some > : subsystems might always return EBUSY for this if there's really no way > : to express "undesirable departure" upwards. > > This is tue current MOD_UNLOAD Well, sort of. It might be well true of device drivers, but in the world of synthetic network interfaces, etc, the MOD_WEAKUNLOAD semantics apply. > : MOD_QUIESCE - Attempt MOD_WEAKUNLOAD, and if that fails, ask the module to > : start draining in some form. I'm a bit unclear on quite what's > : intended, but this seems to be less atomic notion than "unload, or > : don't" at various points on the spectrum. I.e., it kicks off a state > : transition in what is likely a slightly poorly defined state machine. > : Right now, the state machine is "Not loaded", "Loaded", and we use a > : lock to prevent intermediate states from colliding. > > This is the heart of my questions about MOD_QUIESCE. For some module systems this is free, or at least designed in. In the MAC Framework, an unload request is treated as a blocking quiesce request. A module can block unload if it wants to, but by default, the MAC Framework will wait (possibly a long time) for the framework to get to a point where it can safely unload. I.e., it assumes unload can be made atomic, if you wait long enough. With others, that's clearly not the case. You identified a couple of interesting edge cases, such as "State transition driver into 'shutting down', now do something that requires the driver", but I think there's some other big problems with the state machine. What if an application needs the functionality -- does it ask the driver to change its mind, wait for the driver to unload so it can load it again, or can the unload and load overlap? How long might it have to wait? "Drain, oh, unless I need it" may mean one thing for a driver, but something else for a file system. The nice thing about the current model is that it supports two positions: on, and off. This raises another edge case -- the case today where we load a driver, there's a linking or initialization error, and we leave it loaded in a broken state. It becomes more apparent when you load a driver with a mutex initialization in it's load routine. If you've never given it a try, try loading if_tap.ko into a kernel with the tap interface compiled in. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Principal Research Scientist, McAfee Research