From owner-freebsd-arch@FreeBSD.ORG Mon Aug 30 16:26:35 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7E51416A4CE; Mon, 30 Aug 2004 16:26:35 +0000 (GMT) Received: from harmony.village.org (rover.village.org [168.103.84.182]) by mx1.FreeBSD.org (Postfix) with ESMTP id 98ABF43D60; Mon, 30 Aug 2004 16:26:34 +0000 (GMT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.11/8.12.11) with ESMTP id i7UGQ0HQ052687; Mon, 30 Aug 2004 10:26:00 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Mon, 30 Aug 2004 10:26:06 -0600 (MDT) Message-Id: <20040830.102606.130865377.imp@bsdimp.com> To: scottl@freebsd.org From: "M. Warner Losh" In-Reply-To: <41334C3B.4070101@freebsd.org> References: <41334C3B.4070101@freebsd.org> X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: sah@softcardsystems.com cc: freebsd-arch@freebsd.org Subject: Re: splxxx level? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Aug 2004 16:26:35 -0000 In message: <41334C3B.4070101@freebsd.org> Scott Long writes: : Sam wrote: : : > Hello - : > : > I'm almost to testing on my AoE driver for 4.x and have : > a question about interrupt priority levels. : > : > There are currently three entry points into the driver: : > : > a) strategy routine : > b) network frame reception routine : > c) timer rexmit routine : > : > Any of the three can diddle with the device structure : > and thusly I need to ensure they're not running simultaneously. : > For example, the network reception can cause a buf to be completed : > and the rexmit timer can cause a buf to be failed. : > : > So, what kind of contexts are the callout, strategy, and : > network soft interrupt called in? Which splxxx will give : > one of them exclusive access to whatever they need? : > : > Just as a reality check -- I am thinking about this correct, right? : > : > Cheers, : > : > Sam : > : : With 4.x, only one CPU can be in the kernel at a time. You won't have : to worry about multiple processes trying to get into strategy at the : same time and whatnot. However, you can be preempted by your interrupt : handler or by a timeout or by a software interrupt like the netisr. I : don't remember if your driver is for a specific piece of hardware or if : it's a generic layer that sits in between the network interface and the : block layer. If it's for dedicated hardware then you'll need to define : a interrupt type in bus_setup_intr() and use that type for the spl : (i.e. INTR_TYPE_NET translates to splnet(), INTR_TYPE_BIO translates to : splbio(), etc). : : The safe way to go is to protect all of your critical code sections with : the appropriate spl level regardless. spls are very cheap and can be : set/reset from an interrupt context so there is little penalty in using : them liberally at first and then narrowing them down later. Just make : sure that you don't leak an spl references, and don't hold an spl for so : long that it creates priority inversions. Since the only interrupts and : timeouts that you'll likely be dealing with are network related, : splnet() is probably the right one to use. splimp() is what you want to use, not splnet(). Yes, this is confusing, but it appears to be what all the other network drivers use. None of them are using splnet() that I could find. splimp() is also used by the mbuf routines to protect mbuf operations. splnet() is a list of the software interrupts that are network drivers. splimp() is splnet() plus the hardware interrupts, so is more appropriate to block things called from the driver. Especially one that's described as having timeouts. If it is a network driver, you might consider using the timeout functionality in the net stack as opposed to the callout functions. This makes it possible to have almost the entire driver w/o doing any spls (most of the network drivers in 4 don't do spl at all, except for entry points that are outside the scope of the network/interrupt entry points, eg device_suspend). Warner