From owner-freebsd-arch@FreeBSD.ORG Mon Aug 30 17:48:46 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1438016A4CE for ; Mon, 30 Aug 2004 17:48:46 +0000 (GMT) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9C4CB43D54 for ; Mon, 30 Aug 2004 17:48:45 +0000 (GMT) (envelope-from scottl@freebsd.org) Received: from [192.168.0.12] (g4.samsco.home [192.168.0.12]) (authenticated bits=0) by pooker.samsco.org (8.12.11/8.12.10) with ESMTP id i7UHmB7L066967; Mon, 30 Aug 2004 11:48:12 -0600 (MDT) (envelope-from scottl@freebsd.org) Message-ID: <4133682D.3000403@freebsd.org> Date: Mon, 30 Aug 2004 11:47:25 -0600 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7) Gecko/20040514 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "M. Warner Losh" References: <41334C3B.4070101@freebsd.org> <20040830.102606.130865377.imp@bsdimp.com> <20040830.111428.56562495.imp@bsdimp.com> In-Reply-To: <20040830.111428.56562495.imp@bsdimp.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, hits=0.0 required=3.8 tests=none autolearn=no version=2.63 X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on pooker.samsco.org cc: sah@softcardsystems.com cc: freebsd-arch@freebsd.org Subject: Re: splxxx level? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Aug 2004 17:48:46 -0000 M. Warner Losh wrote: > In message: <20040830.102606.130865377.imp@bsdimp.com> > "M. Warner Losh" writes: > : In message: <41334C3B.4070101@freebsd.org> > : Scott Long writes: > : : Sam wrote: > : : > : : > Hello - > : : > > : : > I'm almost to testing on my AoE driver for 4.x and have > : : > a question about interrupt priority levels. > : : > > : : > There are currently three entry points into the driver: > : : > > : : > a) strategy routine > : : > b) network frame reception routine > : : > c) timer rexmit routine > : : > > : : > Any of the three can diddle with the device structure > : : > and thusly I need to ensure they're not running simultaneously. > : : > For example, the network reception can cause a buf to be completed > : : > and the rexmit timer can cause a buf to be failed. > : : > > : : > So, what kind of contexts are the callout, strategy, and > : : > network soft interrupt called in? Which splxxx will give > : : > one of them exclusive access to whatever they need? > : : > > : : > Just as a reality check -- I am thinking about this correct, right? > : : > > : : > Cheers, > : : > > : : > Sam > : : > > : : > : : With 4.x, only one CPU can be in the kernel at a time. You won't have > : : to worry about multiple processes trying to get into strategy at the > : : same time and whatnot. However, you can be preempted by your interrupt > : : handler or by a timeout or by a software interrupt like the netisr. I > : : don't remember if your driver is for a specific piece of hardware or if > : : it's a generic layer that sits in between the network interface and the > : : block layer. If it's for dedicated hardware then you'll need to define > : : a interrupt type in bus_setup_intr() and use that type for the spl > : : (i.e. INTR_TYPE_NET translates to splnet(), INTR_TYPE_BIO translates to > : : splbio(), etc). > : : > : : The safe way to go is to protect all of your critical code sections with > : : the appropriate spl level regardless. spls are very cheap and can be > : : set/reset from an interrupt context so there is little penalty in using > : : them liberally at first and then narrowing them down later. Just make > : : sure that you don't leak an spl references, and don't hold an spl for so > : : long that it creates priority inversions. Since the only interrupts and > : : timeouts that you'll likely be dealing with are network related, > : : splnet() is probably the right one to use. > : > : splimp() is what you want to use, not splnet(). Yes, this is > : confusing, but it appears to be what all the other network drivers > : use. None of them are using splnet() that I could find. splimp() is > : also used by the mbuf routines to protect mbuf operations. > : > : splnet() is a list of the software interrupts that are network > : drivers. > : > : splimp() is splnet() plus the hardware interrupts, so is more > : appropriate to block things called from the driver. Especially one > : that's described as having timeouts. If it is a network driver, you > : might consider using the timeout functionality in the net stack as > : opposed to the callout functions. This makes it possible to have > : almost the entire driver w/o doing any spls (most of the network > : drivers in 4 don't do spl at all, except for entry points that are > : outside the scope of the network/interrupt entry points, eg > : device_suspend). > > Ah, just saw 'AoE' in the reply. > > This likely means that you are writing a network stack that's glued > into the system with the strategy routine. Inside the strategy > routine, splbio() is likely what you need to hold while dealing with > the struct buf that's passed into that routine. I'm not sure what > spl level you need to be at to call into the network code. I'm > thinking it is splimp(), but I'm not 100% sure about that. Stevens > will likely be a good resource for 4.x. > > You may have to define a software interrupt to pass the packets to the > network code to make the spls work out correctly. > > Warner No, splbio() is not explicitely needed in this case. You'll be dealing with a bioq that might need protection, but you have total control over that. There is nothing else in the block layer that will interrupt you, not like the netisr in the network stack or the camisr in the CAM layer. Scott