From owner-freebsd-arch@FreeBSD.ORG Sun Apr 12 11:58:30 2009 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 51F76106564A; Sun, 12 Apr 2009 11:58:30 +0000 (UTC) (envelope-from dfr@rabson.org) Received: from itchy.rabson.org (router.rabson.org [80.177.232.241]) by mx1.freebsd.org (Postfix) with ESMTP id 04C5F8FC1B; Sun, 12 Apr 2009 11:58:29 +0000 (UTC) (envelope-from dfr@rabson.org) Received: from [IPv6:2001:470:909f:1:21b:63ff:feb8:5abc] (unknown [IPv6:2001:470:909f:1:21b:63ff:feb8:5abc]) by itchy.rabson.org (Postfix) with ESMTP id 8DB675C39; Sun, 12 Apr 2009 12:42:48 +0100 (BST) Message-Id: From: Doug Rabson To: Rick Macklem In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Sun, 12 Apr 2009 12:42:24 +0100 References: <49D98461.4000002@elischer.org> X-Mailer: Apple Mail (2.930.3) Cc: Robert Watson , Julian Elischer , freebsd-arch@FreeBSD.org Subject: Re: getting a callback ip address for nfsv4 client X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Apr 2009 11:58:30 -0000 On 6 Apr 2009, at 16:56, Rick Macklem wrote: > > > On Mon, 6 Apr 2009, Robert Watson wrote: > >> On Sun, 5 Apr 2009, Julian Elischer wrote: >> >>>> One possibility is to connect() a socket to the server address, >>>> then call getsockaddr() to query what address the network stack >>>> is using. If the connection completed, then the address could at >>>> least send packets to the server, even if it isn't reachable from >>>> the server. >>>> (Obviously, kernel versions of the above...) >>> we ended up with him doing "whatever the connect code would have >>> done" since as he is in fact inside the kernel, he can just do the >>> same.. >> >> I figured that if the RPC layer was going to make a connection >> anyway, we could just query the RPC layer and ask it what address >> it had used. However, if the decision needs to be made earlier >> than that, then that doesn't make as much sense. This would mean >> NFS could avoid knowing about routes and interfaces, and just know >> about sockets and addresses. >> > > I just took a quick look at Doug's RPC code and the socket doesn't > seem to be exposed to the client side by the rpc layer, so I think > that rtalloc1() is probably easier? I'd also be worried about > knowing when the socket > pointer is valid and in a connected state. (The rtalloc1() code only > uses the > one path rt->rt_ifa->ifa_addr after sanity checking that the > pointers aren't null, so I don't think it will be difficult to > maintain. It does imply > that the code knows the route locking rules, but that just means it > uses RTFREE_LOCKED(rt), so as long as that macro keeps working, it > should > be ok, I think.) While the RPC code doesn't currently expose the socket or local address, I don't think there would be many problems in adding a client control operation to get the local address. Its not a good idea to allow access to the actual socket since that will change over reconnects. In theory, the local address may also change over reconnects but that seems less likely to be a problem. Also, the reconnect machinery doesn't actually create the connection until the first RPC so some care would be needed to account for that. From owner-freebsd-arch@FreeBSD.ORG Sun Apr 12 12:24:25 2009 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AA249106566B for ; Sun, 12 Apr 2009 12:24:25 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 811F88FC0A for ; Sun, 12 Apr 2009 12:24:25 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 381CA46B2C; Sun, 12 Apr 2009 08:24:25 -0400 (EDT) Date: Sun, 12 Apr 2009 13:24:25 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Doug Rabson In-Reply-To: Message-ID: References: <49D98461.4000002@elischer.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Rick Macklem , Julian Elischer , freebsd-arch@FreeBSD.org Subject: Re: getting a callback ip address for nfsv4 client X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Apr 2009 12:24:25 -0000 On Sun, 12 Apr 2009, Doug Rabson wrote: > While the RPC code doesn't currently expose the socket or local address, I > don't think there would be many problems in adding a client control > operation to get the local address. Its not a good idea to allow access to > the actual socket since that will change over reconnects. In theory, the > local address may also change over reconnects but that seems less likely to > be a problem. Also, the reconnect machinery doesn't actually create the > connection until the first RPC so some care would be needed to account for > that. This was pretty much what I had in mind: do an initial no-op, then query the local address from the RPC layer. The question then is, when do you requery the address, and I guess an event is needed from the RPC layer so that the NFS layer knows to re-query and update the server's callback address (I assume that's allowed?). Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-arch@FreeBSD.ORG Sun Apr 12 13:57:28 2009 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B0C0F106566B; Sun, 12 Apr 2009 13:57:28 +0000 (UTC) (envelope-from dfr@rabson.org) Received: from itchy.rabson.org (router.rabson.org [80.177.232.241]) by mx1.freebsd.org (Postfix) with ESMTP id 648038FC16; Sun, 12 Apr 2009 13:57:28 +0000 (UTC) (envelope-from dfr@rabson.org) Received: from [IPv6:2001:470:909f:1:21b:63ff:feb8:5abc] (unknown [IPv6:2001:470:909f:1:21b:63ff:feb8:5abc]) by itchy.rabson.org (Postfix) with ESMTP id D47EC5C60; Sun, 12 Apr 2009 14:57:20 +0100 (BST) Message-Id: From: Doug Rabson To: Robert Watson In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Sun, 12 Apr 2009 14:56:57 +0100 References: <49D98461.4000002@elischer.org> X-Mailer: Apple Mail (2.930.3) Cc: Rick Macklem , Julian Elischer , freebsd-arch@FreeBSD.org Subject: Re: getting a callback ip address for nfsv4 client X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Apr 2009 13:57:29 -0000 On 12 Apr 2009, at 13:24, Robert Watson wrote: > On Sun, 12 Apr 2009, Doug Rabson wrote: > >> While the RPC code doesn't currently expose the socket or local >> address, I don't think there would be many problems in adding a >> client control operation to get the local address. Its not a good >> idea to allow access to the actual socket since that will change >> over reconnects. In theory, the local address may also change over >> reconnects but that seems less likely to be a problem. Also, the >> reconnect machinery doesn't actually create the connection until >> the first RPC so some care would be needed to account for that. > > This was pretty much what I had in mind: do an initial no-op, then > query the local address from the RPC layer. The question then is, > when do you requery the address, and I guess an event is needed from > the RPC layer so that the NFS layer knows to re-query and update the > server's callback address (I assume that's allowed?). There is no need for an initial no-op - the code in sys/rpc/clnt_rc.c can connect the socket on an address query as well as on the first RPC. I'm less sure about how to handle address changes, if at all. I don't think the NFSv4 spec says anything on the matter. In NFSv4.1, callbacks are multiplexed onto the same connection as for regular forward RPCs so the problem will eventually go away. From owner-freebsd-arch@FreeBSD.ORG Sun Apr 12 18:04:24 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EA45410656EB; Sun, 12 Apr 2009 18:04:24 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from gigi.cs.uoguelph.ca (gigi.cs.uoguelph.ca [131.104.94.210]) by mx1.freebsd.org (Postfix) with ESMTP id A02498FC16; Sun, 12 Apr 2009 18:04:24 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from muncher.cs.uoguelph.ca (muncher.cs.uoguelph.ca [131.104.91.102]) by gigi.cs.uoguelph.ca (8.13.1/8.13.1) with ESMTP id n3CI4JLl011723; Sun, 12 Apr 2009 14:04:19 -0400 Received: from localhost (rmacklem@localhost) by muncher.cs.uoguelph.ca (8.11.7p3+Sun/8.11.6) with ESMTP id n3CIAiB17850; Sun, 12 Apr 2009 14:10:44 -0400 (EDT) X-Authentication-Warning: muncher.cs.uoguelph.ca: rmacklem owned process doing -bs Date: Sun, 12 Apr 2009 14:10:44 -0400 (EDT) From: Rick Macklem X-X-Sender: rmacklem@muncher.cs.uoguelph.ca To: Doug Rabson In-Reply-To: Message-ID: References: <49D98461.4000002@elischer.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Scanned-By: MIMEDefang 2.63 on 131.104.94.210 Cc: Robert Watson , Julian Elischer , freebsd-arch@freebsd.org Subject: Re: getting a callback ip address for nfsv4 client X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Apr 2009 18:04:26 -0000 On Sun, 12 Apr 2009, Doug Rabson wrote: [good stuff snipped] > > I'm less sure about how to handle address changes, if at all. I don't think > the NFSv4 spec says anything on the matter. In NFSv4.1, callbacks are > multiplexed onto the same connection as for regular forward RPCs so the > problem will eventually go away. > Address changes are problematic. If the nfsv4.0 client knows about an address change, it can do a fresh SetClientID/SetClientIDConfirm to tell the server about the new address. However, I think the safest thing for the client to do if a callback path isn't going to be stable, is to disable callbacks. (This just implies that the server won't choose to issue delegations and everything still works. To be honest, at this point, there isn't even much of a performance gain from delegations, although I am hoping to figure out how to use them to improve client side caching over the next year or so.) At this point, not starting the callback daemon (nfscbd) implies no callbacks, but I don't know of an "automagic" way to decide if the callback path will be stable. NB: A callback path that never works is just like no callbacks and isn't much of a problem. The problems occur when the callback path breaks after a while, when the client holds delegations issued to it by the server. It might be a good idea to initially ship with callbacks and delegations disabled, documenting them as "bleeding edge experimental features"? rick ps: For now, I think that using rtalloc1() if the callback address hasn't been set via sysctl, is adequate. However, if changes to the correct callback address (or knowing that the address is likely to change) can be detected, that would be nice. From owner-freebsd-arch@FreeBSD.ORG Sun Apr 12 18:12:05 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 19E73106566B for ; Sun, 12 Apr 2009 18:12:05 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id E12AD8FC13 for ; Sun, 12 Apr 2009 18:12:04 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 70DFB46B38; Sun, 12 Apr 2009 14:12:04 -0400 (EDT) Date: Sun, 12 Apr 2009 19:12:04 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Rick Macklem In-Reply-To: Message-ID: References: <49D98461.4000002@elischer.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Julian Elischer , freebsd-arch@freebsd.org Subject: Re: getting a callback ip address for nfsv4 client X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Apr 2009 18:12:05 -0000 On Sun, 12 Apr 2009, Rick Macklem wrote: >> I'm less sure about how to handle address changes, if at all. I don't think >> the NFSv4 spec says anything on the matter. In NFSv4.1, callbacks are >> multiplexed onto the same connection as for regular forward RPCs so the >> problem will eventually go away. > > Address changes are problematic. If the nfsv4.0 client knows about an > address change, it can do a fresh SetClientID/SetClientIDConfirm to tell the > server about the new address. > > However, I think the safest thing for the client to do if a callback path > isn't going to be stable, is to disable callbacks. (This just implies that > the server won't choose to issue delegations and everything still works. To > be honest, at this point, there isn't even much of a performance gain from > delegations, although I am hoping to figure out how to use them to improve > client side caching over the next year or so.) If we want to consider doing client-side disk caching, such as paging out NFS buffer cache to local swap, having delegations so we can invalidate the cache is fairly important. It becomes even more important if we want to do persistent client-side disk caching across reboots, as found in systems like AFS (does the NFSv4 design allow for this, or are client reboots still necessarily considered terminal for all caching?) The kind of scenario I have in mind isn't my IP address changing every 30 seconds, in which case NFS will be fairly useless anyway. It's more the "My IP changes twice a day when I suspend my notebook and commute to or from my office", or alternatively "My desktop is behind a NAT at home, and the ISP reboots my DSL modem once a month, which changes my ISP". In both of those scenarios, quickly noticing and re-establishing the callback path so that more effective caching can be used might make a significant difference. (Obviously, none of this really matters until delegations work...) Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-arch@FreeBSD.ORG Sun Apr 12 19:44:34 2009 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E8BCC106566B; Sun, 12 Apr 2009 19:44:34 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from mailhub.cs.uoguelph.ca (mailhub.cs.uoguelph.ca [131.104.94.205]) by mx1.freebsd.org (Postfix) with ESMTP id 9EB578FC13; Sun, 12 Apr 2009 19:44:34 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from muncher.cs.uoguelph.ca (muncher.cs.uoguelph.ca [131.104.91.102]) by mailhub.cs.uoguelph.ca (8.13.1/8.13.1) with ESMTP id n3CJiQpQ013604; Sun, 12 Apr 2009 15:44:26 -0400 Received: from localhost (rmacklem@localhost) by muncher.cs.uoguelph.ca (8.11.7p3+Sun/8.11.6) with ESMTP id n3CJoo503175; Sun, 12 Apr 2009 15:50:50 -0400 (EDT) X-Authentication-Warning: muncher.cs.uoguelph.ca: rmacklem owned process doing -bs Date: Sun, 12 Apr 2009 15:50:50 -0400 (EDT) From: Rick Macklem X-X-Sender: rmacklem@muncher.cs.uoguelph.ca To: Robert Watson In-Reply-To: Message-ID: References: <49D98461.4000002@elischer.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Scanned-By: MIMEDefang 2.63 on 131.104.94.205 Cc: Julian Elischer , freebsd-arch@FreeBSD.org Subject: Re: getting a callback ip address for nfsv4 client X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Apr 2009 19:44:35 -0000 On Sun, 12 Apr 2009, Robert Watson wrote: > > If we want to consider doing client-side disk caching, such as paging out NFS > buffer cache to local swap, having delegations so we can invalidate the cache > is fairly important. It becomes even more important if we want to do > persistent client-side disk caching across reboots, as found in systems like > AFS (does the NFSv4 design allow for this, or are client reboots still > necessarily considered terminal for all caching?) > Yes, what I am thinking of (I've coded parts of it, but always get distracted before I get it working:-) is whole file caching on local disk for delegated files, up to a certain size. And yes, if the delegation info is stored on stable storage, such as local disk, a client can recover it after a reboot. (Since no client currently does this, that part of current servers isn't heavily tested, so doing so could get "interesting":-) So, yes, I'd like to see delegations working, too. > The kind of scenario I have in mind isn't my IP address changing every 30 > seconds, in which case NFS will be fairly useless anyway. It's more the "My > IP changes twice a day when I suspend my notebook and commute to or from my > office", or alternatively "My desktop is behind a NAT at home, and the ISP > reboots my DSL modem once a month, which changes my ISP". In both of those > scenarios, quickly noticing and re-establishing the callback path so that > more effective caching can be used might make a significant difference. > The first scenario is problematic for nfsv4. If a client doesn't do a lease renewal within the server's chosen lease duration, the server has the right to throw away state, including locks and delegations. Most servers choose a lease duration of 1-2 minutes these days, so it needs to be "fast commute":-) Actually, my server chooses to be lenient and only expires state when it has to, due to a conflicting request or resource exhaustion and not just upon lease expiration, so you might be ok with a FreeBSD server, but not a Linux one (oh wait, you folks might actually consider that a feature:-). The second brings up the question of NAT, which I know diddly about. (I vaguely thought that neither rtalloc() nor getsockaddr() would work, since they would return 192.168.x.x and you really need the address assigned by the isp?) Can someone help w.r.t. what address to use when behind a NAT? Doug, maybe a client could optionally set an "upcall function" that would be called with the local ip address upon each reconnect to a server? (Then the nfsv4 client could check to see if the address had changed and do a fresh SetClientID/SetClientIDConfirm to change it, if it has?) > (Obviously, none of this really matters until delegations work...) Well, depends upon your definition of "work". At this point, they seem to function fine so long as the callback path is working and my client does do local opens and locking when it has a delegation. It doesn't yet do any clever caching and the FreeBSD kernel needs something (I am planning on posting a first stab at this in a few days) done before local VOP_RENAME() and VOP_ADVLOCK() calls to ensure consistency with locallly done ops. be added for correctness. rick From owner-freebsd-arch@FreeBSD.ORG Sun Apr 12 21:07:36 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 62763106564A; Sun, 12 Apr 2009 21:07:36 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: from kientzle.com (kientzle.com [66.166.149.50]) by mx1.freebsd.org (Postfix) with ESMTP id 2AE7F8FC1C; Sun, 12 Apr 2009 21:07:36 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: (from root@localhost) by kientzle.com (8.14.3/8.14.3) id n3CL7Yh1060529; Sun, 12 Apr 2009 14:07:34 -0700 (PDT) (envelope-from kientzle@freebsd.org) Received: from dark.x.kientzle.com (fw2.kientzle.com [10.123.1.2]) by kientzle.com with SMTP id 5mnfiaientg6kqnf57spjmczun; Sun, 12 Apr 2009 14:07:34 -0700 (PDT) (envelope-from kientzle@freebsd.org) Message-ID: <49E25816.9010907@freebsd.org> Date: Sun, 12 Apr 2009 14:07:34 -0700 From: Tim Kientzle User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.1.21) Gecko/20090409 SeaMonkey/1.1.15 MIME-Version: 1.0 To: Rick Macklem References: <49D98461.4000002@elischer.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Robert Watson , Julian Elischer , freebsd-arch@freebsd.org Subject: Re: getting a callback ip address for nfsv4 client X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Apr 2009 21:07:36 -0000 > The second brings up the question of NAT, which I know diddly about. > (I vaguely thought that neither rtalloc() nor getsockaddr() would work, > since they would return 192.168.x.x and you really need the address > assigned by the isp?) Can someone help w.r.t. what address to use > when behind a NAT? In general, the client doesn't know what address to use in this case. In fact, I've been a little confused by this conversation all along. It sounds like the client is looking up it's own address in order to tell the server how to contact the client? Why? The server already knows the source IP address on the incoming packets from the client; that's much more robust than anything the client could look up. Or did I misunderstand something? Tim From owner-freebsd-arch@FreeBSD.ORG Sun Apr 12 22:25:36 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C914C1065670 for ; Sun, 12 Apr 2009 22:25:36 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.188]) by mx1.freebsd.org (Postfix) with ESMTP id 5A9A98FC14 for ; Sun, 12 Apr 2009 22:25:36 +0000 (UTC) (envelope-from max@love2party.net) Received: from vampire.homelinux.org (dslb-088-066-019-008.pools.arcor-ip.net [88.66.19.8]) by mrelayeu.kundenserver.de (node=mreu0) with ESMTP (Nemesis) id 0MKuxg-1Lt87K2864-0001IU; Mon, 13 Apr 2009 00:25:35 +0200 Received: (qmail 26748 invoked from network); 12 Apr 2009 22:25:32 -0000 Received: from fbsd8.laiers.local (192.168.4.200) by mx.laiers.local with SMTP; 12 Apr 2009 22:25:32 -0000 From: Max Laier Organization: FreeBSD To: freebsd-arch@freebsd.org Date: Sun, 12 Apr 2009 23:25:31 +0100 User-Agent: KMail/1.11.0 (FreeBSD/8.0-CURRENT; KDE/4.2.1; i386; ; ) References: <49E25816.9010907@freebsd.org> In-Reply-To: <49E25816.9010907@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200904130025.31771.max@love2party.net> X-Provags-ID: V01U2FsdGVkX1/LbckiVi/Z8MZvKddIINTE7ZHH20WmquKfowr +cjKwNWDYJUdIig14pSnpeE7qK1hHxGA/3NZbEwjVjGQK7iYJt eRxWiNdARuGircRdKFhGw== Cc: Tim Kientzle , Robert Watson , Julian Elischer , Rick Macklem Subject: Re: getting a callback ip address for nfsv4 client X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Apr 2009 22:25:37 -0000 On Sunday 12 April 2009 23:07:34 Tim Kientzle wrote: > > The second brings up the question of NAT, which I know diddly about. > > (I vaguely thought that neither rtalloc() nor getsockaddr() would work, > > since they would return 192.168.x.x and you really need the address > > assigned by the isp?) Can someone help w.r.t. what address to use > > when behind a NAT? > > In general, the client doesn't know what address to use > in this case. In fact, I've been a little confused by this > conversation all along. It sounds like the client is > looking up it's own address in order to tell the server > how to contact the client? Why? The server already > knows the source IP address on the incoming packets > from the client; that's much more robust than anything > the client could look up. Well, the client also needs to listen at the address - that is a local decision. This is much like the problem with active mode FTP - and it has the same problems with NAT (i.e. the NAT service must be aware of the protocol and translate the address inside). The alternative is to use things like UPnP to retrieve an external address mapping ... there are libraries to deal with that. -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News From owner-freebsd-arch@FreeBSD.ORG Mon Apr 13 11:06:49 2009 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E4EF010656E2 for ; Mon, 13 Apr 2009 11:06:49 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id B2A268FC19 for ; Mon, 13 Apr 2009 11:06:49 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n3DB6nlv084870 for ; Mon, 13 Apr 2009 11:06:49 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n3DB6nMi084865 for freebsd-arch@FreeBSD.org; Mon, 13 Apr 2009 11:06:49 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 13 Apr 2009 11:06:49 GMT Message-Id: <200904131106.n3DB6nMi084865@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-arch@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Apr 2009 11:06:52 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. From owner-freebsd-arch@FreeBSD.ORG Mon Apr 13 14:13:50 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 717571065674; Mon, 13 Apr 2009 14:13:50 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from mailhub.cs.uoguelph.ca (mailhub.cs.uoguelph.ca [131.104.94.205]) by mx1.freebsd.org (Postfix) with ESMTP id 258F98FC20; Mon, 13 Apr 2009 14:13:49 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from muncher.cs.uoguelph.ca (muncher.cs.uoguelph.ca [131.104.91.102]) by mailhub.cs.uoguelph.ca (8.13.1/8.13.1) with ESMTP id n3DEDh8x018152; Mon, 13 Apr 2009 10:13:43 -0400 Received: from localhost (rmacklem@localhost) by muncher.cs.uoguelph.ca (8.11.7p3+Sun/8.11.6) with ESMTP id n3DEK9a24696; Mon, 13 Apr 2009 10:20:09 -0400 (EDT) X-Authentication-Warning: muncher.cs.uoguelph.ca: rmacklem owned process doing -bs Date: Mon, 13 Apr 2009 10:20:09 -0400 (EDT) From: Rick Macklem X-X-Sender: rmacklem@muncher.cs.uoguelph.ca To: Tim Kientzle In-Reply-To: <49E25816.9010907@freebsd.org> Message-ID: References: <49D98461.4000002@elischer.org> <49E25816.9010907@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Scanned-By: MIMEDefang 2.63 on 131.104.94.205 Cc: Robert Watson , Julian Elischer , freebsd-arch@freebsd.org Subject: Re: getting a callback ip address for nfsv4 client X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Apr 2009 14:13:50 -0000 On Sun, 12 Apr 2009, Tim Kientzle wrote: > > In general, the client doesn't know what address to use > in this case. In fact, I've been a little confused by this > conversation all along. It sounds like the client is > looking up it's own address in order to tell the server > how to contact the client? Why? The server already > knows the source IP address on the incoming packets > from the client; that's much more robust than anything > the client could look up. > No misunderstanding, except that the nfsv4 designers put the callback address in the SetClientID request, so a client must explicitly specify it. (I can only guess that they wanted the client to be able to specify an alternate routing or maybe some vendors wanted to be able to put the callback server on a separate machine/interface or ???) To me, it would have been simpler to just put callback RPCs in the same TCP stream as the client->server RPCs, but they didn't do that either. For nfsv4.1, there is a large glob of stuff called Sessions, which handles the transport layer and the connections for callbacks are created by the client, which avoids the problem. rick From owner-freebsd-arch@FreeBSD.ORG Mon Apr 13 14:26:44 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EAD0010656C7; Mon, 13 Apr 2009 14:26:44 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from skerryvore.cs.uoguelph.ca (skerryvore.cs.uoguelph.ca [131.104.94.204]) by mx1.freebsd.org (Postfix) with ESMTP id A569B8FC08; Mon, 13 Apr 2009 14:26:44 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from muncher.cs.uoguelph.ca (muncher.cs.uoguelph.ca [131.104.91.102]) by skerryvore.cs.uoguelph.ca (8.13.1/8.13.1) with ESMTP id n3DEQfCU010964; Mon, 13 Apr 2009 10:26:41 -0400 Received: from localhost (rmacklem@localhost) by muncher.cs.uoguelph.ca (8.11.7p3+Sun/8.11.6) with ESMTP id n3DEX7e26970; Mon, 13 Apr 2009 10:33:07 -0400 (EDT) X-Authentication-Warning: muncher.cs.uoguelph.ca: rmacklem owned process doing -bs Date: Mon, 13 Apr 2009 10:33:07 -0400 (EDT) From: Rick Macklem X-X-Sender: rmacklem@muncher.cs.uoguelph.ca To: Max Laier In-Reply-To: <200904130025.31771.max@love2party.net> Message-ID: References: <49E25816.9010907@freebsd.org> <200904130025.31771.max@love2party.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Scanned-By: MIMEDefang 2.63 on 131.104.94.204 Cc: Tim Kientzle , Robert Watson , Julian Elischer , freebsd-arch@freebsd.org Subject: Re: getting a callback ip address for nfsv4 client X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Apr 2009 14:26:46 -0000 On Sun, 12 Apr 2009, Max Laier wrote: [Tim's good stuff snipped] > > Well, the client also needs to listen at the address - that is a local > decision. For now, it just listens on INADDR_ANY, but I suppose there's an argument for adding an option to the daemon to set an address, to restrict it to listening on one interface? (I do currently have a sysctl variable for overriding what rtalloc1() returns, but that requires manual intervention. There is an argument for setting the port#. Does someone have to set the port in the NAT gateway manually or is there a protocol/library for doing that? > This is much like the problem with active mode FTP - and it has the > same problems with NAT (i.e. the NAT service must be aware of the protocol and > translate the address inside). The alternative is to use things like UPnP to > retrieve an external address mapping ... there are libraries to deal with > that. > Are these libraries in FreeBSD-CURRENT? If so, please point me towards them, so I can take a look. Since the callback daemon starts out in userland, talking to userland libraries to handle NAT shouldn't be a problem. Thanks for the help sofar, rick From owner-freebsd-arch@FreeBSD.ORG Mon Apr 13 15:29:03 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C3712106568C; Mon, 13 Apr 2009 15:29:03 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 812228FC24; Mon, 13 Apr 2009 15:29:03 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 7C64E46B35; Mon, 13 Apr 2009 11:29:02 -0400 (EDT) Received: from jhbbsd.hudson-trading.com (unknown [209.249.190.8]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 198E08A04F; Mon, 13 Apr 2009 11:28:43 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Date: Mon, 13 Apr 2009 10:22:53 -0400 User-Agent: KMail/1.9.7 References: <49E0D353.7090308@freebsd.org> In-Reply-To: <49E0D353.7090308@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200904131022.53882.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Mon, 13 Apr 2009 11:28:43 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=0.1 required=4.2 tests=RDNS_NONE autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: arch@freebsd.org, Robert Watson , Nathan Whitehorn Subject: Re: Simple #define for cache line size X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Apr 2009 15:29:04 -0000 On Saturday 11 April 2009 1:28:51 pm Nathan Whitehorn wrote: > Robert Watson wrote: > > > > > > NetBSD, FYI, defines CACHE_LINE_SIZE as a global constant in param.h, > > but I'm going with an MD definition as I suspect people will want to > > do different things on different architectures (and there is > > variation). I've defaulted all architectures to 64 bytes, but I > > suspect a number would prefer to use 32. > For what it's worth, this is per-CPU variable on PowerPC and detected at > runtime. Most of the CPUs we support have 32 byte cache lines, but some > (e.g. the G5) use 128 bytes. I'm not sure there is a general solution in > this case, but that's the situation on PPC. > -Nathan I think UMA can handle a variable size and that should be preserved. However, for the purposes of aligned() I think a constant that is the maximum size (such as 128 on ppc) might work best. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon Apr 13 15:29:03 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C3712106568C; Mon, 13 Apr 2009 15:29:03 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 812228FC24; Mon, 13 Apr 2009 15:29:03 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 7C64E46B35; Mon, 13 Apr 2009 11:29:02 -0400 (EDT) Received: from jhbbsd.hudson-trading.com (unknown [209.249.190.8]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 198E08A04F; Mon, 13 Apr 2009 11:28:43 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Date: Mon, 13 Apr 2009 10:22:53 -0400 User-Agent: KMail/1.9.7 References: <49E0D353.7090308@freebsd.org> In-Reply-To: <49E0D353.7090308@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200904131022.53882.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Mon, 13 Apr 2009 11:28:43 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=0.1 required=4.2 tests=RDNS_NONE autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: arch@freebsd.org, Robert Watson , Nathan Whitehorn Subject: Re: Simple #define for cache line size X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Apr 2009 15:29:04 -0000 On Saturday 11 April 2009 1:28:51 pm Nathan Whitehorn wrote: > Robert Watson wrote: > > > > > > NetBSD, FYI, defines CACHE_LINE_SIZE as a global constant in param.h, > > but I'm going with an MD definition as I suspect people will want to > > do different things on different architectures (and there is > > variation). I've defaulted all architectures to 64 bytes, but I > > suspect a number would prefer to use 32. > For what it's worth, this is per-CPU variable on PowerPC and detected at > runtime. Most of the CPUs we support have 32 byte cache lines, but some > (e.g. the G5) use 128 bytes. I'm not sure there is a general solution in > this case, but that's the situation on PPC. > -Nathan I think UMA can handle a variable size and that should be preserved. However, for the purposes of aligned() I think a constant that is the maximum size (such as 128 on ppc) might work best. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon Apr 13 19:37:57 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 070E31065895 for ; Mon, 13 Apr 2009 19:37:56 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from proxy.meer.net (proxy.meer.net [64.13.141.13]) by mx1.freebsd.org (Postfix) with ESMTP id C1BA98FC26 for ; Mon, 13 Apr 2009 19:37:55 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from mail.meer.net (mail.meer.net [64.13.141.3]) by proxy.meer.net (8.14.3/8.14.3) with ESMTP id n3DJ10sj018796; Mon, 13 Apr 2009 12:01:00 -0700 (PDT) (envelope-from gnn@neville-neil.com) Received: from mail2.meer.net (mail2.meer.net [64.13.141.16]) by mail.meer.net (8.13.3/8.13.3/meer) with ESMTP id n3DJ0fQs093655; Mon, 13 Apr 2009 12:00:41 -0700 (PDT) (envelope-from gnn@neville-neil.com) Received: from gnnbsd.hudson-trading.com.neville-neil.com (209.249.190.8.available.above.net [209.249.190.8] (may be forged)) (authenticated bits=0) by mail2.meer.net (8.14.1/8.14.3) with ESMTP id n3DJ0f6N059596; Mon, 13 Apr 2009 12:00:41 -0700 (PDT) (envelope-from gnn@neville-neil.com) Date: Mon, 13 Apr 2009 15:00:40 -0400 Message-ID: <7iskkcgyzr.wl%gnn@neville-neil.com> From: gnn@freebsd.org To: Zachary Loafman In-Reply-To: <20090409171613.GC9442@isilon.com> References: <20090409171613.GC9442@isilon.com> User-Agent: Wanderlust/2.14.0 (Africa) SEMI/1.14.6 (Maruoka) FLIM/1.14.8 (=?ISO-8859-4?Q?Shij=F2?=) APEL/10.7 Emacs/22.3 (amd64-portbld-freebsd7.1) MULE/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII X-Spam-Score: undef - spam scanning disabled X-CanIt-Geo: No geolocation information available for 64.13.141.3 X-CanItPRO-Stream: default X-Canit-Stats-ID: Bayes signature not available X-Scanned-By: CanIt (www . roaringpenguin . com) on 64.13.141.13 Cc: freebsd-arch@freebsd.org Subject: Re: splice() in FreeBSD X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Apr 2009 19:38:02 -0000 At Thu, 9 Apr 2009 10:16:13 -0700, Zachary Loafman wrote: > > Arch - > > Isilon has internally been using the FreeBSD sendfile() (with > modifications) and our own recvfile() in order to accomplish zero-copy > read/write for the userland portions of our stack (CIFS, > NDMP). However, these interfaces are limited. In particular, > sendfile/recvfile prevent any other thread from dealing with the same > socket until the call is complete. That's somewhat silly - it would be > nicer to split the read-from-file/write-to-file portion from the > read-from-socket/write-to-socket portion. That also eases some of the > decisions that only the layer above can really make - for example, in > the sendfile() case, you don't really know if it's appropriate to send > a partial read or whether the caller really needs all the data. > > What we'd like is something like splice(). The Linux splice interface > is documented here: http://linux.die.net/man/2/splice and the > internals are discussed here: http://kerneltrap.org/node/6505 . We > don't need the sillier portions of it - Isilon could care less about > vmsplice()/tee(). We need the ability to shuffle data from one source > to one sink, and then to turn around later and use that sink as a > source. At first, I found the splice() interface a bit of an > abomination, but a pipe is a somewhat natural place to act as a data > staging area. If we just implemented splice alone, this wouldn't > require any real VM hackery - you can imagine just shuffling mbufs > through the pipe to accomplish a limited form of this (or, say, a unix > domain socket). > > As part of this, and in order to get something upstreamable, it seems > like we would need a few things: > > *) Agreement on syscall APIs - My initial proposal is to adopt splice > verbatim. Initially the interface may not be truly zero-copy for many > cases, but it's a start. It also increases portability for any Linux > apps that are trying to make use of it. > > *) Unification of uio and mbufs somehow? Isilon currently has private > patches that add *_MBUF variants for I/O VOPs (e.g. we have a > VOP_READ_MBUF in addition to the standard VOP_READ). Isilon is in a > somewhat unique place here - I'm not sure a general file system can > handle this as easily. At the top-half, our system in many ways acts a > lot like a router, so we can handle things like VOP_READ_MBUF by > taking file data off our back-end (which comes in as mbufs off IB), > header splitting, then just slinging the mbufs out the > front-end. However, I think our *_MBUF VOP variants are actually a > little gross. I would rather figure out a way to unify the uio and > mbuf APIs - they're both scatter/gather lists in their own special > way, then call into a single VOP. > > Isilon can get a limited, non-upstreamable thing working fairly > quickly - we can use a unix domain socket as the intermediate buffer > and use our existing *_MBUF VOPs. But it would be nice if we had some > consensus going forward, then we can internally march towards > something we can upstream. > I like the idea, though I don't know if I like the name "splice" because to me it's a bit confusing, but we're probably stuck with the name since it's already in use. If/when you have patches send them along next. Best, George From owner-freebsd-arch@FreeBSD.ORG Mon Apr 13 21:02:13 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0337A106564A for ; Mon, 13 Apr 2009 21:02:13 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id A7FBD8FC15 for ; Mon, 13 Apr 2009 21:02:12 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1LtTI7-0001Ow-Jy for freebsd-arch@freebsd.org; Mon, 13 Apr 2009 21:02:07 +0000 Received: from 93-138-97-154.adsl.net.t-com.hr ([93.138.97.154]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 13 Apr 2009 21:02:07 +0000 Received: from ivoras by 93-138-97-154.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 13 Apr 2009 21:02:07 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Ivan Voras Date: Mon, 13 Apr 2009 23:01:23 +0200 Lines: 44 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig8D725566B4379538CB8FF905" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 93-138-97-154.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) In-Reply-To: X-Enigmail-Version: 0.95.7 Sender: news Subject: Re: Simple #define for cache line size X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Apr 2009 21:02:13 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig8D725566B4379538CB8FF905 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Robert Watson wrote: > --- i386/include/param.h (revision 190941) > +++ i386/include/param.h (working copy) > @@ -74,6 +74,10 @@ > #define ALIGNBYTES _ALIGNBYTES > #define ALIGN(p) _ALIGN(p) >=20 > +#ifndef CACHE_LINE_SIZE > +#define CACHE_LINE_SIZE 64 > +#endif Wouldn't it be better to continue the cpu I486_CPU cpu I586_CPU cpu I686_CPU series of defines in kernel configuration and define alignment per CPU architecture? I guess it depends on the trends - are cache lines expected to change in the near future? :) --------------enig8D725566B4379538CB8FF905 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAknjqCgACgkQldnAQVacBcigHgCdHhI3Ju25GNN/MN2ZDv0rfAmP uMoAnRZKeOyJwbKxA5cjb29kRhs0wdxW =2wxO -----END PGP SIGNATURE----- --------------enig8D725566B4379538CB8FF905-- From owner-freebsd-arch@FreeBSD.ORG Mon Apr 13 22:13:14 2009 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 895011065673 for ; Mon, 13 Apr 2009 22:13:14 +0000 (UTC) (envelope-from julian@elischer.org) Received: from mx1.fusionio.com (mx1.fusionio.com [64.244.102.30]) by mx1.freebsd.org (Postfix) with ESMTP id 53D988FC0A for ; Mon, 13 Apr 2009 22:13:14 +0000 (UTC) (envelope-from julian@elischer.org) X-ASG-Debug-ID: 1239659736-6845000c0000-9koHHG X-Barracuda-URL: http://10.101.1.180:8000/cgi-bin/mark.cgi Received: from zim.fusionio.com (localhost [127.0.0.1]) by mx1.fusionio.com (Spam Firewall) with ESMTP id 3900185359; Mon, 13 Apr 2009 15:55:36 -0600 (MDT) Received: from zim.fusionio.com ([10.101.1.182]) by mx1.fusionio.com with ESMTP id gNKw2HvWA4fwnUoH; Mon, 13 Apr 2009 15:55:36 -0600 (MDT) X-Barracuda-Envelope-From: julian@elischer.org Received: from localhost (localhost.localdomain [127.0.0.1]) by zim.fusionio.com (Postfix) with ESMTP id 202E94500003; Mon, 13 Apr 2009 15:55:36 -0600 (MDT) X-Virus-Scanned: amavisd-new at X-Spam-Flag: NO X-Spam-Score: -1.01 X-Spam-Level: X-Spam-Status: No, score=-1.01 tagged_above=-10 required=6.6 tests=[BAYES_05=-1.11, RDNS_NONE=0.1] Received: from zim.fusionio.com ([127.0.0.1]) by localhost (zim.fusionio.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id y9q1a0kBChD4; Mon, 13 Apr 2009 15:55:28 -0600 (MDT) Received: from julian-mac.elischer.org (unknown [64.244.102.2]) by zim.fusionio.com (Postfix) with ESMTP id CE3E14500002; Mon, 13 Apr 2009 15:55:28 -0600 (MDT) Message-ID: <49E3B4D5.3070701@elischer.org> Date: Mon, 13 Apr 2009 14:55:33 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Doug Rabson X-ASG-Orig-Subj: Re: getting a callback ip address for nfsv4 client References: <49D98461.4000002@elischer.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Barracuda-Connect: UNKNOWN[10.101.1.182] X-Barracuda-Start-Time: 1239659736 X-Barracuda-Virus-Scanned: by Barracuda Spam Firewall at fusionio.com Cc: Rick Macklem , Robert Watson , freebsd-arch@FreeBSD.org Subject: Re: getting a callback ip address for nfsv4 client X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Apr 2009 22:13:15 -0000 Doug Rabson wrote: > > While the RPC code doesn't currently expose the socket or local address, > I don't think there would be many problems in adding a client control > operation to get the local address. I like that option.. > Its not a good idea to allow access > to the actual socket since that will change over reconnects. In theory, > the local address may also change over reconnects but that seems less > likely to be a problem. Also, the reconnect machinery doesn't actually > create the connection until the first RPC so some care would be needed > to account for that. > CONFIDENTIAL This document and attachments contain information from Fusion-io, Inc. which is confidential and/or legally privileged. The information is intended only for the use of the individual or entity named on this transmission. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or taking of any action in reliance on the contents of this emailed information is strictly prohibited, and that the documents should be returned to Fusion-io, Inc. immediately. In this regard, if you have received this email in error, please notify us by return email immediately. From owner-freebsd-arch@FreeBSD.ORG Mon Apr 13 23:05:23 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0D51F1065673 for ; Mon, 13 Apr 2009 23:05:23 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.177]) by mx1.freebsd.org (Postfix) with ESMTP id AC2EB8FC1A for ; Mon, 13 Apr 2009 23:05:22 +0000 (UTC) (envelope-from max@love2party.net) Received: from vampire.homelinux.org (dslb-088-066-019-008.pools.arcor-ip.net [88.66.19.8]) by mrelayeu.kundenserver.de (node=mreu2) with ESMTP (Nemesis) id 0MKv5w-1LtVDN1JGF-0001v0; Tue, 14 Apr 2009 01:05:21 +0200 Received: (qmail 47044 invoked from network); 13 Apr 2009 23:05:21 -0000 Received: from fbsd8.laiers.local (192.168.4.200) by ns1.laiers.local with SMTP; 13 Apr 2009 23:05:21 -0000 From: Max Laier Organization: FreeBSD To: freebsd-arch@freebsd.org Date: Tue, 14 Apr 2009 01:05:20 +0200 User-Agent: KMail/1.11.0 (FreeBSD/8.0-CURRENT; KDE/4.2.1; i386; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200904140105.20664.max@love2party.net> X-Provags-ID: V01U2FsdGVkX18pAmnAlfIRpMJ/LmdMqWuszOlXP8ldsw+Tpek lF47zP29YzKS3oBTaT4LxMOfAvKmLBeczLB+gABZnj1e+Kbx8G 83s75jyDVSp6IqFEcihjA== Cc: Robert Watson Subject: Re: Simple #define for cache line size X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Apr 2009 23:05:23 -0000 On Saturday 11 April 2009 18:11:55 Robert Watson wrote: > Dear all: > > We have a number of __aligned() qualifiers scattered around the kernel > intended to space objects to improve alignment with respect to cache lines. > This is important for a number of reasons, not least avoiding cache line > thrashing when using arrays of foo[MAXCPU]. What I'd like to do is provide > a single compile-time constant, CACHE_LINE_SIZE, defined in > machine-dependent param.h, to use for spacing such objects, rather than > hard-coding various values around. Here are some examples of existing > spacing attempts: How much does __aligned(FOO) even achieve? If we allocate these structs at runtime the alignment requirements have to be passed to the allocator as well (eg. uma(9)'s align parameter in uma_zcreate). If my assumption is correct it would make sense to have a global symbol that is initialized early in the boot process instead of a compile time #define. In addition it seems that we need to make the define a worst case value to assure correctness for static arrays as below. > i386/include/pcpu.h, line 67 > > 67 char pc_monitorbuf[128] __aligned(128); /* cache line */ like here. If we define CACHE_LINE to be 64 and happen to run on a CPU that has 128 this will no longer work correctly. > vm/uma_core.c, line 115 > 114 /* The boot-time adjusted value for cache line alignment. */ > > 115 static int uma_align_cache = 64 - 1; As the comment suggest, the value should be boot time adjusted and used from there. > We could then deploy __aligned(CACHE_LINE_SIZE) in various places, such as > on the description of per-CPU caches for UMA, to ensure that, for example, > per-CPU stats for one CPU weren't in the same cache line as stats for other > CPUs. > > NetBSD, FYI, defines CACHE_LINE_SIZE as a global constant in param.h, but > I'm going with an MD definition as I suspect people will want to do > different things on different architectures (and there is variation). I've > defaulted all architectures to 64 bytes, but I suspect a number would > prefer to use 32. > > Patch below. There's undoubtably an argument for doing something more > optimal than what I'm proposing here, but right now I'm just looking for > something that works, and would be happy to see it replaced with something > more mature once it's available. > > Robert N M Watson > Computer Laboratory > University of Cambridge > > Index: arm/include/param.h > =================================================================== > --- arm/include/param.h (revision 190941) > +++ arm/include/param.h (working copy) > @@ -81,6 +81,10 @@ > #define ALIGNBYTES _ALIGNBYTES > #define ALIGN(p) _ALIGN(p) > > +#ifndef CACHE_LINE_SIZE > +#define CACHE_LINE_SIZE 64 > +#endif > + > #define PAGE_SHIFT 12 > #define PAGE_SIZE (1 << PAGE_SHIFT) /* Page size */ > #define PAGE_MASK (PAGE_SIZE - 1) > Index: powerpc/include/param.h > =================================================================== > --- powerpc/include/param.h (revision 190941) > +++ powerpc/include/param.h (working copy) > @@ -79,6 +79,10 @@ > #define ALIGNBYTES _ALIGNBYTES > #define ALIGN(p) _ALIGN(p) > > +#ifndef CACHE_LINE_SIZE > +#define CACHE_LINE_SIZE 64 > +#endif > + > #define PAGE_SHIFT 12 > #define PAGE_SIZE (1 << PAGE_SHIFT) /* Page size */ > #define PAGE_MASK (PAGE_SIZE - 1) > Index: sparc64/include/param.h > =================================================================== > --- sparc64/include/param.h (revision 190941) > +++ sparc64/include/param.h (working copy) > @@ -71,6 +71,10 @@ > #define ALIGNBYTES _ALIGNBYTES > #define ALIGN(p) _ALIGN(p) > > +#ifndef CACHE_LINE_SIZE > +#define CACHE_LINE_SIZE 64 > +#endif > + > #define PAGE_SHIFT_8K 13 > #define PAGE_SIZE_8K (1L< #define PAGE_MASK_8K (PAGE_SIZE_8K-1) > Index: ia64/include/param.h > =================================================================== > --- ia64/include/param.h (revision 190941) > +++ ia64/include/param.h (working copy) > @@ -99,6 +99,10 @@ > #define ALIGN(p) _ALIGN(p) > #define ALIGNED_POINTER(p,t) _ALIGNED_POINTER(p,t) > > +#ifndef CACHE_LINE_SIZE > +#define CACHE_LINE_SIZE 64 > +#endif > + > #ifndef LOG2_PAGE_SIZE > #define LOG2_PAGE_SIZE 13 /* 8K pages by default. */ > #endif > Index: mips/include/param.h > =================================================================== > --- mips/include/param.h (revision 190941) > +++ mips/include/param.h (working copy) > @@ -89,6 +89,10 @@ > #define ALIGNBYTES _ALIGNBYTES > #define ALIGN(p) _ALIGN(p) > > +#ifndef CACHE_LINE_SIZE > +#define CACHE_LINE_SIZE 64 > +#endif > + > #define NBPG 4096 /* bytes/page */ > #define PGOFSET (NBPG-1) /* byte offset into page */ > #define PGSHIFT 12 /* LOG2(NBPG) */ > Index: sun4v/include/param.h > =================================================================== > --- sun4v/include/param.h (revision 190941) > +++ sun4v/include/param.h (working copy) > @@ -71,6 +71,10 @@ > #define ALIGNBYTES _ALIGNBYTES > #define ALIGN(p) _ALIGN(p) > > +#ifndef CACHE_LINE_SIZE > +#define CACHE_LINE_SIZE 64 > +#endif > + > #define PAGE_SHIFT_8K 13 > #define PAGE_SIZE_8K (1L< #define PAGE_MASK_8K (PAGE_SIZE_8K-1) > Index: i386/include/param.h > =================================================================== > --- i386/include/param.h (revision 190941) > +++ i386/include/param.h (working copy) > @@ -74,6 +74,10 @@ > #define ALIGNBYTES _ALIGNBYTES > #define ALIGN(p) _ALIGN(p) > > +#ifndef CACHE_LINE_SIZE > +#define CACHE_LINE_SIZE 64 > +#endif > + > #define PAGE_SHIFT 12 /* LOG2(PAGE_SIZE) */ > #define PAGE_SIZE (1< #define PAGE_MASK (PAGE_SIZE-1) > Index: amd64/include/param.h > =================================================================== > --- amd64/include/param.h (revision 190941) > +++ amd64/include/param.h (working copy) > @@ -89,6 +89,9 @@ > #define ALIGN(p) _ALIGN(p) > #define ALIGNED_POINTER(p,t) _ALIGNED_POINTER(p,t) > > +#ifndef CACHE_LINE_SIZE > +#define CACHE_LINE_SIZE 64 > +#endif > > /* Size of the level 1 page table units */ > #define NPTEPG (PAGE_SIZE/(sizeof (pt_entry_t))) > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > > > !DSPAM:49e0c16232832046891198! -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News From owner-freebsd-arch@FreeBSD.ORG Tue Apr 14 06:53:34 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1A850106564A for ; Tue, 14 Apr 2009 06:53:34 +0000 (UTC) (envelope-from zec@freebsd.org) Received: from labs4.cc.fer.hr (labs4.cc.fer.hr [161.53.72.24]) by mx1.freebsd.org (Postfix) with ESMTP id A14408FC16 for ; Tue, 14 Apr 2009 06:53:33 +0000 (UTC) (envelope-from zec@freebsd.org) Received: from sluga.fer.hr (sluga.cc.fer.hr [161.53.72.14]) by labs4.cc.fer.hr (8.14.2/8.14.2) with ESMTP id n3E6hIIW008709 for ; Tue, 14 Apr 2009 08:43:18 +0200 (CEST) Received: from [192.168.200.100] ([161.53.19.79]) by sluga.fer.hr with Microsoft SMTPSVC(6.0.3790.3959); Tue, 14 Apr 2009 08:42:55 +0200 From: Marko Zec To: freebsd-arch@freebsd.org Date: Tue, 14 Apr 2009 08:42:46 +0200 User-Agent: KMail/1.9.10 References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200904140842.46501.zec@freebsd.org> X-OriginalArrivalTime: 14 Apr 2009 06:42:55.0232 (UTC) FILETIME=[3DC32800:01C9BCCC] X-Scanned-By: MIMEDefang 2.64 on 161.53.72.24 Subject: Re: Simple #define for cache line size X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Apr 2009 06:53:34 -0000 On Monday 13 April 2009 23:01:23 Ivan Voras wrote: > Robert Watson wrote: > > --- i386/include/param.h (revision 190941) > > +++ i386/include/param.h (working copy) > > @@ -74,6 +74,10 @@ > > #define ALIGNBYTES _ALIGNBYTES > > #define ALIGN(p) _ALIGN(p) > > > > +#ifndef CACHE_LINE_SIZE > > +#define CACHE_LINE_SIZE 64 > > +#endif > > Wouldn't it be better to continue the > > cpu I486_CPU > cpu I586_CPU > cpu I686_CPU > > series of defines in kernel configuration and define alignment per > CPU architecture? We would have to extend our notion of "CPU architecture" for that to make sense. For example, Pentium Pro / II CPUs had cache line size of 32 bytes, Intel Netburst CPUs (all Pentium-4 and Xeons of the time) have / had 128 bytes, while Pentium-III, Pentium-M and later Core CPUs have 64 bytes. They are all I686_CPU in our view. Marko > I guess it depends on the trends - are cache lines > expected to change in the near future? :) From owner-freebsd-arch@FreeBSD.ORG Tue Apr 14 14:16:11 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5E9001065676; Tue, 14 Apr 2009 14:16:11 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from redbull.bpaserver.net (redbullneu.bpaserver.net [213.198.78.217]) by mx1.freebsd.org (Postfix) with ESMTP id EA4898FC14; Tue, 14 Apr 2009 14:16:10 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from outgoing.leidinger.net (pD9E2E7C5.dip.t-dialin.net [217.226.231.197]) by redbull.bpaserver.net (Postfix) with ESMTP id 206672E2D9; Tue, 14 Apr 2009 16:00:11 +0200 (CEST) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id DDA12ABF18; Tue, 14 Apr 2009 16:00:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1239717607; bh=anfRN9Ge/onLwwDqBPw4J4UttMfBaRNYt ndD0bPtYOg=; h=Message-ID:Date:From:To:Cc:Subject:References: In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=kkKsYyLPsLw8TLnRU1cgRMxPm9z/sdNQjqgAiS8HqYPedg4KtiKRt/4VXLLsTkBq5 6uzkwMWiV8YA6HlEy9GPWuB25IG+v7MO4JpivecY34TImAE2bZ+9j+pVxV/Ms4Zkhot YuazendV3Vwb7CYWTpBDV+Ews0crFSJBl25jMXN3kDVZxyIiSt2cskzt8N6Ln+RUTeo rhivYtg3F8aKjPM8oI4CjZbvztQqtwJfV6lrxdiDoBns6giqZDvn1jLGfFngi1gVhOw bxsj3p3TEhE4/XQ1eQoNUKI3mEaews62QpDiKw8dZvWGwTUIU9JJQ7ajFwYXvRljf4A ewMu7vEDg== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id n3EE07MG066654; Tue, 14 Apr 2009 16:00:07 +0200 (CEST) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Tue, 14 Apr 2009 16:00:07 +0200 Message-ID: <20090414160007.15649rsc2fuyqxnk@webmail.leidinger.net> X-Priority: 3 (Normal) Date: Tue, 14 Apr 2009 16:00:07 +0200 From: Alexander Leidinger To: Marko Zec References: <200904140842.46501.zec@freebsd.org> In-Reply-To: <200904140842.46501.zec@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Internet Messaging Program (IMP) H3 (4.3) / FreeBSD-8.0 X-BPAnet-MailScanner-Information: Please contact the ISP for more information X-MailScanner-ID: 206672E2D9.778AE X-BPAnet-MailScanner: Found to be clean X-BPAnet-MailScanner-SpamCheck: not spam, ORDB-RBL, SpamAssassin (not cached, score=-14.9, required 6, BAYES_00 -15.00, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00, RDNS_DYNAMIC 0.10) X-BPAnet-MailScanner-From: alexander@leidinger.net X-Spam-Status: No Cc: freebsd-arch@freebsd.org Subject: Re: Simple #define for cache line size X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Apr 2009 14:16:11 -0000 Quoting Marko Zec (from Tue, 14 Apr 2009 08:42:46 +0200): > On Monday 13 April 2009 23:01:23 Ivan Voras wrote: >> Robert Watson wrote: >> > --- i386/include/param.h (revision 190941) >> > +++ i386/include/param.h (working copy) >> > @@ -74,6 +74,10 @@ >> > #define ALIGNBYTES _ALIGNBYTES >> > #define ALIGN(p) _ALIGN(p) >> > >> > +#ifndef CACHE_LINE_SIZE >> > +#define CACHE_LINE_SIZE 64 >> > +#endif >> >> Wouldn't it be better to continue the >> >> cpu I486_CPU >> cpu I586_CPU >> cpu I686_CPU >> >> series of defines in kernel configuration and define alignment per >> CPU architecture? > > We would have to extend our notion of "CPU architecture" for that to > make sense. For example, Pentium Pro / II CPUs had cache line size of > 32 bytes, Intel Netburst CPUs (all Pentium-4 and Xeons of the time) > have / had 128 bytes, while Pentium-III, Pentium-M and later Core CPUs > have 64 bytes. They are all I686_CPU in our view. I missed the beginning of the discussion. AFAIR we had some code which =20 determined the cache parameters at run time at least on i386. It may =20 be the case that this was removed when the VM system changed to not =20 need the page coloring we had. If this is the case, it may be =20 beneficial to look up the VCS history and resurrect this. Bye, Alexander. --=20 These days the necessities of life cost you about three times what they used to, and half the time they aren't even fit to drink. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID =3D B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID =3D 72077137 From owner-freebsd-arch@FreeBSD.ORG Tue Apr 14 19:30:52 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 757D6106568E; Tue, 14 Apr 2009 19:30:52 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 15CD58FC14; Tue, 14 Apr 2009 19:30:52 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id E5F8E46B96; Tue, 14 Apr 2009 15:30:50 -0400 (EDT) Received: from jhbbsd.hudson-trading.com (unknown [209.249.190.8]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 5B2A38A04D; Tue, 14 Apr 2009 15:30:49 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Date: Tue, 14 Apr 2009 09:48:34 -0400 User-Agent: KMail/1.9.7 References: <200904140105.20664.max@love2party.net> In-Reply-To: <200904140105.20664.max@love2party.net> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200904140948.34503.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Tue, 14 Apr 2009 15:30:49 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=1.5 required=4.2 tests=DATE_IN_PAST_03_06,RDNS_NONE autolearn=no version=3.2.5 X-Spam-Level: * X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Max Laier , Robert Watson Subject: Re: Simple #define for cache line size X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Apr 2009 19:30:54 -0000 On Monday 13 April 2009 7:05:20 pm Max Laier wrote: > On Saturday 11 April 2009 18:11:55 Robert Watson wrote: > > Dear all: > > > > We have a number of __aligned() qualifiers scattered around the kernel > > intended to space objects to improve alignment with respect to cache lines. > > This is important for a number of reasons, not least avoiding cache line > > thrashing when using arrays of foo[MAXCPU]. What I'd like to do is provide > > a single compile-time constant, CACHE_LINE_SIZE, defined in > > machine-dependent param.h, to use for spacing such objects, rather than > > hard-coding various values around. Here are some examples of existing > > spacing attempts: > > How much does __aligned(FOO) even achieve? If we allocate these structs at > runtime the alignment requirements have to be passed to the allocator as well > (eg. uma(9)'s align parameter in uma_zcreate). If my assumption is correct it > would make sense to have a global symbol that is initialized early in the boot > process instead of a compile time #define. > > In addition it seems that we need to make the define a worst case value to > assure correctness for static arrays as below. Actually, we need both the boot-time tunable that UMA has I think and a worst-case constant for static arrays. UMA already has the tunable and I think it should remain (though we may want to provide a way for the MD code to adjust its value). I think the constant is still needed for static arrays and should also be used to initialize the constant in UMA. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Apr 15 12:36:06 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B33BB106564A for ; Wed, 15 Apr 2009 12:36:06 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 2EED18FC18 for ; Wed, 15 Apr 2009 12:36:05 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1Lu4LS-0005ZC-9D for freebsd-arch@freebsd.org; Wed, 15 Apr 2009 12:36:02 +0000 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 15 Apr 2009 12:36:02 +0000 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 15 Apr 2009 12:36:02 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Ivan Voras Date: Wed, 15 Apr 2009 14:35:42 +0200 Lines: 56 Message-ID: References: <200904140842.46501.zec@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig32A0C9624B0C38A8DC414FE1" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Thunderbird 2.0.0.21 (X11/20090318) In-Reply-To: <200904140842.46501.zec@freebsd.org> X-Enigmail-Version: 0.95.0 Sender: news Subject: Re: Simple #define for cache line size X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Apr 2009 12:36:07 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig32A0C9624B0C38A8DC414FE1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Marko Zec wrote: > On Monday 13 April 2009 23:01:23 Ivan Voras wrote: >> Robert Watson wrote: >>> --- i386/include/param.h (revision 190941) >>> +++ i386/include/param.h (working copy) >>> @@ -74,6 +74,10 @@ >>> #define ALIGNBYTES _ALIGNBYTES >>> #define ALIGN(p) _ALIGN(p) >>> >>> +#ifndef CACHE_LINE_SIZE >>> +#define CACHE_LINE_SIZE 64 >>> +#endif >> Wouldn't it be better to continue the >> >> cpu I486_CPU >> cpu I586_CPU >> cpu I686_CPU >> >> series of defines in kernel configuration and define alignment per >> CPU architecture? >=20 > We would have to extend our notion of "CPU architecture" for that to=20 > make sense. For example, Pentium Pro / II CPUs had cache line size of = > 32 bytes, Intel Netburst CPUs (all Pentium-4 and Xeons of the time)=20 > have / had 128 bytes, while Pentium-III, Pentium-M and later Core CPUs = > have 64 bytes. They are all I686_CPU in our view. Yes, because it looks like we stopped tracking them after PPro/II - AFAIK they are the ones originally designated as i686. Runtime checks are more certain to be correct but will performance be the same (versus statically computed offsets)? Some code that should take note of cache line alignment is very performance sensitive. --------------enig32A0C9624B0C38A8DC414FE1 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJ5dSoldnAQVacBcgRAoafAJsHyAci/UbvD1IhZsEjk6HHGpLMWACfa+tU cpBH+ABnKK5ZgA4HMk+2Pwc= =vzRJ -----END PGP SIGNATURE----- --------------enig32A0C9624B0C38A8DC414FE1-- From owner-freebsd-arch@FreeBSD.ORG Wed Apr 15 17:24:15 2009 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D286610656CD for ; Wed, 15 Apr 2009 17:24:15 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id A8DB48FC12 for ; Wed, 15 Apr 2009 17:24:15 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 46D3646B85 for ; Wed, 15 Apr 2009 13:24:15 -0400 (EDT) Received: from jhbbsd.hudson-trading.com (unknown [209.249.190.8]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 38A7E8A04D for ; Wed, 15 Apr 2009 13:24:14 -0400 (EDT) From: John Baldwin To: arch@FreeBSD.org Date: Wed, 15 Apr 2009 13:24:06 -0400 User-Agent: KMail/1.9.7 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200904151324.06754.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Wed, 15 Apr 2009 13:24:14 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=0.1 required=4.2 tests=RDNS_NONE autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Subject: Enabling interrupt filters by default X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Apr 2009 17:24:16 -0000 A while ago I changed the interrupt code in 8.x such that all the MD code was the same for both the INTR_FILTER and non-INTR_FILTER case. I would like to flip the switch to enable INTR_FILTER by default. Any objections? -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Apr 15 18:05:36 2009 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 85EFE106566C for ; Wed, 15 Apr 2009 18:05:36 +0000 (UTC) (envelope-from xcllnt@mac.com) Received: from asmtpout018.mac.com (asmtpout018.mac.com [17.148.16.93]) by mx1.freebsd.org (Postfix) with ESMTP id 737FC8FC15 for ; Wed, 15 Apr 2009 18:05:36 +0000 (UTC) (envelope-from xcllnt@mac.com) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Received: from macbook-pro.jnpr.net ([66.129.224.36]) by asmtp018.mac.com (Sun Java(tm) System Messaging Server 6.3-8.01 (built Dec 16 2008; 32bit)) with ESMTPSA id <0KI500ATMM9BZ060@asmtp018.mac.com>; Wed, 15 Apr 2009 11:05:36 -0700 (PDT) Message-id: <16C23095-4023-499D-896C-8A7A478FDCE4@mac.com> From: Marcel Moolenaar To: John Baldwin In-reply-to: <200904151324.06754.jhb@freebsd.org> Date: Wed, 15 Apr 2009 11:04:14 -0700 References: <200904151324.06754.jhb@freebsd.org> X-Mailer: Apple Mail (2.930.3) Cc: arch@FreeBSD.org Subject: Re: Enabling interrupt filters by default X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Apr 2009 18:05:36 -0000 On Apr 15, 2009, at 10:24 AM, John Baldwin wrote: > A while ago I changed the interrupt code in 8.x such that all the MD > code was > the same for both the INTR_FILTER and non-INTR_FILTER case. I would > like to > flip the switch to enable INTR_FILTER by default. Any objections? Last time it was found to be not working. Did we fix it? -- Marcel Moolenaar xcllnt@mac.com From owner-freebsd-arch@FreeBSD.ORG Wed Apr 15 20:19:52 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7FBB01065752 for ; Wed, 15 Apr 2009 20:19:52 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 539498FC0C for ; Wed, 15 Apr 2009 20:19:52 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id E99F246B8D; Wed, 15 Apr 2009 16:19:51 -0400 (EDT) Received: from jhbbsd.hudson-trading.com (unknown [209.249.190.8]) by bigwig.baldwin.cx (Postfix) with ESMTPA id E62168A04D; Wed, 15 Apr 2009 16:19:50 -0400 (EDT) From: John Baldwin To: Marcel Moolenaar Date: Wed, 15 Apr 2009 16:13:50 -0400 User-Agent: KMail/1.9.7 References: <200904151324.06754.jhb@freebsd.org> <16C23095-4023-499D-896C-8A7A478FDCE4@mac.com> In-Reply-To: <16C23095-4023-499D-896C-8A7A478FDCE4@mac.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200904151613.50568.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Wed, 15 Apr 2009 16:19:50 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=0.1 required=4.2 tests=RDNS_NONE autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: arch@freebsd.org Subject: Re: Enabling interrupt filters by default X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Apr 2009 20:19:52 -0000 On Wednesday 15 April 2009 2:04:14 pm Marcel Moolenaar wrote: > > On Apr 15, 2009, at 10:24 AM, John Baldwin wrote: > > > A while ago I changed the interrupt code in 8.x such that all the MD > > code was > > the same for both the INTR_FILTER and non-INTR_FILTER case. I would > > like to > > flip the switch to enable INTR_FILTER by default. Any objections? > > Last time it was found to be not working. Did we fix it? Err, when was that? I know folks have used it on amd64 and i386 ok and I have tested it on both of those platforms. One of the arm kernel configs uses it by default. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Apr 15 20:37:52 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C48D01065672; Wed, 15 Apr 2009 20:37:52 +0000 (UTC) (envelope-from xcllnt@mac.com) Received: from asmtpout012.mac.com (asmtpout012.mac.com [17.148.16.87]) by mx1.freebsd.org (Postfix) with ESMTP id B056C8FC17; Wed, 15 Apr 2009 20:37:52 +0000 (UTC) (envelope-from xcllnt@mac.com) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Received: from macbook-pro.jnpr.net ([66.129.224.36]) by asmtp012.mac.com (Sun Java(tm) System Messaging Server 6.3-8.01 (built Dec 16 2008; 32bit)) with ESMTPSA id <0KI50065STB3AT00@asmtp012.mac.com>; Wed, 15 Apr 2009 13:37:52 -0700 (PDT) Message-id: From: Marcel Moolenaar To: John Baldwin In-reply-to: <200904151613.50568.jhb@freebsd.org> Date: Wed, 15 Apr 2009 13:36:30 -0700 References: <200904151324.06754.jhb@freebsd.org> <16C23095-4023-499D-896C-8A7A478FDCE4@mac.com> <200904151613.50568.jhb@freebsd.org> X-Mailer: Apple Mail (2.930.3) Cc: arch@freebsd.org Subject: Re: Enabling interrupt filters by default X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Apr 2009 20:37:53 -0000 On Apr 15, 2009, at 1:13 PM, John Baldwin wrote: > On Wednesday 15 April 2009 2:04:14 pm Marcel Moolenaar wrote: >> >> On Apr 15, 2009, at 10:24 AM, John Baldwin wrote: >> >>> A while ago I changed the interrupt code in 8.x such that all the MD >>> code was >>> the same for both the INTR_FILTER and non-INTR_FILTER case. I would >>> like to >>> flip the switch to enable INTR_FILTER by default. Any objections? >> >> Last time it was found to be not working. Did we fix it? > > Err, when was that? August 2007. > I know folks have used it on amd64 and i386 ok and I have > tested it on both of those platforms. One of the arm kernel configs > uses it > by default. There was interrupt starvation on sparc64. There were also issues with permanently masking stray interrupts. This is problematic when interrupts are shared and there is at least 1 filter on it. FYI, -- Marcel Moolenaar xcllnt@mac.com From owner-freebsd-arch@FreeBSD.ORG Wed Apr 15 21:40:37 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 657E01065691 for ; Wed, 15 Apr 2009 21:40:37 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 37C678FC1B for ; Wed, 15 Apr 2009 21:40:37 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id CE2DD46B23; Wed, 15 Apr 2009 17:40:36 -0400 (EDT) Received: from jhbbsd.hudson-trading.com (unknown [209.249.190.8]) by bigwig.baldwin.cx (Postfix) with ESMTPA id A36508A04D; Wed, 15 Apr 2009 17:40:35 -0400 (EDT) From: John Baldwin To: Marcel Moolenaar Date: Wed, 15 Apr 2009 17:37:09 -0400 User-Agent: KMail/1.9.7 References: <200904151324.06754.jhb@freebsd.org> <200904151613.50568.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200904151737.09769.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Wed, 15 Apr 2009 17:40:35 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=0.1 required=4.2 tests=RDNS_NONE autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: arch@freebsd.org Subject: Re: Enabling interrupt filters by default X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Apr 2009 21:40:38 -0000 On Wednesday 15 April 2009 4:36:30 pm Marcel Moolenaar wrote: > > On Apr 15, 2009, at 1:13 PM, John Baldwin wrote: > > > On Wednesday 15 April 2009 2:04:14 pm Marcel Moolenaar wrote: > >> > >> On Apr 15, 2009, at 10:24 AM, John Baldwin wrote: > >> > >>> A while ago I changed the interrupt code in 8.x such that all the MD > >>> code was > >>> the same for both the INTR_FILTER and non-INTR_FILTER case. I would > >>> like to > >>> flip the switch to enable INTR_FILTER by default. Any objections? > >> > >> Last time it was found to be not working. Did we fix it? > > > > Err, when was that? > > August 2007. I rototilled all the MD interrupt code to make both the filter and !filter MD code identical and both sets use the same callout routines (post_filter, etc.) in April 2008. > > I know folks have used it on amd64 and i386 ok and I have > > tested it on both of those platforms. One of the arm kernel configs > > uses it > > by default. > > There was interrupt starvation on sparc64. There were also > issues with permanently masking stray interrupts. This is > problematic when interrupts are shared and there is at least > 1 filter on it. > > FYI, The MD interrupt code has changed quite a bit since then and I explicitly worked with marius@ and others to test the aforementioned changes (though various platforms may have only tested the !filter case at the time). -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Apr 15 21:51:42 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6A2F0106566B; Wed, 15 Apr 2009 21:51:42 +0000 (UTC) (envelope-from p.pisati@oltrelinux.com) Received: from joe.mail.tiscali.it (joe.mail.tiscali.it [213.205.33.54]) by mx1.freebsd.org (Postfix) with ESMTP id ED4AC8FC0A; Wed, 15 Apr 2009 21:51:41 +0000 (UTC) (envelope-from p.pisati@oltrelinux.com) Received: from newluxor.wired.org (94.36.100.254) by joe.mail.tiscali.it (8.0.022) id 499F03960284E95E; Wed, 15 Apr 2009 23:40:26 +0200 Message-ID: <49E65448.4090302@oltrelinux.com> Date: Wed, 15 Apr 2009 23:40:24 +0200 From: Paolo Pisati User-Agent: Thunderbird 2.0.0.18 (X11/20081214) MIME-Version: 1.0 To: Marcel Moolenaar References: <200904151324.06754.jhb@freebsd.org> <16C23095-4023-499D-896C-8A7A478FDCE4@mac.com> <200904151613.50568.jhb@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org Subject: Re: Enabling interrupt filters by default X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Apr 2009 21:51:42 -0000 Marcel Moolenaar wrote: > > August 2007. AFAIK there were different menaces to turn it on but i was never done. > > There was interrupt starvation on sparc64. there were problems with some bridges back then, but i don't know the status today: you could try and tell us :) > There were also > issues with permanently masking stray interrupts. This is > problematic when interrupts are shared and there is at least > 1 filter on it. IIRC that was my mistake trying to cool down interrupt storms, and that piece of code was removed. From owner-freebsd-arch@FreeBSD.ORG Thu Apr 16 17:03:33 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 20070106566B; Thu, 16 Apr 2009 17:03:33 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (alchemy.franken.de [194.94.249.214]) by mx1.freebsd.org (Postfix) with ESMTP id A72BE8FC0A; Thu, 16 Apr 2009 17:03:32 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (localhost [127.0.0.1]) by alchemy.franken.de (8.14.3/8.14.3/ALCHEMY.FRANKEN.DE) with ESMTP id n3GH3Voa031290; Thu, 16 Apr 2009 19:03:31 +0200 (CEST) (envelope-from marius@alchemy.franken.de) Received: (from marius@localhost) by alchemy.franken.de (8.14.3/8.14.3/Submit) id n3GH3VWj031289; Thu, 16 Apr 2009 19:03:31 +0200 (CEST) (envelope-from marius) Date: Thu, 16 Apr 2009 19:03:31 +0200 From: Marius Strobl To: John Baldwin Message-ID: <20090416170331.GA30118@alchemy.franken.de> References: <200904151324.06754.jhb@freebsd.org> <200904151613.50568.jhb@freebsd.org> <200904151737.09769.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200904151737.09769.jhb@freebsd.org> User-Agent: Mutt/1.4.2.3i Cc: arch@freebsd.org, Marcel Moolenaar Subject: Re: Enabling interrupt filters by default X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Apr 2009 17:03:33 -0000 On Wed, Apr 15, 2009 at 05:37:09PM -0400, John Baldwin wrote: > On Wednesday 15 April 2009 4:36:30 pm Marcel Moolenaar wrote: > > > > On Apr 15, 2009, at 1:13 PM, John Baldwin wrote: > > > > > On Wednesday 15 April 2009 2:04:14 pm Marcel Moolenaar wrote: > > >> > > >> On Apr 15, 2009, at 10:24 AM, John Baldwin wrote: > > >> > > >>> A while ago I changed the interrupt code in 8.x such that all the MD > > >>> code was > > >>> the same for both the INTR_FILTER and non-INTR_FILTER case. I would > > >>> like to > > >>> flip the switch to enable INTR_FILTER by default. Any objections? > > >> > > >> Last time it was found to be not working. Did we fix it? > > > > > > Err, when was that? > > > > August 2007. > > I rototilled all the MD interrupt code to make both the filter and !filter MD > code identical and both sets use the same callout routines (post_filter, > etc.) in April 2008. > > > > I know folks have used it on amd64 and i386 ok and I have > > > tested it on both of those platforms. One of the arm kernel configs > > > uses it > > > by default. > > > > There was interrupt starvation on sparc64. There were also > > issues with permanently masking stray interrupts. This is > > problematic when interrupts are shared and there is at least > > 1 filter on it. > > > > FYI, > > The MD interrupt code has changed quite a bit since then and I explicitly > worked with marius@ and others to test the aforementioned changes (though > various platforms may have only tested the !filter case at the time). The MI part of INTR_FILTER still doesn't work properly when multiple filters share one interrupt, resulting in a hang during device attachment. After reviewing the INTR_FILTER code back in August 2007 scottl@ wrote a private mail to marcel@ and me confirming that problem and saying he even found more and even excusing for having pushed the switch to INTR_FILTER for 7.0. Given that the INTR_FILTER code in kern_intr.c for the most part seems unchanged since piso@ committed it probably means that these problems still exist today. Apart from the problem when filters share an interrupt, INTR_FILTER looked good on sparc64 though last time I tested. Marius From owner-freebsd-arch@FreeBSD.ORG Thu Apr 16 17:39:36 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 14A361065687 for ; Thu, 16 Apr 2009 17:39:36 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id D7E378FC1F for ; Thu, 16 Apr 2009 17:39:35 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 8C9AA46B35; Thu, 16 Apr 2009 13:39:35 -0400 (EDT) Received: from jhbbsd.hudson-trading.com (unknown [209.249.190.8]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 807D08A01C; Thu, 16 Apr 2009 13:39:34 -0400 (EDT) From: John Baldwin To: Marius Strobl Date: Thu, 16 Apr 2009 13:37:38 -0400 User-Agent: KMail/1.9.7 References: <200904151324.06754.jhb@freebsd.org> <200904151737.09769.jhb@freebsd.org> <20090416170331.GA30118@alchemy.franken.de> In-Reply-To: <20090416170331.GA30118@alchemy.franken.de> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200904161337.38593.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Thu, 16 Apr 2009 13:39:34 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=0.1 required=4.2 tests=RDNS_NONE autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: arch@freebsd.org, Marcel Moolenaar Subject: Re: Enabling interrupt filters by default X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Apr 2009 17:39:39 -0000 On Thursday 16 April 2009 1:03:31 pm Marius Strobl wrote: > On Wed, Apr 15, 2009 at 05:37:09PM -0400, John Baldwin wrote: > > On Wednesday 15 April 2009 4:36:30 pm Marcel Moolenaar wrote: > > > > > > On Apr 15, 2009, at 1:13 PM, John Baldwin wrote: > > > > > > > On Wednesday 15 April 2009 2:04:14 pm Marcel Moolenaar wrote: > > > >> > > > >> On Apr 15, 2009, at 10:24 AM, John Baldwin wrote: > > > >> > > > >>> A while ago I changed the interrupt code in 8.x such that all the MD > > > >>> code was > > > >>> the same for both the INTR_FILTER and non-INTR_FILTER case. I would > > > >>> like to > > > >>> flip the switch to enable INTR_FILTER by default. Any objections? > > > >> > > > >> Last time it was found to be not working. Did we fix it? > > > > > > > > Err, when was that? > > > > > > August 2007. > > > > I rototilled all the MD interrupt code to make both the filter and !filter MD > > code identical and both sets use the same callout routines (post_filter, > > etc.) in April 2008. > > > > > > I know folks have used it on amd64 and i386 ok and I have > > > > tested it on both of those platforms. One of the arm kernel configs > > > > uses it > > > > by default. > > > > > > There was interrupt starvation on sparc64. There were also > > > issues with permanently masking stray interrupts. This is > > > problematic when interrupts are shared and there is at least > > > 1 filter on it. > > > > > > FYI, > > > > The MD interrupt code has changed quite a bit since then and I explicitly > > worked with marius@ and others to test the aforementioned changes (though > > various platforms may have only tested the !filter case at the time). > > The MI part of INTR_FILTER still doesn't work properly when > multiple filters share one interrupt, resulting in a hang > during device attachment. After reviewing the INTR_FILTER code > back in August 2007 scottl@ wrote a private mail to marcel@ > and me confirming that problem and saying he even found more > and even excusing for having pushed the switch to INTR_FILTER > for 7.0. Given that the INTR_FILTER code in kern_intr.c for the > most part seems unchanged since piso@ committed it probably > means that these problems still exist today. Apart from the > problem when filters share an interrupt, INTR_FILTER looked > good on sparc64 though last time I tested. Ok, I have not heard of this before. I will try to devise some test cases. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Apr 16 18:51:26 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B4D2A106566C for ; Thu, 16 Apr 2009 18:51:26 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id D61268FC12 for ; Thu, 16 Apr 2009 18:51:25 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id VAA26996; Thu, 16 Apr 2009 21:35:49 +0300 (EEST) (envelope-from avg@icyb.net.ua) Message-ID: <49E77A84.1080501@icyb.net.ua> Date: Thu, 16 Apr 2009 21:35:48 +0300 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.21 (X11/20090406) MIME-Version: 1.0 To: John Baldwin References: <200904151324.06754.jhb@freebsd.org> <200904151737.09769.jhb@freebsd.org> <20090416170331.GA30118@alchemy.franken.de> <200904161337.38593.jhb@freebsd.org> In-Reply-To: <200904161337.38593.jhb@freebsd.org> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, Marcel Moolenaar , Marius Strobl Subject: Re: Enabling interrupt filters by default X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Apr 2009 18:51:27 -0000 On Thursday 16 April 2009 1:03:31 pm Marius Strobl wrote: > The MI part of INTR_FILTER still doesn't work properly when > multiple filters share one interrupt, resulting in a hang > during device attachment. Marius, is this observation specific to sparc64 or is it more common/universal? -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Thu Apr 16 20:03:34 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DC631106566C; Thu, 16 Apr 2009 20:03:34 +0000 (UTC) (envelope-from xcllnt@mac.com) Received: from asmtpout022.mac.com (asmtpout022.mac.com [17.148.16.97]) by mx1.freebsd.org (Postfix) with ESMTP id C4AAF8FC29; Thu, 16 Apr 2009 20:03:34 +0000 (UTC) (envelope-from xcllnt@mac.com) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Received: from slim-mbp.jnpr.net ([66.129.224.36]) by asmtp022.mac.com (Sun Java(tm) System Messaging Server 6.3-8.01 (built Dec 16 2008; 32bit)) with ESMTPSA id <0KI700BCBMDSED50@asmtp022.mac.com>; Thu, 16 Apr 2009 13:03:29 -0700 (PDT) Message-id: <66B01309-1855-40E9-8EE7-E13423A0A9FD@mac.com> From: Marcel Moolenaar To: Andriy Gapon In-reply-to: <49E77A84.1080501@icyb.net.ua> Date: Thu, 16 Apr 2009 13:02:06 -0700 References: <200904151324.06754.jhb@freebsd.org> <200904151737.09769.jhb@freebsd.org> <20090416170331.GA30118@alchemy.franken.de> <200904161337.38593.jhb@freebsd.org> <49E77A84.1080501@icyb.net.ua> X-Mailer: Apple Mail (2.930.3) Cc: arch@freebsd.org, Marius Strobl Subject: Re: Enabling interrupt filters by default X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Apr 2009 20:03:36 -0000 On Apr 16, 2009, at 11:35 AM, Andriy Gapon wrote: > On Thursday 16 April 2009 1:03:31 pm Marius Strobl wrote: >> The MI part of INTR_FILTER still doesn't work properly when >> multiple filters share one interrupt, resulting in a hang >> during device attachment. > > Marius, > is this observation specific to sparc64 or is it more common/ > universal? Universal. I've seen it on i386, ia64 and powerpc. FYI, -- Marcel Moolenaar xcllnt@mac.com From owner-freebsd-arch@FreeBSD.ORG Sat Apr 18 20:55:15 2009 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 63BBC106566C for ; Sat, 18 Apr 2009 20:55:15 +0000 (UTC) (envelope-from robert@fledge.watson.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 184C18FC0C for ; Sat, 18 Apr 2009 20:55:15 +0000 (UTC) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id C9CA446B49 for ; Sat, 18 Apr 2009 16:55:14 -0400 (EDT) X-Return-Path: X-Received: from cyrus.watson.org ([unix socket]) by cyrus.watson.org (Cyrus v2.3.13) with LMTPA; Sat, 18 Apr 2009 16:54:51 -0400 X-Sieve: CMU Sieve 2.3 X-Received: from mx2.freebsd.org (mx2.freebsd.org [69.147.83.53]) by cyrus.watson.org (Postfix) with ESMTP id 4037146B91 for ; Sat, 18 Apr 2009 16:54:51 -0400 (EDT) X-Received: from hub.freebsd.org (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id D2112153A03; Sat, 18 Apr 2009 20:54:33 +0000 (UTC) (envelope-from owner-freebsd-current@freebsd.org) X-Received: from hub.freebsd.org (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id 526EF1065706; Sat, 18 Apr 2009 20:54:30 +0000 (UTC) (envelope-from owner-freebsd-current@freebsd.org) X-Delivered-To: current@FreeBSD.org X-Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 16859106566C; Sat, 18 Apr 2009 20:54:22 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) X-Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id E6A8F8FC12; Sat, 18 Apr 2009 20:54:21 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) X-Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 9E3C946B23; Sat, 18 Apr 2009 16:54:21 -0400 (EDT) Date: Sat, 18 Apr 2009 21:54:21 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: net@FreeBSD.org, current@FreeBSD.org Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Sender: owner-freebsd-current@freebsd.org Errors-To: owner-freebsd-current@freebsd.org ReSent-Date: Sat, 18 Apr 2009 21:55:10 +0100 (BST) ReSent-From: robert ReSent-To: arch@FreeBSD.org ReSent-Subject: IFF_NEEDSGIANT now gone from 8.x (was: svn commit: r191253 - head/sys/net (fwd)) ReSent-Message-ID: ReSent-User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) Cc: Subject: IFF_NEEDSGIANT now gone from 8.x (was: svn commit: r191253 - head/sys/net (fwd)) X-BeenThere: freebsd-arch@freebsd.org List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Apr 2009 20:55:15 -0000 Dear all: Just under four years ago, the non-MPSAFE network stack de-orbit burn schedule was announced, setting out a plan for eliminating remaining use of the Giant lock in the FreeBSD network stack. With the attached commit, that plan is now complete, and almost all of the network stack neither requires Giant nor runs with it. As always there are some loose ends, especially in IPv6, but with any luck those can be dealt with 8.0 also. Special thanks are due to the people who worked on and shepherded the last steps of this process -- especially Hans Petter Selasky, Alfred Perlstein, Andrew Thompson, Ed Schouten, and John Baldwin, who collectively bought our USB, tty, and other non-MPSAFE device driver stacks into a post-SMPng world. Thanks, Robert N M Watson Computer Laboratory University of Cambridge ---------- Forwarded message ---------- Date: Sat, 18 Apr 2009 20:39:18 +0000 (UTC) From: Robert Watson To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r191253 - head/sys/net Author: rwatson Date: Sat Apr 18 20:39:17 2009 New Revision: 191253 URL: http://svn.freebsd.org/changeset/base/191253 Log: Remove IFF_NEEDSGIANT interface flag: we no longer provide ifnet-layer infrastructure to support non-MPSAFE network device drivers. Modified: head/sys/net/if.h Modified: head/sys/net/if.h ============================================================================== --- head/sys/net/if.h Sat Apr 18 20:10:39 2009 (r191252) +++ head/sys/net/if.h Sat Apr 18 20:39:17 2009 (r191253) @@ -149,7 +149,6 @@ struct if_data { #define IFF_PPROMISC 0x20000 /* (n) user-requested promisc mode */ #define IFF_MONITOR 0x40000 /* (n) user-requested monitor mode */ #define IFF_STATICARP 0x80000 /* (n) static ARP */ -#define IFF_NEEDSGIANT 0x100000 /* (i) hold Giant over if_start calls */ /* * Old names for driver flags so that user space tools can continue to use @@ -163,8 +162,7 @@ struct if_data { /* flags set internally only: */ #define IFF_CANTCHANGE \ (IFF_BROADCAST|IFF_POINTOPOINT|IFF_DRV_RUNNING|IFF_DRV_OACTIVE|\ - IFF_SIMPLEX|IFF_MULTICAST|IFF_ALLMULTI|IFF_SMART|IFF_PROMISC|\ - IFF_NEEDSGIANT) + IFF_SIMPLEX|IFF_MULTICAST|IFF_ALLMULTI|IFF_SMART|IFF_PROMISC) /* * Values for if_link_state. _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"