From owner-freebsd-net@FreeBSD.ORG Fri Jan 15 02:27:00 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C78BF1065695 for ; Fri, 15 Jan 2010 02:27:00 +0000 (UTC) (envelope-from julian@elischer.org) Received: from utility-0.aerioconnect.net (utility-0.aerioconnect.net [216.240.32.11]) by mx1.freebsd.org (Postfix) with ESMTP id AB7718FC12 for ; Fri, 15 Jan 2010 02:27:00 +0000 (UTC) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by utility-0.aerioconnect.net (8.13.1/8.13.1) with ESMTP id o0F2Qvax016246; Thu, 14 Jan 2010 18:26:57 -0800 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id 36E5C2D6015; Thu, 14 Jan 2010 18:26:57 -0800 (PST) Message-ID: <4B4FD270.7010808@elischer.org> Date: Thu, 14 Jan 2010 18:26:56 -0800 From: Julian Elischer User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: Erik Klavon References: <20100114224635.GA27139@malcolm.berkeley.edu> <4B4FB547.8000202@elischer.org> <20100115014254.GA29258@malcolm.berkeley.edu> In-Reply-To: <20100115014254.GA29258@malcolm.berkeley.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org Subject: Re: netgraph mkpeer and connect failures with ng_ipfw and ng_nat X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jan 2010 02:27:00 -0000 Erik Klavon wrote: > On Thu, Jan 14, 2010 at 04:22:31PM -0800, Julian Elischer wrote: >> Erik Klavon wrote: >>> Here are the hooks for one ng_nat(4) node of interest. At the time I >>> obtained this information, this node was working correctly. Everything >>> in this output looks correct. >>> >>> sudo ngctl show ipfw:202182138 >>> Name: WirelessNAT2182138 Type: nat ID: 000062ee Num hooks: >>> 2 >>> Local hook Peer name Peer type Peer ID Peer hook >>> ---------- --------- --------- ------- --------- >>> in ipfw ipfw 00005a83 102182138 >>> out ipfw ipfw 00005a83 202182138 >>> >>> sudo ngctl msg ipfw:102182138 listredirects >>> Rec'd response "listredirects" (10) from "[62ee]:": >>> Args: { total_count=1 redirects=[ { id=1 local_addr=10.10.118.43 >>> alias_addr=136.152.182.138 proto=259 description="Static NAT" } ] } >>> >>> Running show on ipfw:102174202, I see that this hook is pointing to >>> the above ng_nat(4) node WirelessNAT2182138. >> can you show that output? > > The output for ngctl show ipfw:102174202 is below. how about the output for ipfw: ? > >>> sudo ngctl show ipfw:102174202 >>> Name: WirelessNAT2182138 Type: nat ID: 000062ee Num hooks: >>> 2 >>> Local hook Peer name Peer type Peer ID Peer hook >>> ---------- --------- --------- ------- --------- >>> in ipfw ipfw 00005a83 102182138 >>> out ipfw ipfw 00005a83 202182138 >>> >>> But WirelessNAT2182138 has no record of a hook102174202. Somehow, two >>> hooks used to refer to what should be two different NAT sessions are >>> pointing to the same ng_nat(4) object (i.e. one session). If I run >>> shutdown on ipfw:102174202, WirelessNAT2182138 goes away. Given the >>> above calls to ngctl(8), I don't know what is causing these two separate >>> hooks to refer to the same ng_nat(4) object. >> you might name the ipfw nodes to make the output clearer. > > ng_ipfw(4) uses a single node to pass traffic from ipfw(4) to > netgraph(4). You associate as many hooks with this node as you like and > then direct traffic via the single ng_ipfw(4) node to different hooks > using ipfw(8) rules such as > > 01230 netgraph tablearg ip from table(87) to any in > > Suppose a packet comes in for IP address 10.10.113.226 and rule 1230 > is reached during the processing of the ipfw ruleset. According to the > above rule, ipfw(4) will look up 10.10.113.226/32 in table 87 and if > it finds a matching key in the table, it will tag the packet with the > matching table entry's value and send the packet to the ng_ipfw(4) > node. ng_ipfw(4) will then examine the tag and look for an associated > hook with a matching name. If such a hook is found the packet is sent > on to the node associated with the hook. In the case of 10.10.113.226, > table 87 contains the following entry. > > 10.10.113.226/32 202173114 > > Hook 202173114 refers to the following ng_nat node. > > sudo ngctl show ipfw:202173114 > Name: WirelessNAT2173114 Type: nat ID: 000015f0 Num hooks: 2 > Local hook Peer name Peer type Peer ID Peer hook > ---------- --------- --------- ------- --------- > in ipfw ipfw 00000001 102173114 > out ipfw ipfw 00000001 202173114 > > which is configured for one to one NAT as follows. > > sudo ngctl msg WirelessNAT2173114: listredirects > Rec'd response "listredirects" (10) from "[15f0]:": > Args: { total_count=1 redirects=[ { id=1 local_addr=10.10.113.226 alias_addr=x.x.173.114 proto=259 description="Static NAT" } ] } > > There is a similar path for traffic coming in to the globally routable > side of these one to one NAT nodes. > >> I have not looked at the ipfw node type much but it is possible >> that is suffers from races. >> Especially in the face of ipfw rule changes. > > Can you think of an example in any other netgraph module of proper > concurrency handling I can use as a reference when looking at > ng_ipfw(4)? > > By using tables we don't need to update the ruleset itself. The > ruleset is (normally) loaded once after the machine boots. The table > updates take place after all the calls to ngctl(8) and after the call > to arp(8). I will take a look at the interface between ipfw(4) and > ng_ipfw(4). > >> have you tried adding small delays (sleep 0.5) betwenn the calls by >> the way? > > I've now added sleep 0.5 between the calls to ngctl. I'll let you know > if that helps at all; thanks for the suggestion. > > Erik