From owner-freebsd-net@FreeBSD.ORG Thu Jan 1 16:59:12 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 026F01065672 for ; Thu, 1 Jan 2009 16:59:12 +0000 (UTC) (envelope-from Michael.Tuexen@lurchi.franken.de) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) by mx1.freebsd.org (Postfix) with ESMTP id 60A0C8FC08 for ; Thu, 1 Jan 2009 16:59:11 +0000 (UTC) (envelope-from Michael.Tuexen@lurchi.franken.de) Received: from [IPv6:2002:508f:d3ce::21e:c2ff:fe00:76c8] (unknown [IPv6:2002:508f:d3ce:0:21e:c2ff:fe00:76c8]) by mail-n.franken.de (Postfix) with ESMTP id 3BF4D1C0B4604 for ; Thu, 1 Jan 2009 17:59:09 +0100 (CET) Message-Id: From: =?ISO-8859-1?Q?Michael_T=FCxen?= To: FreeBSD Net Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Thu, 1 Jan 2009 17:59:07 +0100 X-Mailer: Apple Mail (2.930.3) Subject: SCTP related issue with recent ARP changes? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Jan 2009 16:59:12 -0000 Dear all, I'm running the current CVS version of FreeBSD 8 in a virtual machine using VMWare 2.0.1 on a Mac (not sure if this is relevant) and bridged networking having an em interface on the virtual machine. I'm using a similar setup with older FreeBSD machines and they are running fine. Loopback communication based on UDP, TCP, SCTP or ICMP works fine. Communicating with other hosts using UDP, TCP or ICMP works fine. Communicating with other hosts using SCTP does not work. The SCTP stack calls ip_output() and an ARP request for the correct destination IP address is send out. A corresponding ARP reply is visible in Wireshark (running on the FreeBSD 8 box) and looks good. However, the corresponding SCTP packet it never sent out. I'm not sure, but this might be related to the recent ARP changes. Is there anything required from the transport layer to be done when calling ip_output() that was not required before? Best regards Michael From owner-freebsd-net@FreeBSD.ORG Thu Jan 1 20:19:21 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6350D1065673; Thu, 1 Jan 2009 20:19:21 +0000 (UTC) (envelope-from qing.li@bluecoat.com) Received: from whisker.bluecoat.com (whisker.bluecoat.com [216.52.23.28]) by mx1.freebsd.org (Postfix) with ESMTP id 473128FC16; Thu, 1 Jan 2009 20:19:21 +0000 (UTC) (envelope-from qing.li@bluecoat.com) Received: from bcs-mail03.internal.cacheflow.com ([10.2.2.95]) by whisker.bluecoat.com (8.14.2/8.14.2) with ESMTP id n01KJKFx026243; Thu, 1 Jan 2009 12:19:20 -0800 (PST) X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Date: Thu, 1 Jan 2009 12:16:37 -0800 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: SCTP related issue with recent ARP changes? Thread-Index: AclsMle5U2WQGWG/SKSzlWZ3/NiGZQAG4G76 References: From: "Li, Qing" To: =?iso-8859-1?Q?Michael_T=FCxen?= , "FreeBSD Net" Cc: qingli@freebsd.org, current@freebsd.org Subject: RE: SCTP related issue with recent ARP changes? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Jan 2009 20:19:22 -0000 Hi Michael, Your problem could be related to the recent ARP changes. I will investigate further to confirm.=20 Thanks, -- Qing -----Original Message----- From: owner-freebsd-net@freebsd.org on behalf of Michael T=FCxen Sent: Thu 1/1/2009 8:59 AM To: FreeBSD Net Subject: SCTP related issue with recent ARP changes? =20 Dear all, I'm running the current CVS version of FreeBSD 8 in a virtual machine using VMWare 2.0.1 on a Mac (not sure if this is relevant) and bridged networking having an em interface on the virtual machine. I'm using a similar setup with older FreeBSD machines and they are running fine. Loopback communication based on UDP, TCP, SCTP or ICMP works fine. Communicating with other hosts using UDP, TCP or ICMP works fine. Communicating with other hosts using SCTP does not work. The SCTP stack calls ip_output() and an ARP request for the correct destination IP address is send out. A corresponding ARP reply is visible in Wireshark (running on the FreeBSD 8 box) and looks good. However, the corresponding SCTP packet it never sent out. I'm not sure, but this might be related to the recent ARP changes. Is there anything required from the transport layer to be done when calling ip_output() that was not required before? Best regards Michael _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Fri Jan 2 06:19:57 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4B0AC106564A for ; Fri, 2 Jan 2009 06:19:57 +0000 (UTC) (envelope-from ericlin@tamama.org) Received: from mail-gx0-f18.google.com (mail-gx0-f18.google.com [209.85.217.18]) by mx1.freebsd.org (Postfix) with ESMTP id EEBE98FC12 for ; Fri, 2 Jan 2009 06:19:56 +0000 (UTC) (envelope-from ericlin@tamama.org) Received: by gxk11 with SMTP id 11so435241gxk.19 for ; Thu, 01 Jan 2009 22:19:56 -0800 (PST) Received: by 10.150.157.11 with SMTP id f11mr15337185ybe.194.1230875239409; Thu, 01 Jan 2009 21:47:19 -0800 (PST) Received: by 10.151.68.13 with HTTP; Thu, 1 Jan 2009 21:47:19 -0800 (PST) Message-ID: <47713ee10901012147k1f25c31bn512dd29b2b294ad5@mail.gmail.com> Date: Fri, 2 Jan 2009 13:47:19 +0800 From: "Lin Jui-Nan Eric" To: freebsd-net@freebsd.org In-Reply-To: <47713ee10812301206j12b35264o715976c154080a1b@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <47713ee10812301206j12b35264o715976c154080a1b@mail.gmail.com> Subject: TCP packet out-of-order problem X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jan 2009 06:19:57 -0000 Dear listers, We recently found our new FreeBSD server (located in some foreign region) has poor network performance. After doing some tcpdump and iperf testing, we found that out-of-order TCP packets are not inserted into queue. This is an 100Mbps line, and TSO is disabled. % uname -a FreeBSD bsd 7.1-RC2 FreeBSD 7.1-RC2 #2: Wed Dec 31 03:12:39 CST 2008 root@bsd:/usr/obj/usr/src/sys/KERNEL amd64 % iperf -c 10.1.1.250 ------------------------------------------------------------ Client connecting to office, TCP port 5001 TCP window size: 3.07 MByte (default) ------------------------------------------------------------ [ 4] local 10.1.1.210 port 61488 connected with 10.1.1.250 port 5001 [ ID] Interval Transfer Bandwidth [ 4] 0.0-10.2 sec 5.74 MBytes 4.74 Mbits/sec 03:47:21.146397 IP 10.1.1.210.54919 > 10.1.1.250.5001: . 159305:160753(1448) ack 1 win 1040 03:47:21.146409 IP 10.1.1.250.5001 > 10.1.1.210.54919: . ack 160753 win 12568 03:47:21.146473 IP 10.1.1.210.54919 > 10.1.1.250.5001: . 160753:162201(1448) ack 1 win 1040 03:47:21.146485 IP 10.1.1.250.5001 > 10.1.1.210.54919: . ack 162201 win 12568 03:47:21.146972 IP 10.1.1.210.54919 > 10.1.1.250.5001: . 163649:165097(1448) ack 1 win 1040 03:47:21.146983 IP 10.1.1.250.5001 > 10.1.1.210.54919: . ack 162201 win 12573 03:47:21.146985 IP 10.1.1.210.54919 > 10.1.1.250.5001: . 162201:163649(1448) ack 1 win 1040 03:47:21.146996 IP 10.1.1.250.5001 > 10.1.1.210.54919: . ack 163649 win 12568 03:47:21.146998 IP 10.1.1.210.54919 > 10.1.1.250.5001: . 165097:166545(1448) ack 1 win 1040 03:47:21.147006 IP 10.1.1.250.5001 > 10.1.1.210.54919: . ack 163649 win 12573 03:47:21.147009 IP 10.1.1.210.54919 > 10.1.1.250.5001: . 166545:167993(1448) ack 1 win 1040 03:47:21.147017 IP 10.1.1.250.5001 > 10.1.1.210.54919: . ack 163649 win 12573 03:47:21.147019 IP 10.1.1.210.54919 > 10.1.1.250.5001: . 167993:169441(1448) ack 1 win 1040 * You can see "ack 163649" repeating, but the packet is transmitted before 163649:165097. % cat /etc/sysctl.conf # $FreeBSD: src/etc/sysctl.conf,v 1.8 2003/03/13 18:43:50 mux Exp $ # # This file is read when going to multi-user and its contents piped thru # ``sysctl'' to adjust kernel values. ``man 5 sysctl.conf'' for details. # # Uncomment this to prevent users from seeing information about processes that # are being run under another UID. #security.bsd.see_other_uids=0 debug.bootverbose=1 kern.ipc.somaxconn=8192 kern.maxfiles=65536 kern.maxfilesperproc=32768 kern.maxprocperuid=65536 net.inet.ip.fastforwarding=1 net.inet.tcp.delayed_ack=0 vm.pmap.shpgperproc=2000 kern.ipc.maxsockbuf=8388608 net.inet.tcp.sendspace=3217968 net.inet.tcp.recvspace=3217968 Is our configuration wrong? Or it is an known bug? I have searched stable & net list, but found no similar discussion. Best Regards, Eric From owner-freebsd-net@FreeBSD.ORG Fri Jan 2 06:49:29 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E4C431065673; Fri, 2 Jan 2009 06:49:29 +0000 (UTC) (envelope-from ericlin@tamama.org) Received: from yx-out-2324.google.com (yx-out-2324.google.com [74.125.44.29]) by mx1.freebsd.org (Postfix) with ESMTP id 96CEA8FC13; Fri, 2 Jan 2009 06:49:29 +0000 (UTC) (envelope-from ericlin@tamama.org) Received: by yx-out-2324.google.com with SMTP id 8so2842707yxb.13 for ; Thu, 01 Jan 2009 22:49:29 -0800 (PST) Received: by 10.150.135.2 with SMTP id i2mr33862606ybd.229.1230878968920; Thu, 01 Jan 2009 22:49:28 -0800 (PST) Received: by 10.151.68.13 with HTTP; Thu, 1 Jan 2009 22:49:28 -0800 (PST) Message-ID: <47713ee10901012249w65c659bbp3366e4d8ef25c59d@mail.gmail.com> Date: Fri, 2 Jan 2009 14:49:28 +0800 From: "Lin Jui-Nan Eric" To: freebsd-net@freebsd.org, stable@freebsd.org In-Reply-To: <47713ee10901012147k1f25c31bn512dd29b2b294ad5@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <47713ee10812301206j12b35264o715976c154080a1b@mail.gmail.com> <47713ee10901012147k1f25c31bn512dd29b2b294ad5@mail.gmail.com> Cc: Subject: Re: TCP packet out-of-order problem X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jan 2009 06:49:30 -0000 Dear All listers, After running "netstat -s -p tcp", we found that lots of packets are discarded due to memory problems. We googled for it, and found that sysctl oid "net.inet.tcp.reass.maxsegments" became 0, therefore packets never reassembled. Then we checked our /boot/loader.conf and /etc/sysctl.conf, and found that setting kern.ipc.nmbclusters="0" makes net.inet.tcp.reass.maxsegments=0. After setting net.inet.tcp.reass.maxsegments="1600" in /boot/loader.conf, the network works perfectly now. Thank you all for the help! From owner-freebsd-net@FreeBSD.ORG Fri Jan 2 09:57:50 2009 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9E1151065673; Fri, 2 Jan 2009 09:57:50 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 727BB8FC24; Fri, 2 Jan 2009 09:57:50 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (linimon@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n029voGI091490; Fri, 2 Jan 2009 09:57:50 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n029voif091486; Fri, 2 Jan 2009 09:57:50 GMT (envelope-from linimon) Date: Fri, 2 Jan 2009 09:57:50 GMT Message-Id: <200901020957.n029voif091486@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-net@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/130109: [ipfw] Can not set fib for packets originated from local host X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jan 2009 09:57:51 -0000 Old Synopsis: Can not set fib for packets originated from local host New Synopsis: [ipfw] Can not set fib for packets originated from local host Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Fri Jan 2 09:57:11 UTC 2009 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=130109 From owner-freebsd-net@FreeBSD.ORG Fri Jan 2 21:01:13 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0F4F110656D4; Fri, 2 Jan 2009 21:01:13 +0000 (UTC) (envelope-from jason.dicioccio@ods.org) Received: from mail-ew0-f21.google.com (mail-ew0-f21.google.com [209.85.219.21]) by mx1.freebsd.org (Postfix) with ESMTP id 99B598FC1E; Fri, 2 Jan 2009 21:01:11 +0000 (UTC) (envelope-from jason.dicioccio@ods.org) Received: by ewy14 with SMTP id 14so7793936ewy.19 for ; Fri, 02 Jan 2009 13:01:10 -0800 (PST) Received: by 10.210.119.16 with SMTP id r16mr14077795ebc.47.1230930070340; Fri, 02 Jan 2009 13:01:10 -0800 (PST) Received: by 10.210.80.20 with HTTP; Fri, 2 Jan 2009 13:01:10 -0800 (PST) Message-ID: Date: Fri, 2 Jan 2009 13:01:10 -0800 From: "Jason DiCioccio" To: vwe@freebsd.org In-Reply-To: MIME-Version: 1.0 References: <200812311353.mBVDraLJ042040@freefall.freebsd.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-net@freebsd.org, freebsd-bugs@freebsd.org, bugs-followup@freebsd.org Subject: Re: kern/130059: [panic] Leaking 50k mbufs/hour X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jan 2009 21:01:13 -0000 Adding a bit more info.. Here's the netstat -m and vmstat -z output after the mbufs have had a while to build up: -- netstat -m -- 959510/235/959745 mbufs in use (current/cache/total) 257/133/390/25600 mbuf clusters in use (current/cache/total/max) 257/127 mbuf+clusters out of packet secondary zone in use (current/cache) 0/117/117/12800 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) 240391K/792K/241184K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/152/6656 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 2716 requests for I/O initiated by sendfile 0 calls to protocol drain routines -- vmstat -z -- ITEM SIZE LIMIT USED FREE REQUESTS FAILURES UMA Kegs: 128, 0, 99, 21, 99, 0 UMA Zones: 120, 0, 99, 21, 99, 0 UMA Slabs: 64, 0, 1132, 48, 15590, 0 UMA RCntSlabs: 104, 0, 312, 21, 856, 0 UMA Hash: 128, 0, 4, 26, 7, 0 16 Bucket: 76, 0, 21, 29, 64, 0 32 Bucket: 140, 0, 32, 24, 107, 0 64 Bucket: 268, 0, 34, 8, 142, 0 128 Bucket: 524, 0, 628, 9, 45698, 5387 VM OBJECT: 124, 0, 9136, 32404, 20553589, 0 MAP: 140, 0, 7, 49, 7, 0 KMAP ENTRY: 68, 56560, 17, 487, 153898, 0 MAP ENTRY: 68, 0, 15360, 2336, 53481139, 0 DP fakepg: 72, 0, 0, 0, 0, 0 mt_zone: 72, 0, 194, 71, 194, 0 16: 16, 0, 1996, 643, 81641919, 0 32: 32, 0, 2542, 283, 9470304, 0 64: 64, 0, 4584, 7039, 61275174, 0 128: 128, 0, 1101, 549, 6280868, 0 256: 256, 0, 1299, 366, 20697494, 0 512: 512, 0, 145, 279, 79529, 0 1024: 1024, 0, 173, 71, 532899, 0 2048: 2048, 0, 104, 146, 14198, 0 4096: 4096, 0, 266, 63, 1242097, 0 Files: 76, 0, 1595, 655, 16940144, 0 TURNSTILE: 76, 0, 295, 41, 295, 0 umtx pi: 52, 0, 0, 0, 0, 0 PROC: 696, 0, 229, 46, 541535, 0 THREAD: 552, 0, 283, 11, 952, 0 UPCALL: 44, 0, 0, 0, 0, 0 SLEEPQUEUE: 32, 0, 295, 157, 295, 0 VMSPACE: 232, 0, 196, 59, 541501, 0 cpuset: 40, 0, 2, 182, 2, 0 mbuf_packet: 256, 0, 258, 126, 20757971, 0 mbuf: 256, 0, 960618, 33, 116817015, 0 mbuf_cluster: 2048, 25600, 384, 6, 384, 0 mbuf_jumbo_pagesize: 4096, 12800, 0, 117, 19593255, 0 mbuf_jumbo_9k: 9216, 6400, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 3200, 0, 0, 0, 0 mbuf_ext_refcnt: 4, 0, 0, 406, 26807, 0 ACL UMA zone: 388, 0, 0, 20, 65073710, 0 g_bio: 132, 0, 12, 423, 10043465, 0 ata_request: 192, 0, 3, 337, 3212743, 0 ata_composite: 184, 0, 0, 0, 0, 0 cryptop: 60, 0, 0, 0, 0, 0 cryptodesc: 56, 0, 0, 0, 0, 0 VNODE: 276, 0, 19330, 33618, 3235279, 0 VNODEPOLL: 64, 0, 0, 0, 0, 0 NAMEI: 1024, 0, 0, 28, 38128632, 0 S VFS Cache: 68, 0, 20134, 23490, 10825579, 0 L VFS Cache: 291, 0, 27, 207, 175327, 0 NFSMOUNT: 560, 0, 0, 0, 0, 0 NFSNODE: 452, 0, 0, 0, 0, 0 DIRHASH: 1024, 0, 1946, 170, 28110, 0 pipe: 396, 0, 89, 31, 229902, 0 ksiginfo: 80, 0, 248, 40, 248, 0 itimer: 220, 0, 0, 36, 1, 0 KNOTE: 68, 0, 175, 217, 2157098, 0 socket: 416, 25605, 404, 127, 3609347, 0 unpcb: 168, 25622, 278, 136, 1352116, 0 ipq: 32, 904, 0, 226, 42, 0 udpcb: 180, 25608, 13, 75, 1774380, 0 inpcb: 180, 25608, 155, 197, 482839, 0 tcpcb: 464, 25600, 92, 60, 482839, 0 tcptw: 52, 5184, 63, 297, 96034, 0 syncache: 104, 15392, 9, 102, 474336, 0 hostcache: 76, 15400, 626, 124, 4537, 0 tcpreass: 20, 1690, 0, 169, 92838, 0 sackhole: 20, 0, 0, 169, 7789, 0 sctp_ep: 804, 25600, 0, 0, 0, 0 sctp_asoc: 1412, 40000, 0, 0, 0, 0 sctp_laddr: 24, 80040, 0, 145, 28, 0 sctp_raddr: 400, 80000, 0, 0, 0, 0 sctp_chunk: 92, 400008, 0, 0, 0, 0 sctp_readq: 76, 400000, 0, 0, 0, 0 sctp_stream_msg_out: 60, 400050, 0, 0, 0, 0 sctp_asconf: 24, 400055, 0, 0, 0, 0 sctp_asconf_ack: 24, 400055, 0, 0, 0, 0 ripcb: 180, 25608, 1, 43, 2, 0 rtentry: 124, 0, 78, 46, 3549, 0 SWAPMETA: 276, 121576, 158, 108, 19610, 0 Mountpoints: 716, 0, 6, 4, 6, 0 FFS inode: 128, 0, 19299, 19191, 3235170, 0 FFS1 dinode: 128, 0, 0, 0, 0, 0 FFS2 dinode: 256, 0, 19299, 16011, 3235170, 0 pfsrctrpl: 124, 10013, 0, 0, 0, 0 pfrulepl: 828, 0, 12, 8, 12, 0 pfstatepl: 284, 10010, 0, 0, 0, 0 pfaltqpl: 224, 0, 0, 0, 0, 0 pfpooladdrpl: 68, 0, 6, 106, 6, 0 pfrktable: 1240, 1002, 1, 5, 2, 0 pfrkentry: 156, 200000, 3, 47, 3, 0 pfrkentry2: 156, 0, 0, 0, 0, 0 pffrent: 16, 5075, 0, 0, 0, 0 pffrag: 48, 0, 0, 0, 0, 0 pffrcache: 48, 10062, 0, 0, 0, 0 pffrcent: 12, 50141, 0, 0, 0, 0 pfstatescrub: 28, 0, 0, 0, 0, 0 pfiaddrpl: 100, 0, 0, 0, 0, 0 pfospfen: 108, 0, 696, 24, 696, 0 pfosfp: 28, 0, 407, 228, 407, 0 From owner-freebsd-net@FreeBSD.ORG Fri Jan 2 21:30:11 2009 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 78CD7106567C for ; Fri, 2 Jan 2009 21:30:11 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 624F88FC12 for ; Fri, 2 Jan 2009 21:30:11 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n02LUBXf010190 for ; Fri, 2 Jan 2009 21:30:11 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n02LUB9x010187; Fri, 2 Jan 2009 21:30:11 GMT (envelope-from gnats) Date: Fri, 2 Jan 2009 21:30:11 GMT Message-Id: <200901022130.n02LUB9x010187@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: "Jason DiCioccio" Cc: Subject: Re: kern/130059: [panic] Leaking 50k mbufs/hour X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Jason DiCioccio List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jan 2009 21:30:11 -0000 The following reply was made to PR kern/130059; it has been noted by GNATS. From: "Jason DiCioccio" To: bug-followup@freebsd.org Cc: Subject: Re: kern/130059: [panic] Leaking 50k mbufs/hour Date: Fri, 2 Jan 2009 13:02:47 -0800 ------=_Part_175155_3729370.1230930167365 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Adding a bit more info.. Here's the netstat -m and vmstat -z output after the mbufs have had a while to build up: -- netstat -m -- 959510/235/959745 mbufs in use (current/cache/total) 257/133/390/25600 mbuf clusters in use (current/cache/total/max) 257/127 mbuf+clusters out of packet secondary zone in use (current/cache) 0/117/117/12800 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) 240391K/792K/241184K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/152/6656 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 2716 requests for I/O initiated by sendfile 0 calls to protocol drain routines -- vmstat -z -- ITEM SIZE LIMIT USED FREE REQUESTS FAILURES UMA Kegs: 128, 0, 99, 21, 99, 0 UMA Zones: 120, 0, 99, 21, 99, 0 UMA Slabs: 64, 0, 1132, 48, 15590, 0 UMA RCntSlabs: 104, 0, 312, 21, 856, 0 UMA Hash: 128, 0, 4, 26, 7, 0 16 Bucket: 76, 0, 21, 29, 64, 0 32 Bucket: 140, 0, 32, 24, 107, 0 64 Bucket: 268, 0, 34, 8, 142, 0 128 Bucket: 524, 0, 628, 9, 45698, 5387 VM OBJECT: 124, 0, 9136, 32404, 20553589, 0 MAP: 140, 0, 7, 49, 7, 0 KMAP ENTRY: 68, 56560, 17, 487, 153898, 0 MAP ENTRY: 68, 0, 15360, 2336, 53481139, 0 DP fakepg: 72, 0, 0, 0, 0, 0 mt_zone: 72, 0, 194, 71, 194, 0 16: 16, 0, 1996, 643, 81641919, 0 32: 32, 0, 2542, 283, 9470304, 0 64: 64, 0, 4584, 7039, 61275174, 0 128: 128, 0, 1101, 549, 6280868, 0 256: 256, 0, 1299, 366, 20697494, 0 512: 512, 0, 145, 279, 79529, 0 1024: 1024, 0, 173, 71, 532899, 0 2048: 2048, 0, 104, 146, 14198, 0 4096: 4096, 0, 266, 63, 1242097, 0 Files: 76, 0, 1595, 655, 16940144, 0 TURNSTILE: 76, 0, 295, 41, 295, 0 umtx pi: 52, 0, 0, 0, 0, 0 PROC: 696, 0, 229, 46, 541535, 0 THREAD: 552, 0, 283, 11, 952, 0 UPCALL: 44, 0, 0, 0, 0, 0 SLEEPQUEUE: 32, 0, 295, 157, 295, 0 VMSPACE: 232, 0, 196, 59, 541501, 0 cpuset: 40, 0, 2, 182, 2, 0 mbuf_packet: 256, 0, 258, 126, 20757971, 0 mbuf: 256, 0, 960618, 33, 116817015, 0 mbuf_cluster: 2048, 25600, 384, 6, 384, 0 mbuf_jumbo_pagesize: 4096, 12800, 0, 117, 19593255, 0 mbuf_jumbo_9k: 9216, 6400, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 3200, 0, 0, 0, 0 mbuf_ext_refcnt: 4, 0, 0, 406, 26807, 0 ACL UMA zone: 388, 0, 0, 20, 65073710, 0 g_bio: 132, 0, 12, 423, 10043465, 0 ata_request: 192, 0, 3, 337, 3212743, 0 ata_composite: 184, 0, 0, 0, 0, 0 cryptop: 60, 0, 0, 0, 0, 0 cryptodesc: 56, 0, 0, 0, 0, 0 VNODE: 276, 0, 19330, 33618, 3235279, 0 VNODEPOLL: 64, 0, 0, 0, 0, 0 NAMEI: 1024, 0, 0, 28, 38128632, 0 S VFS Cache: 68, 0, 20134, 23490, 10825579, 0 L VFS Cache: 291, 0, 27, 207, 175327, 0 NFSMOUNT: 560, 0, 0, 0, 0, 0 NFSNODE: 452, 0, 0, 0, 0, 0 DIRHASH: 1024, 0, 1946, 170, 28110, 0 pipe: 396, 0, 89, 31, 229902, 0 ksiginfo: 80, 0, 248, 40, 248, 0 itimer: 220, 0, 0, 36, 1, 0 KNOTE: 68, 0, 175, 217, 2157098, 0 socket: 416, 25605, 404, 127, 3609347, 0 unpcb: 168, 25622, 278, 136, 1352116, 0 ipq: 32, 904, 0, 226, 42, 0 udpcb: 180, 25608, 13, 75, 1774380, 0 inpcb: 180, 25608, 155, 197, 482839, 0 tcpcb: 464, 25600, 92, 60, 482839, 0 tcptw: 52, 5184, 63, 297, 96034, 0 syncache: 104, 15392, 9, 102, 474336, 0 hostcache: 76, 15400, 626, 124, 4537, 0 tcpreass: 20, 1690, 0, 169, 92838, 0 sackhole: 20, 0, 0, 169, 7789, 0 sctp_ep: 804, 25600, 0, 0, 0, 0 sctp_asoc: 1412, 40000, 0, 0, 0, 0 sctp_laddr: 24, 80040, 0, 145, 28, 0 sctp_raddr: 400, 80000, 0, 0, 0, 0 sctp_chunk: 92, 400008, 0, 0, 0, 0 sctp_readq: 76, 400000, 0, 0, 0, 0 sctp_stream_msg_out: 60, 400050, 0, 0, 0, 0 sctp_asconf: 24, 400055, 0, 0, 0, 0 sctp_asconf_ack: 24, 400055, 0, 0, 0, 0 ripcb: 180, 25608, 1, 43, 2, 0 rtentry: 124, 0, 78, 46, 3549, 0 SWAPMETA: 276, 121576, 158, 108, 19610, 0 Mountpoints: 716, 0, 6, 4, 6, 0 FFS inode: 128, 0, 19299, 19191, 3235170, 0 FFS1 dinode: 128, 0, 0, 0, 0, 0 FFS2 dinode: 256, 0, 19299, 16011, 3235170, 0 pfsrctrpl: 124, 10013, 0, 0, 0, 0 pfrulepl: 828, 0, 12, 8, 12, 0 pfstatepl: 284, 10010, 0, 0, 0, 0 pfaltqpl: 224, 0, 0, 0, 0, 0 pfpooladdrpl: 68, 0, 6, 106, 6, 0 pfrktable: 1240, 1002, 1, 5, 2, 0 pfrkentry: 156, 200000, 3, 47, 3, 0 pfrkentry2: 156, 0, 0, 0, 0, 0 pffrent: 16, 5075, 0, 0, 0, 0 pffrag: 48, 0, 0, 0, 0, 0 pffrcache: 48, 10062, 0, 0, 0, 0 pffrcent: 12, 50141, 0, 0, 0, 0 pfstatescrub: 28, 0, 0, 0, 0, 0 pfiaddrpl: 100, 0, 0, 0, 0, 0 pfospfen: 108, 0, 696, 24, 696, 0 pfosfp: 28, 0, 407, 228, 407, 0 ------=_Part_175155_3729370.1230930167365 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline
Adding a bit more info..  Here's the netstat -m and vmstat -z output after the mbufs have had a while to build up:

-- netstat -m --
959510/235/959745 mbufs in use (current/cache/total)
257/133/390/25600 mbuf clusters in use (current/cache/total/max)
257/127 mbuf+clusters out of packet secondary zone in use (current/cache)
0/117/117/12800 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/6400 9k jumbo clusters in use (current/cache/total/max)
0/0/0/3200 16k jumbo clusters in use (current/cache/total/max)
240391K/792K/241184K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0/152/6656 sfbufs in use (current/peak/max)
0 requests for sfbufs denied
0 requests for sfbufs delayed
2716 requests for I/O initiated by sendfile
0 calls to protocol drain routines

-- vmstat -z --

ITEM                     SIZE     LIMIT      USED      FREE  REQUESTS  FAILURES

UMA Kegs:                 128,        0,       99,       21,       99,        0
UMA Zones:                120,        0,       99,       21,       99,        0
UMA Slabs:               & nbsp; 64,        0,     1132,       48,    15590,        0
UMA RCntSlabs:            104,        0,      312,       21,      856,        0
UMA Hash:                 128,        0,        4,       26,        7,        0
16 Bucket:                 76,   ;      0,       21,       29,       64,        0
32 Bucket:                140,        0,       32,       24,      107,        0
64 Bucket:                268,        0,       34,        8,      142,        0
128 Bucket:               524,   ;      0,      628,        9,    45698,     5387
VM OBJECT:                124,        0,     9136,    32404, 20553589,        0
MAP:                      140,        0,        7,       49,        7,        0
KMAP ENTRY:                68,   ;  56560,       17,      487,   153898,        0
MAP ENTRY:                 68,        0,    15360,     2336, 53481139,        0
DP fakepg:                 72,        0,        0,        0,        0,        0
mt_zone:                   72,   &nb sp;    0,      194,       71,      194,        0
16:                        16,        0,     1996,      643, 81641919,        0
32:                        32,        0,     2542,      283,  9470304,        0
64:                   &nbs p;    64,        0,     4584,     7039, 61275174,        0
128:                      128,        0,     1101,      549,  6280868,        0
256:                      256,        0,     1299,      366, 20697494,        0
512:                       512,        0,      145,      279,    79529,        0
1024:                    1024,        0,      173,       71,   532899,        0
2048:                    2048,        0,      104,      146,    14198,        0
4096:                   &n bsp;4096,        0,      266,       63,  1242097,        0
Files:                     76,        0,     1595,      655, 16940144,        0
TURNSTILE:                 76,        0,      295,       41,      295,        0
umtx pi:                   52,   &nb sp;    0,        0,        0,        0,        0
PROC:                     696,        0,      229,       46,   541535,        0
THREAD:                   552,        0,      283,       11,      952,        0
UPCALL:                    44,        0,        0,        0,        0,        0
SLEEPQUEUE:                32,        0,      295,      157,      295,        0
VMSPACE:                  232,        0,      196,       59,   541501,        0
cpuset:                    40,        0,        2,      182,        2,        0
mbuf_packet:              256,        0,      258,      126, 20757971,        0
mbuf:                     256,        0,   960618,       33, 116817015,        0
mbuf_cluster:            2048,    25600,      384, &nbs p;      6,      384,        0
mbuf_jumbo_pagesize:     4096,    12800,        0,      117, 19593255,        0
mbuf_jumbo_9k:           9216,     6400,        0,        0,        0,        0
mbuf_jumbo_16k:         16384,     3200,        0,        0,   ;      0,        0
mbuf_ext_refcnt:            4,        0,        0,      406,    26807,        0
ACL UMA zone:             388,        0,        0,       20, 65073710,        0
g_bio:                    132,       &nbs p;0,       12,      423, 10043465,        0
ata_request:              192,        0,        3,      337,  3212743,        0
ata_composite:            184,        0,        0,        0,        0,        0
cryptop:                   60,   ;      0,        0,        0,        0,        0
cryptodesc:                56,        0,        0,        0,        0,        0
VNODE:                    276,        0,    19330,    33618,  3235279,        0
VNODEPOLL:                 64,        0,        0,        0,        0,        0
NAMEI:                   1024,        0,        0,       28, 38128632,        0
S VFS Cache:               68,        0,    20134,    23490, 10825579,        0
L VFS Cache:              291,        0,     & nbsp; 27,      207,   175327,        0
NFSMOUNT:                 560,        0,        0,        0,        0,        0
NFSNODE:                  452,        0,        0,        0,        0,        0
DIRHASH:         &nbs p;       1024,        0,     1946,      170,    28110,        0
pipe:                     396,        0,       89,       31,   229902,        0
ksiginfo:                  80,        0,      248,       40,      248,        0
itimer:                   220,        0,        0,       36,        1,        0
KNOTE:                     68,        0,      175,      217,  2157098,        0
socket:                   416,    25605,      404,      127,  3609347,        0
unpcb:                    168,   &nb sp;25622,      278,      136,  1352116,        0
ipq:                       32,      904,        0,      226,       42,        0
udpcb:                    180,    25608,       13,       75,  1774380,        0
inpcb:                   & nbsp;180,    25608,      155,      197,   482839,        0
tcpcb:                    464,    25600,       92,       60,   482839,        0
tcptw:                     52,     5184,       63,      297,    96034,        0
syncache:                 104,    15392,        9,      102,   474336,        0
hostcache:                 76,    15400,      626,      124,     4537,        0
tcpreass:                  20,     1690,        0,      169,    92838,        0
sackhole:                  20,        0,        0,      169,     7789,        0
sctp_ep:                  804,    25600,        0,        0,        0,        0
sctp_asoc:               1412,    40000,        0,        0,        0,        0
sctp_laddr:                 ;24,    80040,        0,      145,       28,        0
sctp_raddr:               400,    80000,        0,        0,        0,        0
sctp_chunk:                92,   400008,        0,        0,        0,        0
sctp_readq:                76,   ; 400000,        0,        0,        0,        0
sctp_stream_msg_out:       60,   400050,        0,        0,        0,        0
sctp_asconf:               24,   400055,        0,        0,        0,        0
sctp_asconf_ack:           24,   400055,        0,   ;      0,        0,        0
ripcb:                    180,    25608,        1,       43,        2,        0
rtentry:                  124,        0,       78,       46,     3549,        0
SWAPMETA:                 276,   121576,      158,      108,    19610,        0
Mountpoints:              716,        0,        6,        4,        6,        0
FFS inode:                128,        0,    19299,    19191,  3235170,        0
FFS1 dinode:              128,        0,        0,        0,        0,        0
FFS2 dinode:              256,        0,    19299,    16011,  3235170,        0
pfsrctrpl:                124,    10013,        0,        0,        0,        0
pfrulepl:                 828,       &nbs p;0,       12,        8,       12,        0
pfstatepl:                284,    10010,        0,        0,        0,        0
pfaltqpl:                 224,        0,        0,        0,        0,        0
pfpooladdrpl:              68,        0,        6,      106,        6,        0
pfrktable:               1240,     1002,        1,        5,        2,        0
pfrkentry:                156,   200000,        3,       47,        3,        0
pfrkentry2:               156,     & nbsp;  0,        0,        0,        0,        0
pffrent:                   16,     5075,        0,        0,        0,        0
pffrag:                    48,        0,        0,        0,        0,        0
pffrcache:         &n bsp;       48,    10062,        0,        0,        0,        0
pffrcent:                  12,    50141,        0,        0,        0,        0
pfstatescrub:              28,        0,        0,        0,        0,        0
pfiaddrpl:             &nb sp;  100,        0,        0,        0,        0,        0
pfospfen:                 108,        0,      696,       24,      696,        0
pfosfp:                    28,        0,      407,      228,      407,        0



------=_Part_175155_3729370.1230930167365-- From owner-freebsd-net@FreeBSD.ORG Fri Jan 2 22:15:07 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E6BD01065670; Fri, 2 Jan 2009 22:15:07 +0000 (UTC) (envelope-from qing.li@bluecoat.com) Received: from whisker.bluecoat.com (whisker.bluecoat.com [216.52.23.28]) by mx1.freebsd.org (Postfix) with ESMTP id C34758FC1A; Fri, 2 Jan 2009 22:15:07 +0000 (UTC) (envelope-from qing.li@bluecoat.com) Received: from bcs-mail03.internal.cacheflow.com ([10.2.2.95]) by whisker.bluecoat.com (8.14.2/8.14.2) with ESMTP id n02MF6v1004651; Fri, 2 Jan 2009 14:15:07 -0800 (PST) X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Date: Fri, 2 Jan 2009 14:15:44 -0800 Message-ID: In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: SCTP related issue with recent ARP changes? Thread-Index: AclsMle5U2WQGWG/SKSzlWZ3/NiGZQAG4G76ADYDKbA= References: From: "Li, Qing" To: =?iso-8859-1?Q?Michael_T=FCxen?= Cc: qingli@freebsd.org, current@freebsd.org, FreeBSD Net Subject: RE: SCTP related issue with recent ARP changes? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jan 2009 22:15:08 -0000 Hi Michael, The SCTP problem you were having was attributed to the arp-v2 changes. The reason TCP and UDP works well is because the TCP and UDP modules does not supply a route entry when calling ip_output(). Consequently in ip_output() the destination (sockaddr) was reinitialized = and the sin_port field is 0. The destination sockaddr supplied in the route entry from the SCTP module has the actual port value in it. The new L2 search was (initially) written as a generic function. So the L3 address comparison was performed using bcmp() of sockaddr_len bytes. The L2 entry was created with a sockaddr object that includes the port value, however, on ARP input the port value does not apply. The mismatch in port value causes the ARP lookup to fail and the SCTP connection never complete its negotiation. Please apply the patch for IPv4 in my home directory at http://people.freebsd.org/~qingli/in.c.diff I did the basic testing using the programs you gave me and now the client connects successful and completes the transfer. I also verified the packets in wireshark trace. Please let me know if this patch fixes your SCTP problems. Thank you for your help. -- Qing > -----Original Message----- > From: owner-freebsd-current@freebsd.org [mailto:owner-freebsd- > current@freebsd.org] On Behalf Of Li, Qing > Sent: Thursday, January 01, 2009 12:17 PM > To: Michael T=FCxen; FreeBSD Net > Cc: qingli@freebsd.org; current@freebsd.org > Subject: RE: SCTP related issue with recent ARP changes? >=20 >=20 > Hi Michael, >=20 > Your problem could be related to the recent ARP changes. > I will investigate further to confirm. >=20 > Thanks, >=20 > -- Qing >=20 >=20 > -----Original Message----- > From: owner-freebsd-net@freebsd.org on behalf of Michael T=FCxen > Sent: Thu 1/1/2009 8:59 AM > To: FreeBSD Net > Subject: SCTP related issue with recent ARP changes? >=20 > Dear all, >=20 > I'm running the current CVS version of FreeBSD 8 in a virtual > machine using VMWare 2.0.1 on a Mac (not sure if this is relevant) > and bridged networking having an em interface on the virtual machine. > I'm using a similar setup with older FreeBSD machines and they are > running fine. >=20 > Loopback communication based on UDP, TCP, SCTP or ICMP works fine. >=20 > Communicating with other hosts using UDP, TCP or ICMP works fine. >=20 > Communicating with other hosts using SCTP does not work. > The SCTP stack calls ip_output() and an ARP request for the > correct destination IP address is send out. A corresponding > ARP reply is visible in Wireshark (running on the FreeBSD 8 box) > and looks good. However, the corresponding SCTP packet it never > sent out. I'm not sure, but this might be related to the > recent ARP changes. Is there anything required from the > transport layer to be done when calling ip_output() that > was not required before? >=20 > Best regards > Michael >=20 From owner-freebsd-net@FreeBSD.ORG Fri Jan 2 23:20:18 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 422EF1065672; Fri, 2 Jan 2009 23:20:18 +0000 (UTC) (envelope-from Michael.Tuexen@lurchi.franken.de) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) by mx1.freebsd.org (Postfix) with ESMTP id 823CF8FC1C; Fri, 2 Jan 2009 23:20:17 +0000 (UTC) (envelope-from Michael.Tuexen@lurchi.franken.de) Received: from [IPv6:2002:508f:ceb2::21e:c2ff:fe00:76c8] (unknown [IPv6:2002:508f:ceb2:0:21e:c2ff:fe00:76c8]) by mail-n.franken.de (Postfix) with ESMTP id A2E161C0B461A; Sat, 3 Jan 2009 00:20:15 +0100 (CET) Message-Id: From: =?ISO-8859-1?Q?Michael_T=FCxen?= To: "Li, Qing" In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed; delsp=yes Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Apple Message framework v930.3) Date: Sat, 3 Jan 2009 00:20:14 +0100 References: X-Mailer: Apple Mail (2.930.3) Cc: qingli@freebsd.org, Randall Stewart , current@freebsd.org, FreeBSD Net Subject: Re: SCTP related issue with recent ARP changes? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jan 2009 23:20:18 -0000 Hi Qing, I have tested your patch and it works! Thanks you very much for your help. Best regards Michael On Jan 2, 2009, at 11:15 PM, Li, Qing wrote: > Hi Michael, > > The SCTP problem you were having was attributed to the arp-v2 changes. > The reason TCP and UDP works well is because the TCP and UDP > modules does not supply a route entry when calling ip_output(). > Consequently in ip_output() the destination (sockaddr) was =20 > reinitialized > and the sin_port field is 0. > > The destination sockaddr supplied in the route entry from the SCTP > module has the actual port value in it. > > The new L2 search was (initially) written as a generic function. > So the L3 address comparison was performed using bcmp() of =20 > sockaddr_len > bytes. The L2 entry was created with a sockaddr object that includes > the port value, however, on ARP input the port value does not > apply. The mismatch in port value causes the ARP lookup to fail > and the SCTP connection never complete its negotiation. > > Please apply the patch for IPv4 in my home directory at > http://people.freebsd.org/~qingli/in.c.diff > > I did the basic testing using the programs you gave me and now > the client connects successful and completes the transfer. I also > verified the packets in wireshark trace. > > Please let me know if this patch fixes your SCTP problems. > > Thank you for your help. > > -- Qing > > >> -----Original Message----- >> From: owner-freebsd-current@freebsd.org [mailto:owner-freebsd- >> current@freebsd.org] On Behalf Of Li, Qing >> Sent: Thursday, January 01, 2009 12:17 PM >> To: Michael T=FCxen; FreeBSD Net >> Cc: qingli@freebsd.org; current@freebsd.org >> Subject: RE: SCTP related issue with recent ARP changes? >> >> >> Hi Michael, >> >> Your problem could be related to the recent ARP changes. >> I will investigate further to confirm. >> >> Thanks, >> >> -- Qing >> >> >> -----Original Message----- >> From: owner-freebsd-net@freebsd.org on behalf of Michael T=FCxen >> Sent: Thu 1/1/2009 8:59 AM >> To: FreeBSD Net >> Subject: SCTP related issue with recent ARP changes? >> >> Dear all, >> >> I'm running the current CVS version of FreeBSD 8 in a virtual >> machine using VMWare 2.0.1 on a Mac (not sure if this is relevant) >> and bridged networking having an em interface on the virtual machine. >> I'm using a similar setup with older FreeBSD machines and they are >> running fine. >> >> Loopback communication based on UDP, TCP, SCTP or ICMP works fine. >> >> Communicating with other hosts using UDP, TCP or ICMP works fine. >> >> Communicating with other hosts using SCTP does not work. >> The SCTP stack calls ip_output() and an ARP request for the >> correct destination IP address is send out. A corresponding >> ARP reply is visible in Wireshark (running on the FreeBSD 8 box) >> and looks good. However, the corresponding SCTP packet it never >> sent out. I'm not sure, but this might be related to the >> recent ARP changes. Is there anything required from the >> transport layer to be done when calling ip_output() that >> was not required before? >> >> Best regards >> Michael >> > > > From owner-freebsd-net@FreeBSD.ORG Fri Jan 2 23:27:12 2009 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0489F1065674 for ; Fri, 2 Jan 2009 23:27:12 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mxout3.cac.washington.edu (mxout3.cac.washington.edu [140.142.32.166]) by mx1.freebsd.org (Postfix) with ESMTP id CFFDA8FC18 for ; Fri, 2 Jan 2009 23:27:11 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from smtp.washington.edu (smtp.washington.edu [140.142.33.7] (may be forged)) by mxout3.cac.washington.edu (8.14.3+UW08.09/8.14.3+UW08.11) with ESMTP id n02NRBOq016515 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 2 Jan 2009 15:27:11 -0800 X-Auth-Received: from [192.168.10.3] (adsl-76-254-6-149.dsl.pltn13.sbcglobal.net [76.254.6.149]) (authenticated authid=youshi10) by smtp.washington.edu (8.14.3+UW08.09/8.14.3+UW08.11) with ESMTP id n02NR9sb024576 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Fri, 2 Jan 2009 15:27:11 -0800 Message-Id: From: Garrett Cooper To: pyunyh@gmail.com In-Reply-To: <20081224021016.GF95088@cdnetworks.co.kr> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Fri, 2 Jan 2009 15:31:33 -0800 References: <20081224021016.GF95088@cdnetworks.co.kr> X-Mailer: Apple Mail (2.930.3) X-PMX-Version: 5.5.0.356843, Antispam-Engine: 2.6.1.350677, Antispam-Data: 2009.1.2.231614 X-Uwash-Spam: Gauge=IIIIIII, Probability=8%, Report='FORGED_FROM_GMAIL 0.1, BODY_SIZE_1000_LESS 0, BODY_SIZE_5000_LESS 0, BODY_SIZE_600_699 0, __CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __FRAUD_419_WEBMAIL 0, __FRAUD_419_WEBMAIL_FROM 0, __FROM_GMAIL 0, __HAS_MSGID 0, __HAS_X_MAILER 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __SANE_MSGID 0' Cc: net@freebsd.org Subject: Annoyance with msk(4) going up and down when initializing interface X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jan 2009 23:27:12 -0000 Hi Pyun, I've noticed an issue for a while now with my chipset (I think that this is post an MFC between 7.0 and 7.1, but I could be wrong). Basically, each CPU (with the ULE scheduler) grabs the task to check for media status, goes out and attempts to get an IP, and if the timing of the status modifications is just right, one of the CPU's will mark the link up and others will mark it down, and it will stay down. Same thing occurs when trying to get a DHCP request -- there will typically be multiple requests and ACK's for any given requests. This occurs with my onboard NICs on my P5K-e motherboards on 7.1- rc[12], and also 8-CURRENT. Thanks! -Garrett From owner-freebsd-net@FreeBSD.ORG Sat Jan 3 04:08:57 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 81399106564A for ; Sat, 3 Jan 2009 04:08:57 +0000 (UTC) (envelope-from perryh@pluto.rain.com) Received: from agora.rdrop.com (agora.rdrop.com [199.26.172.34]) by mx1.freebsd.org (Postfix) with ESMTP id 677058FC19 for ; Sat, 3 Jan 2009 04:08:57 +0000 (UTC) (envelope-from perryh@pluto.rain.com) Received: from agora.rdrop.com (66@localhost [127.0.0.1]) by agora.rdrop.com (8.13.1/8.12.7) with ESMTP id n033arD2062773 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Fri, 2 Jan 2009 19:36:53 -0800 (PST) (envelope-from perryh@pluto.rain.com) Received: (from uucp@localhost) by agora.rdrop.com (8.13.1/8.12.9/Submit) with UUCP id n033arce062769 for freebsd-net@freebsd.org; Fri, 2 Jan 2009 19:36:53 -0800 (PST) Received: from fbsd61 by pluto.rain.com (4.1/SMI-4.1-pluto-M2060407) id AA03745; Fri, 2 Jan 09 19:31:04 PST Date: Fri, 02 Jan 2009 19:33:31 -0800 From: perryh@pluto.rain.com To: freebsd-net@freebsd.org Message-Id: <495edc8b.yfwTDGtb9G/8NMur%perryh@pluto.rain.com> User-Agent: nail 11.25 7/29/05 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: tun0 not responding to ping X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Jan 2009 04:08:57 -0000 Why would a local interface, reported as up in ifconfig, not respond to a ping of its own IP address? The tun0 reported below doesn't, and I have no idea how to debug it. (I've overwritten the two most- significant octets of its IP address, which is Class B, so as not to publicly identify the network to which I am attempting to connect.) $ ifconfig -a xl0: flags=8843 mtu 1500 options=9 inet6 fe80::2b0:d0ff:fe28:ad4f%xl0 prefixlen 64 scopeid 0x1 inet 192.168.200.61 netmask 0xffffff00 broadcast 192.168.200.255 ether 00:b0:d0:28:ad:4f media: Ethernet autoselect (10baseT/UTP) status: active plip0: flags=108810 mtu 1500 lo0: flags=8049 mtu 16384 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 tun0: flags=8051 mtu 1412 inet6 fe80::2b0:d0ff:fe28:ad4f%tun0 prefixlen 64 scopeid 0x4 inet ZZZ.ZZZ.233.42 --> ZZZ.ZZZ.233.42 netmask 0xffffffff Opened by PID 24635 $ ping ZZZ.ZZZ.233.42 PING ZZZ.ZZZ.233.42 (ZZZ.ZZZ.233.42): 56 data bytes ^C --- ZZZ.ZZZ.233.42 ping statistics --- 4 packets transmitted, 0 packets received, 100% packet loss $ traceroute -n ZZZ.ZZZ.233.42 traceroute to ZZZ.ZZZ.233.42 (ZZZ.ZZZ.233.42), 64 hops max, 40 byte packets 1 * * * 2 * * * 3 * * * 4 * * * 5 * * * ^C $ netstat -r -n Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default 192.168.200.254 UGS 0 2209723 xl0 127.0.0.1 127.0.0.1 UH 0 2 lo0 ZZZ.ZZZ ZZZ.ZZZ.233.42 UGS 0 0 tun0 ZZZ.ZZZ.57.128/32 ZZZ.ZZZ.233.42 UGS 0 19 tun0 ZZZ.ZZZ.57.133/32 ZZZ.ZZZ.233.42 UGS 0 18 tun0 ZZZ.ZZZ.233.42 ZZZ.ZZZ.233.42 UH 59 20 tun0 YYY.YYY.127/27 ZZZ.ZZZ.233.42 UGS 0 0 tun0 YYY.YYY.127.96/27 ZZZ.ZZZ.233.42 UGS 0 0 tun0 YYY.YYY.127.224/27 ZZZ.ZZZ.233.42 UGS 0 0 tun0 YYY.YYY.127.228 192.168.200.254 UGHS 0 106 xl0 192.168.200 link#1 UC 0 0 xl0 192.168.200.254 00:09:5b:a1:c8:9e UHLW 3 4318078 xl0 1148 192.168.200.255 ff:ff:ff:ff:ff:ff UHLWb 1 3 xl0 In case it matters the tunnel was set up by the security/vpnc port, however my focus in this inquiry is on the expected behavior of the tun(4) driver, which is part of the base and therefore (I hope) on-topic here. Inquiries regarding vpnc on ports@ and questions@ have not yielded a solution. For background, the network geometry involved is: 192.168.200.0/8 +---------------------+---------------------+ | | | 192.168.200.254 192.168.200.95 192.168.200.61[xl0] +-----------------+ +-----------------+ +-----------------+ | Netgear router | | Windows | | FreeBSD | | | | Cisco VPN client| | vpnc | +-----------------+ +-----------------+ +-----------------+ 71.111.59.66 ZZZ.ZZZ.233.6 ZZZ.ZZZ.233.42[tun0] | | | +-----------------+ | DSL modem | | | +-----------------+ [works] [does not work] | | | -- Internet -- | | | YYY.YYY.127.228 +-----------------+ | | | Cisco 3000 | < - - - - - - - - - - - - - - - -+ +-----------------+ ZZZ.ZZZ.2.13 | ZZZ.ZZZ.0.0/16 +---------------------+---------------------+ (and no, I am not trying to have both the Windows client and the FreeBSD client connected at the same time, although that ought to work -- the Cisco will supposedly allow up to 4 concurrent connections from the same LAN using the same credentials). I think the tunnel's data flow should be something like: +---------------+---------------+ | application (ping, ssh, etc.) | | to (say) ZZZ.ZZZ.11.220 | +---------------+---------------+ ^ [socket] v +----------+----------+ | kernel TCP/IP stack | +----------+----------+ ^ [routing] v +--------+--------+ | tun0 interface | <-- ping packets to ZZZ.ZZZ.233.42 looped | ZZZ.ZZZ.233.42 | <-- back here, all else passed to vpnc +--------+--------+ ^ | v +--------------+--------------+ | userspace vpnc daemon | | [encapsulation/encryption] | | to YYY.YYY.127.228 | +--------------+--------------+ ^ [socket] v +----------+----------+ | kernel TCP/IP stack | +----------+----------+ ^ [routing] v +-------+-------+ | xl0 interface | | 192.168.200.61| +-------+-------+ ^ 192.168.200.0/8 v +--------+--------+ | 192.168.200.254 | | Netgear router | | 71.111.59.66 | +--------+--------+ ^ DSL/Internet v +--------------+--------------+ | YYY.YYY.127.228 | | Cisco 3000 | | [decapsulation/decryption] | | ZZZ.ZZZ.2.13 | +--------------+--------------+ ^ ZZZ.ZZZ.0.0/16 v ZZZ.ZZZ.11.220 From owner-freebsd-net@FreeBSD.ORG Sat Jan 3 04:55:46 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 056C7106564A for ; Sat, 3 Jan 2009 04:55:46 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [220.233.188.227]) by mx1.freebsd.org (Postfix) with ESMTP id 2A5848FC17 for ; Sat, 3 Jan 2009 04:55:44 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id n034tgHF065728; Sat, 3 Jan 2009 15:55:43 +1100 (EST) (envelope-from smithi@nimnet.asn.au) Date: Sat, 3 Jan 2009 15:55:42 +1100 (EST) From: Ian Smith To: perryh@pluto.rain.com In-Reply-To: <495edc8b.yfwTDGtb9G/8NMur%perryh@pluto.rain.com> Message-ID: <20090103154232.P28770@sola.nimnet.asn.au> References: <495edc8b.yfwTDGtb9G/8NMur%perryh@pluto.rain.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-net@freebsd.org Subject: Re: tun0 not responding to ping X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Jan 2009 04:55:46 -0000 On Fri, 2 Jan 2009, perryh@pluto.rain.com wrote: > Why would a local interface, reported as up in ifconfig, not respond > to a ping of its own IP address? The tun0 reported below doesn't, > and I have no idea how to debug it. (I've overwritten the two most- > significant octets of its IP address, which is Class B, so as not to > publicly identify the network to which I am attempting to connect.) > > $ ifconfig -a > xl0: flags=8843 mtu 1500 > options=9 > inet6 fe80::2b0:d0ff:fe28:ad4f%xl0 prefixlen 64 scopeid 0x1 > inet 192.168.200.61 netmask 0xffffff00 broadcast 192.168.200.255 > ether 00:b0:d0:28:ad:4f > media: Ethernet autoselect (10baseT/UTP) > status: active > plip0: flags=108810 mtu 1500 > lo0: flags=8049 mtu 16384 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 > inet 127.0.0.1 netmask 0xff000000 > tun0: flags=8051 mtu 1412 > inet6 fe80::2b0:d0ff:fe28:ad4f%tun0 prefixlen 64 scopeid 0x4 > inet ZZZ.ZZZ.233.42 --> ZZZ.ZZZ.233.42 netmask 0xffffffff > Opened by PID 24635 I don't know if this is relevant or not, but I've never seen a point to point interface use the same IP address on both ends of its link before. I noticed this before when you posed this on questions@ but not knowing anything about vpnc and very little about VPNs in general, I let it go. > $ ping ZZZ.ZZZ.233.42 > PING ZZZ.ZZZ.233.42 (ZZZ.ZZZ.233.42): 56 data bytes > ^C > --- ZZZ.ZZZ.233.42 ping statistics --- > 4 packets transmitted, 0 packets received, 100% packet loss > > $ traceroute -n ZZZ.ZZZ.233.42 > traceroute to ZZZ.ZZZ.233.42 (ZZZ.ZZZ.233.42), 64 hops max, 40 byte packets > 1 * * * > 2 * * * > 3 * * * > 4 * * * > 5 * * * > ^C > > $ netstat -r -n > Routing tables > > Internet: > Destination Gateway Flags Refs Use Netif Expire > default 192.168.200.254 UGS 0 2209723 xl0 > 127.0.0.1 127.0.0.1 UH 0 2 lo0 > ZZZ.ZZZ ZZZ.ZZZ.233.42 UGS 0 0 tun0 > ZZZ.ZZZ.57.128/32 ZZZ.ZZZ.233.42 UGS 0 19 tun0 > ZZZ.ZZZ.57.133/32 ZZZ.ZZZ.233.42 UGS 0 18 tun0 > ZZZ.ZZZ.233.42 ZZZ.ZZZ.233.42 UH 59 20 tun0 There it is again. Looks circular to me, but I may be way off base .. Nice diagrams :) cheers, Ian > YYY.YYY.127/27 ZZZ.ZZZ.233.42 UGS 0 0 tun0 > YYY.YYY.127.96/27 ZZZ.ZZZ.233.42 UGS 0 0 tun0 > YYY.YYY.127.224/27 ZZZ.ZZZ.233.42 UGS 0 0 tun0 > YYY.YYY.127.228 192.168.200.254 UGHS 0 106 xl0 > 192.168.200 link#1 UC 0 0 xl0 > 192.168.200.254 00:09:5b:a1:c8:9e UHLW 3 4318078 xl0 1148 > 192.168.200.255 ff:ff:ff:ff:ff:ff UHLWb 1 3 xl0 > > In case it matters the tunnel was set up by the security/vpnc port, > however my focus in this inquiry is on the expected behavior of the > tun(4) driver, which is part of the base and therefore (I hope) > on-topic here. Inquiries regarding vpnc on ports@ and questions@ > have not yielded a solution. > > For background, the network geometry involved is: > > 192.168.200.0/8 > +---------------------+---------------------+ > | | | > 192.168.200.254 192.168.200.95 192.168.200.61[xl0] > +-----------------+ +-----------------+ +-----------------+ > | Netgear router | | Windows | | FreeBSD | > | | | Cisco VPN client| | vpnc | > +-----------------+ +-----------------+ +-----------------+ > 71.111.59.66 ZZZ.ZZZ.233.6 ZZZ.ZZZ.233.42[tun0] > | | | > +-----------------+ > | DSL modem | | | > +-----------------+ [works] [does not work] > | | | > -- Internet -- > | | | > YYY.YYY.127.228 > +-----------------+ | | > | Cisco 3000 | < - - - - - - - - - - - - - - - -+ > +-----------------+ > ZZZ.ZZZ.2.13 > | ZZZ.ZZZ.0.0/16 > +---------------------+---------------------+ > > (and no, I am not trying to have both the Windows client and the > FreeBSD client connected at the same time, although that ought > to work -- the Cisco will supposedly allow up to 4 concurrent > connections from the same LAN using the same credentials). > > I think the tunnel's data flow should be something like: > > +---------------+---------------+ > | application (ping, ssh, etc.) | > | to (say) ZZZ.ZZZ.11.220 | > +---------------+---------------+ > ^ > [socket] > v > +----------+----------+ > | kernel TCP/IP stack | > +----------+----------+ > ^ > [routing] > v > +--------+--------+ > | tun0 interface | <-- ping packets to ZZZ.ZZZ.233.42 looped > | ZZZ.ZZZ.233.42 | <-- back here, all else passed to vpnc > +--------+--------+ > ^ > | > v > +--------------+--------------+ > | userspace vpnc daemon | > | [encapsulation/encryption] | > | to YYY.YYY.127.228 | > +--------------+--------------+ > ^ > [socket] > v > +----------+----------+ > | kernel TCP/IP stack | > +----------+----------+ > ^ > [routing] > v > +-------+-------+ > | xl0 interface | > | 192.168.200.61| > +-------+-------+ > ^ > 192.168.200.0/8 > v > +--------+--------+ > | 192.168.200.254 | > | Netgear router | > | 71.111.59.66 | > +--------+--------+ > ^ > DSL/Internet > v > +--------------+--------------+ > | YYY.YYY.127.228 | > | Cisco 3000 | > | [decapsulation/decryption] | > | ZZZ.ZZZ.2.13 | > +--------------+--------------+ > ^ > ZZZ.ZZZ.0.0/16 > v > ZZZ.ZZZ.11.220 From owner-freebsd-net@FreeBSD.ORG Sat Jan 3 07:37:01 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E182F106566B for ; Sat, 3 Jan 2009 07:37:01 +0000 (UTC) (envelope-from perryh@pluto.rain.com) Received: from agora.rdrop.com (agora.rdrop.com [199.26.172.34]) by mx1.freebsd.org (Postfix) with ESMTP id C08678FC14 for ; Sat, 3 Jan 2009 07:37:01 +0000 (UTC) (envelope-from perryh@pluto.rain.com) Received: from agora.rdrop.com (66@localhost [127.0.0.1]) by agora.rdrop.com (8.13.1/8.12.7) with ESMTP id n037b1Ln018126 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Fri, 2 Jan 2009 23:37:01 -0800 (PST) (envelope-from perryh@pluto.rain.com) Received: (from uucp@localhost) by agora.rdrop.com (8.13.1/8.12.9/Submit) with UUCP id n037b1hg018125; Fri, 2 Jan 2009 23:37:01 -0800 (PST) Received: from fbsd61 by pluto.rain.com (4.1/SMI-4.1-pluto-M2060407) id AA04251; Fri, 2 Jan 09 23:35:35 PST Date: Fri, 02 Jan 2009 23:38:02 -0800 From: perryh@pluto.rain.com To: smithi@nimnet.asn.au Message-Id: <495f15da.kLIW2g4L+3rMjCXS%perryh@pluto.rain.com> References: <495edc8b.yfwTDGtb9G/8NMur%perryh@pluto.rain.com> <20090103154232.P28770@sola.nimnet.asn.au> In-Reply-To: <20090103154232.P28770@sola.nimnet.asn.au> User-Agent: nail 11.25 7/29/05 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org Subject: Re: tun0 not responding to ping X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Jan 2009 07:37:02 -0000 Ian Smith wrote: ... > > tun0: flags=8051 mtu 1412 > > inet6 fe80::2b0:d0ff:fe28:ad4f%tun0 prefixlen 64 scopeid 0x4 > > inet ZZZ.ZZZ.233.42 --> ZZZ.ZZZ.233.42 netmask 0xffffffff > > Opened by PID 24635 > > I don't know if this is relevant or not, but I've never seen > a point to point interface use the same IP address on both ends > of its link before. I don't know either, nor whether -- and if so how -- it could keep tun0 from responding to a ping of its own IP address. It looks like the same issue described, for a different way of connecting to a Cisco 3000 from FreeBSD, here: http://www.cs.rpi.edu/~flemej/fbsd-cisco-vpn.pdf If I am understanding the article correctly, the 3000 does something unexpected in the course of setting up the P2P connection. However: * Since the FreeBSD config is completely different, I don't know to what extent the w/a described there would be applicable. * Supposing that tun0 does need to be readdressed as inet ZZZ.ZZZ.233.42 --> ZZZ.ZZZ.2.13 netmask 0xffffffff -- where ZZZ.ZZZ.2.13 is the address of the Cisco box on ZZZ.ZZZ.0.0/16 -- I'm not at all clear on how a w/a should get that internal address in the general case. (I got it by running a traceroute from an inside machine to a working VPN-connected Windows system, after not finding anything in the vpnc logs.) * Since vpnc is supposed to have been written specifically to connect with Cisco 3000's and similar, I'd have expected it to somehow take care of the 3000's quirks rather than needing a separate w/a, although I don't know enough about either tun(4) or P2P to understand the details. From owner-freebsd-net@FreeBSD.ORG Sat Jan 3 09:02:30 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 76E4E106566C for ; Sat, 3 Jan 2009 09:02:30 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [220.233.188.227]) by mx1.freebsd.org (Postfix) with ESMTP id D020D8FC18 for ; Sat, 3 Jan 2009 09:02:29 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id n0392OGF073483; Sat, 3 Jan 2009 20:02:25 +1100 (EST) (envelope-from smithi@nimnet.asn.au) Date: Sat, 3 Jan 2009 20:02:24 +1100 (EST) From: Ian Smith To: perryh@pluto.rain.com In-Reply-To: <495f15da.kLIW2g4L+3rMjCXS%perryh@pluto.rain.com> Message-ID: <20090103185837.K28770@sola.nimnet.asn.au> References: <495edc8b.yfwTDGtb9G/8NMur%perryh@pluto.rain.com> <20090103154232.P28770@sola.nimnet.asn.au> <495f15da.kLIW2g4L+3rMjCXS%perryh@pluto.rain.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-net@freebsd.org Subject: Re: tun0 not responding to ping X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Jan 2009 09:02:30 -0000 On Fri, 2 Jan 2009, perryh@pluto.rain.com wrote: > Ian Smith wrote: uucp .. how quaint :) > ... > > > tun0: flags=8051 mtu 1412 > > > inet6 fe80::2b0:d0ff:fe28:ad4f%tun0 prefixlen 64 scopeid 0x4 > > > inet ZZZ.ZZZ.233.42 --> ZZZ.ZZZ.233.42 netmask 0xffffffff > > > Opened by PID 24635 > > > > I don't know if this is relevant or not, but I've never seen > > a point to point interface use the same IP address on both ends > > of its link before. > > I don't know either, nor whether -- and if so how -- it could keep > tun0 from responding to a ping of its own IP address. It looks like > the same issue described, for a different way of connecting to a > Cisco 3000 from FreeBSD, here: > > http://www.cs.rpi.edu/~flemej/fbsd-cisco-vpn.pdf "You don't have permission to access /~flemej/fbsd-cisco-vpn.pdf on this server." Nor http://www.cs.rpi.edu/~flemej .. so I can't consult that, but as I said, I know next to nothing about VPN configuration anyway. > If I am understanding the article correctly, the 3000 does something > unexpected in the course of setting up the P2P connection. However: > > * Since the FreeBSD config is completely different, I don't know > to what extent the w/a described there would be applicable. > > * Supposing that tun0 does need to be readdressed as > > inet ZZZ.ZZZ.233.42 --> ZZZ.ZZZ.2.13 netmask 0xffffffff > > -- where ZZZ.ZZZ.2.13 is the address of the Cisco box on > ZZZ.ZZZ.0.0/16 -- I'm not at all clear on how a w/a should get > that internal address in the general case. (I got it by running > a traceroute from an inside machine to a working VPN-connected > Windows system, after not finding anything in the vpnc logs.) Beyond me .. I don't even know what a w/a is, but I'm pretty sure ppp is going to need a remote address, and a route to it. > * Since vpnc is supposed to have been written specifically to > connect with Cisco 3000's and similar, I'd have expected it to > somehow take care of the 3000's quirks rather than needing a > separate w/a, although I don't know enough about either tun(4) > or P2P to understand the details. Usually you can ping either end; ping is the same as ping localhost, ping is, well, that. With both the same, I'm not too surprised that ppp can't figure out which end you want to talk to? We ran ppp for 10 years on a dialup link but these days for pppoe using mpd, but the routing should come to about the same, given that here it's our default route. ng0: flags=88d1 mtu 1492 inet xxx.yyy.zzz.227 --> xxx.yyy.1.33 netmask 0xffffffff Destination Gateway Flags Refs Use Netif Expire default xxx.yyy.1.33 UGS 0 24390 ng0 [..] xxx.yyy.1.33 xxx.yyy.zzz.227 UH 1 0 ng0 xxx.yyy.zzz.227/32 lo0 US 0 2 lo0 This is a 5.5 system, in case different presentation might mislead. HTH, Ian From owner-freebsd-net@FreeBSD.ORG Sat Jan 3 11:15:41 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6DA08106566B for ; Sat, 3 Jan 2009 11:15:41 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 422E68FC13 for ; Sat, 3 Jan 2009 11:15:41 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id C0D5E46B03; Sat, 3 Jan 2009 06:15:40 -0500 (EST) Date: Sat, 3 Jan 2009 11:15:40 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Yony Yossef In-Reply-To: <000001c96a78$9a5b3c20$220f000a@mtl.com> Message-ID: References: <20def4870812240600n6edbcad7k2100a0ccbe49f0dd@mail.gmail.com> <000001c96a78$9a5b3c20$220f000a@mtl.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Eitan Shefi , freebsd-net@freebsd.org, Liran Liss , Yevgeny Petrilin Subject: RE: net.inet.udp.blackhole issue X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Jan 2009 11:15:41 -0000 On Tue, 30 Dec 2008, Yony Yossef wrote: >>> I'm facing lots of UDP "Connection refused" errors while running >>> multistream iperf test. Analyzing it with wireshark showed several "ICMP >>> Port Unreachable" problems. >>> >>> I've overriden it with "sysctl net.inet.udp.blackhole=1", >> but I'm not >>> sure this is the correct thing to do, I feel like I've sweeped the problem >>> under the carpet. >>> >>> PS - I see similar failures with TCP bidirectional iperf >> test, it can >>> also be overriden by "sysctl net.inet.tcp.blackhole=1". >>> >>> My question is - can it be a NIC problem? If so, how can a driver problem >>> cause an iperf UDP socket to be in a "non listening state"? >> >> This is fairly unlikely to be a NIC problem, although anything is possible >> in software. I'm not familiar with iperf, but generally speaking ICMP port >> unreachable is a result of packets arriving at a closed socket; >> net.inet.udp.blackhole suppresses that ICMP: >> >> if (udp_blackhole) >> goto badheadlocked; >> if (badport_bandlim(BANDLIM_ICMP_UNREACH) < 0) >> goto badheadlocked; >> *ip = save_ip; >> ip->ip_len += iphlen; >> icmp_error(m, ICMP_UNREACH, ICMP_UNREACH_PORT, 0, 0); >> >> I think I'd suspect an application bug/feature, in which socket gets closed >> and opened during execution and once in a while a datagram is delivered in >> that window. Perhaps packets are being delivered with a non-trivial delay >> causing them to arrive after the application has timed out waiting for it? > > I'm talking about a simple multistream UDP iperf test. One stream always > works fine. More than one UDP stream has a chance of failing because of this > problem. Wireshark analysis shows no such delay and no packet loss nor > corruption, for what I've seen and understood. On the other hand, same test > on a 1Gig NIC (I'm using a 10Gig) doesn't suffer from this issue without the > blackhole assistance. Hmm. These results are confusing, given the code. Would it be possible for you to provide a packet trace that includes a few UDP packets and the ICMP errors they resulted in? While I can't preclude a network stack bug, related parts of the UDP code are relatively straight forward and well-exercised, which generally suggests an application-layer bug or interaction. If the packets are becoming corrupted during driver processing, then the ICMP rejections should show the packet header the stack saw leading to their rejection. Also, could you provide sockstat output for the application during its run? thanks, Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-net@FreeBSD.ORG Sat Jan 3 12:33:32 2009 Return-Path: Delivered-To: net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DD295106564A; Sat, 3 Jan 2009 12:33:32 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id B4E7B8FC1A; Sat, 3 Jan 2009 12:33:32 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from freefall.freebsd.org (rwatson@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n03CXWvB031834; Sat, 3 Jan 2009 12:33:32 GMT (envelope-from rwatson@freefall.freebsd.org) Received: (from rwatson@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n03CXWAc031830; Sat, 3 Jan 2009 12:33:32 GMT (envelope-from rwatson) Date: Sat, 3 Jan 2009 12:33:32 GMT Message-Id: <200901031233.n03CXWAc031830@freefall.freebsd.org> To: rwatson@FreeBSD.org, rwatson@FreeBSD.org, net@FreeBSD.org From: rwatson@FreeBSD.org Cc: Subject: Re: kern/117717: [panic] Kernel panic with Bittorrent client. X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Jan 2009 12:33:33 -0000 Synopsis: [panic] Kernel panic with Bittorrent client. Responsible-Changed-From-To: rwatson->net Responsible-Changed-By: rwatson Responsible-Changed-When: Sat Jan 3 12:32:27 UTC 2009 Responsible-Changed-Why: Transfer to net@ after changing to suspended -- I've been unable to reproduce this problem and the original reporter confirms that it isn't present in 7.x, likely due to bms's substantial cleanup and rewrite of the multicast code in 7.x. I think we might reasonable close it at this point, but if someone is able to pick this up for 6.x and do the debugging, that would still be useful for 6.x users. http://www.freebsd.org/cgi/query-pr.cgi?pr=117717 From owner-freebsd-net@FreeBSD.ORG Sat Jan 3 15:02:21 2009 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 82F1A1065670; Sat, 3 Jan 2009 15:02:21 +0000 (UTC) (envelope-from vwe@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 590C98FC0C; Sat, 3 Jan 2009 15:02:21 +0000 (UTC) (envelope-from vwe@FreeBSD.org) Received: from freefall.freebsd.org (vwe@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n03F2Lk7042273; Sat, 3 Jan 2009 15:02:21 GMT (envelope-from vwe@freefall.freebsd.org) Received: (from vwe@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n03F2KbP042269; Sat, 3 Jan 2009 15:02:20 GMT (envelope-from vwe) Date: Sat, 3 Jan 2009 15:02:20 GMT Message-Id: <200901031502.n03F2KbP042269@freefall.freebsd.org> To: kes-kes@yandex.ru, vwe@FreeBSD.org, freebsd-net@FreeBSD.org, vwe@FreeBSD.org From: vwe@FreeBSD.org Cc: Subject: Re: kern/129074: [ppp] [panic] kernel panic with pppoe_server X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Jan 2009 15:02:21 -0000 Synopsis: [ppp] [panic] kernel panic with pppoe_server State-Changed-From-To: feedback->closed State-Changed-By: vwe State-Changed-When: Sat Jan 3 15:01:30 UTC 2009 State-Changed-Why: We're sorry to not see any feedback received for quite some time. If you think this is still an issue that should be worked on, please provide the requested information and we'll be happy to re-open this ticket. Thank you for bringing this problem to attention! Responsible-Changed-From-To: freebsd-net->vwe Responsible-Changed-By: vwe Responsible-Changed-When: Sat Jan 3 15:01:30 UTC 2009 Responsible-Changed-Why: track http://www.freebsd.org/cgi/query-pr.cgi?pr=129074 From owner-freebsd-net@FreeBSD.ORG Sat Jan 3 21:14:36 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1C638106566B for ; Sat, 3 Jan 2009 21:14:36 +0000 (UTC) (envelope-from perryh@pluto.rain.com) Received: from agora.rdrop.com (agora.rdrop.com [199.26.172.34]) by mx1.freebsd.org (Postfix) with ESMTP id 861B48FC18 for ; Sat, 3 Jan 2009 21:14:34 +0000 (UTC) (envelope-from perryh@pluto.rain.com) Received: from agora.rdrop.com (66@localhost [127.0.0.1]) by agora.rdrop.com (8.13.1/8.12.7) with ESMTP id n03LEXsW024237 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sat, 3 Jan 2009 13:14:34 -0800 (PST) (envelope-from perryh@pluto.rain.com) Received: (from uucp@localhost) by agora.rdrop.com (8.13.1/8.12.9/Submit) with UUCP id n03LEXSU024236; Sat, 3 Jan 2009 13:14:33 -0800 (PST) Received: from fbsd61 by pluto.rain.com (4.1/SMI-4.1-pluto-M2060407) id AA06284; Sat, 3 Jan 09 13:10:59 PST Date: Sat, 03 Jan 2009 13:13:24 -0800 From: perryh@pluto.rain.com To: smithi@nimnet.asn.au Message-Id: <495fd4f4.LnYmNJ/Km8Riy79x%perryh@pluto.rain.com> References: <495edc8b.yfwTDGtb9G/8NMur%perryh@pluto.rain.com> <20090103154232.P28770@sola.nimnet.asn.au> <495f15da.kLIW2g4L+3rMjCXS%perryh@pluto.rain.com> <20090103185837.K28770@sola.nimnet.asn.au> In-Reply-To: <20090103185837.K28770@sola.nimnet.asn.au> User-Agent: nail 11.25 7/29/05 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org Subject: Re: tun0 not responding to ping X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Jan 2009 21:14:36 -0000 Ian Smith wrote: > On Fri, 2 Jan 2009, perryh@pluto.rain.com wrote: > > Ian Smith wrote: > > uucp .. how quaint :) Yep, but running over ssh since agora no longer has modems. How's that for a mix of ancient and modern technology? :) > > http://www.cs.rpi.edu/~flemej/fbsd-cisco-vpn.pdf > > "You don't have permission to access /~flemej/fbsd-cisco-vpn.pdf > on this server." Nor http://www.cs.rpi.edu/~flemej .. so I can't > consult that, That's odd. It worked from here as recently as 12/17. The article is FreeBSD interoperability with Cisco VPN Concentrator 3000 series James Flemer jflemer@alum.rpi.edu October 10, 2002 and the relevant excerpt -- after it describes a setup involving netgraph(4) and the mpd port -- is 3.4 Routing Unfortunately, this does not work completely. It successfully establishes the PPTP connection, but cannot send anything over it. The problem is that the PPTP implementation for the concentrator forces its end of the PPP link to have the same IP as the address of its public interface (192.168.0.2 in this network). This causes FreeBSD to have routing problems, because the default gateway becomes 192.168.0.2 (via ng0), but in order to use that tunnel it has to send GRE packets to 192.168.0.2. The solution to this is as follows. Once the PPTP link is up, you need to re-address the ng0 interface and then change your default route. In the example network, you have to execute the following commands (assuming we are assigned 10.0.2.42 for our side of the link): # ifconfig ng0 inet 10.0.2.42 10.0.0.2 netmask 0xffffffff # route delete default # route add default -interface ng0 What I see is a bit different -- both ends get the IP that's supposed to have been assigned to my end, rather than the Cisco end getting the Cisco's public IP -- but perhaps related. > but as I said, I know next to nothing about VPN configuration anyway. I suspect this problem has more to do with PPP, tun(4), and routing than with VPN's as such. vpnc does seem to be establishing the VPN connection. > > * Supposing that tun0 does need to be readdressed as > > > > inet ZZZ.ZZZ.233.42 --> ZZZ.ZZZ.2.13 netmask 0xffffffff > > > > -- where ZZZ.ZZZ.2.13 is the address of the Cisco box on > > ZZZ.ZZZ.0.0/16 -- I'm not at all clear on how a w/a should get > > that internal address in the general case. (I got it by running > > a traceroute from an inside machine to a working VPN-connected > > Windows system, after not finding anything in the vpnc logs.) > > Beyond me .. I don't even know what a w/a is, but I'm pretty sure > ppp is going to need a remote address, and a route to it. w/a = workaround. > Usually you can ping either end; ping is the same as ping > localhost That's what I expected. > ping is, well, that. With both the same, I'm not > too surprised that ppp can't figure out which end you want to > talk to? Maybe that's (part of?) the problem, although I would have thought that the local side would immediately respond to its own address, without even checking anything else. > We ran ppp for 10 years on a dialup link but these days for pppoe > using mpd, but the routing should come to about the same, given > that here it's our default route. > > ng0: > flags=88d1 mtu 1492 > inet xxx.yyy.zzz.227 --> xxx.yyy.1.33 netmask 0xffffffff Hmm. Maybe tun0 needs NOARP and/or SIMPLEX (but, as with the remote IP address, I'd have expected vpnc to configure the interface as required rather than needing help). > Destination Gateway Flags Refs Use Netif Expire > default xxx.yyy.1.33 UGS 0 24390 ng0 > [..] > xxx.yyy.1.33 xxx.yyy.zzz.227 UH 1 0 ng0 > xxx.yyy.zzz.227/32 lo0 US 0 2 lo0 > > This is a 5.5 system, in case different presentation might mislead. This one is not all that much newer (6.1). One thing I notice, which seems odd, is the route to ng0's local IP address via lo0. Shouldn't the stack be able to communicate directly with a local ng (or tun) interface, just as it does with something like an xl0 (or lo0, for that matter)?