From owner-freebsd-net@FreeBSD.ORG Tue Jul 28 21:52:56 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 442A9106566C for ; Tue, 28 Jul 2009 21:52:56 +0000 (UTC) (envelope-from bob@veznat.com) Received: from mail.ttora.com (mail.ttora.com [208.75.243.236]) by mx1.freebsd.org (Postfix) with ESMTP id F19598FC16 for ; Tue, 28 Jul 2009 21:52:55 +0000 (UTC) (envelope-from bob@veznat.com) DomainKey-Signature: s=two; d=veznat.com; c=nofws; q=dns; h=X-MID:X-IronPort-AV:X-IronPort-AV:Received:User-Agent: Date:Subject:From:To:CC:Message-ID:Thread-Topic: Thread-Index:In-Reply-To:Mime-version:Content-type: Content-transfer-encoding; b=c0jHbxGnhntsQ4FSoAD0Sy3zhSYig59vmEdHsGpRUc0lltVsFzUSst+J II+aoMTTv3i3qCpV+9O41g0FcKne+hMLNXmvUSP0lhjVgZzsxS9IgMugt kwyVwut2IxshSFUDKYlsXYoINaL0omj+iRNBLG0cC4vUlXg0Dm/uhDyir w=; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=veznat.com; i=bob@veznat.com; q=dns/txt; s=one; t=1248817976; x=1280353976; h=from:sender:reply-to:subject:date:message-id:to:cc: mime-version:content-transfer-encoding:content-id: content-description:resent-date:resent-from:resent-sender: resent-to:resent-cc:resent-message-id:in-reply-to: references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:list-owner:list-archive; z=From:=20Bob=20Van=20Zant=20|Subject:=20R e:=20IPv6=20traffic=20between=20two=20hosts=20not=20flowi ng|Date:=20Tue,=2028=20Jul=202009=2014:52:52=20-0700 |Message-ID:=20|To:=20Juli an=20Elischer=20|CC:=20|Mime-version:=201.0 |Content-transfer-encoding:=207bit|In-Reply-To:=20<4A6939 5B.4020604@elischer.org>; bh=/12ZhzvFWGnU5LWRdukhOvreZgKN1lGRwFJs1PSim2s=; b=kq1iqSwNGPOrdI8naC6LmGeheNY2jSyqEqNQpJNleXBoRe4+1S1B0I2d hyKvpnCHTy6R+wZ9mBYOOFnDFHtyCq5WTf/sCmRGupkslauwlZiWiab2j guhJ1E0Fs03aLEsODep7hRXbTBbSMsgq4uPZYk3Y9u9GoAlthfWFgI02t o=; X-MID: 1950820 X-IronPort-AV: E=McAfee;i="5300,2777,5691"; a="1950820" X-IronPort-AV: E=Sophos;i="4.43,284,1246863600"; d="scan'208";a="1950820" Received: from nat.ironport.com (HELO [173.37.10.6]) ([63.251.108.100]) by mail.ttora.com with ESMTP/TLS/DES-CBC3-SHA; 28 Jul 2009 14:52:54 -0700 User-Agent: Microsoft-Entourage/12.12.0.080729 Date: Tue, 28 Jul 2009 14:52:52 -0700 From: Bob Van Zant To: Julian Elischer Message-ID: Thread-Topic: IPv6 traffic between two hosts not flowing Thread-Index: AcoPzcFYrKMZlGLf20OBBupQi4aCFw== In-Reply-To: <4A69395B.4020604@elischer.org> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Cc: freebsd-net@freebsd.org Subject: Re: IPv6 traffic between two hosts not flowing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Jul 2009 21:52:56 -0000 The problem ended up being a semi-broken em (yes, em, bge wasn't the problem like I originally thought) driver in 6.2. The driver wasn't bringing in ethernet multicast frames (required for ndp to work) and without ndp things weren't working. If the interface was put into promiscuous mode for a few seconds ndp could take place and everything worked fine. Upgrading the kernel to 6.3 made the problem go away. Just in case it's useful to anyone, these are the 3 em devices I had on this machine (a Dell 2850): em0@pci6:7:0: class=0x020000 card=0x016d1028 chip=0x10768086 rev=0x05 hdr=0x00 em1@pci7:8:0: class=0x020000 card=0x016d1028 chip=0x10768086 rev=0x05 hdr=0x00 em2@pci9:4:0: class=0x020000 card=0x10028086 chip=0x10268086 rev=0x04 hdr=0x00 em2 was the only device functioning properly on 6.2. -Bob On 7/23/09 9:32 PM, "Julian Elischer" wrote: > Bob Van Zant wrote: >> I have 2 FreeBSD machines each with 3 ethernet interfaces each with an IPv6 >> address. >> >> Here's a table showing how the two machines are connected. To try to blame >> the switch or its configuration I have tried directly connecting the >> machines. The results were the same. >> >> 6.3 4948 6.2 >> --- ---- --- >> fxp0 em2 >> fc00:dada::47 fc00:dada::2 >> >> bge0 em0 >> fc00:dada:1::47 fc00:dada:1::2 >> >> bge1 em1 >> fc00:dada:2::47 fc00:dada:2::2 >> >> Traffic from fxp0 to em2 works fine. >> bge0 to em0 and bge1 to em1 doesn't work as I'll explain below and is the >> reason I'm writing to the list. >> >> >> This host is able to communicate with it's IPv6 neighbor over the fxp0 >> interface, however, neither of the bge interfaces are able to communicate >> with their peers. >> >> ndp(8) output on the 6.3 machine (with bge interfaces) seems to highlight >> something weird. In addition to the errors occurring, the Netif column for >> fc00:dada:1::47 shows fxp0. ifconfig shows that address on bge0. The same is >> true for fc00:dada:2::47; in ndp it shows up on fxp0 but ifconfig thinks >> it's on bge1 (it should be). The linklayer address being "(incomplete)" also >> seems weird given that this is a normal ethernet interface on the local >> machine. > > I'm going to guess that if the other connections were disconnected, > the 2nd and 3rd pairs would work. > > I'm also guessing that if you closely look at the data between the 2nd > and 3rd pairs you will somewhere see the addresses of the first pair > in there. FreeBSD has often suffered when trying to run connections in > parallel, even in IPV4. > > I believe this has been fixed in 8.x to some extent.. > > > >> >> >> $ ndp -an >> Neighbor Linklayer Address Netif Expire S >> Flags >> fc00:dada::2 0:e:c:b4:13:8f fxp0 22h38m10s S R >> fc00:dada::47 0:2:b3:e7:4a:56 fxp0 permanent R >> ndp: ioctl(SIOCGNBRINFO_IN6): Invalid argument >> ndp: failed to get neighbor information >> fc00:dada:1::47 (incomplete) fxp0 >> ndp: ioctl(SIOCGNBRINFO_IN6): Invalid argument >> ndp: failed to get neighbor information >> fc00:dada:2::47 (incomplete) fxp0 >> fe80::20d:56ff:fe70:a9ae%bge0 0:d:56:70:a9:ae bge0 permanent R >> fe80::20d:56ff:fe70:a9af%bge1 0:d:56:70:a9:af bge1 permanent R >> fe80::202:b3ff:fee7:4a56%fxp0 0:2:b3:e7:4a:56 fxp0 permanent R >> >> >> Output from ifmcstat looks correct to me (mostly the same as fxp0 but with >> the appropriate addresses different). >> >> I did find this message in the logfile at least once: >> >> nd6_storelladdr: sdl_alen == 0 >> >> >> If I do a ping6 in between the two hosts and then run tcpdump on either of >> the bge interfaces I can see neighbor discovery traffic going back and >> forth. > > but not correct probably or it would be stored in the Link Level tables. > >> >> If the ping is from the bge interface to the corresponding em interface then >> all I ever see is the NDP traffic. >> >> If the ping is from the em interface to the corresponding bge interface then >> I see a normal solicitation and advertisement followed by ICMP echo requests >> (there are never any replies). The 6.2 machine's ndp cache looks correct. >> The 6.3 machine's ndp cache is the one pasted above. >> >> >> I feel like I'm just rambling here and not necessarily putting together a >> sane question. If this is making sense to anyone and it isn't immediately >> obvious what's going on please ask me some follow up questions. >> >> In addition to all of this, after what appears to be about an hour of >> sending ping packets that go nowhere and pulling interfaces in and out of >> promiscuous mode (tcpdump) the system appears to deadlock. I don't have any >> proof that it's deadlock other than a completely unresponsive box that needs >> to have the power cable removed. It's happened twice so far this afternoon. >> >> Thanks for reading this far, > > if you had an 8.x image, you might try it there.. > and a 6.2<->6.2 and 6.3<->6.3 boot as well > probably the person you want to actually speak with is Qing Li > >> >> Bob >> >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"