Date: Tue, 28 Jul 2009 14:52:52 -0700 From: Bob Van Zant <bob@veznat.com> To: Julian Elischer <julian@elischer.org> Cc: freebsd-net@freebsd.org Subject: Re: IPv6 traffic between two hosts not flowing Message-ID: <C694C144.37036%bob@veznat.com> In-Reply-To: <4A69395B.4020604@elischer.org>
next in thread | previous in thread | raw e-mail | index | archive | help
The problem ended up being a semi-broken em (yes, em, bge wasn't the problem like I originally thought) driver in 6.2. The driver wasn't bringing in ethernet multicast frames (required for ndp to work) and without ndp things weren't working. If the interface was put into promiscuous mode for a few seconds ndp could take place and everything worked fine. Upgrading the kernel to 6.3 made the problem go away. Just in case it's useful to anyone, these are the 3 em devices I had on this machine (a Dell 2850): em0@pci6:7:0: class=0x020000 card=0x016d1028 chip=0x10768086 rev=0x05 hdr=0x00 em1@pci7:8:0: class=0x020000 card=0x016d1028 chip=0x10768086 rev=0x05 hdr=0x00 em2@pci9:4:0: class=0x020000 card=0x10028086 chip=0x10268086 rev=0x04 hdr=0x00 em2 was the only device functioning properly on 6.2. -Bob On 7/23/09 9:32 PM, "Julian Elischer" <julian@elischer.org> wrote: > Bob Van Zant wrote: >> I have 2 FreeBSD machines each with 3 ethernet interfaces each with an IPv6 >> address. >> >> Here's a table showing how the two machines are connected. To try to blame >> the switch or its configuration I have tried directly connecting the >> machines. The results were the same. >> >> 6.3 4948 6.2 >> --- ---- --- >> fxp0 em2 >> fc00:dada::47 fc00:dada::2 >> >> bge0 em0 >> fc00:dada:1::47 fc00:dada:1::2 >> >> bge1 em1 >> fc00:dada:2::47 fc00:dada:2::2 >> >> Traffic from fxp0 to em2 works fine. >> bge0 to em0 and bge1 to em1 doesn't work as I'll explain below and is the >> reason I'm writing to the list. >> >> >> This host is able to communicate with it's IPv6 neighbor over the fxp0 >> interface, however, neither of the bge interfaces are able to communicate >> with their peers. >> >> ndp(8) output on the 6.3 machine (with bge interfaces) seems to highlight >> something weird. In addition to the errors occurring, the Netif column for >> fc00:dada:1::47 shows fxp0. ifconfig shows that address on bge0. The same is >> true for fc00:dada:2::47; in ndp it shows up on fxp0 but ifconfig thinks >> it's on bge1 (it should be). The linklayer address being "(incomplete)" also >> seems weird given that this is a normal ethernet interface on the local >> machine. > > I'm going to guess that if the other connections were disconnected, > the 2nd and 3rd pairs would work. > > I'm also guessing that if you closely look at the data between the 2nd > and 3rd pairs you will somewhere see the addresses of the first pair > in there. FreeBSD has often suffered when trying to run connections in > parallel, even in IPV4. > > I believe this has been fixed in 8.x to some extent.. > > > >> >> >> $ ndp -an >> Neighbor Linklayer Address Netif Expire S >> Flags >> fc00:dada::2 0:e:c:b4:13:8f fxp0 22h38m10s S R >> fc00:dada::47 0:2:b3:e7:4a:56 fxp0 permanent R >> ndp: ioctl(SIOCGNBRINFO_IN6): Invalid argument >> ndp: failed to get neighbor information >> fc00:dada:1::47 (incomplete) fxp0 >> ndp: ioctl(SIOCGNBRINFO_IN6): Invalid argument >> ndp: failed to get neighbor information >> fc00:dada:2::47 (incomplete) fxp0 >> fe80::20d:56ff:fe70:a9ae%bge0 0:d:56:70:a9:ae bge0 permanent R >> fe80::20d:56ff:fe70:a9af%bge1 0:d:56:70:a9:af bge1 permanent R >> fe80::202:b3ff:fee7:4a56%fxp0 0:2:b3:e7:4a:56 fxp0 permanent R >> >> >> Output from ifmcstat looks correct to me (mostly the same as fxp0 but with >> the appropriate addresses different). >> >> I did find this message in the logfile at least once: >> >> nd6_storelladdr: sdl_alen == 0 >> >> >> If I do a ping6 in between the two hosts and then run tcpdump on either of >> the bge interfaces I can see neighbor discovery traffic going back and >> forth. > > but not correct probably or it would be stored in the Link Level tables. > >> >> If the ping is from the bge interface to the corresponding em interface then >> all I ever see is the NDP traffic. >> >> If the ping is from the em interface to the corresponding bge interface then >> I see a normal solicitation and advertisement followed by ICMP echo requests >> (there are never any replies). The 6.2 machine's ndp cache looks correct. >> The 6.3 machine's ndp cache is the one pasted above. >> >> >> I feel like I'm just rambling here and not necessarily putting together a >> sane question. If this is making sense to anyone and it isn't immediately >> obvious what's going on please ask me some follow up questions. >> >> In addition to all of this, after what appears to be about an hour of >> sending ping packets that go nowhere and pulling interfaces in and out of >> promiscuous mode (tcpdump) the system appears to deadlock. I don't have any >> proof that it's deadlock other than a completely unresponsive box that needs >> to have the power cable removed. It's happened twice so far this afternoon. >> >> Thanks for reading this far, > > if you had an 8.x image, you might try it there.. > and a 6.2<->6.2 and 6.3<->6.3 boot as well > probably the person you want to actually speak with is Qing Li > >> >> Bob >> >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?C694C144.37036%bob>