From owner-freebsd-net@FreeBSD.ORG Mon Apr 26 22:20:48 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 62817106566B for ; Mon, 26 Apr 2010 22:20:48 +0000 (UTC) (envelope-from cbuechler@gmail.com) Received: from mail-qy0-f181.google.com (mail-qy0-f181.google.com [209.85.221.181]) by mx1.freebsd.org (Postfix) with ESMTP id 1A9918FC18 for ; Mon, 26 Apr 2010 22:20:47 +0000 (UTC) Received: by qyk11 with SMTP id 11so15627582qyk.13 for ; Mon, 26 Apr 2010 15:20:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=xqugpnAchEjOiDK96dfSM3raRD75F6AFvBEK41HZ94o=; b=wYrMyisjJBJ0rN2pywofvRKA+3+08s2m7aH3xPmJI5vrrsI7pplUO/7zHP+3H7PXM6 XJs0HLvlflZQVf8y/9mmM2WDtNfObeJDykjslK1lzoNUZnOEpXaHi7iOTJzzK3qXZmfM Ppl+4alwv7pmzaeLzqnkoZadfuGKT6b/RbOSM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=B/mVk4ZP9ucgP6twR4ZnSVTH5sAygOZNiuAjY2JxliseYIhWlp+U9z2jFSvauEiKiz L5kHMcqF7sdMmYhAeo0MQYquDYsaZHOej7aS8GkD53b3URwOco1PdAbtKhh5ooP5fyJQ SsZfWUSeNPru3sr9iX2sCqvbdN9XSZRLRYlwY= MIME-Version: 1.0 Received: by 10.229.14.157 with SMTP id g29mr5867212qca.57.1272318794585; Mon, 26 Apr 2010 14:53:14 -0700 (PDT) Received: by 10.229.53.136 with HTTP; Mon, 26 Apr 2010 14:53:14 -0700 (PDT) In-Reply-To: <4BD59625.8080000@rusig.ru> References: <4BD59625.8080000@rusig.ru> Date: Mon, 26 Apr 2010 17:53:14 -0400 Message-ID: From: Chris Buechler To: =?KOI8-R?B?4czFy9PBzsTS?= Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org Subject: Re: multi-homed systems stop answering ARP on local addresses w/ifconfig aliases X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Apr 2010 22:20:48 -0000 2010/4/26 =E1=CC=C5=CB=D3=C1=CE=C4=D2 : > On Sun, May 17, 2009 at 4:35 PM, Steven Hartland > > wrote: >> >> / Silly question but something else on the network isn't doing a arp spo= of > > />/ attack is it? > />/ > / > No, there isn't any ARP at all on that address on the network when > this is a problem, verified with tcpdump. That also shouldn't impact > the system's ability to talk to its own IPs. > > thanks for the response though! > > >> / ----- Original Message ----- From: "Chris Buechler" > > />/ > > />/ To: > > />/ Sent: Sunday, May 17, 2009 9:08 PM > />/ Subject: multi-homed systems stop answering ARP on local addresses > />/ w/ifconfig aliases > />/ > />/ > />>/ There seems to be a regression between 6.x and 7.0 and 7.1 related t= o > />>/ ifconfig aliases on multi-homed hosts. Not sure on anything newer th= an > 7.1 > />>/ (this is pfSense, we're just starting to test 7.2 builds). For perio= ds > of > />>/ time, the system will stop answering ARP on some of its own addresse= s > and > />>/ hence anything on that network completely stops functioning. The sam= e > setup > />>/ worked fine on 6.2. > />>/ > />>/ The particular system illustrated here is a router on part of an ISP= 's > />>/ network. IPs are all public, in the info provided here they've been > replaced > />>/ with 10. IPs. The subnets on the inside interfaces are routed to the > outside > />>/ interface. When this problem occurs, the IPs assigned locally on the > system > />>/ will still respond from the Internet, but the system itself loses al= l > />>/ connectivity with that subnet and nothing on that subnet can > communicate > />>/ with the host due to the lack of ARP. That makes some sense, I presu= me > when > />>/ routing to a locally assigned address via another interface, the sys= tem > />>/ doesn't need ARP on the address to respond. But while it still respo= nds > from > />>/ the Internet, even the host itself can't initiate a ping to that IP.= It > />>/ behaves the same whether pf is enabled or disabled. > />>/ > />>/ I see two similar issues in the past, one with a PR: > />>/ http://www.freebsd.org/cgi/query-pr.cgi?pr=3D121437&cat=3D > > />>/ that's exactly the same issue, it's not limited to VLANs, any > multi-homed > />>/ host is affected. > />>/ > />>/ And another: > />>/ http://thread.gmane.org/gmane.os.freebsd.stable/57125 > />>/ > />>/ fxp0 is the outside interface. It doesn't make any difference whethe= r > the > />>/ ifconfig aliases are on the em0 or fxp1 interfaces, they both behave > the > />>/ same if they have any ifconfig aliases assigned. > />>/ > />>/ # ifconfig > />>/ fxp0: flags=3D8843 metric 0 = mtu > 1500 > />>/ =9A =9A =9A options=3D8 > />>/ =9A =9A =9A ether 00:90:27:86:8b:9d > />>/ =9A =9A =9A inet6 fe80::290:27ff:fe86:8b9d%fxp0 prefixlen 64 scopeid= 0x1 > />>/ =9A =9A =9A inet 10.11.185.146 netmask 0xfffffff8 broadcast 10.11.18= 5.151 > />>/ =9A =9A =9A media: Ethernet 100baseTX > />>/ =9A =9A =9A status: active > />>/ em0: flags=3D8843 metric 0 m= tu > 1500 > />>/ =9A =9A =9A options=3D9b > />>/ =9A =9A =9A ether 00:11:43:2c:62:03 > />>/ =9A =9A =9A inet 10.10.0.1 netmask 0xffffff00 broadcast 10.10.0.255 > />>/ =9A =9A =9A inet6 fe80::211:43ff:fe2c:6203%em0 prefixlen 64 scopeid = 0x2 > />>/ =9A =9A =9A inet 10.13.40.1 netmask 0xffffff00 broadcast 10.13.40.25= 5 > />>/ =9A =9A =9A inet 10.13.41.1 netmask 0xffffff00 broadcast 10.13.41.25= 5 > />>/ =9A =9A =9A inet 10.13.42.1 netmask 0xffffff00 broadcast 10.13.42.25= 5 > />>/ =9A =9A =9A inet 10.13.43.1 netmask 0xffffff00 broadcast 10.13.43.25= 5 > />>/ =9A =9A =9A inet 10.13.44.1 netmask 0xffffff00 broadcast 10.13.44.25= 5 > />>/ =9A =9A =9A inet 10.13.45.1 netmask 0xffffff00 broadcast 10.13.45.25= 5 > />>/ =9A =9A =9A inet 10.13.46.1 netmask 0xffffff00 broadcast 10.13.46.25= 5 > />>/ =9A =9A =9A inet 10.13.47.1 netmask 0xffffff00 broadcast 10.13.47.25= 5 > />>/ =9A =9A =9A media: Ethernet autoselect (100baseTX ) > />>/ =9A =9A =9A status: active > />>/ fxp1: flags=3D8843 metric 0 = mtu > 1500 > />>/ =9A =9A =9A options=3D8 > />>/ =9A =9A =9A ether 00:d0:b7:5d:25:9f > />>/ =9A =9A =9A inet 10.1.242.1 netmask 0xffffff00 broadcast 10.1.242.25= 5 > />>/ =9A =9A =9A inet6 fe80::2d0:b7ff:fe5d:259f%fxp1 prefixlen 64 scopeid= 0x3 > />>/ =9A =9A =9A inet 10.1.243.1 netmask 0xffffff00 broadcast 10.1.243.25= 5 > />>/ =9A =9A =9A media: Ethernet autoselect (100baseTX ) > />>/ =9A =9A =9A status: active > />>/ > />>/ > />>/ > />>/ When the problem is occurring, you can't even ping the affected loca= lly > />>/ assigned addresses from the box itself: > />>/ # ping 10.10.0.1 > />>/ PING 10.10.0.1 (10.10.0.1): 56 data bytes > />>/ ping: sendto: Network is unreachable > />>/ ping: sendto: Network is unreachable > />>/ ping: sendto: Network is unreachable > />>/ > />>/ And when trying to ping something on one of the affected attached > subnets, > />>/ you get: > />>/ # ping 10.10.0.30 > />>/ PING 10.10.0.30 (10.10.0.30): 56 data bytes > />>/ ping: sendto: Invalid argument > />>/ ping: sendto: Invalid argument > />>/ > />>/ > />>/ In the logs, you get a flood of these messages: > />>/ May 14 02:55:12 =9A =9Akernel: arpresolve: can't allocate route for > 10.10.0.1 > />>/ May 14 02:55:12 =9A =9Akernel: arplookup 10.10.0.1 failed: host is n= ot on > />>/ local network > />>/ May 14 02:55:12 =9A =9Akernel: arpresolve: can't allocate route for > 10.10.0.1 > />>/ May 14 02:55:12 =9A =9Akernel: arplookup 10.10.0.1 failed: host is n= ot on > />>/ local network > />>/ > />>/ > />>/ It happens both with the primary IP assigned to the interface, and t= he > />>/ aliases assigned, but not all at once. Some of the addresses will > continue > />>/ to work when others are failing. Somehow it thinks IPs that are loca= lly > />>/ assigned are not on a local network... after a couple minutes, it ju= st > />>/ starts working again without making any changes or even touching the > system. > />>/ > />>/ If I can provide any additional information, please let me know. > />>/ > />>/ thanks, > />>/ Chris > />>/ > // > > //The same thing happened to me. FreeBSD 8.0-RELEASE, bge0-bge4, bge0 > configured with two subnets. > ARP for primary subnet dissappear randomly (~once a day). > > > Do you have any resolution for this? > I've yet to test it on 8, but never got a resolution to the above described issue on 7.2. For the scenario where that box was deployed, parts of the functionality were replaced by a Linux box, and the part of the network where it resides was changed so IP aliases are not in use.