From owner-freebsd-net@FreeBSD.ORG Sat May 12 07:21:16 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1C51F106564A for ; Sat, 12 May 2012 07:21:16 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from mail-wi0-f178.google.com (mail-wi0-f178.google.com [209.85.212.178]) by mx1.freebsd.org (Postfix) with ESMTP id 9BD678FC0A for ; Sat, 12 May 2012 07:21:15 +0000 (UTC) Received: by wibhn6 with SMTP id hn6so772140wib.13 for ; Sat, 12 May 2012 00:21:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date :content-transfer-encoding:message-id:references:to:x-mailer; bh=8xTKh6kM3BWEtoAzfTyCdrGAYFum0NtMyQOrXnqKxXs=; b=L8VaQ27Sg1KNDmLuaUrjDf9UsL6X7IZskhg8Vj0IuOmbtKOb1x59N062Qz4y/X+yAU 1Sq9d0FEhyqLfihAa8jwA0y0UsAxA9o7tL12vvWUvmS+N3ncUDkrRDnN59z3RM3Grnvy XVeU5vZEmpbvD/6jjPxx709Oe6rcQ5mLWuVQiMyLKewheY3ru1GOVzNM3crnjPsDJvwf byF6PaUL/DBF+g6FBhtF+rcXfr9Zuv85vY8r51IVaJP3R4m7kCmVkwF/jnrCkOJiuPAv 5JtkCWzumoh2RmmE/mcfIHqSVVdTpB61oovis2SUi/871umoU6+vPmHpq19XCfe/I+Qg 1+Bw== Received: by 10.180.86.5 with SMTP id l5mr2328211wiz.6.1336807269407; Sat, 12 May 2012 00:21:09 -0700 (PDT) Received: from [10.0.0.3] ([93.152.152.135]) by mx.google.com with ESMTPS id e6sm16456038wix.8.2012.05.12.00.21.07 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 12 May 2012 00:21:07 -0700 (PDT) Mime-Version: 1.0 (Apple Message framework v1257) Content-Type: text/plain; charset=iso-8859-1 From: Nikolay Denev In-Reply-To: <4B587DD0.8020700@icritical.com> Date: Sat, 12 May 2012 10:21:05 +0300 Content-Transfer-Encoding: quoted-printable Message-Id: <59FAFD0B-A107-4173-9FA9-BA3349D499E2@gmail.com> References: <4B587DD0.8020700@icritical.com> To: freebsd-net@freebsd.org X-Mailer: Apple Mail (2.1257) Subject: Re: setfib/arpresolve behaviour bug? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 May 2012 07:21:16 -0000 On Jan 21, 2010, at 6:16 PM, Matt Burke wrote: > Box is running 8.0-RELEASE-p2 cvsupped two days ago. >=20 > NICs are em bonded with lagg failover and running a few vlan = interfaces. >=20 > net.my_fibnum: 0 > net.add_addr_allfibs: 1 > net.fibs: 4 >=20 > This is reproducible, but with the lack of (accessible?) documentation = on > multiple routing tables, I don't know if this is intended behaviour or = a bug. >=20 > It seems processes using a non-default fib cannot perform arp lookups > unless the fib 0 has a routing table entry for the attached network: >=20 > [root@host ~]# ifconfig vlan11 a.a.a.92/27 > [root@host ~]# route delete -net a.a.a.64/27 > delete net a.a.a.64 > [root@host ~]# setfib 1 ping a.a.a.65 > PING a.a.a.65 (a.a.a.65): 56 data bytes > ping: sendto: Invalid argument > ^C > --- a.a.a.65 ping statistics --- > 1 packets transmitted, 0 packets received, 100.0% packet loss > [root@host ~]# dmesg |tail -1 > arpresolve: can't allocate llinfo for a.a.a.65 >=20 >=20 > Putting the entry into the arp cache before removing the route results = in > success: >=20 > [root@host ~]# ifconfig vlan11 a.a.a.92/27 > [root@host ~]# setfib 1 ping a.a.a.65 > PING a.a.a.65 (a.a.a.65): 56 data bytes > 64 bytes from a.a.a.65: icmp_seq=3D0 ttl=3D255 time=3D1.437 ms > ^C > --- a.a.a.65 ping statistics --- > 1 packets transmitted, 1 packets received, 0.0% packet loss > round-trip min/avg/max/stddev =3D 1.437/1.437/1.437/0.000 ms > [root@host ~]# route delete -net a.a.a.64/27 > delete net a.a.a.64 > [root@host ~]# setfib 1 ping a.a.a.65 > PING a.a.a.65 (a.a.a.65): 56 data bytes > 64 bytes from a.a.a.65: icmp_seq=3D0 ttl=3D255 time=3D0.762 ms > ^C > --- a.a.a.65 ping statistics --- > 1 packets transmitted, 1 packets received, 0.0% packet loss > round-trip min/avg/max/stddev =3D 0.762/0.762/0.762/0.000 ms >=20 >=20 > and deleting it again results in failure: >=20 > [root@host ~]# arp -an > ? (a.a.a.92) at 00:11:27:00:d7:c4 on vlan11 permanent [vlan] > ? (a.a.a.65) at 00:1a:e4:00:60:bf on vlan11 [vlan] > ... > [root@host ~]# arp -d a.a.a.65 > delete: cannot locate a.a.a.65 > [root@host ~]# setfib 1 arp -d a.a.a.65 > a.a.a.65 (a.a.a.65) deleted > [root@host ~]# setfib 1 ping -c1 a.a.a.65 > PING a.a.a.65 (a.a.a.65): 56 data bytes > ping: sendto: Invalid argument > ^C > --- a.a.a.65 ping statistics --- > 1 packets transmitted, 0 packets received, 100.0% packet loss >=20 >=20 > This behaviour seems a little inconsistent, with fib 1 requesting arp > lookups, fib 0 performing and displaying them, but fib 1 needing to = delete > them... >=20 >=20 >=20 > --=20 >=20 > The information contained in this message is confidential and is = intended for the addressee only. If you have received this message in = error or there are any problems please notify the originator = immediately. The unauthorised use, disclosure, copying or alteration of = this message is strictly forbidden.=20 >=20 > Critical Software Ltd. reserves the right to monitor and record e-mail = messages sent to and from this address for the purposes of investigating = or detecting any unauthorised use of its system and ensuring its = effective operation. >=20 > Critical Software Ltd. registered in England, 04909220. Registered = Office: IC2, Keele Science Park, Keele, Staffordshire, ST5 5NH. >=20 > ------------------------------------------------------------ > This message has been scanned for security threats by iCritical. > For further information, please visit www.icritical.com > ------------------------------------------------------------ I've encountered exactly the same problem today. I have a machine with public addresses, and also a interface for out of = band management with private address, and I wanted to use a separate FIB for the private interface and it's routes. When I've deleted the routes for the private interface form the main = FIB, arpresolve stopped working. The I've patched sys/netinet/in.c with the following patch : --- sys/netinet/in.c.orig 2012-05-12 08:57:17.000000000 +0200 +++ sys/netinet/in.c 2012-05-12 08:56:43.000000000 +0200 @@ -1418,21 +1418,21 @@ =20 static int in_lltable_rtcheck(struct ifnet *ifp, u_int flags, const struct = sockaddr *l3addr) { struct rtentry *rt; =20 KASSERT(l3addr->sa_family =3D=3D AF_INET, ("sin_family %d", l3addr->sa_family)); =20 /* XXX rtalloc1 should take a const param */ - rt =3D rtalloc1(__DECONST(struct sockaddr *, l3addr), 0, 0); + rt =3D rtalloc1_fib(__DECONST(struct sockaddr *, l3addr), 0, 0, = ifp->if_fib); =20 if (rt =3D=3D NULL) return (EINVAL); =20 /* * If the gateway for an existing host route matches the target = L3 * address, which is a special route inserted by some = implementation * such as MANET, and the interface is of the correct type, then * allow for ARP to proceed. */ And this seems to fix the issue. Now that the multi FIB code is in GENERIC probably this (or similar fix) = should be comitted. P.S.: I also wonder why the loopback route for an interface address is = also installed explicitly in the default FIB?