From owner-freebsd-net@FreeBSD.ORG Wed May 16 04:00:30 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C5B2B106564A for ; Wed, 16 May 2012 04:00:30 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from mail-wg0-f50.google.com (mail-wg0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 5005F8FC08 for ; Wed, 16 May 2012 04:00:15 +0000 (UTC) Received: by wgbds11 with SMTP id ds11so254585wgb.31 for ; Tue, 15 May 2012 21:00:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date :content-transfer-encoding:message-id:references:to:x-mailer; bh=gOiHszZjYsn54f2Put+Rvppe0VyCsm5j3GxUdOemywQ=; b=pN1LcloOIUdWzoQf8ygCJ83+iwkkPzFiSxR/fxJ2bWXoTP8A0UeimQeoaLsAKUxljb O5RfZscICikqnvgiUdWV0Ipftp2q3foD/48mP5/TtCyUx3pUPZtyC90nQWed4DB2FA5m 2FG0BhtO6sJmwFoh2jVZ3eBOGsDLCgqpSfOMZdd4lD68jCR81e93YTwr+f1jSlHlTD/L WJzb5aBr9C7aI0BBlGtoQ7L3tjZ9ea1nEUEa1HNzvL5UPlulx7veB+U3WOD4I7tfXc5F 3jINdfhA8MLMg+2RfpuhAtgwmbVQKt8qgAmLh6pmGGBZjZxuIH4Hx6xUyDORamXhdF1h XA/Q== Received: by 10.180.102.9 with SMTP id fk9mr11705006wib.1.1337140814590; Tue, 15 May 2012 21:00:14 -0700 (PDT) Received: from [10.0.0.3] ([93.152.152.135]) by mx.google.com with ESMTPS id gd4sm4632606wib.6.2012.05.15.21.00.12 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 15 May 2012 21:00:13 -0700 (PDT) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=iso-8859-1 From: Nikolay Denev In-Reply-To: <59FAFD0B-A107-4173-9FA9-BA3349D499E2@gmail.com> Date: Wed, 16 May 2012 07:00:12 +0300 Content-Transfer-Encoding: quoted-printable Message-Id: References: <4B587DD0.8020700@icritical.com> <59FAFD0B-A107-4173-9FA9-BA3349D499E2@gmail.com> To: freebsd-net@freebsd.org X-Mailer: Apple Mail (2.1278) Subject: Re: setfib/arpresolve behaviour bug? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 May 2012 04:00:30 -0000 Filed as=20 misc/167947 On May 12, 2012, at 10:21 AM, Nikolay Denev wrote: > On Jan 21, 2010, at 6:16 PM, Matt Burke wrote: >=20 >> Box is running 8.0-RELEASE-p2 cvsupped two days ago. >>=20 >> NICs are em bonded with lagg failover and running a few vlan = interfaces. >>=20 >> net.my_fibnum: 0 >> net.add_addr_allfibs: 1 >> net.fibs: 4 >>=20 >> This is reproducible, but with the lack of (accessible?) = documentation on >> multiple routing tables, I don't know if this is intended behaviour = or a bug. >>=20 >> It seems processes using a non-default fib cannot perform arp lookups >> unless the fib 0 has a routing table entry for the attached network: >>=20 >> [root@host ~]# ifconfig vlan11 a.a.a.92/27 >> [root@host ~]# route delete -net a.a.a.64/27 >> delete net a.a.a.64 >> [root@host ~]# setfib 1 ping a.a.a.65 >> PING a.a.a.65 (a.a.a.65): 56 data bytes >> ping: sendto: Invalid argument >> ^C >> --- a.a.a.65 ping statistics --- >> 1 packets transmitted, 0 packets received, 100.0% packet loss >> [root@host ~]# dmesg |tail -1 >> arpresolve: can't allocate llinfo for a.a.a.65 >>=20 >>=20 >> Putting the entry into the arp cache before removing the route = results in >> success: >>=20 >> [root@host ~]# ifconfig vlan11 a.a.a.92/27 >> [root@host ~]# setfib 1 ping a.a.a.65 >> PING a.a.a.65 (a.a.a.65): 56 data bytes >> 64 bytes from a.a.a.65: icmp_seq=3D0 ttl=3D255 time=3D1.437 ms >> ^C >> --- a.a.a.65 ping statistics --- >> 1 packets transmitted, 1 packets received, 0.0% packet loss >> round-trip min/avg/max/stddev =3D 1.437/1.437/1.437/0.000 ms >> [root@host ~]# route delete -net a.a.a.64/27 >> delete net a.a.a.64 >> [root@host ~]# setfib 1 ping a.a.a.65 >> PING a.a.a.65 (a.a.a.65): 56 data bytes >> 64 bytes from a.a.a.65: icmp_seq=3D0 ttl=3D255 time=3D0.762 ms >> ^C >> --- a.a.a.65 ping statistics --- >> 1 packets transmitted, 1 packets received, 0.0% packet loss >> round-trip min/avg/max/stddev =3D 0.762/0.762/0.762/0.000 ms >>=20 >>=20 >> and deleting it again results in failure: >>=20 >> [root@host ~]# arp -an >> ? (a.a.a.92) at 00:11:27:00:d7:c4 on vlan11 permanent [vlan] >> ? (a.a.a.65) at 00:1a:e4:00:60:bf on vlan11 [vlan] >> ... >> [root@host ~]# arp -d a.a.a.65 >> delete: cannot locate a.a.a.65 >> [root@host ~]# setfib 1 arp -d a.a.a.65 >> a.a.a.65 (a.a.a.65) deleted >> [root@host ~]# setfib 1 ping -c1 a.a.a.65 >> PING a.a.a.65 (a.a.a.65): 56 data bytes >> ping: sendto: Invalid argument >> ^C >> --- a.a.a.65 ping statistics --- >> 1 packets transmitted, 0 packets received, 100.0% packet loss >>=20 >>=20 >> This behaviour seems a little inconsistent, with fib 1 requesting arp >> lookups, fib 0 performing and displaying them, but fib 1 needing to = delete >> them... >>=20 >>=20 >>=20 >> --=20 >>=20 >> The information contained in this message is confidential and is = intended for the addressee only. If you have received this message in = error or there are any problems please notify the originator = immediately. The unauthorised use, disclosure, copying or alteration of = this message is strictly forbidden.=20 >>=20 >> Critical Software Ltd. reserves the right to monitor and record = e-mail messages sent to and from this address for the purposes of = investigating or detecting any unauthorised use of its system and = ensuring its effective operation. >>=20 >> Critical Software Ltd. registered in England, 04909220. Registered = Office: IC2, Keele Science Park, Keele, Staffordshire, ST5 5NH. >>=20 >> ------------------------------------------------------------ >> This message has been scanned for security threats by iCritical. >> For further information, please visit www.icritical.com >> ------------------------------------------------------------ >=20 > I've encountered exactly the same problem today. >=20 > I have a machine with public addresses, and also a interface for out = of band management with private address, and I wanted to use > a separate FIB for the private interface and it's routes. >=20 > When I've deleted the routes for the private interface form the main = FIB, arpresolve stopped working. >=20 > The I've patched sys/netinet/in.c with the following patch : >=20 > --- sys/netinet/in.c.orig 2012-05-12 08:57:17.000000000 +0200 > +++ sys/netinet/in.c 2012-05-12 08:56:43.000000000 +0200 > @@ -1418,21 +1418,21 @@ >=20 > static int > in_lltable_rtcheck(struct ifnet *ifp, u_int flags, const struct = sockaddr *l3addr) > { > struct rtentry *rt; >=20 > KASSERT(l3addr->sa_family =3D=3D AF_INET, > ("sin_family %d", l3addr->sa_family)); >=20 > /* XXX rtalloc1 should take a const param */ > - rt =3D rtalloc1(__DECONST(struct sockaddr *, l3addr), 0, 0); > + rt =3D rtalloc1_fib(__DECONST(struct sockaddr *, l3addr), 0, 0, = ifp->if_fib); >=20 > if (rt =3D=3D NULL) > return (EINVAL); >=20 > /* > * If the gateway for an existing host route matches the target = L3 > * address, which is a special route inserted by some = implementation > * such as MANET, and the interface is of the correct type, then > * allow for ARP to proceed. > */ >=20 >=20 > And this seems to fix the issue. >=20 > Now that the multi FIB code is in GENERIC probably this (or similar = fix) should be comitted. >=20 > P.S.: I also wonder why the loopback route for an interface address is = also installed explicitly in the default FIB? >=20 >=20 >=20