From owner-freebsd-stable@FreeBSD.ORG Mon Oct 13 15:23:58 2014 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7602D109; Mon, 13 Oct 2014 15:23:58 +0000 (UTC) Received: from mail-wi0-x236.google.com (mail-wi0-x236.google.com [IPv6:2a00:1450:400c:c05::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B06D518C; Mon, 13 Oct 2014 15:23:57 +0000 (UTC) Received: by mail-wi0-f182.google.com with SMTP id n3so7736713wiv.15 for ; Mon, 13 Oct 2014 08:23:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=US0Th29h9nJaK8mgK2lxpU+314KEeLr15OP9LiBzylE=; b=FOkT+adxgLpmP1at8zFh2S+mU/ugHTsfXBexvKQZ/COhoF/uaXOPnTtwTS/oA9Wlnf yTf3DiVm30v2zuydUS+vhxa8s4Fu1ESbumQW1Qj91pxLaSfH+GI8eoNKi8f31wi7lgEb oyt55TEZz9bYQg9BwyA6KSWqQVb6re8vQQTfQLJWlld6wdY251GS6NUNdnd+GoJKmPwt cxBHvbjpeDqmPqESlNzyJpUklTE45XyWa76xeQkRt1ic3Dc8NCSTyT4Dd8cD1WCO8Dyf ekcX2bs7U7DiSA7iPBNyQ36jBvCntCWlMf3K9xcUhWRlU34+TGTLNYFD4HnNzYvOwh1Y dmsQ== MIME-Version: 1.0 X-Received: by 10.180.72.45 with SMTP id a13mr1498776wiv.50.1413213835679; Mon, 13 Oct 2014 08:23:55 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.220.227 with HTTP; Mon, 13 Oct 2014 08:23:55 -0700 (PDT) In-Reply-To: <543B9873.3040605@omnilan.de> References: <53569ABA.60007@omnilan.de> <535771F3.4070007@freebsd.org> <543B8ED5.6040206@omnilan.de> <543B9075.2000102@FreeBSD.org> <543B9873.3040605@omnilan.de> Date: Mon, 13 Oct 2014 09:23:55 -0600 X-Google-Sender-Auth: 5fxoXsJaLCIZPcoeGnhdg96ydaI Message-ID: Subject: Re: Deleting IPv4 iface-routes from extra FIBs From: Alan Somers To: Harald Schmalzbauer Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-net@freebsd.org" , "Alexander V. Chernikov" , FreeBSD X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Oct 2014 15:23:58 -0000 On Mon, Oct 13, 2014 at 3:16 AM, Harald Schmalzbauer wrote: > Bez=C3=BCglich Alexander V. Chernikov's Nachricht vom 13.10.2014 10:42 > (localtime): >> On 13.10.2014 12:35, Harald Schmalzbauer wrote: >>> Bez=C3=BCglich Julian Elischer's Nachricht vom 23.04.2014 09:55 >>> (localtime): > ... >>>> yes, we made two behaviours. >>>> Add interface routes to all active FIBS or only add them to the first >>>> fib and let the user populate other fibs as needed. >>>> It appears you want the second behaviour, so I suggest you use that >>>> option and set up all your routes manually. >>> Hello, >>> >>> last time I had the iface-route problem, I just reverted r248895 (for >>> 9.3). There was inconsitent behaviour with v6 iface routes and >>> net.add_addr_allfibs=3D0. >>> Now I checked with 10.1 ans it seems net.add_addr_allfibs=3D0 doesn't w= ork >>> any more: >>> netstat -f inet -nr >>> Routing tables >>> >>> Internet: >>> Destination Gateway Flags Netif Expire >>> default 172.21.32.1 UGS egn >>> 127.0.0.1 link#2 UH lo0 >>> 172.21.32.0/19 link#1 U egn >>> 172.21.35.1 link#1 UHS lo0 >>> >>> netstat -F 1 -f inet -nr >>> Routing tables (fib: 1) >>> >>> Internet: >>> Destination Gateway Flags Netif Expire >>> 127.0.0.1 link#2 UH lo0 >>> 172.21.32.0/19 link#1 U egn >>> >>> 'sysctl net.add_addr_allfibs' >>> net.add_addr_allfibs: 0 >> Are you sure net.add_addr_allfibs was applied before interface address >> added? > > Sorry, I messed it up. Forgot that on my production systems (where I > tested), / is read-only with /etc as union-mount. > Adding net.add_addr_allfibs=3D0 to the correct sysctl.conf made the inet > routing table stay empty. > > But unfortunately not the inet6 routing table :-( > So I still need to delete iface routes for my jail setups, hence need to > revert r248895. What do your ipv6 routing tables look like when the sysctl is set correctly and 248895 is in place? > > Strage thing is that 'rcorder' shows nothing iface related before > mountcritlocal, where I resource /etc/rc.d, so the > 'net.add_addr_allfibs' in my union-mounted sysctl.conf should work!?! > But that's my homemade problem ;-) /> > > For those having similar problems, here's how I currently solve my jail > setups: > > jail.conf: > > jail { > allow.set_hostname; > ... > exec.fib =3D 1; > exec.prestart =3D "/bin/sh /.JAIL$name/etc/rc.jails_fibprepare -f > 1 -i inop"; > interface =3D inop; > ... > > =E2=80=93=E2=80=93=E2=80=93 > rc.jails_fibprepare : > > #!/bin/sh > # format FIB for JAIL usage (remove all but own interface routes) > # Does only work if on FreeBSD-9.2 if r248895 was reverted, since > deleting iface routes is prohibited by default. > # TODO: extend jail (8) and jail.conf for routing parameters and delete > this ugly hack! > # TODO: Do it the other way, not deleleting, but adding if "sysctl > net.add_addr_allfibs=3D0". > # Last edited: 20140605.0 > > > _help(){ > echo "Usage: rc.jails_fibprepare -f FIBNUM -i IFACENAME [-4 > defaultrouterIPv4] [-6 defaultrouterIPv6] [-h]" > if [ "X$1" !=3D "X" ]; then > if [ "$1" =3D "-h" ]; then > echo "Prepare routing tabel of specified FIB for jail usage." > echo "This removes all iface routes not belonging to own interface" > echo "and sets default route(s) if specified or automatically, if" > echo "iface used is the same where fib 0 has set the default gatewa= y." > echo " -f: FIBNUM is the number of the fib whose routing > table will be altered." > echo " -i: IFACENAME is the name of the interface we have > our IP on." > echo " -4: IP (v4) of the defaultrouter." > echo " -6: IP (v6) of the defaultrouter." > echo " -h: This help" > echo > else > echo "ERROR:" > echo " $1" > echo > exit 1 > fi > else > echo "Type \"rc.jails_fibprepare -h\" for more help." > exit 1 > fi > exit 0 > } > > _find_unwanted_destinations(){ > # first, generate complete destination lists (separate for v4+v6) > dest4list=3D`setfib ${fibnum} netstat -f inet -nr | grep -E > '^[[:print:]]+(%[[:alnum:].]+|[[:digit:]])[[:blank:]]+U[[:print:]]+$' | > cut -s -d ' ' -f 1` > dest6list=3D`setfib ${fibnum} netstat -f inet6 -nr | grep -E > '^[[:print:]]+(%[[:alnum:].]+|[[:digit:]])[[:blank:]]+U[[:print:]]+$' | > cut -s -d ' ' -f 1` > # Create lists with wanted destinations (separate for v4+v6) > for ifn in ${ifnames}; do > link=3D`setfib ${fibnum} netstat -I ${ifn} | sed -n -E > 's/^[[:print:]]+<[lL](ink#[[:digit:]]{1,2})>[[:print:]]+$/l\1/p'` > dest4wanted=3D"`setfib ${fibnum} netstat -f inet -nr | grep -E > '^[^[:blank:]]+[[:blank:]]+'"${link}"'[[:blank:]]+.*$' | cut -s -d ' ' > -f 1` ${dest4wanted:-}" > dest6wanted=3D"`setfib ${fibnum} netstat -f inet6 -nr | grep -E > '^[^[:blank:]]+[[:blank:]]+'"${link}"'[[:blank:]]+.*$' | cut -s -d ' ' > -f 1` ${dest6wanted:-}" > done > # remove wanted destinations from v4 list > for dest in ${dest4wanted}; do > dest4list=3D"`echo ${dest4list} | sed -E 's,'"${dest}"' *,,'`" > done > # remove wanted destinations from v6 list > for dest in ${dest6wanted}; do > dest6list=3D"`echo ${dest6list} | sed -E 's,'"${dest}"' *,,'`" > done > } > > _clean_fib(){ > _find_unwanted_destinations || return 1 > # extract default gateway IPv4 if it's on one of our interfaces and > none is set already > for ifn in ${ifnames}; do > if [ "X${dv4gw}" =3D "X" ]; then > dv4gw=3D"`netstat -f inet -nr | sed -n -E > 's/^default[[:print:]]+[[:blank:]]([^[:blank:]]+[.:][^[:blank:]]+)[[:prin= t:]]+[^[:blank:]]+[[:blank:]]+'"${ifn}"'$/\1/p'`" > fi > done > # extract default gateway IPv6 if it's on one of our interfaces and > none is set already > for ifn in ${ifnames}; do > if [ "X${dv6gw}" =3D "X" ]; then > dv6gw=3D"`netstat -f inet6 -nr | sed -n -E > 's/^default[[:print:]]+[[:blank:]]([^[:blank:]]+[.:][^[:blank:]]+)[[:prin= t:]]+[^[:blank:]]+[[:blank:]]+'"${ifn}"'$/\1/p'`" > fi > done > # remove v4 destinations > for dest in ${dest4list}; do > route -q delete -net -inet ${dest} -fib ${fibnum} || return 1 > done > # remove v6 destinations > for dest in ${dest6list}; do > route -q delete -net -inet6 ${dest} -fib ${fibnum} || return 1 > done > # Set v4 defaultrouter > if [ "X${dv4gw}" !=3D "X" ]; then > route -q add -net -inet default ${dv4gw} -fib ${fibnum} || return 1 > fi > # Set v6 defaultrouter > if [ "X${dv6gw}" !=3D "X" ]; then > route -q add -net -inet6 default ${dv6gw} -fib ${fibnum} || return 1 > fi > } > > if [ $# -gt 8 ]; then > _help "Too many arguments!" > else > if [ $# -lt 4 ]; then > _help "At least \"-f FIBUM\" and \"-i IFACENAME\" is required!" > else > if ! expr $# % 2 >/dev/null; then > while [ $# -gt 0 ]; do > case "$1" in > -f) if ! setfib ${2} true; then > _help "FIBNUM too high!" > else > fibnum=3D$2 > fi > ;; > -i) if ! ifconfig ${2} >/dev/null 2>&1; then > _help "No such interface: \"$2\"" > else > ifnames=3D"$2 ${ifnames:-}" > fi > ;; > -4) dv4gw=3D"$2";; > -6) dv6gw=3D"$2";; > -h|*) _help "$1" > esac > shift 2 > done > _clean_fib && exit 0 > else > _help "Wrong number of arguments ($#), only even numbers can be > valid!" > fi > fi > fi > exit 1 > > =E2=80=93=E2=80=93=E2=80=93 > r248895-revert patch against 10.1: > > --- src/sys/net/if.c 2014-10-06 12:56:27.000000000 +0200 > +++ src/sys/net/if.c 2014-10-13 10:47:51.000000000 +0200 > @@ -1371,8 +1371,7 @@ > return (0); > > err =3D rtrequest_fib(RTM_DELETE, rt_key(rt), rt->rt_gateway, > - rt_mask(rt), > - rt->rt_flags|RTF_RNH_LOCKED|RTF_PINNED, > + rt_mask(rt), rt->rt_flags|RTF_RNH_LOCKED, > (struct rtentry **) NULL, rt->rt_fibnum); > if (err) { > log(LOG_WARNING, "if_rtdel: error %d\n", err); > --- src/sys/net/route.c 2014-10-06 12:56:27.000000000 +0200 > +++ src/sys/net/route.c 2014-10-13 10:47:51.000000000 +0200 > @@ -1210,14 +1210,6 @@ > error =3D 0; > } > #endif > - if ((flags & RTF_PINNED) =3D=3D 0) { > - /* Check if target route can be deleted */ > - rt =3D (struct rtentry *)rnh->rnh_lookup(dst, > - netmask, rnh); > - if ((rt !=3D NULL) && (rt->rt_flags & RTF_PINNED)) > - senderr(EADDRINUSE); > - } > - > /* > * Remove the item from the tree and return it. > * Complain if it is not there and do no more processing. > @@ -1521,7 +1513,6 @@ > int didwork =3D 0; > int a_failure =3D 0; > static struct sockaddr_dl null_sdl =3D {sizeof(null_sdl), AF_LINK}; > - struct radix_node_head *rnh; > > if (flags & RTF_HOST) { > dst =3D ifa->ifa_dstaddr; > @@ -1580,6 +1571,7 @@ > */ > for ( fibnum =3D startfib; fibnum <=3D endfib; fibnum++) { > if (cmd =3D=3D RTM_DELETE) { > + struct radix_node_head *rnh; > struct radix_node *rn; > /* > * Look up an rtentry that is in the routing tree and > @@ -1626,8 +1618,7 @@ > */ > bzero((caddr_t)&info, sizeof(info)); > info.rti_ifa =3D ifa; > - info.rti_flags =3D flags | > - (ifa->ifa_flags & ~IFA_RTSELF) | RTF_PINNED; > + info.rti_flags =3D flags | (ifa->ifa_flags & ~IFA_RTSELF); > info.rti_info[RTAX_DST] =3D dst; > /* > * doing this for compatibility reasons > @@ -1639,33 +1630,6 @@ > info.rti_info[RTAX_GATEWAY] =3D ifa->ifa_addr; > info.rti_info[RTAX_NETMASK] =3D netmask; > error =3D rtrequest1_fib(cmd, &info, &rt, fibnum); > - > - if ((error =3D=3D EEXIST) && (cmd =3D=3D RTM_ADD)) { > - /* > - * Interface route addition failed. > - * Atomically delete current prefix generating > - * RTM_DELETE message, and retry adding > - * interface prefix. > - */ > - rnh =3D rt_tables_get_rnh(fibnum, dst->sa_family); > - RADIX_NODE_HEAD_LOCK(rnh); > - > - /* Delete old prefix */ > - info.rti_ifa =3D NULL; > - info.rti_flags =3D RTF_RNH_LOCKED; > - > - error =3D rtrequest1_fib(RTM_DELETE, &info, NULL, fibnum); > - if (error =3D=3D 0) { > - info.rti_ifa =3D ifa; > - info.rti_flags =3D flags | RTF_RNH_LOCKED | > - (ifa->ifa_flags & ~IFA_RTSELF) | RTF_PINNED; > - error =3D rtrequest1_fib(cmd, &info, &rt, fibnum); > - } > - > - RADIX_NODE_HEAD_UNLOCK(rnh); > - } > - > - > if (error =3D=3D 0 && rt !=3D NULL) { > /* > * notify any listening routing agents of the change > --- src/sys/net/route.h 2014-10-06 12:56:27.000000000 +0200 > +++ src/sys/net/route.h 2014-10-13 10:43:59.000000000 +0200 > @@ -148,7 +148,7 @@ > /* 0x20000 unused, was RTF_WASCLONED */ > #define RTF_PROTO3 0x40000 /* protocol specific routing flag *= / > /* 0x80000 unused */ > -#define RTF_PINNED 0x100000 /* route is immutable */ > +#define RTF_PINNED 0x100000 /* future use (route is immutable, > startintg with r248895) */ > #define RTF_LOCAL 0x200000 /* route represents a local address= */ > #define RTF_BROADCAST 0x400000 /* route represents a bcast > address */ > #define RTF_MULTICAST 0x800000 /* route represents a mcast > address */ > >