From owner-freebsd-stable@FreeBSD.ORG Mon Oct 13 09:22:46 2014 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ADAA1EB1; Mon, 13 Oct 2014 09:22:46 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2DD9D925; Mon, 13 Oct 2014 09:22:46 +0000 (UTC) Received: from [2a02:6b8:0:401:222:4dff:fe50:cd2f] (helo=ptichko.yndx.net) by mail.ipfw.ru with esmtpsa (TLSv1:DHE-RSA-AES128-SHA:128) (Exim 4.82 (FreeBSD)) (envelope-from ) id 1XdXqK-000FRi-Bp; Mon, 13 Oct 2014 09:06:48 +0400 Message-ID: <543B998D.2020003@FreeBSD.org> Date: Mon, 13 Oct 2014 13:21:17 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.1.2 MIME-Version: 1.0 To: Harald Schmalzbauer Subject: Re: Deleting IPv4 iface-routes from extra FIBs References: <53569ABA.60007@omnilan.de> <535771F3.4070007@freebsd.org> <543B8ED5.6040206@omnilan.de> <543B9075.2000102@FreeBSD.org> <543B9873.3040605@omnilan.de> In-Reply-To: <543B9873.3040605@omnilan.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Cc: "freebsd-net@freebsd.org" , Julian Elischer , FreeBSD X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Oct 2014 09:22:46 -0000 On 13.10.2014 13:16, Harald Schmalzbauer wrote: > Bezüglich Alexander V. Chernikov's Nachricht vom 13.10.2014 10:42 > (localtime): >> On 13.10.2014 12:35, Harald Schmalzbauer wrote: >>> Bezüglich Julian Elischer's Nachricht vom 23.04.2014 09:55 >>> (localtime): > ... >>>> yes, we made two behaviours. >>>> Add interface routes to all active FIBS or only add them to the first >>>> fib and let the user populate other fibs as needed. >>>> It appears you want the second behaviour, so I suggest you use that >>>> option and set up all your routes manually. >>> Hello, >>> >>> last time I had the iface-route problem, I just reverted r248895 (for >>> 9.3). There was inconsitent behaviour with v6 iface routes and >>> net.add_addr_allfibs=0. >>> Now I checked with 10.1 ans it seems net.add_addr_allfibs=0 doesn't work >>> any more: >>> netstat -f inet -nr >>> Routing tables >>> >>> Internet: >>> Destination Gateway Flags Netif Expire >>> default 172.21.32.1 UGS egn >>> 127.0.0.1 link#2 UH lo0 >>> 172.21.32.0/19 link#1 U egn >>> 172.21.35.1 link#1 UHS lo0 >>> >>> netstat -F 1 -f inet -nr >>> Routing tables (fib: 1) >>> >>> Internet: >>> Destination Gateway Flags Netif Expire >>> 127.0.0.1 link#2 UH lo0 >>> 172.21.32.0/19 link#1 U egn >>> >>> 'sysctl net.add_addr_allfibs' >>> net.add_addr_allfibs: 0 >> Are you sure net.add_addr_allfibs was applied before interface address >> added? > Sorry, I messed it up. Forgot that on my production systems (where I > tested), / is read-only with /etc as union-mount. > Adding net.add_addr_allfibs=0 to the correct sysctl.conf made the inet > routing table stay empty. > > But unfortunately not the inet6 routing table :-( > So I still need to delete iface routes for my jail setups, hence need to > revert r248895. Hm. If the problem happens with inet6 routes only, why do you need to revert r248895 ? > > Strage thing is that 'rcorder' shows nothing iface related before > mountcritlocal, where I resource /etc/rc.d, so the > 'net.add_addr_allfibs' in my union-mounted sysctl.conf should work!?! > But that's my homemade problem ;-) /> > > For those having similar problems, here's how I currently solve my jail > setups: > > jail.conf: > > jail { > allow.set_hostname; > ... > exec.fib = 1; > exec.prestart = "/bin/sh /.JAIL$name/etc/rc.jails_fibprepare -f > 1 -i inop"; > interface = inop; > ... > > ––– > rc.jails_fibprepare : > > #!/bin/sh > # format FIB for JAIL usage (remove all but own interface routes) > # Does only work if on FreeBSD-9.2 if r248895 was reverted, since > deleting iface routes is prohibited by default. > # TODO: extend jail (8) and jail.conf for routing parameters and delete > this ugly hack! > # TODO: Do it the other way, not deleleting, but adding if "sysctl > net.add_addr_allfibs=0". > # Last edited: 20140605.0 > > > _help(){ > echo "Usage: rc.jails_fibprepare -f FIBNUM -i IFACENAME [-4 > defaultrouterIPv4] [-6 defaultrouterIPv6] [-h]" > if [ "X$1" != "X" ]; then > if [ "$1" = "-h" ]; then > echo "Prepare routing tabel of specified FIB for jail usage." > echo "This removes all iface routes not belonging to own interface" > echo "and sets default route(s) if specified or automatically, if" > echo "iface used is the same where fib 0 has set the default gateway." > echo " -f: FIBNUM is the number of the fib whose routing > table will be altered." > echo " -i: IFACENAME is the name of the interface we have > our IP on." > echo " -4: IP (v4) of the defaultrouter." > echo " -6: IP (v6) of the defaultrouter." > echo " -h: This help" > echo > else > echo "ERROR:" > echo " $1" > echo > exit 1 > fi > else > echo "Type \"rc.jails_fibprepare -h\" for more help." > exit 1 > fi > exit 0 > } > > _find_unwanted_destinations(){ > # first, generate complete destination lists (separate for v4+v6) > dest4list=`setfib ${fibnum} netstat -f inet -nr | grep -E > '^[[:print:]]+(%[[:alnum:].]+|[[:digit:]])[[:blank:]]+U[[:print:]]+$' | > cut -s -d ' ' -f 1` > dest6list=`setfib ${fibnum} netstat -f inet6 -nr | grep -E > '^[[:print:]]+(%[[:alnum:].]+|[[:digit:]])[[:blank:]]+U[[:print:]]+$' | > cut -s -d ' ' -f 1` > # Create lists with wanted destinations (separate for v4+v6) > for ifn in ${ifnames}; do > link=`setfib ${fibnum} netstat -I ${ifn} | sed -n -E > 's/^[[:print:]]+<[lL](ink#[[:digit:]]{1,2})>[[:print:]]+$/l\1/p'` > dest4wanted="`setfib ${fibnum} netstat -f inet -nr | grep -E > '^[^[:blank:]]+[[:blank:]]+'"${link}"'[[:blank:]]+.*$' | cut -s -d '' > -f 1` ${dest4wanted:-}" > dest6wanted="`setfib ${fibnum} netstat -f inet6 -nr | grep -E > '^[^[:blank:]]+[[:blank:]]+'"${link}"'[[:blank:]]+.*$' | cut -s -d '' > -f 1` ${dest6wanted:-}" > done > # remove wanted destinations from v4 list > for dest in ${dest4wanted}; do > dest4list="`echo ${dest4list} | sed -E 's,'"${dest}"' *,,'`" > done > # remove wanted destinations from v6 list > for dest in ${dest6wanted}; do > dest6list="`echo ${dest6list} | sed -E 's,'"${dest}"' *,,'`" > done > } > > _clean_fib(){ > _find_unwanted_destinations || return 1 > # extract default gateway IPv4 if it's on one of our interfaces and > none is set already > for ifn in ${ifnames}; do > if [ "X${dv4gw}" = "X" ]; then > dv4gw="`netstat -f inet -nr | sed -n -E > 's/^default[[:print:]]+[[:blank:]]([^[:blank:]]+[.:][^[:blank:]]+)[[:print:]]+[^[:blank:]]+[[:blank:]]+'"${ifn}"'$/\1/p'`" > fi > done > # extract default gateway IPv6 if it's on one of our interfaces and > none is set already > for ifn in ${ifnames}; do > if [ "X${dv6gw}" = "X" ]; then > dv6gw="`netstat -f inet6 -nr | sed -n -E > 's/^default[[:print:]]+[[:blank:]]([^[:blank:]]+[.:][^[:blank:]]+)[[:print:]]+[^[:blank:]]+[[:blank:]]+'"${ifn}"'$/\1/p'`" > fi > done > # remove v4 destinations > for dest in ${dest4list}; do > route -q delete -net -inet ${dest} -fib ${fibnum} || return 1 > done > # remove v6 destinations > for dest in ${dest6list}; do > route -q delete -net -inet6 ${dest} -fib ${fibnum} || return 1 > done > # Set v4 defaultrouter > if [ "X${dv4gw}" != "X" ]; then > route -q add -net -inet default ${dv4gw} -fib ${fibnum} || return 1 > fi > # Set v6 defaultrouter > if [ "X${dv6gw}" != "X" ]; then > route -q add -net -inet6 default ${dv6gw} -fib ${fibnum} || return 1 > fi > } > > if [ $# -gt 8 ]; then > _help "Too many arguments!" > else > if [ $# -lt 4 ]; then > _help "At least \"-f FIBUM\" and \"-i IFACENAME\" is required!" > else > if ! expr $# % 2 >/dev/null; then > while [ $# -gt 0 ]; do > case "$1" in > -f) if ! setfib ${2} true; then > _help "FIBNUM too high!" > else > fibnum=$2 > fi > ;; > -i) if ! ifconfig ${2} >/dev/null 2>&1; then > _help "No such interface: \"$2\"" > else > ifnames="$2 ${ifnames:-}" > fi > ;; > -4) dv4gw="$2";; > -6) dv6gw="$2";; > -h|*) _help "$1" > esac > shift 2 > done > _clean_fib && exit 0 > else > _help "Wrong number of arguments ($#), only even numbers can be > valid!" > fi > fi > fi > exit 1 > > ––– > r248895-revert patch against 10.1: > > --- src/sys/net/if.c 2014-10-06 12:56:27.000000000 +0200 > +++ src/sys/net/if.c 2014-10-13 10:47:51.000000000 +0200 > @@ -1371,8 +1371,7 @@ > return (0); > > err = rtrequest_fib(RTM_DELETE, rt_key(rt), rt->rt_gateway, > - rt_mask(rt), > - rt->rt_flags|RTF_RNH_LOCKED|RTF_PINNED, > + rt_mask(rt), rt->rt_flags|RTF_RNH_LOCKED, > (struct rtentry **) NULL, rt->rt_fibnum); > if (err) { > log(LOG_WARNING, "if_rtdel: error %d\n", err); > --- src/sys/net/route.c 2014-10-06 12:56:27.000000000 +0200 > +++ src/sys/net/route.c 2014-10-13 10:47:51.000000000 +0200 > @@ -1210,14 +1210,6 @@ > error = 0; > } > #endif > - if ((flags & RTF_PINNED) == 0) { > - /* Check if target route can be deleted */ > - rt = (struct rtentry *)rnh->rnh_lookup(dst, > - netmask, rnh); > - if ((rt != NULL) && (rt->rt_flags & RTF_PINNED)) > - senderr(EADDRINUSE); > - } > - > /* > * Remove the item from the tree and return it. > * Complain if it is not there and do no more processing. > @@ -1521,7 +1513,6 @@ > int didwork = 0; > int a_failure = 0; > static struct sockaddr_dl null_sdl = {sizeof(null_sdl), AF_LINK}; > - struct radix_node_head *rnh; > > if (flags & RTF_HOST) { > dst = ifa->ifa_dstaddr; > @@ -1580,6 +1571,7 @@ > */ > for ( fibnum = startfib; fibnum <= endfib; fibnum++) { > if (cmd == RTM_DELETE) { > + struct radix_node_head *rnh; > struct radix_node *rn; > /* > * Look up an rtentry that is in the routing tree and > @@ -1626,8 +1618,7 @@ > */ > bzero((caddr_t)&info, sizeof(info)); > info.rti_ifa = ifa; > - info.rti_flags = flags | > - (ifa->ifa_flags & ~IFA_RTSELF) | RTF_PINNED; > + info.rti_flags = flags | (ifa->ifa_flags & ~IFA_RTSELF); > info.rti_info[RTAX_DST] = dst; > /* > * doing this for compatibility reasons > @@ -1639,33 +1630,6 @@ > info.rti_info[RTAX_GATEWAY] = ifa->ifa_addr; > info.rti_info[RTAX_NETMASK] = netmask; > error = rtrequest1_fib(cmd, &info, &rt, fibnum); > - > - if ((error == EEXIST) && (cmd == RTM_ADD)) { > - /* > - * Interface route addition failed. > - * Atomically delete current prefix generating > - * RTM_DELETE message, and retry adding > - * interface prefix. > - */ > - rnh = rt_tables_get_rnh(fibnum, dst->sa_family); > - RADIX_NODE_HEAD_LOCK(rnh); > - > - /* Delete old prefix */ > - info.rti_ifa = NULL; > - info.rti_flags = RTF_RNH_LOCKED; > - > - error = rtrequest1_fib(RTM_DELETE, &info, NULL, fibnum); > - if (error == 0) { > - info.rti_ifa = ifa; > - info.rti_flags = flags | RTF_RNH_LOCKED | > - (ifa->ifa_flags & ~IFA_RTSELF) | RTF_PINNED; > - error = rtrequest1_fib(cmd, &info, &rt, fibnum); > - } > - > - RADIX_NODE_HEAD_UNLOCK(rnh); > - } > - > - > if (error == 0 && rt != NULL) { > /* > * notify any listening routing agents of the change > --- src/sys/net/route.h 2014-10-06 12:56:27.000000000 +0200 > +++ src/sys/net/route.h 2014-10-13 10:43:59.000000000 +0200 > @@ -148,7 +148,7 @@ > /* 0x20000 unused, was RTF_WASCLONED */ > #define RTF_PROTO3 0x40000 /* protocol specific routing flag */ > /* 0x80000 unused */ > -#define RTF_PINNED 0x100000 /* route is immutable */ > +#define RTF_PINNED 0x100000 /* future use (route is immutable, > startintg with r248895) */ > #define RTF_LOCAL 0x200000 /* route represents a local address */ > #define RTF_BROADCAST 0x400000 /* route represents a bcast > address */ > #define RTF_MULTICAST 0x800000 /* route represents a mcast > address */ > >