Date: Mon, 13 Oct 2014 11:16:35 +0200 From: Harald Schmalzbauer <h.schmalzbauer@omnilan.de> To: "Alexander V. Chernikov" <melifaro@FreeBSD.org> Cc: "freebsd-net@freebsd.org" <freebsd-net@FreeBSD.org>, Julian Elischer <julian@FreeBSD.org>, FreeBSD <freebsd-stable@FreeBSD.org> Subject: Re: Deleting IPv4 iface-routes from extra FIBs Message-ID: <543B9873.3040605@omnilan.de> In-Reply-To: <543B9075.2000102@FreeBSD.org> References: <53569ABA.60007@omnilan.de> <CA%2BP_MZH_iScuJ4S=xiKocnEwTzT1eRJPNpJKbboZDfG3B=TBzA@mail.gmail.com> <535771F3.4070007@freebsd.org> <543B8ED5.6040206@omnilan.de> <543B9075.2000102@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enigDFEC5242399331AD23FB8855
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Bez=C3=BCglich Alexander V. Chernikov's Nachricht vom 13.10.2014 10:42
(localtime):
> On 13.10.2014 12:35, Harald Schmalzbauer wrote:
>> Bez=C3=BCglich Julian Elischer's Nachricht vom 23.04.2014 09:55
>> (localtime):
=2E..
>>> yes, we made two behaviours.
>>> Add interface routes to all active FIBS or only add them to the first=
>>> fib and let the user populate other fibs as needed.
>>> It appears you want the second behaviour, so I suggest you use that
>>> option and set up all your routes manually.
>> Hello,
>>
>> last time I had the iface-route problem, I just reverted r248895 (for
>> 9.3). There was inconsitent behaviour with v6 iface routes and
>> net.add_addr_allfibs=3D0.
>> Now I checked with 10.1 ans it seems net.add_addr_allfibs=3D0 doesn't =
work
>> any more:
>> netstat -f inet -nr
>> Routing tables
>>
>> Internet:
>> Destination Gateway Flags Netif Expire
>> default 172.21.32.1 UGS egn
>> 127.0.0.1 link#2 UH lo0
>> 172.21.32.0/19 link#1 U egn
>> 172.21.35.1 link#1 UHS lo0
>>
>> netstat -F 1 -f inet -nr
>> Routing tables (fib: 1)
>>
>> Internet:
>> Destination Gateway Flags Netif Expire
>> 127.0.0.1 link#2 UH lo0
>> 172.21.32.0/19 link#1 U egn
>>
>> 'sysctl net.add_addr_allfibs'
>> net.add_addr_allfibs: 0
> Are you sure net.add_addr_allfibs was applied before interface address
> added?
Sorry, I messed it up. Forgot that on my production systems (where I
tested), / is read-only with /etc as union-mount.
Adding net.add_addr_allfibs=3D0 to the correct sysctl.conf made the inet
routing table stay empty.
But unfortunately not the inet6 routing table :-(
So I still need to delete iface routes for my jail setups, hence need to
revert r248895.
<Unrelated to the actual problem:
Strage thing is that 'rcorder' shows nothing iface related before
mountcritlocal, where I resource /etc/rc.d, so the
'net.add_addr_allfibs' in my union-mounted sysctl.conf should work!?!
But that's my homemade problem ;-) />
For those having similar problems, here's how I currently solve my jail
setups:
jail.conf:
jail {
allow.set_hostname;
=2E..
exec.fib =3D 1;
exec.prestart =3D "/bin/sh /.JAIL$name/etc/rc.jails_fibprepare -f=
1 -i inop";
interface =3D inop;
=2E..
=E2=80=93=E2=80=93=E2=80=93
rc.jails_fibprepare :
#!/bin/sh
# format FIB for JAIL usage (remove all but own interface routes)
# Does only work if on FreeBSD-9.2 if r248895 was reverted, since
deleting iface routes is prohibited by default.
# TODO: extend jail (8) and jail.conf for routing parameters and delete
this ugly hack!
# TODO: Do it the other way, not deleleting, but adding if "sysctl
net.add_addr_allfibs=3D0".
# Last edited: 20140605.0
_help(){
echo "Usage: rc.jails_fibprepare -f FIBNUM -i IFACENAME [-4
defaultrouterIPv4] [-6 defaultrouterIPv6] [-h]"
if [ "X$1" !=3D "X" ]; then
if [ "$1" =3D "-h" ]; then
echo "Prepare routing tabel of specified FIB for jail usage."
echo "This removes all iface routes not belonging to own interface"=
echo "and sets default route(s) if specified or automatically, if"
echo "iface used is the same where fib 0 has set the default gatewa=
y."
echo " -f: FIBNUM is the number of the fib whose routing
table will be altered."
echo " -i: IFACENAME is the name of the interface we have
our IP on."
echo " -4: IP (v4) of the defaultrouter."
echo " -6: IP (v6) of the defaultrouter."
echo " -h: This help"
echo
else
echo "ERROR:"
echo " $1"
echo
exit 1
fi
else
echo "Type \"rc.jails_fibprepare -h\" for more help."
exit 1
fi
exit 0
}
_find_unwanted_destinations(){
# first, generate complete destination lists (separate for v4+v6)
dest4list=3D`setfib ${fibnum} netstat -f inet -nr | grep -E
'^[[:print:]]+(%[[:alnum:].]+|[[:digit:]])[[:blank:]]+U[[:print:]]+$' |
cut -s -d ' ' -f 1`
dest6list=3D`setfib ${fibnum} netstat -f inet6 -nr | grep -E
'^[[:print:]]+(%[[:alnum:].]+|[[:digit:]])[[:blank:]]+U[[:print:]]+$' |
cut -s -d ' ' -f 1`
# Create lists with wanted destinations (separate for v4+v6)
for ifn in ${ifnames}; do
link=3D`setfib ${fibnum} netstat -I ${ifn} | sed -n -E
's/^[[:print:]]+<[lL](ink#[[:digit:]]{1,2})>[[:print:]]+$/l\1/p'`
dest4wanted=3D"`setfib ${fibnum} netstat -f inet -nr | grep -E
'^[^[:blank:]]+[[:blank:]]+'"${link}"'[[:blank:]]+.*$' | cut -s -d ' '
-f 1` ${dest4wanted:-}"
dest6wanted=3D"`setfib ${fibnum} netstat -f inet6 -nr | grep -E
'^[^[:blank:]]+[[:blank:]]+'"${link}"'[[:blank:]]+.*$' | cut -s -d ' '
-f 1` ${dest6wanted:-}"
done
# remove wanted destinations from v4 list
for dest in ${dest4wanted}; do
dest4list=3D"`echo ${dest4list} | sed -E 's,'"${dest}"' *,,'`"
done
# remove wanted destinations from v6 list
for dest in ${dest6wanted}; do
dest6list=3D"`echo ${dest6list} | sed -E 's,'"${dest}"' *,,'`"
done
}
_clean_fib(){
_find_unwanted_destinations || return 1
# extract default gateway IPv4 if it's on one of our interfaces and
none is set already
for ifn in ${ifnames}; do
if [ "X${dv4gw}" =3D "X" ]; then
dv4gw=3D"`netstat -f inet -nr | sed -n -E
's/^default[[:print:]]+[[:blank:]]([^[:blank:]]+[.:][^[:blank:]]+)[[:prin=
t:]]+[^[:blank:]]+[[:blank:]]+'"${ifn}"'$/\1/p'`"
fi
done
# extract default gateway IPv6 if it's on one of our interfaces and
none is set already
for ifn in ${ifnames}; do
if [ "X${dv6gw}" =3D "X" ]; then
dv6gw=3D"`netstat -f inet6 -nr | sed -n -E
's/^default[[:print:]]+[[:blank:]]([^[:blank:]]+[.:][^[:blank:]]+)[[:prin=
t:]]+[^[:blank:]]+[[:blank:]]+'"${ifn}"'$/\1/p'`"
fi
done
# remove v4 destinations
for dest in ${dest4list}; do
route -q delete -net -inet ${dest} -fib ${fibnum} || return 1
done
# remove v6 destinations
for dest in ${dest6list}; do
route -q delete -net -inet6 ${dest} -fib ${fibnum} || return 1
done
# Set v4 defaultrouter
if [ "X${dv4gw}" !=3D "X" ]; then
route -q add -net -inet default ${dv4gw} -fib ${fibnum} || return 1
fi
# Set v6 defaultrouter
if [ "X${dv6gw}" !=3D "X" ]; then
route -q add -net -inet6 default ${dv6gw} -fib ${fibnum} || return 1
fi
}
if [ $# -gt 8 ]; then
_help "Too many arguments!"
else
if [ $# -lt 4 ]; then
_help "At least \"-f FIBUM\" and \"-i IFACENAME\" is required!"
else
if ! expr $# % 2 >/dev/null; then
while [ $# -gt 0 ]; do
case "$1" in
-f) if ! setfib ${2} true; then
_help "FIBNUM too high!"
else
fibnum=3D$2
fi
;;
-i) if ! ifconfig ${2} >/dev/null 2>&1; then
_help "No such interface: \"$2\""
else
ifnames=3D"$2 ${ifnames:-}"
fi
;;
-4) dv4gw=3D"$2";;
-6) dv6gw=3D"$2";;
-h|*) _help "$1"
esac
shift 2
done
_clean_fib && exit 0
else
_help "Wrong number of arguments ($#), only even numbers can be
valid!"
fi
fi
fi
exit 1
=E2=80=93=E2=80=93=E2=80=93
r248895-revert patch against 10.1:
--- src/sys/net/if.c 2014-10-06 12:56:27.000000000 +0200
+++ src/sys/net/if.c 2014-10-13 10:47:51.000000000 +0200
@@ -1371,8 +1371,7 @@
return (0);
=20
err =3D rtrequest_fib(RTM_DELETE, rt_key(rt), rt->rt_gateway,
- rt_mask(rt),
- rt->rt_flags|RTF_RNH_LOCKED|RTF_PINNED,
+ rt_mask(rt), rt->rt_flags|RTF_RNH_LOCKED,
(struct rtentry **) NULL, rt->rt_fibnum);
if (err) {
log(LOG_WARNING, "if_rtdel: error %d\n", err);
--- src/sys/net/route.c 2014-10-06 12:56:27.000000000 +0200
+++ src/sys/net/route.c 2014-10-13 10:47:51.000000000 +0200
@@ -1210,14 +1210,6 @@
error =3D 0;
}
#endif
- if ((flags & RTF_PINNED) =3D=3D 0) {
- /* Check if target route can be deleted */
- rt =3D (struct rtentry *)rnh->rnh_lookup(dst,
- netmask, rnh);
- if ((rt !=3D NULL) && (rt->rt_flags & RTF_PINNED))
- senderr(EADDRINUSE);
- }
-
/*
* Remove the item from the tree and return it.
* Complain if it is not there and do no more processing.
@@ -1521,7 +1513,6 @@
int didwork =3D 0;
int a_failure =3D 0;
static struct sockaddr_dl null_sdl =3D {sizeof(null_sdl), AF_LINK};
- struct radix_node_head *rnh;
=20
if (flags & RTF_HOST) {
dst =3D ifa->ifa_dstaddr;
@@ -1580,6 +1571,7 @@
*/
for ( fibnum =3D startfib; fibnum <=3D endfib; fibnum++) {
if (cmd =3D=3D RTM_DELETE) {
+ struct radix_node_head *rnh;
struct radix_node *rn;
/*
* Look up an rtentry that is in the routing tree and
@@ -1626,8 +1618,7 @@
*/
bzero((caddr_t)&info, sizeof(info));
info.rti_ifa =3D ifa;
- info.rti_flags =3D flags |
- (ifa->ifa_flags & ~IFA_RTSELF) | RTF_PINNED;
+ info.rti_flags =3D flags | (ifa->ifa_flags & ~IFA_RTSELF);
info.rti_info[RTAX_DST] =3D dst;
/*
* doing this for compatibility reasons
@@ -1639,33 +1630,6 @@
info.rti_info[RTAX_GATEWAY] =3D ifa->ifa_addr;
info.rti_info[RTAX_NETMASK] =3D netmask;
error =3D rtrequest1_fib(cmd, &info, &rt, fibnum);
-
- if ((error =3D=3D EEXIST) && (cmd =3D=3D RTM_ADD)) {
- /*
- * Interface route addition failed.
- * Atomically delete current prefix generating
- * RTM_DELETE message, and retry adding
- * interface prefix.
- */
- rnh =3D rt_tables_get_rnh(fibnum, dst->sa_family);
- RADIX_NODE_HEAD_LOCK(rnh);
-
- /* Delete old prefix */
- info.rti_ifa =3D NULL;
- info.rti_flags =3D RTF_RNH_LOCKED;
-
- error =3D rtrequest1_fib(RTM_DELETE, &info, NULL, fibnum);
- if (error =3D=3D 0) {
- info.rti_ifa =3D ifa;
- info.rti_flags =3D flags | RTF_RNH_LOCKED |
- (ifa->ifa_flags & ~IFA_RTSELF) | RTF_PINNED;
- error =3D rtrequest1_fib(cmd, &info, &rt, fibnum);
- }
-
- RADIX_NODE_HEAD_UNLOCK(rnh);
- }
-
-
if (error =3D=3D 0 && rt !=3D NULL) {
/*
* notify any listening routing agents of the change
--- src/sys/net/route.h 2014-10-06 12:56:27.000000000 +0200
+++ src/sys/net/route.h 2014-10-13 10:43:59.000000000 +0200
@@ -148,7 +148,7 @@
/* 0x20000 unused, was RTF_WASCLONED */
#define RTF_PROTO3 0x40000 /* protocol specific routing flag *=
/
/* 0x80000 unused */
-#define RTF_PINNED 0x100000 /* route is immutable */
+#define RTF_PINNED 0x100000 /* future use (route is immutable,
startintg with r248895) */
#define RTF_LOCAL 0x200000 /* route represents a local address=
*/
#define RTF_BROADCAST 0x400000 /* route represents a bcast
address */
#define RTF_MULTICAST 0x800000 /* route represents a mcast
address */
--------------enigDFEC5242399331AD23FB8855
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (FreeBSD)
iEYEARECAAYFAlQ7mHMACgkQLDqVQ9VXb8gOegCfXiznyHCmkyRMosVBO5uIUlzB
2yQAoKWEezWtKKwXzoBveGim6cb/E6y8
=10vS
-----END PGP SIGNATURE-----
--------------enigDFEC5242399331AD23FB8855--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?543B9873.3040605>