Date: Mon, 14 Jan 2019 16:53:31 -0500 From: Mark Saad <nonesuch@longcount.org> To: FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Re: Removing an alias can remove routes ? Message-ID: <CAMXt9NazdCf5XZoSvu%2Bc4g7hfgbuaW=0EDrvBgmyF98YSyuy=w@mail.gmail.com> In-Reply-To: <CAMXt9NbV%2Bh8tjugcss%2BdafbUOT2o6bh2eWbBFAKNoJ9GeYs1Cw@mail.gmail.com> References: <CAMXt9NZ8VtTETH0_pRBeHZ7P%2Bgocpxa3EypWuJ6P=b8-%2B3VgqA@mail.gmail.com> <201901141821.x0EILVem036356@pdx.rh.CN85.dnsmgr.net> <CAMXt9NbV%2Bh8tjugcss%2BdafbUOT2o6bh2eWbBFAKNoJ9GeYs1Cw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Jan 14, 2019 at 4:48 PM Mark Saad <nonesuch@longcount.org> wrote: > > On Mon, Jan 14, 2019 at 1:21 PM Rodney W. Grimes > <freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote: > > > > > On Mon, Jan 14, 2019 at 11:59 AM Rodney W. Grimes > > > <freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote: > > > > > > > > > All > > > > > I ran into a peculiar over the weekend on two FreeBSD 10-STABLE > > > > > servers that has me at a loss. Both servers have the same setup lacp > > > > > lagg wtih vlans using the lagg as a parent dev. > > > > > On the vlans I have some alias along with the primary ip. When a > > > > > alias was added or deleted routes that sourced out of the primary ip > > > > > on that vlan were dropped from the routing table. > > > > > > > > > > > > > > > So for example. I have lagg0.vlan1044 . (both servers are similarly > > > > > configured as routers / firewalls ) > > > > > > > > > > [root@pineapple01 ~]# ifconfig lagg0.vlan1044 > > > > > lagg0.vlan1044: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> > > > > > metric 0 mtu 1500 > > > > > options=300<TSO4,TSO6> > > > > > ether 00:0f:53:20:9b:d0 > > > > > inet6 fe80::20f:53ff:fe20:9bd0%lagg0.vlan1044 prefixlen 64 > > > > > scopeid 0x80 > > > > > inet 10.24.213.84 netmask 0xffffffe0 broadcast 10.24.213.95 > > > > > inet 10.24.212.129 netmask 0xffffffff broadcast 10.24.212.129 > > > > > inet 10.24.213.163 netmask 0xfffffff0 broadcast 10.24.213.175 > > > > > inet 10.24.213.76 netmask 0xffffffe0 broadcast 10.24.213.95 > > > > > inet 10.24.213.94 netmask 0xffffffe0 broadcast 10.24.213.95 > > > > > inet 10.24.213.89 netmask 0xffffffe0 broadcast 10.24.213.95 > > > > > inet 10.24.213.75 netmask 0xffffffe0 broadcast 10.24.213.95 > > > > > nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> > > > > > media: Ethernet autoselect > > > > > status: active > > > > > vlan: 1044 parent interface: lagg0 > > > > > > > > > > [root@pineapple01 ~]# netstat -nr4Wl > > > > > ... > > > > > 192.168.144.32/27 10.24.213.65 UGS 0 1500 lagg0.vlan1044 > > > > > 192.168.144.96/27 10.24.213.65 UGS 0 1500 lagg0.vlan1044 > > > > > 192.168.23.0/24 10.24.213.65 UGS 0 1500 lagg0.vlan1044 > > > > > 192.168.120.0/21 10.24.213.65 UGS 0 1500 lagg0.vlan1044 > > > > > > > > > > > > > > > So I wanted to remove the alias ended in 163 and fix its netmask back to /32 > > > > > > > > > > I ran this > > > > > > > > > > ifconfig lagg0.vlan1044 inet 10.24.213.163/24 -alias && ifconfig > > > > ^ delete? > > > > > > I use -alias which is an alias of -delete > > > > > > > > > > > > lagg0.vlan1044 inet 10.24.213.163/32 -alias > > > > This is also a delete? > > I am concerend that what you think the command you > > typed is not actually the command you typed and what > > you did actually type has the bad side effects. > > > > > > > > > and shortly there after all of the routes that went out lagg0.vlan1044 > > > > > were gone . I quickly undid my change and put the routes back but I am > > > > > not sure what caused this ? Anyone have any ideas I have done this in > > > > > the past with out issue and I am unsure whats changed other then the > > > > > box have a long up time of 463 days . > > > > > > > > I believe what happened here is that 10.24.213.163/24 when reduced > > > > to a network address is 10.24.213.0/24, which probably got sent to > > > > the route removal code, which since the route to that covers the > > > > gateway at 10.24.213.65 that gateway was no longer accessable > > > > so all routes via it got removed. > > > > > > > > > > ok I see what you are saying and it makes sense to me; do you know why > > > the routing code does not see the primary ip ( the non-alias one) is > > > still > > > live on the interface before deciding to drop the routes associated with it ? > > > > It should, but I can not see enough of your routing table to > > guess as to exactly what the kernel did when you removed > > this interface. The routes I see above all would go through > > the interface you showed you removed, thus they would all > > go away. > > > > -- > > Rod Grimes rgrimes@freebsd.org > > > Ok some further testing shows what appears to be unintended results. > > On my test box I do the following > > ifconfig vlan98 create > ifconfig vlan98 vlan 98 vlandev lagg0 > ifconfig vlan98 inet 10.1.68.12/26 > ifconfig vlan98 inet 10.1.68.13/26 alias > ifconfig vlan98 inet 10.1.68.14/28 alias > ifconfig vlan98 inet 10.1.68.15/32 alias > > route add 10.24.213.0/24 10.1.68.11 > route add 10.24.214.0/24 10.1.68.11 > route add 10.24.215.0/24 10.1.68.11 > > root@potato2:~ # netstat -nr4Wl > Routing tables > > Internet: > Destination Gateway Flags Use Mtu Netif Expire > default 10.21.160.1 UGS 1216 1500 igb0 > 10.1.68.0/26 link#12 U 0 1500 vlan98 > 10.1.68.12 link#12 UHS 0 16384 lo0 > 10.1.68.13 link#12 UHS 0 16384 lo0 > 10.1.68.13/32 link#12 U 0 1500 vlan98 > 10.1.68.14 link#12 UHS 0 16384 lo0 > 10.1.68.14/32 link#12 U 0 1500 vlan98 > 10.1.68.15 link#12 UHS 0 16384 lo0 > 10.1.68.15/32 link#12 U 0 1500 vlan98 > 10.21.160.0/21 link#3 U 26985 1500 igb0 > 10.21.160.85 link#3 UHS 0 16384 lo0 > 10.24.213.0/24 10.1.68.11 UGS 0 1500 vlan98 > 10.24.214.0/24 10.1.68.11 UGS 0 1500 vlan98 > 10.24.215.0/24 10.1.68.11 UGS 0 1500 vlan98 > 127.0.0.1 link#5 UH 840 16384 lo0 > > Then I realize that my masks are wrong and I change the aliases back to "/32's" > > root@potato2:~ # ifconfig vlan98 > vlan98: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 > ether 00:0f:53:20:9d:00 > inet 10.1.68.12 netmask 0xffffffc0 broadcast 10.1.68.63 > inet6 fe80::20f:53ff:fe20:9d00%vlan98 prefixlen 64 scopeid 0xc > inet 10.1.68.13 netmask 0xffffffff broadcast 10.1.68.13 > inet 10.1.68.14 netmask 0xffffffff broadcast 10.1.68.14 > inet 10.1.68.15 netmask 0xffffffff broadcast 10.1.68.15 > nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> > media: Ethernet autoselect > status: active > vlan: 98 parent interface: lagg0 > root@potato2:~ # netstat -nr4Wl > Routing tables > > Internet: > Destination Gateway Flags Use Mtu Netif Expire > default 10.21.160.1 UGS 1217 1500 igb0 > 10.1.68.0/26 link#12 U 0 1500 vlan98 > 10.1.68.12 link#12 UHS 0 16384 lo0 > 10.1.68.13 link#12 UHS 0 16384 lo0 > 10.1.68.13/32 link#12 U 0 1500 vlan98 > 10.1.68.14 link#12 UHS 0 16384 lo0 > 10.1.68.14/32 link#12 U 0 1500 vlan98 > 10.1.68.15 link#12 UHS 0 16384 lo0 > 10.1.68.15/32 link#12 U 0 1500 vlan98 > 10.21.160.0/21 link#3 U 27166 1500 igb0 > 10.21.160.85 link#3 UHS 0 16384 lo0 > 10.24.213.0/24 10.1.68.11 UGS 0 1500 vlan98 > 10.24.214.0/24 10.1.68.11 UGS 0 1500 vlan98 > 10.24.215.0/24 10.1.68.11 UGS 0 1500 vlan98 > 127.0.0.1 link#5 UH 840 16384 lo0 > root@potato2:~ # > > So far no problems. > > Then I see ohh my primary ip needs to be a "/24" > > > root@potato2~ # ifconfig vlan98 inet 10.1.68.12/24 > root@potato2:~ # netstat -nr4Wl > Routing tables > > Internet: > Destination Gateway Flags Use Mtu Netif Expire > default 10.21.160.1 UGS 1218 1500 igb0 > 10.1.68.0/24 link#12 U 0 1500 vlan98 > 10.1.68.12 link#12 UHS 0 16384 lo0 > 10.1.68.13 link#12 UHS 0 16384 lo0 > 10.1.68.13/32 link#12 U 0 1500 vlan98 > 10.1.68.14 link#12 UHS 0 16384 lo0 > 10.1.68.14/32 link#12 U 0 1500 vlan98 > 10.1.68.15 link#12 UHS 0 16384 lo0 > 10.1.68.15/32 link#12 U 0 1500 vlan98 > 10.21.160.0/21 link#3 U 27230 1500 igb0 > 10.21.160.85 link#3 UHS 0 16384 lo0 > 10.24.213.0/24 10.1.68.11 UGS 0 1500 vlan98 > 10.24.214.0/24 10.1.68.11 UGS 0 1500 vlan98 > 10.24.215.0/24 10.1.68.11 UGS 0 1500 vlan98 > 127.0.0.1 link#5 UH 840 16384 lo0 > root@potato2:~ # > > > So far so good; then I accidentally hit up arrow and enter / or re-run > the promotion to /24 again two times > > IE: > root@potato2:~ # ifconfig vlan98 inet 10.1.68.12/24 > root@potato2:~ # ifconfig vlan98 inet 10.1.68.12/24 > > root@potato2:~ # netstat -nr4Wl > Routing tables > > Internet: > Destination Gateway Flags Use Mtu Netif Expire > default 10.21.160.1 UGS 1223 1500 igb0 > 10.1.68.0/24 link#12 U 0 1500 vlan98 > 10.1.68.12 link#12 UHS 0 16384 lo0 > 10.1.68.15 link#12 UHS 0 16384 lo0 > 10.1.68.15/32 link#12 U 0 1500 vlan98 > 10.21.160.0/21 link#3 U 27847 1500 igb0 > 10.21.160.85 link#3 UHS 0 16384 lo0 > 127.0.0.1 link#5 UH 868 16384 lo0 > > > > What just happened to my routes and check out what just happened to my aliases > > root@potato2:~ # ifconfig vlan98 > vlan98: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 > ether 00:0f:53:20:9d:00 > inet6 fe80::20f:53ff:fe20:9d00%vlan98 prefixlen 64 scopeid 0xc > inet 10.1.68.15 netmask 0xffffffff broadcast 10.1.68.15 > inet 10.1.68.12 netmask 0xffffff00 broadcast 10.1.68.255 > nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> > media: Ethernet autoselect > status: active > vlan: 98 parent interface: lagg0 > > > > Anyone have an idea what happened here ? > > -- > mark saad | nonesuch@longcount.org To be clear this is on 10-STABLE from 2017 however on 12-STABLE from Dec 2018 this is still acting odd but in a slightly diferent way. When I change the aliases from /28 and /26 back to /32 when I run root@ostrich:~ # ifconfig vlan98 inet 10.1.68.13/32 alias no issues root@ostrich:~ # ifconfig vlan98 inet 10.1.68.14/32 alias poof my routes are removed. Again anyone have any idea whats going on here ? -- mark saad | nonesuch@longcount.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAMXt9NazdCf5XZoSvu%2Bc4g7hfgbuaW=0EDrvBgmyF98YSyuy=w>