Date: Mon, 14 Jan 2019 16:48:53 -0500 From: Mark Saad <nonesuch@longcount.org> To: FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Re: Removing an alias can remove routes ? Message-ID: <CAMXt9NbV%2Bh8tjugcss%2BdafbUOT2o6bh2eWbBFAKNoJ9GeYs1Cw@mail.gmail.com> In-Reply-To: <201901141821.x0EILVem036356@pdx.rh.CN85.dnsmgr.net> References: <CAMXt9NZ8VtTETH0_pRBeHZ7P%2Bgocpxa3EypWuJ6P=b8-%2B3VgqA@mail.gmail.com> <201901141821.x0EILVem036356@pdx.rh.CN85.dnsmgr.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Jan 14, 2019 at 1:21 PM Rodney W. Grimes
<freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote:
>
> > On Mon, Jan 14, 2019 at 11:59 AM Rodney W. Grimes
> > <freebsd-rwg@pdx.rh.cn85.dnsmgr.net> wrote:
> > >
> > > > All
> > > > I ran into a peculiar over the weekend on two FreeBSD 10-STABLE
> > > > servers that has me at a loss. Both servers have the same setup lacp
> > > > lagg wtih vlans using the lagg as a parent dev.
> > > > On the vlans I have some alias along with the primary ip. When a
> > > > alias was added or deleted routes that sourced out of the primary ip
> > > > on that vlan were dropped from the routing table.
> > > >
> > > >
> > > > So for example. I have lagg0.vlan1044 . (both servers are similarly
> > > > configured as routers / firewalls )
> > > >
> > > > [root@pineapple01 ~]# ifconfig lagg0.vlan1044
> > > > lagg0.vlan1044: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST>
> > > > metric 0 mtu 1500
> > > > options=300<TSO4,TSO6>
> > > > ether 00:0f:53:20:9b:d0
> > > > inet6 fe80::20f:53ff:fe20:9bd0%lagg0.vlan1044 prefixlen 64
> > > > scopeid 0x80
> > > > inet 10.24.213.84 netmask 0xffffffe0 broadcast 10.24.213.95
> > > > inet 10.24.212.129 netmask 0xffffffff broadcast 10.24.212.129
> > > > inet 10.24.213.163 netmask 0xfffffff0 broadcast 10.24.213.175
> > > > inet 10.24.213.76 netmask 0xffffffe0 broadcast 10.24.213.95
> > > > inet 10.24.213.94 netmask 0xffffffe0 broadcast 10.24.213.95
> > > > inet 10.24.213.89 netmask 0xffffffe0 broadcast 10.24.213.95
> > > > inet 10.24.213.75 netmask 0xffffffe0 broadcast 10.24.213.95
> > > > nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
> > > > media: Ethernet autoselect
> > > > status: active
> > > > vlan: 1044 parent interface: lagg0
> > > >
> > > > [root@pineapple01 ~]# netstat -nr4Wl
> > > > ...
> > > > 192.168.144.32/27 10.24.213.65 UGS 0 1500 lagg0.vlan1044
> > > > 192.168.144.96/27 10.24.213.65 UGS 0 1500 lagg0.vlan1044
> > > > 192.168.23.0/24 10.24.213.65 UGS 0 1500 lagg0.vlan1044
> > > > 192.168.120.0/21 10.24.213.65 UGS 0 1500 lagg0.vlan1044
> > > >
> > > >
> > > > So I wanted to remove the alias ended in 163 and fix its netmask back to /32
> > > >
> > > > I ran this
> > > >
> > > > ifconfig lagg0.vlan1044 inet 10.24.213.163/24 -alias && ifconfig
> > > ^ delete?
> >
> > I use -alias which is an alias of -delete
> >
> > >
> > > > lagg0.vlan1044 inet 10.24.213.163/32 -alias
>
> This is also a delete?
> I am concerend that what you think the command you
> typed is not actually the command you typed and what
> you did actually type has the bad side effects.
>
>
> > > > and shortly there after all of the routes that went out lagg0.vlan1044
> > > > were gone . I quickly undid my change and put the routes back but I am
> > > > not sure what caused this ? Anyone have any ideas I have done this in
> > > > the past with out issue and I am unsure whats changed other then the
> > > > box have a long up time of 463 days .
> > >
> > > I believe what happened here is that 10.24.213.163/24 when reduced
> > > to a network address is 10.24.213.0/24, which probably got sent to
> > > the route removal code, which since the route to that covers the
> > > gateway at 10.24.213.65 that gateway was no longer accessable
> > > so all routes via it got removed.
> > >
> >
> > ok I see what you are saying and it makes sense to me; do you know why
> > the routing code does not see the primary ip ( the non-alias one) is
> > still
> > live on the interface before deciding to drop the routes associated with it ?
>
> It should, but I can not see enough of your routing table to
> guess as to exactly what the kernel did when you removed
> this interface. The routes I see above all would go through
> the interface you showed you removed, thus they would all
> go away.
>
> --
> Rod Grimes rgrimes@freebsd.org
Ok some further testing shows what appears to be unintended results.
On my test box I do the following
ifconfig vlan98 create
ifconfig vlan98 vlan 98 vlandev lagg0
ifconfig vlan98 inet 10.1.68.12/26
ifconfig vlan98 inet 10.1.68.13/26 alias
ifconfig vlan98 inet 10.1.68.14/28 alias
ifconfig vlan98 inet 10.1.68.15/32 alias
route add 10.24.213.0/24 10.1.68.11
route add 10.24.214.0/24 10.1.68.11
route add 10.24.215.0/24 10.1.68.11
root@potato2:~ # netstat -nr4Wl
Routing tables
Internet:
Destination Gateway Flags Use Mtu Netif Expire
default 10.21.160.1 UGS 1216 1500 igb0
10.1.68.0/26 link#12 U 0 1500 vlan98
10.1.68.12 link#12 UHS 0 16384 lo0
10.1.68.13 link#12 UHS 0 16384 lo0
10.1.68.13/32 link#12 U 0 1500 vlan98
10.1.68.14 link#12 UHS 0 16384 lo0
10.1.68.14/32 link#12 U 0 1500 vlan98
10.1.68.15 link#12 UHS 0 16384 lo0
10.1.68.15/32 link#12 U 0 1500 vlan98
10.21.160.0/21 link#3 U 26985 1500 igb0
10.21.160.85 link#3 UHS 0 16384 lo0
10.24.213.0/24 10.1.68.11 UGS 0 1500 vlan98
10.24.214.0/24 10.1.68.11 UGS 0 1500 vlan98
10.24.215.0/24 10.1.68.11 UGS 0 1500 vlan98
127.0.0.1 link#5 UH 840 16384 lo0
Then I realize that my masks are wrong and I change the aliases back to "/32's"
root@potato2:~ # ifconfig vlan98
vlan98: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 00:0f:53:20:9d:00
inet 10.1.68.12 netmask 0xffffffc0 broadcast 10.1.68.63
inet6 fe80::20f:53ff:fe20:9d00%vlan98 prefixlen 64 scopeid 0xc
inet 10.1.68.13 netmask 0xffffffff broadcast 10.1.68.13
inet 10.1.68.14 netmask 0xffffffff broadcast 10.1.68.14
inet 10.1.68.15 netmask 0xffffffff broadcast 10.1.68.15
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
media: Ethernet autoselect
status: active
vlan: 98 parent interface: lagg0
root@potato2:~ # netstat -nr4Wl
Routing tables
Internet:
Destination Gateway Flags Use Mtu Netif Expire
default 10.21.160.1 UGS 1217 1500 igb0
10.1.68.0/26 link#12 U 0 1500 vlan98
10.1.68.12 link#12 UHS 0 16384 lo0
10.1.68.13 link#12 UHS 0 16384 lo0
10.1.68.13/32 link#12 U 0 1500 vlan98
10.1.68.14 link#12 UHS 0 16384 lo0
10.1.68.14/32 link#12 U 0 1500 vlan98
10.1.68.15 link#12 UHS 0 16384 lo0
10.1.68.15/32 link#12 U 0 1500 vlan98
10.21.160.0/21 link#3 U 27166 1500 igb0
10.21.160.85 link#3 UHS 0 16384 lo0
10.24.213.0/24 10.1.68.11 UGS 0 1500 vlan98
10.24.214.0/24 10.1.68.11 UGS 0 1500 vlan98
10.24.215.0/24 10.1.68.11 UGS 0 1500 vlan98
127.0.0.1 link#5 UH 840 16384 lo0
root@potato2:~ #
So far no problems.
Then I see ohh my primary ip needs to be a "/24"
root@potato2~ # ifconfig vlan98 inet 10.1.68.12/24
root@potato2:~ # netstat -nr4Wl
Routing tables
Internet:
Destination Gateway Flags Use Mtu Netif Expire
default 10.21.160.1 UGS 1218 1500 igb0
10.1.68.0/24 link#12 U 0 1500 vlan98
10.1.68.12 link#12 UHS 0 16384 lo0
10.1.68.13 link#12 UHS 0 16384 lo0
10.1.68.13/32 link#12 U 0 1500 vlan98
10.1.68.14 link#12 UHS 0 16384 lo0
10.1.68.14/32 link#12 U 0 1500 vlan98
10.1.68.15 link#12 UHS 0 16384 lo0
10.1.68.15/32 link#12 U 0 1500 vlan98
10.21.160.0/21 link#3 U 27230 1500 igb0
10.21.160.85 link#3 UHS 0 16384 lo0
10.24.213.0/24 10.1.68.11 UGS 0 1500 vlan98
10.24.214.0/24 10.1.68.11 UGS 0 1500 vlan98
10.24.215.0/24 10.1.68.11 UGS 0 1500 vlan98
127.0.0.1 link#5 UH 840 16384 lo0
root@potato2:~ #
So far so good; then I accidentally hit up arrow and enter / or re-run
the promotion to /24 again two times
IE:
root@potato2:~ # ifconfig vlan98 inet 10.1.68.12/24
root@potato2:~ # ifconfig vlan98 inet 10.1.68.12/24
root@potato2:~ # netstat -nr4Wl
Routing tables
Internet:
Destination Gateway Flags Use Mtu Netif Expire
default 10.21.160.1 UGS 1223 1500 igb0
10.1.68.0/24 link#12 U 0 1500 vlan98
10.1.68.12 link#12 UHS 0 16384 lo0
10.1.68.15 link#12 UHS 0 16384 lo0
10.1.68.15/32 link#12 U 0 1500 vlan98
10.21.160.0/21 link#3 U 27847 1500 igb0
10.21.160.85 link#3 UHS 0 16384 lo0
127.0.0.1 link#5 UH 868 16384 lo0
What just happened to my routes and check out what just happened to my aliases
root@potato2:~ # ifconfig vlan98
vlan98: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 00:0f:53:20:9d:00
inet6 fe80::20f:53ff:fe20:9d00%vlan98 prefixlen 64 scopeid 0xc
inet 10.1.68.15 netmask 0xffffffff broadcast 10.1.68.15
inet 10.1.68.12 netmask 0xffffff00 broadcast 10.1.68.255
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
media: Ethernet autoselect
status: active
vlan: 98 parent interface: lagg0
Anyone have an idea what happened here ?
--
mark saad | nonesuch@longcount.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAMXt9NbV%2Bh8tjugcss%2BdafbUOT2o6bh2eWbBFAKNoJ9GeYs1Cw>