Date: Tue, 23 Apr 2013 08:08:07 -0400 From: Randall Stewart <rrs@lakerest.net> To: Nick Rogers <ncrogers@gmail.com> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>, Andre Oppermann <andre@freebsd.org> Subject: Re: Default route changes unexpectedly Message-ID: <556E6D18-15FD-4D89-8064-45B139C9C6E7@lakerest.net> In-Reply-To: <CAKOb=YaX%2ByopoofwRbfN7ZXc_yG0uxoKkr3aXsVcXEdLqQ=AXQ@mail.gmail.com> References: <CAKOb=YYGu6mr-3nyydBi9K-FHPnEx-fKSZ2=r_uDVeY9pvrqtQ@mail.gmail.com> <5136FD71.6000408@freebsd.org> <CAKOb=YaX%2ByopoofwRbfN7ZXc_yG0uxoKkr3aXsVcXEdLqQ=AXQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Ok I too have been struck by this *multiple* times on my base home router. I am not sure how its happening, but I have placed in my kernel a = special catch that if the default route is set via the normal route.c path and = it is *not* the default route to my ISP, I will crash the kernel. My thoughts are this is *not* going to happen and its probably some = other memory corruption, but I will start with the most obvious first ;-) Now, I have also put in a cron that checks every 10 min the default = route, and=20 if it finds it *not* correct, it will a) log it, and b) fix it. So if I get a log showing up, I will know its *not* some errant program = in 9.x=20 causing a routing change via a routing socket, and if not I should have = a crash revealing who the guilty party is ;-) I will keep you all informed as I try to gather more information ;-D R On Mar 7, 2013, at 12:07 PM, Nick Rogers wrote: > On Wed, Mar 6, 2013 at 12:25 AM, Andre Oppermann <andre@freebsd.org> = wrote: >> On 05.03.2013 18:39, Nick Rogers wrote: >>>=20 >>> Hello, >>>=20 >>> I am attempting to create awareness of a serious issue affecting = users >>> of FreeBSD 9.x and PF. There appears to be a bug that allows the >>> kernel's routing table to be corrupted by traffic routing through = the >>> system. Under heavy traffic load, the default route can seemingly >>> randomly change to an IP address that is not directly connected to = the >>> network (i.e., is not configured anywhere). Dhclient is not in the >>> mix, nor is routed, bgpd, etc. Running `route monitor` shows no >>> evidence of the change in the default route. The one commonality >>> between all the systems experiencing this problem seems to be the = use >>> of PF. >>>=20 >>> Obviously this is a serious problem as it causes all Internet-bound >>> traffic to stop routing until the default route is corrected. Some >>> users, including myself, are working around this problem by = installing >>> a script that runs multiple times a second to check if the default >>> route is incorrect and fixing it if necessary, which mitigates the >>> amount of downtime caused by the bug. >>=20 >>=20 >> Can you describe your traffic forwarding setup in more detail? >> Is it only pf, or do you run netgraph, or other things as well? >> Do you use flow routing? >=20 > I use PF for NAT, filtering, and rdr rules. ALTQ for bandwidth > management. I do not use netgraph. I use vlans. PF redirects to squid > as a transproxy. I'm not familiar with flow routing so unless its > enabled in 9.1 by default I do not use it. >=20 >>=20 >> How frequent does this happen? > Every other day during periods of heavier Internet-bound traffic. >=20 >>=20 >> I'm trying to create a stack graph to see which parts of the network >> stack are involved in handling your packet. >>=20 >> -- >> Andre >>=20 >>> Please refer to these past posts for more examples and evidence of >>> other users experiencing this problem: >>>=20 >>> http://forums.freebsd.org/showthread.php?p=3D211610#post211610 >>>=20 >>>=20 >>> = http://freebsd.1045724.n5.nabble.com/Default-route-quot-random-quot-gatewa= y-modification-bug-td5750820.html >>>=20 >>> = http://lists.freebsd.org/pipermail/freebsd-net/2012-March/031879.html >>>=20 >>> = http://lists.freebsd.org/pipermail/freebsd-ipfw/2010-September/004361.html= >>>=20 >>> There is also a PR that was incorrectly labeled as an IPFW issue. >>> Myself and others believe this issue is not restricted to the use of >>> IPFW and that the PR should be relabeled. I am inclined to think it = is >>> strictly a PF issue since I am not using IPFW, however there is >>> evidence of the default route changing on people using IPFW for past >>> versions of FreeBSD (7.x/8.x), so perhaps this is related. >>>=20 >>> http://www.freebsd.org/cgi/query-pr.cgi?pr=3Dkern/174749 >>>=20 >>> Another PR for the same problem but specific to IPFW and 8.2-RELEASE >>>=20 >>> http://www.freebsd.org/cgi/query-pr.cgi?pr=3D157796 >>>=20 >>> I am hoping someone reading this can give the problem the attention = it >>> deserves. Thank you. >>>=20 >>> -Nick >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>>=20 >>>=20 >>=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >=20 ------------------------------ Randall Stewart 803-317-4952 (cell)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?556E6D18-15FD-4D89-8064-45B139C9C6E7>