Date: Fri, 2 Jul 2010 20:53:25 -0700 From: Garrett Cooper <yanefbsd@gmail.com> To: Ian Smith <smithi@nimnet.asn.au> Cc: Luigi Rizzo <luigi@freebsd.org>, net@freebsd.org Subject: Re: Deterministic lockup / panic in networking stack with ipfw / natd enabled on recent amd64 STABLE / CURRENT Message-ID: <AANLkTinQc1Gw6j-aLOLpWXLNJ-P65mpRB4_S3z0ipG_F@mail.gmail.com> In-Reply-To: <20100703125804.P54166@sola.nimnet.asn.au> References: <AANLkTilz7_P5jKx5pHtojocdZAeg6OyE4YtLO7AZGwaI@mail.gmail.com> <20100702234212.B54166@sola.nimnet.asn.au> <20100703125804.P54166@sola.nimnet.asn.au>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Jul 2, 2010 at 8:26 PM, Ian Smith <smithi@nimnet.asn.au> wrote:
> On Sat, 3 Jul 2010, Ian Smith wrote:
> =A0> On Tue, 15 Jun 2010, Garrett Cooper wrote:
> =A0> =A0> Hi,
> =A0> =A0> =A0 =A0 I'm experiencing a deterministic situation on a develop=
ment box I
> =A0> =A0> manage when I do the following to enable ipfw and natd to bridg=
e a
> =A0> =A0> network with two bce(4) enabled NICs, where if I do the followi=
ng
> =A0> =A0> steps below, then try to push a few tcp frames through, the ker=
nel
> =A0> =A0> either hardlocks, or panics in the bce(4) code, ipfw(4) code or
> =A0> =A0> networking stack code.
> =A0> =A0> =A0 =A0 My kernel is relatively vanilla (I just turned off a nu=
mber of
> =A0> =A0> drivers that I don't use because the hardware support isn't the=
re),
> =A0> =A0> and all of the networking options available in GENERIC are enab=
led as
> =A0> =A0> well. I have ipfw, ipfw_nat, and libalias built as modules, alo=
ng with
> =A0> =A0> bce and em.
> =A0> =A0> =A0 =A0 I've included the stats on the machine. Note that it is=
a dual
> =A0> =A0> SMT-enabled quad core machine with 8GB of RAM. I haven't done a=
nything
> =A0> =A0> to pimp the box settings via make.conf whatsoever. I would prov=
ide a
> =A0> =A0> crashdump, but dumpon is broken on the box (which is extremely
> =A0> =A0> annoying). Please note that pf doesn't have any issues pushing =
packets
> =A0> =A0> with similar rules.
> =A0> =A0> =A0 =A0 This has occurred on both 8-STABLE (r209169), and 9-CUR=
RENT (r208809).
> =A0> =A0> =A0 =A0 Here's the manual procedure for reproducing the issue:
> =A0> =A0>
> =A0> =A0> # Do the following steps (this isn't automated apparently as it
> =A0> =A0> completely blocks off a running box, when using ipfw restart is=
run).
> =A0> =A0>
> =A0> =A0> # Copy the 8.0-RELEASE copy of rc.firewall over
> =A0> =A0> cp -p /usr/src/etc/rc.firewall /etc
> =A0> =A0>
> =A0> =A0> # Make sure you have access via ssh being redirected via natd.
> =A0> =A0> echo "redirect_port tcp 192.168.10.1:22 22" > /etc/natd.conf
> =A0> =A0>
> =A0> =A0> # Enable all of the required services and knobs
> =A0> =A0> cat >> /etc/rc.conf <<EOF
> =A0> =A0> firewall_enable=3D"YES"
> =A0> =A0> firewall_logging=3D"YES"
> =A0> =A0> firewall_nat_enable=3D"YES"
> =A0> =A0> firewall_nat_interface=3D"bce1"
> =A0> =A0> firewall_type=3D"open"
> =A0> =A0> gateway_enable=3D"YES"
> =A0> =A0> ipfw_enable=3D"YES"
> =A0> =A0> natd_enable=3D"YES"
> =A0> =A0> natd_interface=3D"bce1"
> =A0> =A0> natd_flags=3D"-dynamic -d -m"
> =A0> =A0> EOF
> =A0>
> =A0> Garrett I missed this earlier; here from your ref in the TSO thread.
> =A0>
> =A0> If you enable both firewall_nat and natd as above, on that config yo=
u
> =A0> should have wound up with two of ipfw rule 50, like
> =A0>
> =A0> =A050 divert 8668 ip4 from any to any via bce1
> =A0> =A050 nat 123 ip4 from any to any via bce1
> =A0>
> =A0> but I don't think you really wanted to run natd then firewall_nat ag=
ain
> =A0> like that?
>
> Oh, sorry .. that's not right; I quite forgot the discussions in ipfw@
> about this a while ago, until I re-browsed natd(8):
>
> =A0 =A0 =A0 =A0 =A0After translation by natd, packets re-enter the firewa=
ll at the rule
> =A0 =A0 =A0 =A0 =A0number following the rule number that caused the diver=
sion (not the
> =A0 =A0 =A0 =A0 =A0next rule if there are several at the same number).
>
> so in this case only natd should be invoked and the ipfw nat skipped.
>
> =A0> Also I'm pretty sure you'd need to include '-f /etc/natd.conf' in yo=
ur
> =A0> natd_flags for your redirect_port config, here's no default configfi=
le
> =A0> for natd (AFAIK)
>
> I think that's right - or you can specify -redirect_port in natd_flags.
>
> =A0> I guess rc.firewall ought to be checking that natd_enable and
> =A0> firewall_nat_enable aren't both YES ..
>
> .. and that becomes irrelevant, though it's still an ambiguous config.
I'll look into this more closely on Sunday when I come in to repro
the issue.
Thanks for the feedback :)!
-Garrett
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AANLkTinQc1Gw6j-aLOLpWXLNJ-P65mpRB4_S3z0ipG_F>