Date: Tue, 18 Dec 2007 18:27:57 +0100 From: Max Laier <max@love2party.net> To: freebsd-pf@freebsd.org Subject: Re: occasional "Operation not permitted" on state-mismatch Message-ID: <200712181828.04998.max@love2party.net> In-Reply-To: <14397207.post@talk.nabble.com> References: <200712180934.58755.silver.salonen@gmail.com> <14397207.post@talk.nabble.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--nextPart1995150.QsF0deyv5T Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Tuesday 18 December 2007, Atrox wrote: > Atrox wrote: > > Hello! > > > > I have some FreeBSD-boxes (2x6.3-PRERELEASE (installed on 08.Dec), > > 1x6.2-RELEASE) with PF configured. They are connected with OpenVPN > > LAN-to-LAN > > and the problem is that a few times per hour connection drops between > > computers from one LAN to another. At first I blamed OpenVPN, then I > > blamed > > bridge, but now I've realized that the problem is in PF. > > So I've tried increasing TCP-timeouts and setting optimization > > to "aggressive", but well, it's still the same. > > > > I monitor connections by sending TCP packets once per second to some > > other host and wait for reply. I use Nagios-plugins' check_tcp for > > that. The script > > looks like: > > =3D=3D=3D=3D=3D > > while [ 1 ]; do > > pfctl -si |grep mismatch > > /usr/local/libexec/nagios/check_tcp -H $host -p $port -t 2 > > pfctl -si |grep mismatch > > sleep 1 > > done > > =3D=3D=3D=3D=3D > > > > So if I let this script into action, I see that in 2-3 minutes, > > check_tcp gets "Operation not permitted" error and just in this > > moment packet-mismatch > > counter is increased by one (on machine with lesser traffic, I get > > the timeout > > in a few hours). That's on both 6.3-PRERELEASE as well as on > > 6.2-RELEASE. I've > > tried connections: > > * along WAN to IPFW-enabled machines > > * along WAN to PF-enabled machines > > * along LAN to PF-enabled machines > > * along LAN to Windows machines > > * along VPN to PF-enabled machines > > * along VPN to Windows machines > > > > Sometimes I get just some connection timeout: CRITICAL - Socket > > timeout after > > 2 seconds (I don't know what could cause that). > > > > I can see this behaviour in about every FreeBSD/PF machine I have. > > > > The basic PF-configuration looks like: > > =3D=3D=3D=3D=3D > > set block-policy return > > set loginterface $ext_if > > set timeout tcp.closed 15 > > set optimization aggressive > > scrub in all no-df > > > > block drop out quick on $ext_if from ($ext_if) to 0.0.0.0 > > block log all > > pass quick on lo0 all > > pass out all modulate state > > pass out proto tcp all flags S/SA modulate state > > pass on $int_if all modulate state > > pass on $int_if proto tcp all flags S/SA modulate state > > pass in on $ext_if proto tcp from any to ($ext_if) port $tcp_services > > flags > > S/SA modulate state > > =3D=3D=3D=3D=3D > > > > Is PF buggy or have I misconfigured smth? > > Today I installed an OpenBSD-4.2 box just to see whether PF does the > same thing there. And yes, it does. > pf.conf: > =3D=3D=3D=3D=3D > ext_if =3D rl0 > set block-policy return > set loginterface $ext_if > scrub in all no-df > block drop out quick on $ext_if from ($ext_if) to 0.0.0.0 > pass all modulate state > pass quick on lo0 all > =3D=3D=3D=3D=3D > > I check TCP without "sleep 1" now, and I do it to FreeBSD box without > firewall. state-mismatch gets increased by one, and I get either "No > route to host" or "Socket timeout after 2 seconds". > > Am I still misconfiguring the thing? No idea, you don't give nearly enough data for us to figure out what your=20 setup looks like. Regular tcp state mismatch usually hints that pf isn't=20 seeing all packets of the conversation. This can be caused by triangular=20 routing, load balanceing or if_bridge (which is difficult to get right in=20 some scenarios). You should figure out the exact path your tcp packets are taking (back and= =20 forth) and make sure pf sees all of them. Enabling additional pf logging=20 (pfctl -xm) helps to figure out what kind of mismatch is happening. =2D-=20 /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News --nextPart1995150.QsF0deyv5T Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQBHaAMkXyyEoT62BG0RAgT0AJ4qMX4Jxf3ZciJqtuWa0JAl+SyQYgCeINVX ah60YsBRsxQNfPukoMyCiFA= =/03A -----END PGP SIGNATURE----- --nextPart1995150.QsF0deyv5T--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200712181828.04998.max>