Date: Wed, 19 Apr 2006 11:40:15 GMT From: Daniel Hartmeier <daniel@benzedrine.cx> To: freebsd-bugs@FreeBSD.org Subject: Re: kern/95559: [RELENG_6] write(2) fails with EPERM on TCP socket under certain situations Message-ID: <200604191140.k3JBeFRn049327@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
The following reply was made to PR kern/95559; it has been noted by GNATS.
From: Daniel Hartmeier <daniel@benzedrine.cx>
To: Xin LI <delphij@delphij.net>
Cc: Gleb Smirnoff <glebius@FreeBSD.org>, gnn@FreeBSD.org,
Robert Watson <rwatson@FreeBSD.org>, mlaier@FreeBSD.org,
Xin LI <delphij@FreeBSD.org>, FreeBSD-gnats-submit@FreeBSD.org
Subject: Re: kern/95559: [RELENG_6] write(2) fails with EPERM on TCP socket under certain situations
Date: Wed, 19 Apr 2006 13:37:52 +0200
I haven't read all context yet, but maybe I can tell you how you can
check whether it's really pf blocking any packets.
If you create a state entry in pf for a TCP connection, pf must see and
match all packets of that connection against that state entry, otherwise
things will break. For instance, if pf only associates packets flowing
in one direction with the state entry, the state entry will never
advance to 'established' and pf can't track TCP windows and will sooner
or later start to block packets.
If outgoing packets of one connection are seen (by pf) on interface A,
but incoming packets of the same connection on a different interface B,
things still work, if you create a floating state (not using
'if-bound'). But the direction of packets matters. If pf sees packets
flowing in either direction both as incoming (on different interfaces),
or both as outgoing, things break.
To check whether either of those things occur in your setup, you can try
to establish one connection, then check the following things in pf
a) pfctl -vvss, should show one (or more) states related to the
connection.
Check the "x:y pkts" part on the third line, it shows how many
packets pf has associated with the state entry so far. x is
number of packets in the same direction as the initial packet
that created the state, y is in the reverse direction. If either
one of those is >1 but the other ==0, pf doesn't see replies in
the opposite direction.
The right-most string on the first line tells how advanced the
state entry is. After a successful TCP handshake, while the
connection is not closed from either side, it should read
'ESTABLISHED:ESTABLISHED'.
If there are multiple states related to a single connection,
make sure each one is created as expected, and advancing normally.
A common mistake, for instance, is to create state not on the
initial SYN of the TCP handshake, but on a subsequent packet.
This causes pf to miss the TCP window scaling negotiation, and
can break connections eventually, after they appear to have been
established fine and progress to some degree.
b) pfctl -si, check for increasing counters once the problem occurs.
pf will increase at least one of these counters for every packet
it blocks, for any reason. If no counter is increasing, pf hasn't
blocked a packet.
c) pfctl -xm, enables debug logging to /var/log/messages, enable, then
reproduce the problem, then check the log. If there are any
messages from pf (like 'BAD state'), those will help analysis.
One explanation why you'd see EPERM is that in FreeBSD, the pfil wrapper
simply returns pf_test()'s return value. This is either PF_PASS (0) or
PF_DROP (1), and 1 is also the value of EPERM, by coincidence.
On OpenBSD and NetBSD, the return value PF_DROP of pf_test() is mapped
to errno 65 EHOSTUNREACH, as that is one existing errno that most
network related syscalls that can now additionally fail due to pf blocking
can return otherwise already (according to their individual man pages).
While returning EPERM is somewhat intuitive, it's not an errno an
application must expect to come back from most such syscalls. On the
other hand, people are regularly confused (on Open/Net) when tools (like
ping) fail with 'No route to host' due to pf blocking, when the routing
table is not the problem at all.
Not sure if this is different on FreeBSD intentionally or an oversight.
From a cross-platform supporter's point of view, it would make things
easier if it was the same on all platforms :)
HTH,
Daniel
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200604191140.k3JBeFRn049327>