Date: Tue, 28 Aug 2007 22:53:38 +0200 From: Max Laier <max@love2party.net> To: "Bill Marquette" <bill.marquette@gmail.com> Cc: freebsd-pf@freebsd.org Subject: Re: pfsync errors Message-ID: <200708282253.46740.max@love2party.net> In-Reply-To: <55e8a96c0708281344y62bbb152k3f2a7e9d375a2acd@mail.gmail.com> References: <55e8a96c0708221242h2d5e7d15q847e6fac7cf60554@mail.gmail.com> <55e8a96c0708272114n42c0d9e7h675d1a9043bf3d80@mail.gmail.com> <55e8a96c0708281344y62bbb152k3f2a7e9d375a2acd@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--nextPart8114292.2W39dtXNfE
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
On Tuesday 28 August 2007, Bill Marquette wrote:
> On 8/27/07, Bill Marquette <bill.marquette@gmail.com> wrote:
> > > > Here's what we get with the patch:
> > > > pfsync_senddef: ip_output 64
> > >
> > > that's EHOSTDOWN ... that's strange. Are you using syncpeer?
>
> After converting both sides to using syncpeer, I now got this in my
> dmesg after reboot:
> <snip most of dmesg you've already seen>
> Trying to mount root from ufs:/dev/da0s2a
> pfsync_senddef: ip_output 65
> pfsync_senddef: ip_output 65
>
> $ netstat -s -ppfsync && sleep 1 && netstat -s -ppfsync
> pfsync:
> 699811 packets received (IPv4)
> 0 packets received (IPv6)
> 0 packets discarded for bad interface
> 0 packets discarded for bad ttl
> 0 packets shorter than header
> 0 packets discarded for bad version
> 0 packets discarded for bad HMAC
> 0 packets discarded for bad action
> 0 packets discarded for short packet
> 0 states discarded for bad values
> 0 stale states
> 61740 failed state lookup/inserts
> 46728 packets sent (IPv4)
> 0 packets sent (IPv6)
> 0 send failed due to mbuf memory error
> 23020 send error
> pfsync:
> 704677 packets received (IPv4)
> 0 packets received (IPv6)
> 0 packets discarded for bad interface
> 0 packets discarded for bad ttl
> 0 packets shorter than header
> 0 packets discarded for bad version
> 0 packets discarded for bad HMAC
> 0 packets discarded for bad action
> 0 packets discarded for short packet
> 0 states discarded for bad values
> 0 stale states
> 62032 failed state lookup/inserts
> 47070 packets sent (IPv4)
> 0 packets sent (IPv6)
> 0 send failed due to mbuf memory error
> 23221 send error
>
> The other pfsyncstats.pfsyncs_oerrors++ is after a IF_HANDOFF macro, I
> traced it back to if_handoff() in if.c, but I'm not sure how to track
> it down any further as it look like it's _IF_QFULL(ifq) that's
> triggering the return 0.
>
> I'm going to toy with some settings for the em(4) driver in
> loader.conf and see if I can raise the txd and rxd descriptors since
> we're not running on 82542 or 82543 chipped hardware we can go above
> 256 descriptors.
No that's the internal work deferral queue. Try something like the=20
following in contrib/pf/net/if_pfsync.c:
@@ -229,7 +229,7 @@
callout_init(&sc->sc_bulk_tmo, NET_CALLOUT_MPSAFE);
callout_init(&sc->sc_bulkfail_tmo, NET_CALLOUT_MPSAFE);
callout_init(&sc->sc_send_tmo, NET_CALLOUT_MPSAFE);
=2D sc->sc_ifq.ifq_maxlen =3D ifqmaxlen;
+ sc->sc_ifq.ifq_maxlen =3D 2 * ifqmaxlen;
mtx_init(&sc->sc_ifq.ifq_mtx, ifp->if_xname, "pfsync send queue",
MTX_DEF);
if_attach(ifp);
But there might be other reasons like timing wrt the locks. I'll have to=20
check for details. It might also be a good idea to MFC the taskqueue=20
approach from CURRENT, rather than using the callout ... that's a bit of=20
work however.
=2D-=20
/"\ Best regards, | mlaier@freebsd.org
\ / Max Laier | ICQ #67774661
X http://pf4freebsd.love2party.net/ | mlaier@EFnet
/ \ ASCII Ribbon Campaign | Against HTML Mail and News
--nextPart8114292.2W39dtXNfE
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.4 (FreeBSD)
iD8DBQBG1ItaXyyEoT62BG0RAiQXAJ96MwKuzLsyXcgFzZr7bCEZPxA4xACfTcB4
MHwsRAisyPoCIDsRZUn/6gQ=
=IcpH
-----END PGP SIGNATURE-----
--nextPart8114292.2W39dtXNfE--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200708282253.46740.max>