Date: Wed, 26 Oct 2011 09:54:31 +0200 From: Pawel Jakub Dawidek <pjd@FreeBSD.org> To: John Baldwin <jhb@freebsd.org> Cc: Kostik Belousov <kostikbel@gmail.com>, Lawrence Stewart <lstewart@freebsd.org>, freebsd-current@freebsd.org, Andre Oppermann <andre@freebsd.org>, freebsd-net@freebsd.org Subject: Re: 9.0-RC1 panic in tcp_input: negative winow. Message-ID: <20111026075431.GB1672@garage.freebsd.pl> In-Reply-To: <201110240814.22368.jhb@freebsd.org> References: <20111022084931.GD1697@garage.freebsd.pl> <20111023084445.GB50300@deviant.kiev.zoral.com.ua> <20111023155827.GH1697@garage.freebsd.pl> <201110240814.22368.jhb@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--0eh6TmSyL6TZE2Uz Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Oct 24, 2011 at 08:14:22AM -0400, John Baldwin wrote: > On Sunday, October 23, 2011 11:58:28 am Pawel Jakub Dawidek wrote: > > On Sun, Oct 23, 2011 at 11:44:45AM +0300, Kostik Belousov wrote: > > > On Sun, Oct 23, 2011 at 08:10:38AM +0200, Pawel Jakub Dawidek wrote: > > > > My suggestion would be that if we won't be able to fix it before 9.= 0, > > > > we should turn this assertion off, as the system seems to be able to > > > > recover. > > >=20 > > > Shipped kernels have all assertions turned off. > >=20 > > Yes, I'm aware of that, but many people compile their production kernels > > with INVARIANTS/INVARIANT_SUPPORT to fail early instead of eg. > > corrupting data. I'd be fine in moving this under DIAGNOSTIC or changing > > it into a printf, so it will be visible. >=20 > No, the kernel is corrupting things in other places when this is true, so > if you are running with INVARIANTS, we want to know about it. Specifica= lly, > in several places in TCP we assume that rcv_adv >=3D rcv_nxt, and depend = on > being able to do 'rcv_adv - rcv_nxt'. >=20 > In this case, it looks like the difference is consistently less than one= =20 > frame. I suspect the other end of the connection is sending just beyond = the=20 > end of the advertised window (it probably assumes it is better to send a = full=20 > frame if it has that much pending data even though part of it is beyond t= he=20 > window edge vs sending a truncated packet that just fills the window) and= that > that frame is accepted ok in the header prediction case and it's ACK is= =20 > delayed, but the next packet to arrive then trips over this assumption. >=20 > Since 'win' is guaranteed to be non-negative and we explicitly cast > 'rcv_adv - rcv_nxt' to (int) in the following line that the assert is che= cking > for: >=20 > tp->rcv_wnd =3D imax(win, (int)(tp->rcv_adv - tp->rcv_nxt)); >=20 > I think we already handle this case ok and perhaps the assertion can just= be > removed? Not sure if others feel that it warrants a comment to note that= this > is the case being handled. I added debug to the places where rcv_adv and rcv_nxt are modified. Here is what happens before the panic occurs: tcp_do_segment:1722 negative window: tp 0xfffffe000dab1b70 rcv_nxt 40223615= 48 rcv_adv 4022360100 diff -1448 tcp_do_segment:2847 negative window: tp 0xfffffe000dab1b70 rcv_nxt 40223622= 98 rcv_adv 4022361548 diff -750 tcp_do_segment:1722 negative window: tp 0xfffffe000dab1b70 rcv_nxt 40223637= 46 rcv_adv 4022362298 diff -1448 tcp_do_segment:2847 negative window: tp 0xfffffe000dab1b70 rcv_nxt 40223648= 36 rcv_adv 4022363746 diff -1090 tcp_do_segment:1722 negative window: tp 0xfffffe000dab1b70 rcv_nxt 40223662= 84 rcv_adv 4022364836 diff -1448 tcp_do_segment:1722 negative window: tp 0xfffffe000dab1b70 rcv_nxt 40223706= 28 rcv_adv 4022369690 diff -938 tcp_do_segment:1722 negative window: tp 0xfffffe000dab1b70 rcv_nxt 40223791= 40 rcv_adv 4022377692 diff -1448 tcp_do_segment:1722 negative window: tp 0xfffffe000dab1b70 rcv_nxt 40223877= 92 rcv_adv 4022386344 diff -1448 tcp_do_segment:2847 negative window: tp 0xfffffe000dab1b70 rcv_nxt 40223888= 90 rcv_adv 4022387792 diff -1098 tcp_do_segment:1722 negative window: tp 0xfffffe000dab1b70 rcv_nxt 40223903= 38 rcv_adv 4022388890 diff -1448 tcp_do_segment:2847 negative window: tp 0xfffffe000dab1b70 rcv_nxt 40223945= 63 rcv_adv 4022394342 diff -221 panic: tcp_input negative window: tp 0xfffffe000dab1b70 rcv_nxt 4022394563 = rcv_adv 4022394342 win=3D0 diff -221 I can send you the full log if you want, I've plenty of messages where rcv_adv < rcv_nxt, not all of them trigger this assertion. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --0eh6TmSyL6TZE2Uz Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk6nvLYACgkQForvXbEpPzS/igCgh6TcluOieBa/tfP90xC4Gy1t 5GMAoNQG1Mc8LyevKm/3rQFJoC02DriH =LzuI -----END PGP SIGNATURE----- --0eh6TmSyL6TZE2Uz--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20111026075431.GB1672>