From owner-freebsd-net@freebsd.org Thu Sep 24 07:58:00 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6AB18A08CF3 for ; Thu, 24 Sep 2015 07:58:00 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-wi0-f178.google.com (mail-wi0-f178.google.com [209.85.212.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E6A3E1376; Thu, 24 Sep 2015 07:57:59 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by wicge5 with SMTP id ge5so240029756wic.0; Thu, 24 Sep 2015 00:57:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-type; bh=UkKh9D9MO9F+jbcG4OYPkYN438nuf7qv9paW7Uyueos=; b=EEb9cya+eY9qv05RAYZemOxvu2rlHUjeGx2lrMvQpOtSTJrfEOtuHh0QMUiP9CTRT8 SMFSqFcI4XFP3e0AzkmqjgTThwycKqcmjhORgqNwAmttQJv7z8ug8r8doujGgTQIbVUa fN/bos5fjnrgTBFt/CcEj6tWRroI8hKt1WYP5LTFRfjOml0ULAkoFx9Y1C7AfCzU8upg yaH/9NGu310mzYio3XqFGjf1lXaxQv8+9I+SQ1QaNJqKbM9s40IVY801D7DjQcsZh8dE Shv0bE+7nKf6hhiacwlqTzJEYQvJfXchpk7MNmIc/7wqURfYb/iFYhWOSei9n3tSLTRT 26cg== X-Received: by 10.180.8.164 with SMTP id s4mr8856876wia.5.1443081471762; Thu, 24 Sep 2015 00:57:51 -0700 (PDT) Received: from FRI2JCHARBON-M1.local ([217.30.88.7]) by smtp.googlemail.com with ESMTPSA id mx19sm12368757wic.0.2015.09.24.00.57.50 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 24 Sep 2015 00:57:51 -0700 (PDT) Subject: Re: Kernel panics in tcp_twclose To: Palle Girgensohn References: <26B0FF93-8AE3-4514-BDA1-B966230AAB65@FreeBSD.org> <55FC1809.3070903@freebsd.org> <20150918160605.GN67105@kib.kiev.ua> <55FFBE01.6060706@freebsd.org> <3721F099-F45D-4DCD-8AB3-84D1ABC44145@FreeBSD.org> <73856F2B-3E70-483C-9988-C84E798CEB44@FreeBSD.org> <44EBAC98-4761-4E47-8E47-5032430A1C8A@FreeBSD.org> <56019AF8.8000705@freebsd.org> <5601CF2D.9030307@freebsd.org> <5602E90A.9050504@freebsd.org> <0931591A-23EC-40CB-A109-72E9308B1A2D@pingpong.net> <5602F044.5010606@freebsd.org> <54767991-9D3B-4ECB-A07E-CFA21A54BBDD@pingpong.net> <4E148E2E-F8D2-41C2-B232-9FD1548AA20B@pingpong.net> <30AD333B-EC8B-4EEF-8FE2-8EA8C216601E@FreeBSD.org> <5603A03B.4060002@freebsd.org> Cc: freebsd-net@freebsd.org From: Julien Charbon X-Enigmail-Draft-Status: N1110 Message-ID: <5603ACF7.7040403@freebsd.org> Date: Thu, 24 Sep 2015 09:57:43 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <5603A03B.4060002@freebsd.org> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="2xKnTLrEPFmUogFPILk9iCAhuv6Q8h3jH" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Sep 2015 07:58:00 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --2xKnTLrEPFmUogFPILk9iCAhuv6Q8h3jH Content-Type: multipart/mixed; boundary="------------010008010008030404040209" This is a multi-part message in MIME format. --------------010008010008030404040209 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi -net, On 24/09/15 09:03, Julien Charbon wrote: > On 24/09/15 08:55, Palle Girgensohn wrote: >>> 24 sep 2015 kl. 07:51 skrev Palle Girgensohn >>> : >>>> 24 sep 2015 kl. 00:05 skrev Palle Girgensohn >>>> : >>>>> 23 sep 2015 kl. 20:32 skrev Julien Charbon :=20 >>>>> On 23/09/15 20:26, Palle Girgensohn wrote: >>>> Kernels and userland are updated to 10.2-p3 with the patch >>>> removing the suspicous KASSERT. >>>> dtrace running continously redirecting to a log file. >> Just had a crash. Unfortunately, the kernel was stuck at the db> >> prompt, and the remote keyboard was unresponsive (HP ILO, not >> impressed). So I had to reset the power and never got a core dump... >> >> panic: tcp_tw_2msl_stop: inp should not be released here >> cpuid =3D 0 >> KDB: stack backtrace: >> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame >> 0xfffffe175acd16a0 kdb_backtrace() at kdb_backtrace+0x39/frame >> 0xfffffe175acd1750 vpanic() at vpanic+0x126/frame 0xfffffe175acd1790 >> kassert_panic() at kassert_panic+0x139/frame 0xfffffe175acd1800 >> tcp_twclose() at tcp_twclose+0x2cb/frame 0xfffffe175acd1850 >> tcp_tw_2msl_scan() at tcp_tw_2msl_scan+0x13b/frame >> 0xfffffe175acd1890 tcp_slowtimo() at tcp_slowtimo+0x68/frame >> 0xfffffe175acd18c0 pfslowtimo() at pfslowtimo+0x54/frame >> 0xfffffe175acd18f0 softclock_call_cc() at >> softclock_call_cc+0x193/frame 0xfffffe175acd19d0 softclock() at >> softclock+0x47/frame 0xfffffe175acd19f0 intr_event_execute_handlers() >> at intr_event_execute_handlers+0x93/frame 0xfffffe 175acd1a30 >> ithread_loop() at ithread_loop+0xa6/frame 0xfffffe175acd1a70 >> fork_exit() at fork_exit+0x84/frame 0xfffffe175acd1ab0 >> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe175acd1ab0 >> --- trap 0, rip =3D 0, rsp =3D 0xfffffe175acd1b70, rbp =3D 0 --- >> KDB: enter: panic >> [ thread pid 12 tid 100043 ] >> Stopped at kdb_enter+0x3e: movq $0,kdb_why >> db> >=20 > Thanks a log for this backstrace. This is what at expected, when > tcp_close() in call in INP_TIMEWAIT case, in_pcbfree() can be called on= e > extra time that leads to: >=20 > tcp_tw_2msl_stop: inp should not be released here >=20 > Let me try to come with a tentative fix for this case. See joined my tentative patch for these case. It is only a first tentative patch as I am still waiting on -net feedbacks on what should be the rule here. By the way: - I see nothing specific to VIMAGE here - Anyone aware of tcp_close() (or tcp_drop()) calls modified/introduced recently in 10.2 that could explained why this issue only appears only no= w? -- Julien --------------010008010008030404040209 Content-Type: text/plain; charset=UTF-8; name="tcp-close-fix-v1.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="tcp-close-fix-v1.patch" diff --git a/sys/netinet/tcp_usrreq.c b/sys/netinet/tcp_usrreq.c index be9e0e7..4379e19 100644 --- a/sys/netinet/tcp_usrreq.c +++ b/sys/netinet/tcp_usrreq.c @@ -199,10 +199,11 @@ tcp_detach(struct socket *so, struct inpcb *inp) * In all three cases the tcptw should not be freed here. */ if (inp->inp_flags & INP_DROPPED) { - KASSERT(tp =3D=3D NULL, ("tcp_detach: INP_TIMEWAIT && " - "INP_DROPPED && tp !=3D NULL")); in_pcbdetach(inp); - in_pcbfree(inp); + if (tp =3D=3D NULL) + in_pcbfree(inp); + else + INP_WUNLOCK(inp); } else { in_pcbdetach(inp); INP_WUNLOCK(inp); --------------010008010008030404040209-- --2xKnTLrEPFmUogFPILk9iCAhuv6Q8h3jH Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJWA6z/AAoJEKVlQ5Je6dhx6LUIAK7m6qHVcm75uthiiz44kUq6 qt5OaGmCvvRKuCY1czSXGScQKf6SdQ0njIh8Mu3ZgSUaq6rEI0hB5XWf73vu48Cm 5U41urew/qp3myahlpYn4qrTRr+hO7tFrmQpXkHW31T/a7oAIYE/F+t35P7pxQWI 2HcUcjYkwqShTFAlonSqof5mBRX8YquFnQ0BQ3Jmi80wYoO0eBiZJE2ut3BhSWE0 YkVjui1eoPpxoMuwy2KCuFF72GrhJBJe+NL30lR5W/FhJQu1tf1Yp4eqcANAjn+J ZyYYu3Zt+JFFaQfJabTJIQkQBIvZx0E79f1iJIXbge7a7vq8megbfTRpUPAgQ0I= =/wdk -----END PGP SIGNATURE----- --2xKnTLrEPFmUogFPILk9iCAhuv6Q8h3jH--