From owner-freebsd-net@FreeBSD.ORG Sat Sep 28 22:45:06 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id D31BA983; Sat, 28 Sep 2013 22:45:06 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 71EB22722; Sat, 28 Sep 2013 22:45:06 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.7/8.14.7) with ESMTP id r8SMj16J017182; Sun, 29 Sep 2013 01:45:01 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua r8SMj16J017182 Received: (from kostik@localhost) by tom.home (8.14.7/8.14.7/Submit) id r8SMj1YZ017181; Sun, 29 Sep 2013 01:45:01 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 29 Sep 2013 01:45:01 +0300 From: Konstantin Belousov To: jfv@freebsd.org Subject: Re: igb(4) panic: already enqueue Message-ID: <20130928224501.GX41229@kib.kiev.ua> References: <20130928213038.GT41229@kib.kiev.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="XRLv71ZoqfThaM/0" Content-Disposition: inline In-Reply-To: <20130928213038.GT41229@kib.kiev.ua> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Sep 2013 22:45:07 -0000 --XRLv71ZoqfThaM/0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Sep 29, 2013 at 12:30:38AM +0300, Konstantin Belousov wrote: > On a relatively old test machine, dual-socket X7DWU supermicro, Xeon > X5272, with on-board 82575EB igb controllers, running the stress2 > udp blast test immediately causes the following >=20 > panic: buf=3D0xfffff8001aa5c800 already enqueue at 1445 prod=3D1315 cons= =3D1445 > cpuid =3D 3 > KDB: stack backtrace: > db_trace_self_wrapper() at 0xffffffff802839ab =3D db_trace_self_wrapper+0= x2b/frame 0xfffffe0123570600 > vpanic() at 0xffffffff8033bd96 =3D vpanic+0x126/frame 0xfffffe0123570640 > panic() at 0xffffffff8033be23 =3D panic+0x43/frame 0xfffffe01235706a0 > igb_mq_start() at 0xffffffff80bcb228 =3D igb_mq_start+0xb8/frame 0xfffffe= 0123570710 =20 > ether_output() at 0xffffffff804072a8 =3D ether_output+0x5c8/frame 0xfffff= e0123570780 =20 > ip_output() at 0xffffffff80430be1 =3D ip_output+0xe91/frame 0xfffffe01235= 70860 > udp_send() at 0xffffffff804a5fc4 =3D udp_send+0x834/frame 0xfffffe0123570= 910 > sosend_dgram() at 0xffffffff803b517b =3D sosend_dgram+0x36b/frame 0xfffff= e0123570990 =20 > soo_write() at 0xffffffff8039dbb9 =3D soo_write+0x49/frame 0xfffffe012357= 09d0 > dofilewrite() at 0xffffffff803956c5 =3D dofilewrite+0x85/frame 0xfffffe01= 23570a20 > kern_writev() at 0xffffffff803953d5 =3D kern_writev+0x65/frame 0xfffffe01= 23570a70 > sys_write() at 0xffffffff80395363 =3D sys_write+0x63/frame 0xfffffe012357= 0ac0 > amd64_syscall() at 0xffffffff8053ae7d =3D amd64_syscall+0x28d/frame 0xfff= ffe0123570bf0 >=20 > Test is available at > svn:base/user/pho/stress2/testcases/udp, invocation line is > BLASTHOST=3D192.168.5.1 ./udp -t 10m -i 50 -h -v > Substitute the BLASTHOST with the ip address of a scratch box. >=20 > This is on the yesterday HEAD. Could you, please, fix this ? I also tested on i350 adapters, there the issue is harder to reproduce, but it still panics under the test. Also, I saw the following: Expensive timeout(9) function: 0xffffffff809c7d60(0xfffffe0001864000) 1.929= 7931s Expensive timeout(9) function: 0xffffffff809c7d60(0xfffffe0001864000) 9.966= 9852s db> x/i 0xffffffff809c7d60 0xffffffff809c7d60 =3D igb_local_timer: pushq %rbp --XRLv71ZoqfThaM/0 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.21 (FreeBSD) iQIbBAEBAgAGBQJSR1vsAAoJEJDCuSvBvK1BTCUP+P79Xa3gbBhomSQUboyqW/hZ 1isDKCguScJo6hOW/R1iltnkbn3RMCJNBQS9MBM9dWdiVGI0orBolsk/XKKKCT8l W2pPWjlInN0jUPqoBDj+flF1eM9d0EDnuJVMKIlESMD5cuY+bpMm+OLZo6u8qqwR 6O1Db2kUt4kgqhbjyrjTj65Isdrll0Pa5vSMGMQe1/Y9UI3aBJQVxmJdsgBE8IbZ h3x4Vus6fgzBHjikrYQGer8pUCnc32BfpsJfNiiyKB5ki3ix6mXn6BfAj197r1H+ N++nJ4g6UvHl9YfNoSxRae981FDuqwYod1PfYm1RkMQh9UaB6l+hkihhDI+ncqdS 8NjC4tDaF+NGtuKFc/sX6PTxbrJ4Y5uxUU6Rc3xdcW9wb+7ZHwnDfh/kBumtzh2w sFEVsclx5XgCZfU6PjHLhHs3KawB37jRiiS/5k/98P9ohkOQvRQyuAZMqsghkvyL upRrFl+MtA7koyoZBMvGPqvdA3/ROXW2njUiQitlLPuj/s6KJVUKF9uqGL+hbVjW F2Y9mMM1N7BfsiuEkW0pDXENWquBiu4LcKGo+c3WnsJH5JDkxPblSkYiwvg2LAdj ozf2GmP+6Ug5DEsDfZpMm6PDp5TqPI8rfLmMU6h+FjfCe8TqGQt76x3gCBoAtixt z7gMtMRV1XhQEqkXu44= =c29f -----END PGP SIGNATURE----- --XRLv71ZoqfThaM/0--