Date: Mon, 25 Nov 2013 10:10:47 +0200 From: Konstantin Belousov <kostikbel@gmail.com> To: Don Lewis <truckman@FreeBSD.org> Cc: freebsd-current@FreeBSD.org Subject: Re: panic: double fault with 11.0-CURRENT r258504 Message-ID: <20131125081047.GN59496@kib.kiev.ua> In-Reply-To: <201311240743.rAO7hU9e026518@gw.catspoiler.org> References: <201311240743.rAO7hU9e026518@gw.catspoiler.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--nqw1/iq3c1XOUvXw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Sat, Nov 23, 2013 at 11:43:30PM -0800, Don Lewis wrote:
> I upgraded my 11.0-CURRENT machine to r258504 to get past the uma panic
> that I stumbled across earlier. Now I got this when I started upgrading
> ports:
>=20
> Unread portion of the kernel message buffer:
>=20
> Fatal double fault:
> eip =3D 0xc0b158e0
> esp =3D 0xe4f62000
> ebp =3D 0xe4f62010
> cpuid =3D 0; apic id =3D 00
> panic: double fault
> cpuid =3D 0
> KDB: stack backtrace:
> db_trace_self_wrapper(c113340c,2,10000000,c15a0cf0,c15a0ce8,...) at db_tr=
ace_self_wrapper+0x2d/frame 0xc15a0cb0
> kdb_backtrace(c12f143f,0,c12f2aea,c15a0d6c,0,...) at kdb_backtrace+0x30/f=
rame 0xc15a0d18
> vpanic(c15a0d6c,c15a0d84,c0fc14fb,c12f2aea,0,...) at vpanic+0x11f/frame 0=
xc15a0d54
> panic(c12f2aea,0,0,0,e4f62010,...) at panic+0x12/frame 0xc15a0d60
> dblfault_handler() at dblfault_handler+0xab/frame 0xc15a0d60
> --- trap 0x17, eip =3D 0xc0b158e0, esp =3D 0xe4f62000, ebp =3D 0xe4f62010=
---
> vprintf(c12f2900,c,fffffe7f,fffffeff,bfff75ed,...) at vprintf/frame 0xe4f=
62010
> trap(e4f62164) at trap+0x18a/frame 0xe4f62158
> calltrap() at calltrap+0x6/frame 0xe4f62158
> --- trap 0xc, eip =3D 0xc0b145dd, esp =3D 0xe4f621a4, ebp =3D 0xe4f62270 =
---
> kvprintf(c12f2900,c0b15210,e4f62290,a,e4f6235c,...) at kvprintf+0x1cd/fra=
me 0xe4f62270
> vprintf(c12f2900,e4f6235c,e4f6235c) at vprintf+0x7f/frame 0xe4f6233c
> printf(c12f2900,c,ffefdfff,ebefefff,dfdffedf,...) at printf+0x1b/frame 0x=
e4f62350
> trap(e4f624a4) at trap+0x18a/frame 0xe4f62498
> calltrap() at calltrap+0x6/frame 0xe4f62498
> --- trap 0xc, eip =3D 0xc0b145dd, esp =3D 0xe4f624e4, ebp =3D 0xe4f625b0 =
---
> kvprintf(c12f2900,c0b15210,e4f625d0,a,e4f6269c,...) at kvprintf+0x1cd/fra=
me 0xe4f625b0
> vprintf(c12f2900,e4f6269c,e4f6269c) at vprintf+0x7f/frame 0xe4f6267c
> printf(c12f2900,c,5fd7ff5f,ba77f7fb,bfffb7ff,...) at printf+0x1b/frame 0x=
e4f62690
> trap(e4f627e4) at trap+0x18a/frame 0xe4f627d8
> calltrap() at calltrap+0x6/frame 0xe4f627d8
> --- trap 0xc, eip =3D 0xc0b145dd, esp =3D 0xe4f62824, ebp =3D 0xe4f628f0 =
---
> kvprintf(c12f2900,c0b15210,e4f62910,a,e4f629dc,...) at kvprintf+0x1cd/fra=
me 0xe4f628f0
> vprintf(c12f2900,e4f629dc,e4f629dc) at vprintf+0x7f/frame 0xe4f629bc
> printf(c12f2900,c,0,80000000,c0,...) at printf+0x1b/frame 0xe4f629d0
> trap(e4f62b20) at trap+0x18a/frame 0xe4f62b14
> calltrap() at calltrap+0x6/frame 0xe4f62b14
> --- trap 0xc, eip =3D 0xc0afe270, esp =3D 0xe4f62b60, ebp =3D 0xe4f62b78 =
---
> tdq_choose(c141e090,4,c113144d,917,c2425c80,...) at tdq_choose+0x60/frame=
0xe4f62b78
> sched_choose(e4f62c00,c0afc511,c141e090,14,c113144d,...) at sched_choose+=
0x4c/frame 0xe4f62ba4
> choosethread(c141e090,14,c113144d,78b,c141e116,...) at choosethread+0x1f/=
frame 0xe4f62bac
> sched_switch(c8f04000,0,608,1b7,ef2,...) at sched_switch+0x361/frame 0xe4=
f62c00
> mi_switch(608,0,c112f4e4,d3,c,...) at mi_switch+0x1c9/frame 0xe4f62c34
> critical_exit(0,2,c113144d,411,c141e108,...) at critical_exit+0xa4/frame =
0xe4f62c50
> sched_idletd(0,e4f62d08,c1128634,3db,0,...) at sched_idletd+0x1d6/frame 0=
xe4f62ccc
> fork_exit(c0afeb00,0,e4f62d08) at fork_exit+0x7f/frame 0xe4f62cf4
> fork_trampoline() at fork_trampoline+0x8/frame 0xe4f62cf4
> --- trap 0, eip =3D 0, esp =3D 0xe4f62d40, ebp =3D 0 ---
> KDB: enter: panic
>=20
> (kgdb) list *tdq_choose+0x60
> 0xc0afe270 is in tdq_choose (/usr/src/sys/kern/sched_ule.c:1334).
> 1329 td =3D runq_choose(&tdq->tdq_realtime);
> 1330 if (td !=3D NULL)
> 1331 return (td);
> 1332 td =3D runq_choose_from(&tdq->tdq_timeshare, tdq->tdq_ridx);
> 1333 if (td !=3D NULL) {
> 1334 KASSERT(td->td_priority >=3D PRI_MIN_BATCH,
> 1335 ("tdq_choose: Invalid priority on timeshare queue %d",
> 1336 td->td_priority));
> 1337 return (td);
> 1338 }
>=20
> (kgdb) bt
> #0 doadump (textdump=3D-1051128300) at pcpu.h:233
> #1 0xc052766d in db_fncall (dummy1=3D-1051051648, dummy2=3D0, dummy3=3D-=
1051063684,=20
> dummy4=3D0xc15a0a54 "") at /usr/src/sys/ddb/db_command.c:578
> #2 0xc0527357 in db_command (cmd_table=3D<value optimized out>)
> at /usr/src/sys/ddb/db_command.c:449
> #3 0xc0527090 in db_command_loop () at /usr/src/sys/ddb/db_command.c:502
> #4 0xc0529922 in db_trap (type=3D<value optimized out>, code=3D0)
> at /usr/src/sys/ddb/db_main.c:231
> #5 0xc0b0ff38 in kdb_trap (type=3D<value optimized out>,=20
> code=3D<value optimized out>, tf=3D<value optimized out>)
> at /usr/src/sys/kern/subr_kdb.c:656
> #6 0xc0fc0c07 in trap (frame=3D<value optimized out>)
> at /usr/src/sys/i386/i386/trap.c:712
> #7 0xc0faa0ec in calltrap () at /usr/src/sys/i386/i386/exception.s:170
> #8 0xc0b0f7bd in kdb_enter (why=3D0xc112ee39 "panic", msg=3D<value optim=
ized out>)
> at cpufunc.h:71
> #9 0xc0ad6a93 in vpanic (fmt=3D<value optimized out>, ap=3D<value optimi=
zed out>)
> at /usr/src/sys/kern/kern_shutdown.c:747
> #10 0xc0ad6ad2 in panic (fmt=3D0xc12f2aea "double fault")
> at /usr/src/sys/kern/kern_shutdown.c:683
> #11 0xc0fc14fb in dblfault_handler () at /usr/src/sys/i386/i386/trap.c:10=
72
> #12 0x00000000 in ?? ()
It seems to be a corruption of the td and probably curthread.
Is it repeatable easily ? If yes, you could try to manually inspect first
elements in the (idle) runq queue of the tdq_cpu[paniced cpu].
--nqw1/iq3c1XOUvXw
Content-Type: application/pgp-signature
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (FreeBSD)
iQIbBAEBAgAGBQJSkwYGAAoJEJDCuSvBvK1B08QP9R7nxcxz0UFC/F783Oe2lVbe
iEtmyLKHdJ0qDE5xhcqWw4LOA+kEGWnWItNdH3OjBtwhwv4mo03y1gJO9Bvmn4lz
Ja34J1qYQgiyWcUKsIobsjZRZwZfAK32YIrtkPq6FCOhz6l08KYJcSMWLhG9MgDk
HZAMaJBWwljtAItl38ZXYqfwLsuQfjVQIRpITNB2f0ckj1p6DznwekFQNsCACA5Z
kyqyZIm8FiTO9TDeY6/JD+h2EpKRdxExsD0uBpIyT6Uz9uSq99xFzk7QF6dYRrGU
73gM47CqDreT6BqP+kieAJ46aVLxvLqVJs8QLEZQhxdptRBX7xocCoZVWWylZLeQ
cSJMShcifRqwZDSEAohuZSaT1X3+iCbjgp5Zgqk9fLif+k1xzxJHGFY80m6aCh9B
URlAKGDf8Iv6k3BaQ35jG5bc7i1KJXXi2MlfYg2Juz+tqqSRNRg21tg9fHUNTXKf
pVblKRqLLCyVg2v41+9Oyzd7maZlhjh11nUsDCWIg/Kqo0xYgo/2Y27ck/USsn3R
uorfBZ23osp2IWnu1Xau6iH806N7c9KSCTaT2TjZB7kBQ57PFT1nhtCXP1V5/gjW
p6Bim2ze3ZNMxDTuYofqWxhQL2PDYeQAxWGK/q0CJjsJ1JufP472RpSndLQ40Z85
M5S0jbAGnFPZSmxGFv4=
=OBoQ
-----END PGP SIGNATURE-----
--nqw1/iq3c1XOUvXw--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20131125081047.GN59496>