From owner-freebsd-current@FreeBSD.ORG Fri Jun 6 15:33:29 2014 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from hub.FreeBSD.org (hub.freebsd.org [IPv6:2001:1900:2254:206c::16:88]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3CA1F890; Fri, 6 Jun 2014 15:33:29 +0000 (UTC) Date: Fri, 6 Jun 2014 11:33:24 -0400 From: Glen Barber To: freebsd-current@FreeBSD.org Subject: Re: panic in deadlkres() on r267110 Message-ID: <20140606153324.GK33882@hub.FreeBSD.org> References: <20140606141215.GE33882@hub.FreeBSD.org> <1402064629.1123.46.camel@bruno> <20140606150640.GI33882@hub.FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="GmiNL4+5WUWrod5m" Content-Disposition: inline In-Reply-To: <20140606150640.GI33882@hub.FreeBSD.org> X-Operating-System: FreeBSD 11.0-CURRENT amd64 X-SCUD-Definition: Sudden Completely Unexpected Dataloss X-SULE-Definition: Sudden Unexpected Learning Event User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jun 2014 15:33:29 -0000 --GmiNL4+5WUWrod5m Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jun 06, 2014 at 11:06:40AM -0400, Glen Barber wrote: > On Fri, Jun 06, 2014 at 07:23:49AM -0700, Sean Bruno wrote: > > On Fri, 2014-06-06 at 10:12 -0400, Glen Barber wrote: > > > Two machines in the cluster panic last night with the same backtrace. > > > It is unclear yet exactly what was happening on the systems, but both > > > are port building machines using ports-mgmt/tinderbox. > > >=20 > > > Any ideas or information on how to further debug this would be > > > appreciated. > > >=20 > > These machines were happily running r266621 previously to this update > > yesterday. So, that gives us a bisection point. > >=20 >=20 > Some more debug information. Thank you to Attilio for information on > what data to get. >=20 Sorry, I did not mask the bits properly. Script started on Fri Jun 6 15:20:14 2014 command: /bin/sh # kgdb ./kernel.debug /var/crash/vmcore.0 [...] #0 doadump (textdump=3D-946873840) at pcpu.h:219 219 __asm("movq %%gs:%1,%0" : "=3Dr" (td) (kgdb) p/x allproc_lock.sx_lock $1 =3D 0xfffff813ae7f4924 Current language: auto; currently minimal (kgdb) p ((struct thread *)0xfffff813ae7f4920) $4 =3D (struct thread *) 0xfffff813ae7f4920 (kgdb) p $4->td_tid $5 =3D 195038 (kgdb) tid 195038 [Switching to thread 94414 (Thread 195038)]#0 0xffffffff80daf1f8 in cpusto= p_handler () at /usr/src/sys/amd64/amd64/mp_machdep.c:1432 1432 savectx(&stoppcbs[cpu]); (kgdb) bt #0 0xffffffff80daf1f8 in cpustop_handler () at /usr/src/sys/amd64/amd64/mp_machdep.c:1432 #1 0xffffffff80daf1bf in ipi_nmi_handler () at /usr/src/sys/amd64/amd64/mp_machdep.c:1417 #2 0xffffffff80dbfcba in trap (frame=3D0xfffffe17d5389f30) at /usr/src/sys/amd64/amd64/trap.c:189 #3 0xffffffff80da2e03 in nmi_calltrap () at /usr/src/sys/amd64/amd64/exception.S:504 #4 0xffffffff8092fa39 in fork1 (td=3D0xfffff813ae7f4920, flags=3D20, pages= =3DCannot access memory at address 0x4 ) at /usr/src/sys/kern/kern_fork.c:268 #5 0xffffffff8092f00f in sys_fork (td=3D0xfffff813ae7f4920,=20 uap=3D) at /usr/src/sys/kern/kern_fork.c:106 #6 0xffffffff80dc118b in amd64_syscall (td=3D0xfffff813ae7f4920, traced=3D= 0) at subr_syscall.c:133 #7 0xffffffff80da2cab in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:390 #8 0x0000000800d46d1a in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb)=20 Glen --GmiNL4+5WUWrod5m Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBCAAGBQJTkd9EAAoJELls3eqvi17Q9ZsP/1h/W/y6BDVx1Iu83LXNXei2 KCRUzaQcDmLsswqDRht294mKXH+b9WMrNaE4ewfrD+Sbq9tt+lsyPImoOfcCKRz0 iX7UVTs1V/LzE+TqT8kYTZOC1qqe5/8KO8yzDJ2eY2U6N2jsM6P1ZAb6oBk+W7QM 71ZUkJPPUJmNQDirpTU44CqEPuxM4chkaFj9bt+UFyo3/Qg2m5IMx1YYkNPGAAXv 5Mn51uYllefaMUbxCB00miq+Z60ZYIfEoHePDrNNm9VOm9gf9xkPhbIkewUH1oR+ eiM0d7Qvlv5pGr2nFuXrAXBI5bKcbTH8rFuEQztGSiB7MwlxAjGXuOMe4BMaZTpK W3KM8KlS3dsLUopivvUABFea5uJdpHYrBXrFKSW/lY80ubpqGa355hCDU4famirC YspPBcno6kdahEZ8XQqn5Ad8Mblxwp7wmq5bMOX/dSmADihzkV3hSvbrzgC8QvDy dJfhBKvpGch7a92AbcpZrApSIOWxF0FU9eN3/KHs7+dGXz0YeW28CG1lmuDjhtVA QzYP0KFfYOUkKP+1JGgSA7YQwIA4QMY3SuOn7zo9edVDtxNygCfOnhHFdOqE6ZTf m8kOaswyLcETz4Nt5Mt1wanaFWuopYMrymztcxzQq3T5Dg94cAVJ3BcEkzTibrQz DUbiatuMqtLNUAhH3sBA =F3JJ -----END PGP SIGNATURE----- --GmiNL4+5WUWrod5m--