From owner-freebsd-stable@FreeBSD.ORG Sat Aug 8 15:48:59 2009 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D6BE51065672 for ; Sat, 8 Aug 2009 15:48:59 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (skuns.zoral.com.ua [91.193.166.194]) by mx1.freebsd.org (Postfix) with ESMTP id 706858FC24 for ; Sat, 8 Aug 2009 15:48:58 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id n78FmrSi013319 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 8 Aug 2009 18:48:54 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.3/8.14.3) with ESMTP id n78Fmrgw065758; Sat, 8 Aug 2009 18:48:53 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.3/8.14.3/Submit) id n78FmrbJ065757; Sat, 8 Aug 2009 18:48:53 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 8 Aug 2009 18:48:53 +0300 From: Kostik Belousov To: pluknet Message-ID: <20090808154853.GZ1884@deviant.kiev.zoral.com.ua> References: <20090807115309.GW1884@deviant.kiev.zoral.com.ua> <20090807124609.GX1884@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="h1d7ksSqXDyyuV0m" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-stable Subject: Re: panic in vgonel() X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Aug 2009 15:49:00 -0000 --h1d7ksSqXDyyuV0m Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Aug 07, 2009 at 04:48:37PM +0400, pluknet wrote: > 2009/8/7 Kostik Belousov : > > On Fri, Aug 07, 2009 at 04:37:07PM +0400, pluknet wrote: > >> 2009/8/7 Kostik Belousov : > >> > On Fri, Aug 07, 2009 at 03:37:11PM +0400, pluknet wrote: > >> >> This is on 7.2-R amd64. > >> >> > >> >> I'm curious if it might be due to glusterfs on it. > >> >> > >> >> Fatal trap 12: page fault while in kernel mode > >> >> cpuid =3D 3; apic id =3D 03 > >> >> fault virtual address =9A =3D 0x0 > >> >> fault code =9A =9A =9A =9A =9A =9A =9A=3D supervisor write data, pa= ge not present > >> >> instruction pointer =9A =9A =3D 0x8:0xffffffff805a52ba > >> >> stack pointer =9A =9A =9A =9A =9A =3D 0x10:0xfffffffefc3474a0 > >> >> frame pointer =9A =9A =9A =9A =9A =3D 0x10:0xfffffffefc347510 > >> >> code segment =9A =9A =9A =9A =9A =9A=3D base 0x0, limit 0xfffff, ty= pe > >> >> =9A =9A =9A =9A =9A =9A =9A =9A =9A =9A =9A =9A =3D DPL 0, pres 1, = long 1, def32 0, gran 1 > >> >> processor eflags =9A =9A =9A =9A=3D interrupt enabled, resume, IOPL= =3D 0 > >> >> current process =9A =9A =9A =9A =3D 35425 (find) > >> >> > >> >> db> bt > >> >> Tracing pid 35425 tid 100194 td 0xffffff003c165370 > >> >> vgonel() at vgonel+0x1aa > >> >> vnlru_free() at vnlru_free+0x36c > >> >> getnewvnode() at getnewvnode+0x281 > >> >> ffs_vgetf() at ffs_vgetf+0xdf > >> >> ufs_lookup() at ufs_lookup+0x2dd > >> >> vfs_cache_lookup() at vfs_cache_lookup+0xf3 > >> >> VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x40 > >> >> lookup() at lookup+0x598 > >> >> namei() at namei+0x33e > >> >> kern_lstat() at kern_lstat+0x5e > >> >> lstat() at lstat+0x2a > >> >> syscall() at syscall+0x256 > >> >> Xfast_syscall() at Xfast_syscall+0xab > >> >> --- syscall (190, FreeBSD ELF64, lstat), rip =3D 0x80071063c, rsp = =3D > >> >> 0x7fffffffea48, rbp =3D 0x800a06910 --- > >> > > >> > Did you got the vmcore ? If yes, please find the value for vgonel() > >> > argument, vp, and print the vnode content. > >> > >> I didn't. Same problem as in my another mail. :( > >> > >> > > >> > Regardless of this, look up the source line for vgonel+0x1aa. > >> > > >> > >> I could resolve only address which belongs to instruction pointer > >> =3D 0x8:0xffffffff805a52ba > >> (eh, I don't know if I should sum these numbers, so I did this for bot= h cases): > >> > >> dev2# addr2line -e /boot/kernel/kernel.symbols 0xffffffff805a52ba > >> /usr/src/sys/kern/vfs_subr.c:979 > >> delmntque(): =9A =9A =9A =9ATAILQ_REMOVE(&mp->mnt_nvnodelist, vp, v_nm= ntvnodes); > >> > >> dev2# addr2line -e /boot/kernel/kernel.symbols 0xffffffff805a52c2 > >> /usr/src/sys/kern/vfs_subr.c:981 > >> delmntque(): =9A =9A =9A =9AMNT_REL(mp); > > > > load kernel.debug into gdb, and then do "list *(vgonel+0x1aa)" > > >=20 > Ah, of course. Sorry. >=20 > (gdb) list *(vgonel+0x1aa) > 0xffffffff805a52ba is in vgonel (/usr/src/sys/kern/vfs_subr.c:979). > 974 return; > 975 MNT_ILOCK(mp); > 976 vp->v_mount =3D NULL; > 977 VNASSERT(mp->mnt_nvnodelistsize > 0, vp, > 978 ("bad mount point vnode list size")); > 979 TAILQ_REMOVE(&mp->mnt_nvnodelist, vp, v_nmntvnodes); > 980 mp->mnt_nvnodelistsize--; > 981 MNT_REL(mp); > 982 MNT_IUNLOCK(mp); > 983 } If you can reproduce the issue at will, please try to do reproduce it on the kernel with INVARIANTS and possibly DEBUG_VFS_LOCKS on. --h1d7ksSqXDyyuV0m Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (FreeBSD) iEYEARECAAYFAkp9nmUACgkQC3+MBN1Mb4jVEQCePrkxYLu2/zzXtin4otUSCeO0 4j0AoIHMUwFSljUt/qsI8mu0qwH/QsUl =uNaY -----END PGP SIGNATURE----- --h1d7ksSqXDyyuV0m--