Date: Fri, 15 Dec 2006 23:20:40 +0200 From: Kostik Belousov <kostikbel@gmail.com> To: Kris Kennaway <kris@obsecurity.org> Cc: Sven Willenberger <sven@dmv.com>, stable@freebsd.org Subject: Re: Not panic in nfsd (Re: panic in nfsd on 6.2-RC1) Message-ID: <20061215212040.GG23698@deviant.kiev.zoral.com.ua> In-Reply-To: <20061215192958.GA86926@xor.obsecurity.org> References: <20061205.004323.78708386.hrs@allbsd.org> <20061204160949.GM35681@deviant.kiev.zoral.com.ua> <20061205.123805.59655403.hrs@allbsd.org> <1166194879.6317.11.camel@lanshark.dmv.com> <20061215181548.GA58555@xor.obsecurity.org> <1166209936.6317.21.camel@lanshark.dmv.com> <20061215192958.GA86926@xor.obsecurity.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--9Iq5ULCa7nGtWwZS Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Dec 15, 2006 at 02:29:58PM -0500, Kris Kennaway wrote: > On Fri, Dec 15, 2006 at 02:12:16PM -0500, Sven Willenberger wrote: > > On Fri, 2006-12-15 at 13:15 -0500, Kris Kennaway wrote: > > > On Fri, Dec 15, 2006 at 10:01:19AM -0500, Sven Willenberger wrote: > > > > On Tue, 2006-12-05 at 12:38 +0900, Hiroki Sato wrote: > > > > > Kostik Belousov <kostikbel@gmail.com> wrote > > > > > in <20061204160949.GM35681@deviant.kiev.zoral.com.ua>: > > > > >=20 > > > > > ko> What version of sys/nfsserver/nfs_serv.c do you use ? If it i= s older than > > > > > ko> 1.156.2.7, please, update the system. > > > > >=20 > > > > > Thanks, I updated it just now and see how it works. > > > > >=20 > > > > > -- > > > > > | Hiroki SATO > > > >=20 > > > > I was/am having the same issue. Updating world (6.2-stable) to incl= ude > > > > the above update sadly did not fix the problem for me. This is an a= md64 > > > > box with only one client connecting to it via nfs. Reading further = it > > > > may seem to be an issue with rpc.statd and/or rpc.lockd. As I only = have > > > > one client connecting and it is being used as mail storage (i.e. the > > > > client pops/imaps the storage) would be safe to not using fcntl for= wards > > > > over the wire? Is this same issue present in 6.1-RELENG? I am reall= y at > > > > my wits end at this point and for the first time am actually consid= ering > > > > moving to another OS (solaris more than likely) as I cannot have th= ese > > > > types of issues interrupting services every couple days. > > > >=20 > > > > What other information (spefically) can I provide to help the devs > > > > figure out what is going on? What can I do in the meantime to have = some > > > > semblence of stability? I assume downgrading to 5.5-RELENG is out o= f the > > > > question but perhaps disabling SMP? > > >=20 > > > Just to confirm, can you please post the panic backtrace you are > > > seeing? And can you explain what you mean by "may seem to be an issue > > > with rpc.statd and/or rpc.lockd"? > > >=20 > > > Sometimes people think they're seeing the same problem as someone else > > > when really it's a completely different problem in the same subsystem, > > > so I'd like to rule that out here. > > >=20 > > > Kris > >=20 > > Well I have now added kdb and invariants/witness support to the kernel > > so I should be able to get some backtrace the next time it happens. > > Currently, the system just locks and no error is displayed on the > > console or /var/log/messages; sorry I cannot be of immediate help there. >=20 > OK, so your issue is not, in fact, a "panic in nfsd" as you were > claiming ;-) >=20 > > Regarding the rpc issue, I just ran across mention of those in sshfs/nfs > > threads appearing here and in particular to a link referenced within one > > of them (http://docs.freebsd.org/cgi/getmsg.cgi?fetch=3D1362611+0 > > +archive/2006/freebsd-stable/20060702.freebsd-stable ) - it is more than > > likely not at all related but I am grasping at straws here trying to > > solve this. >=20 > Yes, I think you are grasping at straws. At this point, you need to > do some debugging to find out the source of your problem, and treat it > as a new bug until you find conclusive evidence that it's the same as > a previously reported bug. >=20 > Guessing without evidence that your problem is the same as someone > else's problem, because e.g. both involve your system becoming > unresponsive, is a very good way to confuse the issue and delay > resolution. > =20 > > FWIW, I do see the following appearing in the /var/log/messages: > > ufs_rename: fvp =3D=3D tvp (can't happen)=20 > > about once or twice a day, but cannot correlate those to lockup. Now > > that I have enabled the options mentioned above in the kernel, I am > > seeing some LOR issues: > >=20 > > kernel: lock order reversal: > > kernel: 1st 0xffffff00c3bab200 kqueue (kqueue) @ /usr/src/sys/kern/kern= _event.c:1547 > > kernel: 2nd 0xffffff0005bb6078 struct mount mtx (struct mount mtx) @ /u= sr/src/sys/ufs/ufs/ufs_vnops.c:138 >=20 > OK, this is interesting, so let's proceed from here. >=20 > Kris Try this. Index: ufs/ufs/ufs_vnops.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /usr/local/arch/ncvs/src/sys/ufs/ufs/ufs_vnops.c,v retrieving revision 1.283 diff -u -r1.283 ufs_vnops.c --- ufs/ufs/ufs_vnops.c 6 Nov 2006 13:42:09 -0000 1.283 +++ ufs/ufs/ufs_vnops.c 15 Dec 2006 21:19:51 -0000 @@ -133,19 +133,15 @@ { struct inode *ip; struct timespec ts; - int mnt_locked; =20 ip =3D VTOI(vp); - mnt_locked =3D 0; - if ((vp->v_mount->mnt_flag & MNT_RDONLY) !=3D 0) { - VI_LOCK(vp); + VI_LOCK(vp); + if ((vp->v_mount->mnt_flag & MNT_RDONLY) !=3D 0) goto out; + if ((ip->i_flag & (IN_ACCESS | IN_CHANGE | IN_UPDATE)) =3D=3D 0) { + VI_UNLOCK(vp); + return; } - MNT_ILOCK(vp->v_mount); /* For reading of mnt_kern_flags. */ - mnt_locked =3D 1; - VI_LOCK(vp); - if ((ip->i_flag & (IN_ACCESS | IN_CHANGE | IN_UPDATE)) =3D=3D 0) - goto out_unl; =20 if ((vp->v_type =3D=3D VBLK || vp->v_type =3D=3D VCHR) && !DOINGSOFTDEP(v= p)) ip->i_flag |=3D IN_LAZYMOD; @@ -172,10 +168,7 @@ =20 out: ip->i_flag &=3D ~(IN_ACCESS | IN_CHANGE | IN_UPDATE); - out_unl: VI_UNLOCK(vp); - if (mnt_locked) - MNT_IUNLOCK(vp->v_mount); } =20 /* --9Iq5ULCa7nGtWwZS Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (FreeBSD) iD8DBQFFgxGoC3+MBN1Mb4gRAlhrAJ4j5LNZBdJ+cw9DFNxjr92AHH8TaACeIPLl migJ5N+a/N0XEU6D6gHcwVE= =7lc7 -----END PGP SIGNATURE----- --9Iq5ULCa7nGtWwZS--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20061215212040.GG23698>