From owner-freebsd-current@FreeBSD.ORG Wed Mar 12 13:50:07 2014 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 848A5108 for ; Wed, 12 Mar 2014 13:50:07 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5C1FAE3D for ; Wed, 12 Mar 2014 13:50:07 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s2CDo7ua055682 for ; Wed, 12 Mar 2014 13:50:07 GMT (envelope-from bdrewery@freefall.freebsd.org) Received: (from bdrewery@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s2CDo7L7055679 for freebsd-current@freebsd.org; Wed, 12 Mar 2014 13:50:07 GMT (envelope-from bdrewery) Received: (qmail 40446 invoked from network); 12 Mar 2014 08:50:00 -0500 Received: from unknown (HELO ?10.10.0.24?) (freebsd@shatow.net@10.10.0.24) by sweb.xzibition.com with ESMTPA; 12 Mar 2014 08:50:00 -0500 Message-ID: <53206600.4080602@FreeBSD.org> Date: Wed, 12 Mar 2014 08:49:52 -0500 From: Bryan Drewery Organization: FreeBSD User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Konstantin Belousov , John Baldwin Subject: Re: panic: lockmgr still held [tmpfs] [vm_map_remove()->vdropl()] (r262186: Thu Feb 20) References: <53109ACB.20102@FreeBSD.org> <5316F144.1000105@FreeBSD.org> <20140305110723.GB24664@kib.kiev.ua> <201403051421.04381.jhb@freebsd.org> <20140305195001.GF24664@kib.kiev.ua> In-Reply-To: <20140305195001.GF24664@kib.kiev.ua> X-Enigmail-Version: 1.6 OpenPGP: id=6E4697CF; url=http://www.shatow.net/bryan/bryan2.asc Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="eVT9qDHFH2MeofqkUHEaF0vhVg603Jk2O" Cc: Alan Cox , freebsd-current@freebsd.org, Andriy Gapon X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Mar 2014 13:50:07 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --eVT9qDHFH2MeofqkUHEaF0vhVg603Jk2O Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 3/5/2014 1:50 PM, Konstantin Belousov wrote: > On Wed, Mar 05, 2014 at 02:21:04PM -0500, John Baldwin wrote: >> On Wednesday, March 05, 2014 6:07:23 am Konstantin Belousov wrote: >>> On Wed, Mar 05, 2014 at 11:41:24AM +0200, Andriy Gapon wrote: >>>> on 04/03/2014 18:45 John Baldwin said the following: >>>>> So I'm not sure how to fix this. The crash is in this code in=20 >>>>> vm_object_deallocate(): >>>>> >>>>> if (object->type =3D=3D OBJT_SWAP && >>>>> (object->flags & OBJ_TMPFS) !=3D 0) { >>>>> vp =3D object->un_pager.swp.swp_tmpfs; >>>>> vhold(vp); >>>>> VM_OBJECT_WUNLOCK(object); >>>>> vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); >>>>> vdrop(vp); >>>>> VM_OBJECT_WLOCK(object); >>>>> if (object->type =3D=3D OBJT_DEAD || >>>>> object->ref_count !=3D 1) { >>>>> VM_OBJECT_WUNLOCK(object); >>>>> VOP_UNLOCK(vp, 0); >>>>> return; >>>>> } >>>>> if ((object->flags & OBJ_TMPFS) !=3D 0) >>>>> VOP_UNSET_TEXT(vp); >>>>> VOP_UNLOCK(vp, 0); >>>>> } >>>>> >>>>> The vdrop() is dropping the count to zero and trying to free the vn= ode. The=20 >>>>> real problem I think is that swp_tmpfs doesn't have an implicit vho= ld() on the=20 >>>>> vnode, so in this case, the code is doing a vhold/vn_lock/vdrop of = an already- >>>>> free vnode. For OBJT_VNODE objects, the reference from the object = back to the=20 >>>>> vnode holds a vref() that gets released by a vput() in=20 >>>>> vm_object_vndeallocate(). >>>>> >>>>> One fix might be to chagne smp_tmpfs to hold a vhold reference. Th= is is=20 >>>>> untested but might work (but I'm also not sure that this is the rig= ht thing in=20 >>>>> that I don't know what other effects it might have). >>>> >>>> I agree with your analysis, but I don't think that a filesystem hold= ing its own >>>> vnode is a good idea. If I am not mistaken, that would prevent tmpf= s vnodes >>>> from going to free list. >>>> I'd rather try to modify vm_object_deallocate() code. E.g. vdrop() = could be >>>> called after VOP_UNLOCK(). Alternatively, the code could handle a d= oomed vnode >>>> in a different way. >>> >>> I agree with Andrey, it is just a bug to vdrop() before unlock. >>> Please try this. >> >> Ok, my only worry is in the case of Bryan's panic, the hold count on t= he vnode >> was already zero before vhold() was called, so is it possible that it = is a stale >> pointer or is there some other implicit reference that prevents that? = If it can't >> be stale, I think deferring the vdrop() is fine. >=20 > The object->un_pager.swp.swp_tmpfs is cleared under the object lock > before the vnode is reclaimed, i.e. long before the vnode can be freed.= > swp_tmpfs should be kept in sync with the OBJ_TMPFS flag, so the > vhold() is safe while flag is set and object is locked. >=20 This has been stable. Not sure if it fixes the original problem, but I have not seen any panics since using it and doing many poudriere builds. --=20 Regards, Bryan Drewery --eVT9qDHFH2MeofqkUHEaF0vhVg603Jk2O Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJTIGYBAAoJEDXXcbtuRpfPiv8IAMEAV0LelmkvG9E7CZivwBey ZySy99hyqU/v2utMmfQHqmDtYycwPTlJ1OrFogbseo9tSPVjYYh7nHlCRp4RMexj pDOrYOmMmBJxQFGSJxvQrgXt3GZ/IUu/9FVbBaEBSf/BBG2ewMLIjsHUYNvHyxKK uOcdEeN9xD0GRBryxns5co/uMx5VqPaaiTvS3ljMKHs2N6Q0Zgjm8QL5XIotZ53p 3ZuhMxUhoo5EJhHU/7NIhE2FmO38dY0ufEv8GEpq2spD7+5fwCv/xWxMixlSMMsG RPTQmY/yER1Bn953gqlQ4H4a/f8v5kK/qDhgLCXPZ6X+p39PaPMc0iTcyd4kqwo= =dYN5 -----END PGP SIGNATURE----- --eVT9qDHFH2MeofqkUHEaF0vhVg603Jk2O--