From owner-svn-src-all@freebsd.org Fri Jul 13 22:40:15 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F2FC91028A71; Fri, 13 Jul 2018 22:40:14 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A60F47BD54; Fri, 13 Jul 2018 22:40:14 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 83C6816541; Fri, 13 Jul 2018 22:40:14 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6DMeEeA073098; Fri, 13 Jul 2018 22:40:14 GMT (envelope-from mjg@FreeBSD.org) Received: (from mjg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6DMeEA7073097; Fri, 13 Jul 2018 22:40:14 GMT (envelope-from mjg@FreeBSD.org) Message-Id: <201807132240.w6DMeEA7073097@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mjg set sender to mjg@FreeBSD.org using -f From: Mateusz Guzik Date: Fri, 13 Jul 2018 22:40:14 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336267 - head/sys/kern X-SVN-Group: head X-SVN-Commit-Author: mjg X-SVN-Commit-Paths: head/sys/kern X-SVN-Commit-Revision: 336267 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Jul 2018 22:40:15 -0000 Author: mjg Date: Fri Jul 13 22:40:14 2018 New Revision: 336267 URL: https://svnweb.freebsd.org/changeset/base/336267 Log: lockmgr: tidy up slock/sunlock similar to other locks Modified: head/sys/kern/kern_lock.c Modified: head/sys/kern/kern_lock.c ============================================================================== --- head/sys/kern/kern_lock.c Fri Jul 13 21:30:18 2018 (r336266) +++ head/sys/kern/kern_lock.c Fri Jul 13 22:40:14 2018 (r336267) @@ -112,11 +112,21 @@ CTASSERT(LK_UNLOCKED == (LK_UNLOCKED & } \ } while (0) -#define LK_CAN_SHARE(x, flags) \ - (((x) & LK_SHARE) && \ - (((x) & (LK_EXCLUSIVE_WAITERS | LK_EXCLUSIVE_SPINNERS)) == 0 || \ - (curthread->td_lk_slocks != 0 && !(flags & LK_NODDLKTREAT)) || \ - (curthread->td_pflags & TDP_DEADLKTREAT))) +static bool __always_inline +LK_CAN_SHARE(uintptr_t x, int flags, bool fp) +{ + + if ((x & (LK_SHARE | LK_EXCLUSIVE_WAITERS | LK_EXCLUSIVE_SPINNERS)) == + LK_SHARE) + return (true); + if (fp || (!(x & LK_SHARE))) + return (false); + if ((curthread->td_lk_slocks != 0 && !(flags & LK_NODDLKTREAT)) || + (curthread->td_pflags & TDP_DEADLKTREAT)) + return (true); + return (false); +} + #define LK_TRYOP(x) \ ((x) & LK_NOWAIT) @@ -169,7 +179,7 @@ struct lockmgr_wait { }; static bool __always_inline lockmgr_slock_try(struct lock *lk, uintptr_t *xp, - int flags); + int flags, bool fp); static bool __always_inline lockmgr_sunlock_try(struct lock *lk, uintptr_t *xp); static void @@ -498,7 +508,7 @@ lockdestroy(struct lock *lk) } static bool __always_inline -lockmgr_slock_try(struct lock *lk, uintptr_t *xp, int flags) +lockmgr_slock_try(struct lock *lk, uintptr_t *xp, int flags, bool fp) { /* @@ -509,7 +519,7 @@ lockmgr_slock_try(struct lock *lk, uintptr_t *xp, int * loop back and retry. */ *xp = lk->lk_lock; - while (LK_CAN_SHARE(*xp, flags)) { + while (LK_CAN_SHARE(*xp, flags, fp)) { if (atomic_fcmpset_acq_ptr(&lk->lk_lock, xp, *xp + LK_ONE_SHARER)) { return (true); @@ -523,29 +533,12 @@ lockmgr_sunlock_try(struct lock *lk, uintptr_t *xp) { for (;;) { - /* - * If there is more than one shared lock held, just drop one - * and return. - */ - if (LK_SHARERS(*xp) > 1) { + if (LK_SHARERS(*xp) > 1 || !(*xp & LK_ALL_WAITERS)) { if (atomic_fcmpset_rel_ptr(&lk->lk_lock, xp, *xp - LK_ONE_SHARER)) return (true); continue; } - - /* - * If there are not waiters on the exclusive queue, drop the - * lock quickly. - */ - if ((*xp & LK_ALL_WAITERS) == 0) { - MPASS((*xp & ~LK_EXCLUSIVE_SPINNERS) == - LK_SHARERS_LOCK(1)); - if (atomic_fcmpset_rel_ptr(&lk->lk_lock, xp, - LK_UNLOCKED)) - return (true); - continue; - } break; } return (false); @@ -574,7 +567,7 @@ lockmgr_slock_hard(struct lock *lk, u_int flags, struc WITNESS_CHECKORDER(&lk->lock_object, LOP_NEWORDER, file, line, flags & LK_INTERLOCK ? ilk : NULL); for (;;) { - if (lockmgr_slock_try(lk, &x, flags)) + if (lockmgr_slock_try(lk, &x, flags, false)) break; #ifdef HWPMC_HOOKS PMC_SOFT_CALL( , , lock, failed); @@ -617,7 +610,7 @@ retry_sleepq: * if the lock can be acquired in shared mode, try * again. */ - if (LK_CAN_SHARE(x, flags)) { + if (LK_CAN_SHARE(x, flags, false)) { sleepq_release(&lk->lock_object); continue; } @@ -932,7 +925,7 @@ lockmgr_lock_fast_path(struct lock *lk, u_int flags, s file, line, flags & LK_INTERLOCK ? ilk : NULL); if (__predict_false(lk->lock_object.lo_flags & LK_NOSHARE)) break; - if (lockmgr_slock_try(lk, &x, flags)) { + if (lockmgr_slock_try(lk, &x, flags, true)) { lockmgr_note_shared_acquire(lk, 0, 0, file, line, flags); locked = true;