From owner-svn-src-user@FreeBSD.ORG Tue Apr 20 22:59:56 2010 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BC9051065673; Tue, 20 Apr 2010 22:59:56 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id A15FD8FC12; Tue, 20 Apr 2010 22:59:56 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o3KMxu1e070904; Tue, 20 Apr 2010 22:59:56 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o3KMxuJN070901; Tue, 20 Apr 2010 22:59:56 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <201004202259.o3KMxuJN070901@svn.freebsd.org> From: Kip Macy Date: Tue, 20 Apr 2010 22:59:56 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r206953 - in user/kmacy/head_page_lock_2/sys/amd64: amd64 include X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Apr 2010 22:59:56 -0000 Author: kmacy Date: Tue Apr 20 22:59:56 2010 New Revision: 206953 URL: http://svn.freebsd.org/changeset/base/206953 Log: - handle multiple simultaneous relocks on a pmap using a counter - only return EAGAIN if the pmap lock was acquired by another caller in the meantime or another caller is in the middle of a retry Modified: user/kmacy/head_page_lock_2/sys/amd64/amd64/pmap.c user/kmacy/head_page_lock_2/sys/amd64/include/pmap.h Modified: user/kmacy/head_page_lock_2/sys/amd64/amd64/pmap.c ============================================================================== --- user/kmacy/head_page_lock_2/sys/amd64/amd64/pmap.c Tue Apr 20 22:57:05 2010 (r206952) +++ user/kmacy/head_page_lock_2/sys/amd64/amd64/pmap.c Tue Apr 20 22:59:56 2010 (r206953) @@ -213,6 +213,10 @@ static int pmap_tryrelock_restart; SYSCTL_INT(_vm_pmap, OID_AUTO, tryrelock_restart, CTLFLAG_RD, &pmap_tryrelock_restart, 0, "Number of tryrelock restarts"); +static int pmap_tryrelock_race; +SYSCTL_INT(_vm_pmap, OID_AUTO, tryrelock_race, CTLFLAG_RD, + &pmap_tryrelock_race, 0, "Number of tryrelock pmap race cases"); + static u_int64_t KPTphys; /* phys addr of kernel level 1 */ static u_int64_t KPDphys; /* phys addr of kernel level 2 */ @@ -549,8 +553,10 @@ static int pa_tryrelock(pmap_t pmap, vm_paddr_t pa, vm_paddr_t *locked) { vm_paddr_t lockpa; + uint16_t gen_count; PMAP_LOCK_ASSERT(pmap, MA_OWNED); + gen_count = pmap->pm_gen_count; atomic_add_long((volatile long *)&pmap_tryrelock_calls, 1); lockpa = *locked; *locked = pa; @@ -562,14 +568,20 @@ pa_tryrelock(pmap_t pmap, vm_paddr_t pa, } if (PA_TRYLOCK(pa)) return (0); - pmap->pm_flags |= PMAP_IN_RETRY; + pmap->pm_retry_depth++; PMAP_UNLOCK(pmap); atomic_add_int((volatile int *)&pmap_tryrelock_restart, 1); PA_LOCK(pa); mtx_lock(&(pmap)->pm_mtx); - pmap->pm_flags &= ~PMAP_IN_RETRY; - - return (EAGAIN); + pmap->pm_retry_depth--; + if (pmap->pm_retry_depth) + pmap->pm_gen_count++; + + if (gen_count != pmap->pm_gen_count) { + atomic_add_int((volatile int *)&pmap_tryrelock_race, 1); + return (EAGAIN); + } + return (0); } static u_int64_t Modified: user/kmacy/head_page_lock_2/sys/amd64/include/pmap.h ============================================================================== --- user/kmacy/head_page_lock_2/sys/amd64/include/pmap.h Tue Apr 20 22:57:05 2010 (r206952) +++ user/kmacy/head_page_lock_2/sys/amd64/include/pmap.h Tue Apr 20 22:59:56 2010 (r206953) @@ -237,8 +237,6 @@ struct md_page { int pat_mode; }; -#define PMAP_IN_RETRY 0x1 - /* * The kernel virtual address (KVA) of the level 4 page table page is always * within the direct map (DMAP) region. @@ -249,7 +247,7 @@ struct pmap { TAILQ_HEAD(,pv_chunk) pm_pvchunk; /* list of mappings in pmap */ u_int pm_active; /* active on cpus */ uint16_t pm_gen_count; /* generation count (pmap lock dropped) */ - uint16_t pm_flags; + uint16_t pm_retry_depth; /* number of cases in retry */ struct pmap_statistics pm_stats; /* pmap statistics */ vm_page_t pm_root; /* spare page table pages */ vm_page_t pm_free; /* Temporary free pages. */ @@ -263,7 +261,7 @@ extern struct pmap kernel_pmap_store; #define PMAP_UPDATE_GEN_COUNT(pmap) \ do { \ - if (pmap->pm_flags & PMAP_IN_RETRY) \ + if (pmap->pm_retry_depth) \ pmap->pm_gen_count++; \ } while (0)