From owner-svn-src-stable-11@freebsd.org Fri Feb 3 00:58:09 2017 Return-Path: Delivered-To: svn-src-stable-11@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 77A54CCCF59; Fri, 3 Feb 2017 00:58:09 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 524931479; Fri, 3 Feb 2017 00:58:09 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v130w8hZ081708; Fri, 3 Feb 2017 00:58:08 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v130w8hG081706; Fri, 3 Feb 2017 00:58:08 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201702030058.v130w8hG081706@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Fri, 3 Feb 2017 00:58:08 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r313121 - in stable/11/sys: amd64/amd64 i386/i386 X-SVN-Group: stable-11 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-stable-11@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for only the 11-stable src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Feb 2017 00:58:09 -0000 Author: markj Date: Fri Feb 3 00:58:08 2017 New Revision: 313121 URL: https://svnweb.freebsd.org/changeset/base/313121 Log: MFC r311902: Coalesce TLB shootdowns of global PTEs in pmap_advise() on x86. Modified: stable/11/sys/amd64/amd64/pmap.c stable/11/sys/i386/i386/pmap.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/amd64/amd64/pmap.c ============================================================================== --- stable/11/sys/amd64/amd64/pmap.c Fri Feb 3 00:55:31 2017 (r313120) +++ stable/11/sys/amd64/amd64/pmap.c Fri Feb 3 00:58:08 2017 (r313121) @@ -6054,7 +6054,7 @@ pmap_advise(pmap_t pmap, vm_offset_t sva pdp_entry_t *pdpe; pd_entry_t oldpde, *pde; pt_entry_t *pte, PG_A, PG_G, PG_M, PG_RW, PG_V; - vm_offset_t va_next; + vm_offset_t va, va_next; vm_page_t m; boolean_t anychanged; @@ -6134,11 +6134,11 @@ pmap_advise(pmap_t pmap, vm_offset_t sva } if (va_next > eva) va_next = eva; + va = va_next; for (pte = pmap_pde_to_pte(pde, sva); sva != va_next; pte++, sva += PAGE_SIZE) { - if ((*pte & (PG_MANAGED | PG_V)) != (PG_MANAGED | - PG_V)) - continue; + if ((*pte & (PG_MANAGED | PG_V)) != (PG_MANAGED | PG_V)) + goto maybe_invlrng; else if ((*pte & (PG_M | PG_RW)) == (PG_M | PG_RW)) { if (advice == MADV_DONTNEED) { /* @@ -6153,12 +6153,22 @@ pmap_advise(pmap_t pmap, vm_offset_t sva } else if ((*pte & PG_A) != 0) atomic_clear_long(pte, PG_A); else - continue; - if ((*pte & PG_G) != 0) - pmap_invalidate_page(pmap, sva); - else + goto maybe_invlrng; + + if ((*pte & PG_G) != 0) { + if (va == va_next) + va = sva; + } else anychanged = TRUE; + continue; +maybe_invlrng: + if (va != va_next) { + pmap_invalidate_range(pmap, va, sva); + va = va_next; + } } + if (va != va_next) + pmap_invalidate_range(pmap, va, sva); } if (anychanged) pmap_invalidate_all(pmap); Modified: stable/11/sys/i386/i386/pmap.c ============================================================================== --- stable/11/sys/i386/i386/pmap.c Fri Feb 3 00:55:31 2017 (r313120) +++ stable/11/sys/i386/i386/pmap.c Fri Feb 3 00:58:08 2017 (r313121) @@ -4959,7 +4959,7 @@ pmap_advise(pmap_t pmap, vm_offset_t sva { pd_entry_t oldpde, *pde; pt_entry_t *pte; - vm_offset_t pdnxt; + vm_offset_t va, pdnxt; vm_page_t m; boolean_t anychanged, pv_lists_locked; @@ -5020,11 +5020,11 @@ resume: } if (pdnxt > eva) pdnxt = eva; + va = pdnxt; for (pte = pmap_pte_quick(pmap, sva); sva != pdnxt; pte++, sva += PAGE_SIZE) { - if ((*pte & (PG_MANAGED | PG_V)) != (PG_MANAGED | - PG_V)) - continue; + if ((*pte & (PG_MANAGED | PG_V)) != (PG_MANAGED | PG_V)) + goto maybe_invlrng; else if ((*pte & (PG_M | PG_RW)) == (PG_M | PG_RW)) { if (advice == MADV_DONTNEED) { /* @@ -5039,12 +5039,21 @@ resume: } else if ((*pte & PG_A) != 0) atomic_clear_int((u_int *)pte, PG_A); else - continue; - if ((*pte & PG_G) != 0) - pmap_invalidate_page(pmap, sva); - else + goto maybe_invlrng; + if ((*pte & PG_G) != 0) { + if (va == pdnxt) + va = sva; + } else anychanged = TRUE; + continue; +maybe_invlrng: + if (va != pdnxt) { + pmap_invalidate_range(pmap, va, sva); + va = pdnxt; + } } + if (va != pdnxt) + pmap_invalidate_range(pmap, va, sva); } if (anychanged) pmap_invalidate_all(pmap);