Date: Sun, 6 Dec 2015 17:39:13 +0000 (UTC) From: "Conrad E. Meyer" <cem@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r291906 - in head/sys: amd64/amd64 i386/i386 Message-ID: <201512061739.tB6HdDWh069901@repo.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: cem Date: Sun Dec 6 17:39:13 2015 New Revision: 291906 URL: https://svnweb.freebsd.org/changeset/base/291906 Log: pmap_invalidate_range: For very large ranges, flush the whole TLB Typical TLBs have 40-512 entries available. At some point, iterating every single page in a requested invalidation range and issuing invlpg on it is more expensive than flushing the TLB and allowing it to reload on demand. Broadwell CPUs have 1536 L2 TLB entries, so I've picked the arbitrary number 4096 entries as a hueristic at which point we flush TLB rather than invalidating every single potential page. Reviewed by: alc Feedback from: jhb, kib MFC notes: Depends on r291688 Sponsored by: EMC / Isilon Storage Division Differential Revision: https://reviews.freebsd.org/D4280 Modified: head/sys/amd64/amd64/pmap.c head/sys/i386/i386/pmap.c Modified: head/sys/amd64/amd64/pmap.c ============================================================================== --- head/sys/amd64/amd64/pmap.c Sun Dec 6 17:11:23 2015 (r291905) +++ head/sys/amd64/amd64/pmap.c Sun Dec 6 17:39:13 2015 (r291906) @@ -1421,6 +1421,9 @@ pmap_invalidate_page(pmap_t pmap, vm_off sched_unpin(); } +/* 4k PTEs -- Chosen to exceed the total size of Broadwell L2 TLB */ +#define PMAP_INVLPG_THRESHOLD (4 * 1024 * PAGE_SIZE) + void pmap_invalidate_range(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) { @@ -1428,6 +1431,11 @@ pmap_invalidate_range(pmap_t pmap, vm_of vm_offset_t addr; u_int cpuid, i; + if (eva - sva >= PMAP_INVLPG_THRESHOLD) { + pmap_invalidate_all(pmap); + return; + } + if (pmap_type_guest(pmap)) { pmap_invalidate_ept(pmap); return; Modified: head/sys/i386/i386/pmap.c ============================================================================== --- head/sys/i386/i386/pmap.c Sun Dec 6 17:11:23 2015 (r291905) +++ head/sys/i386/i386/pmap.c Sun Dec 6 17:39:13 2015 (r291906) @@ -1032,6 +1032,9 @@ pmap_invalidate_page(pmap_t pmap, vm_off sched_unpin(); } +/* 4k PTEs -- Chosen to exceed the total size of Broadwell L2 TLB */ +#define PMAP_INVLPG_THRESHOLD (4 * 1024 * PAGE_SIZE) + void pmap_invalidate_range(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) { @@ -1039,6 +1042,11 @@ pmap_invalidate_range(pmap_t pmap, vm_of vm_offset_t addr; u_int cpuid; + if (eva - sva >= PMAP_INVLPG_THRESHOLD) { + pmap_invalidate_all(pmap); + return; + } + sched_pin(); if (pmap == kernel_pmap || !CPU_CMP(&pmap->pm_active, &all_cpus)) { for (addr = sva; addr < eva; addr += PAGE_SIZE)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201512061739.tB6HdDWh069901>