From owner-svn-src-stable-10@FreeBSD.ORG Thu Jul 24 10:45:53 2014 Return-Path: Delivered-To: svn-src-stable-10@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9FCD325D; Thu, 24 Jul 2014 10:45:53 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 80A44215A; Thu, 24 Jul 2014 10:45:53 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s6OAjrk5062163; Thu, 24 Jul 2014 10:45:53 GMT (envelope-from kib@svn.freebsd.org) Received: (from kib@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s6OAjrDs062162; Thu, 24 Jul 2014 10:45:53 GMT (envelope-from kib@svn.freebsd.org) Message-Id: <201407241045.s6OAjrDs062162@svn.freebsd.org> From: Konstantin Belousov Date: Thu, 24 Jul 2014 10:45:53 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r269056 - stable/10/sys/amd64/amd64 X-SVN-Group: stable-10 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-stable-10@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: SVN commit messages for only the 10-stable src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jul 2014 10:45:53 -0000 Author: kib Date: Thu Jul 24 10:45:52 2014 New Revision: 269056 URL: http://svnweb.freebsd.org/changeset/base/269056 Log: MFC r268660: Make amd64 pmap_copy_pages() functional for pages not mapped by DMAP. Modified: stable/10/sys/amd64/amd64/pmap.c Directory Properties: stable/10/ (props changed) Modified: stable/10/sys/amd64/amd64/pmap.c ============================================================================== --- stable/10/sys/amd64/amd64/pmap.c Thu Jul 24 10:43:15 2014 (r269055) +++ stable/10/sys/amd64/amd64/pmap.c Thu Jul 24 10:45:52 2014 (r269056) @@ -390,6 +390,11 @@ SYSCTL_PROC(_vm_pmap, OID_AUTO, pcid_sav CTLFLAG_MPSAFE, NULL, 0, pmap_pcid_save_cnt_proc, "QU", "Count of saved TLB context on switch"); +/* pmap_copy_pages() over non-DMAP */ +static struct mtx cpage_lock; +static vm_offset_t cpage_a; +static vm_offset_t cpage_b; + /* * Crashdump maps. */ @@ -1055,6 +1060,10 @@ pmap_init(void) M_WAITOK | M_ZERO); for (i = 0; i < pv_npg; i++) TAILQ_INIT(&pv_table[i].pv_list); + + mtx_init(&cpage_lock, "cpage", NULL, MTX_DEF); + cpage_a = kva_alloc(PAGE_SIZE); + cpage_b = kva_alloc(PAGE_SIZE); } static SYSCTL_NODE(_vm_pmap, OID_AUTO, pde, CTLFLAG_RD, 0, @@ -4970,19 +4979,58 @@ pmap_copy_pages(vm_page_t ma[], vm_offse vm_offset_t b_offset, int xfersize) { void *a_cp, *b_cp; + vm_page_t m_a, m_b; + vm_paddr_t p_a, p_b; + pt_entry_t *pte; vm_offset_t a_pg_offset, b_pg_offset; int cnt; + boolean_t pinned; + pinned = FALSE; while (xfersize > 0) { a_pg_offset = a_offset & PAGE_MASK; - cnt = min(xfersize, PAGE_SIZE - a_pg_offset); - a_cp = (char *)PHYS_TO_DMAP(ma[a_offset >> PAGE_SHIFT]-> - phys_addr) + a_pg_offset; + m_a = ma[a_offset >> PAGE_SHIFT]; + p_a = m_a->phys_addr; b_pg_offset = b_offset & PAGE_MASK; + m_b = mb[b_offset >> PAGE_SHIFT]; + p_b = m_b->phys_addr; + cnt = min(xfersize, PAGE_SIZE - a_pg_offset); cnt = min(cnt, PAGE_SIZE - b_pg_offset); - b_cp = (char *)PHYS_TO_DMAP(mb[b_offset >> PAGE_SHIFT]-> - phys_addr) + b_pg_offset; + if (__predict_false(p_a < DMAP_MIN_ADDRESS || + p_a > DMAP_MIN_ADDRESS + dmaplimit)) { + mtx_lock(&cpage_lock); + sched_pin(); + pinned = TRUE; + pte = vtopte(cpage_a); + *pte = p_a | X86_PG_A | X86_PG_V | + pmap_cache_bits(kernel_pmap, m_a->md.pat_mode, 0); + invlpg(cpage_a); + a_cp = (char *)cpage_a + a_pg_offset; + } else { + a_cp = (char *)PHYS_TO_DMAP(p_a) + a_pg_offset; + } + if (__predict_false(p_b < DMAP_MIN_ADDRESS || + p_b > DMAP_MIN_ADDRESS + dmaplimit)) { + if (!pinned) { + mtx_lock(&cpage_lock); + sched_pin(); + pinned = TRUE; + } + pte = vtopte(cpage_b); + *pte = p_b | X86_PG_A | X86_PG_M | X86_PG_RW | + X86_PG_V | pmap_cache_bits(kernel_pmap, + m_b->md.pat_mode, 0); + invlpg(cpage_b); + b_cp = (char *)cpage_b + b_pg_offset; + } else { + b_cp = (char *)PHYS_TO_DMAP(p_b) + b_pg_offset; + } bcopy(a_cp, b_cp, cnt); + if (__predict_false(pinned)) { + sched_unpin(); + mtx_unlock(&cpage_lock); + pinned = FALSE; + } a_offset += cnt; b_offset += cnt; xfersize -= cnt;