From owner-svn-src-all@freebsd.org Mon Apr 4 07:16:45 2016 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 66EB0B022F8; Mon, 4 Apr 2016 07:16:45 +0000 (UTC) (envelope-from wma@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3E9121CE5; Mon, 4 Apr 2016 07:16:45 +0000 (UTC) (envelope-from wma@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u347Gib3060838; Mon, 4 Apr 2016 07:16:44 GMT (envelope-from wma@FreeBSD.org) Received: (from wma@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u347Gi5o060834; Mon, 4 Apr 2016 07:16:44 GMT (envelope-from wma@FreeBSD.org) Message-Id: <201604040716.u347Gi5o060834@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: wma set sender to wma@FreeBSD.org using -f From: Wojciech Macek Date: Mon, 4 Apr 2016 07:16:44 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r297538 - in head/sys/arm64: arm64 include X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Apr 2016 07:16:45 -0000 Author: wma Date: Mon Apr 4 07:16:43 2016 New Revision: 297538 URL: https://svnweb.freebsd.org/changeset/base/297538 Log: arm64: pagezero improvement This change has been provided to improve pagezero call performance. Submitted by: Dominik Ermel Obtained from: Semihalf Sponsored by: Cavium Reviewed by: kib Differential Revision: https://reviews.freebsd.org/D5741 Modified: head/sys/arm64/arm64/machdep.c head/sys/arm64/arm64/pmap.c head/sys/arm64/arm64/support.S head/sys/arm64/include/machdep.h Modified: head/sys/arm64/arm64/machdep.c ============================================================================== --- head/sys/arm64/arm64/machdep.c Mon Apr 4 07:11:33 2016 (r297537) +++ head/sys/arm64/arm64/machdep.c Mon Apr 4 07:16:43 2016 (r297538) @@ -110,6 +110,13 @@ int64_t icache_line_size; /* The minimum int64_t idcache_line_size; /* The minimum cache line size */ int64_t dczva_line_size; /* The size of cache line the dc zva zeroes */ +/* pagezero_* implementations are provided in support.S */ +void pagezero_simple(void *); +void pagezero_cache(void *); + +/* pagezero_simple is default pagezero */ +void (*pagezero)(void *p) = pagezero_simple; + static void cpu_startup(void *dummy) { @@ -817,6 +824,9 @@ cache_setup(void) /* Same as with above calculations */ dczva_line_shift = DCZID_BS_SIZE(dczid_el0); dczva_line_size = sizeof(int) << dczva_line_shift; + + /* Change pagezero function */ + pagezero = pagezero_cache; } } Modified: head/sys/arm64/arm64/pmap.c ============================================================================== --- head/sys/arm64/arm64/pmap.c Mon Apr 4 07:11:33 2016 (r297537) +++ head/sys/arm64/arm64/pmap.c Mon Apr 4 07:16:43 2016 (r297538) @@ -271,13 +271,6 @@ pagecopy(void *s, void *d) memcpy(d, s, PAGE_SIZE); } -static __inline void -pagezero(void *p) -{ - - bzero(p, PAGE_SIZE); -} - #define pmap_l0_index(va) (((va) >> L0_SHIFT) & L0_ADDR_MASK) #define pmap_l1_index(va) (((va) >> L1_SHIFT) & Ln_ADDR_MASK) #define pmap_l2_index(va) (((va) >> L2_SHIFT) & Ln_ADDR_MASK) Modified: head/sys/arm64/arm64/support.S ============================================================================== --- head/sys/arm64/arm64/support.S Mon Apr 4 07:11:33 2016 (r297537) +++ head/sys/arm64/arm64/support.S Mon Apr 4 07:16:43 2016 (r297538) @@ -33,6 +33,7 @@ __FBSDID("$FreeBSD$"); #include +#include #include "assym.s" @@ -290,3 +291,38 @@ ENTRY(longjmp) mov x0, x1 ret END(longjmp) + +/* + * pagezero, simple implementation + */ +ENTRY(pagezero_simple) + add x1, x0, #PAGE_SIZE + +1: + stp xzr, xzr, [x0], #0x10 + stp xzr, xzr, [x0], #0x10 + stp xzr, xzr, [x0], #0x10 + stp xzr, xzr, [x0], #0x10 + cmp x0, x1 + b.ne 1b + ret + +END(pagezero_simple) + +/* + * pagezero, cache assisted + */ +ENTRY(pagezero_cache) + add x1, x0, #PAGE_SIZE + + ldr x2, =dczva_line_size + ldr x2, [x2] + +1: + dc zva, x0 + add x0, x0, x2 + cmp x0, x1 + b.ne 1b + ret + +END(pagezero_cache) Modified: head/sys/arm64/include/machdep.h ============================================================================== --- head/sys/arm64/include/machdep.h Mon Apr 4 07:11:33 2016 (r297537) +++ head/sys/arm64/include/machdep.h Mon Apr 4 07:16:43 2016 (r297538) @@ -41,5 +41,6 @@ extern vm_paddr_t physmap[]; extern u_int physmap_idx; void initarm(struct arm64_bootparams *); +extern void (*pagezero)(void *); #endif /* _MACHINE_MACHDEP_H_ */