From owner-svn-src-all@FreeBSD.ORG Sat May 23 23:05:33 2015 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E6B7FE4F; Sat, 23 May 2015 23:05:33 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D484B1968; Sat, 23 May 2015 23:05:33 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.9/8.14.9) with ESMTP id t4NN5X5U052066; Sat, 23 May 2015 23:05:33 GMT (envelope-from ian@FreeBSD.org) Received: (from ian@localhost) by svn.freebsd.org (8.14.9/8.14.9/Submit) id t4NN5WCe052056; Sat, 23 May 2015 23:05:32 GMT (envelope-from ian@FreeBSD.org) Message-Id: <201505232305.t4NN5WCe052056@svn.freebsd.org> X-Authentication-Warning: svn.freebsd.org: ian set sender to ian@FreeBSD.org using -f From: Ian Lepore Date: Sat, 23 May 2015 23:05:32 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r283336 - in stable/10/sys/arm: arm include X-SVN-Group: stable-10 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 May 2015 23:05:34 -0000 Author: ian Date: Sat May 23 23:05:31 2015 New Revision: 283336 URL: https://svnweb.freebsd.org/changeset/base/283336 Log: MFC r279810, r279811: Clean data cache before instruction cache in armv7_icache_sync_range(). Add minimum cache line sizes to struct cpuinfo, use them in the new cache maintenance routines. Also add a routine to invalidate the branch cache. Modified: stable/10/sys/arm/arm/cpufunc_asm_armv7.S stable/10/sys/arm/arm/cpuinfo.c stable/10/sys/arm/arm/genassym.c stable/10/sys/arm/include/cpu-v6.h stable/10/sys/arm/include/cpuinfo.h Directory Properties: stable/10/ (props changed) Modified: stable/10/sys/arm/arm/cpufunc_asm_armv7.S ============================================================================== --- stable/10/sys/arm/arm/cpufunc_asm_armv7.S Sat May 23 22:48:54 2015 (r283335) +++ stable/10/sys/arm/arm/cpufunc_asm_armv7.S Sat May 23 23:05:31 2015 (r283336) @@ -247,8 +247,8 @@ ENTRY(armv7_idcache_wbinv_range) add r0, r0, ip subs r1, r1, ip bhi .Larmv7_id_wbinv_next - isb /* instruction synchronization barrier */ dsb /* data synchronization barrier */ + isb /* instruction synchronization barrier */ RET END(armv7_idcache_wbinv_range) @@ -258,8 +258,8 @@ ENTRY_NP(armv7_icache_sync_all) #else mcr CP15_ICIALLU #endif - isb /* instruction synchronization barrier */ dsb /* data synchronization barrier */ + isb /* instruction synchronization barrier */ RET END(armv7_icache_sync_all) @@ -267,13 +267,13 @@ ENTRY_NP(armv7_icache_sync_range) ldr ip, .Larmv7_icache_line_size ldr ip, [ip] .Larmv7_sync_next: - mcr CP15_ICIMVAU(r0) mcr CP15_DCCMVAC(r0) + mcr CP15_ICIMVAU(r0) add r0, r0, ip subs r1, r1, ip bhi .Larmv7_sync_next - isb /* instruction synchronization barrier */ dsb /* data synchronization barrier */ + isb /* instruction synchronization barrier */ RET END(armv7_icache_sync_range) Modified: stable/10/sys/arm/arm/cpuinfo.c ============================================================================== --- stable/10/sys/arm/arm/cpuinfo.c Sat May 23 22:48:54 2015 (r283335) +++ stable/10/sys/arm/arm/cpuinfo.c Sat May 23 23:05:31 2015 (r283336) @@ -34,7 +34,14 @@ __FBSDID("$FreeBSD$"); #include #include -struct cpuinfo cpuinfo; +struct cpuinfo cpuinfo = +{ + /* Use safe defaults for start */ + .dcache_line_size = 32, + .dcache_line_mask = 31, + .icache_line_size = 32, + .icache_line_mask = 31, +}; /* Read and parse CPU id scheme */ void @@ -122,4 +129,10 @@ cpuinfo_init(void) cpuinfo.generic_timer_ext = (cpuinfo.id_pfr1 >> 16) & 0xF; cpuinfo.virtualization_ext = (cpuinfo.id_pfr1 >> 12) & 0xF; cpuinfo.security_ext = (cpuinfo.id_pfr1 >> 4) & 0xF; + + /* L1 Cache sizes */ + cpuinfo.dcache_line_size = 1 << (CPU_CT_DMINLINE(cpuinfo.ctr ) + 2); + cpuinfo.dcache_line_mask = cpuinfo.dcache_line_size - 1; + cpuinfo.icache_line_size= 1 << (CPU_CT_IMINLINE(cpuinfo.ctr ) + 2); + cpuinfo.icache_line_mask = cpuinfo.icache_line_size - 1; } Modified: stable/10/sys/arm/arm/genassym.c ============================================================================== --- stable/10/sys/arm/arm/genassym.c Sat May 23 22:48:54 2015 (r283335) +++ stable/10/sys/arm/arm/genassym.c Sat May 23 23:05:31 2015 (r283336) @@ -45,6 +45,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -146,3 +147,8 @@ ASSYM(MAXCOMLEN, MAXCOMLEN); ASSYM(MAXCPU, MAXCPU); ASSYM(NIRQ, NIRQ); ASSYM(PCPU_SIZE, sizeof(struct pcpu)); + +ASSYM(DCACHE_LINE_SIZE, offsetof(struct cpuinfo, dcache_line_size)); +ASSYM(DCACHE_LINE_MASK, offsetof(struct cpuinfo, dcache_line_mask)); +ASSYM(ICACHE_LINE_SIZE, offsetof(struct cpuinfo, icache_line_size)); +ASSYM(ICACHE_LINE_MASK, offsetof(struct cpuinfo, icache_line_mask)); Modified: stable/10/sys/arm/include/cpu-v6.h ============================================================================== --- stable/10/sys/arm/include/cpu-v6.h Sat May 23 22:48:54 2015 (r283335) +++ stable/10/sys/arm/include/cpu-v6.h Sat May 23 23:05:31 2015 (r283336) @@ -37,6 +37,9 @@ #define CPU_ASID_KERNEL 0 +vm_offset_t dcache_wb_pou_checked(vm_offset_t, vm_size_t); +vm_offset_t icache_inv_pou_checked(vm_offset_t, vm_size_t); + /* * Macros to generate CP15 (system control processor) read/write functions. */ @@ -295,7 +298,7 @@ icache_sync(vm_offset_t sva, vm_size_t s vm_offset_t eva = sva + size; dsb(); - for (va = sva; va < eva; va += arm_dcache_align) { + for (va = sva; va < eva; va += cpuinfo.dcache_line_size) { #if __ARM_ARCH >= 7 && defined SMP _CP15_DCCMVAU(va); #else @@ -325,6 +328,19 @@ icache_inv_all(void) isb(); } +/* Invalidate branch predictor buffer */ +static __inline void +bpb_inv_all(void) +{ +#if __ARM_ARCH >= 7 && defined SMP + _CP15_BPIALLIS(); +#else + _CP15_BPIALL(); +#endif + dsb(); + isb(); +} + /* Write back D-cache to PoU */ static __inline void dcache_wb_pou(vm_offset_t sva, vm_size_t size) @@ -333,7 +349,7 @@ dcache_wb_pou(vm_offset_t sva, vm_size_t vm_offset_t eva = sva + size; dsb(); - for (va = sva; va < eva; va += arm_dcache_align) { + for (va = sva; va < eva; va += cpuinfo.dcache_line_size) { #if __ARM_ARCH >= 7 && defined SMP _CP15_DCCMVAU(va); #else @@ -351,7 +367,7 @@ dcache_inv_poc(vm_offset_t sva, vm_paddr vm_offset_t eva = sva + size; /* invalidate L1 first */ - for (va = sva; va < eva; va += arm_dcache_align) { + for (va = sva; va < eva; va += cpuinfo.dcache_line_size) { _CP15_DCIMVAC(va); } dsb(); @@ -361,7 +377,7 @@ dcache_inv_poc(vm_offset_t sva, vm_paddr dsb(); /* then L1 again */ - for (va = sva; va < eva; va += arm_dcache_align) { + for (va = sva; va < eva; va += cpuinfo.dcache_line_size) { _CP15_DCIMVAC(va); } dsb(); @@ -376,7 +392,7 @@ dcache_wb_poc(vm_offset_t sva, vm_paddr_ dsb(); - for (va = sva; va < eva; va += arm_dcache_align) { + for (va = sva; va < eva; va += cpuinfo.dcache_line_size) { _CP15_DCCMVAC(va); } dsb(); @@ -394,7 +410,7 @@ dcache_wbinv_poc(vm_offset_t sva, vm_pad dsb(); /* write back L1 first */ - for (va = sva; va < eva; va += arm_dcache_align) { + for (va = sva; va < eva; va += cpuinfo.dcache_line_size) { _CP15_DCCMVAC(va); } dsb(); @@ -403,7 +419,7 @@ dcache_wbinv_poc(vm_offset_t sva, vm_pad cpu_l2cache_wbinv_range(pa, size); /* then invalidate L1 */ - for (va = sva; va < eva; va += arm_dcache_align) { + for (va = sva; va < eva; va += cpuinfo.dcache_line_size) { _CP15_DCIMVAC(va); } dsb(); Modified: stable/10/sys/arm/include/cpuinfo.h ============================================================================== --- stable/10/sys/arm/include/cpuinfo.h Sat May 23 22:48:54 2015 (r283335) +++ stable/10/sys/arm/include/cpuinfo.h Sat May 23 23:05:31 2015 (r283336) @@ -82,6 +82,12 @@ struct cpuinfo { int generic_timer_ext; int virtualization_ext; int security_ext; + + /* L1 cache info */ + int dcache_line_size; + int dcache_line_mask; + int icache_line_size; + int icache_line_mask; }; extern struct cpuinfo cpuinfo;