From owner-svn-src-all@FreeBSD.ORG Sun May 11 04:24:58 2014 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 47A3BD2C; Sun, 11 May 2014 04:24:58 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2A64C2FE2; Sun, 11 May 2014 04:24:58 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s4B4Ow8v039004; Sun, 11 May 2014 04:24:58 GMT (envelope-from ian@svn.freebsd.org) Received: (from ian@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s4B4Ov8w039000; Sun, 11 May 2014 04:24:57 GMT (envelope-from ian@svn.freebsd.org) Message-Id: <201405110424.s4B4Ov8w039000@svn.freebsd.org> From: Ian Lepore Date: Sun, 11 May 2014 04:24:57 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r265870 - in head/sys/arm: arm include X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 May 2014 04:24:58 -0000 Author: ian Date: Sun May 11 04:24:57 2014 New Revision: 265870 URL: http://svnweb.freebsd.org/changeset/base/265870 Log: Add cpu_l2cache_drain_writebuf(), use it to implement generic_bs_barrier(). On modern ARM SoCs the L2 cache controller sits between the CPU and the AXI bus, and most on-chip memory-mapped devices are on the AXI bus. We map the device registers using the 'Device' memory attribute, which means the memory is not cached, but writes to it are buffered. Ensuring that a write has made it all the way to a device may require that the L2 controller take some action. There is currently only one implementation of the new function, for the PL310 cache controller. It invokes a function that the controller manual calls "cache sync" but it actually has nothing to do with cache at all, it triggers a drain of all pending store buffer writes and it blocks until they complete. The sheeva and xscale L2 controllers (which predate the concept of Device memory) don't seem to have a corresponding function. It appears that the standard armv5 drain_writebuf function includes draining all the way through the L2 controller. Modified: head/sys/arm/arm/bus_space_generic.c head/sys/arm/arm/cpufunc.c head/sys/arm/arm/pl310.c head/sys/arm/include/cpufunc.h Modified: head/sys/arm/arm/bus_space_generic.c ============================================================================== --- head/sys/arm/arm/bus_space_generic.c Sun May 11 04:18:51 2014 (r265869) +++ head/sys/arm/arm/bus_space_generic.c Sun May 11 04:24:57 2014 (r265870) @@ -50,6 +50,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include /* Prototypes for all the bus_space structure functions */ @@ -110,5 +111,16 @@ generic_bs_barrier(void *t, bus_space_ha bus_size_t len, int flags) { - /* Nothing to do. */ + /* + * dsb() will drain the L1 write buffer and establish a memory access + * barrier point on platforms where that has meaning. On a write we + * also need to drain the L2 write buffer, because most on-chip memory + * mapped devices are downstream of the L2 cache. Note that this needs + * to be done even for memory mapped as Device type, because while + * Device memory is not cached, writes to it are still buffered. + */ + dsb(); + if (flags & BUS_SPACE_BARRIER_WRITE) { + cpu_l2cache_drain_writebuf(); + } } Modified: head/sys/arm/arm/cpufunc.c ============================================================================== --- head/sys/arm/arm/cpufunc.c Sun May 11 04:18:51 2014 (r265869) +++ head/sys/arm/arm/cpufunc.c Sun May 11 04:24:57 2014 (r265870) @@ -150,6 +150,7 @@ struct cpu_functions arm9_cpufuncs = { (void *)cpufunc_nullop, /* l2cache_wbinv_range */ (void *)cpufunc_nullop, /* l2cache_inv_range */ (void *)cpufunc_nullop, /* l2cache_wb_range */ + (void *)cpufunc_nullop, /* l2cache_drain_writebuf */ /* Other functions */ @@ -214,6 +215,7 @@ struct cpu_functions armv5_ec_cpufuncs = (void *)cpufunc_nullop, /* l2cache_wbinv_range */ (void *)cpufunc_nullop, /* l2cache_inv_range */ (void *)cpufunc_nullop, /* l2cache_wb_range */ + (void *)cpufunc_nullop, /* l2cache_drain_writebuf */ /* Other functions */ @@ -276,6 +278,7 @@ struct cpu_functions sheeva_cpufuncs = { sheeva_l2cache_wbinv_range, /* l2cache_wbinv_range */ sheeva_l2cache_inv_range, /* l2cache_inv_range */ sheeva_l2cache_wb_range, /* l2cache_wb_range */ + (void *)cpufunc_nullop, /* l2cache_drain_writebuf */ /* Other functions */ @@ -338,6 +341,7 @@ struct cpu_functions arm10_cpufuncs = { (void *)cpufunc_nullop, /* l2cache_wbinv_range */ (void *)cpufunc_nullop, /* l2cache_inv_range */ (void *)cpufunc_nullop, /* l2cache_wb_range */ + (void *)cpufunc_nullop, /* l2cache_drain_writebuf */ /* Other functions */ @@ -401,6 +405,7 @@ struct cpu_functions pj4bv7_cpufuncs = { (void *)cpufunc_nullop, /* l2cache_wbinv_range */ (void *)cpufunc_nullop, /* l2cache_inv_range */ (void *)cpufunc_nullop, /* l2cache_wb_range */ + (void *)cpufunc_nullop, /* l2cache_drain_writebuf */ /* Other functions */ @@ -466,6 +471,7 @@ struct cpu_functions xscale_cpufuncs = { (void *)cpufunc_nullop, /* l2cache_wbinv_range */ (void *)cpufunc_nullop, /* l2cache_inv_range */ (void *)cpufunc_nullop, /* l2cache_wb_range */ + (void *)cpufunc_nullop, /* l2cache_drain_writebuf */ /* Other functions */ @@ -530,6 +536,7 @@ struct cpu_functions xscalec3_cpufuncs = xscalec3_l2cache_purge_rng, /* l2cache_wbinv_range */ xscalec3_l2cache_flush_rng, /* l2cache_inv_range */ xscalec3_l2cache_clean_rng, /* l2cache_wb_range */ + (void *)cpufunc_nullop, /* l2cache_drain_writebuf */ /* Other functions */ @@ -593,6 +600,7 @@ struct cpu_functions fa526_cpufuncs = { (void *)cpufunc_nullop, /* l2cache_wbinv_range */ (void *)cpufunc_nullop, /* l2cache_inv_range */ (void *)cpufunc_nullop, /* l2cache_wb_range */ + (void *)cpufunc_nullop, /* l2cache_drain_writebuf */ /* Other functions */ @@ -656,6 +664,7 @@ struct cpu_functions arm1136_cpufuncs = (void *)cpufunc_nullop, /* l2cache_wbinv_range */ (void *)cpufunc_nullop, /* l2cache_inv_range */ (void *)cpufunc_nullop, /* l2cache_wb_range */ + (void *)cpufunc_nullop, /* l2cache_drain_writebuf */ /* Other functions */ @@ -718,6 +727,7 @@ struct cpu_functions arm1176_cpufuncs = (void *)cpufunc_nullop, /* l2cache_wbinv_range */ (void *)cpufunc_nullop, /* l2cache_inv_range */ (void *)cpufunc_nullop, /* l2cache_wb_range */ + (void *)cpufunc_nullop, /* l2cache_drain_writebuf */ /* Other functions */ @@ -789,6 +799,7 @@ struct cpu_functions cortexa_cpufuncs = (void *)cpufunc_nullop, /* l2cache_wbinv_range */ (void *)cpufunc_nullop, /* l2cache_inv_range */ (void *)cpufunc_nullop, /* l2cache_wb_range */ + (void *)cpufunc_nullop, /* l2cache_drain_writebuf */ /* Other functions */ Modified: head/sys/arm/arm/pl310.c ============================================================================== --- head/sys/arm/arm/pl310.c Sun May 11 04:18:51 2014 (r265869) +++ head/sys/arm/arm/pl310.c Sun May 11 04:24:57 2014 (r265870) @@ -350,6 +350,18 @@ pl310_inv_range(vm_paddr_t start, vm_siz } static void +pl310_drain_writebuf(void) +{ + + if ((pl310_softc == NULL) || !pl310_softc->sc_enabled) + return; + + PL310_LOCK(pl310_softc); + pl310_cache_sync(); + PL310_UNLOCK(pl310_softc); +} + +static void pl310_set_way_sizes(struct pl310_softc *sc) { uint32_t aux_value; @@ -484,6 +496,7 @@ pl310_attach(device_t dev) cpufuncs.cf_l2cache_wbinv_range = pl310_wbinv_range; cpufuncs.cf_l2cache_inv_range = pl310_inv_range; cpufuncs.cf_l2cache_wb_range = pl310_wb_range; + cpufuncs.cf_l2cache_drain_writebuf = pl310_drain_writebuf; return (0); } Modified: head/sys/arm/include/cpufunc.h ============================================================================== --- head/sys/arm/include/cpufunc.h Sun May 11 04:18:51 2014 (r265869) +++ head/sys/arm/include/cpufunc.h Sun May 11 04:24:57 2014 (r265870) @@ -151,6 +151,7 @@ struct cpu_functions { void (*cf_l2cache_wbinv_range) (vm_offset_t, vm_size_t); void (*cf_l2cache_inv_range) (vm_offset_t, vm_size_t); void (*cf_l2cache_wb_range) (vm_offset_t, vm_size_t); + void (*cf_l2cache_drain_writebuf) (void); /* Other functions */ @@ -252,6 +253,7 @@ void tlb_broadcast(int); #define cpu_l2cache_wb_range(a, s) cpufuncs.cf_l2cache_wb_range((a), (s)) #define cpu_l2cache_inv_range(a, s) cpufuncs.cf_l2cache_inv_range((a), (s)) #define cpu_l2cache_wbinv_range(a, s) cpufuncs.cf_l2cache_wbinv_range((a), (s)) +#define cpu_l2cache_drain_writebuf() cpufuncs.cf_l2cache_drain_writebuf() #define cpu_flush_prefetchbuf() cpufuncs.cf_flush_prefetchbuf() #define cpu_drain_writebuf() cpufuncs.cf_drain_writebuf()