From owner-svn-src-all@FreeBSD.ORG Tue Jun 9 23:54:22 2015 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 35A9BA08; Tue, 9 Jun 2015 23:54:22 +0000 (UTC) (envelope-from zbb@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1724B1D25; Tue, 9 Jun 2015 23:54:22 +0000 (UTC) (envelope-from zbb@FreeBSD.org) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.9/8.14.9) with ESMTP id t59NsL0f006117; Tue, 9 Jun 2015 23:54:21 GMT (envelope-from zbb@FreeBSD.org) Received: (from zbb@localhost) by svn.freebsd.org (8.14.9/8.14.9/Submit) id t59NsLto006114; Tue, 9 Jun 2015 23:54:21 GMT (envelope-from zbb@FreeBSD.org) Message-Id: <201506092354.t59NsLto006114@svn.freebsd.org> X-Authentication-Warning: svn.freebsd.org: zbb set sender to zbb@FreeBSD.org using -f From: Zbigniew Bodek Date: Tue, 9 Jun 2015 23:54:21 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r284196 - in head/sys/arm64: arm64 include X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Jun 2015 23:54:22 -0000 Author: zbb Date: Tue Jun 9 23:54:20 2015 New Revision: 284196 URL: https://svnweb.freebsd.org/changeset/base/284196 Log: Add options to dmb() and dsb() macros on ARM64 Using plain dsb()/dmb() as full system barriers is usually to much. Adding proper options to those barriers (instead of full system - sy) will most likely reduce the cost of the instructions and will benefit in performance improvement. This commit adds options to barrier macro definitions. Obtained from: Semihalf Reviewed by: andrew, ian Sponsored by: The FreeBSD Foundation Modified: head/sys/arm64/arm64/db_interface.c head/sys/arm64/arm64/vfp.c head/sys/arm64/include/atomic.h Modified: head/sys/arm64/arm64/db_interface.c ============================================================================== --- head/sys/arm64/arm64/db_interface.c Tue Jun 9 23:13:37 2015 (r284195) +++ head/sys/arm64/arm64/db_interface.c Tue Jun 9 23:54:20 2015 (r284196) @@ -156,13 +156,11 @@ db_write_bytes(vm_offset_t addr, size_t } *dst++ = *data++; } + dsb(ish); - dsb(); /* Clean D-cache and invalidate I-cache */ cpu_dcache_wb_range(addr, (vm_size_t)size); cpu_icache_sync_range(addr, (vm_size_t)size); - dsb(); - isb(); return (0); } Modified: head/sys/arm64/arm64/vfp.c ============================================================================== --- head/sys/arm64/arm64/vfp.c Tue Jun 9 23:13:37 2015 (r284195) +++ head/sys/arm64/arm64/vfp.c Tue Jun 9 23:54:20 2015 (r284196) @@ -120,7 +120,7 @@ vfp_save_state(struct thread *td) td->td_pcb->pcb_fpcr = fpcr; td->td_pcb->pcb_fpsr = fpsr; - dsb(); + dsb(ish); vfp_disable(); } critical_exit(); Modified: head/sys/arm64/include/atomic.h ============================================================================== --- head/sys/arm64/include/atomic.h Tue Jun 9 23:13:37 2015 (r284195) +++ head/sys/arm64/include/atomic.h Tue Jun 9 23:54:20 2015 (r284196) @@ -29,13 +29,29 @@ #ifndef _MACHINE_ATOMIC_H_ #define _MACHINE_ATOMIC_H_ -#define isb() __asm __volatile("isb" : : : "memory") -#define dsb() __asm __volatile("dsb sy" : : : "memory") -#define dmb() __asm __volatile("dmb sy" : : : "memory") - -#define mb() dmb() -#define wmb() dmb() -#define rmb() dmb() +#define isb() __asm __volatile("isb" : : : "memory") + +/* + * Options for DMB and DSB: + * oshld Outer Shareable, load + * oshst Outer Shareable, store + * osh Outer Shareable, all + * nshld Non-shareable, load + * nshst Non-shareable, store + * nsh Non-shareable, all + * ishld Inner Shareable, load + * ishst Inner Shareable, store + * ish Inner Shareable, all + * ld Full system, load + * st Full system, store + * sy Full system, all + */ +#define dsb(opt) __asm __volatile("dsb " __STRING(opt) : : : "memory") +#define dmb(opt) __asm __volatile("dmb " __STRING(opt) : : : "memory") + +#define mb() dmb(sy) /* Full system memory barrier all */ +#define wmb() dmb(st) /* Full system memory barrier store */ +#define rmb() dmb(ld) /* Full system memory barrier load */ static __inline void atomic_add_32(volatile uint32_t *p, uint32_t val)