From owner-svn-src-head@FreeBSD.ORG Tue Feb 24 19:09:19 2015 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0A8ECD39; Tue, 24 Feb 2015 19:09:19 +0000 (UTC) Received: from pmta1.delivery2.ore.mailhop.org (pmta1.delivery2.ore.mailhop.org [54.149.210.130]) by mx1.freebsd.org (Postfix) with ESMTP id DB682866; Tue, 24 Feb 2015 19:09:18 +0000 (UTC) Received: from smtp4.ore.mailhop.org (172.31.18.134) by pmta1.delivery1.ore.mailhop.org id htj65q20r84j; Tue, 24 Feb 2015 18:59:05 +0000 (envelope-from ) Received: from c-73-34-117-227.hsd1.co.comcast.net ([73.34.117.227] helo=ilsoft.org) by smtp4.ore.mailhop.org with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.82) (envelope-from ) id 1YQKhE-00084V-Oh; Tue, 24 Feb 2015 18:59:04 +0000 Received: from revolution.hippie.lan (revolution.hippie.lan [172.22.42.240]) by ilsoft.org (8.14.9/8.14.9) with ESMTP id t1OIx2Wv011500; Tue, 24 Feb 2015 11:59:02 -0700 (MST) (envelope-from ian@freebsd.org) X-Mail-Handler: DuoCircle Outbound SMTP X-Originating-IP: 73.34.117.227 X-Report-Abuse-To: abuse@duocircle.com (see https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information for abuse reporting information) X-MHO-User: U2FsdGVkX1+ubGUb8752Vu8/JTDpDnTp Message-ID: <1424804342.3293.9.camel@freebsd.org> Subject: Re: svn commit: r279236 - head/sys/netinet From: Ian Lepore To: John-Mark Gurney Date: Tue, 24 Feb 2015 11:59:02 -0700 In-Reply-To: <20150224173413.GF46794@funkthat.com> References: <201502241257.t1OCv40V097418@svn.freebsd.org> <20150224173413.GF46794@funkthat.com> Content-Type: text/plain; charset="us-ascii" X-Mailer: Evolution 3.12.8 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Cc: Zbigniew Bodek , svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 19:09:19 -0000 On Tue, 2015-02-24 at 09:34 -0800, John-Mark Gurney wrote: > Zbigniew Bodek wrote this message on Tue, Feb 24, 2015 at 12:57 +0000: > > Author: zbb > > Date: Tue Feb 24 12:57:03 2015 > > New Revision: 279236 > > URL: https://svnweb.freebsd.org/changeset/base/279236 > > > > Log: > > Change struct attribute to avoid aligned operations mismatch > > > > Previous __alignment(4) allowed compiler to assume that operations are > > performed on aligned region. On ARM processor, this led to alignment fault > > as shown below: > > trapframe: 0xda9e5b10 > > FSR=00000001, FAR=a67b680e, spsr=60000113 > > r0 =00000000, r1 =00000068, r2 =0000007c, r3 =00000000 > > r4 =a67b6826, r5 =a67b680e, r6 =00000014, r7 =00000068 > > r8 =00000068, r9 =da9e5bd0, r10=00000011, r11=da9e5c10 > > r12=da9e5be0, ssp=da9e5b60, slr=a054f164, pc =a054f2cc > > <...> > > udp_input+0x264: ldmia r5, {r0-r3, r6} > > udp_input+0x268: stmia r12, {r0-r3, r6} > > > > This was due to instructions which do not support unaligned access, > > whereas for __alignment(2) compiler replaced ldmia/stmia with some > > logically equivalent memcpy operations. > > In fact, the assumption that 'struct ip' is always 4-byte aligned > > is definitely false, as we have no impact on data alignment of packet > > stream received. > > So, the whole point of ETHER_ALIGN is to make struct ip aligned on > 4 byte offsets... This will probably impact performance on arm for > properly aligned struct ip... > ETHER_ALIGN is wonderful... if you're on a platform that can DMA to an arbitrary boundary. Of course, if you're on such a platform it can probably just access the word-sized values on halfword boundaries anyway. For arm, the only solution at the driver level is to memcpy() every incoming packet to another buffer to realign it. If you think that makes receive performance really bad, you'd be right. Many arm platforms can only DMA on a cacheline boundary. The size of an mbuf header is like 24 or 28 or something, definitely not cache aligned. So in addition to the extra copying the drivers do for ETHER_ALIGN, there could also be bounce-buffer copying involved due to the alignment in the busdma tag. The latter issue could be fixed with an MD padding field at the end of the mbuf header to make the data portion start on a cache line boundary. When I experimented with that concept on imx6 I gained 10 MB/sec performance from the reduced copying. -- Ian