From owner-freebsd-arch@FreeBSD.ORG Sat Sep 8 16:52:36 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 82E491065674; Sat, 8 Sep 2012 16:52:36 +0000 (UTC) (envelope-from marcel@xcllnt.net) Received: from mail.xcllnt.net (mail.xcllnt.net [70.36.220.4]) by mx1.freebsd.org (Postfix) with ESMTP id D7F418FC12; Sat, 8 Sep 2012 16:52:35 +0000 (UTC) Received: from marcelm-sslvpn-nc.jnpr.net (natint3.juniper.net [66.129.224.36]) (authenticated bits=0) by mail.xcllnt.net (8.14.5/8.14.5) with ESMTP id q88GqOsO028614 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sat, 8 Sep 2012 09:52:24 -0700 (PDT) (envelope-from marcel@xcllnt.net) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) From: Marcel Moolenaar In-Reply-To: <1346857799.59094.66.camel@revolution.hippie.lan> Date: Sat, 8 Sep 2012 09:52:23 -0700 Content-Transfer-Encoding: 7bit Message-Id: <01CF0B3B-1F24-4BAA-A662-356796DE2EDA@xcllnt.net> References: <1346689154.1140.601.camel@revolution.hippie.lan> <3AFC763F-011C-46B4-B500-FE21B704259F@bsdimp.com> <1346857799.59094.66.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1486) Cc: freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Some busdma stats X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Sep 2012 16:52:36 -0000 On Sep 5, 2012, at 8:09 AM, Ian Lepore wrote: > For that to work, the busdma implementation needs to be able to > efficiently allocate buffers that are properly aligned and padded and > especially that are guaranteed not to share a cache line with some other > unrelated data. The busdma implementation can't get those guarantees > from malloc(9), and the alternatives (contigmalloc(), and the kmem_alloc > family) only work in page-sized chunks. We're asking drivers to > allocate individual buffers of sometimes no more than a few bytes each. In my new busdma implementation this is indeed one of the things that get addressed. In fact, it contains a reverse mapping from the busdma tag as we know it (representing the device's perspective) to a new memory tag used to represent the corresponding CPU perspective. It's the memory tag that can only be used for allocating DMA-capable memory, as bus implementation may have non-trivial address transfor- mations. Note that the memory tag is where you assert CPU constraints, like cacheline alignment. See also: http://wiki.freebsd.org/UnifiedBusDma I'm working on the projects/altix2 branch right now. > So that's all I'm addressing in the patchset I submitted: make sure > that when we start fixing drivers to allocate 256 individual 16-byte IO > descriptors that it shares with the hardware, that doesn't result in > allocating 256 pages of memory. Yup. > Also, if the request is for > BUS_DMA_COHERENT memory, make sure that doesn't result in turning off > caching in up to 256 pages that each contain a 16 byte IO buffer and > 4080 bytes of unrelated data. Or worse: you end up with multiple incompatible translations for the same physical page. This tends to cause machine checks. The busdma infrastructure needs to maintain it's own list of physical pages and do so per "attribute". BTW: Are you interested in fixing bounce buffering as well? -- Marcel Moolenaar marcel@xcllnt.net