From owner-freebsd-arch@FreeBSD.ORG Sun Aug 26 17:42:23 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BA3D91065670; Sun, 26 Aug 2012 17:42:23 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id 0634D8FC20; Sun, 26 Aug 2012 17:42:06 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id q7QHg5RZ067335; Sun, 26 Aug 2012 11:42:05 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id q7QHg2M1029199; Sun, 26 Aug 2012 11:42:02 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) From: Ian Lepore To: Warner Losh In-Reply-To: References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> <1345766109.27688.606.camel@revolution.hippie.lan> Content-Type: text/plain; charset="us-ascii" Date: Sun, 26 Aug 2012 11:42:02 -0600 Message-ID: <1346002922.1140.56.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: Adrian Chadd , Hans Petter Selasky , freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Aug 2012 17:42:23 -0000 On Thu, 2012-08-23 at 22:00 -0600, Warner Losh wrote: > The bottom line is that you can't mix things like that when cache > lines are involved. The current code that tries is doomed to failure. > Doomed. You just can't control all flushes, as Ian's missive > demonstrates, and trying to accommodate code that does this I don't > think can possibly work. All the interrupt masking, copying in and > out, etc I fear is doomed to utter and abject failure. > Until last weekend I was in the camp that thought the partial cacheline flush problem was solvable with sufficiently clever code. Now I agree that we're doomed to failure and it's time to try another direction. We're going to have some implementation work to do in arm and mips busdma, but I think the larger part of the task is going to be defining more rigorously how a driver must interact with the busdma system to function correctly on all types of platforms, and then update existing drivers to conform. The busdma manpage currently has some vague words about the usage and sequencing of sync ops, such as "If read and write operations are not preceded and followed by the appropriate synchronization operations, behavior is undefined." I think we should more explicitly spell out what the appropriate sequences are. In particular: * The PRE and POST operations must occur in pairs; a PREREAD must be followed eventually by a POSTREAD and a PREWRITE must be followed by a POSTWRITE. * The CPU is not allowed to access the mapped memory after a PRE sync and before the corresponding POST sync. * The DMA hardware is not allowed to access the mapped memory after a POST sync and before the next PRE sync. * Read and write sync operators may be combined in a single call, PRE and POST operators may not be. E.G., PREREAD|PREWRITE is allowed, PREREAD|POSTREAD is not. We should note that while read and write operations may be combined, on some platforms PREREAD|PREWRITE is needlessly expensive when only a read is being performed. We also need some rules about working with buffers obtained from bus_dmamem_alloc() and external buffers passed to bus_dmamap_load(). I think the rule should be that a buffer obtained from bus_dmamem_alloc(), or more formally any region of memory mapped by a bus_dmamap_load(), is a single logical object which can only be accessed by one entity at a time. That means that there cannot be two concurrent DMA operations happening in different regions of the same buffer, nor can DMA and CPU access be happening concurrently even if in different parts of the buffer. I've always thought that allocating a dma buffer feels like a big hassle. You sometimes have to create a tag for the sole purpose of setting the maxsize to get the buffer size you need when you call bus_dmamem_alloc(). If bus_dmamem_alloc() took a size parm you could just use your parent tag, or a generic tag appropriate to all the IO you're doing for a given device. If you need a variety of buffers for small control and command and status transfers of different sizes, you end up having to manage up to a dozen tags and maps and buffers. It's all very clunky and inconvenient. It's just the sort of thing that makes you want to allocate a big buffer and subdivide it. Surely we could do something to make it easier? -- Ian From owner-freebsd-arch@FreeBSD.ORG Sun Aug 26 18:05:31 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0553D106566B; Sun, 26 Aug 2012 18:05:31 +0000 (UTC) (envelope-from marktinguely@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id B86078FC08; Sun, 26 Aug 2012 18:05:30 +0000 (UTC) Received: by dadr6 with SMTP id r6so1986829dad.13 for ; Sun, 26 Aug 2012 11:05:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Uw70O9ZjXEv0noKSxgiAWrFGrA7R41RTQBz+00qiyXg=; b=lRMVdUjT02PasXSBwruHEy+P+tM/5iIRceigrGli1+Adb6r+aF6KRIASE2HatMyk9R 0ydHl0W5mtYVWe/O6UM0CRSVtswg5BzKmV/rpcxgCSfl1qORXuv3xn1eRyk+sOjPNYZO UsgNWJwnflgqUBll5iG9sKtc1AeLU+U5T60xqEM1AVlrLAejg3fAJ0VKl6nIr65wZPKz vJwP5oR93/LEJv3bWXYUo7FgQ/NHLIvNydb3nQMXpPpqifD1Z0iBjgEHf5KgLaoJo4x2 DsuoqXjIV95lXHpAIcPj2D1MuD9zcHa9+/pDu30eLi0XrQhFBGxfoOO2HXfgIYMawTcW TgTQ== MIME-Version: 1.0 Received: by 10.68.221.70 with SMTP id qc6mr29050561pbc.92.1346004330201; Sun, 26 Aug 2012 11:05:30 -0700 (PDT) Received: by 10.68.229.227 with HTTP; Sun, 26 Aug 2012 11:05:29 -0700 (PDT) In-Reply-To: <1346002922.1140.56.camel@revolution.hippie.lan> References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> <1345766109.27688.606.camel@revolution.hippie.lan> <1346002922.1140.56.camel@revolution.hippie.lan> Date: Sun, 26 Aug 2012 13:05:29 -0500 Message-ID: From: Mark Tinguely To: Ian Lepore Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-arm@freebsd.org, freebsd-arch@freebsd.org, freebsd-mips@freebsd.org, Hans Petter Selasky Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Aug 2012 18:05:31 -0000 On Sun, Aug 26, 2012 at 12:42 PM, Ian Lepore wrote: > On Thu, 2012-08-23 at 22:00 -0600, Warner Losh wrote: >> The bottom line is that you can't mix things like that when cache >> lines are involved. The current code that tries is doomed to failure. >> Doomed. You just can't control all flushes, as Ian's missive >> demonstrates, and trying to accommodate code that does this I don't >> think can possibly work. All the interrupt masking, copying in and >> out, etc I fear is doomed to utter and abject failure. >> > Until last weekend I was in the camp that thought the partial cacheline > flush problem was solvable with sufficiently clever code. Now I agree > that we're doomed to failure and it's time to try another direction. > > We're going to have some implementation work to do in arm and mips > busdma, but I think the larger part of the task is going to be defining > more rigorously how a driver must interact with the busdma system to > function correctly on all types of platforms, and then update existing > drivers to conform. > > The busdma manpage currently has some vague words about the usage and > sequencing of sync ops, such as "If read and write operations are not > preceded and followed by the appropriate synchronization operations, > behavior is undefined." I think we should more explicitly spell out > what the appropriate sequences are. In particular: > > * The PRE and POST operations must occur in pairs; a PREREAD must > be followed eventually by a POSTREAD and a PREWRITE must be > followed by a POSTWRITE. > * The CPU is not allowed to access the mapped memory after a PRE > sync and before the corresponding POST sync. > * The DMA hardware is not allowed to access the mapped memory > after a POST sync and before the next PRE sync. > * Read and write sync operators may be combined in a single call, > PRE and POST operators may not be. E.G., PREREAD|PREWRITE is > allowed, PREREAD|POSTREAD is not. We should note that while > read and write operations may be combined, on some platforms > PREREAD|PREWRITE is needlessly expensive when only a read is > being performed. > > We also need some rules about working with buffers obtained from > bus_dmamem_alloc() and external buffers passed to bus_dmamap_load(). I > think the rule should be that a buffer obtained from bus_dmamem_alloc(), > or more formally any region of memory mapped by a bus_dmamap_load(), is > a single logical object which can only be accessed by one entity at a > time. That means that there cannot be two concurrent DMA operations > happening in different regions of the same buffer, nor can DMA and CPU > access be happening concurrently even if in different parts of the > buffer. > > I've always thought that allocating a dma buffer feels like a big > hassle. You sometimes have to create a tag for the sole purpose of > setting the maxsize to get the buffer size you need when you call > bus_dmamem_alloc(). If bus_dmamem_alloc() took a size parm you could > just use your parent tag, or a generic tag appropriate to all the IO > you're doing for a given device. If you need a variety of buffers for > small control and command and status transfers of different sizes, you > end up having to manage up to a dozen tags and maps and buffers. It's > all very clunky and inconvenient. It's just the sort of thing that > makes you want to allocate a big buffer and subdivide it. Surely we > could do something to make it easier? > > -- Ian I did a quick look at the drivers last summer. Most drivers do the right thing and use memory allocated from bus_dmamem_alloc(). It is easy for us to give them a cache aligned buffer. Some drivers use mbufs - 256 bytes which cache safe. Some drivers directly or indirectly malloc() a buffer and then use it to dma - rather than try to fix them all, I was okay with making the smallest malloc() amount equal to the cache line size. It amounts to getting rid of the 16 byte allocation on some ARM architectures. The power of 2 allocator will then give us cache line safe allocation. A few drivers take a small memory amount from the kernel stack and dma to it <- broken driver. The few drivers that use data from a structure and that memory is not cached aligned <- broken driver. --Mark Tinguely. From owner-freebsd-arch@FreeBSD.ORG Sun Aug 26 18:25:23 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8D04B1065674; Sun, 26 Aug 2012 18:25:23 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id 4881A8FC1E; Sun, 26 Aug 2012 18:25:10 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id q7QIPAH8068013; Sun, 26 Aug 2012 12:25:10 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id q7QIP87W029231; Sun, 26 Aug 2012 12:25:08 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) From: Ian Lepore To: Mark Tinguely In-Reply-To: References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> <1345766109.27688.606.camel@revolution.hippie.lan> <1346002922.1140.56.camel@revolution.hippie.lan> Content-Type: text/plain; charset="us-ascii" Date: Sun, 26 Aug 2012 12:25:07 -0600 Message-ID: <1346005507.1140.69.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-arm@freebsd.org, Hans Petter Selasky , freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Aug 2012 18:25:23 -0000 On Sun, 2012-08-26 at 13:05 -0500, Mark Tinguely wrote: > I did a quick look at the drivers last summer. > > Most drivers do the right thing and use memory allocated from > bus_dmamem_alloc(). It is easy for us to give them a cache aligned > buffer. > > Some drivers use mbufs - 256 bytes which cache safe. > > Some drivers directly or indirectly malloc() a buffer and then use it > to dma - rather than try to fix them all, I was okay with making the > smallest malloc() amount equal to the cache line size. It amounts to > getting rid of the 16 byte allocation on some ARM architectures. The > power of 2 allocator will then give us cache line safe allocation. > > A few drivers take a small memory amount from the kernel stack and dma > to it <- broken driver. > > The few drivers that use data from a structure and that memory is not > cached aligned <- broken driver. > I disagree about those last two points -- drivers that choose to use stack memory or malloc'd memory as IO buffers are not broken. Drivers can do IO directly to/from userland buffers, do we say that an application that calls read(2) and passes the address of a stack variable is broken? In this regard, it's the busdma implementation that's broken, because it should bounce those IOs through a DMA-safe buffer. There's absolutely no rule that I've ever heard of in FreeBSD that says IO can only take place using memory allocated from busdma. The rule is only that the proper sequence of busdma operation must be called, and beyond that it's up to the busdma implementation to make it work. Our biggest problem, I think, is that we don't have a sufficient definition of "the proper sequence of busdma operations." I don't think it will be very hard to make the arm and mips busdma implementations work correctly. It won't even be too hard to make them fairly efficient at bouncing small IOs (my thinking is that we can make small bounces no more expensive than the current partial cacheline flush implementation which copies the data multiple times). Bouncing large IO will never be efficient, but the inefficiency will be a powerful motivator to update drivers that do large IO to work better, such as using buffers allocated from busdma. -- Ian From owner-freebsd-arch@FreeBSD.ORG Sun Aug 26 18:53:47 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3AE0D106566C; Sun, 26 Aug 2012 18:53:47 +0000 (UTC) (envelope-from tim@kientzle.com) Received: from monday.kientzle.com (99-115-135-74.uvs.sntcca.sbcglobal.net [99.115.135.74]) by mx1.freebsd.org (Postfix) with ESMTP id E46D08FC16; Sun, 26 Aug 2012 18:53:46 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.4/8.14.4) id q7QIrb6b086928; Sun, 26 Aug 2012 18:53:37 GMT (envelope-from tim@kientzle.com) Received: from [192.168.2.143] (CiscoE3000 [192.168.1.65]) by kientzle.com with SMTP id y7e92cf8djshe4rhprsvmx2wr2; Sun, 26 Aug 2012 18:53:37 +0000 (UTC) (envelope-from tim@kientzle.com) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=windows-1252 From: Tim Kientzle In-Reply-To: Date: Sun, 26 Aug 2012 11:53:35 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> <1345766109.27688.606.camel@revolution.hippie.lan> <1346002922.1140.56.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1278) Cc: freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, "arch@" Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Aug 2012 18:53:47 -0000 On Sun, Aug 26, 2012 at 12:42 PM, Ian Lepore wrote: > On Thu, 2012-08-23 at 22:00 -0600, Warner Losh wrote: >> The bottom line is that you can't mix things like that when cache >> lines are involved. >>=20 > =85. I think we should more explicitly spell out > what the appropriate sequences are. As someone who is just tinkering with driver code for the first time, I applaud any attempts to better document this area! ;-) > In particular: >=20 > * The PRE and POST operations must occur in pairs; a PREREAD must > be followed eventually by a POSTREAD and a PREWRITE must be > followed by a POSTWRITE. > * The CPU is not allowed to access the mapped memory after a PRE > sync and before the corresponding POST sync. > * The DMA hardware is not allowed to access the mapped memory > after a POST sync and before the next PRE sync. These rules sound reasonable. Good documentation might also give examples of what the PRE/POST operations might entail (e.g., from the preceding discussion, it sounds like PREREAD and PREWRITE require at least a partial cache flush on ARM). That helps folks who are coming to the docs with some hardware background. > * Read and write sync operators may be combined in a single = call, > PRE and POST operators may not be. E.G., PREREAD|PREWRITE is > allowed, PREREAD|POSTREAD is not. We should note that while > read and write operations may be combined, on some platforms > PREREAD|PREWRITE is needlessly expensive when only a read is > being performed. PREREAD|POSTREAD doesn't sound useful to me, but why would it be explicitly forbidden? Would you also forbid POSTREAD|PREWRITE? (For a buffer that has just completed a DMA read and is going to be immediately used for a DMA write?) Tim From owner-freebsd-arch@FreeBSD.ORG Sun Aug 26 20:01:55 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AB765106564A; Sun, 26 Aug 2012 20:01:55 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id 4FF5D8FC1A; Sun, 26 Aug 2012 20:01:55 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id q7QK1s32069554; Sun, 26 Aug 2012 14:01:54 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id q7QK1qGi029391; Sun, 26 Aug 2012 14:01:52 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) From: Ian Lepore To: Tim Kientzle In-Reply-To: References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> <1345766109.27688.606.camel@revolution.hippie.lan> <1346002922.1140.56.camel@revolution.hippie.lan> Content-Type: text/plain; charset="us-ascii" Date: Sun, 26 Aug 2012 14:01:52 -0600 Message-ID: <1346011312.1140.107.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, "arch@" Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Aug 2012 20:01:55 -0000 On Sun, 2012-08-26 at 11:53 -0700, Tim Kientzle wrote: > These rules sound reasonable. Good documentation might > also give examples of what the PRE/POST operations might entail > (e.g., from the preceding discussion, it sounds like PREREAD > and PREWRITE require at least a partial cache flush on ARM). > That helps folks who are coming to the docs with some hardware > background. > I agree, I think it would be good to have something like a RATIONALE section in the manpage that summarizes the issues faced by various categories of platform (hardware coherency, software-assisted coherency, etc) and how they handle it. You don't want people coding to the implementation (some of which is going on now, and I've been guilty of it myself); if there were more info in the docs there'd be less motivation to peek at the implementation. > > * Read and write sync operators may be combined in a single call, > > PRE and POST operators may not be. E.G., PREREAD|PREWRITE is > > allowed, PREREAD|POSTREAD is not. We should note that while > > read and write operations may be combined, on some platforms > > PREREAD|PREWRITE is needlessly expensive when only a read is > > being performed. > > > PREREAD|POSTREAD doesn't sound useful to me, but > why would it be explicitly forbidden? > > Would you also forbid POSTREAD|PREWRITE? > (For a buffer that has just completed a DMA read > and is going to be immediately used for a DMA write?) My thinking on forbidding PREREAD|POSTREAD is at least partly that it removes some temptation to do the wrong thing: treat the busdma API as if it were a general cpu cache manipulation library. With the new definition of the sequences, a PREREAD|POSTREAD operation is nonsensical because it leaves no window during which the DMA hardware has access to memory; it is in effect a no-op. If you think in terms of implementation you might think "This would have to cause a cache invalidation." If you think in terms of API you should be thinking something more like "This marks out a time window during which the DMA hardware has safe access to that memory which has a duration of zero." POSTREAD|PREWRITE is interesting. It is not a no-op in terms of the API. It closes the hardware access window that was opened by an earlier PREREAD, and it opens a new hardware access window with the PREWRITE. Whether it touches hardware or caches or anything in a given implementation isn't the point, in terms of the API it's the right thing to do for a pair of back to back DMA operations, without intervening CPU access, on the same memory. So PREREAD|POSTREAD and PREWRITE|POSTWRITE make no sense, but all other combos should be allowed. Maybe instead of allowing and forbidding specific combos, we should just advise that these two are effectively no-ops. -- Ian From owner-freebsd-arch@FreeBSD.ORG Sun Aug 26 23:03:56 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2A1B71065670 for ; Sun, 26 Aug 2012 23:03:56 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id D1A308FC17 for ; Sun, 26 Aug 2012 23:03:55 +0000 (UTC) Received: by ialo14 with SMTP id o14so8766146ial.13 for ; Sun, 26 Aug 2012 16:03:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=ep5IxGfVukJ1EG6I0J6d7Tuvp5WIjgyi6rZXdyCg3ho=; b=L1UnkBHRbxKYcBFDQ7qmFv6gH7583UA2Rkz6wM8jUupcJympe8cKdtUv4WT8aqkhmc byvGokZdwyRvz/19Db9nsXwJtqX+whkUIiEyCL/W/dJeZOFArf9dCj+VMPSGtdDrWI+r itr3Jo+RBhngw5BjwDoX2vVBWg3tp+cm7/RQCXvFqamP0gQtd8aq59bJLmGOp4XxK0GS zdbT55qrRpiMCIeUc0LMyWOcjR6548wsERh6vX3vr3QM5mrvlIAcJ2ukL+kLB+FZT59T 1Aa0CyVZr8YKoSS5CDAld76hv6h5NQgogJ85T5aPUYrDyGAl3FJZ+GXIVTrqvTyxOD8l 2Vcg== Received: by 10.50.184.198 with SMTP id ew6mr8457358igc.27.1346022235175; Sun, 26 Aug 2012 16:03:55 -0700 (PDT) Received: from 63.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id ce6sm4966875igb.1.2012.08.26.16.03.48 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 26 Aug 2012 16:03:54 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <1346002922.1140.56.camel@revolution.hippie.lan> Date: Sun, 26 Aug 2012 17:03:46 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <6D83AF9D-577B-4C83-84B7-C4E3B32695FC@bsdimp.com> References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> <1345766109.27688.606.camel@revolution.hippie.lan> <1346002922.1140.56.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQnmOkuruuOba95rVN4udBHB3GbYPogQgwdSylZj5kvlK5XJv4Mnww8TCnLe4IoonGlczeUN Cc: Adrian Chadd , Hans Petter Selasky , freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Aug 2012 23:03:56 -0000 On Aug 26, 2012, at 11:42 AM, Ian Lepore wrote: > On Thu, 2012-08-23 at 22:00 -0600, Warner Losh wrote: >> The bottom line is that you can't mix things like that when cache >> lines are involved. The current code that tries is doomed to = failure. >> Doomed. You just can't control all flushes, as Ian's missive >> demonstrates, and trying to accommodate code that does this I don't >> think can possibly work. All the interrupt masking, copying in and >> out, etc I fear is doomed to utter and abject failure. =20 >>=20 > Until last weekend I was in the camp that thought the partial = cacheline > flush problem was solvable with sufficiently clever code. Now I agree > that we're doomed to failure and it's time to try another direction. >=20 > We're going to have some implementation work to do in arm and mips > busdma, but I think the larger part of the task is going to be = defining > more rigorously how a driver must interact with the busdma system to > function correctly on all types of platforms, and then update existing > drivers to conform. >=20 > The busdma manpage currently has some vague words about the usage and > sequencing of sync ops, such as "If read and write operations are not > preceded and followed by the appropriate synchronization operations, > behavior is undefined." I think we should more explicitly spell out > what the appropriate sequences are. In particular: >=20 > * The PRE and POST operations must occur in pairs; a PREREAD must > be followed eventually by a POSTREAD and a PREWRITE must be > followed by a POSTWRITE.=20 PREREAD means "I am about to tell the device to put data here, have = whaterver things might be pending in the CPU complex to get out of the = way." usually this means 'invalidate the cache for that range', but not = always. POSTREAD means 'The device's DMA is done, I'd like to start = accessing it now.' If the memory will be thrown away without being = looked at, then does the driver necessarily need to issue the POSTREAD? = I think so, but I don't know if that's a new requirement. > * The CPU is not allowed to access the mapped memory after a PRE > sync and before the corresponding POST sync. =20 Correct. > * The DMA hardware is not allowed to access the mapped memory > after a POST sync and before the next PRE sync.=20 Correct. > * Read and write sync operators may be combined in a single call, > PRE and POST operators may not be. E.G., PREREAD|PREWRITE is > allowed, PREREAD|POSTREAD is not. We should note that while > read and write operations may be combined, on some platforms > PREREAD|PREWRITE is needlessly expensive when only a read is > being performed. Correct. > We also need some rules about working with buffers obtained from > bus_dmamem_alloc() and external buffers passed to bus_dmamap_load(). = I > think the rule should be that a buffer obtained from = bus_dmamem_alloc(), > or more formally any region of memory mapped by a bus_dmamap_load(), = is > a single logical object which can only be accessed by one entity at a > time. That means that there cannot be two concurrent DMA operations > happening in different regions of the same buffer, nor can DMA and CPU > access be happening concurrently even if in different parts of the > buffer. =20 There's something subtle that I'm missing. Why would two DMA operations = be disallowed? The rest makes good sense. > I've always thought that allocating a dma buffer feels like a big > hassle. You sometimes have to create a tag for the sole purpose of > setting the maxsize to get the buffer size you need when you call > bus_dmamem_alloc(). If bus_dmamem_alloc() took a size parm you could > just use your parent tag, or a generic tag appropriate to all the IO > you're doing for a given device. If you need a variety of buffers for > small control and command and status transfers of different sizes, you > end up having to manage up to a dozen tags and maps and buffers. It's > all very clunky and inconvenient. It's just the sort of thing that > makes you want to allocate a big buffer and subdivide it. Surely we > could do something to make it easier? You'd wind up creating a quick tag on the fly for the bus_dmamap_alloc = if you wanted to do this. Cleanup then becomes unclear. Warner From owner-freebsd-arch@FreeBSD.ORG Sun Aug 26 23:13:35 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BC340106566C for ; Sun, 26 Aug 2012 23:13:35 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 6AFE18FC19 for ; Sun, 26 Aug 2012 23:13:34 +0000 (UTC) Received: by ialo14 with SMTP id o14so8778790ial.13 for ; Sun, 26 Aug 2012 16:13:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=ljCRbjZ9ASHSXhbZZlvbTJ/BcuQ0M4nJMzidj/rAYzc=; b=gQLUWH8H4WF5+fAkTGCGwmQh27HEh6WLHshRdWbQED0en6El9jNVvpdvA809HP0ZqI fBXuif2+qu2KD8ffo5dA3tIopHA7VcTA1MNuBSAZsgN+kNnEEnAcRSvwQM6u1SZAJ1k7 US+tDP8brfdR/8yUlJfgHlwvWLvt5ajhh2o49bh+vcOY+hZXJQHnr9aqKomDiKQguDRe 1HEMxnjEteSGX+YP7Dnf5Djpjfr7bIIsIl0M92sxNaINxm4FbzqeKTq4uL86m4A4zWOa pO64dazQqKMgoO75P0sOKQZsTwHJB4SCazOeP/RG3dPzF8+h5l8iDGyNltQgNlWWZOur E9ug== Received: by 10.50.187.229 with SMTP id fv5mr2371955igc.57.1346022814665; Sun, 26 Aug 2012 16:13:34 -0700 (PDT) Received: from 63.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id df1sm16929663igc.10.2012.08.26.16.13.32 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 26 Aug 2012 16:13:33 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <1346005507.1140.69.camel@revolution.hippie.lan> Date: Sun, 26 Aug 2012 17:13:31 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <10307B47-13F3-45C0-87F7-66FD3ACA3F86@bsdimp.com> References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> <1345766109.27688.606.camel@revolution.hippie.lan> <1346002922.1140.56.camel@revolution.hippie.lan> <1346005507.1140.69.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQninWUHm92egJMDW4ug3EBrCkYlO3yWGOR1Ws+E9H6NtFslwS3L0dzb0o1kAGYUv6PUwxOW Cc: freebsd-arm@freebsd.org, freebsd-arch@freebsd.org, Mark Tinguely , freebsd-mips@freebsd.org, Hans Petter Selasky Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Aug 2012 23:13:35 -0000 On Aug 26, 2012, at 12:25 PM, Ian Lepore wrote: > On Sun, 2012-08-26 at 13:05 -0500, Mark Tinguely wrote: >> I did a quick look at the drivers last summer. >>=20 >> Most drivers do the right thing and use memory allocated from >> bus_dmamem_alloc(). It is easy for us to give them a cache aligned >> buffer. >>=20 >> Some drivers use mbufs - 256 bytes which cache safe. >>=20 >> Some drivers directly or indirectly malloc() a buffer and then use it >> to dma - rather than try to fix them all, I was okay with making the >> smallest malloc() amount equal to the cache line size. It amounts to >> getting rid of the 16 byte allocation on some ARM architectures. The >> power of 2 allocator will then give us cache line safe allocation. >>=20 >> A few drivers take a small memory amount from the kernel stack and = dma >> to it <- broken driver. >>=20 >> The few drivers that use data from a structure and that memory is not >> cached aligned <- broken driver. >>=20 >=20 > I disagree about those last two points -- drivers that choose to use > stack memory or malloc'd memory as IO buffers are not broken. Stack DMA is bad policy, at best, and broken at worst. The reason is = because of alignment of the underlying unit. Since there's no way to = say that something is aligned to a given spot on the stack, you are = asking for random stack corruption. Also, malloced area is similarly problematic: There's no cache line = informing of the allocator, so you can wind up with an allocation of = memory that's corrupted due to cache effects. > Drivers > can do IO directly to/from userland buffers, do we say that an > application that calls read(2) and passes the address of a stack > variable is broken? Yes, if it is smaller than a cache line size, and not aligned to the = cache line. That's the point of the uio load variant. > In this regard, it's the busdma implementation that's broken, because = it > should bounce those IOs through a DMA-safe buffer. There's absolutely > no rule that I've ever heard of in FreeBSD that says IO can only take > place using memory allocated from busdma. That's partially true. Since BUSDMA grew up in the storage area, you = must allocate the memory from busdma, or it must be page aligned has = been the de-facto rule here. The mbuf and uio variants of load were = invented to cope with common cases of mbufs and user I/O to properly = flag things. How does busdma know that it is using memory that's not from its = allocator? > The rule is only that the > proper sequence of busdma operation must be called, and beyond that = it's > up to the busdma implementation to make it work. =20 No. Bouncing is needed due to poor alignment of the underlying device. = Not due to cache effects. There's a limited number of things that we support with busdma. = Arbitrary data from malloc that might be shared with the CPU isn't on = that list. > Our biggest problem, I think, is that we don't have a sufficient > definition of "the proper sequence of busdma operations." I disagree. The sequence has been known for a long time. > I don't think it will be very hard to make the arm and mips busdma > implementations work correctly. It won't even be too hard to make = them > fairly efficient at bouncing small IOs (my thinking is that we can = make > small bounces no more expensive than the current partial cacheline = flush > implementation which copies the data multiple times). Bouncing large = IO > will never be efficient, but the inefficiency will be a powerful > motivator to update drivers that do large IO to work better, such as > using buffers allocated from busdma. I don't think the cache line problem can be solved with bounce buffers. = Trying to accommodate broken drivers is what lead us to this spot. We = need to fix the broken drivers. If that's impossible, then the best we = can do is have the driver set a 'always bounce' flag in the tag it = creates and use that to always bounce for operations through that tag. Warner From owner-freebsd-arch@FreeBSD.ORG Sun Aug 26 23:15:31 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id F1CA61065673 for ; Sun, 26 Aug 2012 23:15:30 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 6381F8FC15 for ; Sun, 26 Aug 2012 23:15:30 +0000 (UTC) Received: by ialo14 with SMTP id o14so8781099ial.13 for ; Sun, 26 Aug 2012 16:15:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=iYEhywsT8AYLxmmanrymiKw7wLde2oh04bmK21Zf8Ko=; b=TwKbb2jYXvKi5wPtCC4mwMaBLnEx9ps0oHPxgV1vVFAomH5vQvmmGEt9YfASeo6i8x 610Egx0PuHmunxXzXAYSZ/uW1DUFcPfTURm6BYZG+gRvGzGYku1kPlWJLQa15qGlINcm UMv96eSvm59vDQcai/t773VUECsiWXa4Mvxzn2VBSeAjK8uUDb0kLDer3bkefgwiT1VW vPuXRjyFMO6tF80DX+d8vUWDsLyB9yPtdAdZi5oP6T682FXt7cD6yoPz5Q80DNcmxrME Uk2N7S1SrXrKPwd5R66jv87WKwF0yR24l6pjOkZhQn2eFW9Octkr8SV9aReGAYQCtjMs MeSg== Received: by 10.50.195.169 with SMTP id if9mr2347874igc.62.1346022930007; Sun, 26 Aug 2012 16:15:30 -0700 (PDT) Received: from 63.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id s2sm9413489igb.5.2012.08.26.16.15.27 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 26 Aug 2012 16:15:28 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=windows-1252 From: Warner Losh In-Reply-To: Date: Sun, 26 Aug 2012 17:15:26 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> <1345766109.27688.606.camel@revolution.hippie.lan> <1346002922.1140.56.camel@revolution.hippie.lan> To: Tim Kientzle X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQlTvEOaeSkGYeyU5t64BsYt3+mAReOXMB9n4IGBJ3Iq/HQgx+WfNRW0ra2zsLbJJSoH//nE Cc: Ian Lepore , freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, "arch@" Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Aug 2012 23:15:31 -0000 On Aug 26, 2012, at 12:53 PM, Tim Kientzle wrote: >=20 > On Sun, Aug 26, 2012 at 12:42 PM, Ian Lepore > wrote: >> On Thu, 2012-08-23 at 22:00 -0600, Warner Losh wrote: >>> The bottom line is that you can't mix things like that when cache >>> lines are involved. >>>=20 >> =85. I think we should more explicitly spell out >> what the appropriate sequences are. >=20 >=20 > As someone who is just tinkering with driver code for > the first time, I applaud any attempts to better document > this area! ;-) >=20 >> In particular: >>=20 >> * The PRE and POST operations must occur in pairs; a PREREAD must >> be followed eventually by a POSTREAD and a PREWRITE must be >> followed by a POSTWRITE. >> * The CPU is not allowed to access the mapped memory after a PRE >> sync and before the corresponding POST sync. >> * The DMA hardware is not allowed to access the mapped memory >> after a POST sync and before the next PRE sync. >=20 >=20 > These rules sound reasonable. Good documentation might > also give examples of what the PRE/POST operations might entail > (e.g., from the preceding discussion, it sounds like PREREAD > and PREWRITE require at least a partial cache flush on ARM). > That helps folks who are coming to the docs with some hardware > background. >=20 >> * Read and write sync operators may be combined in a single = call, >> PRE and POST operators may not be. E.G., PREREAD|PREWRITE is >> allowed, PREREAD|POSTREAD is not. We should note that while >> read and write operations may be combined, on some platforms >> PREREAD|PREWRITE is needlessly expensive when only a read is >> being performed. >=20 >=20 > PREREAD|POSTREAD doesn't sound useful to me, but > why would it be explicitly forbidden? It could be useful when DMAing from multiple devices. Let's say I have = a buffer that I want to get data from two different DMA operations. = Wouldn't this be how you'd specify it? > Would you also forbid POSTREAD|PREWRITE? > (For a buffer that has just completed a DMA read > and is going to be immediately used for a DMA write?) This could be useful for some zero-copy things, no? Warner > Tim >=20 > _______________________________________________ > freebsd-arm@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arm > To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Sun Aug 26 23:18:57 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8245C1065673 for ; Sun, 26 Aug 2012 23:18:57 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 363698FC12 for ; Sun, 26 Aug 2012 23:18:57 +0000 (UTC) Received: by ialo14 with SMTP id o14so8785191ial.13 for ; Sun, 26 Aug 2012 16:18:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=vV/55IW1m35Nojy+yoYqvhV4JtFulO/qaNQbLw6wfac=; b=oQZIoqj9ZnwL1rnFOkiVlUXISpEY9ZMUlruwE5G0aGVzYuxJ8NnK0DsFK56Kb4Q9XL tmI3vqAy4ebzDg4k1bGLQdj0Im4v8VxiyGbFxuhW80Z4d2lmESBZeoyqi2HnMsozqoFJ rsAYFe4gIDc6f8WRKvLLaAukoAQ8nNKOIcrwm59w0HL2I2lHbq7lv3umpbMjy9KxzuAt ngjIIvjZsVcB7ZiPIS0bSGqsCjGBL8ruvyqL+bDD6MXRMONqrebPUXBEM9J53i2KtXkg P8/3fRcfgmnUD2JpqiUdOrLW2/u9zz3Q03krrRrNlYILRA8vbGMVtVJ/KiGDsfR6fs0m 275g== Received: by 10.50.88.229 with SMTP id bj5mr8287211igb.21.1346023136744; Sun, 26 Aug 2012 16:18:56 -0700 (PDT) Received: from 63.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id i2sm4979193igl.8.2012.08.26.16.18.54 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 26 Aug 2012 16:18:55 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <1346011312.1140.107.camel@revolution.hippie.lan> Date: Sun, 26 Aug 2012 17:18:54 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <26F65164-EC99-483E-9FA8-916AD99CBCBC@bsdimp.com> References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> <1345766109.27688.606.camel@revolution.hippie.lan> <1346002922.1140.56.camel@revolution.hippie.lan> <1346011312.1140.107.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQlziAMJONvjseTSWuCERXucnrFbuHswKaESo1zr+FvsdjNpCVV/HoF4pYOhIcdntiwy1Jeg Cc: Tim Kientzle , freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, "arch@" Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Aug 2012 23:18:57 -0000 On Aug 26, 2012, at 2:01 PM, Ian Lepore wrote: > On Sun, 2012-08-26 at 11:53 -0700, Tim Kientzle wrote: >> These rules sound reasonable. Good documentation might >> also give examples of what the PRE/POST operations might entail >> (e.g., from the preceding discussion, it sounds like PREREAD >> and PREWRITE require at least a partial cache flush on ARM). >> That helps folks who are coming to the docs with some hardware >> background. >>=20 >=20 > I agree, I think it would be good to have something like a RATIONALE > section in the manpage that summarizes the issues faced by various > categories of platform (hardware coherency, software-assisted = coherency, > etc) and how they handle it. You don't want people coding to the > implementation (some of which is going on now, and I've been guilty of > it myself); if there were more info in the docs there'd be less > motivation to peek at the implementation. More documentation is good. >>> * Read and write sync operators may be combined in a single = call, >>> PRE and POST operators may not be. E.G., PREREAD|PREWRITE is >>> allowed, PREREAD|POSTREAD is not. We should note that while >>> read and write operations may be combined, on some platforms >>> PREREAD|PREWRITE is needlessly expensive when only a read is >>> being performed. >>=20 >>=20 >> PREREAD|POSTREAD doesn't sound useful to me, but >> why would it be explicitly forbidden? >>=20 >> Would you also forbid POSTREAD|PREWRITE? >> (For a buffer that has just completed a DMA read >> and is going to be immediately used for a DMA write?) >=20 > My thinking on forbidding PREREAD|POSTREAD is at least partly that it > removes some temptation to do the wrong thing: treat the busdma API as > if it were a general cpu cache manipulation library. =20 I think it should be read as POSTREAD + PREREAD, not PREREAD + POSTREAD. > With the new definition of the sequences, a PREREAD|POSTREAD operation > is nonsensical because it leaves no window during which the DMA = hardware > has access to memory; it is in effect a no-op. If you think in terms = of > implementation you might think "This would have to cause a cache > invalidation." If you think in terms of API you should be thinking > something more like "This marks out a time window during which the DMA > hardware has safe access to that memory which has a duration of zero." Acutally, it would allow one to shift the DMA from one device to = another, if read the way I say above. > POSTREAD|PREWRITE is interesting. It is not a no-op in terms of the > API. It closes the hardware access window that was opened by an = earlier > PREREAD, and it opens a new hardware access window with the PREWRITE. > Whether it touches hardware or caches or anything in a given > implementation isn't the point, in terms of the API it's the right = thing > to do for a pair of back to back DMA operations, without intervening = CPU > access, on the same memory. Right. This is useful in many cases. Consider a log structured device = that reads data into memory, and then writes it back to the tail of the = log as a defragging operation. > So PREREAD|POSTREAD and PREWRITE|POSTWRITE make no sense, but all = other > combos should be allowed. Maybe instead of allowing and forbidding > specific combos, we should just advise that these two are effectively > no-ops. I respectfully disagree.. Warner From owner-freebsd-arch@FreeBSD.ORG Mon Aug 27 01:26:09 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E6B57106564A; Mon, 27 Aug 2012 01:26:09 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 9BEEE8FC16; Mon, 27 Aug 2012 01:26:09 +0000 (UTC) Received: by pbbrp2 with SMTP id rp2so6609002pbb.13 for ; Sun, 26 Aug 2012 18:26:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=buRkMCmZXq3F4Cc5OBO5pchRTJfQaZIQI48CqnG/UVM=; b=PAnNo8QWePhAymqZ/mWGbYh8Urc23Jl/DWWf7mLRxsT+q8Oxkez8Ol9YphyhVM3q35 x/ASzYBiRdpuZhiL4yVUHmVmQkRfh5DgoUCSEMxG862EH+bdfqJ/t1lUxC74+V4NwP9X IMQ1/qx0CsZW79aQogmq3fkNjK/c6aBIlf1Rg8vNjATaoGxRqTsGunzWDw50lXV5K+sG 7W9419jOZyDWkUxOu483e5zgItkzYa9JddxiLzDftB90igFeVRO9mKbjHmdA+jT8lxeS LDve5jhPhL6WMxa0canqUNYzriSRFegQcasvJtHFx6zV/HjHn2eqZaRK+Gl9lyoq+Zwp h0DQ== MIME-Version: 1.0 Received: by 10.68.129.131 with SMTP id nw3mr30406099pbb.43.1346030769022; Sun, 26 Aug 2012 18:26:09 -0700 (PDT) Received: by 10.68.36.106 with HTTP; Sun, 26 Aug 2012 18:26:08 -0700 (PDT) Received: by 10.68.36.106 with HTTP; Sun, 26 Aug 2012 18:26:08 -0700 (PDT) In-Reply-To: <26F65164-EC99-483E-9FA8-916AD99CBCBC@bsdimp.com> References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> <1345766109.27688.606.camel@revolution.hippie.lan> <1346002922.1140.56.camel@revolution.hippie.lan> <1346011312.1140.107.camel@revolution.hippie.lan> <26F65164-EC99-483E-9FA8-916AD99CBCBC@bsdimp.com> Date: Sun, 26 Aug 2012 18:26:08 -0700 Message-ID: From: Adrian Chadd To: Warner Losh Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Ian Lepore , freebsd-arm@freebsd.org, "arch@" , freebsd-mips@freebsd.org, Tim Kientzle Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2012 01:26:10 -0000 Hm. So perhaps we should document the busdma assumptions with respect to alignment and coherency? So this doesnt happen again? Adrian On Aug 26, 2012 4:19 PM, "Warner Losh" wrote: > > On Aug 26, 2012, at 2:01 PM, Ian Lepore wrote: > > > On Sun, 2012-08-26 at 11:53 -0700, Tim Kientzle wrote: > >> These rules sound reasonable. Good documentation might > >> also give examples of what the PRE/POST operations might entail > >> (e.g., from the preceding discussion, it sounds like PREREAD > >> and PREWRITE require at least a partial cache flush on ARM). > >> That helps folks who are coming to the docs with some hardware > >> background. > >> > > > > I agree, I think it would be good to have something like a RATIONALE > > section in the manpage that summarizes the issues faced by various > > categories of platform (hardware coherency, software-assisted coherency, > > etc) and how they handle it. You don't want people coding to the > > implementation (some of which is going on now, and I've been guilty of > > it myself); if there were more info in the docs there'd be less > > motivation to peek at the implementation. > > More documentation is good. > > >>> * Read and write sync operators may be combined in a single call, > >>> PRE and POST operators may not be. E.G., PREREAD|PREWRITE is > >>> allowed, PREREAD|POSTREAD is not. We should note that while > >>> read and write operations may be combined, on some platforms > >>> PREREAD|PREWRITE is needlessly expensive when only a read is > >>> being performed. > >> > >> > >> PREREAD|POSTREAD doesn't sound useful to me, but > >> why would it be explicitly forbidden? > >> > >> Would you also forbid POSTREAD|PREWRITE? > >> (For a buffer that has just completed a DMA read > >> and is going to be immediately used for a DMA write?) > > > > My thinking on forbidding PREREAD|POSTREAD is at least partly that it > > removes some temptation to do the wrong thing: treat the busdma API as > > if it were a general cpu cache manipulation library. > > I think it should be read as POSTREAD + PREREAD, not PREREAD + POSTREAD. > > > With the new definition of the sequences, a PREREAD|POSTREAD operation > > is nonsensical because it leaves no window during which the DMA hardware > > has access to memory; it is in effect a no-op. If you think in terms of > > implementation you might think "This would have to cause a cache > > invalidation." If you think in terms of API you should be thinking > > something more like "This marks out a time window during which the DMA > > hardware has safe access to that memory which has a duration of zero." > > Acutally, it would allow one to shift the DMA from one device to another, > if read the way I say above. > > > POSTREAD|PREWRITE is interesting. It is not a no-op in terms of the > > API. It closes the hardware access window that was opened by an earlier > > PREREAD, and it opens a new hardware access window with the PREWRITE. > > Whether it touches hardware or caches or anything in a given > > implementation isn't the point, in terms of the API it's the right thing > > to do for a pair of back to back DMA operations, without intervening CPU > > access, on the same memory. > > Right. This is useful in many cases. Consider a log structured device > that reads data into memory, and then writes it back to the tail of the log > as a defragging operation. > > > So PREREAD|POSTREAD and PREWRITE|POSTWRITE make no sense, but all other > > combos should be allowed. Maybe instead of allowing and forbidding > > specific combos, we should just advise that these two are effectively > > no-ops. > > I respectfully disagree.. > > Warner > > _______________________________________________ > freebsd-arm@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arm > To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Mon Aug 27 06:07:05 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D29CD106566C; Mon, 27 Aug 2012 06:07:05 +0000 (UTC) (envelope-from hans.petter.selasky@bitfrost.no) Received: from smtp02-out.isp.tdc.no (smtp02-out.isp.tdc.no [213.236.144.173]) by mx1.freebsd.org (Postfix) with ESMTP id 5C13D8FC0C; Mon, 27 Aug 2012 06:07:05 +0000 (UTC) Received: from mail.bitfrost.no (mail.bitfrost.no [85.19.79.136]) by smtp02-out.isp.tdc.no (Postfix) with ESMTP id 3X52cd1p2pz3CP; Mon, 27 Aug 2012 08:04:49 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at bitfrost.no From: =?windows-1252?Q?Hans_Petter_Selasky?= To: =?windows-1252?Q?Ian_Lepore?= , =?windows-1252?Q?Warner_Losh?= Date: Mon, 27 Aug 2012 08:06:57 +0200 Mime-Version: 1.0 In-Reply-To: <6D83AF9D-577B-4C83-84B7-C4E3B32695FC@bsdimp.com> References: <6D83AF9D-577B-4C83-84B7-C4E3B32695FC@bsdimp.com> X-Priority: 3 (Normal) Message-Id: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "=?windows-1252?Q?freebsd-arm=40freebsd.org?=" , =?windows-1252?Q?Adrian_Chadd?= , "=?windows-1252?Q?freebsd-mips=40freebsd.org?=" , "=?windows-1252?Q?freebsd-arch=40freebsd.org?=" Subject: RE: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2012 06:07:06 -0000 Hi,=20=0D=0ACorrect.=0D=0A=0D=0A> We also need some rules about working w= ith buffers obtained from=0D=0A> bus_dmamem_alloc() and external buffers = passed to bus_dmamap_load(). =A0I=0D=0A> think the rule should be that a = buffer obtained from bus_dmamem_alloc(),=0D=0A> or more formally any regi= on of memory mapped by a bus_dmamap_load(), is=0D=0A> a single logical ob= ject which can only be accessed by one entity at a=0D=0A> time. =A0That m= eans that there cannot be two concurrent DMA operations=0D=0A> happening = in different regions of the same buffer, nor can DMA and CPU=0D=0A> acces= s be happening concurrently even if in different parts of the=0D=0A> buff= er. =A0=0D=0A=0D=0A=0D=0AIs this something which we can fix using a simpl= e __align(USB_DMA_ALIGN) on elements in C-structures which are allowed to= be DMA loaded.=0D=0A=0D=0A=A0=0D=0A=0D=0AAlso: Why is busdma not complai= ning when loading a non-valid buffer pointer=3F=0D=0A=0D=0A--HPS=0D=0A From owner-freebsd-arch@FreeBSD.ORG Mon Aug 27 13:38:10 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 858DB1065674 for ; Mon, 27 Aug 2012 13:38:10 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 346FE8FC1E for ; Mon, 27 Aug 2012 13:38:10 +0000 (UTC) Received: by dadr6 with SMTP id r6so2521837dad.13 for ; Mon, 27 Aug 2012 06:38:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:x-priority :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to:x-mailer:x-gm-message-state; bh=sDQvljch4pOy5n8q84V4CzIi45qexsJyPkD23yBHMLw=; b=m6bC4w5TPQYzLtEIb5FiG8XdpGghJqmiP8bQqAjVEsimdbVwZ3hljb3asMBEfItX4T irVPiZ+WbgzmEpAyXvbsQykCXmFfoDUM1IyvG91t6PWC4Mayi5hmkc1yMBetkB/aMA76 SF99MO/CLWNB7zweb7YJwmUzD3v8GogrJBWwRLBi/+dA047l9+DfIOpXdKe8pxQ6vKJP Hd7NQjvXcoPv5iPPxG25r3Bq5v7dCfNDG5LQlJgzOTBr2LIUPFRlvz8VfW9YWxtDx+K+ Mauf21xRGVYG019AqyHzbNYP7jTaXWJAydBAC2Z5wNiBHJctC/6CmwpK7nKhc0JqO18o Qb9w== Received: by 10.66.74.37 with SMTP id q5mr30255409pav.29.1346074689751; Mon, 27 Aug 2012 06:38:09 -0700 (PDT) Received: from [10.0.0.63] (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id pj10sm14721644pbb.46.2012.08.27.06.38.07 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 27 Aug 2012 06:38:09 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh X-Priority: 3 (Normal) In-Reply-To: Date: Mon, 27 Aug 2012 07:38:06 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <6D83AF9D-577B-4C83-84B7-C4E3B32695FC@bsdimp.com> To: Hans Petter Selasky X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQn1ygn82aXjJftRqprYWVZalB0LhTRubkxjmCt6Rt7iqx3rKaGbJiFC6SwiAYY9RIhppVke Cc: Ian Lepore , freebsd-arm@freebsd.org, Adrian Chadd , freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2012 13:38:10 -0000 On Aug 27, 2012, at 12:06 AM, Hans Petter Selasky wrote: > Hi, > Correct. >=20 > > We also need some rules about working with buffers obtained from > > bus_dmamem_alloc() and external buffers passed to bus_dmamap_load(). = I > > think the rule should be that a buffer obtained from = bus_dmamem_alloc(), > > or more formally any region of memory mapped by a bus_dmamap_load(), = is > > a single logical object which can only be accessed by one entity at = a > > time. That means that there cannot be two concurrent DMA operations > > happening in different regions of the same buffer, nor can DMA and = CPU > > access be happening concurrently even if in different parts of the > > buffer. =20 >=20 > Is this something which we can fix using a simple = __align(USB_DMA_ALIGN) on elements in C-structures which are allowed to = be DMA loaded. No. I don't think so. the reason is that you can't define USB_DMA_ALGIN = to be a constant on MIPS, at least, or I think ARM because that's = determined at run time. > Also: Why is busdma not complaining when loading a non-valid buffer = pointer? Speed and code simplicity. Warner From owner-freebsd-arch@FreeBSD.ORG Mon Aug 27 14:12:24 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 50F7210658E5 for ; Mon, 27 Aug 2012 14:12:24 +0000 (UTC) (envelope-from bounce-4389-freebsd-arch=freebsd.org@mbmsend.com) Received: from mta045.mbmsend.com (mta045.mbmsend.com [66.85.161.9]) by mx1.freebsd.org (Postfix) with ESMTP id 1B7288FC49 for ; Mon, 27 Aug 2012 14:12:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=sendmail; d=mbmsend.com; h=Date:To:From:Reply-To:Subject:List-Unsubscribe:Message-ID:MIME-Version:Sender:Content-Type; i=killersportslive=3Daol.com@mbmsend.com; bh=+GTLggJD7UeweZwJ1TLWxDmg850=; b=j1E3bwzt4uINL20ySdFEN7ta4Y909veeNYVeRkLyiCnr5aOTw7K9razelB/ab+3b9c382b7GC+DO /AVvXbVgaA== DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; s=sendmail; d=mbmsend.com; b=MNMtVLG8Q0vBa0GI3FHx8Rtw9yynXi7XnxJ1q8mOeojUVwbesaKjq/Pdr457GGvAwLUYCeBpR4vK G2V0xB/rgw==; Received: from mbmsend.com (127.0.0.1) by mta045.mbmsend.com id h7e04819uisc for ; Mon, 27 Aug 2012 10:11:33 -0400 (envelope-from ) Date: Mon, 27 Aug 2012 10:11:20 -0400 To: "" From: "Chase Diamond" Message-ID: <370891932e0a0c58b192ade0f3a30ed7-4389@mbmsend.com> X-Mailer: mybizmailer v1.0 (MyBizMailer.com) MIME-Version: 1.0 X-Report-Abuse: Please report abuse for this campaign here: http://mybizmailer.com/abuse.aspx?aid=00a1i0h3&uid=8bd880e810a8d744ded4e0ab8ab0457c&e=370891932e0a0c58b192ade0f3a30ed7&b=1 Sender: "Chase Diamond" X-CampaignID: 370891932e0a0c58b192ade0f3a30ed7 X-AccountID: 00a1i0h3 X-FBL: 370891932e0a0c58b192ade0f3a30ed7 Content-Type: text/plain; charset="utf-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: $100.00 in free capper cash X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Chase Diamond List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2012 14:12:24 -0000 SIMPLY PUT GET 100.00 IN FREE HANDICAPPER CASH TO USE ON ANY HANDICAPPER AT LASVEGASLINECHANGERS.COM FOR FOOTBALL SEASON. GO TO WWW.LASVEGASLINECHANGERS.COM LOOK FOR A HANDICAPPER UD LIKE SIGHN UP FOR THERE FREE PICKS NEWSLETER SIMPLY BYE ENTERING UR EMAIL IN TOP LEFT OF WEBSITE AND (REPLY BACK TO THIS EMAIL SAYIN WHICH CAPPER UD LIKE UR FREE 100.00 PUT TOWARDS AND YOU WILL HAVE UR FREE TRIAL DONT WAIT FOOTBALL STARTS IN 3 DAYS!!!! BELOW IS A LIST OF HANDICAPPERS TO PICK FROM... Tony Corleone Vernon Croy Chase Diamond Chaz Diamond Matt Fargo Simon Green Tony Karpinski Ross King Steve Merril Stephen Nover TJ Pemberton RJ Robbins Carolina Sports Doc's Sports Kevin Thomas Craig Trapp Paul Wynns >>> Click here to read about our guarantee policy and loyalty program <<< TONY CORLEONE Tony's MLB Play of the Day!! Tony Corleone continues his great success with a strong 2nd half of the MLB season. He was 5-1 with his plays last night and is now 55-36 on the season with his MLB Plays of the Day. He has another one locked and loaded for today. Grab this guaranteed winner now and continue to cash in with Tony!! Tony's 10 Unit MLB Grand Slam!! (21-11 last 32) Tony continues to pick winners. He has been crushing the books all season long. He currently holds down the #1 MLB spot as well as the #1 overall spot. He has his 10 Unit MLB Grand Slam ready for today. Make sure to grab this winner now and absolutely shell your bookie with Tony!! Tony's 10 Unit Thursday NCAAF Enforcer!! Tony is locked and loaded with his 10 Unit Thursday NCAAF Enforcer between Vanderbilt and South Carolina. Last season Tony absolutely shelled the books in college football, as he managed to net his dime bettor cliets over $43,000. Grab this winner now and start the college football season off on a winning note with Tony!! Click here to see all of Tony Corleone's picks CHASE DIAMOND NCAFF KICK OFF 3 PACK 8/30 100/70/50 DIMER CHASE DIAMOND IS READY TO START THE NCAFF SEASON WITH A SWEEP AND HE HAS 3 WINNERS SET FOR THURSDAY 8/30 FOR JUST 29.99 YOU CAN CASH WITH YOUR BOOKS WITH EASE BE SHARP BUY CHASE DIAMONDS NCAFF PLAYS LAST SEASON HE CASHED OVER 64% IN NCAFF. 200 DIME NCAFF GAME OF THE WEEK 51-24 LIFFETIME THIS PLAY IS FOR 9/1 OPENING SATURDAY DONT MISS CHASE DIAMOND COLLEGE FOOTBALL GAME OF THE WEEK. CHASE DIAMOND HAS GOT HIS HANDS OF A GAME THAT WILL CASH WITH EASE. LAST YEAR CHASE DIAMOND CASHED OVER 60% ON HIS COLLEGE FOOTBALL PLAYS THIS YEAR CHASE LOOKS TO IMPROVE ON THAT NUMBER BE HERE OR BE SQUARE.(10.00 OFF NORMAL PRICE) 200 DIME NFL KICK OFF ULTIMATE WINNER 72% LIFETIME CHASE DIAMOND THE BIG PLAY KING HAS LOCKED AND LOADED ON HIS TAKE ON THE COWBOYS AT GIANTS AND YOU ALL KNOW HOW CHASE DOES WITH 200 DIMERS TRY 72% LIFETIME THIS GAME IS BACKED BY 4 SYSTEMS AND SHOULD BE HIT AND HIT HARD WIN BIG GAME 1 GUARANTEED. Click here to see all of Chase Diamond's picks CHAZ DIAMOND 50 DIME KILLER SPORTS KNOCK OUT WINNER CHAZ DIAMOND HAS BEEN VERY SELECTIVE LATELY AND HITTING HIS PLAYS DONT MISS THIS 50 DIME DOG WINNER TODAY AND FOR JUST 14.99 WHO CAN AFFORD NOT TO BE ON IT? Click here to see all of Chaz Diamond's picks MATT FARGO Fargo's MLB BIG BITE BEATDOWN (HUGE 59-35 LIFETIME Matt is coming off a Sunday split on the bases and brings in a POTENT 60% run in baseball into the new week! He comes back in a huge way on Monday with a Big Bite Beatdown and these reports have been OUTSTANDING, going a BLISTERING 59-35 (62.8%) lifetime in MLB! This large chalk UNLEASHES another MASSIVE DESTRUCTION! Hammer it! Do not even think about missing this! Fargo's 10* CFB THURSDAY TOTALS DOMINATOR (53-34) Matt is locked and loaded for a huge CFB year as he carries over what was an EXCEPTIONAL ending last year! He closed on a PERFECT 4-0 run and the numbers were OUTSTANDING going back further as he finished on a SCORCHING 53-34-1 (61%) run! He WINS Thursday with an opening night Totals Dominator that is a SHOOTOUT in the making! This one flies way over the total! Fargo's 10* CFB THURSDAY ENFORCER (53-34 CFB RUN) Matt closed the college football season on a PERFECT 4-0 run and the numbers were OUTSTANDING going back further as he finished on a SCORCHING 53-34-1 (61%) run! He gets things started right where he left off with a MASSIVE 10* Enforcer on opening night! Last year he went a COMMANDING 7-1 (87.5%) in Week One! More of the same! Do not even think about missing this! Click here to see all of Matt Fargo's picks STEVE MERRIL MLB Grand Slam - (PERFECT 5-0 MLB RUN)! Steve Merril has SWEPT his L5 MLB plays and he has uncovered a powerful MLB GRAND SLAM for Monday that is backed by solid winning angles - Grab this value play right now! Guaranteed Run Line that will WIN BIG! Click here to see all of Steve Merril's picks STEPHEN NOVER Stephen Nover's Underdog Play of the Month A wrong favorite is one of many factors making this matchup such a strong play that it rates as Vegas wiseguy Stephen Nover's August Underdog Play of the Month. Don't miss this sure winner and cash big. Read what makes this play so special. Click here to see all of Stephen Nover's picks KEVIN THOMAS KEVIN THURSDAY NITE KICKOFF THREE PACK OF COLLEGE FOOTBALL THIS THURSDAY NAILING OVER 62 PERCENT LIFETIME COLLEGE AND READY TO CASH Click here to see all of Kevin Thomas's picks From owner-freebsd-arch@FreeBSD.ORG Mon Aug 27 15:12:19 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id EDAAC1065690; Mon, 27 Aug 2012 15:12:18 +0000 (UTC) (envelope-from tim@kientzle.com) Received: from monday.kientzle.com (99-115-135-74.uvs.sntcca.sbcglobal.net [99.115.135.74]) by mx1.freebsd.org (Postfix) with ESMTP id BC06E8FC08; Mon, 27 Aug 2012 15:12:18 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.4/8.14.4) id q7RFC818091514; Mon, 27 Aug 2012 15:12:08 GMT (envelope-from tim@kientzle.com) Received: from [192.168.2.143] (CiscoE3000 [192.168.1.65]) by kientzle.com with SMTP id 8wvqwunvxbytgk5wqkfzrzju8s; Mon, 27 Aug 2012 15:12:08 +0000 (UTC) (envelope-from tim@kientzle.com) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=us-ascii From: Tim Kientzle X-Priority: 3 (Normal) In-Reply-To: Date: Mon, 27 Aug 2012 08:12:07 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <6D83AF9D-577B-4C83-84B7-C4E3B32695FC@bsdimp.com> To: Warner Losh X-Mailer: Apple Mail (2.1278) Cc: Ian Lepore , Adrian Chadd , Hans Petter Selasky , freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2012 15:12:19 -0000 On Aug 27, 2012, at 6:38 AM, Warner Losh wrote: >=20 > On Aug 27, 2012, at 12:06 AM, Hans Petter Selasky wrote: >=20 >> Hi, >> Correct. >>=20 >>> We also need some rules about working with buffers obtained from >>> bus_dmamem_alloc() and external buffers passed to bus_dmamap_load(). = I >>> think the rule should be that a buffer obtained from = bus_dmamem_alloc(), >>> or more formally any region of memory mapped by a bus_dmamap_load(), = is >>> a single logical object which can only be accessed by one entity at = a >>> time. That means that there cannot be two concurrent DMA operations >>> happening in different regions of the same buffer, nor can DMA and = CPU >>> access be happening concurrently even if in different parts of the >>> buffer. =20 >>=20 >> Is this something which we can fix using a simple = __align(USB_DMA_ALIGN) on elements in C-structures which are allowed to = be DMA loaded. >=20 > No. I don't think so. the reason is that you can't define = USB_DMA_ALGIN to be a constant on MIPS, at least, or I think ARM because = that's determined at run time. But don't mbuf structures do pretty much what Hans is suggesting? Why is mbuf okay? Tim From owner-freebsd-arch@FreeBSD.ORG Mon Aug 27 15:20:53 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B44911065673; Mon, 27 Aug 2012 15:20:53 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 7601A8FC15; Mon, 27 Aug 2012 15:20:53 +0000 (UTC) Received: by dadr6 with SMTP id r6so2589280dad.13 for ; Mon, 27 Aug 2012 08:20:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=lYrRM128qw04nZGkoy9WgB67sDTxSyD1vHe6xtBexvo=; b=ajYXEeB64Q8sgiJvPDcQkhcYLTpxX+4bxS1dk9blsyBAxhjVkMd5u5tSfiEPg7RmSB kc8Sq1JfWh/cI6jIinuo7tNbnfvkeC3SKBjbZXooJrfhgc9uBHpGQGHhpkO5lyCSkCQ2 i2zMmX8uspoG9YByjg9qYniJdsrthFCXhu7Da/k+7t4xFilPiFrwEIM8OhyYqW0NAxve 1z6Ug3KtaAIRG1Kx7fBG39DhO0ixteho3n4n+m1axU6wpj9gq30F7D+cW3RD9mM8F525 BQ+vOw8oW9uVElLkOygvPFmbcJqyX1TOotxbUMEBtAGyzQWZGau6A9Dy41T+0Husyp83 eoQg== MIME-Version: 1.0 Received: by 10.68.129.131 with SMTP id nw3mr35286569pbb.43.1346080853000; Mon, 27 Aug 2012 08:20:53 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.68.36.106 with HTTP; Mon, 27 Aug 2012 08:20:52 -0700 (PDT) In-Reply-To: References: <6D83AF9D-577B-4C83-84B7-C4E3B32695FC@bsdimp.com> Date: Mon, 27 Aug 2012 08:20:52 -0700 X-Google-Sender-Auth: ohyisdNkmhbkzwukiyHFrt829tI Message-ID: From: Adrian Chadd To: Tim Kientzle Content-Type: text/plain; charset=ISO-8859-1 Cc: Ian Lepore , Hans Petter Selasky , freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2012 15:20:53 -0000 On 27 August 2012 08:12, Tim Kientzle wrote: >> No. I don't think so. the reason is that you can't define USB_DMA_ALGIN to be a constant on MIPS, at least, or I think ARM because that's determined at run time. FYI, I wonder if MIPS may need some run time value for that. Right now we hide it behind needing a custom config file for each mips variant, rather than supporting one mips24k kernel for ${LOTS} of variants. > But don't mbuf structures do pretty much what Hans is suggesting? > > Why is mbuf okay? Which part of mbufs? I think the difference here is that there's no concurrent access. Ie, although you may have an mbuf header + data inside the same cache line, you wouldn't go fondling the mbuf once it's handed to the hardware. But I was under the impression that mbuf + mbuf buffers are already correctly aligned. This all honestly looks like a very i386-centric interpretation of the busdma "intention", which I think illustrates a need to better document what assumptions busdma actually makes. That does remind me, I think the ath(4) driver does the same (since it allocates its own descriptor block and then treats it as an array of descriptors for the hardware to access) - I should ensure that sizeof(ath_desc) is aligned on the relevant architecture. It gets slightly scary - AR93xx TX descriptors are "L1 cache == 128 byte aligned" which is an enormous waste of memory compared to a 16 or 32 byte aligned platform. Alas.. Adrian From owner-freebsd-arch@FreeBSD.ORG Mon Aug 27 15:33:24 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 748681065686; Mon, 27 Aug 2012 15:33:24 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id 8FCB48FC1E; Mon, 27 Aug 2012 15:33:08 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id q7RFX0vq088953; Mon, 27 Aug 2012 09:33:00 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id q7RFWbGs031228; Mon, 27 Aug 2012 09:32:37 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) From: Ian Lepore To: Warner Losh In-Reply-To: <10307B47-13F3-45C0-87F7-66FD3ACA3F86@bsdimp.com> References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> <1345766109.27688.606.camel@revolution.hippie.lan> <1346002922.1140.56.camel@revolution.hippie.lan> <1346005507.1140.69.camel@revolution.hippie.lan> <10307B47-13F3-45C0-87F7-66FD3ACA3F86@bsdimp.com> Content-Type: text/plain; charset="us-ascii" Date: Mon, 27 Aug 2012 09:32:37 -0600 Message-ID: <1346081557.1140.181.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: Mark Tinguely , Petter Selasky , freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org, Hans Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2012 15:33:24 -0000 On Sun, 2012-08-26 at 17:13 -0600, Warner Losh wrote: > On Aug 26, 2012, at 12:25 PM, Ian Lepore wrote: > > In this regard, it's the busdma implementation that's broken, because it > > should bounce those IOs through a DMA-safe buffer. There's absolutely > > no rule that I've ever heard of in FreeBSD that says IO can only take > > place using memory allocated from busdma. > > That's partially true. Since BUSDMA grew up in the storage area, you must allocate the memory from busdma, or it must be page aligned has been the de-facto rule here. Where does anything say that you must allocate the memory from busdma if it's not mbuf/page-aligned? I've never seen it in any docs. I certainly find contrary evidence in existing driver code (and I'm not talking about USB here). > The mbuf and uio variants of load were invented to cope with common cases of mbufs and user I/O to properly flag things. > What does that mean, "to properly flag things"? I think with uio we come to the crux of the issue. Userland buffers are not allocated with knowledge of the busdma constraints. That leads to some choices: * We don't support IO with a userland buffer at all. * We support userland IO if the userland buffers are accidentally aligned properly for the platform, otherwise the call fails. * We support userland buffers by making any driver which wants to do so responsible for always copying the data to aligned buffers (each individual driver handles bounces). * We support userland buffers by having the busdma layer handle the bouncing when required. The first two seem untenable to me. The third one comes down to "every driver must always bounce all userland data" because the driver doesn't actually have access to the info to decide whether a userland buffer is aligned properly or not for a given platform. That leaves the option where the busdma layer handles bouncing for unaligned userland buffers. If that's possible, then the busdma layer could automatically bounce any unaligned request, whether it came from userland or a driver reading a status response into an unaligned local buffer in kernel memory. > How does busdma know that it is using memory that's not from its allocator? The busdma code allocates the map along with the buffer, and can record information in the map that it can use during mapping and sync operations to know whether it allocated the buffer. > > The rule is only that the > > proper sequence of busdma operation must be called, and beyond that it's > > up to the busdma implementation to make it work. > > No. Bouncing is needed due to poor alignment of the underlying device. Not due to cache effects. Says who? This statement seems to me to be more about opinion and dogma than about a documented API or technical concerns about how to implement it. IMO, bouncing is needed when that's the only way to make a particular busdma sequence work correctly given the parameters of the hardware and the transfer. To put it another way, the busdma subsystem is responsible for helping make DMA transfers work on a platform without every driver needing to contain platform-specific code, and bouncing is one of the techniques available to it. > There's a limited number of things that we support with busdma. Arbitrary data from malloc that might be shared with the CPU isn't on that list. > Where are those limitations documented? This isn't some small oversight in the documention, if the rule is "IO, any IO at all, can only be performed using buffers allocated from busdma" that's a pretty major thing that ought to appear in multiple manpages relating to driver development. If you don't think "any IO at all" is the right characterization, read all the way to the end before responding. > > Our biggest problem, I think, is that we don't have a sufficient > > definition of "the proper sequence of busdma operations." > > I disagree. The sequence has been known for a long time. > > > I don't think it will be very hard to make the arm and mips busdma > > implementations work correctly. It won't even be too hard to make them > > fairly efficient at bouncing small IOs (my thinking is that we can make > > small bounces no more expensive than the current partial cacheline flush > > implementation which copies the data multiple times). Bouncing large IO > > will never be efficient, but the inefficiency will be a powerful > > motivator to update drivers that do large IO to work better, such as > > using buffers allocated from busdma. > > I don't think the cache line problem can be solved with bounce buffers. Trying to accommodate broken drivers is what lead us to this spot. We need to fix the broken drivers. If that's impossible, then the best we can do is have the driver set a 'always bounce' flag in the tag it creates and use that to always bounce for operations through that tag. So you'd be okay with a driver setting a flag that says "always bounce" but not okay with the busdma layer bouncing only when it's actually necessary? I'm confused -- do you think the busdma layer will be unable to detect when it's necessary unless directed from the outside? Let me pose a question back to you... if it is up to a driver to allocate DMA buffers using the busdma allocator, how is any given driver to know whether DMA is going to be involved in the transfer? Don't think USB or ATA here... think iicbus/foo_temperature_sensor, or the mmc/sd driver. Those drivers know nothing about the hardware bridge layers beneath them and whether DMA is involved or not. They just know "I need to read a 2-byte value from a register on this IIC chip" or "I need to retrieve the 32-bit CSD data from the SD card" and they accomplish that by calling a read method of some lower-level bridge driver. In each of those cases we have both PIO and DMA implementations of the lower-level drivers that move bits over a wire. If the concensus is that such drivers should always allocate all IO buffers using busdma, even though they have no idea whether DMA is going to be involved, then we certainly have a lot of work ahead of us in terms of "fixing broken drivers." That also implies that bus_dmamem_alloc() is misnamed and implemented in the wrong place, because it's not really about dma at all. If you push the responsibility down to the layer where it is known whether DMA is going to be involved, then we're back to the need for many individual drivers to bounce every request, because they don't have access to the info needed to know whether bouncing is required or not for a given buffer they were handed from higher layers. (Remember when I proposed exporting that info to drivers so they could decide whether a buffer is dma-aligned or not, the response was "No, keep it all hidden in the busdma layer," which I think is the right response.) Or if you push the responsibility down to the busdma layer where all that info resides, all drivers which use DMA and adhere to the busdma rules just work, without needing to know anything about the buffers that were handed to them and without needing to know anything about the platform requirements. -- Ian From owner-freebsd-arch@FreeBSD.ORG Mon Aug 27 15:34:35 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AF2451065678 for ; Mon, 27 Aug 2012 15:34:35 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 628028FC2E for ; Mon, 27 Aug 2012 15:34:34 +0000 (UTC) Received: by obbun3 with SMTP id un3so11030978obb.13 for ; Mon, 27 Aug 2012 08:34:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:x-priority :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to:x-mailer:x-gm-message-state; bh=sv9ysqQt0d8GIYk9LOiQWK2BMH1BDWf3NslmGnAmKbY=; b=gpfdGXR2a9IOYk3zUrKbKFOBKMEmDLccRVBVMrHJpDThtzCigQ4+4/NRgjcWixQfKG 0otavN1GYoaxOVrUVEKmZzKyJV9xF9U6qoAd0UqCY3tqBXa23SIjb40UFok1yHyMuhdR LtxaGMFDZwRwMcYcUvbaf3MUW11V1L1LnAgFfMe89NfM1jPFU/hdfmCmPY1EcdyQxsSM oc47HALDy0cK5WkHoZDX0ADv5iWZfOfPjiq4d7w2WThJD7NmAMhhl+AWZx50fTT90+4X D7DRRONZexUl8yATZOUo62v+a+45hX8Qf5bNuiYuuCzcf9kQzKKKEmqsFzbQ/Cd2pByZ dLeA== Received: by 10.182.190.69 with SMTP id go5mr10597608obc.43.1346081674507; Mon, 27 Aug 2012 08:34:34 -0700 (PDT) Received: from [10.30.101.53] ([209.117.142.2]) by mx.google.com with ESMTPS id cp8sm17348774obc.23.2012.08.27.08.34.33 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 27 Aug 2012 08:34:33 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh X-Priority: 3 (Normal) In-Reply-To: Date: Mon, 27 Aug 2012 09:34:30 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <45C204C3-D4CE-4B00-886A-A88A0FE7CAD7@bsdimp.com> References: <6D83AF9D-577B-4C83-84B7-C4E3B32695FC@bsdimp.com> To: Tim Kientzle X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQlr3ugsgTNsUML7DmyfT3TTCpL5ymwZ86Zb331oN6qSiN7Rs61LAlnFqi+J9F4T4U04ua/d Cc: Ian Lepore , Adrian Chadd , Hans Petter Selasky , freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2012 15:34:35 -0000 On Aug 27, 2012, at 9:12 AM, Tim Kientzle wrote: >=20 > On Aug 27, 2012, at 6:38 AM, Warner Losh wrote: >=20 >>=20 >> On Aug 27, 2012, at 12:06 AM, Hans Petter Selasky wrote: >>=20 >>> Hi, >>> Correct. >>>=20 >>>> We also need some rules about working with buffers obtained from >>>> bus_dmamem_alloc() and external buffers passed to = bus_dmamap_load(). I >>>> think the rule should be that a buffer obtained from = bus_dmamem_alloc(), >>>> or more formally any region of memory mapped by a = bus_dmamap_load(), is >>>> a single logical object which can only be accessed by one entity at = a >>>> time. That means that there cannot be two concurrent DMA = operations >>>> happening in different regions of the same buffer, nor can DMA and = CPU >>>> access be happening concurrently even if in different parts of the >>>> buffer. =20 >>>=20 >>> Is this something which we can fix using a simple = __align(USB_DMA_ALIGN) on elements in C-structures which are allowed to = be DMA loaded. >>=20 >> No. I don't think so. the reason is that you can't define = USB_DMA_ALGIN to be a constant on MIPS, at least, or I think ARM because = that's determined at run time. >=20 > But don't mbuf structures do pretty much what Hans is suggesting? They kinda do, and kinda don't. > Why is mbuf okay? mbuf is OK because it is never changed while the DMA is pending to the = buffers. That is, m_hdr and m_ext are invariant while the device owns = the memory. In addition, mbuf allocations are so large that no two mbufs = share the same cache line (although it looks like 256 might be too small = to avoid that on some MIPS processors). usb buffers do not obey these = same rules, otherwise none of the stuff we're talking about would = matter. Warner= From owner-freebsd-arch@FreeBSD.ORG Mon Aug 27 15:40:24 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A8056106567B for ; Mon, 27 Aug 2012 15:40:24 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 1C1368FC25 for ; Mon, 27 Aug 2012 15:40:23 +0000 (UTC) Received: by obbun3 with SMTP id un3so11051469obb.13 for ; Mon, 27 Aug 2012 08:40:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=t1YrUXxwc6T+C4ku+1nQRySvTX9kp0FRv2Gmd0Bg2co=; b=VoSnMpYWN5R4Hx6xcB1yo0aiA/DyU90OoK1JpYWOqQIcqJUERZ3hOjT1QWLp9Tm2a7 HXVL5AnY0CMMPokVdW/RFHCNKTM+EXEkSXputYqCO4+qU8Zk+PRK/JLgV3WCuHbvi1RO Y9u6IZa0GDYpLCKlS7IHXKz+pyga+AU+INncTDnOJ7EjTOV+7b95lBASeHrM0Ea5JC9E ZeQwbH9rD3XJ4RHX8oCKn132nmsgljenNtw8jMhbNWTn7Q1AR2z7DJQhba/027JLaOqt FMxGoAqBFEg9NQe1vQYpwMP9wewjQALMOBWZj2jsYBW0VMIw+AYYE2+bQqluDO0svVXP L3+w== Received: by 10.60.3.35 with SMTP id 3mr10500585oez.139.1346082023466; Mon, 27 Aug 2012 08:40:23 -0700 (PDT) Received: from [10.30.101.53] ([209.117.142.2]) by mx.google.com with ESMTPS id hz6sm17394331obb.1.2012.08.27.08.40.21 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 27 Aug 2012 08:40:21 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: Date: Mon, 27 Aug 2012 09:40:19 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <9642068B-3C66-42BD-8515-14F734B3FF89@bsdimp.com> References: <6D83AF9D-577B-4C83-84B7-C4E3B32695FC@bsdimp.com> To: Adrian Chadd X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQlRtUmVAQVPg/zYWBxlQTw5sebrGwW8xQ1l/JSXikhdsH3BLBSGYdFhQ+f0ZEnaPA47pyKq Cc: Ian Lepore , Tim Kientzle , Hans Petter Selasky , freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2012 15:40:24 -0000 On Aug 27, 2012, at 9:20 AM, Adrian Chadd wrote: > On 27 August 2012 08:12, Tim Kientzle wrote: >> But don't mbuf structures do pretty much what Hans is suggesting? >>=20 >> Why is mbuf okay? >=20 > Which part of mbufs? >=20 > I think the difference here is that there's no concurrent access. Ie, > although you may have an mbuf header + data inside the same cache > line, you wouldn't go fondling the mbuf once it's handed to the > hardware. >=20 > But I was under the impression that mbuf + mbuf buffers are already > correctly aligned. Not quite. mbufs are large enough and aligned enough so that no two = mbufs are in the same cache line. The data inside the mbuf on 32-bit = platforms is 24 bytes from the start of the mbuf, so the mbuf data and = mbuf header do share the same cache line, but as you say, nobody touches = any part of the mbuf will the hardware has it. Recall that it isn't = alignment of where the buffer starts that matters here, but rather = alignment to ensure that there's no sharing of cache lines between = structures. > This all honestly looks like a very i386-centric interpretation of the > busdma "intention", which I think illustrates a need to better > document what assumptions busdma actually makes. Yes. x86 lets you play fast and loose with many of these things. We've = tried to accommodate this for a long time, but that is lame. > That does remind me, I think the ath(4) driver does the same (since it > allocates its own descriptor block and then treats it as an array of > descriptors for the hardware to access) - I should ensure that > sizeof(ath_desc) is aligned on the relevant architecture. It gets > slightly scary - AR93xx TX descriptors are "L1 cache =3D=3D 128 byte > aligned" which is an enormous waste of memory compared to a 16 or 32 > byte aligned platform. Alas.. The problem is with cache line sharing, not necessarily with alignment. = If you are only ever using one of them at a time, or if you have perfect = hygiene, you can cope with this situation without undue waste. The = perfect hygiene might be hard sometimes. Warner From owner-freebsd-arch@FreeBSD.ORG Mon Aug 27 15:57:54 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 67DA51065670 for ; Mon, 27 Aug 2012 15:57:54 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 0C4B68FC0C for ; Mon, 27 Aug 2012 15:57:53 +0000 (UTC) Received: by obbun3 with SMTP id un3so11111198obb.13 for ; Mon, 27 Aug 2012 08:57:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=+6b5VLtRymOQqJ25JKslOXKH/1wAxmLCqLr7LE+kYFs=; b=aPxdsiLhfH/V+BStWDiSdhAvoxTxR3aZgtWurEBN5Hjbdb6F74cDolaGOFMChPK3f+ LZ21SHV+yGK5r9xCsB2Yg7KSwumV6CXz87/zFADWnFmbh8hKgNYaqJpAfK90m9Rk3e1J OI1rN3cpeltnFq/myAhZjpFWnHGwHv/l6wppFoyjbjtzoIiwsJr0DcIL0rXtN9joCveo rMHkg3kFVY/6OcIXsTqHwLmJ+5TfdxYQAibA6RtWgCG66WdAkq6JTzvMndmIFOoSQD7o o3fJL183lMhNG64jqO2Tyf40SJ9mJ2x4n228vSEJ8hBfDQXJkQx09JU2HH9H12et8BkD IgxQ== Received: by 10.60.19.34 with SMTP id b2mr10134983oee.41.1346083073414; Mon, 27 Aug 2012 08:57:53 -0700 (PDT) Received: from [10.30.101.53] ([209.117.142.2]) by mx.google.com with ESMTPS id ov5sm9250003obc.2.2012.08.27.08.57.51 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 27 Aug 2012 08:57:52 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <1346081557.1140.181.camel@revolution.hippie.lan> Date: Mon, 27 Aug 2012 09:57:48 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> <1345766109.27688.606.camel@revolution.hippie.lan> <1346002922.1140.56.camel@revolution.hippie.lan> <1346005507.1140.69.camel@revolution.hippie.lan> <10307B47-13F3-45C0-87F7-66FD3ACA3F86@bsdimp.com> <1346081557.1140.181.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQmcdy2X5V7iH3x8Q8xqWjwwr6DXF8bahK3BMs0qf/AHQijUp1Pf2hXltP5KMZLYZNQFRVQM Cc: freebsd-arm@freebsd.org, freebsd-arch@freebsd.org, Mark Tinguely , freebsd-mips@freebsd.org, Hans Petter Selasky Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2012 15:57:54 -0000 On Aug 27, 2012, at 9:32 AM, Ian Lepore wrote: > On Sun, 2012-08-26 at 17:13 -0600, Warner Losh wrote: >> On Aug 26, 2012, at 12:25 PM, Ian Lepore wrote: >>> In this regard, it's the busdma implementation that's broken, = because it >>> should bounce those IOs through a DMA-safe buffer. There's = absolutely >>> no rule that I've ever heard of in FreeBSD that says IO can only = take >>> place using memory allocated from busdma. >>=20 >> That's partially true. Since BUSDMA grew up in the storage area, you = must allocate the memory from busdma, or it must be page aligned has = been the de-facto rule here. =20 >=20 > Where does anything say that you must allocate the memory from busdma = if > it's not mbuf/page-aligned? I've never seen it in any docs. I > certainly find contrary evidence in existing driver code (and I'm not > talking about USB here). Where does it say that you are allowed to not use the routines? >> The mbuf and uio variants of load were invented to cope with common = cases of mbufs and user I/O to properly flag things. >>=20 >=20 > What does that mean, "to properly flag things"? >=20 > I think with uio we come to the crux of the issue. Userland buffers = are > not allocated with knowledge of the busdma constraints. That leads to > some choices: >=20 > * We don't support IO with a userland buffer at all. You may have to bounce here. > * We support userland IO if the userland buffers are accidentally > aligned properly for the platform, otherwise the call fails. You may have to bounce here. > * We support userland buffers by making any driver which wants to > do so responsible for always copying the data to aligned = buffers > (each individual driver handles bounces). This is what most drivers that want to do zero copying do. > * We support userland buffers by having the busdma layer handle > the bouncing when required. I thought that's what the uio variants of the map loading routines were = supposed to do. > The first two seem untenable to me. The third one comes down to = "every > driver must always bounce all userland data" because the driver = doesn't > actually have access to the info to decide whether a userland buffer = is > aligned properly or not for a given platform. >=20 > That leaves the option where the busdma layer handles bouncing for > unaligned userland buffers. If that's possible, then the busdma layer > could automatically bounce any unaligned request, whether it came from > userland or a driver reading a status response into an unaligned local > buffer in kernel memory. Right, the uio load option is supposed to do this. >> How does busdma know that it is using memory that's not from its = allocator? >=20 > The busdma code allocates the map along with the buffer, and can = record > information in the map that it can use during mapping and sync > operations to know whether it allocated the buffer. >=20 >>> The rule is only that the >>> proper sequence of busdma operation must be called, and beyond that = it's >>> up to the busdma implementation to make it work. =20 >>=20 >> No. Bouncing is needed due to poor alignment of the underlying = device. Not due to cache effects. >=20 > Says who? This statement seems to me to be more about opinion and = dogma > than about a documented API or technical concerns about how to = implement > it. IMO, bouncing is needed when that's the only way to make a > particular busdma sequence work correctly given the parameters of the > hardware and the transfer. Says the people that wrote and use busdma? There's no requirement for = cache effects for the transfer, so there's no constraints for the = transfer. The requirement is about having a buffer that's suitable for = DMA. This means either the buffer (not the start of the transfer) be = aligned to a cache line and be an integral number of cache lines in = size, or it means that you have 100% perfect hygiene when it comes to = ensuring the CPU touches the buffer or the device touches it and ever = have cross threading. I'd agree that better docs here would help. > To put it another way, the busdma subsystem is responsible for helping > make DMA transfers work on a platform without every driver needing to > contain platform-specific code, and bouncing is one of the techniques > available to it. Buffers allocated through the busdma system do this. >> There's a limited number of things that we support with busdma. = Arbitrary data from malloc that might be shared with the CPU isn't on = that list. >>=20 >=20 > Where are those limitations documented? =20 Where is it documented that memory returned from malloc may be used for = DMA? I agree that docs could be better here. > This isn't some small oversight in the documention, if the rule is = "IO, > any IO at all, can only be performed using buffers allocated from > busdma" that's a pretty major thing that ought to appear in multiple > manpages relating to driver development. >=20 > If you don't think "any IO at all" is the right characterization, read > all the way to the end before responding. Yes. I'd state it another way. "Buffers returned from the busdma = allocation routines is always safe." Everything else may or may not be = safe. mbufs, for example, happen to be safe because there is a rigid = protocol for sharing between the driver and the device. uio buffers can = be made safe. busdma has routines to ensure that these types of data = are handled properly. >>> Our biggest problem, I think, is that we don't have a sufficient >>> definition of "the proper sequence of busdma operations." >>=20 >> I disagree. The sequence has been known for a long time. >>=20 >>> I don't think it will be very hard to make the arm and mips busdma >>> implementations work correctly. It won't even be too hard to make = them >>> fairly efficient at bouncing small IOs (my thinking is that we can = make >>> small bounces no more expensive than the current partial cacheline = flush >>> implementation which copies the data multiple times). Bouncing = large IO >>> will never be efficient, but the inefficiency will be a powerful >>> motivator to update drivers that do large IO to work better, such as >>> using buffers allocated from busdma. >>=20 >> I don't think the cache line problem can be solved with bounce = buffers. Trying to accommodate broken drivers is what lead us to this = spot. We need to fix the broken drivers. If that's impossible, then = the best we can do is have the driver set a 'always bounce' flag in the = tag it creates and use that to always bounce for operations through that = tag. >=20 > So you'd be okay with a driver setting a flag that says "always = bounce" > but not okay with the busdma layer bouncing only when it's actually > necessary? I'm confused -- do you think the busdma layer will be = unable > to detect when it's necessary unless directed from the outside? Actually, it is good you are confused. I was wrong when I said I'd be = happy with a flag that says always bounce. The reason is that the driver = can do this all the time for those cases like the at91 mci driver does. > Let me pose a question back to you... if it is up to a driver to > allocate DMA buffers using the busdma allocator, how is any given = driver > to know whether DMA is going to be involved in the transfer? =20 because it does the dma? > Don't think USB or ATA here... think iicbus/foo_temperature_sensor, or > the mmc/sd driver. Those drivers know nothing about the hardware = bridge > layers beneath them and whether DMA is involved or not. They just = know > "I need to read a 2-byte value from a register on this IIC chip" or "I > need to retrieve the 32-bit CSD data from the SD card" and they > accomplish that by calling a read method of some lower-level bridge > driver. In each of those cases we have both PIO and DMA = implementations > of the lower-level drivers that move bits over a wire. The bridge here would know and have to cope. However, the real issue in = this case wouldn't be the transfer alignment, but what traffic might be = happening to other parts of the cache line. For small things like this, typically data copies are involved. There's = a few places that break these rules (I think mci might be breaking the = rules for small transfers, for example). > If the concensus is that such drivers should always allocate all IO > buffers using busdma, even though they have no idea whether DMA is = going > to be involved, then we certainly have a lot of work ahead of us in > terms of "fixing broken drivers." That also implies that > bus_dmamem_alloc() is misnamed and implemented in the wrong place, > because it's not really about dma at all. You can't get around the hardware requirement that any I/O going to the = device cannot share cache lines with other data on the system. > If you push the responsibility down to the layer where it is known > whether DMA is going to be involved, then we're back to the need for > many individual drivers to bounce every request, because they don't = have > access to the info needed to know whether bouncing is required or not > for a given buffer they were handed from higher layers. (Remember = when > I proposed exporting that info to drivers so they could decide whether = a > buffer is dma-aligned or not, the response was "No, keep it all hidden > in the busdma layer," which I think is the right response.) Right now we support only a few things doing DMA on the system. We = support pages, mbufs and to a limited extent uio buffers (but to be = honest, we may have exposure on smaller transfers here). > Or if you push the responsibility down to the busdma layer where all > that info resides, all drivers which use DMA and adhere to the busdma > rules just work, without needing to know anything about the buffers = that > were handed to them and without needing to know anything about the > platform requirements. I'm not sure I follow this last bit... Warner From owner-freebsd-arch@FreeBSD.ORG Mon Aug 27 16:09:12 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1AA23106566C; Mon, 27 Aug 2012 16:09:12 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id D96788FC15; Mon, 27 Aug 2012 16:08:59 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id q7RG8xAG089488; Mon, 27 Aug 2012 10:08:59 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id q7RG8aPI031283; Mon, 27 Aug 2012 10:08:36 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) From: Ian Lepore To: Warner Losh In-Reply-To: <6D83AF9D-577B-4C83-84B7-C4E3B32695FC@bsdimp.com> References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> <1345766109.27688.606.camel@revolution.hippie.lan> <1346002922.1140.56.camel@revolution.hippie.lan> <6D83AF9D-577B-4C83-84B7-C4E3B32695FC@bsdimp.com> Content-Type: text/plain; charset="us-ascii" Date: Mon, 27 Aug 2012 10:08:36 -0600 Message-ID: <1346083716.1140.212.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: Adrian Chadd , freebsd-arch@freebsd.org, freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, Hans Petter Selasky Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2012 16:09:12 -0000 On Sun, 2012-08-26 at 17:03 -0600, Warner Losh wrote: > On Aug 26, 2012, at 11:42 AM, Ian Lepore wrote: > > > > The busdma manpage currently has some vague words about the usage and > > sequencing of sync ops, such as "If read and write operations are not > > preceded and followed by the appropriate synchronization operations, > > behavior is undefined." I think we should more explicitly spell out > > what the appropriate sequences are. In particular: > > > > * The PRE and POST operations must occur in pairs; a PREREAD must > > be followed eventually by a POSTREAD and a PREWRITE must be > > followed by a POSTWRITE. > > PREREAD means "I am about to tell the device to put data here, have whaterver things might be pending in the CPU complex to get out of the way." usually this means 'invalidate the cache for that range', but not always. POSTREAD means 'The device's DMA is done, I'd like to start accessing it now.' If the memory will be thrown away without being looked at, then does the driver necessarily need to issue the POSTREAD? I think so, but I don't know if that's a new requirement. > One of the things that scares me most is the idea that driver writers will glance at an existing implementation and think "Oh I have no need to ever call POSTWRITE because it's implemented as a no-op and I can save the call overhead." In fact we have drivers coded like that now. We've got an API here to support arbitrary hardware, some of which may not have been designed yet. I think it's really unsafe to say that a driver can decide that it's safe to elide some calls in some situations just because that's safe with the busdma implementations that exist today. > > We also need some rules about working with buffers obtained from > > bus_dmamem_alloc() and external buffers passed to bus_dmamap_load(). I > > think the rule should be that a buffer obtained from bus_dmamem_alloc(), > > or more formally any region of memory mapped by a bus_dmamap_load(), is > > a single logical object which can only be accessed by one entity at a > > time. That means that there cannot be two concurrent DMA operations > > happening in different regions of the same buffer, nor can DMA and CPU > > access be happening concurrently even if in different parts of the > > buffer. > > There's something subtle that I'm missing. Why would two DMA operations be disallowed? The rest makes good sense. > If two DMAs are going on concurrently in the same buffer, one is going to finish before the other, leading to a POSTxxxx sync op happening for one DMA operation while the other is still in progress. The unit of granularity for sync operations is the mapped region, so now you're syncing access to a region which still has active DMA happening within it. While I think it's really an API definition issue, think about it in terms of a potential implementation... What if the CPU had to access the memory as part of the sync for the first DMA that completes, while the second is still running? Now you've got pretty much exactly the same situation as when a driver subdivides a buffer without knowing about the cache alignment; you end up with the CPU and DMA touching data in the same cachline and no sequence of flush/invalidate can be g'teed to preserve all data correctly. > > I've always thought that allocating a dma buffer feels like a big > > hassle. You sometimes have to create a tag for the sole purpose of > > setting the maxsize to get the buffer size you need when you call > > bus_dmamem_alloc(). If bus_dmamem_alloc() took a size parm you could > > just use your parent tag, or a generic tag appropriate to all the IO > > you're doing for a given device. If you need a variety of buffers for > > small control and command and status transfers of different sizes, you > > end up having to manage up to a dozen tags and maps and buffers. It's > > all very clunky and inconvenient. It's just the sort of thing that > > makes you want to allocate a big buffer and subdivide it. Surely we > > could do something to make it easier? > > You'd wind up creating a quick tag on the fly for the bus_dmamap_alloc if you wanted to do this. Cleanup then becomes unclear. > My point is that the only piece of information in the tag that's specific to the allocation is the maxsize. If the allocation size were passed to bus_dmamem_alloc() then you wouldn't need a tag specific to that buffer, a generic tag for the device would work and you could allocate a dozen different-sized buffers all using that one tag, and the allocator would just have to sanity-check the allocation size against the tag's maxsize. -- Ian From owner-freebsd-arch@FreeBSD.ORG Mon Aug 27 16:53:31 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9B291106566B; Mon, 27 Aug 2012 16:53:31 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 2CDD38FC15; Mon, 27 Aug 2012 16:53:31 +0000 (UTC) Received: by obbun3 with SMTP id un3so11293602obb.13 for ; Mon, 27 Aug 2012 09:53:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=gJHpHTLGok1mdDMngy2k8DW11dnWPLHSjDIlGfmoy5o=; b=ezk1B4AI/J8/l3RTz+S6U95QsnA6aZcxymJbdAgyM5BlQwkdg/RMBmt4Dgjwh2l70q z/1AXux2Ac7RLCGrdDiGhpsbFyAXLMgynAst11zcozzCEGe2BmW/T+8NgIQ1VIjc8x76 cB8Cqif38GOFiNDnLq5VZIpqzgLUJf7Q9RhpESC5G9idEgjqMRxOUt7ZT7n7bBJUuwna dO6mfIgIAo6YJzw2pa7mLjUDpbH+X4sCC4+wqByxReoiWrIVX+bmntHNLgTRI72lwwnr Uf/WlLRWeWQj6Ri5/BINCkEi2RKCcBJGA0lnFjgFj1ngZxTbdwJXxuFy890xokaJcrBf 3CFQ== MIME-Version: 1.0 Received: by 10.182.0.13 with SMTP id 13mr10283841oba.55.1346086410479; Mon, 27 Aug 2012 09:53:30 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.76.8.98 with HTTP; Mon, 27 Aug 2012 09:53:29 -0700 (PDT) In-Reply-To: <1346083716.1140.212.camel@revolution.hippie.lan> References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> <1345766109.27688.606.camel@revolution.hippie.lan> <1346002922.1140.56.camel@revolution.hippie.lan> <6D83AF9D-577B-4C83-84B7-C4E3B32695FC@bsdimp.com> <1346083716.1140.212.camel@revolution.hippie.lan> Date: Mon, 27 Aug 2012 09:53:29 -0700 X-Google-Sender-Auth: _mn46Ck_2vyos8KJFH_aDJV2o8M Message-ID: From: Adrian Chadd To: Ian Lepore Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-arm@freebsd.org, freebsd-arch@freebsd.org, freebsd-mips@freebsd.org, Hans Petter Selasky Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2012 16:53:31 -0000 On 27 August 2012 09:08, Ian Lepore wrote: > If two DMAs are going on concurrently in the same buffer, one is going > to finish before the other, leading to a POSTxxxx sync op happening for > one DMA operation while the other is still in progress. The unit of > granularity for sync operations is the mapped region, so now you're > syncing access to a region which still has active DMA happening within > it. Right. But the enforced idea is "DMA up to this point should be flushed to memory." > While I think it's really an API definition issue, think about it in > terms of a potential implementation... What if the CPU had to access the > memory as part of the sync for the first DMA that completes, while the > second is still running? Now you've got pretty much exactly the same > situation as when a driver subdivides a buffer without knowing about the > cache alignment; you end up with the CPU and DMA touching data in the > same cachline and no sequence of flush/invalidate can be g'teed to > preserve all data correctly. Right. So you realise at that point you can't win and you stick each of those pieces in a different cache line. Adrian From owner-freebsd-arch@FreeBSD.ORG Mon Aug 27 18:54:02 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 090AE106564A for ; Mon, 27 Aug 2012 18:54:02 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 796238FC08 for ; Mon, 27 Aug 2012 18:54:00 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q7RIs0nK023175; Mon, 27 Aug 2012 21:54:00 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q7RIrlGG059144; Mon, 27 Aug 2012 21:53:47 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q7RIrked059143; Mon, 27 Aug 2012 21:53:46 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 27 Aug 2012 21:53:46 +0300 From: Konstantin Belousov To: Warner Losh Message-ID: <20120827185346.GE33100@deviant.kiev.zoral.com.ua> References: <1345765503.27688.602.camel@revolution.hippie.lan> <1345766109.27688.606.camel@revolution.hippie.lan> <1346002922.1140.56.camel@revolution.hippie.lan> <1346005507.1140.69.camel@revolution.hippie.lan> <10307B47-13F3-45C0-87F7-66FD3ACA3F86@bsdimp.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="BwgxZU5iS1dWSTZh" Content-Disposition: inline In-Reply-To: <10307B47-13F3-45C0-87F7-66FD3ACA3F86@bsdimp.com> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Ian Lepore , Mark Tinguely , Hans Petter Selasky , freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2012 18:54:02 -0000 --BwgxZU5iS1dWSTZh Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Aug 26, 2012 at 05:13:31PM -0600, Warner Losh wrote: >=20 > On Aug 26, 2012, at 12:25 PM, Ian Lepore wrote: > > In this regard, it's the busdma implementation that's broken, because it > > should bounce those IOs through a DMA-safe buffer. There's absolutely > > no rule that I've ever heard of in FreeBSD that says IO can only take > > place using memory allocated from busdma. >=20 > That's partially true. Since BUSDMA grew up in the storage area, you > must allocate the memory from busdma, or it must be page aligned has > been the de-facto rule here. The mbuf and uio variants of load were > invented to cope with common cases of mbufs and user I/O to properly > flag things. I once looked at x86 bus_dmamap_load_uio(), and I was unable to understand how to use it with usermode uio. I think this is a good moment to ask. Most existing users use UIO_SYSSPACE, but several crypto drivers might allow the UIO_USERSPACE for them. For UIO_USERSPACE, if the page is not resident, the pmap_extract() call from _bus_dmamap_load_buffer() returns 0. So the i/o happens to the page located at 0, which contains real mode IVT and other BIOS sensitive tables. Worse, if the page is resident, but it is mapped at the region which requires COW on write, then DMA will be performed to the wrong page which is typically shared with other innocent users. to the COW area which was not yet copied, Am I missing some trick there ? --BwgxZU5iS1dWSTZh Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlA7wjoACgkQC3+MBN1Mb4hJsACg9rY5d4jEjVBz1gmM0pcHvbwY 1N0AoKmBOQ6HFrErCDyO1ME2rbSRZLt/ =iNrn -----END PGP SIGNATURE----- --BwgxZU5iS1dWSTZh-- From owner-freebsd-arch@FreeBSD.ORG Mon Aug 27 19:01:31 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 89DBC106564A; Mon, 27 Aug 2012 19:01:31 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id 1F8C98FC08; Mon, 27 Aug 2012 19:01:19 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id q7RJ1BMt091960; Mon, 27 Aug 2012 13:01:17 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id q7RJ0mgZ031745; Mon, 27 Aug 2012 13:00:48 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) From: Ian Lepore To: Adrian Chadd In-Reply-To: References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> <1345766109.27688.606.camel@revolution.hippie.lan> <1346002922.1140.56.camel@revolution.hippie.lan> <6D83AF9D-577B-4C83-84B7-C4E3B32695FC@bsdimp.com> <1346083716.1140.212.camel@revolution.hippie.lan> Content-Type: text/plain; charset="us-ascii" Date: Mon, 27 Aug 2012 13:00:47 -0600 Message-ID: <1346094047.1140.264.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-arm@freebsd.org, freebsd-arch@freebsd.org, freebsd-mips@freebsd.org, Hans Petter Selasky Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2012 19:01:31 -0000 On Mon, 2012-08-27 at 09:53 -0700, Adrian Chadd wrote: > On 27 August 2012 09:08, Ian Lepore wrote: > > > If two DMAs are going on concurrently in the same buffer, one is going > > to finish before the other, leading to a POSTxxxx sync op happening for > > one DMA operation while the other is still in progress. The unit of > > granularity for sync operations is the mapped region, so now you're > > syncing access to a region which still has active DMA happening within > > it. > > Right. But the enforced idea is "DMA up to this point should be > flushed to memory." > > > While I think it's really an API definition issue, think about it in > > terms of a potential implementation... What if the CPU had to access the > > memory as part of the sync for the first DMA that completes, while the > > second is still running? Now you've got pretty much exactly the same > > situation as when a driver subdivides a buffer without knowing about the > > cache alignment; you end up with the CPU and DMA touching data in the > > same cachline and no sequence of flush/invalidate can be g'teed to > > preserve all data correctly. > > Right. So you realise at that point you can't win and you stick each > of those pieces in a different cache line. > Actually, I think that even discussing cache lines in this context is a mistake (yeah, I'm the one who did so above, in trying to relate an abstract API design concept to a real-world hardware example). Drivers are not supposed to know about interactions between DMA transfers and cache lines or other machine-specific constraints; that info is supposed to be encapsulated and hidden within busdma. I think a driver making the assumption that it can do DMA safely on a buffer as long as that buffer is cacheline-granular is just as flawed as assuming that it can do DMA safely on any arbitrarily sized and aligned buffer. So the right way to "stick each of those pieces in a different cache line" is to allocate two different buffers, one per concurrent DMA transfer. Or, really, to use two separate busdma mappings would be the more rigorous way to say it, since the mapping is the operation at which constraints come into play. Thinking of it that way then drives the need to document that if multiple mappings describe the same area of physical memory, then concurrent operations on those maps yield unpredictable results. -- Ian From owner-freebsd-arch@FreeBSD.ORG Mon Aug 27 19:18:54 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5EE64106566C for ; Mon, 27 Aug 2012 19:18:54 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 0908B8FC1D for ; Mon, 27 Aug 2012 19:18:53 +0000 (UTC) Received: by obbun3 with SMTP id un3so11731248obb.13 for ; Mon, 27 Aug 2012 12:18:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=NWAxBZraqJ39VsS1Q2psNADkTl2YA8RO4lXjmSt0qtk=; b=P3PwEz/JCwbjmdX8Usz5oE+2t/X3fcH8B3cu2rGVG11Fvum6aqf4QHh6npg0cltMcp sNZykF8GPOLIy9KcWUctRGy5XzbDIvM/RmQ9f90RZ7/noxvkUou1yKX5tXL7QwJGd064 EJronsoMwavcMtdO6vJwSV5Ad+leJse/tuJip73GsIHlZCIOs5R0ucj4psWPNbqdM4e8 Hy/ehmC+JHZUTnLbvgX+nXfIlR4B51XM/5uv3WG93xi7nb+4VGua+6VHf6/JNSKN0TV7 btupHdFWg7si7ynFVVquKoG7PZmShf6qtTGMK0gqH3Pf2C//9mdBS0SCcJwnawj24u7z Kp0w== Received: by 10.182.182.71 with SMTP id ec7mr8049310obc.22.1346095133419; Mon, 27 Aug 2012 12:18:53 -0700 (PDT) Received: from [10.30.101.53] ([209.117.142.2]) by mx.google.com with ESMTPS id r1sm13110813oea.4.2012.08.27.12.18.51 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 27 Aug 2012 12:18:52 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <1346094047.1140.264.camel@revolution.hippie.lan> Date: Mon, 27 Aug 2012 13:18:49 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> <1345766109.27688.606.camel@revolution.hippie.lan> <1346002922.1140.56.camel@revolution.hippie.lan> <6D83AF9D-577B-4C83-84B7-C4E3B32695FC@bsdimp.com> <1346083716.1140.212.camel@revolution.hippie.lan> <1346094047.1140.264.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQluF1C/zouvN9nGOjrjoM50tO3IXtroI8A7XXNQEpXbHwHBOmfk8Wyr6m7CqIdcvLpGv7Sp Cc: Adrian Chadd , freebsd-arch@freebsd.org, freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, Hans Petter Selasky Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2012 19:18:54 -0000 On Aug 27, 2012, at 1:00 PM, Ian Lepore wrote: > On Mon, 2012-08-27 at 09:53 -0700, Adrian Chadd wrote: >> On 27 August 2012 09:08, Ian Lepore = wrote: >>=20 >>> If two DMAs are going on concurrently in the same buffer, one is = going >>> to finish before the other, leading to a POSTxxxx sync op happening = for >>> one DMA operation while the other is still in progress. The unit of >>> granularity for sync operations is the mapped region, so now you're >>> syncing access to a region which still has active DMA happening = within >>> it. >>=20 >> Right. But the enforced idea is "DMA up to this point should be >> flushed to memory." >>=20 >>> While I think it's really an API definition issue, think about it in >>> terms of a potential implementation... What if the CPU had to access = the >>> memory as part of the sync for the first DMA that completes, while = the >>> second is still running? Now you've got pretty much exactly the = same >>> situation as when a driver subdivides a buffer without knowing about = the >>> cache alignment; you end up with the CPU and DMA touching data in = the >>> same cachline and no sequence of flush/invalidate can be g'teed to >>> preserve all data correctly. >>=20 >> Right. So you realise at that point you can't win and you stick each >> of those pieces in a different cache line. >>=20 >=20 > Actually, I think that even discussing cache lines in this context is = a > mistake (yeah, I'm the one who did so above, in trying to relate an > abstract API design concept to a real-world hardware example). >=20 > Drivers are not supposed to know about interactions between DMA > transfers and cache lines or other machine-specific constraints; that > info is supposed to be encapsulated and hidden within busdma. I think = a > driver making the assumption that it can do DMA safely on a buffer as > long as that buffer is cacheline-granular is just as flawed as = assuming > that it can do DMA safely on any arbitrarily sized and aligned buffer. >=20 > So the right way to "stick each of those pieces in a different cache > line" is to allocate two different buffers, one per concurrent DMA > transfer. Or, really, to use two separate busdma mappings would be = the > more rigorous way to say it, since the mapping is the operation at = which > constraints come into play. Thinking of it that way then drives the > need to document that if multiple mappings describe the same area of > physical memory, then concurrent operations on those maps yield > unpredictable results. Despite what I said earlier, I think this is sane. busdma only should = support one DMA active at a time into a buffer. If the driver wants to = do two different ones, they are on their own. Warner= From owner-freebsd-arch@FreeBSD.ORG Mon Aug 27 19:31:53 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0A4EE106568A; Mon, 27 Aug 2012 19:31:53 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id CDF9E8FC15; Mon, 27 Aug 2012 19:31:52 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 0C5F3B983; Mon, 27 Aug 2012 15:31:52 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Date: Mon, 27 Aug 2012 15:31:42 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p17; KDE/4.5.5; amd64; ; ) References: <10307B47-13F3-45C0-87F7-66FD3ACA3F86@bsdimp.com> <20120827185346.GE33100@deviant.kiev.zoral.com.ua> In-Reply-To: <20120827185346.GE33100@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201208271531.42725.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 27 Aug 2012 15:31:52 -0400 (EDT) Cc: Ian Lepore , Mark Tinguely , Hans Petter Selasky , freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, Konstantin Belousov Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2012 19:31:53 -0000 On Monday, August 27, 2012 2:53:46 pm Konstantin Belousov wrote: > On Sun, Aug 26, 2012 at 05:13:31PM -0600, Warner Losh wrote: > > > > On Aug 26, 2012, at 12:25 PM, Ian Lepore wrote: > > > In this regard, it's the busdma implementation that's broken, because it > > > should bounce those IOs through a DMA-safe buffer. There's absolutely > > > no rule that I've ever heard of in FreeBSD that says IO can only take > > > place using memory allocated from busdma. > > > > That's partially true. Since BUSDMA grew up in the storage area, you > > must allocate the memory from busdma, or it must be page aligned has > > been the de-facto rule here. The mbuf and uio variants of load were > > invented to cope with common cases of mbufs and user I/O to properly > > flag things. > > I once looked at x86 bus_dmamap_load_uio(), and I was unable to > understand how to use it with usermode uio. I think this is a good > moment to ask. Most existing users use UIO_SYSSPACE, but several crypto > drivers might allow the UIO_USERSPACE for them. > > For UIO_USERSPACE, if the page is not resident, the pmap_extract() call from > _bus_dmamap_load_buffer() returns 0. So the i/o happens to the page > located at 0, which contains real mode IVT and other BIOS sensitive tables. > > Worse, if the page is resident, but it is mapped at the region which > requires COW on write, then DMA will be performed to the wrong page > which is typically shared with other innocent users. to the COW area > which was not yet copied, > > Am I missing some trick there ? No. The caller is required to wire the pages first in some manner. In general bus_dmamap_load_uio() isn't a good idea. I do believe the crypto drivers are careful to wire the buffer first. I think requiring the caller to wire is the only sane way that can be used. Also, doing DMA to a stack variable is absolutely horrible for a related reason since presumably the thread will block while it waits for the DMA to complete, and a sleeping thread can be swapped out (including having it's stack swapped out). -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon Aug 27 22:08:17 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E93E4106566C; Mon, 27 Aug 2012 22:08:16 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id 6F7F08FC0A; Mon, 27 Aug 2012 22:08:04 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id q7RM7q0C094077; Mon, 27 Aug 2012 16:07:59 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id q7RM7Uu6031952; Mon, 27 Aug 2012 16:07:30 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) From: Ian Lepore To: Warner Losh In-Reply-To: <9642068B-3C66-42BD-8515-14F734B3FF89@bsdimp.com> References: <6D83AF9D-577B-4C83-84B7-C4E3B32695FC@bsdimp.com> <9642068B-3C66-42BD-8515-14F734B3FF89@bsdimp.com> Content-Type: text/plain; charset="us-ascii" Date: Mon, 27 Aug 2012 16:07:30 -0600 Message-ID: <1346105250.1140.314.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: Tim Kientzle , Adrian Chadd , Hans Petter Selasky , freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2012 22:08:17 -0000 On Mon, 2012-08-27 at 09:40 -0600, Warner Losh wrote: > On Aug 27, 2012, at 9:20 AM, Adrian Chadd wrote: > > That does remind me, I think the ath(4) driver does the same (since it > > allocates its own descriptor block and then treats it as an array of > > descriptors for the hardware to access) - I should ensure that > > sizeof(ath_desc) is aligned on the relevant architecture. It gets > > slightly scary - AR93xx TX descriptors are "L1 cache == 128 byte > > aligned" which is an enormous waste of memory compared to a 16 or 32 > > byte aligned platform. Alas.. > > The problem is with cache line sharing, not necessarily with alignment. If you are only ever using one of them at a time, or if you have perfect hygiene, you can cope with this situation without undue waste. The perfect hygiene might be hard sometimes. This brings up an interesting tangential issue for this busdma discussion. For some controller hardware you allocate a block of memory which is treated as an array of "descriptors" or some other shared control information, you set a register in the hardware to point to that block of memory, and then there is some degree of concurrent access of that memory by hardware and CPU. The interesting part is that some such hardware cannot operate in phases as anticipated by our busdma model. That is, there's no clear demarkation points between "the CPU has exclusive access to the memory" and "the hardware has exclusive access to the memory." Usually for these schemes to work correctly, the memory has to be mapped as uncached, unbuffered, strongly ordered, or whatever combo of those makes sense for a given platform. We have arm drivers that use bus_dmamem_alloc() with the BUS_DMA_COHERENT flag to obtain such memory, even though that wasn't the intended meaning for that flag. If the armv4 busdma implementation were changed to stop honoring the COHERENT flag (it's supposed to be an optional feature) those drivers would stop working. So we need to track down such mistakes and fix them, but the question is: fix them how? I think it may make sense to let busdma handle it, because you may get some advantage from the allocation being made based upon the constraints encoded in the inherited chain of tags for the driver. On the other hand, drivers doing this sort of thing are usually pretty close to the silicon and have a good idea for themselves what the hardware constraints are. We could just say that drivers with such needs should call kmem_alloc_contig() or kmem_alloc_attr() for themselves. If we say it's a thing that busdma should handle, then I think we need: * A flag that is universal across all platforms that means unambiguously that you need memory that is mapped however device-register memory is mapped on that platform (uncached, unbuffered, strongly ordered; I'm tempted to say "whatever pmap_mapdev() does" but I'm not sure that's rigorously correct). * If the request cannot be honored for some reason it has to return failure, not quietly give you regular cached memory instead (which is what BUS_DMA_COHERENT does). * The busdma sequence of sync operations does not apply to memory allocated with this flag, and indeed you must not call the sync functions on such memory. The x86 busdma code recently grew a BUS_DMA_NOCACHE flag, perhaps that's the name that should be supported across all platforms? -- Ian From owner-freebsd-arch@FreeBSD.ORG Mon Aug 27 22:12:56 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C81371065670; Mon, 27 Aug 2012 22:12:56 +0000 (UTC) (envelope-from ticso@cicely7.cicely.de) Received: from raven.bwct.de (raven.bwct.de [85.159.14.73]) by mx1.freebsd.org (Postfix) with ESMTP id 55C148FC0A; Mon, 27 Aug 2012 22:12:55 +0000 (UTC) Received: from mail.cicely.de ([10.1.1.37]) by raven.bwct.de (8.13.4/8.13.4) with ESMTP id q7RMCjUO059568 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 28 Aug 2012 00:12:46 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (cicely7.cicely.de [10.1.1.9]) by mail.cicely.de (8.14.4/8.14.4) with ESMTP id q7RMCWPU082659 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 28 Aug 2012 00:12:32 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (localhost [127.0.0.1]) by cicely7.cicely.de (8.14.2/8.14.2) with ESMTP id q7RMCWaC074506; Tue, 28 Aug 2012 00:12:32 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: (from ticso@localhost) by cicely7.cicely.de (8.14.2/8.14.2/Submit) id q7RMCWMQ074505; Tue, 28 Aug 2012 00:12:32 +0200 (CEST) (envelope-from ticso) Date: Tue, 28 Aug 2012 00:12:32 +0200 From: Bernd Walter To: Warner Losh Message-ID: <20120827221231.GB73992@cicely7.cicely.de> References: <6D83AF9D-577B-4C83-84B7-C4E3B32695FC@bsdimp.com> <45C204C3-D4CE-4B00-886A-A88A0FE7CAD7@bsdimp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <45C204C3-D4CE-4B00-886A-A88A0FE7CAD7@bsdimp.com> X-Operating-System: FreeBSD cicely7.cicely.de 7.0-STABLE i386 User-Agent: Mutt/1.5.11 X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED=-1, BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01 autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on spamd.cicely.de Cc: Tim Kientzle , freebsd-arm@freebsd.org, freebsd-arch@freebsd.org, freebsd-mips@freebsd.org, Hans Petter Selasky Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: ticso@cicely.de List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2012 22:12:57 -0000 On Mon, Aug 27, 2012 at 09:34:30AM -0600, Warner Losh wrote: > > On Aug 27, 2012, at 9:12 AM, Tim Kientzle wrote: > > > > > On Aug 27, 2012, at 6:38 AM, Warner Losh wrote: > > > >> > >> On Aug 27, 2012, at 12:06 AM, Hans Petter Selasky wrote: > >> > >>> Hi, > >>> Correct. > >>> > >>>> We also need some rules about working with buffers obtained from > >>>> bus_dmamem_alloc() and external buffers passed to bus_dmamap_load(). I > >>>> think the rule should be that a buffer obtained from bus_dmamem_alloc(), > >>>> or more formally any region of memory mapped by a bus_dmamap_load(), is > >>>> a single logical object which can only be accessed by one entity at a > >>>> time. That means that there cannot be two concurrent DMA operations > >>>> happening in different regions of the same buffer, nor can DMA and CPU > >>>> access be happening concurrently even if in different parts of the > >>>> buffer. > >>> > >>> Is this something which we can fix using a simple __align(USB_DMA_ALIGN) on elements in C-structures which are allowed to be DMA loaded. > >> > >> No. I don't think so. the reason is that you can't define USB_DMA_ALGIN to be a constant on MIPS, at least, or I think ARM because that's determined at run time. > > > > But don't mbuf structures do pretty much what Hans is suggesting? > > They kinda do, and kinda don't. > > > Why is mbuf okay? > > mbuf is OK because it is never changed while the DMA is pending to the buffers. That is, m_hdr and m_ext are invariant while the device owns the memory. In addition, mbuf allocations are so large that no two mbufs share the same cache line (although it looks like 256 might be too small to avoid that on some MIPS processors). usb buffers do not obey these same rules, otherwise none of the stuff we're talking about would matter. I've always hated the design of USB controllers with their zillions of tiny DMA buffers. -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. From owner-freebsd-arch@FreeBSD.ORG Tue Aug 28 08:57:52 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B6B7D106566C for ; Tue, 28 Aug 2012 08:57:52 +0000 (UTC) (envelope-from andrey@zonov.org) Received: from mail-lpp01m010-f54.google.com (mail-lpp01m010-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 2BC918FC0A for ; Tue, 28 Aug 2012 08:57:51 +0000 (UTC) Received: by lage12 with SMTP id e12so3636783lag.13 for ; Tue, 28 Aug 2012 01:57:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:subject :x-enigmail-version:content-type:x-gm-message-state; bh=4lG1wZvlksUYIOlwna4failhumkiEX52ITraAh9scqw=; b=gfxXDIcWWsLsqE/oLyiZ/2Pj9e8tIVKbuug9g0SMAZggUYYdbEPXKXG/EbbESf338X aepMQ4NuUg7b1TjY50QF8hBuMrEGeBN8IzY2QHwxNeq5NLqGfwqbGT62NjdkjU0x8VE0 QIdAtgUdtygLPMFpCA5YCVWSnvO2wqVZOW4DNI8xm4HtHmumVLpLNSGvyucz5C9iSZvI mB140FZyOwPKeP6UEeTvDeUEUVuD6m9qvo1Qj9bBqqhJARpHZsk0hVGs8p/GCjfXNtRR 7muJ4MwviWpASnBwr7g2nAItCY+lY29G2Z4s+akJ66Gdt+N2yhrF1+BzIFP6pjeFVCns xBxA== Received: by 10.152.104.172 with SMTP id gf12mr1020121lab.56.1346144270948; Tue, 28 Aug 2012 01:57:50 -0700 (PDT) Received: from zont-osx.local (ppp95-165-143-86.pppoe.spdop.ru. [95.165.143.86]) by mx.google.com with ESMTPS id sy1sm21487084lab.13.2012.08.28.01.57.49 (version=SSLv3 cipher=OTHER); Tue, 28 Aug 2012 01:57:50 -0700 (PDT) Sender: Andrey Zonov Message-ID: <503C8809.3050507@FreeBSD.org> Date: Tue, 28 Aug 2012 12:57:45 +0400 From: Andrey Zonov User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:14.0) Gecko/20120713 Thunderbird/14.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org X-Enigmail-Version: 1.4.3 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig0788A89F08941CC5D4233F50" X-Gm-Message-State: ALoCoQmLk2trTzpCmgHtG3IqyR8rFGfMDTdZa5QeLnd8qvajE8Qm+Xw2YxJdpDeR9o+XYYS+3dSG Subject: warning: cast increases required alignment of target type X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2012 08:57:52 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig0788A89F08941CC5D4233F50 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi, Does anyone know how to correctly fix this warning for arm/ia64/mips/sparc64? usr.bin/elf2aout/elf2aout.c: In function 'main': usr.bin/elf2aout/elf2aout.c:129: warning: cast increases required alignment of target type I found this explanation from bde, but still don't understand how to correctly fix this issue. ------------------------------------------------------------------------ r99799 | bde | 2002-07-11 22:06:09 +0400 (Thu, 11 Jul 2002) | 10 lines Set NO_WERROR to ignore the following warning which is emitted on alphas: .../elf2aout.c:130: warning: cast increases required alignment of target type The warning is about casting ((char *)e + phoff) to a struct pointer, where e is aligned but phoff might be garbage, so I think the warning should be emitted on most machines (even on i386's, alignment checking might be on) and the correct fix would involve validation phoff before using it. Is this fix correct? Index: usr.bin/elf2aout/elf2aout.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- usr.bin/elf2aout/elf2aout.c (revision 239611) +++ usr.bin/elf2aout/elf2aout.c (working copy) @@ -126,7 +126,7 @@ entry =3D xe64toh(e->e_entry); phoff =3D xe64toh(e->e_phoff); phnum =3D xe16toh(e->e_phnum); - p =3D (Elf64_Phdr *)((char *)e + phoff); + p =3D (Elf64_Phdr *)(void *)((char *)e + phoff); bzero(&a, sizeof(a)); for (i =3D 0; i < phnum; i++) { type =3D xe32toh(p[i].p_type); --=20 Andrey Zonov --------------enig0788A89F08941CC5D4233F50 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.18 (Darwin) Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJQPIgNAAoJEBWLemxX/CvT84YH/ifd1jhi4TimbVr6Ze9roCeF Jy5iZu0mY+0aW6w1guUSrJPE1K5DTqlRlgpWIhRcRogEyCfM6Ami8E4+aCts2OWT BlEXJeHasvzWLuC64KtDLNtl0GLIlFNSk18wxWmRkI1QsfAOy1x94XPcladB9JDV E07lpKJWlFEZfUmSCVSI6/Ui4R0RqHzHh86CaxziOnBgHqKt0s0TJD/UaM0IRf93 T/OzBdp6n/Pivt72CRu45/sbCDrhtGn4yTLTQ4KOjbDTOwBAPoOOyYCgdTk/k+Pe ScGvDubxVWKQJkQToPMB05LWZ206i2Z96L3z253YIIES8dE9rhE3VGARmZPok60= =pGE6 -----END PGP SIGNATURE----- --------------enig0788A89F08941CC5D4233F50-- From owner-freebsd-arch@FreeBSD.ORG Tue Aug 28 13:12:20 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 230DC1065673 for ; Tue, 28 Aug 2012 13:12:20 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id D61D78FC16 for ; Tue, 28 Aug 2012 13:12:19 +0000 (UTC) Received: by ialo14 with SMTP id o14so13523509ial.13 for ; Tue, 28 Aug 2012 06:12:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=E/knWnBt3ho5KsYPmjtbhIS6TCisnVxzJrTW/+MsABk=; b=geDfvKUxXo+8BXmnFs7d4Selhr36ZZL3mTLspeZrOiN351iESzBRK83bsocPVeYeto Sn+MSH+jzxQl7fmLTTDWr7buf6/5FnYFiiL+I5VOtXjlTIds77fS9tD7r5bUsIhaS34x vQymReOopbznUomKJCWtC9I2h5pigrnmy6N9ocmxpC6hPo9C4J4Hkt+UeI6AUpfOheDH iE50i80JMy2unUu41V3cTW1Yj49W3YTR0GGPo5lWgAHqTwreQC6+xL7AiTes6Z06rqWr J8xx7LWJ70ehtZ/rdOT5uznuiMb/KwlTS3Y9i4UzNudRRbsdcspu/t4c/07oCiMv3JPw 9jyw== Received: by 10.50.220.169 with SMTP id px9mr13478929igc.8.1346159539140; Tue, 28 Aug 2012 06:12:19 -0700 (PDT) Received: from 63.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id va9sm3111303igb.17.2012.08.28.06.12.16 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 28 Aug 2012 06:12:16 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <503C8809.3050507@FreeBSD.org> Date: Tue, 28 Aug 2012 07:12:15 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <503C8809.3050507@FreeBSD.org> To: Andrey Zonov X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQkFJv2V1LwoUIRu4SqhMAsP32QSwAigGlvOLFicBmUbdCfVuGT4L5rR30InpqN5X42/tqBv Cc: freebsd-arch@freebsd.org Subject: Re: warning: cast increases required alignment of target type X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2012 13:12:20 -0000 On Aug 28, 2012, at 2:57 AM, Andrey Zonov wrote: > Hi, >=20 > Does anyone know how to correctly fix this warning for > arm/ia64/mips/sparc64? >=20 > usr.bin/elf2aout/elf2aout.c: In function 'main': > usr.bin/elf2aout/elf2aout.c:129: warning: cast increases required > alignment of target type >=20 > I found this explanation from bde, but still don't understand how to > correctly fix this issue. >=20 > = ------------------------------------------------------------------------ > r99799 | bde | 2002-07-11 22:06:09 +0400 (Thu, 11 Jul 2002) | 10 lines >=20 > Set NO_WERROR to ignore the following warning which is emitted on > alphas: > .../elf2aout.c:130: warning: cast increases required alignment of > target type > The warning is about casting ((char *)e + phoff) to a struct pointer, > where e is aligned but phoff might be garbage, so I think the warning > should be emitted on most machines (even on i386's, alignment checking > might be on) and the correct fix would involve validation phoff before > using it. >=20 > Is this fix correct? No. You need to tell the compiler that e has the alignment you think it = has so that it can check to make sure that you are actually right. Just = casting like this defeats the purpose of the check and will break on = other architectures. Warner > Index: usr.bin/elf2aout/elf2aout.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- usr.bin/elf2aout/elf2aout.c (revision 239611) > +++ usr.bin/elf2aout/elf2aout.c (working copy) > @@ -126,7 +126,7 @@ > entry =3D xe64toh(e->e_entry); > phoff =3D xe64toh(e->e_phoff); > phnum =3D xe16toh(e->e_phnum); > - p =3D (Elf64_Phdr *)((char *)e + phoff); > + p =3D (Elf64_Phdr *)(void *)((char *)e + phoff); > bzero(&a, sizeof(a)); > for (i =3D 0; i < phnum; i++) { > type =3D xe32toh(p[i].p_type); >=20 >=20 > --=20 > Andrey Zonov >=20 From owner-freebsd-arch@FreeBSD.ORG Tue Aug 28 13:29:01 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5893A106566B; Tue, 28 Aug 2012 13:29:01 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id C8B0F8FC12; Tue, 28 Aug 2012 13:29:00 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q7SDT3GD014063; Tue, 28 Aug 2012 16:29:03 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q7SDSpjr065432; Tue, 28 Aug 2012 16:28:51 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q7SDSpke065431; Tue, 28 Aug 2012 16:28:51 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 28 Aug 2012 16:28:51 +0300 From: Konstantin Belousov To: Warner Losh Message-ID: <20120828132851.GN33100@deviant.kiev.zoral.com.ua> References: <503C8809.3050507@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="rNtzt+pFA3UwHi4l" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Andrey Zonov , freebsd-arch@freebsd.org Subject: Re: warning: cast increases required alignment of target type X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2012 13:29:01 -0000 --rNtzt+pFA3UwHi4l Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Aug 28, 2012 at 07:12:15AM -0600, Warner Losh wrote: >=20 > On Aug 28, 2012, at 2:57 AM, Andrey Zonov wrote: >=20 > > Hi, > >=20 > > Does anyone know how to correctly fix this warning for > > arm/ia64/mips/sparc64? > >=20 > > usr.bin/elf2aout/elf2aout.c: In function 'main': > > usr.bin/elf2aout/elf2aout.c:129: warning: cast increases required > > alignment of target type > >=20 > > I found this explanation from bde, but still don't understand how to > > correctly fix this issue. > >=20 > > ------------------------------------------------------------------------ > > r99799 | bde | 2002-07-11 22:06:09 +0400 (Thu, 11 Jul 2002) | 10 lines > >=20 > > Set NO_WERROR to ignore the following warning which is emitted on > > alphas: > > .../elf2aout.c:130: warning: cast increases required alignment of > > target type > > The warning is about casting ((char *)e + phoff) to a struct pointer, > > where e is aligned but phoff might be garbage, so I think the warning > > should be emitted on most machines (even on i386's, alignment checking > > might be on) and the correct fix would involve validation phoff before > > using it. > >=20 > > Is this fix correct? >=20 > No. You need to tell the compiler that e has the alignment you think > it has so that it can check to make sure that you are actually right. > Just casting like this defeats the purpose of the check and will break > on other architectures. Does compiler ever checks the alignment ? I thought that it is hardware that performs (transparent) checks, like elfags.AC bit on x86. IMO hardware checks are good enough, since having program headers not aligned as required by ELF standard/ABI means that incoming ELF file is corrupted. Note that the same code is present in libexec/rtld-elf/map_object.c:map_object(), and there it might indeed make sense to introduce manual check for the alignment, to avoid hw triggering trap. Instead, rtld could gracefully refuse to load the mis-formed object. --rNtzt+pFA3UwHi4l Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlA8x5IACgkQC3+MBN1Mb4hQkQCg8otcSRnQ3zwehPQZ/04vvYk5 rq8AoNiZm03DkLPOMCjl4SZHV8YpeVMU =RdCC -----END PGP SIGNATURE----- --rNtzt+pFA3UwHi4l-- From owner-freebsd-arch@FreeBSD.ORG Tue Aug 28 13:31:54 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C7C0D1065670 for ; Tue, 28 Aug 2012 13:31:54 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 82DD88FC15 for ; Tue, 28 Aug 2012 13:31:54 +0000 (UTC) Received: by ialo14 with SMTP id o14so13579751ial.13 for ; Tue, 28 Aug 2012 06:31:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=LKzJL3diBU1leuz/sL4VQw6HmBzrQGFkJCsfmUddqHg=; b=pSIrTB94S6c6YS4Zr1w+rbol2HkkifUnDs/vlLD+BrMW3q7hQo1VXeHNNSRRMfI+7Z YXGm8Hv+7+bLBgtjrp9Jud/ulv6iv3+qr12ozNqyPrGc95761tTtdiHn/OlXqKX4LqDQ V68wq+K278oYmT2Xx7XgBoOfM/M5y/lR6jA5QQBT8tNjr+qElGM9oAZMpxLIgi10rF98 MfWhC1bZJC22GeHQvN1j+HtFvFV+JDvH6ME9m44M8NhxAClVw2zHGvpjqGiYmtiwX8ao EIoIPD/pWvzVC2Zf5E5fSPZUEv+t8wYm/J9bvyMy4iXMp8Mk1vMSJ4tVlbQ+GfaXSi27 B7nQ== Received: by 10.50.222.193 with SMTP id qo1mr2888722igc.70.1346160713521; Tue, 28 Aug 2012 06:31:53 -0700 (PDT) Received: from 63.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id wn5sm81390igc.7.2012.08.28.06.31.52 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 28 Aug 2012 06:31:52 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <20120828132851.GN33100@deviant.kiev.zoral.com.ua> Date: Tue, 28 Aug 2012 07:31:51 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <1B08245D-D467-48A5-B18C-9C8FF6823F6F@bsdimp.com> References: <503C8809.3050507@FreeBSD.org> <20120828132851.GN33100@deviant.kiev.zoral.com.ua> To: Konstantin Belousov X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQkycEEa8wYvRw8PQw2yakodYrUXjBvMaSjJgU6XhpTMAFTBMVm7tlsZjIDih1kyPqmhHJ/0 Cc: Andrey Zonov , freebsd-arch@freebsd.org Subject: Re: warning: cast increases required alignment of target type X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2012 13:31:55 -0000 On Aug 28, 2012, at 7:28 AM, Konstantin Belousov wrote: > On Tue, Aug 28, 2012 at 07:12:15AM -0600, Warner Losh wrote: >>=20 >> On Aug 28, 2012, at 2:57 AM, Andrey Zonov wrote: >>=20 >>> Hi, >>>=20 >>> Does anyone know how to correctly fix this warning for >>> arm/ia64/mips/sparc64? >>>=20 >>> usr.bin/elf2aout/elf2aout.c: In function 'main': >>> usr.bin/elf2aout/elf2aout.c:129: warning: cast increases required >>> alignment of target type >>>=20 >>> I found this explanation from bde, but still don't understand how to >>> correctly fix this issue. >>>=20 >>> = ------------------------------------------------------------------------ >>> r99799 | bde | 2002-07-11 22:06:09 +0400 (Thu, 11 Jul 2002) | 10 = lines >>>=20 >>> Set NO_WERROR to ignore the following warning which is emitted on >>> alphas: >>> .../elf2aout.c:130: warning: cast increases required alignment of >>> target type >>> The warning is about casting ((char *)e + phoff) to a struct = pointer, >>> where e is aligned but phoff might be garbage, so I think the = warning >>> should be emitted on most machines (even on i386's, alignment = checking >>> might be on) and the correct fix would involve validation phoff = before >>> using it. >>>=20 >>> Is this fix correct? >>=20 >> No. You need to tell the compiler that e has the alignment you think >> it has so that it can check to make sure that you are actually right. >> Just casting like this defeats the purpose of the check and will = break >> on other architectures. >=20 > Does compiler ever checks the alignment ? I thought that it is = hardware that > performs (transparent) checks, like elfags.AC bit on x86. Yes. Hence the warning. The hardware does *NOT* transparently fix = misaligned accesses, which causes core dumps which is why this warning = is in place. > IMO hardware checks are good enough, since having program headers > not aligned as required by ELF standard/ABI means that incoming ELF = file > is corrupted. It is alignment in the source code. Telling the compiler about these = alignments is what I'm suggesting. > Note that the same code is present in > libexec/rtld-elf/map_object.c:map_object(), and there it might indeed > make sense to introduce manual check for the alignment, to avoid hw > triggering trap. Instead, rtld could gracefully refuse to load the > mis-formed object. In the rtld case, we should know if things are properly aligned or not = as part of the ABI. A check wouldn't hurt too much, but it is done at = every program execution and too many sanity checks that are never = happens can cause performance hits. Warner= From owner-freebsd-arch@FreeBSD.ORG Tue Aug 28 13:48:53 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2CCE4106566C; Tue, 28 Aug 2012 13:48:53 +0000 (UTC) (envelope-from marktinguely@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id DDDF58FC17; Tue, 28 Aug 2012 13:48:51 +0000 (UTC) Received: by pbbrp2 with SMTP id rp2so9660554pbb.13 for ; Tue, 28 Aug 2012 06:48:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=P71mtBUeKJqJJ7uppZZUYzebtvvWMylC3ZFDuLWwN8U=; b=bHZLuzFKqvhefHZ1LPS5tIpUsqC75R1yub1u8o2tFUZoD5ipMaiREpoTrKdMT7Emxe dnj+MRbW+060ycWl/z6nYyRNRPZTFa448jZz9i2j6C9d9ucVZAoRZeVWkxflBRdoLPMg +/SnpzKDAqAgwiHQJwZoTo4N0mHy64YtmDaxMpnKRl0TJxFZovCxGuVOnie5nHCH/rE9 OneNADf1F1c1eEp3mNDK/8FrZeRFWghdsCcbVk747yOV5YDPmORpnyqMNlfCbVgNqcJF gDNyB6UKcpKjaNyb/zQeH/WsKEDkXVME05CdzU/QWMEhNo7GT3GF2v/ITheHdTzvYmRc C6yQ== MIME-Version: 1.0 Received: by 10.68.231.130 with SMTP id tg2mr3863853pbc.70.1346161731445; Tue, 28 Aug 2012 06:48:51 -0700 (PDT) Received: by 10.68.229.227 with HTTP; Tue, 28 Aug 2012 06:48:51 -0700 (PDT) In-Reply-To: <201208271531.42725.jhb@freebsd.org> References: <10307B47-13F3-45C0-87F7-66FD3ACA3F86@bsdimp.com> <20120827185346.GE33100@deviant.kiev.zoral.com.ua> <201208271531.42725.jhb@freebsd.org> Date: Tue, 28 Aug 2012 08:48:51 -0500 Message-ID: From: Mark Tinguely To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: Ian Lepore , Hans Petter Selasky , freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org, Konstantin Belousov Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2012 13:48:53 -0000 On Mon, Aug 27, 2012 at 2:31 PM, John Baldwin wrote: > On Monday, August 27, 2012 2:53:46 pm Konstantin Belousov wrote: >> On Sun, Aug 26, 2012 at 05:13:31PM -0600, Warner Losh wrote: >> > >> > On Aug 26, 2012, at 12:25 PM, Ian Lepore wrote: >> > > In this regard, it's the busdma implementation that's broken, because it >> > > should bounce those IOs through a DMA-safe buffer. There's absolutely >> > > no rule that I've ever heard of in FreeBSD that says IO can only take >> > > place using memory allocated from busdma. >> > >> > That's partially true. Since BUSDMA grew up in the storage area, you >> > must allocate the memory from busdma, or it must be page aligned has >> > been the de-facto rule here. The mbuf and uio variants of load were >> > invented to cope with common cases of mbufs and user I/O to properly >> > flag things. >> >> I once looked at x86 bus_dmamap_load_uio(), and I was unable to >> understand how to use it with usermode uio. I think this is a good >> moment to ask. Most existing users use UIO_SYSSPACE, but several crypto >> drivers might allow the UIO_USERSPACE for them. >> >> For UIO_USERSPACE, if the page is not resident, the pmap_extract() call from >> _bus_dmamap_load_buffer() returns 0. So the i/o happens to the page >> located at 0, which contains real mode IVT and other BIOS sensitive tables. >> >> Worse, if the page is resident, but it is mapped at the region which >> requires COW on write, then DMA will be performed to the wrong page >> which is typically shared with other innocent users. to the COW area >> which was not yet copied, >> >> Am I missing some trick there ? > > No. The caller is required to wire the pages first in some manner. > In general bus_dmamap_load_uio() isn't a good idea. I do believe the > crypto drivers are careful to wire the buffer first. I think requiring > the caller to wire is the only sane way that can be used. > > Also, doing DMA to a stack variable is absolutely horrible for a related > reason since presumably the thread will block while it waits for the DMA > to complete, and a sleeping thread can be swapped out (including having > it's stack swapped out). > > -- > John Baldwin The POST commands for UIO DMA may be a bit hairy because the address space may not be the same as the caller. Someone once told me they got around that (in a Sparc enviroment?) by shadow mapping the address to KVA. --Mark TInguely. From owner-freebsd-arch@FreeBSD.ORG Tue Aug 28 14:02:29 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3AA94106566B for ; Tue, 28 Aug 2012 14:02:29 +0000 (UTC) (envelope-from andrey@zonov.org) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id 9C67E8FC16 for ; Tue, 28 Aug 2012 14:02:28 +0000 (UTC) Received: by lbbgg13 with SMTP id gg13so3854770lbb.13 for ; Tue, 28 Aug 2012 07:02:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :x-gm-message-state; bh=EmGS27ndmxzBRipFqW0CO+tzpy6q2y3u/+3whMVoaro=; b=niZMgagmfHqsdg6JY3X6GLqC6KSA/rOVZmPfKrx4kEE++FvzWRCuoSOOodXgXuCRNX KQef/ge3zLQPAIqT6hU4dAqTCrqAiyXEi7ghqn1YYuNEL1YTTsk0A3xF12yPGlMDinvY uLQ94iQ9v85uWG6AOowA4vBgLNSkQXxbtbYNAtJeqUAtgHjUcdCTGQVC07GqwsXcoc65 xwmCWCOlwkko4urNxzYnDlmk0h7+J0HIZVaWss+SUKIHbe3WEX8T1DG25ZsIj34+kk7e d/jBLMXkTQ467QzugpEDCdA1kHsPPO4honXyNYtGVTs+ektPXQC1KGK5raFQ5/LRJDS5 fkiA== Received: by 10.152.112.136 with SMTP id iq8mr18687949lab.18.1346162547227; Tue, 28 Aug 2012 07:02:27 -0700 (PDT) Received: from zont-osx.local (ppp95-165-143-86.pppoe.spdop.ru. [95.165.143.86]) by mx.google.com with ESMTPS id y4sm2151756lbg.5.2012.08.28.07.02.26 (version=SSLv3 cipher=OTHER); Tue, 28 Aug 2012 07:02:26 -0700 (PDT) Sender: Andrey Zonov Message-ID: <503CCF6E.7080208@FreeBSD.org> Date: Tue, 28 Aug 2012 18:02:22 +0400 From: Andrey Zonov User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:14.0) Gecko/20120713 Thunderbird/14.0 MIME-Version: 1.0 To: Warner Losh References: <503C8809.3050507@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.4.3 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigC0AA8BFED6D1E4FEBC86BF53" X-Gm-Message-State: ALoCoQl7X+Qtvip6KH09o1NxJyLKATsvTRcP1jXZFaMf3NYNdbdJqKt7eYzxwn63NEsUS08tsbke Cc: freebsd-arch@freebsd.org Subject: Re: warning: cast increases required alignment of target type X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2012 14:02:29 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigC0AA8BFED6D1E4FEBC86BF53 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 8/28/12 5:12 PM, Warner Losh wrote: >=20 > On Aug 28, 2012, at 2:57 AM, Andrey Zonov wrote: >=20 >> Hi, >> >> Does anyone know how to correctly fix this warning for >> arm/ia64/mips/sparc64? >> >> usr.bin/elf2aout/elf2aout.c: In function 'main': >> usr.bin/elf2aout/elf2aout.c:129: warning: cast increases required >> alignment of target type >> >> I found this explanation from bde, but still don't understand how to >> correctly fix this issue. >> >> ----------------------------------------------------------------------= -- >> r99799 | bde | 2002-07-11 22:06:09 +0400 (Thu, 11 Jul 2002) | 10 lines= >> >> Set NO_WERROR to ignore the following warning which is emitted on >> alphas: >> .../elf2aout.c:130: warning: cast increases required alignment of >> target type >> The warning is about casting ((char *)e + phoff) to a struct pointer, >> where e is aligned but phoff might be garbage, so I think the warning >> should be emitted on most machines (even on i386's, alignment checking= >> might be on) and the correct fix would involve validation phoff before= >> using it. >> >> Is this fix correct? >=20 > No. You need to tell the compiler that e has the alignment you think i= t has so that it can check to make sure that you are actually right. Jus= t casting like this defeats the purpose of the check and will break on ot= her architectures. >=20 > Warner >=20 What do you say about this one? Index: usr.bin/elf2aout/elf2aout.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- usr.bin/elf2aout/elf2aout.c (revision 239611) +++ usr.bin/elf2aout/elf2aout.c (working copy) @@ -126,7 +126,7 @@ entry =3D xe64toh(e->e_entry); phoff =3D xe64toh(e->e_phoff); phnum =3D xe16toh(e->e_phnum); - p =3D (Elf64_Phdr *)((char *)e + phoff); + p =3D (Elf64_Phdr *)(e + phoff); bzero(&a, sizeof(a)); for (i =3D 0; i < phnum; i++) { type =3D xe32toh(p[i].p_type); I can build elf2aout without warnings on amd64 in 'buildenv' for all targets. >> Index: usr.bin/elf2aout/elf2aout.c >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> --- usr.bin/elf2aout/elf2aout.c (revision 239611) >> +++ usr.bin/elf2aout/elf2aout.c (working copy) >> @@ -126,7 +126,7 @@ >> entry =3D xe64toh(e->e_entry); >> phoff =3D xe64toh(e->e_phoff); >> phnum =3D xe16toh(e->e_phnum); >> - p =3D (Elf64_Phdr *)((char *)e + phoff); >> + p =3D (Elf64_Phdr *)(void *)((char *)e + phoff); >> bzero(&a, sizeof(a)); >> for (i =3D 0; i < phnum; i++) { >> type =3D xe32toh(p[i].p_type); >> >> >> --=20 >> Andrey Zonov >> >=20 --=20 Andrey Zonov --------------enigC0AA8BFED6D1E4FEBC86BF53 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.18 (Darwin) Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJQPM9wAAoJEBWLemxX/CvTx3YH/RMywLzQHvj1Q4gjdmElavnh v11NF5TNplZxMB5RR/7cY202Jw3GOIV4Jeei1ElqNS1tmBjALtO0h1ROfaGbTGki C8Hb8L/U+Te0TZjI8VSpvv3CwmG99iZNjrxtrIg0RXRmWcAzb7su81eKqrmDhdWl r/YehA6gg4Y2ePWRL2Sq0DLJNMPwTwSjNxJZI82MzJAlBDJlqGSP/PDi+u1maIY6 k1LoNoVnb4MMoBRVZ/pa6KgD3BgSxumPCU5DOl9Nm2k3Xb61mmwBN1aiXrMjjSpJ 7MDvWRcpzU78b4cFNFLtWywTrdpZTEWr1fUPF5rMLcwMqPrVikky2VzHEInVDnQ= =DEZE -----END PGP SIGNATURE----- --------------enigC0AA8BFED6D1E4FEBC86BF53-- From owner-freebsd-arch@FreeBSD.ORG Tue Aug 28 14:07:25 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CAEAD1065676; Tue, 28 Aug 2012 14:07:25 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 5EEE38FC19; Tue, 28 Aug 2012 14:07:25 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q7SE6w0C017253; Tue, 28 Aug 2012 17:06:58 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q7SE6kGa065649; Tue, 28 Aug 2012 17:06:46 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q7SE6kGV065648; Tue, 28 Aug 2012 17:06:46 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 28 Aug 2012 17:06:46 +0300 From: Konstantin Belousov To: Andrey Zonov Message-ID: <20120828140646.GP33100@deviant.kiev.zoral.com.ua> References: <503C8809.3050507@FreeBSD.org> <503CCF6E.7080208@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="V5RPc/ioYsxrZT/b" Content-Disposition: inline In-Reply-To: <503CCF6E.7080208@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-arch@freebsd.org Subject: Re: warning: cast increases required alignment of target type X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2012 14:07:26 -0000 --V5RPc/ioYsxrZT/b Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Aug 28, 2012 at 06:02:22PM +0400, Andrey Zonov wrote: > On 8/28/12 5:12 PM, Warner Losh wrote: > >=20 > > On Aug 28, 2012, at 2:57 AM, Andrey Zonov wrote: > >=20 > >> Hi, > >> > >> Does anyone know how to correctly fix this warning for > >> arm/ia64/mips/sparc64? > >> > >> usr.bin/elf2aout/elf2aout.c: In function 'main': > >> usr.bin/elf2aout/elf2aout.c:129: warning: cast increases required > >> alignment of target type > >> > >> I found this explanation from bde, but still don't understand how to > >> correctly fix this issue. > >> > >> ----------------------------------------------------------------------= -- > >> r99799 | bde | 2002-07-11 22:06:09 +0400 (Thu, 11 Jul 2002) | 10 lines > >> > >> Set NO_WERROR to ignore the following warning which is emitted on > >> alphas: > >> .../elf2aout.c:130: warning: cast increases required alignment of > >> target type > >> The warning is about casting ((char *)e + phoff) to a struct pointer, > >> where e is aligned but phoff might be garbage, so I think the warning > >> should be emitted on most machines (even on i386's, alignment checking > >> might be on) and the correct fix would involve validation phoff before > >> using it. > >> > >> Is this fix correct? > >=20 > > No. You need to tell the compiler that e has the alignment you think i= t has so that it can check to make sure that you are actually right. Just = casting like this defeats the purpose of the check and will break on other = architectures. > >=20 > > Warner > >=20 >=20 > What do you say about this one? >=20 > Index: usr.bin/elf2aout/elf2aout.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- usr.bin/elf2aout/elf2aout.c (revision 239611) > +++ usr.bin/elf2aout/elf2aout.c (working copy) > @@ -126,7 +126,7 @@ > entry =3D xe64toh(e->e_entry); > phoff =3D xe64toh(e->e_phoff); > phnum =3D xe16toh(e->e_phnum); > - p =3D (Elf64_Phdr *)((char *)e + phoff); > + p =3D (Elf64_Phdr *)(e + phoff); This is plain wrong. Pointer arithmetic steps over the object's count, not the char count. You end up incrementing the pointer by sizeof(Elf64_Ehdr) * phoff. --V5RPc/ioYsxrZT/b Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlA80HUACgkQC3+MBN1Mb4gN/wCeIqW6V9e4laTixzkB+x3mEJGJ prwAn2l9tWplARTgmDer/lRXQ98UyQLU =+2F4 -----END PGP SIGNATURE----- --V5RPc/ioYsxrZT/b-- From owner-freebsd-arch@FreeBSD.ORG Tue Aug 28 14:10:36 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 292B6106566C for ; Tue, 28 Aug 2012 14:10:36 +0000 (UTC) (envelope-from andrey@zonov.org) Received: from mail-lpp01m010-f54.google.com (mail-lpp01m010-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 8F71B8FC14 for ; Tue, 28 Aug 2012 14:10:35 +0000 (UTC) Received: by lage12 with SMTP id e12so3872007lag.13 for ; Tue, 28 Aug 2012 07:10:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :x-gm-message-state; bh=10xJHBw8RkGGD/4lwitH9KlSVaI+qtHLyntReqoprsI=; b=kpHMMs5qJH08G6EiVlNq1KeB0/bNKHnr3ADUHetpEXhHjDHuwFtV31fHWx3gzC9mYF g2hMD4opwOQ6f+JKGi9jGZe5oZu4XOLViJB9iVWKdZ0FSCY6kiqIdeG/vcyANszLWoA5 jBcj1WNbFjT4ucpvIhiXuRMBByN7ohCtSt8/6xuKZ8/2DsE+1OaLdpZ8xT74PvqDUqjq 023ibUC7F/Tl0Z/C4xImL7lajlbubJJ7PZZEBQEvNERzFb2A4smGZ9xOFA97j74O8eeV k4YhY/LNDT0M7y8RfWqZ+hsjY3oS8doyiKODhKLeQFSj5CTsqphNszTatf9yo5IQ2e8b QpdA== Received: by 10.152.112.138 with SMTP id iq10mr18737959lab.13.1346163034166; Tue, 28 Aug 2012 07:10:34 -0700 (PDT) Received: from zont-osx.local (ppp95-165-143-86.pppoe.spdop.ru. [95.165.143.86]) by mx.google.com with ESMTPS id xx8sm22205012lab.10.2012.08.28.07.10.32 (version=SSLv3 cipher=OTHER); Tue, 28 Aug 2012 07:10:33 -0700 (PDT) Sender: Andrey Zonov Message-ID: <503CD157.50306@FreeBSD.org> Date: Tue, 28 Aug 2012 18:10:31 +0400 From: Andrey Zonov User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:14.0) Gecko/20120713 Thunderbird/14.0 MIME-Version: 1.0 To: Konstantin Belousov References: <503C8809.3050507@FreeBSD.org> <503CCF6E.7080208@FreeBSD.org> <20120828140646.GP33100@deviant.kiev.zoral.com.ua> In-Reply-To: <20120828140646.GP33100@deviant.kiev.zoral.com.ua> X-Enigmail-Version: 1.4.3 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigDDAEA3B6B075123B90F271DA" X-Gm-Message-State: ALoCoQlCLeDyEreVIs0V15sGaD8hMN0FA/telOnJ86KSxTbuMWalJwfnkz4yKHGvxGv8IyEweVqw Cc: freebsd-arch@freebsd.org Subject: Re: warning: cast increases required alignment of target type X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2012 14:10:36 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigDDAEA3B6B075123B90F271DA Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 8/28/12 6:06 PM, Konstantin Belousov wrote: > On Tue, Aug 28, 2012 at 06:02:22PM +0400, Andrey Zonov wrote: >> On 8/28/12 5:12 PM, Warner Losh wrote: >>> >>> On Aug 28, 2012, at 2:57 AM, Andrey Zonov wrote: >>> >>>> Hi, >>>> >>>> Does anyone know how to correctly fix this warning for >>>> arm/ia64/mips/sparc64? >>>> >>>> usr.bin/elf2aout/elf2aout.c: In function 'main': >>>> usr.bin/elf2aout/elf2aout.c:129: warning: cast increases required >>>> alignment of target type >>>> >>>> I found this explanation from bde, but still don't understand how to= >>>> correctly fix this issue. >>>> >>>> --------------------------------------------------------------------= ---- >>>> r99799 | bde | 2002-07-11 22:06:09 +0400 (Thu, 11 Jul 2002) | 10 lin= es >>>> >>>> Set NO_WERROR to ignore the following warning which is emitted on >>>> alphas: >>>> .../elf2aout.c:130: warning: cast increases required alignment of= >>>> target type >>>> The warning is about casting ((char *)e + phoff) to a struct pointer= , >>>> where e is aligned but phoff might be garbage, so I think the warnin= g >>>> should be emitted on most machines (even on i386's, alignment checki= ng >>>> might be on) and the correct fix would involve validation phoff befo= re >>>> using it. >>>> >>>> Is this fix correct? >>> >>> No. You need to tell the compiler that e has the alignment you think= it has so that it can check to make sure that you are actually right. J= ust casting like this defeats the purpose of the check and will break on = other architectures. >>> >>> Warner >>> >> >> What do you say about this one? >> >> Index: usr.bin/elf2aout/elf2aout.c >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> --- usr.bin/elf2aout/elf2aout.c (revision 239611) >> +++ usr.bin/elf2aout/elf2aout.c (working copy) >> @@ -126,7 +126,7 @@ >> entry =3D xe64toh(e->e_entry); >> phoff =3D xe64toh(e->e_phoff); >> phnum =3D xe16toh(e->e_phnum); >> - p =3D (Elf64_Phdr *)((char *)e + phoff); >> + p =3D (Elf64_Phdr *)(e + phoff); > This is plain wrong. Pointer arithmetic steps over the object's > count, not the char count. You end up incrementing the pointer by > sizeof(Elf64_Ehdr) * phoff. >=20 Ooops, you are totally right. --=20 Andrey Zonov --------------enigDDAEA3B6B075123B90F271DA Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.18 (Darwin) Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJQPNFXAAoJEBWLemxX/CvTyNMH/07DQY9SEqLzWCicIpF5l8Lo 5S97NSBUee97EMaBeI+vs3oWNp2tpl8klcil6GKg5731rhfWG7uwMToV8wBZL1jW fo2xdyUZjV2WgEw0s88uBjXLGhjmYtGu5aMBqTDjeszF+at47IbdGjoAO754GYfe 90gNj8NZyJz9o2xBeyeh7jr8bLylx7OlW9FNLRp7cap1kE3kVe3SG4Uo3CPxdn0I AfZ2bBNhHq0IKBfF++YKMtqCQWTrFAxumUiBy/eNcgGo5LlYDX3g9Oqivm7oDiy8 KY4VIF0Fc/uNeojaKwLJOp2nRy157q9jIRJfYDcmv3oam3gJfEwFok15gFyStW4= =k3PF -----END PGP SIGNATURE----- --------------enigDDAEA3B6B075123B90F271DA-- From owner-freebsd-arch@FreeBSD.ORG Tue Aug 28 14:31:27 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EA758106566B; Tue, 28 Aug 2012 14:31:27 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id 571348FC12; Tue, 28 Aug 2012 14:31:23 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id q7SEVMEj006002; Tue, 28 Aug 2012 08:31:22 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id q7SEVJ5w033171; Tue, 28 Aug 2012 08:31:19 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) From: Ian Lepore To: Warner Losh In-Reply-To: References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> <1345766109.27688.606.camel@revolution.hippie.lan> <1346002922.1140.56.camel@revolution.hippie.lan> <1346005507.1140.69.camel@revolution.hippie.lan> <10307B47-13F3-45C0-87F7-66FD3ACA3F86@bsdimp.com> <1346081557.1140.181.camel@revolution.hippie.lan> Content-Type: text/plain; charset="us-ascii" Date: Tue, 28 Aug 2012 08:31:19 -0600 Message-ID: <1346164279.1140.328.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-arm@freebsd.org, Hans Petter Selasky , Mark Tinguely , freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2012 14:31:28 -0000 On Mon, 2012-08-27 at 09:57 -0600, Warner Losh wrote: > >> I don't think the cache line problem can be solved with bounce > buffers. Trying to accommodate broken drivers is what lead us to this > spot. We need to fix the broken drivers. If that's impossible, then > the best we can do is have the driver set a 'always bounce' flag in > the tag it creates and use that to always bounce for operations > through that tag. > > > > So you'd be okay with a driver setting a flag that says "always > bounce" > > but not okay with the busdma layer bouncing only when it's actually > > necessary? I'm confused -- do you think the busdma layer will be > unable > > to detect when it's necessary unless directed from the outside? > > Actually, it is good you are confused. I was wrong when I said I'd be > happy with a flag that says always bounce. The reason is that the > driver can do this all the time for those cases like the at91 mci > driver does. After studying the busdma implementation code and pondering this some more, I now understand why the automatic bouncing that I think is the ideal solution to this is so hard to accomplish. I'm not sure "Impossible" is the right description, but I'm afraid that in the long run "impractical" might turn out to be the case. I'm going to think all this over for a few days, maybe do a bit of actual paying work, before revisiting it. -- Ian From owner-freebsd-arch@FreeBSD.ORG Tue Aug 28 16:37:14 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C960210656EC for ; Tue, 28 Aug 2012 16:37:14 +0000 (UTC) (envelope-from andrey@zonov.org) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3C7928FC19 for ; Tue, 28 Aug 2012 16:37:13 +0000 (UTC) Received: by lbbgg13 with SMTP id gg13so4000864lbb.13 for ; Tue, 28 Aug 2012 09:37:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:subject :x-enigmail-version:content-type:x-gm-message-state; bh=DrVd+LjAf2mRVnZarJralrvtrBH2ophg15zLAj2Lu2g=; b=mcGJp0EBJg/0zeIOELgPeqNl+JhzJFXTcJ4YJAZOdLVQyiSrC7QXgjQXTfb/0nRv1X YIVp4ZNpQ7vRpyZdb35osl/RQghQZJyNvDOiJrtsiK16O2BqidLQwrA9x8UGjiRCTGAd AgmHFLTi39MjkadJM7O6Cttuonf462YJ8X36depxMoCabaFhP5WtW6Dw1b9BsDK7P+Qr j2yx9w8vIWBdW65XhrtlK76O7dcsC2kRfoO3uJ3YMURvvJuC4+ZFk8pbAgHpqmtCJPRl ooQyFGSlmjeEXaeDNPg/q98N2G8lBKayNjcQhAU2a/XYYmO3CH8MgU198+Icqm1UOznB uPLA== Received: by 10.112.25.106 with SMTP id b10mr8330504lbg.28.1346171832937; Tue, 28 Aug 2012 09:37:12 -0700 (PDT) Received: from zont-osx.local (ppp95-165-143-86.pppoe.spdop.ru. [95.165.143.86]) by mx.google.com with ESMTPS id nf5sm22552302lab.3.2012.08.28.09.37.11 (version=SSLv3 cipher=OTHER); Tue, 28 Aug 2012 09:37:12 -0700 (PDT) Sender: Andrey Zonov Message-ID: <503CF3B1.3050604@FreeBSD.org> Date: Tue, 28 Aug 2012 20:37:05 +0400 From: Andrey Zonov User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:14.0) Gecko/20120713 Thunderbird/14.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org X-Enigmail-Version: 1.4.3 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig936FD783B61C3D5F55FA814F" X-Gm-Message-State: ALoCoQknMyswHAhzAzMVEJ7kriFlFDbBW2nXmWoFlOW6LzDZAVTGhIpb6gBtm1IJOrPsJRXhjktE Subject: [patch] unprivileged mlock(2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2012 16:37:14 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig936FD783B61C3D5F55FA814F Content-Type: multipart/mixed; boundary="------------080705060908010301070308" This is a multi-part message in MIME format. --------------080705060908010301070308 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi, We've got RLIMIT_MEMLOCK for years, but this limit is useless, because only root may call mlock(2), and root may raise any limits. I suggest patch that allows to call mlock(2) for unprivileged users. Are there any objections to got it in tree? --=20 Andrey Zonov --------------080705060908010301070308 Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0"; name="mlock.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="mlock.patch" - Allow non-root users to call mlock(2)/munlock(2) and mlockall(2)/munlockall(2). Now RLIMIT_MEMLOCK makes sense. - Add sysctl security.bsd.unprivileged_mlock to deny ability of calling mlock(2) to non-root users. Approved by: kib (mentor) MFC after: 2 weeks Index: sys/vm/vm_mmap.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- sys/vm/vm_mmap.c (revision 239772) +++ sys/vm/vm_mmap.c (working copy) @@ -1015,6 +1015,10 @@ done2: return (error); } =20 +static int unprivileged_mlock =3D 1; +SYSCTL_INT(_security_bsd, OID_AUTO, unprivileged_mlock, CTLFLAG_RW, + &unprivileged_mlock, 0, "Unprivileged processes may lock the memory"= ); + #ifndef _SYS_SYSPROTO_H_ struct mlock_args { const void *addr; @@ -1035,9 +1039,11 @@ sys_mlock(td, uap) unsigned long nsize; int error; =20 - error =3D priv_check(td, PRIV_VM_MLOCK); - if (error) - return (error); + if (!unprivileged_mlock) { + error =3D priv_check(td, PRIV_VM_MLOCK); + if (error) + return (error); + } addr =3D (vm_offset_t)uap->addr; size =3D uap->len; last =3D addr + size; @@ -1114,9 +1120,11 @@ sys_mlockall(td, uap) } PROC_UNLOCK(td->td_proc); #else - error =3D priv_check(td, PRIV_VM_MLOCK); - if (error) - return (error); + if (!unprivileged_mlock) { + error =3D priv_check(td, PRIV_VM_MLOCK); + if (error) + return (error); + } #endif #ifdef RACCT PROC_LOCK(td->td_proc); @@ -1174,9 +1182,11 @@ sys_munlockall(td, uap) int error; =20 map =3D &td->td_proc->p_vmspace->vm_map; - error =3D priv_check(td, PRIV_VM_MUNLOCK); - if (error) - return (error); + if (!unprivileged_mlock) { + error =3D priv_check(td, PRIV_VM_MUNLOCK); + if (error) + return (error); + } =20 /* Clear the MAP_WIREFUTURE flag from this vm_map. */ vm_map_lock(map); @@ -1215,9 +1225,11 @@ sys_munlock(td, uap) vm_size_t size; int error; =20 - error =3D priv_check(td, PRIV_VM_MUNLOCK); - if (error) - return (error); + if (!unprivileged_mlock) { + error =3D priv_check(td, PRIV_VM_MUNLOCK); + if (error) + return (error); + } addr =3D (vm_offset_t)uap->addr; size =3D uap->len; last =3D addr + size; Index: lib/libc/sys/mlockall.2 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- lib/libc/sys/mlockall.2 (revision 239772) +++ lib/libc/sys/mlockall.2 (working copy) @@ -72,7 +72,9 @@ limit and the per-process .Dv RLIMIT_MEMLOCK resource limit. .Pp -These calls are only available to the super-user. +These calls are only available to the super-user, or to anyone when +.Va security.bsd.unprivileged_mlock +is set to 1. .Pp The .Fn munlockall Index: lib/libc/sys/mlock.2 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- lib/libc/sys/mlock.2 (revision 239772) +++ lib/libc/sys/mlock.2 (working copy) @@ -99,7 +99,9 @@ the per-process .Li RLIMIT_MEMLOCK resource limit. .Pp -These calls are only available to the super-user. +These calls are only available to the super-user, or to anyone when +.Va security.bsd.unprivileged_mlock +is set to 1. .Sh RETURN VALUES .Rv -std .Pp @@ -112,7 +114,9 @@ system call will fail if: .Bl -tag -width Er .It Bq Er EPERM -The caller is not the super-user. +The caller is not the super-user and +.Va security.bsd.unprivileged_mlock +is set to 0. .It Bq Er EINVAL The address given is not page aligned or the length is negative. .It Bq Er EAGAIN @@ -129,7 +133,9 @@ system call will fail if: .Bl -tag -width Er .It Bq Er EPERM -The caller is not the super-user. +The caller is not the super-user and +.Va security.bsd.unprivileged_mlock +is set to 0. .It Bq Er EINVAL The address given is not page aligned or the length is negative. .It Bq Er ENOMEM --------------080705060908010301070308-- --------------enig936FD783B61C3D5F55FA814F Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.18 (Darwin) Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJQPPO2AAoJEBWLemxX/CvTMywIALbazyTRxENDi+KF1JZZHPEs brnI9G6kxNqKPRwP826xAdWgv/5BplizafsPUcPiyXj8OjM9NmP32gjJ8OrW2qqw 5V7Wy+pkgrPy++g43wSH//+JmTDjNlVoAH9c4dRRgVZD8sBz3zt44xTSVK657zRo w58Dpqajf4RPGysavD7W9rV0L96QmN5BvfgkPrzg1w/ykkCrJXvOgx7NRz7ZbRx8 gmT5P9puDk9JA1Kt/axdzV3wAFYcmVG0WyT8gDvMgsiGi7QC9J4k4bvC/HHl5TpJ 6YqkfptanxZIwn+XagQj+z4GPBQ5dA3Piu4zNGez149uqPBpzq9Q5CN03VS5xDM= =XUdc -----END PGP SIGNATURE----- --------------enig936FD783B61C3D5F55FA814F-- From owner-freebsd-arch@FreeBSD.ORG Tue Aug 28 18:07:28 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 189B31065673 for ; Tue, 28 Aug 2012 18:07:28 +0000 (UTC) (envelope-from bryan@shatow.net) Received: from secure.xzibition.com (secure.xzibition.com [173.160.118.92]) by mx1.freebsd.org (Postfix) with ESMTP id A16368FC1E for ; Tue, 28 Aug 2012 18:07:27 +0000 (UTC) DomainKey-Signature: a=rsa-sha1; c=nofws; d=shatow.net; h=message-id :date:from:mime-version:to:cc:subject:references:in-reply-to :content-type:content-transfer-encoding; q=dns; s=sweb; b=Foat9a +CRgsdxyU3CsjMlpQl14M4lY0G+NI7aXJ3vHrkrBFe95OJog3SgmuOjbzmEq6KZZ Z9B0AHvu8xQEEybuuf4JumnKSKFYX2rxGRyA+2WBJVsverL37+f4daGzjcog0XZp nxkwYgSCmPCkh6t5o2vBl6p9Q6v2lc9RIqw/s= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=shatow.net; h=message-id :date:from:mime-version:to:cc:subject:references:in-reply-to :content-type:content-transfer-encoding; s=sweb; bh=Th7Q3+Bn+TfM 5Ct9eEj8Nh58yZTpstCrxTea3gju8wQ=; b=O6XOYzMRMUjppBFzmFzSbgCDKe04 yUNN/d7EP3jHRKhwkyrte861nmF0+6eAFGmElfrSQUDWKb3Yo6ZR+OjFZ++CMMp8 Jg/+VSF/Ox4UQizcYu2WODN0IECuZy1DNVcEl18enJNwoCuv8GYWRmtPuiq8hr3I DziR7CCCMKfBFLE= Received: (qmail 96059 invoked from network); 28 Aug 2012 13:07:20 -0500 Received: from unknown (HELO ?192.168.0.74?) (bryan@shatow.net@74.94.87.209) by sweb.xzibition.com with ESMTPA; 28 Aug 2012 13:07:20 -0500 Message-ID: <503D08D6.1040004@shatow.net> Date: Tue, 28 Aug 2012 13:07:18 -0500 From: Bryan Drewery User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:14.0) Gecko/20120713 Thunderbird/14.0 MIME-Version: 1.0 To: Andrey Zonov References: <503CF3B1.3050604@FreeBSD.org> In-Reply-To: <503CF3B1.3050604@FreeBSD.org> X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-arch@freebsd.org Subject: Re: [patch] unprivileged mlock(2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2012 18:07:28 -0000 On 8/28/2012 11:37 AM, Andrey Zonov wrote: > Hi, > > We've got RLIMIT_MEMLOCK for years, but this limit is useless, because > only root may call mlock(2), and root may raise any limits. > > I suggest patch that allows to call mlock(2) for unprivileged users. > Are there any objections to got it in tree? > FYI, see previous recent thread on this here: http://lists.freebsd.org/pipermail/freebsd-arch/2012-May/012552.html and http://lists.freebsd.org/pipermail/freebsd-arch/2012-June/012606.html Bryan From owner-freebsd-arch@FreeBSD.ORG Tue Aug 28 20:20:47 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AD897106564A; Tue, 28 Aug 2012 20:20:47 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 87FD58FC12; Tue, 28 Aug 2012 20:20:45 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id XAA26585; Tue, 28 Aug 2012 23:20:43 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1T6SHD-000LEK-Fk; Tue, 28 Aug 2012 23:20:43 +0300 Message-ID: <503D281A.3080107@FreeBSD.org> Date: Tue, 28 Aug 2012 23:20:42 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120728 Thunderbird/14.0 MIME-Version: 1.0 To: Andrey Zonov References: <503CF3B1.3050604@FreeBSD.org> <503D08D6.1040004@shatow.net> In-Reply-To: <503D08D6.1040004@shatow.net> X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-arch@FreeBSD.org, Bryan Drewery Subject: Re: [patch] unprivileged mlock(2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2012 20:20:47 -0000 on 28/08/2012 21:07 Bryan Drewery said the following: > On 8/28/2012 11:37 AM, Andrey Zonov wrote: >> Hi, >> >> We've got RLIMIT_MEMLOCK for years, but this limit is useless, because >> only root may call mlock(2), and root may raise any limits. >> >> I suggest patch that allows to call mlock(2) for unprivileged users. >> Are there any objections to got it in tree? >> > > FYI, see previous recent thread on this here: > > http://lists.freebsd.org/pipermail/freebsd-arch/2012-May/012552.html > and > http://lists.freebsd.org/pipermail/freebsd-arch/2012-June/012606.html Yes, Andrey, I highly suggest that you read those threads completely. Here are some observations. It doesn't look like mlockall and mlockall(MCL_FUTURE) in particular properly honor RLIMIT_MEMLOCK. If this is not fixed, then it would be premature to enable the privilege for non-privileged users. I am against adding the sysctl knob. If RLIMIT_MEMLOCK limit is properly implemented then it is sufficient to effectively deny the privilege (and with much finer granularity). When the privilege is allowed to ordinary users, then memorylocked in the default login.conf would need to be set to something much lower than the current 'unlimited' :-) Also, note that currently RLIMIT_MEMLOCK is abused at least in vslock() (used at least by sysctl's kernel side). The temporary wirings performed as an implementation detail or side-effect should not be accounted against the limit. The limit is for wirings that a user makes and controls explicitly. It should not be applied to something that kernel does behind the scenes without user's knowledge. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Tue Aug 28 21:15:14 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A78921065670 for ; Tue, 28 Aug 2012 21:15:13 +0000 (UTC) (envelope-from andrey@zonov.org) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id 1B6E78FC1A for ; Tue, 28 Aug 2012 21:15:12 +0000 (UTC) Received: by lbbgg13 with SMTP id gg13so4222508lbb.13 for ; Tue, 28 Aug 2012 14:15:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :x-gm-message-state; bh=iWLP41KmxZx4le0qRpWQfKHPgJevyUI29xpEsPCElFk=; b=d/TTU3g8DcWnptaJvoSK7SKEKnlctmfmh6jb1ubINM19xTqdhFbh9huq4qCg8EotT3 xoCvBj0SAGjbf7/8gaoKexix0EiQNg1IiFk7unIkNYYucg/uqXkMOOUabGk+GhOQsYrG hDKe//nMqu5Z3c1tQBOYoP/qglSsg8SN22BorTThFPISuL2ZuthJcdnJjzmBYS6VZg56 KWYnas7H1ZCbHJns0A2NxxZcprqgOjZoawSHtKzaMToQtuQCjUKp+ZfQYCDly/3O7kCF 9VLD7IYH87a3b+JZKizyr3TYJlc3SRf2hZV0kbU+ezpoT2fxFqTFaA7LQOdSINygoe+J QDKA== Received: by 10.112.85.35 with SMTP id e3mr8682899lbz.90.1346188511906; Tue, 28 Aug 2012 14:15:11 -0700 (PDT) Received: from zont-osx.local (ppp95-165-143-86.pppoe.spdop.ru. [95.165.143.86]) by mx.google.com with ESMTPS id lr17sm23118377lab.12.2012.08.28.14.15.10 (version=SSLv3 cipher=OTHER); Tue, 28 Aug 2012 14:15:11 -0700 (PDT) Sender: Andrey Zonov Message-ID: <503D34DB.3090000@FreeBSD.org> Date: Wed, 29 Aug 2012 01:15:07 +0400 From: Andrey Zonov User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:14.0) Gecko/20120713 Thunderbird/14.0 MIME-Version: 1.0 To: Andriy Gapon References: <503CF3B1.3050604@FreeBSD.org> <503D08D6.1040004@shatow.net> <503D281A.3080107@FreeBSD.org> In-Reply-To: <503D281A.3080107@FreeBSD.org> X-Enigmail-Version: 1.4.3 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig494F7A79CA0D729A78704F40" X-Gm-Message-State: ALoCoQkOJXF/a16QFrilEMht/0+4LAZAdATL4vGXnM6FAP1jvIxZzWdGf6rMKiD/Oh25GviOx11X Cc: freebsd-arch@FreeBSD.org, Bryan Drewery Subject: Re: [patch] unprivileged mlock(2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2012 21:15:14 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig494F7A79CA0D729A78704F40 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 8/29/12 12:20 AM, Andriy Gapon wrote: > on 28/08/2012 21:07 Bryan Drewery said the following: >> On 8/28/2012 11:37 AM, Andrey Zonov wrote: >>> Hi, >>> >>> We've got RLIMIT_MEMLOCK for years, but this limit is useless, becaus= e >>> only root may call mlock(2), and root may raise any limits. >>> >>> I suggest patch that allows to call mlock(2) for unprivileged users. >>> Are there any objections to got it in tree? >>> >> >> FYI, see previous recent thread on this here: >> >> http://lists.freebsd.org/pipermail/freebsd-arch/2012-May/012552.html >> and >> http://lists.freebsd.org/pipermail/freebsd-arch/2012-June/012606.html >=20 > Yes, Andrey, I highly suggest that you read those threads completely. >=20 > Here are some observations. >=20 > It doesn't look like mlockall and mlockall(MCL_FUTURE) in particular pr= operly > honor RLIMIT_MEMLOCK. If this is not fixed, then it would be premature= to > enable the privilege for non-privileged users. >=20 This should be surely fixed, but I don't know how. Any suggestions are welcome. > I am against adding the sysctl knob. If RLIMIT_MEMLOCK limit is proper= ly > implemented then it is sufficient to effectively deny the privilege (an= d with > much finer granularity). >=20 Until all bugs around this problem will be fixed, to have such sysctl would be nice, and even keep it turned off to not change default behavior (not like in patch). > When the privilege is allowed to ordinary users, then memorylocked in t= he > default login.conf would need to be set to something much lower than th= e current > 'unlimited' :-) >=20 It's not a problem to set it to a new reasonable value in the tree, but it will be a problem to fix this everywhere. > Also, note that currently RLIMIT_MEMLOCK is abused at least in vslock()= (used at > least by sysctl's kernel side). The temporary wirings performed as an > implementation detail or side-effect should not be accounted against th= e limit. > The limit is for wirings that a user makes and controls explicitly. I= t should > not be applied to something that kernel does behind the scenes without = user's > knowledge. >=20 I was surprised when I stepped on this few years ago on machine with thousands processes. top(8) ate 100% CPU in a forever loop, ps(1) didn't work, and that is because memorylocked limit was set to low. Later I submitted two patches which fixed kvm (r230873) and sockstat (r230874), but I totally agree with you here, we shouldn't check for limits in vslock(). Index: sys/vm/vm_glue.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- sys/vm/vm_glue.c (revision 239772) +++ sys/vm/vm_glue.c (working copy) @@ -183,7 +183,6 @@ int vslock(void *addr, size_t len) { vm_offset_t end, last, start; - unsigned long nsize; vm_size_t npages; int error; @@ -195,18 +194,6 @@ vslock(void *addr, size_t len) npages =3D atop(end - start); if (npages > vm_page_max_wired) return (ENOMEM); - PROC_LOCK(curproc); - nsize =3D ptoa(npages + - pmap_wired_count(vm_map_pmap(&curproc->p_vmspace->vm_map))); - if (nsize > lim_cur(curproc, RLIMIT_MEMLOCK)) { - PROC_UNLOCK(curproc); - return (ENOMEM); - } - if (racct_set(curproc, RACCT_MEMLOCK, nsize)) { - PROC_UNLOCK(curproc); - return (ENOMEM); - } - PROC_UNLOCK(curproc); #if 0 /* * XXX - not yet @@ -222,14 +209,6 @@ vslock(void *addr, size_t len) #endif error =3D vm_map_wire(&curproc->p_vmspace->vm_map, start, end, VM_MAP_WIRE_SYSTEM | VM_MAP_WIRE_NOHOLES); -#ifdef RACCT - if (error !=3D KERN_SUCCESS) { - PROC_LOCK(curproc); - racct_set(curproc, RACCT_MEMLOCK, - ptoa(pmap_wired_count(vm_map_pmap(&curproc->p_vmspace->vm_map))));= - PROC_UNLOCK(curproc); - } -#endif /* * Return EFAULT on error to match copy{in,out}() behaviour * rather than returning ENOMEM like mlock() would. --=20 Andrey Zonov --------------enig494F7A79CA0D729A78704F40 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.18 (Darwin) Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJQPTTdAAoJEBWLemxX/CvTM8UH/2S0ihXw3K0esXwUHvIRZUX3 TiF3eqUGfXj+HZtufuZDBpwctrxT1q4h3GZLjgH3fXJn2p8C+MlJCCItL9pemUA1 RYLkKzzKiYIXN7ssDNJbEjetFYHmSb/RmXCveEkuCE8ihOFPfHn3dTY9KdRwE5Fx QinPGrj+U7td2auZadkNwGZxe5O6LDqPN5pc9gLdCD5t5Wz00lvgjOSBHZZmJ9T+ 3aS/xYtKX3MwKksRzX/HxtG9oaAy7885fJ1oAKV3cULf7u7mNyjJ34VZIEpt9kyU pnJNcObywwQkpqLIG61PSJ1EOx7ZKX5IeKMRkL6cz8Td34kRcay3Pavqk6URzxo= =VfND -----END PGP SIGNATURE----- --------------enig494F7A79CA0D729A78704F40-- From owner-freebsd-arch@FreeBSD.ORG Wed Aug 29 00:13:59 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 702BE106564A; Wed, 29 Aug 2012 00:13:59 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 369978FC1A; Wed, 29 Aug 2012 00:13:58 +0000 (UTC) Received: by dadr6 with SMTP id r6so3690837dad.13 for ; Tue, 28 Aug 2012 17:13:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=FTOL9enOulzESx3HzAUNJZmrzfbS0l0UbgMhPxpaTu8=; b=ecbru9QO5TYf74auFBfehelitMgVt4PZGFTEsadKMpNoQmIZhlKXoeZb9ejIyNF/yL Zj46/wtByEI/zrwVlR3FuL+eCJ7eHQOCepgvhpi0I5uc0x5pN0Zo4ULl+mDP3Paj/U0c IwdR7riBwM19sj3NiUewR0Wv0xnqBG8SPFTDW6uCvk+N8P/bHHw5DjpX792sz8J6+6Ff xAQ+399F5nj35K+GHiHBr09SoQ2mqJsvzmCClvRZbSa4s4v66JWJSX5ps0tU8UbWC9kz y2JYJbW57kpmp5c+5CBdC9b/rvp9nQmMwZOvKxTCw6hAqcEa0jzT2eTIdAmLluFR1rOZ wlFA== MIME-Version: 1.0 Received: by 10.68.236.102 with SMTP id ut6mr398584pbc.113.1346199238677; Tue, 28 Aug 2012 17:13:58 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.68.36.106 with HTTP; Tue, 28 Aug 2012 17:13:58 -0700 (PDT) In-Reply-To: References: <15285562-E9BA-431B-A2C1-D0547DFB2663@bsdimp.com> <201201030924.44493.jhb@freebsd.org> Date: Tue, 28 Aug 2012 17:13:58 -0700 X-Google-Sender-Auth: 0AsSXsv1kjFUvEa2_teLFHYXWg0 Message-ID: From: Adrian Chadd To: Warner Losh Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-arch@freebsd.org Subject: Re: Request for help: how do teach module building about kernel options? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Aug 2012 00:13:59 -0000 On 3 January 2012 15:31, Warner Losh wrote: > > On Jan 3, 2012, at 12:06 PM, Adrian Chadd wrote: > >> On 3 January 2012 10:56, Warner Losh wrote: >> >>>> So how about we do up say, the KERNOPTS field first, which would be a >>>> big win. Then KERNDEVICES too, if that's possible? >>> >>> I'd only planned on doing KERNOPTS :) >> >> KERNOPTS would be fine for now. That at least gets me out of my >> initial issues with wlan/ath building and the fun in handling options. > > Please find enclosed a proposed patch. This will not build, of course, since there's no file in the tree mesh_baby.c, so if you have IEEE80211_SUPPORT_MESH defined, it won't work. It keys off the actual define. > > It even works with devices (which define DEV_xxx), but only if you have it in the options file like isa (eg, you need to tell config to generate it). > > Comments? Hiya, I'm now facing this particular issue again for embedded builds. I now understand what you're describing. I wonder though if it's worth doing a slightly larger scale re-architecture. Ie: * teach configure to write out opt_.mk, which sets up the makefile options the same as the .h file options; * split kmod.mk into kmod-pre.mk and kmod-post.mk; * teach kmod-pre.mk to walk a list (say, OPT_MKSRCS?) and link them in as needed, as well as linking in the opt_.mk file as well; * .. then have it include the contents of each one as needed. That way kmod-pre.mk could pre-populate the options early on. If patching config is scary, we could do what you've suggested in kmod-pre.mk for now; just as long as I can populate another list (say OPT_SRCS) of opt_*.h files for it to extract options out of. What do you think? Adrian > > Warner > > Index: conf/kmod.mk > =================================================================== > --- conf/kmod.mk (revision 229436) > +++ conf/kmod.mk (working copy) > @@ -324,7 +324,16 @@ > ${_src}: > ln -sf ${KERNBUILDDIR}/${_src} ${.TARGET} > .endif > +.if exists(${KERNBUILDDIR}/${_src}) > +_opts!=cut -d' ' -f 2 ${KERNBUILDDIR}/${_src} > +.for _o in ${_opts} > +KERNOPT_${_o}=1 > +.if defined(SRCS_${_o}) > +SRCS+=${SRCS_${_o}} > +.endif > .endfor > +.endif > +.endfor > .else > .for _src in ${SRCS:Mopt_*.h} > CLEANFILES+= ${_src} > Index: modules/ath/Makefile > =================================================================== > --- modules/ath/Makefile (revision 229436) > +++ modules/ath/Makefile (working copy) > @@ -41,6 +41,7 @@ > SRCS+= ah_osdep.c ah.c ah_regdomain.c ah_eeprom_v3.c > SRCS+= device_if.h bus_if.h pci_if.h opt_inet.h opt_ath.h opt_ah.h opt_wlan.h > > +SRCS_IEEE80211_SUPPORT_MESH=mesh_baby.c > # > # AR5210 support; these are first generation 11a-only devices. > # > From owner-freebsd-arch@FreeBSD.ORG Wed Aug 29 04:39:12 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0708F1065686; Wed, 29 Aug 2012 04:39:12 +0000 (UTC) (envelope-from alan.l.cox@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 9F14F8FC14; Wed, 29 Aug 2012 04:39:11 +0000 (UTC) Received: by ialo14 with SMTP id o14so396045ial.13 for ; Tue, 28 Aug 2012 21:39:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=N+jR5z5xlIiHFBXQaQCpiRwml3lAhhhbLl8Fk9SsuOc=; b=O2zLp2Z+iTs3P+tJftisCJMkG95Nv4S2faHRKU06k+pnBbcHPjIIUZ6g2k0y//gxkl HeHGr3jfO3Ao+3GfgC4EJJ1zP0I4Ti/TnFZQw3iGRTTAcGG4zzexWLDFGZkvBD/PwagY vefK8m7jvhX/8ethttMHYKvp85JAFvrMGiMvg+u/hY3raDcZq7lcvQDbl6WJgkBtpBvx dZ1n1QITrhX6f1gHJvdbl2SC9L+h/WoNeL8djrWYEtSjD9sHVkOcGD6GWYbf0SHAZYLr kxkMAM9VtNWoBZFB/Rrj+5+b/o4Of+4AYjQ7z95aDX5RjR1WSv6gdzLWI47n7uIBHCEZ RJMg== MIME-Version: 1.0 Received: by 10.50.47.200 with SMTP id f8mr319047ign.6.1346215151078; Tue, 28 Aug 2012 21:39:11 -0700 (PDT) Received: by 10.64.72.165 with HTTP; Tue, 28 Aug 2012 21:39:11 -0700 (PDT) In-Reply-To: <503D34DB.3090000@FreeBSD.org> References: <503CF3B1.3050604@FreeBSD.org> <503D08D6.1040004@shatow.net> <503D281A.3080107@FreeBSD.org> <503D34DB.3090000@FreeBSD.org> Date: Tue, 28 Aug 2012 23:39:11 -0500 Message-ID: From: Alan Cox To: Andrey Zonov Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Bryan Drewery , Andriy Gapon , freebsd-arch@freebsd.org Subject: Re: [patch] unprivileged mlock(2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: alc@freebsd.org List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Aug 2012 04:39:12 -0000 On Tue, Aug 28, 2012 at 4:15 PM, Andrey Zonov wrote: > On 8/29/12 12:20 AM, Andriy Gapon wrote: > > on 28/08/2012 21:07 Bryan Drewery said the following: > >> On 8/28/2012 11:37 AM, Andrey Zonov wrote: > >>> Hi, > >>> > >>> We've got RLIMIT_MEMLOCK for years, but this limit is useless, because > >>> only root may call mlock(2), and root may raise any limits. > >>> > >>> I suggest patch that allows to call mlock(2) for unprivileged users. > >>> Are there any objections to got it in tree? > >>> > >> > >> FYI, see previous recent thread on this here: > >> > >> http://lists.freebsd.org/pipermail/freebsd-arch/2012-May/012552.html > >> and > >> http://lists.freebsd.org/pipermail/freebsd-arch/2012-June/012606.html > > > > Yes, Andrey, I highly suggest that you read those threads completely. > > > > Here are some observations. > > > > It doesn't look like mlockall and mlockall(MCL_FUTURE) in particular > properly > > honor RLIMIT_MEMLOCK. If this is not fixed, then it would be premature > to > > enable the privilege for non-privileged users. > > > > This should be surely fixed, but I don't know how. Any suggestions are > welcome. > > > I am against adding the sysctl knob. If RLIMIT_MEMLOCK limit is properly > > implemented then it is sufficient to effectively deny the privilege (and > with > > much finer granularity). > > > > Until all bugs around this problem will be fixed, to have such sysctl > would be nice, and even keep it turned off to not change default > behavior (not like in patch). > > > When the privilege is allowed to ordinary users, then memorylocked in the > > default login.conf would need to be set to something much lower than the > current > > 'unlimited' :-) > > > > It's not a problem to set it to a new reasonable value in the tree, but > it will be a problem to fix this everywhere. > > > Also, note that currently RLIMIT_MEMLOCK is abused at least in vslock() > (used at > > least by sysctl's kernel side). The temporary wirings performed as an > > implementation detail or side-effect should not be accounted against the > limit. > > The limit is for wirings that a user makes and controls explicitly. It > should > > not be applied to something that kernel does behind the scenes without > user's > > knowledge. > > > > I was surprised when I stepped on this few years ago on machine with > thousands processes. top(8) ate 100% CPU in a forever loop, ps(1) > didn't work, and that is because memorylocked limit was set to low. > Later I submitted two patches which fixed kvm (r230873) and sockstat > (r230874), but I totally agree with you here, we shouldn't check for > limits in vslock(). > > I agree with Andriy's argument for making the following change. Please go ahead and commit it. Alan Index: sys/vm/vm_glue.c > =================================================================== > --- sys/vm/vm_glue.c (revision 239772) > +++ sys/vm/vm_glue.c (working copy) > @@ -183,7 +183,6 @@ int > vslock(void *addr, size_t len) > { > vm_offset_t end, last, start; > - unsigned long nsize; > vm_size_t npages; > int error; > > @@ -195,18 +194,6 @@ vslock(void *addr, size_t len) > npages = atop(end - start); > if (npages > vm_page_max_wired) > return (ENOMEM); > - PROC_LOCK(curproc); > - nsize = ptoa(npages + > - pmap_wired_count(vm_map_pmap(&curproc->p_vmspace->vm_map))); > - if (nsize > lim_cur(curproc, RLIMIT_MEMLOCK)) { > - PROC_UNLOCK(curproc); > - return (ENOMEM); > - } > - if (racct_set(curproc, RACCT_MEMLOCK, nsize)) { > - PROC_UNLOCK(curproc); > - return (ENOMEM); > - } > - PROC_UNLOCK(curproc); > #if 0 > /* > * XXX - not yet > @@ -222,14 +209,6 @@ vslock(void *addr, size_t len) > #endif > error = vm_map_wire(&curproc->p_vmspace->vm_map, start, end, > VM_MAP_WIRE_SYSTEM | VM_MAP_WIRE_NOHOLES); > -#ifdef RACCT > - if (error != KERN_SUCCESS) { > - PROC_LOCK(curproc); > - racct_set(curproc, RACCT_MEMLOCK, > - > ptoa(pmap_wired_count(vm_map_pmap(&curproc->p_vmspace->vm_map)))); > - PROC_UNLOCK(curproc); > - } > -#endif > /* > * Return EFAULT on error to match copy{in,out}() behaviour > * rather than returning ENOMEM like mlock() would. > > -- > Andrey Zonov > > From owner-freebsd-arch@FreeBSD.ORG Wed Aug 29 08:35:04 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1F5F7106573F for ; Wed, 29 Aug 2012 08:35:04 +0000 (UTC) (envelope-from andrey@zonov.org) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 77A498FC19 for ; Wed, 29 Aug 2012 08:35:03 +0000 (UTC) Received: by eaak11 with SMTP id k11so92848eaa.13 for ; Wed, 29 Aug 2012 01:35:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :x-gm-message-state; bh=eALbDMM4+prONm4e9+Kz41jtRVv4DnEnNHRJVXvhzwE=; b=Sjsi7JbF5jbG9evLaJRVWfDhj2oJNXR6aSSSU+FGC6d3NTL0dMFLxPTbZQpNFayMvm m+yVaqL6piJWhvOpTF5RfoLfWMZuVOFfS6zKdDT2YdzlTCeuNcv2yODfwlsvc3soklND JMP+3R6pVGosNo53j7cFh6rgoCt3j9PizDCqVuVIGyS4anO3Rmn/gb1iOgGDcp5RzvCE 8r3gewj2AJ3N32ELQ8WBhAkvt2JX78xshZfYktPShqN5wjRwn6kCgTIgU7NIBF5kb94W HJuUOUJtdxRXmydrK3ZDAmT593aG4rjol3cpP4cfZnCANho2wYwUaIF8g4FRXnCATjWS ld4w== Received: by 10.14.182.134 with SMTP id o6mr940753eem.26.1346229302261; Wed, 29 Aug 2012 01:35:02 -0700 (PDT) Received: from zont-osx.local (ppp95-165-143-86.pppoe.spdop.ru. [95.165.143.86]) by mx.google.com with ESMTPS id z6sm67827743eeo.6.2012.08.29.01.35.00 (version=SSLv3 cipher=OTHER); Wed, 29 Aug 2012 01:35:01 -0700 (PDT) Sender: Andrey Zonov Message-ID: <503DD433.2030108@FreeBSD.org> Date: Wed, 29 Aug 2012 12:34:59 +0400 From: Andrey Zonov User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:14.0) Gecko/20120713 Thunderbird/14.0 MIME-Version: 1.0 To: alc@freebsd.org References: <503CF3B1.3050604@FreeBSD.org> <503D08D6.1040004@shatow.net> <503D281A.3080107@FreeBSD.org> <503D34DB.3090000@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.4.3 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigE2B25C00700D66F00BB7EB91" X-Gm-Message-State: ALoCoQkgEtXxtqThzxaNFW5gJSTW4nnKTC0AkD3rCGfyCwRhM+4RdhamNUl/DqYv3a13TqSZUN2N Cc: Bryan Drewery , Alan Cox , Andriy Gapon , freebsd-arch@freebsd.org Subject: Re: [patch] unprivileged mlock(2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Aug 2012 08:35:04 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigE2B25C00700D66F00BB7EB91 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 8/29/12 8:39 AM, Alan Cox wrote: > On Tue, Aug 28, 2012 at 4:15 PM, Andrey Zonov wrote:= >=20 >> On 8/29/12 12:20 AM, Andriy Gapon wrote: >>> on 28/08/2012 21:07 Bryan Drewery said the following: >>>> On 8/28/2012 11:37 AM, Andrey Zonov wrote: >>>>> Hi, >>>>> >>>>> We've got RLIMIT_MEMLOCK for years, but this limit is useless, beca= use >>>>> only root may call mlock(2), and root may raise any limits. >>>>> >>>>> I suggest patch that allows to call mlock(2) for unprivileged users= =2E >>>>> Are there any objections to got it in tree? >>>>> >>>> >>>> FYI, see previous recent thread on this here: >>>> >>>> http://lists.freebsd.org/pipermail/freebsd-arch/2012-May/012552.html= >>>> and >>>> http://lists.freebsd.org/pipermail/freebsd-arch/2012-June/012606.htm= l >>> >>> Yes, Andrey, I highly suggest that you read those threads completely.= >>> >>> Here are some observations. >>> >>> It doesn't look like mlockall and mlockall(MCL_FUTURE) in particular >> properly >>> honor RLIMIT_MEMLOCK. If this is not fixed, then it would be prematu= re >> to >>> enable the privilege for non-privileged users. >>> >> >> This should be surely fixed, but I don't know how. Any suggestions ar= e >> welcome. >> >>> I am against adding the sysctl knob. If RLIMIT_MEMLOCK limit is prop= erly >>> implemented then it is sufficient to effectively deny the privilege (= and >> with >>> much finer granularity). >>> >> >> Until all bugs around this problem will be fixed, to have such sysctl >> would be nice, and even keep it turned off to not change default >> behavior (not like in patch). >> >>> When the privilege is allowed to ordinary users, then memorylocked in= the >>> default login.conf would need to be set to something much lower than = the >> current >>> 'unlimited' :-) >>> >> >> It's not a problem to set it to a new reasonable value in the tree, bu= t >> it will be a problem to fix this everywhere. >> >>> Also, note that currently RLIMIT_MEMLOCK is abused at least in vslock= () >> (used at >>> least by sysctl's kernel side). The temporary wirings performed as a= n >>> implementation detail or side-effect should not be accounted against = the >> limit. >>> The limit is for wirings that a user makes and controls explicitly. = It >> should >>> not be applied to something that kernel does behind the scenes withou= t >> user's >>> knowledge. >>> >> >> I was surprised when I stepped on this few years ago on machine with >> thousands processes. top(8) ate 100% CPU in a forever loop, ps(1) >> didn't work, and that is because memorylocked limit was set to low. >> Later I submitted two patches which fixed kvm (r230873) and sockstat >> (r230874), but I totally agree with you here, we shouldn't check for >> limits in vslock(). >> >> > I agree with Andriy's argument for making the following change. Please= go > ahead and commit it. >=20 Thanks, I will commit it after approving from kib. But can we do better and don't lock process's memory in sysctl handlers? --=20 Andrey Zonov --------------enigE2B25C00700D66F00BB7EB91 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.18 (Darwin) Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJQPdQzAAoJEBWLemxX/CvTvJoIAI/rRlQIW7hjQYN2Bqt8lD2L ohIi67fCq8lsyGkbqBleFJCGTRpjSGKpQMT6ZD1cTjHi8WkzmJQMRO2sn7m2dBcI uCYs/LNHVf4xnNukeANGJ838jdjWPIrlEfnrh3CiPB8/BCUu05X7vKh1c6np9E7+ hcD8pJGl568Jg3Z2l+SBenbv7c6acWd4tBu2xHtnnz5x2Ly7nSZjbJ553ZbQKXlQ THH2w2GuRc4B1JYhhGWuU5n7t5W4UOWMzQUZnG4YayUTEowcqz6+4vcuEsqEJ2zd 4jwqm7G5eXxaHJ7yCPs6MPrd9JRMKYaGB2SBzAzFo95bdLkuyQCAhNoEGlsybvE= =GQxu -----END PGP SIGNATURE----- --------------enigE2B25C00700D66F00BB7EB91-- From owner-freebsd-arch@FreeBSD.ORG Wed Aug 29 09:06:30 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1F7F4106566B; Wed, 29 Aug 2012 09:06:30 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from gw.catspoiler.org (gw.catspoiler.org [75.1.14.242]) by mx1.freebsd.org (Postfix) with ESMTP id DC1308FC0C; Wed, 29 Aug 2012 09:06:29 +0000 (UTC) Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.13.3/8.13.3) with ESMTP id q7T96C9j032802; Wed, 29 Aug 2012 02:06:16 -0700 (PDT) (envelope-from truckman@FreeBSD.org) Message-Id: <201208290906.q7T96C9j032802@gw.catspoiler.org> Date: Wed, 29 Aug 2012 02:06:12 -0700 (PDT) From: Don Lewis To: zont@FreeBSD.org In-Reply-To: <503DD433.2030108@FreeBSD.org> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Cc: alc@FreeBSD.org, alan.l.cox@gmail.com, freebsd-arch@FreeBSD.org, avg@FreeBSD.org, bryan@shatow.net Subject: Re: [patch] unprivileged mlock(2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Aug 2012 09:06:30 -0000 On 29 Aug, Andrey Zonov wrote: > But can we do better and don't lock process's memory in sysctl handlers? It is necessary to wire the userland buffer to make sure that the sysctl handler doesn't page fault while it is copying data into or out of the buffer. The handler may be holding a mutex that protects the kernel data structure that it is accessing, in which case the handler is not allowed to block on a page fault. If the handler allocated a buffer in kernel memory before grabbing the mutex, the it would be possible to avoid wiring the userland buffer, but this would tie up just as much non-pageable memory and an extra data copy would be required. From owner-freebsd-arch@FreeBSD.ORG Wed Aug 29 09:23:26 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5BA281065676; Wed, 29 Aug 2012 09:23:26 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id C42498FC08; Wed, 29 Aug 2012 09:23:25 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q7T9NU4Q016140; Wed, 29 Aug 2012 12:23:30 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q7T9NIoK071714; Wed, 29 Aug 2012 12:23:18 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q7T9NIGa071713; Wed, 29 Aug 2012 12:23:18 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 29 Aug 2012 12:23:18 +0300 From: Konstantin Belousov To: Don Lewis Message-ID: <20120829092318.GW33100@deviant.kiev.zoral.com.ua> References: <503DD433.2030108@FreeBSD.org> <201208290906.q7T96C9j032802@gw.catspoiler.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="DGJeRcv3Pogw2aF5" Content-Disposition: inline In-Reply-To: <201208290906.q7T96C9j032802@gw.catspoiler.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: alc@freebsd.org, alan.l.cox@gmail.com, avg@freebsd.org, freebsd-arch@freebsd.org, zont@freebsd.org, bryan@shatow.net Subject: Re: [patch] unprivileged mlock(2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Aug 2012 09:23:26 -0000 --DGJeRcv3Pogw2aF5 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 29, 2012 at 02:06:12AM -0700, Don Lewis wrote: > On 29 Aug, Andrey Zonov wrote: >=20 > > But can we do better and don't lock process's memory in sysctl handlers? >=20 > It is necessary to wire the userland buffer to make sure that the sysctl > handler doesn't page fault while it is copying data into or out of the > buffer. The handler may be holding a mutex that protects the kernel > data structure that it is accessing, in which case the handler is not > allowed to block on a page fault. >=20 > If the handler allocated a buffer in kernel memory before grabbing the > mutex, the it would be possible to avoid wiring the userland buffer, but > this would tie up just as much non-pageable memory and an extra data > copy would be required. At least it would not cause a fragmentation of the user map. Another approach could be to use vm_fault_quick_hold_pages() and then use uiomove_fromphys() to copyin/out data inside the handler. This is what currently done by FFS and NFS. --DGJeRcv3Pogw2aF5 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlA934UACgkQC3+MBN1Mb4h1QQCgoAz5M0P2Vn1TP/8B99+y3++G 6+QAoLy75FjsJk1QLTKDTfwgFo7Ot1vS =ujrs -----END PGP SIGNATURE----- --DGJeRcv3Pogw2aF5-- From owner-freebsd-arch@FreeBSD.ORG Wed Aug 29 12:25:46 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 37B45106566C; Wed, 29 Aug 2012 12:25:46 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 0C5788FC08; Wed, 29 Aug 2012 12:25:46 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 60DD1B91A; Wed, 29 Aug 2012 08:25:45 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Date: Wed, 29 Aug 2012 07:41:59 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p17; KDE/4.5.5; amd64; ; ) References: <503CF3B1.3050604@FreeBSD.org> In-Reply-To: <503CF3B1.3050604@FreeBSD.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201208290741.59143.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 29 Aug 2012 08:25:45 -0400 (EDT) Cc: Andrey Zonov , Robert Watson Subject: Re: [patch] unprivileged mlock(2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Aug 2012 12:25:46 -0000 On Tuesday, August 28, 2012 12:37:05 pm Andrey Zonov wrote: > Hi, > > We've got RLIMIT_MEMLOCK for years, but this limit is useless, because > only root may call mlock(2), and root may raise any limits. > > I suggest patch that allows to call mlock(2) for unprivileged users. > Are there any objections to got it in tree? Aside from the other comments (e.g. needing to fix MCL_FUTURE to honor the limit), if you were to keep the unprivileged_mlock variable, I think the right place to patch this would be in kern_priv.c by adding a new check to grant PRIV_VM_MLOCK and PRIV_VM_MUNLOCK to all users if unprivileged_mlock is set. This centralizes the privilege checking logic instead of duplicating it in four different places. Robert may have a different opinion, however. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Aug 30 09:06:52 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B3E0B1065673 for ; Thu, 30 Aug 2012 09:06:52 +0000 (UTC) (envelope-from andrey@zonov.org) Received: from mail-lpp01m010-f54.google.com (mail-lpp01m010-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 176C98FC1D for ; Thu, 30 Aug 2012 09:06:51 +0000 (UTC) Received: by lage12 with SMTP id e12so1734180lag.13 for ; Thu, 30 Aug 2012 02:06:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:subject :references:in-reply-to:x-enigmail-version:content-type :x-gm-message-state; bh=G704ieSYsVu+fKeVf7/74Z7O6N2Cjnc6eU2ACntkF+8=; b=cuj6wZk7lBoI98K8ymcAMXG4LUOEPPusSH3nMZyXhHkhjQRH9f2yfJodj+TGy4dJWv Fz0HJDZnzabHZ1YojsMXwwZ9liDyjYL6agqbZr2Z2VZUd5QbMJ8snIzTgffdZIagP0qf 6j9WYL3CXsa87Y6U6z37UjFA44FNyZI+/cc6Y+H4uywcz7VnPdweGl+iNv/C4Jb/uPSI Pi78gPqW2dV5JpwClxbJ2xRMjOBpxVm0oUqsG1YAc7btp2l1eMzgu1vCckA6xowavop+ dDHz4XAFE4f+zr59B4Zfhyq9W+q43faUYuwbxZc8Aa0uk3TPY5hJiAQdMQG4UTQGiyBb O8RA== Received: by 10.112.31.231 with SMTP id d7mr1048231lbi.60.1346317609802; Thu, 30 Aug 2012 02:06:49 -0700 (PDT) Received: from zont-osx.local (ppp95-165-143-86.pppoe.spdop.ru. [95.165.143.86]) by mx.google.com with ESMTPS id y4sm328463lbg.5.2012.08.30.02.06.48 (version=SSLv3 cipher=OTHER); Thu, 30 Aug 2012 02:06:49 -0700 (PDT) Sender: Andrey Zonov Message-ID: <503F2D24.8050103@FreeBSD.org> Date: Thu, 30 Aug 2012 13:06:44 +0400 From: Andrey Zonov User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) Gecko/20120824 Thunderbird/15.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org References: <503DD433.2030108@FreeBSD.org> <201208290906.q7T96C9j032802@gw.catspoiler.org> <20120829092318.GW33100@deviant.kiev.zoral.com.ua> In-Reply-To: <20120829092318.GW33100@deviant.kiev.zoral.com.ua> X-Enigmail-Version: 1.4.4 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig5C2551C41E5B3832AFE69806" X-Gm-Message-State: ALoCoQk9DeuYzPVnpUmaEB6ohuGoS6CdaxNVlOzfW1g6oWmqxLW9R1toPVUcfPldatv56nAqhzR0 Subject: Re: [patch] unprivileged mlock(2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Aug 2012 09:06:52 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig5C2551C41E5B3832AFE69806 Content-Type: multipart/mixed; boundary="------------020105090900020700090502" This is a multi-part message in MIME format. --------------020105090900020700090502 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi, So, I've got the first version of the patch (attached) which fixes memory locked limit checking and accounting. I also want to change 'pmap_wired_count(vm_map_pmap(map))' with 'vmspace_wired_count(vmspace)', and apply the patch below separately: Index: sys/vm/vm_mmap.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- sys/vm/vm_mmap.c (revision 239611) +++ sys/vm/vm_mmap.c (working copy) @@ -1483,11 +1483,13 @@ vm_mmap(vm_map_t map, vm_offset_t *addr, vm_size_= t PROC_UNLOCK(td->td_proc); return (ENOMEM); } +#ifdef RACCT if (racct_set(td->td_proc, RACCT_VMEM, map->size + size)) { PROC_UNLOCK(td->td_proc); return (ENOMEM); } PROC_UNLOCK(td->td_proc); +#endif } /* It looks like this was missed in r220373. --=20 Andrey Zonov --------------020105090900020700090502 Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0"; name="mlock.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="mlock.patch" Index: sys/vm/vm_map.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- sys/vm/vm_map.c (revision 239772) +++ sys/vm/vm_map.c (working copy) @@ -3247,7 +3247,7 @@ vm_map_stack(vm_map_t map, vm_offset_t addrbos, vm vm_offset_t bot, top; vm_size_t init_ssize; int orient, rv; - rlim_t vmemlim; + rlim_t lmemlim, vmemlim; =20 /* * The stack orientation is piggybacked with the cow argument. @@ -3267,6 +3267,7 @@ vm_map_stack(vm_map_t map, vm_offset_t addrbos, vm init_ssize =3D (max_ssize < sgrowsiz) ? max_ssize : sgrowsiz; =20 PROC_LOCK(curthread->td_proc); + lmemlim =3D lim_cur(curthread->td_proc, RLIMIT_MEMLOCK); vmemlim =3D lim_cur(curthread->td_proc, RLIMIT_VMEM); PROC_UNLOCK(curthread->td_proc); =20 @@ -3278,6 +3279,14 @@ vm_map_stack(vm_map_t map, vm_offset_t addrbos, vm= return (KERN_NO_SPACE); } =20 + if (map->flags & MAP_WIREFUTURE) { + if (ptoa(vmspace_wired_count(curthread->td_proc->p_vmspace)) + + init_ssize > lmemlim) { + vm_map_unlock(map); + return (KERN_NO_SPACE); + } + } + /* If we would blow our VMEM resource limit, no go */ if (map->size + init_ssize > vmemlim) { vm_map_unlock(map); @@ -3358,7 +3367,7 @@ vm_map_growstack(struct proc *p, vm_offset_t addr) vm_map_t map =3D &vm->vm_map; vm_offset_t end; size_t grow_amount, max_grow; - rlim_t stacklim, vmemlim; + rlim_t lmemlim, stacklim, vmemlim; int is_procstack, rv; struct ucred *cred; #ifdef notyet @@ -3370,6 +3379,7 @@ vm_map_growstack(struct proc *p, vm_offset_t addr) =20 Retry: PROC_LOCK(p); + lmemlim =3D lim_cur(p, RLIMIT_MEMLOCK); stacklim =3D lim_cur(p, RLIMIT_STACK); vmemlim =3D lim_cur(p, RLIMIT_VMEM); PROC_UNLOCK(p); @@ -3491,7 +3501,25 @@ Retry: if (is_procstack && (ctob(vm->vm_ssize) + grow_amount > limit)) grow_amount =3D limit - ctob(vm->vm_ssize); #endif - + if (map->flags & MAP_WIREFUTURE) { + if (ptoa(vmspace_wired_count(p->p_vmspace)) + grow_amount > + lmemlim) { + vm_map_unlock_read(map); + rv =3D KERN_NO_SPACE; + goto out; + } +#ifdef RACCT + PROC_LOCK(p); + if (racct_set(p, RACCT_MEMLOCK, + ptoa(vmspace_wired_count(p->p_vmspace)) + grow_amount)) { + PROC_UNLOCK(p); + vm_map_unlock_read(map); + rv =3D KERN_NO_SPACE; + goto out; + } + PROC_UNLOCK(p); +#endif + } /* If we would blow our VMEM resource limit, no go */ if (map->size + grow_amount > vmemlim) { vm_map_unlock_read(map); Index: sys/vm/vm_mmap.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- sys/vm/vm_mmap.c (revision 239772) +++ sys/vm/vm_mmap.c (working copy) @@ -1035,9 +1035,6 @@ sys_mlock(td, uap) unsigned long nsize; int error; =20 - error =3D priv_check(td, PRIV_VM_MLOCK); - if (error) - return (error); addr =3D (vm_offset_t)uap->addr; size =3D uap->len; last =3D addr + size; @@ -1102,22 +1099,18 @@ sys_mlockall(td, uap) if ((uap->how =3D=3D 0) || ((uap->how & ~(MCL_CURRENT|MCL_FUTURE)) !=3D= 0)) return (EINVAL); =20 -#if 0 /* * If wiring all pages in the process would cause it to exceed * a hard resource limit, return ENOMEM. */ - PROC_LOCK(td->td_proc); - if (map->size > lim_cur(td->td_proc, RLIMIT_MEMLOCK)) { + if (uap->how & MCL_CURRENT) { + PROC_LOCK(td->td_proc); + if (map->size > lim_cur(td->td_proc, RLIMIT_MEMLOCK)) { + PROC_UNLOCK(td->td_proc); + return (ENOMEM); + } PROC_UNLOCK(td->td_proc); - return (ENOMEM); } - PROC_UNLOCK(td->td_proc); -#else - error =3D priv_check(td, PRIV_VM_MLOCK); - if (error) - return (error); -#endif #ifdef RACCT PROC_LOCK(td->td_proc); error =3D racct_set(td->td_proc, RACCT_MEMLOCK, map->size); @@ -1174,9 +1167,6 @@ sys_munlockall(td, uap) int error; =20 map =3D &td->td_proc->p_vmspace->vm_map; - error =3D priv_check(td, PRIV_VM_MUNLOCK); - if (error) - return (error); =20 /* Clear the MAP_WIREFUTURE flag from this vm_map. */ vm_map_lock(map); @@ -1215,9 +1205,6 @@ sys_munlock(td, uap) vm_size_t size; int error; =20 - error =3D priv_check(td, PRIV_VM_MUNLOCK); - if (error) - return (error); addr =3D (vm_offset_t)uap->addr; size =3D uap->len; last =3D addr + size; @@ -1479,14 +1466,32 @@ vm_mmap(vm_map_t map, vm_offset_t *addr, vm_size_= t =20 if (map =3D=3D &td->td_proc->p_vmspace->vm_map) { PROC_LOCK(td->td_proc); + if (map->flags & MAP_WIREFUTURE) { + if (ptoa(vmspace_wired_count(td->td_proc->p_vmspace)) + + size > lim_cur(td->td_proc, RLIMIT_MEMLOCK)) { + PROC_UNLOCK(td->td_proc); + return (ENOMEM); + } +#ifdef RACCT + error =3D racct_set(td->td_proc, RACCT_MEMLOCK, + ptoa(vmspace_wired_count(td->td_proc->p_vmspace)) + + size); + if (error !=3D 0) { + PROC_UNLOCK(td->td_proc); + return (error); + } +#endif + } if (map->size + size > lim_cur(td->td_proc, RLIMIT_VMEM)) { PROC_UNLOCK(td->td_proc); return (ENOMEM); } +#ifdef RACCT /* was missed in r220373 */ if (racct_set(td->td_proc, RACCT_VMEM, map->size + size)) { PROC_UNLOCK(td->td_proc); return (ENOMEM); } +#endif PROC_UNLOCK(td->td_proc); } =20 Index: sys/vm/vm_unix.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- sys/vm/vm_unix.c (revision 239772) +++ sys/vm/vm_unix.c (working copy) @@ -77,13 +77,14 @@ sys_obreak(td, uap) { struct vmspace *vm =3D td->td_proc->p_vmspace; vm_offset_t new, old, base; - rlim_t datalim, vmemlim; + rlim_t datalim, lmemlim, vmemlim; int prot, rv; int error =3D 0; boolean_t do_map_wirefuture; =20 PROC_LOCK(td->td_proc); datalim =3D lim_cur(td->td_proc, RLIMIT_DATA); + lmemlim =3D lim_cur(td->td_proc, RLIMIT_MEMLOCK); vmemlim =3D lim_cur(td->td_proc, RLIMIT_VMEM); PROC_UNLOCK(td->td_proc); =20 @@ -116,6 +117,13 @@ sys_obreak(td, uap) goto done; } if (new > old) { + if (vm->vm_map.flags & MAP_WIREFUTURE) { + if (ptoa(vmspace_wired_count(td->td_proc->p_vmspace)) + + (new - old) > lmemlim) { + error =3D ENOMEM; + goto done; + } + } if (vm->vm_map.size + (new - old) > vmemlim) { error =3D ENOMEM; goto done; Index: lib/libc/sys/mlockall.2 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- lib/libc/sys/mlockall.2 (revision 239772) +++ lib/libc/sys/mlockall.2 (working copy) @@ -72,8 +72,6 @@ limit and the per-process .Dv RLIMIT_MEMLOCK resource limit. .Pp -These calls are only available to the super-user. -.Pp The .Fn munlockall call unlocks any locked memory regions in the process address space. Index: lib/libc/sys/mlock.2 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- lib/libc/sys/mlock.2 (revision 239772) +++ lib/libc/sys/mlock.2 (working copy) @@ -98,8 +98,6 @@ a system-wide ``wired pages'' limit and the per-process .Li RLIMIT_MEMLOCK resource limit. -.Pp -These calls are only available to the super-user. .Sh RETURN VALUES .Rv -std .Pp @@ -111,8 +109,6 @@ The system call will fail if: .Bl -tag -width Er -.It Bq Er EPERM -The caller is not the super-user. .It Bq Er EINVAL The address given is not page aligned or the length is negative. .It Bq Er EAGAIN @@ -128,8 +124,6 @@ The system call will fail if: .Bl -tag -width Er -.It Bq Er EPERM -The caller is not the super-user. .It Bq Er EINVAL The address given is not page aligned or the length is negative. .It Bq Er ENOMEM --------------020105090900020700090502-- --------------enig5C2551C41E5B3832AFE69806 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.18 (Darwin) Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJQPy0nAAoJEBWLemxX/CvTF0QIAK0JH1n++jqx9etPFKNP8TRP lcSoyPBdzMwnpqiOPNBxJqXfrh0kW+73gyCDaPhOp65KTVIK+VPaGcZX/3qwTyA1 IdwTji8nRdwGp+L8mbjNRq/2CbHcaS/us31qql/I8krL+kJ0gfaDHOSlxuNvQj10 T3pMXQ5PmPvuioVutSrA6u1HENGCBLQemk5npvfNzDbYdwXe5re0+GhU3v7n4zoy IdX/9sDdj8skboOeVHO+n0Z9Dzr1vuTiYitfTiuy9wXhqCx2CtDOlbefWc0TEoR4 t4bNv8cnVVyXj/h7b2OdBgOOhzFtJt8jN9hjGvOMbfK6+2t17skdKcDB8R++l14= =voP9 -----END PGP SIGNATURE----- --------------enig5C2551C41E5B3832AFE69806-- From owner-freebsd-arch@FreeBSD.ORG Thu Aug 30 10:59:06 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B2F95106564A for ; Thu, 30 Aug 2012 10:59:06 +0000 (UTC) (envelope-from andrey@zonov.org) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 354C88FC18 for ; Thu, 30 Aug 2012 10:59:05 +0000 (UTC) Received: by eaak11 with SMTP id k11so523187eaa.13 for ; Thu, 30 Aug 2012 03:58:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:subject :references:in-reply-to:x-enigmail-version:content-type :x-gm-message-state; bh=2p2bFwPo3zRfSdB+x5Q/UW2pr5sjswsplBRinoeMp4U=; b=ZS3l/WyEywQvVwew7mUkeAkBsZp8DtawnQZRQDkKC87csdo8lu3VvtaJ3bys/j3Ffs IZaeQF9PeewAoi4wluZfnR71KmBjOQlk3H7rubj1GWw6TmJHXqgcj48dDbVYMjAPwVw1 A2TzC5hAjF0OoddPHvDG87hUfwutbHzhuCymHvzT/03FHA8aDFYdiSJdgsJ47U+XDAsF DKRnvjmqVyEhvVSppQHcGD7aCmxVG22Am/uwcocGbyjD8F6L6+YmvBeeYi5xfPrpMLBH m/SBmxxg833J6lXXoO08R5cq6dYwNRM2FhHwJcEV0mdQ1LFsuX/snoWCud86F49HNBKL HdMw== Received: by 10.14.4.201 with SMTP id 49mr6055842eej.0.1346324338885; Thu, 30 Aug 2012 03:58:58 -0700 (PDT) Received: from dhcp170-234-red.yandex.net ([2a02:6b8:0:401:c52:f98c:398a:b0d7]) by mx.google.com with ESMTPS id a7sm3853619eep.14.2012.08.30.03.58.57 (version=SSLv3 cipher=OTHER); Thu, 30 Aug 2012 03:58:58 -0700 (PDT) Sender: Andrey Zonov Message-ID: <503F476E.1010505@FreeBSD.org> Date: Thu, 30 Aug 2012 14:58:54 +0400 From: Andrey Zonov User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) Gecko/20120824 Thunderbird/15.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org References: <503DD433.2030108@FreeBSD.org> <201208290906.q7T96C9j032802@gw.catspoiler.org> <20120829092318.GW33100@deviant.kiev.zoral.com.ua> <503F2D24.8050103@FreeBSD.org> In-Reply-To: <503F2D24.8050103@FreeBSD.org> X-Enigmail-Version: 1.4.4 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig6E4239DCB20C1E641CC3E0B6" X-Gm-Message-State: ALoCoQm411VAfz585MlmG3OP7NegUEkTghBbw7dW7nH50Ya/sKqQmmaD8UIKpV5cuTKJYaKPdiZp Subject: Re: [patch] unprivileged mlock(2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Aug 2012 10:59:06 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig6E4239DCB20C1E641CC3E0B6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 8/30/12 1:06 PM, Andrey Zonov wrote: > Hi, >=20 > So, I've got the first version of the patch (attached) which fixes > memory locked limit checking and accounting. >=20 > I also want to change 'pmap_wired_count(vm_map_pmap(map))' with > 'vmspace_wired_count(vmspace)' It's here [1]. > Index: sys/vm/vm_mmap.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/vm/vm_mmap.c (revision 239611) > +++ sys/vm/vm_mmap.c (working copy) > @@ -1483,11 +1483,13 @@ vm_mmap(vm_map_t map, vm_offset_t *addr, vm_siz= e_t > PROC_UNLOCK(td->td_proc); > return (ENOMEM); > } > +#ifdef RACCT > if (racct_set(td->td_proc, RACCT_VMEM, map->size + size)) { > PROC_UNLOCK(td->td_proc); > return (ENOMEM); > } > PROC_UNLOCK(td->td_proc); > +#endif > } >=20 > /* >=20 I put '#endif' in the wrong place, corrected patch is here [2]. [1] http://people.freebsd.org/~zont/vm_mmap.c.patch [2] http://people.freebsd.org/~zont/racct.patch --=20 Andrey Zonov --------------enig6E4239DCB20C1E641CC3E0B6 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.18 (Darwin) Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJQP0dxAAoJEBWLemxX/CvTO9gH/RsdGYsLBOepleQSctFdabGr 35gtiy+olhOaUOienpUgRPJGKYh2P3urj4w1vYKMyd3OGstCDiUHtO8T8ZwQO3an AM5eLOzDuegnSZjXf5hxryX8qdY05WOrLN7/5Cedxr/3OHR3AIb8Zrbk+aBZ7kf+ 9Cl2a1DkMJzJ79Ipw/U3frN7mZlKhVM2U1sazNT+SImf6SJ3tJfTWuIm0OO8iEq7 w69YjyM1kAmjfwWwnAjV0Z0VO70vmhaRf27Nsww61GzrGRa/GGRQcfz/Lb/QIS5f 0+XHm46pKqbEpkeMJNOZvh2HLpqD2Oh3claWTvGDgmbziSMshdw+Gq18IjXLxMM= =8tIU -----END PGP SIGNATURE----- --------------enig6E4239DCB20C1E641CC3E0B6-- From owner-freebsd-arch@FreeBSD.ORG Thu Aug 30 20:40:24 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B76981065670 for ; Thu, 30 Aug 2012 20:40:24 +0000 (UTC) (envelope-from asp654@gmail.com) Received: from mail-lpp01m010-f54.google.com (mail-lpp01m010-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 3491E8FC0A for ; Thu, 30 Aug 2012 20:40:23 +0000 (UTC) Received: by lage12 with SMTP id e12so2289223lag.13 for ; Thu, 30 Aug 2012 13:40:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=O/Ier/C2/gMaEx1iIBtU95qMVcrM6vSAVdi2yiMh0Ag=; b=ci+4dz9NWKNwywpFJ4f2xmw/kKVvMudqtKArrZHU1t6WRqhQa+l/V+lwGAD15gPeBl xpYTKd2Q7CUPGTwsv2tz7N8tkcFm+JFbHAd/QdxzsgktWyqeIwrpccb5XL+P7U/VfELM eoXCDYkmjzWlQ2HWgZOfJ92Dqy8KQT2co3fradCC+zbuXaK4Z2oJSwIBYklBTvh9+rJA 96fkilj75Ze+7HKiR666OT/P5tSpybm2iqsm7ALdHYgb/aMS96t1jX/BRICPgrLKtmyA RlxnmuiIRXxSOy2knBrii/9brvsuLsWRhkayHfpuLNuEVc5loDt+tdYL2Sm77qrCRvPF arZA== MIME-Version: 1.0 Received: by 10.152.103.244 with SMTP id fz20mr4537307lab.54.1346359222740; Thu, 30 Aug 2012 13:40:22 -0700 (PDT) Received: by 10.112.106.73 with HTTP; Thu, 30 Aug 2012 13:40:22 -0700 (PDT) Date: Thu, 30 Aug 2012 13:40:22 -0700 Message-ID: From: asp imho To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: FreeBSD complies with which ABI X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Aug 2012 20:40:24 -0000 Hi all, Which ABI does FreeBSD comply to for multiple architectures (x86, powerpc)... I've seen that the executable is in the ELF format (specified in System V ABI), so does this mean that we comply to System V ABI completely for all architectures. Please let me know. I've asked this in the FreeBSD forums, I was directed to one of the forums here. If this is not the correct forum please direct me to the correct forum where this question can be posted. Thanks. From owner-freebsd-arch@FreeBSD.ORG Thu Aug 30 21:30:12 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 754831065675 for ; Thu, 30 Aug 2012 21:30:12 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) by mx1.freebsd.org (Postfix) with ESMTP id 527618FC0C for ; Thu, 30 Aug 2012 21:30:12 +0000 (UTC) Received: from troutmask.apl.washington.edu (localhost.apl.washington.edu [127.0.0.1]) by troutmask.apl.washington.edu (8.14.5/8.14.5) with ESMTP id q7ULU5XY081688; Thu, 30 Aug 2012 14:30:05 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.14.5/8.14.5/Submit) id q7ULU4TI081687; Thu, 30 Aug 2012 14:30:04 -0700 (PDT) (envelope-from sgk) Date: Thu, 30 Aug 2012 14:30:04 -0700 From: Steve Kargl To: asp imho Message-ID: <20120830213004.GA81386@troutmask.apl.washington.edu> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-arch@freebsd.org Subject: Re: FreeBSD complies with which ABI X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Aug 2012 21:30:12 -0000 On Thu, Aug 30, 2012 at 01:40:22PM -0700, asp imho wrote: > Hi all, > > Which ABI does FreeBSD comply to for multiple architectures (x86, > powerpc)... I've seen that the executable is in the ELF format (specified > in System V ABI), so does this mean that we comply to System V ABI > completely for all architectures. Please let me know. > man brandelf % brandelf -l known ELF types are: FreeBSD(9) Linux(3) Solaris(6) SVR4(0) On an AMD64 system with compat_linux: % elfdump -e /bin/cat | grep e_ident e_ident: ELFCLASS64 ELFDATA2LSB ELFOSABI_FREEBSD % elfdump -e /compat/linux//bin/cat | grep e_ident e_ident: ELFCLASS32 ELFDATA2LSB ELFOSABI_LINUX On a sparc64 system, % elfdump -e /bin/cat | grep ident e_ident: ELFCLASS64 ELFDATA2MSB ELFOSABI_FREEBSD -- Steve From owner-freebsd-arch@FreeBSD.ORG Thu Aug 30 22:00:37 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 96081106566B for ; Thu, 30 Aug 2012 22:00:37 +0000 (UTC) (envelope-from asp654@gmail.com) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id 0EC5B8FC0A for ; Thu, 30 Aug 2012 22:00:36 +0000 (UTC) Received: by lbbgg13 with SMTP id gg13so1003731lbb.13 for ; Thu, 30 Aug 2012 15:00:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Wsa19unfNgqGM2R2f16BbYfbYbkH/3yUv/npN9VIoR0=; b=hhGqtZZxB4d03kbwZOCj1e11JNDUSt0OnSpcO+mz1vSEZcg2W+3Yny3sj0s4gkgYXi qFZTUBUsji6yTiEmvRMNsuyKqC1yBzzNMDJjFzra5aPXwSJFKSnl9uQ5J30OYqgrwmr+ sPfD1hfxbyt2EePIE1a0DJdvRqHkwahXcNd9TUhx4oq5Gsr8V4fLRyYQyiAm+6hs1zzF kaKyakxM1ERPQansgx7/txljOL36qdEjRYGID2Z19jw/Jrsnd1K5Z2MSpK0rFzjegQQm ZAuuxfPaUj8OVyG1D/p2q3idBV5KdnmjS/WhOZynmYlqlYJwBsG8jHS5EIc75PGJWN8Q PErA== MIME-Version: 1.0 Received: by 10.152.110.40 with SMTP id hx8mr4760341lab.9.1346364035371; Thu, 30 Aug 2012 15:00:35 -0700 (PDT) Received: by 10.112.106.73 with HTTP; Thu, 30 Aug 2012 15:00:35 -0700 (PDT) In-Reply-To: <20120830213004.GA81386@troutmask.apl.washington.edu> References: <20120830213004.GA81386@troutmask.apl.washington.edu> Date: Thu, 30 Aug 2012 15:00:35 -0700 Message-ID: From: asp imho To: Steve Kargl Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-arch@freebsd.org Subject: Re: FreeBSD complies with which ABI X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Aug 2012 22:00:37 -0000 Thanks Steve. On Thu, Aug 30, 2012 at 2:30 PM, Steve Kargl < sgk@troutmask.apl.washington.edu> wrote: > On Thu, Aug 30, 2012 at 01:40:22PM -0700, asp imho wrote: > > Hi all, > > > > Which ABI does FreeBSD comply to for multiple architectures (x86, > > powerpc)... I've seen that the executable is in the ELF format (specified > > in System V ABI), so does this mean that we comply to System V ABI > > completely for all architectures. Please let me know. > > > > man brandelf > > % brandelf -l > known ELF types are: FreeBSD(9) Linux(3) Solaris(6) SVR4(0) > > On an AMD64 system with compat_linux: > % elfdump -e /bin/cat | grep e_ident > e_ident: ELFCLASS64 ELFDATA2LSB ELFOSABI_FREEBSD > % elfdump -e /compat/linux//bin/cat | grep e_ident > e_ident: ELFCLASS32 ELFDATA2LSB ELFOSABI_LINUX > > On a sparc64 system, > % elfdump -e /bin/cat | grep ident > e_ident: ELFCLASS64 ELFDATA2MSB ELFOSABI_FREEBSD > > > -- > Steve > From owner-freebsd-arch@FreeBSD.ORG Fri Aug 31 06:12:28 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 36C521065677 for ; Fri, 31 Aug 2012 06:12:28 +0000 (UTC) (envelope-from andrey@zonov.org) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id 991E78FC20 for ; Fri, 31 Aug 2012 06:12:27 +0000 (UTC) Received: by lbbgg13 with SMTP id gg13so1190639lbb.13 for ; Thu, 30 Aug 2012 23:12:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:subject :references:in-reply-to:x-enigmail-version:content-type :x-gm-message-state; bh=lJDePPNOf7q8/qvBduNOSn5EC/CLK88277Z5/JiaCFM=; b=P9iY9PWSQ7ep2uYyrl+TvspO27QO9NTgGtKJVA45eK81uqryyFIq8jUO5YduBAeWCd 1RvrQiWS0DGgCMW88o1tjPR3B1Pg/mMnQpq+NBEtZEzSexz+/0Qq6LW6W2s+byDBLfUs wNRwf2wvSReDYdhIf6SKIBwKEECG6KqL3eZetVvV2y1OciOM7MX9mROnaUWrJt/k+Q0U 3VrN/R1YZm87GHfHVn9a6Z0HufyG8GBPhPkIQMNwc78QEjBapmFaYQuDoK2/Sr1+Y/aM epPGx2Dl7eq/tEh7YQR4p1hhmPixA2bIfF+dWooKrJ8DCqizCFHoIoDk+fbGoM7U+Uu7 bo7A== Received: by 10.152.46.209 with SMTP id x17mr5437932lam.38.1346393545532; Thu, 30 Aug 2012 23:12:25 -0700 (PDT) Received: from zont-osx.local (ppp95-165-143-86.pppoe.spdop.ru. [95.165.143.86]) by mx.google.com with ESMTPS id gv8sm3809744lab.14.2012.08.30.23.12.24 (version=SSLv3 cipher=OTHER); Thu, 30 Aug 2012 23:12:24 -0700 (PDT) Sender: Andrey Zonov Message-ID: <504055C6.8060704@FreeBSD.org> Date: Fri, 31 Aug 2012 10:12:22 +0400 From: Andrey Zonov User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) Gecko/20120824 Thunderbird/15.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org References: <503DD433.2030108@FreeBSD.org> <201208290906.q7T96C9j032802@gw.catspoiler.org> <20120829092318.GW33100@deviant.kiev.zoral.com.ua> <503F2D24.8050103@FreeBSD.org> <503F476E.1010505@FreeBSD.org> In-Reply-To: <503F476E.1010505@FreeBSD.org> X-Enigmail-Version: 1.4.4 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig16ACCA67F278F16EE0A13A8C" X-Gm-Message-State: ALoCoQmZ0MGh3jvcGBWY/DYMfd+ED5Bfwyn+wSGtCDxs8fKLu3CIlYBi+U1l/3MLOpl58fKXRHgG Subject: Re: [patch] unprivileged mlock(2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 31 Aug 2012 06:12:28 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig16ACCA67F278F16EE0A13A8C Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 8/30/12 2:58 PM, Andrey Zonov wrote: > On 8/30/12 1:06 PM, Andrey Zonov wrote: >> Index: sys/vm/vm_mmap.c >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> --- sys/vm/vm_mmap.c (revision 239611) >> +++ sys/vm/vm_mmap.c (working copy) >> @@ -1483,11 +1483,13 @@ vm_mmap(vm_map_t map, vm_offset_t *addr, vm_si= ze_t >> PROC_UNLOCK(td->td_proc); >> return (ENOMEM); >> } >> +#ifdef RACCT >> if (racct_set(td->td_proc, RACCT_VMEM, map->size + size)) { >> PROC_UNLOCK(td->td_proc); >> return (ENOMEM); >> } >> PROC_UNLOCK(td->td_proc); >> +#endif >> } >> >> /* >> >=20 > I put '#endif' in the wrong place, corrected patch is here [2]. >=20 We don't need this patch, because accounting code doesn't add any extra process locking here. --=20 Andrey Zonov --------------enig16ACCA67F278F16EE0A13A8C Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.18 (Darwin) Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJQQFXJAAoJEBWLemxX/CvTep0H/i30DoHwh+oEgXdpIawicSVN HrRouTTJqG7+bR/qNoXmN0VHZfQtO+mD+MmuTq9jkHS2GwnJKBjACFYqT40vOLTt hd2YxYtec82WpjZoIfadM5xjSI5faBbAY8IVrLKt3/Sy9Zoc+qZ/XrDN9+LP1G1N ZX6NsZnI5mvDMo++2f90NE1CB4C0Hryobvz22wy468R5+q6uDi7ZfcwEVzCJuq8/ /+LLQRMe49Q8igjnMzF560mcd5+MLFdjTG1O/yaMCQUgeux1+nvPPINBZElcDcBu pPRjqpa1v+pY8DH5s/bRNz3qmynepb3eke+tHDHkTdxDt8EEV3mhvZv+gTEFaLk= =ZbPd -----END PGP SIGNATURE----- --------------enig16ACCA67F278F16EE0A13A8C-- From owner-freebsd-arch@FreeBSD.ORG Fri Aug 31 12:41:07 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6EF6F1065673; Fri, 31 Aug 2012 12:41:07 +0000 (UTC) (envelope-from ray@dlink.ua) Received: from smtp.dlink.ua (smtp.dlink.ua [193.138.187.146]) by mx1.freebsd.org (Postfix) with ESMTP id 05F108FC1C; Fri, 31 Aug 2012 12:41:03 +0000 (UTC) Received: from terran.dlink.ua (unknown [192.168.10.90]) (Authenticated sender: ray) by smtp.dlink.ua (Postfix) with ESMTPSA id 7A1CCC4935; Fri, 31 Aug 2012 15:40:56 +0300 (EEST) Date: Fri, 31 Aug 2012 15:42:41 +0300 From: Aleksandr Rybalko To: arch@freebsd.org, embedded@freebsd.org Message-Id: <20120831154241.82902856.ray@dlink.ua> Organization: D-Link X-Mailer: Sylpheed 2.7.1 (GTK+ 2.20.1; i386-portbld-freebsd8.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Subject: [RFC] hintmode switch patch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 31 Aug 2012 12:41:07 -0000 Hi hackers, I already post that patch some time ago with proposed "dynamic attach of hinted devices" patch. [1] But will try to do it again, step-by-step :) So that patch allow switch from static hints to dynamic hints. That way embedded systems, which usually compiled with hints (static) will be able to see/edit/add hints and/or kenv variables. If nobody have objections I will commit it soon. Hope 2-3 days enough for that :) [1] http://lists.freebsd.org/pipermail/freebsd-arch/2012-January/012295.html [2] http://people.freebsd.org/~ray/subr_hints.c.patch Thanks. WBW -- Alexandr Rybalko aka Alex RAY From owner-freebsd-arch@FreeBSD.ORG Fri Aug 31 14:00:15 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D05FB1065673 for ; Fri, 31 Aug 2012 14:00:15 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ie0-f182.google.com (mail-ie0-f182.google.com [209.85.223.182]) by mx1.freebsd.org (Postfix) with ESMTP id 6EDA18FC26 for ; Fri, 31 Aug 2012 14:00:11 +0000 (UTC) Received: by iebc12 with SMTP id c12so2153651ieb.13 for ; Fri, 31 Aug 2012 07:00:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=NSjLsMO4aS5NiSVmXyUKcTrXo2muu6RbkfEgRRMQccQ=; b=LUx8U805gTV7fxp0T6Q/fnTwygzq+d1PSV5/QqXR/ilfu793Vm0mXVonNuH5yuw7eG qSNfAquvF+z5uyqSQlO7pWXv+3OzysPwCQw02fHGOFcUAR4geCRFiqxCLlncOPLoWBMS M+mbaAYYRragaysc+nSAKTKaY8UevLSxgtBe4aeTPqT4Wqb7WX2KPMecIaw24IlNUvux YzQXVx0TuUgJ+BuwJQUFmTNhyF7lt2aEOa3GgknuUp6ZfHZ7hQb3CaWs4iLePA/XkKzP XKykUbZExwgy8ZAueqpvwIGm1ql3Y3OOprV3TYRL1jy5kr1dWZspeervUpGVvSVyb5x/ 1+Xw== Received: by 10.50.169.70 with SMTP id ac6mr2887589igc.12.1346421610915; Fri, 31 Aug 2012 07:00:10 -0700 (PDT) Received: from 63.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id xm2sm1219673igb.3.2012.08.31.07.00.08 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 31 Aug 2012 07:00:10 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <20120831154241.82902856.ray@dlink.ua> Date: Fri, 31 Aug 2012 08:00:08 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <103E211E-75F0-4017-9167-E1AA940D1FC2@bsdimp.com> References: <20120831154241.82902856.ray@dlink.ua> To: Aleksandr Rybalko X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQlI0UDopSXdpuI1L36F2eZme+DXzUH76xm2G1EhqKH/8Gs37gcxa2Sw3PYgpmvYOHCvyQ+7 Cc: arch@freebsd.org, embedded@freebsd.org Subject: Re: [RFC] hintmode switch patch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 31 Aug 2012 14:00:15 -0000 On Aug 31, 2012, at 6:42 AM, Aleksandr Rybalko wrote: > Hi hackers, >=20 > I already post that patch some time ago with proposed "dynamic attach > of hinted devices" patch. [1] >=20 > But will try to do it again, step-by-step :) >=20 > So that patch allow switch from static hints to dynamic hints. > That way embedded systems, which usually compiled with hints (static) > will be able to see/edit/add hints and/or kenv variables. >=20 > If nobody have objections I will commit it soon. > Hope 2-3 days enough for that :) >=20 > [1] > = http://lists.freebsd.org/pipermail/freebsd-arch/2012-January/012295.html > [2] http://people.freebsd.org/~ray/subr_hints.c.patch I'll have to take a look at this. I had it on my list to review, but it = dropped off... Warner From owner-freebsd-arch@FreeBSD.ORG Fri Aug 31 17:55:04 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B63551065675; Fri, 31 Aug 2012 17:55:04 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 83DB08FC12; Fri, 31 Aug 2012 17:55:04 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id CB4B9B945; Fri, 31 Aug 2012 13:55:03 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org, embedded@freebsd.org Date: Fri, 31 Aug 2012 10:31:41 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p17; KDE/4.5.5; amd64; ; ) References: <20120831154241.82902856.ray@dlink.ua> In-Reply-To: <20120831154241.82902856.ray@dlink.ua> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201208311031.42163.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 31 Aug 2012 13:55:03 -0400 (EDT) Cc: Aleksandr Rybalko Subject: Re: [RFC] hintmode switch patch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 31 Aug 2012 17:55:04 -0000 On Friday, August 31, 2012 8:42:41 am Aleksandr Rybalko wrote: > Hi hackers, > > I already post that patch some time ago with proposed "dynamic attach > of hinted devices" patch. [1] > > But will try to do it again, step-by-step :) > > So that patch allow switch from static hints to dynamic hints. > That way embedded systems, which usually compiled with hints (static) > will be able to see/edit/add hints and/or kenv variables. > > If nobody have objections I will commit it soon. > Hope 2-3 days enough for that :) > > [1] > http://lists.freebsd.org/pipermail/freebsd-arch/2012-January/012295.html > [2] http://people.freebsd.org/~ray/subr_hints.c.patch > > Thanks. A few suggestions: 1) You can simplify sysctl_hintmode() if you do something like: value = hintmode; error = sysctl_handle_int(req, &value, 0, oidp); if (error || req->newptr == NULL) return (error); In place of doing SYSCTL_IN/OUT by hand. I prefer doing this in custom sysctl handlers so that the function is focused on the nonstandard functionality. 2) I would just leave the global comment ('Access functions..') where it is as it seems to me to be a comment for the whole file. Other than that I think this is fine. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Aug 31 20:02:03 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 972211065670; Fri, 31 Aug 2012 20:02:03 +0000 (UTC) (envelope-from ray@freebsd.org) Received: from smtp.dlink.ua (smtp.dlink.ua [193.138.187.146]) by mx1.freebsd.org (Postfix) with ESMTP id 469A58FC18; Fri, 31 Aug 2012 20:02:03 +0000 (UTC) Received: from rnote.ddteam.net (77-193-201-46.pool.ukrtel.net [46.201.193.77]) (Authenticated sender: ray) by smtp.dlink.ua (Postfix) with ESMTPSA id AED6DC4936; Fri, 31 Aug 2012 23:02:01 +0300 (EEST) Date: Fri, 31 Aug 2012 23:01:57 +0300 From: Aleksandr Rybalko To: John Baldwin Message-Id: <20120831230157.53c86a85.ray@freebsd.org> In-Reply-To: <201208311031.42163.jhb@freebsd.org> References: <20120831154241.82902856.ray@dlink.ua> <201208311031.42163.jhb@freebsd.org> Organization: FreeBSD.ORG X-Mailer: Sylpheed 3.1.2 (GTK+ 2.24.5; amd64-portbld-freebsd9.0) X-Operating-System: FreeBSD Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Aleksandr Rybalko , embedded@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [RFC] hintmode switch patch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 31 Aug 2012 20:02:03 -0000 On Fri, 31 Aug 2012 10:31:41 -0400 John Baldwin wrote: > On Friday, August 31, 2012 8:42:41 am Aleksandr Rybalko wrote: > > Hi hackers, > > > > I already post that patch some time ago with proposed "dynamic > > attach of hinted devices" patch. [1] > > > > But will try to do it again, step-by-step :) > > > > So that patch allow switch from static hints to dynamic hints. > > That way embedded systems, which usually compiled with hints > > (static) will be able to see/edit/add hints and/or kenv variables. > > > > If nobody have objections I will commit it soon. > > Hope 2-3 days enough for that :) > > > > [1] > > http://lists.freebsd.org/pipermail/freebsd-arch/2012-January/012295.html > > [2] http://people.freebsd.org/~ray/subr_hints.c.patch > > > > Thanks. > > A few suggestions: > > 1) You can simplify sysctl_hintmode() if you do something like: > > value = hintmode; > error = sysctl_handle_int(req, &value, 0, oidp); > if (error || req->newptr == NULL) > return (error); > > In place of doing SYSCTL_IN/OUT by hand. I prefer doing this in > custom sysctl handlers so that the function is focused on the > nonstandard functionality. Very good hint, not seen that helpers before. Thank you John! > > 2) I would just leave the global comment ('Access functions..') where > it is as it seems to me to be a comment for the whole file. > Sometime confusing with that, operations on kenv, made in subr_hints.c. Moved back. > Other than that I think this is fine. > > -- > John Baldwin > _______________________________________________ > freebsd-embedded@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-embedded > To unsubscribe, send any mail to > "freebsd-embedded-unsubscribe@freebsd.org" Thanks! Patch updated, and stay at same place. WBW -- Aleksandr Rybalko From owner-freebsd-arch@FreeBSD.ORG Sat Sep 1 14:01:39 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1D186106564A; Sat, 1 Sep 2012 14:01:39 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id DF4858FC0A; Sat, 1 Sep 2012 14:01:38 +0000 (UTC) Received: by dadr6 with SMTP id r6so2518012dad.13 for ; Sat, 01 Sep 2012 07:01:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=vXzVcVRQHeeqPWE+6b7u01Bz1Xtk47x4FtPsrIOdAK8=; b=biia8uKQDKVtik753wxBYC568xYSnFG+b+ky7dyphW6RvTkP3IMqlUM/KR1BJu9NO7 YCaMfmd4L0TVGXtAvu+u6cT2ssUTxLLwXAPIbwMPypF3YAtEphw0l07NMnz1fo0jgnOe fRT5QgRakQ4t0xqRnfNTWEzNY4JgPsA3t8JyZ7ZgEP7FMH51YzSueK5DtT6DCiNx7+ET xZP4Eg9sXiioGY/n6/wTAj+P9QW+z4SkWaDLg+jFobaqBAB9iHju6V02gtV74h+Nji5o cunckCQKSNE3TvxFamQClzo5N+9gpazGxbg+ivOhnVqJdF+AGgNh8k8ZbGKmERJpwVN8 U2bw== MIME-Version: 1.0 Received: by 10.66.78.73 with SMTP id z9mr22017121paw.9.1346508098631; Sat, 01 Sep 2012 07:01:38 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.68.36.106 with HTTP; Sat, 1 Sep 2012 07:01:38 -0700 (PDT) In-Reply-To: <20120831154241.82902856.ray@dlink.ua> References: <20120831154241.82902856.ray@dlink.ua> Date: Sat, 1 Sep 2012 07:01:38 -0700 X-Google-Sender-Auth: NRLD-1Pxyu_JW_aA2MzW_OvEwjQ Message-ID: From: Adrian Chadd To: Aleksandr Rybalko Content-Type: text/plain; charset=ISO-8859-1 Cc: arch@freebsd.org, embedded@freebsd.org Subject: Re: [RFC] hintmode switch patch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Sep 2012 14:01:39 -0000 Once others have stopped giving you reasons not to commit, please commit. Adrian On 31 August 2012 05:42, Aleksandr Rybalko wrote: > Hi hackers, > > I already post that patch some time ago with proposed "dynamic attach > of hinted devices" patch. [1] > > But will try to do it again, step-by-step :) > > So that patch allow switch from static hints to dynamic hints. > That way embedded systems, which usually compiled with hints (static) > will be able to see/edit/add hints and/or kenv variables. > > If nobody have objections I will commit it soon. > Hope 2-3 days enough for that :) > > [1] > http://lists.freebsd.org/pipermail/freebsd-arch/2012-January/012295.html > [2] http://people.freebsd.org/~ray/subr_hints.c.patch > > Thanks. > > WBW > -- > Alexandr Rybalko > aka Alex RAY > _______________________________________________ > freebsd-embedded@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-embedded > To unsubscribe, send any mail to "freebsd-embedded-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Sat Sep 1 23:13:51 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1DD1D106567B for ; Sat, 1 Sep 2012 23:13:51 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id B8C6F8FC15 for ; Sat, 1 Sep 2012 23:13:50 +0000 (UTC) Received: by ialo14 with SMTP id o14so8318176ial.13 for ; Sat, 01 Sep 2012 16:13:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=hG3rYIFLHathq7ok5nHgcXaOxVp2SConopuVZyO4DQQ=; b=hemlXaddBH96YdXyHJ96PwkEf2weAH/9je7AmDZLBiEA0UnHxXsXrqX3LeauYhpLqN Zn4SFpFOJT9jOzZS2j1yftSNiImK0KMAPCi7h7AmckdM8z8Q1xsAFGU92g/zjCLAlNeI tw4JSQ0zHLHK+Yxio/T2bXkMGGGqqQdfMpouToeRLzpwN5/SiqPTqh9d7xDc1zWkjZI3 08kp1RpwSO3ky9GE+2KMflg08UHHA8LcXY1RXWUJ8nVF4Ikonnwl0s6tSgA82KNM4OWv eJPSfF6YZMJ/QXdunoqXyxpfgUm3iGqzYK4JM/Mw+QGIhUWkPVpusfesRq0ehAsICl9P 1eHw== Received: by 10.50.195.194 with SMTP id ig2mr6995119igc.24.1346541230087; Sat, 01 Sep 2012 16:13:50 -0700 (PDT) Received: from 63.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id p8sm4647288igl.16.2012.09.01.16.13.48 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 01 Sep 2012 16:13:48 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: Date: Sat, 1 Sep 2012 17:13:46 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <02C549DE-D912-4D1A-A08D-650B918E9B99@bsdimp.com> References: <20120831154241.82902856.ray@dlink.ua> To: Adrian Chadd X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQkRZUU2GwTBW8P2inC3Lb7C9mxIr+mzXzNd+wLKNRCc0kpBSb3zKcFN61apnbnBiUQ/AhXu Cc: Aleksandr Rybalko , arch@freebsd.org, embedded@freebsd.org Subject: Re: [RFC] hintmode switch patch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Sep 2012 23:13:51 -0000 This patch looks good to me. It contained none of the stuff I was = worried about before, so please commit this. Warner On Sep 1, 2012, at 8:01 AM, Adrian Chadd wrote: > Once others have stopped giving you reasons not to commit, please = commit. >=20 >=20 >=20 > Adrian >=20 >=20 > On 31 August 2012 05:42, Aleksandr Rybalko wrote: >> Hi hackers, >>=20 >> I already post that patch some time ago with proposed "dynamic attach >> of hinted devices" patch. [1] >>=20 >> But will try to do it again, step-by-step :) >>=20 >> So that patch allow switch from static hints to dynamic hints. >> That way embedded systems, which usually compiled with hints (static) >> will be able to see/edit/add hints and/or kenv variables. >>=20 >> If nobody have objections I will commit it soon. >> Hope 2-3 days enough for that :) >>=20 >> [1] >> = http://lists.freebsd.org/pipermail/freebsd-arch/2012-January/012295.html >> [2] http://people.freebsd.org/~ray/subr_hints.c.patch >>=20 >> Thanks. >>=20 >> WBW >> -- >> Alexandr Rybalko >> aka Alex RAY >> _______________________________________________ >> freebsd-embedded@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-embedded >> To unsubscribe, send any mail to = "freebsd-embedded-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-embedded@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-embedded > To unsubscribe, send any mail to = "freebsd-embedded-unsubscribe@freebsd.org"