From owner-freebsd-mips@FreeBSD.ORG Sun Dec 9 18:50:28 2012 Return-Path: Delivered-To: mips@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 894DC360 for ; Sun, 9 Dec 2012 18:50:28 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: from mail-pa0-f54.google.com (mail-pa0-f54.google.com [209.85.220.54]) by mx1.freebsd.org (Postfix) with ESMTP id 474E28FC15 for ; Sun, 9 Dec 2012 18:50:28 +0000 (UTC) Received: by mail-pa0-f54.google.com with SMTP id bi5so1535890pad.13 for ; Sun, 09 Dec 2012 10:50:22 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=date:from:x-x-sender:to:cc:subject:in-reply-to:message-id :references:user-agent:mime-version:content-type:x-gm-message-state; bh=5fWgylGiq8F/xIkEEo08py/KPhxFu2c4EOHYYrQz/mc=; b=RD+semxIgqRfCF2XGyyCD+IdBdDpM5cCglHnNwBHE2W0sS/vzmMHlwzGEDfirJEIlb +DxRzTfG6mpC7z4Io8bMEDqtY+Q8HlCy/mA35wZdz63hvmrEZoeDABfNWiI/m9wpe1hH ZE4HIArig+tXws9FogxPQ/Lb9Doyk2YkS0pHQjZQh1MJy9FbvY0BsHrwtSD8e3Eg+c0f AHOSDs7WGjIzZ4AhIHKLBSdinSlhG4BXblhdCWV6WW+36ryReypaKflo5UhF/MIx0LTd kXOp2HYeMhUEQsYzk//cF4MidmFvb2/u5Cz9awri0PAMr3mr4v10OlYSrRTYKbqSUAIh /vFQ== Received: by 10.66.84.10 with SMTP id u10mr30029217pay.24.1355079021836; Sun, 09 Dec 2012 10:50:21 -0800 (PST) Received: from rrcs-66-91-135-210.west.biz.rr.com (rrcs-66-91-135-210.west.biz.rr.com. [66.91.135.210]) by mx.google.com with ESMTPS id m4sm10436372pav.17.2012.12.09.10.50.19 (version=SSLv3 cipher=OTHER); Sun, 09 Dec 2012 10:50:21 -0800 (PST) Date: Sun, 9 Dec 2012 08:48:51 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: Ian Lepore Subject: Re: Call for testing and review, busdma changes In-Reply-To: <1355077061.87661.320.camel@revolution.hippie.lan> Message-ID: References: <1355077061.87661.320.camel@revolution.hippie.lan> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Gm-Message-State: ALoCoQlc8cJ+RHQHJOhbu00g0RDo3w+6M/kS23SfDh/+XoYTtEFOnFMmR3tvVW8IjfpQt1e/9TKc X-Mailman-Approved-At: Mon, 10 Dec 2012 00:21:05 +0000 Cc: powerpc@freebsd.org, marcel@freebsd.org, mips@freebsd.org, John Baldwin , mav@freebsd.org, scottl@freebsd.org, attilio@freebsd.org, kib@freebsd.org, sparc64@freebsd.org, arm@freebsd.org X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Dec 2012 18:50:28 -0000 On Sun, 9 Dec 2012, Ian Lepore wrote: > On Sat, 2012-12-08 at 08:51 -1000, Jeff Roberson wrote: >> Hello, >> >> http://people.freebsd.org/~jeff/physbio.diff >> >> I have a relative large patch that reforms the busdma API so that new >> types may be added without modifying every architecture's >> busdma_machdep.c. It does this by unifying the bus_dmamap_load_buffer() >> routines so that they may be called from MI code. The MD busdma is then >> given a chance to do any final processing in the complete() callback. >> This patch also contains cam changes to unify the bus_dmamap_load* >> handling in cam drivers. >> >> The arm and mips implementations saw the largest changes since they have >> to track virtual addresses for sync(). Previously this was done in a type >> specific way. Now it is done in a generic way by recording the list of >> virtuals in the map. >> >> I have verified that this patch passes make universe which includes >> several kernel builds from each architecture. I suspect that if I broke >> anything your machine simply won't boot or will throw I/O errors. There >> is little subtlety, it is mostly refactoring. >> > > Testing on armv4/v5 platforms, the patches applied and compiled cleanly, > but fault-abort with a NULL deref on what appears to be the first call > to bus_dmamap_load(). I'll dig deeper into debugging that after > browsing the new code so I can figure out where to start debugging. Can you give me the file and line of the fault? > > Just from an initial quick browsing of the new code for armv4, I notice > these things... > > The new routine __bus_dmamap_mayblock() does (*flags) |= BUS_DMA_WAITOK; > which is a no-op because BUS_DMA_WAITOK is zero. I'm not sure what the > intent was, but I think any modification of the WAIT-related flags would > be conceptually wrong because it overrides the caller's instructions on > whether or not to do a deferred load when resources aren't available. The intent of the original load function seems to be that it is always blocking. Most architectures force the flag. The mbuf and uio load routines don't support blocking at all because the callback lacks the information needed to restart the operation. > > The way you've refactored the mbuf-related load routines, the MD code is > no longer able to tell that it's an mbuf operation. This comes back to > what I said a few days ago... when you look at the current code on > various platforms you see a lot of false similarities. The code isn't > nearly identical because it really all needs to be the same, it's > because it was all copied from the same source, and it doesn't currently > work correctly on arm and mips. With your changes, the ability to > correct the implementation on those platforms is now gone. Perhaps this > can be fixed easily by defining some new flags that the MI routines can > OR-in to give the MD routines more info on what's happening. (Correcting > the mbuf sync handling on arm and mips requires that we know at sync > time that the buffers involved are mbufs, which means we need to know at > map-load time so we can set a flag or a type code or something in the > map that we can examine at sync time.) Why do you need to know it is an mbuf if you have a record of the virtual addresses? The only type specific information was used to find the addresses. See how arm's busdma_machdep-v6 ws implemented. > >> The next step is to allow for dma loading of physical addresses. This >> will permit unmapped I/O. Which is a significant performance optimization >> targeted for 10.0. > > This sounds scary for arm and mips, or more specifically for VIVT cache > platforms that can only do a sync based on virtual addresses. Can you > talk some more about the plans for this? It will be for addresses which were never mapped into kva. To support unmaapped io. There will only be a need for bounce pages, no syncing. Jeff > > -- Ian > > From owner-freebsd-mips@FreeBSD.ORG Sun Dec 9 20:34:55 2012 Return-Path: Delivered-To: mips@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1EEC849E; Sun, 9 Dec 2012 20:34:55 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id B46EA8FC13; Sun, 9 Dec 2012 20:34:42 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id qB9KYZtS016544; Sun, 9 Dec 2012 13:34:41 -0700 (MST) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id qB9KYAou053413; Sun, 9 Dec 2012 13:34:10 -0700 (MST) (envelope-from freebsd@damnhippie.dyndns.org) Subject: Re: Call for testing and review, busdma changes From: Ian Lepore To: Jeff Roberson In-Reply-To: References: <1355077061.87661.320.camel@revolution.hippie.lan> Content-Type: multipart/mixed; boundary="=-jWdke4I5Nd/QjDc9IHG8" Date: Sun, 09 Dec 2012 13:34:10 -0700 Message-ID: <1355085250.87661.345.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port X-Mailman-Approved-At: Mon, 10 Dec 2012 00:21:06 +0000 Cc: powerpc@freebsd.org, marcel@freebsd.org, mips@freebsd.org, John Baldwin , mav@freebsd.org, scottl@freebsd.org, attilio@freebsd.org, kib@freebsd.org, sparc64@freebsd.org, arm@freebsd.org X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Dec 2012 20:34:55 -0000 --=-jWdke4I5Nd/QjDc9IHG8 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit On Sun, 2012-12-09 at 08:48 -1000, Jeff Roberson wrote: > > On Sun, 9 Dec 2012, Ian Lepore wrote: > > > On Sat, 2012-12-08 at 08:51 -1000, Jeff Roberson wrote: > >> Hello, > >> > >> http://people.freebsd.org/~jeff/physbio.diff > >> > >> I have a relative large patch that reforms the busdma API so that new > >> types may be added without modifying every architecture's > >> busdma_machdep.c. It does this by unifying the bus_dmamap_load_buffer() > >> routines so that they may be called from MI code. The MD busdma is then > >> given a chance to do any final processing in the complete() callback. > >> This patch also contains cam changes to unify the bus_dmamap_load* > >> handling in cam drivers. > >> > >> The arm and mips implementations saw the largest changes since they have > >> to track virtual addresses for sync(). Previously this was done in a type > >> specific way. Now it is done in a generic way by recording the list of > >> virtuals in the map. > >> > >> I have verified that this patch passes make universe which includes > >> several kernel builds from each architecture. I suspect that if I broke > >> anything your machine simply won't boot or will throw I/O errors. There > >> is little subtlety, it is mostly refactoring. > >> First an FYI, my replies to this thread aren't making it to the mailing lists, except when quoted by someone replying to them. They're being held for moderator approval because they're going to too many addresses. > > > > Testing on armv4/v5 platforms, the patches applied and compiled cleanly, > > but fault-abort with a NULL deref on what appears to be the first call > > to bus_dmamap_load(). I'll dig deeper into debugging that after > > browsing the new code so I can figure out where to start debugging. > > Can you give me the file and line of the fault? > Better than that, I can give you patches (attached). There were two little glitches... the allocated slist wasn't getting attached to the map, and when building the slist in _bus_dmamap_load_buffer() the slist needs to participate in the coalescing logic near the end of the loop. I'm not positive the attached quick-fix is perfect, but it allows one of my arm units here to boot and run and pass some basic IO smoke tests. I'll be trying it on other arm platforms later today. > > > > Just from an initial quick browsing of the new code for armv4, I notice > > these things... > > > > The new routine __bus_dmamap_mayblock() does (*flags) |= BUS_DMA_WAITOK; > > which is a no-op because BUS_DMA_WAITOK is zero. I'm not sure what the > > intent was, but I think any modification of the WAIT-related flags would > > be conceptually wrong because it overrides the caller's instructions on > > whether or not to do a deferred load when resources aren't available. > > The intent of the original load function seems to be that it is always > blocking. Most architectures force the flag. The mbuf and uio load > routines don't support blocking at all because the callback lacks the > information needed to restart the operation. > The manpage states that bus_dmamap_load() never blocks for any reason. The mbuf and uio flavors say they are variations of bus_dmapmap_load(); I take the implication of that to mean that they also are defined as never blocking. The docs then say that WAITOK vs. NOWAIT control the scheduling of a deferred load, and that NOWAIT is forced in the mbuf and uio cases. Your refactored code already handles that by adding NOWAIT to the caller-specified flags for mbufs and uio. So the code will do the right thing now (with your patches), but only because (*flags) |= BUS_DMA_WAITOK is a no-op. > > > > The way you've refactored the mbuf-related load routines, the MD code is > > no longer able to tell that it's an mbuf operation. This comes back to > > what I said a few days ago... when you look at the current code on > > various platforms you see a lot of false similarities. The code isn't > > nearly identical because it really all needs to be the same, it's > > because it was all copied from the same source, and it doesn't currently > > work correctly on arm and mips. With your changes, the ability to > > correct the implementation on those platforms is now gone. Perhaps this > > can be fixed easily by defining some new flags that the MI routines can > > OR-in to give the MD routines more info on what's happening. (Correcting > > the mbuf sync handling on arm and mips requires that we know at sync > > time that the buffers involved are mbufs, which means we need to know at > > map-load time so we can set a flag or a type code or something in the > > map that we can examine at sync time.) > > Why do you need to know it is an mbuf if you have a record of the virtual > addresses? The only type specific information was used to find the > addresses. See how arm's busdma_machdep-v6 ws implemented. > Because there is a special rule for mbufs on VIVT-cache architectures: the address of the data buffer is not aligned to a cacheline boundary, but the sync code is allowed to treat the buffer as if it is aligned and thus avoid invoking the partial cacheline flush algorithm. That algorithm was shown a few months ago to be fatally flawed, and we're on a path (albeit in super-slow-motion) to eliminate it. I have patches to fix the mbuf partial sync and other things related to partial sync problems, and my next step will be to try to rework them to fit in with what you've done. Based on what I've seen so far in your refactoring, I think just passing a flag to say it's an mbuf, from the MI to the MD mapping routine, will provide enough info. > > > >> The next step is to allow for dma loading of physical addresses. This > >> will permit unmapped I/O. Which is a significant performance optimization > >> targeted for 10.0. > > > > This sounds scary for arm and mips, or more specifically for VIVT cache > > platforms that can only do a sync based on virtual addresses. Can you > > talk some more about the plans for this? > > It will be for addresses which were never mapped into kva. To support > unmaapped io. There will only be a need for bounce pages, no syncing. > Can the pages be mapped into any non-kernel vmspaces? If they aren't mapped into any vmspace at all, then I think there'll be no implications for VIVT cache architectures. If they appear in any pmap at all then there may be problems. -- Ian --=-jWdke4I5Nd/QjDc9IHG8 Content-Disposition: inline; filename="physbio_armv4fix.diff" Content-Type: text/x-patch; name="physbio_armv4fix.diff"; charset="us-ascii" Content-Transfer-Encoding: 7bit diff -r bee560bf0159 sys/arm/arm/busdma_machdep.c --- a/sys/arm/arm/busdma_machdep.c Sun Dec 09 12:12:11 2012 -0700 +++ b/sys/arm/arm/busdma_machdep.c Sun Dec 09 13:01:12 2012 -0700 @@ -310,9 +310,10 @@ static __inline bus_dmamap_t map->flags = DMAMAP_ALLOCATED; } else map->flags = 0; - if (map != NULL) + if (map != NULL) { STAILQ_INIT(&map->bpages); - else + map->slist = slist; + } else free(slist, M_DEVBUF); return (map); } @@ -764,7 +765,6 @@ int { bus_size_t sgsize; bus_addr_t curaddr, baddr, bmask; - struct sync_list *sl; vm_offset_t vaddr = (vm_offset_t)buf; int seg; int error = 0; @@ -855,14 +855,6 @@ int if (((dmat->flags & BUS_DMA_COULD_BOUNCE) != 0) && map->pagesneeded != 0 && run_filter(dmat, curaddr)) { curaddr = add_bounce_page(dmat, map, vaddr, sgsize); - } else { - sl = &map->slist[map->sync_count]; - if (++map->sync_count > dmat->nsegments) - goto cleanup; - printf("sl %p count %d nseg %d\n", sl, map->sync_count, dmat->nsegments); - sl->vaddr = vaddr; - sl->datacount = sgsize; - sl->busaddr = curaddr; } if (dmat->ranges) { @@ -891,10 +883,15 @@ int (segs[seg].ds_addr & bmask) == (curaddr & bmask))) { segs[seg].ds_len += sgsize; + map->slist[seg].datacount += sgsize; goto segdone; } else { if (++seg >= dmat->nsegments) break; + ++map->sync_count; + map->slist[seg].vaddr = vaddr; + map->slist[seg].busaddr = curaddr; + map->slist[seg].datacount = sgsize; segs[seg].ds_addr = curaddr; segs[seg].ds_len = sgsize; } @@ -906,7 +903,6 @@ segdone: } *segp = seg; -cleanup: /* * Did we fit? */ --=-jWdke4I5Nd/QjDc9IHG8-- From owner-freebsd-mips@FreeBSD.ORG Sun Dec 9 18:18:31 2012 Return-Path: Delivered-To: mips@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7377D882; Sun, 9 Dec 2012 18:18:31 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id E5AC38FC12; Sun, 9 Dec 2012 18:18:18 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id qB9II553012970; Sun, 9 Dec 2012 11:18:11 -0700 (MST) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id qB9IHfHu053325; Sun, 9 Dec 2012 11:17:41 -0700 (MST) (envelope-from freebsd@damnhippie.dyndns.org) Subject: Re: Call for testing and review, busdma changes From: Ian Lepore To: Jeff Roberson In-Reply-To: References: Content-Type: text/plain; charset="us-ascii" Date: Sun, 09 Dec 2012 11:17:41 -0700 Message-ID: <1355077061.87661.320.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Mon, 10 Dec 2012 00:28:50 +0000 Cc: powerpc@freebsd.org, marcel@freebsd.org, mips@freebsd.org, John Baldwin , mav@freebsd.org, scottl@freebsd.org, attilio@freebsd.org, kib@freebsd.org, sparc64@freebsd.org, arm@freebsd.org X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Dec 2012 18:18:31 -0000 On Sat, 2012-12-08 at 08:51 -1000, Jeff Roberson wrote: > Hello, > > http://people.freebsd.org/~jeff/physbio.diff > > I have a relative large patch that reforms the busdma API so that new > types may be added without modifying every architecture's > busdma_machdep.c. It does this by unifying the bus_dmamap_load_buffer() > routines so that they may be called from MI code. The MD busdma is then > given a chance to do any final processing in the complete() callback. > This patch also contains cam changes to unify the bus_dmamap_load* > handling in cam drivers. > > The arm and mips implementations saw the largest changes since they have > to track virtual addresses for sync(). Previously this was done in a type > specific way. Now it is done in a generic way by recording the list of > virtuals in the map. > > I have verified that this patch passes make universe which includes > several kernel builds from each architecture. I suspect that if I broke > anything your machine simply won't boot or will throw I/O errors. There > is little subtlety, it is mostly refactoring. > Testing on armv4/v5 platforms, the patches applied and compiled cleanly, but fault-abort with a NULL deref on what appears to be the first call to bus_dmamap_load(). I'll dig deeper into debugging that after browsing the new code so I can figure out where to start debugging. Just from an initial quick browsing of the new code for armv4, I notice these things... The new routine __bus_dmamap_mayblock() does (*flags) |= BUS_DMA_WAITOK; which is a no-op because BUS_DMA_WAITOK is zero. I'm not sure what the intent was, but I think any modification of the WAIT-related flags would be conceptually wrong because it overrides the caller's instructions on whether or not to do a deferred load when resources aren't available. The way you've refactored the mbuf-related load routines, the MD code is no longer able to tell that it's an mbuf operation. This comes back to what I said a few days ago... when you look at the current code on various platforms you see a lot of false similarities. The code isn't nearly identical because it really all needs to be the same, it's because it was all copied from the same source, and it doesn't currently work correctly on arm and mips. With your changes, the ability to correct the implementation on those platforms is now gone. Perhaps this can be fixed easily by defining some new flags that the MI routines can OR-in to give the MD routines more info on what's happening. (Correcting the mbuf sync handling on arm and mips requires that we know at sync time that the buffers involved are mbufs, which means we need to know at map-load time so we can set a flag or a type code or something in the map that we can examine at sync time.) > The next step is to allow for dma loading of physical addresses. This > will permit unmapped I/O. Which is a significant performance optimization > targeted for 10.0. This sounds scary for arm and mips, or more specifically for VIVT cache platforms that can only do a sync based on virtual addresses. Can you talk some more about the plans for this? -- Ian From owner-freebsd-mips@FreeBSD.ORG Mon Dec 10 00:54:18 2012 Return-Path: Delivered-To: mips@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 30ACD2D7 for ; Mon, 10 Dec 2012 00:54:18 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ie0-f181.google.com (mail-ie0-f181.google.com [209.85.223.181]) by mx1.freebsd.org (Postfix) with ESMTP id DFE888FC17 for ; Mon, 10 Dec 2012 00:54:17 +0000 (UTC) Received: by mail-ie0-f181.google.com with SMTP id 16so7160708iea.12 for ; Sun, 09 Dec 2012 16:54:11 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=NQ/SFUzHRS12x8NX6tNgjKotHD2QRO5QpoGfvtwLETk=; b=Dj5OdkOY4janLQq18h6vVvgCUuVijy0cYfmyt7tm4dl4w3mWmcMvSELG3h6LqxxS8H i/MmMj/ASQHHk9QO7mVNeicWusx6lMw8coervvpysnViQbwNp3dz15B9m//gKcjbRAny 2I5N34wqJfaZx1PmvDzRZqS3fDhvxRH1vxLh5vDbZEWjTsyHrZlbnNo0hr6iqO1WWAEb myNCna5k4DP1razv/2nmRfhIVd6Gr7MlJIvOQQhwLedPHf2H79fuVejGT4WVGgTcPYFP 5jtwLXAV0q4mCrn+v4HJw+X78cvd2rLE3i5oahFGSzalXMfV/Utl6/dUT84jzsLhlb3E KBmA== Received: by 10.50.157.130 with SMTP id wm2mr7911478igb.0.1355100851225; Sun, 09 Dec 2012 16:54:11 -0800 (PST) Received: from 53.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id l8sm5263808igo.13.2012.12.09.16.54.08 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 09 Dec 2012 16:54:10 -0800 (PST) Sender: Warner Losh Subject: Re: Call for testing and review, busdma changes Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: Date: Sun, 9 Dec 2012 17:54:07 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <1355077061.87661.320.camel@revolution.hippie.lan> To: Jeff Roberson X-Mailer: Apple Mail (2.1085) X-Gm-Message-State: ALoCoQnl3LrA9hFsZsFjIaspHaGNRV4wmqUgZ8A9falTWTaAL4ESkIt5EwFxCYvRwPLIpHwdOceq X-Mailman-Approved-At: Mon, 10 Dec 2012 01:01:54 +0000 Cc: powerpc@freebsd.org, marcel@freebsd.org, mips@freebsd.org, John Baldwin , mav@freebsd.org, scottl@freebsd.org, attilio@freebsd.org, kib@freebsd.org, sparc64@freebsd.org, arm@freebsd.org X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 00:54:18 -0000 [[ looks like Ian answered the other questions ]] On Dec 9, 2012, at 11:48 AM, Jeff Roberson wrote: >>> The next step is to allow for dma loading of physical addresses. = This >>> will permit unmapped I/O. Which is a significant performance = optimization >>> targeted for 10.0. >>=20 >> This sounds scary for arm and mips, or more specifically for VIVT = cache >> platforms that can only do a sync based on virtual addresses. Can = you >> talk some more about the plans for this? >=20 > It will be for addresses which were never mapped into kva. To support = unmaapped io. There will only be a need for bounce pages, no syncing. If there's a virtual mapping at all for the page (not just a kva), then = we need to flush/invalidate cache for those pages for safety unless we = know for sure there's nothing in the cache. It these are pages that = have never been mapped (say for zero copy between storage and network = controllers), then there won't be a problem. The whole reason for cache = jockeying is to make sure that any pending writes to the data are = flushed, and that any cached notion of the pages are invalidated once = the read is done. This is more important for mbuf operations, = operations with the USB stack or anything else that marshals data it = generates to be consumed by the stack/hardware. Warner From owner-freebsd-mips@FreeBSD.ORG Mon Dec 10 11:06:46 2012 Return-Path: Delivered-To: freebsd-mips@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CFE35F02 for ; Mon, 10 Dec 2012 11:06:46 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id B40368FC1D for ; Mon, 10 Dec 2012 11:06:46 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id qBAB6ksh064292 for ; Mon, 10 Dec 2012 11:06:46 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id qBAB6klG064290 for freebsd-mips@FreeBSD.org; Mon, 10 Dec 2012 11:06:46 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 10 Dec 2012 11:06:46 GMT Message-Id: <201212101106.qBAB6klG064290@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-mips@FreeBSD.org Subject: Current problem reports assigned to freebsd-mips@FreeBSD.org X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 11:06:46 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/165951 mips [ar913x] [ath] DDR flush isn't being done for the WMAC p kern/163670 mips [mips][arge] arge can't allocate ring buffer on multip 2 problems total. From owner-freebsd-mips@FreeBSD.ORG Tue Dec 11 05:56:45 2012 Return-Path: Delivered-To: mips@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1D5D87E1; Tue, 11 Dec 2012 05:56:45 +0000 (UTC) (envelope-from tinderbox@freebsd.org) Received: from freebsd-stable.sentex.ca (freebsd-stable.sentex.ca [IPv6:2607:f3e0:0:3::6502:9b]) by mx1.freebsd.org (Postfix) with ESMTP id D2E798FC12; Tue, 11 Dec 2012 05:56:44 +0000 (UTC) Received: from freebsd-stable.sentex.ca (localhost [127.0.0.1]) by freebsd-stable.sentex.ca (8.14.5/8.14.5) with ESMTP id qBB5uiSX082042; Tue, 11 Dec 2012 05:56:44 GMT (envelope-from tinderbox@freebsd.org) Received: (from tinderbox@localhost) by freebsd-stable.sentex.ca (8.14.5/8.14.5/Submit) id qBB5ui34082041; Tue, 11 Dec 2012 05:56:44 GMT (envelope-from tinderbox@freebsd.org) Date: Tue, 11 Dec 2012 05:56:44 GMT Message-Id: <201212110556.qBB5ui34082041@freebsd-stable.sentex.ca> X-Authentication-Warning: freebsd-stable.sentex.ca: tinderbox set sender to FreeBSD Tinderbox using -f Sender: FreeBSD Tinderbox From: FreeBSD Tinderbox To: FreeBSD Tinderbox , , Subject: [releng_9 tinderbox] failure on mips/mips Precedence: bulk X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.14 List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Dec 2012 05:56:45 -0000 TB --- 2012-12-11 05:20:23 - tinderbox 2.9 running on freebsd-stable.sentex.ca TB --- 2012-12-11 05:20:23 - FreeBSD freebsd-stable.sentex.ca 8.3-STABLE FreeBSD 8.3-STABLE #0: Tue Oct 16 17:37:58 UTC 2012 mdtancsa@freebsd-stable.sentex.ca:/usr/obj/usr/src/sys/server amd64 TB --- 2012-12-11 05:20:23 - starting RELENG_9 tinderbox run for mips/mips TB --- 2012-12-11 05:20:23 - cleaning the object tree TB --- 2012-12-11 05:20:23 - checking out /src from svn://svn.freebsd.org/base/stable/9 TB --- 2012-12-11 05:20:23 - cd /tinderbox/RELENG_9/mips/mips TB --- 2012-12-11 05:20:23 - /usr/local/bin/svn cleanup /src TB --- 2012-12-11 05:21:04 - /usr/local/bin/svn update /src TB --- 2012-12-11 05:21:11 - At svn revision 244110 TB --- 2012-12-11 05:21:12 - building world TB --- 2012-12-11 05:21:12 - CROSS_BUILD_TESTING=YES TB --- 2012-12-11 05:21:12 - MAKEOBJDIRPREFIX=/obj TB --- 2012-12-11 05:21:12 - PATH=/usr/bin:/usr/sbin:/bin:/sbin TB --- 2012-12-11 05:21:12 - SRCCONF=/dev/null TB --- 2012-12-11 05:21:12 - TARGET=mips TB --- 2012-12-11 05:21:12 - TARGET_ARCH=mips TB --- 2012-12-11 05:21:12 - TZ=UTC TB --- 2012-12-11 05:21:12 - __MAKE_CONF=/dev/null TB --- 2012-12-11 05:21:12 - cd /src TB --- 2012-12-11 05:21:12 - /usr/bin/make -B buildworld >>> World build started on Tue Dec 11 05:21:13 UTC 2012 >>> Rebuilding the temporary build tree >>> stage 1.1: legacy release compatibility shims >>> stage 1.2: bootstrap tools >>> stage 2.1: cleaning up the object tree >>> stage 2.2: rebuilding the object tree >>> stage 2.3: build tools >>> stage 3: cross tools >>> stage 4.1: building includes >>> stage 4.2: building libraries >>> stage 4.3: make dependencies >>> stage 4.4: building everything [...] ===> bin/expr (all) cc -O -pipe -G0 -std=gnu99 -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -c expr.c cc1: warnings being treated as errors /src/bin/expr/expr.y: In function 'main': /src/bin/expr/expr.y:291: warning: implicit declaration of function 'yyparse' /src/bin/expr/expr.y:291: warning: nested extern declaration of 'yyparse' expr.c: At top level: expr.c:813: warning: no previous prototype for 'yyparse' *** Error code 1 Stop in /src/bin/expr. *** Error code 1 Stop in /src/bin. *** Error code 1 Stop in /src. *** Error code 1 Stop in /src. *** Error code 1 Stop in /src. TB --- 2012-12-11 05:56:44 - WARNING: /usr/bin/make returned exit code 1 TB --- 2012-12-11 05:56:44 - ERROR: failed to build world TB --- 2012-12-11 05:56:44 - 1521.03 user 339.12 system 2181.02 real http://tinderbox.freebsd.org/tinderbox-releng_9-RELENG_9-mips-mips.full From owner-freebsd-mips@FreeBSD.ORG Tue Dec 11 16:25:34 2012 Return-Path: Delivered-To: freebsd-mips@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 624B1A0C for ; Tue, 11 Dec 2012 16:25:34 +0000 (UTC) (envelope-from monthadar@gmail.com) Received: from mail-wi0-f174.google.com (mail-wi0-f174.google.com [209.85.212.174]) by mx1.freebsd.org (Postfix) with ESMTP id E04538FC14 for ; Tue, 11 Dec 2012 16:25:33 +0000 (UTC) Received: by mail-wi0-f174.google.com with SMTP id hm9so2125765wib.13 for ; Tue, 11 Dec 2012 08:25:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=D/rqVQlMTGmq96iq0XsLo8iFEad0rnU4UAQHppKcJbE=; b=N12JwI61H1/U7ytnFF8WTPgazRXFvJusGW+JxBfSU2TTOGuMO2xv/Djpg4XcARNEkk k310wnIqz9i+/xrM5tPXNvPw/yfwCh8ItHkNkkMOfgfwLeN17aaUtbhOkrrqmTcPdvvP kwVT+i9dGsV+ylB4effA4lWvs/9f5QD4xaEkoMz2i1ACFTTa3EZKtG/S2Tu0hVNa2T1w y5aTrUw64xTfxZ0SsnEGgFjXnjV3dcF41xm3+AWyBv6NoQvRZ5u0BOAUBeApDnDArL5E wRaD5uTWrYx8dkZwytBiILvtLrRD+dacats41PJyDXARA6z0KAsEXiL5e3qi5vJ2QGjX UT1w== MIME-Version: 1.0 Received: by 10.180.103.106 with SMTP id fv10mr17612788wib.19.1355243126932; Tue, 11 Dec 2012 08:25:26 -0800 (PST) Received: by 10.227.59.19 with HTTP; Tue, 11 Dec 2012 08:25:26 -0800 (PST) In-Reply-To: References: <20120607184354.GB90876@zibbi.meraka.csir.co.za> Date: Tue, 11 Dec 2012 17:25:26 +0100 Message-ID: Subject: Re: Routerboard RB433AH From: Monthadar Al Jaberi To: Luiz Otavio O Souza Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-mips@freebsd.org X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Dec 2012 16:25:34 -0000 On Sat, Dec 8, 2012 at 9:41 PM, Luiz Otavio O Souza wrote: > On 7 December 2012 12:27, Monthadar Al Jaberi wrote: >> Sorry for being too late, have you solved this issue? >> >> br, >> >> On Thu, Jun 7, 2012 at 8:43 PM, John Hay wrote: >>> Hi, >>> >>> I found a RB433AH that is not doing anything and was wondering if one >>> can run FreeBSD on it. For a start I thought of just a NFS boot. I >>> started with the AR71XX_BASE config file and removed the USB and >>> other stuff that I think I do not need and can get it load the kernel >>> and start it. But it looks like there is a problem with the ethernet >>> because I get stuck at: >>> >>> Timecounters tick every 1.000 msec >>> Sending DHCP Discover packet from interface arge0 (00:0c:42:90:f9:de) >>> arge0: link state changed to DOWN >>> arge0: link state changed to UP >>> DHCP/BOOTP timeout for server 255.255.255.255 >>> DHCP/BOOTP timeout for server 255.255.255.255 >>> ... >>> >>> This just keeps on repeating. >>> >>> Maybe I need some specific setup for the ethernet / phy. Anybody with >>> ideas? >>> > > Yeah, sorry for the delay... > > It's been a while since the last boot of my RB433UAH, so my kernel and > hints are not really updated. > > But back then, i have this setup for the PHYs: > > > # arge0 > hint.arge.0.at="nexus0" > hint.arge.0.maddr=0x19000000 > hint.arge.0.msize=0x1000 > hint.arge.0.irq=2 > # PHY0, PHY1 > hint.arge.0.phymask=0x03 > > # arge1 > hint.arge.1.at="nexus0" > hint.arge.1.maddr=0x1A000000 > hint.arge.1.msize=0x1000 > hint.arge.1.irq=3 > # PHY4 > hint.arge.1.phymask=0x10 > > > Note that on RB433,RB450 the wan port is connected to arge1 and not to > arge0 as usually, so you need to adjust your bootp setup. > > In order to keep the original mac address you'll the following patch: > http://loos.no-ip.org/rb/routerboot.diff and then add 'options > AR71XX_ROUTERBOOT' to kernel file. Thnx! I applied your patch on latest current. And your hint file plus the options define. But I am having trouble booting from nfs. kernel boots fine, except arge1 is can't attach: arge1: at mem 0x1a000000-0x1a000fff irq 3 on nexus0 arge1: finishing attachment, phymask 0010, proxy null arge1: unable to attach PHY 4: 6 device_attach: arge1 attach returned 6 And I get: Sending DHCP Discover packet from interface arge0 (d4:ca:6d:57:dc:d4) arge0: link state changed to DOWN DHCP/BOOTP timeout for server 255.255.255.255 DHCP/BOOTP timeout for server 255.255.255.255 ... I don't see any activity in tcpdump. attaching my config & hints file based on AR71XX. br, # # AR71XX -- Kernel configuration file for FreeBSD/MIPS for Atheros 71xx systems # # This includes all the common drivers for the AR71XX boards along with # the usb, net80211 and atheros driver code. # # $FreeBSD$ # machine mips mips ident RB433AH_MFS cpu CPU_MIPS4KC makeoptions KERNLOADADDR=0x80050000 options HZ=1000 options HWPMC_HOOKS files "../atheros/files.ar71xx" # For now, hints are per-board. hints "RB433AH.hints" makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols # Build these as modules so small platform builds will have the # modules already built. makeoptions MODULES_OVERRIDE="random gpio ar71xx if_gif if_gre if_bridge bridgestp usb wlan wlan_xauth wlan_acl wlan_wep wlan_tkip wlan_ccmp wlan_rssadapt wlan_amrr ath ath_pci" options DDB options KDB options SCHED_4BSD #4BSD scheduler options INET #InterNETworking #options INET6 # IPv6 # options NFS_CL #Network Filesystem Client options PSEUDOFS #Pseudo-filesystem framework options _KPOSIX_PRIORITY_SCHEDULING #Posix P1003_1B real-time extensions # options NFS_LEGACYRPC # Debugging for use in -current options INVARIANTS options INVARIANT_SUPPORT options WITNESS options WITNESS_SKIPSPIN options DEBUG_REDZONE options DEBUG_MEMGUARD options FFS #Berkeley Fast Filesystem # options SOFTUPDATES #Enable FFS soft updates support # options UFS_ACL #Support for access control lists # options UFS_DIRHASH #Improve performance on big directories # options MSDOSFS # Read MSDOS filesystems; useful for USB/CF device pci device ar71xx_pci # RTC - requires hackery in the spibus code to work device pcf2123_rtc # GEOM modules device geom_redboot # to get access to the SPI flash partitions device geom_uzip # compressed in-memory filesystem hackery! options GEOM_UZIP ## Boot from the first MFS uzip #options ROOTDEVNAME=\"ufs:md0.uzip\" #options MD_ROOT #options MD_ROOT_SIZE="9000" # Boot from NFS options NFSLOCKD #Network Lock Manager options NFSCLIENT #Network Filesystem Client options NFS_ROOT options BOOTP options BOOTP_NFSROOT options BOOTP_NFSV3 #options BOOTP_WIRED_TO=arge1 options BOOTP_COMPAT options ROOTDEVNAME=\"nfs:172.16.0.101:/usr/obj/rb433ah/nfs\" # 802.11 framework options IEEE80211_DEBUG options IEEE80211_ALQ options IEEE80211_SUPPORT_MESH # This option is currently broken for if_ath_tx. options IEEE80211_SUPPORT_TDMA options IEEE80211_AMPDU_AGE device wlan # 802.11 support device wlan_wep # 802.11 WEP support device wlan_ccmp # 802.11 CCMP support device wlan_tkip # 802.11 TKIP support device wlan_xauth # 802.11 hostap support device wlan_acl # 802.11 ACL support # Atheros wireless NICs device ath # Atheros interface support device ath_pci # Atheros PCI/Cardbus bus options ATH_DEBUG options ATH_DIAGAPI options ATH_ENABLE_11N options AH_DEBUG options AH_DEBUG_ALQ options ALQ device ath_hal option AH_SUPPORT_AR5416 device ath_rate_sample option AH_RXCFG_SDMAMW_4BYTES option AH_AR5416_INTERRUPT_MITIGATION # There's no DFS radar detection support yet so this won't actually # detect radars. It however does enable the rest of the channel change # machinery so DFS can be debugged. option ATH_ENABLE_DFS device mii device arge option ARGE_DEBUG #device usb #options USB_EHCI_BIG_ENDIAN_DESC # handle big-endian byte order #options USB_DEBUG #options USB_HOST_ALIGN=32 # AR71XX (MIPS in general?) requires this #device ehci device scbus #device umass device da # On-board SPI flash device spibus device ar71xx_spi device mx25l device ar71xx_wdog device uart device uart_ar71xx device loop device ether device md device bpf device random device if_bridge device gif # ip[46] in ip[46] tunneling protocol device gre # generic encapsulation - only for IPv4 in IPv4 though atm options ARGE_DEBUG # Enable if_arge debugging for now # Enable GPIO device gpio device gpioled options AR71XX_ROUTERBOOT # # $FreeBSD$ # hint.apb.0.at="nexus0" hint.apb.0.irq=4 # uart0 hint.uart.0.at="apb0" # see atheros/uart_cpu_ar71xx.c why +3 hint.uart.0.maddr=0x18020003 hint.uart.0.msize=0x18 hint.uart.0.irq=3 #ohci hint.ohci.0.at="apb0" hint.ohci.0.maddr=0x1c000000 hint.ohci.0.msize=0x01000000 hint.ohci.0.irq=6 #ehci hint.ehci.0.at="nexus0" hint.ehci.0.maddr=0x1b000000 hint.ehci.0.msize=0x01000000 hint.ehci.0.irq=1 # pci hint.pcib.0.at="nexus0" hint.pcib.0.irq=0 # arge0 hint.arge.0.at="nexus0" hint.arge.0.maddr=0x19000000 hint.arge.0.msize=0x1000 hint.arge.0.irq=2 # PHY0, PHY1 hint.arge.0.phymask=0x03 # arge0 mdio bus hint.argemdio.0.at="nexus0" hint.argemdio.0.maddr=0x19000000 hint.argemdio.0.msize=0x1000 hint.argemdio.0.order=0 # arge0: dedicated switch port hint.arge.0.phymask=0x00 # PHY4 hint.arge.0.miimode=3 # RGMII hint.arge.0.mdio=mdioproxy1 # .. off of the switch mdiobus # phymask, media and fduplex depend upon the specific # board. # So each board will override the settings as needed. # arge1 hint.arge.1.at="nexus0" hint.arge.1.maddr=0x1A000000 hint.arge.1.msize=0x1000 hint.arge.1.irq=3 # PHY4 hint.arge.1.phymask=0x10 #hint.arswitch.0.is_7240=0 #hint.arswitch.0.numphys=4 #hint.arswitch.0.phy4cpu=1 #hint.arswitch.0.is_rgmii=1 #hint.arswitch.0.is_gmii=0 # SPI flash hint.spi.0.at="nexus0" hint.spi.0.maddr=0x1f000000 hint.spi.0.msize=0x10 hint.mx25l.0.at="spibus0" hint.mx25l.0.cs=0 # Watchdog hint.ar71xx_wdog.0.at="nexus0" # GPIO hint.gpio.0.at="apb0" hint.gpio.0.maddr=0x18040000 hint.gpio.0.msize=0x1000 hint.gpio.0.irq=2 # Each board should override the GPIO bus pins with the configuration # relevant to it. Thus no pins are defined here. # hwpmc device hint.ar71xx_pmc.0.at="apb0" hint.ar71xx_pmc.0.irq=5 > > The etherswitch doesn't support the switch chip on RB433 right now, > but i'm working on this. > > Luiz -- Monthadar Al Jaberi