Date: Thu, 27 Oct 2005 14:36:52 +0200 From: Jacques Caron <jc@oxado.com> To: Scott Long <scottl@samsco.org> Cc: freebsd-amd64@FreeBSD.ORG, Søren Schmidt <sos@FreeBSD.ORG> Subject: Re: busdma dflt_lock on amd64 > 4 GB Message-ID: <6.2.3.4.0.20051027121824.03d34dd8@wheresmymailserver.com> In-Reply-To: <42A1B51D-5A5E-4849-96D0-BC5C2DD1AE97@FreeBSD.ORG> References: <6.2.3.4.0.20051025171333.03a15490@pop.interactivemediafactory.net> <6.2.3.4.0.20051026131012.03a80a20@pop.interactivemediafactory.net> <435F8E06.9060507@samsco.org> <08A81034-AB5D-4BFC-8F53-21501073D674@FreeBSD.ORG> <435FA542.3030209@samsco.org> <6.2.3.4.0.20051026180325.03ad7558@wheresmymailserver.com> <42A1B51D-5A5E-4849-96D0-BC5C2DD1AE97@FreeBSD.ORG>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi, At 20:01 26/10/2005, Søren Schmidt wrote: ATA does all the tag/map creates/allocs/loads for the SGlist and >simple workspace stuff at channel attach time, there are NO further >creates or allocs after that. So that is exactly what I would expect >the ALLOCNOW flags to make busdma support, if that doesn't work >busdma needs to be fixed IMNHO. So I tested with ALLOCNOW set on all 4 tags, and the end result is exactly the same, ch->dma->load fails (due to EINPROGRESS, most probably), and a panic is trigerred by busdma dflt_lock . And I agree that this is a busdma bug, as the busdma manpage says, in the description of bus_dmamap_load: EINPROGRESS The mapping has been deferred for lack of resources. The callback will be called as soon as resources are available. Callbacks are serviced in FIFO order. DMA maps created from DMA tags that are allocated with the BUS_DMA_ALLOCNOW flag will never return this status for a load operation. So either the busdma API needs to be changed and ALLOCNOW no longer means that EINPROGRESS cannot happen (and that might break other things), or busdma needs to be fixed to conform to the documentation. Interestingly, if I remove the ALLOCNOW in all tag creations in ata, then: - ata attaches during boot are a bit longer (like fxp which is really long) - 320 bounces pages are allocated - I can do apparently do everything I want and not panic the machine. - I suppose that since there's plenty of available bounce pages, I should not have problems with ENOMEM. I'll go with that configuration for now, but that's most probably not the correct fix, just a workaround. The correct fix is to make busdma work as documented. Here I see several things to look into: - make sure enough ressources are actually allocated at tag create time if ALLOCNOW is set (this does not necessarily mean allocating maxsize additional pages, some logic about already required pages needs to be added I think) - let bus_dmamap_create allocate additional bounce pages if needed even if ALLOCNOW was set initially (i.e. don't use BUS_DMA_MIN_ALLOC_COMP), though I'm not sure the page allocation logic there is entirely correct either, as it will allocate additional pages even if there are already enough - in _bus_dmamap_load_buffer, consider ALLOCNOW like the undocumented NOWAIT, i.e. return ENOMEM if enough ressources aren't available (which I believe wouldn't be correct either: ALLOCNOW should prevent EINPROGRESS but also ENOMEM at map load time and should return ENOMEM immediately if there aren't enough ressources, not later). Another thing to look into is why bounce page allocation is that long (at least I believe that's what is taking so long), but I haven't looked at that bit at all yet. I also wonder if allocating bounce pages on <=4G configurations is really needed/useful? There are probably situations where this is needed, but I'm not sure I understand why. Jacques.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?6.2.3.4.0.20051027121824.03d34dd8>