Date: Sat, 26 May 2018 23:54:17 +0530 From: Pratyush Yadav <pratyush@freebsd.org> To: Warner Losh <imp@bsdimp.com> Cc: freebsd-hackers@freebsd.org, Akshay Jaggi <jaggi@freebsd.org>, Edward Napierala <trasz@freebsd.org> Subject: Re: Why is there no bus_dmamap_load_sg()? Message-ID: <CA%2BX=3TL5sRKUceQ_YF1dYyVc6gNV_=y7E7tjMv=kbY_izfpg1A@mail.gmail.com> In-Reply-To: <CANCZdfrt6eCVx3MAO=qLM5gXjSbx4hzGXQdePHgtktjFKWJA8g@mail.gmail.com> References: <CA%2BX=3TK_v2Sby0fP57XB2T2WQpJ6mebE4doF6AiVJNHYHxPU8A@mail.gmail.com> <CANCZdfrt6eCVx3MAO=qLM5gXjSbx4hzGXQdePHgtktjFKWJA8g@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Warner, On Sat 26 May, 2018, 7:40 PM Warner Losh, <imp@bsdimp.com> wrote: > > > On Sat, May 26, 2018 at 3:07 AM, Pratyush Yadav <pratyush@freebsd.org> > wrote: > >> I was looking at the man page and I notice the function >> bus_dmamap_load_mbuf_sg() which directly returns the segment array to >> the client. But a similar variant of bus_dmamap_load() is missing. >> This variant would be useful to me and help me in my work. >> >> Is there a reason the sg variant was only implemented for mbufs? >> >> I was looking at the code in sys/kern/subr_bus_dma.c and a >> bus_dmamap_load_sg() does not look too difficult to implement. Is it >> ok if I submit a patch implementing it? >> > > The biggest reason is because of the failure modes. If there's not > sufficient memory or other resources to honor the request, we defer until > there is such memory / resources and then call the callback. There's no > convenient hook the drivers can connect to so they can retry the allocation > when memory becomes available (busdma has one since eventually a request > will complete and unload). The original port was for CAM SIMs which had > stringent requirements to always make progress on I/Os to avoid deadlock. > Rather than put this tricky code in all the CAM SIMs, it's centralized to > retry as soon and as robustly as it can. > > Network drivers, however, don't have such requirements and often don't > care. If they can't load a map, they can put the packet back on the > transmit queue (or whatever), set a flag and return without putting it into > the NIC's buffer rings. Eventually, a packet will finish transmitting and > there will be a chance to send it again (or one of a number of other > events, I'm glossing heavily here). It's a better fit for network drivers > than storage drivers. Since bus_dmamap_load_mbuf* is used only for network > drivers, it makes sense. > > So, it would only make sense to add if you can cope with BUS_DMA_NOWAIT > always, and that error handling is simpler than letting busdma call you > back when the shortage is over. Is that the case for your port? > I'm not sure. I looked at the OpenBSD code and they always call with a BUS_DMA_NOWAIT flag, but I'm not sure what the answer is for FreeBSD. Or is it just because OpenBSD doesn't have this sort of interface (the > callback difference dates back to the initial port to FreeBSD by Justin > Gibbs because it never got folded back into NetBSD for reasons too long, > too old and only half remembered by me for this post)? Can the code you are > porting cope with error returns and retry again later? > The other problem with using callbacks is that because I pass the xen-specific callback to the dma load, the driver using the xen specific dma operation can't pass it's own callback. I thought using the _sg() variant would allow the driver to use a callback. But the way you put it, that kind of defeats the purpose of the callback, which is to use it when allocation is defered. The bottom line is, I need to re-evaluate. I will spend some more time on this and discuss it with my mentors. I'll let you (and everyone else) know if I do decide we need it. Thanks for the enlightening answer. Regards, Pratyush Yadav
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2BX=3TL5sRKUceQ_YF1dYyVc6gNV_=y7E7tjMv=kbY_izfpg1A>