Date: Tue, 12 Oct 1999 14:03:01 -0600 From: "Justin T. Gibbs" <gibbs@caspian.plutotech.com> To: Gerard Roudier <groudier@club-internet.fr> Cc: "Justin T. Gibbs" <gibbs@plutotech.com>, scsi@FreeBSD.org Subject: Re: Driver for GDT6517RD RAID controller Message-ID: <199910122003.OAA02480@caspian.plutotech.com> In-Reply-To: Your message of "Tue, 12 Oct 1999 21:53:46 %2B0200." <Pine.LNX.3.95.991012210439.377A-100000@localhost>
next in thread | previous in thread | raw e-mail | index | archive | help
>> Having every driver create their own S/G list is problematic when kern= el >> interfaces change. What happens when we change our clustering code to= >> pass down a list of I/O requests to be combined? Hopefully this will >> just result in the drivers calling a slightly different bus_dma method= =2E > >The bus dma method I have seen requires the caller to wait for a call ba= ck >for the resources to be available. This moves the burden of having to de= al >with such resources leakage in the SIM. This should be an option, in my >opinion and optionnally handled from the caller (that will provide a >scatter list to the SIM). How does the caller know all of the proper parameters your driver needs in order to properly perform the I/O? The caller is some generic peripheral driver that doesn't want to know about the different chipset, bridge, device interactions. In fact, CAM doesn't want to know about you= r particular hardware either, only that your SIM honors the expected CAM API. >I read that such bus dma method was also intended to deal with host brid= ge >that does not ensure cache consistency or any other weirdnesses that can= >be encountered with existing bridges. = Yes. >Let me give you an example: >Let consider a host bridge (such does exist) that posts both IO and MMIO= >and MMIO transactions (By the way, the PCI specs allows host bridge to >also post IOs). Let this crap also not implement the PCI ordering rules,= >but wan to be able to interface properly with PCI-2.1/2.2 PCI-to-PCI >bridges. Let then this utter crap allow to force flushing of transaction= s >by the host bridge by tampering some specific hardware registers, in ord= er >to ensure a consistent view of data when needed. = > >If you want a PCI device driver to really work on such weirdness, then >abstractions will not help at all. I beg to differ. When your hardware is probed, a hierarchical bus tag is= created. If any particular bus, chipset, etc. on the way to your device has particular size, alignment, or cache coherency constraints, it has the ability to impose these restrictions on any operation performed by the 'child' or final owner of the tag. In the case of most x86 machines (the ones without bugs), all tag operations becomes no-ops and you get almost identical performance to the non-busdma case (we added two functio= n calls to create the S/G list, but you are now using code shared with othe= r drivers that has a better chance of being in the cache). Before you begin a dma, the bus dma code has the opportunity to intercede. After a dma completes, the bus dma code has the opportunity to intercede. If bus dma must intercede, sure, it will be slower than if it didn't have= to, but your driver need never know. >On the contrary, you must allow the >drivers to have a real handle on the hardware (including host bridge), a= nd >using normal IOs will not be enough at all. The bus_space primitives are not guaranteed to boil down to "normal I/O primitives". The bus_space tag may well be keeping state so it can enforce ordering behind your back. Luckily, for most hardware out there,= nothing special is required. >There are errata everywhere and one who wants to provide reliable PCI >device driver for portable O/S must be aware on the erratas of the PCI >device and at least of the functionnal description of existing host >bridges when available (erratas will be a plus but aren't always >available). The way the CPU handles write buffers must also be considere= d, >especially when using MMIOs. Even a feature can become a bug, given all >that mess. So each driver author must reinvent the wheel here? That seems silly. Provided the OS provides a rich set of primitives and the author understa= nds when those primitives must be used, the end result is the same without duplicated code and complexity. When a new chipset with new bugs comes out, I don't want to touch all of the 100's of drivers in the system agai= n. I want to update one piece of code and have the rest benefit. >By the way, some constraints as the max size of scatter entries, the >clustering of scatter entries, not traversing some address boundary, etc= >..., when building a scatter list may affect performances and/or trigger= >erratas. I just wanted not to have to do the work twice, even if this ma= y >seem to possibly happen on Linux. In fact under Linux, scatter entries a= re >aligned on some nice address boundaries and you can tell the upper layer= >not to clusterize entries. = In FreeBSD, you can pick whatever boundary, alignment and S/G entry size constraints you want and bus dma will comply. If the knobs currently provided by the bus dma interface are not enough, then point them out so the interface can be improved. >Some O/S designed to be portable (but seems no longer to be so) got slow= >probably due to some infinite of epsilonish design and implementation >errors. I would be sad to ever see any free UNIX O/S to get NTified this= >way. Some O/Ses are designed to be large and difficult to maintain an understa= nd. It would be sad to see FreeBSD go in this direction. There are always tradeoffs in any designd... speed, code size, portability, maintainabilit= y, etc. Perhaps the tradeoffs in bus dma are not quite right (seeing as the= interface is still in flux, this wouldn't surprise me), but I'm not willi= ng to implicitly believe that abstractions cannot help in this situation. >AFAIR, in CAM3, the target peripheral driver can require disconnections = to >be used. In this situation the initiator must just be refused to be serv= ed >when it disallows disconnections. And if both target and initiator parts= >actually want to waste SCSI bandwitch, we just have to 'dont_care' and >behave as both parts did agree. In general practice you can't make this restriction. Often you are not in control of the code on the initiator. Many drivers/controllers will perform REQUEST SENSE operations without the disconnection privilege and you can't simply bail every time a REQUEST SENSE is performed. -- Justin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-scsi" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199910122003.OAA02480>