Date: Tue, 27 Jul 1999 10:55:31 -0700 (PDT) From: Julian Elischer <julian@whistle.com> To: Mike Smith <mike@smith.net.au> Cc: "Justin T. Gibbs" <gibbs@caspian.plutotech.com>, "Justin T. Gibbs" <gibbs@plutotech.com>, cvs-all@FreeBSD.org, cvs-committers@FreeBSD.org Subject: Re: cvs commit: src/release/sysinstall tcpip.c Message-ID: <Pine.BSF.3.95.990727104024.29565A-100000@current1.whistle.com> In-Reply-To: <199907270810.BAA01282@dingo.cdrom.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 27 Jul 1999, Mike Smith wrote: > > >> In zero copy applications, the header and the payload are usually placed > > >> in separate areas. > > > > > >Can you elaborate on this a little? We don't support the two being in > > >a separate allocation unit at the moment, yet I understood the 'fast > > >forwarding' code was essentially a zero-copy operation. > > > > I have not looked at the code Drew Gallatin used for his myrinet work, > > but here at Pluto, we plan to ship the header information into CPU > > memory and the payload to another PCI device's memory. I would expect > > a similar approach to be used for page flipping packet payloads into > > user space in more conventional zero-copy applications. This is nothing new. We used BSD4.3 to do this in 1987. I still have all the code around here to do it somewhere. It was VME bus not PCI (and later EISA). The ethernet drivers and disk drivers both knew about a pseudo device called the "Physmem" device. The physmem driver was in charge of physical ram allocated for this sort of thing. Sometimes it was taken from main ram and both ethernet and disk DMA'd into it, and sometimes it was on the ethernet cards. (4MB/card). A special protocol was developed that passed in Physmem buffer cookies as control information, (just like sendig fds in UNIX domain sockets). You sent the header as the data part of the sendmsg(2), and the cookie as the control info, and the ethernet driver knew to DMA straight from the buffer refered to in the cookie. A IOCTL was added to the disk driver to read and write blocks directly to a buffer identified by a cookie. The ethernet driver wrote all incoming packets into physmem buffers to start with, but copied out very small packets to conserver them (we were using jumbo packets so a 64 byte packet in a 16K buffer was a waste). BTW TRW still does this with FreeBSD, and is the reason that FreeBSD's 'external buffer' support in the mbufs is known to work :-) (even before sendfile). > > Hmm. Would you be using a second mbuf with external storage pointing > to the other device's memory to account for the payload? At on stage We used a 2nd mbuf with the 'control info' flag set, and the control info had the cookie refering to the buffer. The drivers 'cashed in the cookie with the physmem driver to get the address of the data. It also ad the external pointers pointing at the data. All referenced tothe physmem buffers were reference counted so even if the user freed a buffer after sending it, it wasn't actually freed until the driver also freed it after sending it, or the disk driver freed it after writing it. > > Is this a short-circuit routing technique, or just immediate delivery > of payload data? Who unpacks the data that's going to the target > peripheral? > julian To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe cvs-all" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.3.95.990727104024.29565A-100000>