Date: Fri, 19 Mar 1999 09:15:53 -0500 (EST) From: Bill Paul <wpaul@skynet.ctr.columbia.edu> To: dillon@apollo.backplane.com (Matthew Dillon) Cc: hackers@freebsd.org Subject: Re: Gigabit ethernet revisited Message-ID: <199903191415.JAA03718@skynet.ctr.columbia.edu> In-Reply-To: <199903190525.VAA44097@apollo.backplane.com> from "Matthew Dillon" at Mar 18, 99 09:25:12 pm
next in thread | previous in thread | raw e-mail | index | archive | help
Of all the gin joints in all the towns in all the world, Matthew Dillon had to walk into mine and say: > :it's about half what I think it should be. The nic stats counters > :show quite a few nicDmaWriteRingFull and nicNoMoreWrDMADescriptors > :errors, which means that the NIC is receiving the frames but the > :bus isn't transfering them to the host fast enough. > : > :The machines I'm using now are Dell Precision 410 workstations, each > :with one 450Mhz Pentium II CPU, 128MB of RAM and Intel 82443BX PCI > :chipsets. > > It sounds like the DMA is working fine, but the host computer > is not able to drain the receive ring ( which I presume is the > 'write ring' ) quickly enough. No, I don't think that's it. I think the DMA isn't going fast enough. The 'WriteRing' in question is a ring of DMA descriptors inside the NIC, which the firmware is supposed to be managing. There's also a DmaReadRing for the transmit side. > Your current code appears to allocate a sufficient number of buffers, > but your ti_refill_rx_rings() optimization is suspect... you do not > call it until you have *COMPLETELY* drained the RX ring of packets. > What happens if new packets are streaming in faster then you can drain > them? You will completely exhaust your RX ring before you reallocate > it's buffers. There are a couple of tuning parameters that control when the NIC will interrupt the host to check the receive ring: ti_rx_coal_ticks and ti_rx_max_coal_bds. The former is the number of microseconds that can elapse before the NIC will be forced to interrupt, and the latter is the number of descriptors to wait to be accumulated before generating an interrupt. Right now, I set things for 500 microseconds and 64 buffer descriptors, and in my tests this usually results in buffers being DMAed in chunks of 16 or so. (I addedm some instrumentation to check this.) So I don't think the RX ring is getting anywhere near drained. For giggles, I cranked thinks up so that it DMA'ed a couple hundred a shot before interrupting and it was still too slow, and that was with all 512 entries in the receive ring filled up. > I would also suggest figuring out *exactly* why you are getting the > error message... i.e. is the RX ring actually completely used up as of > the time the error occurs? The firmware source code is somewhat... opaque. But basically it means that the NIC has received frames and wants to queue them onto the DMA write ring, but that ring is already full. In other words, the MAC part of the NIC is getting the all the frames and wants to get them into the host, but the DMA is proceeding so slowly that it can't move them all and has to drop some. I'm not discounting the possibility that the receive handler is botching things somehow, but I don't think the problem is lack of space in the receive ring. -Bill -- ============================================================================= -Bill Paul (212) 854-6020 | System Manager, Master of Unix-Fu Work: wpaul@ctr.columbia.edu | Center for Telecommunications Research Home: wpaul@skynet.ctr.columbia.edu | Columbia University, New York City ============================================================================= "It is not I who am crazy; it is I who am mad!" - Ren Hoek, "Space Madness" ============================================================================= To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199903191415.JAA03718>