Date: Sat, 20 Sep 2003 09:48:20 -0700 (PDT) From: John Polstra <jdp@polstra.com> To: Sam Leffler <sam@errno.com> Cc: freebsd-arch@freebsd.org Subject: RE: interrupt latency and driver locking Message-ID: <XFMail.20030920094820.jdp@polstra.com> In-Reply-To: <606095639.1063987155@melange.errno.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 19-Sep-2003 Sam Leffler wrote: > I enabled MUTEX_PROFILING on a fast machine I've got setup as a > firewall/router. The machine had an fxp device on one side and an em on > the other. I then ran a bunch of netperf tests through the machine. > Unfortunately for the moment I'm stuck with 100baseT on the fxp side of the > machine so my tests were all limited by the media. But one thing stuck out: [...] > Note that the fxp driver holds its driver lock for an average 112 us in one > spot. This turns out to be in fxp_tick and the likely culprit is the call > to mii_tick which is done with the lock held. Agreed. The MII calls usually have lots of DELAY() calls in them. > I'm not sure if it's simple to move MII operations outside of the driver > lock but it might be worth investigating. It seems like it ought to be possible to at least lock at a finer grain for those calls. The driver lock is held because the MII routines call back into the driver and use it to do the bit-twiddling necessary to shift data into and out of the PHY registers via the PHY's serial interface. That's normally pretty independent of anything else that's going on in the chip, so it could probably be done under a separate lock. Another idea would be to substitute something else for DELAY() calls of more than a few microseconds. I am not very up-to-date on -current these days, but as far as I know DELAY() is still a spin-wait. If the delay is for more than a few microseconds it would probably be better to switch to a different runnable thread and come back later. In fact, CPU speeds may have reached the point where that is always the right thing to do. Finally, the entire mii_tick() nonsense could be thrown out for many modern network adapters. Both the Broadcom and Intel gigabit chips are capable of interrupting on interesting link events, so there is no need at all to poll the PHY every second. That doesn't appear to be true of the fxp devices, though. They need to be polled. Still, you can avoid the expensive PHY reads and writes most of the time. For example, if you have received any packets on the interface in the past second, you can assume that link is good and never touch the PHY at all. John
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?XFMail.20030920094820.jdp>