Date: Thu, 12 Jun 2003 01:47:26 -0700 From: Terry Lambert <tlambert2@mindspring.com> To: Hiten Pandya <hmp@nxad.com> Cc: Eric Anderson <anderson@centtech.com> Subject: Re: Slow disk write speeds over network Message-ID: <3EE83E1E.BB3E9865@mindspring.com> References: <20030609211526.58641.qmail@web14908.mail.yahoo.com> <3EE4FAED.6090603@centtech.com> <3EE595D2.B223CA19@mindspring.com> <3EE5F8DE.30001@centtech.com> <3EE7021E.F2928B7@mindspring.com> <20030612010349.GA23018@perrin.int.nxad.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hiten Pandya wrote:
> ``NetISR Dispatch'' and friends are not available in
> FreeBSD 4.x versions. I am sure that he is using 4.x and
> that's why he can't find it in his source.
Actually, that hadn't occurred to me. 8-).
> > The net.isr.enable=1 will save you about 100ms per packet,
> > minimum, and more if you have a high interrupt overhead that
> > livelocks you from running NETISR. What it does is turns on
> > direct processing by IP and TCP of packets as they come in
> > the interface and you get the interrupt. Combined with soft
> > interrupt coelescing and polling, they should give you
> > another 1/3 of the receiver livelock fixup. The final third
> > isn't available, unless you are willing to hack network stack
> > code and scheduler code, since FreeBSD doesn't include LRP or
> > Weighted Fair Share Queuing.
>
> Does it help if queueing discipline is changed?
No. The WFQ code is not for packet queueing, it's for the
scheduling of user processes that are servicing the data
that comes in over the network. Looking at the "QLinux" web
site would probably be informative. Note that I've been
advocating these approaches before the QLinux project had a
web site up. 8-).
The problem here is that if you spend all your time handling
interrupts, you never run NETISR; if you spend all your time
handling interrupts AND running NETISR, you never run the
user space program that's supposed to take the data out of
the socket buffer.
> > You should grep for DEVICE_POLLING in the network device
> > drivers you are interested in using to see if they have the
> > support. Also you can get up to 15% by adding soft interrupt
> > coelescing code, if the driver doesn't already support it (I
> > added it for a couple of drivers, and it was committed after
> > the benchmarks showed it was good, but it's not everywhere);
> > the basic idea is you take the interrupt, run rx_eof(), and
> > call ether_input(). Then repeat the process until you hit
> > some count limit, or until there's no more data. The direct
> > dispatch (net.isr.enable) combined with that will process most
> > packet trains to completion at interrupt, saving you 10ms up
> > and 10ms back down per packet exchange (NETISR only runs on
> > exit from spl or at the HZ time, which is default every 10ms).
>
> Hmm, can you point these drivers to me?
Look in if_dc.c, around line 3067 in dc_intr(), for one:
if (curpkts == ifp->if_ipackets) {
while(dc_rx_resync(sc))
dc_rxeof(sc);X-MX-Mozilla-Status: 0009 }
Also if_pcn.c, if_rl.c, if_sf.c, if_sis.c, if_sk.c, etc. ...just
look for a status-check loop that calls the *_rxeof() function in
the *_intr() function; if it's there, that's a software coelescing
driver.
-- Terry
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3EE83E1E.BB3E9865>