Date: Mon, 22 Jan 2007 14:23:26 +1100 (EST) From: Bruce Evans <bde@zeta.org.au> To: Max Laier <max@love2party.net> Cc: freebsd-net@freebsd.org Subject: Re: slow writes on nfs with bge devices Message-ID: <20070122140144.R7272@besplex.bde.org> In-Reply-To: <200701212033.34914.max@love2party.net> References: <20070121155510.C23922@delplex.bde.org> <200701210809.27770.max@love2party.net> <20070121215054.C24780@delplex.bde.org> <200701212033.34914.max@love2party.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, 21 Jan 2007, Max Laier wrote: > On Sunday 21 January 2007 13:25, Bruce Evans wrote: >> On Sun, 21 Jan 2007, Max Laier wrote: >>> On Sunday 21 January 2007 07:25, Bruce Evans wrote: >>>> nfs writes much less well with bge NICs than with other NICs (sk, >>>> fxp, >>> >>> Do you use hardware checksumming on the bge? There is an XXX in >>> bge_start_locked() that looks a bit suspicious to me. >> >> I use the default for that. Wouldn't checksum problems show up as >> errors somwhere? > > Did you look at the code in question? It is concerned with fragmented > packet chains (which NFS over UDP usually generated) and only commits to > sending them, if there are enough descriptors available at once. This > can easily explain burstyness. > > Can you just try to disable the delayed checksums via "ifconfig -txcsum"? > Should be an easy enough test. I haven't looked closely at the bge checksum code, but tried turning off checksums (various combinations) yesterday. THis made no difference. >> tcpdump shows a lot of intervals between nfs write requests of almost >> exactly 10 mS (even with HZ = 1000, but more obvious with HZ = 100) >> for the broken case, so the problem is apparently related to timeouts, > > See above, I really think that there is something about that if_start loop > that might be causing this. I have some local changes in this area (much larger ifq length and watermark stuff to reduce tx interrupts). I think they make no difference, but will have to test an unchanged driver more carefully. I assume that the if_start loop always calls the interface if_start if there is something in the ifq and the interface is not marked as active. Bruce
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20070122140144.R7272>