Date: Thu, 28 Sep 2006 17:40:33 GMT From: Gleb Smirnoff <glebius@FreeBSD.org> To: freebsd-bugs@FreeBSD.org Subject: Re: kern/100519: Suboptimal network polling Message-ID: <200609281740.k8SHeWoM068400@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
The following reply was made to PR kern/100519; it has been noted by GNATS.
From: Gleb Smirnoff <glebius@FreeBSD.org>
To: freebsd-gnats-submit@FreeBSD.org
Cc:
Subject: Re: kern/100519: Suboptimal network polling
Date: Thu, 28 Sep 2006 21:37:43 +0400
attach this to PR, so that it won't be lost in
my mailbox.
----- Forwarded message from Arthur Hartwig <Arthur.Hartwig@nokia.com> -----
>Hmm, interesting. Have you done any profiling?
G'day Gleb,
My work was then largely concerned with fast forwarding performance
between two em interfaces on FreeBSD 6.0 and in particular with trying
to get some performance improvement at high rates by using additional
CPUs. I did some timing using the tsc and found that the em driver in
polling mode was using roughly roughly half the time spent in
ether_input() to the point of putting a frame on the output queue. So I
thought if I ran the polling code on one CPU and ether_input() on
another CPU I would get an observable improvement. This turned out to be
the case.
At some stage I noticed that vmstat showed lots of interrupts even
though I was using polling mode. I noticed v_intr was incremented in
swi_sched() and intr_execute_handlers(). Another system not using
polling mode didn't show the same high interrupt rate. After making the
described change I found the interrupt rate reported by vmstat dropped
significantly. The change didn't result in a significant performance
improvement but clearly reduce the overhead in the polling thread, thus
making more CPU time available to the system.
Arthur
I found interrupt driven fast forwarding performed significantly
worse on a dual CPU system than on a single CPU system. Some profiling
showed a lot of contention for the routing table locks, sometimes with
significant delays between a thread attempting to acquire a lock and its
actual acquisition. Changing to polling mode seemed as if would help
reduce lock contention and so improve performance. A number of tweaks
combined to get noloss fast forwarding performance between two GigE
interfaces on a dual CPU system up to about what it was on a single CPU
interrupt driven system
Arthur
----- End forwarded message -----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200609281740.k8SHeWoM068400>