Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 18 Nov 2004 18:49:08 +0100
From:      "Daniel Eriksson" <daniel_k_eriksson@telia.com>
To:        "'M. Warner Losh'" <imp@bsdimp.com>
Cc:        freebsd-stable@freebsd.org
Subject:   RE: serious networking (em) performance (ggate and NFS) problem
Message-ID:  <!~!UENERkVCMDkAAQACAAAAAAAAAAAAAAAAABgAAAAAAAAA0VcX9IoJqUaXPS8MjT1PdsKAAAAQAAAA/qWWqwitlkyUSHwJEUT+bwEAAAAA@telia.com>
In-Reply-To: <20041118.095348.103081513.imp@bsdimp.com>

Next in thread | Previous in thread | Raw E-Mail | Index | Archive | Help
M. Warner Losh wrote:

> Also, make sure that you aren't sharing interrupts between
> GIANT-LOCKED and non-giant-locked cards.  This might be exposing bugs
> in the network layer that debug.mpsafenet=0 might correct.  Just
> noticed that our setup here has that setup, so I'll be looking into
> that area of things.

I have a Tyan Tiger MPX board (dual AthlonMP) that has two 64bit PCI slots.
I have an Adaptec 29160 and a dual port Intel Pro/1000 MT plugged into those
slots.

As can be seen from the vmstat -i output below, em1 shares ithread with
ahc0. This is with ACPI disabled. With ACPI enabled all devices get their
own ithread (I think, not 100% sure). However, because of some hardware
problem (interrupt routing?), em0 interrupts will somehow leak into
atapci1+, generating a higher interrupt load. I'm not sure how expensive
this is.

Finally, my question. What would you recommend:
1) Run with ACPI disabled and debug.mpsafenet=1 and hope that the mix of
giant-safe and giant-locked (em and ahc) doesn't trigger any bugs. This is
what I currently do.
2) Run with ACPI disabled and debug.mpsafenet=0 and accept lower network
performance (it is a high-traffic server, so I'm not sure this is a valid
option).
3) Run with ACPI enabled and debug.mpsafenet=1 and accept that em0
interrupts "leak" to the atapci1+ ithread. This I have done in the past.


# vmstat -i
interrupt                          total       rate
irq1: atkbd0                           2          0
irq4: sio0                           710          0
irq6: fdc0                             8          0
irq8: rtc                       11470937        127
irq13: npx0                            1          0
irq14: ata0                      1744545         19
irq15: ata1                      1749617         19
irq16: em0 atapci1+            186062858       2074
irq17: em1 ahc0                 27028088        301
irq18: atapci3                   7393468         82
irq19: atapci4+                  7129446         79
irq0: clk                      179054582       1995
Total                          421634262       4699

/Daniel Eriksson




Want to link to this message? Use this URL: <http://docs.FreeBSD.org/cgi/mid.cgi?!~!UENERkVCMDkAAQACAAAAAAAAAAAAAAAAABgAAAAAAAAA0VcX9IoJqUaXPS8MjT1PdsKAAAAQAAAA/qWWqwitlkyUSHwJEUT+bwEAAAAA>