Date: Thu, 18 Nov 2004 18:49:08 +0100 From: "Daniel Eriksson" <daniel_k_eriksson@telia.com> To: "'M. Warner Losh'" <imp@bsdimp.com> Cc: freebsd-stable@freebsd.org Subject: RE: serious networking (em) performance (ggate and NFS) problem Message-ID: <!~!UENERkVCMDkAAQACAAAAAAAAAAAAAAAAABgAAAAAAAAA0VcX9IoJqUaXPS8MjT1PdsKAAAAQAAAA/qWWqwitlkyUSHwJEUT%2BbwEAAAAA@telia.com> In-Reply-To: <20041118.095348.103081513.imp@bsdimp.com>
next in thread | previous in thread | raw e-mail | index | archive | help
M. Warner Losh wrote: > Also, make sure that you aren't sharing interrupts between > GIANT-LOCKED and non-giant-locked cards. This might be exposing bugs > in the network layer that debug.mpsafenet=0 might correct. Just > noticed that our setup here has that setup, so I'll be looking into > that area of things. I have a Tyan Tiger MPX board (dual AthlonMP) that has two 64bit PCI slots. I have an Adaptec 29160 and a dual port Intel Pro/1000 MT plugged into those slots. As can be seen from the vmstat -i output below, em1 shares ithread with ahc0. This is with ACPI disabled. With ACPI enabled all devices get their own ithread (I think, not 100% sure). However, because of some hardware problem (interrupt routing?), em0 interrupts will somehow leak into atapci1+, generating a higher interrupt load. I'm not sure how expensive this is. Finally, my question. What would you recommend: 1) Run with ACPI disabled and debug.mpsafenet=1 and hope that the mix of giant-safe and giant-locked (em and ahc) doesn't trigger any bugs. This is what I currently do. 2) Run with ACPI disabled and debug.mpsafenet=0 and accept lower network performance (it is a high-traffic server, so I'm not sure this is a valid option). 3) Run with ACPI enabled and debug.mpsafenet=1 and accept that em0 interrupts "leak" to the atapci1+ ithread. This I have done in the past. # vmstat -i interrupt total rate irq1: atkbd0 2 0 irq4: sio0 710 0 irq6: fdc0 8 0 irq8: rtc 11470937 127 irq13: npx0 1 0 irq14: ata0 1744545 19 irq15: ata1 1749617 19 irq16: em0 atapci1+ 186062858 2074 irq17: em1 ahc0 27028088 301 irq18: atapci3 7393468 82 irq19: atapci4+ 7129446 79 irq0: clk 179054582 1995 Total 421634262 4699 /Daniel Eriksson
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?!~!UENERkVCMDkAAQACAAAAAAAAAAAAAAAAABgAAAAAAAAA0VcX9IoJqUaXPS8MjT1PdsKAAAAQAAAA/qWWqwitlkyUSHwJEUT%2BbwEAAAAA>