From owner-freebsd-smp@FreeBSD.ORG Mon Oct 6 13:04:15 2003 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5C8DF16A4B3 for ; Mon, 6 Oct 2003 13:04:15 -0700 (PDT) Received: from mta1.srv.hcvlny.cv.net (mta1.srv.hcvlny.cv.net [167.206.5.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id A032343FF2 for ; Mon, 6 Oct 2003 13:04:14 -0700 (PDT) (envelope-from all@biosys.net) Received: from megalomaniac.biosys.net (ool-43529b51.dyn.optonline.net [67.82.155.81]) by mta1.srv.hcvlny.cv.net (iPlanet Messaging Server 5.2 HotFix 1.16 (built May 14 2003)) with ESMTP id <0HMC00FK6PDZKM@mta1.srv.hcvlny.cv.net> for freebsd-smp@freebsd.org; Mon, 06 Oct 2003 15:42:05 -0400 (EDT) Date: Mon, 06 Oct 2003 15:43:52 -0400 From: Allen Landsidel X-Sender: bsdasym@pop.hotpop.com (Unverified) To: freebsd-smp@freebsd.org Message-id: <6.0.0.22.0.20031006154227.023f7928@pop.hotpop.com> MIME-version: 1.0 X-Mailer: QUALCOMM Windows Eudora Version 6.0.0.22 Content-type: text/plain; charset=us-ascii; format=flowed Content-transfer-encoding: 7BIT Subject: systat, vmstat, and interrupts X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Oct 2003 20:04:15 -0000 This is a copy of an email I sent to -questions. After about a week without a response I figure nobody knows and/or cares enough to hazard a response, so I'm going to drop it here and see if anyone has an idea.. ---- Not as complicated a question as one might think, but I still haven't found a way.. so, is there some way I can get tools like systat (systat -vmstat 1) or vmstat (vmstat -i) that show irq rates to display the "proper" IRQs on SMP boxes? I have several SMP boxes right now, and despite all my manpage digging and googling, I can't figure out a way to demux the mux'd IRQ for display purposes; It's quite useless to know that "mux irq2" is handling 1k+ interrupts/sec when there are several devices actually using that irq.. Sample output: interrupt total rate mux irq2 51255486 416 fdc0 irq6 3 0 clk irq0 12300164 99 rtc irq8 15743621 127 Total 79299274 644 NIC, scsi controller, etc are all contained within the mux I believe, along with most everything else on this box from the looks of it. From owner-freebsd-smp@FreeBSD.ORG Mon Oct 6 13:13:16 2003 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A653716A4B3 for ; Mon, 6 Oct 2003 13:13:16 -0700 (PDT) Received: from mail.speakeasy.net (mail9.speakeasy.net [216.254.0.209]) by mx1.FreeBSD.org (Postfix) with ESMTP id D9EB543FE0 for ; Mon, 6 Oct 2003 13:13:15 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Received: (qmail 25305 invoked from network); 6 Oct 2003 20:13:15 -0000 Received: from unknown (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender )encrypted SMTP for ; 6 Oct 2003 20:13:15 -0000 Received: from laptop.baldwin.cx (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.9/8.12.9) with ESMTP id h96KDCce002956; Mon, 6 Oct 2003 16:13:12 -0400 (EDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.5.4 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <6.0.0.22.0.20031006154227.023f7928@pop.hotpop.com> Date: Mon, 06 Oct 2003 16:13:19 -0400 (EDT) From: John Baldwin To: Allen Landsidel X-Spam-Checker-Version: SpamAssassin 2.55 (1.174.2.19-2003-05-19-exp) cc: freebsd-smp@freebsd.org Subject: RE: systat, vmstat, and interrupts X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Oct 2003 20:13:16 -0000 On 06-Oct-2003 Allen Landsidel wrote: > This is a copy of an email I sent to -questions. After about a week > without a response I figure nobody knows and/or cares enough to hazard a > response, so I'm going to drop it here and see if anyone has an idea.. > > ---- > > Not as complicated a question as one might think, but I still haven't found > a way.. so, is there some way I can get tools like systat (systat -vmstat > 1) or vmstat (vmstat -i) that show irq rates to display the "proper" IRQs > on SMP boxes? Nope, not right now. 5.2 probably will show you better output though. -stable's output isn't going to change. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ From owner-freebsd-smp@FreeBSD.ORG Mon Oct 6 14:22:30 2003 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BE82716A4D6 for ; Mon, 6 Oct 2003 14:22:30 -0700 (PDT) Received: from mta6.srv.hcvlny.cv.net (mta6.srv.hcvlny.cv.net [167.206.5.17]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0114843F3F for ; Mon, 6 Oct 2003 14:22:30 -0700 (PDT) (envelope-from all@biosys.net) Received: from megalomaniac.biosys.net (ool-43529b51.dyn.optonline.net [67.82.155.81]) by mta6.srv.hcvlny.cv.net (iPlanet Messaging Server 5.2 HotFix 1.16 (built May 14 2003)) with ESMTP id <0HMC006UETM5KF@mta6.srv.hcvlny.cv.net> for freebsd-smp@freebsd.org; Mon, 06 Oct 2003 17:13:17 -0400 (EDT) Date: Mon, 06 Oct 2003 17:15:22 -0400 From: Allen Landsidel In-reply-to: X-Sender: bsdasym@pop.hotpop.com (Unverified) To: freebsd-smp@freebsd.org Message-id: <6.0.0.22.0.20031006171159.0241d600@pop.hotpop.com> MIME-version: 1.0 X-Mailer: QUALCOMM Windows Eudora Version 6.0.0.22 Content-type: text/plain; charset=us-ascii; format=flowed Content-transfer-encoding: 7BIT References: <6.0.0.22.0.20031006154227.023f7928@pop.hotpop.com> Subject: RE: systat, vmstat, and interrupts X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Oct 2003 21:22:30 -0000 I'm on list, save to forget the cc.. At 04:13 PM 10/6/2003, John Baldwin wrote: > > Not as complicated a question as one might think, but I still haven't > found > > a way.. so, is there some way I can get tools like systat (systat -vmstat > > 1) or vmstat (vmstat -i) that show irq rates to display the "proper" IRQs > > on SMP boxes? > >Nope, not right now. 5.2 probably will show you better output though. >-stable's output isn't going to change. Ok, forgetting the two examples right now, is there any way via some other utility to check these values? I'm eagerly anticipating moving the SMP machines to 5.x, but I (like many people) am in a heavy-usage production environment; can't afford to try updating any of them until we move to -STABLE.. hopefully the website is still up to date and the RELENG_5 branch will be following not too long after, but for now that isn't an option. -Allen From owner-freebsd-smp@FreeBSD.ORG Mon Oct 6 14:55:38 2003 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C59FC16A4B3 for ; Mon, 6 Oct 2003 14:55:38 -0700 (PDT) Received: from mail.speakeasy.net (mail10.speakeasy.net [216.254.0.210]) by mx1.FreeBSD.org (Postfix) with ESMTP id 768AD43F75 for ; Mon, 6 Oct 2003 14:55:37 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Received: (qmail 21170 invoked from network); 6 Oct 2003 21:55:36 -0000 Received: from unknown (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender )encrypted SMTP for ; 6 Oct 2003 21:55:36 -0000 Received: from laptop.baldwin.cx (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.9/8.12.9) with ESMTP id h96LtXce003286; Mon, 6 Oct 2003 17:55:34 -0400 (EDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.5.4 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <6.0.0.22.0.20031006171159.0241d600@pop.hotpop.com> Date: Mon, 06 Oct 2003 17:55:40 -0400 (EDT) From: John Baldwin To: Allen Landsidel X-Spam-Checker-Version: SpamAssassin 2.55 (1.174.2.19-2003-05-19-exp) cc: freebsd-smp@freebsd.org Subject: RE: systat, vmstat, and interrupts X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Oct 2003 21:55:38 -0000 On 06-Oct-2003 Allen Landsidel wrote: > I'm on list, save to forget the cc.. > > At 04:13 PM 10/6/2003, John Baldwin wrote: >> > Not as complicated a question as one might think, but I still haven't >> found >> > a way.. so, is there some way I can get tools like systat (systat -vmstat >> > 1) or vmstat (vmstat -i) that show irq rates to display the "proper" IRQs >> > on SMP boxes? >> >>Nope, not right now. 5.2 probably will show you better output though. >>-stable's output isn't going to change. > > Ok, forgetting the two examples right now, is there any way via some other > utility to check these values? Not really other than trying to correlate with dmesg output. :-/ > I'm eagerly anticipating moving the SMP machines to 5.x, but I (like many > people) am in a heavy-usage production environment; can't afford to try > updating any of them until we move to -STABLE.. hopefully the website is > still up to date and the RELENG_5 branch will be following not too long > after, but for now that isn't an option. Definitely understood. I still run 4.x on all my important machines still. :) -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ From owner-freebsd-smp@FreeBSD.ORG Wed Oct 8 00:35:19 2003 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 91C2216A4B3; Wed, 8 Oct 2003 00:35:19 -0700 (PDT) Received: from sferics.mongueurs.net (sferics.mongueurs.net [81.80.147.197]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1F30743F85; Wed, 8 Oct 2003 00:35:18 -0700 (PDT) (envelope-from david@landgren.net) Received: from landgren.net (81-80-147-206.bpinet.com [81.80.147.206]) by sferics.mongueurs.net (Postfix) with ESMTP id 0B4CAC620; Wed, 8 Oct 2003 09:35:15 +0200 (CEST) Message-ID: <3F83BE31.1040108@landgren.net> Date: Wed, 08 Oct 2003 09:35:13 +0200 From: David Landgren Organization: A thousand golden eyes are watching User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5) Gecko/20030916 X-Accept-Language: en-us, en MIME-Version: 1.0 To: John Baldwin References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-smp@freebsd.org Subject: Re: During boot: "Programming 16 pins in IOAPIC" ... and then hangs X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Oct 2003 07:35:19 -0000 John Baldwin wrote: > On 19-Sep-2003 David Landgren wrote: > >>Greetings list, >> >>I have an HP DL380-G3 with a P4 processor. I upgraded the source tree >>last night to 4.9-PRERELEASE. This is the last server I have that is >>running a GENERIC kernel, two others are successfully running SMP >>kernels (insofar as hw.ncpus = 2). >> >>I copied over a successful kernel configuration file and tried to >>build an SMP kernel. The new kernel boots with the following: >> >>Programming 16 pins in IOAPIc #0 >>IOAPIC #0 intpin 2 -> irq 0 >>Programming 16 pins in IOAPIc #1 >>Programming 16 pins in IOAPIc #2 >>Programming 16 pins in IOAPIc #3 >> >>... and then hangs. If I comment out >> >>options SMP >>options APIC_IO >>options HTT > > > Look in your BIOS and see if there is an option to make it create > a "full mptable". If so, turn that on and try again. For the archives, in case other people search for this problem: Yes indeed, this was the solution. I rebooted the machine and ran the BIOS configuration (F9 at the prompt just before it tries to figure out which device it should boot from. From there, take the "Advanced Options" menu. From there, take the "MPS Table Mode" menu. There, you have a choice between o Full Table APIC o Full Table Mapped o Disabled o Auto Set Table It was set to "Auto Set Table". I changed this to "Full Table APIC" and now my system reports: % sysctl hw.ncpu hw.ncpu: 2 Here's the dmesg, FWIW % dmesg Copyright (c) 1992-2003 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 4.9-PRERELEASE #1: Wed Oct 8 09:05:45 CEST 2003 root@bechet.bpinet.com:/usr/obj/usr/src/sys/BECHET Timecounter "i8254" frequency 1193182 Hz CPU: Intel(R) Xeon(TM) CPU 2.80GHz (2785.61-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf27 Stepping = 7 Features=0xbfebfbff Hyperthreading: 2 logical CPUs real memory = 2147459072 (2097128K bytes) avail memory = 2088259584 (2039316K bytes) Programming 16 pins in IOAPIC #0 IOAPIC #0 intpin 2 -> irq 0 Programming 16 pins in IOAPIC #1 Programming 16 pins in IOAPIC #2 Programming 16 pins in IOAPIC #3 FreeBSD/SMP: Multiprocessor motherboard: 2 CPUs cpu0 (BSP): apic id: 6, version: 0x00050014, at 0xfee00000 cpu1 (AP): apic id: 7, version: 0x00050014, at 0xfee00000 io0 (APIC): apic id: 2, version: 0x000f0011, at 0xfec00000 io1 (APIC): apic id: 3, version: 0x000f0011, at 0xfec01000 io2 (APIC): apic id: 4, version: 0x000f0011, at 0xfec02000 io3 (APIC): apic id: 5, version: 0x000f0011, at 0xfec03000 Preloaded elf kernel "kernel" at 0xc0364000. Pentium Pro MTRR support enabled md0: Malloc disk npx0: on motherboard npx0: INT 16 interface pcib0: on motherboard IOAPIC #1 intpin 0 -> irq 2 IOAPIC #1 intpin 1 -> irq 3 IOAPIC #0 intpin 7 -> irq 5 pci0: on pcib0 pci0: at 3.0 pci0: (vendor=0x0e11, dev=0xb203) at 4.0 irq 2 pci0: (vendor=0x0e11, dev=0xb204) at 4.2 irq 3 isab0: at device 15.0 on pci0 isa0: on isab0 atapci0: port 0x2000-0x200f,0x374-0x377,0x170-0x177,0x3f4-0x3f7,0x1f0-0x1f7 at device 15.1 on pci0 ata0: at 0x1f0 irq 14 on atapci0 ata1: at 0x170 irq 15 on atapci0 pci0: at 15.2 irq 5 pcib1: on motherboard IOAPIC #1 intpin 14 -> irq 7 pci1: on pcib1 ciss0: port 0x3000-0x30ff mem 0xf7bf0000-0xf7bf3fff,0xf7cc0000-0xf7cfffff irq 7 at device 3.0 on pci1 pcib2: on motherboard IOAPIC #1 intpin 13 -> irq 10 IOAPIC #1 intpin 15 -> irq 11 pci2: on pcib2 bge0: mem 0xf7df0000-0xf7dfffff irq 10 at device 1.0 on pci2 bge0: Ethernet address: 00:0b:cd:03:bc:da miibus0: on bge0 brgphy0: on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto bge1: mem 0xf7de0000-0xf7deffff irq 11 at device 2.0 on pci2 bge1: Ethernet address: 00:0b:cd:0e:07:18 miibus1: on bge1 brgphy1: on miibus1 brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto pcib3: on motherboard IOAPIC #1 intpin 4 -> irq 15 IOAPIC #1 intpin 5 -> irq 16 pci3: on pcib3 pci3: (vendor=0x9005, dev=0x00c0) at 1.0 irq 15 pci3: (vendor=0x9005, dev=0x00c0) at 1.1 irq 16 pcib4: on motherboard pci4: on pcib4 pcib6: on motherboard IOAPIC #1 intpin 2 -> irq 17 pci6: on pcib6 pci6: (vendor=0x0e11, dev=0xa0f7) at 30.0 irq 17 pcib5: on motherboard pci5: on pcib5 pcib7: on motherboard pci7: on pcib7 eisa0: on motherboard mainboard0: on eisa0 slot 0 orm0: