From owner-freebsd-smp Sun Mar 28 23:59:39 1999 Delivered-To: freebsd-smp@freebsd.org Received: from excalibur.lps.ens.fr (excalibur.lps.ens.fr [129.199.120.3]) by hub.freebsd.org (Postfix) with ESMTP id 3CC0215438; Sun, 28 Mar 1999 23:59:33 -0800 (PST) (envelope-from Thierry.Besancon@lps.ens.fr) Received: (from besancon@localhost) by excalibur.lps.ens.fr (8.8.5/8.8.6) id HAA23476; Mon, 29 Mar 1999 07:59:09 GMT Message-Id: <199903290759.HAA23476@excalibur.lps.ens.fr> From: Thierry.Besancon@lps.ens.fr (Thierry Besancon) Date: Mon, 29 Mar 1999 09:59:09 +0000 X-Mailer: Mail User's Shell (7.2.5 10/14/92) To: freebsd-stable@FreeBSD.ORG, freebsd-smp@FreeBSD.ORG, Pierre.David@prism.uvsq.fr, jt@ratp.fr Subject: panic: pipeinit Cc: besancon@lps.ens.fr Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hello After some troubles with lockmgr that seem to be fixed after upgrading to 3.1-stable-990311 with a 15 days uptime, I fetched yesterday 3.1-stable and rebooted on a new kernel (with maxusers==512). (I changed stable versions because I got "Out of mbuf clusters - adjust NMBCLUSTERS or increase maxusers!" crashes and I thought that was the opportunity to give stable-990328 a chance to run.) During the night my bi-pentium II (with 512 Mo of ram) crashed with the following ddb trace : panic: pipeinit: cannot allocate pipe -- out of kvm -- code = 3 mp_lock = 01000001; cpuid = 1; lapic.id = 00000000 Debugger("panic") Stopped at Debugger+0x37: movl $0,in_Debugger db> trace Debugger(f0231966) at Debugger+0x37 panic(f023256e,3,ff6088e0,ff790f10,f014e19e) at panic+0xa4 pipespace(ff6088e0) at pipespace+0x57 pipe_write(f28943c0,ff790f40,f287f200,ff6d5100,f024edf8) at pipe_write+0x18a write(ff6d5100,ff790f94,2811c808,2812a4cc,80c3000) at write+0xba syscall(27,27,80c3000,2812a4cc,efbf9228) at syscall+0x187 Xint0x80_syscall() at Xint0x80_syscall+0x4c mp_lock = 01000001; cpuid = 1;lapic.id = 00000000 db> ps 1860 ff6d5100 ff78f000 0 295 000104 2 sendmail-8.9.3 ... If someone has an idea of what to do... Thanks in advance. Best regards. Thierry Besancon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Mar 29 7:57:58 1999 Delivered-To: freebsd-smp@freebsd.org Received: from smtp02.iafrica.com (smtp02.iafrica.com [196.7.0.140]) by hub.freebsd.org (Postfix) with ESMTP id 72B2A14FCC; Mon, 29 Mar 1999 07:54:36 -0800 (PST) (envelope-from bradh@uunet.co.za) Received: from [196.7.161.2] (helo=machine02) by smtp02.iafrica.com with smtp (Exim 1.92 #1) id 10ReMh-000MSn-00; Mon, 29 Mar 1999 17:54:15 +0200 From: "Brad Hendrickse" To: , Subject: panic on 3.1-stable (SMP) from 1990-03-15 Date: Mon, 29 Mar 1999 17:53:09 +0200 Message-ID: <001801be79fc$3e253d60$02a107c4@machine02.rabies.org.za> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0 X-MimeOLE: Produced By Microsoft MimeOLE V4.72.3155.0 Importance: Normal Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org hi folks, i'm running an smp system with 2 x intel pentium 200Mhz cpu's. today, after the machine had been up for almost 2 weeks, i got this: cpu_reset called on cpu#1 cpu_reset: Stopping other CPUs panic: apic_api was stuck mp_lock = 01000001; cpuid = 1; lapic_id = 01000000 boot() called on cpu#1 it then said it was rebooting, but just kept cycling through the above error messages saying i could press a key on the console to boot, which i did and then there was nothing i could do short of a hardware reset. thanks! mptable output: ============================================================================ === MPTable, version 2.0.15 ---------------------------------------------------------------------------- --- MP Floating Pointer Structure: location: BIOS physical address: 0x000f0dd0 signature: '_MP_' length: 16 bytes version: 1.1 checksum: 0xa3 mode: Virtual Wire ---------------------------------------------------------------------------- --- MP Config Table Header: physical address: 0x000f0de4 signature: 'PCMP' base table length: 292 version: 1.1 checksum: 0x2e OEM ID: 'OEM00000' Product ID: 'PROD00000000' OEM table pointer: 0x00000000 OEM table size: 0 entry count: 28 local APIC address: 0xfee00000 extended table length: 0 extended table checksum: 0 ---------------------------------------------------------------------------- --- MP Config Base Table Entries: -- Processors: APIC ID Version State Family Model Step Flags 0 0x11 BSP, usable 5 2 1 0x07bf 1 0x11 AP, usable 5 2 1 0x07bf -- Bus: Bus ID Type 0 PCI 1 ISA -- I/O APICs: APIC ID Version State Address 2 0x11 usable 0xfec00000 -- I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN# ExtINT conforms conforms 1 0 2 0 INT conforms conforms 1 1 2 1 INT conforms conforms 1 0 2 2 INT conforms conforms 1 3 2 3 INT conforms conforms 1 4 2 4 INT conforms conforms 1 5 2 5 INT conforms conforms 1 6 2 6 INT conforms conforms 1 7 2 7 INT conforms conforms 1 8 2 8 INT conforms conforms 1 9 2 9 INT conforms conforms 1 10 2 10 INT conforms conforms 1 11 2 11 INT conforms conforms 1 12 2 12 INT conforms conforms 1 13 2 13 INT conforms conforms 1 14 2 14 INT conforms conforms 1 15 2 15 INT active-lo level 0 8:A 2 16 INT active-lo level 0 9:A 2 17 INT active-lo level 0 10:A 2 18 INT active-lo level 0 12:A 2 19 SMI conforms conforms 1 0 2 23 -- Local Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN# ExtINT conforms conforms 0 0:A 255 0 NMI conforms conforms 0 0:A 255 1 ---------------------------------------------------------------------------- --- # SMP kernel config file options: # Required: options SMP # Symmetric MultiProcessor Kernel options APIC_IO # Symmetric (APIC) I/O # Optional (built-in defaults will work in most cases): #options NCPU=2 # number of CPUs #options NBUS=2 # number of busses #options NAPIC=1 # number of IO APICs #options NINTR=24 # number of INTs ============================================================================ === dmesg output: Mar 29 13:06:58 ns1 /kernel: The Regents of the University of California. All rights reserved. Mar 29 13:06:58 ns1 /kernel: FreeBSD 3.1-STABLE #0: Tue Mar 16 00:50:52 SAST 1999 Mar 29 13:06:58 ns1 /kernel: bradh@ns1.rabies.org.za:/usr/src/sys/compile/kernel.AMNESIA Mar 29 13:06:58 ns1 /kernel: Timecounter "i8254" frequency 1193182 Hz Mar 29 13:06:58 ns1 /kernel: CPU: Pentium/P54C (586-class CPU) Mar 29 13:06:58 ns1 /kernel: Origin = "GenuineIntel" Id = 0x52c Stepping=12 Mar 29 13:06:58 ns1 /kernel: Features=0x3bf Mar 29 13:06:58 ns1 /kernel: real memory = 67108864 (65536K bytes) Mar 29 13:06:58 ns1 /kernel: avail memory = 61956096 (60504K bytes) Mar 29 13:06:58 ns1 /kernel: Programming 24 pins in IOAPIC #0 Mar 29 13:06:58 ns1 /kernel: FreeBSD/SMP: Multiprocessor motherboard Mar 29 13:06:58 ns1 /kernel: cpu0 (BSP): apic id: 0, version: 0x00030010, at 0xfee00000 Mar 29 13:06:58 ns1 /kernel: cpu1 (AP): apic id: 1, version: 0x00030010, at 0xfee00000 Mar 29 13:06:58 ns1 /kernel: io0 (APIC): apic id: 2, version: 0x00170011, at 0xfec00000 Mar 29 13:06:58 ns1 /kernel: Preloaded elf kernel "kernel" at 0xf02d2000. Mar 29 13:06:58 ns1 /kernel: Preloaded userconfig_script "/kernel.config" at 0xf02d209c. Mar 29 13:06:58 ns1 /kernel: Preloaded splash_image_data "/boot/startup.bmp" at 0xf02d20e8. Mar 29 13:06:58 ns1 /kernel: Preloaded elf module "splash_bmp.ko" at 0xf02d2138. Mar 29 13:06:58 ns1 /kernel: Probing for devices on PCI bus 0: Mar 29 13:06:58 ns1 /kernel: chip0: rev 0x03 on pci0.0.0 Mar 29 13:06:58 ns1 /kernel: chip1: rev 0x01 on pci0.7.0 Mar 29 13:06:58 ns1 /kernel: ide_pci0: rev 0x00 on pci0.7.1 Mar 29 13:06:58 ns1 /kernel: vga0: rev 0x00 int a irq 17 on pci0.9.0 Mar 29 13:06:58 ns1 /kernel: ahc0: rev 0x00 int a irq 19 on pci0.12.0 Mar 29 13:06:58 ns1 /kernel: ahc0: Using left over BIOS settings Mar 29 13:06:58 ns1 /kernel: ahc0: aic7880 Wide Channel A, SCSI Id=0, 16/255 SCBs Mar 29 13:06:59 ns1 /kernel: Probing for devices on the ISA bus: Mar 29 13:06:59 ns1 /kernel: sc0 on isa Mar 29 13:06:59 ns1 /kernel: sc0: VGA color <16 virtual consoles, flags=0x0> Mar 29 13:06:59 ns1 /kernel: ed0 at 0x340-0x35f irq 9 on isa Mar 29 13:06:59 ns1 /kernel: ed0: address 00:40:d8:00:3f:64, type NE2000 (16 bit) Mar 29 13:06:59 ns1 /kernel: atkbdc0 at 0x60-0x6f on motherboard Mar 29 13:06:59 ns1 /kernel: atkbd0 irq 1 on isa Mar 29 13:06:59 ns1 /kernel: sio0 at 0x3f8-0x3ff irq 4 on isa Mar 29 13:06:59 ns1 /kernel: sio0: type 16550A Mar 29 13:06:59 ns1 /kernel: sio1 at 0x2f8-0x2ff irq 3 on isa Mar 29 13:06:59 ns1 /kernel: sio1: type 16550A Mar 29 13:06:59 ns1 /kernel: lpt0 at 0x378-0x37f irq 7 on isa Mar 29 13:06:59 ns1 /kernel: lpt0: Interrupt-driven port Mar 29 13:06:59 ns1 /kernel: lp0: TCP/IP capable interface Mar 29 13:06:59 ns1 /kernel: lpt-266019832: this driver is deprecated; use ppbus instead. Mar 29 13:06:59 ns1 /kernel: fdc0 at 0x3f0-0x3f7 irq 6 drq 2 on isa Mar 29 13:06:59 ns1 /kernel: wdc0 at 0x1f0-0x1f7 irq 14 on isa Mar 29 13:06:59 ns1 /kernel: wdc0: unit 0 (wd0): Mar 29 13:06:59 ns1 /kernel: wd0: 3077MB (6303024 sectors), 6253 cyls, 16 heads, 63 S/T, 512 B/S Mar 29 13:06:59 ns1 /kernel: wdc0: unit 1 (wd1): Mar 29 13:06:59 ns1 /kernel: wd1: 2452MB (5021856 sectors), 4982 cyls, 16 heads, 63 S/T, 512 B/S Mar 29 13:06:59 ns1 /kernel: vga0 at 0x3b0-0x3df maddr 0xa0000 msize 131072 on isa Mar 29 13:06:59 ns1 /kernel: npx0 flags 0x1 on motherboard Mar 29 13:06:59 ns1 /kernel: npx0: INT 16 interface Mar 29 13:06:59 ns1 /kernel: ahc0: Someone reset channel A Mar 29 13:06:59 ns1 /kernel: Intel Pentium detected, installing workaround for F00F bug Mar 29 13:06:59 ns1 /kernel: APIC_IO: Testing 8254 interrupt delivery Mar 29 13:06:59 ns1 /kernel: APIC_IO: routing 8254 via pin 2 Mar 29 13:06:59 ns1 /kernel: IP packet filtering initialized, divert disabled, rule-based forwarding disabled, logging limited to 100 packets/entry Mar 29 13:06:59 ns1 /kernel: Waiting 8 seconds for SCSI devices to settle Mar 29 13:06:59 ns1 /kernel: SMP: AP CPU #1 Launched! Mar 29 13:07:00 ns1 /kernel: changing root device to wd0s1a Mar 29 13:07:00 ns1 /kernel: cd0 at ahc0 bus 0 target 1 lun 0 Mar 29 13:07:00 ns1 /kernel: cd0: Removable CD-ROM SCSI-2 device Mar 29 13:07:00 ns1 /kernel: cd0: 8.064MB/s transfers (8.064MHz, offset 15) Mar 29 13:07:00 ns1 /kernel: cd0: cd present [182192 x 2048 byte records] Mar 29 13:07:00 ns1 /kernel: WARNING: / was not properly dismounted Mar 29 13:07:00 ns1 /kernel: ffs_mountfs: superblock updated for soft updates Mar 29 13:07:00 ns1 /kernel: ffs_mountfs: superblock updated for soft updates Mar 29 13:07:00 ns1 /kernel: ffs_mountfs: superblock updated for soft updates Mar 29 13:07:00 ns1 /kernel: ffs_mountfs: superblock updated for soft updates Mar 29 13:07:00 ns1 /kernel: ffs_mountfs: superblock updated for soft updates To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Mar 29 12:16:10 1999 Delivered-To: freebsd-smp@freebsd.org Received: from frmug.org (frmug-gw.frmug.org [193.56.58.252]) by hub.freebsd.org (Postfix) with ESMTP id 92E0D15501 for ; Mon, 29 Mar 1999 12:15:53 -0800 (PST) (envelope-from roberto@keltia.freenix.fr) Received: (from uucp@localhost) by frmug.org (8.9.1/frmug-2.3/nospam) with UUCP id WAA10749; Mon, 29 Mar 1999 22:15:27 +0200 (CEST) (envelope-from roberto@keltia.freenix.fr) Received: by keltia.freenix.fr (Postfix, from userid 101) id AE63987B6; Mon, 29 Mar 1999 21:48:27 +0200 (CEST) Date: Mon, 29 Mar 1999 21:48:27 +0200 From: Ollivier Robert To: freebsd-smp@FreeBSD.ORG Cc: Pierre.David@prism.uvsq.fr, jt@ratp.fr, Thierry.Besancon@lps.ens.fr Subject: Re: panic: pipeinit Message-ID: <19990329214827.A42380@keltia.freenix.fr> Mail-Followup-To: freebsd-smp@FreeBSD.ORG References: <199903290759.HAA23476@excalibur.lps.ens.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii User-Agent: Mutt/0.95.3i In-Reply-To: <199903290759.HAA23476@excalibur.lps.ens.fr>; from Thierry Besancon on Mon, Mar 29, 1999 at 09:59:09AM +0000 X-Operating-System: FreeBSD 4.0-CURRENT/ELF ctm#5173 AMD-K6 MMX @ 200 MHz Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org According to Thierry Besancon: > yesterday 3.1-stable and rebooted on a new kernel (with maxusers==512). Please lower down the value of maxusers. Try with 128. -- Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=- roberto@keltia.freenix.fr FreeBSD keltia.freenix.fr 4.0-CURRENT #70: Sat Feb 27 09:43:08 CET 1999 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Mar 29 13:29: 6 1999 Delivered-To: freebsd-smp@freebsd.org Received: from excalibur.lps.ens.fr (excalibur.lps.ens.fr [129.199.120.3]) by hub.freebsd.org (Postfix) with ESMTP id F18C314D65; Mon, 29 Mar 1999 13:29:03 -0800 (PST) (envelope-from Thierry.Besancon@lps.ens.fr) Received: (from besancon@localhost) by excalibur.lps.ens.fr (8.8.5/8.8.6) id VAA29618; Mon, 29 Mar 1999 21:26:00 GMT To: Ollivier Robert Cc: freebsd-smp@FreeBSD.ORG, Pierre.David@prism.uvsq.fr, jt@ratp.fr, Thierry.Besancon@tournesol.lps.ens.fr, freebsd-stable@FreeBSD.ORG Subject: Re: panic: pipeinit References: <199903290759.HAA23476@excalibur.lps.ens.fr> <19990329214827.A42380@keltia.freenix.fr> From: Thierry.Besancon@lps.ens.fr Date: 29 Mar 1999 23:25:59 +0200 In-Reply-To: Ollivier Robert's message of Mon, 29 Mar 1999 21:48:27 +0200 Message-ID: Lines: 24 X-Mailer: Gnus v5.3/Emacs 19.34 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Dixit Ollivier Robert (le Mon, 29 Mar 1999 21:48:27 +0200) : >> According to Thierry Besancon: >> > yesterday 3.1-stable and rebooted on a new kernel (with maxusers==512). >> >> Please lower down the value of maxusers. Try with 128. I switched to a 3.1-stable-990328 because of "Out of mbuf clusters - adjust NMBCLUSTERS or increase maxusers!" messages with 3.1-stable-990311 that seemed to freeze the workstation. The 3.1-stable-990311 kernel was already configured with maxusers==128. So I increased it to 512 in the 3.1-stable-990328 version (my bi Pentium II has 512 Mo ram). And instead of a stable kernel, I got something crashing every 5 minutes or less. I went back to maxusers==128 with 3.1-stable-990311 but given I got "Out of mbuf clusters - adjust NMBCLUSTERS or increase maxusers!" messages two or three times with that snapshot, I fear getting new ones... What's the rationale between RAM and maxusers and SMP (if concerned) ? Thierry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Mar 29 13:55:34 1999 Delivered-To: freebsd-smp@freebsd.org Received: from qartek.anzen.com (qartek.anzen.com [152.160.231.129]) by hub.freebsd.org (Postfix) with ESMTP id DEF5A14CEF for ; Mon, 29 Mar 1999 13:55:23 -0800 (PST) (envelope-from mundy@anzen.com) Received: by qartek.anzen.com; id QAA21666; Mon, 29 Mar 1999 16:55:04 -0500 (EST) Received: from banzai.anzen.com(152.160.231.5) by qartek.anzen.com via smap (4.1) id xma021657; Mon, 29 Mar 99 16:54:19 -0500 Received: by banzai.anzen.com; id QAA28167; Mon, 29 Mar 1999 16:54:17 -0500 (EST) Date: Mon, 29 Mar 1999 16:54:17 -0500 (EST) From: Matt Undy Reply-To: Matt Undy To: Thierry.Besancon@lps.ens.fr Cc: freebsd-smp@FreeBSD.ORG Subject: Re: panic: pipeinit In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org hmm..then maybe its a more genreal bug. I kept seeing it with a DEC Tulip, but i only saw it maybe once with an Ether EXpress Pro. I also continued to see it after changing maxuser form 32 to 64 and NMBCLUSTERS from 1024 to 4096. Matt On 29 Mar 1999 Thierry.Besancon@lps.ens.fr wrote: > Dixit Matt Undy (le Mon, 29 Mar 1999 16:35:11 -0500 (EST)) : > > >> > >> what eathernet card are you using? > >> > > I'm using a Intel EtherExpressPro 100B. > The driver is named fxp in the FreeBSD kernel. > > Thierry > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Mar 30 7:36:12 1999 Delivered-To: freebsd-smp@freebsd.org Received: from tornado.cisco.com (tornado.cisco.com [171.69.104.22]) by hub.freebsd.org (Postfix) with ESMTP id C5A9214E86; Tue, 30 Mar 1999 07:36:03 -0800 (PST) (envelope-from bmcgover@bmcgover-pc.cisco.com) Received: from bmcgover-pc.cisco.com (bmcgover-pc.cisco.com [171.69.104.147]) by tornado.cisco.com (8.8.5-Cisco.1/8.6.5) with ESMTP id KAA06727; Tue, 30 Mar 1999 10:35:44 -0500 (EST) Received: from bmcgover-pc.cisco.com (localhost.pa.dtd.cisco.com [127.0.0.1]) by bmcgover-pc.cisco.com (8.9.2/8.9.2) with ESMTP id KAA00331; Tue, 30 Mar 1999 10:35:42 -0500 (EST) (envelope-from bmcgover@bmcgover-pc.cisco.com) Message-Id: <199903301535.KAA00331@bmcgover-pc.cisco.com> To: hackers@freebsd.org, freebsd-smp@freebsd.org Subject: Compaq Presario 800... No Joy? Date: Tue, 30 Mar 1999 10:35:42 -0500 From: Brian McGovern Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org After having used a Compaq Presario 400 for a few months with FreeBSD in SMP mode (dual PII/450), I decied I'd try to get a beefier box by getting an 800 with some Ultra-2-Wide SCSI drives, a GB of RAM, and the whole deal. Unfortunately, it appears not to work with both processors installed and trying to run an SMP kernel. Basically, when I boot up (either verbose or not), it gets through the memory checks ok, and then displays: panic: assign_apic_irq: inconsistent table mp_lock = 00000001; cpuid=0; lapic_id = ffffffff Followed by the standard "Rebooting in 15 seconds..." message of the panic/reboot. The listing from mptable is below. I'm suspecting since the extended table is "HOSED" that the motherboard is not Intel MP compliant (from the mptable man page), and I'm screwed. But, I thought I'd ask in case anyone has some ideas, or maybe a fix I'm not aware of, to get the machine up and running. Make world's and make release's scream with the 768MB of MFS... Its too bad I can't get the second processor going. Anyhow, comments welcome. -Brian =============================================================================== MPTable, version 2.0.15 ------------------------------------------------------------------------------- MP Floating Pointer Structure: location: BIOS physical address: 0x000f4ff0 signature: '_MP_' length: 16 bytes version: 1.4 checksum: 0x00 mode: Virtual Wire ------------------------------------------------------------------------------- MP Config Table Header: physical address: 0x000f25dc signature: 'PCMP' base table length: 484 version: 1.4 checksum: 0x9a OEM ID: 'COMPAQ ' Product ID: 'PROLIANT ' OEM table pointer: 0x00000000 OEM table size: 0 entry count: 52 local APIC address: 0xfee00000 extended table length: 76 extended table checksum: 86 ------------------------------------------------------------------------------- MP Config Base Table Entries: -- Processors: APIC ID Version State Family Model Step Flags 0 0x10 BSP, usable 6 2 1 0x0381 0 0x10 AP, usable 6 5 2 0x183fbff -- Bus: Bus ID Type 0 PCI 1 PCI 9 ISA -- I/O APICs: APIC ID Version State Address 8 0x11 usable 0xfec00000 -- I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN# INT active-lo level 1 10:A 8 17 INT active-lo level 1 10:B 8 16 INT active-lo level 1 10:C 8 17 INT active-lo level 1 10:D 8 16 INT active-lo level 1 9:A 8 19 INT active-lo level 1 9:B 8 18 INT active-lo level 1 9:C 8 19 INT active-lo level 1 9:D 8 18 INT active-lo level 1 8:A 8 21 INT active-lo level 1 8:B 8 20 INT active-lo level 1 8:C 8 21 INT active-lo level 1 8:D 8 20 INT active-lo level 1 7:A 8 23 INT active-lo level 1 7:B 8 22 INT active-lo level 1 7:C 8 23 INT active-lo level 1 7:D 8 22 INT active-lo level 0 15:A 8 25 INT active-lo level 0 15:B 8 24 INT active-lo level 0 15:C 8 25 INT active-lo level 0 15:D 8 24 INT active-lo level 0 13:A 8 27 INT active-lo level 0 13:B 8 26 INT active-lo level 0 13:C 8 27 INT active-lo level 0 13:D 8 26 INT active-lo level 0 6:A 8 31 INT active-lo level 0 6:B 8 30 INT active-lo level 0 7:A 8 29 INT active-lo level 0 8:A 8 28 INT active-hi edge 9 1 8 1 INT active-hi edge 9 0 8 2 INT active-hi edge 9 3 8 3 INT active-hi edge 9 4 8 4 INT active-hi edge 9 5 8 5 INT active-hi edge 9 6 8 6 INT active-hi edge 9 7 8 7 INT active-hi edge 9 8 8 8 INT active-hi edge 9 9 8 9 INT active-hi edge 9 10 8 10 INT active-hi edge 9 11 8 11 INT active-hi edge 9 12 8 12 INT active-lo edge 9 13 8 13 INT active-hi edge 9 14 8 14 INT active-hi edge 9 15 8 15 -- Local Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN# ExtINT conforms conforms 9 0 255 0 NMI conforms conforms 9 0 255 1 ------------------------------------------------------------------------------- MP Config Extended Table Entries: Extended Table HOSED! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Mar 30 8:27:48 1999 Delivered-To: freebsd-smp@freebsd.org Received: from verdi.nethelp.no (verdi.nethelp.no [158.36.41.162]) by hub.freebsd.org (Postfix) with SMTP id 4D44B15B22 for ; Tue, 30 Mar 1999 08:27:42 -0800 (PST) (envelope-from sthaug@nethelp.no) Received: (qmail 10691 invoked by uid 1001); 30 Mar 1999 16:27:22 +0000 (GMT) To: bmcgover@cisco.com Cc: hackers@freebsd.org, freebsd-smp@freebsd.org Subject: Re: Compaq Presario 800... No Joy? From: sthaug@nethelp.no In-Reply-To: Your message of "Tue, 30 Mar 1999 10:35:42 -0500" References: <199903301535.KAA00331@bmcgover-pc.cisco.com> X-Mailer: Mew version 1.05+ on Emacs 19.34.2 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Date: Tue, 30 Mar 1999 18:27:21 +0200 Message-ID: <10689.922811241@verdi.nethelp.no> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > After having used a Compaq Presario 400 for a few months with FreeBSD in SMP > mode (dual PII/450), I decied I'd try to get a beefier box by getting an 800 > with some Ultra-2-Wide SCSI drives, a GB of RAM, and the whole deal. > > Unfortunately, it appears not to work with both processors installed and trying > to run an SMP kernel. > > Basically, when I boot up (either verbose or not), it gets through the memory > checks ok, and then displays: > > panic: assign_apic_irq: inconsistent table > mp_lock = 00000001; cpuid=0; lapic_id = ffffffff > > Followed by the standard "Rebooting in 15 seconds..." message of the > panic/reboot. 1. Have you recompiled your kernel with a higher number of NINTR? 2. How recent is your kernel? Spefically, does it include Tor Egge's patch shown below? Steinar Haug, Nethelp consulting, sthaug@nethelp.no ---------------------------------------------------------------------- To: sigpet@islandia.is Cc: freebsd-smp@FreeBSD.ORG Subject: Re: Problem - Compaq Proliant 2500 and SMP From: Tor.Egge@fast.no Date: Fri, 26 Feb 1999 03:45:29 +0100 > Hi. > I've bean scrolling throug the archives and I cant find any solution to my > problem. > > I have Compaq Proliant 2500 with 2x200 Mhz Pentium Pro CPUs. 128 Mb RAM > I have configured the APIC option in System Configuration ( BIOS ) to FULL > TABLE but still it crashes on boot up with this: > > assign_apic_irq:inconsistent table > MP_LOCK=0000001 ; CPUID=0 ; lapic=01000000 Try this patch: Index: mp_machdep.c =================================================================== RCS file: /home/ncvs/src/sys/i386/i386/mp_machdep.c,v retrieving revision 1.89 diff -u -r1.89 mp_machdep.c --- mp_machdep.c 1999/01/28 01:59:50 1.89 +++ mp_machdep.c 1999/02/26 02:43:04 @@ -1090,7 +1090,7 @@ int_to_apicintpin[x].redirindex = 0; } for (x = 0; x < nintrs; x++) { - if (io_apic_ints[x].dst_apic_int <= APIC_INTMAPSIZE && + if (io_apic_ints[x].dst_apic_int < APIC_INTMAPSIZE && io_apic_ints[x].dst_apic_id == IO_TO_ID(0) && io_apic_ints[x].int_vector == 0xff && (io_apic_ints[x].int_type == 0 || - Tor Egge To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Mar 30 8:52:57 1999 Delivered-To: freebsd-smp@freebsd.org Received: from tornado.cisco.com (tornado.cisco.com [171.69.104.22]) by hub.freebsd.org (Postfix) with ESMTP id 655EA15A6C; Tue, 30 Mar 1999 08:52:55 -0800 (PST) (envelope-from bmcgover@bmcgover-pc.cisco.com) Received: from bmcgover-pc.cisco.com (bmcgover-pc.cisco.com [171.69.104.147]) by tornado.cisco.com (8.8.5-Cisco.1/8.6.5) with ESMTP id LAA09266; Tue, 30 Mar 1999 11:52:35 -0500 (EST) Received: from bmcgover-pc.cisco.com (localhost.pa.dtd.cisco.com [127.0.0.1]) by bmcgover-pc.cisco.com (8.9.2/8.9.2) with ESMTP id LAA00482; Tue, 30 Mar 1999 11:52:35 -0500 (EST) (envelope-from bmcgover@bmcgover-pc.cisco.com) Message-Id: <199903301652.LAA00482@bmcgover-pc.cisco.com> To: sthaug@nethelp.no Cc: hackers@freebsd.org, freebsd-smp@freebsd.org Subject: Re: Compaq Presario 800... No Joy? In-reply-to: Your message of "Tue, 30 Mar 1999 18:27:21 +0200." <10689.922811241@verdi.nethelp.no> Date: Tue, 30 Mar 1999 11:52:35 -0500 From: Brian McGovern Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Well, tried the patch provided. It still doesn't work, but it changed the symptoms a bit. Now, rather than the panic, I get (note: the first line might be bogus... my notes aren't perfect): Programming 34 pins on IOAPIC #0: IOAPIC #0 intpin 24 -> irq -1 IOAPIC #0 intpin 25 -> irq -1 IOAPIC #0 intpin 26 -> irq -1 IOAPIC #0 intpin 27 -> irq -1 IOAPIC #0 intpin 28 -> irq -1 IOAPIC #0 intpin 29 -> irq -1 IOAPIC #0 intpin 30 -> irq -1 IOAPIC #0 intpin 31 -> irq -1 Then the machine hangs. -Brian To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Mar 30 9:17:30 1999 Delivered-To: freebsd-smp@freebsd.org Received: from mail.lh.net (mail.lh.net [207.48.52.203]) by hub.freebsd.org (Postfix) with ESMTP id 9E05C15B63 for ; Tue, 30 Mar 1999 09:17:28 -0800 (PST) (envelope-from pepper@lh.net) Received: from [207.48.52.241] by mail.lh.net via ESMTP (8.8.5/970220.SGI.BM.8.8.5) id RAA12511; Tue, 30 Mar 1999 17:16:23 GMT X-Sender: pepper@mail.lh.net Message-Id: In-Reply-To: <199903301652.LAA00482@bmcgover-pc.cisco.com> References: Your message of "Tue, 30 Mar 1999 18:27:21 +0200." <10689.922811241@verdi.nethelp.no> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Tue, 30 Mar 1999 11:16:25 -0600 To: Brian McGovern From: Tom Pepper Subject: Re: Compaq Presario 800... No Joy? Cc: freebsd-smp@freebsd.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Brian: Make sure you've gone into Compaq's disk-based BIOS and set up the APIC as Full Table - Mapped in the system configuration. In order to see this option, you have to press Ctrl-A from the main menu (where you select the System Configuration option -- if you do it successfully you'll see a message that you're in advanced mode). The reason you need to do this is Compaq's wacky BIOS changes the table based on the O/S you intend to load onto the machine, so don't reselect an O/S like NT, Netware, etc. once you've made the change. I've got four Proliant 1850R's which behaved similarly. They now work without any kernel patches whatsoever with dual p2/450s, although mptable still claims the extended table is hosed. My Proliant 5500R with dual Xeons is still broken, however. *sigh* Hope this helps. -T >Well, tried the patch provided. It still doesn't work, but it changed >the symptoms a bit. Now, rather than the panic, I get (note: the first >line might be bogus... my notes aren't perfect): > >Programming 34 pins on IOAPIC #0: >IOAPIC #0 intpin 24 -> irq -1 >IOAPIC #0 intpin 25 -> irq -1 >IOAPIC #0 intpin 26 -> irq -1 >IOAPIC #0 intpin 27 -> irq -1 >IOAPIC #0 intpin 28 -> irq -1 >IOAPIC #0 intpin 29 -> irq -1 >IOAPIC #0 intpin 30 -> irq -1 >IOAPIC #0 intpin 31 -> irq -1 > >Then the machine hangs. > -Brian > > > >To Unsubscribe: send mail to majordomo@FreeBSD.org >with "unsubscribe freebsd-smp" in the body of the message =========================================================================== Tom Pepper Vice President, Engineering pepper@lh.net Lighthouse Communications, Inc. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Mar 30 11:29:14 1999 Delivered-To: freebsd-smp@freebsd.org Received: from tornado.cisco.com (tornado.cisco.com [171.69.104.22]) by hub.freebsd.org (Postfix) with ESMTP id 0C1B214C3A for ; Tue, 30 Mar 1999 11:29:11 -0800 (PST) (envelope-from bmcgover@bmcgover-pc.cisco.com) Received: from bmcgover-pc.cisco.com (bmcgover-pc.cisco.com [171.69.104.147]) by tornado.cisco.com (8.8.5-Cisco.1/8.6.5) with ESMTP id OAA13941; Tue, 30 Mar 1999 14:28:52 -0500 (EST) Received: from bmcgover-pc.cisco.com (localhost.pa.dtd.cisco.com [127.0.0.1]) by bmcgover-pc.cisco.com (8.9.2/8.9.2) with ESMTP id OAA00286; Tue, 30 Mar 1999 14:28:51 -0500 (EST) (envelope-from bmcgover@bmcgover-pc.cisco.com) Message-Id: <199903301928.OAA00286@bmcgover-pc.cisco.com> To: Tom Pepper Cc: freebsd-smp@freebsd.org Subject: Joy! (was: Re: Compaq Presario 800... No Joy?) In-reply-to: Your message of "Tue, 30 Mar 1999 11:16:25 CST." Date: Tue, 30 Mar 1999 14:28:51 -0500 From: Brian McGovern Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Well, the full Compaq utility wasn't loaded on the box because it came driveless (Added 2 9.2GB Ultra-2-Wide SCSI LVD Disks...). However, I was able to get the utility from their website and drop it on some floppies. Found Advanced mode. Found the entry in the setup. Set it. Booted. We're golden (and I've got another reference note in my head about Compaqs). Thanks for all the help. I'm looking forward to doing a make release entirely in MFS. -Brian To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Apr 1 22: 8:39 1999 Delivered-To: freebsd-smp@freebsd.org Received: from mail.bewellnet.com (mail.bewellnet.com [209.54.96.221]) by hub.freebsd.org (Postfix) with SMTP id 3215B14CC2; Thu, 1 Apr 1999 22:08:12 -0800 (PST) (envelope-from epeters@bewellnet.com) Received: from fred [209.54.97.16] by mail.bewellnet.com (SMTPD32-4.06) id AFB73420138; Thu, 01 Apr 1999 23:12:07 MDT Message-ID: <000401be7cd0$6c88d0a0$106136d1@fred.nets> From: "Eric Peters" To: , Subject: Curse of the COMs... Date: Thu, 1 Apr 1999 23:17:01 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 4.72.3110.5 X-MimeOLE: Produced By Microsoft MimeOLE V4.72.3110.3 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org This one has me stumped. So, like any good(?) hardware tech, I try tach # 2 & still, this box despises me. Start: 1 Tyan dual processor 1696DLUA Thunder 2 ATX (Built in SCSI, Dual processor, etc). Misc hardware, plus: 1 33.6k ISA Modem, configured for Com 2: IRQ3 Addr 0x2f8 Modem works great ppp is all and well and happy, using 3-Stable built mid March. The change: Replace the motherboard with: 1 Tyan dual processor S1836 Thunder 100 (Equivalant to the above with a few noted differences: Includes built-in ethernet, supports 100Mhz CPU Bus frequency, & golly, that's a different BIOS to match) All other hardware remains the same. Ah, did need to recompile the kernel with different SMP params, but all was well, except the modem. It wasn't responding. All other parts of the system remained sane. Of note: The modem was on sio1, but opening sio0 (unconfigured in the BIOS, but detected during boot in place of sio1) hung the PC neat as anything. The generic kernel (which found the modem fine earlier) didn't see the modem either. I fiddled with BIOS params & modem jumpers until I turned blue, no help. Time for the other tach: I purchace 1 modem, careful to get a PCI modem & avoid all those extra ISA params in the BIOS. Strangely, all the PCI modems I saw were PnP ;). (3com/USR 56k faxmodem model 5698) controller pnp0 is already in the kernel, swap the modems, boot, & nothing, no PnP devices detected, no flattering comments about sio0-3 man pnp gives a few settings to configure with. pnp 1 0 enable os irq0 3 drq0 0 port 0 0x2f8 pnp 2 0 enable os irq0 4 drq0 0 port 0 0x3f8 (blanket measures...) Again, nothing. boot_verbose offers no clues for me to spot any problems with. I've also cut out all the parts & disabled all the MBoard features (sound & ethernet) I don't need, but something is still sticking. Not included due to lack of immediate availability, but available on request I fear that the Bios Params are causing the ISA modem it's problems, & something either overlooked by me or unsupported on the PnP side is balking my attempts with the PCI modem, but I fear I've run out of places to think to look. I'l entertain suggestions. ;) //Eric And in the good old days, you could tell a piece of hardware to behave thusly. It either behaved, or was in need of a warrenty check. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Apr 2 13:52:57 1999 Delivered-To: freebsd-smp@freebsd.org Received: from luva.lb.bawue.de (luva.lb.bawue.de [193.197.8.17]) by hub.freebsd.org (Postfix) with ESMTP id 2238614DE9 for ; Fri, 2 Apr 1999 13:52:37 -0800 (PST) (envelope-from migieger@luva.lb.bawue.de) Received: (from migieger@localhost) by luva.lb.bawue.de (8.9.0/8.9.0) id XAA02336 for freebsd-smp@freebsd.org; Fri, 2 Apr 1999 23:52:14 +0200 (CEST) From: Michael Giegerich Message-Id: <199904022152.XAA02336@luva.lb.bawue.de> Subject: FBSD-3.1-R w/ ASUS P/E-P55T2P4D: kernel panics - need help To: freebsd-smp@freebsd.org Date: Fri, 2 Apr 1999 23:52:14 +0200 (CEST) X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I've sent this already to cubfm some weeks ago without getting any response... So, sorry if someone is reading this twice. If MPS 1.4 is enabled in BIOS the kernel panics during boot with: =============================================================================== programming 16 pins in IOAPIC #0 EISA INTCONTROL = 00001e00 EISA IRQ 36 ?!?! panic: bad APIC IO INT flags mp_lock = 00000001; cpuid = 0; lapic.id = 00000000 =============================================================================== If MPS 1.1 is enabled I get: =============================================================================== panic: APIC_IO: Cannot route 8254 interrupt to CPU mp_lock = 00000001; cpuid = 0; lapic.id = 00000000 =============================================================================== The config file reads: =============================================================================== options SMP options APIC_IO #options NCPU=2 #options NBUS=2 #options NAPIC=1 #options NINTR=24 =============================================================================== Enabling: options SMP_TIMER_NC or commenting out NCPU etc options didn't help. This is mptable output for MPS 1.1: =============================================================================== MPTable, version 2.0.15 ------------------------------------------------------------------------------- MP Floating Pointer Structure: location: BIOS physical address: 0x000f6480 signature: '_MP_' length: 16 bytes version: 1.4 checksum: 0xbb mode: Virtual Wire ------------------------------------------------------------------------------- MP Config Table Header: physical address: 0x000f6076 signature: 'PCMP' base table length: 228 version: 1.4 checksum: 0x12 OEM ID: 'OEM00000' Product ID: 'PROD00000000' OEM table pointer: 0x00000000 OEM table size: 0 entry count: 20 local APIC address: 0xfee00000 extended table length: 0 extended table checksum: 0 ------------------------------------------------------------------------------- MP Config Base Table Entries: -- Processors: APIC ID Version State Family Model Step Flags 0 0x10 BSP, usable 5 4 3 0x8003bf 1 0x10 AP, usable 5 4 3 0x8003bf -- Bus: Bus ID Type 0 PCI 1 EISA -- I/O APICs: APIC ID Version State Address 2 0x11 usable 0xfec00000 -- I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN# INT conforms conforms 1 1 2 1 INT conforms conforms 1 3 2 3 INT conforms conforms 1 4 2 4 INT conforms conforms 1 5 2 5 INT conforms conforms 1 6 2 6 INT conforms conforms 1 7 2 7 INT conforms conforms 1 8 2 8 INT conforms conforms 1 11 2 11 INT conforms conforms 1 14 2 14 INT conforms conforms 1 15 2 15 INT conforms conforms 1 40 2 12 INT conforms conforms 1 44 2 10 INT conforms conforms 1 36 2 9 -- Local Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN# ExtINT active-hi edge 1 0 255 0 NMI active-hi edge 1 0 255 1 ------------------------------------------------------------------------------- # SMP kernel config file options: # Required: options SMP # Symmetric MultiProcessor Kernel options APIC_IO # Symmetric (APIC) I/O # Optional (built-in defaults will work in most cases): #options NCPU=2 # number of CPUs #options NBUS=2 # number of busses #options NAPIC=1 # number of IO APICs #options NINTR=24 # number of INTs =============================================================================== And here the same for MPS 1.4: =============================================================================== MPTable, version 2.0.15 ------------------------------------------------------------------------------- MP Floating Pointer Structure: location: BIOS physical address: 0x000f6480 signature: '_MP_' length: 16 bytes version: 1.4 checksum: 0xbb mode: Virtual Wire ------------------------------------------------------------------------------- MP Config Table Header: physical address: 0x000f6076 signature: 'PCMP' base table length: 228 version: 1.4 checksum: 0x12 OEM ID: 'OEM00000' Product ID: 'PROD00000000' OEM table pointer: 0x00000000 OEM table size: 0 entry count: 20 local APIC address: 0xfee00000 extended table length: 0 extended table checksum: 0 ------------------------------------------------------------------------------- MP Config Base Table Entries: -- Processors: APIC ID Version State Family Model Step Flags 0 0x10 BSP, usable 5 4 3 0x8003bf 1 0x10 AP, usable 5 4 3 0x8003bf -- Bus: Bus ID Type 0 PCI 1 EISA -- I/O APICs: APIC ID Version State Address 2 0x11 usable 0xfec00000 -- I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN# INT conforms conforms 1 1 2 1 INT conforms conforms 1 3 2 3 INT conforms conforms 1 4 2 4 INT conforms conforms 1 5 2 5 INT conforms conforms 1 6 2 6 INT conforms conforms 1 7 2 7 INT conforms conforms 1 8 2 8 INT conforms conforms 1 11 2 11 INT conforms conforms 1 14 2 14 INT conforms conforms 1 15 2 15 INT conforms conforms 1 40 2 12 INT conforms conforms 1 44 2 10 INT conforms conforms 1 36 2 9 -- Local Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN# ExtINT active-hi edge 1 0 255 0 NMI active-hi edge 1 0 255 1 ------------------------------------------------------------------------------- # SMP kernel config file options: # Required: options SMP # Symmetric MultiProcessor Kernel options APIC_IO # Symmetric (APIC) I/O # Optional (built-in defaults will work in most cases): #options NCPU=2 # number of CPUs #options NBUS=2 # number of busses #options NAPIC=1 # number of IO APICs #options NINTR=24 # number of INTs =============================================================================== And to complete information dmesg's output: Preloaded elf kernel "kernel.GENERIC" at 0xf0340000. eisa0: Probing for devices on the EISA bus ahb0: at 0x4c00-0x4cff irq 11 on eisa0 slot 4 ahb0: AHA1740A Single Ended SCSI Adapter, FW Rev. I , ID=7, 64 ECBs Probing for devices on PCI bus 0: chip0: rev 0x03 on pci0.0.0 chip1: rev 0x05 on pci0.7.0 vga0: rev 0x02 int a irq 9 on pci0.9.0 ahc0: rev 0x03 int a irq 12 on pci0.10.0 ahc0: aic7860 Single Channel A, SCSI Id=7, 3/255 SCBs de0: rev 0x21 int a irq 10 on pci0.11.0 de0: 21041 [10Mb/s] pass 2.1 de0: address 00:40:05:a4:dc:12 ide_pci0: rev 0x01 int a irq 14 on pci0.13.0 Probing for devices on the ISA bus: sc0 on isa sc0: VGA color <16 virtual consoles, flags=0x0> atkbdc0 at 0x60-0x6f on motherboard atkbd0 irq 1 on isa psm0 not found sio0 at 0x3f8-0x3ff irq 4 flags 0x10 on isa sio0: type 16550A sio1 at 0x2f8-0x2ff irq 3 on isa sio1: type 16550A fdc0 at 0x3f0-0x3f7 irq 6 drq 2 on isa fdc0: FIFO enabled, 8 bytes threshold fd0: 1.44MB 3.5in fd1: 1.2MB 5.25in wdc0 at 0x1f0-0x1f7 irq 14 on isa wdc0: unit 0 (wd0): wd0: 407MB (833616 sectors), 827 cyls, 16 heads, 63 S/T, 512 B/S wdc1 not found at 0x170 ppc0 at 0x378 irq 7 on isa ppc0: SMC FDC37C665GT chipset (PS2/NIBBLE) in COMPATIBLE mode nlpt0: on ppbus 0 nlpt0: Interrupt-driven port ppi0: on ppbus 0 plip0: on ppbus 0 vga0 at 0x3b0-0x3df maddr 0xa0000 msize 131072 on isa npx0 on motherboard npx0: INT 16 interface Intel Pentium detected, installing workaround for F00F bug Waiting 15 seconds for SCSI devices to settle ahb0: SCSI Bus Reset Delivered de0: enabling BNC port changing root device to wd0s3a cd0 at ahb0 bus 0 target 2 lun 0 cd0: Removable CD-ROM SCSI-2 device cd0: cd present [322265 x 2048 byte records] da0 at ahb0 bus 0 target 1 lun 0 da0: Fixed Direct Access SCSI-2 device da0: Tagged Queueing Enabled da0: 2063MB (4226725 512 byte sectors: 64H 32S/T 2063C) wd0s2: rejecting BSD label: raw partition offset != slice offset wd0s2: start 65520, end 734831, size 669312 wd0s2c: start 0, end 832607, size 832608 wd0s2: rejecting BSD label: raw partition offset != slice offset wd0s2: start 65520, end 734831, size 669312 wd0s2c: start 0, end 832607, size 832608 wd0s2: rejecting BSD label: raw partition offset != slice offset wd0s2: start 65520, end 734831, size 669312 wd0s2c: start 0, end 832607, size 832608 Anybody able to help? Bye, -- Michael Giegerich, mail: migieger@luva.lb.bawue.de, phone: +49 7144 881806 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Apr 8 12:47:17 1999 Delivered-To: freebsd-smp@freebsd.org Received: from noc.demon.net (server.noc.demon.net [193.195.224.4]) by hub.freebsd.org (Postfix) with ESMTP id 52C3214F0F for ; Thu, 8 Apr 1999 12:46:23 -0700 (PDT) (envelope-from fanf@demon.net) Received: by noc.demon.net; id UAA28762; Thu, 8 Apr 1999 20:44:20 +0100 (BST) Received: from fanf.noc.demon.net(195.11.55.83) by inside.noc.demon.net via smap (3.2) id xma028744; Thu, 8 Apr 99 20:44:10 +0100 Received: from fanf by fanf.noc.demon.net with local (Exim 1.73 #2) id 10VKig-0006Y3-00; Thu, 8 Apr 1999 20:44:10 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit From: Tony Finch To: smp@freebsd.org Subject: concurrent select()s on listen socket broken under SMP X-Mailer: VM 6.34 under Emacs 19.34.1 Message-Id: Date: Thu, 8 Apr 1999 20:44:10 +0100 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org We discovered something odd this evening: We have a modified thttpd which forks several times between opening its listen socket and dropping into the big select() loop. There is a difference in behaviour between uniprocessor machines and SMP machines when a connection arrives. On a uniprocessor machine, select() only tells one process that a connection is available. On a dual processor machine, two processes are told that a connection is available: both processes then go on to accept() the connection and one of them succeeds but the other blocks. This upsets thttpd greatly because it expects the accept() to be instantaneous for the purpose of calculating timeouts. Because we are on a bastard deadline our current fix is to use a uniprocessor kernel, but this is a little bit wasteful. A fix would be nice... Tony. -- f.a.n.finch dot@dotat.at fanf@demon.net To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Apr 8 13: 4:13 1999 Delivered-To: freebsd-smp@freebsd.org Received: from databus.databus.com (databus.databus.com [198.186.154.34]) by hub.freebsd.org (Postfix) with SMTP id CECB114D57 for ; Thu, 8 Apr 1999 13:04:08 -0700 (PDT) (envelope-from barney@databus.databus.com) From: Barney Wolff To: smp@freebsd.org Date: Thu, 8 Apr 1999 15:59 EDT Subject: Re: concurrent select()s on listen socket broken under SMP Content-Length: 1314 Content-Type: text/plain Message-ID: <370d0b3a0.466c@databus.databus.com> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Seems to me you need to set the socket non-blocking, and then handle the EWOULDBLOCK on the fd that loses. I don't think this should be considered a kernel error. Barney Wolff > From: Tony Finch > Date: Thu, 8 Apr 1999 20:44:10 +0100 > > We discovered something odd this evening: > > We have a modified thttpd which forks several times between opening > its listen socket and dropping into the big select() loop. There is a > difference in behaviour between uniprocessor machines and SMP machines > when a connection arrives. > > On a uniprocessor machine, select() only tells one process that a > connection is available. On a dual processor machine, two processes > are told that a connection is available: both processes then go on to > accept() the connection and one of them succeeds but the other blocks. > This upsets thttpd greatly because it expects the accept() to be > instantaneous for the purpose of calculating timeouts. > > Because we are on a bastard deadline our current fix is to use a > uniprocessor kernel, but this is a little bit wasteful. A fix would be > nice... > > Tony. > -- > f.a.n.finch dot@dotat.at fanf@demon.net > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-smp" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Apr 8 13: 7:47 1999 Delivered-To: freebsd-smp@freebsd.org Received: from alive.znep.com (sense-sea-MegaSub-1-222.oz.net [216.39.144.222]) by hub.freebsd.org (Postfix) with ESMTP id D035014DC1 for ; Thu, 8 Apr 1999 13:07:41 -0700 (PDT) (envelope-from marcs@znep.com) Received: from localhost (marcs@localhost) by alive.znep.com (8.9.1/8.9.1) with ESMTP id NAA01900; Thu, 8 Apr 1999 13:06:31 -0700 (PDT) (envelope-from marcs@znep.com) Date: Thu, 8 Apr 1999 13:06:31 -0700 (PDT) From: Marc Slemko To: Tony Finch Cc: smp@FreeBSD.ORG Subject: Re: concurrent select()s on listen socket broken under SMP In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Thu, 8 Apr 1999, Tony Finch wrote: > We discovered something odd this evening: > > We have a modified thttpd which forks several times between opening > its listen socket and dropping into the big select() loop. There is a > difference in behaviour between uniprocessor machines and SMP machines > when a connection arrives. > > On a uniprocessor machine, select() only tells one process that a > connection is available. On a dual processor machine, two processes > are told that a connection is available: both processes then go on to > accept() the connection and one of them succeeds but the other blocks. > This upsets thttpd greatly because it expects the accept() to be > instantaneous for the purpose of calculating timeouts. There is a race condition in the idea that having multiple processes doing a select() then an accept() on the same descriptor should always result in an accept() for each success from accept(). It is not an OS problem, although various OSes may behave in different ways. This is one of the reasons why Apache uses an "accept mutex". It essentially does the same thing if you have multiple listening sockets. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Apr 8 13:51: 0 1999 Delivered-To: freebsd-smp@freebsd.org Received: from chiark.greenend.org.uk (chiark.greenend.org.uk [195.224.76.132]) by hub.freebsd.org (Postfix) with ESMTP id B82DB14E67 for ; Thu, 8 Apr 1999 13:50:44 -0700 (PDT) (envelope-from fanf@chiark.greenend.org.uk) Received: from fanf by chiark.greenend.org.uk with local (Exim 2.02 #1) id 10VLj4-0001LF-00 (Debian); Thu, 8 Apr 1999 21:48:38 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <14093.5670.813002.917842@chiark.greenend.org.uk> Date: Thu, 8 Apr 1999 21:48:38 +0100 (BST) From: Tony Finch To: Marc Slemko Cc: Tony Finch , smp@FreeBSD.ORG Subject: Re: concurrent select()s on listen socket broken under SMP In-Reply-To: References: X-Mailer: VM 6.47 under Emacs 19.34.1 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Marc Slemko writes: > > There is a race condition in the idea that having multiple processes doing > a select() then an accept() on the same descriptor should always result in > an accept() for each success from accept(). It is not an OS problem, > although various OSes may behave in different ways. > > This is one of the reasons why Apache uses an "accept mutex". It > essentially does the same thing if you have multiple listening sockets. Ah, I have heard of this before then, although I didn't think it would have this kind of effect. I had a quick dig around the kernel code to see if there were any likely changes that I could make. What breaks if I change the wakeup((caddr_t)&sb->sb_cc); in sowakeup() (line 319 of uipc_socket2.c) to a wakeup_one()? Tony. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Apr 8 13:57:53 1999 Delivered-To: freebsd-smp@freebsd.org Received: from alpha.netaccess.on.ca (netaccess.on.ca [199.243.225.10]) by hub.freebsd.org (Postfix) with ESMTP id B352214F3D for ; Thu, 8 Apr 1999 13:57:43 -0700 (PDT) (envelope-from rob@ControlQ.com) Received: from fatlady.controlq.com (dial143.netaccess.on.ca [204.101.178.98]) by alpha.netaccess.on.ca (8.8.5/8.7.3) with SMTP id QAA27806; Thu, 8 Apr 1999 16:55:20 -0400 (EDT) Date: Thu, 8 Apr 1999 17:04:27 -0400 (EDT) From: "Robert S. Sciuk" To: Tony Finch Cc: Marc Slemko , smp@FreeBSD.ORG Subject: Re: concurrent select()s on listen socket broken under SMP In-Reply-To: <14093.5670.813002.917842@chiark.greenend.org.uk> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Thu, 8 Apr 1999, Tony Finch wrote: > Marc Slemko writes: > > > > There is a race condition in the idea that having multiple processes doing > > a select() then an accept() on the same descriptor should always result in > > an accept() for each success from accept(). It is not an OS problem, > > although various OSes may behave in different ways. > > > > This is one of the reasons why Apache uses an "accept mutex". It > > essentially does the same thing if you have multiple listening sockets. > > Ah, I have heard of this before then, although I didn't think it would > have this kind of effect. I had a quick dig around the kernel code to > see if there were any likely changes that I could make. What breaks if > I change the wakeup((caddr_t)&sb->sb_cc); in sowakeup() (line 319 of > uipc_socket2.c) to a wakeup_one()? Why not do it and see what breaks?? Just kidding ... you may want to re-consider the suggestion of putting a mutex around the accept ... use SYSVSEM's for portability just in case this behaviour is apparent on other OS'es ... You'll serialize on the accept, but hey! you need to do that anyways. man semop() semctl() semget() This way you won't have to suffer the consequences of an obscurely hacked kernel, nor do you have to explain to your customer why a kernel rebuild is neccessary to install an application 8-). IMHO anyways ... Cheers, Rob. > > Tony. > -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Robert S. Sciuk 1032 Howard Rd. PO Box 6A Ph:905 632-2466 Control-Q Research Burlington, Ont. Canada Fx:905 632-7417 rob@ControlQ.com L7R 3X5 http://www.ControlQ.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Apr 8 14: 7:14 1999 Delivered-To: freebsd-smp@freebsd.org Received: from chiark.greenend.org.uk (chiark.greenend.org.uk [195.224.76.132]) by hub.freebsd.org (Postfix) with ESMTP id 3DE2314D06 for ; Thu, 8 Apr 1999 14:07:05 -0700 (PDT) (envelope-from fanf@chiark.greenend.org.uk) Received: from fanf by chiark.greenend.org.uk with local (Exim 2.02 #1) id 10VLyp-0001UO-00 (Debian); Thu, 8 Apr 1999 22:04:55 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <14093.6647.297622.735051@chiark.greenend.org.uk> Date: Thu, 8 Apr 1999 22:04:55 +0100 (BST) From: Tony Finch To: "Robert S. Sciuk" Cc: Tony Finch , Marc Slemko , smp@FreeBSD.ORG Subject: Re: concurrent select()s on listen socket broken under SMP In-Reply-To: References: <14093.5670.813002.917842@chiark.greenend.org.uk> X-Mailer: VM 6.47 under Emacs 19.34.1 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Robert S. Sciuk writes: > > Why not do it and see what breaks?? Just kidding ... you > may want to re-consider the suggestion of putting a mutex > around the accept ... use SYSVSEM's for portability just > in case this behaviour is apparent on other OS'es ... You'll > serialize on the accept, but hey! you need to do that anyways. Well, that's not entirely desirable if there's more than one connection be accept()ed at any one time. > This way you won't have to suffer the consequences of an obscurely > hacked kernel, nor do you have to explain to your customer why a > kernel rebuild is neccessary to install an application 8-). We are the only people who need to know, and we aren't scared of obscurely hacked kernels :-) Tony. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Apr 8 15:11:23 1999 Delivered-To: freebsd-smp@freebsd.org Received: from alive.znep.com (sense-sea-MegaSub-1-222.oz.net [216.39.144.222]) by hub.freebsd.org (Postfix) with ESMTP id 2814514D7B for ; Thu, 8 Apr 1999 15:11:21 -0700 (PDT) (envelope-from marcs@znep.com) Received: from localhost (marcs@localhost) by alive.znep.com (8.9.1/8.9.1) with ESMTP id PAA02392; Thu, 8 Apr 1999 15:10:13 -0700 (PDT) (envelope-from marcs@znep.com) Date: Thu, 8 Apr 1999 15:10:13 -0700 (PDT) From: Marc Slemko To: Tony Finch Cc: smp@FreeBSD.ORG Subject: Re: concurrent select()s on listen socket broken under SMP In-Reply-To: <14093.5670.813002.917842@chiark.greenend.org.uk> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Thu, 8 Apr 1999, Tony Finch wrote: > Marc Slemko writes: > > > > There is a race condition in the idea that having multiple processes doing > > a select() then an accept() on the same descriptor should always result in > > an accept() for each success from accept(). It is not an OS problem, > > although various OSes may behave in different ways. > > > > This is one of the reasons why Apache uses an "accept mutex". It > > essentially does the same thing if you have multiple listening sockets. > > Ah, I have heard of this before then, although I didn't think it would > have this kind of effect. I had a quick dig around the kernel code to > see if there were any likely changes that I could make. What breaks if > I change the wakeup((caddr_t)&sb->sb_cc); in sowakeup() (line 319 of > uipc_socket2.c) to a wakeup_one()? That could just break a whole lot of things I think. For example, if you have two processes in a select(), and two descriptors are ready, then only one process may return from the select with two descriptors ready, then proceed to handle one, while the other process sits there sleeping until something else wakes it up. In real life, this could lead to starvation. You also need to consider possible other starvation issues (which, in some situations, may actually be beneficial to performance) that could result in unfair distribution of connections between processes. The basic reason behind this is (aside from the non-standard behaviour and fairness issues) is that select() can notify a process that multiple sockets are ready to be dealt with, while a process will probably only be able to deal with one at a time. But I'm no expert. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Apr 8 15:46: 7 1999 Delivered-To: freebsd-smp@freebsd.org Received: from implode.root.com (root.com [209.102.106.178]) by hub.freebsd.org (Postfix) with ESMTP id A528114F03 for ; Thu, 8 Apr 1999 15:45:59 -0700 (PDT) (envelope-from root@implode.root.com) Received: from implode.root.com (localhost [127.0.0.1]) by implode.root.com (8.8.8/8.8.5) with ESMTP id PAA21435; Thu, 8 Apr 1999 15:41:51 -0700 (PDT) Message-Id: <199904082241.PAA21435@implode.root.com> To: Tony Finch Cc: Marc Slemko , smp@FreeBSD.ORG Subject: Re: concurrent select()s on listen socket broken under SMP In-reply-to: Your message of "Thu, 08 Apr 1999 21:48:38 BST." <14093.5670.813002.917842@chiark.greenend.org.uk> From: David Greenman Reply-To: dg@root.com Date: Thu, 08 Apr 1999 15:41:51 -0700 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >Marc Slemko writes: >> >> There is a race condition in the idea that having multiple processes doing >> a select() then an accept() on the same descriptor should always result in >> an accept() for each success from accept(). It is not an OS problem, >> although various OSes may behave in different ways. >> >> This is one of the reasons why Apache uses an "accept mutex". It >> essentially does the same thing if you have multiple listening sockets. > >Ah, I have heard of this before then, although I didn't think it would >have this kind of effect. I had a quick dig around the kernel code to >see if there were any likely changes that I could make. What breaks if >I change the wakeup((caddr_t)&sb->sb_cc); in sowakeup() (line 319 of >uipc_socket2.c) to a wakeup_one()? In addition to the other comments that have already been made, I would like to also point out that wakeup_one() wakes "at least one process", not "at most one process". It can and will wakeup more than one process depending on process states. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Apr 8 19: 5:10 1999 Delivered-To: freebsd-smp@freebsd.org Received: from smtp03.primenet.com (smtp03.primenet.com [206.165.6.133]) by hub.freebsd.org (Postfix) with ESMTP id 65C3514CAB for ; Thu, 8 Apr 1999 19:05:08 -0700 (PDT) (envelope-from tlambert@usr04.primenet.com) Received: (from daemon@localhost) by smtp03.primenet.com (8.8.8/8.8.8) id TAA15310; Thu, 8 Apr 1999 19:03:07 -0700 (MST) Received: from usr04.primenet.com(206.165.6.204) via SMTP by smtp03.primenet.com, id smtpd015288; Thu Apr 8 19:03:04 1999 Received: (from tlambert@localhost) by usr04.primenet.com (8.8.5/8.8.5) id TAA14284; Thu, 8 Apr 1999 19:03:02 -0700 (MST) From: Terry Lambert Message-Id: <199904090203.TAA14284@usr04.primenet.com> Subject: Re: concurrent select()s on listen socket broken under SMP To: dot@dotat.at (Tony Finch) Date: Fri, 9 Apr 1999 02:03:02 +0000 (GMT) Cc: smp@FreeBSD.ORG In-Reply-To: from "Tony Finch" at Apr 8, 99 08:44:10 pm X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > We discovered something odd this evening: > > We have a modified thttpd which forks several times between opening > its listen socket and dropping into the big select() loop. There is a > difference in behaviour between uniprocessor machines and SMP machines > when a connection arrives. > > On a uniprocessor machine, select() only tells one process that a > connection is available. On a dual processor machine, two processes > are told that a connection is available: both processes then go on to > accept() the connection and one of them succeeds but the other blocks. > This upsets thttpd greatly because it expects the accept() to be > instantaneous for the purpose of calculating timeouts. > > Because we are on a bastard deadline our current fix is to use a > uniprocessor kernel, but this is a little bit wasteful. A fix would be > nice... IMO, a socket in a state wating for a connection should not select true to multiple user space processes simultaneously. This is pretty obviously an SMP specific problem, since the uniprocessor case works around the problem. I think that what you would need would be a fix in the select code itself to respect the big giant lock. Looking at this code, it appears that it is not SMP safe. It calculates things based on state before a sleep, and then uses the cached information after the tsleep in the case of a retry. The poll call appears to have the same problem. Since only one process can be in the kernel at a time, it seems to me that what's happening is that the process that is in the kernel tsleep's, allowing another process to enter the kernel, and then at interrupt time, the wakeup causes both to run, one on each processor. And then one of them blocks in the accept() after it loses the race. One fix should be to use the wakeup_one(). It looks like this is broken, though, since it allows a "goto restart" if the process it wanted to wakeup is asleep. In terms of using a select on an fd that represents a mux in order to do "hot scheduling", this appears to be a lose anyway; processes selecting on the same ident should be serviced in LIFO order, given an LRU policy for paging (a fix for this would probably make FreeBSD a much faster server for "work to do" services that run multiple identical engine processes and make no state distinction between them -- e.g., Apache). Another fix might be to obey the lock around the select. This is mildly problematic, since it means that if you have something blocked on it, another process can not select on the same object unlit the first is done. This is both FIFO (and therefore bad, though "fair"), and it doesn't take into account kernel reentrancy on the basis of an interrupt on the same object (e.g. where the wakeup is called). You could probably hack this for network connections, specifically, under the theory that it's not a real interrupt, but a NETISR, that is doing the call, so you are both running in the same protection conflict domain for the lock. This would basically boil down to adding a field to the select structure, blocking entries on sleep, and not blocking them when they want to wakeup. This would leave a "thundering herd" waiting to sleep on the object that just selected true, but it would be one running on a single processor in the kernel. Probably the easiest way to accomplish this would be to make up a structure, call it an "smp_tsleep_context", pass it to an "smp_tsleep" function, and then dereference the select object ident out of the structure. Then you: put an smp_sleep_context in the struct you are passing a "ident", NOT as the first entry (or the ident would be the same) set the "ident" member to point to the real "ident" call smp_sleep smp_tsleep: grab the SMP lock (implicit) sleep on the addr of the sleep context, if it's in use if it's not, set it "in use" sleep on the ident when you wake up, set the context not in use wakeup the address of the context return as if fdrom a normal tsleep This means that the wakeups on the context only occur in kernel mode, not interrupt mode, while the wakeups on the ident can occur at interrupt. This basically means bloating the structure you are selecting on in order to make it SMP safe. This is obviously a trivial implementation; a high granularity SMP kernel would be able to do this serialization with far less work and far less bloat. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Apr 8 19: 5:57 1999 Delivered-To: freebsd-smp@freebsd.org Received: from smtp04.primenet.com (smtp04.primenet.com [206.165.6.134]) by hub.freebsd.org (Postfix) with ESMTP id 3420314DA4 for ; Thu, 8 Apr 1999 19:05:48 -0700 (PDT) (envelope-from tlambert@usr04.primenet.com) Received: (from daemon@localhost) by smtp04.primenet.com (8.8.8/8.8.8) id UAA12221; Thu, 8 Apr 1999 20:22:18 -0700 (MST) Received: from usr04.primenet.com(206.165.6.204) via SMTP by smtp04.primenet.com, id smtpd012199; Thu Apr 8 20:22:06 1999 Received: (from tlambert@localhost) by usr04.primenet.com (8.8.5/8.8.5) id TAA14310; Thu, 8 Apr 1999 19:03:33 -0700 (MST) From: Terry Lambert Message-Id: <199904090203.TAA14310@usr04.primenet.com> Subject: Re: concurrent select()s on listen socket broken under SMP To: barney@databus.com (Barney Wolff) Date: Fri, 9 Apr 1999 02:03:33 +0000 (GMT) Cc: smp@FreeBSD.ORG In-Reply-To: <370d0b3a0.466c@databus.databus.com> from "Barney Wolff" at Apr 8, 99 03:59:00 pm X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Seems to me you need to set the socket non-blocking, and then handle > the EWOULDBLOCK on the fd that loses. I don't think this should be > considered a kernel error. It's SMP specific. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Apr 8 22: 7:27 1999 Delivered-To: freebsd-smp@freebsd.org Received: from databus.databus.com (databus.databus.com [198.186.154.34]) by hub.freebsd.org (Postfix) with SMTP id ADB1A14ED7 for ; Thu, 8 Apr 1999 22:07:18 -0700 (PDT) (envelope-from barney@databus.databus.com) From: Barney Wolff To: smp@FreeBSD.ORG Date: Fri, 9 Apr 1999 00:35 EDT Subject: Re: concurrent select()s on listen socket broken under SMP Content-Length: 1209 Content-Type: text/plain Message-ID: <370d8a8b0.4bf5@databus.databus.com> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Different observed behavior UP vs MP does not constitute kernel error, unless the behavior is specifically defined. In this case, select() on the same fd by two processes has no defined behavior, that I'm aware of. (Not that I've done a lot of searching on this topic.) Nor has there ever been any guarantee that a condition discovered by select() will remain true until a subsequent system call. It *usually* does, but that's all. On a very pragmatic basis, if you really can't ever block, set nonblock and be prepared for the occasional cases when the hint given by select doesn't pan out. As someone who first experienced MP bugs in an application over 20 years ago, let me just say that it is a losing cause to count on *undocumented* system behavior staying the same when going from UP to MP. Then again, it's unwise to count on undocumented system behavior, period. Barney Wolff > From: Terry Lambert > Date: Fri, 9 Apr 1999 02:03:33 +0000 (GMT) > > > Seems to me you need to set the socket non-blocking, and then handle > > the EWOULDBLOCK on the fd that loses. I don't think this should be > > considered a kernel error. > > It's SMP specific. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Apr 8 23:55: 5 1999 Delivered-To: freebsd-smp@freebsd.org Received: from spinner.netplex.com.au (spinner.netplex.com.au [202.12.86.3]) by hub.freebsd.org (Postfix) with ESMTP id EB76614F6F for ; Thu, 8 Apr 1999 23:54:50 -0700 (PDT) (envelope-from peter@netplex.com.au) Received: from spinner.netplex.com.au (localhost [127.0.0.1]) by spinner.netplex.com.au (8.9.3/8.9.3/Netplex) with ESMTP id OAA23047; Fri, 9 Apr 1999 14:52:13 +0800 (WST) (envelope-from peter@spinner.netplex.com.au) Message-Id: <199904090652.OAA23047@spinner.netplex.com.au> X-Mailer: exmh version 2.0.2 2/24/98 To: Terry Lambert Cc: barney@databus.com (Barney Wolff), smp@FreeBSD.ORG, Tony Finch Subject: Re: concurrent select()s on listen socket broken under SMP In-reply-to: Your message of "Fri, 09 Apr 1999 02:03:33 GMT." <199904090203.TAA14310@usr04.primenet.com> Date: Fri, 09 Apr 1999 14:52:13 +0800 From: Peter Wemm Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Terry Lambert wrote: > > Seems to me you need to set the socket non-blocking, and then handle > > the EWOULDBLOCK on the fd that loses. I don't think this should be > > considered a kernel error. > > It's SMP specific. This is *NOT* a SMP specific problem, it's a generic problem in the socket API and implementation. You can trigger it on a UP kernel too, it just happens to be real easy under SMP. See rev 1.52 and 1.54 of uipc_socket.c and related files for a description of what's really going on. (the problem there is from a different angle, it deals with remote users being able to trigger the race remotely as a denial-of-service attack). The original poster didn't mention what version of FreeBSD was in use. 3.1-RELEASE has this bug... It's fixed in 4.0 and 3.1-STABLE (after 26-feb-99). Terry, rest assured, select() and poll() are quite SMP safe. The other serious problem is select collisions. I'd strongly advise using a mutex so that only one process is ever in a select() for waiting for connections on a socket. If two or more processes select() waiting for the listening socket to become readable, when it does - a select collision happens. ie: *EVERY SINGLE PROCESS* that was asleep in select gets woken up, and runs, and scans it's select tables, and 99% go back to sleep. If you have a couple of hundred processes asleep in select(), this hits like a bomb going off. You can get a spontanious load average jump from 0.2 to 100+ in an instant, depending on when loadav() runs. Select collisions are a well understood problem, and there is no cheap, easy, trivial fix that scales well. (Sure, making the selinfo struct have 5 pid's rather than 1 would work for <= 6 processes waiting, but the problem is still there for 6. you can't allocate memory for it due to the nature of the implementation without modifying the driver interface so that there is an "d_unselect()" handler or something - selinfo's get released long after the process has given up and gone onto something else. Changing the driver API _again_ isn't something we want to do yet...) Cheers, -Peter To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Apr 9 2:30:18 1999 Delivered-To: freebsd-smp@freebsd.org Received: from mail2.idgruppen.no (ns2.idgruppen.no [195.204.177.66]) by hub.freebsd.org (Postfix) with ESMTP id 0A0A115000 for ; Fri, 9 Apr 1999 02:30:08 -0700 (PDT) (envelope-from ragnar.nielsen@idcomnet.no) Received: from rnnt ([10.1.18.144]) by mail2.idgruppen.no (8.8.7/8.8.7) with SMTP id LAA11565 for ; Fri, 9 Apr 1999 11:28:05 +0200 From: "Ragnar Nielsen" To: Subject: P-II processor steppings Date: Fri, 9 Apr 1999 11:29:14 +0200 Message-ID: <000301be826b$6e936ea0$9012010a@rnnt.dhcp.osl.idgruppen.no> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0 Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V4.72.3110.3 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Last fall, I got myself an ASUS P2B-DS with a single P-II/450 processor, planning for a second CPU later on. Now I wonder: Do I have to get a processor of the same stepping as the original one, or is it OK to mix steppings in a SMP system as long as the other CPU parameters are identical? If so, I'll postpone the purchase for a little while, saving money as the prices continue to drop. If not, I'll have to grab a stepping 2 processor while they're still available! I would be most grateful for an educated answer. Best regards, Ragnar Nielsen ragnar.nielsen@idcomnet.no To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Apr 9 4:18:32 1999 Delivered-To: freebsd-smp@freebsd.org Received: from noc.demon.net (server.noc.demon.net [193.195.224.4]) by hub.freebsd.org (Postfix) with ESMTP id 3650F14FD0 for ; Fri, 9 Apr 1999 04:18:29 -0700 (PDT) (envelope-from fanf@demon.net) Received: by noc.demon.net; id MAA21357; Fri, 9 Apr 1999 12:16:28 +0100 (BST) Received: from fanf.noc.demon.net(195.11.55.83) by inside.noc.demon.net via smap (3.2) id xma021334; Fri, 9 Apr 99 12:16:22 +0100 Received: from fanf by fanf.noc.demon.net with local (Exim 1.73 #2) id 10VZGo-0006yj-00; Fri, 9 Apr 1999 12:16:22 +0100 To: smp@freebsd.org From: Tony Finch Subject: Re: concurrent select()s on listen socket broken under SMP Newsgroups: chiark.mail.freebsd.smp In-Reply-To: Organization: Deliberate Obfuscation To Amuse Tony References: <14093.5670.813002.917842@chiark.greenend.org.uk> Message-Id: Date: Fri, 9 Apr 1999 12:16:22 +0100 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Marc Slemko wrote: >On Thu, 8 Apr 1999, Tony Finch wrote: >> >> What breaks if I change the wakeup((caddr_t)&sb->sb_cc); in >> sowakeup() (line 319 of uipc_socket2.c) to a wakeup_one()? > >That could just break a whole lot of things I think. :-) >The basic reason behind this is (aside from the non-standard behaviour and >fairness issues) is that select() can notify a process that multiple >sockets are ready to be dealt with, while a process will probably only be >able to deal with one at a time. thttpd deals with all the ready descriptors between calls to select() Tony. -- f.a.n.finch dot@dotat.at fanf@demon.net To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Apr 9 4:24:17 1999 Delivered-To: freebsd-smp@freebsd.org Received: from noc.demon.net (server.noc.demon.net [193.195.224.4]) by hub.freebsd.org (Postfix) with ESMTP id 75CAB15148 for ; Fri, 9 Apr 1999 04:24:13 -0700 (PDT) (envelope-from fanf@demon.net) Received: by noc.demon.net; id MAA22359; Fri, 9 Apr 1999 12:22:11 +0100 (BST) Received: from fanf.noc.demon.net(195.11.55.83) by inside.noc.demon.net via smap (3.2) id xma022331; Fri, 9 Apr 99 12:22:01 +0100 Received: from fanf by fanf.noc.demon.net with local (Exim 1.73 #2) id 10VZMG-000700-00; Fri, 9 Apr 1999 12:22:00 +0100 To: smp@freebsd.org From: Tony Finch Subject: Re: concurrent select()s on listen socket broken under SMP In-Reply-To: <199904090652.OAA23047@spinner.netplex.com.au> References: <199904090203.TAA14310@usr04.primenet.com> Message-Id: Date: Fri, 9 Apr 1999 12:22:00 +0100 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Peter Wemm wrote: > >The original poster didn't mention what version of FreeBSD was in use. >3.1-RELEASE has this bug... It's fixed in 4.0 and 3.1-STABLE (after >26-feb-99). It's an up-to-the-minute -STABLE box. >The other serious problem is select collisions. I'd strongly advise using >a mutex so that only one process is ever in a select() for waiting for >connections on a socket. If two or more processes select() waiting for the >listening socket to become readable, when it does - a select collision >happens. ie: *EVERY SINGLE PROCESS* that was asleep in select gets woken >up, and runs, and scans it's select tables, and 99% go back to sleep. If >you have a couple of hundred processes asleep in select(), this hits like a >bomb going off. You can get a spontanious load average jump from 0.2 to >100+ in an instant, depending on when loadav() runs. We only have a few processes select()ing on the listen socket. The reason for doing this is to cope with certain situations when thttpd takes longer than it should between select()s -- generating directory listings in particular. I suppose we were playing fast-and-loose with this cheapo parallelism and were lucky not to lose sooner. >Select collisions are a well understood problem, and there is no cheap, >easy, trivial fix that scales well. Drat. Tony. -- f.a.n.finch dot@dotat.at fanf@demon.net To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Apr 9 17:41:50 1999 Delivered-To: freebsd-smp@freebsd.org Received: from smtp05.primenet.com (smtp05.primenet.com [206.165.6.135]) by hub.freebsd.org (Postfix) with ESMTP id 253DF14C2B for ; Fri, 9 Apr 1999 17:41:44 -0700 (PDT) (envelope-from tlambert@usr01.primenet.com) Received: (from daemon@localhost) by smtp05.primenet.com (8.9.1/8.9.1) id SAA66240; Fri, 9 Apr 1999 18:40:28 -0600 Received: from usr01.primenet.com(206.165.6.201) via SMTP by smtp05.primenet.com, id smtpdabb27a; Fri Apr 9 18:40:22 1999 Received: (from tlambert@localhost) by usr01.primenet.com (8.8.5/8.8.5) id RAA05457; Fri, 9 Apr 1999 17:39:19 -0700 (MST) From: Terry Lambert Message-Id: <199904100039.RAA05457@usr01.primenet.com> Subject: Re: concurrent select()s on listen socket broken under SMP To: peter@netplex.com.au (Peter Wemm) Date: Sat, 10 Apr 1999 00:39:19 +0000 (GMT) Cc: tlambert@primenet.com, barney@databus.com, smp@FreeBSD.ORG, dot@dotat.at In-Reply-To: <199904090652.OAA23047@spinner.netplex.com.au> from "Peter Wemm" at Apr 9, 99 02:52:13 pm X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > This is *NOT* a SMP specific problem, it's a generic problem in the socket > API and implementation. You can trigger it on a UP kernel too, it just > happens to be real easy under SMP. > > See rev 1.52 and 1.54 of uipc_socket.c and related files for a description > of what's really going on. (the problem there is from a different angle, > it deals with remote users being able to trigger the race remotely > as a denial-of-service attack). > > The original poster didn't mention what version of FreeBSD was in use. > 3.1-RELEASE has this bug... It's fixed in 4.0 and 3.1-STABLE (after > 26-feb-99). He said he was running a curtrent version of 3.1-STABLE. > Terry, rest assured, select() and poll() are quite SMP safe. > > The other serious problem is select collisions. I'd strongly advise using > a mutex so that only one process is ever in a select() for waiting for > connections on a socket. If two or more processes select() waiting for the > listening socket to become readable, when it does - a select collision > happens. ie: *EVERY SINGLE PROCESS* that was asleep in select gets woken > up, and runs, and scans it's select tables, and 99% go back to sleep. Yes. This is the classic "thundering herd" problem for which wakeup_one() (or wake_one()) was invented. The real problem is that for some type of events, you want all of the processes waiting on the event to go, and on other types of events, you want only one process to go. I think for a socket on which a listen has been called, it's pretty pbvious that you can only service a single accept at a time, and so based on the character of the object selected, wakeup_one() is the correct call. I think that select collisions should be outlawed, unless the descriptor in question explicitly permits them, but I'd be OK with having to change the current behaviour by via fcntl on a per fd basis. > If you have a couple of hundred processes asleep in select(), this > hits like a bomb going off. You can get a spontanious load average > jump from 0.2 to 100+ in an instant, depending on when loadav() runs. And this is a design error, which should be corrected, whether you argue that it's a scheduler problem or a wakeup problem. Dynix and SVR4.2 both corrected this problem a while ago, using a wake_one() approach. > Select collisions are a well understood problem, and there is no cheap, > easy, trivial fix that scales well. (Sure, making the selinfo struct have > 5 pid's rather than 1 would work for <= 6 processes waiting, but the > problem is still there for 6. you can't allocate memory for it due to > the nature of the implementation without modifying the driver interface > so that there is an "d_unselect()" handler or something - selinfo's get > released long after the process has given up and gone onto something > else. Changing the driver API _again_ isn't something we want to do > yet...) I think what you want is a LIFO queue for people selecting on an fd, and perhaps a per fd fcntl'able flag that can turn that into a FIFO policy instead for resources for which there are interactive waits (clearly, for a server, LIFO is the correct ordering; at Novell, my suggestion that a LIFO service order be used for the NCP streams mux resulted in a 35% increase in throughput as a result of reduced paging load, making the UNIX soloution faster than Native NetWare). One possibile soloution, which would do dick for the SMP implications, would be to add a "time_in_select", and then have wakeup_one() only wake the one who called last, using a linear iteration. This is basically what it does now (why the hell does it wakeup more than one *just* because the first one is swapped out?!?). Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Apr 9 17:47:54 1999 Delivered-To: freebsd-smp@freebsd.org Received: from smtp03.primenet.com (smtp03.primenet.com [206.165.6.133]) by hub.freebsd.org (Postfix) with ESMTP id 22F6714DD5 for ; Fri, 9 Apr 1999 17:47:48 -0700 (PDT) (envelope-from tlambert@usr01.primenet.com) Received: (from daemon@localhost) by smtp03.primenet.com (8.8.8/8.8.8) id RAA20139; Fri, 9 Apr 1999 17:45:36 -0700 (MST) Received: from usr01.primenet.com(206.165.6.201) via SMTP by smtp03.primenet.com, id smtpd020117; Fri Apr 9 17:45:27 1999 Received: (from tlambert@localhost) by usr01.primenet.com (8.8.5/8.8.5) id RAA05886; Fri, 9 Apr 1999 17:45:27 -0700 (MST) From: Terry Lambert Message-Id: <199904100045.RAA05886@usr01.primenet.com> Subject: Re: concurrent select()s on listen socket broken under SMP To: dot@dotat.at (Tony Finch) Date: Sat, 10 Apr 1999 00:45:26 +0000 (GMT) Cc: smp@FreeBSD.ORG In-Reply-To: from "Tony Finch" at Apr 9, 99 12:16:22 pm X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > >The basic reason behind this is (aside from the non-standard behaviour and > >fairness issues) is that select() can notify a process that multiple > >sockets are ready to be dealt with, while a process will probably only be > >able to deal with one at a time. > > thttpd deals with all the ready descriptors between calls to select() I think it is bogus to put the code to deal with this problem into each and every user space application, just like I think the Apache hacks to alarm out of shutdown(2) and call close(2) and expect the close(2) (which implies a shutdown(2)) to break things out of the FIN_WAIT_2 state to deal with what is an obvious design error (one packet out, two packets in response, just like Window SMB and NetWare NCP file locks). It's on the order of bogosity of having a link manager that can't base link up/down decisons on who's generating the traffic, or having to bind sockets to IP addresses in order to bind them to an interface, such that every time the IP address changes, you have to restart your daemons. Bletch. Poor design is poor design. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Apr 9 17:52:30 1999 Delivered-To: freebsd-smp@freebsd.org Received: from smtp01.primenet.com (smtp01.primenet.com [206.165.6.131]) by hub.freebsd.org (Postfix) with ESMTP id DC46D14FD2 for ; Fri, 9 Apr 1999 17:52:23 -0700 (PDT) (envelope-from tlambert@usr01.primenet.com) Received: (from daemon@localhost) by smtp01.primenet.com (8.8.8/8.8.8) id RAA22307; Fri, 9 Apr 1999 17:50:11 -0700 (MST) Received: from usr01.primenet.com(206.165.6.201) via SMTP by smtp01.primenet.com, id smtpd022296; Fri Apr 9 17:50:06 1999 Received: (from tlambert@localhost) by usr01.primenet.com (8.8.5/8.8.5) id RAA06066; Fri, 9 Apr 1999 17:50:04 -0700 (MST) From: Terry Lambert Message-Id: <199904100050.RAA06066@usr01.primenet.com> Subject: Re: concurrent select()s on listen socket broken under SMP To: barney@databus.com (Barney Wolff) Date: Sat, 10 Apr 1999 00:50:04 +0000 (GMT) Cc: smp@FreeBSD.ORG In-Reply-To: <370d8a8b0.4bf5@databus.databus.com> from "Barney Wolff" at Apr 9, 99 00:35:00 am X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Different observed behavior UP vs MP does not constitute kernel error, If it's the same kernel, it does. If it doesn't, then you aren't talking about symmetry. > unless the behavior is specifically defined. In this case, select() > on the same fd by two processes has no defined behavior, that I'm > aware of. (Not that I've done a lot of searching on this topic.) According to Peter, it does. > Nor has there ever been any guarantee that a condition > discovered by select() will remain true until a subsequent system call. > It *usually* does, but that's all. Yeah, this is as bullshit as signals being persistent conditions instead of events. > On a very pragmatic basis, if you really can't ever block, set nonblock > and be prepared for the occasional cases when the hint given by select > doesn't pan out. As someone who first experienced MP bugs in an > application over 20 years ago, let me just say that it is a losing > cause to count on *undocumented* system behavior staying the same > when going from UP to MP. Then again, it's unwise to count on > undocumented system behavior, period. This is a way to work around the behaviour in your applications. But if you define a well behaved application as something that has the same 30 lines of code that's in every other well behaved application, then you are starting to get into the realm of something that should be done in a central location, such as the kernel and/or the libc function call wrapper for communicating with the kernel. Unfortunately, there aren't any very good anonymous semaphore mechanisms in FreeBSD , where a semaphore can be given to everyone who is a subprocess of a process group leader, and invisibly enforced by libc. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Apr 10 5:41:23 1999 Delivered-To: freebsd-smp@freebsd.org Received: from spinner.netplex.com.au (spinner.netplex.com.au [202.12.86.3]) by hub.freebsd.org (Postfix) with ESMTP id 87B7914E9C for ; Sat, 10 Apr 1999 05:41:14 -0700 (PDT) (envelope-from peter@spinner.netplex.com.au) Received: from spinner.netplex.com.au (localhost [127.0.0.1]) by spinner.netplex.com.au (Postfix) with ESMTP id 44DD21F4D; Sat, 10 Apr 1999 20:38:59 +0800 (WST) (envelope-from peter@spinner.netplex.com.au) X-Mailer: exmh version 2.0.2 2/24/98 To: Terry Lambert Cc: barney@databus.com, smp@FreeBSD.ORG, dot@dotat.at Subject: Re: concurrent select()s on listen socket broken under SMP In-reply-to: Your message of "Sat, 10 Apr 1999 00:39:19 GMT." <199904100039.RAA05457@usr01.primenet.com> Date: Sat, 10 Apr 1999 20:38:58 +0800 From: Peter Wemm Message-Id: <19990410123901.44DD21F4D@spinner.netplex.com.au> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Terry Lambert wrote: > > This is *NOT* a SMP specific problem, it's a generic problem in the socket > > API and implementation. You can trigger it on a UP kernel too, it just > > happens to be real easy under SMP. > > > > See rev 1.52 and 1.54 of uipc_socket.c and related files for a description > > of what's really going on. (the problem there is from a different angle, > > it deals with remote users being able to trigger the race remotely > > as a denial-of-service attack). > > > > The original poster didn't mention what version of FreeBSD was in use. > > 3.1-RELEASE has this bug... It's fixed in 4.0 and 3.1-STABLE (after > > 26-feb-99). > > He said he was running a curtrent version of 3.1-STABLE. Either way, it doesn't matter. As I said, look at the description of the changes for details of what is going on. The problem in the patch was a *specific case* where it was possible to provoke this race with a single process listener. Having multiple processes in a select/accept role listening role is always vulnerable.. select() and accept() are different syscalls, and there is no guarantee that you're going to get a continuous execution run from the select being true and the socket still being ready for select. Eg: Process 1 Process 2 [sleeping in select] [sleeping in select] [incoming connection - socket becomes readable, both become runnable] Runs.. does accept() [connection taken - another accept will hang] Finally runs, does accept(), hangs.. At this point, Process 1 has taken the connection and process 2 is locked in accept(). The only way out of this is to set the listening socket non-blocking so that accept() will abort on a false start. This race is inherent in the design of select() and accept(). It's nothing to do with SMP etc. > > Terry, rest assured, select() and poll() are quite SMP safe. > > > > The other serious problem is select collisions. I'd strongly advise using > > a mutex so that only one process is ever in a select() for waiting for > > connections on a socket. If two or more processes select() waiting for the > > listening socket to become readable, when it does - a select collision > > happens. ie: *EVERY SINGLE PROCESS* that was asleep in select gets woken > > up, and runs, and scans it's select tables, and 99% go back to sleep. > > Yes. This is the classic "thundering herd" problem for which wakeup_one() > (or wake_one()) was invented. wakeup_one() cannot be used for select(). Look at the example above. What if Process 1 got selected for the wakeup, and instead it serviced something else of a higher priority? Suppose it hangs? The connection will be in limbo with nothing to service it. Suppose it (process 1) core dumps? How is process 2 going to get woken up instead? This is not quite 'the' thundering herd BTW. If you have 3 processes selecting on an event and it happens, all three (by design of select() api) need to be woken. The bug is that not just those three get worken but every select process on the system, including those that don't give a damn about the given event. > The real problem is that for some type of events, you want all > of the processes waiting on the event to go, and on other types of > events, you want only one process to go. In select() and poll(), you *cannot* "pick one". All the processes listening on the event *must* be woken - because the processes themselves scan the vectors and decide what they will do and when. A process can't "send back" the wakeup if it's going to do something else instead. > I think for a socket on which a listen has been called, it's pretty > pbvious that you can only service a single accept at a time, and > so based on the character of the object selected, wakeup_one() is > the correct call. Even then, you can't do this safely. Sure, only one can accept the connection, but if you pick one, how will you know if it actually will? Remember, it's doing a select() so that it can look at *multiple* things at once. > I think that select collisions should be outlawed, unless the > descriptor in question explicitly permits them, but I'd be OK > with having to change the current behaviour by via fcntl on a > per fd basis. This won't work for the reasons above.. You'll get missed events all over the place in a scenario where this is actually activated.. Remember, select collisions are pretty rare for 99% of the software out there. > > If you have a couple of hundred processes asleep in select(), this > > hits like a bomb going off. You can get a spontanious load average > > jump from 0.2 to 100+ in an instant, depending on when loadav() runs. > > And this is a design error, which should be corrected, whether you > argue that it's a scheduler problem or a wakeup problem. Dynix and > SVR4.2 both corrected this problem a while ago, using a wake_one() > approach. Yes, it is a problem and should be fixed, but wakeup_one() isn't even remotely the answer.. > > Select collisions are a well understood problem, and there is no cheap, > > easy, trivial fix that scales well. (Sure, making the selinfo struct have > > 5 pid's rather than 1 would work for <= 6 processes waiting, but the > > problem is still there for 6. you can't allocate memory for it due to > > the nature of the implementation without modifying the driver interface > > so that there is an "d_unselect()" handler or something - selinfo's get > > released long after the process has given up and gone onto something > > else. Changing the driver API _again_ isn't something we want to do > > yet...) > > I think what you want is a LIFO queue for people selecting on an fd, > and perhaps a per fd fcntl'able flag that can turn that into a FIFO > policy instead for resources for which there are interactive waits > (clearly, for a server, LIFO is the correct ordering; at Novell, my > suggestion that a LIFO service order be used for the NCP streams mux > resulted in a 35% increase in throughput as a result of reduced > paging load, making the UNIX soloution faster than Native NetWare). A list/queue/whatever would be one way, but this requires a driver API redesign in a big way. At the moment, the drivers manage the select event wakeups. A process calls select/poll(), and this eventually translates into a selscan() call which ends up calling the driver d_select() (now d_poll) entry point. If the event is not presently active, selrecord() is used to log the event in the *driver*'s private selinfo struct that it provides for that event. At some later time, the event "happens", the driver does a selwakeup() on the selinfo. The driver has *no idea* if the process is asleep and waiting for the event or not or even if it's died or otherwise, because there is no call from the select subsystem to tell the driver that it's no longer interested. As a result, the drivers cannot maintain a LIFO or anything like that since they never get a chance to clean out the list of things that are no longer interested in the event. However, if we're prepared to get a little sloppy and do lazy reclaims of buffers or something like that, then selrecord() could prune the table if needed, and selwakeup could do it too. We could probably fudge it a little, so that instead of selinfo holding a short pid, it could hold a union and an array of 4 or 8 pid's which transmutes into a list header or something if all 4 or 8 slots get used up. > One possibile soloution, which would do dick for the SMP implications, > would be to add a "time_in_select", and then have wakeup_one() only > wake the one who called last, using a linear iteration. This is > basically what it does now (why the hell does it wakeup more than > one *just* because the first one is swapped out?!?). If only it were that simple.. > Terry Lambert > terry@lambert.org Cheers, -Peter To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Apr 10 16:19:45 1999 Delivered-To: freebsd-smp@freebsd.org Received: from smtp03.primenet.com (smtp03.primenet.com [206.165.6.133]) by hub.freebsd.org (Postfix) with ESMTP id DA8F915162 for ; Sat, 10 Apr 1999 16:19:42 -0700 (PDT) (envelope-from tlambert@usr08.primenet.com) Received: (from daemon@localhost) by smtp03.primenet.com (8.8.8/8.8.8) id QAA01164; Sat, 10 Apr 1999 16:17:25 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp03.primenet.com, id smtpd001150; Sat Apr 10 16:17:17 1999 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id QAA25435; Sat, 10 Apr 1999 16:17:16 -0700 (MST) From: Terry Lambert Message-Id: <199904102317.QAA25435@usr08.primenet.com> Subject: Re: concurrent select()s on listen socket broken under SMP To: peter@spinner.netplex.com.au (Peter Wemm) Date: Sat, 10 Apr 1999 23:17:16 +0000 (GMT) Cc: tlambert@primenet.com, barney@databus.com, smp@FreeBSD.ORG, dot@dotat.at In-Reply-To: <19990410123901.44DD21F4D@spinner.netplex.com.au> from "Peter Wemm" at Apr 10, 99 08:38:58 pm X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I'll probably respond in detail to other issues later; right now, I just want to make a quick point... > A list/queue/whatever would be one way, but this requires a driver API > redesign in a big way. > > At the moment, the drivers manage the select event wakeups. A process > calls select/poll(), and this eventually translates into a selscan() call > which ends up calling the driver d_select() (now d_poll) entry point. If > the event is not presently active, selrecord() is used to log the event in > the *driver*'s private selinfo struct that it provides for that event. I think that we should look at UDI as an opportunity for redesign in this area: http://www.sco.com/udi/ Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Apr 11 22: 1:41 1999 Delivered-To: freebsd-smp@freebsd.org Received: from dfw-ix10.ix.netcom.com (dfw-ix10.ix.netcom.com [206.214.98.10]) by hub.freebsd.org (Postfix) with ESMTP id ADB0914E21 for ; Sun, 11 Apr 1999 22:01:38 -0700 (PDT) (envelope-from MuleAxe@ix.netcom.com) Received: (from smap@localhost) by dfw-ix10.ix.netcom.com (8.8.4/8.8.4) id XAA15385 for ; Sun, 11 Apr 1999 23:59:18 -0500 (CDT) Received: from ali-ca55-32.ix.netcom.com(209.110.237.160) by dfw-ix10.ix.netcom.com via smap (V1.3) id rma015366; Sun Apr 11 23:58:57 1999 Message-ID: <37117D8D.AB186B23@ix.netcom.com> Date: Sun, 11 Apr 1999 21:58:53 -0700 From: Allen Wong X-Mailer: Mozilla 4.07 [en] (X11; I; FreeBSD 3.0-RELEASE i386) MIME-Version: 1.0 To: freebsd-smp@freebsd.org Subject: SMP kernel Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Dear Sir: Just dropping you a note to let you know that I have successfully compiled SMP support on a dual 90MHz Pentium Micronics M54Pe PCI/EISA motherboard. Here are the vital statistics. I'm not sure about the "NINTR" setting in the kernel configuration file, but I don't seem to be having any problems with the new kernel. =============================================================================== MPTable, version 2.0.15 ------------------------------------------------------------------------------- MP Floating Pointer Structure: location: EBDA physical address: 0x0009fc30 signature: '_MP_' length: 16 bytes version: 1.1 checksum: 0x5a mode: Virtual Wire ------------------------------------------------------------------------------- MP Config Table Header: physical address: 0x0009fc44 signature: 'PCMP' base table length: 260 version: 1.1 checksum: 0x29 OEM ID: 'INTEL ' Product ID: '430 NX EISA ' OEM table pointer: 0x00000000 OEM table size: 0 entry count: 24 local APIC address: 0xfee00000 extended table length: 0 extended table checksum: 0 ------------------------------------------------------------------------------- MP Config Base Table Entries: -- Processors: APIC ID Version State Family Model Step Flags 0 0x10 BSP, usable 5 2 1 0x07bf 1 0x10 AP, usable 5 2 4 0x07bf -- Bus: Bus ID Type 0 ISA 1 EISA 2 PCI -- I/O APICs: APIC ID Version State Address 2 0x11 usable 0xfec00000 -- I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN# ExtINT active-hi edge 0 0 2 0 INT conforms conforms 0 1 2 1 INT conforms conforms 0 0 2 2 INT conforms conforms 0 3 2 3 INT conforms conforms 0 4 2 4 INT conforms conforms 0 5 2 5 INT conforms conforms 0 6 2 6 INT conforms conforms 0 7 2 7 INT conforms conforms 0 8 2 8 INT conforms conforms 0 9 2 9 INT conforms conforms 0 10 2 10 INT conforms conforms 0 11 2 11 INT conforms conforms 0 12 2 12 INT conforms conforms 0 13 2 13 INT conforms conforms 0 14 2 14 INT conforms conforms 0 15 2 15 -- Local Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN# ExtINT active-hi edge 0 0 255 0 NMI active-hi edge 0 0 255 1 ------------------------------------------------------------------------------- # SMP kernel config file options: # Required: options SMP # Symmetric MultiProcessor Kernel options APIC_IO # Symmetric (APIC) I/O # Optional (built-in defaults will work in most cases): #options NCPU=2 # number of CPUs #options NBUS=3 # number of busses #options NAPIC=1 # number of IO APICs #options NINTR=24 # number of INTs =============================================================================== Copyright (c) 1992-1998 FreeBSD Inc. "dmesg" output: Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All rights reserved. FreeBSD 3.0-RELEASE #5: Tue Mar 30 14:35:45 PST 1999 root@FreeBSD1.toxicwaste.com:/usr/src/sys/compile/AMADEUS Timecounter "i8254" frequency 1193182 Hz cost 4334 ns CPU: Pentium/P54C (586-class CPU) Origin = "GenuineIntel" Id = 0x521 Stepping=1 Features=0x7bf real memory = 25165824 (24576K bytes) avail memory = 22368256 (21844K bytes) Programming 16 pins in IOAPIC #0 FreeBSD/SMP: Multiprocessor motherboard cpu0 (BSP): apic id: 0, version: 0x00030010, at 0xfee00000 cpu1 (AP): apic id: 1, version: 0x00030010, at 0xfee00000 io0 (APIC): apic id: 2, version: 0x000f0011, at 0xfec00000 eisa0: Probing for devices on the EISA bus Probing for devices on PCI bus 0: chip0: rev 0x11 on pci0.0. 0 chip1: rev 0x03 on pci0.2.0 xl0: <3Com 3c905B Fast Etherlink XL 10/100BaseTX> rev 0x30 int a irq 10 on pci0. 13.0 xl0: Ethernet address: 00:10:5a:1c:2c:50 xl0: autoneg complete, link status good (half-duplex, 10Mbps) vga0: rev 0xd3 int a irq 9 on pci0.14.0 adv0: rev 0x03 int a irq 11 on pci0.15. 0 adv0: AdvanSys Ultra SCSI Host Adapter, SCSI ID 7, queue depth 240 Probing for devices on the ISA bus: sc0 at 0x60-0x6f irq 1 on motherboard sc0: VGA color <16 virtual consoles, flags=0x0> sio0 at 0x3f8-0x3ff irq 4 flags 0x10 on isa sio0: type 16550A sio1 at 0x2f8-0x2ff irq 3 on isa sio1: type 16550A fdc0 at 0x3f0-0x3f7 irq 6 drq 2 on isa fdc0: FIFO enabled, 8 bytes threshold fd0: 1.44MB 3.5in adv0 not found at 0x330 npx0 on motherboard npx0: INT 16 interface Intel Pentium F00F detected, installing workaround APIC_IO: Testing 8254 interrupt delivery APIC_IO: routing 8254 via pin 2 Waiting 5 seconds for SCSI devices to settle SMP: AP CPU #1 Launched! da1 at adv0 bus 0 target 6 lun 0 da1: Fixed Direct Access SCSI2 device da1: 10.0MB/s transfers (10.0MHz, offset 15) da1: 1025MB (2099880 512 byte sectors: 255H 63S/T 130C) da0 at adv0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI2 device da0: 20.0MB/s transfers (20.0MHz, offset 15) da0: 3090MB (6328861 512 byte sectors: 255H 63S/T 393C) # Create a SMP capable kernel (mandatory options): options SMP # Symmetric MultiProcessor Kernel options APIC_IO # Symmetric (APIC) I/O # Optional, these are the defaults: options NCPU=2 # number of CPUs options NBUS=3 # number of busses options NAPIC=1 # number of IO APICs options NINTR=16 # number of INTs Allen -- NCURSES! Foiled again! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Apr 12 14:51:51 1999 Delivered-To: freebsd-smp@freebsd.org Received: from netscape.com (h-205-217-237-47.netscape.com [205.217.237.47]) by hub.freebsd.org (Postfix) with ESMTP id 15AAE155E5 for ; Mon, 12 Apr 1999 14:51:45 -0700 (PDT) (envelope-from dlb@netscape.com) Received: from dredd.mcom.com (dredd.mcom.com [205.217.237.54]) by netscape.com (8.8.5/8.8.5) with ESMTP id OAA16375 for ; Mon, 12 Apr 1999 14:49:26 -0700 (PDT) Received: from netscape.com ([208.12.60.117]) by dredd.mcom.com (Netscape Messaging Server 4.01) with ESMTP id FA3JAE01.R1D for ; Mon, 12 Apr 1999 14:49:26 -0700 Message-ID: <371269EF.95AC0CD7@netscape.com> Date: Mon, 12 Apr 1999 14:47:28 -0700 From: Debbie Bridygham Organization: Netscape Communications Corporation X-Mailer: Mozilla 4.51C-NSCP [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en-US,en-GB MIME-Version: 1.0 To: freebsd-smp@freebsd.org Subject: FreeBSD 3.0 SMP kernel hang with IWill DBL100 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: base64 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org SGksDQoNCiAgICBJJ3ZlIGJlZW4gdHJ5aW5nIHRvIHVzZSB0aGUgU01QIGZlYXR1cmVzIHdp dGggYW4gSVdpbGwgREJMMTAwIE1CDQooYmlvcyBkYXRlZCBKYW4gMjEgMTk5OSkuICBVbmZv cnR1bmF0ZWx5IHRoZSBrZXJuZWwgaGFuZ3MgcmlnaHQgYWZ0ZXINCnByb2R1Y2luZyB0aGUg J1Byb2dyYW1taW5nIDI0IHBpbiAuLi4nIG1lc3NhZ2UuICBtcHRhYmxlIHJlcG9ydHMgTVBT IGFzDQoxLjEsIGV2ZW4gdGhvdWdoIEkgaGF2ZSAxLjQgc2V0IGluIHRoZSBDTU9TLiAgVGhl IE1CIGhhcyB0d28gUGVybnRpdW0NCklJcyBydW5uaW5nIGF0IDM1ME1Iei4gIEFueSBzdWdn ZXN0aW9ucz8gIEkgZm91bmQgYW4gZWFybGllciBtZXNzYWdlDQphYm91dCBvdGhlciBwcm9i bGVtcyB3aXRoIGFuIEl3aWxsIERCTDEwMCBhbmQgYSByZXBvcnRlZCBwYXRjaCBvbiB0aGUN Cm1haWxpbmcgbGlzdCBhcmNoaXZlcyBkYXRlZCBpbiBPY3RvYmVyIG9mIDE5OTguICBTaG91 bGQgSSB1cGdyYWRlIHRvIDMuMQ0Kb3Igd2FpdCB1bnRpbCAzLjI/DQo= To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Apr 12 17:28:17 1999 Delivered-To: freebsd-smp@freebsd.org Received: from fire.starkreality.com (fire.starkreality.com [208.24.48.226]) by hub.freebsd.org (Postfix) with ESMTP id 2EE4E153FA; Mon, 12 Apr 1999 17:28:13 -0700 (PDT) (envelope-from caesar@starkreality.com) Received: from grail (grail.starkreality.com [208.24.48.235]) by fire.starkreality.com (8.9.3/8.9.2) with SMTP id TAA03162; Mon, 12 Apr 1999 19:25:52 -0500 (CDT) Message-Id: <4.1.19990412191627.009b3100@imap.colltech.com> X-Sender: caesar@mail.starkreality.com (Unverified) X-Mailer: QUALCOMM Windows Eudora Pro Version 4.1 Date: Mon, 12 Apr 1999 19:25:31 -0500 To: freebsd-smp@freebsd.org From: "William S. Duncanson" Subject: SMP broken in -CURRENT? Cc: freebsd-current@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I haven't been able to get a working SMP kernel out of -CURRENT recently. I don't know exactly when it broke, because I usually rebuild on a weekly basis. The kernel hangs after: APIC_IO: Testing 8254 interrupt delivery and doesn't ever come back (panic or otherwise). The one thing that I noticed is that on the older kernels, CPU#1 is launched after the APIC_IO Testing and Routing. On the newer kernels, CPU#1 is launched far earlier. Anybody have any ideas? On a side note, the pn0 driver appears to be broken (uniprocessor kernels will boot, the pn0 device shows up, and is ifconfig'd, but there's no link light on it. It works fine with older (04/04/99) kernels). William S. Duncanson caesar@starkreality.com The driving force behind the NC is the belief that the companies who brought us things like Unix, relational databases, and Windows can make an appliance that is inexpensive and easy to use if they choose to do that. -- Scott Adams To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Apr 12 18: 4:40 1999 Delivered-To: freebsd-smp@freebsd.org Received: from picnic.mat.net (picnic.mat.net [206.246.122.133]) by hub.freebsd.org (Postfix) with ESMTP id 584F814CFD for ; Mon, 12 Apr 1999 18:04:34 -0700 (PDT) (envelope-from chuckr@mat.net) Received: from localhost (chuckr@localhost) by picnic.mat.net (8.9.3/8.8.5) with ESMTP id VAA64079; Mon, 12 Apr 1999 21:00:05 -0400 (EDT) Date: Mon, 12 Apr 1999 21:00:04 -0400 (EDT) From: Chuck Robey To: "William S. Duncanson" Cc: freebsd-smp@FreeBSD.ORG Subject: Re: SMP broken in -CURRENT? In-Reply-To: <4.1.19990412191627.009b3100@imap.colltech.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Mon, 12 Apr 1999, William S. Duncanson wrote: > I haven't been able to get a working SMP kernel out of -CURRENT recently. > I don't know exactly when it broke, because I usually rebuild on a weekly > basis. The kernel hangs after: > APIC_IO: Testing 8254 interrupt delivery > and doesn't ever come back (panic or otherwise). > > The one thing that I noticed is that on the older kernels, CPU#1 is > launched after the APIC_IO Testing and Routing. On the newer kernels, > CPU#1 is launched far earlier. > > Anybody have any ideas? Tell us how recent ... I built my last kernel on last Saturday, and it works fine (freshly cvsupped sources). If yours is not more recent than that, it might be your config or your hardware. I don't crosspost, so I restricted this one to -smp. (Now that I've said that, I'll probably crosspost accidentally on my next reply ... Mr. Murphy ALWAYS gets the last laugh). > > On a side note, the pn0 driver appears to be broken (uniprocessor kernels > will boot, the pn0 device shows up, and is ifconfig'd, but there's no link > light on it. It works fine with older (04/04/99) kernels). No comment, I don't use the pn device, didn't even know what it was (do now). ----------------------------+----------------------------------------------- Chuck Robey | Interests include any kind of voice or data chuckr@picnic.mat.net | communications topic, C programming, and Unix. 213 Lakeside Drive Apt T-1 | Greenbelt, MD 20770 | I run picnic (FreeBSD-current) (301) 220-2114 | and jaunt (Solaris7). ----------------------------+----------------------------------------------- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Apr 12 18:26:45 1999 Delivered-To: freebsd-smp@freebsd.org Received: from cheops.anu.edu.au (cheops.anu.edu.au [150.203.149.24]) by hub.freebsd.org (Postfix) with ESMTP id 0A32C14D58 for ; Mon, 12 Apr 1999 18:26:40 -0700 (PDT) (envelope-from bryan@cheops.anu.edu.au) Received: (from bryan@localhost) by cheops.anu.edu.au (8.9.1/8.9.1) id LAA24023 for freebsd-smp@freebsd.org; Tue, 13 Apr 1999 11:24:19 +1000 (EST) From: bryan collins Message-Id: <199904130124.LAA24023@cheops.anu.edu.au> Subject: 3.1 SMP broke, 3.0 SMP works To: freebsd-smp@freebsd.org Date: Tue, 13 Apr 1999 11:24:18 +1000 (EST) X-Mailer: ELM [version 2.4ME+ PL38 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi, I've been having major problems with a new Dual PII server built about 2 weeks ago. We loaded 3.1R with an SMP kernel, and get reboots about out of kvm and sometimes just reboot without a panic message or dump. We tried all sorts of things, even a patch that I found mentioned about increasing the kernel mem size, reducing MAXUSERS, NMBCLUSTERS etc. No luck. We eventually gave up on 3.1R and found a 3.0R dist on an ftp sever, loaded it up with same SMP params, and so far its been up almost 2 days. More than twice aslong as the uptimes with 3.1R So it seems somethings changed from 3.0 to 3.1 thats causes us this drama. Running the same setup with 3.1R NON-SMP kernel, lasted just over 24 hours. The machine runs squid and mail, so gets a bit of traffic. Anyone else gone to 3.1R SMP and seen hassles? Cheers Bry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Apr 12 18:42:31 1999 Delivered-To: freebsd-smp@freebsd.org Received: from fire.starkreality.com (fire.starkreality.com [208.24.48.226]) by hub.freebsd.org (Postfix) with ESMTP id B5ED114DA0 for ; Mon, 12 Apr 1999 18:42:22 -0700 (PDT) (envelope-from caesar@starkreality.com) Received: from grail (grail.starkreality.com [208.24.48.235]) by fire.starkreality.com (8.9.3/8.9.2) with SMTP id UAA10482; Mon, 12 Apr 1999 20:39:22 -0500 (CDT) Message-Id: <4.1.19990412202347.009bf790@imap.colltech.com> X-Sender: caesar@mail.starkreality.com (Unverified) X-Mailer: QUALCOMM Windows Eudora Pro Version 4.1 Date: Mon, 12 Apr 1999 20:38:35 -0500 To: Chuck Robey From: "William S. Duncanson" Subject: Re: SMP broken in -CURRENT? Cc: freebsd-smp@FreeBSD.ORG In-Reply-To: References: <4.1.19990412191627.009b3100@imap.colltech.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org At 09:00 PM 4/12/99 -0400, Chuck Robey wrote: >On Mon, 12 Apr 1999, William S. Duncanson wrote: > >> I haven't been able to get a working SMP kernel out of -CURRENT recently. >> I don't know exactly when it broke, because I usually rebuild on a weekly >> basis. The kernel hangs after: >> APIC_IO: Testing 8254 interrupt delivery >> and doesn't ever come back (panic or otherwise). >> >> The one thing that I noticed is that on the older kernels, CPU#1 is >> launched after the APIC_IO Testing and Routing. On the newer kernels, >> CPU#1 is launched far earlier. >> >> Anybody have any ideas? > >Tell us how recent ... I built my last kernel on last Saturday, and it >works fine (freshly cvsupped sources). If yours is not more recent than >that, it might be your config or your hardware. > I tried late Saturday (4/10), Sunday (4/11) and today (4/12). The working kernel that I have is from 4/4. I'm using the same config as from the working kernel, and there doesn't appear to have been any relevant changes in LINT. I'm going to try blowing away my source tree completely and re-cvsupping, but... Does your CPU#1 launch right after the kernel loads, or does it wait until the "normal" time? William S. Duncanson caesar@starkreality.com The driving force behind the NC is the belief that the companies who brought us things like Unix, relational databases, and Windows can make an appliance that is inexpensive and easy to use if they choose to do that. -- Scott Adams To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Apr 12 19:16:24 1999 Delivered-To: freebsd-smp@freebsd.org Received: from midten.fast.no (midten.fast.no [195.139.251.11]) by hub.freebsd.org (Postfix) with ESMTP id 4E23A14CEC; Mon, 12 Apr 1999 19:16:20 -0700 (PDT) (envelope-from tegge@fast.no) Received: from fast.no (IDENT:tegge@midten.fast.no [195.139.251.11]) by midten.fast.no (8.9.1/8.9.1) with ESMTP id EAA29557; Tue, 13 Apr 1999 04:13:54 +0200 (CEST) Message-Id: <199904130213.EAA29557@midten.fast.no> To: caesar@starkreality.com Cc: freebsd-smp@FreeBSD.ORG, freebsd-current@FreeBSD.ORG Subject: Re: SMP broken in -CURRENT? From: Tor.Egge@fast.no In-Reply-To: Your message of "Mon, 12 Apr 1999 19:25:31 -0500" References: <4.1.19990412191627.009b3100@imap.colltech.com> X-Mailer: Mew version 1.70 on Emacs 19.34.1 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Tue, 13 Apr 1999 04:13:53 +0200 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > I haven't been able to get a working SMP kernel out of -CURRENT recently. > I don't know exactly when it broke, because I usually rebuild on a weekly > basis. The kernel hangs after: > APIC_IO: Testing 8254 interrupt delivery > and doesn't ever come back (panic or otherwise). > > The one thing that I noticed is that on the older kernels, CPU#1 is > launched after the APIC_IO Testing and Routing. On the newer kernels, > CPU#1 is launched far earlier. > > Anybody have any ideas? You might want to try this patch, which disables the early start of CPU#1. Index: mp_machdep.c =================================================================== RCS file: /home/ncvs/src/sys/i386/i386/mp_machdep.c,v retrieving revision 1.96 diff -u -r1.96 mp_machdep.c --- mp_machdep.c 1999/04/11 00:43:43 1.96 +++ mp_machdep.c 1999/04/13 02:08:54 @@ -1930,9 +1930,11 @@ for (i = 0; i < mp_ncpus; i++) { bcopy( (int *) PTD + KPTDI, (int *) IdlePTDS[i] + KPTDI, NKPDE * sizeof (int)); } +#if 0 wait_ap(1000000); if (smp_started == 0) printf("WARNING: Failed to start all APs\n"); +#endif /* number of APs actually started */ return mp_ncpus - 1; To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Apr 12 19:17:59 1999 Delivered-To: freebsd-smp@freebsd.org Received: from picnic.mat.net (picnic.mat.net [206.246.122.133]) by hub.freebsd.org (Postfix) with ESMTP id 4E7FB14FDB for ; Mon, 12 Apr 1999 19:17:53 -0700 (PDT) (envelope-from chuckr@mat.net) Received: from localhost (chuckr@localhost) by picnic.mat.net (8.9.3/8.8.5) with ESMTP id WAA14794; Mon, 12 Apr 1999 22:13:30 -0400 (EDT) Date: Mon, 12 Apr 1999 22:13:30 -0400 (EDT) From: Chuck Robey To: "William S. Duncanson" Cc: freebsd-smp@FreeBSD.ORG Subject: Re: SMP broken in -CURRENT? In-Reply-To: <4.1.19990412202347.009bf790@imap.colltech.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Mon, 12 Apr 1999, William S. Duncanson wrote: > At 09:00 PM 4/12/99 -0400, Chuck Robey wrote: > >On Mon, 12 Apr 1999, William S. Duncanson wrote: > > > >> I haven't been able to get a working SMP kernel out of -CURRENT recently. > >> I don't know exactly when it broke, because I usually rebuild on a weekly > >> basis. The kernel hangs after: > >> APIC_IO: Testing 8254 interrupt delivery > >> and doesn't ever come back (panic or otherwise). > >> > >> The one thing that I noticed is that on the older kernels, CPU#1 is > >> launched after the APIC_IO Testing and Routing. On the newer kernels, > >> CPU#1 is launched far earlier. > >> > >> Anybody have any ideas? > > > >Tell us how recent ... I built my last kernel on last Saturday, and it > >works fine (freshly cvsupped sources). If yours is not more recent than > >that, it might be your config or your hardware. > > > > I tried late Saturday (4/10), Sunday (4/11) and today (4/12). The working > kernel that I have is from 4/4. I'm using the same config as from the > working kernel, and there doesn't appear to have been any relevant changes > in LINT. I'm going to try blowing away my source tree completely and > re-cvsupping, but... > > Does your CPU#1 launch right after the kernel loads, or does it wait until > the "normal" time? After the timecounter, and the pnp config data gets registered in from kernel.conf, then the IOApic gets checked, and the 2nd cpu fires up. If your data is that recent, I have nothing here to contradict you, my kernel is likely older. I'm building world now, but I won't have data tonight. ----------------------------+----------------------------------------------- Chuck Robey | Interests include any kind of voice or data chuckr@picnic.mat.net | communications topic, C programming, and Unix. 213 Lakeside Drive Apt T-1 | Greenbelt, MD 20770 | I run picnic (FreeBSD-current) (301) 220-2114 | and jaunt (Solaris7). ----------------------------+----------------------------------------------- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Apr 12 20:16: 5 1999 Delivered-To: freebsd-smp@freebsd.org Received: from ix.netcom.com (pax-ca27-08.ix.netcom.com [207.93.145.40]) by hub.freebsd.org (Postfix) with ESMTP id 8E52114BC9; Mon, 12 Apr 1999 20:16:00 -0700 (PDT) (envelope-from tomdean@ix.netcom.com) Received: (from tomdean@localhost) by ix.netcom.com (8.9.3/8.8.8) id UAA00338; Mon, 12 Apr 1999 20:13:39 -0700 (PDT) (envelope-from tomdean) Date: Mon, 12 Apr 1999 20:13:39 -0700 (PDT) Message-Id: <199904130313.UAA00338@ix.netcom.com> From: Thomas Dean To: freebsd-smp@FreeBSD.ORG, freebsd-current@FreeBSD.ORG Subject: CVSUP, Make World, Patch mp_machdep.c, Make Kernel Works Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I am running 4.0-current SMP, as of an hour ago, cvsup about noon, today. I have been holding off, watching the egcs development. It all works. egcs all appears to work, I rebuilt some applications. I did a cvsup, make world, and, after applying the patch to delay starting cpu#1, built a kernel and it all works. I tried building a kernel before I applied the patch and the hang problem was there. Great job in getting it all together. Now, to rebuild 37 ports! Thanks, core team tomdean To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Apr 12 20:34:38 1999 Delivered-To: freebsd-smp@freebsd.org Received: from fire.starkreality.com (fire.starkreality.com [208.24.48.226]) by hub.freebsd.org (Postfix) with ESMTP id BED1014D53; Mon, 12 Apr 1999 20:34:33 -0700 (PDT) (envelope-from caesar@starkreality.com) Received: from armageddon (armageddon.starkreality.com [208.24.48.227]) by fire.starkreality.com (8.9.3/8.9.2) with SMTP id WAA01039; Mon, 12 Apr 1999 22:31:42 -0500 (CDT) Message-Id: <4.1.19990412222940.03bde1e0@fire.starkreality.com> X-Sender: caesar@fire.starkreality.com X-Mailer: QUALCOMM Windows Eudora Pro Version 4.1 Date: Mon, 12 Apr 1999 22:31:28 -0500 To: Tor.Egge@fast.no From: "William S. Duncanson" Subject: Re: SMP broken in -CURRENT? Cc: freebsd-smp@FreeBSD.ORG, freebsd-current@FreeBSD.ORG In-Reply-To: <199904130213.EAA29557@midten.fast.no> References: <4.1.19990412191627.009b3100@imap.colltech.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org That fixes it, thanks. At 04:13 4/13/99 +0200, Tor.Egge@fast.no wrote: >> I haven't been able to get a working SMP kernel out of -CURRENT recently. >> I don't know exactly when it broke, because I usually rebuild on a weekly >> basis. The kernel hangs after: >> APIC_IO: Testing 8254 interrupt delivery >> and doesn't ever come back (panic or otherwise). >> >> The one thing that I noticed is that on the older kernels, CPU#1 is >> launched after the APIC_IO Testing and Routing. On the newer kernels, >> CPU#1 is launched far earlier. >> >> Anybody have any ideas? > >You might want to try this patch, which disables the early start of CPU#1. > >Index: mp_machdep.c >=================================================================== >RCS file: /home/ncvs/src/sys/i386/i386/mp_machdep.c,v >retrieving revision 1.96 >diff -u -r1.96 mp_machdep.c >--- mp_machdep.c 1999/04/11 00:43:43 1.96 >+++ mp_machdep.c 1999/04/13 02:08:54 >@@ -1930,9 +1930,11 @@ > for (i = 0; i < mp_ncpus; i++) { > bcopy( (int *) PTD + KPTDI, (int *) IdlePTDS[i] + KPTDI, NKPDE * sizeof >(int)); > } >+#if 0 > wait_ap(1000000); > if (smp_started == 0) > printf("WARNING: Failed to start all APs\n"); >+#endif > > /* number of APs actually started */ > return mp_ncpus - 1; > > > > >To Unsubscribe: send mail to majordomo@FreeBSD.org >with "unsubscribe freebsd-smp" in the body of the message William S. Duncanson caesar@starkreality.com The driving force behind the NC is the belief that the companies who brought us things like Unix, relational databases, and Windows can make an appliance that is inexpensive and easy to use if they choose to do that. -- Scott Adams To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Apr 12 22:21:45 1999 Delivered-To: freebsd-smp@freebsd.org Received: from thelab.hub.org (nat192.236.mpoweredpc.net [142.177.192.236]) by hub.freebsd.org (Postfix) with ESMTP id 4A56214CFC for ; Mon, 12 Apr 1999 22:21:41 -0700 (PDT) (envelope-from scrappy@hub.org) Received: from localhost (scrappy@localhost) by thelab.hub.org (8.9.3/8.9.1) with ESMTP id CAA55246 for ; Tue, 13 Apr 1999 02:19:28 -0300 (ADT) (envelope-from scrappy@hub.org) X-Authentication-Warning: thelab.hub.org: scrappy owned process doing -bs Date: Tue, 13 Apr 1999 02:19:28 -0300 (ADT) From: The Hermit Hacker To: freebsd-smp@freebsd.org Subject: SMP under 3.x-STABLE ... Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I'm considering upgrading one of my production boxes to a Dual-PII ASUS motherboard ... I want to leave it running 3.x-STABLE, but am curious as to how ppl feel about SMP support in 3.x-STABLE... Basically, is it stable? Or should it be avoided? Thanks... Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy Systems Administrator @ hub.org primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Apr 13 6:36: 8 1999 Delivered-To: freebsd-smp@freebsd.org Received: from fast.cs.utah.edu (fast.cs.utah.edu [155.99.212.1]) by hub.freebsd.org (Postfix) with ESMTP id 1A3051507A for ; Tue, 13 Apr 1999 06:36:05 -0700 (PDT) (envelope-from vanmaren@fast.cs.utah.edu) Received: (from vanmaren@localhost) by fast.cs.utah.edu (8.9.1/8.9.1) id HAA12034; Tue, 13 Apr 1999 07:33:45 -0600 (MDT) Date: Tue, 13 Apr 1999 07:33:45 -0600 (MDT) From: Kevin Van Maren Message-Id: <199904131333.HAA12034@fast.cs.utah.edu> To: freebsd-smp@FreeBSD.ORG, scrappy@hub.org Subject: Re: SMP under 3.x-STABLE ... Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org SMP support in 3.0 was pretty good -- the kernel crashes for different reasons. 3.1 is behaving well for me (although I am a little past 3.1). The Asus P2D-DS is a nice board (although an extra PCI slot would be nice). Kevin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Apr 13 9:35: 6 1999 Delivered-To: freebsd-smp@freebsd.org Received: from noc.demon.net (server.noc.demon.net [193.195.224.4]) by hub.freebsd.org (Postfix) with ESMTP id DECAD15773 for ; Tue, 13 Apr 1999 09:35:00 -0700 (PDT) (envelope-from fanf@demon.net) Received: by noc.demon.net; id RAA28680; Tue, 13 Apr 1999 17:32:39 +0100 (BST) Received: from fanf.noc.demon.net(195.11.55.83) by inside.noc.demon.net via smap (3.2) id xma028651; Tue, 13 Apr 99 17:32:27 +0100 Received: from fanf by fanf.noc.demon.net with local (Exim 1.73 #2) id 10X66t-000323-00; Tue, 13 Apr 1999 17:32:27 +0100 To: smp@freebsd.org From: Tony Finch Subject: Re: SMP under 3.x-STABLE ... Newsgroups: chiark.mail.freebsd.smp In-Reply-To: <199904131333.HAA12034@fast.cs.utah.edu> Organization: Deliberate Obfuscation To Amuse Tony Message-Id: Date: Tue, 13 Apr 1999 17:32:27 +0100 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Kevin Van Maren wrote: > >The Asus P2D-DS is a nice board What speed processors do you have in it? We've had problems with 450MHz PIIs -- see the thread in freebsd-hackers. Tony. -- f.a.n.finch dot@dotat.at fanf@demon.net Arthur: "Oh, that sounds better, have you worked out the controls?" Ford: "No, we just stopped playing with them." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Apr 16 6:59:41 1999 Delivered-To: freebsd-smp@freebsd.org Received: from ringer.cisco.com (ringer.cisco.com [144.254.142.11]) by hub.freebsd.org (Postfix) with ESMTP id 22FBD153FD; Fri, 16 Apr 1999 06:59:32 -0700 (PDT) (envelope-from amcrae@cisco.com) Received: (amcrae@localhost) by ringer.cisco.com (8.8.4-Cisco.1/8.6.5) id XAA15234; Fri, 16 Apr 1999 23:57:29 +1000 (EST) From: Andrew McRae Message-Id: <199904161357.XAA15234@ringer.cisco.com> Subject: SMP kernel on SuperMicro P6DBE To: freebsd-smp@freebsd.org, freebsd-hardware@freebsd.org Date: Fri, 16 Apr 1999 23:57:29 +1000 (EST) X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Comrades, I have swapped in a Supermicro P6DBE dual PII motherboard and am attempting to run FreeBSD current on it. I originally installed two 350MHz PIIs, and booted a prebuilt SMP kernel. It ran fine for a few minutes, long enough for me to start X, run netscape etc., and then it hung. After a few seconds it rebooted. Checking the SMP web details, I tried a kernel with SMP_TIMER_NC, but the same occurred. I removed one of the PIIs, and it works just fine with only one CPU. Are there any known issues with this motherboard? Is there anything I should try to counter this problem? I have attached boot messages and the output of mptable. Cheers, Andrew McRae (amcrae@cisco.com) Boot messages are: Copyright (c) 1992-1999 The FreeBSD Project. Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All rights reserved. FreeBSD 4.0-CURRENT #2: Fri Apr 16 22:29:25 EST 1999 amcrae@sporran:/home/amcrae/perforce/usr/src/sys/compile/SMP-SPORRAN Timecounter "i8254" frequency 1193182 Hz CPU: Pentium II/Xeon/Celeron (686-class CPU) Origin = "GenuineIntel" Id = 0x652 Stepping=2 Features=0x183fbff real memory = 67108864 (65536K bytes) avail memory = 62521344 (61056K bytes) Programming 24 pins in IOAPIC #0 WARNING: Failed to start all APs FreeBSD/SMP: Multiprocessor motherboard cpu0 (BSP): apic id: 0, version: 0x00040011, at 0xfee00000 io0 (APIC): apic id: 2, version: 0x00170011, at 0xfec00000 Preloaded elf kernel "kernel" at 0xc02b8000. Pentium Pro MTRR support enabled, default memory type is uncacheable Probing for devices on PCI bus 0: chip0: rev 0x02 on pci0.0.0 chip1: rev 0x02 on pci0.1.0 chip2: rev 0x02 on pci0.7.0 ide_pci0: rev 0x01 on pci0.7.1 chip3: rev 0x02 on pci0.7.3 vga0: rev 0x01 int a irq 17 on pci0.16.0 ncr0: rev 0x04 int a irq 18 on pci0.18.0 Probing for devices on PCI bus 1: Probing for PnP devices: Probing for devices on the ISA bus: sc0 on isa sc0: VGA color <16 virtual consoles, flags=0x0> ed0 at 0x300-0x31f irq 10 maddr 0xd8000 msize 16384 on isa ed0: address 00:00:c0:f9:e1:8a, type WD8013WC (16 bit) atkbdc0 at 0x60-0x6f on motherboard sio0 at 0x3f8-0x3ff irq 4 on isa sio0: type 16550A sio1 at 0x2f8-0x2ff irq 3 on isa sio1: type 16550A fdc0 at 0x3f0-0x3f7 irq 6 drq 2 on isa fdc0: FIFO enabled, 8 bytes threshold fd0: 1.44MB 3.5in wdc0 not found at 0x1f0 ppc0 at 0x378 irq 7 on isa ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/9 bytes threshold ppb0: IEEE1284 device found /NIBBLE/ECP Probing for PnP devices on ppbus0: ppbus0: MLC,PCL,PML lpt0: on ppbus 0 lpt0: Interrupt-driven port vga0 at 0x3b0-0x3df maddr 0xa0000 msize 131072 on isa npx0 on motherboard npx0: INT 16 interface APIC_IO: Testing 8254 interrupt delivery APIC_IO: routing 8254 via pin 2 Waiting 2 seconds for SCSI devices to settle sa0 at ncr0 bus 0 target 4 lun 0 sa0: Removable Sequential Access SCSI-2 device sa0: 4.032MB/s transfers (4.032MHz, offset 11) cd0 at ncr0 bus 0 target 6 lun 0 cd0: Removable CD-ROM SCSI-2 device cd0: 19.230MB/s transfers (19.230MHz, offset 16) cd0: Attempt to query device size failed: NOT READY, Medium not present da0 at ncr0 bus 0 target 1 lun 0 da0: Fixed Direct Access SCSI-2 device da0: 40.000MB/s transfers (20.000MHz, offset 15, 16bit), Tagged Queueing Enabled da0: 4357MB (8925000 512 byte sectors: 255H 63S/T 555C) changing root device to da0s4a mptable tells me: =============================================================================== MPTable, version 2.0.15 ------------------------------------------------------------------------------- MP Floating Pointer Structure: location: BIOS physical address: 0x000fb500 signature: '_MP_' length: 16 bytes version: 1.1 checksum: 0x5b mode: Virtual Wire ------------------------------------------------------------------------------- MP Config Table Header: physical address: 0x000f69d0 signature: 'PCMP' base table length: 248 version: 1.1 checksum: 0xa9 OEM ID: 'INTEL ' Product ID: '440BX ' OEM table pointer: 0x00000000 OEM table size: 0 entry count: 24 local APIC address: 0xfee00000 extended table length: 0 extended table checksum: 0 ------------------------------------------------------------------------------- MP Config Base Table Entries: -- Processors: APIC ID Version State Family Model Step Flags 0 0x11 BSP, usable 6 5 2 0x183fb ff -- Bus: Bus ID Type 0 PCI 1 PCI 2 ISA -- I/O APICs: APIC ID Version State Address 2 0x11 usable 0xfec00000 -- I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN# ExtINT conforms conforms 2 0 2 0 INT conforms conforms 2 1 2 1 INT conforms conforms 2 0 2 2 INT conforms conforms 2 3 2 3 INT conforms conforms 2 4 2 4 INT conforms conforms 2 5 2 5 INT conforms conforms 2 6 2 6 INT conforms conforms 2 7 2 7 INT active-hi edge 2 8 2 8 INT conforms conforms 2 10 2 10 INT conforms conforms 2 12 2 12 INT conforms conforms 2 13 2 13 INT conforms conforms 2 14 2 14 INT active-lo level 2 11 2 17 INT active-lo level 2 9 2 18 INT active-lo level 2 15 2 19 SMI conforms conforms 2 0 2 23 -- Local Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN# ExtINT conforms conforms 0 0:A 255 0 NMI conforms conforms 0 0:A 255 1 ------------------------------------------------------------------------------- # SMP kernel config file options: # Required: options SMP # Symmetric MultiProcessor Kernel options APIC_IO # Symmetric (APIC) I/O # Optional (built-in defaults will work in most cases): #options NCPU=1 # number of CPUs #options NBUS=3 # number of busses #options NAPIC=1 # number of IO APICs #options NINTR=24 # number of INTs =============================================================================== To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Apr 20 6:55:25 1999 Delivered-To: freebsd-smp@freebsd.org Received: from zed.ludd.luth.se (zed.ludd.luth.se [130.240.16.33]) by hub.freebsd.org (Postfix) with ESMTP id 806DD15755 for ; Tue, 20 Apr 1999 06:55:22 -0700 (PDT) (envelope-from magnus@ludd.luth.se) Received: from speedy.ludd.luth.se (root@speedy.ludd.luth.se [130.240.16.164]) by zed.ludd.luth.se (8.8.5/8.8.5) with ESMTP id PAA20250 for ; Tue, 20 Apr 1999 15:52:54 +0200 From: Magnus Gr|nlund Received: (magnus@localhost) by speedy.ludd.luth.se (8.8.8/8.6.11) id PAA16241 for freebsd-smp@freebsd.org; Tue, 20 Apr 1999 15:49:33 +0200 (CEST) Message-Id: <199904201349.PAA16241@speedy.ludd.luth.se> Subject: Slooow SMP on Tekram P6B40D-A5. To: freebsd-smp@freebsd.org Date: Tue, 20 Apr 1999 15:49:33 +0200 (CEST) X-Mailer: ELM [version 2.4ME+ PL32 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hello. I just got myself an new Tekram motherboard (and two modified Celerons). Apperantly there is a known bug in the bios that can make dual-operation slower than single. At first I tried a 4.0-current kernel from late february, which worked OK. But today i decided to sync up to a "current" current, cvsup:ed and built a new kernel (throwing away the old one.) and got bitten. :-( Any way around this? Thanks... /Magnus To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Apr 20 19:27:45 1999 Delivered-To: freebsd-smp@freebsd.org Received: from cain.gsoft.com.au (genesi.lnk.telstra.net [139.130.136.161]) by hub.freebsd.org (Postfix) with ESMTP id 228F5152F5 for ; Tue, 20 Apr 1999 19:27:39 -0700 (PDT) (envelope-from doconnor@gsoft.com.au) Received: from lot.gsoft.com.au (lot.gsoft.com.au [203.38.152.106]) by cain.gsoft.com.au (8.8.8/8.8.8) with ESMTP id LAA27125; Wed, 21 Apr 1999 11:54:54 +0930 (CST) (envelope-from doconnor@gsoft.com.au) Message-ID: X-Mailer: XFMail 1.3 [p0] on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <199904201349.PAA16241@speedy.ludd.luth.se> Date: Wed, 21 Apr 1999 12:02:33 +0930 (CST) From: "Daniel O'Connor" To: Magnus Gr|nlund Subject: RE: Slooow SMP on Tekram P6B40D-A5. Cc: freebsd-smp@FreeBSD.ORG Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 20-Apr-99 Magnus Gr|nlund wrote: > I just got myself an new Tekram motherboard (and two modified Celerons). > > Apperantly there is a known bug in the bios that can make > dual-operation slower than single. > > At first I tried a 4.0-current kernel from late february, which worked OK. > But today i decided to sync up to a "current" current, cvsup:ed and built > a new kernel (throwing away the old one.) and got bitten. :-( Well, I just updated recently too, and I'm having this problem as well.. Its a bit like doing make -j 16 buildworld in the background :( [guppy 11:39] /usr/src/sys > grep Index: /tmp/kernel-0904-1304.diff | wc -l 110 Bleh! :( 132k of diff's between the last time I updated my kernel. There where changes to SMP code during that time, so I guess thats it.. I'll try and find a date closer to which its broken (ie checkout different kernel sources) --- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Apr 21 5: 8:31 1999 Delivered-To: freebsd-smp@freebsd.org Received: from midget.dons.net.au (daniel.lnk.telstra.net [139.130.137.70]) by hub.freebsd.org (Postfix) with ESMTP id C44A714DC8 for ; Wed, 21 Apr 1999 05:08:20 -0700 (PDT) (envelope-from darius@dons.net.au) Received: from guppy.dons.net.au (guppy.dons.net.au [203.31.81.9]) by midget.dons.net.au (8.9.3/8.9.1) with ESMTP id VAA58063 for ; Wed, 21 Apr 1999 21:35:51 +0930 (CST) (envelope-from darius@dons.net.au) Message-ID: X-Mailer: XFMail 1.3 [p0] on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 Date: Wed, 21 Apr 1999 21:35:50 +0930 (CST) From: "Daniel J. O'Connor" To: freebsd-smp@freebsd.org Subject: Really slow SMP Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Well, the subject says it all :) I have a dual pii-350 system, and I updated to -current on the 13th, it was running -current before (on the 7th or so). It was working on 7th April 20:00, and was broken on the 8th April 22:00. (These times are CST which is 10:30 hrs ahead of GMT) It didn't compile on the 8th at 00:00 (missing getmtrr() and friends) The diff's between those two dates have only one 'interesting' thing.. all the getmtrr() et al calls where #if'd out. Hmm, well now I feel like I have 2 and 2 and it doesn't add up to 4 =) --- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Apr 21 7:54:47 1999 Delivered-To: freebsd-smp@freebsd.org Received: from spinner.netplex.com.au (spinner.netplex.com.au [202.12.86.3]) by hub.freebsd.org (Postfix) with ESMTP id 6C54614BE6 for ; Wed, 21 Apr 1999 07:54:37 -0700 (PDT) (envelope-from peter@netplex.com.au) Received: from netplex.com.au (localhost [127.0.0.1]) by spinner.netplex.com.au (Postfix) with ESMTP id 8A7571F2A; Wed, 21 Apr 1999 22:52:06 +0800 (WST) (envelope-from peter@netplex.com.au) X-Mailer: exmh version 2.0.2 2/24/98 To: "Daniel J. O'Connor" Cc: freebsd-smp@freebsd.org Subject: Re: Really slow SMP In-reply-to: Your message of "Wed, 21 Apr 1999 21:35:50 +0930." Date: Wed, 21 Apr 1999 22:52:06 +0800 From: Peter Wemm Message-Id: <19990421145208.8A7571F2A@spinner.netplex.com.au> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org "Daniel J. O'Connor" wrote: > Well, the subject says it all :) > > I have a dual pii-350 system, and I updated to -current on the 13th, it was > running -current before (on the 7th or so). > > It was working on 7th April 20:00, and was broken on the 8th April 22:00. > (These times are CST which is 10:30 hrs ahead of GMT) > > It didn't compile on the 8th at 00:00 (missing getmtrr() and friends) > > The diff's between those two dates have only one 'interesting' thing.. all th e > getmtrr() et al calls where #if'd out. > > Hmm, well now I feel like I have 2 and 2 and it doesn't add up to 4 =) Hmm! Also, consider this from my dmesg: [..] Preloaded elf kernel "kernel" at 0xc02c8000. Pentium Pro MTRR support enabled, default memory type is uncacheable Probing for PnP devices: [..] Memory type is uncacheable?!? Is that saying what it sounds like? (There is no MTRR synchronization like there was before. Previously the BSP would dump all it's MTRR registers to a table and all the other AP cpus would load that table on startup. That table doesn't exist anymore.) Cheers, -Peter To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Apr 21 8:14:38 1999 Delivered-To: freebsd-smp@freebsd.org Received: from midget.dons.net.au (daniel.lnk.telstra.net [139.130.137.70]) by hub.freebsd.org (Postfix) with ESMTP id 969221523E for ; Wed, 21 Apr 1999 08:14:34 -0700 (PDT) (envelope-from doconnor@gsoft.com.au) Received: from guppy.dons.net.au (guppy.dons.net.au [203.31.81.9]) by midget.dons.net.au (8.9.3/8.9.1) with ESMTP id AAA58341; Thu, 22 Apr 1999 00:41:57 +0930 (CST) (envelope-from doconnor@gsoft.com.au) Message-ID: X-Mailer: XFMail 1.3 [p0] on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <19990421145208.8A7571F2A@spinner.netplex.com.au> Date: Thu, 22 Apr 1999 00:41:57 +0930 (CST) From: "Daniel J. O'Connor" To: Peter Wemm Subject: Re: Really slow SMP Cc: freebsd-smp@freebsd.org, "Daniel J. O'Connor" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 21-Apr-99 Peter Wemm wrote: > Memory type is uncacheable?!? Is that saying what it sounds like? > > (There is no MTRR synchronization like there was before. Previously the > BSP would dump all it's MTRR registers to a table and all the other AP > cpus would load that table on startup. That table doesn't exist anymore.) Hmm.. no memory cache.. Blech. I have the same comment on my machine too in dmesg. I'm assuming its an SMP only thing otherwise the list would be chock full of complaints :) And I didn't realise how useful caching was ;) So is there any solution? --- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Apr 21 8:53:11 1999 Delivered-To: freebsd-smp@freebsd.org Received: from monsoon.dial.pipex.net (monsoon.dial.pipex.net [158.43.128.69]) by hub.freebsd.org (Postfix) with SMTP id 5E3E214ED9 for ; Wed, 21 Apr 1999 08:53:04 -0700 (PDT) (envelope-from pierre.dampure@k2c.co.uk) Received: (qmail 11102 invoked from network); 21 Apr 1999 15:32:39 -0000 Received: from userai32.uk.uudial.com (HELO jfsebastian) (62.188.133.62) by smtp.dial.pipex.com with SMTP; 21 Apr 1999 15:32:39 -0000 Message-ID: <000e01be8c0b$ed1b22a0$0242a8c0@jfsebastian.k2c.co.uk> From: "Pierre Y. Dampure" To: "Daniel J. O'Connor" , "Peter Wemm" Cc: Subject: Re: Really slow SMP Date: Wed, 21 Apr 1999 16:30:45 +0100 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 4.72.3110.5 X-MimeOLE: Produced By Microsoft MimeOLE V4.72.3110.3 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >Hmm.. no memory cache.. Blech. > >I have the same comment on my machine too in dmesg. I'm assuming its an SMP >only thing otherwise the list would be chock full of complaints :) > >And I didn't realise how useful caching was ;) > >So is there any solution? According to the CVS tree, it looks like Mike Smith decided to rewrite the P6 hardware optimisations as a separate file (/sys/i386/i386/i686_mem.c), but the new functions don't seem to be called yet. Peter commented out the old getmtrr() / putmtrr() calls on the 7th because they were not there any further and thus broke a kernel make. I suspect Mike will fix this soon... PYD To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Apr 21 12:21:13 1999 Delivered-To: freebsd-smp@freebsd.org Received: from freebsd.dk (freebsd.dk [212.242.42.178]) by hub.freebsd.org (Postfix) with ESMTP id 553CA1586D for ; Wed, 21 Apr 1999 12:20:41 -0700 (PDT) (envelope-from sos@freebsd.dk) Received: (from sos@localhost) by freebsd.dk (8.9.1/8.9.1) id VAA34556; Wed, 21 Apr 1999 21:17:50 +0200 (CEST) (envelope-from sos) From: Soren Schmidt Message-Id: <199904211917.VAA34556@freebsd.dk> Subject: Re: Really slow SMP In-Reply-To: <19990421145208.8A7571F2A@spinner.netplex.com.au> from Peter Wemm at "Apr 21, 1999 10:52: 6 pm" To: peter@netplex.com.au (Peter Wemm) Date: Wed, 21 Apr 1999 21:17:50 +0200 (CEST) Cc: darius@dons.net.au, freebsd-smp@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL43 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org It seems Peter Wemm wrote: > > I have a dual pii-350 system, and I updated to -current on the 13th, it was > > running -current before (on the 7th or so). > Hmm! Also, consider this from my dmesg: > [..] > Preloaded elf kernel "kernel" at 0xc02c8000. > Pentium Pro MTRR support enabled, default memory type is uncacheable > Probing for PnP devices: > [..] > > Memory type is uncacheable?!? Is that saying what it sounds like? I think it only refers to video memory etc, as my SMP box still runs at its normal pace, I'd be very surpriced if caching was turned off altogether. > (There is no MTRR synchronization like there was before. Previously the > BSP would dump all it's MTRR registers to a table and all the other AP > cpus would load that table on startup. That table doesn't exist anymore.) That should be done again, havnt checked the new code for what it actually does though... -Søren To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Apr 21 17:49:57 1999 Delivered-To: freebsd-smp@freebsd.org Received: from cain.gsoft.com.au (genesi.lnk.telstra.net [139.130.136.161]) by hub.freebsd.org (Postfix) with ESMTP id D8458150C8 for ; Wed, 21 Apr 1999 17:49:46 -0700 (PDT) (envelope-from doconnor@gsoft.com.au) Received: from lot.gsoft.com.au (lot.gsoft.com.au [203.38.152.106]) by cain.gsoft.com.au (8.8.8/8.8.8) with ESMTP id KAA04964; Thu, 22 Apr 1999 10:15:24 +0930 (CST) (envelope-from doconnor@gsoft.com.au) Message-ID: X-Mailer: XFMail 1.3 [p0] on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <199904211917.VAA34556@freebsd.dk> Date: Thu, 22 Apr 1999 10:23:34 +0930 (CST) From: "Daniel O'Connor" To: Soren Schmidt Subject: Re: Really slow SMP Cc: freebsd-smp@FreeBSD.ORG, (Peter Wemm) Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 21-Apr-99 Soren Schmidt wrote: > > Memory type is uncacheable?!? Is that saying what it sounds like? > I think it only refers to video memory etc, as my SMP box still > runs at its normal pace, I'd be very surpriced if caching was > turned off altogether. Hmm.. well, I wonder what the difference is between you and me is :) Is your system Pentium or Pentium II? My box is a dual PII. --- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Apr 21 23:29:16 1999 Delivered-To: freebsd-smp@freebsd.org Received: from freebsd.dk (freebsd.dk [212.242.42.178]) by hub.freebsd.org (Postfix) with ESMTP id 60CA715937 for ; Wed, 21 Apr 1999 23:28:48 -0700 (PDT) (envelope-from sos@freebsd.dk) Received: (from sos@localhost) by freebsd.dk (8.9.1/8.9.1) id IAA35764; Thu, 22 Apr 1999 08:26:02 +0200 (CEST) (envelope-from sos) From: Soren Schmidt Message-Id: <199904220626.IAA35764@freebsd.dk> Subject: Re: Really slow SMP In-Reply-To: from "Daniel O'Connor" at "Apr 22, 1999 10:23:34 am" To: doconnor@gsoft.com.au (Daniel O'Connor) Date: Thu, 22 Apr 1999 08:26:02 +0200 (CEST) Cc: freebsd-smp@FreeBSD.ORG, peter@netplex.com.au X-Mailer: ELM [version 2.4ME+ PL43 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org It seems Daniel O'Connor wrote: > > On 21-Apr-99 Soren Schmidt wrote: > > > Memory type is uncacheable?!? Is that saying what it sounds like? > > I think it only refers to video memory etc, as my SMP box still > > runs at its normal pace, I'd be very surpriced if caching was > > turned off altogether. > > Hmm.. well, I wonder what the difference is between you and me is :) > Is your system Pentium or Pentium II? > My box is a dual PII. Dual PPro in a Tyan 1662D board. -Søren To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Apr 22 19:22:41 1999 Delivered-To: freebsd-smp@freebsd.org Received: from cain.gsoft.com.au (genesi.lnk.telstra.net [139.130.136.161]) by hub.freebsd.org (Postfix) with ESMTP id 476031521B for ; Thu, 22 Apr 1999 19:22:23 -0700 (PDT) (envelope-from doconnor@gsoft.com.au) Received: from lot.gsoft.com.au (lot.gsoft.com.au [203.38.152.106]) by cain.gsoft.com.au (8.8.8/8.8.8) with ESMTP id LAA13040; Fri, 23 Apr 1999 11:47:14 +0930 (CST) (envelope-from doconnor@gsoft.com.au) Message-ID: X-Mailer: XFMail 1.3 [p0] on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <199904220626.IAA35764@freebsd.dk> Date: Fri, 23 Apr 1999 11:47:14 +0930 (CST) From: "Daniel O'Connor" To: Soren Schmidt Subject: Re: Really slow SMP Cc: peter@netplex.com.au, freebsd-smp@FreeBSD.ORG Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 22-Apr-99 Soren Schmidt wrote: > > Hmm.. well, I wonder what the difference is between you and me is :) > > Is your system Pentium or Pentium II? > > My box is a dual PII. > > Dual PPro in a Tyan 1662D board. Hmm.. well effectivly the same core as my system :-/ --- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Apr 22 20:14:33 1999 Delivered-To: freebsd-smp@freebsd.org Received: from osbie.layer8.net (osbie.layer8.net [166.88.69.10]) by hub.freebsd.org (Postfix) with SMTP id E106614BFE for ; Thu, 22 Apr 1999 20:14:26 -0700 (PDT) (envelope-from black@osbie.layer8.net) Received: (qmail 4809 invoked by uid 1001); 23 Apr 1999 03:11:56 -0000 Date: Thu, 22 Apr 1999 20:11:56 -0700 From: Ben Black To: freebsd-smp@freebsd.org Subject: SMP problems on IBM PC Server 325 Message-ID: <19990422201156.B4703@layer8.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.3i Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org According to this mptable output, both CPUs are detected by the hardware, but 3.1-STABLE doesn't see it. Any ideas? (No, I can't find any BIOS parameters to set the floating pointer version to 1.4) =============================================================================== MPTable, version 2.0.15 ------------------------------------------------------------------------------- MP Floating Pointer Structure: location: EBDA physical address: 0x0009fa10 signature: '_MP_' length: 16 bytes version: 1.1 checksum: 0x80 mode: Virtual Wire ------------------------------------------------------------------------------- MP Config Table Header: physical address: 0x0009fa20 signature: 'PCMP' base table length: 292 version: 1.4 checksum: 0x31 OEM ID: 'IBM ' Product ID: '8640 PCI ' OEM table pointer: 0x00000000 OEM table size: 0 entry count: 28 local APIC address: 0xfee00000 extended table length: 0 extended table checksum: 0 ------------------------------------------------------------------------------- MP Config Base Table Entries: -- Processors: APIC ID Version State Family Model Step Flags 1 0x11 BSP, usable 6 3 4 0x80fbff 0 0x11 AP, usable 6 5 2 0x183fbff -- Bus: Bus ID Type 0 PCI 1 PCI 2 ISA -- I/O APICs: APIC ID Version State Address 2 0x11 usable 0xfec00000 -- I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN# ExtINT active-hi edge 2 0 2 0 INT active-hi edge 2 1 2 1 INT active-hi edge 2 0 2 2 INT active-hi edge 2 3 2 3 INT active-hi edge 2 4 2 4 INT active-hi edge 2 5 2 5 INT active-hi edge 2 6 2 6 INT active-hi edge 2 7 2 7 INT active-hi edge 2 8 2 8 INT active-lo level 2 9 2 9 INT active-lo level 2 10 2 10 INT active-lo level 2 11 2 11 INT active-hi edge 2 12 2 12 INT active-lo edge 2 13 2 13 INT active-lo level 2 14 2 14 INT active-lo level 2 15 2 15 INT active-lo level 0 13:A 2 16 INT active-lo level 0 19:D 2 19 INT active-lo level 1 4:A 2 17 INT active-lo level 1 5:A 2 18 INT active-lo level 1 8:A 2 19 -- Local Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN# NMI active-hi edge 0 0:A 255 1 ------------------------------------------------------------------------------- # SMP kernel config file options: # Required: options SMP # Symmetric MultiProcessor Kernel options APIC_IO # Symmetric (APIC) I/O # Optional (built-in defaults will work in most cases): #options NCPU=2 # number of CPUs #options NBUS=3 # number of busses #options NAPIC=1 # number of IO APICs #options NINTR=24 # number of INTs =============================================================================== -- --b This time my name is Ben. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Apr 23 10:49:15 1999 Delivered-To: freebsd-smp@freebsd.org Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.54]) by hub.freebsd.org (Postfix) with ESMTP id 69DD514FFE for ; Fri, 23 Apr 1999 10:49:05 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.9.3/8.9.1) id KAA54441 for freebsd-smp@freebsd.org; Fri, 23 Apr 1999 10:44:20 -0700 (PDT) (envelope-from sgk) From: Steve Kargl Message-Id: <199904231744.KAA54441@troutmask.apl.washington.edu> Subject: Luoqi's patch status To: freebsd-smp@freebsd.org Date: Fri, 23 Apr 1999 10:44:20 -0700 (PDT) X-Mailer: ELM [version 2.4ME+ PL54 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org What is the status of Luoqi's smp patches? Is this the direction that smp on FreeBSD is going, and are these patches slated for inclusion in source tree? -- Steve To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Apr 23 11: 5:27 1999 Delivered-To: freebsd-smp@freebsd.org Received: from cs.rice.edu (cs.rice.edu [128.42.1.30]) by hub.freebsd.org (Postfix) with ESMTP id BBB99154A4 for ; Fri, 23 Apr 1999 11:05:19 -0700 (PDT) (envelope-from alc@cs.rice.edu) Received: from nonpc.cs.rice.edu (nonpc.cs.rice.edu [128.42.1.219]) by cs.rice.edu (8.9.0/8.9.0) with ESMTP id NAA27395; Fri, 23 Apr 1999 13:02:47 -0500 (CDT) Received: (from alc@localhost) by nonpc.cs.rice.edu (8.9.2/8.7.3) id NAA05831; Fri, 23 Apr 1999 13:02:47 -0500 (CDT) Date: Fri, 23 Apr 1999 13:02:47 -0500 From: Alan Cox To: Steve Kargl Cc: freebsd-smp@freebsd.org Subject: Re: Luoqi's patch status Message-ID: <19990423130247.A5317@nonpc.cs.rice.edu> References: <199904231744.KAA54441@troutmask.apl.washington.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.1i In-Reply-To: <199904231744.KAA54441@troutmask.apl.washington.edu>; from Steve Kargl on Fri, Apr 23, 1999 at 10:44:20AM -0700 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Fri, Apr 23, 1999 at 10:44:20AM -0700, Steve Kargl wrote: > What is the status of Luoqi's smp patches? Is this the > direction that smp on FreeBSD is going, and are these patches > slated for inclusion in source tree? > After considerable discussion of the trade-offs and alternatives among John, David, Bruce, Luoqi (of course), Julian and myself, the consensus was to proceed with Luoqi's %fs-based SMP patch. The downside of this approach is that it increases the null syscall cost by about 8%. Otherwise, the decision would have been a no-brainer. Alan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Apr 23 11:25:20 1999 Delivered-To: freebsd-smp@freebsd.org Received: from cantor.boolean.net (cantor.boolean.net [209.133.111.73]) by hub.freebsd.org (Postfix) with ESMTP id 522F914DF1 for ; Fri, 23 Apr 1999 11:25:17 -0700 (PDT) (envelope-from Kurt@OpenLDAP.Org) Received: from gypsy (localhost [127.0.0.1]) by cantor.boolean.net (8.9.2/8.9.1) with SMTP id SAA75408; Fri, 23 Apr 1999 18:23:36 GMT (envelope-from Kurt@OpenLDAP.Org) Message-Id: <3.0.5.32.19990423111825.009fa9d0@localhost> X-Sender: guru@localhost X-Mailer: QUALCOMM Windows Eudora Pro Version 3.0.5 (32) Date: Fri, 23 Apr 1999 11:18:25 -0700 To: Steve Kargl From: "Kurt D. Zeilenga" Subject: Re: Luoqi's patch status Cc: freebsd-smp@FreeBSD.ORG In-Reply-To: <199904231744.KAA54441@troutmask.apl.washington.edu> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org At 10:44 AM 4/23/99 -0700, Steve Kargl wrote: >What is the status of Luoqi's smp patches? Is this the >direction that smp on FreeBSD is going, and are these patches >slated for inclusion in source tree? I hope so! I've tested these patches on both SMP and UP systems and have not unearthed any problems with them. Kurt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Apr 23 15:25:23 1999 Delivered-To: freebsd-smp@freebsd.org Received: from dingo.cdrom.com (castles539.castles.com [208.214.165.103]) by hub.freebsd.org (Postfix) with ESMTP id 4D617154F9 for ; Fri, 23 Apr 1999 15:25:10 -0700 (PDT) (envelope-from mike@dingo.cdrom.com) Received: from dingo.cdrom.com (localhost [127.0.0.1]) by dingo.cdrom.com (8.9.3/8.8.8) with ESMTP id HAA00351; Fri, 23 Apr 1999 07:35:00 -0700 (PDT) (envelope-from mike@dingo.cdrom.com) Message-Id: <199904231435.HAA00351@dingo.cdrom.com> X-Mailer: exmh version 2.0.2 2/24/98 To: "Daniel J. O'Connor" Cc: freebsd-smp@FreeBSD.ORG Subject: Re: Really slow SMP In-reply-to: Your message of "Wed, 21 Apr 1999 21:35:50 +0930." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 23 Apr 1999 07:35:00 -0700 From: Mike Smith Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Well, the subject says it all :) > > I have a dual pii-350 system, and I updated to -current on the 13th, it was > running -current before (on the 7th or so). > > It was working on 7th April 20:00, and was broken on the 8th April 22:00. > (These times are CST which is 10:30 hrs ahead of GMT) > > It didn't compile on the 8th at 00:00 (missing getmtrr() and friends) > > The diff's between those two dates have only one 'interesting' thing.. all the > getmtrr() et al calls where #if'd out. > > Hmm, well now I feel like I have 2 and 2 and it doesn't add up to 4 =) Just out of curiosity; can you send me the output of "memcontrol list"? -- \\ Sometimes you're ahead, \\ Mike Smith \\ sometimes you're behind. \\ mike@smith.net.au \\ The race is long, and in the \\ msmith@freebsd.org \\ end it's only with yourself. \\ msmith@cdrom.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Apr 23 15:25:24 1999 Delivered-To: freebsd-smp@freebsd.org Received: from dingo.cdrom.com (castles539.castles.com [208.214.165.103]) by hub.freebsd.org (Postfix) with ESMTP id 8179A14CC6 for ; Fri, 23 Apr 1999 15:25:10 -0700 (PDT) (envelope-from mike@dingo.cdrom.com) Received: from dingo.cdrom.com (localhost [127.0.0.1]) by dingo.cdrom.com (8.9.3/8.8.8) with ESMTP id HAA00413; Fri, 23 Apr 1999 07:42:17 -0700 (PDT) (envelope-from mike@dingo.cdrom.com) Message-Id: <199904231442.HAA00413@dingo.cdrom.com> X-Mailer: exmh version 2.0.2 2/24/98 To: Peter Wemm Cc: "Daniel J. O'Connor" , freebsd-smp@FreeBSD.ORG Subject: Re: Really slow SMP In-reply-to: Your message of "Wed, 21 Apr 1999 22:52:06 +0800." <19990421145208.8A7571F2A@spinner.netplex.com.au> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 23 Apr 1999 07:42:17 -0700 From: Mike Smith Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org My apologies for the scad of these in a row; I'm doing my mail while I lurk in airports, and it hasn't been fun, I tell you. > Hmm! Also, consider this from my dmesg: > [..] > Preloaded elf kernel "kernel" at 0xc02c8000. > Pentium Pro MTRR support enabled, default memory type is uncacheable > Probing for PnP devices: > [..] > > Memory type is uncacheable?!? Is that saying what it sounds like? Yes, the default memory type is uncacheable. There should be a variable range then set by the BIOS that covers all physical memory; you can look with "memcontrol list". > (There is no MTRR synchronization like there was before. Previously the > BSP would dump all it's MTRR registers to a table and all the other AP > cpus would load that table on startup. That table doesn't exist anymore.) Hmm. It's quite likely that the BIOS is only setting the MTRRs in the BSP; why aren't the APs doing this anymore? -- \\ Sometimes you're ahead, \\ Mike Smith \\ sometimes you're behind. \\ mike@smith.net.au \\ The race is long, and in the \\ msmith@freebsd.org \\ end it's only with yourself. \\ msmith@cdrom.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Apr 25 8:16:36 1999 Delivered-To: freebsd-smp@freebsd.org Received: from midget.dons.net.au (daniel.lnk.telstra.net [139.130.137.70]) by hub.freebsd.org (Postfix) with ESMTP id 10BF5152D6 for ; Sun, 25 Apr 1999 08:16:28 -0700 (PDT) (envelope-from doconnor@gsoft.com.au) Received: from guppy.dons.net.au (guppy.dons.net.au [203.31.81.9]) by midget.dons.net.au (8.9.3/8.9.1) with ESMTP id AAA69376; Mon, 26 Apr 1999 00:46:16 +0930 (CST) (envelope-from doconnor@gsoft.com.au) Message-ID: X-Mailer: XFMail 1.3 [p0] on FreeBSD X-Priority: 3 (Normal) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="_=XFMail.1.3.p0.FreeBSD:990426004614:369=_" In-Reply-To: <199904231435.HAA00351@dingo.cdrom.com> Date: Mon, 26 Apr 1999 00:46:14 +0930 (CST) From: "Daniel J. O'Connor" To: Mike Smith Subject: Re: Really slow SMP Cc: freebsd-smp@freebsd.org, "Daniel J. O'Connor" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org This message is in MIME format --_=XFMail.1.3.p0.FreeBSD:990426004614:369=_ Content-Type: text/plain; charset=us-ascii On 23-Apr-99 Mike Smith wrote: > > Hmm, well now I feel like I have 2 and 2 and it doesn't add up to 4 =) > Just out of curiosity; can you send me the output of "memcontrol list"? OK.. The list seem normal.. (ie ISA manglings, and write back caching for everything with the default of no caching for PCI) BTW I also attached some patches to enable the help text :) --- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum --_=XFMail.1.3.p0.FreeBSD:990426004614:369=_ Content-Disposition: attachment; filename="memcontrol-diff" Content-Transfer-Encoding: 7bit Content-Description: memcontrol-diff Content-Type: text/plain; charset=us-ascii; name=memcontrol-diff; SizeOnDisk=936 Index: memcontrol.c =================================================================== RCS file: /remote1/ncvs/src/usr.sbin/memcontrol/memcontrol.c,v retrieving revision 1.1.1.1 diff -c -r1.1.1.1 memcontrol.c *** memcontrol.c 1999/04/07 04:11:14 1.1.1.1 --- memcontrol.c 1999/04/25 15:08:43 *************** *** 315,325 **** static void helpfunc(int memfd, int argc, char *argv[]) { ! help(NULL); } static void help(char *what) { ! errx(1, "help!"); } --- 315,337 ---- static void helpfunc(int memfd, int argc, char *argv[]) { ! printf("Usage:\n\n"); ! help(argv[1]); } static void help(char *what) { ! int i; ! ! for (i = 0; functions[i].cmd != NULL; i++) { ! if ((what == NULL) || !strcasecmp(functions[i].cmd, what)) { ! printf("Command: %s\n%s\n\n", functions[i].cmd, ! functions[i].desc); ! if (what != NULL) ! break; ! } ! } ! ! /* errx(1, "help!"); */ } --_=XFMail.1.3.p0.FreeBSD:990426004614:369=_ Content-Disposition: attachment; filename="memcontrol-list.txt" Content-Transfer-Encoding: 7bit Content-Description: memcontrol-list.txt Content-Type: text/plain; charset=us-ascii; name=memcontrol-list.txt; SizeOnDisk=6750 0/10000 BIOS write-back fixed-base fixed-length set-by-firmware active 10000/10000 BIOS write-back fixed-base fixed-length set-by-firmware active 20000/10000 BIOS write-back fixed-base fixed-length set-by-firmware active 30000/10000 BIOS write-back fixed-base fixed-length set-by-firmware active 40000/10000 BIOS write-back fixed-base fixed-length set-by-firmware active 50000/10000 BIOS write-back fixed-base fixed-length set-by-firmware active 60000/10000 BIOS write-back fixed-base fixed-length set-by-firmware active 70000/10000 BIOS write-back fixed-base fixed-length set-by-firmware active 80000/4000 BIOS write-back fixed-base fixed-length set-by-firmware active 84000/4000 BIOS write-back fixed-base fixed-length set-by-firmware active 88000/4000 BIOS write-back fixed-base fixed-length set-by-firmware active 8c000/4000 BIOS write-back fixed-base fixed-length set-by-firmware active 90000/4000 BIOS write-back fixed-base fixed-length set-by-firmware active 94000/4000 BIOS write-back fixed-base fixed-length set-by-firmware active 98000/4000 BIOS write-back fixed-base fixed-length set-by-firmware active 9c000/4000 BIOS write-back fixed-base fixed-length set-by-firmware active a0000/4000 BIOS uncacheable fixed-base fixed-length set-by-firmware active a4000/4000 BIOS uncacheable fixed-base fixed-length set-by-firmware active a8000/4000 BIOS uncacheable fixed-base fixed-length set-by-firmware active ac000/4000 BIOS uncacheable fixed-base fixed-length set-by-firmware active b0000/4000 BIOS write-combine fixed-base fixed-length set-by-firmware active b4000/4000 BIOS write-combine fixed-base fixed-length set-by-firmware active b8000/4000 BIOS write-combine fixed-base fixed-length set-by-firmware active bc000/4000 BIOS write-combine fixed-base fixed-length set-by-firmware active c0000/1000 BIOS write-protect fixed-base fixed-length set-by-firmware active c1000/1000 BIOS write-protect fixed-base fixed-length set-by-firmware active c2000/1000 BIOS write-protect fixed-base fixed-length set-by-firmware active c3000/1000 BIOS write-protect fixed-base fixed-length set-by-firmware active c4000/1000 BIOS write-protect fixed-base fixed-length set-by-firmware active c5000/1000 BIOS write-protect fixed-base fixed-length set-by-firmware active c6000/1000 BIOS write-protect fixed-base fixed-length set-by-firmware active c7000/1000 BIOS write-protect fixed-base fixed-length set-by-firmware active c8000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active c9000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active ca000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active cb000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active cc000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active cd000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active ce000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active cf000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active d0000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active d1000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active d2000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active d3000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active d4000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active d5000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active d6000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active d7000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active d8000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active d9000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active da000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active db000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active dc000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active dd000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active de000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active df000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active e0000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active e1000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active e2000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active e3000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active e4000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active e5000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active e6000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active e7000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active e8000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active e9000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active ea000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active eb000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active ec000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active ed000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active ee000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active ef000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active f0000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active f1000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active f2000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active f3000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active f4000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active f5000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active f6000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active f7000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active f8000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active f9000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active fa000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active fb000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active fc000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active fd000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active fe000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active ff000/1000 BIOS uncacheable fixed-base fixed-length set-by-firmware active 0/8000000 BIOS write-back set-by-firmware active --_=XFMail.1.3.p0.FreeBSD:990426004614:369=_-- End of MIME message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Apr 25 8:17:40 1999 Delivered-To: freebsd-smp@freebsd.org Received: from midget.dons.net.au (daniel.lnk.telstra.net [139.130.137.70]) by hub.freebsd.org (Postfix) with ESMTP id B61F4152D6 for ; Sun, 25 Apr 1999 08:17:35 -0700 (PDT) (envelope-from doconnor@gsoft.com.au) Received: from guppy.dons.net.au (guppy.dons.net.au [203.31.81.9]) by midget.dons.net.au (8.9.3/8.9.1) with ESMTP id AAA69385; Mon, 26 Apr 1999 00:47:29 +0930 (CST) (envelope-from doconnor@gsoft.com.au) Message-ID: X-Mailer: XFMail 1.3 [p0] on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <199904231442.HAA00413@dingo.cdrom.com> Date: Mon, 26 Apr 1999 00:47:29 +0930 (CST) From: "Daniel J. O'Connor" To: Mike Smith Subject: Re: Really slow SMP Cc: freebsd-smp@freebsd.org, "Daniel J. O'Connor" , Peter Wemm Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 23-Apr-99 Mike Smith wrote: > My apologies for the scad of these in a row; I'm doing my mail while I > lurk in airports, and it hasn't been fun, I tell you. Huh.. :) > > (There is no MTRR synchronization like there was before. Previously the > > BSP would dump all it's MTRR registers to a table and all the other AP > > cpus would load that table on startup. That table doesn't exist anymore.) > Hmm. It's quite likely that the BIOS is only setting the MTRRs in the > BSP; why aren't the APs doing this anymore? I guess some BIOSen don't set MTRR's on anything but the first CPU.. Maybe time for a BIOS upgrade..? --- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Apr 27 5:30:15 1999 Delivered-To: freebsd-smp@freebsd.org Received: from europa.salford.ac.uk (europa.salford.ac.uk [146.87.3.2]) by hub.freebsd.org (Postfix) with SMTP id C10A814D1D for ; Tue, 27 Apr 1999 05:30:05 -0700 (PDT) (envelope-from M.Choudhury@iti.salford.ac.uk) Received: (qmail 22902 invoked by alias); 27 Apr 1999 12:30:03 -0000 Message-ID: <19990427123003.22901.qmail@europa.salford.ac.uk> Received: (qmail 22896 invoked from network); 27 Apr 1999 12:30:02 -0000 Received: from ashworth-0286.salford.ac.uk (HELO viglen) (146.87.83.30) by europa.salford.ac.uk with SMTP; 27 Apr 1999 12:30:02 -0000 Comments: Authenticated sender is From: M.Choudhury@iti.salford.ac.uk Organization: University of Salford To: freebsd-smp@freebsd.org Date: Tue, 27 Apr 1999 13:32:40 +0100 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Subject: hardware symmetric X-mailer: Pegasus Mail for Win32 (v2.54) Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Please can you tell me what exactly hardware symmetric multiprocessors are? Kind Regards To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Apr 27 12:43:54 1999 Delivered-To: freebsd-smp@freebsd.org Received: from smtp02.primenet.com (smtp02.primenet.com [206.165.6.132]) by hub.freebsd.org (Postfix) with ESMTP id F1C9314DA7 for ; Tue, 27 Apr 1999 12:43:51 -0700 (PDT) (envelope-from tlambert@usr04.primenet.com) Received: (from daemon@localhost) by smtp02.primenet.com (8.8.8/8.8.8) id MAA27037; Tue, 27 Apr 1999 12:43:44 -0700 (MST) Received: from usr04.primenet.com(206.165.6.204) via SMTP by smtp02.primenet.com, id smtpd027007; Tue Apr 27 12:43:40 1999 Received: (from tlambert@localhost) by usr04.primenet.com (8.8.5/8.8.5) id MAA03094; Tue, 27 Apr 1999 12:43:37 -0700 (MST) From: Terry Lambert Message-Id: <199904271943.MAA03094@usr04.primenet.com> Subject: Re: Really slow SMP To: mike@smith.net.au (Mike Smith) Date: Tue, 27 Apr 1999 19:43:37 +0000 (GMT) Cc: peter@netplex.com.au, darius@dons.net.au, freebsd-smp@FreeBSD.ORG In-Reply-To: <199904231442.HAA00413@dingo.cdrom.com> from "Mike Smith" at Apr 23, 99 07:42:17 am X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > (There is no MTRR synchronization like there was before. Previously the > > BSP would dump all it's MTRR registers to a table and all the other AP > > cpus would load that table on startup. That table doesn't exist anymore.) > > Hmm. It's quite likely that the BIOS is only setting the MTRRs in the > BSP; why aren't the APs doing this anymore? Right; this is the crux of the matter: it used to work, now it doesn't. Clearly, the SMP patches are what had this effect; the question is why? Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Apr 27 12:52:21 1999 Delivered-To: freebsd-smp@freebsd.org Received: from dingo.cdrom.com (dingo.cdrom.com [204.216.28.145]) by hub.freebsd.org (Postfix) with ESMTP id 529A615220 for ; Tue, 27 Apr 1999 12:52:07 -0700 (PDT) (envelope-from mike@dingo.cdrom.com) Received: from dingo.cdrom.com (localhost.cdrom.com [127.0.0.1]) by dingo.cdrom.com (8.9.3/8.8.8) with ESMTP id MAA00904; Tue, 27 Apr 1999 12:49:32 -0700 (PDT) (envelope-from mike@dingo.cdrom.com) Message-Id: <199904271949.MAA00904@dingo.cdrom.com> X-Mailer: exmh version 2.0.2 2/24/98 To: Terry Lambert Cc: mike@smith.net.au (Mike Smith), peter@netplex.com.au, darius@dons.net.au, freebsd-smp@FreeBSD.ORG Subject: Re: Really slow SMP In-reply-to: Your message of "Tue, 27 Apr 1999 19:43:37 -0000." <199904271943.MAA03094@usr04.primenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 27 Apr 1999 12:49:32 -0700 From: Mike Smith Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > > (There is no MTRR synchronization like there was before. Previously the > > > BSP would dump all it's MTRR registers to a table and all the other AP > > > cpus would load that table on startup. That table doesn't exist anymore.) > > > > Hmm. It's quite likely that the BIOS is only setting the MTRRs in the > > BSP; why aren't the APs doing this anymore? > > Right; this is the crux of the matter: it used to work, now it doesn't. > > Clearly, the SMP patches are what had this effect; the question is why? I know why; I'm waiting on feedback from peter regarding these patches, but I'd appreciate anyone else seeing the problem to try them too. Also, anyone that can explain why the memory type is printed as "unknown (0x6)" gets a free toy; there are 7 entries in the array. I must be missing something here... Index: i386/i386/i686_mem.c =================================================================== RCS file: /home/ncvs/src/sys/i386/i386/i686_mem.c,v retrieving revision 1.1 diff -u -r1.1 i686_mem.c --- i686_mem.c 1999/04/07 03:57:45 1.1 +++ i686_mem.c 1999/04/26 05:34:23 @@ -62,21 +62,28 @@ #define mrcopyflags(curr, new) (((curr) & ~MDF_ATTRMASK) | ((new) & MDF_ATTRMASK)) -static void i686_mrinit(struct mem_range_softc *); -static int i686_mrset(struct mem_range_softc *, - struct mem_range_desc *, - int *); +static void i686_mrinit(struct mem_range_softc *sc); +static int i686_mrset(struct mem_range_softc *sc, + struct mem_range_desc *mrd, + int *arg); +static void i686_mrAPinit(struct mem_range_softc *sc); static struct mem_range_ops i686_mrops = { i686_mrinit, - i686_mrset + i686_mrset, + i686_mrAPinit }; +/* XXX for AP startup hook */ +extern struct mem_range_softc mem_range_softc; +static u_int64_t mtrrcap, mtrrdef; + static struct mem_range_desc *mem_range_match(struct mem_range_softc *sc, struct mem_range_desc *mrd); static void i686_mrfetch(struct mem_range_softc *sc); static int i686_mtrrtype(int flags); static int i686_mrstore(struct mem_range_softc *sc); +static int i686_mrstoreone(struct mem_range_softc *sc); static struct mem_range_desc *i686_mtrrfixsearch(struct mem_range_softc *sc, u_int64_t addr); static int i686_mrsetlow(struct mem_range_softc *sc, @@ -226,10 +233,6 @@ static int i686_mrstore(struct mem_range_softc *sc) { - struct mem_range_desc *mrd; - u_int64_t msrv; - int i, j, msr; - u_int cr4save; #ifdef SMP /* @@ -241,6 +244,17 @@ return(EOPNOTSUPP); #endif + return(i686_mrstoreone(sc)); +} + +static int +i686_mrstoreone(struct mem_range_softc *sc) +{ + struct mem_range_desc *mrd; + u_int64_t msrv; + int i, j, msr; + u_int cr4save; + disable_intr(); /* disable interrupts */ cr4save = rcr4(); /* save cr4 */ if (cr4save & CR4_PGE) @@ -477,7 +491,6 @@ i686_mrinit(struct mem_range_softc *sc) { struct mem_range_desc *mrd; - u_int64_t mtrrcap, mtrrdef; int nmdesc = 0; int i; @@ -491,8 +504,12 @@ return; } nmdesc = mtrrcap & 0xff; - printf("Pentium Pro MTRR support enabled, default memory type is %s\n", - i686_mtrrtotext[mtrrdef & 0xff]); + printf("Pentium Pro MTRR support enabled, default memory type is "); + if ((mtrrdef & 0xff) < (sizeof(i686_mtrrtotext) / sizeof(i686_mtrrtotext[0]))) { + printf("%s\n", i686_mtrrtotext[mtrrdef & 0xff]); + } else { + printf("unknown (0x%x)\n", mtrrdef & 0xff); + } /* If fixed MTRRs supported and enabled */ if ((mtrrcap & 0x100) && (mtrrdef & 0x400)) { @@ -537,6 +554,17 @@ if (mrd->mr_flags & MDF_ACTIVE) mrd->mr_flags |= MDF_FIRMWARE; } +} + +/* + * Initialise MTRRs on an AP after the BSP has run the init code. + */ +static void +i686_mrAPinit(struct mem_range_softc *sc) +{ + i686_mrstoreone(sc); /* set MTRRs to match BSP */ + wrmsr(MSR_MTRRcap, mtrrcap); /* set MTRR behaviour to match BSP */ + wrmsr(MSR_MTRRdefType, mtrrdef); } static void Index: i386/i386/mem.c =================================================================== RCS file: /home/ncvs/src/sys/i386/i386/mem.c,v retrieving revision 1.55 diff -u -r1.55 mem.c --- mem.c 1999/04/07 03:57:45 1.55 +++ mem.c 1999/04/26 00:46:11 @@ -527,6 +527,12 @@ return(mem_range_softc.mr_op->set(&mem_range_softc, mrd, arg)); } +void +mem_range_AP_init(void) +{ + return(mem_range_softc.mr_op->initAP(&mem_range_softc)); +} + static int random_ioctl(dev, cmd, data, flags, p) dev_t dev; Index: i386/i386/mp_machdep.c =================================================================== RCS file: /home/ncvs/src/sys/i386/i386/mp_machdep.c,v retrieving revision 1.97 diff -u -r1.97 mp_machdep.c --- mp_machdep.c 1999/04/13 03:24:47 1.97 +++ mp_machdep.c 1999/04/26 00:47:20 @@ -496,10 +496,7 @@ PTD[0] = 0; pmap_set_opt((unsigned *)PTD); -#if 0 - putmtrr(); - pmap_setvidram(); -#endif + mem_range_AP_init(); /* copy memory range attributes from the BSP */ invltlb(); } Index: sys/memrange.h =================================================================== RCS file: /home/ncvs/src/sys/sys/memrange.h,v retrieving revision 1.1 diff -u -r1.1 memrange.h --- memrange.h 1999/04/07 03:59:32 1.1 +++ memrange.h 1999/04/26 00:46:51 @@ -47,6 +47,7 @@ { void (*init)(struct mem_range_softc *sc); int (*set)(struct mem_range_softc *sc, struct mem_range_desc *mrd, int *arg); + void (*initAP)(struct mem_range_softc *sc); }; struct mem_range_softc @@ -61,5 +62,6 @@ extern void mem_range_attr_get(struct mem_range_desc *mrd, int *arg); extern int mem_range_attr_set(struct mem_range_desc *mrd, int *arg); +extern void mem_range_AP_init(void); #endif -- \\ Sometimes you're ahead, \\ Mike Smith \\ sometimes you're behind. \\ mike@smith.net.au \\ The race is long, and in the \\ msmith@freebsd.org \\ end it's only with yourself. \\ msmith@cdrom.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Apr 27 16:49:48 1999 Delivered-To: freebsd-smp@freebsd.org Received: from lor.watermarkgroup.com (lor.watermarkgroup.com [207.202.73.33]) by hub.freebsd.org (Postfix) with ESMTP id B0AD7156D5; Tue, 27 Apr 1999 16:49:45 -0700 (PDT) (envelope-from luoqi@watermarkgroup.com) Received: (from luoqi@localhost) by lor.watermarkgroup.com (8.8.8/8.8.8) id TAA28969; Tue, 27 Apr 1999 19:49:44 -0400 (EDT) (envelope-from luoqi) Date: Tue, 27 Apr 1999 19:49:44 -0400 (EDT) From: Luoqi Chen Message-Id: <199904272349.TAA28969@lor.watermarkgroup.com> To: current@freebsd.org, smp@freebsd.org Subject: HEADS UP! to commit SMP vmspace sharing patches Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I'm about to commit the SMP vmspace sharing patch (the %fs approach). All kernel modules will need to be recompiled. Recompilation is not neccessary for user land applications including ps, libkvm and friends. In this %fs approach, per-processor private pages are no longer mapped at identical virtual address for each cpu, instead a new segment descriptor (%fs) is setup to access per-cpu global variables like curproc. As a result the %fs register needs to be saved and restored at the kernel boundary, this would impose a small penalty (cpu model dependent) for each syscall and interrupt. UP kernel will also be affected as efforts were made to ensure portability of kernel modules between UP and SMP architectures. Fast vfork is now possible for SMP and is turned on as default. We're also able to get rid of vmspace juggling kludges in aio code, aio should now work correctly on SMP. -lq To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Apr 27 17:21:33 1999 Delivered-To: freebsd-smp@freebsd.org Received: from cain.gsoft.com.au (genesi.lnk.telstra.net [139.130.136.161]) by hub.freebsd.org (Postfix) with ESMTP id ECFA614A12 for ; Tue, 27 Apr 1999 17:21:29 -0700 (PDT) (envelope-from doconnor@gsoft.com.au) Received: from lot.gsoft.com.au (lot.gsoft.com.au [203.38.152.106]) by cain.gsoft.com.au (8.8.8/8.8.8) with ESMTP id JAA06440; Wed, 28 Apr 1999 09:51:06 +0930 (CST) (envelope-from doconnor@gsoft.com.au) Message-ID: X-Mailer: XFMail 1.3 [p0] on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <199904271949.MAA00904@dingo.cdrom.com> Date: Wed, 28 Apr 1999 09:51:06 +0930 (CST) From: "Daniel O'Connor" To: Mike Smith Subject: Re: Really slow SMP Cc: freebsd-smp@FreeBSD.ORG, peter@netplex.com.au, Terry Lambert Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 27-Apr-99 Mike Smith wrote: > > Clearly, the SMP patches are what had this effect; the question is why? > I know why; I'm waiting on feedback from peter regarding these patches, > but I'd appreciate anyone else seeing the problem to try them too. Hmm.. well they panic'd my machine :( I'll send you a proper error description later, as I only tried the patches before going to work :) Basically it asked if I wanted to panic, so I said no, then it probed, and after that it panic'd and didn't ask.. BTW I applied to patches to a tree checked out on 1999.04.08.12.30 CST because the last time I tried -current it panic'd my machine... --- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Apr 27 22:37: 4 1999 Delivered-To: freebsd-smp@freebsd.org Received: from phk.freebsd.dk (phk.freebsd.dk [212.242.40.153]) by hub.freebsd.org (Postfix) with ESMTP id 95DA014C3B; Tue, 27 Apr 1999 22:37:00 -0700 (PDT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.40.131]) by phk.freebsd.dk (8.9.1/8.8.8) with ESMTP id HAA25819; Wed, 28 Apr 1999 07:36:59 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.9.2/8.9.2) with ESMTP id HAA22150; Wed, 28 Apr 1999 07:36:48 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Luoqi Chen Cc: current@FreeBSD.ORG, smp@FreeBSD.ORG Subject: Re: HEADS UP! to commit SMP vmspace sharing patches In-reply-to: Your message of "Tue, 27 Apr 1999 19:49:44 EDT." <199904272349.TAA28969@lor.watermarkgroup.com> Date: Wed, 28 Apr 1999 07:36:48 +0200 Message-ID: <22148.925277808@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message <199904272349.TAA28969@lor.watermarkgroup.com>, Luoqi Chen writes: >I'm about to commit the SMP vmspace sharing patch (the %fs approach). All >kernel modules will need to be recompiled. Recompilation is not neccessary >for user land applications including ps, libkvm and friends. > >In this %fs approach, per-processor private pages are no longer mapped at >identical virtual address for each cpu, instead a new segment descriptor (%fs) >is setup to access per-cpu global variables like curproc. How is this accessed from C sources ? -- Poul-Henning Kamp FreeBSD coreteam member phk@FreeBSD.ORG "Real hackers run -current on their laptop." FreeBSD -- It will take a long time before progress goes too far! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Apr 28 9:25:32 1999 Delivered-To: freebsd-smp@freebsd.org Received: from lor.watermarkgroup.com (lor.watermarkgroup.com [207.202.73.33]) by hub.freebsd.org (Postfix) with ESMTP id EEE7D14E94; Wed, 28 Apr 1999 09:25:29 -0700 (PDT) (envelope-from luoqi@watermarkgroup.com) Received: (from luoqi@localhost) by lor.watermarkgroup.com (8.8.8/8.8.8) id MAA05043; Wed, 28 Apr 1999 12:25:28 -0400 (EDT) (envelope-from luoqi) Date: Wed, 28 Apr 1999 12:25:28 -0400 (EDT) From: Luoqi Chen Message-Id: <199904281625.MAA05043@lor.watermarkgroup.com> To: luoqi@watermarkgroup.com, phk@critter.freebsd.dk Subject: Re: HEADS UP! to commit SMP vmspace sharing patches Cc: current@FreeBSD.ORG, smp@FreeBSD.ORG Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > In message <199904272349.TAA28969@lor.watermarkgroup.com>, Luoqi Chen writes: > >I'm about to commit the SMP vmspace sharing patch (the %fs approach). All > >kernel modules will need to be recompiled. Recompilation is not neccessary > >for user land applications including ps, libkvm and friends. > > > >In this %fs approach, per-processor private pages are no longer mapped at > >identical virtual address for each cpu, instead a new segment descriptor (%fs) > >is setup to access per-cpu global variables like curproc. > > How is this accessed from C sources ? > curproc is now a macro defined as an inline asm function. > -- > Poul-Henning Kamp FreeBSD coreteam member > phk@FreeBSD.ORG "Real hackers run -current on their laptop." > FreeBSD -- It will take a long time before progress goes too far! > -lq To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Apr 28 11: 0:48 1999 Delivered-To: freebsd-smp@freebsd.org Received: from phk.freebsd.dk (phk.freebsd.dk [212.242.40.153]) by hub.freebsd.org (Postfix) with ESMTP id 0A5C914D80; Wed, 28 Apr 1999 11:00:41 -0700 (PDT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.40.131]) by phk.freebsd.dk (8.9.1/8.8.8) with ESMTP id UAA28701; Wed, 28 Apr 1999 20:00:40 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.9.2/8.9.2) with ESMTP id UAA24331; Wed, 28 Apr 1999 20:00:26 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Luoqi Chen Cc: current@FreeBSD.ORG, smp@FreeBSD.ORG Subject: Re: HEADS UP! to commit SMP vmspace sharing patches In-reply-to: Your message of "Wed, 28 Apr 1999 12:25:28 EDT." <199904281625.MAA05043@lor.watermarkgroup.com> Date: Wed, 28 Apr 1999 20:00:26 +0200 Message-ID: <24329.925322426@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message <199904281625.MAA05043@lor.watermarkgroup.com>, Luoqi Chen writes: >> >In this %fs approach, per-processor private pages are no longer mapped at >> >identical virtual address for each cpu, instead a new segment descriptor (%fs) >> >is setup to access per-cpu global variables like curproc. >> >> How is this accessed from C sources ? >> >curproc is now a macro defined as an inline asm function. Not that I really miss far pointers, but right now we could use one... -- Poul-Henning Kamp FreeBSD coreteam member phk@FreeBSD.ORG "Real hackers run -current on their laptop." FreeBSD -- It will take a long time before progress goes too far! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Apr 28 11: 8:23 1999 Delivered-To: freebsd-smp@freebsd.org Received: from picnic.mat.net (picnic.mat.net [206.246.122.133]) by hub.freebsd.org (Postfix) with ESMTP id AAF09151D1; Wed, 28 Apr 1999 11:08:14 -0700 (PDT) (envelope-from chuckr@picnic.mat.net) Received: from localhost (chuckr@localhost) by picnic.mat.net (8.9.3/8.8.5) with ESMTP id OAA19247; Wed, 28 Apr 1999 14:02:01 -0400 (EDT) Date: Wed, 28 Apr 1999 14:02:01 -0400 (EDT) From: Chuck Robey To: Poul-Henning Kamp Cc: Luoqi Chen , current@FreeBSD.ORG, smp@FreeBSD.ORG Subject: Re: HEADS UP! to commit SMP vmspace sharing patches In-Reply-To: <24329.925322426@critter.freebsd.dk> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Wed, 28 Apr 1999, Poul-Henning Kamp wrote: > In message <199904281625.MAA05043@lor.watermarkgroup.com>, Luoqi Chen writes: > > >> >In this %fs approach, per-processor private pages are no longer mapped at > >> >identical virtual address for each cpu, instead a new segment descriptor (%fs) > >> >is setup to access per-cpu global variables like curproc. > >> > >> How is this accessed from C sources ? > >> > >curproc is now a macro defined as an inline asm function. > > Not that I really miss far pointers, but right now we could use one... No profanity on this list (and Poul, far-pointers were very definitely profane!) Go wash out your mouth! ----------------------------+----------------------------------------------- Chuck Robey | Interests include any kind of voice or data chuckr@picnic.mat.net | communications topic, C programming, and Unix. 213 Lakeside Drive Apt T-1 | Greenbelt, MD 20770 | I run picnic (FreeBSD-current) (301) 220-2114 | and jaunt (Solaris7). ----------------------------+----------------------------------------------- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Apr 28 11:19:26 1999 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [209.157.86.2]) by hub.freebsd.org (Postfix) with ESMTP id B98A71521D; Wed, 28 Apr 1999 11:19:24 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id LAA07937; Wed, 28 Apr 1999 11:19:17 -0700 (PDT) (envelope-from dillon) Date: Wed, 28 Apr 1999 11:19:17 -0700 (PDT) From: Matthew Dillon Message-Id: <199904281819.LAA07937@apollo.backplane.com> To: Chuck Robey Cc: Poul-Henning Kamp , Luoqi Chen , current@FreeBSD.ORG, smp@FreeBSD.ORG Subject: Re: HEADS UP! to commit SMP vmspace sharing patches References: Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I know this is a little late ... but I don't suppose there might be a way to lock a TLB entry in place? That would solve the problem too. Baring that, %fs is the way to go. I am happily compiling up a new SMP kernel as we speak :-). -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Apr 28 13:15:17 1999 Delivered-To: freebsd-smp@freebsd.org Received: from cs.rice.edu (cs.rice.edu [128.42.1.30]) by hub.freebsd.org (Postfix) with ESMTP id E7B351577D; Wed, 28 Apr 1999 13:15:01 -0700 (PDT) (envelope-from alc@cs.rice.edu) Received: from nonpc.cs.rice.edu (nonpc.cs.rice.edu [128.42.1.219]) by cs.rice.edu (8.9.0/8.9.0) with ESMTP id PAA24466; Wed, 28 Apr 1999 15:14:54 -0500 (CDT) Received: (from alc@localhost) by nonpc.cs.rice.edu (8.9.2/8.7.3) id PAA15451; Wed, 28 Apr 1999 15:14:54 -0500 (CDT) Date: Wed, 28 Apr 1999 15:14:54 -0500 From: Alan Cox To: Matthew Dillon Cc: Chuck Robey , Poul-Henning Kamp , Luoqi Chen , current@freebsd.org, smp@freebsd.org Subject: Re: HEADS UP! to commit SMP vmspace sharing patches Message-ID: <19990428151454.O1121@nonpc.cs.rice.edu> References: <199904281819.LAA07937@apollo.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.1i In-Reply-To: <199904281819.LAA07937@apollo.backplane.com>; from Matthew Dillon on Wed, Apr 28, 1999 at 11:19:17AM -0700 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Wed, Apr 28, 1999 at 11:19:17AM -0700, Matthew Dillon wrote: > I know this is a little late ... but I don't suppose there might be a > way to lock a TLB entry in place? That would solve the problem too. > Baring that, %fs is the way to go. > Unfortunately, on the x86, the answer is "No." The only serious alternative was to put the commonly used per processor variables and a pointer to the less commonly used ones at the base of each process's/thread's kernel stack, i.e., the upages, where you could mask off bits from the stack pointer to arrive at the correct address. You'd then have to "refresh" most of these variables on a context switch (in case the process migrated). Alan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Apr 28 14:49: 5 1999 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [209.157.86.2]) by hub.freebsd.org (Postfix) with ESMTP id 0BDDD15520; Wed, 28 Apr 1999 14:48:58 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id OAA09354; Wed, 28 Apr 1999 14:48:56 -0700 (PDT) (envelope-from dillon) Date: Wed, 28 Apr 1999 14:48:56 -0700 (PDT) From: Matthew Dillon Message-Id: <199904282148.OAA09354@apollo.backplane.com> To: Alan Cox Cc: Chuck Robey , Poul-Henning Kamp , Luoqi Chen , current@freebsd.org, smp@freebsd.org Subject: Re: HEADS UP! to commit SMP vmspace sharing patches References: <199904281819.LAA07937@apollo.backplane.com> <19990428151454.O1121@nonpc.cs.rice.edu> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org :On Wed, Apr 28, 1999 at 11:19:17AM -0700, Matthew Dillon wrote: :> I know this is a little late ... but I don't suppose there might be a :> way to lock a TLB entry in place? That would solve the problem too. :> Baring that, %fs is the way to go. :> : :Unfortunately, on the x86, the answer is "No." The only serious :alternative was to put the commonly used per processor variables :and a pointer to the less commonly used ones at the base of each :process's/thread's kernel stack, i.e., the upages, where you could :mask off bits from the stack pointer to arrive at the correct address. :You'd then have to "refresh" most of these variables on a context switch :(in case the process migrated). : : Alan Too bad. There might be less confusion with %fs if we simply use it as a 'cpu number' index and then make all the cpu-dependant variables standard arrays. i.e. instead if 'struct proc *curproc' we would have 'struct proc *curproc[NCPU];'. The assembly macro would simply retrieive the current cpu number from %fs, so: curproc[MYCPU] = ... This would be much less confusing then trying to encapsulate the concept of 'curproc', and we could do away with cpu-specific VM areas entirely. Sure, it would eat a few more cycles, but I don't think it would effect performance. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Apr 28 16: 0:33 1999 Delivered-To: freebsd-smp@freebsd.org Received: from cs.rice.edu (cs.rice.edu [128.42.1.30]) by hub.freebsd.org (Postfix) with ESMTP id 74C4D14EC4; Wed, 28 Apr 1999 16:00:22 -0700 (PDT) (envelope-from alc@cs.rice.edu) Received: from nonpc.cs.rice.edu (nonpc.cs.rice.edu [128.42.1.219]) by cs.rice.edu (8.9.0/8.9.0) with ESMTP id SAA29228; Wed, 28 Apr 1999 18:00:21 -0500 (CDT) Received: (from alc@localhost) by nonpc.cs.rice.edu (8.9.2/8.7.3) id SAA15816; Wed, 28 Apr 1999 18:00:21 -0500 (CDT) Date: Wed, 28 Apr 1999 18:00:20 -0500 From: Alan Cox To: Matthew Dillon Cc: Chuck Robey , Poul-Henning Kamp , Luoqi Chen , current@freebsd.org, smp@freebsd.org Subject: Re: HEADS UP! to commit SMP vmspace sharing patches Message-ID: <19990428180020.P1121@nonpc.cs.rice.edu> References: <199904281819.LAA07937@apollo.backplane.com> <19990428151454.O1121@nonpc.cs.rice.edu> <199904282148.OAA09354@apollo.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.1i In-Reply-To: <199904282148.OAA09354@apollo.backplane.com>; from Matthew Dillon on Wed, Apr 28, 1999 at 02:48:56PM -0700 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Wed, Apr 28, 1999 at 02:48:56PM -0700, Matthew Dillon wrote: > > ... > > There might be less confusion with %fs if we simply use it as a > 'cpu number' index and then make all the cpu-dependant variables > standard arrays. i.e. instead if 'struct proc *curproc' we would > have 'struct proc *curproc[NCPU];'. The assembly macro would > simply retrieive the current cpu number from %fs, so: > > curproc[MYCPU] = ... > > This would be much less confusing then trying to encapsulate the concept > of 'curproc', and we could do away with cpu-specific VM areas entirely. > > Sure, it would eat a few more cycles, but I don't think it would effect > performance. > I don't think the current approach with %fs is that confusing. :-) You can view it as an optimization of struct "per processor data" { struct proc *curproc; ... } ppd[NCPUS]; some_func() { ... ppd[MYCPU]->curproc In some sense, the "ppd[MYCPU]" is precomputed in %fs. Also, I would discourage a "per-variable" approach like struct proc *curproc[NCPU]; This will lead to unnecessary cache coherence traffic (due to false sharing). For example, when processor 0 updates curproc[0] it will cause the invalidation of the cache line containing curproc on processor 1, and vice versa when processor 1 updates curproc[1]. Instead, it's better to aggregate each processor's per-processor data like our current code does. Alan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Apr 28 16:45: 0 1999 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [209.157.86.2]) by hub.freebsd.org (Postfix) with ESMTP id 48B1E14CEE; Wed, 28 Apr 1999 16:44:54 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id QAA10391; Wed, 28 Apr 1999 16:44:52 -0700 (PDT) (envelope-from dillon) Date: Wed, 28 Apr 1999 16:44:52 -0700 (PDT) From: Matthew Dillon Message-Id: <199904282344.QAA10391@apollo.backplane.com> To: Alan Cox Cc: Chuck Robey , Poul-Henning Kamp , Luoqi Chen , current@FreeBSD.ORG, smp@FreeBSD.ORG Subject: Re: HEADS UP! to commit SMP vmspace sharing patches References: <199904281819.LAA07937@apollo.backplane.com> <19990428151454.O1121@nonpc.cs.rice.edu> <199904282148.OAA09354@apollo.backplane.com> <19990428180020.P1121@nonpc.cs.rice.edu> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org :I don't think the current approach with %fs is that confusing. :-) You :can view it as an optimization of : : struct "per processor data" { : struct proc *curproc; : ... : } ppd[NCPUS]; : : some_func() : { : ... ppd[MYCPU]->curproc : :In some sense, the "ppd[MYCPU]" is precomputed in %fs. : :Also, I would discourage a "per-variable" approach like : : struct proc *curproc[NCPU]; : :This will lead to unnecessary cache coherence traffic (due to false :sharing). For example, when processor 0 updates curproc[0] it will :cause the invalidation of the cache line containing curproc on processor 1, :and vice versa when processor 1 updates curproc[1]. Instead, it's better :to aggregate each processor's per-processor data like our current :code does. : :Alan Quite true. But, in that case, the equivalent of struct perprocess { struct process *curproc; ... } perproc[NCPU]; While it is true that you then have to draw out the access: perproc[MYCPU].curproc Or perhaps ( even better ): MYCPU->curproc It would make the code much more readable then trying to 'hide' the fact that curproc ( and other variables ) are actually segmented. We have to keep in mind the fact that SMP is only just now gaining momentum, and I think a considerable amount of additional data and structure is going to be added to the per-cpu structures in the next few years as we begin to parallelize kernel operations. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Apr 29 8:13:28 1999 Delivered-To: freebsd-smp@freebsd.org Received: from midget.dons.net.au (daniel.lnk.telstra.net [139.130.137.70]) by hub.freebsd.org (Postfix) with ESMTP id 4DC7B14F2F for ; Thu, 29 Apr 1999 08:13:18 -0700 (PDT) (envelope-from darius@dons.net.au) Received: from guppy.dons.net.au (guppy.dons.net.au [203.31.81.9]) by midget.dons.net.au (8.9.3/8.9.1) with ESMTP id AAA76293; Fri, 30 Apr 1999 00:43:06 +0930 (CST) (envelope-from darius@dons.net.au) Message-ID: X-Mailer: XFMail 1.3 [p0] on FreeBSD X-Priority: 3 (Normal) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="_=XFMail.1.3.p0.FreeBSD:990430004306:372=_" In-Reply-To: <199904271949.MAA00904@dingo.cdrom.com> Date: Fri, 30 Apr 1999 00:43:06 +0930 (CST) From: "Daniel J. O'Connor" To: Mike Smith Subject: Re: Really slow SMP Cc: freebsd-smp@FreeBSD.ORG, peter@netplex.com.au, Terry Lambert Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org This message is in MIME format --_=XFMail.1.3.p0.FreeBSD:990430004306:372=_ Content-Type: text/plain; charset=us-ascii On 27-Apr-99 Mike Smith wrote: > I know why; I'm waiting on feedback from peter regarding these patches, > but I'd appreciate anyone else seeing the problem to try them too. Well, without the patches I get a panic like so -> ... APIC_IO: routing 8254 via pin 2 Trap 12 fault virtual address = 0x0 fault code = supervisor read, page not present instruction pointer = 0x8:0x0 stack pointer = 0x10:0xc0329f8c frame pointer = 0x10:0xc0329f98 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupts enabled, resume, IOPL = 0 current process = 0 (swapper) interrupt mask = <- SMP: XXX trap number = 12 panic: page fault mp_lock = 0000001d; cpuid = 0; lapic.id = 0 WITH the patch I get -> avail mem = ... Programming 24 pins in IOAPIC #0 AP#1 (PHY #1) failed! panic y/n? [y] (choose n) ... ... panic's same as before. --- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum --_=XFMail.1.3.p0.FreeBSD:990430004306:372=_ Content-Disposition: attachment; filename="GUPPY" Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii; name=GUPPY; SizeOnDisk=3535 machine "i386" cpu "I686_CPU" ident TEST maxusers 32 options INCLUDE_CONFIG_FILE # Include this file in kernel options USERCONFIG #boot -c editor options VISUAL_USERCONFIG #visual boot -c editor options INET #InterNETworking options FFS #Berkeley Fast Filesystem options NFS #Network Filesystem #options "CD9660" #ISO 9660 Filesystem options PROCFS #Process filesystem options "COMPAT_43" #Compatible with BSD 4.3 [KEEP THIS!] options UCONSOLE #Allow users to grab the console options KTRACE #kernel tracing options SYSVSHM options SYSVSEM options SYSVMSG options DDB options DDB_UNATTENDED #options BOUNCE_BUFFERS #For ADV controller options CONSPEED=115200 #default speed for serial console options "MSGBUF_SIZE=32768" options SOFTUPDATES options SMP # Symmetric MultiProcessor Kernel options APIC_IO # Symmetric (APIC) I/O config kernel root on wd0 dumps on wd0 controller isa0 controller pci0 controller fdc0 at isa? port "IO_FD1" bio irq 6 drq 2 vector fdintr disk fd0 at fdc0 drive 0 disk fd1 at fdc0 drive 1 controller wdc0 at isa? port "IO_WD1" bio irq 14 flags 0xa0ffa0ff vector wdintr disk wd0 at wdc0 drive 0 disk wd1 at wdc0 drive 1 controller wdc1 at isa? port "IO_WD2" bio irq 15 flags 0xa0ffa0ff vector wdintr disk wd2 at wdc1 drive 0 disk wd3 at wdc1 drive 1 device wcd0 options IDE_DELAY=2000 # Be optimistic about Joe IDE device options "VM86" options VESA # atkbdc0 controlls both the keyboard and the PS/2 mouse controller atkbdc0 at isa? port IO_KBD tty device psm0 at isa? tty irq 12 device atkbd0 at isa? tty irq 1 device vga0 at isa? port ? conflicts # splash screen/screen saver pseudo-device splash # syscons is the default console driver, resembling an SCO console device sc0 at isa? tty # Mandatory, don't remove device npx0 at isa? port "IO_NPX" irq 13 vector npxintr device sio0 at isa? port "IO_COM1" tty irq 4 vector siointr device sio1 at isa? port "IO_COM2" tty irq 3 vector siointr device ed0 at isa? port 0x280 net irq 5 iomem 0xd8000 controller ppbus0 device nlpt0 at ppbus? device plip0 at ppbus? device ppi0 at ppbus? device pps0 at ppbus? device lpbb0 at ppbus? controller ppc0 at isa? disable port ? tty irq 7 device de0 device fxp0 controller pnp0 controller snd0 device sb0 at isa? port 0x220 irq 5 drq 1 vector sbintr device sbxvi0 at isa? drq 5 device sbmidi0 at isa? port 0x330 device opl0 at isa? port 0x388 device awe0 at isa? port 0x620 device joy0 at isa? port "IO_GAME" device apm0 at isa? conflicts controller smbus0 controller intpm0 device smb0 at smbus? controller iicbus0 controller iicbb0 device ic0 at iicbus? device iic0 at iicbus? device iicsmb0 at iicbus? controller ahc0 controller scbus0 #base SCSI code device ch0 #SCSI media changers device da0 #SCSI direct access devices (aka disks) device sa0 #SCSI tapes device cd0 #SCSI CD-ROMs device od0 #SCSI optical disk device pass0 #CAM passthrough driver pseudo-device loop pseudo-device ether pseudo-device pty 16 pseudo-device bpfilter 4 pseudo-device gzip # Exec gzipped a.out's pseudo-device ccd 4 #Concatenated disk driver pseudo-device vn --_=XFMail.1.3.p0.FreeBSD:990430004306:372=_ Content-Disposition: attachment; filename="mptable" Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii; name=mptable; SizeOnDisk=3020 =============================================================================== MPTable, version 2.0.15 ------------------------------------------------------------------------------- MP Floating Pointer Structure: location: BIOS physical address: 0x000f5630 signature: '_MP_' length: 16 bytes version: 1.1 checksum: 0x80 mode: Virtual Wire ------------------------------------------------------------------------------- MP Config Table Header: physical address: 0x000f1400 signature: 'PCMP' base table length: 292 version: 1.1 checksum: 0x5e OEM ID: 'OEM00000' Product ID: 'PROD00000000' OEM table pointer: 0x00000000 OEM table size: 0 entry count: 28 local APIC address: 0xfee00000 extended table length: 0 extended table checksum: 0 ------------------------------------------------------------------------------- MP Config Base Table Entries: -- Processors: APIC ID Version State Family Model Step Flags 0 0x11 BSP, usable 6 5 3 0xfbff 1 0x11 AP, usable 6 5 3 0xfbff -- Bus: Bus ID Type 0 PCI 1 PCI 2 ISA -- I/O APICs: APIC ID Version State Address 2 0x11 usable 0xfec00000 -- I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN# ExtINT conforms conforms 2 0 2 0 INT conforms conforms 2 1 2 1 INT conforms conforms 2 0 2 2 INT conforms conforms 2 3 2 3 INT conforms conforms 2 4 2 4 INT conforms conforms 2 5 2 5 INT conforms conforms 2 6 2 6 INT conforms conforms 2 7 2 7 INT active-hi edge 2 8 2 8 INT conforms conforms 2 9 2 9 INT conforms conforms 2 10 2 10 INT conforms conforms 2 11 2 11 INT conforms conforms 2 12 2 12 INT conforms conforms 2 13 2 13 INT conforms conforms 2 14 2 14 INT conforms conforms 2 15 2 15 INT active-lo level 0 7:A 2 19 INT active-lo level 0 9:A 2 17 INT active-lo level 0 11:A 2 19 SMI conforms conforms 2 0 2 23 -- Local Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN# ExtINT conforms conforms 0 0:A 255 0 NMI conforms conforms 0 0:A 255 1 ------------------------------------------------------------------------------- # SMP kernel config file options: # Required: options SMP # Symmetric MultiProcessor Kernel options APIC_IO # Symmetric (APIC) I/O # Optional (built-in defaults will work in most cases): #options NCPU=2 # number of CPUs #options NBUS=3 # number of busses #options NAPIC=1 # number of IO APICs #options NINTR=24 # number of INTs =============================================================================== --_=XFMail.1.3.p0.FreeBSD:990430004306:372=_-- End of MIME message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Apr 29 10:14: 0 1999 Delivered-To: freebsd-smp@freebsd.org Received: from dingo.cdrom.com (castles545.castles.com [208.214.165.109]) by hub.freebsd.org (Postfix) with ESMTP id C24D314F94 for ; Thu, 29 Apr 1999 10:13:51 -0700 (PDT) (envelope-from mike@dingo.cdrom.com) Received: from dingo.cdrom.com (localhost [127.0.0.1]) by dingo.cdrom.com (8.9.3/8.8.8) with ESMTP id KAA02932; Thu, 29 Apr 1999 10:12:30 -0700 (PDT) (envelope-from mike@dingo.cdrom.com) Message-Id: <199904291712.KAA02932@dingo.cdrom.com> X-Mailer: exmh version 2.0.2 2/24/98 To: "Daniel J. O'Connor" Cc: Mike Smith , freebsd-smp@FreeBSD.ORG, peter@netplex.com.au, Terry Lambert Subject: Re: Really slow SMP In-reply-to: Your message of "Fri, 30 Apr 1999 00:43:06 +0930." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 29 Apr 1999 10:12:29 -0700 From: Mike Smith Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Symbols, grasshopper. Also, how -current are you? > This message is in MIME format > --_=XFMail.1.3.p0.FreeBSD:990430004306:372=_ > Content-Type: text/plain; charset=us-ascii > > > On 27-Apr-99 Mike Smith wrote: > > I know why; I'm waiting on feedback from peter regarding these patches, > > but I'd appreciate anyone else seeing the problem to try them too. > Well, without the patches I get a panic like so -> > ... > APIC_IO: routing 8254 via pin 2 > Trap 12 > fault virtual address = 0x0 > fault code = supervisor read, page not present > instruction pointer = 0x8:0x0 > stack pointer = 0x10:0xc0329f8c > frame pointer = 0x10:0xc0329f98 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, def32 1, gran 1 > processor eflags = interrupts enabled, resume, IOPL = 0 > current process = 0 (swapper) > interrupt mask = <- SMP: XXX > trap number = 12 > panic: page fault > mp_lock = 0000001d; cpuid = 0; lapic.id = 0 > > WITH the patch I get -> > avail mem = ... > Programming 24 pins in IOAPIC #0 > AP#1 (PHY #1) failed! > panic y/n? [y] (choose n) > ... > ... > > panic's same as before. > > --- > Daniel O'Connor software and network engineer > for Genesis Software - http://www.gsoft.com.au > "The nice thing about standards is that there > are so many of them to choose from." > -- Andrew Tanenbaum > > > > --_=XFMail.1.3.p0.FreeBSD:990430004306:372=_ > Content-Disposition: attachment; filename="GUPPY" > Content-Transfer-Encoding: 7bit > Content-Type: text/plain; charset=us-ascii; name=GUPPY; SizeOnDisk=3535 > > machine "i386" > cpu "I686_CPU" > ident TEST > maxusers 32 > > options INCLUDE_CONFIG_FILE # Include this file in kernel > > options USERCONFIG #boot -c editor > options VISUAL_USERCONFIG #visual boot -c editor > > options INET #InterNETworking > options FFS #Berkeley Fast Filesystem > options NFS #Network Filesystem > #options "CD9660" #ISO 9660 Filesystem > options PROCFS #Process filesystem > options "COMPAT_43" #Compatible with BSD 4.3 [KEEP THIS!] > options UCONSOLE #Allow users to grab the console > options KTRACE #kernel tracing > > options SYSVSHM > options SYSVSEM > options SYSVMSG > > options DDB > options DDB_UNATTENDED > > #options BOUNCE_BUFFERS #For ADV controller > options CONSPEED=115200 #default speed for serial console > options "MSGBUF_SIZE=32768" > > options SOFTUPDATES > > options SMP # Symmetric MultiProcessor Kernel > options APIC_IO # Symmetric (APIC) I/O > > config kernel root on wd0 dumps on wd0 > > controller isa0 > controller pci0 > > controller fdc0 at isa? port "IO_FD1" bio irq 6 drq 2 vector fdintr > disk fd0 at fdc0 drive 0 > disk fd1 at fdc0 drive 1 > > controller wdc0 at isa? port "IO_WD1" bio irq 14 flags 0xa0ffa0ff vector wdintr > disk wd0 at wdc0 drive 0 > disk wd1 at wdc0 drive 1 > controller wdc1 at isa? port "IO_WD2" bio irq 15 flags 0xa0ffa0ff vector wdintr > disk wd2 at wdc1 drive 0 > disk wd3 at wdc1 drive 1 > > device wcd0 > > options IDE_DELAY=2000 # Be optimistic about Joe IDE device > > options "VM86" > options VESA > > # atkbdc0 controlls both the keyboard and the PS/2 mouse > controller atkbdc0 at isa? port IO_KBD tty > device psm0 at isa? tty irq 12 > device atkbd0 at isa? tty irq 1 > > device vga0 at isa? port ? conflicts > > # splash screen/screen saver > pseudo-device splash > > # syscons is the default console driver, resembling an SCO console > device sc0 at isa? tty > > # Mandatory, don't remove > device npx0 at isa? port "IO_NPX" irq 13 vector npxintr > > device sio0 at isa? port "IO_COM1" tty irq 4 vector siointr > device sio1 at isa? port "IO_COM2" tty irq 3 vector siointr > > device ed0 at isa? port 0x280 net irq 5 iomem 0xd8000 > > controller ppbus0 > device nlpt0 at ppbus? > device plip0 at ppbus? > device ppi0 at ppbus? > device pps0 at ppbus? > device lpbb0 at ppbus? > > controller ppc0 at isa? disable port ? tty irq 7 > > device de0 > device fxp0 > > controller pnp0 > > controller snd0 > device sb0 at isa? port 0x220 irq 5 drq 1 vector sbintr > device sbxvi0 at isa? drq 5 > device sbmidi0 at isa? port 0x330 > device opl0 at isa? port 0x388 > device awe0 at isa? port 0x620 > device joy0 at isa? port "IO_GAME" > > device apm0 at isa? conflicts > > controller smbus0 > > controller intpm0 > > device smb0 at smbus? > > controller iicbus0 > controller iicbb0 > > device ic0 at iicbus? > device iic0 at iicbus? > device iicsmb0 at iicbus? > > controller ahc0 > controller scbus0 #base SCSI code > device ch0 #SCSI media changers > device da0 #SCSI direct access devices (aka disks) > device sa0 #SCSI tapes > device cd0 #SCSI CD-ROMs > device od0 #SCSI optical disk > device pass0 #CAM passthrough driver > > > pseudo-device loop > pseudo-device ether > pseudo-device pty 16 > pseudo-device bpfilter 4 > pseudo-device gzip # Exec gzipped a.out's > pseudo-device ccd 4 #Concatenated disk driver > pseudo-device vn > > > > --_=XFMail.1.3.p0.FreeBSD:990430004306:372=_ > Content-Disposition: attachment; filename="mptable" > Content-Transfer-Encoding: 7bit > Content-Type: text/plain; charset=us-ascii; name=mptable; SizeOnDisk=3020 > > > =============================================================================== > > MPTable, version 2.0.15 > > ------------------------------------------------------------------------------- > > MP Floating Pointer Structure: > > location: BIOS > physical address: 0x000f5630 > signature: '_MP_' > length: 16 bytes > version: 1.1 > checksum: 0x80 > mode: Virtual Wire > > ------------------------------------------------------------------------------- > > MP Config Table Header: > > physical address: 0x000f1400 > signature: 'PCMP' > base table length: 292 > version: 1.1 > checksum: 0x5e > OEM ID: 'OEM00000' > Product ID: 'PROD00000000' > OEM table pointer: 0x00000000 > OEM table size: 0 > entry count: 28 > local APIC address: 0xfee00000 > extended table length: 0 > extended table checksum: 0 > > ------------------------------------------------------------------------------- > > MP Config Base Table Entries: > > -- > Processors: APIC ID Version State Family Model Step Flags > 0 0x11 BSP, usable 6 5 3 0xfbff > 1 0x11 AP, usable 6 5 3 0xfbff > -- > Bus: Bus ID Type > 0 PCI > 1 PCI > 2 ISA > -- > I/O APICs: APIC ID Version State Address > 2 0x11 usable 0xfec00000 > -- > I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN# > ExtINT conforms conforms 2 0 2 0 > INT conforms conforms 2 1 2 1 > INT conforms conforms 2 0 2 2 > INT conforms conforms 2 3 2 3 > INT conforms conforms 2 4 2 4 > INT conforms conforms 2 5 2 5 > INT conforms conforms 2 6 2 6 > INT conforms conforms 2 7 2 7 > INT active-hi edge 2 8 2 8 > INT conforms conforms 2 9 2 9 > INT conforms conforms 2 10 2 10 > INT conforms conforms 2 11 2 11 > INT conforms conforms 2 12 2 12 > INT conforms conforms 2 13 2 13 > INT conforms conforms 2 14 2 14 > INT conforms conforms 2 15 2 15 > INT active-lo level 0 7:A 2 19 > INT active-lo level 0 9:A 2 17 > INT active-lo level 0 11:A 2 19 > SMI conforms conforms 2 0 2 23 > -- > Local Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN# > ExtINT conforms conforms 0 0:A 255 0 > NMI conforms conforms 0 0:A 255 1 > > ------------------------------------------------------------------------------- > > # SMP kernel config file options: > > > # Required: > options SMP # Symmetric MultiProcessor Kernel > options APIC_IO # Symmetric (APIC) I/O > > # Optional (built-in defaults will work in most cases): > #options NCPU=2 # number of CPUs > #options NBUS=3 # number of busses > #options NAPIC=1 # number of IO APICs > #options NINTR=24 # number of INTs > > =============================================================================== > > > --_=XFMail.1.3.p0.FreeBSD:990430004306:372=_-- > End of MIME message -- \\ Sometimes you're ahead, \\ Mike Smith \\ sometimes you're behind. \\ mike@smith.net.au \\ The race is long, and in the \\ msmith@freebsd.org \\ end it's only with yourself. \\ msmith@cdrom.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Apr 29 10:36:27 1999 Delivered-To: freebsd-smp@freebsd.org Received: from lor.watermarkgroup.com (lor.watermarkgroup.com [207.202.73.33]) by hub.freebsd.org (Postfix) with ESMTP id 858F414BFC for ; Thu, 29 Apr 1999 10:36:17 -0700 (PDT) (envelope-from luoqi@watermarkgroup.com) Received: (from luoqi@localhost) by lor.watermarkgroup.com (8.8.8/8.8.8) id NAA17634; Thu, 29 Apr 1999 13:35:16 -0400 (EDT) (envelope-from luoqi) Date: Thu, 29 Apr 1999 13:35:16 -0400 (EDT) From: Luoqi Chen Message-Id: <199904291735.NAA17634@lor.watermarkgroup.com> To: darius@dons.net.au, mike@smith.net.au Subject: Re: Really slow SMP Cc: freebsd-smp@FreeBSD.ORG, peter@netplex.com.au, tlambert@primenet.com Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Maybe init_secondary() is too ealier for calling mem_range_AP_init, APs shouldn't be fooling around with locks at that point. I guess what happened was the AP was in a spin loop waiting for a lock and BSP timed out waiting for AP's up signal. Try move the call to ap_init() instead. -lq To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Apr 29 10:42:32 1999 Delivered-To: freebsd-smp@freebsd.org Received: from dingo.cdrom.com (castles545.castles.com [208.214.165.109]) by hub.freebsd.org (Postfix) with ESMTP id 6352115042 for ; Thu, 29 Apr 1999 10:42:19 -0700 (PDT) (envelope-from mike@dingo.cdrom.com) Received: from dingo.cdrom.com (localhost [127.0.0.1]) by dingo.cdrom.com (8.9.3/8.8.8) with ESMTP id KAA03121; Thu, 29 Apr 1999 10:41:01 -0700 (PDT) (envelope-from mike@dingo.cdrom.com) Message-Id: <199904291741.KAA03121@dingo.cdrom.com> X-Mailer: exmh version 2.0.2 2/24/98 To: Luoqi Chen Cc: darius@dons.net.au, mike@smith.net.au, freebsd-smp@FreeBSD.ORG, peter@netplex.com.au, tlambert@primenet.com Subject: Re: Really slow SMP In-reply-to: Your message of "Thu, 29 Apr 1999 13:35:16 EDT." <199904291735.NAA17634@lor.watermarkgroup.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 29 Apr 1999 10:41:01 -0700 From: Mike Smith Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Maybe init_secondary() is too ealier for calling mem_range_AP_init, > APs shouldn't be fooling around with locks at that point. I guess > what happened was the AP was in a spin loop waiting for a lock and > BSP timed out waiting for AP's up signal. Try move the call to ap_init() > instead. Sorry, not sure I follow you here; there's no locking in mem_range_AP_init(), and it's where the MTRRs were being loaded before. The code path is a little more convoluted now, but has the same basic effect. Regardless, Daniel, does that work for you? -- \\ Sometimes you're ahead, \\ Mike Smith \\ sometimes you're behind. \\ mike@smith.net.au \\ The race is long, and in the \\ msmith@freebsd.org \\ end it's only with yourself. \\ msmith@cdrom.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Apr 29 11: 1:48 1999 Delivered-To: freebsd-smp@freebsd.org Received: from lor.watermarkgroup.com (lor.watermarkgroup.com [207.202.73.33]) by hub.freebsd.org (Postfix) with ESMTP id 97C4C150AE for ; Thu, 29 Apr 1999 11:01:42 -0700 (PDT) (envelope-from luoqi@watermarkgroup.com) Received: (from luoqi@localhost) by lor.watermarkgroup.com (8.8.8/8.8.8) id OAA17851; Thu, 29 Apr 1999 14:01:05 -0400 (EDT) (envelope-from luoqi) Date: Thu, 29 Apr 1999 14:01:05 -0400 (EDT) From: Luoqi Chen Message-Id: <199904291801.OAA17851@lor.watermarkgroup.com> To: mike@smith.net.au Subject: Re: Really slow SMP Cc: darius@dons.net.au, freebsd-smp@FreeBSD.ORG, peter@netplex.com.au, tlambert@primenet.com Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > Maybe init_secondary() is too ealier for calling mem_range_AP_init, > > APs shouldn't be fooling around with locks at that point. I guess > > what happened was the AP was in a spin loop waiting for a lock and > > BSP timed out waiting for AP's up signal. Try move the call to ap_init() > > instead. > > Sorry, not sure I follow you here; there's no locking in > mem_range_AP_init(), and it's where the MTRRs were being loaded before. > The code path is a little more convoluted now, but has the same basic > effect. > > Regardless, Daniel, does that work for you? > -- > \\ Sometimes you're ahead, \\ Mike Smith > \\ sometimes you're behind. \\ mike@smith.net.au > \\ The race is long, and in the \\ msmith@freebsd.org > \\ end it's only with yourself. \\ msmith@cdrom.com > IIRC, disable_intr() for SMP needs to get a lock to prevent intr from occuring on all cpus. In any case, it's safer to do it ap_init() when the AP holds the giant lock. -lq To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Apr 29 14:33:53 1999 Delivered-To: freebsd-smp@freebsd.org Received: from dingo.cdrom.com (dingo.cdrom.com [204.216.28.145]) by hub.freebsd.org (Postfix) with ESMTP id F0B5414C09 for ; Thu, 29 Apr 1999 14:33:47 -0700 (PDT) (envelope-from mike@dingo.cdrom.com) Received: from dingo.cdrom.com (localhost.cdrom.com [127.0.0.1]) by dingo.cdrom.com (8.9.3/8.8.8) with ESMTP id OAA00867; Thu, 29 Apr 1999 14:31:38 -0700 (PDT) (envelope-from mike@dingo.cdrom.com) Message-Id: <199904292131.OAA00867@dingo.cdrom.com> X-Mailer: exmh version 2.0.2 2/24/98 To: Luoqi Chen Cc: darius@dons.net.au, freebsd-smp@FreeBSD.ORG, peter@netplex.com.au, tlambert@primenet.com Subject: Re: Really slow SMP In-reply-to: Your message of "Thu, 29 Apr 1999 14:01:05 EDT." <199904291801.OAA17851@lor.watermarkgroup.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 29 Apr 1999 14:31:38 -0700 From: Mike Smith Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > > > Sorry, not sure I follow you here; there's no locking in > > mem_range_AP_init(), and it's where the MTRRs were being loaded before. > > The code path is a little more convoluted now, but has the same basic > > effect. > > IIRC, disable_intr() for SMP needs to get a lock to prevent intr from > occuring on all cpus. In any case, it's safer to do it ap_init() when > the AP holds the giant lock. Point. Here's a new diff; can the people with "slow SMP" problems try this one? Index: i386/i386/i686_mem.c =================================================================== RCS file: /home/ncvs/src/sys/i386/i386/i686_mem.c,v retrieving revision 1.1 diff -u -r1.1 i686_mem.c --- i686_mem.c 1999/04/07 03:57:45 1.1 +++ i686_mem.c 1999/04/26 05:34:23 @@ -62,21 +62,28 @@ #define mrcopyflags(curr, new) (((curr) & ~MDF_ATTRMASK) | ((new) & MDF_ATTRMASK)) -static void i686_mrinit(struct mem_range_softc *); -static int i686_mrset(struct mem_range_softc *, - struct mem_range_desc *, - int *); +static void i686_mrinit(struct mem_range_softc *sc); +static int i686_mrset(struct mem_range_softc *sc, + struct mem_range_desc *mrd, + int *arg); +static void i686_mrAPinit(struct mem_range_softc *sc); static struct mem_range_ops i686_mrops = { i686_mrinit, - i686_mrset + i686_mrset, + i686_mrAPinit }; +/* XXX for AP startup hook */ +extern struct mem_range_softc mem_range_softc; +static u_int64_t mtrrcap, mtrrdef; + static struct mem_range_desc *mem_range_match(struct mem_range_softc *sc, struct mem_range_desc *mrd); static void i686_mrfetch(struct mem_range_softc *sc); static int i686_mtrrtype(int flags); static int i686_mrstore(struct mem_range_softc *sc); +static int i686_mrstoreone(struct mem_range_softc *sc); static struct mem_range_desc *i686_mtrrfixsearch(struct mem_range_softc *sc, u_int64_t addr); static int i686_mrsetlow(struct mem_range_softc *sc, @@ -226,10 +233,6 @@ static int i686_mrstore(struct mem_range_softc *sc) { - struct mem_range_desc *mrd; - u_int64_t msrv; - int i, j, msr; - u_int cr4save; #ifdef SMP /* @@ -241,6 +244,17 @@ return(EOPNOTSUPP); #endif + return(i686_mrstoreone(sc)); +} + +static int +i686_mrstoreone(struct mem_range_softc *sc) +{ + struct mem_range_desc *mrd; + u_int64_t msrv; + int i, j, msr; + u_int cr4save; + disable_intr(); /* disable interrupts */ cr4save = rcr4(); /* save cr4 */ if (cr4save & CR4_PGE) @@ -477,7 +491,6 @@ i686_mrinit(struct mem_range_softc *sc) { struct mem_range_desc *mrd; - u_int64_t mtrrcap, mtrrdef; int nmdesc = 0; int i; @@ -491,8 +504,12 @@ return; } nmdesc = mtrrcap & 0xff; - printf("Pentium Pro MTRR support enabled, default memory type is %s\n", - i686_mtrrtotext[mtrrdef & 0xff]); + printf("Pentium Pro MTRR support enabled, default memory type is "); + if ((mtrrdef & 0xff) < (sizeof(i686_mtrrtotext) / sizeof(i686_mtrrtotext[0]))) { + printf("%s\n", i686_mtrrtotext[mtrrdef & 0xff]); + } else { + printf("unknown (0x%x)\n", mtrrdef & 0xff); + } /* If fixed MTRRs supported and enabled */ if ((mtrrcap & 0x100) && (mtrrdef & 0x400)) { @@ -537,6 +554,17 @@ if (mrd->mr_flags & MDF_ACTIVE) mrd->mr_flags |= MDF_FIRMWARE; } +} + +/* + * Initialise MTRRs on an AP after the BSP has run the init code. + */ +static void +i686_mrAPinit(struct mem_range_softc *sc) +{ + i686_mrstoreone(sc); /* set MTRRs to match BSP */ + wrmsr(MSR_MTRRcap, mtrrcap); /* set MTRR behaviour to match BSP */ + wrmsr(MSR_MTRRdefType, mtrrdef); } static void Index: i386/i386/mem.c =================================================================== RCS file: /home/ncvs/src/sys/i386/i386/mem.c,v retrieving revision 1.55 diff -u -r1.55 mem.c --- mem.c 1999/04/07 03:57:45 1.55 +++ mem.c 1999/04/26 00:46:11 @@ -527,6 +527,12 @@ return(mem_range_softc.mr_op->set(&mem_range_softc, mrd, arg)); } +void +mem_range_AP_init(void) +{ + return(mem_range_softc.mr_op->initAP(&mem_range_softc)); +} + static int random_ioctl(dev, cmd, data, flags, p) dev_t dev; Index: i386/i386/mp_machdep.c =================================================================== RCS file: /home/ncvs/src/sys/i386/i386/mp_machdep.c,v retrieving revision 1.97 diff -u -r1.97 mp_machdep.c --- mp_machdep.c 1999/04/13 03:24:47 1.97 +++ mp_machdep.c 1999/04/29 21:28:19 @@ -496,11 +496,6 @@ PTD[0] = 0; pmap_set_opt((unsigned *)PTD); -#if 0 - putmtrr(); - pmap_setvidram(); -#endif - invltlb(); } @@ -2250,9 +2245,7 @@ panic("cpuid mismatch! boom!!"); } -#if 0 - getmtrr(); -#endif + mem_range_AP_init(); /* copy memory range attributes from the BSP */ /* Init local apic for irq's */ apic_initialize(); Index: sys/memrange.h =================================================================== RCS file: /home/ncvs/src/sys/sys/memrange.h,v retrieving revision 1.1 diff -u -r1.1 memrange.h --- memrange.h 1999/04/07 03:59:32 1.1 +++ memrange.h 1999/04/26 00:46:51 @@ -47,6 +47,7 @@ { void (*init)(struct mem_range_softc *sc); int (*set)(struct mem_range_softc *sc, struct mem_range_desc *mrd, int *arg); + void (*initAP)(struct mem_range_softc *sc); }; struct mem_range_softc @@ -61,5 +62,6 @@ extern void mem_range_attr_get(struct mem_range_desc *mrd, int *arg); extern int mem_range_attr_set(struct mem_range_desc *mrd, int *arg); +extern void mem_range_AP_init(void); #endif -- \\ Sometimes you're ahead, \\ Mike Smith \\ sometimes you're behind. \\ mike@smith.net.au \\ The race is long, and in the \\ msmith@freebsd.org \\ end it's only with yourself. \\ msmith@cdrom.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Apr 29 16:55:40 1999 Delivered-To: freebsd-smp@freebsd.org Received: from ix.netcom.com (sil-wa15-53.ix.netcom.com [207.93.148.53]) by hub.freebsd.org (Postfix) with ESMTP id 08844154C8 for ; Thu, 29 Apr 1999 16:55:01 -0700 (PDT) (envelope-from tomdean@ix.netcom.com) Received: (from tomdean@localhost) by ix.netcom.com (8.9.3/8.8.8) id QAA00466; Thu, 29 Apr 1999 16:53:29 -0700 (PDT) (envelope-from tomdean@ix.netcom.com) Date: Thu, 29 Apr 1999 16:53:29 -0700 (PDT) Message-Id: <199904292353.QAA00466@ix.netcom.com> X-Authentication-Warning: celebris.tddhome: tomdean set sender to tomdean@ix.netcom.com using -f From: Thomas Dean To: freebsd-smp@FreeBSD.ORG Subject: 6 Hours to make -j4 world References: <199904292131.OAA00867@dingo.cdrom.com> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I am running 4/0-current SMP on a DEC CELEBRIS 5133DP. I just finished two make world's. The latest ended about an hour ago, from cvsup this morning. A COMPLETE 'make -j4 world', including cleaning, depend, etc., as from unmodified sources, took some 6 hours. I don't see much change over the past couple months. I am not using soft updates. /usr/src and /usr/obj are on the same scsi disk. The machine was doing nothing else. I think it is I/O bound. Soft updates and another disk should improve things. I can change da2 to use part of it for /usr/obj, say 400MB. It is not an extra-fast disk, but, maybe I will try it. Times and dmesg are attached. I did not include the time for a make world yesterday, which I thought would get Luoqi's patch and make a difference. tomdean ====== make -j4 world times ============================== >>> elf make world started on Thu Jan 14 10:39:01 PST 1999 >>> elf make world completed on Thu Jan 14 15:42:05 PST 1999 >>> elf make world started on Fri Jan 29 10:01:17 PST 1999 >>> elf make world completed on Fri Jan 29 15:13:20 PST 1999 >>> elf make world started on Thu Feb 4 21:54:33 PST 1999 >>> elf make world completed on Fri Feb 5 03:07:14 PST 1999 >>> elf make world started on Sat Feb 13 06:34:17 PST 1999 >>> elf make world completed on Sat Feb 13 11:52:25 PST 1999 >>> elf make world started on Sun Feb 14 11:07:14 PST 1999 >>> elf make world completed on Sun Feb 14 12:22:46 PST 1999 >>> elf make world started on Mon Mar 8 23:34:23 PST 1999 >>> elf make world completed on Tue Mar 9 04:50:12 PST 1999 >>> elf make world started on Thu Apr 1 07:41:43 PST 1999 >>> elf make world completed on Thu Apr 1 14:40:00 PST 1999 >>> elf make world started on Mon Apr 12 12:04:36 PDT 1999 >>> elf make world completed on Mon Apr 12 17:47:20 PDT 1999 >>> elf make world started on Wed Apr 21 10:12:33 PDT 1999 >>> elf make world completed on Wed Apr 21 16:23:25 PDT 1999 >>> elf make world started on Thu Apr 29 09:23:05 PDT 1999 >>> elf make world completed on Thu Apr 29 15:18:43 PDT 1999 ======= dmesg ============================== Copyright (c) 1992-1999 The FreeBSD Project. Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All rights reserved. FreeBSD 4.0-CURRENT #0: Thu Apr 29 16:25:13 PDT 1999 tomdean@celebris:/usr/src/sys/compile/CELEBRIS-SMP Timecounter "i8254" frequency 1193182 Hz CPU: Pentium/P54C (586-class CPU) Origin = "GenuineIntel" Id = 0x525 Stepping=5 Features=0x3bf real memory = 100663296 (98304K bytes) avail memory = 94916608 (92692K bytes) Programming 16 pins in IOAPIC #0 FreeBSD/SMP: Multiprocessor motherboard cpu0 (BSP): apic id: 0, version: 0x00030010, at 0xfee00000 cpu1 (AP): apic id: 1, version: 0x00030010, at 0xfee00000 io0 (APIC): apic id: 2, version: 0x000f0011, at 0xfec00000 Preloaded elf kernel "kernel" at 0xc02c0000. npx0: on motherboard npx0: INT 16 interface pcib0: on motherboard pci0: on pcib0 chip0: at device 0.0 on pci0 ncr0: at device 1.0 on pci0 ncr0: interrupting at irq 11 isab0: at device 2.0 on pci0 de0: at device 8.0 on pci0 de0: interrupting at irq 10 de0: DEC DE450-CA 21041 [10Mb/s] pass 1.1 de0: address 00:00:f8:02:76:db isa0: on motherboard fdc0: interrupting at irq 6 fdc0: at port 0x3f0-0x3f7 irq 6 drq 2 on isa0 fdc0: FIFO enabled, 8 bytes threshold fd0: <1440-KB 3.5" drive> at fdc0 drive 0 atkbdc0: at port 0x60 on isa0 atkbd0: on atkbdc0 atkbd0: interrupting at irq 1 psm0: on atkbdc0 psm0: model Generic PS/2 mouse, device ID 0 psm0: interrupting at irq 12 vga0: on isa0 sc0: on isa0 sc0: VGA color <16 virtual consoles, flags=0x0> sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 sio0: type 16550A sio0: interrupting at irq 4 sio1 at port 0x2f8-0x2ff irq 3 on isa0 sio1: type 16550A sio1: interrupting at irq 3 ppc0 at port 0x378-0x37f irq 7 on isa0 ppc0: PC87334 chipset (PS2/NIBBLE) in COMPATIBLE mode lpt0: on ppbus 0 lpt0: Interrupt-driven port ppc0: interrupting at irq 7 Intel Pentium detected, installing workaround for F00F bug APIC_IO: Testing 8254 interrupt delivery APIC_IO: routing 8254 via pin 2 Waiting 10 seconds for SCSI devices to settle SMP: AP CPU #1 Launched! da0 at ncr0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-2 device da0: 10.000MB/s transfers (10.000MHz, offset 8) da0: 1042MB (2134305 512 byte sectors: 255H 63S/T 132C) cd0 at ncr0 bus 0 target 5 lun 0 cd0: Removable CD-ROM SCSI-2 device cd0: 4.237MB/s transfers (4.237MHz, offset 8) cd0: cd present [322265 x 2048 byte records] da2 at ncr0 bus 0 target 2 lun 0 da2: Fixed Direct Access SCSI-2 device da2: 10.000MB/s transfers (10.000MHz, offset 8), Tagged Queueing Enabled da2: 1029MB (2109376 512 byte sectors: 255H 63S/T 131C) da1 at ncr0 bus 0 target 1 lun 0 da1: Fixed Direct Access SCSI-2 device da1: 10.000MB/s transfers (10.000MHz, offset 8), Tagged Queueing Enabled da1: 3090MB (6328861 512 byte sectors: 255H 63S/T 393C) changing root device to da1s1a cd9660: RockRidge Extension To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Apr 30 2:34:44 1999 Delivered-To: freebsd-smp@freebsd.org Received: from zed.ludd.luth.se (zed.ludd.luth.se [130.240.16.33]) by hub.freebsd.org (Postfix) with ESMTP id F345D15906 for ; Fri, 30 Apr 1999 02:34:37 -0700 (PDT) (envelope-from magnus@ludd.luth.se) Received: from speedy.ludd.luth.se (root@speedy.ludd.luth.se [130.240.16.164]) by zed.ludd.luth.se (8.8.5/8.8.5) with ESMTP id LAA22692; Fri, 30 Apr 1999 11:34:32 +0200 From: Magnus Gr|nlund Received: (magnus@localhost) by speedy.ludd.luth.se (8.8.8/8.6.11) id LAA25670; Fri, 30 Apr 1999 11:34:31 +0200 (CEST) Message-Id: <199904300934.LAA25670@speedy.ludd.luth.se> Subject: Re: Really slow SMP In-Reply-To: <199904292131.OAA00867@dingo.cdrom.com> from Mike Smith at "Apr 29, 99 02:31:38 pm" To: mike@smith.net.au (Mike Smith) Date: Fri, 30 Apr 1999 11:34:31 +0200 (CEST) Cc: freebsd-smp@freebsd.org X-Mailer: ELM [version 2.4ME+ PL32 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > > > > > Sorry, not sure I follow you here; there's no locking in > > > mem_range_AP_init(), and it's where the MTRRs were being loaded before. > > > The code path is a little more convoluted now, but has the same basic > > > effect. > > > > IIRC, disable_intr() for SMP needs to get a lock to prevent intr from > > occuring on all cpus. In any case, it's safer to do it ap_init() when > > the AP holds the giant lock. > > Point. Here's a new diff; can the people with "slow SMP" problems try > this one? > I got the "slow SMP"-problem and applying the patches to yesterdays current resulted in "general protection fault in kernel mode" when mem_range_AP_init() is called. /Magnus To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Apr 30 15:19:37 1999 Delivered-To: freebsd-smp@freebsd.org Received: from dingo.cdrom.com (dingo.cdrom.com [204.216.28.145]) by hub.freebsd.org (Postfix) with ESMTP id 954D114EB7 for ; Fri, 30 Apr 1999 15:19:35 -0700 (PDT) (envelope-from mike@dingo.cdrom.com) Received: from dingo.cdrom.com (localhost.cdrom.com [127.0.0.1]) by dingo.cdrom.com (8.9.3/8.8.8) with ESMTP id PAA01213 for ; Fri, 30 Apr 1999 15:18:39 -0700 (PDT) (envelope-from mike@dingo.cdrom.com) Message-Id: <199904302218.PAA01213@dingo.cdrom.com> X-Mailer: exmh version 2.0.2 2/24/98 To: smp@freebsd.org Subject: Re: Slow SMP Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 30 Apr 1999 15:18:39 -0700 From: Mike Smith Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Ok, this looks like it should do the trick. ------- Forwarded Message From: Mike Smith Date: Fri, 30 Apr 1999 15:09:47 -0700 (PDT) To: cvs-committers@FreeBSD.org, cvs-all@FreeBSD.org Subject: cvs commit: src/sys/i386/i386 i686_mem.c mem.c mp_machdep.c src/sys/sys memrange.h msmith 1999/04/30 15:09:46 PDT Modified files: sys/i386/i386 i686_mem.c mem.c mp_machdep.c sys/sys memrange.h Log: Add a hook that can be called to initialise a slave processor's memory range attributes after they have been extracted from the master. Hook up the i686 MP code to do this for each AP. Be more careful about printing the default memory type for the i686. Suggestions from: luoqi Revision Changes Path 1.2 +49 -18 src/sys/i386/i386/i686_mem.c 1.57 +7 -1 src/sys/i386/i386/mem.c 1.99 +4 -1 src/sys/i386/i386/mp_machdep.c 1.2 +2 -0 src/sys/sys/memrange.h ------- End of Forwarded Message -- \\ Sometimes you're ahead, \\ Mike Smith \\ sometimes you're behind. \\ mike@smith.net.au \\ The race is long, and in the \\ msmith@freebsd.org \\ end it's only with yourself. \\ msmith@cdrom.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon May 3 19:47:27 1999 Delivered-To: freebsd-smp@freebsd.org Received: from miles.lambdawerks.org (dlci096.isomedia.com [209.102.112.96]) by hub.freebsd.org (Postfix) with ESMTP id 01BFE14F17 for ; Mon, 3 May 1999 19:47:07 -0700 (PDT) (envelope-from reggie@lambdawerks.org) Received: from trane.lambdawerks.org (trane.lambdawerks.org [209.102.113.58]) by miles.lambdawerks.org (8.9.2/8.9.3) with ESMTP id TAA87679; Mon, 3 May 1999 19:47:02 -0700 (PDT) (envelope-from reggie@lambdawerks.org) Received: from localhost (localhost [127.0.0.1]) by trane.lambdawerks.org (8.9.3/8.9.3) with ESMTP id TAA04218; Mon, 3 May 1999 19:46:54 -0700 (PDT) (envelope-from reggie@trane.lambdawerks.org) Message-Id: <199905040246.TAA04218@trane.lambdawerks.org> To: Mike Smith Cc: smp@FreeBSD.ORG Subject: Re: Slow SMP In-reply-to: Message from Mike Smith of "Fri, 30 Apr 1999 15:18:39 PDT." <199904302218.PAA01213@dingo.cdrom.com> Mime-Version: 1.0 (generated by tm-edit 7.108) Content-Type: text/plain; charset=US-ASCII Date: Mon, 03 May 1999 19:46:54 -0700 From: "Reginald S. Perry" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Could these changes have created problems? I cannot boot an SMP kernel now. I get a page fault while in kernel mode immediately after CPU 1 launches. It happens at this point: mem_range_AP_init+0x8: cmpl $0, 0x8(%eax) I am running a Tyan TomcatIII dual board. My kernel from April 24th would at least boot. I was setting up to make the machine panic so that I could send reports about panic problems when messing with netscape and lockups when I rm -rf /usr/ports and /usr/src and check them out again in two seperate windows in X. For example: cd /usr/src; rm -rf ./*; cd ..; cvs -d /home/ncvs checkout src and the same thing for /usr/ports would lockup the machine with an SMP kernel. This does not happen when I build a UP kernel. If you need more info, let me know and I will try to produce. -Reggie >"Mike" == Mike Smith writes: > Ok, this looks like it should do the trick. > ------- Forwarded Message > From: Mike Smith Date: Fri, 30 Apr 1999 > 15:09:47 -0700 (PDT) To: cvs-committers@FreeBSD.org, > cvs-all@FreeBSD.org Subject: cvs commit: src/sys/i386/i386 > i686_mem.c mem.c mp_machdep.c src/sys/sys memrange.h > msmith 1999/04/30 15:09:46 PDT > Modified files: sys/i386/i386 i686_mem.c mem.c mp_machdep.c > sys/sys memrange.h Log: Add a hook that can be called to initialise > a slave processor's memory range attributes after they have been > extracted from the master. > Hook up the i686 MP code to do this for each AP. > Be more careful about printing the default memory type for the > i686. > Suggestions from: luoqi > Revision Changes Path 1.2 +49 -18 src/sys/i386/i386/i686_mem.c > 1.57 +7 -1 src/sys/i386/i386/mem.c 1.99 +4 -1 > src/sys/i386/i386/mp_machdep.c 1.2 +2 -0 src/sys/sys/memrange.h > ------- End of Forwarded Message > -- \\ Sometimes you're ahead, \\ Mike Smith \\ sometimes you're > behind. \\ mike@smith.net.au \\ The race is long, and in the \\ > msmith@freebsd.org \\ end it's only with yourself. \\ > msmith@cdrom.com > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe > freebsd-smp" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon May 3 20:10:14 1999 Delivered-To: freebsd-smp@freebsd.org Received: from oldnews.quick.net (unknown [207.212.170.1]) by hub.freebsd.org (Postfix) with ESMTP id ACD671574A for ; Mon, 3 May 1999 20:10:06 -0700 (PDT) (envelope-from donegan@oldnews.quick.net) Received: (from donegan@localhost) by oldnews.quick.net (8.8.5/8.6.9) id UAA05577; Mon, 3 May 1999 20:09:23 -0700 (PDT) Date: Mon, 3 May 1999 20:09:22 -0700 (PDT) From: "Steven P. Donegan" To: "Reginald S. Perry" Cc: Mike Smith , smp@FreeBSD.ORG Subject: Re: Slow SMP In-Reply-To: <199905040246.TAA04218@trane.lambdawerks.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Mon, 3 May 1999, Reginald S. Perry wrote: > Could these changes have created problems? I cannot boot an SMP kernel > now. I get a page fault while in kernel mode immediately after CPU 1 > launches. It happens at this point: > > mem_range_AP_init+0x8: cmpl $0, 0x8(%eax) > This has been the case here all weekend - a TYAN IIID and IVD (two systems) - both panic right after launch into SMP mode. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri May 7 4:22:38 1999 Delivered-To: freebsd-smp@freebsd.org Received: from chmls06.mediaone.net (chmls06.mediaone.net [24.128.1.71]) by hub.freebsd.org (Postfix) with ESMTP id 8527D14F63 for ; Fri, 7 May 1999 04:22:35 -0700 (PDT) (envelope-from housley@frenchknot.ne.mediaone.net) Received: from frenchknot.ne.mediaone.net (frenchknot.ne.mediaone.net [24.128.74.10]) by chmls06.mediaone.net (8.8.7/8.8.7) with ESMTP id HAA09540 for ; Fri, 7 May 1999 07:22:35 -0400 (EDT) Received: from frenchknot.ne.mediaone.net (housley@localhost [127.0.0.1]) by frenchknot.ne.mediaone.net (8.9.3/8.9.1) with ESMTP id HAA58109 for ; Fri, 7 May 1999 07:22:31 -0400 (EDT) (envelope-from housley@frenchknot.ne.mediaone.net) Message-ID: <3732CCF7.B17289C8@frenchknot.ne.mediaone.net> Date: Fri, 07 May 1999 07:22:31 -0400 From: "James E. Housley" X-Mailer: Mozilla 4.51 [en] (X11; U; FreeBSD 3.1-STABLE i386) X-Accept-Language: en MIME-Version: 1.0 To: smp@freebsd.org Subject: Dual MB Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I am in the process of building a dual 3.1-STABLE machine. I was looking at the Asus P2B-D. My normal supplier no longer deals with Asus becuase he was getting too many RMAs. Is this an old problem that has been sloved, or is it on going? What other MB would you suggest. It is for a personal machine. Thanks. -- James E. Housley PGP: 1024/03983B4D System Supply, Inc. 2C 3F 3A 0D A8 D8 C3 13 Pager: pagejim@notepage.com 7C F0 B5 BF 27 8B 92 FE To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri May 7 6:46: 7 1999 Delivered-To: freebsd-smp@freebsd.org Received: from solaris.matti.ee (solaris.matti.ee [194.126.98.135]) by hub.freebsd.org (Postfix) with ESMTP id 989A414DC6 for ; Fri, 7 May 1999 06:46:00 -0700 (PDT) (envelope-from vallo@matti.ee) Received: from myhakas.matti.ee (myhakas.matti.ee [194.126.114.87]) by solaris.matti.ee (8.8.8/8.8.8.s) with ESMTP id QAA11876; Fri, 7 May 1999 16:45:50 +0300 (EET DST) Received: by myhakas.matti.ee (Postfix, from userid 1000) id 421CD1F2D; Fri, 7 May 1999 16:45:51 +0300 (EEST) Date: Fri, 7 May 1999 16:45:51 +0300 From: Vallo Kallaste To: "James E. Housley" Cc: smp@FreeBSD.ORG Subject: Re: Dual MB Message-ID: <19990507164551.A8911@myhakas.matti.ee> Reply-To: vallo@matti.ee References: <3732CCF7.B17289C8@frenchknot.ne.mediaone.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.4i In-Reply-To: <3732CCF7.B17289C8@frenchknot.ne.mediaone.net>; from James E. Housley on Fri, May 07, 1999 at 07:22:31AM -0400 Organization: =?iso-8859-1?Q?AS_Matti_B=FCrootehnika?= Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Fri, May 07, 1999 at 07:22:31AM -0400, "James E. Housley" wrote: > I am in the process of building a dual 3.1-STABLE machine. I was > looking at the Asus P2B-D. My normal supplier no longer deals with Asus > becuase he was getting too many RMAs. Is this an old problem that has > been sloved, or is it on going? > > What other MB would you suggest. It is for a personal machine. Depends how much money you can spend. I can always recommend Tyan S1836 and S1837 boards. These are quite expensive but you get what you expect, I have S1836DLUAN and it's worth of the price. -- Vallo Kallaste vallo@matti.ee To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri May 7 8:14:14 1999 Delivered-To: freebsd-smp@freebsd.org Received: from gidgate.gid.co.uk (gidgate.gid.co.uk [193.123.140.1]) by hub.freebsd.org (Postfix) with ESMTP id D1ACB150D9 for ; Fri, 7 May 1999 08:14:05 -0700 (PDT) (envelope-from rb@gid.co.uk) Received: (from rb@localhost) by gidgate.gid.co.uk (8.8.8/8.8.7) id QAA10266; Fri, 7 May 1999 16:13:14 +0100 (BST) (envelope-from rb) Message-Id: <3.0.6.32.19990507161312.007c1550@192.168.255.1> X-Sender: rbmail@192.168.255.1 X-Mailer: QUALCOMM Windows Eudora Light Version 3.0.6 (32) Date: Fri, 07 May 1999 16:13:12 +0100 To: "James E. Housley" From: Bob Bishop Subject: Re: Dual MB Cc: smp@FreeBSD.ORG In-Reply-To: <3732CCF7.B17289C8@frenchknot.ne.mediaone.net> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi, At 07:22 07/05/99 -0400, James E. Housley wrote: >I am in the process of building a dual 3.1-STABLE machine. I was >looking at the Asus P2B-D. My normal supplier no longer deals with Asus >becuase he was getting too many RMAs. Is this an old problem that has >been sloved, or is it on going? > >What other MB would you suggest. It is for a personal machine. I'm running -current on a GigaByte GA-6BXD which has been trouble-free. Variants are available with onboard SCSI and ethernet. -- Bob Bishop +44 118 977 4017 rb@gid.co.uk fax +44 118 989 4254 (0800-1800 UK) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri May 7 10:28:31 1999 Delivered-To: freebsd-smp@freebsd.org Received: from quark.ChrisBowman.com (crbowman.erols.com [209.122.47.155]) by hub.freebsd.org (Postfix) with ESMTP id 65A5C15247 for ; Fri, 7 May 1999 10:28:00 -0700 (PDT) (envelope-from crb@ChrisBowman.com) Received: from fermion (crb@fermion.ChrisBowman.com [10.0.1.2]) by quark.ChrisBowman.com (8.9.2/8.8.8) with SMTP id NAA09471; Fri, 7 May 1999 13:27:33 -0400 (EDT) (envelope-from crb@ChrisBowman.com) Message-Id: <199905071727.NAA09471@quark.ChrisBowman.com> X-Sender: crb@quark X-Mailer: QUALCOMM Windows Eudora Pro Version 4.0.1 Date: Fri, 07 May 1999 13:26:35 -0400 To: "James E. Housley" From: "Christopher R. Bowman" Subject: Re: Dual MB Cc: smp@FreeBSD.ORG In-Reply-To: <3732CCF7.B17289C8@frenchknot.ne.mediaone.net> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org At 07:22 AM 5/7/99 -0400, James E. Housley wrote: >I am in the process of building a dual 3.1-STABLE machine. I was >looking at the Asus P2B-D. My normal supplier no longer deals with Asus >becuase he was getting too many RMAs. Is this an old problem that has >been sloved, or is it on going? > >What other MB would you suggest. It is for a personal machine. I don't have one, but the TYAN S1837UANG Thunderbolt $529.88 at www.ic-direct.com looks to be a pretty nice board with all the trimmings (see http://www.tyan.com/products/html/s1837uang.html). As I said I haven't actually used it, but I am very happy with my current Tyan board. -------- Christopher R. Bowman crb@ChrisBowman.com http://www.ChrisBowman.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri May 7 10:58:19 1999 Delivered-To: freebsd-smp@freebsd.org Received: from atlrel2.hp.com (atlrel2.hp.com [156.153.255.202]) by hub.freebsd.org (Postfix) with ESMTP id 1227715396 for ; Fri, 7 May 1999 10:58:06 -0700 (PDT) (envelope-from darrylo@sr.hp.com) Received: from srmail.sr.hp.com (srmail.sr.hp.com [15.4.45.14]) by atlrel2.hp.com (8.8.6 (PHNE_17135)/8.8.5tis) with ESMTP id NAA23748; Fri, 7 May 1999 13:57:27 -0400 (EDT) Received: from mina.sr.hp.com by srmail.sr.hp.com with ESMTP (1.37.109.16/15.5+ECS 3.3) id AA156469864; Fri, 7 May 1999 10:57:44 -0700 Received: from localhost (darrylo@mina.sr.hp.com [15.4.42.247]) by mina.sr.hp.com with ESMTP (8.8.6 (PHNE_14041)/8.7.3 TIS 5.0) id KAA14395; Fri, 7 May 1999 10:57:44 -0700 (PDT) Message-Id: <199905071757.KAA14395@mina.sr.hp.com> To: "James E. Housley" , smp@FreeBSD.ORG Subject: Re: Dual MB Reply-To: Darryl Okahata In-Reply-To: Your message of "Fri, 07 May 1999 16:13:12 BST." <3.0.6.32.19990507161312.007c1550@192.168.255.1> Mime-Version: 1.0 (generated by tm-edit 1.1.1.1) Content-Type: text/plain; charset=US-ASCII Date: Fri, 07 May 1999 10:57:43 -0700 From: Darryl Okahata Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Bob Bishop wrote: > I'm running -current on a GigaByte GA-6BXD which has been trouble-free. > Variants are available with onboard SCSI and ethernet. I'll second the GigaByte motherboard. They're also very inexpensive, compared to Asus and Tyan, even though they're based upon the 440BX chipset. You can get the 6BXD for under US$170 (yes, this is a dual-CPU board!), and the 6BXDS (the 6BXD with on-board F/W SCSI) can be had for under US$300. I'm using the 6BXDS, and I'm happy with it (especially since I got it for around US$235, before the prices went up). The only "problem" is that the 6BXDS doesn't support LVD SCSI, just plain F/W (which is probably why it's inexpensive -- the Gigabyte dual CPU board with LVD SCSI costs around US$430, which is comparable to those of Asus and Tyan). -- Darryl Okahata darrylo@sr.hp.com DISCLAIMER: this message is the author's personal opinion and does not constitute the support, opinion, or policy of Hewlett-Packard, or of the little green men that have been following him all day. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri May 7 11:38:29 1999 Delivered-To: freebsd-smp@freebsd.org Received: from dingo.cdrom.com (dingo.cdrom.com [204.216.28.145]) by hub.freebsd.org (Postfix) with ESMTP id 33CD614E46 for ; Fri, 7 May 1999 11:38:27 -0700 (PDT) (envelope-from mike@dingo.cdrom.com) Received: from dingo.cdrom.com (localhost.cdrom.com [127.0.0.1]) by dingo.cdrom.com (8.9.3/8.8.8) with ESMTP id LAA00890; Fri, 7 May 1999 11:36:43 -0700 (PDT) (envelope-from mike@dingo.cdrom.com) Message-Id: <199905071836.LAA00890@dingo.cdrom.com> X-Mailer: exmh version 2.0.2 2/24/98 To: "James E. Housley" Cc: smp@freebsd.org Subject: Re: Dual MB In-reply-to: Your message of "Fri, 07 May 1999 07:22:31 EDT." <3732CCF7.B17289C8@frenchknot.ne.mediaone.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 07 May 1999 11:36:43 -0700 From: Mike Smith Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > I am in the process of building a dual 3.1-STABLE machine. I was > looking at the Asus P2B-D. My normal supplier no longer deals with Asus > becuase he was getting too many RMAs. Is this an old problem that has > been sloved, or is it on going? > > What other MB would you suggest. It is for a personal machine. Your supplier was probably buying on the grey market. Asus are still considered a good buy. See www.tomshardware.com for much more opinion than you'll get here. -- \\ Sometimes you're ahead, \\ Mike Smith \\ sometimes you're behind. \\ mike@smith.net.au \\ The race is long, and in the \\ msmith@freebsd.org \\ end it's only with yourself. \\ msmith@cdrom.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri May 7 13:17:28 1999 Delivered-To: freebsd-smp@freebsd.org Received: from pluto.plutotech.com (mail.plutotech.com [206.168.67.137]) by hub.freebsd.org (Postfix) with ESMTP id 6CED715480 for ; Fri, 7 May 1999 13:17:23 -0700 (PDT) (envelope-from drew@pluto.plutotech.com) Received: (from drew@localhost) by pluto.plutotech.com (8.9.2/8.9.1) id OAA43964; Fri, 7 May 1999 14:17:19 -0600 (MDT) (envelope-from drew) Date: Fri, 7 May 1999 14:17:19 -0600 (MDT) From: Drew Eckhardt Message-Id: <199905072017.OAA43964@pluto.plutotech.com> To: freebsd-smp@freebsd.org, housley@frenchknot.ne.mediaone.net Subject: Re: Dual MB X-Newsgroups: pluto.freebsd.smp In-Reply-To: <3732CCF7.B17289C8@frenchknot.ne.mediaone.net> Organization: Cc: Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In article <3732CCF7.B17289C8@frenchknot.ne.mediaone.net> you write: >I am in the process of building a dual 3.1-STABLE machine. I was >looking at the Asus P2B-D. My normal supplier no longer deals with Asus >becuase he was getting too many RMAs. Is this an old problem that has >been sloved, or is it on going? > >What other MB would you suggest. It is for a personal machine. I have a Supermicro P6DGS D = dual processor G = 440GX chipset (the idea being that 440GX is the lastest rev of 440BX, and should have any bugs fixed in it) S = dual channel ultra wide SCSI (both single channel/bridge U2W SCSI (U suffix) and EIDE only (E) are available. Those flavors also give you 5 PCI slots (1 shared ISA) instead of 4 (1 shared)). I've got piles of old SCSI devices, and didn't want the fast traffic on the same wires as those drives & cables with questionable electrical parameters. About $400. My one complaint is that I can't find anyway in the BIOS to disable the boot-time keyboard check :-( To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri May 7 13:20:37 1999 Delivered-To: freebsd-smp@freebsd.org Received: from gndrsh.aac.dev.com (GndRsh.aac.dev.com [198.145.92.4]) by hub.freebsd.org (Postfix) with ESMTP id 7A89B153DC for ; Fri, 7 May 1999 13:20:27 -0700 (PDT) (envelope-from rgrimes@gndrsh.aac.dev.com) Received: (from rgrimes@localhost) by gndrsh.aac.dev.com (8.9.3/8.9.3) id NAA14597; Fri, 7 May 1999 13:20:00 -0700 (PDT) (envelope-from rgrimes) From: "Rodney W. Grimes" Message-Id: <199905072020.NAA14597@gndrsh.aac.dev.com> Subject: Re: Dual MB In-Reply-To: <199905071836.LAA00890@dingo.cdrom.com> from Mike Smith at "May 7, 1999 11:36:43 am" To: mike@smith.net.au (Mike Smith) Date: Fri, 7 May 1999 13:20:00 -0700 (PDT) Cc: housley@frenchknot.ne.mediaone.net (James E. Housley), smp@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL54 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > I am in the process of building a dual 3.1-STABLE machine. I was > > looking at the Asus P2B-D. My normal supplier no longer deals with Asus > > becuase he was getting too many RMAs. Is this an old problem that has > > been sloved, or is it on going? Having been an authorized ASUS retailer for 5 years I can say it was never true. We sell 100's of ASUS boards/year and I think I have sent 2 of them back to the factory for repair. What is true on the other hand is there are a few companies who have cloned ASUS's product and flooded the market with these substandard low quality knock off's. ASUS sent out notifications on how to check for them and recommended purchasing from the authorized distributors ( which we always have). > > > > What other MB would you suggest. It is for a personal machine. > > Your supplier was probably buying on the grey market. Grey market can have some effect on product quality, but only if it is mis-handled. I've actually had a real problem with a local distributor, who is authorized by ASUS, who likes to tag the motherboard with those stupid white sticky tags. I found my failure rate on incoming product jumped 3%, so I went to check it out. Well, those boys in receiving are working any place they can with no regard to ESD or even decent handling of the product. I showed the distributor my numbers, pointed at the clue- less clones handling product and explained why I would no longer be doing business with them. > Asus are still > considered a good buy. See www.tomshardware.com for much more opinion > than you'll get here. > -- Rod Grimes - KD7CAX - (RWG25) rgrimes@gndrsh.aac.dev.com Accurate Automation, Inc. Reliable computers for FreeBSD http://www.aai.dnsmgr.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri May 7 14:55:47 1999 Delivered-To: freebsd-smp@freebsd.org Received: from eagle.phc.igs.net (eagle.phc.igs.net [207.210.17.201]) by hub.freebsd.org (Postfix) with ESMTP id 7272C14BFF for ; Fri, 7 May 1999 14:55:33 -0700 (PDT) (envelope-from eagle@eagle.phc.igs.net) Received: by eagle.phc.igs.net (Postfix, from userid 1003) id 62E0D18A3; Fri, 7 May 1999 16:55:00 +0000 (GMT) Received: from localhost (localhost [127.0.0.1]) by eagle.phc.igs.net (Postfix) with SMTP id 60C7038; Fri, 7 May 1999 16:55:00 +0000 (GMT) Date: Fri, 7 May 1999 16:55:00 +0000 (GMT) From: eagle To: Bob Bishop Cc: "James E. Housley" , smp@FreeBSD.ORG Subject: Re: Dual MB In-Reply-To: <3.0.6.32.19990507161312.007c1550@192.168.255.1> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I personaly love my Asus P2B-D i've got a few of them and they all work fantasic Rob To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri May 7 16: 4: 5 1999 Delivered-To: freebsd-smp@freebsd.org Received: from smtp6.jps.net (smtp6.jps.net [209.63.224.103]) by hub.freebsd.org (Postfix) with ESMTP id A6CF3154B4 for ; Fri, 7 May 1999 16:03:00 -0700 (PDT) (envelope-from ulairi@jps.net) Received: from default (208-237-196-140.irv.jps.net [208.237.196.140]) by smtp6.jps.net (8.9.0/8.8.5) with SMTP id AAA05759; Sat, 8 May 1999 00:02:01 -0700 (PDT) From: "Ulairi" To: "eagle" Cc: "Smp" Subject: RE: Dual MB Date: Fri, 7 May 1999 16:01:37 -0700 Message-ID: <000f01be98dd$8f67ee20$8cc4edd0@default> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0 In-Reply-To: Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V4.72.2106.4 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'll second the SuperMicro boards. :) I run a P6DLF and no worries under NT/95/FreeBSD (only NT and Fbsd see the second CPU, obviously).... :) | -----Original Message----- | From: owner-freebsd-smp@FreeBSD.ORG | [mailto:owner-freebsd-smp@FreeBSD.ORG]On Behalf Of eagle | Sent: Friday, May 07, 1999 09:55 | To: Bob Bishop | Cc: James E. Housley; smp@FreeBSD.ORG | Subject: Re: Dual MB | | | I personaly love my Asus P2B-D i've got a few of them and | they all work | fantasic | | Rob | | | | | To Unsubscribe: send mail to majordomo@FreeBSD.org | with "unsubscribe freebsd-smp" in the body of the message | -----BEGIN PGP SIGNATURE----- Version: PGPfreeware 6.0 for non-commercial use iQA/AwUBNzNwB1R8Yh25VFLEEQLsSwCgnthBYlIovBFiEkVSGiJb5Dg9QPwAoPmi YMzpsxkZ8Zv9qxTotWS5Jy6W =sK8v -----END PGP SIGNATURE----- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri May 7 17:20:53 1999 Delivered-To: freebsd-smp@freebsd.org Received: from smtp02.primenet.com (smtp02.primenet.com [206.165.6.132]) by hub.freebsd.org (Postfix) with ESMTP id 426EC14DAF for ; Fri, 7 May 1999 17:20:51 -0700 (PDT) (envelope-from tlambert@usr05.primenet.com) Received: (from daemon@localhost) by smtp02.primenet.com (8.8.8/8.8.8) id RAA09729; Fri, 7 May 1999 17:20:48 -0700 (MST) Received: from usr05.primenet.com(206.165.6.205) via SMTP by smtp02.primenet.com, id smtpd009663; Fri May 7 17:20:41 1999 Received: (from tlambert@localhost) by usr05.primenet.com (8.8.5/8.8.5) id RAA02074; Fri, 7 May 1999 17:20:38 -0700 (MST) From: Terry Lambert Message-Id: <199905080020.RAA02074@usr05.primenet.com> Subject: Re: Dual MB To: rgrimes@gndrsh.aac.dev.com (Rodney W. Grimes) Date: Sat, 8 May 1999 00:20:37 +0000 (GMT) Cc: mike@smith.net.au, housley@frenchknot.ne.mediaone.net, smp@FreeBSD.ORG In-Reply-To: <199905072020.NAA14597@gndrsh.aac.dev.com> from "Rodney W. Grimes" at May 7, 99 01:20:00 pm X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Having been an authorized ASUS retailer for 5 years I can say it was > never true. We sell 100's of ASUS boards/year and I think I have sent > 2 of them back to the factory for repair. I have to say that I'm very happy with the ASUS dual P90 machine which I bought from Rod years ago to revive Jack Vogel's SMP FreeBSD project from the dead. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat May 8 8: 0:41 1999 Delivered-To: freebsd-smp@freebsd.org Received: from xwin.nmhtech.com (xwin.nmhtech.com [208.138.46.10]) by hub.freebsd.org (Postfix) with ESMTP id C9E5614CCA for ; Sat, 8 May 1999 08:00:39 -0700 (PDT) (envelope-from nicole@xwin.nmhtech.com) Received: by xwin.nmhtech.com (Postfix, from userid 1001) id D69782EE1B; Sat, 8 May 1999 08:00:38 -0700 (PDT) Content-Length: 2201 Message-ID: X-Mailer: XFMail 1.2 [p0] on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 In-Reply-To: <3732CCF7.B17289C8@frenchknot.ne.mediaone.net> Date: Sat, 08 May 1999 08:00:38 -0700 (PDT) From: Nicole Harrington To: "James E. Housley" Subject: RE: Dual MB Cc: smp@freebsd.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 07-May-99 My Secret Spies Reported That James E. Housley wrote: > I am in the process of building a dual 3.1-STABLE machine. I was > looking at the Asus P2B-D. My normal supplier no longer deals with Asus > becuase he was getting too many RMAs. Is this an old problem that has > been sloved, or is it on going? =20 >=20 > What other MB would you suggest. It is for a personal machine. >=20 I have used several ASUS P2B-DS motherboards.=20 1) No problem. Runs a VERY busy web server with 2 333MHZ CPU's 2) Server with 2 400 MHZ CPU's can't use Top effectively or systat -vmstat= as it complains of clock problems and won't report CPU usage. I have been told that this is perhaps a FreeBSD issue. As I need to upgrade the one above to= 2 450's, I will soon see. 3) A new one, with 2 333CPU's blew up after 2 days, somehow taking the hardrive content with it. It decided that it would only run with one CPU. After replacing the board I had to completely reinstall FreeBSD to get it = to boot. so far 2 days up running OK. Little scary though to loose my content = to a MB. Having the on-board SCSI shouldn't been an issue, should it? I will definatly be trying some of the other boards recommended. Nicole > Thanks. > --=20 > James E. Housley PGP: 1024/03983B4D > System Supply, Inc. 2C 3F 3A 0D A8 D8 C3 13 > Pager: pagejim@notepage.com 7C F0 B5 BF 27 8B 92 FE >=20 >=20 > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-smp" in the body of the message |\ __ /| (`\ =20 | o_o |__ ) ) =20 // \\ =20 nicole@nmhtech.com | http://www.webweaver.net/ webmistress@dangermouse.org | http://www.dangermouse.org -------------------------(((---(((----------------------- =20 - Powered by Coka Cola and FreeBSD - - Strong enough for a man - But made for a Woman - =20 - I'm not ADD - I'm just Multithreaded - - Microsoft: What bug would you like today? - ---------------------------------------------------------- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message