From owner-freebsd-smp Sun Sep 1 00:30:11 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id AAA12648 for smp-outgoing; Sun, 1 Sep 1996 00:30:11 -0700 (PDT) Received: from critter.tfs.com ([140.145.230.177]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id AAA12643; Sun, 1 Sep 1996 00:30:06 -0700 (PDT) Received: from critter.tfs.com (localhost.tfs.com [127.0.0.1]) by critter.tfs.com (8.7.5/8.7.3) with ESMTP id JAA00302; Sun, 1 Sep 1996 09:21:42 +0200 (MET DST) To: Steve Passe cc: freebsd-smp@FreeBSD.ORG, rv@groa.uct.ac.za, erich@uruk.org Subject: Re: SMP on Intel MG15 In-reply-to: Your message of "Sat, 31 Aug 1996 15:43:40 MDT." <199608312143.PAA28776@clem.systemsix.com> Date: Sun, 01 Sep 1996 09:21:41 +0200 Message-ID: <300.841562501@critter.tfs.com> From: Poul-Henning Kamp Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk In message <199608312143.PAA28776@clem.systemsix.com>, Steve Passe writes: >Hi, > >Russel and I are trying to get FreeBSD SMP working on the Intel XXPRESS >board. We are past hte 2nd CPU ID = 2 problem. It appears that >the apic_startup routine never (correctly) launches the 2nd CPU. looking >at the MP spec and Intel pent. manuals it appears that the INIT/Reset >IPI expects that a "warm start vector" will exist for starting the >2nd CPU, but the current code seems to ignore this. See MP spec B.4.1 >for details. the startup code is just the minimum to get neptune based machines flying. much work is needed there. -- Poul-Henning Kamp | phk@FreeBSD.ORG FreeBSD Core-team. http://www.freebsd.org/~phk | phk@login.dknet.dk Private mailbox. whois: [PHK] | phk@ref.tfs.com TRW Financial Systems, Inc. Future will arrive by its own means, progress not so. From owner-freebsd-smp Mon Sep 2 02:21:25 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id CAA29943 for smp-outgoing; Mon, 2 Sep 1996 02:21:25 -0700 (PDT) Received: from groa.uct.ac.za (groa.uct.ac.za [137.158.128.7]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id CAA29936 for ; Mon, 2 Sep 1996 02:21:19 -0700 (PDT) Received: by groa.uct.ac.za via sendmail with stdio id for smp@freebsd.org; Mon, 2 Sep 1996 11:21:11 +0200 (SAT) (Smail-3.2 1996-Jul-4 #1 built 1996-Jul-21) Message-Id: From: rv@groa.uct.ac.za (Russell Vincent) Subject: Testing SMP To: smp@freebsd.org Date: Mon, 2 Sep 1996 11:21:11 +0200 (SAT) X-Mailer: ELM [version 2.4 PL25] Content-Type: text Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Is there a way to test whether a 2nd processor has actually started or not from software running in multiuser mode? i.e: After the machine has booted? Anyone written some code they willing to share? Steve Passe and I (Steve doing all the work) have been working on getting FreeBSD's SMP working on the Intel Xtended Xpress and bootup indications (with extensive debug code, thanks to Steve) are that the the 2nd CPU is not running. However, the OS is running at a load average of '2', based on 2 CPUidle processes, which is making me suspicious. Thanks -Russell PS: Is the SMP CVS repository on Freefall going to be updated with the latest kernel patches? The SMP kernel stops 'ps' from working, presumably due to recent additions to src/sys in -current. From owner-freebsd-smp Mon Sep 2 03:01:14 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id DAA02731 for smp-outgoing; Mon, 2 Sep 1996 03:01:14 -0700 (PDT) Received: from critter.tfs.com ([140.145.230.252]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id DAA02724; Mon, 2 Sep 1996 03:01:09 -0700 (PDT) Received: from critter.tfs.com (localhost.tfs.com [127.0.0.1]) by critter.tfs.com (8.7.5/8.7.3) with ESMTP id MAA03338; Mon, 2 Sep 1996 12:00:38 +0200 (MET DST) To: rv@groa.uct.ac.za (Russell Vincent) cc: smp@freebsd.org Subject: Re: Testing SMP In-reply-to: Your message of "Sat, 02 Sep 1996 11:21:11 +0200." Date: Mon, 02 Sep 1996 12:00:37 +0200 Message-ID: <3336.841658437@critter.tfs.com> From: Poul-Henning Kamp Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk In message , Russell Vincent writes: >Is there a way to test whether a 2nd processor has actually started >or not from software running in multiuser mode? i.e: After the >machine has booted? Anyone written some code they willing to share? > >Steve Passe and I (Steve doing all the work) have been working on >getting FreeBSD's SMP working on the Intel Xtended Xpress and bootup >indications (with extensive debug code, thanks to Steve) are that the >the 2nd CPU is not running. However, the OS is running at a load >average of '2', based on 2 CPUidle processes, which is making me >suspicious. The load avg is no indication. you could examine the SMPcurproc array... >PS: Is the SMP CVS repository on Freefall going to be updated with > the latest kernel patches? The SMP kernel stops 'ps' from > working, presumably due to recent additions to src/sys > in -current. No I hope the smp changes are imported into -current soon... -- Poul-Henning Kamp | phk@FreeBSD.ORG FreeBSD Core-team. http://www.freebsd.org/~phk | phk@login.dknet.dk Private mailbox. whois: [PHK] | phk@ref.tfs.com TRW Financial Systems, Inc. Future will arrive by its own means, progress not so. From owner-freebsd-smp Mon Sep 2 12:21:37 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id MAA08095 for smp-outgoing; Mon, 2 Sep 1996 12:21:37 -0700 (PDT) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id MAA08090 for ; Mon, 2 Sep 1996 12:21:33 -0700 (PDT) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id MAA02683; Mon, 2 Sep 1996 12:20:42 -0700 From: Terry Lambert Message-Id: <199609021920.MAA02683@phaeton.artisoft.com> Subject: Re: SMP on Intel MG15 To: smp@csn.net (Steve Passe) Date: Mon, 2 Sep 1996 12:20:41 -0700 (MST) Cc: freebsd-smp@FreeBSD.ORG, rv@groa.uct.ac.za, erich@uruk.org In-Reply-To: <199608312143.PAA28776@clem.systemsix.com> from "Steve Passe" at Aug 31, 96 03:43:40 pm X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk > Russel and I are trying to get FreeBSD SMP working on the Intel XXPRESS > board. We are past hte 2nd CPU ID = 2 problem. It appears that > the apic_startup routine never (correctly) launches the 2nd CPU. looking > at the MP spec and Intel pent. manuals it appears that the INIT/Reset > IPI expects that a "warm start vector" will exist for starting the > 2nd CPU, but the current code seems to ignore this. See MP spec B.4.1 > for details. > > Is the purpose of this INIT/Reset IPI just to get the 2nd CPU to a known > state, ie NOT to actually start it processing the bootMP code? > > Should not the code setup the BIOS warmstart vector to point @ > a "HLT" instruction b4 using the INIT IPI? B.4.1 specifically applies only to the 82489DX APIC. Is this a 486 MP box you are trying to get running? I think it's more likely that the problem is the system default state doesn't match one of the allowable configurations, and is maybe using a defaul configuration that we aren't handling properly. See chapter 5 of the spec, "Default Configurations". I think it's unlikely that the B.4.1 startup is your problem. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Mon Sep 2 12:34:57 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id MAA08470 for smp-outgoing; Mon, 2 Sep 1996 12:34:57 -0700 (PDT) Received: from uruk.org (uruk.org [198.145.95.253]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id MAA08465 for ; Mon, 2 Sep 1996 12:34:53 -0700 (PDT) From: erich@uruk.org Received: from loopback (loopback [127.0.0.1]) by uruk.org (8.7.4/8.7.3) with SMTP id MAA18360; Mon, 2 Sep 1996 12:34:43 -0700 (PDT) Message-Id: <199609021934.MAA18360@uruk.org> X-Authentication-Warning: uruk.org: Host loopback [127.0.0.1] didn't use HELO protocol To: Terry Lambert cc: smp@csn.net, freebsd-smp@freebsd.org, rv@groa.uct.ac.za Subject: Re: SMP on Intel MG15 In-reply-to: Your message of "Mon, 02 Sep 1996 12:20:41 PDT." <199609021920.MAA02683@phaeton.artisoft.com> Date: Mon, 02 Sep 1996 12:34:43 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Terry Lambert writes: > > Russel and I are trying to get FreeBSD SMP working on the Intel XXPRESS > > board. We are past hte 2nd CPU ID = 2 problem. It appears that > > the apic_startup routine never (correctly) launches the 2nd CPU. looking > > at the MP spec and Intel pent. manuals it appears that the INIT/Reset > > IPI expects that a "warm start vector" will exist for starting the > > 2nd CPU, but the current code seems to ignore this. See MP spec B.4.1 > > for details. > > > > Is the purpose of this INIT/Reset IPI just to get the 2nd CPU to a known > > state, ie NOT to actually start it processing the bootMP code? > > > > Should not the code setup the BIOS warmstart vector to point @ > > a "HLT" instruction b4 using the INIT IPI? > > B.4.1 specifically applies only to the 82489DX APIC. This is incorrect. The XXPRESS box that I had REQUIRED the setting of the "warm start vector" to operate correctly. Only by completing the generic startup code was I able to get many Pentium systems to work correctly on Linux-SMP. It is actually an issue of how the BIOS/hardware combination handles CPU resets. Many Pentium boxes that have any kind of fault-tolerant capability will require this kind of startup. The Pentium Pro hardware is a little more flexible, and I have yet to see any Pentium Pro that didn't work with just one INIT IPI. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" From owner-freebsd-smp Mon Sep 2 12:38:46 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id MAA08705 for smp-outgoing; Mon, 2 Sep 1996 12:38:46 -0700 (PDT) Received: from groa.uct.ac.za (groa.uct.ac.za [137.158.128.7]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id MAA08700 for ; Mon, 2 Sep 1996 12:38:43 -0700 (PDT) Received: by groa.uct.ac.za via sendmail with stdio id for freebsd-smp@FreeBSD.ORG; Mon, 2 Sep 1996 21:38:20 +0200 (SAT) (Smail-3.2 1996-Jul-4 #1 built 1996-Jul-21) Message-Id: From: rv@groa.uct.ac.za (Russell Vincent) Subject: Re: SMP on Intel MG15 To: terry@lambert.org (Terry Lambert) Date: Mon, 2 Sep 1996 21:38:19 +0200 (SAT) Cc: smp@csn.net, freebsd-smp@FreeBSD.ORG, erich@uruk.org In-Reply-To: <199609021920.MAA02683@phaeton.artisoft.com> from "Terry Lambert" at Sep 2, 96 12:20:41 pm X-Mailer: ELM [version 2.4 PL25] Content-Type: text Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk Terry wrote: > B.4.1 specifically applies only to the 82489DX APIC. Is this a 486 > MP box you are trying to get running? Nope - it is a dual P5-133, with the ability to take quad processors. It has a dual PCI bus, dual on-board F/W SCSI controlers and various other server features, making it a monster of a machine (the case alone is a monster!). :-) > I think it's more likely that the problem is the system default state > doesn't match one of the allowable configurations, and is maybe using > a defaul configuration that we aren't handling properly. See chapter > 5 of the spec, "Default Configurations". That is quite likely, so Steve has been adding in a lot of extra code to work out the exact config. Attached is a sample output from dmesg to give you a taste of what he is up to. (Some of the values won't make much sense at this stage because they were added as tests for various stats) -Russell -------------------------------------------------------------------------- FreeBSD/SMP: boot CPU:0, 1 CPUs found 2nd CPU: p1: 5, p2: 10, p3: 15, p4,0:, p5:0 -------------------------------------------------------------------------- MP Floating Pointer Structure found in BIOS @ physical address 0x000f7ba0: signature: '_MP_, length: 16 bytes version: 1.4, checksum: 0x66 MP Virtual Wire Mode -------------------------------------------------------------------------- MP Config Table Header found @ physical address 0x000f7bb0: signature: 'PCMP, base table length: 268 version: 1.4, checksum: 0xdd OEM ID: 'INTEL ', Product ID: 'XXPRESS ' OEM table pointer: 0x00000000, OEM table size: 0 entry count: 25 local APIC address: 0xfee00000 extended table length: 220, extended table checksum: 190 -------------------------------------------------------------------------- Processor apic ID: 0, version: 16 CPU is usable, CPU is is the bootstrap processor family: 5, model: 2, stepping: 11 feature flags: 0x000003bf Processor apic ID: 2, version: 16 CPU is usable, CPU is is NOT the bootstrap processor family: 5, model: 2, stepping: 11 feature flags: 0x000003bf Bus bus ID: 0, bus type: PCI Bus bus ID: 1, bus type: PCI Bus bus ID: 18, bus type: XPRESS Bus bus ID: 19, bus type: EISA I/O APIC apic ID: 14, version: 17 APIC is usable apic address: 0xfec00000 Local INT INT type: 3, flags: 0x0005 source bus ID: 19, IRQ: 0 destination APIC ID: 255, INT: 0 Local INT INT type: 1, flags: 0x0005 source bus ID: 0, IRQ: 0 destination APIC ID: 255, INT: 1 -------------------------------------------------------------------------- FreeBSD 2.2-CURRENT #15: Sat Aug 31 08:59:12 1996 root@erroll.its.uct.ac.za:/usr/src/sys/compile/SMP Calibrating clock(s) relative to mc146818A clock... i586 clock: 133335509 Hz, i8254 clock: 1193190 Hz CPU: Pentium (129.33-MHz 586-class CPU) Origin = "GenuineIntel" Id = 0x52b Stepping=11 Features=0x3bf real memory = 67108864 (65536K bytes) [ etc .... ] From owner-freebsd-smp Mon Sep 2 12:44:19 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id MAA08915 for smp-outgoing; Mon, 2 Sep 1996 12:44:19 -0700 (PDT) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id MAA08904 for ; Mon, 2 Sep 1996 12:44:15 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id NAA06348; Mon, 2 Sep 1996 13:42:45 -0600 Message-Id: <199609021942.NAA06348@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Terry Lambert cc: freebsd-smp@FreeBSD.ORG, rv@groa.uct.ac.za, erich@uruk.org Subject: Re: SMP on Intel MG15 In-reply-to: Your message of "Mon, 02 Sep 1996 12:20:41 PDT." <199609021920.MAA02683@phaeton.artisoft.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 02 Sep 1996 13:42:45 -0600 Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk Terry, >B.4.1 specifically applies only to the 82489DX APIC. Is this a 486 >MP box you are trying to get running? I am using document "Intel MP spec 1.4, July 1995, rev -004" section B.4.1 says: INIT IPIs can be used with systems based on the 82489DX APIC, or on systems that are based on multiple Pentium (735/90, 815/100) processors. THis board has 2: CPU: Pentium (129.33-MHz 586-class CPU) Origin = "GenuineIntel" Id = 0x52b Stepping=11 Features=0x3bf these looks like 815@ parts to me... I wrote code to parse the MP config table and print the results. It finds the table in the BIOS: -------------------------------------------------------------------------- FreeBSD/SMP: boot CPU:0, 1 CPUs found 2nd CPU: p1: 5, p2: 10, p3: 15, p4,0:, p5:0 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ <<< test points added to code ------------------------------------------------------------------------- MP Floating Pointer Structure found in BIOS @ physical address 0x000f7ba0: signature: '_MP_, length: 16 bytes version: 1.4, checksum: 0x66 MP Virtual Wire Mode -------------------------------------------------------------------------- MP Config Table Header found @ physical address 0x000f7bb0: signature: 'PCMP, base table length: 268 version: 1.4, checksum: 0xdd OEM ID: 'INTEL ', Product ID: 'XXPRESS ' OEM table pointer: 0x00000000, OEM table size: 0 entry count: 25 local APIC address: 0xfee00000 extended table length: 220, extended table checksum: 190 -------------------------------------------------------------------------- Processor apic ID: 0, version: 16 CPU is usable, CPU is is the bootstrap processor family: 5, model: 2, stepping: 11 feature flags: 0x000003bf Processor apic ID: 2, version: 16 CPU is usable, CPU is is NOT the bootstrap processor family: 5, model: 2, stepping: 11 feature flags: 0x000003bf Bus bus ID: 0, bus type: PCI Bus bus ID: 1, bus type: PCI Bus bus ID: 18, bus type: XPRESS Bus bus ID: 19, bus type: EISA I/O APIC apic ID: 14, version: 17 APIC is usable apic address: 0xfec00000 Local INT INT type: 3, flags: 0x0005 source bus ID: 19, IRQ: 0 destination APIC ID: 255, INT: 0 Local INT INT type: 1, flags: 0x0005 source bus ID: 0, IRQ: 0 destination APIC ID: 255, INT: 1 -------------------------------------------------------------------------- FreeBSD 2.2-CURRENT #15: Sat Aug 31 08:59:12 1996 root@erroll.its.uct.ac.za:/usr/src/sys/compile/SMP Calibrating clock(s) relative to mc146818A clock... i586 clock: 133335509 Hz, i8254 clock: 1193190 Hz CPU: Pentium (129.33-MHz 586-class CPU) Origin = "GenuineIntel" Id = 0x52b Stepping=11 Features=0x3bf real memory = 67108864 (65536K bytes) [ etc .... ] -------------------------------------------------------------------------- >I think it's more likely that the problem is the system default state >doesn't match one of the allowable configurations, and is maybe using >a defaul configuration that we aren't handling properly. See chapter >5 of the spec, "Default Configurations". As the above printout shows, the system is using an MP configuration table. we have changed the cpu_nmbr to reflect the fact that the AP is #2, NOT #1. this change allows the kernel to boot without hanging, (default AP #1 hangs...). we are monitoring the APIC error registors and they seem happy. -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Mon Sep 2 12:52:15 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id MAA09275 for smp-outgoing; Mon, 2 Sep 1996 12:52:15 -0700 (PDT) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id MAA09268 for ; Mon, 2 Sep 1996 12:52:12 -0700 (PDT) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id MAA02809; Mon, 2 Sep 1996 12:50:20 -0700 From: Terry Lambert Message-Id: <199609021950.MAA02809@phaeton.artisoft.com> Subject: Re: SMP on Intel MG15 To: erich@uruk.org Date: Mon, 2 Sep 1996 12:50:19 -0700 (MST) Cc: terry@lambert.org, smp@csn.net, freebsd-smp@freebsd.org, rv@groa.uct.ac.za In-Reply-To: <199609021934.MAA18360@uruk.org> from "erich@uruk.org" at Sep 2, 96 12:34:43 pm X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > B.4.1 specifically applies only to the 82489DX APIC. > > This is incorrect. The XXPRESS box that I had REQUIRED the setting of the > "warm start vector" to operate correctly. Only by completing the > generic startup code was I able to get many Pentium systems to work > correctly on Linux-SMP. > > It is actually an issue of how the BIOS/hardware combination handles > CPU resets. Many Pentium boxes that have any kind of fault-tolerant > capability will require this kind of startup. The Pentium Pro hardware > is a little more flexible, and I have yet to see any Pentium Pro that > didn't work with just one INIT IPI. -------------------------------------------------------------------------- B.4.1. INIT IPI With Warm Reset This startupt technique is used with systems based on the 82489DX APIC. -------------------------------------------------------------------------- Pretty clearly references the 82489DX APIC, IMO. -------------------------------------------------------------------------- B.4.2. STARTUP IPI This startup technique is used with systems based on version 1.x or higher of the local APIC. These APIC's recognize the STARTUP IPI, which is an APIC Interprocessor Interrupt with trigger mode set to edge and delivery mode set to "110" (bits 8 through 10 of the interprocessor interrupt register). ... The operating system should not issue a STARTUP IPI to an 82489DX. A STARTUP IPI will be ignored instead of forcing the targetted processor to execute from the given address. -------------------------------------------------------------------------- I don't know if there are pre-1.x versions of the local APIC. That could explain the difference. There is also the possibility that the processors are an old stepping. For the 82489DX, I believe the APIC has to be seperate from the processor (ie: not a local APIC). Generally, this means very old and specially build Pentium systems (for processors with cruddy steppings), or 486 systems. There is also the possibility that this motherboard is simply not compliant with the specification. If this is what's going on, then the code to handle it should be special cased to make sure that people who follow on know that its use is an intentional kludge to work around a bug rather than "the way things are done" or a result of an imperfect understanding of the specification by the FreeBSD coders (another kludge). Regards, Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Mon Sep 2 12:53:06 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id MAA09320 for smp-outgoing; Mon, 2 Sep 1996 12:53:06 -0700 (PDT) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id MAA09315 for ; Mon, 2 Sep 1996 12:53:03 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id NAA06411; Mon, 2 Sep 1996 13:51:28 -0600 Message-Id: <199609021951.NAA06411@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: erich@uruk.org cc: Terry Lambert , freebsd-smp@freebsd.org, rv@groa.uct.ac.za Subject: Re: SMP on Intel MG15 In-reply-to: Your message of "Mon, 02 Sep 1996 12:34:43 PDT." <199609021934.MAA18360@uruk.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 02 Sep 1996 13:51:28 -0600 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, >This is incorrect. The XXPRESS box that I had REQUIRED the setting of the >"warm start vector" to operate correctly. Only by completing the >generic startup code was I able to get many Pentium systems to work >correctly on Linux-SMP. I had already decided to pursue this line of attack and added a vector to a "HLT" instruction, then going on to let the STARTUP IPI launch the actual startup code. this works on my (GA586DX, 2 133mHz P5s) box, but no diff on the XXPRESS machine. so I suggested Russel change HLT to "INT $2", ie NMI to this vector point, which caused the machine to hang. so I think the INIT/RESET IPI does something right. so now I am trying to make the INIT/RESET IPI vector directly to the boot code and skip the STARTUP IPI. This seems to cause my mahine to miss the 2nd CPU. Haven't sent this to Russel yet as I have been making all my changes work on my machine b4 trying on the XXPRESS. Question: isn't the STARTUP IPI redundant if the code in the INIT/RESET IPI vectors thru warmstart to the boot code? -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Mon Sep 2 13:01:27 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id NAA09725 for smp-outgoing; Mon, 2 Sep 1996 13:01:27 -0700 (PDT) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id NAA09719 for ; Mon, 2 Sep 1996 13:01:24 -0700 (PDT) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id NAA02837; Mon, 2 Sep 1996 13:00:12 -0700 From: Terry Lambert Message-Id: <199609022000.NAA02837@phaeton.artisoft.com> Subject: Re: SMP on Intel MG15 To: smp@csn.net (Steve Passe) Date: Mon, 2 Sep 1996 13:00:12 -0700 (MST) Cc: terry@lambert.org, freebsd-smp@FreeBSD.ORG, rv@groa.uct.ac.za, erich@uruk.org In-Reply-To: <199609021942.NAA06348@clem.systemsix.com> from "Steve Passe" at Sep 2, 96 01:42:45 pm X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk > Terry, > > >B.4.1 specifically applies only to the 82489DX APIC. Is this a 486 > >MP box you are trying to get running? > > I am using document "Intel MP spec 1.4, July 1995, rev -004" > > section B.4.1 says: > > INIT IPIs can be used with systems based on the 82489DX APIC, or on systems > that are based on multiple Pentium (735/90, 815/100) processors. Ah. I am quopting a 1.1 specification because the current one up for FTP won't print on HP PostScript printers, and being a FreeBSD person, I don't have any Adobe code I can run to use the PDF version instead. I'm waiting for a hardcopy via snail-mail. This is certainly a change, and meshes with what Russell quoted from his dmesg output -- the board claims it is a version 1.4 specification compliant board. > MP Floating Pointer Structure found in BIOS @ physical address 0x000f7ba0: > signature: '_MP_, length: 16 bytes > version: 1.4, checksum: 0x66 ^^^--- the damning version number ******** > >I think it's more likely that the problem is the system default state > >doesn't match one of the allowable configurations, and is maybe using > >a defaul configuration that we aren't handling properly. See chapter > >5 of the spec, "Default Configurations". > > As the above printout shows, the system is using an MP configuration table. > we have changed the cpu_nmbr to reflect the fact that the AP is #2, NOT > #1. this change allows the kernel to boot without hanging, (default AP #1 > hangs...). we are monitoring the APIC error registors and they seem happy. I suppose this still leaves the problem of how to tell which of the startup methods will work, since you don't want to stomp the warm-reset vector if you can avoid it, since the only way to reliably cause a PC-class machine to reset is to force it to real mode and cause it to jump to the warm reset vector. 8-(. Does the B.4.1 specifically state that STARTUP IPI *can't* be use on the (735/90, 815/100) processors, or give some other way to tell, from software, when it is necessary? Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Mon Sep 2 13:06:30 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id NAA09903 for smp-outgoing; Mon, 2 Sep 1996 13:06:30 -0700 (PDT) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id NAA09898 for ; Mon, 2 Sep 1996 13:06:26 -0700 (PDT) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id NAA02867; Mon, 2 Sep 1996 13:04:40 -0700 From: Terry Lambert Message-Id: <199609022004.NAA02867@phaeton.artisoft.com> Subject: Re: SMP on Intel MG15 To: smp@csn.net (Steve Passe) Date: Mon, 2 Sep 1996 13:04:40 -0700 (MST) Cc: erich@uruk.org, terry@lambert.org, freebsd-smp@freebsd.org, rv@groa.uct.ac.za In-Reply-To: <199609021951.NAA06411@clem.systemsix.com> from "Steve Passe" at Sep 2, 96 01:51:28 pm X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > Question: > > isn't the STARTUP IPI redundant if the code in the INIT/RESET IPI vectors > thru warmstart to the boot code? >From my reading of the 1.1 specification, the STARTUP IPI method is the preferred method, and the INIT IPI method is deprecated for use by systems with external APIC's. I derive this implication from the phrase "is used with systems based on version 1.x or higher of the local APIC". It's a pretty clear indicator that local APIC's, version 1.x or higher, *must* honor the STARTUP IPI. This leaves me believing that your board is 1.4 but *not* 1.1 compliant. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Mon Sep 2 13:10:40 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id NAA10008 for smp-outgoing; Mon, 2 Sep 1996 13:10:40 -0700 (PDT) Received: from orion.webspan.net (root@orion.webspan.net [206.154.70.41]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id NAA10003 for ; Mon, 2 Sep 1996 13:10:37 -0700 (PDT) Received: from localhost (gpalmer@localhost [127.0.0.1]) by orion.webspan.net (8.7.5/8.6.12) with SMTP id QAA09884; Mon, 2 Sep 1996 16:09:13 -0400 (EDT) X-Authentication-Warning: orion.webspan.net: Host gpalmer@localhost [127.0.0.1] didn't use HELO protocol To: Terry Lambert cc: freebsd-smp@FreeBSD.ORG From: "Gary Palmer" Subject: Re: SMP on Intel MG15 In-reply-to: Your message of "Mon, 02 Sep 1996 13:00:12 PDT." <199609022000.NAA02837@phaeton.artisoft.com> Date: Mon, 02 Sep 1996 16:09:13 -0400 Message-ID: <9880.841694953@orion.webspan.net> Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk Terry Lambert wrote in message ID <199609022000.NAA02837@phaeton.artisoft.com>: > Ah. I am quopting a 1.1 specification because the current one up for > FTP won't print on HP PostScript printers, and being a FreeBSD person, > I don't have any Adobe code I can run to use the PDF version instead. > I'm waiting for a hardcopy via snail-mail. xpdf Terry :-) Works like a CHARM /usr/ports/graphics/xpdf Gary -- Gary Palmer FreeBSD Core Team Member FreeBSD: Turning PC's into workstations. See http://www.FreeBSD.ORG/ for info From owner-freebsd-smp Mon Sep 2 13:11:39 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id NAA10040 for smp-outgoing; Mon, 2 Sep 1996 13:11:39 -0700 (PDT) Received: from groa.uct.ac.za (groa.uct.ac.za [137.158.128.7]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id NAA10034 for ; Mon, 2 Sep 1996 13:11:31 -0700 (PDT) Received: by groa.uct.ac.za via sendmail with stdio id for freebsd-smp@freebsd.org; Mon, 2 Sep 1996 22:11:22 +0200 (SAT) (Smail-3.2 1996-Jul-4 #1 built 1996-Jul-21) Message-Id: From: rv@groa.uct.ac.za (Russell Vincent) Subject: Re: SMP on Intel MG15 To: terry@lambert.org (Terry Lambert) Date: Mon, 2 Sep 1996 22:11:22 +0200 (SAT) Cc: smp@csn.net, erich@uruk.org, terry@lambert.org, freebsd-smp@freebsd.org In-Reply-To: <199609022004.NAA02867@phaeton.artisoft.com> from "Terry Lambert" at Sep 2, 96 01:04:40 pm X-Mailer: ELM [version 2.4 PL25] Content-Type: text Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > This leaves me believing that your board is 1.4 but *not* 1.1 compliant. Actually, it can do either - there is a BIOS setting for the version number. I have tried both settings at various times and no luck. BTW: This is a real Intel motherboard, sold by Intel themselves, and used in the top-end Compaq and HP Pentium servers, so I would hope it follows their spec. :-) -Russell From owner-freebsd-smp Mon Sep 2 13:33:49 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id NAA11102 for smp-outgoing; Mon, 2 Sep 1996 13:33:49 -0700 (PDT) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id NAA11094 for ; Mon, 2 Sep 1996 13:33:43 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id OAA06728; Mon, 2 Sep 1996 14:32:17 -0600 Message-Id: <199609022032.OAA06728@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Terry Lambert cc: freebsd-smp@FreeBSD.ORG, rv@groa.uct.ac.za, erich@uruk.org Subject: Re: SMP on Intel MG15 In-reply-to: Your message of "Mon, 02 Sep 1996 13:00:12 PDT." <199609022000.NAA02837@phaeton.artisoft.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 02 Sep 1996 14:32:17 -0600 Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk Terry, I am answering your last 3 mailings in this one... ========================================================================= #1: >I don't know if there are pre-1.x versions of the local APIC. That >could explain the difference. > >There is also the possibility that the processors are an old stepping. Processor apic ID: 0, version: 16 CPU is usable, CPU is is the bootstrap processor family: 5, model: 2, stepping: 11 looks reasonable... >There is also the possibility that this motherboard is simply not >compliant with the specification. If this is what's going on, then >the code to handle it should be special cased to make sure that >people who follow on know that its use is an intentional kludge >to work around a bug rather than "the way things are done" or a >result of an imperfect understanding of the specification by the >FreeBSD coders (another kludge). once I get it to work I will do this... ========================================================================= #2: >Ah. I am quopting a 1.1 specification because the current one up for >FTP won't print on HP PostScript printers, and being a FreeBSD person, >I don't have any Adobe code I can run to use the PDF version instead. >I'm waiting for a hardcopy via snail-mail. it worked for me by using ghostscript to an HP lj4p xpdf does a nice job with PDF files for me. beware, when you "print" files the merely end up in a file named xxx.ps, you need to manually go and "lpr xxx.ps" the result. >This is certainly a change, and meshes with what Russell quoted from >his dmesg output -- the board claims it is a version 1.4 specification >compliant board. there was a BIOS option to use version 1.1, which we tried without change. >I suppose this still leaves the problem of how to tell which of the >startup methods will work, since you don't want to stomp the warm-reset >vector if you can avoid it, since the only way to reliably cause a >PC-class machine to reset is to force it to real mode and cause it to >jump to the warm reset vector. 8-(. I solved this problem by saving the current vector and CMOS "reason" b4 altering it, then restoring them once the 2nd CPU has booted (or failed to boot). >Does the B.4.1 specifically state that STARTUP IPI *can't* be use on >the (735/90, 815/100) processors, or give some other way to tell, from >software, when it is necessary? not that I can see. I have used quite a few variations of this theme to get the XXPRESS working. they ALL work on my box, NONE do the trick on the XXPRESS! ========================================================================= #3: >From my reading of the 1.1 specification, the STARTUP IPI method is >the preferred method, and the INIT IPI method is deprecated for use >by systems with external APIC's. It is my current belief that the INIT/RESET IPI should be used even with the P5/integrated APIC for purposes of getting it back to a known real-mode, non-paging state. thus my patch to have it vector to a HLT instruction. I don't think a "standard BIOS warmstart" will necessarily affect the 2nd CPU, hence the need to do this. General algorithm: if ( 80489DX ) { set BIOS warmstart vector to 2nd CPU boot code, CMOS reason to "warmstart" do INIT/RESET /* this resets 2nd CPU (???), causing jump to BIOS, which does warmstart */ } else if ( P5/APIC ) { INTI/RESET it to a "HLT" /* get P5 to real-mode */ set vector to 2nd CPU boot code in APIC register do STARTUP IPI } >I derive this implication from the phrase "is used with systems based >on version 1.x or higher of the local APIC". It's a pretty clear >indicator that local APIC's, version 1.x or higher, *must* honor the >STARTUP IPI. > >This leaves me believing that your board is 1.4 but *not* 1.1 compliant. as I mentioned, above, we set the conformance back to version 1.1 to avoid this complication. But I don't see anything in 1.4 that would prohibit the same course of action. perhaps I'm not hereing you correctly? -- smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Mon Sep 2 13:59:23 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id NAA12602 for smp-outgoing; Mon, 2 Sep 1996 13:59:23 -0700 (PDT) Received: from uruk.org (uruk.org [198.145.95.253]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id NAA12593 for ; Mon, 2 Sep 1996 13:59:17 -0700 (PDT) From: erich@uruk.org Received: from loopback (loopback [127.0.0.1]) by uruk.org (8.7.4/8.7.3) with SMTP id NAA18595; Mon, 2 Sep 1996 13:59:05 -0700 (PDT) Message-Id: <199609022059.NAA18595@uruk.org> X-Authentication-Warning: uruk.org: Host loopback [127.0.0.1] didn't use HELO protocol To: rv@groa.uct.ac.za (Russell Vincent) cc: smp@csn.net, erich@uruk.org, terry@lambert.org, freebsd-smp@freebsd.org Subject: Re: SMP on Intel MG15 In-reply-to: Your message of "Sat, 02 Sep 1996 22:11:22 +0200." Date: Mon, 02 Sep 1996 13:59:04 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk [referring to a comment by Terry Lambert claiming it's sort of a bummer if the CPU startup sequence requires use of the warm reset vector] The implementation of the APIC in the Pentium is not really flexible enough to handle more than 2 CPUs on startup. In particular, there is no boot-time or software way that I know of to set the "secondary CPU" bit (which is what determines if the CPU pays attention to the INIT IPI, I think). I think it is a pin in the CPU socket. The Pentium Pro actually does determine it at boot-time, so no extra hardware or change in startup scheme is required. rv@groa.uct.ac.za (Russell Vincent) writes: > > This leaves me believing that your board is 1.4 but *not* 1.1 compliant. > > Actually, it can do either - there is a BIOS setting for the > version number. I have tried both settings at various times and > no luck. > > BTW: This is a real Intel motherboard, sold by Intel themselves, > and used in the top-end Compaq and HP Pentium servers, so > I would hope it follows their spec. :-) It *does* follow the spec (at least the later one. The earlier spec didn't really cover all the cases of the existing hardware. NOTE that the "recommended CPU startup" code was greatly revised in the later spec version). I would recommend only referring to the later version of the spec. For the more advanced features like bus address range entries in the MP config table, using version 1.1 of the spec will produce bad results (for machines such as the the XXPRESS, with multiple PCI buses, trying to run NT with the BIOS set to version 1.1 of the spec will produce severe problems!). Changing the version of the MPS spec in the BIOS only refers to the information placed in the BIOS tables, and perhaps assignment of interrupts to I/O APICs. I don't think it has any relevance to the operation of the hardware itself. I certainly know the startup hardware isn't influenced by this entry. If it isn't too heathen to suggest it, look at the startup code that is in the file "arch/i386/kernel/smp.c" in any Linux kernel version 1.3.98 or beyond. I wrote it and tested it against a large list of different kinds of 1.1 and 1.4 compatible hardware, and it works with nearly all of them. I'm still working on a "canonical" startup code sequence, and should have it soon on my web site. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" From owner-freebsd-smp Mon Sep 2 14:37:13 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id OAA14121 for smp-outgoing; Mon, 2 Sep 1996 14:37:13 -0700 (PDT) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id OAA14114 for ; Mon, 2 Sep 1996 14:37:10 -0700 (PDT) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id OAA03128; Mon, 2 Sep 1996 14:36:26 -0700 From: Terry Lambert Message-Id: <199609022136.OAA03128@phaeton.artisoft.com> Subject: Re: SMP on Intel MG15 To: smp@csn.net (Steve Passe) Date: Mon, 2 Sep 1996 14:36:26 -0700 (MST) Cc: terry@lambert.org, freebsd-smp@FreeBSD.ORG, rv@groa.uct.ac.za, erich@uruk.org In-Reply-To: <199609022032.OAA06728@clem.systemsix.com> from "Steve Passe" at Sep 2, 96 02:32:17 pm X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk > >I derive this implication from the phrase "is used with systems based > >on version 1.x or higher of the local APIC". It's a pretty clear > >indicator that local APIC's, version 1.x or higher, *must* honor the > >STARTUP IPI. > > > >This leaves me believing that your board is 1.4 but *not* 1.1 compliant. > > as I mentioned, above, we set the conformance back to version 1.1 to avoid > this complication. But I don't see anything in 1.4 that would prohibit > the same course of action. perhaps I'm not hereing you correctly? I think the 1.1 specification expressly leaves the behavior of the "INIT IPI With Warm Reset" method *undefined* for anything other than an 82489DX APIC. The 1.4 (apparently, from what has been quoted here) defines the behaviour, but doesn't define how to tell the difference. The evil motherboard that has the problems doesn't have an 82489DX, right? And it is running a local APIC of version 1.x or higher, right? In 1.1 compliant mode, I would expect the STARTUP IPI method to work on the board, without any change to the Warm Reset vector, or use of the INIT IPI. Clearly this fails. I might be misunderstanding the 1.1 specification, but I don't think so. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Mon Sep 2 14:55:59 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id OAA14953 for smp-outgoing; Mon, 2 Sep 1996 14:55:59 -0700 (PDT) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id OAA14942 for ; Mon, 2 Sep 1996 14:55:55 -0700 (PDT) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id OAA03155; Mon, 2 Sep 1996 14:54:23 -0700 From: Terry Lambert Message-Id: <199609022154.OAA03155@phaeton.artisoft.com> Subject: Re: SMP on Intel MG15 To: erich@uruk.org Date: Mon, 2 Sep 1996 14:54:23 -0700 (MST) Cc: rv@groa.uct.ac.za, smp@csn.net, terry@lambert.org, freebsd-smp@freebsd.org In-Reply-To: <199609022059.NAA18595@uruk.org> from "erich@uruk.org" at Sep 2, 96 01:59:04 pm X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > [referring to a comment by Terry Lambert claiming it's sort of a bummer > if the CPU startup sequence requires use of the warm reset vector] More special case code := ++bummer, in my book. Standards are *supposed* to reduce special case code by reducing the special cases, not by giving people leeway to implement whatever the heck they want. 8-(. > The implementation of the APIC in the Pentium is not really flexible enough > to handle more than 2 CPUs on startup. In particular, there is no boot-time > or software way that I know of to set the "secondary CPU" bit (which is > what determines if the CPU pays attention to the INIT IPI, I think). I > think it is a pin in the CPU socket. I think you can HALT everything, and do it one at a time with different "VV" values for the 000VV000h real mode start address as part of the STARTUP IPI with various VV's as vector. This would mean chopping the heck out of the first meg of memory, but it *is* possible to do (assuming the damn thing listens to the STARTUP IPI like it is supposed to for version 1.x or higher local APIC's). I envision something like: 000VV00h: ... ... HLT Where you then modify the actual code, and startup per CPU ID using the IPI method, to blow the bits. This would let you get to the INIT IPI method, as necessary (not that I like doing things that way, but it *is* conceptually possible to do it). > The Pentium Pro actually does determine it at boot-time, so no extra > hardware or change in startup scheme is required. Yes, it's a much nicer SMP processor, up to the stupid 5 bit limit Intel put on the thing, assuming a bogus arbitration scaling based on an antigue VM design. Silly to limit it to 32 processors, IMO. Nevertheless, the fact that it's "glueless" is a big win, and means that we are more likely to see additional sockets for processors on motherboards, especially with the main costs being in different artwork and BIOS changes (not a real production cost), and the additional socket(s) (a small real cost). You could even put them on a daughterboard that plugged into the main socket, if you could deal with the space and heat dissipation constraints. In any case, we will be seeing a lot more MP systems as a result. > It *does* follow the spec (at least the later one. The earlier spec > didn't really cover all the cases of the existing hardware. NOTE that > the "recommended CPU startup" code was greatly revised in the later > spec version). I would recommend only referring to the later version > of the spec. For the more advanced features like bus address range > entries in the MP config table, using version 1.1 of the spec will > produce bad results (for machines such as the the XXPRESS, with multiple > PCI buses, trying to run NT with the BIOS set to version 1.1 of the > spec will produce severe problems!). Yes, agreed. However, the older specification versions are important because of the existing hardware base. It's also an opportunity to divorce specification level from implementation level -- something that should be done to help out non-Intel SMP ports (the BeBox, for instance). > Changing the version of the MPS spec in the BIOS only refers to the > information placed in the BIOS tables, and perhaps assignment of > interrupts to I/O APICs. I don't think it has any relevance to the > operation of the hardware itself. I certainly know the startup hardware > isn't influenced by this entry. I think the later stuff starts up in "virtual wire mode" almost exclusively, actually. The main issue to deal with is building static MP data tables for use in the "default" cases. As you already sort of pointed out by inference, the 1.1 spec did not deal with the idea of more than 2 processors in its defaults, so it's likely that this would have to be dealt with some time soon anyway. > If it isn't too heathen to suggest it, look at the startup code that > is in the file "arch/i386/kernel/smp.c" in any Linux kernel version > 1.3.98 or beyond. I wrote it and tested it against a large list of > different kinds of 1.1 and 1.4 compatible hardware, and it works with > nearly all of them. 8-). > I'm still working on a "canonical" startup code sequence, and should > have it soon on my web site. This will be a cool thing, IMO. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Mon Sep 2 15:29:06 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id PAA16621 for smp-outgoing; Mon, 2 Sep 1996 15:29:06 -0700 (PDT) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id PAA16616 for ; Mon, 2 Sep 1996 15:29:03 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id QAA07488; Mon, 2 Sep 1996 16:27:37 -0600 Message-Id: <199609022227.QAA07488@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Terry Lambert cc: freebsd-smp@FreeBSD.ORG, rv@groa.uct.ac.za, erich@uruk.org Subject: Re: SMP on Intel MG15 In-reply-to: Your message of "Mon, 02 Sep 1996 14:36:26 PDT." <199609022136.OAA03128@phaeton.artisoft.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 02 Sep 1996 16:27:37 -0600 Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk Hi, again, answering 2 messages: =========================================================================== #1: >The evil motherboard that has the problems doesn't have an 82489DX, right? >And it is running a local APIC of version 1.x or higher, right? correct , and correct >In 1.1 compliant mode, I would expect the STARTUP IPI method to work >on the board, without any change to the Warm Reset vector, or use of >the INIT IPI. Clearly this fails. very clearly, but we just got the 2nd CPU to run to the point where it spins on "kern.smp_active == 2". this was accomplished by setting the warmstart vector to point @ the boot code (bootMP), doing the INIT/RESET IPI, AND then going on to do the STARTUP IPI. note that having the warmstart vector point to a HLT instruction had NO effect. it would appear that this machine NEEDS (as erich said, thanx erich!) the INIT/RESET IPI, AND that vector must actually run the boot code, AND it looks like it might even ignore the following STARTUP IPI!!! >I might be misunderstanding the 1.1 specification, but I don't think so As I said b4, the 1.4 document contradicts itself in several places, so I wouldn't take everything in the 1.1 document as gospel. =========================================================================== #2: >I think you can HALT everything, and do it one at a time with different >"VV" values for the 000VV000h real mode start address as part of the >STARTUP IPI with various VV's as vector. > >This would mean chopping the heck out of the first meg of memory, but >it *is* possible to do (assuming the damn thing listens to the STARTUP >IPI like it is supposed to for version 1.x or higher local APIC's). > >I envision something like: > >000VV00h: ... > ... > HLT > >Where you then modify the actual code, and startup per CPU ID using the >IPI method, to blow the bits. > >This would let you get to the INIT IPI method, as necessary (not that I >like doing things that way, but it *is* conceptually possible to do it). as I stated in the previous answer, this is essentually what I attempted to do, but it doesn't seem to work (too early to say who is not doing the right thing). my immediate goal is to get past the lockup when the second CPU is allowed to run via the sysctl! specifically what we get is: the 2nd CPU runs the boot code then waits for the sysctl. the 2nd CPU sees kern.smp_active == 2 and calls cpu_switch(curproc); the system wedges tight. Russel suggests that it is because the 2nd CPU's ID is 2, NOT 1, and this might cause the lockup. I believe he is correct, the ID is probably being used as an index into some array(s) somewhere (manywheres?). Russel and I are off chasing the source now. stay tuned... -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Mon Sep 2 17:33:03 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA21201 for smp-outgoing; Mon, 2 Sep 1996 17:33:03 -0700 (PDT) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id RAA21185 for ; Mon, 2 Sep 1996 17:33:00 -0700 (PDT) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id RAA03383; Mon, 2 Sep 1996 17:32:11 -0700 From: Terry Lambert Message-Id: <199609030032.RAA03383@phaeton.artisoft.com> Subject: Re: SMP on Intel MG15 To: smp@csn.net (Steve Passe) Date: Mon, 2 Sep 1996 17:32:11 -0700 (MST) Cc: terry@lambert.org, freebsd-smp@FreeBSD.ORG, rv@groa.uct.ac.za, erich@uruk.org In-Reply-To: <199609022227.QAA07488@clem.systemsix.com> from "Steve Passe" at Sep 2, 96 04:27:37 pm X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk > As I said b4, the 1.4 document contradicts itself in several places, so > I wouldn't take everything in the 1.1 document as gospel. Heh. "Document 1 is consistent, and document 2 is inconsistent, so don't believe document 1.". 8-) 8-). I understand we have to implement to 1.4, but it's annoying. 8-(. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Mon Sep 2 17:56:51 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA21990 for smp-outgoing; Mon, 2 Sep 1996 17:56:51 -0700 (PDT) Received: from uruk.org (uruk.org [198.145.95.253]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id RAA21985 for ; Mon, 2 Sep 1996 17:56:46 -0700 (PDT) From: erich@uruk.org Received: from loopback (loopback [127.0.0.1]) by uruk.org (8.7.4/8.7.3) with SMTP id RAA19083; Mon, 2 Sep 1996 17:56:49 -0700 (PDT) Message-Id: <199609030056.RAA19083@uruk.org> X-Authentication-Warning: uruk.org: Host loopback [127.0.0.1] didn't use HELO protocol To: Steve Passe cc: terry@lambert.org, freebsd-smp@freebsd.org, rv@groa.uct.ac.za, erich@uruk.org Subject: Re: SMP on Intel MG15 In-reply-to: Your message of "Mon, 02 Sep 1996 16:27:37 MDT." <199609022227.QAA07488@clem.systemsix.com> Date: Mon, 02 Sep 1996 17:56:49 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk [The messages are flying pretty fast on this thread... I can barely keep up ;-)] Steve Passe writes: > Hi, > > again, answering 2 messages: > > =========================================================================== > #1: > > >The evil motherboard that has the problems doesn't have an 82489DX, right? > >And it is running a local APIC of version 1.x or higher, right? > correct , and correct > > >In 1.1 compliant mode, I would expect the STARTUP IPI method to work > >on the board, without any change to the Warm Reset vector, or use of > >the INIT IPI. Clearly this fails. > very clearly, but we just got the 2nd CPU to run to the point where it > spins on "kern.smp_active == 2". this was accomplished by setting the > warmstart vector to point @ the boot code (bootMP), doing the INIT/RESET > IPI, AND then going on to do the STARTUP IPI. note that having the > warmstart vector point to a HLT instruction had NO effect. it would > appear that this machine NEEDS (as erich said, thanx erich!) the > INIT/RESET IPI, AND that vector must actually run the boot code, AND > it looks like it might even ignore the following STARTUP IPI!!! It does ignore the following STARTUP IPI completely. The Intel CPUs only pay attention to the first STARTUP IPI after a reset (I can't remember where this is documented, but I've both seen it and talked with a few of the hardware geeks who implemented it in the first place). > >I might be misunderstanding the 1.1 specification, but I don't think so > As I said b4, the 1.4 document contradicts itself in several places, so > I wouldn't take everything in the 1.1 document as gospel. I've read the 1.4 document pretty carefully. In my experience it doesn't contradict itself, but it is somewhat ambiguous in a few spots. If you have some real problems, I can probably dredge up my connections at Intel and get the complaints to them. > =========================================================================== > #2: > > >I think you can HALT everything, and do it one at a time with different > >"VV" values for the 000VV000h real mode start address as part of the > >STARTUP IPI with various VV's as vector. > > > >This would mean chopping the heck out of the first meg of memory, but > >it *is* possible to do (assuming the damn thing listens to the STARTUP > >IPI like it is supposed to for version 1.x or higher local APIC's). Huh? Since in general you have to use the warm reset vector, you can only start ONE of the other CPUs at a time. Why go around having separate bootstrap areas for each one? (you only need a separate stack for each one, and that's pretty easy) I'm getting confused here. Is this trying to state that for every context switch (or at least for some of them) you're re-running the startup sequence? The only time the startup sequence needs to be used is at bootstrap time. It's perfectly fine to send some other interrupt via IPIs to wake up a CPU out of the HLT instruction. > specifically what we get is: > > the 2nd CPU runs the boot code then waits for the sysctl. > > the 2nd CPU sees kern.smp_active == 2 and calls cpu_switch(curproc); > > the system wedges tight. > > Russel suggests that it is because the 2nd CPU's ID is 2, NOT 1, and this > might cause the lockup. I believe he is correct, the ID is probably being > used as an index into some array(s) somewhere (manywheres?). Russel and I > are off chasing the source now. Yes, as I mentioned before: -- FreeBSD-SMP presumes the boot CPU is APIC id #0, and the second CPU is APIC id #1. Yes, there are several tables where the APIC id is used as an index. -- The Intel XXPRESS box has CPUs numbered 0, 2, 3, 4. I encountered a similar problem until I changed Linux-SMP to use a "virtual CPU" numbering scheme, where there was an APIC id to internal CPU number mapping. The "virtual number" of the boot CPU was always 0, and the others were numbered consecutively from there. Since reading APIC registers takes a large number of clocks, the hit of this table lookup will be pretty minimal. Most code sequences would end up looking like: /* gets APIC id of current CPU, then maps it to the virtual CPU number */ int cpu_num = get_current_cpu_num(); ... /* do rest of stuff here using "cpu_num" */ ... This will eliminate most of the overhead with lots of SMP code I've seen which plops the "get_current_cpu_id" thing in every spot where getting to the CPUs tables is needed. (This is just carelessness on the part of the programmers in the first place, presuming that the APIC registers are as fast as internal registers) -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" From owner-freebsd-smp Mon Sep 2 18:28:01 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id SAA22698 for smp-outgoing; Mon, 2 Sep 1996 18:28:01 -0700 (PDT) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id SAA22679 for ; Mon, 2 Sep 1996 18:27:57 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id TAA08511; Mon, 2 Sep 1996 19:26:17 -0600 Message-Id: <199609030126.TAA08511@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: erich@uruk.org cc: terry@lambert.org, freebsd-smp@freebsd.org, rv@groa.uct.ac.za Subject: Re: SMP on Intel MG15 In-reply-to: Your message of "Mon, 02 Sep 1996 17:56:49 PDT." <199609030056.RAA19083@uruk.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 02 Sep 1996 19:26:16 -0600 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, >[The messages are flying pretty fast on this thread... I can barely keep up ;-)] i spent most of the day emailing someone or another. but, after a week of almost no progress we have gotten quite far. >It does ignore the following STARTUP IPI completely. > >The Intel CPUs only pay attention to the first STARTUP IPI after a reset >(I can't remember where this is documented, but I've both seen it and >talked with a few of the hardware geeks who implemented it in the first >place). spec 1.4, B.4.2 says: only one time after RESET or after an INIT IPI reception or pin assertion ^^^^^^^^^^^^^ ( I assume this means there is a hardware equivilant pin for INIT IPI) it is because of this line that I assummed the INIT IPI was for getting the 2nd CPU into a state ready for the following STARTUP IPI, and not to actually launch the bootstrap code. >I've read the 1.4 document pretty carefully. In my experience it doesn't >contradict itself, but it is somewhat ambiguous in a few spots. If you >have some real problems, I can probably dredge up my connections at Intel >and get the complaints to them. page B3, example B.1 "Universal Startr-up Algorithm": shows 2 STARTUP IPIs being run in a row this contradits the above "one time" rule. >Yes, as I mentioned before: > > -- FreeBSD-SMP presumes the boot CPU is APIC id #0, and the second CPU > is APIC id #1. Yes, there are several tables where the APIC id is > used as an index. I'm chasing those down now... -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Mon Sep 2 19:53:35 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id TAA26700 for smp-outgoing; Mon, 2 Sep 1996 19:53:35 -0700 (PDT) Received: from intercore.com (num1sun.intercore.com [199.181.243.1]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id TAA26694 for ; Mon, 2 Sep 1996 19:53:31 -0700 (PDT) Received: (robin@localhost) by intercore.com (8.7.1/8.6.4) id WAA03266; Mon, 2 Sep 1996 22:48:30 -0400 (EDT) From: Robin Cutshaw Message-Id: <199609030248.WAA03266@intercore.com> Subject: add a dell smp To: smp@freebsd.org Date: Mon, 2 Sep 1996 22:48:27 -0400 (EDT) Cc: robin@intercore.com (Robin Cutshaw) X-Mailer: ELM [version 2.4 PL21] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk I just got a dell dual-p133 up with the smp kernel. I had to get rid of the EISA 2740 scsi board and go to ncr due to what seems to be a *feature* in the driver. It now seems to be working well. We (the xfree86 core team) will probably use this box as our master sup, nfs, and freebsd build server (if it's sufficiently stable). I look forward to the smp code making it into -current... robin (robin@xfree86.org) From owner-freebsd-smp Mon Sep 2 20:09:04 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id UAA27676 for smp-outgoing; Mon, 2 Sep 1996 20:09:04 -0700 (PDT) Received: from freefall.freebsd.org (localhost.cdrom.com [127.0.0.1]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id UAA27665; Mon, 2 Sep 1996 20:08:59 -0700 (PDT) Message-Id: <199609030308.UAA27665@freefall.freebsd.org> To: Robin Cutshaw cc: smp@freebsd.org Subject: Re: add a dell smp In-reply-to: Your message of "Mon, 02 Sep 1996 22:48:27 EDT." <199609030248.WAA03266@intercore.com> Date: Mon, 02 Sep 1996 20:08:59 -0700 From: "Justin T. Gibbs" Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > >I just got a dell dual-p133 up with the smp kernel. I had to get rid of >the EISA 2740 scsi board and go to ncr due to what seems to be a *feature* >in the driver. It now seems to be working well. Can you tell me what went wrong? I'm sure the NCR will perform better anyway, but I'm always curious about problems with the aic7xxx driver. >robin (robin@xfree86.org) -- Justin T. Gibbs =========================================== FreeBSD: Turning PCs into workstations =========================================== From owner-freebsd-smp Mon Sep 2 20:29:29 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id UAA28378 for smp-outgoing; Mon, 2 Sep 1996 20:29:29 -0700 (PDT) Received: from spooky.eis.net.au (spooky.eis.net.au [203.12.171.2]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id UAA28369 for ; Mon, 2 Sep 1996 20:29:13 -0700 (PDT) Received: (from root@localhost) by spooky.eis.net.au (8.7.5/8.6.12) id NAA09682 for smp@freebsd.org; Tue, 3 Sep 1996 13:28:33 +1000 (EST) From: Ernie Elu Message-Id: <199609030328.NAA09682@spooky.eis.net.au> Subject: Compatible hardware To: smp@freebsd.org Date: Tue, 3 Sep 1996 13:28:33 +1000 (EST) X-Mailer: ELM [version 2.4ME+ PL15 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Is anyone keeping a list of what motherboards have been tested with SMP? I am in the market for a new motherboard for my hacking machine and I am looking at the economy end of dual Pentium boards that will work with FreeBSD SMP. - Ernie. From owner-freebsd-smp Tue Sep 3 06:27:38 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id GAA28713 for smp-outgoing; Tue, 3 Sep 1996 06:27:38 -0700 (PDT) Received: from intercore.com (num1sun.intercore.com [199.181.243.1]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id GAA28704 for ; Tue, 3 Sep 1996 06:27:35 -0700 (PDT) Received: (robin@localhost) by intercore.com (8.7.1/8.6.4) id JAA07163; Tue, 3 Sep 1996 09:22:35 -0400 (EDT) From: Robin Cutshaw Message-Id: <199609031322.JAA07163@intercore.com> Subject: Re: add a dell smp To: gibbs@freefall.freebsd.org (Justin T. Gibbs) Date: Tue, 3 Sep 1996 09:22:33 -0400 (EDT) Cc: robin@intercore.com, smp@freebsd.org In-Reply-To: <199609030308.UAA27665@freefall.freebsd.org> from "Justin T. Gibbs" at Sep 2, 96 08:08:59 pm X-Mailer: ELM [version 2.4 PL21] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > > > >I just got a dell dual-p133 up with the smp kernel. I had to get rid of > >the EISA 2740 scsi board and go to ncr due to what seems to be a *feature* > >in the driver. It now seems to be working well. > > Can you tell me what went wrong? I'm sure the NCR will perform better > anyway, but I'm always curious about problems with the aic7xxx driver. > I got a "panic: cpu#0 ahc:0:brkadrint, Illegal host access at seqaddr:0x0". Interestingly, linux did the same thing but solaris and NT worked fine. robin From owner-freebsd-smp Tue Sep 3 06:52:44 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id GAA29878 for smp-outgoing; Tue, 3 Sep 1996 06:52:44 -0700 (PDT) Received: from freefall.freebsd.org (localhost.cdrom.com [127.0.0.1]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id GAA29871; Tue, 3 Sep 1996 06:52:42 -0700 (PDT) Message-Id: <199609031352.GAA29871@freefall.freebsd.org> To: Robin Cutshaw cc: smp@freebsd.org Subject: Re: add a dell smp In-reply-to: Your message of "Tue, 03 Sep 1996 09:22:33 EDT." <199609031322.JAA07163@intercore.com> Date: Tue, 03 Sep 1996 06:52:42 -0700 From: "Justin T. Gibbs" Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk >> >> > >> >I just got a dell dual-p133 up with the smp kernel. I had to get rid of >> >the EISA 2740 scsi board and go to ncr due to what seems to be a *feature* >> >in the driver. It now seems to be working well. >> >> Can you tell me what went wrong? I'm sure the NCR will perform better >> anyway, but I'm always curious about problems with the aic7xxx driver. >> > >I got a "panic: cpu#0 ahc:0:brkadrint, Illegal host access at seqaddr:0x0". Disable the uha0 device via UserConfig (boot with the -c flag). In the Generic kernel, its probe stomps on the address space of the adpatec after the adaptec is successfully probed, causing the error. Once you're up, you should remove all unneeded devices in your kernel both to save space and to prevent these types of problems from cropping up when you update your kernel source. >robin -- Justin T. Gibbs =========================================== FreeBSD: Turning PCs into workstations =========================================== From owner-freebsd-smp Tue Sep 3 10:58:31 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id KAA14957 for smp-outgoing; Tue, 3 Sep 1996 10:58:31 -0700 (PDT) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id KAA14952 for ; Tue, 3 Sep 1996 10:58:29 -0700 (PDT) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id KAA04598; Tue, 3 Sep 1996 10:55:53 -0700 From: Terry Lambert Message-Id: <199609031755.KAA04598@phaeton.artisoft.com> Subject: Re: SMP on Intel MG15 To: erich@uruk.org Date: Tue, 3 Sep 1996 10:55:52 -0700 (MST) Cc: smp@csn.net, terry@lambert.org, freebsd-smp@freebsd.org, rv@groa.uct.ac.za In-Reply-To: <199609030056.RAA19083@uruk.org> from "erich@uruk.org" at Sep 2, 96 05:56:49 pm X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > >I think you can HALT everything, and do it one at a time with different > > >"VV" values for the 000VV000h real mode start address as part of the > > >STARTUP IPI with various VV's as vector. > > > > > >This would mean chopping the heck out of the first meg of memory, but > > >it *is* possible to do (assuming the damn thing listens to the STARTUP > > >IPI like it is supposed to for version 1.x or higher local APIC's). > > Huh? Since in general you have to use the warm reset vector, you can > only start ONE of the other CPUs at a time. Why go around having separate > bootstrap areas for each one? (you only need a separate stack for each > one, and that's pretty easy) I mean you hack up the code in the first meg to be processor specific to set the values, not that you have one memory area per processor (other than the stack and page tables, anyway). > I'm getting confused here. Is this trying to state that for every > context switch (or at least for some of them) you're re-running the > startup sequence? Rerunning the thing once for each AP to get the settings you wanted but said you couldn't have... you can have them, it's just a pain. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Tue Sep 3 17:50:35 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA11679 for smp-outgoing; Tue, 3 Sep 1996 17:50:35 -0700 (PDT) Received: from intercore.com (num1sun.intercore.com [199.181.243.1]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id RAA11674 for ; Tue, 3 Sep 1996 17:50:32 -0700 (PDT) Received: (robin@localhost) by intercore.com (8.7.1/8.6.4) id UAA10387; Tue, 3 Sep 1996 20:44:58 -0400 (EDT) From: Robin Cutshaw Message-Id: <199609040044.UAA10387@intercore.com> Subject: Re: add a dell smp To: gibbs@freefall.freebsd.org (Justin T. Gibbs) Date: Tue, 3 Sep 1996 20:44:56 -0400 (EDT) Cc: robin@intercore.com, smp@freebsd.org In-Reply-To: <199609031352.GAA29871@freefall.freebsd.org> from "Justin T. Gibbs" at Sep 3, 96 06:52:42 am X-Mailer: ELM [version 2.4 PL21] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > > >I got a "panic: cpu#0 ahc:0:brkadrint, Illegal host access at > seqaddr:0x0". > > Disable the uha0 device via UserConfig (boot with the -c flag). In the > Generic kernel, its probe stomps on the address space of the adpatec > after the adaptec is successfully probed, causing the error. Once you're > up, you should remove all unneeded devices in your kernel both to save > space and to prevent these types of problems from cropping up when you > update your kernel source. > I kinda suspected something like that so I made a minimal kernel but left in uha0. Thanks, robin From owner-freebsd-smp Tue Sep 3 18:01:13 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id SAA12092 for smp-outgoing; Tue, 3 Sep 1996 18:01:13 -0700 (PDT) Received: from intercore.com (num1sun.intercore.com [199.181.243.1]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id SAA12087 for ; Tue, 3 Sep 1996 18:01:09 -0700 (PDT) Received: (robin@localhost) by intercore.com (8.7.1/8.6.4) id UAA10449; Tue, 3 Sep 1996 20:56:01 -0400 (EDT) From: Robin Cutshaw Message-Id: <199609040056.UAA10449@intercore.com> Subject: vmstat on smp To: smp@freebsd.org Date: Tue, 3 Sep 1996 20:55:59 -0400 (EDT) Cc: robin@intercore.com (Robin Cutshaw) X-Mailer: ELM [version 2.4 PL21] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk I've been building the latest release of XFree86 (312Fb) on my SMP box/kernel. I booted /kernel.smp and did a vmstat -N /kernel.smp and got "vmstat: kvm_openfiles: /dev/mem: Permission denied". It seems to work fine without the "-N /kernel.smp" modifier or if run by root. I am doing a non-parallel make and see the cpu idle at 0 and most of the time used in sys. It doesn't look like the second cpu is being counted in total cpu percentage available. The "sys" figure looks suspect as well. free2cpu:robin $ vmstat 5 procs memory page disks faults cpu r b w avm fre flt re pi po fr sr s0 c0 f0 in sy cs us sy id 1 0 0 320 11844 27 0 0 0 26 0 2 0 0 236 60 2241 1 99 0 1 0 0 364 11988 372 0 0 0 364 0 17 0 0 247 708 21121 6 94 0 1 0 0 388 11460 364 0 0 0 330 0 13 0 0 243 626 22385 3 97 0 1 0 0 404 10176 281 0 0 0 220 0 13 0 0 242 515 23507 2 98 0 1 0 0 360 10056 9 0 0 0 0 0 13 0 0 243 24 30659 0 100 0 0 1 0 340 11992 30 0 0 0 128 0 1 0 0 230 33 30990 8 92 0 1 0 0 300 10868 304 0 0 0 233 0 9 0 0 239 536 24626 2 98 0 1 0 0 364 11612 246 0 0 0 272 0 11 0 0 240 404 25153 3 97 0 1 0 0 344 11696 208 0 0 0 206 0 8 0 0 237 391 26441 3 97 0 This machine is a dual-p133 with 64MB ram (and previously mentioned ncr card). robin -- ---- Robin Cutshaw internet: robin@interlabs.com Internet Labs, Inc. BellNet: 404-817-9787 "Time is just one damn thing after another" -- PBS/Nova ---- -- From owner-freebsd-smp Tue Sep 3 19:04:57 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id TAA15277 for smp-outgoing; Tue, 3 Sep 1996 19:04:57 -0700 (PDT) Received: from intercore.com (num1sun.intercore.com [199.181.243.1]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id TAA15261 for ; Tue, 3 Sep 1996 19:04:48 -0700 (PDT) Received: (robin@localhost) by intercore.com (8.7.1/8.6.4) id VAA10680; Tue, 3 Sep 1996 21:59:49 -0400 (EDT) From: Robin Cutshaw Message-Id: <199609040159.VAA10680@intercore.com> Subject: Re: vmstat on smp To: robin@intercore.com (Robin Cutshaw) Date: Tue, 3 Sep 1996 21:59:47 -0400 (EDT) Cc: smp@freebsd.org In-Reply-To: <199609040056.UAA10449@intercore.com> from "Robin Cutshaw" at Sep 3, 96 08:55:59 pm X-Mailer: ELM [version 2.4 PL21] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > > I am doing a non-parallel make and see the cpu idle at 0 and most of the > time used in sys. It doesn't look like the second cpu is being counted > in total cpu percentage available. The "sys" figure looks suspect as > well. > Well, to answer my own question, the cpuidle0 and cpuidle1 kernel idle processes are accumulating all of the sys ticks and they are being charged to them rather that to idle ticks. This explains the displayed behavior. In kern/kern_clock.c:statclock() it looks like the check for non-null process in sys time should also check for the process being cpuidle0 or cpuidle1. robin -- ---- Robin Cutshaw internet: robin@interlabs.com Internet Labs, Inc. BellNet: 404-817-9787 "Time is just one damn thing after another" -- PBS/Nova ---- -- From owner-freebsd-smp Wed Sep 4 00:06:21 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id AAA28880 for smp-outgoing; Wed, 4 Sep 1996 00:06:21 -0700 (PDT) Received: from critter.tfs.com ([140.145.230.252]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id AAA28861; Wed, 4 Sep 1996 00:06:16 -0700 (PDT) Received: from critter.tfs.com (localhost.tfs.com [127.0.0.1]) by critter.tfs.com (8.7.5/8.7.3) with ESMTP id IAA08153; Wed, 4 Sep 1996 08:33:11 +0200 (MET DST) To: Robin Cutshaw cc: smp@freebsd.org Subject: Re: vmstat on smp In-reply-to: Your message of "Tue, 03 Sep 1996 21:59:47 EDT." <199609040159.VAA10680@intercore.com> Date: Wed, 04 Sep 1996 08:33:10 +0200 Message-ID: <8151.841818790@critter.tfs.com> From: Poul-Henning Kamp Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk In message <199609040159.VAA10680@intercore.com>, Robin Cutshaw writes: >> >> >> I am doing a non-parallel make and see the cpu idle at 0 and most of the >> time used in sys. It doesn't look like the second cpu is being counted >> in total cpu percentage available. The "sys" figure looks suspect as >> well. >> > >Well, to answer my own question, the cpuidle0 and cpuidle1 kernel idle >processes are accumulating all of the sys ticks and they are being charged >to them rather that to idle ticks. This explains the displayed behavior. > >In kern/kern_clock.c:statclock() it looks like the check for non-null >process in sys time should also check for the process being cpuidle0 >or cpuidle1. Peter has better plans for the idle than to have those processes, but until then: yes. -- Poul-Henning Kamp | phk@FreeBSD.ORG FreeBSD Core-team. http://www.freebsd.org/~phk | phk@login.dknet.dk Private mailbox. whois: [PHK] | phk@ref.tfs.com TRW Financial Systems, Inc. Future will arrive by its own means, progress not so. From owner-freebsd-smp Thu Sep 5 02:23:24 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id CAA03161 for smp-outgoing; Thu, 5 Sep 1996 02:23:24 -0700 (PDT) Received: from groa.uct.ac.za (groa.uct.ac.za [137.158.128.7]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id CAA03156 for ; Thu, 5 Sep 1996 02:23:18 -0700 (PDT) Received: by groa.uct.ac.za via sendmail with stdio id for smp@freebsd.org; Thu, 5 Sep 1996 11:23:08 +0200 (SAT) (Smail-3.2 1996-Jul-4 #1 built 1996-Jul-21) Message-Id: From: rv@groa.uct.ac.za (Russell Vincent) Subject: SMP on Intel XXpress To: smp@freebsd.org Date: Thu, 5 Sep 1996 11:23:06 +0200 (SAT) X-Mailer: ELM [version 2.4 PL25] Content-Type: text Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Steve and I have come across another problem while getting the FreeBSD SMP code working on the Intel XXpress and I thought I would bounce it off the list: We seem to be able to start the second CPU now. sysctl -w kern.smp_active=2 starts the CPU, but the machine appears to freeze at this point. Steve added some debug code, which shows that the processors are still running. What was added was the following (compressed a little to save space). More comments follow the included code. ------------------------------------------------ --- sys/i386/i386/machdep.c void pp1( int d ); void pp2( int d, int x ); static int pptimes = 0; extern int smp_active; pp1( int d ) { if ( smp_active != 2 ) return; if ( pptimes > 0 ) { if ( d == 9 ) --pptimes; return; } printf( " --- point: %d ---\n", d ); if ( d == 9 ) pptimes = PP_PASSES; } void pp2( int d, int x ) { if ( smp_active != 2 ) return; if ( pptimes > 0 ) { if ( d == 9 ) --pptimes; return; } printf( " --- point: %d, val: 0x%08x ---\n", d, x ); if ( d == 9 ) pptimes = PP_PASSES; } ------------------------------------------------- --- sys/i386/i386/swtch.s /* * cpu_switch() */ ENTRY(cpu_switch) #ifdef MY_DEBUG pushl $1; call _pp1; addl $4,%esp GETCPUID(%ecx) pushl %ecx; pushl $2; call _pp2; addl $4,%esp; popl %ecx #endif /** MY_DEBUG */ /* switch to new process. first, save context as needed */ GETCURPROC(%ecx) #ifdef MY_DEBUG pushl %ecx; pushl $3; call _pp2; addl $4,%esp; popl %ecx #endif /** MY_DEBUG */ /* if no process to save, don't bother */ testl %ecx,%ecx je sw1 [ Code deleted for this message ] swtch_com: #ifdef MY_DEBUGxx pushl %ecx; pushl $4; call _pp2; addl $4,%esp; popl %ecx /* pushfl; pushl $5; call _pp2; addl $8,%esp ** Causes panic - rv ** */ #endif /** MY_DEBUG */ movl $0,%eax movl %eax,_want_resched #ifdef DIAGNOSTIC [ More code deleted for this message ] movb $0,_intr_nesting_level #ifdef SMP movl _apic_base, %eax movl APIC_ID(%eax), %eax #ifdef MY_DEBUG andl $0x0f000000, %eax # ifdef XXPRESS je 9f shrl $24,%eax decl %eax shll $24,%eax 9: # endif /** XXPRESS */ #else andl $0xff000000, %eax #endif /** MY_DEBUG */ orl PCB_MPNEST(%edx),%eax movl %eax, _mp_lock #endif /* SMP */ [ More code deleted for this message ] 2: #endif #ifdef MY_DEBUGxx pushl $9; call _pp1; addl $4,%esp #endif /** MY_DEBUG */ sti ret ------------------------------------------------- After adding that code, the console prints out the 'point' lines, with what appear to be acceptable values. What was even more interesting, was that the machine then became responsive, albeit a little sluggish. Disabling the 'MY_DEBUG' code, causes the machine to go back into it's 'frozen' state. Enabling 'MY_DEBUG' but commenting out the printf statements in pp1() and pp2() also causes the machine to 'freeze'. Increasing 'PP_PASSES' for pp1() and pp2() makes the 'point' lines print out a lot slower, this in turn made the machine even more sluggish. This, to me, sounds as if the machine is no longer handling interrupts when the second processor is enabled with sysctl. What increases my suspicion is that I can adjust the ping reponse times to the machine by adjusting 'PP_PASSES' and hence the frequency that the printf statements are called. i.e: It looks like the call to 'printf' is enabling the interrupts temporaily. The amount that is actually printed in the printf also makes a difference. More confirmation is that I have a little script that times a series of processes, starting with a single, then dual simultaneous processes, followed by triple, followed by quad. Each process is a small C program in a tight loop. A sample result after a sysctl is: Single: 3.95 real 0.00 user 1.95 sys Dual: 3.77 real 3.76 user 0.00 sys 3.87 real 3.75 user 0.05 sys Triple: 3.77 real 0.00 user 3.76 sys 5.68 real 3.72 user 0.05 sys 5.74 real 3.60 user 0.17 sys Quad: 3.79 real 3.76 user 0.00 sys 6.33 real 3.70 user 0.07 sys 7.58 real 3.72 user 0.06 sys 7.66 real 3.67 user 0.09 sys When only a single processor is running, the times show the doubling, tripling, etc of the times, which is correct. An aside comment is that the times are about 3.7 real without the SMP code in src/sys indicating that things are pretty efficient. :-) (Also remember that the kernel is sending streams of printf's to the console while dual processors are alive, using up some CPU) I realise that this is pretty vague, but it does show that only user response (i.e: interrupt driven code) is the only code being affected by the 'sluggishness' - both processors seem to be running perfectly. Have I made enough sense for anyone to prod us in the right direction? Shout if you need any more code samples or info. -Russell From owner-freebsd-smp Thu Sep 5 10:29:43 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id KAA04070 for smp-outgoing; Thu, 5 Sep 1996 10:29:43 -0700 (PDT) Received: from who.cdrom.com (who.cdrom.com [204.216.27.3]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id KAA04054 for ; Thu, 5 Sep 1996 10:29:40 -0700 (PDT) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by who.cdrom.com (8.7.5/8.6.11) with SMTP id JAA19701 for ; Thu, 5 Sep 1996 09:36:31 -0700 (PDT) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id JAA09371; Thu, 5 Sep 1996 09:34:05 -0700 From: Terry Lambert Message-Id: <199609051634.JAA09371@phaeton.artisoft.com> Subject: Re: SMP on Intel XXpress To: rv@groa.uct.ac.za (Russell Vincent) Date: Thu, 5 Sep 1996 09:34:05 -0700 (MST) Cc: smp@freebsd.org In-Reply-To: from "Russell Vincent" at Sep 5, 96 11:23:06 am X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > After adding that code, the console prints out the 'point' > lines, with what appear to be acceptable values. What was even > more interesting, was that the machine then became responsive, > albeit a little sluggish. [ ... ] > Increasing 'PP_PASSES' for pp1() and pp2() makes the 'point' lines > print out a lot slower, this in turn made the machine even more > sluggish. > > This, to me, sounds as if the machine is no longer handling interrupts > when the second processor is enabled with sysctl. What increases my > suspicion is that I can adjust the ping reponse times to the machine > by adjusting 'PP_PASSES' and hence the frequency that the printf > statements are called. i.e: It looks like the call to 'printf' is > enabling the interrupts temporaily. The amount that is actually > printed in the printf also makes a difference. By default, the ASUS and other boards do not initialize to a default of "virtual wire mode". >From your extended boot messages, it's clear that your machine is starting in virtual wire mode. This is probably the source of your interrupt problem, since it would (I think) require a rewrite of the interrupt handler to make it operate correctly in this mode. There are also a number of cache coherency issues for DMA devices once virtual wire mode is in effect; specifically, I think a bus master result might fail on some systems where the DMA completion interrupt is delivered to a processor that did not initiate the I/O. This is further bourne out by the fact that only one processor is allowed to be "in the kernel" (not in the scheduler and not in user space -- ie: for interrupt handling) at one time. This implies that a virtual wire interupt delivery to one processor may be triggered while the other processor is in in progress on handling a prior interrupt. Right now, I believe this could cause a freeze by exercising race conditions at equivalent or lower SPL on the second processor. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Thu Sep 5 13:10:55 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id NAA15268 for smp-outgoing; Thu, 5 Sep 1996 13:10:55 -0700 (PDT) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id NAA15259 for ; Thu, 5 Sep 1996 13:10:52 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id OAA27675; Thu, 5 Sep 1996 14:09:03 -0600 Message-Id: <199609052009.OAA27675@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Terry Lambert cc: rv@groa.uct.ac.za (Russell Vincent), smp@freebsd.org Subject: Re: SMP on Intel XXpress In-reply-to: Your message of "Thu, 05 Sep 1996 09:34:05 PDT." <199609051634.JAA09371@phaeton.artisoft.com> Mime-Version: 1.0 Content-Type: text/plain Date: Thu, 05 Sep 1996 14:09:03 -0600 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, Terry Lambert said: > ... > By default, the ASUS and other boards do not initialize to a default > of "virtual wire mode". > >From your extended boot messages, it's clear that your machine is > starting in virtual wire mode. my gigabyte GA586DX (2x133 P5, MP sepc 1.1) also starts in virtual wirw mode and has no problems. > This is probably the source of your interrupt problem, since it would > (I think) require a rewrite of the interrupt handler to make it operate > correctly in this mode. There are also a number of cache coherency > issues for DMA devices once virtual wire mode is in effect; specifically, > I think a bus master result might fail on some systems where the DMA > completion interrupt is delivered to a processor that did not initiate > the I/O. This is further bourne out by the fact that only one processor > is allowed to be "in the kernel" (not in the scheduler and not in user > space -- ie: for interrupt handling) at one time. This implies that > a virtual wire interupt delivery to one processor may be triggered while > the other processor is in in progress on handling a prior interrupt. > Right now, I believe this could cause a freeze by exercising race > conditions at equivalent or lower SPL on the second processor. MP spec 1.4, 3.6.2.2 says: Virtual Wire Mode provides a uniprocessor hardware environment capable of booting and running all DOS shrink-wrapped software. In virtual wire mode ... the 8259A PIC fields all interrupts ... figure 3-3 clearly shows a setup where the 1st CPU is the only one fielding INT requests. it is the symmetric I/O mode (3.6.2.3) which allows INTs to go to all processors. b4 the "sysctl kern.smp_active=2" the 2nd CPU is running a busy loop in mp_begin, polling smp_active. at this point in time it has the EI bit OFF, ie 'cli', but all the other setup has been done, ie it is in protected mode, paging, etc. after the "sysctl kern.smp_active=2" the 2nd CPU calls secondary_main() which merely prints an info message, then calls cpu_switch(), from which it never returns (thats a good thing, it shouldn't). but from this point on we have the INT loss problem. I'm totally confused... -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Thu Sep 5 14:41:46 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id OAA22121 for smp-outgoing; Thu, 5 Sep 1996 14:41:46 -0700 (PDT) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id OAA22114 for ; Thu, 5 Sep 1996 14:41:39 -0700 (PDT) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id OAA09991; Thu, 5 Sep 1996 14:38:34 -0700 From: Terry Lambert Message-Id: <199609052138.OAA09991@phaeton.artisoft.com> Subject: Re: SMP on Intel XXpress To: smp@csn.net (Steve Passe) Date: Thu, 5 Sep 1996 14:38:34 -0700 (MST) Cc: terry@lambert.org, rv@groa.uct.ac.za, smp@freebsd.org In-Reply-To: <199609052009.OAA27675@clem.systemsix.com> from "Steve Passe" at Sep 5, 96 02:09:03 pm X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > b4 the "sysctl kern.smp_active=2" the 2nd CPU is running a busy loop > in mp_begin, polling smp_active. at this point in time it has the EI > bit OFF, ie 'cli', but all the other setup has been done, ie it is in > protected mode, paging, etc. > > after the "sysctl kern.smp_active=2" the 2nd CPU calls secondary_main() > which merely prints an info message, then calls cpu_switch(), from which > it never returns (thats a good thing, it shouldn't). but from this point > on we have the INT loss problem. > > I'm totally confused... Ugh. Below that is the scheduler, and I have been hiding from the scheduler. I'm afraid I'm not going to be much help here. Poul is the guy for where you are at... Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Thu Sep 5 16:24:24 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id QAA27674 for smp-outgoing; Thu, 5 Sep 1996 16:24:24 -0700 (PDT) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id QAA27668 for ; Thu, 5 Sep 1996 16:24:21 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id RAA28908; Thu, 5 Sep 1996 17:22:44 -0600 Message-Id: <199609052322.RAA28908@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Terry Lambert cc: rv@groa.uct.ac.za, smp@freebsd.org Subject: Re: SMP on Intel XXpress In-reply-to: Your message of "Thu, 05 Sep 1996 14:38:34 PDT." <199609052138.OAA09991@phaeton.artisoft.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Sep 1996 17:22:44 -0600 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, > after the "sysctl kern.smp_active=2" the 2nd CPU calls secondary_main() > which merely prints an info message, then calls cpu_switch(), from which > it never returns (thats a good thing, it shouldn't). but from this point > on we have the INT loss problem. is anything actual using the routine in i386/isa/vector.s: FAST_INTR(irq_num, vec_name, enable_icus) I believe it has a bug where it fails to release the mp_lock for hardware INTerrupts. - -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Thu Sep 5 16:33:06 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id QAA27959 for smp-outgoing; Thu, 5 Sep 1996 16:33:06 -0700 (PDT) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id QAA27954 for ; Thu, 5 Sep 1996 16:33:03 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id RAA28947; Thu, 5 Sep 1996 17:31:27 -0600 Message-Id: <199609052331.RAA28947@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Terry Lambert cc: rv@groa.uct.ac.za, smp@freebsd.org Subject: Re: SMP on Intel XXpress In-reply-to: Your message of "Thu, 05 Sep 1996 14:38:34 PDT." <199609052138.OAA09991@phaeton.artisoft.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Sep 1996 17:31:27 -0600 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, >is anything actual using the routine in i386/isa/vector.s: > >FAST_INTR(irq_num, vec_name, enable_icus) > >I believe it has a bug where it fails to release the mp_lock for >hardware INTerrupts. never mind, I have already answered my question, this is NOT a problem. -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Thu Sep 5 16:46:19 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id QAA28745 for smp-outgoing; Thu, 5 Sep 1996 16:46:19 -0700 (PDT) Received: from spinner.DIALix.COM (spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id QAA28724 for ; Thu, 5 Sep 1996 16:46:11 -0700 (PDT) Received: from spinner.DIALix.COM (localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.7.5/8.7.3) with ESMTP id HAA16950; Fri, 6 Sep 1996 07:43:29 +0800 (WST) Message-Id: <199609052343.HAA16950@spinner.DIALix.COM> X-Mailer: exmh version 1.6.7 5/3/96 To: Steve Passe cc: Terry Lambert , rv@groa.uct.ac.za, smp@freebsd.org Subject: Re: SMP on Intel XXpress In-reply-to: Your message of "Thu, 05 Sep 1996 17:22:44 CST." <199609052322.RAA28908@clem.systemsix.com> Date: Fri, 06 Sep 1996 07:43:29 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Steve Passe wrote: > Hi, > > > after the "sysctl kern.smp_active=2" the 2nd CPU calls secondary_main() > > which merely prints an info message, then calls cpu_switch(), from which > > it never returns (thats a good thing, it shouldn't). but from this point > > on we have the INT loss problem. > > is anything actual using the routine in i386/isa/vector.s: > > FAST_INTR(irq_num, vec_name, enable_icus) > > I believe it has a bug where it fails to release the mp_lock for > hardware INTerrupts. It would not suprise me if it was.. Something certainly is amis somewhere (although different to the problem that you're looking for), I'm almost sure I'm seeing effects of occasionally having both cpu's in the kernel at simultaniously somehow.... The current kernel mutex entry/exit is a nightmare to try and do locking for... Cheers, -Peter From owner-freebsd-smp Thu Sep 5 16:52:11 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id QAA29082 for smp-outgoing; Thu, 5 Sep 1996 16:52:11 -0700 (PDT) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id QAA29076 for ; Thu, 5 Sep 1996 16:52:07 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id RAA29064; Thu, 5 Sep 1996 17:49:52 -0600 Message-Id: <199609052349.RAA29064@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Peter Wemm cc: Terry Lambert , rv@groa.uct.ac.za, smp@freebsd.org Subject: Re: SMP on Intel XXpress In-reply-to: Your message of "Fri, 06 Sep 1996 07:43:29 +0800." <199609052343.HAA16950@spinner.DIALix.COM> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Sep 1996 17:49:52 -0600 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, > It would not suprise me if it was.. Something certainly is amis somewhere I see a path in icu.s where a FAST_INTR holds the mp_lock and jmps thru an address in: ihandlers: /* addresses of interrupt handlers */ this is not my area of knowledge, where exactly is the thread of execution going at this point? -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Thu Sep 5 19:51:13 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id TAA08065 for smp-outgoing; Thu, 5 Sep 1996 19:51:13 -0700 (PDT) Received: from root.com (implode.root.com [198.145.90.17]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id TAA08059 for ; Thu, 5 Sep 1996 19:51:10 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by root.com (8.7.5/8.6.5) with SMTP id TAA05404; Thu, 5 Sep 1996 19:50:30 -0700 (PDT) Message-Id: <199609060250.TAA05404@root.com> X-Authentication-Warning: implode.root.com: Host localhost [127.0.0.1] didn't use HELO protocol To: Steve Passe cc: Peter Wemm , Terry Lambert , rv@groa.uct.ac.za, smp@freebsd.org Subject: Re: SMP on Intel XXpress In-reply-to: Your message of "Thu, 05 Sep 1996 17:49:52 MDT." <199609052349.RAA29064@clem.systemsix.com> From: David Greenman Reply-To: dg@Root.COM Date: Thu, 05 Sep 1996 19:50:30 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk >> It would not suprise me if it was.. Something certainly is amis somewhere > >I see a path in icu.s where a FAST_INTR holds the mp_lock and jmps thru >an address in: > >ihandlers: /* addresses of interrupt handlers */ > >this is not my area of knowledge, where exactly is the thread of >execution going at this point? ihandlers is an array of interrupt handlers that is indexed by the interrupt number (0-15). It is filled with the address of the "vector" routine (specified in your kernel config file on the controller lines, or dynamically during system startup in the PCI case). Does this answer your question? -DG David Greenman Core-team/Principal Architect, The FreeBSD Project From owner-freebsd-smp Thu Sep 5 20:33:34 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id UAA12002 for smp-outgoing; Thu, 5 Sep 1996 20:33:34 -0700 (PDT) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id UAA11991 for ; Thu, 5 Sep 1996 20:33:28 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id VAA00161; Thu, 5 Sep 1996 21:31:25 -0600 Message-Id: <199609060331.VAA00161@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: dg@Root.COM cc: Peter Wemm , Terry Lambert , rv@groa.uct.ac.za, smp@freebsd.org Subject: Re: SMP on Intel XXpress In-reply-to: Your message of "Thu, 05 Sep 1996 19:50:30 PDT." <199609060250.TAA05404@root.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Sep 1996 21:31:25 -0600 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, > ihandlers is an array of interrupt handlers that is indexed by the >interrupt number (0-15). It is filled with the address of the "vector" >routine (specified in your kernel config file on the controller lines, >or dynamically during system startup in the PCI case). Does this answer >your question? it appears that the strings Xresume[0-15] get expanded in the INTR macro when building the "MCOUNT_LABEL(bintr)" table by the sub-macro "__CONCAT(Xresume,irq_num):". from this point the thread goes on to call a function indexed by "*_intr_handler + (irq_num) * 4". is this a correct summation? and can the address Xresume[0-15] in 'ihandlers' ever be overwritten by a driver so that the path thru "__CONCAT(Xresume,irq_num):" would not be taken for a specific INTerrupt? and if so, could those routines return in anyway except thru "_doreti:"? specifically I am looking for a way that the mplock might get lost by the REL_MPLOCK following the Xresume[0-15] being missed: --- vector.s: #define INTR(irq_num, vec_name, icu, enable_icus, reg) \ .text ; \ SUPERALIGN_TEXT ; \ IDTVEC(vec_name) ; \ ... GET_MPLOCK ; /* SMP Spin lock */ \ ... __CONCAT(Xresume,irq_num): ; \ ... call *_intr_handler + (irq_num) * 4 ; \ ... jmp _doreti ; \ ; \ ... REL_MPLOCK ; /* SMP release global lock */ \ ... iret ihandlers: /* addresses of interrupt handlers */ /* actually resumption addresses for HWI's */ .long Xresume0, Xresume1, Xresume2, Xresume3 ... .long Xresume12, Xresume13, Xresume14, Xresume15 -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Fri Sep 6 09:45:55 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id JAA11715 for smp-outgoing; Fri, 6 Sep 1996 09:45:55 -0700 (PDT) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id JAA11697 for freebsd-smp; Fri, 6 Sep 1996 09:45:52 -0700 (PDT) Date: Fri, 6 Sep 1996 09:45:52 -0700 (PDT) From: Peter Wemm Message-Id: <199609061645.JAA11697@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys - Imported sources Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/09/06 09:45:51 sys - Imported sources Update of /home/smp/sys In directory freefall.freebsd.org:/f/peter/work/sys Revision/Branch: 1.1.1 Log Message: Import -current 07/9/96 Status: Vendor Tag: CURRENT Release Tags: v960907 I sys/CVS U sys/Makefile I sys/compile/CVS U sys/compile/.keep_me I sys/netkey/CVS U sys/netkey/key.c U sys/netkey/key.h U sys/netkey/key_debug.c U sys/netkey/key_debug.h I sys/conf/CVS U sys/conf/defines U sys/conf/files U sys/conf/files.newconf U sys/conf/newvers.sh U sys/conf/nfsswapkernel.c U sys/conf/options U sys/conf/param.c U sys/conf/systags.sh I sys/ddb/CVS U sys/ddb/db_access.c U sys/ddb/db_access.h U sys/ddb/db_aout.c U sys/ddb/db_break.c U sys/ddb/db_break.h U sys/ddb/db_command.c U sys/ddb/db_command.h U sys/ddb/db_examine.c U sys/ddb/db_expr.c U sys/ddb/db_input.c U sys/ddb/db_lex.c U sys/ddb/db_lex.h U sys/ddb/db_output.c U sys/ddb/db_output.h U sys/ddb/db_print.c U sys/ddb/db_ps.c U sys/ddb/db_run.c U sys/ddb/db_sym.c U sys/ddb/db_sym.h U sys/ddb/db_trap.c U sys/ddb/db_variables.c U sys/ddb/db_variables.h U sys/ddb/db_watch.c U sys/ddb/db_watch.h U sys/ddb/db_write_cmd.c U sys/ddb/ddb.h I sys/dev/CVS I sys/dev/aic7xxx/CVS U sys/dev/aic7xxx/aic7xxx.seq U sys/dev/aic7xxx/aic7xxx_asm.1 U sys/dev/aic7xxx/aic7xxx_asm.c U sys/dev/aic7xxx/aic7xxx_reg.h I sys/dev/vn/CVS U sys/dev/vn/vn.c I sys/dev/ccd/CVS U sys/dev/ccd/ccd.c I sys/gnu/CVS I sys/gnu/ext2fs/CVS U sys/gnu/ext2fs/COPYRIGHT.INFO U sys/gnu/ext2fs/ext2_alloc.c U sys/gnu/ext2fs/ext2_balloc.c U sys/gnu/ext2fs/ext2_extern.h U sys/gnu/ext2fs/ext2_fs.h U sys/gnu/ext2fs/ext2_fs_i.h U sys/gnu/ext2fs/ext2_fs_sb.h U sys/gnu/ext2fs/ext2_inode.c U sys/gnu/ext2fs/ext2_inode_cnv.c U sys/gnu/ext2fs/ext2_linux_balloc.c U sys/gnu/ext2fs/ext2_linux_ialloc.c U sys/gnu/ext2fs/ext2_lookup.c U sys/gnu/ext2fs/ext2_readwrite.c U sys/gnu/ext2fs/ext2_subr.c U sys/gnu/ext2fs/ext2_vfsops.c U sys/gnu/ext2fs/ext2_vnops.c U sys/gnu/ext2fs/fs.h U sys/gnu/ext2fs/i386-bitops.h I sys/gnu/i386/CVS I sys/gnu/i386/fpemul/CVS U sys/gnu/i386/fpemul/Changelog U sys/gnu/i386/fpemul/README U sys/gnu/i386/fpemul/control_w.h U sys/gnu/i386/fpemul/div_small.s U sys/gnu/i386/fpemul/errors.c U sys/gnu/i386/fpemul/exception.h U sys/gnu/i386/fpemul/fpu_arith.c U sys/gnu/i386/fpemul/fpu_asm.h U sys/gnu/i386/fpemul/fpu_aux.c U sys/gnu/i386/fpemul/fpu_emu.h U sys/gnu/i386/fpemul/fpu_entry.c U sys/gnu/i386/fpemul/fpu_etc.c U sys/gnu/i386/fpemul/fpu_proto.h U sys/gnu/i386/fpemul/fpu_system.h U sys/gnu/i386/fpemul/fpu_trig.c U sys/gnu/i386/fpemul/get_address.c U sys/gnu/i386/fpemul/load_store.c U sys/gnu/i386/fpemul/math_emu.h U sys/gnu/i386/fpemul/poly_2xm1.c U sys/gnu/i386/fpemul/poly_atan.c U sys/gnu/i386/fpemul/poly_div.s U sys/gnu/i386/fpemul/poly_l2.c U sys/gnu/i386/fpemul/poly_mul64.s U sys/gnu/i386/fpemul/poly_sin.c U sys/gnu/i386/fpemul/poly_tan.c U sys/gnu/i386/fpemul/polynomial.s U sys/gnu/i386/fpemul/reg_add_sub.c U sys/gnu/i386/fpemul/reg_compare.c U sys/gnu/i386/fpemul/reg_constant.c U sys/gnu/i386/fpemul/reg_constant.h U sys/gnu/i386/fpemul/reg_div.s U sys/gnu/i386/fpemul/reg_ld_str.c U sys/gnu/i386/fpemul/reg_mul.c U sys/gnu/i386/fpemul/reg_norm.s U sys/gnu/i386/fpemul/reg_round.s U sys/gnu/i386/fpemul/reg_u_add.s U sys/gnu/i386/fpemul/reg_u_div.s U sys/gnu/i386/fpemul/reg_u_mul.s U sys/gnu/i386/fpemul/reg_u_sub.s U sys/gnu/i386/fpemul/status_w.h U sys/gnu/i386/fpemul/version.h U sys/gnu/i386/fpemul/wm_shrx.s U sys/gnu/i386/fpemul/wm_sqrt.s I sys/gnu/i386/isa/CVS U sys/gnu/i386/isa/dgb.c U sys/gnu/i386/isa/dgbios.h U sys/gnu/i386/isa/dgfep.h U sys/gnu/i386/isa/dgreg.h U sys/gnu/i386/isa/nic3008.c U sys/gnu/i386/isa/nic3008.h U sys/gnu/i386/isa/nic3009.c U sys/gnu/i386/isa/nic3009.h U sys/gnu/i386/isa/niccyreg.h I sys/gnu/isdn/CVS U sys/gnu/isdn/if_ii.c U sys/gnu/isdn/iispy.c U sys/gnu/isdn/iitel.c U sys/gnu/isdn/iitty.c U sys/gnu/isdn/isdn.c U sys/gnu/isdn/isdn_ioctl.h I sys/i386/CVS U sys/i386/Makefile I sys/i386/apm/CVS U sys/i386/apm/apm.c U sys/i386/apm/apm_setup.h U sys/i386/apm/apm_setup.s I sys/i386/apm/apm_init/CVS U sys/i386/apm/apm_init/Makefile U sys/i386/apm/apm_init/apm_init.S U sys/i386/apm/apm_init/apm_init.inc U sys/i386/apm/apm_init/bin2asm.c U sys/i386/apm/apm_init/real_prot.S U sys/i386/apm/apm_init/real_prot.h U sys/i386/apm/apm_init/rmaouthdr U sys/i386/apm/apm_init/table.c I sys/i386/boot/CVS U sys/i386/boot/Makefile I sys/i386/boot/biosboot/CVS U sys/i386/boot/biosboot/Makefile U sys/i386/boot/biosboot/README.386BSD U sys/i386/boot/biosboot/README.MACH U sys/i386/boot/biosboot/README.serial U sys/i386/boot/biosboot/asm.S U sys/i386/boot/biosboot/asm.h U sys/i386/boot/biosboot/bios.S U sys/i386/boot/biosboot/boot.c U sys/i386/boot/biosboot/boot.h U sys/i386/boot/biosboot/boot2.S U sys/i386/boot/biosboot/disk.c U sys/i386/boot/biosboot/io.c U sys/i386/boot/biosboot/probe_keyboard.c U sys/i386/boot/biosboot/serial.S U sys/i386/boot/biosboot/start.S U sys/i386/boot/biosboot/sys.c U sys/i386/boot/biosboot/table.c I sys/i386/boot/dosboot/CVS U sys/i386/boot/dosboot/Makefile U sys/i386/boot/dosboot/ansi.h U sys/i386/boot/dosboot/boot.c U sys/i386/boot/dosboot/boot.h U sys/i386/boot/dosboot/bootinfo.h U sys/i386/boot/dosboot/cdefs.h U sys/i386/boot/dosboot/dinode.h U sys/i386/boot/dosboot/dir.h U sys/i386/boot/dosboot/dirent.h U sys/i386/boot/dosboot/disk.c U sys/i386/boot/dosboot/disklabe.h U sys/i386/boot/dosboot/dkbad.h U sys/i386/boot/dosboot/dosboot.c U sys/i386/boot/dosboot/dosboot.h U sys/i386/boot/dosboot/endian.h U sys/i386/boot/dosboot/exec.h U sys/i386/boot/dosboot/fbsdboot.c U sys/i386/boot/dosboot/fbsdboot.exe.uu U sys/i386/boot/dosboot/fbsdboot.mak U sys/i386/boot/dosboot/fs.h U sys/i386/boot/dosboot/imgact.h U sys/i386/boot/dosboot/inode.h U sys/i386/boot/dosboot/mexec.h U sys/i386/boot/dosboot/param.h U sys/i386/boot/dosboot/quota.h U sys/i386/boot/dosboot/protmod.c U sys/i386/boot/dosboot/protmod.h U sys/i386/boot/dosboot/readme U sys/i386/boot/dosboot/reboot.h U sys/i386/boot/dosboot/sys.c U sys/i386/boot/dosboot/syslimit.h U sys/i386/boot/dosboot/sysparam.h U sys/i386/boot/dosboot/types.h I sys/i386/boot/kzipboot/CVS U sys/i386/boot/kzipboot/Makefile U sys/i386/boot/kzipboot/README U sys/i386/boot/kzipboot/boot.c U sys/i386/boot/kzipboot/gzip.h U sys/i386/boot/kzipboot/head.S U sys/i386/boot/kzipboot/malloc.c U sys/i386/boot/kzipboot/misc.c U sys/i386/boot/kzipboot/tail.S U sys/i386/boot/kzipboot/unzip.c I sys/i386/boot/netboot/CVS U sys/i386/boot/netboot/3c509.c U sys/i386/boot/netboot/3c509.h U sys/i386/boot/netboot/Makefile U sys/i386/boot/netboot/bootmenu.c U sys/i386/boot/netboot/main.c U sys/i386/boot/netboot/makerom.c U sys/i386/boot/netboot/misc.c U sys/i386/boot/netboot/netboot.h U sys/i386/boot/netboot/ns8390.c U sys/i386/boot/netboot/ns8390.h U sys/i386/boot/netboot/rpc.c U sys/i386/boot/netboot/start2.S I sys/i386/conf/CVS U sys/i386/conf/GENERIC U sys/i386/conf/LINT U sys/i386/conf/Makefile.i386 U sys/i386/conf/devices.i386 C sys/i386/conf/files.i386 U sys/i386/conf/majors.i386 U sys/i386/conf/options.i386 I sys/i386/eisa/CVS U sys/i386/eisa/3c5x9.c U sys/i386/eisa/aha1742.c U sys/i386/eisa/aic7770.c U sys/i386/eisa/bt74x.c U sys/i386/eisa/eisaconf.c U sys/i386/eisa/eisaconf.h I sys/i386/i386/CVS U sys/i386/i386/autoconf.c U sys/i386/i386/cons.c U sys/i386/i386/cons.h U sys/i386/i386/db_disasm.c C sys/i386/i386/db_interface.c U sys/i386/i386/db_trace.c C sys/i386/i386/exception.s U sys/i386/i386/genassym.c U sys/i386/i386/in_cksum.c U sys/i386/i386/locore.s C sys/i386/i386/machdep.c U sys/i386/i386/math_emu.h U sys/i386/i386/math_emulate.c U sys/i386/i386/mem.c U sys/i386/i386/microtime.s U sys/i386/i386/perfmon.c U sys/i386/i386/pmap.c U sys/i386/i386/procfs_machdep.c U sys/i386/i386/support.s U sys/i386/i386/swapgeneric.c U sys/i386/i386/swtch.s U sys/i386/i386/symbols.raw C sys/i386/i386/trap.c U sys/i386/i386/sys_machdep.c U sys/i386/i386/userconfig.c U sys/i386/i386/vm_machdep.c U sys/i386/i386/identcpu.c N sys/i386/i386/i386-gdbstub.c I sys/i386/ibcs2/CVS U sys/i386/ibcs2/coff.h U sys/i386/ibcs2/ibcs2_dirent.h U sys/i386/ibcs2/ibcs2_errno.c U sys/i386/ibcs2/ibcs2_errno.h U sys/i386/ibcs2/ibcs2_fcntl.c U sys/i386/ibcs2/ibcs2_fcntl.h U sys/i386/ibcs2/ibcs2_ioctl.c U sys/i386/ibcs2/ibcs2_ioctl.h U sys/i386/ibcs2/ibcs2_ipc.c U sys/i386/ibcs2/ibcs2_ipc.h U sys/i386/ibcs2/ibcs2_isc.c U sys/i386/ibcs2/ibcs2_isc_syscall.h U sys/i386/ibcs2/ibcs2_isc_sysent.c U sys/i386/ibcs2/ibcs2_misc.c U sys/i386/ibcs2/ibcs2_mount.h U sys/i386/ibcs2/ibcs2_msg.c U sys/i386/ibcs2/ibcs2_other.c U sys/i386/ibcs2/ibcs2_poll.h U sys/i386/ibcs2/ibcs2_proto.h U sys/i386/ibcs2/ibcs2_signal.c U sys/i386/ibcs2/ibcs2_signal.h U sys/i386/ibcs2/ibcs2_socksys.c U sys/i386/ibcs2/ibcs2_socksys.h U sys/i386/ibcs2/ibcs2_stat.c U sys/i386/ibcs2/ibcs2_stat.h U sys/i386/ibcs2/ibcs2_statfs.h U sys/i386/ibcs2/ibcs2_stropts.h U sys/i386/ibcs2/ibcs2_syscall.h U sys/i386/ibcs2/ibcs2_sysent.c U sys/i386/ibcs2/ibcs2_sysi86.c U sys/i386/ibcs2/ibcs2_sysvec.c U sys/i386/ibcs2/ibcs2_termios.h U sys/i386/ibcs2/ibcs2_time.h U sys/i386/ibcs2/ibcs2_types.h U sys/i386/ibcs2/ibcs2_unistd.h U sys/i386/ibcs2/ibcs2_ustat.h U sys/i386/ibcs2/ibcs2_util.c U sys/i386/ibcs2/ibcs2_util.h U sys/i386/ibcs2/ibcs2_utime.h U sys/i386/ibcs2/ibcs2_utsname.h U sys/i386/ibcs2/ibcs2_xenix.c U sys/i386/ibcs2/ibcs2_xenix.h U sys/i386/ibcs2/ibcs2_xenix_syscall.h U sys/i386/ibcs2/ibcs2_xenix_sysent.c U sys/i386/ibcs2/imgact_coff.c U sys/i386/ibcs2/syscalls.conf U sys/i386/ibcs2/syscalls.isc U sys/i386/ibcs2/syscalls.isc.conf U sys/i386/ibcs2/syscalls.master U sys/i386/ibcs2/syscalls.xenix U sys/i386/ibcs2/syscalls.xenix.conf I sys/i386/include/CVS U sys/i386/include/apm_bios.h U sys/i386/include/apm_segments.h U sys/i386/include/asc_ioctl.h U sys/i386/include/asmacros.h U sys/i386/include/bootinfo.h U sys/i386/include/clock.h U sys/i386/include/conf.h U sys/i386/include/cons.h U sys/i386/include/console.h U sys/i386/include/cpu.h U sys/i386/include/cpufunc.h U sys/i386/include/cputypes.h U sys/i386/include/cronyx.h U sys/i386/include/db_machdep.h U sys/i386/include/devconf.h U sys/i386/include/endian.h U sys/i386/include/exec.h U sys/i386/include/float.h U sys/i386/include/floatingpoint.h U sys/i386/include/frame.h U sys/i386/include/gsc.h U sys/i386/include/ieeefp.h U sys/i386/include/ioctl_ctx.h U sys/i386/include/ioctl_fd.h U sys/i386/include/ioctl_meteor.h U sys/i386/include/ipl.h U sys/i386/include/joystick.h U sys/i386/include/laptops.h U sys/i386/include/limits.h U sys/i386/include/lpt.h U sys/i386/include/md_var.h U sys/i386/include/mouse.h U sys/i386/include/mtpr.h U sys/i386/include/npx.h U sys/i386/include/param.h U sys/i386/include/pcaudioio.h U sys/i386/include/pcb.h U sys/i386/include/pcvt_ioctl.h U sys/i386/include/perfmon.h U sys/i386/include/pmap.h U sys/i386/include/proc.h U sys/i386/include/profile.h U sys/i386/include/psl.h U sys/i386/include/qcam.h U sys/i386/include/ptrace.h U sys/i386/include/spl.h U sys/i386/include/random.h U sys/i386/include/reg.h U sys/i386/include/reloc.h U sys/i386/include/segments.h U sys/i386/include/si.h U sys/i386/include/signal.h U sys/i386/include/soundcard.h U sys/i386/include/speaker.h U sys/i386/include/specialreg.h U sys/i386/include/spigot.h U sys/i386/include/trap.h U sys/i386/include/stdarg.h U sys/i386/include/sysarch.h U sys/i386/include/tss.h U sys/i386/include/types.h U sys/i386/include/ultrasound.h U sys/i386/include/varargs.h U sys/i386/include/vmparam.h U sys/i386/include/wtio.h U sys/i386/include/ansi.h U sys/i386/include/in_cksum.h U sys/i386/include/cdk.h U sys/i386/include/comstats.h I sys/i386/include/pc/CVS U sys/i386/include/pc/display.h U sys/i386/include/pc/msdos.h I sys/i386/isa/CVS U sys/i386/isa/README.le U sys/i386/isa/aic6360.c U sys/i386/isa/asc.c U sys/i386/isa/ascreg.h U sys/i386/isa/atapi.c U sys/i386/isa/atapi.h U sys/i386/isa/b004.c U sys/i386/isa/b004.h U sys/i386/isa/bt5xx-445.c U sys/i386/isa/clock.c U sys/i386/isa/cronyx.c U sys/i386/isa/ctx.c U sys/i386/isa/ctxreg.h U sys/i386/isa/cx.c U sys/i386/isa/cxreg.h U sys/i386/isa/cy.c U sys/i386/isa/cyreg.h U sys/i386/isa/diskslice_machdep.c U sys/i386/isa/elink.c U sys/i386/isa/elink.h U sys/i386/isa/fd.c U sys/i386/isa/fdc.h U sys/i386/isa/fdreg.h U sys/i386/isa/ft.c U sys/i386/isa/ftreg.h U sys/i386/isa/gpib.c U sys/i386/isa/gpib.h U sys/i386/isa/gpibreg.h U sys/i386/isa/gsc.c U sys/i386/isa/gscreg.h U sys/i386/isa/icu.h U sys/i386/isa/icu.s U sys/i386/isa/if_ar.c U sys/i386/isa/if_arregs.h U sys/i386/isa/if_cx.c U sys/i386/isa/if_ed.c U sys/i386/isa/if_edreg.h U sys/i386/isa/if_eg.c U sys/i386/isa/if_egreg.h U sys/i386/isa/if_el.c U sys/i386/isa/if_elreg.h U sys/i386/isa/if_ep.c U sys/i386/isa/if_epreg.h U sys/i386/isa/if_fe.c U sys/i386/isa/if_fereg.h U sys/i386/isa/if_ie.c U sys/i386/isa/if_ie507.h U sys/i386/isa/if_iereg.h U sys/i386/isa/if_ix.c U sys/i386/isa/if_ixreg.h U sys/i386/isa/if_le.c U sys/i386/isa/if_lnc.c U sys/i386/isa/if_lnc.h U sys/i386/isa/if_ze.c U sys/i386/isa/if_zp.c U sys/i386/isa/if_zpreg.h U sys/i386/isa/isa.c U sys/i386/isa/isa.h U sys/i386/isa/isa_device.h U sys/i386/isa/joy.c U sys/i386/isa/kbd.h U sys/i386/isa/kbdtables.h U sys/i386/isa/labpc.c U sys/i386/isa/lpt.c U sys/i386/isa/lptreg.h U sys/i386/isa/mcd.c U sys/i386/isa/mcdreg.h U sys/i386/isa/mse.c U sys/i386/isa/ncr5380.c U sys/i386/isa/npx.c U sys/i386/isa/pcaudio.c U sys/i386/isa/pcibus.c U sys/i386/isa/pcic.h U sys/i386/isa/pcicx.c U sys/i386/isa/prof_machdep.c U sys/i386/isa/psm.c U sys/i386/isa/qcam.c U sys/i386/isa/qcamdefs.h U sys/i386/isa/qcamio.c U sys/i386/isa/qcamreg.h U sys/i386/isa/rc.c U sys/i386/isa/random_machdep.c U sys/i386/isa/rcreg.h U sys/i386/isa/rtc.h U sys/i386/isa/scd.c U sys/i386/isa/scdreg.h U sys/i386/isa/seagate.c U sys/i386/isa/si.c U sys/i386/isa/si_code.c U sys/i386/isa/sio.c U sys/i386/isa/sioreg.h U sys/i386/isa/sireg.h U sys/i386/isa/spigot.c U sys/i386/isa/spkr.c U sys/i386/isa/syscons.c U sys/i386/isa/syscons.h U sys/i386/isa/timerreg.h U sys/i386/isa/tw.c U sys/i386/isa/ultra14f.c U sys/i386/isa/vector.s U sys/i386/isa/wcd.c U sys/i386/isa/wd.c U sys/i386/isa/wd7000.c U sys/i386/isa/wdreg.h U sys/i386/isa/wt.c U sys/i386/isa/wtreg.h U sys/i386/isa/aha1542.c U sys/i386/isa/README.stl U sys/i386/isa/istallion.c U sys/i386/isa/stallion.c U sys/i386/isa/if_sr.c U sys/i386/isa/if_srregs.h I sys/i386/isa/ic/CVS U sys/i386/isa/ic/Am7990.h U sys/i386/isa/ic/am7990.h U sys/i386/isa/ic/cd1400.h U sys/i386/isa/ic/cd180.h U sys/i386/isa/ic/esp.h U sys/i386/isa/ic/hd64570.h U sys/i386/isa/ic/i8042.h U sys/i386/isa/ic/i82365.h U sys/i386/isa/ic/i8237.h U sys/i386/isa/ic/i82586.h U sys/i386/isa/ic/lemac.h U sys/i386/isa/ic/mb86960.h U sys/i386/isa/ic/ncr53400.h U sys/i386/isa/ic/ncr5380.h U sys/i386/isa/ic/nec765.h U sys/i386/isa/ic/ns16450.h U sys/i386/isa/ic/ns16550.h U sys/i386/isa/ic/scd1400.h I sys/i386/isa/matcd/CVS U sys/i386/isa/matcd/TODO U sys/i386/isa/matcd/audio.c U sys/i386/isa/matcd/creative.h U sys/i386/isa/matcd/matcd.c U sys/i386/isa/matcd/matcddrv.h U sys/i386/isa/matcd/options.h I sys/i386/isa/pcvt/CVS U sys/i386/isa/pcvt/pcvt_conf.h U sys/i386/isa/pcvt/pcvt_drv.c U sys/i386/isa/pcvt/pcvt_ext.c U sys/i386/isa/pcvt/pcvt_hdr.h U sys/i386/isa/pcvt/pcvt_kbd.c U sys/i386/isa/pcvt/pcvt_kbd.h U sys/i386/isa/pcvt/pcvt_out.c U sys/i386/isa/pcvt/pcvt_sup.c U sys/i386/isa/pcvt/pcvt_tbl.h U sys/i386/isa/pcvt/pcvt_vtf.c I sys/i386/isa/sound/CVS U sys/i386/isa/sound/CHANGELOG U sys/i386/isa/sound/COPYING U sys/i386/isa/sound/README U sys/i386/isa/sound/Readme.aedsp16 U sys/i386/isa/sound/Readme.modules U sys/i386/isa/sound/Readme.v30 U sys/i386/isa/sound/ad1848.c U sys/i386/isa/sound/ad1848_mixer.h U sys/i386/isa/sound/adlib_card.c U sys/i386/isa/sound/aedsp16.c U sys/i386/isa/sound/audio.c U sys/i386/isa/sound/coproc.h U sys/i386/isa/sound/dev_table.c U sys/i386/isa/sound/dev_table.h U sys/i386/isa/sound/dmabuf.c U sys/i386/isa/sound/finetune.h U sys/i386/isa/sound/gus_card.c U sys/i386/isa/sound/gus_hw.h U sys/i386/isa/sound/gus_linearvol.h U sys/i386/isa/sound/gus_midi.c U sys/i386/isa/sound/gus_vol.c U sys/i386/isa/sound/gus_wave.c U sys/i386/isa/sound/hex2hex.h U sys/i386/isa/sound/ics2101.c U sys/i386/isa/sound/local.h U sys/i386/isa/sound/mad16.h U sys/i386/isa/sound/midi_ctrl.h U sys/i386/isa/sound/midi_synth.c U sys/i386/isa/sound/midi_synth.h U sys/i386/isa/sound/midibuf.c U sys/i386/isa/sound/mpu401.c U sys/i386/isa/sound/opl3.c U sys/i386/isa/sound/opl3.h U sys/i386/isa/sound/os.h U sys/i386/isa/sound/pas.h U sys/i386/isa/sound/pas2_card.c U sys/i386/isa/sound/pas2_midi.c U sys/i386/isa/sound/pas2_mixer.c U sys/i386/isa/sound/pas2_pcm.c U sys/i386/isa/sound/patmgr.c U sys/i386/isa/sound/sb.h U sys/i386/isa/sound/sb16_dsp.c U sys/i386/isa/sound/sb16_midi.c U sys/i386/isa/sound/sb_card.c U sys/i386/isa/sound/sb_dsp.c U sys/i386/isa/sound/sb_midi.c U sys/i386/isa/sound/sb_mixer.c U sys/i386/isa/sound/sb_mixer.h U sys/i386/isa/sound/sequencer.c U sys/i386/isa/sound/sound.doc U sys/i386/isa/sound/sound_calls.h U sys/i386/isa/sound/sound_config.h U sys/i386/isa/sound/sound_switch.c U sys/i386/isa/sound/sound_timer.c U sys/i386/isa/sound/soundcard.c U sys/i386/isa/sound/soundvers.h U sys/i386/isa/sound/sscape.c U sys/i386/isa/sound/sys_timer.c U sys/i386/isa/sound/trix.c U sys/i386/isa/sound/tuning.h U sys/i386/isa/sound/uart6850.c U sys/i386/isa/sound/ulaw.h I sys/i386/linux/CVS U sys/i386/linux/imgact_linux.c U sys/i386/linux/linux.h U sys/i386/linux/linux_dummy.c U sys/i386/linux/linux_file.c U sys/i386/linux/linux_genassym.c U sys/i386/linux/linux_ioctl.c U sys/i386/linux/linux_ipc.c U sys/i386/linux/linux_locore.s U sys/i386/linux/linux_misc.c U sys/i386/linux/linux_proto.h U sys/i386/linux/linux_signal.c U sys/i386/linux/linux_socket.c U sys/i386/linux/linux_stats.c U sys/i386/linux/linux_syscall.h U sys/i386/linux/linux_sysent.c U sys/i386/linux/linux_sysvec.c U sys/i386/linux/linux_util.c U sys/i386/linux/linux_util.h U sys/i386/linux/syscalls.conf U sys/i386/linux/syscalls.master I sys/i386/scsi/CVS U sys/i386/scsi/93cx6.c U sys/i386/scsi/93cx6.h U sys/i386/scsi/aic7xxx.c U sys/i386/scsi/aic7xxx.h U sys/i386/scsi/bt.c U sys/i386/scsi/btreg.h I sys/isofs/CVS I sys/isofs/cd9660/CVS U sys/isofs/cd9660/TODO U sys/isofs/cd9660/TODO.hibler U sys/isofs/cd9660/cd9660_bmap.c U sys/isofs/cd9660/cd9660_lookup.c U sys/isofs/cd9660/cd9660_mount.h U sys/isofs/cd9660/cd9660_node.c U sys/isofs/cd9660/cd9660_node.h U sys/isofs/cd9660/cd9660_rrip.c U sys/isofs/cd9660/cd9660_rrip.h U sys/isofs/cd9660/cd9660_util.c U sys/isofs/cd9660/cd9660_vfsops.c U sys/isofs/cd9660/cd9660_vnops.c U sys/isofs/cd9660/iso.h U sys/isofs/cd9660/iso_rrip.h I sys/kern/CVS U sys/kern/Make.tags.inc U sys/kern/Makefile U sys/kern/imgact_aout.c U sys/kern/imgact_elf.c U sys/kern/imgact_gzip.c U sys/kern/imgact_shell.c U sys/kern/inflate.c C sys/kern/init_main.c U sys/kern/init_sysent.c U sys/kern/init_sysvec.c U sys/kern/kern_acct.c U sys/kern/kern_clock.c U sys/kern/kern_conf.c U sys/kern/kern_descrip.c U sys/kern/kern_devconf.c U sys/kern/kern_exec.c C sys/kern/kern_exit.c C sys/kern/kern_fork.c U sys/kern/kern_ktrace.c U sys/kern/kern_lkm.c U sys/kern/kern_lockf.c U sys/kern/kern_malloc.c U sys/kern/kern_ntptime.c U sys/kern/kern_physio.c U sys/kern/kern_proc.c U sys/kern/kern_prot.c U sys/kern/kern_resource.c U sys/kern/kern_sig.c U sys/kern/kern_subr.c C sys/kern/kern_synch.c U sys/kern/kern_sysctl.c U sys/kern/kern_time.c U sys/kern/kern_xxx.c U sys/kern/makesyscalls.sh U sys/kern/subr_autoconf.c U sys/kern/subr_diskslice.c U sys/kern/subr_dkbad.c U sys/kern/subr_log.c C sys/kern/subr_prf.c U sys/kern/subr_prof.c U sys/kern/subr_rlist.c U sys/kern/subr_xxx.c U sys/kern/sys_pipe.c U sys/kern/sys_generic.c U sys/kern/sys_process.c U sys/kern/sys_socket.c U sys/kern/syscalls.c U sys/kern/syscalls.master U sys/kern/sysv_ipc.c U sys/kern/sysv_msg.c U sys/kern/sysv_sem.c U sys/kern/sysv_shm.c U sys/kern/tty.c U sys/kern/tty_compat.c U sys/kern/tty_conf.c U sys/kern/tty_pty.c U sys/kern/tty_snoop.c U sys/kern/tty_subr.c U sys/kern/tty_tb.c U sys/kern/tty_tty.c U sys/kern/uipc_domain.c U sys/kern/uipc_mbuf.c U sys/kern/uipc_proto.c U sys/kern/uipc_socket.c U sys/kern/uipc_socket2.c U sys/kern/uipc_syscalls.c U sys/kern/uipc_usrreq.c C sys/kern/vfs_bio.c U sys/kern/vfs_cache.c U sys/kern/vfs_cluster.c U sys/kern/vfs_conf.c U sys/kern/vfs_init.c U sys/kern/vfs_lookup.c U sys/kern/vfs_subr.c U sys/kern/vfs_syscalls.c U sys/kern/vfs_vnops.c U sys/kern/vnode_if.sh U sys/kern/vnode_if.src U sys/kern/kern_mib.c N sys/kern/kern_shutdown.c I sys/libkern/CVS U sys/libkern/adddi3.c U sys/libkern/anddi3.c U sys/libkern/ashldi3.c U sys/libkern/ashrdi3.c U sys/libkern/bcd.c U sys/libkern/bcmp.c U sys/libkern/cmpdi2.c U sys/libkern/divdi3.c U sys/libkern/ffs.c U sys/libkern/inet_ntoa.c U sys/libkern/iordi3.c U sys/libkern/locc.c U sys/libkern/lshldi3.c U sys/libkern/lshrdi3.c U sys/libkern/mcount.c U sys/libkern/moddi3.c U sys/libkern/muldi3.c U sys/libkern/negdi2.c U sys/libkern/notdi2.c U sys/libkern/qdivrem.c U sys/libkern/qsort.c U sys/libkern/quad.h U sys/libkern/random.c U sys/libkern/rindex.c U sys/libkern/scanc.c U sys/libkern/skpc.c U sys/libkern/strcat.c U sys/libkern/strcmp.c U sys/libkern/strcpy.c U sys/libkern/strlen.c U sys/libkern/strncmp.c U sys/libkern/strncpy.c U sys/libkern/subdi3.c U sys/libkern/ucmpdi2.c U sys/libkern/udivdi3.c U sys/libkern/umoddi3.c U sys/libkern/xordi3.c U sys/libkern/index.c I sys/miscfs/CVS I sys/miscfs/deadfs/CVS U sys/miscfs/deadfs/dead_vnops.c I sys/miscfs/devfs/CVS U sys/miscfs/devfs/README U sys/miscfs/devfs/devfs_proto.h U sys/miscfs/devfs/devfs_tree.c U sys/miscfs/devfs/devfs_vfsops.c U sys/miscfs/devfs/devfs_vnops.c U sys/miscfs/devfs/devfsdefs.h U sys/miscfs/devfs/reproto.sh I sys/miscfs/fdesc/CVS U sys/miscfs/fdesc/fdesc.h U sys/miscfs/fdesc/fdesc_vfsops.c U sys/miscfs/fdesc/fdesc_vnops.c I sys/miscfs/fifofs/CVS U sys/miscfs/fifofs/fifo.h U sys/miscfs/fifofs/fifo_vnops.c I sys/miscfs/kernfs/CVS U sys/miscfs/kernfs/kernfs.h U sys/miscfs/kernfs/kernfs_vfsops.c U sys/miscfs/kernfs/kernfs_vnops.c I sys/miscfs/nullfs/CVS U sys/miscfs/nullfs/null.h U sys/miscfs/nullfs/null_subr.c U sys/miscfs/nullfs/null_vfsops.c U sys/miscfs/nullfs/null_vnops.c I sys/miscfs/portal/CVS U sys/miscfs/portal/portal.h U sys/miscfs/portal/portal_vfsops.c U sys/miscfs/portal/portal_vnops.c I sys/miscfs/procfs/CVS U sys/miscfs/procfs/README U sys/miscfs/procfs/procfs.h U sys/miscfs/procfs/procfs_ctl.c U sys/miscfs/procfs/procfs_fpregs.c U sys/miscfs/procfs/procfs_mem.c U sys/miscfs/procfs/procfs_note.c U sys/miscfs/procfs/procfs_regs.c U sys/miscfs/procfs/procfs_status.c U sys/miscfs/procfs/procfs_subr.c U sys/miscfs/procfs/procfs_vfsops.c U sys/miscfs/procfs/procfs_vnops.c U sys/miscfs/procfs/procfs_map.c U sys/miscfs/procfs/procfs_type.c I sys/miscfs/specfs/CVS U sys/miscfs/specfs/spec_vnops.c U sys/miscfs/specfs/specdev.h I sys/miscfs/umapfs/CVS U sys/miscfs/umapfs/umap.h U sys/miscfs/umapfs/umap_subr.c U sys/miscfs/umapfs/umap_vfsops.c U sys/miscfs/umapfs/umap_vnops.c I sys/miscfs/union/CVS U sys/miscfs/union/README U sys/miscfs/union/libc.fts.c U sys/miscfs/union/libc.opendir.c U sys/miscfs/union/libc.readdir.c U sys/miscfs/union/union.h U sys/miscfs/union/union_subr.c U sys/miscfs/union/union_vfsops.c U sys/miscfs/union/union_vnops.c I sys/msdosfs/CVS U sys/msdosfs/bootsect.h U sys/msdosfs/bpb.h U sys/msdosfs/denode.h U sys/msdosfs/direntry.h U sys/msdosfs/fat.h U sys/msdosfs/msdosfs_conv.c U sys/msdosfs/msdosfs_denode.c U sys/msdosfs/msdosfs_fat.c U sys/msdosfs/msdosfs_lookup.c U sys/msdosfs/msdosfs_vfsops.c U sys/msdosfs/msdosfs_vnops.c U sys/msdosfs/msdosfsmount.h I sys/net/CVS U sys/net/bpf.c U sys/net/bpf.h U sys/net/bpf_compat.h U sys/net/bpf_filter.c U sys/net/bpfdesc.h U sys/net/bsd_comp.c U sys/net/if.c U sys/net/if.h U sys/net/if_arp.h U sys/net/if_disc.c U sys/net/if_dl.h U sys/net/if_ethersubr.c U sys/net/if_fddisubr.c U sys/net/if_llc.h U sys/net/if_loop.c U sys/net/if_ppp.c U sys/net/if_ppp.h U sys/net/if_pppvar.h U sys/net/if_sl.c U sys/net/if_slvar.h U sys/net/if_sppp.h U sys/net/if_spppsubr.c U sys/net/if_tun.c U sys/net/if_tun.h U sys/net/if_types.h U sys/net/netisr.h U sys/net/ppp_comp.h U sys/net/ppp_defs.h U sys/net/ppp_tty.c U sys/net/radix.c U sys/net/radix.h U sys/net/raw_cb.c U sys/net/raw_cb.h U sys/net/raw_usrreq.c U sys/net/route.c U sys/net/route.h U sys/net/rtsock.c U sys/net/slcompress.c U sys/net/slcompress.h U sys/net/slip.h U sys/net/ethernet.h U sys/net/if_mib.c U sys/net/if_mib.h I sys/netinet/CVS U sys/netinet/icmp_var.h U sys/netinet/if_ether.c U sys/netinet/if_ether.h U sys/netinet/if_fddi.h U sys/netinet/igmp.c U sys/netinet/igmp.h U sys/netinet/igmp_var.h U sys/netinet/in.c U sys/netinet/in.h U sys/netinet/in_cksum.c U sys/netinet/in_pcb.c U sys/netinet/in_pcb.h U sys/netinet/in_proto.c U sys/netinet/in_rmx.c U sys/netinet/in_systm.h U sys/netinet/in_var.h U sys/netinet/ip.h U sys/netinet/ip_fw.c U sys/netinet/ip_fw.h U sys/netinet/ip_icmp.c U sys/netinet/ip_icmp.h U sys/netinet/ip_input.c U sys/netinet/ip_mroute.c U sys/netinet/ip_mroute.h U sys/netinet/ip_output.c U sys/netinet/ip_var.h U sys/netinet/raw_ip.c U sys/netinet/tcp.h U sys/netinet/tcp_debug.c U sys/netinet/tcp_debug.h U sys/netinet/tcp_fsm.h U sys/netinet/tcp_input.c U sys/netinet/tcp_output.c U sys/netinet/tcp_seq.h U sys/netinet/tcp_subr.c U sys/netinet/tcp_timer.c U sys/netinet/tcp_timer.h U sys/netinet/tcp_usrreq.c U sys/netinet/tcp_var.h U sys/netinet/tcpip.h U sys/netinet/udp.h U sys/netinet/udp_usrreq.c U sys/netinet/udp_var.h U sys/netinet/ip_divert.c I sys/netipx/CVS U sys/netipx/README U sys/netipx/ipx.c U sys/netipx/ipx.h U sys/netipx/ipx_cksum.c U sys/netipx/ipx_error.c U sys/netipx/ipx_error.h U sys/netipx/ipx_if.h U sys/netipx/ipx_input.c U sys/netipx/ipx_ip.c U sys/netipx/ipx_ip.h U sys/netipx/ipx_outputfl.c U sys/netipx/ipx_pcb.c U sys/netipx/ipx_pcb.h U sys/netipx/ipx_proto.c U sys/netipx/ipx_tun.c U sys/netipx/ipx_usrreq.c U sys/netipx/ipx_var.h U sys/netipx/spx.h U sys/netipx/spx_debug.c U sys/netipx/spx_debug.h U sys/netipx/spx_timer.h U sys/netipx/spx_usrreq.c U sys/netipx/spx_var.h I sys/nfs/CVS U sys/nfs/nfs.h U sys/nfs/nfs_bio.c U sys/nfs/nfs_node.c U sys/nfs/nfs_nqlease.c U sys/nfs/nfs_serv.c U sys/nfs/nfs_socket.c U sys/nfs/nfs_srvcache.c U sys/nfs/nfs_subs.c U sys/nfs/nfs_syscalls.c U sys/nfs/nfs_vfsops.c U sys/nfs/nfs_vnops.c U sys/nfs/nfsdiskless.h U sys/nfs/nfsm_subs.h U sys/nfs/nfsmount.h U sys/nfs/nfsnode.h U sys/nfs/nfsproto.h U sys/nfs/nfsrtt.h U sys/nfs/nfsrvcache.h U sys/nfs/nfsv2.h U sys/nfs/nqnfs.h U sys/nfs/rpcv2.h U sys/nfs/xdr_subs.h I sys/pccard/CVS U sys/pccard/card.h U sys/pccard/cis.h U sys/pccard/driver.h U sys/pccard/i82365.h U sys/pccard/pccard.c U sys/pccard/pcic.c U sys/pccard/skel.c U sys/pccard/slot.h N sys/pccard/pcic98reg.h I sys/pci/CVS U sys/pci/README.de U sys/pci/README.de-le U sys/pci/aic7870.c U sys/pci/bt9xx.c U sys/pci/dc21040.h U sys/pci/if_de.c U sys/pci/if_fxp.c U sys/pci/if_fxpreg.h U sys/pci/if_pdq.c U sys/pci/if_vx.c U sys/pci/if_vxreg.h U sys/pci/locate.pl U sys/pci/meteor.c U sys/pci/meteor_reg.h U sys/pci/ncr.c U sys/pci/ncrreg.h U sys/pci/pci.c U sys/pci/pcibus.h U sys/pci/pcireg.h U sys/pci/pcisupport.c U sys/pci/pcivar.h U sys/pci/pdq.c U sys/pci/pdq_os.h U sys/pci/pdqreg.h U sys/pci/wd82371.c U sys/pci/wd82371reg.h U sys/pci/if_ed_p.c U sys/pci/if_lnc_p.c I sys/scsi/CVS U sys/scsi/README U sys/scsi/cd.c U sys/scsi/ch.c U sys/scsi/od.c U sys/scsi/pt.c U sys/scsi/scsi_all.h U sys/scsi/scsi_base.c U sys/scsi/scsi_cd.h U sys/scsi/scsi_changer.h U sys/scsi/scsi_debug.h U sys/scsi/scsi_disk.h U sys/scsi/scsi_driver.c U sys/scsi/scsi_driver.h U sys/scsi/scsi_generic.h U sys/scsi/scsi_ioctl.c U sys/scsi/scsi_sense.c U sys/scsi/scsi_tape.h U sys/scsi/scsi_worm.h U sys/scsi/scsiconf.c U sys/scsi/scsiconf.h U sys/scsi/sctarg.c C sys/scsi/sd.c U sys/scsi/ssc.c U sys/scsi/st.c U sys/scsi/su.c U sys/scsi/uk.c U sys/scsi/worm.c I sys/sys/CVS U sys/sys/acct.h U sys/sys/buf.h U sys/sys/callout.h U sys/sys/cdefs.h U sys/sys/cdio.h U sys/sys/chio.h U sys/sys/clist.h U sys/sys/conf.h U sys/sys/dataacq.h U sys/sys/devconf.h U sys/sys/devfsext.h U sys/sys/device.h U sys/sys/dir.h U sys/sys/dirent.h U sys/sys/disk.h U sys/sys/disklabel.h U sys/sys/diskslice.h U sys/sys/dkbad.h U sys/sys/dkstat.h U sys/sys/dmap.h U sys/sys/domain.h U sys/sys/errno.h U sys/sys/exec.h U sys/sys/fbio.h U sys/sys/fcntl.h U sys/sys/file.h U sys/sys/filedesc.h U sys/sys/filio.h U sys/sys/ftape.h U sys/sys/gmon.h U sys/sys/imgact.h U sys/sys/imgact_aout.h U sys/sys/imgact_elf.h U sys/sys/inflate.h U sys/sys/ioccom.h U sys/sys/ioctl.h U sys/sys/ioctl_compat.h U sys/sys/ipc.h C sys/sys/kernel.h U sys/sys/ktrace.h U sys/sys/libkern.h U sys/sys/lkm.h U sys/sys/lockf.h U sys/sys/malloc.h U sys/sys/mbuf.h U sys/sys/mman.h U sys/sys/mount.h U sys/sys/msg.h U sys/sys/msgbuf.h U sys/sys/mtio.h U sys/sys/namei.h U sys/sys/param.h U sys/sys/pipe.h C sys/sys/proc.h U sys/sys/protosw.h U sys/sys/ptrace.h U sys/sys/queue.h U sys/sys/reboot.h U sys/sys/resource.h U sys/sys/resourcevar.h U sys/sys/rlist.h U sys/sys/rtprio.h U sys/sys/scsiio.h U sys/sys/select.h U sys/sys/sem.h U sys/sys/shm.h U sys/sys/signal.h U sys/sys/signalvar.h U sys/sys/snoop.h U sys/sys/socket.h U sys/sys/socketvar.h U sys/sys/sockio.h U sys/sys/stat.h U sys/sys/syscall-hide.h U sys/sys/syscall.h U sys/sys/sysctl.h U sys/sys/sysent.h U sys/sys/syslimits.h U sys/sys/syslog.h U sys/sys/sysproto.h U sys/sys/systm.h U sys/sys/tablet.h U sys/sys/termios.h U sys/sys/time.h U sys/sys/timeb.h U sys/sys/timers.h U sys/sys/times.h U sys/sys/timex.h U sys/sys/tprintf.h U sys/sys/tty.h U sys/sys/ttychars.h U sys/sys/ttycom.h U sys/sys/ttydefaults.h U sys/sys/ttydev.h U sys/sys/types.h U sys/sys/ucred.h U sys/sys/uio.h U sys/sys/un.h U sys/sys/unistd.h U sys/sys/unpcb.h U sys/sys/user.h U sys/sys/utsname.h U sys/sys/vadvise.h U sys/sys/vcmd.h U sys/sys/vlimit.h U sys/sys/vmmeter.h U sys/sys/vnioctl.h U sys/sys/vnode.h U sys/sys/vsio.h U sys/sys/wait.h U sys/sys/wormio.h U sys/sys/ccdvar.h I sys/ufs/CVS I sys/ufs/ffs/CVS U sys/ufs/ffs/ffs_alloc.c U sys/ufs/ffs/ffs_balloc.c U sys/ufs/ffs/ffs_extern.h U sys/ufs/ffs/ffs_inode.c U sys/ufs/ffs/ffs_subr.c U sys/ufs/ffs/ffs_tables.c U sys/ufs/ffs/ffs_vfsops.c U sys/ufs/ffs/ffs_vnops.c U sys/ufs/ffs/fs.h I sys/ufs/lfs/CVS U sys/ufs/lfs/README U sys/ufs/lfs/TODO U sys/ufs/lfs/lfs.h U sys/ufs/lfs/lfs_alloc.c U sys/ufs/lfs/lfs_balloc.c U sys/ufs/lfs/lfs_bio.c U sys/ufs/lfs/lfs_cksum.c U sys/ufs/lfs/lfs_debug.c U sys/ufs/lfs/lfs_extern.h U sys/ufs/lfs/lfs_inode.c U sys/ufs/lfs/lfs_segment.c U sys/ufs/lfs/lfs_subr.c U sys/ufs/lfs/lfs_syscalls.c U sys/ufs/lfs/lfs_vfsops.c U sys/ufs/lfs/lfs_vnops.c I sys/ufs/mfs/CVS U sys/ufs/mfs/mfs_extern.h U sys/ufs/mfs/mfs_vfsops.c U sys/ufs/mfs/mfs_vnops.c U sys/ufs/mfs/mfsiom.h U sys/ufs/mfs/mfsnode.h I sys/ufs/ufs/CVS U sys/ufs/ufs/dinode.h U sys/ufs/ufs/dir.h U sys/ufs/ufs/inode.h U sys/ufs/ufs/quota.h U sys/ufs/ufs/ufs_bmap.c U sys/ufs/ufs/ufs_disksubr.c U sys/ufs/ufs/ufs_extern.h U sys/ufs/ufs/ufs_ihash.c U sys/ufs/ufs/ufs_inode.c U sys/ufs/ufs/ufs_lookup.c U sys/ufs/ufs/ufs_quota.c U sys/ufs/ufs/ufs_readwrite.c U sys/ufs/ufs/ufs_vfsops.c U sys/ufs/ufs/ufs_vnops.c U sys/ufs/ufs/ufsmount.h I sys/vm/CVS U sys/vm/default_pager.c U sys/vm/default_pager.h U sys/vm/device_pager.c U sys/vm/device_pager.h U sys/vm/kern_lock.c U sys/vm/lock.h U sys/vm/pmap.h U sys/vm/swap_pager.c U sys/vm/swap_pager.h U sys/vm/vm.h U sys/vm/vm_extern.h U sys/vm/vm_fault.c U sys/vm/vm_glue.c U sys/vm/vm_inherit.h U sys/vm/vm_init.c U sys/vm/vm_kern.c U sys/vm/vm_kern.h U sys/vm/vm_map.c U sys/vm/vm_map.h U sys/vm/vm_meter.c U sys/vm/vm_mmap.c U sys/vm/vm_object.c U sys/vm/vm_object.h U sys/vm/vm_page.c U sys/vm/vm_page.h U sys/vm/vm_pageout.c U sys/vm/vm_pageout.h U sys/vm/vm_pager.c U sys/vm/vm_pager.h U sys/vm/vm_param.h U sys/vm/vm_prot.h U sys/vm/vm_swap.c U sys/vm/vm_unix.c U sys/vm/vnode_pager.c U sys/vm/vnode_pager.h I sys/netatalk/CVS U sys/netatalk/aarp.c U sys/netatalk/aarp.h U sys/netatalk/at.h U sys/netatalk/at_control.c U sys/netatalk/at_extern.h U sys/netatalk/at_proto.c U sys/netatalk/at_rmx.c U sys/netatalk/at_var.h U sys/netatalk/ddp.h U sys/netatalk/ddp_input.c U sys/netatalk/ddp_output.c U sys/netatalk/ddp_usrreq.c U sys/netatalk/ddp_var.h U sys/netatalk/endian.h U sys/netatalk/phase2.h U sys/netatalk/COPYRIGHT 14 conflicts created by this import. Use the following command to help the merge: cvs checkout -jCURRENT:yesterday -jCURRENT sys From owner-freebsd-smp Fri Sep 6 10:02:46 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id KAA13053 for smp-outgoing; Fri, 6 Sep 1996 10:02:46 -0700 (PDT) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id KAA13040 for freebsd-smp; Fri, 6 Sep 1996 10:02:44 -0700 (PDT) Date: Fri, 6 Sep 1996 10:02:44 -0700 (PDT) From: Peter Wemm Message-Id: <199609061702.KAA13040@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/conf files.i386 sys/scsi sd.c sys/sys kernel.h proc.h sys/i386/i386 db_interface.c exception.s machdep.c trap.c sys/kern init_main.c kern_exit.c kern_fork.c kern_shutdown.c kern_synch.c subr_prf.c vfs_bio.c Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/09/06 10:02:42 Modified: i386/conf files.i386 i386/i386 db_interface.c exception.s machdep.c trap.c kern init_main.c kern_exit.c kern_fork.c kern_shutdown.c kern_synch.c subr_prf.c vfs_bio.c scsi sd.c sys kernel.h proc.h Log: Resolve merge conflicts Revision Changes Path 1.6 +2 -1 sys/i386/conf/files.i386 1.4 +64 -87 sys/i386/i386/db_interface.c 1.10 +3 -1 sys/i386/i386/exception.s 1.21 +65 -173 sys/i386/i386/machdep.c 1.12 +4 -3 sys/i386/i386/trap.c 1.20 +2 -17 sys/kern/init_main.c 1.6 +82 -1 sys/kern/kern_exit.c 1.7 +81 -3 sys/kern/kern_fork.c 1.2 +4 -0 sys/kern/kern_shutdown.c 1.9 +3 -3 sys/kern/kern_synch.c 1.8 +3 -62 sys/kern/subr_prf.c 1.8 +33 -3 sys/kern/vfs_bio.c 1.6 +4 -1 sys/scsi/sd.c 1.7 +5 -4 sys/sys/kernel.h 1.11 +4 -1 sys/sys/proc.h From owner-freebsd-smp Fri Sep 6 18:41:58 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id SAA15672 for smp-outgoing; Fri, 6 Sep 1996 18:41:58 -0700 (PDT) Received: from falco.kuci.uci.edu (root@falco.kuci.uci.edu [128.195.131.10]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id SAA15667 for ; Fri, 6 Sep 1996 18:41:55 -0700 (PDT) Received: from localhost (bmehling@localhost [127.0.0.1]) by falco.kuci.uci.edu (8.6.12/8.6.12) with SMTP id SAA29278 for ; Fri, 6 Sep 1996 18:45:12 GMT Message-Id: <199609061845.SAA29278@falco.kuci.uci.edu> X-Authentication-Warning: falco.kuci.uci.edu: Host localhost didn't use HELO protocol To: freebsd-smp@freebsd.org Subject: subscribe Date: Fri, 06 Sep 1996 18:45:11 +0000 From: Ben Mehling Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk subscribe