From owner-freebsd-amd64@FreeBSD.ORG Tue Jul 13 19:56:03 2004 Return-Path: Delivered-To: freebsd-amd64@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5BCAB16A4CE for ; Tue, 13 Jul 2004 19:56:03 +0000 (GMT) Received: from mail4.speakeasy.net (mail4.speakeasy.net [216.254.0.204]) by mx1.FreeBSD.org (Postfix) with ESMTP id 31DD943D2F for ; Tue, 13 Jul 2004 19:56:03 +0000 (GMT) (envelope-from jhb@FreeBSD.org) Received: (qmail 16324 invoked from network); 13 Jul 2004 19:56:02 -0000 Received: from dsl027-160-063.atl1.dsl.speakeasy.net (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) encrypted SMTP for ; 13 Jul 2004 19:56:00 -0000 Received: from 10.50.41.229 (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.11/8.12.11) with ESMTP id i6DJtqsh018980; Tue, 13 Jul 2004 15:55:52 -0400 (EDT) (envelope-from jhb@FreeBSD.org) From: John Baldwin To: freebsd-amd64@FreeBSD.org Date: Tue, 13 Jul 2004 15:57:05 -0400 User-Agent: KMail/1.6 References: <06b401c46591$49a2b600$471b3dd4@digiware.nl> <02b401c468dc$7c59a4c0$471b3dd4@digiware.nl> In-Reply-To: <02b401c468dc$7c59a4c0$471b3dd4@digiware.nl> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200407131557.05961.jhb@FreeBSD.org> X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on server.baldwin.cx Subject: Re: Help my SMP is broken??? X-BeenThere: freebsd-amd64@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Porting FreeBSD to the AMD64 platform List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jul 2004 19:56:03 -0000 On Tuesday 13 July 2004 09:22 am, Willem Jan Withagen wrote: > > I'm trying to boot my dual opteron box with a recent kernel. > > Without ACPI is boots single processor, with ACPI I get the stuff > > downbelow. > > > > SMAP type=01 base=0000000000000000 len=00000000000a0000 > > SMAP type=02 base=00000000000f0000 len=0000000000010000 > > SMAP type=02 base=00000000fec00000 len=0000000001400000 > > SMAP type=02 base=000000007fef0000 len=0000000000010000 > > SMAP type=01 base=0000000000100000 len=000000007fde0000 > > SMAP type=03 base=000000007fee3000 len=000000000000d000 > > SMAP type=04 base=000000007fee0000 len=0000000000003000 > > Copyright (c) 1992-2004 The FreeBSD Project. > > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > > The Regents of the University of California. All rights reserved. > > FreeBSD 5.2-CURRENT #57: Fri Jul 9 10:46:05 CEST 2004 > > root@opteron.digiware.nl:/usr/obj/home2/src/sys/OPTERON.amd64 > > WARNING: WITNESS option enabled, expect reduced performance. > > WARNING: DIAGNOSTIC option enabled, expect reduced performance. > > Preloaded elf kernel "/boot/kernel/kernel" at 0xffffffff806f1000. > > Calibrating clock(s) ... i8254 clock: 1193255 Hz > > CLK_USE_I8254_CALIBRATION not specified - using default frequency > > Timecounter "i8254" frequency 1193182 Hz quality 0 > > Calibrating TSC clock ... TSC clock: 1603648612 Hz > > CPU: AMD Opteron(tm) Processor 242 (1603.65-MHz K8-class CPU) > > Origin = "AuthenticAMD" Id = 0xf51 Stepping = 1 > > Features=0x78bfbff,CM > > > OV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2> > > AMD Features=0xe0500800 > > L1 2MB data TLB: 8 entries, fully associative > > L1 2MB instruction TLB: 8 entries, fully associative > > L1 4KB data TLB: 32 entries, fully associative > > L1 4KB instruction TLB: 32 entries, fully associative > > L1 data cache: 64 kbytes, 64 bytes/line, 1 lines/tag, 2-way associative > > L1 instruction cache: 64 kbytes, 64 bytes/line, 1 lines/tag, 2-way > > associative L2 2MB unified TLB: 0 entries, disabled/not present > > L2 4KB data TLB: 512 entries, 4-way associative > > L2 4KB instruction TLB: 512 entries, 4-way associative > > L2 unified cache: 1024 kbytes, 64 bytes/line, 1 lines/tag, 16-way > > associative real memory = 2146304000 (2046 MB) > > Physical memory chunk(s): > > 0x0000000000001000 - 0x000000000009cfff, 638976 bytes (156 pages) > > 0x0000000000777000 - 0x000000007c2b7fff, 2075398144 bytes (506689 pages) > > avail memory = 2062643200 (1967 MB) > > ACPI APIC Table: > > panic: AP #1 (PHY# 1) failed! > > cpuid = 0; > > Stack backtrace: > > backtrace() at backtrace+0x17 > > panic() at panic+0x1d2 > > start_all_aps() at start_all_aps+0x187 > > cpu_mp_start() at cpu_mp_start+0x199 > > mp_start() at mp_start+0x54 > > mi_startup() at mi_startup+0xb8 > > btext() at btext+0x2c > > Debugger("panic") > > Stopped at Debugger+0x4d: xchgl %ebx,0x27d461 > > I'm trying to figure this one out..... > So I added a printf into init_secondary. This is namely the place where a > variable (mp_naps) is incremented to make it out of the waiting loop in > start_all_aps. > > BUT we never get to init_secondary, which I expected to be called from an > interrupt routine. ini_secondary is called from an asm-sequence in mpboot.S > which suggest that it is the first thing to be called once we get into > 64bit mode??? > > Is there anywhere a description on how booting is done on UP or SMP (let > alone SMP amd64??) Basically, when a CPU is started up, it starts up in 8086-style real mode executing the code in mpboot.s. That code brings the computer into protected mode and eventually into the same virtual addresses used by the kernel (the mpboot code is dual-mapped to allow for this transition). At that point the AP can safely call into the kernel and it calls init_secondary(). If the AP faults during the early boot code it will basically die and never get into init_secondary(). Also, if the IPI is somehow lost then it will never get into init_secondary(). -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve" = http://www.FreeBSD.org