From owner-freebsd-ppc@freebsd.org Sun Feb 17 00:30:08 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E971814D8C5D for ; Sun, 17 Feb 2019 00:30:07 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-35.consmr.mail.ne1.yahoo.com (sonic317-35.consmr.mail.ne1.yahoo.com [66.163.184.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A825E742D9 for ; Sun, 17 Feb 2019 00:30:06 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: AzkdWtIVM1n4o9UohMlAU.w0Wt6REjqILh5EuXC9d5Jf_g8rZWTwdFdrX.CakmN 5TsXcvJZxQ._mTImHBU9ja5vj7Q6f.BHZ.6joygQFZKDdzmfhCE.c6Au4MU2MtW_egvj7hC_AJX5 ueZ.kWbtRaBCPf557vdLFFmbNoTfxcPO7KZyW4IYFFE2vVeR1DFbWbyuyL8_m0kbh7g43zps4G0v UmomvnXkGsydejLiaSTbdJv4sQv10HTTeYzLT21G2VeFcCZ6L7Y_FHs_jpNJh5KllIv.iTWng7ej Sy85nBZdBQ8KhDb5PJ6jDl1ZFagJBb88IEm7K045b.EpJ8K5rhK58ipwnq2wm6IvTtLRwwjlZTZ2 ZwXeUukyaSVub_TNZahDcZUm8D.KyAuZ63Ed40gZQS1m_aBYMLyjVsPpi322K.SolC3asx0zmGjS 0z6ukqNR9wZufTQldPsmE0I.fH0mZwI8vOKl1y7wAVE.E5Uppk5.B9rnfZVARyrFAqjF9xVWfGI1 6Tl25ajsXnhZDONl91lVGKIW47zncRBXhGppEut7gNqN_g7r.bGqJ_ddoFHAyeNHgovayMFzRnxI Y1Mio7XwsjdlcKAT2fZRDEGhd5EMu6TnN19epK1Y2oGgfYpAJRXePZYL_iID9jV7l_k8Emf4wry1 VtBBRYsK36g7jOoGUkQQaz1Kahzkl1SpxvMiXS6vm.Y2qkxEMQDDcxlMJI8Wn6aOO8qX_4mLXyAc n6BukdTwtB.RCwEy8aUxkmHj_Yvtn_masLXuNXXbFq2GlV7U_Htqz2UZkU_CBCrVUwBFQUA080F4 7lX0q8geVNEwk95OAWlTJanZeTyguKujXurpEmPeoDlgYSORBaTgfjt2vYCFR9SV90_0Ijpu2PEu yT_fyxySA42vHC__TMhdWGT.KisAq3tT3jG.lvVeDWgLKfvt.bmqqa7Sfu9k3NN5x_a36FFDB5Vi isszMSBGE91TSfhpuc8qscWJLeyqLizSjFqJnebcxXsRMYGfQOid4HM.l23C3TwJIw11SHQfrEVn FiDFfARTULIlO3Uih4Mld6A7fCisxvVxvA7MI5WT2nrGQymOgrzkKX2F9Zq8- Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.ne1.yahoo.com with HTTP; Sun, 17 Feb 2019 00:29:59 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.113]) ([67.170.167.181]) by smtp415.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 88dc9ec016cf75fec7969a98b9017f50; Sun, 17 Feb 2019 00:29:54 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: Cleaned -up evidence about the PowerMac G5 multiprocessor boot hang ups with the modern VM_MAX_KERNEL_ADDRESS value Message-Id: Date: Sat, 16 Feb 2019 16:29:52 -0800 Cc: FreeBSD PowerPC ML To: Justin Hibbits X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: A825E742D9 X-Spamd-Bar: + X-Spamd-Result: default: False [1.94 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.23)[0.232,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(1.21)[ip: (4.15), ipnet: 66.163.184.0/21(1.10), asn: 36646(0.88), country: US(-0.07)]; NEURAL_SPAM_MEDIUM(0.87)[0.869,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.14)[0.140,0]; RCVD_IN_DNSWL_NONE(0.00)[46.184.163.66.list.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 Feb 2019 00:30:08 -0000 I eliminated a bunch of reports that gave information that was the same for hang-ups and successful boots. So this is starting over with less material to go through. For an unsuccessful boot: (Values like 0x10, 0x25, 0x51 are use as recorded labels for points in = the code.) Adding CPU 0, hwref=3Dcd38, awake=3D1 Waking up CPU 3 (dev=3Dc480) After reset 4&0 for CPU 3, hwref=3Dc480, awake=3D0, *(unsigned = long*)0xc0000000000000f0=3D0x25 After attempted wait for awake CPU 3, hwref=3Dc480, awake=3D0, = *(unsigned long*)0xc0000000000000f0=3D0x25 Waking up CPU 2 (dev=3Dc768) So neither pc_awake nor *0xc0000000000000f0 have machdep_ap_bootstrap based values as seen by CPU 0 (from trying to start CPU 3). For a successful boot: Adding CPU 0, hwref=3Dcd38, awake=3D1 Waking up CPU 3 (dev=3Dc480) After reset 4&0 for CPU 3, hwref=3Dc480, awake=3D0, *(unsigned = long*)0xc0000000000000f0=3D0x25 After attempted wait for awake CPU 3, hwref=3Dc480, awake=3D1, = *(unsigned long*)0xc0000000000000f0=3D0x51 Adding CPU 3, hwref=3Dc480, awake=3D1 Waking up CPU 2 (dev=3Dc768) After reset 4&0 for CPU 2, hwref=3Dc768, awake=3D0, *(unsigned = long*)0xc0000000000000f0=3D0x51 After attempted wait for awake CPU 2, hwref=3Dc768, awake=3D1, = *(unsigned long*)0xc0000000000000f0=3D0x51 Adding CPU 2, hwref=3Dc768, awake=3D1 Waking up CPU 1 (dev=3Dca50) After reset 4&0 for CPU 1, hwref=3Dca50, awake=3D0, *(unsigned = long*)0xc0000000000000f0=3D0x51 After attempted wait for awake CPU 1, hwref=3Dca50, awake=3D1, = *(unsigned long*)0xc0000000000000f0=3D0x51 Adding CPU 1, hwref=3Dca50, awake=3D1 SMP: AP CPU #3 launched SMP: AP CPU #1 launched SMP: AP CPU #2 launched Trying to mount root from ufs:/dev/ufs/FBSDG5L2rootfs [rw,noatime]... GEOM: new disk ada1 So here waiting on cpu 0 does see pc_awake become 1 and does see 0xc0000000000000f0 updated by machdep_ap_bootstrap for CPU , for example. Note the awake=3D0 (i.e.., pc_awake) examples for each cpu even when there is already a 0x51 label for CPU 2 and CPU 3. (The code is shown later.) This is based on the labeling updates to: ("labels" stored at 0xc0000000000000f0) cpudep_ap_early_bootstrap (before-return labeled with value : 0x10) moea64_cpu_bootstrap_native (labeled with values: 0x21, 0x22, 0x23, = 0x24, 0x25 at various points) pmap_cpu_bootstrap (before-return labeled with value : 0x20) cpudep_ap_bootstrap (before-return labeled with value : 0x30) cpudep_ap_setup (before-return labeled with value : 0x40) machdep_ap_bootstrap (lableled with values: 0x5F, 0x51, and 0x50 = at various points) powermac_smp_start_cpu (reporting values after the resets for a CPU = and after the pa_awake loop) I happen to have experimented with additional isyncs in this variant of the code, not that they made much of a difference. The specifics are: # svnlite diff /usr/src/sys/powerpc/aim/mp_cpudep.c = /usr/src/sys/powerpc/powerpc/pmap_dispatch.c = /usr/src/sys/powerpc/powerpc/mp_machdep.c = /usr/src/sys/powerpc/powermac/platform_powermac.c = /usr/src/sys/powerpc/aim/moea64_native.c | more Index: /usr/src/sys/powerpc/aim/mp_cpudep.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/sys/powerpc/aim/mp_cpudep.c (revision 344018) +++ /usr/src/sys/powerpc/aim/mp_cpudep.c (working copy) @@ -105,6 +105,10 @@ =20 __asm __volatile("mtsprg 0, %0" :: "r"(ap_pcpu)); powerpc_sync(); + + *(unsigned long*)0xc0000000000000f0 =3D 0x10; // HACK!!! + isync(); // HACK!!! + powerpc_sync(); // HACK!!! } =20 uintptr_t @@ -124,6 +128,10 @@ pcpup->pc_curpcb =3D pcpup->pc_curthread->td_pcb; sp =3D pcpup->pc_curpcb->pcb_sp; =20 + *(unsigned long*)0xc0000000000000f0 =3D 0x30; // HACK!!! + isync(); // HACK!!! + powerpc_sync(); // HACK!!! + return (sp); } =20 @@ -416,5 +424,9 @@ "suboptimal.\n"); break; } + + *(unsigned long*)0xc0000000000000f0 =3D 0x40; // HACK!!! + isync(); // HACK!!! + powerpc_sync(); // HACK!!! } =20 Index: /usr/src/sys/powerpc/aim/moea64_native.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/sys/powerpc/aim/moea64_native.c (revision 344018) +++ /usr/src/sys/powerpc/aim/moea64_native.c (working copy) @@ -384,6 +384,9 @@ =20 mtmsr(mfmsr() & ~PSL_DR & ~PSL_IR); =20 + *(unsigned long*)0xc0000000000000f0 =3D 0x21; // HACK!!! + powerpc_sync(); // HACK!!! + /* * Install kernel SLB entries */ @@ -393,6 +396,9 @@ __asm __volatile ("slbmfee %0,%1; slbie %0;" : = "=3Dr"(seg0) : "r"(0)); =20 + *(unsigned long*)0xc0000000000000f0 =3D 0x22; // HACK!!! + powerpc_sync(); // HACK!!! + for (i =3D 0; i < n_slbs; i++) { if (!(slb[i].slbe & SLBE_VALID)) continue; @@ -400,6 +406,9 @@ __asm __volatile ("slbmte %0, %1" ::=20 "r"(slb[i].slbv), "r"(slb[i].slbe));=20 } + + *(unsigned long*)0xc0000000000000f0 =3D 0x23; // HACK!!! + powerpc_sync(); // HACK!!! #else for (i =3D 0; i < 16; i++) mtsrin(i << ADDR_SR_SHFT, = kernel_pmap->pm_sr[i]); @@ -412,7 +421,14 @@ __asm __volatile ("ptesync; mtsdr1 %0; isync" :: "r"(((uintptr_t)moea64_pteg_table & ~DMAP_BASE_ADDRESS) | (uintptr_t)(flsl(moea64_pteg_mask >> 11)))); + + *(unsigned long*)0xc0000000000000f0 =3D 0x24; // HACK!!! + powerpc_sync(); // HACK!!! + tlbia(); + + *(unsigned long*)0xc0000000000000f0 =3D 0x25; // HACK!!! + powerpc_sync(); // HACK!!! } Index: /usr/src/sys/powerpc/powerpc/pmap_dispatch.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/sys/powerpc/powerpc/pmap_dispatch.c (revision = 344018) +++ /usr/src/sys/powerpc/powerpc/pmap_dispatch.c (working copy) @@ -445,6 +445,10 @@ */ =20 return (MMU_CPU_BOOTSTRAP(mmu_obj, ap)); + + *(unsigned long*)0xc0000000000000f0 =3D 0x20; // HACK!!! + isync(); // HACK!!! + powerpc_sync(); // HACK!!! } =20 void * Index: /usr/src/sys/powerpc/powerpc/mp_machdep.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/sys/powerpc/powerpc/mp_machdep.c (revision 344018) +++ /usr/src/sys/powerpc/powerpc/mp_machdep.c (working copy) @@ -73,12 +73,20 @@ void machdep_ap_bootstrap(void) { + *(unsigned long*)0xc0000000000000f0 =3D 0x5F; // HACK!!! + isync(); // HACK!!! + powerpc_sync(); // HACK!!! =20 PCPU_SET(awake, 1); __asm __volatile("msync; isync"); =20 + *(unsigned long*)0xc0000000000000f0 =3D 0x51; // HACK!!! + isync(); // HACK!!! + powerpc_sync(); // HACK!!! + while (ap_letgo =3D=3D 0) __asm __volatile("or 31,31,31"); + isync(); // HACK!!! __asm __volatile("or 6,6,6"); =20 /* @@ -109,10 +117,15 @@ =20 while(smp_started =3D=3D 0) ; + isync(); // HACK!!! =20 /* Start per-CPU event timers. */ cpu_initclocks_ap(); =20 + *(unsigned long*)0xc0000000000000f0 =3D 0x50; // HACK!!! + isync(); // HACK!!! + powerpc_sync(); // HACK!!! + /* Announce ourselves awake, and enter the scheduler */ sched_throw(NULL); } @@ -241,11 +254,13 @@ timeout =3D 2000; /* wait 2sec for the = AP */ while (!pc->pc_awake && --timeout > 0) DELAY(1000); + isync(); // HACK!!! } } else { pc->pc_awake =3D 1; } if (pc->pc_awake) { + isync(); // HACK!!! if (bootverbose) printf("Adding CPU %d, hwref=3D%jx, = awake=3D%x\n", pc->pc_cpuid, = (uintmax_t)pc->pc_hwref, @@ -256,6 +271,8 @@ } =20 ap_awake =3D 1; + isync(); // HACK!!! + powerpc_sync(); // HACK!!! =20 /* Provide our current DEC and TB values for APs */ ap_timebase =3D mftb() + 10; @@ -268,6 +285,7 @@ =20 while (ap_awake < smp_cpus) ; + isync(); // HACK!!! =20 if (smp_cpus !=3D cpus || cpus !=3D mp_ncpus) { printf("SMP: %d CPUs found; %d CPUs usable; %d CPUs = woken\n", Index: /usr/src/sys/powerpc/powermac/platform_powermac.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/sys/powerpc/powermac/platform_powermac.c (revision = 344018) +++ /usr/src/sys/powerpc/powermac/platform_powermac.c (working copy) @@ -382,10 +382,22 @@ (void)(*rstvec); powerpc_sync(); =20 + isync(); // HACK!!! + if (bootverbose) // HACK!!! + printf("After reset 4&0 for CPU %d, hwref=3D%jx, = awake=3D%x, *(unsigned long*)0xc0000000000000f0=3D0x%jx\n", + pc->pc_cpuid, (uintmax_t)pc->pc_hwref, + pc->pc_awake,(uintmax_t)*(unsigned = long*)0xc0000000000000f0); + timeout =3D 10000; while (!pc->pc_awake && timeout--) DELAY(100); + isync(); // HACK!!! =20 + if (bootverbose) // HACK!!! + printf("After attempted wait for awake CPU %d, = hwref=3D%jx, awake=3D%x, *(unsigned long*)0xc0000000000000f0=3D0x%jx\n", + pc->pc_cpuid, (uintmax_t)pc->pc_hwref, + pc->pc_awake,(uintmax_t)*(unsigned = long*)0xc0000000000000f0); + return ((pc->pc_awake) ? 0 : EBUSY); #else /* No SMP support */ =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-ppc@freebsd.org Sun Feb 17 06:50:05 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4251714E5AD1 for ; Sun, 17 Feb 2019 06:50:05 +0000 (UTC) (envelope-from dclarke@blastwave.org) Received: from atl4mhfb04.myregisteredsite.com (atl4mhfb04.myregisteredsite.com [209.17.115.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A19E288CA6 for ; Sun, 17 Feb 2019 06:50:02 +0000 (UTC) (envelope-from dclarke@blastwave.org) Received: from atl4mhob22.registeredsite.com (atl4mhob22.registeredsite.com [209.17.115.116]) by atl4mhfb04.myregisteredsite.com (8.14.4/8.14.4) with ESMTP id x1H6nNCK026607 for ; Sun, 17 Feb 2019 01:49:23 -0500 Received: from mailpod.hostingplatform.com (atl4qobmail02pod2.registeredsite.com [10.30.77.36]) by atl4mhob22.registeredsite.com (8.14.4/8.14.4) with ESMTP id x1H6nGsT175348 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Sun, 17 Feb 2019 01:49:16 -0500 Received: (qmail 34836 invoked by uid 0); 17 Feb 2019 06:49:15 -0000 X-TCPREMOTEIP: 174.118.245.214 X-Authenticated-UID: dclarke@blastwave.org Received: from unknown (HELO ?172.16.35.3?) (dclarke@blastwave.org@174.118.245.214) by 0 with ESMTPA; 17 Feb 2019 06:49:15 -0000 Subject: Re: Cleaned -up evidence about the PowerMac G5 multiprocessor boot hang ups with the modern VM_MAX_KERNEL_ADDRESS value To: freebsd-ppc@freebsd.org References: From: Dennis Clarke Openpgp: preference=signencrypt Message-ID: <0cdabb85-11fe-044c-b768-520775ebdab4@blastwave.org> Date: Sun, 17 Feb 2019 01:49:15 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: A19E288CA6 X-Spamd-Bar: +++++ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [5.42 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_SPAM_SHORT(1.00)[0.999,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-ppc@freebsd.org]; TO_DN_NONE(0.00)[]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_SPAM_MEDIUM(1.00)[1.000,0]; RCVD_TLS_LAST(0.00)[]; MX_GOOD(-0.01)[mx1.netsolmail.net]; NEURAL_SPAM_LONG(1.00)[1.000,0]; RCVD_IN_DNSWL_NONE(0.00)[120.115.17.209.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; DMARC_NA(0.00)[blastwave.org]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:19871, ipnet:209.17.112.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(1.53)[ip: (4.83), ipnet: 209.17.112.0/21(1.62), asn: 19871(1.29), country: US(-0.07)] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 Feb 2019 06:50:05 -0000 On 2/16/19 7:29 PM, Mark Millard via freebsd-ppc wrote: > I eliminated a bunch of reports that gave information > that was the same for hang-ups and successful boots. > So this is starting over with less material to go > through. > Merely a quick follow up here to let you know that I am following along as I can however I am tied up with a RISC-V cross compile issue. -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken GreyBeard and suspenders optional From owner-freebsd-ppc@freebsd.org Sun Feb 17 20:10:25 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 962D814DCD5C for ; Sun, 17 Feb 2019 20:10:25 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic310-21.consmr.mail.gq1.yahoo.com (sonic310-21.consmr.mail.gq1.yahoo.com [98.137.69.147]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2027286C43 for ; Sun, 17 Feb 2019 20:10:24 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: csnuRn0VM1maY4piNnZvCuqfWrxMfPd8Y.ydT11PxF_UM2Aa4.HRRNjzLvjcc5f 13GVOAaciuPqPBitxe1SnP28Pd7aNPkxOk1qzmyrlTx9z0ryww4R6CtMSY0S2PFiJQk6sQYS8cnf QO1maG58WNXrPFJKleTz3vgLCLBy27eZ4m_CRmDY0J4sHc2jz0Gz6LIz.hBsGxvlNvE0jrKVa90m FAtkbahHg8u0reIhCCmLnh53I4..uw3Ca07y9HTn9HLUBP7kiVUP5T.fxqTV86AleQtXVzjmSbud zFb.CqNiZXuQmOFPUearFK6_yp2RHBvzJIumgKkqmT.kHnZynwaRgEdtLoqHsCFiZg3YfPwe2LVJ StD5iNkZy4Z.ztjz1XSwR7luoA61T3j_4XFIkQ5ypOpWcx_Jma6roW9sZ7ktMN7gEWWhFyYCwvW5 i4C00OohhNVyJjkVVcRzpQR4D7Hz4VyaRn.hhbRpfeu3eetKCeNorasDZR7AXbOPbp_4IT_n8eV7 4AKvoiIzYSJYw8P3NsaIxiqP8xUDpTib5_JCRByRn7SUuuBp4g1k4or6Kf.mN4WiX3QePeIEEc4y 0w9j4e3USFRCUftXAi8_rfG4BHYT_.dmnvhpF0GPIHS3m8V__ylp.iT.mMeWvEvrENZo.BqBLWQo zmGZWT.VjtvHnNw3QBPsfgUe3LJDiV.gzFYwXKL_0r2jn5mWTVQ.IqcLA64lvI4hsypKnX8JRTev GWEVDqRE5gpu2UorPPYXNd0d2QKocYJ8UprCC5QblYSkJUk7gb1mg.EvE_NzbK5WZrVnvseanbZb MfPiGsAGllDRQ61ZPE_aD_nlLbGKoyfSCO1ff6GIo1Trv7akTdI6UONdkZdSvIOJG.QdXa0c32sD C42WagjMelT8Fr6vFRLl7BnYG5HIxPmhHq3ZHWBToQCJWNZpgx9CGqXFBrDGb7DlD1KJyfZQoRR5 uoDCvfr1V9.eZAk0Gtf_R7fYlFiU7_tRaR3LBYjNwqPxNUyItyaxDAzjgbvJO7SvSLPmLdNeNYO_ 8_rAH3IUx55d1QYvx.BxFIHwjyfKkG7urkhlVb0fNtfeHDeqyXD38rhhryA-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.gq1.yahoo.com with HTTP; Sun, 17 Feb 2019 20:10:16 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.113]) ([67.170.167.181]) by smtp428.mail.gq1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 6cce6f046e0efb5998242fe650eb18c5; Sun, 17 Feb 2019 20:10:15 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: Re: Cleaned -up evidence about the PowerMac G5 multiprocessor boot hang ups with the modern VM_MAX_KERNEL_ADDRESS value [signficant new evidence] From: Mark Millard In-Reply-To: Date: Sun, 17 Feb 2019 12:10:14 -0800 Cc: FreeBSD PowerPC ML , Dennis Clarke Content-Transfer-Encoding: quoted-printable Message-Id: <924A8F35-88F6-4281-85D6-AF25161DAC8A@yahoo.com> References: To: Justin Hibbits X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: 2027286C43 X-Spamd-Bar: - X-Spamd-Result: default: False [-1.15 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.70)[-0.696,0]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_SPAM_SHORT(0.40)[0.402,0]; NEURAL_HAM_LONG(-0.92)[-0.923,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(0.58)[ip: (1.67), ipnet: 98.137.64.0/21(0.72), asn: 36647(0.58), country: US(-0.07)]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[147.69.137.98.list.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 Feb 2019 20:10:25 -0000 [Reporting the how many times the DELAY(100) in powermac_smp_start_cpu happens proved very interesting: pc_awake for CPU 3 for what turns into a hang does temporarily become non-zero as seen on cpu 0 (stopping the wait loop) but becomes zero again as seen on CPU 0, at least for some hung-up boots.] On 2019-Feb-16, at 16:29, Mark Millard wrote: > I eliminated a bunch of reports that gave information > that was the same for hang-ups and successful boots. > So this is starting over with less material to go > through. >=20 > For an unsuccessful boot: > (Values like 0x10, 0x25, 0x51 are use as recorded labels for points in = the code.) >=20 > Adding CPU 0, hwref=3Dcd38, awake=3D1 > Waking up CPU 3 (dev=3Dc480) > After reset 4&0 for CPU 3, hwref=3Dc480, awake=3D0, *(unsigned = long*)0xc0000000000000f0=3D0x25 > After attempted wait for awake CPU 3, hwref=3Dc480, awake=3D0, = *(unsigned long*)0xc0000000000000f0=3D0x25 > Waking up CPU 2 (dev=3Dc768) >=20 > So neither pc_awake nor *0xc0000000000000f0 have machdep_ap_bootstrap > based values as seen by CPU 0 (from trying to start CPU 3). >=20 > For a successful boot: >=20 > Adding CPU 0, hwref=3Dcd38, awake=3D1 > Waking up CPU 3 (dev=3Dc480) > After reset 4&0 for CPU 3, hwref=3Dc480, awake=3D0, *(unsigned = long*)0xc0000000000000f0=3D0x25 > After attempted wait for awake CPU 3, hwref=3Dc480, awake=3D1, = *(unsigned long*)0xc0000000000000f0=3D0x51 > Adding CPU 3, hwref=3Dc480, awake=3D1 > Waking up CPU 2 (dev=3Dc768) > After reset 4&0 for CPU 2, hwref=3Dc768, awake=3D0, *(unsigned = long*)0xc0000000000000f0=3D0x51 > After attempted wait for awake CPU 2, hwref=3Dc768, awake=3D1, = *(unsigned long*)0xc0000000000000f0=3D0x51 > Adding CPU 2, hwref=3Dc768, awake=3D1 > Waking up CPU 1 (dev=3Dca50) > After reset 4&0 for CPU 1, hwref=3Dca50, awake=3D0, *(unsigned = long*)0xc0000000000000f0=3D0x51 > After attempted wait for awake CPU 1, hwref=3Dca50, awake=3D1, = *(unsigned long*)0xc0000000000000f0=3D0x51 > Adding CPU 1, hwref=3Dca50, awake=3D1 > SMP: AP CPU #3 launched > SMP: AP CPU #1 launched > SMP: AP CPU #2 launched > Trying to mount root from ufs:/dev/ufs/FBSDG5L2rootfs [rw,noatime]... > GEOM: new disk ada1 >=20 > So here waiting on cpu 0 does see pc_awake become 1 and does see > 0xc0000000000000f0 updated by machdep_ap_bootstrap for CPU , for > example. >=20 > Note the awake=3D0 (i.e.., pc_awake) examples for each cpu even when > there is already a 0x51 label for CPU 2 and CPU 3. (The code is > shown later.) >=20 > This is based on the labeling updates to: > ("labels" stored at 0xc0000000000000f0) >=20 > cpudep_ap_early_bootstrap (before-return labeled with value : 0x10) > moea64_cpu_bootstrap_native (labeled with values: 0x21, 0x22, 0x23, = 0x24, 0x25 at various points) > pmap_cpu_bootstrap (before-return labeled with value : 0x20) > cpudep_ap_bootstrap (before-return labeled with value : 0x30) > cpudep_ap_setup (before-return labeled with value : 0x40) > machdep_ap_bootstrap (lableled with values: 0x5F, 0x51, and = 0x50 at various points) >=20 > powermac_smp_start_cpu (reporting values after the resets for a = CPU and after > the pa_awake loop) > . . . >=20 =46rom a successful boot: (note the "delay 100 count =3D 0," exmaples) Adding CPU 0, hwref=3Dcd38, awake=3D1 Waking up CPU 3 (dev=3Dc480) After reset 4&0 for CPU 3, hwref=3Dc480, awake=3D0, *(unsigned = long*)0xc0000000000000f0=3D0x25 After attempted wait for awake CPU 3, hwref=3Dc480, awake=3D1, delay 100 = count =3D 0, *(unsigned long*)0xc0000000000000f0=3D0x51 Adding CPU 3, hwref=3Dc480, awake=3D1 Waking up CPU 2 (dev=3Dc768) After reset 4&0 for CPU 2, hwref=3Dc768, awake=3D0, *(unsigned = long*)0xc0000000000000f0=3D0x51 After attempted wait for awake CPU 2, hwref=3Dc768, awake=3D1, delay 100 = count =3D 0, *(unsigned long*)0xc0000000000000f0=3D0x51 Adding CPU 2, hwref=3Dc768, awake=3D1 Waking up CPU 1 (dev=3Dca50) After reset 4&0 for CPU 1, hwref=3Dca50, awake=3D0, *(unsigned = long*)0xc0000000000000f0=3D0x51 After attempted wait for awake CPU 1, hwref=3Dca50, awake=3D1, delay 100 = count =3D 0, *(unsigned long*)0xc0000000000000f0=3D0x51 Adding CPU 1, hwref=3Dca50, awake=3D1 SMP: AP CPU #1 launched SMP: AP CPU #2 launched SMP: AP CPU #3 launched Trying to mount root from ufs:/dev/ufs/FBSDG5L2rootfs [rw,noatime]... GEOM: new disk ada1 Examples of boot hung ups look something like: (note the "delay 100 count =3D 2711," exmaples) Adding CPU 0, hwref=3Dcd38, awake=3D1 Waking up CPU 3 (dev=3Dc480) After reset 4&0 for CPU 3, hwref=3Dc480, awake=3D0, *(unsigned = long*)0xc0000000000000f0=3D0x25 After attempted wait for awake CPU 3, hwref=3Dc480, awake=3D0, delay 100 = count =3D 2711, *(unsigned long*)0xc0000000000000f0=3D0x25 Waking up CPU 2 (dev=3Dc768) (The above is from a picture of the screen.) 2711 implies that pc_awake was for a time non-zero. The code in question is: if (bootverbose) // HACK!!! printf("After reset 4&0 for CPU %d, hwref=3D%jx, = awake=3D%x, *(unsigned long*)0xc0000000000000f0=3D0x%jx\n", pc->pc_cpuid, (uintmax_t)pc->pc_hwref, pc->pc_awake,(uintmax_t)*(unsigned = long*)0xc0000000000000f0); timeout =3D 10000; while (!pc->pc_awake && timeout--) DELAY(100); if (bootverbose) // HACK!!! printf("After attempted wait for awake CPU %d, = hwref=3D%jx, awake=3D%x, delay 100 count =3D %jx, *(unsigned = long*)0xc0000000000000f0=3D0x%jx\n", pc->pc_cpuid, (uintmax_t)pc->pc_hwref, pc->pc_awake, (uintmax_t)(10000-timeout), = (uintmax_t)*(unsigned long*)0xc0000000000000f0); (If the loop stops for timeout=3D=3D0 the calculation of the count will = be high by 1.) Side note: I've also eliminated the experimental, extra isync() additions. The tested code is below. Index: /usr/src/sys/powerpc/aim/moea64_native.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/sys/powerpc/aim/moea64_native.c (revision 344018) +++ /usr/src/sys/powerpc/aim/moea64_native.c (working copy) @@ -384,6 +384,9 @@ =20 mtmsr(mfmsr() & ~PSL_DR & ~PSL_IR); =20 + *(unsigned long*)0xc0000000000000f0 =3D 0x21; // HACK!!! + powerpc_sync(); // HACK!!! + /* * Install kernel SLB entries */ @@ -393,6 +396,9 @@ __asm __volatile ("slbmfee %0,%1; slbie %0;" : = "=3Dr"(seg0) : "r"(0)); =20 + *(unsigned long*)0xc0000000000000f0 =3D 0x22; // HACK!!! + powerpc_sync(); // HACK!!! + for (i =3D 0; i < n_slbs; i++) { if (!(slb[i].slbe & SLBE_VALID)) continue; @@ -400,6 +406,9 @@ __asm __volatile ("slbmte %0, %1" ::=20 "r"(slb[i].slbv), "r"(slb[i].slbe));=20 } + + *(unsigned long*)0xc0000000000000f0 =3D 0x23; // HACK!!! + powerpc_sync(); // HACK!!! #else for (i =3D 0; i < 16; i++) mtsrin(i << ADDR_SR_SHFT, = kernel_pmap->pm_sr[i]); @@ -412,7 +421,14 @@ __asm __volatile ("ptesync; mtsdr1 %0; isync" :: "r"(((uintptr_t)moea64_pteg_table & ~DMAP_BASE_ADDRESS) | (uintptr_t)(flsl(moea64_pteg_mask >> 11)))); + + *(unsigned long*)0xc0000000000000f0 =3D 0x24; // HACK!!! + powerpc_sync(); // HACK!!! + tlbia(); + + *(unsigned long*)0xc0000000000000f0 =3D 0x25; // HACK!!! + powerpc_sync(); // HACK!!! } =20 static void Index: /usr/src/sys/powerpc/aim/mp_cpudep.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/sys/powerpc/aim/mp_cpudep.c (revision 344018) +++ /usr/src/sys/powerpc/aim/mp_cpudep.c (working copy) @@ -105,6 +105,9 @@ =20 __asm __volatile("mtsprg 0, %0" :: "r"(ap_pcpu)); powerpc_sync(); + + *(unsigned long*)0xc0000000000000f0 =3D 0x10; // HACK!!! + powerpc_sync(); // HACK!!! } =20 uintptr_t @@ -124,6 +127,9 @@ pcpup->pc_curpcb =3D pcpup->pc_curthread->td_pcb; sp =3D pcpup->pc_curpcb->pcb_sp; =20 + *(unsigned long*)0xc0000000000000f0 =3D 0x30; // HACK!!! + powerpc_sync(); // HACK!!! + return (sp); } =20 @@ -416,5 +422,8 @@ "suboptimal.\n"); break; } + + *(unsigned long*)0xc0000000000000f0 =3D 0x40; // HACK!!! + powerpc_sync(); // HACK!!! } =20 Index: /usr/src/sys/powerpc/include/vmparam.h =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/sys/powerpc/include/vmparam.h (revision 344018) +++ /usr/src/sys/powerpc/include/vmparam.h (working copy) @@ -109,9 +109,11 @@ #ifndef LOCORE #define VM_MIN_KERNEL_ADDRESS 0xe000000000000000UL #define VM_MAX_KERNEL_ADDRESS 0xe0000007ffffffffUL +//#define VM_MAX_KERNEL_ADDRESS 0xe0000001c7ffffffUL #else #define VM_MIN_KERNEL_ADDRESS 0xe000000000000000 #define VM_MAX_KERNEL_ADDRESS 0xe0000007ffffffff +//#define VM_MAX_KERNEL_ADDRESS 0xe0000001c7ffffff #endif #define VM_MAX_SAFE_KERNEL_ADDRESS VM_MAX_KERNEL_ADDRESS #endif Index: /usr/src/sys/powerpc/powermac/platform_powermac.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/sys/powerpc/powermac/platform_powermac.c (revision = 344018) +++ /usr/src/sys/powerpc/powermac/platform_powermac.c (working copy) @@ -382,10 +382,20 @@ (void)(*rstvec); powerpc_sync(); =20 + if (bootverbose) // HACK!!! + printf("After reset 4&0 for CPU %d, hwref=3D%jx, = awake=3D%x, *(unsigned long*)0xc0000000000000f0=3D0x%jx\n", + pc->pc_cpuid, (uintmax_t)pc->pc_hwref, + pc->pc_awake,(uintmax_t)*(unsigned = long*)0xc0000000000000f0); + timeout =3D 10000; while (!pc->pc_awake && timeout--) DELAY(100); =20 + if (bootverbose) // HACK!!! + printf("After attempted wait for awake CPU %d, = hwref=3D%jx, awake=3D%x, delay 100 count =3D %jx, *(unsigned = long*)0xc0000000000000f0=3D0x%jx\n", + pc->pc_cpuid, (uintmax_t)pc->pc_hwref, + pc->pc_awake, (uintmax_t)(10000-timeout), = (uintmax_t)*(unsigned long*)0xc0000000000000f0); + return ((pc->pc_awake) ? 0 : EBUSY); #else /* No SMP support */ Index: /usr/src/sys/powerpc/powerpc/machdep.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/sys/powerpc/powerpc/machdep.c (revision 344018) +++ /usr/src/sys/powerpc/powerpc/machdep.c (working copy) @@ -141,8 +141,6 @@ extern vm_paddr_t kernload; #endif =20 -extern void *ap_pcpu; - struct pcpu __pcpu[MAXCPU]; static char init_kenv[2048]; =20 Index: /usr/src/sys/powerpc/powerpc/mp_machdep.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/sys/powerpc/powerpc/mp_machdep.c (revision 344018) +++ /usr/src/sys/powerpc/powerpc/mp_machdep.c (working copy) @@ -73,10 +73,15 @@ void machdep_ap_bootstrap(void) { + *(unsigned long*)0xc0000000000000f0 =3D 0x5F; // HACK!!! + powerpc_sync(); // HACK!!! =20 PCPU_SET(awake, 1); __asm __volatile("msync; isync"); =20 + *(unsigned long*)0xc0000000000000f0 =3D 0x51; // HACK!!! + powerpc_sync(); // HACK!!! + while (ap_letgo =3D=3D 0) __asm __volatile("or 31,31,31"); __asm __volatile("or 6,6,6"); @@ -113,6 +118,9 @@ /* Start per-CPU event timers. */ cpu_initclocks_ap(); =20 + *(unsigned long*)0xc0000000000000f0 =3D 0x50; // HACK!!! + powerpc_sync(); // HACK!!! + /* Announce ourselves awake, and enter the scheduler */ sched_throw(NULL); } @@ -256,6 +264,7 @@ } =20 ap_awake =3D 1; + powerpc_sync(); // HACK!!! =20 /* Provide our current DEC and TB values for APs */ ap_timebase =3D mftb() + 10; Index: /usr/src/sys/powerpc/powerpc/pmap_dispatch.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/sys/powerpc/powerpc/pmap_dispatch.c (revision = 344018) +++ /usr/src/sys/powerpc/powerpc/pmap_dispatch.c (working copy) @@ -445,6 +445,9 @@ */ =20 return (MMU_CPU_BOOTSTRAP(mmu_obj, ap)); + + *(unsigned long*)0xc0000000000000f0 =3D 0x20; // HACK!!! + powerpc_sync(); // HACK!!! } =20 void * =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-ppc@freebsd.org Mon Feb 18 03:11:33 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 848E014F0536 for ; Mon, 18 Feb 2019 03:11:33 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic314-22.consmr.mail.ne1.yahoo.com (sonic314-22.consmr.mail.ne1.yahoo.com [66.163.189.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6E32F717ED for ; Mon, 18 Feb 2019 03:11:32 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: xVRpmEAVM1lFb0q8jZRdeSTIqdXkqWNdDPHdjDybXT5grkyqP_pYgEc20vK_tUm CHwLSIzyK4_ivDN5fAXn1JM_SRlmdRMJZ.7WldNjoXUQ.ot.oGp_2IAULqntHVyiHjouFLzOCSnz kGlPnC8Gc2wD8MWanFunZMHa8DHuaKbSufyU2lVekBzOQpPSMENpHFChkMtmIJ_e17shpCYo6b1K 5QA0KreyXcywfzxKIaIB0YnnCbEXrW29be_nT68P_NQJwpBUmluolAOJ3JKRqOHFnhy27P21mVrH yme.E0QjMFCwM_wsCoQLcChQWf.8_9qyq4m0LL7FEU42of247z9yDY9gzQFcakAgoi75yuiRW8Vt SKzRruVM_sP6DTZswG9.WutuqogqWzZNRJVdepGoHXkuS9YUaxJ8OpHetU1AbUX2MBwqiMCfx1aw YF1kWtgqRTdsEwbBIEQZjQMah1YZakbnjAn9v.DvS2MXqiKDtb3dokIfLbbUtEQEbmRrVpnnQ7M2 epq_u7jaVsCHrLqhwTZRnQSXyquFaJ63OeytZbB9G071LUXFX1DJOgycP8JbhI9QvGP06DbbfF9B Zd7HNewYUKf7BbSBEHs6gXROuFmNhF_ssYlelqw6NxMkVI5T0_SRxp4a5CAgjBhHkxCRwW3.f6fB Ws51JNqIYo6YKwrimeyOJotAdhDVqAkkSTqK8TSdqwp9LVQWgJHWmRu2eBKiu1qm.8ucW0UcoNpY tN0pvlfn_IKGM6YdItJkx1sM2GVClqYqJ8Y16azxXVZA9fDzT6xEELS4_j6sSV_XfRY_yJcfv3Zz OQGRHDCUqBImbg99YYc_pwL1.ttx57Gkioc9ZLB7vWnSHmRt1X8OdgI7JNrXpHI872D451PeDS3F tY1Pz12mkXl_7yhFtN2Z8NMfA6In8vcTRBc2LRmh56V0UGeqKJvCAmUBON24.x_J_NUNjTnOWbCD uf1L6jIurbe6q.k1hfcIwepx4iBHZt22hGpqBNzC1ZiUwMT6xgzLYTGh61.MYzEzYwW7WIMvfRjz U2u_zB1BDW1AsfWaKl6yFxHQOsdKJtGZwsM.FwEB06h6I9tmwA7G_SJ2ryQ-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.ne1.yahoo.com with HTTP; Mon, 18 Feb 2019 03:11:25 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.115]) ([67.170.167.181]) by smtp416.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 7416373e09ddbbebd3e94fd477514ed5; Mon, 18 Feb 2019 03:11:21 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: Re: Cleaned -up evidence about the PowerMac G5 multiprocessor boot hang ups with the modern VM_MAX_KERNEL_ADDRESS value [Wrong radix invaldates prior "new" information] From: Mark Millard In-Reply-To: <924A8F35-88F6-4281-85D6-AF25161DAC8A@yahoo.com> Date: Sun, 17 Feb 2019 19:11:20 -0800 Cc: FreeBSD PowerPC ML , Dennis Clarke Content-Transfer-Encoding: quoted-printable Message-Id: References: <924A8F35-88F6-4281-85D6-AF25161DAC8A@yahoo.com> To: Justin Hibbits X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: 6E32F717ED X-Spamd-Bar: ++ X-Spamd-Result: default: False [2.89 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_SPAM_SHORT(0.96)[0.964,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(1.37)[ip: (4.97), ipnet: 66.163.184.0/21(1.10), asn: 36646(0.88), country: US(-0.07)]; NEURAL_SPAM_MEDIUM(0.78)[0.781,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.28)[0.276,0]; RCVD_IN_DNSWL_NONE(0.00)[148.189.163.66.list.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Feb 2019 03:11:33 -0000 [I used %jx instead of the %jd that I had intended and: 0x2711 =3D=3D 10001. SO IGNORE MY PRIOR CONCLUSION. In fact the hang-up has the loop run until timeout=3D=3D0 . No evidence of pc_awake being temporarily non-zero.] On 2019-Feb-17, at 12:10, Mark Millard wrote: > [Reporting the how many times the DELAY(100) in > powermac_smp_start_cpu happens proved very interesting: > pc_awake for CPU 3 for what turns into a hang does > temporarily become non-zero as seen on cpu 0 (stopping > the wait loop) but becomes zero again as seen on CPU 0, > at least for some hung-up boots.] >=20 > On 2019-Feb-16, at 16:29, Mark Millard wrote: >=20 >> I eliminated a bunch of reports that gave information >> that was the same for hang-ups and successful boots. >> So this is starting over with less material to go >> through. >>=20 >> For an unsuccessful boot: >> (Values like 0x10, 0x25, 0x51 are use as recorded labels for points = in the code.) >>=20 >> Adding CPU 0, hwref=3Dcd38, awake=3D1 >> Waking up CPU 3 (dev=3Dc480) >> After reset 4&0 for CPU 3, hwref=3Dc480, awake=3D0, *(unsigned = long*)0xc0000000000000f0=3D0x25 >> After attempted wait for awake CPU 3, hwref=3Dc480, awake=3D0, = *(unsigned long*)0xc0000000000000f0=3D0x25 >> Waking up CPU 2 (dev=3Dc768) >>=20 >> So neither pc_awake nor *0xc0000000000000f0 have machdep_ap_bootstrap >> based values as seen by CPU 0 (from trying to start CPU 3). >>=20 >> For a successful boot: >>=20 >> Adding CPU 0, hwref=3Dcd38, awake=3D1 >> Waking up CPU 3 (dev=3Dc480) >> After reset 4&0 for CPU 3, hwref=3Dc480, awake=3D0, *(unsigned = long*)0xc0000000000000f0=3D0x25 >> After attempted wait for awake CPU 3, hwref=3Dc480, awake=3D1, = *(unsigned long*)0xc0000000000000f0=3D0x51 >> Adding CPU 3, hwref=3Dc480, awake=3D1 >> Waking up CPU 2 (dev=3Dc768) >> After reset 4&0 for CPU 2, hwref=3Dc768, awake=3D0, *(unsigned = long*)0xc0000000000000f0=3D0x51 >> After attempted wait for awake CPU 2, hwref=3Dc768, awake=3D1, = *(unsigned long*)0xc0000000000000f0=3D0x51 >> Adding CPU 2, hwref=3Dc768, awake=3D1 >> Waking up CPU 1 (dev=3Dca50) >> After reset 4&0 for CPU 1, hwref=3Dca50, awake=3D0, *(unsigned = long*)0xc0000000000000f0=3D0x51 >> After attempted wait for awake CPU 1, hwref=3Dca50, awake=3D1, = *(unsigned long*)0xc0000000000000f0=3D0x51 >> Adding CPU 1, hwref=3Dca50, awake=3D1 >> SMP: AP CPU #3 launched >> SMP: AP CPU #1 launched >> SMP: AP CPU #2 launched >> Trying to mount root from ufs:/dev/ufs/FBSDG5L2rootfs [rw,noatime]... >> GEOM: new disk ada1 >>=20 >> So here waiting on cpu 0 does see pc_awake become 1 and does see >> 0xc0000000000000f0 updated by machdep_ap_bootstrap for CPU , for >> example. >>=20 >> Note the awake=3D0 (i.e.., pc_awake) examples for each cpu even when >> there is already a 0x51 label for CPU 2 and CPU 3. (The code is >> shown later.) >>=20 >> This is based on the labeling updates to: >> ("labels" stored at 0xc0000000000000f0) >>=20 >> cpudep_ap_early_bootstrap (before-return labeled with value : 0x10) >> moea64_cpu_bootstrap_native (labeled with values: 0x21, 0x22, 0x23, = 0x24, 0x25 at various points) >> pmap_cpu_bootstrap (before-return labeled with value : 0x20) >> cpudep_ap_bootstrap (before-return labeled with value : 0x30) >> cpudep_ap_setup (before-return labeled with value : 0x40) >> machdep_ap_bootstrap (lableled with values: 0x5F, 0x51, and = 0x50 at various points) >>=20 >> powermac_smp_start_cpu (reporting values after the resets for a = CPU and after >> the pa_awake loop) >> . . . >>=20 >=20 > =46rom a successful boot: > (note the "delay 100 count =3D 0," exmaples) >=20 > Adding CPU 0, hwref=3Dcd38, awake=3D1 > Waking up CPU 3 (dev=3Dc480) > After reset 4&0 for CPU 3, hwref=3Dc480, awake=3D0, *(unsigned = long*)0xc0000000000000f0=3D0x25 > After attempted wait for awake CPU 3, hwref=3Dc480, awake=3D1, delay = 100 count =3D 0, *(unsigned long*)0xc0000000000000f0=3D0x51 > Adding CPU 3, hwref=3Dc480, awake=3D1 > Waking up CPU 2 (dev=3Dc768) > After reset 4&0 for CPU 2, hwref=3Dc768, awake=3D0, *(unsigned = long*)0xc0000000000000f0=3D0x51 > After attempted wait for awake CPU 2, hwref=3Dc768, awake=3D1, delay = 100 count =3D 0, *(unsigned long*)0xc0000000000000f0=3D0x51 > Adding CPU 2, hwref=3Dc768, awake=3D1 > Waking up CPU 1 (dev=3Dca50) > After reset 4&0 for CPU 1, hwref=3Dca50, awake=3D0, *(unsigned = long*)0xc0000000000000f0=3D0x51 > After attempted wait for awake CPU 1, hwref=3Dca50, awake=3D1, delay = 100 count =3D 0, *(unsigned long*)0xc0000000000000f0=3D0x51 > Adding CPU 1, hwref=3Dca50, awake=3D1 > SMP: AP CPU #1 launched > SMP: AP CPU #2 launched > SMP: AP CPU #3 launched > Trying to mount root from ufs:/dev/ufs/FBSDG5L2rootfs [rw,noatime]... > GEOM: new disk ada1 >=20 >=20 > Examples of boot hung ups look something like: > (note the "delay 100 count =3D 2711," exmaples) >=20 > Adding CPU 0, hwref=3Dcd38, awake=3D1 > Waking up CPU 3 (dev=3Dc480) > After reset 4&0 for CPU 3, hwref=3Dc480, awake=3D0, *(unsigned = long*)0xc0000000000000f0=3D0x25 > After attempted wait for awake CPU 3, hwref=3Dc480, awake=3D0, delay = 100 count =3D 2711, *(unsigned long*)0xc0000000000000f0=3D0x25 > Waking up CPU 2 (dev=3Dc768) >=20 > (The above is from a picture of the screen.) The following is wrong because 2711 was hexadecimal, instead of the decimal I had intended: > 2711 implies that pc_awake was for a time non-zero. The > code in question is: 0x2711 =3D=3D 10001, indicating that the loop ran to its conclusion via timeout=3D=3D0 . > if (bootverbose) // HACK!!! > printf("After reset 4&0 for CPU %d, hwref=3D%jx, = awake=3D%x, *(unsigned long*)0xc0000000000000f0=3D0x%jx\n", > pc->pc_cpuid, (uintmax_t)pc->pc_hwref, > pc->pc_awake,(uintmax_t)*(unsigned = long*)0xc0000000000000f0); >=20 > timeout =3D 10000; > while (!pc->pc_awake && timeout--) > DELAY(100); >=20 > if (bootverbose) // HACK!!! > printf("After attempted wait for awake CPU %d, = hwref=3D%jx, awake=3D%x, delay 100 count =3D %jx, *(unsigned = long*)0xc0000000000000f0=3D0x%jx\n", > pc->pc_cpuid, (uintmax_t)pc->pc_hwref, > pc->pc_awake, (uintmax_t)(10000-timeout), = (uintmax_t)*(unsigned long*)0xc0000000000000f0); Above: I used %jx for the count instead of the intended %jd. > (If the loop stops for timeout=3D=3D0 the calculation of the count = will be > high by 1.) >=20 >=20 >=20 > Side note: >=20 > I've also eliminated the experimental, extra isync() additions. > The tested code is below. >=20 > Index: /usr/src/sys/powerpc/aim/moea64_native.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- /usr/src/sys/powerpc/aim/moea64_native.c (revision 344018) > +++ /usr/src/sys/powerpc/aim/moea64_native.c (working copy) > @@ -384,6 +384,9 @@ >=20 > mtmsr(mfmsr() & ~PSL_DR & ~PSL_IR); >=20 > + *(unsigned long*)0xc0000000000000f0 =3D 0x21; // HACK!!! > + powerpc_sync(); // HACK!!! > + > /* > * Install kernel SLB entries > */ > @@ -393,6 +396,9 @@ > __asm __volatile ("slbmfee %0,%1; slbie %0;" : = "=3Dr"(seg0) : > "r"(0)); >=20 > + *(unsigned long*)0xc0000000000000f0 =3D 0x22; // = HACK!!! > + powerpc_sync(); // HACK!!! > + > for (i =3D 0; i < n_slbs; i++) { > if (!(slb[i].slbe & SLBE_VALID)) > continue; > @@ -400,6 +406,9 @@ > __asm __volatile ("slbmte %0, %1" ::=20 > "r"(slb[i].slbv), "r"(slb[i].slbe));=20 > } > + > + *(unsigned long*)0xc0000000000000f0 =3D 0x23; // = HACK!!! > + powerpc_sync(); // HACK!!! > #else > for (i =3D 0; i < 16; i++) > mtsrin(i << ADDR_SR_SHFT, = kernel_pmap->pm_sr[i]); > @@ -412,7 +421,14 @@ > __asm __volatile ("ptesync; mtsdr1 %0; isync" > :: "r"(((uintptr_t)moea64_pteg_table & ~DMAP_BASE_ADDRESS) > | (uintptr_t)(flsl(moea64_pteg_mask >> 11)))); > + > + *(unsigned long*)0xc0000000000000f0 =3D 0x24; // HACK!!! > + powerpc_sync(); // HACK!!! > + > tlbia(); > + > + *(unsigned long*)0xc0000000000000f0 =3D 0x25; // HACK!!! > + powerpc_sync(); // HACK!!! > } >=20 > static void > Index: /usr/src/sys/powerpc/aim/mp_cpudep.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- /usr/src/sys/powerpc/aim/mp_cpudep.c (revision 344018) > +++ /usr/src/sys/powerpc/aim/mp_cpudep.c (working copy) > @@ -105,6 +105,9 @@ >=20 > __asm __volatile("mtsprg 0, %0" :: "r"(ap_pcpu)); > powerpc_sync(); > + > + *(unsigned long*)0xc0000000000000f0 =3D 0x10; // HACK!!! > + powerpc_sync(); // HACK!!! > } >=20 > uintptr_t > @@ -124,6 +127,9 @@ > pcpup->pc_curpcb =3D pcpup->pc_curthread->td_pcb; > sp =3D pcpup->pc_curpcb->pcb_sp; >=20 > + *(unsigned long*)0xc0000000000000f0 =3D 0x30; // HACK!!! > + powerpc_sync(); // HACK!!! > + > return (sp); > } >=20 > @@ -416,5 +422,8 @@ > "suboptimal.\n"); > break; > } > + > + *(unsigned long*)0xc0000000000000f0 =3D 0x40; // HACK!!! > + powerpc_sync(); // HACK!!! > } >=20 > Index: /usr/src/sys/powerpc/include/vmparam.h > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- /usr/src/sys/powerpc/include/vmparam.h (revision 344018) > +++ /usr/src/sys/powerpc/include/vmparam.h (working copy) > @@ -109,9 +109,11 @@ > #ifndef LOCORE > #define VM_MIN_KERNEL_ADDRESS 0xe000000000000000UL > #define VM_MAX_KERNEL_ADDRESS 0xe0000007ffffffffUL > +//#define VM_MAX_KERNEL_ADDRESS 0xe0000001c7ffffffUL > #else > #define VM_MIN_KERNEL_ADDRESS 0xe000000000000000 > #define VM_MAX_KERNEL_ADDRESS 0xe0000007ffffffff > +//#define VM_MAX_KERNEL_ADDRESS 0xe0000001c7ffffff > #endif > #define VM_MAX_SAFE_KERNEL_ADDRESS VM_MAX_KERNEL_ADDRESS > #endif > Index: /usr/src/sys/powerpc/powermac/platform_powermac.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- /usr/src/sys/powerpc/powermac/platform_powermac.c (revision = 344018) > +++ /usr/src/sys/powerpc/powermac/platform_powermac.c (working copy) > @@ -382,10 +382,20 @@ > (void)(*rstvec); > powerpc_sync(); >=20 > + if (bootverbose) // HACK!!! > + printf("After reset 4&0 for CPU %d, hwref=3D%jx, = awake=3D%x, *(unsigned long*)0xc0000000000000f0=3D0x%jx\n", > + pc->pc_cpuid, (uintmax_t)pc->pc_hwref, > + pc->pc_awake,(uintmax_t)*(unsigned = long*)0xc0000000000000f0); > + > timeout =3D 10000; > while (!pc->pc_awake && timeout--) > DELAY(100); >=20 > + if (bootverbose) // HACK!!! > + printf("After attempted wait for awake CPU %d, = hwref=3D%jx, awake=3D%x, delay 100 count =3D %jx, *(unsigned = long*)0xc0000000000000f0=3D0x%jx\n", > + pc->pc_cpuid, (uintmax_t)pc->pc_hwref, > + pc->pc_awake, (uintmax_t)(10000-timeout), = (uintmax_t)*(unsigned long*)0xc0000000000000f0); > + > return ((pc->pc_awake) ? 0 : EBUSY); > #else > /* No SMP support */ Again (above): I used %jx for the count instead of the intended %jd . > Index: /usr/src/sys/powerpc/powerpc/machdep.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- /usr/src/sys/powerpc/powerpc/machdep.c (revision 344018) > +++ /usr/src/sys/powerpc/powerpc/machdep.c (working copy) > @@ -141,8 +141,6 @@ > extern vm_paddr_t kernload; > #endif >=20 > -extern void *ap_pcpu; > - > struct pcpu __pcpu[MAXCPU]; > static char init_kenv[2048]; >=20 > Index: /usr/src/sys/powerpc/powerpc/mp_machdep.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- /usr/src/sys/powerpc/powerpc/mp_machdep.c (revision 344018) > +++ /usr/src/sys/powerpc/powerpc/mp_machdep.c (working copy) > @@ -73,10 +73,15 @@ > void > machdep_ap_bootstrap(void) > { > + *(unsigned long*)0xc0000000000000f0 =3D 0x5F; // HACK!!! > + powerpc_sync(); // HACK!!! >=20 > PCPU_SET(awake, 1); > __asm __volatile("msync; isync"); >=20 > + *(unsigned long*)0xc0000000000000f0 =3D 0x51; // HACK!!! > + powerpc_sync(); // HACK!!! > + > while (ap_letgo =3D=3D 0) > __asm __volatile("or 31,31,31"); > __asm __volatile("or 6,6,6"); > @@ -113,6 +118,9 @@ > /* Start per-CPU event timers. */ > cpu_initclocks_ap(); >=20 > + *(unsigned long*)0xc0000000000000f0 =3D 0x50; // HACK!!! > + powerpc_sync(); // HACK!!! > + > /* Announce ourselves awake, and enter the scheduler */ > sched_throw(NULL); > } > @@ -256,6 +264,7 @@ > } >=20 > ap_awake =3D 1; > + powerpc_sync(); // HACK!!! >=20 > /* Provide our current DEC and TB values for APs */ > ap_timebase =3D mftb() + 10; > Index: /usr/src/sys/powerpc/powerpc/pmap_dispatch.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- /usr/src/sys/powerpc/powerpc/pmap_dispatch.c (revision = 344018) > +++ /usr/src/sys/powerpc/powerpc/pmap_dispatch.c (working copy) > @@ -445,6 +445,9 @@ > */ >=20 > return (MMU_CPU_BOOTSTRAP(mmu_obj, ap)); > + > + *(unsigned long*)0xc0000000000000f0 =3D 0x20; // HACK!!! > + powerpc_sync(); // HACK!!! > } >=20 > void * >=20 >=20 >=20 =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-ppc@freebsd.org Tue Feb 19 02:58:56 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0D96414D5F09 for ; Tue, 19 Feb 2019 02:58:56 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic308-12.consmr.mail.ne1.yahoo.com (sonic308-12.consmr.mail.ne1.yahoo.com [66.163.187.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B2F9C962A2 for ; Tue, 19 Feb 2019 02:58:53 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: oA4i4NAVM1leUWWN4O.HZIOhKXPAod7.V3sXKMgcXUsiZtF2OPPK2ovmYcWWk9D Y59sD00yx3r.rzb_BiRVCtJeXt88AXND14hAme8T028k6pCDgcyJHo6WQ_heKkkoqhQ9LPleGKWv figjfUiyIZz4vVw52kt3feyjBCJauD3Fn81.j07j1iRNE7mh.w1WC1xxYu0F5kE22wN10v4sJbKS rxLx8ZLpYDZG.cLptLIO1Ls8LYmpQRztjYfg5t9iVr8gWyFyk.3gnvRIn5RciM6DytCsVt8qeRW7 .m9c3mqobWHE06mdkGAaYIZQNzBJD8o46roEhI4ZmHYdOpVHSvz1SlX.unuvLMU0syxbrqdTPZrK yPe8.yFOqJqVgDAxBm9RHaDd3ZCx_S.lwm.cDz5zY.zRoYAkWuM9l_IRVq8.U7KhMs8wgC6cVR9. qV0WF3KZ5li4S9Kh9.kdfj_BtPhX6MIa46vOrwfBg4o1L8lk_uIXB8CtlqKv1vlSDeoY4lmXx5S7 CF0agvSmNKWvfzxMr3WjH.znql03qcL4asiWrGxPR3DPz0o7ZaxB9IzF80daNwWokZP86VszlaUE fawkeDRy63nQozJawDm1RWD4y_Waiz5O1tB4udEcQIp.Ottmsr6p8cgVqFq2tKPuplzJz2T0KnwP I7bvpGkJsqYMdjQ6a0PJ3__3flRZ.jA5m2qrxsiEF.gTpI8Ualqfvz4Hg8fcboQGMevOKGfSTjQv enTvFeDs01qO0yXjFAz72Dw3lHavZaRq1tjbOg9dNJC2edtaewUPKMb.KVYszJeDehZEM8cWt.7d 1YIkVkkRRU4ChpJwcuPZGWTw1jnLiLvOKmfuhdcFAsGM9d6Vgiy1KbGoYJyqwLsEDgiQeuXZOOd_ _eVFGIyLZOymWx2aRxrJbvys9UrzdsXiOGmCa28LUb2_Gn4MPw5odcvkWwJLzy9mlcjHMTGxnv.S UaFJFoLggVZrrDsS1iJBMfeewlzjdr.MdXIjGoeLKVrCWWJgqbub2S3tbcLPbEGj9Y0s9l7p3SHD Sv3mdR32cC6lXFMS1X9k7VDTRdAuNHCeB9Am.9unutOLL7YtltIeukRax9el76Fgc Received: from sonic.gate.mail.ne1.yahoo.com by sonic308.consmr.mail.ne1.yahoo.com with HTTP; Tue, 19 Feb 2019 02:58:46 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.113]) ([67.170.167.181]) by smtp428.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 071f5e2723ba0e299e9b13300a5a998f; Tue, 19 Feb 2019 02:58:45 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: Question on "Map the entire KVA range into the SLB. We must not fault there" vs. the modern VM_MAX_KERNEL_ADDRESS Message-Id: Date: Mon, 18 Feb 2019 18:58:43 -0800 Cc: FreeBSD PowerPC ML To: Justin Hibbits X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: B2F9C962A2 X-Spamd-Bar: + X-Spamd-Result: default: False [1.55 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.73)[0.734,0]; NEURAL_HAM_LONG(-0.19)[-0.188,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(1.09)[ip: (3.58), ipnet: 66.163.184.0/21(1.09), asn: 36646(0.87), country: US(-0.07)]; NEURAL_SPAM_MEDIUM(0.42)[0.421,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[35.187.163.66.list.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Feb 2019 02:58:56 -0000 It takes me a bit to provide context for the question . . . I'll first note: #define VM_MAX_SAFE_KERNEL_ADDRESS VM_MAX_KERNEL_ADDRESS moea64_mid_bootstrap has: virtual_avail = VM_MIN_KERNEL_ADDRESS; virtual_end = VM_MAX_SAFE_KERNEL_ADDRESS; /* * Map the entire KVA range into the SLB. We must not fault there. */ #ifdef __powerpc64__ for (va = virtual_avail; va < virtual_end; va += SEGMENT_LENGTH) moea64_bootstrap_slb_prefault(va, 0); #endif but the prefaulting is based on: #define PCPU_MD_AIM64_FIELDS \ struct slb slb[64]; \ where the slb is based on the upper 36 bits of the passed-in va values (effective address value in powerpc terms). The address range now being covered is: #define VM_MIN_KERNEL_ADDRESS 0xe000000000000000 #define VM_MAX_KERNEL_ADDRESS 0xe0000007ffffffff So for the upper 36 bits: 0xe000_0000_0 0xe000_0007_f and the range 0x00 through 0x7f has 128 possibilities. So more than the slb can track. (Also: slb[USER_SLB_SLOT] is avoided so really 63 of the 64 are available.) Later (larger) va values are the replacing earlier (smaller) va values via slb_insert_kernel usage that in turn involves random replacements via (n_slbs being 64 in the G5 context): i = mftb() % n_slbs; if (i == USER_SLB_SLOT) i = (i+1) % n_slbs; So apparently only the last 63 from: for (va = virtual_avail; va < virtual_end; va += SEGMENT_LENGTH) moea64_bootstrap_slb_prefault(va, 0); are known to still be present in the slb. So the question . . . How can: "Map the entire KVA range into the SLB. We must not fault there" be true? Is it really a requirement? If it is, then something seems to be wrong as far as handling the modern VM_MAX_KERNEL_ADDRESS value goes. The old VM_MAX_KERNEL_ADDRESS value would have 0x00 through 0x1c and so would fit this particular constraint. The old value 0xe0000001c7ffffff looks like something was being avoided (why not 0xe0000001ffffffff ?). But I've not found what or why. === Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-ppc@freebsd.org Tue Feb 19 19:49:53 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F2B0514FB4D4 for ; Tue, 19 Feb 2019 19:49:52 +0000 (UTC) (envelope-from chmeeedalf@gmail.com) Received: from mail-it1-x12b.google.com (mail-it1-x12b.google.com [IPv6:2607:f8b0:4864:20::12b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0FE8C70B6E for ; Tue, 19 Feb 2019 19:49:52 +0000 (UTC) (envelope-from chmeeedalf@gmail.com) Received: by mail-it1-x12b.google.com with SMTP id h6so9306955itl.1 for ; Tue, 19 Feb 2019 11:49:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rcjMA9pvfdXjt76IA5govBG4QzzW3auE+eW4HqKxWGw=; b=L8f7qKPUOv24xkgL3lrduStzR3wbfAMBAlkvHns8eQQCxf65C7Vh0wAIDrHhpMPfMY 0Cw1F/X3ejGJQIZX9gGnrHHAPiqflIsXxQjawuSLB31HkJDLSZwJkOTVWFJsAzCFKXd1 e31deYfO8oK23a4tNzu9lnc4CQl6LVlamOSxc1erOSIui0y425FFEdgqIffZEIpIC0Fi /IqTJ4PC+bKfjNtNPIXDI0nQZHc3qgcY1pY2M1Xo1gnkdsICd4+CWMFmC9kN67sZt1SI mDUoTUW873+UQU+0pdbKJtzsqPI4L3qPnYIw5G0qYHgExWzXtLh2JLatCGqc4nNLyECI TkOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rcjMA9pvfdXjt76IA5govBG4QzzW3auE+eW4HqKxWGw=; b=VDO8+SQ3uy9Lc4s6GHLgTlAXhSXUUXQbGknBn+xVMrx052apVDMH9FNVrpy33WVC8O g27kW4rECMAyQ2A+Mv5+xRrEFLxPjFmgyz3aW4xNtSaK+F/kfqZMgJddaQpZELCmi0DX nPytfK+KAw0R3csKb5Pe6bjvKNwF3ZhGEadxDdCU/pUY9B3eyYHLaP2chnS+TsPJV6Qb KeQhanyc6WVBgJmkOQztpkwJ+VKXhUrC586HSEi232qfXG/s7/vX7tdvBXJlLl06IW1k wZkzIiIfJopaLAytilvChbdUg+r4zjpKoPRriLblfqOgNi7GTwWZ9tDFmtqXVSG/OzqK tuVA== X-Gm-Message-State: AHQUAuY7DGj1JxB3Qcv6yDpojDMmjbkPWV6t2IPxyHlKg8wiNZJ8aizF huYEYIGnn7+fXMcNBj+xUdtpwzQK X-Google-Smtp-Source: AHgI3IbJjTbw6Ogf8+v971wbpYjx6fv4uLCCCZHDhHYo3lYmgSz1Tj6NHrvwAzeLSvyvfCP9D+RiXw== X-Received: by 2002:a24:88:: with SMTP id 130mr3107668ita.92.1550605791174; Tue, 19 Feb 2019 11:49:51 -0800 (PST) Received: from ralga.knownspace (173-25-245-129.client.mchsi.com. [173.25.245.129]) by smtp.gmail.com with ESMTPSA id 2sm1584661itw.11.2019.02.19.11.49.50 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 19 Feb 2019 11:49:50 -0800 (PST) Date: Tue, 19 Feb 2019 13:49:46 -0600 From: Justin Hibbits To: Mark Millard Cc: FreeBSD PowerPC ML Subject: Re: Question on "Map the entire KVA range into the SLB. We must not fault there" vs. the modern VM_MAX_KERNEL_ADDRESS Message-ID: <20190219134946.125d91a9@ralga.knownspace> In-Reply-To: References: X-Mailer: Claws Mail 3.17.3 (GTK+ 2.24.32; powerpc64-portbld-freebsd13.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 0FE8C70B6E X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=L8f7qKPU; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of chmeeedalf@gmail.com designates 2607:f8b0:4864:20::12b as permitted sender) smtp.mailfrom=chmeeedalf@gmail.com X-Spamd-Result: default: False [-6.42 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; FREEMAIL_TO(0.00)[yahoo.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_TLS_LAST(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_SHORT(-0.98)[-0.980,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-ppc@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[b.2.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.43)[ip: (-7.51), ipnet: 2607:f8b0::/32(-2.60), asn: 15169(-1.99), country: US(-0.07)] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Feb 2019 19:49:53 -0000 On Mon, 18 Feb 2019 18:58:43 -0800 Mark Millard wrote: > It takes me a bit to provide context for the question . . . > > I'll first note: > > #define VM_MAX_SAFE_KERNEL_ADDRESS VM_MAX_KERNEL_ADDRESS > > moea64_mid_bootstrap has: > > virtual_avail = VM_MIN_KERNEL_ADDRESS; > virtual_end = VM_MAX_SAFE_KERNEL_ADDRESS; > > /* > * Map the entire KVA range into the SLB. We must not fault > there. */ > #ifdef __powerpc64__ > for (va = virtual_avail; va < virtual_end; va += > SEGMENT_LENGTH) moea64_bootstrap_slb_prefault(va, 0); > #endif > > but the prefaulting is based on: > > #define > PCPU_MD_AIM64_FIELDS \ > struct slb slb[64]; \ > > where the slb is based on the upper 36 bits of the passed-in va values > (effective address value in powerpc terms). > > The address range now being covered is: > > #define VM_MIN_KERNEL_ADDRESS 0xe000000000000000 > #define VM_MAX_KERNEL_ADDRESS 0xe0000007ffffffff > > So for the upper 36 bits: > > 0xe000_0000_0 > 0xe000_0007_f > > and the range 0x00 through 0x7f has 128 possibilities. > > So more than the slb can track. (Also: slb[USER_SLB_SLOT] > is avoided so really 63 of the 64 are available.) > > Later (larger) va values are the replacing earlier > (smaller) va values via slb_insert_kernel usage that > in turn involves random replacements via (n_slbs > being 64 in the G5 context): > > i = mftb() % n_slbs; > if (i == USER_SLB_SLOT) > i = (i+1) % n_slbs; > > So apparently only the last 63 from: > > for (va = virtual_avail; va < virtual_end; va += > SEGMENT_LENGTH) moea64_bootstrap_slb_prefault(va, 0); > > are known to still be present in the slb. > > > > So the question . . . > > How can: > > "Map the entire KVA range into the SLB. We must not fault there" > > be true? Is it really a requirement? If it is, then something seems > to be wrong as far as handling the modern VM_MAX_KERNEL_ADDRESS value > goes. > > > The old VM_MAX_KERNEL_ADDRESS value would have 0x00 through 0x1c > and so would fit this particular constraint. The old value > 0xe0000001c7ffffff looks like something was being avoided > (why not 0xe0000001ffffffff ?). But I've not found what or why. If this were cause for concern I'd likely see it on my POWER9, which actually only has 32 SLB entries, 4 of which are pinned, the remaining 28 being FIFO. Much less than 64 on the G5. - Justin From owner-freebsd-ppc@freebsd.org Tue Feb 19 20:36:27 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 27E6E14D75C2 for ; Tue, 19 Feb 2019 20:36:27 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic309-21.consmr.mail.gq1.yahoo.com (sonic309-21.consmr.mail.gq1.yahoo.com [98.137.65.147]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B906F72729 for ; Tue, 19 Feb 2019 20:36:25 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: oxCtb.QVM1nEBpMChbWtcG8D6GFDHMCbaterjQgMZbnn7F1sMGMAtizAzz4AaX. LQoia3lBcIIk54UNjxN8CDtl0BvbOVqqhg_YyL51wsJA0ysrx_Z.tzf3YJM_vAhA2Ywpcp4sanxC il5xQGhb87VkgCt8pHmcdbfY59s6UgNJW01xJILTFlQgz1rB2ftWeSFTgTVagHaIpsuzLB3FtoYL yW0gxtKBXVwbVp0Re7xiiWJWrhQoxZS5EF5ZEwAPT.nHhov1MPMVoicQbC.7jV0sCfeLcpJ9lvaQ nuEuR9rDwMsaKpQ5dJ9f3eYc8JrcpsHyFKN1moN.T0c1zyiPYTBI8FaJNC3RY1kWqTVskdNnU8b5 7q0ZyxX95IQ7211CL0QU45exIU5Q09zC0wEBcxrh.EamCAqpPN854cgIoEkcBh5Clw88C11Iz4LO MxZtsOZNW0ElnSKCL.HG6KTQsxvILIJuXMFNeo6UJ148r9zJ.9rFyUF7V0wOBE5Wqo8naJ7ofjFq kao5TzjJpjUimhpAesQHlH3UNrYLLyFiBx82rCPKOM.2ipyMtKb8nzW_Fklcq81_rFmHyqiV4wSd 1wEdrEu5BRy3_WRdo1h0ejWVMVF2YiwkKtzSwSPlPe3_9n1osSSHxZU2W7hqHDZgQZLOsWmPyOGL 5EVJr7td9hAJVTspbsZVKpdc3rtaa1Ao4WE0hV8QAOYvBwoBKM6gy_TGv8K2EHD839Q.HEp71hiU bxeY1G4jDe0PKmqKgpzCXOb2cwtVuSU8.9MsyGAWvHe_YAjxQPiC8pRQpXCxW5CRyWwpvyINz6k4 h516An43lVba8dolfDD7Cf.esSLl8R43CrW1n5ge.Raz.ZIjvaLFqJSAV_sW4XpgCNhjENyDoKjj oubq1TrEC2VkccaaVmJe3ja_1MNnEde7ljiKvMPeR4szEcivDeij67RZwfn7kOAhXmgv20.Kv1GN DZyJcUVU1fELowUbKiLplZeUV_7xVgHXay5h1J0POV.KDyVeV7bMu_S.GgNyg5tIbTXEJ0cvyBja MG1Gj2.LE7c_n14cM7cufaKMcLOmefh2HqEsBFJm6eFjuNNZNHaBdkTj1 Received: from sonic.gate.mail.ne1.yahoo.com by sonic309.consmr.mail.gq1.yahoo.com with HTTP; Tue, 19 Feb 2019 20:36:21 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.113]) ([67.170.167.181]) by smtp424.mail.gq1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 81a98bfc8044de29f7cb753b027a4199; Tue, 19 Feb 2019 20:36:21 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: Re: Question on "Map the entire KVA range into the SLB. We must not fault there" vs. the modern VM_MAX_KERNEL_ADDRESS From: Mark Millard In-Reply-To: <20190219134946.125d91a9@ralga.knownspace> Date: Tue, 19 Feb 2019 12:36:20 -0800 Cc: FreeBSD PowerPC ML Content-Transfer-Encoding: 7bit Message-Id: References: <20190219134946.125d91a9@ralga.knownspace> To: Justin Hibbits X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: B906F72729 X-Spamd-Bar: + X-Spamd-Result: default: False [1.59 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.69)[0.688,0]; NEURAL_HAM_LONG(-0.28)[-0.280,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(1.03)[ip: (3.84), ipnet: 98.137.64.0/21(0.78), asn: 36647(0.63), country: US(-0.07)]; NEURAL_SPAM_MEDIUM(0.66)[0.661,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[147.65.137.98.list.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Feb 2019 20:36:27 -0000 On 2019-Feb-19, at 11:49, Justin Hibbits wrote: > On Mon, 18 Feb 2019 18:58:43 -0800 > Mark Millard wrote: > >> It takes me a bit to provide context for the question . . . >> >> I'll first note: >> >> #define VM_MAX_SAFE_KERNEL_ADDRESS VM_MAX_KERNEL_ADDRESS >> >> moea64_mid_bootstrap has: >> >> virtual_avail = VM_MIN_KERNEL_ADDRESS; >> virtual_end = VM_MAX_SAFE_KERNEL_ADDRESS; >> >> /* >> * Map the entire KVA range into the SLB. We must not fault >> there. */ >> #ifdef __powerpc64__ >> for (va = virtual_avail; va < virtual_end; va += >> SEGMENT_LENGTH) moea64_bootstrap_slb_prefault(va, 0); >> #endif >> >> but the prefaulting is based on: >> >> #define >> PCPU_MD_AIM64_FIELDS \ >> struct slb slb[64]; \ >> >> where the slb is based on the upper 36 bits of the passed-in va values >> (effective address value in powerpc terms). >> >> The address range now being covered is: >> >> #define VM_MIN_KERNEL_ADDRESS 0xe000000000000000 >> #define VM_MAX_KERNEL_ADDRESS 0xe0000007ffffffff >> >> So for the upper 36 bits: >> >> 0xe000_0000_0 >> 0xe000_0007_f >> >> and the range 0x00 through 0x7f has 128 possibilities. >> >> So more than the slb can track. (Also: slb[USER_SLB_SLOT] >> is avoided so really 63 of the 64 are available.) >> >> Later (larger) va values are the replacing earlier >> (smaller) va values via slb_insert_kernel usage that >> in turn involves random replacements via (n_slbs >> being 64 in the G5 context): >> >> i = mftb() % n_slbs; >> if (i == USER_SLB_SLOT) >> i = (i+1) % n_slbs; >> >> So apparently only the last 63 from: >> >> for (va = virtual_avail; va < virtual_end; va += >> SEGMENT_LENGTH) moea64_bootstrap_slb_prefault(va, 0); >> >> are known to still be present in the slb. >> >> >> >> So the question . . . >> >> How can: >> >> "Map the entire KVA range into the SLB. We must not fault there" >> >> be true? Is it really a requirement? If it is, then something seems >> to be wrong as far as handling the modern VM_MAX_KERNEL_ADDRESS value >> goes. >> >> >> The old VM_MAX_KERNEL_ADDRESS value would have 0x00 through 0x1c >> and so would fit this particular constraint. The old value >> 0xe0000001c7ffffff looks like something was being avoided >> (why not 0xe0000001ffffffff ?). But I've not found what or why. > > If this were cause for concern I'd likely see it on my POWER9, which > actually only has 32 SLB entries, 4 of which are pinned, the remaining > 28 being FIFO. Much less than 64 on the G5. The code that I referenced does not pin 4 and does not FIFO 28. "i = mftb() % n_slbs;" is not FIFO/queueing logic for determining replacement. Only USER_SLB_SLOT is pinned. So I'm a bit surprised that the contexts are similar enough for comparison. But I'll take your word for it. The quoted comment should probably be replaced as misleading. I've still no clue why sometimes moea64_cpu_bootstrap_native's 0x25 "labeling" that I added stays visible to CPU 0 and the 0x20 label (or later) on returning to pmap_cpu_bootstrap (or later activity) do not show up. One hypothesis is that the return to pmap_cpu_bootstrap failed on CPU 3 so that the 0x20 and the later values are not written. === Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-ppc@freebsd.org Wed Feb 20 13:30:07 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BB97914F437A for ; Wed, 20 Feb 2019 13:30:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 544FC77B45 for ; Wed, 20 Feb 2019 13:30:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 0FDF614F4379; Wed, 20 Feb 2019 13:30:07 +0000 (UTC) Delivered-To: ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EFF3414F4378 for ; Wed, 20 Feb 2019 13:30:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 845C977B3E for ; Wed, 20 Feb 2019 13:30:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id C1FA7E18E for ; Wed, 20 Feb 2019 13:30:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x1KDU59Y010065 for ; Wed, 20 Feb 2019 13:30:05 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x1KDU54B010064 for ppc@FreeBSD.org; Wed, 20 Feb 2019 13:30:05 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: ppc@FreeBSD.org Subject: [Bug 233377] [PowerPC64] Panic during high disk I/O activity Date: Wed, 20 Feb 2019 13:30:05 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: sbruno@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ppc@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: resolution bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Feb 2019 13:30:07 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D233377 Sean Bruno changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Status|New |Closed --- Comment #13 from Sean Bruno --- We just completed a full package set rebuild on pylon.nyi.freebsd.org Marking this as fixed. Thank you! --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-ppc@freebsd.org Thu Feb 21 21:01:30 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4D80714F0A37 for ; Thu, 21 Feb 2019 21:01:30 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic302-4.consmr.mail.bf2.yahoo.com (sonic302-4.consmr.mail.bf2.yahoo.com [74.6.135.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 345258D490 for ; Thu, 21 Feb 2019 21:01:27 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: bD7p1zoVM1nrPr3BfVmhTDiHOJ0QelnZQMGuUTheYHgfEN5lr.aA86Lp3Q5Vi1A .tkJ.7SxogAgBPZa81m3_WUSzVW2B7j_ihRHWcHON6ZW5LaNV7pnYXJbapE6qr3svFlP5TL3hliH im67FCQfoqgSkJR64WghSn1dlx9Gop8BLbay4x1yTsDJanZYpR_mU.ACbm07HnTLuYH3QgWpWtSq .JLMQd045mlTT0oLhvYj8B.k0UENPS0AkVi4NeIvz6ASiNRR0rmdv25OuCR_WNxJ_KMpEPaHvce9 qO8qehD1X.LgtNGiTErB.5NWo7TBAO1MVV3KliLdVSqw314.SKETZ5fn1FaESDQa75O3Fy7um3MG 4nJHFLsNlbBOjyyzsC.qS_BSvcc0WqSVQ1brOpZqTO.3EawJ2Bth_U0HPQPQIsDAezbiwyqGkw24 dsUsJSdAGu57tD2NCdglMSfiNpCIr3L37EHdkhXuilV65.5QtbGGuCqGHaegfjTTc7WkVJGJ7LmS rhOB3MI53vNQQXDf4WNDG5A_YpuwaCysnwEHotK_A4xKtLCT1lS0A567Iy4egRGK6H7Mz1t33.5u i89Lx0lAPNRlibOsfowJWaopMUr3r7nQiP9KePbvmj0_NZQ6oWtQ6VHjJrm91__q9ovJFmF97d_J V_s7z7YhG721jb3ir8pquS_SCGlDKoOjOy.ROtuOV8_3QZiM0gTvTonCqDSBE3EUz7Jb6I9UrR0_ jlkKlb47tjlF.tOFfK66i8UPaaYZXiyNfI.sT.lAQpu.qORWbTBq6U3kNtPhUTUzmJuv9TyxafkC Zv.bDR5qN1CEn9O5VsUgI669pC6kqcuKu07TJK5j2TXl52DpBwF_evEbOD_hUBfbvgyTJcjLHqPL 9uBABhN4P3jv8.nsAcBTqXdP4xDWQCF87UImQjcc_vKSslgsRYC0YlBTeYlfmx9GSU9CCocRkHm9 Fo0zTrJs21ZMr0sJGTmHMoP3PivsF5FKloplYG0NoFRnVaJFYJup0NIWe_R.n0uW_uHj5OSi7CS2 0Pg29DIcHP1Rh.2BcixZhBfATZTQdyynw8mmfxD19fPeJa.AxO10jyLyqEmY1 Received: from sonic.gate.mail.ne1.yahoo.com by sonic302.consmr.mail.bf2.yahoo.com with HTTP; Thu, 21 Feb 2019 21:01:25 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.113]) ([67.170.167.181]) by smtp401.mail.bf1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID bc6b9d2a575ae24626c33fd0f964c126; Thu, 21 Feb 2019 21:01:21 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: Re: Cleaned -up evidence about the PowerMac G5 multiprocessor boot hang ups with the modern VM_MAX_KERNEL_ADDRESS value [pcpup->pc_curpcb->pcb_sp sometimes fails] From: Mark Millard In-Reply-To: Date: Thu, 21 Feb 2019 13:01:18 -0800 Cc: FreeBSD PowerPC ML , Dennis Clarke Content-Transfer-Encoding: quoted-printable Message-Id: References: <924A8F35-88F6-4281-85D6-AF25161DAC8A@yahoo.com> To: Justin Hibbits X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: 345258D490 X-Spamd-Bar: ++ X-Spamd-Result: default: False [2.47 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MX_GOOD(-0.01)[mta6.am0.yahoodns.net, mta5.am0.yahoodns.net, mta7.am0.yahoodns.net, mta6.am0.yahoodns.net, mta5.am0.yahoodns.net, mta7.am0.yahoodns.net, mta6.am0.yahoodns.net, mta5.am0.yahoodns.net, mta7.am0.yahoodns.net, mta6.am0.yahoodns.net, mta5.am0.yahoodns.net, mta7.am0.yahoodns.net, mta6.am0.yahoodns.net, mta5.am0.yahoodns.net, mta7.am0.yahoodns.net, mta6.am0.yahoodns.net, mta5.am0.yahoodns.net, mta7.am0.yahoodns.net, mta6.am0.yahoodns.net, mta5.am0.yahoodns.net, mta7.am0.yahoodns.net, mta6.am0.yahoodns.net, mta5.am0.yahoodns.net, mta7.am0.yahoodns.net, mta6.am0.yahoodns.net, mta5.am0.yahoodns.net, mta7.am0.yahoodns.net, mta6.am0.yahoodns.net, mta5.am0.yahoodns.net, mta7.am0.yahoodns.net, mta6.am0.yahoodns.net, mta5.am0.yahoodns.net, mta7.am0.yahoodns.net, mta6.am0.yahoodns.net, mta5.am0.yahoodns.net, mta7.am0.yahoodns.net, mta6.am0.yahoodns.net, mta5.am0.yahoodns.net, mta7.am0.yahoodns.net, mta6.am0.yahoodns.net, mta5.am0.yahoodns.net, mta7.am0.yahoodns.net, mta6.am0.yahoodns.net, mta5.am0.yahoodns.net, mta7.am0.yaho odns.net,mta6.am0.yahoodns.net,mta5.am0.yahoodns.net,mta7.am0.yahoodns.net,mta6.am0.yahoodns.net,mta5.am0.yahoodns.net,mta7.am0.yahoodns.net]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:26101, ipnet:74.6.128.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_SPAM_SHORT(0.95)[0.952,0]; NEURAL_HAM_LONG(-0.04)[-0.045,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(1.40)[ip: (4.40), ipnet: 74.6.128.0/21(1.49), asn: 26101(1.19), country: US(-0.07)]; NEURAL_SPAM_MEDIUM(0.67)[0.670,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[43.135.6.74.list.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Feb 2019 21:01:30 -0000 [A possible surprise is that the same pcpup->pc_curpcb value for CPU 3 is present for hangs and for completing boots: 0xe0000000740cfac0 . Also: I've dropped historical text for this note.] Justin Hibbits pointed out that of course I'd not see the 0x20 "label": I had stupidly placed it after a return statement without noticing. See below. void pmap_cpu_bootstrap(int ap) { /* * No KTR here because our console probably doesn't work yet */ return (MMU_CPU_BOOTSTRAP(mmu_obj, ap)); =20 *(volatile unsigned long*)0xc0000000000000f0 =3D 0x20; // = HACK!!! powerpc_sync(); // HACK!!! } (The original void-return function has that return with an expression. Not that such should have misdirected my thinking.) So the expected/desired value to see after 0x25 would not be the 0x20 "label". (So I've now eliminated the lines that I had added.) Thus the next value after the 0x25 from moea64_cpu_bootstrap_native for a hang-up would have been 0x30 from cpudep_ap_bootstrap . I've updated cpudep_ap_bootstrap to record more "labels" for places reached: uintptr_t cpudep_ap_bootstrap(void) { register_t msr, sp; *(volatile unsigned long*)0xc0000000000000f0 =3D 0x3F; // = HACK!!! powerpc_sync(); // HACK!!! msr =3D psl_kernset & ~PSL_EE; mtmsr(msr); *(volatile unsigned long*)0xc0000000000000f0 =3D 0x31; // = HACK!!! powerpc_sync(); // HACK!!! pcpup->pc_curthread =3D pcpup->pc_idlethread; *(volatile unsigned long*)0xc0000000000000f0 =3D 0x32; // = HACK!!! powerpc_sync(); // HACK!!! #ifdef __powerpc64__ __asm __volatile("mr 13,%0" :: "r"(pcpup->pc_curthread)); #else __asm __volatile("mr 2,%0" :: "r"(pcpup->pc_curthread)); #endif *(volatile unsigned long*)0xc0000000000000f0 =3D 0x33; // = HACK!!! powerpc_sync(); // HACK!!! pcpup->pc_curpcb =3D pcpup->pc_curthread->td_pcb; =20 *(volatile unsigned long*)0xc0000000000000f0 =3D 0x34; // = HACK!!! powerpc_sync(); // HACK!!! =20 sp =3D pcpup->pc_curpcb->pcb_sp; *(volatile unsigned long*)0xc0000000000000f0 =3D 0x30; // = HACK!!! powerpc_sync(); // HACK!!! return (sp); } The result for hanging boots is "label": 0x34 is reported by CPU 0. Thus it appears that pcpup->pc_curthread->td_pcb and (so) = pcpup->pc_curpcb end up with pointer value(s) that sometimes block: pcpup->pc_curpcb-> from being used, although the pointer values need not be different. (Later below they are shown to not be different for hangs vs. finishes). Thus I added recording of the address in question: pcpup->pc_curpcb =3D pcpup->pc_curthread->td_pcb; *(volatile void**)0xc0000000000000e0 =3D pcpup->pc_curpcb; // = HACK!!! powerpc_sync(); // HACK!!! *(volatile unsigned long*)0xc0000000000000f0 =3D 0x34; // = HACK!!! powerpc_sync(); // HACK!!! sp =3D pcpup->pc_curpcb->pcb_sp; and added reporting of the value placed at 0xc0000000000000e0 : *rstvec =3D 4; powerpc_sync(); (void)(*rstvec); powerpc_sync(); DELAY(1); *rstvec =3D 0; powerpc_sync(); (void)(*rstvec); powerpc_sync(); if (bootverbose) // HACK!!! printf("After reset 4&0 for CPU %d, hwref=3D%jx, = awake=3D%x, n_slbs=3D%jd,\n" " *(volatile void**)0xc0000000000000e0=3D%p,\n" " *(volatile unsigned = long*)0xc0000000000000f0=3D0x%jx\n", pc->pc_cpuid, (uintmax_t)pc->pc_hwref, pc->pc_awake, (uintmax_t)n_slbs, *(volatile void**)0xc0000000000000e0, (uintmax_t)*(volatile unsigned = long*)0xc0000000000000f0); timeout =3D 10000; while (!pc->pc_awake && timeout--) DELAY(100); if (bootverbose) // HACK!!! printf("After attempted wait for awake CPU %d, = hwref=3D%jx, awake=3D%x, n_slbs=3D%jd, delay 100 count =3D %jd,\n" " *(volatile void**)0xc0000000000000e0=3D%p,\n" " *(volatile unsigned = long*)0xc0000000000000f0=3D0x%jx\n", pc->pc_cpuid, (uintmax_t)pc->pc_hwref, pc->pc_awake, (uintmax_t)n_slbs, = (uintmax_t)(10000-timeout), *(volatile void**)0xc0000000000000e0, (uintmax_t)*(volatile unsigned = long*)0xc0000000000000f0); return ((pc->pc_awake) ? 0 : EBUSY); The values of *(volatile void**)0xc0000000000000e0 (i.e., copies of pcpup->pc_curpcb) after attempting to wait for pc->pc_awake for CPU 3 are: boots: 0x0xe0000000740cfac0 hangs: 0x0xe0000000740cfac0 So: no difference in value. So sometimes the address appears valid to dereference on CPU 3 and other times the same address does not. A successful boot looks like: Adding CPU 0, hwref=3Dcd38, awake=3D1 Waking up CPU 3 (dev=3Dc480) After reset 4&0 for CPU 3, hwref=3Dc480, awake=3D0, n_slbs=3D64, *(volatile void**)0xc0000000000000e0=3D0, *(volatile unsigned long*)0xc0000000000000f0=3D0x25 After attempted wait for awake CPU 3, hwref=3Dc480, awake=3D1, = n_slbs=3D64, delay 100 count =3D 0, *(volatile void**)0xc0000000000000e0=3D0xe0000000740cfac0, *(volatile unsigned long*)0xc0000000000000f0=3D0x51 Adding CPU 3, hwref=3Dc480, awake=3D1 Waking up CPU 2 (dev=3Dc768) After reset 4&0 for CPU 2, hwref=3Dc768, awake=3D0, n_slbs=3D64, *(volatile void**)0xc0000000000000e0=3D0xe0000000740cfac0, *(volatile unsigned long*)0xc0000000000000f0=3D0x51 After attempted wait for awake CPU 2, hwref=3Dc768, awake=3D1, = n_slbs=3D64, delay 100 count =3D 0, *(volatile void**)0xc0000000000000e0=3D0xe0000000740d8ac0, *(volatile unsigned long*)0xc0000000000000f0=3D0x51 Adding CPU 2, hwref=3Dc768, awake=3D1 Waking up CPU 1 (dev=3Dca50) After reset 4&0 for CPU 1, hwref=3Dca50, awake=3D0, n_slbs=3D64, *(volatile void**)0xc0000000000000e0=3D0xe0000000740d8ac0, *(volatile unsigned long*)0xc0000000000000f0=3D0x51 After attempted wait for awake CPU 1, hwref=3Dca50, awake=3D1, = n_slbs=3D64, delay 100 count =3D 0, *(volatile void**)0xc0000000000000e0=3D0xe0000000740e1ac0, *(volatile unsigned long*)0xc0000000000000f0=3D0x51 Adding CPU 1, hwref=3Dca50, awake=3D1 SMP: AP CPU #2 launched SMP: AP CPU #3 launched SMP: AP CPU #1 launched A hanging boot looks like (from a picture): Adding CPU 0, hwref=3Dcd38, awake=3D1 Waking up CPU 3 (dev=3Dc480) After reset 4&0 for CPU 3, hwref=3Dc480, awake=3D0, n_slbs=3D64, *(volatile void**)0xc0000000000000e0=3D0, *(volatile unsigned long*)0xc0000000000000f0=3D0x25 After attempted wait for awake CPU 3, hwref=3Dc480, awake=3D1, = n_slbs=3D64, delay 100 count =3D 0, *(volatile void**)0xc0000000000000e0=3D0xe0000000740cfac0, *(volatile unsigned long*)0xc0000000000000f0=3D0x34 Waking up CPU 2 (dev=3Dc768) =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-ppc@freebsd.org Sat Feb 23 13:49:08 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E6A1D1516D19 for ; Sat, 23 Feb 2019 13:49:07 +0000 (UTC) (envelope-from bacon4000@gmail.com) Received: from mail-oi1-x243.google.com (mail-oi1-x243.google.com [IPv6:2607:f8b0:4864:20::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E017F8BA63 for ; Sat, 23 Feb 2019 13:49:06 +0000 (UTC) (envelope-from bacon4000@gmail.com) Received: by mail-oi1-x243.google.com with SMTP id t206so3957676oib.3 for ; Sat, 23 Feb 2019 05:49:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:from:to:cc:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=Z+UtLIaArciZCMcHZdJNec4YgkIa98jMVSWcgCdkQLE=; b=YiIOrEF5kKEgt9TtGgpcmXLsk7z7xnzriES8nQmunWOIhaDXEoJvXd2NCJZqA+74t+ snzJiZ/AdXxyCoPnzGsF6sWF3vrxR5o1VIgKnpadQG1HQ1f5lWDyE6ss5feOWlAEt463 3JA3qj8J9VmoGoWKVMNt4Y1493qaqKlKcmAT+F8/gIaEhlmwPG2qVevi00cKB4VQ+rOT 1TbAckqk1jqbvtuHsiyK5zxZlQUuZOB64roEUn7BuyoV7A3RMt/6aFYaXK6w4zZZaDqH R/NvZRWAFsl77QkYvxN3E6Xu+q3hh6B+R7UlfW0HM370w+GqiuN5NTZ+OlpL9HwnAGGU YN7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:references:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=Z+UtLIaArciZCMcHZdJNec4YgkIa98jMVSWcgCdkQLE=; b=TfE5YbW9ZgyMSz1gBG3LknRxnvLU2RtwhnYRmt8h7RCJUXY1WAliI6VdFcQALb49vI ojHqkB900lfH8sGZG+/tg5J5vTYD9uX9uWBvGCHIEy1mvDvhe/8iIBUgItSJNWp1LQ11 q7MCrlbZqaGRJvgF4EQ4gSnfEXyz3wsOqi60YczXKrejm/liJPe7XYvapAgNkZKd13kq ZEVrxXRkm3nMncqGMWg5avUKdYECstGfTjFWmDJN/IIcYkpu4F5aBNeacvGPiJrTnr7x kNQ878pdgSPiTgxVUdqF6YDyLfj9Y41tvr3FLnTduARcjjH3kSOoUfCAYkWXrthnJ0+I iiYQ== X-Gm-Message-State: AHQUAuZHiJgHGNM9WHfPlZeZr1kVMfNgIbOt7yDxwT6urt6G9UDThq/D OgRgrcYvILs2o018YDh81Wc= X-Google-Smtp-Source: AHgI3IYlETx75KhiWX+Mxy2YTK97zRrWRQ/z6F0D54PoNJSPo/Hpa9HD2Qfl5aO5C+/CDMYYYOBwLQ== X-Received: by 2002:aca:ac45:: with SMTP id v66mr5354697oie.134.1550929746064; Sat, 23 Feb 2019 05:49:06 -0800 (PST) Received: from cray.acadix.biz (cpe-174-102-163-140.wi.res.rr.com. [174.102.163.140]) by smtp.gmail.com with ESMTPSA id v62sm1861418oie.21.2019.02.23.05.49.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 23 Feb 2019 05:49:05 -0800 (PST) Subject: Re: QEMU From: Jason Bacon To: Chuck Tuffli Cc: freebsd-ppc@freebsd.org, Muhammad Moinur Rahman References: <5f291124-612f-6d10-5012-a8701b1cf49e@gmail.com> <5302f073-b51b-c92f-ada2-f7123d27fa3d@gmail.com> <8213518b-14a5-aac2-bcbb-529e49c4f044@gmail.com> Message-ID: <9f96d3ac-ada3-8f41-6c2c-e6fab80e49e9@gmail.com> Date: Sat, 23 Feb 2019 07:49:03 -0600 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.5.1 MIME-Version: 1.0 In-Reply-To: <8213518b-14a5-aac2-bcbb-529e49c4f044@gmail.com> Content-Type: multipart/mixed; boundary="------------B66E56AEA7773B15A0A397F0" Content-Language: en-US X-Rspamd-Queue-Id: E017F8BA63 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=YiIOrEF5; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of bacon4000@gmail.com designates 2607:f8b0:4864:20::243 as permitted sender) smtp.mailfrom=bacon4000@gmail.com X-Spamd-Result: default: False [-1.49 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; HAS_ATTACHMENT(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; CTYPE_MIXED_BOGUS(1.00)[]; MIME_BASE64_TEXT(0.10)[]; SUBJ_ALL_CAPS(0.30)[4]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.92)[-0.923,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_LONG(-1.00)[-0.996,0]; TAGGED_RCPT(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-ppc@freebsd.org]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; NEURAL_HAM_SHORT(-0.14)[-0.143,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[3.4.2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(0.18)[ip: (5.61), ipnet: 2607:f8b0::/32(-2.63), asn: 15169(-1.99), country: US(-0.07)] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 Feb 2019 13:49:08 -0000 This is a multi-part message in MIME format. --------------B66E56AEA7773B15A0A397F0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit On 7/21/18 1:06 PM, Jason Bacon wrote: > On 7/19/18 4:32 PM, Jason Bacon wrote: >> On 07/19/18 14:09, Chuck Tuffli wrote: >>> On Thu, Jul 12, 2018 at 8:09 AM, Jason Bacon >>> wrote: >>>> FYI, I get the exact same behavior under qemu 2.8.1 on Debian. >>>> >>>> So now we have similar symptoms in qemu 2.8.1, 2.9, and 2.12.50 on >>>> FreeBSD >>>> and Linux hosts. >>> FWIW, on an Ubuntu 14.04 system with qemu-system-ppc64 version 2.0.0, >>> the ppc64 snapshot ISO of 12.0, the OS appears to install correctly >>> and subsequently boots correctly. >>> >>> --chuck >> That's worth a lot, actually. >> >> The 12.0 snapshot also works on my FreeBSD 11.1 host with the stock >> qemu package.  Both keyboard and mouse input are processed. >> >> Interestingly, though, while 12.0 works, it seems to be a lot slower >> than 11.1 under qemu.  Below are times to get to the install screen.  >> ( I just close the qemu window as soon as it reaches that point, >> where 11.1 won't accept keyboard input. ) >> >> FreeBSD cray.acadix  bacon ~ 999: time qemu-ppc install >> freebsd-ppc.img >> FreeBSD-12.0-CURRENT-powerpc-powerpc64-20180709-r336134-disc1.iso >> + [ ! -e freebsd-ppc.img ] >> + qemu-system-ppc64 -cdrom >> FreeBSD-12.0-CURRENT-powerpc-powerpc64-20180709-r336134-disc1.iso >> -drive 'file=freebsd-ppc.img,format=raw' -boot d >> 217.327u 3.455s 4:21.41 84.4%    9628+6292k 94+2io 476pf+0w >> >> >> FreeBSD cray.acadix  bacon ~ 1000: time qemu-ppc install >> freebsd-ppc.img Save/FreeBSD-11.1-RELEASE-powerpc-powerpc64-disc1.iso >> + [ ! -e freebsd-ppc.img ] >> + qemu-system-ppc64 -cdrom >> Save/FreeBSD-11.1-RELEASE-powerpc-powerpc64-disc1.iso -drive >> 'file=freebsd-ppc.img,format=raw' -boot d >> 123.001u 1.748s 2:47.05 74.6%    9643+6302k 556+3io 0pf+0w >> >> Maybe these data will provide some clues to the ppc base developers... >> > I'm getting "lock order reversal" errors followed by stack traces when > running portsnap.  Bleeding-edge 12.0 issue? > Poked around at this a bit more and found a workaround.  It seems FreeBSD doesn't support the latest default PPC machine in qemu. Available options are listed below.  After switching from the default pseries-2.6 to pseries-2.5, FreeBSD 12.0 works flawlessly. I attached a script I'm using to install and then boot the VM. So now there's an easy way to test/fix ports for PPC64.  Runs about as fast as a 486, but that's fine since we can install dependencies via "pkg install" to reduce build time. FreeBSD cray.acadix  bacon ~ 1010: qemu-system-ppc -machine help Supported machines are: bamboo               bamboo g3beige              Heathrow based PowerMAC (default) mac99                Mac99 based PowerMAC mpc8544ds            mpc8544ds none                 empty machine ppce500              generic paravirt e500 platform prep                 PowerPC PREP platform ref405ep             ref405ep taihu                taihu virtex-ml507         Xilinx Virtex ML507 reference design FreeBSD cray.acadix  bacon ~ 1011: qemu-system-ppc64 -machine help Supported machines are: bamboo               bamboo g3beige              Heathrow based PowerMAC mac99                Mac99 based PowerMAC mpc8544ds            mpc8544ds none                 empty machine ppce500              generic paravirt e500 platform prep                 PowerPC PREP platform pseries-2.1          pSeries Logical Partition (PAPR compliant) pseries-2.2          pSeries Logical Partition (PAPR compliant) pseries-2.3          pSeries Logical Partition (PAPR compliant) pseries-2.4          pSeries Logical Partition (PAPR compliant) pseries-2.5          pSeries Logical Partition (PAPR compliant) pseries              pSeries Logical Partition (PAPR compliant) (alias of pseries-2.6) pseries-2.6          pSeries Logical Partition (PAPR compliant) (default) ref405ep             ref405ep taihu                taihu virtex-ml507         Xilinx Virtex ML507 reference design -- Earth is a beta site. --------------B66E56AEA7773B15A0A397F0 Content-Type: text/plain; charset=UTF-8; name="qemu-ppc64" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="qemu-ppc64" IyEvYmluL3NoIC1lCgojIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIwojICAgU2NyaXB0IGRlc2NyaXB0 aW9uOgojICAgICAgIEluc3RhbGwvYm9vdCBGcmVlQlNELXBvd2VycGMgb24gcWVtdQojICAg ICAgIAojICAgSGlzdG9yeToKIyAgIERhdGUgICAgICAgIE5hbWUgICAgICAgIE1vZGlmaWNh dGlvbgojICAgMjAxOC0wNy0wNyAgSmFzb24gQmFjb24gQmVnaW4KIwojICAgaHR0cHM6Ly93 aWtpLmZyZWVic2Qub3JnL1FlbXVSZWNpcGVzCiMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjCgp1c2Fn ZSgpCnsKICAgIHByaW50ZiAiVXNhZ2U6ICQwIGluc3RhbGx8Ym9vdCBkaXNrLWltYWdlIHJh d3xxY293MiBbY2QtaW1hZ2VdXG4iCiAgICBleGl0IDEKfQoKCiMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjCiMgICBNYWluCiMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjCgppZiBbICQjIC1sdCAzIF07IHRo ZW4KICAgIHVzYWdlCmZpCgpjbWQ9JDEKZGlza2ltYWdlPSQyCmZvcm1hdD0kMwoKY2FzZSAk Y21kIGluCmluc3RhbGwpCiAgICBjZGltYWdlPSQ0CiAgICBzZXQgLXgKICAgIGlmIFsgISAt ZSAkZGlza2ltYWdlIF07IHRoZW4KCXFlbXUtaW1nIGNyZWF0ZSAtZiAkZm9ybWF0ICRkaXNr aW1hZ2UgMjBnCiAgICBmaQogICAgIyBxZW11LXN5c3RlbS1wcGM2NCAtbm9ncmFwaGljCiAg ICAjIHFlbXUtc3lzdGVtLXBwYzY0IC1tIDIwNDggLW1hY2hpbmUgcHNlcmllcy0yLjUKICAg IHFlbXUtc3lzdGVtLXBwYzY0IC1tYWNoaW5lIHBzZXJpZXMtMi41IFwKCS1jZHJvbSAkY2Rp bWFnZSAtZHJpdmUgZmlsZT0kZGlza2ltYWdlLGZvcm1hdD0kZm9ybWF0IFwKCS1ib290IGQK ICAgIDs7Cgpib290KQogICAgcWVtdS1zeXN0ZW0tcHBjNjQgLW0gMjA0OCAtbWFjaGluZSBw c2VyaWVzLTIuNSBcCgktZHJpdmUgZmlsZT0kZGlza2ltYWdlLGZvcm1hdD0kZm9ybWF0IC1i b290IGMKICAgIDs7CgoqKQogICAgdXNhZ2UKICAgIDs7Cgplc2FjCg== --------------B66E56AEA7773B15A0A397F0-- From owner-freebsd-ppc@freebsd.org Sat Feb 23 18:23:48 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 716331520795 for ; Sat, 23 Feb 2019 18:23:48 +0000 (UTC) (envelope-from gardask@gmail.com) Received: from mail-oi1-x22b.google.com (mail-oi1-x22b.google.com [IPv6:2607:f8b0:4864:20::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5B7F9978A6 for ; Sat, 23 Feb 2019 18:23:47 +0000 (UTC) (envelope-from gardask@gmail.com) Received: by mail-oi1-x22b.google.com with SMTP id s16so4254582oih.9 for ; Sat, 23 Feb 2019 10:23:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=to:from:subject:message-id:date:user-agent:mime-version :content-transfer-encoding:content-language; bh=Jlq+8f7BAjI/GA6JHD7sxmx99vDNLVns71M2fznynLI=; b=WTk6YLEiyvhdOP9/W5mtZkI1s+H2+EDZDYCREeJrnL7aPsSAWgjTMYNYRmC1am2nJC EA5J+FHxzt0pEe7UyaMHCrnQWPQ5W/btmQBLBhdvBDqKN7ZXff9+JFlrCgvx6J2IBjXS kl1bQ1gz0+iFE7LO/ZtkiwftxDPCbbEAKsmIikzDYsjibCzl9FXx9DEBMrp3qbAQ37md TvPCRY9QrBa8kah8yCz6m+BSQ3cxLYkjFIECw749kWgb1Q3RLTEW1xKb60X12pmIcfZs Jf6MdP4Up0DljjtjEjTrRIJ6s6ideoxPW5rMlHArZQQXUjk6/Zs2WaaykEIwBxSasFha +EhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version:content-transfer-encoding:content-language; bh=Jlq+8f7BAjI/GA6JHD7sxmx99vDNLVns71M2fznynLI=; b=ayKIogmXwPRKhyOYi6uqoIKa44AMNh8X3aNUpj1/5f3iS5YPGzrmWwCOXum5DUPGxe OFbAFl6bMm/L1BfLnbJ16qJTzd6UJTF5nG/VFzkR2OpE+gsWOlc9EorWMbHziZxRraqX kuLUa1ESJtuILvGbC+qG8zSECZRg2tEMI3n4gGtJq+Ul3H7k6Q+KcoaCWv1fd0P2CnXp IfFANHSoy8v+Yqnax+RlehjPpaslR6/9MAND3qNfa2lBFhXPhFiM53pPa7U4SOdDJnO7 Fn+rYtcpRbjlCLRxqOukFBbxe3PJ/ClueO3I1f6k17zVZ3I0UPwWK+ZUznuhIlsx7RrQ hoEw== X-Gm-Message-State: AHQUAuZIqySVVBu6/CSe0sdou2oCTaqBbj3tNls/++sFZ5UhUYMIR7Ge dpX6zm/X7hDw/FThB8q/giZhnQcS X-Google-Smtp-Source: AHgI3IYrs62VkTN5t7BAW7amikEUwzUVL4ufjT2sDjea+qQFWVe89PTwx9ZyoU7nrehMhQcNUu2JmQ== X-Received: by 2002:aca:5343:: with SMTP id h64mr6329910oib.69.1550946226318; Sat, 23 Feb 2019 10:23:46 -0800 (PST) Received: from thinkpad.localdomain ([31.47.99.1]) by smtp.gmail.com with ESMTPSA id v8sm1964594otq.4.2019.02.23.10.23.45 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 23 Feb 2019 10:23:45 -0800 (PST) To: freebsd-ppc@freebsd.org From: Karel Gardas Subject: Installation fails on Tyan POWER8 box with ahci command timeouts. Message-ID: <6278e350-1cbd-2882-0a2e-7fd7f67b09e6@gmail.com> Date: Sat, 23 Feb 2019 19:23:40 +0100 User-Agent: Mozilla/5.0 (X11; OpenBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-GB X-Rspamd-Queue-Id: 5B7F9978A6 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=WTk6YLEi; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of gardask@gmail.com designates 2607:f8b0:4864:20::22b as permitted sender) smtp.mailfrom=gardask@gmail.com X-Spamd-Result: default: False [-6.71 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.94)[-0.937,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-ppc@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-2.76)[ip: (-9.13), ipnet: 2607:f8b0::/32(-2.63), asn: 15169(-1.99), country: US(-0.07)]; RCVD_IN_DNSWL_NONE(0.00)[b.2.2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 Feb 2019 18:23:48 -0000 Hello, my attempt to install 12.0 on Tyan TN71-BP012 box with POWER8 CPU fails while extracting distribution files. The message -- probably kernel complain as it "destroys" nice curses dialog of Archive Extraction -- contains this: ahcich1: Timeout on slot 0 port 0 ahcich1: is 00000000 cs fdffffff ss ffffffff rs ffffffff tfd 40 serr 00000000 cmd 00719917 (ada0:ahcich1:0:0:0): WRITE_FPDMA_QUEUED. ACB 61 00 28 e6 d0 40 06 00 00 01 00 00 (ada0:ahcich1:0:0:0): CAM status: Command timeout (ada0:ahcich1:0:0:0): Retrying command, 3 more tries remain ahcich1: Timeout on slot 25 port 0 ahcich1: is 00000000 cs 02000000 ss 00000000 rs 02000000 tfd 150 serr 000000000 cmd 0071c017 (apropbe0:ahcich1:0:0:0): ATA_IDENTITY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (apropbe0:ahcich1:0:0:0): CAM status: Command timeout (apropbe0:ahcich1:0:0:0): Retrying command, 0 more tries remain I'm not sure if I've shoot all before panic. I guess what happened after those messages is kernel panic and then straight reboot. The drive connected to Marvel SATA mezanine board in Tyan is WD GOLD 500GB SATA. Is there any workaround for this issue? As I'd like to have FreeBSD running on bare-metal/powernv instead of inside the PowerKVN provided in also installed ubuntu, where freebsd works fine. Thanks! Karel From owner-freebsd-ppc@freebsd.org Sat Feb 23 18:27:15 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EF9AA152098D for ; Sat, 23 Feb 2019 18:27:14 +0000 (UTC) (envelope-from cam@neo-zeon.de) Received: from neo-zeon.de (neo-zeon.de [96.90.244.226]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.neo-zeon.de", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C47B97A24 for ; Sat, 23 Feb 2019 18:27:14 +0000 (UTC) (envelope-from cam@neo-zeon.de) Received: from [192.168.0.55] (ukyo.nerv.lan [192.168.0.55]) (authenticated bits=0) by neo-zeon.de (8.15.2/8.15.2) with ESMTPSA id x1NI3feY089941 (version=TLSv1.2 cipher=AES128-SHA bits=128 verify=NO) for ; Sat, 23 Feb 2019 10:03:41 -0800 (PST) (envelope-from cam@neo-zeon.de) Subject: Re: QEMU To: freebsd-ppc@freebsd.org References: <5f291124-612f-6d10-5012-a8701b1cf49e@gmail.com> <5302f073-b51b-c92f-ada2-f7123d27fa3d@gmail.com> <8213518b-14a5-aac2-bcbb-529e49c4f044@gmail.com> <9f96d3ac-ada3-8f41-6c2c-e6fab80e49e9@gmail.com> From: Cameron Berkenpas Message-ID: <79ae62b5-f627-5a76-f824-5fdbb3f17876@neo-zeon.de> Date: Sat, 23 Feb 2019 10:03:41 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.4.0 MIME-Version: 1.0 In-Reply-To: <9f96d3ac-ada3-8f41-6c2c-e6fab80e49e9@gmail.com> Content-Language: en-US X-Rspamd-Queue-Id: 4C47B97A24 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of cam@neo-zeon.de designates 96.90.244.226 as permitted sender) smtp.mailfrom=cam@neo-zeon.de X-Spamd-Result: default: False [-3.82 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:96.90.244.224/28]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-ppc@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; DMARC_NA(0.00)[neo-zeon.de]; MX_GOOD(-0.01)[cached: mail02.neo-zeon.de]; NEURAL_HAM_SHORT(-0.83)[-0.833,0]; SUBJ_ALL_CAPS(0.30)[4]; IP_SCORE(-0.97)[ipnet: 96.64.0.0/11(-4.27), asn: 7922(-0.53), country: US(-0.07)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; ASN(0.00)[asn:7922, ipnet:96.64.0.0/11, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 Feb 2019 18:27:15 -0000 Hello, This was very helpful! I switched to pseries-2.5 and my FreeBSD VM (KVM) works again! On 2/23/19 5:49 AM, Jason Bacon wrote: > On 7/21/18 1:06 PM, Jason Bacon wrote: >> On 7/19/18 4:32 PM, Jason Bacon wrote: >>> On 07/19/18 14:09, Chuck Tuffli wrote: >>>> On Thu, Jul 12, 2018 at 8:09 AM, Jason Bacon >>>> wrote: >>>>> FYI, I get the exact same behavior under qemu 2.8.1 on Debian. >>>>> >>>>> So now we have similar symptoms in qemu 2.8.1, 2.9, and 2.12.50 on >>>>> FreeBSD >>>>> and Linux hosts. >>>> FWIW, on an Ubuntu 14.04 system with qemu-system-ppc64 version 2.0.0, >>>> the ppc64 snapshot ISO of 12.0, the OS appears to install correctly >>>> and subsequently boots correctly. >>>> >>>> --chuck >>> That's worth a lot, actually. >>> >>> The 12.0 snapshot also works on my FreeBSD 11.1 host with the stock >>> qemu package.  Both keyboard and mouse input are processed. >>> >>> Interestingly, though, while 12.0 works, it seems to be a lot slower >>> than 11.1 under qemu.  Below are times to get to the install >>> screen.  ( I just close the qemu window as soon as it reaches that >>> point, where 11.1 won't accept keyboard input. ) >>> >>> FreeBSD cray.acadix  bacon ~ 999: time qemu-ppc install >>> freebsd-ppc.img >>> FreeBSD-12.0-CURRENT-powerpc-powerpc64-20180709-r336134-disc1.iso >>> + [ ! -e freebsd-ppc.img ] >>> + qemu-system-ppc64 -cdrom >>> FreeBSD-12.0-CURRENT-powerpc-powerpc64-20180709-r336134-disc1.iso >>> -drive 'file=freebsd-ppc.img,format=raw' -boot d >>> 217.327u 3.455s 4:21.41 84.4%    9628+6292k 94+2io 476pf+0w >>> >>> >>> FreeBSD cray.acadix  bacon ~ 1000: time qemu-ppc install >>> freebsd-ppc.img Save/FreeBSD-11.1-RELEASE-powerpc-powerpc64-disc1.iso >>> + [ ! -e freebsd-ppc.img ] >>> + qemu-system-ppc64 -cdrom >>> Save/FreeBSD-11.1-RELEASE-powerpc-powerpc64-disc1.iso -drive >>> 'file=freebsd-ppc.img,format=raw' -boot d >>> 123.001u 1.748s 2:47.05 74.6%    9643+6302k 556+3io 0pf+0w >>> >>> Maybe these data will provide some clues to the ppc base developers... >>> >> I'm getting "lock order reversal" errors followed by stack traces >> when running portsnap.  Bleeding-edge 12.0 issue? >> > > Poked around at this a bit more and found a workaround.  It seems > FreeBSD doesn't support the latest default PPC machine in qemu. > Available options are listed below.  After switching from the default > pseries-2.6 to pseries-2.5, FreeBSD 12.0 works flawlessly. > > I attached a script I'm using to install and then boot the VM. > > So now there's an easy way to test/fix ports for PPC64.  Runs about as > fast as a 486, but that's fine since we can install dependencies via > "pkg install" to reduce build time. > > > FreeBSD cray.acadix  bacon ~ 1010: qemu-system-ppc -machine help > Supported machines are: > bamboo               bamboo > g3beige              Heathrow based PowerMAC (default) > mac99                Mac99 based PowerMAC > mpc8544ds            mpc8544ds > none                 empty machine > ppce500              generic paravirt e500 platform > prep                 PowerPC PREP platform > ref405ep             ref405ep > taihu                taihu > virtex-ml507         Xilinx Virtex ML507 reference design > FreeBSD cray.acadix  bacon ~ 1011: qemu-system-ppc64 -machine help > Supported machines are: > bamboo               bamboo > g3beige              Heathrow based PowerMAC > mac99                Mac99 based PowerMAC > mpc8544ds            mpc8544ds > none                 empty machine > ppce500              generic paravirt e500 platform > prep                 PowerPC PREP platform > pseries-2.1          pSeries Logical Partition (PAPR compliant) > pseries-2.2          pSeries Logical Partition (PAPR compliant) > pseries-2.3          pSeries Logical Partition (PAPR compliant) > pseries-2.4          pSeries Logical Partition (PAPR compliant) > pseries-2.5          pSeries Logical Partition (PAPR compliant) > pseries              pSeries Logical Partition (PAPR compliant) (alias > of pseries-2.6) > pseries-2.6          pSeries Logical Partition (PAPR compliant) (default) > ref405ep             ref405ep > taihu                taihu > virtex-ml507         Xilinx Virtex ML507 reference design > > > > _______________________________________________ > freebsd-ppc@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-ppc > To unsubscribe, send any mail to "freebsd-ppc-unsubscribe@freebsd.org" From owner-freebsd-ppc@freebsd.org Sat Feb 23 18:22:41 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5C3EB152075A for ; Sat, 23 Feb 2019 18:22:41 +0000 (UTC) (envelope-from cam@neo-zeon.de) Received: from neo-zeon.de (neo-zeon.de [96.90.244.226]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.neo-zeon.de", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7CBFE97896 for ; Sat, 23 Feb 2019 18:22:39 +0000 (UTC) (envelope-from cam@neo-zeon.de) Received: from [192.168.0.55] (ukyo.nerv.lan [192.168.0.55]) (authenticated bits=0) by neo-zeon.de (8.15.2/8.15.2) with ESMTPSA id x1NIMbXE090087 (version=TLSv1.2 cipher=AES128-SHA bits=128 verify=NO) for ; Sat, 23 Feb 2019 10:22:37 -0800 (PST) (envelope-from cam@neo-zeon.de) Subject: Re: QEMU From: Cameron Berkenpas To: freebsd-ppc@freebsd.org References: <5f291124-612f-6d10-5012-a8701b1cf49e@gmail.com> <5302f073-b51b-c92f-ada2-f7123d27fa3d@gmail.com> <8213518b-14a5-aac2-bcbb-529e49c4f044@gmail.com> <9f96d3ac-ada3-8f41-6c2c-e6fab80e49e9@gmail.com> <79ae62b5-f627-5a76-f824-5fdbb3f17876@neo-zeon.de> Message-ID: Date: Sat, 23 Feb 2019 10:22:37 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.4.0 MIME-Version: 1.0 In-Reply-To: <79ae62b5-f627-5a76-f824-5fdbb3f17876@neo-zeon.de> Content-Language: en-US X-Rspamd-Queue-Id: 7CBFE97896 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of cam@neo-zeon.de designates 96.90.244.226 as permitted sender) smtp.mailfrom=cam@neo-zeon.de X-Spamd-Result: default: False [-2.85 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:96.90.244.224/28]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-ppc@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; DMARC_NA(0.00)[neo-zeon.de]; NEURAL_SPAM_SHORT(0.14)[0.136,0]; MX_GOOD(-0.01)[mail02.neo-zeon.de,neo-zeon.de]; SUBJ_ALL_CAPS(0.30)[4]; IP_SCORE(-0.98)[ipnet: 96.64.0.0/11(-4.29), asn: 7922(-0.53), country: US(-0.07)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; ASN(0.00)[asn:7922, ipnet:96.64.0.0/11, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 Feb 2019 18:22:41 -0000 Scratch that, I still get: "KVM can't supply 64kiB CI pages, which guest expects" Seems I'd forgotten my VM was set to qemu mode for testing. The good news is that this works for me in with qemu. At least I can boot up quite slowly with qemu and try other solutions now. On 2/23/19 10:03 AM, Cameron Berkenpas wrote: > Hello, > > This was very helpful! I switched to pseries-2.5 and my FreeBSD VM > (KVM) works again! > > > On 2/23/19 5:49 AM, Jason Bacon wrote: >> On 7/21/18 1:06 PM, Jason Bacon wrote: >>> On 7/19/18 4:32 PM, Jason Bacon wrote: >>>> On 07/19/18 14:09, Chuck Tuffli wrote: >>>>> On Thu, Jul 12, 2018 at 8:09 AM, Jason Bacon >>>>> wrote: >>>>>> FYI, I get the exact same behavior under qemu 2.8.1 on Debian. >>>>>> >>>>>> So now we have similar symptoms in qemu 2.8.1, 2.9, and 2.12.50 >>>>>> on FreeBSD >>>>>> and Linux hosts. >>>>> FWIW, on an Ubuntu 14.04 system with qemu-system-ppc64 version 2.0.0, >>>>> the ppc64 snapshot ISO of 12.0, the OS appears to install correctly >>>>> and subsequently boots correctly. >>>>> >>>>> --chuck >>>> That's worth a lot, actually. >>>> >>>> The 12.0 snapshot also works on my FreeBSD 11.1 host with the stock >>>> qemu package.  Both keyboard and mouse input are processed. >>>> >>>> Interestingly, though, while 12.0 works, it seems to be a lot >>>> slower than 11.1 under qemu.  Below are times to get to the install >>>> screen.  ( I just close the qemu window as soon as it reaches that >>>> point, where 11.1 won't accept keyboard input. ) >>>> >>>> FreeBSD cray.acadix  bacon ~ 999: time qemu-ppc install >>>> freebsd-ppc.img >>>> FreeBSD-12.0-CURRENT-powerpc-powerpc64-20180709-r336134-disc1.iso >>>> + [ ! -e freebsd-ppc.img ] >>>> + qemu-system-ppc64 -cdrom >>>> FreeBSD-12.0-CURRENT-powerpc-powerpc64-20180709-r336134-disc1.iso >>>> -drive 'file=freebsd-ppc.img,format=raw' -boot d >>>> 217.327u 3.455s 4:21.41 84.4%    9628+6292k 94+2io 476pf+0w >>>> >>>> >>>> FreeBSD cray.acadix  bacon ~ 1000: time qemu-ppc install >>>> freebsd-ppc.img Save/FreeBSD-11.1-RELEASE-powerpc-powerpc64-disc1.iso >>>> + [ ! -e freebsd-ppc.img ] >>>> + qemu-system-ppc64 -cdrom >>>> Save/FreeBSD-11.1-RELEASE-powerpc-powerpc64-disc1.iso -drive >>>> 'file=freebsd-ppc.img,format=raw' -boot d >>>> 123.001u 1.748s 2:47.05 74.6%    9643+6302k 556+3io 0pf+0w >>>> >>>> Maybe these data will provide some clues to the ppc base developers... >>>> >>> I'm getting "lock order reversal" errors followed by stack traces >>> when running portsnap.  Bleeding-edge 12.0 issue? >>> >> >> Poked around at this a bit more and found a workaround.  It seems >> FreeBSD doesn't support the latest default PPC machine in qemu. >> Available options are listed below.  After switching from the default >> pseries-2.6 to pseries-2.5, FreeBSD 12.0 works flawlessly. >> >> I attached a script I'm using to install and then boot the VM. >> >> So now there's an easy way to test/fix ports for PPC64.  Runs about >> as fast as a 486, but that's fine since we can install dependencies >> via "pkg install" to reduce build time. >> >> >> FreeBSD cray.acadix  bacon ~ 1010: qemu-system-ppc -machine help >> Supported machines are: >> bamboo               bamboo >> g3beige              Heathrow based PowerMAC (default) >> mac99                Mac99 based PowerMAC >> mpc8544ds            mpc8544ds >> none                 empty machine >> ppce500              generic paravirt e500 platform >> prep                 PowerPC PREP platform >> ref405ep             ref405ep >> taihu                taihu >> virtex-ml507         Xilinx Virtex ML507 reference design >> FreeBSD cray.acadix  bacon ~ 1011: qemu-system-ppc64 -machine help >> Supported machines are: >> bamboo               bamboo >> g3beige              Heathrow based PowerMAC >> mac99                Mac99 based PowerMAC >> mpc8544ds            mpc8544ds >> none                 empty machine >> ppce500              generic paravirt e500 platform >> prep                 PowerPC PREP platform >> pseries-2.1          pSeries Logical Partition (PAPR compliant) >> pseries-2.2          pSeries Logical Partition (PAPR compliant) >> pseries-2.3          pSeries Logical Partition (PAPR compliant) >> pseries-2.4          pSeries Logical Partition (PAPR compliant) >> pseries-2.5          pSeries Logical Partition (PAPR compliant) >> pseries              pSeries Logical Partition (PAPR compliant) >> (alias of pseries-2.6) >> pseries-2.6          pSeries Logical Partition (PAPR compliant) >> (default) >> ref405ep             ref405ep >> taihu                taihu >> virtex-ml507         Xilinx Virtex ML507 reference design >> >> >> >> _______________________________________________ >> freebsd-ppc@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-ppc >> To unsubscribe, send any mail to"freebsd-ppc-unsubscribe@freebsd.org" > From owner-freebsd-ppc@freebsd.org Sat Feb 23 19:36:23 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1D66B1502D14 for ; Sat, 23 Feb 2019 19:36:23 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-20.consmr.mail.ne1.yahoo.com (sonic316-20.consmr.mail.ne1.yahoo.com [66.163.187.146]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 719EB6C733 for ; Sat, 23 Feb 2019 19:36:22 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: WYKm8TYVM1kghtl7SUs3LMjEhjJZfms4.5UquYJsHkLNt1rNn2bNUMJrl50mkaC 3WV66wCLUAHMBvCfQGHDG151dMjCyvirz16ryaZsXiuvF4Z4.sN7QrT2H51kXjRAOQJPukEwdJrC fpR8LI502ztrw3koGqnp3mjY2uSQR_iv_9qGf4kgpZyaQTB2ruAtCu5VQaH2u7NqOHiAhVwBrSII CyNMMJHxLwXL56QblNgmdtWerWE4BXLbqLFQb7iZfFqW8zb98unHJ0Mc3TaLKHwT2Kz3i3l65Vd6 E.MVJ67wvhhusxZNmfSSJDPrR3tHpN84pGg6ZcrfouBmPEgD8DamHsj3aK3jPFIFysc9K.4HRj7r Uy_RQtHecNQctWOmCQKjzfT8LvHfO.p7Fl3VVUas7C2rGwk34c5D4UYvylufi8n5PMFgAG1hGmEi qyuJxHoQdj5x0stI7pJHVMAThvqHg_MxkTA.PjA6Bnsg6dEUIEs.Y366XRknzsVQk3qJKb7o5mkY f5phDnasRtWPT47.ARfTFqTrFoi1qK54ZGzpl6LDNZbJ0dCx_5EZX9y8ZwKVF_gjoemWLri7M.y9 W53tfiyWslGPPDDnEm43smgiXfOWHX_JYcFHL.iMc1yG_Tgr20gezTMH40pj5ryVtNx8X7bJcYd. sOfmP5Ycj8ysr9q3EYrJd7rFpFhyJsOF4LY1Q14fHHi8D2gqYaemi87LLBrNathSxfowEmEuhTgt u1pjpi.abkqgrhOV9.9TTcB9nR7CLCHKAw1Bl8Bj2v2Bw6CbWX9q20zO7eye00Zg6HKHRVXQkNji msv2.Je4zyQr_aRn66DG7e_HRVIkizWq2IvYjXe56qOdm_BShm9HvsbvdvGXgwwVEofCP5GH7.Qq 1MRBGzSn_lyS23VFw3NtC_j66xDddjpCM2znDnF_Am8YjwyEg_OEuBlnIjPMeJxjkkuk_zgwjhVC 5cu4Lq1CYiwYf9I2547gyrIJKuK.h6hchceHihUpLCWOvANtjW4EbTdoP_puQa5wVloZlINgqI7_ iZeg7BBImkxLBmib9It64pnYCMFVVgtVsSpgOHOV4QJQoPzuiwBPAJA4bew-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.ne1.yahoo.com with HTTP; Sat, 23 Feb 2019 19:36:14 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.115]) ([67.170.167.181]) by smtp404.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 4f91947f7643be7adbc4944f35b3d4aa; Sat, 23 Feb 2019 19:36:12 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: An experimental hack that appears to allow old PowerMacG5 4-core (system total) system to boot reliably (head -r343884 based context) Message-Id: Date: Sat, 23 Feb 2019 11:36:11 -0800 Cc: FreeBSD PowerPC ML , Dennis Clarke To: Justin Hibbits X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: 719EB6C733 X-Spamd-Bar: + X-Spamd-Result: default: False [1.30 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_SPAM_SHORT(0.93)[0.932,0]; NEURAL_HAM_LONG(-0.78)[-0.782,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(1.13)[ip: (3.42), ipnet: 66.163.184.0/21(1.29), asn: 36646(1.03), country: US(-0.07)]; NEURAL_SPAM_MEDIUM(0.52)[0.523,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[146.187.163.66.list.dnswl.org : 127.0.5.0]; RWL_MAILSPIKE_POSSIBLE(0.00)[146.187.163.66.rep.mailspike.net : 127.0.0.17] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 Feb 2019 19:36:23 -0000 For sys/powerpc/aim/mp_cpudep.c 's cpudep_ap_bootstrap I added as shown = below: +extern void hack_into_slb_if_needed(void* vap); // HACK!!! + uintptr_t cpudep_ap_bootstrap(void) { . . . + hack_into_slb_if_needed(pcpup->pc_curpcb); // HACK!!! + sp =3D pcpup->pc_curpcb->pcb_sp; and in src/sys/powerpc/aim/slb.c I added an implementation: +void hack_into_slb_if_needed(void* vap); // HACK!!! +void hack_into_slb_if_needed(void* vap) // HACK!!! +{ // HACK!!! + struct slb *cache=3D PCPU_GET(aim.slb); + vm_offset_t va=3D (vm_offset_t)vap; + uint64_t slbv=3D kernel_va_to_slbv(va); + uint64_t esid=3D va>>ADDR_SR_SHFT; + uint64_t slbe=3D (esid<