Date: Sat, 4 May 2019 18:24:18 -0700 From: Mark Millard <marklmi@yahoo.com> To: Justin Hibbits <chmeeedalf@gmail.com>, FreeBSD PowerPC ML <freebsd-ppc@freebsd.org> Subject: Examples of early boot variability in handle_kernel_slb_spill happening on 2-socket/2-core-each G5 PowerMac11,2 Message-ID: <331C36C9-3884-402B-BB1A-1E2C9A2BB511@yahoo.com> References: <331C36C9-3884-402B-BB1A-1E2C9A2BB511.ref@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
I've switched to keeping counts of handle_kernel_slb_spill
usage for my experiments with the slb-miss behavior:
#if defined(__powerpc64__) && defined(AIM)
unsigned int at_or_after_KVA_START_slb_spill_srr0_count=3D 0;
unsigned int from_DMAP_START_upto_KVA_START_slb_spill_srr0_count=3D 0;
unsigned int below_DMAP_START_slb_spill_srr0_count=3D 0;
unsigned int at_or_after_KVA_START_slb_spill_dar_count=3D 0;
unsigned int from_DMAP_START_upto_KVA_START_slb_spill_dar_count=3D 0;
unsigned int below_DMAP_START_slb_spill_dar_count=3D 0;
/* Handle kernel SLB faults -- runs in real mode, all seat belts off */
void
handle_kernel_slb_spill(int type, register_t dar, register_t srr0)
{
struct slb *slbcache;
uint64_t slbe, slbv;
uint64_t esid, addr;
int i;
if (EXC_ISE=3D=3Dtype) {
if (0xe000000000000000u<=3Dsrr0)
at_or_after_KVA_START_slb_spill_srr0_count++;
else if (0xc000000000000000u<=3Dsrr0)
from_DMAP_START_upto_KVA_START_slb_spill_srr0_count++;
else
below_DMAP_START_slb_spill_srr0_count++;
} else {
if (0xe000000000000000u<=3Ddar)
at_or_after_KVA_START_slb_spill_dar_count++;
else if (0xc000000000000000u<=3Ddar)
from_DMAP_START_upto_KVA_START_slb_spill_dar_count++;
else
below_DMAP_START_slb_spill_dar_count++;
}
. . .
That is only designed to deal with the "only bsp" time frame.
The examples below are for my use of prefaulting
about half of the 64-1 kernel entries:
i =3D 0;
for (va =3D virtual_avail; va < virtual_end && i<(n_slbs-1)/2; =
va +=3D SEGMENT_LENGTH, i++)
moea64_bootstrap_slb_prefault(va, 0);
(Some of the numbers checked are from before this loop
as well.)
The boots were a mix of:
The original power-on
shutdown -r now
shutdown -p now then power-on
The summary of the pictures of the samplings
of the counts goes like . . .
The moea64_bootstrap_slb_prefault loop had examples of:
(counts from just-before vs. just-after checked
for increases to non-zero)
A) No handle_kernel_slb_spill use in the boot.
B) dar<DMAP_START happening once
C) srr0<DMAP_START happening once (distinct boot)
bs_remap_earlyboot had examples of:
(counts from just-before vs. just-after checked
for increases)
A) No handle_kernel_slb_spill use in the boot.
B) DMAP_START<=3Ddar<KVA_START happening twice
C) DMAP_START<=3Ddar<KVA_START happening once
dpcpu_init had examples of:
(counts from just-before vs. just-after checked
for increases)
A) No handle_kernel_slb_spill use in the boot.
B) KVA_START<=3Ddar happening once
No other before-vs-after counts examined showed
any changes in the counts: just the 3 stages. The
last sampling was just-after the
numa_mem_regions(...) call.
There were fairly small number of boots tried.
For a few I failed to get a picture soon
enough and so could not see at watch stage
a count had gone to non-zero.
While there may be a bias, both -r reboots and
power-ons varied in their behavior for the counts.
This also make clear the following about the live
slb content on such a G5 (relative to
handle_kernel_slb_spill):
0) There can be examples from/of dar: dar<DMAP_START
1) There can be examples from/of srr0: srr0<DMAP_START
2) There can be examples from/of dar: DMA_START<=3Ddar<KVA_START
3) There can be examples from/of dar: KVA_START<=3Ddar
(beyond what I prefaulted)
But none of those 4 is guaranteed to happen.
Hopefully none of (0-2) can lead to problems. (3)
is at least similar to normal behavior for the KVA.
=3D=3D=3D
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?331C36C9-3884-402B-BB1A-1E2C9A2BB511>