Date: Thu, 15 Jul 2021 13:48:11 -0700 From: Mark Millard via freebsd-arm <freebsd-arm@freebsd.org> To: freebsd-arm <freebsd-arm@freebsd.org> Subject: Re: HoneyComb first-boot notes [a L3/L2/L1/RAM performance oddity: fix identified] Message-ID: <10364E05-8BCD-4A15-9D3A-CB5DD9952AED@yahoo.com> In-Reply-To: <C21C4DE5-5B1E-41FD-9268-F2E818CBCA11@yahoo.com> References: <8A6C415F-A57B-4F2F-861F-052B487166D6.ref@yahoo.com> <8A6C415F-A57B-4F2F-861F-052B487166D6@yahoo.com> <YNGT5hcHOBd6cU4T@x230.ds> <40AE6447-77AF-4D0E-864F-AD52D9F3346F@yahoo.com> <YNGf999RsaTfNhcp@x230.ds> <Rv9QGaKflpIjLPsxUFG3ht12loej__FxMBy7SQ1QzDTk1NLcFjGb4ScQuF32SakZi68wjgPQpIVp2dipMoYteJIAMhSrXbPM6-mRSeL_744=@a9development.com> <C4D3B585-63B6-4C2A-B8DA-264073C6E2C2@yahoo.com> <12A4EDD1-A2AB-4CE3-AB0E-A4B5D6FB4674@yahoo.com> <5B1B5E1A-8AE4-4889-ABE6-50C206F896FB@yahoo.com> <7DBDC8AB-C80B-4E26-B58F-251A3D29CE41@yahoo.com> <5BBF1B55-F02C-4817-B805-677EDDC5B809@yahoo.com> <0B577668-97AB-44B6-B1A7-C68F6CC299E5@yahoo.com> <C0426887-59D9-4524-8542-8DA6DBAFF744@yahoo.com> <C21C4DE5-5B1E-41FD-9268-F2E818CBCA11@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2021-Jul-11, at 18:29, Mark Millard <marklmi at yahoo.com> wrote: >>>> . . . >>>=20 >>> I've run into an issue where what FreeBSD calls cpu 0 has >>> significantly different L3/L2/L1/RAM subsystem performance >>> than all the other cores (cpu 0 being worse). Similarly for >>> compared/contrasted to all 4 MACCHIATObin Double Shot cores. >>>=20 >>> A plot with curves showing the issue is at: >>>=20 >>> = https://github.com/markmi/acpphint/blob/master/acpphint_example_data/Honey= CombFreeBSDcpu0RAMAccessPerformanceIsOdd.png >>>=20 >>> The dark red curves in the plot show the expected general >>> shape for such and are for cpu 0. The lighter colored >>> curves are the MACCHIATObin curves. The darker ones are >>> the HoneyComb curves, where the L3/L2/L1 is relatively >>> effective (other than cpu 0). >>>=20 >>> My notes on Discord (so far) are . . . >>>=20 >>> The curves are from my C++ variant of the old Hierarchical >>> INTegration benchmark (historically abbreviated HINT). You >>> can read the approximate size of a level of cache from=20 >>> the x-axis for where the curve drops faster. So, right >>> (most obvious) to left (least obvious): L3 8 MiByte, L2 1 >>> MiByte (per core pair, as it turns out), L1 32 KiByte. >>>=20 >>> The curves here are for single thread benchmark >>> configurations with cpuset used to control which CPU is >>> used. I first noticed this via odd performance variations >>> in multithreading with more cores allowed than in use (so >>> migrations to a variety of cpus over time). >>>=20 >>> I explored all the CPUs (cores), not just what I plotted. >>> Only the one gets the odd performing memory access >>> structure in its curve. >>>=20 >>> FYI: The FreeBSD boot is UEFI/ACPI based for both systems, >>> not U-Boot based. >>>=20 >>=20 >> Jon Nettleton has replicated the memory access performance >> issue on the one cpu via a different HoneyComb, running >> some Linux kernel, using tinymembench as the benchmark. >>=20 >=20 > Jon reports that for HoneyCombs older and newer, EDK2's older > and newer: All show the behavior on cpu 0. "[I]t may have > always existed." >=20 > Jon also reports that U-Boot based booting does not get the > behavior. >=20 > (I've never used U-Boot to boot the HoneyComb for any OS > media that I've got around. In my U-Boot ignorance, my > quick attempts failed for FreeBSD main and Fedora 34 > Server media that I've been using with EDK2's UEFI/ACPI.) The problem in the: lx2160a_uefi/build/arm-trusted-firmware/plat/nxp/soc-lx2160a/soc.c code has been identified and my testing of the proposed fix indicates things are working. Some very early code setting up the L1 Data prefetch configuration was depending on not-well-initialized memory and an initialization routine needed to be used a little earlier in the sequencing to avoid that. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?10364E05-8BCD-4A15-9D3A-CB5DD9952AED>