Date: Fri, 5 Apr 2019 14:35:04 +0300 From: Konstantin Belousov <kostikbel@gmail.com> To: Bruce Evans <brde@optusnet.com.au> Cc: Mark Millard <marklmi@yahoo.com>, freebsd-hackers Hackers <freebsd-hackers@freebsd.org>, FreeBSD PowerPC ML <freebsd-ppc@freebsd.org> Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] Message-ID: <20190405113504.GA1923@kib.kiev.ua> In-Reply-To: <20190405201055.I2396@besplex.bde.org> References: <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> <E0785613-2B6E-4BB3-95CD-03DD96902CD8@fh-muenster.de> <20190403070045.GW1923@kib.kiev.ua> <20190404011802.E2390@besplex.bde.org> <F22CCA2C-08BB-452E-B00C-A36CD4611540@yahoo.com> <20190405150236.A959@besplex.bde.org> <F3BBD355-5198-41A0-A461-8E5E12984BE7@yahoo.com> <20190405201055.I2396@besplex.bde.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Apr 05, 2019 at 09:13:50PM +1100, Bruce Evans wrote: > On Fri, 5 Apr 2019, Mark Millard wrote: > > > On 2019-Apr-4, at 21:38, Bruce Evans <brde at optusnet.com.au> wrote: > > > >> On Thu, 4 Apr 2019, Mark Millard wrote: > >>> ... > >>> Unfortunately, all the multi-socket contexts that I sometimes have > >>> access to are old PowerMacs. And, currently, the only such context > >>> is the G5 with 2 sockets, 2 cores per socket (powerpc64). So I've > >>> not been able to set up other types of examples to see if problems > >>> repeat. > >>> > >>> I do not have access to a single-socket powerpc64 for contrast in > >>> that direction. > >> > >> Testing 1 socket is time-consuming enough. Do these old systems > >> use the equivalent of an x86 TSC for the timecounter? With multiple > >> sockets, it isn't clear how even a hardware timer independent of the > >> CPUs can be distributed so as to appear to be monotonic on all cors. > > > > "The DEC frequency is based on the same implementation-dependent > > frequency that drives the time base." The frequency may well be > > fixed by the PowerMac G5 model implementation but is not fixed > > by the powerpc64 architecture. > > > > The Time Base Register (TBR) in a powerpc64 core (cpu in FreeBSD terms) > > increments at 33,333,333 Hz (nominal) and is 64 bits wide. Its value > > can be set (mttb instruction) and the boot sequence in FreeBSD does > > attempt to adjust as the FreeBSD CPU is brought-up/started. mftb is > > used to read the 64-bit value. FreeBSD masks it down to 32-bits to > > contribute to the time-counter. > > > > (Is that description sufficient for what you were after? I've never > > seen documentation of how the 33,333,333 MHz is produced.) > > This seems to be equivalent to an x86 TSC. It is equivalent from the interface PoV. I saw some references that Powers do have per-core PLLs, the best I can find now is https://stackoverflow.com/questions/43844438/are-multiple-clock-domains-common-in-modern-processors For Intel there is no much information, the best guess is that TSC is in uncore and effectively shared by all cores. See also https://software.intel.com/en-us/articles/best-timing-function-for-measuring-ipp-api-timing/ > > x86 P-state-invariant TSCs apparently use a 100 MHz clock which corresponds > even more closely to this, except for historical reasons this clock is > scaled and interpolated to a clock resembling the CPU cycle count at a > nominal frequency. For Intel designs, there is indeed a single-source 100MHz signal which is distributed over all consumers using clock fan-out buffers like DB1900Z. > > > As FreeBSD supports multi-socket, what are its criteria for a sufficient > > context for it to work with for supporting sbinuptime and the like? Is > > FreeBSD supposed to then make it appear that sbinputime and the like are > > weakly increasing, even as threads migrate between CPUs (cores, > > hw-threads)? > > CLOCK_MONOTONIC has to be monotonic, but it is only clear what this means > within a single thread: sequential clock_gettime() calls must occur in > program order and the results must be consistent with that order. Across > threads, I think it should mean that the results must be consistent with > any order established using any supported ordering methods. > > >... > >>> One oddity is that the eventtimer's decrementer and timecounter > >>> may change (nearly) together: both change at 33,333,333 Hz, as if > >>> they are tied to the same clock (at least on one socket). > >> > >> I think this is from a normal hardware implementation. On all of > >> my x86 systems with a TSC, the TSC frequency is an exact fractional > >> multiple of the i8254, the ACPI timer (if present) and the HPET (if > >> present). Only the RTC has an independent frequency. The fraction is > >> changed by changing the nominal TSC frequency in the BIOS, but is not > >> changed by temperature variations. This must be because most clocks are > >> derived from a common clock using a PLL. I use this to calibrate all > >> clocks (except the RTC) by calibrating only 1. > > > > I'm not aware of the OpenFirmware having any control over the > > TBR-change frequency behavior. I've no evidence about any variability > > based on temperature. > > Temperature changes usually affect the actual frequency but not the > nominal frequency. > > Bruce
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20190405113504.GA1923>