Date: Mon, 22 Apr 2019 10:48:26 -0500 From: Justin Hibbits <chmeeedalf@gmail.com> To: Mark Millard <marklmi@yahoo.com> Cc: FreeBSD PowerPC ML <freebsd-ppc@freebsd.org> Subject: Re: spinlock_enter, head -r346144 (and before) and use of nop_prio_mhigh vs. PowerISA document suggestions for lock code Message-ID: <20190422104826.1bcdbaa6@titan.knownspace> In-Reply-To: <300DEA60-E9C3-4BB7-A7EF-980EE58257DB@yahoo.com> References: <F0A58D08-4A99-4483-A419-200A1485E73E@yahoo.com> <300DEA60-E9C3-4BB7-A7EF-980EE58257DB@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, 19 Apr 2019 17:22:24 -0700
Mark Millard <marklmi@yahoo.com> wrote:
> [Looks like nop_prio_mhigh has some vintage-specific
> behavior, based on if there is a PSPB (Problem State
> Priority Boost Register) and how it is configured.]
As mentioned in the references, these nops are hints only. They're
completely ignored on older hardware, so the existence of a PSPBR is
inconsequential.
>
> On 2019-Apr-19, at 15:56, Mark Millard <marklmi at yahoo.com> wrote:
>
> > I found the following text in each of the 2.03, 2.04, 2.05,
> > 2.06B V2, 2.07, and 3.0B PowerISA documents, in a "Programming
> > Note" in the "Program Priority Registers" section:
> >
> > ". . . if a program is waiting on a lock (...), it could set low
> > priority with the result that more processor resources would be
> > diverted to the program that holds the lock. This diversion of
> > resources may enable the lock-holding program to complete the
> > operation under the lock more quickly, and then relinquish the lock
> > to the waiting program."
> >
> > The wording suggests working via normal/medium and lower
> > priorities instead of via normal/medium and higher priorities.
> > (May be more than just the relative priority matters in the
> > behavior changes that result each way? Unfortunately the
> > wording is not very explicit.)
But it's not waiting on a lock here, it's holding the lock. The idea
is to hold the lock at high priority so it gets more cycles to complete
its work, then drops back to normal priority as soon as possible.
There is work remaining to drop to very low priority during the
spinwait phase of mutexes, which will be done soon.
> >
> > All of the documents list "or rx,rx,rx" for:
> > Rx being 31 or 1 or 6 or 2 or 5 or 3 or 7
> > (listed lowest to highest relative priority),
> > 2 being normal/medium. Some table(s) might not
> > list 3 or 7 in a document but at least one does
> > in each.
>
> Actually, going back to 2.03, for example, only lists 31
> in one place as well. Only 1, 6, and 2 are in all
> such tables.
>
> > In the following powerpc64 and 32-bit powerpc
> > FreeBSD seem to be going in the opposite direction
> > relative to normal/medium vs. the suggestion
> > of "low priority":
These nops are only relevant on hardware that only properly supports
64-bit kernels.
> >
> > void
> > spinlock_enter(void)
> > {
> > struct thread *td;
> > register_t msr;
> >
> > td = curthread;
> > if (td->td_md.md_spinlock_count == 0) {
> > nop_prio_mhigh();
> > msr = intr_disable();
> > td->td_md.md_spinlock_count = 1;
> > td->td_md.md_saved_msr = msr;
> > } else
> > td->td_md.md_spinlock_count++;
> > critical_enter();
> > }
> >
> > void
> > spinlock_exit(void)
> > {
> > struct thread *td;
> > register_t msr;
> >
> > td = curthread;
> > critical_exit();
> > msr = td->td_md.md_saved_msr;
> > td->td_md.md_spinlock_count--;
> > if (td->td_md.md_spinlock_count == 0) {
> > intr_restore(msr);
> > nop_prio_medium();
> > }
> > }
> >
> > and previously:
> >
> > void
> > spinlock_enter(void)
> > {
> > struct thread *td;
> > register_t msr;
> >
> > td = curthread;
> > if (td->td_md.md_spinlock_count == 0) {
> > __asm __volatile("or 2,2,2"); /* Set high thread
> > priority */ msr = intr_disable();
> > td->td_md.md_spinlock_count = 1;
> > td->td_md.md_saved_msr = msr;
> > } else
> > td->td_md.md_spinlock_count++;
> > critical_enter();
> > }
> >
> > void
> > spinlock_exit(void)
> > {
> > struct thread *td;
> > register_t msr;
> >
> > td = curthread;
> > critical_exit();
> > msr = td->td_md.md_saved_msr;
> > td->td_md.md_spinlock_count--;
> > if (td->td_md.md_spinlock_count == 0) {
> > intr_restore(msr);
> > __asm __volatile("or 6,6,6"); /* Set normal thread
> > priority */ }
> > }
> >
> > (2,2,2 was higher then 6,6,6 but 2,2,2 is
> > normal/medium and 6,6,6 is "medium low" the
> > way the PowerISA documentation reads.)
> >
> > 2.06B V2 and 2.07 also list special meanings for:
> > 27 and 29 and 30. (cpu_spinwait in FreeBSD uses 27.)
> > But 3.0B does not list them any more.
> >
> > 2.07 and 3.0B lists a special meaning for: 26.
> > No prior version that I looked at does.
> >
>
> PSPB (Problem State Priority Boost Register)is not
> mentioned until 2.07 of the PowerISA (for those I've
> been looking at).
>
> PSPB can count down and force Medium High back
> to Medium, for example. A hardware form of
> timed-temporary priority boost when used that way.
>
> It appears that medium and lower priorities do
> not have such a means of control. Nor do
> higher priorities, just Medium High.
Medium-high is the highest non-privileged can go, no higher. You can
see that we make no reference to the priority boost register because it
doesn't matter. If it exists, userspace might get a slight boost for a
few milliseconds, but if it doesn't, or is configured to not permit it,
then the mhigh nop is a complete nop. The nop_prio_*() inlines are
intended for kernel only, but aren't held to it because it's not
strictly necessary.
- Justin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20190422104826.1bcdbaa6>