Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 6 Jul 2018 10:27:02 -0700 (PDT)
From:      "Rodney W. Grimes" <freebsd@pdx.rh.CN85.dnsmgr.net>
To:        Warner Losh <imp@bsdimp.com>
Cc:        "Rodney W. Grimes" <rgrimes@freebsd.org>, Hans Petter Selasky <hselasky@freebsd.org>, src-committers <src-committers@freebsd.org>, svn-src-all@freebsd.org, svn-src-head@freebsd.org
Subject:   Re: svn commit: r336025 - in head/sys: amd64/include i386/include
Message-ID:  <201807061727.w66HR2wY053353@pdx.rh.CN85.dnsmgr.net>
In-Reply-To: <CANCZdfr-x8LW8oD-J9B9nr6DQ17VnA2oy-cQ0hNsof0At7_NYQ@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
> On Fri, Jul 6, 2018 at 9:52 AM, Rodney W. Grimes <
> freebsd@pdx.rh.cn85.dnsmgr.net> wrote:
> 
> > > On Fri, Jul 6, 2018 at 9:32 AM, Rodney W. Grimes <
> > > freebsd@pdx.rh.cn85.dnsmgr.net> wrote:
> > >
> > > > > Author: hselasky
> > > > > Date: Fri Jul  6 10:13:42 2018
> > > > > New Revision: 336025
> > > > > URL: https://svnweb.freebsd.org/changeset/base/336025
> > > > >
> > > > > Log:
> > > > >   Make sure kernel modules built by default are portable between UP
> > and
> > > > >   SMP systems by extending defined(SMP) to include
> > defined(KLD_MODULE).
> > > > >
> > > > >   This is a regression issue after r335873 .
> > > > >
> > > > >   Discussed with:             mmacy@
> > > > >   Sponsored by:               Mellanox Technologies
> > > >
> > > > Though this fixes the issue, it also means that now when
> > > > anyone intentionally builds a UP kernel with modules
> > > > they are getting SMP support in the modules and I am
> > > > not sure they would want that.  I know I don't.
> > > >
> > >
> > >
> > > On UP systems, these additional opcodes are harmless. They take a few
> > extra
> > > cycles (since they lock an uncontested bus) and add a couple extra memory
> > > barriers (which will be NOPs). On MP systems, atomics now work by
> > default.
> > > Had we not defaulted like this, all modules built outside of a kernel
> > build
> > > env would have broken atomics. Given that (a) the overwhelming majority
> > > (99% or more) is SMP and (b) the MP code merely adds a few cycles to
> > what's
> > > already a not-too-expensive operation, this was the right choice.
> > >
> > > It simply doesn't matter for systems that are relevant to the project
> > > today. While one could try to optimize this a little (for example, by
> > > having SMP defined to be 0 or 1, say, and changing all the ifdef SMP to
> > if
> > > (defined(SMP) && SMP != 0)), it's likely not going to matter enough for
> > > anybody to make the effort. UP on x86 is simply not relevant enough to
> > > optimize for it. Even in VMs, people run SMP kernels typically even when
> > > they just allocate one CPU to the VM.
> > >
> > > So while we still support the UP config, and we'll let people build
> > > optimized kernels for x86, we've flipped the switch from pessimized for
> > SMP
> > > modules to pessimized for UP modules, which seems like quite the
> > reasonable
> > > trade-off.
> > >
> > > Were it practical to do so, I'd suggest de-orbiting UP on x86. However,
> > > it's a lot of work for not much benefit and we'd need to invent much
> > crazy
> > > to get there.
> >
> > Trivial to fix this with
> > +#if defined(SMP) || !defined(_KERNEL) || defined(KLD_MODULE) ||
> > !defined(KLD_UP_MODULES)
> 
> 
> Nope. Not so trivial. Who defines KLD_UP_MODULES?

Call it SMP_KLD_MODULES, and it gets defined the same place SMP does.

> 
> And really, it's absolutely not worth it unless someone shows up with
> numbers to show the old 'function call to optimal routine' is actually
> faster than the new 'inline to slightly unoptimal code'. Since I think the
> function call overhead is larger than the pessmizations, I'm not sure what
> the fuss is about.

I have no issues with the SMP converting from function calls to
inline locks, I just want to retain the exact same code I had
before any of these changes, and that was A UP built system
without any SMP locking.  Is it too much to ask to keep what
already worked?

> > >
> > > > > Modified:
> > > > >   head/sys/amd64/include/atomic.h
> > > > >   head/sys/i386/include/atomic.h
> > > > >
> > > > > Modified: head/sys/amd64/include/atomic.h
> > > > > ============================================================
> > > > ==================
> > > > > --- head/sys/amd64/include/atomic.h   Fri Jul  6 10:10:00 2018
> > > > (r336024)
> > > > > +++ head/sys/amd64/include/atomic.h   Fri Jul  6 10:13:42 2018
> > > > (r336025)
> > > > > @@ -132,7 +132,7 @@ void              atomic_store_rel_##TYPE(
> > volatile
> > > > u_##TYPE *p, u_
> > > > >   * For userland, always use lock prefixes so that the binaries will
> > run
> > > > >   * on both SMP and !SMP systems.
> > > > >   */
> > > > > -#if defined(SMP) || !defined(_KERNEL)
> > > > > +#if defined(SMP) || !defined(_KERNEL) || defined(KLD_MODULE)
> > > > >  #define      MPLOCKED        "lock ; "
> > > > >  #else
> > > > >  #define      MPLOCKED
> > > > > @@ -354,7 +354,7 @@ atomic_testandclear_long(volatile u_long *p,
> > u_int
> > > > v)
> > > > >   */
> > > > >  #define      OFFSETOF_MONITORBUF     0x100
> > > > >
> > > > > -#if defined(SMP)
> > > > > +#if defined(SMP) || defined(KLD_MODULE)
> > > > >  static __inline void
> > > > >  __storeload_barrier(void)
> > > > >  {
> > > > >
> > > > > Modified: head/sys/i386/include/atomic.h
> > > > > ============================================================
> > > > ==================
> > > > > --- head/sys/i386/include/atomic.h    Fri Jul  6 10:10:00 2018
> > > > (r336024)
> > > > > +++ head/sys/i386/include/atomic.h    Fri Jul  6 10:13:42 2018
> > > > (r336025)
> > > > > @@ -143,7 +143,7 @@ void              atomic_subtract_64(volatile
> > > > uint64_t *, uint64_t
> > > > >   * For userland, always use lock prefixes so that the binaries will
> > run
> > > > >   * on both SMP and !SMP systems.
> > > > >   */
> > > > > -#if defined(SMP) || !defined(_KERNEL)
> > > > > +#if defined(SMP) || !defined(_KERNEL) || defined(KLD_MODULE)
> > > > >  #define      MPLOCKED        "lock ; "
> > > > >  #else
> > > > >  #define      MPLOCKED
> > > > > @@ -302,7 +302,7 @@ atomic_testandclear_int(volatile u_int *p,
> > u_int v)
> > > > >   */
> > > > >
> > > > >  #if defined(_KERNEL)
> > > > > -#if defined(SMP)
> > > > > +#if defined(SMP) || defined(KLD_MODULE)
> > > > >  #define      __storeload_barrier()   __mbk()
> > > > >  #else /* _KERNEL && UP */
> > > > >  #define      __storeload_barrier()   __compiler_membar()
> > > > >
> > > > >
> > > >
> > > > --
> > > > Rod Grimes
> > > > rgrimes@freebsd.org
> > > >
> > > >
> >
> > --
> > Rod Grimes
> > rgrimes@freebsd.org
> >

-- 
Rod Grimes                                                 rgrimes@freebsd.org



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201807061727.w66HR2wY053353>