Date: Sat, 19 Mar 2016 09:08:37 +0100 From: Gary Jennejohn <gljennjohn@gmail.com> To: John Baldwin <jhb@freebsd.org> Cc: "K. Macy" <kmacy@freebsd.org>, Stanislav Sedov <stas@freebsd.org>, Adrian Chadd <adrian.chadd@gmail.com>, Konstantin Belousov <kostikbel@gmail.com>, Ryan Stone <rysto32@gmail.com>, "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org> Subject: Re: Peformance issues with r278325 Message-ID: <20160319090837.7799561c@ernst.home> In-Reply-To: <1938968.vRTG1GJntD@ralph.baldwin.cx> References: <FA50A68E-7F3D-4361-8A8A-EB7F97EF3D00@FreeBSD.org> <3213721.SRyKE0LtiG@ralph.baldwin.cx> <CAHM0Q_MaHwJhd2YWPV3V8R8KGkM=GwPh79jyEcsaro%2B_d__e9A@mail.gmail.com> <1938968.vRTG1GJntD@ralph.baldwin.cx>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, 18 Mar 2016 14:05:39 -0700
John Baldwin <jhb@freebsd.org> wrote:
> On Friday, March 18, 2016 01:11:15 PM K. Macy wrote:
> > On Friday, March 18, 2016, John Baldwin <jhb@freebsd.org> wrote:
> >
> > > On Friday, March 18, 2016 11:58:19 AM Stanislav Sedov wrote:
> > > >
> > > > > On Mar 18, 2016, at 11:49 AM, Ryan Stone <rysto32@gmail.com
> > > <javascript:;>> wrote:
> > > > >
> > > > > On Fri, Mar 18, 2016 at 1:37 PM, John Baldwin <jhb@freebsd.org
> > > <javascript:;>> wrote:
> > > > > I think I'll likely just convert it to use a direct
> > > > > TSC delay loop always in HEAD (assuming that verifies ok in testing as
> > > well).
> > > > >
> > > > > Couldn't that work incorrectly on VM guests? The tsc is not
> > > guaranteed to be monotonic in that environment.
> > > > >
> > > >
> > > > Another concern is SMP systems where the clock is not synchronized. SMP
> > > TSC requires
> > > > a complicated setup procedure on the system boot which is not followed
> > > properly by all
> > > > vendors, and I have seen some recent systems with SMP TSC skew.
> > > >
> > > > I'm afraid that using TSC in this code will make FreeBSD unusable on
> > > such (arguably buggy)
> > > > systems.
> > >
> > > Eh, SMP does not matter here. DELAY() already uses TSC on FreeBSD. The
> > > current thread is pinned to a single CPU in lapic_ipi_wait(). The idea
> > > would be to do this:
> > >
> > > deadline = rdtsc() + freq * delay / 1000000;
> > >
> > > while (rdtsc() < deadline) {
> > > if (APIC_DELSTAT_IDLE)
> > > return (1);
> > > ia32_pause();
> > > }
> > >
> > >
> > >
> > In a VM the thread is pinned to the _vcpu_. You have no control over what
> > physical core the underlying thread is running on. If the vcpu is migrated
> > to a different package there is a possibility of the next TSC reading be a
> > lower value. The rdtsc instruction reads the value of the physical core.
> > Although I doubt this is a problem in practice, it is a possibility if
> > vcpus are not pinned for the lifetime of the VM.
>
> Again, DELAY() already makes this assumption on FreeBSD/x86. If this is
> broken, it's broken in far, far many more places than this. In addition, a
> hypervisor is capable of handling this if it uses the TSC adjustment field
> in the VMCS. OTOH, TSC's on modern systems generally are in sync. Invariant
> TSCs are generated from a timer in the uncore on all the i[357] CPUs AFAIK, so
> they are identical across cores and threads within a package. They are also
> in sync across packages on all of the i[357] systems I have worked with (and
> generations prior to those) in the last 7-8 years (barring one machine I'm
> aware of that was a quad-socket i7 box where the BIOS overwrote the TSC of
> the first core in each package on each reboot to try to reset it to zero on
> each boot).
>
Just out of curiosity, can one assume that this is also true for AMD
CPUs?
I'm using a Phenom II X6 1090T in my box.
--
Gary Jennejohn
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20160319090837.7799561c>