From owner-freebsd-arch@FreeBSD.ORG Fri Mar 20 12:38:33 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 53B1C94D; Fri, 20 Mar 2015 12:38:33 +0000 (UTC) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0A9EE67E; Fri, 20 Mar 2015 12:38:33 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1YYwBz-000DRo-PE; Fri, 20 Mar 2015 15:38:23 +0300 Date: Fri, 20 Mar 2015 15:38:23 +0300 From: Slawa Olhovchenkov To: John Baldwin Subject: Re: RFC: Simplfying hyperthreading distinctions Message-ID: <20150320123823.GA49621@zxy.spb.ru> References: <1640664.8z9mx3EOQs@ralph.baldwin.cx> <54FA1180.3080605@astrodoggroup.com> <1526311.uylCbgv5VB@ralph.baldwin.cx> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1526311.uylCbgv5VB@ralph.baldwin.cx> User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: Harrison Grundy , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Mar 2015 12:38:33 -0000 On Fri, Mar 06, 2015 at 04:17:37PM -0500, John Baldwin wrote: > On Friday, March 06, 2015 12:43:44 PM Harrison Grundy wrote: > > On 03/06/15 12:44, John Baldwin wrote: > > > Currently we go out of our way a bit to distinguish Pentium4-era > > > hyperthreading from more recent ("modern") hyperthreading. I > > > suspect that this distinction probably results in confusion more > > > than anything else. Intel's documentation does not make near as > > > broad a distinction as far as I can tell. Both types of SMT are > > > called hyperthreading in the SDM for example. However, we have the > > > astonishing behavior that 'machdep.hyperthreading_allowed' only > > > affects "old" hyperthreads, but not "new" ones. We also try to be > > > overly cute in our dmesg output by using HTT for "old" > > > hyperthreading, and SMT for "new" hyperthreading. I propose the > > > following changes to simplify things a bit: > > > > > > 1) Call both "old" and "new" hyperthreading HTT in dmesg. > > > > > > 2) Change machdep.hyperthreading_allowed to apply to both new and > > > old HTT. However, doing this means a POLA violation in that we > > > would now disable modern HTT by default. Balanced against > > > re-enabling "old" HTT by default on an increasingly-shrinking pool > > > of old hardware, I think the better approach here would be to also > > > change the default to allow HTT. > > > > > > 3) Possibly add a different knob (or change the behavior of > > > machdep.hyperthreading_allowed) to still bring up hyperthreads, but > > > leave them out of the default cpuset (set 1). This would allow > > > those threads to be re-enabled dynamically at runtime by adjusting > > > the mask on set 1. The original htt settings back when > > > 'hyperthreading_allowed' was introduced actually permitted this via > > > by adjusting 'machdep.hlt_cpus' at runtime. > > > > > > What do people think? > > > > I'm not sure of how interrupt handling works as it relates to HTT, but > > wouldn't using cpuset potentially leave them active for interrupt > > handling? > > > > Other than that question, this all makes sense to me. > > Interrupt handling works differently. Per my commit a few minutes ago, we do > not send interrupts to hyperthreads by default (either old or new). However, > ithreads that are not explicitly bound to a specific CPU will "float" among > all the CPUs in set 1 so 3) would affect that. Eventually I want to use a > separate cpuset for interrupts that ithreads inherit from (rather than > belonging to set 1). Can you explain interrupt handling some more? How routing real interrupt? Can be real interrupt routing to specific core? Is real interrupt and `cpuset -x irq` is same (I see interrup from chelsio on cpu other then pinned)?