From owner-freebsd-arch@FreeBSD.ORG Tue Feb 26 19:24:01 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3B2121065672 for ; Tue, 26 Feb 2008 19:24:01 +0000 (UTC) (envelope-from nevans@talkpoint.com) Received: from mailbox.talkpoint.com (mailbox.talkpoint.com [204.141.15.162]) by mx1.freebsd.org (Postfix) with ESMTP id D7DCA13C4F0 for ; Tue, 26 Feb 2008 19:24:00 +0000 (UTC) (envelope-from nevans@talkpoint.com) Received: from localhost (localhost [127.0.0.1]) by mailbox.talkpoint.com (Postfix) with ESMTP id 087D82C50023; Tue, 26 Feb 2008 13:54:41 -0500 (EST) X-Virus-Scanned: amavisd-new at X-Spam-Flag: NO X-Spam-Score: -3.829 X-Spam-Level: X-Spam-Status: No, score=-3.829 tagged_above=-10 required=3 tests=[ALL_TRUSTED=-1.8, AWL=0.570, BAYES_00=-2.599] Received: from mailbox.talkpoint.com ([127.0.0.1]) by localhost (mailbox.talkpoint.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SvJqz8WXHnRi; Tue, 26 Feb 2008 13:54:40 -0500 (EST) Received: from pleiades.nextvenue.com (pleiades.nextvenue.com [204.141.15.194]) by mailbox.talkpoint.com (Postfix) with ESMTP id 2B1E22C50022; Tue, 26 Feb 2008 13:54:40 -0500 (EST) Date: Tue, 26 Feb 2008 13:54:39 -0500 From: Nick Evans To: Jeff Roberson Message-ID: <20080226135439.27db7400@pleiades.nextvenue.com> In-Reply-To: <20080224001902.J920@desktop> References: <20080220105333.G44565@fledge.watson.org> <47BCEFDB.5040207@freebsd.org> <20080220175532.Q920@desktop> <20080220213253.A920@desktop> <20080221092011.J52922@fledge.watson.org> <20080222121253.N920@desktop> <20080222231245.GA28788@lor.one-eyed-alien.net> <20080222134923.M920@desktop> <20080223194047.GB38485@lor.one-eyed-alien.net> <20080223111659.K920@desktop> <20080223213507.GD39699@lor.one-eyed-alien.net> <20080224001902.J920@desktop> X-Mailer: Claws Mail 3.3.0 (GTK+ 2.12.3; i386-portbld-freebsd7.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org Subject: Re: cpuset and affinity implementation X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Feb 2008 19:24:01 -0000 On Sun, 24 Feb 2008 00:40:31 -1000 (HST) Jeff Roberson wrote: > Please see: > http://people.freebsd.org/~jeff/cpuset.diff > > This is unfortunately intertwined with ULE's new CPU selection algorithm > so that code is in the patch as well. Otherwise, this includes a simple, > ugly userland tool called cpuset and all of the kernel support required. > I have tested this by creating sets and subsets and modifying their cpu > masks under load. I'm able to dynamically reprovision without issue. > > This doesn't have support for jails but the infrastructure is there. It > also fails to modify sets if it would leave threads without a valid cpu > to run on. I have not implemented a force option but it will be trivial > to do so. The initial cpu set is also created before we know all_cpus so > it's faked up with all cpus set for now. > > I mostly want people to look at the interface in cpuset.h and make sure > they agree with it before I start polishing to commit. I'm fairly happy > with the way the syscall api looks now. The code itself ended up being > much more complicated than I'd hoped due to locking considerations. Try > not to look at cpuset_setproc() ;). > > If you want to actually try the patch, here's a couple of neat things to > do with cpuset: > > cpuset -l 0-4 /bin/sh > > This creates a new group with a list (-l) of cpus 0-4 inclusive and runs > sh in it. > > cpuset -g -p > > This will get (-g) the mask of cpus pid (-p) is allowed to run on. > > cpuset -l 0,2 -p > > This will restrict sh to running on cpus 0, 2 while its group is still > allowed 0-4. > > cpuset -l 0,2 -c -p > > This will modify the cpuset (-c) that the sh belongs to. > > cpuset -l 0-3 -s 1 > > This will modify the set (-s) that all threads are in by default to > contain the first 4 cpus leaving the rest idled. > > Feedback is appreciated. > Jeff, Is it currently, or will it eventually be possible to assign network threads to different cores? Everything appears to be driven by pid, but at least according to top all interrupt "processes" show as pid 12. Also, if kern.sched.topology returns 0 is it safe to assume I'm not getting the benefit of the topology distinction between packages vs cores? Nick