From owner-freebsd-arch@FreeBSD.ORG Sun Feb 24 10:39:07 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 51BB316A406; Sun, 24 Feb 2008 10:39:07 +0000 (UTC) (envelope-from jroberson@chesapeake.net) Received: from webaccess-cl.virtdom.com (webaccess-cl.virtdom.com [216.240.101.25]) by mx1.freebsd.org (Postfix) with ESMTP id F29F013C43E; Sun, 24 Feb 2008 10:39:06 +0000 (UTC) (envelope-from jroberson@chesapeake.net) Received: from [192.168.1.107] (cpe-24-94-75-93.hawaii.res.rr.com [24.94.75.93]) (authenticated bits=0) by webaccess-cl.virtdom.com (8.13.6/8.13.6) with ESMTP id m1OAd2qL086981; Sun, 24 Feb 2008 05:39:03 -0500 (EST) (envelope-from jroberson@chesapeake.net) Date: Sun, 24 Feb 2008 00:40:31 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: arch@freebsd.org In-Reply-To: <20080223213507.GD39699@lor.one-eyed-alien.net> Message-ID: <20080224001902.J920@desktop> References: <20080220105333.G44565@fledge.watson.org> <47BCEFDB.5040207@freebsd.org> <20080220175532.Q920@desktop> <20080220213253.A920@desktop> <20080221092011.J52922@fledge.watson.org> <20080222121253.N920@desktop> <20080222231245.GA28788@lor.one-eyed-alien.net> <20080222134923.M920@desktop> <20080223194047.GB38485@lor.one-eyed-alien.net> <20080223111659.K920@desktop> <20080223213507.GD39699@lor.one-eyed-alien.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Daniel Eischen , Brooks Davis , Robert Watson , David Xu , Andrew Gallatin Subject: cpuset and affinity implementation X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Feb 2008 10:39:07 -0000 Please see: http://people.freebsd.org/~jeff/cpuset.diff This is unfortunately intertwined with ULE's new CPU selection algorithm so that code is in the patch as well. Otherwise, this includes a simple, ugly userland tool called cpuset and all of the kernel support required. I have tested this by creating sets and subsets and modifying their cpu masks under load. I'm able to dynamically reprovision without issue. This doesn't have support for jails but the infrastructure is there. It also fails to modify sets if it would leave threads without a valid cpu to run on. I have not implemented a force option but it will be trivial to do so. The initial cpu set is also created before we know all_cpus so it's faked up with all cpus set for now. I mostly want people to look at the interface in cpuset.h and make sure they agree with it before I start polishing to commit. I'm fairly happy with the way the syscall api looks now. The code itself ended up being much more complicated than I'd hoped due to locking considerations. Try not to look at cpuset_setproc() ;). If you want to actually try the patch, here's a couple of neat things to do with cpuset: cpuset -l 0-4 /bin/sh This creates a new group with a list (-l) of cpus 0-4 inclusive and runs sh in it. cpuset -g -p This will get (-g) the mask of cpus pid (-p) is allowed to run on. cpuset -l 0,2 -p This will restrict sh to running on cpus 0, 2 while its group is still allowed 0-4. cpuset -l 0,2 -c -p This will modify the cpuset (-c) that the sh belongs to. cpuset -l 0-3 -s 1 This will modify the set (-s) that all threads are in by default to contain the first 4 cpus leaving the rest idled. Feedback is appreciated. Thanks, Jeff