Date: Wed, 20 Feb 2008 11:10:01 +0000 (GMT) From: Robert Watson <rwatson@FreeBSD.org> To: Jeff Roberson <jroberson@chesapeake.net> Cc: Daniel Eischen <deischen@FreeBSD.org>, arch@FreeBSD.org, Andrew Gallatin <gallatin@cs.duke.edu> Subject: Re: Linux compatible setaffinity. Message-ID: <20080220105333.G44565@fledge.watson.org> In-Reply-To: <20080220005030.Y920@desktop> References: <20071219211025.T899@desktop> <18311.49715.457070.397815@grasshopper.cs.duke.edu> <20080112182948.F36731@fledge.watson.org> <20080112170831.A957@desktop> <Pine.GSO.4.64.0801122240510.15683@sea.ntplx.net> <20080112194521.I957@desktop> <20080219234101.D920@desktop> <20080220101348.D44565@fledge.watson.org> <20080220005030.Y920@desktop>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, 20 Feb 2008, Jeff Roberson wrote: >> So perhaps this means a slightly more complex API, but not much more >> complex. How about: >> >> int cpuaffinity_get(scope, id, length, mask) >> int cpuaffinity_getmax(scope, id, length, mask) >> int cpuaffinity_set(scope, id, length, mask) >> int cpuaffinity_setmax(scope, id, length, mask) >> >> Scope would be something on the order of process (representing individual >> processes or process groups, potentially), id would be the id in that scope >> namespace, length and mask would be as you propose. You could imagine >> adding a further field to indicate whether it's the current affinity or the >> maximum affinity, but I'm not sure the details matter all that much. Here >> might be some application logic, though: > > Well I'm not sure about the max. How about just a cpuaffinity_get with a > scope that specifies what cpus are available to you? If the set is > restricted by a jail or some other mechanism it would be returned in avail. > Otherwise all cpus would be returned. The thread probably wouldn't directly > mainpulate its max, rather it would be set by changing the jail or cpu group > it belonged to. I think the actual details don't matter too much as long as we can express what we need to, so I'm ok with a special scope for that. You do raise an interesting point, though, on the nature of scope: presumably we'd like to be able to query a few different "maximums" in the interest of having a maximally debuggable system -- be it the hardware limit, the administrative limit, etc. So perhaps the scopes are something more like: #define CPUAFF_SCOPE_HARDWARE 1 /* Hardware limits */ #define CPUAFF_SCOPE_SYSTEM 2 /* System usage mask */ #define CPUAFF_SCOPE_PROCESS 3 /* Processes and process groups */ #define CPUAFF_SCOPE_THREAD 4 /* Threads in process */ CPUAFF_SCOPE_HARDWARE would be what the kernel has probed at the physical layer. This would be get-only, and available simply to provide a consistent interface when dealing with CPU masks. CPUAFF_SCOPE_SYSTEM would be what are exposed by the kernel for use by the application. This would be get-only without privilege, and presumably not exceed CPUAFF_SCOPE_HARDWARE. This would be the practical upper bound on what a process could use, and would likely be how we implement Jail scoping of CPUs. This might or might not be how one tries to discourage userspace use of a particular CPU. CPUAFF_SCOPE_PROCESS would be the process affinity for the process as a whole. It would be get-set without privilege, but limited to CPUAFF_SCOPE_SYSTEM. I'm not sure we want that limitation to be something overridden with privilege, but I guess we can think about that. CPUAFF_SCOPE_THREAD would be the thread affinity for an individual thread. It would be get-set without privilege. Do we limit it to CPUAFF_SCOPE_SYSTEM or CPUAFF_SCOPE_PROCESS? Since this involves some access control now, a pondering on access control: When a process sets the affinity of another thread or process, is it limited to the system scope of the current process, or of the target process? This may be a practical question if we're talking about how to deal with a process outside of a jail setting the affinity of a process inside jail. It's easy to implement the right thing in the kernel, but does that then imply that the system scope query should take a process ID so that a user tool can figure out what valid choices are? :-) I think it's useful to play out a few of these scenarios a bit and see what works; we won't be locked into any particular model until it hits a -STABLE branch, but getting the underlying primitive right will make it a lot easier to implement some of the more mature services we have in mind, making them more likely to happen. Robert N M Watson Computer Laboratory University of Cambridge
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20080220105333.G44565>