Date: Mon, 5 Aug 2002 23:05:56 -0700 From: Luigi Rizzo <rizzo@icir.org> To: Peter Wemm <peter@wemm.org> Cc: Terry Lambert <tlambert2@mindspring.com>, smp@freebsd.org Subject: Re: how to create per-cpu variables in SMP kernels ? Message-ID: <20020805230556.C26751@iguana.icir.org> In-Reply-To: <20020805221415.B0C732A7D6@canning.wemm.org>; from peter@wemm.org on Mon, Aug 05, 2002 at 03:14:15PM -0700 References: <20020805015340.A17716@iguana.icir.org> <20020805221415.B0C732A7D6@canning.wemm.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Peter, thanks for the explaination. I still have a few doubts on this (let's restrict to the -current case where the code seems more readable): --- MINOR DETAIL --- * I wonder why the macro __PCPU_GET() in sys/i386/include/pcpu.h cannot store directly into __result for operand sizes of 1,2,4 instead of going through a temporary variable. I.e. what would be wrong in having #define __PCPU_GET(name) ({ \ __pcpu_type(name) __result; \ \ if (sizeof(__result) == 1) { \ __asm __volatile("movb %%fs:%1,%0" \ : "=r" (__result) \ : "m" (*(u_char *)(__pcpu_offset(name)))); \ } else if (sizeof(__result) == 2) { \ Probably the same holds for __PCPU_SET(). --- OVERALL IMPLEMENTATION OF THE PER-CPU DATA --- Partly following Terry's description, i thought an arrangement like the following could be relatively simple to implement and not require any recourse to assembly code, does not impact the compiler's ability to do optimizations, and does not require an extra segment descriptor to access the struct pcpu. It relies on the following variables, my_pcpu to access the pcpu data of the local processor, all_pcpu to view all pcpu data (including our own, at a different mapping in vm space): struct pcpu *my_pcpu; struct pcpu *all_pcpu[MAXCPU]; /* XXX volatile */ Early in the boot process we allocate MAXCPU physical pages, and MAXCPU+1 entries in the VM space. Individual pcpu structs go at the beginning of each of the physical pages, and the VM -> physical mapping of the first MAXCPU VM entries is the same for all processors. Then all_pcpu[i] can be initialized with a pointer to the beginning of the i-th VM page. The MAXCPU+1-th VM entry maps differently on each CPU, so that it effectively permits access to the per-cpu data. my_pcpu can be initialized with a pointer to the MAXCPU+1-th VM page. At this point, curproc and all other per-cpu variables for the local CPU can be accessed through my_pcpu->curproc and similar, whereas we can get to other cpu's data with all_pcpu[i]->curproc without the need for using %fs or special assembly language to access these fields. Then we can discuss how/where to put "volatile" keywords. In principle, all references through all_pcpu[] should be readonly and treated as volatile, with perhaps the exception of some section of code at machine startup. On the contrary we could safely assume that references through my_pcpu are non-volatile as the local processor should be the only one to mess with them Anything wrong with this description ? cheers luigi On Mon, Aug 05, 2002 at 03:14:15PM -0700, Peter Wemm wrote: ... > Sort-of. There is both a compile time issue and a runtime issue. > > Using the %fs:variable segment overrides doesn't make a lot of difference, > but the compiler is effectively wired so that they are treated as volatile. ... To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020805230556.C26751>