From owner-freebsd-smp Mon Oct 23 9:32:48 2000 Delivered-To: freebsd-smp@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id A5A1337B479 for ; Mon, 23 Oct 2000 09:32:45 -0700 (PDT) Received: from zeppo.feral.com (IDENT:mjacob@zeppo [192.67.166.71]) by feral.com (8.9.3/8.9.3) with ESMTP id JAA04619; Mon, 23 Oct 2000 09:29:33 -0700 Date: Mon, 23 Oct 2000 09:26:42 -0700 (PDT) From: Matthew Jacob Reply-To: mjacob@feral.com To: Poul-Henning Kamp Cc: freebsd-smp@freebsd.org Subject: Re: cvs commit: src/sys/sys mutex.h In-Reply-To: <13301.972324795@critter> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org [ taken to smp... ] > This was the warnings I talked about, hoping they would remind > the SMPng team about their brokenness... > > On i386 those were exactly warnings, they did not break the build. Broke it on alpha :-). It might actually break on an SMP kernel build for i386 too. Yeah- the deal here is that for each file that includes this, it whines. Ugly, but your point gets across. What I've done so far is to push this out to the file- and to move CURTHD out there too because that's where it'll need to be anyway. I'm test compiling the changes now so I can put back a new mutex.h that gives you what you want but doesn't pork the alpha kernel compiles. The ultimate problem is partially that proc.h is included in the wrong place throughout most of the kernel- it's included *after* buf/bio, but any SMP aware Unix system is likely going to need something either from proc.h, or some glue layer (call to assembler, most likely) that will yield the current thread ID, and places like buffer structures, etc., always have locks defined. Is there some reason that proc.h is included later? I mean, the right way to fix this whine breakage seems to me to go thru and put proc.h in the right place so that curproc and their ilk can be defined before being used. Formally speaking, I'd prefer to *not* have CURTHD just a static propagated define in mutex.h- but I suspect that a good performance argument could be made that for non-SMP machines that curproc is what this would resolve to anyway, and why pay even a trivial call/return cost to get it. Thoughts? -matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 23 9:45:51 2000 Delivered-To: freebsd-smp@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 84C8437B479 for ; Mon, 23 Oct 2000 09:45:48 -0700 (PDT) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.9.3) with ESMTP id e9NIfZU13425; Mon, 23 Oct 2000 20:41:35 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: mjacob@feral.com Cc: freebsd-smp@freebsd.org Subject: Re: cvs commit: src/sys/sys mutex.h In-Reply-To: Your message of "Mon, 23 Oct 2000 09:26:42 PDT." Date: Mon, 23 Oct 2000 20:41:35 +0200 Message-ID: <13423.972326495@critter> From: Poul-Henning Kamp Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message , Matthew J acob writes: >[ taken to smp... ] > >> This was the warnings I talked about, hoping they would remind >> the SMPng team about their brokenness... >> >> On i386 those were exactly warnings, they did not break the build. > >Broke it on alpha :-). It might actually break on an SMP kernel build for i386 >too. It didn't, I tested that. I don't have an alpha I can test on here, so my automatic scripts didn't cover that. >Yeah- the deal here is that for each file that includes this, it whines. Ugly, >but your point gets across. :-) >What I've done so far is to push this out to the file- >and to move CURTHD out there too because that's where it'll need to be anyway. >I'm test compiling the changes now so I can put back a new mutex.h that gives >you what you want but doesn't pork the alpha kernel compiles. > >The ultimate problem is partially that proc.h is included in the wrong place >throughout most of the kernel- it's included *after* buf/bio, but any SMP >aware Unix system is likely going to need something either from proc.h, or >some glue layer (call to assembler, most likely) that will yield the current >thread ID, and places like buffer structures, etc., always have locks defined. Well, my take on this is that the main problem is that curproc is two entirely different things for SMP vs UP. That clearly makes all KLD's toast unless compiled for the right thing. That needs fixed. In general "#ifdef SMP" should be minimized and in particular no KLD's should need to know about it. I think a function to get the current thread/proc should not require to be included, most places just need to pass a reference, they don't need to even know how the entrails of a proccess is arranged, so obviously they should not need #included. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 23 10: 0:15 2000 Delivered-To: freebsd-smp@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id B9CF137B4C5 for ; Mon, 23 Oct 2000 10:00:12 -0700 (PDT) Received: from zeppo.feral.com (IDENT:mjacob@zeppo [192.67.166.71]) by feral.com (8.9.3/8.9.3) with ESMTP id JAA04745; Mon, 23 Oct 2000 09:59:52 -0700 Date: Mon, 23 Oct 2000 09:57:01 -0700 (PDT) From: Matthew Jacob Reply-To: mjacob@feral.com To: Poul-Henning Kamp Cc: freebsd-smp@freebsd.org Subject: Re: cvs commit: src/sys/sys mutex.h In-Reply-To: <13423.972326495@critter> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > Well, my take on this is that the main problem is that curproc is two > entirely different things for SMP vs UP. That clearly makes all KLD's > toast unless compiled for the right thing. That needs fixed. > > In general "#ifdef SMP" should be minimized and in particular no > KLD's should need to know about it. > > I think a function to get the current thread/proc should not require > to be included, most places just need to pass a reference, > they don't need to even know how the entrails of a proccess is > arranged, so obviously they should not need #included. From solaris: extern kthread_id_t threadp(void); /* inline, returns thread pointer */ #define curthread (threadp()) /* current thread pointer */ #define curproc (ttoproc(curthread)) /* current process */ Strictly speaking, not an inline- but instead a per-platform locore.s function- on sun4u it's: retl mov THREAD_REG, %o0 On i86 it's: movl %gs:CPU_THREAD, %eax ret This approach solves the KLD issue. But we have to get everyone to agree that even UP kernels have to pay the cost of even trivial function reference instead of linktime binding. Anyone else have some comments? -matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 23 10: 6:41 2000 Delivered-To: freebsd-smp@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id 2E4B737B479 for ; Mon, 23 Oct 2000 10:06:38 -0700 (PDT) Received: from zeppo.feral.com (IDENT:mjacob@zeppo [192.67.166.71]) by feral.com (8.9.3/8.9.3) with ESMTP id KAA04809; Mon, 23 Oct 2000 10:06:30 -0700 Date: Mon, 23 Oct 2000 10:03:38 -0700 (PDT) From: Matthew Jacob Reply-To: mjacob@feral.com To: Poul-Henning Kamp Cc: freebsd-smp@freebsd.org Subject: followup... In-Reply-To: <13423.972326495@critter> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > It didn't, I tested that. I don't have an alpha I can test on here, > so my automatic scripts didn't cover that. [ canned reminder ] There's supposed to be a machine, beast.cdrom.com, that exists just for this purpose (compilees). And if that isn't available, feel free to ask me to give you a slogin on one of my alphas. -matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 23 11:22:29 2000 Delivered-To: freebsd-smp@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 77A5B37B479 for ; Mon, 23 Oct 2000 11:22:26 -0700 (PDT) Received: (qmail 46977 invoked by uid 1142); 23 Oct 2000 18:22:25 -0000 Date: 23 Oct 2000 11:22:25 -0700 Date: Mon, 23 Oct 2000 11:22:07 -0700 From: Jason Evans To: smp@freebsd.org Subject: SMP project status Message-ID: <20001023112207.D3993@canonware.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Here's a reminder that the SMP project is being tracked at: http://people.freebsd.org/~jasone/smp/ Significant progress continues to be made. Many of the tasks that are listed as "in progress" are essentially complete, pending testing and review. In general, the SMP project has been going very smoothly, and -current has been usable a larger percentage of the time than I had hoped for. Note that there are a number of unassigned tasks right now. Some of the tasks are pretty straightforward, so if you're looking for an opportunity to help, please take a look at the list. A couple of weeks ago, John Baldwin and Chuck Paterson created a list of rules for kernel synchronization. If you haven't looked at the rules yet, take a look at: http://people.freebsd.org/~jasone/smp/smp_synch_rules.html Many of the developers currently involved in the SMP project were at BSDcon last week. We discussed a lot of technical issues surrounding the tasks that are currently in progress. We also discussed modes of communication. Much of the SMP-related communication has taken place via IRC, phone, or in person meetings. In my opinion, IRC and voice are much more efficient than email for many things, but I am increasingly concerned about the lack of a permanent record for many of the conversations we've had up to now. In some cases, I've tried to go back and figure out why we made a certain decision, only to realize that there is no record of how the decision was reached. Therefore, I ask that everyone make a concerted effort to document issues and reasoning for decisions on the -smp mailing list. This also means holding technical discussions on the mailing list whenever practical, and when not practical, summarizing on the mailing list. SMP-related work is going to continue for several years in one form or another, and there are probably people who are not even involved with FreeBSD yet that will be doing some of the work, so we need to leave a development documentation trail for those people, as well as for ourselves. To those of you currently involved in the SMP project, thanks, and keep up the good work! To those of you not currently involved, maybe it's time to jump in. =) Jason SMP project manager To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 23 11:47: 5 2000 Delivered-To: freebsd-smp@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id D765837B4C5 for ; Mon, 23 Oct 2000 11:47:02 -0700 (PDT) Received: from laptop.baldwin.cx (ether.osd.bsdi.com [204.216.28.196]) by pike.osd.bsdi.com (8.11.0/8.9.3) with ESMTP id e9NIi0f27436; Mon, 23 Oct 2000 11:44:00 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <13423.972326495@critter> Date: Mon, 23 Oct 2000 11:44:36 -0700 (PDT) From: John Baldwin To: Poul-Henning Kamp Subject: Re: cvs commit: src/sys/sys mutex.h Cc: freebsd-smp@FreeBSD.org, mjacob@feral.com Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 23-Oct-00 Poul-Henning Kamp wrote: > In message , Matthew > J > acob writes: >>[ taken to smp... ] >> >>> This was the warnings I talked about, hoping they would remind >>> the SMPng team about their brokenness... >>> >>> On i386 those were exactly warnings, they did not break the build. >> >>Broke it on alpha :-). It might actually break on an SMP kernel build for >>i386 >>too. > > It didn't, I tested that. I don't have an alpha I can test on here, > so my automatic scripts didn't cover that. ?? I must have missed a thread in here somewhere... >>What I've done so far is to push this out to the file- >>and to move CURTHD out there too because that's where it'll need to be >>anyway. >>I'm test compiling the changes now so I can put back a new mutex.h that gives >>you what you want but doesn't pork the alpha kernel compiles. Hang on a tick. We are going to be adding in a PCU_GET() macro soon for accessing per-CPU data. Once this is done and the kernel source is changed to use these macros, only machine/globaldata.h will be needed. >>The ultimate problem is partially that proc.h is included in the wrong place >>throughout most of the kernel- it's included *after* buf/bio, but any SMP >>aware Unix system is likely going to need something either from proc.h, or >>some glue layer (call to assembler, most likely) that will yield the current >>thread ID, and places like buffer structures, etc., always have locks >>defined. > > Well, my take on this is that the main problem is that curproc is two > entirely different things for SMP vs UP. That clearly makes all KLD's > toast unless compiled for the right thing. That needs fixed. It isn't b0rked. In machine/globaldata.h (or globals.h maybe) we use the same function calls for accessing data for KLD's that we do for SMP. The only case when we optimize to access the variables directly is within a UP kernel. > In general "#ifdef SMP" should be minimized and in particular no > KLD's should need to know about it. > > I think a function to get the current thread/proc should not require > to be included, most places just need to pass a reference, > they don't need to even know how the entrails of a proccess is > arranged, so obviously they should not need #included.\ I have to have the type. The function returns a type. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 23 11:58: 9 2000 Delivered-To: freebsd-smp@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 0D9C837B4C5; Mon, 23 Oct 2000 11:58:06 -0700 (PDT) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.9.3) with ESMTP id e9NL2GU14130; Mon, 23 Oct 2000 23:02:16 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: John Baldwin Cc: freebsd-smp@FreeBSD.ORG, mjacob@feral.com Subject: Re: cvs commit: src/sys/sys mutex.h In-Reply-To: Your message of "Mon, 23 Oct 2000 11:44:36 PDT." Date: Mon, 23 Oct 2000 23:02:16 +0200 Message-ID: <14128.972334936@critter> From: Poul-Henning Kamp Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message , John Baldwin writes: >> I think a function to get the current thread/proc should not require >> to be included, most places just need to pass a reference, >> they don't need to even know how the entrails of a proccess is >> arranged, so obviously they should not need #included.\ > >I have to have the type. The function returns a type. You can do that simply with: struct proc; struct proc *getcurproc(); You don't need the contents of a struct fiddle a pointer to it. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 23 12: 2: 7 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6727F37B479; Mon, 23 Oct 2000 12:02:05 -0700 (PDT) Received: from feral.com (feral.com [192.67.166.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id CD4886E3035; Mon, 23 Oct 2000 12:02:04 -0700 (PDT) Received: from zeppo.feral.com (IDENT:mjacob@zeppo [192.67.166.71]) by feral.com (8.9.3/8.9.3) with ESMTP id MAA05304; Mon, 23 Oct 2000 12:02:01 -0700 Date: Mon, 23 Oct 2000 11:59:09 -0700 (PDT) From: Matthew Jacob Reply-To: mjacob@feral.com To: John Baldwin Cc: Poul-Henning Kamp , freebsd-smp@FreeBSD.org Subject: Re: cvs commit: src/sys/sys mutex.h In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > > > It didn't, I tested that. I don't have an alpha I can test on here, > > so my automatic scripts didn't cover that. > > ?? I must have missed a thread in here somewhere... Yes, alpha got borked. > Hang on a tick. We are going to be adding in a PCU_GET() macro soon for > accessing per-CPU data. Once this is done and the kernel source is changed > to use these macros, only machine/globaldata.h will be needed. Umm... too late! Sorry if I broke things! > > >>The ultimate problem is partially that proc.h is included in the wrong place > >>throughout most of the kernel- it's included *after* buf/bio, but any SMP > >>aware Unix system is likely going to need something either from proc.h, or > >>some glue layer (call to assembler, most likely) that will yield the current > >>thread ID, and places like buffer structures, etc., always have locks > >>defined. > > > > Well, my take on this is that the main problem is that curproc is two > > entirely different things for SMP vs UP. That clearly makes all KLD's > > toast unless compiled for the right thing. That needs fixed. > > It isn't b0rked. In machine/globaldata.h (or globals.h maybe) we use > the same function calls for accessing data for KLD's that we do for > SMP. The only case when we optimize to access the variables directly > is within a UP kernel. That's the point that Poul and I agree on (I think)- this is broken WRT KLDs- they can't (shouldn't) know whether they're on a UP or SMP. > > > In general "#ifdef SMP" should be minimized and in particular no > > KLD's should need to know about it. > > > > I think a function to get the current thread/proc should not require > > to be included, most places just need to pass a reference, > > they don't need to even know how the entrails of a proccess is > > arranged, so obviously they should not need #included.\ > > I have to have the type. The function returns a type. Yeah- but the type you're playing with currently is a kernel thread pointer. It just happens to be punned to curproc, or PCPU_GET(curproc) right now. In point of fact for mutex.h, you don't need any of the contents of what this CURTHD (punned to curproc) points to- you just use it as an identifier. I would suggest that in the case of mutex.h, just a function that returns a uintptr_t sized unique id is needed. This way you don't have to pollute every source file to have to know about the contents of a thread or a proc structure- *and* as a function you don't have to worry about how things got compiled [ note that I'm now moving from a question as to whether or not we want to do this to an assertion that we do ]. But since I'm not leading the design of this, I'll back away and let you and Poul work it out if you feel this isn't the right direction. -matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 23 12: 2:49 2000 Delivered-To: freebsd-smp@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id 6505937B479; Mon, 23 Oct 2000 12:02:47 -0700 (PDT) Received: from zeppo.feral.com (IDENT:mjacob@zeppo [192.67.166.71]) by feral.com (8.9.3/8.9.3) with ESMTP id MAA05312; Mon, 23 Oct 2000 12:02:24 -0700 Date: Mon, 23 Oct 2000 11:59:32 -0700 (PDT) From: Matthew Jacob Reply-To: mjacob@feral.com To: Poul-Henning Kamp Cc: John Baldwin , freebsd-smp@FreeBSD.ORG Subject: Re: cvs commit: src/sys/sys mutex.h In-Reply-To: <14128.972334936@critter> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > > >I have to have the type. The function returns a type. > > You can do that simply with: > > struct proc; > struct proc *getcurproc(); > > You don't need the contents of a struct fiddle a pointer to it. Right- but you get all the whining compile errors. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 23 12: 5:36 2000 Delivered-To: freebsd-smp@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 8EF1A37B479; Mon, 23 Oct 2000 12:05:33 -0700 (PDT) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.9.3) with ESMTP id e9NL9rU14205; Mon, 23 Oct 2000 23:09:53 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: mjacob@feral.com Cc: John Baldwin , freebsd-smp@FreeBSD.ORG Subject: Re: cvs commit: src/sys/sys mutex.h In-Reply-To: Your message of "Mon, 23 Oct 2000 11:59:32 PDT." Date: Mon, 23 Oct 2000 23:09:53 +0200 Message-ID: <14203.972335393@critter> From: Poul-Henning Kamp Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message , Matthew J acob writes: >> > >> >I have to have the type. The function returns a type. >> >> You can do that simply with: >> >> struct proc; >> struct proc *getcurproc(); >> >> You don't need the contents of a struct fiddle a pointer to it. > >Right- but you get all the whining compile errors. Only if you bogusly redeclare struct proc... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 23 12:11: 6 2000 Delivered-To: freebsd-smp@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 4BF5137B4C5 for ; Mon, 23 Oct 2000 12:11:03 -0700 (PDT) Received: from laptop.baldwin.cx (ether.osd.bsdi.com [204.216.28.196]) by pike.osd.bsdi.com (8.11.0/8.9.3) with ESMTP id e9NJAgf28364; Mon, 23 Oct 2000 12:10:42 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Mon, 23 Oct 2000 12:11:18 -0700 (PDT) From: John Baldwin To: Matthew Jacob Subject: Re: cvs commit: src/sys/sys mutex.h Cc: freebsd-smp@FreeBSD.org, Poul-Henning Kamp Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 23-Oct-00 Matthew Jacob wrote: > >> > >> > It didn't, I tested that. I don't have an alpha I can test on here, >> > so my automatic scripts didn't cover that. >> >> ?? I must have missed a thread in here somewhere... > > Yes, alpha got borked. > >> Hang on a tick. We are going to be adding in a PCU_GET() macro soon for >> accessing per-CPU data. Once this is done and the kernel source is changed >> to use these macros, only machine/globaldata.h will be needed. > > Umm... too late! Sorry if I broke things! > >> >> >>The ultimate problem is partially that proc.h is included in the wrong >> >>place >> >>throughout most of the kernel- it's included *after* buf/bio, but any SMP >> >>aware Unix system is likely going to need something either from proc.h, or >> >>some glue layer (call to assembler, most likely) that will yield the >> >>current >> >>thread ID, and places like buffer structures, etc., always have locks >> >>defined. >> > >> > Well, my take on this is that the main problem is that curproc is two >> > entirely different things for SMP vs UP. That clearly makes all KLD's >> > toast unless compiled for the right thing. That needs fixed. >> >> It isn't b0rked. In machine/globaldata.h (or globals.h maybe) we use >> the same function calls for accessing data for KLD's that we do for >> SMP. The only case when we optimize to access the variables directly >> is within a UP kernel. > > That's the point that Poul and I agree on (I think)- this is broken WRT KLDs- > they can't (shouldn't) know whether they're on a UP or SMP. From i386/include/globals.h: #if defined(SMP) || defined(KLD_MODULE) || defined(ACTUALLY_LKM_NOT_KERNEL) /* * The following set of macros works for UP kernel as well, but for maximum * performance we allow the global variables to be accessed directly. On the * other hand, kernel modules should always use these macros to maintain * portability between UP and SMP kernels. */ #define curproc GLOBAL_RVALUE_NV(curproc, struct proc *) #define curpcb GLOBAL_RVALUE_NV(curpcb, struct pcb *) The kld's _don't_ know. For KLD's we use function calls just like we do with SMP. Within a UP kernel we optimize to access the variables directly. > Yeah- but the type you're playing with currently is a kernel thread > pointer. It just happens to be punned to curproc, or PCPU_GET(curproc) right > now. In point of fact for mutex.h, you don't need any of the contents of what > this CURTHD (punned to curproc) points to- you just use it as an identifier. CURTHD needs a definition somewhere. (Although curthd isn't always correct; spin mutexes should be using cpuid instead of curproc for example.) -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 23 15:41:25 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id C0B0D37B479 for ; Mon, 23 Oct 2000 15:41:19 -0700 (PDT) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.8.7/8.8.7) with ESMTP id JAA09629; Tue, 24 Oct 2000 09:40:56 +1100 Date: Tue, 24 Oct 2000 09:41:20 +1100 (EST) From: Bruce Evans X-Sender: bde@besplex.bde.org To: Matthew Jacob Cc: Poul-Henning Kamp , freebsd-smp@FreeBSD.ORG Subject: Re: cvs commit: src/sys/sys mutex.h In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Mon, 23 Oct 2000, Matthew Jacob wrote: > Strictly speaking, not an inline- but instead a per-platform locore.s > function- on sun4u it's: > > > retl > mov THREAD_REG, %o0 > > On i86 it's: > > movl %gs:CPU_THREAD, %eax > ret > > > This approach solves the KLD issue. But we have to get everyone to agree that > even UP kernels have to pay the cost of even trivial function reference > instead of linktime binding. I wouldn't agree. Just #define machine-dependent access macros in all cases. In particular, don't require MI code to know that per-cpu globals are sometimes normal globals and sometimes macros. Only the i386 UP case made the mistake of not #defining curproc, at least until recently. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 23 15:48:38 2000 Delivered-To: freebsd-smp@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id 6913537B479 for ; Mon, 23 Oct 2000 15:48:35 -0700 (PDT) Received: from zeppo.feral.com (IDENT:mjacob@zeppo [192.67.166.71]) by feral.com (8.9.3/8.9.3) with ESMTP id PAA06271; Mon, 23 Oct 2000 15:44:31 -0700 Date: Mon, 23 Oct 2000 15:41:39 -0700 (PDT) From: Matthew Jacob Reply-To: mjacob@feral.com To: Bruce Evans Cc: Poul-Henning Kamp , freebsd-smp@FreeBSD.ORG Subject: Re: cvs commit: src/sys/sys mutex.h In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > > Strictly speaking, not an inline- but instead a per-platform locore.s > > function- on sun4u it's: > > > > > > retl > > mov THREAD_REG, %o0 > > > > On i86 it's: > > > > movl %gs:CPU_THREAD, %eax > > ret > > > > > > This approach solves the KLD issue. But we have to get everyone to agree that > > even UP kernels have to pay the cost of even trivial function reference > > instead of linktime binding. > > I wouldn't agree. Just #define machine-dependent access macros in all > cases. In particular, don't require MI code to know that per-cpu > globals are sometimes normal globals and sometimes macros. Only the > i386 UP case made the mistake of not #defining curproc, at least until > recently. Okay. I guess I bring all my old biases with me where I just plain don't believe that there really is such a thing as true MI when you talk about macros. -matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 23 18: 3:21 2000 Delivered-To: freebsd-smp@freebsd.org Received: from web1704.mail.yahoo.com (web1704.mail.yahoo.com [128.11.23.215]) by hub.freebsd.org (Postfix) with SMTP id 6E3D237B479 for ; Mon, 23 Oct 2000 18:03:18 -0700 (PDT) Received: (qmail 12832 invoked by uid 60001); 24 Oct 2000 01:03:18 -0000 Message-ID: <20001024010318.12831.qmail@web1704.mail.yahoo.com> Received: from [202.101.165.61] by web1704.mail.yahoo.com; Mon, 23 Oct 2000 18:03:18 PDT Date: Mon, 23 Oct 2000 18:03:18 -0700 (PDT) From: Yifeng Xu Subject: Re: SMP project status To: smp@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org what's status of kernel based pthread implemention? will it still be user level pthread library and can not be smp scaled? Regards, XuYifeng ----- Original Message ----- From: Jason Evans To: Sent: Tuesday, October 24, 2000 2:22 AM Subject: SMP project status > Here's a reminder that the SMP project is being tracked at: > > http://people.freebsd.org/~jasone/smp/ > > Significant progress continues to be made. Many of the tasks that are > listed as "in progress" are essentially complete, pending testing and > review. In general, the SMP project has been going very smoothly, and > -current has been usable a larger percentage of the time than I had hoped > for. > > Note that there are a number of unassigned tasks right now. Some of the > tasks are pretty straightforward, so if you're looking for an opportunity > to help, please take a look at the list. > > A couple of weeks ago, John Baldwin and Chuck Paterson created a list of > rules for kernel synchronization. If you haven't looked at the rules yet, > take a look at: > > http://people.freebsd.org/~jasone/smp/smp_synch_rules.html > > Many of the developers currently involved in the SMP project were at BSDcon > last week. We discussed a lot of technical issues surrounding the tasks > that are currently in progress. We also discussed modes of communication. > Much of the SMP-related communication has taken place via IRC, phone, or in > person meetings. In my opinion, IRC and voice are much more efficient than > email for many things, but I am increasingly concerned about the lack of a > permanent record for many of the conversations we've had up to now. In > some cases, I've tried to go back and figure out why we made a certain > decision, only to realize that there is no record of how the decision was > reached. Therefore, I ask that everyone make a concerted effort to > document issues and reasoning for decisions on the -smp mailing list. This > also means holding technical discussions on the mailing list whenever > practical, and when not practical, summarizing on the mailing list. > SMP-related work is going to continue for several years in one form or > another, and there are probably people who are not even involved with > FreeBSD yet that will be doing some of the work, so we need to leave a > development documentation trail for those people, as well as for ourselves. > > To those of you currently involved in the SMP project, thanks, and keep up > the good work! To those of you not currently involved, maybe it's time to > jump in. =) > > Jason > SMP project manager > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-smp" in the body of the message __________________________________________________ Do You Yahoo!? Yahoo! Messenger - Talk while you surf! It's FREE. http://im.yahoo.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 23 18:23:39 2000 Delivered-To: freebsd-smp@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 75B1237B479 for ; Mon, 23 Oct 2000 18:23:38 -0700 (PDT) Received: (qmail 66466 invoked by uid 1142); 24 Oct 2000 01:23:37 -0000 Date: 23 Oct 2000 18:23:37 -0700 Date: Mon, 23 Oct 2000 18:23:31 -0700 From: Jason Evans To: Yifeng Xu Cc: smp@freebsd.org Subject: Re: SMP project status Message-ID: <20001023182331.P3993@canonware.com> References: <20001024010318.12831.qmail@web1704.mail.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20001024010318.12831.qmail@web1704.mail.yahoo.com>; from websoft@yahoo.com on Mon, Oct 23, 2000 at 06:03:18PM -0700 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Mon, Oct 23, 2000 at 06:03:18PM -0700, Yifeng Xu wrote: > what's status of kernel based pthread implemention? will it still be > user level pthread library and can not be smp scaled? Work is ramping up on scheduler activations, which will allow scaling of threaded applications in proportion to the number of processors. Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 23 22:28:18 2000 Delivered-To: freebsd-smp@freebsd.org Received: from gtei1.bellatlantic.net (gtei1.bellatlantic.net [199.45.39.159]) by hub.freebsd.org (Postfix) with ESMTP id 6AD1637B479 for ; Mon, 23 Oct 2000 22:28:15 -0700 (PDT) Received: from seth-f1pgytg8r3 (client-64-223-145-91.bellatlantic.net [64.223.145.91]) by gtei1.bellatlantic.net (8.9.1/8.9.1) with SMTP id BAA05338 for ; Tue, 24 Oct 2000 01:27:50 -0400 (EDT) Message-Id: <3.0.6.32.20001024013147.00c4d798@hobbiton.shire.net> X-Sender: seth-pc@hobbiton.shire.net X-Mailer: QUALCOMM Windows Eudora Light Version 3.0.6 (32) Date: Tue, 24 Oct 2000 01:31:47 -0700 To: smp@freebsd.org From: Seth Leigh Subject: Re: SMP project status In-Reply-To: <20001023182331.P3993@canonware.com> References: <20001024010318.12831.qmail@web1704.mail.yahoo.com> <20001024010318.12831.qmail@web1704.mail.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org What exactly does this mean? Are we going to have something like the Solaris LWP, and schedule those instead of processes? Basically, what will be the nature of the FreeBSD thread, in terms of kernel schedulable entities? Won't this require a whole new thread library implementation? If so, who is leading that effort? Seth At 06:23 PM 10/23/2000 -0700, Jason Evans wrote: >On Mon, Oct 23, 2000 at 06:03:18PM -0700, Yifeng Xu wrote: >> what's status of kernel based pthread implemention? will it still be >> user level pthread library and can not be smp scaled? > >Work is ramping up on scheduler activations, which will allow scaling of >threaded applications in proportion to the number of processors. > >Jason > > >To Unsubscribe: send mail to majordomo@FreeBSD.org >with "unsubscribe freebsd-smp" in the body of the message > > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 23 23:27:26 2000 Delivered-To: freebsd-smp@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 8AC3837B479 for ; Mon, 23 Oct 2000 23:27:24 -0700 (PDT) Received: (qmail 75253 invoked by uid 1142); 24 Oct 2000 06:27:24 -0000 Date: 23 Oct 2000 23:27:23 -0700 Date: Mon, 23 Oct 2000 23:27:17 -0700 From: Jason Evans To: Seth Leigh Cc: smp@freebsd.org Subject: SA project (was Re: SMP project status) Message-ID: <20001023232717.T3993@canonware.com> References: <20001024010318.12831.qmail@web1704.mail.yahoo.com> <20001024010318.12831.qmail@web1704.mail.yahoo.com> <20001023182331.P3993@canonware.com> <3.0.6.32.20001024013147.00c4d798@hobbiton.shire.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <3.0.6.32.20001024013147.00c4d798@hobbiton.shire.net>; from seth@pengar.com on Tue, Oct 24, 2000 at 01:31:47AM -0700 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Tue, Oct 24, 2000 at 01:31:47AM -0700, Seth Leigh wrote: > At 06:23 PM 10/23/2000 -0700, Jason Evans wrote: > > >Work is ramping up on scheduler activations, which will allow scaling of > >threaded applications in proportion to the number of processors. > > What exactly does this mean? > > Are we going to have something like the Solaris LWP, and schedule those > instead of processes? Basically, what will be the nature of the FreeBSD > thread, in terms of kernel schedulable entities? See http://people.freebsd.org/~jasone/refs/p95-anderson.pdf for a description of scheduler activations. > Won't this require a whole new thread library implementation? If so, who > is leading that effort? The kernel modifications and userland work aren't being treated as separate projects, but they probably will not be implemented in parallel. Large portions of libc_r should be useable. As for someone leading the effort, there isn't a formal leader. I'm the instigator, and Dan Eischen and David O'Brien have expressed interest in working together on it. Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 24 0:16: 4 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp04.primenet.com (smtp04.primenet.com [206.165.6.134]) by hub.freebsd.org (Postfix) with ESMTP id D459337B479 for ; Tue, 24 Oct 2000 00:16:01 -0700 (PDT) Received: (from daemon@localhost) by smtp04.primenet.com (8.9.3/8.9.3) id AAA10382; Tue, 24 Oct 2000 00:12:54 -0700 (MST) Received: from usr01.primenet.com(206.165.6.201) via SMTP by smtp04.primenet.com, id smtpdAAAjOaacu; Tue Oct 24 00:12:45 2000 Received: (from tlambert@localhost) by usr01.primenet.com (8.8.5/8.8.5) id AAA10142; Tue, 24 Oct 2000 00:15:34 -0700 (MST) From: Terry Lambert Message-Id: <200010240715.AAA10142@usr01.primenet.com> Subject: Re: SA project (was Re: SMP project status) To: jasone@canonware.com (Jason Evans) Date: Tue, 24 Oct 2000 07:15:34 +0000 (GMT) Cc: seth@pengar.com (Seth Leigh), smp@FreeBSD.ORG In-Reply-To: <20001023232717.T3993@canonware.com> from "Jason Evans" at Oct 23, 2000 11:27:17 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > On Tue, Oct 24, 2000 at 01:31:47AM -0700, Seth Leigh wrote: > > At 06:23 PM 10/23/2000 -0700, Jason Evans wrote: > > >Work is ramping up on scheduler activations, which will allow scaling of > > >threaded applications in proportion to the number of processors. > > > > What exactly does this mean? > > > > Are we going to have something like the Solaris LWP, and schedule those > > instead of processes? Basically, what will be the nature of the FreeBSD > > thread, in terms of kernel schedulable entities? > > See http://people.freebsd.org/~jasone/refs/p95-anderson.pdf for a > description of scheduler activations. In laymans terms, it means that FreeBSDs threading system will end up being vastly superior to everything but high end commercial UNIX implementations, and superior to a number of those, as well. Specifically, it means that a multithreaded process will not incur the context switch overhead that one normally sees as a result of a blocking call on systems where user space threads are implemented using kernel threads (e.g. Linux, Solaris, NT, Windows 95/98/2000, and any SVR4.0.2 or later derived system). In older designs, using kernel threads as backing objects for user space (process level) threads, when a blocking call is made, the kernel goes into the scheduler, and picks another runnable object to schedule; often this is not something in the same process context as the one making the blocking call, so the promise of threading overhead saving the cost of a full context switch is almost never realized on any system with a non-manufactured workload. A scheduler activation, on the other hand, can reactivate in the specific process (picking another thread from the thread group), so long as there is quantum remaining, and thus actually deliver on the promise of reduced context switch overhead. A side benefit is that cache coherency is also maintained, and there is a reduced interprocessor arbitration overhead. The only approach that's significantly better than activations is a full on async call gate, since it reduces the amount of costly protection domain crossing even further; this can be improved to a larger extent, as well, by dual-mapping pages between user and kernel space to use for communicating completion events. It would have significantly reduced the complexity of the kernel and user space scheduler code to have utilized an async call gate approach, but there would have been a significant cost in terms of binary compatability (it would mean that the standard blocking POSIX calls would have to be implemented as library routines that did an explicit wait for the async call completion event). Binary backward compatability would have meant keeping the existing call structure in place, in parallel, which is a heavy maintenance burden, though not necessarily a long term one. > > Won't this require a whole new thread library implementation? If so, who > > is leading that effort? > > The kernel modifications and userland work aren't being treated as separate > projects, but they probably will not be implemented in parallel. Large > portions of libc_r should be useable. > > As for someone leading the effort, there isn't a formal leader. I'm the > instigator, and Dan Eischen and David O'Brien have expressed interest in > working together on it. Archie and Julian have also expressed interest off and on, as have others. It was perhaps the biggest single turnout that we have ever had at a FreeBSD user group meeting (BAFUG/BABUG). Perhaps the biggest pain is the default signal behaviour, which no longer results in system call restart. This means that the signal code must self-mask anything that it wants to pretend is a system call, for every wrapping library routine. POSIX really screwed us over on that account, kind of like what they did to locks when you close one of several file handles to the same file. As Jason has noted, most of this code can be reused out of the existing lib_r, with few or no changes, so that's not a pain that will be returning to haunt us. Most, if not all, of the user space work will be in the user space scheduler, dealing with what to do when an activation occurs. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 24 4:55:57 2000 Delivered-To: freebsd-smp@freebsd.org Received: from pcnet1.pcnet.com (pcnet1.pcnet.com [204.213.232.3]) by hub.freebsd.org (Postfix) with ESMTP id AD8B937B479 for ; Tue, 24 Oct 2000 04:55:54 -0700 (PDT) Received: (from eischen@localhost) by pcnet1.pcnet.com (8.8.7/PCNet) id HAA00420; Tue, 24 Oct 2000 07:55:28 -0400 (EDT) Date: Tue, 24 Oct 2000 07:55:28 -0400 (EDT) From: Daniel Eischen To: Jason Evans Cc: Seth Leigh , smp@FreeBSD.ORG Subject: Re: SA project (was Re: SMP project status) In-Reply-To: <20001023232717.T3993@canonware.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 23 Oct 2000, Jason Evans wrote: > On Tue, Oct 24, 2000 at 01:31:47AM -0700, Seth Leigh wrote: > > Won't this require a whole new thread library implementation? If so, who > > is leading that effort? > > The kernel modifications and userland work aren't being treated as separate > projects, but they probably will not be implemented in parallel. Large > portions of libc_r should be useable. And a fair amount of code can be thrown away without needing reimplementation :-) > > As for someone leading the effort, there isn't a formal leader. I'm the > instigator, and Dan Eischen and David O'Brien have expressed interest in > working together on it. I'm just about ready to start working on this; I just want to make sure libc_r is in a state where we can leave it alone for a while. -- Dan Eischen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 24 5:24:46 2000 Delivered-To: freebsd-smp@freebsd.org Received: from pcnet1.pcnet.com (pcnet1.pcnet.com [204.213.232.3]) by hub.freebsd.org (Postfix) with ESMTP id EBDB537B479 for ; Tue, 24 Oct 2000 05:24:44 -0700 (PDT) Received: (from eischen@localhost) by pcnet1.pcnet.com (8.8.7/PCNet) id IAA04480; Tue, 24 Oct 2000 08:24:24 -0400 (EDT) Date: Tue, 24 Oct 2000 08:24:19 -0400 (EDT) From: Daniel Eischen To: Terry Lambert Cc: Jason Evans , Seth Leigh , smp@FreeBSD.ORG Subject: Re: SA project (was Re: SMP project status) In-Reply-To: <200010240715.AAA10142@usr01.primenet.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Tue, 24 Oct 2000, Terry Lambert wrote: > Perhaps the biggest pain is the default signal behaviour, which > no longer results in system call restart. This means that the > signal code must self-mask anything that it wants to pretend is > a system call, for every wrapping library routine. POSIX really > screwed us over on that account, kind of like what they did to > locks when you close one of several file handles to the same file. Hmm. If the method used by the kernel to notify the threads library that a thread has blocked in the kernel can also tell the threads library that it is/isn't blocked in a restartable system call, then the threads library can DTRT WRT interrupting the thread. Basically, all it would need is the system call number. It could probably get it from the context of the interrupted thread (which is passed out to the threads library under SA). There are only a few system calls that require restarting (assuming SA_RESTART is set), so I'm not quite sure what you mean by self-masking library routines. Perhaps library routines that use read(), for instance, in their implementation? -- Dan Eischen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 24 6:41:49 2000 Delivered-To: freebsd-smp@freebsd.org Received: from gtei2.bellatlantic.net (gtei2.bellatlantic.net [199.45.39.161]) by hub.freebsd.org (Postfix) with ESMTP id F16AB37B479 for ; Tue, 24 Oct 2000 06:41:45 -0700 (PDT) Received: from seth-f1pgytg8r3 (client-64-223-145-91.bellatlantic.net [64.223.145.91]) by gtei2.bellatlantic.net (8.9.1/8.9.1) with SMTP id JAA13077; Tue, 24 Oct 2000 09:37:09 -0400 (EDT) Message-Id: <3.0.6.32.20001024094101.00c4d798@hobbiton.shire.net> X-Sender: seth-pc@hobbiton.shire.net X-Mailer: QUALCOMM Windows Eudora Light Version 3.0.6 (32) Date: Tue, 24 Oct 2000 09:41:01 -0700 To: Terry Lambert , jasone@canonware.com (Jason Evans) From: Seth Leigh Subject: Re: SA project (was Re: SMP project status) Cc: smp@FreeBSD.ORG In-Reply-To: <200010240715.AAA10142@usr01.primenet.com> References: <20001023232717.T3993@canonware.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org At 07:15 AM 10/24/2000 +0000, Terry Lambert wrote: >> On Tue, Oct 24, 2000 at 01:31:47AM -0700, Seth Leigh wrote: >> > At 06:23 PM 10/23/2000 -0700, Jason Evans wrote: >> > >Work is ramping up on scheduler activations, which will allow scaling of >> > >threaded applications in proportion to the number of processors. >> > >> > What exactly does this mean? >> > >> > Are we going to have something like the Solaris LWP, and schedule those >> > instead of processes? Basically, what will be the nature of the FreeBSD >> > thread, in terms of kernel schedulable entities? >> >> See http://people.freebsd.org/~jasone/refs/p95-anderson.pdf for a >> description of scheduler activations. > >In laymans terms, it means that FreeBSDs threading system will >end up being vastly superior to everything but high end commercial >UNIX implementations, and superior to a number of those, as well. > >Specifically, it means that a multithreaded process will not incur >the context switch overhead that one normally sees as a result of >a blocking call on systems where user space threads are implemented >using kernel threads (e.g. Linux, Solaris, NT, Windows 95/98/2000, >and any SVR4.0.2 or later derived system). > >In older designs, using kernel threads as backing objects for user >space (process level) threads, when a blocking call is made, the >kernel goes into the scheduler, and picks another runnable object >to schedule; often this is not something in the same process context >as the one making the blocking call, so the promise of threading >overhead saving the cost of a full context switch is almost never >realized on any system with a non-manufactured workload. > >A scheduler activation, on the other hand, can reactivate in the >specific process (picking another thread from the thread group), >so long as there is quantum remaining, and thus actually deliver >on the promise of reduced context switch overhead. A side benefit >is that cache coherency is also maintained, and there is a reduced >interprocessor arbitration overhead. This may seem petty, but if we always use the whole quantum, won't this have the effect of driving down the priority of any multi-threaded application with respect to single-threaded apps? You will pardon me if I ask dumb questions. After dabbling and reading about it for a long time, I have finally started working on my first major multi-threaded application, and so I am thinking a lot about them but I am not necessarily a guru. Additionally, I aspire someday to being a kernal guy, so I want to learn how these things work. >The only approach that's significantly better than activations is >a full on async call gate, since it reduces the amount of costly >protection domain crossing even further; this can be improved to >a larger extent, as well, by dual-mapping pages between user and >kernel space to use for communicating completion events. > >It would have significantly reduced the complexity of the kernel >and user space scheduler code to have utilized an async call gate >approach, but there would have been a significant cost in terms >of binary compatability (it would mean that the standard blocking >POSIX calls would have to be implemented as library routines that >did an explicit wait for the async call completion event). Binary >backward compatability would have meant keeping the existing call >structure in place, in parallel, which is a heavy maintenance >burden, though not necessarily a long term one. > > >> > Won't this require a whole new thread library implementation? If so, who >> > is leading that effort? >> >> The kernel modifications and userland work aren't being treated as separate >> projects, but they probably will not be implemented in parallel. Large >> portions of libc_r should be useable. >> >> As for someone leading the effort, there isn't a formal leader. I'm the >> instigator, and Dan Eischen and David O'Brien have expressed interest in >> working together on it. > >Archie and Julian have also expressed interest off and on, as have >others. It was perhaps the biggest single turnout that we have >ever had at a FreeBSD user group meeting (BAFUG/BABUG). > >Perhaps the biggest pain is the default signal behaviour, which >no longer results in system call restart. This means that the >signal code must self-mask anything that it wants to pretend is >a system call, for every wrapping library routine. POSIX really >screwed us over on that account, kind of like what they did to >locks when you close one of several file handles to the same file. > >As Jason has noted, most of this code can be reused out of the >existing lib_r, with few or no changes, so that's not a pain that >will be returning to haunt us. Most, if not all, of the user space >work will be in the user space scheduler, dealing with what to do >when an activation occurs. > > > Terry Lambert > terry@lambert.org >--- >Any opinions in this posting are my own and not those of my present >or previous employers. > > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 24 7: 0:21 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp04.primenet.com (smtp04.primenet.com [206.165.6.134]) by hub.freebsd.org (Postfix) with ESMTP id 89DE037B479 for ; Tue, 24 Oct 2000 07:00:17 -0700 (PDT) Received: (from daemon@localhost) by smtp04.primenet.com (8.9.3/8.9.3) id GAA08563; Tue, 24 Oct 2000 06:57:10 -0700 (MST) Received: from usr05.primenet.com(206.165.6.205) via SMTP by smtp04.primenet.com, id smtpdAAA07aaHq; Tue Oct 24 06:56:58 2000 Received: (from tlambert@localhost) by usr05.primenet.com (8.8.5/8.8.5) id GAA27114; Tue, 24 Oct 2000 06:59:53 -0700 (MST) From: Terry Lambert Message-Id: <200010241359.GAA27114@usr05.primenet.com> Subject: Re: SA project (was Re: SMP project status) To: eischen@vigrid.com (Daniel Eischen) Date: Tue, 24 Oct 2000 13:59:53 +0000 (GMT) Cc: tlambert@primenet.com (Terry Lambert), jasone@canonware.com (Jason Evans), seth@pengar.com (Seth Leigh), smp@FreeBSD.ORG In-Reply-To: from "Daniel Eischen" at Oct 24, 2000 08:24:19 AM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > Perhaps the biggest pain is the default signal behaviour, which > > no longer results in system call restart. This means that the > > signal code must self-mask anything that it wants to pretend is > > a system call, for every wrapping library routine. POSIX really > > screwed us over on that account, kind of like what they did to > > locks when you close one of several file handles to the same file. > > Hmm. If the method used by the kernel to notify the threads library > that a thread has blocked in the kernel can also tell the threads > library that it is/isn't blocked in a restartable system call, > then the threads library can DTRT WRT interrupting the thread. > Basically, all it would need is the system call number. It could > probably get it from the context of the interrupted thread (which > is passed out to the threads library under SA). This may be one approach; it assumes a scheduler context in user space which is updated from kernel space, and which contains the call number. It's doable, but potentially gross. > There are only a few system calls that require restarting (assuming > SA_RESTART is set), so I'm not quite sure what you mean by self-masking > library routines. Perhaps library routines that use read(), for > instance, in their implementation? OK, if I have a call that can be interrupted, e.g.: int foo( struct fooargs *args) { } And I want to wrap it in user space so that I can do a call conversion reschedule based on it blocking, and me getting an activation, then I have: int foo( struct fooargs *args) /* wrapper*/ { ... rv = realfoo( args); ... /* Danger, Will Robinson!*/ } ..so I need the code that occurs before and after the call to the realfoo() call to appear as atomic as the realfoo() call, in the event that I either need to restart it, or, worst case, I need the library wrapper foo() to return EINTR as if it were realfoo() in an unwrapped library, not running threaded. This means that I need to mask off any additional signal processing between the time that realfoo() returns EINTR, and processing to the end of the function is completed to the point that foo() can return -- in other words, foo() must look like a system call to realfoo() to the caller, even if we end up with a blocking call in the kernel itself, coupled with an activation. It wouldn't do to, for example, disable SIGINT or SIGHUP, merely because I had some background processing going on in another thread. Under activations, this applies to everything the uses non-blocking fd's in the current situation, coupled with a scheduler entry in user space to convert a call that would have blocked into a call that returns EWOULDBLK. Under activations, this becomes once again a blocking call, with an activation occurring at the point that the current kernel would normally put the process to sleep (the one exception to this would be an involuntary context switch as a result of the quantum being used up). Unfortunately, it gets more complicated, as Julian and Archie could also tell you. Since an involuntary context switch can occurr during the processing of an activation, in other words, in the user space scheduler, instead of the kernel, activations that are pending delivery to user space have to be stalled until the user space scheduler hits a "safe" place, where it can be concurrent again, so it can safely process the pending activations. You need one of these per CPU on which you might be scheduled to run, or more, if you are capable of being in the scheduler more than once on a single CPU (this is necessary for some priority handling). As I said before, this is really more complicated than it would have to be, if all non-blocking scheduling events occurred in user space. You could do this, if you had either a non-blocking system call interface, or a generic AST mechanism, like VMS and NT have, since either approach will result in either all the work being done in user space, or all the work being done in the kernel. The DEC MTS services use AST-based call conversion to schedule several concurrent operations simultaneously; I used that to add the missing interlocks into Mentat Streams on VMS back in 92 or so, since the MTS as it shipped from DEC didn't have timers or interlocks for doing inter-thread semaphoring. OBTW: the signal processing following an interrupted system call is not the only situation where blocking additional interruptions until processing has completed will be an issue. On any architecture with register windows, such as SPARC, you will need an explicit kernel assist to flush the register windows, so it will have to be stubbed in, even it it's a NOOP on x86 architectures (at least until the new Intel stuff becomes common, I think). Personally, I'd probably approach the problem as if I were going to allow multiple concurrent outstanding system calls from a single process on a uniprocessor system (which means that you can have more than one entry in the scheduler for a given process), and then sort out the migration and other issues, as well as the SMP issues, later. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 24 7:32:14 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp05.primenet.com (smtp05.primenet.com [206.165.6.135]) by hub.freebsd.org (Postfix) with ESMTP id 27D9037B479 for ; Tue, 24 Oct 2000 07:32:10 -0700 (PDT) Received: (from daemon@localhost) by smtp05.primenet.com (8.9.3/8.9.3) id HAA18250; Tue, 24 Oct 2000 07:32:35 -0700 (MST) Received: from usr05.primenet.com(206.165.6.205) via SMTP by smtp05.primenet.com, id smtpdAAADFaGIJ; Tue Oct 24 07:32:26 2000 Received: (from tlambert@localhost) by usr05.primenet.com (8.8.5/8.8.5) id HAA28242; Tue, 24 Oct 2000 07:31:55 -0700 (MST) From: Terry Lambert Message-Id: <200010241431.HAA28242@usr05.primenet.com> Subject: Re: SA project (was Re: SMP project status) To: seth@pengar.com (Seth Leigh) Date: Tue, 24 Oct 2000 14:31:54 +0000 (GMT) Cc: tlambert@primenet.com (Terry Lambert), jasone@canonware.com (Jason Evans), smp@FreeBSD.ORG In-Reply-To: <3.0.6.32.20001024094101.00c4d798@hobbiton.shire.net> from "Seth Leigh" at Oct 24, 2000 09:41:01 AM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > >A scheduler activation, on the other hand, can reactivate in the > >specific process (picking another thread from the thread group), > >so long as there is quantum remaining, and thus actually deliver > >on the promise of reduced context switch overhead. A side benefit > >is that cache coherency is also maintained, and there is a reduced > >interprocessor arbitration overhead. > > This may seem petty, but if we always use the whole quantum, won't this > have the effect of driving down the priority of any multi-threaded > application with respect to single-threaded apps? No. If you want to build an application that competes unfairly with other processes for system resources, the correct approach is to use multiple processes in order to get multiple quanta, OR you can define a new scheduler class that implements fair share scheduling or some other scheduling algorithm that gives your program an unfair advantage in being selected for quanta. Realize that the benefit of not paying the context switch overhead will reduce overall system utilization. Realize also that you have a hidden assumption here, which is not necessarily true: that you will always have threads that are ready to run, and are not all blocked pending I/O or other kernel operations. Unless you run one thread in a spin loop, this will most likely never really be the case. Consider how you would fix the context and cache thrashing problem on a Linux or an SVR4 derived system: you could preferrentially choose a thread in your thread group, when making your scheduling decision. But this leads to starvation of other processes, should you make a coding error and go into a spin loop (or simply have a lot of work to do in user space which is CPU bound, such as rendering images). Alternately, you might implement round-robin scheduling or some other scheduling policy, and group threads in a single process next to each other in the runnable queue. But if you have even a moderate number of threads, then you will damage interactive response, perhaps to a considerable degree. Effectively, you are left with a very hard problem. The size of the quantum was chosen such that interactive response would not be damaged, even if a process used the entirety of its allotted CPU doing something compute intensive. Even if you still balked at the "unfairness" of being unable to have your one program compete with sendmail or inted as if it were 16,000 processes (for some definition of "unfairness" 8-)), you could choose to weight it based on system calls currently blocked, so it becomes the amount of time on average that your threads remain blocked. I personally wouldn't do this. If I were worried about my threaded application, I would either use rtprio to force the issue, or I would "manufacture" my server load: for example, it's very rare to see an Oracle server doing anything other than simply running Oracle. > You will pardon me if I ask dumb questions. After dabbling and reading > about it for a long time, I have finally started working on my first major > multi-threaded application, and so I am thinking a lot about them but I am > not necessarily a guru. Additionally, I aspire someday to being a kernal > guy, so I want to learn how these things work. You'd do well to study schedulers and context switch overhead, and cache-busting. The scheduling algorithms are actually much more complicated than they first appear, and it's not obvious on first glance that kernel threads, as an implementation, interact badly with schedulers, or where the system overhead really lives. Sun has a number of good papers on threading that I would recommend looking up on their web site. Really, threading tends to make some types of programming easier, but isn't terribly useful, unless you are trying to achieve SMP scaling. Even then, many OSs do it wrong. There's a long standing fiction that SMP systems start failing to scale at 4 processors, that they reach a point of diminishing returns at a relatively small scale. This isn't really true: mostly it's an implementation failure when you see a limit that small. John Sokol actually gave a nice presentation on using finite state automatons in place of threading in the "AfterBurner" web server product, and backed it up with some nice numbers; but his solution, while incredibly capable on low end hardware, would not scale to better ability on SMP. I don't think he ever hit a CPU binding limitation, but if he were to do so, the only thing he'd be able to do would be to throw bigger iron at it. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 24 8:31:42 2000 Delivered-To: freebsd-smp@freebsd.org Received: from crewsoft.com (ns.aenet.net [157.22.214.1]) by hub.freebsd.org (Postfix) with ESMTP id 2496B37B4C5 for ; Tue, 24 Oct 2000 08:31:40 -0700 (PDT) Received: from [63.197.8.222] (HELO wireless-networks.com) by crewsoft.com (CommuniGate Pro SMTP 3.3b5) with ESMTP id 332471 for smp@freebsd.org; Tue, 24 Oct 2000 08:33:43 -0700 Message-ID: <39F5AB57.EA51946D@wireless-networks.com> Date: Tue, 24 Oct 2000 08:31:35 -0700 From: Cedric Berger X-Mailer: Mozilla 4.73 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: smp@freebsd.org Subject: Re: SA project (was Re: SMP project status) References: <20001024010318.12831.qmail@web1704.mail.yahoo.com> <20001024010318.12831.qmail@web1704.mail.yahoo.com> <20001023182331.P3993@canonware.com> <3.0.6.32.20001024013147.00c4d798@hobbiton.shire.net> <20001023232717.T3993@canonware.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > See http://people.freebsd.org/~jasone/refs/p95-anderson.pdf for a > description of scheduler activations. > Wow, We'll finally be able to run thoses Java applications properly! I can't wait! Cedric To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 26 5:53:31 2000 Delivered-To: freebsd-smp@freebsd.org Received: from imo-d04.mx.aol.com (imo-d04.mx.aol.com [205.188.157.36]) by hub.freebsd.org (Postfix) with ESMTP id 690F937B4C5 for ; Thu, 26 Oct 2000 05:53:27 -0700 (PDT) Received: from Cafetyme@aol.com by imo-d04.mx.aol.com (mail_out_v28.32.) id n.a3.d0e1978 (4358) for ; Thu, 26 Oct 2000 08:53:22 -0400 (EDT) From: Cafetyme@aol.com Message-ID: Date: Thu, 26 Oct 2000 08:53:21 EDT Subject: SMP To: freebsd-smp@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Mailer: Windows AOL sub 114 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org My computer keeps rebooting on its own and shows that I have lost chains , or sometimes its blank . Someone told me I needed to replace my SMP , can you please tell me where is the SMP is in my computer , and how to fix this problem? Sometimes it just keeps rebooting without saying I have lost chains. Alexandra To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 26 5:53:48 2000 Delivered-To: freebsd-smp@freebsd.org Received: from imo-d07.mx.aol.com (imo-d07.mx.aol.com [205.188.157.39]) by hub.freebsd.org (Postfix) with ESMTP id D036B37B4C5 for ; Thu, 26 Oct 2000 05:53:43 -0700 (PDT) Received: from Cafetyme@aol.com by imo-d07.mx.aol.com (mail_out_v28.32.) id n.b5.24b2cf9 (4358) for ; Thu, 26 Oct 2000 08:53:41 -0400 (EDT) From: Cafetyme@aol.com Message-ID: Date: Thu, 26 Oct 2000 08:53:41 EDT Subject: SMP To: freebsd-smp@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Mailer: Windows AOL sub 114 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org My computer keeps rebooting on its own and shows that I have lost chains , or sometimes its blank . Someone told me I needed to replace my SMP , can you please tell me where is the SMP is in my computer , and how to fix this problem? Sometimes it just keeps rebooting without saying I have lost chains. Alexandra To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Oct 27 6:46:46 2000 Delivered-To: freebsd-smp@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 5EF1037B4C5 for ; Fri, 27 Oct 2000 06:46:44 -0700 (PDT) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.9.3) with ESMTP id e9RDkfN10865 for ; Fri, 27 Oct 2000 15:46:43 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: smp@freebsd.org Subject: ktr.h improvement From: Poul-Henning Kamp Date: Fri, 27 Oct 2000 15:46:41 +0200 Message-ID: <10863.972654401@critter> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Is anybody madly in love with the massive inlining used by the ktr stuff, or would an improvement of it be welcome ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Oct 27 9:14:11 2000 Delivered-To: freebsd-smp@freebsd.org Received: from berserker.bsdi.com (berserker.twistedbit.com [199.79.183.1]) by hub.freebsd.org (Postfix) with ESMTP id 22E6E37B479; Fri, 27 Oct 2000 09:14:08 -0700 (PDT) Received: from berserker.bsdi.com (cp@localhost.bsdi.com [127.0.0.1]) by berserker.bsdi.com (8.9.3/8.9.3) with ESMTP id KAA12094; Fri, 27 Oct 2000 10:14:07 -0600 (MDT) (envelope-from cp@berserker.bsdi.com) Message-Id: <200010271614.KAA12094@berserker.bsdi.com> To: Poul-Henning Kamp Cc: smp@FreeBSD.ORG Subject: Re: ktr.h improvement In-reply-to: Your message of "Fri, 27 Oct 2000 15:46:41 +0200." <10863.972654401@critter> From: Chuck Paterson Date: Fri, 27 Oct 2000 10:14:07 -0600 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org The macros that came from BSD/OS actually can be improved very little my changing. In the case of Sparc a small section is done in a function to set up the pointer and then the rest is inline. In X86 the upfront code is quite small so everything was done inline. The code in question is +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- movl $ KTR_ESIZE,reg; /* How much to incr */ \ xaddl reg,%fs:PCPU_KTR_IDX; /* get a slot */ \ rdtsc; /* time -> eax/edx */ \ andl seg ktr_size_mask,reg; /* wrap buffer */ \ addl %fs:PCPU_KTR_BUF,reg; /* Point to actual spot in buffer */ \ movl %eax,seg KTR_CLKV(reg);/* Time low */ \ movl %edx,seg KTR_CLKV+4(reg); /* Time high */ \ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- The rest of the macros as little or less space as passing the data to a function so there really isn't anything to be gained, and speed to be lost. In the case of FreeBSD there is also the KTR_EXTEND. This should be pushed into a function as much as possible. It is already so slow that it really can't be used in any kind of live kernel so any space to time overhead is a win. There is another big issue with the KTR macros which really needs to be addressed. Macros which are defined to not be compiled in are removed by the optimizer. This removes the code, but doesn't remove the strings. I have include a sample to show how this could be done so that the preprocessor rather than the optimizer leave this stuff out. Chuck #include #define KTR_TRACE 2 #define TRACE_KTR_FOO_BIT 2 /* some random bit */ #if KTR_TRACE & TRACE_KTR_FOO_BIT #define TRACE_KTR_FOO(x) printf ("%s\n", x) #else #define TRACE_KTR_FOO(x) #endif #define CTR0(x,y) __CONCAT(TRACE_, x)(y) int main(int ac, char **av) { CTR0(KTR_FOO, "now is the time"); } To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Oct 27 10:58:57 2000 Delivered-To: freebsd-smp@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 9969B37B479; Fri, 27 Oct 2000 10:58:55 -0700 (PDT) Received: from laptop.baldwin.cx (ether.osd.bsdi.com [204.216.28.196]) by pike.osd.bsdi.com (8.11.0/8.9.3) with ESMTP id e9RHwOf84062; Fri, 27 Oct 2000 10:58:24 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <10863.972654401@critter> Date: Fri, 27 Oct 2000 10:59:14 -0700 (PDT) From: John Baldwin To: Poul-Henning Kamp Subject: RE: ktr.h improvement Cc: smp@FreeBSD.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 27-Oct-00 Poul-Henning Kamp wrote: > > Is anybody madly in love with the massive inlining used by the ktr > stuff, or would an improvement of it be welcome ? Define improvement. :) Right now it allows the compiler to optimize out things that aren't in KTR_COMPILE. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Oct 27 16:28:35 2000 Delivered-To: freebsd-smp@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 16BA237B479 for ; Fri, 27 Oct 2000 16:28:34 -0700 (PDT) Received: (qmail 65139 invoked by uid 1142); 27 Oct 2000 23:28:33 -0000 Date: 27 Oct 2000 16:28:33 -0700 Date: Fri, 27 Oct 2000 16:28:18 -0700 From: Jason Evans To: smp@freebsd.org Subject: KSE project announcement Message-ID: <20001027162818.I48771@canonware.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org One of the best reasons to use SMP hardware is to scale up the performance of multi-threaded applications, which isn't currently possible with libc_r. The SMP project is well under way now and continues to make solid progress, so in addition to managing that project, I'm again focusing some attention on SMP-scalable userland threads support. There is now a KSE project web page at: http://people.freebsd.org/~jasone/kse/ There were lengthy design discussions almost a year ago on the mailing lists and at BA[BF]UG (Bay Area [Free]BSD Users Group) meetings. I've written a draft paper that explains the approach we decided on. Note that the paper is a draft, which means it is missing some details, there may be inaccuracies, and some things might change. Nonetheless, if this project is of interest to you, the paper should be enlightening: http://people.freebsd.org/~jasone/refs/freebsd_kse/freebsd_kse.html http://people.freebsd.org/~jasone/refs/freebsd_kse.ps Feedback is welcome, of course. Quite a number of people were involved in the design, and there are presently three of us that are attacking the implementation. If you have the time, interest, and background necessary to aid in implementation, please contact me. Thanks, Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message