From owner-freebsd-smp Tue Dec 25 23:14:15 2001 Delivered-To: freebsd-smp@freebsd.org Received: from technokratis.com (modemcable099.144-201-24.mtl.mc.videotron.ca [24.201.144.99]) by hub.freebsd.org (Postfix) with ESMTP id 231EA37B41A; Tue, 25 Dec 2001 23:14:03 -0800 (PST) Received: (from bmilekic@localhost) by technokratis.com (8.11.4/8.11.3) id fBQ7I1N14795; Wed, 26 Dec 2001 02:18:01 -0500 (EST) (envelope-from bmilekic) Date: Wed, 26 Dec 2001 02:18:00 -0500 From: Bosko Milekic To: freebsd-arch@freebsd.org, freebsd-smp@freebsd.org Subject: SMPng: Interrupt Latency Issues Message-ID: <20011226021800.A14608@technokratis.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Hi, It has become obvious, recently in particular, that some important improvements are required in the way we take interrupts in -CURRENT SMPng. As previously mentionned, we are experiencing lousy interrupt latency in -CURRENT. This comes as no surprise. The purpose of this Email is to get the ball rolling on a discussion pertaining not only to the approach that we're going to take to properly remedy the latency issues, but also on the overall ways that we will be handeling interrupts in SMPng. I know that there are already several ideas floating around, and briefly talking to Jake and some others over IRC, I see that there are a couple that are more common than others. Now I know that this is a sensitive topic but in discussing it, I would appreciate it if all points are strictly technical and deal with the implementation, that we not over-generalize (in other words, that we stay `on topic' [*]), and that after the discussion is done, that we set up some milestones and get the work done _soon_ (I'm prepared to take up some of the work, yes - however, I do not feel that *I* am the right person to oversee _all_ of the technical aspects of this heavily burdened task alone). [*] This means please don't start bashing anything and everything in our system and stating useless things like "well, yeah, but our VM system does not do X, Y, and Z" if it is of no _direct_ relevance to the way we take interrupts. With that out of the way, here's our situation: 1. We presently take an interrupt, and in the general case, proceed to schedule an interrupt thread to run. While placing the thread on the run queue and because we need to check for the "SWAIT" process flag we must acquire the sched_lock. This is where the jist of the problem lies. We bottleneck here because it means that only one CPU can be scheduling the interrupt thread at a time, and the rest have to spin waiting for sched_lock. To make matters worse, interrupts are disabled on the local CPU and we cannot take any other interrupts either. This is where we stand. 2. BSD/OS does things differently. An interrupt comes in and in the general case, the interrupted thread's VM address space is borrowed and the handler is immediately executed after interrupts are re-enabled. Then the handler(s) run and only if it happens to hit a mutex lock for which it must wait, a clean-up is done to provide a thread that can block on the mutex for the interrupt, and to allow the interrupted thread to continue doing its thing. The tradeoff is that the interrupted thread _cannot_ run anywhere else, not even on another CPU, while the interrupt that pre-empted it is not finished executing or has not hit a mutex and could not aquire it quickly. This may result in some kernel thread priority rules not being 100% obeyed but I guess that this is part of the tradeoff. 3. I believe that some others have alternative suggestions. I encourage them to present them here clearly, assuming that they are realistic and implementable approaches, so that we are sure to make the right decision before we setup milestones. In particular, I know that, after briefly speaking to Jake, there is the idea of per-CPU "interrupt-only run queues" floating around. The jist of this method would be to keep per-CPU run queues to which each CPU can schedule their own interrupt threads without having to acquire any locks (i.e., only have interrupts disabled). The un-balancing of the queues as well as special priority cases - should they arise - could be handeled by issuing IPIs. I also know that some others (notably Rik van Riel - I hope I spelled that correctly :-) ) have mentionned realistic ideas that are quite logical in light of what we presently have and the points above. To me, following a very brief overview, some of them shared some of the qualities of the BSD/OS way of doing it so I'd like to invite him (and others) to lay those methods out for us now, so that we have a greater picture of various alternatives. That's all for now. I hope that we can agree on something worthwhile soon so that we can establish clear milestones and get this thing done. It's been long overdue. Best regards and Seasons Greetings to you all, -- Bosko Milekic bmilekic@FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Dec 29 15:50:56 2001 Delivered-To: freebsd-smp@freebsd.org Received: from mail3.registeredsite.com (mail3.registeredsite.com [64.224.9.12]) by hub.freebsd.org (Postfix) with ESMTP id B621337B41A; Sat, 29 Dec 2001 15:50:53 -0800 (PST) Received: from proserver128 ([216.247.248.47]) by mail3.registeredsite.com (8.11.6/8.11.4) with SMTP id fBTMsZ500381; Sat, 29 Dec 2001 17:54:35 -0500 Message-Id: <200112292254.fBTMsZ500381@mail3.registeredsite.com> Date: Sat, 29 Dec 2001 18:50:43 -0500 From: Heatherx19@aol.com () Reply-To: Heatherx19@aol.com Subject: hey babe To: bmah@FreeBSD.org, kbyanc@FreeBSD.org, jhb@FreeBSD.org, smp@FreeBSD.org, bmilekic@FreeBSD.org X-Mailer: WinNT's Blat ver 1.8.2b http://www.interlog.com/~tcharron Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org To: bmah@FreeBSD.org,kbyanc@FreeBSD.org,jhb@FreeBSD.org,smp@FreeBSD.org,bmilekic@FreeBSD.org From: Heatherx19@aol.com () Subject: hey babe Below is the result of your feedback form. It was submitted by (Heatherx19@aol.com) on Saturday, December 29, 2001 at 18:50:42 --------------------------------------------------------------------------- message: Hi, my name is Heather and I am a 19 year old female from San Diego, California. Ever since my 14th birthday, I have been really sexually active, but I am still a virgin. Now I am 19 and away from home, attending school at San Diego State University and sharing a dorm with four of my girlfriends and are all VERY turned on to meet a guy and satisfy ALL of his pleasures. To see our sexy pictures we took just last week and to meet some other couples, go to our site
< a href="http://www.lllil.com/heather/livewebcam">http://www.lllil.com/heather/livewebcam


-1119 --------------------------------------------------------------------------- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message