From owner-freebsd-smp Sun Jan 21 8:42: 5 2001 Delivered-To: freebsd-smp@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id D803737B400 for ; Sun, 21 Jan 2001 08:41:46 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id RAA66028; Sun, 21 Jan 2001 17:41:45 +0100 (CET) (envelope-from des@ofug.org) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: smp@freebsd.org Subject: MP-safing the zone allocator From: Dag-Erling Smorgrav Date: 21 Jan 2001 17:41:44 +0100 Message-ID: Lines: 32 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org [this is my debut as SMP hacker, so I'll be grateful if you bear with me while I think aloud on this list, and even more if you take the time to point out flaws in my reasoning] The zone allocator has a double set of API functions, one which is supposedly MP-safe and one which isn't (but is slightly faster because it doesn't grab the locks). The code is sprinkled with #ifdef SMP's so that SMP kernels only use the MP-safe version of the API. The patch I posted to -arch[1] removes the MP-unsafe API, which changes exactly zilch in SMP kernels since they didn't use it to begin with. On UP kernels, it adds the overhead of grabbing the lock, which is no big deal - I think the reason for the MP-unsafe version was not so much to avoid grabbing the locks as to avoid running at splvm, but that's a no-op these days. And since the UP kernel is multithreaded, we *have* to always use the MP-safe code anyway. This is only the first of (as I see it) three stages in thread-safing the zone allocator. The second stage is to convert the simplelocks to mutices (Jason's simplelock_2.diff does most of that work). The third stage is to add a mutex that protects zlist, and maybe convert zlist to an SLIST or a TAILQ; when that is done, it should be possible to call the zone allocator without holding Giant. I'm pretty confident that the first patch is correct, so I'll commit it once I've tested both UP and SMP kernels built with it. I'll submit patches for the 2nd and 3rd stages for review as work progresses. DES (feels like a newbie again) -- Dag-Erling Smorgrav - des@ofug.org [1] http://people.freebsd.org/~des/software/vm_zone-20010120.diff To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Jan 21 10:33:46 2001 Delivered-To: freebsd-smp@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 8BFB937B400 for ; Sun, 21 Jan 2001 10:33:29 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id TAA66544; Sun, 21 Jan 2001 19:33:28 +0100 (CET) (envelope-from des@ofug.org) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: smp@freebsd.org Subject: debug.witness_skipspin From: Dag-Erling Smorgrav Date: 21 Jan 2001 19:33:28 +0100 Message-ID: Lines: 7 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Is there a particular reason why this sysctl is read-only? Just for laughs, I built a kernel with a writeable debug.witness_skipspin, and experienced no ill effects from switching back and forth. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Jan 21 10:38:10 2001 Delivered-To: freebsd-smp@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id EB6F237B400 for ; Sun, 21 Jan 2001 10:37:51 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id TAA66567; Sun, 21 Jan 2001 19:37:51 +0100 (CET) (envelope-from des@ofug.org) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: smp@FreeBSD.ORG Subject: Re: debug.witness_skipspin References: From: Dag-Erling Smorgrav Date: 21 Jan 2001 19:37:50 +0100 In-Reply-To: Dag-Erling Smorgrav's message of "21 Jan 2001 19:33:28 +0100" Message-ID: Lines: 13 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Dag-Erling Smorgrav writes: > Is there a particular reason why this sysctl is read-only? Just for > laughs, I built a kernel with a writeable debug.witness_skipspin, and > experienced no ill effects from switching back and forth. Hey, I seem to have a thing with stupid questions today. Making it writeable doesn't make a difference because all the spin locks are already initialized (and enrolled or not) before you even get a chance to toggle it. Nothing beats reading the source... DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Jan 21 13:30: 5 2001 Delivered-To: freebsd-smp@freebsd.org Received: from moby.geekhouse.net (moby.geekhouse.net [64.81.6.36]) by hub.freebsd.org (Postfix) with ESMTP id B970937B402 for ; Sun, 21 Jan 2001 13:29:46 -0800 (PST) Received: from laptop.baldwin.cx (john@dhcp150.geekhouse.net [192.168.1.150]) by moby.geekhouse.net (8.11.0/8.9.3) with ESMTP id f0LLY8s78497; Sun, 21 Jan 2001 13:34:08 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Sun, 21 Jan 2001 13:29:41 -0800 (PST) From: John Baldwin To: Dag-Erling Smorgrav Subject: Re: debug.witness_skipspin Cc: smp@FreeBSD.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 21-Jan-01 Dag-Erling Smorgrav wrote: > Dag-Erling Smorgrav writes: >> Is there a particular reason why this sysctl is read-only? Just for >> laughs, I built a kernel with a writeable debug.witness_skipspin, and >> experienced no ill effects from switching back and forth. > > Hey, I seem to have a thing with stupid questions today. Making it > writeable doesn't make a difference because all the spin locks are > already initialized (and enrolled or not) before you even get a chance > to toggle it. Nothing beats reading the source... I think skipspin causes it to not do extra checking during enter/exit though. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Jan 21 13:30: 6 2001 Delivered-To: freebsd-smp@freebsd.org Received: from moby.geekhouse.net (moby.geekhouse.net [64.81.6.36]) by hub.freebsd.org (Postfix) with ESMTP id 9D87937B401 for ; Sun, 21 Jan 2001 13:29:46 -0800 (PST) Received: from laptop.baldwin.cx (john@dhcp150.geekhouse.net [192.168.1.150]) by moby.geekhouse.net (8.11.0/8.9.3) with ESMTP id f0LLY6s78493; Sun, 21 Jan 2001 13:34:07 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Sun, 21 Jan 2001 13:29:39 -0800 (PST) From: John Baldwin To: Dag-Erling Smorgrav Subject: RE: debug.witness_skipspin Cc: smp@FreeBSD.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 21-Jan-01 Dag-Erling Smorgrav wrote: > Is there a particular reason why this sysctl is read-only? Just for > laughs, I built a kernel with a writeable debug.witness_skipspin, and > experienced no ill effects from switching back and forth. Hmmm. I made it read only because you can get inconsistent state if, for example, witness is on when a lock is acquired but not when it is released. In that case, witness never notices that it is released and thinks the lock order is messed up when it isn't. Perhaps sysctl() is safe from this race however since spinlocks disable interrupts. > DES > -- > Dag-Erling Smorgrav - des@ofug.org -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Jan 21 13:43:54 2001 Delivered-To: freebsd-smp@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 18F4237B404; Sun, 21 Jan 2001 13:43:37 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id WAA67453; Sun, 21 Jan 2001 22:43:35 +0100 (CET) (envelope-from des@ofug.org) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: John Baldwin Cc: smp@FreeBSD.org Subject: Re: debug.witness_skipspin References: From: Dag-Erling Smorgrav Date: 21 Jan 2001 22:43:34 +0100 In-Reply-To: John Baldwin's message of "Sun, 21 Jan 2001 13:29:41 -0800 (PST)" Message-ID: Lines: 14 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org John Baldwin writes: > On 21-Jan-01 Dag-Erling Smorgrav wrote: > > Hey, I seem to have a thing with stupid questions today. Making it > > writeable doesn't make a difference because all the spin locks are > > already initialized (and enrolled or not) before you even get a chance > > to toggle it. Nothing beats reading the source... > I think skipspin causes it to not do extra checking during enter/exit though. Umm, I don't think so - the only code it affects is enroll(), which is only called from witness_init(), which is only called from mtx_init(). DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Jan 21 13:52:16 2001 Delivered-To: freebsd-smp@freebsd.org Received: from moby.geekhouse.net (moby.geekhouse.net [64.81.6.36]) by hub.freebsd.org (Postfix) with ESMTP id 8F4FD37B400 for ; Sun, 21 Jan 2001 13:51:58 -0800 (PST) Received: from laptop.baldwin.cx (john@dhcp150.geekhouse.net [192.168.1.150]) by moby.geekhouse.net (8.11.0/8.9.3) with ESMTP id f0LLuJs78537; Sun, 21 Jan 2001 13:56:20 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Sun, 21 Jan 2001 13:51:52 -0800 (PST) From: John Baldwin To: Dag-Erling Smorgrav Subject: Re: debug.witness_skipspin Cc: smp@FreeBSD.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 21-Jan-01 Dag-Erling Smorgrav wrote: > John Baldwin writes: >> On 21-Jan-01 Dag-Erling Smorgrav wrote: >> > Hey, I seem to have a thing with stupid questions today. Making it >> > writeable doesn't make a difference because all the spin locks are >> > already initialized (and enrolled or not) before you even get a chance >> > to toggle it. Nothing beats reading the source... >> I think skipspin causes it to not do extra checking during enter/exit >> though. > > Umm, I don't think so - the only code it affects is enroll(), which is > only called from witness_init(), which is only called from mtx_init(). Erm. Hmm, when I turned it off via ddb, I got it to stop panic'ing about lock reverals between sio and sched_lock when I had them backwards, and I did this from post boot. :) I wonder how that worked then... :-/ > DES > -- > Dag-Erling Smorgrav - des@ofug.org -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Jan 22 19: 5:46 2001 Delivered-To: freebsd-smp@freebsd.org Received: from luke.immure.com (luke.immure.com [207.8.42.74]) by hub.freebsd.org (Postfix) with ESMTP id 0C5EE37B402 for ; Mon, 22 Jan 2001 19:05:28 -0800 (PST) Received: (from bob@localhost) by luke.immure.com (8.11.1/8.11.1) id f0N35P662073; Mon, 22 Jan 2001 21:05:25 -0600 (CST) (envelope-from bob) Date: Mon, 22 Jan 2001 21:05:25 -0600 From: Bob Willcox To: John Gold Cc: FreeBSD SMP list Subject: Re: supermicro 370DL3 Message-ID: <20010122210524.A62027@luke.immure.com> Reply-To: Bob Willcox References: <01af01c08393$e56e0dc0$b8adfea9@kr8desktop> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <01af01c08393$e56e0dc0$b8adfea9@kr8desktop>; from Gold@kr8.com on Sun, Jan 21, 2001 at 10:21:18AM -0000 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Sun, Jan 21, 2001 at 10:21:18AM -0000, John Gold wrote: > Hi, > has anybody successfully used the Supermicro 370DL3 dual processor board under freebsd SMP? I am successfully running a Supermicro S2DL3 (Xeon cousin MB) with success (this is on 4.2-stable). Bob -- Bob Willcox The reason we come up with new versions is not to bob@vieo.com fix bugs. It's absolutely not. Austin, TX -- Bill Gates To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jan 23 7:42:55 2001 Delivered-To: freebsd-smp@freebsd.org Received: from mail.wolves.k12.mo.us (mail.wolves.k12.mo.us [207.160.214.1]) by hub.freebsd.org (Postfix) with ESMTP id 3910737B699 for ; Tue, 23 Jan 2001 07:42:35 -0800 (PST) Received: from mail.wolves.k12.mo.us (cdillon@mail.wolves.k12.mo.us [207.160.214.1]) by mail.wolves.k12.mo.us (8.9.3/8.9.3) with ESMTP id JAA42938; Tue, 23 Jan 2001 09:42:27 -0600 (CST) (envelope-from cdillon@wolves.k12.mo.us) Date: Tue, 23 Jan 2001 09:42:26 -0600 (CST) From: Chris Dillon To: Bob Willcox Cc: John Gold , FreeBSD SMP list Subject: Re: supermicro 370DL3 In-Reply-To: <20010122210524.A62027@luke.immure.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Mon, 22 Jan 2001, Bob Willcox wrote: > On Sun, Jan 21, 2001 at 10:21:18AM -0000, John Gold wrote: > > Hi, > > has anybody successfully used the Supermicro 370DL3 dual processor board under freebsd SMP? > > I am successfully running a Supermicro S2DL3 (Xeon cousin MB) with > success (this is on 4.2-stable). I'm using the 370DE6 with dual PIII-800's and 4.2-STABLE with no problems, yet (its only a few days old). -- Chris Dillon - cdillon@wolves.k12.mo.us - cdillon@inter-linc.net FreeBSD: The fastest and most stable server OS on the planet. For IA32 and Alpha architectures. IA64, PPC, and ARM under development. http://www.freebsd.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jan 23 15:29:21 2001 Delivered-To: freebsd-smp@freebsd.org Received: from panzer.kdm.org (panzer.kdm.org [216.160.178.169]) by hub.freebsd.org (Postfix) with ESMTP id A207F37B400 for ; Tue, 23 Jan 2001 15:29:01 -0800 (PST) Received: (from ken@localhost) by panzer.kdm.org (8.9.3/8.9.1) id QAA93302; Tue, 23 Jan 2001 16:28:52 -0700 (MST) (envelope-from ken) Date: Tue, 23 Jan 2001 16:28:52 -0700 From: "Kenneth D. Merry" To: Chris Dillon Cc: Bob Willcox , John Gold , FreeBSD SMP list Subject: Re: supermicro 370DL3 Message-ID: <20010123162852.A93268@panzer.kdm.org> References: <20010122210524.A62027@luke.immure.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: ; from cdillon@wolves.k12.mo.us on Tue, Jan 23, 2001 at 09:42:26AM -0600 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Tue, Jan 23, 2001 at 09:42:26 -0600, Chris Dillon wrote: > On Mon, 22 Jan 2001, Bob Willcox wrote: > > > On Sun, Jan 21, 2001 at 10:21:18AM -0000, John Gold wrote: > > > Hi, > > > has anybody successfully used the Supermicro 370DL3 dual processor board under freebsd SMP? > > > > I am successfully running a Supermicro S2DL3 (Xeon cousin MB) with > > success (this is on 4.2-stable). > > I'm using the 370DE6 with dual PIII-800's and 4.2-STABLE with no > problems, yet (its only a few days old). Just out of curiosity, where did you get the board? Ken -- Kenneth Merry ken@kdm.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jan 23 17:33:39 2001 Delivered-To: freebsd-smp@freebsd.org Received: from mail.wolves.k12.mo.us (mail.wolves.k12.mo.us [207.160.214.1]) by hub.freebsd.org (Postfix) with ESMTP id 5F7C937B6BD for ; Tue, 23 Jan 2001 17:33:22 -0800 (PST) Received: from mail.wolves.k12.mo.us (cdillon@mail.wolves.k12.mo.us [207.160.214.1]) by mail.wolves.k12.mo.us (8.9.3/8.9.3) with ESMTP id TAA53379; Tue, 23 Jan 2001 19:33:17 -0600 (CST) (envelope-from cdillon@wolves.k12.mo.us) Date: Tue, 23 Jan 2001 19:33:16 -0600 (CST) From: Chris Dillon To: "Kenneth D. Merry" Cc: Bob Willcox , John Gold , FreeBSD SMP list Subject: Re: supermicro 370DL3 In-Reply-To: <20010123162852.A93268@panzer.kdm.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Tue, 23 Jan 2001, Kenneth D. Merry wrote: > On Tue, Jan 23, 2001 at 09:42:26 -0600, Chris Dillon wrote: > > On Mon, 22 Jan 2001, Bob Willcox wrote: > > > > > On Sun, Jan 21, 2001 at 10:21:18AM -0000, John Gold wrote: > > > > Hi, > > > > has anybody successfully used the Supermicro 370DL3 dual processor board under freebsd SMP? > > > > > > I am successfully running a Supermicro S2DL3 (Xeon cousin MB) with > > > success (this is on 4.2-stable). > > > > I'm using the 370DE6 with dual PIII-800's and 4.2-STABLE with no > > problems, yet (its only a few days old). > > Just out of curiosity, where did you get the board? It came in a bundled SuperMicro SuperServer 6040 system (http://www.supermicro.com/PRODUCT/SUPERServer/SUPER%20SERVER%206040%20Server.htm). The only thing you need to add to those bundled systems are the Socket 370 processors, the memory, and the SCA drives. It looked good, so I thought I'd give it a try. Pretty nice system for the money. Put it together with dual PIII-800's, 512MB PC133 ECC Registered SDRAM (thats the only kind it will take), a Mylex AcceleRAID 170 w/32MB, and three IBM 36LZX 10KRPM Ultra160 18.2GB HDs for about $4300. I just put 4.2-STABLE on it about a week ago. I haven't had time to do much with it, but it has survived a couple of new kernels and a couple of 'make -j 8 world's. The only thing I've noticed wrong is this: APIC_IO: Testing 8254 interrupt delivery APIC_IO: Broken MP table detected: 8254 is not connected to IOAPIC #0 intpin 2 APIC_IO: routing 8254 via 8259 and IOAPIC #0 intpin 0 and occasionally a few of these when under moderate disk load (disks are on the AcceleRAID 170, which is on the 64/66 PCI bus running at only 32/33): /kernel: stray irq 7 but even then, everything seems to work fine. -- Chris Dillon - cdillon@wolves.k12.mo.us - cdillon@inter-linc.net FreeBSD: The fastest and most stable server OS on the planet. For IA32 and Alpha architectures. IA64, PPC, and ARM under development. http://www.freebsd.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jan 23 22: 4:23 2001 Delivered-To: freebsd-smp@freebsd.org Received: from onizuka.vmunix.org (onizuka.vmunix.org [194.221.152.19]) by hub.freebsd.org (Postfix) with ESMTP id A85EA37B401 for ; Tue, 23 Jan 2001 22:04:05 -0800 (PST) Received: from localhost (2410 bytes) by onizuka.vmunix.org via sendmail with stdio (sender: ) (ident using unix) id for ; Wed, 24 Jan 2001 07:04:03 +0100 (CET) Message-Id: From: torstenb@vmunix.org (Torsten Blum) Subject: Re: supermicro 370DL3 In-Reply-To: <20010122210524.A62027@luke.immure.com> "from Bob Willcox at Jan 22, 2001 09:05:25 pm" To: Bob Willcox Date: Wed, 24 Jan 2001 07:04:03 +0100 (CET) Cc: John Gold , FreeBSD SMP list X-Mailer: ELM [version 2.4ME+ PL82 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Bob Willcox wrote: Hm, John's mail never arrived here... > On Sun, Jan 21, 2001 at 10:21:18AM -0000, John Gold wrote: > > Hi, > > has anybody successfully used the Supermicro 370DL3 dual processor board under freebsd SMP? > > I am successfully running a Supermicro S2DL3 (Xeon cousin MB) with > success (this is on 4.2-stable). I've got a Supermicro 370DL3 running a two weeks old RELENG_4: FreeBSD 4.2-STABLE #0: Tue Jan 9 11:02:37 GMT 2001 torstenb@rutledge.de.cw.net:/usr/src/sys/compile/RUTLEDGE CPU: Pentium III/Pentium III Xeon/Celeron (866.25-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x686 Stepping = 6 Features=0x383fbff real memory = 2147483648 (2097152K bytes) avail memory = 2088251392 (2039308K bytes) Programming 16 pins in IOAPIC #0 IOAPIC #0 intpin 2 -> irq 0 Programming 16 pins in IOAPIC #1 IOAPIC #1 intpin 0 -> irq 2 IOAPIC #1 intpin 4 -> irq 15 IOAPIC #1 intpin 12 -> irq 9 IOAPIC #1 intpin 15 -> irq 11 FreeBSD/SMP: Multiprocessor motherboard cpu0 (BSP): apic id: 0, version: 0x00040011, at 0xfee00000 cpu1 (AP): apic id: 1, version: 0x00040011, at 0xfee00000 io0 (APIC): apic id: 4, version: 0x000f0011, at 0xfec00000 io1 (APIC): apic id: 5, version: 0x000f0011, at 0xfec01000 running 4.2-RELEASE reboots initiated by the machine itself (shutdown, ie SIGINT to init) did not work always. In most cases the machine when down displayed the "Rebooting Now" text but did not really do it. With 4.2-stable it works. One thing that doesnt is APM in SMP mode. During but the kernel hangs before it wants to start init. Since APM is disabled by default and I'm using it as a server it's not a dig deal. However, since lack of APM also means that I can't soft-poweroff the machine I'd like to take a closer look soon. (If I can get the time ...) -tb To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jan 24 1:17: 3 2001 Delivered-To: freebsd-smp@freebsd.org Received: from mail.west.se (unknown [194.52.130.37]) by hub.freebsd.org (Postfix) with ESMTP id D69E137B404 for ; Wed, 24 Jan 2001 01:16:44 -0800 (PST) Received: from grumpy.west.se (IDENT:ema@grumpy [192.168.10.11]) by mail.west.se (8.11.0/8.9.3) with ESMTP id f0O9GVd04053 for ; Wed, 24 Jan 2001 10:16:33 +0100 (CET) (envelope-from erikmattsson@home.se) Date: Wed, 24 Jan 2001 10:16:30 +0100 (CET) From: Erik Mattsson X-Sender: Reply-To: To: Subject: Bug (?) with sockets/pthreads/freebsd4.2-stable Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Im having problem using threads and sockets on a freebsd smp box. Ive mailed freebsd-questions without any reply, and I wonder if this is a bug in freebsd/smp/pthreads ? (I was told on a newsgroup that this behaviour might be a bug in the OS) The thing is that I've executed the signal(SIGPIPE, SIG_IGN) but still get SIGPIPE's thrown at the program. This shouldnt be possible should it ? Im sending the complete mail (that I posted on questions) --------------------------------------------------------------- A little background: A server listens on sockets (each socket is placed in a thread), the server replies to the incoming data and send the info over the same socket. (and the same thread) But sometimes it also sends data over sockets in another thread, and it seems that when this happens the program 'hangs'. ( it happens every one of 10 times) ((relaying data from another socket)) Ive debugged my program and found that the thread hangs on the send() method. It never finishs and because of that some mutex's are still locked the whole server hangs. But when Im using gdb, I find that the program receives a SIGPIPE even though Ive told the program to ignore it; signal(SIGPIPE, SIG_IGN). Is it necessary to run this signal() in every thread, or does it suffice once in the process ? (ive tried both approaches, with no luck) I'm wondering : if it is possible to send() and recv() on a socket at the same time in different threads ? if it is possible to get SIGPIPE's even though you executed signal(SIGPIPE, SIG_IGN) in every thread ? if it is possible that a send() call can hang a thread ? Could anyone shed some light on this ? Im using pthreads, freebsd4.2 stable, SMP, dual 600 mhz, 512 mb. (what other info do you need ?) Any help is appreciated ! Erik Mattsson ps. It would be nice if you could bcc my mail address since Im not subscribed to this list..(erikmattsson@home.se) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jan 24 1:35:35 2001 Delivered-To: freebsd-smp@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 179F737B400 for ; Wed, 24 Jan 2001 01:35:19 -0800 (PST) Received: (qmail 50486 invoked by uid 1142); 24 Jan 2001 09:35:18 -0000 Date: 24 Jan 2001 01:35:18 -0800 Date: Wed, 24 Jan 2001 01:35:07 -0800 From: Jason Evans To: ema@west.se Cc: freebsd-smp@FreeBSD.ORG Subject: Re: Bug (?) with sockets/pthreads/freebsd4.2-stable Message-ID: <20010124013507.K69199@canonware.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from erikmattsson@home.se on Wed, Jan 24, 2001 at 10:16:30AM +0100 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Wed, Jan 24, 2001 at 10:16:30AM +0100, Erik Mattsson wrote: > Im having problem using threads and sockets on a freebsd smp box. > > Ive mailed freebsd-questions without any reply, and I wonder > if this is a bug in freebsd/smp/pthreads ? > (I was told on a newsgroup that this behaviour might be a bug in the OS) > > The thing is that I've executed the signal(SIGPIPE, SIG_IGN) but > still get SIGPIPE's thrown at the program. This shouldnt be possible > should it ? Here's a snippet of code from a library I wrote that does precisely what you're doing: /* * Ignore SIGPIPE, so that writing to a closed socket * won't crash the program. */ signal(SIGPIPE, SIG_IGN); This code is executed by the initial thread. It works for me... Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jan 24 7: 6: 3 2001 Delivered-To: freebsd-smp@freebsd.org Received: from mail.west.se (unknown [194.52.130.37]) by hub.freebsd.org (Postfix) with ESMTP id 046BA37B404 for ; Wed, 24 Jan 2001 07:05:45 -0800 (PST) Received: from grumpy.west.se (IDENT:ema@grumpy [192.168.10.11]) by mail.west.se (8.11.0/8.9.3) with ESMTP id f0OF5Yd06856; Wed, 24 Jan 2001 16:05:35 +0100 (CET) (envelope-from ema@west.se) Date: Wed, 24 Jan 2001 16:05:33 +0100 (CET) From: Reply-To: To: Jason Evans Cc: Subject: Re: Bug (?) with sockets/pthreads/freebsd4.2-stable In-Reply-To: <20010124013507.K69199@canonware.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Wed, 24 Jan 2001, Jason Evans wrote: > Here's a snippet of code from a library I wrote that does precisely what > you're doing: > > /* > * Ignore SIGPIPE, so that writing to a closed socket > * won't crash the program. > */ > signal(SIGPIPE, SIG_IGN); > > This code is executed by the initial thread. > > It works for me... Unfortunately it doesnt for me, as Ive littered the code with signal(..) before it hangs but it still throws a SIGPIPE at the program. Ive tried executing in each pthread's method and in the main(). With no success. I had no problem with the SIGPIPE signal when using it in one process, but now when with the pthreads, they still missbehave. //Erik To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jan 24 14: 6:38 2001 Delivered-To: freebsd-smp@freebsd.org Received: from panzer.kdm.org (panzer.kdm.org [216.160.178.169]) by hub.freebsd.org (Postfix) with ESMTP id E569F37B401 for ; Wed, 24 Jan 2001 14:06:15 -0800 (PST) Received: (from ken@localhost) by panzer.kdm.org (8.9.3/8.9.1) id PAA02265; Wed, 24 Jan 2001 15:05:31 -0700 (MST) (envelope-from ken) Date: Wed, 24 Jan 2001 15:05:31 -0700 From: "Kenneth D. Merry" To: Chris Dillon Cc: Bob Willcox , John Gold , FreeBSD SMP list Subject: Re: supermicro 370DL3 Message-ID: <20010124150530.A2082@panzer.kdm.org> References: <20010123162852.A93268@panzer.kdm.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: ; from cdillon@wolves.k12.mo.us on Tue, Jan 23, 2001 at 07:33:16PM -0600 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Tue, Jan 23, 2001 at 19:33:16 -0600, Chris Dillon wrote: > On Tue, 23 Jan 2001, Kenneth D. Merry wrote: > > On Tue, Jan 23, 2001 at 09:42:26 -0600, Chris Dillon wrote: > > > On Mon, 22 Jan 2001, Bob Willcox wrote: > > > > > > > On Sun, Jan 21, 2001 at 10:21:18AM -0000, John Gold wrote: > > > > > Hi, > > > > > has anybody successfully used the Supermicro 370DL3 dual processor board under freebsd SMP? > > > > > > > > I am successfully running a Supermicro S2DL3 (Xeon cousin MB) with > > > > success (this is on 4.2-stable). > > > > > > I'm using the 370DE6 with dual PIII-800's and 4.2-STABLE with no > > > problems, yet (its only a few days old). > > > > Just out of curiosity, where did you get the board? > > It came in a bundled SuperMicro SuperServer 6040 system > (http://www.supermicro.com/PRODUCT/SUPERServer/SUPER%20SERVER%206040%20Server.htm). > The only thing you need to add to those bundled systems are the Socket > 370 processors, the memory, and the SCA drives. It looked good, so I > thought I'd give it a try. Pretty nice system for the money. Put it > together with dual PIII-800's, 512MB PC133 ECC Registered SDRAM (thats > the only kind it will take), a Mylex AcceleRAID 170 w/32MB, and three > IBM 36LZX 10KRPM Ultra160 18.2GB HDs for about $4300. Nice system. Where did you get the server system? Most of the places I've seen advertising the 370DE6 don't give you any clear indication of whether it's in stock or not. The one place that did (www.microx-press.com) doesn't have them in stock. > I just put 4.2-STABLE on it about a week ago. I haven't had time to > do much with it, but it has survived a couple of new kernels and a > couple of 'make -j 8 world's. The only thing I've noticed wrong is > this: > > APIC_IO: Testing 8254 interrupt delivery > APIC_IO: Broken MP table detected: 8254 is not connected to IOAPIC #0 intpin 2 > APIC_IO: routing 8254 via 8259 and IOAPIC #0 intpin 0 > > and occasionally a few of these when under moderate disk load (disks > are on the AcceleRAID 170, which is on the 64/66 PCI bus running at > only 32/33): > > /kernel: stray irq 7 > > but even then, everything seems to work fine. Yeah, there was a discussion about the 'stray irq 7' messages on -hackers last week. Ken -- Kenneth Merry ken@kdm.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jan 24 14:18:57 2001 Delivered-To: freebsd-smp@freebsd.org Received: from mail.wolves.k12.mo.us (mail.wolves.k12.mo.us [207.160.214.1]) by hub.freebsd.org (Postfix) with ESMTP id 21CC037B698 for ; Wed, 24 Jan 2001 14:18:40 -0800 (PST) Received: from mail.wolves.k12.mo.us (cdillon@mail.wolves.k12.mo.us [207.160.214.1]) by mail.wolves.k12.mo.us (8.9.3/8.9.3) with ESMTP id QAA71808; Wed, 24 Jan 2001 16:18:34 -0600 (CST) (envelope-from cdillon@wolves.k12.mo.us) Date: Wed, 24 Jan 2001 16:18:34 -0600 (CST) From: Chris Dillon To: "Kenneth D. Merry" Cc: Bob Willcox , John Gold , FreeBSD SMP list Subject: Re: supermicro 370DL3 In-Reply-To: <20010124150530.A2082@panzer.kdm.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Wed, 24 Jan 2001, Kenneth D. Merry wrote: > > It came in a bundled SuperMicro SuperServer 6040 system > > (http://www.supermicro.com/PRODUCT/SUPERServer/SUPER%20SERVER%206040%20Server.htm). > > The only thing you need to add to those bundled systems are the Socket > > 370 processors, the memory, and the SCA drives. It looked good, so I > > thought I'd give it a try. Pretty nice system for the money. Put it > > together with dual PIII-800's, 512MB PC133 ECC Registered SDRAM (thats > > the only kind it will take), a Mylex AcceleRAID 170 w/32MB, and three > > IBM 36LZX 10KRPM Ultra160 18.2GB HDs for about $4300. > > Nice system. Where did you get the server system? From CDW. http://www.cdw.com > Most of the places I've seen advertising the 370DE6 don't give you > any clear indication of whether it's in stock or not. The one > place that did (www.microx-press.com) doesn't have them in stock. We had to wait a couple of weeks to get this, since apparently it was backordered, or maybe Supermicro hadn't even started shipping the boards until recently, even though they had them up on their website and CDW had the part number for it in their database weeks ago. -- Chris Dillon - cdillon@wolves.k12.mo.us - cdillon@inter-linc.net FreeBSD: The fastest and most stable server OS on the planet. For IA32 and Alpha architectures. IA64, PPC, and ARM under development. http://www.freebsd.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jan 24 17: 5: 9 2001 Delivered-To: freebsd-smp@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 4C60337B400 for ; Wed, 24 Jan 2001 17:04:44 -0800 (PST) Received: (qmail 85027 invoked by uid 1142); 25 Jan 2001 01:04:43 -0000 Date: 24 Jan 2001 17:04:43 -0800 Date: Wed, 24 Jan 2001 17:04:07 -0800 From: Jason Evans To: smp@freebsd.org Cc: hackers@freebsd.org Subject: SMP Project Status (24 January 2001) Message-ID: <20010124170407.B87569@canonware.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org A lot of good progress has been made on the SMP project in the past couple of weeks. We are on the verge of being able to move portions of the kernel out of under the Giant lock, which will be the first improvement in performance due to our work, following a long string of developments over the past 6 months that actually (temporarily) decreased performance. John Baldwin is in the process of committing changes that lock accesses to the proc structure. This allows a number of system calls to run without holding Giant, as well as most signal handling. Additionally, a number of other system calls can be made MP-safe with only a moderate amount of additional effort. Jake Burkholder is in the late stages of testing changes that will make the kernel preemptive. As mentioned in the last status report, a number of other tasks hinge on this development. Bosko Milekic has made some important cleanups to mutexes with regard to recursion, and is in the process of completely revamping the mutex API and code structure (converting to non-inlined functions). Dag-Erling Smorgrav has joined the fray and made the zone allocator MP-safe. He has also expressed interest in doing further work locking the VM. Peter Wemm has been a huge help testing and debugging changes with at least John, Jake, and me. I finally committed Jake's condition variable implementation, and completely removed simplelocks from the kernel. Hopefully the sx lock implementation will be committed soon as well. The SMP project page is still at: http://people.freebsd.org/~jasone/smp/ The previous status email can be found at: http://www.FreeBSD.org/cgi/getmsg.cgi?fetch=45111+48223+/usr/local/www/db/text/2001/freebsd-smp/20010114.freebsd-smp To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message