From owner-freebsd-smp Wed Jun 9 15:24:41 1999 Delivered-To: freebsd-smp@freebsd.org Received: from par28.ma.ikos.com (par28.ma.ikos.com [137.103.105.228]) by hub.freebsd.org (Postfix) with ESMTP id 0813714DBB; Wed, 9 Jun 1999 15:24:37 -0700 (PDT) (envelope-from tich@par28.ma.ikos.com) Received: (from tich@localhost) by par28.ma.ikos.com (8.8.7/8.8.7) id SAA01166; Wed, 9 Jun 1999 18:25:05 -0400 Date: Wed, 9 Jun 1999 18:25:05 -0400 From: Richard Cownie Message-Id: <199906092225.SAA01166@par28.ma.ikos.com> To: freebsd-current@freebsd.org, freebsd-smp@freebsd.org Subject: 4-way SMP broken ? Cc: tich@ma.ikos.com Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi, I've been trying to install 19990604-CURRENT on a couple of SC450NX boxes. It works fine with 2 cpu's, but an SMP kernel with 4 cpu's falls over very quickly (I think while it's setting up the APIC stuff, or very shortly after - the messages about APIC bus ids appear on the screen very briefly, then the machine reboots itself). Does anyone know a) when was the last time it worked on 4 cpu's b) what's changed recently which might relate to this. Also in trying to figure this out I looked at the DRAM probing code in /usr/src/sys/i386/i386/machdep.c:getmemsize(), and it looks as though it's not safe for >2GB (e.g. comparisons of byte addresses against signed "int end"). It would also be good if this probing code was carefule not to ventrue past 4GB-64MB (PCI device space) - then a generic kernel could work on a 4GB machine without any tweaking, which would simplify installation - I get nervous shuffling DIMMs in and out of the machine ... Thanks Richard Cownie (tich@ma.ikos.com) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 9 17: 1:10 1999 Delivered-To: freebsd-smp@freebsd.org Received: from lor.watermarkgroup.com (lor.watermarkgroup.com [207.202.73.33]) by hub.freebsd.org (Postfix) with ESMTP id 3AECF14F52; Wed, 9 Jun 1999 17:01:05 -0700 (PDT) (envelope-from luoqi@watermarkgroup.com) Received: (from luoqi@localhost) by lor.watermarkgroup.com (8.8.8/8.8.8) id UAA07014; Wed, 9 Jun 1999 20:01:04 -0400 (EDT) (envelope-from luoqi) Date: Wed, 9 Jun 1999 20:01:04 -0400 (EDT) From: Luoqi Chen Message-Id: <199906100001.UAA07014@lor.watermarkgroup.com> To: freebsd-current@FreeBSD.ORG, freebsd-smp@FreeBSD.ORG, tich@ma.ikos.com Subject: Re: 4-way SMP broken ? Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Hi, > > I've been trying to install 19990604-CURRENT on a couple of SC450NX > boxes. It works fine with 2 cpu's, but an SMP kernel with 4 cpu's > falls over very quickly (I think while it's setting up the APIC > stuff, or very shortly after - the messages about APIC bus ids appear > on the screen very briefly, then the machine reboots itself). > Do you mean messages like these? FreeBSD/SMP: Multiprocessor motherboard cpu0 (BSP): apic id: 0, version: 0x00040011, at 0xfec08000 cpu1 (AP): apic id: 12, version: 0x00040011, at 0xfec08000 io0 (APIC): apic id: 13, version: 0x00170011, at 0xfec00000 By the time you see these messages, all cpus should have been booted up successfully, any crash immediately follows is not likely to be SMP related. It's helpful to pinpoint the crash if you could include the last few lines from a verbose boot. > Does anyone know a) when was the last time it worked on 4 cpu's > b) what's changed recently which might relate to this. > > Also in trying to figure this out I looked at the DRAM probing > code in /usr/src/sys/i386/i386/machdep.c:getmemsize(), and it looks > as though it's not safe for >2GB (e.g. comparisons of byte addresses > against signed "int end"). It would also be good if this probing > code was carefule not to ventrue past 4GB-64MB (PCI device space) - > then a generic kernel could work on a 4GB machine without any tweaking, > which would simplify installation - I get nervous shuffling DIMMs > in and out of the machine ... > > Thanks > Richard Cownie (tich@ma.ikos.com) > -lq To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Jun 10 11:58:36 1999 Delivered-To: freebsd-smp@freebsd.org Received: from par28.ma.ikos.com (par28.ma.ikos.com [137.103.105.228]) by hub.freebsd.org (Postfix) with ESMTP id 7FED01509D; Thu, 10 Jun 1999 11:58:32 -0700 (PDT) (envelope-from tich@par28.ma.ikos.com) Received: from [[UNIX: localhost]] ([[UNIX: localhost]]) by par28.ma.ikos.com (8.8.7/8.8.7) id OAA19650; Thu, 10 Jun 1999 14:58:22 -0400 From: Richard Cownie To: Luoqi Chen , freebsd-current@FreeBSD.ORG, freebsd-smp@FreeBSD.ORG, tich@ma.ikos.com Subject: Re: 4-way SMP broken ? Date: Thu, 10 Jun 1999 14:47:01 -0400 X-Mailer: KMail [version 1.1.0] Content-Type: text/plain References: <199906100001.UAA07014@lor.watermarkgroup.com> MIME-Version: 1.0 Message-Id: <99061014582200.19512@par28.ma.ikos.com> Content-Transfer-Encoding: 8bit X-KMail-Mark: Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Wed, 09 Jun 1999, Luoqi Chen wrote: > > I've been trying to install 19990604-CURRENT on a couple of SC450NX > > boxes. It works fine with 2 cpu's, but an SMP kernel with 4 cpu's > > falls over very quickly (I think while it's setting up the APIC > > stuff, or very shortly after - the messages about APIC bus ids appear > > on the screen very briefly, then the machine reboots itself). > > > Do you mean messages like these? > FreeBSD/SMP: Multiprocessor motherboard > cpu0 (BSP): apic id: 0, version: 0x00040011, at 0xfec08000 > cpu1 (AP): apic id: 12, version: 0x00040011, at 0xfec08000 > io0 (APIC): apic id: 13, version: 0x00170011, at 0xfec00000 > By the time you see these messages, all cpus should have been booted up > successfully, any crash immediately follows is not likely to be SMP related. > It's helpful to pinpoint the crash if you could include the last few lines > from a verbose boot. I have added more debugging messages, and the crash appears to be inside mp_start(). I don't have a log because this is too early in the boot to get the messages saved anywhere, and they go by too quickly to write it down. The evidence that this is an SMP problem is simple - with 2 cpu's plugged in, it works fine; with 3 or 4 cpu's plugged in, it crashes. I believe the hardware is fine because I was previously running 19990421-CURRENT with all 4 cpu's without serious problems (it was a little unstable, but always booted ok). > > Does anyone know a) when was the last time it worked on 4 cpu's > > b) what's changed recently which might relate to this. So if anyone has an answer to these questions I'd still be interested. > > Also in trying to figure this out I looked at the DRAM probing > > code in /usr/src/sys/i386/i386/machdep.c:getmemsize(), and it looks > > as though it's not safe for >2GB (e.g. comparisons of byte addresses > > against signed "int end"). It would also be good if this probing I've tried various hacks to this code, but have not succeeded in making it work for 4GB. Changing "int end" to "vm_offset_t end" is not sufficient. It has a tendency to say "Too many holes in address space" ... Even defining MAXMEM does not solve the problem. Richard Cownie (tich@ma.ikos.com) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Jun 10 14:56: 4 1999 Delivered-To: freebsd-smp@freebsd.org Received: from lor.watermarkgroup.com (lor.watermarkgroup.com [207.202.73.33]) by hub.freebsd.org (Postfix) with ESMTP id 22E391506D; Thu, 10 Jun 1999 14:56:01 -0700 (PDT) (envelope-from luoqi@watermarkgroup.com) Received: (from luoqi@localhost) by lor.watermarkgroup.com (8.8.8/8.8.8) id RAA15728; Thu, 10 Jun 1999 17:56:01 -0400 (EDT) (envelope-from luoqi) Date: Thu, 10 Jun 1999 17:56:01 -0400 (EDT) From: Luoqi Chen Message-Id: <199906102156.RAA15728@lor.watermarkgroup.com> To: tich@ma.ikos.com Subject: Re: 4-way SMP broken ? Cc: freebsd-current@FreeBSD.ORG, freebsd-smp@FreeBSD.ORG Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > I have added more debugging messages, and the crash appears to be inside > mp_start(). I don't have a log because this is too early in the boot > to get the messages saved anywhere, and they go by too quickly to > write it down. The evidence that this is an SMP problem is simple - > with 2 cpu's plugged in, it works fine; with 3 or 4 cpu's plugged in, > it crashes. > Could you narrow down the crash further inside mp_start()? I'd like to know whether the crash occurred inside start_all_aps(). One or two lines of debug messages would be really helpful, you don't have to write down the exact words. Do you have options DDB enabled in the kernel? It helps to stop the last few lines of console messages to scroll of the screen. > I believe the hardware is fine because I was previously running > 19990421-CURRENT with all 4 cpu's without serious problems (it was > a little unstable, but always booted ok). > If possible, could you try a kernel built from sources with the POST_SMP_VMSHARE tag? I may have broken something during the commit. -lq To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Jun 10 15:36:29 1999 Delivered-To: freebsd-smp@freebsd.org Received: from par28.ma.ikos.com (par28.ma.ikos.com [137.103.105.228]) by hub.freebsd.org (Postfix) with ESMTP id 8268F1501A; Thu, 10 Jun 1999 15:36:21 -0700 (PDT) (envelope-from tich@par28.ma.ikos.com) Received: from [[UNIX: localhost]] ([[UNIX: localhost]]) by par28.ma.ikos.com (8.8.7/8.8.7) id SAA22847; Thu, 10 Jun 1999 18:36:18 -0400 From: Richard Cownie To: Luoqi Chen , tich@ma.ikos.com Subject: Re: 4-way SMP broken ? Date: Thu, 10 Jun 1999 18:33:25 -0400 X-Mailer: KMail [version 1.1.0] Content-Type: text/plain Cc: freebsd-current@FreeBSD.ORG, freebsd-smp@FreeBSD.ORG References: <199906102156.RAA15728@lor.watermarkgroup.com> MIME-Version: 1.0 Message-Id: <99061018361800.22794@par28.ma.ikos.com> Content-Transfer-Encoding: 8bit X-KMail-Mark: Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Thu, 10 Jun 1999, Luoqi Chen wrote: > Could you narrow down the crash further inside mp_start()? I'd like to > know whether the crash occurred inside start_all_aps(). One or two lines of > debug messages would be really helpful, you don't have to write down the exact > words. Do you have options DDB enabled in the kernel? It helps to stop > the last few lines of console messages to scroll of the screen. Yes, I added more messages and it's inside start_all_aps() - it seems to start AP #1 ok, then crashes while starting AP #2. > If possible, could you try a kernel built from sources with the > POST_SMP_VMSHARE tag? I may have broken something during the commit. Have to get out the door right now, will try this either tomorrow morning or Monday. Thanks Richard Cownie To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Jun 11 8: 0:43 1999 Delivered-To: freebsd-smp@freebsd.org Received: from leap.innerx.net (leap.innerx.net [38.179.176.25]) by hub.freebsd.org (Postfix) with ESMTP id 3301D151CB; Fri, 11 Jun 1999 08:00:38 -0700 (PDT) (envelope-from chris@holly.dyndns.org) Received: from holly.dyndns.org (ip126.houston14.tx.pub-ip.psi.net [38.27.214.126]) by leap.innerx.net (Postfix) with ESMTP id 930E3370B9; Fri, 11 Jun 1999 11:00:31 -0400 (EDT) Received: (from chris@localhost) by holly.dyndns.org (8.9.3/8.9.3) id JAA96358; Fri, 11 Jun 1999 09:59:30 -0500 (CDT) (envelope-from chris) Date: Fri, 11 Jun 1999 09:59:25 -0500 From: Chris Costello To: "David E. Cross" Cc: freebsd-hackers@FreeBSD.ORG, freebsd-smp@FreeBSD.ORG Subject: Re: High syscall overhead? Message-ID: <19990611095925.V57174@holly.dyndns.org> Reply-To: chris@calldei.com References: <199906111440.KAA70517@cs.rpi.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii User-Agent: Mutt/0.96.3i In-Reply-To: <199906111440.KAA70517@cs.rpi.edu>; from David E. Cross on Fri, Jun 11, 1999 at 10:40:37AM -0400 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Fri, Jun 11, 1999, David E. Cross wrote: > Just doing some performance testing and I noticed something rather > disturbing.... > > Here is the test program: > int main (void) > { > int count=0; > for(count=0;count <10000000;++count) > getppid(); > > return 0; > } > > The time on linux for this program is ~5 seconds (linux "time" reports 3.x, but > a wall clock clearly shows 5.x, go fig). FreeBSD reports 18.x seconds?!. I > have a dual processor system and decided to parallel run them... it took > 52!?! seconds, linux on the same was again about 5. Looking through the > exception.s it appears that on entry to the kernel an MP lock is obtained... > I thought we had splX(); to protect concurancy in the kernel. ('sc' being the program above, compiled without optimization, just with cc -o sc sc.c) $ time ./sc 10.04s real 3.89s user 5.64s system I counted between around 9 and a half to 10 and a half seconds on my wall clock (trusty old GE, same model they have in public schools). Copyright (c) 1992-1999 The FreeBSD Project. Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All rights reserved. FreeBSD 4.0-CURRENT #4: Sun May 30 04:22:23 CDT 1999 root@holly.dyndns.org:/usr/src/sys/compile/Holly Timecounter "i8254" frequency 1193182 Hz CPU: AMD-K6(tm) 3D processor (350.80-MHz 586-class CPU) Origin = "AuthenticAMD" Id = 0x580 Stepping=0 Features=0x8001bf real memory = 67108864 (65536K bytes) sio0: system console avail memory = 62267392 (60808K bytes) SMP specific bug, perhaps? > > I am just curious what's the story with this. On some of my other tests it is > clear that FreeBSD is handling concurancy much better than linux (by an equal > factor actually, and on "real" tasks like real I/O handling). > > -- > David Cross | email: crossd@cs.rpi.edu > Systems Administrator/Research Programmer | Web: http://www.cs.rpi.edu/~crossd > Rensselaer Polytechnic Institute, | Ph: 518.276.2860 > Department of Computer Science | Fax: 518.276.4033 > I speak only for myself. | WinNT:Linux::Linux:FreeBSD > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message -- Chris Costello This message transmitted on 100% recycled electrons. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Jun 11 8:28:21 1999 Delivered-To: freebsd-smp@freebsd.org Received: from cs.rpi.edu (mumble.cs.rpi.edu [128.213.8.16]) by hub.freebsd.org (Postfix) with ESMTP id 0F5671526E; Fri, 11 Jun 1999 08:28:17 -0700 (PDT) (envelope-from crossd@cs.rpi.edu) Received: from cs.rpi.edu (monica.cs.rpi.edu [128.213.7.2]) by cs.rpi.edu (8.9.3/8.9.3) with ESMTP id LAA71507; Fri, 11 Jun 1999 11:28:10 -0400 (EDT) Message-Id: <199906111528.LAA71507@cs.rpi.edu> To: chris@calldei.com Cc: "David E. Cross" , freebsd-hackers@FreeBSD.ORG, freebsd-smp@FreeBSD.ORG, crossd@cs.rpi.edu Subject: Re: High syscall overhead? In-Reply-To: Message from Chris Costello of "Fri, 11 Jun 1999 09:59:25 CDT." <19990611095925.V57174@holly.dyndns.org> Date: Fri, 11 Jun 1999 11:28:09 -0400 From: "David E. Cross" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Oops, here is some additional information from my system: bash-2.02$ cat sc.c int main (void) { int count=0; for(count=0;count <10000000;++count) getppid(); return 0; } bash-2.02$ cc -o sc sc.c bash-2.02$ uptime 11:19AM up 3 hrs, 4 users, load averages: 0.01, 0.01, 0.00 bash-2.02$ time ./sc real 0m18.523s user 0m0.000s sys 0m18.521s bash-2.02$ (date;time ./sc;date) & (date;time ./sc;date) & [3] 516 [4] 517 Fri Jun 11 11:21:51 EDT 1999 [1] Done time ./sc Fri Jun 11 11:21:51 EDT 1999 [2] Done time ./sc bash-2.02$ real 0m52.058s user 0m0.000s sys 0m52.056s real 0m52.056s user 0m0.000s sys 0m52.053s Fri Jun 11 11:22:43 EDT 1999 Fri Jun 11 11:22:43 EDT 1999 bash-2.02$ dmesg | head Copyright (c) 1992-1999 FreeBSD Inc. Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All rights reserved. FreeBSD 3.2-STABLE #0: Fri Jun 11 08:18:12 EDT 1999 root@phoenix.home:/usr/src/sys/compile/PHOENIX_DUAL Timecounter "i8254" frequency 1193182 Hz CPU: Pentium II/Xeon/Celeron (686-class CPU) Origin = "GenuineIntel" Id = 0x652 Stepping=2 Features=0x183fbff> real memory = 268435456 (262144K bytes) avail memory = 257925120 (251880K bytes) Programming 24 pins in IOAPIC #0 FreeBSD/SMP: Multiprocessor motherboard cpu0 (BSP): apic id: 1, version: 0x00040011, at 0xfee00000 cpu1 (AP): apic id: 0, version: 0x00040011, at 0xfee00000 io0 (APIC): apic id: 2, version: 0x00170011, at 0xfec00000 Preloaded elf kernel "kernel" at 0xc02df000. Probing for devices on PCI bus 0: chip0: rev 0x03 on pci0.0.0 chip1: rev 0x03 on pci0.1.0 It is -STABLE from June 7th, mid-day. -- David Cross | email: crossd@cs.rpi.edu Systems Administrator/Research Programmer | Web: http://www.cs.rpi.edu/~crossd Rensselaer Polytechnic Institute, | Ph: 518.276.2860 Department of Computer Science | Fax: 518.276.4033 I speak only for myself. | WinNT:Linux::Linux:FreeBSD To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Jun 11 8:43: 6 1999 Delivered-To: freebsd-smp@freebsd.org Received: from fire.starkreality.com (fire.starkreality.com [208.24.48.226]) by hub.freebsd.org (Postfix) with ESMTP id C2DA215167; Fri, 11 Jun 1999 08:42:57 -0700 (PDT) (envelope-from caesar@fire.starkreality.com) Received: (from caesar@localhost) by fire.starkreality.com (8.9.3/8.9.2) id KAA66007; Fri, 11 Jun 1999 10:41:30 -0500 (CDT) From: "William S. Duncanson" Message-Id: <199906111541.KAA66007@fire.starkreality.com> Subject: Re: High syscall overhead? In-Reply-To: <199906111528.LAA71507@cs.rpi.edu> from "David E. Cross" at "Jun 11, 1999 11:28: 9 am" To: crossd@cs.rpi.edu (David E. Cross) Date: Fri, 11 Jun 1999 10:41:30 -0500 (CDT) Cc: chris@calldei.com, crossd@cs.rpi.edu, freebsd-hackers@FreeBSD.ORG, freebsd-smp@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL43 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org David E. Cross was heard to mumble: > Oops, here is some additional information from my system: > I can reproduce this under -CURRENT: Dual P200MMX running FreeBSD 4.0-CURRENT-990528: fire /home/caesar/src/tmp $ time ./sc real 0m34.695s user 0m20.457s sys 0m13.850s Single P200MMX running RH 6.0 Linux: hindenburg /home/wduncan/src/tmp $ time ./sc 2.24user 5.09system 0:07.53elapsed 97%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (65major+8minor)pagefaults 0swaps -- William S. Duncanson caesar@starkreality.com Smash forehead on keyboard to continue... To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Jun 11 21: 7:24 1999 Delivered-To: freebsd-smp@freebsd.org Received: from ivory.lm.com (ivory.telerama.com [205.201.1.20]) by hub.freebsd.org (Postfix) with ESMTP id 31E0C14EA8; Fri, 11 Jun 1999 21:07:17 -0700 (PDT) (envelope-from evs@telerama.com) Received: from mvehpc (d13-23.dyn.telerama.com [205.201.41.215]) by ivory.lm.com (8.8.5/8.6.12) with SMTP id AAA29650; Sat, 12 Jun 1999 00:07:10 -0400 (EDT) Message-ID: <09f401beb488$bb612390$6f27abcd@mvehpc.evs.slip.lm.com> Reply-To: "Mikhail V. Evstiounin" From: "Mikhail V. Evstiounin" To: , "David E. Cross" Cc: , Subject: Re: High syscall overhead? Date: Sat, 12 Jun 1999 00:04:53 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 4.72.3110.5 X-MimeOLE: Produced By Microsoft MimeOLE V4.72.3110.3 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I have installed 3.0-RELEASE FreeBSD on AMD K6, 233MHz, 128MB, ABit Motherboard, not overclocked/ Ran this program, got the following results: 5.2u 8.5s 0:14.02 98.3% 5+171k 0+0io 0pf+0w. :-( -----Original Message----- From: Chris Costello To: David E. Cross Cc: freebsd-hackers@FreeBSD.ORG ; freebsd-smp@FreeBSD.ORG Date: Friday, June 11, 1999 11:01 AM Subject: Re: High syscall overhead? >On Fri, Jun 11, 1999, David E. Cross wrote: >> Just doing some performance testing and I noticed something rather >> disturbing.... >> >> Here is the test program: >> int main (void) >> { >> int count=0; >> for(count=0;count <10000000;++count) >> getppid(); >> >> return 0; >> } >> >> The time on linux for this program is ~5 seconds (linux "time" reports 3.x, but >> a wall clock clearly shows 5.x, go fig). FreeBSD reports 18.x seconds?!. I >> have a dual processor system and decided to parallel run them... it took >> 52!?! seconds, linux on the same was again about 5. Looking through the >> exception.s it appears that on entry to the kernel an MP lock is obtained... >> I thought we had splX(); to protect concurancy in the kernel. > >('sc' being the program above, compiled without optimization, > just with cc -o sc sc.c) >$ time ./sc > 10.04s real 3.89s user 5.64s system > > I counted between around 9 and a half to 10 and a half seconds >on my wall clock (trusty old GE, same model they have in public >schools). > >Copyright (c) 1992-1999 The FreeBSD Project. >Copyright (c) 1982, 1986, 1989, 1991, 1993 > The Regents of the University of California. All rights reserved. >FreeBSD 4.0-CURRENT #4: Sun May 30 04:22:23 CDT 1999 > root@holly.dyndns.org:/usr/src/sys/compile/Holly >Timecounter "i8254" frequency 1193182 Hz >CPU: AMD-K6(tm) 3D processor (350.80-MHz 586-class CPU) > Origin = "AuthenticAMD" Id = 0x580 Stepping=0 > Features=0x8001bf >real memory = 67108864 (65536K bytes) >sio0: system console >avail memory = 62267392 (60808K bytes) > > SMP specific bug, perhaps? > >> >> I am just curious what's the story with this. On some of my other tests it is >> clear that FreeBSD is handling concurancy much better than linux (by an equal >> factor actually, and on "real" tasks like real I/O handling). >> >> -- >> David Cross | email: crossd@cs.rpi.edu >> Systems Administrator/Research Programmer | Web: http://www.cs.rpi.edu/~crossd >> Rensselaer Polytechnic Institute, | Ph: 518.276.2860 >> Department of Computer Science | Fax: 518.276.4033 >> I speak only for myself. | WinNT:Linux::Linux:FreeBSD >> >> >> To Unsubscribe: send mail to majordomo@FreeBSD.org >> with "unsubscribe freebsd-hackers" in the body of the message > >-- >Chris Costello >This message transmitted on 100% recycled electrons. > > >To Unsubscribe: send mail to majordomo@FreeBSD.org >with "unsubscribe freebsd-smp" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Jun 12 1:15: 6 1999 Delivered-To: freebsd-smp@freebsd.org Received: from cygnus.rush.net (cygnus.rush.net [209.45.245.133]) by hub.freebsd.org (Postfix) with ESMTP id AD88414D88 for ; Sat, 12 Jun 1999 01:15:04 -0700 (PDT) (envelope-from bright@rush.net) Received: from localhost (bright@localhost) by cygnus.rush.net (8.9.3/8.9.3) with SMTP id DAA02294 for ; Sat, 12 Jun 1999 03:34:31 -0500 (EST) Date: Sat, 12 Jun 1999 03:34:27 -0500 (EST) From: Alfred Perlstein To: smp@freebsd.org Subject: simple_lock() ? Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org The simple_lock/unlock/try_lock stuff looks like precursor work towards finer grained SMP. Is this true? right now it looks like it amounts to a NOP in SMP and UP systems, is this also true? thanks, -Alfred To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Jun 12 1:33: 6 1999 Delivered-To: freebsd-smp@freebsd.org Received: from pallas.veritas.com (pallas.veritas.com [204.177.156.25]) by hub.freebsd.org (Postfix) with ESMTP id 1E46515145; Sat, 12 Jun 1999 01:33:03 -0700 (PDT) (envelope-from aaron@sigma.veritas.com) Received: from megami.veritas.com (megami.veritas.com [192.203.46.101]) by pallas.veritas.com (8.9.1a/8.9.1) with SMTP id BAA04617; Sat, 12 Jun 1999 01:33:40 -0700 (PDT) Received: from sigma.veritas.com([192.203.46.125]) (1876 bytes) by megami.veritas.com via sendmail with P:esmtp/R:smart_host/T:smtp (sender: ) id for ; Sat, 12 Jun 1999 01:33:00 -0700 (PDT) (Smail-3.2.0.101 1997-Dec-17 #3 built 1999-Jan-25) Received: from sigma (localhost [127.0.0.1]) by sigma.veritas.com (8.9.2/8.9.1) with ESMTP id BAA40681; Sat, 12 Jun 1999 01:33:00 -0700 (PDT) (envelope-from aaron@sigma.veritas.com) Message-Id: <199906120833.BAA40681@sigma.veritas.com> From: Aaron Smith To: Arun Sharma Cc: "David E. Cross" , freebsd-hackers@FreeBSD.ORG, freebsd-smp@FreeBSD.ORG Subject: Re: High syscall overhead? In-reply-to: Your message of "11 Jun 1999 23:15:15 PDT." Date: Sat, 12 Jun 1999 01:33:00 -0700 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 11 Jun 1999 23:15:15 PDT, Arun Sharma writes: >Can someone explain to me why is SYSCALL_LOCK necessary ? It certainly >seems to hurt system call performance on a MP machine. > >Also, is there any data on lock contention in FreeBSD ? Is anyone >working on decomposing some of the giant locks ? I have a follow-on question: is there any planned work to give FreeBSD some of the basic synch primitives? I would love to help finer-grain the kernel, (having just today built my first SMP FreeBSD system), but first I think I'd need to implement mutexes and condition variables. It looks like there may be spin/sleeplocks and rwlocks, but they're not called that. Is there any work being done in this area? I think implementing the SVR4 synch primitives (mutex, condvars, maybe semas, rwlocks) would be great, since that's what's taught, and they're intuitive. I'm still trying to figure out the deal with "lockmgr". -- Aaron Smith VERITAS Software File System Engineer "I'll call him mini me". To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Jun 12 10: 0:58 1999 Delivered-To: freebsd-smp@freebsd.org Received: from mail.rdc1.sfba.home.com (ha1.rdc1.sfba.home.com [24.0.0.66]) by hub.freebsd.org (Postfix) with ESMTP id 5B76715150 for ; Sat, 12 Jun 1999 10:00:56 -0700 (PDT) (envelope-from adsharma@c62443-a.frmt1.sfba.home.com) Received: from c62443-a.frmt1.sfba.home.com ([24.0.69.165]) by mail.rdc1.sfba.home.com (InterMail v4.01.01.00 201-229-111) with ESMTP id <19990612170056.QGOS8807.mail.rdc1.sfba.home.com@c62443-a.frmt1.sfba.home.com>; Sat, 12 Jun 1999 10:00:56 -0700 Received: (from adsharma@localhost) by c62443-a.frmt1.sfba.home.com (8.8.7/8.8.7) id KAA25921; Sat, 12 Jun 1999 10:00:56 -0700 To: Alfred Perlstein Cc: smp@FreeBSD.ORG Subject: Re: simple_lock() ? References: From: Arun Sharma Mime-Version: 1.0 (generated by tm-edit 7.106) Content-Type: text/plain; charset=US-ASCII Date: 12 Jun 1999 10:00:56 -0700 In-Reply-To: Alfred Perlstein's message of "Sat, 12 Jun 1999 03:34:27 -0500 (EST)" Message-ID: Lines: 10 X-Mailer: Gnus v5.5/Emacs 20.3 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Alfred Perlstein writes: > The simple_lock/unlock/try_lock stuff looks like precursor work towards > finer grained SMP. Is this true? right now it looks like it amounts > to a NOP in SMP and UP systems, is this also true? It is a nop in UP systems, but calls some locking primitive on a SMP system. -Arun To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Jun 12 10:40:36 1999 Delivered-To: freebsd-smp@freebsd.org Received: from mail.rdc1.sfba.home.com (ha1.rdc1.sfba.home.com [24.0.0.66]) by hub.freebsd.org (Postfix) with ESMTP id 03D5E14E84; Sat, 12 Jun 1999 10:40:33 -0700 (PDT) (envelope-from adsharma@c62443-a.frmt1.sfba.home.com) Received: from c62443-a.frmt1.sfba.home.com ([24.0.69.165]) by mail.rdc1.sfba.home.com (InterMail v4.01.01.00 201-229-111) with ESMTP id <19990612174033.QNET8807.mail.rdc1.sfba.home.com@c62443-a.frmt1.sfba.home.com>; Sat, 12 Jun 1999 10:40:33 -0700 Received: (from adsharma@localhost) by c62443-a.frmt1.sfba.home.com (8.8.7/8.8.7) id KAA25937; Sat, 12 Jun 1999 10:40:32 -0700 To: Aaron Smith Cc: "David E. Cross" , freebsd-hackers@FreeBSD.ORG, freebsd-smp@FreeBSD.ORG Subject: Re: High syscall overhead? References: <199906120833.BAA40681@sigma.veritas.com> From: Arun Sharma Mime-Version: 1.0 (generated by tm-edit 7.106) Content-Type: text/plain; charset=US-ASCII Date: 12 Jun 1999 10:40:32 -0700 In-Reply-To: Aaron Smith's message of "Sat, 12 Jun 1999 01:33:00 -0700" Message-ID: Lines: 9 X-Mailer: Gnus v5.5/Emacs 20.3 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Aaron Smith writes: > I'm still trying to figure out the deal with "lockmgr". I found the following doc useful: http://www.freebsd.org/~fsmp/SMP/Locking.html -Arun To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Jun 12 12:49:56 1999 Delivered-To: freebsd-smp@freebsd.org Received: from cygnus.rush.net (cygnus.rush.net [209.45.245.133]) by hub.freebsd.org (Postfix) with ESMTP id 0EBE3150FB for ; Sat, 12 Jun 1999 12:49:46 -0700 (PDT) (envelope-from bright@rush.net) Received: from localhost (bright@localhost) by cygnus.rush.net (8.9.3/8.9.3) with SMTP id PAA29473; Sat, 12 Jun 1999 15:09:17 -0500 (EST) Date: Sat, 12 Jun 1999 15:09:16 -0500 (EST) From: Alfred Perlstein To: Arun Sharma Cc: smp@FreeBSD.ORG Subject: Re: simple_lock() ? In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 12 Jun 1999, Arun Sharma wrote: > Alfred Perlstein writes: > > > The simple_lock/unlock/try_lock stuff looks like precursor work towards > > finer grained SMP. Is this true? right now it looks like it amounts > > to a NOP in SMP and UP systems, is this also true? > > It is a nop in UP systems, but calls some locking primitive on a SMP > system. because of the BGL i don't see why his is nessesary... confused, -Alfred :) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Jun 12 14:22:43 1999 Delivered-To: freebsd-smp@freebsd.org Received: from mail.rdc1.sfba.home.com (ha1.rdc1.sfba.home.com [24.0.0.66]) by hub.freebsd.org (Postfix) with ESMTP id 96DFC14DB2 for ; Sat, 12 Jun 1999 14:22:42 -0700 (PDT) (envelope-from adsharma@c62443-a.frmt1.sfba.home.com) Received: from c62443-a.frmt1.sfba.home.com ([24.0.69.165]) by mail.rdc1.sfba.home.com (InterMail v4.01.01.00 201-229-111) with ESMTP id <19990612212242.RTPG8807.mail.rdc1.sfba.home.com@c62443-a.frmt1.sfba.home.com>; Sat, 12 Jun 1999 14:22:42 -0700 Received: (from adsharma@localhost) by c62443-a.frmt1.sfba.home.com (8.8.7/8.8.7) id OAA26293; Sat, 12 Jun 1999 14:22:42 -0700 To: Alfred Perlstein Cc: smp@FreeBSD.ORG Subject: Re: simple_lock() ? References: From: Arun Sharma Mime-Version: 1.0 (generated by tm-edit 7.106) Content-Type: text/plain; charset=US-ASCII Date: 12 Jun 1999 14:22:42 -0700 In-Reply-To: Alfred Perlstein's message of "Sat, 12 Jun 1999 15:09:16 -0500 (EST)" Message-ID: Lines: 22 X-Mailer: Gnus v5.5/Emacs 20.3 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Alfred Perlstein writes: > On 12 Jun 1999, Arun Sharma wrote: > > > Alfred Perlstein writes: > > > > > The simple_lock/unlock/try_lock stuff looks like precursor work towards > > > finer grained SMP. Is this true? right now it looks like it amounts > > > to a NOP in SMP and UP systems, is this also true? > > > > It is a nop in UP systems, but calls some locking primitive on a SMP > > system. > > because of the BGL i don't see why his is nessesary... > > confused, Here's a guess: the comments in simplelock.s indicate that the locking code came from 4.4BSD lite2. But you're right - in a giant locked kernel, they are wasteful. -Arun To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message