From owner-freebsd-smp@FreeBSD.ORG Wed Feb 8 12:46:10 2006 Return-Path: X-Original-To: freebsd-smp@freebsd.org Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2A67816A425 for ; Wed, 8 Feb 2006 12:46:10 +0000 (GMT) (envelope-from astatech@mail.ru) Received: from f64.mail.ru (f64.mail.ru [194.67.57.98]) by mx1.FreeBSD.org (Postfix) with ESMTP id 705DF43D45 for ; Wed, 8 Feb 2006 12:46:09 +0000 (GMT) (envelope-from astatech@mail.ru) Received: from mail by f64.mail.ru with local id 1F6ohz-0001cf-00; Wed, 08 Feb 2006 15:46:07 +0300 Received: from [217.107.221.51] by win.mail.ru with HTTP; Wed, 08 Feb 2006 15:46:07 +0300 From: astatech@mail.ru To: Joseph Koshy Mime-Version: 1.0 X-Mailer: mPOP Web-Mail 2.19 X-Originating-IP: [217.107.221.51] Date: Wed, 08 Feb 2006 15:46:07 +0300 In-Reply-To: <84dead720602070831g32e14257x16841f31e5b8527d@mail.gmail.com> Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 8bit Message-Id: Cc: freebsd-smp@freebsd.org Subject: 6.0 Stable on 8 Core AMD Opteron scalability. (perf stat) X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: astatech@mail.ru List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Feb 2006 12:46:10 -0000 > a> - 8 cores at 2.2GHz/16GB RAM ... > a> What can I do to utilize all CPU power on FreeBSD? > 1) Which benchmark are you using? jakarta jmeter with customly builded test plan for PHPBB site. It`s my standart benchmark and already tested on others platforms. > 2) Please post the output of dmesg(8). See below... > 3) Please post outputs from vmstat or any other high-level > performance statistics that you've collected while > the benchmark is running. See below... PS: maybe httpd wait for log write? $ dmesg -a Copyright (c) 1992-2005 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 6.0-RELEASE #2: Mon Feb 6 20:50:32 MSK 2006 root@XXXX:/usr/src/sys/amd64/compile/SMP Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Dual Core AMD Opteron(tm) Processor 875 (2190.07-MHz K8-class CPU) Origin = "AuthenticAMD" Id = 0x20f10 Stepping = 0 Features=0x178bfbff Features2=0x1 AMD Features=0xe2500800,LM,3DNow+,3DNow> Hyperthreading: 2 logical CPUs real memory = 17179869184 (16384 MB) avail memory = 16188342272 (15438 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 cpu4 (AP): APIC ID: 4 cpu5 (AP): APIC ID: 5 cpu6 (AP): APIC ID: 6 cpu7 (AP): APIC ID: 7 MADT: Forcing active-low polarity and level trigger for SCI ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-27 on motherboard ioapic2 irqs 28-31 on motherboard ioapic3 irqs 32-35 on motherboard ioapic4 irqs 36-39 on motherboard ioapic5 irqs 40-43 on motherboard ioapic6 irqs 44-47 on motherboard acpi0: on motherboard acpi0: Power Button (fixed) pci_link0: irq 10 on acpi0 pci_link1: irq 5 on acpi0 pci_link2: irq 7 on acpi0 pci_link3: irq 11 on acpi0 unknown: I/O range not supported unknown: I/O range not supported Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0xf008-0xf00b on acpi0 cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 acpi_button0: on acpi0 pcib0: port 0xcf8-0xcff,0xf000-0xf07f,0xf080-0xf0ff iomem 0xd8000-0xdbfff on acpi0 pci0: on pcib0 pcib1: at device 6.0 on pci0 pci1: on pcib1 pci1: at device 0.0 (no driver attached) pci1: at device 0.1 (no driver attached) pci1: at device 5.0 (no driver attached) isab0: at device 7.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1000-0x100f at device 7.1 on pci0 ata0: on atapci0 ata1: on atapci0 pci0: at device 7.3 (no driver attached) pcib2: at device 10.0 on pci0 pci2: on pcib2 bge0: mem 0xe5810000-0xe581ffff,0xe5800000-0xe580ffff irq 25 at device 2.0 on pci2 miibus0: on bge0 brgphy0: on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto bge0: Ethernet address: 00:09:3d:12:e8:b5 bge1: mem 0xe5830000-0xe583ffff,0xe5820000-0xe582ffff irq 26 at device 2.1 on pci2 miibus1: on bge1 brgphy1: on miibus1 brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto bge1: Ethernet address: 00:09:3d:12:e8:b6 mpt0: port 0x2000-0x20ff mem 0xe5850000-0xe585ffff,0xe5840000-0xe584ffff irq 27 at device 4.0 on pci2 mpt0: [GIANT-LOCKED] mpt0: MPI Version=1.2.15.0 mpt0: Unhandled Event Notify Frame. Event 0xa. mpt0: Capabilities: ( RAID-1E RAID-1 SAFTE ) mpt0: 1 Active Volume (1 Max) mpt0: 2 Hidden Drive Members (6 Max) pci0: at device 10.1 (no driver attached) pcib3: at device 11.0 on pci0 pci3: on pcib3 pci0: at device 11.1 (no driver attached) pcib4: iomem 0xe5b01000-0xe5b01fff,0xe5b03000-0xe5b03fff,0xe5b05000-0xe5b05fff,0xe5b07000-0xe5b07fff on acpi0 pci32: on pcib4 pcib5: mem 0xe5b00000-0xe5b00fff irq 32 at device 1.0 on pci32 pci33: on pcib5 pci32: at device 1.1 (no driver attached) pcib6: mem 0xe5b02000-0xe5b02fff irq 36 at device 2.0 on pci32 pci37: on pcib6 pci32: at device 2.1 (no driver attached) pcib7: mem 0xe5b04000-0xe5b04fff irq 40 at device 3.0 on pci32 pci41: on pcib7 pci32: at device 3.1 (no driver attached) pcib8: mem 0xe5b06000-0xe5b06fff irq 44 at device 4.0 on pci32 pci45: on pcib8 pci32: at device 4.1 (no driver attached) atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: flags 0x1 irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: model IntelliMouse Explorer, device ID 4 sio0: configured irq 4 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: [FAST] fd0: <1440-KB 3.5" drive> on fdc0 drive 0 orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xc97ff,0xc9800-0xcafff,0xcb000-0xcefff on isa0 ppc0: cannot reserve I/O port range sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio1: configured irq 3 not in bitmap of probed irqs 0 sio1: port may not be enabled sio1 at port 0x2f8-0x2ff irq 3 on isa0 sio1: type 16550A vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounters tick every 1.000 msec Waiting 5 seconds for SCSI devices to settle acd0: DVDROM at ata1-master UDMA33 mpt0:vol0(mpt0:0:0): Settings ( Hot-Plug-Spares ) mpt0:vol0(mpt0:0:0): Using Spare Pool: 0 mpt0:vol0(mpt0:0:0): 2 Members: (mpt0:0:0): Primary (mpt0:0:1): Secondary mpt0:vol0(mpt0:0:0): RAID-1 - Optimal mpt0:vol0(mpt0:0:0): Status ( Enabled ) (mpt0:vol0:0): Physical (mpt0:0:0), Pass-thru (mpt0:1:0) (mpt0:vol0:0): Online (mpt0:vol0:1): Physical (mpt0:0:1), Pass-thru (mpt0:1:1) (mpt0:vol0:1): Online ses0 at mpt0 bus 0 target 6 lun 0 ses0: Fixed Processor SCSI-2 device ses0: 3.300MB/s transfers ses0: SAF-TE Compliant Device pass2 at mpt0 bus 1 target 0 lun 0 pass2: Fixed unknown SCSI-3 device pass2: 320.000MB/s transfers (160.000MHz, offset 63, 16bit), Tagged Queueing Enabled pass3 at mpt0 bus 1 target 1 lun 0 pass3: Fixed unknown SCSI-3 device pass3: 320.000MB/s transfers (160.000MHz, offset 63, 16bit), Tagged Queueing Enabled da0 at mpt0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-2 device da0: 320.000MB/s transfers (160.000MHz, offset 63, 16bit), Tagged Queueing Enabled da0: 69878MB (143110144 512 byte sectors: 255H 63S/T 8908C) SMP: AP CPU #1 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #5 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #7 Launched! Trying to mount root from ufs:/dev/da0s1a Loading configuration files. /dev/da0s1b is smaller than physical memory No suitable dump device was found. Entropy harvesting: interrupts ethernet point_to_point kickstart . swapon: adding /dev/da0s1b as swap device Starting file system checks: /dev/da0s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/da0s1a: clean, 220365 free (493 frags, 27484 blocks, 0.2% fragmentation) /dev/da0s1e: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/da0s1e: clean, 243870 free (38 frags, 30479 blocks, 0.0% fragmentation) /dev/da0s1f: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/da0s1f: clean, 3364076 free (43252 frags, 415103 blocks, 0.9% fragmentation) /dev/da0s1d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/da0s1d: clean, 460696 free (256 frags, 57555 blocks, 0.1% fragmentation) Setting hostname: XXXXX lo0: flags=8049 mtu 16384 inet 127.0.0.1 netmask 0xff000000 bge0: flags=8843 mtu 1500 options=1a inet 192.168.10.197 netmask 0xffffff00 broadcast 192.168.10.255 ether 00:09:3d:12:e8:b5 media: Ethernet autoselect (none) status: no carrier add net default: gateway 192.168.10.25 Additional routing options: . Starting devd. Mounting NFS file systems: . Creating and/or trimming log files: . Starting syslogd. /etc/rc: WARNING: Dump device does not exist. Savecore not run. ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib ldconfig: /usr/local/lib: ignoring directory not owned by root Starting usbd. ohci0: mem 0xe4120000-0xe4120fff irq 19 at device 0.0 on pci1 ohci0: [GIANT-LOCKED] usb0: OHCI version 1.0, legacy support usb0: SMM does not respond, resetting usb0: on ohci0 usb0: USB revision 1.0 bge0: link state changed to UP uhub0: AMD OHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 3 ports with 3 removable, self powered ohci1: mem 0xe4121000-0xe4121fff irq 19 at device 0.1 on pci1 ohci1: [GIANT-LOCKED] usb1: OHCI version 1.0, legacy support usb1: SMM does not respond, resetting usb1: on ohci1 usb1: USB revision 1.0 uhub1: AMD OHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub1: 3 ports with 3 removable, self powered Starting local daemons: . Updating motd . Configuring syscons: blanktime . Starting sshd. Initial amd64 initialization: . Additional ABI support: . Starting cron. Local package initialization: . Additional TCP options: . Starting background file system checks in 60 seconds. ------------------------------------------------------------------------------------------------------ $ vmstat 2 99999 procs memory page disks faults cpu r b w avm fre flt re pi po fr sr da0 pa0 in sy cs us sy id ..... 10 65 3 1321932 14450328 29310 0 0 0 30897 0 15 0 3009 311093 135143 48 25 26 92 13 4 1363148 14426332 32961 0 0 0 30981 0 14 0 3556 340291 93069 54 21 25 95 48 8 1340540 14440024 26367 0 0 0 27873 0 26 0 3967 294134 68154 49 21 29 80 1 1 1104284 14485872 44924 0 0 0 50532 0 28 0 3447 165200 68333 49 23 28 121 0 0 1365440 14432084 43587 0 0 0 37317 0 23 0 1754 211795 83845 46 30 23 54 11 3 1338724 14446888 19043 0 0 0 21133 0 9 0 2800 291969 13703539 25 36 66 0 0 1375484 14423592 34484 0 0 0 31638 0 15 0 3503 259224 109744 53 21 26 44 40 0 1350336 14437612 27347 0 0 0 29507 0 87 0 2966 281139 131237 44 27 29 71 1 2 1400156 14404564 30590 0 0 0 26273 0 23 0 3256 321923 143233 45 30 26 28 89 0 1381720 14417128 27507 0 0 0 28523 0 13 0 2797 261042 128871 46 21 34 64 0 19 1383200 14414360 34167 0 0 0 34986 0 25 0 4321 269089 106260 52 18 30 40 42 3 1383896 14409896 36772 0 0 0 36896 0 114 0 3747 278703 96025 53 22 25 107 11 3 1332280 14441928 27602 0 0 0 30553 0 10 0 3643 304230 84296 53 23 24 47 59 0 1345140 14431752 29048 0 0 0 29329 0 16 0 3862 346969 120789 49 25 27 44 58 3 1342888 14431316 22663 0 0 0 22538 0 15 0 3323 309007 137080 44 25 30 28 74 2 1410220 14396556 29990 0 0 0 25843 0 9 0 2946 270054 124079 46 26 28 151 0 9 1413572 14392208 41542 0 0 0 41407 0 17 0 3293 242969 77110 55 18 27 ..... $ top 8403 nobody 1 80 0 32548K 8704K RUN 7 0:22 2.25% httpd 8503 nobody 1 77 0 32668K 8796K CPU3 3 0:19 1.46% httpd 8520 nobody 1 79 0 33004K 9136K CPU0 0 0:19 1.46% httpd 8478 nobody 1 4 0 35452K 10340K sbwait 2 0:19 1.46% httpd 8490 nobody 1 -4 0 32600K 8756K ufs 7 0:18 1.46% httpd 8413 nobody 1 77 0 32880K 8848K RUN 6 0:17 1.46% httpd 8505 nobody 1 -4 0 32300K 8456K ufs 7 0:19 0.73% httpd 8416 nobody 1 -4 0 32396K 8552K ufs 7 0:19 0.73% httpd 8443 nobody 1 -4 0 32396K 8568K ufs 7 0:19 0.73% httpd 8414 nobody 1 -4 0 32564K 8720K ufs 7 0:19 0.73% httpd 8539 nobody 1 -4 0 35588K 10476K ufs 7 0:18 0.73% httpd 8485 nobody 1 4 0 32552K 8708K sbwait 2 0:18 0.73% httpd 8473 nobody 1 -4 0 31916K 8072K ufs 7 0:18 0.73% httpd 8444 nobody 1 77 0 32552K 8708K RUN 2 0:18 0.73% httpd 8452 nobody 1 -4 0 32424K 8580K ufs 7 0:18 0.73% httpd 8479 nobody 1 78 0 35372K 10260K CPU2 2 0:18 0.73% httpd 8535 nobody 1 -4 0 35544K 10432K ufs 7 0:17 0.73% httpd 8453 nobody 1 -4 0 35380K 10268K ufs 7 0:17 0.73% httpd 8634 mysql 111 76 0 367M 122M RUN 7 0:52 0.00% mysqld 8423 nobody 1 4 0 35424K 10312K sbwait 2 0:20 0.00% httpd 8436 nobody 1 -4 0 32200K 8356K ufs 7 0:20 0.00% httpd 8427 nobody 1 4 0 32300K 8456K sbwait 6 0:20 0.00% httpd 8402 nobody 1 77 0 35128K 10040K RUN 2 0:20 0.00% httpd 8486 nobody 1 -4 0 35332K 10244K ufs 6 0:20 0.00% httpd 8511 nobody 1 -4 0 32252K 8408K ufs 6 0:20 0.00% httpd 8410 nobody 1 4 0 35400K 10312K sbwait 6 0:20 0.00% httpd 8447 nobody 1 4 0 35428K 10316K sbwait 7 0:19 0.00% httpd 8482 nobody 1 -4 0 32296K 8452K ufs 6 0:19 0.00% httpd 8472 nobody 1 -4 0 32852K 9012K ufs 7 0:19 0.00% httpd 8458 nobody 1 -4 0 32552K 8720K ufs 7 0:19 0.00% httpd 8441 nobody 1 -4 0 35460K 10352K ufs 7 0:19 0.00% httpd 8442 nobody 1 4 0 32608K 8788K sbwait 0 0:19 0.00% httpd 8450 nobody 1 -4 0 32516K 8672K ufs 7 0:19 0.00% httpd 8542 nobody 1 4 0 35400K 10288K sbwait 3 0:19 0.00% httpd 8438 nobody 1 -4 0 35388K 10276K ufs 7 0:19 0.00% httpd $ mount /dev/da0s1a on / (ufs, local) devfs on /dev (devfs, local) /dev/da0s1e on /tmp (ufs, local, soft-updates) /dev/da0s1f on /usr (ufs, local, soft-updates) /dev/da0s1d on /var (ufs, local, soft-updates)