Date: Mon, 13 Mar 2006 08:54:31 -0500 From: Bill Moran <wmoran@collaborativefusion.com> To: freebsd-amd64@freebsd.org Cc: bseklecki@collaborativefusion.com Subject: How is hyperthreading handled on amd64? Message-ID: <20060313085431.0eb059d9.wmoran@collaborativefusion.com>
next in thread | raw e-mail | index | archive | help
We've got some dell servers (Poweredge 2850 is specifically what the following tests were run on). Some of the behaviour we're seeing was unexpected. In particular, I'm rather fuzzy as to how an amd64 kernel handles hyperthreading. (Personally, I'm also a little fuzzy on whether this really is hyperthreading under amd64 at all) After building a kernel with SMP, I do see 2 logical processors, and top(1) shows both of them doing their thing. However, machdep.hypterthreading_allowed is set to 0. Is the amd64 kernel ignoring this value, or is top(1) reporting incorrectly? Additionally, I'm not seeing the performance I'm accustomed to on hypterthreaded machines. Usually, throughput scales when hyperthreading is enabled and more than one process are fighting for CPU. In my tests, it seems as if there is no scaling whatsoever. i.e.: On a standard hyperthreaded machine, running 1 ubench might yield a CPU value of 100,000. Running two ubench simultaneously will usually result in each of them returning ~90,000 or so. With the amd64, I end up with each one around 50,000 - as if there were no second logical CPU (despite the fact that top(1) shows both in use). dmesg attached. I can provide more information on request. -- Bill Moran Collaborative Fusion Inc. Mar 10 15:08:51 kernel: Copyright (c) 1992-2005 The FreeBSD Project. Mar 10 15:08:51 kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 Mar 10 15:08:51 kernel: The Regents of the University of California. All rights reserved. Mar 10 15:08:51 kernel: FreeBSD 6.0-RELEASE #0: Wed Nov 2 19:07:38 UTC 2005 Mar 10 15:08:51 kernel: root@rat.samsco.home:/usr/obj/usr/src/sys/GENERIC Mar 10 15:08:51 kernel: ACPI APIC Table: <DELL PE BKC > Mar 10 15:08:51 kernel: Timecounter "i8254" frequency 1193182 Hz quality 0 Mar 10 15:08:51 kernel: CPU: Intel(R) Xeon(TM) CPU 3.00GHz (2992.71-MHz K8-class CPU) Mar 10 15:08:51 kernel: Origin = "GenuineIntel" Id = 0xf41 Stepping = 1 Mar 10 15:08:51 kernel: Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> Mar 10 15:08:51 kernel: Features2=0x641d<SSE3,RSVD2,MON,DS_CPL,CNTX-ID,CX16,<b14>> Mar 10 15:08:51 kernel: AMD Features=0x20000800<SYSCALL,LM> Mar 10 15:08:51 kernel: Hyperthreading: 2 logical CPUs Mar 10 15:08:51 kernel: real memory = 2147221504 (2047 MB) Mar 10 15:08:51 kernel: avail memory = 2063126528 (1967 MB) Mar 10 15:08:51 kernel: ioapic0: Changing APIC ID to 2 Mar 10 15:08:51 kernel: ioapic1: Changing APIC ID to 3 Mar 10 15:08:51 kernel: ioapic1: WARNING: intbase 32 != expected base 24 Mar 10 15:08:51 kernel: ioapic2: Changing APIC ID to 4 Mar 10 15:08:51 kernel: ioapic2: WARNING: intbase 64 != expected base 56 Mar 10 15:08:51 kernel: ioapic3: Changing APIC ID to 5 Mar 10 15:08:51 kernel: ioapic3: WARNING: intbase 96 != expected base 88 Mar 10 15:08:51 kernel: ioapic0 <Version 2.0> irqs 0-23 on motherboard Mar 10 15:08:51 kernel: ioapic1 <Version 2.0> irqs 32-55 on motherboard Mar 10 15:08:51 kernel: ioapic2 <Version 2.0> irqs 64-87 on motherboard Mar 10 15:08:51 kernel: ioapic3 <Version 2.0> irqs 96-119 on motherboard Mar 10 15:08:51 kernel: acpi0: <DELL PE BKC> on motherboard Mar 10 15:08:51 kernel: acpi0: Power Button (fixed) Mar 10 15:08:51 kernel: pci_link0: <ACPI PCI Link LNKA> irq 11 on acpi0 Mar 10 15:08:51 kernel: pci_link1: <ACPI PCI Link LNKB> irq 3 on acpi0 Mar 10 15:08:51 kernel: pci_link2: <ACPI PCI Link LNKC> irq 7 on acpi0 Mar 10 15:08:51 kernel: pci_link3: <ACPI PCI Link LNKD> irq 10 on acpi0 Mar 10 15:08:51 kernel: pci_link4: <ACPI PCI Link LNKE> on acpi0 Mar 10 15:08:51 kernel: pci_link5: <ACPI PCI Link LNKF> on acpi0 Mar 10 15:08:51 kernel: pci_link6: <ACPI PCI Link LNKG> on acpi0 Mar 10 15:08:51 kernel: pci_link7: <ACPI PCI Link LNKH> irq 5 on acpi0 Mar 10 15:08:51 kernel: Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 Mar 10 15:08:51 kernel: acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 Mar 10 15:08:51 kernel: cpu0: <ACPI CPU> on acpi0 Mar 10 15:08:51 kernel: pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 Mar 10 15:08:51 kernel: pci0: <ACPI PCI bus> on pcib0 Mar 10 15:08:51 kernel: pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0 Mar 10 15:08:51 kernel: pci1: <ACPI PCI bus> on pcib1 Mar 10 15:08:51 kernel: pcib2: <ACPI PCI-PCI bridge> at device 0.0 on pci1 Mar 10 15:08:51 kernel: pci2: <ACPI PCI bus> on pcib2 Mar 10 15:08:51 kernel: amr0: <LSILogic MegaRAID 1.51> mem 0xd80f0000-0xd80fffff,0xdfdc0000-0xdfdfffff irq 46 at device 14.0 on pci2 Mar 10 15:08:51 kernel: amr0: <LSILogic PERC 4e/Di> Firmware 513O, BIOS H418, 256MB RAM Mar 10 15:08:51 kernel: pcib3: <ACPI PCI-PCI bridge> at device 0.2 on pci1 Mar 10 15:08:51 kernel: pci3: <ACPI PCI bus> on pcib3 Mar 10 15:08:51 kernel: pcib4: <ACPI PCI-PCI bridge> at device 4.0 on pci0 Mar 10 15:08:51 kernel: pci4: <ACPI PCI bus> on pcib4 Mar 10 15:08:51 kernel: pcib5: <ACPI PCI-PCI bridge> at device 5.0 on pci0 Mar 10 15:08:51 kernel: pci5: <ACPI PCI bus> on pcib5 Mar 10 15:08:51 kernel: pcib6: <ACPI PCI-PCI bridge> at device 0.0 on pci5 Mar 10 15:08:51 kernel: pci6: <ACPI PCI bus> on pcib6 Mar 10 15:08:51 kernel: em0: <Intel(R) PRO/1000 Network Connection, Version - 2.1.7> port 0xecc0-0xecff mem 0xdfae0000-0xdfafffff irq 64 at device 7.0 on pci6 Mar 10 15:08:51 kernel: em0: Ethernet address: 00:11:43:d5:a4:1a Mar 10 15:08:51 kernel: em0: Speed:N/A Duplex:N/A Mar 10 15:08:51 kernel: pcib7: <ACPI PCI-PCI bridge> at device 0.2 on pci5 Mar 10 15:08:51 kernel: pci7: <ACPI PCI bus> on pcib7 Mar 10 15:08:51 kernel: em1: <Intel(R) PRO/1000 Network Connection, Version - 2.1.7> port 0xdcc0-0xdcff mem 0xdf8e0000-0xdf8fffff irq 65 at device 8.0 on pci7 Mar 10 15:08:51 kernel: em1: Ethernet address: 00:11:43:d5:a4:1b Mar 10 15:08:51 kernel: em1: Speed:N/A Duplex:N/A Mar 10 15:08:51 kernel: pcib8: <ACPI PCI-PCI bridge> at device 6.0 on pci0 Mar 10 15:08:51 kernel: pci8: <ACPI PCI bus> on pcib8 Mar 10 15:08:51 kernel: pcib9: <ACPI PCI-PCI bridge> at device 0.0 on pci8 Mar 10 15:08:51 kernel: pci9: <ACPI PCI bus> on pcib9 Mar 10 15:08:51 kernel: pcib10: <ACPI PCI-PCI bridge> at device 0.2 on pci8 Mar 10 15:08:51 kernel: pci10: <ACPI PCI bus> on pcib10 Mar 10 15:08:51 kernel: uhci0: <Intel 82801EB (ICH5) USB controller USB-A> port 0xbce0-0xbcff irq 16 at device 29.0 on pci0 Mar 10 15:08:51 kernel: uhci0: [GIANT-LOCKED] Mar 10 15:08:51 kernel: usb0: <Intel 82801EB (ICH5) USB controller USB-A> on uhci0 Mar 10 15:08:51 kernel: usb0: USB revision 1.0 Mar 10 15:08:51 kernel: uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 Mar 10 15:08:51 kernel: uhub0: 2 ports with 2 removable, self powered Mar 10 15:08:51 kernel: uhci1: <Intel 82801EB (ICH5) USB controller USB-B> port 0xbcc0-0xbcdf irq 19 at device 29.1 on pci0 Mar 10 15:08:51 kernel: uhci1: [GIANT-LOCKED] Mar 10 15:08:51 kernel: usb1: <Intel 82801EB (ICH5) USB controller USB-B> on uhci1 Mar 10 15:08:51 kernel: usb1: USB revision 1.0 Mar 10 15:08:51 kernel: uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 Mar 10 15:08:51 kernel: uhub1: 2 ports with 2 removable, self powered Mar 10 15:08:51 kernel: uhci2: <Intel 82801EB (ICH5) USB controller USB-C> port 0xbca0-0xbcbf irq 18 at device 29.2 on pci0 Mar 10 15:08:51 kernel: uhci2: [GIANT-LOCKED] Mar 10 15:08:51 kernel: usb2: <Intel 82801EB (ICH5) USB controller USB-C> on uhci2 Mar 10 15:08:51 kernel: usb2: USB revision 1.0 Mar 10 15:08:51 kernel: uhub2: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 Mar 10 15:08:51 kernel: uhub2: 2 ports with 2 removable, self powered Mar 10 15:08:51 kernel: ehci0: <EHCI (generic) USB 2.0 controller> mem 0xdff00000-0xdff003ff irq 23 at device 29.7 on pci0 Mar 10 15:08:51 kernel: ehci0: [GIANT-LOCKED] Mar 10 15:08:51 kernel: usb3: EHCI version 1.0 Mar 10 15:08:51 kernel: usb3: companion controllers, 2 ports each: usb0 usb1 usb2 Mar 10 15:08:51 kernel: usb3: <EHCI (generic) USB 2.0 controller> on ehci0 Mar 10 15:08:51 kernel: usb3: USB revision 2.0 Mar 10 15:08:51 kernel: uhub3: Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1 Mar 10 15:08:51 kernel: uhub3: 6 ports with 6 removable, self powered Mar 10 15:08:51 kernel: uhub4: vendor 0x413c product 0xa001, class 9/0, rev 2.00/0.00, addr 2 Mar 10 15:08:51 kernel: uhub4: multiple transaction translators Mar 10 15:08:51 kernel: uhub4: 2 ports with 2 removable, self powered Mar 10 15:08:51 kernel: ukbd0: DELL DELL USB Keyboard, rev 1.10/1.04, addr 3, iclass 3/1 Mar 10 15:08:51 kernel: kbd0 at ukbd0 Mar 10 15:08:51 kernel: pcib11: <ACPI PCI-PCI bridge> at device 30.0 on pci0 Mar 10 15:08:51 kernel: pci11: <ACPI PCI bus> on pcib11 Mar 10 15:08:51 kernel: pci11: <display, VGA> at device 13.0 (no driver attached) Mar 10 15:08:51 kernel: isab0: <PCI-ISA bridge> at device 31.0 on pci0 Mar 10 15:08:51 kernel: isa0: <ISA bus> on isab0 Mar 10 15:08:51 kernel: atapci0: <Intel ICH5 UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfc00-0xfc0f at device 31.1 on pci0 Mar 10 15:08:51 kernel: ata0: <ATA channel 0> on atapci0 Mar 10 15:08:51 kernel: ata1: <ATA channel 1> on atapci0 Mar 10 15:08:51 kernel: fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 Mar 10 15:08:51 kernel: fdc0: [FAST] Mar 10 15:08:51 kernel: sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 Mar 10 15:08:51 kernel: sio0: type 16550A Mar 10 15:08:51 kernel: orm0: <ISA Option ROMs> at iomem 0xc0000-0xcafff,0xec000-0xeffff on isa0 Mar 10 15:08:51 kernel: atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0 Mar 10 15:08:51 kernel: ppc0: cannot reserve I/O port range Mar 10 15:08:51 kernel: sc0: <System console> at flags 0x100 on isa0 Mar 10 15:08:51 kernel: sc0: VGA <16 virtual consoles, flags=0x300> Mar 10 15:08:51 kernel: sio1: configured irq 3 not in bitmap of probed irqs 0 Mar 10 15:08:51 kernel: sio1: port may not be enabled Mar 10 15:08:51 kernel: vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Mar 10 15:08:51 kernel: Timecounter "TSC" frequency 2992705635 Hz quality 800 Mar 10 15:08:51 kernel: Timecounters tick every 1.000 msec Mar 10 15:08:51 kernel: acd0: CDROM <TEAC CD-ROM CD-224E/K.9A> at ata0-master UDMA33 Mar 10 15:08:51 kernel: amrd0: <LSILogic MegaRAID logical drive> on amr0 Mar 10 15:08:51 kernel: amrd0: 34680MB (71024640 sectors) RAID 1 (optimal) Mar 10 15:08:51 kernel: amrd1: <LSILogic MegaRAID logical drive> on amr0 Mar 10 15:08:51 kernel: amrd1: 209640MB (429342720 sectors) RAID 5 (optimal) Mar 10 15:08:51 kernel: ses0 at amr0 bus 0 target 6 lun 0 Mar 10 15:08:51 kernel: ses0: <PE/PV 1x2 SCSI BP 1.0> Fixed Processor SCSI-2 device Mar 10 15:08:51 kernel: ses0: SAF-TE Compliant Device Mar 10 15:08:51 kernel: ses1 at amr0 bus 1 target 6 lun 0 Mar 10 15:08:51 kernel: ses1: <PE/PV 1x4 SCSI BP 1.0> Fixed Processor SCSI-2 device Mar 10 15:08:51 kernel: ses1: SAF-TE Compliant Device
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20060313085431.0eb059d9.wmoran>