From owner-freebsd-current@freebsd.org Wed Jan 4 19:59:14 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9114DC9F9DD for ; Wed, 4 Jan 2017 19:59:14 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 397F11C5C; Wed, 4 Jan 2017 19:59:14 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id v04Jx95T007285 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 4 Jan 2017 21:59:09 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua v04Jx95T007285 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id v04Jx96x007284; Wed, 4 Jan 2017 21:59:09 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 4 Jan 2017 21:59:09 +0200 From: Konstantin Belousov To: Eric Joyner Cc: FreeBSD Current Subject: Re: How many CPU cores does FreeBSD support? Message-ID: <20170104195909.GB2533@kib.kiev.ua> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.2 (2016-11-26) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jan 2017 19:59:14 -0000 On Wed, Jan 04, 2017 at 06:53:23PM +0000, Eric Joyner wrote: > Adding freebsd-current, because that's a good idea. > > I see these lines in the beginning of dmesg: > > MADT: Ignoring local APIC ID 256 (too high) > > > [907/911] > MADT: Ignoring local APIC ID 260 (too high) > > > [906/911] > MADT: Ignoring local APIC ID 264 (too high) > > > [905/911] > MADT: Ignoring local APIC ID 268 (too high) > > > [904/911] > MADT: Ignoring local APIC ID 272 (too high) > > > [903/911] > MADT: Ignoring local APIC ID 276 (too high) > > > [902/911] > MADT: Ignoring local APIC ID 280 (too high) > > > [901/911] > MADT: Ignoring local APIC ID 272 (too high) > > > [903/911] > MADT: Ignoring local APIC ID 276 (too high) > > > [902/911] > MADT: Ignoring local APIC ID 280 (too high) > > > [901/911] > MADT: Ignoring local APIC ID 276 (too high) > > > [902/911] > MADT: Ignoring local APIC ID 280 (too high) > MADT: Ignoring local APIC ID 284 (too high) > MADT: Ignoring local APIC ID 288 (too high) > MADT: Ignoring local APIC ID 292 (too high) > MADT: Ignoring local APIC ID 296 (too high) > MADT: Ignoring local APIC ID 300 (too high) > MADT: Ignoring local APIC ID 257 (too high) > MADT: Ignoring local APIC ID 261 (too high) > MADT: Ignoring local APIC ID 265 (too high) > MADT: Ignoring local APIC ID 269 (too high) > MADT: Ignoring local APIC ID 273 (too high) > MADT: Ignoring local APIC ID 277 (too high) > MADT: Ignoring local APIC ID 281 (too high) > MADT: Ignoring local APIC ID 285 (too high) > MADT: Ignoring local APIC ID 289 (too high) > MADT: Ignoring local APIC ID 293 (too high) > MADT: Ignoring local APIC ID 297 (too high) > MADT: Ignoring local APIC ID 301 (too high) > MADT: Ignoring local APIC ID 258 (too high) > MADT: Ignoring local APIC ID 262 (too high) > MADT: Ignoring local APIC ID 266 (too high) > MADT: Ignoring local APIC ID 270 (too high) > MADT: Ignoring local APIC ID 274 (too high) > MADT: Ignoring local APIC ID 278 (too high) > MADT: Ignoring local APIC ID 282 (too high) > MADT: Ignoring local APIC ID 286 (too high) > MADT: Ignoring local APIC ID 290 (too high) > MADT: Ignoring local APIC ID 294 (too high) > MADT: Ignoring local APIC ID 298 (too high) > MADT: Ignoring local APIC ID 302 (too high) > MADT: Ignoring local APIC ID 255 (too high) > MADT: Ignoring local APIC ID 259 (too high) > MADT: Ignoring local APIC ID 263 (too high) > MADT: Ignoring local APIC ID 267 (too high) > MADT: Ignoring local APIC ID 271 (too high) > MADT: Ignoring local APIC ID 275 (too high) > MADT: Ignoring local APIC ID 279 (too high) > MADT: Ignoring local APIC ID 283 (too high) > MADT: Ignoring local APIC ID 287 (too high) > MADT: Ignoring local APIC ID 291 (too high) > MADT: Ignoring local APIC ID 295 (too high) > MADT: Ignoring local APIC ID 299 (too high) > MADT: Ignoring local APIC ID 303 (too high) > Copyright (c) 1992-2016 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 11.0-RELEASE-p1 #0 r306420: Thu Sep 29 01:43:23 UTC 2016 > root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 > FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM > 3.8.0) > SRAT: No CPU found for memory domain 1 > VT(efifb): resolution 800x600 > CPU: Intel(R) Xeon Phi(TM) CPU 7250 @ 1.40GHz (1396.80-MHz K8-class CPU) > Origin="GenuineIntel" Id=0x50671 Family=0x6 Model=0x57 Stepping=1 > > Features=0xbfebfbff > > Features2=0x7ff8f39f > AMD Features=0x2c100800 > AMD Features2=0x121 > Structured Extended > Features=0x1c0d23ab > Structured Extended Features2=0x1 > XSAVE Features=0x1 > TSC: P-state invariant, performance statistics > real memory = 223332007936 (212986 MB) > avail memory = 216976429056 (206924 MB) > Event timer "LAPIC" quality 600 > ACPI APIC Table: > FreeBSD/SMP: Multiprocessor System Detected: 223 CPUs > FreeBSD/SMP: Non-uniform topology > > It sounds like those MADT errors point to the problem? In sys/x86/acpica/madt.c file, function madt_add_cpu(), just remove the block if (apic_id > MAX_APIC_ID) { printf("MADT: Ignoring local APIC ID %u (too high)\n", apic_id); return; } Best to do it on HEAD and not on 11.0. If it helps, commit the change. Also, for >= 256 CPUs, you might need to enable interrupt remapping in the DMAR, use loader tunables hw.dmar.enable <= to enable DMAR hw.dmar.ir <= to enable remapping hw.dmar.dma <= to disable IOMMU if it interacts badly with your hw