From owner-freebsd-performance@FreeBSD.ORG Tue May 27 09:45:31 2008 Return-Path: Delivered-To: freebsd-performance@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 18A9B10656C7; Tue, 27 May 2008 09:45:31 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 35FAB8FC0A; Tue, 27 May 2008 09:45:30 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <483BD839.9020804@FreeBSD.org> Date: Tue, 27 May 2008 11:45:29 +0200 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: "Hattori, Shigehiro" References: <011501c8bfcc$a3164b40$c154140a@colors.scs.co.jp> In-Reply-To: <011501c8bfcc$a3164b40$c154140a@colors.scs.co.jp> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-performance@FreeBSD.org, freebsd-questions@freebsd.org Subject: Re: FreeBSD 7.0-RELEASE: Can I specify the maximum number of cores that kernel can recognize ? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 May 2008 09:45:31 -0000 Hattori, Shigehiro wrote: > Hi, > > I try to measure Bind caching name server's multi threading performance on > FreeBSD 7 , like below. > > -- > # of cores query/second > 1 xx > 2 xx > 4 xx > 6 xx > 8 xx > -- > > My testing machine has 8 cores. ( quad core * 2 ) > > I'd like to specify the maximum # of cores that kernel can recognize. > Can I specify the maximum # of cores with boot parameters or something ? You can disable the cores you do not want by turning off their lapic. Add to /boot/loader.conf: hint.lapic.0.disabled=1 turns off lapic 0, etc. Cross-reference against dmesg: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs cpu0 (BSP): APIC ID: 0 <---- disables this core cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 cpu4 (AP): APIC ID: 4 cpu5 (AP): APIC ID: 5 cpu6 (AP): APIC ID: 6 cpu7 (AP): APIC ID: 7 > I've already done Bind multi threading performance test on Linux ( > CentOS5 ) > In case of CentOS5 , I specified maximum # of cores with grub.conf　 > "maxcpus=1 > or 2 or 4 or ..." > > # cat /etc/grub.conf > title CentOS (2.6.18-53.1.21.el5PAE) > root (hd0,1) > kernel /boot/vmlinuz-2.6.18-53.1.21.el5PAE ro root=LABEL=/ rhgb > quiet maxcpus=6 > initrd /boot/initrd-2.6.18-53.1.21.el5PAE.img > # > > The following is the results I did on CentOS5 ( kernel: > 2.6.18-53.1.21.el5 ). > > -- Bind 9.4.2 caching name server on CentOS5 > # of cores query/second CPU ( named ) > 1 3578 99.9 > 2 5070 196 > 4 6608 362 > 6 9042 527 > 8 10552 678 > -- > > - Bind's machine spec > > CPU: Intel Xeon E5346 2.3GH ( quad core * 2 ) > Memory: 4GB > Bind 9.4.2 > Cache size: 1400MB > Max recursive clients: 50000 > > - Traffic generetor: queryperf > Query list: all queries are uniq ( that means "No answer in the cache" ) Make sure you are resolving these queries against another local server. If you're querying random servers in the internet then you're mostly going to be benchmarking your uplink latency and the maximum query rate will be limited by the number of broken servers you query that do not respond but time out. Those numbers are quite low even for Linux; on similar hardware I can achieve 60000 qps on Linux and about 105000 qps on FreeBSD: http://people.freebsd.org/~kris/scaling/bind-pt.png With 1gb ethernet I have to query from multiple clients to get that high because of request latency (with 10gbe I can saturate from a single client). I was not disabling cores but limiting the number of bind threads, which should be approximately the same thing especially for 8 threads. Kris From owner-freebsd-performance@FreeBSD.ORG Wed May 28 07:37:02 2008 Return-Path: Delivered-To: performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5F15E106566B; Wed, 28 May 2008 07:37:02 +0000 (UTC) (envelope-from tataz@tataz.chchile.org) Received: from postfix2-g20.free.fr (postfix2-g20.free.fr [212.27.60.43]) by mx1.freebsd.org (Postfix) with ESMTP id 0F94F8FC12; Wed, 28 May 2008 07:37:01 +0000 (UTC) (envelope-from tataz@tataz.chchile.org) Received: from smtp5-g19.free.fr (smtp5-g19.free.fr [212.27.42.35]) by postfix2-g20.free.fr (Postfix) with ESMTP id EAE2226AD996; Wed, 28 May 2008 07:19:52 +0200 (CEST) Received: from smtp5-g19.free.fr (localhost.localdomain [127.0.0.1]) by smtp5-g19.free.fr (Postfix) with ESMTP id E8BEC3F61D3; Wed, 28 May 2008 09:20:14 +0200 (CEST) Received: from tatooine.tataz.chchile.org (tataz.chchile.org [82.233.239.98]) by smtp5-g19.free.fr (Postfix) with ESMTP id AD2C43F6168; Wed, 28 May 2008 09:20:14 +0200 (CEST) Received: from obiwan.tataz.chchile.org (unknown [192.168.1.25]) by tatooine.tataz.chchile.org (Postfix) with ESMTP id 23AF09B497; Wed, 28 May 2008 07:16:55 +0000 (UTC) Received: by obiwan.tataz.chchile.org (Postfix, from userid 1000) id 0554F4089; Wed, 28 May 2008 09:16:55 +0200 (CEST) Date: Wed, 28 May 2008 09:16:54 +0200 From: Jeremie Le Hen To: Yani Brankov Message-ID: <20080528071654.GH80973@obiwan.tataz.chchile.org> References: <4831A0A5.6040808@bulinfo.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4831A0A5.6040808@bulinfo.net> User-Agent: Mutt/1.5.15 (2007-04-06) Cc: questions@freebsd.org, performance@freebsd.org Subject: Re: Low/Jerky performance in FreeBSD 7 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 May 2008 07:37:02 -0000 On Mon, May 19, 2008 at 08:45:41AM -0700, Yani Brankov wrote: > I recently updated to FreeBSD 7 and noticed that my box started to perform > as windows does under heavier loads. The mouse starts to be jerky when > compiling, window updates/redraws are slow and bump the CPU usage up to > 100%. I attributed this to the mga driver which comes with the distro in the > beginning. However, I later noticed the same happens even when X has not > been started. For instance, I start a kernel compile and the console mouse > becomes jumpy. All these have never happened before with FreeBSD on this > box. It has enough memory (1.5G) and relatively fast CPU (2.5GHz). I started > to think this may be originating from kernel level (irq handling, long times > in giant locked code during syscalls, etc). > > I am wondering whether it is only me who has hit this problem or it is more > common. It may be also related to hardware configuration, etc. I'm trying to > figure out. Try profiling your kernel with PMC: http://lists.freebsd.org/pipermail/freebsd-current/2006-February/061096.html Regards, -- Jeremie Le Hen < jeremie at le-hen dot org >< ttz at chchile dot org >