From owner-freebsd-performance@FreeBSD.ORG Sun Nov 27 05:35:12 2005 Return-Path: X-Original-To: performance@freebsd.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1043F16A41F for ; Sun, 27 Nov 2005 05:35:12 +0000 (GMT) (envelope-from gnn@neville-neil.com) Received: from mrout2.yahoo.com (mrout2.yahoo.com [216.145.54.172]) by mx1.FreeBSD.org (Postfix) with ESMTP id B8A0443D49 for ; Sun, 27 Nov 2005 05:35:11 +0000 (GMT) (envelope-from gnn@neville-neil.com) Received: from minion.local.neville-neil.com (proxy7.corp.yahoo.com [216.145.48.98]) by mrout2.yahoo.com (8.13.4/8.13.4/y.out) with ESMTP id jAR5Yj9k089127; Sat, 26 Nov 2005 21:34:45 -0800 (PST) Date: Sun, 27 Nov 2005 14:34:50 +0900 Message-ID: From: gnn@freebsd.org To: "Ricardo A. Reis" In-Reply-To: <438600DD.10608@yahoo.com.br> References: <438600DD.10608@yahoo.com.br> User-Agent: Wanderlust/2.14.0 (Africa) SEMI/1.14.6 (Maruoka) FLIM/1.14.7 (=?ISO-8859-4?Q?Sanj=F2?=) APEL/10.6 Emacs/21.2 (powerpc-apple-darwin) MULE/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII Cc: performance@freebsd.org Subject: Re: Libmicro!! pthread_create Resource temporarily unavailable X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Nov 2005 05:35:12 -0000 At Thu, 24 Nov 2005 16:05:17 -0200, Ricardo A. Reis wrote: > > Hi all, > > > Recently i uncovered libmicro work in freebsd, but after > install from ports > my first test using pthread_create not work. > > Using default or custom parameters in pthread_create i see this messages > on dmesg , > > > pthread_create: Resource temporarily unavailable > Which version of FreeBSD please? Thanks, George From owner-freebsd-performance@FreeBSD.ORG Sun Nov 27 18:20:23 2005 Return-Path: X-Original-To: performance@freebsd.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0228E16A41F for ; Sun, 27 Nov 2005 18:20:23 +0000 (GMT) (envelope-from ricardo_bsd@yahoo.com.br) Received: from smtp202.mail.sc5.yahoo.com (smtp202.mail.sc5.yahoo.com [216.136.129.92]) by mx1.FreeBSD.org (Postfix) with SMTP id 5F7F943D55 for ; Sun, 27 Nov 2005 18:20:22 +0000 (GMT) (envelope-from ricardo_bsd@yahoo.com.br) Received: (qmail 87406 invoked from network); 27 Nov 2005 12:01:36 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com.br; h=Received:Subject:From:To:Cc:In-Reply-To:References:Content-Type:Date:Message-Id:Mime-Version:X-Mailer:Content-Transfer-Encoding; b=BSiZANv3BLOVAulZH1lg6bPTDOJVarSKlqTi+OZ9eodv00PBaY91qgu9DaHGhhR4R1yeoOs/D7WvQxA8GunYb3GwhN3CRBMSC/Z0yhen/5dplmBDjH99NgoTggLjxliGe0GUsCAa2F4hUPv7DjIQ6SwoWSBqkjmChRIviPgYndw= ; Received: from unknown (HELO 201-1-75-81.dsl.telesp.net.br) (ricardo?bsd@201.1.75.81 with plain) by smtp202.mail.sc5.yahoo.com with SMTP; 27 Nov 2005 12:01:33 -0000 From: "Ricardo A. Reis" To: gnn@freebsd.org In-Reply-To: References: <438600DD.10608@yahoo.com.br> Content-Type: text/plain Date: Sun, 27 Nov 2005 08:00:58 -0200 Message-Id: <1133085658.906.19.camel@myfreebsd.homeunix.org> Mime-Version: 1.0 X-Mailer: Evolution 2.2.3 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: performance@freebsd.org Subject: Re: Libmicro!! pthread_create Resource temporarily unavailable X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Nov 2005 18:20:23 -0000 Hi George, > > pthread_create: Resource temporarily unavailable > > > > Which version of FreeBSD please? I test libmicro in 6.0-RELEASE and last cvsuped RELENG_6, both doesn't work. Thanks Ricardo A. Reis UNIFESP Unix and Network Admin _______________________________________________________ Faça do Yahoo! sua página inicial. http://br.yahoo.com/homepageset.html From owner-freebsd-performance@FreeBSD.ORG Mon Nov 28 00:02:43 2005 Return-Path: X-Original-To: performance@freebsd.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from [127.0.0.1] (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id 4161D16A423; Mon, 28 Nov 2005 00:02:40 +0000 (GMT) (envelope-from davidxu@freebsd.org) Message-ID: <438A4920.9070101@freebsd.org> Date: Mon, 28 Nov 2005 08:02:40 +0800 From: David Xu User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.7.10) Gecko/20050806 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "Ricardo A. Reis" References: <438600DD.10608@yahoo.com.br> <1133085658.906.19.camel@myfreebsd.homeunix.org> In-Reply-To: <1133085658.906.19.camel@myfreebsd.homeunix.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: gnn@freebsd.org, performance@freebsd.org Subject: Re: Libmicro!! pthread_create Resource temporarily unavailable X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Nov 2005 00:02:43 -0000 Ricardo A. Reis wrote: >Hi George, > > > >>>pthread_create: Resource temporarily unavailable >>> >>> >>> >>Which version of FreeBSD please? >> >> > >I test libmicro in 6.0-RELEASE and last cvsuped RELENG_6, both doesn't >work. > > >Thanks > > >Ricardo A. Reis >UNIFESP >Unix and Network Admin > > > the pthread_create benchmark is trying to create 20000 threads, the default allowed number is 1500. see sysctl kern.threads.max_threads_per_proc From owner-freebsd-performance@FreeBSD.ORG Tue Nov 29 11:24:42 2005 Return-Path: X-Original-To: performance@freebsd.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0384C16A41F for ; Tue, 29 Nov 2005 11:24:42 +0000 (GMT) (envelope-from ricardo_bsd@yahoo.com.br) Received: from maritaca.epm.br (disrouter.epm.br [200.17.25.161]) by mx1.FreeBSD.org (Postfix) with ESMTP id 44EDB43D4C for ; Tue, 29 Nov 2005 11:24:40 +0000 (GMT) (envelope-from ricardo_bsd@yahoo.com.br) Received: from localhost (localhost.localdomain [127.0.0.1]) by maritaca.epm.br (Postfix) with ESMTP id B8DCE3AC2 for ; Tue, 29 Nov 2005 09:24:38 -0200 (BRDT) Received: from [172.22.1.166] (ricardo.epm.br [172.22.1.166]) by maritaca.epm.br (Postfix) with ESMTP id 8D6113AB4 for ; Tue, 29 Nov 2005 09:24:34 -0200 (BRDT) Message-ID: <438C3A57.9020507@yahoo.com.br> Date: Tue, 29 Nov 2005 09:24:07 -0200 From: "Ricardo A. Reis" User-Agent: Mozilla Thunderbird 1.0.7 (X11/20051122) X-Accept-Language: en-us, en MIME-Version: 1.0 To: performance@freebsd.org References: <23B6BBC5.F5C7C9B@mails.de> In-Reply-To: <23B6BBC5.F5C7C9B@mails.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit UNIFESP-Virus-Scanned: by amavisd-new at dis.epm.br Cc: Subject: Re: Libmicro!! pthread_create Resource temporarily unavailable X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Nov 2005 11:24:42 -0000 >the pthread_create benchmark is trying to create 20000 threads, the >default allowed number is 1500. Hi, I increase default limits for *per_proc for 40000 sysctl -a |grep threads kern.threads.thr_concurrency: 0 kern.threads.thr_scope: 0 kern.threads.virtual_cpu: 2 kern.threads.max_threads_hits: 0 kern.threads.max_groups_per_proc: 40000 kern.threads.max_threads_per_proc: 40000 kern.threads.debug: 0 vm.stats.vm.v_kthreads: 69 But this not solution, Thanks Ricardo A. Reis UNIFESP Unix and Network Admin From owner-freebsd-performance@FreeBSD.ORG Tue Nov 29 12:17:53 2005 Return-Path: X-Original-To: performance@freebsd.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from [127.0.0.1] (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id 7435716A41F; Tue, 29 Nov 2005 12:17:52 +0000 (GMT) (envelope-from davidxu@freebsd.org) Message-ID: <438C46EB.8040306@freebsd.org> Date: Tue, 29 Nov 2005 20:17:47 +0800 From: David Xu User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.11) Gecko/20050911 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "Ricardo A. Reis" References: <23B6BBC5.F5C7C9B@mails.de> <438C3A57.9020507@yahoo.com.br> In-Reply-To: <438C3A57.9020507@yahoo.com.br> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: performance@freebsd.org Subject: Re: Libmicro!! pthread_create Resource temporarily unavailable X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Nov 2005 12:17:53 -0000 Ricardo A. Reis wrote: > > >the pthread_create benchmark is trying to create 20000 threads, the > >default allowed number is 1500. > > > Hi, > > I increase default limits for *per_proc for 40000 > > sysctl -a |grep threads > kern.threads.thr_concurrency: 0 > kern.threads.thr_scope: 0 > kern.threads.virtual_cpu: 2 > kern.threads.max_threads_hits: 0 > kern.threads.max_groups_per_proc: 40000 > kern.threads.max_threads_per_proc: 40000 > kern.threads.debug: 0 > vm.stats.vm.v_kthreads: 69 > > > But this not solution, > > > > Thanks > > > Ricardo A. Reis > UNIFESP > Unix and Network Admin You can not create 40000 threads, because each thread defaults has 2M stack, you will out of address space. David Xu From owner-freebsd-performance@FreeBSD.ORG Tue Nov 29 18:52:14 2005 Return-Path: X-Original-To: performance@freebsd.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 99A1C16A41F for ; Tue, 29 Nov 2005 18:52:14 +0000 (GMT) (envelope-from ricardo@dis.epm.br) Received: from maritaca.epm.br (disrouter.epm.br [200.17.25.161]) by mx1.FreeBSD.org (Postfix) with ESMTP id 37B7443D46 for ; Tue, 29 Nov 2005 18:52:11 +0000 (GMT) (envelope-from ricardo@dis.epm.br) Received: from localhost (localhost.localdomain [127.0.0.1]) by maritaca.epm.br (Postfix) with ESMTP id 30C3E3B58 for ; Tue, 29 Nov 2005 16:52:10 -0200 (BRDT) Received: from [172.22.1.166] (ricardo.epm.br [172.22.1.166]) by maritaca.epm.br (Postfix) with ESMTP id C4A2C3B38 for ; Tue, 29 Nov 2005 16:52:05 -0200 (BRDT) Message-ID: <438CA33A.7000007@dis.epm.br> Date: Tue, 29 Nov 2005 16:51:38 -0200 From: "Ricardo A. Reis" User-Agent: Mozilla Thunderbird 1.0.7 (X11/20051122) X-Accept-Language: en-us, en MIME-Version: 1.0 To: performance@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit UNIFESP-Virus-Scanned: by amavisd-new at dis.epm.br X-Mailman-Approved-At: Tue, 29 Nov 2005 19:58:47 +0000 Cc: Subject: Libmicro!! pthread_create Resource temporarily unavailable X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Nov 2005 18:52:14 -0000 >>You can not create 40000 threads, because each thread >>defaults has 2M stack, you will out of address space. David, I don't attempt to create 40k threads, i barely increase the limit for single process for 40k. I like a test libmicro with default sets. Ricardo A. Reis UNIFESP From owner-freebsd-performance@FreeBSD.ORG Tue Nov 29 23:41:35 2005 Return-Path: X-Original-To: performance@freebsd.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from [127.0.0.1] (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id 1D72116A41F; Tue, 29 Nov 2005 23:41:34 +0000 (GMT) (envelope-from davidxu@freebsd.org) Message-ID: <438CE72B.4090804@freebsd.org> Date: Wed, 30 Nov 2005 07:41:31 +0800 From: David Xu User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.11) Gecko/20050911 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "Ricardo A. Reis" References: <438CA33A.7000007@dis.epm.br> In-Reply-To: <438CA33A.7000007@dis.epm.br> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: performance@freebsd.org Subject: Re: Libmicro!! pthread_create Resource temporarily unavailable X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Nov 2005 23:41:35 -0000 Ricardo A. Reis wrote: > >>You can not create 40000 threads, because each thread > >>defaults has 2M stack, you will out of address space. > > David, > > I don't attempt to create 40k threads, i barely increase the limit for > single process for 40k. > I like a test libmicro with default sets. > > > > Ricardo A. Reis > UNIFESP > _______________________________________________ try to give it an option, for example: ./pthread_create -B 1499 From owner-freebsd-performance@FreeBSD.ORG Thu Dec 1 04:33:56 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1FF4616A41F for ; Thu, 1 Dec 2005 04:33:56 +0000 (GMT) (envelope-from gbaratto@superb.net) Received: from master4.yvr1.superb.net (master4.yvr1.superb.net [64.251.80.20]) by mx1.FreeBSD.org (Postfix) with ESMTP id AE2F943D53 for ; Thu, 1 Dec 2005 04:33:55 +0000 (GMT) (envelope-from gbaratto@superb.net) Received: from guinness (100.80.251.64.in-addr.arpa [64.251.80.100] (may be forged)) by master4.yvr1.superb.net (8.12.9/8.12.9) with SMTP id jB14XsaE027254 for ; Wed, 30 Nov 2005 20:33:54 -0800 (PST) Message-ID: <04f401c5f630$b9af6570$6450fb40@guinness> From: "Gustavo A. Baratto" To: Date: Wed, 30 Nov 2005 20:35:52 -0800 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2180 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 X-Mailman-Approved-At: Thu, 01 Dec 2005 04:43:42 +0000 Subject: new benchmarks. WAS: FreeBSD MySQL still WAY slower than Linux X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Dec 2005 04:33:56 -0000 following Steve Roome's benchmarks from a few months ago, here it goes a few more, on freebsd 6.0 RELEASE, using libpthreads (system, and process scopes), libthr, and linuxthreads. Unless otherwise specified, the my.cnf values are the ones posted below. Dmesg, and diff of the kernel compared with GENERIC are posted in the end as well. File systems are mounted as default (no fancy mount opts): ---- # more /etc/fstab # Device Mountpoint FStype Options Dump Pass# /dev/da0s1b none swap sw 0 0 /dev/da0s1a / ufs rw 1 1 /dev/da0s1d /tmp ufs rw 2 2 /dev/da0s1e /var ufs rw 2 2 /dev/da0s1f /home ufs rw 2 2 /dev/acd0 /cdrom cd9660 ro,noauto 0 0 ------- Cheers all ++++ lipthread: standard # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 16:20:52 EST 2005 select_index 20000 0 0 14570.99 select_index 20000 2 0 14787.32 select_index 20000 3 0 14621.65 select_index 20000 1 0 13554.19 select_index 20000 0 0 13514.42 Wed Nov 30 16:20:59 EST 2005 date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 22:45:37 EST 2005 select_index 20000 1 0 14567.60 select_index 20000 0 0 13745.24 select_index 20000 1 0 13625.50 select_index 20000 0 0 13558.84 select_index 20000 1 0 13460.09 Wed Nov 30 22:45:45 EST 2005 ++++ lipthread: LIBPTHREAD_PROCESS_SCOPE=yes # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 16:37:16 EST 2005 select_index 20000 2 0 12014.51 select_index 20000 2 0 11991.61 select_index 20000 0 0 10890.61 select_index 20000 1 0 11992.41 select_index 20000 3 0 11775.37 Wed Nov 30 16:37:24 EST 2005 date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 22:49:40 EST 2005 select_index 20000 1 0 11035.18 select_index 20000 4 0 11877.35 select_index 20000 0 0 10097.29 select_index 20000 0 0 10765.29 select_index 20000 1 0 11401.68 Wed Nov 30 22:49:50 EST 2005 ++++ libthr: # more /etc/libmap.conf [mysqld] libpthread.so.2 libthr.so.2 libpthread.so libthr.so # ldd /usr/local/libexec/mysqld /usr/local/libexec/mysqld: libz.so.3 => /lib/libz.so.3 (0x881f8000) libcrypt.so.3 => /lib/libcrypt.so.3 (0x88208000) libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0x88220000) libm.so.4 => /lib/libm.so.4 (0x882ea000) libpthread.so.2 => /usr/lib/libthr.so.2 (0x88300000) libc.so.6 => /lib/libc.so.6 (0x88312000) # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 16:38:45 EST 2005 select_index 20000 0 0 15789.56 select_index 20000 1 0 14804.51 select_index 20000 0 0 14810.69 select_index 20000 0 0 14600.25 select_index 20000 0 0 14478.79 Wed Nov 30 16:38:52 EST 2005 # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 22:47:26 EST 2005 select_index 20000 1 0 14375.95 select_index 20000 0 0 14392.69 select_index 20000 0 0 14478.93 select_index 20000 0 0 14371.66 select_index 20000 0 0 14416.07 Wed Nov 30 22:47:33 EST 2005 ++++ linuxthreads # ldd /usr/local/libexec/mysqld /usr/local/libexec/mysqld: liblthread.so.3 => /usr/local/lib/liblthread.so.3 (0x88201000) liblstdc++.so.5 => /usr/local/lib/liblstdc++.so.5 (0x88222000) libz.so.3 => /lib/libz.so.3 (0x882ed000) libcrypt.so.3 => /lib/libcrypt.so.3 (0x882fd000) libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0x88315000) libm.so.4 => /lib/libm.so.4 (0x883df000) libc.so.6 => /lib/libc.so.6 (0x883f5000) # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 23:30:24 EST 2005 select_index 20000 0 0 14357.28 select_index 20000 0 0 14055.69 select_index 20000 0 0 13694.37 select_index 20000 1 0 13394.22 select_index 20000 0 0 13723.05 Wed Nov 30 23:30:32 EST 2005 # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 23:30:38 EST 2005 select_index 20000 2 0 13555.20 select_index 20000 2 0 13500.62 select_index 20000 0 0 13546.00 select_index 20000 0 0 13161.49 select_index 20000 0 0 13510.82 Wed Nov 30 23:30:46 EST 2005 ++++ linuxthreads with query cache disabled (query_cache_size commented out in my.cnf):: # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 21:59:51 EST 2005 select_index 20000 1 0 8615.98 select_index 20000 3 0 8700.63 select_index 20000 1 0 8590.18 select_index 20000 1 0 8733.90 select_index 20000 1 0 8582.01 Wed Nov 30 22:00:02 EST 2005 ++++ libthr with query cache disabled (query_cache_size commented out in my.cnf): date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 22:52:42 EST 2005 select_index 20000 1 0 9511.13 select_index 20000 1 0 9464.96 select_index 20000 1 0 9229.52 select_index 20000 1 0 9195.97 select_index 20000 1 0 9346.92 Wed Nov 30 22:52:53 EST 2005 ++++ libthr, connecting to a local mysql using TCP, instead of socket # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 22:57:03 EST 2005 select_index 20000 1 0 9286.62 select_index 20000 2 0 9245.66 select_index 20000 1 0 9126.25 select_index 20000 0 0 9303.66 select_index 20000 1 0 9286.14 Wed Nov 30 22:57:15 EST 2005 ++++ linuxthreads, connecting to a local mysql using TCP, instead of socket # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 23:07:16 EST 2005 select_index 20000 2 0 8137.02 select_index 20000 2 0 8220.83 select_index 20000 3 0 8264.19 select_index 20000 1 0 8268.86 select_index 20000 3 0 8234.24 Wed Nov 30 23:07:28 EST 2005 ++++ my.cnf ---------------- [mysqld] server-id=2 log-bin=/var/log/mysql/binlog log-slow-queries=/var/log/mysql/slow-queries.log relay-log=/var/log/mysql/relay-log relay-log-index=/var/log/mysql/relay-log-index skip-innodb skip-bdb safe-user-create skip-name-resolve skip-log-warnings safe-show-database set-variable = flush_time=86400 set-variable = max_connections=300 set-variable = max_connect_errors=100 set-variable = interactive_timeout=600 set-variable = wait_timeout=60 set-variable = myisam-recover=FORCE set-variable = key_buffer_size=400M set-variable = sort_buffer_size=1M set-variable = read_buffer_size=512k set-variable = table_cache=4096 set-variable = max_delayed_threads=0 set-variable = max_user_connections=100 set-variable = query_cache_size=300M set-variable = thread_cache_size=50 [mysql.server] user=mysql [safe_mysqld] err-log=/var/log/mysql/error.log ------------------------ ++++ DMESG: ---------- # dmesg Copyright (c) 1992-2005 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 6.0-RELEASE #1: Tue Nov 29 23:36:08 EST 2005 gbaratto@sh-mysql4.dca1.superb.net:/usr/obj/usr/src/sys/GBARATTO Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Xeon(TM) CPU 3.06GHz (3066.79-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf29 Stepping = 9 Features=0xbfebfbff Features2=0x4400> Hyperthreading: 2 logical CPUs real memory = 1073741824 (1024 MB) avail memory = 1041698816 (993 MB) MPTable: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 6 ioapic0: Assuming intbase of 0 ioapic1: Assuming intbase of 24 ioapic2: Assuming intbase of 48 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard ioapic2 irqs 48-71 on motherboard npx0: [FAST] npx0: on motherboard npx0: INT 16 interface cpu0 on motherboard cpu1 on motherboard pcib0: pcibus 0 on motherboard pci0: on pcib0 pcib1: at device 2.0 on pci0 pci2: on pcib1 pci2: at device 28.0 (no driver attached) pcib2: at device 29.0 on pci2 pci5: on pcib2 em0: port 0xa400-0xa43f mem 0xfe9c0000-0xfe9dffff irq 48 at device 1.0 on pci5 em0: Ethernet address: 00:e0:81:2a:48:64 em0: Speed:N/A Duplex:N/A em1: port 0xa800-0xa83f mem 0xfe9e0000-0xfe9fffff irq 49 at device 1.1 on pci5 em1: Ethernet address: 00:e0:81:2a:48:65 em1: Speed:N/A Duplex:N/A pci2: at device 30.0 (no driver attached) pcib3: at device 31.0 on pci2 pci3: on pcib3 asr0: mem 0xfa000000-0xfbffffff irq 24 at device 3.0 on pci3 asr0: [GIANT-LOCKED] asr0: ADAPTEC 2110S FW Rev. 380E, 1 channel, 256 CCBs, Protocol I2O pcib4: at device 3.1 on pci3 pci4: on pcib4 uhci0: port 0xc400-0xc41f irq 16 at device 29.0 on pci0 uhci0: [GIANT-LOCKED] usb0: on uhci0 usb0: USB revision 1.0 uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered uhci1: port 0xc800-0xc81f irq 19 at device 29.1 on pci0 uhci1: [GIANT-LOCKED] usb1: on uhci1 usb1: USB revision 1.0 uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub1: 2 ports with 2 removable, self powered ehci0: mem 0xfebffc00-0xfebfffff irq 23 at device 29.7 on pci0 ehci0: [GIANT-LOCKED] usb2: waiting for BIOS to give up control usb2: timed out waiting for BIOS usb2: EHCI version 1.0 usb2: companion controllers, 2 ports each: usb0 usb1 usb2: on ehci0 usb2: USB revision 2.0 uhub2: Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1 uhub2: 4 ports with 4 removable, self powered pcib5: at device 30.0 on pci0 pci1: on pcib5 fxp0: port 0x9800-0x983f mem 0xfe6fe000-0xfe6fefff,0xfe6a0000-0xfe6bffff irq 17 at device 1.0 on pci1 miibus0: on fxp0 inphy0: on miibus0 inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto fxp0: Ethernet address: 00:e0:81:2a:48:35 pci1: at device 2.0 (no driver attached) isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfc00-0xfc0f at device 31.1 on pci0 ata0: on atapci0 ata1: on atapci0 pci0: at device 31.3 (no driver attached) pmtimer0 on isa0 orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xcdfff on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] fdc0: at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0 fdc0: [FAST] fd0: <1440-KB 3.5" drive> on fdc0 drive 0 ppc0: parallel port not found. sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio0: configured irq 4 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 sio0: type 16550A sio1: configured irq 3 not in bitmap of probed irqs 0 sio1: port may not be enabled sio1 at port 0x2f8-0x2ff irq 3 on isa0 sio1: type 16550A vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 unknown: can't assign resources (memory) unknown: can't assign resources (port) unknown: can't assign resources (port) unknown: can't assign resources (port) unknown: can't assign resources (port) Timecounters tick every 1.000 msec acd0: CDROM at ata1-master PIO4 da0 at asr0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-2 device da0: Tagged Queueing Enabled da0: 70006MB (143372288 512 byte sectors: 255H 63S/T 8924C) SMP: AP CPU #1 Launched! Trying to mount root from ufs:/dev/da0s1a em0: link state changed to UP ---------------- ++++ DIFF KERNEL # diff -bu /usr/src/sys/i386/conf/GENERIC /usr/src/sys/i386/conf/GBARATTO --- /usr/src/sys/i386/conf/GENERIC Fri Oct 28 15:22:41 2005 +++ /usr/src/sys/i386/conf/GBARATTO Tue Nov 29 23:29:19 2005 @@ -19,10 +19,8 @@ # $FreeBSD: src/sys/i386/conf/GENERIC,v 1.429.2.3.2.1 2005/10/28 19:22:41 jhb Exp $ machine i386 -cpu I486_CPU -cpu I586_CPU cpu I686_CPU -ident GENERIC +ident GBARATTO # To statically compile in device wiring instead of /boot/device.hints #hints "GENERIC.hints" # Default places to look for devices. @@ -64,6 +62,26 @@ options ADAPTIVE_GIANT # Giant mutex is adaptive. device apic # I/O APIC + +# ADDED BY GUSTAVO +options SMP + +device carp + +options ALTQ +options ALTQ_CBQ +options ALTQ_RED +options ALTQ_RIO +options ALTQ_HFSC +options ALTQ_CDNR +options ALTQ_PRIQ +options ALTQ_NOPCC # just for SMP builds + +options MAXDSIZ=(2048UL*1024*1024) #added by gustavo +options MAXSSIZ=(512UL*1024*1024) #added by gustavo +options DFLDSIZ=(2048UL*1024*1024) #added by gustavo +# + # Bus support. device eisa From owner-freebsd-performance@FreeBSD.ORG Thu Dec 1 05:25:49 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A737016A45E for ; Thu, 1 Dec 2005 05:25:49 +0000 (GMT) (envelope-from delphij@gmail.com) Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.199]) by mx1.FreeBSD.org (Postfix) with ESMTP id D2F4143D82 for ; Thu, 1 Dec 2005 05:24:48 +0000 (GMT) (envelope-from delphij@gmail.com) Received: by wproxy.gmail.com with SMTP id i11so280922wra for ; Wed, 30 Nov 2005 21:24:39 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=aEMYdrBwKvj7wOc8vRAgfqddtYuJXKVjCp5H16OOMZ62rVZbLH0WNSnK4NscR1w0CtNxWxgbXdvfQ84CjVUkHbNRaFwmDK9AWmt4I1hC+euv0jj3FA4oV8STKJ9DltVSjyguzTLjyFmAmVpC688/Pr+0g94PrlCGtq4mahdBTKM= Received: by 10.64.195.11 with SMTP id s11mr615711qbf; Wed, 30 Nov 2005 21:24:38 -0800 (PST) Received: by 10.65.72.5 with HTTP; Wed, 30 Nov 2005 21:24:38 -0800 (PST) Message-ID: Date: Thu, 1 Dec 2005 13:24:38 +0800 From: Xin LI To: "Gustavo A. Baratto" In-Reply-To: <04f401c5f630$b9af6570$6450fb40@guinness> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: base64 Content-Disposition: inline References: <04f401c5f630$b9af6570$6450fb40@guinness> Cc: freebsd-performance@freebsd.org Subject: Re: new benchmarks. WAS: FreeBSD MySQL still WAY slower than Linux X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: delphij@delphij.net List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Dec 2005 05:25:49 -0000 SGksCgpPbiAxMi8xLzA1LCBHdXN0YXZvIEEuIEJhcmF0dG8gPGdiYXJhdHRvQHN1cGVyYi5uZXQ+ IHdyb3RlOgpbc25pcF0KPiArKysrIERNRVNHOgo+IC0tLS0tLS0tLS0KPiAgRmVhdHVyZXMyPTB4 NDQwMDxDTlRYLUlELDxiMTQ+Pgo+ICBIeXBlcnRocmVhZGluZzogMiBsb2dpY2FsIENQVXMKCldv dWxkIHlvdSBoYXZlIGEgdHJ5IG9mIGRpc2FibGluZyBIVFQ/ICBJdCBoYXJtcyBwZXJmb3JtYW5j ZSBmb3IKY2VydGFpbiBhcHBsaWNhdGlvbnMuCgpDaGVlcnMsCi0tClhpbiBMSSA8ZGVscGhpakBk ZWxwaGlqLm5ldD4gaHR0cDovL3d3dy5kZWxwaGlqLm5ldAo= From owner-freebsd-performance@FreeBSD.ORG Thu Dec 1 18:57:03 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 13C2416A41F for ; Thu, 1 Dec 2005 18:57:03 +0000 (GMT) (envelope-from gbaratto@superb.net) Received: from master4.yvr1.superb.net (master4.yvr1.superb.net [64.251.80.20]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2136C43D8B for ; Thu, 1 Dec 2005 18:56:53 +0000 (GMT) (envelope-from gbaratto@superb.net) Received: from guinness (100.80.251.64.in-addr.arpa [64.251.80.100] (may be forged)) by master4.yvr1.superb.net (8.12.9/8.12.9) with SMTP id jB1IuiFR021027; Thu, 1 Dec 2005 10:56:44 -0800 (PST) Message-ID: <05c901c5f6a9$43834730$6450fb40@guinness> From: "Gustavo A. Baratto" To: References: <04f401c5f630$b9af6570$6450fb40@guinness> Date: Thu, 1 Dec 2005 10:58:43 -0800 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="UTF-8"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2180 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 Cc: freebsd-performance@freebsd.org Subject: Re: new benchmarks. WAS: FreeBSD MySQL still WAY slower than Linux X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Dec 2005 18:57:03 -0000 The numbers are roughly the same with HTT disabled. For example for libthr: HTT disabled: ----------- # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Thu Dec 1 13:50:24 EST 2005 select_index 20000 1 0 14749.36 select_index 20000 0 0 14838.91 select_index 20000 0 0 14913.73 select_index 20000 2 0 14230.60 select_index 20000 1 0 14288.65 Thu Dec 1 13:50:31 EST 2005 ----------- HTT enabled: ----------- # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 22:47:26 EST 2005 select_index 20000 1 0 14375.95 select_index 20000 0 0 14392.69 select_index 20000 0 0 14478.93 select_index 20000 0 0 14371.66 select_index 20000 0 0 14416.07 Wed Nov 30 22:47:33 EST 2005 ----------- For linuxthreads and libpthread, the numbers are all within the same margin as the other post as well. Cheers ----- Original Message ----- From: "Xin LI" To: "Gustavo A. Baratto" Cc: Sent: Wednesday, November 30, 2005 9:24 PM Subject: Re: new benchmarks. WAS: FreeBSD MySQL still WAY slower than Linux > Hi, > > On 12/1/05, Gustavo A. Baratto wrote: > [snip] >> ++++ DMESG: >> ---------- >> Features2=0x4400> >> Hyperthreading: 2 logical CPUs > > Would you have a try of disabling HTT? It harms performance for > certain applications. > > Cheers, > -- > Xin LI http://www.delphij.net > -------------------------------------------------------------------------------- > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to > "freebsd-performance-unsubscribe@freebsd.org" From owner-freebsd-performance@FreeBSD.ORG Thu Dec 1 19:09:38 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CF5F316A41F for ; Thu, 1 Dec 2005 19:09:38 +0000 (GMT) (envelope-from nevans@talkpoint.com) Received: from relay.talkpoint.com (pobox.talkpoint.com [204.141.15.158]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5E62C43D58 for ; Thu, 1 Dec 2005 19:09:35 +0000 (GMT) (envelope-from nevans@talkpoint.com) Received: from ASSP-nospam ([127.0.0.1]) by relay.talkpoint.com with Microsoft SMTPSVC(5.0.2195.6713); Thu, 1 Dec 2005 14:09:33 -0500 Received: from 204.141.15.194 ([204.141.15.194] helo=pleiades.nextvenue.com) by ASSP-nospam ; 1 Dec 05 19:09:33 -0000 Date: Thu, 1 Dec 2005 14:09:31 -0500 From: Nick Evans To: "Gustavo A. Baratto" Message-ID: <20051201140931.702b6efe@pleiades.nextvenue.com> In-Reply-To: <05c901c5f6a9$43834730$6450fb40@guinness> References: <04f401c5f630$b9af6570$6450fb40@guinness> <05c901c5f6a9$43834730$6450fb40@guinness> X-Mailer: Sylpheed-Claws 1.9.15 (GTK+ 2.6.10; i386-portbld-freebsd6.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 01 Dec 2005 19:09:34.0011 (UTC) FILETIME=[C3D1D8B0:01C5F6AA] Cc: freebsd-performance@freebsd.org Subject: Re: new benchmarks. WAS: FreeBSD MySQL still WAY slower than Linux X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Dec 2005 19:09:38 -0000 On Thu, 1 Dec 2005 10:58:43 -0800 "Gustavo A. Baratto" wrote: > The numbers are roughly the same with HTT disabled. For example for libthr: > > HTT disabled: > ----------- > # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | > grep select_index; done; date > Thu Dec 1 13:50:24 EST 2005 > select_index 20000 1 0 14749.36 > select_index 20000 0 0 14838.91 > select_index 20000 0 0 14913.73 > select_index 20000 2 0 14230.60 > select_index 20000 1 0 14288.65 > Thu Dec 1 13:50:31 EST 2005 > ----------- > > > HTT enabled: > ----------- > # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | > grep select_index; done; date > Wed Nov 30 22:47:26 EST 2005 > select_index 20000 1 0 14375.95 > select_index 20000 0 0 14392.69 > select_index 20000 0 0 14478.93 > select_index 20000 0 0 14371.66 > select_index 20000 0 0 14416.07 > Wed Nov 30 22:47:33 EST 2005 > ----------- > > > For linuxthreads and libpthread, the numbers are all within the same margin > as the other post as well. > > Cheers > > > ----- Original Message ----- > From: "Xin LI" > To: "Gustavo A. Baratto" > Cc: > Sent: Wednesday, November 30, 2005 9:24 PM > Subject: Re: new benchmarks. WAS: FreeBSD MySQL still WAY slower than Linux > > > > Hi, > > > > On 12/1/05, Gustavo A. Baratto wrote: > > [snip] > >> ++++ DMESG: > >> ---------- > >> Features2=0x4400> > >> Hyperthreading: 2 logical CPUs > > > > Would you have a try of disabling HTT? It harms performance for > > certain applications. > > > > Cheers, > > -- > > Xin LI http://www.delphij.net > > > > > -------------------------------------------------------------------------------- > > > > _______________________________________________ > > freebsd-performance@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > > To unsubscribe, send any mail to > > "freebsd-performance-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to > "freebsd-performance-unsubscribe@freebsd.org" Are you running super-smack on localhost or from remote boxes? From owner-freebsd-performance@FreeBSD.ORG Thu Dec 1 19:16:28 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 06F3016A41F for ; Thu, 1 Dec 2005 19:16:28 +0000 (GMT) (envelope-from gbaratto@superb.net) Received: from master4.yvr1.superb.net (master4.yvr1.superb.net [64.251.80.20]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9E1A543D4C for ; Thu, 1 Dec 2005 19:16:27 +0000 (GMT) (envelope-from gbaratto@superb.net) Received: from guinness (100.80.251.64.in-addr.arpa [64.251.80.100] (may be forged)) by master4.yvr1.superb.net (8.12.9/8.12.9) with SMTP id jB1JGQFR023524; Thu, 1 Dec 2005 11:16:26 -0800 (PST) Message-ID: <061701c5f6ac$03f98860$6450fb40@guinness> From: "Gustavo A. Baratto" To: "Nick Evans" References: <04f401c5f630$b9af6570$6450fb40@guinness><05c901c5f6a9$43834730$6450fb40@guinness> <20051201140931.702b6efe@pleiades.nextvenue.com> Date: Thu, 1 Dec 2005 11:18:25 -0800 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2180 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 Cc: freebsd-performance@freebsd.org Subject: Re: new benchmarks. WAS: FreeBSD MySQL still WAY slower than Linux X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Dec 2005 19:16:28 -0000 Localhost using unix socket. In my first post in this thread, there are some numbers for connections from localhost, but using TCP instead of socket. ----- Original Message ----- From: "Nick Evans" To: "Gustavo A. Baratto" Cc: Sent: Thursday, December 01, 2005 11:09 AM Subject: Re: new benchmarks. WAS: FreeBSD MySQL still WAY slower than Linux > On Thu, 1 Dec 2005 10:58:43 -0800 > "Gustavo A. Baratto" wrote: > >> The numbers are roughly the same with HTT disabled. For example for >> libthr: >> >> HTT disabled: >> ----------- >> # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 >> | >> grep select_index; done; date >> Thu Dec 1 13:50:24 EST 2005 >> select_index 20000 1 0 14749.36 >> select_index 20000 0 0 14838.91 >> select_index 20000 0 0 14913.73 >> select_index 20000 2 0 14230.60 >> select_index 20000 1 0 14288.65 >> Thu Dec 1 13:50:31 EST 2005 >> ----------- >> >> >> HTT enabled: >> ----------- >> # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 >> | >> grep select_index; done; date >> Wed Nov 30 22:47:26 EST 2005 >> select_index 20000 1 0 14375.95 >> select_index 20000 0 0 14392.69 >> select_index 20000 0 0 14478.93 >> select_index 20000 0 0 14371.66 >> select_index 20000 0 0 14416.07 >> Wed Nov 30 22:47:33 EST 2005 >> ----------- >> >> >> For linuxthreads and libpthread, the numbers are all within the same >> margin >> as the other post as well. >> >> Cheers >> >> >> ----- Original Message ----- >> From: "Xin LI" >> To: "Gustavo A. Baratto" >> Cc: >> Sent: Wednesday, November 30, 2005 9:24 PM >> Subject: Re: new benchmarks. WAS: FreeBSD MySQL still WAY slower than >> Linux >> >> >> > Hi, >> > >> > On 12/1/05, Gustavo A. Baratto wrote: >> > [snip] >> >> ++++ DMESG: >> >> ---------- >> >> Features2=0x4400> >> >> Hyperthreading: 2 logical CPUs >> > >> > Would you have a try of disabling HTT? It harms performance for >> > certain applications. >> > >> > Cheers, >> > -- >> > Xin LI http://www.delphij.net >> > >> >> >> -------------------------------------------------------------------------------- >> >> >> > _______________________________________________ >> > freebsd-performance@freebsd.org mailing list >> > http://lists.freebsd.org/mailman/listinfo/freebsd-performance >> > To unsubscribe, send any mail to >> > "freebsd-performance-unsubscribe@freebsd.org" >> >> _______________________________________________ >> freebsd-performance@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >> To unsubscribe, send any mail to >> "freebsd-performance-unsubscribe@freebsd.org" > > > Are you running super-smack on localhost or from remote boxes? > From owner-freebsd-performance@FreeBSD.ORG Fri Dec 2 02:24:13 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EE29716A41F for ; Fri, 2 Dec 2005 02:24:13 +0000 (GMT) (envelope-from mv@roq.com) Received: from vault.mel.jumbuck.com (ppp166-27.static.internode.on.net [150.101.166.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1435C43D4C for ; Fri, 2 Dec 2005 02:23:54 +0000 (GMT) (envelope-from mv@roq.com) Received: from vault.mel.jumbuck.com (localhost [127.0.0.1]) by vault.mel.jumbuck.com (Postfix) with ESMTP id 995608A02B; Fri, 2 Dec 2005 13:23:36 +1100 (EST) Received: from [192.168.46.52] (unknown [192.168.46.250]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by vault.mel.jumbuck.com (Postfix) with ESMTP id 397AF8A00A; Fri, 2 Dec 2005 13:23:36 +1100 (EST) Message-ID: <438FB036.3000804@roq.com> Date: Fri, 02 Dec 2005 13:23:50 +1100 From: Michael Vince User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.7.12) Gecko/20051127 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "Gustavo A. Baratto" References: <04f401c5f630$b9af6570$6450fb40@guinness> In-Reply-To: <04f401c5f630$b9af6570$6450fb40@guinness> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV using ClamSMTP Cc: freebsd-performance@freebsd.org Subject: Re: new benchmarks. WAS: FreeBSD MySQL still WAY slower than Linux X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Dec 2005 02:24:14 -0000 It would be good to actually see the Linux performance on the exact same hardware, all we ever see is how it is on FreeBSD. MySQL has very frequent use of queries of the system time and is well known in FreeBSD to be slower because its more expensive to call compared to Linux, which has less precision time. If you are on modern hardware you might want to try changing from kern.timecounter.hardware: ACPI-fast to TSC(-100) and doing a benchmark this has already proven to increase performance of MySQL by a significant amount. Also some new experimental low-precision time code has been added to current source tree to see how much performance increases can be gained some people are against it and I don't know if its usable for tests yet. Mike Gustavo A. Baratto wrote: > following Steve Roome's benchmarks from a few months ago, here it > goes a few more, on freebsd 6.0 RELEASE, using libpthreads (system, > and process scopes), libthr, and linuxthreads. Unless otherwise > specified, the my.cnf values are the ones posted below. Dmesg, and > diff of the kernel compared with GENERIC are posted in the end as > well. File systems are mounted as default (no fancy mount opts): > ---- > # more /etc/fstab > # Device Mountpoint FStype Options Dump > Pass# > /dev/da0s1b none swap sw 0 0 > /dev/da0s1a / ufs rw 1 1 > /dev/da0s1d /tmp ufs rw 2 2 > /dev/da0s1e /var ufs rw 2 2 > /dev/da0s1f /home ufs rw 2 2 > /dev/acd0 /cdrom cd9660 ro,noauto 0 0 > ------- > > Cheers all > > > ++++ lipthread: standard > # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 > 1000 | grep select_index; done; date > Wed Nov 30 16:20:52 EST 2005 > select_index 20000 0 0 14570.99 > select_index 20000 2 0 14787.32 > select_index 20000 3 0 14621.65 > select_index 20000 1 0 13554.19 > select_index 20000 0 0 13514.42 > Wed Nov 30 16:20:59 EST 2005 > > date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 > 1000 | grep select_index; done; date > Wed Nov 30 22:45:37 EST 2005 > select_index 20000 1 0 14567.60 > select_index 20000 0 0 13745.24 > select_index 20000 1 0 13625.50 > select_index 20000 0 0 13558.84 > select_index 20000 1 0 13460.09 > Wed Nov 30 22:45:45 EST 2005 > > > > > ++++ lipthread: LIBPTHREAD_PROCESS_SCOPE=yes > # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 > 1000 | grep select_index; done; date > Wed Nov 30 16:37:16 EST 2005 > select_index 20000 2 0 12014.51 > select_index 20000 2 0 11991.61 > select_index 20000 0 0 10890.61 > select_index 20000 1 0 11992.41 > select_index 20000 3 0 11775.37 > Wed Nov 30 16:37:24 EST 2005 > > date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 > 1000 | grep select_index; done; date > Wed Nov 30 22:49:40 EST 2005 > select_index 20000 1 0 11035.18 > select_index 20000 4 0 11877.35 > select_index 20000 0 0 10097.29 > select_index 20000 0 0 10765.29 > select_index 20000 1 0 11401.68 > Wed Nov 30 22:49:50 EST 2005 > > > > > ++++ libthr: > # more /etc/libmap.conf > [mysqld] > libpthread.so.2 libthr.so.2 > libpthread.so libthr.so > > # ldd /usr/local/libexec/mysqld > /usr/local/libexec/mysqld: > libz.so.3 => /lib/libz.so.3 (0x881f8000) > libcrypt.so.3 => /lib/libcrypt.so.3 (0x88208000) > libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0x88220000) > libm.so.4 => /lib/libm.so.4 (0x882ea000) > libpthread.so.2 => /usr/lib/libthr.so.2 (0x88300000) > libc.so.6 => /lib/libc.so.6 (0x88312000) > > # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 > 1000 | grep select_index; done; date > Wed Nov 30 16:38:45 EST 2005 > select_index 20000 0 0 15789.56 > select_index 20000 1 0 14804.51 > select_index 20000 0 0 14810.69 > select_index 20000 0 0 14600.25 > select_index 20000 0 0 14478.79 > Wed Nov 30 16:38:52 EST 2005 > > # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 > 1000 | grep select_index; done; date > Wed Nov 30 22:47:26 EST 2005 > select_index 20000 1 0 14375.95 > select_index 20000 0 0 14392.69 > select_index 20000 0 0 14478.93 > select_index 20000 0 0 14371.66 > select_index 20000 0 0 14416.07 > Wed Nov 30 22:47:33 EST 2005 > > > > > ++++ linuxthreads > # ldd /usr/local/libexec/mysqld > /usr/local/libexec/mysqld: > liblthread.so.3 => /usr/local/lib/liblthread.so.3 (0x88201000) > liblstdc++.so.5 => /usr/local/lib/liblstdc++.so.5 (0x88222000) > libz.so.3 => /lib/libz.so.3 (0x882ed000) > libcrypt.so.3 => /lib/libcrypt.so.3 (0x882fd000) > libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0x88315000) > libm.so.4 => /lib/libm.so.4 (0x883df000) > libc.so.6 => /lib/libc.so.6 (0x883f5000) > > # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 > 1000 | grep select_index; done; date > Wed Nov 30 23:30:24 EST 2005 > select_index 20000 0 0 14357.28 > select_index 20000 0 0 14055.69 > select_index 20000 0 0 13694.37 > select_index 20000 1 0 13394.22 > select_index 20000 0 0 13723.05 > Wed Nov 30 23:30:32 EST 2005 > > # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 > 1000 | grep select_index; done; date > Wed Nov 30 23:30:38 EST 2005 > select_index 20000 2 0 13555.20 > select_index 20000 2 0 13500.62 > select_index 20000 0 0 13546.00 > select_index 20000 0 0 13161.49 > select_index 20000 0 0 13510.82 > Wed Nov 30 23:30:46 EST 2005 > > > > > ++++ linuxthreads with query cache disabled (query_cache_size > commented out in my.cnf):: > # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 > 1000 | grep select_index; done; date > Wed Nov 30 21:59:51 EST 2005 > select_index 20000 1 0 8615.98 > select_index 20000 3 0 8700.63 > select_index 20000 1 0 8590.18 > select_index 20000 1 0 8733.90 > select_index 20000 1 0 8582.01 > Wed Nov 30 22:00:02 EST 2005 > > > ++++ libthr with query cache disabled (query_cache_size commented out > in my.cnf): > date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 > 1000 | grep select_index; done; date > Wed Nov 30 22:52:42 EST 2005 > select_index 20000 1 0 9511.13 > select_index 20000 1 0 9464.96 > select_index 20000 1 0 9229.52 > select_index 20000 1 0 9195.97 > select_index 20000 1 0 9346.92 > Wed Nov 30 22:52:53 EST 2005 > > > > ++++ libthr, connecting to a local mysql using TCP, instead of socket > # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 > 1000 | grep select_index; done; date > Wed Nov 30 22:57:03 EST 2005 > select_index 20000 1 0 9286.62 > select_index 20000 2 0 9245.66 > select_index 20000 1 0 9126.25 > select_index 20000 0 0 9303.66 > select_index 20000 1 0 9286.14 > Wed Nov 30 22:57:15 EST 2005 > > ++++ linuxthreads, connecting to a local mysql using TCP, instead of > socket > # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 > 1000 | grep select_index; done; date > Wed Nov 30 23:07:16 EST 2005 > select_index 20000 2 0 8137.02 > select_index 20000 2 0 8220.83 > select_index 20000 3 0 8264.19 > select_index 20000 1 0 8268.86 > select_index 20000 3 0 8234.24 > Wed Nov 30 23:07:28 EST 2005 > > > > ++++ my.cnf > ---------------- > [mysqld] > server-id=2 > log-bin=/var/log/mysql/binlog > log-slow-queries=/var/log/mysql/slow-queries.log > relay-log=/var/log/mysql/relay-log > relay-log-index=/var/log/mysql/relay-log-index > skip-innodb > skip-bdb > safe-user-create > skip-name-resolve > skip-log-warnings > safe-show-database > > > set-variable = flush_time=86400 > set-variable = max_connections=300 > set-variable = max_connect_errors=100 > set-variable = interactive_timeout=600 > set-variable = wait_timeout=60 > set-variable = myisam-recover=FORCE > set-variable = key_buffer_size=400M > set-variable = sort_buffer_size=1M > set-variable = read_buffer_size=512k > set-variable = table_cache=4096 > set-variable = max_delayed_threads=0 > set-variable = max_user_connections=100 > set-variable = query_cache_size=300M > set-variable = thread_cache_size=50 > > > [mysql.server] > user=mysql > > [safe_mysqld] > err-log=/var/log/mysql/error.log > ------------------------ > > > > ++++ DMESG: > ---------- > # dmesg > Copyright (c) 1992-2005 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > FreeBSD 6.0-RELEASE #1: Tue Nov 29 23:36:08 EST 2005 > gbaratto@sh-mysql4.dca1.superb.net:/usr/obj/usr/src/sys/GBARATTO > Timecounter "i8254" frequency 1193182 Hz quality 0 > CPU: Intel(R) Xeon(TM) CPU 3.06GHz (3066.79-MHz 686-class CPU) > Origin = "GenuineIntel" Id = 0xf29 Stepping = 9 > Features=0xbfebfbff > > Features2=0x4400> > Hyperthreading: 2 logical CPUs > real memory = 1073741824 (1024 MB) > avail memory = 1041698816 (993 MB) > MPTable: > FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs > cpu0 (BSP): APIC ID: 0 > cpu1 (AP): APIC ID: 6 > ioapic0: Assuming intbase of 0 > ioapic1: Assuming intbase of 24 > ioapic2: Assuming intbase of 48 > ioapic0 irqs 0-23 on motherboard > ioapic1 irqs 24-47 on motherboard > ioapic2 irqs 48-71 on motherboard > npx0: [FAST] > npx0: on motherboard > npx0: INT 16 interface > cpu0 on motherboard > cpu1 on motherboard > pcib0: pcibus 0 on motherboard > pci0: on pcib0 > pcib1: at device 2.0 on pci0 > pci2: on pcib1 > pci2: at device 28.0 (no > driver attached) > pcib2: at device 29.0 on pci2 > pci5: on pcib2 > em0: port > 0xa400-0xa43f mem 0xfe9c0000-0xfe9dffff irq 48 at device 1.0 on pci5 > em0: Ethernet address: 00:e0:81:2a:48:64 > em0: Speed:N/A Duplex:N/A > em1: port > 0xa800-0xa83f mem 0xfe9e0000-0xfe9fffff irq 49 at device 1.1 on pci5 > em1: Ethernet address: 00:e0:81:2a:48:65 > em1: Speed:N/A Duplex:N/A > pci2: at device 30.0 (no > driver attached) > pcib3: at device 31.0 on pci2 > pci3: on pcib3 > asr0: mem 0xfa000000-0xfbffffff irq 24 at > device 3.0 on pci3 > asr0: [GIANT-LOCKED] > asr0: ADAPTEC 2110S FW Rev. 380E, 1 channel, 256 CCBs, Protocol I2O > pcib4: at device 3.1 on pci3 > pci4: on pcib4 > uhci0: port 0xc400-0xc41f > irq 16 at device 29.0 on pci0 > uhci0: [GIANT-LOCKED] > usb0: on uhci0 > usb0: USB revision 1.0 > uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 > uhub0: 2 ports with 2 removable, self powered > uhci1: port 0xc800-0xc81f > irq 19 at device 29.1 on pci0 > uhci1: [GIANT-LOCKED] > usb1: on uhci1 > usb1: USB revision 1.0 > uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 > uhub1: 2 ports with 2 removable, self powered > ehci0: mem 0xfebffc00-0xfebfffff > irq 23 at device 29.7 on pci0 > ehci0: [GIANT-LOCKED] > usb2: waiting for BIOS to give up control > usb2: timed out waiting for BIOS > usb2: EHCI version 1.0 > usb2: companion controllers, 2 ports each: usb0 usb1 > usb2: on ehci0 > usb2: USB revision 2.0 > uhub2: Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1 > uhub2: 4 ports with 4 removable, self powered > pcib5: at device 30.0 on pci0 > pci1: on pcib5 > fxp0: port 0x9800-0x983f mem > 0xfe6fe000-0xfe6fefff,0xfe6a0000-0xfe6bffff irq 17 at device 1.0 on pci1 > miibus0: on fxp0 > inphy0: on miibus0 > inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto > fxp0: Ethernet address: 00:e0:81:2a:48:35 > pci1: at device 2.0 (no driver attached) > isab0: at device 31.0 on pci0 > isa0: on isab0 > atapci0: port > 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfc00-0xfc0f at device 31.1 on pci0 > ata0: on atapci0 > ata1: on atapci0 > pci0: at device 31.3 (no driver attached) > pmtimer0 on isa0 > orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xcdfff on isa0 > atkbdc0: at port 0x60,0x64 on isa0 > atkbd0: irq 1 on atkbdc0 > kbd0 at atkbd0 > atkbd0: [GIANT-LOCKED] > fdc0: at port 0x3f0-0x3f5,0x3f7 irq 6 drq > 2 on isa0 > fdc0: [FAST] > fd0: <1440-KB 3.5" drive> on fdc0 drive 0 > ppc0: parallel port not found. > sc0: at flags 0x100 on isa0 > sc0: VGA <16 virtual consoles, flags=0x300> > sio0: configured irq 4 not in bitmap of probed irqs 0 > sio0: port may not be enabled > sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 > sio0: type 16550A > sio1: configured irq 3 not in bitmap of probed irqs 0 > sio1: port may not be enabled > sio1 at port 0x2f8-0x2ff irq 3 on isa0 > sio1: type 16550A > vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 > unknown: can't assign resources (memory) > unknown: can't assign resources (port) > unknown: can't assign resources (port) > unknown: can't assign resources (port) > unknown: can't assign resources (port) > Timecounters tick every 1.000 msec > acd0: CDROM at ata1-master PIO4 > da0 at asr0 bus 0 target 0 lun 0 > da0: Fixed Direct Access SCSI-2 device > da0: Tagged Queueing Enabled > da0: 70006MB (143372288 512 byte sectors: 255H 63S/T 8924C) > SMP: AP CPU #1 Launched! > Trying to mount root from ufs:/dev/da0s1a > em0: link state changed to UP > ---------------- > > > ++++ DIFF KERNEL > # diff -bu /usr/src/sys/i386/conf/GENERIC /usr/src/sys/i386/conf/GBARATTO > --- /usr/src/sys/i386/conf/GENERIC Fri Oct 28 15:22:41 2005 > +++ /usr/src/sys/i386/conf/GBARATTO Tue Nov 29 23:29:19 2005 > @@ -19,10 +19,8 @@ > # $FreeBSD: src/sys/i386/conf/GENERIC,v 1.429.2.3.2.1 2005/10/28 > 19:22:41 jhb Exp $ > > machine i386 > -cpu I486_CPU > -cpu I586_CPU > cpu I686_CPU > -ident GENERIC > +ident GBARATTO > > # To statically compile in device wiring instead of /boot/device.hints > #hints "GENERIC.hints" # Default places to look for > devices. > @@ -64,6 +62,26 @@ > options ADAPTIVE_GIANT # Giant mutex is adaptive. > > device apic # I/O APIC > + > +# ADDED BY GUSTAVO > +options SMP > + > +device carp > + > +options ALTQ > +options ALTQ_CBQ > +options ALTQ_RED > +options ALTQ_RIO > +options ALTQ_HFSC > +options ALTQ_CDNR > +options ALTQ_PRIQ > +options ALTQ_NOPCC # just for SMP builds > + > +options MAXDSIZ=(2048UL*1024*1024) #added by gustavo > +options MAXSSIZ=(512UL*1024*1024) #added by gustavo > +options DFLDSIZ=(2048UL*1024*1024) #added by gustavo > +# > + > > # Bus support. > device eisa > > > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to > "freebsd-performance-unsubscribe@freebsd.org" From owner-freebsd-performance@FreeBSD.ORG Fri Dec 2 15:53:34 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F051D16A420 for ; Fri, 2 Dec 2005 15:53:34 +0000 (GMT) (envelope-from trashy_bumper@yahoo.com) Received: from web36301.mail.mud.yahoo.com (web36301.mail.mud.yahoo.com [209.191.84.231]) by mx1.FreeBSD.org (Postfix) with SMTP id 1A7E043D49 for ; Fri, 2 Dec 2005 15:53:34 +0000 (GMT) (envelope-from trashy_bumper@yahoo.com) Received: (qmail 52158 invoked by uid 60001); 2 Dec 2005 15:53:33 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=wGpObLvw/SzCW358/OjqE2jtlIhMyomtmD7fRi1pXJ8JzCGa9FEp/oRe94mYsU2fPhpDT1Q/QQ2jc0rD6zcltvU9Kl+ShvU8UuDOW5M7zGZYzkdQIuVMHAKW2IDE8ys8ol4L//bHCAz8cdAhVikf7ekLzh+Gihzf9eEfOrpDaWM= ; Message-ID: <20051202155333.52155.qmail@web36301.mail.mud.yahoo.com> Received: from [213.227.200.79] by web36301.mail.mud.yahoo.com via HTTP; Fri, 02 Dec 2005 07:53:33 PST Date: Fri, 2 Dec 2005 07:53:33 -0800 (PST) From: Nash Nipples To: freebsd-performance@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Constraining CPU usage X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Dec 2005 15:53:35 -0000 Hi Guys! It seems i cannot get a clear answer wherther it is possible to limit a CPU usage by a user process and should i do that at all. I just wana run "tar -xzvf /usr/backup/pub.tar.gz /usr/home/pub" and do not cause something like this: PID USERNAME PRI NICE SIZE RES STATE TIME WCPU CPU 3993 backup 139 20 8672K 8068K RUN 19:56 87.65% 87.65% load averages: 0.99, 0.98, 0.88 As you can see in the figure above, renicing wont work. Any hints? What to read? Thank you. Nash FreeBSD 5.3-RELEASE Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: AMD Duron(tm) processor (1202.73-MHz 686-class CPU) --------------------------------- Yahoo! Personals Skip the bars and set-ups and start using Yahoo! Personals for free From owner-freebsd-performance@FreeBSD.ORG Fri Dec 2 16:44:29 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3981B16A41F for ; Fri, 2 Dec 2005 16:44:29 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: from web30301.mail.mud.yahoo.com (web30301.mail.mud.yahoo.com [68.142.200.94]) by mx1.FreeBSD.org (Postfix) with SMTP id 6D67443D64 for ; Fri, 2 Dec 2005 16:44:28 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 53512 invoked by uid 60001); 2 Dec 2005 16:44:27 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=GA6KgPH6G7ahRIiFGP2ZRdtsNw2YS/qfRcdNPe5VzDntfgMWQ+DGvDFToSDE4OMd948h5oxGZ3aHPIrTqfi790xkY08DsMH+K5deOhWp2fytJeVzIntDwnv6G2HMvbmbCj1AWLzwjEnYmOjZIeumq2YE+Zo4dC8dqD4p4Wjcpw4= ; Message-ID: <20051202164427.53510.qmail@web30301.mail.mud.yahoo.com> Received: from [213.54.72.43] by web30301.mail.mud.yahoo.com via HTTP; Fri, 02 Dec 2005 08:44:27 PST Date: Fri, 2 Dec 2005 08:44:27 -0800 (PST) From: Arne Woerner To: Nash Nipples , freebsd-performance@freebsd.org In-Reply-To: <20051202155333.52155.qmail@web36301.mail.mud.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: Subject: Re: Constraining CPU usage X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Dec 2005 16:44:29 -0000 --- Nash Nipples wrote: > It seems i cannot get a clear answer wherther it > is possible to limit a CPU usage by a user process > and should i do that at all. > Why should somebody want to keep CPU usage of a process below a certain value (e. g. 20%)? > As you can see in the figure above, renicing wont work. > Why? Obviously there is no or nearly no other process, that wants to run, so your "tar" process runs as fast as possible in order to finish as soon as possible... Reniceing to a positive nice-value means that this process gets the processor less often than other processes, who want to run... > Any hints? What to read? Thank you. > You could try idprio(1) against your tar process. That makes sure, that your tar process waits, when other processes (non idprio processes) want to run... -Arne __________________________________ Start your day with Yahoo! - Make it your home page! http://www.yahoo.com/r/hs From owner-freebsd-performance@FreeBSD.ORG Sat Dec 3 12:16:54 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 568D216A41F for ; Sat, 3 Dec 2005 12:16:54 +0000 (GMT) (envelope-from Shane@007Marketing.com) Received: from smtp3.adl2.internode.on.net (smtp3.adl2.internode.on.net [203.16.214.203]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7079E43D60 for ; Sat, 3 Dec 2005 12:16:53 +0000 (GMT) (envelope-from Shane@007Marketing.com) Received: from [192.168.8.50] (ppp19-170.static.internode.on.net [150.101.19.170]) by smtp3.adl2.internode.on.net (8.12.9/8.12.6) with ESMTP id jB3CGogY089910 for ; Sat, 3 Dec 2005 22:46:52 +1030 (CST) (envelope-from Shane@007Marketing.com) User-Agent: Microsoft-Entourage/10.1.4.030702.0 Date: Sat, 03 Dec 2005 22:46:47 +1030 From: Shane Ambler To: FreeBSD Mailing Lists Message-ID: In-Reply-To: <20051202164427.53510.qmail@web30301.mail.mud.yahoo.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Subject: Re: Constraining CPU usage X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Dec 2005 12:16:54 -0000 On 3/12/2005 3:14, "Arne Woerner" wrote: > --- Nash Nipples wrote: >> It seems i cannot get a clear answer wherther it >> is possible to limit a CPU usage by a user process >> and should i do that at all. >> > Why should somebody want to keep CPU usage of a process below a > certain value (e. g. 20%)? Well if your machine isn't smp enabled (specifically pre-HTT) then one process can slow the server down to a point where your web/databse server can't get processor time to give a response - effectively making it non existent for the duration. Some tasks such as getting awstat to update and create stats pages are not a priority and can take as long as they need. (if you can't guess this is a problem I am just starting to look into) >> As you can see in the figure above, renicing wont work. >> > Why? Obviously there is no or nearly no other process, that wants > to run, so your "tar" process runs as fast as possible in order to > finish as soon as possible... > > Reniceing to a positive nice-value means that this process gets > the processor less often than other processes, who want to run... > >> Any hints? What to read? Thank you. >> > You could try idprio(1) against your tar process. That makes sure, > that your tar process waits, when other processes (non idprio > processes) want to run... > > -Arne > > > > > __________________________________ > Start your day with Yahoo! - Make it your home page! > http://www.yahoo.com/r/hs > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" > -- Shane Ambler Sales Department 007Marketing.com Shane@007Marketing.com From owner-freebsd-performance@FreeBSD.ORG Sat Dec 3 13:35:34 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 93ADF16A41F for ; Sat, 3 Dec 2005 13:35:34 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: from web30305.mail.mud.yahoo.com (web30305.mail.mud.yahoo.com [68.142.200.98]) by mx1.FreeBSD.org (Postfix) with SMTP id 14A4543D45 for ; Sat, 3 Dec 2005 13:35:34 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 92333 invoked by uid 60001); 3 Dec 2005 13:35:33 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=okLpgvoivcQ9mQtWNBcGY+URFN6hDlefEOsaS9tpcfSklw5mjk5RESBfFaiGFgsbH/XWE8BMGJ4nEHLD3+12GZvmUx0DWRqiZ23XTpmCNMgx64SEJygS948q9O8z7qEqGQgr+k54ABDASuRjpr+ZYjAkcxsi+ix43iidDPJdtk0= ; Message-ID: <20051203133533.92331.qmail@web30305.mail.mud.yahoo.com> Received: from [213.54.94.134] by web30305.mail.mud.yahoo.com via HTTP; Sat, 03 Dec 2005 05:35:33 PST Date: Sat, 3 Dec 2005 05:35:33 -0800 (PST) From: Arne Woerner To: Shane Ambler , FreeBSD Mailing Lists In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: Subject: Re: Constraining CPU usage X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Dec 2005 13:35:34 -0000 --- Shane Ambler wrote: > On 3/12/2005 3:14, "Arne Woerner" > wrote: > > Why should somebody want to keep CPU usage of a > > process below a certain value (e. g. 20%)? > > Well if your machine isn't smp enabled (specifically > pre-HTT) then one process can slow the server down > to a point where your web/databse server can't get > processor time to give a response - effectively making > it non existent for the duration. > Hmm... Isn't that problem solved already by nice'ing a process? I think that problem is completely solved by idprio'ing a process... Because: When the NIC gets a packet for a database server, the kernel interrupts the executing of a potential idprio'ed process and starts the DB process... Or am I wrong? -Arne __________________________________________ Yahoo! DSL – Something to write home about. Just $16.99/mo. or less. dsl.yahoo.com From owner-freebsd-performance@FreeBSD.ORG Sat Dec 3 15:18:39 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DB04C16A41F for ; Sat, 3 Dec 2005 15:18:39 +0000 (GMT) (envelope-from trashy_bumper@yahoo.com) Received: from web36309.mail.mud.yahoo.com (web36309.mail.mud.yahoo.com [209.191.84.239]) by mx1.FreeBSD.org (Postfix) with SMTP id 3AE8C43D5E for ; Sat, 3 Dec 2005 15:18:39 +0000 (GMT) (envelope-from trashy_bumper@yahoo.com) Received: (qmail 57916 invoked by uid 60001); 3 Dec 2005 15:18:37 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=o5sap3fLd9vTJ/FvRgPpF1ogV8I6JJ03+BUoH6uZzng+ac5cMcovGPjz6Li7zGvzqzP69DtFJtpoB1sYFxZ52IL0omlwpxVFUbulNPVGXPCxFPU6oj1J2PWnII0DXv8jklZbB2B1FmGuuo7kNljZ6HvwcjjSdQa5awyITIaOam4= ; Message-ID: <20051203151837.57912.qmail@web36309.mail.mud.yahoo.com> Received: from [213.227.200.79] by web36309.mail.mud.yahoo.com via HTTP; Sat, 03 Dec 2005 07:18:37 PST Date: Sat, 3 Dec 2005 07:18:37 -0800 (PST) From: Nash Nipples To: freebsd-performance@freebsd.org In-Reply-To: <20051203133533.92331.qmail@web30305.mail.mud.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: Constraining CPU usage X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Dec 2005 15:18:40 -0000 Arne Woerner wrote: --- Shane Ambler wrote: > On 3/12/2005 3:14, "Arne Woerner" > wrote: > > Why should somebody want to keep CPU usage of a > > process below a certain value (e. g. 20%)? > > Well if your machine isn't smp enabled (specifically > pre-HTT) then one process can slow the server down > to a point where your web/databse server can't get > processor time to give a response - effectively making > it non existent for the duration. > Hmm... Isn't that problem solved already by nice'ing a process? I think that problem is completely solved by idprio'ing a process... Because: When the NIC gets a packet for a database server, the kernel interrupts the executing of a potential idprio'ed process and starts the DB process... Or am I wrong? -Arne Looks pretty simple and clear. Thank you Arne. In respect to what Mike said i must admit: "its all about high utilization numbers fear" If the machine was smp enabled i think the number would be around 50% or so.. thank you Shane. Reasoning the topic i believe i should do a little research on: Understanding units and measurements used in FreeBSD performance representation. Calculating a machine's available resources and estimating its possible low/average/peak load when performing user/system tasks during a specific amount of time (24 hours). E.g: 1 user will read $File of $Size Mb from $Storage_media using $Transfer_media. The operation will last $time and consume $CPU CPU time, $Storage_media_capacity $Transfer_media_bandwith, $RAM, $Other shared resources. So eventually, one can build an operations chart/table, estimate timings and balance a target machine's load among the processes that are a subject to order/priority. Gather and analyze collected in a real-time situation statistics. I'm planning to google that for a while. Thank you everyone once again. nash --------------------------------- Yahoo! DSL Something to write home about. Just $16.99/mo. or less