From owner-freebsd-performance@FreeBSD.ORG Tue Dec 6 10:00:04 2005 Return-Path: X-Original-To: performance@freebsd.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4500816A41F; Tue, 6 Dec 2005 10:00:04 +0000 (GMT) (envelope-from mv@roq.com) Received: from vault.mel.jumbuck.com (ppp166-27.static.internode.on.net [150.101.166.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8140E43D60; Tue, 6 Dec 2005 10:00:03 +0000 (GMT) (envelope-from mv@roq.com) Received: from vault.mel.jumbuck.com (localhost [127.0.0.1]) by vault.mel.jumbuck.com (Postfix) with ESMTP id 6C0198A035; Tue, 6 Dec 2005 20:59:57 +1100 (EST) Received: from [192.168.46.52] (unknown [192.168.46.250]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by vault.mel.jumbuck.com (Postfix) with ESMTP id 4FF1D8A014; Tue, 6 Dec 2005 20:59:57 +1100 (EST) Message-ID: <43956122.5020106@roq.com> Date: Tue, 06 Dec 2005 21:00:02 +1100 From: Michael Vince User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.7.12) Gecko/20051202 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-java@FreeBSD.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV using ClamSMTP Cc: performance@freebsd.org, freebsd-threads@FreeBSD.org Subject: More threads X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Dec 2005 10:00:04 -0000 Hi All, I been benchmarking a Java servlet I have created and I want to be able to handle a massive amount of simultaneous connections. So far using benchmarking tools I have been able to get around 2,565 threads on the Tomcat 5.5 Java process (with native 1.4.2 Java) according to ps -auxwH | grep -c 'java' But I just can't seem to get past that mark, I have a lot of memory currently the Tomcat is allocated 2gigs of memory. When I max it out with my benchmarks I get this in catalina Tomcat 5.5 but there is still plenty of free memory to the Tomcat process. SEVERE: Caught exception (java.lang.OutOfMemoryError: unable to create new native thread) executing org.apache.jk.common.SocketAcceptor@f36e59, terminatingthread I have been testing with my libmap.conf with these different types of implementations. [/usr/local/jdk1.4.2/] #libpthread.so libc_r.so #libpthread.so.2 libc_r.so.6 libpthread.so.2 libthr.so.2 libpthread.so libthr.so libc_r.so.6 libthr.so.2 libc_r.so libthr.so I have been getting the best performance with libthr and have been using the above libthr and with these settings below. I have set my max_threads in /etc/sysctl.conf to a massive amount. kern.threads.max_threads_per_proc=40000 kern.threads.max_groups_per_proc=40000 ps -auxwH | grep -c 'java' 2565 I am using Apache2.2 with the new built in AJP module which has been a great addition to Apache 2.2. I have been able to get the setup performing most stable (no 503 status errors) but with less performance / threads with the libmap.conf below. [/usr/local/jdk1.4.2/] libpthread.so libc_r.so libpthread.so.2 libc_r.so.6 I am on 6.0 Release i386, dual Intel P4, generic SMP kernel. With Tomcat on Windows XP I have been able to get it running better. Does any one know of some other sysctls that increase the amount of threads in libthr which I am assuming utilizes the above sysctls. Does any one know how many threads can be created in Java on FreeBSD? Cheers, Mike From owner-freebsd-performance@FreeBSD.ORG Tue Dec 6 10:33:30 2005 Return-Path: X-Original-To: performance@freebsd.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3C17016A420; Tue, 6 Dec 2005 10:33:30 +0000 (GMT) (envelope-from davidxu@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8F12543D5E; Tue, 6 Dec 2005 10:33:29 +0000 (GMT) (envelope-from davidxu@freebsd.org) Received: from [127.0.0.1] (davidxu@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.3/8.13.3) with ESMTP id jB6AXQMD040880; Tue, 6 Dec 2005 10:33:27 GMT (envelope-from davidxu@freebsd.org) Message-ID: <439568FB.1000804@freebsd.org> Date: Tue, 06 Dec 2005 18:33:31 +0800 From: David Xu User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.12) Gecko/20050928 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Michael Vince References: <43956122.5020106@roq.com> In-Reply-To: <43956122.5020106@roq.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-threads@freebsd.org, performance@freebsd.org, freebsd-java@freebsd.org Subject: Re: More threads X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Dec 2005 10:33:30 -0000 Michael Vince wrote: > Hi All, > > I been benchmarking a Java servlet I have created and I want to be able > to handle a massive amount of simultaneous connections. > So far using benchmarking tools I have been able to get around 2,565 > threads on the Tomcat 5.5 Java process (with native 1.4.2 Java) > according to ps -auxwH | grep -c 'java' > > But I just can't seem to get past that mark, I have a lot of memory > currently the Tomcat is allocated 2gigs of memory. > > When I max it out with my benchmarks I get this in catalina Tomcat 5.5 > but there is still plenty of free memory to the Tomcat process. > SEVERE: Caught exception (java.lang.OutOfMemoryError: unable to create > new native thread) executing org.apache.jk.common.SocketAcceptor@f36e59, > terminatingthread > > I have been testing with my libmap.conf with these different types of > implementations. > > [/usr/local/jdk1.4.2/] > #libpthread.so libc_r.so > #libpthread.so.2 libc_r.so.6 > libpthread.so.2 libthr.so.2 > libpthread.so libthr.so > libc_r.so.6 libthr.so.2 > libc_r.so libthr.so > > I have been getting the best performance with libthr and have been using > the above libthr and with these settings below. > > I have set my max_threads in /etc/sysctl.conf to a massive amount. > > kern.threads.max_threads_per_proc=40000 > kern.threads.max_groups_per_proc=40000 > > ps -auxwH | grep -c 'java' > 2565 > > I am using Apache2.2 with the new built in AJP module which has been a > great addition to Apache 2.2. > I have been able to get the setup performing most stable (no 503 status > errors) but with less performance / threads with the libmap.conf below. > [/usr/local/jdk1.4.2/] > libpthread.so libc_r.so > libpthread.so.2 libc_r.so.6 > > I am on 6.0 Release i386, dual Intel P4, generic SMP kernel. > > With Tomcat on Windows XP I have been able to get it running better. > Does any one know of some other sysctls that increase the amount of > threads in libthr which I am assuming utilizes the above sysctls. Does > any one know how many threads can be created in Java on FreeBSD? > > Cheers, > Mike > Number of threads you can create if you use libthr is limited by following factors: 1) sysctl: kern.threads.max_threads_per_proc kern.threads.max_groups_per_proc 2) stack per-thread userland stack, default number on 64 bits platform is 2M, on 32 bits platform, it is 1M. I don't know whether java supports adjusting default per-thread stack size. if can not, we may add an environment variable to thread libraries, for example: LIBPTHREAD_THREAD_STACKSIZE allows user to set default stack size. Thread also needs a kernel mode stack when it is in kernel, if I am right, it is 16K bytes per-thread. if you create too many threads, make sure both side won't exhaust address space, and because kernel stack can not be swapped out when process is running, you'd make sure physical memory can not be exhausted. 3) check memory limits type limits command: Resource limits (current): cputime infinity secs filesize infinity kB datasize 524288 kB stacksize 65536 kB coredumpsize infinity kB memoryuse infinity kB memorylocked infinity kB maxprocesses 5547 openfiles 11095 sbsize infinity bytes vmemoryuse infinity kB if address space is not large enough, you have to reconfigure kernel to allow larger space. Fix me if I am wrong. David Xu From owner-freebsd-performance@FreeBSD.ORG Tue Dec 6 10:12:09 2005 Return-Path: X-Original-To: performance@freebsd.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5F36D16A41F; Tue, 6 Dec 2005 10:12:09 +0000 (GMT) (envelope-from achill@matrix.gatewaynet.com) Received: from matrix.gatewaynet.com (host2.dynacom.ondsl.gr [62.103.35.210]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8E20243D68; Tue, 6 Dec 2005 10:12:08 +0000 (GMT) (envelope-from achill@matrix.gatewaynet.com) Received: from matrix.gatewaynet.com (localhost.localdomain [127.0.0.1]) by matrix.gatewaynet.com (8.12.8/8.12.8) with ESMTP id jB6BAUKc017209; Tue, 6 Dec 2005 13:10:30 +0200 Received: from localhost (achill@localhost) by matrix.gatewaynet.com (8.12.8/8.12.8/Submit) with ESMTP id jB6BAUgg017205; Tue, 6 Dec 2005 13:10:30 +0200 Date: Tue, 6 Dec 2005 13:10:30 +0200 (EET) From: Achilleus Mantzios To: Michael Vince In-Reply-To: <43956122.5020106@roq.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Mailman-Approved-At: Tue, 06 Dec 2005 12:53:47 +0000 Cc: freebsd-threads@freebsd.org, performance@freebsd.org, freebsd-java@freebsd.org Subject: Re: More threads X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Dec 2005 10:12:09 -0000 Maybe take a look at http://mail-archives.apache.org/mod_mbox/jakarta-tomcat-user/200312.mbox/<3FD2066D.2040008%40joedog.org> -- -Achilleus From owner-freebsd-performance@FreeBSD.ORG Tue Dec 6 10:15:22 2005 Return-Path: X-Original-To: performance@freebsd.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5D3D616A41F for ; Tue, 6 Dec 2005 10:15:22 +0000 (GMT) (envelope-from chad.brubaker@gmail.com) Received: from smtp101.sbc.mail.mud.yahoo.com (smtp101.sbc.mail.mud.yahoo.com [68.142.198.200]) by mx1.FreeBSD.org (Postfix) with SMTP id 6A83543D53 for ; Tue, 6 Dec 2005 10:15:21 +0000 (GMT) (envelope-from chad.brubaker@gmail.com) Received: (qmail 67453 invoked from network); 6 Dec 2005 10:15:20 -0000 Received: from unknown (HELO ?192.168.1.106?) (emochila@sbcglobal.net@71.141.245.145 with plain) by smtp101.sbc.mail.mud.yahoo.com with SMTP; 6 Dec 2005 10:15:20 -0000 In-Reply-To: <43956122.5020106@roq.com> References: <43956122.5020106@roq.com> Mime-Version: 1.0 (Apple Message framework v746.2) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Chad Brubaker Date: Tue, 6 Dec 2005 02:15:19 -0800 To: Michael Vince X-Mailer: Apple Mail (2.746.2) X-Mailman-Approved-At: Tue, 06 Dec 2005 12:54:28 +0000 Cc: freebsd-threads@FreeBSD.org, performance@freebsd.org, freebsd-java@FreeBSD.org Subject: Re: More threads X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Dec 2005 10:15:22 -0000 You can try setting your heap size. JAVA_OPTS="-Xms512M - Xmx1024M" - actually, in your case you could set the min and max to be much higher, but this is the command i use on my machine. I'm pretty sure this is definitely your problem. I upgraded to a machine with more memory thinking i had solved my OutOfMemeoryError, - but low and behold, it was my heap size allocation.... hope this helps. On Dec 6, 2005, at 2:00 AM, Michael Vince wrote: > Hi All, > > I been benchmarking a Java servlet I have created and I want to be > able to handle a massive amount of simultaneous connections. > So far using benchmarking tools I have been able to get around > 2,565 threads on the Tomcat 5.5 Java process (with native 1.4.2 > Java) according to ps -auxwH | grep -c 'java' > > But I just can't seem to get past that mark, I have a lot of memory > currently the Tomcat is allocated 2gigs of memory. > > When I max it out with my benchmarks I get this in catalina Tomcat > 5.5 but there is still plenty of free memory to the Tomcat process. > SEVERE: Caught exception (java.lang.OutOfMemoryError: unable to > create new native thread) executing > org.apache.jk.common.SocketAcceptor@f36e59, terminatingthread > > I have been testing with my libmap.conf with these different types > of implementations. > > [/usr/local/jdk1.4.2/] > #libpthread.so libc_r.so > #libpthread.so.2 libc_r.so.6 > libpthread.so.2 libthr.so.2 > libpthread.so libthr.so > libc_r.so.6 libthr.so.2 > libc_r.so libthr.so > > I have been getting the best performance with libthr and have been > using the above libthr and with these settings below. > > I have set my max_threads in /etc/sysctl.conf to a massive amount. > > kern.threads.max_threads_per_proc=40000 > kern.threads.max_groups_per_proc=40000 > > ps -auxwH | grep -c 'java' > 2565 > > I am using Apache2.2 with the new built in AJP module which has > been a great addition to Apache 2.2. > I have been able to get the setup performing most stable (no 503 > status errors) but with less performance / threads with the > libmap.conf below. > [/usr/local/jdk1.4.2/] > libpthread.so libc_r.so > libpthread.so.2 libc_r.so.6 > > I am on 6.0 Release i386, dual Intel P4, generic SMP kernel. > > With Tomcat on Windows XP I have been able to get it running better. > Does any one know of some other sysctls that increase the amount of > threads in libthr which I am assuming utilizes the above sysctls. > Does any one know how many threads can be created in Java on FreeBSD? > > Cheers, > Mike > > > > > > _______________________________________________ > freebsd-java@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-java > To unsubscribe, send any mail to "freebsd-java- > unsubscribe@freebsd.org" From owner-freebsd-performance@FreeBSD.ORG Tue Dec 6 16:15:13 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 28A9316A41F for ; Tue, 6 Dec 2005 16:15:13 +0000 (GMT) (envelope-from pete@he.iki.fi) Received: from rms06.rommon.net (rms06.rommon.net [212.54.5.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9F10643D6A for ; Tue, 6 Dec 2005 16:15:11 +0000 (GMT) (envelope-from pete@he.iki.fi) Received: from [127.0.0.1] (dyn234.helenius.fi [193.64.42.234]) by rms06.rommon.net (Postfix) with ESMTP id 60BF133C1B; Tue, 6 Dec 2005 18:15:01 +0200 (EET) Message-ID: <4395B905.9090308@he.iki.fi> Date: Tue, 06 Dec 2005 18:15:01 +0200 From: Petri Helenius User-Agent: Mozilla Thunderbird 1.0.6 (Windows/20050716) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Mike Tancsa References: <000801c5d434$579de280$b3db87d4@multiplay.co.uk> <000801c5d434$579de280$b3db87d4@multiplay.co.uk> <3.0.1.32.20051102022618.00a73dc0@pop.redshift.com> <436896BD.6090405@multiwise.fi> <436972DE.8070002@roq.com> <6.2.3.4.0.20051102222452.0564eda0@64.7.153.2> In-Reply-To: <6.2.3.4.0.20051102222452.0564eda0@64.7.153.2> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Michael VInce , freebsd-performance@freebsd.org Subject: Re: SATA: RAID 5 controller recommendations X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Dec 2005 16:15:13 -0000 Mike Tancsa wrote: > At 09:15 PM 02/11/2005, Michael VInce wrote: > >> I have seen some network based SMP related performance problems >> vanish in 6.0 tests, admittedly I haven't done hard drive based tests >> but I wouldn't surprise me of performance drops on HDs in SMP on 6.0 >> are gone as well. > > > Yes, I noticed that well. Unfortunately I seem to have a bad drive as > well so I will have to try swapping it tomorrow. Although it might > just be a cable / connector issue as smartmontools doesnt see anything > on the drive. > Has the 3ware patchset for smartmontools been integrated into the port? Pete > da0: Fixed Direct Access SCSI-3 device > da0: 100.000MB/s transfers > da0: 343290MB (703057920 512 byte sectors: 255H 63S/T 43763C) > twa0: ERROR: (0x04: 0x0009): Drive timeout detected: port=0 > twa0: ERROR: (0x04: 0x0009): Drive timeout detected: port=0 > twa0: ERROR: (0x04: 0x0009): Drive timeout detected: port=0 > twa0: ERROR: (0x04: 0x0009): Drive timeout detected: port=0 > twa0: ERROR: (0x04: 0x0009): Drive timeout detected: port=0 > twa0: ERROR: (0x04: 0x0009): Drive timeout detected: port=0 > twa0: ERROR: (0x04: 0x0009): Drive timeout detected: port=0 > twa0: ERROR: (0x04: 0x0009): Drive timeout detected: port=0 > twa0: ERROR: (0x04: 0x0009): Drive timeout detected: port=0 > twa0: ERROR: (0x04: 0x0009): Drive timeout detected: port=0 > twa0: ERROR: (0x04: 0x0009): Drive timeout detected: port=0 > twa0: ERROR: (0x04: 0x0009): Drive timeout detected: port=0 > twa0: ERROR: (0x04: 0x0009): Drive timeout detected: port=0 > twa0: ERROR: (0x04: 0x0009): Drive timeout detected: port=0 > twa0: ERROR: (0x04: 0x0009): Drive timeout detected: port=0 > twa0: ERROR: (0x04: 0x0002): Degraded unit: unit=0, port=0 > > >> Mike >> >> >> Markus Kovero wrote: >> >>> Dell's CERC sata raid thingy works fast for me but when I turn SMP >>> on read performance drops 50%+ (raid1,raid5,raid10 and single >>> volumes). >>> I don't know if its aac problem or what. >>> In leenox it works very well though. >>> >>> Yours >>> Markus Kovero >>> >>> ray@redshift.com wrote: >>> >>>> At 09:17 PM 11/2/2005 +1100, Michael VInce wrote: >>>> | For SATA I have always been getting the Dell 750s (now 850s) >>>> which use | the 'aac' >>>> | Adaptec AdvancedRAID Controller driver, do 'man aac' for more >>>> details. >>>> | Dell supply them with a 64meg cache and are hardware raid, I have >>>> always | just installed FreeBSD on them and only 1 hard drive comes >>>> up as RAID | should be. >>>> | | I have only been ordering the machines in dual drive mirror 1 >>>> Raid, but | you could get a PowerEdge Tower servers like the 830 >>>> which do raid 5, I | never used one though. >>>> | Alternatively all the Dell SCSI RAID systems have 256megs cache >>>> which | use the LSI Logic MegaRAID ( man amr ) >>>> | Stay away from the ultra low end SC1425 systems they have some >>>> kind of | software based raid. >>>> | | The thing I have noticed is that if you compare other (often >>>> claiming to | be much cheaper) servers from other companies they >>>> often really skimp on | their hard drive subsystems with software >>>> based stuff claiming its RAID. >>>> | | I used to be anti Dell and build my own U1 servers from case >>>> and up and | had a keen interest in cheap RAID technologies but I >>>> gave up a little | while ago. >>>> | I believe Dell are great value when you consider their RAID | >>>> implementations and as long as you are happy with Intel chips. >>>> | | Mike >>>> | | Steven Hartland wrote: >>>> | | > Anyone got any SATA RAID 5 controllers they can recommend >>>> | > 64Bit PCIX. >>>> | > >>>> | > Steve >>>> >>>> Steve - 3ware is very good. >>>> >>>> Mike - I have some benchmarks from a guy using a Dell with a raid >>>> and was >>>> surprised to see how slow they were compared to our other hardware. >>>> I didn't >>>> run them myself, but the guy is on the list here some place. I can >>>> send you a >>>> copy of the spreadsheet if you want to look it over. >>>> >>>> Ray >>>> >>>> _______________________________________________ >>>> freebsd-performance@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >>>> To unsubscribe, send any mail to >>>> "freebsd-performance-unsubscribe@freebsd.org" >>>> >> _______________________________________________ >> freebsd-performance@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >> To unsubscribe, send any mail to >> "freebsd-performance-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to > "freebsd-performance-unsubscribe@freebsd.org" > From owner-freebsd-performance@FreeBSD.ORG Tue Dec 6 20:45:49 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1E64616A41F for ; Tue, 6 Dec 2005 20:45:49 +0000 (GMT) (envelope-from oxy@field.hu) Received: from dumballah.tvnet.hu (dumballah.tvnet.hu [195.38.96.23]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9931343D5F for ; Tue, 6 Dec 2005 20:45:48 +0000 (GMT) (envelope-from oxy@field.hu) Received: from localhost (localhost.localdomain [127.0.0.1]) by dumballah.tvnet.hu (Postfix) with ESMTP id 138DA1019D1 for ; Tue, 6 Dec 2005 21:45:46 +0100 (CET) Received: from dumballah.tvnet.hu ([127.0.0.1]) by localhost (dumballah.tvnet.hu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 06380-12 for ; Tue, 6 Dec 2005 21:45:44 +0100 (CET) Received: from oxy (dsl85-238-73-88.pool.tvnet.hu [85.238.73.88]) by dumballah.tvnet.hu (Postfix) with SMTP id 9DA9F1019CB for ; Tue, 6 Dec 2005 21:45:43 +0100 (CET) Message-ID: <003c01c5faa6$08f4e730$0201a8c0@oxy> From: "OxY" To: Date: Tue, 6 Dec 2005 21:45:46 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-2"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2527 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2527 X-Virus-Scanned: by amavisd-new at tvnet.hu Subject: xl driver bug X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Dec 2005 20:45:49 -0000 hi! i have some trouble with xl driver (3Com 905c-TX) in 6.0-release.. after i upgraded my 5.4 to 6.0 i realized that the incoming traffic has been dropped to 1M/s..however outgoing is still ok (7-9M/s).. i tried different protocols, with the same result.. switched off every tcp tuning in sysctl.conf, then rebooted, still the same.. i saw that the xl interrupt jumps to 2500-5000 (systat -vmstat 1 ) while i am downloading with 800-1000K/s (maximum..), but when i upload the interrupt is around 800 with 8M/s speed.. please gime me some advice, patch, anything :) thanks a lot! OxY From owner-freebsd-performance@FreeBSD.ORG Tue Dec 6 23:18:29 2005 Return-Path: X-Original-To: performance@freebsd.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3132516A41F; Tue, 6 Dec 2005 23:18:29 +0000 (GMT) (envelope-from glewis@eyesbeyond.com) Received: from misty.eyesbeyond.com (glewis.dsl.xmission.com [166.70.56.15]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3C1AF43D4C; Tue, 6 Dec 2005 23:18:27 +0000 (GMT) (envelope-from glewis@eyesbeyond.com) Received: from misty.eyesbeyond.com (localhost.eyesbeyond.com [127.0.0.1]) by misty.eyesbeyond.com (8.13.3/8.13.3) with ESMTP id jB6NIQ9q045157; Tue, 6 Dec 2005 16:18:26 -0700 (MST) (envelope-from glewis@eyesbeyond.com) Received: (from glewis@localhost) by misty.eyesbeyond.com (8.13.3/8.13.3/Submit) id jB6NIPEq045156; Tue, 6 Dec 2005 16:18:25 -0700 (MST) (envelope-from glewis@eyesbeyond.com) X-Authentication-Warning: misty.eyesbeyond.com: glewis set sender to glewis@eyesbeyond.com using -f Date: Tue, 6 Dec 2005 16:18:25 -0700 From: Greg Lewis To: David Xu Message-ID: <20051206231825.GA44966@misty.eyesbeyond.com> References: <43956122.5020106@roq.com> <439568FB.1000804@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <439568FB.1000804@freebsd.org> User-Agent: Mutt/1.4.2.1i X-Mailman-Approved-At: Wed, 07 Dec 2005 00:53:03 +0000 Cc: Michael Vince , freebsd-java@freebsd.org, performance@freebsd.org, freebsd-threads@freebsd.org Subject: Re: More threads X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Dec 2005 23:18:29 -0000 On Tue, Dec 06, 2005 at 06:33:31PM +0800, David Xu wrote: > Michael Vince wrote: [...] > >But I just can't seem to get past that mark, I have a lot of memory > >currently the Tomcat is allocated 2gigs of memory. [...] > Number of threads you can create if you use libthr is limited by > following factors: > 1) sysctl: > kern.threads.max_threads_per_proc > kern.threads.max_groups_per_proc > > 2) stack > per-thread userland stack, default number on 64 bits platform is 2M, > on 32 bits platform, it is 1M. I don't know whether java supports > adjusting default per-thread stack size. if can not, we may add > an environment variable to thread libraries, for example: > LIBPTHREAD_THREAD_STACKSIZE allows user to set default stack size. > > Thread also needs a kernel mode stack when it is in kernel, if > I am right, it is 16K bytes per-thread. if you create too many > threads, make sure both side won't exhaust address space, and > because kernel stack can not be swapped out when process is > running, you'd make sure physical memory can not be exhausted. > > 3) check memory limits > type limits command: > > Resource limits (current): > cputime infinity secs > filesize infinity kB > datasize 524288 kB > stacksize 65536 kB > coredumpsize infinity kB > memoryuse infinity kB > memorylocked infinity kB > maxprocesses 5547 > openfiles 11095 > sbsize infinity bytes > vmemoryuse infinity kB > > if address space is not large enough, you have to reconfigure kernel > to allow larger space. Fix me if I am wrong. I think that Michael's real problem is the amount of memory he has allocated. If he has 2GB allocated already then he is out of luck if he needs to allocate more (e.g. to create another thread). See mmap(2), particularly the BUGS section. -- Greg Lewis Email : glewis@eyesbeyond.com Eyes Beyond Web : http://www.eyesbeyond.com Information Technology FreeBSD : glewis@FreeBSD.org From owner-freebsd-performance@FreeBSD.ORG Wed Dec 7 06:06:33 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BE6FB16A41F for ; Wed, 7 Dec 2005 06:06:33 +0000 (GMT) (envelope-from oxy@field.hu) Received: from dumballah.tvnet.hu (dumballah.tvnet.hu [195.38.96.23]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1393343D45 for ; Wed, 7 Dec 2005 06:06:32 +0000 (GMT) (envelope-from oxy@field.hu) Received: from localhost (localhost.localdomain [127.0.0.1]) by dumballah.tvnet.hu (Postfix) with ESMTP id C4A521018A5 for ; Wed, 7 Dec 2005 07:06:31 +0100 (CET) Received: from dumballah.tvnet.hu ([127.0.0.1]) by localhost (dumballah.tvnet.hu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 31185-34 for ; Wed, 7 Dec 2005 07:06:31 +0100 (CET) Received: from oxy (dsl85-238-73-88.pool.tvnet.hu [85.238.73.88]) by dumballah.tvnet.hu (Postfix) with SMTP id D1032101842 for ; Wed, 7 Dec 2005 07:06:22 +0100 (CET) Message-ID: <000601c5faf4$5bec1550$0201a8c0@oxy> From: "OxY" To: References: <003c01c5faa6$08f4e730$0201a8c0@oxy> Date: Wed, 7 Dec 2005 07:06:26 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-2"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2527 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2527 X-Virus-Scanned: by amavisd-new at tvnet.hu Subject: Re: xl driver bug-->geom mirror X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Dec 2005 06:06:33 -0000 i found the real problem, it was not the xl driver, but the geom mirror.. if i try to write to an other disk, not in the main systems raid1, then download is ok, 10M/s... then i tried to remove one disk from the raid1, and write on it.. the download is OK again... when i sync the raid array it gives poor performance again 1M/s.. what's wrong? ----- Original Message ----- From: "OxY" To: Sent: Tuesday, December 06, 2005 9:45 PM Subject: xl driver bug > hi! > > > i have some trouble with xl driver (3Com 905c-TX) in 6.0-release.. > after i upgraded my 5.4 to 6.0 i realized that the incoming traffic > has been dropped to 1M/s..however outgoing is still ok (7-9M/s).. > > i tried different protocols, with the same result.. > switched off every tcp tuning in sysctl.conf, then rebooted, still the > same.. > > i saw that the xl interrupt jumps to 2500-5000 (systat -vmstat 1 ) while i > am > downloading with 800-1000K/s (maximum..), but when i upload the interrupt > is > around 800 with 8M/s speed.. > > please gime me some advice, patch, anything :) > thanks a lot! > OxY > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to > "freebsd-performance-unsubscribe@freebsd.org" From owner-freebsd-performance@FreeBSD.ORG Wed Dec 7 10:39:32 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7F5F516A41F for ; Wed, 7 Dec 2005 10:39:32 +0000 (GMT) (envelope-from imriz@co.zahav.net.il) Received: from mrx.wan.inter.net.il (mrx.wan.inter.net.il [192.117.191.51]) by mx1.FreeBSD.org (Postfix) with SMTP id BC46C43D55 for ; Wed, 7 Dec 2005 10:39:28 +0000 (GMT) (envelope-from imriz@co.zahav.net.il) Received: from igfe.InetGold ([172.33.1.137]) by mrx.wan.inter.net.il (SAVSMTP 3.1.0.29) with SMTP id M2005120712404927593 for ; Wed, 07 Dec 2005 12:40:49 +0200 Received: from IGMAIL.InetGold ([172.31.100.164]) by igfe.InetGold with Microsoft SMTPSVC(6.0.3790.1830); Wed, 7 Dec 2005 12:39:26 +0200 X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="windows-1255" Content-Transfer-Encoding: quoted-printable Date: Wed, 7 Dec 2005 12:44:52 +0200 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: very busy syslog server Thread-Index: AcX7Gn5cTwdte5XdQrKFj7UnjCL8uA== From: "Imri Zvik" To: X-OriginalArrivalTime: 07 Dec 2005 10:39:26.0729 (UTC) FILETIME=[7EEFBF90:01C5FB1A] Subject: very busy syslog server X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Dec 2005 10:39:32 -0000 Hi! I'm trying to setup a syslog server to serve a large group of servers. For the syslog daemon, I have chosen rsyslogd, and the backend is mysql = (on a different machine). The machine has 2 Intel Xeon 2.80GHz CPUs, and 1GB of RAM, and it is = running FreeBSD 6 (6.0-STABLE). The problem is, that I see a lot of UDP packets being dropped: udp: 390202 datagrams received 0 with incomplete header 0 with bad data length field 0 with bad checksum 6 with no checksum 0 dropped due to no socket 0 broadcast/multicast datagrams dropped due to no socket ->>> 123677 dropped due to full socket buffers 0 not for hashed pcb 266525 delivered 133260 datagrams output I have tried to increase net.inet.udp.recvspace, but it didn't solve the = problem. I would appreciate any hint or tips. Thank you in advance! -- Imri Zvik PGP (2.6.3ia) Public Key: http://mariska.inter.net.il/~imriz/imriz.pgp =A0 From owner-freebsd-performance@FreeBSD.ORG Wed Dec 7 11:14:37 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7618B16A420 for ; Wed, 7 Dec 2005 11:14:37 +0000 (GMT) (envelope-from kometen@gmail.com) Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.201]) by mx1.FreeBSD.org (Postfix) with ESMTP id F1F2B43D5F for ; Wed, 7 Dec 2005 11:14:36 +0000 (GMT) (envelope-from kometen@gmail.com) Received: by wproxy.gmail.com with SMTP id i34so323208wra for ; Wed, 07 Dec 2005 03:14:36 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=OSIqEZRu/bzp2OTlPc4cdrfssbt4rEXOaxEzRb+OuvMctaux491pZpis6Dkudq6s3yAKlZaxoDzUJ4CIQXB67qk6Xz6vWSBLDJrFGOQ+XHTTYmbIMG8wBCT3lHM7cnHjna079CDiB3pxXS07Q85MIuLBQFs13N4hg1TkaPv2Clc= Received: by 10.65.216.1 with SMTP id t1mr1653825qbq; Wed, 07 Dec 2005 03:14:35 -0800 (PST) Received: by 10.65.192.14 with HTTP; Wed, 7 Dec 2005 03:14:35 -0800 (PST) Message-ID: Date: Wed, 7 Dec 2005 12:14:35 +0100 From: Claus Guttesen To: Imri Zvik In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline References: Cc: freebsd-performance@freebsd.org Subject: Re: very busy syslog server X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Dec 2005 11:14:37 -0000 > I'm trying to setup a syslog server to serve a large group of servers. > For the syslog daemon, I have chosen rsyslogd, and the backend is mysql (= on a different > machine). Maby you could put the syslog-messages into a file and then import it into mysql? We have 10 webservers sending http-logs to one dual PIII. syslog~#>netstat -s -p udp udp: 167166048 datagrams received 0 with incomplete header 0 with bad data length field 0 with bad checksum 19371 with no checksum 810 dropped due to no socket 18964 broadcast/multicast datagrams dropped due to no socket 22911 dropped due to full socket buffers 0 not for hashed pcb 167123363 delivered 122985843 datagrams output They are kept in flat files. regards Claus From owner-freebsd-performance@FreeBSD.ORG Wed Dec 7 11:17:00 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 01CE016A41F for ; Wed, 7 Dec 2005 11:17:00 +0000 (GMT) (envelope-from trashy_bumper@yahoo.com) Received: from web36314.mail.mud.yahoo.com (web36314.mail.mud.yahoo.com [209.191.84.244]) by mx1.FreeBSD.org (Postfix) with SMTP id 58CB343D5D for ; Wed, 7 Dec 2005 11:16:59 +0000 (GMT) (envelope-from trashy_bumper@yahoo.com) Received: (qmail 49264 invoked by uid 60001); 7 Dec 2005 11:16:58 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=s3U5biPgCh12ReEvmkYZhQNPpv4ikINt6k8bGt4lthq2eZ8X7M3tgniqLk6IxYP4HSP+rGkXjN6bmbdqQGDk2yKHxU+TaZsX7D49LQow4+uTtj3g7gTYtvxowtydSgkH8HKyVVXcrpJ+M8FGWFHsEXohF/Z+yYjfyG43YMM8kaA= ; Message-ID: <20051207111658.49262.qmail@web36314.mail.mud.yahoo.com> Received: from [213.227.200.79] by web36314.mail.mud.yahoo.com via HTTP; Wed, 07 Dec 2005 03:16:58 PST Date: Wed, 7 Dec 2005 03:16:58 -0800 (PST) From: Nash Nipples To: freebsd-performance@freebsd.org In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: very busy syslog server X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Dec 2005 11:17:00 -0000 Or else it could be a udp fun. Just cut the crap. Nash Imri Zvik wrote: Hi! I'm trying to setup a syslog server to serve a large group of servers. For the syslog daemon, I have chosen rsyslogd, and the backend is mysql (on a different machine). The machine has 2 Intel Xeon 2.80GHz CPUs, and 1GB of RAM, and it is running FreeBSD 6 (6.0-STABLE). The problem is, that I see a lot of UDP packets being dropped: udp: 390202 datagrams received 0 with incomplete header 0 with bad data length field 0 with bad checksum 6 with no checksum 0 dropped due to no socket 0 broadcast/multicast datagrams dropped due to no socket ->>> 123677 dropped due to full socket buffers 0 not for hashed pcb 266525 delivered 133260 datagrams output I have tried to increase net.inet.udp.recvspace, but it didn't solve the problem. I would appreciate any hint or tips. Thank you in advance! -- Imri Zvik PGP (2.6.3ia) Public Key: http://mariska.inter.net.il/~imriz/imriz.pgp _______________________________________________ freebsd-performance@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-performance To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" --------------------------------- Yahoo! Shopping Find Great Deals on Gifts at Yahoo! Shopping From owner-freebsd-performance@FreeBSD.ORG Wed Dec 7 11:20:11 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C27DA16A41F for ; Wed, 7 Dec 2005 11:20:11 +0000 (GMT) (envelope-from imriz@co.zahav.net.il) Received: from mrx.wan.inter.net.il (mrx.wan.inter.net.il [192.117.191.51]) by mx1.FreeBSD.org (Postfix) with SMTP id A2BF943D6E for ; Wed, 7 Dec 2005 11:20:10 +0000 (GMT) (envelope-from imriz@co.zahav.net.il) Received: from igfe.InetGold ([172.33.1.137]) by mrx.wan.inter.net.il (SAVSMTP 3.1.0.29) with SMTP id M2005120713212830095 ; Wed, 07 Dec 2005 13:21:28 +0200 Received: from IGMAIL.InetGold ([172.31.100.164]) by igfe.InetGold with Microsoft SMTPSVC(6.0.3790.1830); Wed, 7 Dec 2005 13:20:06 +0200 X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Wed, 7 Dec 2005 13:25:31 +0200 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: very busy syslog server Thread-Index: AcX7ICurMkS5SrFNRrq0aHwuQ1E8rQAAKQcw From: "Imri Zvik" To: "Claus Guttesen" X-OriginalArrivalTime: 07 Dec 2005 11:20:06.0358 (UTC) FILETIME=[2D119760:01C5FB20] Cc: freebsd-performance@freebsd.org Subject: RE: very busy syslog server X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Dec 2005 11:20:11 -0000 Hi, First, thanks for the reply. Now, I see that you too are missing messages.... I don't know when was the last time you rested the counters, but missing 22911 events is quite a lot :/ Thanks! -- Imri Zvik PGP (2.6.3ia) Public Key: http://mariska.inter.net.il/~imriz/imriz.pgp =20 -----Original Message----- From: Claus Guttesen [mailto:kometen@gmail.com]=20 Sent: Wednesday, December 07, 2005 1:15 PM To: Imri Zvik Cc: freebsd-performance@freebsd.org Subject: Re: very busy syslog server > I'm trying to setup a syslog server to serve a large group of servers. > For the syslog daemon, I have chosen rsyslogd, and the backend is mysql (on a different > machine). Maby you could put the syslog-messages into a file and then import it into mysql? We have 10 webservers sending http-logs to one dual PIII. syslog~#>netstat -s -p udp udp: 167166048 datagrams received 0 with incomplete header 0 with bad data length field 0 with bad checksum 19371 with no checksum 810 dropped due to no socket 18964 broadcast/multicast datagrams dropped due to no socket 22911 dropped due to full socket buffers 0 not for hashed pcb 167123363 delivered 122985843 datagrams output They are kept in flat files. regards Claus From owner-freebsd-performance@FreeBSD.ORG Wed Dec 7 12:14:41 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9F11A16A420 for ; Wed, 7 Dec 2005 12:14:41 +0000 (GMT) (envelope-from kometen@gmail.com) Received: from zproxy.gmail.com (zproxy.gmail.com [64.233.162.201]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1D6C743D7B for ; Wed, 7 Dec 2005 12:14:37 +0000 (GMT) (envelope-from kometen@gmail.com) Received: by zproxy.gmail.com with SMTP id i11so314511nzi for ; Wed, 07 Dec 2005 04:14:33 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=eAnuIFe+M8g1BWVVKuBaY1W9JzNzduRU1zWinScU/ARP/4pJXudyD4yRm+4xsgysZnHjZFvKrbYgFfz2EioaDhnuld6bFoZfA8WmeP1NGW56afSpsNTUt24cd3PjYsSXtwtdTHQjmrJIiiF2T4TtlmxniGZrKiIstLBas2k2hvs= Received: by 10.65.20.1 with SMTP id x1mr1352773qbi; Wed, 07 Dec 2005 04:14:32 -0800 (PST) Received: by 10.65.192.14 with HTTP; Wed, 7 Dec 2005 04:14:32 -0800 (PST) Message-ID: Date: Wed, 7 Dec 2005 13:14:32 +0100 From: Claus Guttesen To: Imri Zvik In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline References: Cc: freebsd-performance@freebsd.org Subject: Re: very busy syslog server X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Dec 2005 12:14:41 -0000 2005/12/7, Imri Zvik : > Hi, > > First, thanks for the reply. > > Now, I see that you too are missing messages.... > I don't know when was the last time you rested the counters, but missing > 22911 events is quite a lot :/ Yes, you're right, some messages are dropped. syslog~#>bc bc 1.06 Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc. This is free software with ABSOLUTELY NO WARRANTY. For details type `warranty'. (22911*100)/167166048 .0137 22911 messages out of 167166048 is fine for me. regards Claus From owner-freebsd-performance@FreeBSD.ORG Wed Dec 7 18:41:18 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 56CF316A429 for ; Wed, 7 Dec 2005 18:41:18 +0000 (GMT) (envelope-from sean@sean.gigave.com) Received: from mailhost.gigave.com (mailhost.gigave.com [38.113.228.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 80F8543DA5 for ; Wed, 7 Dec 2005 18:41:04 +0000 (GMT) (envelope-from sean@sean.gigave.com) Date: Wed, 7 Dec 2005 10:41:03 -0800 From: Sean Chittenden To: Imri Zvik Message-ID: <20051207184103.GD56998@sean.gigave.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Cc: freebsd-performance@freebsd.org Subject: Re: very busy syslog server X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Dec 2005 18:41:18 -0000 > I'm trying to setup a syslog server to serve a large group of > servers. For the syslog daemon, I have chosen rsyslogd, and the > backend is mysql (on a different machine). > > The machine has 2 Intel Xeon 2.80GHz CPUs, and 1GB of RAM, and it is > running FreeBSD 6 (6.0-STABLE). > > The problem is, that I see a lot of UDP packets being dropped: > > udp: > 390202 datagrams received > 0 with incomplete header > 0 with bad data length field > 0 with bad checksum > 6 with no checksum > 0 dropped due to no socket > 0 broadcast/multicast datagrams dropped due to no socket > ->>> 123677 dropped due to full socket buffers > 0 not for hashed pcb > 266525 delivered > 133260 datagrams output > > I have tried to increase net.inet.udp.recvspace, but it didn't solve > the problem. > > I would appreciate any hint or tips. When you're doing a large number of packets per second, you may want to look into enabling device polling(4). Right now, every packet results in an interrupt. With device polling, you can handle more than one packet per interrupt. See the man page for details. If your syslogd's are blocking, waiting for the MySQL INSERT to complete, you may be dropping UDP packets and polling won't help you. I'd write messages out to a file then process them once a minute. *shrug* -sc -- Sean Chittenden From owner-freebsd-performance@FreeBSD.ORG Wed Dec 7 19:54:43 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 65C4716A41F for ; Wed, 7 Dec 2005 19:54:43 +0000 (GMT) (envelope-from jinmtb@sbcglobal.net) Received: from smtp113.sbc.mail.mud.yahoo.com (smtp113.sbc.mail.mud.yahoo.com [68.142.198.212]) by mx1.FreeBSD.org (Postfix) with SMTP id 0C19443D45 for ; Wed, 7 Dec 2005 19:54:34 +0000 (GMT) (envelope-from jinmtb@sbcglobal.net) Received: (qmail 12467 invoked from network); 7 Dec 2005 19:54:22 -0000 Received: from unknown (HELO ?192.168.2.10?) (jinmtb@sbcglobal.net@68.127.176.52 with plain) by smtp113.sbc.mail.mud.yahoo.com with SMTP; 7 Dec 2005 19:54:22 -0000 Message-ID: <43973E69.5000504@sbcglobal.net> Date: Wed, 07 Dec 2005 11:56:25 -0800 From: Jin Guojun User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.2) Gecko/20040804 Netscape/7.2 (ax) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Sean Chittenden References: <20051207184103.GD56998@sean.gigave.com> In-Reply-To: <20051207184103.GD56998@sean.gigave.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Imri Zvik , freebsd-performance@freebsd.org Subject: Re: very busy syslog server X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: j_guojun@lbl.gov List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Dec 2005 19:54:43 -0000 Sean Chittenden wrote: >>I'm trying to setup a syslog server to serve a large group of >>servers. For the syslog daemon, I have chosen rsyslogd, and the >>backend is mysql (on a different machine). >> >>The machine has 2 Intel Xeon 2.80GHz CPUs, and 1GB of RAM, and it is >>running FreeBSD 6 (6.0-STABLE). >> >>The problem is, that I see a lot of UDP packets being dropped: >> >>udp: >> 390202 datagrams received >> 0 with incomplete header >> 0 with bad data length field >> 0 with bad checksum >> 6 with no checksum >> 0 dropped due to no socket >> 0 broadcast/multicast datagrams dropped due to no socket >>->>> 123677 dropped due to full socket buffers >> 0 not for hashed pcb >> 266525 delivered >> 133260 datagrams output >> >>I have tried to increase net.inet.udp.recvspace, but it didn't solve >>the problem. >> >>I would appreciate any hint or tips. >> >> > >When you're doing a large number of packets per second, you may want >to look into enabling device polling(4). Right now, every packet >results in an interrupt. With device polling, you can handle more >than one packet per interrupt. See the man page for details. > Not quite, the interrupt interval depends on the device driver, or which NIC is used. A number NICs are able to to interrupt coalescence, which requires to increase buffer descriptor ring size (just for receiving buffer descriptors). Of course, polling is a simple thing to try. Before we can come up a better way to alter a better solution for this case, you also need to monitor a few things: What is NIC on this machine? What is the CUP utilization in average and in case the packet drops? You can simply write a script to do this instead of instructing kernel to do so (since this needs no super accurate): run vmstat to record CPU utilization in every 1 to 3 seconds for use when following event happens: use netstat watch UDP and pipe it to awk "netstat -udp | awk '$2=="drooped" {print $1; exit}'" every 3-5 seconds, and compare the result with previous one to see if any changes. If so, grep the last couple of line from vmstat output records. From your information, it seems that this machine has enough memory bandwidth for syslog needs, since it is not clear what this machine is for rlog daemon or sql server, or both are on the same machine. If the third case is true, then you may run out of memory bandwidth. Under this circumstance, you need to obtain the packet rate and the average packet size in order to determine the I/O and memory bandwidth requirements. -Jin Guojun From owner-freebsd-performance@FreeBSD.ORG Thu Dec 8 08:31:40 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 73C5516A420 for ; Thu, 8 Dec 2005 08:31:40 +0000 (GMT) (envelope-from imriz@co.zahav.net.il) Received: from mrx.wan.inter.net.il (mrx.wan.inter.net.il [192.117.191.51]) by mx1.FreeBSD.org (Postfix) with SMTP id 1C4B343D79 for ; Thu, 8 Dec 2005 08:31:28 +0000 (GMT) (envelope-from imriz@co.zahav.net.il) Received: from igfe.InetGold ([172.33.1.137]) by mrx.wan.inter.net.il (SAVSMTP 3.1.0.29) with SMTP id M2005120810322927914 ; Thu, 08 Dec 2005 10:32:29 +0200 Received: from IGMAIL.InetGold ([172.31.100.164]) by igfe.InetGold with Microsoft SMTPSVC(6.0.3790.1830); Thu, 8 Dec 2005 10:31:06 +0200 X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Thu, 8 Dec 2005 10:36:33 +0200 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: very busy syslog server Thread-Index: AcX7Xoyfe1qH67BURe693YO6GW3j6gActbsw From: "Imri Zvik" To: "Sean Chittenden" X-OriginalArrivalTime: 08 Dec 2005 08:31:06.0741 (UTC) FILETIME=[BBCC9250:01C5FBD1] Cc: freebsd-performance@freebsd.org Subject: RE: very busy syslog server X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Dec 2005 08:31:40 -0000 Hi, I have tried polling, and it did not improve the performance. Regarding the inserts - I am using delayed inserts to solve that problem exactly. I have already tried to log into files, and still had this issue... Thanks for reply :) -- Imri Zvik PGP (2.6.3ia) Public Key: http://mariska.inter.net.il/~imriz/imriz.pgp =20 -----Original Message----- From: Sean Chittenden [mailto:sean@gigave.com]=20 Sent: Wednesday, December 07, 2005 8:41 PM To: Imri Zvik Cc: freebsd-performance@freebsd.org Subject: Re: very busy syslog server > I'm trying to setup a syslog server to serve a large group of > servers. For the syslog daemon, I have chosen rsyslogd, and the > backend is mysql (on a different machine). >=20 > The machine has 2 Intel Xeon 2.80GHz CPUs, and 1GB of RAM, and it is > running FreeBSD 6 (6.0-STABLE). >=20 > The problem is, that I see a lot of UDP packets being dropped: >=20 > udp: > 390202 datagrams received > 0 with incomplete header > 0 with bad data length field > 0 with bad checksum > 6 with no checksum > 0 dropped due to no socket > 0 broadcast/multicast datagrams dropped due to no socket > ->>> 123677 dropped due to full socket buffers > 0 not for hashed pcb > 266525 delivered > 133260 datagrams output >=20 > I have tried to increase net.inet.udp.recvspace, but it didn't solve > the problem. >=20 > I would appreciate any hint or tips. When you're doing a large number of packets per second, you may want to look into enabling device polling(4). Right now, every packet results in an interrupt. With device polling, you can handle more than one packet per interrupt. See the man page for details. If your syslogd's are blocking, waiting for the MySQL INSERT to complete, you may be dropping UDP packets and polling won't help you. I'd write messages out to a file then process them once a minute. *shrug* -sc --=20 Sean Chittenden From owner-freebsd-performance@FreeBSD.ORG Thu Dec 8 08:50:38 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E68BA16A41F for ; Thu, 8 Dec 2005 08:50:37 +0000 (GMT) (envelope-from imriz@co.zahav.net.il) Received: from mrx.wan.inter.net.il (mrx.wan.inter.net.il [192.117.191.51]) by mx1.FreeBSD.org (Postfix) with SMTP id 30D5543D6A for ; Thu, 8 Dec 2005 08:50:30 +0000 (GMT) (envelope-from imriz@co.zahav.net.il) Received: from igfe.InetGold ([172.33.1.137]) by mrx.wan.inter.net.il (SAVSMTP 3.1.0.29) with SMTP id M2005120810515128896 ; Thu, 08 Dec 2005 10:51:51 +0200 Received: from IGMAIL.InetGold ([172.31.100.164]) by igfe.InetGold with Microsoft SMTPSVC(6.0.3790.1830); Thu, 8 Dec 2005 10:50:28 +0200 X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="windows-1255" Content-Transfer-Encoding: quoted-printable Date: Thu, 8 Dec 2005 10:55:55 +0200 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: very busy syslog server Thread-Index: AcX7aMko4yhDKh/mSFeap/gxfsD0eQAaQ6ZA From: "Imri Zvik" To: X-OriginalArrivalTime: 08 Dec 2005 08:50:28.0735 (UTC) FILETIME=[7066C8F0:01C5FBD4] Cc: freebsd-performance@freebsd.org Subject: RE: very busy syslog server X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Dec 2005 08:50:38 -0000 Hi, 1. The NIC being used is "Intel(R) PRO/1000" (the em(4) driver). 2. The CPU utilization in average is between 15% and 20%. 3. This machine is being used _only_ for the sysloging - the database = resides on another server. Meanwhile, I have added some more memory to the machine, and now it has = 3GB of RAM, but I am still seeing packets being dropped due to full = socket buffers. Thanks, -- Imri Zvik PGP (2.6.3ia) Public Key: http://mariska.inter.net.il/~imriz/imriz.pgp =A0 ________________________________________ From: Jin Guojun [mailto:jinmtb@sbcglobal.net]=20 Sent: Wednesday, December 07, 2005 9:56 PM To: Sean Chittenden Cc: Imri Zvik; freebsd-performance@freebsd.org Subject: Re: very busy syslog server Sean Chittenden wrote:=20 I'm trying to setup a syslog server to serve a large group of servers. For the syslog daemon, I have chosen rsyslogd, and the backend is mysql (on a different machine). The machine has 2 Intel Xeon 2.80GHz CPUs, and 1GB of RAM, and it is running FreeBSD 6 (6.0-STABLE). The problem is, that I see a lot of UDP packets being dropped: udp: 390202 datagrams received 0 with incomplete header 0 with bad data length field 0 with bad checksum 6 with no checksum 0 dropped due to no socket 0 broadcast/multicast datagrams dropped due to no socket ->>> 123677 dropped due to full socket buffers 0 not for hashed pcb 266525 delivered 133260 datagrams output I have tried to increase net.inet.udp.recvspace, but it didn't solve the problem. I would appreciate any hint or tips. =20 When you're doing a large number of packets per second, you may want to look into enabling device polling(4). Right now, every packet results in an interrupt. With device polling, you can handle more than one packet per interrupt. See the man page for details.=20 Not quite, the interrupt interval depends on the device driver, or which = NIC is used. A number NICs are able to to interrupt coalescence, which requires to = increase buffer descriptor ring size (just for receiving buffer descriptors). Of course, = polling is a simple thing to try. Before we can come up a better way to alter a better solution for this = case, you also need to monitor a few things: What is NIC on this machine? What is the CUP utilization in average and in case the packet drops? You = can simply write a script to do this instead of instructing kernel to do so (since this = needs no super accurate): run vmstat to record CPU utilization in every 1 to 3 seconds for use = when following event happens: use netstat watch UDP and pipe it to awk "netstat -udp | awk = '$2=3D=3D"drooped" {print $1; exit}'" every 3-5 seconds, and compare the result with previous one to see if = any changes. If so, grep the last couple of line from vmstat output records. >From your information, it seems that this machine has enough memory = bandwidth for syslog needs, since it is not clear what this machine is for rlog daemon or sql = server, or both are on the same machine. If the third case is true, then you may run out of memory bandwidth. = Under this circumstance, you need to obtain the packet rate and the average packet size in order = to determine the I/O and memory bandwidth requirements. =A0=A0=A0 -Jin Guojun From owner-freebsd-performance@FreeBSD.ORG Thu Dec 8 11:43:14 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C0F0616A420 for ; Thu, 8 Dec 2005 11:43:14 +0000 (GMT) (envelope-from jaws@skyinet.net) Received: from smtp1.skyinet.net (smtp1.skyinet.net [202.78.97.6]) by mx1.FreeBSD.org (Postfix) with ESMTP id A841E43D5C for ; Thu, 8 Dec 2005 11:43:05 +0000 (GMT) (envelope-from jaws@skyinet.net) Received: from [202.78.98.98] (unknown [202.78.98.98]) by smtp1.skyinet.net (Postfix) with ESMTP id 72C63582CC; Thu, 8 Dec 2005 19:43:02 +0800 (PHT) Message-ID: <43981D8B.9030605@skyinet.net> Date: Thu, 08 Dec 2005 19:48:27 +0800 From: jaws User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.12) Gecko/20051013 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Imri Zvik References: In-Reply-To: Content-Type: text/plain; charset=windows-1255; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-performance@freebsd.org Subject: Re: very busy syslog server X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Dec 2005 11:43:15 -0000 What you need is more sockets available and a much larger sockbufs. Try adjusting the following parameters: net.inet.udp.maxdgram net.inet.udp.recvspace Dont rely on the default values rather set it to a much higher values. Regards, jaws Imri Zvik wrote: >Hi, > >1. The NIC being used is "Intel(R) PRO/1000" (the em(4) driver). >2. The CPU utilization in average is between 15% and 20%. >3. This machine is being used _only_ for the sysloging - the database resides on another server. > >Meanwhile, I have added some more memory to the machine, and now it has 3GB of RAM, but I am still seeing packets being dropped due to full socket buffers. > >Thanks, > >-- >Imri Zvik >PGP (2.6.3ia) Public Key: http://mariska.inter.net.il/~imriz/imriz.pgp > >________________________________________ >From: Jin Guojun [mailto:jinmtb@sbcglobal.net] >Sent: Wednesday, December 07, 2005 9:56 PM >To: Sean Chittenden >Cc: Imri Zvik; freebsd-performance@freebsd.org >Subject: Re: very busy syslog server > >Sean Chittenden wrote: >I'm trying to setup a syslog server to serve a large group of >servers. For the syslog daemon, I have chosen rsyslogd, and the >backend is mysql (on a different machine). > >The machine has 2 Intel Xeon 2.80GHz CPUs, and 1GB of RAM, and it is >running FreeBSD 6 (6.0-STABLE). > >The problem is, that I see a lot of UDP packets being dropped: > >udp: > 390202 datagrams received > 0 with incomplete header > 0 with bad data length field > 0 with bad checksum > 6 with no checksum > 0 dropped due to no socket > 0 broadcast/multicast datagrams dropped due to no socket >->>> 123677 dropped due to full socket buffers > 0 not for hashed pcb > 266525 delivered > 133260 datagrams output > >I have tried to increase net.inet.udp.recvspace, but it didn't solve >the problem. > >I would appreciate any hint or tips. > > >When you're doing a large number of packets per second, you may want >to look into enabling device polling(4). Right now, every packet >results in an interrupt. With device polling, you can handle more >than one packet per interrupt. See the man page for details. >Not quite, the interrupt interval depends on the device driver, or which NIC is used. >A number NICs are able to to interrupt coalescence, which requires to increase buffer >descriptor ring size (just for receiving buffer descriptors). Of course, polling is a simple thing >to try. > >Before we can come up a better way to alter a better solution for this case, you also need to >monitor a few things: > >What is NIC on this machine? > >What is the CUP utilization in average and in case the packet drops? You can simply write a >script to do this instead of instructing kernel to do so (since this needs no super accurate): > >run vmstat to record CPU utilization in every 1 to 3 seconds for use when following event happens: >use netstat watch UDP and pipe it to awk "netstat -udp | awk '$2=="drooped" {print $1; exit}'" >every 3-5 seconds, and compare the result with previous one to see if any changes. If so, >grep the last couple of line from vmstat output records. > >>From your information, it seems that this machine has enough memory bandwidth for syslog needs, >since it is not clear what this machine is for rlog daemon or sql server, or both are on the same machine. >If the third case is true, then you may run out of memory bandwidth. Under this circumstance, >you need to obtain the packet rate and the average packet size in order to determine the I/O >and memory bandwidth requirements. > > -Jin Guojun >_______________________________________________ >freebsd-performance@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-performance >To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" > > > From owner-freebsd-performance@FreeBSD.ORG Thu Dec 8 11:49:51 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2860816A420 for ; Thu, 8 Dec 2005 11:49:51 +0000 (GMT) (envelope-from imriz@co.zahav.net.il) Received: from mrx.wan.inter.net.il (mrx.wan.inter.net.il [192.117.191.51]) by mx1.FreeBSD.org (Postfix) with SMTP id 6F55D43D8C for ; Thu, 8 Dec 2005 11:49:37 +0000 (GMT) (envelope-from imriz@co.zahav.net.il) Received: from igfe.InetGold ([172.33.1.137]) by mrx.wan.inter.net.il (SAVSMTP 3.1.0.29) with SMTP id M2005120813504605731 ; Thu, 08 Dec 2005 13:50:46 +0200 Received: from IGMAIL.InetGold ([172.31.100.164]) by igfe.InetGold with Microsoft SMTPSVC(6.0.3790.1830); Thu, 8 Dec 2005 13:49:24 +0200 X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Thu, 8 Dec 2005 13:54:51 +0200 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: very busy syslog server Thread-Index: AcX77VSyFQkHCRIkRCKNV70UWt2CBwAAD+4w From: "Imri Zvik" To: "jaws" X-OriginalArrivalTime: 08 Dec 2005 11:49:24.0139 (UTC) FILETIME=[6F336FB0:01C5FBED] Cc: freebsd-performance@freebsd.org Subject: RE: very busy syslog server X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Dec 2005 11:49:51 -0000 Already tried different values, even some really high arbitrary ones... I think I will try splitting the work load on a couple of instances of syslog...=20 I hope that would do the trick -- Imri Zvik PGP (2.6.3ia) Public Key: http://mariska.inter.net.il/~imriz/imriz.pgp =20 -----Original Message----- From: jaws [mailto:jaws@skyinet.net]=20 Sent: Thursday, December 08, 2005 1:48 PM To: Imri Zvik Cc: j_guojun@lbl.gov; freebsd-performance@freebsd.org Subject: Re: very busy syslog server What you need is more sockets available and a much larger sockbufs. Try=20 adjusting the following parameters: net.inet.udp.maxdgram net.inet.udp.recvspace Dont rely on the default values rather set it to a much higher values. Regards, jaws Imri Zvik wrote: >Hi, > >1. The NIC being used is "Intel(R) PRO/1000" (the em(4) driver). >2. The CPU utilization in average is between 15% and 20%. >3. This machine is being used _only_ for the sysloging - the database resides on another server. > >Meanwhile, I have added some more memory to the machine, and now it has 3GB of RAM, but I am still seeing packets being dropped due to full socket buffers. > >Thanks, > >-- >Imri Zvik >PGP (2.6.3ia) Public Key: http://mariska.inter.net.il/~imriz/imriz.pgp >=20 >________________________________________ >From: Jin Guojun [mailto:jinmtb@sbcglobal.net]=20 >Sent: Wednesday, December 07, 2005 9:56 PM >To: Sean Chittenden >Cc: Imri Zvik; freebsd-performance@freebsd.org >Subject: Re: very busy syslog server > >Sean Chittenden wrote:=20 >I'm trying to setup a syslog server to serve a large group of >servers. For the syslog daemon, I have chosen rsyslogd, and the >backend is mysql (on a different machine). > >The machine has 2 Intel Xeon 2.80GHz CPUs, and 1GB of RAM, and it is >running FreeBSD 6 (6.0-STABLE). > >The problem is, that I see a lot of UDP packets being dropped: > >udp: > 390202 datagrams received > 0 with incomplete header > 0 with bad data length field > 0 with bad checksum > 6 with no checksum > 0 dropped due to no socket > 0 broadcast/multicast datagrams dropped due to no socket >->>> 123677 dropped due to full socket buffers > 0 not for hashed pcb > 266525 delivered > 133260 datagrams output > >I have tried to increase net.inet.udp.recvspace, but it didn't solve >the problem. > >I would appreciate any hint or tips. > =20 > >When you're doing a large number of packets per second, you may want >to look into enabling device polling(4). Right now, every packet >results in an interrupt. With device polling, you can handle more >than one packet per interrupt. See the man page for details.=20 >Not quite, the interrupt interval depends on the device driver, or which NIC is used. >A number NICs are able to to interrupt coalescence, which requires to increase buffer >descriptor ring size (just for receiving buffer descriptors). Of course, polling is a simple thing >to try. > >Before we can come up a better way to alter a better solution for this case, you also need to >monitor a few things: > >What is NIC on this machine? > >What is the CUP utilization in average and in case the packet drops? You can simply write a >script to do this instead of instructing kernel to do so (since this needs no super accurate): > >run vmstat to record CPU utilization in every 1 to 3 seconds for use when following event happens: >use netstat watch UDP and pipe it to awk "netstat -udp | awk '$2=3D=3D"drooped" {print $1; exit}'" >every 3-5 seconds, and compare the result with previous one to see if any changes. If so, >grep the last couple of line from vmstat output records. > >>From your information, it seems that this machine has enough memory bandwidth for syslog needs, >since it is not clear what this machine is for rlog daemon or sql server, or both are on the same machine. >If the third case is true, then you may run out of memory bandwidth. Under this circumstance, >you need to obtain the packet rate and the average packet size in order to determine the I/O >and memory bandwidth requirements. > > -Jin Guojun >_______________________________________________ >freebsd-performance@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-performance >To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" > > =20 > From owner-freebsd-performance@FreeBSD.ORG Thu Dec 8 18:36:09 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6774316A420 for ; Thu, 8 Dec 2005 18:36:09 +0000 (GMT) (envelope-from j_guojun@lbl.gov) Received: from smtp106.sbc.mail.mud.yahoo.com (smtp106.sbc.mail.mud.yahoo.com [68.142.198.205]) by mx1.FreeBSD.org (Postfix) with SMTP id C4F9943D67 for ; Thu, 8 Dec 2005 18:36:01 +0000 (GMT) (envelope-from j_guojun@lbl.gov) Received: (qmail 20745 invoked from network); 8 Dec 2005 18:35:58 -0000 Received: from unknown (HELO ?192.168.2.8?) (jinmtb@sbcglobal.net@68.127.176.52 with plain) by smtp106.sbc.mail.mud.yahoo.com with SMTP; 8 Dec 2005 18:35:58 -0000 Message-ID: <43987E6A.8030508@lbl.gov> Date: Thu, 08 Dec 2005 10:41:46 -0800 From: "Jin Guojun [VFFS]" User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8b) Gecko/20051013 MIME-Version: 1.0 To: jaws References: <43981D8B.9030605@skyinet.net> In-Reply-To: <43981D8B.9030605@skyinet.net> Content-Type: text/plain; charset=windows-1255; format=flowed Content-Transfer-Encoding: 7bit Cc: Imri Zvik , freebsd-performance@freebsd.org Subject: Re: very busy syslog server X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Dec 2005 18:36:09 -0000 jaws wrote: > What you need is more sockets available and a much larger sockbufs. > Try adjusting the following parameters: > net.inet.udp.maxdgram > net.inet.udp.recvspace maxdgram has nothing to do with this on receiver side. If you want to play with this parameter, try to REDUCE it on sender side. But ignore this since the default is about 8-9KB, which is good enough. -- ------------ Jin Guojun ----------- v --- jin@george.lbl.gov --- Distributed Systems Department http://www.dsd.lbl.gov/~jin Lawrence Berkeley National Laboratory, Berkeley, CA 94720 From owner-freebsd-performance@FreeBSD.ORG Thu Dec 8 18:59:24 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D317516A432 for ; Thu, 8 Dec 2005 18:59:24 +0000 (GMT) (envelope-from j_guojun@lbl.gov) Received: from smtp104.sbc.mail.mud.yahoo.com (smtp104.sbc.mail.mud.yahoo.com [68.142.198.203]) by mx1.FreeBSD.org (Postfix) with SMTP id 8FD2243DA0 for ; Thu, 8 Dec 2005 18:59:08 +0000 (GMT) (envelope-from j_guojun@lbl.gov) Received: (qmail 85586 invoked from network); 8 Dec 2005 18:59:00 -0000 Received: from unknown (HELO ?192.168.2.8?) (jinmtb@sbcglobal.net@68.127.176.52 with plain) by smtp104.sbc.mail.mud.yahoo.com with SMTP; 8 Dec 2005 18:59:00 -0000 Message-ID: <439883D4.3090503@lbl.gov> Date: Thu, 08 Dec 2005 11:04:52 -0800 From: "Jin Guojun [VFFS]" User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8b) Gecko/20051013 MIME-Version: 1.0 To: Imri Zvik References: In-Reply-To: Content-Type: text/plain; charset=windows-1255; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-performance@freebsd.org Subject: Re: very busy syslog server X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Dec 2005 18:59:25 -0000 Clear enough. em(4) should be able to handle this amount traffic without polling unless all syslog traffics come at the same time that could cause congested resource. That is why I want you to run the script to watch the CPU utilization when the drop happens. The average CPU use does NOT reflect sparkling issue. If CPU utilization is lower than 60%, there is nothing to worry about interrupt, recvspace, etc. because CPU will have enough time to move data in and out. If you see CPU utilization over 60% and interrupt is also over 60%-80%, than interrupt coalescence or polling needs to be considered. At this moment, only one place with three conditions can cause such drop (6.0-Release): --- see function sbappendaddr_locked() in kern/uipc_socket2.c between line 934-942 --- recvspace, asa->sa_len, and number of mbufs. I doubt recvspace will be the problem since sending size (maxdgram) is much smaller than recvspace. sa_len should not be the case unless we had bug in 6.0. The last thing you may check is the mbufs -- type "netstat -m" to see the statistics on mbuf when drop happens. Since you have a lot of CPU time, try to run the script I mentioned to you and add "netstat -m" to the condition when drop count increasing. This should be a few minutes programming work, and run it for hours or a day. If you can get such info., we may know what is going on. We may have a bug since I just reboot my 6.0 box and see numbers of UDP drops, see below. Belkin: netstat -p udp -s udp: 148 datagrams received 0 with incomplete header 0 with bad data length field 0 with bad checksum 0 with no checksum 63 dropped due to no socket 20 broadcast/multicast datagrams dropped due to no socket 0 dropped due to full socket buffers 0 not for hashed pcb 65 delivered 68 datagrams output Belkin: netstat -p udp -s udp: 175 datagrams received 0 with incomplete header 0 with bad data length field 0 with bad checksum 0 with no checksum 69 dropped due to no socket 35 broadcast/multicast datagrams dropped due to no socket 0 dropped due to full socket buffers 0 not for hashed pcb 71 delivered 74 datagrams output Imri Zvik wrote: >Hi, > >1. The NIC being used is "Intel(R) PRO/1000" (the em(4) driver). >2. The CPU utilization in average is between 15% and 20%. >3. This machine is being used _only_ for the sysloging - the database resides on another server. > >Meanwhile, I have added some more memory to the machine, and now it has 3GB of RAM, but I am still seeing packets being dropped due to full socket buffers. > >Thanks, > >-- >Imri Zvik >PGP (2.6.3ia) Public Key: http://mariska.inter.net.il/~imriz/imriz.pgp > >________________________________________ >From: Jin Guojun [mailto:jinmtb@sbcglobal.net] >Sent: Wednesday, December 07, 2005 9:56 PM >To: Sean Chittenden >Cc: Imri Zvik; freebsd-performance@freebsd.org >Subject: Re: very busy syslog server > >Sean Chittenden wrote: >I'm trying to setup a syslog server to serve a large group of >servers. For the syslog daemon, I have chosen rsyslogd, and the >backend is mysql (on a different machine). > >The machine has 2 Intel Xeon 2.80GHz CPUs, and 1GB of RAM, and it is >running FreeBSD 6 (6.0-STABLE). > >The problem is, that I see a lot of UDP packets being dropped: > >udp: > 390202 datagrams received > 0 with incomplete header > 0 with bad data length field > 0 with bad checksum > 6 with no checksum > 0 dropped due to no socket > 0 broadcast/multicast datagrams dropped due to no socket >->>> 123677 dropped due to full socket buffers > 0 not for hashed pcb > 266525 delivered > 133260 datagrams output > >I have tried to increase net.inet.udp.recvspace, but it didn't solve >the problem. > >I would appreciate any hint or tips. > > >When you're doing a large number of packets per second, you may want >to look into enabling device polling(4). Right now, every packet >results in an interrupt. With device polling, you can handle more >than one packet per interrupt. See the man page for details. >Not quite, the interrupt interval depends on the device driver, or which NIC is used. >A number NICs are able to to interrupt coalescence, which requires to increase buffer >descriptor ring size (just for receiving buffer descriptors). Of course, polling is a simple thing >to try. > >Before we can come up a better way to alter a better solution for this case, you also need to >monitor a few things: > >What is NIC on this machine? > >What is the CUP utilization in average and in case the packet drops? You can simply write a >script to do this instead of instructing kernel to do so (since this needs no super accurate): > >run vmstat to record CPU utilization in every 1 to 3 seconds for use when following event happens: >use netstat watch UDP and pipe it to awk "netstat -udp | awk '$2=="drooped" {print $1; exit}'" >every 3-5 seconds, and compare the result with previous one to see if any changes. If so, >grep the last couple of line from vmstat output records. > >>From your information, it seems that this machine has enough memory bandwidth for syslog needs, >since it is not clear what this machine is for rlog daemon or sql server, or both are on the same machine. >If the third case is true, then you may run out of memory bandwidth. Under this circumstance, >you need to obtain the packet rate and the average packet size in order to determine the I/O >and memory bandwidth requirements. > > -Jin Guojun > > From owner-freebsd-performance@FreeBSD.ORG Fri Dec 9 00:06:13 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 62C6C16A41F for ; Fri, 9 Dec 2005 00:06:13 +0000 (GMT) (envelope-from grog@lemis.com) Received: from ext-gw.lemis.com (ext-gw.lemis.com [150.101.14.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 138D443D73 for ; Fri, 9 Dec 2005 00:06:05 +0000 (GMT) (envelope-from grog@lemis.com) Received: from wantadilla.lemis.com (wantadilla.lemis.com [192.109.197.135]) by ext-gw.lemis.com (Postfix) with ESMTP id 0A598131D4A; Fri, 9 Dec 2005 10:36:04 +1030 (CST) Received: by wantadilla.lemis.com (Postfix, from userid 1004) id 50FC1857BE; Fri, 9 Dec 2005 10:37:04 +1030 (CST) Date: Fri, 9 Dec 2005 10:37:04 +1030 From: Greg 'groggy' Lehey To: Michael Vince Message-ID: <20051209000704.GA80362@wantadilla.lemis.com> References: <04f401c5f630$b9af6570$6450fb40@guinness> <438FB036.3000804@roq.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="6TrnltStXW4iwmi0" Content-Disposition: inline In-Reply-To: <438FB036.3000804@roq.com> User-Agent: Mutt/1.4.2.1i Organization: The FreeBSD Project Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.FreeBSD.org/ X-PGP-Fingerprint: 9A1B 8202 BCCE B846 F92F 09AC 22E6 F290 507A 4223 Cc: freebsd-performance@freebsd.org, "Gustavo A. Baratto" Subject: Re: new benchmarks. WAS: FreeBSD MySQL still WAY slower than Linux X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Dec 2005 00:06:13 -0000 --6TrnltStXW4iwmi0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Friday, 2 December 2005 at 13:23:50 +1100, Michael Vince wrote: > It would be good to actually see the Linux performance on the exact same > hardware, all we ever see is how it is on FreeBSD. > > MySQL has very frequent use of queries of the system time and is > well known in FreeBSD to be slower because its more expensive to > call compared to Linux, which has less precision time. I've heard this claim again and again, and I intend to look at it when I have time. I find it difficult to believe that this alone could explain the sometimes horrendous performance differences (3 to 1) that have been reported. Can somebody tell me: 1. How many calls there are per second? 2. Where they're coming from? This would involve profiling, of course. Greg -- See complete headers for address and phone numbers. --6TrnltStXW4iwmi0 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (FreeBSD) iD8DBQFDmMqoIubykFB6QiMRAuF7AJ9E+LWKJQdU7dyKNqAiudj7423pVQCfZsh1 pHccrBQstu500DFvWAMLdTk= =vWLz -----END PGP SIGNATURE----- --6TrnltStXW4iwmi0-- From owner-freebsd-performance@FreeBSD.ORG Fri Dec 9 00:23:03 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from [127.0.0.1] (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id B860416A41F; Fri, 9 Dec 2005 00:23:02 +0000 (GMT) (envelope-from davidxu@freebsd.org) Message-ID: <4398CE7E.5050007@freebsd.org> Date: Fri, 09 Dec 2005 08:23:26 +0800 From: David Xu User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.7.10) Gecko/20050806 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Greg 'groggy' Lehey References: <04f401c5f630$b9af6570$6450fb40@guinness> <438FB036.3000804@roq.com> <20051209000704.GA80362@wantadilla.lemis.com> In-Reply-To: <20051209000704.GA80362@wantadilla.lemis.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: Michael Vince , freebsd-performance@freebsd.org, "Gustavo A. Baratto" Subject: Re: new benchmarks. WAS: FreeBSD MySQL still WAY slower than Linux X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Dec 2005 00:23:03 -0000 Greg 'groggy' Lehey wrote: >I've heard this claim again and again, and I intend to look at it when >I have time. I find it difficult to believe that this alone could >explain the sometimes horrendous performance differences (3 to 1) that >have been reported. > >Can somebody tell me: > >1. How many calls there are per second? >2. Where they're coming from? This would involve profiling, of > course. > >Greg >-- >See complete headers for address and phone numbers. > > You find ktrace result of mysql: http://people.freebsd.org/~davidxu/mysql/mysql_ktrace.txt gettimeofday() almost follows every network I/O. Also you can find its I/O size: http://people.freebsd.org/~davidxu/mysql/iosize.txt I guess the gettimeofday() call is relevant to mysql's connection keepalive work, sounds like a very silly method. David Xu From owner-freebsd-performance@FreeBSD.ORG Sat Dec 10 02:39:07 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0E4FE16A420 for ; Sat, 10 Dec 2005 02:39:07 +0000 (GMT) (envelope-from gbaratto@superb.net) Received: from smail2.superb.net (smail2.superb.net [64.251.80.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id A571843D55 for ; Sat, 10 Dec 2005 02:39:05 +0000 (GMT) (envelope-from gbaratto@superb.net) Received: from garrincha (S01060080c8118809.vc.shawcable.net [24.82.90.177]) by smail2.superb.net (Postfix) with ESMTP id A85A71F6EFC for ; Fri, 9 Dec 2005 18:39:00 -0800 (PST) Message-ID: <032801c5fd32$df28c3b0$6400a8c0@garrincha> From: "Gustavo A. Baratto" To: Date: Fri, 9 Dec 2005 18:38:57 -0800 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2527 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2527 Subject: mysql benchmarks X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Dec 2005 02:39:07 -0000 Since the last post just had freebsd numbers, I'm re-posting it including Linux as well. Both linux and freebsd numbers were taken from the same box: ++++ LINUX # uname -a Linux xoxoxoxox.com 2.6.14-1.1644_FC4smp #1 SMP Sun Nov 27 03:39:31 EST 2005 i686 i686 i386 GNU/Linux Fedora core 4 - mysql 4.0.26 - The RPM provided by mysql com ++++ Linux connecting to localhost # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Fri Dec 9 15:02:31 EST 2005 select_index 20000 0 0 22829.72 select_index 20000 0 0 22944.49 select_index 20000 0 0 23089.30 select_index 20000 0 0 23316.90 select_index 20000 0 0 22487.98 Fri Dec 9 15:02:35 EST 2005 ++++ Linux connecting to a local mysql using TCP, instead of socket: # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; dateFri Dec 9 15:03:59 EST 2005select_index 20000 0 0 21397.17select_index 20000 0 0 21040.54select_index 20000 2 0 21372.57select_index 20000 0 0 21242.07select_index 20000 0 0 22575.78Fri Dec 9 15:04:04 EST 2005+++++ FREEBSD - 6.0 RELEASE++++ lipthread: standard # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 16:20:52 EST 2005 select_index 20000 0 0 14570.99 select_index 20000 2 0 14787.32 select_index 20000 3 0 14621.65 select_index 20000 1 0 13554.19 select_index 20000 0 0 13514.42 Wed Nov 30 16:20:59 EST 2005 date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 22:45:37 EST 2005 select_index 20000 1 0 14567.60 select_index 20000 0 0 13745.24 select_index 20000 1 0 13625.50 select_index 20000 0 0 13558.84 select_index 20000 1 0 13460.09 Wed Nov 30 22:45:45 EST 2005 ++++ lipthread: LIBPTHREAD_PROCESS_SCOPE=yes # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 16:37:16 EST 2005 select_index 20000 2 0 12014.51 select_index 20000 2 0 11991.61 select_index 20000 0 0 10890.61 select_index 20000 1 0 11992.41 select_index 20000 3 0 11775.37 Wed Nov 30 16:37:24 EST 2005 date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 22:49:40 EST 2005 select_index 20000 1 0 11035.18 select_index 20000 4 0 11877.35 select_index 20000 0 0 10097.29 select_index 20000 0 0 10765.29 select_index 20000 1 0 11401.68 Wed Nov 30 22:49:50 EST 2005 ++++ libthr: # more /etc/libmap.conf [mysqld] libpthread.so.2 libthr.so.2 libpthread.so libthr.so # ldd /usr/local/libexec/mysqld /usr/local/libexec/mysqld: libz.so.3 => /lib/libz.so.3 (0x881f8000) libcrypt.so.3 => /lib/libcrypt.so.3 (0x88208000) libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0x88220000) libm.so.4 => /lib/libm.so.4 (0x882ea000) libpthread.so.2 => /usr/lib/libthr.so.2 (0x88300000) libc.so.6 => /lib/libc.so.6 (0x88312000) # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 16:38:45 EST 2005 select_index 20000 0 0 15789.56 select_index 20000 1 0 14804.51 select_index 20000 0 0 14810.69 select_index 20000 0 0 14600.25 select_index 20000 0 0 14478.79 Wed Nov 30 16:38:52 EST 2005 # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 22:47:26 EST 2005 select_index 20000 1 0 14375.95 select_index 20000 0 0 14392.69 select_index 20000 0 0 14478.93 select_index 20000 0 0 14371.66 select_index 20000 0 0 14416.07 Wed Nov 30 22:47:33 EST 2005 ++++ linuxthreads # ldd /usr/local/libexec/mysqld /usr/local/libexec/mysqld: liblthread.so.3 => /usr/local/lib/liblthread.so.3 (0x88201000) liblstdc++.so.5 => /usr/local/lib/liblstdc++.so.5 (0x88222000) libz.so.3 => /lib/libz.so.3 (0x882ed000) libcrypt.so.3 => /lib/libcrypt.so.3 (0x882fd000) libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0x88315000) libm.so.4 => /lib/libm.so.4 (0x883df000) libc.so.6 => /lib/libc.so.6 (0x883f5000) # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 23:30:24 EST 2005 select_index 20000 0 0 14357.28 select_index 20000 0 0 14055.69 select_index 20000 0 0 13694.37 select_index 20000 1 0 13394.22 select_index 20000 0 0 13723.05 Wed Nov 30 23:30:32 EST 2005 # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 23:30:38 EST 2005 select_index 20000 2 0 13555.20 select_index 20000 2 0 13500.62 select_index 20000 0 0 13546.00 select_index 20000 0 0 13161.49 select_index 20000 0 0 13510.82 Wed Nov 30 23:30:46 EST 2005 ++++ linuxthreads with query cache disabled (query_cache_size commented out in my.cnf):: # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 21:59:51 EST 2005 select_index 20000 1 0 8615.98 select_index 20000 3 0 8700.63 select_index 20000 1 0 8590.18 select_index 20000 1 0 8733.90 select_index 20000 1 0 8582.01 Wed Nov 30 22:00:02 EST 2005 ++++ libthr with query cache disabled (query_cache_size commented out in my.cnf): date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 22:52:42 EST 2005 select_index 20000 1 0 9511.13 select_index 20000 1 0 9464.96 select_index 20000 1 0 9229.52 select_index 20000 1 0 9195.97 select_index 20000 1 0 9346.92 Wed Nov 30 22:52:53 EST 2005 ++++ libthr, connecting to a local mysql using TCP, instead of socket # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 22:57:03 EST 2005 select_index 20000 1 0 9286.62 select_index 20000 2 0 9245.66 select_index 20000 1 0 9126.25 select_index 20000 0 0 9303.66 select_index 20000 1 0 9286.14 Wed Nov 30 22:57:15 EST 2005 ++++ linuxthreads, connecting to a local mysql using TCP, instead of socket # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 23:07:16 EST 2005 select_index 20000 2 0 8137.02 select_index 20000 2 0 8220.83 select_index 20000 3 0 8264.19 select_index 20000 1 0 8268.86 select_index 20000 3 0 8234.24 Wed Nov 30 23:07:28 EST 2005 ++++ my.cnf ---------------- [mysqld] server-id=2 log-bin=/var/log/mysql/binlog log-slow-queries=/var/log/mysql/slow-queries.log relay-log=/var/log/mysql/relay-log relay-log-index=/var/log/mysql/relay-log-index skip-innodb skip-bdb safe-user-create skip-name-resolve skip-log-warnings safe-show-database set-variable = flush_time=86400 set-variable = max_connections=300 set-variable = max_connect_errors=100 set-variable = interactive_timeout=600 set-variable = wait_timeout=60 set-variable = myisam-recover=FORCE set-variable = key_buffer_size=400M set-variable = sort_buffer_size=1M set-variable = read_buffer_size=512k set-variable = table_cache=4096 set-variable = max_delayed_threads=0 set-variable = max_user_connections=100 set-variable = query_cache_size=300M set-variable = thread_cache_size=50 [mysql.server] user=mysql [safe_mysqld] err-log=/var/log/mysql/error.log ------------------------ ++++ FREEBSD DMESG (Linux in the very end): ---------- # dmesg Copyright (c) 1992-2005 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 6.0-RELEASE #1: Tue Nov 29 23:36:08 EST 2005 gbaratto at sh-mysql4.dca1.superb.net:/usr/obj/usr/src/sys/GBARATTO Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Xeon(TM) CPU 3.06GHz (3066.79-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf29 Stepping = 9 Features=0xbfebfbff Features2=0x4400> Hyperthreading: 2 logical CPUs real memory = 1073741824 (1024 MB) avail memory = 1041698816 (993 MB) MPTable: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 6 ioapic0: Assuming intbase of 0 ioapic1: Assuming intbase of 24 ioapic2: Assuming intbase of 48 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard ioapic2 irqs 48-71 on motherboard npx0: [FAST] npx0: on motherboard npx0: INT 16 interface cpu0 on motherboard cpu1 on motherboard pcib0: pcibus 0 on motherboard pci0: on pcib0 pcib1: at device 2.0 on pci0 pci2: on pcib1 pci2: at device 28.0 (no driver attached) pcib2: at device 29.0 on pci2 pci5: on pcib2 em0: port 0xa400-0xa43f mem 0xfe9c0000-0xfe9dffff irq 48 at device 1.0 on pci5 em0: Ethernet address: 00:e0:81:2a:48:64 em0: Speed:N/A Duplex:N/A em1: port 0xa800-0xa83f mem 0xfe9e0000-0xfe9fffff irq 49 at device 1.1 on pci5 em1: Ethernet address: 00:e0:81:2a:48:65 em1: Speed:N/A Duplex:N/A pci2: at device 30.0 (no driver attached) pcib3: at device 31.0 on pci2 pci3: on pcib3 asr0: mem 0xfa000000-0xfbffffff irq 24 at device 3.0 on pci3 asr0: [GIANT-LOCKED] asr0: ADAPTEC 2110S FW Rev. 380E, 1 channel, 256 CCBs, Protocol I2O pcib4: at device 3.1 on pci3 pci4: on pcib4 uhci0: port 0xc400-0xc41f irq 16 at device 29.0 on pci0 uhci0: [GIANT-LOCKED] usb0: on uhci0 usb0: USB revision 1.0 uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered uhci1: port 0xc800-0xc81f irq 19 at device 29.1 on pci0 uhci1: [GIANT-LOCKED] usb1: on uhci1 usb1: USB revision 1.0 uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub1: 2 ports with 2 removable, self powered ehci0: mem 0xfebffc00-0xfebfffff irq 23 at device 29.7 on pci0 ehci0: [GIANT-LOCKED] usb2: waiting for BIOS to give up control usb2: timed out waiting for BIOS usb2: EHCI version 1.0 usb2: companion controllers, 2 ports each: usb0 usb1 usb2: on ehci0 usb2: USB revision 2.0 uhub2: Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1 uhub2: 4 ports with 4 removable, self powered pcib5: at device 30.0 on pci0 pci1: on pcib5 fxp0: port 0x9800-0x983f mem 0xfe6fe000-0xfe6fefff,0xfe6a0000-0xfe6bffff irq 17 at device 1.0 on pci1 miibus0: on fxp0 inphy0: on miibus0 inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto fxp0: Ethernet address: 00:e0:81:2a:48:35 pci1: at device 2.0 (no driver attached) isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfc00-0xfc0f at device 31.1 on pci0 ata0: on atapci0 ata1: on atapci0 pci0: at device 31.3 (no driver attached) pmtimer0 on isa0 orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xcdfff on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] fdc0: at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0 fdc0: [FAST] fd0: <1440-KB 3.5" drive> on fdc0 drive 0 ppc0: parallel port not found. sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio0: configured irq 4 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 sio0: type 16550A sio1: configured irq 3 not in bitmap of probed irqs 0 sio1: port may not be enabled sio1 at port 0x2f8-0x2ff irq 3 on isa0 sio1: type 16550A vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 unknown: can't assign resources (memory) unknown: can't assign resources (port) unknown: can't assign resources (port) unknown: can't assign resources (port) unknown: can't assign resources (port) Timecounters tick every 1.000 msec acd0: CDROM at ata1-master PIO4 da0 at asr0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-2 device da0: Tagged Queueing Enabled da0: 70006MB (143372288 512 byte sectors: 255H 63S/T 8924C) SMP: AP CPU #1 Launched! Trying to mount root from ufs:/dev/da0s1a em0: link state changed to UP ---------------- ++++ DIFF KERNEL # diff -bu /usr/src/sys/i386/conf/GENERIC /usr/src/sys/i386/conf/GBARATTO --- /usr/src/sys/i386/conf/GENERIC Fri Oct 28 15:22:41 2005 +++ /usr/src/sys/i386/conf/GBARATTO Tue Nov 29 23:29:19 2005 @@ -19,10 +19,8 @@ # $FreeBSD: src/sys/i386/conf/GENERIC,v 1.429.2.3.2.1 2005/10/28 19:22:41 jhb Exp $ machine i386 -cpu I486_CPU -cpu I586_CPU cpu I686_CPU -ident GENERIC +ident GBARATTO # To statically compile in device wiring instead of /boot/device.hints #hints "GENERIC.hints" # Default places to look for devices. @@ -64,6 +62,26 @@ options ADAPTIVE_GIANT # Giant mutex is adaptive. device apic # I/O APIC + +# ADDED BY GUSTAVO +options SMP + +device carp + +options ALTQ +options ALTQ_CBQ +options ALTQ_RED +options ALTQ_RIO +options ALTQ_HFSC +options ALTQ_CDNR +options ALTQ_PRIQ +options ALTQ_NOPCC # just for SMP builds + +options MAXDSIZ=(2048UL*1024*1024) #added by gustavo +options MAXSSIZ=(512UL*1024*1024) #added by gustavo +options DFLDSIZ=(2048UL*1024*1024) #added by gustavo +# + ++++ LINUX DMESG: # dmesg Linux version 2.6.14-1.1644_FC4smp (bhcompile@hs20-bc1-1.build.redhat.com) (gcc version 4.0.1 20050727 (Red Hat 4.0.1-5)) #1 SMP Sun Nov 27 03:39:31 EST 2005 BIOS-provided physical RAM map: BIOS-e820: 0000000000000000 - 000000000009fc00 (usable) BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved) BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved) BIOS-e820: 0000000000100000 - 0000000040000000 (usable) BIOS-e820: 00000000fec00000 - 00000000fed00000 (reserved) BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved) BIOS-e820: 00000000fff00000 - 0000000100000000 (reserved) 128MB HIGHMEM available. 896MB LOWMEM available. found SMP MP-table at 000ff780 Using x86 segment limits to approximate NX protection On node 0 totalpages: 262144 DMA zone: 4096 pages, LIFO batch:1 Normal zone: 225280 pages, LIFO batch:31 HighMem zone: 32768 pages, LIFO batch:15 DMI 2.3 present. Using APIC driver default ACPI: Unable to locate RSDP Intel MultiProcessor Specification v1.4 Virtual Wire compatibility mode. OEM ID: INTEL Product ID: KingsCanyonC APIC at: 0xFEE00000 Processor #0 15:2 APIC version 20 Processor #6 15:2 APIC version 20 I/O APIC #7 Version 32 at 0xFEC00000. I/O APIC #8 Version 32 at 0xFEC80000. I/O APIC #9 Version 32 at 0xFEC80400. Enabling APIC mode: Flat. Using 3 I/O APICs Processors: 2 Allocating PCI resources starting at 50000000 (gap: 40000000:bec00000) Built 1 zonelists Kernel command line: ro root=/dev/VolGroup00/LogVol00 mapped APIC to ffffd000 (fee00000) mapped IOAPIC to ffffc000 (fec00000) mapped IOAPIC to ffffb000 (fec80000) mapped IOAPIC to ffffa000 (fec80400) Initializing CPU#0 CPU 0 irqstacks, hard=c044a000 soft=c042a000 PID hash table entries: 4096 (order: 12, 65536 bytes) Detected 3067.305 MHz processor. Using tsc for high-res timesource Console: colour VGA+ 80x25 Dentry cache hash table entries: 131072 (order: 7, 524288 bytes) Inode-cache hash table entries: 65536 (order: 6, 262144 bytes) Memory: 1031912k/1048576k available (2177k kernel code, 15744k reserved, 812k data, 224k init, 131072k highmem) Checking if this processor honours the WP bit even in supervisor mode... Ok. Calibrating delay using timer specific routine.. 6142.88 BogoMIPS (lpj=12285760) Security Framework v1.0.0 initialized SELinux: Initializing. SELinux: Starting in permissive mode selinux_register_security: Registering secondary module capability Capability LSM initialized as secondary Mount-cache hash table entries: 512 CPU: After generic identify, caps: bfebfbff 00000000 00000000 00000000 00004400 00000000 00000000 CPU: After vendor identify, caps: bfebfbff 00000000 00000000 00000000 00004400 00000000 00000000 CPU: Trace cache: 12K uops, L1 D cache: 8K CPU: L2 cache: 512K CPU: Hyper-Threading is disabled CPU: After all inits, caps: bfebf3ff 00000000 00000000 00000080 00004400 00000000 00000000 Intel machine check architecture supported. Intel machine check reporting enabled on CPU#0. CPU0: Intel P4/Xeon Extended MCE MSRs (12) available CPU0: Thermal monitoring enabled mtrr: v2.0 (20020519) Enabling fast FPU save and restore... done. Enabling unmasked SIMD FPU exception support... done. Checking 'hlt' instruction... OK. CPU0: Intel(R) Xeon(TM) CPU 3.06GHz stepping 09 Booting processor 1/6 eip 2000 CPU 1 irqstacks, hard=c044b000 soft=c042b000 Initializing CPU#1 Calibrating delay using timer specific routine.. 6134.16 BogoMIPS (lpj=12268339) CPU: After generic identify, caps: bfebfbff 00000000 00000000 00000000 00004400 00000000 00000000 CPU: After vendor identify, caps: bfebfbff 00000000 00000000 00000000 00004400 00000000 00000000 CPU: Trace cache: 12K uops, L1 D cache: 8K CPU: L2 cache: 512K CPU: Hyper-Threading is disabled CPU: After all inits, caps: bfebf3ff 00000000 00000000 00000080 00004400 00000000 00000000 Intel machine check architecture supported. Intel machine check reporting enabled on CPU#1. CPU1: Intel P4/Xeon Extended MCE MSRs (12) available CPU1: Thermal monitoring enabled CPU1: Intel(R) Xeon(TM) CPU 3.06GHz stepping 09 Total of 2 processors activated (12277.04 BogoMIPS). ENABLING IO-APIC IRQs ..TIMER: vector=0x31 pin1=2 pin2=0 checking TSC synchronization across 2 CPUs: passed. Brought up 2 CPUs checking if image is initramfs... it is Freeing initrd memory: 1760k freed NET: Registered protocol family 16 PCI: PCI BIOS revision 2.10 entry at 0xf0031, last bus=5 PCI: Using configuration type 1 ACPI: Subsystem revision 20050902 ACPI: Interpreter disabled. Linux Plug and Play Support v0.97 (c) Adam Belay pnp: PnP ACPI: disabled usbcore: registered new driver usbfs usbcore: registered new driver hub PCI: Probing PCI hardware PCI: Probing PCI hardware (bus 00) PCI quirk: region 0800-087f claimed by ICH4 ACPI/GPIO/TCO PCI quirk: region 0480-04bf claimed by ICH4 GPIO PCI: Ignoring BAR0-3 of IDE controller 0000:00:1f.1 Boot video device is 0000:01:02.0 PCI: Transparent bridge - 0000:00:1e.0 PCI: Using IRQ router PIIX/ICH [8086/24d0] at 0000:00:1f.0 PCI->APIC IRQ transform: 0000:00:1d.0[A] -> IRQ 137 PCI->APIC IRQ transform: 0000:00:1d.1[B] -> IRQ 161 PCI->APIC IRQ transform: 0000:00:1d.7[D] -> IRQ 169 PCI->APIC IRQ transform: 0000:00:1f.1[A] -> IRQ 153 PCI->APIC IRQ transform: 0000:00:1f.3[B] -> IRQ 145 PCI->APIC IRQ transform: 0000:05:01.0[A] -> IRQ 185 PCI->APIC IRQ transform: 0000:05:01.1[B] -> IRQ 193 PCI->APIC IRQ transform: 0000:03:03.0[A] -> IRQ 177 PCI->APIC IRQ transform: 0000:01:01.0[A] -> IRQ 145 PCI->APIC IRQ transform: 0000:01:02.0[A] -> IRQ 153 PCI: Bridge: 0000:02:1d.0 IO window: a000-afff MEM window: fe900000-fe9fffff PREFETCH window: fc300000-fc3fffff PCI: Bridge: 0000:03:03.1 IO window: disabled. MEM window: disabled. PREFETCH window: disabled. PCI: Bridge: 0000:02:1f.0 IO window: disabled. MEM window: fe700000-fe8fffff PREFETCH window: f8100000-fc2fffff PCI: Bridge: 0000:00:02.0 IO window: a000-afff MEM window: fe700000-feafffff PREFETCH window: f8100000-fc4fffff PCI: Bridge: 0000:00:1e.0 IO window: 8000-9fff MEM window: fc600000-fe6fffff PREFETCH window: f8000000-f80fffff PCI: Setting latency timer of device 0000:00:1e.0 to 64 apm: BIOS not found. audit: initializing netlink socket (disabled) audit(1134082350.740:1): initialized highmem bounce pool size: 64 pages Total HugeTLB memory allocated, 0 VFS: Disk quotas dquot_6.5.1 Dquot-cache hash table entries: 1024 (order 0, 4096 bytes) SELinux: Registering netfilter hooks Initializing Cryptographic API ksign: Installing public key data Loading keyring - Added public key D3270678ACC294DC - User ID: Red Hat, Inc. (Kernel Module GPG key) pci_hotplug: PCI Hot Plug PCI Core version: 0.5 isapnp: Scanning for PnP cards... isapnp: No Plug & Play device found Real Time Clock Driver v1.12 Linux agpgart interface v0.101 (c) Dave Jones PNP: No PS/2 controller found. Probing ports directly. serio: i8042 AUX port at 0x60,0x64 irq 12 serio: i8042 KBD port at 0x60,0x64 irq 1 Serial: 8250/16550 driver $Revision: 1.90 $ 32 ports, IRQ sharing enabled ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A io scheduler noop registered io scheduler anticipatory registered io scheduler deadline registered io scheduler cfq registered RAMDISK driver initialized: 16 RAM disks of 16384K size 1024 blocksize Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2 ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx ICH5: IDE controller at PCI slot 0000:00:1f.1 PCI: Enabling device 0000:00:1f.1 (0005 -> 0007) ICH5: chipset revision 2 ICH5: not 100% native mode: will probe irqs later ide0: BM-DMA at 0xfc00-0xfc07, BIOS settings: hda:pio, hdb:pio ide1: BM-DMA at 0xfc08-0xfc0f, BIOS settings: hdc:DMA, hdd:pio Probing IDE interface ide0... Probing IDE interface ide1... hdc: CDU5211, ATAPI CD/DVD-ROM drive ide1 at 0x170-0x177,0x376 on irq 15 Probing IDE interface ide0... hdc: ATAPI 52X CD-ROM drive, 120kB Cache, UDMA(33) Uniform CD-ROM driver Revision: 3.20 ide-floppy driver 0.99.newide usbcore: registered new driver hiddev usbcore: registered new driver usbhid drivers/usb/input/hid-core.c: v2.6:USB HID core driver mice: PS/2 mouse device common for all mice md: md driver 0.90.2 MAX_MD_DEVS=256, MD_SB_DISKS=27 md: bitmap version 3.39 NET: Registered protocol family 2 IP route cache hash table entries: 65536 (order: 6, 262144 bytes) TCP established hash table entries: 131072 (order: 9, 2621440 bytes) TCP bind hash table entries: 65536 (order: 8, 1310720 bytes) TCP: Hash tables configured (established 131072 bind 65536) TCP reno registered TCP bic registered Initializing IPsec netlink socket NET: Registered protocol family 1 NET: Registered protocol family 17 Using IPI Shortcut mode Freeing unused kernel memory: 224k freed Write protecting the kernel read-only data: 331k SCSI subsystem initialized I2O subsystem v1.288 i2o: max drivers = 8 i2o: Checking for PCI I2O controllers... iop0: controller found (0000:03:03.0) iop0: using 64-bit DMA iop0: PCI I2O controller at FA000000 size=1048576 iop0: Installed at IRQ 177 i2o: iop0: Activating I2O controller... i2o: iop0: This may take a few minutes if there are many devices input: AT Translated Set 2 keyboard on isa0060/serio0 iop0: HRT has 1 entries of 16 bytes each. Adapter 00000012: <7>TID 0000:[<7>H<7>P<7>C<7>*<7>]:<7>PCI 1: Bus 1 Device 22 Function 0<7> i2o: iop0: Controller added I2O Block Device OSM v1.287 block-osm: registered device at major 80 i2o/hda: i2o/hda1 i2o/hda2 block-osm: device added (TID: 207): i2o/hda device-mapper: 4.4.0-ioctl (2005-01-12) initialised: dm-devel@redhat.com ide-cd: cmd 0x28 timed out hdc: DMA interrupt recovery hdc: lost interrupt hdc: status timeout: status=0xd0 { Busy } ide: failed opcode was: unknown hdc: DMA disabled hdc: drive not ready for command hdc: ATAPI reset complete EXT3-fs: INFO: recovery required on readonly filesystem. EXT3-fs: write access will be enabled during recovery. kjournald starting. Commit interval 5 seconds EXT3-fs: dm-0: orphan cleanup on readonly fs ext3_orphan_cleanup: deleting unreferenced inode 14403988 EXT3-fs: dm-0: 1 orphan inode deleted EXT3-fs: recovery complete. EXT3-fs: mounted filesystem with ordered data mode. SELinux: Disabled at runtime. SELinux: Unregistering netfilter hooks Floppy drive(s): fd0 is 1.44M FDC 0 is a post-1991 82077 e100: Intel(R) PRO/100 Network Driver, 3.4.14-k2-NAPI e100: Copyright(c) 1999-2005 Intel Corporation e100: eth0: e100_probe: addr 0xfe6fe000, irq 145, MAC addr 00:E0:81:2A:48:35 Intel(R) PRO/1000 Network Driver - version 6.0.60-k2-NAPI Copyright (c) 1999-2005 Intel Corporation. e1000: eth1: e1000_probe: Intel(R) PRO/1000 Network Connection e1000: eth2: e1000_probe: Intel(R) PRO/1000 Network Connection shpchp: acpi_shpchprm:get_device PCI ROOT HID fail=0x1001 shpchp: acpi_shpchprm:get_device PCI ROOT HID fail=0x1001 shpchp: acpi_shpchprm:get_device PCI ROOT HID fail=0x1001 hw_random: RNG not detected shpchp: acpi_shpchprm:get_device PCI ROOT HID fail=0x1001 PCI: Setting latency timer of device 0000:00:1d.7 to 64 ehci_hcd 0000:00:1d.7: EHCI Host Controller ehci_hcd 0000:00:1d.7: debug port 1 ehci_hcd 0000:00:1d.7: BIOS handoff failed (104, 01010001) ehci_hcd 0000:00:1d.7: continuing after BIOS bug... ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 1 ehci_hcd 0000:00:1d.7: irq 169, io mem 0xfebffc00 PCI: cache line size of 128 is not supported by device 0000:00:1d.7 ehci_hcd 0000:00:1d.7: USB 2.0 initialized, EHCI 1.00, driver 10 Dec 2004 hub 1-0:1.0: USB hub found hub 1-0:1.0: 4 ports detected USB Universal Host Controller Interface driver v2.3 PCI: Setting latency timer of device 0000:00:1d.0 to 64 uhci_hcd 0000:00:1d.0: UHCI Host Controller uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2 uhci_hcd 0000:00:1d.0: irq 137, io base 0x0000c400 hub 2-0:1.0: USB hub found hub 2-0:1.0: 2 ports detected PCI: Setting latency timer of device 0000:00:1d.1 to 64 uhci_hcd 0000:00:1d.1: UHCI Host Controller uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3 uhci_hcd 0000:00:1d.1: irq 161, io base 0x0000c800 hub 3-0:1.0: USB hub found hub 3-0:1.0: 2 ports detected md: Autodetecting RAID arrays. md: autorun ... md: ... autorun DONE. EXT3 FS on dm-0, internal journal kjournald starting. Commit interval 5 seconds EXT3 FS on i2o/hda1, internal journal EXT3-fs: mounted filesystem with ordered data mode. Adding 2031608k swap on /dev/VolGroup00/LogVol01. Priority:-1 extents:1 across:2031608k e1000: eth1: e1000_watchdog_task: NIC Link is Up 100 Mbps Half Duplex Bluetooth: Core ver 2.7 NET: Registered protocol family 31 Bluetooth: HCI device and connection manager initialized Bluetooth: HCI socket layer initialized Bluetooth: L2CAP ver 2.7 Bluetooth: L2CAP socket layer initialized Bluetooth: RFCOMM ver 1.5 Bluetooth: RFCOMM socket layer initialized Bluetooth: RFCOMM TTY layer initialized lp: driver loaded but no devices found NET: Registered protocol family 10 Disabled Privacy Extensions on device c0399a00(lo) IPv6 over IPv4 tunneling driver eth1: no IPv6 routers present eth0: no IPv6 routers present From owner-freebsd-performance@FreeBSD.ORG Sat Dec 10 02:52:24 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from [127.0.0.1] (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id 4D8DB16A41F; Sat, 10 Dec 2005 02:52:23 +0000 (GMT) (envelope-from davidxu@freebsd.org) Message-ID: <439A4301.5030404@freebsd.org> Date: Sat, 10 Dec 2005 10:52:49 +0800 From: David Xu User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.7.10) Gecko/20050806 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "Gustavo A. Baratto" References: <032801c5fd32$df28c3b0$6400a8c0@garrincha> In-Reply-To: <032801c5fd32$df28c3b0$6400a8c0@garrincha> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-performance@freebsd.org Subject: Re: mysql benchmarks X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Dec 2005 02:52:24 -0000 Gustavo A. Baratto wrote: > Since the last post just had freebsd numbers, I'm re-posting it including > Linux as well. Both linux and freebsd numbers were taken from the same > box: ... Can you try TSC timer on FreeBSD ? someone reported that using TSC timer boosts performance of super-smack significantly. David Xu From owner-freebsd-performance@FreeBSD.ORG Sat Dec 10 02:35:18 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4AA6116A41F for ; Sat, 10 Dec 2005 02:35:18 +0000 (GMT) (envelope-from gbaratto@hotmail.com) Received: from hotmail.com (bay101-dav6.bay101.hotmail.com [64.4.56.78]) by mx1.FreeBSD.org (Postfix) with ESMTP id D410D43D5C for ; Sat, 10 Dec 2005 02:35:17 +0000 (GMT) (envelope-from gbaratto@hotmail.com) Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Fri, 9 Dec 2005 18:35:17 -0800 Message-ID: Received: from 24.82.90.177 by BAY101-DAV6.phx.gbl with DAV; Sat, 10 Dec 2005 02:35:16 +0000 X-Originating-IP: [24.82.90.177] X-Originating-Email: [gbaratto@hotmail.com] X-Sender: gbaratto@hotmail.com From: "Gustavo A. Baratto" To: Date: Fri, 9 Dec 2005 18:35:10 -0800 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2527 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2527 X-OriginalArrivalTime: 10 Dec 2005 02:35:17.0139 (UTC) FILETIME=[5B455630:01C5FD32] X-Mailman-Approved-At: Sat, 10 Dec 2005 03:15:50 +0000 Subject: new mysql benchmarks X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Dec 2005 02:35:18 -0000 Since the last post just had freebsd numbers, I'm re-posting it including Linux as well. Both linux and freebsd numbers were taken from the same box: ++++ LINUX # uname -a Linux xoxoxoxox.com 2.6.14-1.1644_FC4smp #1 SMP Sun Nov 27 03:39:31 EST 2005 i686 i686 i386 GNU/Linux Fedora core 4 - mysql 4.0.26 - The RPM provided by mysql com ++++ Linux connecting to localhost # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Fri Dec 9 15:02:31 EST 2005 select_index 20000 0 0 22829.72 select_index 20000 0 0 22944.49 select_index 20000 0 0 23089.30 select_index 20000 0 0 23316.90 select_index 20000 0 0 22487.98 Fri Dec 9 15:02:35 EST 2005 ++++ Linux connecting to a local mysql using TCP, instead of socket: # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; dateFri Dec 9 15:03:59 EST 2005select_index 20000 0 0 21397.17select_index 20000 0 0 21040.54select_index 20000 2 0 21372.57select_index 20000 0 0 21242.07select_index 20000 0 0 22575.78Fri Dec 9 15:04:04 EST 2005+++++ FREEBSD - 6.0 RELEASE++++ lipthread: standard # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 16:20:52 EST 2005 select_index 20000 0 0 14570.99 select_index 20000 2 0 14787.32 select_index 20000 3 0 14621.65 select_index 20000 1 0 13554.19 select_index 20000 0 0 13514.42 Wed Nov 30 16:20:59 EST 2005 date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 22:45:37 EST 2005 select_index 20000 1 0 14567.60 select_index 20000 0 0 13745.24 select_index 20000 1 0 13625.50 select_index 20000 0 0 13558.84 select_index 20000 1 0 13460.09 Wed Nov 30 22:45:45 EST 2005 ++++ lipthread: LIBPTHREAD_PROCESS_SCOPE=yes # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 16:37:16 EST 2005 select_index 20000 2 0 12014.51 select_index 20000 2 0 11991.61 select_index 20000 0 0 10890.61 select_index 20000 1 0 11992.41 select_index 20000 3 0 11775.37 Wed Nov 30 16:37:24 EST 2005 date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 22:49:40 EST 2005 select_index 20000 1 0 11035.18 select_index 20000 4 0 11877.35 select_index 20000 0 0 10097.29 select_index 20000 0 0 10765.29 select_index 20000 1 0 11401.68 Wed Nov 30 22:49:50 EST 2005 ++++ libthr: # more /etc/libmap.conf [mysqld] libpthread.so.2 libthr.so.2 libpthread.so libthr.so # ldd /usr/local/libexec/mysqld /usr/local/libexec/mysqld: libz.so.3 => /lib/libz.so.3 (0x881f8000) libcrypt.so.3 => /lib/libcrypt.so.3 (0x88208000) libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0x88220000) libm.so.4 => /lib/libm.so.4 (0x882ea000) libpthread.so.2 => /usr/lib/libthr.so.2 (0x88300000) libc.so.6 => /lib/libc.so.6 (0x88312000) # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 16:38:45 EST 2005 select_index 20000 0 0 15789.56 select_index 20000 1 0 14804.51 select_index 20000 0 0 14810.69 select_index 20000 0 0 14600.25 select_index 20000 0 0 14478.79 Wed Nov 30 16:38:52 EST 2005 # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 22:47:26 EST 2005 select_index 20000 1 0 14375.95 select_index 20000 0 0 14392.69 select_index 20000 0 0 14478.93 select_index 20000 0 0 14371.66 select_index 20000 0 0 14416.07 Wed Nov 30 22:47:33 EST 2005 ++++ linuxthreads # ldd /usr/local/libexec/mysqld /usr/local/libexec/mysqld: liblthread.so.3 => /usr/local/lib/liblthread.so.3 (0x88201000) liblstdc++.so.5 => /usr/local/lib/liblstdc++.so.5 (0x88222000) libz.so.3 => /lib/libz.so.3 (0x882ed000) libcrypt.so.3 => /lib/libcrypt.so.3 (0x882fd000) libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0x88315000) libm.so.4 => /lib/libm.so.4 (0x883df000) libc.so.6 => /lib/libc.so.6 (0x883f5000) # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 23:30:24 EST 2005 select_index 20000 0 0 14357.28 select_index 20000 0 0 14055.69 select_index 20000 0 0 13694.37 select_index 20000 1 0 13394.22 select_index 20000 0 0 13723.05 Wed Nov 30 23:30:32 EST 2005 # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 23:30:38 EST 2005 select_index 20000 2 0 13555.20 select_index 20000 2 0 13500.62 select_index 20000 0 0 13546.00 select_index 20000 0 0 13161.49 select_index 20000 0 0 13510.82 Wed Nov 30 23:30:46 EST 2005 ++++ linuxthreads with query cache disabled (query_cache_size commented out in my.cnf):: # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 21:59:51 EST 2005 select_index 20000 1 0 8615.98 select_index 20000 3 0 8700.63 select_index 20000 1 0 8590.18 select_index 20000 1 0 8733.90 select_index 20000 1 0 8582.01 Wed Nov 30 22:00:02 EST 2005 ++++ libthr with query cache disabled (query_cache_size commented out in my.cnf): date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 22:52:42 EST 2005 select_index 20000 1 0 9511.13 select_index 20000 1 0 9464.96 select_index 20000 1 0 9229.52 select_index 20000 1 0 9195.97 select_index 20000 1 0 9346.92 Wed Nov 30 22:52:53 EST 2005 ++++ libthr, connecting to a local mysql using TCP, instead of socket # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 22:57:03 EST 2005 select_index 20000 1 0 9286.62 select_index 20000 2 0 9245.66 select_index 20000 1 0 9126.25 select_index 20000 0 0 9303.66 select_index 20000 1 0 9286.14 Wed Nov 30 22:57:15 EST 2005 ++++ linuxthreads, connecting to a local mysql using TCP, instead of socket # date; for i in 1 2 3 4 5; do super-smack select-key-mysql.smack 10 1000 | grep select_index; done; date Wed Nov 30 23:07:16 EST 2005 select_index 20000 2 0 8137.02 select_index 20000 2 0 8220.83 select_index 20000 3 0 8264.19 select_index 20000 1 0 8268.86 select_index 20000 3 0 8234.24 Wed Nov 30 23:07:28 EST 2005 ++++ my.cnf ---------------- [mysqld] server-id=2 log-bin=/var/log/mysql/binlog log-slow-queries=/var/log/mysql/slow-queries.log relay-log=/var/log/mysql/relay-log relay-log-index=/var/log/mysql/relay-log-index skip-innodb skip-bdb safe-user-create skip-name-resolve skip-log-warnings safe-show-database set-variable = flush_time=86400 set-variable = max_connections=300 set-variable = max_connect_errors=100 set-variable = interactive_timeout=600 set-variable = wait_timeout=60 set-variable = myisam-recover=FORCE set-variable = key_buffer_size=400M set-variable = sort_buffer_size=1M set-variable = read_buffer_size=512k set-variable = table_cache=4096 set-variable = max_delayed_threads=0 set-variable = max_user_connections=100 set-variable = query_cache_size=300M set-variable = thread_cache_size=50 [mysql.server] user=mysql [safe_mysqld] err-log=/var/log/mysql/error.log ------------------------ ++++ FREEBSD DMESG (Linux in the very end): ---------- # dmesg Copyright (c) 1992-2005 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 6.0-RELEASE #1: Tue Nov 29 23:36:08 EST 2005 gbaratto at sh-mysql4.dca1.superb.net:/usr/obj/usr/src/sys/GBARATTO Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Xeon(TM) CPU 3.06GHz (3066.79-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf29 Stepping = 9 Features=0xbfebfbff Features2=0x4400> Hyperthreading: 2 logical CPUs real memory = 1073741824 (1024 MB) avail memory = 1041698816 (993 MB) MPTable: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 6 ioapic0: Assuming intbase of 0 ioapic1: Assuming intbase of 24 ioapic2: Assuming intbase of 48 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard ioapic2 irqs 48-71 on motherboard npx0: [FAST] npx0: on motherboard npx0: INT 16 interface cpu0 on motherboard cpu1 on motherboard pcib0: pcibus 0 on motherboard pci0: on pcib0 pcib1: at device 2.0 on pci0 pci2: on pcib1 pci2: at device 28.0 (no driver attached) pcib2: at device 29.0 on pci2 pci5: on pcib2 em0: port 0xa400-0xa43f mem 0xfe9c0000-0xfe9dffff irq 48 at device 1.0 on pci5 em0: Ethernet address: 00:e0:81:2a:48:64 em0: Speed:N/A Duplex:N/A em1: port 0xa800-0xa83f mem 0xfe9e0000-0xfe9fffff irq 49 at device 1.1 on pci5 em1: Ethernet address: 00:e0:81:2a:48:65 em1: Speed:N/A Duplex:N/A pci2: at device 30.0 (no driver attached) pcib3: at device 31.0 on pci2 pci3: on pcib3 asr0: mem 0xfa000000-0xfbffffff irq 24 at device 3.0 on pci3 asr0: [GIANT-LOCKED] asr0: ADAPTEC 2110S FW Rev. 380E, 1 channel, 256 CCBs, Protocol I2O pcib4: at device 3.1 on pci3 pci4: on pcib4 uhci0: port 0xc400-0xc41f irq 16 at device 29.0 on pci0 uhci0: [GIANT-LOCKED] usb0: on uhci0 usb0: USB revision 1.0 uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered uhci1: port 0xc800-0xc81f irq 19 at device 29.1 on pci0 uhci1: [GIANT-LOCKED] usb1: on uhci1 usb1: USB revision 1.0 uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub1: 2 ports with 2 removable, self powered ehci0: mem 0xfebffc00-0xfebfffff irq 23 at device 29.7 on pci0 ehci0: [GIANT-LOCKED] usb2: waiting for BIOS to give up control usb2: timed out waiting for BIOS usb2: EHCI version 1.0 usb2: companion controllers, 2 ports each: usb0 usb1 usb2: on ehci0 usb2: USB revision 2.0 uhub2: Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1 uhub2: 4 ports with 4 removable, self powered pcib5: at device 30.0 on pci0 pci1: on pcib5 fxp0: port 0x9800-0x983f mem 0xfe6fe000-0xfe6fefff,0xfe6a0000-0xfe6bffff irq 17 at device 1.0 on pci1 miibus0: on fxp0 inphy0: on miibus0 inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto fxp0: Ethernet address: 00:e0:81:2a:48:35 pci1: at device 2.0 (no driver attached) isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfc00-0xfc0f at device 31.1 on pci0 ata0: on atapci0 ata1: on atapci0 pci0: at device 31.3 (no driver attached) pmtimer0 on isa0 orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xcdfff on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] fdc0: at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0 fdc0: [FAST] fd0: <1440-KB 3.5" drive> on fdc0 drive 0 ppc0: parallel port not found. sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio0: configured irq 4 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 sio0: type 16550A sio1: configured irq 3 not in bitmap of probed irqs 0 sio1: port may not be enabled sio1 at port 0x2f8-0x2ff irq 3 on isa0 sio1: type 16550A vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 unknown: can't assign resources (memory) unknown: can't assign resources (port) unknown: can't assign resources (port) unknown: can't assign resources (port) unknown: can't assign resources (port) Timecounters tick every 1.000 msec acd0: CDROM at ata1-master PIO4 da0 at asr0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-2 device da0: Tagged Queueing Enabled da0: 70006MB (143372288 512 byte sectors: 255H 63S/T 8924C) SMP: AP CPU #1 Launched! Trying to mount root from ufs:/dev/da0s1a em0: link state changed to UP ---------------- ++++ DIFF KERNEL # diff -bu /usr/src/sys/i386/conf/GENERIC /usr/src/sys/i386/conf/GBARATTO --- /usr/src/sys/i386/conf/GENERIC Fri Oct 28 15:22:41 2005 +++ /usr/src/sys/i386/conf/GBARATTO Tue Nov 29 23:29:19 2005 @@ -19,10 +19,8 @@ # $FreeBSD: src/sys/i386/conf/GENERIC,v 1.429.2.3.2.1 2005/10/28 19:22:41 jhb Exp $ machine i386 -cpu I486_CPU -cpu I586_CPU cpu I686_CPU -ident GENERIC +ident GBARATTO # To statically compile in device wiring instead of /boot/device.hints #hints "GENERIC.hints" # Default places to look for devices. @@ -64,6 +62,26 @@ options ADAPTIVE_GIANT # Giant mutex is adaptive. device apic # I/O APIC + +# ADDED BY GUSTAVO +options SMP + +device carp + +options ALTQ +options ALTQ_CBQ +options ALTQ_RED +options ALTQ_RIO +options ALTQ_HFSC +options ALTQ_CDNR +options ALTQ_PRIQ +options ALTQ_NOPCC # just for SMP builds + +options MAXDSIZ=(2048UL*1024*1024) #added by gustavo +options MAXSSIZ=(512UL*1024*1024) #added by gustavo +options DFLDSIZ=(2048UL*1024*1024) #added by gustavo +# + # Bus support. device eisaLinux dmegs:# dmesgLinux version 2.6.14-1.1644_FC4smp (bhcompile@hs20-bc1-1.build.redhat.com) (gcc version 4.0.1 20050727 (Red Hat 4.0.1-5)) #1 SMP Sun Nov 27 03:39:31 EST 2005BIOS-provided physical RAM map: BIOS-e820: 0000000000000000 - 000000000009fc00 (usable) BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved) BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved) BIOS-e820: 0000000000100000 - 0000000040000000 (usable) BIOS-e820: 00000000fec00000 - 00000000fed00000 (reserved) BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved) BIOS-e820: 00000000fff00000 - 0000000100000000 (reserved)128MB HIGHMEM available.896MB LOWMEM available.found SMP MP-table at 000ff780Using x86 segment limits to approximate NX protectionOn node 0 totalpages: 262144 DMA zone: 4096 pages, LIFO batch:1 Normal zone: 225280 pages, LIFO batch:31 HighMem zone: 32768 pages, LIFO batch:15DMI 2.3 present.Using APIC driver defaultACPI: Unable to locate RSDPIntel MultiProcessor Specification v1.4 Virtual Wire compatibility mode.OEM ID: INTEL Product ID: KingsCanyonC APIC at: 0xFEE00000Processor #0 15:2 APIC version 20Processor #6 15:2 APIC version 20I/O APIC #7 Version 32 at 0xFEC00000.I/O APIC #8 Version 32 at 0xFEC80000.I/O APIC #9 Version 32 at 0xFEC80400.Enabling APIC mode: Flat. Using 3 I/O APICsProcessors: 2Allocating PCI resources starting at 50000000 (gap: 40000000:bec00000)Built 1 zonelistsKernel command line: ro root=/dev/VolGroup00/LogVol00mapped APIC to ffffd000 (fee00000)mapped IOAPIC to ffffc000 (fec00000)mapped IOAPIC to ffffb000 (fec80000)mapped IOAPIC to ffffa000 (fec80400)Initializing CPU#0CPU 0 irqstacks, hard=c044a000 soft=c042a000PID hash table entries: 4096 (order: 12, 65536 bytes)Detected 3067.305 MHz processor.Using tsc for high-res timesourceConsole: colour VGA+ 80x25Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)Memory: 1031912k/1048576k available (2177k kernel code, 15744k reserved, 812k data, 224k init, 131072k highmem)Checking if this processor honours the WP bit even in supervisor mode... Ok.Calibrating delay using timer specific routine.. 6142.88 BogoMIPS (lpj=12285760)Security Framework v1.0.0 initializedSELinux: Initializing.SELinux: Starting in permissive modeselinux_register_security: Registering secondary module capabilityCapability LSM initialized as secondaryMount-cache hash table entries: 512CPU: After generic identify, caps: bfebfbff 00000000 00000000 00000000 00004400 00000000 00000000CPU: After vendor identify, caps: bfebfbff 00000000 00000000 00000000 00004400 00000000 00000000CPU: Trace cache: 12K uops, L1 D cache: 8KCPU: L2 cache: 512KCPU: Hyper-Threading is disabledCPU: After all inits, caps: bfebf3ff 00000000 00000000 00000080 00004400 00000000 00000000Intel machine check architecture supported.Intel machine check reporting enabled on CPU#0.CPU0: Intel P4/Xeon Extended MCE MSRs (12) availableCPU0: Thermal monitoring enabledmtrr: v2.0 (20020519)Enabling fast FPU save and restore... done.Enabling unmasked SIMD FPU exception support... done.Checking 'hlt' instruction... OK.CPU0: Intel(R) Xeon(TM) CPU 3.06GHz stepping 09Booting processor 1/6 eip 2000CPU 1 irqstacks, hard=c044b000 soft=c042b000Initializing CPU#1Calibrating delay using timer specific routine.. 6134.16 BogoMIPS (lpj=12268339)CPU: After generic identify, caps: bfebfbff 00000000 00000000 00000000 00004400 00000000 00000000CPU: After vendor identify, caps: bfebfbff 00000000 00000000 00000000 00004400 00000000 00000000CPU: Trace cache: 12K uops, L1 D cache: 8KCPU: L2 cache: 512KCPU: Hyper-Threading is disabledCPU: After all inits, caps: bfebf3ff 00000000 00000000 00000080 00004400 00000000 00000000Intel machine check architecture supported.Intel machine check reporting enabled on CPU#1.CPU1: Intel P4/Xeon Extended MCE MSRs (12) availableCPU1: Thermal monitoring enabledCPU1: Intel(R) Xeon(TM) CPU 3.06GHz stepping 09Total of 2 processors activated (12277.04 BogoMIPS).ENABLING IO-APIC IRQs..TIMER: vector=0x31 pin1=2 pin2=0checking TSC synchronization across 2 CPUs: passed.Brought up 2 CPUschecking if image is initramfs... it isFreeing initrd memory: 1760k freedNET: Registered protocol family 16PCI: PCI BIOS revision 2.10 entry at 0xf0031, last bus=5PCI: Using configuration type 1ACPI: Subsystem revision 20050902ACPI: Interpreter disabled.Linux Plug and Play Support v0.97 (c) Adam Belaypnp: PnP ACPI: disabledusbcore: registered new driver usbfsusbcore: registered new driver hubPCI: Probing PCI hardwarePCI: Probing PCI hardware (bus 00)PCI quirk: region 0800-087f claimed by ICH4 ACPI/GPIO/TCOPCI quirk: region 0480-04bf claimed by ICH4 GPIOPCI: Ignoring BAR0-3 of IDE controller 0000:00:1f.1Boot video device is 0000:01:02.0PCI: Transparent bridge - 0000:00:1e.0PCI: Using IRQ router PIIX/ICH [8086/24d0] at 0000:00:1f.0PCI->APIC IRQ transform: 0000:00:1d.0[A] -> IRQ 137PCI->APIC IRQ transform: 0000:00:1d.1[B] -> IRQ 161PCI->APIC IRQ transform: 0000:00:1d.7[D] -> IRQ 169PCI->APIC IRQ transform: 0000:00:1f.1[A] -> IRQ 153PCI->APIC IRQ transform: 0000:00:1f.3[B] -> IRQ 145PCI->APIC IRQ transform: 0000:05:01.0[A] -> IRQ 185PCI->APIC IRQ transform: 0000:05:01.1[B] -> IRQ 193PCI->APIC IRQ transform: 0000:03:03.0[A] -> IRQ 177PCI->APIC IRQ transform: 0000:01:01.0[A] -> IRQ 145PCI->APIC IRQ transform: 0000:01:02.0[A] -> IRQ 153PCI: Bridge: 0000:02:1d.0 IO window: a000-afff MEM window: fe900000-fe9fffff PREFETCH window: fc300000-fc3fffffPCI: Bridge: 0000:03:03.1 IO window: disabled. MEM window: disabled. PREFETCH window: disabled.PCI: Bridge: 0000:02:1f.0 IO window: disabled. MEM window: fe700000-fe8fffff PREFETCH window: f8100000-fc2fffffPCI: Bridge: 0000:00:02.0 IO window: a000-afff MEM window: fe700000-feafffff PREFETCH window: f8100000-fc4fffffPCI: Bridge: 0000:00:1e.0 IO window: 8000-9fff MEM window: fc600000-fe6fffff PREFETCH window: f8000000-f80fffffPCI: Setting latency timer of device 0000:00:1e.0 to 64apm: BIOS not found.audit: initializing netlink socket (disabled)audit(1134082350.740:1): initializedhighmem bounce pool size: 64 pagesTotal HugeTLB memory allocated, 0VFS: Disk quotas dquot_6.5.1Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)SELinux: Registering netfilter hooksInitializing Cryptographic APIksign: Installing public key dataLoading keyring- Added public key D3270678ACC294DC- User ID: Red Hat, Inc. (Kernel Module GPG key)pci_hotplug: PCI Hot Plug PCI Core version: 0.5isapnp: Scanning for PnP cards...isapnp: No Plug & Play device foundReal Time Clock Driver v1.12Linux agpgart interface v0.101 (c) Dave JonesPNP: No PS/2 controller found. Probing ports directly.serio: i8042 AUX port at 0x60,0x64 irq 12serio: i8042 KBD port at 0x60,0x64 irq 1Serial: 8250/16550 driver $Revision: 1.90 $ 32 ports, IRQ sharing enabledttyS0 at I/O 0x3f8 (irq = 4) is a 16550AttyS1 at I/O 0x2f8 (irq = 3) is a 16550Aio scheduler noop registeredio scheduler anticipatory registeredio scheduler deadline registeredio scheduler cfq registeredRAMDISK driver initialized: 16 RAM disks of 16384K size 1024 blocksizeUniform Multi-Platform E-IDE driver Revision: 7.00alpha2ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xxICH5: IDE controller at PCI slot 0000:00:1f.1PCI: Enabling device 0000:00:1f.1 (0005 -> 0007)ICH5: chipset revision 2ICH5: not 100% native mode: will probe irqs later ide0: BM-DMA at 0xfc00-0xfc07, BIOS settings: hda:pio, hdb:pio ide1: BM-DMA at 0xfc08-0xfc0f, BIOS settings: hdc:DMA, hdd:pioProbing IDE interface ide0...Probing IDE interface ide1...hdc: CDU5211, ATAPI CD/DVD-ROM driveide1 at 0x170-0x177,0x376 on irq 15Probing IDE interface ide0...hdc: ATAPI 52X CD-ROM drive, 120kB Cache, UDMA(33)Uniform CD-ROM driver Revision: 3.20ide-floppy driver 0.99.newideusbcore: registered new driver hiddevusbcore: registered new driver usbhiddrivers/usb/input/hid-core.c: v2.6:USB HID core drivermice: PS/2 mouse device common for all micemd: md driver 0.90.2 MAX_MD_DEVS=256, MD_SB_DISKS=27md: bitmap version 3.39NET: Registered protocol family 2IP route cache hash table entries: 65536 (order: 6, 262144 bytes)TCP established hash table entries: 131072 (order: 9, 2621440 bytes)TCP bind hash table entries: 65536 (order: 8, 1310720 bytes)TCP: Hash tables configured (established 131072 bind 65536)TCP reno registeredTCP bic registeredInitializing IPsec netlink socketNET: Registered protocol family 1NET: Registered protocol family 17Using IPI Shortcut modeFreeing unused kernel memory: 224k freedWrite protecting the kernel read-only data: 331kSCSI subsystem initializedI2O subsystem v1.288i2o: max drivers = 8i2o: Checking for PCI I2O controllers...iop0: controller found (0000:03:03.0)iop0: using 64-bit DMAiop0: PCI I2O controller at FA000000 size=1048576iop0: Installed at IRQ 177i2o: iop0: Activating I2O controller...i2o: iop0: This may take a few minutes if there are many devicesinput: AT Translated Set 2 keyboard on isa0060/serio0iop0: HRT has 1 entries of 16 bytes each.Adapter 00000012: <7>TID 0000:[<7>H<7>P<7>C<7>*<7>]:<7>PCI 1: Bus 1 Device 22 Function 0<7>i2o: iop0: Controller addedI2O Block Device OSM v1.287block-osm: registered device at major 80 i2o/hda: i2o/hda1 i2o/hda2block-osm: device added (TID: 207): i2o/hdadevice-mapper: 4.4.0-ioctl (2005-01-12) initialised: dm-devel@redhat.comide-cd: cmd 0x28 timed outhdc: DMA interrupt recoveryhdc: lost interrupthdc: status timeout: status=0xd0 { Busy }ide: failed opcode was: unknownhdc: DMA disabledhdc: drive not ready for commandhdc: ATAPI reset completeEXT3-fs: INFO: recovery required on readonly filesystem.EXT3-fs: write access will be enabled during recovery.kjournald starting. Commit interval 5 secondsEXT3-fs: dm-0: orphan cleanup on readonly fsext3_orphan_cleanup: deleting unreferenced inode 14403988EXT3-fs: dm-0: 1 orphan inode deletedEXT3-fs: recovery complete.EXT3-fs: mounted filesystem with ordered data mode.SELinux: Disabled at runtime.SELinux: Unregistering netfilter hooksFloppy drive(s): fd0 is 1.44MFDC 0 is a post-1991 82077e100: Intel(R) PRO/100 Network Driver, 3.4.14-k2-NAPIe100: Copyright(c) 1999-2005 Intel Corporatione100: eth0: e100_probe: addr 0xfe6fe000, irq 145, MAC addr 00:E0:81:2A:48:35Intel(R) PRO/1000 Network Driver - version 6.0.60-k2-NAPICopyright (c) 1999-2005 Intel Corporation.e1000: eth1: e1000_probe: Intel(R) PRO/1000 Network Connectione1000: eth2: e1000_probe: Intel(R) PRO/1000 Network Connectionshpchp: acpi_shpchprm:get_device PCI ROOT HID fail=0x1001shpchp: acpi_shpchprm:get_device PCI ROOT HID fail=0x1001shpchp: acpi_shpchprm:get_device PCI ROOT HID fail=0x1001hw_random: RNG not detectedshpchp: acpi_shpchprm:get_device PCI ROOT HID fail=0x1001PCI: Setting latency timer of device 0000:00:1d.7 to 64ehci_hcd 0000:00:1d.7: EHCI Host Controllerehci_hcd 0000:00:1d.7: debug port 1ehci_hcd 0000:00:1d.7: BIOS handoff failed (104, 01010001)ehci_hcd 0000:00:1d.7: continuing after BIOS bug...ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 1ehci_hcd 0000:00:1d.7: irq 169, io mem 0xfebffc00PCI: cache line size of 128 is not supported by device 0000:00:1d.7ehci_hcd 0000:00:1d.7: USB 2.0 initialized, EHCI 1.00, driver 10 Dec 2004hub 1-0:1.0: USB hub foundhub 1-0:1.0: 4 ports detectedUSB Universal Host Controller Interface driver v2.3PCI: Setting latency timer of device 0000:00:1d.0 to 64uhci_hcd 0000:00:1d.0: UHCI Host Controlleruhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2uhci_hcd 0000:00:1d.0: irq 137, io base 0x0000c400hub 2-0:1.0: USB hub foundhub 2-0:1.0: 2 ports detectedPCI: Setting latency timer of device 0000:00:1d.1 to 64uhci_hcd 0000:00:1d.1: UHCI Host Controlleruhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3uhci_hcd 0000:00:1d.1: irq 161, io base 0x0000c800hub 3-0:1.0: USB hub foundhub 3-0:1.0: 2 ports detectedmd: Autodetecting RAID arrays.md: autorun ...md: ... autorun DONE.EXT3 FS on dm-0, internal journalkjournald starting. Commit interval 5 secondsEXT3 FS on i2o/hda1, internal journalEXT3-fs: mounted filesystem with ordered data mode.Adding 2031608k swap on /dev/VolGroup00/LogVol01. Priority:-1 extents:1 across:2031608ke1000: eth1: e1000_watchdog_task: NIC Link is Up 100 Mbps Half DuplexBluetooth: Core ver 2.7NET: Registered protocol family 31Bluetooth: HCI device and connection manager initializedBluetooth: HCI socket layer initializedBluetooth: L2CAP ver 2.7Bluetooth: L2CAP socket layer initializedBluetooth: RFCOMM ver 1.5Bluetooth: RFCOMM socket layer initializedBluetooth: RFCOMM TTY layer initializedlp: driver loaded but no devices foundNET: Registered protocol family 10Disabled Privacy Extensions on device c0399a00(lo)IPv6 over IPv4 tunneling drivereth1: no IPv6 routers presenteth0: no IPv6 routers present From owner-freebsd-performance@FreeBSD.ORG Sat Dec 10 04:03:28 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 85A5416A41F for ; Sat, 10 Dec 2005 04:03:28 +0000 (GMT) (envelope-from j_guojun@lbl.gov) Received: from smtp104.sbc.mail.mud.yahoo.com (smtp104.sbc.mail.mud.yahoo.com [68.142.198.203]) by mx1.FreeBSD.org (Postfix) with SMTP id 4601E43D60 for ; Sat, 10 Dec 2005 04:03:27 +0000 (GMT) (envelope-from j_guojun@lbl.gov) Received: (qmail 38074 invoked from network); 10 Dec 2005 04:03:26 -0000 Received: from unknown (HELO ?192.168.2.10?) (jinmtb@sbcglobal.net@68.127.176.52 with plain) by smtp104.sbc.mail.mud.yahoo.com with SMTP; 10 Dec 2005 04:03:26 -0000 Message-ID: <439A5411.8090005@lbl.gov> Date: Fri, 09 Dec 2005 20:05:37 -0800 From: "Jin Guojun [VFFS]" User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.5) Gecko/20050108 X-Accept-Language: zh, zh-CN, en MIME-Version: 1.0 To: David Xu References: <032801c5fd32$df28c3b0$6400a8c0@garrincha> <439A4301.5030404@freebsd.org> In-Reply-To: <439A4301.5030404@freebsd.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-performance@freebsd.org, "Gustavo A. Baratto" Subject: Re: mysql benchmarks X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Dec 2005 04:03:28 -0000 David Xu wrote: > Gustavo A. Baratto wrote: > >> Since the last post just had freebsd numbers, I'm re-posting it >> including >> Linux as well. Both linux and freebsd numbers were taken from the >> same box: > > > ... > > Can you try TSC timer on FreeBSD ? someone reported that using TSC > timer boosts performance of super-smack significantly. > > David Xu "TSC" can be a few 100 times faster than gettimeofday(), but one needs to know how to calibrate and convert it to timer because TSC is counter, not timer. Also, if FreeBSD uses TSC to compare with Linux using gettimeofday, it does not sound a fun. gettimeofday is a very expensive function on all systems. http://dsd.lbl.gov/DIDC/papers/imc-2003.pdf tells how much it costs on some common systems, especially FreeBSD and Linux. So, if both systems use gettimeofday, then slow may be somewhere else. -- ------------ Jin Guojun ----------- v --- jin@george.lbl.gov --- Distributed Systems Department http://www.dsd.lbl.gov/~jin Lawrence Berkeley National Laboratory, Berkeley, CA 94720 From owner-freebsd-performance@FreeBSD.ORG Sat Dec 10 17:09:34 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4716616A41F for ; Sat, 10 Dec 2005 17:09:34 +0000 (GMT) (envelope-from mat@mat.cc) Received: from plouf.absolight.net (plouf.absolight.net [193.30.224.136]) by mx1.FreeBSD.org (Postfix) with ESMTP id DB89843D5A for ; Sat, 10 Dec 2005 17:09:33 +0000 (GMT) (envelope-from mat@mat.cc) Date: Sat, 10 Dec 2005 18:09:30 +0100 From: Mathieu Arnold To: Imri Zvik , freebsd-performance@freebsd.org Message-ID: In-Reply-To: References: X-Mailer: Mulberry/3.1.6 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Mailman-Approved-At: Sat, 10 Dec 2005 17:39:38 +0000 Cc: Subject: Re: very busy syslog server X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Dec 2005 17:09:34 -0000 +-le 07/12/2005 12:44 +0200, Imri Zvik =E9crivait : | Hi! |=20 | I'm trying to setup a syslog server to serve a large group of servers. | For the syslog daemon, I have chosen rsyslogd, and the backend is mysql (on | a different machine). |=20 | The machine has 2 Intel Xeon 2.80GHz CPUs, and 1GB of RAM, and it is | running FreeBSD 6 (6.0-STABLE). |=20 | The problem is, that I see a lot of UDP packets being dropped: without any tweaking : # netstat -s -p udp udp: 750858021 datagrams received 0 with incomplete header 0 with bad data length field 0 with bad checksum 20003 with no checksum 142075741 dropped due to no socket 1152246 broadcast/multicast datagrams dropped due to no socket 1729027 dropped due to full socket buffers 0 not for hashed pcb 605901007 delivered 801662 datagrams output it's been up for a bit, and I don't use syslog but minirsyslogd, which is in the ports tree I believe. CPU: Intel(R) Celeron(R) CPU 2.40GHz (2400.09-MHz 686-class CPU) Origin =3D "GenuineIntel" Id =3D 0xf29 Stepping =3D 9 Features=3D0xbfebfbff real memory =3D 259194880 (247 MB) avail memory =3D 243978240 (232 MB) --=20 Mathieu Arnold From owner-freebsd-performance@FreeBSD.ORG Sat Dec 10 22:56:09 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D3CB416A41F for ; Sat, 10 Dec 2005 22:56:09 +0000 (GMT) (envelope-from j_guojun@lbl.gov) Received: from smtp109.sbc.mail.mud.yahoo.com (smtp109.sbc.mail.mud.yahoo.com [68.142.198.208]) by mx1.FreeBSD.org (Postfix) with SMTP id 3C48743D5D for ; Sat, 10 Dec 2005 22:56:08 +0000 (GMT) (envelope-from j_guojun@lbl.gov) Received: (qmail 36072 invoked from network); 10 Dec 2005 22:56:08 -0000 Received: from unknown (HELO ?192.168.2.10?) (jinmtb@sbcglobal.net@68.127.176.52 with plain) by smtp109.sbc.mail.mud.yahoo.com with SMTP; 10 Dec 2005 22:56:08 -0000 Message-ID: <439B5D8D.70203@lbl.gov> Date: Sat, 10 Dec 2005 14:58:21 -0800 From: "Jin Guojun [VFFS]" User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.5) Gecko/20050108 X-Accept-Language: zh, zh-CN, en MIME-Version: 1.0 To: Mathieu Arnold References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: Imri Zvik , freebsd-performance@freebsd.org Subject: Re: very busy syslog server X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Dec 2005 22:56:10 -0000 What is "netstat -m" output on your machine? Mathieu Arnold wrote: >+-le 07/12/2005 12:44 +0200, Imri Zvik écrivait : >| Hi! >| >| I'm trying to setup a syslog server to serve a large group of servers. >| For the syslog daemon, I have chosen rsyslogd, and the backend is mysql (on >| a different machine). >| >| The machine has 2 Intel Xeon 2.80GHz CPUs, and 1GB of RAM, and it is >| running FreeBSD 6 (6.0-STABLE). >| >| The problem is, that I see a lot of UDP packets being dropped: > >without any tweaking : ># netstat -s -p udp >udp: > 750858021 datagrams received > 0 with incomplete header > 0 with bad data length field > 0 with bad checksum > 20003 with no checksum > 142075741 dropped due to no socket > 1152246 broadcast/multicast datagrams dropped due to no socket > 1729027 dropped due to full socket buffers > 0 not for hashed pcb > 605901007 delivered > 801662 datagrams output > >it's been up for a bit, and I don't use syslog but minirsyslogd, which is in >the ports tree I believe. > >CPU: Intel(R) Celeron(R) CPU 2.40GHz (2400.09-MHz 686-class CPU) > Origin = "GenuineIntel" Id = 0xf29 Stepping = 9 > >Features=0xbfebfbff >real memory = 259194880 (247 MB) >avail memory = 243978240 (232 MB) > >