From owner-freebsd-performance@FreeBSD.ORG Sun Mar 13 03:54:17 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 20EFB16A4CF for ; Sun, 13 Mar 2005 03:54:17 +0000 (GMT) Received: from salvador.pacific.net.sg (salvador.pacific.net.sg [203.120.90.219]) by mx1.FreeBSD.org (Postfix) with SMTP id A79D643D53 for ; Sun, 13 Mar 2005 03:54:15 +0000 (GMT) (envelope-from oceanare@pacific.net.sg) Received: (qmail 7609 invoked from network); 13 Mar 2005 03:54:13 -0000 Received: from unknown (HELO maxwell6.pacific.net.sg) (203.120.90.212) by salvador with SMTP; 13 Mar 2005 03:54:13 -0000 Received: from [192.168.0.107] ([210.24.122.133]) by maxwell6.pacific.net.sg with ESMTP <20050313035412.VITE1233.maxwell6.pacific.net.sg@[192.168.0.107]>; Sun, 13 Mar 2005 11:54:12 +0800 Message-ID: <4233B901.1090009@pacific.net.sg> Date: Sun, 13 Mar 2005 11:52:33 +0800 From: Erich Dollansky Organization: oceanare pte ltd User-Agent: Mozilla Thunderbird 1.0 (X11/20050224) X-Accept-Language: en-us, en MIME-Version: 1.0 To: ray@redshift.com References: <3.0.1.32.20050310180051.00a7e908@pop.redshift.com> <3.0.1.32.20050310180051.00a7e908@pop.redshift.com> <3.0.1.32.20050310193015.00a7e908@pop.redshift.com> In-Reply-To: <3.0.1.32.20050310193015.00a7e908@pop.redshift.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-performance@freebsd.org Subject: Re: performance modifications X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Mar 2005 03:54:17 -0000 Hi, ray@redshift.com wrote: > Okay, great, thanks. I'll check into that area. My biggest problem right now > is that PHP brings down the speed of everything. I may have to go back to Perl > and use mod_perl or look into some other alternatives. The main thing I wanted Isolate the parts with high number of hits and rewrite them in C as we did in the past. The client was surprised what is possible. The effort is lower than you might expect if you have C knowledge. Erich From owner-freebsd-performance@FreeBSD.ORG Mon Mar 14 03:01:37 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 26E2016A4CE for ; Mon, 14 Mar 2005 03:01:37 +0000 (GMT) Received: from outgoing.redshift.com (outgoing.redshift.com [207.177.231.8]) by mx1.FreeBSD.org (Postfix) with ESMTP id C6E1443D5D for ; Mon, 14 Mar 2005 03:01:36 +0000 (GMT) (envelope-from ray@redshift.com) Received: from workstation (216-228-19-21.dsl.redshift.com [216.228.19.21]) by outgoing.redshift.com (Postfix) with SMTP id 1F02F9700A; Sun, 13 Mar 2005 19:01:35 -0800 (PST) Message-Id: <3.0.1.32.20050313190145.00a8db40@pop.redshift.com> X-Mailer: na X-Sender: redshift.com Date: Sun, 13 Mar 2005 19:01:45 -0800 To: Erich Dollansky From: ray@redshift.com In-Reply-To: <4233B901.1090009@pacific.net.sg> References: <3.0.1.32.20050310193015.00a7e908@pop.redshift.com> <3.0.1.32.20050310180051.00a7e908@pop.redshift.com> <3.0.1.32.20050310180051.00a7e908@pop.redshift.com> <3.0.1.32.20050310193015.00a7e908@pop.redshift.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" cc: freebsd-performance@freebsd.org Subject: Re: performance modifications X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Mar 2005 03:01:37 -0000 Hi Erich, I wrote a small test program in C that just printed a single test line and it was very slow when called as a cgi via apache. Much slower than PHP. Is there something that needs to be done in order for Apache to run C without having to shell out to the OS (?) or something. Ray At 11:52 AM 3/13/2005 +0800, Erich Dollansky wrote: | Hi, | | ray@redshift.com wrote: | | > Okay, great, thanks. I'll check into that area. My biggest problem right now | > is that PHP brings down the speed of everything. I may have to go back to Perl | > and use mod_perl or look into some other alternatives. The main thing I wanted | | Isolate the parts with high number of hits and rewrite them in C as we | did in the past. | | The client was surprised what is possible. | | The effort is lower than you might expect if you have C knowledge. | | Erich | | From owner-freebsd-performance@FreeBSD.ORG Mon Mar 14 07:47:46 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BDC9516A4CE for ; Mon, 14 Mar 2005 07:47:46 +0000 (GMT) Received: from ford.blinkenlights.nl (ford.blinkenlights.nl [213.204.211.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4009143D2F for ; Mon, 14 Mar 2005 07:47:46 +0000 (GMT) (envelope-from sten@blinkenlights.nl) Received: from tea.blinkenlights.nl (tea.blinkenlights.nl [192.168.1.21]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ford.blinkenlights.nl (Postfix) with ESMTP id 8A0BF3F294; Mon, 14 Mar 2005 08:47:44 +0100 (CET) Received: by tea.blinkenlights.nl (Postfix, from userid 101) id 212E727C; Mon, 14 Mar 2005 08:47:44 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by tea.blinkenlights.nl (Postfix) with ESMTP id 0DB78265; Mon, 14 Mar 2005 08:47:44 +0100 (CET) Date: Mon, 14 Mar 2005 08:47:43 +0100 (CET) From: Sten Spans To: ray@redshift.com In-Reply-To: <3.0.1.32.20050313190145.00a8db40@pop.redshift.com> Message-ID: References: <3.0.1.32.20050310193015.00a7e908@pop.redshift.com> <3.0.1.32.20050310180051.00a7e908@pop.redshift.com> <3.0.1.32.20050310180051.00a7e908@pop.redshift.com> <3.0.1.32.20050310193015.00a7e908@pop.redshift.com> <3.0.1.32.20050313190145.00a8db40@pop.redshift.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed cc: freebsd-performance@freebsd.org cc: Erich Dollansky Subject: Re: performance modifications X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Mar 2005 07:47:46 -0000 On Sun, 13 Mar 2005 ray@redshift.com wrote: > Hi Erich, > > I wrote a small test program in C that just printed a single test > line and it was very slow when called as a cgi via apache. Much slower than > PHP. Is there something that needs to be done in order for Apache to run C > without having to shell out to the OS (?) or something. The main overhead should be fork/exec, this can be solved by running persistent cgi's through fastcgi, cgisock, etc. The main trick with tuning anything is to search for the bottlenecks, switching an entire site to another language solves nothing if the database server is the bottleneck. There are however a few easy tricks to speed things up a bit: 1- Move static content to a lightweight webserver, Apache with mod_php uses quite a bit of memory, which is wasted on static content. run thttpd/lighthttpd/etc for pictures.yourdomain.com 2- Database queries are another common bottleneck changing database layout, caching queries in sessions, etc usually helps quite a lot. 3- A simple brute-force solution is to cache generated content for short periods in squid/mod_proxy/etc or even cache it in the site code itself. Good luck. -- Sten Spans "There is a crack in everything, that's how the light gets in." Leonard Cohen - Anthem From owner-freebsd-performance@FreeBSD.ORG Mon Mar 14 09:07:10 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2BC1A16A4CE for ; Mon, 14 Mar 2005 09:07:10 +0000 (GMT) Received: from web26805.mail.ukl.yahoo.com (web26805.mail.ukl.yahoo.com [217.146.176.81]) by mx1.FreeBSD.org (Postfix) with SMTP id 2894243D5D for ; Mon, 14 Mar 2005 09:07:09 +0000 (GMT) (envelope-from cguttesen@yahoo.dk) Received: (qmail 4221 invoked by uid 60001); 14 Mar 2005 09:07:08 -0000 Message-ID: <20050314090708.4219.qmail@web26805.mail.ukl.yahoo.com> Received: from [194.248.174.58] by web26805.mail.ukl.yahoo.com via HTTP; Mon, 14 Mar 2005 10:07:08 CET Date: Mon, 14 Mar 2005 10:07:08 +0100 (CET) From: Claus Guttesen To: freebsd-performance@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Subject: postgresql, max_fsm_pages on FreeBSD 5.3 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Mar 2005 09:07:10 -0000 Hi. I'm running a postgresql 7.4.6 db, where the largest table having 13 mill. records. The db-server is a quad-opteron with 4 GB RAM. I've configured shared_buffers to 8192, sort_mem to 4096 and effective_cache_size to 4096. max_fsm_relations and max_fsm_pages was easy to configure when I found http://www.desknow.com/kb/idx/12/061/article/ (max_fsm_pages and max_fsm_relations). The odd thing is that according to a vacuum verbose where total pages needed always seems to be approx. 15.000 pages larger than what max_fsm_pages is configured to. So I keept increasing max_fsm_pages, from 60000, 80000 and now 100000. This morning my beloved crontab send me a verbose vacuum telling me "114848 total pages needed". max_fsm_relations remains steady at 300 (245 relations according to the output). Not that I mind increasing this value, but is there any limit to this value, or will I reach some thresshold where this setting will become counterproductive? The docs says that max_fsm_pages "Sets the maximum number of disk pages for which free space will be tracked in the shared free-space map." But what does that actually mean? Postgresql does seem to perform better when I tweak max_fsm_pages. The above mentioned shared buffers, sort mem and effective cache size did become counterproductive when I increased them too much. regards Claus From owner-freebsd-performance@FreeBSD.ORG Mon Mar 14 09:13:35 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6E2A816A4CE for ; Mon, 14 Mar 2005 09:13:35 +0000 (GMT) Received: from sarajevo.pacific.net.sg (sarajevo.pacific.net.sg [203.120.90.134]) by mx1.FreeBSD.org (Postfix) with SMTP id 28BCE43D41 for ; Mon, 14 Mar 2005 09:13:34 +0000 (GMT) (envelope-from oceanare@pacific.net.sg) Received: (qmail 9988 invoked from network); 14 Mar 2005 09:13:31 -0000 Received: from unknown (HELO maxwell2.pacific.net.sg) (203.120.90.192) by sarajevo with SMTP; 14 Mar 2005 09:13:30 -0000 Received: from [192.168.0.107] ([210.24.122.227]) by maxwell2.pacific.net.sg with ESMTP <20050314091331.WZRY1191.maxwell2.pacific.net.sg@[192.168.0.107]>; Mon, 14 Mar 2005 17:13:31 +0800 Message-ID: <423555B4.2010900@pacific.net.sg> Date: Mon, 14 Mar 2005 17:13:24 +0800 From: Erich Dollansky Organization: oceanare pte ltd User-Agent: Mozilla Thunderbird 1.0 (X11/20050224) X-Accept-Language: en-us, en MIME-Version: 1.0 To: ray@redshift.com References: <3.0.1.32.20050310193015.00a7e908@pop.redshift.com> <3.0.1.32.20050310180051.00a7e908@pop.redshift.com> <3.0.1.32.20050310180051.00a7e908@pop.redshift.com> <3.0.1.32.20050310193015.00a7e908@pop.redshift.com> <3.0.1.32.20050313190145.00a8db40@pop.redshift.com> In-Reply-To: <3.0.1.32.20050313190145.00a8db40@pop.redshift.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-performance@freebsd.org Subject: Re: performance modifications X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Mar 2005 09:13:35 -0000 Hi Ray, ray@redshift.com wrote: > Hi Erich, > > I wrote a small test program in C that just printed a single test > line and it was very slow when called as a cgi via apache. Much slower than > PHP. Is there something that needs to be done in order for Apache to run C > without having to shell out to the OS (?) or something. > I think you got already the answer to your question. If C would be helpful, writing an Apache module could be a solution. You module could then be linked with Apache. I would use this only as the last resort. When we use C with Apache we do not use any other scripting language. This allows us to remove things which make Apache big. Erich From owner-freebsd-performance@FreeBSD.ORG Mon Mar 14 11:16:26 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4A26816A4CE for ; Mon, 14 Mar 2005 11:16:26 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.192]) by mx1.FreeBSD.org (Postfix) with ESMTP id E37EA43D1D for ; Mon, 14 Mar 2005 11:16:25 +0000 (GMT) (envelope-from joseph.koshy@gmail.com) Received: by rproxy.gmail.com with SMTP id j1so1879328rnf for ; Mon, 14 Mar 2005 03:15:52 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=pnGIPFozWW038nhoiuixMFPpCVa3Qv9c8R/ODCnYiKVpKo6vJVo1/nlfUGzcHPZ+J4mnxP0XuuZ0FlA4p1Pr7fmVJscla2SobdfOiRHW3Rj8QYzi158c9G0yYSiT6luwK9DgGIVsBwnTA2/eNIsvrxax5pfrslb9IT2qWvjoUlg= Received: by 10.38.11.1 with SMTP id 1mr1247947rnk; Sun, 13 Mar 2005 19:43:35 -0800 (PST) Received: by 10.38.209.22 with HTTP; Sun, 13 Mar 2005 19:43:35 -0800 (PST) Message-ID: <84dead72050313194324d82953@mail.gmail.com> Date: Mon, 14 Mar 2005 03:43:35 +0000 From: Joseph Koshy To: "ray@redshift.com" In-Reply-To: <3.0.1.32.20050313190145.00a8db40@pop.redshift.com> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <3.0.1.32.20050310180051.00a7e908@pop.redshift.com> <3.0.1.32.20050310193015.00a7e908@pop.redshift.com> <4233B901.1090009@pacific.net.sg> <3.0.1.32.20050313190145.00a8db40@pop.redshift.com> cc: freebsd-performance@freebsd.org cc: Erich Dollansky Subject: Re: performance modifications X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Joseph Koshy List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Mar 2005 11:16:26 -0000 > without having to shell out to the OS (?) or something. FastCGI? http://www.fastcgi.com/devkit/doc/fastcgi-whitepaper/fastcgi.htm -- FreeBSD Volunteer, http://people.freebsd.org/~jkoshy From owner-freebsd-performance@FreeBSD.ORG Mon Mar 14 13:11:28 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C1D7D16A4CE for ; Mon, 14 Mar 2005 13:11:28 +0000 (GMT) Received: from web26808.mail.ukl.yahoo.com (web26808.mail.ukl.yahoo.com [217.146.176.84]) by mx1.FreeBSD.org (Postfix) with SMTP id 41CCD43D41 for ; Mon, 14 Mar 2005 13:11:28 +0000 (GMT) (envelope-from cguttesen@yahoo.dk) Received: (qmail 12500 invoked by uid 60001); 14 Mar 2005 13:11:27 -0000 Message-ID: <20050314131127.12498.qmail@web26808.mail.ukl.yahoo.com> Received: from [194.248.174.58] by web26808.mail.ukl.yahoo.com via HTTP; Mon, 14 Mar 2005 14:11:27 CET Date: Mon, 14 Mar 2005 14:11:27 +0100 (CET) From: Claus Guttesen To: freebsd-performance@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Subject: nfs and disc-storage X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Mar 2005 13:11:28 -0000 Hi. I have the option of going with an atabeast from nexsan, or a cx300 from dell (emc). Both have fiber. It will be connected to a nfs-server with some TB of data using a qlogic 2310 hba. Will the cx300 have much higher throughput compared to the atabeast? Which one should I go for? I'm primarily "concerned" with io-performance. regards Claus From owner-freebsd-performance@FreeBSD.ORG Mon Mar 14 14:49:38 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5B9E316A4CE for ; Mon, 14 Mar 2005 14:49:38 +0000 (GMT) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6DC4A43D49 for ; Mon, 14 Mar 2005 14:49:37 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id j2EEnZfV079865; Mon, 14 Mar 2005 08:49:35 -0600 (CST) (envelope-from anderson@centtech.com) Message-ID: <4235A479.9020802@centtech.com> Date: Mon, 14 Mar 2005 08:49:29 -0600 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.5) Gecko/20050210 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Claus Guttesen References: <20050314131127.12498.qmail@web26808.mail.ukl.yahoo.com> In-Reply-To: <20050314131127.12498.qmail@web26808.mail.ukl.yahoo.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.82/762/Sun Mar 13 17:35:33 2005 on mh1.centtech.com X-Virus-Status: Clean cc: freebsd-performance@freebsd.org Subject: Re: nfs and disc-storage X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Mar 2005 14:49:38 -0000 Claus Guttesen wrote: > Hi. > > I have the option of going with an atabeast from > nexsan, or a cx300 from dell (emc). Both have fiber. > It will be connected to a nfs-server with some TB of > data using a qlogic 2310 hba. > > Will the cx300 have much higher throughput compared to > the atabeast? Which one should I go for? I'm primarily > "concerned" with io-performance. I know the cx300's are fast - we ended up buying JetStor's (www.acnc.com) 16bay SATA-Fibre box, which is really snappy, and the support has been great. They officially support FreeBSD also, and I can honestly say that of the 6 we have so far, they all have been rock solid, and snappy boxes. If you want the ultimate performance, get the Raptor SATA drives, and configure in a RAID0+1 configuration (more $$ though), or a RAID5 for speed but less $$. I use these in an NFS serving environment also. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology I have seen the future and it is just like the present, only longer. ------------------------------------------------------------------------ From owner-freebsd-performance@FreeBSD.ORG Mon Mar 14 23:15:16 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F23B916A4CE for ; Mon, 14 Mar 2005 23:15:15 +0000 (GMT) Received: from mail.trippynames.com (mail.trippynames.com [38.113.223.19]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5E22A43D2F for ; Mon, 14 Mar 2005 23:15:13 +0000 (GMT) (envelope-from sean@chittenden.org) Received: from localhost (localhost [127.0.0.1]) by mail.trippynames.com (Postfix) with ESMTP id 12F2AC9AB7; Mon, 14 Mar 2005 15:15:13 -0800 (PST) Received: from mail.trippynames.com ([127.0.0.1]) by localhost (rand.nxad.com [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 20273-06; Mon, 14 Mar 2005 15:15:10 -0800 (PST) Received: from [192.168.102.100] (dsl231-047-005.sea1.dsl.speakeasy.net [216.231.47.5]) by mail.trippynames.com (Postfix) with ESMTP id E08C9C9AB6; Mon, 14 Mar 2005 15:15:09 -0800 (PST) In-Reply-To: <3.0.1.32.20050310193015.00a7e908@pop.redshift.com> References: <3.0.1.32.20050310180051.00a7e908@pop.redshift.com> <3.0.1.32.20050310180051.00a7e908@pop.redshift.com> <3.0.1.32.20050310193015.00a7e908@pop.redshift.com> Mime-Version: 1.0 (Apple Message framework v619.2) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <562e6c1c0720b8cc79f03b71942b08c4@chittenden.org> Content-Transfer-Encoding: 7bit From: Sean Chittenden Date: Mon, 14 Mar 2005 15:15:07 -0800 To: ray@redshift.com X-Mailer: Apple Mail (2.619.2) cc: freebsd-performance@freebsd.org Subject: Re: performance modifications X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Mar 2005 23:15:16 -0000 > | There are better web servers than Apache for demanding loads; > | ones that used a non-forking, event-driven I/O model. > | Aolserver and thttpd come to mind. > > I tried thttpd a number of years ago, but due to its limited cgi > support (at > least at the time) I didn't do too much with it. It did seem fast, > however. thttpd is a favorite of mine because of the time I've spent in its guts rewriting it for clients, but embedding PHP into it will never work well because of the way the code is written and organization. Recently I've been playing around with lighttpd and am very impressed. It's BSD licensed (3 clause) and has quickly become my new webserver of choice (*mutters something about Rails*). It even supports keep-alives well, which is something that thttpd doesn't do too well. Here are some benchmarks put out by a commercial webserver company, Lite Speed. http://www.litespeedtech.com/benchmark.html The lightspeed webserver guys are keen to point out that their webserver is the fastest on the block, but I'm skeptical of their results given the utility they're using for testing. I'm in the midst of writing a kqueue(2)/pthreads(3) backed web-benchmark program that will hopefully show that the gap between lighttpd and lite speed is smaller than the above benchmarks point out (if there's any difference at all). Any testing utility that's select(2) backed is going to suck up more resources than the server.... but I digress. If you're doing PHP, lighttpd should work very well for you and certainly be orders of magnitude better than Apache. http://www.lighttpd.net/ Having skimmed through the lighttpd code, the areas that it could improve are: *) pre-forking a pool of backends for CGI requests (lighttpd gets crushed by litespeed here) *) mmap(2)'ing buffers to reduce the overhead of writev(2)'ing data out over the network (ie: reduce copying of userland data to the kernel) *) use Boehm GC or make use of a memory pools that stay around longer than a connection. Right now every connection gets its own memory pool which is a problem in non-keep-alive connections. At the beginning and end of a connection, a pool gets allocated and free'ed. Free'ed memory from other connections is only reused by malloc(3), not by the OS. *) A better method of appending data to buffers (ie, use the same idea from Boehm's Cord library to reduce the costs of appending strings of data). Right now it uses memcpy(3) and it should just add a pointer to an array and realloc(3) the pointer array when the pointer array gets full instead of copy'ing data around. *) Divvy out requests to a pool of worker threads much the same way that Cherokee does (a thttpd based webserver that's been adapted to use pthreads... ie, thttpd on steroids) http://www.alobbs.com/modules.php? op=modload&name=News&file=article&sid=104 The improvement should come with threaded processing and dispatching of requests. Right now that's serialized and is the difference between litespeed pro and litespeed. That's also one of the ways that Cherokee makes its speed improvements over other webservers. -sc -- Sean Chittenden From owner-freebsd-performance@FreeBSD.ORG Tue Mar 15 15:30:44 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C3F7216A4CE for ; Tue, 15 Mar 2005 15:30:44 +0000 (GMT) Received: from mr01.hansenet.de (mr01.hansenet.de [213.191.74.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5BAC143D1F for ; Tue, 15 Mar 2005 15:30:44 +0000 (GMT) (envelope-from db@nipsi.home.net) Received: from nipsi.home.net (213.39.140.190) by mr01.hansenet.de (6.7.010) id 422EC055000276E2 for freebsd-performance@freebsd.org; Tue, 15 Mar 2005 16:30:42 +0100 Received: from localhost (localhost [127.0.0.1]) by nipsi.home.net (Postfix) with ESMTP id A5C2E1E6F3 for ; Tue, 15 Mar 2005 16:32:11 +0100 (CET) Received: from nipsi.home.net ([127.0.0.1]) by localhost (nipsi.home.net [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 07721-09 for ; Tue, 15 Mar 2005 16:32:09 +0100 (CET) Received: by nipsi.home.net (Postfix, from userid 1001) id D1AE31E701; Tue, 15 Mar 2005 16:32:09 +0100 (CET) Date: Tue, 15 Mar 2005 16:32:09 +0100 From: Dennis Berger To: freebsd-performance@freebsd.org Message-ID: <20050315153209.GA8863@nipsi.home.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.6i X-Virus-Scanned: amavisd-new at ARRAY(0x87d7e14) Subject: vfs.ufs.dirhash_maxmem is a bit low X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Mar 2005 15:30:44 -0000 hi all, I recently had the problem to change a line in about 2.5 million files. In about 1530 subdirectories. i started this command. find . \( -not -name restoresym\* -and -not -name \*.log -and -not -name \*.gif \) -type f -print0 | xargs -0 sed -i '' 's/\/wurstbrot\//\/kaesebrot\//' on another shell i watched with gstat how it works after several minutes i noticed a terrible breakdown. no read/write anymore. After watching with top i notice 25% system usage. (it's a 4 processor mashine) I tracked down the error to the vfs.ufs.dirhash. I guess the system is running out of memory for the hashtable so everything is getting terrible slow. I raised vfs.ufs.dirhash_maxmem to 30 mb. after that everything was fine again. My request can we add white kernel message for the behavier of running out of hashmemory? Like it is for running out of open files. A message will appear on /dev/console indicating that the dirhash is too low. Or pointing out in tuning(9) that it should raised of servers with lots of files. Or raised the default from 2MB to 10MB or something. any comments? ragards, -Dennis From owner-freebsd-performance@FreeBSD.ORG Tue Mar 15 15:47:22 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9320816A4CE for ; Tue, 15 Mar 2005 15:47:22 +0000 (GMT) Received: from mailhost.stack.nl (vaak.stack.nl [131.155.140.140]) by mx1.FreeBSD.org (Postfix) with ESMTP id CAA4B43D46 for ; Tue, 15 Mar 2005 15:47:21 +0000 (GMT) (envelope-from marcolz@stack.nl) Received: from hammer.stack.nl (hammer.stack.nl [IPv6:2001:610:1108:5010::153]) by mailhost.stack.nl (Postfix) with ESMTP id C2CC21F05A; Tue, 15 Mar 2005 16:47:20 +0100 (CET) Received: by hammer.stack.nl (Postfix, from userid 333) id A8F726247; Tue, 15 Mar 2005 16:47:20 +0100 (CET) Date: Tue, 15 Mar 2005 16:47:20 +0100 From: Marc Olzheim To: Dennis Berger Message-ID: <20050315154720.GA69524@stack.nl> References: <20050315153209.GA8863@nipsi.home.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="LQksG6bCIzRHxTLp" Content-Disposition: inline In-Reply-To: <20050315153209.GA8863@nipsi.home.net> X-Operating-System: FreeBSD hammer.stack.nl 5.4-PRERELEASE FreeBSD 5.4-PRERELEASE X-URL: http://www.stack.nl/~marcolz/ User-Agent: Mutt/1.5.8i cc: freebsd-performance@freebsd.org Subject: Re: vfs.ufs.dirhash_maxmem is a bit low X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Mar 2005 15:47:22 -0000 --LQksG6bCIzRHxTLp Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Mar 15, 2005 at 04:32:09PM +0100, Dennis Berger wrote: > My request can we add white kernel message for the behavier of running ou= t of hashmemory? Like it is for running out of open files. > A message will appear on /dev/console indicating that the dirhash is too = low. > Or pointing out in tuning(9) that it should raised of servers with lots o= f files. > Or raised the default from 2MB to 10MB or something. >=20 > any comments? I'd say stuff it in tuning(7)... Marc --LQksG6bCIzRHxTLp Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (FreeBSD) iD8DBQFCNwOIezjnobFOgrERArgPAKDUex1ZV3Hs6xO/58z5F5k4gqV96QCfcODq nQ5GiyDX/sSNh/ccswrov8I= =8OAh -----END PGP SIGNATURE----- --LQksG6bCIzRHxTLp-- From owner-freebsd-performance@FreeBSD.ORG Tue Mar 15 15:52:46 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6D90E16A4CE for ; Tue, 15 Mar 2005 15:52:46 +0000 (GMT) Received: from web41211.mail.yahoo.com (web41211.mail.yahoo.com [66.218.93.44]) by mx1.FreeBSD.org (Postfix) with SMTP id 2123143D54 for ; Tue, 15 Mar 2005 15:52:46 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 17640 invoked by uid 60001); 15 Mar 2005 15:52:40 -0000 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=LSCcAfE31JyTHklcZQ3RFT6n/chsnQ1Hx6tI/6WlTxOoNiuLaCSveUN+xlhKDjP/vlCX/BaQ3meUVJhYD2Faft5ooNXdYt/wSAVwouhW1g1rWcQSWaConQAXEibZHspFBqmGgvyPnOs9UyNQspRZnsJai5djAW3aY46Iu1B72Fg= ; Message-ID: <20050315155240.17638.qmail@web41211.mail.yahoo.com> Received: from [83.129.170.146] by web41211.mail.yahoo.com via HTTP; Tue, 15 Mar 2005 07:52:40 PST Date: Tue, 15 Mar 2005 07:52:40 -0800 (PST) From: Arne "Wörner" To: Dennis Berger , freebsd-performance@freebsd.org In-Reply-To: <20050315153209.GA8863@nipsi.home.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: Re: vfs.ufs.dirhash_maxmem is a bit low X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Mar 2005 15:52:46 -0000 --- Dennis Berger wrote: > [...] > I raised vfs.ufs.dirhash_maxmem to 30 mb. after that everything > was fine again. > [...] > any comments? > I would say: It should increase and decrease the value by need and document this via console/syslog... Skoal Arne __________________________________ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/ From owner-freebsd-performance@FreeBSD.ORG Tue Mar 15 15:55:35 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E2E2A16A4CE for ; Tue, 15 Mar 2005 15:55:35 +0000 (GMT) Received: from sollube.sarenet.es (sollube.sarenet.es [192.148.167.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2CEA443D2F for ; Tue, 15 Mar 2005 15:55:35 +0000 (GMT) (envelope-from BORJAMAR@SARENET.ES) Received: from [127.0.0.1] (matahari.sarenet.es [192.148.167.18]) by sollube.sarenet.es (Postfix) with ESMTP id DB10213A5; Tue, 15 Mar 2005 16:55:33 +0100 (CET) In-Reply-To: <20050315154720.GA69524@stack.nl> References: <20050315153209.GA8863@nipsi.home.net> <20050315154720.GA69524@stack.nl> Mime-Version: 1.0 (Apple Message framework v619.2) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <1b2fb21a30bde2f07bb8a29f8e389392@SARENET.ES> Content-Transfer-Encoding: 7bit From: Borja Marcos Date: Tue, 15 Mar 2005 16:55:32 +0100 To: Marc Olzheim X-Mailer: Apple Mail (2.619.2) cc: freebsd-performance@freebsd.org Subject: Re: vfs.ufs.dirhash_maxmem is a bit low X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Mar 2005 15:55:36 -0000 On 15 Mar 2005, at 16:47, Marc Olzheim wrote: > On Tue, Mar 15, 2005 at 04:32:09PM +0100, Dennis Berger wrote: >> My request can we add white kernel message for the behavier of >> running out of hashmemory? I'd say stuff it in tuning(7)... My rule of thumb is: In case you have a machine with lots of huge directories (which is _not_ the same as lots of files), in case you see that vfs.ufs.dirhash_mem is very similar to vfs.ufs.dirhash_maxmem, increase vfs.ufs.dirhash_maxmem until you cannot fill it. It's a radical approach, but it gave me very good results in an imap server with lots of maildir folders. Of course, be sure not to run out of memory. Run vmstat and check the "sr" column. Borja. From owner-freebsd-performance@FreeBSD.ORG Tue Mar 15 20:07:40 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 399BA16A4CE for ; Tue, 15 Mar 2005 20:07:40 +0000 (GMT) Received: from coe.ufrj.br (roma.coe.ufrj.br [146.164.53.65]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6E9ED43D2D for ; Tue, 15 Mar 2005 20:07:39 +0000 (GMT) (envelope-from jonny@jonny.eng.br) Received: from localhost (localhost [127.0.0.1]) by coe.ufrj.br (Postfix) with ESMTP id BCB8017017; Tue, 15 Mar 2005 17:07:35 -0300 (BRT) Received: from coe.ufrj.br ([146.164.53.65]) by localhost (roma.coe.ufrj.br [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 23956-10; Tue, 15 Mar 2005 17:07:29 -0300 (BRT) Received: from [10.0.8.17] (nat.int.gov.br [200.20.196.226]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by coe.ufrj.br (Postfix) with ESMTP id 2458617016; Tue, 15 Mar 2005 17:07:29 -0300 (BRT) Message-ID: <42374080.9040609@jonny.eng.br> Date: Tue, 15 Mar 2005 17:07:28 -0300 From: =?ISO-8859-1?Q?Jo=E3o_Carlos_Mendes_Lu=EDs?= User-Agent: Mozilla Thunderbird 1.0 (Windows/20041206) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Dennis Berger References: <20050315153209.GA8863@nipsi.home.net> In-Reply-To: <20050315153209.GA8863@nipsi.home.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at coe.ufrj.br cc: freebsd-performance@freebsd.org Subject: Re: vfs.ufs.dirhash_maxmem is a bit low X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Mar 2005 20:07:40 -0000 Dennis Berger wrote: > My request can we add white kernel message for the behavier of running out of hashmemory? Like it is for running out of open files. > A message will appear on /dev/console indicating that the dirhash is too low. I like this. > Or pointing out in tuning(9) that it should raised of servers with lots of files. And this. > Or raised the default from 2MB to 10MB or something. But not this! Remember that we still have some servers with low memory capacity. Isn't it possible to make this hash somewhat auto-resizeable? From owner-freebsd-performance@FreeBSD.ORG Fri Mar 18 18:26:38 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2070616A4CE for ; Fri, 18 Mar 2005 18:26:38 +0000 (GMT) Received: from dumballah.tvnet.hu (dumballah.tvnet.hu [195.38.96.23]) by mx1.FreeBSD.org (Postfix) with ESMTP id 57E1143D46 for ; Fri, 18 Mar 2005 18:26:37 +0000 (GMT) (envelope-from banhalmi@field.hu) Received: from localhost (localhost.localdomain [127.0.0.1]) by dumballah.tvnet.hu (Postfix) with ESMTP id 6DA141017E5 for ; Fri, 18 Mar 2005 19:26:36 +0100 (CET) Received: from dumballah.tvnet.hu ([127.0.0.1]) by localhost (dumballah.tvnet.hu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 28279-40 for ; Fri, 18 Mar 2005 19:26:36 +0100 (CET) Received: from oxy (adsl-1-90.pool.tvnet.hu [195.38.103.90]) by dumballah.tvnet.hu (Postfix) with SMTP id 058421017D7 for ; Fri, 18 Mar 2005 19:26:36 +0100 (CET) Message-ID: <000b01c52be8$076c43f0$0201a8c0@oxy> From: =?iso-8859-2?Q?B=E1nhalmi_Csaba?= To: Date: Fri, 18 Mar 2005 19:26:40 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-2"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2527 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2527 X-Virus-Scanned: by amavisd-new at tvnet.hu Subject: mbuf underrun, zombie process X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Mar 2005 18:26:38 -0000 hi! i hope i'm writin' to the right list this time.. my problem is the following: my server has high traffic on 100mbit dedicated line (around 10-11,5mb/s all day) with approx ~100-120 tcp connection. the nic is an intel pro 100, used with polling. now..when i set my data proxy's tcp window size to optimal, the network traffic dies in 2-3hours and the process become zombie, can't even kill. when i use small or large tcp window size the data proxy is stable, however the traffic is not good, dsl users complain, high speed copy is slow, etc. i don't really know what to do, what sysctl should increase or anything, if anyone had same problem, please post.. my machine is a pc, amd 2000+, around 50-60% used cpu, 512 ram and this is the only high load process. i changed couple sysctl settings: kern.ipc.nmbclusters=32768 kern.polling.enable=1 kern.ipc.somaxconn=4096 net.inet.udp.maxdgram=32768 net.inet.tcp.inflight.enable=0 net.inet.ip.portrange.randomized=0 ps: i tried to increase nmbclusters (65536, 131072), increased somaxconn (8192) and tcp.recvspace/sendspace to 65536, but didn't helped. From owner-freebsd-performance@FreeBSD.ORG Fri Mar 18 18:29:55 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 028F916A4CE for ; Fri, 18 Mar 2005 18:29:55 +0000 (GMT) Received: from meisai.numachi.com (meisai.numachi.com [198.175.254.6]) by mx1.FreeBSD.org (Postfix) with SMTP id 3256743D5E for ; Fri, 18 Mar 2005 18:29:54 +0000 (GMT) (envelope-from reichert@numachi.com) Received: (qmail 49141 invoked from network); 18 Mar 2005 18:29:53 -0000 Received: from natto.numachi.com (198.175.254.216) by meisai.numachi.com with SMTP; 18 Mar 2005 18:29:53 -0000 Received: (qmail 64010 invoked by uid 1001); 18 Mar 2005 18:29:53 -0000 Date: Fri, 18 Mar 2005 13:29:53 -0500 From: Brian Reichert To: B?nhalmi Csaba Message-ID: <20050318182953.GD50093@numachi.com> References: <000b01c52be8$076c43f0$0201a8c0@oxy> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <000b01c52be8$076c43f0$0201a8c0@oxy> User-Agent: Mutt/1.5.8i cc: freebsd-performance@freebsd.org Subject: Re: mbuf underrun, zombie process X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Mar 2005 18:29:55 -0000 On Fri, Mar 18, 2005 at 07:26:40PM +0100, B?nhalmi Csaba wrote: > my problem is the following: my server has high traffic on 100mbit dedicated > line (around 10-11,5mb/s all day) > with approx ~100-120 tcp connection. the nic is an intel pro 100, used with > polling. > now..when i set my data proxy's tcp window size to optimal, the network > traffic dies in 2-3hours and > the process become zombie, can't even kill. What sort of data proxy is this? -- Brian Reichert 55 Crystal Ave. #286 Daytime number: (603) 434-6842 Derry NH 03038-1725 USA BSD admin/developer at large From owner-freebsd-performance@FreeBSD.ORG Fri Mar 18 17:06:37 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CB16E16A4CE for ; Fri, 18 Mar 2005 17:06:37 +0000 (GMT) Received: from dumballah.tvnet.hu (dumballah.tvnet.hu [195.38.96.23]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1B81043D48 for ; Fri, 18 Mar 2005 17:06:37 +0000 (GMT) (envelope-from oxygen@field.hu) Received: from localhost (localhost.localdomain [127.0.0.1]) by dumballah.tvnet.hu (Postfix) with ESMTP id 6F39610178B for ; Fri, 18 Mar 2005 18:06:35 +0100 (CET) Received: from dumballah.tvnet.hu ([127.0.0.1]) by localhost (dumballah.tvnet.hu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 27018-12 for ; Fri, 18 Mar 2005 18:06:35 +0100 (CET) Received: from oxy (adsl-1-90.pool.tvnet.hu [195.38.103.90]) by dumballah.tvnet.hu (Postfix) with SMTP id EEE12101773 for ; Fri, 18 Mar 2005 18:06:34 +0100 (CET) Message-ID: <000501c52bdc$d9970600$0201a8c0@oxy> From: "Oxygen" To: Date: Fri, 18 Mar 2005 18:06:39 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-2"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2527 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2527 X-Virus-Scanned: by amavisd-new at tvnet.hu X-Mailman-Approved-At: Sat, 19 Mar 2005 06:29:51 +0000 Subject: mbuf underrun, zombie process X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Mar 2005 17:06:37 -0000 hi! i hope i'm writin' to the right list this time.. my problem is the following: my server has high traffic on 100mbit dedicated line (around 10-11,5mb/s all day) with approx ~100-120 tcp connection. the nic is an intel pro 100, used with polling. now..when i set my data proxy's tcp window size to optimal, the network traffic dies in 2-3hours and the process become zombie, can't even kill. when i use small or large tcp window size the data proxy is stable, however the traffic is not good, dsl users complain, high speed copy is slow, etc. i don't really know what to do, what sysctl should increase or anything, if anyone had same problem, please post.. my machine is a pc, amd 2000+, around 50-60% used cpu, 512 ram and this is the only high load process. i changed couple sysctl settings: kern.ipc.nmbclusters=32768 kern.polling.enable=1 kern.ipc.somaxconn=4096 net.inet.udp.maxdgram=32768 net.inet.tcp.inflight.enable=0 net.inet.ip.portrange.randomized=0 ps: i tried to increase nmbclusters (65536, 131072), increased somaxconn (8192) and tcp.recvspace/sendspace to 65536, but didn't helped. From owner-freebsd-performance@FreeBSD.ORG Sat Mar 19 14:14:21 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 048D716A4CE for ; Sat, 19 Mar 2005 14:14:21 +0000 (GMT) Received: from cyrus.watson.org (cyrus.watson.org [204.156.12.53]) by mx1.FreeBSD.org (Postfix) with ESMTP id CF52E43D48 for ; Sat, 19 Mar 2005 14:14:20 +0000 (GMT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by cyrus.watson.org (Postfix) with SMTP id D968B46B1A; Sat, 19 Mar 2005 09:14:19 -0500 (EST) Date: Sat, 19 Mar 2005 14:11:35 +0000 (GMT) From: Robert Watson X-Sender: robert@fledge.watson.org To: =?iso-8859-2?Q?B=E1nhalmi_Csaba?= In-Reply-To: <000b01c52be8$076c43f0$0201a8c0@oxy> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=ISO-8859-1 Content-Transfer-Encoding: QUOTED-PRINTABLE cc: freebsd-performance@freebsd.org Subject: Re: mbuf underrun, zombie process X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Mar 2005 14:14:21 -0000 On Fri, 18 Mar 2005, [iso-8859-2] B=E1nhalmi Csaba wrote: > the process become zombie, can't even kill. Could you show the output of "ps axl" that includes any processes associated with the proxy? By "zombie", do you mean it has the zombie flag set, or that it is wedged in some other way? Note that in UNIX semantics, 'zombie' specifically means a process that has exited, but whose parent process has not picked up the exit results, and it sounds like maybe you mean it has wedged but not exited?=20 Thanks, Robert N M Watson