From owner-freebsd-performance@FreeBSD.ORG Tue Sep 28 01:35:07 2010 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CA031106566B for ; Tue, 28 Sep 2010 01:35:07 +0000 (UTC) (envelope-from ssanders@softhammer.net) Received: from oproxy3-pub.bluehost.com (oproxy3-pub.bluehost.com [69.89.21.8]) by mx1.freebsd.org (Postfix) with SMTP id 9B04F8FC17 for ; Tue, 28 Sep 2010 01:35:07 +0000 (UTC) Received: (qmail 28703 invoked by uid 0); 28 Sep 2010 01:08:27 -0000 Received: from unknown (HELO host358.hostmonster.com) (66.147.240.158) by oproxy3.bluehost.com with SMTP; 28 Sep 2010 01:08:27 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=default; d=softhammer.net; h=Received:Message-ID:Date:From:User-Agent:MIME-Version:To:Subject:Content-Type:Content-Transfer-Encoding:X-Identified-User; b=X8corEn85EbWw22fZG5mQrYDAK+1tL8q6s1mQSctKk3C9j0PBRprkDSChIMzILA3qXGqXMINGBq3Xc9aCIz5C+jr8uJF4ARUC/KU7sxAfIIVhau49K0NwJDiRduo5hvk; Received: from pool-74-96-233-244.washdc.fios.verizon.net ([74.96.233.244] helo=[192.168.1.54]) by host358.hostmonster.com with esmtpa (Exim 4.69) (envelope-from ) id 1P0OgE-000161-Rd for freebsd-performance@freebsd.org; Mon, 27 Sep 2010 19:08:27 -0600 Message-ID: <4CA14009.4050906@softhammer.net> Date: Mon, 27 Sep 2010 21:08:25 -0400 From: Stephen Sanders User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.9) Gecko/20100921 Fedora/3.1.4-1.fc13 Thunderbird/3.1.4 MIME-Version: 1.0 To: freebsd-performance@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Identified-User: {2492:host358.hostmonster.com:softhamm:softhammer.net} {sentby:smtp auth 74.96.233.244 authed with ssanders@softhammer.net} Subject: gstripe small transaction size X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Sep 2010 01:35:07 -0000 I'm trying a disk throughput experiment where in two 3ware raid 6's are being put into a g_strip raid 0. The raid 6's are using 8 7200RPM disks. The disk transfer rate is ~80MB/s. Using a load generation tool that is using O_DIRECT for I/O, I've generated the following short output from iostat. Needless to say, the write performance is a lot less than I'm expecting. We've modified the kernel such that our KB/t figure is closer to 512KB/t per disk when measured without the g_strip. With g_strip turned on, the KB/t number is more like 60KB/t. The question is how do I get g_stripe to write larger transactions to the disk ? Thanks. gstripe label -v -s 131072 roadkill /dev/da0 /dev/da2 newfs /dev/stripe/roadkill mount /dev/stripe/roadkill /u8 sysctl kern.geom.stripe.fast=1 iostat -n3 -t da 5 tty da0 da1 da2 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 200 39.52 13 0.49 11.97 1 0.01 41.23 12 0.49 0 0 0 0 100 0 47 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100 1 19 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100 20 47 0.00 0 0.00 41.43 3 0.11 0.00 0 0.00 0 0 0 0 100 0 18 56.60 1817 100.41 20.00 1 0.02 57.22 1797 100.44 0 0 6 1 93 0 16 58.87 1978 113.73 0.00 0 0.00 58.27 2000 113.81 0 0 6 1 92 0 16 60.38 1906 112.37 16.00 0 0.00 59.95 1918 112.31 0 0 7 1 92 0 16 59.36 1863 108.01 0.00 0 0.00 60.00 1844 108.07 0 0 6 1 93 0 16 59.88 1933 113.05 0.00 0 0.00 60.40 1915 112.98 0 0 7 1 92 From owner-freebsd-performance@FreeBSD.ORG Tue Sep 28 10:44:48 2010 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 13A79106566B for ; Tue, 28 Sep 2010 10:44:48 +0000 (UTC) (envelope-from gofp-freebsd-performance@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id BF2C88FC1A for ; Tue, 28 Sep 2010 10:44:47 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1P0Xfw-0003wJ-64 for freebsd-performance@freebsd.org; Tue, 28 Sep 2010 12:44:44 +0200 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 28 Sep 2010 12:44:44 +0200 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 28 Sep 2010 12:44:44 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-performance@freebsd.org From: Ivan Voras Date: Tue, 28 Sep 2010 12:44:44 +0200 Lines: 26 Message-ID: References: <4CA14009.4050906@softhammer.net> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.9) Gecko/20100518 Thunderbird/3.0.4 In-Reply-To: <4CA14009.4050906@softhammer.net> X-Enigmail-Version: 1.0.1 Subject: Re: gstripe small transaction size X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Sep 2010 10:44:48 -0000 On 09/28/10 03:08, Stephen Sanders wrote: > I'm trying a disk throughput experiment where in two 3ware raid 6's are > being put into a g_strip raid 0. > > The raid 6's are using 8 7200RPM disks. The disk transfer rate is > ~80MB/s. Using a load generation tool that is using O_DIRECT for I/O, > I've generated the following short output from iostat. Needless to say, > the write performance is a lot less than I'm expecting. > > We've modified the kernel such that our KB/t figure is closer to 512KB/t > per disk when measured without the g_strip. With g_strip turned on, the > KB/t number is more like 60KB/t. > > The question is how do I get g_stripe to write larger transactions to > the disk ? > > gstripe label -v -s 131072 roadkill /dev/da0 /dev/da2 No pratical way. You have a gstripe array of two drives and with MAXPHYS of 128 KiB each would in fact see at most 64 KiB transactions. It gets worse when you add drives. > newfs /dev/stripe/roadkill Depending on what you are doing, adding "-U" would help you here. From owner-freebsd-performance@FreeBSD.ORG Tue Sep 28 05:45:41 2010 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DE024106566B for ; Tue, 28 Sep 2010 05:45:41 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 63EA78FC12 for ; Tue, 28 Sep 2010 05:45:40 +0000 (UTC) Received: by bwz15 with SMTP id 15so5154662bwz.13 for ; Mon, 27 Sep 2010 22:45:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=dyE1f7B50CDTv9ub6SUGOfu1q5jew1FtJ6Es0eoy67g=; b=PETMjQMLE5aVBckOaB58DmhiW4yQ80jBdjE6ZNfPuH2p9aA3zRhI6j4GHrpPX07hQk hHgDy+vvxF3/SacXcQItjUyD4aa3aSuu2+YJ0bKGIFA84nAO0e+79rmzG0ZRlqqG7UUp ZjjcuENIEsUz73UY+cxMoNW2vb2X5jkVTX6L4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=rmRx/uwraKZIeKEA6G7dfx9McLV22aGLTUrpYhZo76Ec+XZeNmJ10qVHaw6iylhlGe 1KXlpFDh7rarEecHrIZ6n7oPLHWon0QvMlJuYMW7AuY4m/Vv6oGAfykbjLYXPvsIX8h7 hk/5baol4FNGDgf7PacGQFh95mgqeDFgR6kFU= Received: by 10.204.69.18 with SMTP id x18mr6284717bki.34.1285652740192; Mon, 27 Sep 2010 22:45:40 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (pc.mavhome.dp.ua [212.86.226.226]) by mx.google.com with ESMTPS id f18sm5193495bkf.15.2010.09.27.22.45.38 (version=SSLv3 cipher=RC4-MD5); Mon, 27 Sep 2010 22:45:39 -0700 (PDT) Sender: Alexander Motin Message-ID: <4CA180FD.9050002@FreeBSD.org> Date: Tue, 28 Sep 2010 08:45:33 +0300 From: Alexander Motin User-Agent: Thunderbird 2.0.0.24 (X11/20100402) MIME-Version: 1.0 To: Stephen Sanders , freebsd-performance@freebsd.org References: In-Reply-To: X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Tue, 28 Sep 2010 11:19:26 +0000 Cc: Subject: Re: gstripe small transaction size X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Sep 2010 05:45:41 -0000 Stephen Sanders wrote: > I'm trying a disk throughput experiment where in two 3ware raid 6's are > being put into a g_strip raid 0. > > The raid 6's are using 8 7200RPM disks. The disk transfer rate is > ~80MB/s. Using a load generation tool that is using O_DIRECT for I/O, > I've generated the following short output from iostat. Needless to say, > the write performance is a lot less than I'm expecting. > > We've modified the kernel such that our KB/t figure is closer to 512KB/t > per disk when measured without the g_strip. With g_strip turned on, the > KB/t number is more like 60KB/t. > > The question is how do I get g_stripe to write larger transactions to > the disk ? How old is your system? There was bug, fixed 8-12 months ago, making fast mode in gstripe not working on systems with increased MAXPHYS. As I understand, it is what you've changed in your kernel. -- Alexander Motin From owner-freebsd-performance@FreeBSD.ORG Tue Sep 28 12:44:51 2010 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5F725106566B for ; Tue, 28 Sep 2010 12:44:51 +0000 (UTC) (envelope-from ssanders@softhammer.net) Received: from oproxy1-pub.bluehost.com (oproxy1-pub.bluehost.com [66.147.249.253]) by mx1.freebsd.org (Postfix) with SMTP id 2DB668FC08 for ; Tue, 28 Sep 2010 12:44:51 +0000 (UTC) Received: (qmail 1399 invoked by uid 0); 28 Sep 2010 11:46:05 -0000 Received: from unknown (HELO host358.hostmonster.com) (66.147.240.158) by oproxy1.bluehost.com.bluehost.com with SMTP; 28 Sep 2010 11:46:04 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=default; d=softhammer.net; h=Received:Message-ID:Date:From:User-Agent:MIME-Version:To:Subject:References:In-Reply-To:Content-Type:Content-Transfer-Encoding:X-Identified-User; b=vdP1DuT9shBPgl8VuNS6KSOVHeprAgRF35LFTVsNIAV3jgWox2iYlteR5PkBmiuGxkmtj944R2xBpQQn+vYqrNCIHyRVTAxrwBCPoOAXQp3+ZooM5LFSkXUp6ljqftRk; Received: from pool-74-96-233-244.washdc.fios.verizon.net ([74.96.233.244] helo=[192.168.1.54]) by host358.hostmonster.com with esmtpa (Exim 4.69) (envelope-from ) id 1P0ZY6-0005v6-7z for freebsd-performance@freebsd.org; Tue, 28 Sep 2010 06:44:46 -0600 Message-ID: <4CA1E33C.70200@softhammer.net> Date: Tue, 28 Sep 2010 08:44:44 -0400 From: Stephen Sanders User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.9) Gecko/20100921 Fedora/3.1.4-1.fc13 Thunderbird/3.1.4 MIME-Version: 1.0 To: freebsd-performance@freebsd.org References: <4CA14009.4050906@softhammer.net> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Identified-User: {2492:host358.hostmonster.com:softhamm:softhammer.net} {sentby:smtp auth 74.96.233.244 authed with ssanders@softhammer.net} Subject: Re: gstripe small transaction size X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Sep 2010 12:44:51 -0000 Increasing MAXPHYS and turning up the stripe size won't have the effect I'm looking for ? I see where MAXPHYS is now a kernel configuration parameter in FreeBSD 8.1. Thanks. On 09/28/2010 06:44 AM, Ivan Voras wrote: > On 09/28/10 03:08, Stephen Sanders wrote: >> I'm trying a disk throughput experiment where in two 3ware raid 6's are >> being put into a g_strip raid 0. >> >> The raid 6's are using 8 7200RPM disks. The disk transfer rate is >> ~80MB/s. Using a load generation tool that is using O_DIRECT for I/O, >> I've generated the following short output from iostat. Needless to say, >> the write performance is a lot less than I'm expecting. >> >> We've modified the kernel such that our KB/t figure is closer to 512KB/t >> per disk when measured without the g_strip. With g_strip turned on, the >> KB/t number is more like 60KB/t. >> >> The question is how do I get g_stripe to write larger transactions to >> the disk ? >> >> gstripe label -v -s 131072 roadkill /dev/da0 /dev/da2 > > No pratical way. You have a gstripe array of two drives and with > MAXPHYS of 128 KiB each would in fact see at most 64 KiB transactions. > It gets worse when you add drives. > >> newfs /dev/stripe/roadkill > > Depending on what you are doing, adding "-U" would help you here. > > > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to > "freebsd-performance-unsubscribe@freebsd.org" > From owner-freebsd-performance@FreeBSD.ORG Tue Sep 28 12:45:10 2010 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5F68A10656C2 for ; Tue, 28 Sep 2010 12:45:10 +0000 (UTC) (envelope-from ssanders@softhammer.net) Received: from oproxy3-pub.bluehost.com (oproxy3-pub.bluehost.com [69.89.21.8]) by mx1.freebsd.org (Postfix) with SMTP id 2DA9F8FC21 for ; Tue, 28 Sep 2010 12:45:09 +0000 (UTC) Received: (qmail 13559 invoked by uid 0); 28 Sep 2010 12:45:09 -0000 Received: from unknown (HELO host358.hostmonster.com) (66.147.240.158) by oproxy3.bluehost.com with SMTP; 28 Sep 2010 12:45:09 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=default; d=softhammer.net; h=Received:Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject:References:In-Reply-To:Content-Type:Content-Transfer-Encoding:X-Identified-User; b=uuI22gSJOvrdTSnpgMIhLF948V8q+FqUSbKldXW1ylFG3hcv3c1769I1Rs1OftA4E9bxqY10U1Bs2jxQWEO7WqwRAE17rV6eU1iSVeILVHp289ScspX2l1d+WOr6UCVi; Received: from pool-74-96-233-244.washdc.fios.verizon.net ([74.96.233.244] helo=[192.168.1.54]) by host358.hostmonster.com with esmtpa (Exim 4.69) (envelope-from ) id 1P0ZYT-0006DG-96; Tue, 28 Sep 2010 06:45:09 -0600 Message-ID: <4CA1E354.8000608@softhammer.net> Date: Tue, 28 Sep 2010 08:45:08 -0400 From: Stephen Sanders User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.9) Gecko/20100921 Fedora/3.1.4-1.fc13 Thunderbird/3.1.4 MIME-Version: 1.0 To: Alexander Motin References: <4CA180FD.9050002@FreeBSD.org> In-Reply-To: <4CA180FD.9050002@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Identified-User: {2492:host358.hostmonster.com:softhamm:softhammer.net} {sentby:smtp auth 74.96.233.244 authed with ssanders@softhammer.net} Cc: freebsd-performance@freebsd.org Subject: Re: gstripe small transaction size X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Sep 2010 12:45:10 -0000 Ooops, didn't put out the particulars of the system. This is a FreeBSD 8.0 system that has not been updated. We upped the block size of the da driver, not MAXPHYS. I'll give that a shot. Thanks. On 09/28/2010 01:45 AM, Alexander Motin wrote: > Stephen Sanders wrote: >> I'm trying a disk throughput experiment where in two 3ware raid 6's are >> being put into a g_strip raid 0. >> >> The raid 6's are using 8 7200RPM disks. The disk transfer rate is >> ~80MB/s. Using a load generation tool that is using O_DIRECT for I/O, >> I've generated the following short output from iostat. Needless to say, >> the write performance is a lot less than I'm expecting. >> >> We've modified the kernel such that our KB/t figure is closer to 512KB/t >> per disk when measured without the g_strip. With g_strip turned on, the >> KB/t number is more like 60KB/t. >> >> The question is how do I get g_stripe to write larger transactions to >> the disk ? > How old is your system? There was bug, fixed 8-12 months ago, making > fast mode in gstripe not working on systems with increased MAXPHYS. As I > understand, it is what you've changed in your kernel. > From owner-freebsd-performance@FreeBSD.ORG Tue Sep 28 12:53:02 2010 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5FDF9106564A for ; Tue, 28 Sep 2010 12:53:02 +0000 (UTC) (envelope-from gofp-freebsd-performance@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 18B908FC18 for ; Tue, 28 Sep 2010 12:53:01 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1P0Zg1-0004ry-Gf for freebsd-performance@freebsd.org; Tue, 28 Sep 2010 14:52:57 +0200 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 28 Sep 2010 14:52:57 +0200 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 28 Sep 2010 14:52:57 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-performance@freebsd.org From: Ivan Voras Date: Tue, 28 Sep 2010 14:52:54 +0200 Lines: 18 Message-ID: References: <4CA14009.4050906@softhammer.net> <4CA1E33C.70200@softhammer.net> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.9) Gecko/20100518 Thunderbird/3.0.4 In-Reply-To: <4CA1E33C.70200@softhammer.net> X-Enigmail-Version: 1.0.1 Subject: Re: gstripe small transaction size X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Sep 2010 12:53:02 -0000 On 09/28/10 14:44, Stephen Sanders wrote: > Increasing MAXPHYS and turning up the stripe size won't have the effect > I'm looking for ? I've missed you tuned MAXPHYS up. Yes, in this case it should work, if the underlying driver supports larger IO sizes. As I see it, if all of these are true: * MAXPHYS is upped to be at least stripe_size * number_of_drives (and possibly 2x that, to use "fast mode" of gstripe) * vfs.hirunningspace is > MAXPHYS * the driver supports IOs of at least your stripe_size * the file system doesn't introduce unnecessary choppiness in IO (i.e. use soft-updates) it is reasonable to expect streaming IO to your array will use larger transactions. From owner-freebsd-performance@FreeBSD.ORG Wed Sep 29 21:09:47 2010 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 05E111065693 for ; Wed, 29 Sep 2010 21:09:47 +0000 (UTC) (envelope-from faust64@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id A45CE8FC16 for ; Wed, 29 Sep 2010 21:09:46 +0000 (UTC) Received: by qyk7 with SMTP id 7so1863142qyk.13 for ; Wed, 29 Sep 2010 14:09:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:from:date :message-id:subject:to:content-type; bh=wfbDSeg3hVLuyl42WOnwpxRUx9c+wvP1s7PlQJbTUKM=; b=fEvFVx2B4aaiiToRq9htgJzkosFfXXDfFAFkyvjn5Notu5tUPomhQZI6U4g3i6M+hX 4JaaaEwRY74n+PK1Op+Gi1YK3QxSpevfq4Qda8dTKzfrNPEKmQPA38nc5aQICU2DigQe ogaDsO2NUmHSb5ZXFterEg203S+Nk/DSFhUu8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:from:date:message-id:subject:to:content-type; b=uV4gNtqK24wrrx6zAQGzU78UYcdOIqVnbm6LChOcSMXxd6Ym0wtzdAiKRkngAY+M1R wcYu2asnjKaNQkgkCrx9uvpX7gEjjwGLbykX4oGCy4uJMFAD93OTpZnQ3Q7Vwsp12X7i UL+3pk8F8sekY3OuvvQfpmg21FBg0P0By7wts= Received: by 10.229.52.38 with SMTP id f38mr1363055qcg.224.1285793137543; Wed, 29 Sep 2010 13:45:37 -0700 (PDT) MIME-Version: 1.0 Received: by 10.229.10.103 with HTTP; Wed, 29 Sep 2010 13:44:53 -0700 (PDT) From: =?ISO-8859-1?Q?Samuel_Mart=EDn_Moro?= Date: Wed, 29 Sep 2010 22:44:53 +0200 Message-ID: To: freebsd-performance@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: freebsd router X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Sep 2010 21:09:47 -0000 Hi, I'm trying to replace my (dying) gateway with a qnap ts-509 (1G DDR, celero= n m420 1.6Ghzs). I'm using mfsBSD, based on FreeBSD-RELEASE-8.1 amd64. It's almost ready (zfs, nfs, dns, pf, ...), I'm checking everything's OK to swap the gate. I noticed that opening a new connection to distant or local computer is (very) slow. After that, everything works perfectly fine. So I had a look at sysctl, and tried to fix that problem. Now, when I start a ping on a client from my network (disabled on the gate)= , I have something like that: faust@alpha ~ : time ping -c 4 google.com PING google.com (66.249.92.104) 56(84) bytes of data. 64 bytes from par03s01-in-f104.1e100.net (66.249.92.104): icmp_seq=3D1 ttl= =3D53 time=3D7.12 ms 64 bytes from par03s01-in-f104.1e100.net (66.249.92.104): icmp_seq=3D2 ttl= =3D53 time=3D7.32 ms 64 bytes from par03s01-in-f104.1e100.net (66.249.92.104): icmp_seq=3D3 ttl= =3D53 time=3D7.18 ms 64 bytes from par03s01-in-f104.1e100.net (66.249.92.104): icmp_seq=3D4 ttl= =3D53 time=3D7.18 ms --- google.com ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 15034ms rtt min/avg/max/mdev =3D 7.126/7.205/7.329/0.128 ms 0.000u 0.000s 0:25.08 0.0% 0+0k 0+0io 0pf+0w So, it takes 5 seconds to display the first line (connect), and then 5 second per ping. 25 seconds, for 4 pings... Obviously, my tries doesn't make it work any better... I found some infos here , here , there and there But I'm still not sure about the tuning implications for most of those vars= . Here is my sysctl.conf: kern.coredump=3D0 kern.ipc.somaxconn=3D4096 net.inet.ip.check_interface=3D1 net.inet.ip.fastforwarding=3D1 net.inet.ip.forwarding=3D1 net.inet.ip.portrange.first=3D1024 net.inet.ip.portrange.last=3D65535 net.inet.ip.rtexpire=3D2 net.inet.ip.rtmaxcache=3D256 net.inet.ip.rtminexpire=3D2 net.inet.ip.ttl=3D42 net.inet.udp.blackhole=3D1 net.inet.tcp.blackhole=3D2 net.inet.tcp.delacktime=3D42 net.inet.tcp.delayed_ack=3D0 net.inet.tcp.drop_synfin=3D1 net.inet.tcp.fast_finwait2_recycle=3D1 net.inet.tcp.icmp_may_rst=3D0 net.inet.icmp.icmplim=3D42 net.inet.tcp.ecn.enable=3D1 net.inet.tcp.msl=3D5000 net.inet.tcp.rfc1323=3D0 net.inet.tcp.inflight.enable=3D1 net.inet.tcp.inflight.max=3D1073725440 net.inet.tcp.inflight.stab=3D20 net.inet.tcp.inflight.min=3D1024 net.inet.tcp.recvspace=3D82320 net.inet.tcp.sendspace=3D82320 net.inet.udp.maxdgram=3D82320 net.inet.udp.recvspace=3D82320 net.inet.raw.maxdgram=3D82320 net.inet.raw.recvspace=3D82320 net.local.dgram.maxdgram=3D82320 net.local.dgram.recvspace=3D82320 net.local.stream.sendspace=3D82320 net.local.stream.recvspace=3D82320 net.inet.tcp.local_slowstart_flightsize=3D10 net.inet.tcp.nolocaltimewait=3D1 vfs.read_max=3D32 So, I was wondering, is something wrong in there? Or should I keep looking somewhere else? Where? Thanks for your help, --=20 Samuel Mart=EDn Moro {EPITECH.} tek5 From owner-freebsd-performance@FreeBSD.ORG Wed Sep 29 21:47:02 2010 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4E25B106566B for ; Wed, 29 Sep 2010 21:47:02 +0000 (UTC) (envelope-from mksmith@adhost.com) Received: from mail-defer01.adhost.com (mail-defer01.adhost.com [216.211.128.176]) by mx1.freebsd.org (Postfix) with ESMTP id 2B1B78FC15 for ; Wed, 29 Sep 2010 21:47:01 +0000 (UTC) Received: from mail-in04.adhost.com (mail-in04.adhost.com [10.212.3.14]) by mail-defer01.adhost.com (Postfix) with ESMTP id 8B06A67EC7E for ; Wed, 29 Sep 2010 14:31:18 -0700 (PDT) (envelope-from mksmith@adhost.com) Received: from ad-exh01.adhost.lan (exchange.adhost.com [216.211.143.69]) by mail-in04.adhost.com (Postfix) with ESMTP id 4C6ED614FD4; Wed, 29 Sep 2010 14:31:17 -0700 (PDT) (envelope-from mksmith@adhost.com) X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Date: Wed, 29 Sep 2010 14:31:16 -0700 Message-ID: <17838240D9A5544AAA5FF95F8D52031608F04693@ad-exh01.adhost.lan> In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: freebsd router Thread-Index: ActgGskg0hfLzrhORx+OJLLGOA++/gAAQExg References: From: "Michael K. Smith - Adhost" To: =?iso-8859-1?Q?Samuel_Mart=EDn_Moro?= , Cc: Subject: RE: freebsd router X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Sep 2010 21:47:02 -0000 Here are my settings for a box doing about 100 Mb/sec. I just included = the values that are different than yours. kern.ipc.somaxconn: 32768 net.inet.ip.check_interface: 0 net.inet.ip.fastforwarding: 0 net.inet.ip.portrange.first: 49152 net.inet.ip.rtexpire: 3600 net.inet.ip.rtmaxcache: 128 net.inet.ip.rtminexpire: 10 net.inet.ip.ttl: 64 net.inet.tcp.delacktime: 100 net.inet.tcp.drop_synfin: 0 net.inet.tcp.fast_finwait2_recycle: 0 net.inet.tcp.icmp_may_rst: 1 net.inet.icmp.icmplim: 2000 net.inet.tcp.msl: 30000 net.inet.tcp.rfc1323: 1 net.inet.tcp.inflight.enable: 0 net.inet.tcp.recvspace: 65536 net.inet.tcp.sendspace: 65536 net.inet.udp.maxdgram: 57344 net.inet.udp.recvspace: 65536 net.inet.raw.maxdgram: 9216 net.inet.raw.recvspace: 9216 net.local.dgram.maxdgram: 2048 net.local.dgram.recvspace: 4096 net.local.stream.sendspace: 8192 net.local.stream.recvspace: 65536 net.inet.tcp.local_slowstart_flightsize: 4 net.inet.tcp.nolocaltimewait: 0 vfs.read_max: 8 In addition, we set: net.inet.tcp.mssdflt=3D1460 kern.ipc.maxsockbuf=3D16777216 kern.ipc.nmbclusters=3D32768 kern.maxfiles=3D65536 kern.maxfilesperproc=3D32768 kern.maxvnodes=3D600000 net.inet.tcp.path_mtu_discovery=3D0 net.inet.tcp.recvbuf_auto=3D1 net.inet.tcp.recvbuf_inc=3D16384 net.inet.tcp.recvbuf_max=3D16777216 net.inet.tcp.sendbuf_auto=3D1 net.inet.tcp.sendbuf_inc=3D8192 net.inet.tcp.sendbuf_max=3D16777216 Regards, Mike -- Michael K. Smith - CISSP, GSEC, GISP Chief Technical Officer - Adhost Internet LLC mksmith@adhost.com w: +1 (206) 404-9500 f: +1 (206) 404-9050 PGP: B49A DDF5 8611 27F3 08B9 84BB E61E 38C0 (Key ID: 0x9A96777D) > -----Original Message----- > From: owner-freebsd-performance@freebsd.org [mailto:owner-freebsd- > performance@freebsd.org] On Behalf Of Samuel Mart=EDn Moro > Sent: Wednesday, September 29, 2010 1:45 PM > To: freebsd-performance@freebsd.org > Subject: freebsd router >=20 > Hi, >=20 >=20 > I'm trying to replace my (dying) gateway with a qnap ts-509 (1G DDR, = celeron > m420 1.6Ghzs). > I'm using mfsBSD, based on FreeBSD-RELEASE-8.1 amd64. > It's almost ready (zfs, nfs, dns, pf, ...), I'm checking everything's = OK to > swap the gate. >=20 >=20 > I noticed that opening a new connection to distant or local computer = is > (very) slow. > After that, everything works perfectly fine. > So I had a look at sysctl, and tried to fix that problem. >=20 > Now, when I start a ping on a client from my network (disabled on the = gate), > I have something like that: > faust@alpha ~ : time ping -c 4 google.com > PING google.com (66.249.92.104) 56(84) bytes of data. > 64 bytes from par03s01-in-f104.1e100.net (66.249.92.104): icmp_seq=3D1 = ttl=3D53 > time=3D7.12 ms > 64 bytes from par03s01-in-f104.1e100.net (66.249.92.104): icmp_seq=3D2 = ttl=3D53 > time=3D7.32 ms > 64 bytes from par03s01-in-f104.1e100.net (66.249.92.104): icmp_seq=3D3 = ttl=3D53 > time=3D7.18 ms > 64 bytes from par03s01-in-f104.1e100.net (66.249.92.104): icmp_seq=3D4 = ttl=3D53 > time=3D7.18 ms >=20 > --- google.com ping statistics --- > 4 packets transmitted, 4 received, 0% packet loss, time 15034ms > rtt min/avg/max/mdev =3D 7.126/7.205/7.329/0.128 ms > 0.000u 0.000s 0:25.08 0.0% 0+0k 0+0io 0pf+0w >=20 >=20 > So, it takes 5 seconds to display the first line (connect), and then 5 > second per ping. > 25 seconds, for 4 pings... > Obviously, my tries doesn't make it work any better... >=20 >=20 > I found some infos > here > , here limits.html> > , there and > there > But I'm still not sure about the tuning implications for most of those = vars. >=20 >=20 >=20 > Here is my sysctl.conf: > kern.coredump=3D0 > kern.ipc.somaxconn=3D4096 >=20 > net.inet.ip.check_interface=3D1 > net.inet.ip.fastforwarding=3D1 > net.inet.ip.forwarding=3D1 > net.inet.ip.portrange.first=3D1024 > net.inet.ip.portrange.last=3D65535 > net.inet.ip.rtexpire=3D2 > net.inet.ip.rtmaxcache=3D256 > net.inet.ip.rtminexpire=3D2 > net.inet.ip.ttl=3D42 >=20 > net.inet.udp.blackhole=3D1 > net.inet.tcp.blackhole=3D2 > net.inet.tcp.delacktime=3D42 > net.inet.tcp.delayed_ack=3D0 > net.inet.tcp.drop_synfin=3D1 > net.inet.tcp.fast_finwait2_recycle=3D1 > net.inet.tcp.icmp_may_rst=3D0 > net.inet.icmp.icmplim=3D42 > net.inet.tcp.ecn.enable=3D1 > net.inet.tcp.msl=3D5000 > net.inet.tcp.rfc1323=3D0 >=20 > net.inet.tcp.inflight.enable=3D1 > net.inet.tcp.inflight.max=3D1073725440 > net.inet.tcp.inflight.stab=3D20 > net.inet.tcp.inflight.min=3D1024 >=20 > net.inet.tcp.recvspace=3D82320 > net.inet.tcp.sendspace=3D82320 > net.inet.udp.maxdgram=3D82320 > net.inet.udp.recvspace=3D82320 > net.inet.raw.maxdgram=3D82320 > net.inet.raw.recvspace=3D82320 > net.local.dgram.maxdgram=3D82320 > net.local.dgram.recvspace=3D82320 > net.local.stream.sendspace=3D82320 > net.local.stream.recvspace=3D82320 > net.inet.tcp.local_slowstart_flightsize=3D10 > net.inet.tcp.nolocaltimewait=3D1 >=20 > vfs.read_max=3D32 >=20 >=20 >=20 >=20 > So, I was wondering, is something wrong in there? > Or should I keep looking somewhere else? > Where? >=20 >=20 >=20 > Thanks for your help, >=20 > -- > Samuel Mart=EDn Moro > {EPITECH.} tek5 > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to "freebsd-performance- > unsubscribe@freebsd.org" From owner-freebsd-performance@FreeBSD.ORG Wed Sep 29 22:30:06 2010 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 147A410656C5 for ; Wed, 29 Sep 2010 22:30:06 +0000 (UTC) (envelope-from faust64@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 024A08FC1C for ; Wed, 29 Sep 2010 22:30:04 +0000 (UTC) Received: by qwd6 with SMTP id 6so931125qwd.13 for ; Wed, 29 Sep 2010 15:30:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:in-reply-to :references:from:date:message-id:subject:to:cc:content-type; bh=FX2iR+Mz11N5Mi6jw2gdjpOqffc6gB8G56Nucg6WoJk=; b=ot1m4mUCMzEefSDsmWxhfoNbmhTSAmQ+9gYdu6Qx6GxrDAKVF4Whva/60ta2HoNL3N FBIZUnHZN3G/4WdlfjIOxTp+VHKKU6OOhtot43LkzJTYB687DigLz1HU8HFTx4JarOe9 8RhD9OKcr4kmAakUzJDbtEdE+Qi4dB2murT6c= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; b=k+qRZau4uLKecRPwEAFmntSHhEJnrpAHDTyJ3obImZwk2pbAaqFaVAPsOf9lRNu7fm AaP7hRN35LDCAb/R3Or3ezdYsKa9FvIvpm7eAPSlCsrPYCtEmMjMeeOXWr3sk8XphS+r KkeWlTqlRxkSSkR6Bm/3nGOnoydN4m+jdJIDE= Received: by 10.229.96.16 with SMTP id f16mr1696874qcn.255.1285799404178; Wed, 29 Sep 2010 15:30:04 -0700 (PDT) MIME-Version: 1.0 Received: by 10.229.10.103 with HTTP; Wed, 29 Sep 2010 15:29:23 -0700 (PDT) In-Reply-To: <17838240D9A5544AAA5FF95F8D52031608F04693@ad-exh01.adhost.lan> References: <17838240D9A5544AAA5FF95F8D52031608F04693@ad-exh01.adhost.lan> From: =?ISO-8859-1?Q?Samuel_Mart=EDn_Moro?= Date: Thu, 30 Sep 2010 00:29:23 +0200 Message-ID: To: "Michael K. Smith - Adhost" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-performance@freebsd.org Subject: Re: freebsd router X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Sep 2010 22:30:06 -0000 Thanks for your quick answer. I tried it out, but still have my slow-connect-problem... I read somewhere (can't remember where...) that my network chip doesn't support jumbo frames. So, I thought I had better to disable rfc1323, and adjust send/recv/dgram values. Am I right? Am I doing it right? About, vfs.read_max, AFAIK (a few tests, seen on google), a "correct" value is more likely between 32 and 128 I kept 32 as 'default', I'll do some tests before choosing mine... And about nmbclusters, I read that it may only be tweaked on the loader.conf, or in the kernel building configuration file. Did I missed something? Or read some outdated doc? Whatever. Thanks for your help! So, is there some other problem I could have missed? What could cause such a lag? How can the connexion "freeze" always for "exactly" five seconds before doing anything? ssh -v confirms: the "connect" string only appears after five seconds, and then everything's just fine. Regards, --=20 Samuel Mart=EDn Moro {EPITECH.} tek5 CamTrace S.A.S (+033) 1 41 38 37 60 1 All=E9e de la Venelle 92150 Suresnes FRANCE "Nobody wants to say how this works. Maybe nobody knows ..." Xorg.conf(5) From owner-freebsd-performance@FreeBSD.ORG Wed Sep 29 22:44:55 2010 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D5EFB106566C for ; Wed, 29 Sep 2010 22:44:55 +0000 (UTC) (envelope-from korvus@comcast.net) Received: from qmta04.westchester.pa.mail.comcast.net (qmta04.westchester.pa.mail.comcast.net [76.96.62.40]) by mx1.freebsd.org (Postfix) with ESMTP id 9708D8FC13 for ; Wed, 29 Sep 2010 22:44:55 +0000 (UTC) Received: from omta06.westchester.pa.mail.comcast.net ([76.96.62.51]) by qmta04.westchester.pa.mail.comcast.net with comcast id CmV41f00316LCl054mXhxu; Wed, 29 Sep 2010 22:31:41 +0000 Received: from [10.0.0.51] ([71.199.122.142]) by omta06.westchester.pa.mail.comcast.net with comcast id CmXg1f00B34Sj4f3SmXgvf; Wed, 29 Sep 2010 22:31:40 +0000 Message-ID: <4CA3BE61.7020702@comcast.net> Date: Wed, 29 Sep 2010 18:32:01 -0400 From: Steve Polyack User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.2.9) Gecko/20100915 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: =?ISO-8859-1?Q?Samuel_Mart=EDn_Moro?= References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-performance@freebsd.org Subject: Re: freebsd router X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Sep 2010 22:44:56 -0000 On 9/29/2010 4:44 PM, Samuel Martín Moro wrote: > Hi, > > > I'm trying to replace my (dying) gateway with a qnap ts-509 (1G DDR, celeron > m420 1.6Ghzs). > I'm using mfsBSD, based on FreeBSD-RELEASE-8.1 amd64. > It's almost ready (zfs, nfs, dns, pf, ...), I'm checking everything's OK to > swap the gate. > > > I noticed that opening a new connection to distant or local computer is > (very) slow. > After that, everything works perfectly fine. > So I had a look at sysctl, and tried to fix that problem. > > Now, when I start a ping on a client from my network (disabled on the gate), > I have something like that: > faust@alpha ~ : time ping -c 4 google.com > PING google.com (66.249.92.104) 56(84) bytes of data. > 64 bytes from par03s01-in-f104.1e100.net (66.249.92.104): icmp_seq=1 ttl=53 > time=7.12 ms > 64 bytes from par03s01-in-f104.1e100.net (66.249.92.104): icmp_seq=2 ttl=53 > time=7.32 ms > 64 bytes from par03s01-in-f104.1e100.net (66.249.92.104): icmp_seq=3 ttl=53 > time=7.18 ms > 64 bytes from par03s01-in-f104.1e100.net (66.249.92.104): icmp_seq=4 ttl=53 > time=7.18 ms > > --- google.com ping statistics --- > 4 packets transmitted, 4 received, 0% packet loss, time 15034ms > rtt min/avg/max/mdev = 7.126/7.205/7.329/0.128 ms > 0.000u 0.000s 0:25.08 0.0% 0+0k 0+0io 0pf+0w > > > So, it takes 5 seconds to display the first line (connect), and then 5 > second per ping. > 25 seconds, for 4 pings... > Obviously, my tries doesn't make it work any better... > Are you certain this isn't an issue with your DNS resolver(s) listed in /etc/resolv.conf? How do things change if you try 'ping -n 66.249.92.104'? Steve > I found some infos > here > , here > , there and > there > But I'm still not sure about the tuning implications for most of those vars. > > > > Here is my sysctl.conf: > kern.coredump=0 > kern.ipc.somaxconn=4096 > > net.inet.ip.check_interface=1 > net.inet.ip.fastforwarding=1 > net.inet.ip.forwarding=1 > net.inet.ip.portrange.first=1024 > net.inet.ip.portrange.last=65535 > net.inet.ip.rtexpire=2 > net.inet.ip.rtmaxcache=256 > net.inet.ip.rtminexpire=2 > net.inet.ip.ttl=42 > > net.inet.udp.blackhole=1 > net.inet.tcp.blackhole=2 > net.inet.tcp.delacktime=42 > net.inet.tcp.delayed_ack=0 > net.inet.tcp.drop_synfin=1 > net.inet.tcp.fast_finwait2_recycle=1 > net.inet.tcp.icmp_may_rst=0 > net.inet.icmp.icmplim=42 > net.inet.tcp.ecn.enable=1 > net.inet.tcp.msl=5000 > net.inet.tcp.rfc1323=0 > > net.inet.tcp.inflight.enable=1 > net.inet.tcp.inflight.max=1073725440 > net.inet.tcp.inflight.stab=20 > net.inet.tcp.inflight.min=1024 > > net.inet.tcp.recvspace=82320 > net.inet.tcp.sendspace=82320 > net.inet.udp.maxdgram=82320 > net.inet.udp.recvspace=82320 > net.inet.raw.maxdgram=82320 > net.inet.raw.recvspace=82320 > net.local.dgram.maxdgram=82320 > net.local.dgram.recvspace=82320 > net.local.stream.sendspace=82320 > net.local.stream.recvspace=82320 > net.inet.tcp.local_slowstart_flightsize=10 > net.inet.tcp.nolocaltimewait=1 > > vfs.read_max=32 > > > > > So, I was wondering, is something wrong in there? > Or should I keep looking somewhere else? > Where? > > > > Thanks for your help, > From owner-freebsd-performance@FreeBSD.ORG Wed Sep 29 22:48:40 2010 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1752D106564A for ; Wed, 29 Sep 2010 22:48:40 +0000 (UTC) (envelope-from faust64@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id C4AA68FC15 for ; Wed, 29 Sep 2010 22:48:39 +0000 (UTC) Received: by qwd6 with SMTP id 6so942121qwd.13 for ; Wed, 29 Sep 2010 15:48:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:in-reply-to :references:from:date:message-id:subject:to:cc:content-type; bh=qKj3/lGSPhkXlNEGehU4R5MYLwOCwiFSZts2BmHuP3A=; b=RfFoTlOIYyOHqZKGkwrlOFlwEvl3Irk6ZITwG7mN2lhfL2Tkg7S4NsJrYr1P/Zhz4h GGV8JEcmDcIINdHHSEl+TiivGt24dmBKvHEj/+WYC9r0CzIg1kLqHcn+S3csK6ptFS3M UqqnO4TuSaQWHIcWfFWkT3MHphHif1QGUzarg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; b=ccpyVNRvSukH62gCISHNZYvXqycDxpAOlpVCqXiDCchDvPPkMCDutJKS9NuA5v1ikW LZIGs8bB3t0Imcc5fGKRCtMq4zy5mjy5kGWwG4voxes0BGsmdzWvPqpW1VLBoP+5wHQ2 HpKG8MabBOSoV9TnfgFZJC05Jqb5g9/OS5MmM= Received: by 10.229.251.197 with SMTP id mt5mr1747047qcb.131.1285800518996; Wed, 29 Sep 2010 15:48:38 -0700 (PDT) MIME-Version: 1.0 Received: by 10.229.10.103 with HTTP; Wed, 29 Sep 2010 15:47:57 -0700 (PDT) In-Reply-To: <4CA3BE61.7020702@comcast.net> References: <4CA3BE61.7020702@comcast.net> From: =?ISO-8859-1?Q?Samuel_Mart=EDn_Moro?= Date: Thu, 30 Sep 2010 00:47:57 +0200 Message-ID: To: Steve Polyack Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-performance@freebsd.org Subject: Re: freebsd router X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Sep 2010 22:48:40 -0000 Oh... good guess... next time I'll try with a fresh VM... faust@alpha ~ : cat /etc/resolv.conf # Generated by dhcpcd from eth1 nameserver 192.168.0.254 domain faust-network nameserver 10.242.42.254 # /etc/resolv.conf.tail can replace this line faust@alpha ~ : cat /etc/resolv.conf.head nameserver 192.168.0.254 I removed the resolv.conf.head, restarted my dhcp clients. Indeed. Works better! Sorry for the disturbance... And thanks again! Cheers, --=20 Samuel Mart=EDn Moro {EPITECH.} tek5 From owner-freebsd-performance@FreeBSD.ORG Thu Sep 30 12:58:46 2010 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 85D8D106566B for ; Thu, 30 Sep 2010 12:58:46 +0000 (UTC) (envelope-from ssanders@softhammer.net) Received: from oproxy2-pub.bluehost.com (oproxy2-pub.bluehost.com [67.222.39.60]) by mx1.freebsd.org (Postfix) with SMTP id 53C078FC13 for ; Thu, 30 Sep 2010 12:58:46 +0000 (UTC) Received: (qmail 26968 invoked by uid 0); 30 Sep 2010 12:58:45 -0000 Received: from unknown (HELO host358.hostmonster.com) (66.147.240.158) by oproxy2.bluehost.com with SMTP; 30 Sep 2010 12:58:45 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=default; d=softhammer.net; h=Received:Message-ID:Date:From:User-Agent:MIME-Version:To:Subject:References:In-Reply-To:Content-Type:Content-Transfer-Encoding:X-Identified-User; b=4SvrUM7RqfjjYS8xLWxTghWAfEiXkAyl09rl9Mob2LuQ2SdikaLfuG72WvQtVg4h9eDP/+jNOQ7hORYCWKhdSJe0AWRJRAfZDhAh2hkViDm/GOm0KzGiC+rM7ipS1V/i; Received: from pool-74-96-233-244.washdc.fios.verizon.net ([74.96.233.244] helo=[192.168.1.54]) by host358.hostmonster.com with esmtpa (Exim 4.69) (envelope-from ) id 1P1Iij-0000Th-IV for freebsd-performance@freebsd.org; Thu, 30 Sep 2010 06:58:45 -0600 Message-ID: <4CA48984.5030403@softhammer.net> Date: Thu, 30 Sep 2010 08:58:44 -0400 From: Stephen Sanders User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.9) Gecko/20100921 Fedora/3.1.4-1.fc13 Thunderbird/3.1.4 MIME-Version: 1.0 To: freebsd-performance@freebsd.org References: <4CA180FD.9050002@FreeBSD.org> In-Reply-To: <4CA180FD.9050002@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Identified-User: {2492:host358.hostmonster.com:softhamm:softhammer.net} {sentby:smtp auth 74.96.233.244 authed with ssanders@softhammer.net} Subject: Re: gstripe small transaction size X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Sep 2010 12:58:46 -0000 On 09/28/2010 01:45 AM, Alexander Motin wrote: > Stephen Sanders wrote: >> I'm trying a disk throughput experiment where in two 3ware raid 6's are >> being put into a g_strip raid 0. >> >> The raid 6's are using 8 7200RPM disks. The disk transfer rate is >> ~80MB/s. Using a load generation tool that is using O_DIRECT for I/O, >> I've generated the following short output from iostat. Needless to say, >> the write performance is a lot less than I'm expecting. >> >> We've modified the kernel such that our KB/t figure is closer to 512KB/t >> per disk when measured without the g_strip. With g_strip turned on, the >> KB/t number is more like 60KB/t. >> >> The question is how do I get g_stripe to write larger transactions to >> the disk ? > How old is your system? There was bug, fixed 8-12 months ago, making > fast mode in gstripe not working on systems with increased MAXPHYS. As I > understand, it is what you've changed in your kernel. > Thanks for all of the suggestions. I merged all of the suggestions and got a good result. What worked in the end was to upgrade to FreeBSD 8.1 and : * MAXPHYS is upped to be at least stripe_size * number_of_drives (and possibly 2x that, to use "fast mode" of gstripe) * vfs.hirunningspace is > MAXPHYS * the driver supports IOs of at least your stripe_size * the file system doesn't introduce unnecessary choppiness in IO (i.e. use soft-updates) From owner-freebsd-performance@FreeBSD.ORG Thu Sep 30 18:02:46 2010 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E4FBD1065695 for ; Thu, 30 Sep 2010 18:02:46 +0000 (UTC) (envelope-from gofp-freebsd-performance@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id A1DC68FC17 for ; Thu, 30 Sep 2010 18:02:46 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1P1NSu-00060K-0B for freebsd-performance@freebsd.org; Thu, 30 Sep 2010 20:02:44 +0200 Received: from h134-215-222-243.plmomi.dedicated.static.tds.net ([134.215.222.243]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 30 Sep 2010 20:02:43 +0200 Received: from aditya by h134-215-222-243.plmomi.dedicated.static.tds.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 30 Sep 2010 20:02:43 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-performance@freebsd.org From: "R.P. Aditya" Date: Thu, 30 Sep 2010 18:02:36 +0000 (UTC) Organization: grot.org Lines: 22 Message-ID: X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: h134-215-222-243.plmomi.dedicated.static.tds.net User-Agent: slrn/0.9.8.1 (FreeBSD) Subject: monitoring per-process disk io X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Sep 2010 18:02:47 -0000 I'm trying to monitor, over the long-term, per-process disk IO (a counter of bytes read and written per pid would be ideal). I currently monitor per pid cpu and memory usage using the SNMP Host-Resources MIB, however I don't see any oids for io (disk or otherwise) per oid. The closest I've come to finding what I want -- per-process disk IO stats -- is the Linux tool dstat -- something like "screenshot 3" at: http://dag.wieers.com/home-made/dstat/ which is the output of: dstat -c --top-cpu -d --top-bio --top-latency would be good...iostat, vmstat don't give that sort of info... any suggestions or hints? Thanks, Adi From owner-freebsd-performance@FreeBSD.ORG Thu Sep 30 18:37:37 2010 Return-Path: Delivered-To: freebsd-performance@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8D988106564A for ; Thu, 30 Sep 2010 18:37:37 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id DA79B8FC13 for ; Thu, 30 Sep 2010 18:37:36 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id VAA26963; Thu, 30 Sep 2010 21:23:59 +0300 (EEST) (envelope-from avg@icyb.net.ua) Message-ID: <4CA4D5BE.5030502@icyb.net.ua> Date: Thu, 30 Sep 2010 21:23:58 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100920 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: "R.P. Aditya" References: In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=KOI8-U Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Thu, 30 Sep 2010 19:35:48 +0000 Cc: freebsd-performance@FreeBSD.org Subject: Re: monitoring per-process disk io X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Sep 2010 18:37:37 -0000 on 30/09/2010 21:02 R.P. Aditya said the following: > I'm trying to monitor, over the long-term, per-process disk IO (a > counter of bytes read and written per pid would be ideal). > > I currently monitor per pid cpu and memory usage using the SNMP > Host-Resources MIB, however I don't see any oids for io (disk or > otherwise) per oid. > > The closest I've come to finding what I want -- per-process disk IO > stats -- is the Linux tool dstat -- something like "screenshot 3" at: > > http://dag.wieers.com/home-made/dstat/ > > which is the output of: > > dstat -c --top-cpu -d --top-bio --top-latency > > would be good...iostat, vmstat don't give that sort of info... > > any suggestions or hints? 1) top -m io 2) rolling your own customized monitoring using dtrace -- Andriy Gapon From owner-freebsd-performance@FreeBSD.ORG Fri Oct 1 07:17:01 2010 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F3DE9106566C for ; Fri, 1 Oct 2010 07:17:00 +0000 (UTC) (envelope-from massimo@cedoc.mo.it) Received: from cp-out7.libero.it (cp-out7.libero.it [212.52.84.107]) by mx1.freebsd.org (Postfix) with ESMTP id BA1A18FC14 for ; Fri, 1 Oct 2010 07:17:00 +0000 (UTC) Received: from intanto (151.65.158.54) by cp-out7.libero.it (8.5.107) id 4C9778C301763610 for freebsd-performance@freebsd.org; Fri, 1 Oct 2010 09:05:45 +0200 Date: Fri, 1 Oct 2010 09:05:45 +0200 From: Massimo Lusetti To: freebsd-performance@freebsd.org Message-ID: <20101001090545.1a7eb487@intanto> In-Reply-To: References: X-Mailer: Claws Mail 3.7.4 (GTK+ 2.20.1; i486-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: freebsd router X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Oct 2010 07:17:01 -0000 On Wed, 29 Sep 2010 22:44:53 +0200 Samuel Mart=EDn Moro wrote: > Hi, >=20 >=20 > I'm trying to replace my (dying) gateway with a qnap ts-509 (1G DDR, > celeron m420 1.6Ghzs). Someway OT but... do you have the LCD display work out from that box!? Cheers --=20 Massimo From owner-freebsd-performance@FreeBSD.ORG Fri Oct 1 07:31:56 2010 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 85EB51065672 for ; Fri, 1 Oct 2010 07:31:56 +0000 (UTC) (envelope-from faust64@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3F4C78FC14 for ; Fri, 1 Oct 2010 07:31:55 +0000 (UTC) Received: by qyk33 with SMTP id 33so48657qyk.13 for ; Fri, 01 Oct 2010 00:31:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:in-reply-to :references:from:date:message-id:subject:to:cc:content-type; bh=cW0PkG38rW0Ya1LnI5IgN3iO87SRKNLZ6mOQPyTLXTQ=; b=MXCHy2lzJl0CDVuzKzd+96TkN2EikRM8U0+qIlUOm+j6Ndjgb6McB+0LlVIpiArMNF QKx+jJQx0XUiSlouw5wvuvRn+zRzsCMiQvXwW6k63ySTpbAMQMVDoFHjfrSM07YCaL61 U1LJY0KwyEviC3vdnbEI5s2TI17dcXu2PBhrQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; b=iFXbgYVzRq2Ws+98b5qDboD1XIe4GmIWqUIeH9tzqQdvtJr6dseI4IQr/bnRLPOtNU 67liOw6YNjkT5EPHq5oRAggqK/BcCiEUW/qXc+thQ8D/oKUbwVbfQXi5t8D6oWVofl85 HZHwo2whhXZUdCD+A9/Ii9K9vQRYaJK6jWJAk= Received: by 10.229.213.212 with SMTP id gx20mr3548690qcb.60.1285918314805; Fri, 01 Oct 2010 00:31:54 -0700 (PDT) MIME-Version: 1.0 Received: by 10.229.10.103 with HTTP; Fri, 1 Oct 2010 00:31:14 -0700 (PDT) In-Reply-To: <20101001090545.1a7eb487@intanto> References: <20101001090545.1a7eb487@intanto> From: =?ISO-8859-1?Q?Samuel_Mart=EDn_Moro?= Date: Fri, 1 Oct 2010 09:31:14 +0200 Message-ID: To: Massimo Lusetti Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-performance@freebsd.org Subject: Re: freebsd router X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Oct 2010 07:31:56 -0000 err, yep, and it only displays 'system booting' when the nas is running. I understood from their forums that the only way to have it working is to keep their FW. but, just take a look at their linux booting. I didn't even try their management interface. If I've some time to waste, I'll investigate that... Regards, --=20 Samuel Mart=EDn Moro {EPITECH.} tek5 CamTrace S.A.S (+033) 1 41 38 37 60 1 All=E9e de la Venelle 92150 Suresnes FRANCE "Nobody wants to say how this works. Maybe nobody knows ..." Xorg.conf(5)