From owner-freebsd-performance@FreeBSD.ORG Thu Nov 18 09:47:29 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ED8CD16A4CE for ; Thu, 18 Nov 2004 09:47:29 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.204]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9987A43D4C for ; Thu, 18 Nov 2004 09:47:29 +0000 (GMT) (envelope-from aaron.glenn@gmail.com) Received: by rproxy.gmail.com with SMTP id b11so1021536rne for ; Thu, 18 Nov 2004 01:47:17 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:mime-version:content-type:content-transfer-encoding; b=Nn9/TYi/Gi0kDn2yhQubY28y2hn6egH59urKdq5vJc/aiDV10O7nECs051+bBIgy8qNTBcuGK7s4jjSGFGloCM0B6wx9RK265+j5KrtNHjPZQ5kWnEZJgRcDNJhT4yz4BdneFz5P9RvjN2MUG+kcl3Hn1+HdLXY6wcM7nmvkln0= Received: by 10.38.59.47 with SMTP id h47mr185314rna; Thu, 18 Nov 2004 01:47:17 -0800 (PST) Received: by 10.38.151.56 with HTTP; Thu, 18 Nov 2004 01:47:17 -0800 (PST) Message-ID: <18f6019404111801471db5bbfd@mail.gmail.com> Date: Thu, 18 Nov 2004 01:47:17 -0800 From: Aaron Glenn To: freebsd-performance@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Aaron Glenn List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Nov 2004 09:47:30 -0000 I'm pushing large files via thttpd over low-end hardware (celeron 1.8GHz, 512MB RAM, UATA 100 drive) and, out of the box, FreeBSD 5.3-RELEASE topped out at 40Mbps sustained. After creating a separate partition with a much larger blocksize, it's hit 50Mbps sustained but won't go past 54Mbps at all. iostat shows the drive pushing 3.5MB/s pretty consistently. There are on average 750 http connections. Interrupts take 12% of the CPU, according to top, and another 2% for thttpd itself. The box doesn't swap. My dmesg is inexplicably gone...which I realize is going to seriously hamper any constructive input. It is an fxp card, and the IDE controller is a generic Intel controller (the motherboard is an old gateway desktop pull). The /var/log/messages file is filled with: Nov 18 04:00:15 d thttpd[38743]: write - Socket is not connected sending /path/to/file.name I'd like to know what else I can to do maximize raw network I/O. I don't see why this box can't push 90Mbps. My good friend, colleague, and Linux zealot, is eating this up. (-: Regards, aaron.glenn From owner-freebsd-performance@FreeBSD.ORG Thu Nov 18 10:10:42 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A375216A4DE for ; Thu, 18 Nov 2004 10:10:42 +0000 (GMT) Received: from web14121.mail.yahoo.com (web14121.mail.yahoo.com [66.163.171.112]) by mx1.FreeBSD.org (Postfix) with SMTP id 0DA2343D5F for ; Thu, 18 Nov 2004 10:10:27 +0000 (GMT) (envelope-from cguttesen@yahoo.dk) Received: (qmail 55890 invoked by uid 60001); 18 Nov 2004 10:10:27 -0000 Message-ID: <20041118101026.55888.qmail@web14121.mail.yahoo.com> Received: from [194.248.174.58] by web14121.mail.yahoo.com via HTTP; Thu, 18 Nov 2004 11:10:26 CET Date: Thu, 18 Nov 2004 11:10:26 +0100 (CET) From: Claus Guttesen To: Aaron Glenn , freebsd-performance@freebsd.org In-Reply-To: <18f6019404111801471db5bbfd@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Nov 2004 10:10:43 -0000 > I'm pushing large files via thttpd over low-end > hardware (celeron > 1.8GHz, 512MB RAM, UATA 100 drive) and, out of the > box, FreeBSD > 5.3-RELEASE topped out at 40Mbps sustained. After > creating a separate > partition with a much larger blocksize, it's hit > 50Mbps sustained but > won't go past 54Mbps at all. You could try adding/tweaking HZ to 1000/2000 in kernel and add kern.polling.enable=1 to /etc/sysctl.conf. > Nov 18 04:00:15 d thttpd[38743]: write - Socket is > not connected > sending /path/to/file.name > Try adding: kern.ipc.somaxconn=2048 kern.maxfiles=65536 net.isr.enable=1 to /etc/sysctl.conf as well. net.isr.enable gave better network-performance on a 5.3 RC3 nfs-server and lower cpu-usage. HTH. Claus From owner-freebsd-performance@FreeBSD.ORG Thu Nov 18 10:55:44 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1A1AF16A4CE for ; Thu, 18 Nov 2004 10:55:44 +0000 (GMT) Received: from web41208.mail.yahoo.com (web41208.mail.yahoo.com [66.218.93.41]) by mx1.FreeBSD.org (Postfix) with SMTP id E087543D46 for ; Thu, 18 Nov 2004 10:55:43 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 10297 invoked by uid 60001); 18 Nov 2004 10:55:43 -0000 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=ost7Z7U6X7I9A5om43VyYAFbpAdznXfEfrsNURRFv9ZaJ4rmL19kSgJcl8CzbGzXF6xHpNNHStv/JluEioFEA1SL9d48Gj6uQSYvwEv0d0L6buZHMlPgH0xJ2gSVe6UcQ2VFpchLh/aubZhfEOilUParbfq+QRm7pUjwj1XscZU= ; Message-ID: <20041118105543.10295.qmail@web41208.mail.yahoo.com> Received: from [83.129.203.165] by web41208.mail.yahoo.com via HTTP; Thu, 18 Nov 2004 02:55:43 PST Date: Thu, 18 Nov 2004 02:55:43 -0800 (PST) From: Arne "Wörner" To: Claus Guttesen , Aaron Glenn , freebsd-performance@freebsd.org In-Reply-To: <20041118101026.55888.qmail@web14121.mail.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Nov 2004 10:55:44 -0000 > > I'm pushing large files via thttpd over low-end > > hardware (celeron > > 1.8GHz, 512MB RAM, UATA 100 drive) and, out of the > > box, FreeBSD > > 5.3-RELEASE topped out at 40Mbps sustained. After > > creating a separate > > partition with a much larger blocksize, it's hit > > 50Mbps sustained but > > won't go past 54Mbps at all. > > You could try adding/tweaking HZ to 1000/2000 in > kernel and add kern.polling.enable=1 to > /etc/sysctl.conf. > Here I would like to mention the man page polling(4) for clarification (it does not look like the GENERIC kernel can do polling...). Furthermore I would like to know, how fast is a "dd if=/dev/discXY of=/dev/null bs=64k count=1000" in single user mode on your box (I can do 42.8MByte/sec with UDMA-100 and ST340015A (Seagate?))... -Arne __________________________________ Do you Yahoo!? The all-new My Yahoo! - Get yours free! http://my.yahoo.com From owner-freebsd-performance@FreeBSD.ORG Thu Nov 18 11:02:49 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CF6F816A4D0 for ; Thu, 18 Nov 2004 11:02:49 +0000 (GMT) Received: from daemon.kr.FreeBSD.org (www.kr.freebsd.org [211.115.73.199]) by mx1.FreeBSD.org (Postfix) with ESMTP id BE65F43D31 for ; Thu, 18 Nov 2004 11:02:47 +0000 (GMT) (envelope-from cjh@kr.FreeBSD.org) Received: from localhost (localhost [127.0.0.1]) by daemon.kr.FreeBSD.org (Postfix) with ESMTP id 3C2591A6D0; Thu, 18 Nov 2004 20:02:41 +0900 (KST) Received: from daemon.kr.FreeBSD.org ([127.0.0.1]) by localhost (daemon.kr.freebsd.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 56666-06; Thu, 18 Nov 2004 20:02:37 +0900 (KST) Received: from localhost (gradius [211.44.63.164]) by daemon.kr.FreeBSD.org (Postfix) with ESMTP id 67A0F1A6C9; Thu, 18 Nov 2004 20:02:37 +0900 (KST) Date: Thu, 18 Nov 2004 19:58:54 +0900 (KST) Message-Id: <20041118.195854.85380612.cjh@kr.FreeBSD.org> To: aaron.glenn@gmail.com From: CHOI Junho In-Reply-To: <18f6019404111801471db5bbfd@mail.gmail.com> References: <18f6019404111801471db5bbfd@mail.gmail.com> Organization: Korea FreeBSD Users Group X-URL: http://www.kr.FreeBSD.org/~cjh X-Mailer: Mew version 4.1.50 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at kr.FreeBSD.org cc: freebsd-performance@freebsd.org Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Nov 2004 11:02:49 -0000 Can you test it on 4.x machines? I think 5.x is not stable/optimized for kqueue operations of thttpd. If you have sufficient(not swapping) memory, you can do 100Mbps/sec easily, as long as on 4.x. And, check several sysctls: somaxconn, mbuf, and polling. From: Aaron Glenn Subject: I've ran out of ideas Date: Thu, 18 Nov 2004 01:47:17 -0800 > I'm pushing large files via thttpd over low-end hardware (celeron > 1.8GHz, 512MB RAM, UATA 100 drive) and, out of the box, FreeBSD > 5.3-RELEASE topped out at 40Mbps sustained. After creating a separate > partition with a much larger blocksize, it's hit 50Mbps sustained but > won't go past 54Mbps at all. > > iostat shows the drive pushing 3.5MB/s pretty consistently. There are > on average 750 http connections. Interrupts take 12% of the CPU, > according to top, and another 2% for thttpd itself. The box doesn't > swap. > > My dmesg is inexplicably gone...which I realize is going to seriously > hamper any constructive input. It is an fxp card, and the IDE > controller is a generic Intel controller (the motherboard is an old > gateway desktop pull). The /var/log/messages file is filled with: > > Nov 18 04:00:15 d thttpd[38743]: write - Socket is not connected > sending /path/to/file.name > > I'd like to know what else I can to do maximize raw network I/O. I > don't see why this box can't push 90Mbps. My good friend, colleague, > and Linux zealot, is eating this up. (-: > > Regards, > aaron.glenn > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" -- CHOI Junho cjh@[kr.]FreeBSD.org Key fingerprint = 1369 7374 A45F F41A F3C0 07E3 4A01 C020 E602 60F5 From owner-freebsd-performance@FreeBSD.ORG Thu Nov 18 12:08:31 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 057AA16A4CE for ; Thu, 18 Nov 2004 12:08:31 +0000 (GMT) Received: from web14107.mail.yahoo.com (web14107.mail.yahoo.com [216.136.172.137]) by mx1.FreeBSD.org (Postfix) with SMTP id E867A43D46 for ; Thu, 18 Nov 2004 12:08:30 +0000 (GMT) (envelope-from cguttesen@yahoo.dk) Received: (qmail 34442 invoked by uid 60001); 18 Nov 2004 12:08:30 -0000 Message-ID: <20041118120830.34440.qmail@web14107.mail.yahoo.com> Received: from [194.248.174.58] by web14107.mail.yahoo.com via HTTP; Thu, 18 Nov 2004 13:08:30 CET Date: Thu, 18 Nov 2004 13:08:30 +0100 (CET) From: Claus Guttesen To: Arne "Wörner" , Aaron Glenn , freebsd-performance@freebsd.org In-Reply-To: <20041118105543.10295.qmail@web41208.mail.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Nov 2004 12:08:31 -0000 > > You could try adding/tweaking HZ to 1000/2000 in > > kernel and add kern.polling.enable=1 to > > /etc/sysctl.conf. > Here I would like to mention the man page polling(4) > for clarification (it does > not look like the GENERIC kernel can do polling...). Add options HZ=2000 options DEVICE_POLLING to your kernel. Claus From owner-freebsd-performance@FreeBSD.ORG Thu Nov 18 17:20:38 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 47E7816A4CE for ; Thu, 18 Nov 2004 17:20:38 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.206]) by mx1.FreeBSD.org (Postfix) with ESMTP id CED1A43D41 for ; Thu, 18 Nov 2004 17:20:37 +0000 (GMT) (envelope-from aaron.glenn@gmail.com) Received: by rproxy.gmail.com with SMTP id b11so1062089rne for ; Thu, 18 Nov 2004 09:19:57 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=qcHRcPk2nuhud2D1Wop0/+8INGls35JjqhapIu/BKgCTduTOpvn/lEsQ3Yr/Ivl1wM27E7V5W7yqseaJtvtqBPGd1nF0HrDeW6M5HSgLREFGdLm31KnoMWxfvG7cWEOg0BbTLGps/vpt9oA+bJTWLcagkkkFxK3vt367xHRDXZc= Received: by 10.38.59.47 with SMTP id h47mr330706rna; Thu, 18 Nov 2004 09:19:57 -0800 (PST) Received: by 10.38.151.56 with HTTP; Thu, 18 Nov 2004 09:19:57 -0800 (PST) Message-ID: <18f6019404111809195ec88089@mail.gmail.com> Date: Thu, 18 Nov 2004 09:19:57 -0800 From: Aaron Glenn To: CHOI Junho In-Reply-To: <20041118.195854.85380612.cjh@kr.FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <18f6019404111801471db5bbfd@mail.gmail.com> <20041118.195854.85380612.cjh@kr.FreeBSD.org> cc: freebsd-performance@freebsd.org Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Aaron Glenn List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Nov 2004 17:20:38 -0000 Unless there is something of interest to the project, I can't. I don't have a spare box of it's kind, unfortunately, and I have zero interest in using 4.x for anything. Would a move to Apache 1.3 or Apache 2 better utilize 5.x? aaron.glenn On Thu, 18 Nov 2004 19:58:54 +0900 (KST), CHOI Junho wrote: > > Can you test it on 4.x machines? I think 5.x is not stable/optimized > for kqueue operations of thttpd. > > If you have sufficient(not swapping) memory, you can do 100Mbps/sec > easily, as long as on 4.x. > > And, check several sysctls: somaxconn, mbuf, and polling. > > From: Aaron Glenn > Subject: I've ran out of ideas > Date: Thu, 18 Nov 2004 01:47:17 -0800 > > > > > I'm pushing large files via thttpd over low-end hardware (celeron > > 1.8GHz, 512MB RAM, UATA 100 drive) and, out of the box, FreeBSD > > 5.3-RELEASE topped out at 40Mbps sustained. After creating a separate > > partition with a much larger blocksize, it's hit 50Mbps sustained but > > won't go past 54Mbps at all. > > > > iostat shows the drive pushing 3.5MB/s pretty consistently. There are > > on average 750 http connections. Interrupts take 12% of the CPU, > > according to top, and another 2% for thttpd itself. The box doesn't > > swap. > > > > My dmesg is inexplicably gone...which I realize is going to seriously > > hamper any constructive input. It is an fxp card, and the IDE > > controller is a generic Intel controller (the motherboard is an old > > gateway desktop pull). The /var/log/messages file is filled with: > > > > Nov 18 04:00:15 d thttpd[38743]: write - Socket is not connected > > sending /path/to/file.name > > > > I'd like to know what else I can to do maximize raw network I/O. I > > don't see why this box can't push 90Mbps. My good friend, colleague, > > and Linux zealot, is eating this up. (-: > > > > Regards, > > aaron.glenn > > _______________________________________________ > > freebsd-performance@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > > To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" > > -- > CHOI Junho cjh@[kr.]FreeBSD.org > Key fingerprint = 1369 7374 A45F F41A F3C0 07E3 4A01 C020 E602 60F5 > From owner-freebsd-performance@FreeBSD.ORG Thu Nov 18 17:22:42 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 06C0C16A4CE for ; Thu, 18 Nov 2004 17:22:42 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.192]) by mx1.FreeBSD.org (Postfix) with ESMTP id A564E43D1F for ; Thu, 18 Nov 2004 17:22:41 +0000 (GMT) (envelope-from aaron.glenn@gmail.com) Received: by rproxy.gmail.com with SMTP id b11so1062554rne for ; Thu, 18 Nov 2004 09:22:34 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=T2KrcnE3Ek8xo9nTUqqqFbP/oJbuukclGxPuOi5NAsI88MFGJzPWx7C7Ucs5MxZ9d6nEdtjABA7aeIrWS5T3rGvJbprSICqQAegwjpzwicPdw3u9bu5SJuHdN0bvYSgOmWQ2Q6Wx7CRNnkLr6McclrCrge/Z7uV2o2dX3yFY38w= Received: by 10.38.71.79 with SMTP id t79mr1768rna; Thu, 18 Nov 2004 09:22:34 -0800 (PST) Received: by 10.38.151.56 with HTTP; Thu, 18 Nov 2004 09:22:34 -0800 (PST) Message-ID: <18f6019404111809224fb97c06@mail.gmail.com> Date: Thu, 18 Nov 2004 09:22:34 -0800 From: Aaron Glenn To: =?ISO-8859-1?Q?Arne_W=F6rner?= In-Reply-To: <20041118105543.10295.qmail@web41208.mail.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable References: <20041118101026.55888.qmail@web14121.mail.yahoo.com> <20041118105543.10295.qmail@web41208.mail.yahoo.com> cc: freebsd-performance@freebsd.org Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Aaron Glenn List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Nov 2004 17:22:42 -0000 d# dd if=3D/dev/ad2s1 of=3D/dev/null bs=3D64k count=3D1000 1000+0 records in 1000+0 records out 65536000 bytes transferred in 1.353203 secs (48430278 bytes/sec) d# iostat has never gone above 3.8MB/s aaron.glenn On Thu, 18 Nov 2004 02:55:43 -0800 (PST), Arne W=F6rner wrote: >=20 >=20 > > > I'm pushing large files via thttpd over low-end > > > hardware (celeron > > > 1.8GHz, 512MB RAM, UATA 100 drive) and, out of the > > > box, FreeBSD > > > 5.3-RELEASE topped out at 40Mbps sustained. After > > > creating a separate > > > partition with a much larger blocksize, it's hit > > > 50Mbps sustained but > > > won't go past 54Mbps at all. > > > > You could try adding/tweaking HZ to 1000/2000 in > > kernel and add kern.polling.enable=3D1 to > > /etc/sysctl.conf. > > > Here I would like to mention the man page polling(4) for clarification (i= t does > not look like the GENERIC kernel can do polling...). >=20 > Furthermore I would like to know, how fast is a "dd if=3D/dev/discXY of= =3D/dev/null > bs=3D64k count=3D1000" in single user mode on your box (I can do 42.8MByt= e/sec with > UDMA-100 and ST340015A (Seagate?))... >=20 > -Arne >=20 >=20 > __________________________________ > Do you Yahoo!? > The all-new My Yahoo! - Get yours free! > http://my.yahoo.com >=20 > From owner-freebsd-performance@FreeBSD.ORG Thu Nov 18 17:23:39 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7E18916A4CE for ; Thu, 18 Nov 2004 17:23:39 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.198]) by mx1.FreeBSD.org (Postfix) with ESMTP id F00FB43D48 for ; Thu, 18 Nov 2004 17:23:38 +0000 (GMT) (envelope-from aaron.glenn@gmail.com) Received: by rproxy.gmail.com with SMTP id b11so1062741rne for ; Thu, 18 Nov 2004 09:23:38 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=dz9E1uwgNlrRvQ8xAXcebQX8rD8jsvNTQOfxFcJ3IcfI+1J83tOKomV/i79QPQg2vTdXL811DhexM1U7KzFSkp4ABNx2cQzPaNwp8tFKRa3UPrm1FHfhipu9U7xX9ysvPFDi4qUM6ROv3fP23P1ciyiraEQYGPlh9GpYjXjna3s= Received: by 10.38.125.42 with SMTP id x42mr384515rnc; Thu, 18 Nov 2004 09:23:38 -0800 (PST) Received: by 10.38.151.56 with HTTP; Thu, 18 Nov 2004 09:23:36 -0800 (PST) Message-ID: <18f60194041118092322751420@mail.gmail.com> Date: Thu, 18 Nov 2004 09:23:36 -0800 From: Aaron Glenn To: freebsd-performance@freebsd.org In-Reply-To: <18f6019404111801471db5bbfd@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <18f6019404111801471db5bbfd@mail.gmail.com> Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Aaron Glenn List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Nov 2004 17:23:39 -0000 On Thu, 18 Nov 2004 01:47:17 -0800, Aaron Glenn wrote: > My dmesg is inexplicably gone... ...and so it returns. d# dmesg Copyright (c) 1992-2004 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 5.3-RELEASE #0: Thu Nov 18 05:49:47 PST 2004 root@d.ish.cc:/usr/src/sys/i386/compile/GENERIC Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Celeron(R) CPU 1.70GHz (1699.95-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf13 Stepping = 3 Features=0x3febfbff real memory = 535035904 (510 MB) avail memory = 513896448 (490 MB) ACPI APIC Table: ioapic0 irqs 0-23 on motherboard npx0: [FAST] npx0: on motherboard npx0: INT 16 interface acpi0: on motherboard acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 cpu0: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 agp0: mem 0xffa80000-0xffafffff,0xf0000000-0xf7ffffff irq 16 at device 2.0 on pci0 agp0: detected 892k stolen memory agp0: aperture size is 128M uhci0: port 0xe800-0xe81f irq 16 at device 29.0 on pci0 uhci0: [GIANT-LOCKED] usb0: on uhci0 usb0: USB revision 1.0 uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered uhci1: port 0xe880-0xe89f irq 19 at device 29.1 on pci0 uhci1: [GIANT-LOCKED] usb1: on uhci1 usb1: USB revision 1.0 uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub1: 2 ports with 2 removable, self powered uhci2: port 0xec00-0xec1f irq 18 at device 29.2 on pci0 uhci2: [GIANT-LOCKED] usb2: on uhci2 usb2: USB revision 1.0 uhub2: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub2: 2 ports with 2 removable, self powered pci0: at device 29.7 (no driver attached) pcib1: at device 30.0 on pci0 pci1: on pcib1 fxp0: port 0xdc00-0xdc3f mem 0xff8ff000-0xff8fffff irq 20 at device 8.0 on pci1 miibus0: on fxp0 inphy0: on miibus0 inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto fxp0: Ethernet address: 00:07:e9:ff:86:dd isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0xffa0-0xffaf,0x376,0x170-0x177,0x3f6,0x1f0-0x1f7 at device 31.1 on pci0 ata0: channel #0 on atapci0 ata1: channel #1 on atapci0 pci0: at device 31.3 (no driver attached) pci0: at device 31.5 (no driver attached) acpi_button0: on acpi0 fdc0: port 0x3f7,0x3f4-0x3f5,0x3f2-0x3f3,0x3f0-0x3f1 irq 6 drq 2 on acpi0 fdc0: [FAST] fd0: <1440-KB 3.5" drive> on fdc0 drive 0 sio0: configured irq 4 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A ppc0: port 0x778-0x77f,0x378-0x37f irq 7 drq 3 on acpi0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/8 bytes threshold ppbus0: on ppc0 plip0: on ppbus0 lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 pmtimer0 on isa0 atkbdc0: at port 0x64,0x60 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio1: configured irq 3 not in bitmap of probed irqs 0 sio1: port may not be enabled vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounter "TSC" frequency 1699953264 Hz quality 800 Timecounters tick every 1.000 msec acpi_cpu: throttling enabled, 8 steps (100% to 12.5%), currently 100.0% ad2: 19092MB [38792/16/63] at ata1-master UDMA100 Mounting root from ufs:/dev/ad2s1a From owner-freebsd-performance@FreeBSD.ORG Thu Nov 18 18:32:00 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0D11016A4CE for ; Thu, 18 Nov 2004 18:32:00 +0000 (GMT) Received: from web41202.mail.yahoo.com (web41202.mail.yahoo.com [66.218.93.35]) by mx1.FreeBSD.org (Postfix) with SMTP id D49C843D41 for ; Thu, 18 Nov 2004 18:31:59 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 87239 invoked by uid 60001); 18 Nov 2004 18:31:57 -0000 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=ixtp1PWhrA0Ur7rlquY/W2UyYNYtpKB77rzqsaqKHQf8cIlOXiIyPZeL4VBLloPt2siHhkWOTR66JoLUY4dZePP+3EnyHPjQ6qNNR3pbsty5MD6zuebVPGYwv6oj5DsFkuDOlJ9WudM29MBOHwcxfbKkFkcpZEoqQ3RCB1/P0hc= ; Message-ID: <20041118183156.87236.qmail@web41202.mail.yahoo.com> Received: from [83.129.203.165] by web41202.mail.yahoo.com via HTTP; Thu, 18 Nov 2004 10:31:55 PST Date: Thu, 18 Nov 2004 10:31:55 -0800 (PST) From: Arne "Wörner" To: Aaron Glenn In-Reply-To: <18f6019404111809224fb97c06@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii cc: freebsd-performance@freebsd.org Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Nov 2004 18:32:00 -0000 > d# dd if=/dev/ad2s1 of=/dev/null bs=64k count=1000 > 1000+0 records in > 1000+0 records out > 65536000 bytes transferred in 1.353203 secs (48430278 bytes/sec) > d# > Hmm... So the hard disc driver delivers data quite quickly... Maybe you could try a read from /dev/zero via ftp (by logging in as a user via ftp)? Then we would see, how fast your network driver + card is (e. g. if you use "wget" there is a speed meter on the right)... -Arne __________________________________ Do you Yahoo!? Meet the all-new My Yahoo! - Try it today! http://my.yahoo.com From owner-freebsd-performance@FreeBSD.ORG Thu Nov 18 23:12:14 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 31A0516A4CE for ; Thu, 18 Nov 2004 23:12:14 +0000 (GMT) Received: from note.orchestra.cse.unsw.EDU.AU (note.orchestra.cse.unsw.EDU.AU [129.94.242.24]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5BEC343D53 for ; Thu, 18 Nov 2004 23:12:13 +0000 (GMT) (envelope-from lukem@cse.unsw.edu.au) Received: From wagner With LocalMail ; Fri, 19 Nov 2004 10:11:59 +1100 From: lukem.freebsd@cse.unsw.edu.au Sender: lukem@cse.unsw.edu.au To: Aaron Glenn Date: Fri, 19 Nov 2004 10:11:59 +1100 (EST) In-Reply-To: <18f6019404111809224fb97c06@mail.gmail.com> Message-ID: References: <20041118101026.55888.qmail@web14121.mail.yahoo.com> <20041118105543.10295.qmail@web41208.mail.yahoo.com> <18f6019404111809224fb97c06@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed cc: freebsd-performance@freebsd.org cc: =?ISO-8859-1?Q?Arne_W=F6rner?= Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Nov 2004 23:12:14 -0000 On Thu, 18 Nov 2004, Aaron Glenn wrote: > d# dd if=/dev/ad2s1 of=/dev/null bs=64k count=1000 > 1000+0 records in > 1000+0 records out > 65536000 bytes transferred in 1.353203 secs (48430278 bytes/sec) > d# > > iostat has never gone above 3.8MB/s The simple answer is that both iostat and dd are giving you the correct results. Try your dd again with a smaller block size (like the 512 byte default) and you should see that disk throughput is closer to what you are seeing for your web server. I would theorise that your web server is probably only issuing fairly small disk reads, so your network performance is being bound by the disk. To test this theory, you could: * create an mfs partition and serve off it to see what kind of performance you can get * see if you can tweak the web server to use larger reads and writes. In your original message you said that increasing the disk block size improved performance, which would also indicate that you are being limited by disk performance. -- Luke From owner-freebsd-performance@FreeBSD.ORG Thu Nov 18 23:29:19 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9DDB516A4CE for ; Thu, 18 Nov 2004 23:29:19 +0000 (GMT) Received: from mail.trippynames.com (mail.trippynames.com [38.113.223.19]) by mx1.FreeBSD.org (Postfix) with ESMTP id 868EE43D1F for ; Thu, 18 Nov 2004 23:29:19 +0000 (GMT) (envelope-from sean@chittenden.org) Received: from localhost (localhost [127.0.0.1]) by mail.trippynames.com (Postfix) with ESMTP id 39BECA6C14; Thu, 18 Nov 2004 15:29:19 -0800 (PST) Received: from mail.trippynames.com ([127.0.0.1]) by localhost (rand.nxad.com [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 17860-04; Thu, 18 Nov 2004 15:29:17 -0800 (PST) Received: from [192.168.123.128] (unknown [38.113.223.82]) by mail.trippynames.com (Postfix) with ESMTP id DB78EA6C2F; Thu, 18 Nov 2004 15:29:17 -0800 (PST) In-Reply-To: <18f6019404111801471db5bbfd@mail.gmail.com> References: <18f6019404111801471db5bbfd@mail.gmail.com> Mime-Version: 1.0 (Apple Message framework v619) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Sean Chittenden Date: Thu, 18 Nov 2004 15:29:14 -0800 To: Aaron Glenn X-Mailer: Apple Mail (2.619) cc: freebsd-performance@freebsd.org Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Nov 2004 23:29:19 -0000 > I'm pushing large files via thttpd over low-end hardware (celeron > 1.8GHz, 512MB RAM, UATA 100 drive) and, out of the box, FreeBSD > 5.3-RELEASE topped out at 40Mbps sustained. After creating a separate > partition with a much larger blocksize, it's hit 50Mbps sustained but > won't go past 54Mbps at all. Are you using thttpd+sendfile(2)? If so, I'm going to guess you've exhausted your sendfile(2) buffers. Can you run top to see what the state of your thttpd process is? I've bumped into this numerous times. If you look through arch@ there's a patch from me to change the behavior of sendfile(2) so that it returns with ENOBUFs instead of blocking when your sendfile buffers are exhausted. I would've thought that alc@'s work on sendfile(2) would've prevented this from happening, but you may have just exhausted them in which case you're getting killed by this. -sc -- Sean Chittenden From owner-freebsd-performance@FreeBSD.ORG Fri Nov 19 01:48:44 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 88B4D16A4CE for ; Fri, 19 Nov 2004 01:48:44 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.193]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3122043D31 for ; Fri, 19 Nov 2004 01:48:44 +0000 (GMT) (envelope-from aaron.glenn@gmail.com) Received: by rproxy.gmail.com with SMTP id b11so28344rne for ; Thu, 18 Nov 2004 17:48:43 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=otLcpiFDgTE5/iB6q/zczjtNjfprqeX3aYptbRfSAlDltyAobzPuvMw56iyM2TmfGT/Pr9kn2eIBkCjSh331jOpPt6mf16iFG0Z2kmf86rp883KkdC0hht3Y6dba4ki0hWkzKk5H0i+A8LgPEejC0H+R5XU5OgIXlX81Xv+qktU= Received: by 10.38.67.26 with SMTP id p26mr191434rna; Thu, 18 Nov 2004 17:48:43 -0800 (PST) Received: by 10.38.151.56 with HTTP; Thu, 18 Nov 2004 17:48:43 -0800 (PST) Message-ID: <18f6019404111817487397f53b@mail.gmail.com> Date: Thu, 18 Nov 2004 17:48:43 -0800 From: Aaron Glenn To: =?ISO-8859-1?Q?Arne_W=F6rner?= In-Reply-To: <20041118183156.87236.qmail@web41202.mail.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable References: <18f6019404111809224fb97c06@mail.gmail.com> <20041118183156.87236.qmail@web41202.mail.yahoo.com> cc: freebsd-performance@freebsd.org Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Aaron Glenn List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 01:48:44 -0000 On Thu, 18 Nov 2004 10:31:55 -0800 (PST), Arne W=F6rner wrote: > Hmm... So the hard disc driver delivers data quite quickly... >=20 > Maybe you could try a read from /dev/zero via ftp (by logging in as a use= r via > ftp)? Then we would see, how fast your network driver + card is (e. g. if= you > use "wget" there is a speed meter on the right)... ftp> get /dev/zero local: /dev/zero remote: /dev/zero 229 Entering Extended Passive Mode (|||58848|) 150 Opening BINARY mode data connection for '/dev/zero'. ^C receive aborted. Waiting for remote to finish abort. 426 Data connection: Interrupted system call. 225 ABOR command successful. 73784040 bytes received in 00:11 (5.86 MB/s) ftp> get /dev/zero local: /dev/zero remote: /dev/zero 229 Entering Extended Passive Mode (|||49748|) 150 Opening BINARY mode data connection for '/dev/zero'. ^C receive aborted. Waiting for remote to finish abort. 426 Data connection: Interrupted system call. 225 ABOR command successful. 461636400 bytes received in 01:13 (6.00 MB/s) aaron.glenn From owner-freebsd-performance@FreeBSD.ORG Fri Nov 19 01:53:05 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A2E9416A4CE for ; Fri, 19 Nov 2004 01:53:05 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.192]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4B8E043D48 for ; Fri, 19 Nov 2004 01:53:05 +0000 (GMT) (envelope-from aaron.glenn@gmail.com) Received: by rproxy.gmail.com with SMTP id b11so28614rne for ; Thu, 18 Nov 2004 17:53:05 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=GR793xSBJ9eM2/N7xMQux1sN52JTBKQV5WaT20bcWCTC1meKjd2k0GHRt94R3OoWQAayxkkGJojzgdZRKdAjrG0gnl0Tum02qgoGsgQAC4/wZBhsmiSmnV5okaDZIaq8bKBkd3xU+dFwEvE5hiZYErlWzjwua9fL5XHHKDPwsxU= Received: by 10.38.181.73 with SMTP id d73mr191043rnf; Thu, 18 Nov 2004 17:53:04 -0800 (PST) Received: by 10.38.151.56 with HTTP; Thu, 18 Nov 2004 17:53:04 -0800 (PST) Message-ID: <18f6019404111817533b93cbba@mail.gmail.com> Date: Thu, 18 Nov 2004 17:53:04 -0800 From: Aaron Glenn To: "lukem.freebsd@cse.unsw.edu.au" In-Reply-To: Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <20041118101026.55888.qmail@web14121.mail.yahoo.com> <20041118105543.10295.qmail@web41208.mail.yahoo.com> <18f6019404111809224fb97c06@mail.gmail.com> cc: freebsd-performance@freebsd.org Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Aaron Glenn List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 01:53:05 -0000 On Fri, 19 Nov 2004 10:11:59 +1100 (EST), lukem.freebsd@cse.unsw.edu.au wrote: > The simple answer is that both iostat and dd are giving you the correct > results. Try your dd again with a smaller block size (like the 512 byte > default) and you should see that disk throughput is closer to what you are > seeing for your web server. The /www partition it's reading from was made with -b 65536 and -f 8192. > I would theorise that your web server is probably only issuing fairly > small disk reads, so your network performance is being bound by the disk. That was my first hunch; hence creating a separate partition with obscene blocksizes (-: > To test this theory, you could: > > * create an mfs partition and serve off it to see what kind of performance > you can get > * see if you can tweak the web server to use larger reads and writes. > > In your original message you said that increasing the disk block size > improved performance, which would also indicate that you are being limited > by disk performance. I doubt increasing it even more would help. Law of diminishing returns and all that; then again I'm no expert. Thoughts? Regards, aaron.glenn From owner-freebsd-performance@FreeBSD.ORG Fri Nov 19 01:55:46 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7DEB616A4CF for ; Fri, 19 Nov 2004 01:55:46 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.200]) by mx1.FreeBSD.org (Postfix) with ESMTP id E2DAA43D49 for ; Fri, 19 Nov 2004 01:55:45 +0000 (GMT) (envelope-from aaron.glenn@gmail.com) Received: by rproxy.gmail.com with SMTP id b11so28916rne for ; Thu, 18 Nov 2004 17:55:45 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=ldjvmGC2uWUeXfOUHje4jTUXgzSrZcTK0/Eie1HECNpGdkoVbhrhtj5BC0DnKLkL9KMGx5Z428jfyV4WfwtwE3e9R+kGZvV9wDFPWEI7d77w0XhSj5Yp83yEnHojjeVJYY12GM3R3xOcEEWQOzCmJwfkD8Z8VBSpUCVqsHDY4YQ= Received: by 10.38.71.67 with SMTP id t67mr189018rna; Thu, 18 Nov 2004 17:55:45 -0800 (PST) Received: by 10.38.151.56 with HTTP; Thu, 18 Nov 2004 17:55:45 -0800 (PST) Message-ID: <18f6019404111817553208789e@mail.gmail.com> Date: Thu, 18 Nov 2004 17:55:45 -0800 From: Aaron Glenn To: Sean Chittenden In-Reply-To: Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <18f6019404111801471db5bbfd@mail.gmail.com> cc: freebsd-performance@freebsd.org Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Aaron Glenn List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 01:55:46 -0000 On Thu, 18 Nov 2004 15:29:14 -0800, Sean Chittenden wrote: > Are you using thttpd+sendfile(2)? If so, I'm going to guess you've > exhausted your sendfile(2) buffers. Can you run top to see what the > state of your thttpd process is? I've bumped into this numerous times. sendfile? I wasn't aware of that option. I'll look into it 11303 nobody -8 0 4124K 3516K biord 4:35 2.83% 2.83% thttpd This is with the previous sysctl changes mentioned, and a new kernel with HZ set to 2000 and DEVICE_POLLING enabled. Absolutely no change on the interrupts - still 12-14% cpu. > If you look through arch@ there's a patch from me to change the > behavior of sendfile(2) so that it returns with ENOBUFs instead of > blocking when your sendfile buffers are exhausted. I would've thought > that alc@'s work on sendfile(2) would've prevented this from happening, > but you may have just exhausted them in which case you're getting > killed by this. -sc > > -- > Sean Chittenden From owner-freebsd-performance@FreeBSD.ORG Fri Nov 19 01:58:54 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 84AC116A4CE for ; Fri, 19 Nov 2004 01:58:54 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.194]) by mx1.FreeBSD.org (Postfix) with ESMTP id 356E643D2D for ; Fri, 19 Nov 2004 01:58:54 +0000 (GMT) (envelope-from aaron.glenn@gmail.com) Received: by rproxy.gmail.com with SMTP id b11so29097rne for ; Thu, 18 Nov 2004 17:58:53 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=e4H1xYo1em7aiuuKy/SzR/4XgdV1GQS8cyvS4/Jew5cSidRZ0vpVnT56UC9GWtv7v6IFlX3gJcj1Sq3QY0nUO1dgbXwubySVcxVxUcjPOQYc6Ixi1Yg1uck85ysCfyIW7YE3c/SiEjdMDLUGfH1ygT9xMIVt0+76YM4YaWcbBi8= Received: by 10.38.161.47 with SMTP id j47mr188125rne; Thu, 18 Nov 2004 17:58:53 -0800 (PST) Received: by 10.38.151.56 with HTTP; Thu, 18 Nov 2004 17:58:53 -0800 (PST) Message-ID: <18f6019404111817582375cf37@mail.gmail.com> Date: Thu, 18 Nov 2004 17:58:53 -0800 From: Aaron Glenn To: Claus Guttesen In-Reply-To: <20041118120830.34440.qmail@web14107.mail.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <20041118105543.10295.qmail@web41208.mail.yahoo.com> <20041118120830.34440.qmail@web14107.mail.yahoo.com> cc: freebsd-performance@freebsd.org Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Aaron Glenn List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 01:58:54 -0000 On Thu, 18 Nov 2004 13:08:30 +0100 (CET), Claus Guttesen wrote: > > Add > > options HZ=2000 > options DEVICE_POLLING > > to your kernel. > Claus No impact on CPU interrupts (still 12-14%) and, if anything, its actually slightly decreased my thoroughput (hovering at 47Mbps instead of 50Mbps+) I've been suggested a custom thttpd package and some more sysctl's offlist. I'm going to try those in a second. Regards, aaron.glenn From owner-freebsd-performance@FreeBSD.ORG Fri Nov 19 02:16:45 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7F7D816A4CE for ; Fri, 19 Nov 2004 02:16:45 +0000 (GMT) Received: from web41211.mail.yahoo.com (web41211.mail.yahoo.com [66.218.93.44]) by mx1.FreeBSD.org (Postfix) with SMTP id 4CB2743D2D for ; Fri, 19 Nov 2004 02:16:45 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 88480 invoked by uid 60001); 19 Nov 2004 02:16:45 -0000 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=zLwhJQ6m5J+zGEUwt5Hd+zPa1koRScrKQpAco3GXx7WKuQFF1gQ4I1bDoDmUkXbJl9mheEFSgGTwb42gEpSfvaVDQsMW7aZ2D/m+0B6NWxR+Tv3zQIAg43HbjJDnFyB0Aj6jAKamd9lKEwWTcfgCTmpimZvChRslQoJoP8ZlJRo= ; Message-ID: <20041119021644.88478.qmail@web41211.mail.yahoo.com> Received: from [83.129.203.165] by web41211.mail.yahoo.com via HTTP; Thu, 18 Nov 2004 18:16:44 PST Date: Thu, 18 Nov 2004 18:16:44 -0800 (PST) From: Arne "Wörner" To: freebsd-performance@freebsd.org In-Reply-To: <18f6019404111817487397f53b@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 02:16:45 -0000 --- Aaron Glenn wrote: > On Thu, 18 Nov 2004 10:31:55 -0800 (PST), Arne Wörner > wrote: > > Hmm... So the hard disc driver delivers data quite quickly... > > > > Maybe you could try a read from /dev/zero via ftp (by logging in as a user > via > > ftp)? Then we would see, how fast your network driver + card is (e. g. if > you > > use "wget" there is a speed meter on the right)... > > ftp> get /dev/zero > local: /dev/zero remote: /dev/zero > 229 Entering Extended Passive Mode (|||58848|) > 150 Opening BINARY mode data connection for '/dev/zero'. > ^C > receive aborted. Waiting for remote to finish abort. > 426 Data connection: Interrupted system call. > 225 ABOR command successful. > 73784040 bytes received in 00:11 (5.86 MB/s) > ftp> get /dev/zero > local: /dev/zero remote: /dev/zero > 229 Entering Extended Passive Mode (|||49748|) > 150 Opening BINARY mode data connection for '/dev/zero'. > ^C > receive aborted. Waiting for remote to finish abort. > 426 Data connection: Interrupted system call. > 225 ABOR command successful. > 461636400 bytes received in 01:13 (6.00 MB/s) > > aaron.glenn > __________________________________ Do you Yahoo!? The all-new My Yahoo! - Get yours free! http://my.yahoo.com From owner-freebsd-performance@FreeBSD.ORG Fri Nov 19 02:23:49 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1F7A416A4CE for ; Fri, 19 Nov 2004 02:23:49 +0000 (GMT) Received: from web41208.mail.yahoo.com (web41208.mail.yahoo.com [66.218.93.41]) by mx1.FreeBSD.org (Postfix) with SMTP id DFF6843D2F for ; Fri, 19 Nov 2004 02:23:48 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 91479 invoked by uid 60001); 19 Nov 2004 02:23:48 -0000 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=ViDXSf7wrHytw6T/Ne0aqY++qzdFfv0NNmIUbVUBLSnFNHfhqmma/03ZjnU+TH8OGN+tCv9zf/tVLNcfKd9zo+If3zGm14iad1opGyF7hwwjgqeqIOQuZ0Ox8FmicenXqaPSGuym7HE1tKJXBdv1uF/neu+MBhFz/5czdGKLXjE= ; Message-ID: <20041119022348.91477.qmail@web41208.mail.yahoo.com> Received: from [83.129.203.165] by web41208.mail.yahoo.com via HTTP; Thu, 18 Nov 2004 18:23:48 PST Date: Thu, 18 Nov 2004 18:23:48 -0800 (PST) From: Arne "Wörner" To: freebsd-performance@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 02:23:49 -0000 Ooopps... My last email has somehow the wrong content... Sorry. --- Aaron Glenn wrote: > On Thu, 18 Nov 2004 10:31:55 -0800 (PST), Arne Wörner > wrote: > > Hmm... So the hard disc driver delivers data quite quickly... > > > > Maybe you could try a read from /dev/zero via ftp (by logging in as a user > > via ftp)? Then we would see, how fast your network driver + card is (e. g. > > if you use "wget" there is a speed meter on the right)... > > ftp> get /dev/zero [...] > 73784040 bytes received in 00:11 (5.86 MB/s) > Hmm... Since /dev/zero delivers on my box about 100-300MByte/sec (100Mbyte/sec for 512byte block size), it looks like your network driver hinders the throughputting proceedings (I assume, that your other network devices can handle 90Mbit/sec; no ethernet collisions, no concurrency, ...). Are you sure, that Linux performs better in the same setting? Furthermore I saw on my box, that indeed a block size of 2KByte slows "dd..." down to 10MByte/sec (which is just 25% of the "maximum"). Maybe the handshaking of TCP causes some slow down? Did you try "ping -c 10 -s 60000 " where denotes another host? Now I do not have any further ideas, but I am interested in the ping-results... -Arne __________________________________ Do you Yahoo!? Meet the all-new My Yahoo! - Try it today! http://my.yahoo.com From owner-freebsd-performance@FreeBSD.ORG Fri Nov 19 02:34:24 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 806D416A4CE for ; Fri, 19 Nov 2004 02:34:24 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.199]) by mx1.FreeBSD.org (Postfix) with ESMTP id BA9F543D31 for ; Fri, 19 Nov 2004 02:34:21 +0000 (GMT) (envelope-from fehwalker@gmail.com) Received: by rproxy.gmail.com with SMTP id a36so1103659rnf for ; Thu, 18 Nov 2004 18:34:17 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=Dk7Mre/VtFWznrPYqbUofQz4puEoQNR7c2uak8HDKLSSf9b1/x3pUM82Dmrick3MGsBwRliBhHzGj6PSzv/7G0vXQPCovkScmcNDKSuIGJrdg5omV8RkQbw2wvpBggO7pDfY8D4H3aXgIgcpPAbFJ1tm5Wnqoxuy9e9KR78QDNw= Received: by 10.38.82.31 with SMTP id f31mr262939rnb; Thu, 18 Nov 2004 18:34:17 -0800 (PST) Received: by 10.38.74.74 with HTTP; Thu, 18 Nov 2004 18:34:17 -0800 (PST) Message-ID: <35de0c30041118183491b383b@mail.gmail.com> Date: Thu, 18 Nov 2004 21:34:17 -0500 From: Bryan Fullerton To: freebsd-performance@freebsd.org In-Reply-To: <18f6019404111801471db5bbfd@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <18f6019404111801471db5bbfd@mail.gmail.com> Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Bryan Fullerton List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 02:34:24 -0000 On Thu, 18 Nov 2004 01:47:17 -0800, Aaron Glenn wrote: > I'm pushing large files via thttpd over low-end hardware (celeron > 1.8GHz, 512MB RAM, UATA 100 drive) and, out of the box, FreeBSD > 5.3-RELEASE topped out at 40Mbps sustained. After creating a separate > partition with a much larger blocksize, it's hit 50Mbps sustained but > won't go past 54Mbps at all. I haven't seen this in any of the other replies (or if it was there I miised it, sorry) - have you confirmed that the network interface is running at full duplex on both your box and the switch? Bryan From owner-freebsd-performance@FreeBSD.ORG Fri Nov 19 03:02:04 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 27DB516A4CE for ; Fri, 19 Nov 2004 03:02:04 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.197]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6D8B543D49 for ; Fri, 19 Nov 2004 03:02:03 +0000 (GMT) (envelope-from aaron.glenn@gmail.com) Received: by rproxy.gmail.com with SMTP id b11so32981rne for ; Thu, 18 Nov 2004 19:02:03 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=BXmfCMgfn1xU0wl2x2LWcWaXN/5lQ14Bx8JnEv6tyMnXMS4I6ntvYJhmGSEPEb64zSeca8kd/tyPQmYwLkJqWM1dV2kBcwgBECDXpDqBA/VV98zHYzf1zmoN8+I0G9hD7c85Y8VbOj24Hj9zueeOI8hj6FZ3v6eXy0IKK67K/hI= Received: by 10.38.67.26 with SMTP id p26mr211876rna; Thu, 18 Nov 2004 19:02:02 -0800 (PST) Received: by 10.38.151.56 with HTTP; Thu, 18 Nov 2004 19:02:02 -0800 (PST) Message-ID: <18f601940411181902605bebbd@mail.gmail.com> Date: Thu, 18 Nov 2004 19:02:02 -0800 From: Aaron Glenn To: Bryan Fullerton In-Reply-To: <35de0c30041118183491b383b@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <18f6019404111801471db5bbfd@mail.gmail.com> <35de0c30041118183491b383b@mail.gmail.com> cc: freebsd-performance@freebsd.org Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Aaron Glenn List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 03:02:04 -0000 On Thu, 18 Nov 2004 21:34:17 -0500, Bryan Fullerton wrote: > I haven't seen this in any of the other replies (or if it was there I > miised it, sorry) - have you confirmed that the network interface is > running at full duplex on both your box and the switch? d# ifconfig fxp0 fxp0: flags=18843 mtu 1500 options=48 inet 64.235.239.37 netmask 0xfffffff8 broadcast 64.235.239.39 inet6 fe80::207:e9ff:feff:86dd%fxp0 prefixlen 64 scopeid 0x1 ether 00:07:e9:ff:86:dd media: Ethernet autoselect (100baseTX ) status: active d# Regards, aaron.glenn From owner-freebsd-performance@FreeBSD.ORG Fri Nov 19 03:21:36 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D0D4B16A4CE for ; Fri, 19 Nov 2004 03:21:36 +0000 (GMT) Received: from web41205.mail.yahoo.com (web41205.mail.yahoo.com [66.218.93.38]) by mx1.FreeBSD.org (Postfix) with SMTP id 75C4843D2F for ; Fri, 19 Nov 2004 03:21:36 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 89882 invoked by uid 60001); 19 Nov 2004 03:21:36 -0000 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=ROpLo9A244D6PnqRGsyHRIOQl5PRBny9Qyt9y5qVWnEQQeJ1CU6eK1xSy4oJsohHWqh5W+jtVhKQ1svTXtKjyB5C4r4hjPVwmk5T3g0OWMsLxpnYIjl1vtjltmHWhKNoVX+Rnbn8X2glIoOO8mcsAqflHjm70eJX9EJD4JqFfBU= ; Message-ID: <20041119032135.89880.qmail@web41205.mail.yahoo.com> Received: from [83.129.203.165] by web41205.mail.yahoo.com via HTTP; Thu, 18 Nov 2004 19:21:35 PST Date: Thu, 18 Nov 2004 19:21:35 -0800 (PST) From: Arne "Wörner" To: Luke In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii cc: freebsd-performance@freebsd.org Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 03:21:36 -0000 --- Luke wrote: > On Thu, 18 Nov 2004, Arne Wörner wrote: > > Hmm... Since /dev/zero delivers on my box about 100-300MByte/sec > > (100Mbyte/sec for 512byte block size), it looks like your network driver > > hinders the throughputting proceedings (I assume, that your other > > network devices can handle 90Mbit/sec; no ethernet collisions, no > > concurrency, ...). Are you sure, that Linux performs better in the same > > setting? > > I think you must mean Mbps, not MBps. If it were bytes, you would be > saturating 2.5 gigabit links with a single connection. Not to mention > most PCI busses. > neo# dd if=/dev/zero of=/dev/null bs=512k count=1000 1000+0 records in 1000+0 records out 524288000 bytes transferred in 1.475860 secs (355242396 bytes/sec) I meant /dev/zero, which does not use PCI or so (it just produces zeros; that's why it is so fast). Just the bus between processor and main memory, I think... With my 133MHz FSB(?) my DDRAM can do 8Gbit/sec??? I am not so good in such things... Looks like each byte gets copied three times before it reaches /dev/null... Does somebody agree? Maybe my contributions are too weird and misleading, but I do not see, that they are technically wrong. -Arne __________________________________ Do you Yahoo!? Meet the all-new My Yahoo! - Try it today! http://my.yahoo.com From owner-freebsd-performance@FreeBSD.ORG Fri Nov 19 03:34:12 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D653D16A4D1 for ; Fri, 19 Nov 2004 03:34:12 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7911243D41 for ; Fri, 19 Nov 2004 03:34:12 +0000 (GMT) (envelope-from aaron.glenn@gmail.com) Received: by rproxy.gmail.com with SMTP id b11so34891rne for ; Thu, 18 Nov 2004 19:34:11 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=XTrc165MPhbFGgZC+Q6ZZHb6bG/SEBo3SriXWxlbSfhvZ2owYVXRCAJf8bRuUsyiJoly+tHYhXkbW/XWwUXjnsMp7egwLtjMYpkim03Ovbz6Rv62akfs2tvGR4+Or+MYORRFxvFlTl6P+KMJ9ePEmCLyWbxc1DiAywpvstGvTdI= Received: by 10.38.67.12 with SMTP id p12mr215498rna; Thu, 18 Nov 2004 19:34:11 -0800 (PST) Received: by 10.38.151.56 with HTTP; Thu, 18 Nov 2004 19:34:11 -0800 (PST) Message-ID: <18f6019404111819343f2a7967@mail.gmail.com> Date: Thu, 18 Nov 2004 19:34:11 -0800 From: Aaron Glenn To: Luke In-Reply-To: Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <20041118101026.55888.qmail@web14121.mail.yahoo.com> <20041118105543.10295.qmail@web41208.mail.yahoo.com> <18f6019404111809224fb97c06@mail.gmail.com> <18f6019404111817533b93cbba@mail.gmail.com> cc: freebsd-performance@freebsd.org Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Aaron Glenn List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 03:34:13 -0000 On Fri, 19 Nov 2004 14:16:51 +1100 (EST), Luke wrote: > Increasing the block size of the disk need not imply that the application > is issuing larger reads. Hmm, true. > I have not suggested that you increase the blocksize further. Please read > the above points again. > > As Sean pointed out, if your web server supports sendfile then that would > be a good option. Indeed I misread your email. I do have some sort optimized thttpd package that was sent to me off list. I'll try that out in a few minutes here. I'm also searching for the patch on arch@ that Sean mentioned. > Your recent post which showed poor performance from /dev/zero does bring > the theory that your disk is the bottleneck into question, however you may > still find it is worth trying the above suggestions. I neglected to mention those numbers were taken while pushing 45Mbps out the network. If you add them up you get 93Mbps. aaron.glenn From owner-freebsd-performance@FreeBSD.ORG Fri Nov 19 03:11:19 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8898D16A4CE for ; Fri, 19 Nov 2004 03:11:19 +0000 (GMT) Received: from note.orchestra.cse.unsw.EDU.AU (note.orchestra.cse.unsw.EDU.AU [129.94.242.24]) by mx1.FreeBSD.org (Postfix) with ESMTP id F170243D5E for ; Fri, 19 Nov 2004 03:11:18 +0000 (GMT) (envelope-from lukem@cse.unsw.edu.au) Received: From wagner With LocalMail ; Fri, 19 Nov 2004 14:11:11 +1100 From: Luke To: =?X-UNKNOWN?Q?Arne_W=F6rner?= Date: Fri, 19 Nov 2004 14:11:11 +1100 (EST) In-Reply-To: <20041119022348.91477.qmail@web41208.mail.yahoo.com> Message-ID: References: <20041119022348.91477.qmail@web41208.mail.yahoo.com> MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="1585517554-865016676-1100833871=:11282" X-Mailman-Approved-At: Fri, 19 Nov 2004 13:18:23 +0000 cc: freebsd-performance@freebsd.org Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 03:11:19 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --1585517554-865016676-1100833871=:11282 Content-Type: TEXT/PLAIN; charset=X-UNKNOWN; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE On Thu, 18 Nov 2004, Arne W=F6rner wrote: > Hmm... Since /dev/zero delivers on my box about 100-300MByte/sec=20 > (100Mbyte/sec for 512byte block size), it looks like your network driver= =20 > hinders the throughputting proceedings (I assume, that your other=20 > network devices can handle 90Mbit/sec; no ethernet collisions, no=20 > concurrency, ...). Are you sure, that Linux performs better in the same= =20 > setting? I think you must mean Mbps, not MBps. If it were bytes, you would be=20 saturating 2.5 gigabit links with a single connection. Not to mention=20 most PCI busses. --=20 Luke --1585517554-865016676-1100833871=:11282-- From owner-freebsd-performance@FreeBSD.ORG Fri Nov 19 03:16:56 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4D71416A4CE for ; Fri, 19 Nov 2004 03:16:56 +0000 (GMT) Received: from note.orchestra.cse.unsw.EDU.AU (note.orchestra.cse.unsw.EDU.AU [129.94.242.24]) by mx1.FreeBSD.org (Postfix) with ESMTP id B501743D45 for ; Fri, 19 Nov 2004 03:16:55 +0000 (GMT) (envelope-from lukem@cse.unsw.edu.au) Received: From wagner With LocalMail ; Fri, 19 Nov 2004 14:16:51 +1100 From: Luke To: Aaron Glenn Date: Fri, 19 Nov 2004 14:16:51 +1100 (EST) In-Reply-To: <18f6019404111817533b93cbba@mail.gmail.com> Message-ID: References: <20041118101026.55888.qmail@web14121.mail.yahoo.com> <20041118105543.10295.qmail@web41208.mail.yahoo.com> <18f6019404111809224fb97c06@mail.gmail.com> <18f6019404111817533b93cbba@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Mailman-Approved-At: Fri, 19 Nov 2004 13:18:23 +0000 cc: freebsd-performance@freebsd.org cc: "lukem.freebsd@cse.unsw.edu.au" Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 03:16:56 -0000 On Thu, 18 Nov 2004, Aaron Glenn wrote: >> I would theorise that your web server is probably only issuing fairly >> small disk reads, so your network performance is being bound by the disk. > > That was my first hunch; hence creating a separate partition with > obscene blocksizes (-: Increasing the block size of the disk need not imply that the application is issuing larger reads. >> To test this theory, you could: >> >> * create an mfs partition and serve off it to see what kind of performance >> you can get >> * see if you can tweak the web server to use larger reads and writes. >> >> In your original message you said that increasing the disk block size >> improved performance, which would also indicate that you are being limited >> by disk performance. > > I doubt increasing it even more would help. Law of diminishing returns > and all that; then again I'm no expert. Thoughts? I have not suggested that you increase the blocksize further. Please read the above points again. As Sean pointed out, if your web server supports sendfile then that would be a good option. Your recent post which showed poor performance from /dev/zero does bring the theory that your disk is the bottleneck into question, however you may still find it is worth trying the above suggestions. -- Luke From owner-freebsd-performance@FreeBSD.ORG Fri Nov 19 15:26:44 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1306216A4CE for ; Fri, 19 Nov 2004 15:26:44 +0000 (GMT) Received: from mail.geeks.org (jacobs.Geeks.ORG [204.153.247.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id A004D43D5F for ; Fri, 19 Nov 2004 15:26:43 +0000 (GMT) (envelope-from drechsau@Geeks.ORG) Received: by mail.geeks.org (Postfix, from userid 400) id EAFB820B51; Fri, 19 Nov 2004 09:26:42 -0600 (CST) Date: Fri, 19 Nov 2004 09:26:42 -0600 From: Mike Horwath To: Aaron Glenn Message-ID: <20041119152642.GA5143@octanews.net> References: <18f6019404111801471db5bbfd@mail.gmail.com> <35de0c30041118183491b383b@mail.gmail.com> <18f601940411181902605bebbd@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <18f601940411181902605bebbd@mail.gmail.com> X-PGP-Fingerprint: D8 24 CC E6 47 5F E4 60 BF B7 6E FA BF C7 6E C5 X-GPG-Fingerprint: 6A89 E78A B8B1 69D9 8CDB E966 4A5A C3F9 A1B0 C381 User-Agent: Mutt/1.5.6i cc: freebsd-performance@freebsd.org cc: Bryan Fullerton Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 15:26:44 -0000 On Thu, Nov 18, 2004 at 07:02:02PM -0800, Aaron Glenn wrote: > On Thu, 18 Nov 2004 21:34:17 -0500, Bryan Fullerton wrote: > > I haven't seen this in any of the other replies (or if it was there I > > miised it, sorry) - have you confirmed that the network interface is > > running at full duplex on both your box and the switch? > > d# ifconfig fxp0 > fxp0: flags=18843 mtu 1500 > options=48 > inet 64.235.239.37 netmask 0xfffffff8 broadcast 64.235.239.39 > inet6 fe80::207:e9ff:feff:86dd%fxp0 prefixlen 64 scopeid 0x1 > ether 00:07:e9:ff:86:dd > media: Ethernet autoselect (100baseTX ) > status: active > d# That just shows you what the driver thinks. How about trying a: % netstat -I and look at the 'errors' area, see if you have collisions, and the rest. -- Mike Horwath, reachable via drechsau@Geeks.ORG From owner-freebsd-performance@FreeBSD.ORG Fri Nov 19 18:50:17 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BB9CB16A4CE for ; Fri, 19 Nov 2004 18:50:17 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.206]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6759D43D1F for ; Fri, 19 Nov 2004 18:50:17 +0000 (GMT) (envelope-from aaron.glenn@gmail.com) Received: by rproxy.gmail.com with SMTP id b11so101017rne for ; Fri, 19 Nov 2004 10:50:16 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=gcNRVKw/vjHTXY6+r6xZ/r33Zu6jFdTE1tk/oWvK6L1SLTGo5fOAa1XqCuTDkSDzkSfZVLXzUSlOasobpvENondgbRaWyPQhP3wuLH3qHHqm/GJLamKyYqBJOa4Q63Q7Qy90oMuoelD61OwiUFBBp8YfgzHLNMfHOMiTpIcEnoI= Received: by 10.39.1.5 with SMTP id d5mr445411rni; Fri, 19 Nov 2004 10:49:25 -0800 (PST) Received: by 10.38.151.56 with HTTP; Fri, 19 Nov 2004 10:49:25 -0800 (PST) Message-ID: <18f60194041119104939fe9156@mail.gmail.com> Date: Fri, 19 Nov 2004 10:49:25 -0800 From: Aaron Glenn To: Mike Horwath In-Reply-To: <20041119152642.GA5143@octanews.net> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <18f6019404111801471db5bbfd@mail.gmail.com> <35de0c30041118183491b383b@mail.gmail.com> <18f601940411181902605bebbd@mail.gmail.com> <20041119152642.GA5143@octanews.net> cc: freebsd-performance@freebsd.org Subject: Re: I've ran out of ideas X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Aaron Glenn List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 18:50:17 -0000 On Fri, 19 Nov 2004 09:26:42 -0600, Mike Horwath wrote: > How about trying a: > > % netstat -I > > and look at the 'errors' area, see if you have collisions, and the rest. d# netstat -I fxp0 Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll fxp0 1500 00:07:e9:ff:FF:FF 70838149 0 110259682 1 0 fxp0 1500 64.235.xxx.xxx 64.235.xxx.xxx 70840600 - 110264374 - - fxp0 1500 fe80:1::207:e fe80:1::207:e9ff: 0 - 4 - - d# aaron.glenn