From owner-freebsd-performance@FreeBSD.ORG Mon Jul 30 08:19:21 2007 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 15E2516A417 for ; Mon, 30 Jul 2007 08:19:21 +0000 (UTC) (envelope-from patpro@patpro.net) Received: from postfix2-g20.free.fr (postfix2-g20.free.fr [212.27.60.43]) by mx1.freebsd.org (Postfix) with ESMTP id 96C3613C468 for ; Mon, 30 Jul 2007 08:19:20 +0000 (UTC) (envelope-from patpro@patpro.net) Received: from smtp1-g19.free.fr (smtp1-g19.free.fr [212.27.42.27]) by postfix2-g20.free.fr (Postfix) with ESMTP id 18A5218A3FF2 for ; Mon, 30 Jul 2007 08:56:50 +0200 (CEST) Received: from smtp1-g19.free.fr (localhost.localdomain [127.0.0.1]) by smtp1-g19.free.fr (Postfix) with ESMTP id 0DFF51AB2C4 for ; Mon, 30 Jul 2007 09:56:04 +0200 (CEST) Received: from boleskine.patpro.net (boleskine.patpro.net [82.235.12.223]) by smtp1-g19.free.fr (Postfix) with ESMTP id C5E601AB2C5 for ; Mon, 30 Jul 2007 09:56:03 +0200 (CEST) Received: from [192.168.0.2] (unknown [192.168.0.2]) by boleskine.patpro.net (Postfix) with ESMTP id 3D4111CC0E for ; Mon, 30 Jul 2007 09:56:03 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v752.2) To: freebsd-performance@freebsd.org Message-Id: Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-1--830951157; protocol="application/pkcs7-signature" From: Patrick Proniewski Date: Mon, 30 Jul 2007 09:56:02 +0200 X-Mailer: Apple Mail (2.752.2) X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: DSL/ethernet network perf problem with pf X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2007 08:19:21 -0000 --Apple-Mail-1--830951157 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Hello, I'm running a FreeBSD 6.2 on a Tyan motherboard. The board has 3 ethernet ports (fpx0, em0, em1). It uses `pf` to share/protect an internet access over xDSL plugged in fxp0 to 2 LANs on em0/1. When pf is loaded, my transfert rate for a file on the internet reaches about 150-200 KB/s max, but I can download 2 or 3 files each at 120-150 KB/s at the same time. If i disable pf (by unloading the kernel module), my transfert rate jumps to 650-700 KB/s Here is my pf.conf : # macros int_if = "em0" int_if_sec = "em1" ext_if = "fxp0" wif_if = "ath0" tcp_services = "{ 22, 113, 80, 443, 25, 53, 554 }" udp_services = "{ 53 }" admin_tcp_services = "{ 311, 625, 5900, 5988 }" admin_udp_services = "{ 3283 }" icmp_types = "echoreq" priv_nets = "{ 127.0.0.0/8, 172.16.0.0/12, 10.0.0.0/8 }" # Tables: similar to macros, but more flexible for many addresses. table persist { --some ip's-- } table persist { --some ip's-- } table persist file "/etc/pf.liste_ip_spamer" table persist file "/etc/pf.liste_ip_ssh_scan" table persist file "/etc/pf.liste_ip_webspam" table persist { --some ip's-- } # options set block-policy return set loginterface $ext_if # scrub scrub in all # nat/rdr nat on $ext_if from $int_if:network to any -> ($ext_if) nat on $ext_if from $int_if_sec:network to any -> ($ext_if) # filter rules block log all block in log quick proto tcp from to any port smtp block in log quick proto tcp from to any port ssh block in log quick proto tcp from to any port http pass quick on lo0 all block drop in log quick on $ext_if from $priv_nets to any block drop out log quick on $ext_if from any to $priv_nets pass in on $ext_if inet proto tcp from any to ($ext_if) port $tcp_services flags S/SA keep state pass in on $ext_if inet proto udp from any to ($ext_if) port $udp_services keep state ##### admin pass in log on $ext_if inet proto tcp from { , } to { ($ext_if), 192.168.0.2 } port $admin_tcp_services flags S/SA keep state pass in log on $ext_if inet proto udp from { , } to { ($ext_if), 192.168.0.2 } port $admin_udp_services keep state ##### friends #pass in log on $ext_if inet proto tcp from to ($ext_if) flags S/SA keep state #pass in log on $ext_if inet proto udp from to ($ext_if) keep state ##### OpenArena pass in on $ext_if inet proto tcp from to ($ext_if) port 56789 flags S/SA keep state pass in on $ext_if inet proto udp from to ($ext_if) port 56789 keep state pass in inet proto icmp all icmp-type $icmp_types keep state pass in on $int_if from $int_if:network to any keep state pass out on $int_if from any to $int_if:network keep state pass in on $int_if_sec from $int_if_sec:network to any keep state pass out on $int_if_sec from any to $int_if_sec:network keep state pass out on $ext_if proto tcp all modulate state flags S/SA pass out on $ext_if proto { udp, icmp } all keep state any idea how I can reach 650-700 KB/s with pf enabled ? regards, patpro --Apple-Mail-1--830951157-- From owner-freebsd-performance@FreeBSD.ORG Mon Jul 30 08:21:08 2007 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 23F2616A41A for ; Mon, 30 Jul 2007 08:21:08 +0000 (UTC) (envelope-from patpro@patpro.net) Received: from smtp1-g19.free.fr (smtp1-g19.free.fr [212.27.42.27]) by mx1.freebsd.org (Postfix) with ESMTP id BB70B13C468 for ; Mon, 30 Jul 2007 08:21:07 +0000 (UTC) (envelope-from patpro@patpro.net) Received: from smtp1-g19.free.fr (localhost.localdomain [127.0.0.1]) by smtp1-g19.free.fr (Postfix) with ESMTP id D5A0B1AB2FE for ; Mon, 30 Jul 2007 10:21:06 +0200 (CEST) Received: from boleskine.patpro.net (boleskine.patpro.net [82.235.12.223]) by smtp1-g19.free.fr (Postfix) with ESMTP id B2EB21AB2E7 for ; Mon, 30 Jul 2007 10:21:06 +0200 (CEST) Received: from [192.168.0.2] (unknown [192.168.0.2]) by boleskine.patpro.net (Postfix) with ESMTP id 5261F1CC5C for ; Mon, 30 Jul 2007 10:20:57 +0200 (CEST) Resent-Message-Id: Mime-Version: 1.0 (Apple Message framework v752.2) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Resent-Date: Mon, 30 Jul 2007 10:21:05 +0200 Message-Id: Content-Transfer-Encoding: 7bit Resent-To: freebsd-performance@freebsd.org From: Patrick Proniewski Resent-From: Patrick Proniewski Date: Mon, 30 Jul 2007 09:56:02 +0200 To: freebsd-performance@freebsd.org X-Mailer: Apple Mail (2.752.2) Subject: DSL/ethernet network perf problem with pf X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2007 08:21:08 -0000 Hello, I'm running a FreeBSD 6.2 on a Tyan motherboard. The board has 3 ethernet ports (fpx0, em0, em1). It uses `pf` to share/protect an internet access over xDSL plugged in fxp0 to 2 LANs on em0/1. When pf is loaded, my transfert rate for a file on the internet reaches about 150-200 KB/s max, but I can download 2 or 3 files each at 120-150 KB/s at the same time. If i disable pf (by unloading the kernel module), my transfert rate jumps to 650-700 KB/s Here is my pf.conf : # macros int_if = "em0" int_if_sec = "em1" ext_if = "fxp0" wif_if = "ath0" tcp_services = "{ 22, 113, 80, 443, 25, 53, 554 }" udp_services = "{ 53 }" admin_tcp_services = "{ 311, 625, 5900, 5988 }" admin_udp_services = "{ 3283 }" icmp_types = "echoreq" priv_nets = "{ 127.0.0.0/8, 172.16.0.0/12, 10.0.0.0/8 }" # Tables: similar to macros, but more flexible for many addresses. table persist { --some ip's-- } table persist { --some ip's-- } table persist file "/etc/pf.liste_ip_spamer" table persist file "/etc/pf.liste_ip_ssh_scan" table persist file "/etc/pf.liste_ip_webspam" table persist { --some ip's-- } # options set block-policy return set loginterface $ext_if # scrub scrub in all # nat/rdr nat on $ext_if from $int_if:network to any -> ($ext_if) nat on $ext_if from $int_if_sec:network to any -> ($ext_if) # filter rules block log all block in log quick proto tcp from to any port smtp block in log quick proto tcp from to any port ssh block in log quick proto tcp from to any port http pass quick on lo0 all block drop in log quick on $ext_if from $priv_nets to any block drop out log quick on $ext_if from any to $priv_nets pass in on $ext_if inet proto tcp from any to ($ext_if) port $tcp_services flags S/SA keep state pass in on $ext_if inet proto udp from any to ($ext_if) port $udp_services keep state ##### admin pass in log on $ext_if inet proto tcp from { , } to { ($ext_if), 192.168.0.2 } port $admin_tcp_services flags S/SA keep state pass in log on $ext_if inet proto udp from { , } to { ($ext_if), 192.168.0.2 } port $admin_udp_services keep state ##### friends #pass in log on $ext_if inet proto tcp from to ($ext_if) flags S/SA keep state #pass in log on $ext_if inet proto udp from to ($ext_if) keep state ##### OpenArena pass in on $ext_if inet proto tcp from to ($ext_if) port 56789 flags S/SA keep state pass in on $ext_if inet proto udp from to ($ext_if) port 56789 keep state pass in inet proto icmp all icmp-type $icmp_types keep state pass in on $int_if from $int_if:network to any keep state pass out on $int_if from any to $int_if:network keep state pass in on $int_if_sec from $int_if_sec:network to any keep state pass out on $int_if_sec from any to $int_if_sec:network keep state pass out on $ext_if proto tcp all modulate state flags S/SA pass out on $ext_if proto { udp, icmp } all keep state any idea how I can reach 650-700 KB/s with pf enabled ? regards, patpro From owner-freebsd-performance@FreeBSD.ORG Sat Aug 4 08:05:36 2007 Return-Path: Delivered-To: performance@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9C79416A421; Sat, 4 Aug 2007 08:05:36 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id A585613C46C; Sat, 4 Aug 2007 08:05:36 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from rot26.obsecurity.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 8349F1A4D7E; Sat, 4 Aug 2007 01:04:57 -0700 (PDT) Received: by rot26.obsecurity.org (Postfix, from userid 1001) id 52514C0F6; Sat, 4 Aug 2007 04:05:35 -0400 (EDT) Date: Sat, 4 Aug 2007 04:05:35 -0400 From: Kris Kennaway To: performance@FreeBSD.org Message-ID: <20070804080535.GA3952@rot26.obsecurity.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="W/nzBZO5zC0uMSeA" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i Cc: arch@FreeBSD.org Subject: read-write SQL performance X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2007 08:05:36 -0000 --W/nzBZO5zC0uMSeA Content-Type: text/plain; charset=us-ascii Content-Disposition: inline I did some benchmarking of sysbench in read-write mode (previous tests have focused on read-only mode). The reason for this is that the disk hardware in my 8-core test system is slow (single disk) and is too easily saturated. In fact mysql and pgsql have identical performance when writing to disk. In other words, I seem to be mostly benchmarking the disk performance and not database or kernel performance. Faster disk hardware is necessary to explore database performance differences or kernel bottlenecks. An upper bound on possible read-write performance comes from using a memory disk instead of physical disk hardware. I replicated the databases onto a suitably large (2gb) tmpfs and reran the tests together with some mutex profiling. Results are here: http://obsecurity.dyndns.org/sysbench-write.png There are a couple of interesting features. mysql has better peak performance than pgsql, but then quickly falls in the toilet. Profiling indicates that at peak there is some contention on lockmgr locks and the proc lock, but most of the contention is in userland (i.e. within mysql itself). At higher loads the bottleneck is overwhelmingly within mysql (and the system is actually 90-100% idle). This seems to be a serious scaling problem within mysql. Peak pgsql performance is lower than mysql, but there is comparatively little degradation at higher loads. Profiling shows that the dominant bottleneck at all workloads is lockmgr. Fortunately there is a lockmgr rewrite in progress by Attilio for SoC, so there is great scope for performance improvements to pgsql. Significant mysql performance improvements may require fundamental architectural work by the mysql developers. Kris --W/nzBZO5zC0uMSeA Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFGtDNPWry0BWjoQKURArsaAJ0TBneLAFm0JZl16qo/wpeCbxJ7NgCgwHcg NIrOTxurHgldrKsD9BdiPLY= =xNb8 -----END PGP SIGNATURE----- --W/nzBZO5zC0uMSeA-- From owner-freebsd-performance@FreeBSD.ORG Sat Aug 4 20:44:48 2007 Return-Path: Delivered-To: performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 20C2B16A419 for ; Sat, 4 Aug 2007 20:44:48 +0000 (UTC) (envelope-from almarrie@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.241]) by mx1.freebsd.org (Postfix) with ESMTP id CF1AC13C468 for ; Sat, 4 Aug 2007 20:44:47 +0000 (UTC) (envelope-from almarrie@gmail.com) Received: by an-out-0708.google.com with SMTP id c14so260668anc for ; Sat, 04 Aug 2007 13:44:47 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=AsqBLMkKarPCF4w4noo3JRiKG6gOt9WLhPHQUs5rc/vdCnmUn1Q/YQwWP7ZxbtAYnxO5BRET29/ZRuoK4NrWCTlwKjqFs4ufR8lbASYoLRZ4kP4K4ECDabCPUZIYoLd33tTkdD9GRLMGBK343rtTRAAhwUZ279t2QS1Uhl3oADI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=cNUSNTe9VsI+LASPQOokydb5SmjxGxEfeubbSwQjMsaXTcTx1PsbmCwrTumLpOCpO8NWNHVbbGmxVvEVG78bHrACKlwMopxRIhGsN7Tp2Yk6xLWRNF2fZypZgvFFTR/Zuu8yow17zJ3f73y577aYjEmUAF7EjeoPtKCGkcgxEiU= Received: by 10.100.168.13 with SMTP id q13mr2369906ane.1186258840847; Sat, 04 Aug 2007 13:20:40 -0700 (PDT) Received: by 10.100.9.14 with HTTP; Sat, 4 Aug 2007 13:20:40 -0700 (PDT) Message-ID: <499c70c0708041320r1f51cb3qe6f05376cfb8a470@mail.gmail.com> Date: Sat, 4 Aug 2007 23:20:40 +0300 From: "Abdullah Ibn Hamad Al-Marri" To: "Kris Kennaway" In-Reply-To: <20070804080535.GA3952@rot26.obsecurity.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20070804080535.GA3952@rot26.obsecurity.org> Cc: Greg 'groggy' Lehey , arch@freebsd.org, performance@freebsd.org Subject: Re: read-write SQL performance X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2007 20:44:48 -0000 On 8/4/07, Kris Kennaway wrote: > I did some benchmarking of sysbench in read-write mode (previous tests > have focused on read-only mode). The reason for this is that the disk > hardware in my 8-core test system is slow (single disk) and is too > easily saturated. In fact mysql and pgsql have identical performance > when writing to disk. In other words, I seem to be mostly > benchmarking the disk performance and not database or kernel > performance. > > Faster disk hardware is necessary to explore database performance > differences or kernel bottlenecks. An upper bound on possible > read-write performance comes from using a memory disk instead of > physical disk hardware. I replicated the databases onto a suitably > large (2gb) tmpfs and reran the tests together with some mutex > profiling. > > Results are here: > > http://obsecurity.dyndns.org/sysbench-write.png > > There are a couple of interesting features. > > mysql has better peak performance than pgsql, but then quickly falls > in the toilet. Profiling indicates that at peak there is some > contention on lockmgr locks and the proc lock, but most of the > contention is in userland (i.e. within mysql itself). At higher loads > the bottleneck is overwhelmingly within mysql (and the system is > actually 90-100% idle). This seems to be a serious scaling problem > within mysql. > > Peak pgsql performance is lower than mysql, but there is comparatively > little degradation at higher loads. Profiling shows that the dominant > bottleneck at all workloads is lockmgr. > > Fortunately there is a lockmgr rewrite in progress by Attilio for SoC, > so there is great scope for performance improvements to pgsql. > Significant mysql performance improvements may require fundamental > architectural work by the mysql developers. > > Kris Maybe Greg would be interested in the MySQL issues? -- Regards, -Abdullah Ibn Hamad Al-Marri Arab Portal http://www.WeArab.Net/