From owner-freebsd-performance@FreeBSD.ORG Mon Mar 3 20:55:16 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DB013669; Mon, 3 Mar 2014 20:55:15 +0000 (UTC) Received: from mail-ve0-x22d.google.com (mail-ve0-x22d.google.com [IPv6:2607:f8b0:400c:c01::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 885E5C4A; Mon, 3 Mar 2014 20:55:15 +0000 (UTC) Received: by mail-ve0-f173.google.com with SMTP id oy12so2098516veb.18 for ; Mon, 03 Mar 2014 12:55:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:from:date:message-id:subject:to:content-type; bh=JdSynC/ldzqDA39xlwY6fjDHiOS3aONJvt6gABRQleA=; b=enPkRVHZgbhuaN3++U/Vsh8BdcR+XZKxIMWRi5gtUmH7e2o4jDbyaTvssSlzA5cXK4 w+YBG9Cz6mLiIg+QnpA2LjsZo3LjyxJjOIdow9SE+5qYo5c4N9kteHdcSwVZmIKWu6WQ 8K/Ygmm+Gltvif7aq5DdlwcvVIvAqcSMjTdfqzBsBUqHHozpSp9lkzl53h+pBVFdGH3n h69FvylWFxp5fU5aQA5ChrOJPsfXN+n6C3qkyVKcKxeQj75TRvoajRReSN1rEatMeVMj cjPiblx8PSZE6oQ2g7AcV1o/Dr1dVCGhiw1JJO+HDFNa/lSplFFHZ/G3G1VQgmi7ZL9T sIOA== X-Received: by 10.220.250.203 with SMTP id mp11mr18781697vcb.2.1393880114469; Mon, 03 Mar 2014 12:55:14 -0800 (PST) MIME-Version: 1.0 Sender: cochard@gmail.com Received: by 10.58.188.35 with HTTP; Mon, 3 Mar 2014 12:54:54 -0800 (PST) From: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Date: Mon, 3 Mar 2014 21:54:54 +0100 X-Google-Sender-Auth: XkFI0EertvH62-zkxCUmMHyRu-w Message-ID: Subject: Strange network performance on Intel Rangeley (8 cores Atom) To: "freebsd-net@freebsd.org" , freebsd-performance@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 20:55:16 -0000 Hi all, I've got a new toy in my network bench lab: a SuperMicro SuperServer 5018A-FTN4. But I've got a problem for understanding and obtaining good throughput for "routing" or "firewalling" usages. I'm using only the embedded 4 gigabit ports of the Atom C2758 SoC. With the default igb(4) parameters which is to create 8 queues (because there is 8 cores) this server is not able to receive more than 585K packet-per-seconds into one port which is far from the gigabit line-rate (1.48Mpps): I was expecting better throughput with 8 cores. Then I did a bunch of new benchmarks by measuring the impact of number of queue and the results are here: http://bsdrp.net/documentation/examples/forwarding_performance_lab_of_a_superserver_5018a-ftn4#graph => I've got better results with only 4 queues than 8... but still low throughput with only 938Kpps. Then I decided to measure the impact of pf and ipfw on the throughput with 4 and 8 queues. And the results are annoying: http://bsdrp.net/documentation/examples/forwarding_performance_lab_of_a_superserver_5018a-ftn4#graph1 => With 8 queues, enabling pf or ipfw improve the input throughput of the igb(4) port. Why so low throughput with 8 queues ? Why better throughput with pf or ipfw enabled than without ? Thanks, Olivier From owner-freebsd-performance@FreeBSD.ORG Mon Mar 3 21:12:35 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6BDED295; Mon, 3 Mar 2014 21:12:35 +0000 (UTC) Received: from mail-vc0-x22d.google.com (mail-vc0-x22d.google.com [IPv6:2607:f8b0:400c:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 132A818C; Mon, 3 Mar 2014 21:12:34 +0000 (UTC) Received: by mail-vc0-f173.google.com with SMTP id ld13so4099497vcb.4 for ; Mon, 03 Mar 2014 13:12:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=2YhHLr2yG3koVqsEuKa4NeHahw8jnCd+XtKlwDNlni4=; b=dtXgVReBZ736joZk6qUJsg+iyUPUkS2ThySZA30JdOIyQEC+Wpy/aT5ueIkk99RFou /kMXBtcEbaCi6FVtZUP/2NY1+lL1vohSVGFGO0TotkavBu/eevQov9Vc/iz2W+3gF8W9 kdgYmccCK1f52GCgXmOpohxcIC7ybAuM3m5XgXJFzqEDhURsbiAGtcL2TQWuabzGwFtK TNF5GTyGpx81952myd4xwB0zGDUmqjijdFuS1etnHvz/sulWeSVp2zHTJ5FYMnlZSgql p5CcTuW0aXVGgYy2dW8dqNe9clWU2gVCjXI1+SYD8+9ykmzpd5lmmWtz8UQmTRT7501B rJug== MIME-Version: 1.0 X-Received: by 10.58.123.70 with SMTP id ly6mr18857384veb.26.1393881154154; Mon, 03 Mar 2014 13:12:34 -0800 (PST) Received: by 10.221.11.135 with HTTP; Mon, 3 Mar 2014 13:12:34 -0800 (PST) In-Reply-To: References: Date: Mon, 3 Mar 2014 13:12:34 -0800 Message-ID: Subject: Re: Strange network performance on Intel Rangeley (8 cores Atom) From: Jack Vogel To: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" , "freebsd-performance@freebsd.org" X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 21:12:35 -0000 What OS version are you running? Jack On Mon, Mar 3, 2014 at 12:54 PM, Olivier Cochard-Labb=E9 wrote: > Hi all, > > I've got a new toy in my network bench lab: a SuperMicro SuperServer > 5018A-FTN4. > But I've got a problem for understanding and obtaining good throughput fo= r > "routing" or "firewalling" usages. > > I'm using only the embedded 4 gigabit ports of the Atom C2758 SoC. > With the default igb(4) parameters which is to create 8 queues (because > there is 8 cores) this server is not able to receive more than 585K > packet-per-seconds into one port which is far from the gigabit line-rate > (1.48Mpps): I was expecting better throughput with 8 cores. > Then I did a bunch of new benchmarks by measuring the impact of number of > queue and the results are here: > > http://bsdrp.net/documentation/examples/forwarding_performance_lab_of_a_s= uperserver_5018a-ftn4#graph > > =3D> I've got better results with only 4 queues than 8... but still low > throughput with only 938Kpps. > > Then I decided to measure the impact of pf and ipfw on the throughput wit= h > 4 and 8 queues. > And the results are annoying: > > http://bsdrp.net/documentation/examples/forwarding_performance_lab_of_a_s= uperserver_5018a-ftn4#graph1 > > =3D> With 8 queues, enabling pf or ipfw improve the input throughput of t= he > igb(4) port. > > Why so low throughput with 8 queues ? > Why better throughput with pf or ipfw enabled than without ? > > Thanks, > > Olivier > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-performance@FreeBSD.ORG Mon Mar 3 21:30:25 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 315BFB8A; Mon, 3 Mar 2014 21:30:25 +0000 (UTC) Received: from mail-ve0-x22a.google.com (mail-ve0-x22a.google.com [IPv6:2607:f8b0:400c:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D01AC2D8; Mon, 3 Mar 2014 21:30:24 +0000 (UTC) Received: by mail-ve0-f170.google.com with SMTP id pa12so4246304veb.15 for ; Mon, 03 Mar 2014 13:30:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=d1JpshKn+2/nVSeWNsMxW4IRotf6XO4I16IftypQjL4=; b=mRG3dMYYv/DFjHjLwK7BBytULYcui2Ugt4cPYH0VeOlegvWCSWlBRnfUhdAjReSjTS L7slnSsCDta4G92AnrXGh9JKybHE1fJSGl9spVCyA05ANPfqLT7B5+6sMJO5Q3RC/Kp2 VJA86Wm5PdVj0T6x2QgifgLtNxuf56dnJ5+5McuDdyywVKzwxeQ0283ZZRcLzgoqO7Wi hrnyccW9DORx9bAsr+QEMjTtIA3E6gNjO6IDKgUWYls5gJqMg8/R1/ymtCVMogch1h78 pj69zxgQcPDIb31rxoKdZl8MqnRuWKEjshCaFeGxyl7GAHCOo1V0NBbPHxMMvVMXxSSE lKmA== X-Received: by 10.58.37.67 with SMTP id w3mr18756606vej.22.1393882224066; Mon, 03 Mar 2014 13:30:24 -0800 (PST) MIME-Version: 1.0 Sender: cochard@gmail.com Received: by 10.58.188.35 with HTTP; Mon, 3 Mar 2014 13:30:03 -0800 (PST) In-Reply-To: References: From: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Date: Mon, 3 Mar 2014 22:30:03 +0100 X-Google-Sender-Auth: gIU4ZUjLPiEuIWuMCMB-LK4AXUw Message-ID: Subject: Re: Strange network performance on Intel Rangeley (8 cores Atom) To: Jack Vogel Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-net@freebsd.org" , "freebsd-performance@freebsd.org" X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 21:30:25 -0000 On Mon, Mar 3, 2014 at 10:12 PM, Jack Vogel wrote: > What OS version are you running? > Hi Jack, More information about the version used: FreeBSD 10.0-STABLE #0 r262601M And about the igb(4) parameters common to all my tests: hw.igb.rxd="2048" hw.igb.txd="2048" hw.igb.rx_process_limit="-1" hw.igb.max_interrupt_rate="16000" Regards, Olivier From owner-freebsd-performance@FreeBSD.ORG Tue Mar 18 03:13:19 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 32AFFBB7 for ; Tue, 18 Mar 2014 03:13:19 +0000 (UTC) Received: from mail-pd0-x235.google.com (mail-pd0-x235.google.com [IPv6:2607:f8b0:400e:c02::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 065D2F41 for ; Tue, 18 Mar 2014 03:13:19 +0000 (UTC) Received: by mail-pd0-f181.google.com with SMTP id p10so6377470pdj.26 for ; Mon, 17 Mar 2014 20:13:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:disposition-notification-to:date:from:user-agent :mime-version:to:subject:content-type:content-transfer-encoding; bh=oxEiMMS9wBQmfDvcRf8eUWIiPeObQ2eZ+YrtB9Q4msU=; b=etnUeGPAT99Mo9r78aPv0zfP/qw02vHEmogoqoewfU71oB8jmHEbf8O4PwgOe6yEpQ ALTQmc8sWE9LU2GyUt+/wyWArgnyxqD1KYIqS3vXjgc/T4NDQ4iuuUL7rcHLSUUBB5A4 yTKqwGBx+Ks+ac8tqzClu3ft3cHrH90cB7zOvwVYpKsn+lmRIaaWP9INjKvC9AJS3Hzv gFD5fL8iWKXNKWvybCPxXn0ucf1tsOAiYg9kzutqT72MvOC40AsPrQaaxkO3Xl2a7r02 DLFzlZz7jLIUf0wUR80Oan1DEB+7x5zik95UUTO4m36YFyTv+2FRtRapiiizF2Y36NaF PwSg== X-Received: by 10.68.12.133 with SMTP id y5mr7954148pbb.134.1395112398615; Mon, 17 Mar 2014 20:13:18 -0700 (PDT) Received: from [10.11.12.12] (203-206-165-226.perm.iinet.net.au. [203.206.165.226]) by mx.google.com with ESMTPSA id dk1sm47744485pbc.46.2014.03.17.20.13.16 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 17 Mar 2014 20:13:17 -0700 (PDT) Message-ID: <5327B9B7.3050103@gmail.com> Date: Tue, 18 Mar 2014 14:12:55 +1100 From: Petr Janda User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-performance@freebsd.org Subject: FreeBSD 10 and PostgreSQL 9.3 scalability issues Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Tue, 18 Mar 2014 11:31:20 +0000 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 03:13:19 -0000 Hi guys, Just want to share these pgbench results done by DragonFlyBSD, and would like some input on why these numbers look so bad and what can be done to improve (ie. kernel tunables etc) the performance. http://lists.dragonflybsd.org/pipermail/users/attachments/20140310/4250b961/attachment-0001.pdf Cheers, Peter From owner-freebsd-performance@FreeBSD.ORG Tue Mar 18 11:47:53 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D6BFE109 for ; Tue, 18 Mar 2014 11:47:53 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 9A51D2A4 for ; Tue, 18 Mar 2014 11:47:53 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 1F6FB20E7088A; Tue, 18 Mar 2014 11:47:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.8 required=8.0 tests=AWL,BAYES_05,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,RDNS_DYNAMIC,STOX_REPLY_TYPE autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id CBE3220E70886; Tue, 18 Mar 2014 11:47:47 +0000 (UTC) Message-ID: <2610F490C952470C9D15999550F67068@multiplay.co.uk> From: "Steven Hartland" To: "Petr Janda" , References: <5327B9B7.3050103@gmail.com> Subject: Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues Date: Tue, 18 Mar 2014 11:47:43 -0000 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 11:47:53 -0000 ----- Original Message ----- From: "Petr Janda" > Hi guys, > > Just want to share these pgbench results done by DragonFlyBSD, and would > like some input on why these numbers look so bad and what can be done to > improve (ie. kernel tunables etc) the performance. > > http://lists.dragonflybsd.org/pipermail/users/attachments/20140310/4250b961/attachment-0001.pdf Do you have the ability to test with FreeBSD 8.x and 9.x to see if this is regression? Also you don't mention the FS used in each case, so I'm wondering if you used a ZFS install of FreeBSD which could help to explain things. Regards Steve From owner-freebsd-performance@FreeBSD.ORG Tue Mar 18 12:00:49 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B1C2261F for ; Tue, 18 Mar 2014 12:00:49 +0000 (UTC) Received: from smtp.infracaninophile.co.uk (smtp6.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3cd3:cd67:fafa:3d78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5691E3D7 for ; Tue, 18 Mar 2014 12:00:49 +0000 (UTC) Received: from ox-dell39.ox.adestra.com (no-reverse-dns.metronet-uk.com [85.199.232.226] (may be forged)) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.14.8/8.14.8) with ESMTP id s2IC0Yrr029050 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Tue, 18 Mar 2014 12:00:40 GMT (envelope-from matthew@freebsd.org) DKIM-Filter: OpenDKIM Filter v2.8.3 smtp.infracaninophile.co.uk s2IC0Yrr029050 Authentication-Results: smtp.infracaninophile.co.uk/s2IC0Yrr029050; dkim=none reason="no signature"; dkim-adsp=none X-Authentication-Warning: lucid-nonsense.infracaninophile.co.uk: Host no-reverse-dns.metronet-uk.com [85.199.232.226] (may be forged) claimed to be ox-dell39.ox.adestra.com Message-ID: <53283557.4070105@freebsd.org> Date: Tue, 18 Mar 2014 12:00:23 +0000 From: Matthew Seaman User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-performance@freebsd.org Subject: Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues References: <5327B9B7.3050103@gmail.com> In-Reply-To: <5327B9B7.3050103@gmail.com> X-Enigmail-Version: 1.6 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="TrvE6DU40kixEQgjbS3vUXBPeNnSpjw8k" X-Virus-Scanned: clamav-milter 0.98.1 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_00,RDNS_NONE, SPF_SOFTFAIL autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lucid-nonsense.infracaninophile.co.uk X-Mailman-Approved-At: Tue, 18 Mar 2014 12:05:47 +0000 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 12:00:49 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --TrvE6DU40kixEQgjbS3vUXBPeNnSpjw8k Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 03/18/14 03:12, Petr Janda wrote: > ust want to share these pgbench results done by DragonFlyBSD, and would= > like some input on why these numbers look so bad and what can be done t= o > improve (ie. kernel tunables etc) the performance. >=20 > http://lists.dragonflybsd.org/pipermail/users/attachments/20140310/4250= b961/attachment-0001.pdf Using ZFS as the backing for a RDBMS without: * Separate (fast) L2ARC devices * Tuning the ZFS block size to match the postgres IO block size * Setting primarycache to metadata * Tuning the ARC max so ZFS doesn't eat all the RAM * probably other things I can remember off-hand. That's what is wrong. ZFS is known to work particularly badly at the sort of small random IOs that RDBMSes generate (mostly because of the copy-on-write thing) without special tuning and extra hardware for caches. ie. You can't construct a fair test of database performance against other OSes/filesystems if you restrict yourself to using exactly the same hardware. Basically, install the FreeBSD box on UFS2 and try again. Cheers, Matthew --TrvE6DU40kixEQgjbS3vUXBPeNnSpjw8k Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQJ8BAEBCgBmBQJTKDVgXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQxOUYxNTRFQ0JGMTEyRTUwNTQ0RTNGMzAw MDUxM0YxMEUwQTlFNEU3AAoJEABRPxDgqeTnFjYP/RL9xQktUXOHKTQLGSoPrH7b mrprdljjuIa8LHKhh08zyiAwU3V+OIk2JDYX4rSAjLYryRNWMEkEjNi7DTctgJny r5yefOGParhOTP6p9hxaxm2qFUQpPBjSS1V/+Igrd8DEjmfI6Uy5VwJwYR1nfQBQ lLf4Kpa9EUMSZnoJk+4SusoweySjcUBq0BjoGt+1wXpjk4410V38gDEVvdT3APe+ QKkobicvSWkjyrVkZ+o1kskjbvSxFefgc5POYOaJeCKrfgVWvFGirwT7vn8nEZ7N EkALwBtofOIKGeXENUMfH5zCdXVEw/D1zoMEQL0n0SPEuq/1GG9nm+Kvqsf7Z0zp 02PWOGzhbBdHE5Vng/o627wMUd2YrsjFxRkYEiq89sxQTmBqNGDWU+LA+WIi3xGT PUHMPwp/6xEIpLnXf2kxZMwt4fwEPiCJxJSzgKK2lmgj/8eIKjWNWbyyFEIsZMQp Q4LRJ9zFDo5rBe+Mo2du4M2B+vyismUsxr34l7lNXPQ4WiPLY69EbiJA24oBC+zI QsZWEUKHChdlnETeVcCwAA3bMB3Q3UmCHczTGktz0JGfpAQ5j9FpVlxbFkV/P9++ OEi4MFpDNzCJAJY5rS7wr5deROZzcpeFBDvbGsv5nkXhAlzqbUeX8EsJO+4AiPBH 9lYFQGVdJA39ttfg7JK5 =fodm -----END PGP SIGNATURE----- --TrvE6DU40kixEQgjbS3vUXBPeNnSpjw8k-- From owner-freebsd-performance@FreeBSD.ORG Tue Mar 18 12:16:46 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AA0118C0; Tue, 18 Mar 2014 12:16:46 +0000 (UTC) Received: from mail-qa0-x230.google.com (mail-qa0-x230.google.com [IPv6:2607:f8b0:400d:c00::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5B9967B2; Tue, 18 Mar 2014 12:16:46 +0000 (UTC) Received: by mail-qa0-f48.google.com with SMTP id m5so6714690qaj.7 for ; Tue, 18 Mar 2014 05:16:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=jOa7NUWuqIg+9uk93t90+i339uodPNU7vp9Gp637Dt0=; b=GmVQ4FnehVrXZahldBQDP8n5pl8monwHIPbKkv8M59UGWy2ENXdutEQZLnXBrLo/bF YYy+PrM8hDyAbgZdSX81fwCu1BMbn13XhEs3L+pftrt16SRAigSTUeqZU02178ZVa9XR 8G+v/YvHTuVIvNdQtFU/0yt5P53h4VgtYnHIi7LNGumbEIMB7DW6J6X5vA4gPZP3Cqgf k+WitgNGsARAy4YQz1rBlPDn1byGl+msw6EAimYt5SViaq1E39emWuPH09SKneemPb4b 1XawwH2xqDOQxvQXWU9k0Ictq7xTzCaKmbr91pzAwsrbmuO7MoCLiQLSQeHVXCkT4ay3 JBaw== MIME-Version: 1.0 X-Received: by 10.224.29.4 with SMTP id o4mr13517328qac.55.1395145005556; Tue, 18 Mar 2014 05:16:45 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.8.137 with HTTP; Tue, 18 Mar 2014 05:16:44 -0700 (PDT) In-Reply-To: <53283557.4070105@freebsd.org> References: <5327B9B7.3050103@gmail.com> <53283557.4070105@freebsd.org> Date: Tue, 18 Mar 2014 05:16:44 -0700 X-Google-Sender-Auth: C9rZDHuz_SsYVQ5V2cGtck4Duuo Message-ID: Subject: Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues From: Adrian Chadd To: Matthew Seaman Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Mailing Lists X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 12:16:46 -0000 Hi, the pgsql testing done has been analysed by a few developers. The TL;DR version is that there's significant lock contention in the VM / mmap path and it sticks out like a sore thumb when one does lock profiling. -a On 18 March 2014 05:00, Matthew Seaman wrote: > On 03/18/14 03:12, Petr Janda wrote: >> ust want to share these pgbench results done by DragonFlyBSD, and would >> like some input on why these numbers look so bad and what can be done to >> improve (ie. kernel tunables etc) the performance. >> >> http://lists.dragonflybsd.org/pipermail/users/attachments/20140310/4250b961/attachment-0001.pdf > > Using ZFS as the backing for a RDBMS without: > > * Separate (fast) L2ARC devices > * Tuning the ZFS block size to match the postgres IO block size > * Setting primarycache to metadata > * Tuning the ARC max so ZFS doesn't eat all the RAM > * probably other things I can remember off-hand. > > That's what is wrong. ZFS is known to work particularly badly at the > sort of small random IOs that RDBMSes generate (mostly because of the > copy-on-write thing) without special tuning and extra hardware for > caches. ie. You can't construct a fair test of database performance > against other OSes/filesystems if you restrict yourself to using exactly > the same hardware. > > Basically, install the FreeBSD box on UFS2 and try again. > > Cheers, > > Matthew > > From owner-freebsd-performance@FreeBSD.ORG Wed Mar 19 22:25:07 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0BEE3FD3 for ; Wed, 19 Mar 2014 22:25:07 +0000 (UTC) Received: from mail-pd0-x233.google.com (mail-pd0-x233.google.com [IPv6:2607:f8b0:400e:c02::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D36AC77D for ; Wed, 19 Mar 2014 22:25:06 +0000 (UTC) Received: by mail-pd0-f179.google.com with SMTP id w10so9318559pde.10 for ; Wed, 19 Mar 2014 15:25:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:disposition-notification-to:date:from:user-agent :mime-version:to:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=zGiZSZp2UU1GxaHrbmup78khIWGKjYhk/oTXALrPYVc=; b=fuCzQr4PVin2P0V176o29JeK/VijZ07PpXVTHmruRwSOsAmBYrt7UpCsMFDpUAk+ta R5BRAGJ51IFW4tI+0Nyn6TuFyy5022JBA0XroUXzKH1clXuH4XSm4Ut9o+bvVYuM8K6k h8CrHICZJYe0SD5YdO73KPEGIKZPrVqsCZTkEoFbuUGTHXHmVgtdHfNfDIbfr6cc92Rv eVd0iw7ZZmw2yoqseB53FXQC3ssoKSyHEqA8wXRRQQiW6nw5RWJ/QTunn8zXIA+1mOlF C9hWd256ahYyAtXcnzzsUkvTJzEolk+EV0wJIUB6B02xYOCevN4QB+hKtgbjayBfzUYL 5mCQ== X-Received: by 10.68.133.163 with SMTP id pd3mr6306992pbb.166.1395267906520; Wed, 19 Mar 2014 15:25:06 -0700 (PDT) Received: from [10.11.12.12] (203-206-165-226.perm.iinet.net.au. [203.206.165.226]) by mx.google.com with ESMTPSA id g6sm109795125pat.2.2014.03.19.15.25.04 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 19 Mar 2014 15:25:05 -0700 (PDT) Message-ID: <532A192A.1070509@gmail.com> Date: Thu, 20 Mar 2014 09:24:42 +1100 From: Petr Janda User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Steven Hartland , freebsd-performance@freebsd.org Subject: Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues References: <5327B9B7.3050103@gmail.com> <2610F490C952470C9D15999550F67068@multiplay.co.uk> In-Reply-To: <2610F490C952470C9D15999550F67068@multiplay.co.uk> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Wed, 19 Mar 2014 22:33:39 +0000 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 22:25:07 -0000 Hi, As far as I know, the test was done on both UFS2 and ZFS and the difference was marginal. Petr On 18/03/2014 10:47 PM, Steven Hartland wrote: > ----- Original Message ----- From: "Petr Janda" > > >> Hi guys, >> >> Just want to share these pgbench results done by DragonFlyBSD, and would >> like some input on why these numbers look so bad and what can be done to >> improve (ie. kernel tunables etc) the performance. >> >> http://lists.dragonflybsd.org/pipermail/users/attachments/20140310/4250b961/attachment-0001.pdf >> > > > Do you have the ability to test with FreeBSD 8.x and 9.x to see if this is > regression? > > Also you don't mention the FS used in each case, so I'm wondering if you > used a ZFS install of FreeBSD which could help to explain things. > > Regards > Steve From owner-freebsd-performance@FreeBSD.ORG Wed Mar 19 23:40:07 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8158EC1 for ; Wed, 19 Mar 2014 23:40:07 +0000 (UTC) Received: from mail01.lax1.stackjet.com (mon01.lax1.stackjet.com [174.136.104.178]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 62CA1E1A for ; Wed, 19 Mar 2014 23:40:06 +0000 (UTC) Received: from [10.23.9.200] (localhost [127.0.0.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: sean@chittenden.org) by mail01.lax1.stackjet.com (Postfix) with ESMTPSA id C54423E8EA4; Wed, 19 Mar 2014 16:33:13 -0700 (PDT) Received: from [10.23.9.200] ([173.228.12.189] helo=[10.23.9.200]) by nutscrape.filter with SMTPS(ECDHE-RSA-AES256-SHA) (2.4.2); 19 Mar 2014 16:33:10 -0700 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues From: Sean Chittenden In-Reply-To: <532A192A.1070509@gmail.com> Date: Wed, 19 Mar 2014 16:33:10 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <5327B9B7.3050103@gmail.com> <2610F490C952470C9D15999550F67068@multiplay.co.uk> <532A192A.1070509@gmail.com> To: Petr Janda X-Mailer: Apple Mail (2.1874) X-Assp-Version: 2.4.2(14058) on nutscrape.filter X-Assp-ID: nutscrape.filter m-71993-40633 X-Assp-Session: 850424720 (mail 1) X-Assp-Envelope-From: sean@chittenden.org X-Assp-Intended-For: janda.petr@gmail.com X-Assp-Intended-For: killing@multiplay.co.uk X-Assp-Intended-For: freebsd-performance@freebsd.org X-Assp-Client-TLS: yes X-Assp-Server-TLS: yes X-Assp-Whitelisted: Yes Cc: freebsd-performance@freebsd.org, Steven Hartland X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 23:40:07 -0000 > As far as I know, the test was done on both UFS2 and ZFS and the > difference was marginal. As Adrian pointed out, there is an mmap(2) mutex in the way. Starting in = PostgreSQL 9.3, shared buffers are allocated out of mmap(2) instead of = shm. shm is only used to notify the PostgreSQL postmaster that a child = process exited/crashed (when a pid detaches from a shm segment, there is = a kernel event, but there is no kernel event when detaching from an = mmap(2) region). -sc http://www.postgresql.org/docs/9.3/static/release-9-3.html#AEN115039 >>> Just want to share these pgbench results done by DragonFlyBSD, and = would >>> like some input on why these numbers look so bad and what can be = done to >>> improve (ie. kernel tunables etc) the performance. >>>=20 >>> = http://lists.dragonflybsd.org/pipermail/users/attachments/20140310/4250b96= 1/attachment-0001.pdf >>>=20 >>=20 >>=20 >> Do you have the ability to test with FreeBSD 8.x and 9.x to see if = this is >> regression? >>=20 >> Also you don't mention the FS used in each case, so I'm wondering if = you >> used a ZFS install of FreeBSD which could help to explain things. -- Sean Chittenden sean@chittenden.org From owner-freebsd-performance@FreeBSD.ORG Mon Apr 21 22:52:38 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3A7959B0 for ; Mon, 21 Apr 2014 22:52:38 +0000 (UTC) Received: from mail-yh0-f63.google.com (mail-yh0-f63.google.com [209.85.213.63]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EBF9912D3 for ; Mon, 21 Apr 2014 22:52:36 +0000 (UTC) Received: by mail-yh0-f63.google.com with SMTP id 29so1273816yhl.8 for ; Mon, 21 Apr 2014 15:52:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:message-id:in-reply-to :references:subject:mime-version:content-type; bh=lEZoLCK7FiDgEb0CO1IOfOytG3gg2BFULA+IUJZoGpc=; b=VD/17Ycails/vkZitqCxRY7jH7rwq83ahJEJYeSCHOLW5ZzvNQPhA3BE3jxXXk4i2f rPOMwNKMUTOCSGcAVmfSRzRyz2ibGR8ODkp6fljwnuLd3Ee9vI3zcu22rcQTX0KhUWCZ ByIZqiuGuP9Q1xP6RFhnXhnKlEpxayAu79cGtcW5Abtz8gJ/Me9CUE0bXIN7qIp4iAfX OmBWsJjKL+h+SXBmVwSTNtGGKDo8/ZKFvb3/kHbtH3qpptLDv5kQMaQ/+HpfUrB/5q9r EPo1C96qrtRyWMqqZrVE7W6svfxUdqhGHYH1LVe9HPfNFysCqEeu4WEG2EKZtN7eAVAS kqRw== X-Gm-Message-State: ALoCoQnIkm6Oj8IQFYqo6LoPUnT3s+jqrNmo/r4S5lFWYhc+jmXmxFirc8R7HLBNIICjOCV9Kick X-Received: by 10.140.36.6 with SMTP id o6mr54108qgo.26.1398114702242; Mon, 21 Apr 2014 14:11:42 -0700 (PDT) X-Google-Doc-Id: e87f9ad4ab539159 X-Google-Web-Client: true Date: Mon, 21 Apr 2014 14:11:41 -0700 (PDT) From: Palle Girgensohn To: bsdmailinglist@googlegroups.com Message-Id: In-Reply-To: References: <5327B9B7.3050103@gmail.com> <2610F490C952470C9D15999550F67068@multiplay.co.uk> <532A192A.1070509@gmail.com> Subject: Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues MIME-Version: 1.0 X-Google-IP: 79.136.43.145 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: sean@chittenden.org, Steven Hartland , freebsd-performance@freebsd.org, Petr Janda , adrian@FreeBSD.org X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2014 22:52:38 -0000 Den torsdagen den 20:e mars 2014 kl. 00:33:10 UTC+1 skrev Sean Chittenden: > > > As far as I know, the test was done on both UFS2 and ZFS and the > > difference was marginal. > > As Adrian pointed out, there is an mmap(2) mutex in the way. Starting in > PostgreSQL 9.3, shared buffers are allocated out of mmap(2) instead of shm. > shm is only used to notify the PostgreSQL postmaster that a child process > exited/crashed (when a pid detaches from a shm segment, there is a kernel > event, but there is no kernel event when detaching from an mmap(2) region). > -sc > > http://www.postgresql.org/docs/9.3/static/release-9-3.html#AEN115039 > > > >>> Just want to share these pgbench results done by DragonFlyBSD, and > would > >>> like some input on why these numbers look so bad and what can be done > to > >>> improve (ie. kernel tunables etc) the performance. > >>> > >>> > http://lists.dragonflybsd.org/pipermail/users/attachments/20140310/4250b961/attachment-0001.pdf > >>> > >> > >> > >> Do you have the ability to test with FreeBSD 8.x and 9.x to see if this > is > >> regression? > >> > >> Also you don't mention the FS used in each case, so I'm wondering if > you > >> used a ZFS install of FreeBSD which could help to explain things. > > > -- > Sean Chittenden > se...@chittenden.org > > > Hi, There is a fresh thread about this in postgresql-hackers [1]. There are two parallel approaches suggested there, where one is to have an option to continue using the old SYSV shared memory in PostgreSQL, and the other is the suggestion that "somebody needs to hold the FreeBSD folks' feet to the fire about when we can expect to see a fix from their side." Looking at the original post in this thread, it seems to me that FreeBSD has scalability problems beyond what the SYSV vs mmap change in PostgreSQL introduces? Check my test of PostgreSQL 9.2 vs 9.3 on FreeBSD 10.0 at [1]. The difference between PG92 and PG93 is not huge, ~17%. The difference between FreeBSD and the other OS:es in this thread's original post's performance chart seems to be about a lot more? Palle [1] http://www.postgresql.org/message-id/2AE143D2-87D3-4AD1-AC78-CE2258230C05@FreeBSD.org From owner-freebsd-performance@FreeBSD.ORG Tue Apr 22 23:04:24 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 23C0FBDE for ; Tue, 22 Apr 2014 23:04:24 +0000 (UTC) Received: from mail-qc0-x22f.google.com (mail-qc0-x22f.google.com [IPv6:2607:f8b0:400d:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D33F11838 for ; Tue, 22 Apr 2014 23:04:23 +0000 (UTC) Received: by mail-qc0-f175.google.com with SMTP id e16so171372qcx.34 for ; Tue, 22 Apr 2014 16:04:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=xi8DS/coqaM3xkcYDJzbnw/CHswvIVC883Zjc3JZjs8=; b=O14cmk0CDZzQ8whJxXKQL3pYLBBP9trPIXGiRq+F73Ltra4dEg/Sgi1P0+sEUe7ce8 TxrDPZ6qRtFYuCofIw6obH4UHyJrLE0Hkk6ighKTnY8zT0gRWU/7Qsr8Y1H16KlWXomt PhF57GvYB8myhZZdSG5LdYEyWZGEGUuUrFDwRqlLDkpDMcKEDTYNCeEJbdqQ/JiNap3J 7yxEyTHyZSUOgwBaPZjknaQ6BYPx7FNO0AdeCNms9m21Ad4KSw1XhReus5BRL9jW/aXs XeGLfdjuHRZnbkwciam4nMdoI1vTt/symphewOhQ2YTod8NecaWSVRB1FvXYCh4dV/Ym u2Eg== MIME-Version: 1.0 X-Received: by 10.224.38.138 with SMTP id b10mr15617871qae.98.1398207863014; Tue, 22 Apr 2014 16:04:23 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Tue, 22 Apr 2014 16:04:22 -0700 (PDT) In-Reply-To: References: <5327B9B7.3050103@gmail.com> <2610F490C952470C9D15999550F67068@multiplay.co.uk> <532A192A.1070509@gmail.com> Date: Tue, 22 Apr 2014 16:04:22 -0700 X-Google-Sender-Auth: 3wgQo0leiwD8tQE820ab9VRQrq4 Message-ID: Subject: Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues From: Adrian Chadd To: Palle Girgensohn Content-Type: text/plain; charset=UTF-8 Cc: bsdmailinglist@googlegroups.com, FreeBSD Mailing Lists , Sean Chittenden , Petr Janda , Steven Hartland X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2014 23:04:24 -0000 Hi, Are you able to repeat these tests (for both 9.2 and 9.3) whilst grabbing some performance data from lock profiling and hwpmc? The benchmarking is great but it doesn't tell us enough information as to "why" things behave poorly compared to Linux and why the mmap drop isn't so great. What about with more clients? 64? 128? 256? Thanks! -a On 21 April 2014 14:11, Palle Girgensohn wrote: > > > Den torsdagen den 20:e mars 2014 kl. 00:33:10 UTC+1 skrev Sean Chittenden: >> >> > As far as I know, the test was done on both UFS2 and ZFS and the >> > difference was marginal. >> >> As Adrian pointed out, there is an mmap(2) mutex in the way. Starting in >> PostgreSQL 9.3, shared buffers are allocated out of mmap(2) instead of shm. >> shm is only used to notify the PostgreSQL postmaster that a child process >> exited/crashed (when a pid detaches from a shm segment, there is a kernel >> event, but there is no kernel event when detaching from an mmap(2) region). >> -sc >> >> http://www.postgresql.org/docs/9.3/static/release-9-3.html#AEN115039 >> >> >> >>> Just want to share these pgbench results done by DragonFlyBSD, and >> would >> >>> like some input on why these numbers look so bad and what can be done >> to >> >>> improve (ie. kernel tunables etc) the performance. >> >>> >> >>> >> http://lists.dragonflybsd.org/pipermail/users/attachments/20140310/4250b961/attachment-0001.pdf >> >>> >> >> >> >> >> >> Do you have the ability to test with FreeBSD 8.x and 9.x to see if this >> is >> >> regression? >> >> >> >> Also you don't mention the FS used in each case, so I'm wondering if >> you >> >> used a ZFS install of FreeBSD which could help to explain things. >> >> >> -- >> Sean Chittenden >> se...@chittenden.org >> >> >> > Hi, > > There is a fresh thread about this in postgresql-hackers [1]. > > There are two parallel approaches suggested there, where one is to have an > option to continue using the old SYSV shared memory in PostgreSQL, and the > other is the suggestion that "somebody needs to hold the FreeBSD folks' > feet to the fire about when we can expect to see a fix from their side." > > Looking at the original post in this thread, it seems to me that FreeBSD > has scalability problems beyond what the SYSV vs mmap change in PostgreSQL > introduces? Check my test of PostgreSQL 9.2 vs 9.3 on FreeBSD 10.0 at [1]. > The difference between PG92 and PG93 is not huge, ~17%. The difference > between FreeBSD and the other OS:es in this thread's original post's > performance chart seems to be about a lot more? > > Palle > > [1] > http://www.postgresql.org/message-id/2AE143D2-87D3-4AD1-AC78-CE2258230C05@FreeBSD.org > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" From owner-freebsd-performance@FreeBSD.ORG Wed Apr 23 06:53:05 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 685F6246; Wed, 23 Apr 2014 06:53:05 +0000 (UTC) Received: from melon.pingpong.net (melon.pingpong.net [79.136.116.200]) by mx1.freebsd.org (Postfix) with ESMTP id EADFB17B8; Wed, 23 Apr 2014 06:53:04 +0000 (UTC) Received: from [10.19.58.119] (c-5eeaaaae-74736162.cust.telenor.se [94.234.170.174]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by melon.pingpong.net (Postfix) with ESMTPSA id 010AA34753; Wed, 23 Apr 2014 08:52:57 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues From: Palle Girgensohn X-Mailer: iPhone Mail (11D167) In-Reply-To: Date: Wed, 23 Apr 2014 08:52:57 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <572540F9-13E4-4BA9-88AE-5F47FB19450A@pingpong.net> References: <5327B9B7.3050103@gmail.com> <2610F490C952470C9D15999550F67068@multiplay.co.uk> <532A192A.1070509@gmail.com> To: Adrian Chadd Cc: "bsdmailinglist@googlegroups.com" , FreeBSD Mailing Lists , Sean Chittenden , Petr Janda , Steven Hartland X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Apr 2014 06:53:05 -0000 > 23 apr 2014 kl. 01:04 skrev Adrian Chadd : >=20 > Hi, >=20 > Are you able to repeat these tests (for both 9.2 and 9.3) whilst > grabbing some performance data from lock profiling and hwpmc? I sure can, but I'd love some pointers as to how this is done. Please? :-) >=20 > The benchmarking is great but it doesn't tell us enough information as > to "why" things behave poorly compared to Linux and why the mmap drop > isn't so great. As per the discussion on postresql-hackers, the regression between pg9.2 and= pg9.3, which includes the sysv->mmap shift, *might* also exist, at least pa= rtly, on Linux as well. The initial post in *this* thread does however indicate that freebsd perform= s poorer than Linux and dragonflybsd, but does not really compare PostgreSQL= versions. Just so we're not pursuing the wrong problem here, let's be open minded abou= t the definition of the problem. :-) >=20 > What about with more clients? 64? 128? 256? My test went to 80. I can go higher as well, though other sources say 50 is a= reasonable limit for PostgreSQL.=20 Palle=20 >=20 >=20 > Thanks! >=20 >=20 >=20 > -a >=20 >=20 >> On 21 April 2014 14:11, Palle Girgensohn wrote: >>=20 >>=20 >>> Den torsdagen den 20:e mars 2014 kl. 00:33:10 UTC+1 skrev Sean Chittende= n: >>>=20 >>>> As far as I know, the test was done on both UFS2 and ZFS and the >>>> difference was marginal. >>>=20 >>> As Adrian pointed out, there is an mmap(2) mutex in the way. Starting in= >>> PostgreSQL 9.3, shared buffers are allocated out of mmap(2) instead of s= hm. >>> shm is only used to notify the PostgreSQL postmaster that a child proces= s >>> exited/crashed (when a pid detaches from a shm segment, there is a kerne= l >>> event, but there is no kernel event when detaching from an mmap(2) regio= n). >>> -sc >>>=20 >>> http://www.postgresql.org/docs/9.3/static/release-9-3.html#AEN115039 >>>=20 >>>=20 >>>>>> Just want to share these pgbench results done by DragonFlyBSD, and >>> would >>>>>> like some input on why these numbers look so bad and what can be done= >>> to >>>>>> improve (ie. kernel tunables etc) the performance. >>> http://lists.dragonflybsd.org/pipermail/users/attachments/20140310/4250b= 961/attachment-0001.pdf >>>>>=20 >>>>>=20 >>>>> Do you have the ability to test with FreeBSD 8.x and 9.x to see if thi= s >>> is >>>>> regression? >>>>>=20 >>>>> Also you don't mention the FS used in each case, so I'm wondering if >>> you >>>>> used a ZFS install of FreeBSD which could help to explain things. >>>=20 >>>=20 >>> -- >>> Sean Chittenden >>> se...@chittenden.org >> Hi, >>=20 >> There is a fresh thread about this in postgresql-hackers [1]. >>=20 >> There are two parallel approaches suggested there, where one is to have a= n >> option to continue using the old SYSV shared memory in PostgreSQL, and th= e >> other is the suggestion that "somebody needs to hold the FreeBSD folks' >> feet to the fire about when we can expect to see a fix from their side." >>=20 >> Looking at the original post in this thread, it seems to me that FreeBSD >> has scalability problems beyond what the SYSV vs mmap change in PostgreSQ= L >> introduces? Check my test of PostgreSQL 9.2 vs 9.3 on FreeBSD 10.0 at [1]= . >> The difference between PG92 and PG93 is not huge, ~17%. The difference >> between FreeBSD and the other OS:es in this thread's original post's >> performance chart seems to be about a lot more? >>=20 >> Palle >>=20 >> [1] >> http://www.postgresql.org/message-id/2AE143D2-87D3-4AD1-AC78-CE2258230C05= @FreeBSD.org >> _______________________________________________ >> freebsd-performance@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >> To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd= .org" From owner-freebsd-performance@FreeBSD.ORG Wed May 21 00:14:33 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3DD3DBBC; Wed, 21 May 2014 00:14:33 +0000 (UTC) Received: from mail-la0-x232.google.com (mail-la0-x232.google.com [IPv6:2a00:1450:4010:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 75C5B2260; Wed, 21 May 2014 00:14:32 +0000 (UTC) Received: by mail-la0-f50.google.com with SMTP id b8so1003685lan.9 for ; Tue, 20 May 2014 17:14:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=QnS3juKGBwwY+AhFcO+2hyadp0uVMoT+TIdGWPAX46g=; b=gNcjXnSEzLZzPZq8sU5WbbQT3RRuWwKAFjdXOUn9jFIW8Sm9RCZ+pvoPCwXoC73uLb +/wX7Kczen+MudEYGNKqWlbnZnLLsQ0vQMy8MLHQfCIze2ZpMfw7Jz/MXuM4VCJPFe72 66k9sB+1/of1ASHzkWtTYe13Z1tm9kfprqggRmnjb+jJ33MwAeVHaY6sIdMwkDp4Gbgq SRXlbmBEsHG2kXYmveW52MbQR30kdmDmL6n8ruDu9cu0tvlbvG4mKwvJ8bdpTfzOjQKc 8JVVqRix41RnuWlMfezjA28AlR7Bw1ZlQj/amPxJcgkuBmXmISAnNbAfjKH+qUZrmFf9 jjBA== X-Received: by 10.152.87.52 with SMTP id u20mr10312152laz.52.1400631270344; Tue, 20 May 2014 17:14:30 -0700 (PDT) MIME-Version: 1.0 Received: by 10.115.4.4 with HTTP; Tue, 20 May 2014 17:13:50 -0700 (PDT) In-Reply-To: <572540F9-13E4-4BA9-88AE-5F47FB19450A@pingpong.net> References: <5327B9B7.3050103@gmail.com> <2610F490C952470C9D15999550F67068@multiplay.co.uk> <532A192A.1070509@gmail.com> <572540F9-13E4-4BA9-88AE-5F47FB19450A@pingpong.net> From: =?UTF-8?Q?Gezeala_M=2E_Bacu=C3=B1o_II?= Date: Tue, 20 May 2014 17:13:50 -0700 Message-ID: Subject: Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues To: Palle Girgensohn Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: Adrian Chadd , "bsdmailinglist@googlegroups.com" , Petr Janda , Steven Hartland , FreeBSD Mailing Lists , Sean Chittenden X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 00:14:33 -0000 Do you guys have any updates on this? -- regards gezeala bacu=C3=B1o II On Tue, Apr 22, 2014 at 11:52 PM, Palle Girgensohn wro= te: > > > > 23 apr 2014 kl. 01:04 skrev Adrian Chadd : > > > > Hi, > > > > Are you able to repeat these tests (for both 9.2 and 9.3) whilst > > grabbing some performance data from lock profiling and hwpmc? > > I sure can, but I'd love some pointers as to how this is done. Please? :-= ) > > > > > The benchmarking is great but it doesn't tell us enough information as > > to "why" things behave poorly compared to Linux and why the mmap drop > > isn't so great. > > > As per the discussion on postresql-hackers, the regression between pg9.2 > and pg9.3, which includes the sysv->mmap shift, *might* also exist, at > least partly, on Linux as well. > > The initial post in *this* thread does however indicate that freebsd > performs poorer than Linux and dragonflybsd, but does not really compare > PostgreSQL versions. > > Just so we're not pursuing the wrong problem here, let's be open minded > about the definition of the problem. :-) > > > > > What about with more clients? 64? 128? 256? > > My test went to 80. I can go higher as well, though other sources say 50 > is a reasonable limit for PostgreSQL. > > Palle > > > > > > > > Thanks! > > > > > > > > -a > > > > > >> On 21 April 2014 14:11, Palle Girgensohn wrote: > >> > >> > >>> Den torsdagen den 20:e mars 2014 kl. 00:33:10 UTC+1 skrev Sean > Chittenden: > >>> > >>>> As far as I know, the test was done on both UFS2 and ZFS and the > >>>> difference was marginal. > >>> > >>> As Adrian pointed out, there is an mmap(2) mutex in the way. Starting > in > >>> PostgreSQL 9.3, shared buffers are allocated out of mmap(2) instead o= f > shm. > >>> shm is only used to notify the PostgreSQL postmaster that a child > process > >>> exited/crashed (when a pid detaches from a shm segment, there is a > kernel > >>> event, but there is no kernel event when detaching from an mmap(2) > region). > >>> -sc > >>> > >>> http://www.postgresql.org/docs/9.3/static/release-9-3.html#AEN115039 > >>> > >>> > >>>>>> Just want to share these pgbench results done by DragonFlyBSD, and > >>> would > >>>>>> like some input on why these numbers look so bad and what can be > done > >>> to > >>>>>> improve (ie. kernel tunables etc) the performance. > >>> > http://lists.dragonflybsd.org/pipermail/users/attachments/20140310/4250b9= 61/attachment-0001.pdf > >>>>> > >>>>> > >>>>> Do you have the ability to test with FreeBSD 8.x and 9.x to see if > this > >>> is > >>>>> regression? > >>>>> > >>>>> Also you don't mention the FS used in each case, so I'm wondering i= f > >>> you > >>>>> used a ZFS install of FreeBSD which could help to explain things. > >>> > >>> > >>> -- > >>> Sean Chittenden > >>> se...@chittenden.org > >> Hi, > >> > >> There is a fresh thread about this in postgresql-hackers [1]. > >> > >> There are two parallel approaches suggested there, where one is to hav= e > an > >> option to continue using the old SYSV shared memory in PostgreSQL, and > the > >> other is the suggestion that "somebody needs to hold the FreeBSD folks= ' > >> feet to the fire about when we can expect to see a fix from their side= ." > >> > >> Looking at the original post in this thread, it seems to me that FreeB= SD > >> has scalability problems beyond what the SYSV vs mmap change in > PostgreSQL > >> introduces? Check my test of PostgreSQL 9.2 vs 9.3 on FreeBSD 10.0 at > [1]. > >> The difference between PG92 and PG93 is not huge, ~17%. The difference > >> between FreeBSD and the other OS:es in this thread's original post's > >> performance chart seems to be about a lot more? > >> > >> Palle > >> > >> [1] > >> > http://www.postgresql.org/message-id/2AE143D2-87D3-4AD1-AC78-CE2258230C05= @FreeBSD.org > >> _______________________________________________ > >> freebsd-performance@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-performance > >> To unsubscribe, send any mail to " > freebsd-performance-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to " > freebsd-performance-unsubscribe@freebsd.org" > From owner-freebsd-performance@FreeBSD.ORG Wed May 21 06:16:14 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1B9E0929; Wed, 21 May 2014 06:16:14 +0000 (UTC) Received: from melon.pingpong.net (melon.pingpong.net [79.136.116.200]) by mx1.freebsd.org (Postfix) with ESMTP id 88EF32E23; Wed, 21 May 2014 06:16:12 +0000 (UTC) Received: from [10.0.1.21] (h-43-145.a357.priv.bahnhof.se [79.136.43.145]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by melon.pingpong.net (Postfix) with ESMTPSA id F13DC3593F; Wed, 21 May 2014 08:16:04 +0200 (CEST) Mime-Version: 1.0 (1.0) Subject: Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues From: Palle Girgensohn X-Mailer: iPhone Mail (11D201) In-Reply-To: Date: Wed, 21 May 2014 08:16:04 +0200 Message-Id: <1BC3D447-2044-4AB8-B183-B83957BC9112@pingpong.net> References: <5327B9B7.3050103@gmail.com> <2610F490C952470C9D15999550F67068@multiplay.co.uk> <532A192A.1070509@gmail.com> <572540F9-13E4-4BA9-88AE-5F47FB19450A@pingpong.net> To: =?utf-8?Q?"Gezeala_M._Bacu=C3=B1o_II"?= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: Adrian Chadd , "bsdmailinglist@googlegroups.com" , Petr Janda , Steven Hartland , FreeBSD Mailing Lists , Sean Chittenden X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 06:16:14 -0000 I got no response about how to grab performance data.=20 The PostgreSQL team is also making an effort by setting up machines dedicate= d to performance measuring and tuning.=20 And freebsd guys and PostgreSQL guys are apparently meeting at pgcon this we= ek.=20 We'll see where that leads.=20 In the mean time, if I for some pointers on how to grab performance data, I c= ould do some more tests.=20 Palle > 21 maj 2014 kl. 02:13 skrev Gezeala M. Bacu=C3=B1o II := >=20 >=20 > Do you guys have any updates on this?=20 >=20 > -- >=20 > regards >=20 > gezeala bacu=C3=B1o II >=20 >=20 >> On Tue, Apr 22, 2014 at 11:52 PM, Palle Girgensohn w= rote: >>=20 >>=20 >> > 23 apr 2014 kl. 01:04 skrev Adrian Chadd : >> > >> > Hi, >> > >> > Are you able to repeat these tests (for both 9.2 and 9.3) whilst >> > grabbing some performance data from lock profiling and hwpmc? >>=20 >> I sure can, but I'd love some pointers as to how this is done. Please? :-= ) >>=20 >> > >> > The benchmarking is great but it doesn't tell us enough information as >> > to "why" things behave poorly compared to Linux and why the mmap drop >> > isn't so great. >>=20 >>=20 >> As per the discussion on postresql-hackers, the regression between pg9.2 a= nd pg9.3, which includes the sysv->mmap shift, *might* also exist, at least p= artly, on Linux as well. >>=20 >> The initial post in *this* thread does however indicate that freebsd perf= orms poorer than Linux and dragonflybsd, but does not really compare Postgre= SQL versions. >>=20 >> Just so we're not pursuing the wrong problem here, let's be open minded a= bout the definition of the problem. :-) >>=20 >> > >> > What about with more clients? 64? 128? 256? >>=20 >> My test went to 80. I can go higher as well, though other sources say 50 i= s a reasonable limit for PostgreSQL. >>=20 >> Palle >>=20 >>=20 >> > >> > >> > Thanks! >> > >> > >> > >> > -a >> > >> > >> >> On 21 April 2014 14:11, Palle Girgensohn wrote: >> >> >> >> >> >>> Den torsdagen den 20:e mars 2014 kl. 00:33:10 UTC+1 skrev Sean Chitte= nden: >> >>> >> >>>> As far as I know, the test was done on both UFS2 and ZFS and the >> >>>> difference was marginal. >> >>> >> >>> As Adrian pointed out, there is an mmap(2) mutex in the way. Starting= in >> >>> PostgreSQL 9.3, shared buffers are allocated out of mmap(2) instead o= f shm. >> >>> shm is only used to notify the PostgreSQL postmaster that a child pro= cess >> >>> exited/crashed (when a pid detaches from a shm segment, there is a ke= rnel >> >>> event, but there is no kernel event when detaching from an mmap(2) re= gion). >> >>> -sc >> >>> >> >>> http://www.postgresql.org/docs/9.3/static/release-9-3.html#AEN115039 >> >>> >> >>> >> >>>>>> Just want to share these pgbench results done by DragonFlyBSD, and= >> >>> would >> >>>>>> like some input on why these numbers look so bad and what can be d= one >> >>> to >> >>>>>> improve (ie. kernel tunables etc) the performance. >> >>> http://lists.dragonflybsd.org/pipermail/users/attachments/20140310/42= 50b961/attachment-0001.pdf >> >>>>> >> >>>>> >> >>>>> Do you have the ability to test with FreeBSD 8.x and 9.x to see if t= his >> >>> is >> >>>>> regression? >> >>>>> >> >>>>> Also you don't mention the FS used in each case, so I'm wondering i= f >> >>> you >> >>>>> used a ZFS install of FreeBSD which could help to explain things. >> >>> >> >>> >> >>> -- >> >>> Sean Chittenden >> >>> se...@chittenden.org >> >> Hi, >> >> >> >> There is a fresh thread about this in postgresql-hackers [1]. >> >> >> >> There are two parallel approaches suggested there, where one is to hav= e an >> >> option to continue using the old SYSV shared memory in PostgreSQL, and= the >> >> other is the suggestion that "somebody needs to hold the FreeBSD folks= ' >> >> feet to the fire about when we can expect to see a fix from their side= ." >> >> >> >> Looking at the original post in this thread, it seems to me that FreeB= SD >> >> has scalability problems beyond what the SYSV vs mmap change in Postgr= eSQL >> >> introduces? Check my test of PostgreSQL 9.2 vs 9.3 on FreeBSD 10.0 at [= 1]. >> >> The difference between PG92 and PG93 is not huge, ~17%. The difference= >> >> between FreeBSD and the other OS:es in this thread's original post's >> >> performance chart seems to be about a lot more? >> >> >> >> Palle >> >> >> >> [1] >> >> http://www.postgresql.org/message-id/2AE143D2-87D3-4AD1-AC78-CE2258230= C05@FreeBSD.org >> >> _______________________________________________ >> >> freebsd-performance@freebsd.org mailing list >> >> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >> >> To unsubscribe, send any mail to "freebsd-performance-unsubscribe@free= bsd.org" >> _______________________________________________ >> freebsd-performance@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >> To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd= .org" >=20 From owner-freebsd-performance@FreeBSD.ORG Wed May 21 16:33:11 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D2B6AEFB; Wed, 21 May 2014 16:33:11 +0000 (UTC) Received: from mail-lb0-x233.google.com (mail-lb0-x233.google.com [IPv6:2a00:1450:4010:c04::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1727F2773; Wed, 21 May 2014 16:33:10 +0000 (UTC) Received: by mail-lb0-f179.google.com with SMTP id c11so1748225lbj.38 for ; Wed, 21 May 2014 09:33:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=I8Wc5WIQO5m5nhQHtymbhQPJ/U3a+PpXfggLiVPuECY=; b=AtIRp84zFmDTohmNSnAYxKd5H1syUeJTZGQEkecHfFOMFlJ1lo7YFOlK60NCtWiOJ8 /glivXH/zkW6kBfBsmkUmBNYXirXQFnb7OgGI06c+zGqPi44TxuRk9bVubM/wJwVNJIk OKlyhizD4TpxD2cdpvy5CnXINTmxZPzCjG6bWsQ9IoHHgf/V5gFNSsZ2KwBboPqHuz9I RfgO1NjSLm0EuBs5HZY1xR2lrh/0toeMRrbUH4H+RpU4gA93vCYE69RHGHCyrL7N60Xp RjV4Etuu8BTdJAZKn6iql6WgFSc1CIjyYe/bbVHd2R9mlIrNY1dG6PFL8v6J5kY4PosC FiMw== MIME-Version: 1.0 X-Received: by 10.152.19.195 with SMTP id h3mr19073182lae.47.1400689988290; Wed, 21 May 2014 09:33:08 -0700 (PDT) Received: by 10.115.4.4 with HTTP; Wed, 21 May 2014 09:33:08 -0700 (PDT) Received: by 10.115.4.4 with HTTP; Wed, 21 May 2014 09:33:08 -0700 (PDT) In-Reply-To: <1BC3D447-2044-4AB8-B183-B83957BC9112@pingpong.net> References: <5327B9B7.3050103@gmail.com> <2610F490C952470C9D15999550F67068@multiplay.co.uk> <532A192A.1070509@gmail.com> <572540F9-13E4-4BA9-88AE-5F47FB19450A@pingpong.net> <1BC3D447-2044-4AB8-B183-B83957BC9112@pingpong.net> Date: Wed, 21 May 2014 09:33:08 -0700 Message-ID: Subject: Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues From: =?UTF-8?Q?Gezeala_M=2E_Bacu=C3=B1o_II?= To: Palle Girgensohn Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: Adrian Chadd , "bsdmailinglist@googlegroups.com" , Petr Janda , Steven Hartland , FreeBSD Mailing Lists , Sean Chittenden X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 16:33:11 -0000 Gotcha. I've been testing using pgbench on FreeBSD 9.0 release + ZFS + pg 9.3.. I can reach the freebsd 10 stats on the pdf files if the dataset RAM. Test was done on pools without L2ARC and with/without compression. I also remember increasing a vm.pmap sysctl. I'm out of the office right now sick so I can't provide the stats but yes, with mmap it is pretty bad.. Keep us in the loop. I'd like to help on getting the performance data they need. On May 20, 2014 11:16 PM, "Palle Girgensohn" wrote: > I got no response about how to grab performance data. > > The PostgreSQL team is also making an effort by setting up machines > dedicated to performance measuring and tuning. > > And freebsd guys and PostgreSQL guys are apparently meeting at pgcon this > week. > > We'll see where that leads. > > In the mean time, if I for some pointers on how to grab performance data, > I could do some more tests. > > Palle > > 21 maj 2014 kl. 02:13 skrev Gezeala M. Bacu=C3=B1o II = : > > > Do you guys have any updates on this? > > -- > > regards > > gezeala bacu=C3=B1o II > > > On Tue, Apr 22, 2014 at 11:52 PM, Palle Girgensohn w= rote: > >> >> >> > 23 apr 2014 kl. 01:04 skrev Adrian Chadd : >> > >> > Hi, >> > >> > Are you able to repeat these tests (for both 9.2 and 9.3) whilst >> > grabbing some performance data from lock profiling and hwpmc? >> >> I sure can, but I'd love some pointers as to how this is done. Please? := -) >> >> > >> > The benchmarking is great but it doesn't tell us enough information as >> > to "why" things behave poorly compared to Linux and why the mmap drop >> > isn't so great. >> >> >> As per the discussion on postresql-hackers, the regression between pg9.2 >> and pg9.3, which includes the sysv->mmap shift, *might* also exist, at >> least partly, on Linux as well. >> >> The initial post in *this* thread does however indicate that freebsd >> performs poorer than Linux and dragonflybsd, but does not really compare >> PostgreSQL versions. >> >> Just so we're not pursuing the wrong problem here, let's be open minded >> about the definition of the problem. :-) >> >> > >> > What about with more clients? 64? 128? 256? >> >> My test went to 80. I can go higher as well, though other sources say 50 >> is a reasonable limit for PostgreSQL. >> >> Palle >> >> >> > >> > >> > Thanks! >> > >> > >> > >> > -a >> > >> > >> >> On 21 April 2014 14:11, Palle Girgensohn wrote: >> >> >> >> >> >>> Den torsdagen den 20:e mars 2014 kl. 00:33:10 UTC+1 skrev Sean >> Chittenden: >> >>> >> >>>> As far as I know, the test was done on both UFS2 and ZFS and the >> >>>> difference was marginal. >> >>> >> >>> As Adrian pointed out, there is an mmap(2) mutex in the way. Startin= g >> in >> >>> PostgreSQL 9.3, shared buffers are allocated out of mmap(2) instead >> of shm. >> >>> shm is only used to notify the PostgreSQL postmaster that a child >> process >> >>> exited/crashed (when a pid detaches from a shm segment, there is a >> kernel >> >>> event, but there is no kernel event when detaching from an mmap(2) >> region). >> >>> -sc >> >>> >> >>> http://www.postgresql.org/docs/9.3/static/release-9-3.html#AEN115039 >> >>> >> >>> >> >>>>>> Just want to share these pgbench results done by DragonFlyBSD, an= d >> >>> would >> >>>>>> like some input on why these numbers look so bad and what can be >> done >> >>> to >> >>>>>> improve (ie. kernel tunables etc) the performance. >> >>> >> http://lists.dragonflybsd.org/pipermail/users/attachments/20140310/4250b= 961/attachment-0001.pdf >> >>>>> >> >>>>> >> >>>>> Do you have the ability to test with FreeBSD 8.x and 9.x to see if >> this >> >>> is >> >>>>> regression? >> >>>>> >> >>>>> Also you don't mention the FS used in each case, so I'm wondering = if >> >>> you >> >>>>> used a ZFS install of FreeBSD which could help to explain things. >> >>> >> >>> >> >>> -- >> >>> Sean Chittenden >> >>> se...@chittenden.org >> >> Hi, >> >> >> >> There is a fresh thread about this in postgresql-hackers [1]. >> >> >> >> There are two parallel approaches suggested there, where one is to >> have an >> >> option to continue using the old SYSV shared memory in PostgreSQL, an= d >> the >> >> other is the suggestion that "somebody needs to hold the FreeBSD folk= s' >> >> feet to the fire about when we can expect to see a fix from their >> side." >> >> >> >> Looking at the original post in this thread, it seems to me that >> FreeBSD >> >> has scalability problems beyond what the SYSV vs mmap change in >> PostgreSQL >> >> introduces? Check my test of PostgreSQL 9.2 vs 9.3 on FreeBSD 10.0 at >> [1]. >> >> The difference between PG92 and PG93 is not huge, ~17%. The differenc= e >> >> between FreeBSD and the other OS:es in this thread's original post's >> >> performance chart seems to be about a lot more? >> >> >> >> Palle >> >> >> >> [1] >> >> >> http://www.postgresql.org/message-id/2AE143D2-87D3-4AD1-AC78-CE2258230C0= 5@FreeBSD.org >> >> _______________________________________________ >> >> freebsd-performance@freebsd.org mailing list >> >> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >> >> To unsubscribe, send any mail to " >> freebsd-performance-unsubscribe@freebsd.org" >> _______________________________________________ >> freebsd-performance@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >> To unsubscribe, send any mail to " >> freebsd-performance-unsubscribe@freebsd.org" >> > > From owner-freebsd-performance@FreeBSD.ORG Wed May 21 16:50:54 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7FAE56DE; Wed, 21 May 2014 16:50:54 +0000 (UTC) Received: from melon.pingpong.net (melon.pingpong.net [79.136.116.200]) by mx1.freebsd.org (Postfix) with ESMTP id 2866A28DD; Wed, 21 May 2014 16:50:53 +0000 (UTC) Received: from [10.19.39.26] (c-5eeaaaa2-74736162.cust.telenor.se [94.234.170.162]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by melon.pingpong.net (Postfix) with ESMTPSA id 9EA7F378E0; Wed, 21 May 2014 18:50:51 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (1.0) Subject: Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues From: Palle Girgensohn X-Mailer: iPhone Mail (11D201) In-Reply-To: Date: Wed, 21 May 2014 18:50:50 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <1473AF7C-B190-4CD4-B611-BA4090A081CB@pingpong.net> References: <5327B9B7.3050103@gmail.com> <2610F490C952470C9D15999550F67068@multiplay.co.uk> <532A192A.1070509@gmail.com> <572540F9-13E4-4BA9-88AE-5F47FB19450A@pingpong.net> <1BC3D447-2044-4AB8-B183-B83957BC9112@pingpong.net> To: =?utf-8?Q?"Gezeala_M._Bacu=C3=B1o_II"?= Cc: Adrian Chadd , "bsdmailinglist@googlegroups.com" , Petr Janda , Steven Hartland , FreeBSD Mailing Lists , Sean Chittenden X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 16:50:54 -0000 I did some tests with zfs, and results where appallingly bad, but that was w= ith db size > ram.=20 I think the model used by PostgreSQL, as most databases, are very disk block= centric. Using zfs makes it hard to get good performance. But this was some= time ago, maybe things have improved.=20 Palle > 21 maj 2014 kl. 18:33 skrev Gezeala M. Bacu=C3=B1o II := >=20 > Gotcha. >=20 > I've been testing using pgbench on FreeBSD 9.0 release + ZFS + pg 9.3.. I > can reach the freebsd 10 stats on the pdf files if the dataset gets way lower if the dataset >RAM. Test was done on pools without L2ARC > and with/without compression. I also remember increasing a vm.pmap sysctl.= > I'm out of the office right now sick so I can't provide the stats but yes,= > with mmap it is pretty bad.. >=20 > Keep us in the loop. I'd like to help on getting the performance data they= > need. >> On May 20, 2014 11:16 PM, "Palle Girgensohn" wrote:= >>=20 >> I got no response about how to grab performance data. >>=20 >> The PostgreSQL team is also making an effort by setting up machines >> dedicated to performance measuring and tuning. >>=20 >> And freebsd guys and PostgreSQL guys are apparently meeting at pgcon this= >> week. >>=20 >> We'll see where that leads. >>=20 >> In the mean time, if I for some pointers on how to grab performance data,= >> I could do some more tests. >>=20 >> Palle >>=20 >> 21 maj 2014 kl. 02:13 skrev Gezeala M. Bacu=C3=B1o II = : >>=20 >>=20 >> Do you guys have any updates on this? >>=20 >> -- >>=20 >> regards >>=20 >> gezeala bacu=C3=B1o II >>=20 >>=20 >> On Tue, Apr 22, 2014 at 11:52 PM, Palle Girgensohn w= rote: >>=20 >>>=20 >>>=20 >>>> 23 apr 2014 kl. 01:04 skrev Adrian Chadd : >>>>=20 >>>> Hi, >>>>=20 >>>> Are you able to repeat these tests (for both 9.2 and 9.3) whilst >>>> grabbing some performance data from lock profiling and hwpmc? >>>=20 >>> I sure can, but I'd love some pointers as to how this is done. Please? := -) >>>=20 >>>>=20 >>>> The benchmarking is great but it doesn't tell us enough information as >>>> to "why" things behave poorly compared to Linux and why the mmap drop >>>> isn't so great. >>>=20 >>>=20 >>> As per the discussion on postresql-hackers, the regression between pg9.2= >>> and pg9.3, which includes the sysv->mmap shift, *might* also exist, at >>> least partly, on Linux as well. >>>=20 >>> The initial post in *this* thread does however indicate that freebsd >>> performs poorer than Linux and dragonflybsd, but does not really compare= >>> PostgreSQL versions. >>>=20 >>> Just so we're not pursuing the wrong problem here, let's be open minded >>> about the definition of the problem. :-) >>>=20 >>>>=20 >>>> What about with more clients? 64? 128? 256? >>>=20 >>> My test went to 80. I can go higher as well, though other sources say 50= >>> is a reasonable limit for PostgreSQL. >>>=20 >>> Palle >>>=20 >>>=20 >>>>=20 >>>>=20 >>>> Thanks! >>>>=20 >>>>=20 >>>>=20 >>>> -a >>>>=20 >>>>=20 >>>>> On 21 April 2014 14:11, Palle Girgensohn wrote: >>>>>=20 >>>>>=20 >>>>>> Den torsdagen den 20:e mars 2014 kl. 00:33:10 UTC+1 skrev Sean >>> Chittenden: >>>>>>=20 >>>>>>> As far as I know, the test was done on both UFS2 and ZFS and the >>>>>>> difference was marginal. >>>>>>=20 >>>>>> As Adrian pointed out, there is an mmap(2) mutex in the way. Starting= >>> in >>>>>> PostgreSQL 9.3, shared buffers are allocated out of mmap(2) instead >>> of shm. >>>>>> shm is only used to notify the PostgreSQL postmaster that a child >>> process >>>>>> exited/crashed (when a pid detaches from a shm segment, there is a >>> kernel >>>>>> event, but there is no kernel event when detaching from an mmap(2) >>> region). >>>>>> -sc >>>>>>=20 >>>>>> http://www.postgresql.org/docs/9.3/static/release-9-3.html#AEN115039 >>>>>>=20 >>>>>>=20 >>>>>>>>> Just want to share these pgbench results done by DragonFlyBSD, and= >>>>>> would >>>>>>>>> like some input on why these numbers look so bad and what can be >>> done >>>>>> to >>>>>>>>> improve (ie. kernel tunables etc) the performance. >>> http://lists.dragonflybsd.org/pipermail/users/attachments/20140310/4250b= 961/attachment-0001.pdf >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> Do you have the ability to test with FreeBSD 8.x and 9.x to see if >>> this >>>>>> is >>>>>>>> regression? >>>>>>>>=20 >>>>>>>> Also you don't mention the FS used in each case, so I'm wondering i= f >>>>>> you >>>>>>>> used a ZFS install of FreeBSD which could help to explain things. >>>>>>=20 >>>>>>=20 >>>>>> -- >>>>>> Sean Chittenden >>>>>> se...@chittenden.org >>>>> Hi, >>>>>=20 >>>>> There is a fresh thread about this in postgresql-hackers [1]. >>>>>=20 >>>>> There are two parallel approaches suggested there, where one is to >>> have an >>>>> option to continue using the old SYSV shared memory in PostgreSQL, and= >>> the >>>>> other is the suggestion that "somebody needs to hold the FreeBSD folks= ' >>>>> feet to the fire about when we can expect to see a fix from their >>> side." >>>>>=20 >>>>> Looking at the original post in this thread, it seems to me that >>> FreeBSD >>>>> has scalability problems beyond what the SYSV vs mmap change in >>> PostgreSQL >>>>> introduces? Check my test of PostgreSQL 9.2 vs 9.3 on FreeBSD 10.0 at >>> [1]. >>>>> The difference between PG92 and PG93 is not huge, ~17%. The difference= >>>>> between FreeBSD and the other OS:es in this thread's original post's >>>>> performance chart seems to be about a lot more? >>>>>=20 >>>>> Palle >>>>>=20 >>>>> [1] >>> http://www.postgresql.org/message-id/2AE143D2-87D3-4AD1-AC78-CE2258230C0= 5@FreeBSD.org >>>>> _______________________________________________ >>>>> freebsd-performance@freebsd.org mailing list >>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >>>>> To unsubscribe, send any mail to " >>> freebsd-performance-unsubscribe@freebsd.org" >>> _______________________________________________ >>> freebsd-performance@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >>> To unsubscribe, send any mail to " >>> freebsd-performance-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.= org" From owner-freebsd-performance@FreeBSD.ORG Wed May 21 17:17:48 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AA70C108; Wed, 21 May 2014 17:17:48 +0000 (UTC) Received: from mail01.lax1.stackjet.com (mon01.lax1.stackjet.com [174.136.104.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7D24D2B4F; Wed, 21 May 2014 17:17:47 +0000 (UTC) Received: from hormesis.group.on (localhost [127.0.0.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: sean@chittenden.org) by mail01.lax1.stackjet.com (Postfix) with ESMTPSA id A9D1C3E8DDB; Wed, 21 May 2014 10:17:38 -0700 (PDT) Received: from hormesis.group.on ([64.125.69.70] helo=hormesis.group.on) by ASSP.nospam with SMTPS(ECDHE-RSA-AES256-SHA) (2.4.2); 21 May 2014 10:17:35 -0700 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues From: Sean Chittenden In-Reply-To: <1473AF7C-B190-4CD4-B611-BA4090A081CB@pingpong.net> Date: Wed, 21 May 2014 10:17:37 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <5327B9B7.3050103@gmail.com> <2610F490C952470C9D15999550F67068@multiplay.co.uk> <532A192A.1070509@gmail.com> <572540F9-13E4-4BA9-88AE-5F47FB19450A@pingpong.net> <1BC3D447-2044-4AB8-B183-B83957BC9112@pingpong.net> <1473AF7C-B190-4CD4-B611-BA4090A081CB@pingpong.net> To: Palle Girgensohn X-Mailer: Apple Mail (2.1878.2) X-Assp-Version: 2.4.2(14097) on ASSP.nospam X-Assp-ID: ASSP.nospam m1-92658-01683 X-Assp-Session: 8437F81F8 (mail 1) X-Assp-Envelope-From: sean@chittenden.org X-Assp-Intended-For: girgen@pingpong.net X-Assp-Intended-For: gezeala@gmail.com X-Assp-Intended-For: adrian@freebsd.org X-Assp-Intended-For: bsdmailinglist@googlegroups.com X-Assp-Intended-For: janda.petr@gmail.com X-Assp-Intended-For: killing@multiplay.co.uk X-Assp-Intended-For: freebsd-performance@freebsd.org X-Assp-Client-TLS: yes X-Assp-Server-TLS: yes Cc: Adrian Chadd , "bsdmailinglist@googlegroups.com" , Steven Hartland , FreeBSD Mailing Lists , Petr Janda X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 17:17:48 -0000 > I did some tests with zfs, and results where appallingly bad, but that = was with db size > ram.=20 >=20 > I think the model used by PostgreSQL, as most databases, are very disk = block centric. Using zfs makes it hard to get good performance. But this = was some time ago, maybe things have improved.=20 I have some hardware that I ran with last week wherein I was *not* able = to reproduce any performance difference between ZFS and UFS2. On both = UFS2 and ZFS I was seeing the same performance when using a a RAID10 / = set of mirrors. I talked with the Dragonfly folk who originally = performed these tests and they also saw the same thing: no real = performance difference between ZFS and UFS. I ran my tests on a host = with 16 drive, 10K SAS, 192GB RAM. I also created a kernel profiling = image and ran the 20 concurrent user test under kgprof(1), dtrace, and = pmcstat and have the results available: http://people.freebsd.org/~seanc/pg9.3-fbsd10-profiling/ There are some investigations that are ongoing as a result of these = findings. The dfly methodology was observed when generating these = results. Stay tuned. -sc -- Sean Chittenden sean@chittenden.org >=20 > Palle >=20 >> 21 maj 2014 kl. 18:33 skrev Gezeala M. Bacu=F1o II = : >>=20 >> Gotcha. >>=20 >> I've been testing using pgbench on FreeBSD 9.0 release + ZFS + pg = 9.3.. I >> can reach the freebsd 10 stats on the pdf files if the dataset > gets way lower if the dataset >RAM. Test was done on pools without = L2ARC >> and with/without compression. I also remember increasing a vm.pmap = sysctl. >> I'm out of the office right now sick so I can't provide the stats but = yes, >> with mmap it is pretty bad.. >>=20 >> Keep us in the loop. I'd like to help on getting the performance data = they >> need. >>> On May 20, 2014 11:16 PM, "Palle Girgensohn" = wrote: >>>=20 >>> I got no response about how to grab performance data. >>>=20 >>> The PostgreSQL team is also making an effort by setting up machines >>> dedicated to performance measuring and tuning. >>>=20 >>> And freebsd guys and PostgreSQL guys are apparently meeting at pgcon = this >>> week. >>>=20 >>> We'll see where that leads. >>>=20 >>> In the mean time, if I for some pointers on how to grab performance = data, >>> I could do some more tests. >>>=20 >>> Palle >>>=20 >>> 21 maj 2014 kl. 02:13 skrev Gezeala M. Bacu=F1o II = : >>>=20 >>>=20 >>> Do you guys have any updates on this? >>>=20 >>> -- >>>=20 >>> regards >>>=20 >>> gezeala bacu=F1o II >>>=20 >>>=20 >>> On Tue, Apr 22, 2014 at 11:52 PM, Palle Girgensohn = wrote: >>>=20 >>>>=20 >>>>=20 >>>>> 23 apr 2014 kl. 01:04 skrev Adrian Chadd : >>>>>=20 >>>>> Hi, >>>>>=20 >>>>> Are you able to repeat these tests (for both 9.2 and 9.3) whilst >>>>> grabbing some performance data from lock profiling and hwpmc? >>>>=20 >>>> I sure can, but I'd love some pointers as to how this is done. = Please? :-) >>>>=20 >>>>>=20 >>>>> The benchmarking is great but it doesn't tell us enough = information as >>>>> to "why" things behave poorly compared to Linux and why the mmap = drop >>>>> isn't so great. >>>>=20 >>>>=20 >>>> As per the discussion on postresql-hackers, the regression between = pg9.2 >>>> and pg9.3, which includes the sysv->mmap shift, *might* also exist, = at >>>> least partly, on Linux as well. >>>>=20 >>>> The initial post in *this* thread does however indicate that = freebsd >>>> performs poorer than Linux and dragonflybsd, but does not really = compare >>>> PostgreSQL versions. >>>>=20 >>>> Just so we're not pursuing the wrong problem here, let's be open = minded >>>> about the definition of the problem. :-) >>>>=20 >>>>>=20 >>>>> What about with more clients? 64? 128? 256? >>>>=20 >>>> My test went to 80. I can go higher as well, though other sources = say 50 >>>> is a reasonable limit for PostgreSQL. >>>>=20 >>>> Palle >>>>=20 >>>>=20 >>>>>=20 >>>>>=20 >>>>> Thanks! >>>>>=20 >>>>>=20 >>>>>=20 >>>>> -a >>>>>=20 >>>>>=20 >>>>>> On 21 April 2014 14:11, Palle Girgensohn = wrote: >>>>>>=20 >>>>>>=20 >>>>>>> Den torsdagen den 20:e mars 2014 kl. 00:33:10 UTC+1 skrev Sean >>>> Chittenden: >>>>>>>=20 >>>>>>>> As far as I know, the test was done on both UFS2 and ZFS and = the >>>>>>>> difference was marginal. >>>>>>>=20 >>>>>>> As Adrian pointed out, there is an mmap(2) mutex in the way. = Starting >>>> in >>>>>>> PostgreSQL 9.3, shared buffers are allocated out of mmap(2) = instead >>>> of shm. >>>>>>> shm is only used to notify the PostgreSQL postmaster that a = child >>>> process >>>>>>> exited/crashed (when a pid detaches from a shm segment, there is = a >>>> kernel >>>>>>> event, but there is no kernel event when detaching from an = mmap(2) >>>> region). >>>>>>> -sc >>>>>>>=20 >>>>>>> = http://www.postgresql.org/docs/9.3/static/release-9-3.html#AEN115039 >>>>>>>=20 >>>>>>>=20 >>>>>>>>>> Just want to share these pgbench results done by = DragonFlyBSD, and >>>>>>> would >>>>>>>>>> like some input on why these numbers look so bad and what can = be >>>> done >>>>>>> to >>>>>>>>>> improve (ie. kernel tunables etc) the performance. >>>> = http://lists.dragonflybsd.org/pipermail/users/attachments/20140310/4250b96= 1/attachment-0001.pdf >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> Do you have the ability to test with FreeBSD 8.x and 9.x to = see if >>>> this >>>>>>> is >>>>>>>>> regression? >>>>>>>>>=20 >>>>>>>>> Also you don't mention the FS used in each case, so I'm = wondering if >>>>>>> you >>>>>>>>> used a ZFS install of FreeBSD which could help to explain = things. >>>>>>>=20 >>>>>>>=20 >>>>>>> -- >>>>>>> Sean Chittenden >>>>>>> se...@chittenden.org >>>>>> Hi, >>>>>>=20 >>>>>> There is a fresh thread about this in postgresql-hackers [1]. >>>>>>=20 >>>>>> There are two parallel approaches suggested there, where one is = to >>>> have an >>>>>> option to continue using the old SYSV shared memory in = PostgreSQL, and >>>> the >>>>>> other is the suggestion that "somebody needs to hold the FreeBSD = folks' >>>>>> feet to the fire about when we can expect to see a fix from their >>>> side." >>>>>>=20 >>>>>> Looking at the original post in this thread, it seems to me that >>>> FreeBSD >>>>>> has scalability problems beyond what the SYSV vs mmap change in >>>> PostgreSQL >>>>>> introduces? Check my test of PostgreSQL 9.2 vs 9.3 on FreeBSD = 10.0 at >>>> [1]. >>>>>> The difference between PG92 and PG93 is not huge, ~17%. The = difference >>>>>> between FreeBSD and the other OS:es in this thread's original = post's >>>>>> performance chart seems to be about a lot more? >>>>>>=20 >>>>>> Palle >>>>>>=20 >>>>>> [1] >>>> = http://www.postgresql.org/message-id/2AE143D2-87D3-4AD1-AC78-CE2258230C05@= FreeBSD.org >>>>>> _______________________________________________ >>>>>> freebsd-performance@freebsd.org mailing list >>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >>>>>> To unsubscribe, send any mail to " >>>> freebsd-performance-unsubscribe@freebsd.org" >>>> _______________________________________________ >>>> freebsd-performance@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >>>> To unsubscribe, send any mail to " >>>> freebsd-performance-unsubscribe@freebsd.org" >> _______________________________________________ >> freebsd-performance@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >> To unsubscribe, send any mail to = "freebsd-performance-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to = "freebsd-performance-unsubscribe@freebsd.org" From owner-freebsd-performance@FreeBSD.ORG Wed May 21 18:05:08 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 22C6F93F for ; Wed, 21 May 2014 18:05:08 +0000 (UTC) Received: from smtp.infracaninophile.co.uk (smtp6.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3cd3:cd67:fafa:3d78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.infracaninophile.co.uk", Issuer "ca.infracaninophile.co.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id BF18B2FC4 for ; Wed, 21 May 2014 18:05:07 +0000 (UTC) Received: from seedling.black-earth.co.uk (seedling.black-earth.co.uk [81.2.117.99]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.14.8/8.14.8) with ESMTP id s4LI52NF024193 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Wed, 21 May 2014 19:05:03 +0100 (BST) (envelope-from matthew@FreeBSD.org) Authentication-Results: lucid-nonsense.infracaninophile.co.uk; dmarc=none header.from=FreeBSD.org DKIM-Filter: OpenDKIM Filter v2.8.3 smtp.infracaninophile.co.uk s4LI52NF024193 Authentication-Results: smtp.infracaninophile.co.uk/s4LI52NF024193; dkim=none reason="no signature"; dkim-adsp=none Message-ID: <537CEACD.8090701@FreeBSD.org> Date: Wed, 21 May 2014 19:05:01 +0100 From: Matthew Seaman User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: freebsd-performance@freebsd.org Subject: Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues References: <5327B9B7.3050103@gmail.com> <2610F490C952470C9D15999550F67068@multiplay.co.uk> <532A192A.1070509@gmail.com> <572540F9-13E4-4BA9-88AE-5F47FB19450A@pingpong.net> <1BC3D447-2044-4AB8-B183-B83957BC9112@pingpong.net> <1473AF7C-B190-4CD4-B611-BA4090A081CB@pingpong.net> In-Reply-To: X-Enigmail-Version: 1.6 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="OtaoFqCRpb7S0cvx3pgqWm3nsT3B7GROf" X-Virus-Scanned: clamav-milter 0.98.3 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-3.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lucid-nonsense.infracaninophile.co.uk X-Mailman-Approved-At: Wed, 21 May 2014 19:28:42 +0000 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 18:05:08 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --OtaoFqCRpb7S0cvx3pgqWm3nsT3B7GROf Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 21/05/2014 18:17, Sean Chittenden wrote: >> I did some tests with zfs, and results where appallingly bad, but that= was with db size > ram.=20 >> >=20 >> > I think the model used by PostgreSQL, as most databases, are very di= sk block centric. Using zfs makes it hard to get good performance. But th= is was some time ago, maybe things have improved.=20 > I have some hardware that I ran with last week wherein I was *not* able= to reproduce any performance difference between ZFS and UFS2. On both UF= S2 and ZFS I was seeing the same performance when using a a RAID10 / set = of mirrors. I talked with the Dragonfly folk who originally performed the= se tests and they also saw the same thing: no real performance difference= between ZFS and UFS. I ran my tests on a host with 16 drive, 10K SAS, 19= 2GB RAM. I also created a kernel profiling image and ran the 20 concurren= t user test under kgprof(1), dtrace, and pmcstat and have the results ava= ilable: >=20 > http://people.freebsd.org/~seanc/pg9.3-fbsd10-profiling/ >=20 > There are some investigations that are ongoing as a result of these fin= dings. The dfly methodology was observed when generating these results. S= tay tuned. -sc >=20 I'm not sure that the ZFS vs UFS2 question is at the core of the performance problem. We're definitely seeing marked slowdowns between Pg 9.2 and 9.3 on UFS2 (RAID10 + Dell H710p (mfi) raid controller with 1GB NVRAM) In our case the DB size is significantly bigger than RAM, and we also run with a large (3GB) work mem, which seems to exacerbate the slow-down effect. Cheers, Matthew --=20 Dr Matthew J Seaman MA, D.Phil. PGP: http://www.infracaninophile.co.uk/pgpkey --OtaoFqCRpb7S0cvx3pgqWm3nsT3B7GROf Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.20 (Darwin) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQJ8BAEBCgBmBQJTfOrOXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ2NTNBNjhCOTEzQTRFNkNGM0UxRTEzMjZC QjIzQUY1MThFMUE0MDEzAAoJELsjr1GOGkATwOEP/jfYOhm6Tz/YzWPtPFAtrd6F gA5EEZ/KxGOxjPUIGhK3+uK6MhxSsDohO7cLWmz2MJd5PXv6scKOgsW7bvFwC0Lp z6GY5LhfH+x3W0Rx4vtsTWaVLz7QBABkNziiSNcux5sc3vBMn0QMEis6req4smRX Lz63WZSkTbrX63fxbTNwyTa0bd0ySzqCDbOA+ggVOgtxwzVwMC9+Y5YJhddmuWbO O4xXXghUKneYVHeMQ4dn5lsVa0t+3s//PASnwCn2aEK5cDSfGUE3jtfLHyxkgPPq XHnmVfbPZ93rlbKI0JBxK7bOkdmXu0J+DeEl8EadjudoO314Kopn5kJ1JoTs3pPj u+AuMG0xcbSUDzhCbE4a5wpdxg3OfC9gxmVeXfVNTfRo9N+gJdorp2lx9bMT/jXJ bTXgw5ZZWuRlZlBhOgzT6W/ta1lX3ZcNDpsOGrFNw4ASLo0RYf1NViC5oUhR0y3n AtToaBlMObmIavPBb0tdiZH+EWbQdRMwt3+bCveDruMx+xHzgLtvpZeP+GqAwG6M ZfAYha/ypLEUq6jYDNEd+Pl/+5MkTAsMulL6gzLVYn+D/RYTssO8sBTHRLvbHE4E 5appLQhjFbn0ZjKamGboIdqObtpJOln5M7jE+hcXMygtzVMIbApYGqxPZ5j4jL+A ZMW6beR+1pwAK6mqgfdj =p4Wy -----END PGP SIGNATURE----- --OtaoFqCRpb7S0cvx3pgqWm3nsT3B7GROf-- From owner-freebsd-performance@FreeBSD.ORG Wed May 21 20:05:34 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AF39E9B8; Wed, 21 May 2014 20:05:34 +0000 (UTC) Received: from mail01.lax1.stackjet.com (mon01.lax1.stackjet.com [174.136.104.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 886962AFB; Wed, 21 May 2014 20:05:33 +0000 (UTC) Received: from hormesis.group.on (localhost [127.0.0.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: sean@chittenden.org) by mail01.lax1.stackjet.com (Postfix) with ESMTPSA id 488563E8DD8; Wed, 21 May 2014 13:05:30 -0700 (PDT) Received: from hormesis.group.on ([64.125.69.70] helo=hormesis.group.on) by ASSP.nospam with SMTPS(ECDHE-RSA-AES256-SHA)(ECDHE-RSA-AES256-SHA) (2.4.2); 21 May 2014 13:05:04 -0700 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues From: Sean Chittenden In-Reply-To: <537CEACD.8090701@FreeBSD.org> Date: Wed, 21 May 2014 13:05:30 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <5327B9B7.3050103@gmail.com> <2610F490C952470C9D15999550F67068@multiplay.co.uk> <532A192A.1070509@gmail.com> <572540F9-13E4-4BA9-88AE-5F47FB19450A@pingpong.net> <1BC3D447-2044-4AB8-B183-B83957BC9112@pingpong.net> <1473AF7C-B190-4CD4-B611-BA4090A081CB@pingpong.net> <537CEACD.8090701@FreeBSD.org> To: Matthew Seaman X-Mailer: Apple Mail (2.1878.2) X-Assp-Version: 2.4.2(14097) on ASSP.nospam X-Assp-ID: ASSP.nospam m1-02730-01908 X-Assp-Session: 842F2ADF8 (mail 1) X-Assp-Envelope-From: sean@chittenden.org X-Assp-Intended-For: matthew@FreeBSD.org X-Assp-Intended-For: freebsd-performance@freebsd.org X-Assp-Client-TLS: yes X-Assp-Server-TLS: yes Cc: freebsd-performance@freebsd.org X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 20:05:34 -0000 >>> I did some tests with zfs, and results where appallingly bad, but = that was with db size > ram.=20 >>>>=20 >>>> I think the model used by PostgreSQL, as most databases, are very = disk block centric. Using zfs makes it hard to get good performance. But = this was some time ago, maybe things have improved.=20 >> I have some hardware that I ran with last week wherein I was *not* = able to reproduce any performance difference between ZFS and UFS2. On = both UFS2 and ZFS I was seeing the same performance when using a a = RAID10 / set of mirrors. I talked with the Dragonfly folk who originally = performed these tests and they also saw the same thing: no real = performance difference between ZFS and UFS. I ran my tests on a host = with 16 drive, 10K SAS, 192GB RAM. I also created a kernel profiling = image and ran the 20 concurrent user test under kgprof(1), dtrace, and = pmcstat and have the results available: >>=20 >> http://people.freebsd.org/~seanc/pg9.3-fbsd10-profiling/ >>=20 >> There are some investigations that are ongoing as a result of these = findings. The dfly methodology was observed when generating these = results. Stay tuned. -sc >>=20 >=20 > I'm not sure that the ZFS vs UFS2 question is at the core of the > performance problem. We're definitely seeing marked slowdowns between > Pg 9.2 and 9.3 on UFS2 (RAID10 + Dell H710p (mfi) raid controller with > 1GB NVRAM) When the working set fits in RAM (OS + PG), there isn't a performance = difference between 9.2 and 9.3. This is a good data point. I will try and reproduce this workload and = will run the performance profiling again to see if something else pops = up in the profiling. -sc -- Sean Chittenden sean@chittenden.org From owner-freebsd-performance@FreeBSD.ORG Wed May 21 20:19:43 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7933AE62; Wed, 21 May 2014 20:19:43 +0000 (UTC) Received: from melon.pingpong.net (melon.pingpong.net [79.136.116.200]) by mx1.freebsd.org (Postfix) with ESMTP id 3342B2BFE; Wed, 21 May 2014 20:19:42 +0000 (UTC) Received: from [10.19.39.26] (c-5eeaaaa2-74736162.cust.telenor.se [94.234.170.162]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by melon.pingpong.net (Postfix) with ESMTPSA id 6896835C7F; Wed, 21 May 2014 22:19:40 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues From: Palle Girgensohn X-Mailer: iPhone Mail (11D201) In-Reply-To: Date: Wed, 21 May 2014 22:19:39 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <5327B9B7.3050103@gmail.com> <2610F490C952470C9D15999550F67068@multiplay.co.uk> <532A192A.1070509@gmail.com> <572540F9-13E4-4BA9-88AE-5F47FB19450A@pingpong.net> <1BC3D447-2044-4AB8-B183-B83957BC9112@pingpong.net> <1473AF7C-B190-4CD4-B611-BA4090A081CB@pingpong.net> <537CEACD.8090701@FreeBSD.org> To: Sean Chittenden Cc: "freebsd-performance@freebsd.org" , Matthew Seaman X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 20:19:43 -0000 21 maj 2014 kl. 22:05 skrev Sean Chittenden : >>>> I did some tests with zfs, and results where appallingly bad, but that w= as with db size > ram.=20 >>>>>=20 >>>>> I think the model used by PostgreSQL, as most databases, are very disk= block centric. Using zfs makes it hard to get good performance. But this wa= s some time ago, maybe things have improved. >>> I have some hardware that I ran with last week wherein I was *not* able t= o reproduce any performance difference between ZFS and UFS2. On both UFS2 an= d ZFS I was seeing the same performance when using a a RAID10 / set of mirro= rs. I talked with the Dragonfly folk who originally performed these tests an= d they also saw the same thing: no real performance difference between ZFS a= nd UFS. I ran my tests on a host with 16 drive, 10K SAS, 192GB RAM. I also c= reated a kernel profiling image and ran the 20 concurrent user test under kg= prof(1), dtrace, and pmcstat and have the results available: >>>=20 >>> http://people.freebsd.org/~seanc/pg9.3-fbsd10-profiling/ >>>=20 >>> There are some investigations that are ongoing as a result of these find= ings. The dfly methodology was observed when generating these results. Stay t= uned. -sc >>=20 >> I'm not sure that the ZFS vs UFS2 question is at the core of the >> performance problem. We're definitely seeing marked slowdowns between >> Pg 9.2 and 9.3 on UFS2 (RAID10 + Dell H710p (mfi) raid controller with >> 1GB NVRAM) I never meant that it was the core of our problem. Zfs vs ufs is out of scop= e here.=20 >=20 > When the working set fits in RAM (OS + PG), there isn't a performance diff= erence between 9.2 and 9.3. >=20 > This is a good data point. I will try and reproduce this workload and will= run the performance profiling again to see if something else pops up in the= profiling. -sc >=20 I don't agree. My original measurements showed a 20% slowdown for a reasonab= ly small ( -- > Sean Chittenden > sean@chittenden.org >=20 > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.= org" From owner-freebsd-performance@FreeBSD.ORG Thu May 22 06:07:07 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 84A50487; Thu, 22 May 2014 06:07:07 +0000 (UTC) Received: from mail-wi0-x229.google.com (mail-wi0-x229.google.com [IPv6:2a00:1450:400c:c05::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C27AA2AAA; Thu, 22 May 2014 06:07:06 +0000 (UTC) Received: by mail-wi0-f169.google.com with SMTP id hi2so9413170wib.0 for ; Wed, 21 May 2014 23:07:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:sender:date:message-id:subject:from:to :content-type; bh=FdhgYqVCaPh+BeIHnJlDrpGL0fR054JT1JRFG0JHSmI=; b=MDTAZGzhCnDl/bDO4QorM48b/4GyuA9M4UC+BTBbV386lyNRJZD5cLN4jINsixoW+u egEA2doU5icE+uy8VA1tUyfAdxlVJn1G9pVcujD7A5awSj5NU9bHKN7pkuiaPOBp3qDx g9gijWrQHPQ4R9wxXLsJBgCno6AZ/1eCwD1T1+4vn4XOA3HMNn1GV41Q6iwuKk52z2es QaWu6UYkm1UMlJ0y7iYaW0qDMHbHvF+aG0LqaRRPXYPO5T75cd/fAVEoNLZZM1rZc4q6 j5INumeygYQuR6aWi1w6kCwnSMefy8+LLIM52mQhVZCkTbatYEceg9QoLsYtxdqn2IZM TFFA== MIME-Version: 1.0 X-Received: by 10.194.60.114 with SMTP id g18mr697807wjr.61.1400738824857; Wed, 21 May 2014 23:07:04 -0700 (PDT) Reply-To: attilio@FreeBSD.org Sender: asmrookie@gmail.com Received: by 10.217.61.196 with HTTP; Wed, 21 May 2014 23:07:04 -0700 (PDT) Date: Wed, 21 May 2014 23:07:04 -0700 X-Google-Sender-Auth: xGaUHRF-S8ov-FjEsNMKgy42p5U Message-ID: Subject: FreeBSD 10 and PostgreSQL 9.3 scalability issues From: Attilio Rao To: freebsd-performance@freebsd.org, Bryan Drewery , Florian Smeets Content-Type: text/plain; charset=UTF-8 X-Mailman-Approved-At: Thu, 22 May 2014 17:33:35 +0000 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 May 2014 06:07:07 -0000 [ Please CC me as I'm not subscribed to FreeBSD mailing lists ] Recently Bryan Drewery and I have been looking at this issue, in particular after that some people has been pointing us to DragonflyBSD / Linux benchmarks. Usually DB workloads are interesting mostly because they can expose some real performance problems in kernel intensive workloads (scalability of scheduler, VM, VFS, network stack, etc.). More generally, however, some extra-attention must be put when the test is performed, especially by avoiding I/O (to increase predictability and avoid latency fluctuations). We have done tests similar to what Florian Smeets has been doing on netperf's cluster giant-ape1 (a XEON E7 4(nodes)x10(cores) machine) and I've come to the conclusion that the tests comparing dragonflybsd, Linux and FreeBSD have intrinsics problems. Essentially having the client and the backend of PGSQL on the same machine makes them share the data, getting to much faster results. However, the more cache levels they share, the faster the results will be. When the client becomes heavily multithreaded, in particular, the data becomes so unpredictably spreaded that it is difficult to say how much cache-sharing/trashing effects are coming into play. However I can make you an example that explains it well: with a full DB in memory and writes to tmpfs (so no real I/O) and a *single client* configuration, we were getting around +20% if the client and the backend were running on the same chip (so sharing L2 cache) rather than running on 2 different domains. If you consider multiple clients, all touching the same data, caming to play, it becomes a pretty unpredictable behaviour. I can also tell that Florian has tried to benchmark on the same machine in the past and got very unstable numbers, as when using all the 40-cores available, with fluctuations in the range of +/-10%, when trying around 10 times. I think that this explains why this was really the case. I'm not going to claim that FreeBSD will be kick-ass on this type of workload but I think that results reported so far are biased and I think that a more realistic behaviour (that I hope to start exploring soon) would involve having PGSQL clients to run on separate machines so that we can just benchmark the backend behaviour. After all, PGSQL people raccoment this as well: http://www.postgresql.org/docs/devel/static/pgbench.html "A limitation of pgbench is that it can itself become the bottleneck when trying to test a large number of client sessions. This can be alleviated by running pgbench on a different machine from the database server, although low network latency will be essential. It might even be useful to run several pgbench instances concurrently, on several client machines, against the same database server." To be honest, I'm a bit worried that with a realistic/physical test FreeBSD is going to be more limited by NIC / TCP stack bottlenecks than real CPU / memory ones (the ones really interesting to analyze further kernel scalability) but there is no way than try it with performing hardware to see where we are staying. A fairer approach would be maybe to just stick all the clients into a single, different domain and the backend into another. However there will still be some data sharing among them, invalidating the test at all the effects and having clients/backend to compete for the same resources. While looking into this, however I noted something that is interesting: the EST / cpufreq driver is essentially broken. It is not going to attach to newest Intel microarchs (Nehalem, SB, etc). From what my experience is, I can tell that enabling EST and possibly disabling turbo-boost makes a nice difference. Without such capabilities controlled by the est driver we can end up in having sub-optimal performances on Intel CPUs (I can tell that for giant-ape1 this wasn't the case, as everything was already setup properly, but we cannot assume this for all the machines booting FreeBSD I think). Possibly some time might be spent on this part of the code to be properly available. Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-performance@FreeBSD.ORG Fri Jun 27 12:56:20 2014 Return-Path: Delivered-To: performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 53C5CA10; Fri, 27 Jun 2014 12:56:20 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 418C82BDB; Fri, 27 Jun 2014 12:56:19 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id s5RCuDgL053532; Fri, 27 Jun 2014 15:56:13 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua s5RCuDgL053532 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.8/Submit) id s5RCuDkd053531; Fri, 27 Jun 2014 15:56:13 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 27 Jun 2014 15:56:13 +0300 From: Konstantin Belousov To: performance@freebsd.org Subject: PostgreSQL performance on FreeBSD Message-ID: <20140627125613.GT93733@kib.kiev.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="gw/jrspf8iiPX0nu" Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: current@freebsd.org X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jun 2014 12:56:20 -0000 --gw/jrspf8iiPX0nu Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, I did some measurements and hacks to see about the performance and scalability of PostgreSQL 9.3 on FreeBSD, sponsored by The FreeBSD Foundation. The results are described in https://kib.kiev.ua/kib/pgsql_perf.pdf. The uncommitted patches, referenced in the article, are available as https://kib.kiev.ua/kib/pig1.patch.txt https://kib.kiev.ua/kib/patch-2 --gw/jrspf8iiPX0nu Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJTrWnsAAoJEJDCuSvBvK1Bh5sP/2QSi4pgfmziDoMVip/rKwFB DxMiTu2rl8sVWjWTB3Qd0zlk/glJwwZDmhprA+ovXJFsLlZBXSrZKvz+tP9Ug1RP iZ5z87XCN37GMDnvYO7uh8dt/Z2HVRiPQA7GXTaDUEdycNUED1OhlH9X62fw4wM8 uuU5J1dsyf9Eh+MXyDhJPqf83l6wvgSbX8UOEhOOmcx/57jSaFm2elpGKHdWMhHV bLGqbHv2TPek5BbsSUQhpAvrWG7j8iLf1QIbbYTkssu8jN4HtyceBKHLHm0vKU6E uNWrvdRzl3ZSLWc1/oVKd2IESeSCDhQI4053gV4VQ+tD89D53d5eaIaxdEoYEuVO TFyzAS6D4FtAnSiVuq9GKV4ChX2cA7hhI+l6AxM6RQ745gn3o60jQDNGvMyd0hg+ Mdy9e9vNSv9PQWwTOs2oUJXt9Y9Mkni01xq2fUz7w6ZBBlHmSdPH1H8GAHeFfDfh 9r3t1CIgY26Ff9kUHMgkWs7uBp6ly9QSAOIEPyVDWhbj6x88N3vIQ/V2k58vrhJW VofYgmlz1M/6D7gi2KnVQll4n9EAzuviow05yfRJVVnGgAhvyKPrmyNxML6nQF26 /MvYBYpC9ceWPP7Mk7+xbWy3S1qqot1H+8U1nyUCYv2Qm40xRMFIH7hLcC3kb8M7 L9Hd8WjQ9Z9r/NxF0/Zz =gNTr -----END PGP SIGNATURE----- --gw/jrspf8iiPX0nu-- From owner-freebsd-performance@FreeBSD.ORG Fri Jun 27 14:00:08 2014 Return-Path: Delivered-To: performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8D2B9F84; Fri, 27 Jun 2014 14:00:08 +0000 (UTC) Received: from mail.feld.me (mail.feld.me [66.170.3.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.feld.me", Issuer "Gandi Standard SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2D8992221; Fri, 27 Jun 2014 14:00:07 +0000 (UTC) Received: from mail.feld.me (mail.feld.me [66.170.3.6]); by mail.feld.me (OpenSMTPD) with ESMTP id 18724661; Fri, 27 Jun 2014 08:53:26 -0500 (CDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=feld.me; h=mime-version :date:content-type:message-id:from:subject:to:cc:in-reply-to :references:sender; s=blargle2; bh=QHp5kyOwcji/8sYH0H7go2vvmGE=; b= T2Q4GMvkZGKMJ1mud/wS5wbWvmWGHQ5SgB3RxSuCgvSgl1HAD4Jh9cYB6JhtZ+ic srRIXnyLw2R42A9PTSXqJUcGxsnzFmXv1nADSYyXmSUutJZyRIuFYYjUGJ6MZkNT 34QWulARgX5b1NueJjLFbLhDfeRaDQM0R1yqsVllaGn78ZY4aV5pfaJEWPoLTPib tb+ZN3tpI0GCsiBboqRZUX0qdTBmGOzE3ikVu9MM18cGU+uVuM5MLWefEV5fKWEz wgAM7eNHOzuCttTUoiL6RMBbjgPpezJ0MYk6p8QDz/i5xTnW9vrRPJpj/Mw75Za8 fekadck5sFQ/yDuFg4zwrA== DomainKey-Signature: a=rsa-sha1; c=nofws; d=feld.me; h=mime-version:date :content-type:message-id:from:subject:to:cc:in-reply-to :references:sender; q=dns; s=blargle2; b=CrY0WRhU+qJEtFWq3NYkyA4 HXVjhGJplG84j/v8/JZ6NZ9lfd2emgRjtJEYNvXS5OkLT6waxh3WbLJzfqTf91JA e5Q1SOIZhabisIquvXO32lLFNFI6qN2xNu8FEtOJFYgH87JSBwZ9VA5/KDuc2cSj 1JITPBh47TNM7RHWYZ+pOe+2fOsNPt1+Y6tb+S4aCw417xJUdkKPKKXM0cA65PXD shcfGWMMNBJ8jDJ/6a07e1Ms28SSRRRyc+cn/wMClMmmKkehDpQQDP2aZMjJ6/VA QoTDilUoIQf2HOfKPws0AdZjQU8RMlZWleTEeQAOft6y8xKDbDQNTUouJmsUksw= = Received: from mail.feld.me (mail.feld.me [66.170.3.6]); by mail.feld.me (OpenSMTPD) with ESMTP id 966a52e7; Fri, 27 Jun 2014 08:53:26 -0500 (CDT) Received: from feld@feld.me by mail.feld.me (Archiveopteryx 3.2.0) with esmtpa id 1403877205-4188-4185/5/6; Fri, 27 Jun 2014 13:53:25 +0000 Mime-Version: 1.0 Date: Fri, 27 Jun 2014 13:53:25 +0000 Content-Type: text/plain; charset=utf-8 Message-Id: <3fc51f4ac40910455d761c6cd2b150c7@mail.feld.me> X-Mailer: RainLoop/1.6.7.132 From: Mark Felder Subject: Re: PostgreSQL performance on FreeBSD To: Konstantin Belousov , performance@freebsd.org In-Reply-To: <20140627125613.GT93733@kib.kiev.ua> References: <20140627125613.GT93733@kib.kiev.ua> Sender: feld@feld.me Cc: current@freebsd.org X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jun 2014 14:00:08 -0000 June 27 2014 7:56 AM, "Konstantin Belousov" wrote: > Hi, > I did some measurements and hacks to see about the performance and > scalability of PostgreSQL 9.3 on FreeBSD, sponsored by The FreeBSD > Foundation. > > The results are described in https://kib.kiev.ua/kib/pgsql_perf.pdf. > The uncommitted patches, referenced in the article, are available as > https://kib.kiev.ua/kib/pig1.patch.txt > https://kib.kiev.ua/kib/patch-2 > Thank you for taking the time to do this benchmark. From owner-freebsd-performance@FreeBSD.ORG Fri Jun 27 15:03:58 2014 Return-Path: Delivered-To: performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A5B547A6 for ; Fri, 27 Jun 2014 15:03:58 +0000 (UTC) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 728D32935 for ; Fri, 27 Jun 2014 15:03:57 +0000 (UTC) Received: from compute2.internal (compute2.nyi.mail.srv.osa [10.202.2.42]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 2FA3C209D9 for ; Fri, 27 Jun 2014 11:03:36 -0400 (EDT) Received: from frontend1 ([10.202.2.160]) by compute2.internal (MEProxy); Fri, 27 Jun 2014 11:03:36 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=grosser.es; h= message-id:date:from:mime-version:to:cc:subject:references :in-reply-to:content-type:content-transfer-encoding; s=mesmtp; bh=KT4zPOliJqCKFoxdhuogF14OWBU=; b=MGeXP8Vidg5vhxA6biYZLM0neai7 cvYiAxoFXpqYOTezy72MraOoPw/G9rGKqRbSSCwZOYhuRC88svPf06pQlf5aznB9 SVNGWhUHea0dPGGZYoU+g/NvV8TY00R5eTUlf8gHcSQib5CHG3U5HEFMKkwnalCC XhnLL/86EqHt1GI= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:date:from:mime-version:to:cc :subject:references:in-reply-to:content-type :content-transfer-encoding; s=smtpout; bh=KT4zPOliJqCKFoxdhuogF1 4OWBU=; b=p0mt/lwRKr67vPF50cdJK3KDCDn+xeNpzqCgMNjok6AupVuCGjoVJp ODR71z/G3tnHEJrP139fQMni+adYK5Qjppb6co+Xci8ckd3bmKJ8f4xGUdP3g2lT A5DNoSM0XqzVfaa6jXTVzObK+kyk5cUqyPZ1cTE/n31zUOXvGGsOc= X-Sasl-enc: GI4l6yXT3gkABetPWw4jpWp+0aOGKiKEleMakpCdRv8i 1403881415 Received: from [129.199.97.210] (unknown [129.199.97.210]) by mail.messagingengine.com (Postfix) with ESMTPA id B0840C007B1; Fri, 27 Jun 2014 11:03:35 -0400 (EDT) Message-ID: <53AD87C6.7050408@grosser.es> Date: Fri, 27 Jun 2014 17:03:34 +0200 From: Tobias Grosser User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Konstantin Belousov , performance@freebsd.org Subject: Re: PostgreSQL performance on FreeBSD References: <20140627125613.GT93733@kib.kiev.ua> In-Reply-To: <20140627125613.GT93733@kib.kiev.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Fri, 27 Jun 2014 15:44:20 +0000 Cc: current@freebsd.org X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jun 2014 15:03:58 -0000 On 27/06/2014 14:56, Konstantin Belousov wrote: > Hi, > I did some measurements and hacks to see about the performance and > scalability of PostgreSQL 9.3 on FreeBSD, sponsored by The FreeBSD > Foundation. > > The results are described in https://kib.kiev.ua/kib/pgsql_perf.pdf. > The uncommitted patches, referenced in the article, are available as > https://kib.kiev.ua/kib/pig1.patch.txt > https://kib.kiev.ua/kib/patch-2 Interesting. Did you report the clang bug upstream? Tobias From owner-freebsd-performance@FreeBSD.ORG Fri Jun 27 15:51:40 2014 Return-Path: Delivered-To: performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 50211A31; Fri, 27 Jun 2014 15:51:40 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 27DF22D7C; Fri, 27 Jun 2014 15:51:40 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id CAD70B965; Fri, 27 Jun 2014 11:51:38 -0400 (EDT) From: John Baldwin To: freebsd-current@freebsd.org Subject: Re: PostgreSQL performance on FreeBSD Date: Fri, 27 Jun 2014 10:57:53 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <20140627125613.GT93733@kib.kiev.ua> In-Reply-To: <20140627125613.GT93733@kib.kiev.ua> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201406271057.53599.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 27 Jun 2014 11:51:39 -0400 (EDT) X-Mailman-Approved-At: Fri, 27 Jun 2014 16:28:43 +0000 Cc: Konstantin Belousov , performance@freebsd.org, current@freebsd.org X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jun 2014 15:51:40 -0000 On Friday, June 27, 2014 8:56:13 am Konstantin Belousov wrote: > Hi, > I did some measurements and hacks to see about the performance and > scalability of PostgreSQL 9.3 on FreeBSD, sponsored by The FreeBSD > Foundation. > > The results are described in https://kib.kiev.ua/kib/pgsql_perf.pdf. > The uncommitted patches, referenced in the article, are available as > https://kib.kiev.ua/kib/pig1.patch.txt > https://kib.kiev.ua/kib/patch-2 Did you run the same benchmark on the same hardware with any other OS's to compare results? -- John Baldwin From owner-freebsd-performance@FreeBSD.ORG Fri Jun 27 16:34:14 2014 Return-Path: Delivered-To: performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 228465A7; Fri, 27 Jun 2014 16:34:14 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 84A2B2247; Fri, 27 Jun 2014 16:34:13 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id s5RGY7OJ003317 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 27 Jun 2014 19:34:08 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua s5RGY7OJ003317 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id s5RGY7qo003316; Fri, 27 Jun 2014 19:34:07 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 27 Jun 2014 19:34:07 +0300 From: Konstantin Belousov To: John Baldwin Subject: Re: PostgreSQL performance on FreeBSD Message-ID: <20140627163407.GX93733@kib.kiev.ua> References: <20140627125613.GT93733@kib.kiev.ua> <201406271057.53599.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="vD4R8EutdGn5UgVK" Content-Disposition: inline In-Reply-To: <201406271057.53599.jhb@freebsd.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: freebsd-current@freebsd.org, performance@freebsd.org X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jun 2014 16:34:14 -0000 --vD4R8EutdGn5UgVK Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jun 27, 2014 at 10:57:53AM -0400, John Baldwin wrote: > On Friday, June 27, 2014 8:56:13 am Konstantin Belousov wrote: > > Hi, > > I did some measurements and hacks to see about the performance and > > scalability of PostgreSQL 9.3 on FreeBSD, sponsored by The FreeBSD > > Foundation. > >=20 > > The results are described in https://kib.kiev.ua/kib/pgsql_perf.pdf. > > The uncommitted patches, referenced in the article, are available as > > https://kib.kiev.ua/kib/pig1.patch.txt > > https://kib.kiev.ua/kib/patch-2 >=20 > Did you run the same benchmark on the same hardware with any other OS's t= o=20 > compare results? No. FWIW, before the failing after the 30 clients is corrected, I do not think it is much interesting to do such comparision. --vD4R8EutdGn5UgVK Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJTrZz/AAoJEJDCuSvBvK1BivAP/jQn+zI9GOk+h3mQtuQVH9KE t1W8101cwctwZ2ysLbg0OTOPGzmKeGm7NOK+PtDnoS6pl4BkV9QVhkE63WyQ3Ptg FclUM60nE15P6Z+sbS9Ol4SNGDAo6rPmEkDtKkOEIM2VvFr+F9pOyPlH4LwcP4g3 OiKI3+Wp8BaJTDrRwZRV2+f1VxuY2/kxFOPTTtjvOXwMMnUSRuCqpAvxB2nYny+H VurzG3Gac1yYHRfBnbfoFObbkYXo0ayFh5lWOCEIoCkF/G8NRZUgOuEvrj7yPWeL EnvehQZQT59c0fbwm2QoVczD9F85hiiF9mIHXxA+mpLWYIJZHdhCMNE0OecbRWOF uw9fE0iSDAguCMOmprPBJZM+EB0pqaJ40eS8a/AQ99SQBES60mCQIXlG7IF62Tiq exqQRrQEYTrMWUIaMIJfTHyQPIu9Xla2me3wG5vsSkpIpZPH2NCqXhL+kUiukLgS kNWbxMSF2zdev8KPZu065CUSOpLKSunxZyn8b0Y3eiW1IGmgKFpqpavq9xArNxjZ M2/i/ogVLPy/Di9l6ZUOkWEag+ypIFd+5P8GawQbNE+qnpXwGZEOAdBHqeBbAx9Z 2EZaNahwSikcWrjr3y0hMfyg2iM1M9TUiWlc7dt44TNpcM0yTegHJVrVkfWY0657 zpH131slsx43J2fb0iJZ =k5Mx -----END PGP SIGNATURE----- --vD4R8EutdGn5UgVK-- From owner-freebsd-performance@FreeBSD.ORG Sat Jun 28 10:14:33 2014 Return-Path: Delivered-To: performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AB804102; Sat, 28 Jun 2014 10:14:33 +0000 (UTC) Received: from melon.pingpong.net (melon.pingpong.net [79.136.116.200]) by mx1.freebsd.org (Postfix) with ESMTP id 6BCE22915; Sat, 28 Jun 2014 10:14:33 +0000 (UTC) Received: from [10.0.1.16] (h-43-145.a357.priv.bahnhof.se [79.136.43.145]) by melon.pingpong.net (Postfix) with ESMTPA id D11A037FFC; Sat, 28 Jun 2014 12:08:39 +0200 (CEST) References: <20140627125613.GT93733@kib.kiev.ua> <201406271057.53599.jhb@freebsd.org> <20140627163407.GX93733@kib.kiev.ua> In-Reply-To: <20140627163407.GX93733@kib.kiev.ua> Mime-Version: 1.0 (1.0) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Message-Id: <35090A62-2DB8-493C-A5ED-ADB1BC193640@pingpong.net> X-Mailer: iPad Mail (9B206) From: Palle Girgensohn Subject: Re: PostgreSQL performance on FreeBSD Date: Sat, 28 Jun 2014 12:08:39 +0200 To: Konstantin Belousov Cc: "freebsd-current@freebsd.org" , "performance@freebsd.org" , John Baldwin X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jun 2014 10:14:33 -0000 27 jun 2014 kl. 18:34 skrev Konstantin Belousov : > On Fri, Jun 27, 2014 at 10:57:53AM -0400, John Baldwin wrote: >> On Friday, June 27, 2014 8:56:13 am Konstantin Belousov wrote: >>> Hi, >>> I did some measurements and hacks to see about the performance and >>> scalability of PostgreSQL 9.3 on FreeBSD, sponsored by The FreeBSD >>> Foundation. >>>=20 >>> The results are described in https://kib.kiev.ua/kib/pgsql_perf.pdf. >>> The uncommitted patches, referenced in the article, are available as >>> https://kib.kiev.ua/kib/pig1.patch.txt >>> https://kib.kiev.ua/kib/patch-2 >>=20 >> Did you run the same benchmark on the same hardware with any other OS's t= o=20 >> compare results? >=20 > No. >=20 > FWIW, before the failing after the 30 clients is corrected, I do not > think it is much interesting to do such comparision. This is great work! Does anybody know how far back in FreeBSD versions using posix semaphore ins= tead of sysv would make a difference? It seems we need a rather current ver= sion? 8.x did not support it at all, at some point at lest, and in 9 it was b= uggy. I could add he patch-2 to the port, but I reckon it needs a conditiona= l based on FreeBSD version? The clang bug should go upstreams, right? I have seen similar curves, presented by Greg Smith (PostgreSQL hacker) wher= e he concluded that there is no point in running more than 50 concurrent con= nections. This was for Linux. In your measures, the knee is at 30. That's sa= id, FreeBSD could and should do better, but probably there is a limit where t= here will be a knee in the graph and performance will drop. It should be mor= e than 30, though, as you rightly commented. Do you any ideas to pursue this further apart from complicated rewrites like= DragonFly? Palle= From owner-freebsd-performance@FreeBSD.ORG Sat Jun 28 10:21:44 2014 Return-Path: Delivered-To: performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 309BE2F0; Sat, 28 Jun 2014 10:21:44 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AAF4F29C4; Sat, 28 Jun 2014 10:21:43 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id s5SALbTk058878 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 28 Jun 2014 13:21:37 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua s5SALbTk058878 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id s5SALbMN058877; Sat, 28 Jun 2014 13:21:37 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 28 Jun 2014 13:21:37 +0300 From: Konstantin Belousov To: Palle Girgensohn Subject: Re: PostgreSQL performance on FreeBSD Message-ID: <20140628102137.GA93733@kib.kiev.ua> References: <20140627125613.GT93733@kib.kiev.ua> <201406271057.53599.jhb@freebsd.org> <20140627163407.GX93733@kib.kiev.ua> <35090A62-2DB8-493C-A5ED-ADB1BC193640@pingpong.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="4TjORqrrURmsgxxJ" Content-Disposition: inline In-Reply-To: <35090A62-2DB8-493C-A5ED-ADB1BC193640@pingpong.net> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: "freebsd-current@freebsd.org" , "performance@freebsd.org" , John Baldwin X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jun 2014 10:21:44 -0000 --4TjORqrrURmsgxxJ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jun 28, 2014 at 12:08:39PM +0200, Palle Girgensohn wrote: >=20 >=20 > 27 jun 2014 kl. 18:34 skrev Konstantin Belousov : >=20 > > On Fri, Jun 27, 2014 at 10:57:53AM -0400, John Baldwin wrote: > >> On Friday, June 27, 2014 8:56:13 am Konstantin Belousov wrote: > >>> Hi, > >>> I did some measurements and hacks to see about the performance and > >>> scalability of PostgreSQL 9.3 on FreeBSD, sponsored by The FreeBSD > >>> Foundation. > >>>=20 > >>> The results are described in https://kib.kiev.ua/kib/pgsql_perf.pdf. > >>> The uncommitted patches, referenced in the article, are available as > >>> https://kib.kiev.ua/kib/pig1.patch.txt > >>> https://kib.kiev.ua/kib/patch-2 > >>=20 > >> Did you run the same benchmark on the same hardware with any other OS'= s to=20 > >> compare results? > >=20 > > No. > >=20 > > FWIW, before the failing after the 30 clients is corrected, I do not > > think it is much interesting to do such comparision. >=20 > This is great work! >=20 > Does anybody know how far back in FreeBSD versions using posix semaphore = instead of sysv would make a difference? It seems we need a rather current= version? 8.x did not support it at all, at some point at lest, and in 9 it= was buggy. I could add he patch-2 to the port, but I reckon it needs a con= ditional based on FreeBSD version? >=20 I recommend to add it as an option. The currently supported versions of stable/9 and higher have new posix semaphores implementation. The stable/8 also has posix semaphores, but there it is kernel-based interface, I do not plan to evaluate it in any way. > The clang bug should go upstreams, right? I believe there is already some activity about it. I do not follow clang development. >=20 > I have seen similar curves, presented by Greg Smith (PostgreSQL > hacker) where he concluded that there is no point in running more > than 50 concurrent connections. This was for Linux. In your measures, > the knee is at 30. That's said, FreeBSD could and should do better, > but probably there is a limit where there will be a knee in the graph > and performance will drop. It should be more than 30, though, as you > rightly commented. > > Do you any ideas to pursue this further apart from complicated > rewrites like DragonFly? > I do. The scope of the current work was done to obtain understanding where do we = stay and, if possible, evaluate ideas, possibly in the hackish way. I hope and almost sure that this will be continued, but cannot provide any time estimation. --4TjORqrrURmsgxxJ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJTrpcxAAoJEJDCuSvBvK1BYE8P/RYeGpegRSM7KT2+HlrW2zRO nGiXx7phK6+CJtUdzOvgrhuHkniI0e54t6qPF8q1XTtVNnPC+5nMWGuNoeJrMfPu +R29XDGcFNONovDDsbIuQgNq1J9foIQyKY6zFlgXl2jTtzuv2hwBoVP/BJ18mKRO ly51Ys+gRFQmdCcsU3GD+tp4q5VesNdOes9V3CBefEXNZPnFPgtULNh+mfVd+0aS jXfMo/2SRecUXG0rWBnB+5qJlySTJUmMzsaBaiWjTPQ0IlrMVcOTcwxc2cqM0GMt ihrPzdUEXoKZQt8qa36Otb71dcKdxQQVFjOf7aQMRbg4C66m2vwTIbAFurJnh8m/ GtAc9ebvYUBeAsqz2I6aacjfFCgc/Lg3dkLf5h0sCGfQoF8KhO0UGbSjWhPDmHw1 2MFIWs1q/gVsXgtpG2rnHwgEGVDbUQB/NwA8lwqyzU/x1+He2g8cT+Q2SO2dfa9u NEXoZhH7mJvbRQbXaJ+zhthtRC/wAr6RwWYA+T64/1RJxMv8Y2fjwMykYxVDQI6C rC3dddI+tgNOxe4hzoxLAd89K4kU0yA/xSz3q5pmZc/MO7bZ7zH1e26SwpGP5/yo TNTsNuo+buZUYqLvHYpbBwA0D+Uu/AGH14vkKpOJjoZAGd6/amt6jrX3rgJGVOYK sdGFeytYByDakUQfLu1H =3QOa -----END PGP SIGNATURE----- --4TjORqrrURmsgxxJ-- From owner-freebsd-performance@FreeBSD.ORG Sat Jun 28 11:37:23 2014 Return-Path: Delivered-To: performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A45C26E4; Sat, 28 Jun 2014 11:37:23 +0000 (UTC) Received: from melon.pingpong.net (melon.pingpong.net [79.136.116.200]) by mx1.freebsd.org (Postfix) with ESMTP id 2BA0C2F5C; Sat, 28 Jun 2014 11:37:22 +0000 (UTC) Received: from [10.0.1.26] (h-43-145.a357.priv.bahnhof.se [79.136.43.145]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by melon.pingpong.net (Postfix) with ESMTPSA id BE6C1375D2; Sat, 28 Jun 2014 13:37:20 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: PostgreSQL performance on FreeBSD From: Palle Girgensohn X-Mailer: iPhone Mail (11D201) In-Reply-To: <20140628102137.GA93733@kib.kiev.ua> Date: Sat, 28 Jun 2014 13:37:20 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20140627125613.GT93733@kib.kiev.ua> <201406271057.53599.jhb@freebsd.org> <20140627163407.GX93733@kib.kiev.ua> <35090A62-2DB8-493C-A5ED-ADB1BC193640@pingpong.net> <20140628102137.GA93733@kib.kiev.ua> To: Konstantin Belousov Cc: "freebsd-current@freebsd.org" , "performance@freebsd.org" , John Baldwin X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jun 2014 11:37:23 -0000 > 28 jun 2014 kl. 12:21 skrev Konstantin Belousov : >=20 >> On Sat, Jun 28, 2014 at 12:08:39PM +0200, Palle Girgensohn wrote: >>=20 >>=20 >>> 27 jun 2014 kl. 18:34 skrev Konstantin Belousov : >>>=20 >>>> On Fri, Jun 27, 2014 at 10:57:53AM -0400, John Baldwin wrote: >>>>> On Friday, June 27, 2014 8:56:13 am Konstantin Belousov wrote: >>>>> Hi, >>>>> I did some measurements and hacks to see about the performance and >>>>> scalability of PostgreSQL 9.3 on FreeBSD, sponsored by The FreeBSD >>>>> Foundation. >>>>>=20 >>>>> The results are described in https://kib.kiev.ua/kib/pgsql_perf.pdf. >>>>> The uncommitted patches, referenced in the article, are available as >>>>> https://kib.kiev.ua/kib/pig1.patch.txt >>>>> https://kib.kiev.ua/kib/patch-2 >>>>=20 >>>> Did you run the same benchmark on the same hardware with any other OS's= to=20 >>>> compare results? >>>=20 >>> No. >>>=20 >>> FWIW, before the failing after the 30 clients is corrected, I do not >>> think it is much interesting to do such comparision. >>=20 >> This is great work! >>=20 >> Does anybody know how far back in FreeBSD versions using posix semaphore i= nstead of sysv would make a difference? It seems we need a rather current v= ersion? 8.x did not support it at all, at some point at lest, and in 9 it wa= s buggy. I could add he patch-2 to the port, but I reckon it needs a conditi= onal based on FreeBSD version? > I recommend to add it as an option. The currently supported versions > of stable/9 and higher have new posix semaphores implementation. > The stable/8 also has posix semaphores, but there it is kernel-based > interface, I do not plan to evaluate it in any way. According to one source, posix semaphores uses O(N^2) file descriptors, wher= e N is the number of connections. Do you know if this is true? (I'll try it,= naturally, just checking).=20 >=20 >=20 >> The clang bug should go upstreams, right? > I believe there is already some activity about it. I do not follow > clang development. Sounds good enough.=20 >=20 >>=20 >> I have seen similar curves, presented by Greg Smith (PostgreSQL >> hacker) where he concluded that there is no point in running more >> than 50 concurrent connections. This was for Linux. In your measures, >> the knee is at 30. That's said, FreeBSD could and should do better, >> but probably there is a limit where there will be a knee in the graph >> and performance will drop. It should be more than 30, though, as you >> rightly commented. >>=20 >> Do you any ideas to pursue this further apart from complicated >> rewrites like DragonFly? > I do. >=20 > The scope of the current work was done to obtain understanding where do we= stay > and, if possible, evaluate ideas, possibly in the hackish way. I hope > and almost sure that this will be continued, but cannot provide any time > estimation. Great. If you need help testing, I might be able to help.=20 Cheers, Palle= From owner-freebsd-performance@FreeBSD.ORG Sat Jun 28 16:33:14 2014 Return-Path: Delivered-To: performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 34419621; Sat, 28 Jun 2014 16:33:14 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C73EC2424; Sat, 28 Jun 2014 16:33:13 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id s5SGX7M3084358 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 28 Jun 2014 19:33:07 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua s5SGX7M3084358 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id s5SGX77I084357; Sat, 28 Jun 2014 19:33:07 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 28 Jun 2014 19:33:07 +0300 From: Konstantin Belousov To: Palle Girgensohn Subject: Re: PostgreSQL performance on FreeBSD Message-ID: <20140628163307.GC93733@kib.kiev.ua> References: <20140627125613.GT93733@kib.kiev.ua> <201406271057.53599.jhb@freebsd.org> <20140627163407.GX93733@kib.kiev.ua> <35090A62-2DB8-493C-A5ED-ADB1BC193640@pingpong.net> <20140628102137.GA93733@kib.kiev.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="xqJRvRgB/be8tbVJ" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: "freebsd-current@freebsd.org" , "performance@freebsd.org" , John Baldwin X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jun 2014 16:33:14 -0000 --xqJRvRgB/be8tbVJ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jun 28, 2014 at 01:37:20PM +0200, Palle Girgensohn wrote: >=20 >=20 > > 28 jun 2014 kl. 12:21 skrev Konstantin Belousov : > >=20 > >> On Sat, Jun 28, 2014 at 12:08:39PM +0200, Palle Girgensohn wrote: > >>=20 > >>=20 > >>> 27 jun 2014 kl. 18:34 skrev Konstantin Belousov : > >>>=20 > >>>> On Fri, Jun 27, 2014 at 10:57:53AM -0400, John Baldwin wrote: > >>>>> On Friday, June 27, 2014 8:56:13 am Konstantin Belousov wrote: > >>>>> Hi, > >>>>> I did some measurements and hacks to see about the performance and > >>>>> scalability of PostgreSQL 9.3 on FreeBSD, sponsored by The FreeBSD > >>>>> Foundation. > >>>>>=20 > >>>>> The results are described in https://kib.kiev.ua/kib/pgsql_perf.pdf. > >>>>> The uncommitted patches, referenced in the article, are available as > >>>>> https://kib.kiev.ua/kib/pig1.patch.txt > >>>>> https://kib.kiev.ua/kib/patch-2 > >>>>=20 > >>>> Did you run the same benchmark on the same hardware with any other O= S's to=20 > >>>> compare results? > >>>=20 > >>> No. > >>>=20 > >>> FWIW, before the failing after the 30 clients is corrected, I do not > >>> think it is much interesting to do such comparision. > >>=20 > >> This is great work! > >>=20 > >> Does anybody know how far back in FreeBSD versions using posix semapho= re instead of sysv would make a difference? It seems we need a rather curr= ent version? 8.x did not support it at all, at some point at lest, and in 9= it was buggy. I could add he patch-2 to the port, but I reckon it needs a = conditional based on FreeBSD version? > > I recommend to add it as an option. The currently supported versions > > of stable/9 and higher have new posix semaphores implementation. > > The stable/8 also has posix semaphores, but there it is kernel-based > > interface, I do not plan to evaluate it in any way. >=20 > According to one source, posix semaphores uses O(N^2) file descriptors, w= here N is the number of connections. Do you know if this is true? (I'll try= it, naturally, just checking).=20 >=20 (New) posix semaphores implementation, done by David Xu, opens a file descriptor during the sem_open(3), which is used to mmap the area carrying the lock word, and is immediately closed afterward in sem_open(). In other words, if you have N semaphores and M processes, there would be N*M open(2)/close(2) pairs, and N files, each mmaped to M processes. New implementation does not use file descriptor during semaphore use, and does not keep the backing file open. > >=20 > >=20 > >> The clang bug should go upstreams, right? > > I believe there is already some activity about it. I do not follow > > clang development. >=20 > Sounds good enough.=20 >=20 > >=20 > >>=20 > >> I have seen similar curves, presented by Greg Smith (PostgreSQL > >> hacker) where he concluded that there is no point in running more > >> than 50 concurrent connections. This was for Linux. In your measures, > >> the knee is at 30. That's said, FreeBSD could and should do better, > >> but probably there is a limit where there will be a knee in the graph > >> and performance will drop. It should be more than 30, though, as you > >> rightly commented. > >>=20 > >> Do you any ideas to pursue this further apart from complicated > >> rewrites like DragonFly? > > I do. > >=20 > > The scope of the current work was done to obtain understanding where do= we stay > > and, if possible, evaluate ideas, possibly in the hackish way. I hope > > and almost sure that this will be continued, but cannot provide any time > > estimation. >=20 > Great. If you need help testing, I might be able to help.=20 I have the test set up and the graphing mostly automated, although the repeat of the configuration would be quite laborous. On the other hand, if you get access to zoo, replication could be easier. --xqJRvRgB/be8tbVJ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJTru5DAAoJEJDCuSvBvK1B4R0P/2hZk6AgtyDcvVnrq135PNDn JeGNzD2cMIa8rYVc3UMAUCar8VzFXPxgNSKJSKhEEEfonD/BS94f4J75vC6okUiO b+5CidshF4KWElXAv6vzA5xnPverGzqeHpU6LRDGRotkgSUH2fgQ3EiR4g5a2kTC 4SZpRcUW708HoGHsvl0ivrBgQRIX0r6isXiUIJntG5OrbGUa/2VlVqlmIuR6Ckw5 +Vao8HAOL0JENuu1PPdzp0MWtZ0O+Klkt0vMcNN4SR7ePFHGT95OSAg+XG5rfSi+ 7fRQmmT5MjimnArCP2nQkBVBvI4XRsvCoo2n02e+A2OkPRYJNGyY7pWzY39/OPtJ cyfLkaxR3PidGSFxrU5N4YFQwgfsP/rBbC4+07Tskf6rGoRelUOmjbMkobyHB7JC Gfa5QeTVNYL+6CbVjD4Y9o5FPof+FUHi2ul4kcGcJCtprfvrx9h9FjBa0NT2URrD CMtdF7vA+/pVsJ6K22x7/QSVnpLc5tLpOWi8ULCdnliOANlZMWLZjci+EhhkC5sa GX6duPXf8zSHifb+AaPwjF/XAGzD90zLBe2SEIAHMs6Q179LRBKN5N+ub/JyoVY8 5PMKpDbV2YCJMcyvkDXztDWJ0gv+C+OXeHywdwAW6kUOP3Pe/7713Yq8nERKC+Tk 8Epwei1gXgeP+0ZG7Bix =YV3S -----END PGP SIGNATURE----- --xqJRvRgB/be8tbVJ-- From owner-freebsd-performance@FreeBSD.ORG Mon Jun 30 23:39:51 2014 Return-Path: Delivered-To: performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C57FED20; Mon, 30 Jun 2014 23:39:51 +0000 (UTC) Received: from mailuogwhop.emc.com (mailuogwhop.emc.com [168.159.213.141]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mailuogwprd01.lss.emc.com", Issuer "RSA Corporate Server CA v2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7403622B7; Mon, 30 Jun 2014 23:39:51 +0000 (UTC) Received: from maildlpprd06.lss.emc.com (maildlpprd06.lss.emc.com [10.253.24.38]) by mailuogwprd01.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s5UNJEpH006353 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 30 Jun 2014 19:19:14 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd01.lss.emc.com s5UNJEpH006353 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=isilon.com; s=jan2013; t=1404170354; bh=Bde1GjU26eP1xNvV3fpwGHrQoPE=; h=From:To:CC:Subject:Date:Message-ID:References:In-Reply-To: Content-Type:Content-Transfer-Encoding:MIME-Version; b=p+kMmU4OM2fl1xjeZ9JmkNDy57qCowYkefrDYrUH+TjwZSHEjDn2X4stKq1CHelZU 1/mNF32eaHmwVykFswdhXryA6ABIii1tV1nYRtZ/rPcnljrbIxzzApWyHSXXhmso+P jK4U/u2pXoKJm36bY7QeooIFyf4zdA0LFAy8fTDY= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd01.lss.emc.com s5UNJEpH006353 Received: from mailusrhubprd51.lss.emc.com (mailusrhubprd51.lss.emc.com [10.106.48.24]) by maildlpprd06.lss.emc.com (RSA Interceptor); Mon, 30 Jun 2014 16:18:57 -0700 Received: from mxhub16.corp.emc.com (mxhub16.corp.emc.com [128.222.70.237]) by mailusrhubprd51.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s5UNIuFB010951 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Mon, 30 Jun 2014 19:18:56 -0400 Received: from MXHUB102.corp.emc.com (10.253.58.15) by mxhub16.corp.emc.com (128.222.70.237) with Microsoft SMTP Server (TLS) id 8.3.327.1; Mon, 30 Jun 2014 19:18:56 -0400 Received: from MX104CL01.corp.emc.com ([169.254.7.16]) by MXHUB102.corp.emc.com ([::1]) with mapi id 14.03.0181.006; Mon, 30 Jun 2014 19:18:55 -0400 From: "Rang, Anton" To: Konstantin Belousov , "performance@freebsd.org" Subject: RE: PostgreSQL performance on FreeBSD Thread-Topic: PostgreSQL performance on FreeBSD Thread-Index: AQHPkgc47QChxXd+UUuS516FdEccE5uKTpdQ Date: Mon, 30 Jun 2014 23:18:55 +0000 Message-ID: References: <20140627125613.GT93733@kib.kiev.ua> In-Reply-To: <20140627125613.GT93733@kib.kiev.ua> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.13.52.17] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd51.lss.emc.com X-RSA-Classifications: DLM_1, public X-Mailman-Approved-At: Tue, 01 Jul 2014 00:30:54 +0000 Cc: "current@freebsd.org" X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jun 2014 23:39:52 -0000 Thanks for this. The cpu_search problem you reference came up here at Isilon as well. Here'= s a patch which should get clang to do the "right thing" (inlining 3 specia= lized copies of cpu_search); I haven't checked to make sure it doesn't hurt= gcc, though. Anton Index: sched_ule.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- sched_ule.c (revision 268043) +++ sched_ule.c (working copy) @@ -622,11 +622,11 @@ for ((cpu) =3D 0; (cpu) <=3D mp_maxid; (cpu)++) \ if (CPU_ISSET(cpu, &mask)) =20 -static __inline int cpu_search(const struct cpu_group *cg, struct cpu_sear= ch *low, +static __always_inline int cpu_search(const struct cpu_group *cg, struct c= pu_search *low, struct cpu_search *high, const int match); -int cpu_search_lowest(const struct cpu_group *cg, struct cpu_search *low); -int cpu_search_highest(const struct cpu_group *cg, struct cpu_search *high= ); -int cpu_search_both(const struct cpu_group *cg, struct cpu_search *low, +int __noinline cpu_search_lowest(const struct cpu_group *cg, struct cpu_se= arch *low); +int __noinline cpu_search_highest(const struct cpu_group *cg, struct cpu_s= earch *high); +int __noinline cpu_search_both(const struct cpu_group *cg, struct cpu_sear= ch *low, struct cpu_search *high); =20 /* @@ -640,7 +640,7 @@ * match argument. It is reduced to the minimum set for each case. It is * also recursive to the depth of the tree. */ -static __inline int +static __always_inline int cpu_search(const struct cpu_group *cg, struct cpu_search *low, struct cpu_search *high, const int match) { -----Original Message----- From: owner-freebsd-current@freebsd.org [mailto:owner-freebsd-current@freeb= sd.org] On Behalf Of Konstantin Belousov Sent: Friday, June 27, 2014 7:56 AM To: performance@freebsd.org Cc: current@freebsd.org Subject: PostgreSQL performance on FreeBSD Hi, I did some measurements and hacks to see about the performance and scalabil= ity of PostgreSQL 9.3 on FreeBSD, sponsored by The FreeBSD Foundation. The results are described in https://kib.kiev.ua/kib/pgsql_perf.pdf. The uncommitted patches, referenced in the article, are available as https:= //kib.kiev.ua/kib/pig1.patch.txt https://kib.kiev.ua/kib/patch-2 From owner-freebsd-performance@FreeBSD.ORG Tue Jul 8 10:59:08 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EC9635F3 for ; Tue, 8 Jul 2014 10:59:08 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A309B2526 for ; Tue, 8 Jul 2014 10:59:08 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1X4T6y-00018K-Cr for freebsd-performance@freebsd.org; Tue, 08 Jul 2014 12:59:00 +0200 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 08 Jul 2014 12:59:00 +0200 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 08 Jul 2014 12:59:00 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-performance@freebsd.org From: Ivan Voras Subject: Re: PostgreSQL performance on FreeBSD Date: Tue, 08 Jul 2014 12:58:49 +0200 Lines: 39 Message-ID: References: <20140627125613.GT93733__16271.4854618118$1403873797$gmane$org@kib.kiev.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="X1RaXo0XW8M5KXax3u3jfpSV4sWvrvig7" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 In-Reply-To: <20140627125613.GT93733__16271.4854618118$1403873797$gmane$org@kib.kiev.ua> X-Enigmail-Version: 1.6 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 10:59:09 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --X1RaXo0XW8M5KXax3u3jfpSV4sWvrvig7 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 27/06/2014 14:56, Konstantin Belousov wrote: > Hi, > I did some measurements and hacks to see about the performance and > scalability of PostgreSQL 9.3 on FreeBSD, sponsored by The FreeBSD > Foundation. >=20 > The results are described in https://kib.kiev.ua/kib/pgsql_perf.pdf. > The uncommitted patches, referenced in the article, are available as > https://kib.kiev.ua/kib/pig1.patch.txt > https://kib.kiev.ua/kib/patch-2 I'm waiting to upgrade some PostgreSQL machines running FreeBSD 9 to FreeBSD 10 - are the patches committed yet / will they be committed for 10.1? --X1RaXo0XW8M5KXax3u3jfpSV4sWvrvig7 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iKYEARECAGYFAlO7zulfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDYxNDE4MkQ3ODMwNDAwMDJFRUIzNDhFNUZE MDhENTA2M0RGRjFEMkMACgkQ/QjVBj3/HSzxCQCdEu8ZPkOzLxRMU+hHnbP9nhIo Eh8AoIzUZAU1eNuiVNjgL5qg3eAbc6si =4Iff -----END PGP SIGNATURE----- --X1RaXo0XW8M5KXax3u3jfpSV4sWvrvig7-- From owner-freebsd-performance@FreeBSD.ORG Tue Jul 15 19:00:16 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0AEFF31F; Tue, 15 Jul 2014 19:00:16 +0000 (UTC) Received: from mail.feld.me (mail.feld.me [66.170.3.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.feld.me", Issuer "Gandi Standard SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7DEAA2845; Tue, 15 Jul 2014 19:00:15 +0000 (UTC) Received: from mail.feld.me (mail.feld.me [66.170.3.6]); by mail.feld.me (OpenSMTPD) with ESMTP id ae4424fa; Tue, 15 Jul 2014 13:53:32 -0500 (CDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=feld.me; h=content-type :mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:sender; s= blargle2; bh=p6RoxLIql1QwD75BRABtNQKN1/g=; b=1AR92nZMndod8CeA3KE +0G1MUUcKTim1IkA2LYSQOCqnec0XgVkt+7OoEbIvQYmiqLiX7ljyA5ZL4w6OACi ddoDngWviKzsd0fBU/DfX/Ff7GTIz9tFMGjd5MfeZHYJuTZeNL0Y4eAnSyurXRE8 emrWeHpF+XQp5opXTS7+mBoyec0mo1IgoRCVc7MI4TGpiCzIf1SGZR3HY3w+zoki aPO0RokdXIexkgzLDSnnq/n+h3DSfmhlePxlFFIWpwiJNWcTwxViaKESj6qpGRzY jhVls+vZSzxMzT6kfOz0Fkzrg2HiUsZ/oCkX02NaNtYBn0FjcYfCmH3OWWkKxHKI kiw== DomainKey-Signature: a=rsa-sha1; c=nofws; d=feld.me; h=content-type :mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:sender; q= dns; s=blargle2; b=aNv1aXnENLjvGmD2RuZh4V7O6PxuUgOmBqm8LeAvOYL/q u7vvn1mpa3+sMmroSq3EpvFknux46C/e4a/Ok93cEQDVz9bkAmVSXk0nUZtGy8LQ nt9gKcsdsXeIqEIAk7cqnN4MQn9R+qZXhLblikLBTcoCTzbWg6eIJXy5i2NRVhlU E6d9HWcZE0gVhSuR3IrhfNfjvQF8LRJTaxshu6xJEv0z7cCdbXsuQqwVm9HWlpgQ kC/4i39iZAc4UeOeZUD2Jsa2LMavMbeoby/3QHbz6wQewiHymQ4AUxjNPj2VKBtx 69Xi/CX2UL2V8SXzAGvHChKUIo4QY/Kj594+K2xvQ== Received: from mail.feld.me (mail.feld.me [66.170.3.6]); by mail.feld.me (OpenSMTPD) with ESMTP id 58d01fd9; Tue, 15 Jul 2014 13:53:32 -0500 (CDT) Received: from feld@feld.me by mail.feld.me (Archiveopteryx 3.2.0) with esmtpa id 1404819669-4187-4185/5/15; Tue, 8 Jul 2014 11:41:09 +0000 Content-Type: text/plain Mime-Version: 1.0 Subject: Re: PostgreSQL performance on FreeBSD From: Mark Felder In-Reply-To: Date: Tue, 8 Jul 2014 06:41:07 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <9D3FE91D-7806-4D89-8158-E54D7050D8D5@FreeBSD.org> References: <20140627125613.GT93733__16271.4854618118$1403873797$gmane$org@kib.kiev.ua> To: Ivan Voras X-Mailer: Apple Mail (2.1878.6) Sender: feld@feld.me Cc: freebsd-performance@freebsd.org X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 19:00:16 -0000 On Jul 8, 2014, at 5:58, Ivan Voras wrote: > On 27/06/2014 14:56, Konstantin Belousov wrote: >> Hi, >> I did some measurements and hacks to see about the performance and >> scalability of PostgreSQL 9.3 on FreeBSD, sponsored by The FreeBSD >> Foundation. >>=20 >> The results are described in https://kib.kiev.ua/kib/pgsql_perf.pdf. >> The uncommitted patches, referenced in the article, are available as >> https://kib.kiev.ua/kib/pig1.patch.txt >> https://kib.kiev.ua/kib/patch-2 >=20 > I'm waiting to upgrade some PostgreSQL machines running FreeBSD 9 to > FreeBSD 10 - are the patches committed yet / will they be committed for > 10.1? >=20 What Postgres version are you running? If you're still on 9.2 I don't = believe it would show a performance degradation. From owner-freebsd-performance@FreeBSD.ORG Tue Jul 15 20:46:25 2014 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CFC5ED21; Tue, 15 Jul 2014 20:46:25 +0000 (UTC) Received: from mail-ob0-x231.google.com (mail-ob0-x231.google.com [IPv6:2607:f8b0:4003:c01::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8E67422CB; Tue, 15 Jul 2014 20:46:25 +0000 (UTC) Received: by mail-ob0-f177.google.com with SMTP id wp18so6243147obc.22 for ; Tue, 15 Jul 2014 13:46:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=zMnM9QOXPtVagQ5aOuXf8cm+BMIh2qls2SKpcYGkhNY=; b=lQeRFmV7EHEtgBf5G5S0SNxxwBgwm7Gg0Qp4uEg6CmKRWLuRLZH0bcTl+2k3kxMtWR hc1e4+WMPWGFJ725VzdUEiu5FiUIRu0mRDIbxA9gWIerxTGUU3ukCEoV1ZfDqM7YxVRW uZcwM7Z1FzUd+TMHVC0uSIEzYdPrMNRWLO+A15BLvXMT6OFk2zobADJb1KqZu6UMhbvY MzGZ85E3oYS7pY3y7mgjl+WQa7tCNBQkl/qopfzsccpxkHh/1MGQYc9p0Z6CCZ1Tg+0q bodrCeyXTlcP4PG8Xg/Dj2yXCl67gRKMIg7c8cddClRiELHKXyjgoVRIrZTElHDuZrDW N2aw== X-Received: by 10.182.233.230 with SMTP id tz6mr29243221obc.53.1405457184804; Tue, 15 Jul 2014 13:46:24 -0700 (PDT) MIME-Version: 1.0 Sender: ivoras@gmail.com Received: by 10.202.231.195 with HTTP; Tue, 15 Jul 2014 13:45:44 -0700 (PDT) In-Reply-To: <9D3FE91D-7806-4D89-8158-E54D7050D8D5@FreeBSD.org> References: <20140627125613.GT93733__16271.4854618118$1403873797$gmane$org@kib.kiev.ua> <9D3FE91D-7806-4D89-8158-E54D7050D8D5@FreeBSD.org> From: Ivan Voras Date: Tue, 15 Jul 2014 22:45:44 +0200 X-Google-Sender-Auth: st0lsVey92HvWHWeoJD_-2CWFNM Message-ID: Subject: Re: PostgreSQL performance on FreeBSD To: Mark Felder Content-Type: text/plain; charset=UTF-8 Cc: freebsd-performance@freebsd.org X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 20:46:25 -0000 On 8 July 2014 13:41, Mark Felder wrote: > > On Jul 8, 2014, at 5:58, Ivan Voras wrote: >> I'm waiting to upgrade some PostgreSQL machines running FreeBSD 9 to >> FreeBSD 10 - are the patches committed yet / will they be committed for >> 10.1? >> > > What Postgres version are you running? If you're still on 9.2 I don't believe it would show a performance degradation. Since I'm going into downtime for the upgrade anyway, I'm going to upgrade both PostgreSQL and FreeBSD to 10.x at the same time, and since it looks that it would be prudent NOT to upgrade to PostgreSQL 9.3+ from any release of FreeBSD, I'm going to wait for the patches to hit a RELEASE. Btw PostgreSQL 9.4 is just around the corner. From owner-freebsd-performance@FreeBSD.ORG Wed Jul 16 13:29:44 2014 Return-Path: Delivered-To: performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 59C1AA4A; Wed, 16 Jul 2014 13:29:44 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BD0612877; Wed, 16 Jul 2014 13:29:43 +0000 (UTC) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id s6GDTc1E083792 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 16 Jul 2014 16:29:38 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua s6GDTc1E083792 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id s6GDTcmK083791; Wed, 16 Jul 2014 16:29:38 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 16 Jul 2014 16:29:38 +0300 From: Konstantin Belousov To: performance@freebsd.org, current@freebsd.org Subject: Re: PostgreSQL performance on FreeBSD Message-ID: <20140716132938.GB93733@kib.kiev.ua> References: <20140627125613.GT93733@kib.kiev.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="MS56pFM3hjr8joCS" Content-Disposition: inline In-Reply-To: <20140627125613.GT93733@kib.kiev.ua> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 13:29:44 -0000 --MS56pFM3hjr8joCS Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jun 27, 2014 at 03:56:13PM +0300, Konstantin Belousov wrote: > Hi, > I did some measurements and hacks to see about the performance and > scalability of PostgreSQL 9.3 on FreeBSD, sponsored by The FreeBSD > Foundation. >=20 > The results are described in https://kib.kiev.ua/kib/pgsql_perf.pdf. > The uncommitted patches, referenced in the article, are available as > https://kib.kiev.ua/kib/pig1.patch.txt > https://kib.kiev.ua/kib/patch-2 A followup to the original paper. Most importantly, I identified the cause for the drop on the graph after the 30 clients, which appeared to be the debugging version of malloc(3) in libc. Also there are some updates on the patches. New version of the paper is available at https://www.kib.kiev.ua/kib/pgsql_perf_v2.0.pdf The changes are marked as 'update for version 2.0'. --MS56pFM3hjr8joCS Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJTxn5CAAoJEJDCuSvBvK1Bv2YP/2CLnhKXwVqvqY314KyRc2mN dD9JM6lQrzzG59MmEAWjdwmSI04OCJ77etczzbrpPtay9aQYWGitYOtqumAPVGSa g3nV/IW1orZPaZESlbFepSwqR8P56FEWXREt5VA/BbFIdHgJAqjIQTxxSveoQY1u htY8R6BDzww6e7qDFMnUP/sL6MOnnEkz2XJimZT/D2w2ve0Z5A12wpqy336p40bH utaM44RQwIMBKoTLKlmRhGfwCrd6GCRBvvEYFgAWyjcZqxeUARS5i1h1PyiduQ+w kWNY9Y1IjUAC7PCRQ92/DoNXlZVI6peOSuDZF6lYA0inTMwNDqcWlQBlujk77mzu gUG9fp1Cy5dp01LiLJHQBdi8hnxkEfZ6ufm6jdttaVcPGmPAqSNyB70v80QpOg/C 8hMtRG080u4fpEsvm0qvHdTHK/+LoIZ+Mr6FZYUij26BxFABzOD/lP7yXvplwsWV ac7MkfvMgltG1U1MChqb5bwNc6qDgaOJOVQtgMD9wQNU29J94+fkL+I5xYUDjwQZ +LREG8yz3a082aujy4coT+HYyRFnzsYlM6ixjISBqoeG1gZ1EPf2UnpmrK/baGSh xq9+08bnW8lshgRn/qqrOczIWPQm6ONYRVEgbdn7dLCWxtRw7mNbj36FORXCK6kU 0yzHzfqqz7CBbqXfiqSR =X0m2 -----END PGP SIGNATURE----- --MS56pFM3hjr8joCS-- From owner-freebsd-performance@FreeBSD.ORG Wed Jul 16 17:52:47 2014 Return-Path: Delivered-To: performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3A77DABA; Wed, 16 Jul 2014 17:52:47 +0000 (UTC) Received: from mail-qc0-x236.google.com (mail-qc0-x236.google.com [IPv6:2607:f8b0:400d:c01::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DC75B217C; Wed, 16 Jul 2014 17:52:46 +0000 (UTC) Received: by mail-qc0-f182.google.com with SMTP id r5so1101075qcx.13 for ; Wed, 16 Jul 2014 10:52:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=tbZl6MpAurodEO9oy5ctH4e7jfjLNzJO/2pF+5Hv2as=; b=VGBR2FJvgeUUcqO6TUWYU55fiGbZXaPooeQbuwv4zl7BHWZPVD65RtedTuCSGvb6wQ mX1s2f+vTc3du+tDFi/0F9hlJWbaRPOE8+8sdLZiI/WrGXwPt1tH6myh1I9+LcK1iejG QSn5YnTFXgFa1eO0Dy7nx1w9jSxj6Y6ffD8Txqgsd6Mwr6aCkSgaeBs9McuFG0X91A1/ P8xdaglBnLvu2HXqLMcEebSvywGg3iR1BFkXLrEtejSpf6PT2RXMB31HpePIbQNBbzIN blruwVmIsjeTZr6jMlfI0RrR/80SiDepFUXWLRVUALNVcNdKsJO6FSbJOIKKAS9AWGtb yCjQ== MIME-Version: 1.0 X-Received: by 10.140.22.134 with SMTP id 6mr45383977qgn.4.1405533166037; Wed, 16 Jul 2014 10:52:46 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Wed, 16 Jul 2014 10:52:45 -0700 (PDT) In-Reply-To: <20140716132938.GB93733@kib.kiev.ua> References: <20140627125613.GT93733@kib.kiev.ua> <20140716132938.GB93733@kib.kiev.ua> Date: Wed, 16 Jul 2014 10:52:45 -0700 X-Google-Sender-Auth: lJXcZq-CeDhZh07j61Osw64Q0d0 Message-ID: Subject: Re: PostgreSQL performance on FreeBSD From: Adrian Chadd To: Konstantin Belousov Content-Type: text/plain; charset=UTF-8 Cc: performance@freebsd.org, "current@freebsd.org" X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 17:52:47 -0000 Hi! On 16 July 2014 06:29, Konstantin Belousov wrote: > On Fri, Jun 27, 2014 at 03:56:13PM +0300, Konstantin Belousov wrote: >> Hi, >> I did some measurements and hacks to see about the performance and >> scalability of PostgreSQL 9.3 on FreeBSD, sponsored by The FreeBSD >> Foundation. >> >> The results are described in https://kib.kiev.ua/kib/pgsql_perf.pdf. >> The uncommitted patches, referenced in the article, are available as >> https://kib.kiev.ua/kib/pig1.patch.txt >> https://kib.kiev.ua/kib/patch-2 > > A followup to the original paper. > > Most importantly, I identified the cause for the drop on the graph > after the 30 clients, which appeared to be the debugging version > of malloc(3) in libc. > > Also there are some updates on the patches. > > New version of the paper is available at > https://www.kib.kiev.ua/kib/pgsql_perf_v2.0.pdf > The changes are marked as 'update for version 2.0'. Would you mind trying a default (non-PRODUCTION) build, but with junk filling turned off? adrian@adrian-hackbox:~ % ls -l /etc/malloc.conf lrwxr-xr-x 1 root wheel 10 Jun 24 04:37 /etc/malloc.conf -> junk:false That fixes almost all of the malloc debug performance issues that I see without having to recompile. I'd like to know if you see any after that. Thanks! -a