From owner-freebsd-performance@FreeBSD.ORG Thu Jun 12 01:07:47 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 437FD1065672 for ; Thu, 12 Jun 2008 01:07:47 +0000 (UTC) (envelope-from felipebgn@gmail.com) Received: from ik-out-1112.google.com (ik-out-1112.google.com [66.249.90.177]) by mx1.freebsd.org (Postfix) with ESMTP id C102C8FC13 for ; Thu, 12 Jun 2008 01:07:46 +0000 (UTC) (envelope-from felipebgn@gmail.com) Received: by ik-out-1112.google.com with SMTP id c30so2699743ika.3 for ; Wed, 11 Jun 2008 18:07:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:mime-version:content-type:content-transfer-encoding :content-disposition; bh=aKH1Rig6dQDz4f1++PEt86kRj6EwSu8u0DZ5TcTMHc0=; b=nhla02c5QL/T+onGHcPGe99VfFoqNB+jfAPhICddA3uV25oK9WJUWtQ43IQh2IFYTE lhrsl7A8axIuKIhiOV1p2vY3/nW/X2RzuZHzM/gCcshva/igS8RRGDT3G0u5dKTONdxZ ZV477aYe3y2BWsylT+ujb+NBjWK3/odazJ9G8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type :content-transfer-encoding:content-disposition; b=I74IVKzf349lI+OJag7u60QJSBN5+MMH1xvsNIDqi3O62voTGEAGdAh8KWV9tG0Hvi iGM5ORarim0kihG86EYef5LGTkkCFPxQLxGGn2r0r4KPHrGO39cgIHDsRbH2Oo+q8fNj 2CnbhkNIvlMto1a6WucfD+cEJ549SEhdB5OrA= Received: by 10.210.19.11 with SMTP id 11mr436168ebs.111.1213231110244; Wed, 11 Jun 2008 17:38:30 -0700 (PDT) Received: by 10.210.19.16 with HTTP; Wed, 11 Jun 2008 17:38:30 -0700 (PDT) Message-ID: <928b5da90806111738h55bbbbb0y3e9731323a1561f4@mail.gmail.com> Date: Wed, 11 Jun 2008 21:38:30 -0300 From: "Felipe Neuwald" To: freebsd-performance@freebsd.org, freebsd-python@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Cc: Subject: Performance with python and FreeBSD 7.0 amd64 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2008 01:07:47 -0000 Hi all, We have a few servers running zope + plone. On one server running FreeBSD 6.3-STABLE i386, I got no problems, but, with one server running FreeBSD 7.0-STABLE amd64, same versions of applications, I got some errors, like the following: dmesg result: pid 74775 (python), uid 1002: exited on signal 11 pid 74861 (python), uid 1002: exited on signal 11 pid 74911 (python), uid 1002: exited on signal 11 pid 74926 (python), uid 1002: exited on signal 11 pid 74970 (python), uid 1002: exited on signal 11 pid 75038 (python), uid 1002: exited on signal 11 pid 75069 (python), uid 1002: exited on signal 11 pid 75095 (python), uid 1002: exited on signal 11 pid 75131 (python), uid 1002: exited on signal 11 pid 75136 (python), uid 1002: exited on signal 11 pid 75204 (python), uid 1002: exited on signal 11 pid 75842 (python), uid 1002: exited on signal 11 pid 75949 (python), uid 1002: exited on signal 10 pid 75962 (python), uid 1002: exited on signal 11 pid 75999 (python), uid 1002: exited on signal 4 pid 76097 (python), uid 1002: exited on signal 10 pid 77452 (python), uid 1002: exited on signal 11 pid 78012 (python), uid 1002: exited on signal 10 pid 78044 (python), uid 1002: exited on signal 11 pid 78425 (python), uid 1002: exited on signal 4 pid 78464 (python), uid 1002: exited on signal 11 pid 78615 (python), uid 1002: exited on signal 11 pid 78638 (python), uid 1002: exited on signal 11 pid 78656 (python), uid 1002: exited on signal 11 pid 78809 (python), uid 1002: exited on signal 11 pid 79076 (python), uid 1002: exited on signal 11 pid 84577 (python), uid 1002: exited on signal 11 I'm using python version 2.4 on every server. So, if someone can help me: - How can I debug the system to find the error? - How can I configure the server for plone + zope (python) best performance? Thank you very much, Felipe Neuwald. From owner-freebsd-performance@FreeBSD.ORG Thu Jun 12 12:40:43 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6A01B106568F; Thu, 12 Jun 2008 12:40:43 +0000 (UTC) (envelope-from andymac@bullseye.apana.org.au) Received: from ipmail01.adl6.internode.on.net (ipmail01.adl6.internode.on.net [203.16.214.146]) by mx1.freebsd.org (Postfix) with ESMTP id C4A4C8FC1D; Thu, 12 Jun 2008 12:40:42 +0000 (UTC) (envelope-from andymac@bullseye.apana.org.au) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AtMEALeyUEg7pyt2/2dsb2JhbACBW6wX X-IronPort-AV: E=Sophos;i="4.27,631,1204464600"; d="scan'208";a="124714950" Received: from ppp59-167-43-118.lns2.cbr1.internode.on.net (HELO bullseye.apana.org.au) ([59.167.43.118]) by ipmail01.adl6.internode.on.net with ESMTP; 12 Jun 2008 21:55:23 +0930 Received: from [192.168.63.10] (tenring.andymac.org [192.168.63.10]) by bullseye.apana.org.au (8.14.2/8.14.2) with ESMTP id m5CCNFeW016382; Thu, 12 Jun 2008 22:23:16 +1000 (EST) (envelope-from andymac@bullseye.andymac.org) Message-ID: <485107C2.7080202@bullseye.andymac.org> Date: Thu, 12 Jun 2008 22:25:54 +1100 From: Andrew MacIntyre User-Agent: Thunderbird 2.0.0.14 (OS/2/20080509) MIME-Version: 1.0 To: Felipe Neuwald References: <928b5da90806111738h55bbbbb0y3e9731323a1561f4@mail.gmail.com> In-Reply-To: <928b5da90806111738h55bbbbb0y3e9731323a1561f4@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Thu, 12 Jun 2008 13:18:10 +0000 Cc: freebsd-performance@freebsd.org, freebsd-python@freebsd.org Subject: Re: Performance with python and FreeBSD 7.0 amd64 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2008 12:40:43 -0000 Felipe Neuwald wrote: > We have a few servers running zope + plone. On one server running > FreeBSD 6.3-STABLE i386, I got no problems, but, with one server > running FreeBSD 7.0-STABLE amd64, same versions of applications, I got > some errors, like the following: > > dmesg result: > pid 74775 (python), uid 1002: exited on signal 11 segmentation violation {...} > pid 75949 (python), uid 1002: exited on signal 10 bus error {...} > pid 75999 (python), uid 1002: exited on signal 4 illegal instruction {...} Hmm... that's an interesting mix of failures. I have seen bus errors when Python runs out of stack space either in the main thread or child threads (not an unknown issue with Zope). gcc 4.x in my limited experience generates sometimes noticeably larger stack frames than gcc 3.x (which is standard on 6.x), which can provoke unexpected stack exhaustion. You don't mention whether you're using a local build or a binary package. Nor do you mention the point release (python 2.4.5 is the most recent in the 2.4 series). The default thread stack size according to my 6.3 box's ports is 1MB for Python 2.4.4) which should be adequate for most circumstances. The illegal instruction failure suggests something wrong with your binaries (including those built for Zope). The segmentation violations often indicate a problem with reference counts, frequently attributable to bugs in 3rd party extensions. You might want to check that all binaries for Python, Zope & Plone (if it has any) link against the same libraries. If you can snaffle cores, you might want to try and extract backtraces from gdb (debugging symbols would make this more productive...) -- ------------------------------------------------------------------------- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac@pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From owner-freebsd-performance@FreeBSD.ORG Thu Jun 12 16:32:25 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BDF80106568C for ; Thu, 12 Jun 2008 16:32:25 +0000 (UTC) (envelope-from felipebgn@gmail.com) Received: from ik-out-1112.google.com (ik-out-1112.google.com [66.249.90.179]) by mx1.freebsd.org (Postfix) with ESMTP id 4DF018FC14 for ; Thu, 12 Jun 2008 16:32:24 +0000 (UTC) (envelope-from felipebgn@gmail.com) Received: by ik-out-1112.google.com with SMTP id c30so2974402ika.3 for ; Thu, 12 Jun 2008 09:32:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references; bh=8tHB4N7dyXI05zA/eWkW5r4+nxp/1cxTiBnXWX9SY9U=; b=sZ8LVlsFYeoti2xJ15co7Z/NPmZuF/VfF3HWo4ep9yGVLtyCEuVvbVYxkR186Xk2KD eb+O8n850/yOtVqiMObMcwq0XZ7VYHwW3qbG70jSVyR+B6AgTY+nOxGI95KDJXvehk+i BkTD4LUBFQko0o+MicbpBktY7PqejmdGlYhx0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references; b=YFyfc1XgOZUXnbtuxxuzoCINqccTF6zJfpGVl3DXQ5AWBh3bgGxbdr35tnfG04a4+5 Q/CezLDrT4swGl/4Z1/1GDRmB2Y/f5iwCuissd3xN14aU6Zcz6Jkpcbwv2+5DeGH6DNE OEG0oEQuNkELxvszjM8SpKSNfJSXQRSFnoDvw= Received: by 10.210.92.8 with SMTP id p8mr1354481ebb.182.1213288343894; Thu, 12 Jun 2008 09:32:23 -0700 (PDT) Received: by 10.210.19.16 with HTTP; Thu, 12 Jun 2008 09:32:23 -0700 (PDT) Message-ID: <928b5da90806120932v49113d35if81b12c45c86c662@mail.gmail.com> Date: Thu, 12 Jun 2008 13:32:23 -0300 From: "Felipe Neuwald" To: "Andrew MacIntyre" In-Reply-To: <485107C2.7080202@bullseye.andymac.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <928b5da90806111738h55bbbbb0y3e9731323a1561f4@mail.gmail.com> <485107C2.7080202@bullseye.andymac.org> Cc: freebsd-performance@freebsd.org, freebsd-python@freebsd.org Subject: Re: Performance with python and FreeBSD 7.0 amd64 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2008 16:32:25 -0000 > > Hmm... that's an interesting mix of failures. > > I have seen bus errors when Python runs out of stack space either in the > main thread or child threads (not an unknown issue with Zope). > > gcc 4.x in my limited experience generates sometimes noticeably larger > stack frames than gcc 3.x (which is standard on 6.x), which can provoke > unexpected stack exhaustion. > > You don't mention whether you're using a local build or a binary package. > Nor do you mention the point release (python 2.4.5 is the most recent in > the 2.4 series). I'm using a local build, installed via ports tree (python24-2.4.5_1). > The default thread stack size according to my 6.3 box's ports is 1MB for > Python 2.4.4) which should be adequate for most circumstances. > > The illegal instruction failure suggests something wrong with your > binaries (including those built for Zope). > > The segmentation violations often indicate a problem with reference > counts, frequently attributable to bugs in 3rd party extensions. > > You might want to check that all binaries for Python, Zope & Plone (if it > has any) link against the same libraries. Ok, I'll try to check these. I'm not the python + zope + plone guy, I'm the FreeBSD administrator. I'll have to work with the application team to find the solution for these problem. > If you can snaffle cores, you might want to try and extract backtraces > from gdb (debugging symbols would make this more productive...) Ok, I'll also try to get more information with cores. Thanks, Felipe Neuwald. From owner-freebsd-performance@FreeBSD.ORG Thu Jun 12 17:57:07 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1E806106567E for ; Thu, 12 Jun 2008 17:57:07 +0000 (UTC) (envelope-from felipebgn@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.175]) by mx1.freebsd.org (Postfix) with ESMTP id A08248FC23 for ; Thu, 12 Jun 2008 17:57:06 +0000 (UTC) (envelope-from felipebgn@gmail.com) Received: by ug-out-1314.google.com with SMTP id q2so417109uge.37 for ; Thu, 12 Jun 2008 10:57:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references; bh=V3+UDLI7uVFCcg7hazI6OpnC6r2xyaWQJZ2oRasdVPI=; b=k7MV7l1iGEnLobDDJzQpGFAdPGBhIhY0aWbct/AOZ/HkmOLbN9O+ck5rF4t++KSJ5L gL4jkZsA4Dgy3q+nuzEh8QyTDzySQh2SJ/W+iifqL2uFK2plUbses46a4v7683FewziX oOR7w0WtkoC8UXL6Jese7seWsPHwBYR+T8ISg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references; b=ZaWThVLtv8zAjNZLUXlsnoqOMHoZnBuwhwCOA7vpk9piIJOgK9jMr2wQlS1BSEeEV5 ODuBk3I7n0fXY5BFT0sT6AzIK/JS/m8+9TaKCG8oBk/KNuc2Ynt0LRoqrh3+xfM8jvrA LJBT7plQ9QKfZX/SSGXLi1BGtJd7UQWAUIKII= Received: by 10.210.49.19 with SMTP id w19mr1515049ebw.11.1213293425231; Thu, 12 Jun 2008 10:57:05 -0700 (PDT) Received: by 10.210.19.16 with HTTP; Thu, 12 Jun 2008 10:57:05 -0700 (PDT) Message-ID: <928b5da90806121057j48c178bdp773adfe759561e15@mail.gmail.com> Date: Thu, 12 Jun 2008 14:57:05 -0300 From: "Felipe Neuwald" To: "Andrew MacIntyre" In-Reply-To: <485107C2.7080202@bullseye.andymac.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <928b5da90806111738h55bbbbb0y3e9731323a1561f4@mail.gmail.com> <485107C2.7080202@bullseye.andymac.org> Cc: freebsd-performance@freebsd.org, freebsd-python@freebsd.org Subject: Re: Performance with python and FreeBSD 7.0 amd64 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2008 17:57:07 -0000 Andrew, I'll try to recompile python with "HUGE STACK SIZE" option. Let's see. Felipe Neuwald. From owner-freebsd-performance@FreeBSD.ORG Fri Jun 13 12:45:53 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F31A61065671 for ; Fri, 13 Jun 2008 12:45:53 +0000 (UTC) (envelope-from felipebgn@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.168]) by mx1.freebsd.org (Postfix) with ESMTP id 7323C8FC1A for ; Fri, 13 Jun 2008 12:45:53 +0000 (UTC) (envelope-from felipebgn@gmail.com) Received: by ug-out-1314.google.com with SMTP id q2so73218uge.37 for ; Fri, 13 Jun 2008 05:45:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references; bh=hje+WtmV4sOu8lQJZPWX+viBhFAe7K/6u7D/CmfWyFI=; b=cIYhNkI4J4yc8qUlsXgxEdHdTohVkdNyN7jgqQvAAzHbAdjpnZgko0DqnAsf/KqKE7 lPa6BWYEe+vUS5pv17C3dG9Ax/YUiKVq2M2J0EAe9KloRQCg9UeG/wU3EPWPaZ3cSF3y WbYWeF7hN8jZrqAaiq3r23fXToVJxwVvxpw2I= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references; b=I+F1Ol8kuI9aH8KHcCbcbEyE5L6JFYefDo75OZITq/Nbmg127DnL4GcPS6AsrIoLsW yBiIMRXKwMXut4Y0NXx7Cx/OgY1yJ4iEvN8bSJcW/smyLsI5AuZhZaFrOvLFRtUFqyge q/t08+OslBfHOADd3dlhRRcJB0w2puIZahVsE= Received: by 10.210.58.13 with SMTP id g13mr2545601eba.54.1213361151745; Fri, 13 Jun 2008 05:45:51 -0700 (PDT) Received: by 10.210.19.16 with HTTP; Fri, 13 Jun 2008 05:45:51 -0700 (PDT) Message-ID: <928b5da90806130545w37621619t20dab5495318459a@mail.gmail.com> Date: Fri, 13 Jun 2008 09:45:51 -0300 From: "Felipe Neuwald" To: "Andrew MacIntyre" In-Reply-To: <928b5da90806121057j48c178bdp773adfe759561e15@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <928b5da90806111738h55bbbbb0y3e9731323a1561f4@mail.gmail.com> <485107C2.7080202@bullseye.andymac.org> <928b5da90806121057j48c178bdp773adfe759561e15@mail.gmail.com> Cc: freebsd-performance@freebsd.org, freebsd-python@freebsd.org Subject: Re: Performance with python and FreeBSD 7.0 amd64 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Jun 2008 12:45:54 -0000 Andrew and all, After recompile python 2.4 with HUGE_STACK_SIZE option, I got no more problems. I'll still wait the weekend to say it again, and wait for the customer reply about system performance / errors. If I got news, I'll send to you. Thank you very much, Felipe Neuwald. 2008/6/12 Felipe Neuwald : > Andrew, I'll try to recompile python with "HUGE STACK SIZE" option. Let's see. > > Felipe Neuwald. >