From owner-freebsd-apache@FreeBSD.ORG Fri Oct 29 20:05:23 2010 Return-Path: Delivered-To: freebsd-apache@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8CA0A106566B for ; Fri, 29 Oct 2010 20:05:23 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta11.emeryville.ca.mail.comcast.net (qmta11.emeryville.ca.mail.comcast.net [76.96.27.211]) by mx1.freebsd.org (Postfix) with ESMTP id 74AB18FC16 for ; Fri, 29 Oct 2010 20:05:23 +0000 (UTC) Received: from omta21.emeryville.ca.mail.comcast.net ([76.96.30.88]) by qmta11.emeryville.ca.mail.comcast.net with comcast id Qitn1f00A1u4NiLABjsDGt; Fri, 29 Oct 2010 19:52:13 +0000 Received: from koitsu.dyndns.org ([98.248.41.155]) by omta21.emeryville.ca.mail.comcast.net with comcast id QjsC1f00G3LrwQ28hjsC4w; Fri, 29 Oct 2010 19:52:12 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 1AD659B425; Fri, 29 Oct 2010 12:52:12 -0700 (PDT) Date: Fri, 29 Oct 2010 12:52:12 -0700 From: Jeremy Chadwick To: freebsd-apache@freebsd.org Message-ID: <20101029195212.GA17998@icarus.home.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Cc: ade@freebsd.org Subject: autoconf 2.68 causing issues with www/apache22 + ITK MPM X-BeenThere: freebsd-apache@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Support of apache-related ports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Oct 2010 20:05:23 -0000 I'm CC'ing ade@ because this really does look like an autoconf problem of some sort, but I wanted to make freebsd-apache@ aware of it. I've spent pretty much the past 4 days off-and-on trying to debug this. I first thought it was a hardware issue (clock skew/drift or ntpd doing something super weird, which commonly is the source of "make" issues), then bothered jilles thinking it was a /bin/sh problem, then dug deeper to find it looks more like an autoconf problem. The problem itself happens "most of the time". Sounds crazy, but true. About 80-90% of the time its broken, but there's occasional windows where it works fine. Better yet: it happens on multiple boxes which have completely different hardware specifications. All machines are reliable, use ECC RAM, show no signs of software segfaults or other anomalies (they can build work 100% reliably). powerd is not in use, and the system clock is in sync, does not skew, and ntpd shows no signs of excessive drift/offset or jitter. We use the ITK MPM (WITH_MPM=itk in make.conf). Use of this feature applies a patch: files/mpm-itk-20090414-00. The patch has not changed in over a year, and it does apply cleanly. The patch has the following change in it: --- orig/server/mpm/config.m4 2007-01-29 21:30:26.000000000 +0100 +++ server/mpm/config.m4 2007-01-29 21:30:35.000000000 +0100 @@ -1,7 +1,7 @@ AC_MSG_CHECKING(which MPM to use) AC_ARG_WITH(mpm, APACHE_HELP_STRING(--with-mpm=MPM,Choose the process model for Apache to use. - MPM={beos|event|worker|prefork|mpmt_os2}),[ + MPM={beos|event|worker|prefork|mpmt_os2|itk}),[ APACHE_MPM=$withval ],[ if test "x$APACHE_MPM" = "x"; then @@ -23,7 +23,7 @@ ap_mpm_is_experimental () { - if test "$apache_cv_mpm" = "event" ; then + if test "$apache_cv_mpm" = "event" -o "$apache_cv_mpm" = "itk" ; then return 0 else return 1 Once this patch is applied, and autoconf is run (I assume), it *should* result in the main "configure" script having those changes. Except most of the time (80-90% of the time) "make clean ; make configure" fails in the configure phase with this message: checking for times... (cached) yes checking which MPM to use... itk configure: error: the selected mpm -- itk -- is not supported ===> Script "configure" failed unexpectedly. When failing, upon reviewing the configure script itself, I found that the ap_mpm_is_experimental function doesn't even have the changes that were applied to server/mpm/config.m4. Validation: icarus# make clean ; make configure {...} checking for times... (cached) yes checking which MPM to use... itk configure: error: the selected mpm -- itk -- is not supported ===> Script "configure" failed unexpectedly. Please report the problem to apache@FreeBSD.org [maintainer] and attach the "/usr/ports/www/apache22/work/httpd-2.2.17/config.log" including the output of the failure of your make command. Also, it might be a good idea to provide an overview of all packages installed on your system (e.g. an `ls /var/db/pkg`). *** Error code 1 Stop in /usr/ports/www/apache22. *** Error code 1 Stop in /usr/ports/www/apache22. icarus# ls -l work/httpd-2.2.17/configure* -rwxr-xr-x 1 root wheel 657359 Oct 29 12:34 work/httpd-2.2.17/configure -rw-r--r-- 1 root wheel 24246 Oct 29 12:34 work/httpd-2.2.17/configure.in -rw-r--r-- 1 root wheel 23953 Dec 1 2008 work/httpd-2.2.17/configure.in.orig icarus# ls -l work/httpd-2.2.17/server/mpm/config* -rw-r--r-- 1 root wheel 2097 Oct 29 12:34 work/httpd-2.2.17/server/mpm/config.m4 -rw-r--r-- 1 root wheel 2022 Oct 29 12:34 work/httpd-2.2.17/server/mpm/config.m4.orig icarus# egrep -n -A 3 '^ap_mpm_is_experimental' work/httpd-2.2.17/configure 17548:ap_mpm_is_experimental () 17549-{ 17550- if test "$apache_cv_mpm" = "event" ; then 17551- return 0 I verified via dmesg that there's no segfaults or indication of hardware issues or disk problems (the filesystem is on an SSD). Remember, this happens on two separate boxes. So what do things look like when it *does* work? icarus# make clean ; make configure {...} checking for times... (cached) yes checking which MPM to use... itk configure: WARNING: You have selected an EXPERIMENTAL MPM. Be warned! checking for cap_init in -lcap... no {...} icarus# ls -l work/httpd-2.2.17/configure* -rwxr-xr-x 1 root wheel 571760 Oct 29 12:36 work/httpd-2.2.17/configure -rw-r--r-- 1 root wheel 24246 Oct 29 12:36 work/httpd-2.2.17/configure.in -rw-r--r-- 1 root wheel 23953 Dec 1 2008 work/httpd-2.2.17/configure.in.orig icarus# ls -l work/httpd-2.2.17/server/mpm/config* -rw-r--r-- 1 root wheel 2097 Oct 29 12:36 work/httpd-2.2.17/server/mpm/config.m4 -rw-r--r-- 1 root wheel 2022 Oct 29 12:36 work/httpd-2.2.17/server/mpm/config.m4.orig icarus# egrep -n -A 3 '^ap_mpm_is_experimental' work/httpd-2.2.17/configure 14706:ap_mpm_is_experimental () 14707-{ 14708- if test "$apache_cv_mpm" = "event" -o "$apache_cv_mpm" = "itk" ; then 14709- return 0 That's ~80KBytes of shell code that exists in the broken one but not in the working one. Umm... :-) Ade and/or Philip, please let me know what I can provide which can help debug this problem (including if you need root-level access to the box via SSH or serial console). If either of you are available on IRC or via some IM method, we can work from there, just contact me off-list. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |