From owner-freebsd-questions@FreeBSD.ORG Sat Dec 18 21:25:00 2004 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E684016A4CE for ; Sat, 18 Dec 2004 21:25:00 +0000 (GMT) Received: from pimout1-ext.prodigy.net (pimout1-ext.prodigy.net [207.115.63.77]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6EDE843D1D for ; Sat, 18 Dec 2004 21:25:00 +0000 (GMT) (envelope-from mark@antsclimbtree.com) Received: from lilbuddy.antsclimbtree.com (adsl-66-122-112-171.dsl.snfc21.pacbell.net [66.122.112.171]) iBILOsDa030168 for ; Sat, 18 Dec 2004 16:24:59 -0500 Received: from antslaptop.antsclimbtree.com ([192.168.1.192]) by lilbuddy.antsclimbtree.com with esmtpsa (TLSv1:RC4-SHA:128) (Exim 4.43 (FreeBSD)) id 1Cfm4F-00014D-Nc for questions@FreeBSD.ORG; Sat, 18 Dec 2004 13:24:54 -0800 Mime-Version: 1.0 (Apple Message framework v619) Content-Transfer-Encoding: 7bit Message-Id: <3CF60F58-513B-11D9-BF6E-000393A5ED5E@antsclimbtree.com> Content-Type: text/plain; charset=US-ASCII; format=flowed To: "'questions@freebsd.org'" From: Mark Edwards Date: Sat, 18 Dec 2004 13:24:45 -0800 X-Mailer: Apple Mail (2.619) X-Spam-Score: -5.9 (-----) Subject: Runaway Apache X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Dec 2004 21:25:01 -0000 In the last week or so, my FreeBSD 4.10p5 server has started locking up every day or so, to the point where it becomes unusable and must be rebooted to resume service. I've noticed that when it happens, the following type of thing appears in /var/log/httpd-error.log [Sat Dec 18 13:00:18 2004] [error] child process 248 still did not exit, sending a SIGKILL [Sat Dec 18 13:00:18 2004] [error] child process 464 still did not exit, sending a SIGKILL [Sat Dec 18 13:00:18 2004] [error] child process 465 still did not exit, sending a SIGKILL [Sat Dec 18 13:00:18 2004] [error] child process 466 still did not exit, sending a SIGKILL [Sat Dec 18 13:00:18 2004] [error] child process 554 still did not exit, sending a SIGKILL [Sat Dec 18 13:00:18 2004] [error] child process 2121 still did not exit, sending a SIGKILL [Sat Dec 18 13:00:18 2004] [error] child process 2126 still did not exit, sending a SIGKILL [Sat Dec 18 13:00:18 2004] [error] child process 2129 still did not exit, sending a SIGKILL [Sat Dec 18 13:00:18 2004] [error] child process 2130 still did not exit, sending a SIGKILL and on and on and on... So, apparently, Apache is having a problem and taking down the server. I eventually also see complaints about user 80 exceeding the kern.maxfiles limit. That's probably when the server really takes a dump. I've been monitoring top periodically to see if I can spot the problem, and an httpd process was consuming 95% of the cpu just now, and sure enough the above messages were streaming through the log. I also notice the following: httpd in free(): warning: chunk is already free httpd in free(): warning: chunk is already free httpd in free(): warning: chunk is already free httpd in free(): warning: chunk is already free httpd in free(): warning: chunk is already free httpd in free(): warning: chunk is already free httpd in free(): warning: chunk is already free Now, my problem is I'm not sure how to find the source of this problem and stop it. A google search on those log entries suggests that it may be an attempt to exploit the Chunk Handling Vulnerability, but my Apache is newer than the fix for that. http://httpd.apache.org/info/security_bulletin_20020617.txt Anyhow, can anyone give me a suggestion on how to troubleshoot this? Thanks! Here is the Apache in question: Server Version: Apache/1.3.33 (Unix) mod_ssl/2.8.22 OpenSSL/0.9.7e PHP/4.3.10 DAV/1.0.3