From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 20 20:55:11 2005 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A721E16A4CF for ; Sun, 20 Mar 2005 20:55:11 +0000 (GMT) Received: from mail.mundomateo.com (static-24-56-193-117.chrlmi.cablespeed.com [24.56.193.117]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2BB2643D41 for ; Sun, 20 Mar 2005 20:55:11 +0000 (GMT) (envelope-from matthew@digitalstratum.com) Received: from [10.0.81.14] (ws14.mundomateo.com [10.0.81.14]) by mail.mundomateo.com (Postfix) with ESMTP id 36F6C60AB for ; Sun, 20 Mar 2005 15:55:09 -0500 (EST) Message-ID: <423DE326.9000203@digitalstratum.com> Date: Sun, 20 Mar 2005 15:55:02 -0500 From: Matthew Hagerty Organization: Digital Stratum User-Agent: Mozilla Thunderbird 1.0 (Windows/20041206) X-Accept-Language: en-us, en MIME-Version: 1.0 To: FreeBSD Hackers Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Causing a process switch to test a theory. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: matthew@digitalstratum.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Mar 2005 20:55:11 -0000 Greetings, I posted a similar message earlier, but I think I was too vague to solicit any kind of response. I'm also trying to stay relevant to the forum, but I can only offer the fact that I'm using FreeBSD as my OS of choice. If this is too far removed from being on topic, I apologize in advance and will take my questions elsewhere. I think I have found a possible bug in Apache's logging when using their "reliable pipe" feature, but I'd like to test it prior to submitting a bug report (or possibly a patch.) Of course I posted a message on the Apache development forum before posting here, but I have had no response from that group. My understanding of PIPE_BUF is that it is the largest amount of data the kernel will guarantee to be atomic when writing to a pipe. Thus if more than one process is writing to the same pipe, and more than PIPE_BUF bytes needs to be written, there is the chance of the data being interleaved due to a context switch during write(), or between multiple calls to write() in order to write all required data. I've been reading the Apache source code to try and determine if PIPE_BUF is taken into consideration while logging entries to a pipe. What appears to happen is that if a single log entry is more than 512 bytes, it is simply written to the the pipe without regard to PIPE_BUF. But there is a lot going on in Apache and I might very easily miss what's really happening somewhere else in the code. I'm pretty sure I can see what's going on and that the code needs to be fixed, but I don't want to go blowing my horn without some proof. What fstat seems to show me is that each child process (I'm running the 1.x branch of Apache) has the pipe open for writing, thus each child logs it's own entries instead of passing them to a single logging process (which could be used as some sort of serialization and control to prevent interleaving the log entries.) In this situation (each child logging it's own entries) it seems there is the possibility that a child could be preempted during it's call to write() when trying to write more than PIPE_BUF bytes of log data. What I'd like to do is create a test where I would be making requests to Apache that would cause log entries longer than PIPE_BUF in length, then be able to show the interleaving of log entries due to the PIPE_BUF limit being exceeded. The 2.x branch of Apache has an experimental features called BufferedLogs, that if used seems to protect the writing of the log entries with a mutex. However, it is not enabled by default and the normal logging does not use the mutex. I would have assumed the PIPE_BUF limit would not have been overlooked by the Apache developers, especially since they don't ignore log entires until they reach 8190 bytes (which is certainly longer than PIPE_BUF.) Basically it comes down to what happens when making this call: write(cls->log_fd, str, len); Under the conditions such that cls->log_fd is a pipe (inherited from the parent), len > PIPE_BUF, and there are multiple child processes all logging entries with this code. Knowing if Apache could possibly write interleaved logs when writing to a pipe is critical to a program I'm developing which receives log entries from Apache via a pipe. Thank you, Matthew