From owner-freebsd-questions@FreeBSD.ORG Mon Mar 21 08:50:44 2011 Return-Path: Delivered-To: questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9D133106566C for ; Mon, 21 Mar 2011 08:50:44 +0000 (UTC) (envelope-from gandalf@shopzeus.com) Received: from shopzeus.com (135-shost.hostoffice.hu [195.228.74.135]) by mx1.freebsd.org (Postfix) with ESMTP id 329B28FC08 for ; Mon, 21 Mar 2011 08:50:43 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by shopzeus.com (Postfix) with ESMTP id 1B6F0889739E; Mon, 21 Mar 2011 04:35:24 -0400 (EDT) Message-ID: <4D870DCD.9000404@shopzeus.com> Date: Mon, 21 Mar 2011 09:35:25 +0100 From: Laszlo Nagy User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; hu; rv:1.9.2.15) Gecko/20110303 Thunderbird/3.1.9 MIME-Version: 1.0 To: questions@freebsd.org, Daniel Fekete Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit X-shopzeus-MailScanner-Information: Please contact the ISP for more information X-shopzeus-MailScanner-ID: 1B6F0889739E.A8208 X-shopzeus-MailScanner: Found to be clean X-shopzeus-MailScanner-From: gandalf@shopzeus.com X-Spam-Status: No Cc: Subject: fstat bug? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Mar 2011 08:50:44 -0000 Hi All, I have a Python program that goes up to 100% CPU. Just like this (top): PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 80212 user1 2 44 0 70520K 16212K select 1 0:30 100.00% /usr/local/bin/python process_updates_ss_od.py -l 10 I have added extra logs and it turns out that there are two threads. One thread is calling "time.sleep()" and the other is calling "os.stat" call. (Actually it is calling os.path.isfile, but I hunted down the last link in the chain.) The most interesting thing is that the process is in "SELECT" state. As far as I know, CPU load should be 0% because "select" state should block program execution until the I/O completes. I must also tell you that the os.stat call is taking long because this system has about 7 million files on a slow disk under heavy load. It would be normal for an os.stat call to return after 10 seconds. I have no problem with that. But I think that the 100% CPU is not acceptable. I guess that the code is running a system call in kernel mode. I think this because I can send a KILL signal to it and the state changes to the following: PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 80212 user1 2 44 0 70520K 15256K STOP 5 1:27 100.00% /usr/local/bin/python process_updates_ss_od.py -l 10 So the state of the process changes to "STOP", but the program does not stop until the os.stat call returns back. Sometimes for a minute. Could it be a problem with the operation system? Is it possible that an os.stat call requires 100% CPU power from the OS? (I believe that os.stat ends in fstat()). Or is it a problem with the Python implementation? (Unfortunately I cannot give you an example program. Giving an example would require giving you a slow I/O device with millions of files on it.) OS version: FreeBSD 8.1-STABLE amd64 Python version: 2.6.6 Thanks, Laszlo -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.