From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 12 15:01:05 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 09BF6106564A for ; Mon, 12 Mar 2012 15:01:05 +0000 (UTC) (envelope-from onwahe@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id C0A6F8FC0C for ; Mon, 12 Mar 2012 15:01:04 +0000 (UTC) Received: by yhgm50 with SMTP id m50so3142118yhg.13 for ; Mon, 12 Mar 2012 08:00:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=UrFWM2eOdKm98zU/IIKOiFjhVL7Zx/4r/oobUPCqkAM=; b=MEDpnG2LHZBeUA4yutng/Z/wBIZzJfOf0prQYNQChb9Vx4QkaVj7jEXoa6UzxKKpA4 inUvxTZyJiDafRmQhoAys9B2aDE8e+4suAiEHBpMxvkxQG3BYTwaK9NbWrYlaqmOOxc7 CDpYK50YbpsAWbEJXQh2a2NXMFGERKaw+t/yxHMVCXTrdK3pPO8AY1wBy0ewp9wvkrtc T4oUFt0eAFk8VMBD0UeVmr+e7bO0ixkvRNKuepT6ovYfG8+XVtfLHrdMToh/Xo+iwJxd OxzsyziT/pchGD4s64jPoRnfkwsAmCZeDg0fZnEDsl2f9mQtXllNTWhqGtdVy0nNoYYG PeTw== MIME-Version: 1.0 Received: by 10.236.177.6 with SMTP id c6mr13586171yhm.42.1331564458798; Mon, 12 Mar 2012 08:00:58 -0700 (PDT) Received: by 10.236.75.162 with HTTP; Mon, 12 Mar 2012 08:00:58 -0700 (PDT) Date: Mon, 12 Mar 2012 16:00:58 +0100 Message-ID: From: Svatopluk Kraus To: hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Subject: [vfs] buf_daemon() slows down write() severely on low-speed CPU X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Mar 2012 15:01:05 -0000 Hi, I have solved a following problem. If a big file (according to 'hidirtybuffers') is being written, the write speed is very poor. It's observed on system with elan 486 and 32MB RAM (i.e., low speed CPU and not too much memory) running FreeBSD-9. Analysis: A file is being written. All or almost all dirty buffers belong to the file. The file vnode is almost all time locked by writing process. The buf_daemon() can not flush any dirty buffer as a chance to acquire the file vnode lock is very low. A number of dirty buffers grows up very slow and with each new dirty buffer slower, because buf_daemon() eats more and more CPU time by looping on dirty buffers queue (with very low or no effect). This slowing down effect is started by buf_daemon() itself, when 'numdirtybuffers' reaches 'lodirtybuffers' threshold and buf_daemon() is waked up by own timeout. The timeout fires at 'hz' period, but starts to fire at 'hz/10' immediately as buf_daemon() fails to reach 'lodirtybuffers' threshold. When 'numdirtybuffers' (now slowly) reaches ((lodirtybuffers + hidirtybuffers) / 2) threshold, the buf_daemon() can be waked up within bdwrite() too and it's much worse. Finally and with very slow speed, the 'hidirtybuffers' or 'dirtybufthresh' is reached, the dirty buffers are flushed, and everything starts from beginning... On the system, a buffer size is 512 bytes and the default thresholds are following: vfs.hidirtybuffers = 134 vfs.lodirtybuffers = 67 vfs.dirtybufthresh = 120 For example, a 2MB file is copied into flash disk in about 3 minutes and 15 second. If dirtybufthresh is set to 40, the copy time is about 20 seconds. My solution is a mix of three things: 1. Suppresion of buf_daemon() wakeup by setting bd_request to 1 in the main buf_daemon() loop. 2. Increment of buf_daemon() fast timeout from hz/10 to hz/4. 3. Tuning dirtybufthresh to (((lodirtybuffers + hidirtybuffers) / 2) - 15) magic. The mention copy time is about 30 seconds now. The described problem is just for information to anyone who can be interested in. Comments are welcome. However, the bd_request thing is more general. bd_request (despite its description) should be 0 only when buf_daemon() is in sleep(). Otherwise, wakeup() on &bd_request channel is useless. Therefore, setting bd_request to 1 in the main buf_daemon() loop is correct and better as it saves time spent by wakeup() on not existing channel. Svata