From owner-freebsd-questions@FreeBSD.ORG Sat Dec 2 01:35:12 2006 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2A1BC16A407 for ; Sat, 2 Dec 2006 01:35:12 +0000 (UTC) (envelope-from wmoran@collaborativefusion.com) Received: from mx00.pub.collaborativefusion.com (mx00.pub.collaborativefusion.com [206.210.89.199]) by mx1.FreeBSD.org (Postfix) with ESMTP id 797C243C9D for ; Sat, 2 Dec 2006 01:34:54 +0000 (GMT) (envelope-from wmoran@collaborativefusion.com) Received: from working (c-71-60-174-60.hsd1.pa.comcast.net [71.60.174.60]) (AUTH: LOGIN wmoran, TLS: TLSv1/SSLv3,256bits,AES256-SHA) by wingspan with esmtp; Fri, 01 Dec 2006 20:35:10 -0500 id 000564A0.4570D84E.0000075C Date: Fri, 1 Dec 2006 20:35:09 -0500 From: Bill Moran To: Sean Murphy Message-Id: <20061201203509.444401d9.wmoran@collaborativefusion.com> In-Reply-To: <4570C4D6.5030708@calarts.edu> References: <4570C4D6.5030708@calarts.edu> Organization: Collaborative Fusion Inc. X-Mailer: Sylpheed version 2.2.9 (GTK+ 2.10.6; i386-portbld-freebsd6.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-questions@freebsd.org Subject: Re: Soft Updates Help X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Dec 2006 01:35:12 -0000 Sean Murphy wrote: > > I have read up on soft updates and have some questions. > > The way that I am understanding soft updates purpose is to allow file > systems to be mounted dirty after an unclean shutdown of the system. That's not the purpose. The purpose is to improve performance by taking advantage of delayed writes much the way an asynchronous filesystem does, while preventing horrendous data corruption by ordering those writes, much the way a journalling filesystem does. The fact that you can generate filesystem snapshots is a side-benefit. The fact that you can use filesystem snapshots to validate the filesystem after it's been mounted is a further side-benefit. > If this is a safe way to restore consistency why is it not used on /? Because writes are delayed, it's possible for data to be lost in the event of a crash -- it acts like a database, either the entire transaction is committed or it's rolled back, either way, the data is guaranteed not to be corrupt. Also, on heavily used filesystems, softupdates can lead to the filesystem temporarily having less space available than it really does. I.e. you update /kernel, softupdates completely replaces the file with a new one, but the blocks for the old file haven't been reclaimed yet. For a short period, you might have 1 kernel file, but there's 2x that being allocated for it. For these two reasons, / is traditionally _not_ mounted with softupdates enabled, since it's critical to system startup. > If a file system is not heavily written to is it better not to use soft > updates? Weigh the good vs. the bad: *) synchronous mounted filesystem is almost guaranteed to keep your data safe at all times, but is abysmally slow. *) softupdates _may_ lose some data if your system crashes before all writes are flushed, but will never _corrupt_ it. Additionally, you get a LOT better speed. *) Asynchronous is a little faster than softupdates, but it's damn near guaranteed to be corrupt in the event of a crash. > When file systems are mounted dirty and our being used while the > backgound fsck is running on the file systems how does it prevent files > from being lost? It doesn't. It guarantees that your filesystem will always be mountable and never corrupt, but it doesn't guarantee against data loss. Here's a simplified example: Let's say you're saving a big file and the power goes out. When the power comes back on, there are basically 3 states that file can be in: A) It was fully written to disk -- you got lucky. B) Nothing had been written to disk yet -- "data loss" C) It was partially written to disk -- your filesystem is corrupt, you either need to allow a filesystem repair program to fix it (fsck -- or chkdsk on Windows, for example) or you'll have weird problems with it until you do so. Softupdates guarantees against C. It does this by (essentially) writing the file "backwards": 1) it writes all the data to data blocks, and once that's done 2) _then_ it creates a directory entry for the file. If the system crashes between #1 and #2, it looks like B happened, but you never get in scenario C where the filesystem is corrupt and gets more corrupt as you continue to use it. Instead, when fsck runs (in the background) it realizes that there are data blocks in use that don't belong to any file, and it can free them up for the filesystem to use. That's somewhat simplified, but it gives you the basic idea. HTH Bill