From owner-freebsd-fs@FreeBSD.ORG Mon Feb 7 00:38:44 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6D87816A4CE for ; Mon, 7 Feb 2005 00:38:44 +0000 (GMT) Received: from mxfep02.bredband.com (mxfep02.bredband.com [195.54.107.73]) by mx1.FreeBSD.org (Postfix) with ESMTP id B62FA43D2F for ; Mon, 7 Feb 2005 00:38:42 +0000 (GMT) (envelope-from ola.theander@otsystem.com) Received: from c0003 ([82.182.185.144] [82.182.185.144]) by mxfep02.bredband.com with ESMTP id <20050207003841.MKER24781.mxfep02.bredband.com@c0003> for ; Mon, 7 Feb 2005 01:38:41 +0100 From: "Ola Theander" To: Date: Mon, 7 Feb 2005 01:38:42 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.6353 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2527 Thread-Index: AcUMrV+t8lMD3qYOST2tIPJf+7NQNQ== Message-Id: <20050207003841.MKER24781.mxfep02.bredband.com@c0003> Subject: Prblm: software RAID boot, FreeBSD panic. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Feb 2005 00:38:44 -0000 Dear subscribers I'm trying to setup an old Dell Dimension computer as a simple file server in a small network. For this I've purchased two ATA-disks at 200 GB each. The idea is to use the software RAID features of FreeBSD to have some primitive redundancy. The computer doesn't have any RAID controller whatsoever so I'm stuck with just plain software RAID. What I tried to do is to use "atacontroller create mirror ad0 ad1" to create mirroring between the two disks. Then I installed FreeBSD 5.3 stable as usual on ad0. When the installation was finished I booted the OS and edited the /etc/fstab file, replacing all "ad0*" with "ar0*" and rebooted. I.e fstab is changed to this: # Device Mountpoint FStype Options Dump Pass# /dev/ar0s1b none swap sw 0 0 /dev/ar0s1a / ufs rw 1 1 /dev/ar0s1e /tmp ufs rw 2 2 /dev/ar0s1f /usr ufs rw 2 2 /dev/ar0s1d /var ufs rw 2 2 /dev/acd0 /cdrom cd9660 ro,noauto 0 0 At the boot, after changing fstab, the kernel panics and says: ar0: 194xxxxxxxx MB [24792/255/63] Status: READY subdisks: disk0 READY on ad0 at ata0-master disk1 READY on ad1 at ata0-slave mounting root from ufs: /dev/ar0s1a /: bad dir at ino 2 at offset 0: mangled entry . . init: not found in path . panic: no init That's it. Any help with this matter would be greatly appreciated. From reading some posting I figure that I maybe must have a RAID controller, e.g. a Promise, in order to use "atacontrol ." software RAID. In that case, that might be the problem. Would this be the best way to configure software RAID. It seems like there are several different techniques for this e.g. vinum, atacontrol, geom, gmirror etc. Which one of these is the recommended one? Note that I would like to be able to boot the RAID1 array, i.e. I don't want a separate boot hd if it can be avoided. Any help on this matter would be greatly appreciated. Kind regards, Ola Theander From owner-freebsd-fs@FreeBSD.ORG Mon Feb 7 01:11:51 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0918C16A4CE for ; Mon, 7 Feb 2005 01:11:51 +0000 (GMT) Received: from web41203.mail.yahoo.com (web41203.mail.yahoo.com [66.218.93.36]) by mx1.FreeBSD.org (Postfix) with SMTP id 6F42343D31 for ; Mon, 7 Feb 2005 01:11:48 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 37377 invoked by uid 60001); 7 Feb 2005 01:11:48 -0000 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=QUAHgJinTOHXqI94C/pYruUFFNrXdGWnZ5lzwGj2gbxSZeXIf07GI2Ke1aJ2vEYGe7fFx7RpMWuOYEJGcUZNjdQOUFTkz6sc38MkIiuv9/+76O49+E5k54jfujBnZvzP1MEC5pgEQgnrPbQG8b/eSyTHxrdp4UvBXRZec4yWOXc= ; Message-ID: <20050207011148.37375.qmail@web41203.mail.yahoo.com> Received: from [83.129.181.214] by web41203.mail.yahoo.com via HTTP; Sun, 06 Feb 2005 17:11:48 PST Date: Sun, 6 Feb 2005 17:11:48 -0800 (PST) From: Arne "Wörner" To: Ola Theander , freebsd-fs@freebsd.org In-Reply-To: <20050207003841.MKER24781.mxfep02.bredband.com@c0003> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: Re: Prblm: software RAID boot, FreeBSD panic. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Feb 2005 01:11:51 -0000 Hi Ola! --- Ola Theander wrote: > What I tried to do is to use "atacontroller create mirror ad0 > ad1" to create > Have you tried a cmp /dev/ad0s1a /dev/ad1s1a in single user mode with / and /usr mounted read-only? > mirroring between the two disks. Then I installed FreeBSD 5.3 > stable as usual on ad0. > Are you sure, that writes to ad0 are automatically copied to ad1? Maybe you should try to install to ar0? Maybe you should do a dd if=/dev/ad0 of=/dev/ad1 bs=256k > Would this be the best way to configure software RAID. It seems > like there are several different techniques for this e.g. > vinum, atacontrol, geom, > gmirror etc. Which one of these is the recommended one? > I like to use geom_vinum, which works quite fine, if the disk labels are ok... :-) (if the labels are ko, vinum behaves funny.) Bye Arne __________________________________ Do you Yahoo!? Yahoo! Mail - Helps protect you from nasty viruses. http://promotions.yahoo.com/new_mail From owner-freebsd-fs@FreeBSD.ORG Tue Feb 8 13:53:43 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 72D7616A4CE for ; Tue, 8 Feb 2005 13:53:43 +0000 (GMT) Received: from vsmtp3.tin.it (vsmtp3alice.tin.it [212.216.176.143]) by mx1.FreeBSD.org (Postfix) with ESMTP id C92E043D39 for ; Tue, 8 Feb 2005 13:53:42 +0000 (GMT) (envelope-from marco@pizzi.name) Received: from reaper.marcolin.info (62.211.6.220) by vsmtp3.tin.it (7.0.027) id 4200F9A7003695EB for freebsd-fs@freebsd.org; Tue, 8 Feb 2005 14:53:40 +0100 Received: from DAMIEN.pizzi.name (reaper.marcolin.info [192.168.0.253]) by reaper.marcolin.info (8.13.3/8.13.3) with ESMTP id j18Drd6p007495 for ; Tue, 8 Feb 2005 14:53:40 +0100 Message-Id: <6.1.2.0.2.20050208144950.0343e3c0@127.0.0.1> X-Sender: casellone@127.0.0.1 (Unverified) X-Mailer: QUALCOMM Windows Eudora Version 6.1.2.0 Date: Tue, 08 Feb 2005 14:53:39 +0100 To: freebsd-fs@freebsd.org From: Marco Pizzi Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: Swap problem on FreeBSD 5.2.1-RC X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Feb 2005 13:53:43 -0000 Hello, We've installed (a long time ago) the FreeBSD version 5.2.1-RC on a server. We've noticed, at least two times, very bad problems on the swap system; in the detail, we've found the well known error in the messages file: swap_pager: indefinite wait buffer: device: da0s1a The fact strange is that the da0s1a device is not a swap partition! ------------------------------------------------------------------------------------------------------- # Device Mountpoint FStype Options Dump Pass# /dev/da0s1b none swap sw 0 0 /dev/da0s1a / ufs rw 1 1 ------------------------------------------------------------------------------------------------------- Have you ever seen such a problem? Thanks, -- Marco. From owner-freebsd-fs@FreeBSD.ORG Tue Feb 8 13:59:53 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 26D8016A4CE for ; Tue, 8 Feb 2005 13:59:53 +0000 (GMT) Received: from obsecurity.dyndns.org (CPE0050040655c8-CM00111ae02aac.cpe.net.cable.rogers.com [69.199.47.57]) by mx1.FreeBSD.org (Postfix) with ESMTP id E483B43D41 for ; Tue, 8 Feb 2005 13:59:52 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id BCD1A512AF; Tue, 8 Feb 2005 05:59:51 -0800 (PST) Date: Tue, 8 Feb 2005 05:59:51 -0800 From: Kris Kennaway To: Marco Pizzi Message-ID: <20050208135951.GA94269@xor.obsecurity.org> References: <6.1.2.0.2.20050208144950.0343e3c0@127.0.0.1> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Nq2Wo0NMKNjxTN9z" Content-Disposition: inline In-Reply-To: <6.1.2.0.2.20050208144950.0343e3c0@127.0.0.1> User-Agent: Mutt/1.4.2.1i cc: freebsd-fs@freebsd.org Subject: Re: Swap problem on FreeBSD 5.2.1-RC X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Feb 2005 13:59:53 -0000 --Nq2Wo0NMKNjxTN9z Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Feb 08, 2005 at 02:53:39PM +0100, Marco Pizzi wrote: > Hello, >=20 > We've installed (a long time ago) the FreeBSD version 5.2.1-RC on a serve= r. > We've noticed, at least two times, very bad problems on the swap system; = in=20 > the > detail, we've found the well known error in the messages file: >=20 > swap_pager: indefinite wait buffer: device: da0s1a >=20 > The fact strange is that the da0s1a device is not a swap partition! This message usually indicates a problem reading/writing to the disk, i.e. disk failure. You might also like to try updating to 5.3-stable in case it's a kernel bug that was already fixed. Kris --Nq2Wo0NMKNjxTN9z Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (FreeBSD) iD8DBQFCCMXXWry0BWjoQKURAiC+AKCKxLdcBKAZW8aBtIQDs31OsGrH+ACggDvn rio75pDpCglsmCMBGpy0uDQ= =pjlT -----END PGP SIGNATURE----- --Nq2Wo0NMKNjxTN9z-- From owner-freebsd-fs@FreeBSD.ORG Tue Feb 8 15:06:35 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F112C16A4CE for ; Tue, 8 Feb 2005 15:06:35 +0000 (GMT) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2361543D41 for ; Tue, 8 Feb 2005 15:06:34 +0000 (GMT) (envelope-from avg@icyb.net.ua) Received: from oddity (oddity.topspin.kiev.ua [212.40.38.87]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA06623 for ; Tue, 08 Feb 2005 17:06:31 +0200 (EET) (envelope-from avg@icyb.net.ua) Message-ID: <4207DECD.5030403@icyb.net.ua> Date: Mon, 07 Feb 2005 16:34:05 -0500 From: Andriy Gapon User-Agent: Mozilla Thunderbird 1.0RC1 (Windows/20041201) X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=KOI8-U; format=flowed Content-Transfer-Encoding: 7bit Status: RO X-UID: 3846 X-Keywords: ReSent-Date: Tue, 8 Feb 2005 17:06:23 +0200 (EET) Resent-From: Andriy Gapon Resent-To: freebsd-fs@freebsd.org ReSent-Subject: [Fwd: Re: kern/77234: corrupted data is read from UDF filesystem if read starts at non-aligned offset] ReSent-Message-ID: <20050208170623.H21928@oddity.topspin.kiev.ua> cc: Scott Long Subject: [Fwd: Re: kern/77234: corrupted data is read from UDF filesystem if read starts at non-aligned offset] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Feb 2005 15:06:36 -0000 -------- Original Message -------- Subject: Re: kern/77234: corrupted data is read from UDF filesystem if read starts at non-aligned offset Date: Mon, 7 Feb 2005 20:10:24 GMT From: FreeBSD-gnats-submit@FreeBSD.org Reply-To: FreeBSD-gnats-submit@FreeBSD.org, freebsd-bugs@FreeBSD.org To: Andriy Gapon Thank you very much for your problem report. It has the internal identification `kern/77234'. The individual assigned to look at your report is: freebsd-bugs. You can access the state of your problem report at any time via this link: http://www.freebsd.org/cgi/query-pr.cgi?pr=77234 >Category: kern >Responsible: freebsd-bugs >Synopsis: corrupted data is read from UDF filesystem if read starts at non-aligned offset >Arrival-Date: Mon Feb 07 20:10:24 GMT 2005 -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Tue Feb 8 22:37:29 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DB65B16A4CE for ; Tue, 8 Feb 2005 22:37:29 +0000 (GMT) Received: from VARK.MIT.EDU (VARK.MIT.EDU [18.95.3.179]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7D22143D39 for ; Tue, 8 Feb 2005 22:37:29 +0000 (GMT) (envelope-from das@FreeBSD.ORG) Received: from VARK.MIT.EDU (localhost [127.0.0.1]) by VARK.MIT.EDU (8.13.1/8.13.1) with ESMTP id j18MbK4H016075; Tue, 8 Feb 2005 17:37:20 -0500 (EST) (envelope-from das@FreeBSD.ORG) Received: (from das@localhost) by VARK.MIT.EDU (8.13.1/8.13.1/Submit) id j18MbKSg016074; Tue, 8 Feb 2005 17:37:20 -0500 (EST) (envelope-from das@FreeBSD.ORG) Date: Tue, 8 Feb 2005 17:37:20 -0500 From: David Schultz To: Marco Pizzi Message-ID: <20050208223720.GA16034@VARK.MIT.EDU> Mail-Followup-To: Marco Pizzi , freebsd-fs@FreeBSD.ORG References: <6.1.2.0.2.20050208144950.0343e3c0@127.0.0.1> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <6.1.2.0.2.20050208144950.0343e3c0@127.0.0.1> cc: freebsd-fs@FreeBSD.ORG Subject: Re: Swap problem on FreeBSD 5.2.1-RC X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Feb 2005 22:37:30 -0000 On Tue, Feb 08, 2005, Marco Pizzi wrote: > Hello, > > We've installed (a long time ago) the FreeBSD version 5.2.1-RC on a server. > We've noticed, at least two times, very bad problems on the swap system; in > the > detail, we've found the well known error in the messages file: > > swap_pager: indefinite wait buffer: device: da0s1a > > The fact strange is that the da0s1a device is not a swap partition! > > ------------------------------------------------------------------------------------------------------- > # Device Mountpoint FStype Options Dump > Pass# > /dev/da0s1b none swap sw 0 0 > /dev/da0s1a / ufs rw 1 1 As Kris mentioned, this warning means that an I/O took too long, and it usually means you have a failing disk. As for the fact that it reported the wrong partition on the disk, Poul-Henning ``fixed'' the problem in -CURRENT by making the warning less useful. (It no longer tells you which disk is failing at all.) From owner-freebsd-fs@FreeBSD.ORG Thu Feb 10 03:01:21 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 499D216A4CE for ; Thu, 10 Feb 2005 03:01:21 +0000 (GMT) Received: from hosea.tallye.com (joel.tallye.com [216.99.199.78]) by mx1.FreeBSD.org (Postfix) with ESMTP id 26B2D43D41 for ; Thu, 10 Feb 2005 03:01:20 +0000 (GMT) (envelope-from lorenl@alzatex.com) Received: from hosea.tallye.com (hosea.tallye.com [127.0.0.1]) by hosea.tallye.com (8.12.8/8.12.10) with ESMTP id j1A31JGf030061 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 9 Feb 2005 19:01:19 -0800 Received: (from sttng359@localhost) by hosea.tallye.com (8.12.8/8.12.10/Submit) id j1A31JHs030059 for freebsd-fs@freebsd.org; Wed, 9 Feb 2005 19:01:19 -0800 X-Authentication-Warning: hosea.tallye.com: sttng359 set sender to lorenl@alzatex.com using -f Date: Wed, 9 Feb 2005 19:01:19 -0800 From: "Loren M. Lang" To: freebsd-fs@freebsd.org Message-ID: <20050210030119.GD29396@alzatex.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.1i X-GPG-Key: ftp://ftp.tallye.com/pub/lorenl_pubkey.asc X-GPG-Fingerprint: B3B9 D669 69C9 09EC 1BCD 835A FAF3 7A46 E4A3 280C Subject: Journalling FS and Soft Updates comparision X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Feb 2005 03:01:21 -0000 Traditionally, filesystems have been designed with the idea that the data will always be written to disk safely and not much effort was put into making then Journalling Filesystems and Soft Updates are two different techniques designed to solve the problem of keeping data consistent even in the case of a major interruption like a power blackout. Both work solely on the meta data, not the real data. This means increasing a file's size is protected, but not neccessarily the data that's being written. (Does this also mean that the data will be written to free space before the file size is increased so extraneous data won't be left in the file?) Journally works be recording in a special place on the hard drive called the journal every meta data change that it is about to execute before it does it, then it updates all the meta data and finally marks the journal completed. Soft updates are simply a way to order meta data so that it happens in a safe order. An example is moving file a from directory x to directory y would first delete file a from dir x, then add it to dir y. If a crash happens in the middle, then the data becomes lost right? Now this shouldn't be a big deal since it's harmless to anything else, just some free space is eaten up. Since all meta data updates have this same kind of harmless behavior, that why fsck can be done in the background now instead of foreground. Now comparing the two, perfomance wise journalling has an advantage since every group of meta data updates that are written to the journal at the same time can be reordered to optimize the disk performance. The disk head just has to move across the disk in order instead of seeking back and forth. Now this performance is usually lost because the journal is constantly needing to be updated and it probably lies in one small ares of the disk. The other benefit of the journal is very quick fsck times since all it has do to it see what the journal was updating and make sure it all completed. Soft updates still require a full fsck, but since it can be done in the background unlike journalling, it mean even faster startup time, but more cpu and i/o time spent on it. Now if the journal of a journalling fs could be kept somewhere else, say, in some kind of nvram, then journalling might be overall more efficient as far as disk i/o and cpu time than soft updates. I'm mainly just trying to get an understanding of these two techniques, not neccessarily saying one is better. In the real world, it's probably very dependent on many other things like lot of random access vs. sequential, many files and file ops per seconds, vs. mostly read-only with noatime set, etc. -- I sense much NT in you. NT leads to Bluescreen. Bluescreen leads to downtime. Downtime leads to suffering. NT is the path to the darkside. Powerful Unix is. Public Key: ftp://ftp.tallye.com/pub/lorenl_pubkey.asc Fingerprint: B3B9 D669 69C9 09EC 1BCD 835A FAF3 7A46 E4A3 280C From owner-freebsd-fs@FreeBSD.ORG Thu Feb 10 03:23:25 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7464E16A4CE for ; Thu, 10 Feb 2005 03:23:25 +0000 (GMT) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0574543D46 for ; Thu, 10 Feb 2005 03:23:25 +0000 (GMT) (envelope-from scottl@freebsd.org) Received: from [192.168.254.12] (g4.samsco.home [192.168.254.12]) (authenticated bits=0) by pooker.samsco.org (8.13.1/8.13.1) with ESMTP id j1A3NXUP013286; Wed, 9 Feb 2005 20:23:33 -0700 (MST) (envelope-from scottl@freebsd.org) Message-ID: <420AD3A7.3000102@freebsd.org> Date: Wed, 09 Feb 2005 20:23:19 -0700 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7) Gecko/20040514 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "Loren M. Lang" References: <20050210030119.GD29396@alzatex.com> In-Reply-To: <20050210030119.GD29396@alzatex.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-2.8 required=3.8 tests=ALL_TRUSTED autolearn=failed version=3.0.2 X-Spam-Checker-Version: SpamAssassin 3.0.2 (2004-11-16) on pooker.samsco.org cc: freebsd-fs@freebsd.org Subject: Re: Journalling FS and Soft Updates comparision X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Feb 2005 03:23:25 -0000 Loren M. Lang wrote: > Traditionally, filesystems have been designed with the idea that the > data will always be written to disk safely and not much effort was put > into making then > > Journalling Filesystems and Soft Updates are two different techniques > designed to solve the problem of keeping data consistent even in the > case of a major interruption like a power blackout. Both work solely on > the meta data, not the real data. This isn't always true. There are journaling implementations that journal the data as well as the metadata. > This means increasing a file's size > is protected, but not neccessarily the data that's being written. (Does > this also mean that the data will be written to free space before the > file size is increased so extraneous data won't be left in the file?) > Journally works be recording in a special place on the hard drive called > the journal every meta data change that it is about to execute before it > does it, then it updates all the meta data and finally marks the journal > completed. Soft updates are simply a way to order meta data so that it > happens in a safe order. An example is moving file a from directory x to > directory y would first delete file a from dir x, then add it to dir y. > If a crash happens in the middle, then the data becomes lost right? > Part of the reordering of metadata in softupdates involves generating dependency graphs that prevent data loss like this. > Now this shouldn't be a big deal since it's harmless to anything else, > just some free space is eaten up. Since all meta data updates have this > same kind of harmless behavior, that why fsck can be done in the > background now instead of foreground. The theory of softupdates is that whatever metadata made it to disk before shutdown/crash is consistent enough to be trusted after just a quick preen. The rest of the background checking is just to clean up blocks that became unallocated but weren't committed. > > Now comparing the two, perfomance wise journalling has an advantage > since every group of meta data updates that are written to the journal > at the same time can be reordered to optimize the disk performance. The > disk head just has to move across the disk in order instead of seeking > back and forth. Now this performance is usually lost because the > journal is constantly needing to be updated and it probably lies in one > small ares of the disk. The other benefit of the journal is very quick > fsck times since all it has do to it see what the journal was updating > and make sure it all completed. Soft updates still require a full fsck, > but since it can be done in the background unlike journalling, it mean > even faster startup time, but more cpu and i/o time spent on it. Now if > the journal of a journalling fs could be kept somewhere else, say, in > some kind of nvram, then journalling might be overall more efficient as > far as disk i/o and cpu time than soft updates. Performance between softupdates and journalling is still hotly debated, and your statements border on the 'flaimbait' side of the argument. > > I'm mainly just trying to get an understanding of these two techniques, > not neccessarily saying one is better. In the real world, it's probably > very dependent on many other things like lot of random access vs. > sequential, many files and file ops per seconds, vs. mostly read-only > with noatime set, etc. Softupdates really aren't a whole lot different from journalling. Both turn metadata operations into a sequence of ordered atomic updates. The only difference is that journalling writes these updates to the on-disk journal right away and then commits them later on, and softupdates keeps (most of) them in RAM and then commits them later on. You are correct that journalling has a key advantage in that a fsck, either foreground or background, is not strictly required after an unexpected shutdown. For further information, I'd suggest reading: http://www.mckusick.com/softdep/index.html Scott From owner-freebsd-fs@FreeBSD.ORG Thu Feb 10 08:34:29 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6CF3216A4CE for ; Thu, 10 Feb 2005 08:34:29 +0000 (GMT) Received: from ferengi.borderworlds.dk (ferengi.borderworlds.dk [80.166.152.7]) by mx1.FreeBSD.org (Postfix) with ESMTP id E577643D31 for ; Thu, 10 Feb 2005 08:34:28 +0000 (GMT) (envelope-from xi@borderworlds.dk) Received: from borg.borderworlds.dk (localhost [127.0.0.1]) by ferengi.borderworlds.dk (Postfix) with ESMTP id 22F4BB829; Thu, 10 Feb 2005 09:34:27 +0100 (CET) Received: by borg.borderworlds.dk (Postfix, from userid 1001) id E93771143A; Thu, 10 Feb 2005 09:34:26 +0100 (CET) Sender: xi@borderworlds.dk To: "Loren M. Lang" References: <20050210030119.GD29396@alzatex.com> From: Christian Laursen Date: 10 Feb 2005 09:34:26 +0100 In-Reply-To: <20050210030119.GD29396@alzatex.com> Message-ID: <86vf90ak4t.fsf@borg.borderworlds.dk> Lines: 13 User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.3 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii cc: freebsd-fs@freebsd.org Subject: Re: Journalling FS and Soft Updates comparision X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Feb 2005 08:34:29 -0000 "Loren M. Lang" writes: > I'm mainly just trying to get an understanding of these two techniques, > not neccessarily saying one is better. In the real world, it's probably > very dependent on many other things like lot of random access vs. > sequential, many files and file ops per seconds, vs. mostly read-only > with noatime set, etc. Some relevant reading about this: http://www.usenix.org/publications/library/proceedings/usenix2000/general/seltzer.html -- Christian Laursen From owner-freebsd-fs@FreeBSD.ORG Thu Feb 10 08:47:42 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0629F16A4CE; Thu, 10 Feb 2005 08:47:42 +0000 (GMT) Received: from vsmtp12.tin.it (vsmtp12.tin.it [212.216.176.206]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7FD5643D2F; Thu, 10 Feb 2005 08:47:41 +0000 (GMT) (envelope-from marco@pizzi.name) Received: from reaper.marcolin.info (62.211.6.220) by vsmtp12.tin.it (7.0.027) id 41F508DE00A06310; Thu, 10 Feb 2005 09:47:39 +0100 Received: from DAMIEN.pizzi.name (reaper.marcolin.info [192.168.0.253]) by reaper.marcolin.info (8.13.3/8.13.3) with ESMTP id j1A8lbhh001221; Thu, 10 Feb 2005 09:47:39 +0100 Message-Id: <6.1.2.0.2.20050210094537.028a47b8@127.0.0.1> X-Sender: casellone@127.0.0.1 (Unverified) X-Mailer: QUALCOMM Windows Eudora Version 6.1.2.0 Date: Thu, 10 Feb 2005 09:47:37 +0100 To: David Schultz From: Marco Pizzi In-Reply-To: <20050208223720.GA16034@VARK.MIT.EDU> References: <6.1.2.0.2.20050208144950.0343e3c0@127.0.0.1> <20050208223720.GA16034@VARK.MIT.EDU> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed cc: freebsd-fs@FreeBSD.ORG Subject: Re: Swap problem on FreeBSD 5.2.1-RC X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Feb 2005 08:47:42 -0000 Hello, Thanks for the answers. >As Kris mentioned, this warning means that an I/O took too long, >and it usually means you have a failing disk. The problem is that the device is a SCSI 1 array, and the hardware is not showing any damaged disk. Thanks, -- Marco Pizzi. From owner-freebsd-fs@FreeBSD.ORG Thu Feb 10 08:50:11 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3F90316A4CE; Thu, 10 Feb 2005 08:50:11 +0000 (GMT) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.FreeBSD.org (Postfix) with ESMTP id D60E143D46; Thu, 10 Feb 2005 08:50:10 +0000 (GMT) (envelope-from scottl@samsco.org) Received: from [192.168.254.19] (ibook-nai.samsco.home [192.168.254.19]) (authenticated bits=0) by pooker.samsco.org (8.13.1/8.13.1) with ESMTP id j1A8oIJS014534; Thu, 10 Feb 2005 01:50:19 -0700 (MST) (envelope-from scottl@samsco.org) Message-ID: <420B203F.7030000@samsco.org> Date: Thu, 10 Feb 2005 01:50:07 -0700 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.5) Gecko/20041217 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Marco Pizzi References: <6.1.2.0.2.20050208144950.0343e3c0@127.0.0.1> <20050208223720.GA16034@VARK.MIT.EDU> <6.1.2.0.2.20050210094537.028a47b8@127.0.0.1> In-Reply-To: <6.1.2.0.2.20050210094537.028a47b8@127.0.0.1> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-2.8 required=3.8 tests=ALL_TRUSTED autolearn=failed version=3.0.2 X-Spam-Checker-Version: SpamAssassin 3.0.2 (2004-11-16) on pooker.samsco.org cc: freebsd-fs@freebsd.org cc: David Schultz Subject: Re: Swap problem on FreeBSD 5.2.1-RC X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Feb 2005 08:50:11 -0000 Marco Pizzi wrote: > Hello, > > Thanks for the answers. > >> As Kris mentioned, this warning means that an I/O took too long, >> and it usually means you have a failing disk. > > > The problem is that the device is a SCSI 1 array, and the hardware is not > showing any damaged disk. > > Thanks, > > -- Exactly what kind of hardware are you using? Scott From owner-freebsd-fs@FreeBSD.ORG Thu Feb 10 09:11:44 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A311E16A4CE; Thu, 10 Feb 2005 09:11:44 +0000 (GMT) Received: from vsmtp4.tin.it (vsmtp4.tin.it [212.216.176.224]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4820443D62; Thu, 10 Feb 2005 09:11:44 +0000 (GMT) (envelope-from marco@pizzi.name) Received: from reaper.marcolin.info (62.211.6.220) by vsmtp4.tin.it (7.0.027) id 420A62A8000481C0; Thu, 10 Feb 2005 10:11:42 +0100 Received: from DAMIEN.pizzi.name (reaper.marcolin.info [192.168.0.253]) by reaper.marcolin.info (8.13.3/8.13.3) with ESMTP id j1A9BaEK001469; Thu, 10 Feb 2005 10:11:37 +0100 Message-Id: <6.1.2.0.2.20050210095628.028b9ef8@127.0.0.1> X-Sender: casellone@127.0.0.1 (Unverified) X-Mailer: QUALCOMM Windows Eudora Version 6.1.2.0 Date: Thu, 10 Feb 2005 10:11:35 +0100 To: Scott Long From: Marco Pizzi In-Reply-To: <420B203F.7030000@samsco.org> References: <6.1.2.0.2.20050208144950.0343e3c0@127.0.0.1> <20050208223720.GA16034@VARK.MIT.EDU> <6.1.2.0.2.20050210094537.028a47b8@127.0.0.1> <420B203F.7030000@samsco.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed cc: freebsd-fs@FreeBSD.ORG cc: David Schultz Subject: Re: Swap problem on FreeBSD 5.2.1-RC X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Feb 2005 09:11:44 -0000 >Exactly what kind of hardware are you using? The server is a HP ProLiant DL360 G3. The controller is an embedded Smart Array 5i+ with two 72,8GB Ultra320 SCSI 10.000 rpm disks. So, our next step will be to try to put the HP diagnostic utilities under a Linux capability box, in order to discover if there's something wrong with the hardware, and update the FreeBSD to a stable version. It's strange however... Thanks, -- Marco. From owner-freebsd-fs@FreeBSD.ORG Thu Feb 10 22:23:10 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C4D1F16A4CF; Thu, 10 Feb 2005 22:23:10 +0000 (GMT) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3857E43D53; Thu, 10 Feb 2005 22:23:10 +0000 (GMT) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.13.1/8.13.1) id j1AMN8E6047407; Thu, 10 Feb 2005 16:23:08 -0600 (CST) (envelope-from dan) Resent-From: dan@dan.emsphone.com Resent-Date: Thu, 10 Feb 2005 16:23:08 -0600 Resent-Message-ID: <20050210222308.GA47039@dan.emsphone.com> Resent-To: Scott Long , lorenl@alzatex.com, freebsd-fs@freebsd.org Date: Thu, 10 Feb 2005 16:21:15 -0600 From: Dan Nelson To: Scott Long Message-ID: <20050210222114.GA40240@dan.emsphone.com> References: <20050210030119.GD29396@alzatex.com> <420AD3A7.3000102@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <420AD3A7.3000102@freebsd.org> X-OS: FreeBSD 5.3-STABLE X-message-flag: Outlook Error User-Agent: Mutt/1.5.7i Subject: Re: Journalling FS and Soft Updates comparision X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Feb 2005 22:23:10 -0000 In the last episode (Feb 10), Scott Long said: > > journal completed. Soft updates are simply a way to order meta > > data so that it happens in a safe order. An example is moving file > > a from directory x to directory y would first delete file a from > > dir x, then add it to dir y. If a crash happens in the middle, then > > the data becomes lost right? > > Part of the reordering of metadata in softupdates involves generating > dependency graphs that prevent data loss like this. Also, note that softupdates' only job is to guarantee that the on-disk filesystem layout is consistent. There is no guarantee that multiple independant metadata changes will be written to disk in the same order they were made. Due to the consistency rule, adding inodes to directories is done first for things like file deletions, and last for file creation. A directory tree with lots of file creation and deletion happening in it at the time of a reboot will tend to lose files (relative to a userland application's view of the fs) when it is remounted. > Softupdates really aren't a whole lot different from journalling. > Both turn metadata operations into a sequence of ordered atomic > updates. The only difference is that journalling writes these > updates to the on-disk journal right away and then commits them later > on, and softupdates keeps (most of) them in RAM and then commits them > later on. This is the big drawback to softupdates for me. You run the risk of losing the last `sysctl kern.metadelay` seconds worth of files, because of all the state that softupdates keeps in RAM. -- Dan Nelson dnelson@allantgroup.com From owner-freebsd-fs@FreeBSD.ORG Thu Feb 10 22:56:32 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E566916A4CF; Thu, 10 Feb 2005 22:56:32 +0000 (GMT) Received: from mail.vicor-nb.com (bigwoop.vicor-nb.com [208.206.78.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 24F1143D46; Thu, 10 Feb 2005 22:56:32 +0000 (GMT) (envelope-from julian@elischer.org) Received: from elischer.org (julian.vicor-nb.com [208.206.78.97]) by mail.vicor-nb.com (Postfix) with ESMTP id 139547A424; Thu, 10 Feb 2005 14:56:32 -0800 (PST) Message-ID: <420BE69F.6070009@elischer.org> Date: Thu, 10 Feb 2005 14:56:31 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.3.1) Gecko/20030516 X-Accept-Language: en, hu MIME-Version: 1.0 To: Dan Nelson References: <20050210030119.GD29396@alzatex.com> <420AD3A7.3000102@freebsd.org> <20050210222114.GA40240@dan.emsphone.com> In-Reply-To: <20050210222114.GA40240@dan.emsphone.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit cc: Scott Long cc: fs@freebsd.org Subject: Re: Journalling FS and Soft Updates comparision X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Feb 2005 22:56:33 -0000 uh.. isn't this backwards? Dan Nelson wrote: > >This is the big drawback to softupdates for me. You run the risk of >losing the last `sysctl kern.metadelay` seconds worth of files, because >of all the state that softupdates keeps in RAM. > > kern.filedelay: 30 kern.dirdelay: 29 kern.metadelay: 28 according to this we write the metadata first and then the directory block and then the file data.. This is just wrong. In softupdates you want to have your data down before you do your metadata write, and you need to have your directory block written last of all. When I committed the softupdates code for Kirk, I think that the numbers were: data in 10 seconds, metadata in 15 and directory block in 20. (from memory). From owner-freebsd-fs@FreeBSD.ORG Fri Feb 11 12:39:20 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0C78716A4CE; Fri, 11 Feb 2005 12:39:20 +0000 (GMT) Received: from mailout2.pacific.net.au (mailout2.pacific.net.au [61.8.0.85]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5F2B243D45; Fri, 11 Feb 2005 12:39:19 +0000 (GMT) (envelope-from bde@zeta.org.au) Received: from mailproxy2.pacific.net.au (mailproxy2.pacific.net.au [61.8.0.87])j1BCd7Hn017251; Fri, 11 Feb 2005 23:39:07 +1100 Received: from epsplex.bde.org (katana.zip.com.au [61.8.7.246]) j1BCd4wF032082; Fri, 11 Feb 2005 23:39:05 +1100 Date: Fri, 11 Feb 2005 23:39:03 +1100 (EST) From: Bruce Evans X-X-Sender: bde@epsplex.bde.org To: Julian Elischer In-Reply-To: <420BE69F.6070009@elischer.org> Message-ID: <20050211231020.S1985@epsplex.bde.org> References: <20050210030119.GD29396@alzatex.com> <420AD3A7.3000102@freebsd.org><420BE69F.6070009@elischer.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: Scott Long cc: Dan Nelson cc: fs@freebsd.org Subject: Re: Journalling FS and Soft Updates comparision X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2005 12:39:20 -0000 On Thu, 10 Feb 2005, Julian Elischer wrote: > uh.. isn't this backwards? > > Dan Nelson wrote: > > > > >This is the big drawback to softupdates for me. You run the risk of > >losing the last `sysctl kern.metadelay` seconds worth of files, because > >of all the state that softupdates keeps in RAM. > > kern.filedelay: 30 > kern.dirdelay: 29 > kern.metadelay: 28 > > according to this we write the metadata first and then the directory > block and > then the file data.. The kern.metadata sysctl has nothing to do with the delay for syncing metadata, unlike what its comment says. It controls the delay for syncing files of type VCHR. Since these files have no data to sync, only metadata can be synced for them. In -current and mostly in RELENG_5, since phk has completed or almost completed axing support for device files outside of devfs, these files usually have nothing to sync. The kern.filedelay sysctl contrils the delay for syncing files of type VDIR (directories) and the kern.filedelay sysctl controls the delay for syncing files of type VREG (regular files). > This is just wrong. In softupdates you want to have your data down > before you do > your metadata write, and you need to have your directory block written > last of all. Hopefully this (and correct order for metadata) is controlled by dependencies. The syncer alone can't control the order since files may be synced asynchronously relative to the syncer via fsync(2) or for recycling vnodes, etc. > When I committed the softupdates code for Kirk, I think that the numbers > were: > data in 10 seconds, metadata in 15 and directory block in 20. > (from memory). That commit (vfs_subr.c 1.201, etc.) is hard to find and understand because it was a mega-commit with no details (not even the keyowrd "soft") in its log message. It has only syncer_maxdelay = 32 and syncer_delay = 30, and some magic scaling of these. The current numbers were added in rev.1.201. It is easy to see what the old scaling was in the diff for that: syncdelay / 2 -> dirdelay (default 15 -> 29) syncdelay / 3 -> metadelay ( 10 -> 28) sycdelay -> filedelay ( 30 -> 30) So the numbers have been bunched up but not reordered. Bruce From owner-freebsd-fs@FreeBSD.ORG Sat Feb 12 16:00:37 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 21C9216A4CE for ; Sat, 12 Feb 2005 16:00:37 +0000 (GMT) Received: from mail.freebsd.org.cn (dns3.freebsd.org.cn [61.129.66.75]) by mx1.FreeBSD.org (Postfix) with SMTP id 77A3443D1D for ; Sat, 12 Feb 2005 16:00:28 +0000 (GMT) (envelope-from delphij@frontfree.net) Received: (qmail 9407 invoked by uid 0); 12 Feb 2005 15:51:49 -0000 Received: from unknown (HELO beastie.frontfree.net) (219.239.99.7) by mail.freebsd.org.cn with SMTP; 12 Feb 2005 15:51:49 -0000 Received: from localhost (localhost.frontfree.net [127.0.0.1]) by beastie.frontfree.net (Postfix) with ESMTP id CCDEC135015 for ; Sun, 13 Feb 2005 00:00:18 +0800 (CST) Received: from beastie.frontfree.net ([127.0.0.1]) by localhost (beastie.frontfree.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 94316-12 for ; Sun, 13 Feb 2005 00:00:05 +0800 (CST) Received: by beastie.frontfree.net (Postfix, from userid 1001) id 821BB1339B0; Sun, 13 Feb 2005 00:00:04 +0800 (CST) Date: Sun, 13 Feb 2005 00:00:04 +0800 From: Xin LI To: freebsd-fs@FreeBSD.org Message-ID: <20050212160004.GA94520@frontfree.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="cWoXeonUoKmBZSoM" Content-Disposition: inline User-Agent: Mutt/1.4.2.1i X-GPG-key-ID/Fingerprint: 0xCAEEB8C0 / 43B8 B703 B8DD 0231 B333 DC28 39FB 93A0 CAEE B8C0 X-GPG-Public-Key: http://www.delphij.net/delphij.asc X-Operating-System: FreeBSD beastie.frontfree.net 5.3-RELEASE-p2 FreeBSD 5.3-RELEASE-p2 #15: Wed Dec 15 10:43:16 CST 2004 delphij@beastie.frontfree.net:/usr/obj/usr/src/sys/BEASTIE i386 X-URL: http://www.delphij.net X-By: delphij@beastie.frontfree.net X-Location: Beijing, China X-Virus-Scanned: by amavisd-new at frontfree.net Subject: SoftUpdates's softdep_mount: Why do we need to recompute the superblock summary when bgfsck is there? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2005 16:00:37 -0000 --cWoXeonUoKmBZSoM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, When a FFS is SoftUpdates enabled and was incorrectly dismounted, the mount process will recompute the superblock summary (fs->fs_cstotal) by traversing every cylinder group and accumulate the summary. While this approach will guarantee the correctness of the summary, it can lead to a painful latency of mount() when the volume is very big. With background fsck, I think this can be splited off the softdep_mount() call with the following background fsck process: - When mounting the file system, and when we know that background fsck *would* be run, we skip the recomputation process. (recompute only if no background fsck can be run, or is explicitly specified) - During pass5 of fsck_ffs, the newly generated cstotal is then computed as deltas against the original snapshot's summary. Then, apply the delta to the current number. - Maintain the superblock summary by sync'ing it periodically, so it won't lag behind too much upon a crash. Would this improve the situation? Cheers, --=20 Xin LI http://www.delphij.net/ See complete headers for GPG key and other information. --cWoXeonUoKmBZSoM Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (FreeBSD) iD8DBQFCDigE/cVsHxFZiIoRAkDDAJ9b0yY4D13Lod9y94tSoRcVi6TQVwCfT0mN zwSx4WMZKZ2hDIbxHnbDc1Y= =G+LS -----END PGP SIGNATURE----- --cWoXeonUoKmBZSoM--