Date: Fri, 21 Feb 2003 01:56:56 -0800 From: Terry Lambert <tlambert2@mindspring.com> To: kalts@estpak.ee Cc: Darryl Okahata <darrylo@soco.agilent.com>, current@FreeBSD.ORG Subject: Re: Vinum R5 [was: Re: background fsck deadlocks with ufs2 and big disk] Message-ID: <3E55F7E8.BD5BF990@mindspring.com> References: <20030220200317.GA5136@kevad.internal> <200302202228.OAA03775@mina.soco.agilent.com> <20030221080046.GA1103@kevad.internal>
next in thread | previous in thread | raw e-mail | index | archive | help
Vallo Kallaste wrote: > The crashes and anomalies with filesystem residing on R5 volume were > related to vinum(R5)/softupdates combo. The vinum R5 and system as > a whole were stable without softupdates. Only one problem remained > after disabling softupdates, while being online and user I/O going > on, rebuilding of failed disk corrupt the R5 volume completely. > Don't know is it fixed or not as I don't have necessary hardware at > the moment. The only way around was to quiesce the volume before > rebuilding, umount it, and wait until rebuild finished. I'll suggest > extensive testing cycle for everyone who's going to work with > vinum R5. Concat, striping and mirroring has been a breeze but not > so with R5. I think this is an expected problem with a lot of concatenation, whether through Vinum, GEOM, RAIDFrame, or whatever. This comes about for the same reason that you can't "mount -u" to turn Soft Updates from "off" to "on": Soft Updates does not tolerate dirty buffers for which a dependency does not exist, and will crap out when a pending dirty buffer causes a write. This could be fixed in the "mount -u" case for Soft Updates, and it can also be fixed for Vinum (et. al.). The key is the difference between a "mount -u" vs. a "umount ; mount", which comes down to flushing and invalidating all buffers on the underlying device, e.g.: vn_lock(devvp, LK_EXCLUSIVE | LK_RETRY, p); vinvalbuf(devvp, V_SAVE, NOCRED, p, 0, 0); error = VOP_CLOSE(devvp, ronly ? FREAD : FREAD|FWRITE, FSCRED, p); error = VOP_OPEN(devvp, ronly ? FREAD : FREAD|FWRITE, FSCRED, p); VOP_UNLOCK(devvp, 0, p); ... Basically, after rebuilding, before allowing the mount to proceed, the Vinum (and GEOM and RAIDFRame, etc.) code needs to cause all the pending dirty buffers to be written. This will guarantee that there are no outstanding dirty buffers at mount time, which in turn guarantees that there will be no dirty buffers that the dependency tracking in Soft Updates does not know about. FWIW: I've maintained for over 6 years now that the mount update code should be modified to do this automatically (and provided patches; see early 1997 mailing list archives), essentially turning a "mount -u" into a "umount ; mount", without invalidating outstanding vnodes and in-core inodes or their references (so that open files do not break... they just get all their buffers taken away from them). Of course, the only open files that matter for device layering are the device exporting the layered block store, and the underlying component block stores that make it up (i.e. no open files there). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3E55F7E8.BD5BF990>