From owner-freebsd-fs@FreeBSD.ORG Mon Sep 5 06:44:36 2005 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6B33116A41F for ; Mon, 5 Sep 2005 06:44:36 +0000 (GMT) (envelope-from jebc@c4solutions.net) Received: from outbound.mailhop.org (outbound.mailhop.org [63.208.196.171]) by mx1.FreeBSD.org (Postfix) with ESMTP id 119BB43D45 for ; Mon, 5 Sep 2005 06:44:35 +0000 (GMT) (envelope-from jebc@c4solutions.net) Received: from pcp0012210017pcs.blairblvd.tn.nash.comcast.net ([69.245.61.169] helo=ares.c4solutions.net) by outbound.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.51) id 1ECAiZ-0002AJ-Dg for freebsd-fs@freebsd.org; Mon, 05 Sep 2005 02:44:35 -0400 Received: from [10.10.1.115] (jeb-64.internal.c4solutions.net [10.10.1.115]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ares.c4solutions.net (Postfix) with ESMTP id 6E008ADAF7 for ; Mon, 5 Sep 2005 01:44:31 -0500 (CDT) X-Mail-Handler: MailHop Outbound by DynDNS.org X-Originating-IP: 69.245.61.169 X-Report-Abuse-To: abuse@dyndns.org (see http://www.mailhop.org/outbound/abuse.html for abuse reporting information) X-MHO-User: jebutk Message-ID: <431BE951.4070101@c4solutions.net> Date: Mon, 05 Sep 2005 01:44:33 -0500 From: Jeb Campbell User-Agent: Mozilla Thunderbird 1.0.6 (X11/20050816) X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: kernel newb -- read/write to a local file from in kernel X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Sep 2005 06:44:36 -0000 I'm trying to learn about kernel programming and as a weekend project I've started writing a Netgraph to file/dev module. The Netgraph stuff was pretty easy, but I'm at a loss on the file access part as I've never done anything in-kernel. After much searching (most of it not knowing what I was looking for), I think that I'm looking for something like "Vnode interface mechanics, write to a local file" as taught in Mr. McKusick's "FreeBSD Kernel Internals: An Intensive Code Walkthrough". I've looked at nullfs/null_vnops.c, but I think that is more of a pass-through example. Does anyone know of any online code that shows the process of opening a file/device for read/write? Thanks in advance if someone has any tips or more info. (and sorry to bother the kernel hackers with such a newbie question) -- Jeb Campbell jeb@c4solutions.net From owner-freebsd-fs@FreeBSD.ORG Mon Sep 5 06:46:13 2005 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9B8D516A41F for ; Mon, 5 Sep 2005 06:46:13 +0000 (GMT) (envelope-from phk@phk.freebsd.dk) Received: from haven.freebsd.dk (haven.freebsd.dk [130.225.244.222]) by mx1.FreeBSD.org (Postfix) with ESMTP id 46A0043D45 for ; Mon, 5 Sep 2005 06:46:13 +0000 (GMT) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (unknown [192.168.48.2]) by haven.freebsd.dk (Postfix) with ESMTP id 1EB70BC66; Mon, 5 Sep 2005 06:46:11 +0000 (UTC) To: Jeb Campbell From: "Poul-Henning Kamp" In-Reply-To: Your message of "Mon, 05 Sep 2005 01:44:33 CDT." <431BE951.4070101@c4solutions.net> Date: Mon, 05 Sep 2005 08:46:11 +0200 Message-ID: <51974.1125902771@phk.freebsd.dk> Sender: phk@phk.freebsd.dk Cc: freebsd-fs@freebsd.org Subject: Re: kernel newb -- read/write to a local file from in kernel X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Sep 2005 06:46:13 -0000 In message <431BE951.4070101@c4solutions.net>, Jeb Campbell writes: >I'm trying to learn about kernel programming and as a weekend project >I've started writing a Netgraph to file/dev module. > >The Netgraph stuff was pretty easy, but I'm at a loss on the file access >part as I've never done anything in-kernel. > >After much searching (most of it not knowing what I was looking for), I >think that I'm looking for something like "Vnode interface mechanics, >write to a local file" as taught in Mr. McKusick's "FreeBSD Kernel >Internals: An Intensive Code Walkthrough". > >I've looked at nullfs/null_vnops.c, but I think that is more of a >pass-through example. Does anyone know of any online code that shows >the process of opening a file/device for read/write? In the spirit of education I won't tell you "how to" but only "where to look": ktrace, coredumps and accounting all writes to files from the kernel. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-fs@FreeBSD.ORG Mon Sep 5 06:55:16 2005 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5E84916A41F for ; Mon, 5 Sep 2005 06:55:16 +0000 (GMT) (envelope-from jebc@c4solutions.net) Received: from outbound.mailhop.org (outbound.mailhop.org [63.208.196.171]) by mx1.FreeBSD.org (Postfix) with ESMTP id 02A8C43D49 for ; Mon, 5 Sep 2005 06:55:15 +0000 (GMT) (envelope-from jebc@c4solutions.net) Received: from pcp0012210017pcs.blairblvd.tn.nash.comcast.net ([69.245.61.169] helo=ares.c4solutions.net) by outbound.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.51) id 1ECAst-0003a6-8n; Mon, 05 Sep 2005 02:55:15 -0400 Received: from [10.10.1.115] (jeb-64.internal.c4solutions.net [10.10.1.115]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ares.c4solutions.net (Postfix) with ESMTP id 5A6E9ADAF7; Mon, 5 Sep 2005 01:55:11 -0500 (CDT) X-Mail-Handler: MailHop Outbound by DynDNS.org X-Originating-IP: 69.245.61.169 X-Report-Abuse-To: abuse@dyndns.org (see http://www.mailhop.org/outbound/abuse.html for abuse reporting information) X-MHO-User: jebutk Message-ID: <431BEBD1.8030001@c4solutions.net> Date: Mon, 05 Sep 2005 01:55:13 -0500 From: Jeb Campbell User-Agent: Mozilla Thunderbird 1.0.6 (X11/20050816) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Poul-Henning Kamp References: <51974.1125902771@phk.freebsd.dk> In-Reply-To: <51974.1125902771@phk.freebsd.dk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: kernel newb -- read/write to a local file from in kernel X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Sep 2005 06:55:16 -0000 Poul-Henning Kamp wrote: > In the spirit of education I won't tell you "how to" but only "where > to look": ktrace, coredumps and accounting all writes to files > from the kernel. Wow -- don't I feel stupid. After 2 days of searching, I couldn't see the forest for the trees. (and 4 hours of sleep doesn't help) Thanks you for your time on this question and for all your other work on FreeBSD! Jeb (cc'ing the list for a record of my stupidity) From owner-freebsd-fs@FreeBSD.ORG Mon Sep 5 15:58:26 2005 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 19EFF16A41F for ; Mon, 5 Sep 2005 15:58:26 +0000 (GMT) (envelope-from stas@core.310.ru) Received: from core.310.ru (core.310.ru [83.97.105.75]) by mx1.FreeBSD.org (Postfix) with ESMTP id 917D543D75 for ; Mon, 5 Sep 2005 15:58:13 +0000 (GMT) (envelope-from stas@core.310.ru) Received: from core.310.ru (localhost [127.0.0.1]) by core.310.ru (8.13.3/8.12.11) with ESMTP id j85FrcRM012928; Mon, 5 Sep 2005 19:53:38 +0400 (MSD) (envelope-from stas@core.310.ru) Received: (from stas@localhost) by core.310.ru (8.13.3/8.12.11/Submit) id j85FrcdF012927; Mon, 5 Sep 2005 19:53:38 +0400 (MSD) (envelope-from stas) Date: Mon, 5 Sep 2005 19:53:38 +0400 From: Stanislav Sedov To: Jeb Campbell Message-ID: <20050905155338.GB9283@core.310.ru> References: <431BE951.4070101@c4solutions.net> Mime-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <431BE951.4070101@c4solutions.net> User-Agent: Mutt/1.4.2.1i Cc: freebsd-fs@freebsd.org Subject: Re: kernel newb -- read/write to a local file from in kernel X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Sep 2005 15:58:26 -0000 On Mon, Sep 05, 2005 at 01:44:33AM -0500, Jeb Campbell wrote: > I'm trying to learn about kernel programming and as a weekend project > I've started writing a Netgraph to file/dev module. > > The Netgraph stuff was pretty easy, but I'm at a loss on the file access > part as I've never done anything in-kernel. > > After much searching (most of it not knowing what I was looking for), I > think that I'm looking for something like "Vnode interface mechanics, > write to a local file" as taught in Mr. McKusick's "FreeBSD Kernel > Internals: An Intensive Code Walkthrough". > > I've looked at nullfs/null_vnops.c, but I think that is more of a > pass-through example. Does anyone know of any online code that shows > the process of opening a file/device for read/write? > > Thanks in advance if someone has any tips or more info. > > (and sorry to bother the kernel hackers with such a newbie question) > > -- > Jeb Campbell > jeb@c4solutions.net > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > Look throgh md(4) driver source (/sys/dev/md) - it is a good example. From owner-freebsd-fs@FreeBSD.ORG Mon Sep 5 23:53:25 2005 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 76DD016A420 for ; Mon, 5 Sep 2005 23:53:25 +0000 (GMT) (envelope-from mi+kde@aldan.algebra.com) Received: from mail21.sea5.speakeasy.net (mail21.sea5.speakeasy.net [69.17.117.23]) by mx1.FreeBSD.org (Postfix) with ESMTP id D708A43D48 for ; Mon, 5 Sep 2005 23:53:24 +0000 (GMT) (envelope-from mi+kde@aldan.algebra.com) Received: (qmail 17456 invoked from network); 5 Sep 2005 23:53:24 -0000 Received: from aldan.algebra.com (HELO blue.virtual-estates.net) ([216.254.65.224]) (envelope-sender ) by mail21.sea5.speakeasy.net (qmail-ldap-1.03) with AES256-SHA encrypted SMTP for ; 5 Sep 2005 23:53:24 -0000 Received: from blue.virtual-estates.net (blue [127.0.0.1]) by blue.virtual-estates.net (8.13.3/8.13.3) with ESMTP id j85NrNdM032621 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 5 Sep 2005 19:53:23 -0400 (EDT) (envelope-from mi+kde@aldan.algebra.com) Received: from localhost (localhost [[UNIX: localhost]]) by blue.virtual-estates.net (8.13.4/8.13.3/Submit) id j85NrMgl032620; Mon, 5 Sep 2005 19:53:22 -0400 (EDT) (envelope-from mi+kde@aldan.algebra.com) X-Authentication-Warning: blue.virtual-estates.net: mi set sender to mi+kde@aldan.algebra.com using -f From: Mikhail Teterin To: fs@freebsd.org, questions@freebsd.org Date: Mon, 5 Sep 2005 19:53:21 -0400 User-Agent: KMail/1.8.1 X-Face: %UW#n0|w>ydeGt/b@1-.UFP=K^~-:0f#O:D7whJ5G_<5143Bb3kOIs9XpX+"V+~$adGP:J|SLieM31VIhqXeLBli" Cc: Subject: Strange case of filesystem corruption? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Sep 2005 23:53:25 -0000 Can this be explained by anything other than a (nasty) bug? % ls -la audio/shorten/files total 0 % rmdir audio/shorten/files rmdir: audio/shorten/files: Directory not empty This is on 5.4-stable from July 21 -- up ever since... Thanks! -mi From owner-freebsd-fs@FreeBSD.ORG Tue Sep 6 00:38:21 2005 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6AAF516A41F; Tue, 6 Sep 2005 00:38:21 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id DEA2743D49; Tue, 6 Sep 2005 00:38:20 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [192.168.42.23] (andersonbox3.centtech.com [192.168.42.23]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id j860c420016953; Mon, 5 Sep 2005 19:38:04 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <431CE4F8.8000404@centtech.com> Date: Mon, 05 Sep 2005 19:38:16 -0500 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.11) Gecko/20050904 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Mikhail Teterin References: <200509051953.22337@aldan> In-Reply-To: <200509051953.22337@aldan> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.82/1064/Mon Sep 5 16:05:55 2005 on mh1.centtech.com X-Virus-Status: Clean Cc: questions@freebsd.org, fs@freebsd.org Subject: Re: Strange case of filesystem corruption? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Sep 2005 00:38:21 -0000 Mikhail Teterin wrote: > Can this be explained by anything other than a (nasty) bug? > > % ls -la audio/shorten/files > total 0 > % rmdir audio/shorten/files > rmdir: audio/shorten/files: Directory not empty > > This is on 5.4-stable from July 21 -- up ever since... Thanks! I doubt it's file system corruption, but there isn't a ton of info here. Can you show permissions on the directories audio, shorten, and files? Also - what is your securelevel set to, and have you checked to see if there are processes with any open files in those directories? Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Tue Sep 6 01:37:17 2005 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A271716A41F for ; Tue, 6 Sep 2005 01:37:17 +0000 (GMT) (envelope-from mi+kde@aldan.algebra.com) Received: from mail22.sea5.speakeasy.net (mail22.sea5.speakeasy.net [69.17.117.24]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8169D43D48 for ; Tue, 6 Sep 2005 01:37:16 +0000 (GMT) (envelope-from mi+kde@aldan.algebra.com) Received: (qmail 1036 invoked from network); 6 Sep 2005 01:37:16 -0000 Received: from aldan.algebra.com (HELO blue.virtual-estates.net) ([216.254.65.224]) (envelope-sender ) by mail22.sea5.speakeasy.net (qmail-ldap-1.03) with AES256-SHA encrypted SMTP for ; 6 Sep 2005 01:37:15 -0000 Received: from blue.virtual-estates.net (blue [127.0.0.1]) by blue.virtual-estates.net (8.13.3/8.13.3) with ESMTP id j861bDQf040371 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 5 Sep 2005 21:37:13 -0400 (EDT) (envelope-from mi+kde@aldan.algebra.com) Received: from localhost (localhost [[UNIX: localhost]]) by blue.virtual-estates.net (8.13.4/8.13.3/Submit) id j861bAjC040365; Mon, 5 Sep 2005 21:37:10 -0400 (EDT) (envelope-from mi+kde@aldan.algebra.com) X-Authentication-Warning: blue.virtual-estates.net: mi set sender to mi+kde@aldan.algebra.com using -f From: Mikhail Teterin To: Eric Anderson , Beecher Rintoul Date: Mon, 5 Sep 2005 21:37:10 -0400 User-Agent: KMail/1.8.1 References: <200509051953.22337@aldan> <431CE4F8.8000404@centtech.com> In-Reply-To: <431CE4F8.8000404@centtech.com> X-Face: %UW#n0|w>ydeGt/b@1-.UFP=K^~-:0f#O:D7whJ5G_<5143Bb3kOIs9XpX+"V+~$adGP:J|SLieM31VIhqXeLBli" Cc: questions@freebsd.org, fs@freebsd.org Subject: Re: Strange case of filesystem corruption? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Sep 2005 01:37:17 -0000 On Monday 05 September 2005 08:38 pm, Eric Anderson wrote: = Mikhail Teterin wrote: = > Can this be explained by anything other than a (nasty) bug? = > = > % ls -la audio/shorten/files = > total 0 = > % rmdir audio/shorten/files = > rmdir: audio/shorten/files: Directory not empty = > = > This is on 5.4-stable from July 21 -- up ever since... Thanks! [...] = Can you show permissions on the directories audio, shorten, and files? Well, ls of the directory succeds above, so it can not be the permission problem. But here: % cd audio/shorten % ls -lds 2 drwxrwxr-x 4 mi wheel 512 Jul 21 01:13 . % ls -loas files total 0 % rmdir files rmdir: files: Directory not empty = Also - what is your securelevel set to Default. = and have you checked to see if there are processes with any open files = in those directories? I doubt there are any, and why would that affect anything anyway? Here: % mkdir /tmp/q % touch /tmp/q/meow % tail -F /tmp/q/meow & [2] 39947 % rm /tmp/q/meow % rmdir /tmp/q In other words, the directory (/tmp/q) is removable even if a process (tail) still has a deleted file (meow) in it opened. On Monday 05 September 2005 08:42 pm, Beecher Rintoul wrote: = Try rm -R audio/shorten/files Thank you, but I'm afraid, it may succeed in deleting the directory, while I try to figure out, what is happening -- the directory is empty according to ls, but not empty according to rmdir. -mi From owner-freebsd-fs@FreeBSD.ORG Tue Sep 6 02:17:09 2005 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 52C2316A420 for ; Tue, 6 Sep 2005 02:17:09 +0000 (GMT) (envelope-from bv@bilver.wjv.com) Received: from wjv.com (fl-65-40-24-38.sta.sprint-hsd.net [65.40.24.38]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8D41643D48 for ; Tue, 6 Sep 2005 02:17:08 +0000 (GMT) (envelope-from bv@bilver.wjv.com) Received: from bilver.wjv.com (localhost.wjv.com [127.0.0.1]) by wjv.com (8.12.11/8.13.1) with ESMTP id j862GYYN093131; Mon, 5 Sep 2005 22:16:35 -0400 (EDT) (envelope-from bv@bilver.wjv.com) Received: (from bv@localhost) by bilver.wjv.com (8.12.11/8.13.1/Submit) id j862GXVQ093130; Mon, 5 Sep 2005 22:16:33 -0400 (EDT) (envelope-from bv) Date: Mon, 5 Sep 2005 22:16:33 -0400 From: Bill Vermillion To: Mikhail Teterin Message-ID: <20050906021633.GA93053@wjv.com> References: <200509051953.22337@aldan> <431CE4F8.8000404@centtech.com> <200509052137.10847@aldan> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200509052137.10847@aldan> Organization: W.J.Vermillion / Orlando - Winter Park ReplyTo: bv@wjv.com User-Agent: Mutt/1.5.6i X-Spam-Status: No, score=-2.8 required=5.0 tests=ALL_TRUSTED autolearn=failed version=3.0.4 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on bilver.wjv.com Cc: fs@freebsd.org, Beecher Rintoul Subject: Re: Strange case of filesystem corruption? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: bv@wjv.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Sep 2005 02:17:09 -0000 In the last exciting episode of the Mikhail Teterin saga on Mon, Sep 05, 2005 at 21:37 , Mikhail Teterin as heard to say: > On Monday 05 September 2005 08:38 pm, Eric Anderson wrote: > = Mikhail Teterin wrote: > = > Can this be explained by anything other than a (nasty) bug? > = > % ls -la audio/shorten/files > = > total 0 > = > % rmdir audio/shorten/files > = > rmdir: audio/shorten/files: Directory not empty > = > This is on 5.4-stable from July 21 -- up ever since... Thanks! > [...] > = Can you show permissions on the directories audio, shorten, and files? > Well, ls of the directory succeds above, so it can not be the permission > problem. But here: > % cd audio/shorten > % ls -lds > 2 drwxrwxr-x 4 mi wheel 512 Jul 21 01:13 . > % ls -loas files > total 0 > % rmdir files > rmdir: files: Directory not empty > = Also - what is your securelevel set to > Default. > = and have you checked to see if there are processes with any open files > = in those directories? > I doubt there are any, and why would that affect anything anyway? Here: You could check to be sure by running 'lsof'. That should remove all doubts. > % mkdir /tmp/q > % touch /tmp/q/meow > % tail -F /tmp/q/meow & > [2] 39947 > % rm /tmp/q/meow > % rmdir /tmp/q > > In other words, the directory (/tmp/q) is removable even if a process > (tail) still has a deleted file (meow) in it opened. > > On Monday 05 September 2005 08:42 pm, Beecher Rintoul wrote: > = Try rm -R audio/shorten/files > > Thank you, but I'm afraid, it may succeed in deleting the directory, > while I try to figure out, what is happening -- the directory is empty > according to ls, but not empty according to rmdir. I've seen this fairly recenlty in a 4.x [it's either 9, 10, or 11 - and I don't remember which server it was on so I can't verify the version number]. And I'm sorry I don't remember exactly what I did to fix it, as I tried many ways before I got the right one. Every now and then almost every Unix system does something that is almost unfathomable. Bill -- Bill Vermillion - bv @ wjv . com From owner-freebsd-fs@FreeBSD.ORG Tue Sep 6 09:18:37 2005 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2904216A41F; Tue, 6 Sep 2005 09:18:37 +0000 (GMT) (envelope-from killing@multiplay.co.uk) Received: from multiplay.co.uk (core6.multiplay.co.uk [85.236.96.23]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6CA0243D46; Tue, 6 Sep 2005 09:18:36 +0000 (GMT) (envelope-from killing@multiplay.co.uk) Received: from vader ([212.135.219.179]) by multiplay.co.uk (multiplay.co.uk [85.236.96.23]) (MDaemon.PRO.v8.1.1.R) with ESMTP id md50001796763.msg; Tue, 06 Sep 2005 10:17:47 +0100 Message-ID: <002a01c5b2c3$d4383f30$b3db87d4@multiplay.co.uk> From: "Steven Hartland" To: "Mikhail Teterin" , , References: <200509051953.22337@aldan> Date: Tue, 6 Sep 2005 10:17:35 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2670 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2670 X-Spam-Processed: multiplay.co.uk, Tue, 06 Sep 2005 10:17:47 +0100 (not processed: message from valid local sender) X-MDRemoteIP: 212.135.219.179 X-Return-Path: killing@multiplay.co.uk X-MDAV-Processed: multiplay.co.uk, Tue, 06 Sep 2005 10:17:48 +0100 Cc: Subject: Re: Strange case of filesystem corruption? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Sep 2005 09:18:37 -0000 have u run fsck on the fs in question while unmounted? ----- Original Message ----- From: "Mikhail Teterin" > Can this be explained by anything other than a (nasty) bug? > > % ls -la audio/shorten/files > total 0 > % rmdir audio/shorten/files > rmdir: audio/shorten/files: Directory not empty > > This is on 5.4-stable from July 21 -- up ever since... Thanks! ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone (023) 8024 3137 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-fs@FreeBSD.ORG Tue Sep 6 10:51:03 2005 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D030216A470; Tue, 6 Sep 2005 10:51:03 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [204.156.12.53]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5199743D45; Tue, 6 Sep 2005 10:51:03 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by cyrus.watson.org (Postfix) with ESMTP id CED6D46B2D; Tue, 6 Sep 2005 06:51:02 -0400 (EDT) Date: Tue, 6 Sep 2005 11:51:02 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Mikhail Teterin In-Reply-To: <200509051953.22337@aldan> Message-ID: <20050906114055.R51625@fledge.watson.org> References: <200509051953.22337@aldan> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: questions@freebsd.org, fs@freebsd.org Subject: Re: Strange case of filesystem corruption? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Sep 2005 10:51:04 -0000 On Mon, 5 Sep 2005, Mikhail Teterin wrote: > Can this be explained by anything other than a (nasty) bug? > > % ls -la audio/shorten/files > total 0 > % rmdir audio/shorten/files > rmdir: audio/shorten/files: Directory not empty > > This is on 5.4-stable from July 21 -- up ever since... Thanks! Mikhail, Have you recently experienced a system crash or hard reboot without proper shutdown? I know of at least one case prior to 6.x where this can occur -- a bug I reported to Kirk relating to bgfsck. Basically, soft update guarantees that upon reboot after failure (power or otherwise), the on-disk layout of UFS meta-data will be consistent, except with respect to reference and link counts, which may be elevated. What bgfsck does is walk the file system to identify and correct the elevated counts. Here's a specific example of such a problem: Directory A is created Write A directory data (., ..) Write A directory inode (new inode) Write A parent inode (link count++) Write A parent data (add name) Directory B is created Write B directory data (., ..) Write B directory inode (new inode) Write A directory inode (link count++) XXX Write A directory data (add name) Directory B is removed Write A directory data (remove name) XXX Write A directory inode (link count--) Note that if the sequence of events his halted at either of the XXX's above, the inode link count on directory A will be elevated, even though the name for B has been removed from A. Background fsck comes alone later, notices that the counts are elevated, and drops them. However, until ufs_vnops.c:1.269, this caused a problem: because the link count was elevated, UFS assumed that the directory contained a reference to another directory, and would not let it be removed. Once bgfsck catches up with the directory, it can be removed. I've seen this symptom most frequently following a crash or a power outage during an rm -Rf of a /usr/obj, which I then immediately restart on reboot, and rm -Rf gets there before bgfsck does. Here's the commit message for ufs_vnops.c:1.269, which should be MFC'd: revision 1.269 date: 2005/05/18 22:18:21; author: mckusick; state: Exp; lines: +2 -3 Allow removal of empty directories with high link counts. These can occur on a filesystem running with soft updates after a crash and before a background fsck has been run. To prevent discrepancies from arising in a background fsck that may already be running, the directory is removed but its inode is not freed and is left with the residual reference count. When encountered by the background fsck it will be reclaimed. I'll e-mail Kirk and ask if he's comfortable enough with the change to this point to merge it. Robert N M Watson From owner-freebsd-fs@FreeBSD.ORG Tue Sep 6 13:31:37 2005 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BAE1016A41F for ; Tue, 6 Sep 2005 13:31:37 +0000 (GMT) (envelope-from mi+kde@aldan.algebra.com) Received: from mail28.sea5.speakeasy.net (mail28.sea5.speakeasy.net [69.17.117.30]) by mx1.FreeBSD.org (Postfix) with ESMTP id 76EC843D45 for ; Tue, 6 Sep 2005 13:31:36 +0000 (GMT) (envelope-from mi+kde@aldan.algebra.com) Received: (qmail 28746 invoked from network); 6 Sep 2005 13:31:36 -0000 Received: from aldan.algebra.com (HELO blue.virtual-estates.net) ([216.254.65.224]) (envelope-sender ) by mail28.sea5.speakeasy.net (qmail-ldap-1.03) with AES256-SHA encrypted SMTP for ; 6 Sep 2005 13:31:36 -0000 Received: from blue.virtual-estates.net (blue [127.0.0.1]) by blue.virtual-estates.net (8.13.3/8.13.3) with ESMTP id j86DVYIe023176 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 6 Sep 2005 09:31:35 -0400 (EDT) (envelope-from mi+kde@aldan.algebra.com) Received: from localhost (localhost [[UNIX: localhost]]) by blue.virtual-estates.net (8.13.4/8.13.3/Submit) id j86DVYY7023175; Tue, 6 Sep 2005 09:31:34 -0400 (EDT) (envelope-from mi+kde@aldan.algebra.com) X-Authentication-Warning: blue.virtual-estates.net: mi set sender to mi+kde@aldan.algebra.com using -f From: Mikhail Teterin To: Robert Watson Date: Tue, 6 Sep 2005 09:31:33 -0400 User-Agent: KMail/1.8.1 References: <200509051953.22337@aldan> <20050906114055.R51625@fledge.watson.org> In-Reply-To: <20050906114055.R51625@fledge.watson.org> X-Face: %UW#n0|w>ydeGt/b@1-.UFP=K^~-:0f#O:D7whJ5G_<5143Bb3kOIs9XpX+"V+~$adGP:J|SLieM31VIhqXeLBli" Cc: questions@freebsd.org, fs@freebsd.org Subject: Re: Strange case of filesystem corruption? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Sep 2005 13:31:37 -0000 On Tuesday 06 September 2005 06:51 am, Robert Watson wrote: = Have you recently experienced a system crash or hard reboot without proper = shutdown? According to dmesg.boot, this filesystem was flagged as "not properly dismounted" back then. The machine's uptime is currently 47 days and no background fskcs are running, of course. Yours analysis is, likely, correct then... I guess, the fix should be MFCed. (What about 6.0-release?) Thanks! -mi From owner-freebsd-fs@FreeBSD.ORG Tue Sep 6 13:46:34 2005 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B927E16A420; Tue, 6 Sep 2005 13:46:34 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [204.156.12.53]) by mx1.FreeBSD.org (Postfix) with ESMTP id F279A43D6A; Tue, 6 Sep 2005 13:46:30 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by cyrus.watson.org (Postfix) with ESMTP id 5734546B92; Tue, 6 Sep 2005 09:46:30 -0400 (EDT) Date: Tue, 6 Sep 2005 14:46:30 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Mikhail Teterin In-Reply-To: <200509060931.33977@aldan> Message-ID: <20050906144512.T75104@fledge.watson.org> References: <200509051953.22337@aldan> <20050906114055.R51625@fledge.watson.org> <200509060931.33977@aldan> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: questions@freebsd.org, fs@freebsd.org Subject: Re: Strange case of filesystem corruption? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Sep 2005 13:46:34 -0000 On Tue, 6 Sep 2005, Mikhail Teterin wrote: > On Tuesday 06 September 2005 06:51 am, Robert Watson wrote: > = Have you recently experienced a system crash or hard reboot without proper > = shutdown? > > According to dmesg.boot, this filesystem was flagged as "not properly > dismounted" back then. The machine's uptime is currently 47 days and no > background fskcs are running, of course. > > Yours analysis is, likely, correct then... I guess, the fix should be > MFCed. (What about 6.0-release?) Thanks! The change was made in HEAD before the 6.x branch, so it's currently in RELENG_6. I fired off an e-mail to Kirk to ask about MFC'ing it to RELENG_5, and will do so once I hear back, assuming he thinks there is no reason not to. Do you have back logs to when bgfsck was running, btw? Normally it should correct this sort of thing, suggestion that bgfsck exited without correcting them, which is a little worrying. You may want to boot to single user and fsck manually... Robert N M Watson From owner-freebsd-fs@FreeBSD.ORG Tue Sep 6 15:19:34 2005 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C483D16A41F; Tue, 6 Sep 2005 15:19:34 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3BD2543D46; Tue, 6 Sep 2005 15:19:33 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id j86FJLAF033327; Tue, 6 Sep 2005 10:19:21 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <431DB382.7000904@centtech.com> Date: Tue, 06 Sep 2005 10:19:30 -0500 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.11) Gecko/20050904 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Robert Watson References: <200509051953.22337@aldan> <20050906114055.R51625@fledge.watson.org> <200509060931.33977@aldan> <20050906144512.T75104@fledge.watson.org> In-Reply-To: <20050906144512.T75104@fledge.watson.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.82/1066/Tue Sep 6 06:42:41 2005 on mh1.centtech.com X-Virus-Status: Clean Cc: questions@freebsd.org, fs@freebsd.org, Mikhail Teterin Subject: Re: Strange case of filesystem corruption? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Sep 2005 15:19:34 -0000 Robert Watson wrote: > On Tue, 6 Sep 2005, Mikhail Teterin wrote: > >> On Tuesday 06 September 2005 06:51 am, Robert Watson wrote: >> = Have you recently experienced a system crash or hard reboot without >> proper >> = shutdown? >> >> According to dmesg.boot, this filesystem was flagged as "not properly >> dismounted" back then. The machine's uptime is currently 47 days and >> no background fskcs are running, of course. >> >> Yours analysis is, likely, correct then... I guess, the fix should be >> MFCed. (What about 6.0-release?) Thanks! > > > The change was made in HEAD before the 6.x branch, so it's currently in > RELENG_6. I fired off an e-mail to Kirk to ask about MFC'ing it to > RELENG_5, and will do so once I hear back, assuming he thinks there is > no reason not to. > > Do you have back logs to when bgfsck was running, btw? Normally it > should correct this sort of thing, suggestion that bgfsck exited without > correcting them, which is a little worrying. You may want to boot to > single user and fsck manually... I thought that having background_fsck="NO" in your rc.conf would make the partition get mounted even though it is unclean, and no fsck's would happen (I use this 'feature' right now, since doing bgfsck on a 2TB partition takes about 2 hours just to make the snapshot, which means filesystem activity is suspended for that time). If it was set to "YES", and bgfsck exited abnormally, I don't think it will get mounted automatically, so he would have had to manually mount it, correct? Mikhail, do any of those scenario's sound like a match? Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Tue Sep 6 15:46:57 2005 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 81AAE16A41F for ; Tue, 6 Sep 2005 15:46:57 +0000 (GMT) (envelope-from mi+mx@aldan.algebra.com) Received: from mail28.sea5.speakeasy.net (mail28.sea5.speakeasy.net [69.17.117.30]) by mx1.FreeBSD.org (Postfix) with ESMTP id 372F343D49 for ; Tue, 6 Sep 2005 15:46:55 +0000 (GMT) (envelope-from mi+mx@aldan.algebra.com) Received: (qmail 21691 invoked from network); 6 Sep 2005 15:46:54 -0000 Received: from aldan.algebra.com (HELO blue.virtual-estates.net) ([216.254.65.224]) (envelope-sender ) by mail28.sea5.speakeasy.net (qmail-ldap-1.03) with AES256-SHA encrypted SMTP for ; 6 Sep 2005 15:46:54 -0000 Received: from corbulon.video-collage.com (static-151-204-231-237.bos.east.verizon.net [151.204.231.237]) by blue.virtual-estates.net (8.13.3/8.13.3) with ESMTP id j86Fknjk078197 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 6 Sep 2005 11:46:50 -0400 (EDT) (envelope-from mi+mx@aldan.algebra.com) Received: from mteterin.us.murex.com (195-11.customer.cloud9.net [168.100.195.11]) by corbulon.video-collage.com (8.13.4/8.13.1) with ESMTP id j86Fkh9b046569 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 6 Sep 2005 11:46:44 -0400 (EDT) (envelope-from mi+mx@aldan.algebra.com) Received: from mteterin.us.murex.com (mteterin@localhost [127.0.0.1]) by mteterin.us.murex.com (8.13.3/8.13.3) with ESMTP id j86FkUJ7042838; Tue, 6 Sep 2005 11:46:30 -0400 (EDT) (envelope-from mi+mx@aldan.algebra.com) Received: from localhost (localhost [[UNIX: localhost]]) by mteterin.us.murex.com (8.13.3/8.13.3/Submit) id j86FkTDQ042837; Tue, 6 Sep 2005 11:46:29 -0400 (EDT) (envelope-from mi+mx@aldan.algebra.com) X-Authentication-Warning: mteterin.us.murex.com: mteterin set sender to mi+mx@aldan.algebra.com using -f From: Mikhail Teterin Organization: Virtual Estates, Inc. To: Robert Watson Date: Tue, 6 Sep 2005 11:46:28 -0400 User-Agent: KMail/1.8.2 References: <200509051953.22337@aldan> <200509060931.33977@aldan> <20050906144512.T75104@fledge.watson.org> In-Reply-To: <20050906144512.T75104@fledge.watson.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200509061146.28926.mi+mx@aldan.algebra.com> X-Virus-Scanned: ClamAV devel-20050525/1064/Mon Sep 5 17:05:55 2005 on corbulon.video-collage.com X-Virus-Status: Clean X-Scanned-By: MIMEDefang 2.43 Cc: questions@freebsd.org, fs@freebsd.org Subject: Re: Strange case of filesystem corruption? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Sep 2005 15:46:57 -0000 > Do you have back logs to when bgfsck was running, btw? Yes, indeed. And there is stuff in them... The machine crashed on July 21st at 00:20 (/var is troublesome fs): Jul 21 00:20:37 blue kernel: WARNING: /var was not properly dismounted Jul 21 00:20:37 blue kernel: /var: mount pending error: blocks 28 files 2 Jul 21 00:29:05 blue kernel: root@blue.virtual-estates.net:/var/obj/var/src/sys/SILVER Jul 21 00:29:05 blue kernel: WARNING: /var was not properly dismounted Jul 21 00:29:05 blue kernel: /var: mount pending error: blocks 28 files 2 Jul 21 00:32:21 blue fsck: /dev/da0s1d: UNREF FILE I=11559 OWNER=root MODE=100400 Jul 21 00:32:21 blue fsck: /dev/da0s1d: SIZE=12884902464 MTIME=Jul 21 00:22 2005 (CLEARED) Jul 21 00:32:21 blue fsck: /dev/da0s1d: UNREF FILE I=1413219 OWNER=root MODE=100600 Jul 21 00:32:21 blue fsck: /dev/da0s1d: SIZE=12288 MTIME=Jul 21 00:18 2005 (CLEARED) Jul 21 00:32:21 blue fsck: /dev/da0s1d: UNREF FILE I=1423432 OWNER=root MODE=100600 Jul 21 00:32:21 blue fsck: /dev/da0s1d: SIZE=12288 MTIME=Jul 21 00:22 2005 (CLEARED) Jul 21 00:32:21 blue fsck: /dev/da0s1d: UNREF FILE I=1423433 OWNER=root MODE=100600 Jul 21 00:32:21 blue fsck: /dev/da0s1d: SIZE=451 MTIME=Jul 21 00:22 2005 (CLEARED) Jul 21 00:32:21 blue fsck: /dev/da0s1d: Reclaimed: 0 directories, 4294967269 files, -26 fragments Jul 21 00:32:21 blue fsck: /dev/da0s1d: 495013 files, 4631802 used, 1461319 free (59391 frags, 175241 blocks, 1.0% fragmentation) So that was a successful fsck above (save for the negative number of reclaimed fragments). I am not sure, what caused the next reboot (crash or orderly reboot) at 1:17, but this time fsck failed: Jul 21 01:17:18 blue kernel: root@blue.virtual-estates.net:/var/obj/var/src/sys/SILVER Jul 21 01:17:18 blue kernel: WARNING: /var was not properly dismounted Jul 21 01:17:18 blue kernel: /var: mount pending error: blocks 44 files 12 Jul 21 01:20:30 blue fsck: /dev/da0s1d: INCORRECT BLOCK COUNT I=11599 (4 should be 0) (CORRECTED) Jul 21 01:20:30 blue fsck: /dev/da0s1d: INCORRECT BLOCK COUNT I=34376 (4 should be 0) (CORRECTED) Jul 21 01:20:30 blue fsck: /dev/da0s1d: SETTING DIRTY FLAG IN READ_ONLY MODE Jul 21 01:20:30 blue fsck: Jul 21 01:20:30 blue fsck: /dev/da0s1d: UNEXPECTED SOFT UPDATE INCONSISTENCY; RUN fsck MANUALLY. What is interesting is that my ports tree is also on this same fs and has gone through numerous cvs updates and port builds (including large items like mozilla and openoffice)... Thanks! -mi From owner-freebsd-fs@FreeBSD.ORG Wed Sep 7 17:45:43 2005 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 492B016A41F; Wed, 7 Sep 2005 17:45:43 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: from fileserver.fields.utoronto.ca (fileserver.fields.utoronto.ca [128.100.216.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id C063F43D55; Wed, 7 Sep 2005 17:45:42 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: from fields.fields.utoronto.ca (fields.localdomain [192.168.216.11]) by fileserver.fields.utoronto.ca (8.12.8/8.12.8/Fields 6.0) with ESMTP id j87HjE0r010962 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 7 Sep 2005 13:45:14 -0400 Received: from obsecurity.dyndns.org (localhost.localdomain [127.0.0.1]) by fields.fields.utoronto.ca (8.12.8/8.12.8/Fields WS 6.0) with ESMTP id j87HjD6P027747; Wed, 7 Sep 2005 13:45:14 -0400 Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id 90A5451226; Wed, 7 Sep 2005 13:45:13 -0400 (EDT) Date: Wed, 7 Sep 2005 13:45:13 -0400 From: Kris Kennaway To: Eric Anderson Message-ID: <20050907174513.GA32771@xor.obsecurity.org> References: <200509051953.22337@aldan> <20050906114055.R51625@fledge.watson.org> <200509060931.33977@aldan> <20050906144512.T75104@fledge.watson.org> <431DB382.7000904@centtech.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="9jxsPFA5p3P2qPhR" Content-Disposition: inline In-Reply-To: <431DB382.7000904@centtech.com> User-Agent: Mutt/1.4.2.1i Cc: questions@freebsd.org, Robert Watson , Mikhail Teterin , fs@freebsd.org Subject: Re: Strange case of filesystem corruption? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Sep 2005 17:45:43 -0000 --9jxsPFA5p3P2qPhR Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Sep 06, 2005 at 10:19:30AM -0500, Eric Anderson wrote: > Robert Watson wrote: > >On Tue, 6 Sep 2005, Mikhail Teterin wrote: > > > >>On Tuesday 06 September 2005 06:51 am, Robert Watson wrote: > >>=3D Have you recently experienced a system crash or hard reboot without= =20 > >>proper > >>=3D shutdown? > >> > >>According to dmesg.boot, this filesystem was flagged as "not properly= =20 > >>dismounted" back then. The machine's uptime is currently 47 days and=20 > >>no background fskcs are running, of course. > >> > >>Yours analysis is, likely, correct then... I guess, the fix should be= =20 > >>MFCed. (What about 6.0-release?) Thanks! > > > > > >The change was made in HEAD before the 6.x branch, so it's currently in= =20 > >RELENG_6. I fired off an e-mail to Kirk to ask about MFC'ing it to=20 > >RELENG_5, and will do so once I hear back, assuming he thinks there is= =20 > >no reason not to. > > > >Do you have back logs to when bgfsck was running, btw? Normally it=20 > >should correct this sort of thing, suggestion that bgfsck exited without= =20 > >correcting them, which is a little worrying. You may want to boot to=20 > >single user and fsck manually... >=20 > I thought that having background_fsck=3D"NO" in your rc.conf would make= =20 > the partition get mounted even though it is unclean, and no fsck's would= =20 > happen (I use this 'feature' right now, since doing bgfsck on a 2TB=20 > partition takes about 2 hours just to make the snapshot, which means=20 > filesystem activity is suspended for that time). AFAIK, no, this just does a regular old fg fsck. Kris --9jxsPFA5p3P2qPhR Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFDHycpWry0BWjoQKURAro6AKDbtwdlhFeFMkrLC6dMmAyX7NTmmgCg/Q7w jXqBhbOvhpqUueaoI9j7+7w= =O1pD -----END PGP SIGNATURE----- --9jxsPFA5p3P2qPhR-- From owner-freebsd-fs@FreeBSD.ORG Wed Sep 7 18:08:12 2005 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B0F5916A41F; Wed, 7 Sep 2005 18:08:12 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 242AE43D46; Wed, 7 Sep 2005 18:08:11 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id j87I83ZX066786; Wed, 7 Sep 2005 13:08:03 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <431F2C8C.3010203@centtech.com> Date: Wed, 07 Sep 2005 13:08:12 -0500 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.11) Gecko/20050904 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Kris Kennaway References: <200509051953.22337@aldan> <20050906114055.R51625@fledge.watson.org> <200509060931.33977@aldan> <20050906144512.T75104@fledge.watson.org> <431DB382.7000904@centtech.com> <20050907174513.GA32771@xor.obsecurity.org> In-Reply-To: <20050907174513.GA32771@xor.obsecurity.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.82/1069/Wed Sep 7 10:08:51 2005 on mh1.centtech.com X-Virus-Status: Clean Cc: questions@freebsd.org, Robert Watson , Mikhail Teterin , fs@freebsd.org Subject: Re: Strange case of filesystem corruption? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Sep 2005 18:08:12 -0000 Kris Kennaway wrote: > On Tue, Sep 06, 2005 at 10:19:30AM -0500, Eric Anderson wrote: > >>Robert Watson wrote: >> >>>On Tue, 6 Sep 2005, Mikhail Teterin wrote: >>> >>> >>>>On Tuesday 06 September 2005 06:51 am, Robert Watson wrote: >>>>= Have you recently experienced a system crash or hard reboot without >>>>proper >>>>= shutdown? >>>> >>>>According to dmesg.boot, this filesystem was flagged as "not properly >>>>dismounted" back then. The machine's uptime is currently 47 days and >>>>no background fskcs are running, of course. >>>> >>>>Yours analysis is, likely, correct then... I guess, the fix should be >>>>MFCed. (What about 6.0-release?) Thanks! >>> >>> >>>The change was made in HEAD before the 6.x branch, so it's currently in >>>RELENG_6. I fired off an e-mail to Kirk to ask about MFC'ing it to >>>RELENG_5, and will do so once I hear back, assuming he thinks there is >>>no reason not to. >>> >>>Do you have back logs to when bgfsck was running, btw? Normally it >>>should correct this sort of thing, suggestion that bgfsck exited without >>>correcting them, which is a little worrying. You may want to boot to >>>single user and fsck manually... >> >>I thought that having background_fsck="NO" in your rc.conf would make >>the partition get mounted even though it is unclean, and no fsck's would >>happen (I use this 'feature' right now, since doing bgfsck on a 2TB >>partition takes about 2 hours just to make the snapshot, which means >>filesystem activity is suspended for that time). > > > AFAIK, no, this just does a regular old fg fsck. You are absolutely right! My mistake - I had forgotten I did some other hacks to ignore those filesystems (so I can fsck manually)! Too bad the mksnap_ffs is so slow on large (>20GB) partitions.. :( Thanks for the correction! Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Thu Sep 8 12:56:54 2005 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1A3AE16A41F for ; Thu, 8 Sep 2005 12:56:54 +0000 (GMT) (envelope-from deepak_nai@yahoo.com) Received: from web41002.mail.yahoo.com (web41002.mail.yahoo.com [66.218.93.1]) by mx1.FreeBSD.org (Postfix) with SMTP id C1F3B43D45 for ; Thu, 8 Sep 2005 12:56:53 +0000 (GMT) (envelope-from deepak_nai@yahoo.com) Received: (qmail 97585 invoked by uid 60001); 8 Sep 2005 12:56:53 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=rKROpd6eyD5cMb4fS/kTBXQKr//Vde+wdjPGzJwxOT54XqY6NRbp8RYImg+5fsO0nv0PfX3j4lPb/IebEObMxXMo7uw0xVIysJVttaaallVwvVk97X5f/iRng9IDSH+Ta6NyvaK1hdoqzgkGst2ocT0IJRzYedQLr136wBnZgtE= ; Message-ID: <20050908125653.97583.qmail@web41002.mail.yahoo.com> Received: from [202.149.214.46] by web41002.mail.yahoo.com via HTTP; Thu, 08 Sep 2005 13:56:53 BST Date: Thu, 8 Sep 2005 13:56:53 +0100 (BST) From: Deepak Naidu To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Subject: Porting from linux to FreeBSD X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Sep 2005 12:56:54 -0000 Hi, I am thinking of porting my mailserver from fedora 2.6 to freebsd 5.4. In this regards i have a question regarding chosing the stable and fast filesystem. I used reiserfs in fedora due to its faster input & output operation with file size of 1-5 Mb. Is UFS2 stable and fast(does it journalize). It seems to be slow when untaring a file. I am thing about XFS, how do i implement it in FreeBSD 5.4 in kernel, how do i get the XFS option during installation(I know for that i have to make own CD, even then). What are my options regarding file syetm in FreeBSD 5.4 considering mailing server performance. Thnax for the help... Regards, Deepak Naidu ___________________________________________________________ Yahoo! Messenger - NEW crystal clear PC to PC calling worldwide with voicemail http://uk.messenger.yahoo.com From owner-freebsd-fs@FreeBSD.ORG Thu Sep 8 15:18:49 2005 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 47DC116A41F for ; Thu, 8 Sep 2005 15:18:49 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id D413343D46 for ; Thu, 8 Sep 2005 15:18:48 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id j88FIl1q091967; Thu, 8 Sep 2005 10:18:47 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <43205660.6040309@centtech.com> Date: Thu, 08 Sep 2005 10:18:56 -0500 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.11) Gecko/20050904 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Deepak Naidu References: <20050908125653.97583.qmail@web41002.mail.yahoo.com> In-Reply-To: <20050908125653.97583.qmail@web41002.mail.yahoo.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.82/1069/Wed Sep 7 10:08:51 2005 on mh1.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: Porting from linux to FreeBSD X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Sep 2005 15:18:49 -0000 Deepak Naidu wrote: > Hi, > > I am thinking of porting my mailserver from fedora > 2.6 to freebsd 5.4. In this regards i have a question > regarding chosing the stable and fast filesystem. > > I used reiserfs in fedora due to its faster input & > output operation with file size of 1-5 Mb. Is UFS2 > stable and fast(does it journalize). It seems to be > slow when untaring a file. > > I am thing about XFS, how do i implement it in FreeBSD > 5.4 in kernel, how do i get the XFS option during > installation(I know for that i have to make own CD, > even then). > > > What are my options regarding file syetm in FreeBSD > 5.4 considering mailing server performance. UFS2 is actually a pretty fast filesystem for most cases. UFS2 does not currently support journaling, however some work is underway by Scott Long to implement this. UFS2 has soft-updates, which keeps meta-data consistant in case of system failure (reboots,etc), but does not avoid an fsck. The 5.x series and above has support for background fsck, which allows you to mount the filesystem and begin using it, while background fsck does the checking live. There are read-only ports of reiserfs and xfs available for 6.0- and 7.0. The reiserfs code is in the src tree, and the xfs code can be found here: http://people.freebsd.org/~rodrigc/xfs/ I'm sure Craig and Alexander would enjoy the help with the porting of XFS. I'm unsure of the status of reiserfs. Porting XFS to FreeBSD (full support) would be awesome! Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Thu Sep 8 15:34:05 2005 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D20B216A41F for ; Thu, 8 Sep 2005 15:34:05 +0000 (GMT) (envelope-from deepak_nai@yahoo.com) Received: from web41005.mail.yahoo.com (web41005.mail.yahoo.com [66.218.93.4]) by mx1.FreeBSD.org (Postfix) with SMTP id 91B0043D48 for ; Thu, 8 Sep 2005 15:34:05 +0000 (GMT) (envelope-from deepak_nai@yahoo.com) Received: (qmail 12319 invoked by uid 60001); 8 Sep 2005 15:34:05 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=w4ERD0Op5mkIAfJhE2LDMCNNsSsj1vYCaQxzxkKukMgveiiOMeU4kbSVLRU/3TPwh9Mjo6aqvGFj0NbEiaBL/8hcNe1Qu/g1Xs9ffPWdLbllnG9RRxuaxe09uuNocG3BFX5Gp3po9qgJlPOSyrqzin26MvMjJtZjku+e/xI0SMs= ; Message-ID: <20050908153405.12317.qmail@web41005.mail.yahoo.com> Received: from [202.149.214.46] by web41005.mail.yahoo.com via HTTP; Thu, 08 Sep 2005 16:34:05 BST Date: Thu, 8 Sep 2005 16:34:05 +0100 (BST) From: Deepak Naidu To: Eric Anderson In-Reply-To: <43205660.6040309@centtech.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: freebsd-fs@freebsd.org Subject: Re: Porting from linux to FreeBSD X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Sep 2005 15:34:05 -0000 Thanx for the detailed info. hope it comes soon. --- Eric Anderson wrote: > Deepak Naidu wrote: > > Hi, > > > > I am thinking of porting my mailserver from > fedora > > 2.6 to freebsd 5.4. In this regards i have a > question > > regarding chosing the stable and fast filesystem. > > > > I used reiserfs in fedora due to its faster > input & > > output operation with file size of 1-5 Mb. Is > UFS2 > > stable and fast(does it journalize). It seems to > be > > slow when untaring a file. > > > > I am thing about XFS, how do i implement it in > FreeBSD > > 5.4 in kernel, how do i get the XFS option during > > installation(I know for that i have to make own > CD, > > even then). > > > > > > What are my options regarding file syetm in > FreeBSD > > 5.4 considering mailing server performance. > > UFS2 is actually a pretty fast filesystem for most > cases. UFS2 does not > currently support journaling, however some work is > underway by Scott > Long to implement this. UFS2 has soft-updates, > which keeps meta-data > consistant in case of system failure (reboots,etc), > but does not avoid > an fsck. The 5.x series and above has support for > background fsck, > which allows you to mount the filesystem and begin > using it, while > background fsck does the checking live. > > There are read-only ports of reiserfs and xfs > available for 6.0- and > 7.0. The reiserfs code is in the src tree, and the > xfs code can be > found here: > http://people.freebsd.org/~rodrigc/xfs/ > > I'm sure Craig and Alexander would enjoy the help > with the porting of > XFS. I'm unsure of the status of reiserfs. Porting > XFS to FreeBSD > (full support) would be awesome! > > Eric > > > > > > -- > ------------------------------------------------------------------------ > Eric Anderson Sr. Systems Administrator > Centaur Technology > Anything that works is better than anything that > doesn't. > ------------------------------------------------------------------------ > Linux your Life, Don't Window it [[]] { All for the best } ___________________________________________________________ Yahoo! Messenger - NEW crystal clear PC to PC calling worldwide with voicemail http://uk.messenger.yahoo.com From owner-freebsd-fs@FreeBSD.ORG Thu Sep 8 15:43:18 2005 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 434A416A41F for ; Thu, 8 Sep 2005 15:43:18 +0000 (GMT) (envelope-from freebsd-fs@merdin.com) Received: from lancia.kaluga.ru (lancia.kaluga.ru [62.148.128.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id B0A6B43D49 for ; Thu, 8 Sep 2005 15:43:16 +0000 (GMT) (envelope-from freebsd-fs@merdin.com) Received: from localhost ([62.148.144.242]) by lancia.kaluga.ru (8.13.4/8.13.4) with ESMTP id j88FhCFP078784 for ; Thu, 8 Sep 2005 19:43:13 +0400 (MSD) Received: from localhost ([127.0.0.1]) by [127.0.0.1] with ESMTP (SpamPal v1.583) sender ; 08 Sep 2005 19:43:13 +0400 Date: Thu, 8 Sep 2005 19:43:12 +0400 From: Pavel Merdine X-Mailer: The Bat! (v3.0.1.33) UNREG / CD5BF9353B3B7091 X-Priority: 3 (Normal) Message-ID: <1523969635.20050908194312@merdin.com> To: Eric Anderson In-Reply-To: <43205660.6040309@centtech.com> References: <20050908125653.97583.qmail@web41002.mail.yahoo.com> <43205660.6040309@centtech.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Antivirus: Dr.Web (R) for Mail Servers on lancia.kaluga.ru host X-Antivirus-Code: 100000 Subject: Re[2]: Porting from linux to FreeBSD X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Sep 2005 15:43:18 -0000 Hello , I dont recommend using background fsck in production environment, because there can be a kernel panic before background fsck finished. Foreground fsck can last more than half an hour for IDE 300G+ disks. In my subjective experience, ufs is quite slow comparing to Linux supported filesystems like reiserfs and xfs. However anybody can benefit from the FreeBSD kernel stability. Also, I have to note that ufs seems to be poorly supported despites the fact this is the only filesystem in FreeBSD. At least I'm aware of two huge bugs which are not being fixed for years. First bug is in dirpref (the allocation optimisation) with lack of huge disk support. Another bug is in inode allocation. That bug causes rare panics on heavily loaded server (with "dup alloc" message). That bug seems to be fixed in DragonFly. We are going to review their patch and try it on our servers. Thursday, September 8, 2005, 7:18:56 PM, you wrote: > Deepak Naidu wrote: >> Hi, >> >> I am thinking of porting my mailserver from fedora >> 2.6 to freebsd 5.4. In this regards i have a question >> regarding chosing the stable and fast filesystem. >> >> I used reiserfs in fedora due to its faster input & >> output operation with file size of 1-5 Mb. Is UFS2 >> stable and fast(does it journalize). It seems to be >> slow when untaring a file. >> >> I am thing about XFS, how do i implement it in FreeBSD >> 5.4 in kernel, how do i get the XFS option during >> installation(I know for that i have to make own CD, >> even then). >> >> >> What are my options regarding file syetm in FreeBSD >> 5.4 considering mailing server performance. > UFS2 is actually a pretty fast filesystem for most cases. UFS2 does not > currently support journaling, however some work is underway by Scott > Long to implement this. UFS2 has soft-updates, which keeps meta-data > consistant in case of system failure (reboots,etc), but does not avoid > an fsck. The 5.x series and above has support for background fsck, > which allows you to mount the filesystem and begin using it, while > background fsck does the checking live. > There are read-only ports of reiserfs and xfs available for 6.0- and > 7.0. The reiserfs code is in the src tree, and the xfs code can be > found here: > http://people.freebsd.org/~rodrigc/xfs/ > I'm sure Craig and Alexander would enjoy the help with the porting of > XFS. I'm unsure of the status of reiserfs. Porting XFS to FreeBSD > (full support) would be awesome! > Eric -- / Pavel Merdine From owner-freebsd-fs@FreeBSD.ORG Thu Sep 8 15:51:03 2005 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4FB7B16A41F for ; Thu, 8 Sep 2005 15:51:03 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id E2A8843D45 for ; Thu, 8 Sep 2005 15:51:02 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id j88Fp14J092736; Thu, 8 Sep 2005 10:51:01 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <43205DEE.5010300@centtech.com> Date: Thu, 08 Sep 2005 10:51:10 -0500 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.11) Gecko/20050904 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Pavel Merdine References: <20050908125653.97583.qmail@web41002.mail.yahoo.com> <43205660.6040309@centtech.com> <1523969635.20050908194312@merdin.com> In-Reply-To: <1523969635.20050908194312@merdin.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.82/1069/Wed Sep 7 10:08:51 2005 on mh1.centtech.com X-Virus-Status: Clean Cc: Eric Anderson Subject: Re: Porting from linux to FreeBSD X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Sep 2005 15:51:03 -0000 Pavel Merdine wrote: > Hello , > > I dont recommend using background fsck in production environment, > because there can be a kernel panic before background fsck finished. > Foreground fsck can last more than half an hour for IDE 300G+ disks. > In my subjective experience, ufs is quite slow comparing to Linux > supported filesystems like reiserfs and xfs. However anybody can > benefit from the FreeBSD kernel stability. > Also, I have to note that ufs seems to be poorly supported despites the > fact this is the only filesystem in FreeBSD. At least I'm aware of two > huge bugs which are not being fixed for years. First bug is in dirpref > (the allocation optimisation) with lack of huge disk support. Another > bug is in inode allocation. That bug causes rare panics on heavily > loaded server (with "dup alloc" message). That bug seems to be fixed > in DragonFly. We are going to review their patch and try it on our > servers. Hi Pavel, A number of bugs have been recently fixed in 7/6/5- branches regarding ufs and vfs. I don't know if they affect you or not, but they are significant. Have you posted detailed debug info about these bugs to any lists, or submitted PR's? As far as support, I'm not sure what you are referring to - are you saying the FreeBSD developers are not supporting it, or are you saying that there isn't support on other platforms? Any patches you have I'm sure would be interesting to see by kernel developers and others interested in FreeBSD filesystem performance and stability. Eric > Thursday, September 8, 2005, 7:18:56 PM, you wrote: > > >>Deepak Naidu wrote: >> >>>Hi, >>> >>> I am thinking of porting my mailserver from fedora >>>2.6 to freebsd 5.4. In this regards i have a question >>>regarding chosing the stable and fast filesystem. >>> >>> I used reiserfs in fedora due to its faster input & >>>output operation with file size of 1-5 Mb. Is UFS2 >>>stable and fast(does it journalize). It seems to be >>>slow when untaring a file. >>> >>>I am thing about XFS, how do i implement it in FreeBSD >>>5.4 in kernel, how do i get the XFS option during >>>installation(I know for that i have to make own CD, >>>even then). >>> >>> >>>What are my options regarding file syetm in FreeBSD >>>5.4 considering mailing server performance. > > >>UFS2 is actually a pretty fast filesystem for most cases. UFS2 does not >>currently support journaling, however some work is underway by Scott >>Long to implement this. UFS2 has soft-updates, which keeps meta-data >>consistant in case of system failure (reboots,etc), but does not avoid >>an fsck. The 5.x series and above has support for background fsck, >>which allows you to mount the filesystem and begin using it, while >>background fsck does the checking live. > > >>There are read-only ports of reiserfs and xfs available for 6.0- and >>7.0. The reiserfs code is in the src tree, and the xfs code can be >>found here: >>http://people.freebsd.org/~rodrigc/xfs/ > > >>I'm sure Craig and Alexander would enjoy the help with the porting of >>XFS. I'm unsure of the status of reiserfs. Porting XFS to FreeBSD >>(full support) would be awesome! > > >>Eric > > > > > > > > -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Fri Sep 9 12:12:16 2005 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8836816A41F for ; Fri, 9 Sep 2005 12:12:16 +0000 (GMT) (envelope-from kamalpr@gmail.com) Received: from xproxy.gmail.com (xproxy.gmail.com [66.249.82.205]) by mx1.FreeBSD.org (Postfix) with ESMTP id EE92D43D48 for ; Fri, 9 Sep 2005 12:12:15 +0000 (GMT) (envelope-from kamalpr@gmail.com) Received: by xproxy.gmail.com with SMTP id i27so1302568wxd for ; Fri, 09 Sep 2005 05:12:15 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:sender:to:subject:in-reply-to:mime-version:content-type:references; b=FeSOL+eSUC2p7nWFiShVMSEX0VSVf/dTwPiSAvYUwxLpAnFGzCjU27ud9gtq8tnWWWFgN3/1XV6j7aY7lvIrlzeegKTXv48PclMz9DgCx2jiFVfDIXJracYglLSRGwPAuL1eHyGp+HuKIqgQ/wE4pWX87UNWhMniH8TEHkvim4Q= Received: by 10.70.131.13 with SMTP id e13mr4317wxd; Fri, 09 Sep 2005 05:12:15 -0700 (PDT) Received: by 10.70.23.12 with HTTP; Fri, 9 Sep 2005 05:12:15 -0700 (PDT) Message-ID: Date: Fri, 9 Sep 2005 17:42:15 +0530 From: "Kamal R. Prasad" Sender: kamalpr@gmail.com To: freebsd-fs@freebsd.org In-Reply-To: <20050909122506.K33344@fledge.watson.org> Mime-Version: 1.0 References: <868xyack37.fsf@xps.des.no> <20050906191929.E78038@fledge.watson.org> <200509070215.j872FeQE040259@apollo.backplane.com> <20050907111035.B85520@fledge.watson.org> <200509071623.j87GNpal043201@apollo.backplane.com> <20050909122506.K33344@fledge.watson.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Fwd: JFS2 on freebsd X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: "Kamal R. Prasad" List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Sep 2005 12:12:16 -0000 Hello, Can someone tell me about journaling support being added into freebsd=20 (ufs2) and/or support for jfs2? thanks -kamal ---------- Forwarded message ---------- From: Robert Watson Date: Sep 9, 2005 4:58 PM Subject: Re: JFS2 on freebsd To: "Kamal R. Prasad" Cc: freebsd-hackers@freebsd.org On Thu, 8 Sep 2005, Kamal R. Prasad wrote: > Has there been any work on porting JFS2 onto Freebsd? There has been recent work to port several of the newer Linux file systems to FreeBSD, including: - Pretty old work to get the basic JFS userland tools working (status unknown, likely very stale due to the passage of time). - Pretty recent work to get read-only reiserfs working (committed and in the CVS repository). - Pretty recent work to get read-only XFS working (external repository, but publicly available). Also potentially of interesting: - Increasingly dated work to port the pre-journalled version of HFS+ to FreeBSD, which works well subject to the datedness and pre-journalledness of the work. There's also on-going work on a journalled version of UFS. I'm sure the authors of any of these would be interested in someone lending a hand -- I know there have been specific appeals for interest from the XFS crowd in the last month, along with test patches, etc, for example. You might want to post to freebsd-fs looking for details on the various projects. Robert N M Watson From owner-freebsd-fs@FreeBSD.ORG Sat Sep 10 15:28:19 2005 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4A49D16A459 for ; Sat, 10 Sep 2005 15:28:19 +0000 (GMT) (envelope-from sebastien.bourdeauducq@gmail.com) Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.200]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7416343D48 for ; Sat, 10 Sep 2005 15:28:18 +0000 (GMT) (envelope-from sebastien.bourdeauducq@gmail.com) Received: by wproxy.gmail.com with SMTP id i21so1362097wra for ; Sat, 10 Sep 2005 08:28:17 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:from:to:subject:date:user-agent:mime-version:content-type:content-transfer-encoding:content-disposition:message-id; b=km7XNHAZT7UVmEFN7JH4BLDvCu+Z7BEOmC5wXXZXbMyAOLXnUW/PvXUCCajk/p3MRhNKzosOljkxhgPR+gs564G7bMkNHdMtJuxZK3HPGcsEkmH0QtqfqEUYVrZq9dIFgw3vJ6J9rhuzAxaWSUwbPrMkmoM2RwzLSl88L9AnGoc= Received: by 10.54.33.57 with SMTP id g57mr1229879wrg; Sat, 10 Sep 2005 08:28:17 -0700 (PDT) Received: from AAmiens-152-1-15-78.w83-192.abo.wanadoo.fr ( [83.192.72.78]) by mx.gmail.com with ESMTP id 16sm253471wrl.2005.09.10.08.28.15; Sat, 10 Sep 2005 08:28:16 -0700 (PDT) From: Sebastien To: freebsd-fs@freebsd.org Date: Sat, 10 Sep 2005 17:29:24 +0200 User-Agent: KMail/1.8 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200509101729.24912.sebastien.bourdeauducq@gmail.com> Subject: Filesystem access from a KLD causes "vrele: negative ref cnt" panic X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Sep 2005 15:28:19 -0000 Hello, When the following kernel code has been run : ------------------------ struct thread *td = curthread; struct nameidata nd; int flags, resid; struct vattr vat; /* Some threads don't have a current working directory, set this to avoid a page fault on vn_open() call */ if(td->td_proc->p_fd->fd_rdir == NULL) td->td_proc->p_fd->fd_rdir = rootvnode; if(td->td_proc->p_fd->fd_cdir == NULL) td->td_proc->p_fd->fd_cdir = rootvnode; NDINIT(&nd, LOOKUP, FOLLOW, UIO_SYSSPACE, &filename[0], td); flags = FREAD; vn_open(&nd, &flags, 0, -1); NDFREE(&nd, NDF_ONLY_PNBUF); /* Get the file size. */ VOP_GETATTR(nd.ni_vp, &vat, td->td_ucred, td); VOP_UNLOCK(nd.ni_vp, 0, td); vn_rdwr(UIO_READ, nd.ni_vp, buf, vat.va_size, 0, UIO_SYSSPACE, IO_NODELOCKED, td->td_ucred, NOCRED, &resid, td); vn_close(nd.ni_vp, FREAD, td->td_ucred, td); ------------------------ I have a random panic "vrele: negative ref cnt" when I shutdown the system. Am I double-freeing something in my code ? The fact that the panic is caused randomly suggests there is a synchronization problem - but the above code is always running under the protection of the Giant mutex. Regards, Sebastien