From owner-freebsd-fs@FreeBSD.ORG Mon Jun 7 15:18:55 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3E04716A4CE for ; Mon, 7 Jun 2004 15:18:55 +0000 (GMT) Received: from web21509.mail.yahoo.com (web21509.mail.yahoo.com [66.163.169.58]) by mx1.FreeBSD.org (Postfix) with SMTP id 2C4AF43D62 for ; Mon, 7 Jun 2004 15:18:55 +0000 (GMT) (envelope-from email_saju@yahoo.com) Message-ID: <20040607151854.40229.qmail@web21509.mail.yahoo.com> Received: from [193.32.3.82] by web21509.mail.yahoo.com via HTTP; Mon, 07 Jun 2004 08:18:54 PDT Date: Mon, 7 Jun 2004 08:18:54 -0700 (PDT) From: Saju Pillai To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: pointer to FS implementation docs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jun 2004 15:18:55 -0000 Hello, I am trying to get familiar with the vfs and specific the n/w filesystems available on freebsd (5.1). Apart from "man 9 VFS" and the /usr/src/fs, is there any external documentation I can look at ? regards srp __________________________________ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/ From owner-freebsd-fs@FreeBSD.ORG Tue Jun 8 07:32:20 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5BC5416A4CE for ; Tue, 8 Jun 2004 07:32:20 +0000 (GMT) Received: from lucius.provo.novell.com (lucius.provo.novell.com [137.65.81.172]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1196443D2D for ; Tue, 8 Jun 2004 07:32:20 +0000 (GMT) (envelope-from vrashmi@novell.com) Received: from INET-PRV1-MTA by lucius.provo.novell.com with Novell_GroupWise; Tue, 08 Jun 2004 01:32:17 -0600 Message-Id: X-Mailer: Novell GroupWise Internet Agent 6.5.2 Beta Date: Tue, 08 Jun 2004 01:32:02 -0600 From: "Rashmi Vittal" To: Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Disposition: inline Subject: Re: pointer to FS implementation docs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jun 2004 07:32:20 -0000 http://www.nobius.org/~dbg/practical-file-system-design.pdf should help. Rashmi >>> Saju Pillai 6/7/2004 8:48:54 PM >>> Hello, I am trying to get familiar with the vfs and specific the n/w filesystems available on freebsd (5.1). Apart from "man 9 VFS" and the /usr/src/fs, is there any external documentation I can look at ? regards srp __________________________________ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/ _______________________________________________ freebsd-fs@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Tue Jun 8 21:49:00 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AE52616A4CE for ; Tue, 8 Jun 2004 21:49:00 +0000 (GMT) Received: from hewey.af.speednet.com.au (udsl-3-062.QLD.dft.com.au [202.168.108.62]) by mx1.FreeBSD.org (Postfix) with ESMTP id 71C1643D2F for ; Tue, 8 Jun 2004 21:48:59 +0000 (GMT) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [172.22.2.17])i58LmgKV004276; Wed, 9 Jun 2004 07:48:43 +1000 (EST) (envelope-from andyf@speednet.com.au) Date: Wed, 9 Jun 2004 07:48:42 +1000 (EST) From: Andy Farkas X-X-Sender: andyf@hewey.af.speednet.com.au To: Kris Kennaway In-Reply-To: <20040602221825.GB89451@xor.obsecurity.org> Message-ID: <20040609074726.G868@hewey.af.speednet.com.au> References: <20040531213127.1eb7224c@vixen42.24-119-122-191.cpe.cableone.net> <20040602130952.49fb2561@vixen42.24-119-122-191.cpe.cableone.net> <20040602221825.GB89451@xor.obsecurity.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-fs@freebsd.org cc: Vulpes Velox Subject: Re: file space question X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jun 2004 21:49:00 -0000 On Wed, 2 Jun 2004, Kris Kennaway wrote: > On Wed, Jun 02, 2004 at 01:09:52PM -0500, Vulpes Velox wrote: > > > So storing bookmarks and the like each in their own file, on UFS is a > > bad idea then? Or given modern disk sizes probally can easily be > > ignored? > > Is disk space really so tight for you that you have to worry where > each 2K goes? :-) > > Kris > Some people may have to worry about it: sd0 at scsibus0 target 2 lun 0: SCSI1 0/direct fixed sd0: 100 MB, 776 cyl, 8 head, 33 sec, 512 bytes/sect x 204864 sectors sd0: async, 8-bit transfers sd1 at scsibus0 target 3 lun 0: SCSI1 0/direct fixed sd1: 100 MB, 776 cyl, 8 head, 33 sec, 512 bytes/sect x 204864 sectors sd1: async, 8-bit transfers ;) -- :{ andyf@speednet.com.au Andy Farkas System Administrator Speednet Communications http://www.speednet.com.au/ From owner-freebsd-fs@FreeBSD.ORG Tue Jun 8 23:35:05 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 80AE616A4CE for ; Tue, 8 Jun 2004 23:35:05 +0000 (GMT) Received: from mail5.dslextreme.com (mail5.dslextreme.com [66.51.199.81]) by mx1.FreeBSD.org (Postfix) with SMTP id 51A5A43D53 for ; Tue, 8 Jun 2004 23:35:05 +0000 (GMT) (envelope-from jmlewis@dslextreme.com) Received: (qmail 29528 invoked from network); 8 Jun 2004 23:35:02 -0000 Received: from unknown (HELO www.dslextreme.com) (66.51.199.92) by 192.168.8.93 with SMTP; Tue, 08 Jun 2004 23:35:02 +0000 Message-ID: <4eca4272a1905a2139a.20040608163503.wzyrjvf@www.dslextreme.com> Date: Tue, 8 Jun 2004 16:35:03 -0700 (PDT) From: "Joshua Lewis" To: freebsd-fs@freebsd.org User-Agent: DSL Extreme Webmail (www.dslextreme.com) MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 Importance: Normal X-AntiVirus: scanned for viruses by AMaViS 0.2.1 (http://amavis.org/) Subject: Frequent Power Failures X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: jmlewis@dslextreme.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jun 2004 23:35:05 -0000 My office is located in Southern California and of late we have had almost ten power failures. I have set up my first FreeBSD box and am learning quickly. If my memory serves I recall Unix writes to disk as little as possible to keep the system running at peak efficiency. However I don't want to risk data loss. So until I can get a UPS (that is absolutely next on my list) I want set the system to write to the disk as soon as possible. I am currently looking through sysctl in the handbook to find some setting that may enable writing data immediately. Am I in the correct place? There are A LOT of settings here can anyone point me to the correct setting so I don't spend the rest of my life reading. Thank you, Joshua Lewis From owner-freebsd-fs@FreeBSD.ORG Tue Jun 8 23:40:01 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BF6CC16A4CE for ; Tue, 8 Jun 2004 23:40:01 +0000 (GMT) Received: from mail5.dslextreme.com (mail5.dslextreme.com [66.51.199.81]) by mx1.FreeBSD.org (Postfix) with SMTP id 9259D43D2F for ; Tue, 8 Jun 2004 23:40:01 +0000 (GMT) (envelope-from jmlewis@dslextreme.com) Received: (qmail 30628 invoked from network); 8 Jun 2004 23:40:00 -0000 Received: from unknown (HELO www.dslextreme.com) (66.51.199.92) by 192.168.8.93 with SMTP; Tue, 08 Jun 2004 23:40:00 +0000 Message-ID: <1e0a1950a988aca8a.20040608164001.wzyrjvf@www.dslextreme.com> In-Reply-To: <834a6ebea29b3a375fa.20040608163505.wzyrjvf@www.dslextreme.com> References: <834a6ebea29b3a375fa.20040608163505.wzyrjvf@www.dslextreme.com> Date: Tue, 8 Jun 2004 16:40:01 -0700 (PDT) From: "Joshua Lewis" To: freebsd-fs@freebsd.org User-Agent: DSL Extreme Webmail (www.dslextreme.com) MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 Importance: Normal X-AntiVirus: scanned for viruses by AMaViS 0.2.1 (http://amavis.org/) Subject: Re: Frequent Power Failures X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: jmlewis@dslextreme.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jun 2004 23:40:01 -0000 I think I found the setting. Ironic is was the next setting. Is this the correct setting to help prevent data loss (at from what I guess is a MAJOR performance hit.) hw.ata.wc Thank you, Joshua Lewis Joshua Lewis > My office is located in Southern California and of late we have had almost > ten power failures. > > I have set up my first FreeBSD box and am learning quickly. If my memory > serves I recall Unix writes to disk as little as possible to keep the > system running at peak efficiency. However I don't want to risk data loss. > So until I can get a UPS (that is absolutely next on my list) I want set > the system to write to the disk as soon as possible. > > I am currently looking through sysctl in the handbook to find some setting > that may enable writing data immediately. > > Am I in the correct place? There are A LOT of settings here can anyone > point me to the correct setting so I don't spend the rest of my life > reading. > > > Thank you, > Joshua Lewis > > > > > From owner-freebsd-fs@FreeBSD.ORG Fri Jun 11 16:32:25 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EB23716A4CE for ; Fri, 11 Jun 2004 16:32:25 +0000 (GMT) Received: from sccmmhc91.asp.att.net (sccmmhc91.asp.att.net [204.127.203.211]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8BED543D58 for ; Fri, 11 Jun 2004 16:32:25 +0000 (GMT) (envelope-from brian@classicalguitar.net) Received: from [192.168.1.100] (12-217-90-214.client.mchsi.com[12.217.90.214]) by sccmmhc91.asp.att.net (sccmmhc91) with SMTP id <20040611163148m9100cqq1oe>; Fri, 11 Jun 2004 16:31:48 +0000 Mime-Version: 1.0 (Apple Message framework v618) Content-Transfer-Encoding: 7bit Message-Id: Content-Type: text/plain; charset=US-ASCII; format=fixed To: fs@freebsd.org From: Brian Bergstrand Date: Fri, 11 Jun 2004 11:31:47 -0500 X-Pgp-Rfc2646-Fix: 1 X-Mailer: Apple Mail (2.618) Subject: Ext2 vs UFS getlbns X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jun 2004 16:32:26 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I just noticed something in ext2_getlbns() (ext2_bmap.c, v1.57) vs. ufs_getlbns() (ufs_bmap.c, v1.60) In the last loop to setup the indir array, UFS does: { ... blockcnt /= MNINDIR(ump); off = (bn / blockcnt) % MNINDIR(ump); ++numlevels; ap->in_lbn = metalbn; ap->in_off = off; ap->in_exists = 0; ++ap; metalbn -= -1 + off * blockcnt; } While Ext2 does: { ... off = (bn / blockcnt) % MNINDIR(ump); ++numlevels; ap->in_lbn = metalbn; ap->in_off = off; ap->in_exists = 0; ++ap; metalbn -= -1 + off * blockcnt; blockcnt /= MNINDIR(ump); } Notice that blockcnt is changed AFTER offset/metalbn in Ext2 and BEFORE those in UFS. Was this change in Ext2 done on purpose for some reason? It makes a difference in calculating in_off and metalbn for some block #'s. Thanks. Brian Bergstrand , AIM: triryche206 PGP Key: If all else fails, lower your standards. As of 11:31:08 AM, iTunes is playing "Tristessa" from "Gish" by "Smashing Pumpkins" -----BEGIN PGP SIGNATURE----- Version: PGP 8.0.3 iQA/AwUBQMnQZHnR2Fu2x7aiEQK+QgCeJynMXuz9NsR+HBh+LDGKjdDT5SUAnAqc x2FZQ7uaURUzxOOTItxByl4D =5IRG -----END PGP SIGNATURE----- From owner-freebsd-fs@FreeBSD.ORG Fri Jun 11 18:14:49 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BDBFB16A4CE for ; Fri, 11 Jun 2004 18:14:49 +0000 (GMT) Received: from mailout2.pacific.net.au (mailout2.pacific.net.au [61.8.0.85]) by mx1.FreeBSD.org (Postfix) with ESMTP id 15D6D43D31 for ; Fri, 11 Jun 2004 18:14:49 +0000 (GMT) (envelope-from bde@zeta.org.au) Received: from mailproxy1.pacific.net.au (mailproxy1.pacific.net.au [61.8.0.86])i5BIEj5v032655; Sat, 12 Jun 2004 04:14:45 +1000 Received: from gamplex.bde.org (katana.zip.com.au [61.8.7.246]) i5BIEg2O022504; Sat, 12 Jun 2004 04:14:44 +1000 Date: Sat, 12 Jun 2004 04:14:42 +1000 (EST) From: Bruce Evans X-X-Sender: bde@gamplex.bde.org To: Brian Bergstrand In-Reply-To: Message-ID: <20040612035325.D15028@gamplex.bde.org> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: fs@freebsd.org Subject: Re: Ext2 vs UFS getlbns X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jun 2004 18:14:49 -0000 On Fri, 11 Jun 2004, Brian Bergstrand wrote: > I just noticed something in ext2_getlbns() (ext2_bmap.c, v1.57) vs. > ufs_getlbns() (ufs_bmap.c, v1.60) > > In the last loop to setup the indir array, > > UFS does: > > { > ... > blockcnt /= MNINDIR(ump); > off = (bn / blockcnt) % MNINDIR(ump); > > ++numlevels; > ap->in_lbn = metalbn; > ap->in_off = off; > ap->in_exists = 0; > ++ap; > > metalbn -= -1 + off * blockcnt; > } > > While Ext2 does: > > { > ... > off = (bn / blockcnt) % MNINDIR(ump); > > ++numlevels; > ap->in_lbn = metalbn; > ap->in_off = off; > ap->in_exists = 0; > ++ap; > > metalbn -= -1 + off * blockcnt; > blockcnt /= MNINDIR(ump); > } > > Notice that blockcnt is changed AFTER offset/metalbn in Ext2 and BEFORE > those in UFS. > > Was this change in Ext2 done on purpose for some reason? It makes a > difference in calculating in_off and metalbn for some block #'s. This is to fix overflow in the calculation of block numbers for triple indirect blocks. ffs used to do this, and ext2_getlbns() was a copy ufs_getlbns(), but ffs was changed back to use simpler code when its daddr type (ufs2_daddr_t) was changed to 64 bits and the longs in ufs_getlbns() were fixed to use ufs_daddr_t. Overflow is probably theoretically possible again, but it would take a 128-bit calculation to avoid it and 64 bit block numbers should be enough for anyone. This difference shouldn't affect in_off or metalbn for any reachable block number (32 bit ones in ext2fs). There is another variable "int64_t qblockcnt" that is used instead of "long blockcnt" in some places in ext2_getlbns(). The logic for using blockcnt in the above is a little different because earlier calculations set qblockcnt instead of qblockcnt. Bruce From owner-freebsd-fs@FreeBSD.ORG Fri Jun 11 19:33:36 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E727616A4D0 for ; Fri, 11 Jun 2004 19:33:36 +0000 (GMT) Received: from liquidsky.homeunix.org (12-217-90-214.client.mchsi.com [12.217.90.214]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7980543D46 for ; Fri, 11 Jun 2004 19:33:36 +0000 (GMT) (envelope-from brian@classicalguitar.net) Received: from [127.0.0.1] (localhost [127.0.0.1]) by postfix.bergstrand.org (Postfix) with ESMTP id C553A39C13 for ; Thu, 10 Jun 2004 20:13:35 -0500 (CDT) Mime-Version: 1.0 (Apple Message framework v618) Content-Transfer-Encoding: 7bit Message-Id: <8C40210D-BB44-11D8-B357-0003930A674E@classicalguitar.net> Content-Type: text/plain; charset=US-ASCII; format=fixed To: fs@freebsd.org From: Brian Bergstrand Date: Thu, 10 Jun 2004 20:13:29 -0500 X-Pgp-Rfc2646-Fix: 1 X-Mailer: Apple Mail (2.618) Subject: Ext2 vs UFS getlbns X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jun 2004 19:33:37 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I just noticed something in ext2_getlbns() (ext2_bmap.c, v1.57) vs. ufs_getlbns() (ufs_bmap.c, v1.60) In the last loop to setup the indir array, UFS does: { ... blockcnt /= MNINDIR(ump); off = (bn / blockcnt) % MNINDIR(ump); ++numlevels; ap->in_lbn = metalbn; ap->in_off = off; ap->in_exists = 0; ++ap; metalbn -= -1 + off * blockcnt; } While Ext2 does: { ... off = (bn / blockcnt) % MNINDIR(ump); ++numlevels; ap->in_lbn = metalbn; ap->in_off = off; ap->in_exists = 0; ++ap; metalbn -= -1 + off * blockcnt; blockcnt /= MNINDIR(ump); } Notice that blockcnt is calculated AFTER the offset in Ext2 and BEFORE the offset in UFS. Was this change in Ext2 done on purpose for some reason, or does it not make a difference? I would think with some block block #'s it would. Thanks. Brian Bergstrand , AIM: triryche206 PGP Key: Hell, there are no rules here--we're trying to accomplish something. - Thomas A. Edison As of 07:50:41 PM, iTunes is playing "Bouncing Around The Room" from "Lawn Boy" by "Phish" -----BEGIN PGP SIGNATURE----- Version: PGP 8.0.3 iQA/AwUBQMj5L3nR2Fu2x7aiEQJJPACePBqrrfWVXW+VJjx5ucfo+WPJ7+oAoODa v40zOqo1FAmMVcgL2NSAoMsL =jtnG -----END PGP SIGNATURE----- From owner-freebsd-fs@FreeBSD.ORG Fri Jun 11 19:34:34 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9E3A416A4CE for ; Fri, 11 Jun 2004 19:34:34 +0000 (GMT) Received: from liquidsky.homeunix.org (12-217-90-214.client.mchsi.com [12.217.90.214]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5B19D43D48 for ; Fri, 11 Jun 2004 19:34:34 +0000 (GMT) (envelope-from brian@classicalguitar.net) Received: from [127.0.0.1] (localhost [127.0.0.1]) by liquidsky.homeunix.org (Postfix) with ESMTP id A130139C82; Fri, 11 Jun 2004 14:34:33 -0500 (CDT) In-Reply-To: <20040612035325.D15028@gamplex.bde.org> References: <20040612035325.D15028@gamplex.bde.org> Mime-Version: 1.0 (Apple Message framework v618) Content-Type: text/plain; charset=US-ASCII; format=fixed Message-Id: <5A58608E-BBDE-11D8-8C2A-0003930A674E@classicalguitar.net> Content-Transfer-Encoding: 7bit From: Brian Bergstrand Date: Fri, 11 Jun 2004 14:34:28 -0500 To: Bruce Evans X-Pgp-Rfc2646-Fix: 1 X-Mailer: Apple Mail (2.618) cc: fs@freebsd.org Subject: Re: Ext2 vs UFS getlbns X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jun 2004 19:34:34 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 11, 2004, at 1:14 PM, Bruce Evans wrote: > On Fri, 11 Jun 2004, Brian Bergstrand wrote: > >> I just noticed something in ext2_getlbns() (ext2_bmap.c, v1.57) vs. >> ufs_getlbns() (ufs_bmap.c, v1.60) >> >> ... >> >> Notice that blockcnt is changed AFTER offset/metalbn in Ext2 and >> BEFORE >> those in UFS. >> >> Was this change in Ext2 done on purpose for some reason? It makes a >> difference in calculating in_off and metalbn for some block #'s. > > This is to fix overflow in the calculation of block numbers for triple > indirect blocks. ffs used to do this, and ext2_getlbns() was a copy > ufs_getlbns(), but ffs was changed back to use simpler code when its > daddr type (ufs2_daddr_t) was changed to 64 bits and the longs in > ufs_getlbns() were fixed to use ufs_daddr_t. Overflow is probably > theoretically possible again, but it would take a 128-bit calculation > to avoid it and 64 bit block numbers should be enough for anyone. > > This difference shouldn't affect in_off or metalbn for any reachable > block number (32 bit ones in ext2fs). There is another variable > "int64_t qblockcnt" that is used instead of "long blockcnt" in > some places in ext2_getlbns(). The logic for using blockcnt in the > above is a little different because earlier calculations set > qblockcnt instead of qblockcnt. > > Bruce > Bruce, thanks for the explanation. The reason that I originally asked, is because I'm seeing different offsets and metalbns for relatively small block #'s in the OS X port. The OS X code is derived from FreeBSD 5.x and ext2_getlbns() has not changed For instance, on a 1KB block FS (which therefore has 256 block entries per indirect block) I see the following with the original algo.: Write 21 bytes at offset 0: lbn = 0 indir not set because this falls in the direct block range Write 21 bytes at offset 12288: lbn=12 {{ in_lbn = -12, in_off = 0, in_exists = 0 }, { in_lbn = -12, in_off = 0, in_exists = 0 }, { .... Write 21 bytes at offset 4194304 lbn=4096 {{ in_lbn = -269, in_off = 1, in_exists = 0 }, { in_lbn = -269, in_off = 14, in_exists = 0 }, { in_lbn = -3852, in_off = 244, in_exists = 0 }, { ... But, if I move the blockcnt cal to where UFS has it I get: Write 21 bytes at offset 0: same Write 21 bytes at offset 12288: same Write 21 bytes at offset 4194304 lbn=4096 {{ in_lbn = -269, in_off = 1, in_exists = 0 }, { in_lbn = -269, in_off = 244, in_exists = 0 }, { in_lbn = -512, in_off = 0, in_exists = 0 }, { ... Notice how indir[1].off, indir[2].off and indir[2].in_lbn are different from the first run (with the current ext2 algo). The same thing happens with 8MB and 16MB offset writes too. Any ideas why this happens? Here's my simplified test case to simulate what happens on a 1KB block FS: #include #include #include struct vnode { }; struct indir { int in_lbn; int in_off; int in_exists; }; struct ext2mount { int u_mindir; }; #define MNINDIR(m) (m)->u_mindir; #define NDADDR 12 #define NIADDR 3 int ext2_getlbns(vp, bn, ap, nump) struct vnode *vp; int32_t bn; struct indir *ap; int *nump; { long blockcnt, metalbn, realbn; struct ext2mount *ump; int i, numlevels, off; int64_t qblockcnt; //ump = VFSTOEXT2(vp->v_mount); struct ext2mount e2mt = {256}; ump = &e2mt; if (nump) *nump = 0; numlevels = 0; realbn = bn; if ((long)bn < 0) bn = -(long)bn; /* The first NDADDR blocks are direct blocks. */ if (bn < NDADDR) return (0); /* * Determine the number of levels of indirection. After this loop * is done, blockcnt indicates the number of data blocks possible * at the previous level of indirection, and NIADDR - i is the number * of levels of indirection needed to locate the requested block. */ for (blockcnt = 1, i = NIADDR, bn -= NDADDR;; i--, bn -= blockcnt) { if (i == 0) return (EFBIG); /* * Use int64_t's here to avoid overflow for triple indirect * blocks when longs have 32 bits and the block size is more * than 4K. */ qblockcnt = (int64_t)blockcnt * MNINDIR(ump); if (bn < qblockcnt) break; blockcnt = qblockcnt; } /* Calculate the address of the first meta-block. */ if (realbn >= 0) metalbn = -(realbn - bn + NIADDR - i); else metalbn = -(-realbn - bn + NIADDR - i); /* * At each iteration, off is the offset into the bap array which is * an array of disk addresses at the current level of indirection. * The logical block number and the offset in that block are stored * into the argument array. */ ap->in_lbn = metalbn; ap->in_off = off = NIADDR - i; ap->in_exists = 0; ap++; for (++numlevels; i <= NIADDR; i++) { /* If searching for a meta-data block, quit when found. */ if (metalbn == realbn) break; //blockcnt /= MNINDIR(ump); off = (bn / blockcnt) % MNINDIR(ump); ++numlevels; ap->in_lbn = metalbn; ap->in_off = off; ap->in_exists = 0; ++ap; metalbn -= -1 + off * blockcnt; blockcnt /= MNINDIR(ump); } if (nump) *nump = numlevels; return (0); } int main (int argc, char *argvp[]) { struct vnode v; struct indir indir[NIADDR+2]; int num; (void)ext2_getlbns(&v, 0, indir, &num); (void)ext2_getlbns(&v, (12288>>10), indir, &num); (void)ext2_getlbns(&v, (4194304>>10), indir, &num); (void)ext2_getlbns(&v, (24576>>10), indir, &num); (void)ext2_getlbns(&v, (8388608>>10), indir, &num); (void)ext2_getlbns(&v, (49152>>10), indir, &num); (void)ext2_getlbns(&v, (16777216>>10), indir, &num); } Thanks. Brian Bergstrand , AIM: triryche206 PGP Key: It is easy to convert a bug to a feature; document it. - The Microsoft Employee Handbook, p.215, section B, article VII As of 02:00:08 PM, iTunes is playing "Tears In Heaven" from "MTV Unplugged" by "Eric Clapton" -----BEGIN PGP SIGNATURE----- Version: PGP 8.0.3 iQA/AwUBQMn7OXnR2Fu2x7aiEQKNoACg2wQGJNSmmZLNe8iyyh7QhaP5ZwAAoPmK f0DJmDF8IEJ5C4Vke4jxtBwC =yBtM -----END PGP SIGNATURE----- From owner-freebsd-fs@FreeBSD.ORG Sat Jun 12 15:26:13 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 083A916A4CE for ; Sat, 12 Jun 2004 15:26:13 +0000 (GMT) Received: from mailout1.pacific.net.au (mailout1.pacific.net.au [61.8.0.84]) by mx1.FreeBSD.org (Postfix) with ESMTP id 53E7143D2D for ; Sat, 12 Jun 2004 15:26:12 +0000 (GMT) (envelope-from bde@zeta.org.au) Received: from mailproxy2.pacific.net.au (mailproxy2.pacific.net.au [61.8.0.87])i5CFPW4u015018; Sun, 13 Jun 2004 01:25:33 +1000 Received: from gamplex.bde.org (katana.zip.com.au [61.8.7.246]) i5CFPTLS032650; Sun, 13 Jun 2004 01:25:31 +1000 Date: Sun, 13 Jun 2004 01:25:29 +1000 (EST) From: Bruce Evans X-X-Sender: bde@gamplex.bde.org To: Brian Bergstrand In-Reply-To: <5A58608E-BBDE-11D8-8C2A-0003930A674E@classicalguitar.net> Message-ID: <20040613002619.J1310@gamplex.bde.org> References: <5A58608E-BBDE-11D8-8C2A-0003930A674E@classicalguitar.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: fs@freebsd.org Subject: Re: Ext2 vs UFS getlbns X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jun 2004 15:26:13 -0000 On Fri, 11 Jun 2004, Brian Bergstrand wrote: > On Jun 11, 2004, at 1:14 PM, Bruce Evans wrote: > > > On Fri, 11 Jun 2004, Brian Bergstrand wrote: > > > >> I just noticed something in ext2_getlbns() (ext2_bmap.c, v1.57) vs. > >> ufs_getlbns() (ufs_bmap.c, v1.60) > >> > >> ... > >> > >> Notice that blockcnt is changed AFTER offset/metalbn in Ext2 and > >> BEFORE > >> those in UFS. > >> > >> Was this change in Ext2 done on purpose for some reason? It makes a > >> difference in calculating in_off and metalbn for some block #'s. > > > > This is to fix overflow in the calculation of block numbers for triple > > indirect blocks. ffs used to do this, and ext2_getlbns() was a copy > > .. > > > > This difference shouldn't affect in_off or metalbn for any reachable > > block number (32 bit ones in ext2fs). There is another variable > > ... > > The reason that I originally asked, is because I'm seeing different > offsets and metalbns for relatively small block #'s in the OS X port. > The OS X code is derived from FreeBSD 5.x and ext2_getlbns() has not > changed > > For instance, on a 1KB block FS (which therefore has 256 block entries > per indirect block) I see the following with the original algo.: > > Write 21 bytes at offset 0: > lbn = 0 > indir not set because this falls in the direct block range > > Write 21 bytes at offset 12288: > lbn=12 > {{ > in_lbn = -12, > in_off = 0, > in_exists = 0 > }, { > in_lbn = -12, > in_off = 0, > in_exists = 0 > }, { > .... > > Write 21 bytes at offset 4194304 > lbn=4096 > {{ > in_lbn = -269, > in_off = 1, > in_exists = 0 > }, { > in_lbn = -269, > in_off = 14, > in_exists = 0 > }, { > in_lbn = -3852, > in_off = 244, > in_exists = 0 > }, { > ... This seems to be correct. E.g., the first 14 blocks pointed to by the second indirect block are full and point to 256 data blocks. The 15th block pointed to by the second indirect block points to 244 data blocks (4096 = 12 + 256 + 14 * 256 + 244). > But, if I move the blockcnt cal to where UFS has it I get: > > Write 21 bytes at offset 0: same > > Write 21 bytes at offset 12288: same > > Write 21 bytes at offset 4194304 > lbn=4096 > {{ > in_lbn = -269, > in_off = 1, > in_exists = 0 > }, { > in_lbn = -269, > in_off = 244, > in_exists = 0 > }, { > in_lbn = -512, > in_off = 0, > in_exists = 0 > }, { > ... This seems to be incorrect. It doesn't have the magic number 14, and I can't see how the magic number of -512 can be relevant. > > Notice how indir[1].off, indir[2].off and indir[2].in_lbn are different > from the first run (with the current ext2 algo). The same thing happens > with 8MB and 16MB offset writes too. > > Any ideas why this happens? > > Here's my simplified test case to simulate what happens on a 1KB block > FS: I merged the current version of ufs_getblns() into your test and got the same results as with the current ext2_lbns(). %%% diff -c2 z.c~ z.c *** z.c~ Sat Jun 12 23:29:43 2004 --- z.c Sun Jun 13 00:25:36 2004 *************** *** 23,36 **** int ! ext2_getlbns(vp, bn, ap, nump) ! struct vnode *vp; ! int32_t bn; ! struct indir *ap; ! int *nump; { ! long blockcnt, metalbn, realbn; struct ext2mount *ump; int i, numlevels, off; - int64_t qblockcnt; //ump = VFSTOEXT2(vp->v_mount); --- 23,31 ---- int ! ext2_getlbns(struct vnode *vp, int64_t bn, struct indir *ap, int *nump) { ! int64_t blockcnt, metalbn, realbn; struct ext2mount *ump; int i, numlevels, off; //ump = VFSTOEXT2(vp->v_mount); *************** *** 44,49 **** numlevels = 0; realbn = bn; ! if ((long)bn < 0) ! bn = -(long)bn; /* The first NDADDR blocks are direct blocks. */ --- 39,44 ---- numlevels = 0; realbn = bn; ! if (bn < 0) ! bn = -bn; /* The first NDADDR blocks are direct blocks. */ *************** *** 60,72 **** if (i == 0) return (EFBIG); ! /* ! * Use int64_t's here to avoid overflow for triple indirect ! * blocks when longs have 32 bits and the block size is more ! * than 4K. ! */ ! qblockcnt = (int64_t)blockcnt * MNINDIR(ump); ! if (bn < qblockcnt) break; - blockcnt = qblockcnt; } --- 55,61 ---- if (i == 0) return (EFBIG); ! blockcnt *= MNINDIR(ump); ! if (bn < blockcnt) break; } *************** *** 92,95 **** --- 81,85 ---- break; + blockcnt /= MNINDIR(ump); //blockcnt /= MNINDIR(ump); off = (bn / blockcnt) % MNINDIR(ump); *************** *** 102,106 **** metalbn -= -1 + off * blockcnt; - blockcnt /= MNINDIR(ump); } if (nump) --- 92,95 ---- %%% This patch was adapted from "cvs diff -c1 -r 1.51 -r 1.60 ufs_bmap.c". Bruce