From owner-freebsd-fs@FreeBSD.ORG Tue Feb 10 10:33:20 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3BBF216A4CE for ; Tue, 10 Feb 2004 10:33:20 -0800 (PST) Received: from mail-out3.apple.com (mail-out3.apple.com [17.254.13.22]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3396C43D1F for ; Tue, 10 Feb 2004 10:33:20 -0800 (PST) (envelope-from mday@apple.com) Received: from mailgate1.apple.com (a17-128-100-225.apple.com [17.128.100.225]) by mail-out3.apple.com (8.12.11/8.12.11) with ESMTP id i1AIXJ2j018922 for ; Tue, 10 Feb 2004 10:33:19 -0800 (PST) Received: from relay3.apple.com (relay3.apple.com) by mailgate1.apple.com ; Tue, 10 Feb 2004 10:33:19 -0800 Received: from [17.202.44.244] (daylight.apple.com [17.202.44.244]) by relay3.apple.com (8.12.11/8.12.11) with ESMTP id i1AIXFSX027228; Tue, 10 Feb 2004 18:33:16 GMT In-Reply-To: <20040207155819.186335d4@it.buh.cameradicommercio.ro> References: <20040207155819.186335d4@it.buh.cameradicommercio.ro> Mime-Version: 1.0 (Apple Message framework v612) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <4F987C5C-5BF7-11D8-A867-00039354009A@apple.com> Content-Transfer-Encoding: 7bit From: Mark Day Date: Tue, 10 Feb 2004 10:31:15 -0800 To: Ion-Mihai Tetcu X-Mailer: Apple Mail (2.612) cc: freebsd-fs@freebsd.org Subject: Re: Painful speed for copying to fat32 partitions with small cluster size. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Feb 2004 18:33:20 -0000 On Feb 7, 2004, at 5:58 AM, Ion-Mihai Tetcu wrote: > Copying to FAT32 partitions is very slow with small cluster sizes (I > was > copying some 300 - 700MB files): (My FAT experience is mostly from Darwin/Mac OS X, which was ported from FreeBSD a few years ago. I haven't looked at the FreeBSD sources recently to know how relevant these remarks are for FreeBSD). First of all, smaller clusters sizes means larger FAT tables (more entries per table). This means you end up doing more I/O for a given file size. The Darwin code as originally ported from FreeBSD was using buffer cache blocks sized the same as a cluster (that is, one cluster per buffer cache block). But the real problem was that the VOP_BMAP (and VOP_CMAP in Darwin) implementation was very naive and always claimed that only there were no additional blocks of the file contiguous with the one being mapped. So, read and write ended up doing lots of tiny maps and I/Os. When this was fixed to scan the FAT for contiguous clusters, the buffer cache code (what Darwin calls Cluster I/O) was able to send much larger I/O requests to the driver. This made a huge difference, and brought streaming I/O performance into line with HFS Plus and UFS. -Mark From owner-freebsd-fs@FreeBSD.ORG Tue Feb 10 15:41:15 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F22E016A4CE; Tue, 10 Feb 2004 15:41:14 -0800 (PST) Received: from genius.tao.org.uk (genius.tao.org.uk [212.135.162.51]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2F03643D1D; Tue, 10 Feb 2004 15:41:14 -0800 (PST) (envelope-from joe@genius.tao.org.uk) Received: by genius.tao.org.uk (Postfix, from userid 100) id C261542A3; Tue, 10 Feb 2004 23:41:10 +0000 (GMT) Date: Tue, 10 Feb 2004 23:41:10 +0000 From: Josef Karthauser To: freebsd-fs@FreeBSD.org, current@FreeBSD.org Message-ID: <20040210234110.GD27830@genius.tao.org.uk> Mail-Followup-To: Josef Karthauser , freebsd-fs@FreeBSD.org, current@FreeBSD.org References: <20031021120918.GC15345@genius.tao.org.uk> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="IpbVkmxF4tDyP/Kb" Content-Disposition: inline In-Reply-To: <20031021120918.GC15345@genius.tao.org.uk> User-Agent: Mutt/1.5.5.1i Subject: Re: Problems with NFS (client) under 5.1. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Feb 2004 23:41:15 -0000 --IpbVkmxF4tDyP/Kb Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Oct 21, 2003 at 01:09:18PM +0100, Josef Karthauser wrote: > I'm trying to set a FreeBSD 5.1 machine up as an NFS client. The > server is on an SGI box. Things are strange: >=20 > phoenix# uname -a > FreeBSD phoenix.mydomain 5.1-CURRENT FreeBSD 5.1-CURRENT #0: Thu Sep = 18 15:20:19 GMT 2003 root@pheonix.mydomain:/usr/obj/usr/src/sys/GENERIC i3= 86 >=20 > phoenix# ls -ld /mnt > drwxr-xr-x 2 root wheel 512 Jun 5 01:53 /mnt >=20 > phoenix# mount rebus:/rebus/home /mnt > phoenix# ls -ld /mnt > ls: /mnt: Permission denied > phoenix# ls -ld /* | grep mnt >=20 > phoenix# umount /mnt > phoenix# ls -ld /* | grep mnt > drwxr-xr-x 2 root wheel 512 Jun 5 01:53 /mnt >=20 > What's going on here? Is it a bug or something that I'm doing wrong? >=20 > phoenix# grep nfs /etc/rc.conf > nfs_client_enable=3D"YES" # This host is an NFS client (or NO= ). >=20 > The NFS server is: > IRIX64 rebus 6.5 04101930 IP35 mips For the record, I upgraded this box to FreeBSD 5.2 over the last few days and tried again, and to my surprise NFS worked! I don't know what's changed since then, but thanks :) Joe --=20 Josef Karthauser (joe@tao.org.uk) http://www.josef-k.net/ FreeBSD (cvs meister, admin and hacker) http://www.uk.FreeBSD.org/ Physics Particle Theory (student) http://www.pact.cpes.sussex.ac.uk/ =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D An eclectic mix of fact an= d theory. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --IpbVkmxF4tDyP/Kb Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (FreeBSD) iEYEARECAAYFAkApbBYACgkQXVIcjOaxUBZQoQCg2x2YJAo5KJ7ZsSi8f61aG4Im 45cAn01mMC5dL8jA7EO3eUDH7s/p/A/q =gY0N -----END PGP SIGNATURE----- --IpbVkmxF4tDyP/Kb-- From owner-freebsd-fs@FreeBSD.ORG Wed Feb 11 10:36:51 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AEDE516A4CE for ; Wed, 11 Feb 2004 10:36:51 -0800 (PST) Received: from comp.chem.msu.su (comp.chem.msu.su [158.250.32.97]) by mx1.FreeBSD.org (Postfix) with ESMTP id C7F1A43D1D for ; Wed, 11 Feb 2004 10:36:50 -0800 (PST) (envelope-from yar@comp.chem.msu.su) Received: from comp.chem.msu.su (localhost [127.0.0.1]) by comp.chem.msu.su (8.12.9p2/8.12.9) with ESMTP id i1BIam0V019601; Wed, 11 Feb 2004 21:36:48 +0300 (MSK) (envelope-from yar@comp.chem.msu.su) Received: (from yar@localhost) by comp.chem.msu.su (8.12.9p2/8.12.9/Submit) id i1BIakmT019597; Wed, 11 Feb 2004 21:36:46 +0300 (MSK) (envelope-from yar) Date: Wed, 11 Feb 2004 21:36:45 +0300 From: Yar Tikhiy To: Murata Shuuichirou Message-ID: <20040211183645.GA18497@comp.chem.msu.su> References: <1074080151.733.51.camel@cyclops.thehouse> <87ad41z6ru.fsf@fons-adae.s.notwork.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87ad41z6ru.fsf@fons-adae.s.notwork.org> User-Agent: Mutt/1.5.3i cc: Zack Hobson cc: fs@freebsd.org Subject: Re: updating HFS for 5.2R [patch] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Feb 2004 18:36:51 -0000 On Tue, Feb 03, 2004 at 03:16:21AM +0900, Murata Shuuichirou wrote: > > > With these changes, the code compiles. I can install and load the > > resulting kernel module, and I can sucessfully use newfs_hfs and > > fsck_hfs, but mount_hfs on the same volume always fails with an > > "Input/output error". > > If you have not gotten good results yet, try attached patch. Of > course, your patch is also needed. > > With this patch, I can mount hfs successfully. Creating and > removing files on the filesystem are also succeeded. Excellent work, gentlemen! While currently I have very little time left for hacking, you helped me to bring the HFS port back into functional state. Thank you! > But, I have not tested this fully and found some problems such as: > > 1. Sometime, hfs partitions become unmountable by FreeBSD > (mount_hfs returns "Invalid argument"), although the > partition can still be mounted by MacOSX. Did you try to fsck_hfs such a broken volume? > 2. After editing files on hfs filesystem with vi(1), umounting > the filesystem causes these errors: > > Feb 2 21:13:11 roma kernel: hfs_fsync: dirty: 0xc2d74000: tag hfs, type VREG, usecount 2, writecount 0, refcount 2, flags (VV_SYSTEM), lock type cnode: EXCL (count 1) by thread 0xc2b42a80 (pid 1068) > Feb 2 21:13:11 roma kernel: tag VT_HFS, cnid 4, on dev 4, 24 lock type cnode: EXCL (count 1) by thread 0xc2b42a80 (pid 1068) > (lots of same errors continue) > > Then system crashed. It's me who introduced this bug. I forgot to unlock a buffer at one place. Please try the patch attached. -- Yar --- hfs_vnops.c 20 Jan 2004 17:48:16 -0000 1.52.2.1 +++ hfs_vnops.c 11 Feb 2004 18:23:34 -0000 @@ -1295,6 +1295,7 @@ hfs_fsync(ap) /* * Flush all dirty buffers associated with a vnode. */ +#ifdef DARWIN loop: s = splbio(); VI_LOCK(vp); @@ -1381,6 +1382,9 @@ loop: } VI_UNLOCK(vp); splx(s); +#else /* !DARWIN */ + vop_stdfsync(ap); +#endif /* DARWIN */ metasync: getmicrotime(&tv); @@ -1449,6 +1453,7 @@ hfs_metasync(struct hfsmount *hfsmp, dad (void) VOP_BWRITE(bp); goto exit; } + BUF_UNLOCK(bp); VI_LOCK(vp); } VI_UNLOCK(vp); From owner-freebsd-fs@FreeBSD.ORG Thu Feb 12 15:56:13 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4F01F16A4CE for ; Thu, 12 Feb 2004 15:56:13 -0800 (PST) Received: from beelzebubba.sysabend.org (alcatraz.inna.net [209.201.74.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 10D6543D1F for ; Thu, 12 Feb 2004 15:56:13 -0800 (PST) (envelope-from xyzzy@moo.sysabend.org) Received: from moo.sysabend.org (moo.sysabend.org [66.111.41.70]) by beelzebubba.sysabend.org (Postfix) with SMTP id 4932814E9F for ; Thu, 12 Feb 2004 18:56:24 -0500 (EST) Received: (nullmailer pid 77568 invoked by uid 14); Thu, 12 Feb 2004 23:56:05 -0000 Date: Thu, 12 Feb 2004 15:56:05 -0800 From: Tom Arnold To: freebsd-fs@freebsd.org Message-ID: <20040212235605.GP13780@moo.sysabend.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4i Organization: The Sysabend Dump X-Operating-System: CPM2.2 X-Bucket-Brigade-Devices: Rah! X-8-Bit-Samples-And-Analog-Filters: Rah! Subject: Upcoming filesystems? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: xyzzy@sysabend.org List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Feb 2004 23:56:13 -0000 Are there any Journaling FS projects ongoing for FreeBSD that arn't on the general radar? UFS2/Softupdates isnt working out for us so unfortunatly we are getting a lot of push to start using the L word with ReiserFS&Friends. -- ------------------------------------------------------------------------ - Tom Arnold - When I was small, I was in love, - - Sysabend - In love with everything. - - CareTaker - And now there's only you... - -------------- -- Thomas Dolby, "Cloudburst At Shingle Street" - From owner-freebsd-fs@FreeBSD.ORG Thu Feb 12 18:20:36 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 63FD316A4CE for ; Thu, 12 Feb 2004 18:20:36 -0800 (PST) Received: from ftp.bjpu.edu.cn (ftp.bjpu.edu.cn [202.112.78.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2690F43D1F for ; Thu, 12 Feb 2004 18:20:36 -0800 (PST) (envelope-from delphij@frontfree.net) Received: from localhost (localhost [127.0.0.1]) by ftp.bjpu.edu.cn (Postfix) with ESMTP id E1048535A for ; Fri, 13 Feb 2004 10:20:33 +0800 (CST) Received: from ftp.bjpu.edu.cn ([127.0.0.1]) by localhost (ftp.bjpu.edu.cn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 76747-04 for ; Fri, 13 Feb 2004 10:20:33 +0800 (CST) Received: from beastie.frontfree.net (beastie.frontfree.net [218.107.145.7]) by ftp.bjpu.edu.cn (Postfix) with ESMTP id C952252BD for ; Fri, 13 Feb 2004 10:20:32 +0800 (CST) Received: from localhost (localhost [127.0.0.1]) by beastie.frontfree.net (Postfix) with ESMTP id 0FC6D1177E; Fri, 13 Feb 2004 10:20:32 +0800 (CST) Received: from beastie.frontfree.net ([127.0.0.1]) by localhost (beastie.frontfree.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 02478-07; Fri, 13 Feb 2004 10:20:11 +0800 (CST) Received: by beastie.frontfree.net (Postfix, from userid 1001) id 5B77011585; Fri, 13 Feb 2004 10:20:11 +0800 (CST) Date: Fri, 13 Feb 2004 10:20:10 +0800 From: Xin LI To: Tom Arnold Message-ID: <20040213022010.GA2331@frontfree.net> References: <20040212235605.GP13780@moo.sysabend.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="4Ckj6UjgE2iN1+kY" Content-Disposition: inline In-Reply-To: <20040212235605.GP13780@moo.sysabend.org> User-Agent: Mutt/1.4.1i X-GPG-key-ID/Fingerprint: 0xCAEEB8C0 / 43B8 B703 B8DD 0231 B333 DC28 39FB 93A0 CAEE B8C0 X-GPG-Public-Key: http://www.delphij.net/delphij.asc X-Operating-System: FreeBSD beastie.frontfree.net 5.2-CURRENT FreeBSD 5.2-CURRENT #22: Sun Feb 8 05:09:45 CST 2004 delphij@beastie.frontfree.net:/usr/obj/usr/src/sys/BEASTIE i386 X-URL: http://www.delphij.net X-By: delphij@beastie.frontfree.net X-Location: Beijing, China X-Virus-Scanned: by amavisd-new at frontfree.net X-Virus-Scanned: by amavisd-new at frontfree.net cc: freebsd-fs@freebsd.org Subject: Re: Upcoming filesystems? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Feb 2004 02:20:36 -0000 --4Ckj6UjgE2iN1+kY Content-Type: text/plain; charset=us-ascii Content-Disposition: inline There was a journalling fs in FreeBSD in the past, however, it was removed (after it was imported from 4.4BSD-Lite2's distribution) because there are few people who want it. Not so sure whether there's an ongoing projects to have a journaling FS in FreeBSD, I guess not, however, I have heard that there're many efforts on other FS's, for example, tjr's "tfs" (an extent-filesystem) etc. You can find them in p4. I think the only issue with ReiserFS to prevent it from getting its way into FreeBSD's kernel source code might be the licence. Implementing it from scratch is possible, but it really needs a lot of work. Personally I'd prefer the SoftUpdates approach, so I am more interested in what made you to think it doesn't fit your needs, performance? security features? or others? Best Regards, -- Xin LI http://www.delphij.net/ See complete headers for GPG key and other information. --4Ckj6UjgE2iN1+kY Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (FreeBSD) iD8DBQFALDRaOfuToMruuMARAunhAJ4n6DDkhXIS93BW8CzJazxIOYIUYgCfaVQO jwzieeLzwKDNVXl9KjIw84o= =EsIR -----END PGP SIGNATURE----- --4Ckj6UjgE2iN1+kY-- From owner-freebsd-fs@FreeBSD.ORG Thu Feb 12 18:48:39 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DEE7B16A4CE for ; Thu, 12 Feb 2004 18:48:39 -0800 (PST) Received: from beelzebubba.sysabend.org (alcatraz.inna.net [209.201.74.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id A66E243D1F for ; Thu, 12 Feb 2004 18:48:39 -0800 (PST) (envelope-from xyzzy@moo.sysabend.org) Received: from moo.sysabend.org (moo.sysabend.org [66.111.41.70]) by beelzebubba.sysabend.org (Postfix) with SMTP id 1C9F014E9F; Thu, 12 Feb 2004 21:48:54 -0500 (EST) Received: (nullmailer pid 83742 invoked by uid 14); Fri, 13 Feb 2004 02:48:38 -0000 Date: Thu, 12 Feb 2004 18:48:38 -0800 From: Tom Arnold To: Xin LI Message-ID: <20040213024837.GQ13780@moo.sysabend.org> References: <20040212235605.GP13780@moo.sysabend.org> <20040213022010.GA2331@frontfree.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040213022010.GA2331@frontfree.net> User-Agent: Mutt/1.4i Organization: The Sysabend Dump X-Operating-System: CPM2.2 X-Bucket-Brigade-Devices: Rah! X-8-Bit-Samples-And-Analog-Filters: Rah! cc: freebsd-fs@freebsd.org Subject: Re: Upcoming filesystems? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: xyzzy@sysabend.org List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Feb 2004 02:48:40 -0000 On Fri, Feb 13, 2004 at 10:20:10AM +0800, Xin LI wrote: > Personally I'd prefer the SoftUpdates approach, so I am more interested > in what made you to think it doesn't fit your needs, performance? > security features? or others? Boot time. Large small-block filesystems take forever to come up at bootime Forever being approx 10 minutes per filesystem for the snapshot to occur, with 3 filesystems on the machine it adds up. If you go really insane on your block size on a large enough disk, in my case 1TB disk, 4k block size, 2billion inodes, the snapshot will just error out saying it doesnt have enough space. This isnt as much of an issue as we've determined the average filesize to be markedly over 512 bytes and it seems to handle 8k/1k just fine, but I can see this sneaking up on someone as drives get larger and larger... Minor issue, there are times when you cannot remove a file or subdirectory while the bgfsck is running. Important to us because of how the space is being used. Has to be large numbers of small blocks/fragments because its an extreme numbers of small files. I may not agree with how its being done, but I have to deploy a machine to handle it. I can expect hundreds of millions of 1k files on the each 1TB slice of a 3TB array. The boot-time is whats really killing us. Performance-wise, I've found softupdates to easily outperform ReiserFS. -- ------------------------------------------------------------------------ - Tom Arnold - When I was small, I was in love, - - Sysabend - In love with everything. - - CareTaker - And now there's only you... - -------------- -- Thomas Dolby, "Cloudburst At Shingle Street" - From owner-freebsd-fs@FreeBSD.ORG Thu Feb 12 19:09:41 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4D7AF16A4CE for ; Thu, 12 Feb 2004 19:09:41 -0800 (PST) Received: from VARK.homeunix.com (adsl-68-122-2-18.dsl.pltn13.pacbell.net [68.122.2.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2CC1143D1F for ; Thu, 12 Feb 2004 19:09:41 -0800 (PST) (envelope-from das@FreeBSD.ORG) Received: from VARK.homeunix.com (localhost [127.0.0.1]) by VARK.homeunix.com (8.12.10/8.12.10) with ESMTP id i1D39QOa037284; Thu, 12 Feb 2004 19:09:26 -0800 (PST) (envelope-from das@FreeBSD.ORG) Received: (from das@localhost) by VARK.homeunix.com (8.12.10/8.12.10/Submit) id i1D39Qjr037283; Thu, 12 Feb 2004 19:09:26 -0800 (PST) (envelope-from das@FreeBSD.ORG) Date: Thu, 12 Feb 2004 19:09:26 -0800 From: David Schultz To: Tom Arnold Message-ID: <20040213030926.GA37208@VARK.homeunix.com> Mail-Followup-To: Tom Arnold , Xin LI , freebsd-fs@FreeBSD.ORG References: <20040212235605.GP13780@moo.sysabend.org> <20040213022010.GA2331@frontfree.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040213022010.GA2331@frontfree.net> cc: freebsd-fs@FreeBSD.ORG Subject: Re: Upcoming filesystems? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Feb 2004 03:09:41 -0000 On Thu, Feb 12, 2004, Tom Arnold wrote: > Are there any Journaling FS projects ongoing for FreeBSD that arn't on the > general radar? I believe there is an effort underway to port SGI's XFS to FreeBSD. On Fri, Feb 13, 2004, Xin LI wrote: > There was a journalling fs in FreeBSD in the past, however, it was removed > (after it was imported from 4.4BSD-Lite2's distribution) because there > are few people who want it. You're thinking of LFS, which isn't exactly the same as journalling. In LFS, the journal *is* the filesystem. LFS does share the advantage of requiring no fsck, though. (It requires an incremental cleaner, but that runs continuously in the background.) Modern-day ``LFS-like'' filesystems include HP AutoRAID, Network Appliances' filesystem, and Sun's Zettabyte Filesystem. The latter two are not technically log-structured, but they perform block allocation and provide consistency in similar ways. NetBSD has an LFS implementation that is true to the original. From owner-freebsd-fs@FreeBSD.ORG Thu Feb 12 20:18:31 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E433C16A4CE for ; Thu, 12 Feb 2004 20:18:31 -0800 (PST) Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id 285FA43D1F for ; Thu, 12 Feb 2004 20:18:31 -0800 (PST) (envelope-from areilly@bigpond.net.au) Received: from areilly.bpc-users.org (ppp154.dyn230b.pacific.net.au [203.143.230.154]) by mailman.zeta.org.au (8.9.3p2/8.8.7) with SMTP id PAA19171 for ; Fri, 13 Feb 2004 15:18:28 +1100 Received: (qmail 12486 invoked by uid 1000); 13 Feb 2004 04:18:31 -0000 From: "Andrew Reilly" Date: Fri, 13 Feb 2004 15:18:31 +1100 To: Tom Arnold Message-ID: <20040213041831.GA12350@gurney.reilly.home> References: <20040212235605.GP13780@moo.sysabend.org> <20040213022010.GA2331@frontfree.net> <20040213024837.GQ13780@moo.sysabend.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040213024837.GQ13780@moo.sysabend.org> User-Agent: Mutt/1.4.1i cc: freebsd-fs@freebsd.org Subject: Re: Upcoming filesystems? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Feb 2004 04:18:32 -0000 On Thu, Feb 12, 2004 at 06:48:38PM -0800, Tom Arnold wrote: > On Fri, Feb 13, 2004 at 10:20:10AM +0800, Xin LI wrote: > > Personally I'd prefer the SoftUpdates approach, so I am more interested > > in what made you to think it doesn't fit your needs, performance? > > security features? or others? > > Boot time. Large small-block filesystems take forever to come > up at bootime Forever being approx 10 minutes per filesystem > for the snapshot to occur, with 3 filesystems on the machine > it adds up. Why are your filsystems going down dirty? Are you crashing the system, or is your power supply dodgy? If it's the latter, then a UPS ought to sort out the problem. If you shutdown cleanly, then it should come up without needing a fsck at all (background or otherwise). -- Andrew From owner-freebsd-fs@FreeBSD.ORG Fri Feb 13 01:16:35 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 590A016A4CE for ; Fri, 13 Feb 2004 01:16:35 -0800 (PST) Received: from mail.synology.com (dns1.synology.com [210.202.102.129]) by mx1.FreeBSD.org (Postfix) with ESMTP id 32DA043D1F for ; Fri, 13 Feb 2004 01:16:34 -0800 (PST) (envelope-from rjshang@synology.com) Received: from synology.com (localhost.synology.com [127.0.0.1]) by mail.synology.com (8.12.11/8.12.11) with ESMTP id i1D9GHFQ014682 for ; Fri, 13 Feb 2004 17:16:17 +0800 (CST) From: "rjshang" To: freebsd-fs@freebsd.org Date: Fri, 13 Feb 2004 17:16:17 +0800 Message-Id: <20040213091617.M29405@synology.com> X-Mailer: Open WebMail 1.81 20021127 X-OriginatingIP: 192.168.101.227 (rjshang) MIME-Version: 1.0 Content-Type: text/plain; charset=big5 Subject: Is that a typo in ffs_softdep.c X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Feb 2004 09:16:35 -0000 Hi, there, It looks like a typo in ffs_softdep.c softdep_sync_metadata(). The second parameter of drain_output is used to tell if it is protected by &lk. Is there other concern? Please check the following patch from 4.9 release. Thanks! R.J. Shang ************************************************************************* --- ffs_softdep.c Wed Feb 6 02:46:53 2002 +++ ffs_softdep.new.c Fri Feb 13 16:48:52 2004 @@ -4299,29 +4299,29 @@ return (0); } FREE_LOCK(&lk); /* * If we are trying to sync a block device, some of its buffers may * contain metadata that cannot be written until the contents of some * partially written files have been written to disk. The only easy * way to accomplish this is to sync the entire filesystem (luckily * this happens rarely). * * We must wait for any I/O in progress to finish so that * all potential buffers on the dirty list will be visible. */ - drain_output(vp, 1); + drain_output(vp, 0); if (vn_isdisk(vp, NULL) && vp->v_specmountpoint && !VOP_ISLOCKED(vp, NULL) && (error = VFS_SYNC(vp->v_specmountpoint, MNT_WAIT, ap->a_cred, ap->a_p)) != 0) return (error); return (0); } /* * Flush the dependencies associated with an inodedep. * Called with splbio blocked. */ static int flush_inodedep_deps(fs, ino) From owner-freebsd-fs@FreeBSD.ORG Fri Feb 13 03:05:51 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 09DDB16A4CE for ; Fri, 13 Feb 2004 03:05:51 -0800 (PST) Received: from mail-1.zoominternet.net (mail-1.zoominternet.net [63.67.120.5]) by mx1.FreeBSD.org (Postfix) with SMTP id A9DAA43D1F for ; Fri, 13 Feb 2004 03:05:50 -0800 (PST) (envelope-from dmmiller@cvzoom.net) Received: (qmail 8182 invoked from network); 13 Feb 2004 11:05:48 -0000 Received: from acs-24-239-218-57.zoominternet.net (HELO cvzoom.net) ([24.239.218.57]) (envelope-sender ) by mail-1.zoominternet.net (qmail-ldap-1.03) with SMTP for ; 13 Feb 2004 11:05:48 -0000 Message-ID: <402CAF83.2000201@cvzoom.net> Date: Fri, 13 Feb 2004 06:05:39 -0500 From: Donn Miller User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.6) Gecko/20040204 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Tom Arnold References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-fs@freebsd.org Subject: Re: Upcoming filesystems? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Feb 2004 11:05:51 -0000 Tom Arnold wrote: > Are there any Journaling FS projects ongoing for FreeBSD that arn't on the > general radar? > > UFS2/Softupdates isnt working out for us so unfortunatly we are getting a > lot of push to start using the L word with ReiserFS&Friends. Not really. There is an ongoing port of XFS. Check the status report. As for UFS+softupdates, softupdates obviates (according to theory, anyways) the need for journalling. But I was wondering myself if there was some way of using softupdates and journalling together. For example, there could be some way of logging what softupdates is doing exactly so that background fsck can have better hints of which part of the filesystem needs fscked, if any. For example, there could be some "hints" stored in a journal and log that softupdates allocated a data block, but no inode was written. So fsck would know to de-allocate that particular data block, and know exactly where it is from the hints stored in the log so fsck could finish faster. But this sounds like how the typical journalling filesystem works in the first place. I did take a look at some ReiserFS code. I proceeded to do some very quick preliminary research. The first things I noted are the differences of Linux's kmalloc() vs. FreeBSD's malloc(9). Linux's kmalloc() seems to be tied down to its slab allocator, and it incorporates the filesystem buffer cache whereas FreeBSD's malloc(9) doesn't. This would be the first obstacle in porting ReiserFS to FreeBSD, as ReiserFS does call kmalloc a fair bit, and in addition, malloc(9) takes 3 arguments compared to kmalloc's 2. I'll have to take another look at kmalloc, malloc(9), and the ReiserFS code, because I'm sure I'm way off the mark. :-) From owner-freebsd-fs@FreeBSD.ORG Fri Feb 13 18:01:04 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0222616A4D2 for ; Fri, 13 Feb 2004 18:01:04 -0800 (PST) Received: from ganymede.hub.org (u46n208.hfx.eastlink.ca [24.222.46.208]) by mx1.FreeBSD.org (Postfix) with ESMTP id CE94443D2F for ; Fri, 13 Feb 2004 18:01:03 -0800 (PST) (envelope-from scrappy@hub.org) Received: by ganymede.hub.org (Postfix, from userid 1000) id 06BE0354CF; Fri, 13 Feb 2004 21:56:46 -0400 (AST) Received: from localhost (localhost [127.0.0.1]) by ganymede.hub.org (Postfix) with ESMTP id 055FD35383; Fri, 13 Feb 2004 21:56:46 -0400 (AST) Date: Fri, 13 Feb 2004 21:56:45 -0400 (AST) From: "Marc G. Fournier" To: Andrew Reilly In-Reply-To: <20040213041831.GA12350@gurney.reilly.home> Message-ID: <20040213215212.U96890@ganymede.hub.org> References: <20040212235605.GP13780@moo.sysabend.org> <20040213022010.GA2331@frontfree.net> <20040213041831.GA12350@gurney.reilly.home> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-fs@freebsd.org Subject: Re: Upcoming filesystems? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Feb 2004 02:01:04 -0000 On Fri, 13 Feb 2004, Andrew Reilly wrote: > On Thu, Feb 12, 2004 at 06:48:38PM -0800, Tom Arnold wrote: > > On Fri, Feb 13, 2004 at 10:20:10AM +0800, Xin LI wrote: > > > Personally I'd prefer the SoftUpdates approach, so I am more interested > > > in what made you to think it doesn't fit your needs, performance? > > > security features? or others? > > > > Boot time. Large small-block filesystems take forever to come > > up at bootime Forever being approx 10 minutes per filesystem > > for the snapshot to occur, with 3 filesystems on the machine > > it adds up. > > Why are your filsystems going down dirty? Are you crashing the > system, or is your power supply dodgy? If it's the latter, then > a UPS ought to sort out the problem. I have problems with 4.9-STABLE where doing a 'reboot' with all file systems mounted will result in it hanging while syncing disks ... the last time I did it, I umounted *everything* except those reported as busy, so that all I had left mounted was: Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/da0s1a 516062 191016 283762 40% / /dev/da0s1e 1032142 19454 930118 2% /tmp /dev/da0s1f 10322414 5805016 3691606 61% /usr /dev/da0s1g 1032142 146584 802988 15% /var /dev/da0s1h 119837208 90385742 19864490 82% /vm procfs 4 4 0 100% /proc and it hung, requiring a cold boot ... the higher the uptime, the better chance it happens ... I've gotten into the habit of umounting everything I can, and shutting down all the processes, before rebooting to avoid the issue ... the servers are remote, so a hang tends to mean waiting for someone to get to that machine ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664