From owner-freebsd-fs@FreeBSD.ORG Sun Nov 8 11:20:09 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 95CC91065693 for ; Sun, 8 Nov 2009 11:20:09 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 6B6028FC1E for ; Sun, 8 Nov 2009 11:20:09 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id nA8BK9YM010754 for ; Sun, 8 Nov 2009 11:20:09 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id nA8BK9qf010747; Sun, 8 Nov 2009 11:20:09 GMT (envelope-from gnats) Date: Sun, 8 Nov 2009 11:20:09 GMT Message-Id: <200911081120.nA8BK9qf010747@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: Dominic Fandrey Cc: Subject: Re: kern/140134: [msdosfs] write and fsck destroy filesystem integrity X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Dominic Fandrey List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Nov 2009 11:20:09 -0000 The following reply was made to PR kern/140134; it has been noted by GNATS. From: Dominic Fandrey To: bug-followup@FreeBSD.org, kamikaze@bsdforen.de Cc: Subject: Re: kern/140134: [msdosfs] write and fsck destroy filesystem integrity Date: Sun, 08 Nov 2009 12:18:53 +0100 I just rebuilt world and kernel with RELENG_8 from ~3 hours ago. Afterwards I repeated the How-To-Repeat procedure and the problems shown therein didn't appear. In the next step I put my Cowon S9 (audio/video player with 16GB fat32 partition) on the line. I mounted the device and created a file with touch. fsck_msdosfs said the drive was fine. Windows chkdsk reported the drive was fine. So the next attempt was to remove the empty file and copy a JPG ~2.5MB onto the drive. fsck_msdosfs reported the following: > fsck_msdosfs -n /dev/msdosfs/VALHALLA ** /dev/msdosfs/VALHALLA ** Phase 1 - Read and Compare FATs ** Phase 2 - Check Cluster Chains ** Phase 3 - Checking Directories ** Phase 4 - Checking for Lost Files Lost cluster chain at cluster 4 49 Cluster(s) lost Reconnect? no Clear? no Lost cluster chain at cluster 53 37 Cluster(s) lost Reconnect? no Clear? no Lost cluster chain at cluster 90 9 Cluster(s) lost Reconnect? no Clear? no Lost cluster chain at cluster 99 9 Cluster(s) lost Reconnect? no Clear? no Lost cluster chain at cluster 108 1126 Cluster(s) lost Reconnect? no Clear? no Lost cluster chain at cluster 1234 9 Cluster(s) lost Reconnect? no Clear? no Lost cluster chain at cluster 1244 5 Cluster(s) lost Reconnect? no Clear? no Lost cluster chain at cluster 1248 49 Cluster(s) lost Reconnect? no Clear? no Lost cluster chain at cluster 1302 282 Cluster(s) lost Reconnect? no Clear? no Lost cluster chain at cluster 1600 5 Cluster(s) lost Reconnect? no Clear? no Lost cluster chain at cluster 1605 59 Cluster(s) lost Reconnect? no Clear? no Lost cluster chain at cluster 449483 1 Cluster(s) lost Reconnect? no Clear? no Free space in FSInfo block (-1) not correct (577592) Fix? no Next free cluster in FSInfo block (2) not free Fix? no 2130 files, 426432 free (577592 clusters) Kinda panicked I ran Windows chkdsk and it reported the drive was just fine fine. fsck_msdosfs still reports the same errors. I conclude that writing to msdosfs seems to be fine, now, however fsck_msdosfs is a dangerous threat to fat32 file systems. From owner-freebsd-fs@FreeBSD.ORG Mon Nov 9 09:13:01 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 885E9106566B for ; Mon, 9 Nov 2009 09:13:01 +0000 (UTC) (envelope-from gerrit@pmp.uni-hannover.de) Received: from mrelay1.uni-hannover.de (mrelay1.uni-hannover.de [130.75.2.106]) by mx1.freebsd.org (Postfix) with ESMTP id 134E08FC0A for ; Mon, 9 Nov 2009 09:13:00 +0000 (UTC) Received: from www.pmp.uni-hannover.de (www.pmp.uni-hannover.de [130.75.117.2]) by mrelay1.uni-hannover.de (8.14.2/8.14.2) with ESMTP id nA99CtQm029150; Mon, 9 Nov 2009 10:12:56 +0100 Received: from pmp.uni-hannover.de (arc.pmp.uni-hannover.de [130.75.117.1]) by www.pmp.uni-hannover.de (Postfix) with SMTP id 82EB424; Mon, 9 Nov 2009 10:12:55 +0100 (CET) Date: Mon, 9 Nov 2009 10:12:55 +0100 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: "James R. Van Artsdalen" Message-Id: <20091109101255.e81774e4.gerrit@pmp.uni-hannover.de> In-Reply-To: <4AF4AAFF.2080104@jrv.org> References: <20091106094734.4b056899.gerrit@pmp.uni-hannover.de> <4AF4123A.4080301@andric.com> <20091106231440.4f0f2cbb.gerrit@pmp.uni-hannover.de> <4AF4AAFF.2080104@jrv.org> Organization: Albert-Einstein-Institut (MPI =?ISO-8859-1?Q?f=FCr?= Gravitationsphysik & IGP =?ISO-8859-1?Q?Universit=E4t?= Hannover) X-Mailer: Sylpheed 2.7.1 (GTK+ 2.12.11; i386-portbld-freebsd7.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-PMX-Version: 5.5.5.374460, Antispam-Engine: 2.7.1.369594, Antispam-Data: 2009.11.9.90052 Cc: freebsd-fs@freebsd.org Subject: Re: trace for zfs panic mounting fs after crash with RC2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Nov 2009 09:13:01 -0000 On Fri, 06 Nov 2009 17:02:23 -0600 "James R. Van Artsdalen" wrote about Re: trace for zfs panic mounting fs after crash with RC2: JRVA> How the ZIL got corrupted - if it did - is a harder question. I think it is. Otherwise zfs would not crash while trying to replay the ZIL, wouldn't it? It seems that this happens rather easily with the system I have at hand (it happend twice to me so far - and I crashed the system only twice, that makes 100%, although I doubt that it is that reproducible). Searching around I found some reports of the same or similar issues (but no solution). So apart from recovering my fs (I did not try your suggested patch yet), there are two things I regard as very important: 1. Find you why the ZIL gets corrupted under some circumstances. 2. Find a safe way to recover a fs with a corrupted ZIL. I guess I could live with a corrupted ZIL after a crash, if there was some kind of --ignore-zil switch to get my data back online. In any case, zfs should not panic on corrupted ZIL data, should it? As I do not dare to use the system for storing data until this is sorted out, I can try out almost anything to get more information about the problem. Please let me know what I should do to support debugging. cu Gerrit From owner-freebsd-fs@FreeBSD.ORG Mon Nov 9 11:06:51 2009 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A68291065692 for ; Mon, 9 Nov 2009 11:06:51 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 7AF868FC2B for ; Mon, 9 Nov 2009 11:06:51 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id nA9B6pFl078978 for ; Mon, 9 Nov 2009 11:06:51 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id nA9B6oeh078976 for freebsd-fs@FreeBSD.org; Mon, 9 Nov 2009 11:06:50 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 9 Nov 2009 11:06:50 GMT Message-Id: <200911091106.nA9B6oeh078976@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Nov 2009 11:06:51 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/140134 fs [msdosfs] write and fsck destroy filesystem integrity o kern/140068 fs [smbfs] [patch] smbfs does not allow semicolon in file o kern/139725 fs [zfs] zdb(1) dumps core on i386 when examining zpool c o kern/139715 fs [zfs] vfs.numvnodes leak on busy zfs o bin/139651 fs [nfs] mount(8): read-only remount of NFS volume does n o kern/139597 fs [patch] [tmpfs] tmpfs initializes va_gen but doesn't u o kern/139564 fs [zfs] [panic] 8.0-RC1 - Fatal trap 12 at end of shutdo o kern/139407 fs [smbfs] [panic] smb mount causes system crash if remot o kern/139363 fs [nfs] diskless root nfs mount from non FreeBSD server o kern/138790 fs [zfs] ZFS ceases caching when mem demand is high o kern/138524 fs [msdosfs] disks and usb flashes/cards with Russian lab o kern/138421 fs [ufs] [patch] remove UFS label limitations o kern/138367 fs [tmpfs] [panic] 'panic: Assertion pages > 0 failed' wh o kern/138202 fs mount_msdosfs(1) see only 2Gb o kern/138109 fs [extfs] [patch] Minor cleanups to the sys/gnu/fs/ext2f f kern/137037 fs [zfs] [hang] zfs rollback on root causes FreeBSD to fr o kern/136968 fs [ufs] [lor] ufs/bufwait/ufs (open) o kern/136945 fs [ufs] [lor] filedesc structure/ufs (poll) o kern/136944 fs [ffs] [lor] bufwait/snaplk (fsync) o kern/136873 fs [ntfs] Missing directories/files on NTFS volume o kern/136865 fs [nfs] [patch] NFS exports atomic and on-the-fly atomic o kern/136470 fs [nfs] Cannot mount / in read-only, over NFS o kern/135594 fs [zfs] Single dataset unresponsive with Samba o kern/135546 fs [zfs] zfs.ko module doesn't ignore zpool.cache filenam o kern/135469 fs [ufs] [panic] kernel crash on md operation in ufs_dirb o kern/135050 fs [zfs] ZFS clears/hides disk errors on reboot o kern/134491 fs [zfs] Hot spares are rather cold... o kern/133980 fs [panic] [ffs] panic: ffs_valloc: dup alloc o kern/133676 fs [smbfs] [panic] umount -f'ing a vnode-based memory dis o kern/133614 fs [panic] panic: ffs_truncate: read-only filesystem o kern/133174 fs [msdosfs] [patch] msdosfs must support utf-encoded int f kern/133150 fs [zfs] Page fault with ZFS on 7.1-RELEASE/amd64 while w o kern/132960 fs [ufs] [panic] panic:ffs_blkfree: freeing free frag o kern/132597 fs [tmpfs] [panic] tmpfs-related panic while interrupting o kern/132397 fs reboot causes filesystem corruption (failure to sync b o kern/132331 fs [ufs] [lor] LOR ufs and syncer o kern/132237 fs [msdosfs] msdosfs has problems to read MSDOS Floppy o kern/132145 fs [panic] File System Hard Crashes o kern/131995 fs [nfs] Failure to mount NFSv4 server o kern/131441 fs [unionfs] [nullfs] unionfs and/or nullfs not combineab o kern/131360 fs [nfs] poor scaling behavior of the NFS server under lo o kern/131342 fs [nfs] mounting/unmounting of disks causes NFS to fail o bin/131341 fs makefs: error "Bad file descriptor" on the mount poin o kern/130979 fs [smbfs] [panic] boot/kernel/smbfs.ko o kern/130920 fs [msdosfs] cp(1) takes 100% CPU time while copying file o kern/130229 fs [iconv] usermount fails on fs that need iconv o kern/130210 fs [nullfs] Error by check nullfs o kern/129760 fs [nfs] after 'umount -f' of a stale NFS share FreeBSD l o kern/129488 fs [smbfs] Kernel "bug" when using smbfs in smbfs_smb.c: o kern/129231 fs [ufs] [patch] New UFS mount (norandom) option - mostly o kern/129152 fs [panic] non-userfriendly panic when trying to mount(8) o kern/129059 fs [zfs] [patch] ZFS bootloader whitelistable via WITHOUT f kern/128829 fs smbd(8) causes periodic panic on 7-RELEASE o kern/127659 fs [tmpfs] tmpfs memory leak o kern/127420 fs [gjournal] [panic] Journal overflow on gmirrored gjour o kern/127029 fs [panic] mount(8): trying to mount a write protected zi o kern/126287 fs [ufs] [panic] Kernel panics while mounting an UFS file s kern/125738 fs [zfs] [request] SHA256 acceleration in ZFS p kern/124621 fs [ext3] [patch] Cannot mount ext2fs partition f bin/124424 fs [zfs] zfs(8): zfs list -r shows strange snapshots' siz o kern/123939 fs [msdosfs] corrupts new files o kern/122380 fs [ffs] ffs_valloc:dup alloc (Soekris 4801/7.0/USB Flash o bin/122172 fs [fs]: amd(8) automount daemon dies on 6.3-STABLE i386, o bin/121898 fs [nullfs] pwd(1)/getcwd(2) fails with Permission denied o bin/121779 fs [ufs] snapinfo(8) (and related tools?) only work for t o bin/121366 fs [zfs] [patch] Automatic disk scrubbing from periodic(8 o bin/121072 fs [smbfs] mount_smbfs(8) cannot normally convert the cha f kern/120991 fs [panic] [fs] [snapshot] System crashes when manipulati o kern/120483 fs [ntfs] [patch] NTFS filesystem locking changes o kern/120482 fs [ntfs] [patch] Sync style changes between NetBSD and F f kern/119735 fs [zfs] geli + ZFS + samba starting on boot panics 7.0-B o kern/118912 fs [2tb] disk sizing/geometry problem with large array o kern/118713 fs [minidump] [patch] Display media size required for a k o bin/118249 fs mv(1): moving a directory changes its mtime o kern/118107 fs [ntfs] [panic] Kernel panic when accessing a file at N o bin/117315 fs [smbfs] mount_smbfs(8) and related options can't mount o kern/117314 fs [ntfs] Long-filename only NTFS fs'es cause kernel pani o kern/117158 fs [zfs] zpool scrub causes panic if geli vdevs detach on o bin/116980 fs [msdosfs] [patch] mount_msdosfs(8) resets some flags f o kern/116913 fs [ffs] [panic] ffs_blkfree: freeing free block p kern/116608 fs [msdosfs] [patch] msdosfs fails to check mount options o kern/116583 fs [ffs] [hang] System freezes for short time when using o kern/116170 fs [panic] Kernel panic when mounting /tmp o kern/115645 fs [snapshots] [panic] lockmgr: thread 0xc4c00d80, not ex o bin/115361 fs [zfs] mount(8) gets into a state where it won't set/un o kern/114955 fs [cd9660] [patch] [request] support for mask,dirmask,ui o kern/114847 fs [ntfs] [patch] [request] dirmask support for NTFS ala o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o bin/114468 fs [patch] [request] add -d option to umount(8) to detach o kern/113852 fs [smbfs] smbfs does not properly implement DFS referral o bin/113838 fs [patch] [request] mount(8): add support for relative p o bin/113049 fs [patch] [request] make quot(8) use getopt(3) and show o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/111843 fs [msdosfs] Long Names of files are incorrectly created o kern/111782 fs [ufs] dump(8) fails horribly for large filesystems s bin/111146 fs [2tb] fsck(8) fails on 6T filesystem o kern/109024 fs [msdosfs] mount_msdosfs: msdosfs_iconv: Operation not o kern/109010 fs [msdosfs] can't mv directory within fat32 file system o bin/107829 fs [2TB] fdisk(8): invalid boundary checking in fdisk / w o kern/106030 fs [ufs] [panic] panic in ufs from geom when a dead disk o kern/104406 fs [ufs] Processes get stuck in "ufs" state under persist o kern/104133 fs [ext2fs] EXT2FS module corrupts EXT2/3 filesystems o kern/103035 fs [ntfs] Directories in NTFS mounted disc images appear o kern/101324 fs [smbfs] smbfs sometimes not case sensitive when it's s o kern/99290 fs [ntfs] mount_ntfs ignorant of cluster sizes o kern/97377 fs [ntfs] [patch] syntax cleanup for ntfs_ihash.c o kern/95222 fs [iso9660] File sections on ISO9660 level 3 CDs ignored o kern/94849 fs [ufs] rename on UFS filesystem is not atomic o kern/94769 fs [ufs] Multiple file deletions on multi-snapshotted fil o kern/94733 fs [smbfs] smbfs may cause double unlock o kern/93942 fs [vfs] [patch] panic: ufs_dirbad: bad dir (patch from D o kern/92272 fs [ffs] [hang] Filling a filesystem while creating a sna f kern/91568 fs [ufs] [panic] writing to UFS/softupdates DVD media in o kern/91134 fs [smbfs] [patch] Preserve access and modification time a kern/90815 fs [smbfs] [patch] SMBFS with character conversions somet o kern/89991 fs [ufs] softupdates with mount -ur causes fs UNREFS o kern/88657 fs [smbfs] windows client hang when browsing a samba shar o kern/88266 fs [smbfs] smbfs does not implement UIO_NOCOPY and sendfi o kern/87859 fs [smbfs] System reboot while umount smbfs. o kern/86587 fs [msdosfs] rm -r /PATH fails with lots of small files o kern/85326 fs [smbfs] [panic] saving a file via samba to an overquot o kern/84589 fs [2TB] 5.4-STABLE unresponsive during background fsck 2 o kern/80088 fs [smbfs] Incorrect file time setting on NTFS mounted vi o kern/73484 fs [ntfs] Kernel panic when doing `ls` from the client si o bin/73019 fs [ufs] fsck_ufs(8) cannot alloc 607016868 bytes for ino o kern/71774 fs [ntfs] NTFS cannot "see" files on a WinXP filesystem o kern/68978 fs [panic] [ufs] crashes with failing hard disk, loose po o kern/65920 fs [nwfs] Mounted Netware filesystem behaves strange o kern/65901 fs [smbfs] [patch] smbfs fails fsx write/truncate-down/tr o kern/61503 fs [smbfs] mount_smbfs does not work as non-root o kern/55617 fs [smbfs] Accessing an nsmb-mounted drive via a smb expo o kern/51685 fs [hang] Unbounded inode allocation causes kernel to loc o kern/51583 fs [nullfs] [patch] allow to work with devices and socket o kern/36566 fs [smbfs] System reboot with dead smb mount and umount o kern/18874 fs [2TB] 32bit NFS servers export wrong negative values t 135 problems total. From owner-freebsd-fs@FreeBSD.ORG Mon Nov 9 21:45:58 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C42E0106566C; Mon, 9 Nov 2009 21:45:58 +0000 (UTC) (envelope-from gavin@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 8CB198FC1B; Mon, 9 Nov 2009 21:45:58 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id nA9LjwM4037228; Mon, 9 Nov 2009 21:45:58 GMT (envelope-from gavin@freefall.freebsd.org) Received: (from gavin@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id nA9LjwAC037224; Mon, 9 Nov 2009 21:45:58 GMT (envelope-from gavin) Date: Mon, 9 Nov 2009 21:45:58 GMT Message-Id: <200911092145.nA9LjwAC037224@freefall.freebsd.org> To: gavin@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: gavin@FreeBSD.org Cc: Subject: Re: kern/140433: ZFS panics kernel while replaying ZIL after crash X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Nov 2009 21:45:58 -0000 Synopsis: ZFS panics kernel while replaying ZIL after crash Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: gavin Responsible-Changed-When: Mon Nov 9 21:30:36 UTC 2009 Responsible-Changed-Why: Over to maintainer(s). To submitter: can you also please give the output of "zdb -C"? http://www.freebsd.org/cgi/query-pr.cgi?pr=140433 From owner-freebsd-fs@FreeBSD.ORG Mon Nov 9 23:40:51 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CD95C106568D for ; Mon, 9 Nov 2009 23:40:51 +0000 (UTC) (envelope-from gleb.kurtsou@gmail.com) Received: from mail-bw0-f213.google.com (mail-bw0-f213.google.com [209.85.218.213]) by mx1.freebsd.org (Postfix) with ESMTP id 5C0028FC16 for ; Mon, 9 Nov 2009 23:40:51 +0000 (UTC) Received: by bwz5 with SMTP id 5so4207653bwz.3 for ; Mon, 09 Nov 2009 15:40:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:date:from:to:subject :message-id:mime-version:content-type:content-disposition:user-agent; bh=Fn1ZOznFGFUtA5nj/bvERDGHUytmWWyd8SnWFizbPY0=; b=emT0HRYT2K7x9NBHOVxYgA7JXM/aCtYduBPqkCd7Cfa8EHrMSDwXhoAWBqXza/0iXm Vfhter8t0mrcmYpxJXjTamr3KCH5MDNdsXwQVCa9haGUOnS8G3TbG8x9/QoByGMKCUAR ZfFWhk5Ig430ZviKXBiwWa0lWF4tVjar8RYqk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:to:subject:message-id:mime-version:content-type :content-disposition:user-agent; b=XEQnRMKkF96IPUao9bY784s4LLTb92Grs9TbbRB/XTnKHlWW+7MerYaMmAJr6Ihygb 1cJN9H7DQOwLr1cWAQuJKUdGIN0nD/ya4quKTe55CJdJXN/zNGa2q2BaB9s8zPvWOZJb QdzEcsvId34N8rf2pTn91IJVghU9nJPY4dO/U= Received: by 10.204.34.71 with SMTP id k7mr218211bkd.206.1257810050382; Mon, 09 Nov 2009 15:40:50 -0800 (PST) Received: from localhost (lan-78-157-90-54.vln.skynet.lt [78.157.90.54]) by mx.google.com with ESMTPS id 22sm255913fkq.51.2009.11.09.15.40.49 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 09 Nov 2009 15:40:49 -0800 (PST) Date: Tue, 10 Nov 2009 01:40:45 +0200 From: Gleb Kurtsou To: freebsd-fs@freebsd.org Message-ID: <20091109234045.GA3679@tops.skynet.lt> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline User-Agent: Mutt/1.5.20 (2009-06-14) Subject: Deleting files/dirs from partially damaged ZFS filesystem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Nov 2009 23:40:52 -0000 Hello, I have ZFS filesystem with some inconsistencies I'd like to fix. It's root filesystem on my laptop, so backup/restore everything is rather troublesome. ZFS scrub doesn't show/fix any errors. These inconsistencies occurred several months ago and I had no other issues with filesystem since then. These issues can be interesting for those working on ZFS stability. I wasn't able to find any useful information on ZFS debugging tools, so posting it here. 1. ~/.mozilla-bug2/firefox/5iyxnqmf.default-1 The name used to be different but I was able to rename it. Process accessing this directory stalls and can't be killed. System used to panic on accessing it before (NULL pointer dereference, have to stack trace any longer), but that was fixed just before 9-CURRENT. It seems that kernel buffer that contained structures related to this directory was changed with random data, and then written to disk (So that checksums are fine but data is incorrect) 2. /test Directory looks empty, but can't be deleted. / # ls -Al /test total 0 / # rm -rf /test rm: /test: Directory not empty Most likely because I've managed to create file/directory with name longer then MAXNAMLEN (255 bytes) in this directory. I was running tools/regression/fstest suite to test stacked filesystem that performed manipulations on file names. Files can be added/deleted in this directory. It seems both these issues can be fixed by marking appropriate device blocks bad, running scrub, and marking blocks good, or simply trashing appropriate checksums, so that scrub can fix them. But I can't find block offsets nor the way to change checksums. / # uname -a FreeBSD tops 9.0-CURRENT FreeBSD 9.0-CURRENT #16 r198029+762c399-dirty: Wed Oct 28 16:11:49 EET 2009 root@tops:/usr/obj/usr/freebsd-src/local/sys/TOPS amd64 / # zpool status pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 ada0s3d ONLINE 0 0 0 errors: No known data errors / # zfs list NAME USED AVAIL REFER MOUNTPOINT tank 116G 8,39G 18,2G legacy tank/home 87,5G 8,39G 62,7G /home tank/local 4,45G 8,39G 4,45G /usr/local tank/ports 5,48G 8,39G 5,48G /usr/ports / # zdb -C tank version=13 name='tank' state=0 txg=298580 pool_guid=12986731317200074631 hostid=1869410071 hostname='tops' vdev_tree type='root' id=0 guid=12986731317200074631 children[0] type='disk' id=0 guid=11828906155092156003 path='/dev/ad0s3d' whole_disk=0 metaslab_array=23 metaslab_shift=30 ashift=9 asize=135652442112 is_log=0 DTL=153 From owner-freebsd-fs@FreeBSD.ORG Tue Nov 10 15:02:21 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EE2C1106568F for ; Tue, 10 Nov 2009 15:02:21 +0000 (UTC) (envelope-from gtodd@bellanet.org) Received: from smtp122.rog.mail.re2.yahoo.com (smtp122.rog.mail.re2.yahoo.com [206.190.53.27]) by mx1.freebsd.org (Postfix) with SMTP id 91DDD8FC2D for ; Tue, 10 Nov 2009 15:02:21 +0000 (UTC) Received: (qmail 83227 invoked from network); 10 Nov 2009 15:02:20 -0000 Received: from CPE0080c8f208a5-CM001371173cf8.cpe.net.cable.rogers.com (gtodd@99.246.61.82 with login) by smtp122.rog.mail.re2.yahoo.com with SMTP; 10 Nov 2009 07:02:20 -0800 PST X-Yahoo-SMTP: nTCfdR6swBBGLC_L2D5qGu5iTT2NdwV8DpHvzb.tTA-- X-YMail-OSG: 2tU2yCIVM1lbQrkfdjg4NQQk6nEXXDT1OTI5qCxsorEsBWqt8ar8A_a0bbx2MWO_og-- X-Yahoo-Newman-Property: ymail-3 Received: from wawanesa.iciti.ca (wawanesa.iciti.ca [192.168.2.4]) by wawanesa.iciti.ca (Postfix) with ESMTP id BEB48100; Tue, 10 Nov 2009 10:01:06 -0500 (EST) Message-ID: <4AF98032.9050808@bellanet.org> Date: Tue, 10 Nov 2009 10:01:06 -0500 From: Graham Todd User-Agent: Thunderbird 2.0.0.21 (X11/20090511) MIME-Version: 1.0 To: =?ISO-8859-1?Q?Gerrit_K=FChn?= References: <20091106094734.4b056899.gerrit@pmp.uni-hannover.de> <4AF4123A.4080301@andric.com> <20091106231440.4f0f2cbb.gerrit@pmp.uni-hannover.de> <4AF4AAFF.2080104@jrv.org> <20091109101255.e81774e4.gerrit@pmp.uni-hannover.de> In-Reply-To: <20091109101255.e81774e4.gerrit@pmp.uni-hannover.de> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Cc: freebsd-fs@freebsd.org Subject: Re: trace for zfs panic mounting fs after crash with RC2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Nov 2009 15:02:22 -0000 Gerrit Kühn wrote: > On Fri, 06 Nov 2009 17:02:23 -0600 "James R. Van Artsdalen" > wrote about Re: trace for zfs panic mounting > fs after crash with RC2: > > JRVA> How the ZIL got corrupted - if it did - is a harder question. > > I think it is. Otherwise zfs would not crash while trying to replay the > ZIL, wouldn't it? > It seems that this happens rather easily with the system I have at hand > (it happend twice to me so far - and I crashed the system only twice, > that makes 100%, although I doubt that it is that reproducible). Searching > around I found some reports of the same or similar issues (but no > solution). So apart from recovering my fs (I did not try your suggested > patch yet), there are two things I regard as very important: > > 1. Find you why the ZIL gets corrupted under some circumstances. > 2. Find a safe way to recover a fs with a corrupted ZIL. > > I guess I could live with a corrupted ZIL after a crash, if there was some > kind of --ignore-zil switch to get my data back online. In any case, zfs > should not panic on corrupted ZIL data, should it? Is there is a way to "manually" use zdb to mimic the "zpool clear" command introduced in OpenSolaris's ZFS with PSARC-2009479? http://www.c0t0d0s0.org/archives/6067-PSARC-2009479-zpool-recovery-a.html I have no idea if this would help: in fact it might very well be dangerous for the pool that Gerrit is trying to recover. Are you able to copy the pool somehow before trying experiments? I think the current state of "disaster recovery" tools and methods for ZFS makes some folks nervous. With so much error checking "built in" there's fewer tried and true "old school" sysadmin approaches to recovering lost data after the fact. So thanks for debugging your problem in public. I hope you can resolve things and document how you did it for everyone. Good luck. From owner-freebsd-fs@FreeBSD.ORG Tue Nov 10 15:46:27 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 855351065692 for ; Tue, 10 Nov 2009 15:46:27 +0000 (UTC) (envelope-from gerrit@pmp.uni-hannover.de) Received: from mrelay1.uni-hannover.de (mrelay1.uni-hannover.de [130.75.2.106]) by mx1.freebsd.org (Postfix) with ESMTP id 0F41A8FC22 for ; Tue, 10 Nov 2009 15:46:26 +0000 (UTC) Received: from www.pmp.uni-hannover.de (www.pmp.uni-hannover.de [130.75.117.2]) by mrelay1.uni-hannover.de (8.14.2/8.14.2) with ESMTP id nAAFkM7S014765; Tue, 10 Nov 2009 16:46:23 +0100 Received: from pmp.uni-hannover.de (arc.pmp.uni-hannover.de [130.75.117.1]) by www.pmp.uni-hannover.de (Postfix) with SMTP id 2E28424; Tue, 10 Nov 2009 16:46:22 +0100 (CET) Date: Tue, 10 Nov 2009 16:46:22 +0100 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: Graham Todd Message-Id: <20091110164622.6bc7aca1.gerrit@pmp.uni-hannover.de> In-Reply-To: <4AF98032.9050808@bellanet.org> References: <20091106094734.4b056899.gerrit@pmp.uni-hannover.de> <4AF4123A.4080301@andric.com> <20091106231440.4f0f2cbb.gerrit@pmp.uni-hannover.de> <4AF4AAFF.2080104@jrv.org> <20091109101255.e81774e4.gerrit@pmp.uni-hannover.de> <4AF98032.9050808@bellanet.org> Organization: Albert-Einstein-Institut (MPI =?ISO-8859-1?Q?f=FCr?= Gravitationsphysik & IGP =?ISO-8859-1?Q?Universit=E4t?= Hannover) X-Mailer: Sylpheed 2.7.1 (GTK+ 2.12.11; i386-portbld-freebsd7.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-PMX-Version: 5.5.5.374460, Antispam-Engine: 2.7.1.369594, Antispam-Data: 2009.11.10.153621 Cc: freebsd-fs@freebsd.org Subject: Re: trace for zfs panic mounting fs after crash with RC2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Nov 2009 15:46:27 -0000 On Tue, 10 Nov 2009 10:01:06 -0500 Graham Todd wrote about Re: trace for zfs panic mounting fs after crash with RC2: GT> > I guess I could live with a corrupted ZIL after a crash, if there GT> > was some kind of --ignore-zil switch to get my data back online. In GT> > any case, zfs should not panic on corrupted ZIL data, should it? GT> GT> Is there is a way to "manually" use zdb to mimic the "zpool clear" GT> command introduced in OpenSolaris's ZFS with PSARC-2009479? GT> GT> http://www.c0t0d0s0.org/archives/6067-PSARC-2009479-zpool-recovery-a.html FYI: Meanwhile I opened a PR for the issue (kern/140433) and got some request for additional zdb input (that I will hopefully be able to provide later this evening). The page above looks interesting, though. There it is mentioned (in the comments) that you can achieve the same thing zpool clear does... but it is not mentioned how. Does anyone here know? GT> I have no idea if this would help: in fact it might very well be GT> dangerous for the pool that Gerrit is trying to recover. Are you able GT> to copy the pool somehow before trying experiments? I do not care that much about this specific pool, since I only installed the system and some software. But I want to know I can handle this situation before I put data on the disks. :-) GT> I think the current state of "disaster recovery" tools and methods for GT> ZFS makes some folks nervous. With so much error checking "built in" GT> there's fewer tried and true "old school" sysadmin approaches to GT> recovering lost data after the fact. As long as these situations do not happen, it's ok for me to have no way to recover. :-) I am using zfs since Pawel made the first patchset available in autumn 2006 and never had to face a situation like this so far. As it happend two times in a row now on this new machine, I guess it must have something to do with the hardware. OTOH, everything seems to run fine, unless the machine crashes and corrupts the zil in some strange way. GT> So thanks for debugging your GT> problem in public. I hope you can resolve things and document how you GT> did it for everyone. I hope we get this resolved, too. As long as I do not have to fear to loose important data, I can do almost anything with the machine for debugging. cu Gerrit From owner-freebsd-fs@FreeBSD.ORG Tue Nov 10 16:17:56 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8D8741065696 for ; Tue, 10 Nov 2009 16:17:56 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-iw0-f190.google.com (mail-iw0-f190.google.com [209.85.223.190]) by mx1.freebsd.org (Postfix) with ESMTP id 4E43A8FC1F for ; Tue, 10 Nov 2009 16:17:56 +0000 (UTC) Received: by iwn28 with SMTP id 28so146704iwn.3 for ; Tue, 10 Nov 2009 08:17:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type; bh=hJz1s3CgciE6AfuYfvJ6GZ+utUyJ+0SH9li40jwDkbw=; b=NwKv2sQThNcWKTomPY2EKWxaFDVbMWD7put+bMHbom7gH/zXMN6Xe117VuOeKN87ig 0afdSYrhNIcMvLyjImzVVLnOxiD7AUUbjw72hi6DPXI4LWW/jniX6zQMi1Tv6+kaoQUs /g0u76gEThg4CqeGMMCV1Im8w+cVmC5pCA/0w= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=jqyK6PqVGYYPg615OSTIySftkpVFiYyjuP33eRXwjNFkuxdPa5vjEFVVpLqYL3jHAe H5qHUWDuAGtO0l/QNkmg6fHRYfNEoIs6CvB21BUUKknTDHvCz1xVyt7BSvLuefJTF0oR e4BifUXG/7blFscMNz39LT2HaZISCvf3nXJuM= MIME-Version: 1.0 Sender: artemb@gmail.com Received: by 10.231.170.201 with SMTP id e9mr457655ibz.17.1257869875586; Tue, 10 Nov 2009 08:17:55 -0800 (PST) In-Reply-To: <20091110164622.6bc7aca1.gerrit@pmp.uni-hannover.de> References: <20091106094734.4b056899.gerrit@pmp.uni-hannover.de> <4AF4123A.4080301@andric.com> <20091106231440.4f0f2cbb.gerrit@pmp.uni-hannover.de> <4AF4AAFF.2080104@jrv.org> <20091109101255.e81774e4.gerrit@pmp.uni-hannover.de> <4AF98032.9050808@bellanet.org> <20091110164622.6bc7aca1.gerrit@pmp.uni-hannover.de> Date: Tue, 10 Nov 2009 08:17:55 -0800 X-Google-Sender-Auth: bb8cd3212e1100af Message-ID: From: Artem Belevich To: =?ISO-8859-1?Q?Gerrit_K=FChn?= Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-fs@freebsd.org Subject: Re: trace for zfs panic mounting fs after crash with RC2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Nov 2009 16:17:56 -0000 > GT> http://www.c0t0d0s0.org/archives/6067-PSARC-2009479-zpool-recovery-a.html > > The page above looks interesting, though. There it is mentioned (in the > comments) that you can achieve the same thing zpool clear does... but it > is not mentioned how. Does anyone here know? Perhaps some of the links on the following post on zfs-discuss may help: http://www.mail-archive.com/zfs-discuss@opensolaris.org/msg26704.html Another option would be to boot from OpenSolaris LiveCD that contains latest zfs changes, import your pool there, fix, export and then re-import it on FreeBSD. Make sure you don't upgrade your pool while running OpenSolaris. --Artem From owner-freebsd-fs@FreeBSD.ORG Tue Nov 10 19:00:11 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B725B10656A4 for ; Tue, 10 Nov 2009 19:00:11 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 8C07B8FC1E for ; Tue, 10 Nov 2009 19:00:11 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id nAAJ0BH2068842 for ; Tue, 10 Nov 2009 19:00:11 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id nAAJ0BT8068841; Tue, 10 Nov 2009 19:00:11 GMT (envelope-from gnats) Date: Tue, 10 Nov 2009 19:00:11 GMT Message-Id: <200911101900.nAAJ0BT8068841@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: Gerrit =?ISO-8859-1?Q?K=FChn?= Cc: Subject: Re: kern/140433: [zfs] [panic] panic while replaying ZIL after crash X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Gerrit =?ISO-8859-1?Q?K=FChn?= List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Nov 2009 19:00:11 -0000 The following reply was made to PR kern/140433; it has been noted by GNATS. From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/140433: [zfs] [panic] panic while replaying ZIL after crash Date: Tue, 10 Nov 2009 19:40:42 +0100 Output of "zdb -C" as requested: tank version=13 name='tank' state=0 txg=32618 pool_guid=17523106262699816181 hostname='' vdev_tree type='root' id=0 guid=17523106262699816181 children[0] type='raidz' id=0 guid=2668789775933362751 nparity=1 metaslab_array=14 metaslab_shift=33 ashift=9 asize=1600334594048 is_log=0 children[0] type='disk' id=0 guid=4872680480919708890 path='/dev/label/disk0' whole_disk=0 DTL=63 children[1] type='disk' id=1 guid=14727435584907659484 path='/dev/label/disk1' whole_disk=0 DTL=60 children[2] type='disk' id=2 guid=1501397252321623055 path='/dev/label/disk2' whole_disk=0 DTL=62 children[3] type='disk' id=3 guid=15105917771654568537 path='/dev/label/disk3' whole_disk=0 DTL=61 From owner-freebsd-fs@FreeBSD.ORG Tue Nov 10 19:19:16 2009 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D74E0106566C; Tue, 10 Nov 2009 19:19:16 +0000 (UTC) (envelope-from jh@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id AEE7F8FC08; Tue, 10 Nov 2009 19:19:16 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id nAAJJGpc086189; Tue, 10 Nov 2009 19:19:16 GMT (envelope-from jh@freefall.freebsd.org) Received: (from jh@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id nAAJJGRv086185; Tue, 10 Nov 2009 19:19:16 GMT (envelope-from jh) Date: Tue, 10 Nov 2009 19:19:16 GMT Message-Id: <200911101919.nAAJJGRv086185@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org, jh@FreeBSD.org, freebsd-fs@FreeBSD.org From: jh@FreeBSD.org Cc: Subject: Re: kern/89991: [ufs] softupdates with mount -ur causes fs UNREFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Nov 2009 19:19:16 -0000 Synopsis: [ufs] softupdates with mount -ur causes fs UNREFS State-Changed-From-To: open->closed State-Changed-By: jh State-Changed-When: Tue Nov 10 19:16:50 UTC 2009 State-Changed-Why: Probably fixed by r183074. http://www.freebsd.org/cgi/query-pr.cgi?pr=89991 From owner-freebsd-fs@FreeBSD.ORG Tue Nov 10 19:19:16 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D74E0106566C; Tue, 10 Nov 2009 19:19:16 +0000 (UTC) (envelope-from jh@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id AEE7F8FC08; Tue, 10 Nov 2009 19:19:16 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id nAAJJGpc086189; Tue, 10 Nov 2009 19:19:16 GMT (envelope-from jh@freefall.freebsd.org) Received: (from jh@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id nAAJJGRv086185; Tue, 10 Nov 2009 19:19:16 GMT (envelope-from jh) Date: Tue, 10 Nov 2009 19:19:16 GMT Message-Id: <200911101919.nAAJJGRv086185@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org, jh@FreeBSD.org, freebsd-fs@FreeBSD.org From: jh@FreeBSD.org Cc: Subject: Re: kern/89991: [ufs] softupdates with mount -ur causes fs UNREFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Nov 2009 19:19:17 -0000 Synopsis: [ufs] softupdates with mount -ur causes fs UNREFS State-Changed-From-To: open->closed State-Changed-By: jh State-Changed-When: Tue Nov 10 19:16:50 UTC 2009 State-Changed-Why: Probably fixed by r183074. http://www.freebsd.org/cgi/query-pr.cgi?pr=89991 From owner-freebsd-fs@FreeBSD.ORG Tue Nov 10 22:45:32 2009 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 09BE3106566B; Tue, 10 Nov 2009 22:45:32 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077035009.chello.pl [89.77.35.9]) by mx1.freebsd.org (Postfix) with ESMTP id 657258FC16; Tue, 10 Nov 2009 22:45:30 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 40F0945E49; Tue, 10 Nov 2009 23:45:29 +0100 (CET) Received: from localhost (chello089077035009.chello.pl [89.77.35.9]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 08C5E45CDC; Tue, 10 Nov 2009 23:45:23 +0100 (CET) Date: Tue, 10 Nov 2009 23:45:24 +0100 From: Pawel Jakub Dawidek To: freebsd-current@FreeBSD.org Message-ID: <20091110224524.GC3194@garage.freebsd.pl> References: <200911102227.nAAMRXTf073603@svn.freebsd.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="w7PDEPdKQumQfZlR" Content-Disposition: inline In-Reply-To: <200911102227.nAAMRXTf073603@svn.freebsd.org> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-fs@FreeBSD.org Subject: HEADS UP: Important bug fix in ZFS replay code! X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Nov 2009 22:45:32 -0000 --w7PDEPdKQumQfZlR Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi. There was important bug in ZFS replay code. If there were setattr logs (not related to permission change) in ZIL during unclean shutdown, one can end up with files that have mode set to 07777. This is very dangerous, especially if you have untrusted local users, as this will set setuid bit on such files. Note that FreeBSD will remove setuid bits when someone will try to modify the file, but it is still dangerous. You can locate such files with the following command: # find / -perm -7777 -print0 | xargs -0 ls -ld You can locate and fix such files with the following command: # find / -perm -7777 -print0 | xargs -0 chmod a-s,o-w,-t On Tue, Nov 10, 2009 at 10:27:33PM +0000, Pawel Jakub Dawidek wrote: > Author: pjd > Date: Tue Nov 10 22:27:33 2009 > New Revision: 199157 > URL: http://svn.freebsd.org/changeset/base/199157 >=20 > Log: > Be careful which vattr fields are set during setattr replay. > Without this fix strange things can appear after unclean shutdown like > files with mode set to 07777. > =20 > Reported by: des > MFC after: 3 days >=20 > Modified: > head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_replay.c >=20 > Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_replay.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_replay.c Tue = Nov 10 22:25:46 2009 (r199156) > +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_replay.c Tue = Nov 10 22:27:33 2009 (r199157) > @@ -60,10 +60,14 @@ zfs_init_vattr(vattr_t *vap, uint64_t ma > { > VATTR_NULL(vap); > vap->va_mask =3D (uint_t)mask; > - vap->va_type =3D IFTOVT(mode); > - vap->va_mode =3D mode & MODEMASK; > - vap->va_uid =3D (uid_t)(IS_EPHEMERAL(uid)) ? -1 : uid; > - vap->va_gid =3D (gid_t)(IS_EPHEMERAL(gid)) ? -1 : gid; > + if (mask & AT_TYPE) > + vap->va_type =3D IFTOVT(mode); > + if (mask & AT_MODE) > + vap->va_mode =3D mode & MODEMASK; > + if (mask & AT_UID) > + vap->va_uid =3D (uid_t)(IS_EPHEMERAL(uid)) ? -1 : uid; > + if (mask & AT_GID) > + vap->va_gid =3D (gid_t)(IS_EPHEMERAL(gid)) ? -1 : gid; > vap->va_rdev =3D zfs_cmpldev(rdev); > vap->va_nodeid =3D nodeid; > } --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --w7PDEPdKQumQfZlR Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFK+e0EForvXbEpPzQRAmJnAKDwdm/rs75bZuvUBuiCqRusQWaxAgCg0NWE EJ16L2S+kqi3fsBxJPtIqNs= =FH86 -----END PGP SIGNATURE----- --w7PDEPdKQumQfZlR-- From owner-freebsd-fs@FreeBSD.ORG Wed Nov 11 20:32:49 2009 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8CBE11065679 for ; Wed, 11 Nov 2009 20:32:49 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.freebsd.org (Postfix) with ESMTP id 4FDFB8FC14 for ; Wed, 11 Nov 2009 20:32:49 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 887B66D41B; Wed, 11 Nov 2009 20:14:49 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 2FE3F844B4; Wed, 11 Nov 2009 21:14:49 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Pawel Jakub Dawidek References: <200911102227.nAAMRXTf073603@svn.freebsd.org> <20091110224524.GC3194@garage.freebsd.pl> Date: Wed, 11 Nov 2009 21:14:48 +0100 In-Reply-To: <20091110224524.GC3194@garage.freebsd.pl> (Pawel Jakub Dawidek's message of "Tue, 10 Nov 2009 23:45:24 +0100") Message-ID: <86k4xwom2v.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.95 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org Subject: Re: HEADS UP: Important bug fix in ZFS replay code! X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Nov 2009 20:32:49 -0000 Pawel Jakub Dawidek writes: > You can locate such files with the following command: > > # find / -perm -7777 -print0 | xargs -0 ls -ld or 'grep rws /var/run/setuid.today' :) DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-fs@FreeBSD.ORG Wed Nov 11 21:30:09 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A48B71065670 for ; Wed, 11 Nov 2009 21:30:09 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 7A1FD8FC14 for ; Wed, 11 Nov 2009 21:30:09 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id nABLU9MB074549 for ; Wed, 11 Nov 2009 21:30:09 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id nABLU9VR074548; Wed, 11 Nov 2009 21:30:09 GMT (envelope-from gnats) Date: Wed, 11 Nov 2009 21:30:09 GMT Message-Id: <200911112130.nABLU9VR074548@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: kickbsd kickbsd Cc: Subject: Re: kern/139715: [zfs] vfs.numvnodes leak on busy zfs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: kickbsd kickbsd List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Nov 2009 21:30:09 -0000 The following reply was made to PR kern/139715; it has been noted by GNATS. From: kickbsd kickbsd To: bug-followup@freebsd.org Cc: Subject: Re: kern/139715: [zfs] vfs.numvnodes leak on busy zfs Date: Thu, 12 Nov 2009 00:16:04 +0300 Same issue observed on RC3 [root@testzfs /tmp]# sysctl vfs.numvnodes ; i=1 ; while [ $i -le 10000 ] ; do echo "sdfsdfsdf" > `mktemp -t ABC` ; i=$(($i+1)) ; done ; sysctl vfs.numvnodes vfs.numvnodes: 860 vfs.numvnodes: 10861 [root@testzfs /tmp]# sysctl vfs.numvnodes ; i=1 ; while [ $i -le 10000 ] ; do echo "sdfsdfsdf" > `mktemp -t ABC` ; i=$(($i+1)) ; done ; sysctl vfs.numvnodes vfs.numvnodes: 10863 vfs.numvnodes: 20863 From owner-freebsd-fs@FreeBSD.ORG Thu Nov 12 02:44:48 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 34A8C1065672 for ; Thu, 12 Nov 2009 02:44:48 +0000 (UTC) (envelope-from tamgya@gmail.com) Received: from mail-gx0-f218.google.com (mail-gx0-f218.google.com [209.85.217.218]) by mx1.freebsd.org (Postfix) with ESMTP id DAD218FC17 for ; Thu, 12 Nov 2009 02:44:47 +0000 (UTC) Received: by gxk10 with SMTP id 10so1689564gxk.3 for ; Wed, 11 Nov 2009 18:44:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:x-opensmtpd-loop:received :x-envelope-to:to:subject:mime-version:content-type:user-agent:from :organization:mail-followup-to:date:message-id; bh=j0ZSWGAv6zbi5R+ytR/shSPPXJKwrtfQdp47tcjcGPc=; b=ZqCnpfW+Gf/oQqJcNPvX3ALZrbnTyN3De4lxeoQ3vofwUYMr2FX8wTn1lMJaA2AMRb hmYn7/XJQHf7scoPkOWBeKrql9P7MLZcV6q4k2v2A5YeFzL8qkZj+iHfJWL+jkLGzJrW CeoVLXZN9xvd33E7ZbiYy/bzND0vch6Dl0f58= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=x-opensmtpd-loop:x-envelope-to:to:subject:mime-version:content-type :user-agent:from:organization:mail-followup-to:date:message-id; b=r8Z77Md58U3oMgdgwFR4PdQXGzTYWzGyAMdxh5y0nNOfQutleCH9pD19W27UKrzLMg cVAqDzwwBCdMkB/jHWhOWgWXEWeS7DI/pvgBRTU7+hDNmQpxcqIgopznN4+72RHsFoCe uHiikIPcSTC39G/9joHgeUVKiwIF8N+X2q8Uc= Received: by 10.90.16.35 with SMTP id 35mr3536080agp.54.1257991928909; Wed, 11 Nov 2009 18:12:08 -0800 (PST) Received: from mecuria.xbsd.name ([124.207.115.45]) by mx.google.com with ESMTPS id 9sm1088038yxf.23.2009.11.11.18.12.05 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 11 Nov 2009 18:12:08 -0800 (PST) X-OpenSMTPD-Loop: freebsd-fs@freebsd.org Received: from mecuria.xbsd.name (localhost.xbsd.name [IPv6:::1]) by mecuria.xbsd.name (OpenSMTPD) with ESMTP id 1257991917.cR3wD5HTzji8PiTK; Thu, 12 Nov 2009 10:11:57 +0800 (CST) X-Envelope-To: freebsd-fs@freebsd.org To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.3 (berkeley-unix) From: tamgya@gmail.com (Denise H. G.) Organization: Terra Firma Mail-Followup-To: freebsd-fs@freebsd.org Date: Thu, 12 Nov 2009 10:11:56 +0800 Message-ID: <87vdhglcer.fsf@mecuria.xbsd.name> Subject: Is 'zfs prefetch' dangerous?CC: X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Nov 2009 02:44:48 -0000 Hi list. I've been wondering if I should turn on prefetch for ZFS, but after a search in google, I found there seem to be some issues on ZFS prefetch. Since I have got 4GB RAM, maybe it is a good idea to turn it on. My machine is mainly for home use and has no important data on it. If I turn it on, will my machine run smooth? Thanks, regards. -- tamgya |aT| GmAiL |DoT| cOm From owner-freebsd-fs@FreeBSD.ORG Thu Nov 12 08:26:36 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 42A06106566C for ; Thu, 12 Nov 2009 08:26:36 +0000 (UTC) (envelope-from gerrit@pmp.uni-hannover.de) Received: from mrelay1.uni-hannover.de (mrelay1.uni-hannover.de [130.75.2.106]) by mx1.freebsd.org (Postfix) with ESMTP id C02128FC2A for ; Thu, 12 Nov 2009 08:26:35 +0000 (UTC) Received: from www.pmp.uni-hannover.de (www.pmp.uni-hannover.de [130.75.117.2]) by mrelay1.uni-hannover.de (8.14.2/8.14.2) with ESMTP id nAC8QUwd010258; Thu, 12 Nov 2009 09:26:32 +0100 Received: from pmp.uni-hannover.de (arc.pmp.uni-hannover.de [130.75.117.1]) by www.pmp.uni-hannover.de (Postfix) with SMTP id 642A124; Thu, 12 Nov 2009 09:26:30 +0100 (CET) Date: Thu, 12 Nov 2009 09:26:30 +0100 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: Artem Belevich Message-Id: <20091112092630.e7cd6836.gerrit@pmp.uni-hannover.de> In-Reply-To: References: <20091106094734.4b056899.gerrit@pmp.uni-hannover.de> <4AF4123A.4080301@andric.com> <20091106231440.4f0f2cbb.gerrit@pmp.uni-hannover.de> <4AF4AAFF.2080104@jrv.org> <20091109101255.e81774e4.gerrit@pmp.uni-hannover.de> <4AF98032.9050808@bellanet.org> <20091110164622.6bc7aca1.gerrit@pmp.uni-hannover.de> Organization: Albert-Einstein-Institut (MPI =?ISO-8859-1?Q?f=FCr?= Gravitationsphysik & IGP =?ISO-8859-1?Q?Universit=E4t?= Hannover) X-Mailer: Sylpheed 2.7.1 (GTK+ 2.12.11; i386-portbld-freebsd7.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-PMX-Version: 5.5.5.374460, Antispam-Engine: 2.7.1.369594, Antispam-Data: 2009.11.12.81824 Cc: freebsd-fs@freebsd.org Subject: Re: trace for zfs panic mounting fs after crash with RC2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Nov 2009 08:26:36 -0000 On Tue, 10 Nov 2009 08:17:55 -0800 Artem Belevich wrote about Re: trace for zfs panic mounting fs after crash with RC2: AB> Perhaps some of the links on the following post on zfs-discuss may AB> help: AB> http://www.mail-archive.com/zfs-discuss@opensolaris.org/msg26704.html Interesting stuff, thanks. At a first glance I do not see an easy way to roll back my pool to a slightly previous (consistent) state, but all the posts state that it is possible. I guess I have to dive into this a bit deeper. "zpool clear -F" definitely would be the easier-to-use solution. AB> Another option would be to boot from OpenSolaris LiveCD that contains AB> latest zfs changes, import your pool there, fix, export and then AB> re-import it on FreeBSD. Make sure you don't upgrade your pool while AB> running OpenSolaris. Uh, yes, not really an option in this case, I guess. Unless I buy an additional external CD drive and stuff. But thanks for the hint, anyway. I will have a look around how difficult it is to get recent OpenSolaris on a USB stick... cu Gerrit From owner-freebsd-fs@FreeBSD.ORG Thu Nov 12 12:06:18 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ADE0910656A6 for ; Thu, 12 Nov 2009 12:06:18 +0000 (UTC) (envelope-from gerrit@pmp.uni-hannover.de) Received: from mrelay1.uni-hannover.de (mrelay1.uni-hannover.de [130.75.2.106]) by mx1.freebsd.org (Postfix) with ESMTP id 206358FC16 for ; Thu, 12 Nov 2009 12:06:17 +0000 (UTC) Received: from www.pmp.uni-hannover.de (www.pmp.uni-hannover.de [130.75.117.2]) by mrelay1.uni-hannover.de (8.14.2/8.14.2) with ESMTP id nACC6F0C019641; Thu, 12 Nov 2009 13:06:16 +0100 Received: from pmp.uni-hannover.de (arc.pmp.uni-hannover.de [130.75.117.1]) by www.pmp.uni-hannover.de (Postfix) with SMTP id 430124F; Thu, 12 Nov 2009 13:06:15 +0100 (CET) Date: Thu, 12 Nov 2009 13:06:15 +0100 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: "James R. Van Artsdalen" Message-Id: <20091112130615.64b44914.gerrit@pmp.uni-hannover.de> In-Reply-To: <4AF4AAFF.2080104@jrv.org> References: <20091106094734.4b056899.gerrit@pmp.uni-hannover.de> <4AF4123A.4080301@andric.com> <20091106231440.4f0f2cbb.gerrit@pmp.uni-hannover.de> <4AF4AAFF.2080104@jrv.org> Organization: Albert-Einstein-Institut (MPI =?ISO-8859-1?Q?f=FCr?= Gravitationsphysik & IGP =?ISO-8859-1?Q?Universit=E4t?= Hannover) X-Mailer: Sylpheed 2.7.1 (GTK+ 2.12.11; i386-portbld-freebsd7.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-PMX-Version: 5.5.5.374460, Antispam-Engine: 2.7.1.369594, Antispam-Data: 2009.11.12.115426 Cc: freebsd-fs@freebsd.org Subject: Re: trace for zfs panic mounting fs after crash with RC2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Nov 2009 12:06:18 -0000 On Fri, 06 Nov 2009 17:02:23 -0600 "James R. Van Artsdalen" wrote about Re: trace for zfs panic mounting fs after crash with RC2: JRVA> How the ZIL got corrupted - if it did - is a harder question. What JRVA> kind of hard disk is this, and how is it connected to the system? JRVA> Was there any redundancy (mirror, raidz)? I have been thinking about this for some time now. I have almost the same controller (low-profile version, different bios, but otherwise identical) in use without these problems. Can the 2.5" disks cause any problems? The problematic system is the only one I have with the small drives. Maybe they somehow "lie" to the system about the data actually being written? I remember that a long time ago (about 10 years?) FreeBSD people suggested to turn off the write cache of disk drives to prevent data losses. I see that the sysctl hw.ata.wc is still there. Do people here think that this is worth giving a try? Are there any recent experiences concerning the performance-wise impact on zfs when turning off wc? cu Gerrit From owner-freebsd-fs@FreeBSD.ORG Thu Nov 12 14:42:50 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BE06D106566B for ; Thu, 12 Nov 2009 14:42:50 +0000 (UTC) (envelope-from matthias.andree@gmx.de) Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by mx1.freebsd.org (Postfix) with SMTP id 07D088FC1D for ; Thu, 12 Nov 2009 14:42:49 +0000 (UTC) Received: (qmail invoked by alias); 12 Nov 2009 14:16:09 -0000 Received: from balu.cs.uni-paderborn.de (EHLO balu.cs.uni-paderborn.de) [131.234.21.37] by mail.gmx.net (mp021) with SMTP; 12 Nov 2009 15:16:09 +0100 X-Authenticated: #428038 X-Provags-ID: V01U2FsdGVkX1/iJwQJXLPiVxQ5BfarlpYCy4+YaxPaFsvlc06rhn GI7c+9p0Es0kdm Received: from localhost ([127.0.0.1] helo=balu.cs.uni-paderborn.de) by balu.cs.uni-paderborn.de with esmtp (Exim 4.69) (envelope-from ) id KT02AW-0003F0-PK; Thu, 12 Nov 2009 15:16:08 +0100 Content-Type: text/plain; charset=iso-8859-15; format=flowed; delsp=yes To: "Pawel Jakub Dawidek" , freebsd-current@freebsd.org References: <200911102227.nAAMRXTf073603@svn.freebsd.org> <20091110224524.GC3194@garage.freebsd.pl> Date: Thu, 12 Nov 2009 15:16:08 +0100 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: "Matthias Andree" Organization: Message-ID: In-Reply-To: <20091110224524.GC3194@garage.freebsd.pl> User-Agent: Opera Mail/10.01 (Win32) X-Y-GMX-Trusted: 0 X-FuHaFi: 0.64 Cc: freebsd-fs@freebsd.org Subject: Re: HEADS UP: Important bug fix in ZFS replay code! X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Nov 2009 14:42:50 -0000 Am 10.11.2009, 23:45 Uhr, schrieb Pawel Jakub Dawidek : > You can locate such files with the following command: > > # find / -perm -7777 -print0 | xargs -0 ls -ld Use ls -ldb to be on the safe side (control characters!). So how about these refinements: find / -perm -7777 -exec ls -ldb '{}' + find / -perm -7777 -ls (not sure what that does with escapes) > You can locate and fix such files with the following command: > > # find / -perm -7777 -print0 | xargs -0 chmod a-s,o-w,-t find / -perm -7777 -exec chmod a-s,o-w,-t '{}' + -- Matthias Andree From owner-freebsd-fs@FreeBSD.ORG Thu Nov 12 14:44:34 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5F3151065672 for ; Thu, 12 Nov 2009 14:44:34 +0000 (UTC) (envelope-from tevans.uk@googlemail.com) Received: from ey-out-2122.google.com (ey-out-2122.google.com [74.125.78.27]) by mx1.freebsd.org (Postfix) with ESMTP id E5B158FC1B for ; Thu, 12 Nov 2009 14:44:33 +0000 (UTC) Received: by ey-out-2122.google.com with SMTP id 9so587330eyd.9 for ; Thu, 12 Nov 2009 06:44:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=yv6lwIM8ceS6zAsqeTE3MNY9IE9H0vL/Xf9nqIkkBbk=; b=jM3mmuERBYXPwulHjAfBf3kiLzlMZUkO7IUbwSvnWQy22NE1XBMcbL7gZyJN8vDYPy uLck2pRzqQa75qwpl+OGPuRfW6r9XYtIzqwhPNHSICbl0yvnbl3+uUQq8H7lpFRckv10 6gJeddPHE3+PGFQ1yFLUvAHbTnukXdT9uZm4I= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=gL6qrkRR2zCTnq2rFKw12gGZldk/rJuqwDYa2sWdzVPcXQsS5fy8wOVAG/X5In6Wnl 2e/My6IOuvzIegVOg4vtg6N4C2OV/CXMSmhQPB94QIbYYtZD9qxdfF/C/ABC7qm7W6Hr E2aW99eLCWrx99jSKX1TN0znG+N/9268PNumQ= MIME-Version: 1.0 Received: by 10.213.103.144 with SMTP id k16mr2545073ebo.66.1258035823265; Thu, 12 Nov 2009 06:23:43 -0800 (PST) In-Reply-To: References: <200911102227.nAAMRXTf073603@svn.freebsd.org> <20091110224524.GC3194@garage.freebsd.pl> Date: Thu, 12 Nov 2009 14:23:43 +0000 Message-ID: <2e027be00911120623v2019be2euc48a6f0ec9a049a6@mail.gmail.com> From: Tom Evans To: Matthias Andree , freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Re: HEADS UP: Important bug fix in ZFS replay code! X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Nov 2009 14:44:34 -0000 On Thu, Nov 12, 2009 at 2:16 PM, Matthias Andree wrote: > Am 10.11.2009, 23:45 Uhr, schrieb Pawel Jakub Dawidek : > > > You can locate such files with the following command: >> >> # find / -perm -7777 -print0 | xargs -0 ls -ld >> > > Use ls -ldb to be on the safe side (control characters!). > > So how about these refinements: > > find / -perm -7777 -exec ls -ldb '{}' + > find / -perm -7777 -ls (not sure what that does with escapes) > > > You can locate and fix such files with the following command: >> >> # find / -perm -7777 -print0 | xargs -0 chmod a-s,o-w,-t >> > > find / -perm -7777 -exec chmod a-s,o-w,-t '{}' + > > -- > Matthias Andree > > > -exec causes a fork()/exec() for each file found doesn't it? xargs would be more efficient (since we're bikeshedding :) Cheers Tom From owner-freebsd-fs@FreeBSD.ORG Thu Nov 12 17:24:17 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E30C8106566C for ; Thu, 12 Nov 2009 17:24:17 +0000 (UTC) (envelope-from gerrit@pmp.uni-hannover.de) Received: from mrelay1.uni-hannover.de (mrelay1.uni-hannover.de [130.75.2.106]) by mx1.freebsd.org (Postfix) with ESMTP id 5A3EB8FC19 for ; Thu, 12 Nov 2009 17:24:17 +0000 (UTC) Received: from www.pmp.uni-hannover.de (www.pmp.uni-hannover.de [130.75.117.2]) by mrelay1.uni-hannover.de (8.14.2/8.14.2) with ESMTP id nACHOEIE005128 for ; Thu, 12 Nov 2009 18:24:15 +0100 Received: from pmp.uni-hannover.de (arc.pmp.uni-hannover.de [130.75.117.1]) by www.pmp.uni-hannover.de (Postfix) with SMTP id A238E24 for ; Thu, 12 Nov 2009 18:24:14 +0100 (CET) Date: Thu, 12 Nov 2009 18:24:14 +0100 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: freebsd-fs@freebsd.org Message-Id: <20091112182414.cebec1df.gerrit@pmp.uni-hannover.de> Organization: Albert-Einstein-Institut (MPI =?ISO-8859-1?Q?f=FCr?= Gravitationsphysik & IGP =?ISO-8859-1?Q?Universit=E4t?= Hannover) X-Mailer: Sylpheed 2.7.1 (GTK+ 2.12.11; i386-portbld-freebsd7.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-PMX-Version: 5.5.5.374460, Antispam-Engine: 2.7.1.369594, Antispam-Data: 2009.11.12.170928 Subject: nfsv4 FreeBSD server vs. Linux client I/O error X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Nov 2009 17:24:18 -0000 Hi all, ist this the right place to talk about nfsv4 issues, or does this better go to -net (or even somewhere else)? Anyways, I'll start here and now: I have a FreeBSD8-RC2 server which I set up for nfsv4 serving according to nfsv4(4). I have a Linux client (Kernel 2.6.25) trying to do "mount.nfs4 / /mnt -v". This takes *exactly* 30s. (looks like running into some kind of timeout). After that, the mount is there. It is displayed by "mount" and gives the right sizes with "du -h". However, as soon as I try to access /mnt with "cd /mnt" or "ls /mnt" I get an Input/Output Error on the client side. Googleing around I found only very few information about nfsv4 and FreeBSD (ok, it's a new feature after all :-). Does anyone here already have some experiences to share? Anyone already using FreeBSD servers against Linux clients? Any suggestions how to debug and solve the problem above? Any hints are as always greatly appreciated. cu Gerrit From owner-freebsd-fs@FreeBSD.ORG Thu Nov 12 17:53:28 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CA5DE106568D for ; Thu, 12 Nov 2009 17:53:28 +0000 (UTC) (envelope-from matthias.andree@gmx.de) Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by mx1.freebsd.org (Postfix) with SMTP id 107468FC14 for ; Thu, 12 Nov 2009 17:53:27 +0000 (UTC) Received: (qmail invoked by alias); 12 Nov 2009 17:53:25 -0000 Received: from balu.cs.uni-paderborn.de (EHLO balu.cs.uni-paderborn.de) [131.234.21.37] by mail.gmx.net (mp036) with SMTP; 12 Nov 2009 18:53:25 +0100 X-Authenticated: #428038 X-Provags-ID: V01U2FsdGVkX1+oYhRrOASoWkB/w0fG/itFhSI90AaIIKynFuJDMb rXYtriQ7Ob6+UK Received: from localhost ([127.0.0.1] helo=balu.cs.uni-paderborn.de) by balu.cs.uni-paderborn.de with esmtp (Exim 4.69) (envelope-from ) id KT0CD1-00023O-75; Thu, 12 Nov 2009 18:53:25 +0100 Content-Type: text/plain; charset=iso-8859-15; format=flowed; delsp=yes To: "Tom Evans" , freebsd-fs@freebsd.org References: <200911102227.nAAMRXTf073603@svn.freebsd.org> <20091110224524.GC3194@garage.freebsd.pl> <2e027be00911120623v2019be2euc48a6f0ec9a049a6@mail.gmail.com> Date: Thu, 12 Nov 2009 18:53:25 +0100 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: "Matthias Andree" Organization: Message-ID: In-Reply-To: <2e027be00911120623v2019be2euc48a6f0ec9a049a6@mail.gmail.com> User-Agent: Opera Mail/10.01 (Win32) X-Y-GMX-Trusted: 0 X-FuHaFi: 0.61 Cc: Subject: Re: HEADS UP: Important bug fix in ZFS replay code! X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Nov 2009 17:53:28 -0000 Am 12.11.2009, 15:23 Uhr, schrieb Tom Evans : >> So how about these refinements: >> >> find / -perm -7777 -exec ls -ldb '{}' + >> find / -perm -7777 -ls (not sure what that does with escapes) >> >> >> You can locate and fix such files with the following command: >>> >>> # find / -perm -7777 -print0 | xargs -0 chmod a-s,o-w,-t >>> >> >> find / -perm -7777 -exec chmod a-s,o-w,-t '{}' + >> >> -- >> Matthias Andree >> >> >> -exec causes a fork()/exec() for each file found doesn't it? xargs >> would be > more efficient (since we're bikeshedding :) That's the subtle difference between "+" and "\;" at the end if using '{}' :) -- Matthias Andree From owner-freebsd-fs@FreeBSD.ORG Thu Nov 12 19:37:25 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 37356106568D for ; Thu, 12 Nov 2009 19:37:25 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id B29D78FC12 for ; Thu, 12 Nov 2009 19:37:23 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApoEAG/y+0qDaFvJ/2dsb2JhbADdeYQ8BIFs X-IronPort-AV: E=Sophos;i="4.44,730,1249272000"; d="scan'208";a="53658853" Received: from ganges.cs.uoguelph.ca ([131.104.91.201]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 12 Nov 2009 14:37:22 -0500 Received: from localhost (localhost.localdomain [127.0.0.1]) by ganges.cs.uoguelph.ca (Postfix) with ESMTP id CC937FB80AD; Thu, 12 Nov 2009 14:37:22 -0500 (EST) X-Virus-Scanned: amavisd-new at ganges.cs.uoguelph.ca Received: from ganges.cs.uoguelph.ca ([127.0.0.1]) by localhost (ganges.cs.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kyGCKWF9Ptop; Thu, 12 Nov 2009 14:37:21 -0500 (EST) Received: from muncher.cs.uoguelph.ca (muncher.cs.uoguelph.ca [131.104.91.102]) by ganges.cs.uoguelph.ca (Postfix) with ESMTP id 2CE99FB801A; Thu, 12 Nov 2009 14:37:21 -0500 (EST) Received: from localhost (rmacklem@localhost) by muncher.cs.uoguelph.ca (8.11.7p3+Sun/8.11.6) with ESMTP id nACJj4q17144; Thu, 12 Nov 2009 14:45:04 -0500 (EST) X-Authentication-Warning: muncher.cs.uoguelph.ca: rmacklem owned process doing -bs Date: Thu, 12 Nov 2009 14:45:04 -0500 (EST) From: Rick Macklem X-X-Sender: rmacklem@muncher.cs.uoguelph.ca To: =?utf-8?B?R2Vycml0IEvDvGhu?= In-Reply-To: <20091112182414.cebec1df.gerrit@pmp.uni-hannover.de> Message-ID: References: <20091112182414.cebec1df.gerrit@pmp.uni-hannover.de> MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="-559023410-1804928587-1258055104=:15244" Cc: freebsd-fs@freebsd.org Subject: Re: nfsv4 FreeBSD server vs. Linux client I/O error X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Nov 2009 19:37:25 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. ---559023410-1804928587-1258055104=:15244 Content-Type: TEXT/PLAIN; charset=X-UNKNOWN; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE On Thu, 12 Nov 2009, Gerrit K=C3=BChn wrote: > Hi all, > > ist this the right place to talk about nfsv4 issues, or does this better > go to -net (or even somewhere else)? > > Anyways, I'll start here and now: > I have a FreeBSD8-RC2 server which I set up for nfsv4 serving according > to nfsv4(4). I have a Linux client (Kernel 2.6.25) trying to do > "mount.nfs4 / /mnt -v". This takes *exactly* 30s. (looks like running int= o > some kind of timeout). After that, the mount is there. It is displayed by > "mount" and gives the right sizes with "du -h". > However, as soon as I try to access /mnt with "cd /mnt" or "ls /mnt" I ge= t > an Input/Output Error on the client side. > A few things to check on the server: - Did you add a "V4:" line to your /etc/exports and what did you set as the root path in it? If you used "V4: /" then the root file system would need to be exported by another line in /etc/exports for it to work. - If you are only exporting another filesystem, lets say "/exports", then your mount command would have to look like: =09mount -t nfs4 /exports /mnt (assuming "V4: /" was used) - If you used "V4: /exports", then "mount -t nfs4 :/ /mnt" would work and you would see /exports at /mnt. Beyond something like the above, if you capture packets using "tcpdump -s 0 -w host " on the server and then email me "", I can take a look at it. (tcpdump doesn't know diddly about NFSv4, but wireshark does and can handle tcpdump captures.) > Googleing around I found only very few information about nfsv4 and FreeBS= D > (ok, it's a new feature after all :-). Does anyone here already have some > experiences to share? Anyone already using FreeBSD servers against Linux > clients? Any suggestions how to debug and solve the problem above? > I don't usually test against Linux, but I'll try a quick mount here, in case something is obviously broken. rick ---559023410-1804928587-1258055104=:15244-- From owner-freebsd-fs@FreeBSD.ORG Thu Nov 12 22:10:16 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C9507106566B for ; Thu, 12 Nov 2009 22:10:16 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 4C77D8FC12 for ; Thu, 12 Nov 2009 22:10:15 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApoEAMIW/EqDaFvK/2dsb2JhbADdLYQ8BIFt X-IronPort-AV: E=Sophos;i="4.44,731,1249272000"; d="scan'208";a="53680372" Received: from fraser.cs.uoguelph.ca ([131.104.91.202]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 12 Nov 2009 17:10:14 -0500 Received: from localhost (localhost.localdomain [127.0.0.1]) by fraser.cs.uoguelph.ca (Postfix) with ESMTP id EAE40109C311; Thu, 12 Nov 2009 17:10:14 -0500 (EST) X-Virus-Scanned: amavisd-new at fraser.cs.uoguelph.ca Received: from fraser.cs.uoguelph.ca ([127.0.0.1]) by localhost (fraser.cs.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2UfuJ1DumiPK; Thu, 12 Nov 2009 17:10:14 -0500 (EST) Received: from muncher.cs.uoguelph.ca (muncher.cs.uoguelph.ca [131.104.91.102]) by fraser.cs.uoguelph.ca (Postfix) with ESMTP id 6673A109C2EC; Thu, 12 Nov 2009 17:10:14 -0500 (EST) Received: from localhost (rmacklem@localhost) by muncher.cs.uoguelph.ca (8.11.7p3+Sun/8.11.6) with ESMTP id nACMHtG16356; Thu, 12 Nov 2009 17:17:56 -0500 (EST) X-Authentication-Warning: muncher.cs.uoguelph.ca: rmacklem owned process doing -bs Date: Thu, 12 Nov 2009 17:17:55 -0500 (EST) From: Rick Macklem X-X-Sender: rmacklem@muncher.cs.uoguelph.ca To: =?utf-8?B?R2Vycml0IEvDvGhu?= In-Reply-To: <20091112182414.cebec1df.gerrit@pmp.uni-hannover.de> Message-ID: References: <20091112182414.cebec1df.gerrit@pmp.uni-hannover.de> MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="-559023410-684387517-1258064275=:14430" Cc: freebsd-fs@freebsd.org Subject: Re: nfsv4 FreeBSD server vs. Linux client I/O error X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Nov 2009 22:10:16 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. ---559023410-684387517-1258064275=:14430 Content-Type: TEXT/PLAIN; charset=X-UNKNOWN; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE On Thu, 12 Nov 2009, Gerrit K=C3=BChn wrote: > Hi all, > > ist this the right place to talk about nfsv4 issues, or does this better > go to -net (or even somewhere else)? > > Anyways, I'll start here and now: > I have a FreeBSD8-RC2 server which I set up for nfsv4 serving according > to nfsv4(4). I have a Linux client (Kernel 2.6.25) trying to do > "mount.nfs4 / /mnt -v". This takes *exactly* 30s. (looks like running int= o > some kind of timeout). After that, the mount is there. It is displayed by > "mount" and gives the right sizes with "du -h". > However, as soon as I try to access /mnt with "cd /mnt" or "ls /mnt" I ge= t > an Input/Output Error on the client side. > One more thing that came to mind. If your root fs in NFS mounted, it can't be exported, so you have to use the version (assuming /exports is a local on-disk file system that is exported): V4: /export and not V4: / and then the mount needs to look like: =09mount -t nfs4 :/export /mnt I tried a relatively recent Ubuntu client here and it seemed to mount ok. (There are many variants of the mount utilities and patch versions of the nfs4 client for Linux, so your mileage definitely may vary.) I'll be happy to look at a tcpdump capture, if you get one, rick ---559023410-684387517-1258064275=:14430-- From owner-freebsd-fs@FreeBSD.ORG Fri Nov 13 01:12:09 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B1CB1106568D; Fri, 13 Nov 2009 01:12:09 +0000 (UTC) (envelope-from mattjreimer@gmail.com) Received: from mail-pw0-f44.google.com (mail-pw0-f44.google.com [209.85.160.44]) by mx1.freebsd.org (Postfix) with ESMTP id 775F88FC1C; Fri, 13 Nov 2009 01:12:09 +0000 (UTC) Received: by pwj15 with SMTP id 15so1866771pwj.3 for ; Thu, 12 Nov 2009 17:12:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=qwcZdGxprnJhQlYTq8U1772wRGxMSE2twEDuMwip9Xk=; b=ehhUWK84/U9xHNCCgYVMuuqLTVHJumzcLMfiAl8zcL3D0jIgB/qoEE9kK7mVX2+2v7 tBHmlHQ6c1BzpgTdp4N9IOZlg47CppjYNcABIMaUlbnTQEOueYZY0uARR27ViQJ9fWgg LXKURfksOBHBZe8jDwgf4rg1LQqiIo/R8fik4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=P8dX6ejB2fK9Io3m5BaWS2cGX0FoHmEbVLcE205gpYiW595Q5yGH3lkY5HbCX36bN/ vuRNLKdxZ4zPGJuWCYhOWkP7DnfnIi0bxKMxbDA5GuF4c7ZoNT8s+wUzoS7tfv+UG65L 2kExQ3SWGUpNbYMDBLtcUHDOfYFqudLEmCDnE= MIME-Version: 1.0 Received: by 10.142.196.18 with SMTP id t18mr422896wff.32.1258074281549; Thu, 12 Nov 2009 17:04:41 -0800 (PST) In-Reply-To: References: <4AD710D6.70404@buchlovice.org> Date: Thu, 12 Nov 2009 17:04:41 -0800 Message-ID: From: Matt Reimer To: =?ISO-8859-2?Q?Radek_Val=E1=B9ek?= Content-Type: multipart/mixed; boundary=000e0cd32f54a7210c0478363e64 Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, rnoland@freebsd.org Subject: Re: GPT boot with ZFS RAIDZ "ZFS: i/o error - all block copies unavailable" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2009 01:12:09 -0000 --000e0cd32f54a7210c0478363e64 Content-Type: text/plain; charset=ISO-8859-1 2009/11/12 Matt Reimer : > > Radek, > > Try the attached patch (sponsored by VPOP Technologies). I found an > overflow in /sys/cddl/boot/zfs/zfssubr.c:vdev_raidz_read() that was > causing my 6x1TB raidz2 array to fail to boot. > > Apply the patch, build everything in /sys/boot, and then make sure you > update both gptzfsboot and /boot/loader. Oops, here's the patch. Matt --000e0cd32f54a7210c0478363e64 Content-Type: application/octet-stream; name="zfssubr.c.patch" Content-Disposition: attachment; filename="zfssubr.c.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_g1y8xrsy0 LS0tIHpmc3N1YnIuYy5vcmlnCTIwMDktMTAtMjQgMTg6MTA6MjkuMDAwMDAwMDAwIC0wNzAwCisr KyB6ZnNzdWJyLmMJMjAwOS0xMS0xMiAxNjozNzowNS4wMDAwMDAwMDAgLTA4MDAKQEAgLTU1MCw3 ICs1NTAsOCBAQAogCXVpbnQ2NF90IHMgPSBwc2l6ZSA+PiB1bml0X3NoaWZ0OwogCXVpbnQ2NF90 IGYgPSBiICUgZGNvbHM7CiAJdWludDY0X3QgbyA9IChiIC8gZGNvbHMpIDw8IHVuaXRfc2hpZnQ7 Ci0JaW50IHEsIHIsIGMsIGMxLCBiYywgY29sLCBhY29scywgY29mZiwgZGV2aWR4LCBhc2l6ZSwg bjsKKwl1aW50NjRfdCBxLCByLCBjb2ZmOworCWludCBjLCBjMSwgYmMsIGNvbCwgYWNvbHMsIGRl dmlkeCwgYXNpemUsIG47CiAJc3RhdGljIHJhaWR6X2NvbF90IGNvbHNbMTZdOwogCXJhaWR6X2Nv bF90ICpyYywgKnJjMTsKIAo= --000e0cd32f54a7210c0478363e64-- From owner-freebsd-fs@FreeBSD.ORG Fri Nov 13 01:27:04 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2F0E8106566C; Fri, 13 Nov 2009 01:27:04 +0000 (UTC) (envelope-from mattjreimer@gmail.com) Received: from mail-pw0-f44.google.com (mail-pw0-f44.google.com [209.85.160.44]) by mx1.freebsd.org (Postfix) with ESMTP id EB4128FC0A; Fri, 13 Nov 2009 01:27:03 +0000 (UTC) Received: by pwj15 with SMTP id 15so1873518pwj.3 for ; Thu, 12 Nov 2009 17:27:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=7BJZAVO8VE2F9u3NLNTioMpmeUWFlAT5ZbDXB2Qt/V0=; b=hlEhMU3Q7gL0GdYFnNZXl5Y/osWVg117D5gds/w2FNMzk1dKS3DDBu2bHIHV97oV7v /MzZdowTpLFHY3hYRbjoDTGfS1442KtBM8n6m4/X6gsv732Dg2EtvCwtSiyzoCo6afhR G6KlHRgEHsG2xBxk4ce4BUkIQ90M4w3zXmnZ0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=CW7n4dWMzsK/gcSTDAG2nh1SSSvokNLBBGOcahNm0e2Ft7ytFHSJ19XlYOu8M4krEf yQsQ2oCfM7/J1A1hG8Kgqugtfg9KGhSDE2u0u2aXUdfogQu0RKMfcwluwDsQtVN9c2md nf/8aURqL4MWiae30Tq1tnlcFUF0TdjBwMoMw= MIME-Version: 1.0 Received: by 10.143.138.7 with SMTP id q7mr420029wfn.314.1258073698747; Thu, 12 Nov 2009 16:54:58 -0800 (PST) In-Reply-To: <4AD710D6.70404@buchlovice.org> References: <4AD710D6.70404@buchlovice.org> Date: Thu, 12 Nov 2009 16:54:58 -0800 Message-ID: From: Matt Reimer To: =?ISO-8859-2?Q?Radek_Val=E1=B9ek?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, rnoland@freebsd.org Subject: Re: GPT boot with ZFS RAIDZ "ZFS: i/o error - all block copies unavailable" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2009 01:27:04 -0000 2009/10/15 Radek Val=C3=A1=C5=A1ek : > Hi, > > I want to ask if there is something new in adding support to > gptzfsboot/zfsboot for reading gang-blocks? > > From Sun's docs: > > Gang blocks > > When there is not enough contiguous space to write a complete block, the = ZIO > pipeline will break the I/O up into smaller 'gang blocks' which can later= be > assembled transparently to appear as complete blocks. > > Everything works fine for me, until I rewrite kernel/world after system > upgrade to latest one (releng_8). After this am I no longer able to boot > from zfs raidz1 pool with following messages: > >>/ ZFS: i/o error - all block copies unavailable > />/ ZFS: can't read MOS > />/ ZFS: unexpected object set type lld > />/ ZFS: unexpected object set type lld > />/ > />/ FreeBSD/i386 boot > />/ Default: z:/boot/kernel/kernel > />/ boot: > />/ ZFS: unexpected object set type lld > />/ > />/ FreeBSD/i386 boot > />/ Default: tank:/boot/kernel/kernel > />/ boot: Radek, Try the attached patch (sponsored by VPOP Technologies). I found an overflow in /sys/cddl/boot/zfs/zfssubr.c:vdev_raidz_read() that was causing my 6x1TB raidz2 array to fail to boot. Apply the patch, build everything in /sys/boot, and then make sure you update both gptzfsboot and /boot/loader. Robert, I'm guessing you couldn't replicate this because your array was small enough not to result in block numbers overflowing an int. The kernel source for the corresponding functionality is in /sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c:vdev_raidz_map= _alloc(). There all these variables are uint64_t, but I think unnecessarily. I tried changing the boot loader's vdev_raidz_read() variables to all uint64_t but then gptzfsboot would reboot itself, likely due to a stack overflow. The attached patch just changes a few variables that, after a quick analysis, seemed likely to overflow. If this looks good, would someone commit it? Matt From owner-freebsd-fs@FreeBSD.ORG Fri Nov 13 04:53:17 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1B2A71065679; Fri, 13 Nov 2009 04:53:17 +0000 (UTC) (envelope-from rnoland@FreeBSD.org) Received: from gizmo.2hip.net (gizmo.2hip.net [64.74.207.195]) by mx1.freebsd.org (Postfix) with ESMTP id D55008FC17; Fri, 13 Nov 2009 04:53:16 +0000 (UTC) Received: from [192.168.1.4] (adsl-157-60-44.bna.bellsouth.net [70.157.60.44]) (authenticated bits=0) by gizmo.2hip.net (8.14.3/8.14.3) with ESMTP id nAD4r8ev029626 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Thu, 12 Nov 2009 23:53:10 -0500 (EST) (envelope-from rnoland@FreeBSD.org) From: Robert Noland To: Matt Reimer In-Reply-To: References: <4AD710D6.70404@buchlovice.org> Content-Type: text/plain; charset="iso-8859-2" Organization: FreeBSD Date: Thu, 12 Nov 2009 22:53:03 -0600 Message-Id: <1258087983.2303.23.camel@balrog.2hip.net> Mime-Version: 1.0 X-Mailer: Evolution 2.26.3 FreeBSD GNOME Team Port Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-2.1 required=5.0 tests=AWL,BAYES_00,RCVD_IN_PBL, RDNS_DYNAMIC,SPF_SOFTFAIL autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on gizmo.2hip.net Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: GPT boot with ZFS RAIDZ "ZFS: i/o error - all block copies unavailable" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2009 04:53:17 -0000 On Thu, 2009-11-12 at 16:54 -0800, Matt Reimer wrote: > 2009/10/15 Radek Valášek : > > Hi, > > > > I want to ask if there is something new in adding support to > > gptzfsboot/zfsboot for reading gang-blocks? > > > > From Sun's docs: > > > > Gang blocks > > > > When there is not enough contiguous space to write a complete block, the ZIO > > pipeline will break the I/O up into smaller 'gang blocks' which can later be > > assembled transparently to appear as complete blocks. > > > > Everything works fine for me, until I rewrite kernel/world after system > > upgrade to latest one (releng_8). After this am I no longer able to boot > > from zfs raidz1 pool with following messages: > > > >>/ ZFS: i/o error - all block copies unavailable > > />/ ZFS: can't read MOS > > />/ ZFS: unexpected object set type lld > > />/ ZFS: unexpected object set type lld > > />/ > > />/ FreeBSD/i386 boot > > />/ Default: z:/boot/kernel/kernel > > />/ boot: > > />/ ZFS: unexpected object set type lld > > />/ > > />/ FreeBSD/i386 boot > > />/ Default: tank:/boot/kernel/kernel > > />/ boot: > > Radek, > > Try the attached patch (sponsored by VPOP Technologies). I found an > overflow in /sys/cddl/boot/zfs/zfssubr.c:vdev_raidz_read() that was > causing my 6x1TB raidz2 array to fail to boot. > > Apply the patch, build everything in /sys/boot, and then make sure you > update both gptzfsboot and /boot/loader. > > Robert, I'm guessing you couldn't replicate this because your array > was small enough not to result in block numbers overflowing an int. This is likely, all of my raidz tests were with vnode backed 1GB memory disks. So my largest configuration was a 6 x 1GB raidz2. > The kernel source for the corresponding functionality is in > /sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c:vdev_raidz_map_alloc(). > There all these variables are uint64_t, but I think unnecessarily. I > tried changing the boot loader's vdev_raidz_read() variables to all > uint64_t but then gptzfsboot would reboot itself, likely due to a > stack overflow. The attached patch just changes a few variables that, > after a quick analysis, seemed likely to overflow. > > If this looks good, would someone commit it? ps@ grabbed it up already, but I may handle the MFC for him. I have some other minor fixups in my tree right now... like teaching printf to handle %llx. Thanks for finding this... It's been really frustrating that I couldn't produce a failing system. robert. > Matt > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" -- Robert Noland FreeBSD From owner-freebsd-fs@FreeBSD.ORG Fri Nov 13 06:15:08 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 94F47106566B; Fri, 13 Nov 2009 06:15:08 +0000 (UTC) (envelope-from mattjreimer@gmail.com) Received: from mail-gx0-f220.google.com (mail-gx0-f220.google.com [209.85.217.220]) by mx1.freebsd.org (Postfix) with ESMTP id 2594D8FC15; Fri, 13 Nov 2009 06:15:08 +0000 (UTC) Received: by gxk20 with SMTP id 20so3387526gxk.14 for ; Thu, 12 Nov 2009 22:15:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=vUTwUGgyFk04YUSe5pbleVBMzI0jYfPkGe1rT2lte9o=; b=JJAguO05nGs0JCkxivYP7LcOiDmD7tWyYZ7udPT9VxaT53WzoRF9SX/MUg/l/GvtGr MxLF1UpXTHJZHY/lFdGVYx3jWqCqnBBNdGE3pL0VmvVhMQ7FgQqBmnmQ1ho5XlCns0kJ HaC7brpRIqLVs5cIelI2edLGN/z0WKzNO+JzA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=l0JlIHPFaZ9TiKdm9O/KJYqe7O86BC4nYH6IWdqu0a9Ss8Lr/CywBYMKQcJFKTGvjf 7ZGrlgf6KRLakpPdXvTVKNqCzLhCaY7XDCzLjxuXOWF2SMyFGGnqWLdrF7mhZ1IAvbjN oKRA/kBilh+e4OMpU7vfzb3Py9wEPY3I62QkY= MIME-Version: 1.0 Received: by 10.150.233.2 with SMTP id f2mr7000213ybh.259.1258092907509; Thu, 12 Nov 2009 22:15:07 -0800 (PST) In-Reply-To: <1258087983.2303.23.camel@balrog.2hip.net> References: <4AD710D6.70404@buchlovice.org> <1258087983.2303.23.camel@balrog.2hip.net> Date: Thu, 12 Nov 2009 22:15:07 -0800 Message-ID: From: Matt Reimer To: Robert Noland Content-Type: text/plain; charset=ISO-8859-2 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: GPT boot with ZFS RAIDZ "ZFS: i/o error - all block copies unavailable" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2009 06:15:08 -0000 2009/11/12 Robert Noland : > On Thu, 2009-11-12 at 16:54 -0800, Matt Reimer wrote: >> 2009/10/15 Radek Val=E1=B9ek : >> > Everything works fine for me, until I rewrite kernel/world after syste= m >> > upgrade to latest one (releng_8). After this am I no longer able to bo= ot >> > from zfs raidz1 pool with following messages: >> > >> >>/ ZFS: i/o error - all block copies unavailable >> > />/ ZFS: can't read MOS >> > />/ ZFS: unexpected object set type lld >> > />/ ZFS: unexpected object set type lld >> > />/ >> > />/ FreeBSD/i386 boot >> > />/ Default: z:/boot/kernel/kernel >> > />/ boot: >> > />/ ZFS: unexpected object set type lld >> > />/ >> > />/ FreeBSD/i386 boot >> > />/ Default: tank:/boot/kernel/kernel >> > />/ boot: >> >> Radek, >> >> Try the attached patch (sponsored by VPOP Technologies). I found an >> overflow in /sys/cddl/boot/zfs/zfssubr.c:vdev_raidz_read() that was >> causing my 6x1TB raidz2 array to fail to boot. ... >> The kernel source for the corresponding functionality is in >> /sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c:vdev_raidz_= map_alloc(). >> There all these variables are uint64_t, but I think unnecessarily. I >> tried changing the boot loader's vdev_raidz_read() variables to all >> uint64_t but then gptzfsboot would reboot itself, likely due to a >> stack overflow. The attached patch just changes a few variables that, >> after a quick analysis, seemed likely to overflow. >> >> If this looks good, would someone commit it? > > ps@ grabbed it up already, but I may handle the MFC for him. =A0I have > some other minor fixups in my tree right now... like teaching printf to > handle %llx. =A0Thanks for finding this... It's been really frustrating > that I couldn't produce a failing system. Is it possible for this patch to get into 8.0-RELEASE, or is it too late? I suppose it doesn't matter that much since the loader isn't built with LOADER_ZFS_SUPPORT by default anyway, so folks are going to have to compile it themselves. Matt From owner-freebsd-fs@FreeBSD.ORG Fri Nov 13 09:28:52 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 572E7106566C for ; Fri, 13 Nov 2009 09:28:52 +0000 (UTC) (envelope-from gerrit@pmp.uni-hannover.de) Received: from mrelay1.uni-hannover.de (mrelay1.uni-hannover.de [130.75.2.106]) by mx1.freebsd.org (Postfix) with ESMTP id D8EB88FC0C for ; Fri, 13 Nov 2009 09:28:51 +0000 (UTC) Received: from www.pmp.uni-hannover.de (www.pmp.uni-hannover.de [130.75.117.2]) by mrelay1.uni-hannover.de (8.14.2/8.14.2) with ESMTP id nAD9Smwi031787; Fri, 13 Nov 2009 10:28:49 +0100 Received: from pmp.uni-hannover.de (theq.pmp.uni-hannover.de [130.75.117.4]) by www.pmp.uni-hannover.de (Postfix) with SMTP id A569F4F; Fri, 13 Nov 2009 10:28:48 +0100 (CET) Date: Fri, 13 Nov 2009 10:28:48 +0100 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: Rick Macklem Message-Id: <20091113102848.4a8298e8.gerrit@pmp.uni-hannover.de> In-Reply-To: References: <20091112182414.cebec1df.gerrit@pmp.uni-hannover.de> Organization: Albert-Einstein-Institut (MPI =?ISO-8859-1?Q?f=FCr?= Gravitationsphysik & IGP =?ISO-8859-1?Q?Universit=E4t?= Hannover) X-Mailer: Sylpheed 2.4.2 (GTK+ 2.10.12; i386-portbld-freebsd6.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-PMX-Version: 5.5.5.374460, Antispam-Engine: 2.7.1.369594, Antispam-Data: 2009.11.13.91539 Cc: freebsd-fs@freebsd.org Subject: Re: nfsv4 FreeBSD server vs. Linux client I/O error X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2009 09:28:52 -0000 On Thu, 12 Nov 2009 14:45:04 -0500 (EST) Rick Macklem wrote about Re: nfsv4 FreeBSD server vs. Linux client I/O error: RM> A few things to check on the server: RM> - Did you add a "V4:" line to your /etc/exports and what did you set as RM> the root path in it? If you used "V4: /" then the root file system RM> would need to be exported by another line in /etc/exports for it to RM> work. RM> RM> - If you are only exporting another filesystem, lets say "/exports", RM> then your mount command would have to look like: RM> mount -t nfs4 /exports /mnt RM> (assuming "V4: /" was used) I think I do not yet understand the last part. How do I restrict the export of... oh, I guess I see. If I put / in the exports list, this will merely only allow for the full path still being used on the client side, but I have still to add the file systems actually to be exported. I was wondering about the notes in the manpages about this. RM> - If you used "V4: /exports", then "mount -t nfs4 :/ /mnt" RM> would work and you would see /exports at /mnt. However, this is exactly the way I went. My exports line on my server (cliff) looks like this: V4: /tank -network 192.168.0.0 -mask 255.255.0.0 On the client I try to mount like this: --- pt-ws1 ~ # time mount -t nfs4 cliff:/ /mnt real 0m30.005s user 0m0.000s sys 0m0.002s --- As you see, there is a timeout of about 30s involved. After that the nfs appears to be there: pt-ws1 ~ # mount [...] nfsd (rw) cliff:/ on /mnt type nfs4 (rw,addr=192.168.33.96,clientaddr=192.168.32.3) But it cannot be accessed: pt-ws1 ~ # ls /mnt ls: cannot open directory /mnt: Input/output error Unmounting gives the same timeout, but afterwards the nfs mount is indeed unmounted: pt-ws1 ~ # time umount /mnt real 0m30.002s user 0m0.001s sys 0m0.001s RM> Beyond something like the above, if you capture packets using RM> "tcpdump -s 0 -w host " on the server and then RM> email me "", I can take a look at it. (tcpdump doesn't know RM> diddly about NFSv4, but wireshark does and can handle tcpdump RM> captures.) I will do that, thanks. cu Gerrit From owner-freebsd-fs@FreeBSD.ORG Fri Nov 13 09:44:29 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D170F106566C for ; Fri, 13 Nov 2009 09:44:29 +0000 (UTC) (envelope-from gerrit@pmp.uni-hannover.de) Received: from mrelay1.uni-hannover.de (mrelay1.uni-hannover.de [130.75.2.106]) by mx1.freebsd.org (Postfix) with ESMTP id 5F57B8FC13 for ; Fri, 13 Nov 2009 09:44:29 +0000 (UTC) Received: from www.pmp.uni-hannover.de (www.pmp.uni-hannover.de [130.75.117.2]) by mrelay1.uni-hannover.de (8.14.2/8.14.2) with ESMTP id nAD9iQ35032336; Fri, 13 Nov 2009 10:44:27 +0100 Received: from pmp.uni-hannover.de (theq.pmp.uni-hannover.de [130.75.117.4]) by www.pmp.uni-hannover.de (Postfix) with SMTP id D33814F; Fri, 13 Nov 2009 10:44:26 +0100 (CET) Date: Fri, 13 Nov 2009 10:44:26 +0100 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: Rick Macklem Message-Id: <20091113104426.e8e85871.gerrit@pmp.uni-hannover.de> In-Reply-To: References: <20091112182414.cebec1df.gerrit@pmp.uni-hannover.de> Organization: Albert-Einstein-Institut (MPI =?ISO-8859-1?Q?f=FCr?= Gravitationsphysik & IGP =?ISO-8859-1?Q?Universit=E4t?= Hannover) X-Mailer: Sylpheed 2.4.2 (GTK+ 2.10.12; i386-portbld-freebsd6.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-PMX-Version: 5.5.5.374460, Antispam-Engine: 2.7.1.369594, Antispam-Data: 2009.11.13.93327 Cc: freebsd-fs@freebsd.org Subject: Re: nfsv4 FreeBSD server vs. Linux client I/O error X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2009 09:44:29 -0000 On Thu, 12 Nov 2009 17:17:55 -0500 (EST) Rick Macklem wrote about Re: nfsv4 FreeBSD server vs. Linux client I/O error: RM> One more thing that came to mind. If your root fs in NFS mounted, it RM> can't be exported, so you have to use the version (assuming /exports RM> is a local on-disk file system that is exported): RM> V4: /export RM> and not RM> V4: / I guess this does not apply do my situation, both client and server have their root fs on hard disks. RM> and then the mount needs to look like: RM> mount -t nfs4 :/export /mnt I tried that, just to make sure: pt-ws1 ~ # time mount -t nfs4 cliff:/tank /mnt mount.nfs4: mounting cliff:/tank failed, reason given by server: No such file or directory real 1m0.006s user 0m0.002s sys 0m0.002s I guess this is consistent, although I do not know why I see 30s timeout interval twice here. RM> I tried a relatively recent Ubuntu client here and it seemed to mount RM> ok. (There are many variants of the mount utilities and patch versions RM> of the nfs4 client for Linux, so your mileage definitely may vary.) I'm using Sabayon 3.5 with kernel 2.6.25 here. I guess I could try and update to the new Sabayon 5 for testing (if that might help). The installed nfs packages are: net-libs/libnfsidmap-0.20 net-fs/nfs4-acl-tools-0.3.2 net-fs/nfs-utils-1.1.2 RM> I'll be happy to look at a tcpdump capture, if you get one, rick You should have received it by now. I'm looking forward to your answer. Thanks again. cu Gerrit From owner-freebsd-fs@FreeBSD.ORG Fri Nov 13 10:49:11 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 308DD1065697 for ; Fri, 13 Nov 2009 10:49:11 +0000 (UTC) (envelope-from gerrit@pmp.uni-hannover.de) Received: from mrelay1.uni-hannover.de (mrelay1.uni-hannover.de [130.75.2.106]) by mx1.freebsd.org (Postfix) with ESMTP id 97E648FC1D for ; Fri, 13 Nov 2009 10:49:10 +0000 (UTC) Received: from www.pmp.uni-hannover.de (www.pmp.uni-hannover.de [130.75.117.2]) by mrelay1.uni-hannover.de (8.14.2/8.14.2) with ESMTP id nADAn7fm002798; Fri, 13 Nov 2009 11:49:09 +0100 Received: from pmp.uni-hannover.de (theq.pmp.uni-hannover.de [130.75.117.4]) by www.pmp.uni-hannover.de (Postfix) with SMTP id 08C6724; Fri, 13 Nov 2009 11:49:08 +0100 (CET) Date: Fri, 13 Nov 2009 11:49:07 +0100 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: freebsd-fs@freebsd.org Message-Id: <20091113114907.cb04885e.gerrit@pmp.uni-hannover.de> In-Reply-To: <20091113104426.e8e85871.gerrit@pmp.uni-hannover.de> References: <20091112182414.cebec1df.gerrit@pmp.uni-hannover.de> <20091113104426.e8e85871.gerrit@pmp.uni-hannover.de> Organization: Albert-Einstein-Institut (MPI =?ISO-8859-1?Q?f=FCr?= Gravitationsphysik & IGP =?ISO-8859-1?Q?Universit=E4t?= Hannover) X-Mailer: Sylpheed 2.4.2 (GTK+ 2.10.12; i386-portbld-freebsd6.1) Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-PMX-Version: 5.5.5.374460, Antispam-Engine: 2.7.1.369594, Antispam-Data: 2009.11.13.103324 Cc: Subject: Re: nfsv4 FreeBSD server vs. Linux client I/O error X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2009 10:49:11 -0000 On Fri, 13 Nov 2009 10:44:26 +0100 Gerrit K=FChn wrote about Re: nfsv4 FreeBSD server vs. Linux client I/O error: GK> RM> I'll be happy to look at a tcpdump capture, if you get one, rick GK>=20 GK> You should have received it by now. I'm looking forward to your answer. GK> Thanks again. One more thing from my side: I guess I fixed the timeout problem by restarting all concerned services on the client side (namely rpc.idmapd and remounting rpc_pipefs). Now I get quick responses, but I still see the i/o error when trying to access the mounted volume. cu Gerrit From owner-freebsd-fs@FreeBSD.ORG Fri Nov 13 14:26:05 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 256D7106566C; Fri, 13 Nov 2009 14:26:04 +0000 (UTC) (envelope-from rnoland@FreeBSD.org) Received: from gizmo.2hip.net (gizmo.2hip.net [64.74.207.195]) by mx1.freebsd.org (Postfix) with ESMTP id 6FCD68FC14; Fri, 13 Nov 2009 14:26:04 +0000 (UTC) Received: from [192.168.1.4] (adsl-157-60-44.bna.bellsouth.net [70.157.60.44]) (authenticated bits=0) by gizmo.2hip.net (8.14.3/8.14.3) with ESMTP id nADEQ0MJ040245 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Fri, 13 Nov 2009 09:26:00 -0500 (EST) (envelope-from rnoland@FreeBSD.org) From: Robert Noland To: Matt Reimer In-Reply-To: References: <4AD710D6.70404@buchlovice.org> <1258087983.2303.23.camel@balrog.2hip.net> Content-Type: text/plain; charset="iso-8859-2" Organization: FreeBSD Date: Fri, 13 Nov 2009 08:25:54 -0600 Message-Id: <1258122354.2303.24.camel@balrog.2hip.net> Mime-Version: 1.0 X-Mailer: Evolution 2.26.3 FreeBSD GNOME Team Port Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-2.1 required=5.0 tests=AWL,BAYES_00,RCVD_IN_PBL, RDNS_DYNAMIC,SPF_SOFTFAIL autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on gizmo.2hip.net Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: GPT boot with ZFS RAIDZ "ZFS: i/o error - all block copies unavailable" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2009 14:26:05 -0000 On Thu, 2009-11-12 at 22:15 -0800, Matt Reimer wrote: > 2009/11/12 Robert Noland : > > On Thu, 2009-11-12 at 16:54 -0800, Matt Reimer wrote: > >> 2009/10/15 Radek Valášek : > >> > Everything works fine for me, until I rewrite kernel/world after system > >> > upgrade to latest one (releng_8). After this am I no longer able to boot > >> > from zfs raidz1 pool with following messages: > >> > > >> >>/ ZFS: i/o error - all block copies unavailable > >> > />/ ZFS: can't read MOS > >> > />/ ZFS: unexpected object set type lld > >> > />/ ZFS: unexpected object set type lld > >> > />/ > >> > />/ FreeBSD/i386 boot > >> > />/ Default: z:/boot/kernel/kernel > >> > />/ boot: > >> > />/ ZFS: unexpected object set type lld > >> > />/ > >> > />/ FreeBSD/i386 boot > >> > />/ Default: tank:/boot/kernel/kernel > >> > />/ boot: > >> > >> Radek, > >> > >> Try the attached patch (sponsored by VPOP Technologies). I found an > >> overflow in /sys/cddl/boot/zfs/zfssubr.c:vdev_raidz_read() that was > >> causing my 6x1TB raidz2 array to fail to boot. > ... > >> The kernel source for the corresponding functionality is in > >> /sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c:vdev_raidz_map_alloc(). > >> There all these variables are uint64_t, but I think unnecessarily. I > >> tried changing the boot loader's vdev_raidz_read() variables to all > >> uint64_t but then gptzfsboot would reboot itself, likely due to a > >> stack overflow. The attached patch just changes a few variables that, > >> after a quick analysis, seemed likely to overflow. > >> > >> If this looks good, would someone commit it? > > > > ps@ grabbed it up already, but I may handle the MFC for him. I have > > some other minor fixups in my tree right now... like teaching printf to > > handle %llx. Thanks for finding this... It's been really frustrating > > that I couldn't produce a failing system. > > Is it possible for this patch to get into 8.0-RELEASE, or is it too > late? I suppose it doesn't matter that much since the loader isn't > built with LOADER_ZFS_SUPPORT by default anyway, so folks are going to > have to compile it themselves. I think we have missed the boat, but I'll talk to re@ and see if we can get it in. robert. > Matt -- Robert Noland FreeBSD From owner-freebsd-fs@FreeBSD.ORG Fri Nov 13 20:49:01 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BBB34106566B for ; Fri, 13 Nov 2009 20:49:01 +0000 (UTC) (envelope-from se@freebsd.org) Received: from smtp126.mail.ukl.yahoo.com (smtp126.mail.ukl.yahoo.com [77.238.184.57]) by mx1.freebsd.org (Postfix) with SMTP id 15B5F8FC0A for ; Fri, 13 Nov 2009 20:49:00 +0000 (UTC) Received: (qmail 63617 invoked from network); 13 Nov 2009 20:22:19 -0000 Received: from xdsl-81-173-236-126.netcologne.de (se@81.173.236.126 with plain) by smtp126.mail.ukl.yahoo.com with SMTP; 13 Nov 2009 20:22:19 +0000 GMT X-Yahoo-SMTP: iDf2N9.swBDAhYEh7VHfpgq0lnq. X-YMail-OSG: q3RiWL4VM1nRdm9HcyhPfcr2FQTvf15Gwh69fIBgWU6UNUbQ5Sado_77DORai_gvWHnWPtV3ugR_Tb3jMu0XfSqV3buxJhCrgClC1Z8rg0icq7v9NjqKQoQ7ovjN55smVuIqfMGCfS5DkWe9mcly1LGN6wX6sFJ1B08BnD.lEczTl4MxzUk9pO4dUg3X_FB8vauTll_Kl0D2eLM7iaFbssxHnjsGk6TlpFvEfHuRlF_uPmnHtdH.EqDeni28ZtSH X-Yahoo-Newman-Property: ymail-3 Message-ID: <4AFDBFFB.3070009@freebsd.org> Date: Fri, 13 Nov 2009 21:22:19 +0100 From: Stefan Esser User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.4pre) Gecko/20090915 Lightning/1.0pre Thunderbird/3.0b4 MIME-Version: 1.0 To: Robert Noland References: <4AD710D6.70404@buchlovice.org> <1258087983.2303.23.camel@balrog.2hip.net> <1258122354.2303.24.camel@balrog.2hip.net> In-Reply-To: <1258122354.2303.24.camel@balrog.2hip.net> X-Enigmail-Version: 0.97b Content-Type: text/plain; charset=ISO-8859-2 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: GPT boot with ZFS RAIDZ "ZFS: i/o error - all block copies unavailable" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2009 20:49:01 -0000 On 13.11.2009 15:25, Robert Noland wrote: > I think we have missed the boat, but I'll talk to re@ and see if we > can get it in. The patch fixed GPT/ZFS booting for me, too. It would be good to have it in 8.0 (since it is definitely required to boot from ZFS pools with non-trivial sizes), and does not affect anybody not trying to boot this way. OTOH, it since you cannot just install FreeBSD on pure ZFS from sysinstall, it might be sufficient to prominently warn about this problem and point at the required patch, to prevent foot-shooting. But having this patch that has been successfully tested by a number of people that suffered from the GPT/ZFS boot problem looks highly preferable to me ... Regards, STefan From owner-freebsd-fs@FreeBSD.ORG Sat Nov 14 09:05:24 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 52C8E106566C; Sat, 14 Nov 2009 09:05:24 +0000 (UTC) (envelope-from valin@buchlovice.org) Received: from smtp-sfn.sitkom.cz (smtp-sfn.sitkom.cz [88.146.175.4]) by mx1.freebsd.org (Postfix) with ESMTP id 0DF2D8FC1A; Sat, 14 Nov 2009 09:05:23 +0000 (UTC) Received: from osiris.buchlovice.sfn (osiris.buchlovice.sfn [10.6.193.10]) by smtp-sfn.sitkom.cz (Postfix) with ESMTP id 70AFF1FA4A5; Sat, 14 Nov 2009 10:05:21 +0100 (CET) Message-ID: <4AFE72D1.5020902@buchlovice.org> Date: Sat, 14 Nov 2009 10:05:21 +0100 From: =?ISO-8859-2?Q?Radek_Val=E1=B9ek?= User-Agent: Thunderbird 2.0.0.23 (X11/20091015) MIME-Version: 1.0 To: Robert Noland References: <4AD710D6.70404@buchlovice.org> <1258087983.2303.23.camel@balrog.2hip.net> <1258122354.2303.24.camel@balrog.2hip.net> <4AFDBFFB.3070009@freebsd.org> In-Reply-To: <4AFDBFFB.3070009@freebsd.org> Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: GPT boot with ZFS RAIDZ "ZFS: i/o error - all block copies unavailable" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Nov 2009 09:05:24 -0000 Stefan Esser napsal(a): > On 13.11.2009 15:25, Robert Noland wrote: > >> I think we have missed the boat, but I'll talk to re@ and see if we >> can get it in. >> > > The patch fixed GPT/ZFS booting for me, too. It would be good to have > it in 8.0 (since it is definitely required to boot from ZFS pools with > non-trivial sizes), and does not affect anybody not trying to boot > this way. OTOH, it since you cannot just install FreeBSD on pure ZFS > from sysinstall, it might be sufficient to prominently warn about this > problem and point at the required patch, to prevent foot-shooting. > > But having this patch that has been successfully tested by a number > of people that suffered from the GPT/ZFS boot problem looks highly > preferable to me ... > > Regards, STefan > I can confirm that the patch is working for me too, and I'm able now to boot from raidz/raidz2 pool after rewriting loader.conf/loader/kernel. I agree with Stefan, having it in 8.0-RELEASE would be good, catch the boat :) Big thnx to Matt, great work from all. From owner-freebsd-fs@FreeBSD.ORG Sat Nov 14 09:46:41 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EA4A1106566C for ; Sat, 14 Nov 2009 09:46:41 +0000 (UTC) (envelope-from alteriks@gmail.com) Received: from mail-fx0-f227.google.com (mail-fx0-f227.google.com [209.85.220.227]) by mx1.freebsd.org (Postfix) with ESMTP id 7C8098FC12 for ; Sat, 14 Nov 2009 09:46:41 +0000 (UTC) Received: by fxm27 with SMTP id 27so4437204fxm.3 for ; Sat, 14 Nov 2009 01:46:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=L90OO1JE60Ep8TwCwBLlrGjJ/AzM3IcZOo5+HSkwYfQ=; b=iz3QRxSdVSekqsIlZhzlzR4JxNOPCYnunivCZWt+Bihg8bOhAdGssm+wJNcTeMOKhh f3VWezyaAhAyVnI23peCFeK+qLSED9bgYsQGYt+u+WftsSsmTCR1xlGSOsXC/aFbuxLH RkHLxSVj6woOHjvhc/QN12UXwQX/m8MGjcl9A= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=oLYhYGJf6gnen+bqaNM/ZnYxI7qFym9uFPkkRUlmF3KydlamkYj8ZDyW+tVsnGCOiG qf47pWumeks9AmSTNcazTETTEPare19wi245e3Ly82qSXWiXjFyhdTl2tO05djW//ct2 HD257S7O0Tya1yFNNItdmGV9lhWzfVSiIwKIo= MIME-Version: 1.0 Received: by 10.204.156.28 with SMTP id u28mr187656bkw.74.1258190153589; Sat, 14 Nov 2009 01:15:53 -0800 (PST) In-Reply-To: <4AFDBFFB.3070009@freebsd.org> References: <4AD710D6.70404@buchlovice.org> <1258087983.2303.23.camel@balrog.2hip.net> <1258122354.2303.24.camel@balrog.2hip.net> <4AFDBFFB.3070009@freebsd.org> Date: Sat, 14 Nov 2009 10:15:52 +0100 Message-ID: <684e57ec0911140115oa5d3c63xec7b2913847ce2c6@mail.gmail.com> From: Krzysztof Dajka To: Stefan Esser Content-Type: text/plain; charset=UTF-8 Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, Robert Noland Subject: Re: GPT boot with ZFS RAIDZ "ZFS: i/o error - all block copies unavailable" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Nov 2009 09:46:42 -0000 Thanks Matt for the patch. I used it with 8.0RC3 release. I installed FreeBSD under Linux (KVM) my 3x500GB drives were mounted as a scsi drives. Installation went smoothly but when I rebooted FreeBSD guest it hang as usual ;) with "ZFS: i/o error - all block copies unavailable", well it also spit out some LBA errors for the first time. I was a little disappointed, because I've been trying for three weeks to replace my Debian system with broken ext3 fs with FreeBSD on raidz. But I thought to myself I'll give it a try, and run FreeBSD native. To my suprise it welcomed me with login prompt. Once again thanks for the patch. It would be good idea to merge it with final release. On 11/13/09, Stefan Esser wrote: > On 13.11.2009 15:25, Robert Noland wrote: >> I think we have missed the boat, but I'll talk to re@ and see if we >> can get it in. > > The patch fixed GPT/ZFS booting for me, too. It would be good to have > it in 8.0 (since it is definitely required to boot from ZFS pools with > non-trivial sizes), and does not affect anybody not trying to boot > this way. OTOH, it since you cannot just install FreeBSD on pure ZFS > from sysinstall, it might be sufficient to prominently warn about this > problem and point at the required patch, to prevent foot-shooting. > > But having this patch that has been successfully tested by a number > of people that suffered from the GPT/ZFS boot problem looks highly > preferable to me ... > > Regards, STefan > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Sat Nov 14 16:40:27 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9AF63106566C for ; Sat, 14 Nov 2009 16:40:27 +0000 (UTC) (envelope-from lopez.on.the.lists@yellowspace.net) Received: from mail.yellowspace.net (mail.yellowspace.net [80.190.200.164]) by mx1.freebsd.org (Postfix) with ESMTP id 1C70E8FC0A for ; Sat, 14 Nov 2009 16:40:26 +0000 (UTC) Received: from [192.168.178.21] ([85.181.139.246]) (AUTH: CRAM-MD5 lopez.on.the.lists@yellowspace.net, TLS: TLSv1/SSLv3, 128bits, AES128-SHA) by mail.yellowspace.net with esmtp; Sat, 14 Nov 2009 17:30:20 +0100 id 0036A172.000000004AFEDB1C.0000BC5B From: Lorenzo Perone Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Date: Sat, 14 Nov 2009 17:30:09 +0100 Message-Id: <1A8F306A-8749-471B-94EA-FC8435A30C34@yellowspace.net> To: freebsd-fs@freebsd.org Mime-Version: 1.0 (Apple Message framework v1076) X-Mailer: Apple Mail (2.1076) Subject: gmirroring slices X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Nov 2009 16:40:27 -0000 Hello, I was wondering if anyone could give me an advice on how viable and reliable it is, to use gmirror on a slice of an MBR-style partitioned disk, and use the second slice(s) within a zpool. I remember a discussion here on where metadata is kept (always at the end of the disk as opposed to the end of the given consumer?), so I wasn't sure about how much of a good idea this might be. The reason I'd like to have it like this is, that I had mixed bad experiences in the effort of using ZFS as a boot and root volume, so I'd rather keep a traditional slice for booting/rooting, and a zpool for the production jails on that machine. The example would be provider: mirror/gm0 consumers: ad6s1 and ad8s1 zpool mirror made out of ad6s2 and ad8s2 while experimenting, I got into the problem that gmirror label -v -b round-robin gm0 ad6s1 got a permission denied (even with sysctl kern.geom.debugflags=16/17). Any hints on what can cause this (I might have screwed up something with fdisk/bsdlabel, but after doublechecking I wonder what it could be..) Is a GPT partition table better for this (I got further with another machine by using GPT partitions)? Thanx for any advice. Regards, Lorenzo From owner-freebsd-fs@FreeBSD.ORG Sat Nov 14 17:21:11 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1B3B91065692 for ; Sat, 14 Nov 2009 17:21:11 +0000 (UTC) (envelope-from hugo@barafranca.com) Received: from mail.barafranca.com (mail.barafranca.com [67.213.67.47]) by mx1.freebsd.org (Postfix) with ESMTP id E46348FC08 for ; Sat, 14 Nov 2009 17:21:10 +0000 (UTC) Received: from localhost (unknown [172.16.100.24]) by mail.barafranca.com (Postfix) with ESMTP id F1E359D7; Sat, 14 Nov 2009 17:04:34 +0000 (UTC) X-Virus-Scanned: amavisd-new at barafranca.com Received: from mail.barafranca.com ([172.16.100.24]) by localhost (mail.barafranca.com [172.16.100.24]) (amavisd-new, port 10024) with ESMTP id da5jDFIkedsd; Sat, 14 Nov 2009 17:03:53 +0000 (UTC) Received: from [192.168.10.4] (unknown [81.193.1.26]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.barafranca.com (Postfix) with ESMTPSA id 838BE9C6; Sat, 14 Nov 2009 17:03:52 +0000 (UTC) Message-ID: <4AFEE2F2.6000609@barafranca.com> Date: Sat, 14 Nov 2009 17:03:46 +0000 From: Hugo Silva User-Agent: Thunderbird 2.0.0.23 (X11/20091030) MIME-Version: 1.0 To: Lorenzo Perone References: <1A8F306A-8749-471B-94EA-FC8435A30C34@yellowspace.net> In-Reply-To: <1A8F306A-8749-471B-94EA-FC8435A30C34@yellowspace.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: gmirroring slices X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Nov 2009 17:21:11 -0000 Lorenzo Perone wrote: > > Hello, > > I was wondering if anyone could give me an advice on how viable and > reliable it is, to use gmirror on a slice of an MBR-style partitioned > disk, and use the second slice(s) within a zpool. > > I remember a discussion here on where metadata is kept (always at the > end of the disk as opposed to the end of the given consumer?), so I > wasn't sure about how much of a good idea this might be. The reason > I'd like to have it like this is, that I had mixed bad experiences in > the effort of using ZFS as a boot and root volume, so I'd rather keep > a traditional slice for booting/rooting, and a zpool for the > production jails on that machine. > > The example would be > > provider: mirror/gm0 > consumers: ad6s1 and ad8s1 > > zpool mirror made out of > ad6s2 and ad8s2 > > while experimenting, I got into the problem that gmirror label -v -b > round-robin gm0 ad6s1 got a permission denied (even with sysctl > kern.geom.debugflags=16/17). Any hints on what can cause this (I might > have screwed up something with fdisk/bsdlabel, but after > doublechecking I wonder what it could be..) > > Is a GPT partition table better for this (I got further with another > machine by using GPT partitions)? > > Thanx for any advice. > > Regards, > Lorenzo > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" I setup an 8.0-RC3 server the other day just like this: Name Status Components mirror/gm0s1 COMPLETE ad4s1 ad6s1 pool: storage state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM storage ONLINE 0 0 0 mirror ONLINE 0 0 0 ad4s2 ONLINE 0 0 0 ad6s2 ONLINE 0 0 0 errors: No known data errors I would like to try setting this up with gpart too, but I had to get this server running asap and I knew I could make it work like this. From owner-freebsd-fs@FreeBSD.ORG Sat Nov 14 18:44:35 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 953BC106566C; Sat, 14 Nov 2009 18:44:35 +0000 (UTC) (envelope-from rnoland@FreeBSD.org) Received: from gizmo.2hip.net (gizmo.2hip.net [64.74.207.195]) by mx1.freebsd.org (Postfix) with ESMTP id 3DDC08FC16; Sat, 14 Nov 2009 18:44:35 +0000 (UTC) Received: from [192.168.1.4] (adsl-157-60-44.bna.bellsouth.net [70.157.60.44]) (authenticated bits=0) by gizmo.2hip.net (8.14.3/8.14.3) with ESMTP id nAEIiQNf050281 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sat, 14 Nov 2009 13:44:28 -0500 (EST) (envelope-from rnoland@FreeBSD.org) From: Robert Noland To: Krzysztof Dajka In-Reply-To: <684e57ec0911140115oa5d3c63xec7b2913847ce2c6@mail.gmail.com> References: <4AD710D6.70404@buchlovice.org> <1258087983.2303.23.camel@balrog.2hip.net> <1258122354.2303.24.camel@balrog.2hip.net> <4AFDBFFB.3070009@freebsd.org> <684e57ec0911140115oa5d3c63xec7b2913847ce2c6@mail.gmail.com> Content-Type: text/plain Organization: FreeBSD Date: Sat, 14 Nov 2009 12:44:21 -0600 Message-Id: <1258224261.2303.31.camel@balrog.2hip.net> Mime-Version: 1.0 X-Mailer: Evolution 2.26.3 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-2.1 required=5.0 tests=AWL,BAYES_00,RCVD_IN_PBL, RDNS_DYNAMIC,SPF_SOFTFAIL autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on gizmo.2hip.net Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: GPT boot with ZFS RAIDZ "ZFS: i/o error - all block copies unavailable" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Nov 2009 18:44:35 -0000 On Sat, 2009-11-14 at 10:15 +0100, Krzysztof Dajka wrote: > Thanks Matt for the patch. I used it with 8.0RC3 release. I installed > FreeBSD under Linux (KVM) my 3x500GB drives were mounted as a scsi > drives. Installation went smoothly but when I rebooted FreeBSD guest > it hang as usual ;) with "ZFS: i/o error - all block copies > unavailable", well it also spit out some LBA errors for the first > time. I was a little disappointed, because I've been trying for three > weeks to replace my Debian system with broken ext3 fs with FreeBSD on > raidz. But I thought to myself I'll give it a try, and run FreeBSD > native. To my suprise it welcomed me with login prompt. Once again > thanks for the patch. It would be good idea to merge it with final > release. This was approved by re@ and has been merged to the release branch. It should be included in 8.0-RELEASE. robert. > On 11/13/09, Stefan Esser wrote: > > On 13.11.2009 15:25, Robert Noland wrote: > >> I think we have missed the boat, but I'll talk to re@ and see if we > >> can get it in. > > > > The patch fixed GPT/ZFS booting for me, too. It would be good to have > > it in 8.0 (since it is definitely required to boot from ZFS pools with > > non-trivial sizes), and does not affect anybody not trying to boot > > this way. OTOH, it since you cannot just install FreeBSD on pure ZFS > > from sysinstall, it might be sufficient to prominently warn about this > > problem and point at the required patch, to prevent foot-shooting. > > > > But having this patch that has been successfully tested by a number > > of people that suffered from the GPT/ZFS boot problem looks highly > > preferable to me ... > > > > Regards, STefan > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > > -- Robert Noland FreeBSD From owner-freebsd-fs@FreeBSD.ORG Sat Nov 14 20:03:37 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E24D81065672; Sat, 14 Nov 2009 20:03:37 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from koef.zs64.net (koef.zs64.net [212.12.50.230]) by mx1.freebsd.org (Postfix) with ESMTP id 7B8B88FC0A; Sat, 14 Nov 2009 20:03:37 +0000 (UTC) Received: from localhost by koef.zs64.net (8.14.3/8.14.3) with ESMTP id nAEK3Z99080637 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sat, 14 Nov 2009 21:03:35 +0100 (CET) (envelope-from stb@lassitu.de) (authenticated as stb) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Stefan Bethke In-Reply-To: Date: Sat, 14 Nov 2009 21:03:34 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <97D7A06D-98EC-4702-9E3D-A7B85DB39A20@lassitu.de> References: <4AD710D6.70404@buchlovice.org> To: Matt Reimer X-Mailer: Apple Mail (2.1077) Cc: freebsd-fs@freebsd.org, FreeBSD current , rnoland@freebsd.org Subject: Re: GPT boot with ZFS RAIDZ "ZFS: i/o error - all block copies unavailable" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Nov 2009 20:03:38 -0000 Am 13.11.2009 um 01:54 schrieb Matt Reimer: > Try the attached patch (sponsored by VPOP Technologies). I found an > overflow in /sys/cddl/boot/zfs/zfssubr.c:vdev_raidz_read() that was > causing my 6x1TB raidz2 array to fail to boot. I can confirm as well that the patch (as committed to -current as = r199241) makes my loader happy. Now I just need to figure out why the = kernel won't mount root... Thanks, Stefan --=20 Stefan Bethke Fon +49 151 14070811 From owner-freebsd-fs@FreeBSD.ORG Sat Nov 14 20:36:22 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 97AAE106566C; Sat, 14 Nov 2009 20:36:22 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from koef.zs64.net (koef.zs64.net [212.12.50.230]) by mx1.freebsd.org (Postfix) with ESMTP id 1BCC98FC17; Sat, 14 Nov 2009 20:36:21 +0000 (UTC) Received: from localhost by koef.zs64.net (8.14.3/8.14.3) with ESMTP id nAEKaJne085317 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sat, 14 Nov 2009 21:36:20 +0100 (CET) (envelope-from stb@lassitu.de) (authenticated as stb) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Apple Message framework v1077) From: Stefan Bethke In-Reply-To: <97D7A06D-98EC-4702-9E3D-A7B85DB39A20@lassitu.de> Date: Sat, 14 Nov 2009 21:36:18 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <45EBBE61-7D43-4853-AC86-2FD42334808D@lassitu.de> References: <4AD710D6.70404@buchlovice.org> <97D7A06D-98EC-4702-9E3D-A7B85DB39A20@lassitu.de> To: freebsd-fs@freebsd.org, FreeBSD current X-Mailer: Apple Mail (2.1077) Cc: Subject: Re: GPT boot with ZFS RAIDZ "ZFS: i/o error - all block copies unavailable" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Nov 2009 20:36:22 -0000 Am 14.11.2009 um 21:03 schrieb Stefan Bethke: > Am 13.11.2009 um 01:54 schrieb Matt Reimer: >=20 >> Try the attached patch (sponsored by VPOP Technologies). I found an >> overflow in /sys/cddl/boot/zfs/zfssubr.c:vdev_raidz_read() that was >> causing my 6x1TB raidz2 array to fail to boot. >=20 > I can confirm as well that the patch (as committed to -current as = r199241) makes my loader happy. Now I just need to figure out why the = kernel won't mount root... I was trying to boot off a raw ZFS pool. When using GPT partitions, it = works just fine. Stefan --=20 Stefan Bethke Fon +49 151 14070811 From owner-freebsd-fs@FreeBSD.ORG Sat Nov 14 20:40:18 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BA1EB1065679 for ; Sat, 14 Nov 2009 20:40:18 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) by mx1.freebsd.org (Postfix) with ESMTP id 774368FC1D for ; Sat, 14 Nov 2009 20:40:18 +0000 (UTC) Received: from localhost (localhost.codelab.cz [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 15C3919E023; Sat, 14 Nov 2009 21:40:17 +0100 (CET) Received: from [192.168.1.2] (r5bb235.net.upc.cz [86.49.61.235]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 6401419E019; Sat, 14 Nov 2009 21:40:14 +0100 (CET) Message-ID: <4AFF15AE.4070902@quip.cz> Date: Sat, 14 Nov 2009 21:40:14 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.4) Gecko/20091017 SeaMonkey/2.0 MIME-Version: 1.0 To: Lorenzo Perone References: <1A8F306A-8749-471B-94EA-FC8435A30C34@yellowspace.net> In-Reply-To: <1A8F306A-8749-471B-94EA-FC8435A30C34@yellowspace.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: gmirroring slices X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Nov 2009 20:40:18 -0000 Lorenzo Perone wrote: > > Hello, > > I was wondering if anyone could give me an advice on how viable and > reliable it is, to use gmirror on a slice of an MBR-style partitioned > disk, and use the second slice(s) within a zpool. > > I remember a discussion here on where metadata is kept (always at the > end of the disk as opposed to the end of the given consumer?), so I > wasn't sure about how much of a good idea this might be. I think metadata is stored at the end of the provider (slice in this case), but I am not a GEOM expert. > The reason I'd > like to have it like this is, that I had mixed bad experiences in the > effort of using ZFS as a boot and root volume, so I'd rather keep a > traditional slice for booting/rooting, and a zpool for the production > jails on that machine. > > The example would be > > provider: mirror/gm0 > consumers: ad6s1 and ad8s1 > > zpool mirror made out of > ad6s2 and ad8s2 I am running following setup for year without any configuration problems # gmirror status Name Status Components mirror/gms1 COMPLETE ad4s1 ad6s1 # zpool status pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 mirror ONLINE 0 0 0 ad4s2 ONLINE 0 0 0 ad6s2 ONLINE 0 0 0 The first slice is 20GB partitioned as usual: # mount -t ufs /dev/mirror/gms1a on / (ufs, local) /dev/mirror/gms1e on /usr (ufs, local, soft-updates) /dev/mirror/gms1d on /var (ufs, local, nosuid, soft-updates) /dev/mirror/gms1f on /tmp (ufs, local, noexec, nosuid, soft-updates) The rest (450GB) is used in ZFS mirrored zpool for jails (each jail has its own filesystem) > while experimenting, I got into the problem that gmirror label -v -b > round-robin gm0 ad6s1 got a permission denied (even with sysctl > kern.geom.debugflags=16/17). Any hints on what can cause this (I might > have screwed up something with fdisk/bsdlabel, but after doublechecking > I wonder what it could be..) I did it in non-standard way - converting already installed system on one disk to mirrored. So when I was in system running off ad6 I created two slices on ad4, setup gmirror gms1 from first slice of ad4, create partitions, newfs, mount it and transfer files from running system by dump & restore, edit fstab. Then I rebooted system from gms1, destroy content of ad6, create slices on ad6 and insert first slice in to gms1. After this I had ad4s2 and ad6s2 ready for zpool. All was done remotely through ssh. Miroslav Lachman