From owner-freebsd-fs@FreeBSD.ORG Sun Dec 11 16:37:34 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0B3B8106564A for ; Sun, 11 Dec 2011 16:37:34 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id C13918FC13 for ; Sun, 11 Dec 2011 16:37:33 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ap4EAIzb5E6DaFvO/2dsb2JhbABDFoRxpm+BdyWBCwINGQKJApUzjgKQUoE0gkSEW4IEgRYEiDGMQJJF X-IronPort-AV: E=Sophos;i="4.71,335,1320642000"; d="scan'208";a="148050947" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 11 Dec 2011 11:37:32 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id BE593B3F40 for ; Sun, 11 Dec 2011 11:37:32 -0500 (EST) Date: Sun, 11 Dec 2011 11:37:32 -0500 (EST) From: Rick Macklem To: freebsd-fs@freebsd.org Message-ID: <1388258959.33468.1323621452764.JavaMail.root@erie.cs.uoguelph.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Subject: NFSv4.1 status X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2011 16:37:34 -0000 Hi, I thought I'd post to let anyone who is interested know what the status of NFSv4.1 is. First off, the slides here give you an overview of where the vendors are: http://people.redhat.com/steved/Lisa-11/LISA-11-pNFS-BoF-final.pdf You'll notice that FreeBSD isn't mentioned, which is fair, since no NFSv4.1 support is in FreeBSD9.0. However, I now have the basic NFSv4.1 client support written and lightly tested. For releng9: http://people.freebsd.org/~rmacklem/nfsv4.1-client/nfsv4.1-releng9.patch For head/-current, there is an up-to-date source tree at: http://svnweb.freebsd.org/base/projects/nfsv4.1-client/sys/ It does not include any pNFS support at this point, but I'm am just starting to work on that and hopefully will have something for the file layout soon and hope to get to test it against various servers at the NFSv4 Bakeathon in June. rick ps: There has been no work done yet on a FreeBSD NFSv4.1 server as far as I know. From owner-freebsd-fs@FreeBSD.ORG Mon Dec 12 02:43:14 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 40ECC1065672 for ; Mon, 12 Dec 2011 02:43:14 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id 0166D8FC12 for ; Mon, 12 Dec 2011 02:43:13 +0000 (UTC) Received: from julian-mac.elischer.org (c-67-180-24-15.hsd1.ca.comcast.net [67.180.24.15]) (authenticated bits=0) by vps1.elischer.org (8.14.4/8.14.4) with ESMTP id pBC2Hif9032370 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sun, 11 Dec 2011 18:17:46 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <4EE5645C.9040401@freebsd.org> Date: Sun, 11 Dec 2011 18:18:04 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.24) Gecko/20111103 Thunderbird/3.1.16 MIME-Version: 1.0 To: Rick Macklem References: <1388258959.33468.1323621452764.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <1388258959.33468.1323621452764.JavaMail.root@erie.cs.uoguelph.ca> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: NFSv4.1 status X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 02:43:14 -0000 On 12/11/11 8:37 AM, Rick Macklem wrote: > Hi, > > I thought I'd post to let anyone who is interested know what the > status of NFSv4.1 is. First off, the slides here give you an > overview of where the vendors are: > http://people.redhat.com/steved/Lisa-11/LISA-11-pNFS-BoF-final.pdf > > You'll notice that FreeBSD isn't mentioned, which is fair, since no > NFSv4.1 support is in FreeBSD9.0. However, I now have the basic NFSv4.1 > client support written and lightly tested but Panassus and Netapp were.. have you talked to them about interactions between your code and theirs when the time comes to merge? > For releng9: > http://people.freebsd.org/~rmacklem/nfsv4.1-client/nfsv4.1-releng9.patch > > For head/-current, there is an up-to-date source tree at: > http://svnweb.freebsd.org/base/projects/nfsv4.1-client/sys/ > > It does not include any pNFS support at this point, but I'm am just starting > to work on that and hopefully will have something for the file layout soon > and hope to get to test it against various servers at the NFSv4 Bakeathon in June. > > rick > ps: There has been no work done yet on a FreeBSD NFSv4.1 server as far as I know. Pannassus (how DO you spell that?) and netapp must have that part worked out. > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Mon Dec 12 07:55:06 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4900F106567A; Mon, 12 Dec 2011 07:55:06 +0000 (UTC) (envelope-from c.kworr@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 26F278FC18; Mon, 12 Dec 2011 07:55:04 +0000 (UTC) Received: by bkbzv15 with SMTP id zv15so6467314bkb.13 for ; Sun, 11 Dec 2011 23:55:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=ZX31i1nPbSNA0sJwnHXugw8Uo1q4wFUwz6qUMCjrrgY=; b=Ag4wyVetM8HqLIdtYvWI9zZzh3pqTOLfDjymxFmCRttzadoi4Eo/h5CGyCMoYnjj80 rMHjkJRLJLWuwQTjyA+Bh2n4r50nuwCjKrfP6Iir8JYGvgTWCMOaqDTuqbekQJOQ2WI3 Cc1w1T0c3PG0JvbFxTn/ufcQr7VtTk+po5kSI= Received: by 10.204.156.12 with SMTP id u12mr9447960bkw.8.1323676504113; Sun, 11 Dec 2011 23:55:04 -0800 (PST) Received: from green.tandem.local (utwig.xim.bz. [91.216.237.46]) by mx.google.com with ESMTPS id h7sm29631973bkw.12.2011.12.11.23.55.00 (version=SSLv3 cipher=OTHER); Sun, 11 Dec 2011 23:55:01 -0800 (PST) Message-ID: <4EE5B353.4080505@gmail.com> Date: Mon, 12 Dec 2011 09:54:59 +0200 From: Volodymyr Kostyrko User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111111 Thunderbird/8.0 MIME-Version: 1.0 To: Robert Millan References: <20111208134307.GA5266@thorin> <4EE221F9.9010505@gmail.com> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, Adrian Chadd Subject: Re: [PATCH] Wipe other file systems when creating new UFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 07:55:06 -0000 10.12.2011 18:46, Robert Millan wrote: > 2011/12/9 Volodymyr Kostyrko: >> >> Shouldn't it make us a coffee then? Making precautions against >> shooting-the-foot is a wrong choice for me. This way we need to extend it to >> support any other file system that can repair thyself upon mount in case >> user mistakenly mounted the wrong one. >> > > This suggests you don't agree with the premise that file systems need > to be identifiable. We could argue about this, but note that existing > behaviour is entirely consistent with this premise: > > - newfs stores an UFSv2 signature on disk > - fsck checks for this signature and refuses to operate > - newfs checks for UFSv1 signature and makes sure it is cleared out > before creating a new UFSv2 > - the kernel checks for UFS signature before attempting to mount a file system And how about ZFS? 1. You create zfs pool on the disk. 2. When the pool is created it will be automounted each time. You can't format disk with active pool. 3. If you do `export` the pool or move the disk to another computer `zpool import` will only report about found pool doing nothing. You need explicitly specify pool ident or pool name to import it. 4. If you `destroy` the pool it will not automount in any case. The most possible way of running into what your patch tries to solve is: - create ZFS pool on one machine; - don't export it, just move the drive to another machine; - on the other machine format the drive as UFS2; - accidentally attach the drive to the first machine. It's not easy to trigger this one as with UFS. The way I would like this patch would be tasting the drive before formatting and refusing to format when any known filesystem is found. I know it's very far from the current codebase, but this feels like a right way to me. >> You messing up ZFS terminology a bit. Uberblock size is 1k and what you >> refer is ZFS vdev label. Size of vdev label is 256k. There are four vdev >> labels for each vdev - two at the beginning of the vdev and two at the end. > > Ah yes, I probably confused the concepts, I was working from memory > and it's been a while since I read the spec. > >> All vdev labels are _aligned_ to 256k. So technically this is not just last >> 512k. > > I see that you mean. I guess just aligning down "mediasize" to 256 > kiB before using it would suffice? Yep I think will do. -- Sphinx of black quartz judge my vow. From owner-freebsd-fs@FreeBSD.ORG Mon Dec 12 08:52:18 2011 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AE4B6106564A; Mon, 12 Dec 2011 08:52:18 +0000 (UTC) (envelope-from Karli.Sjoberg@slu.se) Received: from Edge1-3.slu.se (edge1-3.slu.se [193.10.100.98]) by mx1.freebsd.org (Postfix) with ESMTP id 5F01A8FC12; Mon, 12 Dec 2011 08:52:16 +0000 (UTC) Received: from Exchange2.ad.slu.se (193.10.100.95) by Edge1-3.slu.se (193.10.100.98) with Microsoft SMTP Server (TLS) id 8.3.213.0; Mon, 12 Dec 2011 09:47:18 +0100 Received: from exmbx3.ad.slu.se ([193.10.100.93]) by Exchange2.ad.slu.se ([193.10.100.95]) with mapi; Mon, 12 Dec 2011 09:46:44 +0100 From: =?iso-8859-1?Q?Karli_Sj=F6berg?= To: "fs@freebsd.org" , Jeremy Chadwick , "Kenneth D. Merry" Date: Mon, 12 Dec 2011 09:46:42 +0100 Thread-Topic: Consistant panics trying to access zfs filesystems replicated from Sun/Oracle appliance Thread-Index: Acy4qpM4hZuUnBeqTeKAiHqcH/fH0w== Message-ID: <73B607D0-8C3E-4980-B901-B986E060D32E@slu.se> References: <82B38DBF-DD3A-46CD-93F6-02CDB6506E05@slu.se> Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: sv-SE, en-US MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Consistant panics trying to access zfs filesystems replicated from Sun/Oracle appliance X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 08:52:18 -0000 Hi, the zfs send/recv is now complete and instead facing a more severe problem.= If I have any of the replicated filesystems mounted and try to go anywhere= near them somehow, the systems panics, and it=B4s the exact same panic as = before. Any attempt to cd, ls, whatever and it panics immediately. The syst= em can be stable and up for weeks with all filesystems on zfs, including ro= ot, but as soon as I mount any of replicated filesystems and try to access = them, blam! Any suggestions? Best Regards Karli Sj=F6berg ---------------------------------------------------------- Hi, I=B4m in the process of vacating a Sun/Oracle system to a another Supermicr= o/FreeBSD system, doing zfs send/recv between. Two times now, the system ha= s panicked while not doing anything at all, and it=B4s throwing alot of SCS= I/CAM-related errors while doing IO-intensive operations, like send/recv, r= esilver, and zpool has sometimes reported read/write errors on the hard dri= ves. Best part is that the errors in messages are about all hard drives at = one time or another, and they are connected with separate cables, controlle= rs and caddies. Specs: HW: 1x Supermicro X8SIL-F 2x Supermicro AOC-USAS2-L8i 2x Supermicro CSE-M35T-1B 1x Intel Core i5 650 3,2GHz 4x 2GB 1333MHZ DDR3 ECC UDIMM 10x SAMSUNG HD204UI (in a raidz2 zpool) 1x OCZ Vertex 3 240GB (L2ARC) SW: # uname -a FreeBSD server 8.2-STABLE FreeBSD 8.2-STABLE #0: Mon Oct 10 09:12:25 UTC 20= 11 root@server:/usr/obj/usr/src/sys/GENERIC amd64 # zpool get version pool1 NAME PROPERTY VALUE SOURCE pool1 version 28 default[/CODE] I got the panic from the IPMI KVM: http://i55.tinypic.com/synpzk.png From owner-freebsd-fs@FreeBSD.ORG Mon Dec 12 11:07:22 2011 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B87C21065675 for ; Mon, 12 Dec 2011 11:07:22 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id A5ADD8FC0A for ; Mon, 12 Dec 2011 11:07:22 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id pBCB7MR6029999 for ; Mon, 12 Dec 2011 11:07:22 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id pBCB7MFo029996 for freebsd-fs@FreeBSD.org; Mon, 12 Dec 2011 11:07:22 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 12 Dec 2011 11:07:22 GMT Message-Id: <201112121107.pBCB7MFo029996@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 11:07:22 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/162944 fs [coda] Coda file system module looks broken in 9.0 o kern/162860 fs [zfs] Cannot share ZFS filesystem to hosts with a hyph o kern/162751 fs [zfs] [panic] kernel panics during file operations o kern/162591 fs [nullfs] cross-filesystem nullfs does not work as expe o kern/162564 fs [ext2fs][patch] fs/ext2fs: Add include guard o kern/162519 fs [zfs] "zpool import" relies on buggy realpath() behavi o kern/162362 fs [snapshots] [panic] ufs with snapshot(s) panics when g o kern/162083 fs [zfs] [panic] zfs unmount -f pool o kern/161968 fs [zfs] [hang] renaming snapshot with -r including a zvo o kern/161897 fs [zfs] [patch] zfs partition probing causing long delay o kern/161864 fs [ufs] removing journaling from UFS partition fails on o bin/161807 fs [patch] add option for explicitly specifying metadata o kern/161674 fs [ufs] snapshot on journaled ufs doesn't work o kern/161579 fs [smbfs] FreeBSD sometimes panics when an smb share is o kern/161533 fs [zfs] [panic] zfs receive panic: system ioctl returnin o kern/161511 fs [unionfs] Filesystem deadlocks when using multiple uni o kern/161438 fs [zfs] [panic] recursed on non-recursive spa_namespace_ o kern/161424 fs [nullfs] __getcwd() calls fail when used on nullfs mou o kern/161280 fs [zfs] Stack overflow in gptzfsboot o kern/161205 fs [nfs] [pfsync] [regression] [build] Bug report freebsd o kern/161169 fs [zfs] [panic] ZFS causes kernel panic in dbuf_dirty o kern/161112 fs [ufs] [lor] filesystem LOR in FreeBSD 9.0-BETA3 o kern/160893 fs [zfs] [panic] 9.0-BETA2 kernel panic o kern/160860 fs Random UFS root filesystem corruption with SU+J [regre o kern/160801 fs [zfs] zfsboot on 8.2-RELEASE fails to boot from root-o o kern/160790 fs [fusefs] [panic] VPUTX: negative ref count with FUSE o kern/160777 fs [zfs] [hang] RAID-Z3 causes fatal hang upon scrub/impo o kern/160706 fs [zfs] zfs bootloader fails when a non-root vdev exists o kern/160591 fs [zfs] Fail to boot on zfs root with degraded raidz2 [r o kern/160410 fs [smbfs] [hang] smbfs hangs when transferring large fil o kern/160283 fs [zfs] [patch] 'zfs list' does abort in make_dataset_ha o kern/159971 fs [ffs] [panic] panic with soft updates journaling durin o kern/159930 fs [ufs] [panic] kernel core o kern/159402 fs [zfs][loader] symlinks cause I/O errors o kern/159357 fs [zfs] ZFS MAXNAMELEN macro has confusing name (off-by- o kern/159356 fs [zfs] [patch] ZFS NAME_ERR_DISKLIKE check is Solaris-s o kern/159351 fs [nfs] [patch] - divide by zero in mountnfs() o kern/159251 fs [zfs] [request]: add FLETCHER4 as DEDUP hash option o kern/159233 fs [ext2fs] [patch] fs/ext2fs: finish reallocblk implemen o kern/159232 fs [ext2fs] [patch] fs/ext2fs: merge ext2_readwrite into o kern/159077 fs [zfs] Can't cd .. with latest zfs version o kern/159048 fs [smbfs] smb mount corrupts large files o kern/159045 fs [zfs] [hang] ZFS scrub freezes system o kern/158839 fs [zfs] ZFS Bootloader Fails if there is a Dead Disk o kern/158802 fs amd(8) ICMP storm and unkillable process. o kern/158711 fs [ffs] [panic] panic in ffs_blkfree and ffs_valloc o kern/158231 fs [nullfs] panic on unmounting nullfs mounted over ufs o f kern/157929 fs [nfs] NFS slow read o kern/157722 fs [geli] unable to newfs a geli encrypted partition o kern/157399 fs [zfs] trouble with: mdconfig force delete && zfs strip o kern/157179 fs [zfs] zfs/dbuf.c: panic: solaris assert: arc_buf_remov o kern/156797 fs [zfs] [panic] Double panic with FreeBSD 9-CURRENT and o kern/156781 fs [zfs] zfs is losing the snapshot directory, p kern/156545 fs [ufs] mv could break UFS on SMP systems o kern/156193 fs [ufs] [hang] UFS snapshot hangs && deadlocks processes o kern/156039 fs [nullfs] [unionfs] nullfs + unionfs do not compose, re o kern/155615 fs [zfs] zfs v28 broken on sparc64 -current o kern/155587 fs [zfs] [panic] kernel panic with zfs f kern/155411 fs [regression] [8.2-release] [tmpfs]: mount: tmpfs : No o kern/155199 fs [ext2fs] ext3fs mounted as ext2fs gives I/O errors o bin/155104 fs [zfs][patch] use /dev prefix by default when importing o kern/154930 fs [zfs] cannot delete/unlink file from full volume -> EN o kern/154828 fs [msdosfs] Unable to create directories on external USB o kern/154491 fs [smbfs] smb_co_lock: recursive lock for object 1 p kern/154228 fs [md] md getting stuck in wdrain state o kern/153996 fs [zfs] zfs root mount error while kernel is not located o kern/153847 fs [nfs] [panic] Kernel panic from incorrect m_free in nf o kern/153753 fs [zfs] ZFS v15 - grammatical error when attempting to u o kern/153716 fs [zfs] zpool scrub time remaining is incorrect o kern/153695 fs [patch] [zfs] Booting from zpool created on 4k-sector o kern/153680 fs [xfs] 8.1 failing to mount XFS partitions o kern/153520 fs [zfs] Boot from GPT ZFS root on HP BL460c G1 unstable o kern/153418 fs [zfs] [panic] Kernel Panic occurred writing to zfs vol o kern/153351 fs [zfs] locking directories/files in ZFS o bin/153258 fs [patch][zfs] creating ZVOLs requires `refreservation' s kern/153173 fs [zfs] booting from a gzip-compressed dataset doesn't w o kern/153126 fs [zfs] vdev failure, zpool=peegel type=vdev.too_small o kern/152022 fs [nfs] nfs service hangs with linux client [regression] o kern/151942 fs [zfs] panic during ls(1) zfs snapshot directory o kern/151905 fs [zfs] page fault under load in /sbin/zfs o bin/151713 fs [patch] Bug in growfs(8) with respect to 32-bit overfl o kern/151648 fs [zfs] disk wait bug o kern/151629 fs [fs] [patch] Skip empty directory entries during name o kern/151330 fs [zfs] will unshare all zfs filesystem after execute a o kern/151326 fs [nfs] nfs exports fail if netgroups contain duplicate o kern/151251 fs [ufs] Can not create files on filesystem with heavy us o kern/151226 fs [zfs] can't delete zfs snapshot o kern/151111 fs [zfs] vnodes leakage during zfs unmount o kern/150503 fs [zfs] ZFS disks are UNAVAIL and corrupted after reboot o kern/150501 fs [zfs] ZFS vdev failure vdev.bad_label on amd64 o kern/150390 fs [zfs] zfs deadlock when arcmsr reports drive faulted o kern/150336 fs [nfs] mountd/nfsd became confused; refused to reload n o kern/149208 fs mksnap_ffs(8) hang/deadlock o kern/149173 fs [patch] [zfs] make OpenSolaris installa o kern/149015 fs [zfs] [patch] misc fixes for ZFS code to build on Glib o kern/149014 fs [zfs] [patch] declarations in ZFS libraries/utilities o kern/149013 fs [zfs] [patch] make ZFS makefiles use the libraries fro o kern/148504 fs [zfs] ZFS' zpool does not allow replacing drives to be o kern/148490 fs [zfs]: zpool attach - resilver bidirectionally, and re o kern/148368 fs [zfs] ZFS hanging forever on 8.1-PRERELEASE o kern/148138 fs [zfs] zfs raidz pool commands freeze o kern/147903 fs [zfs] [panic] Kernel panics on faulty zfs device o kern/147881 fs [zfs] [patch] ZFS "sharenfs" doesn't allow different " o kern/147560 fs [zfs] [boot] Booting 8.1-PRERELEASE raidz system take o kern/147420 fs [ufs] [panic] ufs_dirbad, nullfs, jail panic (corrupt o kern/146941 fs [zfs] [panic] Kernel Double Fault - Happens constantly o kern/146786 fs [zfs] zpool import hangs with checksum errors o kern/146708 fs [ufs] [panic] Kernel panic in softdep_disk_write_compl o kern/146528 fs [zfs] Severe memory leak in ZFS on i386 o kern/146502 fs [nfs] FreeBSD 8 NFS Client Connection to Server s kern/145712 fs [zfs] cannot offline two drives in a raidz2 configurat o kern/145411 fs [xfs] [panic] Kernel panics shortly after mounting an f bin/145309 fs bsdlabel: Editing disk label invalidates the whole dev o kern/145272 fs [zfs] [panic] Panic during boot when accessing zfs on o kern/145246 fs [ufs] dirhash in 7.3 gratuitously frees hashes when it o kern/145238 fs [zfs] [panic] kernel panic on zpool clear tank o kern/145229 fs [zfs] Vast differences in ZFS ARC behavior between 8.0 o kern/145189 fs [nfs] nfsd performs abysmally under load o kern/144929 fs [ufs] [lor] vfs_bio.c + ufs_dirhash.c p kern/144447 fs [zfs] sharenfs fsunshare() & fsshare_main() non functi o kern/144416 fs [panic] Kernel panic on online filesystem optimization s kern/144415 fs [zfs] [panic] kernel panics on boot after zfs crash o kern/144234 fs [zfs] Cannot boot machine with recent gptzfsboot code o kern/143825 fs [nfs] [panic] Kernel panic on NFS client o bin/143572 fs [zfs] zpool(1): [patch] The verbose output from iostat o kern/143212 fs [nfs] NFSv4 client strange work ... o kern/143184 fs [zfs] [lor] zfs/bufwait LOR o kern/142878 fs [zfs] [vfs] lock order reversal o kern/142597 fs [ext2fs] ext2fs does not work on filesystems with real o kern/142489 fs [zfs] [lor] allproc/zfs LOR o kern/142466 fs Update 7.2 -> 8.0 on Raid 1 ends with screwed raid [re o kern/142306 fs [zfs] [panic] ZFS drive (from OSX Leopard) causes two o kern/142068 fs [ufs] BSD labels are got deleted spontaneously o kern/141897 fs [msdosfs] [panic] Kernel panic. msdofs: file name leng o kern/141463 fs [nfs] [panic] Frequent kernel panics after upgrade fro o kern/141305 fs [zfs] FreeBSD ZFS+sendfile severe performance issues ( o kern/141091 fs [patch] [nullfs] fix panics with DIAGNOSTIC enabled o kern/141086 fs [nfs] [panic] panic("nfs: bioread, not dir") on FreeBS o kern/141010 fs [zfs] "zfs scrub" fails when backed by files in UFS2 o kern/140888 fs [zfs] boot fail from zfs root while the pool resilveri o kern/140661 fs [zfs] [patch] /boot/loader fails to work on a GPT/ZFS- o kern/140640 fs [zfs] snapshot crash o kern/140068 fs [smbfs] [patch] smbfs does not allow semicolon in file o kern/139725 fs [zfs] zdb(1) dumps core on i386 when examining zpool c o kern/139715 fs [zfs] vfs.numvnodes leak on busy zfs p bin/139651 fs [nfs] mount(8): read-only remount of NFS volume does n o kern/139597 fs [patch] [tmpfs] tmpfs initializes va_gen but doesn't u o kern/139564 fs [zfs] [panic] 8.0-RC1 - Fatal trap 12 at end of shutdo o kern/139407 fs [smbfs] [panic] smb mount causes system crash if remot o kern/138662 fs [panic] ffs_blkfree: freeing free block o kern/138421 fs [ufs] [patch] remove UFS label limitations o kern/138202 fs mount_msdosfs(1) see only 2Gb o kern/136968 fs [ufs] [lor] ufs/bufwait/ufs (open) o kern/136945 fs [ufs] [lor] filedesc structure/ufs (poll) o kern/136944 fs [ffs] [lor] bufwait/snaplk (fsync) o kern/136873 fs [ntfs] Missing directories/files on NTFS volume o kern/136865 fs [nfs] [patch] NFS exports atomic and on-the-fly atomic p kern/136470 fs [nfs] Cannot mount / in read-only, over NFS o kern/135546 fs [zfs] zfs.ko module doesn't ignore zpool.cache filenam o kern/135469 fs [ufs] [panic] kernel crash on md operation in ufs_dirb o kern/135050 fs [zfs] ZFS clears/hides disk errors on reboot o kern/134491 fs [zfs] Hot spares are rather cold... o kern/133676 fs [smbfs] [panic] umount -f'ing a vnode-based memory dis o kern/132960 fs [ufs] [panic] panic:ffs_blkfree: freeing free frag o kern/132397 fs reboot causes filesystem corruption (failure to sync b o kern/132331 fs [ufs] [lor] LOR ufs and syncer o kern/132237 fs [msdosfs] msdosfs has problems to read MSDOS Floppy o kern/132145 fs [panic] File System Hard Crashes o kern/131441 fs [unionfs] [nullfs] unionfs and/or nullfs not combineab o kern/131360 fs [nfs] poor scaling behavior of the NFS server under lo o kern/131342 fs [nfs] mounting/unmounting of disks causes NFS to fail o bin/131341 fs makefs: error "Bad file descriptor" on the mount poin o kern/130920 fs [msdosfs] cp(1) takes 100% CPU time while copying file o kern/130210 fs [nullfs] Error by check nullfs o kern/129760 fs [nfs] after 'umount -f' of a stale NFS share FreeBSD l o kern/129488 fs [smbfs] Kernel "bug" when using smbfs in smbfs_smb.c: o kern/129231 fs [ufs] [patch] New UFS mount (norandom) option - mostly o kern/129152 fs [panic] non-userfriendly panic when trying to mount(8) o kern/127787 fs [lor] [ufs] Three LORs: vfslock/devfs/vfslock, ufs/vfs o bin/127270 fs fsck_msdosfs(8) may crash if BytesPerSec is zero o kern/127029 fs [panic] mount(8): trying to mount a write protected zi o kern/126287 fs [ufs] [panic] Kernel panics while mounting an UFS file o kern/125895 fs [ffs] [panic] kernel: panic: ffs_blkfree: freeing free s kern/125738 fs [zfs] [request] SHA256 acceleration in ZFS o kern/123939 fs [msdosfs] corrupts new files f sparc/123566 fs [zfs] zpool import issue: EOVERFLOW o kern/122380 fs [ffs] ffs_valloc:dup alloc (Soekris 4801/7.0/USB Flash o bin/122172 fs [fs]: amd(8) automount daemon dies on 6.3-STABLE i386, o bin/121898 fs [nullfs] pwd(1)/getcwd(2) fails with Permission denied o bin/121072 fs [smbfs] mount_smbfs(8) cannot normally convert the cha o kern/120483 fs [ntfs] [patch] NTFS filesystem locking changes o kern/120482 fs [ntfs] [patch] Sync style changes between NetBSD and F o kern/118912 fs [2tb] disk sizing/geometry problem with large array o kern/118713 fs [minidump] [patch] Display media size required for a k o bin/118249 fs [ufs] mv(1): moving a directory changes its mtime o kern/118126 fs [nfs] [patch] Poor NFS server write performance o kern/118107 fs [ntfs] [panic] Kernel panic when accessing a file at N o kern/117954 fs [ufs] dirhash on very large directories blocks the mac o bin/117315 fs [smbfs] mount_smbfs(8) and related options can't mount o kern/117314 fs [ntfs] Long-filename only NTFS fs'es cause kernel pani o kern/117158 fs [zfs] zpool scrub causes panic if geli vdevs detach on o bin/116980 fs [msdosfs] [patch] mount_msdosfs(8) resets some flags f o conf/116931 fs lack of fsck_cd9660 prevents mounting iso images with o kern/116583 fs [ffs] [hang] System freezes for short time when using o bin/115361 fs [zfs] mount(8) gets into a state where it won't set/un o kern/114955 fs [cd9660] [patch] [request] support for mask,dirmask,ui o kern/114847 fs [ntfs] [patch] [request] dirmask support for NTFS ala o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o bin/114468 fs [patch] [request] add -d option to umount(8) to detach o kern/113852 fs [smbfs] smbfs does not properly implement DFS referral o bin/113838 fs [patch] [request] mount(8): add support for relative p o bin/113049 fs [patch] [request] make quot(8) use getopt(3) and show o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/111843 fs [msdosfs] Long Names of files are incorrectly created o kern/111782 fs [ufs] dump(8) fails horribly for large filesystems s bin/111146 fs [2tb] fsck(8) fails on 6T filesystem o kern/109024 fs [msdosfs] [iconv] mount_msdosfs: msdosfs_iconv: Operat o kern/109010 fs [msdosfs] can't mv directory within fat32 file system o bin/107829 fs [2TB] fdisk(8): invalid boundary checking in fdisk / w o kern/106107 fs [ufs] left-over fsck_snapshot after unfinished backgro o kern/104406 fs [ufs] Processes get stuck in "ufs" state under persist o kern/104133 fs [ext2fs] EXT2FS module corrupts EXT2/3 filesystems o kern/103035 fs [ntfs] Directories in NTFS mounted disc images appear o kern/101324 fs [smbfs] smbfs sometimes not case sensitive when it's s o kern/99290 fs [ntfs] mount_ntfs ignorant of cluster sizes s bin/97498 fs [request] newfs(8) has no option to clear the first 12 o kern/97377 fs [ntfs] [patch] syntax cleanup for ntfs_ihash.c o kern/95222 fs [cd9660] File sections on ISO9660 level 3 CDs ignored o kern/94849 fs [ufs] rename on UFS filesystem is not atomic o bin/94810 fs fsck(8) incorrectly reports 'file system marked clean' o kern/94769 fs [ufs] Multiple file deletions on multi-snapshotted fil o kern/94733 fs [smbfs] smbfs may cause double unlock o kern/93942 fs [vfs] [patch] panic: ufs_dirbad: bad dir (patch from D o kern/92272 fs [ffs] [hang] Filling a filesystem while creating a sna o kern/91134 fs [smbfs] [patch] Preserve access and modification time a kern/90815 fs [smbfs] [patch] SMBFS with character conversions somet o kern/88657 fs [smbfs] windows client hang when browsing a samba shar o kern/88555 fs [panic] ffs_blkfree: freeing free frag on AMD 64 o kern/88266 fs [smbfs] smbfs does not implement UIO_NOCOPY and sendfi o bin/87966 fs [patch] newfs(8): introduce -A flag for newfs to enabl o kern/87859 fs [smbfs] System reboot while umount smbfs. o kern/86587 fs [msdosfs] rm -r /PATH fails with lots of small files o bin/85494 fs fsck_ffs: unchecked use of cg_inosused macro etc. o kern/80088 fs [smbfs] Incorrect file time setting on NTFS mounted vi o bin/74779 fs Background-fsck checks one filesystem twice and omits o kern/73484 fs [ntfs] Kernel panic when doing `ls` from the client si o bin/73019 fs [ufs] fsck_ufs(8) cannot alloc 607016868 bytes for ino o kern/71774 fs [ntfs] NTFS cannot "see" files on a WinXP filesystem o bin/70600 fs fsck(8) throws files away when it can't grow lost+foun o kern/68978 fs [panic] [ufs] crashes with failing hard disk, loose po o kern/65920 fs [nwfs] Mounted Netware filesystem behaves strange o kern/65901 fs [smbfs] [patch] smbfs fails fsx write/truncate-down/tr o kern/61503 fs [smbfs] mount_smbfs does not work as non-root o kern/55617 fs [smbfs] Accessing an nsmb-mounted drive via a smb expo o kern/51685 fs [hang] Unbounded inode allocation causes kernel to loc o kern/51583 fs [nullfs] [patch] allow to work with devices and socket o kern/36566 fs [smbfs] System reboot with dead smb mount and umount o bin/27687 fs fsck(8) wrapper is not properly passing options to fsc o kern/18874 fs [2TB] 32bit NFS servers export wrong negative values t 259 problems total. From owner-freebsd-fs@FreeBSD.ORG Mon Dec 12 19:04:36 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C07A81065673 for ; Mon, 12 Dec 2011 19:04:36 +0000 (UTC) (envelope-from siur.vvm@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 83BE88FC14 for ; Mon, 12 Dec 2011 19:04:36 +0000 (UTC) Received: by vbbfr13 with SMTP id fr13so6874363vbb.13 for ; Mon, 12 Dec 2011 11:04:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; bh=wqY3/4Ps6qLO9nFGlZPa5mAwcQKZkjHvT1v4T51FbbY=; b=FQHk4WkN39kPjp5nKwr+5JhaTTLXbjIGZXFrz8FwFOtOGcvU1MB7xUf8On1IxBXgQt MmVveqEPcIex3SZlBQKBx4Q5xKmi38clwLBKnC2BvIfXu+OXB/9h6ihI7cySXJBc6aew 6XqrbNjW1BQZ3RSE2M4ppQMLl4fuN6lm8t8ZQ= MIME-Version: 1.0 Received: by 10.52.88.51 with SMTP id bd19mr10559381vdb.118.1323715072324; Mon, 12 Dec 2011 10:37:52 -0800 (PST) Received: by 10.52.28.8 with HTTP; Mon, 12 Dec 2011 10:37:52 -0800 (PST) Date: Mon, 12 Dec 2011 22:37:52 +0400 Message-ID: From: siur To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: epic UFS crash X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 19:04:36 -0000 Hello! Some days ago I had situation with server running under FreeBSD 7.3. Power supply unit suddenly broke down so server switched off incorrectly. After that hardware raid controller marked one disk in RAID1 as degrated. We've boot system from other disk and found out, that almost all data was gone. To be exactly, it's looks like everything created during last uptime had just dissappeared. I mean, literally, server looked like from the past -- there was no data newer than summer 2010. All databases, websites, user's files -- everything was created/modified a year ago. Fsck created much staff in lost+found directories, but, for example, directories looks like empty for 'ls -la', but hex dump shows information (that's, actually more like file, not a directory). So, could anybody help me find out how it possibly could happen? (my creepy story contains very few details, I understand). Does anybody saw something like that? I appreciate any ideas, all that shit just blowing my mind. P.S. Generally, is there possibility to recover data from filesystem with corrupted metadata?(My assumption was about corrupted meta, I hope, superblock and data itself is alive). P.P.S Sorry for my english 8( From owner-freebsd-fs@FreeBSD.ORG Mon Dec 12 19:38:59 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 017E9106564A for ; Mon, 12 Dec 2011 19:38:59 +0000 (UTC) (envelope-from numisemis@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 85C458FC08 for ; Mon, 12 Dec 2011 19:38:58 +0000 (UTC) Received: by bkbzv15 with SMTP id zv15so7546895bkb.13 for ; Mon, 12 Dec 2011 11:38:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=references:from:in-reply-to:mime-version:date:message-id:subject:to :content-type; bh=YFeqmqagAfmbho4hOttn/fDNK7g+kvzv450GyGvm0bE=; b=ILrYZOSOH3j+JLltLt7AFfc44ZxlKmMV4SGkSKPvau0ejM4cEUqxoL/OBK+NVMLS1a LJcO/PDCyapGluLA43p44GVohKJNX3DMMYbTZbIzxC2lj2V/cdwarrfpyA8rCB1Ysw+B hpYkFbadaY65Wbu6Cr8gnVsK4K8q177PZPdGk= Received: by 10.204.7.77 with SMTP id c13mr10628304bkc.32.1323717340040; Mon, 12 Dec 2011 11:15:40 -0800 (PST) References: From: =?UTF-8?Q?=C5=A0imun_Mikecin?= In-Reply-To: Mime-Version: 1.0 (1.0) Date: Mon, 12 Dec 2011 20:15:36 +0100 Message-ID: <4996422587176353892@unknownmsgid> To: "freebsd-fs@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 Subject: Re: epic UFS crash X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 19:38:59 -0000 12. 12. 2011., u 20:04, siur napisao: > Some days ago I had situation with server running under FreeBSD 7.3. > Power supply unit suddenly broke down so server switched off > incorrectly. After that hardware raid controller marked one disk in > RAID1 as degrated. We've boot system from other disk and found out, > that almost all data was gone. To be exactly, it's looks like > everything created during last uptime had just dissappeared. I mean, > literally, server looked like from the past -- there was no data newer > than summer 2010. All databases, websites, user's files -- everything > was created/modified a year ago. > Fsck created much staff in lost+found directories, but, for example, > directories looks like empty for 'ls -la', but hex dump shows > information (that's, actually more like file, not a directory). > > So, could anybody help me find out how it possibly could happen? (my > creepy story contains very few details, I understand). Does anybody > saw something like that? > I appreciate any ideas, all that shit just blowing my mind. That is what could happen if you are using RAID controller in write-back mode without battery-backed RAM for write-back. What kind of UFS is it? UFS1 or UFS2? UFS+SU? GEOM Journaling? From owner-freebsd-fs@FreeBSD.ORG Mon Dec 12 20:01:02 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C5D2A106564A for ; Mon, 12 Dec 2011 20:01:02 +0000 (UTC) (envelope-from dak.col@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 52ACD8FC0A for ; Mon, 12 Dec 2011 20:01:01 +0000 (UTC) Received: by faaf16 with SMTP id f16so776747faa.13 for ; Mon, 12 Dec 2011 12:01:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ayqCYxWw0xuUF8FrAnAKv/LPPbGE9so6kOLXtqQ8zL4=; b=Xa5INZqBQO+iSjBjLsjn9ctDupoCCdvijZDsKBceMlbnDOwEDOOkiaiid7DMIFkIaz jDMGmgwyyOJTiu13SQGnc8N6/ETzRhxYM0giEQQbkvFcmAVCQXNUccCfrhB1ICch4TeD pXWee8E48Wh/qnpy/+pDDtz3i4wf/sWuJtulQ= MIME-Version: 1.0 Received: by 10.180.103.131 with SMTP id fw3mr23365408wib.57.1323718555797; Mon, 12 Dec 2011 11:35:55 -0800 (PST) Received: by 10.216.9.84 with HTTP; Mon, 12 Dec 2011 11:35:55 -0800 (PST) In-Reply-To: References: Date: Mon, 12 Dec 2011 11:35:55 -0800 Message-ID: From: Diego Arias To: siur Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: epic UFS crash X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 20:01:02 -0000 On Mon, Dec 12, 2011 at 10:37 AM, siur wrote: > Hello! > > Some days ago I had situation with server running under FreeBSD 7.3. > Power supply unit suddenly broke down so server switched off > incorrectly. After that hardware raid controller marked one disk in > RAID1 as degrated. We've boot system from other disk and found out, > that almost all data was gone. To be exactly, it's looks like > everything created during last uptime had just dissappeared. I mean, > literally, server looked like from the past -- there was no data newer > than summer 2010. All databases, websites, user's files -- everything > was created/modified a year ago. > Fsck created much staff in lost+found directories, but, for example, > directories looks like empty for 'ls -la', but hex dump shows > information (that's, actually more like file, not a directory). > > So, could anybody help me find out how it possibly could happen? (my > creepy story contains very few details, I understand). Does anybody > saw something like that? > I appreciate any ideas, all that shit just blowing my mind. > > P.S. Generally, is there possibility to recover data from filesystem > with corrupted metadata?(My assumption was about corrupted meta, I > hope, superblock and data itself is alive). > P.P.S Sorry for my english 8( > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > Hi: Maybe the other drive have the info updated, do you check that? -- Still Going Strong!!! From owner-freebsd-fs@FreeBSD.ORG Mon Dec 12 21:08:18 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 05AEA1065672 for ; Mon, 12 Dec 2011 21:08:18 +0000 (UTC) (envelope-from Lee@Dilkie.com) Received: from spock.dilkie.com (spock.dilkie.com [IPv6:2001:470:8900::40]) by mx1.freebsd.org (Postfix) with ESMTP id 6338D8FC08 for ; Mon, 12 Dec 2011 21:08:16 +0000 (UTC) Received: from [10.39.164.100] ([216.191.234.70]) (authenticated bits=0) by spock.dilkie.com (8.14.5/8.14.5) with ESMTP id pBCL89Qn040381 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Mon, 12 Dec 2011 16:08:13 -0500 (EST) (envelope-from Lee@Dilkie.com) X-DKIM: Sendmail DKIM Filter v2.8.3 spock.dilkie.com pBCL89Qn040381 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=dilkie.com; s=mail; t=1323724095; bh=6bcED00MTSSaaEx8H84vQUMQFAHwmOoJ4NU03WbI4Nw=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type; b=RPy3Wm08xPEKgiRvR0atzg8nj82bDKC7PGzS3qIeRcT1vcrr+tJJcTcAwkRhFVJdt lxt9+PB+O48ypu17RvuuJVy5UELbX2Flv1XVrmViRMFZAal9K3zgxdiEM8eIujG Message-ID: <4EE66D35.2060602@Dilkie.com> Date: Mon, 12 Dec 2011 16:08:05 -0500 From: Lee Dilkie User-Agent: Mozilla/5.0 (Windows NT 5.2; WOW64; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 To: Diego Arias References: In-Reply-To: X-Enigmail-Version: 1.3.4 OpenPGP: id=9C23E113 X-Scanned-By: MIMEDefang 2.72 on 142.46.160.214 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: epic UFS crash X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 21:08:18 -0000 On 12/12/2011 2:35 PM, Diego Arias wrote: > Hi: > > Maybe the other drive have the info updated, do you check that? That's what occurred to me, the system was running with the other drive as the only one with this drive as degraded and then things got reversed on the re-boot. If so, the other drive should be up to date. -lee From owner-freebsd-fs@FreeBSD.ORG Mon Dec 12 22:28:41 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B66BA106564A for ; Mon, 12 Dec 2011 22:28:41 +0000 (UTC) (envelope-from rabe@uugrn.org) Received: from mail.uugrn.org (mail.uugrn.org [195.49.138.123]) by mx1.freebsd.org (Postfix) with ESMTP id 494068FC14 for ; Mon, 12 Dec 2011 22:28:40 +0000 (UTC) Received: from rabe.uugrn.org (root@rabe.uugrn.org [195.49.138.102]) by mail.uugrn.org (8.14.4/8.14.3) with ESMTP id pBCMFaTi015730 for ; Mon, 12 Dec 2011 23:15:46 +0100 (CET) (envelope-from rabe@uugrn.org) Received: from rabox.fritz.box (rabe@rabe.uugrn.org [195.49.138.102]) by rabe.uugrn.org (8.14.4/8.13.8) with ESMTP id pBCMFap6015726 for ; Mon, 12 Dec 2011 23:15:36 +0100 (CET) (envelope-from rabe@uugrn.org) Received: from rabox.fritz.box (localhost [127.0.0.1]) by rabox.fritz.box (8.14.4/8.14.4/Debian-2ubuntu1) with ESMTP id pBCMFXi3007829 for ; Mon, 12 Dec 2011 23:15:33 +0100 Received: (from rabe@localhost) by rabox.fritz.box (8.14.4/8.14.4/Submit) id pBCMFXDE007828 for freebsd-fs@freebsd.org; Mon, 12 Dec 2011 23:15:33 +0100 X-Authentication-Warning: rabox.fritz.box: rabe set sender to rabe@uugrn.org using -f Date: Mon, 12 Dec 2011 23:15:33 +0100 From: Raphael Eiselstein To: freebsd-fs@freebsd.org Message-ID: <20111212221533.GB3696@ma.sigsys.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="qcHopEYAB45HaUaB" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Subject: [XFS][9.0RC2] xfs_repair hangs on umtxn X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 22:28:41 -0000 --qcHopEYAB45HaUaB Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, I just tried to do some experiments using xfs on FreeBSD 9.0RC2. My setup is: a plain file /var/cache.img attached as /dev/md0 some things work fine: fbsd9rc2# mdconfig -l fbsd9rc2# mdconfig -a -t vnode -f /var/cache.img=20 md0 fbsd9rc2# kldload xfs fbsd9rc2# mount -t xfs -o ro /dev/md0 /mnt fbsd9rc2# df -h /mnt Filesystem Size Used Avail Capacity Mounted on /dev/md0 5G 144k 5G 0% /mnt others don't: fbsd9rc2# mount -t xfs /dev/md0 /mnt mount: /dev/md0 : Operation not permitted fbsd9rc2# /usr/local/sbin/xfs_repair /dev/md0 Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... - scan filesystem freespace and inode maps... - found root inode chunk Phase 3 - for each AG... - scan and clear agi unlinked lists... - process known inodes and perform inode discovery... - agno =3D 0 ^C =2E.. hangs forever in umtxn, even when operating on the flat file: fbsd9rc2# /usr/local/sbin/xfs_repair -L -f /var/cache.img Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... - scan filesystem freespace and inode maps... - found root inode chunk Phase 3 - for each AG... - scan and clear agi unlinked lists... - process known inodes and perform inode discovery... - agno =3D 0 ^C =2E.. even without any xfs-support in kernel... What is xfs_repair requesting from kernel when getting stuck in umtxn? For me it's not a problem with xfs kernel support but with the xfs-tools from the ports. xfsprogs-2.9.4_3 A set of utilities and library to manipulate an xfs fil= esys FreeBSD fbsd9rc2 9.0-RC2 FreeBSD 9.0-RC2 #0: Sat Nov 12 18:35:25 UTC 2011 root@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 My hardware is a virtual box 4.0.4 OSE running on linux 2.6.38-13-generic What's going on there? Any more debugging information? Best regards Raphael --=20 Raphael Eiselstein http://rabe.uugrn.org/ xmpp:freibyter@gmx.de | https://www.xing.com/profile/Raphael_Eiselstein = =20 GnuPG: E7B2 1D66 3AF2 EDC7 9828 6D7A 9CDA 3E7B 10CA 9F2D =2E........|.........|.........|.........|.........|.........|.........|.. --qcHopEYAB45HaUaB Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAk7mfQUACgkQnNo+exDKny1kKgCcDG6H21mw4IGjcFc/5rZKZ8zx +KIAoKSUCAf1C/WJiuhE1YfpMLfuUnXq =Wn9G -----END PGP SIGNATURE----- --qcHopEYAB45HaUaB-- From owner-freebsd-fs@FreeBSD.ORG Mon Dec 12 22:36:02 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1A51F1065670 for ; Mon, 12 Dec 2011 22:36:02 +0000 (UTC) (envelope-from rabe@uugrn.org) Received: from mail.uugrn.org (mail.uugrn.org [195.49.138.123]) by mx1.freebsd.org (Postfix) with ESMTP id 9B9EC8FC08 for ; Mon, 12 Dec 2011 22:36:01 +0000 (UTC) Received: from rabe.uugrn.org (root@rabe.uugrn.org [195.49.138.102]) by mail.uugrn.org (8.14.4/8.14.3) with ESMTP id pBCMZoFY019450 for ; Mon, 12 Dec 2011 23:36:00 +0100 (CET) (envelope-from rabe@uugrn.org) Received: from rabox.fritz.box (rabe@rabe.uugrn.org [195.49.138.102]) by rabe.uugrn.org (8.14.4/8.13.8) with ESMTP id pBCMZo91019445 for ; Mon, 12 Dec 2011 23:35:50 +0100 (CET) (envelope-from rabe@uugrn.org) Received: from rabox.fritz.box (localhost [127.0.0.1]) by rabox.fritz.box (8.14.4/8.14.4/Debian-2ubuntu1) with ESMTP id pBCMZkqX008137 for ; Mon, 12 Dec 2011 23:35:46 +0100 Received: (from rabe@localhost) by rabox.fritz.box (8.14.4/8.14.4/Submit) id pBCMZkHx008136 for freebsd-fs@freebsd.org; Mon, 12 Dec 2011 23:35:46 +0100 X-Authentication-Warning: rabox.fritz.box: rabe set sender to rabe@uugrn.org using -f Date: Mon, 12 Dec 2011 23:35:46 +0100 From: Raphael Eiselstein To: freebsd-fs@freebsd.org Message-ID: <20111212223546.GC3696@ma.sigsys.de> References: <20111212221533.GB3696@ma.sigsys.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="i7F3eY7HS/tUJxUd" Content-Disposition: inline In-Reply-To: <20111212221533.GB3696@ma.sigsys.de> User-Agent: Mutt/1.5.21 (2010-09-15) Subject: Re: [XFS][9.0RC2] xfs_repair hangs on umtxn X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 22:36:02 -0000 --i7F3eY7HS/tUJxUd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Dec 12, 2011 at 11:15:33PM +0100, Raphael Eiselstein wrote: > I just tried to do some experiments using xfs on FreeBSD 9.0RC2. My > setup is: a plain file /var/cache.img attached as /dev/md0 Just in advance to my flatfile-experiment: Working an on a more-or-less real SATA drive ... ada2 at ahcich2 bus 0 scbus4 target 0 lun 0 ada2: ATA-6 SATA 2.x device ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada2: Command Queueing enabled ada2: 5120MB (10485760 512 byte sectors: 16H 63S/T 10402C) ada2: Previously was known as ad8 =2E.. xfs_repair still doesn't work: fbsd9rc2# gpart add -t freebsd /dev/ada2 ada2s1 added fbsd9rc2# bsdlabel -w /dev/ada2s1=20 fbsd9rc2# bsdlabel /dev/ada2s1 # /dev/ada2s1: 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 10485677 16 unused 0 0 =20 c: 10485693 0 unused 0 0 # "raw" part, don't edit fbsd9rc2# ls -la /dev/ada2* crw-r----- 1 root operator 0, 114 Dec 12 23:23 /dev/ada2 crw-r----- 1 root operator 0, 93 Dec 12 23:49 /dev/ada2s1 crw-r----- 1 root operator 0, 98 Dec 12 23:49 /dev/ada2s1a fbsd9rc2# /usr/local/sbin/mkfs.xfs /dev/ada2s1a meta-data=3D/dev/ada2s1a isize=3D256 agcount=3D8, agsize=3D163= 838 blks =3D sectsz=3D512 attr=3D0 data =3D bsize=3D4096 blocks=3D1310704, imaxpct= =3D25 =3D sunit=3D0 swidth=3D0 blks, unwritte= n=3D1 naming =3Dversion 2 bsize=3D4096 =20 log =3Dinternal log bsize=3D4096 blocks=3D2560, version=3D1 =3D sectsz=3D512 sunit=3D0 blks, lazy-coun= t=3D0 realtime =3Dnone extsz=3D4096 blocks=3D0, rtextents=3D0 fbsd9rc2# /usr/local/sbin/xfs_repair /dev/ada2s1a Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... - scan filesystem freespace and inode maps... - found root inode chunk Phase 3 - for each AG... - scan and clear agi unlinked lists... - process known inodes and perform inode discovery... - agno =3D 0 ^C (hangs in umtxn but can be killed by ^C) Regards Raphael --=20 Raphael Eiselstein http://rabe.uugrn.org/ xmpp:freibyter@gmx.de | https://www.xing.com/profile/Raphael_Eiselstein = =20 GnuPG: E7B2 1D66 3AF2 EDC7 9828 6D7A 9CDA 3E7B 10CA 9F2D =2E........|.........|.........|.........|.........|.........|.........|.. --i7F3eY7HS/tUJxUd Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAk7mgcIACgkQnNo+exDKny1h5QCfa+vz9egMSFuCvx+uogFqPhvR s/cAn2BzungpcAqXxFXrsKIbmWT4l1Oq =px5z -----END PGP SIGNATURE----- --i7F3eY7HS/tUJxUd-- From owner-freebsd-fs@FreeBSD.ORG Mon Dec 12 23:25:56 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AA8AB106566B for ; Mon, 12 Dec 2011 23:25:56 +0000 (UTC) (envelope-from mj@feral.com) Received: from ns1.feral.com (ns1.feral.com [192.67.166.1]) by mx1.freebsd.org (Postfix) with ESMTP id 5F7DC8FC14 for ; Mon, 12 Dec 2011 23:25:55 +0000 (UTC) Received: from [172.16.135.105] (lportal.in1.lcl [172.16.1.9]) (authenticated bits=0) by ns1.feral.com (8.14.4/8.14.4) with ESMTP id pBCMlLwI026573 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Mon, 12 Dec 2011 14:47:22 -0800 (PST) (envelope-from mj@feral.com) Message-ID: <4EE68474.4000601@feral.com> Date: Mon, 12 Dec 2011 14:47:16 -0800 From: Matthew Jacob Organization: Feral Software User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <20111212221533.GB3696@ma.sigsys.de> In-Reply-To: <20111212221533.GB3696@ma.sigsys.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (ns1.feral.com [192.67.166.1]); Mon, 12 Dec 2011 14:47:22 -0800 (PST) Subject: Re: [XFS][9.0RC2] xfs_repair hangs on umtxn X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: mj@feral.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 23:25:56 -0000 Oh, XFS works at all? That's news to me. From owner-freebsd-fs@FreeBSD.ORG Mon Dec 12 23:50:36 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 761CF1065670 for ; Mon, 12 Dec 2011 23:50:36 +0000 (UTC) (envelope-from dieterbsd@engineer.com) Received: from mailout-us.mail.com (mailout-us.gmx.com [74.208.5.67]) by mx1.freebsd.org (Postfix) with SMTP id 1D44F8FC0C for ; Mon, 12 Dec 2011 23:50:36 +0000 (UTC) Received: (qmail 26511 invoked by uid 0); 12 Dec 2011 23:50:35 -0000 Received: from 67.206.184.126 by rms-us005.v300.gmx.net with HTTP Content-Type: text/plain; charset="utf-8" Date: Mon, 12 Dec 2011 18:50:33 -0500 From: "Dieter BSD" Message-ID: <20111212235034.218250@gmx.com> MIME-Version: 1.0 To: freebsd-performance@freebsd.org,freebsd-fs@freebsd.org X-Authenticated: #74169980 X-Flags: 0001 X-Mailer: GMX.com Web Mailer x-registered: 0 Content-Transfer-Encoding: 8bit X-GMX-UID: HfaUbiU03zOlNR3dAHAhLPF+IGRvbwBx X-Mailman-Approved-At: Tue, 13 Dec 2011 01:52:34 +0000 Cc: Subject: Maximum blocksize for FFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 23:50:36 -0000 Many recent disks have a 4KiB sector size, so newfs's default 2KiB frag size seems suboptimal for these drives. Newfs's man page states: "The optimal block:fragment ratio is 8:1. Other ratios are possible, but are not recommended, and may produce poor results."  (It is not clear to me what the 8:1 ratio optimizes, or exactly what poor results one should expect with a different ratio?) Thus one would logically think of using 32 KiB blocksize 4KiB fragsize at a minimum with these drives. But, from a discussion in 2009: Bruce Evans wrote: > Any block size above the default (16K) tends to thrash and fragment buffer > cache virtual memory.  This is obviously a good pessimization with lots of > small files, and apparently, as you have found, it is a good pessimization > with a few large files too.  I think severe fragmentation can easily take > several seconds to recover from.  The worst case for causing fragmentaion > is probably a mixture of small and large files. This caused a *severe* performance problem and I was forced to reduce to reduce my blocksize to 16KiB.  :-( For data filesystems with large files (multi GB), there are many advantages to using large blocksizes such as less space wasted on bookkeeping, and faster fsck times. So I'm wondering if this buffer cache virtual memory thrashing/fragmenting problem has been fixed?  Is it safe to use 64KiB/8KiB yet?  IIRC I've read concerns about thrashing/fragmenting due to different filesystems having different sizes, say one filesystem being 16K/2K and another being 64k/8K? Also, has the "bug in vfs_bio.c" been fixed? (64KiB blocksize can hang the system) Any other gottchas? From owner-freebsd-fs@FreeBSD.ORG Tue Dec 13 02:11:02 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D59481065670 for ; Tue, 13 Dec 2011 02:11:02 +0000 (UTC) (envelope-from spork@bway.net) Received: from xena.bway.net (xena.bway.net [216.220.96.26]) by mx1.freebsd.org (Postfix) with ESMTP id 9EAFD8FC1D for ; Tue, 13 Dec 2011 02:11:02 +0000 (UTC) Received: (qmail 18207 invoked by uid 0); 13 Dec 2011 01:44:21 -0000 Received: from smtp.bway.net (216.220.96.25) by xena.bway.net with (DHE-RSA-AES256-SHA encrypted) SMTP; 13 Dec 2011 01:44:21 -0000 Received: (qmail 18200 invoked by uid 90); 13 Dec 2011 01:44:21 -0000 Received: from unknown (HELO ?10.3.2.41?) (spork@96.57.144.66) by smtp.bway.net with (AES128-SHA encrypted) SMTP; 13 Dec 2011 01:44:21 -0000 References: In-Reply-To: Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii Message-Id: Content-Transfer-Encoding: 7bit From: Charles Sprickman Date: Mon, 12 Dec 2011 20:44:20 -0500 To: siur X-Mailer: Apple Mail (2.1084) Cc: freebsd-fs@freebsd.org Subject: Re: epic UFS crash X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 02:11:02 -0000 On Dec 12, 2011, at 1:37 PM, siur wrote: > Hello! > > Some days ago I had situation with server running under FreeBSD 7.3. > Power supply unit suddenly broke down so server switched off > incorrectly. After that hardware raid controller marked one disk in > RAID1 as degrated. Just curious, what type of RAID controller was this? Charles > We've boot system from other disk and found out, > that almost all data was gone. To be exactly, it's looks like > everything created during last uptime had just dissappeared. I mean, > literally, server looked like from the past -- there was no data newer > than summer 2010. All databases, websites, user's files -- everything > was created/modified a year ago. > Fsck created much staff in lost+found directories, but, for example, > directories looks like empty for 'ls -la', but hex dump shows > information (that's, actually more like file, not a directory). > > So, could anybody help me find out how it possibly could happen? (my > creepy story contains very few details, I understand). Does anybody > saw something like that? > I appreciate any ideas, all that shit just blowing my mind. > > P.S. Generally, is there possibility to recover data from filesystem > with corrupted metadata?(My assumption was about corrupted meta, I > hope, superblock and data itself is alive). > P.P.S Sorry for my english 8( > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" -- Charles Sprickman NetEng/SysAdmin Bway.net - New York's Best Internet www.bway.net spork@bway.net - 212.655.9344 From owner-freebsd-fs@FreeBSD.ORG Tue Dec 13 02:11:19 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E01A1106564A for ; Tue, 13 Dec 2011 02:11:19 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 7C9EC8FC12 for ; Tue, 13 Dec 2011 02:11:18 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqAEACC05k6DaFvO/2dsb2JhbABEFoRxpweBcgEBAQIBAQEBASArIAsbGAICDRkCKQEJJgYIBwQBHASHZwijapFTgTSCRIZfgRYEiDGKG4IlkkU X-IronPort-AV: E=Sophos;i="4.71,342,1320642000"; d="scan'208";a="148235225" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 12 Dec 2011 21:10:54 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 95DE8B3F09; Mon, 12 Dec 2011 21:10:54 -0500 (EST) Date: Mon, 12 Dec 2011 21:10:54 -0500 (EST) From: Rick Macklem To: Julian Elischer Message-ID: <1964582720.114395.1323742254568.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <4EE5645C.9040401@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-fs@freebsd.org Subject: Re: NFSv4.1 status X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 02:11:20 -0000 Julian Elischer wrote: > On 12/11/11 8:37 AM, Rick Macklem wrote: > > Hi, > > > > I thought I'd post to let anyone who is interested know what the > > status of NFSv4.1 is. First off, the slides here give you an > > overview of where the vendors are: > > http://people.redhat.com/steved/Lisa-11/LISA-11-pNFS-BoF-final.pdf > > > > You'll notice that FreeBSD isn't mentioned, which is fair, since no > > NFSv4.1 support is in FreeBSD9.0. However, I now have the basic > > NFSv4.1 > > client support written and lightly tested > but Panassus and Netapp were.. have you talked to them about > interactions > between your code and theirs when the time comes to merge? Well, I was able to do some testing at the NFSv4.1 Bakeathon last June. These Bakeathons + Connectathon are where the interoperability testing goes on. Unfortunately, I can't normally attend, since going to these costs $$$ and, as I suspect you already know, I'm not funded to do this work. I was able to make it to the one last June, because it was in Ann Arbor MI and I could drive my little Toyota Echo there (doesn't burn much gas) and stay in the el-cheapo Motel 6. (Also, the Bakeathons don't have any registration fee, unlike Connectathon.) Now, you normally sign an NDA which covers what you learn at these for 1 year. For some reason, that didn't happen last June, but I am still hesitate to discuss what I learned, except to say that the code was mostly written during that week and was working well against the servers that I could test against there by the end of the week. I hope to be able to repeat the above, since they are planning another Ann Arbor Bakeathon in the June 2012 timeframe. Also, someone at Netapp was attempting to make their server simulator (runs just like a real Netapp Filer, except very slowly as a Linux task) available to me, but I haven't heard from him lately, so I suspect that it may never happen. He's a good guy, but can't do anything the company won't allow. (He also mentioned that I would probably have to sign an NDA and until I see and read the document, I cannot say if I will be willing to do so.) This person is also trying to convince the testing team at Netapp to do FreeBSD client to Netapp filer testing, but I have no idea if he`s made progress on that either. > > For releng9: > > http://people.freebsd.org/~rmacklem/nfsv4.1-client/nfsv4.1-releng9.patch > > > > For head/-current, there is an up-to-date source tree at: > > http://svnweb.freebsd.org/base/projects/nfsv4.1-client/sys/ > > > > It does not include any pNFS support at this point, but I'm am just > > starting > > to work on that and hopefully will have something for the file > > layout soon > > and hope to get to test it against various servers at the NFSv4 > > Bakeathon in June. > > > > rick > > ps: There has been no work done yet on a FreeBSD NFSv4.1 server as > > far as I know. > Pannassus (how DO you spell that?) and netapp must have that part > worked out. If you are referring to a server, then yes, the slides would indicate that they are ready to ship soon. I suspect both companies have invested many man years of engineering (ie. many $$$$) to this effort. They would definitely consider the results proprietary. (Although Netapp uses some FreeBSD code, it is my understanding that their NFS code is a separate world from that of FreeBSD. As noted above, even getting to test against a pre-release binary is considered proprietary and they haven't, as yet, agreed to let me do that even if I sign some NDA.) Btw, both companies have sponsored work related to the Linux client. For example, Trond Myklebust, who is a/the major Linux NFS client developer, is a Netapp employee and Panasus has contributed Linux client work that they have done "in house" for their object layout. (Someone from Panasus has indicated that I should try to adhere to the Linux client API that sits between the generic layout handler and the layout specific driver. I am hoping that implies that I may have access to their layout driver under a non-GPL`d open source license at some point, but this is just dreaming on my part at this point.:-) I have recently received some email related to how to patch a Linux kernel to get a basic pNFS server going to testing. Hopefully I can get this going. (I currently use a Fedora15 Linux NFSv4.1 server for testing, but it doesn`t have pNFS support in it.) Did this long winded writeup cover what you were asking...rick > > _______________________________________________ > > freebsd-fs@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > > To unsubscribe, send any mail to > > "freebsd-fs-unsubscribe@freebsd.org" > > From owner-freebsd-fs@FreeBSD.ORG Tue Dec 13 02:46:36 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D77621065670; Tue, 13 Dec 2011 02:46:36 +0000 (UTC) (envelope-from jwd@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id ABFB38FC0C; Tue, 13 Dec 2011 02:46:36 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id pBD2kaCI002374; Tue, 13 Dec 2011 02:46:36 GMT (envelope-from jwd@freefall.freebsd.org) Received: (from jwd@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id pBD2kanc002373; Tue, 13 Dec 2011 02:46:36 GMT (envelope-from jwd) Date: Tue, 13 Dec 2011 02:46:36 +0000 From: John To: freebsd-fs@freebsd.org, freebsd-current@freebsd.org Message-ID: <20111213024636.GA47103@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i Cc: Subject: multihomed nfs server - NLM lock failure on additional interfaces X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 02:46:36 -0000 Hi Folks, I have a 9-prerelease system where I've been testing nfs/zfs. The system has been working quite well until moving the server to a multihomed configuration. Given the following: nfsd: master (nfsd) nfsd: server (nfsd) /usr/sbin/rpcbind -h 10.24.6.38 -h 172.1.1.2 -h 172.21.201.1 -h 172.21.202.1 -h 172.21.203.1 -h 172.21.204.1 -h 172.21.205.1 -h 10.24.6.34 -h 10.24.6.33 /usr/sbin/mountd -r -l -h 10.24.6.38 -h 172.1.1.2 -h 172.21.201.1 -h 172.21.202.1 -h 172.21.203.1 -h 172.21.204.1 -h 172.21.205.1 -h 10.24.6.34 -h 10.24.6.33 /usr/sbin/rpc.statd -h 10.24.6.38 -h 172.1.1.2 -h 172.21.201.1 -h 172.21.202.1 -h 172.21.203.1 -h 172.21.204.1 -h 172.21.205.1 -h 10.24.6.34 -h 10.24.6.33 /usr/sbin/rpc.lockd -h 10.24.6.38 -h 172.1.1.2 -h 172.21.201.1 -h 172.21.202.1 -h 172.21.203.1 -h 172.21.204.1 -h 172.21.205.1 -h 10.24.6.34 -h 10.24.6.33 10.24.6.38 is the default interface on 1G. The 172 nets are 10G connected to compute systems. ifconfig_bce0=' inet 10.24.6.38 netmask 255.255.0.0 -rxcsum -txcsum' _c='physical addr which never changes' ifconfig_bce1=' inet 172.1.1.2 netmask 255.255.255.0' _c='physcial addr on crossover cable' ifconfig_cxgb2='inet 172.21.21.129 netmask 255.255.255.0' _c='physical backside 10g compute net' ifconfig_cxgb3='inet 172.21.201.1 netmask 255.255.255.0 mtu 9000' _c='physical backside 10g compute net' ifconfig_cxgb6='inet 172.21.202.1 netmask 255.255.255.0 mtu 9000' _c='physical backside 10g compute net' ifconfig_cxgb8='inet 172.21.203.1 netmask 255.255.255.0 mtu 9000' _c='physical backside 10g compute net' ifconfig_cxgb4='inet 172.21.204.1 netmask 255.255.255.0 mtu 9000' _c='physical backside 10g compute net' ifconfig_cxgb0='inet 172.21.205.1 netmask 255.255.255.0 mtu 9000' _c='physical backside 10g compute net' The 10.24.6.34 and 10.24.6.33 are alias addresses for the system. Destination Gateway Flags Refs Use Netif Expire default 10.24.0.1 UGS 0 1049 bce0 The server works correctly (and quite well) for both udp & tcp mounts. Basically, all nfs traffic is great! However, locking only works for clients connected to the 10.24.6.38 interface. A tcpdump file from good & bad runs: http://www.freebsd.org/~jwd/lockgood.pcap http://www.freebsd.org/~jwd/lockbad.pcap Basically, the clients (both FreeBSD & Linux) query the servers rpcbind for the address of the nlm which is returned correctly. For the good run, the NLM is then called. For the bad call, it is not. I've started digging through code, but I do not claim to be an rpc expert. If anyone has suggestions I would appreciate any pointers. Thanks! John From owner-freebsd-fs@FreeBSD.ORG Tue Dec 13 08:05:46 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 09C77106564A for ; Tue, 13 Dec 2011 08:05:46 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id B87938FC13 for ; Tue, 13 Dec 2011 08:05:45 +0000 (UTC) Received: from julian-mac.elischer.org (c-67-180-24-15.hsd1.ca.comcast.net [67.180.24.15]) (authenticated bits=0) by vps1.elischer.org (8.14.4/8.14.4) with ESMTP id pBD85gHD039608 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 13 Dec 2011 00:05:44 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <4EE7076A.8030108@freebsd.org> Date: Tue, 13 Dec 2011 00:06:02 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.24) Gecko/20111103 Thunderbird/3.1.16 MIME-Version: 1.0 To: Rick Macklem References: <1964582720.114395.1323742254568.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <1964582720.114395.1323742254568.JavaMail.root@erie.cs.uoguelph.ca> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: NFSv4.1 status X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 08:05:46 -0000 On 12/12/11 6:10 PM, Rick Macklem wrote: > > Did this long winded writeup cover what you were asking...rick > yep >>> _______________________________________________ >>> freebsd-fs@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >>> To unsubscribe, send any mail to >>> "freebsd-fs-unsubscribe@freebsd.org" >>> From owner-freebsd-fs@FreeBSD.ORG Tue Dec 13 09:15:09 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6D3E21065672 for ; Tue, 13 Dec 2011 09:15:09 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 057A68FC17 for ; Tue, 13 Dec 2011 09:15:08 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id pBD9Ewnh076089 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 13 Dec 2011 11:14:58 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id pBD9Evt9038621; Tue, 13 Dec 2011 11:14:57 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id pBD9EvLw038620; Tue, 13 Dec 2011 11:14:57 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 13 Dec 2011 11:14:57 +0200 From: Kostik Belousov To: Dieter BSD Message-ID: <20111213091457.GU50300@deviant.kiev.zoral.com.ua> References: <20111212235034.218250@gmx.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="MzUVogtTSRhL5bvu" Content-Disposition: inline In-Reply-To: <20111212235034.218250@gmx.com> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-fs@freebsd.org, freebsd-performance@freebsd.org Subject: Re: Maximum blocksize for FFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 09:15:09 -0000 --MzUVogtTSRhL5bvu Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Dec 12, 2011 at 06:50:33PM -0500, Dieter BSD wrote: > Many recent disks have a 4KiB sector size, so newfs's default > 2KiB frag size seems suboptimal for these drives. Newfs's man > page states: "The optimal block:fragment ratio is 8:1. Other > ratios are possible, but are not recommended, and may produce > poor results." =9A(It is not clear to me what the 8:1 ratio optimizes, > or exactly what poor results one should expect with a different ratio?) > Thus one would logically think of using 32 KiB blocksize 4KiB fragsize > at a minimum with these drives. >=20 > But, from a discussion in 2009: >=20 > Bruce Evans wrote: > > Any block size above the default (16K) tends to thrash and fragment buf= fer > > cache virtual memory. =9AThis is obviously a good pessimization with lo= ts of > > small files, and apparently, as you have found, it is a good pessimizat= ion > > with a few large files too. =9AI think severe fragmentation can easily = take > > several seconds to recover from. =9AThe worst case for causing fragment= aion > > is probably a mixture of small and large files. >=20 > This caused a *severe* performance problem and I was forced to reduce to > reduce my blocksize to 16KiB. =9A:-( >=20 > For data filesystems with large files (multi GB), there are many advantag= es > to using large blocksizes such as less space wasted on bookkeeping, > and faster fsck times. >=20 > So I'm wondering if this buffer cache virtual memory thrashing/fragmenting > problem has been fixed? =9AIs it safe to use 64KiB/8KiB yet? =9AIIRC I've > read concerns about thrashing/fragmenting due to different filesystems > having different sizes, say one filesystem being 16K/2K and another being > 64k/8K? >=20 > Also, has the "bug in vfs_bio.c" been fixed? (64KiB blocksize can > hang the system) >=20 > Any other gottchas? I think the default KVA allocated for the buffer cache might be too small for the 64KB blocks. The only known issue that prevented defragmentation from working was fixed in r189878. I did not see further reports of the hangs caused by fragmented buffer KVA. --MzUVogtTSRhL5bvu Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk7nF5EACgkQC3+MBN1Mb4gXLQCfc5Avzx04azOdEpi+0xfQ+Ufz EWMAoJhCmRlybh4j1kgdqK88tX+aWEAN =lIRB -----END PGP SIGNATURE----- --MzUVogtTSRhL5bvu-- From owner-freebsd-fs@FreeBSD.ORG Tue Dec 13 11:04:25 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DAF4C1065670 for ; Tue, 13 Dec 2011 11:04:25 +0000 (UTC) (envelope-from joh.hendriks@gmail.com) Received: from mail-ee0-f54.google.com (mail-ee0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 754848FC12 for ; Tue, 13 Dec 2011 11:04:24 +0000 (UTC) Received: by eekc50 with SMTP id c50so3803615eek.13 for ; Tue, 13 Dec 2011 03:04:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=99EMPDFgoFZdI1tobaZIco8oJDRB3X0p1D78ehso1rU=; b=j6hl5Xg5z30/S4A3vIGHCsUT3KKWMfueUcWgo7C2rn3eXpshpSFZXTDu+R03p6qHtB R43NlqOErhSMCQ8D3cFw8gj7DABb/BriiJ0nFBhuK9mXXwNFZni/hrOt1as02fa4/7ci 7sSly2Meq/BXtWXyolaY/88IxikVCia2XlBrY= Received: by 10.14.13.138 with SMTP id b10mr4006201eeb.239.1323774264233; Tue, 13 Dec 2011 03:04:24 -0800 (PST) Received: from [192.168.1.129] (schavemaker.nl. [213.84.84.186]) by mx.google.com with ESMTPS id a60sm87757463eeb.4.2011.12.13.03.04.23 (version=SSLv3 cipher=OTHER); Tue, 13 Dec 2011 03:04:23 -0800 (PST) Message-ID: <4EE73135.2080702@gmail.com> Date: Tue, 13 Dec 2011 12:04:21 +0100 From: Johan Hendriks User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: FreeBSD 9.0 and NFS async X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 11:04:25 -0000 Hello all. I used to use async on my 8.x nfs servers! On the FreeBSD 9.0 server i can not do it through the old 8.x sysctl. Is there an other way to set async on FreeBSD 9.x regards, Johan Hendriks From owner-freebsd-fs@FreeBSD.ORG Tue Dec 13 13:51:34 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5B795106564A for ; Tue, 13 Dec 2011 13:51:34 +0000 (UTC) (envelope-from siur.vvm@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 165F28FC0C for ; Tue, 13 Dec 2011 13:51:33 +0000 (UTC) Received: by vcbfk1 with SMTP id fk1so7909243vcb.13 for ; Tue, 13 Dec 2011 05:51:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=jK3n4pjCxjo107LNaEdAN3I32Kj1tEl7nhXY/K99vgE=; b=C7M816xhCuMv6ZwBhgBjSFUChlLMGt0q/YO8bVpf98/gXRX09SnaMCVnwzdo/Sn2iK FIBmUgYnyOPAmVwC2DLQLe6LGLs4th9rolpA1AP7Fn8RcU7oxBegxjWGqnvC9YbOMqPq QUdVjqdLOXZIwC4Hw+Awg+UzJnMax664VWIEg= MIME-Version: 1.0 Received: by 10.52.35.70 with SMTP id f6mr1262563vdj.84.1323784293324; Tue, 13 Dec 2011 05:51:33 -0800 (PST) Received: by 10.52.28.8 with HTTP; Tue, 13 Dec 2011 05:51:33 -0800 (PST) In-Reply-To: <4996422587176353892@unknownmsgid> References: <4996422587176353892@unknownmsgid> Date: Tue, 13 Dec 2011 17:51:33 +0400 Message-ID: From: siur To: =?UTF-8?Q?=C5=A0imun_Mikecin?= Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-fs@freebsd.org" Subject: Re: epic UFS crash X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 13:51:34 -0000 > That is what could happen if you are using RAID controller in > write-back mode without battery-backed RAM for write-back. > What kind of UFS is it? UFS1 or UFS2? > UFS+SU? GEOM Journaling? Why RAID-controller could write data only in a cache for almost a year?? It was UFS2, without journaling. From owner-freebsd-fs@FreeBSD.ORG Tue Dec 13 13:54:27 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A3FE91065675 for ; Tue, 13 Dec 2011 13:54:27 +0000 (UTC) (envelope-from siur.vvm@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 657D88FC12 for ; Tue, 13 Dec 2011 13:54:27 +0000 (UTC) Received: by vbbfr13 with SMTP id fr13so7897177vbb.13 for ; Tue, 13 Dec 2011 05:54:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=mkqRGYNlz6/FxS/eO2ZGp3XHz5YmmeWevm5vFj6eWlA=; b=a4Exf8ecjjL1EEl/jTXtADYaEzy5P/LLwJxJuU+t+Dyd5bgTO49oDRK7TEhq5MsN6D NzZ91hpGP7Me/pXEHIumAqd9ZSd4QGbi5ognWmA8qubz2NB70cN6Vja2x/OL2V5An+oT ZqBEGVwVA/pxNS29PQ7/lDuq9PwuOYsoyph3A= MIME-Version: 1.0 Received: by 10.52.99.37 with SMTP id en5mr1269218vdb.101.1323784466999; Tue, 13 Dec 2011 05:54:26 -0800 (PST) Received: by 10.52.28.8 with HTTP; Tue, 13 Dec 2011 05:54:26 -0800 (PST) In-Reply-To: References: Date: Tue, 13 Dec 2011 17:54:26 +0400 Message-ID: From: siur To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: Re: epic UFS crash X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 13:54:27 -0000 > Maybe the other drive have the info updated, do you check that? Yes, I did. There is exactly the same old staff on degrated disc as on bootable. > Just curious, what type of RAID controller was this? Oh, I don't remember. Some built-in intel controller. From owner-freebsd-fs@FreeBSD.ORG Tue Dec 13 14:34:58 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A044D106566B for ; Tue, 13 Dec 2011 14:34:58 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 5B3378FC0C for ; Tue, 13 Dec 2011 14:34:58 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1RaTRY-0003KI-6C for freebsd-fs@freebsd.org; Tue, 13 Dec 2011 15:34:56 +0100 Received: from ib-jtotz.ib.ic.ac.uk ([155.198.110.220]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 13 Dec 2011 15:34:56 +0100 Received: from johannes by ib-jtotz.ib.ic.ac.uk with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 13 Dec 2011 15:34:56 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Johannes Totz Date: Tue, 13 Dec 2011 14:34:40 +0000 Lines: 15 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: ib-jtotz.ib.ic.ac.uk User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0) Gecko/20111105 Thunderbird/8.0 Subject: zpool failmode=continue X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 14:34:58 -0000 Hi! Is failmode=continue for zpool working correctly? I have a test pool sitting on the network (which drops out every few hours) but all io eventually just hangs instead of failing. To get the pool back up running (and get rid of all the hung processes) I always need to reboot the machine. This is on: FreeBSD XXX 9.0-PRERELEASE FreeBSD 9.0-PRERELEASE #1 r227793M: Mon Dec 5 22:57:54 GMT 2011 root@XXX:/usr/obj/usr/src/sys/GENERIC amd64 Johannes From owner-freebsd-fs@FreeBSD.ORG Tue Dec 13 14:44:45 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2AA4A106564A for ; Tue, 13 Dec 2011 14:44:45 +0000 (UTC) (envelope-from peter.maloney@brockmann-consult.de) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.186]) by mx1.freebsd.org (Postfix) with ESMTP id B6FDF8FC08 for ; Tue, 13 Dec 2011 14:44:44 +0000 (UTC) Received: from [10.3.0.26] ([141.4.215.32]) by mrelayeu.kundenserver.de (node=mreu4) with ESMTP (Nemesis) id 0LplfX-1R3uGK1k7r-00fnPA; Tue, 13 Dec 2011 15:44:43 +0100 Message-ID: <4EE764DA.4030206@brockmann-consult.de> Date: Tue, 13 Dec 2011 15:44:42 +0100 From: Peter Maloney User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.23) Gecko/20110922 Thunderbird/3.1.15 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Provags-ID: V02:K0:v0/2dmVKEQil7BdAdOHWSJZA23cGTHsbzmRTaAtl8m8 b6TOH1wXE1uW4ryYCunzyTK0HUnY+oyacRoE/iefYl8XFInwdl 7By0CvnVkOQxTig/3YANiUR5T/qpFLeXBFsF1RBWNvfv9ZDI6h KvS+GAn412pbcL35G9FWmr0LTHWNzsGmdwxCds0eQ/0g214BOB oQ6WDGw8w8VBfNWMZS+apfGDSzjkTYCQZrxpUGc6tU56lcLQTA TJpl+M6UsYGYcyumiFeTZDlYasqZeLhKhD+IQ3+xYvOIKq6f2t xVhunm7iRyz1osKyk4TbywfB1phaKTmziB3zOytkyXWuOMvsFX mjfn/2LUb+yZb2O/ng7fGHwsNF7nG7R346K+9Fd/v Subject: Re: zpool failmode=continue X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 14:44:45 -0000 Are you using NFS or ZVOLs? My zfs hangs (all IO) if I go into the .zfs/snapshots directory over NFS. (planning to file a PR after I find a way to reproduce it reliably, but it depends on specific snapshots). My workaround is to mount /var/empty on top of the .zfs directory on the nfs client, and give nobody else access. Another workaround I thought of is to have another parent directory in the dataset, and share the 2nd level down which doesn't contain the .zfs directory. And a single dataset hangs if I rename a ZVOL snapshot (likely not the only way to cause this): http://www.freebsd.org/cgi/query-pr.cgi?pr=161968 I am using: FreeBSD bcnas1.bc.local 8.2-STABLE FreeBSD 8.2-STABLE #0: Thu Sep 29 15:06:03 CEST 2011 root@bcnas1.bc.local:/usr/obj/usr/src/sys/GENERIC amd64 On 12/13/2011 03:34 PM, Johannes Totz wrote: > Hi! > > Is failmode=continue for zpool working correctly? > > I have a test pool sitting on the network (which drops out every few > hours) but all io eventually just hangs instead of failing. > > To get the pool back up running (and get rid of all the hung > processes) I always need to reboot the machine. This is on: > FreeBSD XXX 9.0-PRERELEASE FreeBSD 9.0-PRERELEASE #1 r227793M: Mon Dec > 5 22:57:54 GMT 2011 root@XXX:/usr/obj/usr/src/sys/GENERIC amd64 > > > > Johannes > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" -- -------------------------------------------- Peter Maloney Brockmann Consult Max-Planck-Str. 2 21502 Geesthacht Germany Tel: +49 4152 889 300 Fax: +49 4152 889 333 E-mail: peter.maloney@brockmann-consult.de Internet: http://www.brockmann-consult.de -------------------------------------------- From owner-freebsd-fs@FreeBSD.ORG Tue Dec 13 14:51:13 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A01541065673 for ; Tue, 13 Dec 2011 14:51:13 +0000 (UTC) (envelope-from tevans.uk@googlemail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 578838FC0C for ; Tue, 13 Dec 2011 14:51:13 +0000 (UTC) Received: by vbbfr13 with SMTP id fr13so7980491vbb.13 for ; Tue, 13 Dec 2011 06:51:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=lUujENSfedgC+COIXF7i2hHIJsayACIqaOkq3HGzjzo=; b=a7Q+oSmM9tao1kpxwlxNkuvmMdFsqWlJVBKRefeAMJx3KBeotzN5e2n8wXZltybfsB SFyUwVkUI1uNk9/i4AWoWBenW+ZTlrIRP7yKX1Fb4MrolDxHCS9TXGIB7pWjDUdF1PNT ZVkBfcR10ohnkvXXJfQ/SR+bTnSo8fy5jv2FE= MIME-Version: 1.0 Received: by 10.52.173.197 with SMTP id bm5mr1366642vdc.7.1323785985840; Tue, 13 Dec 2011 06:19:45 -0800 (PST) Received: by 10.52.162.202 with HTTP; Tue, 13 Dec 2011 06:19:45 -0800 (PST) In-Reply-To: References: Date: Tue, 13 Dec 2011 14:19:45 +0000 Message-ID: From: Tom Evans To: siur Content-Type: text/plain; charset=UTF-8 Cc: freebsd-fs@freebsd.org Subject: Re: epic UFS crash X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 14:51:13 -0000 On Tue, Dec 13, 2011 at 1:54 PM, siur wrote: >> Just curious, what type of RAID controller was this? > > Oh, I don't remember. Some built-in intel controller. That doesn't sound like a real RAID controller, sounds more like a soft-raid - were you using ataraid or gmirror? Cheers Tom From owner-freebsd-fs@FreeBSD.ORG Tue Dec 13 14:53:55 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0E3FE106566B for ; Tue, 13 Dec 2011 14:53:55 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 8E1FE8FC15 for ; Tue, 13 Dec 2011 14:53:54 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1RaTjt-0004gs-DR for freebsd-fs@freebsd.org; Tue, 13 Dec 2011 15:53:53 +0100 Received: from ib-jtotz.ib.ic.ac.uk ([155.198.110.220]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 13 Dec 2011 15:53:53 +0100 Received: from johannes by ib-jtotz.ib.ic.ac.uk with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 13 Dec 2011 15:53:53 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Johannes Totz Date: Tue, 13 Dec 2011 14:53:34 +0000 Lines: 57 Message-ID: References: <4EE764DA.4030206@brockmann-consult.de> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: ib-jtotz.ib.ic.ac.uk User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0) Gecko/20111105 Thunderbird/8.0 In-Reply-To: <4EE764DA.4030206@brockmann-consult.de> Subject: Re: zpool failmode=continue X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 14:53:55 -0000 On 13/12/2011 14:44, Peter Maloney wrote: > Are you using NFS or ZVOLs? Neither, see below. > My zfs hangs (all IO) if I go into the .zfs/snapshots directory over > NFS. (planning to file a PR after I find a way to reproduce it reliably, > but it depends on specific snapshots). My workaround is to mount > /var/empty on top of the .zfs directory on the nfs client, and give > nobody else access. Another workaround I thought of is to have another > parent directory in the dataset, and share the 2nd level down which > doesn't contain the .zfs directory. My pool is not exported to any clients. My situation is actually the other way around, should have been more clear: the block device on which I created the pool is a on the network. It's kind of a crazy setup: - sshfs to another (Linux) machine - create big image file - create pool from file vdev mounted via sshfs Eventually the network drops out, zpool shows read and write errors, fine so far. But all new io just hangs instead of failing with an error. > And a single dataset hangs if I rename a ZVOL snapshot (likely not the > only way to cause this): http://www.freebsd.org/cgi/query-pr.cgi?pr=161968 > > I am using: > FreeBSD bcnas1.bc.local 8.2-STABLE FreeBSD 8.2-STABLE #0: Thu Sep 29 > 15:06:03 CEST 2011 > root@bcnas1.bc.local:/usr/obj/usr/src/sys/GENERIC amd64 > > > > On 12/13/2011 03:34 PM, Johannes Totz wrote: >> Hi! >> >> Is failmode=continue for zpool working correctly? >> >> I have a test pool sitting on the network (which drops out every few >> hours) but all io eventually just hangs instead of failing. >> >> To get the pool back up running (and get rid of all the hung >> processes) I always need to reboot the machine. This is on: >> FreeBSD XXX 9.0-PRERELEASE FreeBSD 9.0-PRERELEASE #1 r227793M: Mon Dec >> 5 22:57:54 GMT 2011 root@XXX:/usr/obj/usr/src/sys/GENERIC amd64 >> >> >> >> Johannes >> >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > From owner-freebsd-fs@FreeBSD.ORG Tue Dec 13 15:41:54 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3163D1065679 for ; Tue, 13 Dec 2011 15:41:54 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id E11C78FC16 for ; Tue, 13 Dec 2011 15:41:53 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqAEAPZx506DaFvO/2dsb2JhbABDFoRxpyCBcgEBAQQBAQEgKyALGw4KAgINGQIpAQkmBggHBAEcBIdjpC6RVIEvgkSGX4EWBIgwihyCJZJH X-IronPort-AV: E=Sophos;i="4.71,345,1320642000"; d="scan'208";a="148305194" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 13 Dec 2011 10:41:52 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id DAB92B3FB5; Tue, 13 Dec 2011 10:41:52 -0500 (EST) Date: Tue, 13 Dec 2011 10:41:52 -0500 (EST) From: Rick Macklem To: Johan Hendriks Message-ID: <1888801930.136947.1323790912860.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <4EE73135.2080702@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-fs@freebsd.org Subject: Re: FreeBSD 9.0 and NFS async X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 15:41:54 -0000 Johan Hendriks wrote: > Hello all. > > I used to use async on my 8.x nfs servers! > On the FreeBSD 9.0 server i can not do it through the old 8.x sysctl. > > Is there an other way to set async on FreeBSD 9.x > You have two choices: 1 - Apply this patch to your NFS server's kernel sources and then set vfs.nfsd.async=1 http://people.freebsd.org/~rmacklem/async.patch 2 - switch to using the old server by setting oldnfs_server_enable="YES" in your /etc/rc.conf and then setting the sysctl. I'll assume that you realize that doing this violates the NFS RFCs because it runs your server in a way where there is a risk of data loss (that the client won't know to re-write) when the server crashes. rick > regards, > Johan Hendriks > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Tue Dec 13 16:14:26 2011 Return-Path: Delivered-To: fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D89E3106564A; Tue, 13 Dec 2011 16:14:26 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id EE3848FC0C; Tue, 13 Dec 2011 16:14:25 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA17055; Tue, 13 Dec 2011 17:54:54 +0200 (EET) (envelope-from avg@FreeBSD.org) Message-ID: <4EE7754D.3050605@FreeBSD.org> Date: Tue, 13 Dec 2011 17:54:53 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111109 Thunderbird/8.0 MIME-Version: 1.0 To: =?ISO-8859-1?Q?Karli_Sj=F6berg?= References: <82B38DBF-DD3A-46CD-93F6-02CDB6506E05@slu.se> <73B607D0-8C3E-4980-B901-B986E060D32E@slu.se> In-Reply-To: <73B607D0-8C3E-4980-B901-B986E060D32E@slu.se> X-Enigmail-Version: undefined Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Cc: "Kenneth D. Merry" , "fs@freebsd.org" Subject: Re: Consistant panics trying to access zfs filesystems replicated from Sun/Oracle appliance X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 16:14:26 -0000 on 12/12/2011 10:46 Karli Sjöberg said the following: > I got the panic from the IPMI KVM: > http://i55.tinypic.com/synpzk.png I think that your best bet is to debug what's going on in zfs_fuid_table_load function. Add a few printfs in it to see what values get passed to avl_add calls. It would make sense to print f_idx and f_ksid->kd_name fields for each domnode. It looks like a duplicate entry is getting inserted into one of the trees. -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Tue Dec 13 16:18:40 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5C9131065670 for ; Tue, 13 Dec 2011 16:18:40 +0000 (UTC) (envelope-from joh.hendriks@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id DCE868FC1A for ; Tue, 13 Dec 2011 16:18:39 +0000 (UTC) Received: by eaaf13 with SMTP id f13so1636992eaa.13 for ; Tue, 13 Dec 2011 08:18:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=BdNSC8RjgS5MgSqsj6VA9onz7A2L+oUlQwMsAIHi+h8=; b=VbzTchAtelu0Qw2QYGASaEHl7tX7AAYx+/Zg45P76FrfPuLJgH26lmi+kzaGtMekZf 2Ky51fY0X7jOi+YpMwc3g7/ycGmd68l7mMZU4LPQqSIcPO6z4YjTCjcgo1EjVeBlZed+ YezkxK33HSf9Bf9Pmp6A83Haaz5wAvo+tF43U= Received: by 10.204.14.208 with SMTP id h16mr12993353bka.2.1323793118822; Tue, 13 Dec 2011 08:18:38 -0800 (PST) Received: from [192.168.1.129] (schavemaker.nl. [213.84.84.186]) by mx.google.com with ESMTPS id jf4sm41714025bkc.5.2011.12.13.08.18.37 (version=SSLv3 cipher=OTHER); Tue, 13 Dec 2011 08:18:38 -0800 (PST) Message-ID: <4EE77ADC.9050501@gmail.com> Date: Tue, 13 Dec 2011 17:18:36 +0100 From: Johan Hendriks User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <1888801930.136947.1323790912860.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <1888801930.136947.1323790912860.JavaMail.root@erie.cs.uoguelph.ca> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: FreeBSD 9.0 and NFS async X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 16:18:40 -0000 Rick Macklem schreef: > Johan Hendriks wrote: >> Hello all. >> >> I used to use async on my 8.x nfs servers! >> On the FreeBSD 9.0 server i can not do it through the old 8.x sysctl. >> >> Is there an other way to set async on FreeBSD 9.x >> > You have two choices: > 1 - Apply this patch to your NFS server's kernel sources and then set > vfs.nfsd.async=1 > http://people.freebsd.org/~rmacklem/async.patch > > 2 - switch to using the old server by setting > oldnfs_server_enable="YES" > in your /etc/rc.conf and then setting the sysctl. > > I'll assume that you realize that doing this violates the NFS RFCs because > it runs your server in a way where there is a risk of data loss (that the > client won't know to re-write) when the server crashes. > > rick >> regards, >> Johan Hendriks >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" Yes i do know the risk. The thing is we want a dataset shared to a ESXi client using NFS. I use NFS for my normal usage, (sharing ports tree and so on.) but now we want to use it to share a ZFS dataset for a ESXi client. We use iscsi now, but this way we miss some zfs goodies. like snapshots(not a zvol) and most important, we can reach the files directly. But with a virtual machine shared over NFS i get horrible performance. If i copy a file to whatever virtual machine from a windows client shared with iscsi , i get arround 80Mb per second (in the windows copy window) almost at a steady pace. we are really pleased with that. !! If i copy a virtual machine to the NFS share, fire it up, and do a file copy, it never gets higher than 50 Mb and it sometimes drop to 1 Mb then goes to 20 back to 10 and so on. Also the machines feels sluggish in performance. Are there other less dangerous things i can try to boost performance? regards, Johan Hendriks From owner-freebsd-fs@FreeBSD.ORG Tue Dec 13 16:25:39 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D65AE106564A for ; Tue, 13 Dec 2011 16:25:39 +0000 (UTC) (envelope-from peter.maloney@brockmann-consult.de) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.187]) by mx1.freebsd.org (Postfix) with ESMTP id 838EA8FC0C for ; Tue, 13 Dec 2011 16:25:39 +0000 (UTC) Received: from [10.3.0.26] ([141.4.215.32]) by mrelayeu.kundenserver.de (node=mreu0) with ESMTP (Nemesis) id 0M7Fkw-1QmVK02CmM-00xCoj; Tue, 13 Dec 2011 17:25:38 +0100 Message-ID: <4EE77C81.1060807@brockmann-consult.de> Date: Tue, 13 Dec 2011 17:25:37 +0100 From: Peter Maloney User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.23) Gecko/20110922 Thunderbird/3.1.15 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <1888801930.136947.1323790912860.JavaMail.root@erie.cs.uoguelph.ca> <4EE77ADC.9050501@gmail.com> In-Reply-To: <4EE77ADC.9050501@gmail.com> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Provags-ID: V02:K0:Cqe+irjarl668+dfS2R53//XghC+YMxXTIgsIMled5n m+NM8CUFObcuP1x0ekV4/s7NUeGtp+8URlBxJOC6b00C69MsZ1 OH6OfjeW08tgAE8bwzWQkQqX6OKYzPanFK94B7LzzgfDHN91PN OKfpqdUx1ssGHAFUxCt2tTTVHDepxqH5wLdALB21CK1mNIIIfA V24/cd3QuK5gYgsbQC50T5815bP1e8wVSvur+GhZFZ4bj/pSu0 plnyVYOipDaItDMkZzsFaCR/Ty4YneyRtSB4FiPY+5U1d/A+hn Fu1MwwxgyNlDw77esJ7s15E+pXjbbBrxpoLMyg7dRL6zsXFFH0 8d/pv2Nh88tGkXAZCKUHe4mH+qrzQiTbssMl2XgNC Subject: Re: FreeBSD 9.0 and NFS async X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 16:25:40 -0000 On 12/13/2011 05:18 PM, Johan Hendriks wrote: > Rick Macklem schreef: >> Johan Hendriks wrote: >>> Hello all. >>> >>> I used to use async on my 8.x nfs servers! >>> On the FreeBSD 9.0 server i can not do it through the old 8.x sysctl. >>> >>> Is there an other way to set async on FreeBSD 9.x >>> >> You have two choices: >> 1 - Apply this patch to your NFS server's kernel sources and then set >> vfs.nfsd.async=1 >> http://people.freebsd.org/~rmacklem/async.patch >> >> 2 - switch to using the old server by setting >> oldnfs_server_enable="YES" >> in your /etc/rc.conf and then setting the sysctl. >> >> I'll assume that you realize that doing this violates the NFS RFCs >> because >> it runs your server in a way where there is a risk of data loss (that >> the >> client won't know to re-write) when the server crashes. >> >> rick >>> regards, >>> Johan Hendriks >>> _______________________________________________ >>> freebsd-fs@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > Yes i do know the risk. > > The thing is we want a dataset shared to a ESXi client using NFS. > I use NFS for my normal usage, (sharing ports tree and so on.) but now > we want to use it to share a ZFS dataset for a ESXi client. > We use iscsi now, but this way we miss some zfs goodies. like > snapshots(not a zvol) and most important, we can reach the files > directly. > > But with a virtual machine shared over NFS i get horrible performance. > If i copy a file to whatever virtual machine from a windows client > shared with iscsi , i get arround 80Mb per second (in the windows copy > window) almost at a steady pace. we are really pleased with that. !! > If i copy a virtual machine to the NFS share, fire it up, and do a > file copy, it never gets higher than 50 Mb and it sometimes drop to 1 > Mb then goes to 20 back to 10 and so on. > Also the machines feels sluggish in performance. > That is interesting. I am doing the same thing (and settling for the horrible 5-9MB/s writes right now with a decent ZIL that goes 65 MB/s with any other NFS sync client). But without any NFS settings, I found I can simply run: # zfs set sync=disabled tank/esxidatasetname with the same file integrity sacrifice Rick mentions above. I found that vfs.nfsd.async=1 doesn't actually do anything for me (8.2-STABLE Sept 29th). > Are there other less dangerous things i can try to boost performance? > > regards, > Johan Hendriks > > > > > > > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" -- -------------------------------------------- Peter Maloney Brockmann Consult Max-Planck-Str. 2 21502 Geesthacht Germany Tel: +49 4152 889 300 Fax: +49 4152 889 333 E-mail: peter.maloney@brockmann-consult.de Internet: http://www.brockmann-consult.de -------------------------------------------- From owner-freebsd-fs@FreeBSD.ORG Tue Dec 13 16:29:18 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 29CA61065672 for ; Tue, 13 Dec 2011 16:29:18 +0000 (UTC) (envelope-from joh.hendriks@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id B1E6A8FC12 for ; Tue, 13 Dec 2011 16:29:17 +0000 (UTC) Received: by eaaf13 with SMTP id f13so1651243eaa.13 for ; Tue, 13 Dec 2011 08:29:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=AlYaFNNkPtO6h3j0iMLlzLXSaI1NRmhV8Tb7ZIEZzXk=; b=ZfrPd1Sdz9rTY2pHuZLqgUUMzRf71+ZvZzxk+q2wlh5bRBwG6y+8fNOTbWRxJq+Bji 0jiRdeofGXUO8xkeurArFx5m1sXI5NXqKkrTQX2z4ME2onqLbqbg7yX4qNmNhftSLVT7 jat/mf0pulitET/AgBwfnr/nYqpdcjJfyH7dk= Received: by 10.204.155.65 with SMTP id r1mr13055777bkw.110.1323793756657; Tue, 13 Dec 2011 08:29:16 -0800 (PST) Received: from [192.168.1.129] (schavemaker.nl. [213.84.84.186]) by mx.google.com with ESMTPS id t6sm5745339bka.12.2011.12.13.08.29.15 (version=SSLv3 cipher=OTHER); Tue, 13 Dec 2011 08:29:15 -0800 (PST) Message-ID: <4EE77D59.2020008@gmail.com> Date: Tue, 13 Dec 2011 17:29:13 +0100 From: Johan Hendriks User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <1888801930.136947.1323790912860.JavaMail.root@erie.cs.uoguelph.ca> <4EE77ADC.9050501@gmail.com> <4EE77C81.1060807@brockmann-consult.de> In-Reply-To: <4EE77C81.1060807@brockmann-consult.de> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: FreeBSD 9.0 and NFS async X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 16:29:18 -0000 Peter Maloney schreef: > That is interesting. I am doing the same thing (and settling for the > horrible 5-9MB/s writes right now with a decent ZIL that goes 65 MB/s > with any other NFS sync client). But without any NFS settings, I found I > can simply run: > # zfs set sync=disabled tank/esxidatasetname > with the same file integrity sacrifice Rick mentions above. > > I found that > vfs.nfsd.async=1 > doesn't actually do anything for me (8.2-STABLE Sept 29th). Ok i will try that, and leave the NFS server alone. thanks, i will report back with my findings. regards Johan Hendrisk From owner-freebsd-fs@FreeBSD.ORG Tue Dec 13 18:17:11 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CF1131065672 for ; Tue, 13 Dec 2011 18:17:11 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta02.emeryville.ca.mail.comcast.net (qmta02.emeryville.ca.mail.comcast.net [76.96.30.24]) by mx1.freebsd.org (Postfix) with ESMTP id B5DEF8FC0C for ; Tue, 13 Dec 2011 18:17:11 +0000 (UTC) Received: from omta22.emeryville.ca.mail.comcast.net ([76.96.30.89]) by qmta02.emeryville.ca.mail.comcast.net with comcast id 8i3Q1i0011vN32cA2iH4Wo; Tue, 13 Dec 2011 18:17:04 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta22.emeryville.ca.mail.comcast.net with comcast id 8iZV1i0041t3BNj8iiZVmZ; Tue, 13 Dec 2011 18:33:29 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 40A9A102C19; Tue, 13 Dec 2011 10:17:10 -0800 (PST) Date: Tue, 13 Dec 2011 10:17:10 -0800 From: Jeremy Chadwick To: siur Message-ID: <20111213181710.GA81059@icarus.home.lan> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs@freebsd.org Subject: Re: epic UFS crash X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 18:17:11 -0000 On Tue, Dec 13, 2011 at 05:54:26PM +0400, siur wrote: > > Just curious, what type of RAID controller was this? > > Oh, I don't remember. Some built-in intel controller. Intel MatrixRAID hasn't worked well on FreeBSD for quite some time. See Operating System Support, note the bugs: http://en.wikipedia.org/wiki/Intel_Rapid_Storage_Technology Be aware there is, in RELENG_8 and newer, better support for this using the graid(8) utility. I have been meaning to test this for months now, but haven't had the time. You'd probably be better off scrapping use of the built-in Intel stuff and going with gmirror(8). -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Tue Dec 13 18:19:00 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A0839106566B for ; Tue, 13 Dec 2011 18:19:00 +0000 (UTC) (envelope-from mckusick@mckusick.com) Received: from chez.mckusick.com (unknown [IPv6:2001:5a8:4:7e72:4a5b:39ff:fe12:452]) by mx1.freebsd.org (Postfix) with ESMTP id 622A88FC1D for ; Tue, 13 Dec 2011 18:19:00 +0000 (UTC) Received: from chez.mckusick.com (localhost [127.0.0.1]) by chez.mckusick.com (8.14.3/8.14.3) with ESMTP id pBDIIlPm071707; Tue, 13 Dec 2011 10:18:47 -0800 (PST) (envelope-from mckusick@chez.mckusick.com) Message-Id: <201112131818.pBDIIlPm071707@chez.mckusick.com> To: Dieter BSD In-reply-to: <20111213091457.GU50300@deviant.kiev.zoral.com.ua> Date: Tue, 13 Dec 2011 10:18:47 -0800 From: Kirk McKusick X-Spam-Status: No, score=0.0 required=5.0 tests=MISSING_MID, UNPARSEABLE_RELAY autolearn=failed version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on chez.mckusick.com Cc: freebsd-fs@freebsd.org Subject: Re: Maximum blocksize for FFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 18:19:00 -0000 The default blocksize in FreeBSD 9.0 is 32K/4K. We have been running with this size in -current for a almost a year with no reported problems. Kirk McKusick From owner-freebsd-fs@FreeBSD.ORG Tue Dec 13 22:51:56 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 116B51065672 for ; Tue, 13 Dec 2011 22:51:56 +0000 (UTC) (envelope-from rabe@uugrn.org) Received: from mail.uugrn.org (mail.uugrn.org [195.49.138.123]) by mx1.freebsd.org (Postfix) with ESMTP id 8D2C68FC14 for ; Tue, 13 Dec 2011 22:51:55 +0000 (UTC) Received: from rabe.uugrn.org (root@rabe.uugrn.org [195.49.138.102]) by mail.uugrn.org (8.14.4/8.14.3) with ESMTP id pBDMphHl005917 for ; Tue, 13 Dec 2011 23:51:53 +0100 (CET) (envelope-from rabe@uugrn.org) Received: from rabox.fritz.box (rabe@rabe.uugrn.org [195.49.138.102]) by rabe.uugrn.org (8.14.4/8.13.8) with ESMTP id pBDMphmV005913 for ; Tue, 13 Dec 2011 23:51:43 +0100 (CET) (envelope-from rabe@uugrn.org) Received: from rabox.fritz.box (localhost [127.0.0.1]) by rabox.fritz.box (8.14.4/8.14.4/Debian-2ubuntu1) with ESMTP id pBDMpe9a005620 for ; Tue, 13 Dec 2011 23:51:40 +0100 Received: (from rabe@localhost) by rabox.fritz.box (8.14.4/8.14.4/Submit) id pBDMpeVm005618 for freebsd-fs@freebsd.org; Tue, 13 Dec 2011 23:51:40 +0100 X-Authentication-Warning: rabox.fritz.box: rabe set sender to rabe@uugrn.org using -f Date: Tue, 13 Dec 2011 23:51:40 +0100 From: Raphael Eiselstein To: freebsd-fs@freebsd.org Message-ID: <20111213225140.GA4085@ma.sigsys.de> References: <20111212221533.GB3696@ma.sigsys.de> <4EE68474.4000601@feral.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="tKW2IUtsqtDRztdT" Content-Disposition: inline In-Reply-To: <4EE68474.4000601@feral.com> User-Agent: Mutt/1.5.21 (2010-09-15) Subject: Re: [XFS][9.0RC2] xfs_repair hangs on umtxn X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 22:51:56 -0000 --tKW2IUtsqtDRztdT Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Dec 12, 2011 at 02:47:16PM -0800, Matthew Jacob wrote: > Oh, XFS works at all? That's news to me. read-only seems to work: fbsd9rc2# kldload xfs fbsd9rc2# mount -t xfs -o ro /dev/md0 /mnt fbsd9rc2# df -h /mnt Filesystem Size Used Avail Capacity Mounted on /dev/md0 5G 144k 5G 0% /mnt read-write won't: fbsd9rc2# mount -t xfs /dev/md0 /mnt mount: /dev/md0 : Operation not permitted I wonder what makes xfs_repair hanging. Regards Raphael --=20 Raphael Eiselstein http://rabe.uugrn.org/ xmpp:freibyter@gmx.de | https://www.xing.com/profile/Raphael_Eiselstein = =20 GnuPG: E7B2 1D66 3AF2 EDC7 9828 6D7A 9CDA 3E7B 10CA 9F2D =2E........|.........|.........|.........|.........|.........|.........|.. --tKW2IUtsqtDRztdT Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAk7n1vwACgkQnNo+exDKny3KGQCgnsnYRedrEOaAygW9n1kPIb0H tYcAoJKdUIRsWzEgpVSE+YeYRsqbkApd =7Lqz -----END PGP SIGNATURE----- --tKW2IUtsqtDRztdT-- From owner-freebsd-fs@FreeBSD.ORG Wed Dec 14 00:35:18 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0D9E2106564A for ; Wed, 14 Dec 2011 00:35:18 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 263568FC12 for ; Wed, 14 Dec 2011 00:35:16 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqAEAObt506DaFvO/2dsb2JhbABDFoRypx2BcgEBAQMBAQEBICsgCwUWGAICDRkCKQEJJgYIBwQBHASHWwikYJFRgS+CRoZogRYEiDGKHIIlkkc X-IronPort-AV: E=Sophos;i="4.71,349,1320642000"; d="scan'208";a="150179725" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 13 Dec 2011 19:35:15 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id EF526B3F3C; Tue, 13 Dec 2011 19:35:15 -0500 (EST) Date: Tue, 13 Dec 2011 19:35:15 -0500 (EST) From: Rick Macklem To: Peter Maloney Message-ID: <466034446.173998.1323822915962.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <4EE77C81.1060807@brockmann-consult.de> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-fs@freebsd.org Subject: Re: FreeBSD 9.0 and NFS async X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 00:35:18 -0000 Peter Maloney wrote: > On 12/13/2011 05:18 PM, Johan Hendriks wrote: > > Rick Macklem schreef: > >> Johan Hendriks wrote: > >>> Hello all. > >>> > >>> I used to use async on my 8.x nfs servers! > >>> On the FreeBSD 9.0 server i can not do it through the old 8.x > >>> sysctl. > >>> > >>> Is there an other way to set async on FreeBSD 9.x > >>> > >> You have two choices: > >> 1 - Apply this patch to your NFS server's kernel sources and then > >> set > >> vfs.nfsd.async=1 > >> http://people.freebsd.org/~rmacklem/async.patch > >> > >> 2 - switch to using the old server by setting > >> oldnfs_server_enable="YES" > >> in your /etc/rc.conf and then setting the sysctl. > >> > >> I'll assume that you realize that doing this violates the NFS RFCs > >> because > >> it runs your server in a way where there is a risk of data loss > >> (that > >> the > >> client won't know to re-write) when the server crashes. > >> > >> rick > >>> regards, > >>> Johan Hendriks > >>> _______________________________________________ > >>> freebsd-fs@freebsd.org mailing list > >>> http://lists.freebsd.org/mailman/listinfo/freebsd-fs > >>> To unsubscribe, send any mail to > >>> "freebsd-fs-unsubscribe@freebsd.org" > > Yes i do know the risk. > > > > The thing is we want a dataset shared to a ESXi client using NFS. > > I use NFS for my normal usage, (sharing ports tree and so on.) but > > now > > we want to use it to share a ZFS dataset for a ESXi client. > > We use iscsi now, but this way we miss some zfs goodies. like > > snapshots(not a zvol) and most important, we can reach the files > > directly. > > > > But with a virtual machine shared over NFS i get horrible > > performance. > > If i copy a file to whatever virtual machine from a windows client > > shared with iscsi , i get arround 80Mb per second (in the windows > > copy > > window) almost at a steady pace. we are really pleased with that. !! > > If i copy a virtual machine to the NFS share, fire it up, and do a > > file copy, it never gets higher than 50 Mb and it sometimes drop to > > 1 > > Mb then goes to 20 back to 10 and so on. > > Also the machines feels sluggish in performance. > > > That is interesting. I am doing the same thing (and settling for the > horrible 5-9MB/s writes right now with a decent ZIL that goes 65 MB/s > with any other NFS sync client). But without any NFS settings, I found > I > can simply run: > # zfs set sync=disabled tank/esxidatasetname > with the same file integrity sacrifice Rick mentions above. > > I found that > vfs.nfsd.async=1 > doesn't actually do anything for me (8.2-STABLE Sept 29th). This sysctl won't exist unless you apply the patch (that isn't even in head yet). For FreeBSD8.n, the default server is the old one and vfs.nfsrv.async=1 would be the sysctl. (If that was what you meant, then don't worry about the above comments, rick) > > > Are there other less dangerous things i can try to boost > > performance? > > > > regards, > > Johan Hendriks > > > > > > > > > > > > > > > > _______________________________________________ > > freebsd-fs@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > > To unsubscribe, send any mail to > > "freebsd-fs-unsubscribe@freebsd.org" > > > -- > > -------------------------------------------- > Peter Maloney > Brockmann Consult > Max-Planck-Str. 2 > 21502 Geesthacht > Germany > Tel: +49 4152 889 300 > Fax: +49 4152 889 333 > E-mail: peter.maloney@brockmann-consult.de > Internet: http://www.brockmann-consult.de > -------------------------------------------- > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Wed Dec 14 00:38:08 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 42A5A1065670 for ; Wed, 14 Dec 2011 00:38:08 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id EFD028FC15 for ; Wed, 14 Dec 2011 00:38:07 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqAEAErv506DaFvO/2dsb2JhbABDFoRypx6BcgEBAQMBAQEBICsgCwUWDgoCAg0ZAikBCSYGCAcEARwEh1sIpFqRUoEvgkaGaIEWBIgxihyCJZJH X-IronPort-AV: E=Sophos;i="4.71,349,1320642000"; d="scan'208";a="148384619" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 13 Dec 2011 19:38:06 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id DDA14B3EB1; Tue, 13 Dec 2011 19:38:06 -0500 (EST) Date: Tue, 13 Dec 2011 19:38:06 -0500 (EST) From: Rick Macklem To: Johan Hendriks Message-ID: <961254586.174058.1323823086880.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <4EE77ADC.9050501@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-fs@freebsd.org Subject: Re: FreeBSD 9.0 and NFS async X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 00:38:08 -0000 Johan Hendriks wrote: > Rick Macklem schreef: > > Johan Hendriks wrote: > >> Hello all. > >> > >> I used to use async on my 8.x nfs servers! > >> On the FreeBSD 9.0 server i can not do it through the old 8.x > >> sysctl. > >> > >> Is there an other way to set async on FreeBSD 9.x > >> > > You have two choices: > > 1 - Apply this patch to your NFS server's kernel sources and then > > set > > vfs.nfsd.async=1 > > http://people.freebsd.org/~rmacklem/async.patch > > > > 2 - switch to using the old server by setting > > oldnfs_server_enable="YES" > > in your /etc/rc.conf and then setting the sysctl. > > > > I'll assume that you realize that doing this violates the NFS RFCs > > because > > it runs your server in a way where there is a risk of data loss > > (that the > > client won't know to re-write) when the server crashes. > > > > rick > >> regards, > >> Johan Hendriks > >> _______________________________________________ > >> freebsd-fs@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-fs > >> To unsubscribe, send any mail to > >> "freebsd-fs-unsubscribe@freebsd.org" > Yes i do know the risk. > > The thing is we want a dataset shared to a ESXi client using NFS. > I use NFS for my normal usage, (sharing ports tree and so on.) but now > we want to use it to share a ZFS dataset for a ESXi client. > We use iscsi now, but this way we miss some zfs goodies. like > snapshots(not a zvol) and most important, we can reach the files > directly. > > But with a virtual machine shared over NFS i get horrible performance. > If i copy a file to whatever virtual machine from a windows client > shared with iscsi , i get arround 80Mb per second (in the windows copy > window) almost at a steady pace. we are really pleased with that. !! > If i copy a virtual machine to the NFS share, fire it up, and do a > file > copy, it never gets higher than 50 Mb and it sometimes drop to 1 Mb > then > goes to 20 back to 10 and so on. > Also the machines feels sluggish in performance. > > Are there other less dangerous things i can try to boost performance? > I don't use ZFS, but others have reported using a dedicated SSD that has good write performance for the ZIL log in order to get better write performance for ZFS. > regards, > Johan Hendriks From owner-freebsd-fs@FreeBSD.ORG Wed Dec 14 02:36:30 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 448C81065670; Wed, 14 Dec 2011 02:36:30 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id CAA478FC14; Wed, 14 Dec 2011 02:36:28 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqAEAH8L6E6DaFvO/2dsb2JhbABDFoRypx6BcgEBAQMBAQEBICsdAwsFFhgCAg0ZAikBCSYGCAcEAQgUBIdbCKR4kWGBL4JGhmiBFgSIMYocgiWSRw X-IronPort-AV: E=Sophos;i="4.71,349,1320642000"; d="scan'208";a="148393557" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 13 Dec 2011 21:36:08 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id B6354B3F64; Tue, 13 Dec 2011 21:36:08 -0500 (EST) Date: Tue, 13 Dec 2011 21:36:08 -0500 (EST) From: Rick Macklem To: John Message-ID: <1513776106.179512.1323830168731.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <20111213024636.GA47103@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: multihomed nfs server - NLM lock failure on additional interfaces X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 02:36:30 -0000 John De wrote: > Hi Folks, > > I have a 9-prerelease system where I've been testing nfs/zfs. The > system has been working quite well until moving the server to a > multihomed > configuration. > > Given the following: > > nfsd: master (nfsd) > nfsd: server (nfsd) > /usr/sbin/rpcbind -h 10.24.6.38 -h 172.1.1.2 -h 172.21.201.1 -h > 172.21.202.1 -h 172.21.203.1 -h 172.21.204.1 -h 172.21.205.1 -h > 10.24.6.34 -h 10.24.6.33 > /usr/sbin/mountd -r -l -h 10.24.6.38 -h 172.1.1.2 -h 172.21.201.1 -h > 172.21.202.1 -h 172.21.203.1 -h 172.21.204.1 -h 172.21.205.1 -h > 10.24.6.34 -h 10.24.6.33 > /usr/sbin/rpc.statd -h 10.24.6.38 -h 172.1.1.2 -h 172.21.201.1 -h > 172.21.202.1 -h 172.21.203.1 -h 172.21.204.1 -h 172.21.205.1 -h > 10.24.6.34 -h 10.24.6.33 > /usr/sbin/rpc.lockd -h 10.24.6.38 -h 172.1.1.2 -h 172.21.201.1 -h > 172.21.202.1 -h 172.21.203.1 -h 172.21.204.1 -h 172.21.205.1 -h > 10.24.6.34 -h 10.24.6.33 > > 10.24.6.38 is the default interface on 1G. The 172 nets are 10G > connected > to compute systems. > > ifconfig_bce0=' inet 10.24.6.38 netmask 255.255.0.0 -rxcsum -txcsum' > _c='physical addr which never changes' > ifconfig_bce1=' inet 172.1.1.2 netmask 255.255.255.0' _c='physcial > addr on crossover cable' > ifconfig_cxgb2='inet 172.21.21.129 netmask 255.255.255.0' _c='physical > backside 10g compute net' > ifconfig_cxgb3='inet 172.21.201.1 netmask 255.255.255.0 mtu 9000' > _c='physical backside 10g compute net' > ifconfig_cxgb6='inet 172.21.202.1 netmask 255.255.255.0 mtu 9000' > _c='physical backside 10g compute net' > ifconfig_cxgb8='inet 172.21.203.1 netmask 255.255.255.0 mtu 9000' > _c='physical backside 10g compute net' > ifconfig_cxgb4='inet 172.21.204.1 netmask 255.255.255.0 mtu 9000' > _c='physical backside 10g compute net' > ifconfig_cxgb0='inet 172.21.205.1 netmask 255.255.255.0 mtu 9000' > _c='physical backside 10g compute net' > > The 10.24.6.34 and 10.24.6.33 are alias addresses for the system. > > Destination Gateway Flags Refs Use Netif Expire > default 10.24.0.1 UGS 0 1049 bce0 > > > The server works correctly (and quite well) for both udp & tcp mounts. > Basically, all nfs traffic is great! > > However, locking only works for clients connected to the 10.24.6.38 > interface. > > A tcpdump file from good & bad runs: > > http://www.freebsd.org/~jwd/lockgood.pcap > http://www.freebsd.org/~jwd/lockbad.pcap > > Basically, the clients (both FreeBSD & Linux) query the servers > rpcbind > for the address of the nlm which is returned correctly. For the good > run, the > NLM is then called. For the bad call, it is not. > Well, first off I think your packet traces are missing packets. If you look at nlm_get_rpc(), which is the function in sys/nlm/nlm_prot_impl.c that is doing this, you will see that it first attempts UDP and then falls back to TCP when talking to rpcbind. Your packet traces only show TCP, so I suspect that the UDP case went through a different interface (or missed getting captured some other way?). My guess would be that the attempt to connect to the server's NLM does the same thing, since the lockbad.pcap doesn't show any SYN,... to port 844. If I were you, I'd put lottsa printfs in nlm_get_rpc() showing what is in the address structure "ss" and, in particular when it calls clnt_reconnect_create(). { For the client. } For the server, it starts at sys_nlm_syscall(), which calls ... until you get to nlm_register_services(). It copies in a list of address(es) and I would printf those address(es) once copied into the kernel, to see if they make sense. These are the address(es) that are going to get sobind()'d later by a function called svn_tli_create() { over is sys/rpc/rpc_generic.c }. That's as far as I got. Good luck with it, rick > I've started digging through code, but I do not claim to be an rpc > expert. > If anyone has suggestions I would appreciate any pointers. > > Thanks! > John > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to > "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Wed Dec 14 07:08:16 2011 Return-Path: Delivered-To: fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CA873106566B; Wed, 14 Dec 2011 07:08:16 +0000 (UTC) (envelope-from Karli.Sjoberg@slu.se) Received: from edge1-1.slu.se (edge1-1.slu.se [193.10.100.96]) by mx1.freebsd.org (Postfix) with ESMTP id 45DCA8FC0A; Wed, 14 Dec 2011 07:08:15 +0000 (UTC) Received: from Exchange1.ad.slu.se (193.10.100.94) by edge1-1.slu.se (193.10.100.96) with Microsoft SMTP Server (TLS) id 8.3.213.0; Wed, 14 Dec 2011 08:08:12 +0100 Received: from exmbx3.ad.slu.se ([193.10.100.93]) by Exchange1.ad.slu.se ([193.10.100.94]) with mapi; Wed, 14 Dec 2011 08:08:12 +0100 From: =?iso-8859-1?Q?Karli_Sj=F6berg?= To: Andriy Gapon Date: Wed, 14 Dec 2011 08:08:11 +0100 Thread-Topic: Consistant panics trying to access zfs filesystems replicated from Sun/Oracle appliance Thread-Index: Acy6LyR8tNWUH9P3T1+pSN6XTXsvrA== Message-ID: <385AB6A6-6D37-4D6D-A11B-93F91FAC4A8A@slu.se> References: <82B38DBF-DD3A-46CD-93F6-02CDB6506E05@slu.se> <73B607D0-8C3E-4980-B901-B986E060D32E@slu.se> <4EE7754D.3050605@FreeBSD.org> In-Reply-To: <4EE7754D.3050605@FreeBSD.org> Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: sv-SE, en-US MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "Kenneth D. Merry" , "fs@freebsd.org" Subject: Re: Consistant panics trying to access zfs filesystems replicated from Sun/Oracle appliance X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 07:08:16 -0000 Hi Andriy, I would love to do anything that could help resolve this issue. Unfortunate= ly, I have never ever tried to debug any kind of programming before in my l= ife. But I believe myself to be somewhere around the capacity of a somewhat= trained monkey, capable of following clear instructions. I would also agre= e to create an account for any person who=B4d give this debugging a try. Best Regards Karli Sj=F6berg 13 dec 2011 kl. 16.54 skrev Andriy Gapon: on 12/12/2011 10:46 Karli Sj=F6berg said the following: I got the panic from the IPMI KVM: http://i55.tinypic.com/synpzk.png I think that your best bet is to debug what's going on in zfs_fuid_table_lo= ad function. Add a few printfs in it to see what values get passed to avl_add= calls. It would make sense to print f_idx and f_ksid->kd_name fields for each domn= ode. It looks like a duplicate entry is getting inserted into one of the trees. -- Andriy Gapon Med V=E4nliga H=E4lsningar ---------------------------------------------------------------------------= ---- Karli Sj=F6berg Swedish University of Agricultural Sciences Box 7079 (Visiting Address Kron=E5sv=E4gen 8) S-750 07 Uppsala, Sweden Phone: +46-(0)18-67 15 66 karli.sjoberg@slu.se From owner-freebsd-fs@FreeBSD.ORG Wed Dec 14 07:45:07 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5D03A1065670 for ; Wed, 14 Dec 2011 07:45:07 +0000 (UTC) (envelope-from peter.maloney@brockmann-consult.de) Received: from mo-p05-ob6.rzone.de (mo-p05-ob6.rzone.de [IPv6:2a01:238:20a:202:53f5::1]) by mx1.freebsd.org (Postfix) with ESMTP id BE1898FC0C for ; Wed, 14 Dec 2011 07:45:06 +0000 (UTC) X-RZG-AUTH: :LWIKdA2leu0bPbLmhzXgqn0MTG6qiKEwQRWfNxSw4HzYIwjsnvdDt2QV8d370WOpHNvplOs= X-RZG-CLASS-ID: mo05 Received: from [192.168.179.42] (hmbg-4d06f666.pool.mediaWays.net [77.6.246.102]) by smtp.strato.de (klopstock mo7) (RZmta 26.15 DYNA|AUTH) with (DHE-RSA-AES128-SHA encrypted) ESMTPA id 604ae0nBE6OsBo for ; Wed, 14 Dec 2011 08:44:54 +0100 (MET) Message-ID: <4EE853F4.7060003@brockmann-consult.de> Date: Wed, 14 Dec 2011 08:44:52 +0100 From: Peter Maloney User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <961254586.174058.1323823086880.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <961254586.174058.1323823086880.JavaMail.root@erie.cs.uoguelph.ca> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Subject: Re: FreeBSD 9.0 and NFS async -off topic about ZFS ZIL devices X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 07:45:07 -0000 Am 14.12.2011 01:38, schrieb Rick Macklem: > Johan Hendriks wrote: >> Rick Macklem schreef: >>> Johan Hendriks wrote: >>>> Hello all. >>>> >>>> I used to use async on my 8.x nfs servers! >>>> On the FreeBSD 9.0 server i can not do it through the old 8.x >>>> sysctl. >>>> >>>> Is there an other way to set async on FreeBSD 9.x >>>> >>> You have two choices: >>> 1 - Apply this patch to your NFS server's kernel sources and then >>> set >>> vfs.nfsd.async=1 >>> http://people.freebsd.org/~rmacklem/async.patch >>> >>> 2 - switch to using the old server by setting >>> oldnfs_server_enable="YES" >>> in your /etc/rc.conf and then setting the sysctl. >>> >>> I'll assume that you realize that doing this violates the NFS RFCs >>> because >>> it runs your server in a way where there is a risk of data loss >>> (that the >>> client won't know to re-write) when the server crashes. >>> >>> rick >>>> regards, >>>> Johan Hendriks >>>> _______________________________________________ >>>> freebsd-fs@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >>>> To unsubscribe, send any mail to >>>> "freebsd-fs-unsubscribe@freebsd.org" >> Yes i do know the risk. >> >> The thing is we want a dataset shared to a ESXi client using NFS. >> I use NFS for my normal usage, (sharing ports tree and so on.) but now >> we want to use it to share a ZFS dataset for a ESXi client. >> We use iscsi now, but this way we miss some zfs goodies. like >> snapshots(not a zvol) and most important, we can reach the files >> directly. >> >> But with a virtual machine shared over NFS i get horrible performance. >> If i copy a file to whatever virtual machine from a windows client >> shared with iscsi , i get arround 80Mb per second (in the windows copy >> window) almost at a steady pace. we are really pleased with that. !! >> If i copy a virtual machine to the NFS share, fire it up, and do a >> file >> copy, it never gets higher than 50 Mb and it sometimes drop to 1 Mb >> then >> goes to 20 back to 10 and so on. >> Also the machines feels sluggish in performance. >> >> Are there other less dangerous things i can try to boost performance? >> > I don't use ZFS, but others have reported using a dedicated SSD that has > good write performance for the ZIL log in order to get better write > performance for ZFS. Indeed, I can get 65 MB/s using a consumer SSD with normal sync nfs clients. But once you use ESXi's NFS client, it will drop to somewhere between 5 and 9 MB/s. Mine currently goes around 7 MB/s. I also tried adding a ramdisk as my ZIL, and then it still only goes 80 MB/s, even though obviously the ramdisk should be able to handle random writes better than anything, and should go GB/s if there was no other bottleneck, not only 80 MB/s (over 10Gbps network, which is tested at about 6 Gbps with 1500 MTU). Anything else has to go through RAM at some point anyway. Unfortunatlely, I didn't test a normal sync nfs client with the ramdisk... I'll put that on my mental todo list. I also tried a UFS zvol, whidh was not using the log, but only went somewhere between 40 and 60 MB/s. Here is a very nice thread with some scores with various SSDs using a normal NFS client. http://forums.freebsd.org/showthread.php?t=23566 Here is my post in the same thread comparing the linux NFS client with sync option, to the ESXi client (writing to a virtual disk filesystem with dd): http://forums.freebsd.org/showpost.php?p=157154&postcount=55 >> regards, >> Johan Hendriks > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Wed Dec 14 12:21:07 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7A247106566B for ; Wed, 14 Dec 2011 12:21:07 +0000 (UTC) (envelope-from cochard@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 369F28FC19 for ; Wed, 14 Dec 2011 12:21:06 +0000 (UTC) Received: by qcse13 with SMTP id e13so673466qcs.13 for ; Wed, 14 Dec 2011 04:21:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type; bh=WHA0AiRdAC2Tl8tR2jHZXcT2GXwKAfmRjGFY1RJivjs=; b=eSB6ILJ675oeVum353lXJEzAtslI2xXXWP+kHlKXv2J5pAixhMTCRcH4iwjwLZX5dS L9+9P03CPYfC0a6+a5U3q6fX3Wu7BGExpuUZjzw5PigvmXJM68wv0pYCRhGrP3A2OV7B +3VsM/eSz+ny6Nk/T+SJFBB3HPSBJLRP6A/e8= Received: by 10.224.192.8 with SMTP id do8mr7006267qab.46.1323863426267; Wed, 14 Dec 2011 03:50:26 -0800 (PST) MIME-Version: 1.0 Sender: cochard@gmail.com Received: by 10.229.230.210 with HTTP; Wed, 14 Dec 2011 03:50:05 -0800 (PST) In-Reply-To: <201112131818.pBDIIlPm071707@chez.mckusick.com> References: <20111213091457.GU50300@deviant.kiev.zoral.com.ua> <201112131818.pBDIIlPm071707@chez.mckusick.com> From: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Date: Wed, 14 Dec 2011 12:50:05 +0100 X-Google-Sender-Auth: h39bPbaifNNEYWi9bSE8vFK7Yz8 Message-ID: To: Kirk McKusick Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-fs@freebsd.org, Dieter BSD Subject: Re: Maximum blocksize for FFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 12:21:07 -0000 On Tue, Dec 13, 2011 at 7:18 PM, Kirk McKusick wrote: > The default blocksize in FreeBSD 9.0 is 32K/4K. We have been > running with this size in -current for a almost a year with no > reported problems. Hi, There is a reported problem: The number of inode was divided by two with FreeBSD 9.0 (PR bin/162659) and this create some problems because "the number of fragments per inode (NFPI) was not adapted to the new default block size" (Bruce Evans'explanation [1]). Regards, Olivier [1] http://lists.freebsd.org/pipermail/freebsd-bugs/2011-December/046713.html From owner-freebsd-fs@FreeBSD.ORG Wed Dec 14 14:13:17 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CE6E7106564A for ; Wed, 14 Dec 2011 14:13:17 +0000 (UTC) (envelope-from joh.hendriks@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 644488FC0A for ; Wed, 14 Dec 2011 14:13:16 +0000 (UTC) Received: by eaaf13 with SMTP id f13so1139437eaa.13 for ; Wed, 14 Dec 2011 06:13:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=6WsnvOpZPwamMqcgAj3QC/CGR1pYwUi6JFlirYMIFWM=; b=Pix6dBxg+mrJ1ryuJIAA95Ov7DTlH06KB1238dej4E2JzsWV2GX1ImYfu5pVV8MJEd wzXMiepOXsXJP2Qdf0FJfIDEqCFLDJmpFZcrai2Ft6r7s3y4FQ3LWbIXOkwWXKV4tiSS gfoxrUF2wu1Iy+/WH7OWec5A1OXljqZxtOA/k= Received: by 10.204.36.8 with SMTP id r8mr914891bkd.129.1323871995900; Wed, 14 Dec 2011 06:13:15 -0800 (PST) Received: from [192.168.50.104] (double-l.xs4all.nl. [80.126.205.144]) by mx.google.com with ESMTPS id fa8sm6206589bkc.14.2011.12.14.06.13.15 (version=SSLv3 cipher=OTHER); Wed, 14 Dec 2011 06:13:15 -0800 (PST) Message-ID: <4EE8AEF8.1000409@gmail.com> Date: Wed, 14 Dec 2011 15:13:12 +0100 From: Johan Hendriks User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <4EE73135.2080702@gmail.com> In-Reply-To: <4EE73135.2080702@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: FreeBSD 9.0 and NFS async X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 14:13:17 -0000 Johan Hendriks schreef: > Hello all. > > I used to use async on my 8.x nfs servers! > On the FreeBSD 9.0 server i can not do it through the old 8.x sysctl. > > Is there an other way to set async on FreeBSD 9.x > > regards, > Johan Hendriks Ok things look good after the zfs set sync=disabled on the NFS datastore. We have a single cpu supermicro server with 16 GB mem running 9.0 RC3 And we have a HP ML 110 with 8 GB mem as standby backup server, also on 9.0 RC3 Both in the same network. Here are the graphs. http://doub.home.xs4all.nl/bench/iscsi-nfs.PNG I did a clone from the SM (supermicro) ISCSI datastore to the SM NFS datastore. grey block I did a clone from the SM ISCSI datastore to the HP ISCSI datastore. red block I did a clone from the SM ISCSI datastore to the HP NFS datastore green block (not finished but it did go steady on). Since this is our first NAS/SAN server and vmware environment, i do not know if this is good, but it feels ok. But maybe we should be embarrassed with these results :D We do not have a HP or IBM SAN solution to test agains. And this is all over normal GB ethernet, not trunked and so on. Also no Jumbo frames. Thanks for the pointers all. regards Johan Hendriks From owner-freebsd-fs@FreeBSD.ORG Wed Dec 14 15:40:09 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1335010656D1 for ; Wed, 14 Dec 2011 15:40:09 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 42F7F8FC13 for ; Wed, 14 Dec 2011 15:40:03 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqAEAHDC6E6DaFvO/2dsb2JhbAApGhaEcqckgXIBAQEDAQEBASArIAsFFhgCAg0ZAikBCSYGCAcEARwEh1kII6UWkgSBL4JKhnqBFgSIMoodgiWSSA X-IronPort-AV: E=Sophos;i="4.71,352,1320642000"; d="scan'208";a="150268748" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 14 Dec 2011 10:40:03 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 311CFB3F2C; Wed, 14 Dec 2011 10:40:03 -0500 (EST) Date: Wed, 14 Dec 2011 10:40:03 -0500 (EST) From: Rick Macklem To: Peter Maloney Message-ID: <1834792825.201945.1323877203185.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <4EE853F4.7060003@brockmann-consult.de> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-fs@freebsd.org Subject: Re: FreeBSD 9.0 and NFS async -off topic about ZFS ZIL devices X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 15:40:09 -0000 Peter Maloney wrote: > Am 14.12.2011 01:38, schrieb Rick Macklem: > > Johan Hendriks wrote: > >> Rick Macklem schreef: > >>> Johan Hendriks wrote: > >>>> Hello all. > >>>> > >>>> I used to use async on my 8.x nfs servers! > >>>> On the FreeBSD 9.0 server i can not do it through the old 8.x > >>>> sysctl. > >>>> > >>>> Is there an other way to set async on FreeBSD 9.x > >>>> > >>> You have two choices: > >>> 1 - Apply this patch to your NFS server's kernel sources and then > >>> set > >>> vfs.nfsd.async=1 > >>> http://people.freebsd.org/~rmacklem/async.patch > >>> > >>> 2 - switch to using the old server by setting > >>> oldnfs_server_enable="YES" > >>> in your /etc/rc.conf and then setting the sysctl. > >>> > >>> I'll assume that you realize that doing this violates the NFS RFCs > >>> because > >>> it runs your server in a way where there is a risk of data loss > >>> (that the > >>> client won't know to re-write) when the server crashes. > >>> > >>> rick > >>>> regards, > >>>> Johan Hendriks > >>>> _______________________________________________ > >>>> freebsd-fs@freebsd.org mailing list > >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-fs > >>>> To unsubscribe, send any mail to > >>>> "freebsd-fs-unsubscribe@freebsd.org" > >> Yes i do know the risk. > >> > >> The thing is we want a dataset shared to a ESXi client using NFS. > >> I use NFS for my normal usage, (sharing ports tree and so on.) but > >> now > >> we want to use it to share a ZFS dataset for a ESXi client. > >> We use iscsi now, but this way we miss some zfs goodies. like > >> snapshots(not a zvol) and most important, we can reach the files > >> directly. > >> > >> But with a virtual machine shared over NFS i get horrible > >> performance. > >> If i copy a file to whatever virtual machine from a windows client > >> shared with iscsi , i get arround 80Mb per second (in the windows > >> copy > >> window) almost at a steady pace. we are really pleased with that. > >> !! > >> If i copy a virtual machine to the NFS share, fire it up, and do a > >> file > >> copy, it never gets higher than 50 Mb and it sometimes drop to 1 Mb > >> then > >> goes to 20 back to 10 and so on. > >> Also the machines feels sluggish in performance. > >> > >> Are there other less dangerous things i can try to boost > >> performance? > >> > > I don't use ZFS, but others have reported using a dedicated SSD that > > has > > good write performance for the ZIL log in order to get better write > > performance for ZFS. > Indeed, I can get 65 MB/s using a consumer SSD with normal sync nfs > clients. But once you use ESXi's NFS client, it will drop to somewhere > between 5 and 9 MB/s. Mine currently goes around 7 MB/s. I also tried > adding a ramdisk as my ZIL, and then it still only goes 80 MB/s, even > though obviously the ramdisk should be able to handle random writes > better than anything, and should go GB/s if there was no other > bottleneck, not only 80 MB/s (over 10Gbps network, which is tested at > about 6 Gbps with 1500 MTU). Anything else has to go through RAM at > some > point anyway. Unfortunatlely, I didn't test a normal sync nfs client > with the ramdisk... I'll put that on my mental todo list. I also tried > a > UFS zvol, whidh was not using the log, but only went somewhere between > 40 and 60 MB/s. > It might be interesting to take a look at the Write RPC requests generated by the ESXi client in wireshark. I wouldn't be surprised if it: #1 - specifies FILE_SYNC for all writes (this is telling the server it "must" commit the data to stable storage before replying and the "hacks" you are discussing make the server lie about/ignore this request) #2 - uses a relatively small block size (if there is a way to tell the ESXi client to use a larger block size, I would try that) If I'm correct w.r.t. the above assumptions, especially #1, all I can say is that the ESXi client is poorly implemented (or worse;-). rick > Here is a very nice thread with some scores with various SSDs using a > normal NFS client. > http://forums.freebsd.org/showthread.php?t=23566 > > Here is my post in the same thread comparing the linux NFS client with > sync option, to the ESXi client (writing to a virtual disk filesystem > with dd): > http://forums.freebsd.org/showpost.php?p=157154&postcount=55 > > > >> regards, > >> Johan Hendriks > > _______________________________________________ > > freebsd-fs@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > > To unsubscribe, send any mail to > > "freebsd-fs-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Wed Dec 14 17:19:33 2011 Return-Path: Delivered-To: fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 09B931065672 for ; Wed, 14 Dec 2011 17:19:33 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 56A358FC19 for ; Wed, 14 Dec 2011 17:19:31 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA09628; Wed, 14 Dec 2011 19:19:28 +0200 (EET) (envelope-from avg@FreeBSD.org) Message-ID: <4EE8DA9F.3030708@FreeBSD.org> Date: Wed, 14 Dec 2011 19:19:27 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111109 Thunderbird/8.0 MIME-Version: 1.0 To: =?ISO-8859-1?Q?Karli_Sj=F6berg?= References: <82B38DBF-DD3A-46CD-93F6-02CDB6506E05@slu.se> <73B607D0-8C3E-4980-B901-B986E060D32E@slu.se> <4EE7754D.3050605@FreeBSD.org> <385AB6A6-6D37-4D6D-A11B-93F91FAC4A8A@slu.se> In-Reply-To: <385AB6A6-6D37-4D6D-A11B-93F91FAC4A8A@slu.se> X-Enigmail-Version: undefined Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Cc: "fs@freebsd.org" Subject: Re: Consistant panics trying to access zfs filesystems replicated from Sun/Oracle appliance X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 17:19:33 -0000 on 14/12/2011 09:08 Karli Sjöberg said the following: > Hi Andriy, > > I would love to do anything that could help resolve this issue. Unfortunately, I > have never ever tried to debug any kind of programming before in my life. But I > believe myself to be somewhere around the capacity of a somewhat trained monkey, > capable of following clear instructions. I would also agree to create an account > for any person who´d give this debugging a try. It seems your system is 8-STABLE amd64? What method of installation/update do you use? Is it binary or through sources? Can you find out at what exact svn revision your system is now? With this information I might be able to provide you with a zfs.ko binary with the debugging printfs. > 13 dec 2011 kl. 16.54 skrev Andriy Gapon: > >> on 12/12/2011 10:46 Karli Sjöberg said the following: >>> I got the panic from the IPMI KVM: >>> http://i55.tinypic.com/synpzk.png >> >> I think that your best bet is to debug what's going on in zfs_fuid_table_load >> function. Add a few printfs in it to see what values get passed to avl_add calls. >> It would make sense to print f_idx and f_ksid->kd_name fields for each domnode. >> It looks like a duplicate entry is getting inserted into one of the trees. >> >> -- >> Andriy Gapon > > > > Med Vänliga Hälsningar > ------------------------------------------------------------------------------- > Karli Sjöberg > Swedish University of Agricultural Sciences > Box 7079 (Visiting Address Kronåsvägen 8) > S-750 07 Uppsala, Sweden > Phone: +46-(0)18-67 15 66 > karli.sjoberg@slu.se > -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Wed Dec 14 22:10:09 2011 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 672E2106566B for ; Wed, 14 Dec 2011 22:10:09 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 4BC348FC1A for ; Wed, 14 Dec 2011 22:10:09 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id pBEMA9T0086835 for ; Wed, 14 Dec 2011 22:10:09 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id pBEMA9vU086834; Wed, 14 Dec 2011 22:10:09 GMT (envelope-from gnats) Date: Wed, 14 Dec 2011 22:10:09 GMT Message-Id: <201112142210.pBEMA9vU086834@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: dfilter@FreeBSD.ORG (dfilter service) Cc: Subject: Re: kern/159232: commit references a PR X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: dfilter service List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 22:10:09 -0000 The following reply was made to PR kern/159232; it has been noted by GNATS. From: dfilter@FreeBSD.ORG (dfilter service) To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/159232: commit references a PR Date: Wed, 14 Dec 2011 22:04:29 +0000 (UTC) Author: pfg Date: Wed Dec 14 22:04:14 2011 New Revision: 228507 URL: http://svn.freebsd.org/changeset/base/228507 Log: Merge ext2_readwrite.c into ext2_vnops.c as done in UFS in r101729. This removes the obfuscations mentioned in ext2_readwrite and places the clustering funtion in a location similar to other UFS-based implementations. No performance or functional changeses are expected from this move. PR: kern/159232 Suggested by: bde Approved by: jhb (mentor) MFC after: 2 weeks Deleted: head/sys/fs/ext2fs/ext2_readwrite.c Modified: head/sys/fs/ext2fs/ext2_vnops.c Modified: head/sys/fs/ext2fs/ext2_vnops.c ============================================================================== --- head/sys/fs/ext2fs/ext2_vnops.c Wed Dec 14 20:57:41 2011 (r228506) +++ head/sys/fs/ext2fs/ext2_vnops.c Wed Dec 14 22:04:14 2011 (r228507) @@ -64,9 +64,13 @@ #include #include +#include +#include #include #include +#include "opt_directio.h" + #include #include @@ -159,8 +163,6 @@ struct vop_vector ext2_fifoops = { .vop_vptofh = ext2_vptofh, }; -#include - /* * A virgin directory (no blushing please). * Note that the type and namlen fields are reversed relative to ext2. @@ -1675,3 +1677,328 @@ bad: vput(tvp); return (error); } + +/* + * Vnode op for reading. + */ +static int +ext2_read(ap) + struct vop_read_args /* { + struct vnode *a_vp; + struct uio *a_uio; + int a_ioflag; + struct ucred *a_cred; + } */ *ap; +{ + struct vnode *vp; + struct inode *ip; + struct uio *uio; + struct m_ext2fs *fs; + struct buf *bp; + daddr_t lbn, nextlbn; + off_t bytesinfile; + long size, xfersize, blkoffset; + int error, orig_resid, seqcount; + int ioflag; + + vp = ap->a_vp; + uio = ap->a_uio; + ioflag = ap->a_ioflag; + + seqcount = ap->a_ioflag >> IO_SEQSHIFT; + ip = VTOI(vp); + +#ifdef INVARIANTS + if (uio->uio_rw != UIO_READ) + panic("%s: mode", "ext2_read"); + + if (vp->v_type == VLNK) { + if ((int)ip->i_size < vp->v_mount->mnt_maxsymlinklen) + panic("%s: short symlink", "ext2_read"); + } else if (vp->v_type != VREG && vp->v_type != VDIR) + panic("%s: type %d", "ext2_read", vp->v_type); +#endif + orig_resid = uio->uio_resid; + KASSERT(orig_resid >= 0, ("ext2_read: uio->uio_resid < 0")); + if (orig_resid == 0) + return (0); + KASSERT(uio->uio_offset >= 0, ("ext2_read: uio->uio_offset < 0")); + fs = ip->i_e2fs; + if (uio->uio_offset < ip->i_size && + uio->uio_offset >= fs->e2fs_maxfilesize) + return (EOVERFLOW); + + for (error = 0, bp = NULL; uio->uio_resid > 0; bp = NULL) { + if ((bytesinfile = ip->i_size - uio->uio_offset) <= 0) + break; + lbn = lblkno(fs, uio->uio_offset); + nextlbn = lbn + 1; + size = blksize(fs, ip, lbn); + blkoffset = blkoff(fs, uio->uio_offset); + + xfersize = fs->e2fs_fsize - blkoffset; + if (uio->uio_resid < xfersize) + xfersize = uio->uio_resid; + if (bytesinfile < xfersize) + xfersize = bytesinfile; + + if (lblktosize(fs, nextlbn) >= ip->i_size) + error = bread(vp, lbn, size, NOCRED, &bp); + else if ((vp->v_mount->mnt_flag & MNT_NOCLUSTERR) == 0) + error = cluster_read(vp, ip->i_size, lbn, size, + NOCRED, blkoffset + uio->uio_resid, seqcount, &bp); + else if (seqcount > 1) { + int nextsize = blksize(fs, ip, nextlbn); + error = breadn(vp, lbn, + size, &nextlbn, &nextsize, 1, NOCRED, &bp); + } else + error = bread(vp, lbn, size, NOCRED, &bp); + if (error) { + brelse(bp); + bp = NULL; + break; + } + + /* + * If IO_DIRECT then set B_DIRECT for the buffer. This + * will cause us to attempt to release the buffer later on + * and will cause the buffer cache to attempt to free the + * underlying pages. + */ + if (ioflag & IO_DIRECT) + bp->b_flags |= B_DIRECT; + + /* + * We should only get non-zero b_resid when an I/O error + * has occurred, which should cause us to break above. + * However, if the short read did not cause an error, + * then we want to ensure that we do not uiomove bad + * or uninitialized data. + */ + size -= bp->b_resid; + if (size < xfersize) { + if (size == 0) + break; + xfersize = size; + } + error = uiomove((char *)bp->b_data + blkoffset, + (int)xfersize, uio); + if (error) + break; + + if (ioflag & (IO_VMIO|IO_DIRECT)) { + /* + * If it's VMIO or direct I/O, then we don't + * need the buf, mark it available for + * freeing. If it's non-direct VMIO, the VM has + * the data. + */ + bp->b_flags |= B_RELBUF; + brelse(bp); + } else { + /* + * Otherwise let whoever + * made the request take care of + * freeing it. We just queue + * it onto another list. + */ + bqrelse(bp); + } + } + + /* + * This can only happen in the case of an error + * because the loop above resets bp to NULL on each iteration + * and on normal completion has not set a new value into it. + * so it must have come from a 'break' statement + */ + if (bp != NULL) { + if (ioflag & (IO_VMIO|IO_DIRECT)) { + bp->b_flags |= B_RELBUF; + brelse(bp); + } else { + bqrelse(bp); + } + } + + if ((error == 0 || uio->uio_resid != orig_resid) && + (vp->v_mount->mnt_flag & MNT_NOATIME) == 0) + ip->i_flag |= IN_ACCESS; + return (error); +} + +/* + * Vnode op for writing. + */ +static int +ext2_write(ap) + struct vop_write_args /* { + struct vnode *a_vp; + struct uio *a_uio; + int a_ioflag; + struct ucred *a_cred; + } */ *ap; +{ + struct vnode *vp; + struct uio *uio; + struct inode *ip; + struct m_ext2fs *fs; + struct buf *bp; + daddr_t lbn; + off_t osize; + int blkoffset, error, flags, ioflag, resid, size, seqcount, xfersize; + + ioflag = ap->a_ioflag; + uio = ap->a_uio; + vp = ap->a_vp; + + seqcount = ioflag >> IO_SEQSHIFT; + ip = VTOI(vp); + +#ifdef INVARIANTS + if (uio->uio_rw != UIO_WRITE) + panic("%s: mode", "ext2_write"); +#endif + + switch (vp->v_type) { + case VREG: + if (ioflag & IO_APPEND) + uio->uio_offset = ip->i_size; + if ((ip->i_flags & APPEND) && uio->uio_offset != ip->i_size) + return (EPERM); + /* FALLTHROUGH */ + case VLNK: + break; + case VDIR: + /* XXX differs from ffs -- this is called from ext2_mkdir(). */ + if ((ioflag & IO_SYNC) == 0) + panic("ext2_write: nonsync dir write"); + break; + default: + panic("ext2_write: type %p %d (%jd,%jd)", (void *)vp, + vp->v_type, (intmax_t)uio->uio_offset, + (intmax_t)uio->uio_resid); + } + + KASSERT(uio->uio_resid >= 0, ("ext2_write: uio->uio_resid < 0")); + KASSERT(uio->uio_offset >= 0, ("ext2_write: uio->uio_offset < 0")); + fs = ip->i_e2fs; + if ((uoff_t)uio->uio_offset + uio->uio_resid > fs->e2fs_maxfilesize) + return (EFBIG); + /* + * Maybe this should be above the vnode op call, but so long as + * file servers have no limits, I don't think it matters. + */ + if (vn_rlimit_fsize(vp, uio, uio->uio_td)) + return (EFBIG); + + resid = uio->uio_resid; + osize = ip->i_size; + if (seqcount > BA_SEQMAX) + flags = BA_SEQMAX << BA_SEQSHIFT; + else + flags = seqcount << BA_SEQSHIFT; + if ((ioflag & IO_SYNC) && !DOINGASYNC(vp)) + flags |= IO_SYNC; + + for (error = 0; uio->uio_resid > 0;) { + lbn = lblkno(fs, uio->uio_offset); + blkoffset = blkoff(fs, uio->uio_offset); + xfersize = fs->e2fs_fsize - blkoffset; + if (uio->uio_resid < xfersize) + xfersize = uio->uio_resid; + if (uio->uio_offset + xfersize > ip->i_size) + vnode_pager_setsize(vp, uio->uio_offset + xfersize); + + /* + * We must perform a read-before-write if the transfer size + * does not cover the entire buffer. + */ + if (fs->e2fs_bsize > xfersize) + flags |= BA_CLRBUF; + else + flags &= ~BA_CLRBUF; + error = ext2_balloc(ip, lbn, blkoffset + xfersize, + ap->a_cred, &bp, flags); + if (error != 0) + break; + + /* + * If the buffer is not valid and we did not clear garbage + * out above, we have to do so here even though the write + * covers the entire buffer in order to avoid a mmap()/write + * race where another process may see the garbage prior to + * the uiomove() for a write replacing it. + */ + if ((bp->b_flags & B_CACHE) == 0 && fs->e2fs_bsize <= xfersize) + vfs_bio_clrbuf(bp); + if ((ioflag & (IO_SYNC|IO_INVAL)) == (IO_SYNC|IO_INVAL)) + bp->b_flags |= B_NOCACHE; + if (uio->uio_offset + xfersize > ip->i_size) + ip->i_size = uio->uio_offset + xfersize; + size = blksize(fs, ip, lbn) - bp->b_resid; + if (size < xfersize) + xfersize = size; + + error = + uiomove((char *)bp->b_data + blkoffset, (int)xfersize, uio); + if (ioflag & (IO_VMIO|IO_DIRECT)) { + bp->b_flags |= B_RELBUF; + } + + /* + * If IO_SYNC each buffer is written synchronously. Otherwise + * if we have a severe page deficiency write the buffer + * asynchronously. Otherwise try to cluster, and if that + * doesn't do it then either do an async write (if O_DIRECT), + * or a delayed write (if not). + */ + if (ioflag & IO_SYNC) { + (void)bwrite(bp); + } else if (vm_page_count_severe() || + buf_dirty_count_severe() || + (ioflag & IO_ASYNC)) { + bp->b_flags |= B_CLUSTEROK; + bawrite(bp); + } else if (xfersize + blkoffset == fs->e2fs_fsize) { + if ((vp->v_mount->mnt_flag & MNT_NOCLUSTERW) == 0) { + bp->b_flags |= B_CLUSTEROK; + cluster_write(vp, bp, ip->i_size, seqcount); + } else { + bawrite(bp); + } + } else if (ioflag & IO_DIRECT) { + bp->b_flags |= B_CLUSTEROK; + bawrite(bp); + } else { + bp->b_flags |= B_CLUSTEROK; + bdwrite(bp); + } + if (error || xfersize == 0) + break; + } + /* + * If we successfully wrote any data, and we are not the superuser + * we clear the setuid and setgid bits as a precaution against + * tampering. + */ + if ((ip->i_mode & (ISUID | ISGID)) && resid > uio->uio_resid && + ap->a_cred) { + if (priv_check_cred(ap->a_cred, PRIV_VFS_RETAINSUGID, 0)) + ip->i_mode &= ~(ISUID | ISGID); + } + if (error) { + if (ioflag & IO_UNIT) { + (void)ext2_truncate(vp, osize, + ioflag & IO_SYNC, ap->a_cred, uio->uio_td); + uio->uio_offset -= resid - uio->uio_resid; + uio->uio_resid = resid; + } + } + if (uio->uio_resid != resid) { + ip->i_flag |= IN_CHANGE | IN_UPDATE; + if (ioflag & IO_SYNC) + error = ext2_update(vp, 1); + } + return (error); +} _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Thu Dec 15 03:51:01 2011 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4E292106564A; Thu, 15 Dec 2011 03:51:01 +0000 (UTC) (envelope-from pfg@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 229F78FC0A; Thu, 15 Dec 2011 03:51:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id pBF3p1la005875; Thu, 15 Dec 2011 03:51:01 GMT (envelope-from pfg@freefall.freebsd.org) Received: (from pfg@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id pBF3p00q005846; Thu, 15 Dec 2011 03:51:00 GMT (envelope-from pfg) Date: Thu, 15 Dec 2011 03:51:00 GMT Message-Id: <201112150351.pBF3p00q005846@freefall.freebsd.org> To: giffunip@tutopia.com, pfg@FreeBSD.org, freebsd-fs@FreeBSD.org, pfg@FreeBSD.org From: pfg@FreeBSD.org Cc: Subject: Re: kern/159232: [ext2fs] [patch] fs/ext2fs: merge ext2_readwrite into ext2_vnops X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2011 03:51:01 -0000 Synopsis: [ext2fs] [patch] fs/ext2fs: merge ext2_readwrite into ext2_vnops State-Changed-From-To: open->patched State-Changed-By: pfg State-Changed-When: Thu Dec 15 03:44:25 UTC 2011 State-Changed-Why: Assign to myself. Patched in head. Responsible-Changed-From-To: freebsd-fs->pfg Responsible-Changed-By: pfg Responsible-Changed-When: Thu Dec 15 03:44:25 UTC 2011 Responsible-Changed-Why: Assign to myself. Applied to head. http://www.freebsd.org/cgi/query-pr.cgi?pr=159232 From owner-freebsd-fs@FreeBSD.ORG Thu Dec 15 11:59:53 2011 Return-Path: Delivered-To: fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AB100106564A; Thu, 15 Dec 2011 11:59:53 +0000 (UTC) (envelope-from Karli.Sjoberg@slu.se) Received: from Edge1-2.slu.se (edge1-2.slu.se [193.10.100.97]) by mx1.freebsd.org (Postfix) with ESMTP id 29E878FC0A; Thu, 15 Dec 2011 11:59:52 +0000 (UTC) Received: from Exchange1.ad.slu.se (193.10.100.94) by Edge1-2.slu.se (193.10.100.97) with Microsoft SMTP Server (TLS) id 8.3.213.0; Thu, 15 Dec 2011 12:59:21 +0100 Received: from exmbx3.ad.slu.se ([193.10.100.93]) by Exchange1.ad.slu.se ([193.10.100.94]) with mapi; Thu, 15 Dec 2011 12:59:21 +0100 From: =?iso-8859-1?Q?Karli_Sj=F6berg?= To: "fs@freebsd.org" , Jeremy Chadwick , "Kenneth D. Merry" Date: Thu, 15 Dec 2011 12:59:20 +0100 Thread-Topic: Consistant panics trying to access zfs filesystems replicated from Sun/Oracle appliance Thread-Index: Acy7IPsFWHzSUlTvRy+iTYEhIjHFSw== Message-ID: <558C926F-14FA-458D-BB8E-D20BA46BE6D2@slu.se> References: <82B38DBF-DD3A-46CD-93F6-02CDB6506E05@slu.se> <73B607D0-8C3E-4980-B901-B986E060D32E@slu.se> <4EE7754D.3050605@FreeBSD.org> In-Reply-To: <4EE7754D.3050605@FreeBSD.org> Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: sv-SE, en-US MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Andriy Gapon Subject: Re: Consistant panics trying to access zfs filesystems replicated from Sun/Oracle appliance X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2011 11:59:53 -0000 Hi all, with the help of Andriy Gapon, I managed to capture what happened: # cd /export/Portfolio/ci (TAB) http://oi40.tinypic.com/b3lsog.jpg # cd /export/Portfolio/cifs_share http://oi42.tinypic.com/6e40op.jpg # ls /export/Portfolio/cifs_share http://oi42.tinypic.com/23rn60j.jpg And this was Andriy=B4s response: Hmm, so it adds the "FreeBSD" string twice. I am not sure what that means, consider sharing this result with the publi= c, maybe someone will have a better idea. -- Andriy Gapon 13 dec 2011 kl. 16.54 skrev Andriy Gapon: on 12/12/2011 10:46 Karli Sj=F6berg said the following: I got the panic from the IPMI KVM: http://i55.tinypic.com/synpzk.png I think that your best bet is to debug what's going on in zfs_fuid_table_lo= ad function. Add a few printfs in it to see what values get passed to avl_add= calls. It would make sense to print f_idx and f_ksid->kd_name fields for each domn= ode. It looks like a duplicate entry is getting inserted into one of the trees. -- Andriy Gapon Med V=E4nliga H=E4lsningar ---------------------------------------------------------------------------= ---- Karli Sj=F6berg Swedish University of Agricultural Sciences Box 7079 (Visiting Address Kron=E5sv=E4gen 8) S-750 07 Uppsala, Sweden Phone: +46-(0)18-67 15 66 karli.sjoberg@slu.se From owner-freebsd-fs@FreeBSD.ORG Thu Dec 15 12:47:50 2011 Return-Path: Delivered-To: fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 82674106566B; Thu, 15 Dec 2011 12:47:50 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 63F728FC14; Thu, 15 Dec 2011 12:47:49 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id OAA27461; Thu, 15 Dec 2011 14:47:45 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1RbAiv-0002Hc-Au; Thu, 15 Dec 2011 14:47:45 +0200 Message-ID: <4EE9EC6F.2080808@FreeBSD.org> Date: Thu, 15 Dec 2011 14:47:43 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111206 Thunderbird/8.0 MIME-Version: 1.0 To: =?windows-1252?Q?Karli_Sj=F6berg?= , Pawel Jakub Dawidek , =?windows-1252?Q?Martin_Matu=9Aka?= References: <82B38DBF-DD3A-46CD-93F6-02CDB6506E05@slu.se> <73B607D0-8C3E-4980-B901-B986E060D32E@slu.se> <4EE7754D.3050605@FreeBSD.org> <558C926F-14FA-458D-BB8E-D20BA46BE6D2@slu.se> In-Reply-To: <558C926F-14FA-458D-BB8E-D20BA46BE6D2@slu.se> X-Enigmail-Version: undefined Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Cc: "fs@freebsd.org" Subject: Re: Consistant panics trying to access zfs filesystems replicated from Sun/Oracle appliance X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2011 12:47:50 -0000 on 15/12/2011 13:59 Karli Sjöberg said the following: > Hi all, > > with the help of Andriy Gapon, I managed to capture what happened: > > # cd /export/Portfolio/ci (TAB) > http://oi40.tinypic.com/b3lsog.jpg > > # cd /export/Portfolio/cifs_share > http://oi42.tinypic.com/6e40op.jpg > > # ls /export/Portfolio/cifs_share > http://oi42.tinypic.com/23rn60j.jpg > > > And this was Andriy´s response: > Hmm, so it adds the "FreeBSD" string twice. > I am not sure what that means, consider sharing this result with the public, > maybe someone will have a better idea. Ah, hah, no wonder there is a panic: static __inline ksiddomain_t * ksid_lookupdomain(const char *domain) { ksiddomain_t *kd; kd = kmem_alloc(sizeof(*kd), KM_SLEEP); strlcpy(kd->kd_name, "FreeBSD", sizeof(kd->kd_name)); return (kd); } So, no matter what input domain value is, the returned ksiddomain_t is going to have kd_name of "FreeBSD". Basically it means that if an on-disk fuid_nvlist has more than one entry then we always are going to hit this panic. Not good. -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Thu Dec 15 15:42:22 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 956991065670 for ; Thu, 15 Dec 2011 15:42:22 +0000 (UTC) (envelope-from danno@internet2.edu) Received: from int-proxy02.merit.edu (int-proxy02.merit.edu [207.75.116.231]) by mx1.freebsd.org (Postfix) with ESMTP id 554D48FC14 for ; Thu, 15 Dec 2011 15:42:21 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by int-proxy02.merit.edu (Postfix) with ESMTP id 66C5F12002C for ; Thu, 15 Dec 2011 10:42:21 -0500 (EST) X-Virus-Scanned: amavisd-new at int-proxy02.merit.edu Received: from int-proxy02.merit.edu ([127.0.0.1]) by localhost (int-proxy02.merit.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ybVBbZtjxice for ; Thu, 15 Dec 2011 10:42:20 -0500 (EST) Received: from shrubbery.internet2.edu (desk174.internet2.edu [207.75.165.174]) by int-proxy02.merit.edu (Postfix) with ESMTPSA id ADE98120022 for ; Thu, 15 Dec 2011 10:42:20 -0500 (EST) Message-ID: <4EEA155C.5050305@internet2.edu> Date: Thu, 15 Dec 2011 10:42:20 -0500 From: Dan Pritts User-Agent: Postbox 3.0.2 (Macintosh/20111203) MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <4EE118C7.8030803@internet2.edu> <4EE12632.4070309@internet2.edu> <4EE21936.6020502@egr.msu.edu> In-Reply-To: <4EE21936.6020502@egr.msu.edu> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: ZFS hangs with 8.2-release X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2011 15:42:22 -0000 Hi all, as a followup to my notes from last week. Short answer, I have followed most or all of the list's suggestions and I still get crashes when scrubbing. In fact, It is now reliably crashing after <10 minutes. Does anyone have any other suggestions? Are the ZFS devs here, and would crash dumps be useful? Below are my responses to specific things that folks suggested. > do a memory test my colleague reminded me that we have run a test in the last month or two, since we started troubleshooting this. 24 hours with memtest86+ with no errors reported. FWIW this system was stable running solaris for several years. > Recommendations to upgrade to 8.2-STABLE and then polite explanations > after i did it wrong We've upgraded to 8.2-STABLE and applied the 1-line patch suggested by Adam McDougall. > FreeBSD netflow3.internet2.edu 8.2-STABLE FreeBSD 8.2-STABLE #1: Mon > Dec 12 15:45:06 UTC 2011 > root@netflow3.internet2.edu:/usr/obj/usr/src/sys/GENERIC amd64 And many recommendations from Adam McDougall that resulted in the following /boot/loader.conf. I also tried removing all of the zfs and vm lines, same problems. I think that something in here is causing the lockups - with the empty loader.conf it reboots instead of locking. > verbose_loading="YES" > rootdev="disk16s1a" > > #I have 16G of Ram > > vfs.zfs.prefetch_disable=1 > vfs.zfs.txg.timeout="5" > vfs.zfs.arc_min="512M" > vfs.zfs.arc_max="4G" > vm.kmem_size="32G" Specifics from Adam: >> >> - In my experience running with prefetch disabled is a significant >> impact to speed, once you are comfortable with doing some performance >> testing I would evaluate that and decide for yourself about "some >> discussion suggests that the prefetch sucks" Just to confirm, is there any STABILITY reason not to disable prefetch? The notes I saw suggested that it hurt stability. >> - Be wary of using dedupe in v28, it seems to have a huge performance >> drag when working with files that were written while dedupe was >> enabled; I won't comment more on that except to suggest not adding >> that variable to your issue Good to know. Not appropriate for our data set anyway. >> - These comments mostly relate to speed, but I had to give the ARC >> enough room to work without deadlocking the system so they may help >> you there. "enough to work" meaning along the lines of 2-4G as suggested above? thanks! danno -- Dan Pritts, Sr. Systems Engineer Internet2 office: +1-734-352-4953 | mobile: +1-734-834-7224 From owner-freebsd-fs@FreeBSD.ORG Thu Dec 15 16:12:09 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 357E31065672 for ; Thu, 15 Dec 2011 16:12:09 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta05.emeryville.ca.mail.comcast.net (qmta05.emeryville.ca.mail.comcast.net [76.96.30.48]) by mx1.freebsd.org (Postfix) with ESMTP id 196518FC0A for ; Thu, 15 Dec 2011 16:12:08 +0000 (UTC) Received: from omta16.emeryville.ca.mail.comcast.net ([76.96.30.72]) by qmta05.emeryville.ca.mail.comcast.net with comcast id 9RzM1i0021ZMdJ4A5UC2CL; Thu, 15 Dec 2011 16:12:02 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta16.emeryville.ca.mail.comcast.net with comcast id 9TzF1i00D1t3BNj8cTzFrh; Thu, 15 Dec 2011 15:59:15 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 853BD102C1A; Thu, 15 Dec 2011 08:12:07 -0800 (PST) Date: Thu, 15 Dec 2011 08:12:07 -0800 From: Jeremy Chadwick To: Dan Pritts Message-ID: <20111215161207.GA26990@icarus.home.lan> References: <4EE118C7.8030803@internet2.edu> <4EE12632.4070309@internet2.edu> <4EE21936.6020502@egr.msu.edu> <4EEA155C.5050305@internet2.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4EEA155C.5050305@internet2.edu> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs@freebsd.org Subject: Re: ZFS hangs with 8.2-release X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2011 16:12:09 -0000 On Thu, Dec 15, 2011 at 10:42:20AM -0500, Dan Pritts wrote: > Hi all, as a followup to my notes from last week. > > Short answer, I have followed most or all of the list's suggestions > and I still get crashes when scrubbing. In fact, It is now reliably > crashing after <10 minutes. > > Does anyone have any other suggestions? Are the ZFS devs here, and > would crash dumps be useful? > > > Below are my responses to specific things that folks suggested. > > > >do a memory test > my colleague reminded me that we have run a test in the last month > or two, since we started troubleshooting this. 24 hours with > memtest86+ with no errors reported. FWIW this system was stable > running solaris for several years. > >Recommendations to upgrade to 8.2-STABLE and then polite > >explanations after i did it wrong > We've upgraded to 8.2-STABLE and applied the 1-line patch suggested > by Adam McDougall. > > >FreeBSD netflow3.internet2.edu 8.2-STABLE FreeBSD 8.2-STABLE #1: > >Mon Dec 12 15:45:06 UTC 2011 > >root@netflow3.internet2.edu:/usr/obj/usr/src/sys/GENERIC amd64 > > And many recommendations from Adam McDougall that resulted in the > following /boot/loader.conf. I also tried removing all of the zfs > and vm lines, same problems. > > I think that something in here is causing the lockups - with the > empty loader.conf it reboots instead of locking. > >verbose_loading="YES" > >rootdev="disk16s1a" > > > >#I have 16G of Ram > > > >vfs.zfs.prefetch_disable=1 > >vfs.zfs.txg.timeout="5" > >vfs.zfs.arc_min="512M" > >vfs.zfs.arc_max="4G" > >vm.kmem_size="32G" These settings are incorrect by my standards. You're running 8.2-RELEASE though I would strongly recommend you go with 8.2-STABLE and stay with that instead. Regardless of which you run, these are the settings you should be using in /boot/loader.conf: vfs.zfs.arc_max="4G" You could increase this value to 8G if you wanted, or maybe even 12G (and possibly larger, but I would not recommend above 14G). There is "an art" to tuning this variable, as memory fragmentation and other things I'd rather not get into can cause the ARC size to exceed that variable sometimes (this is even further addressed in 8.2-STABLE -- consider it another reason to run that instead of -RELEASE). So, you need to "give it some headroom". The extra "art" involved is that you want to make sure you don't give too much memory to the ARC; e.g. if you have a big/fat mysqld running on that system, you should probably diminish the ARC size so that you have a "good balance" between what MySQL can/will use (based on mysql tunings and some other loader.conf tunings) and what's available for ARC/kernel. Start small (4GB with a 16GB RAM system is fine), see how things are, then increase it. With a 16GB system I would go with 4GB -> 8GB -> 10GB, with about a week in between each test. DO NOT pick a value like 16GB or 15GB; again, it's better to be safe than sorry, else you'll experience a kernel panic. :-) Further comments: 1. vfs.zfs.txg.timeout defaults to 5 now. 2. There is no point in messing with vfs.zfs.arc_min. It seems to be calculated on its own, and reliably so. 3. vm.kmem_size should not be adjusted/touched at all. Messing about with this *could* cause a reboot or possibly stability problems (but the latter would show up differently, not just a reboot). Next let's talk about vfs.zfs.prefetch_disable="1". For a very long time (years?), I strongly advocated this setting (which disables prefetching). The performance on my home storage system, as well as our production systems in our co-lo, suffered when prefetching was enabled; I/O throughput was generally blah. You can find old posts from me on the mailing list, and many posts from me on the web talking about this and advocating its setting. However, we have removed it entirely and leave prefetching enabled. We haven't noticed any particular massive performance loss as such, so it's very likely something was changed/improved in this regard. Maybe ZFSv28 is what did it, I really don't know (meaning I am not sure which commit may have addressed it). Prefetching being enabled or disabled has absolutely no bearing on stability, other than your drives may get taxed a tiny bit more (more data read into the ARC in advance). However, if you have the time (after you get this lock-up problem solved), you can play with the setting if you want. Find what works best for you with your workload. Be aware you should change the setting and then let it sit for about a week or so if possible, to get a full feel for the difference. Next let's talk about dedupe as well as compression. I recommend not enabling either one of these unless you absolutely want them/need them **and** are willing to suffer from sporadic "stalls" on the system during ZFS I/O. The stalling is double-worse if you enable both. "Stalls" means while ZFS is writing (dedupe only) or writing/reading (compression), things like typing via SSH, or on the console, or doing ANYTHING on the system just "stops" and catches up when ZFS does its stuff. This is a known problem and has to do with lack of "prioritisation queue" code for dedupe/compression on ZFS for FreeBSD. Solaris does not have this problem (it was solved by implementing said prio queue thing). I can refer you to the exact post from Bob Friesenhahn on this subject if you wish to read it. There is no ETA on getting this fixed in FreeBSD (meaning I have seen no one discuss fixing it or anything of that sort). Both of these features will tax your system CPU and memory more than if you didn't use them. If you do wish to use compression, I recommend using the lzjb algorithm instead of the default (gzip) as it diminishes the stalling by quite a bit -- but it's still easily noticeable. Finally, let's talk about your system problem: Can you take ZFS out of the picture on this system? If so, please do. That would be a great way to start. But I will give you my opinion: I strongly doubt ZFS is responsible for your problem. ZFS is probably "tickling" another problem. I'm inclined to believe your problem is hardware-related, or (and this continues to get my vote because of the continual non-stop problems I keep reading with these damn controllers) firmware or driver related pertaining to your mpt(4) cards. I will recall what Dan said initially -- you have to read very close to understand the implications. Quote: > internal LSI mpt-driver hardware raid for boot. > 3x LSI parallel-scsi cards for primary storage. 48 SATA disks > attached. Using Infortrend RAIDs as JBODs. So you effectively have 4 LSI cards in this system. Would you like me to spend a few hours digging through mailing lists and PRs listing off all the problems people continually report with either mpt(4), mps(4) or mfi(4) on FreeBSD, ESPECIALLY when ZFS is in use? Heck, there were even commits done not too long ago to one of the drivers "to help relieve problems when heavy I/O happens under ZFS". Then there's the whole debacle with card firmware versions (and you've FOUR cards! :-) ). Some people report problems with some firmware versions, while others work great. Then there's the whole provided-by-FreeBSD vs. provided-by-LSI driver ordeal. I don't even want to get into this nonsense -- seriously, it's all on the mailing lists, and it keeps coming up. It would take me, as I said, hours to put it all together and give you *LOTS* of references. Finally, there is ALWAYS the possibility of bad hardware. I don't mean RAM -- I'm talking weird motherboard problems that are exacerbated when doing lots of PCIe I/O, or drawing too much power -- neither of these would be stress-tested by memtest86, obviously. The number of possibilities is practically endless, I'm sorry to say. Hardware troubleshooting 101 says replace things piece-by-piece until you figure it out. :-( Otherwise, I'd consider just running OpenIndiana on this system, assuming their LSI card driver support is good. Finally: http://people.internet2.edu/~danno/zfs/ returns HTTP 403 Forbidden, so I have no idea what your photos/screen shots contained, if anything. :-( -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Thu Dec 15 20:24:10 2011 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 07773106566B; Thu, 15 Dec 2011 20:24:10 +0000 (UTC) (envelope-from pfg@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id D2F928FC1A; Thu, 15 Dec 2011 20:24:09 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id pBFKO9Sq061013; Thu, 15 Dec 2011 20:24:09 GMT (envelope-from pfg@freefall.freebsd.org) Received: (from pfg@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id pBFKO9xa061009; Thu, 15 Dec 2011 20:24:09 GMT (envelope-from pfg) Date: Thu, 15 Dec 2011 20:24:09 GMT Message-Id: <201112152024.pBFKO9xa061009@freefall.freebsd.org> To: pfg@FreeBSD.org, freebsd-fs@FreeBSD.org, pfg@FreeBSD.org From: pfg@FreeBSD.org Cc: Subject: Re: kern/159233: [ext2fs] [patch] fs/ext2fs: finish reallocblk implementation X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2011 20:24:10 -0000 Synopsis: [ext2fs] [patch] fs/ext2fs: finish reallocblk implementation Responsible-Changed-From-To: freebsd-fs->pfg Responsible-Changed-By: pfg Responsible-Changed-When: Thu Dec 15 20:22:45 UTC 2011 Responsible-Changed-Why: Assign to myself. http://www.freebsd.org/cgi/query-pr.cgi?pr=159233 From owner-freebsd-fs@FreeBSD.ORG Thu Dec 15 20:53:39 2011 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 81967106566B; Thu, 15 Dec 2011 20:53:39 +0000 (UTC) (envelope-from pfg@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 590948FC13; Thu, 15 Dec 2011 20:53:39 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id pBFKrdbb089361; Thu, 15 Dec 2011 20:53:39 GMT (envelope-from pfg@freefall.freebsd.org) Received: (from pfg@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id pBFKrdpB089357; Thu, 15 Dec 2011 20:53:39 GMT (envelope-from pfg) Date: Thu, 15 Dec 2011 20:53:39 GMT Message-Id: <201112152053.pBFKrdpB089357@freefall.freebsd.org> To: pfg@FreeBSD.org, freebsd-fs@FreeBSD.org, pfg@FreeBSD.org From: pfg@FreeBSD.org Cc: Subject: Re: kern/162564: [ext2fs][patch] fs/ext2fs: Add include guard X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2011 20:53:39 -0000 Synopsis: [ext2fs][patch] fs/ext2fs: Add include guard Responsible-Changed-From-To: freebsd-fs->pfg Responsible-Changed-By: pfg Responsible-Changed-When: Thu Dec 15 20:52:40 UTC 2011 Responsible-Changed-Why: I will take this. http://www.freebsd.org/cgi/query-pr.cgi?pr=162564 From owner-freebsd-fs@FreeBSD.ORG Fri Dec 16 01:00:30 2011 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4ACFD106564A for ; Fri, 16 Dec 2011 01:00:30 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 3982D8FC1D for ; Fri, 16 Dec 2011 01:00:30 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id pBG10Uug012290 for ; Fri, 16 Dec 2011 01:00:30 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id pBG10UlK012289; Fri, 16 Dec 2011 01:00:30 GMT (envelope-from gnats) Date: Fri, 16 Dec 2011 01:00:30 GMT Message-Id: <201112160100.pBG10UlK012289@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: dfilter@FreeBSD.ORG (dfilter service) Cc: Subject: Re: kern/118126: commit references a PR X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: dfilter service List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2011 01:00:30 -0000 The following reply was made to PR kern/118126; it has been noted by GNATS. From: dfilter@FreeBSD.ORG (dfilter service) To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/118126: commit references a PR Date: Fri, 16 Dec 2011 00:58:55 +0000 (UTC) Author: rmacklem Date: Fri Dec 16 00:58:41 2011 New Revision: 228560 URL: http://svn.freebsd.org/changeset/base/228560 Log: Patch the new NFS server in a manner analagous to r228520 for the old NFS server, so that it correctly handles a count == 0 argument for Commit. PR: kern/118126 MFC after: 2 weeks Modified: head/sys/fs/nfsserver/nfs_nfsdport.c Modified: head/sys/fs/nfsserver/nfs_nfsdport.c ============================================================================== --- head/sys/fs/nfsserver/nfs_nfsdport.c Fri Dec 16 00:48:53 2011 (r228559) +++ head/sys/fs/nfsserver/nfs_nfsdport.c Fri Dec 16 00:58:41 2011 (r228560) @@ -1254,7 +1254,13 @@ nfsvno_fsync(struct vnode *vp, u_int64_t { int error = 0; - if (cnt > MAX_COMMIT_COUNT) { + /* + * RFC 1813 3.3.21: if count is 0, a flush from offset to the end of + * file is done. At this time VOP_FSYNC does not accept offset and + * byte count parameters so call VOP_FSYNC the whole file for now. + * The same is true for NFSv4: RFC 3530 Sec. 14.2.3. + */ + if (cnt == 0 || cnt > MAX_COMMIT_COUNT) { /* * Give up and do the whole thing */ _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Fri Dec 16 19:49:31 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 92F10106564A for ; Fri, 16 Dec 2011 19:49:31 +0000 (UTC) (envelope-from wmn@siberianet.ru) Received: from mail.siberianet.ru (mail.siberianet.ru [89.105.136.7]) by mx1.freebsd.org (Postfix) with ESMTP id 02F658FC0C for ; Fri, 16 Dec 2011 19:49:30 +0000 (UTC) Received: from wmn.localnet (wmn.siberianet.ru [89.105.137.12]) by mail.siberianet.ru (Postfix) with ESMTPA id 3A770A1D89 for ; Sat, 17 Dec 2011 03:49:29 +0800 (KRAT) From: Sergey Lobanov To: freebsd-fs@freebsd.org Date: Sat, 17 Dec 2011 03:49:26 +0800 User-Agent: KMail/1.13.7 (Linux/2.6.38-ARCH; KDE/4.6.2; i686; ; ) References: <201112062004.03759.wmn@siberianet.ru> In-Reply-To: <201112062004.03759.wmn@siberianet.ru> Organization: ISP "SiberiaNet" MIME-Version: 1.0 X-Length: 8156 X-UID: 28 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201112170349.26859.wmn@siberianet.ru> Subject: Re: ZFS v28: mount dataset hangs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2011 19:49:31 -0000 On Tuesday 06 December 2011 at 20:04:03 Sergey Lobanov wrote: > On Thursday 06 October 2011 at 16:04:30 Sergey Gavrilov wrote: > > Hello, all. > > > > After "chown -R" hanging and reboot I cannot mount one of the > > datasets, while other pools work well. > > CTRL-T shows state [tx->tx_sync_done_cv)] > > > > Hardware: 3ware 9690SA-8I, 8 SEAGATE ST32000445SS in raidz2, 48 Gb > > RAM, 2 E5520 CPU > > Software: FreeBSD 8.2-STABLE #3: Wed Oct 5 18:04:50 MSD 2011 > > /usr/obj/usr/src/sys/GENERIC amd64, ZFS V28 compression=on > > > > I can clone and mount any snapshot of the dataset, but not itself. > > No errors in zpool status or system messages. Scrub has finished clear. > > > > Let me know if I could provide any additional information to solve the > > issue. > > Hello. > > Same problem here. > > raidz2 using 4 disks (made with fake 4k sector size using gnop for possible > future replacement by native 4k disks) with dedup, lzjb and NTFSv4 ACL > which is shared over samba3.6 for windows clients. System freezed (even > local log-in via ipmi didn't work) during flush of recycle (trash) over > smb from windows station on one of datasets (a lot of files were being > removed). After hard reset via ipmi, zfs hangs on mount of that dataset > under 8.2 and 9.0-RC2 (but "zfs set mountpoint" works well). Scrubs under > 8.2 and 9.0-RC2 did not help and found nothing, other datasets can be > mounted without problems. > > Storage consists of 2 seagate disks (ST3160318AS CC38) with ufs for system, > 4 WD (WD10EALX-009BA0 15.01H15) for zfs share and Qlogic 2432 FC adapter > which is connected to HP tape library via FC-switch for bacula backups. > > All WD disks were checked with smart long self-test, went without errors, > also no reallocated sectors (and events) in smart output. here is example > for one of the disks: > ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED > WHEN_FAILED RAW_VALUE > 1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always > - 0 > 3 Spin_Up_Time 0x0027 177 175 021 Pre-fail Always > - 4108 > 4 Start_Stop_Count 0x0032 100 100 000 Old_age Always > - 60 > 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always > - 0 > 7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always > - 0 > 9 Power_On_Hours 0x0032 092 092 000 Old_age Always > - 6183 > 10 Spin_Retry_Count 0x0032 100 253 000 Old_age Always > - 0 > 11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always > - 0 > 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always > - 58 > 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always > - 47 > 193 Load_Cycle_Count 0x0032 200 200 000 Old_age Always > - 12 > 194 Temperature_Celsius 0x0022 122 110 000 Old_age Always > - 25 > 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always > - 0 > 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always > - 0 > 198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline > - 0 > 199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always > - 0 > 200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline > - 85 > > Here is some other output: > --- > # zfs mount data/storage/public > load: 0.15 cmd: zfs 1352 [tx->tx_sync_done_cv)] 348.58r 0.00u 0.92s 0% > 2296k > > # procstat -kk 1352 > PID TID COMM TDNAME KSTACK > 1352 100081 zfs initial thread mi_switch+0x176 > sleepq_wait+0x42 _cv_wait+0x129 txg_wait_synced+0x85 zil_replay+0x10a > zfsvfs_setup+0x117 zfs_mount+0x52f vfs_donmount+0xdc5 nmount+0x63 > amd64_syscall+0x1f4 Xfast_syscall+0xfc > --- > # zfs snapshot data/storage/public@test > load: 0.02 cmd: zfs 1382 [tx->tx_sync_done_cv)] 4.34r 0.00u 0.00s 0% 2280k > load: 0.09 cmd: zfs 1382 [tx->tx_sync_done_cv)] 19.13r 0.00u 0.00s 0% > 2280k load: 0.01 cmd: zfs 1352 [tx->tx_sync_done_cv)] 611.74r 0.00u 0.92s > 0% 2296k > > # procstat -kk 1382 > PID TID COMM TDNAME KSTACK > 1382 100295 zfs initial thread mi_switch+0x176 > sleepq_wait+0x42 _cv_wait+0x129 txg_wait_synced+0x85 > dsl_sync_task_group_wait+0x128 dmu_objset_snapshot+0x302 > zfs_ioc_snapshot+0x1a8 zfsdev_ioctl+0xe6 devfs_ioctl_f+0x7b > kern_ioctl+0x102 ioctl+0xfd amd64_syscall+0x1f4 Xfast_syscall+0xfc > --- > # procstat -kk 6 > PID TID COMM TDNAME KSTACK > 6 100058 zfskern arc_reclaim_thre mi_switch+0x176 > sleepq_timedwait+0x42 _cv_timedwait+0x134 arc_reclaim_thread+0x29d > fork_exit+0x11f fork_trampoline+0xe > 6 100059 zfskern l2arc_feed_threa mi_switch+0x176 > sleepq_timedwait+0x42 _cv_timedwait+0x134 l2arc_feed_thread+0x1a8 > fork_exit+0x11f fork_trampoline+0xe > 6 100279 zfskern txg_thread_enter mi_switch+0x176 > sleepq_wait+0x42 _cv_wait+0x129 txg_thread_wait+0x79 > txg_quiesce_thread+0xb5 fork_exit+0x11f fork_trampoline+0xe > 6 100280 zfskern txg_thread_enter mi_switch+0x176 > sleepq_wait+0x42 _cv_wait+0x129 zio_wait+0x61 dbuf_read+0x5e5 > dmu_buf_hold+0xe0 zap_idx_to_blk+0xa3 zap_deref_leaf+0x4f fzap_remove+0x33 > zap_remove_uint64+0x85 ddt_sync+0x267 spa_sync+0x383 txg_sync_thread+0x139 > fork_exit+0x11f fork_trampoline+0xe > --- > > And here is some interesting screenshot of output from 9.0-RC2 livecd > shell, which leads to beleive that there was swap exhaustion while zfs > mount were freezed: > http://imageshack.us/photo/my-images/64/90rc2zfsmounthangswap.gif/ (didn't > get this on 8.2 and 9.0-RC1 though didn't specially try to let system be > in freeze for a long time). > > Hardware is supermicro X8SIA-F, core i3, 4Gb of memory. > > releng8 r226341 amd64 (with e1000 from 8.2-release for another reason), > custom kernel with config: > include GENERIC > ident IPFW_FWD > options IPFIREWALL > options IPFIREWALL_DEFAULT_TO_ACCEPT > options IPFIREWALL_FORWARD > > ZFS-related tuning in loader.conf is only following line: > vfs.zfs.arc_max="1073741824" > > I have couple of days for debugging (i plan to try releng9 today or > tomorrow because i can see some zfs commits there after RC2), releng9 r228308 successfully mounted filesystem after some backstage work. I've typed "zfs mount", then was waiting for few minutes during which there was some work being done (top was showing zfs threads eating little amount of cpu time and gstat was showing some reads and writes for all of zfs disks), then didn't watch for some time and after 30 minutes or so, job was already done. > after which > i'll have to move this machine to production with zfs or without it. The system is in production on releng9 already a week without any issues. But i had to give up on dedup and recreate the pool from scratch, because system locked up on simultaneous zfs destroy of dedup filesystem and zfs destroy of another non-dedup filesystem snapshot, and after hard reset did not come up on any of releng9, mfsbsd8.2+zfsv28 or oi151a, eating all memory even on zpool status or zfs list. > > If you need something else, let me know, i've subscribed to the list, so > there is no need to CC me. -- ISP SiberiaNet System and Network Administrator