From owner-freebsd-stable@FreeBSD.ORG Sun May 17 04:40:52 2009 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CC92E1065673 for ; Sun, 17 May 2009 04:40:52 +0000 (UTC) (envelope-from alessandro.dellavedova@ifom-ieo-campus.it) Received: from lupin.ifom-ieo-campus.it (lupin.ifom-ieo-campus.it [85.239.175.139]) by mx1.freebsd.org (Postfix) with ESMTP id 218E88FC13 for ; Sun, 17 May 2009 04:40:51 +0000 (UTC) (envelope-from alessandro.dellavedova@ifom-ieo-campus.it) Received: (qmail 76603 invoked by uid 106); 17 May 2009 04:14:10 -0000 Received: from 93-35-112-55.ip54.fastwebnet.it by lupin.ifom-ieo-campus.it (envelope-from , uid 803) with qmail-scanner-2.06 (clamdscan: 0.95.1/9216. spamassassin: 3.2.5. Clear:RC:1(93.35.112.55):. Processed in 0.036852 secs); 17 May 2009 04:14:10 -0000 Received: from 93-35-112-55.ip54.fastwebnet.it (HELO [1.64.130.49]) (adellave@[93.35.112.55]) (envelope-sender ) by lupin.ifom-ieo-campus.it (qmail-ldap-1.03) with AES128-SHA encrypted SMTP for ; 17 May 2009 04:14:10 -0000 Message-Id: <28D35184-3EAF-43A9-89A7-C434FD23A967@ifom-ieo-campus.it> From: Alessandro Dellavedova To: Adam McDougall In-Reply-To: <20090516065019.GM82547@egr.msu.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Sun, 17 May 2009 06:14:09 +0200 References: <3c1674c90905151702l81c2b88off1d2b2ffed39ca2@mail.gmail.com> <20090516065019.GM82547@egr.msu.edu> X-Mailer: Apple Mail (2.930.3) Cc: freebsd-stable@freebsd.org, Kip Macy Subject: Re: RFT: ZFS MFC X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 May 2009 04:40:53 -0000 On May 16, 2009, at 8:50 AM, Adam McDougall wrote: > On Fri, May 15, 2009 at 05:02:22PM -0700, Kip Macy wrote: > > I've MFC'd ZFS v13 to RELENG_7 in a work branch. Please test if you > can. > > http://svn.freebsd.org/base/user/kmacy/ZFS_MFC/ > > The standard disclaimers apply. This has only been lightly tested > in a > VM. Please do not use it with data you care about at this time. > > > Thanks, > Kip > > > Seems to work for me so far. I had a zfs send hang part way through > and > with a notable speed difference depending on the direction but this is > literally the first time I've tried zfs send/recv and the systems are > setup different so I have no idea if it would have happened anyway. > Eventually I could probably make these test systems more similar to > give a > fair test, but wanted to mention it so others could check. > > Thanks for working on the MFC, I'm excited to see progress there! > It will play a factor in some upcoming server plans even if the MFC > doesn't happen for months. > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org > " We're now testing ZFS under FreeBSD (7.2 and CURRENT) quite extensively, because our primary goal is to setup a fileserver that can scale up to 80TB using high-density, low-cost storage (2TB SATA II disks in a 42 bay, 4 unit storage). The fact that ZFS v13 has been MFC'ed sounds good to me. I'd like to recommend to test the most basic physical stress conditions, eg: - Pulling out one healthy disk in a raidz2 pool; - Pulling it back and see if resilvering starts in a reasonable time (we experienced consolle locks, access via SSH to the box was still possible and fully responsive); - Pulling out TWO healthy disks in a raidz2 pool; - Pulling them back one at a time (and two at the same time) and see if the pool is resilvered in a reasonable time; - Too many combinations to be listed here. I'll be more than happy if the FreeBSD community will came out with a sort of ZFS standard stress test suite (both physical and logical), in order to be able to compare the ZFS reliability when used in different scenarios (eg: our scenario is made of a v20Z server with 6Gb RAM directly attached via a 2Gb Qlogic Fibre Channel adapter, the JBOD is an Apple Xraid fully loaded with 450Gb PATA disks, all sysctl parameters set accordingly to the ZFS tuning guide). I'm strongly conviced that, for a number of reasons, FreeBSD + ZFS can be the silver bullet in the enterprise storage just because of the following: PROS: - Solaris/OpenSolaris are limited to an NGROUPS_MAX value of 16 (http://bugs.opensolaris.org/view_bug.do?bug_id=4088757 , http://www.j3e.de/ngroups.html) while FreeBSD isn't (we are using an NGROUPS_MAX value of 64 right now). This is a good point if you are serving hundreds of clients, authenticated via LDAP (Posix accounts RFC 2307) and you are living in a mixed shop (MACs and PCs.. like we do); - Apple is introducing ZFS support on non-bootable storage under OS X 10.6 but nobody really knows what ZFS version will be made available; - License constraints on Linux. ZFS can't be used in kernel mode, only using FUSE at the price of degraded performance (AFAIK); - Maybe I'm missing something but it's ok since I'm slightly drunk as of now. Please help (by suggesting missing points, not by suggesting me to refrain from drinking on Saturday's night ;-). CONS: - FreeBSD is currently lacking an enterprise-level support for modern 4Gb Fibre Channel HBAs, as you can read here: http://lists.freebsd.org/pipermail/freebsd-scsi/2009-April/date.html#3876 <-- No reply, AFAIK; http://lists.freebsd.org/pipermail/freebsd-scsi/2008-October/003686.html http://lists.freebsd.org/pipermail/freebsd-scsi/2008-November/003705.html http://lists.freebsd.org/pipermail/freebsd-stable/2008-November/046556.html http://lists.freebsd.org/pipermail/freebsd-stable/2008-November/046602.html Before throwing rotting tomatoes straight to me please consider the following facts: - Our infrastructure is 90% based on FreeBSD. It has proven to be rock solid over a 7 year timespan. Are we happy ? Definitely yes.; - We are grateful to the FreeBSD community, this mail should be intended as a constructive criticism in order to exploit a "sweet spot" in storage enterprise. This can dramatically leverage the adoption of FreeBSD in the enterprise, if we consider the fact that, due to the global crisis, most enterprises will start to look at "low- cost, highly reliable, higly scalable storage systems"; - We are also grateful to the Pawel Jakub Dawidek and all the other FreeBSD ZFS contributors, you are doing a great job guys, thanks. What can I do to foster this trend, and being able to further enhance the ZFS support under FreeBSD, provided that we cannot contribute coding skills ? Unfortunately not a lot but we can do the following: - Donate some hardware (Fibre Channel HBAs) to the FreeBSD project (paid from my pocket, not my employer's one); - Donate some money (paid from my employer's pocket, if I can demonstrate that this can help us to save big bucks on high-end storage systems); - Detail in a very precise way all the tests that we are doing on ZFS, using the following storage: Apple Xraid, Nexsan Sas/SATAbeast, EMC (waiting for the final configuration) as a contribution to the FreeBSD community. In a few words, please let the community know what we can do in order to make this dream come true. FreeBSD: The power to serve, on steroids. Alessandro Dellavedova European Institute of Oncology Department of Experimental Oncology Via Adamello, 16 - 20139 Milan, Italy