From owner-freebsd-fs@FreeBSD.ORG Thu Aug 2 14:40:36 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4F2F716A418 for ; Thu, 2 Aug 2007 14:40:36 +0000 (UTC) (envelope-from eksffa@freebsdbrasil.com.br) Received: from capeta.freebsdbrasil.com.br (vrrp.freebsdbrasil.com.br [200.210.70.30]) by mx1.freebsd.org (Postfix) with SMTP id 9A05813C457 for ; Thu, 2 Aug 2007 14:40:34 +0000 (UTC) (envelope-from eksffa@freebsdbrasil.com.br) Received: (qmail 60431 invoked from network); 2 Aug 2007 11:44:05 -0300 Received: by simscan 1.1.0 ppid: 60419, pid: 60420, t: 1.2353s scanners: clamav: 0.91.1/m: spam: 3.1.1 X-Spam-Checker-Version: SpamAssassin: -last, FreeBSD Brasil LTDA rulesets: Yes X-Spam-Status: No, hits=-2.2 required=3.7 Received: from unknown (HELO claire.bh.freebsdbrasil.com.br) (201.78.44.31) by capeta.freebsdbrasil.com.br with SMTP; 2 Aug 2007 11:44:04 -0300 Message-ID: <46B1ECDF.10407@freebsdbrasil.com.br> Date: Thu, 02 Aug 2007 11:40:31 -0300 From: Patrick Tracanelli Organization: FreeBSD Brasil LTDA User-Agent: Thunderbird 2.0.0.0 (X11/20070612) MIME-Version: 1.0 To: Eric Anderson References: <46B0F505.8090102@freebsdbrasil.com.br> <46B10798.5050504@freebsdbrasil.com.br> <200708011536.37926.matt@ixsystems.com> <46B12D0C.20808@freebsd.org> <46B1D167.4030206@freebsdbrasil.com.br> <46B1E234.7010005@freebsd.org> In-Reply-To: <46B1E234.7010005@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: Xsan (Apple) on FreeBSD X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2007 14:40:36 -0000 Eric Anderson wrote: > Patrick Tracanelli wrote: >> Eric Anderson wrote: >>> Matt Olander wrote: >>>> On Wednesday 01 August 2007 3:22 pm, Patrick Tracanelli wrote: >>>>> Hello Jeff, >>>>> >>>>> Jeff Mohler wrote: >>>>>> Im yet to hear of a large Xsan install that stayed Xsan once it grew. >>>>>> >>>>>> Most, if not all, have gone to netapp or umm..Isilon (spelling) >>>>>> that ive >>>>>> been close to. Latest large dump of Xsan that I know of was >>>>>> Current TV >>>>>> in San Francisco, for Isilon. >>>>> Hmm, good to know. I have tested XServe RAID only, which has shown >>>>> to be >>>>> a good solution as storage system for the usage profile I need, but >>>>> Xsan, never saw it working. Believed it to be the usual path to >>>>> follow, >>>>> but have hear of people running Stornext instead of xsan. >>>>> >>>>> So, now my question goes on a different path. Will Isilon OneFS run >>>>> FreeBSD? I head from people at Zoic Studios it is based on FreeBSD, >>>>> but >>>>> I am not sure how true this information is. Anyway, "based on" wont >>>>> always mean fully compatible. >>>>> >>>>> Are you aware of OneFS running on FreeBSD? >>>> >>>> It is indeed true. OneFS is FreeBSD. The Isilon product is really >>>> neat and you can buy it modularly, starting with just one unit. >>> >>> To be clear, OneFS isn't a solution to add to FreeBSD - it is an >>> all-in-one solution, that happens to be built with FreeBSD (or parts >>> of it anyway). It's like a NetApp. You don't run NetApp on FreeBSD, >>> you use whatever clients you want, and they connect *to* Isilon or >>> NetApp. As far as I understand, they are all just NFS/CIFS/iSCSI >>> servers/targets. There's really no solution for sharing a SAN block >>> device safely using FreeBSD (using the same blocks with the same fs). >>> That would require a clustered filesystem, and there is no such beast >>> for FreeBSD at this point. Simultaneous read-write activity from >>> different hosts to the same file system leads to Bad Stuff. >> >> Hello Eric, >> >> Thank you for clearing up some things. I believed OneFS to be an >> extension the the file system which would allow shared access. So, >> OneFS seem to be exclusively used by the appliance itself, as you >> mention. >> >> FreeBSD unfortunately don't have a shared file system (it would be a >> solution to this matter, combining a shared FS with ggate and gmirror, >> or using a central gvirstor/zfs storage solution, 100% FreeBSD-only). >> It really is a missing piece of feature which would boost >> usage/combination of many other ones. > > > Agreed - I've been beating that drum for some time. It's a *lot* of > work, and not enough developers/money to do it right now. I can imagine how much work would be needed for a whole new FS. Just curious, no chance for a geom module to do the trick? Or "tricky" would also be writing a geom to do this? =) > > >> I dont know about iSCSI support on FreeBSD. A quick research on the >> archieves seem to show that there is no iSCSI support at this time. So > > There is iSCSI support, and -CURRENT recently got an iSCSI initiator in > the base system. The iSCSI target is in the ports collection. This > doesn't fix the issue, as it's still a block device transport. Well, so, on 7.0 I can have a FreeBSD as target sharing some data and two 7.0 boxes as initiator acessing the same data? If so, that would be an approach to test. > > >> NFS/CIFS and something like that would the option? In this case, a >> FreeBSD solution seem a lot more flexible than a storage appliance. In >> fact I run NFS today, but performance is becoming a problem as the >> usage increases. I have never used CIFS on Unix-to-Unix enviroment, >> and I dont believe it to be better than NFS anyway. But maybe I should >> give it a try. Is there any other CIFS implementation other than >> Samba? Samba just happen to have so many features Ill never need in >> this enviroment. Is there any chance it will perform better than NFS? > > NFS will beat CIFS in performance almost always. NFS is a commonly used > protocol for shared file access, and should perform fairly well. If you > are hitting NFS performance issues, you might want to dig there first, > since there are things that you can do to improve your performance, > depending on your usage. It may in fact be that NFS itself is not the > bottleneck for you. Right. The situation I have is among many 5.5-STABLE systems. Sometimes, under high load, the client systems get high load averages, but userland apps are sort of sitting idle. I can see however, that "system" starts using ~ 80% of CPU (from top). Clients are quad-processed servers. On server, gstat shows me that the disk is on 100% usage, but not 100% throughput (since they are many, but small disk operations). nfsstat also shows we are on limit of what I could find to be the limit via dd parallely and massively writing and read tests. And thus, nfs server "stops responding" for a while, and later becomes responsive again. I didnt find out something I could tune up to make it better. Side question: is 5.5-STABLE to 6.2-STABLE NFS code too different? Would the behavior on 6.2 be different? -- Patrick Tracanelli FreeBSD Brasil LTDA. (31) 3281-9633 / 3281-3547 316601@sip.freebsdbrasil.com.br http://www.freebsdbrasil.com.br "Long live Hanin Elias, Kim Deal!"