From owner-freebsd-stable@FreeBSD.ORG Sun Apr 20 02:29:20 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7E7C7106564A for ; Sun, 20 Apr 2008 02:29:20 +0000 (UTC) (envelope-from bri@brianwhalen.net) Received: from entwistle.sonicboom.org (entwistle.sonicboom.org [66.93.34.170]) by mx1.freebsd.org (Postfix) with ESMTP id 422E38FC13 for ; Sun, 20 Apr 2008 02:29:20 +0000 (UTC) (envelope-from bri@brianwhalen.net) Received: from [127.0.0.1] (localhost.sonicboom.org [127.0.0.1]) by entwistle.sonicboom.org (8.14.2/8.14.2) with ESMTP id m3K2TJv4013479; Sat, 19 Apr 2008 19:29:19 -0700 (PDT) (envelope-from bri@brianwhalen.net) Message-ID: <480AAA7F.3010707@brianwhalen.net> Date: Sat, 19 Apr 2008 19:29:19 -0700 From: Brian User-Agent: Thunderbird 2.0.0.12 (Windows/20080213) MIME-Version: 1.0 To: Nenhum_de_Nos References: <396418019.20080409104542@serebryakov.spb.ru> <47FCBAFB.9060508@tzim.net> <4956a5e50804191912g52833c35q868827dc2b54e5ae@mail.gmail.com> In-Reply-To: <4956a5e50804191912g52833c35q868827dc2b54e5ae@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: lev@freebsd.org, freebsd-stable@freebsd.org, Arnaud Houdelette Subject: Re: [SO]HO Software RAID5 server: which implementation should I choice? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Apr 2008 02:29:20 -0000 Nenhum_de_Nos wrote: > On Wed, Apr 9, 2008 at 9:47 AM, Arnaud Houdelette > wrote: > >> Lev Serebryakov a écrit : >> >> >> >> >>> Hello, freebsd-stable. >>> >>> Does somebody use some software RAID5 on FreeBSD in real production >>> system? >>> >>> I want to build storage server for my home: RAW photos, multi-layer >>> PhotoShop files and FLAC-encoded music consume a lot of space, and >>> they should be availible both from desktop & notebook. >>> >>> Also, all photo-content is unique, so I need some insuranse from >>> single HDD crash. I understand, that I will not safe from fire, PSU >>> failure and thing slike this. >>> >>> I selected hardware platform: Intel Q35-based MoBo with 6xSATA-II ports >>> (all of them is chipset-based, so no SiliconImage/JMicron/Whatever >>> crappy controllers), some low-end Core2Duo, 2Gb of memory. >>> Storage will be 5x500Gb WD HDDs for RAID + one small HDD for boot, >>> >> system, >> >>> swap, etc. I want to have 2Tb (ok, not real Tb, I know) of "protected" >>> storage. >>> I want to have maximum speed via 1Gb network, because graphic files >>> are big and should open fast. Not as fast as local ones, I understand >>> that, but speeds about 12-15Mb/s is not enough for sure :) >>> >>> Only problem I see: which software RAID5 solution should I prefer? >>> FreeBSD-based, of course! >>> >>> I see these variants: >>> >>> (1) FreeBSD 6(7?) + graid3. Slow, one disk for checksums is bottleneck, >>> as far as I understand. >>> >>> (2) FreeBSD 6(7?) + gvinum/radi5. Is it stable enough?! Is it complete? >>> when I try it about 6 months ago in VMWare installation with 5 >>> virtual disks, I got panics and strange behaviour after "crashing" >>> one of virtual disks. >>> >>> (3) FreeBSD 6(7?) + graid5. Again, is it stable enough? There are >>> THREE versions of it. Which one should I prefer? There was long >>> thread about it some times ago without any clear conclusion. Does >>> something changed? >>> >>> (4) FreeBSD 7 + ZFS "zraid". And again: stability. Too many messages >>> about locks, crashes, etc. Code is experemental. Is it only for >>> 32 bit systems? >>> >>> (5) Do I miss something? >>> >>> (6) Solaris + ZFS? I don't want it, I know a little about Solaris >>> administaration, and I already have FreeBSD servers and routers. >>> >>> I know, that 3ware or Areca controllers are very good. I know, that >>> "gmirror" is very stable. But these variants are too expensive for >>> home server :( >>> >>> Does somebody use some software RAID5 on FreeBSD in real production? >>> Any advices? >>> >>> >>> >> Hi ! >> >> I personally use the 3 option for my personal Home File Server. I got >> approximatly the same usage for the file server (mostly video, music, >> photo). I built my own about 12 month ago. >> I reviewed the about the same variants as the one you propose : >> (1) Discarded for performance issues. Raid3 is slow. Really. >> (2) raid5/vinum is also slow. And as I understood at that time, recovery >> from lost hard drive wasn't easy enough for the freebsd niubee I was then. >> (4) ZFS wasn't there yet. But I did test it on a test VMWare, and wasn't >> convinced (mostly stability and memory issues). >> >> So I use geom_raid5. I sticked to the main distributions, which seemed more >> stable at the moment. The kernel module is fairly simple to build/install. >> Performance is (very) good for a software raid. >> I successfully switched the raid array from an i386 6.2-RELEASE to an an >> amd64 7.0-RELEASE (with motherboard and CPU change) without any assle. >> >> For the moment, I use one big UFS+SU (and snapshots) on the whole array. I >> successfuly tried unpplugging then replugging on of the drives, suddent >> power loss, using the array with a missing disk (degraded mode). All did >> work fine. (still, I use an UPS on the file server). >> >> The sole issue I had is with ataidle. I had to patch ata-disc.c to increase >> the IO timeout. Without, the raid5 module detected temporary disk loss and >> constantly launched rebuilds of the array. >> >> With 7.0, I wondered if I should use gjournal, but I'm not sure if it's >> really the way to go on a file system dedicated to store many big files. So >> I stick to soft updates. >> >> Current configuration is : >> / on a 2GB usb key >> /tmp on memory >> ports and source trees (and some portsnap stuff) on a small disk >> 4x250 GB sata for the raid5 array. >> AMD A64 3200+ and 512 GB DDRII >> Realtek Gigabit nics. >> >> Copy from raid5 to /dev/null gives about 100MB/s >> Copy from /dev/random to raid5 about 40MB/s >> >> I use samba shares. I get about 40MB/s in both ways from another computer >> on the network (enabling jumbo-frames gives a big boost). >> >> Hope my own story can help you in any way. >> >> Regards, >> >> Arnaud Houdelette >> > > I know its been quite some time from the mail, but if you could say > where to find this module. eikipedia says its on freebsd 7 but there > is not this module for me (/boot/kernel/ there is no raid5 file). > everytime I search the internet I find old stuff about it. > > If you could point me the site/article/anything :) > > thanks, > > matheus > > > Do you know about freenas at http://www.freenas.org?