From owner-freebsd-questions@freebsd.org Fri Jun 3 15:01:27 2016 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6A92FB690DB for ; Fri, 3 Jun 2016 15:01:27 +0000 (UTC) (envelope-from julien.cigar@gmail.com) Received: from relay-b02.edpnet.be (relay-b02.edpnet.be [212.71.1.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "edpnet.email", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 11EEA150F for ; Fri, 3 Jun 2016 15:01:26 +0000 (UTC) (envelope-from julien.cigar@gmail.com) X-ASG-Debug-ID: 1464966081-0a7b8d120f269ce00001-jLrpzn Received: from mordor.lan (109.236.137.241.wls.msr91gkk3.adsl.dyn.edpnet.net [109.236.137.241]) by relay-b02.edpnet.be with ESMTP id hW6HAitfFqGwXqxX (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 03 Jun 2016 17:01:22 +0200 (CEST) X-Barracuda-Envelope-From: julien.cigar@gmail.com X-Barracuda-Effective-Source-IP: 109.236.137.241.wls.msr91gkk3.adsl.dyn.edpnet.net[109.236.137.241] X-Barracuda-Apparent-Source-IP: 109.236.137.241 Date: Fri, 3 Jun 2016 17:01:20 +0200 From: Julien Cigar To: Valeri Galtsev Cc: Steve O'Hara-Smith , freebsd-questions@freebsd.org Subject: Re: redundant storage Message-ID: <20160603150120.GP95511@mordor.lan> X-ASG-Orig-Subj: Re: redundant storage References: <20160603083843.GK95511@mordor.lan> <20160603104138.fdf3c0ac4be93769be6da401@sohara.org> <20160603101446.GM95511@mordor.lan> <20160603114746.6b75e6e79ecd51fe14311e40@sohara.org> <20160603115020.GN95511@mordor.lan> <61821.128.135.52.6.1464964464.squirrel@cosmo.uchicago.edu> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="Zfao1/4IORAeFOVj" Content-Disposition: inline In-Reply-To: <61821.128.135.52.6.1464964464.squirrel@cosmo.uchicago.edu> User-Agent: Mutt/1.6.1 (2016-04-27) X-Barracuda-Connect: 109.236.137.241.wls.msr91gkk3.adsl.dyn.edpnet.net[109.236.137.241] X-Barracuda-Start-Time: 1464966081 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://212.71.1.222:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 5639 X-Virus-Scanned: by bsmtpd at edpnet.be X-Barracuda-BRTS-Status: 1 X-Barracuda-Bayes: INNOCENT GLOBAL 0.5000 1.0000 0.0100 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=9.0 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.30143 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Jun 2016 15:01:27 -0000 --Zfao1/4IORAeFOVj Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jun 03, 2016 at 09:34:24AM -0500, Valeri Galtsev wrote: >=20 > On Fri, June 3, 2016 6:50 am, Julien Cigar wrote: > > On Fri, Jun 03, 2016 at 11:47:46AM +0100, Steve O'Hara-Smith wrote: > >> On Fri, 3 Jun 2016 12:14:46 +0200 > >> Julien Cigar wrote: > >> > >> > On Fri, Jun 03, 2016 at 10:41:38AM +0100, Steve O'Hara-Smith wrote: > >> > > Hi, > >> > > > >> > > Just one change - don't use RAID1 use ZFS mirrors. ZFS does > >> > > better RAID than any hardware controller. > >> > > >> > right.. I must admit that I haven't looked at ZFS yet (I'm still usi= ng > >> > UFS + gmirror), but it will be the opportunity to do so..! > >> > > >> > Does ZFS play well with HAST? > >> > >> Never tried it but it should work well enough, ZFS sits on top of > >> geom providers so it should be possible to use the pool on the primary. > >> > >> One concern would be that since all reads come from local storage > >> the secondary machine never gets scrubbed and silent corruption never > >> gets > >> detected on the secondary. A periodic (say weekly) switch over and scr= ub > >> takes care of this concern. Silent corruption is rare, but the bigger > >> the > >> pool and the longer it's used the more likely it is to happen > >> eventually, > >> detection and repair of this is one of ZFSs advantages over hardware > >> RAID > >> so it's good not to defeat it. > > > > Thanks, I'll read a bit on ZFS this week-end ..! > > > > My ultimate goal would be that the HAST storage survives an hard reboot/ > > unplugged network cable/... during an heavy I/O write, and that the > > switch between the two nodes is transparent to the clients, without any > > data loss of course ... feasible or utopian? Needless to say that what > > I want to avoid at all cost is that the storage becomes corrupted and > > unrecoverable..! >=20 > Sounds pretty much like distributed file system solution. I tried one > (moosefs) which I gave up on, and after I asked (on this list) for advise > about other options, next candidate for me emerged: glusterfs, which I > hadn't chance to set up yet. You may want to search this list archives, > those were really good advises that experts gave me. sorry but: I avoid distributed FS like the plague :) >=20 > Valeri >=20 > > > >> > >> Drive failures on the primary will wind up causing both the primary > >> and the secondary to be rewritten when the drive is replaced - this > >> could > >> probably be avoided by switching primaries and letting HAST deal with > >> the > >> replacement. > >> > >> Another very minor issue would be that any corrective rewrites (for > >> detected corruption) will happen on both copies but that's harmless and > >> there really should be *very* few of these. > >> > >> One final concern, but it's HAST purely and not really ZFS. Writing > >> a large file flat out will likely saturate your LAN with half the > >> capacity > >> going to copying the data for HAST. A private backend link between the > >> two > >> boxes would be a good idea (or 10 gigabit ethernet). > > > > yep, that's what I had in mind..! one nic for the replication between > > the two HAST node, and one (CARP) nic by which clients access to > > storage.. > > > >> > >> > > On Fri, 3 Jun 2016 10:38:43 +0200 > >> > > Julien Cigar wrote: > >> > > > >> > > > Hello, > >> > > > > >> > > > I'm looking for a low-cost redundant HA storage solution for our > >> > > > (small) team here (~30 people). It will be used to store files > >> > > > generated by some webapps, to provide a redundant dovecot (imap) > >> > > > server, etc. > >> > > > > >> > > > For the hardware I have to go with HP (no choice), so I planned = to > >> buy > >> > > > 2 x HP ProLiant DL320e Gen8 v2 E3-1241v3 (768645-421) with > >> > > > 4 x WD Hard Drive Re SATA 4TB 3.5in 6gb/s 7200rpm 64MB Buffer > >> > > > (WD4000FYYZ) in a RAID1 config (the machine has a smartarray P222 > >> > > > controller, which is apparently supported by the ciss driver) > >> > > > > >> > > > On the FreeBSD side I plan to use HAST with CARP, and the volumes > >> > > > will be exported through NFS4. > >> > > > > >> > > > Any comments on this setup (or other recommendations) ? :) > >> > > > > >> > > > Thanks! > >> > > > Julien > >> > > > > >> > > > >> > > > >> > > -- > >> > > Steve O'Hara-Smith > >> > > > >> > > >> > >> > >> -- > >> Steve O'Hara-Smith | Directable Mirror Arra= ys > >> C:>WIN | A better way to focus the > >> sun > >> The computer obeys and wins. | licences available see > >> You lose and Bill collects. | http://www.sohara.org/ > >> > > > > -- > > Julien Cigar > > Belgian Biodiversity Platform (http://www.biodiversity.be) > > PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 > > No trees were killed in the creation of this message. > > However, many electrons were terribly inconvenienced. > > >=20 >=20 > ++++++++++++++++++++++++++++++++++++++++ > Valeri Galtsev > Sr System Administrator > Department of Astronomy and Astrophysics > Kavli Institute for Cosmological Physics > University of Chicago > Phone: 773-702-4247 > ++++++++++++++++++++++++++++++++++++++++ --=20 Julien Cigar Belgian Biodiversity Platform (http://www.biodiversity.be) PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. --Zfao1/4IORAeFOVj Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCgAGBQJXUZu9AAoJELK7NxCiBCPAtKkP/17yPdnVNZ3nPpRnZrOp/mWp rlRXanexejYcEJm0aRjfG0l1aWUivVGu6Q/+5KXWsK5fy5oMWkrMLVGq333adS69 E1t+a0hovRWs6gKAQNKpeATO99112QCFHghjJgeH1NSutok1tSAJrjLRsyXg+3SN t029iFK6T2snZFpTCsBozJKuOFAOryUQ6AnFPIi+WFkj5wAL/ydOCziKM/90W6U3 R0JxDAwteCeFXXorMFQbV8dXEJInZISdWusEl94LN3th3GaOrr+aUzO3WfN3RpE4 RDBpnUIaCacYyV2LHgTvIJSD2HZfQf5VkJR692eC/JbsRf+Xzb3gfJUGjXOgJb64 WO1r5T+9PiL31+P4cQXtGug3u/KdrjFND5K6qbrJf5ZCAsrcbLRSydUD+2MLjxaK Y/oGkgYp0M4kroXHv8TFlmsiPlfVk3vDed2EnkKzvaQLLXPqGeLNyChmK932SrzE emMQBdWR+Y+2SLEZOtSPp1T8L5bHscA8RC2Zt70X5X8N/byBaLBalf2ipYgp302H F2aXZatBOo+g9jQBWCXc/KsW/AXJRFxzChAHJktNw+2t2NQBMjC9KVNbuIuzm6II TOZTMj5Zr3CH9VtBO5swCCLhnYJGugmqWB+xaNQ5Cnmr2yV8U2bEwKEfaC1jnKe8 9zJgyigdPYag68iw5fla =3rDv -----END PGP SIGNATURE----- --Zfao1/4IORAeFOVj--