Date: Thu, 29 Oct 2015 19:17:40 +0100 From: Jan Bramkamp <crest@rlwinm.de> To: freebsd-fs@freebsd.org Subject: Re: iSCSI/ZFS strangeness Message-ID: <563262C4.1040706@rlwinm.de> In-Reply-To: <20151029015721.GA95057@mail.michaelwlucas.com> References: <20151029015721.GA95057@mail.michaelwlucas.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 29/10/15 02:57, Michael W. Lucas wrote: > The initiators can both access the iSCSI-based pool--not > simultaneously, of course. But CARP, devd, and some shell scripting > should get me a highly available pool that can withstand the demise of > any one iSCSI server and any one initiator. > > The hope is that the pool would continue to work even if an iSCSI host > shuts down. When the downed iSCSI host returns, the initiators should > log back in and the pool auto-resilver. I would recommend against using CARP for this because CARP is prone to split-brain situations and in this case they could destroy your whole storage pool. If the current head node fails the replacement has to `zpool import -f` the pool and and in the case of a split-brain situation both head nodes would continue writing to the iSCSI targets. I would move the leader election to an external service like consul, etcd or zookeeper. This is one case where the added complexity is worth it. If you can't run an external service for this e.g. it would exceed the scope of the chapter you're writing please simplify the setup with more reliable hardware, good monitoring and manual failover for maintenance. CARP isn't designed to implement reliable (enough) master election for your storage cluster. Adding iSCSI to your storage stack adds complexity and overhead. For setups which still fit inside a single rack SAS (with geom_multipath) is normally faster and cheaper. On the other hand you can't spread out SAS storage far enough to implement disaster tolerance should you really need it and it certainly is an interesting setup.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?563262C4.1040706>