Date: Thu, 17 Apr 2008 14:55:12 -0400 From: "Zaphod Beeblebrox" <zbeeble@gmail.com> To: "Pete French" <petefrench@ticketswitch.com> Cc: stable@freebsd.org Subject: Re: Dreadful gmirror performance, though each half works fine Message-ID: <5f67a8c40804171155o72b2ab1ctbc116510c39025f3@mail.gmail.com> In-Reply-To: <E1JmYn9-000Fe7-R4@dilbert.ticketswitch.com> References: <5f67a8c40804171109o264ad120wa442f21be8a4bb33@mail.gmail.com> <E1JmYn9-000Fe7-R4@dilbert.ticketswitch.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Apr 17, 2008 at 2:25 PM, Pete French <petefrench@ticketswitch.com> wrote: > > I have experimented with this rather extensively and have operationally > > decided not to use ggated in combination with gmirror --- it doesn't > appear > > to work as well as one might expect. > > Ah, thats unmfortunate :-( I oroginally started off using the > iscsi initiator and target, which did work O.K., but when actually > ran live ended up locking up after several hours,a nd then panicing > the kernel. So not ideal - but when it was working it was fine. ggated > seems the opposite - doesnt crash, but performance is not suitable for any > kind of real use. > In the end we found that ggate was crashy after a week or two of heavy use, too... dispite it's performance problems (which can be somewhat fixed by telling gmirror to only read from the local disk) > I'm somewhat vaguely wondering if zfs with one local and one ggated disk > > will work well. > > I tried ZFS for a while myself, and it works O.K., but has a tendency > to panic if it wants memory which it can't get. Despite the many different > guides available, I never managed to get it to the point where I would > be happy to use it on a production system without worrying about it > suddenly becomming memory hungry and dieing. Certainly ZFS needs lots of memory --- most of my systems running ZFS have 4G of RAM and are running in 64 bit mode. With the wiki's recomendation of a large number of kernel pages, I havn't had a problem with crashing. I am using ZFS RAIDZ as a large data store and ZFS mirroring (separately) on my workstation as /usr, /var, and home directories. > Thanks for the inout though - I am doing some more experimentation > with ggate (basically raing some buffers as per a thread I found) and > seeing if that helps. > > BTW, I think ggate is the problem and not gmirror here - gmirror on top > of iscsi works fine as I said. > I would agree... save the fact that it may be an interaction between the two and/or UFS that is causing the problems. Certainly gmirror on local disks works fine (I've run gmirror/gstripe combinations for several years now as RAID 10 store with UFS on top). This is all going to be latency sensitive --- ggate needs to allow a larger number of oustanding transactions to be efficient. Removing the read load from the ggated drive seems to help quite a bit in overall performance. But even with this change, I still found that ggate would crash after several days to a week of heavy use.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5f67a8c40804171155o72b2ab1ctbc116510c39025f3>