Date: Mon, 21 Jun 2004 21:04:38 -0400 From: Chuck Swiger <cswiger@mac.com> To: Bill Moran <wmoran@potentialtech.com> Cc: questions@freebsd.org Subject: Re: What's the best possible email failover solution Message-ID: <40D785A6.1040408@mac.com> In-Reply-To: <20040621203245.1f0e7444.wmoran@potentialtech.com> References: <20040621132006.2b1a296f.wmoran@potentialtech.com> <a22ff294040621115173bad2e0@mail.gmail.com> <20040621172520.3544d6fe.wmoran@potentialtech.com> <20040621214348.GB63857@happy-idiot-talk.infracaninophile.co.uk> <20040621175626.3e762448.wmoran@potentialtech.com> <40D76DA3.9090809@mac.com> <20040621203245.1f0e7444.wmoran@potentialtech.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Bill Moran wrote: > Chuck Swiger <cswiger@mac.com> wrote: [ ... ] >> The latter uses one-message-per-file, and ought to work *much* better both in >> terms of performance and stability, and in terms of playing nice with the way >> rsync wants to back things up. > > Doesn't really matter. Fact is, the mail directories are something on the order > of 3G. No matter how efficiently I store them, rsync is not going to be able > to back them up fast enough to hit the level of redundancy I'm shooting for. You may well be right, as you aren't really talking about performing backups, you're talking about creating a fully redundant storage which is kept up-to-date in realtime. > Although Maildirs might work a little better, since I wouldn't have to stop > the IMAP server during backup. That, and the granularity of one-message-per-file fits perfectly with rsync's file-driven model. > It takes about 30 minutes to rsync the system to the backup server right now. > That's perfectly acceptable for nightly backup purposes. This is a 1.5Ghz > with 256M RAM and 80G ATA 100 HDDs. If the system runs rysnc continuously > 24/7, I still have 30mins old data. Oh, yes. Just don't forget that if you do eliminate this time gap, you still ought to have another system actually taking backups. Any change the system encounters will be replicated to the redundant mail storage system in real time, including bad changes. >>[ I don't think that stuffing email into a database is a particularly good >>idea since that means keeping large blobs of non-relational data floating >>around, something that the filesystem can do a better job of handling... ] > > It's a good idea if I want real-time redundancy. I see where you're coming > from, and it's true that a RDBMS isn't the best way to store emails. But, > when you look at the features available, it's the best way for this > circumstance. With something like Slony, I'd have real-time redundancy > with (I'm expecting) only a minor performance drop. Although I can't be > sure until I can put something together to test. Reliability is much more > important than performance in this case. Who cares if their email takes > and extra 60 seconds to deliver, as long as it doesn't get lost! If the > email arrives fast, it's useless if the server fails and the email is > lost because the SMTP server told the delivering server that it had > arrived and then crashed before it could be backed up. I suspect that the relatively heavy weight of database transactions compared with filesystem access is going to slow things down a fair amount, too, particularly when running against a replicated DB. But reliability over performance is a fine choice to make. :-) Using RAID improves fault-tolerance, but you still end up with a single-point-of-failure at the system level; using database replication gives you higher availability, which seems to be what you mean when you talk about "reliability". Perhaps SAN or NAS concepts might be worth considering, as you can set up a fully-redundant fibre channel configuration where the storage is shared between two or more systems, thus with no single-point-of-failure. -- -Chuck
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?40D785A6.1040408>