Date: Thu, 17 Sep 2009 09:09:48 -0700 From: "Michael K. Smith - Adhost" <mksmith@adhost.com> To: "Steve Bertrand" <steve@ibctech.ca> Cc: questions@freebsd.org Subject: RE: New mail server setup Message-ID: <17838240D9A5544AAA5FF95F8D52031606AFCB5B@ad-exh01.adhost.lan> In-Reply-To: <4AB2306D.1070009@ibctech.ca> References: <4AB0584D.3050206@eagle.ca><4AB0844B.5080804@infracaninophile.co.uk><4AB0E97B.4060606@ibctech.ca><4AB0ECE8.2080905@infracaninophile.co.uk> <4AB0F17B.1030400@ibctech.ca> <17838240D9A5544AAA5FF95F8D52031606AFCA07@ad-exh01.adhost.lan> <4AB2306D.1070009@ibctech.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
Hello Steve: I'll try to answer your questions in line. =20 <snip> >=20 > > Another approach would be a cluster of Postfix servers and Dovecot > > servers behind PF load balancers. We have 3 "POP" servers > (IMAP/POP), 9 > > Mail Servers, 2 Defer servers and 5 Filter servers that process over > 20 > > million messages a day without a blip. We can take individual > servers > > out of the pool for maintenance, etc. Everything is fed to a set of > > redundant NAS for the data storage and common configuration files. >=20 > Thanks Mike, >=20 > I'm interested to learn a little more about your setup. I was going to > take it off-list, but if you can provide some further details, it would > probably add long-term value to keep it here. >=20 > So, a couple of questions: >=20 > - can your PF load balancers 'sense' when one of the Postfix/Dovecot > units are down, or is this a manual change in config to prevent any > time-out conditions? Not natively. When we initially implemented this setup, ifstated wasn't up to snuff, so we wrote some PERL scripts that make connections to the required ports and, if no connection is established, pull the server from the table and send us an alarm. We also have scripts so that we can pull servers out when we're doing maintenance. >=20 > I like this load balancer idea. In my environment, it would be trivial > to set up a couple of them, throw Quagga on them, and integrate them > directly into our iBGP setup. On the other side, I could use VRRP or > the > like to ensure redundancy from front to back. We use two PF boxes and CARP with PFSync for failover, so no dynamic protocols are needed. =20 >=20 > - do the Postfix/Dovecot servers communicate with each other, or are > they simply stand-alone units that don't know/care that they have other > peers helping with the workload? >=20 They are standalone. All of the user authentication is handled from a centralized database, so there are no local credentials stored on the server. =20 > - are your filter servers in front of, or behind the load balancers > (iow, is all of your inbound email passed through the balancers, and > then filtered/processed/delivered in behind them)? > They are behind the PF boxes. We have other hooks in PF that we use to block SPAM in PF, including Cloudmark and some custom stuff that looks for multiple mails to non-existent addresses. We also use the overload tables for abusive connections. =20 > - how do all of the pieces communicate with the NAS...NFS? Yes. Originally we used TCP but we found performance to be much better with UDP. NFSv3 by the way. >=20 > - could you share a small snip of your PF config in relation to > load-balancing, so I can get a bit of a better understanding config- > wise > on how that piece hangs together? (I've never used PF, only IFPW ;) That might be difficult because it's about 720 lines. :-) Here are some highlights, though. 1) Our customers use mail.adhost.com for everything - SMTP, POP and IMAP. We use redirects in PF so that traffic coming in on the associated ports goes to the appropriate servers. 2) We have our load-balanced DNS servers behind the same PF boxes so we localize the tons of DNS queries related to mail. 3) We do a lot of our rejecting in PF, including Spamhaus, Cloudmark, check scripts for Phishing, Porn and Viruses, as well as our own list of "Nefarious" IP's culled from various sources. When traffic matches these originators, we send them to mail reject servers that send out a 550 message with the group name so we can find false positives more quickly. 4) Because 3 does have false positives, we have a whitelist that we can add to that will pass traffic to the mail servers before they match against any of the tables in 3. 5) We use POP before SMTP, so once we authenticate a user to send, their IP address is also added to an allow table. 6) The filter servers are load balanced to and from the mail servers so we can take them in and out of their pool for maintenance. If you have a particular scenario you're thinking about I could help you with the rules to make it work. Regards, Mike
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?17838240D9A5544AAA5FF95F8D52031606AFCB5B>