Date: Mon, 30 Apr 2001 16:37:20 -0500 From: Gerd Knops <gerti@bitart.com> To: Ken Bolingbroke <hacker@bolingbroke.com> Cc: questions@freebsd.org Subject: Re: Redundant Internet connections [partial solution, comments requested] Message-ID: <20010430213721.1592.qmail@camelot.bitart.com> In-Reply-To: <Pine.BSF.4.21.0104281651400.87921-100000@fremont.bolingbroke.com> References: <Pine.BSF.4.21.0104281651400.87921-100000@fremont.bolingbroke.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Networking experts: Not being a networking expert myself, I would love some feedback on the concepts outlined below. Maybe this can be improved upon. Ken Bolingbroke wrote: > Given a FreeBSD box with _two_ independent connections to the > internet, and also serving as the gateway to a third, private > network, how would I configure it to use both Internet links as > "default" routes? I would prefer one over the other, but need it to > fall back to the second if the first goes offline. > As Ted an others said there is no built in way in FreeBSD to do this. However there are a number of tricks that can help you get the most out of this. I have a frame relay line as 'main' access to the internet. I used to use a DSL line as backup. It left the building on a different cable, but closer to the CO both ended up in the same cable. And as murphy had it, earlier this year a snow plow took a box out where both circuits were going through... So I now use 'cable' as backup. Cable is above ground here, telephone below ground, so that should give me better redundancy. Anyhow, back to the 'tricks' I mentioned. I assume you box connecting to both ISPs has it's external interface set to an IP inside the 'main' network (ISP A, network a.a.a.xx/yy), let us assume the address is a.a.a.s, and the gateway is a.a.a.gw. The external interface is also aliased aliased to an IP on the backup ISP's network (b.b.b.xx/yy), let's assume the address is b.b.b.s, and the gateway of ISP B is b.b.b.gw. The FreeBSD machine is set up with a.a.a.gw as the default gateway. In the above setup all responding packets go via the default gateway, no matter if the original request came through the a.a.a or the b.b.b network. ipfw (which comes with FreeBSD) gives us a tool to change this. To do so, use a rule set like this: add 1000 allow all from a.a.a.s to any add 1010 allow all from any to b.b.b.xx/yy add 1020 allow all from any to a.a.a.xx/yy add 1030 fwd b.b.b.gw all from b.b.b.s to any The trick here is the fourth line. If a packet originates on the aliased (b.b.b.s) address of your FreeBSD box, the fwd directive forces it to use the b.b.b.gw gateway instead the default (a.a.a.gw) gateway. The first three lines are to insure that local packets and packets coming from the a.a.a.s address are not modified. Now with this trick in place you can do a number of things to make your services redundant. If a.a.a.s is your mail server, you can add a second mx record (with lower priority) for b.b.b.s to your DNS. If your primary ISP goes down, incoming mail will automatically use the backup address. Even better: if somewhere in the internet something breaks so that a machine trying to send you EMail can not 'see' the a.a.a.s address, it will automatically switch to the b.b.b.s address. If a.a.a.s is your primary name server, you can also register b.b.b.s as a secondary name server for added redundancy. As far as WWW is concerned, things are a little more difficult. Obviously you can use two different names (www1.my.domain and www2.my.domain) and point them to a.a.a.s and b.b.b.s respectively. But that would require that people know to try www2.my.domain if www1.my.domain is not responding. Another option is to assign two ip addresses to the same name (eg point both a.a.a.s and b.b.b.s to www.my.domain). If bind is used as name server, it will return both addresses, but it will 'round robin' through those addresses (eg it alternates between returning 'a.a.a.s b.b.b.s' and 'b.b.b.s a.a.a.s'). So it will do some crude load balancing instead of preferring the a.a.a.s address. It depends on the clients what they do with those addresses. Some simple minded ones may just use the first address. Smarter ones will try the first address, and if that times out use the second address. So with a setup like this you do get some redundancy, but it will not prefer your main ISP and so not be the best choice if the pipes are of different speeds. It might be interesting to look into a patch for bind (or maybe djbdns) so that one could force the order in which addresses are returned. I also thought about using different instances of bind for the 2 networks, one bound to a.a.a.s returning addresses in the a.a.a network, and one bound to b.b.b.s returning addresses in the b.b.b network. However the order in which name servers are looked up is not determined, so you still could not direct 'default' traffic to the a.a.a network. Also it is generally expected that primary and secondary name servers return identical information, breaking this might have unexpected side effects. With all it's complexity, it is surprising that bind doesn't offer better tools to handle that kind of setup. The above discussed 'inbound connections'. 'Outbound connections' will still go out through the default gateway (unless you use tricks to bind them to the b.b.b.s address). I use a set of scripts to switch the default gateways and do that manually when required. However I guess that could be automated to some extend by using a program that pings an address 'just outside' of ISP A, switches to ISP B when that address does not respond, and switches back after it responds again. Note that when you switch default gateways, you also need to switch the ipfw rules outlined above. My scripts go a step further: They down the external interface, then set it to the address of the 'active' ISP and alias the address of the other ISP. I do this because I use ipf/ipnat (instead of ipfw) for the actual firewalling and NAT. In their configuration files you can use 0.0.0.0/32 instead of the 'real' address of your system, and ipf/ipnat will substitute that with the address of the interface. So by switching the address of the interface I do not have to restart ipf/ipnat, and can use the same rule sets. That keeps ipf's state tables intact, though that might be of little use if the connection is down. Any comments and suggestions are highly welcome. Gerd To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20010430213721.1592.qmail>