Date: Fri, 17 Dec 2010 10:19:51 +0100 From: Daniel Hartmeier <daniel@benzedrine.cx> To: Jayster <jayster@comcast.net> Cc: freebsd-net@freebsd.org Subject: Re: Web Server supporting up to 4 WANs/Interfaces Message-ID: <20101217091951.GA9013@insomnia.benzedrine.cx> In-Reply-To: <17835728.248313.1292567569116.JavaMail.root@sz0077a.emeryville.ca.mail.comcast.net> References: <17835728.248313.1292567569116.JavaMail.root@sz0077a.emeryville.ca.mail.comcast.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Dec 17, 2010 at 06:32:49AM +0000, Jayster wrote: > I have tried both PF and IPFW. Different posts around the web claim Multiple Gateway solutions using both of them. I have tried each of the recommended setups, but had no luck. If you read the last responses to each of those posts, others also state they could not duplicate what is claimed, as well. PF looks the most promising. It has "if-bound", which is supposed to keep interface traffic on the same interface. That is a good first step. But pointing it to the gateway on that interface is still an issue. Please HELP!!! I haven't slept in days and I've been stuck for a week now!!! This is our last showstopper. Ignore "if-bound", it's not relevant in your case. Ignore "route-to", as it only applies to outgoing connections (from the perspective of the web server), and those HTTP connections are incoming. Look at "reply-to". It is used on "pass in" rules that create state, causing the replies (the packets sent by the server matching the state) to get sent through a specific interface to a specific gateway, overriding any routing table lookups (like the default gateway). Let's say your default gateway is connected through bge0, and you have a second interface bge1. The problem is that when a non-local client's connection arrives in through bge1, the reply is sent out through bge0, where the default gateway is, instead of the bge1. You add a rule like pass in on bge1 reply-to (bge1 1.2.3.4) proto tcp to bge1 port 80 where 1.2.3.4 is the IP address of the gateway in the bge1 network, i.e. the next hop router towards the client. If it doesn't work right away, there are several things to check: - Make sure pf is enabled (pfctl -si) and the right ruleset is loaded (pfctl -sr) - If you have a non-trivial ruleset already, start over with a trivial one just for this purpose, you can merge them later, when everything works - Verify the connections on bge1 really do come in on that interface (tcpdump -ni bge1 tcp port 80) - Verify those incoming connections match the right pf.conf rule (pfctl -vvss, pfctl -vvsr) - Verify that 1.2.3.4 is within bge1's subnet (ifconfig bge1), and that you can ping it from the web server - Verify that 1.2.3.4's MAC address resolves (arp -an) - Check /var/log/messages for any errors - Does the SYN+ACK reply from the web server go out anywhere? (tcpdump -ni bge0/bge1/... tcp port 80) HTH, Daniel
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20101217091951.GA9013>