From owner-freebsd-pf@FreeBSD.ORG Mon Oct 19 16:55:34 2009 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 11A57106566C for ; Mon, 19 Oct 2009 16:55:34 +0000 (UTC) (envelope-from mike@jellydonut.org) Received: from mail-fx0-f210.google.com (mail-fx0-f210.google.com [209.85.220.210]) by mx1.freebsd.org (Postfix) with ESMTP id 914F08FC14 for ; Mon, 19 Oct 2009 16:55:33 +0000 (UTC) Received: by fxm6 with SMTP id 6so5054095fxm.43 for ; Mon, 19 Oct 2009 09:55:32 -0700 (PDT) MIME-Version: 1.0 Received: by 10.223.14.145 with SMTP id g17mr1032303faa.51.1255970047643; Mon, 19 Oct 2009 09:34:07 -0700 (PDT) In-Reply-To: <36b1f3e60910190848h382cde04l104f2a9f466af3fa@mail.gmail.com> References: <36b1f3e60910190848h382cde04l104f2a9f466af3fa@mail.gmail.com> Date: Mon, 19 Oct 2009 12:34:07 -0400 Message-ID: <1de79840910190934w358e711t781f39061e16991@mail.gmail.com> From: Michael Proto To: Jed Gainer Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-pf@freebsd.org Subject: Re: PF - load balancing outgoing connections X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Oct 2009 16:55:34 -0000 On Mon, Oct 19, 2009 at 11:48 AM, Jed Gainer wrote: > I wanted to setup a machine as my LAN gateway and have it load balance ov= er > multiple WANs. When I found http://www.openbsd.org/faq/pf/pools.html I > choose FreeBSD as the machines OS. After getting it up and running, and > acting as a gateway just using one WAN via > > *# macros > wan1=3D"nfe0" > lan1=3D"rl0" > > pc1=3D"10.0.0.2" > xb1=3D"10.0.0.3" > > # options > #set block-policy return > #set loginterface $wan1 > set skip on lo0 > > # scrub > scrub in > > # nat/rdr > nat on $wan1 from !($wan1) -> ($wan1:0) static-port > > # uTorrent > rdr on $wan1 proto tcp from any to any port 41016 -> $pc1 > > # Xbox Live > rdr on $wan1 proto {tcp, udp} from any to any port 3074 -> $xb1* > > I decided to try the load balancing and came up with quite a few differen= t > pf.confs that did not work, my LAN just lost all connectivity when I load= ed > them. > * > lan1r =3D "10.0.0.0/24" > lan1 =A0=3D "rl0" > wan1 =3D "nfe0" > wan2 =3D "rl1" > gw1 =3D "10.0.1.2" > gw2 =3D "10.0.2.2" > > # nat outgoing connections on each internet interface > nat on $wan1 from $lan1r to any -> ($wan1) #static-port > nat on $wan2 from $lan1r to any -> ($wan2) #static-port > > # default deny > block in from any to any > block out from any to any > > # pass all outgoing packets on internal interface > pass out on $lan1 from any to $lan1r > > # pass in quick any packets destined for the gateway itself > pass in quick on $lan1 from $lan1r to $lan1 > > # load balance outgoing tcp traffic from internal network. > pass in on $lan1 route-to { ($wan1 $gw1), ($wan2 $gw2) } round-robin prot= o > tcp from $lan1r to any flags S/SA modulate state > > # load balance outgoing udp and icmp traffic from internal network > pass in on $lan1 route-to { ($wan1 $gw1), ($wan2 $gw2) } round-robin prot= o { > udp, icmp } from $lan1r to any keep state > > # general "pass out" rules for external interfaces > pass out on $wan1 proto tcp from any to any flags S/SA modulate state > pass out on $wan1 proto { udp, icmp } from any to any keep state > pass out on $wan2 proto tcp from any to any flags S/SA modulate state > pass out on $wan2 proto { udp, icmp } from any to any keep state > > # route packets from any IPs on $ext_if1 to $ext_gw1 and the same for > $ext_if2 and $ext_gw2 > pass out on $wan1 route-to ($wan2 $gw2) from $wan2 to any > pass out on $wan2 route-to ($wan1 $gw1) from $wan1 to any* > > ... and ... > > *lan =3D rl0 > wan1 =3D nfe0 > wan2 =3D rl1 > wan1_gw =3D 173.183.32.254 > wan2_gw =3D 10.0.1.2 > > nat on $wan1 from any to any -> ($wan1) > nat on $wan2 from any to any -> ($wan2) > > pass in quick on $lan route-to { ($wan1 $wan1_gw), ($wan2 $wan2_gw) } \ > =A0round-robin inet from ($lan:network) to any flags S/SA keep state* > > Neither of the above worked, or the many other attempts I made. > > No errors are reported when I `pfctl -f /etc/pf.lb.conf` and my LAN loose= s > internet connectivity. > > Does any one see the problem? I can ping Google fine using either WAN as > default route so it has to be my PF conf. > > I am at the point where I will pay someone to get it working! > -- > ~ Jed Gainer > _______________________________________________ > freebsd-pf@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-pf > To unsubscribe, send any mail to "freebsd-pf-unsubscribe@freebsd.org" > Correct me if I'm wrong, but I don't think you can do this without running a routing protocol with your upstream ISP. The problem is, regardless of which connection you send your traffic out, the return traffic will always come the same route from your ISP(s). If you send your traffic out $wan2 but your IP space is advertised by your ISP on $wan1 the traffic will always come back in $wan1 and you'll have an asymmetric route (as well as messed-up states in pf on the $wan1 and $wan2 interfaces). The only way I've been able to load-balance outbound traffic is to have different upstream routers advertise different routes back to my network via BGP and work the load-balancing that way. -Proto