Date: Mon, 23 Jul 2007 06:47:58 +0200
From: Sten Daniel Soersdal <netslists@gmail.com>
To: m_wlist@weirdwire.ru
Cc: freebsd-net@freebsd.org
Subject: Re: Policy-based routing for packets originating from local machine ('reinject' packets back into kernel?)
Message-ID: <46A432FE.8000100@gmail.com>
In-Reply-To: <51976.10.23.23.1.1185116844.squirrel@mail.weirdwire.ru>
References: In-Reply-To: <51976.10.23.23.1.1185116844.squirrel@mail.weirdwire.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
m_wlist@weirdwire.ru wrote:
> Hello.
>
> I have a FreeBSD machine connected to internal network and several ISPs. I
> have set up (with pf) nat and balanced (policy-based) routing for machines
> from internal network, proper routing for remote connections to interfaces
> that connected to corresponding ISPs.
>
> One thing that I can't manage to get done is to make balanced routing to
> work with packets originating from the router itself. Simple pf route-to
> rules don't work as it seems local packets don't have any 'in' interface
> -- they just emerge somewhere in kernel just before routing table and when
> they are accessible to pf they are already in 'out' part where policy
> routing is not possible (the packet are already going out on interface
> that is determined from rather static routing table).
> When there are no default route in routing table packets just have nowhere
> to go and fail not even reaching any pf rules.
>
> I suddenly came up with an idea how to try to get that done -- somehow
> 'reinject' packets back into kernel to get them processed by 'in' rule
> with 'route-to'. The idea sounds quite simple -- make some virtual
> interface with non-existing address and subnet and make default route be
> somewhere in that subnet. Then make another virtual interface and
> bridge/connect (with netgraph for example) it to first one the way that
> any packets sent out to first one appear back in on second.
>
> The question number one: Is there any less retarded way to do PBR for
> local packets?
> The question number two: How to do that properly?
>
> At the moment I'm trying to get that working with netgraph's ngeth
> interfaces. But they seem to behave in some really weird way.
> Details:
> # ifconfig ngeth0 10.42.42.1 netmask 255.255.255.250
> # ngctl connect ngeth0: ngeth1: lower upper
> # ngctl connect ngeth1: ngeth0: lower upper
> ('tcpdump -ni ngeth0' on other terminal for great justice)
> # ping 10.42.42.2
> (here after some delay I get 'host is down' messages with no output from
> tcpdump).
> # ping 10.42.42.5
> (broadcast address, gives nothing from ping, and 'blal blah 10.42.42.1 >
> 10.42.42.5: ICMP echo request, blah' from tcpdump)
> (here i change tcpdump from ngeth0 to ngeth1)
> # ping 10.42.42.2 and # ping 10.42.42.3
> give 'host is down' from ping and nothing from tcpdump
> # ping 10.42.42.4
> (LOL WUT!) still gives 'host is down' from ping, but tcpdump -ni ngeth1
> gives 'arp who-has 10.42.42.4 tell 10.42.42.1'!
>
> That raises two questins:
> 1) Wtf is going on?
> 2) How to make ngeth just send ip packet, avoiding that arp stuff (or is
> there any other virtual interface devices available that do that)?
>
> Seems like I don't understand something completely.
Perhaps i'm not following completely but wouldn't this normally be the
job of the loopback interface?
I may be wrong about the specifics, as i haven't done this in many many
years and i used ipfw on 4.x which may or may not be using different
"paths".
Basically you could route the default route to a loopback interface ip
(127.0.0.1??) and it should, theoretically, exit and enter lo0.
*If* lo0 is clonable in some way you should be able to create additional
loopbacks if necessary (kernel compiletime?) in case you want to skip
filter checks on lo0 specifically or just want to see how twisted it can
all turn out.
--
Sten Daniel Soersdal
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?46A432FE.8000100>