Date: Tue, 23 Apr 2024 19:12:15 -0700 From: Gregory Shapiro <gshapiro@freebsd.org> To: freebsd-net@freebsd.org Subject: Source IPv4 address selection vs BGP IX connection Message-ID: <xrxvyz6h3t45tfbqxag2ueqe6ocg2myxhdg7kqsbjx6czj4xeo@jqwioylxcb2c>
next in thread | raw e-mail | index | archive | help
Short version:
Using FreeBSD as a BGP router has network issues caused by suboptimal
default IPv4 source address selection when connected to Internet
Exchanges (which are required to use IPs that aren't routable on the
Internet). I was hoping to find more elegant workarounds or encourage
FreeBSD to add source IPv4 selection akin to the existing IPv6 source
address selection (no_prefer_iface and prefer_source).
Long version:
Unless I'm mistaken, today, there is no way to set the default
IPv4 source address for connections like there is with IPv6 (using
no_prefer_iface and prefer_source).
It appears the default source IP is chosen based on IP address of
the outbound interface for the packet. This presents a problem on
FreeBSD systems acting as BGP routers that have connections to Internet
exchanges (IX). One of the rules of IX IP addresses is that they are
must not be routable on the Internet.
As a simple example, a system with two Ethernet interfaces, one to the
transit provider and one to an IX would look like this:
vtnet0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
description: Uplink
inet 193.148.250.141 netmask 0xffffff00 broadcast 193.148.250.255
vtnet1: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
description: IX
inet 185.1.147.211 netmask 0xffffff00 broadcast 185.1.147.255
Then if /etc/resolv.conf contains 8.8.8.8 and BGP selects a route for
8.8.8.0/24 over the IX, you end up with:
# route -n get 8.8.8.8
route to: 8.8.8.8
destination: 8.8.8.0
mask: 255.255.255.0
gateway: 185.1.147.22
fib: 0
interface: vtnet1
flags: <UP,GATEWAY,DONE,PROTO1>
recvpipe sendpipe ssthresh rtt,msec mtu weight expire
0 0 0 0 1500 1 0
And DNS on the system doesn't work as all DNS requests go out with a
source address of 185.1.147.211 (the IX endpoint) which isn't exported
as an Internet route.
While I can set a static route for 8.8.8.8 for this particular case, it
would be messy to have to set up static routes for every possible local
connection (other DNS servers, outbound SMTP for periodic/cron mail,
etc.).
I assume that there is a group of BGP enthusiasts using FreeBSD lurking
on freebsd-net. What have you done to solve this problem?
I'd also love to hear other tips for running BGP on FreeBSD.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?xrxvyz6h3t45tfbqxag2ueqe6ocg2myxhdg7kqsbjx6czj4xeo>