e@freebsd.org> List-Unsubscribe: Sender: owner-freebsd-net@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Source IPv4 address selection vs BGP IX connection To: Gregory Shapiro , freebsd-net@freebsd.org References: Content-Language: en-US From: Marek Zarychta Autocrypt: addr=zarychtam@plan-b.pwste.edu.pl; keydata= xsBNBFfi3cMBCADLecMTFXad4uDXqv3eRuB4qJJ8G9tzzFezeRnnwxOsPdytW5ES2z1ibSrR IsiImx6+PTqrAmXpTInxAi7yiZGdSiONRI4CCxKY9d1YFiNYT/2WyNXCekm9x29YeIU7x0JB Llbz0f/9HC+styBIu2H+PY/X98Clzm110CS+n/b9l1AtiGxTiVFj7/uavYAKxH6LNWnbkuc5 v8EVNc7NkEcl5h7Z9X5NEtzDxTOiBIFQ/kOT7LAtkYUPo1lqLeOM2DtWSXTXQgXl0zJI4iP1 OAu4qQYm2nXwq4b2AH9peknelvnt1mpfgDCGSKnhc26q6ibTfMwydp+tvUtQIQYpA6b9ABEB AAHNN01hcmVrIFphcnljaHRhIChQbGFuLWIpIDx6YXJ5Y2h0YW1AcGxhbi1iLnB3c3RlLmVk dS5wbD7CwHcEEwEIACEFAlfi4LkCGwMFCwkIBwIGFQgJCgsCBBYCAwECHgECF4AACgkQHZW8 vIFppoJXdgf8D9X3VRFSNaR9lthSx/+uqas17J3FJKBo1xMQsC2a+44vzNvYJSuPGLLJ+LW2 HPVazjP/BWZJbxOYpliY4zxNRU0YCp0BLIVLibc//yax+mE42FND/+NiIZhqJscl6MLPrSwo sIwXec4XYkldkyqW/xBbBYXoIkBqdKB9j5j42Npy1IV/RizOSdmvTWY27ir8e/yGMR1RLr4F 8P5K3OWTdlGy2H2F/3J8bIPBLG6FpaIyLQw4dHSx8V02PYqDxK1cNo2kAOnU8PnZL/AGuMOH iv3MN1VYL8ehcmpBBsrZGebQJxrjY2/5IaTSgp9xHYT70kshuU6Qb97vk1mOjNZxgc7ATQRX 4t3DAQgA10h6RCXuBLMHxq5B8X/ZIlj9sgLoeyfRdDZEc9rT2KUeUJVHDsbvOFf4/7F1ovWY hJbA6GK/LUZeHHTjnbZcH1uDYQeHly4UOLxeEvhGoz4JhS2C7JzN/uRnwbdOAUbJr8rUj/IY a7gk906rktsc/Ldrxrxh7O6WO0JCh2XO/p4pDfEwwB37g4xHprSab28ECYJ9JMbtA8Sy4M55 g3+GQ28FvSlGnx48OoGXU2BZdc1vZKSQmNOlikB+9/hDX8zdYWVfDaX1TLQ8Ib4+xTUmapza mV/bxIsaZRBw+jFjLQHhTbIMfPEU+4mxFDvTdbKPruKPqVf1ydgMnPZWngowdwARAQABwsBf BBgBCAAJBQJX4t3DAhsMAAoJEB2VvLyBaaaC6qkIAJs9sDPqrqW0bYoRfzY6XjDWQ59p9tJi v8aogxacQNCfAu+WkJ8PNVUtC1dlVcG5NnZ80gXzd1rc8ueIvXlvdanUt/jZd8jbb3gaDbK3 wh1yMCGBl/1fOJTyEGYv1CRojv97KK89KP5+r8x1P1iHcSrunlDNqGxTMydNCwBH23QcOM+m u4spKnJ/s0VRBkw3xoKBZfZza6fTQ4gTpAipjyk7ldOGBV+PvkKATdhK2yLwuWXhKbg/GRlD 1r5P0gxzSqfV4My+KJuc2EDcrqp1y0wOpE1m9iZqCcd0fup5f7HDsYlLWshr7NQl28f6+fQb sylq/j672BHXsdeqf/Ip9V4= In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:206006, ipnet:2001:678:618::/48, country:PL] X-Rspamd-Queue-Id: 4VPRsP74KQz4VJJ W dniu 24.04.2024 o 04:12, Gregory Shapiro pisze: > Short version: > > Using FreeBSD as a BGP router has network issues caused by suboptimal > default IPv4 source address selection when connected to Internet > Exchanges (which are required to use IPs that aren't routable on the > Internet). I was hoping to find more elegant workarounds or encourage > FreeBSD to add source IPv4 selection akin to the existing IPv6 source > address selection (no_prefer_iface and prefer_source). > > > Long version: > > Unless I'm mistaken, today, there is no way to set the default > IPv4 source address for connections like there is with IPv6 (using > no_prefer_iface and prefer_source). > > It appears the default source IP is chosen based on IP address of > the outbound interface for the packet. This presents a problem on > FreeBSD systems acting as BGP routers that have connections to Internet > exchanges (IX). One of the rules of IX IP addresses is that they are > must not be routable on the Internet. > > As a simple example, a system with two Ethernet interfaces, one to the > transit provider and one to an IX would look like this: > > vtnet0: flags=1008843 metric 0 mtu 1500 > description: Uplink > inet 193.148.250.141 netmask 0xffffff00 broadcast 193.148.250.255 > vtnet1: flags=1008843 metric 0 mtu 1500 > description: IX > inet 185.1.147.211 netmask 0xffffff00 broadcast 185.1.147.255 > > Then if /etc/resolv.conf contains 8.8.8.8 and BGP selects a route for > 8.8.8.0/24 over the IX, you end up with: > > # route -n get 8.8.8.8 > route to: 8.8.8.8 > destination: 8.8.8.0 > mask: 255.255.255.0 > gateway: 185.1.147.22 > fib: 0 > interface: vtnet1 > flags: > recvpipe sendpipe ssthresh rtt,msec mtu weight expire > 0 0 0 0 1500 1 0 > > And DNS on the system doesn't work as all DNS requests go out with a > source address of 185.1.147.211 (the IX endpoint) which isn't exported > as an Internet route. > > While I can set a static route for 8.8.8.8 for this particular case, it > would be messy to have to set up static routes for every possible local > connection (other DNS servers, outbound SMTP for periodic/cron mail, > etc.). > > I assume that there is a group of BGP enthusiasts using FreeBSD lurking > on freebsd-net. What have you done to solve this problem? > > I'd also love to hear other tips for running BGP on FreeBSD. > In this case, probably best solution will probably be using multiple FIBs. Running a BGP routing daemon under not default FIB after assigning its interface to this FIB should solve the problem but it might create eventually new problems to solve (for example in which FIB should imported routes be stored). It's also possible to set and use non-default FIB for DNS lookups and maintenance tasks like pkg upgrade (setfib -1 pkg ....). This approach is probably more straightforward to conduct. -- Marek Zarychta