From owner-freebsd-net@FreeBSD.ORG Sat Mar 14 18:05:10 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 43CEF1065672 for ; Sat, 14 Mar 2009 18:05:10 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mail.cksoft.de (mail.cksoft.de [195.88.108.3]) by mx1.freebsd.org (Postfix) with ESMTP id CA7848FC13 for ; Sat, 14 Mar 2009 18:05:09 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from localhost (amavis.fra.cksoft.de [192.168.74.71]) by mail.cksoft.de (Postfix) with ESMTP id 8BBD241C6EA; Sat, 14 Mar 2009 19:05:08 +0100 (CET) X-Virus-Scanned: amavisd-new at cksoft.de Received: from mail.cksoft.de ([195.88.108.3]) by localhost (amavis.fra.cksoft.de [192.168.74.71]) (amavisd-new, port 10024) with ESMTP id Jmt8oE5LRSrS; Sat, 14 Mar 2009 19:05:06 +0100 (CET) Received: by mail.cksoft.de (Postfix, from userid 66) id 20B7141C6A1; Sat, 14 Mar 2009 19:05:06 +0100 (CET) Received: from maildrop.int.zabbadoz.net (maildrop.int.zabbadoz.net [10.111.66.10]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.int.zabbadoz.net (Postfix) with ESMTP id D9C214448E6; Sat, 14 Mar 2009 18:01:58 +0000 (UTC) Date: Sat, 14 Mar 2009 18:01:58 +0000 (UTC) From: "Bjoern A. Zeeb" X-X-Sender: bz@maildrop.int.zabbadoz.net To: Stefan Bethke In-Reply-To: Message-ID: <20090314174526.E96785@maildrop.int.zabbadoz.net> References: X-OpenPGP-Key: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-net@freebsd.org Subject: Re: Multi-homing, jails, and source address selection X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Mar 2009 18:05:10 -0000 On Thu, 12 Mar 2009, Stefan Bethke wrote: Hi, > I'm having some trouble configuring a dual-homed jail host, running -current > from about 4 weeks ago. > > My machine has one external interface em0 connected to an /27 IPv4 network. > Additionally, I have a VPN interface tun0 provided by an OpenVPN instance > with a private /18 range. > > I'd like my jails to be dual-homed, with a public and a VPN address each. > Processes in the jail should pick the appropriate source address depending on > the destination address, so that the source address for a connection going to > a VPN address will be the jails' VPN address, and all other connections will > use the jails' public IP. > > I have a couple of questions that I can't seem to find answers to: > > How do I get the VPN addresses configured? tun0 won't accept them (since ptp > interfaces require a destination address). If I use lo0, I seem to have > source address selection issues. I've experimented with various setups, but > haven't found one that would work just right. In the example below, if I > ping from foo to a VPN address, the source address is foo's public IP. If I > run ping with -S10.0.63.3, the source address still is 192.0.2.3. > > Is there any documentation on how source addresses are selected? I thought I > remembered that on unbound sockets the destination route would be used to > pick the first address of the outgoing interface as the source address; the > same address would be picked on connecting a socket. > > I'm currently running with this configuration in rc.conf: > > cloned_interfaces="tun0" > ifconfig_em0="192.0.2.2/27" > ifconfig_tun0="10.0.63.1 10.0.63.255" > > defaultrouter="192.0.2.1" > inetd_flags="-wW -a 192.0.2.2" > static_routes="openvpn" > route_openvpn="10.0.0.0/18 10.0.63.255" > > jail_enable="YES" > jail_set_hostname_allow="NO" > jail_sysvipc_allow="YES" > jail_devfs_enable="YES" > jail_mount_enable="YES" > > jail_list="foo bar baz" > jail_foo_rootdir="/jail/foo.example.com" > jail_foo_hostname="foo.example.com" > jail_foo_ip="em0|192.0.2.3,lo0|10.0.63.3" > > > Any suggestions? sys/netinet/in_pcb.c:in_pcbladdr() is your friend - http://fxr.watson.org/fxr/source/netinet/in_pcb.c#L546 This is the case you are running into: http://fxr.watson.org/fxr/source/netinet/in_pcb.c#L628 /* * If the outgoing interface on the route found is not * a loopback interface, use the address from that interface. * In case of jails do those three steps: * 1. check if the interface address belongs to the jail. If so use it. * 2. check if we have any address on the outgoing interface * belonging to this jail. If so use it. * 3. as a last resort return the 'default' jail address. */ so you are hitting "3." . I am not sure but I'd assume ifconfig tun0 10.0.63.3 10.0.63.255 alias would work, just not with the logic to create the IPs upon jail start (and we will not accept patches to handle that;). I'd also suggest to use telnet instead of ping to run tests as ping imho has some private magic. /bz -- Bjoern A. Zeeb The greatest risk is not taking one.