From owner-freebsd-net@FreeBSD.ORG Thu Mar 3 06:30:34 2011 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7C7AD106564A for ; Thu, 3 Mar 2011 06:30:34 +0000 (UTC) (envelope-from tarkhil@webmail.sub.ru) Received: from mail.sub.ru (mail.sub.ru [88.212.205.2]) by mx1.freebsd.org (Postfix) with SMTP id BC1F98FC13 for ; Thu, 3 Mar 2011 06:30:33 +0000 (UTC) Received: (qmail 10130 invoked from network); 3 Mar 2011 09:30:42 +0300 Received: from tarkhil147-9.rostokino.net (tarkhil147-9.rostokino.net [89.222.147.9]) by mail.sub.ru ([88.212.205.2]) with ESMTP via TCP; 03 Mar 2011 06:30:42 -0000 Message-ID: <4D6F3581.6010906@webmail.sub.ru> Date: Thu, 03 Mar 2011 09:30:25 +0300 From: Alex Povolotsky User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.9.2.14) Gecko/20110221 Thunderbird/3.1.8 MIME-Version: 1.0 To: "Bjoern A. Zeeb" References: <4D4FA3DA.7010004@webmail.sub.ru> <20110302214601.S13400@maildrop.int.zabbadoz.net> In-Reply-To: <20110302214601.S13400@maildrop.int.zabbadoz.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-net@FreeBSD.org Subject: Re: jail source address selection doesn't work? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Mar 2011 06:30:34 -0000 03.03.2011 0:48, Bjoern A. Zeeb пишет: > On Mon, 7 Feb 2011, Alex Povolotsky wrote: > >> Hello! >> >> On a multihomed FreeBSD 8.1-RELEASE, in a multihomed jail, source IP >> selection suddenly refused to work. >> >> ifconfig on a box: > .... >> Seems reasonable, yes? >> >> Pinging from the box >> >> # ping 192.168.75.59 >> PING 192.168.75.59 (192.168.75.59): 56 data bytes >> 64 bytes from 192.168.75.59: icmp_seq=0 ttl=64 time=0.993 ms >> 64 bytes from 192.168.75.59: icmp_seq=1 ttl=64 time=0.986 ms >> 64 bytes from 192.168.75.59: icmp_seq=2 ttl=64 time=0.988 ms >> ^C >> --- 192.168.75.59 ping statistics --- >> 3 packets transmitted, 3 packets received, 0.0% packet loss >> round-trip min/avg/max/stddev = 0.986/0.989/0.993/0.003 ms >> >> 10:45:31.425232 IP 192.168.75.4 > 192.168.75.59: ICMP echo request, >> id 12430, seq 0, length 64 >> 10:45:31.426283 IP 192.168.75.59 > 192.168.75.4: ICMP echo reply, id >> 12430, seq 0, length 64 >> 10:45:32.425415 IP 192.168.75.4 > 192.168.75.59: ICMP echo request, >> id 12430, seq 1, length 64 >> 10:45:32.426404 IP 192.168.75.59 > 192.168.75.4: ICMP echo reply, id >> 12430, seq 1, length 64 >> >> Okay, yes? >> >> From jail: >> >> # ping 192.168.75.59 >> PING 192.168.75.59 (192.168.75.59): 56 data bytes >> ^C >> --- 192.168.75.59 ping statistics --- >> 2 packets transmitted, 0 packets received, 100.0% packet loss >> >> 10:45:52.146600 IP 83.69.203.1 > 192.168.75.59: ICMP echo request, id >> 14222, seq 0, length 64 >> 10:45:53.146702 IP 83.69.203.1 > 192.168.75.59: ICMP echo request, id >> 14222, seq 1, length 64 >> >> Setting ip.saddrsel to 1 or 0 did not change anything. Kernel is >> GENERIC+ALTQ >> >> What could I miss?... > > Don't use ping to test this. a) for ping inside the jail to work you > need to enable raw sockets b) a) could give you a hint that ping does > it's own thing. Telnet did all the same thing. > > Try a telnet to a random port to the destination and verify with > tcpdump whether things are still not working correctly, of if you > establish the connection with netstat. I used telnet to connect to specific ports. Ok, let's try again 104:tarkhil@box2.u.energodata.local:...local/etc/ezjail # jls JID IP Address Hostname Path 1 192.168.82.2 test /usr/jails/test 107:tarkhil@box2.u.energodata.local:...local/etc/ezjail # jls -j 1 ip4.saddrsel true 108:tarkhil@box2.u.energodata.local:...local/etc/ezjail # jls -j 1 ip4.addr 192.168.82.2,192.168.75.2 114:tarkhil@box2.u.energodata.local:...local/etc/ezjail # tcpdump -l -n -i bce0 host 192.168.82.2 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on bce0, link-type EN10MB (Ethernet), capture size 96 bytes 09:27:54.492105 IP 192.168.82.2.50823 > 192.168.72.3.22: Flags [S], seq 3819433473, win 65535, options [mss 1460,nop,wscale 3,sackOK,TS val 1306232522 ecr 0], length 0 115:tarkhil@box2.u.energodata.local:...local/etc/ezjail # ifconfig bce0 bce0: flags=8843 metric 0 mtu 1500 options=c01bb ether 00:14:5e:1a:a6:27 inet 192.168.80.41 netmask 0xffffff00 broadcast 192.168.80.255 media: Ethernet autoselect (100baseTX ) status: active test# sysctl security.jail.jailed security.jail.jailed: 1 test# ifconfig bce0: flags=8843 metric 0 mtu 1500 options=c01bb ether 00:14:5e:1a:a6:27 media: Ethernet autoselect (100baseTX ) status: active bce1: flags=8843 metric 0 mtu 1500 options=c01bb ether 00:14:5e:1a:a6:29 media: Ethernet autoselect (100baseTX ) status: active lo0: flags=8049 metric 0 mtu 16384 options=3 vlan75: flags=8843 metric 0 mtu 1500 options=103 ether 00:14:5e:1a:a6:29 inet 192.168.75.2 netmask 0xffffff00 broadcast 192.168.75.255 media: Ethernet autoselect (100baseTX ) status: active vlan: 75 parent interface: bce1 vlan82: flags=8843 metric 0 mtu 1500 options=103 ether 00:14:5e:1a:a6:29 inet 192.168.82.2 netmask 0xffffff00 broadcast 192.168.82.255 media: Ethernet autoselect (100baseTX ) status: active vlan: 82 parent interface: bce1 In other words, source address is selected as primary IP, and packet runs out on 100% improper interface. No specific routing, no firewall. Alex.