From owner-freebsd-bugs@FreeBSD.ORG Fri Feb 14 08:00:00 2014 Return-Path: Delivered-To: freebsd-bugs@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8BF6EB40 for ; Fri, 14 Feb 2014 08:00:00 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6BFA41A1E for ; Fri, 14 Feb 2014 08:00:00 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id s1E800JB093227 for ; Fri, 14 Feb 2014 08:00:00 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s1E800pE093226; Fri, 14 Feb 2014 08:00:00 GMT (envelope-from gnats) Resent-Date: Fri, 14 Feb 2014 08:00:00 GMT Resent-Message-Id: <201402140800.s1E800pE093226@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Ben Wilber Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CA70992C for ; Fri, 14 Feb 2014 07:51:37 +0000 (UTC) Received: from exile.desync.com (exile.desync.com [IPv6:2607:f178::167]) by mx1.freebsd.org (Postfix) with ESMTP id 8679D19CE for ; Fri, 14 Feb 2014 07:51:37 +0000 (UTC) Received: by exile.desync.com (Postfix, from userid 666) id B72493861; Fri, 14 Feb 2014 02:51:27 -0500 (EST) Message-Id: <20140214075127.B72493861@exile.desync.com> Date: Fri, 14 Feb 2014 02:51:27 -0500 (EST) From: Ben Wilber To: FreeBSD-gnats-submit@freebsd.org X-Send-Pr-Version: 3.114 Subject: kern/186755: ipsec tunnels don't work with pf or ipfw X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: Ben Wilber List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 08:00:00 -0000 >Number: 186755 >Category: kern >Synopsis: ipsec tunnels don't work with pf or ipfw >Confidential: no >Severity: serious >Priority: medium >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Fri Feb 14 08:00:00 UTC 2014 >Closed-Date: >Last-Modified: >Originator: Ben Wilber >Release: FreeBSD 10.0-STABLE amd64 >Organization: >Environment: System: FreeBSD server 10.0-STABLE FreeBSD 10.0-STABLE #2 r261258M: Fri Jan 31 17:36:49 EST 2014 bw@exodus:/usr/obj/factory/stable/sys/COMRADE amd64 >Description: Traffic forwarded from an IPsec SA in tunnel mode appears to bypass pf and ipfw. Last tested working in 9.2-STABLE around December 2013. >How-To-Repeat: Quick lab with two instances of bhyve, server and client. Set up basic connectivity: root@server:~ # ifconfig vtnet0 208.83.20.170/27 root@server:~ # route add default 208.83.20.161 add net default: gateway 208.83.20.161 fib 0 root@client:~ # ifconfig vtnet0 208.83.20.171/27 Create an IPsec tunnel. On server: echo add 208.83.20.170 208.83.20.171 esp 31337 -m tunnel -E des-cbc 0xc001c001c001c001\; | setkey -c echo add 208.83.20.171 208.83.20.170 esp 31338 -m tunnel -E des-cbc 0xc001c001c001c001\; | setkey -c echo 'spdadd 0.0.0.0/0[any] 10.0.0.1/32[any] any -P out ipsec esp/tunnel/208.83.20.170-208.83.20.171/require;' | setkey -c echo 'spdadd 10.0.0.1/32[any] 0.0.0.0/0[any] any -P in ipsec esp/tunnel/208.83.20.171-208.83.20.170/require;' | setkey -c On client: echo add 208.83.20.170 208.83.20.171 esp 31337 -m tunnel -E des-cbc 0xc001c001c001c001\; | setkey -c echo add 208.83.20.171 208.83.20.170 esp 31338 -m tunnel -E des-cbc 0xc001c001c001c001\; | setkey -c echo 'spdadd 10.0.0.1/32[any] 0.0.0.0/0[any] any -P out ipsec esp/tunnel/208.83.20.171-208.83.20.170/require;' | setkey -c echo 'spdadd 0.0.0.0/0[any] 10.0.0.1/32[any] any -P in ipsec esp/tunnel/208.83.20.170-208.83.20.171/require;' | setkey -c Start a running ping from client and verify that traffic is arriving on server over the tunnel: root@client:~ # ifconfig lo0 inet alias 10.0.0.1/32 root@client:~ # ping -S 10.0.0.1 8.8.8.8 root@server:~ # ifconfig enc0 up && tcpdump -ni enc0 tcpdump: WARNING: enc0: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on enc0, link-type ENC (OpenBSD encapsulated IP), capture size 65535 bytes capability mode sandbox enabled 01:54:35.376875 (confidential): SPI 0x00007a6a: IP 208.83.20.171 > 208.83.20.170: IP 10.0.0.1 > 8.8.8.8: ICMP echo request, id 60677, seq 0, length 64 (ipip-proto-4) 01:54:36.434174 (confidential): SPI 0x00007a6a: IP 208.83.20.171 > 208.83.20.170: IP 10.0.0.1 > 8.8.8.8: ICMP echo request, id 60677, seq 1, length 64 (ipip-proto-4) 01:54:37.450706 (confidential): SPI 0x00007a6a: IP 208.83.20.171 > 208.83.20.170: IP 10.0.0.1 > 8.8.8.8: ICMP echo request, id 60677, seq 2, length 64 (ipip-proto-4) 01:54:38.472992 (confidential): SPI 0x00007a6a: IP 208.83.20.171 > 208.83.20.170: IP 10.0.0.1 > 8.8.8.8: ICMP echo request, id 60677, seq 3, length 64 (ipip-proto-4) Enable IP forwarding on server and verify that the tunneled traffic is being forwarded: root@server:~ # sysctl net.inet.ip.forwarding=1 net.inet.ip.forwarding: 0 -> 1 root@server:~ # tcpdump -n listening on vtnet0, link-type EN10MB (Ethernet), capture size 65535 bytes capability mode sandbox enabled 01:48:32.311286 IP 208.83.20.171 > 208.83.20.170: ESP(spi=0x00007a6a,seq=0x2a), length 104 01:48:32.311367 IP 10.0.0.1 > 8.8.8.8: ICMP echo request, id 59141, seq 41, length 64 01:48:33.332164 IP 208.83.20.171 > 208.83.20.170: ESP(spi=0x00007a6a,seq=0x2b), length 104 01:48:33.332246 IP 10.0.0.1 > 8.8.8.8: ICMP echo request, id 59141, seq 42, length 64 01:48:34.352446 IP 208.83.20.171 > 208.83.20.170: ESP(spi=0x00007a6a,seq=0x2c), length 104 01:48:34.352516 IP 10.0.0.1 > 8.8.8.8: ICMP echo request, id 59141, seq 43, length 64 Enable NAT via pf: root@server:~ # kldload pf root@server:~ # echo "nat on vtnet0 inet all -> vtnet0" | pfctl -ef - No ALTQ support in kernel ALTQ related functions disabled pf enabled No states are created for the tunneled traffic and NAT does not function. Similarly, it doesn't appear possible to construct a filter rule that applies to the tunneled traffic. The same happens with ipfw using kernel NAT. >Fix: >Release-Note: >Audit-Trail: >Unformatted: