From owner-freebsd-bugs Wed Sep 6 9:10: 9 2000 Delivered-To: freebsd-bugs@freebsd.org Received: from freefall.freebsd.org (freefall.FreeBSD.org [216.136.204.21]) by hub.freebsd.org (Postfix) with ESMTP id 8BBA537B43F for ; Wed, 6 Sep 2000 09:10:04 -0700 (PDT) Received: (from gnats@localhost) by freefall.freebsd.org (8.9.3/8.9.2) id JAA06146; Wed, 6 Sep 2000 09:10:04 -0700 (PDT) (envelope-from gnats@FreeBSD.org) Received: by hub.freebsd.org (Postfix, from userid 32767) id E841B37B42C; Wed, 6 Sep 2000 09:00:51 -0700 (PDT) Message-Id: <20000906160051.E841B37B42C@hub.freebsd.org> Date: Wed, 6 Sep 2000 09:00:51 -0700 (PDT) From: B.Candler@pobox.com To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-1.0 Subject: kern/21079: IPSEC, kernel ARPs for tunnel endpoint instead of next-hop gateway Sender: owner-freebsd-bugs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >Number: 21079 >Category: kern >Synopsis: IPSEC, kernel ARPs for tunnel endpoint instead of next-hop gateway >Confidential: no >Severity: critical >Priority: medium >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Wed Sep 06 09:10:04 PDT 2000 >Closed-Date: >Last-Modified: >Originator: Brian Candler >Release: 4.1-RELEASE >Organization: >Environment: FreeBSD godl-vpn.example.com 4.1-RELEASE FreeBSD 4.1-RELEASE #0: Sun Sep 3 14:00:33 BST 2000 root@example.com:/usr/src/sys/compile/GODL-VPN i386 >Description: When sending IPSEC packets, and the ARP cache for the next hop expires, the machine tries to ARP for the tunnel endpoint address instead of the next hop router. Thus the symptom is that connectivity drops after a few minutes. e.g. +- - - - - - - - + R1 R2 | | A B -+-+--- -+-+--- W1 W2 Box A is a FreeBSD-4.1 PC configured as IPSEC VPN gateway. W1, W2 are workstations. A points defaultroute at R1. This works for a couple of minutes, until A's ARP entry for R1 expires. At that point, A sends out ARP packets for B's IP address, not R1's IP address! The kernel logs the following message: Sep 4 10:33:01 godl-vpn /kernel: arplookup b.b.b.b failed: host is not on local network (where b.b.b.b is B's IP address, i.e. the remote tunnel endpoint) arp -an shows: ? (b.b.b.b) at (incomplete) [ethernet] Connectivity is lost until you manually do # arp -d b.b.b.b # ping b.b.b.b At this point the IPSEC packets start to flow, until the ARP cache expires again. >How-To-Repeat: You can do this with just one PC running FreeBSD, as it doesn't matter that the remote end does not exist. (1) Create /etc/ipsec.conf [Replace 192.168.1.180 with your PC's ethernet address, but leave all the other numbers as they are here] flush; add 192.168.1.180 192.0.2.1 esp 256 -E des-cbc 0x1111111111111111 -A hmac-md5 0x22222222222222222222222222222222; add 192.0.2.1 192.168.1.180 esp 256 -E des-cbc 0x1111111111111111 -A hmac-md5 0x22222222222222222222222222222222; spdflush; spdadd 10.0.0.0/24[any] 10.0.1.0/24[any] any -P out ipsec esp/tunnel/192.168.1.180-192.0.2.1/require; spdadd 10.0.1.0/24[any] 10.0.0.0/24[any] any -P in ipsec esp/tunnel/192.0.2.1-192.168.1.180/require; (2) ifconfig lo0 10.0.0.1 netmask 255.255.255.0 alias (3) setkey -f /etc/ipsec.conf (4) ping -S 10.0.0.1 10.0.1.1 Make sure there is no other IP traffic being generated by this PC (i.e. no ntpd etc) (5) On another VC, run a tcpdump. You should see 16:49:18.950061 192.168.1.180 > 192.0.2.1: ESP(spi=256,seq=0x1b) 16:49:19.960064 192.168.1.180 > 192.0.2.1: ESP(spi=256,seq=0x1c) 16:49:20.970104 192.168.1.180 > 192.0.2.1: ESP(spi=256,seq=0x1d) 16:49:21.980124 192.168.1.180 > 192.0.2.1: ESP(spi=256,seq=0x1e) (except with 192.168.1.180 changed to your local ethernet address) (6) On a third VC, type "arp -d 192.168.1.254" (but use your PC's gateway address instead of 192.168.1.254) Go back to the second VC and you will see: 16:49:22.990120 arp who-has 192.0.2.1 tell 192.168.1.180 ^^^^^^^^^ i.e. it is ARPing for the tunnel endpoint, not the gateway. If you have a Cisco on your network in its default mode (gratiously proxy ARP) then the Cisco will respond, but the kernel will ignore it.  (7) arp -n 192.168.1.254 ? (192.168.1.254) at (incomplete) [ethernet] (8) Stop the ping -S process and do arp -d 192.168.1.254 ping 192.168.1.254 Then restart the ping -S; the packets will be flowing again. >Fix: Workaround: add static ARP entry for the gateway, so that it never expires. arp -S 192.168.1.254 gg:gg:gg:gg:gg:gg >Release-Note: >Audit-Trail: >Unformatted: To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-bugs" in the body of the message