From owner-freebsd-stable Tue Nov 19 8:13:34 2002 Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0C50737B401 for ; Tue, 19 Nov 2002 08:13:31 -0800 (PST) Received: from grumpy.dyndns.org (user-24-214-34-52.knology.net [24.214.34.52]) by mx1.FreeBSD.org (Postfix) with ESMTP id 19FF243E75 for ; Tue, 19 Nov 2002 08:13:30 -0800 (PST) (envelope-from dkelly@grumpy.dyndns.org) Received: from grumpy.dyndns.org (localhost [127.0.0.1]) by grumpy.dyndns.org (8.12.6/8.12.6) with ESMTP id gAJGBbgx064148; Tue, 19 Nov 2002 10:11:37 -0600 (CST) (envelope-from dkelly@grumpy.dyndns.org) Received: (from dkelly@localhost) by grumpy.dyndns.org (8.12.6/8.12.6/Submit) id gAJGBYsZ064147; Tue, 19 Nov 2002 10:11:34 -0600 (CST) Date: Tue, 19 Nov 2002 10:11:29 -0600 From: David Kelly To: Guido van Rooij Cc: Scott Ullrich , "'Archie Cobbs'" , "'greg.panula@dolaninformation.com'" , FreeBSD-stable@FreeBSD.ORG Subject: IPsec packets seen on wrong interface by ipfw (was Re: IPsec/gif VPN tunnel packets on wrong NIC in ipfw?) Message-ID: <20021119161129.GA63943@grumpy.dyndns.org> References: <2F6DCE1EFAB3BC418B5C324F13934C9601D23C35@exchange.corp.cre8.com> <20021119110336.GA12956@gvr.gvr.org> <200211190754.29355.dkelly@HiWAAY.net> <20021119150826.GA42097@gvr.gvr.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20021119150826.GA42097@gvr.gvr.org> User-Agent: Mutt/1.4i Sender: owner-freebsd-stable@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG To help clarify gif is no longer suspect I have changed the subject. On Tue, Nov 19, 2002 at 04:08:26PM +0100, Guido van Rooij wrote: > On Tue, Nov 19, 2002 at 07:54:29AM -0600, David Kelly wrote: > > > > The problem is that while ESP packets arrive to be processed by > > IPsec just fine thru my ipfw rules, when the packets are de-encrypted > > and re-inserted into the kernel they appear to ipfw to be coming from > > my external interface (the one they arrived on via ESP). tcpdump can't > > find them (decrypted) on the external interface. > > THe reason tcpdump cannot find them on the external interface is because > they are coming out of your gif interface. In case ipfw thinks they are coming > from your physical external interface, ipfw has a bug that needs to be fixed. Yes, the purpose of my tcpdump expermiment was to document the packets were not really on fxp1. Insofar as I can trust tcpdump sniffs packets at a different tap than ipfw. But how can they be coming out of my gif interface when I no longer have a gif? Is gif still operating behind the scense? I would be *happy* if they were coming thru gif0. This discussion started with mention of gif because in using snippets of information I could find hither and yonder to create a VPN nothing was clear (still isn't really) and some were using gif and some might not have been using gif so I did a little of everything. Since this incident I have learned my gif efforts were for naught and have been removed. The same new situation persists. IPFW sees IPsec de-tunneled packets on the wrong interface (hence the clarified subject of this email thread). When gif was enabled the packets *never* came out of my gif interface. Not that ipfw logging could detect. > > 01100 0 0 deny ip from any to 192.0.2.0/24 via fxp1 > > 01200 0 0 deny ip from any to 224.0.0.0/4 via fxp1 > > 01300 13738 4506273 deny ip from any to 240.0.0.0/4 via fxp1 > > 01400 787470 364186135 divert 8668 ip from any to any via fxp1 > > [...] > > I dont see any rules for your gif interface? I don't have a gif anymore but the rule is still there. It never got any hits. Machine has been rebooted between above and below examples: [...] 05800 13 2329 allow udp from me 500 to 24.yy.yy.yy 500 via fxp1 05900 6 1845 allow udp from 24.yy.yy.yy 500 to me 500 via fxp1 06000 0 0 allow ip from any to any via gif0 I have never seen a hit on the above rule 06000. 06100 30 2914 allow ip from 192.168.100.0/24 to 10.0.0.0/24 via fxp0 06200 115 10016 allow ip from 10.0.0.0/24 to 192.168.100.0/24 via fxp0 06300 132 19072 allow esp from me to 24.yy.yy.yy out xmit fxp1 06400 132 18736 allow esp from 24.yy.yy.yy to me in recv fxp1 [...] The remote end (running an older FreeBSD than my local end) does have gif configured but also has racoon and setkeys dealing with IPsec same as the end now without gif, has been up 4 days, ipfw accounting was cleared about an hour after reboot: 05200 4361 1113812 allow udp from me 500 to REMOTE 500 via xl0 05300 5700 1168049 allow udp from 24.xx.xx.xx 500 to me 500 via xl0 05400 0 0 allow ip from any to any via gif0 05500 13539 1307801 allow ip from 10.0.0.0/24 to 192.168.100.0/24 via xl1 05600 14486 1408517 allow ip from 192.168.100.0/24 to 10.0.0.0/24 via xl1 05700 32215 5258080 allow esp from me to 24.xx.xx.xx out xmit xl0 05800 31552 6213544 allow esp from 24.xx.xx.xx to me in recv xl0 Once again I don't have a gif0 (any longer): % ifconfig fxp0: flags=8843 mtu 1500 inet 10.0.0.5 netmask 0xffffff00 broadcast 10.0.0.255 ether 00:d0:b7:1c:cc:66 media: Ethernet autoselect (100baseTX ) status: active fxp1: flags=8843 mtu 1500 inet 24.xx.xx.xx netmask 0xffffff00 broadcast 24.XX.XX.255 ether 00:90:27:87:a6:ad media: Ethernet autoselect (100baseTX ) status: active lp0: flags=8810 mtu 1500 lo0: flags=8049 mtu 16384 inet 127.0.0.1 netmask 0xff000000 faith0: flags=8002 mtu 1500 % Your commit of 2002/10/16 seems to be the point where things broke: : revision 1.214 : date: 2002/10/16 09:01:48; author: guido; state: Exp; lines: +0 -5 : Get rid of checking for ip sec history. It is true that packets are not : supposed to be checked by the firewall rules twice. However, because the : various ipsec handlers never call ip_input(), this never happens anyway. Once the ipsec history is removed from the packet then how/what/where is the packet tagged as having come from? In my case it appears to have retained properties of the ESP packet it was encased within. Don't really know as I don't have multiple interfaces with ESP packets. The system is in production so I can't casually swap interfaces to verify. At some point since early October when this system was previously updated these IPsec packets started appearing on the wrong interface in ipfw. Currently only one end of my link is updated, the other end is running with the same configuration it has used for the past 9 months. With configured (but apparently unused) gif and everything. -- David Kelly N4HHE, dkelly@hiwaay.net ===================================================================== The human mind ordinarily operates at only ten percent of its capacity -- the rest is overhead for the operating system. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message