From owner-freebsd-net@FreeBSD.ORG Tue Oct 28 07:32:39 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 092EF74D for ; Tue, 28 Oct 2014 07:32:39 +0000 (UTC) Received: from mail-oi0-x236.google.com (mail-oi0-x236.google.com [IPv6:2607:f8b0:4003:c06::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A05B8F64 for ; Tue, 28 Oct 2014 07:32:38 +0000 (UTC) Received: by mail-oi0-f54.google.com with SMTP id v63so63095oia.27 for ; Tue, 28 Oct 2014 00:32:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=iOO25ZIobwtSeXzjC88VGDr1pH49Omya2Dv65jeZvp0=; b=Q46+jAKQA1Nx832aybhk4EsYy9BsvOPBL1jkk44nFUr4AGvwIBc94OUhXayWGOYq1Z 0RLB61CuIA5u09XxJdn68Gen3PGdcqkcfED/ZkqVoSt++QcHSpYrs1iHp39jgGuu4Rc9 29zn3ybinBCEJZ7WvIZa2jI+3RXtbTqi62XrNhXyib1NPECFowTF/OqPGF/k0uvZsLOm /g/PynVi7Bi8owi3/ZiiRZELBsynw9dJAaoDSR/KOJjNZfUGL1k1R32GSBZxPuJ/U7/c 9V8u4NoWr82ganurtyTmacWyRG45ygpCPOcpgPZt8iBkE21/PcDSRhN8078Qx9oUel/j HziQ== X-Received: by 10.202.68.2 with SMTP id r2mr1285505oia.49.1414481557767; Tue, 28 Oct 2014 00:32:37 -0700 (PDT) MIME-Version: 1.0 Received: by 10.60.93.40 with HTTP; Tue, 28 Oct 2014 00:32:17 -0700 (PDT) From: Raimundo Santos Date: Tue, 28 Oct 2014 05:32:17 -0200 Message-ID: Subject: ipfw fwd duplicating packets in 9.3-RELEASE To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Oct 2014 07:32:39 -0000 Hello list! I was testing the behaviour of fwd in ipfw from FreeBSd 9.3-RELEASE, latest updates, GENERIC kernel, in this setup: 1x FreeBSD 9.3 as router, with 3 network interfaces 5x OpenBSD 5.5 as network machines, each one connected to FreeBSD via one port. It is a virtual env. FreeBSD em0 (192.168.0.1) -> OpenBSD#1 em0 (192.168.0.2) FreeBSD em1 (192.168.1.1) -> OpenBSD#2 em0 (192.168.1.2) FreeBSD em2 (192.168.2.1) -> OpenBSD#3 em0 (192.168.2.2) FreeBSD em3 (192.168.3.1) -> OpenBSD#4 em0 (192.168.3.2) FreeBSD em4 (192.168.4.1) -> OpenBSD#5 em1 (192.168.4.2) ipfw rule: fwd 192.168.0.2 proto icmp src-ip 192.168.4.2 in recv em4 Then a ping 192.168.1.2 was issued from OpenBSD#5. Interestingly, this rule put a packet on em0 and em1 in FreeBSD. The packet successfully arrived at OpenBSD#1, where it was discarded, and at OpenBSD#2, where it got its reply. Only these combinations of interface direction do not duplicate the packet: out xmit fwd 192.168.0.2 proto icmp src-ip 192.168.4.2 out xmit em1 and fwd 192.168.0.2 proto icmp src-ip 192.168.4.2 out via em1 Even out recv em4 xmit em1 leads to packet duplication. I think that it is a bad thing for PBR. As I can see from these tests, I can not use all the options to do PBR. In my real needs I have to: 1. let web traffic flow to an cache appliance (from internal network to cache, from internet to cache) 2. do NAT for the internal network under three different links In theory, fwd + in kernel NAT + one_pass=0 could solve the problem. But I am hitting my head in the wall for almost three weeks on this, only now I have the time to test in a more clear way. First I blamed the NAT, after that the one_pass=0, and now, with these results, well... Someone has an explanation about it? Something related to https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=129036? For my real needs, I figured out that it works with <...before NAT aliasing...> fwd CACHE_IP proto tcp src-ip table(INT) dst-port 80 out recv INT_IFACE <...after NAT dealiasing...> fwd CACHE_IP proto tcp dst-ip table(INT) src-port 80 out recv EXT_IFACE But I am not confident that it will remains in good shape without knowing exactly why fwd behaves that way. Thank you in advance for your time, Raimundo Santos