From owner-freebsd-pf@FreeBSD.ORG Tue May 20 16:20:29 2008 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DB9F9106568B for ; Tue, 20 May 2008 16:20:29 +0000 (UTC) (envelope-from jdc@parodius.com) Received: from mx01.sc1.parodius.com (mx01.sc1.parodius.com [72.20.106.3]) by mx1.freebsd.org (Postfix) with ESMTP id 506598FC0C for ; Tue, 20 May 2008 16:20:29 +0000 (UTC) (envelope-from jdc@parodius.com) Received: by mx01.sc1.parodius.com (Postfix, from userid 1000) id 3E70D1CC05B; Tue, 20 May 2008 09:20:29 -0700 (PDT) Date: Tue, 20 May 2008 09:20:29 -0700 From: Jeremy Chadwick To: Cristian Bradiceanu Message-ID: <20080520162029.GA41273@eos.sc1.parodius.com> References: <2f12f40a0805200830l7836d640s69c55af837d475d9@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2f12f40a0805200830l7836d640s69c55af837d475d9@mail.gmail.com> User-Agent: Mutt/1.5.17 (2007-11-01) Cc: freebsd-pf@freebsd.org Subject: Re: pf reply-to tcp connections stall X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2008 16:20:30 -0000 On Tue, May 20, 2008 at 06:30:58PM +0300, Cristian Bradiceanu wrote: > I am trying to set up split routing on two Internet links, each with > one IP address: > > em0 = wan1, $em0_gw gateway > em1 = lan, NATed on em0 and em2 > em2 = wan2, default gateway > > pass in on em0 reply-to (em0 $em0_gw) inet proto tcp from any to em0 flags S/SA keep state > pass in on em0 reply-to (em0 $em0_gw) inet proto udp from any to em0 keep state > pass in on em0 reply-to (em0 $em0_gw) inet proto icmp from any to em0 keep state > > wan2 connections are working correct, no pf rules for policy routing > > wan1 tcp connections to IP of em0 (e.g. ssh) stall when a large amount > of data is sent (e.g. running dmesg or cat file). States are created > correctly. When ssh stalls there are some icmp packets out on lo0 with > source and destination ip address of em0, which I believe is not > correct (set skip on lo0 does not help). Also tried with tcp ... > modulate state but same result. modulate state is known to be broken: http://wiki.freebsd.org/JeremyChadwick/Commonly_reported_issues Regarding the "when large amounts of data is sent, the connection breaks" issue: I've reproduced this a few times on our systems (using the exact same method you do: dmesg, cat'ing large files, or scp'ing -- anything using large TCP packets), and it's always been caused by improper pf(4) rules where state was broken. In every case, the "state mismatch" counter shown in pfctl -s info would increase. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |