From owner-freebsd-pf@FreeBSD.ORG Wed May 21 13:19:18 2008 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CF5E81065670 for ; Wed, 21 May 2008 13:19:18 +0000 (UTC) (envelope-from cbredi@bofhserver.net) Received: from yw-out-2324.google.com (yw-out-2324.google.com [74.125.46.31]) by mx1.freebsd.org (Postfix) with ESMTP id 864AF8FC13 for ; Wed, 21 May 2008 13:19:18 +0000 (UTC) (envelope-from cbredi@bofhserver.net) Received: by yw-out-2324.google.com with SMTP id 9so1566510ywe.13 for ; Wed, 21 May 2008 06:19:12 -0700 (PDT) Received: by 10.150.52.2 with SMTP id z2mr210492ybz.48.1211375952512; Wed, 21 May 2008 06:19:12 -0700 (PDT) Received: by 10.150.206.14 with HTTP; Wed, 21 May 2008 06:19:12 -0700 (PDT) Message-ID: <2f12f40a0805210619t4aae9fa0w43737b2098f7d042@mail.gmail.com> Date: Wed, 21 May 2008 16:19:12 +0300 From: "Cristian Bradiceanu" To: "Vlad GALU" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <2f12f40a0805200830l7836d640s69c55af837d475d9@mail.gmail.com> <20080520162029.GA41273@eos.sc1.parodius.com> <2f12f40a0805201349g6ee6be5cxa6f2a029b5150bec@mail.gmail.com> Cc: freebsd-pf@freebsd.org Subject: Re: pf reply-to tcp connections stall X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2008 13:19:18 -0000 On Wed, May 21, 2008 at 1:27 AM, Vlad GALU wrote: > On 5/20/08, Cristian Bradiceanu wrote: >> On Tue, May 20, 2008 at 7:20 PM, Jeremy Chadwick wrote: >> > On Tue, May 20, 2008 at 06:30:58PM +0300, Cristian Bradiceanu wrote: >> >> I am trying to set up split routing on two Internet links, each with >> >> one IP address: >> >> >> >> em0 = wan1, $em0_gw gateway >> >> em1 = lan, NATed on em0 and em2 >> >> em2 = wan2, default gateway >> >> >> >> pass in on em0 reply-to (em0 $em0_gw) inet proto tcp from any to em0 flags S/SA keep state >> >> pass in on em0 reply-to (em0 $em0_gw) inet proto udp from any to em0 keep state >> >> pass in on em0 reply-to (em0 $em0_gw) inet proto icmp from any to em0 keep state >> >> >> >> wan2 connections are working correct, no pf rules for policy routing >> >> >> >> wan1 tcp connections to IP of em0 (e.g. ssh) stall when a large amount >> >> of data is sent (e.g. running dmesg or cat file). States are created >> >> correctly. When ssh stalls there are some icmp packets out on lo0 with >> >> source and destination ip address of em0, which I believe is not >> >> correct (set skip on lo0 does not help). Also tried with tcp ... >> >> modulate state but same result. >> > >> > modulate state is known to be broken: >> > >> > http://wiki.freebsd.org/JeremyChadwick/Commonly_reported_issues >> > >> > Regarding the "when large amounts of data is sent, the connection >> > breaks" issue: >> > >> > I've reproduced this a few times on our systems (using the exact same >> > method you do: dmesg, cat'ing large files, or scp'ing -- anything using >> > large TCP packets), and it's always been caused by improper pf(4) rules >> > where state was broken. In every case, the "state mismatch" counter >> > shown in pfctl -s info would increase. >> >> >> state-mismatch counter does not increase, all "Counters" are 0 except >> match (pfctl -si). When large amounts of data is sent the connection >> stalls and continues from time to time very slow; when it continues >> there are logged icmp packets out on lo0 from (em0) to (em0) which >> looks pretty weird to me. >> >> >> Cristian > > This may be a PMTUD issue. Make sure your ICMP packets can travel > back and forth unhindered and that there are no scrub rules that may > clear out the DF flag on them. There's no no-df scrub flag, also no icmp filters. Cristian