From owner-freebsd-pf@FreeBSD.ORG Mon Oct 15 01:55:19 2007 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 092A216A41B for ; Mon, 15 Oct 2007 01:55:19 +0000 (UTC) (envelope-from m@obmail.net) Received: from unclebob.obfuscated.net (stewie.obfuscated.net [69.8.202.125]) by mx1.freebsd.org (Postfix) with ESMTP id A5E0513C44B for ; Mon, 15 Oct 2007 01:55:17 +0000 (UTC) (envelope-from m@obmail.net) Received: from [10.0.1.196] (pool-96-228-136-165.tampfl.fios.verizon.net [96.228.136.165]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by unclebob.obfuscated.net (Postfix) with ESMTP id 8C7A817085; Sun, 14 Oct 2007 21:55:16 -0400 (EDT) In-Reply-To: <33acb3db0710141624g3647ddaasf720b78c3df4a208@mail.gmail.com> References: <33acb3db0710141624g3647ddaasf720b78c3df4a208@mail.gmail.com> Mime-Version: 1.0 (Apple Message framework v752.3) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <8B91857B-4898-41DF-ABFC-AEA53F375CF3@obmail.net> Content-Transfer-Encoding: 7bit From: Michael Conlen Date: Sun, 14 Oct 2007 17:55:09 -0400 To: "Matthew Franz" X-Mailer: Apple Mail (2.752.3) Cc: freebsd-pf@freebsd.org Subject: Re: PF in FreeBSD 5.3 versus 6.x X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Oct 2007 01:55:19 -0000 I am not using pfsync. I'm using a pair of foundry layer 7 switches to do firewall load balancing. I've since set optimization to aggressive and have seen a reduction in packet loss. One issue I've discovered is that mytraceroute 0.72 appears to be buggy with respect to statistics so I can't trust the results for standard deviation and mean response time. In particular the mean response time tends towards the minimum response time over time despite continuously higher numbers. Without an accurate mean there's no good way to get a idea of the distribution using mytraceroute, and I didn't use ping times before I made the switch. On the other hand my NTP server getting time from across the firewalls does show improvement in stability and jitter, and this tends to be the first application that shows network problems for me. The NTP server is tracking time to wtihin +300/-200 microseconds which is impossible with a unstable network. With the change the state table is running around 20k entries. Do you know if these issues are present in the betas of 7.0, which I understand is using pf 4.1? -- Michael Conlen On Oct 14, 2007, at 7:24 PM, Matthew Franz wrote: > HI Michael, > > You don't say whether you are running pfsync because Bill Marquette > (who I work with) and Max have been discussing a pretty nasty pfsync > bug (on 6.2) on this list under high loads (probably starting where > you are at in terms of pps throughput but going up to 70-90kpps) where > the backup is unable to clear states and there is eventually a huge > discrepancy between the master and the backup. > > If you are seeing this with a single box. Its on my list to try to > reproduce this in the lab (and test some of the patches Max has > developed) with smartbits but I still haven't had time. We are > definitely seeing some PF losing state entries, but sort of assumed > this was a pfsync issue (or an effect thereof) but if you are seeing > this without pfsync, that would point to so more fundamental problems > with PF under high load. I can also share so more specific stats > offline if that would be helpful. > > - mdf > > > > > On 10/9/07, Michael Conlen wrote: >> I've noticed at some point between 5.3 and 6.0 that PF seems to be >> dropping more packets than with 5.3 and there is increased deviation >> in latency. Using the same equipment handling about 25k PPS each way >> I see about 0.3% packet loss with FreeBSD 6.2 and 6.0 with sub 0.1% >> loss with FreeBSD 5.3. Similarly the worst case response times for >> ICMP packets is much less in 5.3 than in either version of 6. >> >> I'm using something pretty vanilla in terms of setup. No ALTQ support >> or features, no redirects, just a lot of blocking and allowing. The >> firewalls are using server class 3Com and Intel Gigabit (Fiber) >> cards. The changes were noticed going forward and undone by going >> back to FreeBSD 5.3 so I don't suspect physical problems at the >> moment. >> >> My pf.conf is essentially a block in all followed by a block in quick >> against a table with 2000 entries, many of the /24 or /16 followed by >> pass rules to the various host:ports we allow. >> >> If I login to the firewalls themselves and run mtr in each direction >> I don't see any traffic loss. It's only when crossing the firewalls. >> >> Usage is about 25k packets per second and 100Mbit/sec 5 minute max >> traffic. The switches are Foundry SI-800g. >> >> Also doing about 25k/sec searches with 400 inserts a second and 270 >> removals and 407 matches/sec. The state table seems to run about >> 70,000 to 90,000 >> >> Are there issues I should be aware of and should pf be able to handle >> this kind of load? >> >> -- >> Michael Conlen >> >> _______________________________________________ >> freebsd-pf@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-pf >> To unsubscribe, send any mail to "freebsd-pf-unsubscribe@freebsd.org" >> > > > -- > Matthew Franz > http://www.threatmind.net/