From owner-freebsd-questions@FreeBSD.ORG Wed Feb 21 08:27:04 2007 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 427C316A53E for ; Wed, 21 Feb 2007 08:27:04 +0000 (UTC) (envelope-from admin@azuni.net) Received: from mail.azuni.net (ns0.azuni.net [217.25.25.3]) by mx1.freebsd.org (Postfix) with ESMTP id 7641113C47E for ; Wed, 21 Feb 2007 08:27:02 +0000 (UTC) (envelope-from admin@azuni.net) Received: (qmail 83913 invoked by uid 1004); 21 Feb 2007 08:27:01 -0000 Received: from admin@azuni.net by mail.azuni.net by uid 89 with qmail-scanner-1.20 (clamscan: 0.65. spamassassin: 2.63. Clear:RC:1(217.25.23.4):. Processed in 0.129857 secs); 21 Feb 2007 08:27:01 -0000 Received: from unknown (HELO ?217.25.23.4?) (217.25.23.4) by ns0.azuni.net with AES256-SHA encrypted SMTP; 21 Feb 2007 08:27:01 -0000 Message-ID: <45DC024F.7040604@azuni.net> Date: Wed, 21 Feb 2007 12:26:55 +0400 From: admin Organization: UniNet User-Agent: Debian Thunderbird 1.0.2 (X11/20070113) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Ian Smith References: In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, Julian Elischer , freebsd-questions@freebsd.org Subject: Re: ipfw limit src-addr woes X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Feb 2007 08:27:04 -0000 Ian Smith wrote: > On Tue, 20 Feb 2007, Julian Elischer wrote: > > admin wrote: > > > > > Wrong: the implied "check-state" done by the "limit" lets the connection > > > through (i.e. performs the action) iff there's state recorded for it > > > (src-addr+src-port+dst-addr+dst-port). If however it's a SYN packet > > > incoming and the number of current states is trying to cross the limit, > > > the SYN packet is implicitly dropped and the search terminates. > > > > > > This is not to say that I completely understand the things going on when > > > the connections start building up (different timeouts?) but the above > > > conclusion is based on what simulation has shown. The whole ruleset fits > > > on one screen, there's an "allow ip from any to any" in the end, so I'm > > > pretty sure I'm not crazy :-) > > > > One thing to keep in mind is that a 'check-state' rule works by effectively > > jumping to the rule that did the 'keep-state' and re-executing it.. > > (and incrementing its stats). > > What if the action of the rule that does the 'keep-state' (here a limit > src-addr) is a skipto, rather than an allow / fwd / divert etc rule that > would terminate the search? Does 're-executing' here imply anything > about whether the skipto's conditional branch is or is not taken? > > I bought into this because admin said that more connections were being > allowed than the limit src-addr clause should allow, and I assumed that > the skipto branch was not being taken on over-limit packets, and that > the following fwd rule (allowing any type of packets including SYN) was > being executed, which would account for what he'd said was happening. > > admin above asserts that my assumption was wrong, and that in a match > beyond the limit number of connections for that src/dest address, the > setup packet is 'implicitly dropped and the search terminates', and > while I can't find that stated as such in ipfw(8), he may be right. > > Which still doesn't explain why connections from a particular IP beyond > his specified limit are allowed to be established, as originally stated. > I've changed "limit src-addr N" to "limit src-addr dst-addr N" - to key the limit on client parties trying to pose some single external IP address under attack (and, admittedly, the transparent proxy down with it :)) Some bit of scripting below allows me to monitor ipfw's dynamic rules usage keyed by src-addr+dst-addr, sorted by rule count (top 10 users): ipfw -d show | sed -n '/^## Dynamic rules /,$p' | tail -n+2 | awk '$5 == "LIMIT" { k=sprintf("%s %s", $7, $10); a[k]++ } END { for (i in a) {printf "%3d %s\n", a[i], i }}' | sort -nr -k1,1 | head Typical output: 32 client1.ip.ad.dr x.x.x.x 30 client1.ip.ad.dr y.y.y.y 20 client1.ip.ad.dr z.z.z.z 14 client2.ip.ad.dr e.f.g.h ... it shows that client1.ip.ad.dr has 32 connections to x.x.x.x, 30 connections to y.y.y.y, etc. And here's the thing: when under "attack", netstat -na | fgrep client1.ip.ad.dr shows a huge number of connections to some single x.x.x.x, port 80 in the ESTABLISHED state - 2, 3 times the value of limit - with full send queue usage (as determined by net.inet.tcp.sendspace) - all opening at a very fast pace!! The ipfw -d script, though, shows that the limit is not yet even crossed! Only when the netstat's connection count gets to 4-5 times the limit does ipfw start to realize something's wrong and drop packets - the usage count shown by the script above is then strictly that of "limit src-addr N"! It _must_ some different timeouts, I don't know _which_ timeouts though. True, some buggy web-browsers of many "good" clients leave the connection in the FIN_WAIT_2 state until timed out - and this still counts as open (eats up a socket etc.) - that's why I was forced to change "limit src-addr N" to "limit src-addr dst-addr N" to better get the idea. > [shrug] Over to the ipfw gurus. > Ditto. > Cheers, Ian > >