Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 21 Feb 2007 12:26:55 +0400
From:      admin <admin@azuni.net>
To:        Ian Smith <smithi@nimnet.asn.au>
Cc:        freebsd-net@freebsd.org, Julian Elischer <julian@elischer.org>, freebsd-questions@freebsd.org
Subject:   Re: ipfw limit src-addr woes
Message-ID:  <45DC024F.7040604@azuni.net>
In-Reply-To: <Pine.BSF.3.96.1070221180206.7164B-100000@gaia.nimnet.asn.au>
References:  <Pine.BSF.3.96.1070221180206.7164B-100000@gaia.nimnet.asn.au>

Next in thread | Previous in thread | Raw E-Mail | Index | Archive | Help
Ian Smith wrote:
> On Tue, 20 Feb 2007, Julian Elischer wrote:
>  > admin wrote:
>  > 
>  > > Wrong: the implied "check-state" done by the "limit" lets the connection 
>  > > through (i.e. performs the action) iff there's state recorded for it 
>  > > (src-addr+src-port+dst-addr+dst-port). If however it's a SYN packet 
>  > > incoming and the number of current states is trying to cross the limit, 
>  > > the SYN packet is implicitly dropped and the search terminates.
>  > > 
>  > > This is not to say that I completely understand the things going on when 
>  > > the connections start building up (different timeouts?) but the above 
>  > > conclusion is based on what simulation has shown. The whole ruleset fits 
>  > > on one screen, there's an "allow ip from any to any" in the end, so I'm 
>  > > pretty sure I'm not crazy :-)
>  > 
>  > One thing to keep in mind is that a 'check-state' rule works by effectively 
>  > jumping to the rule that did the 'keep-state' and re-executing it..
>  > (and incrementing its stats).
> 
> What if the action of the rule that does the 'keep-state' (here a limit
> src-addr) is a skipto, rather than an allow / fwd / divert etc rule that
> would terminate the search?  Does 're-executing' here imply anything
> about whether the skipto's conditional branch is or is not taken?
> 
> I bought into this because admin said that more connections were being
> allowed than the limit src-addr clause should allow, and I assumed that
> the skipto branch was not being taken on over-limit packets, and that
> the following fwd rule (allowing any type of packets including SYN) was
> being executed, which would account for what he'd said was happening.
> 
> admin above asserts that my assumption was wrong, and that in a match
> beyond the limit number of connections for that src/dest address, the
> setup packet is 'implicitly dropped and the search terminates', and
> while I can't find that stated as such in ipfw(8), he may be right.
> 
> Which still doesn't explain why connections from a particular IP beyond
> his specified limit are allowed to be established, as originally stated.
> 

I've changed "limit src-addr N" to "limit src-addr dst-addr N" - to key 
the limit on client parties trying to pose some single external IP 
address under attack (and, admittedly, the transparent proxy down with 
it :))

Some bit of scripting below allows me to monitor ipfw's dynamic rules 
usage keyed by src-addr+dst-addr, sorted by rule count (top 10 users):

ipfw -d show | sed -n '/^## Dynamic rules /,$p' | tail -n+2 | awk '$5 == 
"LIMIT" { k=sprintf("%s %s", $7, $10); a[k]++ } END { for (i in a) 
{printf "%3d %s\n", a[i], i }}' | sort -nr -k1,1 | head

Typical output:
  32 client1.ip.ad.dr x.x.x.x
  30 client1.ip.ad.dr y.y.y.y
  20 client1.ip.ad.dr z.z.z.z
  14 client2.ip.ad.dr e.f.g.h
...
it shows that client1.ip.ad.dr has 32 connections to x.x.x.x, 30 
connections to y.y.y.y, etc.

And here's the thing: when under "attack", netstat -na | fgrep 
client1.ip.ad.dr shows a huge number of connections to some single 
x.x.x.x, port 80 in the ESTABLISHED state - 2, 3 times the value of 
limit - with full send queue usage (as determined by 
net.inet.tcp.sendspace) - all opening at a very fast pace!! The ipfw -d 
script, though, shows that the limit is not yet even crossed! Only when 
the netstat's connection count gets to 4-5 times the limit does ipfw 
start to realize something's wrong and drop packets - the usage count 
shown by the script above is then strictly that of "limit src-addr N"! 
It _must_ some different timeouts, I don't know _which_ timeouts though.

True, some buggy web-browsers of many "good" clients leave the 
connection in the FIN_WAIT_2 state until timed out - and this still 
counts as open (eats up a socket etc.) - that's why I was forced to 
change "limit src-addr N" to "limit src-addr dst-addr N" to better get 
the idea.

> [shrug]  Over to the ipfw gurus.
> 

Ditto.

> Cheers, Ian
> 
> 




Want to link to this message? Use this URL: <http://docs.FreeBSD.org/cgi/mid.cgi?45DC024F.7040604>