Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 24 Sep 2009 08:35:41 -0400
From:      "B. Cook" <bcook@poughkeepsieschools.org>
To:        freebsd-questions@freebsd.org
Subject:   net.inet.ip.random_id possible ASA problems?
Message-ID:  <4ABB679D.7030604@poughkeepsieschools.org>

Next in thread | Raw E-Mail | Index | Archive | Help
Morning,

I am running several FreeBSD 7.x servers in an setting where we recently 
went from controlling the border firewall with PFSense; We were mandated 
to replace it with an outside provider which has an ASA in place.

And we are having TONS of issues..

http://lists.freebsd.org/pipermail/freebsd-net/2009-July/022532.html

This seems to be the only thing I can find that might possibly be what 
he is seeing.  The operator of the upstream ASA does not give us access 
to see the config or output of the debug(s) he runs.. so we are totally 
blind to help ourselves..

We have a 30Mbit feed from them, and find that people downloading 3.5Mb 
pdf files from our lighttpd webserver takes about 2-3 minutes.  Running 
anywhere from 600 down to 2..

Below is his email to us regarding his cisco case:

...
I spent close to 2 hours this afternoon with two Cisco engineers 
troubleshooting the download issue while on the ASA.

The root cause of the problem is that the ASA is being fed a significant 
stream of out-of-order TCP packets when the file download is launched 
from the PokCSD Web Server.   With HTTP inspection enabled on the ASA, 
the ASA is required to process the HTTP stream in order, so it buffers 
out-of-order packets until it can create a proper order for processing. 
   In this case, the volume is so high the buffers fill, and the ASA is 
forced to begin dropping packets from the conversation.  (Oddly, the 
faster the connection the faster the buffers overload which appears to 
be why slower WAN connections appeared to have more consistent 
throughput since they minimized or avoided the buffer drops)

While some out-of-order TCP packets are normal, this volume was deemed 
excessive.  Even setting the buffer sizes to their maximum configurable 
limit on the ASA would not contain the volume of out of order packets 
and prevent drops from occurring.

We did some additional troubleshooting by enabling HTTP inspection and 
some captures on the PokCSD-ASA, which was not enabled by default.  Once 
enabled, we re-ran the download test  and the PokCSD-ASA  began dropping 
the out-of-order TCP packets.  So that leads us to conclude that the 
source of the out-of-order TCP stream was downstream of the PokCSD-ASA 
not some issue further upstream towards BOCES.  We returned the HTTP 
inspection on the PokCSD ASA to its original disabled state at the end 
of testing.

Given the volume of out-of-order TCP packets being sent, it is likely 
that there is a network or server issue somewhere inside the PokCSD 
network.  But all of that is outside of our purview to access or 
troubleshoot.

Beyond looking for a duplex miss-match someplace in the LAN gear I have 
no obvious initial guess for you but I donít think there is much between 
that ASA and the server as I recall.

I does make me wonder if whatever this is isnít in some way tied into 
your other issues with direct e-mail transfer.

So at this point I have closed the case with Cisco since HTTP inspection 
is identifying the issue and is not the cause.

Some things have changed related to web filtering such that the original 
purpose of the inspection may or may not be needed at this point.  I 
will investigate that further tomorrow.  Even if we decide we can turn 
that inspection off it would seem we are just avoiding the real problem 
if we donít root out the out-of-order issue so either way I would 
encourage troubleshooting that to conclusion.

That is it for now,. you are up to date.

Let me know if there is something else I can do.

"

So after 6 hours of cisco techs.. all they could come up with is a "... 
possible duplex mis-match.. "

*sigh*

So dropping my pf rules (which contain scrub settings) made no 
difference, I found the above URL which seeme to point to 
net.inet.ip.random_id.

I can not find any 'freebsd.org' documentation pertaining to it 
regarding what it actually does.  I do however find it scattered amongst 
tons of 'FreeBSD hardening' docs..

Can anyone shed some light on what this does?

Thanks in advance.



Want to link to this message? Use this URL: <http://docs.FreeBSD.org/cgi/mid.cgi?4ABB679D.7030604>