Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 29 Feb 2012 17:06:01 -0800
From:      Gaurav Puri <puri.gaurav@gmail.com>
To:        freebsd-net@freebsd.org
Subject:   FreeBSD7.2 http proxy not reading data off socket.
Message-ID:  <CAGByH0YX4-gzgLMU9wohqMNFWJ6T%2ByBkaJa65oaTRvAYV3z=Sw@mail.gmail.com>

next in thread | raw e-mail | index | archive | help
We have a http proxy running on FreeBSD 7.2

The problem I am seeing is that the proxy accepts a connection from a
client, in this case  yy.94.74.40 and receives the GET request for
disney.com. The application does not get the data off the socket for
several seconds ~90 seconds when it actually responds back to the
client.

Netstat shows that the proxy which is listening on port 8080 has a
full TCP listen queue.

netstat -aLn | grep 8080
tcp4  1742/0/8192    127.0.0.1.8080

curl requests from the client result in delays of ~90 seconds.

Packet catpure with relevant information.

05:46:19.357420 IP yy.94.74.40.16523 > xx.94.74.43.8080: Flags [S],
seq 1120042155, win 65535, options [mss 1460,nop,wscale 3,sackOK,TS
val 40704817 ecr 0], length 0
05:46:19.357438 IP yy.94.74.43.8080 > xx.94.74.40.16523: Flags [S.],
seq 3793473898, ack 1120042156, win 65535, options [mss
1460,nop,wscale 3,sackOK,TS val 1031458393 ecr 40704817], length 0
05:46:19.357669 IP yy.94.74.40.16523 > xx.94.74.43.8080: Flags [.],
ack 1, win 16471, options [nop,nop,TS val 40704817 ecr 1031458393],
length 0
05:46:19.357684 IP yy.94.74.43.8080 > xx.94.74.40.16523: Flags [.],
ack 1, win 16471, options [nop,nop,TS val 1031458393 ecr 40704817],
length 0
05:46:19.357686 IP yy.94.74.40.16523 > xx.94.74.43.8080: Flags [P.],
seq 1:286, ack 1, win 16471, options [nop,nop,TS val 40704817 ecr
1031458393], length 285
.... GET http://www.disney.com HTTP/1.1
05:46:19.357692 IP yy.94.74.43.8080 > xx.94.74.40.16523: Flags [.],
ack 286, win 16435, options [nop,nop,TS val 1031458393 ecr 40704817],
length 0
05:47:54.509303 IP yy.94.74.43.8080 > xx.94.74.40.16523: Flags [P.],
seq 1:554, ack 286, win 16471, options [nop,nop,TS val 1031467800 ecr
40704817], length 553
... HTTP/1.1 301 Moved Permanently


netstat -an shows that the connection is established and then there is
a long delay before the proxy process picks it off the socket.

I also have ktrace output which shows kevent being called ~10 times a
second but the data is not picked up off the socket. So event though
kevent is getting called several times a second the proxy process does
not get the data for quite sometime after.

  6042 prox     1330523858.652689 CALL
kevent(0xa,0x91ab4c8,0x20,0x91adcc8,0x200,0x80ee858)
  6042 prox     1330523858.798931 CALL
kevent(0xa,0x91ab4c8,0x2e,0x91adcc8,0x200,0x80ee858)
  6042 prox     1330523858.894449 CALL
kevent(0xa,0x91ab4c8,0x20,0x91adcc8,0x200,0x80ee858)
  6042 prox     1330523858.964911 CALL
kevent(0xa,0x91ab4c8,0x14,0x91adcc8,0x200,0x80ee858)
  6042 prox     1330523859.051830 CALL
kevent(0xa,0x91ab4c8,0xf,0x91adcc8,0x200,0x80ee858)
  6042 prox     1330523859.093017 CALL
kevent(0xa,0x91ab4c8,0x26,0x91adcc8,0x200,0x80ee858)
  6042 prox     1330523859.137018 CALL
kevent(0xa,0x91ab4c8,0x24,0x91adcc8,0x200,0x80ee858)
  6042 prox     1330523859.138366 CALL
kevent(0xa,0x91ab4c8,0x19,0x91adcc8,0x200,0x80ee858)
  6042 prox     1330523859.332275 CALL
kevent(0xa,0x91ab4c8,0x17,0x91adcc8,0x200,0x80ee858)
  6042 prox     1330523859.576070 CALL
kevent(0xa,0x91ab4c8,0x1f,0x91adcc8,0x200,0x80ee858)
  6042 prox     1330523859.711850 CALL
kevent(0xa,0x91ab4c8,0x21,0x91adcc8,0x200,0x80ee858)
  6042 prox     1330523859.902660 CALL
kevent(0xa,0x91ab4c8,0x2a,0x91adcc8,0x200,0x80ee858)

Has anyone observed this behavior before and could suggest what else
to look at on the appliance?

-- 
~Gaurav



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAGByH0YX4-gzgLMU9wohqMNFWJ6T%2ByBkaJa65oaTRvAYV3z=Sw>