Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 1 Mar 2007 14:45:49 +0000 (GMT)
From:      Robert Watson <rwatson@FreeBSD.org>
To:        Andre Oppermann <andre@freebsd.org>
Cc:        freebsd-net@freebsd.org, gallatin@freebsd.org, freebsd-current@freebsd.org, kmacy@freebsd.org
Subject:   Re: Large TCP send socket buffer optimizations
Message-ID:  <20070301144321.O98688@fledge.watson.org>
In-Reply-To: <45E6E2E8.5060408@freebsd.org>
References:  <45E6E2E8.5060408@freebsd.org>

next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, 1 Mar 2007, Andre Oppermann wrote:

> With the TCP socket buffer autosizing and generally larger socket buffers 
> for high bandwidth and high delay connections tcp_output() has become 
> increasingly inefficient for sending segments.  For every segment sent it is 
> traversing the entire socket buffer mbuf chain until it finds the offset to 
> continue from.  Usually this is close to the end of the chain.  Once it got 
> past a few dozen mbufs it starts to bust the CPU caches and performance 
> starts to fall off.
>
> This patch solves the problem by maintaining an offset pointer in the socket 
> buffer to give tcp_output() the closest mbuf right away avoiding the 
> traversal from the beginning.
>
> With this patch we should be able to compete nicely for the Internet land 
> speed record again.
>
> The patch is here:
>
> http://people.freebsd.org/~andre/sockbuf_sndptr-20070301.diff
>
> Any testing, especially on 10Gig cards, and feedback appreciated.

I'm going to be on- and off-line from now through about the last week of March 
on travel to LinuxForum.dk, AsiaBSDCon, and UKUUG.  However, once I get back 
online from that in late March through the end of April, I will be spending 
quite a bit of time looking at high performance packet capture issues, so can 
certainly spend some time looking at this.  Per our earlier discussions and my 
previous exploration of this issue, I see precisely this sort of problem with 
TCP as socket buffer size grows, so should be set up to fairly easily test for 
an improvement once I have my feet on the ground.  If I'm lucky I may get a 
breather for a few days from about 15 March - 18 March and could take an early 
look at it then.

Thanks,

Robert N M Watson
Computer Laboratory
University of Cambridge



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20070301144321.O98688>