From owner-freebsd-hackers  Fri Oct  8 13:51:55 1999
Delivered-To: freebsd-hackers@freebsd.org
Received: from cs.rice.edu (cs.rice.edu [128.42.1.30])
	by hub.freebsd.org (Postfix) with ESMTP
	id 79FC514D29; Fri,  8 Oct 1999 13:51:34 -0700 (PDT)
	(envelope-from aron@cs.rice.edu)
Received: (from aron@localhost)
	by cs.rice.edu (8.9.0/8.9.0) id PAA25028;
	Fri, 8 Oct 1999 15:51:29 -0500 (CDT)
From: Mohit Aron <aron@cs.rice.edu>
Message-Id: <199910082051.PAA25028@cs.rice.edu>
Subject: sbappend() is not scalable
To: freebsd-hackers@freebsd.org, freebsd-net@freebsd.org
Date: Fri, 8 Oct 1999 15:51:29 -0500 (CDT)
Cc: tech-net@netbsd.org
X-Mailer: ELM [version 2.4 PL25]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Sender: owner-freebsd-hackers@FreeBSD.ORG
Precedence: bulk
X-Loop: FreeBSD.ORG

Hi,
	I recently did some experiments with TCP over a high b/w-delay path
and found a scalability problem in sbappend(). The experimental setup
consisted of a 100Mbps network with a round-trip delay of 100ms. Under this
situation, FreeBSD's TCP version is incapable of attaining more than 65 Mbps
on a 300MHz Pentium II - even without slow-start.

I tracked down the problem to sbappend() - the routine that appends user data
into the socket buffers for network transmission. Every time a TCP ACK 
acknowledges some data, space is created in the socket buffer that permits
more data to be appended. Unfortunately, the implementation does not maintain
a pointer to the end of the list of mbufs in the socket buffer. Thus each 
time any data is added, the whole list of mbufs is traversed to reach the 
very end where the data is added. Since the b/w-delay product is large, there
can be about 600 mbufs in the socket buffer waiting to be acknowledged. Thus
upon every ACK, about 600 mbufs are traversed causing the TCP sender to run 
out of CPU.

The problem is not limited only to high b/w networks - it is also present in
long latency paths (satellite links). Thus a server transferring a large file
over a satellite link can spend lot of CPU due to the above problem.

Hope the problem shall be fixed in future releases,


- Mohit


To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message