Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 20 Jun 2005 22:11:27 +0200
From:      Pieter de Boer <pieter@thedarkside.nl>
To:        freebsd-net@freebsd.org
Subject:   Issues with a Large Fat pipe Network simulation
Message-ID:  <42B722EF.2090203@thedarkside.nl>

next in thread | raw e-mail | index | archive | help
Hello there,

For a project about TCP (performance) enhancements, we have been trying
to simulate a network with a high bandwidth*delay product. Although we
haven't started our real tests just yet, we already stumbled upon some
issues :). For one (advertising an invalid window scale in some
situations), we'll file a PR soon.


We have three systems: 'client', 'network'
and 'server'. All three systems have two intel gigabit NICs (em) in
them. They run 5.4-RELEASE using the SMP-kernel. 'network' has HZ bumped
to 2000 and nmbclusters to 128*1024. The setup is as follows:

'client' <-----> 'network' <-----> 'server'
100.2	       100.1   200.1       200.2

'network' routes traffic between 192.168.100.0/24 and 192.168.200.0/24
and is equipped with ipfw/dummynet for simulation purposes.

We had the following ipfw pipes on 'network':
	pipe 1 ip from client to server via em0
	pipe 2 ip from server to client via em1

We're testing using iperf ('client' actually runs the iperf server)
	client# iperf -s -l64K -N
	server# iperf -c client -i 5 -N -t 900 -l 64k

When testing without any extra delay on 'network' and send/recvspaces of
65535 bytes, we can sustain around 800mbit/s. The interrupts on
'network' may be the limiting factor here. However, when we set the
send/recv space to 65535*2, we can only sustain around 200-300mbit/s. It
seems the speed isn't as 'stable' either (peaks of more than 300mbit/s,
sometimes up to 500mbit/s). We also used read/write sizes of 128KB using
the -l option on iperf, but this didn't seem to have any  noticeable
effect.

When adding extra latency on 'network' and adjusting the
send/recv-spaces to correct for the greater bandwidth*delay product, we
weren't able to sustain rates much higher than 200mbit/s either.


Can anyone shed some light on what we're seeing here?

-- 
With regards,
Pieter de Boer




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?42B722EF.2090203>