From owner-freebsd-net@FreeBSD.ORG Mon Jun 20 21:14:55 2005 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EBA3916A41C for ; Mon, 20 Jun 2005 21:14:55 +0000 (GMT) (envelope-from pieter@thedarkside.nl) Received: from mail.thelostparadise.com (129pc197.sshunet.nl [145.97.197.129]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8C88F43D58 for ; Mon, 20 Jun 2005 21:14:55 +0000 (GMT) (envelope-from pieter@thedarkside.nl) Received: from [195.16.84.92] (92-unused.virt-ix.net [195.16.84.92]) by mail.thelostparadise.com (8.12.11/8.12.8) with ESMTP id j5KLEr9J092467; Mon, 20 Jun 2005 23:14:54 +0200 (CEST) (envelope-from pieter@thedarkside.nl) Message-ID: <42B731CD.1040104@thedarkside.nl> Date: Mon, 20 Jun 2005 23:14:53 +0200 From: Pieter de Boer User-Agent: Mozilla Thunderbird 1.0.2 (X11/20050527) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Luigi Rizzo References: <42B722EF.2090203@thedarkside.nl> <20050620135044.B35720@xorpc.icir.org> In-Reply-To: <20050620135044.B35720@xorpc.icir.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org Subject: Re: Issues with a Large Fat pipe Network simulation X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2005 21:14:56 -0000 Luigi Rizzo wrote: >>When testing without any extra delay on 'network' and send/recvspaces of >>65535 bytes, we can sustain around 800mbit/s. The interrupts on >>'network' may be the limiting factor here. However, when we set the >>send/recv space to 65535*2, we can only sustain around 200-300mbit/s. It > > > have you checked that these two > kern.ipc.maxsockbuf: 262144 > kern.ipc.sockbuf_waste_factor: 8 > > aren't responsible for the trouble ? I've seen 200-300mbit when using a 1MB maxsockbuf. Raising it to 2, 4 or 8MB doesn't seem to have much (if any) effect. Altering sockbuf_waste_factor doesn't seem to do anything, either. However.. when I deleted the pipe rules on 'network', the speed suddenly went up to around 800mbit/s too! I remade them, and voila, 200mbit/s. So apparantly it's an issue in the dummynet configuration I have. So, to reiterate: On 'network': pipe 1 from client to server via em0 pipe 2 from server to client via em1 allow ip from any to any Will give me 200MBit/s when using 128KB window sizes, but 800MBit/s when using 64KB window sizes. On 'network': allow ip from any to any Will give me 800Mbit/s when using 64KB or 128KB window sizes. I haven't configured the pipes in any way, though. So it appears it's due to the 'network' box afterall.. -- Pieter