From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 08:09:52 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1E79A106564A for ; Wed, 25 Feb 2009 08:09:52 +0000 (UTC) (envelope-from eugen@kuzbass.ru) Received: from www.svzserv.kemerovo.su (www.svzserv.kemerovo.su [213.184.65.80]) by mx1.freebsd.org (Postfix) with ESMTP id 3AE708FC15 for ; Wed, 25 Feb 2009 08:09:50 +0000 (UTC) (envelope-from eugen@kuzbass.ru) Received: from www.svzserv.kemerovo.su (eugen@localhost [127.0.0.1]) by www.svzserv.kemerovo.su (8.13.8/8.13.8) with ESMTP id n1P7rAHK086759; Wed, 25 Feb 2009 14:53:10 +0700 (KRAT) (envelope-from eugen@www.svzserv.kemerovo.su) Received: (from eugen@localhost) by www.svzserv.kemerovo.su (8.13.8/8.13.8/Submit) id n1P7rAQJ086758; Wed, 25 Feb 2009 14:53:10 +0700 (KRAT) (envelope-from eugen) Date: Wed, 25 Feb 2009 14:53:10 +0700 From: Eugene Grosbein To: Siquijor Philips Message-ID: <20090225075310.GA85904@svzserv.kemerovo.su> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org Subject: Re: Questions on processing smaller frame size X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2009 08:09:52 -0000 On Wed, Feb 25, 2009 at 03:14:11PM +0800, Siquijor Philips wrote: > Can someone explain why FreeBSD network throughput (both incoming and > outgoing traffic) decreases when smaller frame size being processed? > With smaller frame size, corresponding packet rate (packet/sec or pps) > increases and experiencing dropped packets. What causes dropping of > packets with small frame size? Traffic bandwidth does not matter (or much less), PPS rate matters. Packets drop due to high pps rate. Higher packet size, lesser pps saturates link and pps just can't grow high. It can with smaller packets. I've tried to make FreeBSD 7.1 act as packet generator with Intel dualcore 2.8Ghz processor and onboard gigabit ethernet em0 using ng_source(4) low-overhead packet emitter. And it can't saturate gigabit link with UDP packets (64 bytes payload, 130 bytes at wire - including inter-packet gaps, FCSs etc.) It takes all CPU cycles of one 2.8Ghz core to send 750Kpps - I've profiled kernel with DTrace. It seems it spends many cycles inside inlined mtx_lock/mtx_unlock functions. My em(4) driver is statically compiled into the kernel so mtx_lock/mtx_unload are inlined assembler ops. Eugene Grosbein