From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 12:39:16 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 428091065670 for ; Wed, 25 Feb 2009 12:39:16 +0000 (UTC) (envelope-from siquijorphilips@gmail.com) Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.228]) by mx1.freebsd.org (Postfix) with ESMTP id 140D88FC15 for ; Wed, 25 Feb 2009 12:39:16 +0000 (UTC) (envelope-from siquijorphilips@gmail.com) Received: by rv-out-0506.google.com with SMTP id f6so3046538rvb.43 for ; Wed, 25 Feb 2009 04:39:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=Gf2N+NGL7QWj37fvsmy7Tyto7PtKrC4n7ea2Z4feGFc=; b=SCRxTaRQciEntO7Ls+na9MetFJxINNfMn5n6kXV3bvBCZNXDhrQD3iG7332Z7XzE4H C03UGLql5luVfDW8ZFmgDxR9ByRUZvQlaVT2FjpxYfPhFRf9dXrCYPRWwxku4Ujha/wm 5WHyOiN0LFU2EX+coLhjlWdoDcQYbmrs73crM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=guP9gS3GD36t2NPy/lq07z4AyLqaf8feGnqBRgFh6tjGMOEk4cImgil6a8PicSjZ9m TwdNtFQpwTjE4WzZuul2/d9TIA0Y7uUnSMCNPrW+P0cRIiujUq3FYNE6MxFz3f2WtjIv /YMDgXi92WvyJmkpvlXVXSWs/cPzPCmZorviY= MIME-Version: 1.0 Received: by 10.142.135.16 with SMTP id i16mr27184wfd.250.1235565217808; Wed, 25 Feb 2009 04:33:37 -0800 (PST) In-Reply-To: References: <20090225075310.GA85904@svzserv.kemerovo.su> Date: Wed, 25 Feb 2009 20:33:37 +0800 Message-ID: From: Siquijor Philips To: ivoras@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org Subject: Re: Questions on processing smaller frame size X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2009 12:39:17 -0000 Hello Ivan, This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig69E41D4C44B97AD296C94242 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Siquijor Philips wrote: > Hello Eugene, >=20 >> Traffic bandwidth does not matter (or much less), PPS rate matters. >> Packets drop due to high pps rate. Higher packet size, lesser pps >> saturates link and pps just can't grow high. It can with smaller packe= ts. >> >=20 > All the test scenarios here are bombarded with 1-Gig of network > traffic. When packet drops due to high pps rate, meaning to say that > the current FreeBSD system can't still handle this kind of situation > with high packet rate?=20 > Not unlikely. See other similar findings by other users, usually also > with em cards. Ok, let me check. > Or just it depends on your hardware? I just > can't imagine that with 2x quad-core system processing on high packet > rate, average CPU utilization consumes a total of 98%. >Total across all CPUs? With 64-byte and 128-byte frame, the total average CPU utilization will vary from 92-98%. Below is one of the top output. CPU # 6,7,5,4 and 0 are in 0% idle state already. By default, Chelsio NIC were using MSI/MSI-X interrupt on multiple RX/TX queues. # top -S PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 338 root 1 -68 - 0K 16K CPU7 1 639:53 98.93% irq262: cxgbc 340 root 1 -68 - 0K 16K CPU4 0 631:19 98.19% irq264: cxgbc 337 root 1 -68 - 0K 16K CPU6 3 642:28 98.10% irq261: cxgbc 339 root 1 -68 - 0K 16K RUN 1 616:07 96.63% irq263: cxgbc 336 root 1 -68 - 0K 16K CPU3 2 621:11 90.33% irq260: cxgbc 335 root 1 -68 - 0K 16K CPU0 2 633:18 89.50% irq259: cxgbc 334 root 1 -68 - 0K 16K CPU1 3 642:27 88.87% irq258: cxgbc 333 root 1 -68 - 0K 16K CPU5 1 648:13 88.57% irq257: cxgbc 341 root 1 -83 - 0K 16K RUN 0 157:14 13.53% cxgbsp 16 root 1 171 ki31 0K 16K RUN 1 484:55 8.59% idle: cpu1 15 root 1 171 ki31 0K 16K RUN 2 483:39 7.76% idle: cpu2 14 root 1 171 ki31 0K 16K RUN 3 490:02 7.37% idle: cpu3 11 root 1 171 ki31 0K 16K RUN 6 485:50 0.00% idle: cpu6 10 root 1 171 ki31 0K 16K RUN 7 484:51 0.00% idle: cpu7 12 root 1 171 ki31 0K 16K RUN 5 475:38 0.00% idle: cpu5 13 root 1 171 ki31 0K 16K RUN 4 412:14 0.00% idle: cpu4 17 root 1 171 ki31 0K 16K RUN 0 409:06 0.00% idle: cpu0 342 root 1 -83 - 0K 16K RUN 4 155:04 0.00% cxgbsp > Try reducing the number of CPUs, it might help by reducing contention. Ok, I'll try. Regards, Siquijor --------------enig69E41D4C44B97AD296C94242 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJpSIBldnAQVacBcgRAvGvAJ99YslOqGaklehf6uQjLrAEm/hJ6gCgyTUd rf3LDBNfsymm+jxbN0WHyU0= =mjmo -----END PGP SIGNATURE----- --------------enig69E41D4C44B97AD296C94242--