From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 10:39:41 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8AD95106566C for ; Wed, 25 Feb 2009 10:39:41 +0000 (UTC) (envelope-from siquijorphilips@gmail.com) Received: from wf-out-1314.google.com (wf-out-1314.google.com [209.85.200.168]) by mx1.freebsd.org (Postfix) with ESMTP id 5DC5D8FC14 for ; Wed, 25 Feb 2009 10:39:41 +0000 (UTC) (envelope-from siquijorphilips@gmail.com) Received: by wf-out-1314.google.com with SMTP id 27so3113364wfd.7 for ; Wed, 25 Feb 2009 02:39:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=oV/QQrBR/PPnXKjAkNm+tYFQx8dU3yxqY5VxSWtfsa0=; b=Qze1wPVvjVBICyCVlmf98DobpU0FhrKrdiBa1E5KiGA5SXQhHXav3nRVMCVp+sIkbw himazvg9cqHYJZYGIo6y9wB5Amf+nwmE8B633LUFeYgYSkvaUKgs9G+57zXYQoIyFc5d OPFqFJCOX3v0d83fHvnyGwyAXfRltlRDGsWOA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=F3mqS2M0LnbdHCng3KjitZV6iDK9MU0nFIxD9VWDKuYDXOfTNZAuzdyeVsaLuZfPv2 YRqEj7iHcWHBlzyKGeXt1+R/75xBPQoyimP1uggImjF3JQBNk9WyRJyEIKvpLQORTUbJ NJYxuMKGBY9Tbzv5HEFeLx0b05lC1kpp1blSo= MIME-Version: 1.0 Received: by 10.142.241.15 with SMTP id o15mr3071244wfh.258.1235558380945; Wed, 25 Feb 2009 02:39:40 -0800 (PST) In-Reply-To: <20090225075310.GA85904@svzserv.kemerovo.su> References: <20090225075310.GA85904@svzserv.kemerovo.su> Date: Wed, 25 Feb 2009 18:39:40 +0800 Message-ID: From: Siquijor Philips To: Eugene Grosbein Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org Subject: Re: Questions on processing smaller frame size X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2009 10:39:42 -0000 Hello Eugene, > Traffic bandwidth does not matter (or much less), PPS rate matters. > Packets drop due to high pps rate. Higher packet size, lesser pps > saturates link and pps just can't grow high. It can with smaller packets. > All the test scenarios here are bombarded with 1-Gig of network traffic. When packet drops due to high pps rate, meaning to say that the current FreeBSD system can't still handle this kind of situation with high packet rate? Or just it depends on your hardware? I just can't imagine that with 2x quad-core system processing on high packet rate, average CPU utilization consumes a total of 98%. > I've tried to make FreeBSD 7.1 act as packet generator > with Intel dualcore 2.8Ghz processor and onboard gigabit ethernet em0 > using ng_source(4) low-overhead packet emitter. And it can't saturate > gigabit link with UDP packets (64 bytes payload, 130 bytes at wire - > including inter-packet gaps, FCSs etc.) > > It takes all CPU cycles of one 2.8Ghz core to send 750Kpps - Maybe there's a way we can optimize this, but just don't know how and what particular component to optimize? > I've profiled kernel with DTrace. It seems it spends many cycles > inside inlined mtx_lock/mtx_unlock functions. I would like to assume this is also what had happened to my Chelsio NIC. But I might be wrong here, so if someone is familiar with what's going on, then your idea is highly appreciated. I used to install Dtrace but got no luck with installation. My em(4) driver is statically > compiled into the kernel so mtx_lock/mtx_unload are inlined assembler ops. > The same is true with my cxgb(4) driver, statically compiled in the kernel. Regards, Siquijor