From owner-freebsd-net@FreeBSD.ORG Fri Sep 4 16:28:13 2009 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 125071065679 for ; Fri, 4 Sep 2009 16:28:13 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from mail.digiware.nl (unknown [IPv6:2001:4cb8:1:106::2]) by mx1.freebsd.org (Postfix) with ESMTP id A3EB18FC1C for ; Fri, 4 Sep 2009 16:28:12 +0000 (UTC) Received: from localhost (localhost.digiware.nl [127.0.0.1]) by mail.digiware.nl (Postfix) with ESMTP id E95AE153436 for ; Fri, 4 Sep 2009 18:28:10 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from mail.digiware.nl ([127.0.0.1]) by localhost (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rygiwvgcPbPk; Fri, 4 Sep 2009 18:28:09 +0200 (CEST) Received: from [212.61.27.67] (opteron [212.61.27.67]) by mail.digiware.nl (Postfix) with ESMTP id 15474153435 for ; Fri, 4 Sep 2009 18:28:09 +0200 (CEST) Message-ID: <4AA14018.3010102@digiware.nl> Date: Fri, 04 Sep 2009 18:28:08 +0200 From: Willem Jan Withagen Organization: Digiware User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: UDP output performance X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Sep 2009 16:28:13 -0000 First of: I've been googleing for about a day, but I'll take any suggestions for more info. What I'm trying to do is get as much 1440 byte UDP packets out of an em device. And when tat works, get as much out of the 7 em devices that this board has. :) Currently I run into trouble at 250*174 = 43500 packets/sec. How is the setup: em0 gets 1 stream of 174 p/s which is ~ 2Mbit this gets repeated to 250 streams currently to 2 other servers, 125 streams each. each on their own 1 Gbit port This works uptil 123 streams each, going high gives packet loss. So this is at about 500Mbit/sec on a 1Gbit port And why do I know that the packetloss is not in the network? Well there are no errors on the output interface the ports on the switch the input ports on the receivers the mib of the switch does not show any signs of dropped packets, or likes. Also I can change the order of the queing in my repeater, and then the packetloss moves to the host which is last the outputlist. I tried raising the socketbuffer " sysctl -w net.inet.udp.maxdgram=184320" in a few steps. But that did not bring anything. So my guess is that I'm dropping packets somewhere from the output socket to the wire. BTW al stats in systat -vm are close to 0%. What tunables are there to turn? And if not tuneable, what parts of the code would be target for closer inspection. Any help is more than welcome --WjW