From owner-freebsd-net@FreeBSD.ORG Fri May 25 21:02:38 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 90D3C1065670 for ; Fri, 25 May 2012 21:02:38 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 21CF48FC08 for ; Fri, 25 May 2012 21:02:37 +0000 (UTC) Received: by werg1 with SMTP id g1so1094083wer.13 for ; Fri, 25 May 2012 14:02:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=Cjw7LB0764MT/1kkX61KjS0r/ylM72HttM92KrAMFi0=; b=Ugyg3AMHJ9PA7P7nojUkb8jV2yqDYbTDmJ6Lia5SuDcIJePzZgCNOUkvPjmBLQo/YP MxsVxbsvLQmzhEwsAsoP3OGu2lzv27S822botF4p5m5ybOMP5BWgJzjW+31B0tcCo2Kz DHA03SqZseu1tEQ3pozHZuPHX5aEwKJ+hoBn1EMgC3+/wrn+yBifTb3/fVTaCHZl7cLA d3xuD+9X/pEttuqVZjV93RTPmgSf9Dax2eCn7Be5QT2FE+orpX66XVDe72mEEVh1y0zl /IR6VI+9DLKQEBnVwOpA2iQW659X8ekgifN3qP9GoRyfAwjMCH3teU0zekyH/ZUqyCUQ IWqQ== MIME-Version: 1.0 Received: by 10.180.109.197 with SMTP id hu5mr673389wib.8.1337979757093; Fri, 25 May 2012 14:02:37 -0700 (PDT) Received: by 10.223.155.4 with HTTP; Fri, 25 May 2012 14:02:37 -0700 (PDT) In-Reply-To: <4FBF88CE.20209@cs.duke.edu> References: <4FBF88CE.20209@cs.duke.edu> Date: Fri, 25 May 2012 14:02:37 -0700 Message-ID: From: Kevin Oberman To: Andrew Gallatin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org, Andrew Gallatin Subject: Re: Major performance hit with ToS setting X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 May 2012 21:02:38 -0000 On Fri, May 25, 2012 at 6:27 AM, Andrew Gallatin wro= te: > On 05/24/12 18:55, Kevin Oberman wrote: > >> >> This is,of course, on a 10G interface. On 7.3 there is little > > > Hi Kevin, > > > What you're seeing looks almost like a checksum is bad, or > there is some other packet damage. =A0Do you see any > error counters increasing if you run netstat -s before > and after the test & compare the results? > > Thinking that, perhaps, this was a bug in my mxge(4), I attempted > to reproduce it this morning between =A08.3 and 9.0 boxes and > failed to see the bad behavior.. > > % nuttcp-6.1.2 -c32t -t diablo1-m < /dev/zero > =A09161.7500 MB / =A010.21 sec =3D 7526.5792 Mbps 53 %TX 97 %RX 0 host-re= trans > 0.11 msRTT > % nuttcp-6.1.2 =A0-t diablo1-m < /dev/zero > =A09140.6180 MB / =A010.21 sec =3D 7509.8270 Mbps 53 %TX 97 %RX 0 host-re= trans > 0.11 msRTT > > > However, I don't have any 8.2-r box handy, so I cannot > exactly repro your experiment... As seems to always be the case, the test system I set up to try other versions of FreeBSD does not exhibit the problem. I have no difficulty generating traffic at speeds above 1G. (I saw a max on one run over just a few miles of over 9G.) So I am scratching my head and re-testing this. FWIW, three different engineers tested production systems and all saw the same issue, whether using nuttcp or iperf. If I run hte tests now on one of those systems, I see the problem. So I am off to do a bunch more testing and I'll try to have some real information soon. Thanks for looking into this. --=20 R. Kevin Oberman, Network Engineer E-mail: kob6558@gmail.com