From owner-freebsd-net@FreeBSD.ORG Mon Oct 3 01:39:55 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B8203106566C for ; Mon, 3 Oct 2011 01:39:55 +0000 (UTC) (envelope-from lacombar@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 46C6A8FC08 for ; Mon, 3 Oct 2011 01:39:54 +0000 (UTC) Received: by wyj26 with SMTP id 26so3604942wyj.13 for ; Sun, 02 Oct 2011 18:39:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=1NLgMroqNBM9FEB2hqPpyiOBTL4Qj+0jWn9kI2a5oRU=; b=HtP0jOa02XLGyQtJshltnWNT7oc7X3PkWVavz/kc/lplb0rKdlZshtM6qUXB4smSCb +/ZUNxdQ0XO5yv3jAYUFNq6z/9GmxAVaYIujJXWzbJCloEz4upTA4iZ1HouuIcQ4p+8H qyNUV5TUofgf1dSknVeGDCdQ5ARnl0e4Y47x4= MIME-Version: 1.0 Received: by 10.227.153.211 with SMTP id l19mr181113wbw.51.1317605993809; Sun, 02 Oct 2011 18:39:53 -0700 (PDT) Received: by 10.180.103.33 with HTTP; Sun, 2 Oct 2011 18:39:53 -0700 (PDT) In-Reply-To: References: <4E886A3B.7000502@sepehrs.com> Date: Sun, 2 Oct 2011 21:39:53 -0400 Message-ID: From: Arnaud Lacombe To: Jack Vogel Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-net@freebsd.org" , Hooman Fazaeli Subject: Re: em(4) high latency w/o msix X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 01:39:55 -0000 Hi, On Sun, Oct 2, 2011 at 8:29 PM, Arnaud Lacombe wrote: > Hi, > > On Sun, Oct 2, 2011 at 12:08 PM, Jack Vogel wrote: >> On what hardware? >> > Only the 82574 is using em(4)'s MSI-X. > FWIW, I'm not seeing this issue on a 82574 box I've access to: result of the following command: # ping -nqt 30 -f The two box are connected on an old 10/100 switch MSI-X enabled: 239361 packets transmitted, 239358 packets received, 0.0% packet loss round-trip min/avg/max/stddev =3D 0.090/0.117/2.488/0.027 ms MSI-X disabled: 233122 packets transmitted, 233119 packets received, 0.0% packet loss round-trip min/avg/max/stddev =3D 0.096/0.120/2.589/0.027 ms On another interface (same boxes involved, but through a 10/100/1000 switch= ): MSI-X enabled: 240080 packets transmitted, 240079 packets received, 0.0% packet loss round-trip min/avg/max/stddev =3D 0.090/0.116/0.175/0.008 ms MSI-X disabled: 207310 packets transmitted, 207309 packets received, 0.0% packet loss round-trip min/avg/max/stddev =3D 0.095/0.136/0.310/0.044 ms However, there seem to be a performance drop with MSI-X disabled and Gigabit link. - Arnaud >> Jack >> >> >> On Sun, Oct 2, 2011 at 6:42 AM, Hooman Fazaeli wro= te: >> >>> >>> Latest em(4) driver from HEAD seems to have high latency >>> when MSIX is disabled. >>> >>> With MSIX enabled (hw.em.enable_msix=3D1): >>> >>> # ping -c5 192.168.1.83 >>> PING 192.168.1.83 (192.168.1.83): 56 data bytes >>> 64 bytes from 192.168.1.83: icmp_seq=3D0 ttl=3D64 time=3D0.055 ms >>> 64 bytes from 192.168.1.83: icmp_seq=3D1 ttl=3D64 time=3D0.076 ms >>> 64 bytes from 192.168.1.83: icmp_seq=3D2 ttl=3D64 time=3D0.066 ms >>> 64 bytes from 192.168.1.83: icmp_seq=3D3 ttl=3D64 time=3D0.051 ms >>> 64 bytes from 192.168.1.83: icmp_seq=3D4 ttl=3D64 time=3D0.063 ms >>> >>> --- 192.168.1.83 ping statistics --- >>> 5 packets transmitted, 5 packets received, 0.0% packet loss >>> round-trip min/avg/max/stddev =3D 0.051/0.062/0.076/0.009 ms >>> >>> With MSIX disabled: >>> >>> # ping -c5 192.168.1.83 >>> PING 192.168.1.83 (192.168.1.83): 56 data bytes >>> 64 bytes from 192.168.1.83: icmp_seq=3D0 ttl=3D64 time=3D0.180 ms >>> 64 bytes from 192.168.1.83: icmp_seq=3D1 ttl=3D64 time=3D0.164 ms >>> 64 bytes from 192.168.1.83: icmp_seq=3D2 ttl=3D64 time=3D0.169 ms >>> 64 bytes from 192.168.1.83: icmp_seq=3D3 ttl=3D64 time=3D0.172 ms >>> 64 bytes from 192.168.1.83: icmp_seq=3D4 ttl=3D64 time=3D0.167 ms >>> >>> --- 192.168.1.83 ping statistics --- >>> 5 packets transmitted, 5 packets received, 0.0% packet loss >>> round-trip min/avg/max/stddev =3D 0.164/0.170/0.180/0.005 ms >>> >>> As you see, w/o MSIX, RTT increases by a factor of 3. >>> >>> I also tested the following drivers: >>> =A0 =A0- igb(4) from HEAD: OK. >>> =A0 =A0- Stock 7.3-RELEASE: OK. >>> =A0 =A0- Stock 7.4-RELEASE: problem exist. >>> >>> Any ideas? >>> >>> >>> >>> >>> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >