From owner-freebsd-arch@FreeBSD.ORG Wed Jul 16 01:01:42 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 300359A3; Wed, 16 Jul 2014 01:01:42 +0000 (UTC) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mx1.freebsd.org (Postfix) with ESMTP id 0AEFD2988; Wed, 16 Jul 2014 01:01:41 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga102.fm.intel.com with ESMTP; 15 Jul 2014 18:01:35 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.01,668,1400050800"; d="scan'208";a="562368945" Received: from orsmsx109.amr.corp.intel.com ([10.22.240.7]) by fmsmga001.fm.intel.com with ESMTP; 15 Jul 2014 18:01:34 -0700 Received: from orsmsx153.amr.corp.intel.com (10.22.226.247) by ORSMSX109.amr.corp.intel.com (10.22.240.7) with Microsoft SMTP Server (TLS) id 14.3.123.3; Tue, 15 Jul 2014 18:01:34 -0700 Received: from orsmsx111.amr.corp.intel.com ([169.254.11.5]) by ORSMSX153.amr.corp.intel.com ([169.254.12.126]) with mapi id 14.03.0123.003; Tue, 15 Jul 2014 18:01:34 -0700 From: "Pieper, Jeffrey E" To: Hooman Fazaeli , Adrian Chadd Subject: RE: UDP/TCP versus IP frames - subtle out of order packets with hardware hashing Thread-Topic: UDP/TCP versus IP frames - subtle out of order packets with hardware hashing Thread-Index: AQHPn8XtuhYKXvJgqkGKs4ElIQPwu5uhTKMAgACVKiA= Date: Wed, 16 Jul 2014 01:01:33 +0000 Message-ID: <2A35EA60C3C77D438915767F458D65687E8FC789@ORSMSX111.amr.corp.intel.com> References: <53C4EE00.5090705@gmail.com> In-Reply-To: <53C4EE00.5090705@gmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.22.254.138] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 01:01:42 -0000 I believe the Linux driver DOES however support UDP RSS port hash. It is us= ed for environments where you can ensure there is no fragmentation. In such= use cases, this allows for a significant performance benefit. Jeff -----Original Message----- From: owner-freebsd-net@freebsd.org [mailto:owner-freebsd-net@freebsd.org] = On Behalf Of Hooman Fazaeli Sent: Tuesday, July 15, 2014 2:02 AM To: Adrian Chadd Cc: FreeBSD Net; freebsd-arch@freebsd.org Subject: Re: UDP/TCP versus IP frames - subtle out of order packets with ha= rdware hashing On 7/15/2014 5:14 AM, Adrian Chadd wrote: > Hi, > > Whilst digging into UDP receive side scaling on the intel ixgbe(4) > NIC, I stumbled across how it hashes traffic between IP fragmented > traffic and non IP-fragmented traffic. > > Here's how it surfaced: > > * the ixgbe(4) NIC is configured to hash on both IP (2-tuple) and > TCP/UDP (4-tuple); > * when a non-fragmented UDP frame comes in, it's hashed on the 4-tuple > and comes into queue A; > * when a fragmented UDP frame comes in, it's hashed on the IP 2-tuple > and comes into queue B. > > So if there's a mix of small and large datagrams, we'll end up with > some packets coming in via queue A and some by queue B. In normal > operation that'll result in out of order packets. > > For the RSS stuff I'm working on it means that some packets will match > the PCBGROUP setup and some won't. By default UDP configures a 2-tuple > hash so it expects packets to come in hashed appropriately. But that > only matches for large frames. For small frames it'll be hashed via > the 4-tuple and it won't match. > > The ip reassembly code doesn't recalculate the flowid/flowtype once > it's finished. It'd be nice to do that before further processing so it > can be placed in the right netisr. > > So there's a couple of semi-overlapping issues: > > * Right now we could get TCP and UDP frames out of order. I'd like to > at least have ixgbe(4) hash on the 2-tuple for UDP rather than the > 4-tuple. That fixes that silly corner case. It's not likely going to > show up except for things like forwarding workloads. Maybe people > doing memcached work, I'm not sure. > > * Whether or not to calculate the flowid/flowtype in ip_reass() (or > maybe in the netisr input path, in case there's no flowid assigned) so > work is better distributed; > > * .. then if we do that, we could do 4-tuple UDP hashing again and > we'd just recalculate for any large frames. > > Here's what happened with Linux and ixgbe in 2010 on this topic: > > http://comments.gmane.org/gmane.linux.network/166687 > > What do people think? > > > -a > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" >Doesn't the problem applies to TCP too? >TCP may be fragmented too but is less likely because of MSS. > >--=20 > >Best regards. >Hooman Fazaeli _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"