From owner-svn-src-all@FreeBSD.ORG Mon Mar 17 19:58:27 2014 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8A23FD72; Mon, 17 Mar 2014 19:58:27 +0000 (UTC) Received: from mail-qc0-x22c.google.com (mail-qc0-x22c.google.com [IPv6:2607:f8b0:400d:c01::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 10B7816D; Mon, 17 Mar 2014 19:58:26 +0000 (UTC) Received: by mail-qc0-f172.google.com with SMTP id i8so6567653qcq.17 for ; Mon, 17 Mar 2014 12:58:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=RSppg1VBene+fOQlxAWC5uv0qBTVoHL5bvt9rygeCD4=; b=CZpUEJzoyAgHCxy3f3Bjvb0XJHOQQYiQ1iOEqkswNeuHmxPHY466r5J6XvZ0aQYt+A BZsl1emBlenNbXY9bPM68L1mmfiYApxmO5Tk/ziKbdM01rxoMRQq6BrYYplt6yq8Odlv iHkMeZNIWvY1T7ix65XilVUYbDai4iWrtW0lS06wtP3HkvC3c2bz4ZuiXPVDB6JlCWzj ePh4p18uO6JdVBsNBTl0d05Q9gRVtP5wMqeIPROz39I7VlLR1AOBbMzz2NdMgBNcKUmX NMuV+y0zbYS3q/37WMmTjNUywzkTrqJsu/WIeatD5veStOkDhC6DLAyr8mN+02dsmo7T kfdg== MIME-Version: 1.0 X-Received: by 10.224.79.133 with SMTP id p5mr4503035qak.98.1395086306229; Mon, 17 Mar 2014 12:58:26 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.8.137 with HTTP; Mon, 17 Mar 2014 12:58:26 -0700 (PDT) In-Reply-To: References: <201403150057.s2F0vofg081606@svn.freebsd.org> <20140317075436.GA5864@walton.maths.tcd.ie> Date: Mon, 17 Mar 2014 12:58:26 -0700 X-Google-Sender-Auth: B6UR16eVeNUnyoRITuiReFJG5Ts Message-ID: Subject: Re: svn commit: r263198 - in head/sys: amd64/conf conf net netinet netinet6 sys From: Adrian Chadd To: "Robert N. M. Watson" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: David Malone , "svn-src-head@freebsd.org" , "svn-src-all@freebsd.org" , "src-committers@freebsd.org" X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Mar 2014 19:58:27 -0000 On 17 March 2014 01:26, Robert N. M. Watson wrote: > > On 17 Mar 2014, at 07:54, David Malone wrote: > >>> (1) Merge a software implementation of the Toeplitz hash specified in >>> RSS implemented by David Malone. This is used to allow suitable >>> pcbgroup placement of connections before the first packet is >>> received from the NIC. Software hashing is generally avoided, >>> however, due to high cost of the hash on general-purpose CPUs. >> >> I could look at a faster software implementation, but I guess most of >> the value is when hashing is done on the NIC. > > In my benchmarking (a couple of years ago) the software path never really= turned up a lot. If you end up with a flow migrating from an RSS NIC to a = non-RSS NIC, you fall back on the 'reservation' hash table, using the conve= ntional hash and picking up contention, but you don't end up doing the soft= ware version of Toeplitz per-packet. The only reason we need a software imp= lementation (currently) is to do an initial placement of a new outbound flo= w into the hash table prior to receiving a packet with the hardware-generat= ed hash on it. For the inbound direction, we can pick it up from the first = packet. (Although, actually, we also need to do it for inbound flows that c= ome from non-RSS NICs -- or localhost or such). If it's done right, then yes, we should only see a per-packet RSS hash on: * NICs / paths that don't do it in hardware on the receive side; * Doing per-packet sanity checking - ie, doing a software RSS as well as the hardware RSS in order to verify that they both line up. I plan on adding the latter at some point soon so we can ensure that things end up on the right queue. For higher connection rate workloads, we want to ensure that the initial hash selection for outbound matches the inbound, as we don't want to be grabbing non-local-CPU locks to migrate connections to other pcbgroups after they've been established. We absolutely want to keep things on the local CPU as much as we can. Thanks! -a