From owner-freebsd-net@FreeBSD.ORG Fri Sep 27 19:47:20 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 38883432 for ; Fri, 27 Sep 2013 19:47:20 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: from mail-lb0-x230.google.com (mail-lb0-x230.google.com [IPv6:2a00:1450:4010:c04::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B3980275B for ; Fri, 27 Sep 2013 19:47:19 +0000 (UTC) Received: by mail-lb0-f176.google.com with SMTP id y6so2631379lbh.35 for ; Fri, 27 Sep 2013 12:47:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=DB9ElekQhlHN3bT+pq1pPVOBsU+c+LCtv/uxh+EKYHo=; b=rneTDh4nSJHfq+uLX7TBasyMHHZLEyaeo6bW7F1FmUbb5PphBy1ADm0bFIa18XFPL4 au3RecwLywXHuFX5wPQ9V5EBFMlJMfrCxDBBQ3V1pObiMDQUhNGlR0ivfnT7E+eQ0aFP Gx/sIxNJmi8Qyixk8waxT/EM/so9CZriKkC6LA59uW/IBjZnzHW/OwycAbiTIIGFfrXo lb15LFm0y/wgD/E1P8UUp35TtBwXWCdubglYQc/tbbCmlhfTJIZQSaYuzc5qCPkYm2zH nlVcAb+sOKocjywZ3M+SOvRxPMLxFw1oSaLPFuT3r42CdfxG8TlTwwal2bvnSBv7ICYg 5SHg== MIME-Version: 1.0 X-Received: by 10.152.228.130 with SMTP id si2mr3532574lac.32.1380311237530; Fri, 27 Sep 2013 12:47:17 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.114.172.105 with HTTP; Fri, 27 Sep 2013 12:47:17 -0700 (PDT) In-Reply-To: References: Date: Fri, 27 Sep 2013 21:47:17 +0200 X-Google-Sender-Auth: JPZrY_FzWKoEMUrLDRAFSpbEu1Q Message-ID: Subject: Re: netmap: traffic distribution From: Luigi Rizzo To: chintu hetam Content-Type: text/plain; charset=ISO-8859-1 Cc: Michio Honda , hiren panchasara , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Sep 2013 19:47:20 -0000 On Fri, Sep 27, 2013 at 8:16 PM, chintu hetam wrote: > As far as i know, flow director is Intel terminology it addresses both RSS > and RFS. I think FreeBSD implementation is RFS. > > Luigi, you touched upon SW de-multiplexer, i would like to know why it's > necessary. the sw demux is only necessary if you expect to have traffic with heavy processing requirements for which you cannot easily guarantee that the RSS more or less balances the queues. Since the number of RSS queues is small, unbalance is not unlikely; in software you could in principle demux each flow to its own queue and then have one worker process per thread process queues. Note though that scheduling may become complicated if you care for CPU locality, but if the sw demux only reads buffers, then the situation is not too bad. > let say i have 82599 ixgbe driver (RSS enabled)configured with 5 tuple hash. > My application reads from netmap queue 0-7(1-8), for testing purpose i will > define tuples in such a way that i know for sure that each packet will be > filtered to specific hw queue(0-7). Now is it safe to assume that in netmap > mode in user space i will receive the packet as in ixgbe hw queue? yes. > > Michio, reason i asked for performance values > http://arxiv.org/ftp/arxiv/papers/1106/1106.0443.pdf The problem that the paper describes is due to the incorrect implementation (at least according to the description give in the paper) of the Flow Director (FD) feature in linux. The hardware demux (RSS or RFS) per se does not cause reordering unless it is misued (as described in the paper). Basically when FD is active, the RFS entry in the NIC is generated by looking at the core used for the transmit side of a flow, and (according to what the paper says; not sure if it has been fixed since), the association is updated dynamically and not generated just once at the beginning of the flow. So if the userspace thread floats between CPUs, the RFS entry keepfloating, and incoming packets for the same flow can be delivered to different queues. cheers luigi -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2211611 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+-------------------------------