From owner-freebsd-net@FreeBSD.ORG Tue Jun 24 02:40:55 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 40B56359; Tue, 24 Jun 2014 02:40:55 +0000 (UTC) Received: from mail-wi0-x235.google.com (mail-wi0-x235.google.com [IPv6:2a00:1450:400c:c05::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9E7A0213D; Tue, 24 Jun 2014 02:40:54 +0000 (UTC) Received: by mail-wi0-f181.google.com with SMTP id n3so5083417wiv.8 for ; Mon, 23 Jun 2014 19:40:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=CJnWdBKuKehgHB3KCFM7ocJ2UdlAiuIGAFml0TdUVis=; b=gHRUOSDQ8KVqFucVSjyeNiuOscfqEl8Wcf7rTC9f0QoTpQtn90RZlOI1ThlyLlXCDD 3eA1BCHlhki1ehUghZdnQty5npDMEpnRGV2Z1d4+nMcfCBvnSO4s5nal20ffX5ToKmG2 knpVCBSwibrDciMQ/rIcuN1JZsYfpZiPbDGOQzMVo6fxaHLvedQV6UuGx7s3OQu/OAy9 wrxQtIXYXVvGJ2p1iuIpE8rCgFaygOXcYYLHnghJp2dcbgWRyLCLrUbEln7PbY7q+BiL rowqpiUAH7uaCHfJmljzLMrDAbW8fKDusC1qLJfi7XwJlH9kkLNUCNj+sM9wSCPKwaJI X+xg== MIME-Version: 1.0 X-Received: by 10.180.19.37 with SMTP id b5mr30371361wie.16.1403577652768; Mon, 23 Jun 2014 19:40:52 -0700 (PDT) Received: by 10.217.9.134 with HTTP; Mon, 23 Jun 2014 19:40:52 -0700 (PDT) Reply-To: araujo@FreeBSD.org In-Reply-To: References: Date: Tue, 24 Jun 2014 10:40:52 +0800 Message-ID: Subject: Re: [patch][lagg] - Set a better granularity and distribution on roundrobin protocol. From: Marcelo Araujo To: Adrian Chadd Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2014 02:40:55 -0000 2014-06-24 6:54 GMT+08:00 Adrian Chadd : > Hi, > > No, don't introduce out of order behaviour. Ever. Yes, it has out of order behavior; with my patch much less. I upload two pcap files and you can see by yourself, if you don't believe in what I'm talking about. Test done using: "iperf -s" and "iperf -c -i 1 -t 10". 1) Don't change the number of packets(default round robin behavior). http://people.freebsd.org/~araujo/lagg/lagg-nop.cap 8 out of order packets. Several SACKs. 2) Set the number of packets to 50. http://people.freebsd.org/~araujo/lagg/lagg.cap 0 out of order packets. Less SACKs. > You may not think > it's a problem for TCP, but UDP things and VPN things will start > getting very angry. There are VPN configurations out there that will > drop the VPN if frames are out of order. > I'm not thinking that will be a problem for TCP, but, in somehow it will be, less throughput as I showed before, and less SACK. About the VPN, please, tell me which softwares, and let me know where I can get a sample to make a testbed. However to be very honest, I don't believe anyone here when change something at network protocols will make this extensive testbed. It is almost impossible to predict what software it will works or not, and I don't believe anyone here has all these stuff in hands. > > The ixgbe driver is setting the flowid to the msix queue ID, rather > than a 32 bit unique flow id hash value for the flow. That makes it > hard to do traffic distribution where the flowid is available. > Thanks for the explanation. > > There's an lagg option to re-hash the mbuf rather than rely on the > flowid for outbound port choice - have you looked at using that? Did > that make any difference? > Yes, I set to 0 the net.link.lagg.0.use _flowid, it make a little difference to the default round robin implementation, but yet I can't reach more than 5 Gbit/s. With my patch and set the packets to 50, it improved a bit too. So, thank you so much for all review, I don't know if you have time and a testbed to make a real test, as I'm doing. I would be happy if you or more people could make tests on that patch. Also, I have only ixgbe(4) to make tests, would appreciate if this patch could be tested with other NICs too. Best Regards, -- Marcelo Araujo (__)araujo@FreeBSD.org \\\'',)http://www.FreeBSD.org \/ \ ^ Power To Server. .\. /_)