From owner-freebsd-questions@FreeBSD.ORG Tue Apr 8 20:55:48 2014 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B2EC3D99; Tue, 8 Apr 2014 20:55:48 +0000 (UTC) Received: from mail-wi0-x22b.google.com (mail-wi0-x22b.google.com [IPv6:2a00:1450:400c:c05::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1B43313C6; Tue, 8 Apr 2014 20:55:47 +0000 (UTC) Received: by mail-wi0-f171.google.com with SMTP id q5so7819573wiv.16 for ; Tue, 08 Apr 2014 13:55:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=9UFct1aiFF/XWjC8+5wMpel5PE6Lc8zLflvhSnZ7TmM=; b=0wruonyXeRM68BQvPJoY+SbRPr8yvAaGPF3fhyHJx7ylbNWNLj/JD7GA5fRheeQKVk hIzm9w1eUdm9fQ3iJSQQC392qWvAObrif2hsYxE0Fqelqs/l2xwtsQr7YfrbSBTCzCOz LwtA0OJO7y0XTJs/AeNmT6C5XTu+yFa3AXaLTnmzFwp5d5ihvNNLgfkB6DGouA+zd/xP /Wnwvvdoz64OPL/coaGeIpozPpuHZhqBEfvvrlR2pQ+CRv3MXcchUHlp//wpIQrtK5eN K+TwV1rFwxkcZXzQLqlCZkDEtbX3d7X8oE1sE/Z7iQ6nhoyAfg7P0/kmzRS1MPu6UMeY MahA== MIME-Version: 1.0 X-Received: by 10.194.82.35 with SMTP id f3mr5932982wjy.36.1396990546393; Tue, 08 Apr 2014 13:55:46 -0700 (PDT) Received: by 10.216.61.203 with HTTP; Tue, 8 Apr 2014 13:55:46 -0700 (PDT) In-Reply-To: References: <6876ba1714363dcbbdaf6b23f294fa2a@mail.feld.me> Date: Tue, 8 Apr 2014 21:55:46 +0100 Message-ID: Subject: Re: FreeBSD 10-R, Xen 4.1 guest, pf/NAT performance question From: "seanrees@gmail.com" To: Mark Felder Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 20:55:48 -0000 I did notice one more odd thing today: Good VPS: throughput at ~5mbps: ~1800 context switches Bad VPS: throughput at 0.5mbps: ~240 context switches measured using systat -v. In general, the number of context switches, traps, and system calls were about an order of magnitude more on the good VPS vs. the bad VPS. I'm wondering if this perhaps rings any bells? (or opens up avenues of tuning or investigation) Sean On Mon, Apr 7, 2014 at 3:50 PM, seanrees@gmail.com wrote: > Thanks for the tip. Yes, I had already run into TSO4 causing issues with > this VM, so it was switched off (ifconfig xn0 -tso4). > > I also set net.inet.tcp.tso=0 (was 1) and tried again - no change. :( > > > Sean > > > On Mon, Apr 7, 2014 at 3:10 PM, Mark Felder wrote: > >> On 2014-04-07 07:57, seanrees@gmail.com wrote: >> >>> Hi there freebsd-questions, >>> >>> I've been batting my head against this problem for a few days now and not >>> having much progress, so I'm hoping to get pointers at what to look at >>> next. >>> >>> I've got a FreeBSD 10-R guest in Xen 4.1 (I am just a customer of the Xen >>> provider; I don't run the Xen hypervisor myself). I use this instance to >>> terminate a VPN, for which I also NAT VPN clients with PF. I am seeing >>> unusually slow packet forwarding performance: 0.5mbit internet -> vpn >>> client, 2.0 mbit vpn client -> internet. (the numbers should be closer to >>> 10mbit/5mbit). >>> >>> This guest is a duplicate of another Xen instance I have in another data >>> centre. I manage the configurations and packages centrally and aside from >>> IP address differences, the machines are configured identically. The >>> differences: it's 30ms closer to me and runs in Xen 3.4. I see >>> performance >>> from this machine in the 10mbps range. >>> >>> I've eliminated the obvious: >>> - The problem VPS is fine network wise; can download tarballs from the >>> Internet at 100mbps. >>> - VPS -> Home is fine; can download at ~10mbps; the problem is isolated >>> to forwarding Home -> VPS -> Internet and back. >>> - I excluded OpenVPN as the cause by replicating the setup with ssh -w; >>> same performance. >>> - SSH port forwarding (ssh -L) is fast; indicating to me the issue is >>> somewhere in the PF/kernel. >>> - I checked TCP options by capturing traffic at varying points; these >>> seem fine. I see a good deal of TCP retransmits but the window sizes stay >>> the same. >>> >>> Any thoughts on what to check next? >>> >>> >> Have you turned off TSO? >> >> ifconfig xn0 -tso >> >> or >> >> sysctl net.inet.tcp.tso=0 >> _______________________________________________ >> freebsd-questions@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-questions >> To unsubscribe, send any mail to "freebsd-questions- >> unsubscribe@freebsd.org" >> > >