From owner-freebsd-performance@FreeBSD.ORG Mon Feb 6 21:09:59 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6B7CE106566C for ; Mon, 6 Feb 2012 21:09:59 +0000 (UTC) (envelope-from julianwissmann@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id E85998FC1A for ; Mon, 6 Feb 2012 21:09:58 +0000 (UTC) Received: by eaan10 with SMTP id n10so3115980eaa.13 for ; Mon, 06 Feb 2012 13:09:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:content-type:subject:date:message-id:to:mime-version:x-mailer; bh=UhLinY5qsWaLnD4aVIWlhkIRZ+8cmBKOY/RURWERFGs=; b=Ru7j7+RUmsVgl53VRrA+dx7kZ3kaRYySN1UrzyB73jCXLfbQ8I1aG2mOggobmROxal JUvMODZUpsrtcW2zexho0PGx3VB52qS+km3trsHFPOwsH+OOB/wL06PvxMMTeejUvF/S LvHgLPC/7fZceOu+OezRdwhnODIx7Pdp+7710= Received: by 10.213.21.148 with SMTP id j20mr1669564ebb.74.1328560697793; Mon, 06 Feb 2012 12:38:17 -0800 (PST) Received: from [192.168.10.134] (g225217060.adsl.alicedsl.de. [92.225.217.60]) by mx.google.com with ESMTPS id t11sm64752397eea.10.2012.02.06.12.38.15 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 06 Feb 2012 12:38:17 -0800 (PST) From: Julian Wissmann Content-Type: multipart/signed; boundary="Apple-Mail=_8979D47F-1B0A-4AE9-B7B7-BE2295364A9D"; protocol="application/pgp-signature"; micalg=pgp-sha1 Date: Mon, 6 Feb 2012 21:38:07 +0100 Message-Id: <7FE9C466-2459-4C0E-949A-7C0843B04BB4@gmail.com> To: freebsd-performance@freebsd.org Mime-Version: 1.0 (Apple Message framework v1251.1) X-Mailer: Apple Mail (2.1251.1) X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Tor on FreeBSD Performance issues X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Feb 2012 21:09:59 -0000 --Apple-Mail=_8979D47F-1B0A-4AE9-B7B7-BE2295364A9D Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi, I'm an admin for a non-profit that runs Tor exit nodes, most of them on = Linux currently, but due to problems on our high bandwidth nodes, we = decided to migrate one of them to FreeBSD to do some testing.=20 I've been using FreeBSD for quite some years now, longer than Linux, so = I figured this would probably be a no-brainer, but turns out, that it = isn't. On FreeBSD I currently manage to push 150-200Mbits with some heavy = tuning applied already, on Linux it is roughly 500Mbits. Therefor I'm wondering if I'm really running into some limitations here = or if I'm actually doing something wrong. ifconfig bge0 bge0: flags=3D8843 metric 0 mtu = 1500 = options=3Dc01db As you can see I've compiled polling and currently I'm running on = kern.hz=3D16000 as that's given me the best performance, so far. This is my netstat output for tcp on ipv4: netstat -s tcp: 265086752 packets sent 84255022 data packets (155791599035 bytes) 2244601 data packets (2698410010 bytes) retransmitted 73011 data packets unnecessarily retransmitted 684 resends initiated by MTU discovery 153151729 ack-only packets (0 delayed) 0 URG only packets 19864 window probe packets 16854982 window update packets 8551532 control packets 236720260 packets received 86600847 acks (for 155842386836 bytes) 5062568 duplicate acks 76267 acks for unsent data 138258588 packets (150041170335 bytes) received = in-sequence 1502804 completely duplicate packets (562243206 bytes) 44193 old duplicate packets 91821 packets with some dup. data (23693391 bytes duped) 6598536 out-of-order packets (7168950882 bytes) 32074 packets (20848106 bytes) of data after window 7806 window probes 1457705 window update packets 159860 packets received after close 5219 discarded for bad checksums 3 discarded for bad header offset fields 0 discarded because packet too short 1468 discarded due to memory problems 5665849 connection requests 694088 connection accepts 0 bad connection attempts 129 listen queue overflows 9308 ignored RSTs in the windows 3289250 connections established (including accepts) 6334698 connections closed (including 449721 drops) 916420 connections updated cached RTT on close 923786 connections updated cached RTT variance on close 273103 connections updated cached ssthresh on close 354989 embryonic connections dropped 81015541 segments updated rtt (of 56772442 attempts) 9127304 retransmit timeouts 19875 connections dropped by rexmit timeout 21274 persist timeouts 215 connections dropped by persist timeout 4541 Connections (fin_wait_2) dropped because of timeout 10657 keepalive timeouts 0 keepalive probes sent 10657 connections dropped by keepalive 39113689 correct ACK header predictions 121244352 correct data packet header predictions 698461 syncache entries added 21576 retransmitted 14546 dupsyn 0 dropped 694088 completed 0 bucket overflow 0 cache overflow 1046 reset 3338 stale 135 aborted 0 badack 6 unreach 0 zone failures 698461 cookies sent 232 cookies received 173007 hostcache entries added 0 bucket overflow 285122 SACK recovery episodes 584154 segment rexmits in SACK recovery episodes 730380132 byte rexmits in SACK recovery episodes 3053612 SACK options (SACK blocks) received 6689960 SACK options (SACK blocks) sent 0 SACK scoreboard overflow 8236 packets with ECN CE bit set 26367032 packets with ECN ECT(0) bit set 56 packets with ECN ECT(1) bit set 312220 successful ECN handshakes 34178 times ECN reduced the congestion window My sysctls are roughly equivalent to these: = http://serverfault.com/questions/64356/freebsd-performance-tuning-sysctls-= loader-conf-kernel Any hints? Do I need to provide more info? Julian= --Apple-Mail=_8979D47F-1B0A-4AE9-B7B7-BE2295364A9D Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) iQIcBAEBAgAGBQJPMDo1AAoJELZlJJDYKNKtv3MP/Ra0Ho113SwVHn5NcMgX3o11 x7i9bLqTzy5LoXK+PDE1iyjqGLGB4HzoxTTKvzcnGh9qBQYLdrkiF726Lvl7Ykgj T32rT9I2o8igztVV9vhcu5BDzHc6M1XVAekv4JNfxwZlL6rhibCC7SkdB6/JyqEB RUI2SFBOX6qiA8vs2VzFE8ywTkmzDIWNlU+u8BNQT9hezEwh0ZqMPMt2IWA4dNYR 0cKdqwco67+sn+pVsMztdI4cWElE6rHlZ0eoIJb/BcX2evxv3/lGSLCxL/R+xiwM zelPz9WqHNxpQ5h7a45oe1Z7DCpE+XO2t4vlwZyLSYV75hTlKLGWWVaFsQ6PWDrs Mr7YWqjYJH5tVzzxck9zBPHAw5uTtaO9TEtAxoeGrfjlfebHuOv6dAW49Jr2BcQZ mkdSJvFnW7NFu7ng8cPyJoaT0+N89S4C7vYIKyNZECiVIr8XsklczKbvb3Vym8y0 Ho9irRACNxR7DwY7Xkxa1SrHqxweNdt5G/vF7Wtfoo6Z4WDWfKKMXrSygebVDOlG 83VS9pA/xIrK0emkAUKi9TOh+tpofgbGV6x/AqANoi8aI398dfRawZJQ+lJObiHF E0Vc/KV3jxHvU/Uf0lPooziXls1/GZtMICF55vbkdIAwLytdvZ3kDLkbsQC6gAQk d19oOTlyNYLGKH3Cf0cy =luSy -----END PGP SIGNATURE----- --Apple-Mail=_8979D47F-1B0A-4AE9-B7B7-BE2295364A9D-- From owner-freebsd-performance@FreeBSD.ORG Mon Feb 6 22:55:43 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D7463106566B for ; Mon, 6 Feb 2012 22:55:43 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 9D7BF8FC13 for ; Mon, 6 Feb 2012 22:55:43 +0000 (UTC) Received: by iaeo4 with SMTP id o4so13670426iae.13 for ; Mon, 06 Feb 2012 14:55:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=sKExvNHWqM3p4cQomiwdklFyi7CoveVxjJgnNTgpT7g=; b=Wq4JDuG4btq+dKZ3/xedc/vdBLaiBxXDJ+RICaQEoPR4tDJebhUmIgOUzjTjCKDFP4 SRA2IJQXIC44R2/noGX5rJcmekRl8TNSV7me+kZ/OWxD9/ULBs6kjfQRut/qBD+9RNOQ QkzMySUKWv7csb24UqlLgujqoNY11aKbr2uH8= MIME-Version: 1.0 Received: by 10.50.6.194 with SMTP id d2mr23486344iga.24.1328567523000; Mon, 06 Feb 2012 14:32:03 -0800 (PST) Sender: kmacybsd@gmail.com Received: by 10.50.170.4 with HTTP; Mon, 6 Feb 2012 14:32:02 -0800 (PST) In-Reply-To: <7FE9C466-2459-4C0E-949A-7C0843B04BB4@gmail.com> References: <7FE9C466-2459-4C0E-949A-7C0843B04BB4@gmail.com> Date: Mon, 6 Feb 2012 23:32:02 +0100 X-Google-Sender-Auth: S6pRquX1gpEemJAUg43qWayIad0 Message-ID: From: "K. Macy" To: Julian Wissmann Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Cc: freebsd-performance@freebsd.org Subject: Re: Tor on FreeBSD Performance issues X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Feb 2012 22:55:43 -0000 On Mon, Feb 6, 2012 at 9:38 PM, Julian Wissmann wrote: > Hi, > > I'm an admin for a non-profit that runs Tor exit nodes, most of them on L= inux currently, but due to problems on our high bandwidth nodes, we decided= to migrate one of them to FreeBSD to do some testing. > I've been using FreeBSD for quite some years now, longer than Linux, so I= figured this would probably be a no-brainer, but turns out, that it isn't. > > On FreeBSD I currently manage to push 150-200Mbits with some heavy tuning= applied already, on Linux it is roughly 500Mbits. Tor tends to keep open a lot of connections but that is really very little bandwidth and you really shouldn't need to have polling on or have hz set that high. What does CPU utilization look like on these systems? I don't know if it is part of the problem, but TSO really isn't very useful with large numbers of connections, it is better suited to helping a single connection saturate an interface - could please you turn that off. Thanks > Therefor I'm wondering if I'm really running into some limitations here o= r if I'm actually doing something wrong. > > ifconfig bge0 > bge0: flags=3D8843 metric 0 mtu 1= 500 > =A0 =A0 =A0 =A0options=3Dc01db > > As you can see I've compiled polling and currently I'm running on kern.hz= =3D16000 as that's given me the best performance, so far. > > This is my netstat output for tcp on ipv4: > > netstat -s > tcp: > =A0 =A0 =A0 =A0265086752 packets sent > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A084255022 data packets (155791599035 bytes) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A02244601 data packets (2698410010 bytes) re= transmitted > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A073011 data packets unnecessarily retransmi= tted > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0684 resends initiated by MTU discovery > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0153151729 ack-only packets (0 delayed) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A00 URG only packets > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A019864 window probe packets > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A016854982 window update packets > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A08551532 control packets > =A0 =A0 =A0 =A0236720260 packets received > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A086600847 acks (for 155842386836 bytes) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A05062568 duplicate acks > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A076267 acks for unsent data > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0138258588 packets (150041170335 bytes) rec= eived in-sequence > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A01502804 completely duplicate packets (5622= 43206 bytes) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A044193 old duplicate packets > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A091821 packets with some dup. data (2369339= 1 bytes duped) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A06598536 out-of-order packets (7168950882 b= ytes) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A032074 packets (20848106 bytes) of data aft= er window > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A07806 window probes > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A01457705 window update packets > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0159860 packets received after close > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A05219 discarded for bad checksums > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A03 discarded for bad header offset fields > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A00 discarded because packet too short > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A01468 discarded due to memory problems > =A0 =A0 =A0 =A05665849 connection requests > =A0 =A0 =A0 =A0694088 connection accepts > =A0 =A0 =A0 =A00 bad connection attempts > =A0 =A0 =A0 =A0129 listen queue overflows > =A0 =A0 =A0 =A09308 ignored RSTs in the windows > =A0 =A0 =A0 =A03289250 connections established (including accepts) > =A0 =A0 =A0 =A06334698 connections closed (including 449721 drops) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0916420 connections updated cached RTT on c= lose > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0923786 connections updated cached RTT vari= ance on close > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0273103 connections updated cached ssthresh= on close > =A0 =A0 =A0 =A0354989 embryonic connections dropped > =A0 =A0 =A0 =A081015541 segments updated rtt (of 56772442 attempts) > =A0 =A0 =A0 =A09127304 retransmit timeouts > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A019875 connections dropped by rexmit timeou= t > =A0 =A0 =A0 =A021274 persist timeouts > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0215 connections dropped by persist timeout > =A0 =A0 =A0 =A04541 Connections (fin_wait_2) dropped because of timeout > =A0 =A0 =A0 =A010657 keepalive timeouts > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A00 keepalive probes sent > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A010657 connections dropped by keepalive > =A0 =A0 =A0 =A039113689 correct ACK header predictions > =A0 =A0 =A0 =A0121244352 correct data packet header predictions > =A0 =A0 =A0 =A0698461 syncache entries added > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A021576 retransmitted > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A014546 dupsyn > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A00 dropped > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0694088 completed > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A00 bucket overflow > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A00 cache overflow > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A01046 reset > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A03338 stale > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0135 aborted > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A00 badack > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A06 unreach > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A00 zone failures > =A0 =A0 =A0 =A0698461 cookies sent > =A0 =A0 =A0 =A0232 cookies received > =A0 =A0 =A0 =A0173007 hostcache entries added > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A00 bucket overflow > =A0 =A0 =A0 =A0285122 SACK recovery episodes > =A0 =A0 =A0 =A0584154 segment rexmits in SACK recovery episodes > =A0 =A0 =A0 =A0730380132 byte rexmits in SACK recovery episodes > =A0 =A0 =A0 =A03053612 SACK options (SACK blocks) received > =A0 =A0 =A0 =A06689960 SACK options (SACK blocks) sent > =A0 =A0 =A0 =A00 SACK scoreboard overflow > =A0 =A0 =A0 =A08236 packets with ECN CE bit set > =A0 =A0 =A0 =A026367032 packets with ECN ECT(0) bit set > =A0 =A0 =A0 =A056 packets with ECN ECT(1) bit set > =A0 =A0 =A0 =A0312220 successful ECN handshakes > =A0 =A0 =A0 =A034178 times ECN reduced the congestion window > > My sysctls are roughly equivalent to these: http://serverfault.com/questi= ons/64356/freebsd-performance-tuning-sysctls-loader-conf-kernel > > Any hints? > Do I need to provide more info? > > Julian --=20 =A0 =A0=93The real damage is done by those millions who want to 'get by.' The ordinary men who just want to be left in peace. Those who don=92t want their little lives disturbed by anything bigger than themselves. Those with no sides and no causes. Those who won=92t take measure of their own strength, for fear of antagonizing their own weakness. Those who don=92t like to make waves=97or enemies. =A0 =A0Those for whom freedom, honour, truth, and principles are only literature. Those who live small, love small, die small. It=92s the reductionist approach to life: if you keep it small, you=92ll keep it under control. If you don=92t make any noise, the bogeyman won=92t find you. =A0 =A0But it=92s all an illusion, because they die too, those people who roll up their spirits into tiny little balls so as to be safe. Safe?! >From what? Life is always on the edge of death; narrow streets lead to the same place as wide avenues, and a little candle burns itself out just like a flaming torch does. =A0 =A0I choose my own way to burn.=94 =A0 =A0Sophie Scholl From owner-freebsd-performance@FreeBSD.ORG Wed Feb 8 16:39:48 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E58BC106564A for ; Wed, 8 Feb 2012 16:39:48 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 77BB08FC0C for ; Wed, 8 Feb 2012 16:39:47 +0000 (UTC) Received: by werm13 with SMTP id m13so787377wer.13 for ; Wed, 08 Feb 2012 08:39:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=Y/n1jgF0F8hjXl8bFrcS+VY0xl5SwLWKejIrtWXNDYk=; b=jXNacdBxfVtWctHpfmhnPaFkVQ0Bu1nT6GbHHbJfs+I2trvAWW6880l+4ABMgOj84O tNrosFmwvs/ARLdFYYX/BnMFk+/FtkpfPMZcf2FL04m9Hetoz1itp+oqLREGixh0GLG8 AiaxWBsgZKDEmZgSfB1bQ/2g6JPQ+iDcwaRuE= MIME-Version: 1.0 Received: by 10.216.135.76 with SMTP id t54mr10983205wei.14.1328719187172; Wed, 08 Feb 2012 08:39:47 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.216.175.136 with HTTP; Wed, 8 Feb 2012 08:39:47 -0800 (PST) In-Reply-To: References: <7FE9C466-2459-4C0E-949A-7C0843B04BB4@gmail.com> Date: Wed, 8 Feb 2012 08:39:47 -0800 X-Google-Sender-Auth: sE4FwYnLP7shKxi7eto_HWjNa-0 Message-ID: From: Adrian Chadd To: "K. Macy" Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-performance@freebsd.org, Julian Wissmann Subject: Re: Tor on FreeBSD Performance issues X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Feb 2012 16:39:49 -0000 Can you verify that it's properly using kqueue, rather than poll? Adrian From owner-freebsd-performance@FreeBSD.ORG Wed Feb 8 17:34:05 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CC3C0106566C for ; Wed, 8 Feb 2012 17:34:05 +0000 (UTC) (envelope-from julianwissmann@gmail.com) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 5C26F8FC13 for ; Wed, 8 Feb 2012 17:34:04 +0000 (UTC) Received: by werm13 with SMTP id m13so850803wer.13 for ; Wed, 08 Feb 2012 09:34:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=subject:mime-version:content-type:from:in-reply-to:date:cc :message-id:references:to:x-mailer; bh=81IOZJKnHonVhJhgfiCaJPAkPxZjodeqlnzGeXchqW0=; b=Ii5K3EhZ3TBrsfWkJygZJjhquMkIbdCPLldiBWlis93M8ExJSAi9U7otJXo/zgmF17 aF0fYB2j53X521hnaVl2jAIjY34XKB7ewyTPh7aZdFq3xaapMI4aooLUgjb/BW4ThsCc tbjbrYpSvw2XKOc97Psq/xzj0ktzTouht8an4= Received: by 10.180.102.35 with SMTP id fl3mr24495745wib.22.1328722444186; Wed, 08 Feb 2012 09:34:04 -0800 (PST) Received: from [192.168.10.134] (g225192155.adsl.alicedsl.de. [92.225.192.155]) by mx.google.com with ESMTPS id ho4sm1300778wib.3.2012.02.08.09.34.02 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 08 Feb 2012 09:34:02 -0800 (PST) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: multipart/signed; boundary="Apple-Mail=_1B9ADEE8-3DDC-40ED-9E82-3FF47ADEC18F"; protocol="application/pgp-signature"; micalg=pgp-sha1 From: Julian Wissmann In-Reply-To: Date: Wed, 8 Feb 2012 18:33:53 +0100 Message-Id: References: <7FE9C466-2459-4C0E-949A-7C0843B04BB4@gmail.com> To: Adrian Chadd X-Mailer: Apple Mail (2.1251.1) Cc: freebsd-performance@freebsd.org Subject: Re: Tor on FreeBSD Performance issues X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Feb 2012 17:34:06 -0000 --Apple-Mail=_1B9ADEE8-3DDC-40ED-9E82-3FF47ADEC18F Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 1086 tor RET read 1568/0x620 1086 tor CALL clock_gettime(0xd,0x7fffffffd910) 1086 tor RET clock_gettime 0 1086 tor CALL = kevent(0x3,0x3e18000,0x2,0x3dbe000,0x400,0x7fffffffd920) 1086 tor GIO fd 3 wrote 64 bytes 1086 tor RET kevent 3 1086 tor CALL clock_gettime(0x4,0x7fffffffd940) 1086 tor RET clock_gettime 0 1086 tor CALL gettimeofday(0x7fffffffd930,0) 1086 tor RET gettimeofday 0 1086 tor CALL recvfrom(0xbc2,0x4822020,0x3e40,0,0,0) 1086 tor GIO fd 3010 read 4096 bytes As I understand kevent to be part of queue I go with yes here. What I = also take from it is a huge number of clock_gettime() and gettimeofday() = calls. Julian Am 08.02.2012 um 17:39 schrieb Adrian Chadd: > Can you verify that it's properly using kqueue, rather than poll? >=20 >=20 > Adrian --Apple-Mail=_1B9ADEE8-3DDC-40ED-9E82-3FF47ADEC18F Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) iQIcBAEBAgAGBQJPMrIHAAoJELZlJJDYKNKt5zYP/Rj/pCnSvgitPe1JL5EXhJkd AcQuLqaDNR+iBEFXUJBkrHH7aAdIVsLzz5L/ADYHTTBUaZxk1fGNWEchwk/dS/SI 2adPzQBAmhT5KXS+lQGczejuznc1zoBYAxyUsdLKXM6oV6jo2woMmX3k77OSahRR /OXbpzKzKo2SRNWCco9/GHFhYto+AvwATjYPfHuZGFAXVLO9D+XP+UlwonBoHYNr oEMrJFKhJBAnnqIfG2DiMwk5CnK/iwMbqpQ1hEGZZaDv4SxVR/iRa35ED8H/ttbR 8hyJ6VsKU1meHnzmA3MipAsCH+rx8f41+zSPNAqhuI9JU2CpRrDHHRvrB6ROug2z EU/LOUe/kT4R4rOrXb5uK+vR8JaS7P5JzulHaTnMVAFrGA14b38+WZ4ApTf4HRF0 X5V+4t5gF9fdsc5BsXX2/kxvyjtibYvLS/IKwU4YcLWWAGOdUblgI6mejMKvZ5wV BG3P7/d1pz4edt6ech9A8la+sCCSvo+VCkIvippDPu+sg8iSz/fkFKoESAi7S9gB V/dLR16BRtLqvkLuwfDl3lo5UL5l1pr9cvy3MW9qSasTkakJdJTY9vGvKdzG3Xg0 8bxD8E7C3KkKhCi7jRx/J43bKhK47uhuMV/pJ28sm+GJU9b6Fve7nsvS0Ro7xSdm wSWnCkZXy8E2ujimCNru =uNbY -----END PGP SIGNATURE----- --Apple-Mail=_1B9ADEE8-3DDC-40ED-9E82-3FF47ADEC18F-- From owner-freebsd-performance@FreeBSD.ORG Fri Feb 10 10:56:27 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F363C106564A for ; Fri, 10 Feb 2012 10:56:26 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 6E49F8FC14 for ; Fri, 10 Feb 2012 10:56:26 +0000 (UTC) Received: from outgoing.leidinger.net (p4FC41EEA.dip.t-dialin.net [79.196.30.234]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 6F3E0844720; Fri, 10 Feb 2012 11:40:19 +0100 (CET) Received: from webmail.leidinger.net (webmail.Leidinger.net [IPv6:fd73:10c7:2053:1::3:102]) by outgoing.leidinger.net (Postfix) with ESMTPS id A94682C4D; Fri, 10 Feb 2012 11:40:16 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=leidinger.net; s=outgoing-alex; t=1328870416; bh=04pfdGFOGqEsrnxAmHTPLcdv+owcE9/t8Jk9+Y6Uv60=; h=Date:Message-ID:From:To:Cc:Subject:References:In-Reply-To: Content-Type:MIME-Version; b=V6Gtf1lx52uHnW0afbW/2rIWVSCyNrK1nALCxf0wlC27t8L4r48B0xK4gBbDHaqwn dUza6XKDHsBOVh8x/evVi2zD4iSQH+o+wGwoGHo8LO2M6uSC/HlMeRBWbcDv9PFi+G B5ovtlBQb8KUj6zvIJiayUZu0wT3RCXaOliXbIbaUZ1he1JPaeKRtX2PvLueBE/2su p6BUWi6cpfRJUWCfGkOzkhsJ7uWq61BOxpxLrmg6d1knKwZ47+bGy6lg8BJCZ9y2p8 SVH0o6d1m9PGVhwI4MPfhCPGNPqYer2fQZ9pVk9u+eag2yPODBbqUQL4M2yj4KpR1r osoWMLCEsvt6A== Received: (from www@localhost) by webmail.leidinger.net (8.14.5/8.14.4/Submit) id q1AAeG6N085477; Fri, 10 Feb 2012 11:40:16 +0100 (CET) (envelope-from Alexander@Leidinger.net) X-Authentication-Warning: webmail.leidinger.net: www set sender to Alexander@Leidinger.net using -f Received: from 85.94.224.19 ([85.94.224.19]) by webmail.leidinger.net (Horde Framework) with HTTP; Fri, 10 Feb 2012 11:40:16 +0100 Date: Fri, 10 Feb 2012 11:40:16 +0100 Message-ID: <20120210114016.Horde.NRmcWJjmRSRPNPQQLsmBORA@webmail.leidinger.net> From: Alexander Leidinger To: Julian Wissmann References: <7FE9C466-2459-4C0E-949A-7C0843B04BB4@gmail.com> In-Reply-To: User-Agent: Internet Messaging Program (IMP) H4 (5.0.18) Content-Type: text/plain; charset=ISO-8859-1; format=flowed; DelSp=Yes MIME-Version: 1.0 Content-Disposition: inline X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 6F3E0844720.A1400 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-0.614, required 6, autolearn=disabled, AWL -0.50, DKIM_SIGNED 0.10, DKIM_VALID -0.10, DKIM_VALID_AU -0.10, T_RP_MATCHES_RCVD -0.01) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1329475220.28695@tPTA8c6JA7/Nj07axfVE3A X-EBL-Spam-Status: No Cc: Adrian Chadd , freebsd-performance@freebsd.org Subject: Re: Tor on FreeBSD Performance issues X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Feb 2012 10:56:27 -0000 Quoting Julian Wissmann (from Wed, 8 Feb 2012 18:33:53 +0100): > 1086 tor RET read 1568/0x620 > 1086 tor CALL clock_gettime(0xd,0x7fffffffd910) > 1086 tor RET clock_gettime 0 > 1086 tor CALL > kevent(0x3,0x3e18000,0x2,0x3dbe000,0x400,0x7fffffffd920) > 1086 tor GIO fd 3 wrote 64 bytes > > 1086 tor RET kevent 3 > 1086 tor CALL clock_gettime(0x4,0x7fffffffd940) > 1086 tor RET clock_gettime 0 > 1086 tor CALL gettimeofday(0x7fffffffd930,0) > 1086 tor RET gettimeofday 0 > 1086 tor CALL recvfrom(0xbc2,0x4822020,0x3e40,0,0,0) > 1086 tor GIO fd 3010 read 4096 bytes > > As I understand kevent to be part of queue I go with yes here. What > I also take from it is a huge number of clock_gettime() and > gettimeofday() calls. And there you have probably the cause of the slowdown. The clock subsystem in FreeBSD is working at a higher prcession than what you get in FreeBSD. If possible, try to trim down the number of calls, or use a less precise (and faster) clocksource, e.g. one of the _FAST ones (see http://www.freebsd.org/cgi/man.cgi?clock_gettime). Bye, Alexander. -- Corruption is not the No. 1 priority of the Police Commissioner. His job is to enforce the law and fight crime. -- P.B.A. President E. J. Kiernan http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137