From owner-freebsd-arch@freebsd.org Tue May 30 18:26:28 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EFD6BBD30E9 for ; Tue, 30 May 2017 18:26:28 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wr0-x232.google.com (mail-wr0-x232.google.com [IPv6:2a00:1450:400c:c0c::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8CA7374534 for ; Tue, 30 May 2017 18:26:28 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-wr0-x232.google.com with SMTP id j27so5424310wre.3 for ; Tue, 30 May 2017 11:26:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=h4PLh3mB1GIPT+AfW3Q2JNhcM0+Wzp24xu/4QyrzUYk=; b=mhnvJrpVdVApuaVU/54WDvvJFhGADyqGcZ+Al0Kux++v+VWbaf623gHcyzTvQmeshD Q6f6XKW4dvVOW1/6dDiXAU6MimASvxAh5Tf+CA5NxZcMsJcwH4viB2GIdBZ/fpy/njwa i6P/Ci74KA8D2qRaYHhPjn9QTQa+QhMOhFretGywoz5Bs7Buu9lizw19HowZ1E+YYhzI EAN8ZFh1D+DM4HLEAeT8h1t4YmjEr97kp33RFaPCnSUx63iL710Nb/Dg2L3/TZ8+z21z YtHAP0pZMtvrIpMK+NPZ1G4dqbAMfHfTx0DuRQg0UzmoGFePyHYe7IT/QZG3pZg+39cf GN6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=h4PLh3mB1GIPT+AfW3Q2JNhcM0+Wzp24xu/4QyrzUYk=; b=tcVF9VY9yDs20PNfa83mGGoJdWeEV+eHK/ROgoQFb2bivr2XNgZ8t03E4vD+L57Z8b FT8o3ht0GTEuvs+OqRkJ3vW+1jmJnS0pEnUOHlXSp/b2o2LC//H4Ao2V11KsGAOYlvsr ejsSmDjqpkxTJChOltLccWWaRNDWq2i54V8jFThQ+3fK0EMuRVAbj/BnNydh5J7UNPnZ vnZXY3whwaPJcuqr8mhT3N7Lw9dE23+GoXxB/kuBELJ8kwf3evDxoKn7YYP2HVG/tk5F Huc5NUiDCDAW0lekvrXPnGnyn9d7vfA4UK1SxchFEWnjqJgb+2KAa0MwxQSQUiO7pUiO /t2Q== X-Gm-Message-State: AODbwcBa4X8Ff310Dj57+0GTjQTYHauDKCbmwpTwXCLYpsiulUS30dL8 hDAckTKNgxXdKNqjfgvAw0luj1LC3Msg X-Received: by 10.223.134.80 with SMTP id 16mr18272180wrw.62.1496168786997; Tue, 30 May 2017 11:26:26 -0700 (PDT) MIME-Version: 1.0 Sender: adrian.chadd@gmail.com Received: by 10.28.193.134 with HTTP; Tue, 30 May 2017 11:26:26 -0700 (PDT) In-Reply-To: <608664209.55736023.1496155561181.JavaMail.zimbra@stormshield.eu> References: <1914359731.54283525.1495178031163.JavaMail.zimbra@stormshield.eu> <816581118.55670987.1496141816904.JavaMail.zimbra@stormshield.eu> <608664209.55736023.1496155561181.JavaMail.zimbra@stormshield.eu> From: Adrian Chadd Date: Tue, 30 May 2017 11:26:26 -0700 X-Google-Sender-Auth: xUCwwL6UACISWRMREXYcr77W0G4 Message-ID: Subject: Re: numa and taskqueues To: Emeric POUPON Cc: freebsd-arch Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 May 2017 18:26:29 -0000 On 30 May 2017 at 07:46, Emeric POUPON wrote: > Hi, > >> >>> 2/ about https://reviews.freebsd.org/D10680, I think it would be great to have >>> this commited as a first step. >>> Since it seems to be stuck, maybe I can add more people on this. Any suggestion? >> >> Well, what's with the ~ 8% performance decrease? Do you know what's >> going on? For a "we're parallelising IPSEC operations", seeing it get >> slower with more flows is a bit concerning. >> >> Thanks, >> > > Actually, there is a performance boost only when few flows are involved. > That's why this is not activated by default and a sysctl is here to enable the feature. > > To sum up, the more different flows you process (both ciphered and unciphered), the more network queues are hit and the more CPU units are triggered from ipsec. > In this case, we indeed notice a loss, certainly due to the extra queing/reordering performed. Can you dig into that a bit more? Do you know exactly what's going on? eg, is it a "lock contention" problem? Is it a "stuff is context switching, thus latency" problem? etc, etc. -adrian