From owner-svn-src-head@freebsd.org Thu Sep 17 17:49:27 2020 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7573D3E8493; Thu, 17 Sep 2020 17:49:27 +0000 (UTC) (envelope-from olivier@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BskxM2W5Nz4M34; Thu, 17 Sep 2020 17:49:27 +0000 (UTC) (envelope-from olivier@freebsd.org) Received: from mail-qt1-f175.google.com (mail-qt1-f175.google.com [209.85.160.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) (Authenticated sender: olivier/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 340B027908; Thu, 17 Sep 2020 17:49:27 +0000 (UTC) (envelope-from olivier@freebsd.org) Received: by mail-qt1-f175.google.com with SMTP id c18so2592261qtw.5; Thu, 17 Sep 2020 10:49:27 -0700 (PDT) X-Gm-Message-State: AOAM531etO6xZsbLzio/me4A8T2MlPlPxbkFKC1ss4SnMIAmBLE+K/H/ Myd7aSCBwLcsjCIm5NegKH0TgSWR+2YsQ3yYzds= X-Google-Smtp-Source: ABdhPJy/TNTq+ga6Z1SF7Zcn7xAs618XjQjEuBg+KaDEQhMonmQ7EbRXlopIctzRqejQlHrTeskMS/a/PWna7PKb4Bs= X-Received: by 2002:ac8:24f1:: with SMTP id t46mr16074893qtt.93.1600364965960; Thu, 17 Sep 2020 10:49:25 -0700 (PDT) MIME-Version: 1.0 References: <202009081036.088AaCk8085096@repo.freebsd.org> In-Reply-To: <202009081036.088AaCk8085096@repo.freebsd.org> From: =?UTF-8?Q?Olivier_Cochard=2DLabb=C3=A9?= Date: Thu, 17 Sep 2020 19:49:14 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: svn commit: r365449 - head/sbin/rcorder To: "Andrey V. Elsukov" Cc: src-committers , svn-src-all , svn-src-head Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Sep 2020 17:49:27 -0000 On Tue, Sep 8, 2020 at 12:36 PM Andrey V. Elsukov wrote: > Author: ae > Date: Tue Sep 8 10:36:11 2020 > New Revision: 365449 > URL: https://svnweb.freebsd.org/changeset/base/365449 > > Log: > Add a few features to rcorder: > > > Hi Andrey, I've spent some time bisecting an IPSec gateway performance regression on -head that points to this commit. So my bench uses a simple static IPSec aes-gcm-16 setup, and their results vary a lot between those 2 revisions: - r365448: Estimated Equilibrium Ethernet throughput= 1413 Mb/s (maximum value seen: 1419 Mb/s) - r365449: Estimated Equilibrium Ethernet throughput= 469 Mb/s (maximum value seen: 469 Mb/s) How could a modification to the rcoder be the source cause of that ? My rc.conf contains this: kld_list="aesni" ipsec_enable="YES" So My first thought was that the aesni module wasn't loaded anymore: Before upgrade: [root@hp]~# uname -a FreeBSD hp 13.0-CURRENT FreeBSD 13.0-CURRENT #15 r365448M: Thu Sep 17 18:17:58 CEST 2020 olivier@lame4.bsdrp.net:/usr/src/amd64.amd64/sys/amd64 amd64 [root@hp]~# kldstat Id Refs Address Size Name 1 11 0xffffffff80200000 1ee58b0 kernel 2 1 0xffffffff82319000 34c8 fdescfs.ko 3 1 0xffffffff8231d000 a240 aesni.ko 4 1 0xffffffff82328000 8c98 ioat.ko 5 1 0xffffffff82331000 e350 ipsec.ko Then after upgrade: [root@hp]~# uname -a FreeBSD hp 13.0-CURRENT FreeBSD 13.0-CURRENT #14 r365449M: Thu Sep 17 17:01:35 CEST 2020 olivier@lame4.bsdrp.net:/usr/src/amd64.amd64/sys/amd64 amd64 [root@hp]~# kldstat Id Refs Address Size Name 1 11 0xffffffff80200000 1ee58b0 kernel 2 1 0xffffffff82319000 34c8 fdescfs.ko 3 1 0xffffffff8231d000 e350 ipsec.ko 4 1 0xffffffff8232c000 a240 aesni.ko 5 1 0xffffffff82337000 8c98 ioat.ko => aesni.ko is correctly loaded, so it is not the problem, but notice the order of the kernel modules that have changed. Could be this the source of the problem ? Let's try: [root@hp]~# service ipsec stop Clearing ipsec manual keys/policies. [root@hp]~# kld kldconfig kldload kldstat kldunload kldxref [root@hp]~# kldunload ioat [root@hp]~# kldunload aesni [root@hp]~# kldunload ipsec [root@hp]~# kldload aesni [root@hp]~# kldload ipsec [root@hp]~# service ipsec start Installing ipsec manual keys/policies. [root@hp]~# kldstat Id Refs Address Size Name 1 11 0xffffffff80200000 1ee58b0 kernel 2 1 0xffffffff82319000 34c8 fdescfs.ko 5 1 0xffffffff82337000 8c98 ioat.ko 6 1 0xffffffff8231d000 a240 aesni.ko 7 1 0xffffffff82328000 e350 ipsec.ko And after that the IPSec bench results are back to their previous value :-) So rcorder needs to load aesni before ipsec. Regards, Olivier