From owner-svn-src-head@freebsd.org Wed Sep 25 12:03:18 2019 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CE02F1256B8; Wed, 25 Sep 2019 12:03:18 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-ot1-x343.google.com (mail-ot1-x343.google.com [IPv6:2607:f8b0:4864:20::343]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46dcCB0c24z3KDg; Wed, 25 Sep 2019 12:03:18 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: by mail-ot1-x343.google.com with SMTP id g13so4529424otp.8; Wed, 25 Sep 2019 05:03:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:cc:references:from:autocrypt:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=I8ZI+/lK0mYcd2kISTpB3smaN2n33gToXJ0I7l/6/KM=; b=GaxRlweJnqnQVjrSxYzZ0OdHgI06BJ0NzLKNvdrSz2uB0Z8L2Tx7L7COJxuUTn5f/+ JZuBi9FzUgU5wYh8qWkmPHsIlcTYJaUZxZCKSrmfUX6HHDFNfwYSccxI8D043p1+U1o7 ClwmTZPB/idlSS8x3od6mihzr15LnarQyzgFFJLXdlZ9faodNM47c7byFdutULsrNFMm XdaHZUDJMUgZqx1NO5TIdmaozYsJyhx1E2DBwM6vnQsY6iXlC6B0ok5yDgVFANvQMCJq Uvc64gCbGb1iSecLvtxoyQOr6IKaUL08c4HrURTlrpneYj9QfD3L1Y6CA7m400Gro3Ly q2xw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:cc:references:from:autocrypt :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=I8ZI+/lK0mYcd2kISTpB3smaN2n33gToXJ0I7l/6/KM=; b=CjAY0njy9te51XYLd0MUioO93LQEtXieqmxG7N6AYSN6Q7X3uMUT1nkm8CUUQ9Aido gnSM5zN9w76yCEdhoNlDwayv/36IUx+8+9lL/rbp5t2ucao2xFtwP7H9PuZwyKGD4wHq MMGq1F45uW3/OONhghlbYIuiLqzK2LImw2iHuiijL5q9yYYXIBnnYN3OtsiQHaEPiicH RuS41LcnnlxuKyDJvJiI2wDWBytxrBiQJ8mblHkSyCVUV12dzpSoaKUg63QVCj2GY8JO Sw9IrCmFQysKQsbJye/8CckA87gxnhoPRMPASd8n85AT20Flp/2/KSnSiOflGo5idser ZS0Q== X-Gm-Message-State: APjAAAVRtbPx3s7DexSiMTlyNstMQMQ0BXrGKlCFhNOkZ1m8mefIARaA S5y1U8Q2Lsu44t3Aln3jQgem7qpq X-Google-Smtp-Source: APXvYqz4I/N5vuVRHtB0SKpns3hy7wLx9bKOlfv9cEF5geINweqEkiSqskYhLqa05+sLJptO0lkThQ== X-Received: by 2002:a9d:3b02:: with SMTP id z2mr5433020otb.71.1569412996413; Wed, 25 Sep 2019 05:03:16 -0700 (PDT) Received: from spectre.mavhome.dp.ua (104-55-12-234.lightspeed.knvltn.sbcglobal.net. [104.55.12.234]) by smtp.gmail.com with ESMTPSA id w13sm1376969oih.54.2019.09.25.05.03.14 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 25 Sep 2019 05:03:15 -0700 (PDT) Sender: Alexander Motin Subject: Re: svn commit: r352658 - head/sys/kern To: Peter Holm Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org References: <201909242001.x8OK1KAV099968@repo.freebsd.org> <20190925051317.GA77827@x8.osted.lan> From: Alexander Motin Autocrypt: addr=mav@FreeBSD.org; prefer-encrypt=mutual; keydata= mQENBFOzxAwBCADkPrax0pI2W/ig0CK9nRJJwsHitAGEZ2HZiFEuti+6/4UVxj81yr4ak/4g 9bKUyC7rMEAp/ZHNhd+MFCPAAcHPvtovnfykqE/vuosCS3wlSLloix2iKVLks0CwbLHGAyne 46lTQW74Xl/33c3W1Z6d8jD9gVFT/xaVzZ0U9xdzOmsYAZaAj4ki0tuxO9F7L+ct9grRe7iP g8t9hai7BL4ee3VRwk2JXnKb7UvBiVITKYWKz1jRvZIrjPokgEcCLOSlv7x/1kjuFnj3xWZU 7HSFFT8J93epBbrSSCsYsppIk2fZH41kaaFXsMQfTPH8wkeM6qwrvOh4HiQM08R+9tThABEB AAG0IUFsZXhhbmRlciBNb3RpbiA8bWF2QEZyZWVCU0Qub3JnPokBVwQTAQoAQQIbAwULCQgH AwUVCgkICwUWAwIBAAIeAQIXgAIZARYhBOmM88TmnMPNDledVYMYw5VbqyJ/BQJZYMKuBQkN McyiAAoJEIMYw5VbqyJ/tuUIAOG3ONOSNYqjK4eTZ1TVh9jdUBAhWk5nhDFnODN49Wj0AbYm 7aIqy8O1hnCDSZG5LttjSAo3UfXJZDKQM0BLb0gpRMBnAYqO6tdolLNqAbPGJBnGoPjsh24y 6KcbDaNnis+lD4GwPXwQM+92wZGhCUFElPV9NciZGVS65TNIgk7X+yEjjhD1MSWKKijZ1r9Z zIt4OzUTxxNOvzdlABZS88nNRdJkatOQJPmFdd1mpP6UzTNCiLUo1pIqOEtJgvVVDYq5WHY6 tciWWYdmZG/tIBexJmv2mV2OLVjXR6ZeKmntVH14H72/wRHJuYHQC+r5SVRcWWayrThsY6jZ Yr4+raS5AQ0EU7PEDAEIAOZgWf2cJIu+58IzP2dkXE/urj3tr4OqrB/yHGWUf71Lz6D0Fi6Z AXgDtmcFLGPfMyWuLAvSM+xmoguk7zC4hRBYvQycmIhuqBq1jO1Wp/Z+lpoPM/1cDYLn8Flv mI/c40MhUZh345DA4jYWWaZNjQHUWVQ1fPf595vdVVMPT/abE8E5DaF6fSkRmqFTmfYRkfbt 3ytU8NdUapDcJVY7cEP2nJBVNZPnOIObR/ZIgSxjjrG5o34yXoqeup8JvwEv+/NylzzuyXEZ R1EdEIzQ/a1nh/0j4NXtzZEqKW4aTWlmSqb6wN8jh1OSOOqkYsfnE3nfxcZbxi4IRoNQYlm5 9R8AEQEAAYkBPAQYAQoAJgIbDBYhBOmM88TmnMPNDledVYMYw5VbqyJ/BQJZYMLYBQkNMczM AAoJEIMYw5VbqyJ/TqgH/RQHClkvecE0262lwKoP/m0Mh4I5TLRgoJJn8S7G1BnqohYJkiLq A6xe6urGD7OqdNAl12UbrjWbdJV+zvea3vJoM4MZuYiYrGaXWxzFXqWJcPwMU9sAh8MRghHu uC5vgPb45Tnftw9/+n0i8GfVhQhOqepUGdQg4NPcXviSkoAvig6pp9Lcxisn0groUQKt15Gc sS9YcQWg3j9Hnipc6Mu416HX98Fb113NHJqc2geTHLkRyuBFOoyIqB6N9GKjzOAIzxxsVdl9 TevwGsrp4M4/RFzWbSgsbOnbE7454lmuVZGfReEjnUm8RHp9Q2UWKXlp3exlZjvOp/uVEpCg lz65AQ0EU7PEDAEIAOZgWf2cJIu+58IzP2dkXE/urj3tr4OqrB/yHGWUf71Lz6D0Fi6ZAXgD tmcFLGPfMyWuLAvSM+xmoguk7zC4hRBYvQycmIhuqBq1jO1Wp/Z+lpoPM/1cDYLn8FlvmI/c 40MhUZh345DA4jYWWaZNjQHUWVQ1fPf595vdVVMPT/abE8E5DaF6fSkRmqFTmfYRkfbt3ytU 8NdUapDcJVY7cEP2nJBVNZPnOIObR/ZIgSxjjrG5o34yXoqeup8JvwEv+/NylzzuyXEZR1Ed EIzQ/a1nh/0j4NXtzZEqKW4aTWlmSqb6wN8jh1OSOOqkYsfnE3nfxcZbxi4IRoNQYlm59R8A EQEAAYkBPAQYAQoAJgIbDBYhBOmM88TmnMPNDledVYMYw5VbqyJ/BQJZYMLYBQkNMczMAAoJ EIMYw5VbqyJ/TqgH/RQHClkvecE0262lwKoP/m0Mh4I5TLRgoJJn8S7G1BnqohYJkiLqA6xe 6urGD7OqdNAl12UbrjWbdJV+zvea3vJoM4MZuYiYrGaXWxzFXqWJcPwMU9sAh8MRghHuuC5v gPb45Tnftw9/+n0i8GfVhQhOqepUGdQg4NPcXviSkoAvig6pp9Lcxisn0groUQKt15GcsS9Y cQWg3j9Hnipc6Mu416HX98Fb113NHJqc2geTHLkRyuBFOoyIqB6N9GKjzOAIzxxsVdl9Tevw Gsrp4M4/RFzWbSgsbOnbE7454lmuVZGfReEjnUm8RHp9Q2UWKXlp3exlZjvOp/uVEpCglz4= Message-ID: Date: Wed, 25 Sep 2019 08:03:13 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.0 MIME-Version: 1.0 In-Reply-To: <20190925051317.GA77827@x8.osted.lan> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46dcCB0c24z3KDg X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=GaxRlweJ; dmarc=none; spf=pass (mx1.freebsd.org: domain of mavbsd@gmail.com designates 2607:f8b0:4864:20::343 as permitted sender) smtp.mailfrom=mavbsd@gmail.com X-Spamd-Result: default: False [-2.56 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[FreeBSD.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-0.37)[ip: (3.02), ipnet: 2607:f8b0::/32(-2.61), asn: 15169(-2.18), country: US(-0.05)]; DKIM_TRACE(0.00)[gmail.com:+]; RCVD_IN_DNSWL_NONE(0.00)[3.4.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; FORGED_SENDER(0.30)[mav@FreeBSD.org,mavbsd@gmail.com]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mav@FreeBSD.org,mavbsd@gmail.com]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Sep 2019 12:03:18 -0000 On 25.09.2019 01:13, Peter Holm wrote: > On Tue, Sep 24, 2019 at 08:01:20PM +0000, Alexander Motin wrote: >> Author: mav >> Date: Tue Sep 24 20:01:20 2019 >> New Revision: 352658 >> URL: https://svnweb.freebsd.org/changeset/base/352658 >> >> Log: >> Fix/improve interrupt threads scheduling. >> >> Doing some tests with very high interrupt rates I've noticed that one of >> conditions I added in r232207 to make interrupt threads in most cases >> run on local CPU never worked as expected (worked only if previous time >> it was executed on some other CPU, that is quite opposite). It caused >> additional CPU usage to run full CPU search and could schedule interrupt >> threads to some other CPU. >> >> This patch removes that code and instead reuses existing non-interrupt >> code path with some tweaks for interrupt case: >> - On SMT systems, if current thread is idle, don't look on other threads. >> Even if they are busy, it may take more time to do fill search and bounce >> the interrupt thread to other core then execute it locally, even sharing >> CPU resources. It is other threads should migrate, not bound interrupts. >> - Try hard to keep interrupt threads within LLC of their original CPU. >> This improves scheduling cost and supposedly cache and memory locality. >> >> On a test system with 72 threads doing 2.2M IOPS to NVMe this saves few >> percents of CPU time while adding few percents to IOPS. >> >> MFC after: 1 month >> Sponsored by: iXsystems, Inc. >> >> Modified: >> head/sys/kern/sched_ule.c >> >> Modified: head/sys/kern/sched_ule.c >> ============================================================================== >> --- head/sys/kern/sched_ule.c Tue Sep 24 18:18:11 2019 (r352657) >> +++ head/sys/kern/sched_ule.c Tue Sep 24 20:01:20 2019 (r352658) >> @@ -1251,7 +1251,7 @@ sched_pickcpu(struct thread *td, int flags) > > Could this be yours? > > FreeBSD/SMP: Multiprocessor System Detected: 24 CPUs > FreeBSD/SMP: 2 package(s) x 6 core(s) x 2 hardware threads > random: unblocking device. > Firmware Warning (ACPI): Invalid length for FADT/Pm1aControlBlock: 32, using default 16 (20190703/tbfadt-850) > ioapic0 irqs 0-23 > ioapic1 irqs 24-47 > ioapic2 irqs 48-71 > Launching APs: 13 6 18 17 5 9 8 19 7 10 1 11 2 12 14 15 20 4 21 16 22 3 23 > panic: sched_pickcpu: Failed to find a cpu. > cpuid = 0 > time = 1 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xffffffff8254a830 > vpanic() at vpanic+0x19d/frame 0xffffffff8254a880 > panic() at panic+0x43/frame 0xffffffff8254a8e0 > sched_pickcpu() at sched_pickcpu+0x4c1/frame 0xffffffff8254a990 > sched_add() at sched_add+0x6e/frame 0xffffffff8254a9d0 > gtaskqueue_start_threads() at gtaskqueue_start_threads+0x124/frame 0xffffffff8254aa70 > taskqgroup_cpu_create() at taskqgroup_cpu_create+0x135/frame 0xffffffff8254aab0 > taskqgroup_adjust() at taskqgroup_adjust+0x1ad/frame 0xffffffff8254ab20 > mi_startup() at mi_startup+0x210/frame 0xffffffff8254ab70 > btext() at btext+0x2c > KDB: enter: panic > [ thread pid 0 tid 100000 ] > Stopped at kdb_enter+0x3b: movq $0,kdb_why > db> Should be fixed by r352677. Sorry. -- Alexander Motin