From owner-freebsd-net@freebsd.org Mon Nov 12 03:45:03 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 572EA112F417 for ; Mon, 12 Nov 2018 03:45:03 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:d12:604::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id EEFC17BFB1 for ; Mon, 12 Nov 2018 03:44:51 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id wAC3iiPO084242 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 12 Nov 2018 04:44:44 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: wolfgang@lyxys.ka.sub.org Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id wAC3ihbf005082 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Mon, 12 Nov 2018 10:44:43 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: epair(4) and net.route.netisr_maxqlen To: Wolfgang Zenker , freebsd-net@freebsd.org References: <20181111232320.GA87730@lyxys.ka.sub.org> From: Eugene Grosbein Message-ID: <5afe5838-d2df-009d-af85-99e793f1927c@grosbein.net> Date: Mon, 12 Nov 2018 10:44:35 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <20181111232320.GA87730@lyxys.ka.sub.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: EEFC17BFB1 X-Spamd-Result: default: False [-5.53 / 200.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; MX_INVALID(0.50)[greylisted]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; R_SPF_PERMFAIL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-1.00)[-0.996,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; IP_SCORE(-2.94)[ip: (-8.76), ipnet: 2a01:4f8::/29(-2.98), asn: 24940(-2.93), country: DE(-0.01)]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Nov 2018 03:45:03 -0000 12.11.2018 6:23, Wolfgang Zenker wrote: > on a jail with quite a lot of somewhat bursty network traffic I get > warnings from netdata recently about packets being dropped because of > net.route.netisr_maxqlen being to small. Before I start setting this > value to some random value I'ld like to find out what it actually means. > A search for documentation turned up nothing; a look at the sources > found that it is used for the size of a "software interrupt queue" in > epair(4). But what does it mean? And does this give me enough > information to find a good value to set for this sysctl? netisr packet queues keep packets received by the interface and not yet processed by destined subsystem or userland application that may be short of CPU cycles or blocked for some reason. First, the system won't allow you to raise net.route.netisr_maxqlen over the limit net.isr.maxqlimit. The limit itself can be changed with /boot/loader.conf and reboot. Default value of limit is 10240. I generally raise the limit upto 102400 for hosts with heavy/bursty traffic. Note that actual increase of net.route.netisr_maxqlen somewhat increases usage of kernel memory and that could be important for 32 bit kernel and/or system with very low amount of RAM. There may be several netisr packet queues in the system and raising net.route.netisr_maxqlen allows all of them to grow. epair(4) has its own setting net.link.epair.netisr_maxqlen that defaults to net.route.netisr_maxqlen if unset, so you may be start experimenting with net.link.epair.netisr_maxqlen first, instead of system global net.route.netisr_maxqlen. Don't set net.route.netisr_maxqlen to random value but double its current value and see if that would be enough. If not, double it again. If 4096 apears not enough, you should check your applications why they can't keep with incoming traffic rate.