From owner-freebsd-net@FreeBSD.ORG Thu Feb 2 08:59:37 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 838491065672; Thu, 2 Feb 2012 08:59:37 +0000 (UTC) (envelope-from kes-kes@yandex.ru) Received: from forward8.mail.yandex.net (forward8.mail.yandex.net [IPv6:2a02:6b8:0:202::3]) by mx1.freebsd.org (Postfix) with ESMTP id 5C2388FC13; Thu, 2 Feb 2012 08:59:36 +0000 (UTC) Received: from smtp8.mail.yandex.net (smtp8.mail.yandex.net [77.88.61.54]) by forward8.mail.yandex.net (Yandex) with ESMTP id A57C0F61FC7; Thu, 2 Feb 2012 12:59:34 +0400 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1328173174; bh=Z66qFfSUmvG8yBgLP67s6h0ctmCW0H/VhodzloIW89s=; h=Date:From:Reply-To:Message-ID:To:CC:Subject:In-Reply-To: References:MIME-Version:Content-Type:Content-Transfer-Encoding; b=udMCSdvw4NN4uPt3RyonmS6SXjyDzRfAg7+OPcPm/GLYdUr3WvWIsj2hp8FlbsPx0 jida+4YFe96C+oSaHYg7xj6CO3pDGpbtBRuD81Y/kDd37/W6EtHUcbg8sJ+q88oZ5C 2r9FSoGV+qAfxCilH29Sedkm6bkeNsf6V85ES6RY= Received: from smtp8.mail.yandex.net (localhost [127.0.0.1]) by smtp8.mail.yandex.net (Yandex) with ESMTP id 6F9D41B604F8; Thu, 2 Feb 2012 12:59:34 +0400 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1328173174; bh=Z66qFfSUmvG8yBgLP67s6h0ctmCW0H/VhodzloIW89s=; h=Date:From:Reply-To:Message-ID:To:CC:Subject:In-Reply-To: References:MIME-Version:Content-Type:Content-Transfer-Encoding; b=udMCSdvw4NN4uPt3RyonmS6SXjyDzRfAg7+OPcPm/GLYdUr3WvWIsj2hp8FlbsPx0 jida+4YFe96C+oSaHYg7xj6CO3pDGpbtBRuD81Y/kDd37/W6EtHUcbg8sJ+q88oZ5C 2r9FSoGV+qAfxCilH29Sedkm6bkeNsf6V85ES6RY= Received: from unknown (unknown [77.93.52.19]) by smtp8.mail.yandex.net (nwsmtp/Yandex) with ESMTP id xXtuKItO-xXtudeWl; Thu, 2 Feb 2012 12:59:34 +0400 X-Yandex-Spam: 1 Date: Thu, 2 Feb 2012 10:59:12 +0200 From: =?utf-8?B?0JrQvtC90YzQutC+0LIg0JXQstCz0LXQvdC40Lk=?= X-Mailer: The Bat! (v4.0.24) Professional Organization: =?utf-8?B?0KfQnyDQmtC+0L3RjNC60L7QsiwgRnJlZUxpbmU=?= X-Priority: 3 (Normal) Message-ID: <1446971288.20120202105912@yandex.ru> To: Andrey Zonov In-Reply-To: <4F2A2EAB.3010700@zonov.org> References: <4F29A464.3080302@zonov.org> <4F29E2C8.5000909@FreeBSD.org> <4F2A2EAB.3010700@zonov.org> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: 8bit Cc: freebsd-net@freebsd.org, "Alexander V. Chernikov" Subject: Re[2]: netisr defered - active only one thread X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: =?utf-8?B?0JrQvtC90YzQutC+0LIg0JXQstCz0LXQvdC40Lk=?= List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Feb 2012 08:59:37 -0000 Здравствуйте, Andrey. Вы писали 2 февраля 2012 г., 8:35:23: AZ> On 02.02.2012 5:11, Alexander V. Chernikov wrote: >> On 01.02.2012 20:45, Andrey Zonov wrote: >>> Hi, >>> >>> I'm trying to tune machine with 8.2-STABLE for heavy network load and >>> now playing with netisr. Could anyone explain me why actually works only >>> one netisr thread if I set them to 8? >> >> Can you please supply `nestat -Q` output and clarify you usage pattern ? >> (I mean, this is router/web server/some kind of traffic receiver/etc..). >> For example, flow policy does not balance traffic from single flow >> between different CPUs. >> AZ> This is a web server with multiple nginx instances. 5k/sec accepted AZ> connections. Input packet rate is 35kpps, output - 25kpps. AZ> I thought of changing policy for IP, but how can I do this (without AZ> patching)? Is it safe? AZ> netstat -Q (I turned on direct & direct force for now): AZ> Configuration: AZ> Setting Value Maximum AZ> Thread count 8 8 AZ> Default queue limit 256 10240 AZ> Direct dispatch enabled n/a AZ> Forced direct dispatch enabled n/a AZ> Threads bound to CPUs enabled n/a AZ> Protocols: AZ> Name Proto QLimit Policy Flags AZ> ip 1 5000 flow --- AZ> igmp 2 256 source --- AZ> rtsock 3 256 source --- AZ> arp 7 256 source --- AZ> ip6 10 256 flow --- AZ> Workstreams: AZ> WSID CPU Name Len WMark Disp'd HDisp'd QDrops Queued Handled AZ> 0 0 ip 0 0 1125716 0 0 0 1125716 AZ> igmp 0 0 0 0 0 0 AZ> rtsock 0 1 0 0 0 102 102 AZ> arp 0 0 27 0 0 0 27 AZ> ip6 0 0 0 0 0 0 AZ> 1 1 ip 0 0 1222701 0 0 0 1222701 AZ> igmp 0 0 0 0 0 0 AZ> rtsock 0 0 0 0 0 0 AZ> arp 0 0 46 0 0 0 46 AZ> ip6 0 0 0 0 0 0 AZ> 2 2 ip 0 0 1184381 0 0 0 1184381 AZ> igmp 0 0 0 0 0 0 AZ> rtsock 0 0 0 0 0 0 AZ> arp 0 0 45 0 0 0 45 AZ> ip6 0 0 0 0 0 0 AZ> 3 3 ip 0 0 1191094 0 0 0 1191094 AZ> igmp 0 0 0 0 0 0 AZ> rtsock 0 0 0 0 0 0 AZ> arp 0 0 54 0 0 0 54 AZ> ip6 0 0 0 0 0 0 AZ> 4 4 ip 0 0 846165 0 0 0 846165 AZ> igmp 0 0 0 0 0 0 AZ> rtsock 0 0 0 0 0 0 AZ> arp 0 0 19 0 0 0 19 AZ> ip6 0 0 0 0 0 0 AZ> 5 5 ip 0 0 849478 0 0 0 849478 AZ> igmp 0 0 0 0 0 0 AZ> rtsock 0 0 0 0 0 0 AZ> arp 0 0 27 0 0 0 27 AZ> ip6 0 0 0 0 0 0 AZ> 6 6 ip 0 0 870836 0 0 0 870836 AZ> igmp 0 0 0 0 0 0 AZ> rtsock 0 0 0 0 0 0 AZ> arp 0 0 29 0 0 0 29 AZ> ip6 0 0 0 0 0 0 AZ> 7 7 ip 0 5000 594320 5 910862 3453459 4047784 AZ> igmp 0 0 0 0 0 0 AZ> rtsock 0 0 0 0 0 0 AZ> arp 0 5 21 0 0 109 130 AZ> ip6 0 1 0 0 0 1 same problem, it is because one netisr take 100% so other threads stops?? to work fine. or packet scheduler has disbalanced scheduler and still trying to schedule packet to netisr:7 despite on it is 100% busy. -- С уважением, Коньков mailto:kes-kes@yandex.ru