From owner-freebsd-net@FreeBSD.ORG Tue Oct 6 09:21:43 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 773EF1065672 for ; Tue, 6 Oct 2009 09:21:43 +0000 (UTC) (envelope-from rihad@mail.ru) Received: from mx71.mail.ru (mx71.mail.ru [94.100.176.85]) by mx1.freebsd.org (Postfix) with ESMTP id 2E2DA8FC17 for ; Tue, 6 Oct 2009 09:21:43 +0000 (UTC) Received: from [217.25.27.27] (port=19823 helo=[217.25.27.27]) by mx71.mail.ru with asmtp id 1Mv6Eo-000DXv-00; Tue, 06 Oct 2009 13:21:42 +0400 Message-ID: <4ACB0C22.4000008@mail.ru> Date: Tue, 06 Oct 2009 14:21:38 +0500 From: rihad User-Agent: Mozilla-Thunderbird 2.0.0.22 (X11/20090706) MIME-Version: 1.0 To: Julian Elischer References: <20091005061025.GB55845@onelab2.iet.unipi.it> <4AC9B400.9020400@mail.ru> <20091005090102.GA70430@svzserv.kemerovo.su> <4AC9BC5A.50902@mail.ru> <20091005095600.GA73335@svzserv.kemerovo.su> <4AC9CFF7.3090208@mail.ru> <20091005110726.GA62598@onelab2.iet.unipi.it> <4AC9D87E.7000005@mail.ru> <20091005120418.GA63131@onelab2.iet.unipi.it> <4AC9E29B.6080908@mail.ru> <20091005123230.GA64167@onelab2.iet.unipi.it> <4AC9EFDF.4080302@mail.ru> <4ACA2CC6.70201@elischer.org> <4ACAFF2A.1000206@mail.ru> In-Reply-To: <4ACAFF2A.1000206@mail.ru> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam: Not detected X-Mras: Ok Cc: freebsd-net@freebsd.org, Luigi Rizzo Subject: Re: dummynet dropping too many packets X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Oct 2009 09:21:43 -0000 rihad wrote: > Julian Elischer wrote: >> rihad wrote: >>> Luigi Rizzo wrote: >>>> 2. your test with 'ipfw allow ip from any to any' does not >>>> prove that the interface queue is not saturating, because >>>> you also remove the burstiness that dummynet introduces, >>>> and so the queue is driven differently. >>>> >>> >>> How do I investigate and fix this burstiness issue? >> >> higher Hz rate? >> > > Rebooted with HZ=2000 10 minutes ago. Due to application design the ipfw > table (pipe tablearg) was flushed, so there are now 350 (and increasing > at a rate 1 per 1-2 seconds as I type this) or so users in the table, > and not 4k as normally would be. The box is servicing 450+ mbit/s > without a single drop. I want to monitor how things change once the > number of users in ipfw tables gradually increases up to several thousands. > It starts dropping packets at around 2000 online users (ipfw table load). I've set up a shell script to monitor this: # while :; do ipfw table 0 list | wc -l; netstat -s 2>/dev/null |fgrep -w 'output packets dropped'; sleep 10; done ... # all zeroes above this 1999 0 output packets dropped due to no bufs, etc. 2001 0 output packets dropped due to no bufs, etc. 2008 0 output packets dropped due to no bufs, etc. 2017 0 output packets dropped due to no bufs, etc. 2027 156 output packets dropped due to no bufs, etc. 2037 156 output packets dropped due to no bufs, etc. 2045 156 output packets dropped due to no bufs, etc. 2372 202 output packets dropped due to no bufs, etc. 2377 207 output packets dropped due to no bufs, etc. 2391 338 output packets dropped due to no bufs, etc. 2402 394 output packets dropped due to no bufs, etc. 2415 531 output packets dropped due to no bufs, etc. 2421 725 output packets dropped due to no bufs, etc. Is there some limit on the number of IP addresses in an ipfw table?