From owner-freebsd-net@FreeBSD.ORG Sat Oct 17 06:28:55 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 169101065670; Sat, 17 Oct 2009 06:28:55 +0000 (UTC) (envelope-from rihad@mail.ru) Received: from mx34.mail.ru (mx34.mail.ru [94.100.176.48]) by mx1.freebsd.org (Postfix) with ESMTP id C64108FC0A; Sat, 17 Oct 2009 06:28:53 +0000 (UTC) Received: from [217.25.27.27] (port=26012 helo=[217.25.27.27]) by mx34.mail.ru with asmtp id 1Mz2mZ-0009Jx-00; Sat, 17 Oct 2009 10:28:51 +0400 Message-ID: <4AD96422.1040008@mail.ru> Date: Sat, 17 Oct 2009 11:28:50 +0500 From: rihad User-Agent: Mozilla-Thunderbird 2.0.0.22 (X11/20090706) MIME-Version: 1.0 To: rihad References: <4AD6D99E.10805@mail.ru> <4AD95493.40200@mail.ru> In-Reply-To: <4AD95493.40200@mail.ru> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam: Not detected X-Mras: Ok Cc: freebsd-net@freebsd.org, Robert Watson Subject: Re: dummynet dropping too many packets X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Oct 2009 06:28:55 -0000 rihad wrote: > rihad wrote: >> For now we've mostly disabled dummynet and the drops have stopped, >> thanks to some extra unused bandwidth we have. But this isn't a real >> solution, of course, so this weekend I'm going to try the suggestion >> made by Robert Watson: >> >> > In the driver init code in if_bce, the following code appears: >> > >> > ifp->if_snd.ifq_drv_maxlen = USABLE_TX_BD; >> > IFQ_SET_MAXLEN(&ifp->if_snd, ifp->if_snd.ifq_drv_maxlen); >> > IFQ_SET_READY(&ifp->if_snd); >> > >> > Which evaluates to a architecture-specific value due to varying >> pagesize. You might just try forcing it to 1024. > >> In a few days I'm going to try it anyway, and if the system locks up >> I'll just revert back to the original code > >> TIA >> > > Just rebooted with the "ifp->if_snd.ifq_drv_maxlen = 1024;" kernel, all > ok so far. There's currenlty only 1000 or so entries in the ipfw table > and around 350-400 net mbps load, so I'll wait a few hours for the > numbers to grow to >2000 and 460-480 respectively and see if the drops > still occur. > The change definitely helped! There are now more than 3200 users online, 460-500 mbps net traffic load, and normally 10-60 (up to 150 once or twice) consistent drops per second as opposed to several hundred up to 1000-1500 packets dropped per second before the rebuild. What's interesting is that the drops now began only after the ipfw table had around 3000 entries, not 2000 like before, so the change definitely helped. Just how high can maxlen be? Should I try 2048? 4096?