From owner-freebsd-ipfw@FreeBSD.ORG Wed Nov 5 14:35:19 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9D26516A4CE for ; Wed, 5 Nov 2003 14:35:19 -0800 (PST) Received: from firewall.floating-oak.com (adsl-67-117-120-153.dsl.scrm01.pacbell.net [67.117.120.153]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7C24043FE9 for ; Wed, 5 Nov 2003 14:35:16 -0800 (PST) (envelope-from mnadler@floating-oak.org) Received: from floating-oak.org (dell.floating-oak.com [192.168.1.3]) by firewall.floating-oak.com (8.9.3/8.9.3) with ESMTP id OAA09425 for ; Wed, 5 Nov 2003 14:35:16 -0800 Message-ID: <3FA97B22.7010808@floating-oak.org> Date: Wed, 05 Nov 2003 14:35:14 -0800 From: Michael Nadler User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 X-Accept-Language: en-us, en To: freebsd-ipfw@freebsd.org References: <20031105200049.B897F16A51A@hub.freebsd.org> In-Reply-To: <20031105200049.B897F16A51A@hub.freebsd.org> Content-Transfer-Encoding: 7bit MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" X-Content-Filtered-By: Mailman/MimeDel 2.1.1 Subject: Re: maximum pipes in dummynet? X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Nov 2003 22:35:19 -0000 The basic problem you are having is with ipfw, not merely dummynet. ipfw uses a linked list to hold the rules. Every packet must be matched against these rules using a linear search. And this search happens in the interrupt thread. Packets that match none of the rules end up having the worst latency. There is an inverse relationship between the number of packets and the number of rules. In my experience, 4000 ipfw rules is quite a lot. We had to implement a custom modification to ipfw, creating a special class of static rules that were handled similar to the dynamic rules. This mod permitted more than 100,000 rules with litte performance degredation. The ipfw pipe rule, just like a regular rule, can match multiple ports. See ipfw(8). Date: Wed, 5 Nov 2003 10:59:50 -0800 From: Yuchung Chen Subject: maximum pipes in dummynet? To: [1]ipfw@freebsd.org Cc: [2]ycheng@cs.ucsd.edu Message-ID: [3]<20031105105949.A15915@cs.ucsd.edu> Content-Type: text/plain; charset=us-ascii Hi all, I am using dummynet to simulate many different links. After dummynet seems to slow down after more than 4000 (static) pipes. That is, the link delay is more than the specified delay. My config file looks like ipfw add 1 pipe 7 tcp from 192.168.2.100 10003 to 192.168.1.100 out ipfw pipe 7 config bw 730KByte/s queue 40 delay 4ms plr 0.01 ipfw add 1 pipe 8 tcp from 192.168.1.100 to 192.168.2.100 10003 in ipfw pipe 8 config bw 730KByte/s queue 40 delay 4ms plr 0.01 ipfw add 1 pipe 9 tcp from 192.168.2.100 10004 to 192.168.1.100 out ipfw pipe 9 config bw 1400KByte/s queue 40 delay 28ms plr 0.00 ipfw add 1 pipe 10 tcp from 192.168.1.100 to 192.168.2.100 10004 in ipfw pipe 10 config bw 1503KByte/s queue 40 delay 28ms plr 0.00 My machine is 2.66 P4 w/ 1G mem. Does anybody know how to make dummynet support more pipes? Also is it possible to specify one pipe for more than one port? i.e., ipfw add 1 pipe 10 tcp from 192.168.1.100 to 192.168.2.100 10004,10537,3045 in Thanks. Yu-chung References 1. mailto:ipfw@freebsd.org 2. mailto:ycheng@cs.ucsd.edu 3. mailto:20031105105949.A15915@cs.ucsd.edu