From owner-freebsd-ipfw@FreeBSD.ORG Fri Oct 1 09:47:08 2010 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 819F3106566B for ; Fri, 1 Oct 2010 09:47:08 +0000 (UTC) (envelope-from cosmic17@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 3F3E88FC13 for ; Fri, 1 Oct 2010 09:47:07 +0000 (UTC) Received: by ywt2 with SMTP id 2so1162511ywt.13 for ; Fri, 01 Oct 2010 02:47:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=yvr7nggUCwOfWfFBrOs7ADCuJwHcwTwYyJjVBs+P5jM=; b=eewgSNkn8Ymj4q05Bxlff6B+HbaWAiAIGxzDDF6vt4PpVVQETcrx650Wyw4iaNs8f1 W7vtyIvGez+AeVjM4R7fky9E7FTC71xznSlxGE3TUVYYTcoLW+AGSI1YSYyVV1OxL43l nAYAcoJs50gVU54G8z/l+F0a4+gCdLj6HbM/s= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=bFStHuDgjh33fd0aztpfNkOCkFOibDlXafBChgBjYhFkElLxZFyb20Y7wmYH8SY+7P dzgC+76Jl7v0fcok1iJMlX2c19z/X/8sBsHquWZdi+6OOrcZc+OXVrMxsdH1d+y5F1J5 XvEconeBmbLINEMxqW9VTb3ejiFGMem+iiZTM= MIME-Version: 1.0 Received: by 10.151.27.6 with SMTP id e6mr672740ybj.262.1285924633876; Fri, 01 Oct 2010 02:17:13 -0700 (PDT) Received: by 10.220.161.149 with HTTP; Fri, 1 Oct 2010 02:17:13 -0700 (PDT) Date: Fri, 1 Oct 2010 13:17:13 +0400 Message-ID: From: =?KOI8-R?B?7snLz8zByiDkzdXIwQ==?= To: freebsd-ipfw@freebsd.org X-Mailman-Approved-At: Fri, 01 Oct 2010 11:13:03 +0000 Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Strange behavior of packet scheduling in ipfw3 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Oct 2010 09:47:08 -0000 Hello! The system is: FreeBSD mysystem 8.0-STABLE-201005 FreeBSD 8.0-STABLE-201005 #0: Wed Jul 28 12:04:29 MSD 2010 root@mysystem:/usr/src/sys/amd64/compile/MYKERNEL amd64 There is firewall "ipfw3" from Luigi Rizzo with packet scheduling. There is part of firewall config (tariff with 1Mbit/s speed, for example), below (the rules for another speeds are the same): $IPFW pipe 11 config bw 1040Kbit/s mask dst-ip 0xffffffff $IPFW pipe 12 config bw 1040Kbit/s mask src-ip 0xffffffff ########pipe 11 $IPFW sched 11 config type QFQ mask dst-ip 0xffffff00 $IPFW queue 111 config sched 11 weight 10 $IPFW queue 112 config sched 11 weight 8 $IPFW queue 113 config sched 11 weight 4 $IPFW queue 114 config sched 11 weight 1 $IPFW add queue 111 ip from any to table\(10\) via igb0 out proto udp src-port 5060 $IPFW add queue 112 ip from any to table\(10\) via igb0 out proto tcp src-port 80,443,8080 $IPFW add queue 113 ip from any to table\(10\) via igb0 out proto tcp src-port 5223, 2009, 2106, 3724, 6112, 6881-6999, 7777, 27000-27050, 42292 $IPFW add queue 113 ip from any to table\(10\) via igb0 out proto icmp $IPFW add queue 114 ip from any to table\(10\) via igb0 out $IPFW add queue 111 ip from any to table\(10\) via igb2 out proto udp src-port 5060 $IPFW add queue 112 ip from any to table\(10\) via igb2 out proto tcp src-port 80,443,8080 $IPFW add queue 113 ip from any to table\(10\) via igb2 out proto tcp src-port 5223, 2009, 2106, 3724, 6112, 6881-6999, 7777, 27000-27050, 42292 $IPFW add queue 113 ip from any to table\(10\) via igb2 out proto icmp $IPFW add queue 114 ip from any to table\(10\) via igb2 out ########pipe 12 $IPFW sched 12 config type QFQ mask src-ip 0xffffff00 $IPFW queue 121 config sched 12 weight 10 $IPFW queue 122 config sched 12 weight 8 $IPFW queue 123 config sched 12 weight 4 $IPFW queue 124 config sched 12 weight 1 $IPFW add queue 1210 ip from table\(11\) to any via igb1 out proto udp dst-port 5060 $IPFW add queue 122 ip from table\(11\) to any via igb1 out proto tcp dst-port 80,443,8080 $IPFW add queue 123 ip from table\(11\) to any via igb1 out proto tcp dst-port 5223, 2009, 2106, 3724, 6112, 6881-6999, 7777, 27000-27050, 42292 $IPFW add queue 123 ip from table\(11\) to any via igb1 out proto icmp $IPFW add queue 124 ip from table\(11\) to any via igb1 out $IPFW add queue 121 ip from table\(11\) to any via igb3 out proto udp dst-port 5060 $IPFW add queue 122 ip from table\(11\) to any via igb3 out proto tcp dst-port 80,443,8080 $IPFW add queue 123 ip from table\(11\) to any via igb3 out proto tcp dst-port 5223, 2009, 2106, 3724, 6112, 6881-6999, 7777, 27000-27050, 42292 $IPFW add queue 123 ip from table\(11\) to any via igb3 out proto icmp $IPFW add queue 124 ip from table\(11\) to any via igb3 out Firstly, we have been tested firewall by ourself. And we had no any bad results or any problems or maybe we have not seen them in our synthetic tests. After that we have started this firewall in production. A few months later we received a message from our subscriber with speed 1Mbit/s. He had a problems with online game (big answer delay from the server). We spent a lot of time to solve this problem. Finaly we solved it. The reason was in packet scheduling: 1. we`ve tried to give to subscriber another channel (4Mbit/s) with packet scheduling - there are no such problems; 2. we`ve tried to "turn off" the packet scheduling on 1Mbit channel - there are no such problems. The utilization of subscibers channel was always 0.4Mbit/s. But the traffic from this subscriber was go on under the packet scheduling rules. That`s very strange because of: 1. net.inet.ip.dummynet.io_fast=1; 2. subscribers channel utilization 0.4Mbit/s. As I know with this option, with this firewall config and with this channel utilization (0.4Mbit/s) traffic should bypass the pipe without packet scheduling. Why subscribers traffic with all these conditions doesn`t bypass through pipe without any delays? Why his traffic was on packet scheduling rules? Thanks.