From owner-freebsd-pf@FreeBSD.ORG Sun Feb 14 00:30:59 2010 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B21FD106568F for ; Sun, 14 Feb 2010 00:30:59 +0000 (UTC) (envelope-from ddesimone@verio.net) Received: from relay2-bcrtfl2.verio.net (relay2-bcrtfl2.verio.net [131.103.218.177]) by mx1.freebsd.org (Postfix) with ESMTP id 662B48FC12 for ; Sun, 14 Feb 2010 00:30:59 +0000 (UTC) Received: from iad-wprd-xchw01.corp.verio.net (unknown [198.87.7.164]) by relay2-bcrtfl2.verio.net (Postfix) with ESMTP id 6E3C71FF0AF4 for ; Sat, 13 Feb 2010 19:30:58 -0500 (EST) thread-index: AcqtDPoGGhO/W4rKQFmdz4TXUg0zfQ== Received: from dllstx1-8sst9f1.corp.verio.net ([10.144.0.33]) by iad-wprd-xchw01.corp.verio.net over TLS secured channel with Microsoft SMTPSVC(6.0.3790.3959); Sat, 13 Feb 2010 19:30:56 -0500 Received: by dllstx1-8sst9f1.corp.verio.net (sSMTP sendmail emulation); Sat, 13 Feb 2010 18:30:56 +0000 Content-Transfer-Encoding: 7bit Date: Sat, 13 Feb 2010 18:30:55 -0600 From: "David DeSimone" Content-Class: urn:content-classes:message Importance: normal Priority: normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.4325 To: Message-ID: <20100214003055.GE952@verio.net> Mail-Followup-To: freebsd-pf@freebsd.org References: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: Precedence: bulk User-Agent: Mutt/1.5.18 (2008-05-17) X-OriginalArrivalTime: 14 Feb 2010 00:30:56.0866 (UTC) FILETIME=[F9624420:01CAAD0C] Subject: Re: pf: nat works, ip blocking and logging do not X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Feb 2010 00:30:59 -0000 Alex Teslik wrote: > > Unfortunately, I can still go to 11.11.11.111 or 22.22.22.222 with no > blocking and no logging on /var/log/pflog. Your outbound traffic first matches this rule: > pass out all [keep state] The [keep state] I added because it is implied; you would have to explicitly add "no state" in order to avoid it, and you probably actually do want state-tracking on most outbound connections. This is important to your question, though, because if you open an outbound connection to 11.11.11.111 or 22.22.22.222, it will store a state entry, and state entries are checked BEFORE any rule-checking, for reply traffic coming back from those hosts. So, these rules: > block drop in log (all) quick on $ext_if from { 11.11.11.111, 22.22.22.222 } to any > block drop out log (all) quick on $ext_if from { 11.11.11.111, 22.22.22.222 } to any will never be checked, because they match the previously built state. In order to prevent communications with these hosts, you must also add "block out [quick]" rules which prevent you from initiating the connection to them and thus building state entries. -- David DeSimone == Network Admin == fox@verio.net "I don't like spinach, and I'm glad I don't, because if I liked it I'd eat it, and I just hate it." -- Clarence Darrow This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio, Inc. makes no warranty that this email is error or virus free. Thank you. From owner-freebsd-pf@FreeBSD.ORG Mon Feb 15 07:06:01 2010 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CC96E1065676 for ; Mon, 15 Feb 2010 07:06:01 +0000 (UTC) (envelope-from k@kevinkevin.com) Received: from mail-iw0-f180.google.com (mail-iw0-f180.google.com [209.85.223.180]) by mx1.freebsd.org (Postfix) with ESMTP id 9C98F8FC0C for ; Mon, 15 Feb 2010 07:06:01 +0000 (UTC) Received: by iwn10 with SMTP id 10so353499iwn.13 for ; Sun, 14 Feb 2010 23:06:00 -0800 (PST) Received: by 10.231.144.15 with SMTP id x15mr7265897ibu.99.1266217560787; Sun, 14 Feb 2010 23:06:00 -0800 (PST) Received: from kkPC (not.enough.unixsluts.com [76.10.166.187]) by mx.google.com with ESMTPS id 21sm6093961iwn.10.2010.02.14.23.05.59 (version=SSLv3 cipher=RC4-MD5); Sun, 14 Feb 2010 23:05:59 -0800 (PST) From: "kevin" To: Date: Mon, 15 Feb 2010 02:05:56 -0500 Message-ID: <006401caae0d$530b5560$f9220020$@com> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 thread-index: AcquDVIZ8/RvpCPKR+OnOH1/kDqsiA== Content-Language: en-us Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Server unresponsive when using transparent bridging w/ pf + pfsync X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Feb 2010 07:06:02 -0000 Hello, I'd like to get thoughts / input to the following application of 2x FreeBSD redundant firewalls. I have two firewalls with transparent bridges of the inside/outside interfaces (2 interfaces each firewall). A third interface is used for PFSYNC state synchronization. Synchronization of states is fine. However, if the bridge0 interface is enabled on the second firewall, the first firewall completely halts and is unresponsive ,with no message on the console or in /var/log/messages. Unfortunately debug flags are disabled on both machines. One is 7.1-PRERELEASE and the other is 7.2-STABLE. I am in the process of upgrading the first one to 7.2-STABLE as well. Is this a viable scenario in the first place? Forgive my inexperience , any suggestions are welcome. Many Thanks. From owner-freebsd-pf@FreeBSD.ORG Mon Feb 15 10:56:34 2010 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 378A61065694 for ; Mon, 15 Feb 2010 10:56:34 +0000 (UTC) (envelope-from Albert.Shih@obspm.fr) Received: from smtp-int-m.obspm.fr (smtp-int-m.obspm.fr [145.238.187.15]) by mx1.freebsd.org (Postfix) with ESMTP id AB0858FC22 for ; Mon, 15 Feb 2010 10:56:33 +0000 (UTC) Received: from obspm.fr (pcjas.obspm.fr [145.238.184.233]) by smtp-int-m.obspm.fr (8.14.3/8.14.3/SIO Observatoire de Paris - 07/2009) with ESMTP id o1FAuTv8017988 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 15 Feb 2010 11:56:30 +0100 Date: Mon, 15 Feb 2010 11:56:29 +0100 From: Albert Shih To: geoffroy desvernay Message-ID: <20100215105629.GJ44403@obspm.fr> References: <20100205123254.GN11310@obspm.fr> <4B748700.70409@centrale-marseille.fr> <20100212164454.GA23456@obspm.fr> <4B765EAC.9020201@centrale-marseille.fr> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <4B765EAC.9020201@centrale-marseille.fr> User-Agent: Mutt/1.5.20 (2009-06-14) X-Miltered: at smtp-int-m.obspm.fr with ID 4B79285D.000 by Joe's j-chkmail (http : // j-chkmail dot ensmp dot fr)! X-j-chkmail-Enveloppe: 4B79285D.000/145.238.184.233/pcjas.obspm.fr/obspm.fr/ X-j-chkmail-Score: MSGID : 4B79285D.000 on smtp-int-m.obspm.fr : j-chkmail score : . : R=. U=. O=. B=0.020 -> S=0.020 X-j-chkmail-Status: Ham Cc: freebsd-pf@freebsd.org Subject: Re: How make the route-to working ? X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Feb 2010 10:56:34 -0000 Le 13/02/2010 à 09:11:24+0100, geoffroy desvernay a écrit > Albert Shih a écrit : > > OK now it's working. But I have some big trouble about the bandwith. > > > > Now when I try to do something like a scp, or ftp or wget from inside a > > jail to outside, everything work fine. The traffic go to right interface, > > the answer too. > > > > But when I try to do some network connection (ssh, scp etc..) from outside > > to a jail the bandwith is catastrophic (~40kB/s on 1Gbit/s). > > > > And for you ? > > > Using this kind of setup since at least two years for ~500 real users > without complains... (three different 'ssh jails' on the same machine > with many vlans and three "default" gateways) > OK I find the problem. It's come from TSO. If I disable TSO by sysctl everything become «normal» and works fine. Thanks again for your help. Regards. JAS -- Albert SHIH SIO batiment 15 Observatoire de Paris Meudon 5 Place Jules Janssen 92195 Meudon Cedex Téléphone : 01 45 07 76 26/06 86 69 95 71 Heure local/Local time: Lun 15 fév 2010 11:54:01 CET From owner-freebsd-pf@FreeBSD.ORG Mon Feb 15 11:07:06 2010 Return-Path: Delivered-To: freebsd-pf@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D66EC1065679 for ; Mon, 15 Feb 2010 11:07:06 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id C4F6D8FC23 for ; Mon, 15 Feb 2010 11:07:06 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id o1FB76cR070403 for ; Mon, 15 Feb 2010 11:07:06 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id o1FB76HY070401 for freebsd-pf@FreeBSD.org; Mon, 15 Feb 2010 11:07:06 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 15 Feb 2010 11:07:06 GMT Message-Id: <201002151107.o1FB76HY070401@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-pf@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-pf@FreeBSD.org X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Feb 2010 11:07:07 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/143543 pf [pf] [panic] PF route-to causes kernel panic o bin/143504 pf [patch] outgoing states are not killed by authpf(8) o conf/142961 pf [pf] No way to adjust pidfile in pflogd o conf/142817 pf [patch] etc/rc.d/pf: silence pfctl o kern/141905 pf [pf] [panic] pf kernel panic on 7.2-RELEASE with empty o kern/140697 pf [pf] pf behaviour changes - must be documented o kern/137982 pf [pf] when pf can hit state limits, random IP failures o kern/136781 pf [pf] Packets appear to drop with pf scrub and if_bridg o kern/135948 pf [pf] [gre] pf not natting gre protocol o kern/135162 pf [pfsync] pfsync(4) not usable with GENERIC kernel o kern/134996 pf [pf] Anchor tables not included when pfctl(8) is run w o kern/133732 pf [pf] max-src-conn issue o kern/132769 pf [pf] [lor] 2 LOR's with pf task mtx / ifnet and rtent f kern/132176 pf [pf] pf stalls connection when using route-to [regress o conf/130381 pf [rc.d] [pf] [ip6] ipv6 not fully configured when pf st o kern/129861 pf [pf] [patch] Argument names reversed in pf_table.c:_co o kern/127920 pf [pf] ipv6 and synproxy don't play well together o conf/127814 pf [pf] The flush in pf_reload in /etc/rc.d/pf does not w o kern/127439 pf [pf] deadlock in pf f kern/127345 pf [pf] Problem with PF on FreeBSD7.0 [regression] o kern/127121 pf [pf] [patch] pf incorrect log priority o kern/127042 pf [pf] [patch] pf recursion panic if interface group is o kern/125467 pf [pf] pf keep state bug while handling sessions between s kern/124933 pf [pf] [ip6] pf does not support (drops) IPv6 fragmented o kern/124364 pf [pf] [panic] Kernel panic with pf + bridge o kern/122773 pf [pf] pf doesn't log uid or pid when configured to o kern/122014 pf [pf] [panic] FreeBSD 6.2 panic in pf o kern/121704 pf [pf] PF mangles loopback packets o kern/120281 pf [pf] [request] lost returning packets to PF for a rdr o kern/120057 pf [pf] [patch] Allow proper settings of ALTQ_HFSC. The c o bin/118355 pf [pf] [patch] pfctl(8) help message options order false o kern/114567 pf [pf] [lor] pf_ioctl.c + if.c o kern/114095 pf [carp] carp+pf delay with high state limit o kern/111220 pf [pf] repeatable hangs while manipulating pf tables s conf/110838 pf [pf] tagged parameter on nat not working on FreeBSD 5. o kern/103283 pf pfsync fails to sucessfully transfer some sessions o kern/103281 pf pfsync reports bulk update failures o kern/93825 pf [pf] pf reply-to doesn't work o sparc/93530 pf [pf] Incorrect checksums when using pf's route-to on s o kern/92949 pf [pf] PF + ALTQ problems with latency o bin/86635 pf [patch] pfctl(8): allow new page character (^L) in pf. o kern/82271 pf [pf] cbq scheduler cause bad latency 42 problems total. From owner-freebsd-pf@FreeBSD.ORG Mon Feb 15 13:53:37 2010 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 266D4106568F for ; Mon, 15 Feb 2010 13:53:37 +0000 (UTC) (envelope-from dgeo@centrale-marseille.fr) Received: from melo.ec-m.fr (melo.ec-m.fr [147.94.19.139]) by mx1.freebsd.org (Postfix) with ESMTP id D88A38FC26 for ; Mon, 15 Feb 2010 13:53:36 +0000 (UTC) Received: from localhost (amavis3.serv.int [10.3.0.47]) by melo.ec-m.fr (GrosseBox 1743 XXL) with ESMTP id 661EBAC840; Mon, 15 Feb 2010 14:53:34 +0100 (CET) X-Virus-Scanned: amavisd-new at centrale-marseille.fr Received: from melo.ec-m.fr ([10.3.0.13]) by localhost (amavis3.serv.int [10.3.0.47]) (amavisd-new, port 10024) with LMTP id NDgfebTkbR3W; Mon, 15 Feb 2010 14:53:29 +0100 (CET) Received: from dgeo.sysadm.ec-m.fr (dgeo.sysadm.ec-m.fr [147.94.19.169]) (Authenticated sender: dgeo) by melo.ec-m.fr (GrosseBox 1743 XXL) with ESMTPSA id DF79EAC832; Mon, 15 Feb 2010 14:53:28 +0100 (CET) Message-ID: <4B7951D7.8080703@centrale-marseille.fr> Date: Mon, 15 Feb 2010 14:53:27 +0100 From: geoffroy desvernay User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.7) Gecko/20100210 Thunderbird/3.0.1 MIME-Version: 1.0 To: Albert Shih References: <20100205123254.GN11310@obspm.fr> <4B748700.70409@centrale-marseille.fr> <20100212164454.GA23456@obspm.fr> <4B765EAC.9020201@centrale-marseille.fr> <20100215105629.GJ44403@obspm.fr> In-Reply-To: <20100215105629.GJ44403@obspm.fr> X-Enigmail-Version: 1.0 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Cc: freebsd-pf@freebsd.org Subject: Re: How make the route-to working ? X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Feb 2010 13:53:37 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 15.02.2010 11:56, Albert Shih wrote: > Le 13/02/2010 à 09:11:24+0100, geoffroy desvernay a écrit >> Albert Shih a écrit : >>> OK now it's working. But I have some big trouble about the bandwith. >>> >>> Now when I try to do something like a scp, or ftp or wget from inside a >>> jail to outside, everything work fine. The traffic go to right interface, >>> the answer too. >>> >>> But when I try to do some network connection (ssh, scp etc..) from outside >>> to a jail the bandwith is catastrophic (~40kB/s on 1Gbit/s). >>> >>> And for you ? >>> >> Using this kind of setup since at least two years for ~500 real users >> without complains... (three different 'ssh jails' on the same machine >> with many vlans and three "default" gateways) >> > > OK I find the problem. It's come from TSO. > > If I disable TSO by sysctl everything become «normal» and works fine. > > Thanks again for your help. > You're welcome... Just to be sure to get it: you do use an 'fxp' ethernet card and 7.2-RELEASE or not ? (In other words, is it the fxp bug described here http://www.freebsd.org/releases/7.2R/errata.html ?) On the server I use, I've server# sysctl -a|grep tso net.inet.tcp.tso: 1 … without any (known) problem, on a: FreeBSD 7.2-STABLE #3: Thu Sep 24 21:32:33 CEST 2009 with two bge(4) interfaces... - -- *geoffroy desvernay* C.R.I - Administration systèmes et réseaux Ecole Centrale de Marseille -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkt5UdcACgkQGbFYzwF8gKrAVACfV9Kuq6jef7nQHzzRREvllCUg 7a8AoI4BhBP4WciZgrPSw1/E2TPkcflo =spTC -----END PGP SIGNATURE----- From owner-freebsd-pf@FreeBSD.ORG Mon Feb 15 20:47:19 2010 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DCE011065670 for ; Mon, 15 Feb 2010 20:47:19 +0000 (UTC) (envelope-from Albert.Shih@obspm.fr) Received: from smtp-int-m.obspm.fr (smtp-int-m.obspm.fr [145.238.187.15]) by mx1.freebsd.org (Postfix) with ESMTP id 5BF218FC0A for ; Mon, 15 Feb 2010 20:47:18 +0000 (UTC) Received: from obspm.fr (pcjas.obspm.fr [145.238.184.233]) by smtp-int-m.obspm.fr (8.14.3/8.14.3/SIO Observatoire de Paris - 07/2009) with ESMTP id o1FKlFTR025368 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 15 Feb 2010 21:47:17 +0100 Date: Mon, 15 Feb 2010 21:47:16 +0100 From: Albert Shih To: geoffroy desvernay Message-ID: <20100215204716.GH96648@obspm.fr> References: <20100205123254.GN11310@obspm.fr> <4B748700.70409@centrale-marseille.fr> <20100212164454.GA23456@obspm.fr> <4B765EAC.9020201@centrale-marseille.fr> <20100215105629.GJ44403@obspm.fr> <4B7951D7.8080703@centrale-marseille.fr> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <4B7951D7.8080703@centrale-marseille.fr> User-Agent: Mutt/1.5.20 (2009-06-14) X-Miltered: at smtp-int-m.obspm.fr with ID 4B79B2D3.000 by Joe's j-chkmail (http : // j-chkmail dot ensmp dot fr)! X-j-chkmail-Enveloppe: 4B79B2D3.000/145.238.184.233/pcjas.obspm.fr/obspm.fr/ X-j-chkmail-Score: MSGID : 4B79B2D3.000 on smtp-int-m.obspm.fr : j-chkmail score : . : R=. U=. O=. B=0.011 -> S=0.011 X-j-chkmail-Status: Ham Cc: freebsd-pf@freebsd.org Subject: Re: How make the route-to working ? X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Feb 2010 20:47:19 -0000 Le 15/02/2010 à 14:53:27+0100, geoffroy desvernay a écrit > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > > Thanks again for your help. > > > You're welcome... > > Just to be sure to get it: you do use an 'fxp' ethernet card and > 7.2-RELEASE or not ? No it's bce (the server is Dell PowerEdge 2900). I'm running 7.2-p6. > > On the server I use, I've > server# sysctl -a|grep tso > net.inet.tcp.tso: 1 > > ? without any (known) problem, on a: > FreeBSD 7.2-STABLE #3: Thu Sep 24 21:32:33 CEST 2009 > > with two bge(4) interfaces... > I'm going to send a email to freebsd-net. I don't knwon if it's a bug or not. Regards. Thanks for your help (I insist ;-) ). JAS -- Albert SHIH SIO batiment 15 Observatoire de Paris Meudon 5 Place Jules Janssen 92195 Meudon Cedex Téléphone : 01 45 07 76 26/06 86 69 95 71 Heure local/Local time: Lun 15 fév 2010 21:44:39 CET From owner-freebsd-pf@FreeBSD.ORG Mon Feb 15 21:11:45 2010 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7309F106566B; Mon, 15 Feb 2010 21:11:45 +0000 (UTC) (envelope-from Albert.Shih@obspm.fr) Received: from smtp-int-m.obspm.fr (smtp-int-m.obspm.fr [145.238.187.15]) by mx1.freebsd.org (Postfix) with ESMTP id 0EDB88FC08; Mon, 15 Feb 2010 21:11:44 +0000 (UTC) Received: from obspm.fr (pcjas.obspm.fr [145.238.184.233]) by smtp-int-m.obspm.fr (8.14.3/8.14.3/SIO Observatoire de Paris - 07/2009) with ESMTP id o1FLBfMo009454 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 15 Feb 2010 22:11:43 +0100 Date: Mon, 15 Feb 2010 22:11:41 +0100 From: Albert Shih To: freebsd-net@freebsd.org, freebsd-pf@freebsd.org Message-ID: <20100215211141.GK96648@obspm.fr> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit User-Agent: Mutt/1.5.20 (2009-06-14) X-Miltered: at smtp-int-m.obspm.fr with ID 4B79B88D.000 by Joe's j-chkmail (http : // j-chkmail dot ensmp dot fr)! X-j-chkmail-Enveloppe: 4B79B88D.000/145.238.184.233/pcjas.obspm.fr/obspm.fr/ X-j-chkmail-Score: MSGID : 4B79B88D.000 on smtp-int-m.obspm.fr : j-chkmail score : . : R=. U=. O=. B=0.010 -> S=0.010 X-j-chkmail-Status: Ham Cc: Subject: Possible bug in TSO or in pf on bce X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Feb 2010 21:11:45 -0000 Hi all, I'm not a tcp/ip guru, so I don't known if it's a bug or not. The situation is little complexe, so I'm going to explain that. I've one server with tree interfaces two bce and one bge. All test is on two bce. This server running FreeBSD-7.2-p6 and have lot of jail (but the problem is the same for one jail, so I assume I've just one jail). The bce0 and bce1 are in different vlan. The jail is on bce1 (meaning the jail IP is on the bce1 subnet). The default gateway is on bce0 So to make all traffic of the jail pass only throught bce1 and not using bce0 I'm using pf with something like pass out route-to (bce1 bce1_subnet_gw) from jail_IP to ! bce1_subnet keep state pass in on bce1 reply-to (bce1 bce1_subnet_gw) from ! bce1_subnet to jail_IP keep state if I do that all traffic pass through the right interface (bce1), but...the bandwith drop to ~60kb/s (on gigabit interface). So I find the problem is with TSO, if I deactivated the TSO the bandwith is return to normal. I don't knwon if it's a bug in PF (the problem is same if I use scrub or not) or in the TSO support of bce. I can run some few tests if someone like to debug, but because the server is in production I cannot make lot of test. Regards. JAS -- Albert SHIH SIO batiment 15 Observatoire de Paris Meudon 5 Place Jules Janssen 92195 Meudon Cedex Téléphone : 01 45 07 76 26/06 86 69 95 71 Heure local/Local time: Lun 15 fév 2010 22:10:52 CET From owner-freebsd-pf@FreeBSD.ORG Tue Feb 16 18:46:17 2010 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CF8AF10656A3 for ; Tue, 16 Feb 2010 18:46:17 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 78EF28FC20 for ; Tue, 16 Feb 2010 18:46:17 +0000 (UTC) Received: by vws20 with SMTP id 20so434734vws.13 for ; Tue, 16 Feb 2010 10:46:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=T7lzqGxlcA07Upfmcp5lJ+DReoD3esjudY94WvNZzeE=; b=T163rQByPfN9blW72VAl04TbpdMY+2yeLqk2rDCjVTo1H9rURrR3OYk3Da+gm+OcxU n5PMZyW+8UVWNh0DGuE/XwzgrhbMjlsaJFCzMYQgf+6brOth37I5XDyb3UJe4pvp+RdD /iFxuiHzdMy3XRktKVLeI17W/uL0HgXt9N9Q8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=GQ7bNfdW6PtFsWykW7/XwwK/+dMVtSK+B0QiNJfehn592a+GUp9X4nhjt5y6p6w32L aTSnJHs77odcCze2YQwNc/DKuwCYjneH7pcjcmNSKdvPz62gTKLgh3bDh1OqhA9xfj2S abhuQEM2SAyg6KWTrEsS1ZYFX3Qwlrvi2MX5w= Received: by 10.220.124.135 with SMTP id u7mr1095185vcr.231.1266344628532; Tue, 16 Feb 2010 10:23:48 -0800 (PST) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id 28sm9456522vws.11.2010.02.16.10.23.46 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 16 Feb 2010 10:23:47 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Tue, 16 Feb 2010 10:23:06 -0800 From: Pyun YongHyeon Date: Tue, 16 Feb 2010 10:23:06 -0800 To: Albert Shih Message-ID: <20100216182306.GC1394@michelle.cdnetworks.com> References: <20100215211141.GK96648@obspm.fr> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="9jxsPFA5p3P2qPhR" Content-Disposition: inline In-Reply-To: <20100215211141.GK96648@obspm.fr> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org, freebsd-pf@freebsd.org Subject: Re: Possible bug in TSO or in pf on bce X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 18:46:18 -0000 --9jxsPFA5p3P2qPhR Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Mon, Feb 15, 2010 at 10:11:41PM +0100, Albert Shih wrote: > Hi all, > > I'm not a tcp/ip guru, so I don't known if it's a bug or not. > > The situation is little complexe, so I'm going to explain that. > > I've one server with tree interfaces two bce and one bge. All test is on > two bce. > > This server running FreeBSD-7.2-p6 and have lot of jail (but the problem is > the same for one jail, so I assume I've just one jail). The bce0 and bce1 > are in different vlan. > > The jail is on bce1 (meaning the jail IP is on the bce1 subnet). > > The default gateway is on bce0 > > So to make all traffic of the jail pass only throught bce1 and not using > bce0 I'm using pf with something like > > pass out route-to (bce1 bce1_subnet_gw) from jail_IP to ! bce1_subnet keep state > pass in on bce1 reply-to (bce1 bce1_subnet_gw) from ! bce1_subnet to jail_IP keep state > > if I do that all traffic pass through the right interface (bce1), but...the > bandwith drop to ~60kb/s (on gigabit interface). > > So I find the problem is with TSO, if I deactivated the TSO the bandwith is > return to normal. > > I don't knwon if it's a bug in PF (the problem is same if I use scrub or > not) or in the TSO support of bce. > At first I thought you hit one of edge case of TSO on bce(4). But it seems the issue comes from pf's route handling. When I ported pf from OpenBSD, there was no TSO capability in FreeBSD at that time so the pf_route() had no special handling code for TSO. Since it was long time ago I'm not sure whether it's correct or not but try attached patch. Apart from TSO FreeBSD got several new features like fib, flow-table and vnet. We may need to check whether these new features are still working with pf(4). --9jxsPFA5p3P2qPhR Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="pf.routeto.patch" Index: sys/contrib/pf/net/pf.c =================================================================== --- sys/contrib/pf/net/pf.c (revision 203960) +++ sys/contrib/pf/net/pf.c (working copy) @@ -6375,6 +6375,7 @@ m0->m_pkthdr.csum_flags &= ifp->if_hwassist; if (ntohs(ip->ip_len) <= ifp->if_mtu || + (m0->m_pkthdr.csum_flags & ifp->if_hwassist & CSUM_TSO) != 0 || (ifp->if_hwassist & CSUM_FRAGMENT && ((ip->ip_off & htons(IP_DF)) == 0))) { /* @@ -6449,7 +6450,7 @@ * Too large for interface; fragment if possible. * Must be able to put at least 8 bytes per fragment. */ - if (ip->ip_off & htons(IP_DF)) { + if (ip->ip_off & htons(IP_DF) || (m0->m_pkthdr.csum_flags & CSUM_TSO)) { KMOD_IPSTAT_INC(ips_cantfrag); if (r->rt != PF_DUPTO) { #ifdef __FreeBSD__ --9jxsPFA5p3P2qPhR-- From owner-freebsd-pf@FreeBSD.ORG Tue Feb 16 19:03:54 2010 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9B0631065697 for ; Tue, 16 Feb 2010 19:03:54 +0000 (UTC) (envelope-from julian@elischer.org) Received: from out-0.mx.aerioconnect.net (out-0-3.mx.aerioconnect.net [216.240.47.63]) by mx1.freebsd.org (Postfix) with ESMTP id 732298FC18 for ; Tue, 16 Feb 2010 19:03:54 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id o1GIkjYS017278; Tue, 16 Feb 2010 10:46:45 -0800 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id AC2CB2D6013; Tue, 16 Feb 2010 10:46:44 -0800 (PST) Message-ID: <4B7AE844.3090007@elischer.org> Date: Tue, 16 Feb 2010 10:47:32 -0800 From: Julian Elischer User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: pyunyh@gmail.com References: <20100215211141.GK96648@obspm.fr> <20100216182306.GC1394@michelle.cdnetworks.com> In-Reply-To: <20100216182306.GC1394@michelle.cdnetworks.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: Albert Shih , freebsd-pf@freebsd.org, freebsd-net@freebsd.org Subject: Re: Possible bug in TSO or in pf on bce X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 19:03:54 -0000 Pyun YongHyeon wrote: > On Mon, Feb 15, 2010 at 10:11:41PM +0100, Albert Shih wrote: >> Hi all, >> >> I'm not a tcp/ip guru, so I don't known if it's a bug or not. >> >> The situation is little complexe, so I'm going to explain that. >> >> I've one server with tree interfaces two bce and one bge. All test is on >> two bce. >> >> This server running FreeBSD-7.2-p6 and have lot of jail (but the problem is >> the same for one jail, so I assume I've just one jail). The bce0 and bce1 >> are in different vlan. >> >> The jail is on bce1 (meaning the jail IP is on the bce1 subnet). >> >> The default gateway is on bce0 >> >> So to make all traffic of the jail pass only throught bce1 and not using >> bce0 I'm using pf with something like >> >> pass out route-to (bce1 bce1_subnet_gw) from jail_IP to ! bce1_subnet keep state >> pass in on bce1 reply-to (bce1 bce1_subnet_gw) from ! bce1_subnet to jail_IP keep state >> >> if I do that all traffic pass through the right interface (bce1), but...the >> bandwith drop to ~60kb/s (on gigabit interface). >> >> So I find the problem is with TSO, if I deactivated the TSO the bandwith is >> return to normal. >> >> I don't knwon if it's a bug in PF (the problem is same if I use scrub or >> not) or in the TSO support of bce. >> > > At first I thought you hit one of edge case of TSO on bce(4). But > it seems the issue comes from pf's route handling. When I ported pf > from OpenBSD, there was no TSO capability in FreeBSD at that time > so the pf_route() had no special handling code for TSO. Since it > was long time ago I'm not sure whether it's correct or not but try > attached patch. > > Apart from TSO FreeBSD got several new features like fib, > flow-table and vnet. We may need to check whether these new > features are still working with pf(4). yes, in 8.0 you have options ot do what you want to do in several ways. From a quick look, both multi-FIBs and vnet may be directly applicable to you. As for pf, it works with multi fibs but the patch for vnet depends upon teh installation of a newer revision of pf and that may not be possible in 8.x. (ceri may be able so shed more light on that, I'm assuming he's seeing this.) > > > ------------------------------------------------------------------------ > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"