From owner-freebsd-net@FreeBSD.ORG Fri Feb 27 07:54:27 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3AD581065670 for ; Fri, 27 Feb 2009 07:54:27 +0000 (UTC) (envelope-from ady@ady.ro) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.191]) by mx1.freebsd.org (Postfix) with ESMTP id 690788FC1A for ; Fri, 27 Feb 2009 07:54:26 +0000 (UTC) (envelope-from ady@ady.ro) Received: by nf-out-0910.google.com with SMTP id d21so79078nfb.33 for ; Thu, 26 Feb 2009 23:54:25 -0800 (PST) MIME-Version: 1.0 Received: by 10.210.90.20 with SMTP id n20mr1750475ebb.72.1235721264974; Thu, 26 Feb 2009 23:54:24 -0800 (PST) In-Reply-To: <200902262341.35069.shawn@tandac.com> References: <3650.206.108.16.89.1235691792.squirrel@alder.hosix.com> <3853.206.108.16.89.1235693214.squirrel@alder.hosix.com> <78cb3d3f0902261619t71a054fet43779c37e2981603@mail.gmail.com> <200902262341.35069.shawn@tandac.com> Date: Fri, 27 Feb 2009 08:54:24 +0100 Message-ID: <78cb3d3f0902262354j6c22b43do565b75523a8007e3@mail.gmail.com> From: Adrian Penisoara To: Shawn Everett Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-net@freebsd.org Subject: Re: FreeBSD Router Problem X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2009 07:54:27 -0000 Hi, On Fri, Feb 27, 2009 at 8:41 AM, Shawn Everett wrote: > > Any error messages in dmesg output ? > > Significant changes in "netstat -m" output before and after ? > > The same for "pfctl -s all" output... > > The box has been up for about 12 hours now. As a point of discussion here > is the output from netstat and pfctl in case anything obvious jumps out. > > 385/905/1290 mbufs in use (current/cache/total) > 384/484/868/25600 mbuf clusters in use (current/cache/total/max) > 256/384 mbuf+clusters out of packet secondary zone in use (current/cache) > 0/44/44/12800 4k (page size) jumbo clusters in use > (current/cache/total/max) > 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) > 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) > 864K/1370K/2234K bytes allocated to network (current/cache/total) > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > 0/5/6656 sfbufs in use (current/peak/max) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > 0 calls to protocol drain routines > This looks OK to me... > > > # pfctl -s all > No ALTQ support in kernel > ALTQ related functions disabled > TRANSLATION RULES: > nat on ste0 inet from 172.16.3.0/24 to any -> (ste0) round-robin > nat on ste1 inet from 172.16.3.0/24 to any -> (ste1) round-robin > > FILTER RULES: > pass out on em0 inet from any to 172.16.3.0/24 flags S/SA keep state > pass in quick on em0 inet from 172.16.3.0/24 to 172.16.3.253 flags S/SA > keep state > pass in on em0 route-to { (ste0 204.244.159.254), (ste1 204.244.159.254) } > round-robin inet proto tcp from 172.16.3.0/24 to any flags S/SA modulate > state > pass in on em0 route-to { (ste0 204.244.159.254), (ste1 204.244.159.254) } > round-robin inet proto udp from 172.16.3.0/24 to any keep state > pass in on em0 route-to { (ste0 204.244.159.254), (ste1 204.244.159.254) } > round-robin inet proto icmp from 172.16.3.0/24 to any keep state > pass out on ste0 proto tcp all flags S/SA modulate state > pass out on ste0 proto udp all keep state > pass out on ste0 proto icmp all keep state > pass out on ste1 proto tcp all flags S/SA modulate state > pass out on ste1 proto udp all keep state > pass out on ste1 proto icmp all keep state > pass out on ste0 route-to (ste1 204.244.159.254) inet from 204.244.159.55 > to any flags S/SA keep state > pass out on ste1 route-to (ste0 204.244.159.254) inet from 204.244.159.68 > to any flags S/SA keep state > Quite an evolved route forwarding setup ;)... > > STATES: > all udp 172.16.3.255:137 <- 172.16.3.17:137 NO_TRAFFIC:SINGLE > all udp 172.16.3.17:137 -> 204.244.159.68:57827 -> 172.16.3.255:137 > SINGLE:NO_TRAFFIC > all tcp 10.170.54.1:81 <- 172.16.3.71:3064 CLOSED:SYN_SENT > all tcp 172.16.3.71:3064 -> 204.244.159.55:56563 -> 10.170.54.1:81 > SYN_SENT:CLOSED > all tcp 10.170.54.1:81 <- 172.16.3.30:2021 CLOSED:SYN_SENT > all tcp 172.16.3.30:2021 -> 204.244.159.68:54557 -> 10.170.54.1:81 > SYN_SENT:CLOSED > all tcp 10.170.54.1:81 <- 172.16.3.72:1414 CLOSED:SYN_SENT > all tcp 172.16.3.72:1414 -> 204.244.159.55:52567 -> 10.170.54.1:81 > SYN_SENT:CLOSED > all tcp 10.170.54.1:81 <- 172.16.3.31:2865 CLOSED:SYN_SENT > all tcp 172.16.3.31:2865 -> 204.244.159.68:59429 -> 10.170.54.1:81 > SYN_SENT:CLOSED > all tcp 10.170.54.1:81 <- 172.16.3.72:1415 CLOSED:SYN_SENT > all tcp 172.16.3.72:1415 -> 204.244.159.55:61425 -> 10.170.54.1:81 > SYN_SENT:CLOSED > all tcp 10.170.54.1:81 <- 172.16.3.49:1914 CLOSED:SYN_SENT > all tcp 172.16.3.49:1914 -> 204.244.159.68:58532 -> 10.170.54.1:81 > SYN_SENT:CLOSED > all udp 172.16.3.255:138 <- 172.16.3.39:138 NO_TRAFFIC:SINGLE > all udp 172.16.3.39:138 -> 204.244.159.68:62224 -> 172.16.3.255:138 > SINGLE:NO_TRAFFIC > all tcp 64.56.145.72:110 <- 172.16.3.48:1494 FIN_WAIT_2:FIN_WAIT_2 > all tcp 172.16.3.48:1494 -> 204.244.159.55:62928 -> 64.56.145.72:110 > FIN_WAIT_2:FIN_WAIT_2 > all udp 172.16.3.255:137 <- 172.16.3.49:137 NO_TRAFFIC:SINGLE > all udp 172.16.3.49:137 -> 204.244.159.55:61053 -> 172.16.3.255:137 > SINGLE:NO_TRAFFIC > all tcp 10.170.54.1:81 <- 172.16.3.37:1508 CLOSED:SYN_SENT > all tcp 172.16.3.37:1508 -> 204.244.159.68:54656 -> 10.170.54.1:81 > SYN_SENT:CLOSED > all tcp 10.170.54.1:81 <- 172.16.3.74:3126 CLOSED:SYN_SENT > all tcp 172.16.3.74:3126 -> 204.244.159.55:61282 -> 10.170.54.1:81 > SYN_SENT:CLOSED > all tcp 10.170.54.1:81 <- 172.16.3.18:2446 CLOSED:SYN_SENT > all tcp 172.16.3.18:2446 -> 204.244.159.68:58385 -> 10.170.54.1:81 > SYN_SENT:CLOSED > all tcp 10.170.54.1:81 <- 172.16.3.73:2057 CLOSED:SYN_SENT > all tcp 172.16.3.73:2057 -> 204.244.159.55:61692 -> 10.170.54.1:81 > SYN_SENT:CLOSED > all udp 198.208.22.27:53 <- 172.16.3.74:58071 SINGLE:MULTIPLE > all udp 172.16.3.74:58071 -> 204.244.159.68:54669 -> 198.208.22.27:53 > MULTIPLE:SINGLE > all udp 198.208.22.27:53 <- 172.16.3.74:57503 SINGLE:MULTIPLE > all udp 172.16.3.74:57503 -> 204.244.159.55:64923 -> 198.208.22.27:53 > MULTIPLE:SINGLE > all udp 198.208.22.27:53 <- 172.16.3.74:51153 SINGLE:MULTIPLE > all udp 172.16.3.74:51153 -> 204.244.159.68:61637 -> 198.208.22.27:53 > MULTIPLE:SINGLE > all udp 172.16.3.255:137 <- 172.16.3.74:137 NO_TRAFFIC:SINGLE > all udp 172.16.3.74:137 -> 204.244.159.55:53474 -> 172.16.3.255:137 > SINGLE:NO_TRAFFIC > all tcp 10.170.54.1:81 <- 172.16.3.71:3065 CLOSED:SYN_SENT > all tcp 172.16.3.71:3065 -> 204.244.159.68:63354 -> 10.170.54.1:81 > SYN_SENT:CLOSED > all tcp 10.170.54.1:81 <- 172.16.3.29:4434 CLOSED:SYN_SENT > all tcp 172.16.3.29:4434 -> 204.244.159.55:62977 -> 10.170.54.1:81 > SYN_SENT:CLOSED > all udp 172.16.3.255:137 <- 172.16.3.30:137 NO_TRAFFIC:SINGLE > all udp 172.16.3.30:137 -> 204.244.159.68:61298 -> 172.16.3.255:137 > SINGLE:NO_TRAFFIC > all tcp 63.241.234.60:443 <- 172.16.3.37:1509 > ESTABLISHED:ESTABLISHED > all tcp 172.16.3.37:1509 -> 204.244.159.68:61873 -> 63.241.234.60:443 > ESTABLISHED:ESTABLISHED > all udp 198.208.22.27:53 <- 172.16.3.72:59314 SINGLE:MULTIPLE > all udp 172.16.3.72:59314 -> 204.244.159.55:62186 -> 198.208.22.27:53 > MULTIPLE:SINGLE > all udp 198.208.22.27:53 <- 172.16.3.72:55934 SINGLE:MULTIPLE > all udp 172.16.3.72:55934 -> 204.244.159.68:51479 -> 198.208.22.27:53 > MULTIPLE:SINGLE > all udp 198.208.22.27:53 <- 172.16.3.72:52983 SINGLE:MULTIPLE > all udp 172.16.3.72:52983 -> 204.244.159.55:55523 -> 198.208.22.27:53 > MULTIPLE:SINGLE > all udp 172.16.3.255:137 <- 172.16.3.72:137 NO_TRAFFIC:SINGLE > all udp 172.16.3.72:137 -> 204.244.159.68:58218 -> 172.16.3.255:137 > SINGLE:NO_TRAFFIC > all tcp 10.170.54.1:81 <- 172.16.3.31:2868 CLOSED:SYN_SENT > all tcp 172.16.3.31:2868 -> 204.244.159.55:60911 -> 10.170.54.1:81 > SYN_SENT:CLOSED > all udp 172.16.3.255:137 <- 172.16.3.77:137 NO_TRAFFIC:SINGLE > all udp 172.16.3.77:137 -> 204.244.159.55:59287 -> 172.16.3.255:137 > SINGLE:NO_TRAFFIC > all tcp 10.170.54.1:81 <- 172.16.3.72:1416 CLOSED:SYN_SENT > all tcp 172.16.3.72:1416 -> 204.244.159.68:59828 -> 10.170.54.1:81 > SYN_SENT:CLOSED > all tcp 10.170.54.1:81 <- 172.16.3.49:1915 CLOSED:SYN_SENT > all tcp 172.16.3.49:1915 -> 204.244.159.55:64580 -> 10.170.54.1:81 > SYN_SENT:CLOSED > all tcp 10.170.54.1:81 <- 172.16.3.29:4435 CLOSED:SYN_SENT > all tcp 172.16.3.29:4435 -> 204.244.159.68:60089 -> 10.170.54.1:81 > SYN_SENT:CLOSED > all udp 172.16.3.255:137 <- 172.16.3.8:137 NO_TRAFFIC:SINGLE > all udp 172.16.3.8:137 -> 204.244.159.68:60176 -> 172.16.3.255:137 > SINGLE:NO_TRAFFIC > all tcp 10.170.54.1:81 <- 172.16.3.51:3433 CLOSED:SYN_SENT > all tcp 172.16.3.51:3433 -> 204.244.159.55:63158 -> 10.170.54.1:81 > SYN_SENT:CLOSED > all tcp 10.170.54.1:81 <- 172.16.3.37:1510 CLOSED:SYN_SENT > all tcp 172.16.3.37:1510 -> 204.244.159.68:63197 -> 10.170.54.1:81 > SYN_SENT:CLOSED > all tcp 10.170.54.1:81 <- 172.16.3.74:3127 CLOSED:SYN_SENT > all tcp 172.16.3.74:3127 -> 204.244.159.55:61760 -> 10.170.54.1:81 > SYN_SENT:CLOSED > all tcp 10.170.54.1:81 <- 172.16.3.18:2447 CLOSED:SYN_SENT > all tcp 172.16.3.18:2447 -> 204.244.159.68:61951 -> 10.170.54.1:81 > SYN_SENT:CLOSED > all tcp 10.170.54.1:81 <- 172.16.3.73:2058 CLOSED:SYN_SENT > all tcp 172.16.3.73:2058 -> 204.244.159.55:53396 -> 10.170.54.1:81 > SYN_SENT:CLOSED > all udp 198.208.22.27:53 <- 172.16.3.74:62024 SINGLE:MULTIPLE > all udp 172.16.3.74:62024 -> 204.244.159.55:63136 -> 198.208.22.27:53 > MULTIPLE:SINGLE > all tcp 72.14.162.41:80 <- 172.16.3.74:3128 TIME_WAIT:TIME_WAIT > all tcp 172.16.3.74:3128 -> 204.244.159.68:58088 -> 72.14.162.41:80 > TIME_WAIT:TIME_WAIT > all tcp 72.14.162.41:80 <- 172.16.3.74:3129 FIN_WAIT_2:FIN_WAIT_2 > all tcp 172.16.3.74:3129 -> 204.244.159.55:62718 -> 72.14.162.41:80 > FIN_WAIT_2:FIN_WAIT_2 > all udp 172.16.3.255:138 <- 172.16.3.71:138 NO_TRAFFIC:SINGLE > all udp 172.16.3.71:138 -> 204.244.159.68:52993 -> 172.16.3.255:138 > SINGLE:NO_TRAFFIC > all tcp 10.170.54.1:81 <- 172.16.3.71:3066 CLOSED:SYN_SENT > all tcp 172.16.3.71:3066 -> 204.244.159.68:50898 -> 10.170.54.1:81 > SYN_SENT:CLOSED > Ooops, all TCP states are closed/dead ? No new TCP flows registered ? You better check with "pfctl -v -s state" what's going on there, what's the aging and expiring times... Regards, Adrian.