Date: Fri, 27 Feb 2009 08:54:24 +0100 From: Adrian Penisoara <ady@freebsd.ady.ro> To: Shawn Everett <shawn@tandac.com> Cc: freebsd-net@freebsd.org Subject: Re: FreeBSD Router Problem Message-ID: <78cb3d3f0902262354j6c22b43do565b75523a8007e3@mail.gmail.com> In-Reply-To: <200902262341.35069.shawn@tandac.com> References: <3650.206.108.16.89.1235691792.squirrel@alder.hosix.com> <3853.206.108.16.89.1235693214.squirrel@alder.hosix.com> <78cb3d3f0902261619t71a054fet43779c37e2981603@mail.gmail.com> <200902262341.35069.shawn@tandac.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi,
On Fri, Feb 27, 2009 at 8:41 AM, Shawn Everett <shawn@tandac.com> wrote:
> > Any error messages in dmesg output ?
> > Significant changes in "netstat -m" output before and after ?
> > The same for "pfctl -s all" output...
>
> The box has been up for about 12 hours now. As a point of discussion here
> is the output from netstat and pfctl in case anything obvious jumps out.
>
> 385/905/1290 mbufs in use (current/cache/total)
> 384/484/868/25600 mbuf clusters in use (current/cache/total/max)
> 256/384 mbuf+clusters out of packet secondary zone in use (current/cache)
> 0/44/44/12800 4k (page size) jumbo clusters in use
> (current/cache/total/max)
> 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max)
> 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max)
> 864K/1370K/2234K bytes allocated to network (current/cache/total)
> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
> 0/0/0 requests for jumbo clusters denied (4k/9k/16k)
> 0/5/6656 sfbufs in use (current/peak/max)
> 0 requests for sfbufs denied
> 0 requests for sfbufs delayed
> 0 requests for I/O initiated by sendfile
> 0 calls to protocol drain routines
>
This looks OK to me...
>
>
> # pfctl -s all
> No ALTQ support in kernel
> ALTQ related functions disabled
> TRANSLATION RULES:
> nat on ste0 inet from 172.16.3.0/24 to any -> (ste0) round-robin
> nat on ste1 inet from 172.16.3.0/24 to any -> (ste1) round-robin
>
> FILTER RULES:
> pass out on em0 inet from any to 172.16.3.0/24 flags S/SA keep state
> pass in quick on em0 inet from 172.16.3.0/24 to 172.16.3.253 flags S/SA
> keep state
> pass in on em0 route-to { (ste0 204.244.159.254), (ste1 204.244.159.254) }
> round-robin inet proto tcp from 172.16.3.0/24 to any flags S/SA modulate
> state
> pass in on em0 route-to { (ste0 204.244.159.254), (ste1 204.244.159.254) }
> round-robin inet proto udp from 172.16.3.0/24 to any keep state
> pass in on em0 route-to { (ste0 204.244.159.254), (ste1 204.244.159.254) }
> round-robin inet proto icmp from 172.16.3.0/24 to any keep state
> pass out on ste0 proto tcp all flags S/SA modulate state
> pass out on ste0 proto udp all keep state
> pass out on ste0 proto icmp all keep state
> pass out on ste1 proto tcp all flags S/SA modulate state
> pass out on ste1 proto udp all keep state
> pass out on ste1 proto icmp all keep state
> pass out on ste0 route-to (ste1 204.244.159.254) inet from 204.244.159.55
> to any flags S/SA keep state
> pass out on ste1 route-to (ste0 204.244.159.254) inet from 204.244.159.68
> to any flags S/SA keep state
>
Quite an evolved route forwarding setup ;)...
>
> STATES:
> all udp 172.16.3.255:137 <- 172.16.3.17:137 NO_TRAFFIC:SINGLE
> all udp 172.16.3.17:137 -> 204.244.159.68:57827 -> 172.16.3.255:137
> SINGLE:NO_TRAFFIC
> all tcp 10.170.54.1:81 <- 172.16.3.71:3064 CLOSED:SYN_SENT
> all tcp 172.16.3.71:3064 -> 204.244.159.55:56563 -> 10.170.54.1:81
> SYN_SENT:CLOSED
> all tcp 10.170.54.1:81 <- 172.16.3.30:2021 CLOSED:SYN_SENT
> all tcp 172.16.3.30:2021 -> 204.244.159.68:54557 -> 10.170.54.1:81
> SYN_SENT:CLOSED
> all tcp 10.170.54.1:81 <- 172.16.3.72:1414 CLOSED:SYN_SENT
> all tcp 172.16.3.72:1414 -> 204.244.159.55:52567 -> 10.170.54.1:81
> SYN_SENT:CLOSED
> all tcp 10.170.54.1:81 <- 172.16.3.31:2865 CLOSED:SYN_SENT
> all tcp 172.16.3.31:2865 -> 204.244.159.68:59429 -> 10.170.54.1:81
> SYN_SENT:CLOSED
> all tcp 10.170.54.1:81 <- 172.16.3.72:1415 CLOSED:SYN_SENT
> all tcp 172.16.3.72:1415 -> 204.244.159.55:61425 -> 10.170.54.1:81
> SYN_SENT:CLOSED
> all tcp 10.170.54.1:81 <- 172.16.3.49:1914 CLOSED:SYN_SENT
> all tcp 172.16.3.49:1914 -> 204.244.159.68:58532 -> 10.170.54.1:81
> SYN_SENT:CLOSED
> all udp 172.16.3.255:138 <- 172.16.3.39:138 NO_TRAFFIC:SINGLE
> all udp 172.16.3.39:138 -> 204.244.159.68:62224 -> 172.16.3.255:138
> SINGLE:NO_TRAFFIC
> all tcp 64.56.145.72:110 <- 172.16.3.48:1494 FIN_WAIT_2:FIN_WAIT_2
> all tcp 172.16.3.48:1494 -> 204.244.159.55:62928 -> 64.56.145.72:110
> FIN_WAIT_2:FIN_WAIT_2
> all udp 172.16.3.255:137 <- 172.16.3.49:137 NO_TRAFFIC:SINGLE
> all udp 172.16.3.49:137 -> 204.244.159.55:61053 -> 172.16.3.255:137
> SINGLE:NO_TRAFFIC
> all tcp 10.170.54.1:81 <- 172.16.3.37:1508 CLOSED:SYN_SENT
> all tcp 172.16.3.37:1508 -> 204.244.159.68:54656 -> 10.170.54.1:81
> SYN_SENT:CLOSED
> all tcp 10.170.54.1:81 <- 172.16.3.74:3126 CLOSED:SYN_SENT
> all tcp 172.16.3.74:3126 -> 204.244.159.55:61282 -> 10.170.54.1:81
> SYN_SENT:CLOSED
> all tcp 10.170.54.1:81 <- 172.16.3.18:2446 CLOSED:SYN_SENT
> all tcp 172.16.3.18:2446 -> 204.244.159.68:58385 -> 10.170.54.1:81
> SYN_SENT:CLOSED
> all tcp 10.170.54.1:81 <- 172.16.3.73:2057 CLOSED:SYN_SENT
> all tcp 172.16.3.73:2057 -> 204.244.159.55:61692 -> 10.170.54.1:81
> SYN_SENT:CLOSED
> all udp 198.208.22.27:53 <- 172.16.3.74:58071 SINGLE:MULTIPLE
> all udp 172.16.3.74:58071 -> 204.244.159.68:54669 -> 198.208.22.27:53
> MULTIPLE:SINGLE
> all udp 198.208.22.27:53 <- 172.16.3.74:57503 SINGLE:MULTIPLE
> all udp 172.16.3.74:57503 -> 204.244.159.55:64923 -> 198.208.22.27:53
> MULTIPLE:SINGLE
> all udp 198.208.22.27:53 <- 172.16.3.74:51153 SINGLE:MULTIPLE
> all udp 172.16.3.74:51153 -> 204.244.159.68:61637 -> 198.208.22.27:53
> MULTIPLE:SINGLE
> all udp 172.16.3.255:137 <- 172.16.3.74:137 NO_TRAFFIC:SINGLE
> all udp 172.16.3.74:137 -> 204.244.159.55:53474 -> 172.16.3.255:137
> SINGLE:NO_TRAFFIC
> all tcp 10.170.54.1:81 <- 172.16.3.71:3065 CLOSED:SYN_SENT
> all tcp 172.16.3.71:3065 -> 204.244.159.68:63354 -> 10.170.54.1:81
> SYN_SENT:CLOSED
> all tcp 10.170.54.1:81 <- 172.16.3.29:4434 CLOSED:SYN_SENT
> all tcp 172.16.3.29:4434 -> 204.244.159.55:62977 -> 10.170.54.1:81
> SYN_SENT:CLOSED
> all udp 172.16.3.255:137 <- 172.16.3.30:137 NO_TRAFFIC:SINGLE
> all udp 172.16.3.30:137 -> 204.244.159.68:61298 -> 172.16.3.255:137
> SINGLE:NO_TRAFFIC
> all tcp 63.241.234.60:443 <- 172.16.3.37:1509
> ESTABLISHED:ESTABLISHED
> all tcp 172.16.3.37:1509 -> 204.244.159.68:61873 -> 63.241.234.60:443
> ESTABLISHED:ESTABLISHED
> all udp 198.208.22.27:53 <- 172.16.3.72:59314 SINGLE:MULTIPLE
> all udp 172.16.3.72:59314 -> 204.244.159.55:62186 -> 198.208.22.27:53
> MULTIPLE:SINGLE
> all udp 198.208.22.27:53 <- 172.16.3.72:55934 SINGLE:MULTIPLE
> all udp 172.16.3.72:55934 -> 204.244.159.68:51479 -> 198.208.22.27:53
> MULTIPLE:SINGLE
> all udp 198.208.22.27:53 <- 172.16.3.72:52983 SINGLE:MULTIPLE
> all udp 172.16.3.72:52983 -> 204.244.159.55:55523 -> 198.208.22.27:53
> MULTIPLE:SINGLE
> all udp 172.16.3.255:137 <- 172.16.3.72:137 NO_TRAFFIC:SINGLE
> all udp 172.16.3.72:137 -> 204.244.159.68:58218 -> 172.16.3.255:137
> SINGLE:NO_TRAFFIC
> all tcp 10.170.54.1:81 <- 172.16.3.31:2868 CLOSED:SYN_SENT
> all tcp 172.16.3.31:2868 -> 204.244.159.55:60911 -> 10.170.54.1:81
> SYN_SENT:CLOSED
> all udp 172.16.3.255:137 <- 172.16.3.77:137 NO_TRAFFIC:SINGLE
> all udp 172.16.3.77:137 -> 204.244.159.55:59287 -> 172.16.3.255:137
> SINGLE:NO_TRAFFIC
> all tcp 10.170.54.1:81 <- 172.16.3.72:1416 CLOSED:SYN_SENT
> all tcp 172.16.3.72:1416 -> 204.244.159.68:59828 -> 10.170.54.1:81
> SYN_SENT:CLOSED
> all tcp 10.170.54.1:81 <- 172.16.3.49:1915 CLOSED:SYN_SENT
> all tcp 172.16.3.49:1915 -> 204.244.159.55:64580 -> 10.170.54.1:81
> SYN_SENT:CLOSED
> all tcp 10.170.54.1:81 <- 172.16.3.29:4435 CLOSED:SYN_SENT
> all tcp 172.16.3.29:4435 -> 204.244.159.68:60089 -> 10.170.54.1:81
> SYN_SENT:CLOSED
> all udp 172.16.3.255:137 <- 172.16.3.8:137 NO_TRAFFIC:SINGLE
> all udp 172.16.3.8:137 -> 204.244.159.68:60176 -> 172.16.3.255:137
> SINGLE:NO_TRAFFIC
> all tcp 10.170.54.1:81 <- 172.16.3.51:3433 CLOSED:SYN_SENT
> all tcp 172.16.3.51:3433 -> 204.244.159.55:63158 -> 10.170.54.1:81
> SYN_SENT:CLOSED
> all tcp 10.170.54.1:81 <- 172.16.3.37:1510 CLOSED:SYN_SENT
> all tcp 172.16.3.37:1510 -> 204.244.159.68:63197 -> 10.170.54.1:81
> SYN_SENT:CLOSED
> all tcp 10.170.54.1:81 <- 172.16.3.74:3127 CLOSED:SYN_SENT
> all tcp 172.16.3.74:3127 -> 204.244.159.55:61760 -> 10.170.54.1:81
> SYN_SENT:CLOSED
> all tcp 10.170.54.1:81 <- 172.16.3.18:2447 CLOSED:SYN_SENT
> all tcp 172.16.3.18:2447 -> 204.244.159.68:61951 -> 10.170.54.1:81
> SYN_SENT:CLOSED
> all tcp 10.170.54.1:81 <- 172.16.3.73:2058 CLOSED:SYN_SENT
> all tcp 172.16.3.73:2058 -> 204.244.159.55:53396 -> 10.170.54.1:81
> SYN_SENT:CLOSED
> all udp 198.208.22.27:53 <- 172.16.3.74:62024 SINGLE:MULTIPLE
> all udp 172.16.3.74:62024 -> 204.244.159.55:63136 -> 198.208.22.27:53
> MULTIPLE:SINGLE
> all tcp 72.14.162.41:80 <- 172.16.3.74:3128 TIME_WAIT:TIME_WAIT
> all tcp 172.16.3.74:3128 -> 204.244.159.68:58088 -> 72.14.162.41:80
> TIME_WAIT:TIME_WAIT
> all tcp 72.14.162.41:80 <- 172.16.3.74:3129 FIN_WAIT_2:FIN_WAIT_2
> all tcp 172.16.3.74:3129 -> 204.244.159.55:62718 -> 72.14.162.41:80
> FIN_WAIT_2:FIN_WAIT_2
> all udp 172.16.3.255:138 <- 172.16.3.71:138 NO_TRAFFIC:SINGLE
> all udp 172.16.3.71:138 -> 204.244.159.68:52993 -> 172.16.3.255:138
> SINGLE:NO_TRAFFIC
> all tcp 10.170.54.1:81 <- 172.16.3.71:3066 CLOSED:SYN_SENT
> all tcp 172.16.3.71:3066 -> 204.244.159.68:50898 -> 10.170.54.1:81
> SYN_SENT:CLOSED
>
Ooops, all TCP states are closed/dead ? No new TCP flows registered ?
You better check with "pfctl -v -s state" what's going on there, what's the
aging and expiring times...
Regards,
Adrian.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?78cb3d3f0902262354j6c22b43do565b75523a8007e3>