From owner-freebsd-pf@FreeBSD.ORG Sun Jan 2 17:50:43 2011 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F06D61065673 for ; Sun, 2 Jan 2011 17:50:43 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 7C6CD8FC08 for ; Sun, 2 Jan 2011 17:50:43 +0000 (UTC) Received: by wyf19 with SMTP id 19so12715953wyf.13 for ; Sun, 02 Jan 2011 09:50:42 -0800 (PST) Received: by 10.227.20.78 with SMTP id e14mr11347368wbb.175.1293990640545; Sun, 02 Jan 2011 09:50:40 -0800 (PST) Received: from dfleuriot.local (did75-17-88-165-130-96.fbx.proxad.net [88.165.130.96]) by mx.google.com with ESMTPS id q18sm13471145wbe.5.2011.01.02.09.50.37 (version=SSLv3 cipher=RC4-MD5); Sun, 02 Jan 2011 09:50:39 -0800 (PST) Message-ID: <4D20BAEB.10101@my.gd> Date: Sun, 02 Jan 2011 18:50:35 +0100 From: Damien Fleuriot User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: jay@experts-exchange.com References: <8fb3caa1300a9fcc5c2f23a70ade23a8.squirrel@mail.experts-exchange.com> <4D208AE2.6000402@my.gd> <3020c1e8b0ecb5e9bacb1033ddea2b3e.squirrel@mail.experts-exchange.com> In-Reply-To: <3020c1e8b0ecb5e9bacb1033ddea2b3e.squirrel@mail.experts-exchange.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-pf@freebsd.org Subject: Re: transparent proxy X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Jan 2011 17:50:44 -0000 In other software such as HTTP that you took for example, there's this special X-Forwarded-For header which covers this very need. IMO you shouldn't have to tweak around with the firewall or the IP stack to make up for a missing capability but nvm. Perhaps these 2 PF rules would be of use to you: route-to The route-to option routes the packet to the specified interface with an optional address for the next hop. When a route-to rule creates state, only packets that pass in the same direction as the filter rule specifies will be routed in this way. Packets passing in the opposite direction (replies) are not affected and are routed normally. reply-to The reply-to option is similar to route-to, but routes packets that pass in the opposite direction (replies) to the specified inter- face. Opposite direction is only defined in the context of a state entry, and reply-to is useful only in rules that create state. It can be used on systems with multiple external connections to route all outgoing packets of a connection through the interface the incoming connection arrived through (symmetric routing enforce- ment). On 1/2/11 6:37 PM, jay@experts-exchange.com wrote: > Hi Damien, > > Here I am using HTTP traffic as an illustration, but for other generic > services without the built in SSL layer, it would be highly advantageous > to be able to add stunnel to do the job. The target application (e.g. VNC, > database client/server connection, and so on) need not be re-coded. > Running stunnel as an intermediate layer has the side effect of connecting > to the internal services from a local IP address. In this configuration, > it becomes a proxy service that takes the incoming network communication, > filters out the SSL traffic and passes along the service payload as well > as back again. Stunnel is being used for SSL/TLS connectivity. The > "transparent" setting, available on Linux platforms, would bind() to a > foreign address prior calling connect(), issue an IP_TPROXY_ASSIGN > setsockopt to register the local address as a proxy, and use iptables and > ip routing rules to keep track of the forward and reverse routing. To the > service on the other side, it appears the client network traffic > originates from a foreign IP address. We patched stunnel for v.8.1 of > FreeBSD to use setsockopt IP_BINDANY, and thus preserve the foreign > address, but for the service on the other side, it tries to establish the > connection and fails because the routing is not set up to return the > traffic to stunnel. Without transparent mode, network communication is : > > ext ip -> stunnel -> int ip -> http service > > For int ip -> http svc, real tcpdump traffic is : > > 00:26:53.435415 IP 127.0.0.1.30655 > 127.0.0.1.80: Flags [P.], ack 1, ... > > On the other side, e.g. the return path: > > http -> int ip -> stunnel -> ext ip > > At the http svc, packets are sent as : > > 00:26:53.435864 IP 127.0.0.1.80 > 127.0.0.1.30655: Flags [P.], ack 7, ... > > This works perfectly, but according the apache logs, the request looks > like they are all originate from the internal '127.0.0.1' address. Now > with transparent mode, it is : > > ext ip -> stunnel -> ext ip -> http > > 00:40:28.313038 IP 192.168.103.2.51791 > 127.0.0.1.80: Flags [S], ack ... > > But the return path is bad, and the network connection is stuck in a SYNC > state. > > http X-> ext ip > > How can I create PF rules to create a reverse rule then to say if > (hypothetically) : > > http -> ext ip (proxy/rdr) -> stunnel -> ext ip > > nn:nn:nn.nnnnnn IP 127.0.0.1.51791 > 192.168.103.2.80: Flags [S], ack ... > > In this case it would look like a REVERSE proxy of "internal 127.x.x.x, > port any (except port 80) to the 'external' 192.x.x.x port 80" to > "internal 127.x.x.x, port any to 127.x.x.x port 80". This is what I tried > unsuccessfully to create with the rdr rule. > > rdr on $int_if inet proto tcp from $int_if to any port 80 -> 127.0.0.1 > port 80 > > Really I am trying to reproduce the Linux version of the kernel level > DIVERT proxy filters, so my version is not capturing that function. From > my lack of understanding of PF, I don't know that it can work this way. > > Thanks > >> I'm not sure what you're trying to achieve here. >> >> Are you actually using proxy software at all, or only a PF redirect rule ? >> >> Are you trying to set up a FORWARD or a REVERSE proxy ? >> >> What do you use stunnel for, SSL/TLS connectivity ? >> >> >> On 1/2/11 5:38 AM, jay@experts-exchange.com wrote: >>> Folks, >>> >>> I am trying to use stunnel & pf to devise a transparent proxy, but am >>> unable to figure out how to do it. What I have is ext ip -> stunnel -> >>> http service, but the http service does not know where to route back the >>> packets, and remains in a sync state. >>> >>> >>> 00:40:28.313038 IP 192.168.103.2.51791 > 127.0.0.1.80: Flags [S], seq >>> 2806128000, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val >>> 2027735 ecr 0], length 0 >>> 00:40:31.306553 IP 192.168.103.2.51791 > 127.0.0.1.80: Flags [S], seq >>> 2806128000, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val >>> 2028035 ecr 0], length 0 >>> 00:40:34.506518 IP 192.168.103.2.51791 > 127.0.0.1.80: Flags [S], seq >>> 2806128000, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val >>> 2028355 ecr 0], length 0 >>> 00:40:37.706528 IP 192.168.103.2.51791 > 127.0.0.1.80: Flags [S], seq >>> 2806128000, win 65535, options [mss 16344,sackOK,eol], length 0 >>> >>> rpminit# netstat -ln >>> Active Internet connections >>> Proto Recv-Q Send-Q Local Address Foreign Address >>> (state) >>> tcp4 0 0 192.168.103.2.51218 127.0.0.1.80 >>> SYN_SENT >>> tcp4 0 0 192.168.103.62.443 192.168.103.2.51218 >>> ESTABLISHED >>> >>> >>> If I disable the transparent config setting the communication is works, >>> but http access logs show the request coming from local host. >>> >>> 00:26:53.435415 IP 127.0.0.1.30655 > 127.0.0.1.80: Flags [P.], ack 1, >>> win >>> 8960, options [nop,nop,TS val 1946248 ecr 3625203070], length 6 >>> 00:26:53.435864 IP 127.0.0.1.80 > 127.0.0.1.30655: Flags [P.], ack 7, >>> win >>> 8960, options [nop,nop,TS val 3625203735 ecr 1946248], length 44 >>> 00:26:53.436426 IP 127.0.0.1.80 > 127.0.0.1.30655: Flags [F.], seq 45, >>> ack >>> 7, win 8960, options [nop,nop,TS val 3625203735 ecr 1946248], length 0 >>> 00:26:53.436463 IP 127.0.0.1.30655 > 127.0.0.1.80: Flags [.], ack 46, >>> win >>> 8960, options [nop,nop,TS val 1946248 ecr 3625203735], length 0 >>> 00:26:53.526062 IP 127.0.0.1.30655 > 127.0.0.1.80: Flags [F.], seq 7, >>> ack >>> 46, win 8960, options [nop,nop,TS val 1946257 ecr 3625203735], length 0 >>> 00:26:53.526112 IP 127.0.0.1.80 > 127.0.0.1.30655: Flags [.], ack 8, win >>> 8959, options [nop,nop,TS val 3625203744 ecr 1946257], length 0 >>> 00:28:03.523841 IP 127.0.0.1.47994 > 127.0.0.1.80: Flags [S], seq >>> 1128551040, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val >>> 1953257 ecr 0], length 0 >>> 00:28:03.523924 IP 127.0.0.1.80 > 127.0.0.1.47994: Flags [S.], seq >>> 4120370047, ack 1128551041, win 65535, options [mss 16344,nop,wscale >>> 3,sackOK,TS val 727165180 ecr 1953257], length 0 >>> 00:28:03.523942 IP 127.0.0.1.47994 > 127.0.0.1.80: Flags [.], ack 1, win >>> 8960, options [nop,nop,TS val 1953257 ecr 727165180], length 0 >>> 00:28:05.254567 IP 127.0.0.1.47994 > 127.0.0.1.80: Flags [P.], ack 1, >>> win >>> 8960, options [nop,nop,TS val 1953430 ecr 727165180], length 6 >>> 00:28:05.254888 IP 127.0.0.1.80 > 127.0.0.1.47994: Flags [P.], ack 7, >>> win >>> 8960, options [nop,nop,TS val 727165353 ecr 1953430], length 44 >>> 00:28:05.255194 IP 127.0.0.1.80 > 127.0.0.1.47994: Flags [F.], seq 45, >>> ack >>> 7, win 8960, options [nop,nop,TS val 727165353 ecr 1953430], length 0 >>> 00:28:05.255234 IP 127.0.0.1.47994 > 127.0.0.1.80: Flags [.], ack 46, >>> win >>> 8960, options [nop,nop,TS val 1953430 ecr 727165353], length 0 >>> 00:28:05.408742 IP 127.0.0.1.47994 > 127.0.0.1.80: Flags [F.], seq 7, >>> ack >>> 46, win 8960, options [nop,nop,TS val 1953445 ecr 727165353], length 0 >>> 00:28:05.408799 IP 127.0.0.1.80 > 127.0.0.1.47994: Flags [.], ack 8, win >>> 8959, options [nop,nop,TS val 727165368 ecr 1953445], length 0 >>> 00:28:59.372253 IP 192.168.103.2.60900 > 127.0.0.1.80: Flags [S], seq >>> 2362825029, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val >>> 1958842 ecr 0], length 0 >>> 00:29:02.371384 IP 192.168.103.2.60900 > 127.0.0.1.80: Flags [S], seq >>> 2362825029, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val >>> 1959142 ecr 0], length 0 >>> >>> >>> ==> /var/log/httpd-access.log <== >>> 127.0.0.1 - - [01/Jan/2011:23:18:44 -0800] "GET /" 200 44 "-" "-" >>> 127.0.0.1 - - [01/Jan/2011:23:18:53 -0800] "GET /" 200 44 "-" "-" >>> 127.0.0.1 - - [01/Jan/2011:23:21:48 -0800] "GET /" 200 44 "-" "-" >>> >>> I've tried to set up a rdr rule to redirect requests from internal ip to >>> external port 80 to internal port 80, but no luck. >>> >>> /etc/pf.conf >>> >>> int_if="lo0" >>> ext_if="ed0" >>> >>> rdr on $int_if inet proto tcp from $int_if to any port 80 -> 127.0.0.1 >>> port 80 >>> >>> Under Linux, it is possible to set up rules to perform internal proxy >>> diverting thereby, "Re-write address to appear as if wrapped daemon is >>> connecting from the SSL client machine instead of the machine running >>> stunnel." See also http://www.stunnel.org/faq/stunnel.html and >>> http://www.stunnel.org/faq/transparent.html. >>> >>> iptables -t mangle -N DIVERT >>> iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT >>> iptables -t mangle -A DIVERT -j MARK --set-mark 1 >>> iptables -t mangle -A DIVERT -j ACCEPT >>> ip rule add fwmark 1 lookup 100 >>> ip route add local 0.0.0.0/0 dev lo table 100 >>> >>> Is there a way to devise pf rdr rules to do the same? >>> >>> Thanks > >