Date: Sun, 2 Jan 2011 09:37:34 -0800 From: jay@experts-exchange.com To: "Damien Fleuriot" <ml@my.gd> Cc: freebsd-pf@freebsd.org Subject: Re: transparent proxy Message-ID: <3020c1e8b0ecb5e9bacb1033ddea2b3e.squirrel@mail.experts-exchange.com> In-Reply-To: <4D208AE2.6000402@my.gd> References: <8fb3caa1300a9fcc5c2f23a70ade23a8.squirrel@mail.experts-exchange.com> <4D208AE2.6000402@my.gd>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Damien, Here I am using HTTP traffic as an illustration, but for other generic services without the built in SSL layer, it would be highly advantageous to be able to add stunnel to do the job. The target application (e.g. VNC, database client/server connection, and so on) need not be re-coded. Running stunnel as an intermediate layer has the side effect of connecting to the internal services from a local IP address. In this configuration, it becomes a proxy service that takes the incoming network communication, filters out the SSL traffic and passes along the service payload as well as back again. Stunnel is being used for SSL/TLS connectivity. The "transparent" setting, available on Linux platforms, would bind() to a foreign address prior calling connect(), issue an IP_TPROXY_ASSIGN setsockopt to register the local address as a proxy, and use iptables and ip routing rules to keep track of the forward and reverse routing. To the service on the other side, it appears the client network traffic originates from a foreign IP address. We patched stunnel for v.8.1 of FreeBSD to use setsockopt IP_BINDANY, and thus preserve the foreign address, but for the service on the other side, it tries to establish the connection and fails because the routing is not set up to return the traffic to stunnel. Without transparent mode, network communication is : ext ip -> stunnel -> int ip -> http service For int ip -> http svc, real tcpdump traffic is : 00:26:53.435415 IP 127.0.0.1.30655 > 127.0.0.1.80: Flags [P.], ack 1, ... On the other side, e.g. the return path: http -> int ip -> stunnel -> ext ip At the http svc, packets are sent as : 00:26:53.435864 IP 127.0.0.1.80 > 127.0.0.1.30655: Flags [P.], ack 7, ... This works perfectly, but according the apache logs, the request looks like they are all originate from the internal '127.0.0.1' address. Now with transparent mode, it is : ext ip -> stunnel -> ext ip -> http 00:40:28.313038 IP 192.168.103.2.51791 > 127.0.0.1.80: Flags [S], ack ... But the return path is bad, and the network connection is stuck in a SYNC state. http X-> ext ip How can I create PF rules to create a reverse rule then to say if (hypothetically) : http -> ext ip (proxy/rdr) -> stunnel -> ext ip nn:nn:nn.nnnnnn IP 127.0.0.1.51791 > 192.168.103.2.80: Flags [S], ack ... In this case it would look like a REVERSE proxy of "internal 127.x.x.x, port any (except port 80) to the 'external' 192.x.x.x port 80" to "internal 127.x.x.x, port any to 127.x.x.x port 80". This is what I tried unsuccessfully to create with the rdr rule. rdr on $int_if inet proto tcp from $int_if to any port 80 -> 127.0.0.1 port 80 Really I am trying to reproduce the Linux version of the kernel level DIVERT proxy filters, so my version is not capturing that function. From my lack of understanding of PF, I don't know that it can work this way. Thanks > I'm not sure what you're trying to achieve here. > > Are you actually using proxy software at all, or only a PF redirect rule ? > > Are you trying to set up a FORWARD or a REVERSE proxy ? > > What do you use stunnel for, SSL/TLS connectivity ? > > > On 1/2/11 5:38 AM, jay@experts-exchange.com wrote: >> Folks, >> >> I am trying to use stunnel & pf to devise a transparent proxy, but am >> unable to figure out how to do it. What I have is ext ip -> stunnel -> >> http service, but the http service does not know where to route back the >> packets, and remains in a sync state. >> >> >> 00:40:28.313038 IP 192.168.103.2.51791 > 127.0.0.1.80: Flags [S], seq >> 2806128000, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val >> 2027735 ecr 0], length 0 >> 00:40:31.306553 IP 192.168.103.2.51791 > 127.0.0.1.80: Flags [S], seq >> 2806128000, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val >> 2028035 ecr 0], length 0 >> 00:40:34.506518 IP 192.168.103.2.51791 > 127.0.0.1.80: Flags [S], seq >> 2806128000, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val >> 2028355 ecr 0], length 0 >> 00:40:37.706528 IP 192.168.103.2.51791 > 127.0.0.1.80: Flags [S], seq >> 2806128000, win 65535, options [mss 16344,sackOK,eol], length 0 >> >> rpminit# netstat -ln >> Active Internet connections >> Proto Recv-Q Send-Q Local Address Foreign Address >> (state) >> tcp4 0 0 192.168.103.2.51218 127.0.0.1.80 >> SYN_SENT >> tcp4 0 0 192.168.103.62.443 192.168.103.2.51218 >> ESTABLISHED >> >> >> If I disable the transparent config setting the communication is works, >> but http access logs show the request coming from local host. >> >> 00:26:53.435415 IP 127.0.0.1.30655 > 127.0.0.1.80: Flags [P.], ack 1, >> win >> 8960, options [nop,nop,TS val 1946248 ecr 3625203070], length 6 >> 00:26:53.435864 IP 127.0.0.1.80 > 127.0.0.1.30655: Flags [P.], ack 7, >> win >> 8960, options [nop,nop,TS val 3625203735 ecr 1946248], length 44 >> 00:26:53.436426 IP 127.0.0.1.80 > 127.0.0.1.30655: Flags [F.], seq 45, >> ack >> 7, win 8960, options [nop,nop,TS val 3625203735 ecr 1946248], length 0 >> 00:26:53.436463 IP 127.0.0.1.30655 > 127.0.0.1.80: Flags [.], ack 46, >> win >> 8960, options [nop,nop,TS val 1946248 ecr 3625203735], length 0 >> 00:26:53.526062 IP 127.0.0.1.30655 > 127.0.0.1.80: Flags [F.], seq 7, >> ack >> 46, win 8960, options [nop,nop,TS val 1946257 ecr 3625203735], length 0 >> 00:26:53.526112 IP 127.0.0.1.80 > 127.0.0.1.30655: Flags [.], ack 8, win >> 8959, options [nop,nop,TS val 3625203744 ecr 1946257], length 0 >> 00:28:03.523841 IP 127.0.0.1.47994 > 127.0.0.1.80: Flags [S], seq >> 1128551040, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val >> 1953257 ecr 0], length 0 >> 00:28:03.523924 IP 127.0.0.1.80 > 127.0.0.1.47994: Flags [S.], seq >> 4120370047, ack 1128551041, win 65535, options [mss 16344,nop,wscale >> 3,sackOK,TS val 727165180 ecr 1953257], length 0 >> 00:28:03.523942 IP 127.0.0.1.47994 > 127.0.0.1.80: Flags [.], ack 1, win >> 8960, options [nop,nop,TS val 1953257 ecr 727165180], length 0 >> 00:28:05.254567 IP 127.0.0.1.47994 > 127.0.0.1.80: Flags [P.], ack 1, >> win >> 8960, options [nop,nop,TS val 1953430 ecr 727165180], length 6 >> 00:28:05.254888 IP 127.0.0.1.80 > 127.0.0.1.47994: Flags [P.], ack 7, >> win >> 8960, options [nop,nop,TS val 727165353 ecr 1953430], length 44 >> 00:28:05.255194 IP 127.0.0.1.80 > 127.0.0.1.47994: Flags [F.], seq 45, >> ack >> 7, win 8960, options [nop,nop,TS val 727165353 ecr 1953430], length 0 >> 00:28:05.255234 IP 127.0.0.1.47994 > 127.0.0.1.80: Flags [.], ack 46, >> win >> 8960, options [nop,nop,TS val 1953430 ecr 727165353], length 0 >> 00:28:05.408742 IP 127.0.0.1.47994 > 127.0.0.1.80: Flags [F.], seq 7, >> ack >> 46, win 8960, options [nop,nop,TS val 1953445 ecr 727165353], length 0 >> 00:28:05.408799 IP 127.0.0.1.80 > 127.0.0.1.47994: Flags [.], ack 8, win >> 8959, options [nop,nop,TS val 727165368 ecr 1953445], length 0 >> 00:28:59.372253 IP 192.168.103.2.60900 > 127.0.0.1.80: Flags [S], seq >> 2362825029, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val >> 1958842 ecr 0], length 0 >> 00:29:02.371384 IP 192.168.103.2.60900 > 127.0.0.1.80: Flags [S], seq >> 2362825029, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val >> 1959142 ecr 0], length 0 >> >> >> ==> /var/log/httpd-access.log <== >> 127.0.0.1 - - [01/Jan/2011:23:18:44 -0800] "GET /" 200 44 "-" "-" >> 127.0.0.1 - - [01/Jan/2011:23:18:53 -0800] "GET /" 200 44 "-" "-" >> 127.0.0.1 - - [01/Jan/2011:23:21:48 -0800] "GET /" 200 44 "-" "-" >> >> I've tried to set up a rdr rule to redirect requests from internal ip to >> external port 80 to internal port 80, but no luck. >> >> /etc/pf.conf >> >> int_if="lo0" >> ext_if="ed0" >> >> rdr on $int_if inet proto tcp from $int_if to any port 80 -> 127.0.0.1 >> port 80 >> >> Under Linux, it is possible to set up rules to perform internal proxy >> diverting thereby, "Re-write address to appear as if wrapped daemon is >> connecting from the SSL client machine instead of the machine running >> stunnel." See also http://www.stunnel.org/faq/stunnel.html and >> http://www.stunnel.org/faq/transparent.html. >> >> iptables -t mangle -N DIVERT >> iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT >> iptables -t mangle -A DIVERT -j MARK --set-mark 1 >> iptables -t mangle -A DIVERT -j ACCEPT >> ip rule add fwmark 1 lookup 100 >> ip route add local 0.0.0.0/0 dev lo table 100 >> >> Is there a way to devise pf rdr rules to do the same? >> >> Thanks
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3020c1e8b0ecb5e9bacb1033ddea2b3e.squirrel>