Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 4 Jan 2012 09:03:38 +0100
From:      =?ISO-8859-1?Q?Ermal_Lu=E7i?= <eri@freebsd.org>
To:        Ed Carrel <azanar@carrel.org>
Cc:        freebsd-net@freebsd.org
Subject:   Re: pf not seeing inbound packets on netgraph interface
Message-ID:  <CAPBZQG2z0GeOoV_RfcnmnD=pbQ4CYQAYEs9PqrhE_JXirLMBOA@mail.gmail.com>
In-Reply-To: <CAC6u2XeiUP2jouYv1Qc%2B1gmjx-1twSmKeLMbNZNq_3CU625mFw@mail.gmail.com>
References:  <CAC6u2XeLijriLrYgiUf32BMLzYF-uza18c_e9Rk9jXmxe0fW2w@mail.gmail.com> <CAC6u2XeiUP2jouYv1Qc%2B1gmjx-1twSmKeLMbNZNq_3CU625mFw@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Jan 4, 2012 at 5:29 AM, Ed Carrel <azanar@carrel.org> wrote:

> Hi freebsd-net,
>
> I originally sent this to -questions@, but was redirected here by that
> list. My original question is below:
>
> I am running into a roadblock getting PF to filter traffic on a Netgraph
> interface representing an L2TP/IPSec connection. I have done some narrowing
> down of the problem, but was hoping to get some advice on figuring out
> where to go digging next, or things to try.
>
> For context, here is what I have setup so far: I am running FreeBSD 8.2
> with IPSec support compiled into the kernel. Here's the details from uname:
>
> # uname -a
> FreeBSD **** 8.2-RELEASE-p4 FreeBSD 8.2-RELEASE-p4 #2: Sun Nov 27 04:12:16
> PST 2011 ****  i386
>
> I am following what seems like a typical setup of racoon(8) and setkey(8),
> and am having mpd5 handle construction of the L2TP nodes in netgraph. I can
> provide the details on the configuration of each of these, if desired. When
> I startup racoon in the foreground and ask mpd to construct an L2TP link, I
> can see both the IPSec tunnel and the L2TP link establish without a
> problem. I am able to ping the remote end, and, if I set up a routing rule,
> can contact and ssh to hosts at the other end of the tunnel.
>
> Here's how netgraph sees the world when thing are established:
>
> # ngctl list
> There are 13 total nodes:
>  Name: <unnamed>       Type: ksocket         ID: 000001b3   Num hooks: 1
>  Name: <unnamed>       Type: l2tp            ID: 000001b1   Num hooks: 3
>  Name: <unnamed>       Type: socket          ID: 000001b0   Num hooks: 1
>  Name: ng0             Type: iface           ID: 000001b6   Num hooks: 1
>  Name: ngctl26124      Type: socket          ID: 000001bd   Num hooks: 0
>  Name: ngctl19375      Type: socket          ID: 000000ba   Num hooks: 0
>  Name: mpd25875-stats  Type: socket          ID: 000001b8   Num hooks: 0
>  Name: mpd25875-WPLink-lt Type: tee             ID: 000001af   Num hooks: 2
>  Name: mpd25875-cso    Type: socket          ID: 000001ad   Num hooks: 0
>  Name: mpd25875-eso    Type: socket          ID: 000001ae   Num hooks: 0
>  Name: mpd25875-lso    Type: socket          ID: 000001ac   Num hooks: 1
>  Name: mpd25875-WPBundle-1 Type: ppp             ID: 000001b7   Num hooks:
> 3
>  Name: ng0-tee         Type: tee             ID: 000001b9   Num hooks: 2
> #
>
> The problem I have is that PF only sees traffic on the outbound side of the
> netgraph interface. But, the rest of the network stack appears to see data
> going both ways:
>
> # ifconfig ng0
> ng0: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> metric 0
> mtu 1322
> inet 10.10.7.40 --> 10.10.0.2 netmask 0xffffffff
>
> # pfctl -vvs Interfaces -i ng0
> ng0
> Cleared:     Sun Dec 25 21:14:44 2011
> References:  [ States:  0                  Rules: 9                  ]
>  In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
> In4/Block:   [ Packets: 0                  Bytes: 0                  ]
>  Out4/Pass:   [ Packets: 5555               Bytes: 446225             ]
> Out4/Block:  [ Packets: 622                Bytes: 56336              ]
>  In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
> In6/Block:   [ Packets: 0                  Bytes: 0                  ]
>  Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
> Out6/Block:  [ Packets: 0                  Bytes: 0                  ]
>
> # netstat -I ng0 -bn
> Name    Mtu Network       Address              Ipkts Ierrs Idrop     Ibytes
>   Opkts Oerrs     Obytes  Coll
> ng0    1322 <Link#8>                              56     0     0       5069
>      98     0       6032     0
> ng0    1322 10.10.7.40/32 10.10.7.40              56     -     -
> 5069
>      54     -       3472     -
>
> I have stood up this interface several times, hence the differing numbers
> between the PF and netstat. The cause for concern is the lack of packets
> going through PF when inbound on ng0 -- no problem both seeing them and
> applying rules going outbound. There isn't a peep about the inbound traffic
> within pflog0, either.
>
> I can see traffic going both ways in tcpdump, and nothing looks peculiar
> about the packets.
>
> # tcpdump -c 10 -i ng0
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on ng0, link-type NULL (BSD loopback), capture size 96 bytes
> 22:06:37.201732 IP 10.10.7.40.43113 > 10.10.4.3.ssh: Flags [S], seq
> 3442571726, win 65535, options [mss 1282,nop,wscale 3,sackOK,TS val
> 694436002 ecr 0], length 0
> 22:06:37.263336 IP 10.10.4.3.ssh > 10.10.7.40.43113: Flags [S.], seq
> 1974232057, ack 3442571727, win 14480, options [mss 1282,sackOK,TS val
> 370681934 ecr 694436002,nop,wscale 7], length 0
> 22:06:37.263577 IP 10.10.7.40.43113 > 10.10.4.3.ssh: Flags [.], ack 1, win
> 8255, options [nop,nop,TS val 694436064 ecr 370681934], length 0
> 22:06:37.282835 IP 10.10.4.3.ssh > 10.10.7.40.43113: Flags [P.], ack 1, win
> 114, options [nop,nop,TS val 370681940 ecr 694436064], length 21
> 22:06:37.283931 IP 10.10.7.40.43113 > 10.10.4.3.ssh: Flags [P.], ack 22,
> win 8255, options [nop,nop,TS val 694436084 ecr 370681940], length 40
> 22:06:37.300729 IP 10.10.4.3.ssh > 10.10.7.40.43113: Flags [.], ack 41, win
> 114, options [nop,nop,TS val 370681945 ecr 694436084], length 0
> 22:06:37.300943 IP 10.10.7.40.43113 > 10.10.4.3.ssh: Flags [P.], ack 22,
> win 8255, options [nop,nop,TS val 694436101 ecr 370681945], length 848
> 22:06:37.304154 IP 10.10.4.3.ssh > 10.10.7.40.43113: Flags [P.], ack 41,
> win 114, options [nop,nop,TS val 370681945 ecr 694436084], length 984
> 22:06:37.372460 IP 10.10.4.3.ssh > 10.10.7.40.43113: Flags [.], ack 889,
> win 127, options [nop,nop,TS val 370681967 ecr 694436101], length 0
> 22:06:37.372663 IP 10.10.7.40.43113 > 10.10.4.3.ssh: Flags [P.], ack 1006,
> win 8255, options [nop,nop,TS val 694436173 ecr 370681945], length 24
> 10 packets captured
> 22 packets received by filter
> 0 packets dropped by kernel
>
> As I noted above, I can interact with hosts over the tunnel so long as PF
> blindly passes traffic. Attempting to do any sort of stateful connection
> tracking causes PF to litter /var/log/messages with notices of a "loose
> state match," which I think is to be expected since it is only seeing the
> outbound half the network conversation.
>
> A suggestion someone on questions@ had was to leverage a gif interface,
> but
> with this already creating the ng0 interface, I'm not sure what that would
> accomplish, or if it is possible to bridge a gif with ng in that way. I'd
> be happy to research this more if enlightenment lies down that path.
>
> Ideas on things to try or investigate next? I can provide a paste of any
> relevant config or log files, just let me know.
>
>
Can you see if on the enc(4) interface pf(4) sees both side of the traffic?
Also please describe/post what is the ruleset of blindly passing packets
and the ruleset that you define as 'keep state'!?



> Thanks,
>
> Ed Carrel
> _______________________________________________
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
>



-- 
Ermal



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAPBZQG2z0GeOoV_RfcnmnD=pbQ4CYQAYEs9PqrhE_JXirLMBOA>