Date: Tue, 3 Jan 2012 20:29:27 -0800 From: Ed Carrel <azanar@carrel.org> To: freebsd-net@freebsd.org Subject: pf not seeing inbound packets on netgraph interface Message-ID: <CAC6u2XeiUP2jouYv1Qc%2B1gmjx-1twSmKeLMbNZNq_3CU625mFw@mail.gmail.com> In-Reply-To: <CAC6u2XeLijriLrYgiUf32BMLzYF-uza18c_e9Rk9jXmxe0fW2w@mail.gmail.com> References: <CAC6u2XeLijriLrYgiUf32BMLzYF-uza18c_e9Rk9jXmxe0fW2w@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi freebsd-net,
I originally sent this to -questions@, but was redirected here by that
list. My original question is below:
I am running into a roadblock getting PF to filter traffic on a Netgraph
interface representing an L2TP/IPSec connection. I have done some narrowing
down of the problem, but was hoping to get some advice on figuring out
where to go digging next, or things to try.
For context, here is what I have setup so far: I am running FreeBSD 8.2
with IPSec support compiled into the kernel. Here's the details from uname:
# uname -a
FreeBSD **** 8.2-RELEASE-p4 FreeBSD 8.2-RELEASE-p4 #2: Sun Nov 27 04:12:16
PST 2011 **** i386
I am following what seems like a typical setup of racoon(8) and setkey(8),
and am having mpd5 handle construction of the L2TP nodes in netgraph. I can
provide the details on the configuration of each of these, if desired. When
I startup racoon in the foreground and ask mpd to construct an L2TP link, I
can see both the IPSec tunnel and the L2TP link establish without a
problem. I am able to ping the remote end, and, if I set up a routing rule,
can contact and ssh to hosts at the other end of the tunnel.
Here's how netgraph sees the world when thing are established:
# ngctl list
There are 13 total nodes:
Name: <unnamed> Type: ksocket ID: 000001b3 Num hooks: 1
Name: <unnamed> Type: l2tp ID: 000001b1 Num hooks: 3
Name: <unnamed> Type: socket ID: 000001b0 Num hooks: 1
Name: ng0 Type: iface ID: 000001b6 Num hooks: 1
Name: ngctl26124 Type: socket ID: 000001bd Num hooks: 0
Name: ngctl19375 Type: socket ID: 000000ba Num hooks: 0
Name: mpd25875-stats Type: socket ID: 000001b8 Num hooks: 0
Name: mpd25875-WPLink-lt Type: tee ID: 000001af Num hooks: 2
Name: mpd25875-cso Type: socket ID: 000001ad Num hooks: 0
Name: mpd25875-eso Type: socket ID: 000001ae Num hooks: 0
Name: mpd25875-lso Type: socket ID: 000001ac Num hooks: 1
Name: mpd25875-WPBundle-1 Type: ppp ID: 000001b7 Num hooks:
3
Name: ng0-tee Type: tee ID: 000001b9 Num hooks: 2
#
The problem I have is that PF only sees traffic on the outbound side of the
netgraph interface. But, the rest of the network stack appears to see data
going both ways:
# ifconfig ng0
ng0: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> metric 0
mtu 1322
inet 10.10.7.40 --> 10.10.0.2 netmask 0xffffffff
# pfctl -vvs Interfaces -i ng0
ng0
Cleared: Sun Dec 25 21:14:44 2011
References: [ States: 0 Rules: 9 ]
In4/Pass: [ Packets: 0 Bytes: 0 ]
In4/Block: [ Packets: 0 Bytes: 0 ]
Out4/Pass: [ Packets: 5555 Bytes: 446225 ]
Out4/Block: [ Packets: 622 Bytes: 56336 ]
In6/Pass: [ Packets: 0 Bytes: 0 ]
In6/Block: [ Packets: 0 Bytes: 0 ]
Out6/Pass: [ Packets: 0 Bytes: 0 ]
Out6/Block: [ Packets: 0 Bytes: 0 ]
# netstat -I ng0 -bn
Name Mtu Network Address Ipkts Ierrs Idrop Ibytes
Opkts Oerrs Obytes Coll
ng0 1322 <Link#8> 56 0 0 5069
98 0 6032 0
ng0 1322 10.10.7.40/32 10.10.7.40 56 - - 5069
54 - 3472 -
I have stood up this interface several times, hence the differing numbers
between the PF and netstat. The cause for concern is the lack of packets
going through PF when inbound on ng0 -- no problem both seeing them and
applying rules going outbound. There isn't a peep about the inbound traffic
within pflog0, either.
I can see traffic going both ways in tcpdump, and nothing looks peculiar
about the packets.
# tcpdump -c 10 -i ng0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ng0, link-type NULL (BSD loopback), capture size 96 bytes
22:06:37.201732 IP 10.10.7.40.43113 > 10.10.4.3.ssh: Flags [S], seq
3442571726, win 65535, options [mss 1282,nop,wscale 3,sackOK,TS val
694436002 ecr 0], length 0
22:06:37.263336 IP 10.10.4.3.ssh > 10.10.7.40.43113: Flags [S.], seq
1974232057, ack 3442571727, win 14480, options [mss 1282,sackOK,TS val
370681934 ecr 694436002,nop,wscale 7], length 0
22:06:37.263577 IP 10.10.7.40.43113 > 10.10.4.3.ssh: Flags [.], ack 1, win
8255, options [nop,nop,TS val 694436064 ecr 370681934], length 0
22:06:37.282835 IP 10.10.4.3.ssh > 10.10.7.40.43113: Flags [P.], ack 1, win
114, options [nop,nop,TS val 370681940 ecr 694436064], length 21
22:06:37.283931 IP 10.10.7.40.43113 > 10.10.4.3.ssh: Flags [P.], ack 22,
win 8255, options [nop,nop,TS val 694436084 ecr 370681940], length 40
22:06:37.300729 IP 10.10.4.3.ssh > 10.10.7.40.43113: Flags [.], ack 41, win
114, options [nop,nop,TS val 370681945 ecr 694436084], length 0
22:06:37.300943 IP 10.10.7.40.43113 > 10.10.4.3.ssh: Flags [P.], ack 22,
win 8255, options [nop,nop,TS val 694436101 ecr 370681945], length 848
22:06:37.304154 IP 10.10.4.3.ssh > 10.10.7.40.43113: Flags [P.], ack 41,
win 114, options [nop,nop,TS val 370681945 ecr 694436084], length 984
22:06:37.372460 IP 10.10.4.3.ssh > 10.10.7.40.43113: Flags [.], ack 889,
win 127, options [nop,nop,TS val 370681967 ecr 694436101], length 0
22:06:37.372663 IP 10.10.7.40.43113 > 10.10.4.3.ssh: Flags [P.], ack 1006,
win 8255, options [nop,nop,TS val 694436173 ecr 370681945], length 24
10 packets captured
22 packets received by filter
0 packets dropped by kernel
As I noted above, I can interact with hosts over the tunnel so long as PF
blindly passes traffic. Attempting to do any sort of stateful connection
tracking causes PF to litter /var/log/messages with notices of a "loose
state match," which I think is to be expected since it is only seeing the
outbound half the network conversation.
A suggestion someone on questions@ had was to leverage a gif interface, but
with this already creating the ng0 interface, I'm not sure what that would
accomplish, or if it is possible to bridge a gif with ng in that way. I'd
be happy to research this more if enlightenment lies down that path.
Ideas on things to try or investigate next? I can provide a paste of any
relevant config or log files, just let me know.
Thanks,
Ed Carrel
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAC6u2XeiUP2jouYv1Qc%2B1gmjx-1twSmKeLMbNZNq_3CU625mFw>