Date: Mon, 2 May 2011 17:19:32 -0500 From: Korodev <korodev@gmail.com> To: freebsd-ipfw@freebsd.org Subject: IPFW Table Insertion in C, Dummynet, and an interesting problem. Message-ID: <BANLkTi=HXWu=Pix9-kj=B%2B%2BJRq17YbWyPg@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Hey guys,
I'm currently running some custom C code ,via an output plugin for
Snort, which takes an IP and sticks it in an ipfw table. Once the
packet enters the box, I'm using dummynet to delay the packet while
snort analyzes it and inserts the IP into a table, after the piping
delay is complete the rule is reinserted at the appropriate point and
checked via a deny table lookup rule. I've done some fairly extensive
testing which has led me here. I believe I'm either doing my IPFW
insertion wrong, or there's a bug or tuning setting I'm unaware of.
I'd be delighted if you guys could take the time to look through my
explanation below and let me know if anything comes to mind :)
First my physical setup is as follows:
Pinger --> { eth 0 --> bridge0 --> eth1 } --> Host
#My Freebsd/IPFW setup:
FreeBSD 8.2, net.link.bridge.ipfw=1, net.inet.ip.fw.one_pass=0
#IPFW Ruleset
00100 count icmp from any to any
00300 pipe 1 icmp from any to any //pipe is configured with config delay 150ms
00400 deny log ip from any to any src-ip table(0)
00500 count icmp from any to any
#IPFW C Insertion Code
....
#include <net/if.h>
#include <netinet/ip_fw.h>
//data is a custom struct
data->s = socket(AF_INET, SOCK_RAW, IPPROTO_RAW);
data->ent.tbl = 0;
data->ent.value = 0;
data->ent.masklen = 32;
data->ip.s_addr = p->iph->ip_src.s_addr;
memcpy(&(data->ent.addr), &(data->ip), sizeof(struct in_addr));
setsockopt(data->s, IPPROTO_IP, IP_FW_TABLE_ADD, &(data->ent),
sizeof(data->ent));
...
This is slightly simplified for this test case, but it's important to
know the insertion works, just not in less than 150ms. (~200ms
actually). I've been talking to the Snort guys a bit, and I won't post
my snort conf here, but please take my word that it's quite stripped
down. My test case as follows, consisingt of a sending a single ICMP
ping packet from Pinger to Host. In theory, Snort, listening passively
on eth0 (libpcap 1.1.1), will alert on the ICMP ping to Host insert. .
I conducted the following timing tests using tcpdump, Snort's
performance profiling, and my own C timing code. Here are my results:
tcpdump shows that the packet hits eth0 at 51.647347 seconds, bridge0
at 51.647350 seconds, and the exit interface, eth1, at 51.797320. This
(as expected) equates to 149.97300 milliseconds of time.
Snort is passively listening on the eth0 interface using the pcap daq
module. It's configured to spend a maximum time of 250 microseconds
before triggering my output plugin. Using some C timing methods on the
output plugin code above says that grabbing the IP and sending it to
the IPFW socket (which is cached and already open at that point),
takes about 7.8120 milliseconds (1 CPU tick). Since that's where I
call setsockopt, the it just sits in the pipe for the remaining time,
but upon falling to the next rule (deny src-ip table 0), it misses the
check. The count rule following my deny rule verifies that the rule
did reach the deny table rule.
There's only one untested spot that I can see, and that's the time
between I actually make the insertion via setsockopt and the time IPFW
"actually" update the table in memory. All of my other operations are
executing with testable and consistent speeds, which are FAR less than
a 150 millisecond dummynet pipe.
Am I inserting the IP into the table correctly? Is there another
command I need to call to force IPFW to update the table faster? Or
perhaps there's some kernel tuning I'm unaware of? If you've made it
this far, then thanks for taking the time to read this and please let
me know if you have thoughts on why the IP isn't making it in the
table fast enough. If more info is needed, I'll be happy to provide
it :)
Thanks,
\\korodev
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?BANLkTi=HXWu=Pix9-kj=B%2B%2BJRq17YbWyPg>