From owner-freebsd-net@FreeBSD.ORG Sun Jul 18 21:38:52 2004 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1727116A4CE for ; Sun, 18 Jul 2004 21:38:52 +0000 (GMT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id AF7CD43D46 for ; Sun, 18 Jul 2004 21:38:51 +0000 (GMT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.11/8.12.11) with ESMTP id i6ILcM4q039314; Sun, 18 Jul 2004 17:38:22 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i6ILcMpL039311; Sun, 18 Jul 2004 17:38:22 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Sun, 18 Jul 2004 17:38:22 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Don Bowman In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: "'net@freebsd.org'" Subject: Re: Question on SOCK_RAW, implement a bpf->other host tee X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Jul 2004 21:38:52 -0000 On Sat, 17 Jul 2004, Don Bowman wrote: > I'm trying to implement a 'tee' which reads from bpf, and sends matching > packets to another layer-2 adjacent host. Just FYI, I would normally do this by writing back out to BPF on the interface you're "teeing" to, perhaps with some rewriting of the ethernet layer header. Since I don't have a big picture of what your application is actually doing, I can't speak to the details there :-). > I'm doing this with SOCK_RAW to try and write the packet back out. The > 'sendto' passes, but i don't see a packet anywhere. > > Am i correct that i can hand an arbitrarily crafted IP packet into > sendto, and the stack will write the ethernet header on, pick an > interface, etc, based on the address in the sendto? Hmm. I'm not 100% sure that this is correct. When you don't set IP_HDRINCL, the raw IP output code will indeed do the route lookup on the requested address, as well as insert an IP header that includes the requested address. However, when you use IP_HDRINCL to provide your own IP address, I believe that the target address argument to sendto() will be ignored, and the address in the IP header you provide used as the destination instead. What could well be happening is that you're re-injecting the packet at the IP forwarding layer, and it is being forwarded to the original destination IP (possibly localhost) and processed as a dup packet. I.e., you are making copies of the packet, but they're not being sent where you think. If you're sniffing packets destined for the local IP, you might well be able to see the tee'd packets by running tcpdump on lo0. > I have swapped the ip_len, ip_off fields. Are you sure you need to do this? I thought BPF/PCAP provided those fields in network byte order already, in which case you shouldn't need to touch these fields unless you need to adjust them. Also, I notice that you don't appear to be initializing the 'sin_len' field of struct sockaddr_in, which should be set to sizeof(to) when you initialize the structure. Generally speaking, the notion of "tee" is a little poorly defined. I think what you want to do is what I suggest above -- use BPF to write the packet out another interface, and do the link layer routing yourself, rewriting packet header fields if you want to. That is, basically write the new target ethernet address into the BPF layer ethernet header, and write it out on a BPF device attached to the network interface you're teeing to. This will leave all the IP layer stuff intact. I've previously written a BPF layer bridging application that uses this technique to do filtered bridging and stateful TCP transformation in user space. You don't get the performance of the kernel, but it's a lot easier to debug. One caution: be careful of bridging loops and the like. If you do mean to be changing the IP layer addresses, remember that you'll need to recalculate header checksums for IP, TCP/UDP, etc. If you change any other IP layer fields, you may need to just recalculate the IP layer checksum. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Principal Research Scientist, McAfee Research > > The program I have is below. This is on 4.7. > The handler gets called, the packet there looks > correct, no error on any system call, yet no > output :( > > Suggestions? > > /* > * Copyright 2004 Sandvine Incorporated. All rights reserved > */ > > #include > #include > #include > #include > #include > #include > #include > #include > > void > usage(const char *name) > { > fprintf(stderr, "Usage: %s [-I input_interface] [-O output_interface] > [-i output_ip(arp for mac)] [-v]\n", name); > exit(1); > } > > typedef struct > { > int s; > struct in_addr output_ip; > } > context; > > static int verbose; > > static void > handler(unsigned char *ct, > const struct pcap_pkthdr *hdr, > const unsigned char *pkt) > { > struct ip *ip = (struct ip *)(pkt + 14); > context *ctxt = (context *)ct; > struct sockaddr_in to; > memset(&to,0,sizeof(to)); > to.sin_family = AF_INET; > to.sin_addr = ctxt->output_ip; > if (verbose) > { > fprintf(stderr, "Send %d byte packet\n", hdr->len); > } > ip->ip_len = htons(ip->ip_len); > ip->ip_off = htons(ip->ip_off); > if (sendto(ctxt->s, > ip, > hdr->len-14, > 0, > (struct sockaddr *)&to, > sizeof(to)) != (hdr->len-14) ) > { > err(1, "sendto"); > } > } > > static int > doit(const char *input_interface, > const char *output_interface, > struct in_addr output_ip) > { > char errbuf[PCAP_ERRBUF_SIZE]; > pcap_t *in_d, *out_d; > context ctxt; > int on = 1; > struct bpf_program fp; > > in_d = pcap_open_live((char *)input_interface, 1600, 1, 20, errbuf); > if (in_d == 0) > { > errx(1, "open of %s failed: %s", input_interface, errbuf); > } > > ctxt.output_ip.s_addr = htonl(output_ip.s_addr); > ctxt.s = socket(PF_INET, SOCK_RAW, IPPROTO_RAW); > if (ctxt.s < 0) > errx(1, "can't open raw socket"); > if (setsockopt(ctxt.s, IPPROTO_IP, IP_HDRINCL, (char *)&on, sizeof(on)) > < 0) > { > err(1,"setsockopt"); > } > > memset(&fp,0,sizeof(fp));/ > if (pcap_compile(in_d, &fp, "ip", 0, 0xfffffff0) < 0) > { > errx(1, "failed to compile: %s",pcap_geterr(in_d)); > } > if (pcap_setfilter(in_d, &fp) < 0) > { > errx(1, "failed to set filter"); > } > > pcap_loop(in_d, -1, handler, (unsigned char *)&ctxt); > } > > int > main(int argc, char *argv[]) > { > int ch; > char *input_interface = "ipfw0"; > char *output_interface = "em2"; > struct in_addr output_ip; > output_ip.s_addr = 0; > > while ((ch = getopt(argc, argv, "I:O:i:vh?")) != -1) > { > switch (ch) > { > case 'I': > input_interface = optarg; > break; > case 'O': > output_interface = optarg; > break; > case 'i': > if (inet_aton(optarg,&output_ip) < 0) > { > errx(1, "unknown ip %s", optarg); > } > break; > case 'v': > verbose = 1; > break; > case 'h': > case '?': > default: > usage(argv[0]); > } > } > if (verbose) > fprintf(stderr, "%s->%s(%s)\n", > input_interface,output_interface,inet_ntoa(output_ip)); > return doit(input_interface,output_interface,output_ip); > } > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >