Date: Wed, 6 Oct 2010 16:06:56 -0300 From: Eduardo Meyer <dudu.meyer@gmail.com> To: Julian Elischer <julian@freebsd.org>, Adrian Chadd <adrian@ucc.gu.uwa.edu.au>, ipfw@freebsd.org Subject: Re: layer2 ipfw 'fwd' support Message-ID: <AANLkTi=hoe%2BCaV6%2BbyagXYwzDRAHqCseh-M_44OxEeJO@mail.gmail.com> In-Reply-To: <4CAB8B35.7020703@freebsd.org> References: <AANLkTi=wHkmfDmoPrKN1SRcE9m=1_5iieAd85hQNWHs1@mail.gmail.com> <AANLkTinj8wd9AbROwRzUAUK=XraYmTDkoB3MGddqq-Tn@mail.gmail.com> <AANLkTin1vXOMPT6m8ybhNQk9G7WjDrCcSArP3Zwf65cR@mail.gmail.com> <4CAA1E7B.1020107@freebsd.org> <AANLkTikExTKMWvvDwn=rVUSqwz6UeVXi8WOSsHROQYq%2B@mail.gmail.com> <4CAA45CC.8020304@freebsd.org> <AANLkTikAd_fke1HfMgRy3h4fXpo7_DcX3E4%2BTu__3my8@mail.gmail.com> <4CAB8B35.7020703@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Oct 5, 2010 at 5:31 PM, Julian Elischer <julian@freebsd.org> wrote: > On 10/5/10 12:56 PM, Eduardo Meyer wrote: >> >> On Mon, Oct 4, 2010 at 6:23 PM, Julian Elischer<julian@freebsd.org> >> wrote: >>> >>> On 10/4/10 12:18 PM, Eduardo Meyer wrote: >>>> >>>> On Mon, Oct 4, 2010 at 3:35 PM, Julian Elischer<julian@freebsd.org> >>>> wrote: >>>>> >>>>> On 10/4/10 10:16 AM, Eduardo Meyer wrote: >>>>>> >>>>>> On Mon, Oct 4, 2010 at 2:02 PM, Brandon Gooch >>>>>> <jamesbrandongooch@gmail.com> wrote: >>>>>>> >>>>>>> On Mon, Oct 4, 2010 at 9:44 AM, Eduardo Meyer<dudu.meyer@gmail.com> >>>>>>> wrote: >>>>>>>> >>>>>>>> Hello, >>>>>>>> >>>>>>>> In the past I have used this patch by Luigi Rizzo, which helped me >>>>>>>> well. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> http://lists.freebsd.org/pipermail/freebsd-ipfw/2003-September/000526.html >>>>>>>> >>>>>>>> I tried with a friend to port it to -STABLE, but we were not able to >>>>>>>> find out what has replaced mt_tag. Also on ip_input.c we dirty >>>>>>>> hacked >>>>>>>> to following piece of code: >>>>>>>> >>>>>>>> #ifdef IPFIREWALL_FORWARD >>>>>>>> if (m->m_flags& M_FASTFWD_OURS) { >>>>>>>> m->m_flags&= ~M_FASTFWD_OURS; >>>>>>>> goto pass; /* XXX was 'ours' - SHOULD WE MODIFY IT >>>>>>>> HERE >>>>>>>> */ >>>>>>>> } >>>>>>>> if ((dchg = (m_tag_find(m, PACKET_TAG_IPFORWARD, NULL) != >>>>>>>> NULL)) >>>>>>>> != 0) { >>>>>>>> /* >>>>>>>> * Directly ship the packet on. This allows >>>>>>>> forwarding >>>>>>>> * packets originally destined to us to some other >>>>>>>> directly >>>>>>>> * connected host. >>>>>>>> */ >>>>>>>> ip_forward(m, dchg); >>>>>>>> return; >>>>>>>> } >>>>>>>> #endif /* IPFIREWALL_FORWARD */ >>>>>>>> >>>>>>>> And this is something we are not sure if its correct. >>>>>>>> >>>>>>>> So my very obvious question is: >>>>>>>> >>>>>>>> Does anyone has a recent version of this patch to share? >>>>>>>> >>>>>>>> Can anyone familiar with ipfw source code help me with that? >>>>>>>> >>>>>>> I'm certainly not an expert, but I wonder if the patch your referring >>>>>>> to is still required? Can you provide more detail about your >>>>>>> particular application? >>>>>>> >>>>>>> -Brandon >>>>>> >>>>>> Yes, its still required since ipfw fwd ignores layer2 frames. >>>>>> >>>>>> The application is the very same: squid. I mean, Lusca in fact (squid >>>>>> fork). >>>>>> >>>>>> Thank you for your interest. >>>>> >>>>> Cisco/Ironport have a patch that does this.. >>>>> I had permission to bring it back when I worked there but never got it >>>>> committed. >>>>> >>>>> Adrian, was it part of the set I gave you? >>>> >>>> Hello Elischer, >>>> >>>> Was this made public? >>>> >>>> I hope Chadd has some good news. In fact I tent to use with Lusca in >>>> tproxy mode. I bet this is the only missing piece of software. >>>> >>> I just dug up my old changes. >>> do you want to fwd from a bridge? or what? >>> (it makes a difference what patches are needed) >>> >>> If you want to fwd from a bridge to make a transparent layer 2 proxy, >>> this >>> may help.. >>> >>> >>> Here are parts of it that may be relevent: >>> these are old (2007 I think) but may be of use still. >>> >>> adrian had the full set at >>> >>> ==quote adrian===== >>> The stuff is in p4 now, but I haven't tested it out at all. >>> >>> //depo/projects/adrian_spoof_clientip/ I -think-. >>> == end quote=== >>> >>> >>> >>> >>> Index: net/if_bridge.c >>> =================================================================== >>> RCS file: /usr/local/cvsroot/freebsd/src/sys/net/if_bridge.c,v >>> retrieving revision 1.107 >>> diff -u -r1.107 if_bridge.c >>> --- net/if_bridge.c 6 Nov 2007 23:01:42 -0000 1.107 >>> +++ net/if_bridge.c 28 Nov 2007 06:59:10 -0000 >>> @@ -2908,6 +2908,11 @@ >>> struct ip *ip; >>> struct llc llc1; >>> u_int16_t ether_type; >>> + int is_ip = 0; >>> +#ifdef IPFIREWALL_FORWARD >>> + struct m_tag *fwd_tag; >>> +#endif >>> + >>> >>> snap = 0; >>> error = -1; /* Default error if not error == 0 */ >>> @@ -2967,6 +2972,7 @@ >>> #ifdef INET6 >>> case ETHERTYPE_IPV6: >>> #endif /* INET6 */ >>> + is_ip = 1; >>> break; >>> default: >>> /* >>> @@ -3024,6 +3030,30 @@ >>> >>> if (*mp == NULL) >>> return (error); >>> + >>> +#ifdef IPFIREWALL_FORWARD >>> + /* >>> + * Did the firewall want to forward it somewhere? >>> + * If so, let the ip stack handle it. >>> + */ >>> + if (i == 0&& args.next_hop != NULL&& >>> + is_ip /*&& src != NULL */) { >>> + >>> + fwd_tag = m_tag_get(PACKET_TAG_IPFORWARD, >>> + sizeof(struct sockaddr_in), >>> M_NOWAIT); >>> + if (fwd_tag == NULL) >>> + goto drop; >>> + bcopy(args.next_hop, (fwd_tag+1), >>> + sizeof(struct sockaddr_in)); >>> + m_tag_prepend(*mp, fwd_tag); >>> + >>> + if (in_localip(args.next_hop->sin_addr)) >>> + (*mp)->m_flags |= M_FASTFWD_OURS; >>> + ether_demux(src, *mp); >>> + return (NULL); >>> + } >>> +#endif >>> + >>> >>> if (DUMMYNET_LOADED&& (i == IP_FW_DUMMYNET)) { >>> >>> ================== >>> Index: netinet/ip_fw2.c >>> =================================================================== >>> RCS file: /usr/local/cvsroot/freebsd/src/sys/netinet/ip_fw2.c,v >>> retrieving revision 1.178 >>> diff -u -r1.178 ip_fw2.c >>> --- netinet/ip_fw2.c 28 Oct 2007 17:12:47 -0000 1.178 >>> +++ netinet/ip_fw2.c 28 Nov 2007 06:59:10 -0000 >>> >>> @@ -3446,8 +3507,10 @@ >>> case O_FORWARD_IP: { >>> struct sockaddr_in *sa; >>> sa =&(((ipfw_insn_sa *)cmd)->sa); >>> +#if 0 >>> if (args->eh) /* not valid on layer2 >>> pkts >>> */ >>> break; >>> +#endif >>> if (!q || dyn_dir == MATCH_FORWARD) { >>> if (sa->sin_addr.s_addr == >>> INADDR_ANY) { >>> bcopy(sa,&args->hopstore, >>> >>> ============================================= >>> Index: netinet/ip_output.c >> >> Dear Julian, >> >> Is anything missing from the above code? Say, like ip_output stuff? >> >> I have tried what you sent me, compiled fine but did not work. >> >> Here is my only rule (I have tried both with and without layer2 on the >> rule): >> >> 00001 36 4338 fwd 127.0.0.1,80 tcp from any to not me >> dst-port 80 layer2 >> 65535 32842101 2107060460 allow ip from any to any >> >> Here are the sysctl tunables: >> >> net.link.bridge.ipfw: 1 >> net.link.bridge.inherit_mac: 0 >> net.link.bridge.log_stp: 0 >> net.link.bridge.pfil_local_phys: 0 >> net.link.bridge.pfil_member: 0 >> net.link.bridge.pfil_bridge: 1 >> net.link.bridge.ipfw_arp: 0 >> net.link.bridge.pfil_onlyip: 0 >> net.link.ether.inet.log_arp_permanent_modify: 1 >> net.link.ether.inet.log_arp_movements: 1 >> net.link.ether.inet.log_arp_wrong_iface: 1 >> net.link.ether.inet.proxyall: 0 >> net.link.ether.inet.useloopback: 1 >> net.link.ether.inet.maxtries: 5 >> net.link.ether.inet.max_age: 1200 >> net.link.ether.ipfw: 1 >> >> And my bridge: >> >> bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu >> 1500 >> ether 16:52:8e:91:2f:45 >> id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 >> maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200 >> root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 >> member: vr0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> >> ifmaxaddr 0 port 5 priority 128 path cost 200000 >> member: sis0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> >> ifmaxaddr 0 port 1 priority 128 path cost 200000 >> >> The ipfw counter gets increased by nothing hits by Apache. Instead I >> go to Internet directly. >> >> sis0 is on internet, vr0 is cross-over to the laptop (customer). >> >> How should I debug it? >> >> > basically I woud suggest code inspection for a start.. > > look at where ipfw is called (just before where the patch went in) and > follow the packet up into ipfw > and back, and read what it would do.. > > It's actually not a very hard path to follow. > > I'll try look at it after work.. Hello Julian / Adrian. Thank you for your attention. A friend added some log entries so we could try to find out what gets run and what doesnt. Here is my current patch against RELENG_8: --- if_bridge.c.orig 2010-09-11 22:02:36.000000000 +0000 +++ if_bridge.c 2010-10-05 17:59:13.000000000 +0000 @@ -2957,6 +2957,13 @@ struct ip *ip; struct llc llc1; u_int16_t ether_type; + int is_ip = 0; +#ifdef IPFIREWALL_FORWARD + struct m_tag *fwd_tag; +#endif + + + snap = 0; error = -1; /* Default error if not error == 0 */ @@ -3016,6 +3023,8 @@ #ifdef INET6 case ETHERTYPE_IPV6: #endif /* INET6 */ + is_ip=1; + log(LOG_NOTICE, "Entered 0: is_ip=%i\n",is_ip); break; default: /* @@ -3091,6 +3100,32 @@ if (*mp == NULL) return (error); +#ifdef IPFIREWALL_FORWARD + /* + * Did the firewall want to forward it somewhere? + * If so, let the ip stack handle it. + */ + log(LOG_NOTICE, "Entered 1"); + if (i == 0&& args.next_hop != NULL&& + is_ip /*&& src != NULL */) { + log(LOG_NOTICE, "Entered 2"); + + fwd_tag = m_tag_get(PACKET_TAG_IPFORWARD, + sizeof(struct sockaddr_in), M_NOWAIT); + if (fwd_tag == NULL) + goto drop; + bcopy(args.next_hop, (fwd_tag+1), + sizeof(struct sockaddr_in)); + m_tag_prepend(*mp, fwd_tag); + + if (in_localip(args.next_hop->sin_addr)) + (*mp)->m_flags |= M_FASTFWD_OURS; + ether_demux(src, *mp); + return (NULL); + } +#endif + + if (ip_dn_io_ptr && (i == IP_FW_DUMMYNET)) { /* put the Ethernet header back on */ --- ../netinet/ipfw/ip_fw2.c.orig 2010-09-16 15:11:17.000000000 +0000 +++ ../netinet/ipfw/ip_fw2.c 2010-10-06 12:17:12.000000000 +0000 @@ -2059,8 +2059,14 @@ break; case O_FORWARD_IP: - if (args->eh) /* not valid on layer2 pkts */ +#if 0 + /* not valid on layer2 pkts */ + if (args->eh) { + log(LOG_NOTICE, "ip_fw2.c Entered 1"); break; + } + log(LOG_NOTICE, "ip_fw2.c Entered 2"); +#endif if (!q || dyn_dir == MATCH_FORWARD) { struct sockaddr_in *sa; sa = &(((ipfw_insn_sa *)cmd)->sa); Please notice the log entries for debugging. When I try with the very same rule: fwd 127.0.0.1,80 tcp from any to not me dst-port 80 layer2 This is what I get on /var/log/messages: Oct 6 15:58:44 phoenix kernel: Entered 0: is_ip=1 Oct 6 15:59:16 phoenix last message repeated 93 times Oct 6 16:01:20 phoenix last message repeated 189 times So we never "Entered 1", never "Entered 2" nor ever entered "ip_fw2.c Entered 2"; Seem like FWD is never triggered... I don't know where to move forth. Any help is appreciated. > > -- =========== Eduardo Meyer pessoal: dudu.meyer@gmail.com profissional: ddm.farmaciap@saude.gov.br
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AANLkTi=hoe%2BCaV6%2BbyagXYwzDRAHqCseh-M_44OxEeJO>