Date: Thu, 7 Oct 2010 22:23:57 +0300 From: Eduardo Meyer <dudu.meyer@gmail.com> To: Julian Elischer <julian@freebsd.org>, ipfw@freebsd.org Subject: Re: layer2 ipfw 'fwd' support Message-ID: <AANLkTik2KEYACzjfTS%2BXpB3OiaJL-uYckbLbf2C0DWaS@mail.gmail.com> In-Reply-To: <4CACE7DE.9020106@freebsd.org> References: <AANLkTi=wHkmfDmoPrKN1SRcE9m=1_5iieAd85hQNWHs1@mail.gmail.com> <AANLkTinj8wd9AbROwRzUAUK=XraYmTDkoB3MGddqq-Tn@mail.gmail.com> <AANLkTin1vXOMPT6m8ybhNQk9G7WjDrCcSArP3Zwf65cR@mail.gmail.com> <4CAA1E7B.1020107@freebsd.org> <AANLkTikExTKMWvvDwn=rVUSqwz6UeVXi8WOSsHROQYq%2B@mail.gmail.com> <4CAA45CC.8020304@freebsd.org> <AANLkTikAd_fke1HfMgRy3h4fXpo7_DcX3E4%2BTu__3my8@mail.gmail.com> <4CAB8B35.7020703@freebsd.org> <AANLkTi=hoe%2BCaV6%2BbyagXYwzDRAHqCseh-M_44OxEeJO@mail.gmail.com> <4CACE7DE.9020106@freebsd.org>
index | next in thread | previous in thread | raw e-mail
On Thu, Oct 7, 2010 at 12:19 AM, Julian Elischer <julian@freebsd.org> wrote: > On 10/6/10 12:06 PM, Eduardo Meyer wrote: >> >> On Tue, Oct 5, 2010 at 5:31 PM, Julian Elischer<julian@freebsd.org> >> wrote: >>> >>> On 10/5/10 12:56 PM, Eduardo Meyer wrote: >>>> >>>> On Mon, Oct 4, 2010 at 6:23 PM, Julian Elischer<julian@freebsd.org> >>>> wrote: >>>>> >>>>> On 10/4/10 12:18 PM, Eduardo Meyer wrote: >>>>>> >>>>>> On Mon, Oct 4, 2010 at 3:35 PM, Julian Elischer<julian@freebsd.org> >>>>>> wrote: >>>>>>> >>>>>>> On 10/4/10 10:16 AM, Eduardo Meyer wrote: >>>>>>>> >>>>>>>> On Mon, Oct 4, 2010 at 2:02 PM, Brandon Gooch >>>>>>>> <jamesbrandongooch@gmail.com> wrote: >>>>>>>>> >>>>>>>>> On Mon, Oct 4, 2010 at 9:44 AM, Eduardo Meyer<dudu.meyer@gmail.com> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Hello, >>>>>>>>>> >>>>>>>>>> In the past I have used this patch by Luigi Rizzo, which helped me >>>>>>>>>> well. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> http://lists.freebsd.org/pipermail/freebsd-ipfw/2003-September/000526.html >>>>>>>>>> >>>>>>>>>> I tried with a friend to port it to -STABLE, but we were not able >>>>>>>>>> to >>>>>>>>>> find out what has replaced mt_tag. Also on ip_input.c we dirty >>>>>>>>>> hacked >>>>>>>>>> to following piece of code: >>>>>>>>>> >>>>>>>>>> #ifdef IPFIREWALL_FORWARD >>>>>>>>>> if (m->m_flags& M_FASTFWD_OURS) { >>>>>>>>>> m->m_flags&= ~M_FASTFWD_OURS; >>>>>>>>>> goto pass; /* XXX was 'ours' - SHOULD WE MODIFY IT >>>>>>>>>> HERE >>>>>>>>>> */ >>>>>>>>>> } >>>>>>>>>> if ((dchg = (m_tag_find(m, PACKET_TAG_IPFORWARD, NULL) != >>>>>>>>>> NULL)) >>>>>>>>>> != 0) { >>>>>>>>>> /* >>>>>>>>>> * Directly ship the packet on. This allows >>>>>>>>>> forwarding >>>>>>>>>> * packets originally destined to us to some other >>>>>>>>>> directly >>>>>>>>>> * connected host. >>>>>>>>>> */ >>>>>>>>>> ip_forward(m, dchg); >>>>>>>>>> return; >>>>>>>>>> } >>>>>>>>>> #endif /* IPFIREWALL_FORWARD */ >>>>>>>>>> >>>>>>>>>> And this is something we are not sure if its correct. >>>>>>>>>> >>>>>>>>>> So my very obvious question is: >>>>>>>>>> >>>>>>>>>> Does anyone has a recent version of this patch to share? >>>>>>>>>> >>>>>>>>>> Can anyone familiar with ipfw source code help me with that? >>>>>>>>>> >>>>>>>>> I'm certainly not an expert, but I wonder if the patch your >>>>>>>>> referring >>>>>>>>> to is still required? Can you provide more detail about your >>>>>>>>> particular application? >>>>>>>>> >>>>>>>>> -Brandon >>>>>>>> >>>>>>>> Yes, its still required since ipfw fwd ignores layer2 frames. >>>>>>>> >>>>>>>> The application is the very same: squid. I mean, Lusca in fact >>>>>>>> (squid >>>>>>>> fork). >>>>>>>> >>>>>>>> Thank you for your interest. >>>>>>> >>>>>>> Cisco/Ironport have a patch that does this.. >>>>>>> I had permission to bring it back when I worked there but never got >>>>>>> it >>>>>>> committed. >>>>>>> >>>>>>> Adrian, was it part of the set I gave you? >>>>>> >>>>>> Hello Elischer, >>>>>> >>>>>> Was this made public? >>>>>> >>>>>> I hope Chadd has some good news. In fact I tent to use with Lusca in >>>>>> tproxy mode. I bet this is the only missing piece of software. >>>>>> >>>>> I just dug up my old changes. >>>>> do you want to fwd from a bridge? or what? >>>>> (it makes a difference what patches are needed) >>>>> >>>>> If you want to fwd from a bridge to make a transparent layer 2 proxy, >>>>> this >>>>> may help.. >>>>> >>>>> >>>>> Here are parts of it that may be relevent: >>>>> these are old (2007 I think) but may be of use still. >>>>> >>>>> adrian had the full set at >>>>> >>>>> ==quote adrian===== >>>>> The stuff is in p4 now, but I haven't tested it out at all. >>>>> >>>>> //depo/projects/adrian_spoof_clientip/ I -think-. >>>>> == end quote=== >>>>> >>>>> >>>>> >>>>> >>>>> Index: net/if_bridge.c >>>>> =================================================================== >>>>> RCS file: /usr/local/cvsroot/freebsd/src/sys/net/if_bridge.c,v >>>>> retrieving revision 1.107 >>>>> diff -u -r1.107 if_bridge.c >>>>> --- net/if_bridge.c 6 Nov 2007 23:01:42 -0000 1.107 >>>>> +++ net/if_bridge.c 28 Nov 2007 06:59:10 -0000 >>>>> @@ -2908,6 +2908,11 @@ >>>>> struct ip *ip; >>>>> struct llc llc1; >>>>> u_int16_t ether_type; >>>>> + int is_ip = 0; >>>>> +#ifdef IPFIREWALL_FORWARD >>>>> + struct m_tag *fwd_tag; >>>>> +#endif >>>>> + >>>>> >>>>> snap = 0; >>>>> error = -1; /* Default error if not error == 0 */ >>>>> @@ -2967,6 +2972,7 @@ >>>>> #ifdef INET6 >>>>> case ETHERTYPE_IPV6: >>>>> #endif /* INET6 */ >>>>> + is_ip = 1; >>>>> break; >>>>> default: >>>>> /* >>>>> @@ -3024,6 +3030,30 @@ >>>>> >>>>> if (*mp == NULL) >>>>> return (error); >>>>> + >>>>> +#ifdef IPFIREWALL_FORWARD >>>>> + /* >>>>> + * Did the firewall want to forward it somewhere? >>>>> + * If so, let the ip stack handle it. >>>>> + */ >>>>> + if (i == 0&& args.next_hop != NULL&& >>>>> + is_ip /*&& src != NULL */) { >>>>> + >>>>> + fwd_tag = m_tag_get(PACKET_TAG_IPFORWARD, >>>>> + sizeof(struct sockaddr_in), >>>>> M_NOWAIT); >>>>> + if (fwd_tag == NULL) >>>>> + goto drop; >>>>> + bcopy(args.next_hop, (fwd_tag+1), >>>>> + sizeof(struct sockaddr_in)); >>>>> + m_tag_prepend(*mp, fwd_tag); >>>>> + >>>>> + if (in_localip(args.next_hop->sin_addr)) >>>>> + (*mp)->m_flags |= M_FASTFWD_OURS; >>>>> + ether_demux(src, *mp); >>>>> + return (NULL); >>>>> + } >>>>> +#endif >>>>> + >>>>> >>>>> if (DUMMYNET_LOADED&& (i == IP_FW_DUMMYNET)) { >>>>> >>>>> ================== >>>>> Index: netinet/ip_fw2.c >>>>> =================================================================== >>>>> RCS file: /usr/local/cvsroot/freebsd/src/sys/netinet/ip_fw2.c,v >>>>> retrieving revision 1.178 >>>>> diff -u -r1.178 ip_fw2.c >>>>> --- netinet/ip_fw2.c 28 Oct 2007 17:12:47 -0000 1.178 >>>>> +++ netinet/ip_fw2.c 28 Nov 2007 06:59:10 -0000 >>>>> >>>>> @@ -3446,8 +3507,10 @@ >>>>> case O_FORWARD_IP: { >>>>> struct sockaddr_in *sa; >>>>> sa =&(((ipfw_insn_sa *)cmd)->sa); >>>>> +#if 0 >>>>> if (args->eh) /* not valid on layer2 >>>>> pkts >>>>> */ >>>>> break; >>>>> +#endif >>>>> if (!q || dyn_dir == MATCH_FORWARD) { >>>>> if (sa->sin_addr.s_addr == >>>>> INADDR_ANY) { >>>>> >>>>> bcopy(sa,&args->hopstore, >>>>> >>>>> ============================================= >>>>> Index: netinet/ip_output.c >>>> >>>> Dear Julian, >>>> >>>> Is anything missing from the above code? Say, like ip_output stuff? >>>> >>>> I have tried what you sent me, compiled fine but did not work. >>>> >>>> Here is my only rule (I have tried both with and without layer2 on the >>>> rule): >>>> >>>> 00001 36 4338 fwd 127.0.0.1,80 tcp from any to not me >>>> dst-port 80 layer2 >>>> 65535 32842101 2107060460 allow ip from any to any >>>> >>>> Here are the sysctl tunables: >>>> >>>> net.link.bridge.ipfw: 1 >>>> net.link.bridge.inherit_mac: 0 >>>> net.link.bridge.log_stp: 0 >>>> net.link.bridge.pfil_local_phys: 0 >>>> net.link.bridge.pfil_member: 0 >>>> net.link.bridge.pfil_bridge: 1 >>>> net.link.bridge.ipfw_arp: 0 >>>> net.link.bridge.pfil_onlyip: 0 >>>> net.link.ether.inet.log_arp_permanent_modify: 1 >>>> net.link.ether.inet.log_arp_movements: 1 >>>> net.link.ether.inet.log_arp_wrong_iface: 1 >>>> net.link.ether.inet.proxyall: 0 >>>> net.link.ether.inet.useloopback: 1 >>>> net.link.ether.inet.maxtries: 5 >>>> net.link.ether.inet.max_age: 1200 >>>> net.link.ether.ipfw: 1 >>>> >>>> And my bridge: >>>> >>>> bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 >>>> mtu >>>> 1500 >>>> ether 16:52:8e:91:2f:45 >>>> id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 >>>> maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200 >>>> root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 >>>> member: vr0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> >>>> ifmaxaddr 0 port 5 priority 128 path cost 200000 >>>> member: sis0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> >>>> ifmaxaddr 0 port 1 priority 128 path cost 200000 >>>> >>>> The ipfw counter gets increased by nothing hits by Apache. Instead I >>>> go to Internet directly. >>>> >>>> sis0 is on internet, vr0 is cross-over to the laptop (customer). >>>> >>>> How should I debug it? >>>> >>>> >>> basically I woud suggest code inspection for a start.. >>> >>> look at where ipfw is called (just before where the patch went in) and >>> follow the packet up into ipfw >>> and back, and read what it would do.. >>> >>> It's actually not a very hard path to follow. >>> >>> I'll try look at it after work.. >> >> Hello Julian / Adrian. >> >> Thank you for your attention. A friend added some log entries so we >> could try to find out what gets run and what doesnt. >> >> Here is my current patch against RELENG_8: >> >> --- if_bridge.c.orig 2010-09-11 22:02:36.000000000 +0000 >> +++ if_bridge.c 2010-10-05 17:59:13.000000000 +0000 >> @@ -2957,6 +2957,13 @@ >> struct ip *ip; >> struct llc llc1; >> u_int16_t ether_type; >> + int is_ip = 0; >> +#ifdef IPFIREWALL_FORWARD >> + struct m_tag *fwd_tag; >> +#endif >> + >> + >> + >> >> snap = 0; >> error = -1; /* Default error if not error == 0 */ >> @@ -3016,6 +3023,8 @@ >> #ifdef INET6 >> case ETHERTYPE_IPV6: >> #endif /* INET6 */ >> + is_ip=1; >> + log(LOG_NOTICE, "Entered 0: is_ip=%i\n",is_ip); >> break; >> default: >> /* >> @@ -3091,6 +3100,32 @@ >> if (*mp == NULL) >> return (error); >> >> +#ifdef IPFIREWALL_FORWARD >> + /* >> + * Did the firewall want to forward it somewhere? >> + * If so, let the ip stack handle it. >> + */ >> + log(LOG_NOTICE, "Entered 1"); >> + if (i == 0&& args.next_hop != NULL&& >> + is_ip /*&& src != NULL */) { >> + log(LOG_NOTICE, "Entered 2"); >> + >> + fwd_tag = m_tag_get(PACKET_TAG_IPFORWARD, >> + sizeof(struct sockaddr_in), >> M_NOWAIT); >> + if (fwd_tag == NULL) >> + goto drop; >> + bcopy(args.next_hop, (fwd_tag+1), >> + sizeof(struct sockaddr_in)); >> + m_tag_prepend(*mp, fwd_tag); >> + >> + if (in_localip(args.next_hop->sin_addr)) >> + (*mp)->m_flags |= M_FASTFWD_OURS; >> + ether_demux(src, *mp); >> + return (NULL); >> + } >> +#endif >> + >> + >> if (ip_dn_io_ptr&& (i == IP_FW_DUMMYNET)) { >> >> /* put the Ethernet header back on */ >> --- ../netinet/ipfw/ip_fw2.c.orig 2010-09-16 15:11:17.000000000 >> +0000 >> +++ ../netinet/ipfw/ip_fw2.c 2010-10-06 12:17:12.000000000 +0000 >> @@ -2059,8 +2059,14 @@ >> break; >> >> case O_FORWARD_IP: >> - if (args->eh) /* not valid on layer2 >> pkts */ >> +#if 0 >> + /* not valid on layer2 pkts */ >> + if (args->eh) { >> + log(LOG_NOTICE, "ip_fw2.c Entered >> 1"); >> break; >> + } >> + log(LOG_NOTICE, "ip_fw2.c Entered 2"); > > these will never happen as they are in the #if 0 section. > > the #if 0 is to REMOVE that code from being compiled. > > >> +#endif Hello Julian, Thank you again for your feedback. I appreciate it very much. On my understanding this "if 0" was to really ignore this portion of code, because as I understand what is does is to break (leave the loop) if the packet is on layer2, and this is something we would not want, but I guess I am wrong. I tested now with your suggestion, and what we get is: Oct 7 15:45:16 phoenix kernel: Entered 0: is_ip=1 Oct 7 15:45:16 phoenix kernel: ip_fw2.c Entered 1 Oct 7 15:45:16 phoenix kernel: Entered 0: is_ip=1 Oct 7 15:45:50 phoenix last message repeated 29 times Oct 7 15:47:53 phoenix last message repeated 237 times Oct 7 15:57:56 phoenix last message repeated 1029 times Oct 7 16:02:51 phoenix last message repeated 655 times Oct 7 16:02:51 phoenix kernel: ip_fw2.c Entered 1 Oct 7 16:02:51 phoenix kernel: Entered 0: is_ip=1 Oct 7 16:03:23 phoenix last message repeated 54 times Oct 7 16:05:24 phoenix last message repeated 345 times Oct 7 16:15:26 phoenix last message repeated 1135 times Oct 7 16:15:33 phoenix last message repeated 8 times So yes, we entered on ipfw code now, and executed only the instruction before we "break". The curious thing is that the counter did not count now with both: 00001 0 0 fwd 127.0.0.1,80 tcp from any to not me dst-port 80 layer2 00001 0 0 fwd 127.0.0.1,80 tcp from any to not me dst-port 80 How can I move forth? -- =========== Eduardo Meyer pessoal: dudu.meyer@gmail.com profissional: ddm.farmaciap@saude.gov.brhelp
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AANLkTik2KEYACzjfTS%2BXpB3OiaJL-uYckbLbf2C0DWaS>