Date: Tue, 13 Oct 2015 15:57:49 +1000 From: Nathan Aherne <nathan@reddog.com.au> Cc: freebsd-ipfw@freebsd.org Subject: Re: Kernel NAT issues Message-ID: <5B1C303D-49F6-4EC2-B5B1-5F5D6BE8D4BE@reddog.com.au> In-Reply-To: <C1C25100-FBD4-42F4-94F7-965B270D927F@reddog.com.au> References: <94B91F98-DE01-4A10-8AB5-4193FE11AF3F@reddog.com.au> <20151013142301.B67283@sola.nimnet.asn.au> <C1C25100-FBD4-42F4-94F7-965B270D927F@reddog.com.au>
next in thread | previous in thread | raw e-mail | index | archive | help
To further illustrate my issue, this is a small log output. I am running =E2=80=9Chost google.com <http://google.com/>=E2=80=9D in = the jail, which has the IP 10.0.0.1. The UNKNOWN line is logging on the = check-state rule. I would expect the first piece of traffic out would be = UNKNOWN (does not have an entry in the state table) but it seems the = returning traffic is also showing as UNKNOWN (the second 101). You can = see that the traffic is returning on the same port it went out on, so = its obviously the returning traffic. I am not sure why state is not = being kept? Oct 13 15:50:42 host4 kernel: ipfw: 101 UNKNOWN UDP 10.0.0.1:57446 = 8.8.8.8:53 out via bce0 Oct 13 15:50:42 host4 kernel: ipfw: 123 SkipTo 65501 UDP 10.0.0.1:57446 = 8.8.8.8:53 out via bce0 Oct 13 15:50:42 host4 kernel: ipfw: 65501 Nat UDP 10.0.0.1:57446 = 8.8.8.8:53 out via bce0 Oct 13 15:50:42 host4 kernel: ipfw: 101 UNKNOWN UDP 8.8.8.8:53 = 10.0.0.1:57446 in via bce0 Oct 13 15:50:42 host4 kernel: ipfw: 123 SkipTo 65501 UDP 8.8.8.8:53 = 10.0.0.1:57446 in via bce0 Oct 13 15:50:42 host4 kernel: ipfw: 65534 Deny UDP 8.8.8.8:53 = 10.0.0.1:57446 in via bce0 Regards, Nathan > On 13 Oct 2015, at 1:50 pm, Nathan Aherne <nathan@reddog.com.au> = wrote: >=20 > Hi Ian, >=20 > Thank you for your response. >=20 > I didn=E2=80=99t post my ruleset because I should be able to fix the = issue myself but I see now that my request to explain =E2=80=9Chow NAT = works=E2=80=9D was incorrect. >=20 > I have now included my ruleset below (as well as my initial email). >=20 > # Enable NAT > ipfw nat 1 config ip $jip same_ports log >=20 >=20 > 00005 allow ip from any to any via lo0 > 00006 deny ip from any to not me in via bce0 > 00100 nat 1 log ip from any to AAA.BBB.CCC.DDD recv bce0 > 00101 check-state > 00110 allow icmp from any to WWW.XXX.YYY <http://www.xxx.yyy/>.ZZZ = recv bce0 keep-state > 00111 allow tcp from any to WWW.XXX.YYY <http://www.xxx.yyy/>.ZZZ = dst-port 65222 recv bce0 setup keep-state > 00112 allow icmp from WWW.XXX.YYY <http://www.xxx.yyy/>.ZZZ to any = xmit bce0 keep-state > 00113 allow tcp from WWW.XXX.YYY <http://www.xxx.yyy/>.ZZZ to any = dst-port 53,80,443,22,65222 xmit bce0 setup keep-state > 00114 allow udp from WWW.XXX.YYY <http://www.xxx.yyy/>.ZZZ to any = dst-port 53,123 xmit bce0 keep-state > 00120 skipto 65501 log tcp from any to 10.0.0.0/16 recv bce0 setup = keep-state > 00121 skipto 65501 log udp from any to 10.0.0.0/16 recv bce0 = keep-state > 00122 skipto 65501 log tcp from 10.0.0.0/16 to not 10.0.0.0/16 xmit = bce0 setup keep-state > 00123 skipto 65501 log udp from 10.0.0.0/16 to not 10.0.0.0/16 xmit = bce0 keep-state > 00200 allow log tcp from any to 10.0.0.1 dst-port 22,80,443 in setup = keep-state > 00200 allow log tcp from 10.0.0.1 to any dst-port 22,80,443 out setup = keep-state > 00200 allow log udp from 10.0.0.1 to any dst-port 53 out keep-state > 00201 allow log tcp from any to 10.0.0.2 dst-port 22,80,443 in setup = keep-state > 00201 allow log tcp from 10.0.0.2 to any dst-port 22,80,443 out setup = keep-state > 00201 allow log udp from 10.0.0.2 to any dst-port 53 out keep-state > 65500 deny log ip from any to any > 65501 nat 1 log ip from 10.0.0.0/16 to not 10.0.0.0/16 xmit bce0 = keep-state > 65502 allow log ip from AAA.BBB.CCC.DDD to any xmit bce0 keep-state > 65534 deny log ip from any to any > 65535 deny ip from any to any >=20 > = **************************************************************************= ************ > I sent through a question to this list a little while ago and have = been trying to get IPFW NAT working since then. I have had some success = but not the success I need, everything is working correctly except NAT = rules for my particular use case.=20 >=20 > I have read every Google result on the first 50 pages when searching = for =E2=80=9CIPFW NAT=E2=80=9D or =E2=80=9CIPFW kernel NAT=E2=80=9D. I = would really appreciate it if someone could help me out. >=20 > My use case is as follows: >=20 > 1. I need to use hairpin NAT - I am using Jails behind a http proxy = and some jails need to be able to communicate with each other but only = over the WAN IP. This is why I have not use PF. > 2. Some jails need to be able to communicate with each other on the = private interface (lo1) > 3. IPFW is configured as default deny > 4. Each jail has a list of allowed ports for incoming and outgoing = connections, these are set on the jails private IP (10.0.0.0/16) > 5. I am using a stateful firewall. >=20 > At the moment I am testing my IPFW ruleset using =E2=80=9Chost = google.com <http://google.com/>; <http://google.com/ = <http://google.com/>>=E2=80=9D I can see the traffic leave the Jail, get = natted, the response come back from 8.8.8.8 and the traffic is then = denied. It seems like the state is not being checked or my rules are in = the wrong place. I feel that I should be able to fix this but I am = obviously misunderstanding is how NAT works.=20 >=20 > I was under the assumption that traffic flowed like this: >=20 > 1. Traffic comes from Jail 10.0.0.1 on lo1 interface, if traffic is = for public IP, the traffic is natted, it goes out the WAN interface, = comes back, is natted and switched to lo1 interface, state is checked = and it passes as returning traffic. >=20 > 2. Traffic comes from Jail 10.0.0.1 on lo1 interface, if traffic is = for private IP, the traffic is not natted, it stays on the lo1 interface = and goes directly to the 10.0.0.2 Jail. >=20 > I know I could answer my last question if =E2=80=9CI read the code=E2=80= =9D and I have tried but am not getting it. Is my understanding of IPFW = kernel NAT correct? >=20 > Regards, >=20 > Nathan >=20 > _______________________________________________ > freebsd-ipfw@freebsd.org <mailto:freebsd-ipfw@freebsd.org> mailing = list > https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw = <https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw>; > To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@freebsd.org = <mailto:freebsd-ipfw-unsubscribe@freebsd.org>=E2=80=9D >=20 > = **************************************************************************= ************ >=20 > Regards, >=20 > Nathan >=20 >> On 13 Oct 2015, at 1:37 pm, Ian Smith <smithi@nimnet.asn.au> wrote: >>=20 >> On Tue, 13 Oct 2015 12:33:52 +1000, Nathan Aherne wrote: >>=20 >>> I sent through a question to this list a little while ago and have=20= >>> been trying to get IPFW NAT working since then. I have had some=20 >>> success but not the success I need, everything is working correctly=20= >>> except NAT rules for my particular use case. >>=20 >> Unfortunately the rest of your message failed to quote properly here,=20= >> i.e not quoted indented as above, so I'll leave it out for now; = perhaps=20 >> it's my old mailer (pine) at fault. Maybe plain ASCII text would = help. >>=20 >> That said, without sharing your actual ruleset with us, sanitised if=20= >> need be, it seems unlikely that anyone will be able to work out what=20= >> might be happening here solely from your textual description. >>=20 >> cheers, Ian >=20 > _______________________________________________ > freebsd-ipfw@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw > To unsubscribe, send any mail to = "freebsd-ipfw-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5B1C303D-49F6-4EC2-B5B1-5F5D6BE8D4BE>