Date: Sat, 22 Aug 2015 14:25:07 +1000 From: Nathan Aherne <nathan@reddog.com.au> Cc: freebsd-ipfw@freebsd.org Subject: Re: IPFW and in kernel nat Message-ID: <C84335C7-3F27-42BC-A682-D1CB13321DC2@reddog.com.au> In-Reply-To: <1440215094.231702740.aw1gnxg5@frv34.fwdcdn.com> References: <C513D7E5-D890-4230-A4EE-B94180444396@reddog.com.au> <1440215094.231702740.aw1gnxg5@frv34.fwdcdn.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Vitaliy, Thanks for sending this though, its much appreciated. I will take = onboard your recommendation about using vimage and have a bit more of a = read about it, in concert with the rules you have posted. At the moment, = I would like to get my server working without vimage with the view to = implementing it later. Regards, Nathan > On 22 Aug 2015, at 2:03 pm, wishmaster <artemrts@ukr.net> wrote: >=20 > Hi, =20 >=20 > --- Original message --- > From: "Nathan Aherne" <nathan@reddog.com.au> > Date: 22 August 2015, 06:28:51 >=20 > Hi Everyone, >=20 > First time message to this list, so I am sorry if I do something = against the rules. >=20 > I have posted this questions on the FreeBSD forums in two different = places but have not had a single response in several days, so thought I = might get more success here. I have spent many days solely reading about = IPFW and in kernel nat in the hopes that I would be able to get what I = want to achieve working without having to ask for help. I have found it = extremely hard to find what I would think is a regular use use case for = IPFW and nat. There are examples but there are just so many that are bad = or hard to follow, including the handbook and IPFW that its extremely = easy to get confused, as I have. Because of this, I will be posting my = eventual solution back to the forums in the hopes that it stops someone = from wasting days. >=20 > I have a single wan interface (bce0) that has two public IPs attached = to it. I would like to use one of these IPs for the host (wanip1) and = jails and another for a jail (wanip2) that requires a public IP. I have = a cloned lo0 interface, lo1 with a subnet of 10.1.0.0/24 which all the = jails reside on. I would like to forward ports 80,443 coming in on = wanip1 to the jail proxy, which then forwards the traffic off to the = correct jail. I would like for the host server to be able to have its = own firewall and also receive traffic on port 65222. I have a couple of = other jails that require some other ports as you will see in my = ipfw.rules script. I would also like each jail to be able to connect to = the internet. >=20 > I would like the jails to be able to have their own firewalls, which I = currently allow by including a firewall script from within each jail. >=20 > I have net.inet.ip.fw.one_pass=3D0 set in /etc/sysctl.conf >=20 > Below is my ipfw.rules script. I would super appreciate it if someone = could show me where I have gone wrong. >=20 > = **************************************************************************= **************************************************************************= ************************* > #!/bin/sh >=20 > ###################################################### > # Configuration > wif=3D"bce0" # WAN interface > wip=3D"119.111.111.111" # WAN IP >=20 > j1if=3D"lo1" # Jails Interface > j1net=3D"10.1.0.0/24" # Jails Network > skip=3D"skipto 30000" > jcmd=3D"ipfw -q add 10000=E2=80=9D # Jails rules are inserted here > ###################################################### > # IPFW variables > cmd=3D"ipfw -q add" > ks=3D"keep-state" > sks=3D"setup keep-state" > ###################################################### > ipfw -q -f flush # Flush all rules > ###################################################### > # NAT on Jail1 WAN IP > ipfw nat 1 config ip $wip same_ports unreg_only reset \ > redirect_port tcp 10.1.0.1:80 80 \ > redirect_port tcp 10.1.0.1:443 443 \ > redirect_port tcp 10.1.0.2:65432 65432 \ > redirect_port tcp 10.1.0.3:65444 65444 \ > redirect_port tcp 10.1.0.3:65333 65333 >=20 > # Jail1 Network - allow all traffic > $cmd 10 allow ip from any to any via $j1if > ###################################################### > # Allow all traffic on Loopback > $cmd 999 allow ip from any to any via lo0 >=20 > # NAT Rule for incoming packets on WAN IP > $cmd 1000 nat 1 ip4 from any to any in via $wif >=20 > # Check stage table > $cmd 2000 check-state > ###################################################### > # HOST ONLY > # Ping > $cmd 2100 allow icmp from $wip to any out $ks > $cmd 2101 allow icmp from any to $wip in $ks > # DNS > $cmd 2102 allow tcp from $wip to any 53 out $sks > $cmd 2103 allow udp from $wip to any 53 out $ks > # Ports > $cmd 2104 allow tcp from $wip to any 80 out $ks > $cmd 2105 allow tcp from $wip to any 433 out $ks > # SSH > $cmd 2106 allow tcp from $wip to any 22 out $ks > $cmd 2107 allow tcp from $wip to any 65222 out $ks > $cmd 2108 allow tcp from any to $wip 65222 in $ks > # OpenNTP > $cmd 2109 allow udp from $wip to any 123 out $ks > ###################################################### > # Jails =20 > # Out > $cmd 3004 $skip ip4 from any to any out xmit $wif $ks > # In > $cmd 3000 $skip tcp from any to any dst-port 80 in via $wif $sks > $cmd 3001 $skip tcp from any to any dst-port 443 in via $wif $sks > $cmd 3002 $skip tcp from any to any dst-port 65444 in via $wif $sks > $cmd 3003 $skip tcp from any to any dst-port 65432 in via $wif $sks > $cmd 3004 $skip tcp from any to any dst-port 65333 in via $wif $sks > ###################################################### >=20 > # jail.example.com > . /usr/jails/jail.example.com/ipfw.rules >=20 > $cmd 30000 nat 1 ip from $jnet to any out >=20 > ###################################################### > # Deny Remainder and Log > $cmd deny log all from any to any > ##################################################### > As for me, ipfw is quite complex to configure in not trivial = configurations where there are many interfaces. So my advise, you should = use per-interface ACL and VIMAGE for jails for better traffic control. = Small example below from the real machine. >=20 > # tables with interfaces, in > ipfw -fq table 10 flush > ipfw table 10 add nfe0 3000 > ipfw table 10 add ue0 3500 > ipfw table 10 add lo0 4000 > ipfw table 10 add sk0 5000 > ipfw table 10 add epair1a 6000 > ipfw table 10 add epair2a 6000 > ipfw table 10 add epair3a 6000 > ipfw table 10 add epair4a 6000 > ipfw table 10 add epair999a 6000 > ipfw table 10 add tun1 6100 >=20 > # out > ipfw -fq table 11 flush > ipfw table 11 add nfe0 13000 > ipfw table 11 add ue0 13500 > ipfw table 11 add lo0 14000 > ipfw table 11 add sk0 15000 > ipfw table 11 add epair1a 16000 > ipfw table 11 add epair2a 16000 > ipfw table 11 add epair3a 16000 > ipfw table 11 add epair4a 16000 > ipfw table 11 add epair999a 16000 > ipfw table 11 add tun1 16100 >=20 > #nat rules ..... >=20 > $cmd 100 skipto tablearg log all from any to any in recv "table(10)" > $cmd 110 skipto tablearg log all from any to any out xmit "table(11)" >=20 > #rules for interfaces >=20 > # rl0 > $cmd 1000 allow log ip4 from any to any > $cmd 1099 deny log all from any to any >=20 > $cmd 11000 allow log ip4 from any to any > $cmd 11099 deny log all from any to any >=20 > # epair[0-9]a in (from jail) > $cmd 6000 nat 2 log ip4 from "table(12)" 80,81,443,5555 to me > $cmd 6001 allow log ip4 from "table(12)" 80,81,443,5555 to $nonroute = 1025-65535 > $cmd 6003 fwd 127.0.0.1,25 log ip4 from any to me 25 > $cmd 6010 check-state log >=20 > $cmd 6020 allow log ip4 from 192.168.254.10 to not $nonroute $ks > $cmd 6030 allow log ip4 from 192.168.254.2,192.168.254.254 to not = $nonroute $ks # allows requests to wild world from basejail & j1 = only! >=20 > $cmd 6099 deny log all from any to any >=20 > # epair[0-9]a out (to jail) > $cmd 16000 nat 2 log ip4 from $nonroute 1025-65535 to "table(12)" = 80,81,443,5555 > $cmd 16010 allow log ip4 from me to "table(12)" $ks # for www = redirect ext_if -> jail_if and for $lan -> www jail >=20 > $cmd 16090 check-state log >=20 > $cmd 16099 deny log all from any to any >=20 > ### IN ext_if sk0 > $cmd 5000 call 30000 log all from any to any >=20 > $cmd 5010 nat 1 log ip4 from any to me in > $cmd 5011 call 25000 log all from any to any >=20 > $cmd 5020 check-state log >=20 > $cmd 5100 allow log tcp from any to me 10001 setup limit src-addr 5 > $cmd 5122 allow log tcp from any to me 25 setup limit src-addr 100 > $cmd 5127 allow log tcp from any to me 2112 setup limit src-addr 10 > $cmd 5128 allow log tcp from any to me 49152-65535 limit src-addr 10 > $cmd 5150 allow log udp from any to me 11944 limit src-addr 5 > $cmd 5152 allow log ip4 from any to me 67,68 > $cmd 5200 allow log tcp from any to "table(12)" 80,81,443,5555 setup = limit src-addr 20 > $cmd 5210 allow log tcp from any to 192.168.10.2 55551 setup limit = src-addr 100 # my torrent > $cmd 5211 allow log udp from any to 192.168.10.2 55551 limit src-addr = 100 # my torrent > $cmd 5215 allow log udp from any to 192.168.254.10 11945 limit = src-addr 2 >=20 > $cmd 5310 allow log icmp from any to any icmptypes 0,3,4,8,11 >=20 > $cmd 5999 deny log all from any to any >=20 > ... > and so on > ..... >=20 >=20 > I massively use stateful feature and call subrequests. Hope this = helps. >=20 > Cheers, > Vitaliy >=20 >=20 >=20
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?C84335C7-3F27-42BC-A682-D1CB13321DC2>