Date: Thu, 22 Jul 2010 20:16:01 +1000 (EST) From: Ian Smith <smithi@nimnet.asn.au> To: Spil Oss <spil.oss@gmail.com> Cc: freebsd-ipfw@freebsd.org, freebsd-stable@freebsd.org, snasonov@bcc.ru Subject: Re: Changes to ipfw in 8.1 Message-ID: <20100722153846.S62748@sola.nimnet.asn.au> In-Reply-To: <AANLkTinZmQ2sjIAwvcKv62nWvKlODtkFCvh32sbC2-k3@mail.gmail.com> References: <AANLkTin6Ao8etOxX180N1NcI7M20eYFCO1ooCMREk7pg@mail.gmail.com> <AANLkTimDth1nShNSrCY_OPjqNELl8oWFD54_hdIKtN-j@mail.gmail.com> <AANLkTinZmQ2sjIAwvcKv62nWvKlODtkFCvh32sbC2-k3@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --0-1630664149-1279790660=:62748 Content-Type: TEXT/PLAIN; CHARSET=ISO-8859-1 Content-Transfer-Encoding: 8BIT Content-ID: <20100722194247.L62748@sola.nimnet.asn.au> On Wed, 21 Jul 2010, Spil Oss wrote: > Hi Sergey, > > I'm dumbstruck! > > Switching 'ip' to 'ip4' in both the divert rules fixed my problem. > Personally I think that should go into the UPDATING file as well. I > wouldn't have found it if you hadn't told me! > > Many thanks, > > Spil. This points to a number of issues, not least the effect of people being led to using the currently dreadful IPFW section of the handbook, which contains so many conceptual and factual errors that I really don't know where to begin on a solution .. so I won't discuss that further except as it relates to the actual ruleset you're using. In your just-filed PR http://www.freebsd.org/cgi/query-pr.cgi?pr=148827 you say your ruleset is based on '30.6.5.7 An Example NAT and Stateful Ruleset', so I'm assuming it's broadly based on example #2 there. > On Wed, Jul 21, 2010 at 9:08 PM, Spil Oss <spil.oss@gmail.com> wrote: > > Hi Sergey, > > > > Has the change from ip to ip4 solved the problem for you? The > > documentation states that proto 'ip' is the same as 'all' "Matches any > > packet." > > > > Rule # 60 > > $cmd 060 skipto 1000 ip6 from any to any > > will have already skipped to the ipv6 rules block thus proto 'ip' > > should always match remaining packets. You don't show any rules beyond number 510, so we must take it on faith that you're handling your ip6 traffic appropriately. I don't know much about it, and will be guided by the IPv6 rules lately in rc.firewall. However if natd is barfing on being passed ip6 packets, that should be fixed in natd, which should do nothing with packets it doesn't care about - as it does with ip4 packets not eligible for NAT translation - except to re-enter the firewall at the next higher-numbered rule, which of course relies on sysctl net.inet.ip.fw.one_pass being set to 0. Either that or ipfw should explicitly decline to divert non-ip4 traffic. I don't know whether or how this would also affect ipfw in-kernel nat. If in fact no ip6 packets are being passed to natd as rule 60 indicates, at least for traffic inbound to ipfw, then that is indeed strange, but I suspect that on the outbound pass, using this rather confusing 'skipto 500 .. keep-state' logic, perhaps some outbound ip6 packets may have been being passed to natd .. Could you check to see if it works only changing the _outbound_ divert rule from ip to ip4, leaving the inbound one at 'ip'? If so, this would validate your original theory re rule 60, on the inbound pass. If not, it may be a useful data point for resolving the problem. > > Meanwhile I found bug 148137 [ipfw] call order of natd and ipfw startup scripts > > http://www.freebsd.org/cgi/query-pr.cgi?pr=148137&cat=conf > > Don't know if that's directly related, but it may be worth a try to > > revert back to the RELENG_8_0 script. > > > > Will let you now my findings. Did you try that, to see whether it was an issue? More below .. > > Kind regards, > > > > Spil. > > > > > > On Wed, Jul 21, 2010 at 2:57 PM, Sergey G Nasonov <snasonov@bcc.ru> wrote: > >> Hello Spill, > >> > >> I have get the same trouble after updating my 8.0 Stable. I thing you need > >> modify some firewall rules. > >> > >> Please change > >> > >> $cmd 100 divert natd ip from any to any in via $pif # Mangle inbound > >> > >> to > >> > >> $cmd 100 divert natd ip4 from any to any in via $pif # Mangle inbound > >> > >> and > >> > >> $cmd 500 divert natd ip from any to any out via $pif > >> > >> to > >> > >> $cmd 500 divert natd ip4 from any to any out via $pif > >> > >> accordingly. > >> > >> -- > >> > >> Best Regards, > >> > >> Nasonov Sergey > > > > > > On Wed, Jul 21, 2010 at 11:40 AM, Spil Oss <spil.oss@gmail.com> wrote: > >> Hi, > >> > >> Testing FreeBSD 8.1 I noticed that I seem to have routing or nat or > >> firewall issues. (csupped RELENG_8_1 which was -RELEASE not -RC last > >> night?) > >> - 8.1 booted fine > >> - connections from the system itself were fine > >> - connections from my jails to the internet were not working > >> - connections from my LAN/WLAN to the internet were not working > >> Reverting back to 8.0-p2 with the same configuration works fine. > >> > >> In UPDATING I see that rc.firewall and rc.firewall6 were unified. > >> > >> Setup is > >> - xl0 connected to internet/public IP via dhcp > >> - bge0/wlan0(ath0) connected to LAN > >> - jails have ip's on bge0 in the same subnet as the LAN > >> - allow all from any to any via bge0|wlan0|lo0 The latter point looks problematic, see below. > >> - NAT using natd > >> > >> My guess is that something's changed to ipfw that is affecting my > >> network settings. Any clues where I went wrong? > >> > >> Help appreciated/ Kind regards, > >> > >> Spil. > >> > >> rc.conf: > >> firewall_enable="YES" > >> firewall_script="/etc/ipfw.rules" > >> > >> natd.conf > >> interface xl0 > >> dynamic yes > >> same_ports yes > >> # http/https to http jail > >> redirect_port tcp 192.168.2.3:80 80 > >> redirect_port tcp 192.168.2.3:443 443 > >> > >> Part of /etc/ipfw.rules > >> #!/bin/sh > >> cmd="ipfw -q add" > >> skip="skipto 500" > >> pif=xl0 > >> pif6=gif0 > >> ext6="2001:dead:beef:1::1" > >> ks="keep-state" > >> > >> ipfw -q -f flush > >> > >> # Allow internal traffic > >> $cmd 002 allow all from any to any via bge0 # exclude LAN traffic > >> $cmd 003 allow all from any to any via lo0 # exclude loopback traffic > >> $cmd 004 allow all from any to any via wlan0 # exclude WLAN traffic > >> $cmd 005 allow all from any to any via bridge0 # exclude WLAN traffic > >> $cmd 006 allow all from any to any via tun0 # exclude WLAN traffic There are problems wih this, based on the apparent misunderstanding (by the present IPFW section's author) of how 'via' works when direction (in or out) and/or a specific interface (recv or xmit) is not specified. I'm assuming that apart from lo0, traffic coming in on some or all of bge0, wlan0, bridge0 and tun0 is to be translated by NAT if destined to be going out on the public interface xl0, right? Certainly you've indicated that your jails are on bge0 with 192.168.x.x addresses, but these are local aliases which wouldn't show bge0 as the recv interface: "A packet may not have a receive or transmit interface: packets originating from the local host have no receive interface, while packets destined for the local host have no transmit interface." Let's consider just one initial packet coming in from bge0 from some private LAN address, other than your local jail IPs, that's destined for an outside address and so needs to be NAT'd before transmission. On the inbound pass, the packet comes in on bge0 and so satisfies 'via bge0' so is immediately allowed in, and that's it. That may be ok, as NAT is here only to be performed on the outbound pass, but it could be anything, and there's no protection here against it being spoofed, as opposed to placement of NAT rules in the rc.firewall 'simple' ruleset. So having been allowed in, after kernel routing (but before NAT, as we haven't reached any check-state rule yet) it reenters the firewall for the outbound pass with its xmit interface set to xl0, but with its recv interface still set to bge0. Thus the unqualified 'via bge0' is still true, so this packet is passed immediately, with source address still set to the private LAN address, without having been translated by NAT. As are similarly any packets bound for the outside initially coming in on wlan0, bridge0 or tun0 that originate from other than the local host. This could be obviated by replacing 'via $if' by 'in recv $if' in the rules above, so such packets going out pass through the firewall rules, including setting state for reply packets received on the outside iface. So as is, the rules below are only applied to outbound packets that are NOT received on those interfaces, which doesn't sound like what you'd intend, except packets generated by or destined for the local box. > >> # Allow all encapulated IPv6 to/from tunnel PoP > >> $cmd 010 allow ip4 from <tunnel-provider-ipv4> to me via $pif > >> $cmd 010 allow ip4 from me to <tunnel-provider-ipv4> via $pif Taken on faith. > >> # Black-hole some stuff using tables > >> $cmd 050 drop ip from "table(17)" to any in via $pif > >> $cmd 050 drop ip from any to "table(17)" out via $pif > >> > >> # Separate IPv6 rules (no NAT!) > >> $cmd 060 skipto 1000 ip6 from any to any > >> > >> $cmd 100 divert natd ip from any to any in via $pif # Mangle inbound > >> packets from external > >> $cmd 101 check-state > >> > >> # Authorized outbound packets > >> $cmd 130 $skip icmp from any to any out via $pif $ks > >> $cmd 150 $skip tcp from any to any out via $pif $ks > >> $cmd 151 $skip udp from any to any out via $pif $ks > >> > >> $cmd 200 allow udp from 10.50.0.1 to me 68 in $ks > >> > >> # Deny all inbound traffic from non-routable reserved address spaces > >> $cmd 300 unreach host all from 192.168.0.0/16 to any in via $pif > >> #RFC 1918 private IP > >> $cmd 301 unreach host all from 172.16.0.0/12 to any in via $pif > >> #RFC 1918 private IP > >> $cmd 302 unreach host all from 10.0.0.0/8 to any in via $pif > >> #RFC 1918 private IP > >> $cmd 303 unreach host all from 127.0.0.0/8 to any in via $pif #loopback > >> $cmd 304 unreach host all from 0.0.0.0/8 to any in via $pif #loopback > >> $cmd 305 unreach host all from 169.254.0.0/16 to any in via $pif > >> #DHCP auto-config > >> $cmd 306 unreach host all from 192.0.2.0/24 to any in via $pif > >> #reserved for docs > >> $cmd 307 unreach host all from 204.152.64.0/23 to any in via $pif #Sun cluster > >> $cmd 308 unreach host all from 224.0.0.0/3 to any in via $pif > >> #Class D & E multicast > >> > >> # Deny packets that did not match the dynamic rule table > >> #$cmd 330 deny all from any to any frag in via $pif # All late fragments The author (here and in other writings) expounds that fragmented packets are necessarily bad, and indicate an attack of some sort. Not in fact so; eg if you were using say zen.spamhaus.org as an RBL you'll be seeing valid fragmented UDP port 53 packets of around 2000 bytes total all day. I gather DNSSEC will also require acceptance of fragmented datagrams. The default rc.firewall rules pass IP frags, and I suggest you do too. > >> #$cmd 332 deny tcp from any to any established in via $pif # Deny ACK > >> > >> # Authorized inbound packets > >> $cmd 400 allow icmp from any to any icmptypes 0,11 # echo reply and TTL-exceeded Add icmptype 3 here to pass unreachables including path MTU discovery. > >> $cmd 420 allow tcp from any to me ssh in via $pif setup $ks > >> $cmd 421 allow tcp from any to me smtp in via $pif > >> $cmd 422 allow tcp from any to me http in via $pif > >> $cmd 423 allow tcp from any to me https in via $pif > >> $cmd 424 allow tcp from any to me imaps in via $pif > >> > >> #$cmd 449 unreach host ip from any to any in via $pif > >> $cmd 448 reject log all from any to any in via $pif > >> $cmd 449 reject log all from any to any out via $pif > >> $cmd 450 reject log ip from any to any Security-wise you'd be better off just dropping these using deny than reject (deprecated, equivalent to unreach host), here and above. However I don't wish to let such details obscure the more major issue. > >> > >> # This is skipto location for outbound stateful rules > >> $cmd 500 divert natd ip from any to any out via $pif > >> $cmd 510 allow ip from any to any cheers, Ian --0-1630664149-1279790660=:62748--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20100722153846.S62748>