Date: Sun, 25 Jul 2010 03:23:45 +1000 (EST) From: Ian Smith <smithi@nimnet.asn.au> To: Spil Oss <spil.oss@gmail.com> Cc: freebsd-ipfw@freebsd.org Subject: Re: Changes to ipfw in 8.1 Message-ID: <20100725020910.B62748@sola.nimnet.asn.au> In-Reply-To: <AANLkTinNqt24QSfFlTn6A-e_AKZbt0wr7FewKiMhzalZ@mail.gmail.com> References: <AANLkTin6Ao8etOxX180N1NcI7M20eYFCO1ooCMREk7pg@mail.gmail.com> <AANLkTimDth1nShNSrCY_OPjqNELl8oWFD54_hdIKtN-j@mail.gmail.com> <AANLkTinZmQ2sjIAwvcKv62nWvKlODtkFCvh32sbC2-k3@mail.gmail.com> <20100722153846.S62748@sola.nimnet.asn.au> <AANLkTinNqt24QSfFlTn6A-e_AKZbt0wr7FewKiMhzalZ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --0-23540912-1279992225=:62748 Content-Type: TEXT/PLAIN; charset=ISO-8859-1 Content-Transfer-Encoding: 8BIT On Thu, 22 Jul 2010, Spil Oss wrote: > On Thu, Jul 22, 2010 at 12:16 PM, Ian Smith <smithi@nimnet.asn.au> wrote: > > > > This points to a number of issues, not least the effect of people being > > led to using the currently dreadful IPFW section of the handbook, which > > contains so many conceptual and factual errors that I really don't know > > where to begin on a solution .. so I won't discuss that further except > > as it relates to the actual ruleset you're using. > > I'm not a network admin, and happy that it does what I want at all. My > concern is mainly that an upgrade to 8.1 will break many systems. A fair enough concern. > > In your just-filed PR http://www.freebsd.org/cgi/query-pr.cgi?pr=148827 > > you say your ruleset is based on '30.6.5.7 An Example NAT and Stateful > > Ruleset', so I'm assuming it's broadly based on example #2 there. > > > > > On Wed, Jul 21, 2010 at 9:08 PM, Spil Oss <spil.oss@gmail.com> wrote: > > > > Hi Sergey, > > > > > > > > Has the change from ip to ip4 solved the problem for you? The > > > > documentation states that proto 'ip' is the same as 'all' "Matches any > > > > packet." > > > > > > > > Rule # 60 > > > > $cmd 060 skipto 1000 ip6 from any to any > > > > will have already skipped to the ipv6 rules block thus proto 'ip' > > > > should always match remaining packets. > > > > You don't show any rules beyond number 510, so we must take it on faith > > that you're handling your ip6 traffic appropriately. I don't know much > > about it, and will be guided by the IPv6 rules lately in rc.firewall. > > > > For completeness > ######################## IPv6 rules ################## > > # Allow the uptime pings from SixXS > $cmd 1000 allow icmp6 from 2001:dead:beef:c10::1 to > 2001:dead:beef:c10::2 in via $pif6 > $cmd 1000 allow icmp6 from me6 to 2001:dead:beef:c10::1 out via $pif6 > > # Open up all IPv6 > #$cmd 006 allow ip6 from any to any via $pif6 > > $cmd 1100 check-state > > # Authorized outbound packets > $cmd 1160 allow ip6 from any to any out via $pif6 $ks > $cmd 1161 allow udp from any to any out via $pif6 $ks > $cmd 1162 allow icmp6 from any to any out via $pif6 $ks I expect ip6 includes all ipv6 protocols, so udp (udp6?) and icmp6 - and tcp6 - will also pass on the first of these rules. If you want separate counts (or options) for different protocols, put the more specific ones first (tcp, udp, icmp etc) then an 'all' or here ip6 last to catch other protocols too. I know nothing about other ipv6-specific protocols. > # Authorized inbound packets > $cmd 1450 allow tcp from any to $ext6 dst-port ssh in via $pif6 > > $cmd 1999 reject log all from any to any in via $pif6 > $cmd 1999 reject log all from any to any out via $pif6 > $cmd 1999 reject log ip from any to any > > ######################## end of rules ################## > > However if natd is barfing on being passed ip6 packets, that should be > > fixed in natd, which should do nothing with packets it doesn't care > > about - as it does with ip4 packets not eligible for NAT translation - > > except to re-enter the firewall at the next higher-numbered rule, which > > of course relies on sysctl net.inet.ip.fw.one_pass being set to 0. > > That may be an issue.... > net.inet.ip.fw.one_pass: 1 Ah, then you'd best add net.inet.ip.fw.one_pass=0 to /etc/sysctl.conf .. > > Either that or ipfw should explicitly decline to divert non-ip4 traffic. > > I don't know whether or how this would also affect ipfw in-kernel nat. > > > > If in fact no ip6 packets are being passed to natd as rule 60 indicates, > > at least for traffic inbound to ipfw, then that is indeed strange, but I > > suspect that on the outbound pass, using this rather confusing 'skipto > > 500 .. keep-state' logic, perhaps some outbound ip6 packets may have > > been being passed to natd .. > > > > Could you check to see if it works only changing the _outbound_ divert > > rule from ip to ip4, leaving the inbound one at 'ip'? If so, this would > > validate your original theory re rule 60, on the inbound pass. If not, > > it may be a useful data point for resolving the problem. > > ipfw add 99 divert natd ip from any to any in via $pif > ipfw delete 100 > > NATting still works > > ipfw delete 500 > ipfw add 500 divert natd ip from any to any out via $pif > > NATting broken With your later correction: : ipfw delete 500 : ipfw add 500 divert natd ip4 from any to any out via $pif : : NATting works again So no ip6 packets hit the inbound divert rule as rule 60 bypassed it, but it looks like an ip6 pkt must have hit your outbound divert rule. Adding a 'count ip6 from any to any' just before the outbound divert rule would prove that, one way or the other. Or a 'deny log ip6 ...' or even a 'skipto $somewhere ip6 ...' > > > > Meanwhile I found bug 148137 [ipfw] call order of natd and ipfw startup scripts > > > > http://www.freebsd.org/cgi/query-pr.cgi?pr=148137&cat=conf > > > > Don't know if that's directly related, but it may be worth a try to > > > > revert back to the RELENG_8_0 script. > > > > > > > > Will let you now my findings. > > > > Did you try that, to see whether it was an issue? More below .. > > Sorry, didn't try reverting back to the 8.0 script. The ip -> ip4 > change fixed it so I didn't need the other option. > > > > > > > Kind regards, > > > > > > > > Spil. > > > > > > > > > > > > On Wed, Jul 21, 2010 at 2:57 PM, Sergey G Nasonov <snasonov@bcc.ru> wrote: > > > >> Hello Spill, > > > >> > > > >> I have get the same trouble after updating my 8.0 Stable. I thing you need > > > >> modify some firewall rules. > > > >> > > > >> Please change > > > >> > > > >> $cmd 100 divert natd ip from any to any in via $pif # Mangle inbound > > > >> > > > >> to > > > >> > > > >> $cmd 100 divert natd ip4 from any to any in via $pif # Mangle inbound > > > >> > > > >> and > > > >> > > > >> $cmd 500 divert natd ip from any to any out via $pif > > > >> > > > >> to > > > >> > > > >> $cmd 500 divert natd ip4 from any to any out via $pif > > > >> > > > >> accordingly. > > > >> > > > >> -- > > > >> > > > >> Best Regards, > > > >> > > > >> Nasonov Sergey > > > > > > > > > > > > On Wed, Jul 21, 2010 at 11:40 AM, Spil Oss <spil.oss@gmail.com> wrote: > > > >> Hi, > > > >> > > > >> Testing FreeBSD 8.1 I noticed that I seem to have routing or nat or > > > >> firewall issues. (csupped RELENG_8_1 which was -RELEASE not -RC last > > > >> night?) > > > >> - 8.1 booted fine > > > >> - connections from the system itself were fine > > > >> - connections from my jails to the internet were not working > > > >> - connections from my LAN/WLAN to the internet were not working > > > >> Reverting back to 8.0-p2 with the same configuration works fine. > > > >> > > > >> In UPDATING I see that rc.firewall and rc.firewall6 were unified. > > > >> > > > >> Setup is > > > >> - xl0 connected to internet/public IP via dhcp > > > >> - bge0/wlan0(ath0) connected to LAN > > > >> - jails have ip's on bge0 in the same subnet as the LAN > > > >> - allow all from any to any via bge0|wlan0|lo0 > > > > The latter point looks problematic, see below. > > > > > >> - NAT using natd > > > >> > > > >> My guess is that something's changed to ipfw that is affecting my > > > >> network settings. Any clues where I went wrong? > > > >> > > > >> Help appreciated/ Kind regards, > > > >> > > > >> Spil. > > > >> > > > >> rc.conf: > > > >> firewall_enable="YES" > > > >> firewall_script="/etc/ipfw.rules" > > > >> > > > >> natd.conf > > > >> interface xl0 > > > >> dynamic yes > > > >> same_ports yes > > > >> # http/https to http jail > > > >> redirect_port tcp 192.168.2.3:80 80 > > > >> redirect_port tcp 192.168.2.3:443 443 > > > >> > > > >> Part of /etc/ipfw.rules > > > >> #!/bin/sh > > > >> cmd="ipfw -q add" > > > >> skip="skipto 500" > > > >> pif=xl0 > > > >> pif6=gif0 > > > >> ext6="2001:dead:beef:1::1" > > > >> ks="keep-state" > > > >> > > > >> ipfw -q -f flush > > > >> > > > >> # Allow internal traffic > > > >> $cmd 002 allow all from any to any via bge0 # exclude LAN traffic > > > >> $cmd 003 allow all from any to any via lo0 # exclude loopback traffic > > > >> $cmd 004 allow all from any to any via wlan0 # exclude WLAN traffic > > > >> $cmd 005 allow all from any to any via bridge0 # exclude WLAN traffic > > > >> $cmd 006 allow all from any to any via tun0 # exclude WLAN traffic > > > > There are problems wih this, based on the apparent misunderstanding (by > > the present IPFW section's author) of how 'via' works when direction (in > > or out) and/or a specific interface (recv or xmit) is not specified. > > > > I'm assuming that apart from lo0, traffic coming in on some or all of > > bge0, wlan0, bridge0 and tun0 is to be translated by NAT if destined to > > be going out on the public interface xl0, right? Certainly you've > > indicated that your jails are on bge0 with 192.168.x.x addresses, but > > these are local aliases which wouldn't show bge0 as the recv interface: > > right! bge0, wlan0 and bridge0 should be translated by NAT. > > I think I almost understand all below. Just can't figure out (yet) why > the jails have no problem accessing the internet. Many thanks for this > detailed explanation! Will use it to my advantage. The jails are on your local machine so packets from them 'originate on the local host' as below, so have no recv interface (despite being IP aliases on bge0) and don't appear on the inbound pass at all - they're not routed, so are unaffected by rule 2 (as opposed to packets 'in recv bge0' from or 'out xmit bge0' to your LAN hosts, which are routed). So packets from your jails are being properly NAT'd and state is being set awaiting replies, as they're sent out. Whenever struggling to figure out what's going on, I tend to just stick 'count' rules around sections to log unexpected - or confirm expected - traffic. Defensive programming is testing for what "shouldn't happen". Have a good look through rc.firewall 'simple' ruleset, especially re the placement of nat(d) and antispoofing rules; perhaps that'd help clarify? If in doubt between the handbook section and ipfw(8), trust the latter. HTH, Ian > > "A packet may not have a receive or transmit interface: packets > > originating from the local host have no receive interface, while > > packets destined for the local host have no transmit interface." > > > > Let's consider just one initial packet coming in from bge0 from some > > private LAN address, other than your local jail IPs, that's destined for > > an outside address and so needs to be NAT'd before transmission. > > > > On the inbound pass, the packet comes in on bge0 and so satisfies 'via > > bge0' so is immediately allowed in, and that's it. That may be ok, as > > NAT is here only to be performed on the outbound pass, but it could be > > anything, and there's no protection here against it being spoofed, as > > opposed to placement of NAT rules in the rc.firewall 'simple' ruleset. > > > > So having been allowed in, after kernel routing (but before NAT, as we > > haven't reached any check-state rule yet) it reenters the firewall for > > the outbound pass with its xmit interface set to xl0, but with its recv > > interface still set to bge0. > > > > Thus the unqualified 'via bge0' is still true, so this packet is passed > > immediately, with source address still set to the private LAN address, > > without having been translated by NAT. As are similarly any packets > > bound for the outside initially coming in on wlan0, bridge0 or tun0 that > > originate from other than the local host. > > > > This could be obviated by replacing 'via $if' by 'in recv $if' in the > > rules above, so such packets going out pass through the firewall rules, > > including setting state for reply packets received on the outside iface. > > > > So as is, the rules below are only applied to outbound packets that are > > NOT received on those interfaces, which doesn't sound like what you'd > > intend, except packets generated by or destined for the local box. > > > > > >> # Allow all encapulated IPv6 to/from tunnel PoP > > > >> $cmd 010 allow ip4 from <tunnel-provider-ipv4> to me via $pif > > > >> $cmd 010 allow ip4 from me to <tunnel-provider-ipv4> via $pif > > > > Taken on faith. > > > > > >> # Black-hole some stuff using tables > > > >> $cmd 050 drop ip from "table(17)" to any in via $pif > > > >> $cmd 050 drop ip from any to "table(17)" out via $pif > > > >> > > > >> # Separate IPv6 rules (no NAT!) > > > >> $cmd 060 skipto 1000 ip6 from any to any > > > >> > > > >> $cmd 100 divert natd ip from any to any in via $pif # Mangle inbound > > > >> packets from external > > > >> $cmd 101 check-state > > > >> > > > >> # Authorized outbound packets > > > >> $cmd 130 $skip icmp from any to any out via $pif $ks > > > >> $cmd 150 $skip tcp from any to any out via $pif $ks > > > >> $cmd 151 $skip udp from any to any out via $pif $ks > > > >> > > > >> $cmd 200 allow udp from 10.50.0.1 to me 68 in $ks > > > >> > > > >> # Deny all inbound traffic from non-routable reserved address spaces > > > >> $cmd 300 unreach host all from 192.168.0.0/16 to any in via $pif > > > >> #RFC 1918 private IP > > > >> $cmd 301 unreach host all from 172.16.0.0/12 to any in via $pif > > > >> #RFC 1918 private IP > > > >> $cmd 302 unreach host all from 10.0.0.0/8 to any in via $pif > > > >> #RFC 1918 private IP > > > >> $cmd 303 unreach host all from 127.0.0.0/8 to any in via $pif #loopback > > > >> $cmd 304 unreach host all from 0.0.0.0/8 to any in via $pif #loopback > > > >> $cmd 305 unreach host all from 169.254.0.0/16 to any in via $pif > > > >> #DHCP auto-config > > > >> $cmd 306 unreach host all from 192.0.2.0/24 to any in via $pif > > > >> #reserved for docs > > > >> $cmd 307 unreach host all from 204.152.64.0/23 to any in via $pif #Sun cluster > > > >> $cmd 308 unreach host all from 224.0.0.0/3 to any in via $pif > > > >> #Class D & E multicast > > > >> > > > >> # Deny packets that did not match the dynamic rule table > > > >> #$cmd 330 deny all from any to any frag in via $pif # All late fragments > > > > The author (here and in other writings) expounds that fragmented packets > > are necessarily bad, and indicate an attack of some sort. Not in fact > > so; eg if you were using say zen.spamhaus.org as an RBL you'll be seeing > > valid fragmented UDP port 53 packets of around 2000 bytes total all day. > > I gather DNSSEC will also require acceptance of fragmented datagrams. > > The default rc.firewall rules pass IP frags, and I suggest you do too. > > > > > >> #$cmd 332 deny tcp from any to any established in via $pif # Deny ACK > > > >> > > > >> # Authorized inbound packets > > > >> $cmd 400 allow icmp from any to any icmptypes 0,11 # echo reply and TTL-exceeded > > > > Add icmptype 3 here to pass unreachables including path MTU discovery. > > > > > >> $cmd 420 allow tcp from any to me ssh in via $pif setup $ks > > > >> $cmd 421 allow tcp from any to me smtp in via $pif > > > >> $cmd 422 allow tcp from any to me http in via $pif > > > >> $cmd 423 allow tcp from any to me https in via $pif > > > >> $cmd 424 allow tcp from any to me imaps in via $pif > > > >> > > > >> #$cmd 449 unreach host ip from any to any in via $pif > > > >> $cmd 448 reject log all from any to any in via $pif > > > >> $cmd 449 reject log all from any to any out via $pif > > > >> $cmd 450 reject log ip from any to any > > > > Security-wise you'd be better off just dropping these using deny than > > reject (deprecated, equivalent to unreach host), here and above. > > > > However I don't wish to let such details obscure the more major issue. > > > > > >> > > > >> # This is skipto location for outbound stateful rules > > > >> $cmd 500 divert natd ip from any to any out via $pif > > > >> $cmd 510 allow ip from any to any > > > > cheers, Ian --0-23540912-1279992225=:62748--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20100725020910.B62748>