Date: Sun, 11 Aug 2002 12:36:44 +0200 (SAST) From: Willie Viljoen <will@highveldcs.com> To: freebsd-stable@freebsd.org Subject: Dummynet and Fastrack/Kazaa (FALSE ALARM) Message-ID: <20020811114945.W6225-100000@phoenix.vh.laserfence.net>
next in thread | raw e-mail | index | archive | help
Sorry about the double post for some people, I sorted out this problem yesterday, and in the follow up post, I accidentally replied only to the people who replied to me. The problem wasn't the P2P clients not taking well to having their bandwidth limited, but the fact that the site having the problem were also running natd. Their firewall was in one pass mode (net.inet.ip.fw.one_pass=1), which caused packets that go through dummynet, not to go through natd, and thus not making it back to the private IPs of the hosts from which the P2P connections were coming. For the "record of future generations", here's how to make dummynet and natd work together without trodding on each other's toes, and also how to set up a reasonably secure stateful firewall for such a site. The customer having a problem is a clusterhome development. They run two 128kbps ISDN connections, to provide a combined 256kbps link. It's managed by a strange proprietary piece of software provided to them by their previous ISP, and adopted by us (because we didn't want to mess about with reconfiguring everything) It basically established a multilink ISDN connection, and then creates a file which can be read from and written to asif it were a serial device. This runs on both sides, and then a pppd on both sides is connected to the "device" to provide kernel PPP support. To alleviate the problem, I decided to limit bandwidth for FTP and P2P connections (mostly the Kazaa-like P2Ps, which are causing the problem) to 200kbps, allowing a small extra gap for e-mail and other traffic, and not to limit HTTP, because the client refused to have that. They use the private network 192.168.0.0/24, with two ethernet interfaces, ed0 and ed1, ed0 being the only one having an IP address, and the interfaces acting as a bridge for the rather large ethernet, which is devided rather strangely into two sections, one for the east side of the complex, and the other for the west. The two sections also have a number of badly set up switches and hubs connecting their hosts, but I won't get into that right now. Below are firewall rules to set up a secure stateful firewall, alloing only outgoing connections from the complex. Their e-mail is handled by a server on our premisis, so there's no need to allow connections in to their network. These rules also implement dummynet and natd as I mentioned above. -- /etc/ipfw.conf: -f flush pipe 00100 config bw 200Kbit/s queue 40KB add 00100 divert natd ip from any to any via ppp0 add 00200 pipe 00100 tcp from any 20,21,1214 to any recv ppp0 add 00300 check-state add 00400 allow ip from any to any out keep-state add 00500 allow ip from any to any via lo0 keep-state add 00600 allow ip from 192.168.0.0/24 to any via ed* keep-state -- The above rules are the minimum for getting this running. Note that the natd rule is placed before the dummynet rule. This order is not significant, but for reasons unknown (or not understood properly by me), this configuration work better when the rules are in the order mentioned above. A briefe explanation: Rules 00100 and 00200 (obviously) direct the packets to natd to allow address translation, and then to dummynet (where applicable) to perform bandwidth limiting. Rule 00300 is part of the fairly new set of advanced stateful rules available in ipfw. It checks if a packet has previously been associated with the dynamic ruleset table, and if so, allows the packet to be forwarded on to it's destination. Rule 00400 may be considered by some people to be insecure, but it serves a perpose here. It matches packets generated by the local host that are outbound, and adds dynamic rules to the state table to remember these outgoing connections. Rule 00500 is abit out of place, but it basically allows traffic on the local interface to pass through the firewall at the "check-state" point on the next match, mostly not needed. Rule 00600 matches outbound traffic from the private networks. It adds dynamic rules for these connections to the state table to allow packets back in at the "check-state" point, right after they pass through natd and dummynet pipes. Note that interfaces are specified. This is to protect from people outside the firewall somehow setting their addresses to these private ranges, and attempting to route through our system in order to "spoof" their IP address as ours. In my previous post (which only some people got) I actually had an unoptimized rule set which had two separate rules for ed0 and ed1, but this can be shortened to ed* to allow packets on this range from both these interfaces. Usually, only ed0 should be needed, but even though the bridged ethernet frames are not being passed through ipfw directly, the kernel still records packets that came in over ed1 as having been received over ed1, even though they were intended for the IP address on ed0. Had the system been using different kinds of interfaces, a rule would be required for each, for instance, ed0 and ep0. This is all that's needed for the basic functionality, however, using just the above (minimal) rules, will break some things, below are the rest of the rules recommended for a more complete implementation: -- /etc/ipfw.conf add 00700 allow icmp from any to any icmptypes 0,3,4,8,11,12,13,14 add 00800 deny ip from any to any frag in recv ppp0 add 00900 deny tcp from any to any established in recv ppp0 add 01000 reset tcp from any to any 113 in setup add 01100 unreach port udp from any to any 33400-33556 in recv ppp0 -- Another briefe explanation: Rule 00700 allows certain ICMP packets which are important to some software and configurations. Most people may disable source quench (4), which is typically not allowed, and if you want to disable ping packets enterering your network, you may also remove type 8. Note that this will not affect pings leaving your network from the private IP ranges, as they are allowed through by rule 00600, and allowed back in by rule 00700 if ICMP type 0 is allowed. ICMP type 11 allows traceroute to function from within your network to outside hosts. The other ICMP types are important for MTU discovery and control messages, and should not be blocked. Also, you may choose to allow ICMP type 5 (redirects), but this can be dangerous, and should be used with care. Rule 00800 stops fragmented IP packets from entering your network from the outside internet. Because the relevant ICMP packets are passing through your firewall, MTU discovery should work properly, and fragments should not exist at this point. Rule 00900 stops packets "claiming" to be from an established TCP connection from entering your network. Packets from legitimate connections will have been allowed through at the "check-state" point, so these packets will typically be stray ACKs from lost connections, or nmap et al "ACK scans" (designed to reveal simple state based firewalls) Rule 01000 sends a TCP RESET in response to connection attempts to port 113. This deflects "ident" or "auth" requests, mostly sent by FTP and IRC servers, but which is unneeded and in many cases not secure. Deflecting this is handy because some (broken) servers will wait forever for an response from a firewalled port, or disconnect your users for having a "time-out" on the auth request (which will not otherwise make it through the firewall). When a reset is sent, the remote machines will assume your host is not running identd (the default these days) and allow your users to continue without this. Note that some IRC servers on older IRC networks will not allow users to connect without identd running, you will have to address this problem differently if it should arise. Rule 01100 is a courtesy to people attempting to use traceroute to your network's public IP. Normally, because ipfw defaults to blocking all packets not explicitly allowed through, a traceroute to your host would "star out", and continue up to the maximum hop count specified by the originator. With this rule in place, ICMP port unreachable packets are returned in response to UDP traceroute packets. This will cause traceroute not to "star out", but stop the trace at your firewall's public IP address, thinking it reached the destination host. This is also handy for networks routing to public IPs behind their firewall, as it will not cause traceroute to "star out", but also not allow the trace to continue beyond your firewall. Keep in mind, that for this configuration to be secure, ipfirewall(4) will have to deny all packets by default (the default). If your kernel has been compiled to accept by default, you will either need to recompile it to deny by default, or add a rule like this, to act as a "pseudo-default" rule: -- /etc/ipfw.conf: add 65534 deny ip from any to any -- That's it for the firewall configuration, however, you will need still more configuraton for this implementation to work entirely as expected. Most notibly, your firewall must not be in one pass mode (which is the default). To disable this at run time, use: sysctl net.inet.ip.fw.one_pass=0 Here (for reference) are the complete options in rc.conf(5) and sysctl.conf(5) to get a working implementation as explained above: -- /etc/rc.conf: firewall_enable="YES" firewall_type="/etc/ipfw.conf" gateway_enable="YES" ifconfig_ed0="inet 192.168.0.254 netmask 255.255.255.0" # Assuming your public IP address is 196.25.25.25 (fictional): natd_enable="YES" natd_interface="196.25.25.25" natd_flags="-s -u -punch_fw 10000:10000" -- -- /etc/sysctl.conf: #Only for bridging net.link.ether.bridge=1 net.link.ether.bridge_cfg="ed0,ed1" #Disable one pass mode net.inet.ip.fw.one_pass=0 -- Also, for all of the above to work, your kernel will have to have been configured with all the relevant options, the use of and meaning of which I won't explain here, as it is covered in various places, basically, you will need the following: options IPFIREWALL options IPDIVERT options DUMMYNET options BRIDGE #Only for bridging That's pretty much it. Sorry for the long post, but I'm hoping this will make it into the archives soon, sothat other people who have this problem in future can find it quickly, that also being the reason for it being a little bit too complete, apology to anybody who just read pages and pages of things they aready know :) Will -- Willie Viljoen Highveld Computing Solutions 214 Paul Kruger Avenue Universitas Bloemfontein 9321 South Africa +27 51 522 15 60, a/h +27 51 522 44 36 +27 82 404 03 27 will@highveldcs.com To find out how we can help you with inventive solutions, visit http://www.highveldcs.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020811114945.W6225-100000>