From owner-freebsd-pf@FreeBSD.ORG Wed Nov 16 23:35:44 2005 Return-Path: X-Original-To: freebsd-pf@freebsd.org Delivered-To: freebsd-pf@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 63B7016A41F for ; Wed, 16 Nov 2005 23:35:44 +0000 (GMT) (envelope-from dhartmei@insomnia.benzedrine.cx) Received: from insomnia.benzedrine.cx (insomnia.benzedrine.cx [62.65.145.30]) by mx1.FreeBSD.org (Postfix) with ESMTP id B3D3F43D45 for ; Wed, 16 Nov 2005 23:35:43 +0000 (GMT) (envelope-from dhartmei@insomnia.benzedrine.cx) Received: from insomnia.benzedrine.cx (dhartmei@localhost [127.0.0.1]) by insomnia.benzedrine.cx (8.13.4/8.12.11) with ESMTP id jAGNZca8007805 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO); Thu, 17 Nov 2005 00:35:38 +0100 (MET) Received: (from dhartmei@localhost) by insomnia.benzedrine.cx (8.13.4/8.12.10/Submit) id jAGNZc30029540; Thu, 17 Nov 2005 00:35:38 +0100 (MET) Date: Thu, 17 Nov 2005 00:35:37 +0100 From: Daniel Hartmeier To: Matthew Grooms Message-ID: <20051116233537.GT29615@insomnia.benzedrine.cx> References: <437BB031.9090504@seton.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <437BB031.9090504@seton.org> User-Agent: Mutt/1.5.10i Cc: freebsd-pf@freebsd.org Subject: Re: Traffic Shaping with pf ... X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Nov 2005 23:35:44 -0000 On Wed, Nov 16, 2005 at 04:18:25PM -0600, Matthew Grooms wrote: > I have a couple of firewalls running freebsd 5.4 and pf and was > planning to use ALTQ for traffic shaping. But after doing a bit of > reading, it would seem that ALTQ only works on traffic passing outbound > on an interface. Since most of the traffic passing through my firewall > is http and ftp traffic, the inbound direction is the path being > saturated. Did I read the ALTQ documentation wrong or is there another > mechanism available for use with pf that could help me prioritize > bandwidth usage? This question pops up frequently, if this reply is too wordy, that's just so I can reference it in the future and safe typing. My apologies to the poster if this is all obvious already. ;) Rate-limiting network packets means dropping packets. It's not like a water utility pipe where you can shut the faucet incrementally and slow down the water running towards you from the water company, leaving unused water in their tanks. There are no reservoirs like that in a network (ignoring some very small buffers). If a sender is sending you packets at a rate higher than you can receive them, packets are dropped whereever there are gaps of decreasing bandwidth. And these gaps are on routers at your ISP and further upstream. Many of them will drop random packets. Some can be configured to drop based on criteria, but you don't control those criteria, because they're not your routers. Imagine you have a 1024 kbit/s downlink from your ISP to you. Assume your ISP himself has a much larger downlink himself. You're downloading a file from a web server on the Internet. Then some evil person starts sending you a flood of pings. Let's say that person has an uplink of 2048 kbit/s. Now your ISP is receiving two kinds of packets destined for you: a stream of TCP packets from the web server, and a stream of pings at 2048 kbits/. He can't possibly forward all these packets to you, since only less than half of them fit onto the link to you. So he simply forwards as many packets as he can, randomly dropping the rest. Obviously many of the TCP packets will get dropped randomly now. TCP is clever and adjust to this, the sender recognizes that there is loss between him and you, and will start to send at a lower rate. Meanwhile the flooder continues to send you pings at a happy rate of 2048 kbit/s. You'll notice how your download gets slower and slower, and you consider using rate-limiting incoming packets. You identify that the http packets are those that you prefer over the pings, and tell ALTQ to drop incoming pings exceeding, say, a harmless rate of 64 kbit/s and reserve the rest for the more important http packets. Fine, it could do that. But it wouldn't change anything, because the congestion is upstream of your ALTQ box. You can drop as many packets as you like after you received them, that doesn't free up any bandwidth on your downlink. The downlink will continue to carry mostly ping packets, because you dropping packets has no influence on what happens at either sender, at your ISP's router dropping random packets, or on your downlink. Just like the rate of water you can draw from your water line isn't influenced by what you do with the water that has already come out of it. Rate-limiting is not like a faucet. This is the reason you often get the answer "It just doesn't make sense to rate-limit incoming packets", and I guess that's the reason why ALTQ simply doesn't add queues for incoming packets, but only outgoing ones. Now, if we forget about the DoS case, and assume you have only flow-controlled TCP connections with cooperative peers, things are a little different. If you receive two streams of TCP packets, and you start dropping packets of one stream (after you have received them and they have taken up bandwidth on your downlink), the corresponding peer will detect that loss and helpfully slow down sending, freeing up bandwidth on your downlink for the other peer. In fact, if you tell your ALTQ box to limit one stream to, say, 128 kbit/s, and drop all excess packets, that peer will (eventually, by trial and error) come to the conclusion that sending you packets at a rate of about 128 kbit/s is the optimal thing to do. But it's important to realize that you're not really "enforcing" this limit, but it's the peer that kindly reacts in that way. If you want to do this with ALTQ, you can do so by limiting outgoing packets on the "other" interface, assuming the box is forwarding all packets between two interfaces. If a browser (on a separate local box) is downloading a file from an external web server _through_ the ALTQ box, you rate-limit packets going out through the internal interface. Every packet coming in on the external interface obviously goes out through the internal interface, hence rate-limiting outgoing packets on the internal interface has the same effect as rate-limiting incoming packets on the external interface. This does not work if the client is on the ALTQ box itself, obviously (there is no "other" interface to rate-limit on). In this case you're facing a limitation of ALTQ itself. You might have to move ALTQ onto an additional intermediate box, just so you do have a second interface. I don't think there are any plans to introduce incoming queues in ALTQ. Daniel