From owner-freebsd-net@FreeBSD.ORG Mon Feb 19 12:40:27 2007 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 21C8916A402 for ; Mon, 19 Feb 2007 12:40:27 +0000 (UTC) (envelope-from admin@azuni.net) Received: from mail.azuni.net (ns0.azuni.net [217.25.25.3]) by mx1.freebsd.org (Postfix) with ESMTP id 219A813C4BB for ; Mon, 19 Feb 2007 12:40:25 +0000 (UTC) (envelope-from admin@azuni.net) Received: (qmail 28259 invoked by uid 1004); 19 Feb 2007 12:40:24 -0000 Received: from admin@azuni.net by mail.azuni.net by uid 89 with qmail-scanner-1.20 (clamscan: 0.65. spamassassin: 2.63. Clear:RC:1(217.25.23.11):. Processed in 0.309482 secs); 19 Feb 2007 12:40:24 -0000 Received: from unknown (HELO ?217.25.23.11?) (217.25.23.11) by ns0.azuni.net with AES256-SHA encrypted SMTP; 19 Feb 2007 12:40:23 -0000 Message-ID: <45D99AA2.7070406@azuni.net> Date: Mon, 19 Feb 2007 16:40:02 +0400 From: admin Organization: UniNet User-Agent: Debian Thunderbird 1.0.2 (X11/20070113) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Ian Smith References: In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, freebsd-questions@freebsd.org, Andre Santos Subject: Re: ipfw limit src-addr woes X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Feb 2007 12:40:27 -0000 Ian Smith wrote: > On Mon, 19 Feb 2007, admin wrote: > > Andre Santos wrote: > > > On 2/18/07, admin wrote: > > > > > >> Hi, I'm trying to use ipfw's limit clause to limit the number of > > >> connections a single IP can have at the same time in a transparent > > >> web-proxy environment: > > >> > > >> 00350 skipto 401 tcp from x.x.x.x/x,y.y.y.y/y,z.z.z.z/z to any dst-port > > >> 80 in via if0 setup limit src-addr 10 > > >> 00401 fwd local.ip.ad.dr,8080 tcp from x.x.x.x/x to any dst-port 80 > > >> ... the rest fwd... > > >> > > >> as I understand the manpage, when the current number of connectiions is > > >> below 10, the action "skipto" is performed, else, the packet is dropped > > >> and the search terminates. But... > > No, a packet is not dropped on a condition that fails a skipto test. > The manpage doesn't make this point clear. limit {src-addr | src-port | dst-addr | dst-port} N The firewall will only allow N connections with the same set of parameters as specified in the rule. To limit the number of connections a user can open you can use the following type of rules: ipfw add allow tcp from my-net/24 to any setup limit src-addr 10 ipfw add allow tcp from any to me setup limit src-addr 4 I'm assuming the packet gets silently dropped when the limit is overloaded but gets acted upon otherwise due to the stateful "limit" behaviour (keep-state in disguise). Just do a "skipto" when there's a state entry and that's it. And that's why the counter grows for established connections too, even though there's a "setup" modifier. skipto is a nice thing as it allows you to AND rules ;-) Besides, that's what my humble testing came up with - connections over the limit DO get dropped... if done nicely. As for the problem, it seems to me that all this noise is because of different timeouts in ipfw and TCP layer/whatever. The dynamic state entry for a connection expires while netstat -na still show the connection as ESTABLISHED, or, worse, the state entry is still there but the corresponding connection is in some half-closed state (FIN_WAIT_2, CLOSE_WAIT, LAST_ACK). The first case allows many more connections than "limit", while the second case won't let many good clients connect due to their buggy browsers not closing connections and letting the count build up. Could this be it? > skipto number > Skip all subsequent rules numbered less than number. The search > continues with the first rule numbered number or higher. > > You'll need a specific allow or deny rule; skipto does neither, it just > branches to 401 if the condition is matched, otherwise proceeds to the > next rule, which is also 401. This runs rule 401 and on, either way. > > > >> the problem is that the src-addr limit is not enforced as some clients > > >> somehow open a huge number (3-5 times the prescribed value) of > > >> www-connections to some single address Out There, forcing you to bump up > > >> certain sysctl variables (such as kern.ipc.nmbclusters, > > >> kern.ipc.maxsockets, etc.) to mitigate the DOS effects. What might be > > >> going on? Is ipfw broken, or am I misusing it? > > You've misread skipto, is all. As it stands, the counts will show how > many packets passed the test, but all packets proceed to the next rule. > > I'd rephrase rules to use skipto only for branching on condition, or > !condition, past specific allow and/or deny rules to deal with this. > > > >> OS: FreeBSD 6.2 > > > > > > > > > The following command worked here (6.2-RC1). Only one connection was > > > allowed to 1.2.3.4. > > > # ipfw add 1 allow tcp from any to 1.2.3.4 22 out via rl1 limit dst-addr 1 > > > > > > Use the command "ipfw -d show" to see what connections are matching > > > your dynamic rules. > > > > > > > # ipfw -d show | fgrep x.x.x.x | wc -l > > 20 > > $ netstat -na|fgrep x.x.x.x|fgrep ESTABLISHED|wc -l > > 113 > > > > Why is it that only 20 connections have been accounted for by ipfw's > > dynamic rules but there are actually 113 active connections from that IP > > at the moment? The limit src-addr is 75. > > See above. Sorry I didn't notice this when you first posted it. I've > not yet used limit src-addr myself, but use skipto a lot :) > > Cheers, Ian > >