From owner-freebsd-current@FreeBSD.ORG Tue Dec 1 17:34:12 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B15C110656CC; Tue, 1 Dec 2009 17:34:12 +0000 (UTC) (envelope-from oleg@lath.rinet.ru) Received: from lath.rinet.ru (lath.rinet.ru [195.54.192.90]) by mx1.freebsd.org (Postfix) with ESMTP id 301578FC15; Tue, 1 Dec 2009 17:34:11 +0000 (UTC) Received: by lath.rinet.ru (Postfix, from userid 222) id 25BDF704B; Tue, 1 Dec 2009 20:34:11 +0300 (MSK) Date: Tue, 1 Dec 2009 20:34:11 +0300 From: Oleg Bulyzhin To: Ben Kelly Message-ID: <20091201173411.GA3637@lath.rinet.ru> References: <4B129960.9030107@gmail.com> <4B12B6B9.3030106@bsd.com.br> <4B12C32F.3060709@gmail.com> <4B12C543.2070204@mgwigglesworth.net> <4B141A77.4030102@gmail.com> <20091130201222.GC72710@lath.rinet.ru> <20091130234537.GA78185@lath.rinet.ru> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="wac7ysb48OaltWcw" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.18 (2008-05-17) Cc: freebsd-ipfw@freebsd.org, Kevin Smith , freebsd-current@freebsd.org Subject: Re: dummynet issues X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Dec 2009 17:34:12 -0000 --wac7ysb48OaltWcw Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Mon, Nov 30, 2009 at 11:58:55PM -0500, Ben Kelly wrote: > > I actually have not measured my bandwidth to validate dummynet. I have simply observed these messages repeating in my log: > > dummynet: OUCH! pipe should have been idle! > > Under normal conditions I don't really need the dummynet rules to shape traffic for my configuration to work, so it has not been a high priority for me yet. Do you see the log messages? > > Thanks. > > - Ben It seems i've found the problem. Please test attached patch (it's for R8.0 sources and include r198845). I'm interested in some feedback: 1) does it solve 'OUCH' messages problem? 2) does it solve bandwidth problem (if there was any)? -- Oleg. ================================================================ === Oleg Bulyzhin -- OBUL-RIPN -- OBUL-RIPE -- oleg@rinet.ru === ================================================================ --wac7ysb48OaltWcw Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="wf2q-fix.r80.diff" Index: sys/netinet/ipfw/ip_dummynet.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/ipfw/ip_dummynet.c,v retrieving revision 1.5.2.1.2.1 diff -u -r1.5.2.1.2.1 ip_dummynet.c --- sys/netinet/ipfw/ip_dummynet.c 25 Oct 2009 01:10:29 -0000 1.5.2.1.2.1 +++ sys/netinet/ipfw/ip_dummynet.c 1 Dec 2009 17:23:45 -0000 @@ -244,6 +244,17 @@ static int dummynet_io(struct mbuf **, int , struct ip_fw_args *); /* + * Flow queue is idle if: + * 1) it's empty for at least 1 tick + * 2) it has invalid timestamp (WF2Q case) + * 3) parent pipe has no 'exhausted' burst. + */ +#define QUEUE_IS_IDLE(q) ((q)->head == NULL && (q)->S == (q)->F + 1 && \ + curr_time > (q)->idle_time + 1 && \ + ((q)->numbytes + (curr_time - (q)->idle_time - 1) * \ + (q)->fs->pipe->bandwidth >= (q)->fs->pipe->burst)) + +/* * Heap management functions. * * In the heap, first node is element 0. Children of i are 2i+1 and 2i+2. @@ -1004,7 +1015,7 @@ fs->last_expired = time_uptime ; for (i = 0 ; i <= fs->rq_size ; i++) /* last one is overflow */ for (prev=NULL, q = fs->rq[i] ; q != NULL ; ) - if (q->head != NULL || q->S != q->F+1) { + if (!QUEUE_IS_IDLE(q)) { prev = q ; q = q->next ; } else { /* entry is idle, expire it */ @@ -1134,7 +1145,7 @@ break ; /* found */ /* No match. Check if we can expire the entry */ - if (pipe_expire && q->head == NULL && q->S == q->F+1 ) { + if (pipe_expire && QUEUE_IS_IDLE(q)) { /* entry is idle and not in any heap, expire it */ struct dn_flow_queue *old_q = q ; @@ -1408,18 +1419,20 @@ if (q->idle_time < curr_time) { /* Calculate available burst size. */ q->numbytes += - (curr_time - q->idle_time) * pipe->bandwidth; + (curr_time - q->idle_time - 1) * pipe->bandwidth; if (q->numbytes > pipe->burst) q->numbytes = pipe->burst; if (io_fast) q->numbytes += pipe->bandwidth; } } else { /* WF2Q. */ - if (pipe->idle_time < curr_time) { + if (pipe->idle_time < curr_time && + pipe->scheduler_heap.elements == 0 && + pipe->not_eligible_heap.elements == 0) { /* Calculate available burst size. */ pipe->numbytes += - (curr_time - pipe->idle_time) * pipe->bandwidth; - if (pipe->numbytes > pipe->burst) + (curr_time - pipe->idle_time - 1) * pipe->bandwidth; + if (pipe->numbytes > 0 && pipe->numbytes > pipe->burst) pipe->numbytes = pipe->burst; if (io_fast) pipe->numbytes += pipe->bandwidth; --wac7ysb48OaltWcw--