From owner-freebsd-ipfw Mon Feb 10 4: 0:43 2003 Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 076E337B401; Mon, 10 Feb 2003 04:00:38 -0800 (PST) Received: from mx1.lublin.pl (mx1.lublin.pl [212.182.63.90]) by mx1.FreeBSD.org (Postfix) with ESMTP id BFC1243F93; Mon, 10 Feb 2003 04:00:36 -0800 (PST) (envelope-from pawmal@unia.3lo.lublin.pl) Received: from proxy.zin.lublin.pl ([212.182.126.66]:26748 "EHLO towah-xp") by mx1.lublin.pl with ESMTP id ; Mon, 10 Feb 2003 13:00:17 +0100 From: "Pawel Malachowski" Organization: unidentified flying modems To: Maxim Konovalov Date: Mon, 10 Feb 2003 13:00:33 +0100 MIME-Version: 1.0 Content-type: text/plain; charset=ISO-8859-2 Content-transfer-encoding: 8BIT Comments: Sender has elected to use 8-bit data in this message. If problems arise, refer to postmaster at sender's site. Subject: Re: kern/48009: dummynet(4) related machine hangs Cc: bug-followup@freebsd.org, freebsd-ipfw@freebsd.org References: <20030206152157.52736.qmail@unia.3lo.lublin.pl> In-reply-to: <20030206183303.X25010@news1.macomnet.ru> X-mailer: Pegasus Mail for Win32 (v3.12b) Message-Id: <20030210120026Z3132496-961+1@mx1.lublin.pl> Sender: owner-freebsd-ipfw@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hello, I've checked if really q->numbytes is getting negative as Mike Hibler described in kern/37573 and I realized this is true. My machine goes into infinite loop in splimp()/splx() sections and that's why it looks as if it was frozen. I've decided the check if the problem exists on other 2 machines (4.7-RELEASE and 4.7-STABLE) and I must confirm, all of them hang is the same way when I frequently modify bandwith parameter. Once again, * Install fresh FreeBSD 4.7-RELEASE * Recompile GENERIC kernel with IPFW2, rebuild ipfw and libalias as described in ipfw(8), reinstall and reboot * Do: ipfw pipe 10 config bw 0 ipfw pipe 20 config bw 0 ipfw add 10 pipe 10 ip from any to any in recv NIC ipfw add 20 pipe 20 ip from any to any out xmit NIC (my NIC was: rl0, fxp0 -- a NIC one is using to connect with LAN) Then connect for example to the fast FTP-server in LAN and GET some big file (hundreds of MB). While this file is downloading, run the following script: #!/bin/sh while [ 1 ] i=1 do echo Test number $i \(`date`\). echo Step 1. ipfw pipe 10 config bw 512kbit/s echo Step 2. ipfw pipe 20 config bw 512kbit/s sleep 1 echo Step 3. ipfw pipe 10 config bw 2Mbit/s echo Step 4. ipfw pipe 20 config bw 2Mbit/s sleep 1 i=`expr $i + 1` done * And look q->numbytes after a while will grow fast up to 2^31-1 and revert into negative causing system hang. What is going on? In ip_dummynet.c, in ready_event() the following line (551) reverts q->numbytes into negative: q->numbytes += ( curr_time - q->sched_time ) * p->bandwidth; causing dummynet() to go into infinite loop with ready_event(). Note, in line 557 we are decreasing q->numbytes preventing it from growing to much: q->numbytes -= len_scaled ; (where len_scaled = pkt->dn_m->m_pkthdr.len * 8 * hz) When we frequently modify bandwith using ipfw pipe config bw xxx, q->numbytes starts growing up to maximum signed integer size. This was easy to observe cause I've added diagnostic printf to config_pipe() showing current q->numbytes value every time config_pipe() was called (every time ipfw pipe config bw xxx was used). It looks, when we are downloading something big (means, we have high traffic on pipe 10 (recv), the q->numbytes associated with pipe 20 (xmit!) grows fast. This patch (work-around) comes from kern/37573 and was changed a bit to cleanly apply on RELENG_4_7 ip_dummynet.c. It works for me preventing machine from hanging. ==================== *** ip_dummynet.c.origThu Jan 23 22:06:45 2003 --- ip_dummynet.cSun Feb 9 23:51:49 2003 *************** *** 549,554 **** --- 549,559 ---- * setting len_scaled = 0 does the job. */ q->numbytes += ( curr_time - q->sched_time ) * p->bandwidth; + if (q->numbytes<0) { + /* This shouldn't happen, I clear q->numbytes in config_pipe() */ + printf("Oops, ready_event has a problem with q->numbytes<0.\n"); + q->numbytes=0 ; + } while ( (pkt = q->head) != NULL ) { int len = pkt->dn_m->m_pkthdr.len; int len_scaled = p->bandwidth ? len*8*hz : 0 ; *************** *** 1515,1521 **** static int config_pipe(struct dn_pipe *p) { ! int s ; struct dn_flow_set *pfs = &(p->fs); /* --- 1520,1526 ---- static int config_pipe(struct dn_pipe *p) { ! int s = 0; struct dn_flow_set *pfs = &(p->fs); /* *************** *** 1549,1561 **** x->idle_heap.size = x->idle_heap.elements = 0 ; x->idle_heap.offset=OFFSET_OF(struct dn_flow_queue, heap_pos); } else x = b; ! x->bandwidth = p->bandwidth ; x->numbytes = 0; /* just in case... */ bcopy(p->if_name, x->if_name, sizeof(p->if_name) ); x->ifp = NULL ; /* reset interface ptr */ ! x->delay = p->delay ; set_fs_parms(&(x->fs), pfs); --- 1554,1579 ---- x->idle_heap.size = x->idle_heap.elements = 0 ; x->idle_heap.offset=OFFSET_OF(struct dn_flow_queue, heap_pos); } else + { struct dn_flow_queue *q; + int i; + x = b; + s = splimp(); /* protect mods to active pipe/flow set */ + + /* Obtained from kern/37573 Audit-Trail */ + /* flush accumulated credit for all queues */ + for (i = 0 ; i <= x->fs.rq_size ; i++ ) + for (q = x->fs.rq[i] ; q ; q = q->next ) { + q->numbytes = 0; + } + } + ! x->bandwidth = p->bandwidth ; x->numbytes = 0; /* just in case... */ bcopy(p->if_name, x->if_name, sizeof(p->if_name) ); x->ifp = NULL ; /* reset interface ptr */ ! x->delay = p->delay ; set_fs_parms(&(x->fs), pfs); *************** *** 1571,1578 **** all_pipes = x ; else a->next = x ; - splx(s); } } else { /* config queue */ struct dn_flow_set *x, *a, *b ; --- 1589,1596 ---- all_pipes = x ; else a->next = x ; } + splx(s); } else { /* config queue */ struct dn_flow_set *x, *a, *b ; *************** *** 1600,1605 **** --- 1618,1624 ---- if (pfs->parent_nr != 0 && b->parent_nr != pfs->parent_nr) return EINVAL ; x = b; + s = splimp(); /* protect mods to active pipe/flow set */ } set_fs_parms(x, pfs); *************** *** 1615,1622 **** all_flow_sets = x; else a->next = x; - splx(s); } } return 0 ; } --- 1634,1641 ---- all_flow_sets = x; else a->next = x; } + splx(0); } return 0 ; } ==================== -- Paweł Małachowski To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-ipfw" in the body of the message