From owner-freebsd-current@FreeBSD.ORG Thu Nov 3 21:59:44 2005 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1424416A41F; Thu, 3 Nov 2005 21:59:44 +0000 (GMT) (envelope-from snezhko@indorsoft.ru) Received: from indor.net.tomline.ru (indor.net.tomline.ru [213.183.100.90]) by mx1.FreeBSD.org (Postfix) with ESMTP id ECADB43D46; Thu, 3 Nov 2005 21:59:42 +0000 (GMT) (envelope-from snezhko@indorsoft.ru) Received: from SNEZHKO by indorsoft.ru (MDaemon.PRO.v7.2.2.R) with ESMTP id md50000027601.msg; Fri, 04 Nov 2005 03:59:35 +0600 X-AntiVirus: Checked by Dr.Web [version: 4.32b, engine: 4.32b, virus records: 88046, updated: 3.11.2005] To: freebsd-current@freebsd.org References: <20051027022313.R675@kushnir1.kiev.ua> <200511030059.05946.max@love2party.net> <200511031500.00839.jhb@freebsd.org> From: Victor Snezhko Date: Fri, 04 Nov 2005 03:59:32 +0600 In-Reply-To: <200511031500.00839.jhb@freebsd.org> (John Baldwin's message of "Thu, 3 Nov 2005 14:59:59 -0500") Message-ID: User-Agent: Gnus/5.110002 (No Gnus v0.2) Emacs/21.3 (windows-nt) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Spam-Processed: indor.net.tomline.ru, Fri, 04 Nov 2005 03:59:35 +0600 (not processed: spam filter disabled) X-Return-Path: snezhko@indorsoft.ru X-VVS-Spam: false Cc: Max Laier Subject: Re: CURRENT + amd64 + user-ppp = panic X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Nov 2005 21:59:44 -0000 John Baldwin writes: >> (kgdb) up 11 >> #11 0xc066e0c2 in softclock (dummy=0x0) at >> /usr/src/sys/kern/kern_timeout.c:220 220 if (c->c_time != curticks) { >> (kgdb) list >> 215 curticks = softticks; >> 216 bucket = &callwheel[curticks & callwheelmask]; >> 217 c = TAILQ_FIRST(bucket); >> 218 while (c) { >> 219 depth++; >> 220 if (c->c_time != curticks) { >> 221 c = TAILQ_NEXT(c, c_links.tqe); >> 222 ++steps; >> 223 if (steps >= MAX_SOFTCLOCK_STEPS) { >> 224 nextsoftcheck = c; >> (kgdb) print *bucket >> $1 = {tqh_first = 0xc1891d80, tqh_last = 0xc1891d80} >> (kgdb) print c >> $2 = (struct callout *) 0xdeadc0de >> (kgdb) print *(bucket->tqh_first) >> $3 = {c_links = {sle = {sle_next = 0xdeadc0de}, tqe = {tqe_next = >> 0xdeadc0de, tqe_prev = 0xdeadc0de}}, c_time = -559038242, c_arg = >> 0xdeadc0de, c_func = 0xdeadc0de, c_mtx = 0xdeadc0de, c_flags = -559038242} >> (kgdb) print steps >> $4 = 1 > > Well, from thus it seems that a callout was free'd while it was still on the > list. Perhaps there is a case wehre callout_stop() isn't called. Also, > callout_drain() should really be used. If the callout function is rearming, > then it might have been running when callout_stop() is called, and it could > have rearmed itself and then been overwritten when it was freed. In fact, > that is likely your problem. You can try this patch, but there might be lock > order problems that would require the callout_drain() to happen later when > locks aren't held: > > Index: nd6.c > =================================================================== > RCS file: /usr/cvs/src/sys/netinet6/nd6.c,v > retrieving revision 1.62 > diff -u -r1.62 nd6.c > --- nd6.c 22 Oct 2005 05:07:16 -0000 1.62 > +++ nd6.c 3 Nov 2005 19:56:42 -0000 > @@ -398,7 +398,7 @@ > if (tick < 0) { > ln->ln_expire = 0; > ln->ln_ntick = 0; > - callout_stop(&ln->ln_timer_ch); > + callout_drain(&ln->ln_timer_ch); > } else { > ln->ln_expire = time_second + tick / hz; > if (tick > INT_MAX) { Hmmm, no, this patch didn't change anything for me. The same trap, the same bucket full of garbage. Tomorrow I'll try to trace all callout-related operations in nd6 and/or the whole netinet6. If there are more thoughts - I'll be happy to test. -- WBR, Victor V. Snezhko EMail: snezhko@indorsoft.ru