From owner-freebsd-current@freebsd.org Sat Sep 12 12:32:37 2015 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 70A68A03EF7 for ; Sat, 12 Sep 2015 12:32:37 +0000 (UTC) (envelope-from melifaro@ipfw.ru) Received: from forward11j.cmail.yandex.net (forward11j.cmail.yandex.net [IPv6:2a02:6b8:0:1630::b1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Certum Level IV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E4C3413AE; Sat, 12 Sep 2015 12:32:36 +0000 (UTC) (envelope-from melifaro@ipfw.ru) Received: from web27j.yandex.ru (web27j.yandex.ru [5.45.198.68]) by forward11j.cmail.yandex.net (Yandex) with ESMTP id 63CA720E07; Sat, 12 Sep 2015 15:32:32 +0300 (MSK) Received: from 127.0.0.1 (localhost [127.0.0.1]) by web27j.yandex.ru (Yandex) with ESMTP id 68A4CDC20D1; Sat, 12 Sep 2015 15:32:31 +0300 (MSK) Received: by web27j.yandex.ru with HTTP; Sat, 12 Sep 2015 15:32:30 +0300 From: Alexander V. Chernikov Envelope-From: melifaro@ipfw.ru To: hiren panchasara , Hans Petter Selasky Cc: "freebsd-current@FreeBSD.org" , "jch@FreeBSD.org" In-Reply-To: <20150911232154.GS64965@strugglingcoder.info> References: <20150910192351.GF64965@strugglingcoder.info> <55F27D68.6080501@selasky.org> <20150911232154.GS64965@strugglingcoder.info> Subject: Re: Panic on kldload/kldunload in/near callout MIME-Version: 1.0 Message-Id: <1405441442061150@web27j.yandex.ru> X-Mailer: Yamail [ http://yandex.ru ] 5.0 Date: Sat, 12 Sep 2015 15:32:30 +0300 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=koi8-r X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Sep 2015 12:32:37 -0000 12.09.2015, 02:22, "hiren panchasara" : > On 09/11/15 at 09:06P, Hans Petter Selasky wrote: >> šOn 09/10/15 21:23, hiren panchasara wrote: >> š> I am on 11.0-CURRENT FreeBSD 11.0-CURRENT #4 r286760M: Thu Sep 10 >> š> 08:15:43 MST 2015 >> š> >> š> I get random (1 out of 10 tries) panics when I do: >> š> # kldunload dummynet ; kldunload ipfw ;kldload ipfw ; kldload dummynet >> š> >> š> I used to get panics on a couple months old -head also. >> š> >> š> kernel trap 12 with interrupts disabled >> š> >> š> Fatal trap 12: page fault while in kernel mode >> š> cpuid = 0; apic id = 00 >> š> fault virtual address = 0xffffffff8225cf58 >> š> fault code = supervisor read data, page not present >> š> instruction pointer = 0x20:0xffffffff80aad500 >> š> stack pointer = 0x28:0xfffffe1f9d588700 >> š> frame pointer = 0x28:0xfffffe1f9d588790 >> š> code segment = base 0x0, limit 0xfffff, type 0x1b >> š> = DPL 0, pres 1, long 1, def32 0, gran 1 >> š> >> š> Following https://www.freebsd.org/doc/faq/advanced.html, I did: >> š> # nm -n /boot/kernel/kernel | grep ffffffff80aad500 >> š> # nm -n /boot/kernel/kernel | grep ffffffff80aad50 >> š> # nm -n /boot/kernel/kernel | grep ffffffff80aad5 >> š> # nm -n /boot/kernel/kernel | grep ffffffff80aad >> š> ffffffff80aad030 t itimers_event_hook_exec >> š> ffffffff80aad040 t realtimer_expire >> š> ffffffff80aad360 T callout_process >> š> ffffffff80aad6b0 t softclock_call_cc >> š> ffffffff80aadc10 T softclock >> š> ffffffff80aadd20 T timeout >> š> ffffffff80aade90 T callout_reset_sbt_on >> š> >> š> So I guess " ffffffff80aad360 T callout_process" is the closest match? >> š> >> š> I'll try to get real dump to get more information but that may take a >> š> while. >> š> >> š> ccing jch and hans who've been playing in this area. >> >> šHi, >> >> šPossibly it means some timer was not drained before the module was >> šunloaded. It is not enough to only stop timers before freeing its >> šmemory. Or maybe a timer was restarted after drain. >> >> šCan you get the full backtrace and put debugging symbols into the kernel? > > I'll try to get it. Meanwhile I am getting another panic on idle box: > http://pastebin.com/9qJTFMik The easiest explanation could be lack of lla_create() result check, fixed in r286945. This panic is triggered by fast interface down-up (or just up), when ARP packet is received but there are no (matching) IPv4 prefix on the interface. If this is not the case (e.g. it paniced w/o any interface changes and there were no other subnets in given L2 segment) I'd be happy to debug this further. > > This "looks" similar to > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=156026 which got fixed > via https://svnweb.freebsd.org/base?view=revision&revision=r214675 > "Don't leak the LLE lock if the arptimer callout is pending or > inactive." > > Is what I am seeing similar to this? > > I'll try and get more info. > > Cheers, > Hiren