From owner-freebsd-net@freebsd.org Mon Dec 14 09:10:24 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E7EF2A43829 for ; Mon, 14 Dec 2015 09:10:24 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 939971CB0 for ; Mon, 14 Dec 2015 09:10:24 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id AF7B91FE024; Mon, 14 Dec 2015 10:10:22 +0100 (CET) Subject: Re: Random kernel panic when unloading Dummynet module in multicore processor To: Rasool Al-Saadi , "freebsd-net@freebsd.org" References: <6545444AE21C2749939E637E56594CEA3C10B9DA@gsp-ex02.ds.swin.edu.au> From: Hans Petter Selasky Message-ID: <566E87F1.3040606@selasky.org> Date: Mon, 14 Dec 2015 10:12:17 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <6545444AE21C2749939E637E56594CEA3C10B9DA@gsp-ex02.ds.swin.edu.au> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Dec 2015 09:10:25 -0000 On 12/14/15 01:06, Rasool Al-Saadi wrote: > Hi everyone, > > I am not sure if this bug was reported before or not (I didn't find exact problem) . > > When I try to unload dummynet module (kldunload dummynet) in FreeBSD11-CURRENT, sometimes the system is halt or panic. I noticed that this only happens with multicore processor. > When I disable all CPU's cores except one (using hint.lapic.x.disabled=1), the panic does not appear. > I tried to reproduce the panic in a system with Intel Core 2 Due 2.33HGz processor and also in VirtualBox VM (I set number of processors in VM setting to 2), and in both cases I got system panic. > > Most of the times the panic is: > Fatal trap 12: page fault while in kernel mode > ... > Stopped at callout_process+... > > Kernel version: > FreeBSD 11.0-CURRENT #0 r291495: Mon Nov 30 23:14:34 UTC 2015 root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 > > To reproduce the panic: > Run 'kldload dummynet && kldunload dummynet' many times (or write a shell script loop to do that) > > Debug Info: > > DUMMYNET 0 with IPv6 initialized (100409) > load_dn_sched dn_sched FIFO loaded > load_dn_sched dn_sched QFQ loaded > load_dn_sched dn_sched RR loaded > load_dn_sched dn_sched WF2Q+ loaded > load_dn_sched dn_sched PRIO loaded > unload_dn_sched dn_sched PRIO unloaded > unload_dn_sched dn_sched WF2Q+ unloaded > unload_dn_sched dn_sched RR unloaded > unload_dn_sched dn_sched QFQ unloaded > unload_dn_sched dn_sched FIFO unloaded > kernel trap 12 with interrupts disabled > > > Fatal trap 12: page fault while in kernel mode > cpuid = 0; apic id = 00 > fault virtual address = 0xffffffff82251c68 > fault code = supervisor read data, page not present > instruction pointer = 0x20:0xffffffff80a36c50 > stack pointer = 0x28:0xfffffe0116b0c710 > frame pointer = 0x28:0xfffffe0116b0c7a0 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = resume, IOPL = 0 > current process = 11 (idle: cpu0) > > Backtrace: > > #0 doadump (textdump=380682592) at pcpu.h:221 > 221 pcpu.h: No such file or directory. > in pcpu.h > (kgdb) backtrace > #0 doadump (textdump=380682592) at pcpu.h:221 > #1 0xffffffff8037d906 in db_fncall (dummy1=, > dummy2=, dummy3=, > dummy4=) at /usr/src/sys/ddb/db_command.c:568 > #2 0xffffffff8037d39e in db_command (cmd_table=0x0) > at /usr/src/sys/ddb/db_command.c:440 > #3 0xffffffff8037d134 in db_command_loop () > at /usr/src/sys/ddb/db_command.c:493 > #4 0xffffffff8037fbcb in db_trap (type=, code=0) > at /usr/src/sys/ddb/db_main.c:251 > #5 0xffffffff80a5e593 in kdb_trap (type=12, code=0, tf=) > at /usr/src/sys/kern/subr_kdb.c:654 > #6 0xffffffff80e69d01 in trap_fatal (frame=0xfffffe0116b0c660, > eva=) at /usr/src/sys/amd64/amd64/trap.c:829 > #7 0xffffffff80e69f4d in trap_pfault (frame=0xfffffe0116b0c660, > usermode=) at /usr/src/sys/amd64/amd64/trap.c:684 > #8 0xffffffff80e6967f in trap (frame=0xfffffe0116b0c660) > at /usr/src/sys/amd64/amd64/trap.c:435 > #9 0xffffffff80e49447 in calltrap () > at /usr/src/sys/amd64/amd64/exception.S:234 > #10 0xffffffff80a36c50 in callout_process (now=307238251753) > at /usr/src/sys/kern/kern_timeout.c:480 > #11 0xffffffff80f6809d in handleevents (now=307238251753, fake=0) > at /usr/src/sys/kern/kern_clocksource.c:212 > #12 0xffffffff80f68725 in timercb (et=, > arg=) at /usr/src/sys/kern/kern_clocksource.c:345 > #13 0xffffffff80e6f766 in hpet_intr_single (arg=) > at /usr/src/sys/dev/acpica/acpi_hpet.c:273 > #14 0xffffffff80e6f819 in hpet_intr (arg=0xfffffe0000dcb000) > at /usr/src/sys/dev/acpica/acpi_hpet.c:291 > #15 0xffffffff809e9f9c in intr_event_handle (ie=0xfffff80003ca1a00, > frame=0xfffffe0116b0c970) at /usr/src/sys/kern/kern_intr.c:1436 > #16 0xffffffff80fa1fb8 in intr_execute_handlers (isrc=0xfffff80003cea590, > frame=0xfffffe0116b0c970) at /usr/src/sys/x86/x86/intr_machdep.c:275 > #17 0xffffffff80fa6a78 in lapic_handle_intr (vector=, > frame=0xfffffe0116b0c970) at /usr/src/sys/x86/x86/local_apic.c:1008 > #18 0xffffffff80e49b27 in Xapic_isr1 () at apic_vector.S:116 > #19 0xffffffff80f9f6b6 in acpi_cpu_c1 () > at /usr/src/sys/x86/x86/cpu_machdep.c:133 > #20 0xffffffff8039f855 in acpi_cpu_idle (sbt=) > at /usr/src/sys/dev/acpica/acpi_cpu.c:1157 > #21 0xffffffff80f9f99f in cpu_idle_acpi (sbt=160862) > at /usr/src/sys/x86/x86/cpu_machdep.c:263 > #22 0xffffffff80f9fa47 in cpu_idle (busy=0) > at /usr/src/sys/x86/x86/cpu_machdep.c:415 > #23 0xffffffff80a4c565 in sched_idletd (dummy=) > at /usr/src/sys/kern/sched_ule.c:2688 > #24 0xffffffff809e7714 in fork_exit ( > callout=0xffffffff80a4c0c0 , arg=0x0, > frame=0xfffffe0116b0cc00) at /usr/src/sys/kern/kern_fork.c:1011 > #25 0xffffffff80e4997e in fork_trampoline () > at /usr/src/sys/amd64/amd64/exception.S:609 > #26 0x0000000000000000 in ?? () > Hi, Can you try this DIFF: https://reviews.freebsd.org/D3855 --HPS