From owner-freebsd-bugs@FreeBSD.ORG Sun May 25 09:50:00 2014 Return-Path: Delivered-To: freebsd-bugs@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BD6CF7AD for ; Sun, 25 May 2014 09:50:00 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9953D2FDF for ; Sun, 25 May 2014 09:50:00 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s4P9o09T049902 for ; Sun, 25 May 2014 09:50:00 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s4P9o0La049901; Sun, 25 May 2014 09:50:00 GMT (envelope-from gnats) Resent-Date: Sun, 25 May 2014 09:50:00 GMT Resent-Message-Id: <201405250950.s4P9o0La049901@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Kuang-che Wu Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4A005528 for ; Sun, 25 May 2014 09:42:50 +0000 (UTC) Received: from cgiserv.freebsd.org (cgiserv.freebsd.org [IPv6:2001:1900:2254:206a::50:4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3616E2FB5 for ; Sun, 25 May 2014 09:42:50 +0000 (UTC) Received: from cgiserv.freebsd.org ([127.0.1.6]) by cgiserv.freebsd.org (8.14.8/8.14.8) with ESMTP id s4P9gmuD053812 for ; Sun, 25 May 2014 09:42:48 GMT (envelope-from nobody@cgiserv.freebsd.org) Received: (from nobody@localhost) by cgiserv.freebsd.org (8.14.8/8.14.8/Submit) id s4P9gm9G053811; Sun, 25 May 2014 09:42:48 GMT (envelope-from nobody) Message-Id: <201405250942.s4P9gm9G053811@cgiserv.freebsd.org> Date: Sun, 25 May 2014 09:42:48 GMT From: Kuang-che Wu To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-3.1 Subject: kern/190197: Kernel panic if dial multiple ppp at the same time X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 May 2014 09:50:00 -0000 >Number: 190197 >Category: kern >Synopsis: Kernel panic if dial multiple ppp at the same time >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Sun May 25 09:50:00 UTC 2014 >Closed-Date: >Last-Modified: >Originator: Kuang-che Wu >Release: 11.0-CURRENT >Organization: >Environment: FreeBSD kcwu.csie.org 11.0-CURRENT FreeBSD 11.0-CURRENT #1 3bde90f(master): Sat May 17 17:26:09 CST 2014 root@kcwu.csie.org:/usr/obj/usr/src/sys/DESKTOP amd64 >Description: I found dialing multiple ppp profiles may make kernel panic. And I found the panic rate is related to PCIe-to-SATA adapter. Without the adapter, the panic rate is low, maybe less than 10%. With the adapter, the panic rate is pretty high, maybe more than 90%. ahci0: port 0xcf00-0xcf07,0xce00-0xce03,0xcd00-0xcd07,0xcc00-0xcc03,0xcb00-0xcb1f mem 0xfbeff000-0xfbeff1ff irq 16 at device 0.0 on pci1 ahci0: AHCI v1.20 with 2 6Gbps ports, Port Multiplier supported (I have tested other adapters (different brand) and the panic rate is still high) # kgdb kernel.debug /usr/crash/vmcore.9 ..skip some lines... <118>Starting PPP profile: hinet_static hi1 hi2 WARNING: attempt to domain_add(netgraph) after domainfinalize() Fatal trap 12: page fault while in kernel mode cpuid = 2; apic id = 04 fault virtual address = 0x378 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff80934881 stack pointer = 0x28:0xfffffe045f3d7930 frame pointer = 0x28:0xfffffe045f3d79b0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 656 (ppp) trap number = 12 panic: page fault cpuid = 2 KDB: stack backtrace: #0 0xffffffff80988d50 at kdb_backtrace+0x60 #1 0xffffffff8094a865 at panic+0x155 #2 0xffffffff80dce2ef at trap_fatal+0x38f #3 0xffffffff80dce63c at trap_pfault+0x33c #4 0xffffffff80dcdc7a at trap+0x47a #5 0xffffffff80db1102 at calltrap+0x8 #6 0xffffffff809347ba at __mtx_lock_flags+0x5a #7 0xffffffff81e2a5d3 at ng_attach_common+0x73 #8 0xffffffff81e2a673 at ngc_attach+0x43 #9 0xffffffff809c7fbf at socreate+0x1af #10 0xffffffff809cfe77 at sys_socket+0xf7 #11 0xffffffff80dced0b at amd64_syscall+0x3fb #12 0xffffffff80db13eb at Xfast_syscall+0xfb ..skip some lines... #0 doadump (textdump=) at pcpu.h:219 219 __asm("movq %%gs:%1,%0" : "=r" (td) (kgdb) bt #0 doadump (textdump=) at pcpu.h:219 #1 0xffffffff8094a3e8 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:449 #2 0xffffffff8094a8a4 in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:756 #3 0xffffffff80dce2ef in trap_fatal (frame=, eva=) at /usr/src/sys/amd64/amd64/trap.c:882 #4 0xffffffff80dce63c in trap_pfault (frame=0xfffffe045f72a880, usermode=) at /usr/src/sys/amd64/amd64/trap.c:693 #5 0xffffffff80dcdc7a in trap (frame=0xfffffe045f72a880) at /usr/src/sys/amd64/amd64/trap.c:457 #6 0xffffffff80db1102 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:231 #7 0xffffffff80934881 in __mtx_lock_sleep (c=0xffffffff81e2bc40, tid=18446735278230332560, opts=0, file=0x200
, line=5005456) at /usr/src/sys/kern/kern_mutex.c:430 #8 0xffffffff809347ba in __mtx_lock_flags (c=, opts=613803152, file=0x0, line=512) at /usr/src/sys/kern/kern_mutex.c:223 #9 0xffffffff81e2a5d3 in ng_attach_common (so=0xfffff800248052b8, type=2) at /usr/src/sys/modules/netgraph/socket/../../../netgraph/ng_socket.c:605 #10 0xffffffff81e2a673 in ngc_attach (so=0xfffff800248052b8, proto=, td=) at /usr/src/sys/modules/netgraph/socket/../../../netgraph/ng_socket.c:538 #11 0xffffffff809c7fbf in socreate (dom=, aso=0xfffffe045f72aab0, type=, proto=2, cred=0xfffff8001a5bd300, td=0xfffff8002495e490) at /usr/src/sys/kern/uipc_socket.c:458 #12 0xffffffff809cfe77 in sys_socket (td=0xfffff8002495e490, uap=) at /usr/src/sys/kern/uipc_syscalls.c:265 #13 0xffffffff80dced0b in amd64_syscall (td=0xfffff8002495e490, traced=0) at subr_syscall.c:133 #14 0xffffffff80db13eb in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:390 #15 0x0000000801e0fbca in ?? () Previous frame inner to this frame (corrupt stack?) Current language: auto; currently minimal (kgdb) up #1 0xffffffff8094a3e8 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:449 449 doadump(TRUE); (kgdb) up #2 0xffffffff8094a8a4 in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:756 756 kern_reboot(bootopt); (kgdb) up #3 0xffffffff80dce2ef in trap_fatal (frame=, eva=) at /usr/src/sys/amd64/amd64/trap.c:882 882 panic("%s", trap_msg[type]); (kgdb) up #4 0xffffffff80dce63c in trap_pfault (frame=0xfffffe045f72a880, usermode=) at /usr/src/sys/amd64/amd64/trap.c:693 693 trap_fatal(frame, eva); (kgdb) up #5 0xffffffff80dcdc7a in trap (frame=0xfffffe045f72a880) at /usr/src/sys/amd64/amd64/trap.c:457 457 (void) trap_pfault(frame, FALSE); (kgdb) up #6 0xffffffff80db1102 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:231 231 call trap Current language: auto; currently asm (kgdb) up #7 0xffffffff80934881 in __mtx_lock_sleep (c=0xffffffff81e2bc40, tid=18446735278230332560, opts=0, file=0x200
, line=5005456) at /usr/src/sys/kern/kern_mutex.c:430 430 owner = (struct thread *)(v & ~MTX_FLAGMASK); Current language: auto; currently minimal (kgdb) up #8 0xffffffff809347ba in __mtx_lock_flags (c=, opts=613803152, file=0x0, line=512) at /usr/src/sys/kern/kern_mutex.c:223 223 __mtx_lock(m, curthread, opts, file, line); >How-To-Repeat: Add multiple ppp profiles into rc.conf and reboot. >Fix: I found adding sleep between ppp dialing can work around this issue. --- a/rc.d/ppp +++ b/rc.d/ppp @@ -86,6 +86,7 @@ ppp_start() for _p in $_ppp_profile; do echo -n " $_p" ppp_start_profile $_p + sleep 2 done echo "." >Release-Note: >Audit-Trail: >Unformatted: