From owner-freebsd-bugs@freebsd.org Mon Apr 29 19:42:28 2019 Return-Path: Delivered-To: freebsd-bugs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C54DB159B30D for ; Mon, 29 Apr 2019 19:42:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 5E2276F3C7 for ; Mon, 29 Apr 2019 19:42:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 21C95159B30B; Mon, 29 Apr 2019 19:42:27 +0000 (UTC) Delivered-To: bugs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F20C6159B30A for ; Mon, 29 Apr 2019 19:42:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8A15A6F3BF for ; Mon, 29 Apr 2019 19:42:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id C7F9C1E4C9 for ; Mon, 29 Apr 2019 19:42:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x3TJgPA6094162 for ; Mon, 29 Apr 2019 19:42:25 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x3TJgPT6094161 for bugs@FreeBSD.org; Mon, 29 Apr 2019 19:42:25 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 237655] Non-deterministic panic with pf Date: Mon, 29 Apr 2019 19:42:25 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ngie@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 19:42:28 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D237655 Bug ID: 237655 Summary: Non-deterministic panic with pf Product: Base System Version: CURRENT Hardware: Any OS: Any Status: New Severity: Affects Some People Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: ngie@FreeBSD.org The last few test runs have been failing as follows, with panics in strncmp= () managing (what I assume is epair or tun) interfaces via the ioctl handler. There are some questionable LoR messages also printed out in the pf code ab= out lock holding, but this is likely not the root cause. >From https://ci.freebsd.org/job/FreeBSD-head-amd64-test/11014/consoleText : sys/netpfil/pf/pfsync:basic -> lock order reversal: (sleepable after non-sleepable) 1st 0xfffff8003a43f070 pfsync (pfsync) @ /usr/src/sys/netpfil/pf/if_pfsync.c:1402 2nd 0xffffffff820c48c0 in_multi_sx (in_multi_sx) @ /usr/src/sys/netinet/in_mcast.c:1251 stack backtrace: #0 0xffffffff80c47773 at witness_debugger+0x73 #1 0xffffffff80c474bd at witness_checkorder+0xa7d #2 0xffffffff80be7ed8 at _sx_xlock+0x68 #3 0xffffffff80d65271 at in_joingroup+0x31 #4 0xffffffff82839086 at pfsyncioctl+0x6e6 #5 0xffffffff80d60116 at in_control+0x376 #6 0xffffffff80ce168b at ifioctl+0x57b #7 0xffffffff80c4c6ba at kern_ioctl+0x28a #8 0xffffffff80c4c3bd at sys_ioctl+0x15d #9 0xffffffff810b2e16 at amd64_syscall+0x276 #10 0xffffffff8108b5fd at fast_syscall_common+0x101 passed [2.316s] sys/netpfil/pf/pfsync:defer -> passed [2.248s] sys/netpfil/pf/rdr:basic -> Apr 29 18:28:59 kernel: nd6_dad_timer: cancel DAD on epair3a because of ND6_IFF_IFDISABLED. passed [4.153s] sys/netpfil/pf/route_to:v4 -> passed [3.195s] sys/netpfil/pf/route_to:v6 -> Apr 29 18:29:07 kernel: nd6_dad_timer: cal= led with non-tentative address fe80:7::39:3ff:fe4c:500a(epair4a) Apr 29 18:29:07 kernel: nd6_dad_timer: called with non-tentative address fe80:3::35:96ff:fe61:640b(epair3b) Apr 29 18:29:07 kernel: nd6_dad_timer: called with non-tentative address fe80:4::39:3ff:fe4c:500b(epair4b) Apr 29 18:29:07 kernel: nd6_dad_timer: called with non-tentative address fe80:5::35:96ff:fe61:640a(epair3a) passed [3.181s] sys/netpfil/pf/set_skip:set_skip_group -> passed [0.097s] sys/netpfil/pf/set_skip:set_skip_group_lo -> passed [0.113s] sys/netpfil/pf/set_tos:v4 -> passed [8.702s] sys/netpfil/pf/synproxy:synproxy -> passed [0.161s] Fatal trap 9: general protection fault while in kernel mode cpuid =3D 0; apic id =3D 00 instruction pointer =3D 0x20:0xffffffff80ccb525 stack pointer =3D 0x28:0xfffffe0030ec9740 frame pointer =3D 0x28:0xfffffe0030ec9740 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 69669 (ifconfig) trap number =3D 9 panic: general protection fault cpuid =3D 0 time =3D 1556562559 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0030ec9= 450 vpanic() at vpanic+0x19d/frame 0xfffffe0030ec94a0 panic() at panic+0x43/frame 0xfffffe0030ec9500 trap_fatal() at trap_fatal+0x394/frame 0xfffffe0030ec9560 trap() at trap+0x6c/frame 0xfffffe0030ec9670 calltrap() at calltrap+0x8/frame 0xfffffe0030ec9670 --- trap 0x9, rip =3D 0xffffffff80ccb525, rsp =3D 0xfffffe0030ec9740, rbp = =3D 0xfffffe0030ec9740 --- strncmp() at strncmp+0x15/frame 0xfffffe0030ec9740 ifunit_ref() at ifunit_ref+0x51/frame 0xfffffe0030ec9780 ifioctl() at ifioctl+0x508/frame 0xfffffe0030ec9850 kern_ioctl() at kern_ioctl+0x28a/frame 0xfffffe0030ec98c0 sys_ioctl() at sys_ioctl+0x15d/frame 0xfffffe0030ec9990 amd64_syscall() at amd64_syscall+0x276/frame 0xfffffe0030ec9ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0030ec9ab0 --- syscall (54, FreeBSD ELF64, sys_ioctl), rip =3D 0x80048531a, rsp =3D 0x7fffffffe458, rbp =3D 0x7fffffffe4c0 --- KDB: enter: panic [ thread pid 69669 tid 100143 ] Stopped at kdb_enter+0x3b: movq $0,kdb_why db:0:kdb.enter.panic> show pcpu cpuid =3D 0 dynamic pcpu =3D 0xb84800 curthread =3D 0xfffff80004f305a0: pid 69669 tid 100143 "ifconfig" curpcb =3D 0xfffffe0030ec9b80 fpcurthread =3D 0xfffff80004f305a0: pid 69669 "ifconfig" idlethread =3D 0xfffff80003272000: tid 100003 "idle: cpu0" curpmap =3D 0xfffff8003e6fb130 tssp =3D 0xffffffff821cd320 commontssp =3D 0xffffffff821cd320 rsp0 =3D 0xfffffe0030ec9b80 gs32p =3D 0xffffffff821d3f58 ldt =3D 0xffffffff821d3f98 tss =3D 0xffffffff821d3f88 tlb gen =3D 455364 curvnet =3D 0xfffff8000307aec0 spin locks held: db:0:kdb.enter.panic> alltrace Tracing command ifconfig pid 69669 tid 100143 td 0xfffff80004f305a0 (CPU 0) kdb_enter() at kdb_enter+0x3b/frame 0xfffffe0030ec9450 vpanic() at vpanic+0x1ba/frame 0xfffffe0030ec94a0 panic() at panic+0x43/frame 0xfffffe0030ec9500 trap_fatal() at trap_fatal+0x394/frame 0xfffffe0030ec9560 trap() at trap+0x6c/frame 0xfffffe0030ec9670 calltrap() at calltrap+0x8/frame 0xfffffe0030ec9670 --- trap 0x9, rip =3D 0xffffffff80ccb525, rsp =3D 0xfffffe0030ec9740, rbp = =3D 0xfffffe0030ec9740 --- strncmp() at strncmp+0x15/frame 0xfffffe0030ec9740 ifunit_ref() at ifunit_ref+0x51/frame 0xfffffe0030ec9780 ifioctl() at ifioctl+0x508/frame 0xfffffe0030ec9850 kern_ioctl() at kern_ioctl+0x28a/frame 0xfffffe0030ec98c0 sys_ioctl() at sys_ioctl+0x15d/frame 0xfffffe0030ec9990 amd64_syscall() at amd64_syscall+0x276/frame 0xfffffe0030ec9ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0030ec9ab0 --- syscall (54, FreeBSD ELF64, sys_ioctl), rip =3D 0x80048531a, rsp =3D 0x7fffffffe458, rbp =3D 0x7fffffffe4c0 --- --=20 You are receiving this mail because: You are the assignee for the bug.=