From owner-freebsd-bugs@freebsd.org Thu Oct 18 16:20:27 2018 Return-Path: Delivered-To: freebsd-bugs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6A626247 for ; Thu, 18 Oct 2018 16:20:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 0814571E5F for ; Thu, 18 Oct 2018 16:20:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id C09F6246; Thu, 18 Oct 2018 16:20:26 +0000 (UTC) Delivered-To: bugs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9EA7F245 for ; Thu, 18 Oct 2018 16:20:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4025C71E5E for ; Thu, 18 Oct 2018 16:20:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 6E04A78 for ; Thu, 18 Oct 2018 16:20:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w9IGKPYL060547 for ; Thu, 18 Oct 2018 16:20:25 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w9IGKPKW060546 for bugs@FreeBSD.org; Thu, 18 Oct 2018 16:20:25 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 232387] head -r339076: system crash in vnet_epair_init during kern_jail_set in a kyua test on powerpc64 Date: Thu, 18 Oct 2018 16:20:25 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: marklmi26-fbsd@yahoo.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter cc Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Oct 2018 16:20:27 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D232387 Bug ID: 232387 Summary: head -r339076: system crash in vnet_epair_init during kern_jail_set in a kyua test on powerpc64 Product: Base System Version: CURRENT Hardware: powerpc OS: Any Status: New Severity: Affects Only Me Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: marklmi26-fbsd@yahoo.com CC: bz@FreeBSD.org [https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D222314 is a different issue, as noted there.] A powerpc64 head -r339076 based context running # kyua test -k /usr/tests/Kyuafile reliably crashes (so far) during kyua displaying: sys/netinet/reuseport_lb:basic_ipv4 -> failed: /usr/src/tests/sys/netinet/reuseport_lb.c:165: bind() failed: Address alrea= dy in use [0.013s] sys/netinet/reuseport_lb:basic_ipv6 -> failed: /usr/src/tests/sys/netinet/reuseport_lb.c:221: bind() failed: Address alrea= dy in use [0.013s] sys/netipsec/tunnel/aes_cbc_128_hmac_sha1:v4 ->=20=20 Example details based on a debug kernel (invariants, witness, and diagnostics) . . . Note the LOR backtrace and the crash backtrace are the same for the call chain that calls vnet_sysinit. . . . epair3a: Ethernet address: 02:60:27:70:4b:0a epair3b: Ethernet address: 02:60:27:70:4b:0b epair3a: link state changed to UP epair3b: link state changed to UP lock order reversal: 1st 0x13be260 allprison (allprison) @ /usr/src/sys/kern/kern_jail.c:960 2nd 0x15964a0 vnet_sysinit_sxlock (vnet_sysinit_sxlock) @ /usr/src/sys/net/vnet.c:575 stack backtrace: #0 0x6f6520 at witness_debugger+0xf4 #1 0x6f8440 at witness_checkorder+0xa1c #2 0x675690 at _sx_slock_int+0x70 #3 0x675810 at _sx_slock+0x1c #4 0x7f4338 at vnet_sysinit+0x38 #5 0x7f44dc at vnet_alloc+0x118 #6 0x62ab84 at kern_jail_set+0x3274 #7 0x62b62c at sys_jail_set+0x8c #8 0xa8a798 at trap+0x9a0 #9 0xa7e660 at powerpc_interrupt+0x140 fatal kernel trap: exception =3D 0x300 (data storage interrupt) virtual address =3D 0xc00000008df1df30 dsisr =3D 0x42000000 srr0 =3D 0xe000000047854e98 (0xe000000047854e98) srr1 =3D 0x9000000000009032 current msr =3D 0x9000000000009032 lr =3D 0xe000000047854e90 (0xe000000047854e90) curthread =3D 0xc0000000206b6000 pid =3D 9464, comm =3D jail (Hand transcribed from here on:) [ thread pid 9464 tid 100296 ] Stopped at vnet_epair_init+0x78: stdx r3,r29,r30 db:0:kdb.enter.default> bt Tracing pid 9464 tid 100296 td 0xc0000000206b6000 0xe000000047274240: at vnet_sysinit+0x70 0xe000000047274270: at vnet_alloc+0x118 0xe000000047274300: at kern_jail_set+0x3274 0xe000000047274610: at sys_jail_set+0x8c 0xe000000047274660: at trap+0x9a0 0xe000000047274790: at powerpc_interrupt+0x140 0xe000000047274820: user sc trap by 0x81016a888 srr1 =3D 0x900000000000f032 r1 =3D 0x3fffffffffffd080 cr =3D 0x28002482 xer =3D 0x20000000 ctr =3D 0x81016a880 r2 =3D 0x810322300 There are past reports of the lock order reversal, such as: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210907 but this did not report any crash. Notes: The powerpc64 -r339076 based system was built via devel/powerpc-xtoolchain-gcc and created system-cc-is-clang and is using base/binutils as well. kyua is as of ports -r480180 and system-clang built it (and other things). I experiment with what the issues are with using fairly modern compiler toolchains for powerpc64 instead of gcc 4.2.1 . At this point I do not see this as likely to be responsible for the above crash. I'll see about adding a objdump or kgdb disass of vnet_epair_init in a bit. --=20 You are receiving this mail because: You are the assignee for the bug.=