From owner-freebsd-bugs@freebsd.org Tue Apr 10 07:26:27 2018 Return-Path: Delivered-To: freebsd-bugs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E339FF83F3A for ; Tue, 10 Apr 2018 07:26:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 68B2278DAC for ; Tue, 10 Apr 2018 07:26:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 29CACF83F39; Tue, 10 Apr 2018 07:26:26 +0000 (UTC) Delivered-To: bugs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E33CEF83F38 for ; Tue, 10 Apr 2018 07:26:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 81FF778DA8 for ; Tue, 10 Apr 2018 07:26:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id A661667D5 for ; Tue, 10 Apr 2018 07:26:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w3A7QOhh011303 for ; Tue, 10 Apr 2018 07:26:24 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w3A7QOcQ011302 for bugs@FreeBSD.org; Tue, 10 Apr 2018 07:26:24 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 227404] UP FreeBSD VM always hangs on reboot since 20180329-r331740 Date: Tue, 10 Apr 2018 07:26:24 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: decui@microsoft.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2018 07:26:27 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D227404 --- Comment #5 from Dexuan Cui --- (In reply to Dexuan Cui from comment #4) When the bug reproduces, the log is: Stopping cron. Stopping sshd. appending output to nohup.out Stopping devd. Writing entropy file:. Writing early boot entropy file:. Terminated . Apr 10 14:46:40 decui-b11 syslogd: exiting on signal 15 Waiting (max 60 seconds) for system process `vnlru' to stop... done Waiting (max 60 seconds) for system process `bufdaemon' to stop... done Waiting (max 60 seconds) for system process `syncer' to stop... Syncing disks, vnodes remaining... 4 (It hangs here) After I revert 63a483ed5f4eaadb8979992c7a5de24c7a471c61, the bug can't reproduce, despite the messages: Apr 10 14:28:44 decui-b11 syslogd: exiting on signal 15 Waiting (max 60 seconds) for system process `vnlru' to stop... done Waiting (max 60 seconds) for system process `bufdaemon' to stop... done Waiting (max 60 seconds) for system process `syncer' to stop... Syncing disks, vnodes remaining... 4 1 0 done All buffers synced. lock order reversal: 1st 0xfffff80008533ba8 ufs (ufs) @ /root/bsd.git/sys/kern/vfs_mount.c:1335 2nd 0xfffff800085d0428 syncer (syncer) @ /root/bsd.git/sys/kern/vfs_subr.c:2732 stack backtrace: #0 0xffffffff80bccfa3 at witness_debugger+0x73 #1 0xffffffff80bcce24 at witness_checkorder+0xe34 #2 0xffffffff80b3bb9b at lockmgr_lock_fast_path+0x17b #3 0xffffffff8119d069 at VOP_LOCK1_APV+0xd9 #4 0xffffffff80c488a6 at _vn_lock+0x66 #5 0xffffffff80c379a7 at vputx+0x157 #6 0xffffffff80c2f7d9 at dounmount+0x4d9 #7 0xffffffff80c3919b at vfs_unmountall+0x6b #8 0xffffffff80c14a25 at bufshutdown+0x2c5 #9 0xffffffff80b66d7a at kern_reboot+0x21a #10 0xffffffff80b66b09 at sys_reboot+0x3a9 #11 0xffffffff8102706b at amd64_syscall+0x79b #12 0xffffffff8100191d at fast_syscall_common+0x101 lock order reversal: 1st 0xfffff80008533ba8 ufs (ufs) @ /root/bsd.git/sys/kern/vfs_mount.c:1335 2nd 0xfffff800085d07e8 devfs (devfs) @ /root/bsd.git/sys/ufs/ffs/ffs_vfsops.c:1371 stack backtrace: #0 0xffffffff80bccfa3 at witness_debugger+0x73 #1 0xffffffff80bcce24 at witness_checkorder+0xe34 #2 0xffffffff80b3bb9b at lockmgr_lock_fast_path+0x17b #3 0xffffffff8119d069 at VOP_LOCK1_APV+0xd9 #4 0xffffffff80c488a6 at _vn_lock+0x66 #5 0xffffffff80e67a93 at ffs_flushfiles+0x93 #6 0xffffffff80e4adf2 at softdep_flushfiles+0x82 #7 0xffffffff80e6a147 at ffs_unmount+0x77 #8 0xffffffff80c2f819 at dounmount+0x519 #9 0xffffffff80c3919b at vfs_unmountall+0x6b #10 0xffffffff80c14a25 at bufshutdown+0x2c5 #11 0xffffffff80b66d7a at kern_reboot+0x21a #12 0xffffffff80b66b09 at sys_reboot+0x3a9 #13 0xffffffff8102706b at amd64_syscall+0x79b #14 0xffffffff8100191d at fast_syscall_common+0x101 Uptime: 2m50s acpi0: Powering system off BTW, when the patch is reverted, I occasionally get this when the VM boots,= but I guess that's a different issue: (da1:storvsc2:0:0:0): storvsc inquiry (6) [0 b2 0 4 1 ... ] da2 at storvsc2 bus 0 scbus4 target 0 lun 2 da2: Fixed Direct Access SPC-3 SCSI device da2: 300.000MB/s transfers da2: Command Queueing enabled da2: 51200MB (104857600 512 byte sectors) s_debugger+0x73 #1 0xffffffff80(da2:storvsc2:0:0:2): storvsc inquiry (6) [0 b2 0 4 1 ... ] (da1:storvsc2:0:0:0): storvsc inquiry (5) [0 b0 0 3c 0 ... ] bce381 at witness_warn+0x461 #2 0xffffffff81026273 at trap_pfa(da2:storvsc2:0:0:2): storvsc inquiry (5) = [0 b0 0 3c 0 ... ] (da1:storvsc2:0:0:0): storvsc inquiry (5) [0 b1 0 3c 0 ... ] da1: Delete methods: ult+0x53 #3 0xffffffff81025a72 (da2:storvsc2:0:0:2): storvsc inquiry (5) [0 b1 0 3c 0 ... ] da2: Delete methods: at trap+0x2f2 #4 0xffffffff810010cc at calltrap+0x8 #5 0xffffffff80c1af78 at vfs_vmio_unwire+0x78 #6 0xffffffff80c16350 at bGEOM: new disk da2 relse+0x3c0 #7 0xffffffff80e6af3a at ffs_use_bread+0x9a #8 0xffffffff80e6659c at ffs_sbget+0x8c #9 0xffffffff80e69213 at ffs_mount+0xe03 #10 0xffffffff80c2e449 at vfs_domount+0x719 #11 0xffffffff80c2d727 at vfs_donmount+0x7f7 #12 0xffffffff80c30a32 at kernel_mount+0x62 #13 0xffffffff80c32ddd at parse_mount+0x43d #14 0xffffffff80c3150c at vfs_mountroot+0x68c #15 0xffffffff80afe567 at start_init+0x27 #16 0xffffffff80b277b4 at fork_exit+0x84 #17 0xffffffff81001dee at fork_trampoline+0xe Fatal trap 12: page fault while in kernel mode cpuid =3D 12; apic id =3D 0c fault virtual address =3D 0x0 fault code =3D supervisor read data, page not present instruction pointer =3D 0x20:0xffffffff80ea6081 stack pointer =3D 0x28:0xfffffe002d0b1140 frame pointer =3D 0x28:0xfffffe002d0b1150 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 1 (kernel) [ thread pid 1 tid 100002 ] Stopped at _vm_page_deactivate+0xb1: cmpq %rcx,(%rax) db> bt Tracing pid 1 tid 100002 td 0xfffff800032e0560 _vm_page_deactivate() at _vm_page_deactivate+0xb1/frame 0xfffffe002d0b1150 vfs_vmio_unwire() at vfs_vmio_unwire+0x78/frame 0xfffffe002d0b1170 brelse() at brelse+0x3c0/frame 0xfffffe002d0b11c0 ffs_use_bread() at ffs_use_bread+0x9a/frame 0xfffffe002d0b1200 ffs_sbget() at ffs_sbget+0x8c/frame 0xfffffe002d0b1270 ffs_mount() at ffs_mount+0xe03/frame 0xfffffe002d0b1410 vfs_domount() at vfs_domount+0x719/frame 0xfffffe002d0b1630 vfs_donmount() at vfs_donmount+0x7f7/frame 0xfffffe002d0b16e0 kernel_mount() at kernel_mount+0x62/frame 0xfffffe002d0b1740 parse_mount() at parse_mount+0x43d/frame 0xfffffe002d0b1880 vfs_mountroot() at vfs_mountroot+0x68c/frame 0xfffffe002d0b1a00 start_init() at start_init+0x27/frame 0xfffffe002d0b1a70 fork_exit() at fork_exit+0x84/frame 0xfffffe002d0b1ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe002d0b1ab0 --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- db> --=20 You are receiving this mail because: You are the assignee for the bug.=