From owner-freebsd-virtualization@freebsd.org Thu Feb 27 12:59:28 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 607AE240B99 for ; Thu, 27 Feb 2020 12:59:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 48St6R4RVBz4FBr for ; Thu, 27 Feb 2020 12:59:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id C6092240B94; Thu, 27 Feb 2020 12:59:26 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C4A51240B91 for ; Thu, 27 Feb 2020 12:59:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48St6P60Ltz4F95 for ; Thu, 27 Feb 2020 12:59:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4FF1B19038 for ; Thu, 27 Feb 2020 12:59:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 01RCxPHw044496 for ; Thu, 27 Feb 2020 12:59:25 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 01RCxP9R044495 for virtualization@FreeBSD.org; Thu, 27 Feb 2020 12:59:25 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 236989] AWS EC2 lockups "Missing interrupt" Date: Thu, 27 Feb 2020 12:59:25 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: cao@bus.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 12:59:28 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D236989 --- Comment #13 from Charles O'Donnell --- Most recent incident: Feb 27 04:23:31 s4 kernel: nvme4: Missing interrupt Feb 27 04:24:34 s4 syslogd: last message repeated 1 times Feb 27 04:24:34 s4 kernel: nvme1: Missing interrupt Feb 27 04:24:34 s4 kernel: nvme0: Missing interrupt Feb 27 04:24:34 s4 syslogd: last message repeated 1 times Feb 27 04:24:34 s4 kernel: nvme1: nvme1: cpl does not map to outstanding cmd Feb 27 04:24:34 s4 kernel: cdw0:00000000 sqhd:0012 sqid:0001 cid:0017 p:0 s= c:00 sct:0 m:0 dnr:0 Feb 27 04:24:34 s4 kernel: nvme1: Missing interrupt Feb 27 04:24:34 s4 kernel: nvme4: nvme1: Missing interrupt Feb 27 04:24:34 s4 kernel: Missing interrupt Feb 27 04:24:34 s4 syslogd: last message repeated 1 times Feb 27 04:24:34 s4 kernel: nvme4: Missing interrupt Feb 27 04:24:34 s4 kernel: nvme0: Missing interrupt Feb 27 04:24:34 s4 kernel: nvme1: nvme1: nvme4: Missing interrupt Feb 27 04:24:34 s4 kernel: Missing interrupt # freebsd-version -ukr 12.1-RELEASE-p2 12.1-RELEASE-p2 12.1-RELEASE-p2 # ls -lahtuT /etc/periodic/daily/ total 128 -rwxr-xr-x 1 root wheel 811B Feb 27 03:05:34 2020 999.local -rwxr-xr-x 1 root wheel 2.8K Feb 27 03:05:34 2020 800.scrub-zfs -rwxr-xr-x 1 root wheel 845B Feb 27 03:05:34 2020 510.status-world-kern= el -rwxr-xr-x 1 root wheel 737B Feb 27 03:05:34 2020 500.queuerun -rwxr-xr-x 1 root wheel 498B Feb 27 03:05:34 2020 480.status-ntpd -rwxr-xr-x 1 root wheel 451B Feb 27 03:05:34 2020 480.leapfile-ntpd -rwxr-xr-x 1 root wheel 2.0K Feb 27 03:05:34 2020 460.status-mail-rejec= ts -rwxr-xr-x 1 root wheel 1.0K Feb 27 03:01:00 2020 450.status-security -rwxr-xr-x 1 root wheel 1.4K Feb 27 03:01:00 2020 440.status-mailq -rwxr-xr-x 1 root wheel 705B Feb 27 03:01:00 2020 430.status-uptime -rwxr-xr-x 1 root wheel 611B Feb 27 03:01:00 2020 420.status-network -rwxr-xr-x 1 root wheel 684B Feb 27 03:01:00 2020 410.status-mfi -rwxr-xr-x 1 root wheel 590B Feb 27 03:01:00 2020 409.status-gconcat -rwxr-xr-x 1 root wheel 590B Feb 27 03:01:00 2020 408.status-gstripe -rwxr-xr-x 1 root wheel 591B Feb 27 03:01:00 2020 407.status-graid3 -rwxr-xr-x 1 root wheel 596B Feb 27 03:01:00 2020 406.status-gmirror -rwxr-xr-x 1 root wheel 807B Feb 27 03:01:00 2020 404.status-zfs -rwxr-xr-x 1 root wheel 583B Feb 27 03:01:00 2020 401.status-graid -rwxr-xr-x 1 root wheel 773B Feb 27 03:01:00 2020 400.status-disks -rwxr-xr-x 1 root wheel 724B Feb 27 03:01:00 2020 330.news -r-xr-xr-x 1 root wheel 1.4K Feb 27 03:01:00 2020 310.accounting -rwxr-xr-x 1 root wheel 693B Feb 27 03:01:00 2020 300.calendar -rwxr-xr-x 1 root wheel 1.0K Feb 27 03:01:00 2020 210.backup-aliases -rwxr-xr-x 1 root wheel 1.7K Feb 27 03:01:00 2020 200.backup-passwd -rwxr-xr-x 1 root wheel 603B Feb 27 03:01:00 2020 150.clean-hoststat -rwxr-xr-x 1 root wheel 1.0K Feb 27 03:01:00 2020 140.clean-rwho -rwxr-xr-x 1 root wheel 709B Feb 27 03:01:00 2020 130.clean-msgs -rwxr-xr-x 1 root wheel 1.1K Feb 27 03:01:00 2020 120.clean-preserve -rwxr-xr-x 1 root wheel 1.5K Feb 27 03:01:00 2020 110.clean-tmps -rwxr-xr-x 1 root wheel 1.3K Feb 27 03:01:00 2020 100.clean-disks drwxr-xr-x 2 root wheel 1.0K Dec 7 01:23:36 2018 . drwxr-xr-x 6 root wheel 512B Dec 7 01:23:36 2018 .. --=20 You are receiving this mail because: You are the assignee for the bug.=