From owner-freebsd-virtualization@freebsd.org Mon Apr 20 08:35:29 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F1C092BB265 for ; Mon, 20 Apr 2020 08:35:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 495KlP69lMz3QYV for ; Mon, 20 Apr 2020 08:35:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id D204D2BB264; Mon, 20 Apr 2020 08:35:29 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D1C992BB263 for ; Mon, 20 Apr 2020 08:35:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 495KlP5FHRz3QYT for ; Mon, 20 Apr 2020 08:35:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id AF64DCA57 for ; Mon, 20 Apr 2020 08:35:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 03K8ZTie072710 for ; Mon, 20 Apr 2020 08:35:29 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 03K8ZTqI072709 for virtualization@FreeBSD.org; Mon, 20 Apr 2020 08:35:29 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 235856] FreeBSD freezes on AWS EC2 t3 machines Date: Mon, 20 Apr 2020 08:35:29 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: mail@rubenvos.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Apr 2020 08:35:30 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D235856 --- Comment #50 from mail@rubenvos.com --- (In reply to Colin Percival from comment #49) Hi Colin, The machine that was exhibiting the most freezes will be tested extensively during the next couple of days. No errors on that instance yet. We will probably have to decommission this particular instance end of this week tho= ugh :( I just had a look on one of the other machines and there is an error in one= of the logs: Apr 5 09:31:41 volume3 ec2[7704]: -----END SSH HOST KEY FINGERPRINTS----- Apr 5 09:31:41 volume3 ec2[14622]: ############################################################# Apr 16 04:08:31 volume3 kernel: nvme1: nvme_timeout Apr 16 04:08:31 volume3 kernel: nvme1:=20 Apr 16 04:08:31 volume3 kernel: Resetting controller due to a timeout. Apr 16 04:08:31 volume3 kernel: nvme1: resetting controller Apr 16 04:08:31 volume3 kernel: nvme1: temperature threshold not supported Apr 16 04:08:31 volume3 kernel: nvme1: resubmitting queued i/o Apr 16 04:08:31 volume3 kernel: nvme1: READ sqid:2 cid:0 nsid:1 lba:2029427= 280 len:8 Apr 16 04:08:31 volume3 kernel: nvme1: resubmitting queued i/o Apr 16 04:08:31 volume3 kernel: nvme1: READ sqid:2 cid:0 nsid:1 lba:1776954= 392 len:8 Apr 16 04:08:31 volume3 kernel: nvme1: resubmitting queued i/o Apr 16 04:08:31 volume3 kernel: nvme1: READ sqid:2 cid:0 nsid:1 lba:1315250= 264 len:8 Apr 16 04:08:31 volume3 kernel: nvme1: resubmitting queued i/o Apr 16 04:08:31 volume3 kernel: nvme1: READ sqid:2 cid:0 nsid:1 lba:8718688= 18 len:8 Apr 16 04:08:31 volume3 kernel: nvme1: resubmitting queued i/o Apr 16 04:08:31 volume3 kernel: nvme1: READ sqid:2 cid:0 nsid:1 lba:4230187= 43 len:7 Apr 16 04:08:31 volume3 kernel: nvme1: resubmitting queued i/o Apr 16 04:08:31 volume3 kernel: nvme1: READ sqid:2 cid:0 nsid:1 lba:2561980= 904 len:8 Apr 16 04:08:31 volume3 kernel: nvme1: resubmitting queued i/o Apr 16 04:08:31 volume3 kernel: nvme1: READ sqid:2 cid:0 nsid:1 lba:1987706= 424 len:8 Apr 16 04:08:31 volume3 kernel: nvme1: resubmitting queued i/o Apr 16 04:08:31 volume3 kernel: nvme1: READ sqid:2 cid:0 nsid:1 lba:1154816= 96 len:8 Apr 16 04:08:31 volume3 kernel: nvme1: resubmitting queued i/o Apr 16 04:08:31 volume3 kernel: nvme1: READ sqid:2 cid:0 nsid:1 lba:1095204= 568 len:8 Apr 16 04:08:31 volume3 kernel: nvme1: resubmitting queued i/o Apr 16 04:08:31 volume3 kernel: nvme1: READ sqid:2 cid:0 nsid:1 lba:9482113= 34 len:12 No freezing or any other symptoms yet. This machine runs the patched kernel= but doesn't have a "hw.nvme.timeout_period" setting configured in its /boot/loader.conf . Will provide feedback on new developments. Regards, Ruben(In reply to Colin Percival from comment #49) (In reply to Colin Percival from comment #49) --=20 You are receiving this mail because: You are the assignee for the bug.=