From nobody Sun May 23 04:44:22 2021 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id F1D789F4B80 for ; Sun, 23 May 2021 04:44:31 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic313-21.consmr.mail.gq1.yahoo.com (sonic313-21.consmr.mail.gq1.yahoo.com [98.137.65.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FnnnC019qz4XJs for ; Sun, 23 May 2021 04:44:30 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1621745068; bh=h8lhi9VGQFjMSKJ5Ij48BLy+oq8s24/zGTS5U8Frm78=; h=From:Subject:Date:Cc:To:References:From:Subject:Reply-To; b=QQxXOyubuym9H+rTMU6H3m97AImFhIjNGF1RjKGyY6DZMw/s98t6lr3lpC+Bb6Ps0q9MuojiqQv050qaq9tcx24lqBaM7VCgDy5+Gmds7YTdruIH9nEEIEOteqAvaDeksHYimdKXjZqlSZV/MlOUicRK0D5bvYqFtihiI5IPq9S1TVcYr05T2JPZezLZnsznIMtUTqnAi3Sk9Jns1hEtDFo+TA6uW2jKEB3pxKBOHWc/+MVQPifunoptQugVZn6BVdGByzlLjVEiFAAblfFo07rV8XB3p4DjNTb2SyaZyn+4vGnHMFG+pBRApGiOcNj0a7vZ+wdL/GVwyHARR1/R1w== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1621745068; bh=01ACMmVdFqqr89Qh2xkFkgBWXK+0Aqbaz1vhYbkcF4m=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=pY/mFZLDK1XvsQY5EKv8LvcpMaUT9COiZMt2sLaowf8ZOxBDqcFjJUOuJvldf8ikWrz4kwPa7Afs7gd7xEy9BAf5qtmn01p22uDb7jD5AWRNQB9n1E7AIqAEiy8IzHpMJhQUUJTvkI+fq68WtwBwHvxPup0OjJyiLHoYB5Bw+ia1GJVZ0gFlYSh+/JFAEuepNqZsdr2Pw/nTdcjisGnK7PisWfuGLSqtNBDtF8nlBe2EVS2ZEh8ASiSzSvraT8mSa8LOzaLNyK4kIv+b0LIMNlbGhF3KMKWBFG8v9GU1ryc1MQGwbGb+HqgWvkDiE8M4SzoO0/KyEPX96WrcTTL3KQ== X-YMail-OSG: DesjV4UVM1kUlzOqr2KFNY.61tK3Rx1KxyCLHr3naUZBNEvNjVbmEDg00zTiC0x ux_iKmaLFDau0CIaudKnSrgbpZ8VDZ_m7dNQFQCmC_bbNNJrjPaxCwAoZfy4AXYf4tHjz6sMbeci TAvbJztRh7P3CY_Wl1VFn27hosBkXd8AR3DdMIneMN4eUh_tGAAAXwE0PHCYP8VJHVu.FmPfSOvn Zpmd7plLXUpo57qoeNQlD50RVQ8HtPVRbMn_U48U74laC6D.RiTWvCuZUzDqhkKIw_AjJxH73HZJ DOMUqqYbVoncNECe0r.yzAhWgb4RjuhOmyVdhQ.CQywUBpXMIScKEk5vE6K_FI2gEIO3gPVyeOHo 65sN.ZzvKRDU5js2Ux53_hMfkTR0X7I_.1v3oxB_OFSw8I93OvO4ia5cL6j.Tk7vVI5pjO52BVAN Yr6uiclEKdXKVS4HY2z9sY.vkhmlZLIG3XZfeAVhWdsjQu6F201MzrkLNd5f9PPnQkTYSyNr.z4Z m8bLODEqg1jC7bha65y8vZH2sBGVIICOk362B8k1d5GxIOXBPzluIYDqZKTUgN978pWptaz_8rxM zbF_wc0Ap0ZflKkjy5j5QIwFgvkMpFwqQ7Q5hG5HhiwnhpgjpJjky6._Us65.40kSPqB.zY4Honq _H8IWA5h80Tu5WMTWwEi.vXHnVIsO0Nhd683uL_J28Z00OzC1vJvAlWUZcCO3CNTiEmNovmRRC.W TQQfZZXHwMYZZT.YvXJ6YAJ_ZoIGJY5.WJX6lD.DxeUQ_k0ia.alEBwgw.Jr_1B3KNQpAKEJMgfQ y6hIIYrdFKOVBnqD83i49E0AUTB5gfi1EnSwpSkdVLoNZPw8ExfoOzM3918CYH0x8jB2.PS8KGgw R.RHVQrAJBXv6NEgCczQKgFE6wO0FK19MW7o68VOcFtcjIFFo5l5aVWgRqGaT4czzX_QhKNki0YC eu7KTtIDjTY_tEub5vGLlwrpHINwGiOVTjv8NjREnWsND2uSXRqUBUeHMuTc_ptop2aFmIvNDN6s K7tFVx5._REviDUwXAYvzYTuCh6CvZy3zGovX9siNgGBwnHLS4rrEl0Emy6La23P7xRY5Miw3iNU gWXM3Wj8i5h7iH1oZ3EFUWIdzf3BkR1Ml_n97KsR.x99EJstMcqU3Pvbmn4S64HRB6eern.78gQ4 qmQQzdHltTSJ6UA9YCwWUEFzzI1qI_jDIPqtPNo.8PVERs96oeBqIb2uf5y_dScjyU3GFTIaB2Yn vnTm9oVNWi1tk.WcJjjyDwibWaJXQ6FOfTHgcu3I7G72nUqBTcjmuWZgNFS3Wp9Zsvpp6mONmklr BbSCP_UpxF5Mk35h9z3xNb6KZA3X1nx5gwr.ot8dhShSJiqT0e.7BmJMpYvuWQ8_7Qcyxto.0vXf IRKJf9O8a9c0fAx_t7qQzUXNSKioO2rfoCsRORqcs0.1W72Ypmi1gmQ1gdfzZZhY9XDijxBt9IWn hfGfhM6u5vUkR3ZDg6w.QHPO7wTjhI.ii1TdMjjuxWp1CJgoTDKGLsrqjadlz4jtdVCI12fSWt1y Nt4qD7joDpzjNr2o6fQSwWI9s9E6_seDmTzN8YSAF_CMfIAgpLOTm0n.XJpo0AxlHyKXIjLUdPNq JEgIXJ_.UbkfTAoOtwDw_c5HhgcD2bOBQT2HVOvvg2aGTEstgxoU5MWAy82M5d2HjTFHjD0yEgmo HbJ8c83EKTpRHSiT_PUEekHZO2c5Jri9hyfynaPaeXITUXlYEEvrtlLMj8..uVw22U9fi2GaUOcM aaOuJIn30NY4KLiqz8Kk9WSD80KxppzzG9DO6ijhgmxtoeKCXgrDv.J3Y97q46O75WfWTGB_.ezr 0eWZrZDrbfZcljs9mjgSPfUrS5MNIP3VpobcYRru2LoMcAv3VPuW50R5Upg6cYTR3lvm.tKNOtUU X4pw5SaD31apmEDOcVkD0X9CgFdfLVyOMdKMSWbWG0zxui96kLUq3DgzDNnvGtk3bqFeFiht7q2a 7Xx3PWsri77Qq_GNxlY0RGWdFm57uaOyyH_AkfIu.juDputhDseFedi2wBtFolHWOeUl_Pj7eog9 wyZU.yNMM41cbTztzkXuB1djQ2P7VpjyZN0NtK47VO_i.XZT0MeWlFWrwV821CtJtKIc855oVDIp SGBrkAXL2RjAf9h9ksWEWdCueB2AVhf1fMQhE5v_4uojVzoWP4VStsE8iOGvN5Y9BeQNbJTdNCIs 7VwJ.EXR2FtBTf4fR1rbEwS1C8xWjcTD3UClD6yebrGziuQ_gvyvTXt5uIkHSeNHgj8c2vsDS7TV mtF.1hjIt8_BnP95KO3uZ6oFqH66m4KkawCZpxqqe_rLUNzUI_X0PMjNz5FIATyKI.WuFH0Sgfy6 OCnnOjfKBZcQUS15OgW1WJMFv6TfSY1EVnA4Z4FslPfauWkasOmgDS2SHcApX3n7gE7dEZKmwX0o 6itUj4gAE6p35DTasn.evBv2FJ6ftwtlHX1kOVj9BFEdBw15LJaqNI8hG5WL8jaBHQKnLg9J7RNh vovq7mS4T5nMY4FGQBFtRc9VodqatQY9tHBykPSOHDdc6otlDqqAuVwua5zj42A-- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic313.consmr.mail.gq1.yahoo.com with HTTP; Sun, 23 May 2021 04:44:28 +0000 Received: by kubenode524.mail-prod1.omega.ne1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 7c16d8f57e0774362519e50201b7eb51; Sun, 23 May 2021 04:44:23 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.80.0.2.43\)) Subject: I got a panic for "nvme0: cpl does not map to outstanding cmd" on a MACHIATObin Double Shot Message-Id: <063D5E36-126F-497C-97AF-827BADC1ED2F@yahoo.com> Date: Sat, 22 May 2021 21:44:22 -0700 Cc: freebsd-arm To: freebsd-current X-Mailer: Apple Mail (2.3654.80.0.2.43) References: <063D5E36-126F-497C-97AF-827BADC1ED2F.ref@yahoo.com> X-Rspamd-Queue-Id: 4FnnnC019qz4XJs X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=QQxXOyub; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.84 as permitted sender) smtp.mailfrom=marklmi@yahoo.com X-Spamd-Result: default: False [-3.50 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.65.84:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[98.137.65.84:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.84:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.84:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-current] Reply-To: marklmi@yahoo.com From: Mark Millard via freebsd-arm X-Original-From: Mark Millard # mount -onoatime 192.168.1.187:/usr/ports/ /mnt/ # diff -r /usr/ports/ /mnt/ | more nvme0: cpl does not map to outstanding cmd cdw0:00000000 sqhd:0020 sqid:0003 cid:007e p:1 sc:00 sct:0 m:0 dnr:0 panic: received completion for unknown cmd cpuid =3D 3 time =3D 1621743752 KDB: stack backtrace: db_trace_self() at db_trace_self db_trace_self_wrapper() at db_trace_self_wrapper+0x30 vpanic() at vpanic+0x188 panic() at panic+0x44 nvme_qpair_process_completions() at nvme_qpair_process_completions+0x1fc nvme_timeout() at nvme_timeout+0x3c softclock_call_cc() at softclock_call_cc+0x124 softclock() at softclock+0x60 ithread_loop() at ithread_loop+0x2a8 fork_exit() at fork_exit+0x74 fork_trampoline() at fork_trampoline+0x14 KDB: enter: panic [ thread pid 12 tid 100028 ] Stopped at kdb_enter+0x48: undefined f904411f db>=20 Based on the "nvme" references, I expect this is tied to handling the Optane 480 GiByte that is in the PCIe slot and is the boot/only media for the machine doing the diff. "db> dump" seems to have worked. After the reboot, zpool scrub found no errors. For reference: # uname -apKU FreeBSD CA72_16Gp_ZFS 14.0-CURRENT FreeBSD 14.0-CURRENT #1 = main-n246854-03b0505b8fe8-dirty: Sat May 22 16:25:04 PDT 2021 = root@CA72_16Gp_ZFS:/usr/obj/BUILDs/main-CA72-dbg-clang/usr/main-src/arm64.= aarch64/sys/GENERIC-DBG-CA72 arm64 aarch64 1400013 1400013 =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)