From nobody Fri Oct 8 20:42:01 2021 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id B7E7E17E52D5 for ; Fri, 8 Oct 2021 20:42:20 +0000 (UTC) (envelope-from timp87@gmail.com) Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [IPv6:2a00:1450:4864:20::52a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HR0Vg5cJ4z3rFB for ; Fri, 8 Oct 2021 20:42:19 +0000 (UTC) (envelope-from timp87@gmail.com) Received: by mail-ed1-x52a.google.com with SMTP id d3so13311556edp.3 for ; Fri, 08 Oct 2021 13:42:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=rrLaITBLrYUsJhB2IksC6p8ZQVfheM5uihcXVhnQoJU=; b=SmKjC7SjeXp18Nr4Sii8BQVzf7G4x/u19sra5A2rlVoUbFlpMedzB/6GzeNqba+YIB Qs5t0d9jj6sS/5bY44Rz6ebi2FqpF+AnX7mhkdMrnyrmlK2O2emUoSA+scA42xQapQFN LvAgrF74Sdq5w2V9UXy0ICbBhhJlbCy8ZiLNY4QThKJ6TC2ijsyU7DoMQk8LpCKo2Gsi oWXo/Lb9IrsPEfy9xt7RCouo6FhDf+M+JpI9Y4q6kxLBcSPp2FehTuBFKJ32LTwlyTOt w0omfPci5hUixjjFmSIlAMQgj1xvulwr68N8DlgjxC1xNv97WAoBwWWwf47GBdVOetD3 yWmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=rrLaITBLrYUsJhB2IksC6p8ZQVfheM5uihcXVhnQoJU=; b=hhrpTkHOnHa+1++T57/pQhMotpWESCN4hCXbYxLw8XnFSr65TpM6o2JDMdV9JBhvHH wU47dlB9FQYmu2U9qGcDmQMnG1vtqKxkG64ghBsjrGq+1DJslpAHm6gMLG9eLAnDioVI WJ4L64kImrvU6ZawSgJefR4g4yKZk+JQoFeT3ZvInnaVn4y1GCgGETZG/J6sveoBtQEJ vTEdkN9M0/VwC3OU5NftVIIaeSjzjzgC7v0Lyu16ryDpdrokiigOvJ0Lop6T4A80zHCp wvUAAp9EwPHsRVFKWby1EgkiVcHQNKFZuWjcfkZRNR9oXHkjjwqlMJ8Kf2M/Tz/nGYnQ Tq1w== X-Gm-Message-State: AOAM530vRXCadb6zWCSA1lTz4zqjzbjVJqqb8RH/oU9ogPZ9LcBm8QBN sQJa4CSTSLwgm47Mffle+wMwsgvg9eTRBGB1NMV1Wvj+wdY= X-Google-Smtp-Source: ABdhPJyBiu67+jMdjE4kwzFmtPM//ezqOtL6ya4kHqtrtWPHGYH45I3MbPf40BY0SXG39Yuhz7qpfuG1V0gAtncMIao= X-Received: by 2002:a05:6402:35c4:: with SMTP id z4mr17841637edc.197.1633725732804; Fri, 08 Oct 2021 13:42:12 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Pavel Timofeev Date: Fri, 8 Oct 2021 14:42:01 -0600 Message-ID: Subject: Re: Dell Latitude 7400 - nvme0: Missing interrupt To: Warner Losh Cc: Chuck Tuffli , freebsd-current Content-Type: multipart/alternative; boundary="000000000000f768a705cddd694e" X-Rspamd-Queue-Id: 4HR0Vg5cJ4z3rFB X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=SmKjC7Sj; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of timp87@gmail.com designates 2a00:1450:4864:20::52a as permitted sender) smtp.mailfrom=timp87@gmail.com X-Spamd-Result: default: False [-3.23 / 15.00]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; NEURAL_HAM_MEDIUM(-0.23)[-0.229]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MID_RHS_MATCH_FROMTLD(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::52a:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_TLS_ALL(0.00)[]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org] X-ThisMailContainsUnwantedMimeParts: Y --000000000000f768a705cddd694e Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable =D1=81=D0=B1, 21 =D0=B0=D0=B2=D0=B3. 2021 =D0=B3. =D0=B2 15:22, Warner Losh= : > > > On Sat, Aug 21, 2021 at 3:06 PM Pavel Timofeev wrote: > >> >> >> Warner Losh : >> >>> >>> >>> On Fri, Aug 20, 2021 at 10:42 PM Pavel Timofeev >>> wrote: >>> >>>> Pavel Timofeev : >>>> >>>> > >>>> > Chuck Tuffli : >>>> > >>>> >> On Mon, Aug 16, 2021 at 7:43 PM Pavel Timofeev >>>> wrote: >>>> >> > >>>> >> > Hello >>>> >> > I've got a Dell Latitude 7400 and tried installing the latest >>>> >> 14.0-CURRENT >>>> >> > (main-n248636-d20e9e02db3) on it. >>>> >> > Despite other things the weird one which concerns me is >>>> >> > nvme0: Missing interrupt >>>> >> > message I get sometimes on the console. >>>> >> > It seems like I get it only after the reboot of the laptop, i. e. >>>> not >>>> >> > getting that message if I power cycle the laptop, at least I >>>> haven't >>>> >> seen >>>> >> > them for now in such cases. >>>> >> > So when the laptop is rebooted I can't even take advantage of >>>> >> > nvmecontrol(8) quickly. >>>> >> > Well, it still works, but it takes tens of seconds to return the >>>> output. >>>> >> ... >>>> >> > dmesg when power cycled - >>>> >> > https://drive.google.com/file/d/1dB27oB1O2CcnZy6DvOOhmFO8SN8V8SwJ >>>> >> > dmesg when rebooted - >>>> >> > https://drive.google.com/file/d/1DsKTMkihp_OmUcirByLaVO4o2mU38Bxh >>>> >> >>>> >> I'm sort of curious about the time stamps for the log messages in t= he >>>> >> failing case. Something like: >>>> >> >>>> >> $ grep "nv\(me\|d\)" /var/log/messages >>>> >> >>>> >> --chuck >>>> >> >>>> > >>>> > Well, I can't see timestamps in the verbose boot log. Am I missing >>>> some >>>> > configuration for that? >>>> > >>>> > $ grep "nv\(me\|d\)" /var/log/messages >>>> > nvme0: mem >>>> > 0xcc100000-0xcc103fff,0xcc105000-0xcc105fff,0xcc104000-0xcc104fff at >>>> device >>>> > 0.0 on pci6 >>>> > nvme0: attempting to allocate 5 MSI-X vectors (17 supported) >>>> > nvme0: using IRQs 133-137 for MSI-X >>>> > nvme0: CapLo: 0x140103ff: MQES 1023, CQR, TO 20 >>>> > nvme0: CapHi: 0x00000030: DSTRD 0, NSSRS, CSS 1, MPSMIN 0, MPSMAX 0 >>>> > nvme0: Version: 0x00010300: 1.3 >>>> > nvme0: Missing interrupt >>>> > nvme0: Missing interrupt >>>> > nvme0: Missing interrupt >>>> > nvme0: Missing interrupt >>>> > nvme0: Missing interrupt >>>> > nvme0: Missing interrupt >>>> > nvme0: Missing interrupt >>>> > nvme0: Missing interrupt >>>> > nvme0: Missing interrupt >>>> > nvme0: Missing interrupt >>>> > nvme0: Missing interrupt >>>> > nvme0: Missing interrupt >>>> > nvd0: NVMe namespace >>>> > GEOM: new disk nvd0 >>>> > nvd0: 488386MB (1000215216 512 byte sectors) >>>> > >>>> >>>> >>>> Ah, sorry, provided wrong output. >>>> Here is what you requested: >>>> $ grep "nv\(me\|d\)" /var/log/messages >>>> Aug 21 04:34:36 nostromo kernel: nvme0: mem >>>> 0xcc100000-0xcc103fff,0xcc105000-0xcc105fff,0xcc104000-0xcc104fff at >>>> device >>>> 0.0 on pci6 >>>> Aug 21 04:34:36 nostromo kernel: nvme0: attempting to allocate 5 MSI-X >>>> vectors (17 supported) >>>> Aug 21 04:34:36 nostromo kernel: nvme0: using IRQs 133-137 for MSI-X >>>> Aug 21 04:34:36 nostromo kernel: nvme0: CapLo: 0x140103ff: MQES 1023, >>>> CQR, >>>> TO 20 >>>> Aug 21 04:34:36 nostromo kernel: nvme0: CapHi: 0x00000030: DSTRD 0, >>>> NSSRS, >>>> CSS 1, MPSMIN 0, MPSMAX 0 >>>> Aug 21 04:34:36 nostromo kernel: nvme0: Version: 0x00010300: 1.3 >>>> Aug 21 04:34:36 nostromo kernel: nvme0: Missing interrupt >>>> Aug 21 04:34:36 nostromo kernel: nvme0: Missing interrupt >>>> Aug 21 04:34:36 nostromo kernel: nvme0: Missing interrupt >>>> Aug 21 04:34:36 nostromo kernel: nvd0: NVM= e >>>> namespace >>>> Aug 21 04:34:36 nostromo kernel: GEOM: new disk nvd0 >>>> Aug 21 04:34:36 nostromo kernel: nvd0: 488386MB (1000215216 512 byte >>>> sectors) >>>> Aug 21 04:34:42 nostromo kernel: nvme0: Missing interrupt >>>> Aug 21 04:35:36 nostromo kernel: nvme0: Missing interrupt >>>> Aug 21 04:35:50 nostromo kernel: nvme0: Missing interrupt >>>> >>> >>> What happens if you set hw.nvme.use_nvd=3D0 and hw.cam.nda.nvd_compat= =3D1 >>> in the boot loader and reboot? Same thing except nda where nvd was? Or >>> does >>> it work? >>> >>> Something weird is going on in the interrupt assignment, I think, but I >>> wanted to get any nvd vs nda issues out of the way first. >>> >>> Warner >>> >> >> Do you mean kern.cam.nda.nvd_compat instead of hw.cam.nda.nvd_compat? >> kern.cam.nda.nvd_compat is 1 by default now. >> >> So I tried to set hw.nvme.use_nvd to 1 as suggested, but I still see >> nvme0: Missing interrupt >> and now also >> Root mount waiting for: CAM >> messages besides those >> > > OK. That all makes sense. I'd forgotten that nvd_compat=3D1 by default th= ese > days. > > I'll take a look on monday starting at the differences in interrupt > assignment that > are apparent when you cold boot vs reboot. > > Thanks for checking... I'd hoped this was a cheap fix, but also didn't > really > expect it to be. > > Warner > > I've recently upgraded to main-n249974-17f790f49f5 and it got even worse now. So clean poweron works as before. But if rebooted nvme drive refuses to work, while before the code upgrade it was just complaining about missing interrupts. currently dmesg show this: nvme0: mem 0xcc100000-0xcc103fff,0xcc105000-0xcc105fff,0xcc104000-0xcc104fff at device 0.0 on pci6 nvd0: NVMe namespace nvd0: 488386MB (1000215216 512 byte sectors) nvme0: mem 0xcc100000-0xcc103fff,0xcc105000-0xcc105fff,0xcc104000-0xcc104fff at device 0.0 on pci6 nvme0: RECOVERY_START 9585870784 vs 9367036288 nvme0: timeout with nothing complete, resetting nvme0: Resetting controller due to a timeout. nvme0: RECOVERY_WAITING nvme0: resetting controller nvme0: aborting outstanding admin command nvme0: IDENTIFY (06) sqid:0 cid:15 nsid:0 cdw10:00000001 cdw11:00000000 nvme0: ABORTED - BY REQUEST (00/07) sqid:0 cid:15 cdw0:0 nvme0: nvme_identify_controller failed! nvme0: waiting nvme0: mem 0xcc100000-0xcc103fff,0xcc105000-0xcc105fff,0xcc104000-0xcc104fff at device 0.0 on pci6 nvme0: RECOVERY_START 9362778467 vs 9361830561 nvme0: timeout with nothing complete, resetting nvme0: Resetting controller due to a timeout. nvme0: RECOVERY_WAITING nvme0: resetting controller nvme0: aborting outstanding admin command nvme0: IDENTIFY (06) sqid:0 cid:15 nsid:0 cdw10:00000001 cdw11:00000000 nvme0: ABORTED - BY REQUEST (00/07) sqid:0 cid:15 cdw0:0 nvme0: nvme_identify_controller failed! nvme0: waiting --000000000000f768a705cddd694e--