From nobody Fri Apr 1 05:09:45 2022 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A71221A52877 for ; Fri, 1 Apr 2022 05:09:48 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-vk1-xa2f.google.com (mail-vk1-xa2f.google.com [IPv6:2607:f8b0:4864:20::a2f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4KV7Ww03X8z3pKG for ; Fri, 1 Apr 2022 05:09:48 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-vk1-xa2f.google.com with SMTP id d129so903916vkh.13 for ; Thu, 31 Mar 2022 22:09:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20210112.gappssmtp.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=uneaOpu9b9XzN12Xpq+EPkU5q7mHfvBcqHUPggARom8=; b=1Vvu4thJqp/CHlxOpmuOv4ePfA7bYnbikUsGnX3zhh1kHKcPBqQenzBruDTl8ICnGb /XyS9/QgBoi07+X2fFwu+J9c6/eq2ld/DONtDAdfpfaogTlEMJPVn6vYHK9UvhfHBQys 1/mhhrBtaJLinz0eXo8XJ2YIegdyNMB6SuSBGtN1xwvZUYVvzSfEkDFQWs08u+j8L0YS Wih5uM2v3V5YdF/GkxwG9DLUd/s31IFZWO9w3s16aX8NTLkTh9nMKBWShSvZ/DlKvmEm HhI6W037Jke2n5ipeWjRKAGZw1UAISAHNJAQvccbp+DVYO10ekgYAVyFsFZ3q9N/Iau6 Fj0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=uneaOpu9b9XzN12Xpq+EPkU5q7mHfvBcqHUPggARom8=; b=wzg5Noa4uGsWQB+Lugb7db9j6ZS33ce00SDPpK5urA7hp3eMud5hyqZcfW3f5dE/UA tgbQR8mCHathwESO6aM5pR9yFfYvnDi08tMxpOvIMD7TVFS7d+Cm3qhADd3zxogMnJHV pndCxdYhO4faw6MYXmfHz0nFREsvfAitcV6kRyg+9pmS9cPr0c0lrdYcuRxmHtCx4cDt TavR2wpqLsDUnEHImvx6yCrtcVcGQuS3UcYIfCj+jT3FoM29fROUfFj2RhB3H9DY0w2l IV820nP4DL7wXVnfLlxlfVDdt0GZZNqOe0SoCFwSo2srecwplVUK/eydqHwqqM3/Fozs L8DA== X-Gm-Message-State: AOAM531EOcM00C+FD/RMiVbgSZ/xGbtIRLLBw1pDAdFsueSDAIzbg7Yf VvKlLLzo/fsG6gbSGjBYevWsHMtrLuByUMgR+sJnDF+rFr9IuA== X-Google-Smtp-Source: ABdhPJwsjv8vSvWoNf3qF6vNik9kINwzGOCkZtS++CjykE28sVRsGi5hKMHJ8HRMdnKryNGp5g62q+f6v0Qsj7cu9Eg= X-Received: by 2002:a05:6122:508:b0:342:e9cd:3177 with SMTP id x8-20020a056122050800b00342e9cd3177mr3641313vko.40.1648789787341; Thu, 31 Mar 2022 22:09:47 -0700 (PDT) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Thu, 31 Mar 2022 23:09:45 -0600 Message-ID: Subject: Re: CAM timeouts on boot (13.0-p8, Dell R730xd, mrsas controller) To: George Michaelson Cc: FreeBSD Stable Content-Type: multipart/alternative; boundary="0000000000009588ac05db90c982" X-Rspamd-Queue-Id: 4KV7Ww03X8z3pKG X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20210112.gappssmtp.com header.s=20210112 header.b=1Vvu4thJ; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::a2f) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-2.49 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20210112.gappssmtp.com:s=20210112]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20210112.gappssmtp.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::a2f:from]; NEURAL_HAM_SHORT(-0.49)[-0.495]; MLMMJ_DEST(0.00)[freebsd-stable]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; R_SPF_NA(0.00)[no SPF record]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-ThisMailContainsUnwantedMimeParts: N --0000000000009588ac05db90c982 Content-Type: text/plain; charset="UTF-8" On Thu, Mar 31, 2022 at 9:40 PM George Michaelson wrote: > I checked /boot/zfs/zpool.cache and /etc/zfs/zpool.cache both have the > tank in question, zdb agrees, they are identical, multiple reboots > re-tested: without a manual zpool import tank, only the zfs zroot > which is magically mounted appears. The other tank does not unless you > call it into being. Doing that after multiuser/daemon is bad if your > daemons work in it. (and bad for NFS exports) > > The zfs pages are a bit cryptic about altroot and other zpool get all > values, but the basics seem to be that if you do zpool import, and its > in cache, then it should be detected during zfs/zpool activation. > People expect that once you import, it will re-import automatically > thereafter unless you tell it not to. I checked zpool values on a > system which doesnt have the problem and I don't see any variance for > zpool params. zpool.cache was the recommended "have you checked" > reference. > > Not that this can't be PBCAK, but I have tried not to be the root > cause here. It happened across my upgrade, and the CAM timeout > intruded in the same window. > I'd expect this to work, though there was a change in this file's location between pre-openzfs and now. But if you are using 12.x, then that's not going to be the case... dmesg is interesting, but not completely helpful... Do can you share your zfs config details (what disks are in what datasets?) Warner > G > > On Fri, Apr 1, 2022 at 1:31 PM George Michaelson wrote: > > > > Thanks for the cluestick. I don't disagree with anything you said btw. > > > > Here's the dmesg. (attachment, can in-line if thats better) > --0000000000009588ac05db90c982 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Thu, Mar 31, 2022 at 9:40 PM Georg= e Michaelson <ggm@algebras.org&g= t; wrote:
I chec= ked /boot/zfs/zpool.cache and /etc/zfs/zpool.cache both have the
tank in question, zdb agrees, they are identical, multiple reboots
re-tested: without a manual zpool import tank, only the zfs zroot
which is magically mounted appears. The other tank does not unless you
call it into being. Doing that after multiuser/daemon is bad if your
daemons work in it. (and bad for NFS exports)

The zfs pages are a bit cryptic about altroot and other zpool get all
values, but the basics seem to be that if you do zpool import, and its
in cache, then it should be detected during zfs/zpool activation.
People expect that once you import, it will re-import automatically
thereafter unless you tell it not to. I checked zpool values on a
system which doesnt have the problem and I don't see any variance for zpool params.=C2=A0 zpool.cache was the recommended "have you checked&= quot;
reference.

Not that this can't be PBCAK, but I have tried not to be the root
cause here. It happened across my upgrade, and the CAM timeout
intruded in the same window.

I'd ex= pect this to work, though there was a change in this file's location be= tween
pre-openzfs and now. But if you are using 12.x, then that&#= 39;s not going to be the case...

dmesg is interest= ing, but not completely helpful... Do can you share your zfs config
details (what disks are in what datasets?)

Warn= er
=C2=A0
G

On Fri, Apr 1, 2022 at 1:31 PM George Michaelson <ggm@algebras.org> wrote:
>
> Thanks for the cluestick. I don't disagree with anything you said = btw.
>
> Here's the dmesg.=C2=A0 (attachment, can in-line if thats better)<= br>
--0000000000009588ac05db90c982--