From owner-freebsd-current@freebsd.org Tue Dec 1 11:51:26 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 48D304A640A for ; Tue, 1 Dec 2020 11:51:26 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "CN", Issuer "CN" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ClgRd1j0sz3PLK for ; Tue, 1 Dec 2020 11:51:24 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (mh0.gentlemail.de [78.138.80.135]) by mx0.gentlemail.de (8.15.2/8.15.2) with ESMTP id 0B1BpLft029869; Tue, 1 Dec 2020 12:51:22 +0100 (CET) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id A650D7CC; Tue, 1 Dec 2020 12:51:21 +0100 (CET) Subject: Re: rc.d/zpool runs before ada(4) attaches To: Ronald Klop , FreeBSD Current References: <1439301337.11.1606815206810@localhost> From: Harry Schmalzbauer Organization: OmniLAN Message-ID: <08815f92-742c-2934-e746-fd04ca9b4e16@omnilan.de> Date: Tue, 1 Dec 2020 12:51:21 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <1439301337.11.1606815206810@localhost> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Greylist: inspected by milter-greylist-4.6.2 (mx0.gentlemail.de [78.138.80.130]); Tue, 01 Dec 2020 12:51:22 +0100 (CET) for IP:'78.138.80.135' DOMAIN:'mh0.gentlemail.de' HELO:'mh0.gentlemail.de' FROM:'freebsd@omnilan.de' RCPT:'' X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx0.gentlemail.de [78.138.80.130]); Tue, 01 Dec 2020 12:51:22 +0100 (CET) X-Rspamd-Queue-Id: 4ClgRd1j0sz3PLK X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.30 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:e10:2800::a130:from]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[omnilan.de]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; HAS_ORG_HEADER(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; SPAMHAUS_ZRD(0.00)[2a00:e10:2800::a130:from:127.0.2.255]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:61157, ipnet:2a00:e10:2800::/38, country:DE]; RCVD_TLS_LAST(0.00)[]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Dec 2020 11:51:26 -0000 Am 01.12.2020 um 10:33 schrieb Ronald Klop: : : : >> One machine fails importing zpool because the correponding vdevs >> (ada0-ada2) >> are not available at the time rc.d/zpool runs. >> >> >> Adhoc  I'm not aware of any rc(8) vs. driver awareness. >> Is there any? >> >> Suggestions how to fix else than 'sleep 1'? >> >> Thanks, >> >> -harry >> >> P.S.: ahci(4) is compiled into kernel, machine is a HPE U48 (Gen 10 >> plus MicroServer), zfsloader loads root_MFS kernel module >> > > > There have been some changes to etc/rc.d/zpool in September. > Do you have the latest version? Compare with: > https://github.com/freebsd/freebsd/blob/master/libexec/rc/rc.d/zpool > or > https://svnweb.freebsd.org/base/head/libexec/rc/rc.d/zpool?revision=365354&view=markup > > > Otherwise it would be helpful for readers if you could post some logs > which indicate what is happening. > /var/run/dmesg.boot or the output of "dmesg" > Part of /var/log/messages > Part of /var/log/console.log if it exists. > Thanks, I'm on -current from view days ago. The problem is that cam(4) is still probing devices, when rc.d/zpool runs, since mount_root_from succeeded, because it is a RAM disk, so succeeds independent of any real drive/controller probing. I can imagine of other seldom edgecases hitting the issue too. So my proposed patch, working for me, looks like this: Index: libexec/rc/rc.d/zpool =================================================================== --- libexec/rc/rc.d/zpool       (revision 368202) +++ libexec/rc/rc.d/zpool       (working copy) @@ -18,8 +18,16 @@  zpool_start()  { -       local cachefile +        local cachefile n=0 camlist=`camcontrol devlist -v` +       # Wait for cam(4) devices attaching, 4 times at max by increasing +       # 1s each (10s max in total) +        while [ X"${camlist#*target*lun*probe}" != X"${camlist}" ]; do +               [ $n -lt 4 ] || break +               sleep $((n+=1)) +               camlist=`camcontrol devlist -v` +       done +         for cachefile in /etc/zfs/zpool.cache /boot/zfs/zpool.cache; do                 if [ -r $cachefile ]; then                         zpool import -c $cachefile -a -N && break best, -harry