From owner-freebsd-current@freebsd.org Sat Dec 5 17:46:35 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5A3F74A2E6A for ; Sat, 5 Dec 2020 17:46:35 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "CN", Issuer "CN" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CpH7Z2rWKz4VFL; Sat, 5 Dec 2020 17:46:34 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (mh0.gentlemail.de [IPv6:2a00:e10:2800:0:0:0:0:a135]) by mx0.gentlemail.de (8.15.2/8.15.2) with ESMTP id 0B5HkNOm065983; Sat, 5 Dec 2020 18:46:24 +0100 (CET) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 4B0761FC; Sat, 5 Dec 2020 18:46:23 +0100 (CET) Subject: Re: rc.d/zpool runs before ada(4) attaches To: Ian Lepore , Ronald Klop , FreeBSD Current References: <1439301337.11.1606815206810@localhost> <08815f92-742c-2934-e746-fd04ca9b4e16@omnilan.de> <286917313.21.1606836130991@localhost> <786faeee90e79aa0175b298ec859265ff57a3129.camel@freebsd.org> From: Harry Schmalzbauer Organization: OmniLAN Message-ID: Date: Sat, 5 Dec 2020 18:46:22 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <786faeee90e79aa0175b298ec859265ff57a3129.camel@freebsd.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Greylist: inspected by milter-greylist-4.6.2 (mx0.gentlemail.de [IPv6:2a00:e10:2800:0:0:0:0:a130]); Sat, 05 Dec 2020 18:46:25 +0100 (CET) for IP:'2a00:e10:2800::a135' DOMAIN:'mh0.gentlemail.de' HELO:'mh0.gentlemail.de' FROM:'freebsd@omnilan.de' RCPT:'' X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx0.gentlemail.de [IPv6:2a00:e10:2800:0:0:0:0:a130]); Sat, 05 Dec 2020 18:46:25 +0100 (CET) X-Rspamd-Queue-Id: 4CpH7Z2rWKz4VFL X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.27 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[omnilan.de]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:e10:2800::a130:from]; HAS_ORG_HEADER(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.97)[-0.971]; SPAMHAUS_ZRD(0.00)[2a00:e10:2800::a130:from:127.0.2.255]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:61157, ipnet:2a00:e10:2800::/38, country:DE]; RCVD_TLS_LAST(0.00)[]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Dec 2020 17:46:35 -0000 Am 01.12.2020 um 16:34 schrieb Ian Lepore:: : : : >> You can define these in /boot/loader.conf: >> #kern.cam.boot_delay="10000" # Delay (in ms) of root mount for CAM >> bus >> #kern.cam.scsi_delay="2000" # Delay (in ms) before probing SCSI >> >> Maybe that helps. >> >> Ronald. >> > Those settings control waiting before mounting root. Harry's problem > is that root is mounted quickly, before other drives are ready for zfs. > > The zpool script waits for 'disks'. It would be nice if the cam > subsystem had something like a sysctl it set to indicate when initial > probing for disks was done, then there could be an rc.d/camprobe script > with 'PROVIDE: disks' which waits for the probing to complete. > > -- Ian Until something described above is available, or anybody is aware of any other trick, here's a tested workaround: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251610 It turned out that also swapon and dumpon suffer from early root_hold_wait() release. For dumpon, cam(4) doesn't even start probing.  Luckily all target:LUNs are visible at that earliest stage in rc.d/dumpon. So that's the point where I check if any real target is in state unattached (()) or probing ((aprobe)). I don't know details of the involved vfs.root_mount_hold sysctl, but assume this is dead end currently... Best regards, -harry