From nobody Thu Sep 5 09:40:24 2024 X-Original-To: freebsd-questions@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WzvVN4M14z5W4s3 for ; Thu, 05 Sep 2024 09:40:28 +0000 (UTC) (envelope-from ludovit.koren@gmail.com) Received: from mail-lj1-x236.google.com (mail-lj1-x236.google.com [IPv6:2a00:1450:4864:20::236]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WzvVM54cDz4sWv for ; Thu, 5 Sep 2024 09:40:27 +0000 (UTC) (envelope-from ludovit.koren@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=XTcit8NE; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of ludovit.koren@gmail.com designates 2a00:1450:4864:20::236 as permitted sender) smtp.mailfrom=ludovit.koren@gmail.com Received: by mail-lj1-x236.google.com with SMTP id 38308e7fff4ca-2f51b67e16dso6164831fa.3 for ; Thu, 05 Sep 2024 02:40:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725529226; x=1726134026; darn=freebsd.org; h=mime-version:user-agent:message-id:date:user-mail-address:subject :to:from:from:to:cc:subject:date:message-id:reply-to; bh=t4KYyh47mdyh/9skS1eV2ICCTaV0f/BrT2Z2wFutfks=; b=XTcit8NEcTz+aSPksfs+jku40gSVwQq7iCsF3gEF0sHDuFD5g3FLEw3eQ6vB2jSwp7 h0f+rBQJy7bwvuqlTc+Md/NiFkVrEF65rEyVNaSPy8tzqu6d4RjFC6a/z7KLc3ZeSqKM enIBUKYrQ34kq+KILOKu9Rh+6Ke9LoQo2LNuvQozaa40e2s8/0McRi+VOc2bsNecoNZ/ mlEvcp+f0PM2N9WTnR5ltUxu6VzEpq3afOlMYkus/+aUQDKCDI6buGr+oYvwwyOedxXp gbuqfBZo4RC4UjH2V6kDzwOOK9lSzo2OQ3Z3d5e3GCrQd83sxxTEpRA55L7BZDexwrUH WCAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725529226; x=1726134026; h=mime-version:user-agent:message-id:date:user-mail-address:subject :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=t4KYyh47mdyh/9skS1eV2ICCTaV0f/BrT2Z2wFutfks=; b=iplmd+Od6N70nx/V8Uwlyh3geKL8dX5L6PeAgKapvtTtF/MUtwTVXHijoH/zLx6v8O LMgF6LoKcZPJqg9NRohQYK9BW5F9b5LSsuvltIPav3pQjultyGk52xm2/XY4yrutY/iO 7cdb5rWXPPRHVoX3b7ZrBJByeOiZFygmIgE2btDd0L+q06xrXKOTfjNf121Vu7rEk8tk j69il7jriLScKSE5Cw0WP6LtUr9goR+JjMpjs94pD3S9stgfd79lNay7i5umB5d0+JxG hXJmvow6op+f7H+jidlio6EwV3ocbIEUSBwV0BfHwQRuA8ivGzP7QSP6gKQNans5ncTT hPqw== X-Gm-Message-State: AOJu0Yy4yKRxFLFokxPc6CCUoRSn7d4XsbNBUZ5oaBD0xIOjLTr2sJ+R uHxsI4PIAjCwOfLL+E2gZBdYUfB06Yh/GHTB55wZMsBP4CTFfnvOzWluBw== X-Google-Smtp-Source: AGHT+IFnJDxrj1hfrjtwLCmVZeDNHkUB2mzbcE3P78cEfcRSItsvZ6G1aWxyJ/HRH5lOnNkQuhfV/Q== X-Received: by 2002:a2e:719:0:b0:2f3:f2b6:6ccb with SMTP id 38308e7fff4ca-2f64d4aaffcmr43977311fa.26.1725529225620; Thu, 05 Sep 2024 02:40:25 -0700 (PDT) Received: from jedi.localdomain ([195.28.109.130]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c3cc56c501sm1007534a12.52.2024.09.05.02.40.24 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 05 Sep 2024 02:40:25 -0700 (PDT) Received: by jedi.localdomain (Postfix, from userid 1001) id 334DFE0236; Thu, 05 Sep 2024 11:40:24 +0200 (CEST) From: Ludovit Koren To: freebsd-questions@freebsd.org Subject: ZFS boot devices specification User-Mail-Address: ludovit.koren@gmail.com Date: Thu, 05 Sep 2024 11:40:24 +0200 Message-ID: <86y146zbef.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (berkeley-unix) List-Id: User questions List-Archive: https://lists.freebsd.org/archives/freebsd-questions List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-questions@freebsd.org Sender: owner-freebsd-questions@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain X-Spamd-Bar: --- X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; FREEMAIL_FROM(0.00)[gmail.com]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; PREVIOUSLY_DELIVERED(0.00)[freebsd-questions@freebsd.org]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; TAGGED_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-questions@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::236:from] X-Rspamd-Queue-Id: 4WzvVM54cDz4sWv Hi, I cannot find the recommended way to specify the devices to boot from on 13.4-STABLE FreeBSD 13.4-STABLE stable/13-n258224-f702110bc4bc. I am using internal disks for the system and external for data: # gmultipath list Geom name: DellEMC Type: AUTOMATIC Mode: Active/Active UUID: b1d143d0-577d-11ee-b35f-b47af1b52de4 State: OPTIMAL Providers: 1. Name: multipath/DellEMC Mediasize: 536870911488 (500G) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r1w1e1 State: OPTIMAL Consumers: 1. Name: da3 Mediasize: 536870912000 (500G) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r2w2e2 State: ACTIVE 2. Name: da4 Mediasize: 536870912000 (500G) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r2w2e2 State: ACTIVE 3. Name: da5 Mediasize: 536870912000 (500G) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r2w2e2 State: ACTIVE 4. Name: da6 Mediasize: 536870912000 (500G) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r2w2e2 State: ACTIVE # zpool status pool: DellEMC state: ONLINE config: NAME STATE READ WRITE CKSUM DellEMC ONLINE 0 0 0 multipath/DellEMC ONLINE 0 0 0 errors: No known data errors pool: zroot state: ONLINE config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gpt/zfs0 ONLINE 0 0 0 gpt/zfs1 ONLINE 0 0 0 errors: No known data errors isp0: mem 0xe2a05000-0xe2a05fff,0xe2a02000-0xe2a03fff,0xe2900000-0xe29fffff at device 0.0 numa-domain 0 on pci4 isp0: Primary aux image not addressed isp0: Secondary aux image not addressed isp1: mem 0xe2a04000-0xe2a04fff,0xe2a00000-0xe2a01fff,0xe2800000-0xe28fffff at device 0.1 numa-domain 0 on pci4 isp1: Primary aux image not addressed isp1: Secondary aux image not addressed da3 at isp0 bus 0 scbus2 target 0 lun a da3: Fixed Direct Access SPC-4 SCSI device da3: Serial Number 6BL4L83 da3: 1600.000MB/s transfers WWNN 0x58ccf090cd2001ad WWPN 0x58ccf0904d2101ad PortID 0x190800 da3: Command Queueing enabled da3: 512000MB (1048576000 512 byte sectors) da4 at isp0 bus 0 scbus2 target 1 lun a da4: Fixed Direct Access SPC-4 SCSI device da4: Serial Number 6BL4L83 da4: 1600.000MB/s transfers WWNN 0x58ccf090cd2001ad WWPN 0x58ccf0984d2101ad PortID 0x190900 da4: Command Queueing enabled da4: 512000MB (1048576000 512 byte sectors) GEOM_MULTIPATH: DellEMC created GEOM_MULTIPATH: da3 added to DellEMC GEOM_MULTIPATH: da4 added to DellEMC da5 at isp1 bus 0 scbus3 target 0 lun a da5: Fixed Direct Access SPC-4 SCSI device da5: Serial Number 6BL4L83 da5: 1600.000MB/s transfers WWNN 0x58ccf090cd2001ad WWPN 0x58ccf0904d2001ad PortID 0xf0800 da5: Command Queueing enabled da5: 512000MB (1048576000 512 byte sectors) da6 at isp1 bus 0 scbus3 target 1 lun a da6: Fixed Direct Access SPC-4 SCSI device da6: Serial Number 6BL4L83 da6: 1600.000MB/s transfers WWNN 0x58ccf090cd2001ad WWPN 0x58ccf0984d2001ad PortID 0xf0900 da6: Command Queueing enabled da6: 512000MB (1048576000 512 byte sectors) The problem occurs when the disks from data array are assigned to /dev/da0 and /dev/da1. The booting process starts and finishes on remounting root RW, because it cannot find root file system. It depends on the order of hardware initialization. What is the recommended way of specifying internal disks in ZFS, something like disks uid? The mount process tries to mount ZFS gpt/zfs0 and gpt/zfs1 (which are shown in zpool status), but they are not on /dev/da0 and /dev/da1. Thank you very much for your suggestions. Regards, lk