From owner-freebsd-current@freebsd.org Tue Apr 10 03:33:13 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 97186F99461 for ; Tue, 10 Apr 2018 03:33:13 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from mx1.scaleengine.net (mx1.scaleengine.net [209.51.186.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46447838BB for ; Tue, 10 Apr 2018 03:33:13 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 71C7914FCC for ; Tue, 10 Apr 2018 03:33:12 +0000 (UTC) Subject: Re: Odd ZFS boot module issue on r332158 To: freebsd-current@freebsd.org References: From: Allan Jude Openpgp: preference=signencrypt Message-ID: <935ad20e-017c-5c34-61b4-9db58788a663@freebsd.org> Date: Mon, 9 Apr 2018 23:33:08 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="GJGpJoZeIZ30FlmN3X4bEmNmIDvTY4DjL" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2018 03:33:13 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --GJGpJoZeIZ30FlmN3X4bEmNmIDvTY4DjL Content-Type: multipart/mixed; boundary="JvzDtnG2I0n6zmhhiKQVAjGsOQ4Sncttc"; protected-headers="v1" From: Allan Jude To: freebsd-current@freebsd.org Message-ID: <935ad20e-017c-5c34-61b4-9db58788a663@freebsd.org> Subject: Re: Odd ZFS boot module issue on r332158 References: In-Reply-To: --JvzDtnG2I0n6zmhhiKQVAjGsOQ4Sncttc Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 2018-04-09 19:11, Andrew Gallatin wrote: > I updated my main amd64 workstation to r332158 from something much > earlier (mid Jan). >=20 > Upon reboot, all seemed well.=C2=A0 However, I later realized that the = vmm.ko > module was not loaded at boot, because bhyve PCI passthru did not > work.=C2=A0 My loader.conf looks like (I'm passing a USB interface thro= ugh): >=20 > ####### > vmm_load=3D"YES" > opensolaris_load=3D"YES" > zfs_load=3D"YES" > nvidia_load=3D"YES" > nvidia-modeset_load=3D"YES" >=20 > # Tune ZFS Arc Size - Change to adjust memory used for disk cache > vfs.zfs.arc_max=3D"4096M" > hint.xhci.2.disabled=3D"1" > pptdevs=3D"8/0/0" > hw.dmar.enable=3D"0" > cuse_load=3D"YES" > ####### >=20 > The problem seems "random".=C2=A0 I rebooted into single-user to > see if somehow, vmm.ko was loaded at boot and something > was unloading vmm.ko.=C2=A0 However, on this boot it was loaded.=C2=A0 = I then > ^D'ed and continued to multi-user, where X failed to start because > this time, the nvidia modules were not loaded.=C2=A0 (but nvidia had > been loaded on the 1st boot). >=20 > So it *seems* like different modules are randomly not loaded by the > loader, at boot.=C2=A0=C2=A0 The ZFS config is: >=20 > config: >=20 > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 NAME=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 STATE=C2=A0=C2=A0=C2=A0=C2=A0 READ WRITE CKSUM > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 tank=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 ONLINE=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0= =C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0 0 > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 mirror-0=C2=A0 O= NLINE=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0= =C2=A0=C2=A0=C2=A0 0 > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ada0= p2=C2=A0 ONLINE=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2= =A0 0=C2=A0=C2=A0=C2=A0=C2=A0 0 > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 da3p= 2=C2=A0=C2=A0 ONLINE=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2= =A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0 0 > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 mirror-1=C2=A0 O= NLINE=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0= =C2=A0=C2=A0=C2=A0 0 > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ada1= p2=C2=A0 ONLINE=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2= =A0 0=C2=A0=C2=A0=C2=A0=C2=A0 0 > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 da0p= 2=C2=A0=C2=A0 ONLINE=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2= =A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0 0 > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 cache > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 da2s1d=C2=A0=C2=A0= =C2=A0 ONLINE=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0= 0=C2=A0=C2=A0=C2=A0=C2=A0 0 >=20 > The data drives in the pool are all exactly like this: >=20 > =3D>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 34=C2=A0 9767541101=C2=A0= ada0=C2=A0 GPT=C2=A0 (4.5T) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 34=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 6=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 - free -=C2=A0 (3.0K) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 204800=C2=A0=C2=A0=C2=A0=C2=A0 1=C2=A0 efi=C2=A0 (100M) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 204840=C2=A0 9763209216=C2=A0=C2=A0=C2=A0= =C2=A0 2=C2=A0 freebsd-zfs=C2=A0 (4.5T) > =C2=A0 9763414056=C2=A0=C2=A0=C2=A0=C2=A0 4096000=C2=A0=C2=A0=C2=A0=C2=A0= 3=C2=A0 freebsd-swap=C2=A0 (2.0G) > =C2=A0 9767510056=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 31079=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free -=C2=A0 (15M) >=20 >=20 > There is about 1.44T used in the pool.=C2=A0 I have no idea > how ZFS mirrors work, but I'm wondering if somehow this > is a 2T problem, and there are issues with blocks on > difference sides of the mirror being across the 2T boundary. >=20 > Sorry to be so vague.. but this is the one machine I *don't* have > a serial console on, so I don't have good logs. >=20 > Drew >=20 > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.o= rg" What makes you think it is related to ZFS? Are there any error messages when the nvidia module did not load? --=20 Allan Jude --JvzDtnG2I0n6zmhhiKQVAjGsOQ4Sncttc-- --GJGpJoZeIZ30FlmN3X4bEmNmIDvTY4DjL Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJazDB3AAoJEBmVNT4SmAt+ymMQANMJJAfQ1M8nmX99hWOFIZI0 28ufaHxQ54tVuz7aOxYAb0ZappGmNIQ1WQbsoajPzlGMfs2NM3YSCGqOGaYWvYUn z313Umh2MeXRoApCykVN33RgI1nB5AfNRB+3H/xUAUiEO6acjqlEV7h7S//+YXDZ nqGTGNk8SS6Tgwuut61Sd25Tl+OSBothreed4CgQX/CtugOq4s4nb2vUvcn3skg4 uVPXG99GjMKby5ySBYVaaUeuyMuZnG+CxXUfWvsA+CsV0J5/UXxEUpM4AdHxWj98 gfw6WNJ35TBKt7zcfTsU+Hv2PAi+retOwkOMWBFhllyXlYidYFHu5uZz331MRg1A r8Bd1qEdjaHcZX2WrXHq6jAgl0eQJF+123qb9bf6zs9KjbT7flnO7gGeesqnE7pX iFi5nigDnEan0rCoeCkrbjvNlSnuE//DgKObdjPeJNhTpeKz+iL2sktk4FHLpNNo Kv4Zq2MQvHPyW3ozofBLEL5CR3/LNjnul2k6bX0MbvXv8Z7buYZl773ukob0AS4A CK+48I0rjD4mw2mEuJdwihhgK5g/chk45ZksdggFJmxRcxsV/8NEAmsm6VXCA0CL 0mkGIlHRMgAuEildDhePEMSSiwiPCkGlxbyTOAWLHjEYwf/N8BxxEkMHPrrmbDgC UF26atezGKruhx648P0h =bvCR -----END PGP SIGNATURE----- --GJGpJoZeIZ30FlmN3X4bEmNmIDvTY4DjL--