From owner-freebsd-current@freebsd.org Wed Dec 9 02:50:14 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D76084B3E7A for ; Wed, 9 Dec 2020 02:50:14 +0000 (UTC) (envelope-from warlock@phouka1.phouka.net) Received: from phouka1.phouka.net (phouka1.phouka.net [107.170.196.116]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "phouka.net", Issuer "Go Daddy Secure Certificate Authority - G2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CrM3V0Xr6z3nKD for ; Wed, 9 Dec 2020 02:50:13 +0000 (UTC) (envelope-from warlock@phouka1.phouka.net) Received: from phouka1.phouka.net (localhost [127.0.0.1]) by phouka1.phouka.net (8.16.1/8.16.1) with ESMTPS id 0B92mubD043734 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Tue, 8 Dec 2020 18:48:56 -0800 (PST) (envelope-from warlock@phouka1.phouka.net) Received: (from warlock@localhost) by phouka1.phouka.net (8.16.1/8.16.1/Submit) id 0B92mudp043733; Tue, 8 Dec 2020 18:48:56 -0800 (PST) (envelope-from warlock) Date: Tue, 8 Dec 2020 18:48:56 -0800 From: John Kennedy To: Alban Hertroys Cc: freebsd-current@freebsd.org Subject: Re: KLD zfs.ko: depends on kernel - not available or version mismatch Message-ID: References: <42AC7323-5AD6-401D-9A7D-F1D962EE5717@gmail.com> <2B044A92-500F-4121-85DB-D486865C75B5@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2B044A92-500F-4121-85DB-D486865C75B5@gmail.com> X-Rspamd-Queue-Id: 4CrM3V0Xr6z3nKD X-Spamd-Bar: - X-Spamd-Result: default: False [-1.80 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[phouka.net]; RBL_DBL_DONT_QUERY_IPS(0.00)[107.170.196.116:from]; AUTH_NA(1.00)[]; SPAMHAUS_ZRD(0.00)[107.170.196.116:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; R_SPF_NA(0.00)[no SPF record]; FREEMAIL_TO(0.00)[gmail.com]; FORGED_SENDER(0.30)[warlock@phouka.net,warlock@phouka1.phouka.net]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:14061, ipnet:107.170.192.0/18, country:US]; FROM_NEQ_ENVFROM(0.00)[warlock@phouka.net,warlock@phouka1.phouka.net]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Dec 2020 02:50:14 -0000 On Tue, Dec 08, 2020 at 07:10:26PM +0100, Alban Hertroys wrote: > > You didn't say that you've installed the new kernel, which at least starts > > you down the road towards a driver/kernel mismatch. You presumably have a > > non-ZFS boot+root. > > I???m fairly sure I did, actually. > > Last time I checked, "make buildworld buildkernel" was equivalent to "make buildworld && make buildkernel", while "make kernel??? is a shorthand for ???make buildkernel && make installkernel??? > > So, unless I???m mistaken, ???make buildworld kernel??? should be equivalent to your first two lines. > > Nevertheless, I retried without these assumptions, the result was the same. I forgot to ???make delete-old??? though, I rarely remember to do that??? Ah, the dangers of command syntax being close to human syntax. You're trying to do the right thing, so maybe we can sanity check that. > I had to copy over several files from /etc and /usr/local/etc and re-installed the most important packages. This was admittedly a bit messy, it is possible that I forgot to copy something over. > (Originally my intention was to dd the contents of the spinning disk over, but apparently that disk has a few wonky sectors, dd failed after a few device timeouts) ... so, no guarantee that things are totally sane. The "sane" we're looking for is how you can presumably be booting a kernel located at /boot/kernel/kernel and not have it match the kernel modules found under /boot/kernel. The fact that it is happy with the old kernel modules (presumably under found in /boot/kernel.old) may be a red herring if they're just compatible enough. I can see what I'm expecting to boot here: # grep -E 'boot\/kernel|f7b0aedd1c50' /var/log/messages | tail -2 Dec 6 08:59:04 ouroboros syslogd: kernel boot file is /boot/kernel/kernel Dec 6 08:59:04 ouroboros kernel: FreeBSD 13.0-CURRENT #237 r368388+f7b0aedd1c50-c273383(master): Sun Dec 6 08:27:47 PST 2020 So, I build my system with WITHOUT_REPRODUCIBLE_BUILD=YES in /etc/src.conf, so I can easily see my build version with uname -v: FreeBSD 13.0-CURRENT #237 r368388+f7b0aedd1c50-c273383(master): ... That matches my source tree: # git log -n1 /usr/src | grep revision svn path=/head/; revision=368388 (I've always used git for my sources, but I'm sure there is a svn equivalent.) The version I'm running is what and where I'd expect it to be: # strings -a < /boot/kernel/kernel | grep 'FreeBSD 13' | tail -1 FreeBSD 13.0-CURRENT #237 r368388+f7b0aedd1c50-c273383(master): Sun Dec 6 08:27:47 PST 2020 It certainly isn't the previous kernel: # strings -a < /boot/kernel.old/kernel | grep 'FreeBSD 13' | tail -1 FreeBSD 13.0-CURRENT #236 r368353+0252bfaea893-c273359(master): Fri Dec 4 16:55:41 PST 2020 Not sure what that'll look like with reproducible builds. The hash-check below is a decent stamp, in case the timestamps in /boot/kernel are misleading. What I have built in my source tree is the kernel/zfs module I'd expect: # md5 -r /usr/obj/usr/src/amd64.amd64/sys/GENERIC/kernel /usr/obj/usr/src/amd64.amd64/sys/GENERIC/modules/usr/src/sys/modules/zfs/zfs.ko /boot/kernel/kernel /boot/kernel/zfs.ko | sort 941ab52d075e444da6eea7fb56213e10 /boot/kernel/kernel 941ab52d075e444da6eea7fb56213e10 /usr/obj/usr/src/amd64.amd64/sys/GENERIC/kernel 97d4e0c8ffed1f75e924bf8768a95ff1 /boot/kernel/zfs.ko 97d4e0c8ffed1f75e924bf8768a95ff1 /usr/obj/usr/src/amd64.amd64/sys/GENERIC/modules/usr/src/sys/modules/zfs/zfs.ko What are you seeing after your installkernel equivalent? Your hashes won't match mine due to non-reproducible build. I'd make sure you don't have anything in /boot/modules or otherwise load any extra modules until sanity is restored (just to reduce random variables).