From owner-freebsd-stable@freebsd.org Thu Nov 21 13:54:21 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 150081BF4FB for ; Thu, 21 Nov 2019 13:54:21 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47Jgz06mCjz4QB9 for ; Thu, 21 Nov 2019 13:54:20 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from mail-qv1-f54.google.com (mail-qv1-f54.google.com [209.85.219.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) (Authenticated sender: kevans) by smtp.freebsd.org (Postfix) with ESMTPSA id BFC8A7FE0 for ; Thu, 21 Nov 2019 13:54:20 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: by mail-qv1-f54.google.com with SMTP id g18so1402992qvp.8 for ; Thu, 21 Nov 2019 05:54:20 -0800 (PST) X-Gm-Message-State: APjAAAU7EYEmiEQ0vSDu6mX3ZdLXbEykG/4Cx3ybty5hvezWyKrB7zTq RitYQsSdrp04cJmsQhKWGNHN+X1Fop0/ohIulh0= X-Google-Smtp-Source: APXvYqwt3zSh7vhutFNqxT+rrmCboCehPmnLeqq+NAutz7WaHFVa4vKqRobpS41+zNOjHR6hvInbbEJcjxxDO2m7PCE= X-Received: by 2002:a0c:8e87:: with SMTP id x7mr8168189qvb.112.1574344460212; Thu, 21 Nov 2019 05:54:20 -0800 (PST) MIME-Version: 1.0 References: <4c45a371-c352-b8c4-65c6-d1bff55b4a2e@grosbein.net> In-Reply-To: <4c45a371-c352-b8c4-65c6-d1bff55b4a2e@grosbein.net> From: Kyle Evans Date: Thu, 21 Nov 2019 07:54:07 -0600 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Logging boot messages from the loader To: Eugene Grosbein Cc: "Bjoern A. Zeeb" , FreeBSD stable Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Nov 2019 13:54:21 -0000 On Thu, Nov 21, 2019 at 4:48 AM Eugene Grosbein wrote: > > 21.11.2019 17:22, Bjoern A. Zeeb wrote: > > > On 21 Nov 2019, at 9:51, Eugene Grosbein wrote: > > > >> Hi! > >> > >> How do I get boot messages generated by loader itself written to some = log after successfull boot? > >> This is generic BIOS-based 11.3-STABLE/amd64 system with syscons conso= le driver. > >> > >> /var/run/dmesg.boot starts with kernel messages without previous lines= from the loader. > >> Specifically, I need messages about loading kernel modules with full p= aths. > > > > loader cannot write to the kernel message buffer as there is no kernel = yet. There might be possible ways to preserve that but I doubt loader is f= it for that task. > > Some hardware does not clear memory while warm booting, so KDB trace from= previous panic sometimes survive reboot > and ends up in /var/run/dmesg.boot for successful boot. I wonder why load= er messages cannot be there, too. > In fact, kernel message buffer can be found even in absence of kernel :-) > > I've already searched through the sources in stand/ and see there is no c= ode to support this yet. > > > A boot -v (boot_verbose=3D=E2=80=9CYES=E2=80=9D) might spit the informa= tion out from the kernel as well. Not sure if that helps you. However on= modern server machines a boot -v often overflows the message buffer.. so y= ou might have to tune that as well. > > > > Extracted from an older console log of a random lab machine of someone = else: > > .. > > WARNING: WITNESS option enabled, expect reduced performance.^M^@ > > WARNING: DIAGNOSTIC option enabled, expect reduced performance.^M^@ > > Preloaded elf kernel "/boot/kernel/kernel" at 0xffffffff80e66000.^M^@ > > Preloaded elf obj module "/boot/kernel/if_igb.ko" at 0xffffffff80e66968= .^M^@ > > Preloaded elf obj module "/boot/kernel/nfscl.ko" at 0xffffffff80e66fd0.= ^M^@ > > Preloaded elf obj module "/boot/kernel/nfslock.ko" at 0xffffffff80e6763= 8.^M^@ > > Preloaded elf obj module "/boot/kernel/nfssvc.ko" at 0xffffffff80e67ba0= .^M^@ > > Preloaded elf obj module "/boot/kernel/krpc.ko" at 0xffffffff80e68088.^= M^@ > > Preloaded elf obj module "/boot/kernel/nfscommon.ko" at 0xffffffff80e68= 6b0.^M^@ > > Preloaded elf obj module "/boot/kernel/ahci.ko" at 0xffffffff80e68d20.^= M^@ > > Preloaded elf obj module "/boot/kernel/cam.ko" at 0xffffffff80e69308.^M= ^@ > > Preloaded elf obj module "/boot/kernel/mfi.ko" at 0xffffffff80e699f0.^M= ^@ > > Preloaded elf obj module "/boot/kernel/geom_part_gpt.ko" at 0xffffffff8= 0e6a058.^M^@ > > Preloaded elf obj module "/boot/kernel/sysvsem.ko" at 0xffffffff80e6a64= 8.^M^@ > > Preloaded elf obj module "/boot/kernel/sysvshm.ko" at 0xffffffff80e6acb= 0.^M^@ > > Calibrating TSC clock ... TSC clock: 3400185886 Hz^M^@ > > CPU: Intel(R) Xeon(R) CPU X5272 @ 3.40GHz (3400.19-MHz K8-cl= ass CPU)^M^@ > > Yes, but this only shows results of loader's work and kldstat -v shows pa= ths too. > I need to know why changing modules_path in the nextboot.conf does not wo= rk > and why does loader uses stock zfs.ko insted of my patched version /boot/= nextboot/zfs.ko > if I use nextboot.conf with module_path=3D"/boot/nextboot;/boot/kernel;/b= oot/modules" > > I pointed this out on IRC, but for list's sake: for something like this, it's good to verify module_path that loader used by inspecting kenv. This is my guess at the output: $ kenv module_path /boot/kernel;/boot/nextboot;/boot/kernel;/boot/modules With lualoader, it would be slightly different as we go through the cosmetic exercise of removing the loaded kernel path from the middle of module_path if it's there. Both 4th and lua will prepend the kernel path to module_path once the kernel's loaded, which should always happen after nextboot processing. For anyone else doing the same, the best option here would likely be fully qualifying the module names for every module you want out of a separate path: zfs_name=3D"/boot/nextboot/zfs.ko" This should/will work with both 4thloader and lualoader. It's not ideal for general configuration, but if you're doing a one-off load of a module in loader it works.