From nobody Tue Oct 29 12:28:24 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xd8gK2KSfz5bPbx for ; Tue, 29 Oct 2024 12:28:29 +0000 (UTC) (envelope-from void@f-m.fm) Received: from fout-a1-smtp.messagingengine.com (fout-a1-smtp.messagingengine.com [103.168.172.144]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xd8gJ0hg6z43yv for ; Tue, 29 Oct 2024 12:28:28 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm3 header.b=lC6UBNXW; dkim=pass header.d=messagingengine.com header.s=fm3 header.b="d qa6TU8"; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 103.168.172.144 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from phl-compute-05.internal (phl-compute-05.phl.internal [10.202.2.45]) by mailfout.phl.internal (Postfix) with ESMTP id 40ED413801DE for ; Tue, 29 Oct 2024 08:28:27 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-05.internal (MEProxy); Tue, 29 Oct 2024 08:28:27 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :message-id:mime-version:reply-to:subject:subject:to:to; s=fm3; t=1730204907; x=1730291307; bh=wnbjEs9J1wJrINKjw7sMlbHiPaEiMmhg rMzTVDYy9iM=; b=lC6UBNXWJzJV+Aakeg8nUdo+TtpiCzm4RNVX5u3znwQDBltK 9uq5+e/BYx8CvW0dVWnf1fMXNliaOvwJVltbky7R3Q1dWidHzhp9gXHC7EMBaboR IURkqfpyaRhEYJe9DQkUXCr6wG0xqbgN2RF/K0jE3E85Pm+XJI9CXOCtJSclyEJT DwnYBVGrpVYQRulB4Gz5uBw8ZxBALFLK38NXdOVwut3WuVEqFs5hwPVYTkrtnevd n+AXYOHHsElnbNnjHxoRAxLJ8yDpyWUXinF8TSUFrO78287sGAxTQ6vNtH45y75Z nulyAZ3+Ux8Q7deLhL0iadZJsotOmguaRCpPAw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:message-id :mime-version:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1730204907; x= 1730291307; bh=wnbjEs9J1wJrINKjw7sMlbHiPaEiMmhgrMzTVDYy9iM=; b=d qa6TU8WisErPCOax//+pdNtqEw0GcLCOoiyiqIYdSyg0pYJpcImdVq+D3N9K4zji Ghv7J06J8erhgQbIgeCFcIxuH2E/j4nwmGCKvE/bIVuG21NykEh4M7wY9NVWeGGl Gpal67P+sh1+ryCPNZdkXqTaLnsFREHmi2/TeQbnUjqbvaMwcU78fIV1lmsaU9U4 9EH1wq3EC2gAJoZggUYJe/9UQ3ZNreuO2u8ulKjVTS6EQMIuaD+N2BBo3z6mMqKe zHGL07YpQYJxb0N9yjGYZuFK2KAd48mKPCwx9dKgW7UM7++bw59ni4p0lckDaLo1 oiXkXvrfZNQAMNY+Kdy2g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrvdekuddgfeeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuf fkgggtugesthdtredttddtvdenucfhrhhomhepvhhoihguuceovhhoihgusehfqdhmrdhf mheqnecuggftrfgrthhtvghrnhepveduffeivdfffffghfegfeejfefftdeiteehteekfe fhvdefgfettdeuheegffeunecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehm rghilhhfrhhomhepvhhoihgusehfqdhmrdhfmhdpnhgspghrtghpthhtohepuddpmhhoug gvpehsmhhtphhouhhtpdhrtghpthhtohepfhhrvggvsghsugdqhhgrtghkvghrshesfhhr vggvsghsugdrohhrgh X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Tue, 29 Oct 2024 08:28:26 -0400 (EDT) Date: Tue, 29 Oct 2024 12:28:24 +0000 From: void To: freebsd-hackers@freebsd.org Subject: WITHOUT_LIB32= Message-ID: Mail-Followup-To: freebsd-hackers@freebsd.org List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline X-Spamd-Result: default: False [-2.62 / 15.00]; SUBJ_ALL_CAPS(1.05)[14]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.97)[-0.966]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; R_SPF_ALLOW(-0.20)[+ip4:103.168.172.128/27]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm3,messagingengine.com:s=fm3]; RCVD_IN_DNSWL_LOW(-0.10)[103.168.172.144:from]; RWL_MAILSPIKE_GOOD(-0.10)[103.168.172.144:from]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; FREEMAIL_FROM(0.00)[f-m.fm]; RCVD_COUNT_THREE(0.00)[3]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_ONE(0.00)[1]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; ASN(0.00)[asn:151847, ipnet:103.168.172.0/24, country:AU]; RCVD_VIA_SMTP_AUTH(0.00)[]; MISSING_XM_UA(0.00)[]; DWL_DNSWL_NONE(0.00)[messagingengine.com:dkim] X-Rspamd-Queue-Id: 4Xd8gJ0hg6z43yv X-Spamd-Bar: -- Hello, On a machine not using emulation, will building a world with WITHOUT_LIB32= break anything in base? I mean, is anything in base dependent on it? Is there a way to scan the ports tree that will reliably tell if it's a hard dependency? -- From nobody Tue Oct 29 23:46:51 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XdRkF4mJRz5b9Rj for ; Tue, 29 Oct 2024 23:47:01 +0000 (UTC) (envelope-from steffen@sdaoden.eu) Received: from sdaoden.eu (sdaoden.eu [217.144.132.164]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XdRkD6jPBz46WP for ; Tue, 29 Oct 2024 23:47:00 +0000 (UTC) (envelope-from steffen@sdaoden.eu) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=sdaoden.eu header.s=citron header.b=kvUqqLwt; spf=pass (mx1.freebsd.org: domain of steffen@sdaoden.eu designates 217.144.132.164 as permitted sender) smtp.mailfrom=steffen@sdaoden.eu; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sdaoden.eu; s=citron; t=1730245612; x=1730912278; h=date:author:from:to:subject: message-id:in-reply-to:references:openpgp:blahblahblah:author:from: subject:date:to:cc:resent-author:resent-date:resent-from:resent-sender: resent-to:resent-cc:resent-reply-to:resent-message-id:in-reply-to: references:mime-version:content-type:content-transfer-encoding: content-disposition:content-id:content-description:message-id: mail-followup-to:openpgp:blahblahblah; bh=oIfaaX76Z+yqGtbjje6xN0pbOyXb0mG3gH0KhSh5tVk=; b=kvUqqLwtQ9bOaK9L8pKXqPXQuFgnlP4IZ1Iei1h207dv8JQw8t4doJ2JKY6K3vMpOPxooIhi cDKt9PL+FJXuLL4d8pyx3Q9sVtGREkmmGOlLurV/o99wZFHbuGtzKUGl/kevo86TUib8nJWHUg o5FwpumyoEjlIubKDIKBs2wTtg7QDKXI9xlqjoErBh5JpEQqnLxK3/uK2tlc0Oee+TrRkOlyP9 2q3MEMvq8kKAcIJP6qPWXeRKvrJ4L3U9xpHLsl6Oj97MT8ZPIzu4DXPE5kozlgsGiS0DkhJBiH oCm3xDksSnCv8eOdLzmbhrjvLuMXe0j+dj5jgSNyQrkdpXAQ== DKIM-Signature: v=1; a=adaed25519-sha256; c=relaxed/relaxed; d=sdaoden.eu; s=orange; t=1730245612; x=1730912278; h=date:author:from:to:subject: message-id:in-reply-to:references:openpgp:blahblahblah:author:from: subject:date:to:cc:resent-author:resent-date:resent-from:resent-sender: resent-to:resent-cc:resent-reply-to:resent-message-id:in-reply-to: references:mime-version:content-type:content-transfer-encoding: content-disposition:content-id:content-description:message-id: mail-followup-to:openpgp:blahblahblah; bh=oIfaaX76Z+yqGtbjje6xN0pbOyXb0mG3gH0KhSh5tVk=; b=67CJPIyGpvSjeaRJEAMapeAXmucpmESy0sk2yeZxTBWhtYevoixDTMboXGmUt29W41FLtYry 5mHWTPzCTpiBBQ== Date: Wed, 30 Oct 2024 00:46:51 +0100 Author: Steffen Nurpmeso From: Steffen Nurpmeso To: freebsd-hackers@freebsd.org Subject: Re: FreeBSD Errata Notice FreeBSD-EN-24:17.pam_xdg Message-ID: <20241029234651.ij8s6Vq5@steffen%sdaoden.eu> In-Reply-To: <20241029213236.B9E469155@freefall.freebsd.org> References: <20241029213236.B9E469155@freefall.freebsd.org> User-Agent: s-nail v14.9.25-623-g805238bd9b OpenPGP: id=EE19E1C1F2F7054F8D3954D8308964B51883A0DD; url=https://ftp.sdaoden.eu/steffen.asc; preference=signencrypt BlahBlahBlah: Any stupid boy can crush a beetle. But all the professors in the world can make no bugs. X-Spamd-Result: default: False [-3.44 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.94)[-0.943]; R_DKIM_ALLOW(-0.20)[sdaoden.eu:s=citron]; R_SPF_ALLOW(-0.20)[+a]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:15987, ipnet:217.144.128.0/20, country:DE]; MIME_TRACE(0.00)[0:+]; FROM_HAS_DN(0.00)[]; DMARC_NA(0.00)[sdaoden.eu]; ARC_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_ZERO(0.00)[0]; TO_DN_NONE(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[sdaoden.eu:+] X-Rspamd-Queue-Id: 4XdRkD6jPBz46WP X-Spamd-Bar: --- List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org FreeBSD Errata Notices wrote in <20241029213236.B9E469155@freefall.freebsd.org>: .. |FreeBSD-EN-24:17.pam_xdg Errata \ .. |Category: core |Module: pam_xdg |Announced: 2024-10-29 |Credits: Olivier Certner |Affects: FreeBSD 14.1 Honestly, at least my hmm request to change the name of this thing could have been honoured, *if* you really have a need to "pamper over" me like that at first. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) | |And in Fall, feel "The Dropbear Bard"s ball(s). | |The banded bear |without a care, |Banged on himself fore'er and e'er | |Farewell, dear collar bear From nobody Thu Oct 31 05:18:11 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XfC1p4g5Fz5b6WS for ; Thu, 31 Oct 2024 05:18:06 +0000 (UTC) (envelope-from vini.ipsmaker@gmail.com) Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com [IPv6:2a00:1450:4864:20::12a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XfC1n4Z1Vz583N for ; Thu, 31 Oct 2024 05:18:05 +0000 (UTC) (envelope-from vini.ipsmaker@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=TofRfaEb; spf=pass (mx1.freebsd.org: domain of vini.ipsmaker@gmail.com designates 2a00:1450:4864:20::12a as permitted sender) smtp.mailfrom=vini.ipsmaker@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-lf1-x12a.google.com with SMTP id 2adb3069b0e04-539e59dadebso657937e87.0 for ; Wed, 30 Oct 2024 22:18:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1730351883; x=1730956683; darn=freebsd.org; h=content-transfer-encoding:to:subject:message-id:date:from :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=JpTK4NuFsNXFPW+B9FlMPlBOSN4e2TMrKMOP/5Um9YE=; b=TofRfaEbG1RiJz1airpRcZ0Ln1m65L+kp/axdKpQqtZ+aIPcQLSPEBk44k3JuhAcu7 vfeedVAM3Ols6VuQiTtyO/+MAwxoNlRf1fSR0ilQXk1LBICaxmVmPnCA0XQdmR/l4XpZ eiUEbIM5y6yIQ9rAmoEoitinC7lUJ2kqOjW6dZBcy6C1BA4H9vfZ8VDvOqb6mugarajU oHEy3sGBbvWCIpYOxzZ5utx7dT0lc1JZplFDdyxgyqg20lazz1oV69l0ncUFqIKeghjQ RtQSNAR0jCgBhhkYwKYmrcczmqkaIdW3ZJAaRnktX4C/Jgy486lW74AjMqjjRZrVYgEt oHBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730351883; x=1730956683; h=content-transfer-encoding:to:subject:message-id:date:from :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=JpTK4NuFsNXFPW+B9FlMPlBOSN4e2TMrKMOP/5Um9YE=; b=hYpm1tarURtFjtLTgJkBHfgeeRKZAE1jwzsYPRhFatqrbXzSe9wweTR8zLo7SEXjzB RCBkSFG/xYHwqIa9AYdROH3XgnDO8Z8mkGYKtfzmLWGKfw9ph0MyhgSqDWYI5n9W4d6e h//WBWooe/D7ueLZ4QRBlRql9pEPXe5RAR58YRlEjsqLlkNP7SATXIJOU81mV1y9Xn67 NdQX9Spyydfc+Z6b2ppqjjJuyp7NHVfU0UtDjK3sLpHtKOxiOze/LpEUrlemj19au20A XbReJfHN7Awc9CcjCpI0LU65aqoZnB5FjBi24rhyqELyHnsjvCbS1vHaPli91BMo0sbn gL5Q== X-Gm-Message-State: AOJu0YzKucRqMlx8gB9HZwvH8u9Vnf/2630PzEe6BGMb4a0pG89FctVw 0CyECw1XaZLaX+5AGUkU2Ip8thj9cFWKfASKRYfYeTSNp8P+G3RuWaDTUd8m26ybBboiW9pRhNX 9a8vv0oCVclsjVpAmt/9b/18tjFvKKRs3 X-Google-Smtp-Source: AGHT+IHflOLCJ8xWVeWz0CX4oDi6ki5THrHk2a+yU+mbpPOb6yZVhMZefeslkrSA1g82/7bcLCoSNBH97s24oLmYODA= X-Received: by 2002:a05:6512:23a3:b0:539:e513:1f69 with SMTP id 2adb3069b0e04-53b348f97c6mr8060102e87.17.1730351883149; Wed, 30 Oct 2024 22:18:03 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 From: =?UTF-8?Q?Vin=C3=ADcius_dos_Santos_Oliveira?= Date: Thu, 31 Oct 2024 02:18:11 -0300 Message-ID: Subject: [rtld] fdlopen(), fdlputpath() and capsicum To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; TAGGED_FROM(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+]; FROM_HAS_DN(0.00)[]; MISSING_XM_UA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MID_RHS_MATCH_FROMTLD(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::12a:from] X-Rspamd-Queue-Id: 4XfC1n4Z1Vz583N X-Spamd-Bar: --- I'm trying to use capsicum to load untrusted plugins (fdlopen after cap_enter), but I've been facing problems as plugins might depend on not yet loaded libraries (LD_LIBRARY_PATH no longer works after we disable ambient authority). 10 years ago rtld gained the option/env-var to specify directory descriptors to be searched instead[1]. However this env-var is only queried at program startup (the use case is not sandboxing plugins through fdlopen(), but whole programs through exec()). exec() is not an option for my use case. Plugins are meant to load functions that extend the current program image (e.g. manipulating data structures or some shared memory). exec() would just kill any use I have for plugins. My project just stalled on this problem this week. A workaround I've been planning was to statically link the plugins to the libraries (they are untrusted, so this step would occur in a jail), but this won't be an option for every plugin (maybe closed source, maybe licensing problems for the library linked with, ...). I'd like to request a new function to use from C programs: int fdlputpath(int fd); fdlputpath() would just modify the private variable ld_library_dirs introduced in the commit which introduced the env var LD_LIBRARY_PATH_FDS. fdlputpath() would have some restrictions similar to putenv[2][3][4]. The change is simple and it'd open the doors to use fdlopen() for capsicum processes. Thoughts? [1] https://git.quacker.org/d/freebsd-nq/commit/02d3b38e0a766bde374de052a2c= 65a095282f302?style=3Dunified&whitespace=3Dignore-all&show-outdated=3D [2] putenv() modifies a global and it's not thread-safe. fdlputpath() would have the same restriction. [3] It's not safe to deallocate old values inserted by previous calls to putenv() as copies/references to old values might be hanging around. fdlputpath() would have the same restriction. [4] Some implementations of putenv() might provoke intentional leaks to avoid the previous problem. Users of fdlputpath() might adopt a similar approach and keep the passed values hanging around as valid resources forever just in case. --=20 Vin=C3=ADcius dos Santos Oliveira https://vinipsmaker.github.io/ From nobody Thu Oct 31 06:08:16 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XfD802lRwz5bBqg for ; Thu, 31 Oct 2024 06:08:32 +0000 (UTC) (envelope-from kib@freebsd.org) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XfD7z2qQfz40VV for ; Thu, 31 Oct 2024 06:08:31 +0000 (UTC) (envelope-from kib@freebsd.org) Authentication-Results: mx1.freebsd.org; dkim=none; spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kib@freebsd.org) smtp.mailfrom=kib@freebsd.org; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=freebsd.org (policy=none) Received: from tom.home (kib@localhost [127.0.0.1] (may be forged)) by kib.kiev.ua (8.18.1/8.18.1) with ESMTP id 49V68Gmf088094; Thu, 31 Oct 2024 08:08:19 +0200 (EET) (envelope-from kib@freebsd.org) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 49V68Gmf088094 Received: (from kostik@localhost) by tom.home (8.18.1/8.18.1/Submit) id 49V68GSV088093; Thu, 31 Oct 2024 08:08:16 +0200 (EET) (envelope-from kib@freebsd.org) X-Authentication-Warning: tom.home: kostik set sender to kib@freebsd.org using -f Date: Thu, 31 Oct 2024 08:08:16 +0200 From: Konstantin Belousov To: =?utf-8?B?Vmluw61jaXVz?= dos Santos Oliveira Cc: freebsd-hackers@freebsd.org Subject: Re: [rtld] fdlopen(), fdlputpath() and capsicum Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=4.0.1 X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on tom.home X-Spamd-Result: default: False [-3.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.995]; MIME_GOOD(-0.10)[text/plain]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : No valid SPF, No valid DKIM,none]; RCPT_COUNT_TWO(0.00)[2]; TAGGED_RCPT(0.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[kib]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; HAS_XAW(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all]; R_DKIM_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; MISSING_XM_UA(0.00)[] X-Rspamd-Queue-Id: 4XfD7z2qQfz40VV X-Spamd-Bar: -- On Thu, Oct 31, 2024 at 02:18:11AM -0300, Vinícius dos Santos Oliveira wrote: > I'm trying to use capsicum to load untrusted plugins (fdlopen after > cap_enter), but I've been facing problems as plugins might depend on > not yet loaded libraries (LD_LIBRARY_PATH no longer works after we > disable ambient authority). > > 10 years ago rtld gained the option/env-var to specify directory > descriptors to be searched instead[1]. > > However this env-var is only queried at program startup (the use case > is not sandboxing plugins through fdlopen(), but whole programs > through exec()). exec() is not an option for my use case. Plugins are > meant to load functions that extend the current program image (e.g. > manipulating data structures or some shared memory). exec() would just > kill any use I have for plugins. > > My project just stalled on this problem this week. A workaround I've > been planning was to statically link the plugins to the libraries > (they are untrusted, so this step would occur in a jail), but this > won't be an option for every plugin (maybe closed source, maybe > licensing problems for the library linked with, ...). > > I'd like to request a new function to use from C programs: > > int fdlputpath(int fd); > > fdlputpath() would just modify the private variable ld_library_dirs > introduced in the commit which introduced the env var > LD_LIBRARY_PATH_FDS. fdlputpath() would have some restrictions similar > to putenv[2][3][4]. > > The change is simple and it'd open the doors to use fdlopen() for > capsicum processes. > > Thoughts? > > [1] https://git.quacker.org/d/freebsd-nq/commit/02d3b38e0a766bde374de052a2c65a095282f302?style=unified&whitespace=ignore-all&show-outdated= > [2] putenv() modifies a global and it's not thread-safe. fdlputpath() > would have the same restriction. > [3] It's not safe to deallocate old values inserted by previous calls > to putenv() as copies/references to old values might be hanging > around. fdlputpath() would have the same restriction. > [4] Some implementations of putenv() might provoke intentional leaks > to avoid the previous problem. Users of fdlputpath() might adopt a > similar approach and keep the passed values hanging around as valid > resources forever just in case. Try this https://reviews.freebsd.org/D47351 From nobody Thu Oct 31 10:28:14 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XfKvX5pytz5bdMP for ; Thu, 31 Oct 2024 10:28:08 +0000 (UTC) (envelope-from vini.ipsmaker@gmail.com) Received: from mail-lj1-x229.google.com (mail-lj1-x229.google.com [IPv6:2a00:1450:4864:20::229]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XfKvX26yNz4TGc; Thu, 31 Oct 2024 10:28:08 +0000 (UTC) (envelope-from vini.ipsmaker@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-lj1-x229.google.com with SMTP id 38308e7fff4ca-2fb50e84ec7so4993131fa.1; Thu, 31 Oct 2024 03:28:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1730370486; x=1730975286; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=Z+WBXGQUW/Ib/SNCPTXLdjCFH610552pXLhn51jNlCQ=; b=iVYprHA0iUszqUxwGcYq+l1roWu7V3hCp5OAbpareNB5G5/FS2KHmQ70GyUXIDki6B dyy2t8ejzHGjvZuhBezQaSK2agXn8z9Cap5C51318C4OGNLqBPAeDjeW6mBDZfplihoI 6NtlcGII5ddx50FLT8V7ckhWFfrh9o7suQ0djeCkfw8yBr3WoPvLxJzHZzi64wx9RwYZ hn/geaODFznHr++JXX39rnPib7BS4kYBqPHJDvhmD+8oJYoxKjznn06VEtz9O1R+t++1 VIPqJPIs05VGGyREn4mv3O1S2OBL2ligc4vKJMeBkiVO/A92xzfUrx4aXPPOtJQBmi81 nGGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730370486; x=1730975286; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Z+WBXGQUW/Ib/SNCPTXLdjCFH610552pXLhn51jNlCQ=; b=M7PL0hDc4Kp+9yJcrGoC7O3Rdj+IZHKD41fKH5gU/2Yz02r2qU+xMBbCfxnxdEwd22 8pN9algovUFvvSgxY17ZbQw52lGRZ+2uqF6CwVPJmBs+qsvKghzzSh9vLUM0Gv39cOaZ 2aOwN008nQAmyT96hdv6bpZ1w9DJZJ37KBzVym/3MT3aIdGBR67wv8g+iJi8UVGGHlb5 olCdU6DgLx7pdukaujTFaEvuNsaR+viIHI2LmEiqqj35FDoYIalJJc0qUdGc2OilgCHJ 2p5UtlloUSbqeRSzObtnUwjw+V7erEuLiQD012r2kvJyq9zRjYWK9OYiYjyJe/wFIDuk U6SQ== X-Gm-Message-State: AOJu0YwXmYw8K1jPC/lM4pEKR+fn0SXBUCEFdScBEKgJkhnsrsTiYzkG K2VgrA6Yh9d4xR9LwfQ3SIN2RzGELjU3xiDpyCTasLpR/Rr/AkmoT6aHFAEAFKcdfZtzxAVRazq 5zlYROr1yEyghvse1gTBP66LLtgsqLztY X-Google-Smtp-Source: AGHT+IGWhycsmUHRRz71zz4RudlOvGmkmkHAlMrERMF3XKVf0KEj1qohALPWPgNQ3baUXimM1NHoBgbeP4sk0qx8g8w= X-Received: by 2002:a2e:a984:0:b0:2fb:5d41:bdac with SMTP id 38308e7fff4ca-2fcbdf5faa7mr108557301fa.1.1730370485862; Thu, 31 Oct 2024 03:28:05 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: =?UTF-8?Q?Vin=C3=ADcius_dos_Santos_Oliveira?= Date: Thu, 31 Oct 2024 07:28:14 -0300 Message-ID: Subject: Re: [rtld] fdlopen(), fdlputpath() and capsicum To: Konstantin Belousov Cc: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4XfKvX26yNz4TGc X-Spamd-Bar: ---- Em qui., 31 de out. de 2024 =C3=A0s 03:09, Konstantin Belousov escreveu: > Try this https://reviews.freebsd.org/D47351 An API like this should work. It's also more flexible as it'd allow me to remove inherited already set fd numbers from the search set. > if (!lvd->can_update || (lvd->unsecure && !trust)) Maybe it's also okay to allow it if the process has already called cap_ente= r()? --=20 Vin=C3=ADcius dos Santos Oliveira https://vinipsmaker.github.io/ From nobody Thu Oct 31 13:37:27 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XfQ5t4XJ2z5bjDB for ; Thu, 31 Oct 2024 13:37:22 +0000 (UTC) (envelope-from vini.ipsmaker@gmail.com) Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com [IPv6:2a00:1450:4864:20::12d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XfQ5t1drwz4qyp; Thu, 31 Oct 2024 13:37:22 +0000 (UTC) (envelope-from vini.ipsmaker@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-lf1-x12d.google.com with SMTP id 2adb3069b0e04-539fe76e802so1214395e87.1; Thu, 31 Oct 2024 06:37:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1730381839; x=1730986639; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=EJAcPRcXZZvXeco/JGOXVlujhVre7WHP9HuHJ7eLCA8=; b=YvvIezZGpHLsw5/Un1Or58sKwrinCO98btws5FBeK0RpM4bi4c0cdBT7xEOudZRXtI E5qQBQYk1SEyqe8HyVIoRDFMOHvbfi2WWxPVgmdG5seI93lVNz5fhr3oHs6oNF1zygAj nYn3U4/ruhRVnogSq0RNSe6cun/XVBs4SvbrHoR7Is3e1usnrC8zQKDv6cNxMkKBBwUd 1WpP41lYLXS5lRe/2KfF6Tj+fV1ZPFF5lzIoGq/AphvXYTDaGLI6tnVAJqnSiJChrK3Y DUQr+NEGYBag/V6aB7HG0Vgqdv4UvOUBapdJH21CsYJtdL7P80fxDZDJucPoDXiFt/fi neLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730381839; x=1730986639; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EJAcPRcXZZvXeco/JGOXVlujhVre7WHP9HuHJ7eLCA8=; b=obb+hSL1rCitTQIRN3NJguIFhVarrNZe6PWI9eKDVtlehusU+yqLAZNlj4YtnK9bl1 d9brDJ/HcdcG+xFRw7Sf+3mJBQ2Nxz9Cc1+NRJOyTw5GAHuNr95jEZyf3Kkz9H65+tY8 Ajv6BngZf8VzC+R4aArRpSjEfg7XjcykzPiu15h4bn6KJfUet1BqejbBppu9Orv4rPZy IFd5gRm/X+hRdMKhqBKW/ILzo1fmMNvJvKPkVcwsOvpQieKntLJLnFnAZSE/zLvCwE9y jmhSe5Is38OykhCqF+2w3crvgQH/u09axWilvJX3u1F38/4cQ96EacR/8S+Dm0y3Gqob rxIw== X-Gm-Message-State: AOJu0YzMYsILlnM3fiMrEJfs9rUZ76bWeHaQEOVvKCEoZwCRcUzNkr9G MCGXY0I6PtKna0OTNCEl7ENl1+zaDW2cyKlkXvUK0QUZSkUPxntaNJrNRafCSXL09tNXxnfNdRE cTWLBw3EAlz2pfh//zsLlz8GJxqRJO964 X-Google-Smtp-Source: AGHT+IHA8vlPgs5CrIDOHSBpEldycKMyQ6p3bJ17CRGBhZ0f5IGliGDtNGyctbYAzZhkAOnTkYNDRqPy4TuvoennRV8= X-Received: by 2002:a05:6512:318c:b0:539:918c:5124 with SMTP id 2adb3069b0e04-53b348df188mr9731782e87.31.1730381839161; Thu, 31 Oct 2024 06:37:19 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: =?UTF-8?Q?Vin=C3=ADcius_dos_Santos_Oliveira?= Date: Thu, 31 Oct 2024 10:37:27 -0300 Message-ID: Subject: Re: [rtld] fdlopen(), fdlputpath() and capsicum To: Konstantin Belousov Cc: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4XfQ5t1drwz4qyp X-Spamd-Bar: ---- Em qui., 31 de out. de 2024 =C3=A0s 03:09, Konstantin Belousov escreveu: > Try this https://reviews.freebsd.org/D47351 Your approach would solve not only my problem, but problems like this as well: https://stackoverflow.com/questions/1178094/change-current-process= -environments-ld-library-path Unless someone else has an even better idea, I think we should go with your approach. --=20 Vin=C3=ADcius dos Santos Oliveira https://vinipsmaker.github.io/ From nobody Thu Oct 31 22:23:54 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XfdnV37Ddz5cVLr; Thu, 31 Oct 2024 22:23:58 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XfdnV1gz8z4tq0; Thu, 31 Oct 2024 22:23:58 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1730413438; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=uMG1XdZro36FyxNtkFCnob3r+xI21wRLEEdX2uFrUl8=; b=R2+oJFvv37tkqre8oaiycFihh1VlHMYY1G6WMP9QascZ/RkiqMykiEV5HTODCVjsBEsXE2 9uDVme2/yn60qmLwughotbAt0CV5hnfFcCs7ogtg2cIRbWkjchGUj6UitKv4l5RJGIrXje TArzQs+mVPSTou4Fv0Nh1ehSdfoq4lz/9suocoJdaWPKt/E/PHHU8NN/9j73GSnYp2qLIG 1xY6JrPZuN8G0Ubo8kx+fdMIA4qz5jpkIxRvMtBmnZj5SsdlkJzuKK/WwK3MDtRxtVeMtC VNu9Oel4kG74nXK4t03YDQhGoqFgZSMkIwLDe+CzZigcp8VY3Ts2vP2LimzNdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1730413438; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=uMG1XdZro36FyxNtkFCnob3r+xI21wRLEEdX2uFrUl8=; b=g1c6y/wl6iuyKmeEKBDweiUv3uCt1qm3dJn7p07G+kjl1FjangNh+uGtXCenKBcGTA/s4C v3aBEL03GOP9EhsSgM7JZq/8Ugn/2zD6nmw2+tVhsQ8unwLJk0uR8sO/eiPThnHzG83W1j GENw+Zfzb8s6r0uGscIS88snwZRSYfEPPumNHfjWB4e4/MHl6iaxitI8di6d1TcIgEFa6y Q5qz6vChlqfsdiwe/CdxxmtEiW4WuuekTaI94SRNoeBEs4nQ5gO/yUWZn8xP14SLj4V9V6 aRX/UA4s7qm7G3/hRvccgjMWA/cGriLxRdx71qcwN745DeUWAvsiLDfvYweEqA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1730413438; a=rsa-sha256; cv=none; b=RppXejGHte842hrJ670RrLGskvtwQJCEPUVmUVyTQPkMs/0bZqNoAPhl64vreV2NmDo4xr 0Fez8t1hYiL/6Y3kLEzxF9XUEQdGlN8PUHCs8WbtgNFGlJ6Qd3WgjmGC7Mjs5GNKzZiZcy HBOrFyEaZZLxT0bh5xluQDWhnDJdinntc9dZqFk1+Gca++V9k5dRhHsfkqR29zrngRU6q9 jGUjbDCRxD/VHXjQ/2YpnKwHz9VzOwK796FXQ4uOdWZLL9rs1+UnEZHAVy4NxFjK4LJDQR zSWefvkuGTIW0r2PkcjxMEHFhLJFZfF/w0iGFXLBimsb9Wy6KsnIUCO8sTmukg== Received: from ralga.knownspace (unknown [IPv6:2600:2b00:a720:d301:9f03:382a:d672:81f0]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: jhibbits) by smtp.freebsd.org (Postfix) with ESMTPSA id 4XfdnT5TpDz1Ff7; Thu, 31 Oct 2024 22:23:57 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Date: Thu, 31 Oct 2024 18:23:54 -0400 From: Justin Hibbits To: , Subject: Direct dumped kernel cores Message-ID: <20241031182354.14fa48aa@ralga.knownspace> X-Mailer: Claws Mail 4.3.0 (GTK 3.24.43; powerpc64le-unknown-linux-gnu) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Hi everyone, At Juniper we've been using a so-called 'rescue' kernel for dumping vmcores directly to the filesystem after a panic. We're now contributing this feature, implemented by Klara Systems, to FreeBSD, and looking for feedback. I posted a review at https://reviews.freebsd.org/D47358 for anyone interested. Interesting bits to keep in mind: * It requires a 2-stage build process, one to build the rescue kernel, the other to build the main kernel, which embeds the rescue kernel inside its image. This might need some further work. * Thus far it's been implemented for amd64 and arm64, once proven out, other architectures (powerpc64/le, riscv64) can follow suit. * Kernel environment bits to pass down to the rescue kernel are prefixed `debug.rescue.`, for instance `debug.rescue.vfs.root.mountfrom`. There are many more details in the review summary. We'd love to get feedback from anyone interested. Thanks, Justin Hibbits From nobody Thu Oct 31 22:32:51 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xff005yW6z5cWKL for ; Thu, 31 Oct 2024 22:33:04 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-pf1-x42b.google.com (mail-pf1-x42b.google.com [IPv6:2607:f8b0:4864:20::42b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xff0047y1z3xpk for ; Thu, 31 Oct 2024 22:33:04 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-71e8235f0b6so1206315b3a.3 for ; Thu, 31 Oct 2024 15:33:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1730413983; x=1731018783; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=H26D70ZxWtgAZfgUQiYcsEce7YGQBqm3RU3p92sQdug=; b=rxDBWvNVH+YJYuQGjD7ZswTBjmMOKAhEmj38V16/lbZiE60iaE7a8dz2OvZDrCVJXo U5SoiwIsQbmEONj6pv+EadY5N7nYKDNki5xKQuQG8bov+kUN7a2tzffudvO8RANnZJAA US950XQNz0+7v05tW0OGY5yQT7h0T6CNRrNqxNS9weHFx9QsiLhchl6MPjqUupI9uoPL DvkauR7dIucAt3XnhGBsR9zmyJoPKR+KEETpyTSBjZeZ2HZLP8Ym6rdkig5QiZrUcW/t ZN5W6roRI6CNaoOrA59mcgpzlmcxH4NMhRFM3r/FH3PZVZDVWKfibowvTPXQiwX50lo5 y7EQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730413983; x=1731018783; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=H26D70ZxWtgAZfgUQiYcsEce7YGQBqm3RU3p92sQdug=; b=lNLGUcFNLFxDpC4N/zh9j6JHnN8x9bNzaVjAa1uzgDM8Vsv6Jmwb/5xwWrlXugV1kL +9BZ0c16lku2FWCLSBnlxeZOSrLKVbgjeG/sXy5wIJHkDrDmnuKBrC3pA6kKnnABHdCe sJriJ3EDAVtdj98d85tpEVHm5tcMuJuvDOZ9s9zERQRXPdb7GhP8j/9PdBu5jOEHTMvH 5Iw/fcmhdAegEbcqDHvkhU3TlsdnJXFwW9P1wpNyJNdR6KN5qIqNiBuInmRt8jd9mxGm H236fi6saVLBWQ//yXO5QwkNWdbjoGC9eaRNrUNcsbv3kJyC+s+4go+9zc05mY4dXWqK Hhxg== X-Gm-Message-State: AOJu0YxjsMSszBhUTbwD40n1CgvL9bSUmU02ABGy+UGucaIM0y1VCuNP R8Lv1omv7pIJUahWyJSkCv2ByHfhGxcQf3it7pR91TqXx8JW8NDZsS1gQrVuFZ9WIDwdZ8XurPd D8Qzr5ggBCP7zQtS6y4/JnJhDs219mcgN1sLqbzPJnRClKVBrh9Y= X-Google-Smtp-Source: AGHT+IEiyAAlUCE0W4knf0cQnC4Y/2pjlVTyQ5R2P6HaLmxe3nUJVVmu1r1ODyxm6kaYZHH5Cnjb0ZVIT2VsfpANGGg= X-Received: by 2002:a17:90b:3c46:b0:2e2:d239:84be with SMTP id 98e67ed59e1d1-2e93c159030mr6189766a91.5.1730413982795; Thu, 31 Oct 2024 15:33:02 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: <20241031182354.14fa48aa@ralga.knownspace> In-Reply-To: <20241031182354.14fa48aa@ralga.knownspace> From: Warner Losh Date: Thu, 31 Oct 2024 16:32:51 -0600 Message-ID: Subject: Re: Direct dumped kernel cores To: Justin Hibbits Cc: freebsd-hackers@freebsd.org, freebsd-arch@freebsd.org Content-Type: multipart/alternative; boundary="000000000000c2670d0625cd66ec" X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4Xff0047y1z3xpk X-Spamd-Bar: ---- --000000000000c2670d0625cd66ec Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Oct 31, 2024 at 4:24=E2=80=AFPM Justin Hibbits wrote: > Hi everyone, > > At Juniper we've been using a so-called 'rescue' kernel for dumping > vmcores directly to the filesystem after a panic. We're now > contributing this feature, implemented by Klara Systems, to FreeBSD, and > looking for feedback. I posted a review > at https://reviews.freebsd.org/D47358 for anyone interested. > > Interesting bits to keep in mind: > * It requires a 2-stage build process, one to build the rescue kernel, > the other to build the main kernel, which embeds the rescue kernel > inside its image. This might need some further work. > * Thus far it's been implemented for amd64 and arm64, once proven out, > other architectures (powerpc64/le, riscv64) can follow suit. > * Kernel environment bits to pass down to the rescue kernel are > prefixed `debug.rescue.`, for instance > `debug.rescue.vfs.root.mountfrom`. > First off, this is kinda cool. I've wanted this occasionally when my swap partition is too small (though in my case, it was easy enough to add anothe= r drive to the system that was panicking and dump to that). I do have a question: I'm curious why you didn't follow the Linux lead of having a kexec_load(2) system call to load the 'rescue kernel' to make this more generic. That would make the leap to having full kexec support (eg reboot(CMD_KEXEC) a lot easier to implement. Warner > There are many more details in the review summary. > > We'd love to get feedback from anyone interested. > > Thanks, > Justin Hibbits > > --000000000000c2670d0625cd66ec Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Thu, Oct 31, 2024 at 4:24=E2=80=AF= PM Justin Hibbits <jhibbits@free= bsd.org> wrote:
Hi everyone,

At Juniper we've been using a so-called 'rescue' kernel for dum= ping
vmcores directly to the filesystem after a panic.=C2=A0 We're now
contributing this feature, implemented by Klara Systems, to FreeBSD, and looking for feedback. I posted a review
at https://reviews.freebsd.org/D47358 for anyone interested.
Interesting bits to keep in mind:
* It requires a 2-stage build process, one to build the rescue kernel,
=C2=A0 the other to build the main kernel, which embeds the rescue kernel =C2=A0 inside its image.=C2=A0 This might need some further work.
* Thus far it's been implemented for amd64 and arm64, once proven out,<= br> =C2=A0 other architectures (powerpc64/le, riscv64) can follow suit.
* Kernel environment bits to pass down to the rescue kernel are
=C2=A0 prefixed `debug.rescue.`, for instance
=C2=A0 `debug.rescue.vfs.root.mountfrom`.

First off, this is kinda cool. I've wanted=C2=A0this occasionally wh= en my swap
partition=C2=A0is too small (though in my case, it was= easy enough to add another
drive to the system that was panickin= g and dump to that).

I do have a question: I'm= curious why you didn't follow the Linux lead of having
a kex= ec_load(2) system call to load the 'rescue kernel' to make this mor= e generic.
That would make the leap to having full kexec support = (eg reboot(CMD_KEXEC)
a lot easier to implement.

Warner
=C2=A0
There are many more details in the review summary.

We'd love to get feedback from anyone interested.

Thanks,
Justin Hibbits

--000000000000c2670d0625cd66ec-- From nobody Thu Oct 31 22:33:39 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xff0l4yg8z5b2mv; Thu, 31 Oct 2024 22:33:43 +0000 (UTC) (envelope-from rpokala@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xff0l4Jfpz40v8; Thu, 31 Oct 2024 22:33:43 +0000 (UTC) (envelope-from rpokala@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1730414023; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QUzbJs+AL9SLoQCEzfTEkA2UcHhGE2Gu44JLMQh/ZO8=; b=V6Z07h3dU2NOoNlmLNmxkP0ccxv3X7PVk6Ka4LHA+fy9Gn3cMNlP4bJifG0AbzrQBeFihf J/jspbbKKexLYdFm3bua2xsAMHlZZcp6Yrll7ZbO96kFbIWU7GaxSYMWzxVb6HFjnj4F2W JbwqtzLApZUEO3capo3A2ayzsA6hs38q+q6ga+I8TKW0bPHtHlJ2OY5MybDDfhjq4Zx3Be sLZiYwcfI+3wTwYbisXpykpWbBDk/Dm28zo1gumKsYpHX3AiRBuDNnvtoWV12V5tDOns9Z fknqbVCEwmgF98CwYTn+nctyqx+drNr8fy773XWf40ZhaJq+99o5f8a0vrL4aA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1730414023; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QUzbJs+AL9SLoQCEzfTEkA2UcHhGE2Gu44JLMQh/ZO8=; b=RtQ1/vKqI6l9SshZWYQXmxs+nzy6OCJlCRWiTrjdrBXUm/CWBTpFA4zsZUhNavaD5+S+kf 7+6uOhZLq7ZrrAL7XfbI7cFA+tJj+0U6xMgc0gfv3z+iMCASCRsgyx/i4HRip6WqSWY25a 78AFHINN+LCHmrSc+C3FKOU7Cv4SIbyhUL8se9GFx7v5aJnavQJBzHxsSJmJf/D8JM8qLQ YuP+4J16N1ldN2MuLyLppfYsqajXkBgN2RHy4sXRpE6yA2hF/byCX6kySZUvYtYM8oUjGv iW6C54sSbU2W9ZiuOKWME9o992W6dfm8tELsR4zKqpQTsYNKOalkTsRW/1L8Eg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1730414023; a=rsa-sha256; cv=none; b=VMSKMoTqR7CgPKLSmxLluZFWDu8EhMC+90netp4LtZu/dQQHuOF3K2oy4tTlODI2L1wFML h3PMhRCwsWFSuHdNb91XyRPCtEJ1i/J96kaA+RvPKPubDEB7EH94itRSph37L5ipk6woog qnaWnMuIX8j9cT9WgFzQhSgY6p7PEwqGwMKoDR3p5sHlU5TMmMEl54UiOqviTz1SqTd3rD ZiH+ka6UgqfIq//FXiIn/I0QQZ2vkFwO7cxPsDUHLKDSTqQ8pOWfHw2BPopLrZO0UAgmi0 3j7OPu+EzkSFtuZyfg31naFiE57E3nMgOQWy1BhwGA++ca/2mVRkPg1L4k8GsQ== Received: from [192.168.1.10] (c-73-231-46-254.hsd1.ca.comcast.net [73.231.46.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: rpokala) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Xff0l0Mt6z1FSD; Thu, 31 Oct 2024 22:33:42 +0000 (UTC) (envelope-from rpokala@freebsd.org) User-Agent: Microsoft-MacOutlook/16.90.24102719 Date: Thu, 31 Oct 2024 15:33:39 -0700 Subject: Re: Direct dumped kernel cores From: Ravi Pokala To: Justin Hibbits , , Message-ID: <63B5260F-C835-4BFF-92ED-767AD0CEFE05@panasas.com> Thread-Topic: Direct dumped kernel cores References: <20241031182354.14fa48aa@ralga.knownspace> In-Reply-To: <20241031182354.14fa48aa@ralga.knownspace> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-version: 1.0 Content-type: text/plain; charset="UTF-8" Content-transfer-encoding: quoted-printable Hi Justin, So, this is like the 'crashkernel' thing Linux has, where it kexec()s an al= ternate kernel when the main one panics? I haven't looked at the patch -- most of it will be way out of my expertise= -- but what, if anything, is done to make sure the on-disk state of the tar= get filesystem is okay, before the "rescue" kernel starts writing to it? Thanks, Ravi (rpokala@) =EF=BB=BF-----Original Message----- From: > on behalf of Justin Hibbits > Date: Thursday, October 31, 2024 at 15:23 To: >, > Subject: Direct dumped kernel cores Hi everyone, At Juniper we've been using a so-called 'rescue' kernel for dumping vmcores directly to the filesystem after a panic. We're now contributing this feature, implemented by Klara Systems, to FreeBSD, and looking for feedback. I posted a review at https://reviews.freebsd.org/D47358 = for anyone interested. Interesting bits to keep in mind: * It requires a 2-stage build process, one to build the rescue kernel, the other to build the main kernel, which embeds the rescue kernel inside its image. This might need some further work. * Thus far it's been implemented for amd64 and arm64, once proven out, other architectures (powerpc64/le, riscv64) can follow suit. * Kernel environment bits to pass down to the rescue kernel are prefixed `debug.rescue.`, for instance `debug.rescue.vfs.root.mountfrom`. There are many more details in the review summary. We'd love to get feedback from anyone interested. Thanks, Justin Hibbits From nobody Fri Nov 1 01:07:34 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XfjQH4Hm8z5bH3h; Fri, 01 Nov 2024 01:07:35 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XfjQH3pBTz4JBq; Fri, 1 Nov 2024 01:07:35 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1730423255; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6iEbChIGCuhkZMoX8e4sfyKdKv5aBuFKrd1g4ncZJz8=; b=s4HzTpuQR98BDMoN5cLnUyhqwQu42GZBo9GkQTJd4bEwtKcNULsujoQGKnxFfKoN0q5+Uq Y2Xw4SAhE54fdx0BDpsE8y+hZ5UZb2203SibQ2K0l4PRWUz6DLD2/sd0xWW7AiPMPP2UY/ hNmmUbb8IH+FNAoS1PyLmBR1SDHFQoDN0nFXc9+UY0vy/pLkwzJJdQAO+Bfrc48q4iMglJ oSUZVjDkp0PEa7Va6ggpXgiE+l2sKbXLmhmtL7BMaavecxm7qnngjLbkhH95wBIegzveoV iFIUUVy3yXZ/M2S6E9JdNg5YHYXqtsRZuaCTpA9gRpsg2p1F4W8lpGQgfp15vw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1730423255; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6iEbChIGCuhkZMoX8e4sfyKdKv5aBuFKrd1g4ncZJz8=; b=AQKaajgvRVcA42S6WsN8fiFeq7o/bVJUbxu73nuWPMNVSrrFbB/ZzYDTJ8R/jA5EfQyGuN B6Ir9V5y06Orvke+09bIP3lrIt22iC04i1eiFu+4hyqrnKBlkfxfa2uwaRY8fzZQ0Ah9NK kN0d//DlOqQBpdKGBACmuFBFcArJmJiGdYJfNoQs53wnUGSf7JbCWy8zZ8NIreBFDqkmc5 trAbBvTPquG4FLF00YYGvSyMl6EVrPqkiz42p39DR2WYJtMrrVhR4gR8gs7ODbpjTiDrg+ rW0qlB3MfOmXeAG5Ps4T971ZSPdZLZmsLvqa+bT/Rz19vO21T7mGYFDAHePRbA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1730423255; a=rsa-sha256; cv=none; b=Ee0dMRekntMAe/GAZI8pk999A7XlRWB7aqVsKImYcO+y+fpZTbrUZ1CXOlCclaRN/uX+tn S3vTPYL7ue8jEVmK9frncWDB+ThjhuxJGgMn5ludlyQo/zzIAjvSbZJydI/39gYMk/OUfG SVU4FTTfyQRgDivQAkkkDsKDSf7HEY1I36kp9n0iIj4ua1PjKI9dxSROmDbyphesdh4hk7 y2JW3i4GunW5LbNKyxjkeTmiuSDV5a2r0AJRMu426/eRWXZxMSR5fPw+wR3Ns3Ns/6ntkY 5Qs55Y/S8J0ljUq6KyCdLEYWDAlnDHSz/xZwe3/Sy23jT47nlzeAnjlli7P+DA== Received: from ralga.knownspace (unknown [IPv6:2600:2b00:a720:d301:9f03:382a:d672:81f0]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: jhibbits) by smtp.freebsd.org (Postfix) with ESMTPSA id 4XfjQH2JpKz1JQs; Fri, 1 Nov 2024 01:07:35 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Date: Thu, 31 Oct 2024 21:07:34 -0400 From: Justin Hibbits To: Ravi Pokala Cc: , Subject: Re: Direct dumped kernel cores Message-ID: <20241031210734.283a2fb5@ralga.knownspace> In-Reply-To: <63B5260F-C835-4BFF-92ED-767AD0CEFE05@panasas.com> References: <20241031182354.14fa48aa@ralga.knownspace> <63B5260F-C835-4BFF-92ED-767AD0CEFE05@panasas.com> X-Mailer: Claws Mail 4.3.0 (GTK 3.24.43; powerpc64le-unknown-linux-gnu) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Thu, 31 Oct 2024 15:33:39 -0700 Ravi Pokala wrote: > Hi Justin, >=20 > So, this is like the 'crashkernel' thing Linux has, where it kexec()s > an alternate kernel when the main one panics? If your description is accurate, then probably. I don't know if the crashkernel thing from Linux existed when this was started, and actually hadn't even heard of it until now. >=20 > I haven't looked at the patch -- most of it will be way out of my > expertise -- but what, if anything, is done to make sure the on-disk > state of the target filesystem is okay, before the "rescue" kernel > starts writing to it? Good question. The rescue kernel embeds its own small rootfs it fscks the target fs before writing to it. - Justin >=20 > Thanks, >=20 > Ravi (rpokala@) >=20 > =EF=BB=BF-----Original Message----- > From: > on behalf of Justin Hibbits > > Date: Thursday, > October 31, 2024 at 15:23 To: >, > Subject: Direct dumped kernel cores >=20 >=20 > Hi everyone, >=20 >=20 > At Juniper we've been using a so-called 'rescue' kernel for dumping > vmcores directly to the filesystem after a panic. We're now > contributing this feature, implemented by Klara Systems, to FreeBSD, > and looking for feedback. I posted a review > at https://reviews.freebsd.org/D47358 > for anyone interested. >=20 >=20 > Interesting bits to keep in mind: > * It requires a 2-stage build process, one to build the rescue kernel, > the other to build the main kernel, which embeds the rescue kernel > inside its image. This might need some further work. > * Thus far it's been implemented for amd64 and arm64, once proven out, > other architectures (powerpc64/le, riscv64) can follow suit. > * Kernel environment bits to pass down to the rescue kernel are > prefixed `debug.rescue.`, for instance > `debug.rescue.vfs.root.mountfrom`. >=20 >=20 > There are many more details in the review summary. >=20 >=20 > We'd love to get feedback from anyone interested. >=20 >=20 > Thanks, > Justin Hibbits >=20 >=20 >=20 >=20 >=20 >=20 From nobody Fri Nov 1 01:11:51 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XfjWF1YgKz5bHpg; Fri, 01 Nov 2024 01:11:53 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XfjWF0dhmz4Krw; Fri, 1 Nov 2024 01:11:53 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1730423513; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UCbfWZBo6vDa1tkge/31CJ0yV6YoudWvtuNpzz/RRPg=; b=aBLKVCck3BAMYNym4vKwj9yyCJC+4reYcgwsHhSeofO9huR8wJzLUc5s6q1v92nnDzZ4db uwdHoClUXbaoMvBEDTEQqxkz249EOUBqrGHgHLrH8phcUD5kw/43fldNXNFYC4y2ypvuWa eXKVdP0TtguVUWe18mElKZzYc/UowTo+tV1OwPTVJdU7oxuFvfHCAnLg7YBvOhcvibwa1r 9BSdVIQIHilL6GaALS4lJ/b+ZaizduedQUFxYmtlOPtqgt9aeyn1LGr18NDToYmVto4b0I 9AhlFAGPefTqO4EJEc2jmUa96uekulMOCvtfY65BjsTdQ/5Rpq6/E8ityMomXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1730423513; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UCbfWZBo6vDa1tkge/31CJ0yV6YoudWvtuNpzz/RRPg=; b=ugQWYuHmbOG8v/nBKSgfh4Vkk7uE/hGWyDPKkk797psnCIbhO6S1BE6XGH2+5kuPF8G/bA 9uzs7ykQ7kH8poGZyoq4MlpvdvGEXyktEKyodIYQFZO90sZQx5rJRd9eqV7may2815h4jX CpfGzgavZaHAtSv74GnSg2Ft98IseInzM/YWZXRDzoseHtPf2ANwSnYeSCv7csQ7xzcMSR 0XOdeMWMlKqDQUBRZ1ZdF5tQGiGFPSrKTv/DFor7CVreIaK9+DRiVBkcOzTD6QE8QyTukj 85qDdtSW2s+MbVJVa26olccvIEpXWX8v/BpDD0fKNClXCx8Bq4W9TarNbkwcnQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1730423513; a=rsa-sha256; cv=none; b=YNnrjzv8CX2cb9TfiTcA26NitE05hurHmcAKOZRklXpFE/SG6VGkTn3VckXODcJWX0GpRS ke1/RK1FNbMmnOdmsq+1RKotKbWGqhNRKtVudy27v7YwH70xn+UdaaoqGVJz9LI6EDlSvU oRePaSe2MYlhH9XVSJH7TwQO4cr997QqPYRT0g2SNCUDi9nmqxB3D7L+IQcJDuetMCjuN/ Eyaavu9V2cE9AuPqjEEHpOrbUPgXX+j2CPHlmSMLZDFhZCCxt/picw9RQCuOVM4pNkboJn UQ9oLQfqJfEBDidgqMkbidJXsLA3hzCu7sFKIiWz+f8OONgpT639amwh0wJwvg== Received: from ralga.knownspace (unknown [IPv6:2600:2b00:a720:d301:9f03:382a:d672:81f0]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: jhibbits) by smtp.freebsd.org (Postfix) with ESMTPSA id 4XfjWD6S4rz1HQ2; Fri, 1 Nov 2024 01:11:52 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Date: Thu, 31 Oct 2024 21:11:51 -0400 From: Justin Hibbits To: Warner Losh Cc: freebsd-hackers@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Direct dumped kernel cores Message-ID: <20241031211151.795eba3e@ralga.knownspace> In-Reply-To: References: <20241031182354.14fa48aa@ralga.knownspace> X-Mailer: Claws Mail 4.3.0 (GTK 3.24.43; powerpc64le-unknown-linux-gnu) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Thu, 31 Oct 2024 16:32:51 -0600 Warner Losh wrote: > On Thu, Oct 31, 2024 at 4:24=E2=80=AFPM Justin Hibbits > wrote: >=20 > > Hi everyone, > > > > At Juniper we've been using a so-called 'rescue' kernel for dumping > > vmcores directly to the filesystem after a panic. We're now > > contributing this feature, implemented by Klara Systems, to > > FreeBSD, and looking for feedback. I posted a review > > at https://reviews.freebsd.org/D47358 for anyone interested. > > > > Interesting bits to keep in mind: > > * It requires a 2-stage build process, one to build the rescue > > kernel, the other to build the main kernel, which embeds the rescue > > kernel inside its image. This might need some further work. > > * Thus far it's been implemented for amd64 and arm64, once proven > > out, other architectures (powerpc64/le, riscv64) can follow suit. > > * Kernel environment bits to pass down to the rescue kernel are > > prefixed `debug.rescue.`, for instance > > `debug.rescue.vfs.root.mountfrom`. > > =20 >=20 > First off, this is kinda cool. I've wanted this occasionally when my > swap partition is too small (though in my case, it was easy enough to > add another drive to the system that was panicking and dump to that). >=20 > I do have a question: I'm curious why you didn't follow the Linux > lead of having > a kexec_load(2) system call to load the 'rescue kernel' to make this > more generic. > That would make the leap to having full kexec support (eg > reboot(CMD_KEXEC) a lot easier to implement. >=20 > Warner One problem with trying to kexec_load() a rescue kernel is that the rescue kernel needs its own memory to work with, a contiguous block, so needs to be loaded early, or at least reserved early. Without its reserved memory it would be stomping over the 'host' kernel's memory. That said, I do like that direction, and it's definitely worth exploring. - Justin >=20 >=20 > > There are many more details in the review summary. > > > > We'd love to get feedback from anyone interested. > > > > Thanks, > > Justin Hibbits > > > > =20 From nobody Fri Nov 1 01:39:41 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xfk7Z0r71z5bMLT for ; Fri, 01 Nov 2024 01:39:54 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-pf1-x42c.google.com (mail-pf1-x42c.google.com [IPv6:2607:f8b0:4864:20::42c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xfk7Y66xqz4PLY for ; Fri, 1 Nov 2024 01:39:53 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-71e70c32cd7so1218408b3a.1 for ; Thu, 31 Oct 2024 18:39:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1730425192; x=1731029992; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=JQ+kIjupGU6iGTXg9KApbOdVYwdm9wanbRyqQf8riE4=; b=1V7gakLFoyPar32jhn3XrD3bE1ghrB2H4yGt3UUykPyGxqCKX4bk4rh+nLtaetj1KV 8/c2HESIgETEeA4FvRg4GwG9OUPGkQOGK+ls5TfiOu4NEJparS4gSd8tQ9NQ7lF4M9qU 4O7rHsGaA3zlzu5CUjSrtST5g2dgPJUFcFldj0RvUaLTCljyaAxMpJFRS6a50W7V5Woz Yu3/n56JgJRZfcxSXARZFkgef7RDtY2v+rDIAUbndxsVFSy2cyeFD5JSV936WV+ENpvN 1qCZnBaWvnzaeBamLEH35xqkdMsVgsUTzIvXG6zCZ+fLhvbRJ98WCBGogNYzi6WRpuHS p7DQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730425192; x=1731029992; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=JQ+kIjupGU6iGTXg9KApbOdVYwdm9wanbRyqQf8riE4=; b=b2X4E34LQVSTpECRjrzSmwipzzZOXeyd5hD6caiqNPtUNuUxUyyFZt+iS4aBxH8QOC haS60/0ujQ8dh8kWOXoIrImqRMMY2n/cGEy/IVvDIOU1tX9+0asPEezGPTsIE1nipoTl JBe3RKmwG4yaSc+0MwkaSLIaX7Fb8+K8wwPwCke0LcAXtri3cS8oY71l7xtZlkXwgxC/ BB/1cO52+pq+UXbm3OcIw54vUtiZau/3TYsHW3bH/Xjvk3AFwxB4xlqhgkYFSrcliWh/ YSkHISU3YZHChTZ1qqG9Bf/u05xZgSiwsd+kTs0/yXNHqrKs6KHIEi98CSp45tESHmDK jFQg== X-Forwarded-Encrypted: i=1; AJvYcCUc26V82luzmcOv4jsbVbzUOcjRuXEsdWcoODek5YgKdkuGo8QlXeOpZZwWxrz/pVFUOljelzdBucv17msPw3A=@freebsd.org X-Gm-Message-State: AOJu0Yy8+TBVjU5aKXviYarKSamSQ6RMlw56RLZKmCol9rlRaejWJU5X XCk0SwRg4/g5OmUZh1p2ffRusH/0OyognJ+5hB6KOviF/52yBuRgmjLnk6wbo1ufNn3rvj9ABcn XBCqyd27CEnvRAdNXHx3dS4qZNS2IZ+eu7f6THA== X-Google-Smtp-Source: AGHT+IGgFogaCt9ytVMbIEnJPZfnnuf1LzPl8Kls1WnTSFbgacVQz3DvMEBWHCwr6kXtehHNw2/BcWGRrXXh9rojgWI= X-Received: by 2002:a05:6a20:d8b:b0:1cf:3d14:6921 with SMTP id adf61e73a8af0-1d9a84d168emr29256835637.35.1730425192501; Thu, 31 Oct 2024 18:39:52 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: <20241031182354.14fa48aa@ralga.knownspace> <63B5260F-C835-4BFF-92ED-767AD0CEFE05@panasas.com> <20241031210734.283a2fb5@ralga.knownspace> In-Reply-To: <20241031210734.283a2fb5@ralga.knownspace> From: Warner Losh Date: Thu, 31 Oct 2024 19:39:41 -0600 Message-ID: Subject: Re: Direct dumped kernel cores To: Justin Hibbits Cc: Ravi Pokala , FreeBSD Hackers , "freebsd-arch@freebsd.org" Content-Type: multipart/alternative; boundary="000000000000e9054b0625d00238" X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4Xfk7Y66xqz4PLY X-Spamd-Bar: ---- --000000000000e9054b0625d00238 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Oct 31, 2024, 7:07=E2=80=AFPM Justin Hibbits = wrote: > On Thu, 31 Oct 2024 15:33:39 -0700 > Ravi Pokala wrote: > > > Hi Justin, > > > > So, this is like the 'crashkernel' thing Linux has, where it kexec()s > > an alternate kernel when the main one panics? > > If your description is accurate, then probably. I don't know if the > crashkernel thing from Linux existed when this was started, and > actually hadn't even heard of it until now. > Yeah. There's three types of kernels: debug, crash, and replacement. > > > I haven't looked at the patch -- most of it will be way out of my > > expertise -- but what, if anything, is done to make sure the on-disk > > state of the target filesystem is okay, before the "rescue" kernel > > starts writing to it? > > Good question. The rescue kernel embeds its own small rootfs it fscks > the target fs before writing to it. > Yea. My loader.kboot loads the kernel and initrd too... Warner > - Justin > > > > > Thanks, > > > > Ravi (rpokala@) > > > > =EF=BB=BF-----Original Message----- > > From: > > on behalf of Justin Hibbits > > > Date: Thursday, > > October 31, 2024 at 15:23 To: > >, > > Subject: Direct dumped kernel cores > > > > > > Hi everyone, > > > > > > At Juniper we've been using a so-called 'rescue' kernel for dumping > > vmcores directly to the filesystem after a panic. We're now > > contributing this feature, implemented by Klara Systems, to FreeBSD, > > and looking for feedback. I posted a review > > at https://reviews.freebsd.org/D47358 > > for anyone interested. > > > > > > Interesting bits to keep in mind: > > * It requires a 2-stage build process, one to build the rescue kernel, > > the other to build the main kernel, which embeds the rescue kernel > > inside its image. This might need some further work. > > * Thus far it's been implemented for amd64 and arm64, once proven out, > > other architectures (powerpc64/le, riscv64) can follow suit. > > * Kernel environment bits to pass down to the rescue kernel are > > prefixed `debug.rescue.`, for instance > > `debug.rescue.vfs.root.mountfrom`. > > > > > > There are many more details in the review summary. > > > > > > We'd love to get feedback from anyone interested. > > > > > > Thanks, > > Justin Hibbits > > > > > > > > > > > > > > > --000000000000e9054b0625d00238 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Thu, Oct 31, 2024, 7:07=E2=80=AFPM Justin Hibbits &= lt;jhibbits@freebsd.org> wro= te:
On Thu, 31 Oct 2024 15:33:39 -0= 700
Ravi Pokala <rpokala@freebsd.org> wrote:

> Hi Justin,
>
> So, this is like the 'crashkernel' thing Linux has, where it k= exec()s
> an alternate kernel when the main one panics?

If your description is accurate, then probably.=C2=A0 I don't know if t= he
crashkernel thing from Linux existed when this was started, and
actually hadn't even heard of it until now.

Yeah. There's three type= s of kernels: debug, crash, and replacement.=C2=A0
<= br>
>
> I haven't looked at the patch -- most of it will be way out of my<= br> > expertise -- but what, if anything, is done to make sure the on-disk > state of the target filesystem is okay, before the "rescue" = kernel
> starts writing to it?

Good question.=C2=A0 The rescue kernel embeds its own small rootfs it fscks=
the target fs before writing to it.

Yea. My loader.kboot loads the kernel an= d initrd too...

Warner


- Justin

>
> Thanks,
>
> Ravi (rpokala@)
>
> =EF=BB=BF-----Original Message-----
> From: <owner-freebsd-arch@FreeBSD.org
> <mailto:owner-freebsd-arch@FreeBSD.org>> on b= ehalf of Justin Hibbits
> <jhibbits@FreeBSD.org <mailto:jhibbits@FreeBSD.org>>= Date: Thursday,
> October 31, 2024 at 15:23 To: <freebsd-hackers@FreeBSD.org
> <mailto:freebsd-hackers@FreeBSD.org>>, <freebsd-arch@freebsd.org
> <mailto:freebsd-arch@freebsd.org>> Subject: Direct = dumped kernel cores
>
>
> Hi everyone,
>
>
> At Juniper we've been using a so-called 'rescue' kernel fo= r dumping
> vmcores directly to the filesystem after a panic. We're now
> contributing this feature, implemented by Klara Systems, to FreeBSD, > and looking for feedback. I posted a review
> at https://reviews.freebsd.org/D47358
> <https://reviews.freebsd.org/D47358> for= anyone interested.
>
>
> Interesting bits to keep in mind:
> * It requires a 2-stage build process, one to build the rescue kernel,=
> the other to build the main kernel, which embeds the rescue kernel
> inside its image. This might need some further work.
> * Thus far it's been implemented for amd64 and arm64, once proven = out,
> other architectures (powerpc64/le, riscv64) can follow suit.
> * Kernel environment bits to pass down to the rescue kernel are
> prefixed `debug.rescue.`, for instance
> `debug.rescue.vfs.root.mountfrom`.
>
>
> There are many more details in the review summary.
>
>
> We'd love to get feedback from anyone interested.
>
>
> Thanks,
> Justin Hibbits
>
>
>
>
>
>


--000000000000e9054b0625d00238-- From nobody Fri Nov 1 01:48:53 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XfkLB0TGdz5bNSP for ; Fri, 01 Nov 2024 01:49:06 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XfkL95j9Wz4RCc for ; Fri, 1 Nov 2024 01:49:05 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pj1-x1034.google.com with SMTP id 98e67ed59e1d1-2e2eba31d3aso1156635a91.2 for ; Thu, 31 Oct 2024 18:49:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1730425744; x=1731030544; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=ZgIa43oSCKv7TBLcdTPRvgK19rp0VCREzaHEPW59GOs=; b=K44SUPHndncmLyuL6ToKCacufCxBHI2btrh1CgOSwSosBSVThrcFwuRnCFG7r/5WPw xdlu1nziTfxvNNsnab334g4Y7ZBmr+Mng/MZpXv1njRIfqH5xMIZlkm5mqujsqJL5/nO 87hXuDZMp8bVURriT76i+csyaT5ZUOXwUTxibc2lOwMwtdIzXEUxyGrSnr27IlojuGH7 l+iTYEl8ZfCQXZI23JI0s864HjKFTYHGOfEjI0RXV+RH6oUiP9WZLu3g9UoIM0yLABHg BgnOsBePMICZMhE53hXfQeiOP/cyjjD1E0etsfyAjT8ZkSPUBfVHTihE3BbhRkIXs1Y6 GDZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730425744; x=1731030544; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ZgIa43oSCKv7TBLcdTPRvgK19rp0VCREzaHEPW59GOs=; b=uw93miIwEPNWmi7EjktbjOgXdMY+djIOwQ/YeaitAEfOysozM1Kp6M1YYkNJj2pXKn Bwk4WllbyqgICFBIcKdkDBcqOohjup/HPsVd1SaL07u9TrdIstBtev90hUvG4Q/p3Itt RrWjDklJ8OxPrTaSbaj4JoSTU8VJBgIYxzwKGS8dKB5mb2VskLkWDkgLiAdQ/VzmjVgi Gnw4hpTcqZjE+JUIZ6oYDxfxJfAdN4R1Gr3tHJrSzmxxtcXq+svtU7ZgjeJP+Vqu0btG o7EWA5g+72ncZc2DwxNRrXlGQul1yHvtUs/+DJJNL3AoSDEtWheX7AI2U6FIpNOSyu9R 4xYA== X-Gm-Message-State: AOJu0Yzyczr2mNXBlz/wPX9IjnWkz2ad7361U+9WxP61B5oP1tJAL8dR RLgFKa3C9+9RO+UFeY/r/497W5vBNPvp/9pgThrpnsDmpW1gt7C2uAWVP/zHK7zI5InFTxcTfyY InnvEZ9ya8fULSpAZ5IMiPoKw0P6fRJCY/JCG9/HQfXs7OGgzgZsv8A== X-Google-Smtp-Source: AGHT+IGlO/fhRa2dr7ooHebdla/xfQr7VLhFZD6Zv/bHA1oWvouHxTwoENZX7fNWz/mfK3mUKzsrqfhHAbgy6w9f+9U= X-Received: by 2002:a17:90a:1b8e:b0:2e1:e19f:609b with SMTP id 98e67ed59e1d1-2e8f1088115mr23748499a91.24.1730425744395; Thu, 31 Oct 2024 18:49:04 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: <20241031182354.14fa48aa@ralga.knownspace> <20241031211151.795eba3e@ralga.knownspace> In-Reply-To: <20241031211151.795eba3e@ralga.knownspace> From: Warner Losh Date: Thu, 31 Oct 2024 19:48:53 -0600 Message-ID: Subject: Re: Direct dumped kernel cores To: Justin Hibbits Cc: FreeBSD Hackers , "freebsd-arch@freebsd.org" Content-Type: multipart/alternative; boundary="000000000000ce2d4e0625d02319" X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4XfkL95j9Wz4RCc X-Spamd-Bar: ---- --000000000000ce2d4e0625d02319 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Oct 31, 2024, 7:11=E2=80=AFPM Justin Hibbits = wrote: > On Thu, 31 Oct 2024 16:32:51 -0600 > Warner Losh wrote: > > > On Thu, Oct 31, 2024 at 4:24=E2=80=AFPM Justin Hibbits > > wrote: > > > > > Hi everyone, > > > > > > At Juniper we've been using a so-called 'rescue' kernel for dumping > > > vmcores directly to the filesystem after a panic. We're now > > > contributing this feature, implemented by Klara Systems, to > > > FreeBSD, and looking for feedback. I posted a review > > > at https://reviews.freebsd.org/D47358 for anyone interested. > > > > > > Interesting bits to keep in mind: > > > * It requires a 2-stage build process, one to build the rescue > > > kernel, the other to build the main kernel, which embeds the rescue > > > kernel inside its image. This might need some further work. > > > * Thus far it's been implemented for amd64 and arm64, once proven > > > out, other architectures (powerpc64/le, riscv64) can follow suit. > > > * Kernel environment bits to pass down to the rescue kernel are > > > prefixed `debug.rescue.`, for instance > > > `debug.rescue.vfs.root.mountfrom`. > > > > > > > First off, this is kinda cool. I've wanted this occasionally when my > > swap partition is too small (though in my case, it was easy enough to > > add another drive to the system that was panicking and dump to that). > > > > I do have a question: I'm curious why you didn't follow the Linux > > lead of having > > a kexec_load(2) system call to load the 'rescue kernel' to make this > > more generic. > > That would make the leap to having full kexec support (eg > > reboot(CMD_KEXEC) a lot easier to implement. > > > > Warner > > One problem with trying to kexec_load() a rescue kernel is that the > rescue kernel needs its own memory to work with, a contiguous block, so > needs to be loaded early, or at least reserved early. Without its > reserved memory it would be stomping over the 'host' kernel's > memory. That said, I do like that direction, and it's definitely worth > exploring. > That's exactly what kexec_load does. When the crash happens, the current kernel constructs a new memory map and passes that to the preloaded crash kernel so it knows what memory can safely be used plus info needed to do the crash dump. For the replacement kernel, the reboot copies a miniloader that copies the kernel to the load address, tears the cpu down to the warm reset state and jumps to the trampoline used to start the kernel. Loader.kboot writes that trampoline, creates the EFIlike style metadata and a memory map. And then calls reboot to boot into the new kernel. Warner - Justin > > > > > > > > There are many more details in the review summary. > > > > > > We'd love to get feedback from anyone interested. > > > > > > Thanks, > > > Justin Hibbits > > > > > > > > --000000000000ce2d4e0625d02319 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Thu, Oct 31, 2024, 7:11=E2=80=AFPM Justin Hibbits &= lt;jhibbits@freebsd.org> wro= te:
On Thu, 31 Oct 2024 16:32:51 -0= 600
Warner Losh <imp@bsdimp.com> wrote:

> On Thu, Oct 31, 2024 at 4:24=E2=80=AFPM Justin Hibbits <jhibbits@= freebsd.org>
> wrote:
>
> > Hi everyone,
> >
> > At Juniper we've been using a so-called 'rescue' kern= el for dumping
> > vmcores directly to the filesystem after a panic.=C2=A0 We're= now
> > contributing this feature, implemented by Klara Systems, to
> > FreeBSD, and looking for feedback. I posted a review
> > at https://reviews.freebsd.org/D47358 for= anyone interested.
> >
> > Interesting bits to keep in mind:
> > * It requires a 2-stage build process, one to build the rescue > > kernel, the other to build the main kernel, which embeds the resc= ue
> > kernel inside its image.=C2=A0 This might need some further work.=
> > * Thus far it's been implemented for amd64 and arm64, once pr= oven
> > out, other architectures (powerpc64/le, riscv64) can follow suit.=
> > * Kernel environment bits to pass down to the rescue kernel are > >=C2=A0 =C2=A0prefixed `debug.rescue.`, for instance
> >=C2=A0 =C2=A0`debug.rescue.vfs.root.mountfrom`.
> >=C2=A0
>
> First off, this is kinda cool. I've wanted this occasionally when = my
> swap partition is too small (though in my case, it was easy enough to<= br> > add another drive to the system that was panicking and dump to that).<= br> >
> I do have a question: I'm curious why you didn't follow the Li= nux
> lead of having
> a kexec_load(2) system call to load the 'rescue kernel' to mak= e this
> more generic.
> That would make the leap to having full kexec support (eg
> reboot(CMD_KEXEC) a lot easier to implement.
>
> Warner

One problem with trying to kexec_load() a rescue kernel is that the
rescue kernel needs its own memory to work with, a contiguous block, so
needs to be loaded early, or at least reserved early.=C2=A0 Without its
reserved memory it would be stomping over the 'host' kernel's memory.=C2=A0 That said, I do like that direction, and it's definitely = worth
exploring.

That's exactly what kexec_load does. When the crash happens, = the current kernel constructs a new memory map and passes that to the prelo= aded crash kernel so it knows what memory can safely be used plus info need= ed to do the crash dump.

For the replacement kernel, the reboot copies a miniloader that copies the= kernel to the load address, tears the cpu down to the warm reset state and= jumps to the trampoline used to start the kernel.
<= br>
Loader.kboot writes that trampoline, creates the= EFIlike style metadata and a memory map. And then calls reboot to boot int= o the new kernel.

Warner=

- Justin

>
>
> > There are many more details in the review summary.
> >
> > We'd love to get feedback from anyone interested.
> >
> > Thanks,
> > Justin Hibbits
> >
> >=C2=A0

--000000000000ce2d4e0625d02319--