From nobody Mon Nov 25 08:17:06 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xxdpr6gWHz5fS83; Mon, 25 Nov 2024 08:17:08 +0000 (UTC) (envelope-from des@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xxdpr4kdpz4S4d; Mon, 25 Nov 2024 08:17:08 +0000 (UTC) (envelope-from des@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732522628; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xGOw1XW0lyXNfq8qyh6849nwjVtN3ejMtVyPwhuH4f8=; b=e5FOkAWpHG8Ew+L9o1PpRpUx/mMjwnMZCtCElLXBdQbnFFCSkes7d7CMDCvzx0WrwiDK0Z G1c2lkeeaLolGPdjjlDCvZlfcPnHgS1NZ/WE3tBR0UFJnu60seZKREJtXG1HqEULd3oQDK dlLr0zrak2aGUAVM/IPbYIXBuZCBZKgziK0QiOOnWuByE9y7RjYY2jvB8lA0YEDsZsCTeV 4xSpPsbijsD+kJan03kzo6SmvPY9PXLsLi2h454JkC0csGwYPwh72TevIXXN/r1LMC/HKi wN2y8Y6wy+NCAXUT2vHlZq3EYUvgNNqnwS4ThV0X/Oj4zQPKy97XVI0X8Fdrzw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732522628; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xGOw1XW0lyXNfq8qyh6849nwjVtN3ejMtVyPwhuH4f8=; b=UWA8Mj/izIXyCh+PJqXLO3q7HhLD6uWIle6oHm1AMu1t6xjNDSNxOBz1eOgSisGRuSl6XU tmdttRgXN8L8zaxsGkj/omLUiD8doI/QKZF3Qj7K1Zu5OumNCFfhwM6rVNNvibj/4gpDzF /2yXpvy06MY7nIFXeK8ZFBlWkkWUpbsdBI40piFW/bBBZMQhCE919vRBHUHKAlw9PANXNl nf9FP/7Bhg85k0IMFbD12/BTdVxN/3aXMONfMZv3RuZkSi7P3Hj1JyzHHQqbyuk19qUfwS py1yRF/7S914eMALXa/+Qs2cOUhZX76prjTmHSY74WpZh5QxQUUZAdxk6G1Wbg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1732522628; a=rsa-sha256; cv=none; b=mVpoTfai2k3MZtITkL7hedhSu81Krih3EbvAk9g8geaTDbGiWGQybOuK/aSqZy+HjhN979 Bv8MkbBgYfP5V/UqEkvuC6TODOcYyoDfzA6gBSop7aXU1w2Hu1hE+Qo8NRi4HGSiRTkrDq BfbC8iXO5i4OlPc0a/xZCf8TATQ4i0kRr5c8NzSgiwEYUTv26vwokMlrLJJK6JJwzGQgoM LMAqGwLE4i+R8P+6mZGOP2WVt95bB7jyjRPf1QahS3b1OBhOqBatBCeUE5HpGnm6cxkiqi vyYBawEycpucr2YnsOKZr7vNrS7AaEjNMGztbqaEwIOGxcECZ3j+UjXTzM/i/g== Received: from ltc.des.dev (88-177-82-251.subs.proxad.net [88.177.82.251]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: des) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Xxdpr2zMCz1K3k; Mon, 25 Nov 2024 08:17:08 +0000 (UTC) (envelope-from des@freebsd.org) Received: by ltc.des.dev (Postfix, from userid 1001) id 2BF4A9AD1; Mon, 25 Nov 2024 09:17:06 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Dimitry Andric Cc: Guido Falsi , ports@freebsd.org, FreeBSD Current Subject: Re: port binary dumping core on recent head in poudriere In-Reply-To: <3127C3BA-FC93-4636-ADDB-89518DE9C60D@FreeBSD.org> (Dimitry Andric's message of "Sun, 24 Nov 2024 18:18:51 +0100") References: <46E3A370-A3E0-4BAF-B707-87F94F98E248@FreeBSD.org> <5ee47c3d-f80e-4d50-9b6a-acb3c98e80e0@madpilot.net> <7c9c3cf5-bbd1-4642-8d04-33aa07a4db02@madpilot.net> <9df256a8-c6ed-46d9-b955-fc2657c12d36@madpilot.net> <5c502054-7353-4a1e-8350-c403482e9c0d@madpilot.net> <3127C3BA-FC93-4636-ADDB-89518DE9C60D@FreeBSD.org> User-Agent: Gnus/5.13 (Gnus v5.13) Date: Mon, 25 Nov 2024 09:17:06 +0100 Message-ID: <86ed2zsp6l.fsf@ltc.des.dev> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Dimitry Andric writes: > Probably best to create a bugzilla ticket, but as I said before, I > cannot reproduce this. I can. My builder is running 15 and sees segfaults while building packages for 14 and 15 but not for 13. % uname -a FreeBSD pkg.des.dev 15.0-CURRENT FreeBSD 15.0-CURRENT #0 main-645f8bcba9c: = Mon Nov 18 17:39:08 UTC 2024 root@pkg.des.dev:/usr/obj/poudriere/jails/= 15amd64/usr/src/amd64.amd64/sys/GENERIC-NODEBUG amd64 % poudriere jail -l=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 JAILNAME VERSION ARCH METHOD TIMESTAMP = PATH 13amd64 13.3-RELEASE-p8 amd64 http 2024-11-18 11:59:= 09 /poudriere/jails/13amd64 14amd64 14.1-RELEASE-p6 amd64 http 2024-11-18 12:03:= 11 /poudriere/jails/14amd64 15amd64 15.0-CURRENT 1500027 645f8bcba9c amd64 git+https 2024-11-18 17:57:= 27 /poudriere/jails/15amd64 % grep 'signal 11' /var/log/messages | grep '^Nov [12]' | grep -v conftest Nov 19 16:28:10 pkg kernel: pid 78795 (ngc77601), jid 3, uid 65534: exited = on signal 11 (core dumped) Nov 19 16:59:13 pkg kernel: pid 142 (python3.11), jid 19, uid 65534: exited= on signal 11 (core dumped) Nov 20 03:56:38 pkg kernel: pid 78001 (ngc76093), jid 51, uid 65534: exited= on signal 11 (core dumped) Nov 20 05:52:59 pkg kernel: pid 94860 (python3.11), jid 43, uid 65534: exit= ed on signal 11 (core dumped) Nov 20 06:20:02 pkg kernel: pid 71702 (sassc), jid 59, uid 65534: exited on= signal 11 (core dumped) Nov 20 06:33:26 pkg kernel: pid 20048 (sassc), jid 64, uid 65534: exited on= signal 11 (core dumped) Nov 20 06:33:26 pkg kernel: pid 20050 (sassc), jid 64, uid 65534: exited on= signal 11 (core dumped) Nov 20 06:52:56 pkg kernel: pid 59635 (hugo), jid 45, uid 65534: exited on = signal 11 (core dumped) Nov 20 13:17:40 pkg kernel: pid 53442 (ngc52509), jid 80, uid 65534: exited= on signal 11 (core dumped) Nov 20 14:47:46 pkg kernel: pid 53083 (b), jid 80, uid 65534: exited on sig= nal 11 (core dumped) Nov 20 15:06:58 pkg kernel: pid 72673 (b), jid 99, uid 65534: exited on sig= nal 11 (core dumped) Nov 20 18:09:09 pkg kernel: pid 58589 (python3.11), jid 99, uid 65534: exit= ed on signal 11 (core dumped) Nov 20 20:05:34 pkg kernel: pid 94944 (sassc), jid 71, uid 65534: exited on= signal 11 (core dumped) Nov 20 20:30:56 pkg kernel: pid 60340 (sassc), jid 93, uid 65534: exited on= signal 11 (core dumped) Nov 20 20:30:56 pkg kernel: pid 60349 (sassc), jid 93, uid 65534: exited on= signal 11 (core dumped) Nov 20 20:54:49 pkg kernel: pid 11477 (hugo), jid 73, uid 65534: exited on = signal 11 (core dumped) Nov 22 00:45:20 pkg kernel: pid 66499 (hugo), jid 155, uid 65534: exited on= signal 11 (core dumped) Nov 22 02:49:30 pkg kernel: pid 30727 (sassc), jid 161, uid 65534: exited o= n signal 11 (core dumped) Nov 22 04:16:17 pkg kernel: pid 21254 (sassc), jid 159, uid 65534: exited o= n signal 11 (core dumped) Nov 22 04:16:17 pkg kernel: pid 21264 (sassc), jid 159, uid 65534: exited o= n signal 11 (core dumped) Nov 22 10:39:35 pkg kernel: pid 56707 (b), jid 183, uid 65534: exited on si= gnal 11 (core dumped) Nov 22 10:39:38 pkg kernel: pid 59131 (b), jid 186, uid 65534: exited on si= gnal 11 (core dumped) Nov 22 10:40:39 pkg kernel: pid 37560 (hugo), jid 203, uid 65534: exited on= signal 11 (core dumped) Nov 22 13:34:27 pkg kernel: pid 777 (sassc), jid 191, uid 65534: exited on = signal 11 (core dumped) Nov 22 15:32:56 pkg kernel: pid 63994 (sassc), jid 189, uid 65534: exited o= n signal 11 (core dumped) Nov 22 15:32:56 pkg kernel: pid 63995 (sassc), jid 189, uid 65534: exited o= n signal 11 (core dumped) Nov 23 00:00:10 pkg kernel: pid 20566 (httpd), jid 0, uid 80: exited on sig= nal 11 (no core dump - bad address) Nov 24 21:01:55 pkg kernel: pid 28694 (python3.11), jid 269, uid 65534: exi= ted on signal 11 (core dumped) Nov 24 21:12:22 pkg kernel: pid 56539 (sassc), jid 267, uid 65534: exited o= n signal 11 (core dumped) Nov 24 21:23:06 pkg kernel: pid 71275 (sassc), jid 253, uid 65534: exited o= n signal 11 (core dumped) Nov 24 21:23:06 pkg kernel: pid 71277 (sassc), jid 253, uid 65534: exited o= n signal 11 (core dumped) Nov 24 21:30:06 pkg kernel: pid 13629 (hugo), jid 257, uid 65534: exited on= signal 11 (core dumped) Nov 25 05:21:25 pkg kernel: pid 65922 (b), jid 291, uid 65534: exited on si= gnal 11 (core dumped) Nov 25 05:21:40 pkg kernel: pid 84046 (b), jid 291, uid 65534: exited on si= gnal 11 (core dumped) Nov 25 07:23:57 pkg kernel: pid 40818 (python3.11), jid 299, uid 65534: exi= ted on signal 11 (core dumped) Nov 25 07:37:35 pkg kernel: pid 71735 (sassc), jid 289, uid 65534: exited o= n signal 11 (core dumped) I also see a bunch of SIGABRT from dot but I assume those are unrelated. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@FreeBSD.org From nobody Mon Nov 25 09:01:05 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xxfnd3V58z5fWJX for ; Mon, 25 Nov 2024 09:01:09 +0000 (UTC) (envelope-from glebius@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xxfnd0Tvxz4XBL; Mon, 25 Nov 2024 09:01:09 +0000 (UTC) (envelope-from glebius@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732525269; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=qyJ9nKKSzJ65qGl0Cpqa9lAzoj311C0xaOxApZuK+gs=; b=ExAeraRS+mcok9oslkzOdiwyAFXXHPFwzWHKGFoxc/L2r7d0TEQdMkn4xmWMqjI2GvEINv CuFuwnJIcdiBwTwxD5U22WzWdLWY80JL0MEVRtHiwbmmOpPTkEbJiwYI3UZwM0n338kuE1 PTmz9aJ5OPBHVN/014grIZE9IAYoRTs4mU0ndsuNZktKb7+3BmBVpCWkKOywPIW1Kp4QBT BEOYlVmj++Sukk1qwvagx7i48MBWNZiHCYDn5jUW/9rBbCcpYmMUBC35D5/zRiQxUYqTC7 DyxWgi8OKGHVTU4IEXgBJfk2pBzLxgB1qdl+ozyaKCSRR68WcILAq6/T6gKblw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732525269; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=qyJ9nKKSzJ65qGl0Cpqa9lAzoj311C0xaOxApZuK+gs=; b=JdHPtTXNijnJLG9Sdr68gH2mJhR2T5nrwcw8GAxuIha0ZX+iDLJlrSgi2AR00k3OzcfZrW 2liwigeq/gf4u8BhQd8EzWm/RoaTdJYsySzxWBvKFbhIqg5KCbllDlVggOMJwkMdvFnh8a JrCHv2saYacUPvvy5EETE64sT5n7WgVE3p5/qSY64/o5nH424o67Ab0IFErCbxV1PhJnpF J+n6kOu1VJUdLMH2iMhVcCcC3G7j8JHB+6IYf+Aqan3589sL4VgXPvosu9nTaEcC/tn94b P14+h2Il274rZLcPz3GAfGoAjVxHT1Yv5iPyRoSM++ksKMVLjyHjT8dzHzXPkA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1732525269; a=rsa-sha256; cv=none; b=JrM4pBcpqKl5Fw52ze3AuiGmc46dhBrbg89huZklI/eCPpYluq8K34uIwzW3SmaacbiVLi 2TQZYpzi/0Z2Jhpia/fE3lV/uG+PxUuzEyZcVF9zDxMcAS94jagp8I5LAnYaOrnS/Zcq9q +VCu7eq8tx2uDWpf+wQ6zOk8EkBmXtww4J+Kfb1KaIRwlwhlppNQS1Pk1nbYtafG+yHcSC Eua89nFi5KfK85817mTPu5hFF8w+dxc7xhdRMTgGgQzuRgECLGYUb2j+L9Q/z99aVfQi6l deVjVGYbP3cDFwA76Jwu9+mQPvdSjcEhvMbBBByM757DCPpgg1FCDnY+icZcyQ== Received: from cell.glebi.us (glebi.us [162.251.186.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: glebius) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Xxfnc4WXWz1LDn; Mon, 25 Nov 2024 09:01:08 +0000 (UTC) (envelope-from glebius@freebsd.org) Date: Mon, 25 Nov 2024 01:01:05 -0800 From: Gleb Smirnoff To: freebsd-current@freebsd.org, src-committers@freebsd.org Subject: November 2024 stabilization week Message-ID: List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi FreeBSD/main users & developers: This is an automated email to inform you that the November 2024 stabilization week started with FreeBSD/main at main-n273822-ff4c19bb5427, which was tagged as main-stabweek-2024-Nov. Those who want to participate in the stabilization week are encouraged to update to the above revision/tag and test their systems. The tag main-stabweek-2024-Nov has been published at Gleb Smirnoff's github repo. To connect this repo as an additional remote you need to run: git remote add glebius https://github.com/glebius/FreeBSD Once remote is configured, to checkout the tag run: git fetch glebius --tags git checkout main-stabweek-2024-Nov If you want to use only the official FreeBSD repo, then update to the revision: git pull git checkout ff4c19bb5427 Developers are encouraged to avoid pushing new features to FreeBSD/main during the stabilization week, but focus on bugfixes instead. The stabilization week runs up to Friday 18:00 UTC, but if there is consensus that any regressions discovered by participants have been fixed, it will end early. Once that happens, the advisory freeze of FreeBSD/main branch is thawed. -- Gleb Smirnoff From nobody Mon Nov 25 09:20:54 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XxgDb39DKz5fXXh for ; Mon, 25 Nov 2024 09:21:03 +0000 (UTC) (envelope-from cglogic@protonmail.com) Received: from mail-40138.protonmail.ch (mail-40138.protonmail.ch [185.70.40.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XxgDZ1Vz6z4ZfQ for ; Mon, 25 Nov 2024 09:21:02 +0000 (UTC) (envelope-from cglogic@protonmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=protonmail.com header.s=protonmail3 header.b=jfwHXCHf; spf=pass (mx1.freebsd.org: domain of cglogic@protonmail.com designates 185.70.40.138 as permitted sender) smtp.mailfrom=cglogic@protonmail.com; dmarc=pass (policy=quarantine) header.from=protonmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail3; t=1732526459; x=1732785659; bh=CueqFXGuMszJcY6+jqpXxXnLFl28lX0x/4VRhLIkvqg=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector:List-Unsubscribe:List-Unsubscribe-Post; b=jfwHXCHfhymDOfKv/nDSu9zBNZgsi3I7TVnV2cBvV8T1pP/lUURTyP6tRhv23Ht50 bREGmhi4aooLGdGPa6znYbDEoB2NfK3palOCiMsGliFdeA3i8NJAYIdwoeHIMCZ/qw WQwv6tPK/maR7YJJRPn7MFbukkdzZwafb/AAGOweJtsa/mnvoGlmRWEqEq453GuGwN MqIh5wMue/iIYNmc5ct4coME/l4X9XNert/uE8lY4wfN4uO2iulZYwvpEYAm1mYfnM 8TJYmhlAyBjyygb9l1JD8hs56xx5Y5jpgIHaW+pUxB3WcGD7UmVkt/kp8g87Eh66aT ChIYvIj/WT68w== Date: Mon, 25 Nov 2024 09:20:54 +0000 To: FreeBSD CURRENT From: cglogic Cc: Minsoo Choo , Warner Losh Subject: Re: Long time outdated jemalloc Message-ID: In-Reply-To: References: Feedback-ID: 25313618:user:proton X-Pm-Message-ID: 5270a27caecce4ca07bd8d3ed28d5efaeaaf9e80 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="b1=_vl02V6Ci2mX2eUT6hUgUhSgbrJUbwGQyaUqY5v18wI" X-Spamd-Result: default: False [-2.86 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-0.53)[-0.531]; DMARC_POLICY_ALLOW(-0.50)[protonmail.com,quarantine]; NEURAL_HAM_SHORT(-0.23)[-0.230]; R_SPF_ALLOW(-0.20)[+ip4:185.70.40.0/24]; R_DKIM_ALLOW(-0.20)[protonmail.com:s=protonmail3]; RWL_MAILSPIKE_VERYGOOD(-0.20)[185.70.40.138:from]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MIME_BASE64_TEXT(0.10)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; MISSING_XM_UA(0.00)[]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH]; FROM_HAS_DN(0.00)[]; FREEMAIL_ENVFROM(0.00)[protonmail.com]; FREEMAIL_FROM(0.00)[protonmail.com]; TO_DN_ALL(0.00)[]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_ZERO(0.00)[0]; RCPT_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[protonmail.com:+] X-Rspamd-Queue-Id: 4XxgDZ1Vz6z4ZfQ X-Spamd-Bar: -- --b1=_vl02V6Ci2mX2eUT6hUgUhSgbrJUbwGQyaUqY5v18wI Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 SGVsbG8gZ3V5cywKCkhvdyB0aGUgdXBkYXRlIG9mIGplbWFsbG9jIGlzIGdvaW5nPyBJdCdzIE5v dmVtYmVyIG5vdy4KClRoYW5rcy4KT24gTW9uZGF5LCBKdWx5IDIybmQsIDIwMjQgYXQgNzowMiBQ TSwgTWluc29vIENob28gPG1pbnNvb2Nob28wMTIyQHByb3Rvbi5tZT4gd3JvdGU6Cgo+IEZpcnN0 LCBzb3JyeSBmb3IgbGF0ZSByZXNwb25zZS4KPgo+IGNnbG9naWMsIHRoYW5rIHlvdSBmb3IgYnJp bmdpbmcgdXAgdGhpcyBpc3N1ZSBhZ2FpbiBzaW5jZSBJIG5lYXJseSBmb3Jnb3QgdGhhdCB0aGlz IGlzc3VlIHdhcyBzdGlsbCBvcGVuLgo+Cj4gV2FybmVyLCBhcyBJIGNhbid0IGFjY2VzcyB0byBt eSBGcmVlQlNEIGluc3RhbmNlIHVudGlsIHRoZSBlbmQgb2YgQXVndXN0LCBidXQgSSBjYW4gc3Rp bGwgZWRpdCBhbmQgcHVzaCB0aGUgY29kZSB0aHJvdWdoIG15IEFybSBNYWMuIFRoaXMgbWVhbnMg dGhhdCBJIGNhbid0IHRlc3QgdGhlIHVwZGF0ZWQgY29kZSBvbiBteSBtYWNoaW5lLCBidXQgSSBj YW4gam9pbiB0aGUgcmV2aWV3IHByb2Nlc3MgYW5kIGxpc3RlbiB0byBjaGFuZ2UgcHJvcG9zYWxz Lgo+Cj4gSSdsbCBvcGVuIGEgR2l0aHViIFBSIGluIGEgZmV3IGhvdXJzLiAoVGhlIHBoYWJyaWNh dG9yIHJldmlldyB3aWxsIHN0YXkgb3BlbmVkIGp1c3QgaW4gY2FzZSkKPiBPbiBNb25kYXksIEp1 bHkgMjJuZCwgMjAyNCBhdCA1OjA4IEFNLCBXYXJuZXIgTG9zaCA8aW1wQGJzZGltcC5jb20+IHdy b3RlOgo+Cj4+IE9uIFN1biwgSnVsIDIxLCAyMDI0IGF0IDI6MDPigK9QTSBjZ2xvZ2ljIDxjZ2xv Z2ljQHByb3Rvbm1haWwuY29tPiB3cm90ZToKPj4KPj4+IE9uIFN1bmRheSwgSnVseSAyMXN0LCAy MDI0IGF0IDY6NTQgQU0sIFdhcm5lciBMb3NoIDxpbXBAYnNkaW1wLmNvbT4gd3JvdGU6Cj4+Pgo+ Pj4+IE9uIFNhdCwgSnVsIDIwLCAyMDI0IGF0IDE6NTnigK9BTSBjZ2xvZ2ljIDxjZ2xvZ2ljQHBy b3Rvbm1haWwuY29tPiB3cm90ZToKPj4+Pgo+Pj4+PiBIZWxsbyBGcmVlQlNEIGNvbW11bml0eSwK Pj4+Pj4KPj4+Pj4gQWZ0ZXIgSmFzb24gRXZhbnMgc3RlcHBlZCBhc2lkZSBmcm9tIG1haW50YWlu aW5nIGplbWFsbG9jIGluIEZyZWVCU0QsIGl0J3Mgbm90IHVwZGF0aW5nIGluIHRpbWUgYW55bW9y ZS4KPj4+Pj4gVmVyc2lvbiA1LjMuMCB3YXMgcmVsZWFzZWQgTWF5IDYsIDIwMjIgYW5kIEZyZWVC U0Qgc3RpbGwgbm90IGltcG9ydGVkIGl0IGludG8gdGhlIHRyZWUuCj4+Pj4+Cj4+Pj4+IFRoZXJl IGlzIGEgcGVuZGluZyByZXZpZXcgaHR0cHM6Ly9yZXZpZXdzLmZyZWVic2Qub3JnL0Q0MTQyMSBm cm9tIEF1ZyAxMSwgMjAyMy4KPj4+Pj4gSSdtIHN1Y2Nlc3NmdWxseSBydW5uaW5nIEZyZWVCU0Qv YW1kNjQgc3lzdGVtIHdpdGggRDQxNDIxIGFwcGxpZWQgZm9yIDggbW9udGhzLCBhcyB3ZWxsIGFz IG1hbnkgb3RoZXIgcGVvcGxlLgo+Pj4+Pgo+Pj4+PiBDYW4gaXQgYmUgcmV2aWV3ZWQgYW5kIGNv bW1pdHRlZCB0byBDVVJSRU5UPwo+Pj4+PiBPciwgaWYgdGhlcmUgaXMgbm8gY29tbWl0dGVycyB3 aWxsaW5nIHRvIGRvIGl0LCBjYW4gY29tbWl0IGJpdCBiZSBnaXZlbiB0byBzdWJtaXR0ZXIgb3Ig YW5vdGhlciBwZXJzb24gd2lsbGluZyB0byBkbyB0aGlzPwo+Pj4+Pgo+Pj4+PiBJdCdzIHZlcnkg ZGlzYXBwb2ludGluZyB3aGVuIHVzZXJzIHNwZW5kIHRoZWlyIHRpbWUgdG8gZmlsbCBzdWNoIGdh cHMgYW5kIHRoZWlyIGVmZm9ydHMganVzdCBpZ25vcmVkIGJ5IHRoZSBkZXZlbG9wZXJzLgo+Pj4+ Pgo+Pj4+PiBFdmVyeSB5ZWFyIEZyZWVCU0QgQ29tbXVuaXR5IFN1cnZleSBhc2tpbmcgYWJvdXQg dXNlciBleHBlcmllbmNlIGluIGNvbnRyaWJ1dGluZyB0byBGcmVlQlNELgo+Pj4+Pgo+Pj4+PiBI ZXJlIHlvdSBjYW4gc2VlIGFuIGV4YW1wbGUgb2Ygc3VjaCBjb250cmlidXRpbmcuCj4+Pj4KPj4+ PiBGaXJzdCwgdGhhbmsgeW91IGZvciBiZWluZyBwZXJzaXN0ZW50IGFuZCBjb250aW51aW5nIHRv IGJyaW5nIGl0IHVwLiBJdCdzIGltcG9ydGFudCB0byBkbyB0aGF0IHRvIG1ha2Ugc3VyZSB0aGlz IChhbmQgeW91ciBtYW55IG90aGVyKSBjb250cmlidXRpb24gZG9lc24ndCBmYWxsIG9uIHRoZSBm bG9vci4KPj4+Pgo+Pj4+IEFuZCB0byBiZSBmYWlyLCB3ZSdyZSBvbmx5IDMgbW9udGhzIHNpbmNl IHRoZSBsYXN0IHVwZGF0ZS4gU3RpbGwsIHF1aXRlIGEgYml0IGxvbmdlciB0aGFuIHlvdSBzaG91 bGQgaGF2ZSB0byB3YWl0LCBidXQgbm90IG5lYXJseSB0aGUgeWVhciB0aGUgb3JpZ2luYWwgZGF0 ZSBzdWdnZXN0cy4KPj4+Pgo+Pj4+IEFuZCB0aGlzIGlzIGEgcGVyZmVjdCBzdG9ybSBvZiAiaG93 IHRoZSBwcm9qZWN0IGlzIGJhZCBhdCBhY2NlcHRpbmcgY29udHJpYnV0aW9ucyI6Cj4+Pj4gKDEp IFRoZSBvcmlnaW5hbCBzdWJtaXNzaW9uIHdhcyBjbG9zZSB0byB0aGUgMTQgYnJhbmNoIGNyZWF0 aW9uIHRpbWUuIFRoaXMgbWVhbnQgdGhhdCB3ZSB3ZXJlbid0IHdlbGwgcHJlcGFyZWQgdG8gbG9v ayBhdCBpdCBzaW5jZSBpdCBpcyBzdWNoIGFuIGludmFzaXZlIGNoYW5nZSAoYXQgbGVhc3Qgb24g aXRzIHN1cmZhY2UpLiBJdCBhbHNvIHNsb3dlZCB0aGUgaW5pdGlhbCByZXNwb25zZS4uLgo+Pj4+ ICgyKSBUaGVyZSB3YXMgYSBudW1iZXIgb2YgYmFjayBhbmQgZm9ydGggcmVxdWVzdHMgZm9yIGNo YW5nZXMsIHdoaWNoIHRvb2sgdGltZSB0byBzb3J0IG91dC4uLgo+Pj4+ICgzKSBUaGUgc2l6ZSBv ZiB0aGlzIGlzIGh1Z2UsIHdlbGwgYmV5b25kIHRoZSBjYXBhY2l0eSBvZiBQaGFicmljYXRvciB0 byByZXZpZXcgYWNjdXJhdGVseS4uLgo+Pj4+ICg0KSBJdCdzIGEgdmVuZG9yIGltcG9ydC4gVGhh dCBtZWFucyB3ZSBjYW4ndCBqdXN0IGRyb3AgdGhlIFBoYWJyaWNhdG9yIHJldmlldyBpbnRvIHRo ZSB0cmVlLi4uCj4+Pj4gKDUpIEl0J3MgcGhhYnJpY2F0b3I6IHRoaXMgaXMgYSBncmVhdCB0b29s IGZvciBkZXZlbG9wZXJzLCBidXQgd2UgaGF2ZSBhIHRlcnJpYmxlIHRyYWNrIHJlY29yZCBvZiB1 c2luZyBpdCBmb3IgaW50YWtlIGZyb20gbmV3IGNvbnRyaWJ1dG9ycy4gV2UgZG9uJ3QgaGF2ZSBh bnkgb3ZlcnNpZ2h0IGF0IGFsbCBvdmVyIHRoaXMgdG9vbCwgYXQgdGhlcmUncyBhdCBiZXN0IHRl cGlkIGFuZCBsdWtlIHdhcm0gYXR0ZW1wdHMgdG8gbG9vayBmb3IgZHJvcCBiYWxscy4KPj4+Pgo+ Pj4+IEFsbCBvZiB0aGVzZSB0aGluZ3MgYXJlIGEgdGVycmlibGUgZXhwZXJpZW5jZS4gSSBjYW4g b25seSBhcG9sb2dpemUuIFRoZXNlIGRheXMsIHdlIG1pZ2h0IHN0ZWVyIHRoaXMgdG93YXJkcyBn aXRodWIsIGJ1dCB0aGUgJ3ZlbmRvciBpbXBvcnQnIG1lYW5zIHlvdSByZWFsbHkgbmVlZCBzb21l b25lIG9uIHRoZSBpbnNpZGUsIG9yIHlvdSBuZWVkIHRvIGJlIG9uIHRoZSBpbnNpZGUgdG8gbWFr ZSB0aGF0IHdvcmsuCj4+Pj4KPj4+PiBTbywgaG93IHRvIG1vdmUgZm9yd2FyZD8gV2VsbCwgSSdk IGxpa2UgdG8gcHJvcG9zZSB0aGUgZm9sbG93aW5nOgo+Pj4+ICgxKSBzdWJtaXQgYWxsIHRoZSBv dGhlciBQaGFicmljYXRvciByZXZpZXdzIHlvdSBoYXZlIG9wZW4gKHRoZXkgYXJlIG1vc3RseSBn b29kLCBvciBjbG9zZSB0byBnb29kKSB0byBnaXRodWIuIEdpdGh1YiBpcyBiZWluZyBhY3RpdmVs eSBtYW5hZ2VkIGFuZCB3aWxsIG1ha2UgaXQgZmFzdGVyIHRvIGdldCB0aGluZ3MgaXQuIEl0J3Mg YSBtdWNoIGJldHRlciB0b29sIGZvciBuZXcgY29udHJpYnV0b3JzIChhbmQgZXZlbiBmcmVxdWVu dCBjb250cmlidXRvcnMgb2Ygc21hbGxpc2ggdGhpbmdzKS4KPj4+PiAoMikgSSBzaG91bGQgZG8g YW4gdmVuZG9yIGltcG9ydCBvZiA1LjMuMCBmcm9tIGdpdGh1YiwgYW5kIGRvIHRoZSBtZXJnZSB0 byBhIGJyYW5jaCBhbmQgcHVzaCB0aGF0IHRvIGdpdGh1Yi4gWW91IGNhbiB0aGVuIGxheWVyIG9u IHlvdXIgY2hhbmdlcyBhbmQgdGhvc2UgY2FuIGJlIHJldmlld2VkIG1vcmUgY2xvc2VseSBhcyBh IHB1bGwgcmVxdWVzdCBhZ2FpbnN0IHRoZSBicmFuY2ggSSBwdXNoLiBJIHN1c3BlY3QgdGhhdCBt b3N0IG9mIHRoZSBpc3N1ZXMgYXJlIHNvcnRlZCBvdXQgYWxyZWFkeQo+Pj4+ICgzKSBJJ2xsIGxh bmQgaXQgdmlhIHRoYXQgcm91dGUuLi4KPj4+Pgo+Pj4+IEFuZCwgaWYgdGhlIHN1bSBvZiB0aGUg b3RoZXIgcHVsbCByZXF1ZXN0cyBhbmQgdGhpcyBhcmUgZ29vZCAoYW5kIEkgc3VzcGVjdCB0aGV5 IHdpbGwgYmUpLCB0aGVuIHdlIGNhbiB0YWxrIGFib3V0IGNvbW1pdCBiaXRzIGFuZCBzdWNoLgo+ Pj4+Cj4+Pj4gSXQncyBleHBlcmllbmNlcyBsaWtlIHRoaXMgd2hpY2ggaXMgd2h5IEknbSB0cnlp bmcgdG8gc3RhbmQgdXAgZ2l0aHViIHB1bGwgcmVxdWVzdHMgYXMgYSByZWxpYWJsZSB3YXkgdG8g Z2V0IHRoaW5ncyBhbmQgYW5kIHRoZSBiZXN0IHBsYWNlIHRvIHNlbmQgcGVvcGxlLi4uCj4+Pj4K Pj4+PiBUaGFua3MgYWdhaW4gZm9yIHBlcnNpc3RpbmcsIGFuZCBhbHNvIGZvciBleHByZXNzaW5n IHRoaXMgY3JpdGljaXNtIHRoYXQgd2UgKGhvcGVmdWxseSkgY2FuIHVzZSB0byBtYWtlIGl0IGJl dHRlci4KPj4+Pgo+Pj4+IFdhcm5lcgo+Pj4KPj4+IEhlbGxvLgo+Pj4KPj4+IEknbSBub3QgdGhl IGF1dGhvciBvZiBENDE0MjEuIEp1c3QgYXBwbGllZCB0aGUgcGF0Y2ggdG8gdGVzdCBpdCA4IG1v bnRocyBhZ28uIEFuZCByZWNlbnRseSBkaXNjb3ZlcmVkIHRoYXQgaXQncyBzdGlsbCBub3QgY29t bWl0dGVkLgo+Pj4gSSBjYW4ndCBjb3B5IHlvdXIgbWVzc2FnZSB0byBQaGFicmljYXRvciBiZWNh dXNlIGRvbid0IGhhdmUgYW4gYWNjb3VudC4gUGxlYXNlLCBpZiB5b3UgaGF2ZSB0aW1lLCBoZWxw IHRoZSBhdXRob3IgaW4gRDQxNDIxLgo+Pgo+PiBBaCB5ZXMuIEkndmUgYmVlbiBpbiB0b3VjaCB3 aXRoIHRoZSBhdXRob3IgZm9yIG90aGVyIHRoaW5ncywgYW5kIHNvbWVob3cgdGhvdWdodCBpdCB3 YXMgeW91Li4uLiBJJ2xsIHJlYWNoIG91dCB0byBoaW0gdmlhIG90aGVyIG1lYW5zLi4uCj4+Cj4+ IFdhcm5lcg== --b1=_vl02V6Ci2mX2eUT6hUgUhSgbrJUbwGQyaUqY5v18wI Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: base64 PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0 cHg7Ij5IZWxsbyBndXlzLDwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fu cy1zZXJpZjsgZm9udC1zaXplOiAxNHB4OyI+PGJyPjwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFt aWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxNHB4OyI+SG93IHRoZSB1cGRhdGUg b2YgamVtYWxsb2MgaXMgZ29pbmc/IEl0J3MgTm92ZW1iZXIgbm93LjwvZGl2PjxkaXYgc3R5bGU9 ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxNHB4OyI+PGJyPjwv ZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9udC1zaXpl OiAxNHB4OyI+VGhhbmtzLjwvZGl2PjxkaXYgY2xhc3M9InByb3Rvbm1haWxfcXVvdGUiPg0KICAg ICAgICBPbiBNb25kYXksIEp1bHkgMjJuZCwgMjAyNCBhdCA3OjAyIFBNLCBNaW5zb28gQ2hvbyAm bHQ7bWluc29vY2hvbzAxMjJAcHJvdG9uLm1lJmd0OyB3cm90ZTo8YnI+DQogICAgICAgIDxibG9j a3F1b3RlIGNsYXNzPSJwcm90b25tYWlsX3F1b3RlIiB0eXBlPSJjaXRlIj4NCiAgICAgICAgICAg IDxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAx NHB4OyI+Rmlyc3QsIHNvcnJ5IGZvciBsYXRlIHJlc3BvbnNlLjwvZGl2PjxkaXYgc3R5bGU9ImZv bnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxNHB4OyI+PGJyPjwvZGl2 PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAx NHB4OyI+Y2dsb2dpYywgdGhhbmsgeW91IGZvciBicmluZ2luZyB1cCB0aGlzIGlzc3VlIGFnYWlu IHNpbmNlIEkgbmVhcmx5IGZvcmdvdCB0aGF0IHRoaXMgaXNzdWUgd2FzIHN0aWxsIG9wZW4uPC9k aXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6 IDE0cHg7Ij48YnI+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNl cmlmOyBmb250LXNpemU6IDE0cHg7Ij5XYXJuZXIsIGFzIEkgY2FuJ3QgYWNjZXNzIHRvIG15IEZy ZWVCU0QgaW5zdGFuY2UgdW50aWwgdGhlIGVuZCBvZiBBdWd1c3QsIGJ1dCBJIGNhbiBzdGlsbCBl ZGl0IGFuZCBwdXNoIHRoZSBjb2RlIHRocm91Z2ggbXkgQXJtIE1hYy4gVGhpcyBtZWFucyB0aGF0 IEkgY2FuJ3QgdGVzdCB0aGUgdXBkYXRlZCBjb2RlIG9uIG15IG1hY2hpbmUsIGJ1dCBJIGNhbiBq b2luIHRoZSByZXZpZXcgcHJvY2VzcyBhbmQgbGlzdGVuIHRvIGNoYW5nZSBwcm9wb3NhbHMuPC9k aXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6 IDE0cHg7Ij48YnI+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNl cmlmOyBmb250LXNpemU6IDE0cHg7Ij5JJ2xsIG9wZW4gYSBHaXRodWIgUFIgaW4gYSBmZXcgaG91 cnMuIChUaGUgcGhhYnJpY2F0b3IgcmV2aWV3IHdpbGwgc3RheSBvcGVuZWQganVzdCBpbiBjYXNl KTwvZGl2PjxkaXYgY2xhc3M9InByb3Rvbm1haWxfcXVvdGUiPg0KICAgICAgICBPbiBNb25kYXks IEp1bHkgMjJuZCwgMjAyNCBhdCA1OjA4IEFNLCBXYXJuZXIgTG9zaCAmbHQ7aW1wQGJzZGltcC5j b20mZ3Q7IHdyb3RlOjxicj4NCiAgICAgICAgPGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSIgY2xhc3M9 InByb3Rvbm1haWxfcXVvdGUiPg0KICAgICAgICAgICAgPGRpdiBkaXI9Imx0ciI+PGRpdiBkaXI9 Imx0ciI+PGJyPjwvZGl2Pjxicj48ZGl2IGNsYXNzPSJnbWFpbF9xdW90ZSI+PGRpdiBkaXI9Imx0 ciIgY2xhc3M9ImdtYWlsX2F0dHIiPk9uIFN1biwgSnVsIDIxLCAyMDI0IGF0IDI6MDPigK9QTSBj Z2xvZ2ljICZsdDs8YSB0YXJnZXQ9Il9ibGFuayIgcmVsPSJub3JlZmVycmVyIG5vZm9sbG93IG5v b3BlbmVyIiBocmVmPSJtYWlsdG86Y2dsb2dpY0Bwcm90b25tYWlsLmNvbSI+Y2dsb2dpY0Bwcm90 b25tYWlsLmNvbTwvYT4mZ3Q7IHdyb3RlOjxicj48L2Rpdj48YmxvY2txdW90ZSBjbGFzcz0iZ21h aWxfcXVvdGUiIHN0eWxlPSJtYXJnaW46MHB4IDBweCAwcHggMC44ZXg7Ym9yZGVyLWxlZnQ6MXB4 IHNvbGlkIHJnYigyMDQsMjA0LDIwNCk7cGFkZGluZy1sZWZ0OjFleCI+PGRpdiBzdHlsZT0iZm9u dC1mYW1pbHk6QXJpYWwsc2Fucy1zZXJpZjtmb250LXNpemU6MTRweCI+PGJyPjwvZGl2PjxkaXY+ DQogICAgICAgIE9uIFN1bmRheSwgSnVseSAyMXN0LCAyMDI0IGF0IDY6NTQgQU0sIFdhcm5lciBM b3NoICZsdDs8YSByZWw9Im5vcmVmZXJyZXIgbm9mb2xsb3cgbm9vcGVuZXIiIGhyZWY9Im1haWx0 bzppbXBAYnNkaW1wLmNvbSIgdGFyZ2V0PSJfYmxhbmsiPmltcEBic2RpbXAuY29tPC9hPiZndDsg d3JvdGU6PGJyPg0KICAgICAgICA8YmxvY2txdW90ZSB0eXBlPSJjaXRlIj4NCiAgICAgICAgICAg IDxkaXYgZGlyPSJsdHIiPjxkaXYgZGlyPSJsdHIiPjxicj48L2Rpdj48YnI+PGRpdiBjbGFzcz0i Z21haWxfcXVvdGUiPjxkaXYgZGlyPSJsdHIiIGNsYXNzPSJnbWFpbF9hdHRyIj5PbiBTYXQsIEp1 bCAyMCwgMjAyNCBhdCAxOjU54oCvQU0gY2dsb2dpYyAmbHQ7PGEgcmVsPSJub3JlZmVycmVyIG5v Zm9sbG93IG5vb3BlbmVyIiBocmVmPSJtYWlsdG86Y2dsb2dpY0Bwcm90b25tYWlsLmNvbSIgdGFy Z2V0PSJfYmxhbmsiPmNnbG9naWNAcHJvdG9ubWFpbC5jb208L2E+Jmd0OyB3cm90ZTo8YnI+PC9k aXY+PGJsb2NrcXVvdGUgY2xhc3M9ImdtYWlsX3F1b3RlIiBzdHlsZT0ibWFyZ2luOjBweCAwcHgg MHB4IDAuOGV4O2JvcmRlci1sZWZ0OjFweCBzb2xpZCByZ2IoMjA0LDIwNCwyMDQpO3BhZGRpbmct bGVmdDoxZXgiPjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OkFyaWFsLHNhbnMtc2VyaWY7Zm9udC1z aXplOjE0cHg7Y29sb3I6cmdiKDAsMCwwKTtiYWNrZ3JvdW5kLWNvbG9yOnJnYigyNTUsMjU1LDI1 NSkiPkhlbGxvIEZyZWVCU0QgY29tbXVuaXR5LDwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5 OkFyaWFsLHNhbnMtc2VyaWY7Zm9udC1zaXplOjE0cHg7Y29sb3I6cmdiKDAsMCwwKTtiYWNrZ3Jv dW5kLWNvbG9yOnJnYigyNTUsMjU1LDI1NSkiPjxicj48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZh bWlseTpBcmlhbCxzYW5zLXNlcmlmO2ZvbnQtc2l6ZToxNHB4O2NvbG9yOnJnYigwLDAsMCk7YmFj a2dyb3VuZC1jb2xvcjpyZ2IoMjU1LDI1NSwyNTUpIj48c3BhbiBzdHlsZT0iZGlzcGxheTppbmxp bmU7YmFja2dyb3VuZC1jb2xvcjpyZ2IoMjU1LDI1NSwyNTUpIj5BZnRlciA8L3NwYW4+PHNwYW4g c3R5bGU9ImJhY2tncm91bmQtY29sb3I6cmdiKDI1NSwyNTUsMjU1KSI+SmFzb24gRXZhbnMgc3Rl cHBlZCBhc2lkZSBmcm9tIG1haW50YWluaW5nIGplbWFsbG9jIGluIEZyZWVCU0QsIGl0J3Mgbm90 IHVwZGF0aW5nIGluIHRpbWUgYW55bW9yZS48L3NwYW4+PGJyPjwvZGl2PjxkaXYgc3R5bGU9ImZv bnQtZmFtaWx5OkFyaWFsLHNhbnMtc2VyaWY7Zm9udC1zaXplOjE0cHg7Y29sb3I6cmdiKDAsMCww KTtiYWNrZ3JvdW5kLWNvbG9yOnJnYigyNTUsMjU1LDI1NSkiPlZlcnNpb24gNS4zLjAgd2FzIHJl bGVhc2VkIDxzcGFuPk1heSA2LCAyMDIyIGFuZCBGcmVlQlNEIHN0aWxsIG5vdCBpbXBvcnRlZCBp dCBpbnRvIHRoZSB0cmVlLjwvc3Bhbj48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTpBcmlh bCxzYW5zLXNlcmlmO2ZvbnQtc2l6ZToxNHB4O2NvbG9yOnJnYigwLDAsMCk7YmFja2dyb3VuZC1j b2xvcjpyZ2IoMjU1LDI1NSwyNTUpIj48c3Bhbj48YnI+PC9zcGFuPjwvZGl2PjxkaXYgc3R5bGU9 ImZvbnQtZmFtaWx5OkFyaWFsLHNhbnMtc2VyaWY7Zm9udC1zaXplOjE0cHg7Y29sb3I6cmdiKDAs MCwwKTtiYWNrZ3JvdW5kLWNvbG9yOnJnYigyNTUsMjU1LDI1NSkiPlRoZXJlIGlzIGEgcGVuZGlu ZyByZXZpZXcgPHNwYW4+PGEgcmVsPSJub3JlZmVycmVyIG5vZm9sbG93IG5vb3BlbmVyIiBocmVm PSJodHRwczovL3Jldmlld3MuZnJlZWJzZC5vcmcvRDQxNDIxIiB0YXJnZXQ9Il9ibGFuayI+aHR0 cHM6Ly9yZXZpZXdzLmZyZWVic2Qub3JnL0Q0MTQyMTwvYT4gZnJvbSA8c3Bhbj5BdWcgMTEsIDIw MjMuPC9zcGFuPjwvc3Bhbj48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTpBcmlhbCxzYW5z LXNlcmlmO2ZvbnQtc2l6ZToxNHB4O2NvbG9yOnJnYigwLDAsMCk7YmFja2dyb3VuZC1jb2xvcjpy Z2IoMjU1LDI1NSwyNTUpIj48c3Bhbj48c3Bhbj5JJ20gc3VjY2Vzc2Z1bGx5IHJ1bm5pbmcgRnJl ZUJTRC9hbWQ2NCBzeXN0ZW0gd2l0aCBENDE0MjEgYXBwbGllZCBmb3IgOCBtb250aHMsIGFzIHdl bGwgYXMgbWFueSBvdGhlciBwZW9wbGUuPC9zcGFuPjwvc3Bhbj48L2Rpdj48ZGl2IHN0eWxlPSJm b250LWZhbWlseTpBcmlhbCxzYW5zLXNlcmlmO2ZvbnQtc2l6ZToxNHB4O2NvbG9yOnJnYigwLDAs MCk7YmFja2dyb3VuZC1jb2xvcjpyZ2IoMjU1LDI1NSwyNTUpIj48c3Bhbj48c3Bhbj48YnI+PC9z cGFuPjwvc3Bhbj48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTpBcmlhbCxzYW5zLXNlcmlm O2ZvbnQtc2l6ZToxNHB4O2NvbG9yOnJnYigwLDAsMCk7YmFja2dyb3VuZC1jb2xvcjpyZ2IoMjU1 LDI1NSwyNTUpIj48c3Bhbj48c3Bhbj5DYW4gaXQgYmUgcmV2aWV3ZWQgYW5kIGNvbW1pdHRlZCB0 byBDVVJSRU5UPzwvc3Bhbj48L3NwYW4+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6QXJp YWwsc2Fucy1zZXJpZjtmb250LXNpemU6MTRweDtjb2xvcjpyZ2IoMCwwLDApO2JhY2tncm91bmQt Y29sb3I6cmdiKDI1NSwyNTUsMjU1KSI+PHNwYW4+PHNwYW4+T3IsIGlmIHRoZXJlIGlzIG5vIGNv bW1pdHRlcnMgd2lsbGluZyB0byBkbyBpdCwgY2FuIGNvbW1pdCBiaXQgYmUgZ2l2ZW4gdG8gc3Vi bWl0dGVyIG9yIGFub3RoZXIgcGVyc29uIHdpbGxpbmcgdG8gZG8gdGhpcz88L3NwYW4+PC9zcGFu PjwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OkFyaWFsLHNhbnMtc2VyaWY7Zm9udC1zaXpl OjE0cHg7Y29sb3I6cmdiKDAsMCwwKTtiYWNrZ3JvdW5kLWNvbG9yOnJnYigyNTUsMjU1LDI1NSki PjxzcGFuPjxzcGFuPjxicj48L3NwYW4+PC9zcGFuPjwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFt aWx5OkFyaWFsLHNhbnMtc2VyaWY7Zm9udC1zaXplOjE0cHg7Y29sb3I6cmdiKDAsMCwwKTtiYWNr Z3JvdW5kLWNvbG9yOnJnYigyNTUsMjU1LDI1NSkiPjxzcGFuPjxzcGFuPjxzcGFuPkl0J3MgdmVy eSBkaXNhcHBvaW50aW5nIHdoZW4gdXNlcnMgc3BlbmQgdGhlaXIgdGltZSB0byBmaWxsIHN1Y2gg Z2FwcyBhbmQgdGhlaXIgZWZmb3J0cyBqdXN0IGlnbm9yZWQgYnkgdGhlIGRldmVsb3BlcnMuPC9z cGFuPjxicj48L3NwYW4+PC9zcGFuPjwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OkFyaWFs LHNhbnMtc2VyaWY7Zm9udC1zaXplOjE0cHg7Y29sb3I6cmdiKDAsMCwwKTtiYWNrZ3JvdW5kLWNv bG9yOnJnYigyNTUsMjU1LDI1NSkiPjxzcGFuPjxzcGFuPjxzcGFuPjxzcGFuPkV2ZXJ5IHllYXIg RnJlZUJTRCBDb21tdW5pdHkgU3VydmV5IGFza2luZyBhYm91dCB1c2VyIGV4cGVyaWVuY2UgaW4g Y29udHJpYnV0aW5nIHRvIEZyZWVCU0QuIDwvc3Bhbj48YnI+PC9zcGFuPjwvc3Bhbj48L3NwYW4+ PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6QXJpYWwsc2Fucy1zZXJpZjtmb250LXNpemU6 MTRweDtjb2xvcjpyZ2IoMCwwLDApO2JhY2tncm91bmQtY29sb3I6cmdiKDI1NSwyNTUsMjU1KSI+ PHNwYW4+PHNwYW4+PHNwYW4+PHNwYW4+SGVyZSB5b3UgY2FuIHNlZSBhbiBleGFtcGxlIG9mIHN1 Y2ggY29udHJpYnV0aW5nLjwvc3Bhbj48L3NwYW4+PC9zcGFuPjwvc3Bhbj48L2Rpdj48ZGl2IHN0 eWxlPSJmb250LWZhbWlseTpBcmlhbCxzYW5zLXNlcmlmO2ZvbnQtc2l6ZToxNHB4O2NvbG9yOnJn YigwLDAsMCk7YmFja2dyb3VuZC1jb2xvcjpyZ2IoMjU1LDI1NSwyNTUpIj48c3Bhbj48c3Bhbj48 c3Bhbj48L3NwYW4+PC9zcGFuPjwvc3Bhbj48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTpB cmlhbCxzYW5zLXNlcmlmO2ZvbnQtc2l6ZToxNHB4O2NvbG9yOnJnYigwLDAsMCk7YmFja2dyb3Vu ZC1jb2xvcjpyZ2IoMjU1LDI1NSwyNTUpIj48c3Bhbj48YnI+PC9zcGFuPjwvZGl2PjwvYmxvY2tx dW90ZT48ZGl2Pjxicj48L2Rpdj48ZGl2PkZpcnN0LCB0aGFuayB5b3UgZm9yIGJlaW5nIHBlcnNp c3RlbnQgYW5kIGNvbnRpbnVpbmcgdG8gYnJpbmcgaXQgdXAuIEl0J3MgaW1wb3J0YW50IHRvIGRv IHRoYXQgdG8gbWFrZSBzdXJlIHRoaXMgKGFuZCB5b3VyIG1hbnkgb3RoZXIpIGNvbnRyaWJ1dGlv biBkb2Vzbid0IGZhbGwgb24gdGhlIGZsb29yLjxicj48L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2 PkFuZCB0byBiZSBmYWlyLCB3ZSdyZSBvbmx5IDMgbW9udGhzIHNpbmNlIHRoZSBsYXN0IHVwZGF0 ZS4gU3RpbGwsIHF1aXRlIGEgYml0IGxvbmdlciB0aGFuIHlvdSBzaG91bGQgaGF2ZSB0byB3YWl0 LCBidXQgbm90IG5lYXJseSB0aGUgeWVhciB0aGUgb3JpZ2luYWwgZGF0ZSBzdWdnZXN0cy48YnI+ PC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj5BbmQgdGhpcyBpcyBhIHBlcmZlY3Qgc3Rvcm0gb2Yg ImhvdyB0aGUgcHJvamVjdCBpcyBiYWQgYXQgYWNjZXB0aW5nIGNvbnRyaWJ1dGlvbnMiOjwvZGl2 PjxkaXY+KDEpIFRoZSBvcmlnaW5hbCBzdWJtaXNzaW9uIHdhcyBjbG9zZSB0byB0aGUgMTQgYnJh bmNoIGNyZWF0aW9uIHRpbWUuIFRoaXMgbWVhbnQgdGhhdCB3ZSB3ZXJlbid0IHdlbGwgcHJlcGFy ZWQgdG8gbG9vayBhdCBpdCBzaW5jZSBpdCBpcyBzdWNoIGFuIGludmFzaXZlIGNoYW5nZSAoYXQg bGVhc3Qgb24gaXRzIHN1cmZhY2UpLiBJdCBhbHNvIHNsb3dlZCB0aGUgaW5pdGlhbCByZXNwb25z ZS4uLjxicj48L2Rpdj48ZGl2PigyKSBUaGVyZSB3YXMgYSBudW1iZXIgb2YgYmFjayBhbmQgZm9y dGggcmVxdWVzdHMgZm9yIGNoYW5nZXMsIHdoaWNoIHRvb2sgdGltZSB0byBzb3J0IG91dC4uLjwv ZGl2PjxkaXY+KDMpIFRoZSBzaXplIG9mIHRoaXMgaXMgaHVnZSwgd2VsbCBiZXlvbmQgdGhlIGNh cGFjaXR5IG9mIFBoYWJyaWNhdG9yIHRvIHJldmlldyBhY2N1cmF0ZWx5Li4uPC9kaXY+PGRpdj4o NCkgSXQncyBhIHZlbmRvciBpbXBvcnQuIFRoYXQgbWVhbnMgd2UgY2FuJ3QganVzdCBkcm9wIHRo ZSBQaGFicmljYXRvciByZXZpZXcgaW50byB0aGUgdHJlZS4uLjwvZGl2PjxkaXY+KDUpIEl0J3Mg cGhhYnJpY2F0b3I6IHRoaXMgaXMgYSBncmVhdCB0b29sIGZvciBkZXZlbG9wZXJzLCBidXQgd2Ug aGF2ZSBhIHRlcnJpYmxlIHRyYWNrIHJlY29yZCBvZiB1c2luZyBpdCBmb3IgaW50YWtlIGZyb20g bmV3IGNvbnRyaWJ1dG9ycy4gV2UgZG9uJ3QgaGF2ZSBhbnkgb3ZlcnNpZ2h0IGF0IGFsbCBvdmVy IHRoaXMgdG9vbCwgYXQgdGhlcmUncyBhdCBiZXN0IHRlcGlkIGFuZCBsdWtlIHdhcm0gYXR0ZW1w dHMgdG8gbG9vayBmb3IgZHJvcCBiYWxscy48L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2PkFsbCBv ZiB0aGVzZSB0aGluZ3MgYXJlIGEgdGVycmlibGUgZXhwZXJpZW5jZS4gSSBjYW4gb25seSBhcG9s b2dpemUuIFRoZXNlIGRheXMsIHdlIG1pZ2h0IHN0ZWVyIHRoaXMgdG93YXJkcyBnaXRodWIsIGJ1 dCB0aGUgJ3ZlbmRvciBpbXBvcnQnIG1lYW5zIHlvdSByZWFsbHkgbmVlZCBzb21lb25lIG9uIHRo ZSBpbnNpZGUsIG9yIHlvdSBuZWVkIHRvIGJlIG9uIHRoZSBpbnNpZGUgdG8gbWFrZSB0aGF0IHdv cmsuPC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj5TbywgaG93IHRvIG1vdmUgZm9yd2FyZD8gV2Vs bCwgSSdkIGxpa2UgdG8gcHJvcG9zZSB0aGUgZm9sbG93aW5nOjwvZGl2PjxkaXY+KDEpIHN1Ym1p dCBhbGwgdGhlIG90aGVyIFBoYWJyaWNhdG9yIHJldmlld3MgeW91IGhhdmUgb3BlbiAodGhleSBh cmUgbW9zdGx5IGdvb2QsIG9yIGNsb3NlIHRvIGdvb2QpIHRvIGdpdGh1Yi4gR2l0aHViIGlzIGJl aW5nIGFjdGl2ZWx5IG1hbmFnZWQgYW5kIHdpbGwgbWFrZSBpdCBmYXN0ZXIgdG8gZ2V0IHRoaW5n cyBpdC4gSXQncyBhIG11Y2ggYmV0dGVyIHRvb2wgZm9yIG5ldyBjb250cmlidXRvcnMgKGFuZCBl dmVuIGZyZXF1ZW50IGNvbnRyaWJ1dG9ycyBvZiBzbWFsbGlzaCB0aGluZ3MpLjwvZGl2PjxkaXY+ KDIpIEkgc2hvdWxkIGRvIGFuIHZlbmRvciBpbXBvcnQgb2YgNS4zLjAgZnJvbSBnaXRodWIsIGFu ZCBkbyB0aGUgbWVyZ2UgdG8gYSBicmFuY2ggYW5kIHB1c2ggdGhhdCB0byBnaXRodWIuIFlvdSBj YW4gdGhlbiBsYXllciBvbiB5b3VyIGNoYW5nZXMgYW5kIHRob3NlIGNhbiBiZSByZXZpZXdlZCBt b3JlIGNsb3NlbHkgYXMgYSBwdWxsIHJlcXVlc3QgYWdhaW5zdCB0aGUgYnJhbmNoIEkgcHVzaC4g SSBzdXNwZWN0IHRoYXQgbW9zdCBvZiB0aGUgaXNzdWVzIGFyZSBzb3J0ZWQgb3V0IGFscmVhZHkg PGJyPjwvZGl2PjxkaXY+KDMpIEknbGwgbGFuZCBpdCB2aWEgdGhhdCByb3V0ZS4uLjwvZGl2Pjxk aXY+PGJyPjwvZGl2PjxkaXY+QW5kLCBpZiB0aGUgc3VtIG9mIHRoZSBvdGhlciBwdWxsIHJlcXVl c3RzIGFuZCB0aGlzIGFyZSBnb29kIChhbmQgSSBzdXNwZWN0IHRoZXkgd2lsbCBiZSksIHRoZW4g d2UgY2FuIHRhbGsgYWJvdXQgY29tbWl0IGJpdHMgYW5kIHN1Y2guPC9kaXY+PGRpdj48YnI+PC9k aXY+PGRpdj5JdCdzIGV4cGVyaWVuY2VzIGxpa2UgdGhpcyB3aGljaCBpcyB3aHkgSSdtIHRyeWlu ZyB0byBzdGFuZCB1cCBnaXRodWIgcHVsbCByZXF1ZXN0cyBhcyBhIHJlbGlhYmxlIHdheSB0byBn ZXQgdGhpbmdzIGFuZCBhbmQgdGhlIGJlc3QgcGxhY2UgdG8gc2VuZCBwZW9wbGUuLi4gIDxicj48 L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2PlRoYW5rcyBhZ2FpbiBmb3IgcGVyc2lzdGluZywgYW5k IGFsc28gZm9yIGV4cHJlc3NpbmcgdGhpcyBjcml0aWNpc20gdGhhdCB3ZSAoaG9wZWZ1bGx5KSBj YW4gdXNlIHRvIG1ha2UgaXQgYmV0dGVyLjxicj48L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2Pldh cm5lcjxicj48L2Rpdj48L2Rpdj48L2Rpdj4NCg0KICAgICAgICA8L2Jsb2NrcXVvdGU+PC9kaXY+ PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6QXJpYWwsc2Fucy1zZXJpZjtmb250LXNpemU6MTRweDtj b2xvcjpyZ2IoMCwwLDApO2JhY2tncm91bmQtY29sb3I6cmdiKDI1NSwyNTUsMjU1KSI+PGJyPjwv ZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OkFyaWFsLHNhbnMtc2VyaWY7Zm9udC1zaXplOjE0 cHg7Y29sb3I6cmdiKDAsMCwwKTtiYWNrZ3JvdW5kLWNvbG9yOnJnYigyNTUsMjU1LDI1NSkiPkhl bGxvLjwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OkFyaWFsLHNhbnMtc2VyaWY7Zm9udC1z aXplOjE0cHg7Y29sb3I6cmdiKDAsMCwwKTtiYWNrZ3JvdW5kLWNvbG9yOnJnYigyNTUsMjU1LDI1 NSkiPjxicj48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTpBcmlhbCxzYW5zLXNlcmlmO2Zv bnQtc2l6ZToxNHB4O2NvbG9yOnJnYigwLDAsMCk7YmFja2dyb3VuZC1jb2xvcjpyZ2IoMjU1LDI1 NSwyNTUpIj5JJ20gbm90IHRoZSBhdXRob3Igb2YgPHNwYW4+RDQxNDIxLiBKdXN0IGFwcGxpZWQg dGhlIHBhdGNoIHRvIHRlc3QgaXQgOCBtb250aHMgYWdvLiBBbmQgcmVjZW50bHkgZGlzY292ZXJl ZCB0aGF0IGl0J3Mgc3RpbGwgbm90IGNvbW1pdHRlZC48L3NwYW4+PC9kaXY+PGRpdiBzdHlsZT0i Zm9udC1mYW1pbHk6QXJpYWwsc2Fucy1zZXJpZjtmb250LXNpemU6MTRweDtjb2xvcjpyZ2IoMCww LDApO2JhY2tncm91bmQtY29sb3I6cmdiKDI1NSwyNTUsMjU1KSI+PHNwYW4+SSBjYW4ndCBjb3B5 IHlvdXIgbWVzc2FnZSB0byBQaGFicmljYXRvciBiZWNhdXNlIGRvbid0IGhhdmUgYW4gYWNjb3Vu dC4gPC9zcGFuPlBsZWFzZSwgaWYgeW91IGhhdmUgdGltZSwgaGVscCB0aGUgYXV0aG9yIGluIEQ0 MTQyMS48L2Rpdj48L2Jsb2NrcXVvdGU+PGRpdj48YnI+PC9kaXY+PGRpdj5BaCB5ZXMuIEkndmUg YmVlbiBpbiB0b3VjaCB3aXRoIHRoZSBhdXRob3IgZm9yIG90aGVyIHRoaW5ncywgYW5kIHNvbWVo b3cgdGhvdWdodCBpdCB3YXMgeW91Li4uLiAgSSdsbCByZWFjaCBvdXQgdG8gaGltIHZpYSBvdGhl ciBtZWFucy4uLjwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+V2FybmVyPC9kaXY+PC9kaXY+PC9k aXY+DQoNCiAgICAgICAgPC9ibG9ja3F1b3RlPjxicj4NCiAgICA8L2Rpdj4NCiAgICAgICAgPC9i bG9ja3F1b3RlPjxicj4NCiAgICA8L2Rpdj4= --b1=_vl02V6Ci2mX2eUT6hUgUhSgbrJUbwGQyaUqY5v18wI-- From nobody Mon Nov 25 20:24:41 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XxxyZ2cxmz5f8Sn for ; Mon, 25 Nov 2024 20:24:54 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-io1-f50.google.com (mail-io1-f50.google.com [209.85.166.50]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XxxyY4t1zz4jRq for ; Mon, 25 Nov 2024 20:24:53 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of carpeddiem@gmail.com designates 209.85.166.50 as permitted sender) smtp.mailfrom=carpeddiem@gmail.com; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=freebsd.org (policy=none) Received: by mail-io1-f50.google.com with SMTP id ca18e2360f4ac-84198253284so46506539f.3 for ; Mon, 25 Nov 2024 12:24:53 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732566292; x=1733171092; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=WROf7AXlWD5blUA+Ef4lL+RhQJ9Q0B9+refFaBH4aOM=; b=g9iZSWiKPkiBlgqL+CQkfayfi0nJmXEEBxfKVrrgY0MyqHm1a84iedDkEPtG2pmvu8 j4K++uQSPMLMLxyXf8WcySdGpRAQYLwHK8KJkMSIJvmzMg8AipZ5hykEdMUikrWbJ6dY UoCTyQLZ7K6DOCRiiO47aI71MN7eZngN3dVf/fcCW1wdWHr/XjcpB5E73DsPbhuAXPVP DpcJWERMCqw46MIkBLrBl9uJRZr1DazvEzoS1O91n/QlTAePNEwmbDNIRQ60Jvh6uDhh 84aIMJuyZQghUzanttmpAxpWsmyR/hD3ER+7FAL9JHU0w7TzR7ZvbWr3z44OdJQnbqv1 LplA== X-Gm-Message-State: AOJu0Yw+MgvJIOsg549dmFRDwVnPDHdfgrwR8uwiPlFCrrcF5UgGywiI veyeuucWvDElGAXZ8zDbOniz4m7xMt/F3iladUfAD14owcfPDrwX1skjx4iE1JvAcl8ZeOPnNpq a2ZYXfOsnZVLbmSmTKz+qLVI4mj/CXUs6 X-Gm-Gg: ASbGncsK7Q6ItpcD7q3rZvAGiiJnbNoHBGRhBXWgqi/d1Ymf6mnfB6Yu6iuu11uSaSn 81HU9lQHWtcpK8ql+FgrLFGh/xP0aEbns5UhuOgJwQUrr8tGTK64o1GmrrbBEz8II8Q== X-Google-Smtp-Source: AGHT+IH6eM/XsqSgotw9sedX8CP0QnsJegaJT84cs0zzmiSJgFAp0aTr3zAiZ3asGpw3IruNAqDA17Cd7Q2KdxCiBlo= X-Received: by 2002:a05:6602:490:b0:82a:3552:6b26 with SMTP id ca18e2360f4ac-83ecdd3a01bmr1447187839f.15.1732566292510; Mon, 25 Nov 2024 12:24:52 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 From: Ed Maste Date: Mon, 25 Nov 2024 15:24:41 -0500 Message-ID: Subject: IBM ServeRAID M5110e and other mfi/mrsas users - call for testing / request for information To: FreeBSD Current Content-Type: text/plain; charset="UTF-8" X-Spamd-Result: default: False [-0.97 / 15.00]; NEURAL_HAM_MEDIUM(-0.99)[-0.993]; NEURAL_HAM_SHORT(-0.98)[-0.979]; NEURAL_SPAM_LONG(0.91)[0.907]; FORGED_SENDER(0.30)[emaste@freebsd.org,carpeddiem@gmail.com]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : SPF not aligned (relaxed), No valid DKIM,none]; MIME_GOOD(-0.10)[text/plain]; TO_DN_ALL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; FROM_NEQ_ENVFROM(0.00)[emaste@freebsd.org,carpeddiem@gmail.com]; MISSING_XM_UA(0.00)[]; R_DKIM_NA(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.166.50:from]; TO_DOM_EQ_FROM_DOM(0.00)[]; FREEFALL_USER(0.00)[carpeddiem]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; RCVD_IN_DNSWL_NONE(0.00)[209.85.166.50:from] X-Rspamd-Queue-Id: 4XxxyY4t1zz4jRq X-Spamd-Bar: / I have a review open to switch to mrsas(4) by default, for devices supported by both it and mfi(4). mrsas(4) should be better supported and more functional. The review: https://reviews.freebsd.org/D45476 mfi: default to using mrsas(4) for newer cards However, we have a report of volume corruption with the IBM ServeRAID M5110e when used with the mrsas driver: PR196820 IBM x3650, ServeRAID M5110e, volumes corrupted after boot with mrsas driver Clearly we do not want to corrupt users' RAID volumes, so I would like to request feedback from mfi/mrsas users in general, and M5110e users in particular. Note that there may be a possibility of data corruption, so please investigate only on test systems. To test, set the loader tunable hw.mfi.mrsas_enable=1 in /boot/loader.conf. Confirm that mrsas is being used, and that the system functions properly. From nobody Mon Nov 25 20:41:24 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XxyKq03pjz5f9fk; Mon, 25 Nov 2024 20:41:35 +0000 (UTC) (envelope-from mad@madpilot.net) Received: from vogon.madpilot.net (vogon.madpilot.net [159.69.1.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XxyKp4zJtz4lYF; Mon, 25 Nov 2024 20:41:34 +0000 (UTC) (envelope-from mad@madpilot.net) Authentication-Results: mx1.freebsd.org; none Received: from mail (mail [IPv6:fd5c:5351:d272::3]) by vogon.madpilot.net (Postfix) with ESMTP id 4XxyKg2LbpzL7vm; Mon, 25 Nov 2024 21:41:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=madpilot.net; h= content-transfer-encoding:content-type:content-type:in-reply-to :from:from:content-language:references:subject:subject:date:date :message-id:received; s=cyq4qetkgngm; t=1732567285; x= 1734381686; bh=82y3906ZPGIkGJatSsTMWUa1h6smNOzMEk96Dv8++Vs=; b=G hIewfOd3/F1i/54hYw0+hHswmvcTNf4VVqyZ4R8hdyoma9SZFx0ZAEAaA32SevXg NnEI9JJZMMHIQhDWdXA0ExsXiGSNOzdQZQ4UPvU63Am0SUhUHUryT1IWzWSdVwbs OD0xVpc3kMcgv4Lc0xNFf7PXZ0RlDo25zxawmdToHL1x7wwvJOOwYdxdCACnptDE yExnsKpjGfT5VSHPC5OjzTvJJPwmikgnOgujdCfTNk+TO9CuJ6AOmB1e+O5dEIRP PWI9qZPPjOf1sb3WaHm3a07txcznf5qIDKXX5hz5NRk7bnFTmSrOQmFk3d2wTZB4 nZwGXDUYQ20dtZkzoNAWA== Received: from vogon.madpilot.net ([IPv6:fd5c:5351:d272::3]) by mail (vogon.madpilot.net [IPv6:fd5c:5351:d272::3]) (amavisd-new, port 10026) with ESMTP id 39Qtc-Qwjj6a; Mon, 25 Nov 2024 21:41:25 +0100 (CET) Message-ID: <5f24a570-26e0-4c0a-817f-591a234fd07b@madpilot.net> Date: Mon, 25 Nov 2024 21:41:24 +0100 Subject: Re: port binary dumping core on recent head in poudriere To: =?UTF-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , Dimitry Andric , marklmi@yahoo.com Cc: ports@freebsd.org, FreeBSD Current References: <46E3A370-A3E0-4BAF-B707-87F94F98E248@FreeBSD.org> <5ee47c3d-f80e-4d50-9b6a-acb3c98e80e0@madpilot.net> <7c9c3cf5-bbd1-4642-8d04-33aa07a4db02@madpilot.net> <9df256a8-c6ed-46d9-b955-fc2657c12d36@madpilot.net> <5c502054-7353-4a1e-8350-c403482e9c0d@madpilot.net> <3127C3BA-FC93-4636-ADDB-89518DE9C60D@FreeBSD.org> <86ed2zsp6l.fsf@ltc.des.dev> Content-Language: en-US From: Guido Falsi Autocrypt: addr=mad@madpilot.net; keydata= xsBNBE+G+l0BCADi/WBQ0aRJfnE7LBPsM0G3m/m3Yx7OPu4iYFvS84xawmRHtCNjWIntsxuX fptkmEo3Rsw816WUrek8dxoUAYdHd+EcpBcnnDzfDH5LW/TZ4gbrFezrHPdRp7wdxi23GN80 qPwHEwXuF0X4Wy5V0OO8B6VT/nA0ADYnBDhXS52HGIJ/GCUjgqJn+phDTdCFLvrSFdmgx4Wl c0W5Z1p5cmDF9l8L/hc959AeyNf7I9dXnjekGM9gVv7UDUYzCifR3U8T0fnfdMmS8NeI9NC+ wuREpRO4lKOkTnj9TtQJRiptlhcHQiAlG1cFqs7EQo57Tqq6cxD1FycZJLuC32bGbgalABEB AAHNHkd1aWRvIEZhbHNpIDxtYWRAbWFkcGlsb3QubmV0PsLAeQQTAQgAIwIbAwIeAQIXgAUL CQgHAwUVCgkICwQWAgMBBQJS79AgAhkBAAoJEBrmhg5Wy9KTc0kH/RO64ORBlTbTHaUaOj8F Je5O5NU2Pt9Cyt5ZWBRvxntr1zPTJGKRPS9ihlIfqT4ZvEngQGp57EUyFbCpI0UWasTerImM tt5WACnGmCzUTB39UXx8Oy4b1EgWeTJQ747e/F1mQLXTNa6ijRBE9fYlTb4gAkPN88/wVV9v 3PZozKLTg16ghBzHM/P7Lk8L7clPEZChX1FTa/6eSt3nvzfCuTMZbBPJF/ph+q1KyPqRgVfh tyhu5dvgMoPz/ni41IfeSrkJTD5RXzdyGR9q4Z1NYeBsLkRjC4LxKAP5KqUsvlOUjKvO1byj ApYdMarol+IGkaSk9e3zVYAJkWKjn/ni8XbOwU0EUxB7QQEQAKFhrDceoPdK/IHDSmoj6SQY isvM7VdhcleS7E9DoEAVt7yMbf6HbbMVTTY6ckvwTWQssywLBXNVqxgc4WLJjzfUhgef+WE7 5M3+WFYlOVQLGZY/zEVgma1raYnOHNAOzeHLDmEXjbZP6vGAeDyBbGfQPpE7qGYZ7ubeT3Xw QO+PklcCrvOPj2ZPcAxGNS2xVU/LzONqCrJqLMJSIcCdsbiSP4G5PnDFHtMokaTY6OEr8OEQ fOAerhcHUa/z7Uu8YtmaqKH+QGkE/WEgaRqSiTnv0JOTD+DxehaqvoKPPZ++2NpCZMHB2i6A /xifmQwEiIjEXtcueBRzkNUQkxhqZyS13SrhocL9ydtaVPBzZatAEjUDDEJmAMLVFs45qfyh MiNapHJo2n3MW/E5omqCvEkDdWX/en3P7CK2TemeaDghMsgkNKax/z0wNo5UZCkOPOz0xpNi UilOVbkuezZZNg65741qee2lfXhQIaZ66yT7hphc/N/z3PIAtLeze4u1VR2EXAuZ2sWAdlKC NTlJMsaU/x70BV11Wd/ypnVzM68dfdQIIAj1iMFAD/lXGlEUmKXg5Ov2VQDlTntQoanCYrAg +8CttPzjrydgLZFq3hrtQmfc0se5yv1WHS69+BsUOG09RvvawUDZxUjW19kyeN9THaNRgow3 kSuArUp6zSmJABEBAAHCwF8EGAEIAAkFAlMQe0ECGwwACgkQGuaGDlbL0pMN5wgA4bCkX/qw EVC06ToeR6C2putmSWQMgpDaqrv65Hubo+QGmg2P4ewTYQQ4g6oYWS03qHxqVVWhKz7FjfrV +dH8qbCLfSgIcvdBha7ayGZVrsiuMLKGbw36fcmkZPpSDOfHcP0XH8Z+u9CWj0xUkTxAlZ/7 i6gYSUpG2JWNtdmE/X8VVEyXusCLwy0K0BI60A/4dRTIX3C4QKrJ3ZbUXegz70ynjHf+lQMZ 9IZKASoRMuS5FozPQh6abvmwZEPdf5I9riUElzvHrqJ8Bx0t3Pujdoth+yNHpnBxrtO8LkQd rQ58P0SwcaIX33T2U9pG8bhu5YVR88FQ8OQ0cEsPBpDncg== In-Reply-To: <86ed2zsp6l.fsf@ltc.des.dev> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:159.69.0.0/16, country:DE] X-Rspamd-Queue-Id: 4XxyKp4zJtz4lYF X-Spamd-Bar: ---- List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org On 25/11/24 09:17, Dag-Erling Smørgrav wrote: > Dimitry Andric writes: >> Probably best to create a bugzilla ticket, but as I said before, I >> cannot reproduce this. > > I can. My builder is running 15 and sees segfaults while building > packages for 14 and 15 but not for 13. Interesting. Luckily everything is working for me in 14.1. BTW removing optimizations (CPUTYPE) for only the affected ports made guile2 work again. Did not solve the issue with sassc though. I'm not sure I can provide a strict procedure to reproduce this, except suggesting building in poudriere, with a recent head both in host and in the jails. I'm also using ccache, but that does not look relevant. -- Guido Falsi From nobody Mon Nov 25 21:05:57 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XxytG1l5yz5fC7b for ; Mon, 25 Nov 2024 21:06:14 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-21.consmr.mail.gq1.yahoo.com (sonic317-21.consmr.mail.gq1.yahoo.com [98.137.66.147]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XxytF5FXNz4sL3 for ; Mon, 25 Nov 2024 21:06:13 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732568771; bh=mhGBt4etBL2N+8b0scCo4C6HwsaFNfrgeddifrj0Skc=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=SnF0KfjBvIG0i9/Jk0Gb53nD8DSCxD93zJc9LGfzcVyW3864OIEq/xyaPcs5UyOjs/Yhagjx+I6MTdsB0NqwX2bXLpCagoVj7+/f1HCWd1NUUic82LHwyFn727B5EVYYogRe972qn8c68lpQbMTQJ1AEYDRbXj8DMIPb/vkcFXhcEVf3KgJUI5pwF8e7iTY9IMG9ktgcTNnWuyWB6QSkMiUiZJw9+wZI6xRzpPklIJSxHtGqJ7xocAi45KaASPHAs6eT7QoBtdt8bfuz8DN8Cd5B86Ox/ZkzFbg7yTfuSLfS2B1of/pLJUOtZvIZpXqDxdT4ZyIyYeolIinWug22ig== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732568771; bh=BO5DYzc1BmbWTwnxcVVGpQF7oodAlqRcgpHDqPFiIch=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=Ol+mpejzhlIaleDRWn8p7WeYnpO3FXlpK5lwQ3Eb11uB4ZlwlcbYnlV8nxRMMDlbLYNBVATc+cV0a24KvIixSnJ5t8v44TZinwyXXHpSW6CCBg82P5+4yH92wHOfqXLGrMW3y6xojMFQg5HXbDRr5z44SZfzX/epvrdrx7ofx023COs1fETUBfILnhqLE3jYhaGnjekqmbB7JClFJb6Jk0t3ZhUs6oxRVnFUBgSe3mfut1kr7UNsvPvxaPPJZCrLgOBSIwg+uwIRx7sbBZXfwZ3JIy+/g5ANqgokfE/t90751i2WzHXt2MKlMRfhSDuLU5Zq4PZqtHbO4E9Hdx4aJg== X-YMail-OSG: uE0ETQ8VM1lWd5P_3KnRneukMuui_CZq_tmkl.OVnMxVBMYhHxJEZu.WZUvz4jk ynbRpLrfZrJg1bZhQpU.P9GzLYfgPoN7f1SqvfUa7YVerM_W1CTDzbB2678XWH9eIFZ5yu.pqYP6 e1Iocng.CcDvPsADhdmb2x1tfG9hl0ElDVP.reLrWN125LAL9oAa5CGLXblAjmOr_.WwRTa.W7B8 fjjot90S1kbhKjRQWQwIOWcRI9pYBJE6wUYfFS4snIYdb_4UNMvLppxemQryfLKpcZ5MqelbbEjI AkkQlmg_Vm4XcC.TZ4zyM_tAed4wrUpIVn_RR0ArMR0xxJOFXZbsgjkKmTbS6EYLaO5LMvPC32H. VHI.sUdRkZb6Zu3CFwyH_b6IoOEDxBdsg_Xt900.W_OmgVbowOiHCy1vk6kyVYbfZTIop.hAdaGJ RctS4YZ7R6rWOKdV2jPpsWDLW9AonxdVeqP4jo2qE4QdfdaCfAuF4CuD6WE66tWx9lGYvBcQWCpU TlOl30sjuX0X5eGq4qALyDniGzu_UAY6O9Ez243O93uy3IYklmF.H_zXO9V9.HC3cIBB6YWOOUUs xnD869DWUV_zUNzTNVfQ0Ma9xn0zGklrIGJTxoLkZghWxbIGplfomwHjhVFEGmycYHOmrrDP89qW GLwBDuw7a5MZXzGMrSUpoObovIlE50cbsmd7un1x3oKOdOI3KPeD1hOG7tgRSQMLTYzczmBp..GE ZoplvVSdqOJlw6yxgbxMQ6ovVE_cuX3wNzQRgBxQaC.AKllRNp_I2wzbxbGssJxIToE0XM4Ky7z0 LLWAzUJ1FFv1_clG0OUcHVTG6TLrU5cNEU1CN3dx0PU5fdzaJR71DrE_4eEnH4u._8jW7FTt4Gg8 7_IP7sAzY7ryZd3c7ZUUuR0b25aBtioiljGRM0deS2x2q5tuR64J2qPtuJQhDU9Mhqmbb.Zlt_j3 z9h186iIXsuUuTFqy4dSRBjdH75IoMQZaJqd1eKUuFhNVvw_GlTkdNsR3k5X7rKgOHRFBc2M23Kp 0nTYrTAdTXVEj44N7QVgO6ynIYum36fode.wOAXb9u1CKL5f85W34Pb7i4GcJ5ZOrbX0Z_YhT2Os ZL6wDdDmD9b5.c8_37CnJshHnSblrhuB7abQfepiOdRj3uMQnJxYakFMDL74rQyRiU8ZrgaFwwpj kAQ39TzDAy8c_GdnbNGHbvyfTWyf39c1mdIyey2W7oKMMLXNOQvMQZrf2lrL7S7zUukgr74.lp66 _iIqCuBJg6qTV73IRcmaHAH6VKuE0Zrdk38W4KkIWqWm6B09zzsiNcO13E7aMj.iD.B4z4fRqMTR CB_Z_ewFLAspGZpTEIwCNi62urIZly25KMEMtoKChK9buKP6tMxDyceu9t1vbBM608BhdnDWQYFY ZcT26XiNLUnZFg_aoPhFvKHyvJ0XLOzxBDJPrdjuqKuyuEEZ8Kxmcj1_Ht8Zv2DRl1QORItgwfCt uExGyGsXhgoHYAWJcjesEYRtikhrIQBJoEZI8gSmnRbV7mia8ZekDrY7AS2SzaIBArUeh3en.jbM CCejuLa4NDYO2Nuhg_KQN423oyZrLja5FboNY6DFU_Yr5vM_MfgQBj7zSeRcTpXO2v4NGnygJ6cy xxFArz_CIlaCtbmjNQSv7AbalItAtlE9KCM8B1UsaxwU8PLXMamILlKRnjONyhM4z05fOtEbUE.4 7r5D.5tDtsS5gppoyM.m25HM4CQgXQ_hDLWcldcH62_6TQbRW8XWAeYBXESlb3NpNGPjcVpH40ZH qi3jFZbKsWjw5.qmYZ3ZNHOk.pmnOage37SBAVolAbSxShPJkZuPkuXYSsKaHZ3RRAEbPI6ubYvW GKa2vJL5VueZo2p3jnbfaUlnNgezpDC28CIfjxz1_O5rokPTKQCqHsMEtL35NdaeJBa7pQ5ykpzn nVTNzIhj68HpXLnPaDEhVUVNQSTyYWuk3BrlRZIh17Y5vesOF2dYqmrEbkJFnX9evDBync23FGad Q3oHJbtJjcfegt8ao49so2Dy2Y7scSolXJXEaZWsN8RzIkeSmAVCgh4h8nMw9CZfCnaiCrnhTiVp r.bFYKtYhJf1VVmpXicmlfn16d2i3qTg6eeW9pkKJO8uX0ipTy6GsStXGRHulncmnV1AIGv0vMFJ .oKM1o8zjLyHA..6OCLfBkqw9ZapTnEaRWYk21f2Ce22kRHKHGK6aG5UfpTaE12QxezGCkrDeDcX 5WsEkOZ25gQEeJnCaBX6Wt6oS.Yh_WNmiMXc- X-Sonic-MF: X-Sonic-ID: bd420104-5386-494d-9de2-bc01d8893863 Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.gq1.yahoo.com with HTTP; Mon, 25 Nov 2024 21:06:11 +0000 Received: by hermes--production-gq1-5dd4b47f46-whghm (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 05646dc613fef0e0d34da7d700f4c084; Mon, 25 Nov 2024 21:06:08 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51.11.1\)) Subject: Re: port binary dumping core on recent head in poudriere From: Mark Millard In-Reply-To: <5f24a570-26e0-4c0a-817f-591a234fd07b@madpilot.net> Date: Mon, 25 Nov 2024 13:05:57 -0800 Cc: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , Dimitry Andric , ports@freebsd.org, FreeBSD Current Content-Transfer-Encoding: quoted-printable Message-Id: <5918C6A1-8FDB-40CA-8C86-EB7B7BE75A2E@yahoo.com> References: <46E3A370-A3E0-4BAF-B707-87F94F98E248@FreeBSD.org> <5ee47c3d-f80e-4d50-9b6a-acb3c98e80e0@madpilot.net> <7c9c3cf5-bbd1-4642-8d04-33aa07a4db02@madpilot.net> <9df256a8-c6ed-46d9-b955-fc2657c12d36@madpilot.net> <5c502054-7353-4a1e-8350-c403482e9c0d@madpilot.net> <3127C3BA-FC93-4636-ADDB-89518DE9C60D@FreeBSD.org> <86ed2zsp6l.fsf@ltc.des.dev> <5f24a570-26e0-4c0a-817f-591a234fd07b@madpilot.net> To: Guido Falsi X-Mailer: Apple Mail (2.3776.700.51.11.1) X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4XxytF5FXNz4sL3 X-Spamd-Bar: ---- On Nov 25, 2024, at 12:41, Guido Falsi wrote: > On 25/11/24 09:17, Dag-Erling Sm=C3=B8rgrav wrote: >> Dimitry Andric writes: >>> Probably best to create a bugzilla ticket, but as I said before, I >>> cannot reproduce this. >> I can. My builder is running 15 and sees segfaults while building >> packages for 14 and 15 but not for 13. >=20 > Interesting. Luckily everything is working for me in 14.1. >=20 >=20 > BTW removing optimizations (CPUTYPE) for only the affected ports made = guile2 work again. Did not solve the issue with sassc though. >=20 > I'm not sure I can provide a strict procedure to reproduce this, = except suggesting building in poudriere, with a recent head both in host = and in the jails. >=20 > I'm also using ccache, but that does not look relevant. FYI: I've never used ccache or analogous and get the libsass.so = .1.0.0 .got.plt corruption that I've reported on the lists anyway. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Mon Nov 25 21:18:38 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xxz8c5K6nz5fDBZ; Mon, 25 Nov 2024 21:18:40 +0000 (UTC) (envelope-from des@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xxz8c4T1Qz41N8; Mon, 25 Nov 2024 21:18:40 +0000 (UTC) (envelope-from des@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732569520; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=H88ixeF2j5CH5e6mXgExutKNW9f8KNpuv8mp0e5uilg=; b=Y5PdQHDyd1XdfynG5AedK5/oS/dH5Ls8g8kYOT+/3UYTjAlohvbcksdbSeRf1xQBOg1/V1 VmGtRnBY2WZwHQTPnQDmmv1OCSyFnsSCAZQInsBvAwOiL9XrttM/KjmoGFSiFtp3brITNm Ol8QAWgdqvMcnnfZZM3tPSQGrD+AMINeI9a/YHYShmqBFzdLPcnGOXtecdAvEnLwNqGwhy IJqap0iD0UHq4X2VGFl7t9A7/U9u5Uyl//zx8gNwpmZWyAhLz23/7ZAUcw+bA2NycIOa2/ QGaqjfqNfsxHrs5AZETNokslxvxqMhN4pdTCswUi0Baw7odcNwK9P/Z9K7eQ/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732569520; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=H88ixeF2j5CH5e6mXgExutKNW9f8KNpuv8mp0e5uilg=; b=LkT1nGyv+o7lmmUat5ifaXKfpxQbg0+brI2IBtMMnG/wkH8UNKJVLMW9LONoVddXQJj8O1 NpuiW6/8XsWDKHpP4pK8VXgJ/JwFRfM9PSZtVdwC+HC1AsRmqU4kxkw8r+ria/HUtZZtqt Pg8qaCrxsMVmqMBzFmM5FXR0gHZov/FaEb9SODpKgM9jpRpbd9f3TwRKOdh4SQAbS0lgzx Qxt1HPw4b8z3XkNp7FArtjBFxfZG8JoYo9h1lmikBL8z2Bai2hY2Yrg8hk+avEX26ll8qd VGh9frPr5FoN9o2nsl+57beNckUmk7c+o3n0ii7WGtUJWoHuQmiwE7akTfV06w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1732569520; a=rsa-sha256; cv=none; b=fdWBLxNXHOXv1Vx+Xefk0WOaJ1K5ZXgJgMmgUIsXEYOYJ2VXtc4RJ/yU1vTxxSipJL5qTy YI9PlDFC6cEL6twskcvcfhGEJmL4VO0g3IRzZy5H4qmpDU3ksK0+NkRqrDzYhegoX4j28o JT4f1LCjCVGckvUxwGgfurT/cV4O+xnPt9f8aq0/aCiXg0SaUcmcJkTdh63//JAm9U5Hai hQnBtxlaNMcgJdArMn+EhCryvJ027Z22kfawYxfT2w85jIw6UUSQprIfw6XJNdIfcz4Ijs v0A8OT/dzgwUCw2QFUI6kvn8U1SGHSK4HoQoMB3xWG2Sr5e8nbHDrTKsoD8pGw== Received: from ltc.des.dev (unknown [IPv6:2a01:e0a:c54:bed0:922e:16ff:fef1:acef]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: des) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Xxz8c3NRWzMSy; Mon, 25 Nov 2024 21:18:40 +0000 (UTC) (envelope-from des@freebsd.org) Received: by ltc.des.dev (Postfix, from userid 1001) id 9451D60787; Mon, 25 Nov 2024 22:18:38 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Mark Millard Cc: Guido Falsi , Dimitry Andric , ports@freebsd.org, FreeBSD Current Subject: Re: port binary dumping core on recent head in poudriere In-Reply-To: <5918C6A1-8FDB-40CA-8C86-EB7B7BE75A2E@yahoo.com> (Mark Millard's message of "Mon, 25 Nov 2024 13:05:57 -0800") References: <46E3A370-A3E0-4BAF-B707-87F94F98E248@FreeBSD.org> <5ee47c3d-f80e-4d50-9b6a-acb3c98e80e0@madpilot.net> <7c9c3cf5-bbd1-4642-8d04-33aa07a4db02@madpilot.net> <9df256a8-c6ed-46d9-b955-fc2657c12d36@madpilot.net> <5c502054-7353-4a1e-8350-c403482e9c0d@madpilot.net> <3127C3BA-FC93-4636-ADDB-89518DE9C60D@FreeBSD.org> <86ed2zsp6l.fsf@ltc.des.dev> <5f24a570-26e0-4c0a-817f-591a234fd07b@madpilot.net> <5918C6A1-8FDB-40CA-8C86-EB7B7BE75A2E@yahoo.com> User-Agent: Gnus/5.13 (Gnus v5.13) Date: Mon, 25 Nov 2024 22:18:38 +0100 Message-ID: <86ed2zc8r5.fsf@ltc.des.dev> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mark Millard writes: > Guido Falsi writes: >> On 25/11/24 09:17, Dag-Erling Sm=C3=B8rgrav wrote: >>> Dimitry Andric writes: >>>> Probably best to create a bugzilla ticket, but as I said before, I >>>> cannot reproduce this. >>> I can. My builder is running 15 and sees segfaults while building >>> packages for 14 and 15 but not for 13. >> BTW removing optimizations (CPUTYPE) for only the affected ports made >> guile2 work again. Did not solve the issue with sassc though. [...] >> I'm also using ccache, but that does not look relevant. > I've never used ccache or analogous and get the libsass.so.1.0.0 > .got.plt corruption that I've reported on the lists anyway. I don't use ccache or optimizations. Here's an example of sassc segfaulting in a 14.1-RELEASE-p6 jail: https://pkg.des.dev/logs/data/14amd64-default/2024-11-24_19h29m04s/logs/e= rrors/plasma5-breeze-gtk-5.27.11.log which matches the following entry from `/var/log/messages`: Nov 24 21:23:06 pkg kernel: pid 71277 (sassc), jid 253, uid 65534: exited= on signal 11 (core dumped) The poudriere host is a bhyve VM with 48 cores and 192 GB RAM on a 32c/64t AMD EPYC 7502P with 256 GB RAM. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@FreeBSD.org From nobody Mon Nov 25 21:27:14 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XxzLk0RqFz5fDVk; Mon, 25 Nov 2024 21:27:26 +0000 (UTC) (envelope-from mad@madpilot.net) Received: from vogon.madpilot.net (vogon.madpilot.net [IPv6:2a01:4f8:1c1c:11e5::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XxzLj4ttPz44HW; Mon, 25 Nov 2024 21:27:25 +0000 (UTC) (envelope-from mad@madpilot.net) Authentication-Results: mx1.freebsd.org; none Received: from mail (mail [IPv6:fd5c:5351:d272::3]) by vogon.madpilot.net (Postfix) with ESMTP id 4XxzLY56hfzL7vr; Mon, 25 Nov 2024 22:27:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=madpilot.net; h= content-transfer-encoding:content-type:content-type:in-reply-to :from:from:content-language:references:subject:subject:date:date :message-id:received; s=cyq4qetkgngm; t=1732570035; x= 1734384436; bh=X6gvzpjG0XCcVpnKw3zmCPo6JFUcqj//W2NW6jCZapw=; b=b pxV4rm8ZZhxT5zCm2PJhE0pz57WWinqz5z+ECaDmzwIK/BI15dyaQePssnNedVS9 UDxSR+KXqfzmzrMpbDuAJtOda/PxsGerJp4+BLh2YOPG5akm8zRMqCfMYDDltUI2 do5zHFuXj4ykdImHeOd2DG3lINDUO8MbdYzqNbmCoKnP0+yBbp9aC0uC3LaDwPuz qOSIfldX0mEWr6Vfqdv6z4aE6wsxL1PynlQ3xcEBGVQLF/L2TolJin47H/uFqI6Y Rq6gLto5fdPYns1gHRR4Ozz2T2q1MaLm/b/oNXi6gwpWYY2OgSlVtJK2NlSFPQIG zPL8tmFZd8lJJjVr4Z9mw== Received: from vogon.madpilot.net ([IPv6:fd5c:5351:d272::3]) by mail (vogon.madpilot.net [IPv6:fd5c:5351:d272::3]) (amavisd-new, port 10026) with ESMTP id TIXq0WQwc0Iu; Mon, 25 Nov 2024 22:27:15 +0100 (CET) Message-ID: <45098ccf-4dc6-426c-849a-c923805d6723@madpilot.net> Date: Mon, 25 Nov 2024 22:27:14 +0100 Subject: Re: port binary dumping core on recent head in poudriere To: =?UTF-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , Mark Millard Cc: Dimitry Andric , ports@freebsd.org, FreeBSD Current References: <46E3A370-A3E0-4BAF-B707-87F94F98E248@FreeBSD.org> <5ee47c3d-f80e-4d50-9b6a-acb3c98e80e0@madpilot.net> <7c9c3cf5-bbd1-4642-8d04-33aa07a4db02@madpilot.net> <9df256a8-c6ed-46d9-b955-fc2657c12d36@madpilot.net> <5c502054-7353-4a1e-8350-c403482e9c0d@madpilot.net> <3127C3BA-FC93-4636-ADDB-89518DE9C60D@FreeBSD.org> <86ed2zsp6l.fsf@ltc.des.dev> <5f24a570-26e0-4c0a-817f-591a234fd07b@madpilot.net> <5918C6A1-8FDB-40CA-8C86-EB7B7BE75A2E@yahoo.com> <86ed2zc8r5.fsf@ltc.des.dev> Content-Language: en-US From: Guido Falsi Autocrypt: addr=mad@madpilot.net; keydata= xsBNBE+G+l0BCADi/WBQ0aRJfnE7LBPsM0G3m/m3Yx7OPu4iYFvS84xawmRHtCNjWIntsxuX fptkmEo3Rsw816WUrek8dxoUAYdHd+EcpBcnnDzfDH5LW/TZ4gbrFezrHPdRp7wdxi23GN80 qPwHEwXuF0X4Wy5V0OO8B6VT/nA0ADYnBDhXS52HGIJ/GCUjgqJn+phDTdCFLvrSFdmgx4Wl c0W5Z1p5cmDF9l8L/hc959AeyNf7I9dXnjekGM9gVv7UDUYzCifR3U8T0fnfdMmS8NeI9NC+ wuREpRO4lKOkTnj9TtQJRiptlhcHQiAlG1cFqs7EQo57Tqq6cxD1FycZJLuC32bGbgalABEB AAHNHkd1aWRvIEZhbHNpIDxtYWRAbWFkcGlsb3QubmV0PsLAeQQTAQgAIwIbAwIeAQIXgAUL CQgHAwUVCgkICwQWAgMBBQJS79AgAhkBAAoJEBrmhg5Wy9KTc0kH/RO64ORBlTbTHaUaOj8F Je5O5NU2Pt9Cyt5ZWBRvxntr1zPTJGKRPS9ihlIfqT4ZvEngQGp57EUyFbCpI0UWasTerImM tt5WACnGmCzUTB39UXx8Oy4b1EgWeTJQ747e/F1mQLXTNa6ijRBE9fYlTb4gAkPN88/wVV9v 3PZozKLTg16ghBzHM/P7Lk8L7clPEZChX1FTa/6eSt3nvzfCuTMZbBPJF/ph+q1KyPqRgVfh tyhu5dvgMoPz/ni41IfeSrkJTD5RXzdyGR9q4Z1NYeBsLkRjC4LxKAP5KqUsvlOUjKvO1byj ApYdMarol+IGkaSk9e3zVYAJkWKjn/ni8XbOwU0EUxB7QQEQAKFhrDceoPdK/IHDSmoj6SQY isvM7VdhcleS7E9DoEAVt7yMbf6HbbMVTTY6ckvwTWQssywLBXNVqxgc4WLJjzfUhgef+WE7 5M3+WFYlOVQLGZY/zEVgma1raYnOHNAOzeHLDmEXjbZP6vGAeDyBbGfQPpE7qGYZ7ubeT3Xw QO+PklcCrvOPj2ZPcAxGNS2xVU/LzONqCrJqLMJSIcCdsbiSP4G5PnDFHtMokaTY6OEr8OEQ fOAerhcHUa/z7Uu8YtmaqKH+QGkE/WEgaRqSiTnv0JOTD+DxehaqvoKPPZ++2NpCZMHB2i6A /xifmQwEiIjEXtcueBRzkNUQkxhqZyS13SrhocL9ydtaVPBzZatAEjUDDEJmAMLVFs45qfyh MiNapHJo2n3MW/E5omqCvEkDdWX/en3P7CK2TemeaDghMsgkNKax/z0wNo5UZCkOPOz0xpNi UilOVbkuezZZNg65741qee2lfXhQIaZ66yT7hphc/N/z3PIAtLeze4u1VR2EXAuZ2sWAdlKC NTlJMsaU/x70BV11Wd/ypnVzM68dfdQIIAj1iMFAD/lXGlEUmKXg5Ov2VQDlTntQoanCYrAg +8CttPzjrydgLZFq3hrtQmfc0se5yv1WHS69+BsUOG09RvvawUDZxUjW19kyeN9THaNRgow3 kSuArUp6zSmJABEBAAHCwF8EGAEIAAkFAlMQe0ECGwwACgkQGuaGDlbL0pMN5wgA4bCkX/qw EVC06ToeR6C2putmSWQMgpDaqrv65Hubo+QGmg2P4ewTYQQ4g6oYWS03qHxqVVWhKz7FjfrV +dH8qbCLfSgIcvdBha7ayGZVrsiuMLKGbw36fcmkZPpSDOfHcP0XH8Z+u9CWj0xUkTxAlZ/7 i6gYSUpG2JWNtdmE/X8VVEyXusCLwy0K0BI60A/4dRTIX3C4QKrJ3ZbUXegz70ynjHf+lQMZ 9IZKASoRMuS5FozPQh6abvmwZEPdf5I9riUElzvHrqJ8Bx0t3Pujdoth+yNHpnBxrtO8LkQd rQ58P0SwcaIX33T2U9pG8bhu5YVR88FQ8OQ0cEsPBpDncg== In-Reply-To: <86ed2zc8r5.fsf@ltc.des.dev> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE] X-Rspamd-Queue-Id: 4XxzLj4ttPz44HW X-Spamd-Bar: ---- List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org On 25/11/24 22:18, Dag-Erling Smørgrav wrote: > Mark Millard writes: >> Guido Falsi writes: >>> On 25/11/24 09:17, Dag-Erling Smørgrav wrote: >>>> Dimitry Andric writes: >>>>> Probably best to create a bugzilla ticket, but as I said before, I >>>>> cannot reproduce this. >>>> I can. My builder is running 15 and sees segfaults while building >>>> packages for 14 and 15 but not for 13. >>> BTW removing optimizations (CPUTYPE) for only the affected ports made >>> guile2 work again. Did not solve the issue with sassc though. [...] >>> I'm also using ccache, but that does not look relevant. >> I've never used ccache or analogous and get the libsass.so.1.0.0 >> .got.plt corruption that I've reported on the lists anyway. > > I don't use ccache or optimizations. Here's an example of sassc > segfaulting in a 14.1-RELEASE-p6 jail: > > https://pkg.des.dev/logs/data/14amd64-default/2024-11-24_19h29m04s/logs/errors/plasma5-breeze-gtk-5.27.11.log > > which matches the following entry from `/var/log/messages`: > > Nov 24 21:23:06 pkg kernel: pid 71277 (sassc), jid 253, uid 65534: exited on signal 11 (core dumped) > > The poudriere host is a bhyve VM with 48 cores and 192 GB RAM on a > 32c/64t AMD EPYC 7502P with 256 GB RAM. I sincerely hope this is not relevant but my CPU is also AMD: AMD Ryzen 5 5600G -- Guido Falsi From nobody Mon Nov 25 22:12:53 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xy0MW75Xpz5fHQj for ; Mon, 25 Nov 2024 22:13:11 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic305-21.consmr.mail.gq1.yahoo.com (sonic305-21.consmr.mail.gq1.yahoo.com [98.137.64.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xy0MW52Nfz4Fy0 for ; Mon, 25 Nov 2024 22:13:11 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732572789; bh=DBxtrFZlAu73tOmHP37uYaGAupqyhTjZk2vHSltl8Y4=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=St0mClmQmygYRL0E86o08iTXYBB3kFwg1U4bYq1jlxKjICgr0+QKF7Xfgf0wzEOgffe6PwSwh+O3NEvYGbIEWPUirlXYUWmllBfYM1gqpsMB6pctM5wwVXtmoLW24B2E/DZvGzR4IhGssL712c2eXF/Vw1aTn69eLnQppiiDh8R3BJX2Jl6pgoH8OWMFNiPm/6x60+ocka0yst7cxGJXiTabDILeeJNL/Yt/ilp4+b/85+2aOlUg67NHSKNCcRmu3ZCWF0ZI0VqoytwbzZM267bY2VqoF4xaZgDxkL/kTYh8fo+6WovUjw6sfnOyzfYg0Oo5UNgUMIjSoCrso0QYjA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732572789; bh=uA9W3T0Wgy6USBB67uyxYBTfb5rfQzJC9Zq4T2EJl79=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=jOQT5GB0CVtoduUpanRTz1G9N2A2Youc2K5f5i0Kw01QMPtjLrD1PvPhz4vzl21buLi8WvXFoZdQFPUEtVeIRa0CPeId9lh8M39WWPxQjRfrcko56qf7dQnSUEoRduhZzZZFvWucf0Pxc92LwigaftFEZeDeN5X0qSfF2hC6eZUOveubsuwy40/Tyty3Kjbg8kyi5wfKVQTR0E9XPXSJ0DKof4GRGi06iX36Hj1LjEVdmjY/TGHsmmZgqwtxgmVKm7HNgb8Xks3nCaZzZJ+D/IhWu83Mr2WSTU9MMt+oDObwnaYy+sLemz4IkfK2YBcHVDCCK5mI77J3tD/g3pzDKw== X-YMail-OSG: d9a1ZcgVM1nZG9HHqHgvqtjFt4AXi5hxw7v0TlJ_Gw884K.2gtLvSckKC7ma44q V9cWfA.dzAxDsDJOh3f_YkoDjSFPcSLPTGBNcctjMvaLHkrQpd7ffzntnFkzItZojB7etZc3eoC4 1OkqexT0hM1MIQP0wXfPULcoBKkm6v8TIZHpoc2xec7KlLjxqNGZQXf1LhddQF59tdlATwVMN2_u Y.b.x64KQiTAQGWC.jqES1f3XBfxoww9W5R.WDIWFDdoT7DKDyBLgVRzQqEWV3IBo92NwWN_NVfb sN.ejMqM.wfCQ24BjsQWOiFmjFtJJnhzn1ax2JGPH06qDNoFG5656jKZDPlpURLayWSNtIDDags5 mZEgMH97vnPBmn2FuHEC3953T9BYEvM0iXDB96gx0Y.DChvN4BAHTboV_SNorfYn.cxKZX_oiDsw Y0iaTLyn79HvFqgDcpazEXcCFaT1ipLm5Bf3qbJ9SOzc1KyVWsJnKrR_KZR7iUB4xL3l7f3hFM4Q 9hASY.gei8S3vDoCK5BdlG9TI.zkAjvNtZSNAF2GJaF6D7AutRFVpYf2DEOr_JuvkcTKUmUg0mts .SV0ikgtpxs3Ls6ioGpkZLsOcQaDZyJxicp..HxKSbzJIY87FxhNqcCfEPRCz1EBCxcaw0gTFOYe i0ghijDnmCFmZ864e_QByonLRQuIHW1EHbccQeYIYpHWfLyLqFIxQrqDgNrzZ7fWsIQgRqq35410 pDchzAB_h5Xlh.zrqqEEwn1zjQxBIcQfqBClc3LpJPIC03sn8NU.HtW93ZWm2kpsZIjk.LgtZFHG T9pQM.PXd5VDb9yDxcd0yJ9_liFQebfriPBsarQ1tOG1SQD5kduvp7Y2qWkIFlsAJhZXhYLNw3GN 1DVXgzLledYxWTok37_7X5A7bn5Eqo2CqP2Qwln4xDxuBpjS9wluHorPzr41hQxrErbhfceZWyyW CXJJZxdhXP82zJ3iB0E0FI3T7xbfC4KeVqDuWVrxseTRQ_odASkXu7xlBA7nyEZmuVP6RTdHs.CR ogv7bmlXF_ZT1ERgASLmEkltxJw4LPjbK1zu5AXov7b7UOvlfP6GgAsUFJEXf7Yp.Q3bTFldIxxe XTYW6kwasnA155vlZwA46zsxUB_FUjqKzoizfEQxFDmgVRYkxERFC9jvpQ5LJ0u8hlZ79zPwYymB QBG66RFRj_X3Pd3oSB0htEE0o_SrqJ9ix3rp6PdhW1TZf05bOfRPiPYUlDD6uvuShyoLtGY0fXSO Vs8BirrLTNlL6.agK1zfxS8ucvwQ3XEQEnabv9EdcZBgYKeLKKlwaB8yZ4i_jYpoaGLLQlk4Min5 J3tnQXeL.mFHUNAmTWqUJmnuer.hSpan2jt8Jp67XdjwC8K9mfhjCp5nZvlB3ULi6OvRoMkZ_yh0 6bdagBnUkozNiWv4fRZ35EpGE.s0uByP5Cm6xRbS0LYjqnKEJklfwCf0L5483eCukiIdzReIPcJa JXkGByMD81ub1d2LgN0Vu_CBtVchl_xPNDhRAnhuy8v_y3q_Fnd_AHYYi3TEKSajFHlL5C.O6mHF sHps93Rxzh9HxMn6U05m4lyU6weN.N_rl5CWGINF6O0g4xU5mjKoy36qcugbpu1eiHZdSx7t.jWQ mo_roLttSTVszYrxHx0YDphO9j37iR52mu8kHJ_Jr0W5G_ErfVqhVXC5i2I9HC6wzvcQh2R5Gz58 W8JPKyrz3TB43U2.7CgzmlDBayNIMuMUCBr740SrZ7_Ut0TtHwn8SW0tEfWPVwenpbRj_xgIhVpM X6WjDPLXxjVahMdVwhLmX.wMSeasm2oUHYF479ZNF_ERoToHQvuvozcrIdPc2jrmoJn3RC15xxi3 zaEnVW4jZ9fJm_7bBwARxq.Iwc.NovN6FDJUAb0YQqOQLMUklHy6yNA_ruUAhR1B_0_cNkhOpAr5 whdoZedlRM1HE0Azbc1YfyEbiLcRCxkXEd11QCuoAmW1fYfRDDLbp8y7MshQg8CqxWu4lJZdUKBC _N28Hd4rxbeLjrA8PlyS.08f_0RQ2NmSmhMUZm859qos64H4pTYKhHuZ6Bkrw2i1ZsZLgBnyb7Qz YGY0FlokckKxUChWxGE2uZjc5caQmCfE5rQ6JEooxc6jxcb5eq_m1wAtGhHXWcxhrOzgq0QZ9x3h e4e_a3xP5ZWPTZ1R5Jo74wAmNZhb9gIVeXjMHDOrnjXtAkxEOtjJbdJ2LAAlVtRqL.6Ziry7Wflm .A5GpSIADf5kk51zCaNnaSjmh9vBvX5d7AT0Baf9Qm4cky1NWCI1UECm2sq4xleHIbc.1hoAg8Ck se.OkzQ-- X-Sonic-MF: X-Sonic-ID: dee44fbc-2065-4e91-a28a-d48a20dc7be3 Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.gq1.yahoo.com with HTTP; Mon, 25 Nov 2024 22:13:09 +0000 Received: by hermes--production-gq1-5dd4b47f46-bxhh2 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID ea5ea66511aa0b72dcbbb30241f266d0; Mon, 25 Nov 2024 22:13:04 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51.11.1\)) Subject: Re: port binary dumping core on recent head in poudriere From: Mark Millard In-Reply-To: <45098ccf-4dc6-426c-849a-c923805d6723@madpilot.net> Date: Mon, 25 Nov 2024 14:12:53 -0800 Cc: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , Dimitry Andric , ports@freebsd.org, FreeBSD Current Content-Transfer-Encoding: quoted-printable Message-Id: References: <46E3A370-A3E0-4BAF-B707-87F94F98E248@FreeBSD.org> <5ee47c3d-f80e-4d50-9b6a-acb3c98e80e0@madpilot.net> <7c9c3cf5-bbd1-4642-8d04-33aa07a4db02@madpilot.net> <9df256a8-c6ed-46d9-b955-fc2657c12d36@madpilot.net> <5c502054-7353-4a1e-8350-c403482e9c0d@madpilot.net> <3127C3BA-FC93-4636-ADDB-89518DE9C60D@FreeBSD.org> <86ed2zsp6l.fsf@ltc.des.dev> <5f24a570-26e0-4c0a-817f-591a234fd07b@madpilot.net> <5918C6A1-8FDB-40CA-8C86-EB7B7BE75A2E@yahoo.com> <86ed2zc8r5.fsf@ltc.des.dev> <45098ccf-4dc6-426c-849a-c923805d6723@madpilot.net> To: Guido Falsi X-Mailer: Apple Mail (2.3776.700.51.11.1) X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4Xy0MW52Nfz4Fy0 X-Spamd-Bar: ---- On Nov 25, 2024, at 13:27, Guido Falsi wrote: > On 25/11/24 22:18, Dag-Erling Sm=C3=B8rgrav wrote: >> Mark Millard writes: >>> Guido Falsi writes: >>>> On 25/11/24 09:17, Dag-Erling Sm=C3=B8rgrav wrote: >>>>> Dimitry Andric writes: >>>>>> Probably best to create a bugzilla ticket, but as I said before, = I >>>>>> cannot reproduce this. >>>>> I can. My builder is running 15 and sees segfaults while building >>>>> packages for 14 and 15 but not for 13. >>>> BTW removing optimizations (CPUTYPE) for only the affected ports = made >>>> guile2 work again. Did not solve the issue with sassc though. = [...] >>>> I'm also using ccache, but that does not look relevant. >>> I've never used ccache or analogous and get the libsass.so.1.0.0 >>> .got.plt corruption that I've reported on the lists anyway. >> I don't use ccache or optimizations. Here's an example of sassc >> segfaulting in a 14.1-RELEASE-p6 jail: >> = https://pkg.des.dev/logs/data/14amd64-default/2024-11-24_19h29m04s/logs/er= rors/plasma5-breeze-gtk-5.27.11.log >> which matches the following entry from `/var/log/messages`: >> Nov 24 21:23:06 pkg kernel: pid 71277 (sassc), jid 253, uid 65534: = exited on signal 11 (core dumped) >> The poudriere host is a bhyve VM with 48 cores and 192 GB RAM on a >> 32c/64t AMD EPYC 7502P with 256 GB RAM. >=20 > I sincerely hope this is not relevant but my CPU is also AMD: AMD = Ryzen 5 5600G The amd64 system type that I have access to and used for my testing: AMD 7950X3D (16 core, 32 thread, so 32 FreeBSD-cpus) with 192 GiBytes of = RAM =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Mon Nov 25 22:15:20 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xy0Q24gq6z5fHNJ; Mon, 25 Nov 2024 22:15:22 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xy0Q24BLHz4HYr; Mon, 25 Nov 2024 22:15:22 +0000 (UTC) (envelope-from dim@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732572922; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PSDc+kFf5Lt7sPpHTCWhAoYReTDHVvKeuHVQF/5/cbk=; b=QxTLUls006X8r52ePjP335cBemAoOGTL5zoZWlHV7wWoOAVXmZU4sy9GU+QXgP7HvYkG53 qW2H9fzKdr9/5IiozrWa0eQ1XxlR+TxQc3OfEji1/UFDG/EYNJbPuYIPLKL/IH37ox+eTX ngj4vQC01bVyZKtn7rBCMeHYk+J7DfJWgDS19G52MQF50IOq3dP5YeIihpSkZxIBMiwA7S 7Nl7XHvBYSqc04uAiVinccKAS89EZabS6GmLiqw0i/RZpH63yUuv0j5b9aNHWrrM/kSY0v brw3ijZ1Uu7S0GFgfD0fZL/yiIqn9PbM641187B7Tpg3xtWpyNI0DVc7wcIFUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732572922; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PSDc+kFf5Lt7sPpHTCWhAoYReTDHVvKeuHVQF/5/cbk=; b=B8KYhytxuFiwLOgfAf06m2wRJmld8+a1GFI/VMZPFheF45aiojPHN2GrJnw3sRv3VCHP7h FKx8Q0UPgxRgmMmol/ViKP9k+JwDKKLiYuZCCU4rR5VI+2Pygxqky6BeQvX+6M9gOo81ks qdyo76Q0Xe9VkARvsbUOT83F7r3FBCojSNKMcGznbFZT2OryOGUnwdfxjqdUkwBbfwyHHw kB4R3ZAnAHrr9bcVWzwNcQiMdfa3gIqvIg8g/2OR+wsDFfWvyIg6njFACJhE08Dng7/26p EvMkyNdYJZi42t0/f3KeVxhCphNxSKak5MMKtQhf4eoOl4waR+HNuO0vy+r94A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1732572922; a=rsa-sha256; cv=none; b=YXAzx8h8lOtwNbxVRYACxD/6K2A91kULlLX2YzlpqUCa1Ms2CV2BAjKXRFz7Ze73kyLsx/ a2E6pCD77zIjP5RqgplRZIRHNZQ48ku/yNTM0J51heMCKfxMH5vTra5MW7JAtHBbcrkUSU AvFsk97oofr4fbZtDjU4Cu/nYFr0MVi23ouYU/RIMyo8ht1nfIAI9JOU6c+5zi0imfMBVz oetDr8Wo6rwlsBsZK2KCuGUgaNs4WTvOjDzY2t6H5KXruQP2z1/GiyHBXQLZqb1WCPAOJN CZ03rlE7uhQQMuDNNOWrRh+GXJ74Jxr7iYOLgLS21R5m5ljowBioL2aA6XOzaw== Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "tensor.andric.com", Issuer "R10" (verified OK)) (Authenticated sender: dim) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Xy0Q235h6zNSS; Mon, 25 Nov 2024 22:15:22 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtpclient.apple (longrow.home.andric.com [192.168.0.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id D0D573C36F; Mon, 25 Nov 2024 23:15:20 +0100 (CET) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6.1.9\)) Subject: Re: port binary dumping core on recent head in poudriere From: Dimitry Andric In-Reply-To: Date: Mon, 25 Nov 2024 23:15:20 +0100 Cc: Guido Falsi , =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , ports@freebsd.org, FreeBSD Current Content-Transfer-Encoding: quoted-printable Message-Id: <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> References: <46E3A370-A3E0-4BAF-B707-87F94F98E248@FreeBSD.org> <5ee47c3d-f80e-4d50-9b6a-acb3c98e80e0@madpilot.net> <7c9c3cf5-bbd1-4642-8d04-33aa07a4db02@madpilot.net> <9df256a8-c6ed-46d9-b955-fc2657c12d36@madpilot.net> <5c502054-7353-4a1e-8350-c403482e9c0d@madpilot.net> <3127C3BA-FC93-4636-ADDB-89518DE9C60D@FreeBSD.org> <86ed2zsp6l.fsf@ltc.des.dev> <5f24a570-26e0-4c0a-817f-591a234fd07b@madpilot.net> <5918C6A1-8FDB-40CA-8C86-EB7B7BE75A2E@yahoo.com> <86ed2zc8r5.fsf@ltc.des.dev> <45098ccf-4dc6-426c-849a-c923805d6723@madpilot.net> To: Mark Millard X-Mailer: Apple Mail (2.3731.700.6.1.9) On 25 Nov 2024, at 23:12, Mark Millard wrote: >=20 > On Nov 25, 2024, at 13:27, Guido Falsi wrote: >=20 >> On 25/11/24 22:18, Dag-Erling Sm=C3=B8rgrav wrote: >>> Mark Millard writes: >>>> Guido Falsi writes: >>>>> On 25/11/24 09:17, Dag-Erling Sm=C3=B8rgrav wrote: >>>>>> Dimitry Andric writes: >>>>>>> Probably best to create a bugzilla ticket, but as I said before, = I >>>>>>> cannot reproduce this. >>>>>> I can. My builder is running 15 and sees segfaults while = building >>>>>> packages for 14 and 15 but not for 13. >>>>> BTW removing optimizations (CPUTYPE) for only the affected ports = made >>>>> guile2 work again. Did not solve the issue with sassc though. = [...] >>>>> I'm also using ccache, but that does not look relevant. >>>> I've never used ccache or analogous and get the libsass.so.1.0.0 >>>> .got.plt corruption that I've reported on the lists anyway. >>> I don't use ccache or optimizations. Here's an example of sassc >>> segfaulting in a 14.1-RELEASE-p6 jail: >>> = https://pkg.des.dev/logs/data/14amd64-default/2024-11-24_19h29m04s/logs/er= rors/plasma5-breeze-gtk-5.27.11.log >>> which matches the following entry from `/var/log/messages`: >>> Nov 24 21:23:06 pkg kernel: pid 71277 (sassc), jid 253, uid 65534: = exited on signal 11 (core dumped) >>> The poudriere host is a bhyve VM with 48 cores and 192 GB RAM on a >>> 32c/64t AMD EPYC 7502P with 256 GB RAM. >>=20 >> I sincerely hope this is not relevant but my CPU is also AMD: AMD = Ryzen 5 5600G >=20 > The amd64 system type that I have access to and used > for my testing: >=20 > AMD 7950X3D (16 core, 32 thread, so 32 FreeBSD-cpus) with 192 GiBytes = of RAM I'm on Intel, and I don't see any crashes at all. So, are we looking at = some CPU specific issue here? -Dimitry From nobody Mon Nov 25 22:23:26 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xy0bQ6rY3z5fJTY; Mon, 25 Nov 2024 22:23:30 +0000 (UTC) (envelope-from mad@madpilot.net) Received: from vogon.madpilot.net (vogon.madpilot.net [IPv6:2a01:4f8:1c1c:11e5::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xy0bQ451vz4KYb; Mon, 25 Nov 2024 22:23:30 +0000 (UTC) (envelope-from mad@madpilot.net) Authentication-Results: mx1.freebsd.org; none Received: from mail (mail [IPv6:fd5c:5351:d272::3]) by vogon.madpilot.net (Postfix) with ESMTP id 4Xy0bP1vDYzL7yM; Mon, 25 Nov 2024 23:23:29 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=madpilot.net; h= content-transfer-encoding:content-type:content-type:in-reply-to :from:from:content-language:references:subject:subject:date:date :message-id:received; s=cyq4qetkgngm; t=1732573407; x= 1734387808; bh=cRthysDvpuVnt9WVGrHAc9agq3X35GCvS0L3AMxXcU8=; b=X oTaKpgX8EopJ68E3D4YyCHfged7iwdhFqdAq9mS4zMEvYgR5NP206YpSrW6y1dQB vS1IuCFSl/grllXyDz4hpdaPl8tHFfSZDKJjEmE+UElLzf9/zmqtISf1cFkDpMBE sdtmRhczkMJmOLzqA8lAg3ET4Y624z5CFX2xnpHRB5E3R/+32tLJqwHtLByYchOW zO8vcFvObmt5C2t/5VWYk3mLhO0bKVy7w8Kz1/SDd7+Gx52+q7j6n5zu3hbK8TH9 zJz3daS8i7SgeOzX5xuwFn1tckDp9xy/S7lAUnCmQ9cnenrB/ibda6R4UYnoZzQ+ yS604OKF3qXR+Kv8Yn+NQ== Received: from vogon.madpilot.net ([IPv6:fd5c:5351:d272::3]) by mail (vogon.madpilot.net [IPv6:fd5c:5351:d272::3]) (amavisd-new, port 10026) with ESMTP id u7yyCJUTRSVJ; Mon, 25 Nov 2024 23:23:27 +0100 (CET) Message-ID: <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> Date: Mon, 25 Nov 2024 23:23:26 +0100 Subject: Re: port binary dumping core on recent head in poudriere To: Dimitry Andric , Mark Millard Cc: =?UTF-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , ports@freebsd.org, FreeBSD Current References: <46E3A370-A3E0-4BAF-B707-87F94F98E248@FreeBSD.org> <5ee47c3d-f80e-4d50-9b6a-acb3c98e80e0@madpilot.net> <7c9c3cf5-bbd1-4642-8d04-33aa07a4db02@madpilot.net> <9df256a8-c6ed-46d9-b955-fc2657c12d36@madpilot.net> <5c502054-7353-4a1e-8350-c403482e9c0d@madpilot.net> <3127C3BA-FC93-4636-ADDB-89518DE9C60D@FreeBSD.org> <86ed2zsp6l.fsf@ltc.des.dev> <5f24a570-26e0-4c0a-817f-591a234fd07b@madpilot.net> <5918C6A1-8FDB-40CA-8C86-EB7B7BE75A2E@yahoo.com> <86ed2zc8r5.fsf@ltc.des.dev> <45098ccf-4dc6-426c-849a-c923805d6723@madpilot.net> <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> Content-Language: en-US From: Guido Falsi Autocrypt: addr=mad@madpilot.net; keydata= xsBNBE+G+l0BCADi/WBQ0aRJfnE7LBPsM0G3m/m3Yx7OPu4iYFvS84xawmRHtCNjWIntsxuX fptkmEo3Rsw816WUrek8dxoUAYdHd+EcpBcnnDzfDH5LW/TZ4gbrFezrHPdRp7wdxi23GN80 qPwHEwXuF0X4Wy5V0OO8B6VT/nA0ADYnBDhXS52HGIJ/GCUjgqJn+phDTdCFLvrSFdmgx4Wl c0W5Z1p5cmDF9l8L/hc959AeyNf7I9dXnjekGM9gVv7UDUYzCifR3U8T0fnfdMmS8NeI9NC+ wuREpRO4lKOkTnj9TtQJRiptlhcHQiAlG1cFqs7EQo57Tqq6cxD1FycZJLuC32bGbgalABEB AAHNHkd1aWRvIEZhbHNpIDxtYWRAbWFkcGlsb3QubmV0PsLAeQQTAQgAIwIbAwIeAQIXgAUL CQgHAwUVCgkICwQWAgMBBQJS79AgAhkBAAoJEBrmhg5Wy9KTc0kH/RO64ORBlTbTHaUaOj8F Je5O5NU2Pt9Cyt5ZWBRvxntr1zPTJGKRPS9ihlIfqT4ZvEngQGp57EUyFbCpI0UWasTerImM tt5WACnGmCzUTB39UXx8Oy4b1EgWeTJQ747e/F1mQLXTNa6ijRBE9fYlTb4gAkPN88/wVV9v 3PZozKLTg16ghBzHM/P7Lk8L7clPEZChX1FTa/6eSt3nvzfCuTMZbBPJF/ph+q1KyPqRgVfh tyhu5dvgMoPz/ni41IfeSrkJTD5RXzdyGR9q4Z1NYeBsLkRjC4LxKAP5KqUsvlOUjKvO1byj ApYdMarol+IGkaSk9e3zVYAJkWKjn/ni8XbOwU0EUxB7QQEQAKFhrDceoPdK/IHDSmoj6SQY isvM7VdhcleS7E9DoEAVt7yMbf6HbbMVTTY6ckvwTWQssywLBXNVqxgc4WLJjzfUhgef+WE7 5M3+WFYlOVQLGZY/zEVgma1raYnOHNAOzeHLDmEXjbZP6vGAeDyBbGfQPpE7qGYZ7ubeT3Xw QO+PklcCrvOPj2ZPcAxGNS2xVU/LzONqCrJqLMJSIcCdsbiSP4G5PnDFHtMokaTY6OEr8OEQ fOAerhcHUa/z7Uu8YtmaqKH+QGkE/WEgaRqSiTnv0JOTD+DxehaqvoKPPZ++2NpCZMHB2i6A /xifmQwEiIjEXtcueBRzkNUQkxhqZyS13SrhocL9ydtaVPBzZatAEjUDDEJmAMLVFs45qfyh MiNapHJo2n3MW/E5omqCvEkDdWX/en3P7CK2TemeaDghMsgkNKax/z0wNo5UZCkOPOz0xpNi UilOVbkuezZZNg65741qee2lfXhQIaZ66yT7hphc/N/z3PIAtLeze4u1VR2EXAuZ2sWAdlKC NTlJMsaU/x70BV11Wd/ypnVzM68dfdQIIAj1iMFAD/lXGlEUmKXg5Ov2VQDlTntQoanCYrAg +8CttPzjrydgLZFq3hrtQmfc0se5yv1WHS69+BsUOG09RvvawUDZxUjW19kyeN9THaNRgow3 kSuArUp6zSmJABEBAAHCwF8EGAEIAAkFAlMQe0ECGwwACgkQGuaGDlbL0pMN5wgA4bCkX/qw EVC06ToeR6C2putmSWQMgpDaqrv65Hubo+QGmg2P4ewTYQQ4g6oYWS03qHxqVVWhKz7FjfrV +dH8qbCLfSgIcvdBha7ayGZVrsiuMLKGbw36fcmkZPpSDOfHcP0XH8Z+u9CWj0xUkTxAlZ/7 i6gYSUpG2JWNtdmE/X8VVEyXusCLwy0K0BI60A/4dRTIX3C4QKrJ3ZbUXegz70ynjHf+lQMZ 9IZKASoRMuS5FozPQh6abvmwZEPdf5I9riUElzvHrqJ8Bx0t3Pujdoth+yNHpnBxrtO8LkQd rQ58P0SwcaIX33T2U9pG8bhu5YVR88FQ8OQ0cEsPBpDncg== In-Reply-To: <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE] X-Rspamd-Queue-Id: 4Xy0bQ451vz4KYb X-Spamd-Bar: ---- List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org On 25/11/24 23:15, Dimitry Andric wrote: > On 25 Nov 2024, at 23:12, Mark Millard wrote: >> >> On Nov 25, 2024, at 13:27, Guido Falsi wrote: >> >>> On 25/11/24 22:18, Dag-Erling Smørgrav wrote: >>>> Mark Millard writes: >>>>> Guido Falsi writes: >>>>>> On 25/11/24 09:17, Dag-Erling Smørgrav wrote: >>>>>>> Dimitry Andric writes: >>>>>>>> Probably best to create a bugzilla ticket, but as I said before, I >>>>>>>> cannot reproduce this. >>>>>>> I can. My builder is running 15 and sees segfaults while building >>>>>>> packages for 14 and 15 but not for 13. >>>>>> BTW removing optimizations (CPUTYPE) for only the affected ports made >>>>>> guile2 work again. Did not solve the issue with sassc though. [...] >>>>>> I'm also using ccache, but that does not look relevant. >>>>> I've never used ccache or analogous and get the libsass.so.1.0.0 >>>>> .got.plt corruption that I've reported on the lists anyway. >>>> I don't use ccache or optimizations. Here's an example of sassc >>>> segfaulting in a 14.1-RELEASE-p6 jail: >>>> https://pkg.des.dev/logs/data/14amd64-default/2024-11-24_19h29m04s/logs/errors/plasma5-breeze-gtk-5.27.11.log >>>> which matches the following entry from `/var/log/messages`: >>>> Nov 24 21:23:06 pkg kernel: pid 71277 (sassc), jid 253, uid 65534: exited on signal 11 (core dumped) >>>> The poudriere host is a bhyve VM with 48 cores and 192 GB RAM on a >>>> 32c/64t AMD EPYC 7502P with 256 GB RAM. >>> >>> I sincerely hope this is not relevant but my CPU is also AMD: AMD Ryzen 5 5600G >> >> The amd64 system type that I have access to and used >> for my testing: >> >> AMD 7950X3D (16 core, 32 thread, so 32 FreeBSD-cpus) with 192 GiBytes of RAM > > I'm on Intel, and I don't see any crashes at all. So, are we looking at some CPU specific issue here? We can't say for sure, but we definitely have all people reporting the issue on the same CPU brand, so it's some indication I guess. I was hoping it would not come to this because I suspect such issues are quite difficult to diagnose. -- Guido Falsi From nobody Mon Nov 25 23:21:20 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xy1tQ4SgVz5fNCl for ; Mon, 25 Nov 2024 23:21:34 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic305-21.consmr.mail.gq1.yahoo.com (sonic305-21.consmr.mail.gq1.yahoo.com [98.137.64.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xy1tQ13GLz4TZj for ; Mon, 25 Nov 2024 23:21:34 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732576892; bh=2wvIXmGM/HoIso3uEHyF51Uav0DR3H3PSK9mrzLh9yM=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=OrFTWnGdhxHaFU7uaz4T/9BcnVcwSgHxg3mQaKn1piDhozhehJv4PIIHErT4L7a+Ri2Eyzsyre/oRvJakgy0tKfQp9PH4MuTukiXPxD/kleKnpeOkuEPR30sWJi9XBxZ+zCsbf3IWSoXlbmsmjcvarndpoaXbcSrWuo/P/ynhjgAr7h/xJAiCZztEn6IhOyd+c2gQ0yyhmkdBgnqD3UK2VRFL3d8ZK39Prpc3ivijk0ok+AHCnmxb4/sUjL3Y5BzcvmkcCvF8SXhtm6XD+T1kzB0vN+U3Rtt4gGToAnh7k4bFI2ASy2FK93TBeLxWEfUGZdFlzpvZZlqNkK+msTt5w== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732576892; bh=YXjU16JOR0j898yyvkKixhuuOpB725bfPtRBDgUXMp6=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=X+3viY31zU10OWc+kCSixqMEO8XEXo2+2xLALKHncxlXybRYdUqkQZCJTAGIQBHPuT42RNMrueAu7VyPOU1E+c5ViTaldT3Sjs1dwHMPQS7CBf3S3DOieTRxrmmO/ZmkBTN8/35d+bi+VQspSWVOyZyKz8nR+rsU9UP7fPXUteamxY+wv5A5TAXsFLw3v3TUpLM3RB4sDCKcR7A4RtxQOz5CbA9EHBUltdTnQprJeJ+CWxeC3w96PCg6HGVvwopwn0eFYNMy8hWpyeO7ltFaPn/t0D6vvMAbPIOtjcg+Zh2nnnyeKrJnIUXkTfSbhDbaJiLKsHWRCnHWCfteCybE+g== X-YMail-OSG: 9XspJLQVM1mGCn9CT9V0Nk.bJtD7NmxfT.AHCn3KID5BUqO8MBPQRMaaFRif5gj 76BMn70p1BnlbK2EVoytZb_4grp96E6I6Pfyu3qmzd10yxpyyMq.g6FYf3sRdtT_KYlBlpy4gc1x KiRLz2cma6aZd_INQCQygXeF3v5oHMS3Ji6lylB2OVOhd0BJZUiPEpxTKyLL_Av.mNGKq2BX4jol kwTGBBDLMBlK1UM6z1Q24e4TuEUZ0vcHMAegajGH6vyBiJ_sQt2kwmdTdbm9L8m9uyGhZ5tOdmCD xaCIgr33skqk65mSKlvL8qWB5VVlUCnt8bpCcS6q7UX2VVe2h7dqIKN_k8YhCZCXajzuXPnfgcdK LnJhZI.yHVSUe4T3eDEpXGCeWQUW3GualNQFjy6rFFDdj06_t3ZYg8D0Tnu6LY6L87JSht.T_nuX eap_OnXl7k8A8VFFI_HoWpC5hwWHIA_nDPjOzHTeFkejarNLJtrkdSo.7IhvniWut.8XN_WKPM1P UD6z71A.kqMlFYBkFHp0AlIQ2Qqwcg3ptOfEHLDpu0H7HurisZmVPJShrSWQ_Gua.9OFXYqNAYaR zxtioojUQKwpibpNyFADVQGgIoWmzhcci4yZXGzg4Ox6CWP.lhNaOepVAdtty9ea1lxz0lgXzNtO N.NcaUbSFG2l9l_AwKk2_VDpi3QsE0oXVT7GIMhMG7tyyqZKvpxrhiF_33GCOB4coYdykHoT15H8 Q0GzIE1kCthBZMEn0MWpCclUo4TYrB2PD7jaIWwtif4bGZ_EQi9HoNpezlgW3sjNLUajZR0keW7v gFy3WcIKdDVeno_2vKUmqr0RpAl8EXjo72.dLBlQOexcnIyJc2D.jif56XYj.w_vr96a0ZecmWW0 ZQ6pmNMj7D7Gea0NaCqvC3F17Dcg8PMAFnt958GLYCpVximkL9NdCll3yUEbqNZL77ZVMB0MCCfK xmZWISV.eiWaByUSUvAY3u2CvRf2YZbkUj3Cj0eylo7xaMtEyMjpenxqYcyCBODG.tbmxIQe.cXS 7AC0CZr_6daAUWgrPLSHATW58.ijNoiBVzof1UjDcp2DjXfLa4BskUhTSg8xFuU2v5rlpAu6MJIY 0sb5ufG1i8QR6LJFuwiEatz4LSzfj2hAfcquZyShK1Hk3PI1K.KBUFxQjziBHemLD.xDZTHJMnp5 x_rIffrddxKjiukvSOODFbOTAOJsVDMKS220IISiCzZhkneH4b7Z3_pVn6Y6N3.e6iGUZLQXdP1K yDjH7Edp76Up9IWZ8MlMsGwibD8hAg53c9z15_oX2Voau6ISLQ5xgZSBDY_eEptweiWnALj2NYMz izQctbS8Wb8_nsqtRZ_OBlqefYOfn7_VnYXzacn.Z.QuEKBxP7xmTfKcLOcvVdF91aGzEOV1ortV UILXxnee1mqTDWakzvrARj1rtZpai0YkrQKZGMdIv5N9fxB4uV4gLdrfLZJwQC1cnnL5BrIB5Sy. mdL2mPa5DzU00PkDhRw82c1P0yedP3l6QRmj0hJdWwmOyMNLPGhMuiYU139U4gepCj7swNCGC7zU MNrD4AvXYTrgm9mnY3aoU_1YSdp7QqWY8D0pLEiANaADiiLpAZYvqDEptKL6wE04wdTdDLg6kWuk lSMaA8kt7uR9hXcvLfXCnK8DYtrLKtdZy3DLSmNZqD55RdElivzWEILHmUTlNAlxhfQYmNzb2pe8 fDz0SDkuf_aJ4JJU7RGbSDF11TPmzkIH4ZtSM9HiJ.k2U8s0fIGBVnTwnKC3zzbbfq_nGInl8g9k 2wUwF.3DeHLo306OcCr31hUEcOQ13pW.rgoFuq2c8FhHbhXLNygmzB6aEe.nse7caoq_Jw4.I1hq Ywnb_HoopaAhobzT25Ru_Wjf_Kb3COyRMP6nhgGJp4NQlZaVQbfpnkJlGadc0jE6c9o6zUqzENxu YP8IjOTGqcNXjzoRYNcK8fdNezt6NAdKymJh1S0AmxDNCojlnIM03_P4HNBs9.aIFcXQt6PK_lYx anOSVKmEA.6GvuElKqCqigGiXX_YiZlQyvzin8SZeL3f8hvssBIR9WqNuXvVGRKP_PvvOhJlXdBw bbTaWDynnQ4uVv.rJvwUbEnuIPmKe_i.DQeaVHAYYP_t3wHpnpRT.dseL1iu7Ei8flS1flElP.ZS kDME8POOnuv.3Uj1WQY97mOrAArC4caGNv9dtES1em9.ZGwyEAqtxKR5Dilm8QFPOCvE9DMxfddv qjBwuazm2mjVuDYmBacPLyJIKvb.lOjrBVw-- X-Sonic-MF: X-Sonic-ID: 2cac72ba-1c42-402b-b2c9-635a1c7b5f9b Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.gq1.yahoo.com with HTTP; Mon, 25 Nov 2024 23:21:32 +0000 Received: by hermes--production-gq1-5dd4b47f46-mb2l9 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID cf9a47b5593f00b63cf44df9537267a2; Mon, 25 Nov 2024 23:21:31 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51.11.1\)) Subject: Re: port binary dumping core on recent head in poudriere From: Mark Millard In-Reply-To: <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> Date: Mon, 25 Nov 2024 15:21:20 -0800 Cc: Dimitry Andric , =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , ports@freebsd.org, FreeBSD Current Content-Transfer-Encoding: quoted-printable Message-Id: References: <46E3A370-A3E0-4BAF-B707-87F94F98E248@FreeBSD.org> <5ee47c3d-f80e-4d50-9b6a-acb3c98e80e0@madpilot.net> <7c9c3cf5-bbd1-4642-8d04-33aa07a4db02@madpilot.net> <9df256a8-c6ed-46d9-b955-fc2657c12d36@madpilot.net> <5c502054-7353-4a1e-8350-c403482e9c0d@madpilot.net> <3127C3BA-FC93-4636-ADDB-89518DE9C60D@FreeBSD.org> <86ed2zsp6l.fsf@ltc.des.dev> <5f24a570-26e0-4c0a-817f-591a234fd07b@madpilot.net> <5918C6A1-8FDB-40CA-8C86-EB7B7BE75A2E@yahoo.com> <86ed2zc8r5.fsf@ltc.des.dev> <45098ccf-4dc6-426c-849a-c923805d6723@madpilot.net> <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> To: Guido Falsi X-Mailer: Apple Mail (2.3776.700.51.11.1) X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4Xy1tQ13GLz4TZj X-Spamd-Bar: ---- On Nov 25, 2024, at 14:23, Guido Falsi wrote: > On 25/11/24 23:15, Dimitry Andric wrote: >> On 25 Nov 2024, at 23:12, Mark Millard wrote: >>>=20 >>> On Nov 25, 2024, at 13:27, Guido Falsi wrote: >>>=20 >>>> On 25/11/24 22:18, Dag-Erling Sm=C3=B8rgrav wrote: >>>>> Mark Millard writes: >>>>>> Guido Falsi writes: >>>>>>> On 25/11/24 09:17, Dag-Erling Sm=C3=B8rgrav wrote: >>>>>>>> Dimitry Andric writes: >>>>>>>>> Probably best to create a bugzilla ticket, but as I said = before, I >>>>>>>>> cannot reproduce this. >>>>>>>> I can. My builder is running 15 and sees segfaults while = building >>>>>>>> packages for 14 and 15 but not for 13. >>>>>>> BTW removing optimizations (CPUTYPE) for only the affected ports = made >>>>>>> guile2 work again. Did not solve the issue with sassc though. = [...] >>>>>>> I'm also using ccache, but that does not look relevant. >>>>>> I've never used ccache or analogous and get the libsass.so.1.0.0 >>>>>> .got.plt corruption that I've reported on the lists anyway. >>>>> I don't use ccache or optimizations. Here's an example of sassc >>>>> segfaulting in a 14.1-RELEASE-p6 jail: >>>>> = https://pkg.des.dev/logs/data/14amd64-default/2024-11-24_19h29m04s/logs/er= rors/plasma5-breeze-gtk-5.27.11.log >>>>> which matches the following entry from `/var/log/messages`: >>>>> Nov 24 21:23:06 pkg kernel: pid 71277 (sassc), jid 253, uid = 65534: exited on signal 11 (core dumped) >>>>> The poudriere host is a bhyve VM with 48 cores and 192 GB RAM on a >>>>> 32c/64t AMD EPYC 7502P with 256 GB RAM. >>>>=20 >>>> I sincerely hope this is not relevant but my CPU is also AMD: AMD = Ryzen 5 5600G >>>=20 >>> The amd64 system type that I have access to and used >>> for my testing: >>>=20 >>> AMD 7950X3D (16 core, 32 thread, so 32 FreeBSD-cpus) with 192 = GiBytes of RAM >> I'm on Intel, and I don't see any crashes at all. So, are we looking = at some CPU specific issue here? >=20 > We can't say for sure, but we definitely have all people reporting the = issue on the same CPU brand, so it's some indication I guess. >=20 > I was hoping it would not come to this because I suspect such issues = are quite difficult to diagnose. Unfortunately, for amd64 I only have access to: ) An old ThreadRipper 1950X system (untested so far) ) The 7950X3D system No Intel systems. If someone had both AMD and Intel and could have boot&operate media that should work for both, say USB that can be simply moved between machines, running test on both would be appropriate. (Implication: the media not being tailored to the cpu specifics so the same system software is tested in both places.) I'll note that the media in my context is PCIe Optane, ZFS based. I could try a U.2 Optane in a PCIe adaptor that has UFS instead for building textproc/libsass . (The U.2 content is an basically a rsync of the ZFS Optane media's live directory tree, with node naming and such adjusted afterwards.) What do other folks have for the file system(s) involved? =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Tue Nov 26 00:15:09 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xy35y4f6cz5fRBp; Tue, 26 Nov 2024 00:16:38 +0000 (UTC) (envelope-from yasu@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xy35y44JCz4gQk; Tue, 26 Nov 2024 00:16:38 +0000 (UTC) (envelope-from yasu@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732580198; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=i1Q8AJJed9RkNCrxe9kZ3w3z40us91vhv6KyUPV80Ko=; b=XtvW8SR2Pt0ySAgk87U3KGu9RJuWRDJIVRgISBxUecMKQzd5C0C/Rt9VOTBRB23PMoVXrO xnNZG2789XlRRltEgooHuWyTdsj1ZJ2NFJuOCQ5J0rjVu/5SfAOVbt4jio+n1lgAHlaPGt dY9wneOLW3VxZBPY+ZKPSfqvtNsFcgNcCiH+idCwBlxoVREl754Wss4WOX3my7/DSNdD3F UeFGM63yBjhQt+yY0V9ki3kMKLVu9M/Lk2IkjjUc2O3WarjZ4mUk+Pp5j4qLBZHCx8e4Y3 zTJsN9PzmCjka6xQ+z2G2AYMFtJLgP5x3/AKTYeahsTW2F/d5AzW64YnKWSPFg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732580198; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=i1Q8AJJed9RkNCrxe9kZ3w3z40us91vhv6KyUPV80Ko=; b=xLx1WH2nUTbkloQH0vX0EOSA7wcpWOxOZ+3ZAYI7G1+P3BwefHmHydWlSlK8BNtPilLzoq dJ2puWbbN8nXjVGgmznd+qtrHQdugf62Wp58j2SIC31HLH3UbjWhyovIcEBW4BzLSokKyO byIZAMrXsPdWxlN79Iyv2ATJdPDmCj16xipjZDNNv1D/uqXeatKMYxMxKDCjHb1jEEFLDT e+IaNEcrUer7Y68v7EPNy1RQvMEBP52m/P+d+ZYIcwE+fTn7wg6STqUO9z2oTCQEgDgo7k V+SXmZ0c+IiZcNPnJUrYT8I6eKCSUavKPCtjXsutlcZeYcPbNJgjGPzj9OicHA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1732580198; a=rsa-sha256; cv=none; b=KVt9m3H1LuUDZLzkfPJ4sDKTEw5weOu4fczWDkFk1jt+yD/YN32PJoNIh309YDj5rLKmuc IUjiWYzH3KIAq2hQKB9eIpiFmNju529WlAqF2C77inRv/Mwvv4CANoTi3qc6Z1MrHXUUCl /1GaqLZNsYo3DJonmD/1SDjkHYklQ/lwRrPTskwurncb+sAYc9POdE7vWzgxA4HwwxVm5t A4VnsKDjlSsBiCi/TMNrm+Qo9WxTYwfmYFisRvZ6ZEGPb64G9g5UI+qlklQhrsYBA2+/IG ib2O+7px7u7/OB9YdFslVcaL1QiE+5rOYvqsUFUau0qj9uyJcbmwlXx+K62/sw== Received: from localhost (unknown [IPv6:240b:11:220:fe00::174:11]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) (Authenticated sender: yasu/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Xy35w71DszPv1; Tue, 26 Nov 2024 00:16:36 +0000 (UTC) (envelope-from yasu@FreeBSD.org) Date: Tue, 26 Nov 2024 09:15:09 +0900 (JST) Message-Id: <20241126.091509.940753556636888621.yasu@FreeBSD.org> To: dim@FreeBSD.org Cc: marklmi@yahoo.com, mad@madpilot.net, des@FreeBSD.org, ports@freebsd.org, freebsd-current@freebsd.org Subject: Re: port binary dumping core on recent head in poudriere From: Yasuhiro Kimura In-Reply-To: <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> References: <45098ccf-4dc6-426c-849a-c923805d6723@madpilot.net> <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> X-Mailer: Mew version 6.9 on Emacs 31.0.50 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 Content-Type: Text/Plain; charset=utf-8 Content-Transfer-Encoding: base64 RnJvbTogRGltaXRyeSBBbmRyaWMgPGRpbUBGcmVlQlNELm9yZz4NClN1YmplY3Q6IFJlOiBwb3J0 IGJpbmFyeSBkdW1waW5nIGNvcmUgb24gcmVjZW50IGhlYWQgaW4gcG91ZHJpZXJlDQpEYXRlOiBN b24sIDI1IE5vdiAyMDI0IDIzOjE1OjIwICswMTAwDQoNCj4gT24gMjUgTm92IDIwMjQsIGF0IDIz OjEyLCBNYXJrIE1pbGxhcmQgPG1hcmtsbWlAeWFob28uY29tPiB3cm90ZToNCj4+IA0KPj4gT24g Tm92IDI1LCAyMDI0LCBhdCAxMzoyNywgR3VpZG8gRmFsc2kgPG1hZEBtYWRwaWxvdC5uZXQ+IHdy b3RlOg0KPj4gDQo+Pj4gT24gMjUvMTEvMjQgMjI6MTgsIERhZy1FcmxpbmcgU23DuHJncmF2IHdy b3RlOg0KPj4+PiBNYXJrIE1pbGxhcmQgPG1hcmtsbWlAeWFob28uY29tPiB3cml0ZXM6DQo+Pj4+ PiBHdWlkbyBGYWxzaSA8bWFkQG1hZHBpbG90Lm5ldD4gd3JpdGVzOg0KPj4+Pj4+IE9uIDI1LzEx LzI0IDA5OjE3LCBEYWctRXJsaW5nIFNtw7hyZ3JhdiB3cm90ZToNCj4+Pj4+Pj4gRGltaXRyeSBB bmRyaWMgPGRpbUBGcmVlQlNELm9yZz4gd3JpdGVzOg0KPj4+Pj4+Pj4gUHJvYmFibHkgYmVzdCB0 byBjcmVhdGUgYSBidWd6aWxsYSB0aWNrZXQsIGJ1dCBhcyBJIHNhaWQgYmVmb3JlLCBJDQo+Pj4+ Pj4+PiBjYW5ub3QgcmVwcm9kdWNlIHRoaXMuDQo+Pj4+Pj4+IEkgY2FuLiAgTXkgYnVpbGRlciBp cyBydW5uaW5nIDE1IGFuZCBzZWVzIHNlZ2ZhdWx0cyB3aGlsZSBidWlsZGluZw0KPj4+Pj4+PiBw YWNrYWdlcyBmb3IgMTQgYW5kIDE1IGJ1dCBub3QgZm9yIDEzLg0KPj4+Pj4+IEJUVyByZW1vdmlu ZyBvcHRpbWl6YXRpb25zIChDUFVUWVBFKSBmb3Igb25seSB0aGUgYWZmZWN0ZWQgcG9ydHMgbWFk ZQ0KPj4+Pj4+IGd1aWxlMiB3b3JrIGFnYWluLiBEaWQgbm90IHNvbHZlIHRoZSBpc3N1ZSB3aXRo IHNhc3NjIHRob3VnaC4gIFsuLi5dDQo+Pj4+Pj4gSSdtIGFsc28gdXNpbmcgY2NhY2hlLCBidXQg dGhhdCBkb2VzIG5vdCBsb29rIHJlbGV2YW50Lg0KPj4+Pj4gSSd2ZSBuZXZlciB1c2VkIGNjYWNo ZSBvciBhbmFsb2dvdXMgYW5kIGdldCB0aGUgbGlic2Fzcy5zby4xLjAuMA0KPj4+Pj4gLmdvdC5w bHQgY29ycnVwdGlvbiB0aGF0IEkndmUgcmVwb3J0ZWQgb24gdGhlIGxpc3RzIGFueXdheS4NCj4+ Pj4gSSBkb24ndCB1c2UgY2NhY2hlIG9yIG9wdGltaXphdGlvbnMuICBIZXJlJ3MgYW4gZXhhbXBs ZSBvZiBzYXNzYw0KPj4+PiBzZWdmYXVsdGluZyBpbiBhIDE0LjEtUkVMRUFTRS1wNiBqYWlsOg0K Pj4+PiAgaHR0cHM6Ly9wa2cuZGVzLmRldi9sb2dzL2RhdGEvMTRhbWQ2NC1kZWZhdWx0LzIwMjQt MTEtMjRfMTloMjltMDRzL2xvZ3MvZXJyb3JzL3BsYXNtYTUtYnJlZXplLWd0ay01LjI3LjExLmxv Zw0KPj4+PiB3aGljaCBtYXRjaGVzIHRoZSBmb2xsb3dpbmcgZW50cnkgZnJvbSBgL3Zhci9sb2cv bWVzc2FnZXNgOg0KPj4+PiAgTm92IDI0IDIxOjIzOjA2IHBrZyBrZXJuZWw6IHBpZCA3MTI3NyAo c2Fzc2MpLCBqaWQgMjUzLCB1aWQgNjU1MzQ6IGV4aXRlZCBvbiBzaWduYWwgMTEgKGNvcmUgZHVt cGVkKQ0KPj4+PiBUaGUgcG91ZHJpZXJlIGhvc3QgaXMgYSBiaHl2ZSBWTSB3aXRoIDQ4IGNvcmVz IGFuZCAxOTIgR0IgUkFNIG9uIGENCj4+Pj4gMzJjLzY0dCBBTUQgRVBZQyA3NTAyUCB3aXRoIDI1 NiBHQiBSQU0uDQo+Pj4gDQo+Pj4gSSBzaW5jZXJlbHkgaG9wZSB0aGlzIGlzIG5vdCByZWxldmFu dCBidXQgbXkgQ1BVIGlzIGFsc28gQU1EOiBBTUQgUnl6ZW4gNSA1NjAwRw0KPj4gDQo+PiBUaGUg YW1kNjQgc3lzdGVtIHR5cGUgdGhhdCBJIGhhdmUgYWNjZXNzIHRvIGFuZCB1c2VkDQo+PiBmb3Ig bXkgdGVzdGluZzoNCj4+IA0KPj4gQU1EIDc5NTBYM0QgKDE2IGNvcmUsIDMyIHRocmVhZCwgc28g MzIgRnJlZUJTRC1jcHVzKSB3aXRoIDE5MiBHaUJ5dGVzIG9mIFJBTQ0KPiANCj4gSSdtIG9uIElu dGVsLCBhbmQgSSBkb24ndCBzZWUgYW55IGNyYXNoZXMgYXQgYWxsLiBTbywgYXJlIHdlIGxvb2tp bmcgYXQgc29tZSBDUFUgc3BlY2lmaWMgaXNzdWUgaGVyZT8NCg0KVGhvdWdoIGhvc3QgaXRzZWxm IGlzIGd1ZXN0IG9mIFZpcnR1YWxCb3gsIEkgY2FuIHJlcHJvZHVjZSB0aGUgaXNzdWUNCndpdGgg SW50ZWwgQ29yZSBpNyAxMjcwMC4gVGhlIGRldGFpbCBpcw0KDQpCYXJlIG1ldGFsIGhvc3Q6DQog IENQVTogSW50ZWwgSW50ZWwgQ29yZSBpNyAxMjcwMA0KICBNZW06IDk2R0INCiAgT1M6ICBXaW5k b3dzIDExIDIzSDINClZpcnR1YWxCb3g6IDcuMS40DQpHdWVzdCBzeXN0ZW06DQogIENQVTogeDgN CiAgTWVtOiAzMkdCDQogIE9TOiAgRnJlZUJTRCAxNS4wLUNVUlJFTlQgbWFpbi1uMjczNzk2LTY2 NDM5NjU5OTgyIEdFTkVSSUMgYW1kNjQNCiAgUG91ZHJpZXJlOiAzLjQuMg0KICBKYWlsOiBTYW1l IGFzIHN5c3RlbSBPUw0KICBQb3J0cyB0cmVlOiAwYWJiMDAwZTFlZDUgb2YgbWFpbiBicmFuY2gN Cg0KSFRILg0KDQotLS0NCllhc3VoaXJvIEtpbXVyYQ0K From nobody Tue Nov 26 01:05:52 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xy4C5310yz5dVw5 for ; Tue, 26 Nov 2024 01:06:09 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-21.consmr.mail.gq1.yahoo.com (sonic317-21.consmr.mail.gq1.yahoo.com [98.137.66.147]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xy4C42g3Rz4pM4 for ; Tue, 26 Nov 2024 01:06:08 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=GgZDkinl; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.66.147 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732583165; bh=qKOT7cF2VobtoHjV68hknMDGVCXfMSro+q9lW8cmwyo=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=GgZDkinleLahgju9NikHwNMWxsW7zgUeN1JsbkD6sSyZiaD9e5j+bvbMsn/Fa+DWr/1zd69AwTII1BQ5uM96EeH25eLdTu9BM+QvwdTLZLSwSXh5oevcWTwXKLSy1spk5ulhSP4q/+JZqk9XP/iZgDNn8193CQgO7wZNya/ODmaqsUoLPdmt9sP5jsxEQeJQISr2XW3npxhYTTWPA6oJg36rkTwaIKbR6QmkUIXfBRBKBRXNx4B+ncYie/w9eq2SYDkd4WxAMoa8BgxRTDbbVK27O9E2WSr5xq51gl9JcZ9cdwDUkCWaUOMakfcsB7zWLhKf7ewjDyFXt/kd2Ys5sw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732583165; bh=mdzeFetYww+O+8f9MBLm3uULRil4EySbJCmQxp3istl=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=st5YvdX0aa/LLNRu7lnl6MFSCcYWHjmLTbvBcaCnim7GQamM7ubRvCw+06Bcx9sN57hC9Txom9sdxp/oDJaIeFTDTmxwpZw8MRusuzXp2WyhcpRDSWXRcec1MgzCjNQJYx8U98zIvzLw1/4LH6adot2WoAauEH8be0J3rPdLdAzsLAPqfESDLHk47fz72SQz81fkmXQDrSjbUg/0vZEoWmoD7WpsCKONb/RpbWxiK28iREuJnWDkskqx2ymOVnLp7CHHPEAhLiIhdzvbrNbM/+6OZI24xYww+fQYsjr6JROEeio5bDU4Rom2M8CEw7BRFRl1GJaB/ZHJ4Wz3DRnUAQ== X-YMail-OSG: glUj_eMVM1lHEaP2x4dyA9crTj66wB4aNjzEdSa7T8f3OSHJFdNx3qP9.w5bYXB 2CVff0m06rzypSN5aeuVLiGYI8FxnMprQMpm5PYSao1AnS5i1X7bC7EvP4NQ67kxuYnmNhLcgKf. DjDZCSDsHP3rWB4nfNKx0KDMiXJV_ks17HRuJIMcaNhEGxYm4Pv_1JNn7kdTNQHya88XY5qfD498 exzEFLytcV.VL6bA2VNWVnnVObOWPiBeAjLm_W58TriyvzMFW6BMqWzld6vKlW3e2ajpB4mJJJ4b AnhjAG9kILRG4O3yAILuslvsuv2MrDT_kcLI2Kre8ISXSKrp1oCQ_zJj3FgXYRxGzi9wB.4kQ6bm OnEjlJg.2JOz9750OFRs.zxcXmhrtG.9ouWpH84P79whSwXcwAqa1d8RzZZ0mHcaOKvcUTmafqPj 7ROIskxYkFin8Sp_p0_W3B9QQuc0oZfE95StOxSg7Ryv2x9NvQSR2Fjc.RggmWWyB.00zVu0unnL aPRKE.H2ucvWJeFhgwIMY6SxnfWrxMnkYuQOWOS5m6DfjOzYAZGzpfRg_rc3RfEE4EHnykS2E3QT dMG_ou_FdvvEQarv2ChUIl11GAIxz6ZpmLtOl6H2VQOVM4DtsOaybvmUcynGobfXpUj5bSe5BJu4 tkRKaaWtDEE4_B6DWxrfjF9n5JlGqcHMNqV2uvE2iy45Z7NpIUs2eTe9CY2WI.DNk.4T0Wnj4b06 xvBmBliV3EYk93g.GPjupmTvjnjxMpuDK21YiNXGNcRzDS.X9t5.4vdl401dLo2IJxuUsWbM0ccn jAtuEdSeTwDzTKjSDqcm2j0CLBdQkgJ5NgrLbA.zVOwcf_53OWjKP9x1r0XYvN047XXLX9Ir2aIX ek2waCWYKaOyTcM2nitRe9mFTZOqZhQcTTjW3Rw0JCNlAQsvKtAYcaVrWT4hdy0MFwZ8llHJEJRg 6XJRS7An47LpjbaqbUwgZZ0gx_R2HsfJqWbx1SjqZOOWWjgjXj30Arhbkvx3mOu_zBsiKvF7WHqK Mxpc5irIwLfZ1D_osN8FdnwMkaDjOFtvYnrow5t38Bd5YIpaEmzrVfU8c.k1sajHQ2.eial9PI5E BCFASp8xUT2WXob.1R2DUYEv_fX3PqiWGVSZssoPNa73IqeFL6Brm9cNFmCeN9x69nV08to5VKXx Nvqk2BhaWvlBs97Twt3yiURDonB1TeWRJNHb2ZUX7Ia2Gd0fu.xuIFU6Oxn8at6gTlizIv2b_JGK Y_Qf9S9_nJHTTrfCIlfYU0kXODrtkUkrIds.ZcR1ctaqyG4THZ5UOBpZ2XuMZ2e5sl74B54l6wL. MMEoZ1MkdRFb6uKRSiX3X5uOTsqtzf.GpEGNHQ93uDoCyrWgUUlCo2zvWtSZ.WIyDoPDdyo7m17y oZ4ILcorJsf_Lep66rIi4tXV88w3n4sA9pLdyowP5BaXvIQs8J774udmPABAjzF2bFcCAsuqEjCq kcJLJpYc_bef6HEGxqsXoGkbpAZx2dyE2AKrFN1pADPLABMZvQMJTa7YZPLOwSSUlJl.MTguX3HK UPDjRlHF6Ji4fyxB.V3CLpNEraPCZ4k5Ad9wV.RHXoYMaVMY1M.0mb1zomH8onuW6St0.aobhbAG ivfGFkHg6lUECbI0gmAqN9.524tKMPpfbDs0Ksqs.8X6X7sDPORPOjke.bRjkt2w_2f0OMV0_kD_ e8KSJUmW7nCNB2uolpq5J0NqKGANUpOFPEkdJ55Od032bHcL_Ral4XvgZUAydkuITjsFNGk8UWw9 v7wtGpGQfgrCAdTMI.wf94hSnfyOvQeFOD8JLnPRQtJGbqFGqF6kvyOMxVD9a232AO1F1yGsOCeE VdkDapHm0kNghvD1AUUB.XMDqgpmHKlInA.iWY.XJT0z_OnLMkizi5tqNK1euSE_Q0yszHjrSz4. GlUlAUxEWPWRR2fx3xrB7I.2AH1K9JG8r7i4ilBEoNygfnmX5c2fmXFUs5XjzN2qhTSvSFywyvTb oSHF3w09vFNRCmAnFztF6pdJQNCtXNG1VIl.drPrSwhoRiu04YTQsXnr2x116fFBR5u52ZU5b3yW MfVqhvvIz5uG4zVwtrn.owypuU16OtQT8Zd8aq4agGVoYIhecEPv4oOpOVlT799dSCFeYj31ddSz gFv8RkYQX_Xaj3Ni3B3T2NFod7hevZfijSwAI8Mmul6z9Zqt4.IAGItT3YUrvpLaMpw4G5jByM_O SaL5zpQEN3.IlHb29nxYOrCnAb8qlTaxCyKH9VYT_sXwiROSHeH9nx24jgWd7mMgINUw.EV_8dMf oUQwp.yTM X-Sonic-MF: X-Sonic-ID: beedd218-edf1-452f-94e2-2b8a065f61ec Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.gq1.yahoo.com with HTTP; Tue, 26 Nov 2024 01:06:05 +0000 Received: by hermes--production-gq1-5dd4b47f46-ps69l (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 33f59f478d8ac8e023c55ee347f98df2; Tue, 26 Nov 2024 01:06:03 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51.11.1\)) Subject: Re: port binary dumping core on recent head in poudriere From: Mark Millard In-Reply-To: Date: Mon, 25 Nov 2024 17:05:52 -0800 Cc: Dimitry Andric , =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , ports@freebsd.org, FreeBSD Current Content-Transfer-Encoding: quoted-printable Message-Id: References: <46E3A370-A3E0-4BAF-B707-87F94F98E248@FreeBSD.org> <5ee47c3d-f80e-4d50-9b6a-acb3c98e80e0@madpilot.net> <7c9c3cf5-bbd1-4642-8d04-33aa07a4db02@madpilot.net> <9df256a8-c6ed-46d9-b955-fc2657c12d36@madpilot.net> <5c502054-7353-4a1e-8350-c403482e9c0d@madpilot.net> <3127C3BA-FC93-4636-ADDB-89518DE9C60D@FreeBSD.org> <86ed2zsp6l.fsf@ltc.des.dev> <5f24a570-26e0-4c0a-817f-591a234fd07b@madpilot.net> <5918C6A1-8FDB-40CA-8C86-EB7B7BE75A2E@yahoo.com> <86ed2zc8r5.fsf@ltc.des.dev> <45098ccf-4dc6-426c-849a-c923805d6723@madpilot.net> <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> To: Guido Falsi X-Mailer: Apple Mail (2.3776.700.51.11.1) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FROM_HAS_DN(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_IN_DNSWL_NONE(0.00)[98.137.66.147:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.66.147:from]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_FIVE(0.00)[5] X-Rspamd-Queue-Id: 4Xy4C42g3Rz4pM4 X-Spamd-Bar: --- On Nov 25, 2024, at 15:21, Mark Millard wrote: > On Nov 25, 2024, at 14:23, Guido Falsi wrote: >=20 >> On 25/11/24 23:15, Dimitry Andric wrote: >>> On 25 Nov 2024, at 23:12, Mark Millard wrote: >>>>=20 >>>> On Nov 25, 2024, at 13:27, Guido Falsi wrote: >>>>=20 >>>>> On 25/11/24 22:18, Dag-Erling Sm=C3=B8rgrav wrote: >>>>>> Mark Millard writes: >>>>>>> Guido Falsi writes: >>>>>>>> On 25/11/24 09:17, Dag-Erling Sm=C3=B8rgrav wrote: >>>>>>>>> Dimitry Andric writes: >>>>>>>>>> Probably best to create a bugzilla ticket, but as I said = before, I >>>>>>>>>> cannot reproduce this. >>>>>>>>> I can. My builder is running 15 and sees segfaults while = building >>>>>>>>> packages for 14 and 15 but not for 13. >>>>>>>> BTW removing optimizations (CPUTYPE) for only the affected = ports made >>>>>>>> guile2 work again. Did not solve the issue with sassc though. = [...] >>>>>>>> I'm also using ccache, but that does not look relevant. >>>>>>> I've never used ccache or analogous and get the libsass.so.1.0.0 >>>>>>> .got.plt corruption that I've reported on the lists anyway. >>>>>> I don't use ccache or optimizations. Here's an example of sassc >>>>>> segfaulting in a 14.1-RELEASE-p6 jail: >>>>>> = https://pkg.des.dev/logs/data/14amd64-default/2024-11-24_19h29m04s/logs/er= rors/plasma5-breeze-gtk-5.27.11.log >>>>>> which matches the following entry from `/var/log/messages`: >>>>>> Nov 24 21:23:06 pkg kernel: pid 71277 (sassc), jid 253, uid = 65534: exited on signal 11 (core dumped) >>>>>> The poudriere host is a bhyve VM with 48 cores and 192 GB RAM on = a >>>>>> 32c/64t AMD EPYC 7502P with 256 GB RAM. >>>>>=20 >>>>> I sincerely hope this is not relevant but my CPU is also AMD: AMD = Ryzen 5 5600G >>>>=20 >>>> The amd64 system type that I have access to and used >>>> for my testing: >>>>=20 >>>> AMD 7950X3D (16 core, 32 thread, so 32 FreeBSD-cpus) with 192 = GiBytes of RAM >>> I'm on Intel, and I don't see any crashes at all. So, are we looking = at some CPU specific issue here? >>=20 >> We can't say for sure, but we definitely have all people reporting = the issue on the same CPU brand, so it's some indication I guess. >>=20 >> I was hoping it would not come to this because I suspect such issues = are quite difficult to diagnose. >=20 > Unfortunately, for amd64 I only have access to: >=20 > ) An old ThreadRipper 1950X system (untested so far) > ) The 7950X3D system >=20 > No Intel systems. >=20 > If someone had both AMD and Intel and could have > boot&operate media that should work for both, say > USB that can be simply moved between machines, > running test on both would be appropriate. > (Implication: the media not being tailored to the > cpu specifics so the same system software is > tested in both places.) >=20 > I'll note that the media in my context is PCIe Optane, > ZFS based. I could try a U.2 Optane in a PCIe adaptor > that has UFS instead for building textproc/libsass . > (The U.2 content is an basically a rsync of the ZFS > Optane media's live directory tree, with node naming > and such adjusted afterwards.) >=20 > What do other folks have for the file system(s) > involved? I get the sassc failure from a a pure UFS live-context as well. Interestingly, there is variation in the .got.plt oddity. Earlier: Bad .got.plt: Contents of section .got.plt: 2bed60 00000000 00000000 00000000 00000000 ................ . . . 2befc0 00000000 00000000 00000000 00000000 ................ 2befd0 00000000 00000000 00000000 00000000 ................ 2befe0 00000000 00000000 00000000 00000000 ................ 2beff0 00000000 00000000 00000000 00000000 ................ 2bf000 96ab2a00 00000000 a6ab2a00 00000000 ..*.......*..... 2bf010 b6ab2a00 00000000 c6ab2a00 00000000 ..*.......*..... 2bf020 d6ab2a00 00000000 e6ab2a00 00000000 ..*.......*..... 2bf030 f6ab2a00 00000000 06ac2a00 00000000 ..*.......*..... . . . The new bad .got.plt ended up with a bigger zero area, the nonzero area again being nicely aligned for where it starts. (The .got.plt starts at the same address as above.) Contents of section .got.plt: 2bed60 00000000 00000000 00000000 00000000 ................ . . . 2befc0 00000000 00000000 00000000 00000000 ................ 2befd0 00000000 00000000 00000000 00000000 ................ 2befe0 00000000 00000000 00000000 00000000 ................ 2beff0 00000000 00000000 00000000 00000000 ................ 2bf000 00000000 00000000 00000000 00000000 ................ 2bf010 00000000 00000000 00000000 00000000 ................ 2bf020 00000000 00000000 00000000 00000000 ................ 2bf030 00000000 00000000 00000000 00000000 ................ . . . 2bffc0 00000000 00000000 00000000 00000000 ................ 2bffd0 00000000 00000000 00000000 00000000 ................ 2bffe0 00000000 00000000 00000000 00000000 ................ 2bfff0 00000000 00000000 00000000 00000000 ................ 2c0000 96cb2a00 00000000 a6cb2a00 00000000 ..*.......*..... 2c0010 b6cb2a00 00000000 c6cb2a00 00000000 ..*.......*..... 2c0020 d6cb2a00 00000000 e6cb2a00 00000000 ..*.......*..... 2c0030 f6cb2a00 00000000 06cc2a00 00000000 ..*.......*..... . . . =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Tue Nov 26 02:05:12 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xy5WX23yPz5db9y for ; Tue, 26 Nov 2024 02:05:28 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic306-21.consmr.mail.gq1.yahoo.com (sonic306-21.consmr.mail.gq1.yahoo.com [98.137.68.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xy5WW02Drz3wtK for ; Tue, 26 Nov 2024 02:05:26 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=bakeiUTm; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.84 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732586724; bh=8sJfliBR6dtrYZF1TA/cgxCMZ71Vz2+0NmqB4mslM0c=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=bakeiUTmsWOqdWYIeUW1ol0DEsrCQk3ZAej0sCnthhu1akKGTDk/D4yHRkp7a1AfvNwlCXm4BxrZ5axL7yAPHJN4F2YugIE648HLRNhG4pApEa4y32Dj44knjdrd18SqgEwnhqhQYmiRRNMPl04RxWNMDqfO5WbswH2VDlRrAt+2CQEGR3wGJIvxw9p58Q8mYdA+rmq/+qeVb/qYNG1ZNYyni/f6I7ENNe+PLnIKFkcjCaiY11ST2cY6hRPPacCz5ZgkcOjOan0g7LyaYL3XeFktVm9MU3OaX0z80KQ/n9+oADcoemQtmLh42yfCDhyuBBmH9EJO06G4RP4N79a7SQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732586724; bh=zAtLuFB3uJEy0+yLcdD2koeO0MyT67/PBVck9r0wRiw=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=pKLhqlEX9ZqTURprTzGUvszKIgNWo8voV4+q2YxBTb3Ue6X5l610J7pTfeXIdYZAZMbHCbUQndKNjI7HBXJUWVB3hLt3guwpK9p+K1fuJWAifKYb6akR959CWLtrSFQb3chJdnFXQgaVjISwHcNcjut6MrNny61ucbZHk25AE4Qmd3gB05SFf5+V9vlBo8xuK24OfySg9etpJ4JHy3NgtXcYK2H2l0uEtYd2epRO6Y1h0nU8kjB3HQ5BKe5gqeidlXD+UGKC2XMxjJTxvKQ0kaRpOisjwb1q/sic8PnEGhcox7tEyi0592GQcI+GIhqHqKkp93CghOhuEtfAShEo+Q== X-YMail-OSG: f2d36S0VM1kig0bn1C5OztVM1rG7fw.Lwktigve2lApyygQYQgzDnMugEosOCiF XZ8onTLeNz9jzQet.UsAdCJNtgpb5ECuRWTiFvb1EBt1b9lN.DhkDUwM7CgozLsziPzKhwEh.5Vu U9DN6JCUVktvDsV6KRfCAC_CC6_NzmzOUCPVYvZHifN_ct8YlPa9gN0RXinIGD4G2PBhlxWMaFjg 3UC0CBaBtiggN.AskXHOX5fLuhAsXaq8.MtINPB.NsL6BvylNluoO41L2X4dWrNDqUP7UANRjLx5 fhG7u8V9ftWZpYXJt7tGZNSC1XXO.oKkYKjhvsvJv.DRH8yRWnpOX5C64N_IwTAgTdluFBmN9eaL RqWndPjgBp8arDCPoSj5K9mdnPnGhKqbAJ0ciwpzO9lJGK6s0uYZsHjdgF3mzWcD0UHgy.pSnQlw aXDl0YATguTQYpkRr7Ot.tz5djuGwht5GzfbwHAKOTHDe0IT0x5JyQdfLIBtJuMv7KLYa3D.v9h4 qXTzaU6Epokihh2O7M7h8QbtYCUluKrMLtC1gt1TnFkRpW_f4tK5M5yM2zCMuopSZs7SpsgubX9r ysm_YZY2oNeKZXFFU7A5Gw8QcqK_TqUQVPFUArEQen1orxHlGbkUIip1uA7LB3.dNEyErBLAkUra 0YYACXb6rXK84wgpner4GpU93dWY7_.1JRIRSo4yYAxdPsihp6gq9KGkpcwTprlD4CZ5ePLt20Ua Q3JHqX0PgNh40MZ8FJR3CMUsXqzdvhtiivDEPFXIo55aq0f2fQrKSd4jByLKp682nM6_QC8RTJHx IEsXcOLVU_inuQ11zvHZwVeZRg4lYMuhG1b3Z8_smD4IghAVRo.weNJRSiJU51e..wgXZZPP.mhN KPHtxhjIxLeeS80Ti7AVyVM0KEX8Oe3__ysia61VPRTc_Dbr0gvZD.b_Cq_.RbSfgXSrXwHOW8NE 5NzVhu0Brgu.b8TJPj6O1uhBWrjkMxnM0JWAJNz7Y0Kt7Salecqjh2_..Vla_wAQlqkaSuIOMMN7 rC_oIkV_y.wU5rmxVVqPqi5TfwMifRIfF_Mh2_scyrssTyomGCaEddGdKkV4gTWpHItcBsn5lb6o JfpK5BbFZekxody6GjWJc07Swu2F2PNV7wOSb3o_Uypp4kIIQYzZ1pjalDI.r9SI0..XiCIWpMet pXpu2UypjwG3tOrixPkOmacDIYr5GFcR.G6PojdFtec_GFCWgkIO3qvSp1RMYIW9yslzc6NQJZpF i53.Y.ck.JvBH9NLSVUk2R9SByn6hgjT57adaX93Vr5K7TxZlZNSfGB3gZU21fLGjUqAvb1PzGfI ZKjRrdg2QccpJ2vVyQy7dFx07OYOWiuxhjRjQqMiThxLB9js0TKD9UD5SbGyAYqApOcoB_9SJcTW 6i5aEywNf7qEV2rDXSAnqQhYhQxwaUYGfCgYwb8xylcHQF_qvn6eZ9A9vv9.6l_8lxzfFtyQlg7e Ro9I.DYQNLXH6zRG4sD6_RfC.0VzVVc8A.MqUlZGOZqzZj7e7jbi6DdTSsXaAh7LXkKpzMn1Rv.4 7trngds0Qu4ISaLoLSVSAJs4tboxuYXmMHi4d.M0u7LpCgJJFljdaUh4_wF1tDeDbfnziNkcg45Z tPQWbIMed08oiJu5J5W.GuWj7GM_DbmVS5kEfyPond7xdlVEbCt6cU_IHyak2qdIDtParDNl0FXQ Ox0oQvfm8bgr2pSLYTfpFbORG29vuZcB3FyEyug7sglMOIRvqrPcO8_80IHMZKw1BV7unjPXzzMz rrzhIEbZzqLKAvfA4GQ6ZI_agin4sQ6AaTjNRlLq9KLERLCEd4yzw5CPtqRp1gx0GjBNTH5Cir0A s_V1tWsXHPgQvMb.DYfyVKeDUod8TLG5jj5.PXoGdC_hKnszeiErrnjbS1Kp4SQdVa6OHPXoGCK6 UQtLK3YlbNbl5dnmrvQHKlMZFgyzMjXs6NyrU0vEqeedjISkO6S075dPAI1KcjmT5nhxxk3qHrkm daDHU29OKgBeEnLQFPr7.VDFMNi3T09lktDSkpr4v5Guzlct0b.Li3f68SMNDQFLsTT4b_sHP5jV xh.7u7ewHd3otZMfwJnF2QKh2NEBhY9VJsqQ18q6f9X1PtIJn.MwVH0V2ISuPKsnmIx6VBs0EzNT tePigwEzCNmsm3L9jmDO9aqs8kUYLPgQEvOVflunwW49QYx2p_5wC2VF.SbNOcigixn1T00HiMgT 3fJWoydKnTZoQJzJRjeQ7z.nnS7O_tKybp83vDI9v5CwbGQR2iPeULpejXXN3LHx_dizoXxERZeS Z8aAQvCGW6__IpOLL X-Sonic-MF: X-Sonic-ID: b3d737c3-4961-4965-aacd-456482b901e9 Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.gq1.yahoo.com with HTTP; Tue, 26 Nov 2024 02:05:24 +0000 Received: by hermes--production-gq1-5dd4b47f46-n48bg (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 1b1fcf937bee5059b4da103627d27a30; Tue, 26 Nov 2024 02:05:22 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51.11.1\)) Subject: Re: port binary dumping core on recent head in poudriere From: Mark Millard In-Reply-To: Date: Mon, 25 Nov 2024 18:05:12 -0800 Cc: ports@freebsd.org, FreeBSD Current Content-Transfer-Encoding: quoted-printable Message-Id: References: <46E3A370-A3E0-4BAF-B707-87F94F98E248@FreeBSD.org> <5ee47c3d-f80e-4d50-9b6a-acb3c98e80e0@madpilot.net> <7c9c3cf5-bbd1-4642-8d04-33aa07a4db02@madpilot.net> <9df256a8-c6ed-46d9-b955-fc2657c12d36@madpilot.net> <5c502054-7353-4a1e-8350-c403482e9c0d@madpilot.net> <3127C3BA-FC93-4636-ADDB-89518DE9C60D@FreeBSD.org> <86ed2zsp6l.fsf@ltc.des.dev> <5f24a570-26e0-4c0a-817f-591a234fd07b@madpilot.net> <5918C6A1-8FDB-40CA-8C86-EB7B7BE75A2E@yahoo.com> <86ed2zc8r5.fsf@ltc.des.dev> <45098ccf-4dc6-426c-849a-c923805d6723@madpilot.net> <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> To: Guido Falsi , Dimitry Andric , =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , Yasuhiro Kimura X-Mailer: Apple Mail (2.3776.700.51.11.1) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.84:from]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; FROM_HAS_DN(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCPT_COUNT_FIVE(0.00)[6]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.84:from]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim] X-Rspamd-Queue-Id: 4Xy5WW02Drz3wtK X-Spamd-Bar: --- Top posting going in a different direction that established a way to control the behavior in my context . . . I changed USE_TMPFS=3Dall to USE_TMPFS=3Dno : USE_TMPFS=3Dall gets the failure vs. USE_TMPFS=3Dno works just fine So it is a FreeBSD system error associated with use of tmpfs . Now back to what I looked at before trying the above . . . On Nov 25, 2024, at 17:05, Mark Millard wrote: > On Nov 25, 2024, at 15:21, Mark Millard wrote: >=20 >> On Nov 25, 2024, at 14:23, Guido Falsi wrote: >>=20 >>> On 25/11/24 23:15, Dimitry Andric wrote: >>>> On 25 Nov 2024, at 23:12, Mark Millard wrote: >>>>>=20 >>>>> On Nov 25, 2024, at 13:27, Guido Falsi wrote: >>>>>=20 >>>>>> On 25/11/24 22:18, Dag-Erling Sm=C3=B8rgrav wrote: >>>>>>> Mark Millard writes: >>>>>>>> Guido Falsi writes: >>>>>>>>> On 25/11/24 09:17, Dag-Erling Sm=C3=B8rgrav wrote: >>>>>>>>>> Dimitry Andric writes: >>>>>>>>>>> Probably best to create a bugzilla ticket, but as I said = before, I >>>>>>>>>>> cannot reproduce this. >>>>>>>>>> I can. My builder is running 15 and sees segfaults while = building >>>>>>>>>> packages for 14 and 15 but not for 13. >>>>>>>>> BTW removing optimizations (CPUTYPE) for only the affected = ports made >>>>>>>>> guile2 work again. Did not solve the issue with sassc though. = [...] >>>>>>>>> I'm also using ccache, but that does not look relevant. >>>>>>>> I've never used ccache or analogous and get the = libsass.so.1.0.0 >>>>>>>> .got.plt corruption that I've reported on the lists anyway. >>>>>>> I don't use ccache or optimizations. Here's an example of sassc >>>>>>> segfaulting in a 14.1-RELEASE-p6 jail: >>>>>>> = https://pkg.des.dev/logs/data/14amd64-default/2024-11-24_19h29m04s/logs/er= rors/plasma5-breeze-gtk-5.27.11.log >>>>>>> which matches the following entry from `/var/log/messages`: >>>>>>> Nov 24 21:23:06 pkg kernel: pid 71277 (sassc), jid 253, uid = 65534: exited on signal 11 (core dumped) >>>>>>> The poudriere host is a bhyve VM with 48 cores and 192 GB RAM on = a >>>>>>> 32c/64t AMD EPYC 7502P with 256 GB RAM. >>>>>>=20 >>>>>> I sincerely hope this is not relevant but my CPU is also AMD: AMD = Ryzen 5 5600G >>>>>=20 >>>>> The amd64 system type that I have access to and used >>>>> for my testing: >>>>>=20 >>>>> AMD 7950X3D (16 core, 32 thread, so 32 FreeBSD-cpus) with 192 = GiBytes of RAM >>>> I'm on Intel, and I don't see any crashes at all. So, are we = looking at some CPU specific issue here? >>>=20 >>> We can't say for sure, but we definitely have all people reporting = the issue on the same CPU brand, so it's some indication I guess. >>>=20 >>> I was hoping it would not come to this because I suspect such issues = are quite difficult to diagnose. >>=20 >> Unfortunately, for amd64 I only have access to: >>=20 >> ) An old ThreadRipper 1950X system (untested so far) >> ) The 7950X3D system >>=20 >> No Intel systems. >>=20 >> If someone had both AMD and Intel and could have >> boot&operate media that should work for both, say >> USB that can be simply moved between machines, >> running test on both would be appropriate. >> (Implication: the media not being tailored to the >> cpu specifics so the same system software is >> tested in both places.) >>=20 >> I'll note that the media in my context is PCIe Optane, >> ZFS based. I could try a U.2 Optane in a PCIe adaptor >> that has UFS instead for building textproc/libsass . >> (The U.2 content is an basically a rsync of the ZFS >> Optane media's live directory tree, with node naming >> and such adjusted afterwards.) >>=20 >> What do other folks have for the file system(s) >> involved? >=20 > I get the sassc failure from a a pure UFS live-context as > well. >=20 > Interestingly, there is variation in the .got.plt oddity. >=20 > Earlier: >=20 > Bad .got.plt: >=20 > Contents of section .got.plt: > 2bed60 00000000 00000000 00000000 00000000 ................ > . . . > 2befc0 00000000 00000000 00000000 00000000 ................ > 2befd0 00000000 00000000 00000000 00000000 ................ > 2befe0 00000000 00000000 00000000 00000000 ................ > 2beff0 00000000 00000000 00000000 00000000 ................ > 2bf000 96ab2a00 00000000 a6ab2a00 00000000 ..*.......*..... > 2bf010 b6ab2a00 00000000 c6ab2a00 00000000 ..*.......*..... > 2bf020 d6ab2a00 00000000 e6ab2a00 00000000 ..*.......*..... > 2bf030 f6ab2a00 00000000 06ac2a00 00000000 ..*.......*..... > . . . Interestingly, a later retest of the ZFS context did not get the above. Instead it ended up like the below bad case. I'll also note that scrubbing reports: # zpool status pool: zoptb state: ONLINE scan: scrub repaired 0B in 00:00:47 with 0 errors on Mon Nov 25 = 17:50:44 2024 config: NAME STATE READ WRITE CKSUM zoptb ONLINE 0 0 0 gpt/OptBzfs ONLINE 0 0 0 errors: No known data errors This should mean that the unexpected zeros were present before zfs did its checksum prior to writing the data. > The new bad .got.plt ended up with a bigger zero area, > the nonzero area again being nicely aligned for where > it starts. (The .got.plt starts at the same address > as above.) >=20 > Contents of section .got.plt: > 2bed60 00000000 00000000 00000000 00000000 ................ > . . . > 2befc0 00000000 00000000 00000000 00000000 ................ > 2befd0 00000000 00000000 00000000 00000000 ................ > 2befe0 00000000 00000000 00000000 00000000 ................ > 2beff0 00000000 00000000 00000000 00000000 ................ > 2bf000 00000000 00000000 00000000 00000000 ................ > 2bf010 00000000 00000000 00000000 00000000 ................ > 2bf020 00000000 00000000 00000000 00000000 ................ > 2bf030 00000000 00000000 00000000 00000000 ................ > . . . > 2bffc0 00000000 00000000 00000000 00000000 ................ > 2bffd0 00000000 00000000 00000000 00000000 ................ > 2bffe0 00000000 00000000 00000000 00000000 ................ > 2bfff0 00000000 00000000 00000000 00000000 ................ > 2c0000 96cb2a00 00000000 a6cb2a00 00000000 ..*.......*..... > 2c0010 b6cb2a00 00000000 c6cb2a00 00000000 ..*.......*..... > 2c0020 d6cb2a00 00000000 e6cb2a00 00000000 ..*.......*..... > 2c0030 f6cb2a00 00000000 06cc2a00 00000000 ..*.......*..... > . . . >=20 Adding the comparison of the good .got.plt from the PkgBase based chroot with the official packages installed: Contents of section .got.plt: 2bed60 78ba2b00 00000000 00000000 00000000 x.+............. 2bed70 00000000 00000000 86a62a00 00000000 ..........*..... 2bed80 96a62a00 00000000 a6a62a00 00000000 ..*.......*..... 2bed90 b6a62a00 00000000 c6a62a00 00000000 ..*.......*..... . . . 2befc0 16ab2a00 00000000 26ab2a00 00000000 ..*.....&.*..... 2befd0 36ab2a00 00000000 46ab2a00 00000000 6.*.....F.*..... 2befe0 56ab2a00 00000000 66ab2a00 00000000 V.*.....f.*..... 2beff0 76ab2a00 00000000 86ab2a00 00000000 v.*.......*..... 2bf000 96ab2a00 00000000 a6ab2a00 00000000 ..*.......*..... 2bf010 b6ab2a00 00000000 c6ab2a00 00000000 ..*.......*..... 2bf020 d6ab2a00 00000000 e6ab2a00 00000000 ..*.......*..... 2bf030 f6ab2a00 00000000 06ac2a00 00000000 ..*.......*..... . . . 2bffc0 16cb2a00 00000000 26cb2a00 00000000 ..*.....&.*..... 2bffd0 36cb2a00 00000000 46cb2a00 00000000 6.*.....F.*..... 2bffe0 56cb2a00 00000000 66cb2a00 00000000 V.*.....f.*..... 2bfff0 76cb2a00 00000000 86cb2a00 00000000 v.*.......*..... 2c0000 96cb2a00 00000000 a6cb2a00 00000000 ..*.......*..... 2c0010 b6cb2a00 00000000 c6cb2a00 00000000 ..*.......*..... 2c0020 d6cb2a00 00000000 e6cb2a00 00000000 ..*.......*..... 2c0030 f6cb2a00 00000000 06cc2a00 00000000 ..*.......*..... . . . The contents of the non-zero parts of any pair of the examples agree. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Tue Nov 26 06:10:00 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XyBy11bn3z5dwmJ for ; Tue, 26 Nov 2024 06:10:17 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-8.consmr.mail.gq1.yahoo.com (sonic316-8.consmr.mail.gq1.yahoo.com [98.137.69.32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XyBxz4j8Xz4Nmb for ; Tue, 26 Nov 2024 06:10:15 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=C3rdzGV0; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.32 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732601412; bh=iNg7Vxtzvk5cxXD8x6BPbMHv6+JWtPW/rCmd4mHEKXM=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=C3rdzGV0j6XM7WGjRAwp02YQ/855PLVFD0GjuTCfhKCucT7lL/rAauGtaiwYKkcszhUVTI37kkS1g8f6cZCsfDSPlUyduUDwwom0OIzGGzZVj2v0+Kq6c03gGqjxxjtbzEXDP86BBteCMAe19MwUHh0e5rvOmpjym2J3HB4OH/yQzGdUmHZV/7+C6eIz3RWVmgMBoplANMSxEq/uNoy3iMKP+cBKFimT8tivkg4BUUQC72nn0nIGmfUq4EhZzotoe07xV+I75Uu7xVwEV4nWF4Hx31Fy4dti1LCX8mAzyTio3MPmVlphYSrzWukq2js9AlHvEjGg0pWO1AeXaYuz0g== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732601412; bh=OgA97Cg1BFFCOLNhEa6EykPAD6NmuaJcGas/zL6/b6A=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=GzMdVpkyre7yT+l/OUolj2tO0AmFDNLVBgRkouZTL1o3qzyBiyOgc4co5GNG1405EjDBXbFmMTuO+t1w0mNx98NU68MjGE4MgpMvr/HPCq//SznPE46AiHaRT0NciINPr2atMZNqtcFJttg7ybPn3+dQX/fS02X1t978blxFUNzyFkcm4u6nmyDU7IhaJBPXcGEbBqGUxuNaiBxbuvK6wSUeWcFScK2yS2EHkSxZe3xxDwJvGaOu0vasICGKYIvPwpGP24atzOz1EAbhDLJwuakePX9W5GB/6rlIRHk1vfGDeN2HE5wM/pvciyly2Ref6fMIvYxK966+5JNaNQeHxA== X-YMail-OSG: qnOFeucVM1mNyNeosL_9CVCkgHrk.AO4cNpaUR5E4GPLKO2sNc8L3UhVwrutHjp HxUlvCDOrtJhnCa3zpodZz4QcetGhFeUMslfAH9RxAQnaB0uXtsfgtntpWjvrXPf7cXXAoaqCezT VjSNljSDGylk98jV4sdsfK1huLv5ZBHwG0LqBy1Xzm2zA9qVmrDgJZlvxH6DERI88EGAfK4QuvMq _qumxoeLIGlQULi4U0F0mfoPb2VFP2zlBPu2RWNoWAoHwfaryVZHsFukEoyVQYgKNcT3VWuRxL2l zVl_GTJpFYzIfj1Iwj_8XiRXjFTs72GvRWLyH1Av40kvV08ZYI_PwiuU2TypO4VihhFyPYVOrKym L9TAa9bpfp_wwJyb7y5V9TDysKxlMcjwfXQNlAThPLHFMDULbjK17F76nrXlMv1RvGD3Qgq8w4AJ CvoHkYtCD5whLZSJ_uTOZCpRiT_tf0dsWjr75itTEdb2CIw2_mMgASbUNspGk9KNQSPRpU3OlPzT Ut.FUAhYt7e3y29OxTE_03TGeIExIitObI_93zKwd7kOL.joRMWZhXYauLw9f84y8vkGsu.7I_yk 8oiPQ877kDxzT0zUhupwlRcDnlRNIrlCoyBvctgI7myVlwC2QcDNSKYCxB03fq4q07LTwGvm0bMb dqyASph_oi9WPpH4IbXopuBQm2ODEidJ32SaLh62T2uNlrUL56fBX.vpF0R.tJ23htkHwynQkvVz dLlyDOjOhZyEgOGxEVVPTMfPmoZhdMj3AZJR4h3CYtDIqRVgSSvA5muB6NB0E3lx6ZO.zPTrzKM_ iVpmtO5rt_jGQXos89C75mq7yasiq5SFMQZHUZ9.lA5AFpM1rcWyjynxWhNkuc0AdlgTcETHQ7A6 ue9qmgvfYOfm2dJWEi32GmWvwsl311fj3NI5COBoctO4xQkgptLRtkwkZ4ZqS_D8N.Hjwz7gD8WS 15haeLXrKes4rE7mHjIjgezh7YXgKv.2pwInEdgyOd2T1hGWImxIJoQfL70fDNwuKFEgySqVrKdD ez1AGyQVU7od1zE2Sp6FccLtdO9u9ba0bNrCywz_OlJQ2ZT28dcnv.cQKbGRbII.wmTi85a9PZdw yb4XDzh.t7zct4eaP5gHnE2_viOxYw1jWQlEfwUlJr.5oFdW9v0YyBe8aYHLPCpm57lY5HbDoHxM _jJjRTC2DTbcy1SkA501XPZIUjLH1ZEy2XE0h9.Hfs7bvoKS25Niv2twnSO1.jyfpsA2u05kNwdd cU.NMav1LKQnqFtQV7rCp4T5gPAji9SDkfhN9NJxGaQIDqPKY2K98y4wXzKFK0Jr.6ZXOFPMs1yh dD7lK8DIHYJXjo0a_iu2rFtbQZMp4W1v1Vo0PaNFo_VCtVMax8qzVIj_1XBaoa1QBzutX0gayngE BhQKyhbOUlL4M76HtIwyWXYuwt_CeJgiPGqMRW0kaIloDGV60GWFmzVXTN9JNQKBEXTdTm2UNV1V Nv0bx4D8_ojfOYZhdNZs3WgOrR4ykPH07J_LlrNWRIHu0dVaS9BaSkqcdGSbXXuC.wrQF0708y5W OQORbGGckFtAKsMP1C8_vE6Bg_ptgS8e_ZAdZZuA_lip28CUAH3icsR.Sxh6bu2xOZ2HehHjjrSv msiuZP8ruugfespX38NutbGcuFXdE3zZXmsm3ZJtvvgZvYuq9UQYUsYB7Jf0TxsvhuTjL3rWCkLw jk2QHqQR4QeMAAZfSJvKdpwIuhrGfOsBsNqhUnAN9DYy3qOfZ.Ex39VfzwrYWQkC2cZ0Td3CYud0 UYqXUll2xufHN3FPi2B_7hJ7SZEhVNr_DWJe0cA7qdytnbzgfvytOQjlRlACY3rPum5tx6BouDyO EVQ.IhU7b7TqeX3THcxkQWiylW_AWr.mkjm2_DiW4g4KC1DIBcslweHBnlNsOhUH5b.Ka2GpHhRh giEMSPx_nO_h8axOgpx3tXagkqCkZ1bJgGJwEuHB0uZXbL32aUFR3_KtCT_XhDyj_e2oLDbnWwez TfVhV.CRTZKsQUGV51fFtmD.oCTRtga60Rpq5vcDfNkQPgfk17O8ByTxY7nc.dZyV_nudCPFQus_ QugAkTI7KIzGOsLcaoSde7eNjmA.hFuZx2margMj.11fIEdUYQVzzeYSmiLwCgcNa4VifG7FUOeG VYWhaynilEhLtcKHh5nJwO3GfO1VGjQuvD0fdL7zVoY6S2xn23CO39wtkhlfIh1vEpRbYptp6nRB hdRFIkhrc20Rgxo9CoTO38tiGs8EnCnwSvieTw377ApWn6vLmhyBVRNN3N3r2TnZjyQFF5yE9Lnc 0NuWvqto1vqFjT8IsJOeNyDQ- X-Sonic-MF: X-Sonic-ID: 71d23aa7-2c2f-425b-8de1-ba97eb6e18fb Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Tue, 26 Nov 2024 06:10:12 +0000 Received: by hermes--production-gq1-5dd4b47f46-k4d2j (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID c97bb757c80f8d8e548e0e0aeb72e0b0; Tue, 26 Nov 2024 06:10:11 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51.11.1\)) Subject: Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros] From: Mark Millard In-Reply-To: Date: Mon, 25 Nov 2024 22:10:00 -0800 Cc: Dimitry Andric , Guido Falsi , =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , Yasuhiro Kimura , ports@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <0690CFB1-6A6D-4B63-916C-BAB7F6256000@yahoo.com> References: <46E3A370-A3E0-4BAF-B707-87F94F98E248@FreeBSD.org> <5ee47c3d-f80e-4d50-9b6a-acb3c98e80e0@madpilot.net> <7c9c3cf5-bbd1-4642-8d04-33aa07a4db02@madpilot.net> <9df256a8-c6ed-46d9-b955-fc2657c12d36@madpilot.net> <5c502054-7353-4a1e-8350-c403482e9c0d@madpilot.net> <3127C3BA-FC93-4636-ADDB-89518DE9C60D@FreeBSD.org> <86ed2zsp6l.fsf@ltc.des.dev> <5f24a570-26e0-4c0a-817f-591a234fd07b@madpilot.net> <5918C6A1-8FDB-40CA-8C86-EB7B7BE75A2E@yahoo.com> <86ed2zc8r5.fsf@ltc.des.dev> <45098ccf-4dc6-426c-849a-c923805d6723@madpilot.net> <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> To: "jah@freebsd.org" , dougm@freebsd.org, asomers@freebsd.org, Mark Johnston , FreeBSD Current X-Mailer: Apple Mail (2.3776.700.51.11.1) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; TO_DN_EQ_ADDR_SOME(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.32:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCPT_COUNT_SEVEN(0.00)[10]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.32:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim] X-Rspamd-Queue-Id: 4XyBxz4j8Xz4Nmb X-Spamd-Bar: --- On Nov 25, 2024, at 18:05, Mark Millard wrote: > Top posting going in a different direction that > established a way to control the behavior in my > context . . . For folks new to the discoveries: the context here is poudriere bulk builds, for USE_TMPFS=3Dall vs. USE_TMPFS=3Dno . My test context is amd64 on a 7950X3D system with 192 GiBytes of RAM. Others have other contexts, including an Intel system. > I changed USE_TMPFS=3Dall to USE_TMPFS=3Dno : >=20 > USE_TMPFS=3Dall gets the failure Note: The test case is corruptions of the likes of parts of the .got.plt in libsass.so.1.0.0 from text/proc/libsass . The corruptions are well 4 KiByte aligned blocks of zeros showing up in the files that should not be that way. 2 examples of bad libsass.so.1.0.0 builds have: Contents of section .got.plt: 2bed60 00000000 00000000 00000000 00000000 ................ . . . 2befc0 00000000 00000000 00000000 00000000 ................ 2befd0 00000000 00000000 00000000 00000000 ................ 2befe0 00000000 00000000 00000000 00000000 ................ 2beff0 00000000 00000000 00000000 00000000 ................ 2bf000 96ab2a00 00000000 a6ab2a00 00000000 ..*.......*..... 2bf010 b6ab2a00 00000000 c6ab2a00 00000000 ..*.......*..... 2bf020 d6ab2a00 00000000 e6ab2a00 00000000 ..*.......*..... 2bf030 f6ab2a00 00000000 06ac2a00 00000000 ..*.......*..... . . . Contents of section .got.plt: 2bed60 00000000 00000000 00000000 00000000 ................ . . . 2befc0 00000000 00000000 00000000 00000000 ................ 2befd0 00000000 00000000 00000000 00000000 ................ 2befe0 00000000 00000000 00000000 00000000 ................ 2beff0 00000000 00000000 00000000 00000000 ................ 2bf000 00000000 00000000 00000000 00000000 ................ 2bf010 00000000 00000000 00000000 00000000 ................ 2bf020 00000000 00000000 00000000 00000000 ................ 2bf030 00000000 00000000 00000000 00000000 ................ . . . 2bffc0 00000000 00000000 00000000 00000000 ................ 2bffd0 00000000 00000000 00000000 00000000 ................ 2bffe0 00000000 00000000 00000000 00000000 ................ 2bfff0 00000000 00000000 00000000 00000000 ................ 2c0000 96cb2a00 00000000 a6cb2a00 00000000 ..*.......*..... 2c0010 b6cb2a00 00000000 c6cb2a00 00000000 ..*.......*..... 2c0020 d6cb2a00 00000000 e6cb2a00 00000000 ..*.......*..... 2c0030 f6cb2a00 00000000 06cc2a00 00000000 ..*.......*..... . . . So: Where the zeros end varies but the start of good data end up's at some 0x...000 offset: a multiple of 4 KiBytes. > vs. > USE_TMPFS=3Dno works just fine >=20 > So it is a FreeBSD system error associated with > use of tmpfs . Recent work on tmpfs includes: Mon, 09 Sep 2024 =E2=80=A2 git: 8fa5e0f21fd1 - main - tmpfs: Account for whiteouts during = rename/rmdir Jason A. Harmening Fri, 04 Oct 2024 =E2=80=A2 git: 75734c4360fc - main - tmpfs: check residence in = data_locked Doug Moore Sun, 13 Oct 2024 =E2=80=A2 git: ec22e705c266 - main - tmpfs: remove duplicate flags check = in tmpfs_rmdir Alan Somers Thu, 24 Oct 2024 =E2=80=A2 git: db08b0b04dec - main - tmpfs_vnops: move swap work to = swap_pager Doug Moore swap_pager (given the reference to it above): Tue, 08 Oct 2024 =E2=80=A2 git: d0b225d16418 - main - swap_pager: use iterators in = swp_pager_meta_build Doug Moore Fri, 11 Oct 2024 =E2=80=A2 git: 1107834090be - main - swap_pager: swapoff detecting = object death Doug Moore Thu, 24 Oct 2024 =E2=80=A2 git: 34951b0b9e78 - main - swap_pager: move = scan_all_shadowed, use iterators Doug Moore =E2=80=A2 git: 02e85d1c8a41 - main - swap_pager: fix assert in = seek_data Doug Moore=20 =E2=80=A2 git: faa9356f97d2 - main - swap_pager: fix seek_hole = assert Doug Moore Sat, 26 Oct 2024 =E2=80=A2 git: 39f6d1e7f835 - main - swap_pager: iter in haspage, = lookup, getpages Doug Moore Wed, 13 Nov 2024 =E2=80=A2 git: d11d407aee48 - main - swap_pager: Ensure that swapoff = puts swapped-in pages in page queues Mark Johnston I do not know at this time when the corruptions started. The above is only suggestive. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Tue Nov 26 06:59:12 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XyD2Y1C9lz5f0TZ; Tue, 26 Nov 2024 06:59:17 +0000 (UTC) (envelope-from mad@madpilot.net) Received: from vogon.madpilot.net (vogon.madpilot.net [IPv6:2a01:4f8:1c1c:11e5::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XyD2X5vm0z4V7F; Tue, 26 Nov 2024 06:59:16 +0000 (UTC) (envelope-from mad@madpilot.net) Authentication-Results: mx1.freebsd.org; none Received: from mail (mail [IPv6:fd5c:5351:d272::3]) by vogon.madpilot.net (Postfix) with ESMTP id 4XyD2W03N9zL7xP; Tue, 26 Nov 2024 07:59:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=madpilot.net; h= content-transfer-encoding:content-type:content-type:in-reply-to :from:from:content-language:references:subject:subject:date:date :message-id:received; s=cyq4qetkgngm; t=1732604353; x= 1734418754; bh=DNDpC0yQ1BdAj5gpzvTFzEkVnQ4tSfWIpewNN2prpA4=; b=A HYeTTJzR4lapX38WE8JSldFXTnnJm22KFrjup6NZoqFK/B5bHWkOwSIIgk9qBHRt YkAlcZGrReaOc5e4lPTzqZ8Xlao57vNIActP/ZsKl/jvyCltK+8o8Fo8PKOTXC2r ONT4Zq56shnFEzan3kTIBy5JJMA/f2byXc5vjGzvt1WkaRL8CNpJPBFM4wBNhZJl o30muQQrPCp8/Q0kAyeUWZ8XU0XTQR0LTzN4xbptYdoJdXnbL1WdquuYzlO4T+1a Asvf+1bFNDBXKdV9dVkFEQlPEHPPP+1CbkAyZ2kdyIy2n8UFHK1XKuadHs54bFZa +oeYAuXIHsG7OYJQofLJw== Received: from vogon.madpilot.net ([IPv6:fd5c:5351:d272::3]) by mail (vogon.madpilot.net [IPv6:fd5c:5351:d272::3]) (amavisd-new, port 10026) with ESMTP id RJFMVG-c3Oeq; Tue, 26 Nov 2024 07:59:13 +0100 (CET) Message-ID: Date: Tue, 26 Nov 2024 07:59:12 +0100 Subject: Re: port binary dumping core on recent head in poudriere To: Mark Millard , Dimitry Andric , =?UTF-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , Yasuhiro Kimura Cc: ports@freebsd.org, FreeBSD Current References: <5ee47c3d-f80e-4d50-9b6a-acb3c98e80e0@madpilot.net> <7c9c3cf5-bbd1-4642-8d04-33aa07a4db02@madpilot.net> <9df256a8-c6ed-46d9-b955-fc2657c12d36@madpilot.net> <5c502054-7353-4a1e-8350-c403482e9c0d@madpilot.net> <3127C3BA-FC93-4636-ADDB-89518DE9C60D@FreeBSD.org> <86ed2zsp6l.fsf@ltc.des.dev> <5f24a570-26e0-4c0a-817f-591a234fd07b@madpilot.net> <5918C6A1-8FDB-40CA-8C86-EB7B7BE75A2E@yahoo.com> <86ed2zc8r5.fsf@ltc.des.dev> <45098ccf-4dc6-426c-849a-c923805d6723@madpilot.net> <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> Content-Language: en-US, it From: Guido Falsi Autocrypt: addr=mad@madpilot.net; keydata= xsBNBE+G+l0BCADi/WBQ0aRJfnE7LBPsM0G3m/m3Yx7OPu4iYFvS84xawmRHtCNjWIntsxuX fptkmEo3Rsw816WUrek8dxoUAYdHd+EcpBcnnDzfDH5LW/TZ4gbrFezrHPdRp7wdxi23GN80 qPwHEwXuF0X4Wy5V0OO8B6VT/nA0ADYnBDhXS52HGIJ/GCUjgqJn+phDTdCFLvrSFdmgx4Wl c0W5Z1p5cmDF9l8L/hc959AeyNf7I9dXnjekGM9gVv7UDUYzCifR3U8T0fnfdMmS8NeI9NC+ wuREpRO4lKOkTnj9TtQJRiptlhcHQiAlG1cFqs7EQo57Tqq6cxD1FycZJLuC32bGbgalABEB AAHNHkd1aWRvIEZhbHNpIDxtYWRAbWFkcGlsb3QubmV0PsLAeQQTAQgAIwIbAwIeAQIXgAUL CQgHAwUVCgkICwQWAgMBBQJS79AgAhkBAAoJEBrmhg5Wy9KTc0kH/RO64ORBlTbTHaUaOj8F Je5O5NU2Pt9Cyt5ZWBRvxntr1zPTJGKRPS9ihlIfqT4ZvEngQGp57EUyFbCpI0UWasTerImM tt5WACnGmCzUTB39UXx8Oy4b1EgWeTJQ747e/F1mQLXTNa6ijRBE9fYlTb4gAkPN88/wVV9v 3PZozKLTg16ghBzHM/P7Lk8L7clPEZChX1FTa/6eSt3nvzfCuTMZbBPJF/ph+q1KyPqRgVfh tyhu5dvgMoPz/ni41IfeSrkJTD5RXzdyGR9q4Z1NYeBsLkRjC4LxKAP5KqUsvlOUjKvO1byj ApYdMarol+IGkaSk9e3zVYAJkWKjn/ni8XbOwU0EUxB7QQEQAKFhrDceoPdK/IHDSmoj6SQY isvM7VdhcleS7E9DoEAVt7yMbf6HbbMVTTY6ckvwTWQssywLBXNVqxgc4WLJjzfUhgef+WE7 5M3+WFYlOVQLGZY/zEVgma1raYnOHNAOzeHLDmEXjbZP6vGAeDyBbGfQPpE7qGYZ7ubeT3Xw QO+PklcCrvOPj2ZPcAxGNS2xVU/LzONqCrJqLMJSIcCdsbiSP4G5PnDFHtMokaTY6OEr8OEQ fOAerhcHUa/z7Uu8YtmaqKH+QGkE/WEgaRqSiTnv0JOTD+DxehaqvoKPPZ++2NpCZMHB2i6A /xifmQwEiIjEXtcueBRzkNUQkxhqZyS13SrhocL9ydtaVPBzZatAEjUDDEJmAMLVFs45qfyh MiNapHJo2n3MW/E5omqCvEkDdWX/en3P7CK2TemeaDghMsgkNKax/z0wNo5UZCkOPOz0xpNi UilOVbkuezZZNg65741qee2lfXhQIaZ66yT7hphc/N/z3PIAtLeze4u1VR2EXAuZ2sWAdlKC NTlJMsaU/x70BV11Wd/ypnVzM68dfdQIIAj1iMFAD/lXGlEUmKXg5Ov2VQDlTntQoanCYrAg +8CttPzjrydgLZFq3hrtQmfc0se5yv1WHS69+BsUOG09RvvawUDZxUjW19kyeN9THaNRgow3 kSuArUp6zSmJABEBAAHCwF8EGAEIAAkFAlMQe0ECGwwACgkQGuaGDlbL0pMN5wgA4bCkX/qw EVC06ToeR6C2putmSWQMgpDaqrv65Hubo+QGmg2P4ewTYQQ4g6oYWS03qHxqVVWhKz7FjfrV +dH8qbCLfSgIcvdBha7ayGZVrsiuMLKGbw36fcmkZPpSDOfHcP0XH8Z+u9CWj0xUkTxAlZ/7 i6gYSUpG2JWNtdmE/X8VVEyXusCLwy0K0BI60A/4dRTIX3C4QKrJ3ZbUXegz70ynjHf+lQMZ 9IZKASoRMuS5FozPQh6abvmwZEPdf5I9riUElzvHrqJ8Bx0t3Pujdoth+yNHpnBxrtO8LkQd rQ58P0SwcaIX33T2U9pG8bhu5YVR88FQ8OQ0cEsPBpDncg== In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE] X-Rspamd-Queue-Id: 4XyD2X5vm0z4V7F X-Spamd-Bar: ---- List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org On 26/11/24 03:05, Mark Millard wrote: > Top posting going in a different direction that > established a way to control the behavior in my > context . . . > > I changed USE_TMPFS=all to USE_TMPFS=no : > > USE_TMPFS=all gets the failure > vs. > USE_TMPFS=no works just fine > > So it is a FreeBSD system error associated with > use of tmpfs . > Interesting discovery, I'm also using USE_TMPFS=all I'll try to do a test without that but I won't be able before tomorrow I think. -- Guido Falsi From nobody Tue Nov 26 08:21:40 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XyFsy2c3fz5f665 for ; Tue, 26 Nov 2024 08:21:58 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-8.consmr.mail.gq1.yahoo.com (sonic316-8.consmr.mail.gq1.yahoo.com [98.137.69.32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XyFsx35YXz4cW5 for ; Tue, 26 Nov 2024 08:21:57 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=qpOEkUSv; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.32 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732609314; bh=QIEZ2DNXMUtg+aiD7YkTdtaJBvLckFlUsUjdEF1Cg0A=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=qpOEkUSv6GupDlTbmV+Mao/Aj3au98HwkhML9EFtjm2wqde368JX6Rz6yEz14MBm+1l3aJDtsMz4aba73lgjfn/yMvzAsgki/9fNFii5o5AuQOQbL21XoTHwq8fc8ZeeajE1I1bI3VhMjytDhP66HVwgARvOcmGP4x1vS26F+EVaP/GVIrAkdSlfQg9/Z3jxhgaK/qn+IOsPW4uipkI80iRIC8VO8762I4S/7KryDySpQB29qNj8rzGAmOx/UascQDJsk+Uln18rAZX+KNd9j9halYfC6JE/B9dw2oE3Jz9WGc34QKt/obz5TQ8HD1SHZFGX5+hn+zkYia7lPktFoQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732609314; bh=v7wQoLy1sCxlomNBFGk1T9NSQm3r+d0L9LVxDYNgrEV=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=APF+WHmwwZesWOBC024O5FKnEQtNYwZVhlZUzAaN2dISonUwDun1E5v48x+Fw9UW+LBWUeXjd8LmQRfQsHhd6wM1GRynatLGGjczzmDCpd3GfmoRu5032B/Cv0bqwCXz8zBs9agDKmO1JBi7T9uz9LIEh5xGgO0jKeaMa2/5JSANXDWTCOxDDnkTPFccHilOgvN9EiTDTQeB0Cmw8o4bYeTvUl6ynOs5PLcw9x9NbjuxXXwqHOZvvwem7Ta2EI94a+xaU6xHN7Ac87TOml7PL8apsSHC7CBaepE9lqXg3XsUUnL6Spl+FNPP2J1gy7MMLU9OJWDdGx0uq+r/KRWtSA== X-YMail-OSG: _aotRj0VM1kexqeJGyK7lE0I2W0mpVvUbpWwdhYSeSnT1I1aYol_vf72mbj8Wgb 4j9ewMWqhh.jh2Ubevd0Oq.iWoOjsfgLxrZ0nh.1xNauGua2JCxu0H5205ml_zfZUBQpGYgMSe6C aGJ2nKQCfh1P64lLenpEj88.wU8HrRo4yhV3ggVEY72VkfSFAI.IZXDYdYA.2.TllDg.DqeVpZgO StXBL54YqjsesyaNryqAyN6N3C_fioCXHXPyevQSecLOloesOI6UyktcG7iDznd2zHRuKmUJ62MH lg4aXdYiqUjZeyux2VVpylLH2vzZXMoKWjUft3RXft8Fj3vJuiJsuWbqmDY1fWXBMu8u8q_HVSWf 8A4U1dfkpUWeIMOw6JoW4ewxGbOnAo2KuX0q9n7_VKxYJwaiI6MjbaV6DxML.2cI7lpAaUkG251e aYJmnDR07wYwevYsvGwxqy1YqtW9hpq1AUktyBVSE0EguSOeqBpW.ZNDK_oBoqZRZap7wI_o8FCA MHBFGtaLoiDvPkNbNvHLpI.pDLPdYfLxAmj28iR_.XZ9jZ2mMDMFJog.Vb_15WNb.voFiWRPvx5w Wc1J30MkRtILW7RA0ssu3lP3h0cFRe5wzP3K_CE87qPkdh2VKdRI9Ecn8daO_MBfSa2y.1Io9pDn bPt_KE5R.8K6EteKacDEgu3_p8jV79XkODMGGjoA0lcBQhnwpoG6oSqasXBnuBesAH._AV2n1QNV gSmEgROtrlaWHtuYvmP9_sFZVLdDjCtvdLc6GTbbh1mfbXwEJGPdVxKAqH5QDsEnJT93t9QeGurI 3yh62wDVRj1YEm222DT8kJa4Jvb9lz1QJgmdyDjJEPrGqzgShSj.ZOavsCQGUBh7h8Yes25lBUk7 1zdR.KIntXfUOfajYgIZQ6J87QLHq19jj3Um4rwrwOcNBiLmOc_rH2KCmX7K2_MKNxSCsZTDrnFG UXXWwC.EllgFX8VxiwY_U81ZegXicbgJ1KzGQoM9kJSHRWfryXIO6uxnIxLnIdh1uELQrPTfDr5X yOIX9Ng4miTc4KA7r.zrGsshHkr7DSk7W_jsOIaqQL7xNrrZlaoyfltB1wpW5Jsl.wcFDgV2pX6x AOCp0.2ezhyTsgg_2SEq.wIfwca60O6ZrryxlhN4MgPgQ_4neEv0s86SW.S.amK8UH.SNngqQaA. ELQaFKOBPKiPASaVmlfuu6BSdtipnCSkr7IlwMSqao1HJjhrOg8psLdOTX2APDIV2UKkhCvXcuPv ZU6xEaOCe_UJ0GKNa2495A_ZTrL2FZSJ9cUW9horE2CMb4qWBDYq43TizmcEmKX4_OdAGDDGo2.L sDHsTdINgekMErkBusuSeBsySYKf2rhq0QPN5aPJPiiA5NvkC_Id.dwfEKQ0XDw41GDTf8.7WKWa RGThiT9wRUfksn3Jt.G6opTlyZj7IpATjJDsZ1yDu79KObW_voCx15ay6I4T7jptAuKcmx5Hxlpg 6my0YhlFwjK4jHr_AFgHQHwrGQC4.yje0GF1cEm6x6A71W_mvqJV5LjWBworLHCgi4n9BGB7XDEj anXqbSv_Gcr1miID2ydhKLoFYoKsKgVKfUODt53PYsnZcW05OgxrDY3xZEsZ9fMReTLVNeCg.jGn pWvvgdp78MB0o0wNl4A1pg76bLR4G9Pt9cMYagsHEPjedS2OFkN5.R0ku2Oz8B3c6T7dIGMYRcKy tgeABGwjILdRKbssxzPgJfvua10IhwFtcWPKpPD78y3K00wEtkeAGzp9ph_KdcPB_I8fG9UOURBJ ssRlrCF7SZZVB9qd5VdG5bgX1m1ubVyf5FVgYI2NwX8WucVudf2o0ulmsPbwNngqT7RhWOMiyu.a q5Wg7tL_Vs7Rtv44bjNiTA50UEsuPFvBP6NArn.BaWkEnut1SCqSwfUyANtL1YMgVsCiG1kDSjZ3 atlAbhzTFpSAvLtqm2D7v1CO4aluccz1mZwQNLNoTDJ_uVJ7Nzp4KehdshEWIcSqYaUcdsgshfuL PnHXD5Nxdyh2oNftL8BqaOWaTmC9bH2m5YP8oelZ8mz_E6W2SzTm3SSRZ8wOUg3SvBTSU7e63Cul zHTHsS0BZg8H7KmyEPKxDREVzNmaa.Bnoo3gDI5Rj.eMbA1ND4C1nZSVqBHiXV3KB9TZMVHEJIP2 12rMe2MR58mUXyPP4_TIJnxHQZ6xmidFf5bT1UOhAxD6oNQbsCs5ldp_bdSOYXPh1EAknR0QTem1 iuMw31Zf3eagqY41GBa7UJ0us.VE9dBdXi4EbSjUtA.Vb33p9CXH37lEu.u8MU20Eli0qm6qlO0o KAsK7IN0VkD15XQYD_CbDsNe.0A-- X-Sonic-MF: X-Sonic-ID: ea04d37d-2b05-471e-8ebe-3e7371f0b010 Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Tue, 26 Nov 2024 08:21:54 +0000 Received: by hermes--production-gq1-5dd4b47f46-5qmz7 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 9288cee7416325367ace5b6e545e01a6; Tue, 26 Nov 2024 08:21:51 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51.11.1\)) Subject: Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros] From: Mark Millard In-Reply-To: <0690CFB1-6A6D-4B63-916C-BAB7F6256000@yahoo.com> Date: Tue, 26 Nov 2024 00:21:40 -0800 Cc: Dimitry Andric , Guido Falsi , =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , Yasuhiro Kimura , ports@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <3660625A-0EE8-40DA-A248-EC18C734718C@yahoo.com> References: <46E3A370-A3E0-4BAF-B707-87F94F98E248@FreeBSD.org> <5ee47c3d-f80e-4d50-9b6a-acb3c98e80e0@madpilot.net> <7c9c3cf5-bbd1-4642-8d04-33aa07a4db02@madpilot.net> <9df256a8-c6ed-46d9-b955-fc2657c12d36@madpilot.net> <5c502054-7353-4a1e-8350-c403482e9c0d@madpilot.net> <3127C3BA-FC93-4636-ADDB-89518DE9C60D@FreeBSD.org> <86ed2zsp6l.fsf@ltc.des.dev> <5f24a570-26e0-4c0a-817f-591a234fd07b@madpilot.net> <5918C6A1-8FDB-40CA-8C86-EB7B7BE75A2E@yahoo.com> <86ed2zc8r5.fsf@ltc.des.dev> <45098ccf-4dc6-426c-849a-c923805d6723@madpilot.net> <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> <0690CFB1-6A6D-4B63-916C-BAB7F6256000@yahoo.com> To: "jah@freebsd.org" , dougm@freebsd.org, asomers@freebsd.org, Mark Johnston , FreeBSD Current X-Mailer: Apple Mail (2.3776.700.51.11.1) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.995]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCPT_COUNT_SEVEN(0.00)[10]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.32:from]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.32:from] X-Rspamd-Queue-Id: 4XyFsx35YXz4cW5 X-Spamd-Bar: --- On Nov 25, 2024, at 22:10, Mark Millard wrote: > On Nov 25, 2024, at 18:05, Mark Millard wrote: >=20 >> Top posting going in a different direction that >> established a way to control the behavior in my >> context . . . >=20 > For folks new to the discoveries: the context here > is poudriere bulk builds, for USE_TMPFS=3Dall vs. > USE_TMPFS=3Dno . My test context is amd64 on a > 7950X3D system with 192 GiBytes of RAM. Others have > other contexts, including an Intel system. >=20 >> I changed USE_TMPFS=3Dall to USE_TMPFS=3Dno : >>=20 >> USE_TMPFS=3Dall gets the failure >=20 > Note: The test case is corruptions of the likes of parts of > the .got.plt in libsass.so.1.0.0 from text/proc/libsass . > The corruptions are well 4 KiByte aligned blocks of zeros > showing up in the files that should not be that way. >=20 > 2 examples of bad libsass.so.1.0.0 builds have: >=20 > Contents of section .got.plt: > 2bed60 00000000 00000000 00000000 00000000 ................ > . . . > 2befc0 00000000 00000000 00000000 00000000 ................ > 2befd0 00000000 00000000 00000000 00000000 ................ > 2befe0 00000000 00000000 00000000 00000000 ................ > 2beff0 00000000 00000000 00000000 00000000 ................ > 2bf000 96ab2a00 00000000 a6ab2a00 00000000 ..*.......*..... > 2bf010 b6ab2a00 00000000 c6ab2a00 00000000 ..*.......*..... > 2bf020 d6ab2a00 00000000 e6ab2a00 00000000 ..*.......*..... > 2bf030 f6ab2a00 00000000 06ac2a00 00000000 ..*.......*..... > . . . >=20 > Contents of section .got.plt: > 2bed60 00000000 00000000 00000000 00000000 ................ > . . . > 2befc0 00000000 00000000 00000000 00000000 ................ > 2befd0 00000000 00000000 00000000 00000000 ................ > 2befe0 00000000 00000000 00000000 00000000 ................ > 2beff0 00000000 00000000 00000000 00000000 ................ > 2bf000 00000000 00000000 00000000 00000000 ................ > 2bf010 00000000 00000000 00000000 00000000 ................ > 2bf020 00000000 00000000 00000000 00000000 ................ > 2bf030 00000000 00000000 00000000 00000000 ................ > . . . > 2bffc0 00000000 00000000 00000000 00000000 ................ > 2bffd0 00000000 00000000 00000000 00000000 ................ > 2bffe0 00000000 00000000 00000000 00000000 ................ > 2bfff0 00000000 00000000 00000000 00000000 ................ > 2c0000 96cb2a00 00000000 a6cb2a00 00000000 ..*.......*..... > 2c0010 b6cb2a00 00000000 c6cb2a00 00000000 ..*.......*..... > 2c0020 d6cb2a00 00000000 e6cb2a00 00000000 ..*.......*..... > 2c0030 f6cb2a00 00000000 06cc2a00 00000000 ..*.......*..... > . . . >=20 > So: Where the zeros end varies but the start of > good data end up's at some 0x...000 offset: a > multiple of 4 KiBytes. >=20 >> vs. >> USE_TMPFS=3Dno works just fine >>=20 >> So it is a FreeBSD system error associated with >> use of tmpfs . >=20 > Recent work on tmpfs includes: >=20 > Mon, 09 Sep 2024 > =E2=80=A2 git: 8fa5e0f21fd1 - main - tmpfs: Account for whiteouts = during rename/rmdir Jason A. Harmening > Fri, 04 Oct 2024 > =E2=80=A2 git: 75734c4360fc - main - tmpfs: check residence in = data_locked Doug Moore > Sun, 13 Oct 2024 > =E2=80=A2 git: ec22e705c266 - main - tmpfs: remove duplicate flags = check in tmpfs_rmdir Alan Somers > Thu, 24 Oct 2024 > =E2=80=A2 git: db08b0b04dec - main - tmpfs_vnops: move swap work to = swap_pager Doug Moore >=20 > swap_pager (given the reference to it above): >=20 > Tue, 08 Oct 2024 > =E2=80=A2 git: d0b225d16418 - main - swap_pager: use iterators in = swp_pager_meta_build Doug Moore > Fri, 11 Oct 2024 > =E2=80=A2 git: 1107834090be - main - swap_pager: swapoff detecting = object death Doug Moore > Thu, 24 Oct 2024 > =E2=80=A2 git: 34951b0b9e78 - main - swap_pager: move = scan_all_shadowed, use iterators Doug Moore > =E2=80=A2 git: 02e85d1c8a41 - main - swap_pager: fix assert in = seek_data Doug Moore=20 > =E2=80=A2 git: faa9356f97d2 - main - swap_pager: fix seek_hole = assert Doug Moore > Sat, 26 Oct 2024 > =E2=80=A2 git: 39f6d1e7f835 - main - swap_pager: iter in haspage, = lookup, getpages Doug Moore > Wed, 13 Nov 2024 > =E2=80=A2 git: d11d407aee48 - main - swap_pager: Ensure that = swapoff puts swapped-in pages in page queues Mark Johnston >=20 > I do not know at this time when the corruptions started. The > above is only suggestive. With a bulk -i active but from outside the bulk -i : # df -m | sort -k6,6 | grep ^tmpfs tmpfs = 182907 0 182907 0% = /usr/local/poudriere/data/.m/main-amd64-default tmpfs = 184770 1863 182907 1% = /usr/local/poudriere/data/.m/main-amd64-default/ref tmpfs = 2048 45 2002 2% = /usr/local/poudriere/data/.m/main-amd64-default/ref/.p tmpfs = 182907 0 182907 0% = /usr/local/poudriere/data/.m/main-amd64-default/ref/var/db/ports Note: bulk -i lands one in = /usr/local/poudriere/data/.m/main-amd64-default/ref/ =46rom inside a bulk -i where I did a manual make command after it built and installed libsass.so.1.0.0 . The manual make produced a /wrkdirs/ : # find -s / -name libsass.so.1.0.0 -exec ls -ilodT {} \; 6417 -rwxr-xr-x 1 root wheel - 42444424 Nov 26 07:24:37 2024 = /usr/local/lib/libsass.so.1.0.0 11872 -rwxr-xr-x 1 root wheel - 42444424 Nov 26 07:26:48 2024 = /wrkdirs/usr/ports/textproc/libsass/work/libsass-3.6.6/src/.libs/libsass.s= o.1.0.0 12294 -rwxr-xr-x 1 root wheel - 42444424 Nov 26 07:26:48 2024 = /wrkdirs/usr/ports/textproc/libsass/work/stage/usr/local/lib/libsass.so.1.= 0.0 # objdump -hs = /wrkdirs/usr/ports/textproc/libsass/work/libsass-3.6.6/src/.libs/libsass.s= o.1.0.0 | less . . . 2bed60 78ba2b00 00000000 00000000 00000000 x.+............. 2bed70 00000000 00000000 86a62a00 00000000 ..........*..... 2bed80 96a62a00 00000000 a6a62a00 00000000 ..*.......*..... 2bed90 b6a62a00 00000000 c6a62a00 00000000 ..*.......*..... . . . So the original creation looks okay. But . . . # objdump -hs = /wrkdirs/usr/ports/textproc/libsass/work/stage/usr/local/lib/libsass.so.1.= 0.0 | less . . . 2bed60 00000000 00000000 00000000 00000000 ................ 2bed70 00000000 00000000 00000000 00000000 ................ 2bed80 00000000 00000000 00000000 00000000 ................ 2bed90 00000000 00000000 00000000 00000000 ................ . . . 2befc0 00000000 00000000 00000000 00000000 ................ 2befd0 00000000 00000000 00000000 00000000 ................ 2befe0 00000000 00000000 00000000 00000000 ................ 2beff0 00000000 00000000 00000000 00000000 ................ 2bf000 96ab2a00 00000000 a6ab2a00 00000000 ..*.......*..... 2bf010 b6ab2a00 00000000 c6ab2a00 00000000 ..*.......*..... 2bf020 d6ab2a00 00000000 e6ab2a00 00000000 ..*.......*..... 2bf030 f6ab2a00 00000000 06ac2a00 00000000 ..*.......*..... . . . So: The later, staged copy is a bad copy. Both are in the tmpfs. So copying to the staging area makes a corrupted copy inside the same tmpfs. After that, further copies of staging's bad copy can be expected to be messed up. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Tue Nov 26 10:19:52 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XyJV30Lq0z5fFT5; Tue, 26 Nov 2024 10:19:55 +0000 (UTC) (envelope-from des@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XyJV26wwGz4nBr; Tue, 26 Nov 2024 10:19:54 +0000 (UTC) (envelope-from des@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732616395; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=PINMBuu0iwZzr3F/s3L9g0j4rp08jQ1nqeoc2rsw66M=; b=W7pmLmuGNv6iT3Sefax8D2IqQiI3ClmE0c5ccSSZcW5KH5dNHoABmvWBli+ReoG01Rit50 2oBo/zzgZj4hAGkuMUpHgRVK4tbKqQhIC3Y5+AuQMRzhn6363BkxlTi+lvoqMfHOVuH9Em RJ4zdH0WR73xpqpGNvIz41p6yC26UtUuRniRuNxEFOBHSD0NeigzMei0cQr2UOJIZbYaV9 EPtVWqKSaGaDMk3r6Ww+g7Y5ka5HVugSgMEYQgCZ3M1enl6RLh6JhVvkpgVKFv6ACSYunE uSCHI1vaylukFKGQrTKmz/tWpgM/iqDVVEb88gvUWNKSxq2gRq6KW2A6MEz6rA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732616395; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=PINMBuu0iwZzr3F/s3L9g0j4rp08jQ1nqeoc2rsw66M=; b=g45e1TLeqali2+A8Jl/SUG3V+6zkKzs5Bs1xukDB7MlVxKvRaatWjdOLhM/PnXGPg4VYKI snE2QjUy/mzVzSvMGt4VB7aVHxyvFxRNgnjK4t+126IIW+ZWiOJjYA2nFw2Rk/eixlKI0Y kiDSBP0Ce83kiA5+kIZzye5a00e1V3KLefe4WgO46MvnigDg8mXud6wI6jLenVqe/RB5SA elRSVW50HbI8PqrXCvSPjURwofQfopSFPpP+Q0ZpvmWtVAhdbpsPmWP3/3gnrPW+I2ldhi 06qxPePbXiwNIgUA3gSu61fVlvXY+tX2IUWRGNH/vMgCXz3Cy9w6eE6sxS+MIw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1732616395; a=rsa-sha256; cv=none; b=gZCYD80x2SGlHtiLMC0kbnZBGPVlV/jpfWX+qay8Y2DG1JaoZQ7+RWVVbKsSYElJWDC4fn u75pRaRjl/IgldAuOQklnRzMSHsEj3fphtchn43BwS7KKlm1484e9qRfG6AruFxIY/8755 LhDb2WJUHQcLzuAcarl7qm73kjx/5mAPkrTwzvVzExqWxcmSrSElhARZlOBgGcJPr2j+j+ Kf2gahDcMMqW8FE0QGTHcOtPy9u8Oitf10Df7/Xm1PJ85mNp5O8eul1/coYLLjoFCepVOF g/ebvlyoVFLZeNBxZ4uCPSlWQI4k2DFVzuXcHqUemq4fDR1FZCk14TCNKbYo9g== Received: from ltc.des.dev (unknown [IPv6:2a01:e0a:c54:bed0:922e:16ff:fef1:acef]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: des) by smtp.freebsd.org (Postfix) with ESMTPSA id 4XyJV25nzlzcxx; Tue, 26 Nov 2024 10:19:54 +0000 (UTC) (envelope-from des@freebsd.org) Received: by ltc.des.dev (Postfix, from userid 1001) id 585F86028F; Tue, 26 Nov 2024 11:19:52 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Mark Millard Cc: "jah@freebsd.org" , dougm@freebsd.org, asomers@freebsd.org, Mark Johnston , FreeBSD Current , Dimitry Andric , Guido Falsi , Yasuhiro Kimura , ports@freebsd.org Subject: Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros] In-Reply-To: <3660625A-0EE8-40DA-A248-EC18C734718C@yahoo.com> (Mark Millard's message of "Tue, 26 Nov 2024 00:21:40 -0800") References: <7c9c3cf5-bbd1-4642-8d04-33aa07a4db02@madpilot.net> <9df256a8-c6ed-46d9-b955-fc2657c12d36@madpilot.net> <5c502054-7353-4a1e-8350-c403482e9c0d@madpilot.net> <3127C3BA-FC93-4636-ADDB-89518DE9C60D@FreeBSD.org> <86ed2zsp6l.fsf@ltc.des.dev> <5f24a570-26e0-4c0a-817f-591a234fd07b@madpilot.net> <5918C6A1-8FDB-40CA-8C86-EB7B7BE75A2E@yahoo.com> <86ed2zc8r5.fsf@ltc.des.dev> <45098ccf-4dc6-426c-849a-c923805d6723@madpilot.net> <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> <0690CFB1-6A6D-4B63-916C-BAB7F6256000@yahoo.com> <3660625A-0EE8-40DA-A248-EC18C734718C@yahoo.com> User-Agent: Gnus/5.13 (Gnus v5.13) Date: Tue, 26 Nov 2024 11:19:52 +0100 Message-ID: <865xoa2t6f.fsf@ltc.des.dev> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mark Millard writes: > From inside a bulk -i where I did a manual make command > after it built and installed libsass.so.1.0.0 . The > manual make produced a /wrkdirs/ : > [...] > So the original creation looks okay. But . . . > [...] > So: The later, staged copy is a bad copy. Both are in the > tmpfs. So copying to the staging area makes a corrupted > copy inside the same tmpfs. After that, further copies of > staging's bad copy can be expected to be messed up. This and the fact that it happens on 14 and 15 but not on 13 strongly suggests an issue wth `copy_file_range(2)`, since `install(1)` in 14 and 15 (but not in 13) now uses `copy_file_range(2)` if at all possible. My educated guess is that hole detection doesn't work reliably for files that have had holes filled while memory-mapped, so `copy_file_range(2)` thinks there is a hole where there isn't one and skips some of the data when `install(1)` uses it to copy the library from `${WRKSRC}` to `${STAGEDIR}`. This may or may not be specific to tmpfs. You may want to try applying the attached patch to your FreeBSD 14 and 15 jails. It prevents `cp(1)` and `install(1)` from trying to use `copy_file_range(2)`. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@FreeBSD.org --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-cp-install-disable-copy_file_range.patch >From 18eb75139045c30609d93f6a138526d3288acbd9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Dag-Erling=20Sm=C3=B8rgrav?= Date: Tue, 26 Nov 2024 10:54:14 +0100 Subject: [PATCH] cp, install: disable copy_file_range. --- bin/cp/utils.c | 2 +- usr.bin/xinstall/xinstall.c | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/bin/cp/utils.c b/bin/cp/utils.c index cfbb2022caaf..c6a688235bf1 100644 --- a/bin/cp/utils.c +++ b/bin/cp/utils.c @@ -104,7 +104,7 @@ copy_file(const FTSENT *entp, int dne) ssize_t wcount; off_t wtotal; int ch, checkch, from_fd, rval, to_fd; - int use_copy_file_range = 1; + int use_copy_file_range = 0; fs = entp->fts_statp; from_fd = to_fd = -1; diff --git a/usr.bin/xinstall/xinstall.c b/usr.bin/xinstall/xinstall.c index 2823a9040b7a..4ab0a6dd0de2 100644 --- a/usr.bin/xinstall/xinstall.c +++ b/usr.bin/xinstall/xinstall.c @@ -1194,6 +1194,7 @@ copy(int from_fd, const char *from_name, int to_fd, const char *to_name, err(EX_OSERR, "lseek: %s", to_name); #ifndef BOOTSTRAP_XINSTALL +#if 0 /* Try copy_file_range() if no digest is requested */ if (digesttype == DIGEST_NONE) { do { @@ -1210,6 +1211,7 @@ copy(int from_fd, const char *from_name, int to_fd, const char *to_name, } /* Fall back */ } +#endif #endif digest_init(&ctx); -- 2.46.0 --=-=-=-- From nobody Tue Nov 26 10:45:49 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XyK4h17VMz5fHDr for ; Tue, 26 Nov 2024 10:46:28 +0000 (UTC) (envelope-from zarychtam@plan-b.pwste.edu.pl) Received: from plan-b.pwste.edu.pl (plan-b.pwste.edu.pl [IPv6:2001:678:618::40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "plan-b.pwste.edu.pl", Issuer "GEANT OV RSA CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XyK4g4Pfsz4sLt for ; Tue, 26 Nov 2024 10:46:27 +0000 (UTC) (envelope-from zarychtam@plan-b.pwste.edu.pl) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=plan-b.pwste.edu.pl header.s=plan-b-mailer header.b=rNhAUN+N; spf=pass (mx1.freebsd.org: domain of zarychtam@plan-b.pwste.edu.pl designates 2001:678:618::40 as permitted sender) smtp.mailfrom=zarychtam@plan-b.pwste.edu.pl; dmarc=pass (policy=quarantine) header.from=plan-b.pwste.edu.pl Received: from [IPV6:2001:678:618:402f:f59a:b19b:4650:c749] ([IPv6:2001:678:618:402f:f59a:b19b:4650:c749]) (authenticated bits=0) by plan-b.pwste.edu.pl (8.18.1/8.17.2) with ESMTPSA id 4AQAkHt4070476 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Tue, 26 Nov 2024 11:46:17 +0100 (CET) (envelope-from zarychtam@plan-b.pwste.edu.pl) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=plan-b.pwste.edu.pl; s=plan-b-mailer; t=1732617978; bh=ywJGz9sR8ygJIjALq34nZUjjLzkR5uuC+NdI7JHAOp4=; h=Date:Subject:To:References:From:In-Reply-To; b=rNhAUN+NbZe+mC7ZgJtB8jKjYHuSJ5BMKOwlXbw8zYaHNkcYHHvfZfF1YoCJa8Jf5 7q5YneS79fG1cP20B3GPBUpc8f/b96Wv/g6yONy/kLbTXX7kctvzumcrIRQYgY+I2i ry/qGl+KX48v2rfWMROFVTDkkFgcLaBBDb9pMz9dpyHpUM0A5tXrOeadEp58Qr5vtU vz5ipSVrVNl0eLzUCa8vFxXdTT0wFLKYGPS4xjEA9E2QGpaFHgUp2jmWpAuwLkSbY2 Tzmmsy8CpuBOqvOHSdBBhr0jmwQ96dqTvqxJYDCElPP5pVY4zyiJbL2+jN/FHqQ5ar 8DaRGqH0icHoQ== Content-Type: multipart/alternative; boundary="------------JJwlfiTBiX1FfoeyMehVHV0n" Message-ID: <165139e3-d31a-4f8f-8157-4bc47eaca9cf@plan-b.pwste.edu.pl> Date: Tue, 26 Nov 2024 12:45:49 +0200 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: port binary dumping core on recent head in poudriere To: ports@freebsd.org, freebsd-current@freebsd.org References: <7c9c3cf5-bbd1-4642-8d04-33aa07a4db02@madpilot.net> <9df256a8-c6ed-46d9-b955-fc2657c12d36@madpilot.net> <5c502054-7353-4a1e-8350-c403482e9c0d@madpilot.net> <3127C3BA-FC93-4636-ADDB-89518DE9C60D@FreeBSD.org> <86ed2zsp6l.fsf@ltc.des.dev> <5f24a570-26e0-4c0a-817f-591a234fd07b@madpilot.net> <5918C6A1-8FDB-40CA-8C86-EB7B7BE75A2E@yahoo.com> <86ed2zc8r5.fsf@ltc.des.dev> <45098ccf-4dc6-426c-849a-c923805d6723@madpilot.net> <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> Content-Language: en-US From: Marek Zarychta In-Reply-To: X-Spamd-Result: default: False [-3.89 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.996]; DMARC_POLICY_ALLOW(-0.50)[plan-b.pwste.edu.pl,quarantine]; R_DKIM_ALLOW(-0.20)[plan-b.pwste.edu.pl:s=plan-b-mailer]; R_SPF_ALLOW(-0.20)[+mx:c]; ONCE_RECEIVED(0.10)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; XM_UA_NO_VERSION(0.01)[]; ASN(0.00)[asn:206006, ipnet:2001:678:618::/48, country:PL]; RCVD_COUNT_ONE(0.00)[1]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_IN_DNSWL_NONE(0.00)[2001:678:618::40:from]; DWL_DNSWL_NONE(0.00)[pwste.edu.pl:dkim]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[plan-b.pwste.edu.pl:+] X-Rspamd-Queue-Id: 4XyK4g4Pfsz4sLt X-Spamd-Bar: --- This is a multi-part message in MIME format. --------------JJwlfiTBiX1FfoeyMehVHV0n Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit W dniu 26.11.2024 o 08:59, Guido Falsi pisze: > On 26/11/24 03:05, Mark Millard wrote: >> Top posting going in a different direction that >> established a way to control the behavior in my >> context . . . >> >> I changed USE_TMPFS=all to USE_TMPFS=no : >> >> USE_TMPFS=all gets the failure >> vs. >> USE_TMPFS=no works just fine >> >> So it is a FreeBSD system error associated with >> use of tmpfs . >> > > Interesting discovery, I'm also using USE_TMPFS=all > > I'll try to do a test without that but I won't be able before tomorrow > I think. Interesting thread. With the setting USE_TMPFS=all I was  able to build neither www/chromium nor www/qt6-webengine on stable/14, and even submitted the PR[1] regarding to this. 1. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=278293 -- Marek Zarychta --------------JJwlfiTBiX1FfoeyMehVHV0n Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit

W dniu 26.11.2024 o 08:59, Guido Falsi pisze:

On 26/11/24 03:05, Mark Millard wrote:
Top posting going in a different direction that
established a way to control the behavior in my
context . . .

I changed USE_TMPFS=all to USE_TMPFS=no :

USE_TMPFS=all gets the failure
vs.
USE_TMPFS=no works just fine

So it is a FreeBSD system error associated with
use of tmpfs .


Interesting discovery, I'm also using USE_TMPFS=all

I'll try to do a test without that but I won't be able before tomorrow I think.

Interesting thread. With the setting USE_TMPFS=all I was  able to build neither www/chromium nor www/qt6-webengine on stable/14, and even submitted the PR[1] regarding to this.

1. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=278293


-- 
Marek Zarychta
--------------JJwlfiTBiX1FfoeyMehVHV0n-- From nobody Tue Nov 26 12:31:07 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XyMPY517nz5fPRV for ; Tue, 26 Nov 2024 12:31:13 +0000 (UTC) (envelope-from joh.hendriks@gmail.com) Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com [IPv6:2a00:1450:4864:20::52f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XyMPX0g6yz455j for ; Tue, 26 Nov 2024 12:31:12 +0000 (UTC) (envelope-from joh.hendriks@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=bsGfvA5G; spf=pass (mx1.freebsd.org: domain of joh.hendriks@gmail.com designates 2a00:1450:4864:20::52f as permitted sender) smtp.mailfrom=joh.hendriks@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-ed1-x52f.google.com with SMTP id 4fb4d7f45d1cf-5cf6f367f97so6816376a12.0 for ; Tue, 26 Nov 2024 04:31:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1732624270; x=1733229070; darn=freebsd.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id:from :to:cc:subject:date:message-id:reply-to; bh=f8fpeoa6rISlNnOCGVLI7aOXrCNpjYMVbYJeNswGv18=; b=bsGfvA5GN5oGKByNLcI0SuCvPTgZe+Hk48Ec32SONbG1tQRWwJua7Nmv+GGR0UwKI+ q/9lkc8+DQdqTlHwLM6GX95et6McZYQxnea3D+CRv4k1eJ6ZSM/4xp2oF0KS4/V9giuS /F0nBPIoMtFHM0bd4DzTFCbeLU8wO4jbkofPXYhfvG5jxVIkEbQQ1A+CokJvOpUT3tRn AFgMJarPcKFUZ2e50HKvUZa/m1SpxbeTzeFBpW31kKrYmzIRiGlSCBYEhs11ffDAo4wb 87NerWYEY1hTRr5oGo7CWgUn0rKMvtX5kaWLrQ3wGbvXzEHmg8reA5ULzhQirJjsa1Mk T5RQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732624270; x=1733229070; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=f8fpeoa6rISlNnOCGVLI7aOXrCNpjYMVbYJeNswGv18=; b=DZxAvUupwxsEWNjXPJgrfNI9UGYqg9+zxgOP1cDm8+H04FCeD5lnwW3Ax9ST2bRgVq jXSG7DhFCbWNUe/EURZS5kQsRHmMPUE/7EFxCIyNjrljuhDo0ZmWfaYgkT9wZeaUNR5d SWLtMi3JD1HK+QhGDJIQ5FkkwgcIf/Abic1Bjg12B15z9+kXnttIyX29LAPteBJg8/TO 4Z5XxrAwI8S4Novpv7pJgof/W4ROmk55asvPD0sSxZMIMsfzileDY/RJMfQrwBdsj/5d vMBlMfpiIZl3QYnw4E0gREuJ9UJivfL4sniLZgDXNlj8jTrpGw/sJ8jVhFNff0Kbpf2o jzZA== X-Gm-Message-State: AOJu0Yzf1NCa+r16lUtXNQhaRlYI72+ZzyVWa/mV71IYzDdnaqcUWHm3 bIU6doPtj9DmdkOQ4EO/H0Gw7UWq0mMOIMsqIDCIfZw9/GNSFqdsseu+3BiU X-Gm-Gg: ASbGncvHwbyAS1TEZIxJtUkqTTq147VIo0g/pUX/oVCUMn4XJ2JZt7NaThtsVoZBmY2 KNuEvHLVYGy8j3bUfV2FBF7AgglSduNcNcSZLJkyOyl9HbNg06uMIhqA/0PvdmN4WpLhgaemMmC lSdx7xaeTOG7/YdTWYSBybJaUscM00zuCQL6MmZBr6ncK1UUF02BZzOk9t90glwoyW4mF5e/5IW cfrEzfNRzJAMtwY1nSkDxtczUGSGmzSMbEcBd5prPnLiScsdNqiB10vgioAo5wOlAouG6FQEX1G 4F+37NpyM8IzQLrYta5yAeO4UwbDtc6xOZEbEAw/ejPgqL60+ZkItdMfevuTVowVD4EPcQwdVOO NDxLirG9AMYQ+ooPTbdtH36rUIgS1p7V4C7T+Uk2LFxkm0c02 X-Google-Smtp-Source: AGHT+IG0OwXxiSfBj3eFK8zRifXR4okkrRAy8cNaY77RdZ78wycwcEdfYmNZI2dWmxFmQbZ3pa3CKQ== X-Received: by 2002:a05:6402:5212:b0:5cf:bb9e:cca7 with SMTP id 4fb4d7f45d1cf-5d0207b2521mr14374144a12.28.1732624269591; Tue, 26 Nov 2024 04:31:09 -0800 (PST) Received: from ?IPV6:2001:1c04:1337:b400:b001:3f19:b5e4:2ffd? (2001-1c04-1337-b400-b001-3f19-b5e4-2ffd.cable.dynamic.v6.ziggo.nl. [2001:1c04:1337:b400:b001:3f19:b5e4:2ffd]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aa50b28dd59sm588385866b.7.2024.11.26.04.31.08 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 26 Nov 2024 04:31:08 -0800 (PST) Message-ID: <9f572320-bc01-41c2-9820-74844b6ec9f1@gmail.com> Date: Tue, 26 Nov 2024 13:31:07 +0100 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: IBM ServeRAID M5110e and other mfi/mrsas users - call for testing / request for information To: freebsd-current@freebsd.org References: Content-Language: en-US From: Johan Hendriks In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-3.86 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.87)[-0.868]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; RCPT_COUNT_ONE(0.00)[1]; TAGGED_FROM(0.00)[]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; TO_DN_NONE(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::52f:from] X-Rspamd-Queue-Id: 4XyMPX0g6yz455j X-Spamd-Bar: --- On 25/11/2024 21:24, Ed Maste wrote: > I have a review open to switch to mrsas(4) by default, for devices > supported by both it and mfi(4). mrsas(4) should be better supported > and more functional. The review: > > https://reviews.freebsd.org/D45476 mfi: default to using mrsas(4) for > newer cards > > However, we have a report of volume corruption with the IBM ServeRAID > M5110e when used with the mrsas driver: > > PR196820 IBM x3650, ServeRAID M5110e, volumes corrupted after boot > with mrsas driver > > Clearly we do not want to corrupt users' RAID volumes, so I would like > to request feedback from mfi/mrsas users in general, and M5110e users > in particular. > > Note that there may be a possibility of data corruption, so please > investigate only on test systems. > > To test, set the loader tunable hw.mfi.mrsas_enable=1 in > /boot/loader.conf. Confirm that mrsas is being used, and that the > system functions properly. > I use Dell servers and i set the loader tuneable hw.mfi.mrsasa_enable=1 on these machines. I use ZFS, so no raid functionality off the card itself. This works fine for me. I have not have any issues. I use this setting as mrsas give me way nicer device names, i do remember having boot issues with the mfi default. But that is a long time ago. From nobody Tue Nov 26 12:32:59 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XyMRd2xqPz5fPVj; Tue, 26 Nov 2024 12:33:01 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XyMRd1MZyz46F2; Tue, 26 Nov 2024 12:33:01 +0000 (UTC) (envelope-from dim@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732624381; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tix2TSlScJEv85RcKuPOW0PXrwyrPYhassD35aj5hCs=; b=Bh2vNmqv9JXY/c2/PQ3oo6gX9EyCHJidt6u/Tvn+OexR7FzieBdE7j6T2zxGdzHXNUe7NC MpqodmeskxHJ4KOIjr6oW/cpwsM/Dgdr7Rv0+RGVeebS6sdLEpJ3Ql40ECyCuvQlqfEEyu ZcSiYeEgJ1EhbYkk8vj1wxxpY/vksLDBVV8dO/YnbJyMWgAyfuRoo1MsE/2dROU5+qu7Lr Kg7USWpZSJBLhY+KrKTOStiMrqNoo3GdMuxt+xDBKeEbZv5jBMgmfkBNoBcogVuTLobTZJ VXd63ZE3tgcnFbidrCXDPIsVVCMXeaRAqBC/MSC4OxzsPA/pXe3Adg3ktnxVRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732624381; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tix2TSlScJEv85RcKuPOW0PXrwyrPYhassD35aj5hCs=; b=pgIybOSb3Hca5VNXjHoQgkRpEj3QQuVdL35MO9GEl3O6i+Xn4UmG8zBEIwH+CHm1Q8vtBH aNHWtyZftEYxqNvykpeJ89D7FZ+rJ2KglCI6tV4vtbDTXv7gm8bvfN69cqX6iigVZwORco /yphDeoNJrtdT8A2CZwUb5XSMH9vOvCioJdiluQYoHVeaBZcjJJQcl9oCPtNXBUZSuuykO hpv9QZ085tyOfgwikBp5b9vCjJLAnM63peGd8SSucEsFrAy8VhiwmC5cx2F1W1XZ8oB7YG 2AUftUcvH+S+tpU9EvwlqJ6t0cGySsZIKj8FUZyiSF7srfMD+fwsPnMho9h13w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1732624381; a=rsa-sha256; cv=none; b=nn+3ewm7dGT0BEK+JgRo8IQG4Tld4L1e16hYTQuR7/Y+Z43dJEDulDsm33lkB2o/f9kaBr raKYCF1qLtPWjnvDHG7+T1HRJE8jvDB1RbTN9/IN62sO4Q/ThKMwFRsrClRnzxTXAGQm9y mNYAP6qnm/peRjWUHJUmh785jytEJ1gWJaqimI/k48con8xiNh1CG3bY5FCcEXqfxINlWg YDj/LDw19h8oaeDgOBvpd+QCOMN43zepBJRIJVe5JNq/yepvZP5q5bs5gZihnhfjzTfYoF uCg7wiq0Dr5ATpG5tyAn/WUVFpTZ0RcFCVbH99PFgjtfPVMRh05g1YdozbzAVA== Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "tensor.andric.com", Issuer "R10" (verified OK)) (Authenticated sender: dim) by smtp.freebsd.org (Postfix) with ESMTPSA id 4XyMRd0Q8Gzg64; Tue, 26 Nov 2024 12:33:01 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtpclient.apple (longrow.wg.andric.com [10.69.1.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id B21B13D086; Tue, 26 Nov 2024 13:32:59 +0100 (CET) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6.1.9\)) Subject: Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros] From: Dimitry Andric In-Reply-To: <865xoa2t6f.fsf@ltc.des.dev> Date: Tue, 26 Nov 2024 13:32:59 +0100 Cc: Mark Millard , "jah@freebsd.org" , dougm@freebsd.org, Alan Somers , Mark Johnston , FreeBSD Current , Guido Falsi , Yasuhiro Kimura , ports@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <69A2E921-F5E3-40D2-977D-0964EE27349A@FreeBSD.org> References: <7c9c3cf5-bbd1-4642-8d04-33aa07a4db02@madpilot.net> <9df256a8-c6ed-46d9-b955-fc2657c12d36@madpilot.net> <5c502054-7353-4a1e-8350-c403482e9c0d@madpilot.net> <3127C3BA-FC93-4636-ADDB-89518DE9C60D@FreeBSD.org> <86ed2zsp6l.fsf@ltc.des.dev> <5f24a570-26e0-4c0a-817f-591a234fd07b@madpilot.net> <5918C6A1-8FDB-40CA-8C86-EB7B7BE75A2E@yahoo.com> <86ed2zc8r5.fsf@ltc.des.dev> <45098ccf-4dc6-426c-849a-c923805d6723@madpilot.net> <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> <0690CFB1-6A6D-4B63-916C-BAB7F6256000@yahoo.com> <3660625A-0EE8-40DA-A248-EC18C734718C@yahoo.com> <865xoa2t6f.fsf@ltc.des.dev> To: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= X-Mailer: Apple Mail (2.3731.700.6.1.9) On 26 Nov 2024, at 11:19, Dag-Erling Sm=C3=B8rgrav = wrote: >=20 > Mark Millard writes: >> =46rom inside a bulk -i where I did a manual make command >> after it built and installed libsass.so.1.0.0 . The >> manual make produced a /wrkdirs/ : >> [...] >> So the original creation looks okay. But . . . >> [...] >> So: The later, staged copy is a bad copy. Both are in the >> tmpfs. So copying to the staging area makes a corrupted >> copy inside the same tmpfs. After that, further copies of >> staging's bad copy can be expected to be messed up. >=20 > This and the fact that it happens on 14 and 15 but not on 13 strongly > suggests an issue wth `copy_file_range(2)`, since `install(1)` in 14 = and > 15 (but not in 13) now uses `copy_file_range(2)` if at all possible. >=20 > My educated guess is that hole detection doesn't work reliably for = files > that have had holes filled while memory-mapped, so = `copy_file_range(2)` > thinks there is a hole where there isn't one and skips some of the = data > when `install(1)` uses it to copy the library from `${WRKSRC}` to > `${STAGEDIR}`. This may or may not be specific to tmpfs. >=20 > You may want to try applying the attached patch to your FreeBSD 14 and > 15 jails. It prevents `cp(1)` and `install(1)` from trying to use > `copy_file_range(2)`. Yes, tmpfs is indeed the culprit (or at least involved). I have had = USE_TMPFS=3Dlocalbase in my poudriere.conf for a long time, since = otherwise my build machine would run out of memory very quickly, so I = didn't encounter any issues. Now I changed it to USE_TMPFS=3Dyes, rebuilt only textproc/libsass and = textproc/sassc, and then after reinstalling those packages: $ /usr/local/bin/sassc Segmentation fault -Dimitry From nobody Tue Nov 26 12:58:19 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XyN0s3yz1z5fR6k; Tue, 26 Nov 2024 12:58:21 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XyN0s3H1tz4F44; Tue, 26 Nov 2024 12:58:21 +0000 (UTC) (envelope-from dim@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732625901; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3YjY0F0J58P1bQ73abAZVdvR5QyDjcaV0WB09HY53J4=; b=S/u0iHh7xAR8o7K4lXyAElGhpF0KnZzuqL4KSBXi+arsqefa8NXFW5CX6cKCI2sjeWl+bZ cieJAmka+K/zQQVK0EoL9GHTCVli4Zx/dbiA/Swz/f0lrOyL5E770uEkAyML0OANO5xOk6 dPsaXWX8DeCUTybDKBDhbqx+wmAs+ZvXkjPCF5itUCLdEEjfOqGvld2xaseZio4k/VKZ6H kUvRl+kg/3phwoMkp2cFMcq85w2tfUuGGlFkqaaO6rT7NZZKy86r/HqYIMUzcmgfGeO3PL cDJ7iDPYY4LX4g0N4LnUocCia0rjxHoK4wUjElzHUzlbxE2Xwzqnd/duKWbk8A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732625901; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3YjY0F0J58P1bQ73abAZVdvR5QyDjcaV0WB09HY53J4=; b=pDGiBxARQt/esJs7vgJ5TaYmyH7cUn4eCz2VnW6dV/y1tSaXMobb8xDQTi95ke5rgQXOgn kNC9AJO8oNNzmV/c3Xn7cDh0MaWfUn9PgzqDzy9MUn0kS50i/3H/CDbXJSetsDTD8X6mS+ JmHC7Es3iypUY52UQNw5DpX8u67ZUcYuKhLW2lZAtHqFSd486konTVihG9BkIv6iiEYGNE ZRvkEcNaz9bBQ6Cc1Je6azLia8+uHbtywIHrZCTZqLnSRAM7aqeHAxKZvmI3D5xSm3opFz hDmuXFyeJCtNSSc5pnpJLynQGWNNeQGqdrH/aRDYknhxpjN9+Cqz1O7zbgZ0zA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1732625901; a=rsa-sha256; cv=none; b=s2qTQJ/kYiKusEWSEuxhHkERL36aPWNXohKTROQYRUq0cwrBgQQ7PBi7IS+fi8YOf/B3l3 FIJM+TUfgOs+PwyICqmd6klXZs9jcMIfxkudOhf8cS2DOGT2+9vxT2xq1y2SCEGA7d/iha aCuMvsjz93VVEx6HVL2bxTRWmp/BacuOOtoA6GvXMwsOeBQvMexDZqpFRRCBXP6vsvB5tk O9wpw+kFKz3Fegj8Kh+nb9whBvIvtWk+RFke4+KQKS7WECt7w4noOiIezd9Gtbd+b5geqx 7VkA5rVbmTKJP7sE2MQKt6vAFJ03I9dwDzQBWAFXTI08r4o2WygQmWt9dofSxg== Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "tensor.andric.com", Issuer "R10" (verified OK)) (Authenticated sender: dim) by smtp.freebsd.org (Postfix) with ESMTPSA id 4XyN0s21SczhJ1; Tue, 26 Nov 2024 12:58:21 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtpclient.apple (longrow.wg.andric.com [10.69.1.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 033FF3CDEC; Tue, 26 Nov 2024 13:58:19 +0100 (CET) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6.1.9\)) Subject: Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros] From: Dimitry Andric In-Reply-To: <69A2E921-F5E3-40D2-977D-0964EE27349A@FreeBSD.org> Date: Tue, 26 Nov 2024 13:58:19 +0100 Cc: Mark Millard , "jah@freebsd.org" , dougm@freebsd.org, Alan Somers , Mark Johnston , FreeBSD Current , Guido Falsi , Yasuhiro Kimura , ports@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <4AE5B316-D7EB-4290-8D52-7FBF244EA7A4@FreeBSD.org> References: <7c9c3cf5-bbd1-4642-8d04-33aa07a4db02@madpilot.net> <9df256a8-c6ed-46d9-b955-fc2657c12d36@madpilot.net> <5c502054-7353-4a1e-8350-c403482e9c0d@madpilot.net> <3127C3BA-FC93-4636-ADDB-89518DE9C60D@FreeBSD.org> <86ed2zsp6l.fsf@ltc.des.dev> <5f24a570-26e0-4c0a-817f-591a234fd07b@madpilot.net> <5918C6A1-8FDB-40CA-8C86-EB7B7BE75A2E@yahoo.com> <86ed2zc8r5.fsf@ltc.des.dev> <45098ccf-4dc6-426c-849a-c923805d6723@madpilot.net> <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> <0690CFB1-6A6D-4B63-916C-BAB7F6256000@yahoo.com> <3660625A-0EE8-40DA-A248-EC18C734718C@yahoo.com> <865xoa2t6f.fsf@ltc.des.dev> <69A2E921-F5E3-40D2-977D-0964EE27349A@FreeBSD.org> To: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= X-Mailer: Apple Mail (2.3731.700.6.1.9) On 26 Nov 2024, at 13:32, Dimitry Andric wrote: >=20 > On 26 Nov 2024, at 11:19, Dag-Erling Sm=C3=B8rgrav = wrote: >>=20 >> Mark Millard writes: >>> =46rom inside a bulk -i where I did a manual make command >>> after it built and installed libsass.so.1.0.0 . The >>> manual make produced a /wrkdirs/ : >>> [...] >>> So the original creation looks okay. But . . . >>> [...] >>> So: The later, staged copy is a bad copy. Both are in the >>> tmpfs. So copying to the staging area makes a corrupted >>> copy inside the same tmpfs. After that, further copies of >>> staging's bad copy can be expected to be messed up. >>=20 >> This and the fact that it happens on 14 and 15 but not on 13 strongly >> suggests an issue wth `copy_file_range(2)`, since `install(1)` in 14 = and >> 15 (but not in 13) now uses `copy_file_range(2)` if at all possible. >>=20 >> My educated guess is that hole detection doesn't work reliably for = files >> that have had holes filled while memory-mapped, so = `copy_file_range(2)` >> thinks there is a hole where there isn't one and skips some of the = data >> when `install(1)` uses it to copy the library from `${WRKSRC}` to >> `${STAGEDIR}`. This may or may not be specific to tmpfs. >>=20 >> You may want to try applying the attached patch to your FreeBSD 14 = and >> 15 jails. It prevents `cp(1)` and `install(1)` from trying to use >> `copy_file_range(2)`. >=20 > Yes, tmpfs is indeed the culprit (or at least involved). I have had = USE_TMPFS=3Dlocalbase in my poudriere.conf for a long time, since = otherwise my build machine would run out of memory very quickly, so I = didn't encounter any issues. >=20 > Now I changed it to USE_TMPFS=3Dyes, rebuilt only textproc/libsass and = textproc/sassc, and then after reinstalling those packages: >=20 > $ /usr/local/bin/sassc > Segmentation fault And after applying Dag-Erling's patch to disable copy_file_range for cp = and install, it works correctly again. -Dimitry From nobody Tue Nov 26 13:21:31 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XyNWw1gH8z5fSKQ for ; Tue, 26 Nov 2024 13:21:48 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic306-20.consmr.mail.gq1.yahoo.com (sonic306-20.consmr.mail.gq1.yahoo.com [98.137.68.83]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XyNWv5vRJz4Hyh for ; Tue, 26 Nov 2024 13:21:47 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732627306; bh=CQcaDV1rCASwiOGxrw4JwGMYjI2wJFN6viGlqigTp7E=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=KMo9Os87mXgoIB4l6PYk8M6QxXnWy+LGe7waePoGb0Ru+5i0ZmUk4tFaNAYYn1ULLENsTvRoryQp368qc8uM3IaYEsbpxOP2vKc8d0XjZqvWY4NAPyoBoLz7f2nRat7iZ5DuSjuBBKVcJx1eIumP2+QufOPqhNiTitEM4gn0pR8igMDYKO955vREMKZyyIvRp1apdwz0b19gNbkRXjCbXARPH1CaD5KjGuHnNVKvhHvzN+FrbjRBps4EohOe84n8V5tl6xQHjQvo9NWL8Dcffrxre5ZpDUIlg7mX6l6K1VJoXGnsiFB9EpYz4PPXNN0li0L1mHBjzz9Y4vkGzzs2QQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732627306; bh=01Jxo3cFYVuAoCFx6R1vU7Q5pwNh9eQQ3kMifFjgStA=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=srcqweq3w7ismN6vcv0X/p9mz3MJOCPZ5jgXw5lWr3vhh+fE83EePzvfMsnyUIWhGkMVibij1t/1zHQgCsBXhgP8zzDkqEDOkXHsQmp8+GAySTNUUW2q3Zd1i7zQviuPP+TDZqg79HMO3t0LTGFY/x4Aw5JV0GCs3PMymHssSqsvAMUHd0+rkKTrHOaaIIXzVQLOFVbNIMufp7853D96tFCVPKhTYV9UD0XUa5r0EdzcXNB3D2y4Cvokn0pVCpNGT+Rs6LB8TkGLNu2BLWnjYJNPgQ85JkNnYAaB5gzx8G9QnLZ4NEjIBNHAuJIJ6m7ZAH8FDpIPydQwmbiH4qXOzw== X-YMail-OSG: ZEx4BOQVM1kxQzv1wlw.TJreZHe8BByOquud0zf7CqJonsSI.Vz6Dxi0JZLhmBt .GO3EEoyNSqhYMj2eHp_GAX_u1nF5fQdYmZ32yQ8sN3Y_anqJ1rqhi06GgM6P8GO.3NEvDF4UPB3 1Q_QZ3bagXECiCBsPifFlG.D6yaDfwsEZL0h_9dCfukdPlJbS65bXxOiL1wyHkjkQAjjAsLsRrIb 9Od8q0UbthaPF0PAmK2wbgGDUJG83Tv0KyoMXJpb3i4ImGIsB4ABz1jJ_DF2nYZWCLWK3ojRMp2P bjr2UPZWadx56J96XwV0TvFo4y8nz9cNC6HEXfNklYaAqa46oQxyHxEQdD5HRivZ_gch03zU_80d G62P30p.K32ZnRlSiUScYZUhf_hU9cO6y2A8XRNxObQopRqJhlzhZKm9nkof_SUiwr3me6WmCi5k poPjcSud3uA2RljpJsFtXOfVEQaOvhz2cCSGjD_CeDAIADuncU1pxZVKrnD9ylXiFIO16Acf9Y8u 7NMMVlDLySc1xLY_foN6_UwH3DSsTMuNxRKLSXf0pIKBuVnCU6fUhXJWC_uxdhXz6RH6l8wIWTFM wJRDUZOwtivecnzFKVZR19.udsOxb9ArGrWLvl4fCJbQO8LMbg05E8OVXZGKG_730_LKBFUu_dVk eE8h3oY5A8ZvO4LnvVA2jt6EoKlFbJsc2CxznbwQDsMDSC.28xlZHH0YXo5__BpnpE2rEXGBHuYm J0ek4vyW7xBKkg2arlO3m27nNJzSLc8fxtk1d2iDStsE_.eIzKJ_6sZ2w5GAK_m1LHw9K836bQHy 4S0blBENu6B3iy9Gl1KyUhlyE6oeayb73y3JDrZipknd0I9q0koCxpeyzyDpUPX.4zN2m8zLyNIT G.lbUD7lvHgFYNdP3PgZGiPvuC1m.6_jY1Xi7ODHvKP.Hx9h3cHq346lox2JexWbYEANK54UHtRG T50jEP5x1mUx9mFcU7QXBAkmTqcbigieJs8OUGI9eaSXtf_R34kcujWmdClI5d3qo9HzgInhS68X KRIDLa69yTusX83YuEJv.k.Zqx18znOEPaJcA5eddEsfxaJo0p9FaxpVFDXo63Ohouz5kVDXtqHa AnJze6FkXC3t0Uo9ibaMA4ZoQ_QCBBvu43v9E2ct42WNJEvI86wZWpzS1smKkcGtQxHROC5Q122l M1D0JOxbQqopeiMVMhbEDExeI8AWQw01y9ygT96kZYX28tFd7NA63nBuZ_DQRj_2DoelCNfRBGte ivJqbA8wTy9k24s1DyGphVqfYqVUgmGruSdSwaMKWSILoLHIfrQWuRV7qVFmnwSVOWVnLt_AxmEK h4j8v4iw1roji.mmxcJvemd82IeBXwYjppV.CHYCD4i9KE8U6WLyUyhXQ6lnhvrbCdK4qnJ5XblO oxw_5Jp46Rxz3i5TA7teSEiOOFaioeu_ZIotSj4jcnpx3WCIMBsEEYDJMrH5mLOFYbMSd4ksaUv0 2xArwyU_HsMhf5ZzTdWk.1T.ZKpyxSx15OQOFwgciUHkUADWQVetby5d.4DvrfGVox_jIlGotRiW 1YHSvdUJ4rsIZjIxbN.jVtRGpJmZgur7_2ayl_PA8Gmg4PnSVKx9xZHjwO8A26dlL_mhdrygB0aa ErzKMnRn4uefxV0c0pE2WiKHbWLXEGAVWfZbBhomG5Hp08g2oYL6JpWn1l6BSQu..P2Kvb_UeRWR AEpQMNQSDPs43cHZSKNun1R9kaBgBViaEbiF5Om.Rnt3IkViwUKELjmJDobMcwfWnaAzVInL1aKz tgo1Nkj18rY6cPkZeJA4WTD9LVXBFLcpaxDCb9qlDyVUGUCQxNJXv1KNWRaQiz6ZVIBQcZae3t4K _9ERLIO0lDv_4ep62KjTSYWIv0VF.Utx3nji5.K5ywSCGYC.j_G99OUAHCsfxjck0.HIxf1forjD _iZMfGSY_O4BoKDhbSpfWT3RC3hvG4NRUK_DmWQEPrLay9Kgp4SHAXNfJnUrH0GOmvUmUDq.LHIx Niu.Torw5VpRBfpImWwDDh7jphJmp4EnrF1_oazveElhldG7VPsHQlMsOm.AwOBpoVy5FLAYy7AF ew2dv4HJZM298Bsz3JSuj5HkLnCqCMJctbOOlMK8F8VyACj2QU9bTNZIP7zJUHONaK4DKVbDMjmb iJ5SdO1glmBB8wn_PU29XpTBMw9NXG9TQxDIxYexaoB2oth_Of.M4xt5dqWPaKEiQuZ8fBv2nx8M XuO48mBQO60TUz4La_o2DCcAH2RdysTM54.uzwHEeIQUbp8zql6m7RZPN_Ql59mP3dpTd5_buFnY zaBk5XF1oh1rdenHu X-Sonic-MF: X-Sonic-ID: 06851145-5804-421b-bb7e-4406c40da9c2 Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.gq1.yahoo.com with HTTP; Tue, 26 Nov 2024 13:21:46 +0000 Received: by hermes--production-gq1-5dd4b47f46-whghm (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID c7bc7af1bd60671337a8ffcde8537c81; Tue, 26 Nov 2024 13:21:42 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51.11.1\)) Subject: Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros] From: Mark Millard In-Reply-To: <4AE5B316-D7EB-4290-8D52-7FBF244EA7A4@FreeBSD.org> Date: Tue, 26 Nov 2024 05:21:31 -0800 Cc: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , "jah@freebsd.org" , dougm@freebsd.org, Alan Somers , Mark Johnston , FreeBSD Current , Guido Falsi , Yasuhiro Kimura , ports@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <4EDAA149-2D45-40D3-A2CA-F6FBD06C18BF@yahoo.com> References: <7c9c3cf5-bbd1-4642-8d04-33aa07a4db02@madpilot.net> <9df256a8-c6ed-46d9-b955-fc2657c12d36@madpilot.net> <5c502054-7353-4a1e-8350-c403482e9c0d@madpilot.net> <3127C3BA-FC93-4636-ADDB-89518DE9C60D@FreeBSD.org> <86ed2zsp6l.fsf@ltc.des.dev> <5f24a570-26e0-4c0a-817f-591a234fd07b@madpilot.net> <5918C6A1-8FDB-40CA-8C86-EB7B7BE75A2E@yahoo.com> <86ed2zc8r5.fsf@ltc.des.dev> <45098ccf-4dc6-426c-849a-c923805d6723@madpilot.net> <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> <0690CFB1-6A6D-4B63-916C-BAB7F6256000@yahoo.com> <3660625A-0EE8-40DA-A248-EC18C734718C@yahoo.com> <865xoa2t6f.fsf@ltc.des.dev> <69A2E921-F5E3-40D2-977D-0964EE27349A@FreeBSD.org> <4AE5B316-D7EB-4290-8D52-7FBF244EA7A4@FreeBSD.org> To: Dimitry Andric X-Mailer: Apple Mail (2.3776.700.51.11.1) X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4XyNWv5vRJz4Hyh X-Spamd-Bar: ---- On Nov 26, 2024, at 04:58, Dimitry Andric wrote: > On 26 Nov 2024, at 13:32, Dimitry Andric wrote: >>=20 >> On 26 Nov 2024, at 11:19, Dag-Erling Sm=C3=B8rgrav = wrote: >>>=20 >>> Mark Millard writes: >>>> =46rom inside a bulk -i where I did a manual make command >>>> after it built and installed libsass.so.1.0.0 . The >>>> manual make produced a /wrkdirs/ : >>>> [...] >>>> So the original creation looks okay. But . . . >>>> [...] >>>> So: The later, staged copy is a bad copy. Both are in the >>>> tmpfs. So copying to the staging area makes a corrupted >>>> copy inside the same tmpfs. After that, further copies of >>>> staging's bad copy can be expected to be messed up. >>>=20 >>> This and the fact that it happens on 14 and 15 but not on 13 = strongly >>> suggests an issue wth `copy_file_range(2)`, since `install(1)` in 14 = and >>> 15 (but not in 13) now uses `copy_file_range(2)` if at all possible. >>>=20 >>> My educated guess is that hole detection doesn't work reliably for = files >>> that have had holes filled while memory-mapped, so = `copy_file_range(2)` >>> thinks there is a hole where there isn't one and skips some of the = data >>> when `install(1)` uses it to copy the library from `${WRKSRC}` to >>> `${STAGEDIR}`. This may or may not be specific to tmpfs. >>>=20 >>> You may want to try applying the attached patch to your FreeBSD 14 = and >>> 15 jails. It prevents `cp(1)` and `install(1)` from trying to use >>> `copy_file_range(2)`. >>=20 >> Yes, tmpfs is indeed the culprit (or at least involved). I have had = USE_TMPFS=3Dlocalbase in my poudriere.conf for a long time, since = otherwise my build machine would run out of memory very quickly, Use of TMPFS_BLACKLIST and TMPFS_BLACKLIST_TMPDIR can allow the use of USE_TMPFS=3Dall in many contexts. I'll later show my list that tries to exclude most everything using more than 7 or so GiBytes of tmpfs for the builder. If nothing else, it can help have a context for testing for the failure at hand for fairly general builds, including "bulk -a" . >> so I didn't encounter any issues. >>=20 >> Now I changed it to USE_TMPFS=3Dyes, rebuilt only textproc/libsass = and textproc/sassc, and then after reinstalling those packages: >>=20 >> $ /usr/local/bin/sassc >> Segmentation fault >=20 > And after applying Dag-Erling's patch to disable copy_file_range for = cp and install, it works correctly again. For reference (a very long line in its original form, noted in case something splits the line): TMPFS_BLACKLIST=3D"*-emacs_devel *-emacs_devel_nox *-emacs_nox *-gcc14 = *-rust-bootstrap 0ad RStudio aarch64-none-elf-gcc afni alliance anki = apache-openoffice apache-openoffice-devel arm-none-eabi-gcc binutils = biostar-tools blender boost-libs chezmoi chromium = chrono-physics-simulation-engine clickhouse cmake-core code_saturne deno = diaspora digikam dotnet dune-common dune-localfunctions dynare eclipse = eksctl electron[1-9][0-9] ess ess-emacs_canna firefox firefox-esr = foundationdb fr-aster freebsd-gcc14 gcc-arm-embedded = gcc-msp430-ti-toolchain gcc14 gcc1[45]-devel gdb geant4 ghc ghc810 ghc92 = ghc94 ghemical giacxcas grafana grafana-loki grafana9 gretl = gstreamer1-plugins-rust heyoka hs-cardano-db-sync = intel-graphics-compiler-llvm1[4321] iridium-browser julia kde5 kicad = kicad-devel kicad-doc kicad-library-packages3d* kosmorro kstars = libghemical libint2-psi4 libreoffice librewolf librsvg2-rust = libva-intel-media-driver llvm-devel llvm1[98764321] mesa-dri = mongodb[4-9][0-9] mpqc nerd-fonts nextcloudclient nextpnr octave = octave-forge octave-forge-bim octave-forge-msh octave-forge-sec*d = octave-forge-sole onlyoffice-documentserver paraview piglit = py39-orange3-single py39-pytorch pydio-cells qemu qemu-devel qemu-nox11 = qemu7 qemu7-nox11 qgis qgis-ltr qt*-webengine qt[56]*-webengine = qt[56]-tools quantum-espresso-pseudopotentials ringrtc rust rust-nightly = signal-desktop simpleitk telegraf tex-dvipdfmx tex-luatex tex-xetex = texlive-docs texlive-full thunderbird tor-browser trilinos trivy ttk = ungoogled-chromium vault vaultnextcloudclient virtualbox-ose = virtualbox-ose-legacy virtualbox-ose-nox11 vscode vuls = wasi-compiler-rt-* wasi-libcxx* webkit2-gtk3 wx30-gtk3 yazi ztop" =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Tue Nov 26 13:38:04 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XyNv21vMWz5fTpr; Tue, 26 Nov 2024 13:38:22 +0000 (UTC) (envelope-from kib@freebsd.org) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XyNv13hslz4PZ2; Tue, 26 Nov 2024 13:38:21 +0000 (UTC) (envelope-from kib@freebsd.org) Authentication-Results: mx1.freebsd.org; none Received: from tom.home (kib@localhost [127.0.0.1] (may be forged)) by kib.kiev.ua (8.18.1/8.18.1) with ESMTP id 4AQDc4Ii097783; Tue, 26 Nov 2024 15:38:07 +0200 (EET) (envelope-from kib@freebsd.org) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 4AQDc4Ii097783 Received: (from kostik@localhost) by tom.home (8.18.1/8.18.1/Submit) id 4AQDc4cf097782; Tue, 26 Nov 2024 15:38:04 +0200 (EET) (envelope-from kib@freebsd.org) X-Authentication-Warning: tom.home: kostik set sender to kib@freebsd.org using -f Date: Tue, 26 Nov 2024 15:38:04 +0200 From: Konstantin Belousov To: Dimitry Andric Cc: Dag-Erling =?utf-8?B?U23DuHJncmF2?= , Mark Millard , "jah@freebsd.org" , dougm@freebsd.org, Alan Somers , Mark Johnston , FreeBSD Current , Guido Falsi , Yasuhiro Kimura , ports@freebsd.org Subject: Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros] Message-ID: References: <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> <0690CFB1-6A6D-4B63-916C-BAB7F6256000@yahoo.com> <3660625A-0EE8-40DA-A248-EC18C734718C@yahoo.com> <865xoa2t6f.fsf@ltc.des.dev> <69A2E921-F5E3-40D2-977D-0964EE27349A@FreeBSD.org> <4AE5B316-D7EB-4290-8D52-7FBF244EA7A4@FreeBSD.org> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <4AE5B316-D7EB-4290-8D52-7FBF244EA7A4@FreeBSD.org> X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=4.0.1 X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on tom.home X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US] X-Rspamd-Queue-Id: 4XyNv13hslz4PZ2 X-Spamd-Bar: ---- On Tue, Nov 26, 2024 at 01:58:19PM +0100, Dimitry Andric wrote: > On 26 Nov 2024, at 13:32, Dimitry Andric wrote: > > > > On 26 Nov 2024, at 11:19, Dag-Erling Smørgrav wrote: > >> > >> Mark Millard writes: > >>> From inside a bulk -i where I did a manual make command > >>> after it built and installed libsass.so.1.0.0 . The > >>> manual make produced a /wrkdirs/ : > >>> [...] > >>> So the original creation looks okay. But . . . > >>> [...] > >>> So: The later, staged copy is a bad copy. Both are in the > >>> tmpfs. So copying to the staging area makes a corrupted > >>> copy inside the same tmpfs. After that, further copies of > >>> staging's bad copy can be expected to be messed up. > >> > >> This and the fact that it happens on 14 and 15 but not on 13 strongly > >> suggests an issue wth `copy_file_range(2)`, since `install(1)` in 14 and > >> 15 (but not in 13) now uses `copy_file_range(2)` if at all possible. > >> > >> My educated guess is that hole detection doesn't work reliably for files > >> that have had holes filled while memory-mapped, so `copy_file_range(2)` > >> thinks there is a hole where there isn't one and skips some of the data > >> when `install(1)` uses it to copy the library from `${WRKSRC}` to > >> `${STAGEDIR}`. This may or may not be specific to tmpfs. > >> > >> You may want to try applying the attached patch to your FreeBSD 14 and > >> 15 jails. It prevents `cp(1)` and `install(1)` from trying to use > >> `copy_file_range(2)`. > > > > Yes, tmpfs is indeed the culprit (or at least involved). I have had USE_TMPFS=localbase in my poudriere.conf for a long time, since otherwise my build machine would run out of memory very quickly, so I didn't encounter any issues. > > > > Now I changed it to USE_TMPFS=yes, rebuilt only textproc/libsass and textproc/sassc, and then after reinstalling those packages: > > > > $ /usr/local/bin/sassc > > Segmentation fault > > And after applying Dag-Erling's patch to disable copy_file_range for cp and install, it works correctly again. So indeed there might be an issue in tmpfs seeking for data. Could you try the following? commit f4b848946a131dab260b44eab2cfabceb82bee0c Author: Konstantin Belousov Date: Tue Nov 26 15:34:56 2024 +0200 tmpfs: do not skip pages searching for data If the iterator finds invalid page at the requested pindex in swap_pager_seek_data(), the current code only looks at the swap blocks to search for data. This is not correct, valid pages may appear at the higher indexes still. diff --git a/sys/vm/swap_pager.c b/sys/vm/swap_pager.c index db925f4ae7f6..390b2c10d680 100644 --- a/sys/vm/swap_pager.c +++ b/sys/vm/swap_pager.c @@ -2503,12 +2503,9 @@ swap_pager_seek_data(vm_object_t object, vm_pindex_t pindex) VM_OBJECT_ASSERT_RLOCKED(object); vm_page_iter_init(&pages, object); m = vm_page_iter_lookup_ge(&pages, pindex); - if (m != NULL) { - if (!vm_page_any_valid(m)) - m = NULL; - else if (pages.index == pindex) - return (pages.index); - } + if (m != NULL && pages.index == pindex) + return (pages.index); + swblk_iter_init_only(&blks, object); swap_index = swap_pager_iter_find_least(&blks, pindex); if (swap_index == pindex) From nobody Tue Nov 26 15:52:47 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XyRtS3mPDz5fdlD for ; Tue, 26 Nov 2024 15:53:04 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic303-24.consmr.mail.gq1.yahoo.com (sonic303-24.consmr.mail.gq1.yahoo.com [98.137.64.205]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XyRtS0vTGz4kLD for ; Tue, 26 Nov 2024 15:53:04 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732636381; bh=9W4SS+9c+VUA6jT8z6ziXEi2kIsWjHHqFWnZq/IVqqw=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=dw6HXWIMJGpEo1R2neesRul2nOs/811ouqMiPngtJ+EwrQzUyLP5CYiCKM/XwHmeC33IZJcOQNlxBR6DgQ6AvDXVYmTjS70I7tRSq0T5rt+RPvVtUZdV/2jMf78sEpVfCzWFjePFwBcvdoriBVrW1PbXcp+pa3IeXDtxaDP19aTw2QrPCpRNg1LEGTaextlNIf3davRF6RWx99WQ6JVWx/mZPzWX0oJqOiKL5LS+Ipnaq6diUOiVFMraOieq2FYt731fI9XE/07MeqFHjuO4lknfwIswdhbp2RSTthl/ClVRdpjXvntsICCB3BW5oZ6wGZp5pxeAy8h+AngnecmTtQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732636381; bh=0mcoOQ84cHq6yZ6dJGtkcSkqCsAcGyPTWkbV//P7PL2=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=dr13rgFFxYNoZUR8AYmMaB8enGirZ//8Yge8ns3X+5KdLfVC2V4t+M0VHNPZK1Y94tBAx4zouDCzIvcg2D5g5VAQvivsBsHS8MCkCkntKet0CijxetY1gEe4Ta27uofOdb/ZiBlztNqPoKCObgN4ZvY8F6kV4ffQH3Ia3Ysk7JKDuPYBbsWZ8bIMVrwQzR/BYFfRKaQPEL9rJu61zxnQ5CuNK6g23VjiXMPh/rsXH5smC/SyoKvszyNhF00uyuqZJbCt7G3KMF58Y6RMl4dLS+Xl4nxfnJFgSRES6g24o9dEYao/F3T94X1QjHbTcgIgqbCUWc4NJnHxo7QKdFDeqw== X-YMail-OSG: ZelxJlkVM1nTpVsn2IKzL4jn_doQarWh94PSAF7Qvky.GFGw2UDMFNQs2xjjo4B 1P8qOZApHmIBY6rkUs2Ms.AswikF6ZesgHbhqXZE3OyUwIkIjeM5gnAKJfnIqIgAHE9IbUFokR.j yeiag1TmiBG0JSqkM9_VctIFVUXUXgA7ChnYb6pHVEWTWyX5uP3QRfhcNokbHxlPtUVTcj7Tmxpg vT657EUOOrMANtXWCHDE3vz8FXkPhaPjXsCNcNUOaj0jASOUikiCAWirbcav5XwzJAn8xtdmQcJL 1iz2fqWSP3EmyM5TKYkKyLeWkjYzaO129hq48fdAgWPm15AdaKiuokW873be9viQfs2___yROMb0 NPfa6cwqH8udg3xvSppP8JocPfjtuca1WNFMmP02Q4503nXezwNEVG4MGWrQkB_O6SHBJgYtUdvZ 3FLeNeVWv2Ppi1DmszecmgkH_VsPSArajD6EXq5mrjJqMSZJckX5ME4JPN6ISmWEtwodw2PYcv.D YsNtJ0rFfo_A5Q.4JWVuh_Zqztmne7eaiSYIAeUAIdQ.VZhMbITmL0Wbi1b9IsOIaFjqKLxEHoD2 6NOH2QsxLR0sds9_2Oc7KB_NSlK3l9rn0YMy_SPOhmqKI8sG3HF9EQ2mXLgnob92qziIEE3_C2Ip z_s.Fyy._kQNZQ4piGBGoTsLkFU3fGlWqdYm7aGPj8HdVa4DydnHHlRKgXsEm1kCHGFRsQRIPmr7 Ee0cHY7ucozMXfj4y536C9eMhifrT.C8LJu_wNoJjO.Tw1.L327UOeI4JxZWSkgjuzyPH7c5XXgM Ek.KjghRu.mnnibkp1AjA.ff0ArMlqBpARNvo9uVVtyYyRU7IrF01lxanFVTV62O5Dojfswdekqu 1Sm7TP8Y1qLWK5S25WK_1pN_iNHikdIXb5IxGc4t04XgnHKuPx.uSbqssb35peEoZqDrB2ugF5EB wqIYV9pVEhiE4vI_EgVOJhFIdFigQ3HnvmP3szWX1gpX6RYTTUvGDTSW5IJyIB._qtiwYS47j.G1 a43BGjO3JZQBgZgFuL5JvDD2IszqcWaD3lNSrgoU987ZEXXtAw2LMGIfq_9gft.kmEZMZ_vfoVhg sKJgMvHth5E64dS1LERI05YjRUuuvuZN7h1UXnaLFuJIpPsGOCFcLPA4HeZTmxsSpmiP.CyuytfS YcSxyqpUgT2m3O9SyvL9cPK2D9FPX.SfDDAKaOnzlxAZ1E8euwg3Aubp34Zf8OAeIrGjgZBo0szf SLJz.kxOPUKpfMPR7qQ6rlGvs4qju8mW5zB_79gfc2RAPhIRxbqBWCq8TMpVhx33RVs9p_GvQE0l XE9FgxtSF7JVWvqfq98CXD.w940wgJJfy5wpCpEJYkvQmqmg3smVZZIm2o6snZXhHn_qayqvrKRd ZisupVYqey1cxR.ha0d1cPycqnwWAxc8E.fnncOYGeqH47zEFkPQl_YjQPC35SsVXfDBA8uE4VhN U6yZNqGBvyYhGFQG9.9ADdFZou7BTz3qTP3H9D6rCPFrCLqk54HbpK93DzplQP7Pt7RoXnnNvxl6 x4M.PpYkdUoVg5fbP_gY8.J2GY2yDEQJ03dvtwM0BRlVqMSDGSxQRC5bF6NbtDJCPx.FdcE4cUWc qSkUwZh5C3QATeqInLRBqeZ_o42agyzyGLdxIC2p3oRpO4EDTrGokM7rsTwq8c2xwfpZEZCLUPob kGWkYwidWduJtdaM5YvuZBGtabvMcjMzfVS8y.QqYxieDtaI_r2PLh99bhJhoZKQVTZiE5SEYAG1 7E1gJ8gIjLzaUmnH1WHkIFuc8LN5fKwELvpDrp0ksBkV2l6ZblchkCAYN2AWr_nRWHobytmj0vhZ vWbeW1L2AIKTV.52O3V0cwt04AE.snZiAFsXrG.IML_ij4kwJ1HI8sHXcy56uhgalOeRs0t.b7Vt lERidbT.kHentHCbJ09OdY.HhuNdTgxy_l_6SlTiH8iY.C0gxYnXRoc6_qZ7yyYP8VD8bFgsTiuj e3znxwdzXvRb1tbkifsXprHm9j8h5FiLtPNHc6Y9JSjnGWmyT0H2aJSZhRsB0Nlr42uh0VZ96A5c N8_8Y8.tQw7aDVOYOPjoOPIIjeUXykUUob2YNvGX2QsCaz3uOH7cuU4RjA5XElY_g7cdcEYX0UK4 PNKYKEvwyPmjogD4yI.ouLdv14A3MTgObgCMBq8MkWWpbLI_4VrVUq4Jkdzr.mOPUylmxVqQeyUo Nr1.67WCOM7S_Qcspde1xU0QEosKsYhM1zUZ.9Ik5Y7.3ir12eEWj18X2J5Fs X-Sonic-MF: X-Sonic-ID: 14b2793b-3458-4ee2-acb9-8106e44e98f9 Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.gq1.yahoo.com with HTTP; Tue, 26 Nov 2024 15:53:01 +0000 Received: by hermes--production-gq1-5dd4b47f46-pfhh2 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 5db59a8d4adc5a7b76cc431e6aaf5eb1; Tue, 26 Nov 2024 15:52:58 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51.11.1\)) Subject: Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros] From: Mark Millard In-Reply-To: Date: Tue, 26 Nov 2024 07:52:47 -0800 Cc: Dimitry Andric , =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , "jah@freebsd.org" , dougm@freebsd.org, Alan Somers , Mark Johnston , FreeBSD Current , Guido Falsi , Yasuhiro Kimura , ports@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <33D56E3E-6476-48E8-B115-B906629B8AF5@yahoo.com> References: <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> <0690CFB1-6A6D-4B63-916C-BAB7F6256000@yahoo.com> <3660625A-0EE8-40DA-A248-EC18C734718C@yahoo.com> <865xoa2t6f.fsf@ltc.des.dev> <69A2E921-F5E3-40D2-977D-0964EE27349A@FreeBSD.org> <4AE5B316-D7EB-4290-8D52-7FBF244EA7A4@FreeBSD.org> To: Konstantin Belousov X-Mailer: Apple Mail (2.3776.700.51.11.1) X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4XyRtS0vTGz4kLD X-Spamd-Bar: ---- On Nov 26, 2024, at 05:38, Konstantin Belousov wrote: > On Tue, Nov 26, 2024 at 01:58:19PM +0100, Dimitry Andric wrote: >> On 26 Nov 2024, at 13:32, Dimitry Andric wrote: >>>=20 >>> On 26 Nov 2024, at 11:19, Dag-Erling Sm=C3=B8rgrav = wrote: >>>>=20 >>>> Mark Millard writes: >>>>> =46rom inside a bulk -i where I did a manual make command >>>>> after it built and installed libsass.so.1.0.0 . The >>>>> manual make produced a /wrkdirs/ : >>>>> [...] >>>>> So the original creation looks okay. But . . . >>>>> [...] >>>>> So: The later, staged copy is a bad copy. Both are in the >>>>> tmpfs. So copying to the staging area makes a corrupted >>>>> copy inside the same tmpfs. After that, further copies of >>>>> staging's bad copy can be expected to be messed up. >>>>=20 >>>> This and the fact that it happens on 14 and 15 but not on 13 = strongly >>>> suggests an issue wth `copy_file_range(2)`, since `install(1)` in = 14 and >>>> 15 (but not in 13) now uses `copy_file_range(2)` if at all = possible. >>>>=20 >>>> My educated guess is that hole detection doesn't work reliably for = files >>>> that have had holes filled while memory-mapped, so = `copy_file_range(2)` >>>> thinks there is a hole where there isn't one and skips some of the = data >>>> when `install(1)` uses it to copy the library from `${WRKSRC}` to >>>> `${STAGEDIR}`. This may or may not be specific to tmpfs. >>>>=20 >>>> You may want to try applying the attached patch to your FreeBSD 14 = and >>>> 15 jails. It prevents `cp(1)` and `install(1)` from trying to use >>>> `copy_file_range(2)`. >>>=20 >>> Yes, tmpfs is indeed the culprit (or at least involved). I have had = USE_TMPFS=3Dlocalbase in my poudriere.conf for a long time, since = otherwise my build machine would run out of memory very quickly, so I = didn't encounter any issues. >>>=20 >>> Now I changed it to USE_TMPFS=3Dyes, rebuilt only textproc/libsass = and textproc/sassc, and then after reinstalling those packages: >>>=20 >>> $ /usr/local/bin/sassc >>> Segmentation fault >>=20 >> And after applying Dag-Erling's patch to disable copy_file_range for = cp and install, it works correctly again. >=20 > So indeed there might be an issue in tmpfs seeking for data. Could = you try > the following? >=20 > commit f4b848946a131dab260b44eab2cfabceb82bee0c > Author: Konstantin Belousov > Date: Tue Nov 26 15:34:56 2024 +0200 >=20 > tmpfs: do not skip pages searching for data >=20 > If the iterator finds invalid page at the requested pindex in > swap_pager_seek_data(), the current code only looks at the swap = blocks > to search for data. This is not correct, valid pages may appear at = the > higher indexes still. >=20 > diff --git a/sys/vm/swap_pager.c b/sys/vm/swap_pager.c > index db925f4ae7f6..390b2c10d680 100644 > --- a/sys/vm/swap_pager.c > +++ b/sys/vm/swap_pager.c > @@ -2503,12 +2503,9 @@ swap_pager_seek_data(vm_object_t object, = vm_pindex_t pindex) > VM_OBJECT_ASSERT_RLOCKED(object); > vm_page_iter_init(&pages, object); > m =3D vm_page_iter_lookup_ge(&pages, pindex); > - if (m !=3D NULL) { > - if (!vm_page_any_valid(m)) > - m =3D NULL; > - else if (pages.index =3D=3D pindex) > - return (pages.index); > - } > + if (m !=3D NULL && pages.index =3D=3D pindex) > + return (pages.index); > + > swblk_iter_init_only(&blks, object); > swap_index =3D swap_pager_iter_find_least(&blks, pindex); > if (swap_index =3D=3D pindex) Not sufficient, unfortunately . . . I patched what I've been running and rebooted into: # uname -apKU FreeBSD 7950X3D-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT #152 = main-n273696-43e045c1733d-dirty: Tue Nov 26 07:21:27 PST 2024 = root@7950X3D-ZFS:/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64= .amd64/sys/GENERIC-NODBG amd64 amd64 1500027 1500027 Note: 43e045c1733d is from 2024-Nov-18 . I then built libsass : [00:00:02] [01] [00:00:00] Building textproc/libsass | libsass-3.6.6 [00:00:20] [01] [00:00:18] Finished textproc/libsass | libsass-3.6.6: = Success ending TMPFS: 3.42 GiB I then installed it, resulting in: # pkg info libsass libsass-3.6.6 Name : libsass Version : 3.6.6 Installed on : Tue Nov 26 07:33:15 2024 PST Origin : textproc/libsass Architecture : FreeBSD:15:amd64 Prefix : /usr/local Categories : textproc Licenses : MIT Maintainer : nivit@FreeBSD.org WWW : https://sass-lang.com/libsass Comment : C/C++ implementation of a Sass compiler Shared Libs provided: libsass.so.1 Annotations : FreeBSD_version: 1500027 build_timestamp: 2024-11-26T15:32:33+0000 built_by : poudriere-git-3.4.99.20240811 . . . libsass.so.1.0.0 still has .got.plt starting with (this time): 2bed60 00000000 00000000 00000000 00000000 ................ 2bed70 00000000 00000000 00000000 00000000 ................ 2bed80 00000000 00000000 00000000 00000000 ................ 2bed90 00000000 00000000 00000000 00000000 ................ . . . 2bffc0 00000000 00000000 00000000 00000000 ................ 2bffd0 00000000 00000000 00000000 00000000 ................ 2bffe0 00000000 00000000 00000000 00000000 ................ 2bfff0 00000000 00000000 00000000 00000000 ................ 2c0000 96cb2a00 00000000 a6cb2a00 00000000 ..*.......*..... 2c0010 b6cb2a00 00000000 c6cb2a00 00000000 ..*.......*..... 2c0020 d6cb2a00 00000000 e6cb2a00 00000000 ..*.......*..... 2c0030 f6cb2a00 00000000 06cc2a00 00000000 ..*.......*..... . . . And still results in: # sassc Segmentation fault (core dumped) =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Tue Nov 26 17:29:23 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XyV1g4DJmz5fkcx; Tue, 26 Nov 2024 17:29:27 +0000 (UTC) (envelope-from unkadoug@gmail.com) Received: from mail-oi1-x22d.google.com (mail-oi1-x22d.google.com [IPv6:2607:f8b0:4864:20::22d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XyV1g2KTGz4x4Q; Tue, 26 Nov 2024 17:29:27 +0000 (UTC) (envelope-from unkadoug@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-oi1-x22d.google.com with SMTP id 5614622812f47-3ea696c4dcaso299027b6e.2; Tue, 26 Nov 2024 09:29:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1732642165; x=1733246965; darn=freebsd.org; h=in-reply-to:references:cc:to:content-language:subject:user-agent :mime-version:date:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=jhbT+hQoxcEr8V7GXf6nJnWgsO+6vE+8Vte1YuFS8C0=; b=ZDu+rntCMhaR/cAqZLBzMuBmbVHkcJjdaPpwjWs+3a8IVVHK86LW505+j6Ud88QYKv bLil4mOKYHWrJO/C85vZDRBzQQiFVP2Ah+C8GOTJ+USpjC+Vld3UkEmbrf82EuryxAHa 05YwFlfqPKb9vB//xjIi9R3I7NABEAX/eAWMYpNHT1mfj2GYw64pnVRXM8xbqQF54giQ LhoZ+RJkuPZs0tb+Nogil2hNlQPy3k0UVv+dAxdMIR8tnWckdsQ3zgKYI1JZmMWEvCG+ Q4xtgMLdktpQ+mpq8ysd1NQgvbV5EmwiMmSL3vP7vVYi2qxESyiN0fZjW4FfFFaT8eY5 qbbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732642165; x=1733246965; h=in-reply-to:references:cc:to:content-language:subject:user-agent :mime-version:date:message-id:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jhbT+hQoxcEr8V7GXf6nJnWgsO+6vE+8Vte1YuFS8C0=; b=cd/niDHXCQnIW/4qQ5GaR37X39ej988tweMp/T/R7FJuTWTZAB8f72dE5r1PMEvLrz VI2y3l1NAnNDQx8xEuvP6UK6he25a99WH0HVzLQjCR9pnkAvEqXDIu7CB/lVc4Icb+Us 82TfLxZkvT3kVP8392CkFrdFwLkto9KzU7jnhOz8wdDnHDWMeTkvVCtPtZwbtrJzhAPB QvtSErHsqrGI8I9OqTK7h6m6rCEf9qjp7LOhLGLCUu6qswnnuikV8VOfqpAhPzf+IfZk HQrtZp41kwemZcmCBSqG8PpXX0XodlgaIsEbYqi8WiTp7EdOtKqS6VVO/iJBuXVBC1Ga OW4g== X-Forwarded-Encrypted: i=1; AJvYcCU9c7kXYN8iZnkGIqoKueG4RFYfnXYikRPrbq8ND9AJRNRGfCNwmdRuQYFFeMeybwkpkqog@freebsd.org, AJvYcCUN/HTPxnoLkroh5Z9CY6LyL7GeFOdtDIK75MHLePsJWdcnQa7sWZ3VLB05vHt9US6RSG7cmQk=@freebsd.org, AJvYcCUlec/zvCon3WiQiugOW63DPUe7nFQ122jnElZyYt0vX3uxrTqn8C8Moh4z/LbxICkAScMKW1j16C7VCc3pC65R@freebsd.org, AJvYcCUqRRDuA1J3tjZ6Eop2ukPyoaGnnEDVV0w4QfjsQR+4kwypG56r1tO7HkgQ+2WCqYabDxWEGA==@freebsd.org, AJvYcCWbzCO5hCXZ13nAr3nkIZ6hvphyz7t+qQToS4sNdzrpde8YtwaWoIiKjgf/D2Dz67XPndaqqLc=@freebsd.org, AJvYcCWvalVXjuPQ0tIHUQRa7r2s7xMu/DxGZCUeZvj3UOIdzlMsTnI+bDdTTpw2IC5kFGVW3lJNlRY/@freebsd.org, AJvYcCXetwrs+Ru9JxOxb4mpqYhcDbAWc0V/0AvzxWD4sli0P0X5db+j5XEst+YSmnySpfZz4HuN@freebsd.org, AJvYcCXjb1INcBcs82qQeqdcKOI9He9q7wlMr2A29qDZcAUwXQKG4/H2J4hSiDadtcIWeeYk+UoR@freebsd.org X-Gm-Message-State: AOJu0YxvPwvPYeH6In0Fp+1hEC0HWahJwx0AO9czwXt04CzuG1nGOEqO sUSX4ioY1636N4kzymVqWiw6TY+nUbT2qsJKZrbJfze+SX/g54TGKHeOs6CM X-Gm-Gg: ASbGnctpes4snoFK7QwUcw8/p2yyPIQorkty3qYBUSbQsmTBggnCPv6Ji9Ajpr28kj2 wfq7PIMtgA4VroF7Xf1o5o0u8vRl7Nmd3mC2smKdfei5seGEReeL0WQU5xuyXn/4JFufkaS8xnp gU/f43nT+dzwD956WDMUZ6JtPTa6oXCcZJM4+jbo35SweyksIRwymG+gbYn4ufIXL1tYYr+jTjn g9xFITNABZNWqZ1Lus6Zazk0NfOK+MceIDyh207Z1WKnFDntlsYpf1oxh3AAy0kMnqgalNTfEqp VnJbXI+nBg3h6MuW+yAOKqWY0T9+3Sg3 X-Google-Smtp-Source: AGHT+IH4dj8GoAtsSCgESZPL6BULKHlafkDX8sNE/mZWmccDBf2m0hDyJFg9LjvRGg7bq7lq3FYqrg== X-Received: by 2002:a05:6808:2025:b0:3ea:4140:e7dd with SMTP id 5614622812f47-3ea6dd48c3emr128702b6e.32.1732642165445; Tue, 26 Nov 2024 09:29:25 -0800 (PST) Received: from [108.254.203.202] (108-254-203-202.lightspeed.hstntx.sbcglobal.net. [108.254.203.202]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3ea542d4033sm843593b6e.35.2024.11.26.09.29.24 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 26 Nov 2024 09:29:24 -0800 (PST) From: Doug Moore X-Google-Original-From: Doug Moore Content-Type: multipart/mixed; boundary="------------dLvnPAVEPNGf2FLWnmem0aRA" Message-ID: Date: Tue, 26 Nov 2024 11:29:23 -0600 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros] Content-Language: en-US To: Mark Millard , Konstantin Belousov Cc: Dimitry Andric , =?UTF-8?Q?Dag-Erling_Sm=c3=b8rgrav?= , "jah@freebsd.org" , Alan Somers , Mark Johnston , FreeBSD Current , Guido Falsi , Yasuhiro Kimura , ports@freebsd.org References: <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> <0690CFB1-6A6D-4B63-916C-BAB7F6256000@yahoo.com> <3660625A-0EE8-40DA-A248-EC18C734718C@yahoo.com> <865xoa2t6f.fsf@ltc.des.dev> <69A2E921-F5E3-40D2-977D-0964EE27349A@FreeBSD.org> <4AE5B316-D7EB-4290-8D52-7FBF244EA7A4@FreeBSD.org> <33D56E3E-6476-48E8-B115-B906629B8AF5@yahoo.com> In-Reply-To: <33D56E3E-6476-48E8-B115-B906629B8AF5@yahoo.com> X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4XyV1g2KTGz4x4Q X-Spamd-Bar: ---- This is a multi-part message in MIME format. --------------dLvnPAVEPNGf2FLWnmem0aRA Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit I think @kib has found the source of the problem.  I've attached an attempt to fix it. On 11/26/24 09:52, Mark Millard wrote: > On Nov 26, 2024, at 05:38, Konstantin Belousov wrote: > >> On Tue, Nov 26, 2024 at 01:58:19PM +0100, Dimitry Andric wrote: >>> On 26 Nov 2024, at 13:32, Dimitry Andric wrote: >>>> On 26 Nov 2024, at 11:19, Dag-Erling Smørgrav wrote: >>>>> Mark Millard writes: >>>>>> From inside a bulk -i where I did a manual make command >>>>>> after it built and installed libsass.so.1.0.0 . The >>>>>> manual make produced a /wrkdirs/ : >>>>>> [...] >>>>>> So the original creation looks okay. But . . . >>>>>> [...] >>>>>> So: The later, staged copy is a bad copy. Both are in the >>>>>> tmpfs. So copying to the staging area makes a corrupted >>>>>> copy inside the same tmpfs. After that, further copies of >>>>>> staging's bad copy can be expected to be messed up. >>>>> This and the fact that it happens on 14 and 15 but not on 13 strongly >>>>> suggests an issue wth `copy_file_range(2)`, since `install(1)` in 14 and >>>>> 15 (but not in 13) now uses `copy_file_range(2)` if at all possible. >>>>> >>>>> My educated guess is that hole detection doesn't work reliably for files >>>>> that have had holes filled while memory-mapped, so `copy_file_range(2)` >>>>> thinks there is a hole where there isn't one and skips some of the data >>>>> when `install(1)` uses it to copy the library from `${WRKSRC}` to >>>>> `${STAGEDIR}`. This may or may not be specific to tmpfs. >>>>> >>>>> You may want to try applying the attached patch to your FreeBSD 14 and >>>>> 15 jails. It prevents `cp(1)` and `install(1)` from trying to use >>>>> `copy_file_range(2)`. >>>> Yes, tmpfs is indeed the culprit (or at least involved). I have had USE_TMPFS=localbase in my poudriere.conf for a long time, since otherwise my build machine would run out of memory very quickly, so I didn't encounter any issues. >>>> >>>> Now I changed it to USE_TMPFS=yes, rebuilt only textproc/libsass and textproc/sassc, and then after reinstalling those packages: >>>> >>>> $ /usr/local/bin/sassc >>>> Segmentation fault >>> And after applying Dag-Erling's patch to disable copy_file_range for cp and install, it works correctly again. >> So indeed there might be an issue in tmpfs seeking for data. Could you try >> the following? >> >> commit f4b848946a131dab260b44eab2cfabceb82bee0c >> Author: Konstantin Belousov >> Date: Tue Nov 26 15:34:56 2024 +0200 >> >> tmpfs: do not skip pages searching for data >> >> If the iterator finds invalid page at the requested pindex in >> swap_pager_seek_data(), the current code only looks at the swap blocks >> to search for data. This is not correct, valid pages may appear at the >> higher indexes still. >> >> diff --git a/sys/vm/swap_pager.c b/sys/vm/swap_pager.c >> index db925f4ae7f6..390b2c10d680 100644 >> --- a/sys/vm/swap_pager.c >> +++ b/sys/vm/swap_pager.c >> @@ -2503,12 +2503,9 @@ swap_pager_seek_data(vm_object_t object, vm_pindex_t pindex) >> VM_OBJECT_ASSERT_RLOCKED(object); >> vm_page_iter_init(&pages, object); >> m = vm_page_iter_lookup_ge(&pages, pindex); >> - if (m != NULL) { >> - if (!vm_page_any_valid(m)) >> - m = NULL; >> - else if (pages.index == pindex) >> - return (pages.index); >> - } >> + if (m != NULL && pages.index == pindex) >> + return (pages.index); >> + >> swblk_iter_init_only(&blks, object); >> swap_index = swap_pager_iter_find_least(&blks, pindex); >> if (swap_index == pindex) > Not sufficient, unfortunately . . . > > I patched what I've been running and rebooted into: > > # uname -apKU > FreeBSD 7950X3D-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT #152 main-n273696-43e045c1733d-dirty: Tue Nov 26 07:21:27 PST 2024 root@7950X3D-ZFS:/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG amd64 amd64 1500027 1500027 > > Note: 43e045c1733d is from 2024-Nov-18 . > > I then built libsass : > > [00:00:02] [01] [00:00:00] Building textproc/libsass | libsass-3.6.6 > [00:00:20] [01] [00:00:18] Finished textproc/libsass | libsass-3.6.6: Success ending TMPFS: 3.42 GiB > > I then installed it, resulting in: > > # pkg info libsass > libsass-3.6.6 > Name : libsass > Version : 3.6.6 > Installed on : Tue Nov 26 07:33:15 2024 PST > Origin : textproc/libsass > Architecture : FreeBSD:15:amd64 > Prefix : /usr/local > Categories : textproc > Licenses : MIT > Maintainer : nivit@FreeBSD.org > WWW : https://sass-lang.com/libsass > Comment : C/C++ implementation of a Sass compiler > Shared Libs provided: > libsass.so.1 > Annotations : > FreeBSD_version: 1500027 > build_timestamp: 2024-11-26T15:32:33+0000 > built_by : poudriere-git-3.4.99.20240811 > . . . > > libsass.so.1.0.0 still has .got.plt starting with (this time): > > 2bed60 00000000 00000000 00000000 00000000 ................ > 2bed70 00000000 00000000 00000000 00000000 ................ > 2bed80 00000000 00000000 00000000 00000000 ................ > 2bed90 00000000 00000000 00000000 00000000 ................ > . . . > 2bffc0 00000000 00000000 00000000 00000000 ................ > 2bffd0 00000000 00000000 00000000 00000000 ................ > 2bffe0 00000000 00000000 00000000 00000000 ................ > 2bfff0 00000000 00000000 00000000 00000000 ................ > 2c0000 96cb2a00 00000000 a6cb2a00 00000000 ..*.......*..... > 2c0010 b6cb2a00 00000000 c6cb2a00 00000000 ..*.......*..... > 2c0020 d6cb2a00 00000000 e6cb2a00 00000000 ..*.......*..... > 2c0030 f6cb2a00 00000000 06cc2a00 00000000 ..*.......*..... > . . . > > And still results in: > > # sassc > Segmentation fault (core dumped) > > > > === > Mark Millard > marklmi at yahoo.com > --------------dLvnPAVEPNGf2FLWnmem0aRA Content-Type: text/x-patch; charset=UTF-8; name="seek_data_fix.patch" Content-Disposition: attachment; filename="seek_data_fix.patch" Content-Transfer-Encoding: base64 ZGlmZiAtLWdpdCBhL3N5cy92bS9zd2FwX3BhZ2VyLmMgYi9zeXMvdm0vc3dhcF9wYWdlci5j CmluZGV4IGRiOTI1ZjRhZTdmNi4uM2QwMmYzNjVjYWQ5IDEwMDY0NAotLS0gYS9zeXMvdm0v c3dhcF9wYWdlci5jCisrKyBiL3N5cy92bS9zd2FwX3BhZ2VyLmMKQEAgLTI1MDMsMjYgKzI1 MDMsMjMgQEAgc3dhcF9wYWdlcl9zZWVrX2RhdGEodm1fb2JqZWN0X3Qgb2JqZWN0LCB2bV9w aW5kZXhfdCBwaW5kZXgpCiAJVk1fT0JKRUNUX0FTU0VSVF9STE9DS0VEKG9iamVjdCk7CiAJ dm1fcGFnZV9pdGVyX2luaXQoJnBhZ2VzLCBvYmplY3QpOwogCW0gPSB2bV9wYWdlX2l0ZXJf bG9va3VwX2dlKCZwYWdlcywgcGluZGV4KTsKLQlpZiAobSAhPSBOVUxMKSB7Ci0JCWlmICgh dm1fcGFnZV9hbnlfdmFsaWQobSkpCi0JCQltID0gTlVMTDsKLQkJZWxzZSBpZiAocGFnZXMu aW5kZXggPT0gcGluZGV4KQotCQkJcmV0dXJuIChwYWdlcy5pbmRleCk7Ci0JfQorCWlmICht ICE9IE5VTEwgJiYgcGFnZXMuaW5kZXggPT0gcGluZGV4ICYmIHZtX3BhZ2VfYW55X3ZhbGlk KG0pKQorCQlyZXR1cm4gKHBhZ2VzLmluZGV4KTsKIAlzd2Jsa19pdGVyX2luaXRfb25seSgm Ymxrcywgb2JqZWN0KTsKIAlzd2FwX2luZGV4ID0gc3dhcF9wYWdlcl9pdGVyX2ZpbmRfbGVh c3QoJmJsa3MsIHBpbmRleCk7CiAJaWYgKHN3YXBfaW5kZXggPT0gcGluZGV4KQogCQlyZXR1 cm4gKHN3YXBfaW5kZXgpOwotCWlmIChzd2FwX2luZGV4ID09IE9CSl9NQVhfU0laRSkKLQkJ c3dhcF9pbmRleCA9IG9iamVjdC0+c2l6ZTsKLQlpZiAobSA9PSBOVUxMKQotCQlyZXR1cm4g KHN3YXBfaW5kZXgpOwogCi0Jd2hpbGUgKChtID0gdm1fcmFkaXhfaXRlcl9zdGVwKCZwYWdl cykpICE9IE5VTEwgJiYKLQkgICAgcGFnZXMuaW5kZXggPCBzd2FwX2luZGV4KSB7CisJLyoK KwkgKiBGaW5kIHRoZSBmaXJzdCByZXNpZGVudCBwYWdlIGFmdGVyIG0sIGJlZm9yZSBzd2Fw X2luZGV4LgorCSAqLworCXdoaWxlIChtICE9IE5VTEwgJiYgcGFnZXMuaW5kZXggPCBzd2Fw X2luZGV4KSB7CiAJCWlmICh2bV9wYWdlX2FueV92YWxpZChtKSkKIAkJCXJldHVybiAocGFn ZXMuaW5kZXgpOworCQltID0gdm1fcmFkaXhfaXRlcl9zdGVwKCZwYWdlcyk7CiAJfQorCWlm IChzd2FwX2luZGV4ID09IE9CSl9NQVhfU0laRSkKKwkJc3dhcF9pbmRleCA9IG9iamVjdC0+ c2l6ZTsKIAlyZXR1cm4gKHN3YXBfaW5kZXgpOwogfQogCg== --------------dLvnPAVEPNGf2FLWnmem0aRA-- From nobody Tue Nov 26 17:59:08 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XyVhC1Hbvz5dnNb for ; Tue, 26 Nov 2024 17:59:23 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic301-22.consmr.mail.gq1.yahoo.com (sonic301-22.consmr.mail.gq1.yahoo.com [98.137.64.148]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XyVhB5SHbz52GT for ; Tue, 26 Nov 2024 17:59:22 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732643961; bh=hrJigfJv2bPxK6GZurULYk3OWDMc4KIJq6s5m3SU3Kc=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=OLFZYBe8bPn0fpDDZ0ZxX64c03EPz5q0/iixEekCR+5/XO8iAjFsykltYBDX7DVsP7JMs1dISC2/M3YzyCMkKMZlAN2RihcYTN/3S+MYpZevmysYQKlekBxQ3l0xu7TX4QidpOBrqcEUX473Q1It3tW2EwC1uWdAYitufbHLe3HcM1mY5Ikcf6uC5B4qOO4O5JpGNFDxBsVHJ+9NgDvBa+VywzPNg1UdHl4eIeOc1QCTBdBbgmA5+LscaRtxcPY4xp4+AL54Wlv+CLZMShKBTB2vIPUatSu7lQc9VzEV/SOSQpXK2gR5HvanQO14IPvKFX8iLXlFqbIKawkLXUkheQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732643961; bh=iZmauvUjuErKk3iG+v6NiJTl2JVF91OHcwwcYWb8T/J=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=CTFNrn1s44As4dGLSjSMZf+m2Y+aWv7FJ//Tt07NY9W7CUsPyJ44LwKpVZjkZl79JZlwKF88Lz9OlntDhTYqWhs36XmRaqpmpKxS6TTUFzcPZinH8vT1YkGvBXcdqZyESo9ShCd41cI8IJjy3EIrjAG7qjDUIJu0k3MQ9DhCO+Ets5Q7JPFdUgQB+bYi5zSODZjGl2ib69TnziZL/P0qB6ZWFd/vCrRU6W9lv0PpFeS3uwr4OmiA7SUkyCDPrUzawpZeJNZMaH2wFXrZvIBvqXTKDp1vfxnQdetANhTWiKXmyhbArt5C3wjYJHIC/lZAorH+NU45LC0kVKa/pt3A6w== X-YMail-OSG: S1fAFHAVM1lZV7dUJgm8NyQXOAfzWdliHjCIgRYs5KJ4mtchikBX2cm_Veubu2l JGiyH5BFdNBai5xQo2DiLQm28cjCmR87eD.L3gjM.w69BP.OtYgef6EQcr.pvXR00tQmr60sDtEv UvHQ0MwLNX6i9KQLl1iYp1m1j2MaJi3g666UNhSdvrN4AlmvlZazlEVtQAEVAJ4ppdM02FY_DgU2 YsjTbGagf9P7zgfVNiyxmi7SwRrdbQhI6ZZzaqQ6TT4FUBg_2yZajXLcSdwl5f38Mxj_dQKGqJSr VEK7dHqJkRc2EO77w0YcA0a.PSSxXaqWAyfeDvLZILIZln99kBBxIOVkPn.DuNCzfqRQkQ125cAR 6vc1pjKox3BDj7hOk0Ma5skuiPtai4jwTFJFTJa6dI50omGAg_ZP5NXzv6xBhBtph_fasdutAvpt Z4EYdXL5AvMWmbwxXJIOTNGkq_H9726i_KHxGQY9zQ9sp.Dw35p.ABl95dxNnePfq1jsKRwrGWqU ZCjI.1Jt6HIUUFxRs.EF8rY6idPB_HhLdS6bNOz7vMB3sJ02XgxsInq34kdcPCYdT.fDYeIJoJeq iX6QL8.4S_QwDIiwx78Pv6kbVMjCVpNWNGVh4VuMA08.f7g3uD0oGeTwt1773tlrGXr2xUGntPnh hhlKRIQJ8qJQG0jOGOQlra4n2lgmB_E1hTYO3JAdZ.EWOOcaB5NJbOagNubWJPYG1MuQaBlUfi0U ccsalqlFuo8cjKMVe9knPmosPj0imxvDHzQY_.qSuM952Mwn5z8xvGWv3pK3t_KAqUwCZwgFxEE. Uxw0sS_L0bRCZaFfqgoIM9BIVT9UbaC7coX5JIjIGLqYTvUPDygTph.dIbhXUnr6hIjdWtIL8Sky jtVhIWZ3AFTQsT_AVGYOd0Yu3x2dtMbMLiqDnApcA_z4YnChiwDHN.qzVVBHOiSWZL8UTomJRW_W MgCCBurc4nppHi.bnyC08NAYCMxq61pWWbCwH0YoXwwkYDp9axWBtwlTjLQeeaEhZox..VA8L9NB LOfsrkOxupROKLuCACZRZeDWnf6ltaEEu3aU4N5T_LNUreLfSrJYZYGtkMPOQDZE_cDhRPkHsRwY FkaSkiBeyy.lw8LPYF35f.WRH2eWrD8ZSfvG6pR358XZ..pXV5Ck7mLsP4ao2K8GdFF3bLQAHDur 3AvzjwpLadBwd9ft4wPPRsJaVogaIzHCew7tZR2euiS.YTglYtRyI9ApYALA0HY_.KM6YcMDfWvO S493avF91YHJEu8iKmUAXlhFgumehWWc1OLumBJgMUSD1d1pSoWHMFHf085UDeSVFllhr55a7PeG K5cpLBPLYlajh1uEc.tuhqtw_j8FbX4wlt2sh7YXyOqzwb1zbo9rPu9cpMHYbSM3Law0YBv_coQA QGTCVM1rTo39u2yiWSMlcpF6Lac_9SvlCIjqXZQkrnyMY0Epg5IaaEqWOqU71AjZjqTWZwQ3ahhn DBcbMsAPs9DCcm3nPakCWghMKyEqYcagh0tDVuv.PcNFqioAlxNDjbgtfRgVeQCjPYUAQGdoMiCL sD4nlhUKYTi.pF96qrR2RK.p_6.KRWNlD0qq2888R2sCdUuRoaBl8Bd1ePXwuXyCjdi6pv54yRL5 fszeX.gNkGQtaX22AcHxV7s77DKNA8iHjf8iXFoRCEemEOFXbJMR5mzOX.9tcvTNi7u12YvH1oUX hEe9WDNOgwe0fQEAi8oew_a4aoN7dEiVt6EwjWhXjmbRAE7HWhKdCmbX_T4vLrognHFt4KFwRLi3 IogFB6pIcUYiHmv7XqKn8M.jaBs6uRoNisYGY2x0S5TijHRwJqIuypRwgaGBzSFkOOgH.IROo_Qt 8ZsDVHZ1j1j8s5zsUhZwNBRWOwLq7zQXPElAMRQ2J0q2txjP7Bf0hZkNtvIULPoZfnLcbdijM1j7 Eh6uYOO1gQiCPWAl7QJv1et__Q18Rne2uc_.GwttuLIOZLazyvcoPSfa0FP2lAwxgoPSj22y37Tf Ir9x8Pf5u83VxmCfS6Jxt39dN3odxzvRdrZvYYONBfhQIzOxcZRKSyBgPa_cQvgu1jnlKkAmfnb_ Eo__p.ZBJ_kYejxGOVlNrv4soUUeRPwQ.JhbDXOKeHcqBKEA9.avyhcv.zSGifquUYDI.VmbzIyp Gn0FcRMEzviAszZ27J2gvBl5ZwcDLNY_AX4EemMMizNZZb9AZuQsVHudfbPZ.lH6W07Y4C1Wpg3F EY6c68HQx8chcDc_Yz6jrzkbprW2f7GrImOVjjiFtj.aHUmwgqkK8X5BXQtCWvkmAPT7SnXFdisB bqSdETXpOaVG1ZsCfCMUjwGbdEsk2q8REqWKRKvAe X-Sonic-MF: X-Sonic-ID: 867300d6-66c1-4ac0-99a9-70e4835f0c19 Received: from sonic.gate.mail.ne1.yahoo.com by sonic301.consmr.mail.gq1.yahoo.com with HTTP; Tue, 26 Nov 2024 17:59:21 +0000 Received: by hermes--production-gq1-5dd4b47f46-zz6g6 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 96ab8f073a80e750fb8e02192beccda2; Tue, 26 Nov 2024 17:59:20 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51.11.1\)) Subject: Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros] From: Mark Millard In-Reply-To: Date: Tue, 26 Nov 2024 09:59:08 -0800 Cc: Konstantin Belousov , Dimitry Andric , =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , "jah@freebsd.org" , Alan Somers , Mark Johnston , FreeBSD Current , Guido Falsi , Yasuhiro Kimura , ports@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <5E0432D6-41D1-4A54-AA21-CCB5B8DC08E6@yahoo.com> References: <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> <0690CFB1-6A6D-4B63-916C-BAB7F6256000@yahoo.com> <3660625A-0EE8-40DA-A248-EC18C734718C@yahoo.com> <865xoa2t6f.fsf@ltc.des.dev> <69A2E921-F5E3-40D2-977D-0964EE27349A@FreeBSD.org> <4AE5B316-D7EB-4290-8D52-7FBF244EA7A4@FreeBSD.org> <33D56E3E-6476-48E8-B115-B906629B8AF5@yahoo.com> To: Doug Moore X-Mailer: Apple Mail (2.3776.700.51.11.1) X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4XyVhB5SHbz52GT X-Spamd-Bar: ---- On Nov 26, 2024, at 09:29, Doug Moore wrote: > I think @kib has found the source of the problem. I've attached an = attempt to fix it. That worked for what I'm testing. Following the same procedure but with the new patch content, .got.plt looks good at the beginning: 2bed60 78ba2b00 00000000 00000000 00000000 x.+............. 2bed70 00000000 00000000 86a62a00 00000000 ..........*..... 2bed80 96a62a00 00000000 a6a62a00 00000000 ..*.......*..... 2bed90 b6a62a00 00000000 c6a62a00 00000000 ..*.......*..... . . . And sassc no longer fails: # sassc Usage: sassc [options] [INPUT] [OUTPUT] Options: -s, --stdin Read input from standard input instead of an = input file. -t, --style NAME Output style. Can be: nested, expanded, = compact, compressed. -l, --line-numbers Emit comments showing original line numbers. --line-comments -I, --load-path PATH Set Sass import path. -P, --plugin-path PATH Set path to autoload plugins. -m, --sourcemap[=3DTYPE] Emit source map (auto or inline). -M, --omit-map-comment Omits the source map url comment. -p, --precision Set the precision for numbers. -a, --sass Treat input as indented syntax. -v, --version Display compiled versions. -h, --help Display this help message. > On 11/26/24 09:52, Mark Millard wrote: >> On Nov 26, 2024, at 05:38, Konstantin Belousov = wrote: >>=20 >>> On Tue, Nov 26, 2024 at 01:58:19PM +0100, Dimitry Andric wrote: >>>> On 26 Nov 2024, at 13:32, Dimitry Andric wrote: >>>>> On 26 Nov 2024, at 11:19, Dag-Erling Sm=C3=B8rgrav = wrote: >>>>>> Mark Millard writes: >>>>>>> =46rom inside a bulk -i where I did a manual make command >>>>>>> after it built and installed libsass.so.1.0.0 . The >>>>>>> manual make produced a /wrkdirs/ : >>>>>>> [...] >>>>>>> So the original creation looks okay. But . . . >>>>>>> [...] >>>>>>> So: The later, staged copy is a bad copy. Both are in the >>>>>>> tmpfs. So copying to the staging area makes a corrupted >>>>>>> copy inside the same tmpfs. After that, further copies of >>>>>>> staging's bad copy can be expected to be messed up. >>>>>> This and the fact that it happens on 14 and 15 but not on 13 = strongly >>>>>> suggests an issue wth `copy_file_range(2)`, since `install(1)` in = 14 and >>>>>> 15 (but not in 13) now uses `copy_file_range(2)` if at all = possible. >>>>>>=20 >>>>>> My educated guess is that hole detection doesn't work reliably = for files >>>>>> that have had holes filled while memory-mapped, so = `copy_file_range(2)` >>>>>> thinks there is a hole where there isn't one and skips some of = the data >>>>>> when `install(1)` uses it to copy the library from `${WRKSRC}` to >>>>>> `${STAGEDIR}`. This may or may not be specific to tmpfs. >>>>>>=20 >>>>>> You may want to try applying the attached patch to your FreeBSD = 14 and >>>>>> 15 jails. It prevents `cp(1)` and `install(1)` from trying to = use >>>>>> `copy_file_range(2)`. >>>>> Yes, tmpfs is indeed the culprit (or at least involved). I have = had USE_TMPFS=3Dlocalbase in my poudriere.conf for a long time, since = otherwise my build machine would run out of memory very quickly, so I = didn't encounter any issues. >>>>>=20 >>>>> Now I changed it to USE_TMPFS=3Dyes, rebuilt only textproc/libsass = and textproc/sassc, and then after reinstalling those packages: >>>>>=20 >>>>> $ /usr/local/bin/sassc >>>>> Segmentation fault >>>> And after applying Dag-Erling's patch to disable copy_file_range = for cp and install, it works correctly again. >>> So indeed there might be an issue in tmpfs seeking for data. Could = you try >>> the following? >>>=20 >>> commit f4b848946a131dab260b44eab2cfabceb82bee0c >>> Author: Konstantin Belousov >>> Date: Tue Nov 26 15:34:56 2024 +0200 >>>=20 >>> tmpfs: do not skip pages searching for data >>>=20 >>> If the iterator finds invalid page at the requested pindex in >>> swap_pager_seek_data(), the current code only looks at the swap = blocks >>> to search for data. This is not correct, valid pages may appear = at the >>> higher indexes still. >>>=20 >>> diff --git a/sys/vm/swap_pager.c b/sys/vm/swap_pager.c >>> index db925f4ae7f6..390b2c10d680 100644 >>> --- a/sys/vm/swap_pager.c >>> +++ b/sys/vm/swap_pager.c >>> @@ -2503,12 +2503,9 @@ swap_pager_seek_data(vm_object_t object, = vm_pindex_t pindex) >>> VM_OBJECT_ASSERT_RLOCKED(object); >>> vm_page_iter_init(&pages, object); >>> m =3D vm_page_iter_lookup_ge(&pages, pindex); >>> - if (m !=3D NULL) { >>> - if (!vm_page_any_valid(m)) >>> - m =3D NULL; >>> - else if (pages.index =3D=3D pindex) >>> - return (pages.index); >>> - } >>> + if (m !=3D NULL && pages.index =3D=3D pindex) >>> + return (pages.index); >>> + >>> swblk_iter_init_only(&blks, object); >>> swap_index =3D swap_pager_iter_find_least(&blks, pindex); >>> if (swap_index =3D=3D pindex) >> Not sufficient, unfortunately . . . >>=20 >> I patched what I've been running and rebooted into: >>=20 >> # uname -apKU >> FreeBSD 7950X3D-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT #152 = main-n273696-43e045c1733d-dirty: Tue Nov 26 07:21:27 PST 2024 = root@7950X3D-ZFS:/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64= .amd64/sys/GENERIC-NODBG amd64 amd64 1500027 1500027 >>=20 >> Note: 43e045c1733d is from 2024-Nov-18 . >>=20 >> I then built libsass : >>=20 >> [00:00:02] [01] [00:00:00] Building textproc/libsass | = libsass-3.6.6 >> [00:00:20] [01] [00:00:18] Finished textproc/libsass | = libsass-3.6.6: Success ending TMPFS: 3.42 GiB >>=20 >> I then installed it, resulting in: >>=20 >> # pkg info libsass >> libsass-3.6.6 >> Name : libsass >> Version : 3.6.6 >> Installed on : Tue Nov 26 07:33:15 2024 PST >> Origin : textproc/libsass >> Architecture : FreeBSD:15:amd64 >> Prefix : /usr/local >> Categories : textproc >> Licenses : MIT >> Maintainer : nivit@FreeBSD.org >> WWW : https://sass-lang.com/libsass >> Comment : C/C++ implementation of a Sass compiler >> Shared Libs provided: >> libsass.so.1 >> Annotations : >> FreeBSD_version: 1500027 >> build_timestamp: 2024-11-26T15:32:33+0000 >> built_by : poudriere-git-3.4.99.20240811 >> . . . >>=20 >> libsass.so.1.0.0 still has .got.plt starting with (this time): >>=20 >> 2bed60 00000000 00000000 00000000 00000000 ................ >> 2bed70 00000000 00000000 00000000 00000000 ................ >> 2bed80 00000000 00000000 00000000 00000000 ................ >> 2bed90 00000000 00000000 00000000 00000000 ................ >> . . . >> 2bffc0 00000000 00000000 00000000 00000000 ................ >> 2bffd0 00000000 00000000 00000000 00000000 ................ >> 2bffe0 00000000 00000000 00000000 00000000 ................ >> 2bfff0 00000000 00000000 00000000 00000000 ................ >> 2c0000 96cb2a00 00000000 a6cb2a00 00000000 ..*.......*..... >> 2c0010 b6cb2a00 00000000 c6cb2a00 00000000 ..*.......*..... >> 2c0020 d6cb2a00 00000000 e6cb2a00 00000000 ..*.......*..... >> 2c0030 f6cb2a00 00000000 06cc2a00 00000000 ..*.......*..... >> . . . >>=20 >> And still results in: >>=20 >> # sassc >> Segmentation fault (core dumped) >>=20 >>=20 >>=20 >> =3D=3D=3D >> Mark Millard >> marklmi at yahoo.com > =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Tue Nov 26 19:58:14 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XyYKg2PsQz5dwTQ for ; Tue, 26 Nov 2024 19:58:31 +0000 (UTC) (envelope-from soren.schmidt@gmail.com) Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XyYKf2NBkz41l3 for ; Tue, 26 Nov 2024 19:58:30 +0000 (UTC) (envelope-from soren.schmidt@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=SE7Opm1M; spf=pass (mx1.freebsd.org: domain of soren.schmidt@gmail.com designates 2a00:1450:4864:20::32c as permitted sender) smtp.mailfrom=soren.schmidt@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-wm1-x32c.google.com with SMTP id 5b1f17b1804b1-434a736518eso10670455e9.1 for ; Tue, 26 Nov 2024 11:58:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1732651107; x=1733255907; darn=freebsd.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=N9K2lFcNJBlWiOvA211xuAco6BLvHNTNk54+UiualdU=; b=SE7Opm1Mo1F34fIL7SFbgePPJGYFvfd5Ch4AE72v6YL84ySPCXAQrZkIcBvXDC6cqm VHxqtQfKeY9UFwmd14eBIdNgPu+RBiHvoNfjyXIQwaB/JML+AH4BhUOFES+ZqIGQq53u 6uIn+ninD3olODtha/0e3M6UV9EfoUl90uoSWWs3Ytd8Cu6G4Q6/MfqFFC50PUrUmemx kg7b2et5eWwdoodFohcwTVya/Iu+0+bi+QKNuUbzEZuBQOT70p6TUa8fUXRYmbr/+s9K sCUw5YRE5k2cBq91VKIc581Uw592tJe+XLRjG/Zg/2zCVEmyJP3JbR8CNtvt8bohDw28 HGqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732651107; x=1733255907; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=N9K2lFcNJBlWiOvA211xuAco6BLvHNTNk54+UiualdU=; b=hqex+4a3fOfyH9vDbgeHbvX8ssFizHJHliamK7eX9Ou7X7Pwc/SfEMc211jfIEk3Os wQNgSGQserrfHcTKIJO3IdJWrDuMQokAlp9HW2VemhDJ3rFGLfkSFkrsO7vQKsNt4WLQ 0OtZ+HjTJkvrsq0VNSMp0hc+wFO+OLotbo2yscvw1fQx4JKrBRvdRM4iWQRkHimKqb1X C+ksjYa1MtFmW1Z9h3IKQYrOsuKKMMQZRVjl3cibbwuFRXrBtKS4lusPvlMc9gQcbq+d IoqpjXeIOCbEXQ21bjQ8P68AVf0PdRTG6TLZSm4v/brCz0/zuY9hv2eE2DBSCZWiQkcU hruA== X-Gm-Message-State: AOJu0YwLxm+4ykhEO7btSqp/GzvHI+sN65rxYrlrX32wg4XQKwXoRKXF 8LJw1rIwgGXhqMi/V2FeqfVAp2LjoprWfxzbnEKVQi+TjGCmtvqUn1xApw== X-Gm-Gg: ASbGncusYlg2e2CXLp+cDLHcPabCYops/pe2eB1/NHYDAJZ2dkHeyXu908BBoohuwJ+ C2+q8kWk0g8s5vEdt+UV7gtb6GwuH2pASfUcZt4u/Rw3Q1ViT9cvmRdcMJscKbY/wmsAERVT0wS NqoSKJsALHB4cauWHcnNYg4fyDM5dZlK1etGDQn0k2EHJxKeBAYP01eXkT4dwXENUOkyt69XaaC onbF2/VWvJ7UQTIPlYsmprNQ4bo/kOTssvuX3jKH8DDFQUOrXq58pyCeGOYMB2xPOoT04qVlQ== X-Google-Smtp-Source: AGHT+IFiUeImqpuayD+M8/JjKORuqxZURA4+Jnw9CClTso5pnhQowQZx7WhF3S7wlPRt5YLl43jMfw== X-Received: by 2002:a05:6000:701:b0:382:440e:4e88 with SMTP id ffacd0b85a97d-385c6eb783dmr261170f8f.16.1732651106503; Tue, 26 Nov 2024 11:58:26 -0800 (PST) Received: from smtpclient.apple (s61.dk. [85.27.186.9]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3825fb27386sm14224377f8f.51.2024.11.26.11.58.25 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 26 Nov 2024 11:58:26 -0800 (PST) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6.1.2\)) Subject: Re: IBM ServeRAID M5110e and other mfi/mrsas users - call for testing / request for information From: =?utf-8?Q?S=C3=B8ren_Schmidt?= In-Reply-To: <9f572320-bc01-41c2-9820-74844b6ec9f1@gmail.com> Date: Tue, 26 Nov 2024 20:58:14 +0100 Cc: Johan Hendriks Content-Transfer-Encoding: quoted-printable Message-Id: <6FCA01E4-FEFB-447B-9034-4D3BC0D0C335@gmail.com> References: <9f572320-bc01-41c2-9820-74844b6ec9f1@gmail.com> To: FreeBSD Current X-Mailer: Apple Mail (2.3731.700.6.1.2) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[text/plain]; TO_DN_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; TAGGED_FROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; FREEMAIL_CC(0.00)[gmail.com]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+]; FROM_HAS_DN(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; TAGGED_RCPT(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::32c:from] X-Rspamd-Queue-Id: 4XyYKf2NBkz41l3 X-Spamd-Bar: --- > On 26 Nov 2024, at 13.31, Johan Hendriks = wrote: >>=20 >> To test, set the loader tunable hw.mfi.mrsas_enable=3D1 in >> /boot/loader.conf. Confirm that mrsas is being used, and that the >> system functions properly. >>=20 > I use Dell servers and i set the loader tuneable = hw.mfi.mrsasa_enable=3D1 on these machines. I use ZFS, so no raid = functionality off the card itself. This works fine for me. I have not = have any issues. I use this setting as mrsas give me way nicer device = names, i do remember having boot issues with the mfi default. But that = is a long time ago. I have several Dell servers (R640/R740) running 14-stable using mrsas = and RAID10 volumes, UFS on FreeBSD, and I have experienced corrupted = filesystems on reboot on 3 out of 14 servers so its not consistent, its = the same servers that always fail though :)=20 However for our purposes we have set cache policy to write-through = instead of write-back (default) and that seems to have solved the issue. There is no real performance penalty at least with our workload. Would be nice to have this fixed though, seems like the cache is not = written to disk on reboot but if you force it through it works. Its just like when the backup battery on the controller dies, then a = reboot also messes up the disk :) -S=C3=B8ren From nobody Wed Nov 27 02:16:19 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xyjjf2kymz5fN7Q for ; Wed, 27 Nov 2024 02:16:22 +0000 (UTC) (envelope-from glebius@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xyjjf2Bx7z4jfJ; Wed, 27 Nov 2024 02:16:22 +0000 (UTC) (envelope-from glebius@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732673782; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=+9P80dRwO0EF9uu+9bFbzNIiAxr81EyVZo2krMZHoik=; b=oAB1+tM87g4GMmS3NtIS247HCBJnfHO/KA3lU+NjLlcoWZG3yu3dztkctr2bMMMVk6aRHz CXCVDl0hKFR5duc8+WRRXdItrnKLz6hu7jyTvjzJYA4AxEsGr+ueSPlY5US7/hu5HAPrJj 10S6xkMZJuM0wSxKS42X49QSssE9S52dz/nkH9pQADOkOGrmPI24g9xaxhByEotIVAP1jx hiDWb8T1bksqNDlVkLKFMhgAjCcvlKf4g84z0grUdvHJ8cioViiKZnMkjNvkHPGT51J7M1 yDFyJgzfJykolXWjTBIfTpgLGVl7pDEN5G6hCYKbiuDFrlp+pQ+4fI2eJdVUFw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732673782; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=+9P80dRwO0EF9uu+9bFbzNIiAxr81EyVZo2krMZHoik=; b=QRtgFhTBtI0cc3BRd4BLpxolLi2ib8tubeaIkc9bvGfpyA3zVz+78dhs3jK0wihscZl+Th G9CgtQns18VpznFm1b9uC4yd90nadxTufyO0SrrZxygaOEJt7M4+q2EINr7Qc2fKnxwe9N qEMqQQ+iodt8k30rWQiYYwiPgLFuVshlrXYh6NmBj2v2dIFelCxRu+ZlAwdIK4k4GRc5WV zK7B8BpNFEmrpHKyttPrxfRowyLrAeKUFnRggjYemmFFj5FuOrsT/D9EhlhN6TRVLzn1qR Sc6OiYgkFABlyHjXAxXQdaS9go7CLL+8RqHZOmuGyyW1OvyxDbf4nBbmEI7Vzg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1732673782; a=rsa-sha256; cv=none; b=j8BRv5cmOa7DC5DK1m8vD8OB3mCMu2znsrLanLwHMJERpICF2nW7TWHNi8383oZVyXT/p9 miK2COEE1G5fElqv0PI7wgL/o+P8h6Qb7EGtgLZsWIKFeNYicM/j1ee5hAe4jI2LgKb7O+ m6+yNIqe1yb8/LkyEB0DCEydzRvZfsgpG/zD0iQAlAFnhyxuAYkaxuHs9r5uOZF0+PCN/M tYwdGYKamZ8Oq8lGBj6uI8mHJluOXhV3wtZlTg7neGOVtaNxLAGlgpKoBVMnnd0fDSwHqq WcoAp48fVzEFr8eDemVYb9soG8awbLe2vd3qiBMrcN/dQnnTWiB4p/IPYe10Bg== Received: from cell.glebi.us (glebi.us [162.251.186.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: glebius) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Xyjjd6JVDz1BMs; Wed, 27 Nov 2024 02:16:21 +0000 (UTC) (envelope-from glebius@freebsd.org) Date: Tue, 26 Nov 2024 18:16:19 -0800 From: Gleb Smirnoff To: freebsd-current@freebsd.org, src-committers@freebsd.org Subject: Re: November 2024 stabilization week Message-ID: References: List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Hi, On Mon, Nov 25, 2024 at 01:01:05AM -0800, Gleb Smirnoff wrote: T> This is an automated email to inform you that the November 2024 stabilization week T> started with FreeBSD/main at main-n273822-ff4c19bb5427, which was tagged as T> main-stabweek-2024-Nov. At Netflix testing we didn't discover any new regressions comparing to the October stabweek. My personal machines on the new stanpshot are also doing well. I didn't receive any emails reporting regressions through the last days, hence releasing the advisory freeze. P.S. We are aware of regression in ZFS, that happened between September and October stabweeks and are working on a reliable reproducer. A panic happens when using md(4) device backed by a file on ZFS. -- Gleb Smirnoff From nobody Wed Nov 27 17:00:52 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xz5LK6gtbz5fTMn; Wed, 27 Nov 2024 17:00:57 +0000 (UTC) (envelope-from christos@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xz5LK5GZMz4spN; Wed, 27 Nov 2024 17:00:57 +0000 (UTC) (envelope-from christos@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732726857; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=cGEs9xoL8s4C0kmeR38VAZBuCbEmzp+HsKi5C+UsYXQ=; b=s6BFlFmZTYgsZm4PXRNKHFlfVkrgp7prb4Woi5EcNE+dqN/YQxB+VQxF+UcRApuLuHJC/8 SV4JK4Irv4JRmxYZnlTlWo1L3uGeWJtZzM5OX3jLkwOA96Rw2cMKjwSIx2cMty9nczrZ0K +hO6uD5f6Mujz9iaPWtXKCH50UQCZ0PNeZAGF28MQAWdLyfk2Gov/CZxCatk58NMYpz2Ri TS4Phq07rlZuvvPNcBjsBHiuQYr4b//x8cR41azfiTV9qThA2hpLnu/sTUJiwdz6iB1ayr VLQsqcAI6UDvFyYKgmAVe98GDpz+ms+QBcsiZFxRrr+7ZDwIHThOHrlHotRmDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732726857; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: dkim-signature; bh=cGEs9xoL8s4C0kmeR38VAZBuCbEmzp+HsKi5C+UsYXQ=; b=k8V8BQi0aNfHMI0B2ycFpTcnKLPxb7rhqHF4wy/7qpodRknpj3/b0uhKfLUeo9hWk+HiOR 1Xuyjm8u+A155Op5EzyI0st4HiS7LSEc2p5zmwJPWCNnIqxgFsKseG8/Se/MU/nKZSfPLK ugOrkzimD1c01YqwnlBRUCzgubX2ymIBcuwsOvacyUZeIKfIALkykdxozqMr63nrbz4Egx v16q9UVmVx1guvn69M/XWwMsPKoXwDBZDQectaTbK45/g0JEiKmzOBNALzzt9NzviGDOhm OPQJW/drsME4y4ZSAhL3jWxnLGrj9uV3SiFJTCnVAA0ZWfnEC/xiQM7lHE0Y3w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1732726857; a=rsa-sha256; cv=none; b=PgTIGEfam88T6vjdZcm41mA8yGIzGNbXr/DNpb07p7y68zDRm6UMABsmHYoHIm7G/Ib0FH zG9hhVGKagBmFG335GdaEK4C848L6C+nHXg4lwyrQf3RUsj8NvMIWPXABDyHtUa+k1vN7f 8bo78txGebV+r9EhbD9icsh+aCMCfmIg/CNG6la18pKsvlSQKbmaRfsBczmwWbINtl8tmJ yGPAGHraAPxGSEtMXhXlrg3z/Kj9vGECAOqTLqpDxRajVThc5Kt9Bk4LldRCV4X15rCEuG 5530O25Oao9aE4jMdF3M9wwg/UsXkssuGKrVC2XAwx9y/o8Swhjr6/R/X5kh2Q== Received: from margiolis.net (mail.margiolis.net [95.179.159.8]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA512) (Client did not present a certificate) (Authenticated sender: christos/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Xz5LK1vjPzJBG; Wed, 27 Nov 2024 17:00:57 +0000 (UTC) (envelope-from christos@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; s=mail; bh=cGEs9xoL8s4C0km eR38VAZBuCbEmzp+HsKi5C+UsYXQ=; h=subject:cc:to:from:date; d=margiolis.net; b=KXf+kYk2j2j2jH56C2IYHnWsc3QWZCV2ruOrjTcXMHUTPyJuInk 9zfdAk/WezlDow9dyQCoYILklQHtl3Qdooa/cidSR8SL0fvFJwXiA+SFkJLPSEX5w+L+3r qr6can0NX8WIBp/obz6TA7VIzlOgOuLveACdTFRVcgAh4NhEIg= Received: from tpad (public-gprs242282.centertel.pl [31.60.93.171]) by margiolis.net (OpenSMTPD) with ESMTPSA id e3062791 (TLSv1.3:AEAD-AES256-GCM-SHA384:256:NO); Wed, 27 Nov 2024 17:00:54 +0000 (UTC) Date: Wed, 27 Nov 2024 18:00:52 +0100 From: Christos Margiolis To: freebsd-current@freebsd.org Cc: freebsd-hackers@freebsd.org, freebsd-multimedia@freebsd.org Subject: [Call for testing] sound(4) races and panics Message-ID: List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hello, I recently pushed a series of commits [1][2][3][4] to -CURRENT (will MFC to stable/14 soon) that fix several panics and races. If anyone is interested, I would appreciate some testing to make sure I haven't missed anything. Thanks. Christos [1] https://cgit.freebsd.org/src/commit/?id=5bd08172b4150503c9cf60ffe3c97716c5bf6fa1 [2] https://cgit.freebsd.org/src/commit/?id=5ac39263d825d7b2f8a89614a63fee90ffc77c07 [3] https://cgit.freebsd.org/src/commit/?id=2839ad58dd8a4cf5294180fc599800c437a8d4d8 [4] https://cgit.freebsd.org/src/commit/?id=5317480967bfc8bf678e4da3fce81bcb3f5b7836 From nobody Wed Nov 27 18:18:41 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xz74H203Cz5fZTR for ; Wed, 27 Nov 2024 18:18:55 +0000 (UTC) (envelope-from shuriku@shurik.kiev.ua) Received: from mail.flex-it.com.ua (mail.flex-it.com.ua [193.239.74.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xz74G0ccdz42rn for ; Wed, 27 Nov 2024 18:18:53 +0000 (UTC) (envelope-from shuriku@shurik.kiev.ua) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of shuriku@shurik.kiev.ua designates 193.239.74.7 as permitted sender) smtp.mailfrom=shuriku@shurik.kiev.ua; dmarc=none Received: from 93.183.208.50.ipv4.datagroup.ua ([93.183.208.50] helo=[192.168.200.135]) by mail.flex-it.com.ua with esmtpsa (TLS1.3) tls TLS_AES_128_GCM_SHA256 (Exim 4.98 (FreeBSD)) (envelope-from ) id 1tGMce-00000000FFq-2asQ for freebsd-current@freebsd.org; Wed, 27 Nov 2024 20:18:44 +0200 Content-Type: multipart/alternative; boundary="------------XTZuZ002M080oymEhh1jD0bk" Message-ID: <208081a4-846f-4e70-aad6-c3e75a6ce777@shurik.kiev.ua> Date: Wed, 27 Nov 2024 20:18:41 +0200 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [Call for testing] sound(4) races and panics To: freebsd-current@freebsd.org References: Content-Language: uk-UA From: Oleksandr Kryvulia In-Reply-To: X-ACL-Warn: SPF failed. 93.183.208.50 is not allowed to send mail from shurik.kiev.ua. X-Spamd-Result: default: False [-1.73 / 15.00]; HFILTER_URL_ONLY(1.55)[0.70588235294118]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.993]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; XM_UA_NO_VERSION(0.01)[]; ASN(0.00)[asn:35297, ipnet:193.239.72.0/22, country:UA]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCPT_COUNT_ONE(0.00)[1]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; RCVD_TLS_ALL(0.00)[]; R_DKIM_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; DMARC_NA(0.00)[shurik.kiev.ua]; TO_DN_NONE(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[] X-Rspamd-Queue-Id: 4Xz74G0ccdz42rn X-Spamd-Bar: - This is a multi-part message in MIME format. --------------XTZuZ002M080oymEhh1jD0bk Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit 27.11.24 19:00, Christos Margiolis: > Hello, > > I recently pushed a series of commits [1][2][3][4] to -CURRENT (will MFC > to stable/14 soon) that fix several panics and races. If anyone is > interested, I would appreciate some testing to make sure I haven't > missed anything. Thanks. > > Christos > > [1]https://cgit.freebsd.org/src/commit/?id=5bd08172b4150503c9cf60ffe3c97716c5bf6fa1 > [2]https://cgit.freebsd.org/src/commit/?id=5ac39263d825d7b2f8a89614a63fee90ffc77c07 > [3]https://cgit.freebsd.org/src/commit/?id=2839ad58dd8a4cf5294180fc599800c437a8d4d8 > [4]https://cgit.freebsd.org/src/commit/?id=5317480967bfc8bf678e4da3fce81bcb3f5b7836 > Is there a need to test on hardware that worked before? --------------XTZuZ002M080oymEhh1jD0bk Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit
27.11.24 19:00, Christos Margiolis:
Hello,

I recently pushed a series of commits [1][2][3][4] to -CURRENT (will MFC
to stable/14 soon) that fix several panics and races. If anyone is
interested, I would appreciate some testing to make sure I haven't
missed anything. Thanks.

Christos

[1] https://cgit.freebsd.org/src/commit/?id=5bd08172b4150503c9cf60ffe3c97716c5bf6fa1
[2] https://cgit.freebsd.org/src/commit/?id=5ac39263d825d7b2f8a89614a63fee90ffc77c07
[3] https://cgit.freebsd.org/src/commit/?id=2839ad58dd8a4cf5294180fc599800c437a8d4d8
[4] https://cgit.freebsd.org/src/commit/?id=5317480967bfc8bf678e4da3fce81bcb3f5b7836

Is there a need to test on hardware that worked before? --------------XTZuZ002M080oymEhh1jD0bk-- From nobody Wed Nov 27 18:39:40 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xz7XV1DNfz5fcCv for ; Wed, 27 Nov 2024 18:39:54 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-io1-f46.google.com (mail-io1-f46.google.com [209.85.166.46]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xz7XT23D9z46kC for ; Wed, 27 Nov 2024 18:39:53 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of carpeddiem@gmail.com designates 209.85.166.46 as permitted sender) smtp.mailfrom=carpeddiem@gmail.com; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=freebsd.org (policy=none) Received: by mail-io1-f46.google.com with SMTP id ca18e2360f4ac-8418a2f596fso134424939f.1 for ; Wed, 27 Nov 2024 10:39:53 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732732792; x=1733337592; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jxFmb9jylw/5q1O2z+JSa/SaMmCq8nl5/l0G4h27SBM=; b=GQfckt7WIwOFuWF4894ZkjGCoIzYcUgG3fdxVgBG/eyCJv1LAc//KaI1uhp42w/YVQ a3flwF3xuaFZc/Qv8Uf7ThoTbT+pq6E7GtW40F7eWfpJHiMYbsQoBhF2Ezo2ZogRKpne 5ZiI3a6nMzvPQVam9JfzaW+J/bROh2twWs8UrBgMMX/fMAUQEuRiK0fg/HAptn6lAOR9 bF5/ksgbdRsE2JBA7UKO3f7tPFYqwDZ252p2Mlxlya/5lok+GA8F/Bls/0uW2tv+YkXZ 1/6GJghndY1P+Rz8+67A38AYVeBvu3bS0Noto3syH1dyQpQfskRqo3Uv6k5oThBv8gHI 8/Fw== X-Gm-Message-State: AOJu0YxX7VsPS9DfdUA8KuZioBOHcjFjljVEsV14D215ECxbfGU1m/1s wIGNB1XGihLh9nE6OhWuFrjsxjomTkgfyguAipyiov+H0pLjJGSEkqJhSeR6G3XTv8EzDVnjoy5 pRJtb4KbGMGVKyLDDvxTVYhp/vJLZSw== X-Gm-Gg: ASbGncviMTLXRtli0vAHY8LW4drl5Hm3fIC5wipdVm1Ub7AI8nhjxLVH7YvnWwPoC0L 1P6LvytLnxHAAsnNQofW2XV8OSzIakoaNSfcVa4KfcfpdmduBvNzQ2wO/DmRZTUeqlg== X-Google-Smtp-Source: AGHT+IELH0SgSdDe4RSe2n/wOiyLM0EstFpgbyqSLvjq2ExGdChw3rx/YzsWWZWc/fgOgpo472E62yn1jXg5uLo8T3c= X-Received: by 2002:a05:6602:3417:b0:83a:b7c8:a3dc with SMTP id ca18e2360f4ac-843ece77139mr529964439f.1.1732732792415; Wed, 27 Nov 2024 10:39:52 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 References: <9f572320-bc01-41c2-9820-74844b6ec9f1@gmail.com> <6FCA01E4-FEFB-447B-9034-4D3BC0D0C335@gmail.com> In-Reply-To: <6FCA01E4-FEFB-447B-9034-4D3BC0D0C335@gmail.com> From: Ed Maste Date: Wed, 27 Nov 2024 13:39:40 -0500 Message-ID: Subject: Re: IBM ServeRAID M5110e and other mfi/mrsas users - call for testing / request for information To: =?UTF-8?Q?S=C3=B8ren_Schmidt?= Cc: FreeBSD Current , Johan Hendriks Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-2.89 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.993]; FORGED_SENDER(0.30)[emaste@freebsd.org,carpeddiem@gmail.com]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; MIME_GOOD(-0.10)[text/plain]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : SPF not aligned (relaxed), No valid DKIM,none]; RCPT_COUNT_THREE(0.00)[3]; FREEMAIL_CC(0.00)[freebsd.org,gmail.com]; RCVD_TLS_LAST(0.00)[]; FROM_HAS_DN(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; TO_DN_ALL(0.00)[]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; MISSING_XM_UA(0.00)[]; FREEFALL_USER(0.00)[carpeddiem]; FREEMAIL_ENVFROM(0.00)[gmail.com]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_NEQ_ENVFROM(0.00)[emaste@freebsd.org,carpeddiem@gmail.com]; RCVD_COUNT_ONE(0.00)[1]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; TAGGED_RCPT(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.166.46:from]; RCVD_IN_DNSWL_NONE(0.00)[209.85.166.46:from] X-Rspamd-Queue-Id: 4Xz7XT23D9z46kC X-Spamd-Bar: -- On Tue, 26 Nov 2024 at 14:59, S=C3=B8ren Schmidt = wrote: > > I have several Dell servers (R640/R740) running 14-stable using mrsas and= RAID10 volumes, UFS on FreeBSD, and I have experienced corrupted filesyste= ms on reboot on 3 out of 14 servers so its not consistent, its the same ser= vers that always fail though :) Is this "a regression in mrsas vs mfi" though or "a bug observed with mrsas, and mfi is unknown"? (Or even, "this hardware is not supported by mfi?") From nobody Wed Nov 27 20:20:52 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xz9nG2ZPLz5fkKD for ; Wed, 27 Nov 2024 20:21:06 +0000 (UTC) (envelope-from soren.schmidt@gmail.com) Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xz9nG0w1Bz4HdZ; Wed, 27 Nov 2024 20:21:06 +0000 (UTC) (envelope-from soren.schmidt@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-wr1-x42f.google.com with SMTP id ffacd0b85a97d-3824aef833bso92919f8f.0; Wed, 27 Nov 2024 12:21:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1732738865; x=1733343665; darn=freebsd.org; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:from:to:cc:subject:date:message-id:reply-to; bh=4D8m+BzgG0soTpyFgBkorAaH3nPsvGlYhu6Il0oCsGY=; b=CkBoPN6duOPIJ7wDc/OCZac4uf1TQ5wrVtYlEcqEb0X2FIKNS75zXxsyLx6ypRrlIR pSUSRPJzdT2g6F3VE211ZeVI38ExAwl8yqnGWdCw59QyZS3JAilxxrEsiOZ3/QG4ff2B cnYy4gm5NURNaVdorW0B4WhALQ7THGEAGsGOoiibNR7LSIPFm9nHZxC0NU0Deob642u8 bmMGxk1TCxES3ZuOmHCECj5LrU2v7IAGZ+Z4uyo4dXzNPvp485vjPtl/utxSePxbVPW0 8J1t2bT4EYdRNBS8W/7LN7dSebGdv5H7mAC0h/YM0p6Fqe+xpFbHzHqFcbH/jDUAsKoa fl2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732738865; x=1733343665; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=4D8m+BzgG0soTpyFgBkorAaH3nPsvGlYhu6Il0oCsGY=; b=dwwchVJ7AAykkK4m1SrGV1oFbW5VW/eCAukPrqWhpOrspmxbWaA+0KtbsKYAP4ka05 vWtA3oCHd5sX98xrTmeWhMA/qDPHH7llLcV/W62AQeiGQKPe57GTdsbqanYPueAuSztQ 9QrvNonUbYHcRrxE6HZGuXgDCrQRkuqBRSyGzFXb1lxdVHN3rQQSMpyykos0StjurxEd UmmHdjRKs/cyQRTzrQ56aIOH17+gpxDpyrz/hn/SRnE4c4UPx3paituskkUsLyK3zYpJ QV5p74LUz6+RRC6gVFSxf4u4EhZ4hXAd4luaR083wrAYJWEYY8EwRy0yqgK8c2IipF8w T2mg== X-Gm-Message-State: AOJu0Yw8ZgDAookHh56klkynGS5EnOI9b3JeBnbWrPo8sYGx4pRbidMy c5yyW0EW0Saf1vXJ1Zaj4BS8FEY7NpZSy1uAD7mjO2ywST5Kygpee4VXgw== X-Gm-Gg: ASbGncvTg5K/ueqFB7+P7a0wROtRTfUpCutP5PfUpevgF4eb76DwVo8jwMRiCPCVLOl mpOUV5haUAx8C1ewNCLymg/7z3mKMWapMqkntmM4GddWqR4LyBrOx0fy4yauNByySdhv9aZVZDG FbPoC7FC2EO4WlluJ0pMrWrZ5Cm5hwEO2fMDON863e0LZWN8Oot0ci6J/RPwjEVrO/oqPwj5G0a /tNYfgOUiS7bL97OPYqZHx1HrGeR+vDVI1ntai1rQPsEEPDB6tBrZU8P1DDAPZJ61wrImWXgQ== X-Google-Smtp-Source: AGHT+IHfEdERQEPqyC2RkjGc3MVW2LXO7thQfVmC0TYyFwFreV208DqvkNUza28bW6C63CcLkVg4yA== X-Received: by 2002:a5d:59ae:0:b0:382:5010:c8c0 with SMTP id ffacd0b85a97d-385c6ed7651mr3935830f8f.39.1732738864485; Wed, 27 Nov 2024 12:21:04 -0800 (PST) Received: from smtpclient.apple (s61.dk. [85.27.186.9]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-434aa74f2c7sm31830095e9.2.2024.11.27.12.21.03 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Nov 2024 12:21:04 -0800 (PST) From: =?utf-8?Q?S=C3=B8ren_Schmidt?= Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_0C09ECF4-32F7-472F-B496-9F1AF60936D9" List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6.1.2\)) Subject: Re: IBM ServeRAID M5110e and other mfi/mrsas users - call for testing / request for information Date: Wed, 27 Nov 2024 21:20:52 +0100 In-Reply-To: Cc: FreeBSD Current , Johan Hendriks To: Ed Maste References: <9f572320-bc01-41c2-9820-74844b6ec9f1@gmail.com> <6FCA01E4-FEFB-447B-9034-4D3BC0D0C335@gmail.com> X-Mailer: Apple Mail (2.3731.700.6.1.2) X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; TAGGED_RCPT(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4Xz9nG0w1Bz4HdZ X-Spamd-Bar: ---- --Apple-Mail=_0C09ECF4-32F7-472F-B496-9F1AF60936D9 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On 27 Nov 2024, at 19.39, Ed Maste wrote: >=20 > On Tue, 26 Nov 2024 at 14:59, S=C3=B8ren Schmidt = wrote: >>=20 >> I have several Dell servers (R640/R740) running 14-stable using mrsas = and RAID10 volumes, UFS on FreeBSD, and I have experienced corrupted = filesystems on reboot on 3 out of 14 servers so its not consistent, its = the same servers that always fail though :) >=20 > Is this "a regression in mrsas vs mfi" though or "a bug observed with > mrsas, and mfi is unknown"? (Or even, "this hardware is not supported > by mfi?=E2=80=9D) =E2=80=9CBug observed with mrsas=E2=80=9D, I don=E2=80=99t think the = controller in those are supported by mfi but that I can find out ofc. Its been like that since these servers was moved from VMware to = FreeBSD/bhyve back I April. I do have a few R630=E2=80=99s lying aound here in the lab with the same = controller, I could rig up one of those for experiments if need be... -S=C3=B8ren --Apple-Mail=_0C09ECF4-32F7-472F-B496-9F1AF60936D9 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8

On 27 Nov 2024, at 19.39, Ed Maste = <emaste@freebsd.org> wrote:

On Tue, 26 Nov 2024 at = 14:59, S=C3=B8ren Schmidt <soren.schmidt@gmail.com> = wrote:

I have several Dell servers = (R640/R740) running 14-stable using mrsas and RAID10 volumes, UFS on = FreeBSD, and I have experienced corrupted filesystems on reboot on 3 out = of 14 servers so its not consistent, its the same servers that always = fail though :)

Is this "a regression in mrsas vs = mfi" though or "a bug observed with
mrsas, and mfi is unknown"? (Or = even, "this hardware is not supported
by = mfi?=E2=80=9D)

=E2=80=9CBug = observed with mrsas=E2=80=9D, I don=E2=80=99t think the controller in = those are supported by mfi but that I can find out = ofc.

Its been like that since these servers was = moved from VMware to FreeBSD/bhyve back I = April.

I do have a few R630=E2=80=99s lying = aound here in the lab with the same controller, I could rig up one of = those for experiments if need = be...

-S=C3=B8ren


= = --Apple-Mail=_0C09ECF4-32F7-472F-B496-9F1AF60936D9-- From nobody Wed Nov 27 21:56:56 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XzCw600csz5fr2Y for ; Wed, 27 Nov 2024 21:57:10 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XzCw52PDMz4RCq for ; Wed, 27 Nov 2024 21:57:09 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=TZzaerRD; spf=pass (mx1.freebsd.org: domain of rick.macklem@gmail.com designates 2a00:1450:4864:20::536 as permitted sender) smtp.mailfrom=rick.macklem@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-ed1-x536.google.com with SMTP id 4fb4d7f45d1cf-5cefa22e9d5so198701a12.3 for ; Wed, 27 Nov 2024 13:57:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1732744628; x=1733349428; darn=freebsd.org; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=xQn+3UnVn6LOCCInVQ4cCDqgSyMHjL43Cr267bTv6Os=; b=TZzaerRD568vp61iAvVUlhUJlsE/5Vt/9cGLkByn3iC2D84Y6SMdBft9e0QP6eDYyC tpMkwfzeDL7KgIsGYBWaPuyH9PKcCkk8hFKyyvut3qQz/jIjm/S8EPgaEJCvFF0KFgk1 WGWmDgWaeLA1tbJmukeJjZG/ED/XYX+CCSiS7H9BBrkX9U3DnbmxOqIwTqTWUMcwbpqN LkKRs5szZZbvWuqL2UYpjfRtQpPhtMj0q8nPnIy8/QktePmQLr8RbCzc6xTTAvRSS5ep 2vaWcUfd6fLNLVs60mha9VFD5z2bMkAa9ESwdyOtKwNlaxqvzVdpuwT/SgC6LBiG8bZG YuSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732744628; x=1733349428; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=xQn+3UnVn6LOCCInVQ4cCDqgSyMHjL43Cr267bTv6Os=; b=M4weRBiuzt1vghydpZTbOg4nZJLV6mA0vwWfq5sF5kGdIiUHi6hTn+4KMzzOn6Xt22 VHyJCKM122Wbe3Qef+5xTQ5H0QzrzE850etME+0QtnNiJcwqdmZ6W7KHkGh3fpK/DT8+ o85iz0FTojAFS6J67QHvkEJZIZq+Unyc3ncoq/SX0LUrEiUBgZdCZL36bSqaeV2uySCH vF83dBhImReofAN8JtlS+kQfrvixJh+j+EnDycffBe585oHDVKvDqItjmQ7jlXdroHPt GuuIJTzx0EgB8xPIShPHSyNa76mtqhrc6f9kPQHIH5aIRylbuP7wRKy++CLYNUvJUz6p 01Kg== X-Gm-Message-State: AOJu0YwvaCZr8IgykFsF7BR5RL5kYWhFfANkB0lUaOYpill2tAlJpstB 48WPePqLT4L/uo9AIOYIrGqJ295i+hzr7qlzfTGknKJLad9BP0V0KLTyO4pgGHry/EnB3I3ar7F u5qh1UtzUwiCngihWU01NKE7P96DL X-Gm-Gg: ASbGncvf8iVeXskDGjpjuqmugf5+MX7xrg7mLDRwuH9tXnUq25TPvgY5FL4cO89A3QE l4VDoAl5Gd/jpoRpeqtcb9DA42jxA+4ORRaqLXX2GK043SmafxHGiII+zNBGe5lY= X-Google-Smtp-Source: AGHT+IFg9dUu4YjRziNG9OXImi/xD4KbC2fPjETIk7WTyypPbTrTuVj4ZGdACOgE+xU5mOnRl1CDakHxUOSK9cjrODU= X-Received: by 2002:a05:6402:3489:b0:5cf:e460:43e2 with SMTP id 4fb4d7f45d1cf-5d080bcbf9cmr4073454a12.15.1732744627709; Wed, 27 Nov 2024 13:57:07 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 From: Rick Macklem Date: Wed, 27 Nov 2024 13:56:56 -0800 Message-ID: Subject: RFC: fixing PR#282995 To: FreeBSD CURRENT Cc: Michael Proto Content-Type: text/plain; charset="UTF-8" X-Spamd-Result: default: False [-3.95 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.95)[-0.951]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[text/plain]; TO_DN_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; TAGGED_FROM(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+]; FROM_HAS_DN(0.00)[]; MISSING_XM_UA(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MID_RHS_MATCH_FROMTLD(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::536:from] X-Rspamd-Queue-Id: 4XzCw52PDMz4RCq X-Spamd-Bar: --- Hi, PR#282995 reports that the "-alldirs" export option is broken, since it allows an export where the directory path is not a mount point. I'll admit I did not recall this semantic for -alldirs and I now see it is only documented in the "Examples" section of exports(5). Looking at the code, it appears this was broken between releng1 and releng2.0 (about 30years ago) when the call to mount(2) in mountd.c was changed from using the path in the exports line to using f_mntonname. (The check for "it is a mount point" depended on mount(2) failing because the path was not a mount point.) I do believe the semantic is a useful one, although making it that way after 30years might be construed as a POLA violation? So, what do others think I should do with this? (A) - Patch mountd to enforce the "must be a mount point when -alldirs is specified, plus update exports(5) to state this semantic clearly. or (B) - Patch mountd so that it enforces "must be a mount point when -alldirs is specified, but only enabled via a new mountd command line option. --> ie. Leave the default as not enforced, but allow enforcement based on a new mountd option. - Document this in both exports(5) and mountd(8). or ??? Thanks in advance for your comments, rick From nobody Thu Nov 28 10:47:40 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XzY1G0NGYz5fY29; Thu, 28 Nov 2024 10:47:46 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XzY1F70VXz4RXh; Thu, 28 Nov 2024 10:47:45 +0000 (UTC) (envelope-from avg@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732790866; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UdJS/Q4y0SjR265XnwEP4itIx26OOwyBPalG3yQ9YYA=; b=UOFUZF4DG/E6sKjIBlf8O1VeOYyN0NxEdjYUtj9XOnnsjENqvcMFgniHKk5Lpwo7oc4A3D P3erbzquSeyUKnMB2wzJ9fxCC4Q8w0iPoiUielfL9CatXQUBmbfdoO+Aa43HDOpZtN7MdS XGkbDWpLIJwVcqzW/fLz/LSYNbEk+8afPyyoI4L4QFT8nerNb5MqrXrjtw+zVZ0evpgHz1 2U5M0M8TV6lGCuptzaxLoyZNdW5kTbbRoljQM3MlWaB3SQhoAVYfWtCytqjMFhuKQek465 ESDVmGPyLVI+m1VN8AgluihFOGXdsyiDghRVM0OgqaWCdBHf9fO9YCbooWp2Pg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732790866; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UdJS/Q4y0SjR265XnwEP4itIx26OOwyBPalG3yQ9YYA=; b=Lv6NBByMK53FkzXKB7p4F2tJ6H59phDj4koz9RjETaJ5pI7HEQi1+/nNFK4Kpq5Pa0ACVe iPYXRbVrx7231OJFuWphT4PLdQWTqbJexoJvZhkqUFWmcxb40F+rCXccFDwVzZ0jM5e1zN 7Khxl6Hx6qUB0gVd4+LXlauYHsukKRxoNFMrfnyqeD4Zca0OdiCscG7BzdWQnPsLg8ORvT F4FrUpIGUYXxftGxgpFwDBhPcTj1B36/4Yp9pAD83j/YfTJTmQ41SCdmM1RZe1ZYElUqZT yVelYxvo8ZpbQxbufdNZdHLnbk4C55wAxMDLCw4aDRcr7Zw4NP2fXSyYiz3Plw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1732790866; a=rsa-sha256; cv=none; b=EoDY/xMm35j/RgJvhWCmygNvWQoHTWFUA7Nj/0Iwuu0MzpLwAso1wUQwQxGXzY9q0RlZhR lsKIiHNbrJershjYayVAleu7/NgvziXNqcnQjzbA1Gfxn8LvMWr5DdPWrTL2hnjzYd60vd 9hF8n3KwGm7qLycwoxwYCoxhXgifGIm/npuVUl5O93RP1+ZNYxySc1y+HE0YELulBAQ2nY YpsgT800zE2c9f0mDPV7BpAk8gUFTJVjaiiZXjuAeFQ47y6JelUAxPZDKMis8VlWcGIYqm us78hcA//6raSOVu7qZBhcPeie+KolFJI8jYcP7qaNvmAzZTyoHYjWy9iqwU1Q== Received: from [192.168.0.88] (unknown [93.188.39.137]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: avg/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4XzY1D2pfJzhFq; Thu, 28 Nov 2024 10:47:44 +0000 (UTC) (envelope-from avg@FreeBSD.org) Message-ID: <65d47ca6-b0b9-4c03-9e36-d0f2cf6b4937@FreeBSD.org> Date: Thu, 28 Nov 2024 12:47:40 +0200 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros] To: Mark Millard , Konstantin Belousov Cc: Dimitry Andric , =?UTF-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , "jah@freebsd.org" , dougm@freebsd.org, Alan Somers , Mark Johnston , FreeBSD Current , Guido Falsi , Yasuhiro Kimura , ports@freebsd.org References: <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> <0690CFB1-6A6D-4B63-916C-BAB7F6256000@yahoo.com> <3660625A-0EE8-40DA-A248-EC18C734718C@yahoo.com> <865xoa2t6f.fsf@ltc.des.dev> <69A2E921-F5E3-40D2-977D-0964EE27349A@FreeBSD.org> <4AE5B316-D7EB-4290-8D52-7FBF244EA7A4@FreeBSD.org> <33D56E3E-6476-48E8-B115-B906629B8AF5@yahoo.com> Content-Language: en-US From: Andriy Gapon In-Reply-To: <33D56E3E-6476-48E8-B115-B906629B8AF5@yahoo.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 26/11/2024 17:52, Mark Millard wrote: > libsass.so.1.0.0 still has .got.plt starting with (this time): > > 2bed60 00000000 00000000 00000000 00000000 ................ > 2bed70 00000000 00000000 00000000 00000000 ................ > 2bed80 00000000 00000000 00000000 00000000 ................ > 2bed90 00000000 00000000 00000000 00000000 ................ > . . . > 2bffc0 00000000 00000000 00000000 00000000 ................ > 2bffd0 00000000 00000000 00000000 00000000 ................ > 2bffe0 00000000 00000000 00000000 00000000 ................ > 2bfff0 00000000 00000000 00000000 00000000 ................ > 2c0000 96cb2a00 00000000 a6cb2a00 00000000 ..*.......*..... > 2c0010 b6cb2a00 00000000 c6cb2a00 00000000 ..*.......*..... > 2c0020 d6cb2a00 00000000 e6cb2a00 00000000 ..*.......*..... > 2c0030 f6cb2a00 00000000 06cc2a00 00000000 ..*.......*..... > . . . FWIW, I am not sure if it's relevant but I am seeing a similar pattern of corruption on tmpfs although in a different context, on FreeBSD 13.3. Details: https://lists.freebsd.org/archives/freebsd-fs/2024-November/003855.html -- Andriy Gapon From nobody Thu Nov 28 11:42:04 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XzZCz23stz5fbZb; Thu, 28 Nov 2024 11:42:07 +0000 (UTC) (envelope-from des@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XzZCz0wJ9z4WvX; Thu, 28 Nov 2024 11:42:07 +0000 (UTC) (envelope-from des@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732794127; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bn6X7qvkkKVNljg82ig5qNFXU4WM8JD9EBkC7giInUs=; b=qgY9WAcHjD/+KwXdMzAO2K2UMEG5ah2+PVNfQ9pbvvnt0MJ3iCK1LK4nJzrxH4GX+RAwMp NPui4ZGk2vPWdHGyHytDG88eh4Fl5xxZVL5Tngc94i13GlpYZ2xSKRt1H8hHsrP9iYdvvn sZ+yGF6XVAVA6kgRn7jnTbJsbRDaaSggmucinZC4kn+DC/5jwkDH4ad0ZkyZdHtYFwOX8F h+lKVLcWzD4H6LK5qeGAsU2Uk/p+ixxFKzmCEm+quVS5oS9HCKD2tEPByxbi73FRJKiI8T G2S+E4LYXDDeFRDMZRjfSxTMI029i6sAO1qJotaVGjcn6b5EQBBhjwgRRCbyLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732794127; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bn6X7qvkkKVNljg82ig5qNFXU4WM8JD9EBkC7giInUs=; b=VN2q0CaOJ00D0PykyY2inc9R9U9A36AnVU0Hk5VWnwrKR1C1wNOPD+xB88Zx91gMg/8BOc Ne9VpFxseRBz8shd1Uk7ZlskLQUpm4Utm457L1/Qls0Muo11gA/MSCaSreKyDol9oJGz2g oDbVfngxB0cOIgsVwkeyRXVwimaFY1OlDdmc9ax7E1BKqTTy2kZoh/i42isKFRNkdCdIRL iqdVzJBuZLxaGof1D6GpX0BFQaKJtQlVv9grSVeypNNbGBzvLNrB290pJHQqBZcag7T+Dz +K9Zy3ehfzs2GNklMkKBkwKQ3nPc4kHKNB4PmTeYuTJnJo9+IDBiblNYU7gzJA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1732794127; a=rsa-sha256; cv=none; b=hNQE+REB8jWFbOO7/I76lhKYH51jwlPAZ/FP3XaYNIrPqthcxaQuboE6qt8Oe11ZvyXFvU Czk8sogz9jrZQMcYs2BWUDvp//+jhe4WHFXfVpOuSmVLl5V2B280m3IZkyGAShW9Bvl8tm NYbVPYFZxapkYKjvomxTcwq9XinQEdWYmcvDjyiy29ToTv7dO3A+AKVkBfSiCESFC47M7s 9CfS0u9cXcdOU0BLptDtS0SRqTXnL8b7WVNpgg+VNkVVXcFG8StBosehh/iXEiXNogwEti F1a9knsoEDC0QXUTtPlfHpUz/3KeMHUVVSFuCHtbdQL1v5BZR2oPipItWCRJkA== Received: from ltc.des.dev (unknown [IPv6:2a01:e0a:c54:bed0:922e:16ff:fef1:acef]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: des) by smtp.freebsd.org (Postfix) with ESMTPSA id 4XzZCy72J1zjCs; Thu, 28 Nov 2024 11:42:06 +0000 (UTC) (envelope-from des@freebsd.org) Received: by ltc.des.dev (Postfix, from userid 1001) id D58866078D; Thu, 28 Nov 2024 12:42:04 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Andriy Gapon Cc: Mark Millard , Konstantin Belousov , Dimitry Andric , "jah@freebsd.org" , dougm@freebsd.org, Alan Somers , Mark Johnston , FreeBSD Current , Guido Falsi , Yasuhiro Kimura , ports@freebsd.org Subject: Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros] In-Reply-To: <65d47ca6-b0b9-4c03-9e36-d0f2cf6b4937@FreeBSD.org> (Andriy Gapon's message of "Thu, 28 Nov 2024 12:47:40 +0200") References: <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> <0690CFB1-6A6D-4B63-916C-BAB7F6256000@yahoo.com> <3660625A-0EE8-40DA-A248-EC18C734718C@yahoo.com> <865xoa2t6f.fsf@ltc.des.dev> <69A2E921-F5E3-40D2-977D-0964EE27349A@FreeBSD.org> <4AE5B316-D7EB-4290-8D52-7FBF244EA7A4@FreeBSD.org> <33D56E3E-6476-48E8-B115-B906629B8AF5@yahoo.com> <65d47ca6-b0b9-4c03-9e36-d0f2cf6b4937@FreeBSD.org> User-Agent: Gnus/5.13 (Gnus v5.13) Date: Thu, 28 Nov 2024 12:42:04 +0100 Message-ID: <86zflj1t6b.fsf@ltc.des.dev> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Andriy Gapon writes: > FWIW, I am not sure if it's relevant but I am seeing a similar pattern > of corruption on tmpfs although in a different context, on FreeBSD > 13.3. Not relevant at all. In this case the file is not actually corrupted but `install(1)` skips over some of it when copying because `SEEK_DATA` is implemented incorrectly. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@FreeBSD.org From nobody Thu Nov 28 12:19:18 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xzb2y4MTrz5fdbr; Thu, 28 Nov 2024 12:19:22 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xzb2y3ndZz4c4w; Thu, 28 Nov 2024 12:19:22 +0000 (UTC) (envelope-from avg@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732796362; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=VGhBxBzqJVljgiwjDtUNazRT+7F+4DhC8n47BKuxDVI=; b=qBIyy4YZqRjDMvE5Wv4CAlUE7BlCGv0ldwO02zFV5cA3P3Dn6WJizEHAOrmyloHq4FiM6m hkt8X/RTOKLOse1D5L6VMgO3t/dxmPjUJOPnP0ORqLSzjcq/SyPHVfb1h7jtkmnq3yt0HE 4cEL6Hb+oiZT6+28xclAemyMZyuf6vt84xctBAaPvjVCOIqy/3g/IiGOK+9ThZzkvOf/vp BXZnxrGa/9xwJ8vKxDDc82zdk5aj5Zxtq/19P89hHd7U3/cOsuGtHiuUMgjKf6xZX6eJDx 6T7c2s/wFMkw3I/2HtWL15tHOrZUMX5t4xRQ9QIF7vcVqDcaoPL6rduQDSy+Uw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732796362; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=VGhBxBzqJVljgiwjDtUNazRT+7F+4DhC8n47BKuxDVI=; b=hOCttXmLpt3AVlQc2u6JH4BJGRMHxfzWh2ldvcAU5pUIFjUS2tfNUZZi5+pb2BoM4iIO6o aJ2uwgIOSBcYblzJsM0HEkfVpInoLwsHFvTW5uzCHD3UNtsyhYP4Jc9tRER85oKYKeGygz lgD7xrBpWTQ2ZSbayckGPjh1vw5mLjXsypZSjCLyWto/9M0mE42xwlX0fe3+KiTkxO//Hi YF+06p00puyFFgJitXkrVTl2dy5JwJGVNAwvY9ZkhHmFtNbcX51KFIdbIIoN/jrrFB6NF0 M0quj0tznRLo22R7ogDD4YptSQ7ugxsv6c4RSpG0lXGYfeoT/eL/9u1W8bIBdw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1732796362; a=rsa-sha256; cv=none; b=RNCgnfqjRNLHIEg/q3RJ5jFflMF0HoTHNVpP+LPqQn7fEM5eqtWxMY9xC4BxfS58g6s+5k Mdm6GJ9U0Mqi67V8W3l046wiqe9RcrbM5nfijFieNVq6j3XN1VyXlqgyD1KuOBxVl0/Ycl mtUjgXb4CdGZox7JV3m0b/p5PfZr/+tvhbyknWudBQhCpKACz+Rv2QbW2QiZgGs2WiIT3v aZM31UaCtqNwbvw5vZGpBIXfRnLCWn1sYsmzllfhE7myV9f7sq1XBom29z/umXAKDmuxkU yCCm3kzJuq7+ApituswfIOB/eeIMvXIpOXvws7Vmta0IqWf7y7AfWzjYiPl9ww== Received: from [192.168.0.88] (unknown [93.188.39.137]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: avg/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Xzb2w6b0szyZB; Thu, 28 Nov 2024 12:19:20 +0000 (UTC) (envelope-from avg@FreeBSD.org) Message-ID: <5e37b8a5-2bd2-49b5-9746-674bd26ad770@FreeBSD.org> Date: Thu, 28 Nov 2024 14:19:18 +0200 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird From: Andriy Gapon Subject: Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros] To: =?UTF-8?Q?Dag-Erling_Sm=C3=B8rgrav?= Cc: Mark Millard , Konstantin Belousov , Dimitry Andric , "jah@freebsd.org" , dougm@freebsd.org, Alan Somers , Mark Johnston , FreeBSD Current , Guido Falsi , Yasuhiro Kimura , ports@freebsd.org References: <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> <0690CFB1-6A6D-4B63-916C-BAB7F6256000@yahoo.com> <3660625A-0EE8-40DA-A248-EC18C734718C@yahoo.com> <865xoa2t6f.fsf@ltc.des.dev> <69A2E921-F5E3-40D2-977D-0964EE27349A@FreeBSD.org> <4AE5B316-D7EB-4290-8D52-7FBF244EA7A4@FreeBSD.org> <33D56E3E-6476-48E8-B115-B906629B8AF5@yahoo.com> <65d47ca6-b0b9-4c03-9e36-d0f2cf6b4937@FreeBSD.org> <86zflj1t6b.fsf@ltc.des.dev> Content-Language: en-US Autocrypt: addr=avg@FreeBSD.org; keydata= xsDNBGcKrHEBDADRvwQOK0b/yo4ys5cs6bOQMhEh4xtfbaZ/CU00cpPgUip3sOZCdrtMWlRC g25z97prxE9pKueZi+HXDhIPpa9xl14ghqF4oYScuJ1i18HyiOH2y5Q3Vv/TtFiSzicd3EAu QgS3jVidpgDSPDdj2Yz3UxYpZ+PuFl6nOnvCvqOFcjUlzKCyPaiN2b86l1Nscmhnc+zQ/faB erUOEFEDQbWMA5YfXi8HrbeR16hfRfGt7E0aMDlIj9FIPIq71UWMN9CimPgs4+rbNr1MAlLa z4GxSDhVYZEY5rqtCzr+PLXboRQWnaUwXl0/biw9enf17NHdYv1SNAFTX2eC4dZ3qBVI74dS PgNprm+PMfz+6Hhs/dAv+Nan5nVhg3EFIjYTiy0MnjMSq8uI0v0ykpAGAcJJ5xl6d23aLxgN 6f0z6pJRCO0hGPgU7UzvFD0MxJxmbzqdT1R51KDan1oD41b+tjl2LMBuCDCoB0U44Pu0zLdp xMfFTxCXtwIYKIUxwd28jwMAEQEAAc0eQW5kcml5IEdhcG9uIDxhdmdARnJlZUJTRC5vcmc+ wsENBBMBCAA3FiEEmXvSmjiQFHPVOpLnzDOt5NLj67sFAmcKrHEFCQeEzgACGwMECwkIBwUV CAkKCwUWAgMBAAAKCRDMM63k0uPru5tSDACFK15LLbq89RSQ6QMnjiIm1t/wYJyumb519MHu Dhzxx1lbr8oghf0RHtF6kYRLQPaW2VdToi74pRobd3CN4bhZKDLSL6WfTn17RfavDjL6Njwp KBo30CkOeYKWq1mDmo0xEoQj8cc7ybEZnus+YScZOpj8Ti4EFwhRt6SHer7YDb161IHKL8m4 MsCxpFSGEjbKj8Iul3Ri/fTOO8w14ivcuEEQIvJt4/+4YV5Az8G23wKzL/3aJ7SOT3oYGmR9 atBTmVO3DlODjM+rZLegd8SfLSPTcBTHspWE5duemIzZbEX3BP77r3Qx4Fo5Tkit3bG1XVar yPQato+sFGFEGifdE9USBQoAoOaaeZevwAWjDU0TIuCT0CUe0sKtQuNP4LRq0n9EEHOXBu9a CfdMhFUSkAZnuE7miSVwgPvoVNJ1stA37EXLN/sVsWik7wslTQ5vF81VpdGFiwoQPOe2XEKh ogcwGSnXbwv1gD4x+Gz/7Y+kFyr1NY+4/nSaeXVcS2fOwM0EZwqscgEMAMQTe6ypAmQe/TFO HqKD2hfFKdksTptKi6uEh8xIwct8G/0FBldDWXo9eu8CGr/ZrDg0/bAwJxbaLRQCMH19Gq2Y hLvZ1QK5GQJVzZKcqfxbF2LiDUTs6WkdOBIhGpdDy7p1xFrvqCGCtNFYHuGYm067EozibBSF BWAPstKu2FQuVHZNMOfs7p3OIz3Yfqu9woXDeg3/8G2qVQJINe+8EwXKlhgh4CyDbq7nAZoA kIu1SE9z9u3WI5mcNy/0dFmVUsFxBqRC3ewbvzie8tKyZ9yFOlaZPT0Y4nRBXQTI3mLZ8zQ8 mtrWK5OOmrJ02kdeO9RBXe+OMaUUWMf92ZIoBFb4HP6N+B+4N1y1OwULousfl7JRoYxA4MRL ls7E2sSoJvrEBTJB3Pc34xu8rsJ1A5V3NgN6djX8yEZYpTRkcmrBeWy/ofDqZPVqneAx0LRm eldDS9msXDW4KXODyPZ+9unvmHAcoH0xaBYaSH44CDZDQDg4LNcmbOvuu1TEXBJhjQARAQAB wsD8BBgBCAAmFiEEmXvSmjiQFHPVOpLnzDOt5NLj67sFAmcKrHMFCQeEzgACGwwACgkQzDOt 5NLj67sUCAv5AXqgWnYN9EblapMbZjkiqL8pZQ0GNqh+Pg9FwbyULxjtRTO6rD4D0IxizByb ef+neeUNyYlagt5nfKMysEr0SU/gHKCi8vyTF/63ukMrGUNGmJJxrndl5ZYKC6j6eX7twrZF L1Uvlmn6FnQ22red5kHO93fDjG4zaDIZvHfwj7kzjZ4tpC7Byinf88s14mdZeScc0PnU2hj4 UGYju/wg2FF4YxaZYhcmdTiRYY0Wx85XSMZv19pnn78sadEuRvfRd4JTmw++j1xGXeqQGWzz /CTG5/Ex9GAkQ02hZbmi236byDXoet4G8TEyOph9QFVkV9bNd0jQZaFZPGEj4PSPUYGAF7s5 xJaNGgctC3aZ7WjEv1FBoo44XCU4xcjJ1wZQUrHxRhx6TW0Jtcl0U9qfKFW30TSPo6RyiXuj X4ltWKAtjoXB8nUmEJckaz7IRu2b4pXDeazZuz5JBygUs10yJjDxh2vFQZo0KaBAPx9MZlPn gpPTjT15L8xGftEjQXF6 In-Reply-To: <86zflj1t6b.fsf@ltc.des.dev> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 28/11/2024 13:42, Dag-Erling Smørgrav wrote: > Andriy Gapon writes: >> FWIW, I am not sure if it's relevant but I am seeing a similar pattern >> of corruption on tmpfs although in a different context, on FreeBSD >> 13.3. > > Not relevant at all. In this case the file is not actually corrupted > but `install(1)` skips over some of it when copying because `SEEK_DATA` > is implemented incorrectly. Still could be relevant... I don't know the "true state" of my corrupted files, I only observe the consequences. And the files get some post-processing, then they are uploaded and originals are removed. So, the problem could be not during the write phase, but during the read phase of post-processing. -- Andriy Gapon From nobody Thu Nov 28 12:35:40 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XzbQC6zD3z5ffY8 for ; Thu, 28 Nov 2024 12:36:03 +0000 (UTC) (envelope-from rb@gid.co.uk) Received: from gid2.gid.co.uk (ns0.gid.co.uk [IPv6:2001:470:94de::240]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gid2.gid.co.uk", Issuer "gid2.gid.co.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XzbQB2Gfnz4fdv for ; Thu, 28 Nov 2024 12:36:02 +0000 (UTC) (envelope-from rb@gid.co.uk) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of rb@gid.co.uk designates 2001:470:94de::240 as permitted sender) smtp.mailfrom=rb@gid.co.uk; dmarc=pass (policy=none) header.from=gid.co.uk Received: from mx0.gid.co.uk (mx0.gid.co.uk [194.32.164.250]) by gid2.gid.co.uk (8.15.2/8.15.2) with ESMTP id 4ASCZtai029570; Thu, 28 Nov 2024 12:35:55 GMT (envelope-from rb@gid.co.uk) Received: from smtpclient.apple ([89.248.30.154]) by mx0.gid.co.uk (8.14.2/8.14.2) with ESMTP id 4ASCZoWj007414; Thu, 28 Nov 2024 12:35:50 GMT (envelope-from rb@gid.co.uk) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\)) Subject: Re: RFC: fixing PR#282995 From: Bob Bishop In-Reply-To: Date: Thu, 28 Nov 2024 12:35:40 +0000 Cc: FreeBSD CURRENT , Michael Proto Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Rick Macklem X-Mailer: Apple Mail (2.3776.700.51) X-Spamd-Result: default: False [-3.80 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; DMARC_POLICY_ALLOW(-0.50)[gid.co.uk,none]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; APPLE_MAILER_COMMON(0.00)[]; ARC_NA(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; TAGGED_RCPT(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; R_DKIM_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@FreeBSD.org]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; TO_DN_ALL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; RCPT_COUNT_THREE(0.00)[3] X-Rspamd-Queue-Id: 4XzbQB2Gfnz4fdv X-Spamd-Bar: --- Hi, > On 27 Nov 2024, at 21:56, Rick Macklem wrote: >=20 > Hi, >=20 > PR#282995 reports that the "-alldirs" export option is broken, > since it allows an export where the directory path is not a mount = point. >=20 > I'll admit I did not recall this semantic for -alldirs and I now see = it is only > documented in the "Examples" section of exports(5). >=20 > Looking at the code, it appears this was broken between releng1 and > releng2.0 (about 30years ago) when the call to mount(2) in mountd.c > was changed from using the path in the exports line to using = f_mntonname. > (The check for "it is a mount point" depended on mount(2) failing = because > the path was not a mount point.) >=20 > I do believe the semantic is a useful one, Why? > although making it that way > after 30years might be construed as a POLA violation? >=20 > So, what do others think I should do with this? > (A) - Patch mountd to enforce the "must be a mount point when -alldirs > is specified, plus update exports(5) to state this semantic = clearly. > or > (B) - Patch mountd so that it enforces "must be a mount point when = -alldirs > is specified, but only enabled via a new mountd command line = option. > --> ie. Leave the default as not enforced, but allow = enforcement based > on a new mountd option. > - Document this in both exports(5) and mountd(8). > or > ??? (C) - Patch mountd so that it enforces "must be a mount point when = -alldirs is specified, but provide a new mountd command line option to = restore the old behaviour. --> ie. Default as enforced, but allow an override based on a = new mountd option. - Document this in both exports(5) and mountd(8). I think that (A) is too POLA-unfriendly. > Thanks in advance for your comments, rick >=20 -- Bob Bishop rb@gid.co.uk From nobody Thu Nov 28 13:05:39 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xzc4S6gWwz5fhLp for ; Thu, 28 Nov 2024 13:05:44 +0000 (UTC) (envelope-from dclarke@blastwave.org) Received: from mail.oetec.com (mail.oetec.com [108.160.241.186]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "mail.oetec.com", Issuer "E6" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xzc4S1Cxmz4kJF for ; Thu, 28 Nov 2024 13:05:44 +0000 (UTC) (envelope-from dclarke@blastwave.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=blastwave.org header.s=default header.b=hAlCXcfE; spf=pass (mx1.freebsd.org: domain of dclarke@blastwave.org designates 108.160.241.186 as permitted sender) smtp.mailfrom=dclarke@blastwave.org; dmarc=pass (policy=quarantine) header.from=blastwave.org Received: from [172.16.35.3] (pool-99-253-118-250.cpe.net.cable.rogers.com [99.253.118.250]) (authenticated bits=0) by mail.oetec.com (8.17.1/8.17.1) with ESMTPSA id 4ASD5d8N026619 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT) for ; Thu, 28 Nov 2024 08:05:41 -0500 (EST) (envelope-from dclarke@blastwave.org) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=blastwave.org; s=default; t=1732799141; bh=rwYNqpusMvqLR/mcUHo5FVUgXHL0k65I13RTn1m3tFg=; h=Date:To:From:Subject; b=hAlCXcfEzANsT+3L3v0CW9IjGLQBZUL6b0XudoDOjuj8Btx0s/ESBwDVxiot1f8Kf yFOXN51ViGhAxp3B+DXebFfHabF5PjWisQinOuVNGkldEWRRgSrAB2UNi5AwpXwKZU Jg0Fa5uPP1YL/9PN2Ypt1ytdKTWxUEn4lI5u4XRYSCUqSYZ5Brq9jwp/7blbqig++p I1dfqri5nm2cr6XLaZjg5wjdCplWEQ9Ah0yzUs76h/s49IGSWW6YBliMov869FpJEq YiBtzSnC/XhPnVeh/sghVPdDgdK5nnjUZrLxsQoI4ILx0yxhI2ImUnHxAPDQmXN8Qp wUKg2W2yjqOrw== Message-ID: <5798b0db-bc73-476a-908a-dd1f071bfe43@blastwave.org> Date: Thu, 28 Nov 2024 08:05:39 -0500 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: Current FreeBSD Content-Language: en-CA From: Dennis Clarke Subject: zpools no longer exist after boot Organization: GENUNIX Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-oetec-MailScanner-Information: Please contact the ISP for more information X-oetec-MailScanner-ID: 4ASD5d8N026619 X-oetec-MailScanner: Found to be clean X-oetec-MailScanner-From: dclarke@blastwave.org X-Spam-Status: No X-Spamd-Result: default: False [-4.69 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[blastwave.org,quarantine]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; RCVD_IN_DNSWL_MED(-0.20)[108.160.241.186:from]; R_DKIM_ALLOW(-0.20)[blastwave.org:s=default]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; TO_DN_ALL(0.00)[]; HAS_ORG_HEADER(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[blastwave.org:+]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:812, ipnet:108.160.240.0/20, country:CA]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[] X-Rspamd-Queue-Id: 4Xzc4S1Cxmz4kJF X-Spamd-Bar: ---- This is a baffling problem wherein two zpools no longer exist after boot. This is : titan# uname -apKU FreeBSD titan 15.0-CURRENT FreeBSD 15.0-CURRENT #1 main-n273749-4b65481ac68a-dirty: Wed Nov 20 15:08:52 GMT 2024 root@titan:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG amd64 amd64 1500027 1500027 titan# titan# zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT t0 444G 91.2G 353G - - 27% 20% 1.00x ONLINE - titan# The *only* zpool that seems to exist in any reliable way is the little NVME based unit for booting. The other two zpools vanished and yet the devices exist just fine : titan# titan# camcontrol devlist at scbus0 target 0 lun 0 (pass0,ada0) at scbus1 target 0 lun 0 (pass1,ada1) at scbus2 target 0 lun 0 (ses0,pass2) at scbus6 target 0 lun 0 (ses1,pass3) at scbus7 target 0 lun 1 (pass4,nda0) at scbus8 target 0 lun 0 (da0,pass5) titan# titan# nvmecontrol devlist nvme0: SAMSUNG MZVKW512HMJP-000L7 nvme0ns1 (488386MB) titan# titan# zpool status t0 pool: t0 state: ONLINE status: Some supported and requested features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. scan: scrub repaired 0B in 00:00:44 with 0 errors on Wed Feb 7 09:56:40 2024 config: NAME STATE READ WRITE CKSUM t0 ONLINE 0 0 0 nda0p3 ONLINE 0 0 0 errors: No known data errors titan# Initially I thought the problem was related to cachefile being empty for these zpools. However if I set the cachefile to something reasonable then the cachefile property vanishes at a reboot. The file, of course, exists just fine : titan# zpool get cachefile proteus NAME PROPERTY VALUE SOURCE proteus cachefile - default titan# titan# zpool set cachefile="/var/log/zpool_cache" proteus titan# zpool get cachefile proteus NAME PROPERTY VALUE SOURCE proteus cachefile /var/log/zpool_cache local titan# ls -ladb /var/log/zpool_cache -rw-r--r-- 1 root wheel 1440 Nov 28 11:45 /var/log/zpool_cache titan# So there we have 1440 bytes of data in that file. titan# zpool set cachefile="/var/log/zpool_cache" t0 titan# zpool get cachefile t0 NAME PROPERTY VALUE SOURCE t0 cachefile /var/log/zpool_cache local titan# titan# ls -ladb /var/log/zpool_cache -rw-r--r-- 1 root wheel 2880 Nov 28 11:46 /var/log/zpool_cache titan# Now we have 2 * 1440 bytes = 2880 bytes of some zpool cache data. titan# zpool set cachefile="/var/log/zpool_cache" leaf titan# zpool get cachefile leaf NAME PROPERTY VALUE SOURCE leaf cachefile /var/log/zpool_cache local titan# titan# zpool get cachefile t0 NAME PROPERTY VALUE SOURCE t0 cachefile /var/log/zpool_cache local titan# titan# zpool get cachefile proteus NAME PROPERTY VALUE SOURCE proteus cachefile /var/log/zpool_cache local titan# titan# reboot From here on ... the only zpool that exists after boot is the local little NVME samsung unit. So here I can import those pools and then see that the cachefile property has been wiped out : titan# titan# zpool import proteus titan# zpool import leaf titan# titan# zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT leaf 18.2T 984K 18.2T - - 0% 0% 1.00x ONLINE - proteus 1.98T 361G 1.63T - - 1% 17% 1.00x ONLINE - t0 444G 91.2G 353G - - 27% 20% 1.00x ONLINE - titan# titan# zpool get cachefile leaf NAME PROPERTY VALUE SOURCE leaf cachefile - default titan# titan# zpool get cachefile proteus NAME PROPERTY VALUE SOURCE proteus cachefile - default titan# titan# zpool get cachefile t0 NAME PROPERTY VALUE SOURCE t0 cachefile - default titan# titan# ls -l /var/log/zpool_cache -rw-r--r-- 1 root wheel 4960 Nov 28 11:52 /var/log/zpool_cache titan# The cachefile exists and seems to have grown in size. However a reboot will once again provide nothing but the t0 pool. Baffled. Any thoughts would be welcome. -- -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken From nobody Thu Nov 28 13:49:59 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xzd3Z4sS7z5fjWX for ; Thu, 28 Nov 2024 13:50:02 +0000 (UTC) (envelope-from SRS0=O5G7=SX=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int.realworks.nl (smtp-relay-int.realworks.nl [194.109.157.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xzd3Z0Gqbz4pwp for ; Thu, 28 Nov 2024 13:50:01 +0000 (UTC) (envelope-from SRS0=O5G7=SX=klop.ws=ronald-lists@realworks.nl) Authentication-Results: mx1.freebsd.org; none Date: Thu, 28 Nov 2024 14:49:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1732801800; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=4e/WqyrGSAZkRsd70iWD9IaGjPiox64VskDd2gvIcYo=; b=GbF+kaq0ewHNGaUwpn4jNESriXgP9aAwsGWfURDT/YZlnMVTcTU878/dVNwqzYlmbWU9Cr A/h6hMyKplg6kSBYwJi8ejGxti2Wjzbl5Skj0dhDkBYouXkAGyAJx21sPHoPXfH/81tq9c jB0x7UPOzqPbS1MVWdXUPiTarE5NkDg3dmBmDaSsiaBua+wj6jWOc+BQFDaweK46VrjIMw tD/CqJvVQhLzKKNnISzRuuwc9oI8IMmX6eF6wZ5f6YA+Zg/1uaYmYsOj3wIlCyWsNDPyvo 8vVjHYIE1x+Uw1HuQOMkXsjFj4OCzcCGo2BOM1dVLXNGg/26rQUyViVsWQMgJg== From: Ronald Klop To: Dennis Clarke Cc: Current FreeBSD Message-ID: <1784014555.6851.1732801799724@localhost> In-Reply-To: <5798b0db-bc73-476a-908a-dd1f071bfe43@blastwave.org> Subject: Re: zpools no longer exist after boot List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_6850_62102694.1732801799721" X-Mailer: Realworks (730.92) Importance: Normal X-Priority: 3 (Normal) X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:3265, ipnet:194.109.0.0/16, country:NL] X-Rspamd-Queue-Id: 4Xzd3Z0Gqbz4pwp X-Spamd-Bar: ---- ------=_Part_6850_62102694.1732801799721 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Are the other disks available at the moment the boot process does zpool import? Regards, Ronald Van: Dennis Clarke Datum: 28 november 2024 14:06 Aan: Current FreeBSD Onderwerp: zpools no longer exist after boot > > > > This is a baffling problem wherein two zpools no longer exist after > boot. This is : > > titan# uname -apKU > FreeBSD titan 15.0-CURRENT FreeBSD 15.0-CURRENT #1 main-n273749-4b65481ac68a-dirty: Wed Nov 20 15:08:52 GMT 2024 root@titan:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG amd64 amd64 1500027 1500027 > titan# > > titan# zpool list > NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > t0 444G 91.2G 353G - - 27% 20% 1.00x ONLINE - > titan# > > The *only* zpool that seems to exist in any reliable way is the little > NVME based unit for booting. The other two zpools vanished and yet the > devices exist just fine : > > titan# > titan# camcontrol devlist > at scbus0 target 0 lun 0 (pass0,ada0) > at scbus1 target 0 lun 0 (pass1,ada1) > 0001=""> at scbus2 target 0 lun 0 (ses0,pass2) > 0001=""> at scbus6 target 0 lun 0 (ses1,pass3) > at scbus7 target 0 lun 1 (pass4,nda0) > at scbus8 target 0 lun 0 (da0,pass5) > titan# > titan# nvmecontrol devlist > nvme0: SAMSUNG MZVKW512HMJP-000L7 > nvme0ns1 (488386MB) > titan# > titan# zpool status t0 > pool: t0 > state: ONLINE > status: Some supported and requested features are not enabled on the pool. > The pool can still be used, but some features are unavailable. > action: Enable all features using 'zpool upgrade'. Once this is done, > the pool may no longer be accessible by software that does not support > the features. See zpool-features(7) for details. > scan: scrub repaired 0B in 00:00:44 with 0 errors on Wed Feb 7 09:56:40 2024 > config: > > NAME STATE READ WRITE CKSUM > t0 ONLINE 0 0 0 > nda0p3 ONLINE 0 0 0 > > errors: No known data errors > titan# > > > Initially I thought the problem was related to cachefile being empty for > these zpools. However if I set the cachefile to something reasonable > then the cachefile property vanishes at a reboot. The file, of course, exists just fine : > > titan# zpool get cachefile proteus > NAME PROPERTY VALUE SOURCE > proteus cachefile - default > titan# > titan# zpool set cachefile="/var/log/zpool_cache" proteus > titan# zpool get cachefile proteus > NAME PROPERTY VALUE SOURCE > proteus cachefile /var/log/zpool_cache local > titan# ls -ladb /var/log/zpool_cache > -rw-r--r-- 1 root wheel 1440 Nov 28 11:45 /var/log/zpool_cache > titan# > > So there we have 1440 bytes of data in that file. > > titan# zpool set cachefile="/var/log/zpool_cache" t0 > titan# zpool get cachefile t0 > NAME PROPERTY VALUE SOURCE > t0 cachefile /var/log/zpool_cache local > titan# > titan# ls -ladb /var/log/zpool_cache > -rw-r--r-- 1 root wheel 2880 Nov 28 11:46 /var/log/zpool_cache > titan# > > Now we have 2 * 1440 bytes = 2880 bytes of some zpool cache data. > > titan# zpool set cachefile="/var/log/zpool_cache" leaf > titan# zpool get cachefile leaf > NAME PROPERTY VALUE SOURCE > leaf cachefile /var/log/zpool_cache local > titan# > titan# zpool get cachefile t0 > NAME PROPERTY VALUE SOURCE > t0 cachefile /var/log/zpool_cache local > titan# > titan# zpool get cachefile proteus > NAME PROPERTY VALUE SOURCE > proteus cachefile /var/log/zpool_cache local > titan# > titan# reboot > > From here on ... the only zpool that exists after boot is the local > little NVME samsung unit. > > So here I can import those pools and then see that the cachefile property has been wiped out : > > titan# > titan# zpool import proteus > titan# zpool import leaf > titan# > titan# zpool list > NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > leaf 18.2T 984K 18.2T - - 0% 0% 1.00x ONLINE - > proteus 1.98T 361G 1.63T - - 1% 17% 1.00x ONLINE - > t0 444G 91.2G 353G - - 27% 20% 1.00x ONLINE - > titan# > titan# zpool get cachefile leaf > NAME PROPERTY VALUE SOURCE > leaf cachefile - default > titan# > titan# zpool get cachefile proteus > NAME PROPERTY VALUE SOURCE > proteus cachefile - default > titan# > titan# zpool get cachefile t0 > NAME PROPERTY VALUE SOURCE > t0 cachefile - default > titan# > titan# ls -l /var/log/zpool_cache > -rw-r--r-- 1 root wheel 4960 Nov 28 11:52 /var/log/zpool_cache > titan# > > The cachefile exists and seems to have grown in size. > > However a reboot will once again provide nothing but the t0 pool. > > Baffled. > > Any thoughts would be welcome. > > -- > -- > Dennis Clarke > RISC-V/SPARC/PPC/ARM/CISC > UNIX and Linux spoken > > > > > ------=_Part_6850_62102694.1732801799721 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit Are the other disks available at the moment the boot process does zpool import?

Regards,
Ronald

Van: Dennis Clarke <dclarke@blastwave.org>
Datum: 28 november 2024 14:06
Aan: Current FreeBSD <freebsd-current@freebsd.org>
Onderwerp: zpools no longer exist after boot


This is a baffling problem wherein two zpools no longer exist after
boot. This is :

titan# uname -apKU
FreeBSD titan 15.0-CURRENT FreeBSD 15.0-CURRENT #1 main-n273749-4b65481ac68a-dirty: Wed Nov 20 15:08:52 GMT 2024 root@titan:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG amd64 amd64 1500027 1500027
titan#

titan# zpool list
NAME   SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP HEALTH  ALTROOT
t0     444G  91.2G   353G        -         -    27%    20%  1.00x ONLINE  -
titan#

The *only* zpool that seems to exist in any reliable way is the little
NVME based unit for booting. The other two zpools vanished and yet the
devices exist just fine :

titan#
titan# camcontrol devlist
       at scbus0 target 0 lun 0 (pass0,ada0)
       at scbus1 target 0 lun 0 (pass1,ada1)
  at scbus2 target 0 lun 0 (ses0,pass2)
  at scbus6 target 0 lun 0 (ses1,pass3)
 at scbus7 target 0 lun 1 (pass4,nda0)
            at scbus8 target 0 lun 0 (da0,pass5)
titan#
titan# nvmecontrol devlist
  nvme0: SAMSUNG MZVKW512HMJP-000L7
     nvme0ns1 (488386MB)
titan#
titan# zpool status t0
   pool: t0
  state: ONLINE
status: Some supported and requested features are not enabled on the pool.
         The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
         the pool may no longer be accessible by software that does not support
         the features. See zpool-features(7) for details.
   scan: scrub repaired 0B in 00:00:44 with 0 errors on Wed Feb  7 09:56:40 2024
config:

         NAME        STATE     READ WRITE CKSUM
         t0          ONLINE       0     0     0
           nda0p3    ONLINE       0     0     0

errors: No known data errors
titan#


Initially I thought the problem was related to cachefile being empty for
these zpools. However if I set the cachefile to something reasonable
then the cachefile property vanishes at a reboot.  The file, of course, exists just fine :

titan# zpool get cachefile proteus
NAME     PROPERTY   VALUE      SOURCE
proteus  cachefile  -          default
titan#
titan# zpool set cachefile="/var/log/zpool_cache" proteus
titan# zpool get cachefile proteus
NAME     PROPERTY   VALUE                 SOURCE
proteus  cachefile  /var/log/zpool_cache  local
titan# ls -ladb /var/log/zpool_cache
-rw-r--r--  1 root wheel 1440 Nov 28 11:45 /var/log/zpool_cache
titan#

So there we have 1440 bytes of data in that file.

titan# zpool set cachefile="/var/log/zpool_cache" t0
titan# zpool get cachefile t0
NAME  PROPERTY   VALUE                 SOURCE
t0    cachefile  /var/log/zpool_cache  local
titan#
titan# ls -ladb /var/log/zpool_cache
-rw-r--r--  1 root wheel 2880 Nov 28 11:46 /var/log/zpool_cache
titan#

Now we have 2 * 1440 bytes = 2880 bytes of some zpool cache data.

titan# zpool set cachefile="/var/log/zpool_cache" leaf
titan# zpool get cachefile leaf
NAME  PROPERTY   VALUE                 SOURCE
leaf  cachefile  /var/log/zpool_cache  local
titan#
titan# zpool get cachefile t0
NAME  PROPERTY   VALUE                 SOURCE
t0    cachefile  /var/log/zpool_cache  local
titan#
titan# zpool get cachefile proteus
NAME     PROPERTY   VALUE                 SOURCE
proteus  cachefile  /var/log/zpool_cache  local
titan#
titan# reboot

 From here on ... the only zpool that exists after boot is the local
little NVME samsung unit.

So here I can import those pools and then see that the cachefile property has been wiped out :

titan#
titan# zpool import proteus
titan# zpool import leaf
titan#
titan# zpool list
NAME      SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP HEALTH  ALTROOT
leaf     18.2T   984K  18.2T        -         -     0%     0%  1.00x ONLINE  -
proteus  1.98T   361G  1.63T        -         -     1%    17%  1.00x ONLINE  -
t0        444G  91.2G   353G        -         -    27%    20%  1.00x ONLINE  -
titan#
titan# zpool get cachefile leaf
NAME  PROPERTY   VALUE      SOURCE
leaf  cachefile  -          default
titan#
titan# zpool get cachefile proteus
NAME     PROPERTY   VALUE      SOURCE
proteus  cachefile  -          default
titan#
titan# zpool get cachefile t0
NAME  PROPERTY   VALUE      SOURCE
t0    cachefile  -          default
titan#
titan# ls -l /var/log/zpool_cache
-rw-r--r--  1 root wheel 4960 Nov 28 11:52 /var/log/zpool_cache
titan#

The cachefile exists and seems to have grown in size.

However a reboot will once again provide nothing but the t0 pool.

Baffled.

Any thoughts would be welcome.

-- 
--
Dennis Clarke
RISC-V/SPARC/PPC/ARM/CISC
UNIX and Linux spoken




------=_Part_6850_62102694.1732801799721-- From nobody Thu Nov 28 13:52:35 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xzd6m1SdXz5fkcD for ; Thu, 28 Nov 2024 13:52:48 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ej1-f43.google.com (mail-ej1-f43.google.com [209.85.218.43]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xzd6l6gQHz4rBK for ; Thu, 28 Nov 2024 13:52:47 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ej1-f43.google.com with SMTP id a640c23a62f3a-aa1e6ecd353so49638366b.1 for ; Thu, 28 Nov 2024 05:52:47 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732801966; x=1733406766; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=kkm2deqqq4BGJzJRtiy/u4PLzZArKAB5Gkvo/D29CaE=; b=EkVWnxxitXE/NODhjzJJtp3pOotVP8iVM2xZDNwWJxW9/2vsMr0tNGU6ruJ9Vrg1+b ELSsVa9/Lkpd+IMY+AIEOky3mPLe8XcmbbqUkPKD9BpQUEfY7jmAwN8keNBf0HnxZzAA pLcLEyIGLr7sFchQwSC+xyuB8KwTGSDyHrPW30yMT3ScxAXEUJaKkvlvoqAAJynBLX9L r6nL5f03BvF6fIZVzd9fkMQuzeHz/ThyGoiBMFwi1zoCzORMil3+2kt+3hTBA4kijipM Up/lXVgdDbjDFbtzbJ9a5uvZU1VLiStvlexfaxdm83YpK81pD5bcDUZF026E+XkxuZ4q +hPA== X-Gm-Message-State: AOJu0Yw2WbjwjygF2Tm2PWOkN5EKay+Me41W8YfEmYN7miE1/+xCuGJ9 Z7yXlRgmtxxmxmj/HNyCzUyOFyy40UwEWVAhb1EV1VdSsIv2T76xxQhvWL7QbwxEZ6mvisXaDRI TvZ1T+JQYC7q7j7eCVUk5kueywFBE/w== X-Gm-Gg: ASbGncuLxEOEXH1VXQQm1AY2nkdrGxD/GZ9mT0vnjtSce5UlInH/13/BB2O95xVh7Ji 7iaeYo9ddI+xsHq+XOqSMY9osy2zV9kWqa6aYP0LzcCXNTDugMWTry3AqFJ0PibNu X-Google-Smtp-Source: AGHT+IFVazhxOXfm/uBW5b3RwUVbDIBJ7iDkFrbg1zD90k30Gzt39JK2r31aX9vmIHf2gwyAkm1lK/5k4FjaVQGco88= X-Received: by 2002:a05:6402:3553:b0:5cf:f361:1c11 with SMTP id 4fb4d7f45d1cf-5d080c07623mr7653162a12.20.1732801966172; Thu, 28 Nov 2024 05:52:46 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 References: <5798b0db-bc73-476a-908a-dd1f071bfe43@blastwave.org> In-Reply-To: <5798b0db-bc73-476a-908a-dd1f071bfe43@blastwave.org> From: Alan Somers Date: Thu, 28 Nov 2024 07:52:35 -0600 Message-ID: Subject: Re: zpools no longer exist after boot To: Dennis Clarke Cc: Current FreeBSD Content-Type: multipart/alternative; boundary="000000000000a8f3c40627f96554" X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4Xzd6l6gQHz4rBK X-Spamd-Bar: ---- --000000000000a8f3c40627f96554 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Nov 28, 2024, 7:06=E2=80=AFAM Dennis Clarke = wrote: > > This is a baffling problem wherein two zpools no longer exist after > boot. This is : > > titan# uname -apKU > FreeBSD titan 15.0-CURRENT FreeBSD 15.0-CURRENT #1 > main-n273749-4b65481ac68a-dirty: Wed Nov 20 15:08:52 GMT 2024 > root@titan:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG amd64 amd64 > 1500027 1500027 > titan# > > titan# zpool list > NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP > HEALTH ALTROOT > t0 444G 91.2G 353G - - 27% 20% 1.00x > ONLINE - > titan# > > The *only* zpool that seems to exist in any reliable way is the little > NVME based unit for booting. The other two zpools vanished and yet the > devices exist just fine : > > titan# > titan# camcontrol devlist > at scbus0 target 0 lun 0 (pass0,ada0) > at scbus1 target 0 lun 0 (pass1,ada1) > at scbus2 target 0 lun 0 (ses0,pass2) > at scbus6 target 0 lun 0 (ses1,pass3) > at scbus7 target 0 lun 1 > (pass4,nda0) > at scbus8 target 0 lun 0 (da0,pass5) > titan# > titan# nvmecontrol devlist > nvme0: SAMSUNG MZVKW512HMJP-000L7 > nvme0ns1 (488386MB) > titan# > titan# zpool status t0 > pool: t0 > state: ONLINE > status: Some supported and requested features are not enabled on the pool= . > The pool can still be used, but some features are unavailable. > action: Enable all features using 'zpool upgrade'. Once this is done, > the pool may no longer be accessible by software that does not > support > the features. See zpool-features(7) for details. > scan: scrub repaired 0B in 00:00:44 with 0 errors on Wed Feb 7 > 09:56:40 2024 > config: > > NAME STATE READ WRITE CKSUM > t0 ONLINE 0 0 0 > nda0p3 ONLINE 0 0 0 > > errors: No known data errors > titan# > > > Initially I thought the problem was related to cachefile being empty for > these zpools. However if I set the cachefile to something reasonable > then the cachefile property vanishes at a reboot. The file, of course, > exists just fine : > > titan# zpool get cachefile proteus > NAME PROPERTY VALUE SOURCE > proteus cachefile - default > titan# > titan# zpool set cachefile=3D"/var/log/zpool_cache" proteus > titan# zpool get cachefile proteus > NAME PROPERTY VALUE SOURCE > proteus cachefile /var/log/zpool_cache local > titan# ls -ladb /var/log/zpool_cache > -rw-r--r-- 1 root wheel 1440 Nov 28 11:45 /var/log/zpool_cache > titan# > > So there we have 1440 bytes of data in that file. > > titan# zpool set cachefile=3D"/var/log/zpool_cache" t0 > titan# zpool get cachefile t0 > NAME PROPERTY VALUE SOURCE > t0 cachefile /var/log/zpool_cache local > titan# > titan# ls -ladb /var/log/zpool_cache > -rw-r--r-- 1 root wheel 2880 Nov 28 11:46 /var/log/zpool_cache > titan# > > Now we have 2 * 1440 bytes =3D 2880 bytes of some zpool cache data. > > titan# zpool set cachefile=3D"/var/log/zpool_cache" leaf > titan# zpool get cachefile leaf > NAME PROPERTY VALUE SOURCE > leaf cachefile /var/log/zpool_cache local > titan# > titan# zpool get cachefile t0 > NAME PROPERTY VALUE SOURCE > t0 cachefile /var/log/zpool_cache local > titan# > titan# zpool get cachefile proteus > NAME PROPERTY VALUE SOURCE > proteus cachefile /var/log/zpool_cache local > titan# > titan# reboot > > From here on ... the only zpool that exists after boot is the local > little NVME samsung unit. > > So here I can import those pools and then see that the cachefile > property has been wiped out : > > titan# > titan# zpool import proteus > titan# zpool import leaf > titan# > titan# zpool list > NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP > HEALTH ALTROOT > leaf 18.2T 984K 18.2T - - 0% 0% 1.00x > ONLINE - > proteus 1.98T 361G 1.63T - - 1% 17% 1.00x > ONLINE - > t0 444G 91.2G 353G - - 27% 20% 1.00x > ONLINE - > titan# > titan# zpool get cachefile leaf > NAME PROPERTY VALUE SOURCE > leaf cachefile - default > titan# > titan# zpool get cachefile proteus > NAME PROPERTY VALUE SOURCE > proteus cachefile - default > titan# > titan# zpool get cachefile t0 > NAME PROPERTY VALUE SOURCE > t0 cachefile - default > titan# > titan# ls -l /var/log/zpool_cache > -rw-r--r-- 1 root wheel 4960 Nov 28 11:52 /var/log/zpool_cache > titan# > > The cachefile exists and seems to have grown in size. > > However a reboot will once again provide nothing but the t0 pool. > > Baffled. > > Any thoughts would be welcome. > > -- > -- > Dennis Clarke > RISC-V/SPARC/PPC/ARM/CISC > UNIX and Linux spoken > Do you have zfs_enable=3D"YES" set in /etc/rc.conf? If not then nothing wil= l get imported. Regarding the cachefile property, it's expected that "zpool import" will change it, unless you do "zpool import -O cachefile=3Dwhatever". > --000000000000a8f3c40627f96554 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Thu, Nov 28, 2024, 7:06=E2=80=AFAM Dennis Clarke <dclarke@blastwave.org> wrote:
=

This is a baffling problem wherein two zpools no longer exist after
boot. This is :

titan# uname -apKU
FreeBSD titan 15.0-CURRENT FreeBSD 15.0-CURRENT #1
main-n273749-4b65481ac68a-dirty: Wed Nov 20 15:08:52 GMT 2024
root@titan:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG amd64 amd64 1500027 1500027
titan#

titan# zpool list
NAME=C2=A0 =C2=A0SIZE=C2=A0 ALLOC=C2=A0 =C2=A0FREE=C2=A0 CKPOINT=C2=A0 EXPA= NDSZ=C2=A0 =C2=A0FRAG=C2=A0 =C2=A0 CAP=C2=A0 DEDUP
HEALTH=C2=A0 ALTROOT
t0=C2=A0 =C2=A0 =C2=A0444G=C2=A0 91.2G=C2=A0 =C2=A0353G=C2=A0 =C2=A0 =C2=A0= =C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-=C2=A0 =C2=A0 27%=C2=A0 =C2=A0 = 20%=C2=A0 1.00x
ONLINE=C2=A0 -
titan#

The *only* zpool that seems to exist in any reliable way is the little
NVME based unit for booting. The other two zpools vanished and yet the
devices exist just fine :

titan#
titan# camcontrol devlist
<ST20000NM007D-3DJ103 SN03>=C2=A0 =C2=A0 =C2=A0 =C2=A0 at scbus0 targ= et 0 lun 0 (pass0,ada0)
<ST20000NM007D-3DJ103 SN03>=C2=A0 =C2=A0 =C2=A0 =C2=A0 at scbus1 targ= et 0 lun 0 (pass1,ada1)
<AHCI SGPIO Enclosure 2.00 0001>=C2=A0 =C2=A0at scbus2 target 0 lun 0= (ses0,pass2)
<AHCI SGPIO Enclosure 2.00 0001>=C2=A0 =C2=A0at scbus6 target 0 lun 0= (ses1,pass3)
<SAMSUNG MZVKW512HMJP-000L7 6L6QCXA7>=C2=A0 at scbus7 target 0 lun 1 = (pass4,nda0)
<FREEBSD CTLDISK 0001>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0at scbus8 target 0 lun 0 (da0,pass5)
titan#
titan# nvmecontrol devlist
=C2=A0 nvme0: SAMSUNG MZVKW512HMJP-000L7
=C2=A0 =C2=A0 =C2=A0nvme0ns1 (488386MB)
titan#
titan# zpool status t0
=C2=A0 =C2=A0pool: t0
=C2=A0 state: ONLINE
status: Some supported and requested features are not enabled on the pool.<= br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0The pool can still be used, but some feat= ures are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is don= e,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0the pool may no longer be accessible by s= oftware that does not
support
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0the features. See zpool-features(7) for d= etails.
=C2=A0 =C2=A0scan: scrub repaired 0B in 00:00:44 with 0 errors on Wed Feb= =C2=A0 7
09:56:40 2024
config:

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0NAME=C2=A0 =C2=A0 =C2=A0 =C2=A0 STATE=C2= =A0 =C2=A0 =C2=A0READ WRITE CKSUM
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0t0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ONLI= NE=C2=A0 =C2=A0 =C2=A0 =C2=A00=C2=A0 =C2=A0 =C2=A00=C2=A0 =C2=A0 =C2=A00 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0nda0p3=C2=A0 =C2=A0 ONLINE=C2=A0 = =C2=A0 =C2=A0 =C2=A00=C2=A0 =C2=A0 =C2=A00=C2=A0 =C2=A0 =C2=A00

errors: No known data errors
titan#


Initially I thought the problem was related to cachefile being empty for these zpools. However if I set the cachefile to something reasonable
then the cachefile property vanishes at a reboot.=C2=A0 The file, of course= ,
exists just fine :

titan# zpool get cachefile proteus
NAME=C2=A0 =C2=A0 =C2=A0PROPERTY=C2=A0 =C2=A0VALUE=C2=A0 =C2=A0 =C2=A0 SOUR= CE
proteus=C2=A0 cachefile=C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 default titan#
titan# zpool set cachefile=3D"/var/log/zpool_cache" proteus
titan# zpool get cachefile proteus
NAME=C2=A0 =C2=A0 =C2=A0PROPERTY=C2=A0 =C2=A0VALUE=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0SOURCE
proteus=C2=A0 cachefile=C2=A0 /var/log/zpool_cache=C2=A0 local
titan# ls -ladb /var/log/zpool_cache
-rw-r--r--=C2=A0 1 root wheel 1440 Nov 28 11:45 /var/log/zpool_cache
titan#

So there we have 1440 bytes of data in that file.

titan# zpool set cachefile=3D"/var/log/zpool_cache" t0
titan# zpool get cachefile t0
NAME=C2=A0 PROPERTY=C2=A0 =C2=A0VALUE=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0SOURCE
t0=C2=A0 =C2=A0 cachefile=C2=A0 /var/log/zpool_cache=C2=A0 local
titan#
titan# ls -ladb /var/log/zpool_cache
-rw-r--r--=C2=A0 1 root wheel 2880 Nov 28 11:46 /var/log/zpool_cache
titan#

Now we have 2 * 1440 bytes =3D 2880 bytes of some zpool cache data.

titan# zpool set cachefile=3D"/var/log/zpool_cache" leaf
titan# zpool get cachefile leaf
NAME=C2=A0 PROPERTY=C2=A0 =C2=A0VALUE=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0SOURCE
leaf=C2=A0 cachefile=C2=A0 /var/log/zpool_cache=C2=A0 local
titan#
titan# zpool get cachefile t0
NAME=C2=A0 PROPERTY=C2=A0 =C2=A0VALUE=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0SOURCE
t0=C2=A0 =C2=A0 cachefile=C2=A0 /var/log/zpool_cache=C2=A0 local
titan#
titan# zpool get cachefile proteus
NAME=C2=A0 =C2=A0 =C2=A0PROPERTY=C2=A0 =C2=A0VALUE=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0SOURCE
proteus=C2=A0 cachefile=C2=A0 /var/log/zpool_cache=C2=A0 local
titan#
titan# reboot

=C2=A0From here on ... the only zpool that exists after boot is the local little NVME samsung unit.

So here I can import those pools and then see that the cachefile
property has been wiped out :

titan#
titan# zpool import proteus
titan# zpool import leaf
titan#
titan# zpool list
NAME=C2=A0 =C2=A0 =C2=A0 SIZE=C2=A0 ALLOC=C2=A0 =C2=A0FREE=C2=A0 CKPOINT=C2= =A0 EXPANDSZ=C2=A0 =C2=A0FRAG=C2=A0 =C2=A0 CAP=C2=A0 DEDUP
HEALTH=C2=A0 ALTROOT
leaf=C2=A0 =C2=A0 =C2=A018.2T=C2=A0 =C2=A0984K=C2=A0 18.2T=C2=A0 =C2=A0 =C2= =A0 =C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-=C2=A0 =C2=A0 =C2=A00%=C2=A0= =C2=A0 =C2=A00%=C2=A0 1.00x
ONLINE=C2=A0 -
proteus=C2=A0 1.98T=C2=A0 =C2=A0361G=C2=A0 1.63T=C2=A0 =C2=A0 =C2=A0 =C2=A0= -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-=C2=A0 =C2=A0 =C2=A01%=C2=A0 =C2=A0 17= %=C2=A0 1.00x
ONLINE=C2=A0 -
t0=C2=A0 =C2=A0 =C2=A0 =C2=A0 444G=C2=A0 91.2G=C2=A0 =C2=A0353G=C2=A0 =C2= =A0 =C2=A0 =C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-=C2=A0 =C2=A0 27%=C2= =A0 =C2=A0 20%=C2=A0 1.00x
ONLINE=C2=A0 -
titan#
titan# zpool get cachefile leaf
NAME=C2=A0 PROPERTY=C2=A0 =C2=A0VALUE=C2=A0 =C2=A0 =C2=A0 SOURCE
leaf=C2=A0 cachefile=C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 default
titan#
titan# zpool get cachefile proteus
NAME=C2=A0 =C2=A0 =C2=A0PROPERTY=C2=A0 =C2=A0VALUE=C2=A0 =C2=A0 =C2=A0 SOUR= CE
proteus=C2=A0 cachefile=C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 default titan#
titan# zpool get cachefile t0
NAME=C2=A0 PROPERTY=C2=A0 =C2=A0VALUE=C2=A0 =C2=A0 =C2=A0 SOURCE
t0=C2=A0 =C2=A0 cachefile=C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 default=
titan#
titan# ls -l /var/log/zpool_cache
-rw-r--r--=C2=A0 1 root wheel 4960 Nov 28 11:52 /var/log/zpool_cache
titan#

The cachefile exists and seems to have grown in size.

However a reboot will once again provide nothing but the t0 pool.

Baffled.

Any thoughts would be welcome.

--
--
Dennis Clarke
RISC-V/SPARC/PPC/ARM/CISC
UNIX and Linux spoken

Do you have zfs_enable=3D"YES" set in /etc/r= c.conf? If not then nothing will get imported.=C2=A0

Regarding the cachefile property, it's exp= ected that "zpool import" will change it, unless you do "zpo= ol import -O cachefile=3Dwhatever".
--000000000000a8f3c40627f96554-- From nobody Thu Nov 28 13:58:29 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XzdFN0RCtz5fkbb for ; Thu, 28 Nov 2024 13:58:32 +0000 (UTC) (envelope-from SRS0=O5G7=SX=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int.realworks.nl (smtp-relay-int.realworks.nl [194.109.157.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XzdFM2SvFz4sT2 for ; Thu, 28 Nov 2024 13:58:31 +0000 (UTC) (envelope-from SRS0=O5G7=SX=klop.ws=ronald-lists@realworks.nl) Authentication-Results: mx1.freebsd.org; none Date: Thu, 28 Nov 2024 14:58:29 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1732802309; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=gMIe/uasxtbsqZL0vvAZcxrB5N8fUd3nvlt4+l7Jkjc=; b=kDyVGQnk/ecaGnu4sDPWFvUM0OLBwdgpdRsQcgK6Io0SIlL3+OEur33kcu6SiHme2YrjQB FBaumEMz+B4fhk3Eh6femtpoGqIDpwyqjWfdXb6Ztrg5SZzqPgqj1VLjAmrvfu0DXGet39 i/pGipQekQymXco6O0gd/wT/PHrdA79gStn9dqaGPMeG0lxJ+zox+4myrnxvpCx4gZ08cA ZvUy1IBV9llXdNCjdIAz+fBHMwFWgQTIP8Tu5T+3jLtQla8sxGNlL/oh3lVXyB58lkXghD UgakRUsSv6/+X9F6Zrmh2O4yx4SY8TUEdF5noPNXf+asYm25uS/R7zK3jrP/iw== From: Ronald Klop To: Ronald Klop Cc: Current FreeBSD , Dennis Clarke Message-ID: <1764191396.6959.1732802309600@localhost> In-Reply-To: <1784014555.6851.1732801799724@localhost> Subject: Re: zpools no longer exist after boot List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_6958_1940271980.1732802309596" X-Mailer: Realworks (730.92) Importance: Normal X-Priority: 3 (Normal) X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:3265, ipnet:194.109.0.0/16, country:NL] X-Rspamd-Queue-Id: 4XzdFM2SvFz4sT2 X-Spamd-Bar: ---- ------=_Part_6958_1940271980.1732802309596 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Btw: The /etc/rc.d/zpool script looks into these cachefiles: for cachefile in /etc/zfs/zpool.cache /boot/zfs/zpool.cache; do I didn=E2=80=99t check where the cachefile pool property is used.=20 Hope this helps resolving the issue. Or maybe helps you to provide more inf= ormation about your setup.=20 Regards, Ronald.=20 Van: Ronald Klop Datum: 28 november 2024 14:50 Aan: Dennis Clarke CC: Current FreeBSD Onderwerp: Re: zpools no longer exist after boot >=20 >=20 >=20 > Are the other disks available at the moment the boot process does zpool i= mport? >=20 > Regards, > Ronald >=20 > Van: Dennis Clarke=20 > Datum: 28 november 2024 14:06 > Aan: Current FreeBSD=20 > Onderwerp: zpools no longer exist after boot >=20 >>=20 >> This is a baffling problem wherein two zpools no longer exist after >> boot. This is : >>=20 >> titan# uname -apKU >> FreeBSD titan 15.0-CURRENT FreeBSD 15.0-CURRENT #1 main-n273749-4b65481a= c68a-dirty: Wed Nov 20 15:08:52 GMT 2024 root@titan:/usr/obj/usr/src/amd64.= amd64/sys/GENERIC-NODEBUG amd64 amd64 1500027 1500027 >> titan# >>=20 >> titan# zpool list >> NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH= ALTROOT >> t0 444G 91.2G 353G - - 27% 20% 1.00x ONLINE= - >> titan# >>=20 >> The *only* zpool that seems to exist in any reliable way is the little >> NVME based unit for booting. The other two zpools vanished and yet the >> devices exist just fine : >>=20 >> titan# >> titan# camcontrol devlist >> at scbus0 target 0 lun 0 (pass0,ada0) >> at scbus1 target 0 lun 0 (pass1,ada1) >> at scbus2 target 0 lun 0 (ses0,pass2) >> at scbus6 target 0 lun 0 (ses1,pass3) >> at scbus7 target 0 lun 1 (pass4,nda0) >> at scbus8 target 0 lun 0 (da0,pass5) >> titan# >> titan# nvmecontrol devlist >> nvme0: SAMSUNG MZVKW512HMJP-000L7 >> nvme0ns1 (488386MB) >> titan# >> titan# zpool status t0 >> pool: t0 >> state: ONLINE >> status: Some supported and requested features are not enabled on the poo= l. >> The pool can still be used, but some features are unavailable. >> action: Enable all features using 'zpool upgrade'. Once this is done, >> the pool may no longer be accessible by software that does not = support >> the features. See zpool-features(7) for details. >> scan: scrub repaired 0B in 00:00:44 with 0 errors on Wed Feb 7 09:56= :40 2024 >> config: >>=20 >> NAME STATE READ WRITE CKSUM >> t0 ONLINE 0 0 0 >> nda0p3 ONLINE 0 0 0 >>=20 >> errors: No known data errors >> titan# >>=20 >>=20 >> Initially I thought the problem was related to cachefile being empty for >> these zpools. However if I set the cachefile to something reasonable >> then the cachefile property vanishes at a reboot. The file, of course, = exists just fine : >>=20 >> titan# zpool get cachefile proteus >> NAME PROPERTY VALUE SOURCE >> proteus cachefile - default >> titan# >> titan# zpool set cachefile=3D"/var/log/zpool_cache" proteus >> titan# zpool get cachefile proteus >> NAME PROPERTY VALUE SOURCE >> proteus cachefile /var/log/zpool_cache local >> titan# ls -ladb /var/log/zpool_cache >> -rw-r--r-- 1 root wheel 1440 Nov 28 11:45 /var/log/zpool_cache >> titan# >>=20 >> So there we have 1440 bytes of data in that file. >>=20 >> titan# zpool set cachefile=3D"/var/log/zpool_cache" t0 >> titan# zpool get cachefile t0 >> NAME PROPERTY VALUE SOURCE >> t0 cachefile /var/log/zpool_cache local >> titan# >> titan# ls -ladb /var/log/zpool_cache >> -rw-r--r-- 1 root wheel 2880 Nov 28 11:46 /var/log/zpool_cache >> titan# >>=20 >> Now we have 2 * 1440 bytes =3D 2880 bytes of some zpool cache data. >>=20 >> titan# zpool set cachefile=3D"/var/log/zpool_cache" leaf >> titan# zpool get cachefile leaf >> NAME PROPERTY VALUE SOURCE >> leaf cachefile /var/log/zpool_cache local >> titan# >> titan# zpool get cachefile t0 >> NAME PROPERTY VALUE SOURCE >> t0 cachefile /var/log/zpool_cache local >> titan# >> titan# zpool get cachefile proteus >> NAME PROPERTY VALUE SOURCE >> proteus cachefile /var/log/zpool_cache local >> titan# >> titan# reboot >>=20 >> From here on ... the only zpool that exists after boot is the local >> little NVME samsung unit. >>=20 >> So here I can import those pools and then see that the cachefile propert= y has been wiped out : >>=20 >> titan# >> titan# zpool import proteus >> titan# zpool import leaf >> titan# >> titan# zpool list >> NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEA= LTH ALTROOT >> leaf 18.2T 984K 18.2T - - 0% 0% 1.00x ONL= INE - >> proteus 1.98T 361G 1.63T - - 1% 17% 1.00x ONL= INE - >> t0 444G 91.2G 353G - - 27% 20% 1.00x ONL= INE - >> titan# >> titan# zpool get cachefile leaf >> NAME PROPERTY VALUE SOURCE >> leaf cachefile - default >> titan# >> titan# zpool get cachefile proteus >> NAME PROPERTY VALUE SOURCE >> proteus cachefile - default >> titan# >> titan# zpool get cachefile t0 >> NAME PROPERTY VALUE SOURCE >> t0 cachefile - default >> titan# >> titan# ls -l /var/log/zpool_cache >> -rw-r--r-- 1 root wheel 4960 Nov 28 11:52 /var/log/zpool_cache >> titan# >>=20 >> The cachefile exists and seems to have grown in size. >>=20 >> However a reboot will once again provide nothing but the t0 pool. >>=20 >> Baffled. >>=20 >> Any thoughts would be welcome. >>=20 >> --=20 >> -- >> Dennis Clarke >> RISC-V/SPARC/PPC/ARM/CISC >> UNIX and Linux spoken >>=20 >>=20 >>=20 >>=20 >>=20 >=20 >=20 >=20 >=20 >=20 >=20 ------=_Part_6958_1940271980.1732802309596 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable Btw:

The /etc/rc.d/zpool script= looks into these cachefiles:

for cachefile in /et= c/zfs/zpool.cache /boot/zfs/zpool.cache; do

I didn= =E2=80=99t check where the cachefile pool property is used. 

Hope this helps resolving the issue. Or maybe helps you to = provide more information about your setup. 

R= egards,
Ronald. 

Van: Ronald Klop <ronald-lists@klop.ws>
Datum: 28 n= ovember 2024 14:50
Aan: Dennis Clarke <dclarke@blast= wave.org>
CC: Current FreeBSD <freebsd-current@fr= eebsd.org>
Onderwerp: Re: zpools no longer exist aft= er boot

Are the other disks available at the moment the boot process does zpool imp= ort?

Reg= ards,
Ronald

Van: Dennis Clarke
Datum: 28 november 2024 14:06
Aan: Current FreeBSD
Onderwerp: zpools no longer exist after boot
<= /dclarke@blastwave.org>


This is a baffling problem wherein two zpools no longer exist after
boot. This is :

titan# uname -apKU
FreeBSD titan 15.0-CURRENT FreeBSD 15.0-CURRENT #1 main-n273749-4b65481ac68= a-dirty: Wed Nov 20 15:08:52 GMT 2024 root@titan:/usr/obj/usr/src/amd64.amd= 64/sys/GENERIC-NODEBUG amd64 amd64 1500027 1500027
titan#

titan# zpool list
NAME   SIZE  ALLOC   FREE  CKPOINT  EXPA= NDSZ   FRAG    CAP  DEDUP HEALTH  ALTROO= T
t0     444G  91.2G   353G   &n= bsp;    -        &nb= sp;-    27%    20%  1.00x ONLINE  -=
titan#

The *only* zpool that seems to exist in any reliable way is the little
NVME based unit for booting. The other two zpools vanished and yet the
devices exist just fine :

titan#
titan# camcontrol devlist
       at scbus0 target 0 lun 0 (pass0,= ada0)
       at scbus1 target 0 lun 0 (pass1,= ada1)
  at scbus2 target 0 lun 0 (ses0,pass2)
  at scbus6 target 0 lun 0 (ses1,pass3)
 at scbus7 target 0 lun 1 (pass4,nda0)
            at= scbus8 target 0 lun 0 (da0,pass5)
titan#
titan# nvmecontrol devlist
  nvme0: SAMSUNG MZVKW512HMJP-000L7
     nvme0ns1 (488386MB)
titan#
titan# zpool status t0
   pool: t0
  state: ONLINE
status: Some supported and requested features are not enabled on the pool.<= br>          The pool can still be= used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
         the pool may no longe= r be accessible by software that does not support
         the features. See zpo= ol-features(7) for details.
   scan: scrub repaired 0B in 00:00:44 with 0 errors on Wed = Feb  7 09:56:40 2024
config:

         NAME   &nbs= p;    STATE     READ WRITE CKSUM          t0    =       ONLINE      &n= bsp;0     0     0
           nda0p3 &n= bsp;  ONLINE       0   &n= bsp; 0     0

errors: No known data errors
titan#


Initially I thought the problem was related to cachefile being empty for these zpools. However if I set the cachefile to something reasonable
then the cachefile property vanishes at a reboot.  The file, of course= , exists just fine :

titan# zpool get cachefile proteus
NAME     PROPERTY   VALUE    &= nbsp; SOURCE
proteus  cachefile  -        &= nbsp; default
titan#
titan# zpool set cachefile=3D"/var/log/zpool_cache" proteus
titan# zpool get cachefile proteus
NAME     PROPERTY   VALUE    &= nbsp;           &nbs= p;SOURCE
proteus  cachefile  /var/log/zpool_cache  local
titan# ls -ladb /var/log/zpool_cache
-rw-r--r--  1 root wheel 1440 Nov 28 11:45 /var/log/zpool_cache
titan#

So there we have 1440 bytes of data in that file.

titan# zpool set cachefile=3D"/var/log/zpool_cache" t0
titan# zpool get cachefile t0
NAME  PROPERTY   VALUE       &= nbsp;         SOURCE
t0    cachefile  /var/log/zpool_cache  local
titan#
titan# ls -ladb /var/log/zpool_cache
-rw-r--r--  1 root wheel 2880 Nov 28 11:46 /var/log/zpool_cache
titan#

Now we have 2 * 1440 bytes =3D 2880 bytes of some zpool cache data.

titan# zpool set cachefile=3D"/var/log/zpool_cache" leaf
titan# zpool get cachefile leaf
NAME  PROPERTY   VALUE       &= nbsp;         SOURCE
leaf  cachefile  /var/log/zpool_cache  local
titan#
titan# zpool get cachefile t0
NAME  PROPERTY   VALUE       &= nbsp;         SOURCE
t0    cachefile  /var/log/zpool_cache  local
titan#
titan# zpool get cachefile proteus
NAME     PROPERTY   VALUE    &= nbsp;           &nbs= p;SOURCE
proteus  cachefile  /var/log/zpool_cache  local
titan#
titan# reboot

 From here on ... the only zpool that exists after boot is the local little NVME samsung unit.

So here I can import those pools and then see that the cachefile property h= as been wiped out :

titan#
titan# zpool import proteus
titan# zpool import leaf
titan#
titan# zpool list
NAME      SIZE  ALLOC   FREE  = CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP H= EALTH  ALTROOT
leaf     18.2T   984K  18.2T   = ;     -        =  -     0%     0%  1.00x O= NLINE  -
proteus  1.98T   361G  1.63T     &n= bsp;  -         -  &= nbsp;  1%    17%  1.00x ONLINE  -
t0        444G  91.2G   3= 53G        -     &nb= sp;   -    27%    20%  1.= 00x ONLINE  -
titan#
titan# zpool get cachefile leaf
NAME  PROPERTY   VALUE      SOURCE<= br> leaf  cachefile  -        &nbs= p; default
titan#
titan# zpool get cachefile proteus
NAME     PROPERTY   VALUE    &= nbsp; SOURCE
proteus  cachefile  -        &= nbsp; default
titan#
titan# zpool get cachefile t0
NAME  PROPERTY   VALUE      SOURCE<= br> t0    cachefile  -       =    default
titan#
titan# ls -l /var/log/zpool_cache
-rw-r--r--  1 root wheel 4960 Nov 28 11:52 /var/log/zpool_cache
titan#

The cachefile exists and seems to have grown in size.

However a reboot will once again provide nothing but the t0 pool.

Baffled.

Any thoughts would be welcome.

-- 
--
Dennis Clarke
RISC-V/SPARC/PPC/ARM/CISC
UNIX and Linux spoken







------=_Part_6958_1940271980.1732802309596-- From nobody Thu Nov 28 14:06:54 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XzdR612qyz5dltG for ; Thu, 28 Nov 2024 14:06:58 +0000 (UTC) (envelope-from dclarke@blastwave.org) Received: from mail.oetec.com (mail.oetec.com [108.160.241.186]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "mail.oetec.com", Issuer "E6" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XzdR53SsGz4vws for ; Thu, 28 Nov 2024 14:06:57 +0000 (UTC) (envelope-from dclarke@blastwave.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=blastwave.org header.s=default header.b="KVJ/t/8R"; spf=pass (mx1.freebsd.org: domain of dclarke@blastwave.org designates 108.160.241.186 as permitted sender) smtp.mailfrom=dclarke@blastwave.org; dmarc=pass (policy=quarantine) header.from=blastwave.org Received: from [172.16.35.3] (pool-99-253-118-250.cpe.net.cable.rogers.com [99.253.118.250]) (authenticated bits=0) by mail.oetec.com (8.17.1/8.17.1) with ESMTPSA id 4ASE6shc027760 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT) for ; Thu, 28 Nov 2024 09:06:55 -0500 (EST) (envelope-from dclarke@blastwave.org) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=blastwave.org; s=default; t=1732802815; bh=dVTaVf1nmpJqrUMVvtDHGEWzozocXAbziL/ig7hY/iM=; h=Date:Subject:To:References:From:In-Reply-To; b=KVJ/t/8RFMe5EJmi/n8UY7CNHaucP7xw+PQ4WH55R2f+8d4GooZmyvmHaJQ989VWE XwAAzCGqLb7go21NrdN2kxMaqW2USVe2WyqD+jxaf5DMF9PtO8Ks9DWodVb/9TTfk7 MjQWiq62TdA5woRyjR/+50swvJ4Nz/QsG81hEDAJEUn28Q3bDE+bnQHD5ej3xpPGGG x0XzB3WYWLiF/Bzm2UfY1xCaHdcTdZ6ynjSRiZGnDDOYPdkJqLOCZf57xIEQlfUnLG MuAxDBZHxpCGAfA3/dYwfLvGrqDctH/yeasVGFNvu9aRYpbSDMttLB88uCAPciKfeq BVNdrv2doVh2A== Message-ID: <6bf5884f-9871-4c5b-912a-c64daf9577b5@blastwave.org> Date: Thu, 28 Nov 2024 09:06:54 -0500 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: zpools no longer exist after boot Content-Language: en-CA To: freebsd-current@freebsd.org References: <5798b0db-bc73-476a-908a-dd1f071bfe43@blastwave.org> From: Dennis Clarke Organization: GENUNIX In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-oetec-MailScanner-Information: Please contact the ISP for more information X-oetec-MailScanner-ID: 4ASE6shc027760 X-oetec-MailScanner: Found to be clean X-oetec-MailScanner-From: dclarke@blastwave.org X-Spam-Status: No X-Spamd-Result: default: False [-4.69 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[blastwave.org,quarantine]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; R_SPF_ALLOW(-0.20)[+mx]; RCVD_IN_DNSWL_MED(-0.20)[108.160.241.186:from]; R_DKIM_ALLOW(-0.20)[blastwave.org:s=default]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_ONE(0.00)[1]; TO_MATCH_ENVRCPT_ALL(0.00)[]; HAS_ORG_HEADER(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; ARC_NA(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[blastwave.org:+]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:812, ipnet:108.160.240.0/20, country:CA]; FROM_HAS_DN(0.00)[] X-Rspamd-Queue-Id: 4XzdR53SsGz4vws X-Spamd-Bar: ---- On 11/28/24 08:52, Alan Somers wrote: > On Thu, Nov 28, 2024, 7:06 AM Dennis Clarke wrote: > >> >> This is a baffling problem wherein two zpools no longer exist after >> boot. This is : ... > > Do you have zfs_enable="YES" set in /etc/rc.conf? If not then nothing will > get imported. > > Regarding the cachefile property, it's expected that "zpool import" will > change it, unless you do "zpool import -O cachefile=whatever". > >> > Oh absolutely. Also : zfs_enable="YES" # # the iSCSI initiator iscsid_enable="YES" iscsictl_enable="YES" iscsictl_flags="-Aa" # That explains the iSCSI device provided over 10Gbit : titan# titan# camcontrol devlist at scbus0 target 0 lun 0 (pass0,ada0) at scbus1 target 0 lun 0 (pass1,ada1) at scbus2 target 0 lun 0 (ses0,pass2) at scbus6 target 0 lun 0 (ses1,pass3) at scbus7 target 0 lun 1 (pass4,nda0) at scbus8 target 0 lun 0 (da0,pass5) titan# See the FREEBSD CTLDISK 001 ? That is over iSCSI. However, as I say, the devices exist but the pools vanish unless I import them and deal with them one by one. -- -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken From nobody Thu Nov 28 14:10:36 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XzdWb5T4Xz5dmVm for ; Thu, 28 Nov 2024 14:10:51 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XzdWb5236z3xwk; Thu, 28 Nov 2024 14:10:51 +0000 (UTC) (envelope-from otis@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732803051; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xFl9x5nxn7BxPUmJcoPSCXaI1ji6ujThOHloPBXVUho=; b=KVV9dfSTyOuNoy4fYrr/09hPJKMyY3jKU1Jwj+G2uuC7OzM46Ld83e+MC06wxRasjPLQNZ 2xhQfCymSMqmEf2nU93y14+Us1ARGVO2DRxB/ezj8NPOc2Ngor4cBp7fxNAxpMM/DSMXRu jiiBsajbdsFo8fxPDvqAJ+VF2jDIBhrl/JDqlcbU3x+d5Pb9FARxlEBOMYCM9bcIl8Sd2M tfAqIIsX1UD9E9e8IPje6UyyYvknZp3cESDCMSRh/a2H6Zh0icaF/d5+XCuRLC6uMExlrG O35Xyyw7BBpyC8xL3GRsdIzikRIrGiwkFtFPhjoqFsDZjm+l7EJAqvN7DwGtrw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732803051; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xFl9x5nxn7BxPUmJcoPSCXaI1ji6ujThOHloPBXVUho=; b=WD2B7YSlKjNNbeGn+qczo/hOoQQMG6sVNL52ELuwC2yjhljrDOZKUPDgJVM9J+kLrbmhxp 2BNpqk4jCZEynDuwV/S1l33Al2ufIhgMvgB9Ila0/Mwc1jwe7cymcV21ninKEhiSOFT4gA 00tWdTrfaG0u9aV/2Bez3H3Rr8Odi4joJNvwB9j4ubInqjT1+FQUkb+/FTP8HrXz7N7CuC WWayHyUh8mri6z0LTMUhmfaFFQXSvjBtHc+5mU2QnNNxcm9N6f4wzvJTDwVy4ySNU8gbGh IUIWW4QvheZUvJiNe4BYXR++IosQvtT44fW4Fh26rNEfHMwRjeD4IM8Q1ppWFw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1732803051; a=rsa-sha256; cv=none; b=fpT0wFO8JQ8WkOyMPb+R7jwn1J7hfaPVAqG7elLZ7RaETAstFvrdKRj/f9u+NdPa/0fbnv Y9upBgWYuHzCZAYkgncbxXdnRQjEx5vsOVqD+5wEwRQkIau+QxG1reeii/D3XV/bwA11kq tQBxDwADMbj6Q5v36Ru6HQXglMwtvUHayFWFLfrHp+421A/SSplh69/LNnc+yE9gGP/uwj LZeT+VvH+T/Ju3SJsEY5gZoL0DXkBuWGgi9hjvrUlRfwZYWF73kwM6WaM4OUMoOg/2bnOO Smqw63xFm9tTxv74zgRN3kUJYD/6oqKRawisJo9YM2CIhdwMQ1ogINWjnyzB4Q== Received: from ns2.wilbury.net (ns2.wilbury.net [92.60.51.55]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "svc.wilbury.net", Issuer "E5" (verified OK)) (Authenticated sender: otis) by smtp.freebsd.org (Postfix) with ESMTPSA id 4XzdWb3mJ4z11dW; Thu, 28 Nov 2024 14:10:51 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtpclient.apple (unknown [217.73.28.193]) (Authenticated sender: juraj@lutter.sk) by svc.wilbury.net (Postfix) with ESMTPSA id B911B621B6; Thu, 28 Nov 2024 15:10:46 +0100 (CET) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51.11.1\)) Subject: Re: zpools no longer exist after boot From: Juraj Lutter In-Reply-To: <6bf5884f-9871-4c5b-912a-c64daf9577b5@blastwave.org> Date: Thu, 28 Nov 2024 15:10:36 +0100 Cc: freebsd-current@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <132E5715-C717-4BA6-A5C7-E5F2CA81013B@FreeBSD.org> References: <5798b0db-bc73-476a-908a-dd1f071bfe43@blastwave.org> <6bf5884f-9871-4c5b-912a-c64daf9577b5@blastwave.org> To: Dennis Clarke X-Mailer: Apple Mail (2.3776.700.51.11.1) > On 28 Nov 2024, at 15:06, Dennis Clarke wrote: >=20 > On 11/28/24 08:52, Alan Somers wrote: >> On Thu, Nov 28, 2024, 7:06=E2=80=AFAM Dennis Clarke = wrote: >>>=20 >=20 > See the FREEBSD CTLDISK 001 ? That is over iSCSI. >=20 > However, as I say, the devices exist but the pools vanish > unless I import them and deal with them one by one. >=20 Are there any differences in each pool=E2=80=99s properties? (zpool get = all =E2=80=A6) otis =E2=80=94 Juraj Lutter otis@FreeBSD.org From nobody Thu Nov 28 14:16:57 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xzdfl6vTyz5dmp0 for ; Thu, 28 Nov 2024 14:17:03 +0000 (UTC) (envelope-from dclarke@blastwave.org) Received: from mail.oetec.com (mail.oetec.com [108.160.241.186]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "mail.oetec.com", Issuer "E6" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xzdfl6MB2z40pK for ; Thu, 28 Nov 2024 14:17:03 +0000 (UTC) (envelope-from dclarke@blastwave.org) Authentication-Results: mx1.freebsd.org; none Received: from [172.16.35.3] (pool-99-253-118-250.cpe.net.cable.rogers.com [99.253.118.250]) (authenticated bits=0) by mail.oetec.com (8.17.1/8.17.1) with ESMTPSA id 4ASEGw0U028023 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT); Thu, 28 Nov 2024 09:16:59 -0500 (EST) (envelope-from dclarke@blastwave.org) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=blastwave.org; s=default; t=1732803419; bh=6YksORMobAmf2if8VHo6jVlEQY/fgNs0R5j8rlYD5Uo=; h=Date:Subject:To:Cc:References:From:In-Reply-To; b=CSegFLM8cMRoKzk47/zS9C1BaRQkSG0PtEFkoKa8A4+kDn3Ur5BAH5Q2p+4wUdz/O fsUc2lkINC6h3ZZOSM+7wZJy1g2tv5eBSUqLOXrv9k7VuN4IU2kTFZ7xASIaWtORg6 styXh2OTCaI1w1kNj/wMXhgw2DsFvucvJGB9h650Snilqy7UOP+yHBKFvVmXokQSNf vQxyVY5BKIlYylGan4FmhagE09jzGUUHOgdppiPl52IhgwHh1MEY3DOxEC7q/KHkvu T/SfacAoX+nKIpqAshuykEyM2M6YASItirPza6avUa8aHCNIICHVNdN5Vcl5xUtdb7 Vy8tmO9LvjOQg== Message-ID: <043aee93-3dbd-4e6f-b0ee-fc6ebae9b8ef@blastwave.org> Date: Thu, 28 Nov 2024 09:16:57 -0500 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: zpools no longer exist after boot Content-Language: en-CA To: Ronald Klop Cc: Current FreeBSD References: <1764191396.6959.1732802309600@localhost> From: Dennis Clarke Organization: GENUNIX In-Reply-To: <1764191396.6959.1732802309600@localhost> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-oetec-MailScanner-Information: Please contact the ISP for more information X-oetec-MailScanner-ID: 4ASEGw0U028023 X-oetec-MailScanner: Found to be clean X-oetec-MailScanner-From: dclarke@blastwave.org X-Spam-Status: No X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:812, ipnet:108.160.240.0/20, country:CA] X-Rspamd-Queue-Id: 4Xzdfl6MB2z40pK X-Spamd-Bar: ---- On 11/28/24 08:58, Ronald Klop wrote: > Btw: > > The /etc/rc.d/zpool script looks into these cachefiles: > > for cachefile in /etc/zfs/zpool.cache /boot/zfs/zpool.cache; do > > I didn’t check where the cachefile pool property is used. > Hope this helps resolving the issue. Or maybe helps you to provide more > information about your setup. Ah ha ! So the cachefile property needs to be a specific filename or that script will have no clue. Take a look : titan# zpool get cachefile leaf proteus t0 NAME PROPERTY VALUE SOURCE leaf cachefile - default proteus cachefile - default t0 cachefile - default titan# Those files exist and they seem to be either from early this year or today : titan# ls -l /etc/zfs/zpool.cache /boot/zfs/zpool.cache -rw-r--r-- 1 root wheel 1424 Jan 16 2024 /boot/zfs/zpool.cache -rw-r--r-- 1 root wheel 4960 Nov 28 13:03 /etc/zfs/zpool.cache titan# titan# date -u Thu Nov 28 14:13:51 UTC 2024 titan# titan# uptime 2:13PM up 2:19, 1 user, load averages: 0.00, 0.00, 0.00 titan# titan# zpool set cachefile="/etc/zfs/zpool.cache" leaf titan# zpool set cachefile="/etc/zfs/zpool.cache" proteus titan# zpool set cachefile="/etc/zfs/zpool.cache" t0 However that will not work : titan# zpool get cachefile leaf proteus t0 NAME PROPERTY VALUE SOURCE leaf cachefile - default proteus cachefile - default t0 cachefile - default Let me try just one pool : titan# zpool get cachefile leaf NAME PROPERTY VALUE SOURCE leaf cachefile - default titan# zpool set cachefile="/etc/zfs/zpool.cache" leaf titan# zpool get cachefile leaf NAME PROPERTY VALUE SOURCE leaf cachefile - default titan# So this is even worse. -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken From nobody Thu Nov 28 14:26:17 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XzdsW107tz5dnNy for ; Thu, 28 Nov 2024 14:26:23 +0000 (UTC) (envelope-from dclarke@blastwave.org) Received: from mail.oetec.com (mail.oetec.com [108.160.241.186]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "mail.oetec.com", Issuer "E6" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XzdsV6vqHz42gT; Thu, 28 Nov 2024 14:26:22 +0000 (UTC) (envelope-from dclarke@blastwave.org) Authentication-Results: mx1.freebsd.org; none Received: from [172.16.35.3] (pool-99-253-118-250.cpe.net.cable.rogers.com [99.253.118.250]) (authenticated bits=0) by mail.oetec.com (8.17.1/8.17.1) with ESMTPSA id 4ASEQH5a028250 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT); Thu, 28 Nov 2024 09:26:18 -0500 (EST) (envelope-from dclarke@blastwave.org) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=blastwave.org; s=default; t=1732803979; bh=8S6aUBrYGf20OqVgWV/eOJBGNOQLMtfp9hr3AVwnGAA=; h=Date:Subject:To:Cc:References:From:In-Reply-To; b=B7TZ9pR8nn1IjDtULwqN+VvR376V21Zu1PJZLhpDPqj/1yhXyoiUZaqovMYipQAmP lFb7fhjKzgHhlSUDqyuckzcuupTUW+FTAe1STNRT5Tvpg/6EJVaV2Jl7HWfTUietFy bZMugPCx7OfaAUkXJRCOk8Q1s7ISXaYpxFri2zeATxs1VwZ4AzBzxUdunUyEvzKcxX RygDcyWv9nymzvm473NfqDn0hGMi54on4meiIFApV+G6imQCF5vWL7jzb/Uhwj5TgK KWIaUr8eBbywzevKc3ir2GHLOzxfV8AaESW5Ssm4nbzeVt9cN9wXgAZLCeOG2cDGv5 GxYtcOuMRDKIA== Message-ID: <8846124d-5a84-48e3-97c5-7a9b3db63e02@blastwave.org> Date: Thu, 28 Nov 2024 09:26:17 -0500 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: zpools no longer exist after boot Content-Language: en-CA To: Juraj Lutter Cc: freebsd-current@freebsd.org References: <5798b0db-bc73-476a-908a-dd1f071bfe43@blastwave.org> <6bf5884f-9871-4c5b-912a-c64daf9577b5@blastwave.org> <132E5715-C717-4BA6-A5C7-E5F2CA81013B@FreeBSD.org> From: Dennis Clarke Organization: GENUNIX In-Reply-To: <132E5715-C717-4BA6-A5C7-E5F2CA81013B@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-oetec-MailScanner-Information: Please contact the ISP for more information X-oetec-MailScanner-ID: 4ASEQH5a028250 X-oetec-MailScanner: Found to be clean X-oetec-MailScanner-From: dclarke@blastwave.org X-Spam-Status: No X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:812, ipnet:108.160.240.0/20, country:CA] X-Rspamd-Queue-Id: 4XzdsV6vqHz42gT X-Spamd-Bar: ---- On 11/28/24 09:10, Juraj Lutter wrote: > > > > Are there any differences in each pool’s properties? (zpool get all …) > Well, they are all different. There is a pool called leaf which is a mirror of two disks on two SATA/SAS backplanes. There is proteus which *was* working great over iSCSI and then the local pool t0 from the local little nvme Samsung device. Which thankfully exists or the machine would likely not boot at all. titan# zpool list leaf NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT leaf 18.2T 900K 18.2T - - 0% 0% 1.00x ONLINE - titan# titan# zpool status leaf pool: leaf state: ONLINE config: NAME STATE READ WRITE CKSUM leaf ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 ada0 ONLINE 0 0 0 ada1 ONLINE 0 0 0 errors: No known data errors titan# titan# zpool get all leaf NAME PROPERTY VALUE SOURCE leaf size 18.2T - leaf capacity 0% - leaf altroot - default leaf health ONLINE - leaf guid 227678941907208615 - leaf version - default leaf bootfs - default leaf delegation on default leaf autoreplace off default leaf cachefile - default leaf failmode continue local leaf listsnapshots off default leaf autoexpand off default leaf dedupratio 1.00x - leaf free 18.2T - leaf allocated 900K - leaf readonly off - leaf ashift 0 default leaf comment - default leaf expandsize - - leaf freeing 0 - leaf fragmentation 0% - leaf leaked 0 - leaf multihost off default leaf checkpoint - - leaf load_guid 6926439177379939855 - leaf autotrim off default leaf compatibility openzfs-2.0-freebsd local leaf bcloneused 0 - leaf bclonesaved 0 - leaf bcloneratio 1.00x - leaf dedup_table_size 0 - leaf dedup_table_quota auto default leaf feature@async_destroy enabled local leaf feature@empty_bpobj enabled local leaf feature@lz4_compress active local leaf feature@multi_vdev_crash_dump enabled local leaf feature@spacemap_histogram active local leaf feature@enabled_txg active local leaf feature@hole_birth active local leaf feature@extensible_dataset active local leaf feature@embedded_data active local leaf feature@bookmarks enabled local leaf feature@filesystem_limits enabled local leaf feature@large_blocks enabled local leaf feature@large_dnode enabled local leaf feature@sha512 enabled local leaf feature@skein enabled local leaf feature@edonr disabled local leaf feature@userobj_accounting enabled local leaf feature@encryption enabled local leaf feature@project_quota enabled local leaf feature@device_removal enabled local leaf feature@obsolete_counts enabled local leaf feature@zpool_checkpoint enabled local leaf feature@spacemap_v2 active local leaf feature@allocation_classes enabled local leaf feature@resilver_defer enabled local leaf feature@bookmark_v2 enabled local leaf feature@redaction_bookmarks enabled local leaf feature@redacted_datasets enabled local leaf feature@bookmark_written enabled local leaf feature@log_spacemap active local leaf feature@livelist enabled local leaf feature@device_rebuild enabled local leaf feature@zstd_compress active local leaf feature@draid disabled local leaf feature@zilsaxattr disabled local leaf feature@head_errlog disabled local leaf feature@blake3 disabled local leaf feature@block_cloning disabled local leaf feature@vdev_zaps_v2 disabled local leaf feature@redaction_list_spill disabled local leaf feature@raidz_expansion disabled local leaf feature@fast_dedup disabled local leaf feature@longname disabled local leaf feature@large_microzap disabled local titan# Nothing of interest there other than the blank cachefile which I can not set to anything. At least it seems to reject my attempts to set it. titan# titan# zpool status proteus pool: proteus state: ONLINE scan: scrub repaired 0B in 00:53:43 with 0 errors on Mon Jul 1 18:56:34 2024 config: NAME STATE READ WRITE CKSUM proteus ONLINE 0 0 0 da0p1 ONLINE 0 0 0 errors: No known data errors titan# titan# camcontrol devlist | grep 'FREEBSD' at scbus8 target 0 lun 0 (da0,pass5) titan# titan# zpool get all proteus NAME PROPERTY VALUE SOURCE proteus size 1.98T - proteus capacity 17% - proteus altroot - default proteus health ONLINE - proteus guid 4488185358894371950 - proteus version - default proteus bootfs - default proteus delegation on default proteus autoreplace on local proteus cachefile - default proteus failmode continue local proteus listsnapshots off default proteus autoexpand off default proteus dedupratio 1.00x - proteus free 1.63T - proteus allocated 361G - proteus readonly off - proteus ashift 0 default proteus comment - default proteus expandsize - - proteus freeing 0 - proteus fragmentation 1% - proteus leaked 0 - proteus multihost off default proteus checkpoint - - proteus load_guid 3646341449300914421 - proteus autotrim off default proteus compatibility openzfs-2.0-freebsd local proteus bcloneused 0 - proteus bclonesaved 0 - proteus bcloneratio 1.00x - proteus dedup_table_size 0 - proteus dedup_table_quota auto default proteus feature@async_destroy enabled local proteus feature@empty_bpobj active local proteus feature@lz4_compress active local proteus feature@multi_vdev_crash_dump enabled local proteus feature@spacemap_histogram active local proteus feature@enabled_txg active local proteus feature@hole_birth active local proteus feature@extensible_dataset active local proteus feature@embedded_data active local proteus feature@bookmarks enabled local proteus feature@filesystem_limits enabled local proteus feature@large_blocks enabled local proteus feature@large_dnode enabled local proteus feature@sha512 active local proteus feature@skein enabled local proteus feature@edonr disabled local proteus feature@userobj_accounting active local proteus feature@encryption enabled local proteus feature@project_quota active local proteus feature@device_removal enabled local proteus feature@obsolete_counts enabled local proteus feature@zpool_checkpoint enabled local proteus feature@spacemap_v2 active local proteus feature@allocation_classes enabled local proteus feature@resilver_defer enabled local proteus feature@bookmark_v2 enabled local proteus feature@redaction_bookmarks enabled local proteus feature@redacted_datasets enabled local proteus feature@bookmark_written enabled local proteus feature@log_spacemap active local proteus feature@livelist enabled local proteus feature@device_rebuild enabled local proteus feature@zstd_compress active local proteus feature@draid disabled local proteus feature@zilsaxattr disabled local proteus feature@head_errlog disabled local proteus feature@blake3 disabled local proteus feature@block_cloning disabled local proteus feature@vdev_zaps_v2 disabled local proteus feature@redaction_list_spill disabled local proteus feature@raidz_expansion disabled local proteus feature@fast_dedup disabled local proteus feature@longname disabled local proteus feature@large_microzap disabled local titan# Again here we see cachefile is blank. Lastly there is the little samsung nvme bootable device : titan# titan# zpool list t0 NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT t0 444G 91.2G 353G - - 27% 20% 1.00x ONLINE - titan# titan# zpool status t0 pool: t0 state: ONLINE status: Some supported and requested features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. scan: scrub repaired 0B in 00:00:44 with 0 errors on Wed Feb 7 09:56:40 2024 config: NAME STATE READ WRITE CKSUM t0 ONLINE 0 0 0 nda0p3 ONLINE 0 0 0 errors: No known data errors titan# titan# zpool get all t0 NAME PROPERTY VALUE SOURCE t0 size 444G - t0 capacity 20% - t0 altroot - default t0 health ONLINE - t0 guid 2604455524152494878 - t0 version - default t0 bootfs t0/ROOT/default local t0 delegation on default t0 autoreplace off default t0 cachefile - default t0 failmode wait default t0 listsnapshots off default t0 autoexpand off default t0 dedupratio 1.00x - t0 free 353G - t0 allocated 91.2G - t0 readonly off - t0 ashift 0 default t0 comment - default t0 expandsize - - t0 freeing 0 - t0 fragmentation 27% - t0 leaked 0 - t0 multihost off default t0 checkpoint - - t0 load_guid 5797689675549497497 - t0 autotrim off default t0 compatibility off default t0 bcloneused 12K - t0 bclonesaved 12K - t0 bcloneratio 2.00x - t0 dedup_table_size 0 - t0 dedup_table_quota auto default t0 feature@async_destroy enabled local t0 feature@empty_bpobj active local t0 feature@lz4_compress active local t0 feature@multi_vdev_crash_dump enabled local t0 feature@spacemap_histogram active local t0 feature@enabled_txg active local t0 feature@hole_birth active local t0 feature@extensible_dataset active local t0 feature@embedded_data active local t0 feature@bookmarks enabled local t0 feature@filesystem_limits enabled local t0 feature@large_blocks enabled local t0 feature@large_dnode enabled local t0 feature@sha512 active local t0 feature@skein enabled local t0 feature@edonr enabled local t0 feature@userobj_accounting active local t0 feature@encryption enabled local t0 feature@project_quota active local t0 feature@device_removal enabled local t0 feature@obsolete_counts enabled local t0 feature@zpool_checkpoint enabled local t0 feature@spacemap_v2 active local t0 feature@allocation_classes enabled local t0 feature@resilver_defer enabled local t0 feature@bookmark_v2 enabled local t0 feature@redaction_bookmarks enabled local t0 feature@redacted_datasets enabled local t0 feature@bookmark_written enabled local t0 feature@log_spacemap active local t0 feature@livelist enabled local t0 feature@device_rebuild enabled local t0 feature@zstd_compress active local t0 feature@draid enabled local t0 feature@zilsaxattr active local t0 feature@head_errlog active local t0 feature@blake3 enabled local t0 feature@block_cloning active local t0 feature@vdev_zaps_v2 active local t0 feature@redaction_list_spill enabled local t0 feature@raidz_expansion enabled local t0 feature@fast_dedup disabled local t0 feature@longname disabled local t0 feature@large_microzap disabled local titan# There is nothing of interest in the properties other than the absent cachefile setting. However I guess I could try to delete the previous cache files seen in /etc/rc.d/zpool : titan# cat /etc/rc.d/zpool #!/bin/sh # # # PROVIDE: zpool # REQUIRE: hostid disks # BEFORE: mountcritlocal # KEYWORD: nojail . /etc/rc.subr name="zpool" desc="Import ZPOOLs" rcvar="zfs_enable" start_cmd="zpool_start" required_modules="zfs" zpool_start() { local cachefile for cachefile in /etc/zfs/zpool.cache /boot/zfs/zpool.cache; do if [ -r $cachefile ]; then zpool import -c $cachefile -a -N if [ $? -ne 0 ]; then echo "Import of zpool cache ${cachefile} failed," \ "will retry after root mount hold release" root_hold_wait zpool import -c $cachefile -a -N fi break fi done } load_rc_config $name run_rc_command "$1" titan# titan# titan# titan# ls -l /etc/zfs/zpool.cache /boot/zfs/zpool.cache -rw-r--r-- 1 root wheel 1424 Jan 16 2024 /boot/zfs/zpool.cache -rw-r--r-- 1 root wheel 4960 Nov 28 14:15 /etc/zfs/zpool.cache titan# May as well delete them. I have nothing to lose at this point. -- -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken From nobody Thu Nov 28 14:45:08 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XzfHG2mp9z5dp8D for ; Thu, 28 Nov 2024 14:45:14 +0000 (UTC) (envelope-from dclarke@blastwave.org) Received: from mail.oetec.com (mail.oetec.com [108.160.241.186]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "mail.oetec.com", Issuer "E6" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XzfHF6LFyz45s7; Thu, 28 Nov 2024 14:45:13 +0000 (UTC) (envelope-from dclarke@blastwave.org) Authentication-Results: mx1.freebsd.org; none Received: from [172.16.35.3] (pool-99-253-118-250.cpe.net.cable.rogers.com [99.253.118.250]) (authenticated bits=0) by mail.oetec.com (8.17.1/8.17.1) with ESMTPSA id 4ASEj8CG028746 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT); Thu, 28 Nov 2024 09:45:09 -0500 (EST) (envelope-from dclarke@blastwave.org) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=blastwave.org; s=default; t=1732805110; bh=hApc6sl6YMErDaInrBYNJ6rk/T5pSLByd0huRTibQoY=; h=Date:Subject:To:Cc:References:From:In-Reply-To; b=l8ZImI8SARhsSWEswRbReNxmnO/Ca/F00cOGuzxwc62TctT5Oxy+7HXjQPWqffXpd U3nwNTWsfIcSw3mi/6QvrXywV1fIxeeMZPsVYXTNH5uyf8lDvUq8gZZewCYVD2QJFs T7E10ong2HJTEQhJnqB5bvOP24LGW7INkopNibmaeANTyeHgkC+/xBL+jEPpr8ZhMA +iQKEgZhgvJNkv6sc6TItZpTI+t+i20bP5kg/T6lIehL1s696lDU0aYeN/xhm8zK8L 6eJTWyIL3SsvNyEoII8y3IzSU1P7TPsz3IJJg8hFhU4FjQa3owg66kNwxsRDyeP+No 7CC51ds6nfNjA== Message-ID: <22187e59-b6e9-4f2e-ba9b-f43944d1a37b@blastwave.org> Date: Thu, 28 Nov 2024 09:45:08 -0500 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: zpools no longer exist after boot Content-Language: en-CA To: Alan Somers Cc: Current FreeBSD References: <5798b0db-bc73-476a-908a-dd1f071bfe43@blastwave.org> From: Dennis Clarke Organization: GENUNIX In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-oetec-MailScanner-Information: Please contact the ISP for more information X-oetec-MailScanner-ID: 4ASEj8CG028746 X-oetec-MailScanner: Found to be clean X-oetec-MailScanner-From: dclarke@blastwave.org X-Spam-Status: No X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:812, ipnet:108.160.240.0/20, country:CA] X-Rspamd-Queue-Id: 4XzfHF6LFyz45s7 X-Spamd-Bar: ---- On 11/28/24 08:52, Alan Somers wrote: > On Thu, Nov 28, 2024, 7:06 AM Dennis Clarke wrote: > >> >> This is a baffling problem wherein two zpools no longer exist after >> boot. This is : . . . > Do you have zfs_enable="YES" set in /etc/rc.conf? If not then nothing will > get imported. > > Regarding the cachefile property, it's expected that "zpool import" will > change it, unless you do "zpool import -O cachefile=whatever". > The rc script seems to do something slightly different with zpool import -c $FOOBAR thus : titan# cat /etc/rc.d/zpool #!/bin/sh # # # PROVIDE: zpool # REQUIRE: hostid disks # BEFORE: mountcritlocal # KEYWORD: nojail . /etc/rc.subr name="zpool" desc="Import ZPOOLs" rcvar="zfs_enable" start_cmd="zpool_start" required_modules="zfs" zpool_start() { local cachefile for cachefile in /etc/zfs/zpool.cache /boot/zfs/zpool.cache; do if [ -r $cachefile ]; then zpool import -c $cachefile -a -N if [ $? -ne 0 ]; then echo "Import of zpool cache ${cachefile} failed," \ "will retry after root mount hold release" root_hold_wait zpool import -c $cachefile -a -N fi break fi done } load_rc_config $name run_rc_command "$1" titan# I may as well nuke the pre-existing cache file and start over : titan# ls -l /etc/zfs/zpool.cache /boot/zfs/zpool.cache -rw-r--r-- 1 root wheel 1424 Jan 16 2024 /boot/zfs/zpool.cache -rw-r--r-- 1 root wheel 4960 Nov 28 14:15 /etc/zfs/zpool.cache titan# titan# titan# rm /boot/zfs/zpool.cache titan# zpool set cachefile="/boot/zfs/zpool.cache" t0 titan# titan# ls -l /boot/zfs/zpool.cache -rw-r--r-- 1 root wheel 1456 Nov 28 14:27 /boot/zfs/zpool.cache titan# titan# zpool set cachefile="/boot/zfs/zpool.cache" leaf titan# titan# ls -l /boot/zfs/zpool.cache -rw-r--r-- 1 root wheel 3536 Nov 28 14:28 /boot/zfs/zpool.cache titan# titan# zpool set cachefile="/boot/zfs/zpool.cache" proteus titan# titan# ls -l /boot/zfs/zpool.cache -rw-r--r-- 1 root wheel 4960 Nov 28 14:28 /boot/zfs/zpool.cache titan# titan# zpool get cachefile t0 NAME PROPERTY VALUE SOURCE t0 cachefile /boot/zfs/zpool.cache local titan# titan# zpool get cachefile leaf NAME PROPERTY VALUE SOURCE leaf cachefile /boot/zfs/zpool.cache local titan# titan# zpool get cachefile proteus NAME PROPERTY VALUE SOURCE proteus cachefile /boot/zfs/zpool.cache local titan# titan# titan# reboot Nov 28 14:34:05 Waiting (max 60 seconds) for system process `vnlru' to stop... done Waiting (max 60 seconds) for system process `syncer' to stop... Syncing disks, vnodes remaining... 0 0 0 0 0 0 done All buffers synced. Uptime: 2h38m57s GEOM_MIRROR: Device swap: provider destroyed. GEOM_MIRROR: Device swap destroyed. uhub5: detached uhub1: detached uhub4: detached uhub2: detached uhub3: detached uhub6: detached uhub0: detached ix0: link state changed to DOWN . . . Starting iscsid. Starting iscsictl. Clearing /tmp. Updating /var/run/os-release done. Updating motd:. Creating and/or trimming log files. Starting syslogd. No core dumps found. Starting local daemons:failed to open cache file: No such file or directory . Starting ntpd. Starting powerd. Mounting late filesystems:. Starting cron. Performing sanity check on sshd configuration. Starting sshd. Starting background file system FreeBSD/amd64 (titan) (ttyu0) login: root Password: Nov 28 14:36:29 titan login[4162]: ROOT LOGIN (root) ON ttyu0 Last login: Thu Nov 28 14:33:45 on ttyu0 FreeBSD 15.0-CURRENT (GENERIC-NODEBUG) #1 main-n273749-4b65481ac68a-dirty: Wed Nov 20 15:08:52 GMT 2024 Welcome to FreeBSD! Release Notes, Errata: https://www.FreeBSD.org/releases/ Security Advisories: https://www.FreeBSD.org/security/ FreeBSD Handbook: https://www.FreeBSD.org/handbook/ FreeBSD FAQ: https://www.FreeBSD.org/faq/ Questions List: https://www.FreeBSD.org/lists/questions/ FreeBSD Forums: https://forums.FreeBSD.org/ Documents installed with the system are in the /usr/local/share/doc/freebsd/ directory, or can be installed later with: pkg install en-freebsd-doc For other languages, replace "en" with a language code like de or fr. Show the version of FreeBSD installed: freebsd-version ; uname -a Please include that output and any error messages when posting questions. Introduction to manual pages: man man FreeBSD directory layout: man hier To change this login announcement, see motd(5). You have new mail. titan# titan# zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT leaf 18.2T 984K 18.2T - - 0% 0% 1.00x ONLINE - proteus 1.98T 361G 1.63T - - 1% 17% 1.00x ONLINE - t0 444G 91.2G 353G - - 27% 20% 1.00x ONLINE - titan# This is progress ... however the cachefile property is wiped out again : titan# zpool get cachefile t0 NAME PROPERTY VALUE SOURCE t0 cachefile - default titan# zpool get cachefile leaf NAME PROPERTY VALUE SOURCE leaf cachefile - default titan# zpool get cachefile proteus NAME PROPERTY VALUE SOURCE proteus cachefile - default titan# Also, strangely, none of the filesystem in proteus are mounted : titan# titan# zfs list -o name,exec,checksum,canmount,mounted,mountpoint -r proteus NAME EXEC CHECKSUM CANMOUNT MOUNTED MOUNTPOINT proteus on sha512 on no none proteus/bhyve off sha512 on no /bhyve proteus/bhyve/disk off sha512 on no /bhyve/disk proteus/bhyve/isos off sha512 on no /bhyve/isos proteus/obj on sha512 on no /usr/obj proteus/src on sha512 on no /usr/src titan# If I reboot again without doing anything will the zpools re-appear ? titan# titan# Nov 28 14:37:08 titan su[4199]: admsys to root on /dev/pts/0 titan# reboot Nov 28 14:40:29 Waiting (max 60 seconds) for system process `vnlru' to stop... done Waiting (max 60 seconds) for system process `syncer' to stop... Syncing disks, vnodes remaining... 0 0 0 0 0 done All buffers synced. Uptime: 4m50s GEOM_MIRROR: Device swap: provider destroyed. GEOM_MIRROR: Device swap destroyed. uhub4: detached uhub1: detached uhub5: detached uhub0: detached uhub3: detached uhub6: detached uhub2: detached ix0: link state changed to DOWN . . . Starting iscsid. Starting iscsictl. Clearing /tmp. Updating /var/run/os-release done. Updating motd:. Creating and/or trimming log files. Starting syslogd. No core dumps found. Starting local daemons:failed to open cache file: No such file or directory . Starting ntpd. Starting powerd. Mounting late filesystems:. Starting cron. Performing sanity check on sshd configuration. Starting sshd. Starting background file system FreeBSD/amd64 (titan) (ttyu0) login: root Password: Nov 28 14:43:01 titan login[4146]: ROOT LOGIN (root) ON ttyu0 Last login: Thu Nov 28 14:36:29 on ttyu0 FreeBSD 15.0-CURRENT (GENERIC-NODEBUG) #1 main-n273749-4b65481ac68a-dirty: Wed Nov 20 15:08:52 GMT 2024 Welcome to FreeBSD! Release Notes, Errata: https://www.FreeBSD.org/releases/ Security Advisories: https://www.FreeBSD.org/security/ FreeBSD Handbook: https://www.FreeBSD.org/handbook/ FreeBSD FAQ: https://www.FreeBSD.org/faq/ Questions List: https://www.FreeBSD.org/lists/questions/ FreeBSD Forums: https://forums.FreeBSD.org/ Documents installed with the system are in the /usr/local/share/doc/freebsd/ directory, or can be installed later with: pkg install en-freebsd-doc For other languages, replace "en" with a language code like de or fr. Show the version of FreeBSD installed: freebsd-version ; uname -a Please include that output and any error messages when posting questions. Introduction to manual pages: man man FreeBSD directory layout: man hier To change this login announcement, see motd(5). You have new mail. titan# titan# zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT leaf 18.2T 1.01M 18.2T - - 0% 0% 1.00x ONLINE - proteus 1.98T 361G 1.63T - - 1% 17% 1.00x ONLINE - t0 444G 91.2G 353G - - 27% 20% 1.00x ONLINE - titan# titan# zfs list -o name,exec,checksum,canmount,mounted,mountpoint -r proteus NAME EXEC CHECKSUM CANMOUNT MOUNTED MOUNTPOINT proteus on sha512 on no none proteus/bhyve off sha512 on no /bhyve proteus/bhyve/disk off sha512 on no /bhyve/disk proteus/bhyve/isos off sha512 on no /bhyve/isos proteus/obj on sha512 on no /usr/obj proteus/src on sha512 on no /usr/src titan# OKay so the zpools appear to be back in spite of the strange situation with the cachefile property is empty everywhere. My guess is the zpool rc script is bring in information during early boot. Why the zfs filesystems on proteus do not mount? Well that is a strange problem but at least the zpool can be used. -- -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken From nobody Thu Nov 28 14:52:45 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XzfSB6ZLhz5dpb7 for ; Thu, 28 Nov 2024 14:52:58 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ed1-f43.google.com (mail-ed1-f43.google.com [209.85.208.43]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XzfSB46Jwz49sc; Thu, 28 Nov 2024 14:52:58 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-f43.google.com with SMTP id 4fb4d7f45d1cf-5cefa22e9d5so977128a12.3; Thu, 28 Nov 2024 06:52:58 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732805577; x=1733410377; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=G/ZCwB9hDNaGed150n6byhQNCP+Xl44ozauxN44kdZ8=; b=VHyarrnzVhR3qFs8X2tkGXqbJg521TOgjTnmQvAW6s9upgLm/FHBadmNT0J+yFmzYY iBCytiGkE4Hfmkc1MNMjGX4A+dksu4ssoM0lQobWuNb5O6JIXZaNqGvoAGCmvOwMYRr3 evMnqi8tScMe3g6zbYRIFike2Tn0wla/k4iPPOs577r2HpMb8QEfp6I9KcBgaEN7i9O9 4y7IQ6msdRwVa1JcULifm1t+FVllMPkLWU3QWRvyAXfZWXs+Ox4VTq0pvY76KbSeti7Q e2xmMCEy1dYFSbi1qWXCRRyoEZfe9zYPAMv473xlo36AF2zxYbZ5K2G5ObzpZ4W5TnXL G+uA== X-Forwarded-Encrypted: i=1; AJvYcCWY6zWZBqef5edIIxU1890t4dguXmpAyUJ2DZ4MZinpsHQbgmMfsH1dveb47nJkDZ3Q9VtbbSLeVwcFwq37Avg=@freebsd.org X-Gm-Message-State: AOJu0YwUacyW15kHhN9ee1ehqWQJrG/489f+0T/kb9Oe8HjoK1vFt/fh kGob7RnNL2eStXABjZWgmZXZ2VUmgyP6Q68FwSiRUeOmwuiC+Qk9Lf1H3rxQWC0Zo31ZrhW2rk1 K47G79TNUNSbsUmXvhmTR5/GjvEM7Jw== X-Gm-Gg: ASbGncu+cVTtdmJsWlBtkMgSbJ5Rbce99A338cOeiykekvawKWJjnf8rlLRTjNLmGQ3 80hLMIPFAsYgkVKd5MEcRY6T1CLrouAB67a4dBkxgZ6n25LxgL3S8Y+1Q9MjDwQwQ X-Google-Smtp-Source: AGHT+IEg1fnYOxVVPmSzvH7OhgkuAIYjHBZkVTKTqrFsxGeFhAPtIhtrgNKDnUUMm9Ew7sRJFNiGV3ruLKcJhEXMatA= X-Received: by 2002:a05:6402:5243:b0:5d0:225b:f4e4 with SMTP id 4fb4d7f45d1cf-5d080c5cdaamr5601003a12.29.1732805576964; Thu, 28 Nov 2024 06:52:56 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 References: <5798b0db-bc73-476a-908a-dd1f071bfe43@blastwave.org> <22187e59-b6e9-4f2e-ba9b-f43944d1a37b@blastwave.org> In-Reply-To: <22187e59-b6e9-4f2e-ba9b-f43944d1a37b@blastwave.org> From: Alan Somers Date: Thu, 28 Nov 2024 08:52:45 -0600 Message-ID: Subject: Re: zpools no longer exist after boot To: Dennis Clarke Cc: Alan Somers , Current FreeBSD Content-Type: multipart/alternative; boundary="000000000000e1437d0627fa3c5e" X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4XzfSB46Jwz49sc X-Spamd-Bar: ---- --000000000000e1437d0627fa3c5e Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Nov 28, 2024, 8:45=E2=80=AFAM Dennis Clarke = wrote: > On 11/28/24 08:52, Alan Somers wrote: > > On Thu, Nov 28, 2024, 7:06=E2=80=AFAM Dennis Clarke > wrote: > > > >> > >> This is a baffling problem wherein two zpools no longer exist after > >> boot. This is : > . > . > . > > Do you have zfs_enable=3D"YES" set in /etc/rc.conf? If not then nothing > will > > get imported. > > > > Regarding the cachefile property, it's expected that "zpool import" wil= l > > change it, unless you do "zpool import -O cachefile=3Dwhatever". > > > > The rc script seems to do something slightly different with zpool import > -c $FOOBAR thus : > > > titan# cat /etc/rc.d/zpool > #!/bin/sh > # > # > > # PROVIDE: zpool > # REQUIRE: hostid disks > # BEFORE: mountcritlocal > # KEYWORD: nojail > > . /etc/rc.subr > > name=3D"zpool" > desc=3D"Import ZPOOLs" > rcvar=3D"zfs_enable" > start_cmd=3D"zpool_start" > required_modules=3D"zfs" > > zpool_start() > { > local cachefile > > for cachefile in /etc/zfs/zpool.cache /boot/zfs/zpool.cache; do > if [ -r $cachefile ]; then > zpool import -c $cachefile -a -N > if [ $? -ne 0 ]; then > echo "Import of zpool cache > ${cachefile} failed," \ > "will retry after root mount hold > release" > root_hold_wait > zpool import -c $cachefile -a -N > fi > break > fi > done > } > > load_rc_config $name > run_rc_command "$1" > titan# > > > > I may as well nuke the pre-existing cache file and start over : > > > titan# ls -l /etc/zfs/zpool.cache /boot/zfs/zpool.cache > -rw-r--r-- 1 root wheel 1424 Jan 16 2024 /boot/zfs/zpool.cache > -rw-r--r-- 1 root wheel 4960 Nov 28 14:15 /etc/zfs/zpool.cache > titan# > titan# > titan# rm /boot/zfs/zpool.cache > titan# zpool set cachefile=3D"/boot/zfs/zpool.cache" t0 > titan# > titan# ls -l /boot/zfs/zpool.cache > -rw-r--r-- 1 root wheel 1456 Nov 28 14:27 /boot/zfs/zpool.cache > titan# > titan# zpool set cachefile=3D"/boot/zfs/zpool.cache" leaf > titan# > titan# ls -l /boot/zfs/zpool.cache > -rw-r--r-- 1 root wheel 3536 Nov 28 14:28 /boot/zfs/zpool.cache > titan# > titan# zpool set cachefile=3D"/boot/zfs/zpool.cache" proteus > titan# > titan# ls -l /boot/zfs/zpool.cache > -rw-r--r-- 1 root wheel 4960 Nov 28 14:28 /boot/zfs/zpool.cache > titan# > titan# zpool get cachefile t0 > NAME PROPERTY VALUE SOURCE > t0 cachefile /boot/zfs/zpool.cache local > titan# > titan# zpool get cachefile leaf > NAME PROPERTY VALUE SOURCE > leaf cachefile /boot/zfs/zpool.cache local > titan# > titan# zpool get cachefile proteus > NAME PROPERTY VALUE SOURCE > proteus cachefile /boot/zfs/zpool.cache local > titan# > > titan# > titan# reboot > Nov 28 14:34:05 Waiting (max 60 seconds) for system process `vnlru' to > stop... done > Waiting (max 60 seconds) for system process `syncer' to stop... > Syncing disks, vnodes remaining... 0 0 0 0 0 0 done > All buffers synced. > Uptime: 2h38m57s > GEOM_MIRROR: Device swap: provider destroyed. > GEOM_MIRROR: Device swap destroyed. > uhub5: detached > uhub1: detached > uhub4: detached > uhub2: detached > uhub3: detached > uhub6: detached > uhub0: detached > ix0: link state changed to DOWN > . > . > . > > Starting iscsid. > Starting iscsictl. > Clearing /tmp. > Updating /var/run/os-release done. > Updating motd:. > Creating and/or trimming log files. > Starting syslogd. > No core dumps found. > Starting local daemons:failed to open cache file: No such file or directo= ry > . > Starting ntpd. > Starting powerd. > Mounting late filesystems:. > Starting cron. > Performing sanity check on sshd configuration. > Starting sshd. > Starting background file system > FreeBSD/amd64 (titan) (ttyu0) > > login: root > Password: > Nov 28 14:36:29 titan login[4162]: ROOT LOGIN (root) ON ttyu0 > Last login: Thu Nov 28 14:33:45 on ttyu0 > FreeBSD 15.0-CURRENT (GENERIC-NODEBUG) #1 > main-n273749-4b65481ac68a-dirty: Wed Nov 20 15:08:52 GMT 2024 > > Welcome to FreeBSD! > > Release Notes, Errata: https://www.FreeBSD.org/releases/ > Security Advisories: https://www.FreeBSD.org/security/ > FreeBSD Handbook: https://www.FreeBSD.org/handbook/ > FreeBSD FAQ: https://www.FreeBSD.org/faq/ > Questions List: https://www.FreeBSD.org/lists/questions/ > FreeBSD Forums: https://forums.FreeBSD.org/ > > Documents installed with the system are in the > /usr/local/share/doc/freebsd/ > directory, or can be installed later with: pkg install en-freebsd-doc > For other languages, replace "en" with a language code like de or fr. > > Show the version of FreeBSD installed: freebsd-version ; uname -a > Please include that output and any error messages when posting questions. > Introduction to manual pages: man man > FreeBSD directory layout: man hier > > To change this login announcement, see motd(5). > You have new mail. > titan# > titan# zpool list > NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP > HEALTH ALTROOT > leaf 18.2T 984K 18.2T - - 0% 0% 1.00x > ONLINE - > proteus 1.98T 361G 1.63T - - 1% 17% 1.00x > ONLINE - > t0 444G 91.2G 353G - - 27% 20% 1.00x > ONLINE - > titan# > > This is progress ... however the cachefile property is wiped out again : > > titan# zpool get cachefile t0 > NAME PROPERTY VALUE SOURCE > t0 cachefile - default > titan# zpool get cachefile leaf > NAME PROPERTY VALUE SOURCE > leaf cachefile - default > titan# zpool get cachefile proteus > NAME PROPERTY VALUE SOURCE > proteus cachefile - default > titan# > > Also, strangely, none of the filesystem in proteus are mounted : > > titan# > titan# zfs list -o name,exec,checksum,canmount,mounted,mountpoint -r > proteus > NAME EXEC CHECKSUM CANMOUNT MOUNTED MOUNTPOINT > proteus on sha512 on no none > proteus/bhyve off sha512 on no /bhyve > proteus/bhyve/disk off sha512 on no /bhyve/disk > proteus/bhyve/isos off sha512 on no /bhyve/isos > proteus/obj on sha512 on no /usr/obj > proteus/src on sha512 on no /usr/src > titan# > > If I reboot again without doing anything will the zpools re-appear ? > > > titan# > titan# Nov 28 14:37:08 titan su[4199]: admsys to root on /dev/pts/0 > > titan# reboot > Nov 28 14:40:29 Waiting (max 60 seconds) for system process `vnlru' to > stop... done > Waiting (max 60 seconds) for system process `syncer' to stop... > Syncing disks, vnodes remaining... 0 0 0 0 0 done > All buffers synced. > Uptime: 4m50s > GEOM_MIRROR: Device swap: provider destroyed. > GEOM_MIRROR: Device swap destroyed. > uhub4: detached > uhub1: detached > uhub5: detached > uhub0: detached > uhub3: detached > uhub6: detached > uhub2: detached > ix0: link state changed to DOWN > . > . > . > Starting iscsid. > Starting iscsictl. > Clearing /tmp. > Updating /var/run/os-release done. > Updating motd:. > Creating and/or trimming log files. > Starting syslogd. > No core dumps found. > Starting local daemons:failed to open cache file: No such file or directo= ry > . > Starting ntpd. > Starting powerd. > Mounting late filesystems:. > Starting cron. > Performing sanity check on sshd configuration. > Starting sshd. > Starting background file system > FreeBSD/amd64 (titan) (ttyu0) > > login: root > Password: > Nov 28 14:43:01 titan login[4146]: ROOT LOGIN (root) ON ttyu0 > Last login: Thu Nov 28 14:36:29 on ttyu0 > FreeBSD 15.0-CURRENT (GENERIC-NODEBUG) #1 > main-n273749-4b65481ac68a-dirty: Wed Nov 20 15:08:52 GMT 2024 > > Welcome to FreeBSD! > > Release Notes, Errata: https://www.FreeBSD.org/releases/ > Security Advisories: https://www.FreeBSD.org/security/ > FreeBSD Handbook: https://www.FreeBSD.org/handbook/ > FreeBSD FAQ: https://www.FreeBSD.org/faq/ > Questions List: https://www.FreeBSD.org/lists/questions/ > FreeBSD Forums: https://forums.FreeBSD.org/ > > Documents installed with the system are in the > /usr/local/share/doc/freebsd/ > directory, or can be installed later with: pkg install en-freebsd-doc > For other languages, replace "en" with a language code like de or fr. > > Show the version of FreeBSD installed: freebsd-version ; uname -a > Please include that output and any error messages when posting questions. > Introduction to manual pages: man man > FreeBSD directory layout: man hier > > To change this login announcement, see motd(5). > You have new mail. > titan# > titan# zpool list > NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP > HEALTH ALTROOT > leaf 18.2T 1.01M 18.2T - - 0% 0% 1.00x > ONLINE - > proteus 1.98T 361G 1.63T - - 1% 17% 1.00x > ONLINE - > t0 444G 91.2G 353G - - 27% 20% 1.00x > ONLINE - > titan# > titan# zfs list -o name,exec,checksum,canmount,mounted,mountpoint -r > proteus > NAME EXEC CHECKSUM CANMOUNT MOUNTED MOUNTPOINT > proteus on sha512 on no none > proteus/bhyve off sha512 on no /bhyve > proteus/bhyve/disk off sha512 on no /bhyve/disk > proteus/bhyve/isos off sha512 on no /bhyve/isos > proteus/obj on sha512 on no /usr/obj > proteus/src on sha512 on no /usr/src > titan# > > OKay so the zpools appear to be back in spite of the strange situation > with the cachefile property is empty everywhere. My guess is the zpool > rc script is bring in information during early boot. > > Why the zfs filesystems on proteus do not mount? Well that is a strange > problem but at least the zpool can be used. > > -- > -- > Dennis Clarke > RISC-V/SPARC/PPC/ARM/CISC > UNIX and Linux spoken > For "zpool import", the "-c" argument instructs zfs which cachefile to search for importable pools. "-O", on the other hand, specifies how the cachefile property should be set after the pool is imported. > --000000000000e1437d0627fa3c5e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Thu, Nov 28, 2024, 8:45=E2=80=AFAM Dennis Clarke &l= t;dclarke@blastwave.org> wr= ote:
On 11/28/24= 08:52, Alan Somers wrote:
> On Thu, Nov 28, 2024, 7:06=E2=80=AFAM Dennis Clarke <dclarke@bla= stwave.org> wrote:
>
>>
>> This is a baffling problem wherein two zpools no longer exist afte= r
>> boot. This is :
.
.
.
> Do you have zfs_enable=3D"YES" set in /etc/rc.conf? If not t= hen nothing will
> get imported.
>
> Regarding the cachefile property, it's expected that "zpool i= mport" will
> change it, unless you do "zpool import -O cachefile=3Dwhatever&qu= ot;.
>

The rc script seems to do something slightly different with zpool import -c $FOOBAR thus :


titan# cat=C2=A0 /etc/rc.d/zpool
#!/bin/sh
#
#

# PROVIDE: zpool
# REQUIRE: hostid disks
# BEFORE: mountcritlocal
# KEYWORD: nojail

. /etc/rc.subr

name=3D"zpool"
desc=3D"Import ZPOOLs"
rcvar=3D"zfs_enable"
start_cmd=3D"zpool_start"
required_modules=3D"zfs"

zpool_start()
{
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0local cachefile

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for cachefile in /etc/zfs/zpool.cache /bo= ot/zfs/zpool.cache; do
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if [ -r $cach= efile ]; then
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0zpool import -c $cachefile -a -N
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0if [ $? -ne 0 ]; then
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0echo "Import of zpool cac= he
${cachefile} failed," \
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"will retry= after root mount hold
release"
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0root_hold_wait
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0zpool import -c $cachefile -a = -N
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0fi
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0break
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0fi
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0done
}

load_rc_config $name
run_rc_command "$1"
titan#



I may as well nuke the pre-existing cache file and start over :


titan# ls -l /etc/zfs/zpool.cache /boot/zfs/zpool.cache
-rw-r--r--=C2=A0 1 root wheel 1424 Jan 16=C2=A0 2024 /boot/zfs/zpool.cache<= br> -rw-r--r--=C2=A0 1 root wheel 4960 Nov 28 14:15 /etc/zfs/zpool.cache
titan#
titan#
titan# rm /boot/zfs/zpool.cache
titan# zpool set cachefile=3D"/boot/zfs/zpool.cache" t0
titan#
titan# ls -l /boot/zfs/zpool.cache
-rw-r--r--=C2=A0 1 root wheel 1456 Nov 28 14:27 /boot/zfs/zpool.cache
titan#
titan# zpool set cachefile=3D"/boot/zfs/zpool.cache" leaf
titan#
titan# ls -l /boot/zfs/zpool.cache
-rw-r--r--=C2=A0 1 root wheel 3536 Nov 28 14:28 /boot/zfs/zpool.cache
titan#
titan# zpool set cachefile=3D"/boot/zfs/zpool.cache" proteus
titan#
titan# ls -l /boot/zfs/zpool.cache
-rw-r--r--=C2=A0 1 root wheel 4960 Nov 28 14:28 /boot/zfs/zpool.cache
titan#
titan# zpool get cachefile t0
NAME=C2=A0 PROPERTY=C2=A0 =C2=A0VALUE=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 SOURCE
t0=C2=A0 =C2=A0 cachefile=C2=A0 /boot/zfs/zpool.cache=C2=A0 local
titan#
titan# zpool get cachefile leaf
NAME=C2=A0 PROPERTY=C2=A0 =C2=A0VALUE=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 SOURCE
leaf=C2=A0 cachefile=C2=A0 /boot/zfs/zpool.cache=C2=A0 local
titan#
titan# zpool get cachefile proteus
NAME=C2=A0 =C2=A0 =C2=A0PROPERTY=C2=A0 =C2=A0VALUE=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 SOURCE
proteus=C2=A0 cachefile=C2=A0 /boot/zfs/zpool.cache=C2=A0 local
titan#

titan#
titan# reboot
Nov 28 14:34:05 Waiting (max 60 seconds) for system process `vnlru' to =
stop... done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining... 0 0 0 0 0 0 done
All buffers synced.
Uptime: 2h38m57s
GEOM_MIRROR: Device swap: provider destroyed.
GEOM_MIRROR: Device swap destroyed.
uhub5: detached
uhub1: detached
uhub4: detached
uhub2: detached
uhub3: detached
uhub6: detached
uhub0: detached
ix0: link state changed to DOWN
.
.
.

Starting iscsid.
Starting iscsictl.
Clearing /tmp.
Updating /var/run/os-release done.
Updating motd:.
Creating and/or trimming log files.
Starting syslogd.
No core dumps found.
Starting local daemons:failed to open cache file: No such file or directory=
.
Starting ntpd.
Starting powerd.
Mounting late filesystems:.
Starting cron.
Performing sanity check on sshd configuration.
Starting sshd.
Starting background file system
FreeBSD/amd64 (titan) (ttyu0)

login: root
Password:
Nov 28 14:36:29 titan login[4162]: ROOT LOGIN (root) ON ttyu0
Last login: Thu Nov 28 14:33:45 on ttyu0
FreeBSD 15.0-CURRENT (GENERIC-NODEBUG) #1
main-n273749-4b65481ac68a-dirty: Wed Nov 20 15:08:52 GMT 2024

Welcome to FreeBSD!

Release Notes, Errata: https://www.FreeBSD.org/releases/=
Security Advisories:=C2=A0 =C2=A0https://www.FreeBSD.org= /security/
FreeBSD Handbook:=C2=A0 =C2=A0 =C2=A0 https://www.FreeBS= D.org/handbook/
FreeBSD FAQ:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0https://= www.FreeBSD.org/faq/
Questions List:=C2=A0 =C2=A0 =C2=A0 =C2=A0 https:= //www.FreeBSD.org/lists/questions/
FreeBSD Forums:=C2=A0 =C2=A0 =C2=A0 =C2=A0 https://forums.Free= BSD.org/

Documents installed with the system are in the /usr/local/share/doc/freebsd= /
directory, or can be installed later with:=C2=A0 pkg install en-freebsd-doc=
For other languages, replace "en" with a language code like de or= fr.

Show the version of FreeBSD installed:=C2=A0 freebsd-version ; uname -a
Please include that output and any error messages when posting questions. Introduction to manual pages:=C2=A0 man man
FreeBSD directory layout:=C2=A0 =C2=A0 =C2=A0 man hier

To change this login announcement, see motd(5).
You have new mail.
titan#
titan# zpool list
NAME=C2=A0 =C2=A0 =C2=A0 SIZE=C2=A0 ALLOC=C2=A0 =C2=A0FREE=C2=A0 CKPOINT=C2= =A0 EXPANDSZ=C2=A0 =C2=A0FRAG=C2=A0 =C2=A0 CAP=C2=A0 DEDUP
HEALTH=C2=A0 ALTROOT
leaf=C2=A0 =C2=A0 =C2=A018.2T=C2=A0 =C2=A0984K=C2=A0 18.2T=C2=A0 =C2=A0 =C2= =A0 =C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-=C2=A0 =C2=A0 =C2=A00%=C2=A0= =C2=A0 =C2=A00%=C2=A0 1.00x
ONLINE=C2=A0 -
proteus=C2=A0 1.98T=C2=A0 =C2=A0361G=C2=A0 1.63T=C2=A0 =C2=A0 =C2=A0 =C2=A0= -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-=C2=A0 =C2=A0 =C2=A01%=C2=A0 =C2=A0 17= %=C2=A0 1.00x
ONLINE=C2=A0 -
t0=C2=A0 =C2=A0 =C2=A0 =C2=A0 444G=C2=A0 91.2G=C2=A0 =C2=A0353G=C2=A0 =C2= =A0 =C2=A0 =C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-=C2=A0 =C2=A0 27%=C2= =A0 =C2=A0 20%=C2=A0 1.00x
ONLINE=C2=A0 -
titan#

This is progress ... however the cachefile property is wiped out again :
titan# zpool get cachefile t0
NAME=C2=A0 PROPERTY=C2=A0 =C2=A0VALUE=C2=A0 =C2=A0 =C2=A0 SOURCE
t0=C2=A0 =C2=A0 cachefile=C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 default=
titan# zpool get cachefile leaf
NAME=C2=A0 PROPERTY=C2=A0 =C2=A0VALUE=C2=A0 =C2=A0 =C2=A0 SOURCE
leaf=C2=A0 cachefile=C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 default
titan# zpool get cachefile proteus
NAME=C2=A0 =C2=A0 =C2=A0PROPERTY=C2=A0 =C2=A0VALUE=C2=A0 =C2=A0 =C2=A0 SOUR= CE
proteus=C2=A0 cachefile=C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 default titan#

Also, strangely, none of the filesystem in proteus are mounted :

titan#
titan# zfs list -o name,exec,checksum,canmount,mounted,mountpoint -r proteu= s
NAME=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 EXEC=C2=A0 CHEC= KSUM=C2=A0 =C2=A0CANMOUNT=C2=A0 MOUNTED=C2=A0 MOUNTPOINT
proteus=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0on=C2=A0 =C2=A0 sha5= 12=C2=A0 =C2=A0 =C2=A0on=C2=A0 =C2=A0 =C2=A0 =C2=A0 no=C2=A0 =C2=A0 =C2=A0 = =C2=A0none
proteus/bhyve=C2=A0 =C2=A0 =C2=A0 =C2=A0off=C2=A0 =C2=A0sha512=C2=A0 =C2=A0= =C2=A0on=C2=A0 =C2=A0 =C2=A0 =C2=A0 no=C2=A0 =C2=A0 =C2=A0 =C2=A0/bhyve proteus/bhyve/disk=C2=A0 off=C2=A0 =C2=A0sha512=C2=A0 =C2=A0 =C2=A0on=C2=A0= =C2=A0 =C2=A0 =C2=A0 no=C2=A0 =C2=A0 =C2=A0 =C2=A0/bhyve/disk
proteus/bhyve/isos=C2=A0 off=C2=A0 =C2=A0sha512=C2=A0 =C2=A0 =C2=A0on=C2=A0= =C2=A0 =C2=A0 =C2=A0 no=C2=A0 =C2=A0 =C2=A0 =C2=A0/bhyve/isos
proteus/obj=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0on=C2=A0 =C2=A0 sha512=C2=A0 = =C2=A0 =C2=A0on=C2=A0 =C2=A0 =C2=A0 =C2=A0 no=C2=A0 =C2=A0 =C2=A0 =C2=A0/us= r/obj
proteus/src=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0on=C2=A0 =C2=A0 sha512=C2=A0 = =C2=A0 =C2=A0on=C2=A0 =C2=A0 =C2=A0 =C2=A0 no=C2=A0 =C2=A0 =C2=A0 =C2=A0/us= r/src
titan#

If I reboot again without doing anything will the zpools re-appear ?


titan#
titan# Nov 28 14:37:08 titan su[4199]: admsys to root on /dev/pts/0

titan# reboot
Nov 28 14:40:29 Waiting (max 60 seconds) for system process `vnlru' to =
stop... done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining... 0 0 0 0 0 done
All buffers synced.
Uptime: 4m50s
GEOM_MIRROR: Device swap: provider destroyed.
GEOM_MIRROR: Device swap destroyed.
uhub4: detached
uhub1: detached
uhub5: detached
uhub0: detached
uhub3: detached
uhub6: detached
uhub2: detached
ix0: link state changed to DOWN
.
.
.
Starting iscsid.
Starting iscsictl.
Clearing /tmp.
Updating /var/run/os-release done.
Updating motd:.
Creating and/or trimming log files.
Starting syslogd.
No core dumps found.
Starting local daemons:failed to open cache file: No such file or directory=
.
Starting ntpd.
Starting powerd.
Mounting late filesystems:.
Starting cron.
Performing sanity check on sshd configuration.
Starting sshd.
Starting background file system
FreeBSD/amd64 (titan) (ttyu0)

login: root
Password:
Nov 28 14:43:01 titan login[4146]: ROOT LOGIN (root) ON ttyu0
Last login: Thu Nov 28 14:36:29 on ttyu0
FreeBSD 15.0-CURRENT (GENERIC-NODEBUG) #1
main-n273749-4b65481ac68a-dirty: Wed Nov 20 15:08:52 GMT 2024

Welcome to FreeBSD!

Release Notes, Errata: https://www.FreeBSD.org/releases/=
Security Advisories:=C2=A0 =C2=A0https://www.FreeBSD.org= /security/
FreeBSD Handbook:=C2=A0 =C2=A0 =C2=A0 https://www.FreeBS= D.org/handbook/
FreeBSD FAQ:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0https://= www.FreeBSD.org/faq/
Questions List:=C2=A0 =C2=A0 =C2=A0 =C2=A0 https:= //www.FreeBSD.org/lists/questions/
FreeBSD Forums:=C2=A0 =C2=A0 =C2=A0 =C2=A0 https://forums.Free= BSD.org/

Documents installed with the system are in the /usr/local/share/doc/freebsd= /
directory, or can be installed later with:=C2=A0 pkg install en-freebsd-doc=
For other languages, replace "en" with a language code like de or= fr.

Show the version of FreeBSD installed:=C2=A0 freebsd-version ; uname -a
Please include that output and any error messages when posting questions. Introduction to manual pages:=C2=A0 man man
FreeBSD directory layout:=C2=A0 =C2=A0 =C2=A0 man hier

To change this login announcement, see motd(5).
You have new mail.
titan#
titan# zpool list
NAME=C2=A0 =C2=A0 =C2=A0 SIZE=C2=A0 ALLOC=C2=A0 =C2=A0FREE=C2=A0 CKPOINT=C2= =A0 EXPANDSZ=C2=A0 =C2=A0FRAG=C2=A0 =C2=A0 CAP=C2=A0 DEDUP
HEALTH=C2=A0 ALTROOT
leaf=C2=A0 =C2=A0 =C2=A018.2T=C2=A0 1.01M=C2=A0 18.2T=C2=A0 =C2=A0 =C2=A0 = =C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-=C2=A0 =C2=A0 =C2=A00%=C2=A0 =C2= =A0 =C2=A00%=C2=A0 1.00x
ONLINE=C2=A0 -
proteus=C2=A0 1.98T=C2=A0 =C2=A0361G=C2=A0 1.63T=C2=A0 =C2=A0 =C2=A0 =C2=A0= -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-=C2=A0 =C2=A0 =C2=A01%=C2=A0 =C2=A0 17= %=C2=A0 1.00x
ONLINE=C2=A0 -
t0=C2=A0 =C2=A0 =C2=A0 =C2=A0 444G=C2=A0 91.2G=C2=A0 =C2=A0353G=C2=A0 =C2= =A0 =C2=A0 =C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-=C2=A0 =C2=A0 27%=C2= =A0 =C2=A0 20%=C2=A0 1.00x
ONLINE=C2=A0 -
titan#
titan# zfs list -o name,exec,checksum,canmount,mounted,mountpoint -r proteu= s
NAME=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 EXEC=C2=A0 CHEC= KSUM=C2=A0 =C2=A0CANMOUNT=C2=A0 MOUNTED=C2=A0 MOUNTPOINT
proteus=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0on=C2=A0 =C2=A0 sha5= 12=C2=A0 =C2=A0 =C2=A0on=C2=A0 =C2=A0 =C2=A0 =C2=A0 no=C2=A0 =C2=A0 =C2=A0 = =C2=A0none
proteus/bhyve=C2=A0 =C2=A0 =C2=A0 =C2=A0off=C2=A0 =C2=A0sha512=C2=A0 =C2=A0= =C2=A0on=C2=A0 =C2=A0 =C2=A0 =C2=A0 no=C2=A0 =C2=A0 =C2=A0 =C2=A0/bhyve proteus/bhyve/disk=C2=A0 off=C2=A0 =C2=A0sha512=C2=A0 =C2=A0 =C2=A0on=C2=A0= =C2=A0 =C2=A0 =C2=A0 no=C2=A0 =C2=A0 =C2=A0 =C2=A0/bhyve/disk
proteus/bhyve/isos=C2=A0 off=C2=A0 =C2=A0sha512=C2=A0 =C2=A0 =C2=A0on=C2=A0= =C2=A0 =C2=A0 =C2=A0 no=C2=A0 =C2=A0 =C2=A0 =C2=A0/bhyve/isos
proteus/obj=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0on=C2=A0 =C2=A0 sha512=C2=A0 = =C2=A0 =C2=A0on=C2=A0 =C2=A0 =C2=A0 =C2=A0 no=C2=A0 =C2=A0 =C2=A0 =C2=A0/us= r/obj
proteus/src=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0on=C2=A0 =C2=A0 sha512=C2=A0 = =C2=A0 =C2=A0on=C2=A0 =C2=A0 =C2=A0 =C2=A0 no=C2=A0 =C2=A0 =C2=A0 =C2=A0/us= r/src
titan#

OKay so the zpools appear to be back in spite of the strange situation
with the cachefile property is empty everywhere.=C2=A0 My guess is the zpoo= l
rc script is bring in information during early boot.

Why the zfs filesystems on proteus do not mount? Well that is a strange problem but at least the zpool can be used.

--
--
Dennis Clarke
RISC-V/SPARC/PPC/ARM/CISC
UNIX and Linux spoken

For "zpool import", the "-c" argum= ent instructs zfs which cachefile to search for importable pools. "-O&= quot;, on the other hand, specifies how the cachefile property should be se= t after the pool is imported.=C2=A0
--000000000000e1437d0627fa3c5e-- From nobody Thu Nov 28 15:02:26 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xzfg83r7Jz5dqPq for ; Thu, 28 Nov 2024 15:02:28 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xzfg83Mc7z4D8c; Thu, 28 Nov 2024 15:02:28 +0000 (UTC) (envelope-from dim@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732806148; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LsNR7zc21fUAeDNY6arFEhppR1vjE/1RxufU4C51Dis=; b=q4Jnbz93t0wv2uQFd9qNh5sMUrPF+qimJ0mK+DG7RiBlhpQxjRuP5c3rd6it4Xt8uT5/b3 dEDPOxho2WboTZHCQmf0DCdquQ0od3NOWNCGc2SrI4HHo34ADOcbT44KA2C6yKGCXKrm6D 6YtLPVoQWMl0tUj7BkvRyA1iCAttN/jNcwzr8VaQKTHaYruFs+6ZPbJwF7foUS3kmf5LQw 5lNXHNAL9QcZscOGZIKfIEI3kMmXr9RiEgqi1QT388DzRhJGhW87g6CGAmGmOoD41OUZLX e0tzSRElvaM2K54PbEJa90TTmr3aDhGt4KJUiwJ5NyzKQ4ANKoBwjQyKDCMBRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732806148; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LsNR7zc21fUAeDNY6arFEhppR1vjE/1RxufU4C51Dis=; b=gzGJwFLapWwDDwq1isVC5Ib2HcVxmW9Z1RWloXDX1z8eBXwjY7Ioq8c6L1h0yng1KOogdj fN9Y3H2VX+ikGGQvCH0UFH6emedhyILUOOoX+kNQcl7y6hCb46bEMYWnjGW73Gmf2bMljz VI6Dt/RAKbKqiccziAbHYAmzx3ZAQPzee1R3namvdGmG7iI0S8Ms0KNjl+CnYDHfXB/z3J M5ufkXaEjdEi7dEh9JlQGojPPggKXHBcJOBJCFKOTNoxP4wu8NHbb3yZNITE6d21xyMdGg zFqwK2ccajIQw7npvyNUjo0PddUyITdr0MFohXauIsb30X+aCS1gH93Tw7PxzA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1732806148; a=rsa-sha256; cv=none; b=ODCYRIbJG6ObZjS1I68FT8Ph1wREZp4s3J+lauoqDGWvqgM70S0a/bFUGdjPu322+kFNZb LERe+7lL8fMRb+/1wUk+tCUDP5t49PV00tjrFX1ZMC9w22W2hJ294vudkB1T+iVpUdz5Np fwMKGqf+MCQ3rZYRpA0/1X8cNnt95Q+CCiwSZm5iABSNSjoqRd1RwY5a4MIb201AsRwmi+ Q8imqVRZ2Jk5mpks1YWDKnro00EnTTRYngwOJjgajPYr5lnYcPdsMGLk22x1weskxedu2+ mr1vcCdqRMUHPAHx/GwfODrEmC8fSEjbksbkJaqhORu0f8QDjtkL/JXZfoztAg== Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "tensor.andric.com", Issuer "R10" (verified OK)) (Authenticated sender: dim) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Xzfg82R59z12kV; Thu, 28 Nov 2024 15:02:28 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtpclient.apple (longrow.home.andric.com [192.168.0.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id E5F1951CB3; Thu, 28 Nov 2024 16:02:26 +0100 (CET) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6.1.9\)) Subject: Re: zpools no longer exist after boot From: Dimitry Andric In-Reply-To: <5798b0db-bc73-476a-908a-dd1f071bfe43@blastwave.org> Date: Thu, 28 Nov 2024 16:02:26 +0100 Cc: Current FreeBSD Content-Transfer-Encoding: quoted-printable Message-Id: <5165C4DB-8BA0-4579-A808-F98FBF3B797F@FreeBSD.org> References: <5798b0db-bc73-476a-908a-dd1f071bfe43@blastwave.org> To: Dennis Clarke X-Mailer: Apple Mail (2.3731.700.6.1.9) On 28 Nov 2024, at 14:05, Dennis Clarke wrote: >=20 > This is a baffling problem wherein two zpools no longer exist after > boot. This is : >=20 > titan# uname -apKU > FreeBSD titan 15.0-CURRENT FreeBSD 15.0-CURRENT #1 = main-n273749-4b65481ac68a-dirty: Wed Nov 20 15:08:52 GMT 2024 = root@titan:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG amd64 amd64 = 1500027 1500027 > titan# >=20 > titan# zpool list > NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP = HEALTH ALTROOT > t0 444G 91.2G 353G - - 27% 20% 1.00x = ONLINE - > titan# >=20 > The *only* zpool that seems to exist in any reliable way is the little > NVME based unit for booting. The other two zpools vanished and yet the > devices exist just fine : >=20 > titan# > titan# camcontrol devlist > at scbus0 target 0 lun 0 = (pass0,ada0) > at scbus1 target 0 lun 0 = (pass1,ada1) > at scbus2 target 0 lun 0 = (ses0,pass2) > at scbus6 target 0 lun 0 = (ses1,pass3) > at scbus7 target 0 lun 1 = (pass4,nda0) > at scbus8 target 0 lun 0 = (da0,pass5) It has been my experience that with disks connected to external = enclosures, these sometimes take a long time to come up. If they come up = only after the kernel initially detects ZFS pools, they will not be = shown or imported at all. If you look at dmesg or boot logs, are the external disks detected = around the same time as the NVMe device? -Dimitry From nobody Thu Nov 28 15:04:10 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XzfjN3w6Wz5dqMP for ; Thu, 28 Nov 2024 15:04:24 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [IPv6:2a00:1450:4864:20::52a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XzfjM6CkVz4FB0 for ; Thu, 28 Nov 2024 15:04:23 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-x52a.google.com with SMTP id 4fb4d7f45d1cf-5cfcb7183deso3507979a12.0 for ; Thu, 28 Nov 2024 07:04:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1732806262; x=1733411062; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=eFzKvMVtYWXhgHQjjHjiNwtJVbRXNb1I8B3+u25jQYg=; b=CUgkPMeqX1iJiKH2GgL1SdUwTJkyHGUwq2GgMJ43y58XIRRNkglm25DgutYoRjVUP4 ISdrzbUEk6k0w/oBfslaNwaz5LMqYdk+ugPLgC6rfopzGZhZ/iPFe0sltp/yncNPD92W 2fPwGZJQYJWMuWc1+HL1Y0wQJ/gJdxWHr4nmT8xXD4mJATAuTFgL/x3GRuB+r1f8ljLA sSnYxPMVpMiXJnsGVItEiSDTKjET9l3PWbU8kPM+LGKE/2zmHCFtmfwIR5Z9VIqKtHJP TXW+4s946Tj2hMVJCmCWX+CXFtvwKceaXRL0O9EX11g0j+yvMuCzHb6QsmHaqzMAso8m A5LA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732806262; x=1733411062; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eFzKvMVtYWXhgHQjjHjiNwtJVbRXNb1I8B3+u25jQYg=; b=Sp1NyxnEg00a+KPtl2ZwcUZwXxEUXBXddEGnTnG4QCKm5wevCRIrhKeb5ZBZS39cT8 zRym8W6+1wp9w8c2Utiqj9K+WqWgsUkdJhPE9UdLBpdkTy1XiPu16AeGYyffTJWPTuL2 NcFlOqAmCxvF0EsYe9ivE0JdhyEsIGKtfgtCmE5yzsdBFvbcCQIQaDHcuKZLtktQAy3S tpd7QR3Gs6SwujEjHCVkVhGICagFAe1CXfEMRxZRc69i3giFVi4862kfOrtNhihJpDy0 kgDFjjiy6s2FpRwqKEuqvDMyk/VkfN2zDBkVNlr8QFaGZrW5TJ2Of7mM6tx5+JdWv+I/ XPDg== X-Gm-Message-State: AOJu0Yy/lRkD69pdd6uhk//7njFPSa/iXTkYWIsdUS8FbiEgRRYIH6eP XC7kO1m7NT/fwuj/vS+9Y46jquxPyY0Aew9P0uzVjrrQJr954Uk6J4pQBI2ruBlguqvZAU0RGVa Bty/14vc5O/kf7pex6FCI88WgOWSMDgc= X-Gm-Gg: ASbGncuaHGJvbhbIRGw04/Nn6MmmOTdNUzAinwTSecZQgycvMAwzo1a5BKHYLyOic1Q FY4GKF8b0chEfWQdXHGY73s1K24FxX2ZW4u524WViDDT7JuaVxPgXBasXHFLcYQ== X-Google-Smtp-Source: AGHT+IHoIcOucxdsNxuODIeJvp6/VmI7v8aSNBuwmELVDGnX1Pvabl6xoT2P1FDrGN7CwvjYB/q8DrbmZ7dtGJWvO8Q= X-Received: by 2002:a05:6402:2089:b0:5cf:c1b2:c6ae with SMTP id 4fb4d7f45d1cf-5d0951706afmr4121567a12.17.1732806261481; Thu, 28 Nov 2024 07:04:21 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Rick Macklem Date: Thu, 28 Nov 2024 07:04:10 -0800 Message-ID: Subject: Re: RFC: fixing PR#282995 To: Bob Bishop Cc: FreeBSD CURRENT , Michael Proto Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4XzfjM6CkVz4FB0 X-Spamd-Bar: ---- On Thu, Nov 28, 2024 at 4:36=E2=80=AFAM Bob Bishop wrote: > > Hi, > > > On 27 Nov 2024, at 21:56, Rick Macklem wrote: > > > > Hi, > > > > PR#282995 reports that the "-alldirs" export option is broken, > > since it allows an export where the directory path is not a mount point= . > > > > I'll admit I did not recall this semantic for -alldirs and I now see it= is only > > documented in the "Examples" section of exports(5). > > > > Looking at the code, it appears this was broken between releng1 and > > releng2.0 (about 30years ago) when the call to mount(2) in mountd.c > > was changed from using the path in the exports line to using f_mntonnam= e. > > (The check for "it is a mount point" depended on mount(2) failing becau= se > > the path was not a mount point.) > > > > I do believe the semantic is a useful one, > > Why? Suppose /cdrom is where a CD is mounted sometimes. If this is exported when the CD is not mounted, it will export the "/" file system. --> This export is probably not what the sysadmin wanted. mountd does now generate a warning about this, which was how the exporter spotted the bug. For example (the line in /etc/exports): /cdrom -alldirs will export "/" to "the world" if /cdrom is not mounted. The example in the exports(5) man page claims the export line will fail, so "/" would not be exported. This seems like a better semantic to me. rick > > > although making it that way > > after 30years might be construed as a POLA violation? > > > > So, what do others think I should do with this? > > (A) - Patch mountd to enforce the "must be a mount point when -alldirs > > is specified, plus update exports(5) to state this semantic clea= rly. > > or > > (B) - Patch mountd so that it enforces "must be a mount point when -all= dirs > > is specified, but only enabled via a new mountd command line opt= ion. > > --> ie. Leave the default as not enforced, but allow enforcement= based > > on a new mountd option. > > - Document this in both exports(5) and mountd(8). > > or > > ??? > > (C) - Patch mountd so that it enforces "must be a mount point when -alldi= rs > is specified, but provide a new mountd command line option to rest= ore the old behaviour. > --> ie. Default as enforced, but allow an override based on a new= mountd option. > - Document this in both exports(5) and mountd(8). > > I think that (A) is too POLA-unfriendly. > > > Thanks in advance for your comments, rick > > > > -- > Bob Bishop > rb@gid.co.uk > > > > From nobody Thu Nov 28 15:46:11 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xzgdj0KrLz5dtS0 for ; Thu, 28 Nov 2024 15:46:17 +0000 (UTC) (envelope-from dclarke@blastwave.org) Received: from mail.oetec.com (mail.oetec.com [108.160.241.186]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "mail.oetec.com", Issuer "E6" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xzgdh5GXVz4Hyv; Thu, 28 Nov 2024 15:46:16 +0000 (UTC) (envelope-from dclarke@blastwave.org) Authentication-Results: mx1.freebsd.org; none Received: from [172.16.35.3] (pool-99-253-118-250.cpe.net.cable.rogers.com [99.253.118.250]) (authenticated bits=0) by mail.oetec.com (8.17.1/8.17.1) with ESMTPSA id 4ASFkBWg030302 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT); Thu, 28 Nov 2024 10:46:13 -0500 (EST) (envelope-from dclarke@blastwave.org) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=blastwave.org; s=default; t=1732808773; bh=w8FmMOGyf7x8QQMBEzugM5kKktxy9UXYyCV/H6LPQbs=; h=Date:Subject:To:Cc:References:From:In-Reply-To; b=gT9O3CNs5OFPLk3sgH8ZOV4ZNpc1NsE+M+JP8ITqyJFDMiiWP/6PrH+pTjriLBLZL a/fHqHUYtF+pNC9Vfn7CXVRPWVzjDEyNAtjOz1y4SbPADcV+NYt/OrhJ7/vNgGTZuJ D8OE5p3e4wseEAaI2jDRf2x80Kf0XqKiVU4Mw2T2qCIGszEKefYmnHK6dEZWr/WDKA H+Z4PMA8AG4eMAz7wd8jyszbrZTUQ3HDuOZ0Y+lwa11YMWVou88l5D3EFeD8SJ0Syg JzmW7A7JNNyMvjVbXh/riNJxEC6kdCSumppEYqMLvW5hvIG7djE2lyuAiYag0h53iM 232MNl6p3fi4g== Message-ID: <6d5be5e0-ee68-4858-814b-d0886cef3f85@blastwave.org> Date: Thu, 28 Nov 2024 10:46:11 -0500 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: zpools no longer exist after boot Content-Language: en-CA To: Dimitry Andric Cc: Current FreeBSD References: <5798b0db-bc73-476a-908a-dd1f071bfe43@blastwave.org> <5165C4DB-8BA0-4579-A808-F98FBF3B797F@FreeBSD.org> From: Dennis Clarke Organization: GENUNIX In-Reply-To: <5165C4DB-8BA0-4579-A808-F98FBF3B797F@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-oetec-MailScanner-Information: Please contact the ISP for more information X-oetec-MailScanner-ID: 4ASFkBWg030302 X-oetec-MailScanner: Found to be clean X-oetec-MailScanner-From: dclarke@blastwave.org X-Spam-Status: No X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:812, ipnet:108.160.240.0/20, country:CA] X-Rspamd-Queue-Id: 4Xzgdh5GXVz4Hyv X-Spamd-Bar: ---- On 11/28/24 10:02, Dimitry Andric wrote: > On 28 Nov 2024, at 14:05, Dennis Clarke wrote: >> >> This is a baffling problem wherein two zpools no longer exist after >> boot. This is : >>... >> titan# camcontrol devlist >> at scbus0 target 0 lun 0 (pass0,ada0) >> at scbus1 target 0 lun 0 (pass1,ada1) >> at scbus2 target 0 lun 0 (ses0,pass2) >> at scbus6 target 0 lun 0 (ses1,pass3) >> at scbus7 target 0 lun 1 (pass4,nda0) >> at scbus8 target 0 lun 0 (da0,pass5) > > It has been my experience that with disks connected to external enclosures, these sometimes take a long time to come up. If they come up only after the kernel initially detects ZFS pools, they will not be shown or imported at all. > > If you look at dmesg or boot logs, are the external disks detected around the same time as the NVMe device? > The only remote device is the iSCSI based unit and it seems to be visible neatly. In fact the zpools, all of them, are now available at boot time thanks to using the correct cachefile name. There may be a need to make some note somewhere in a man page that the zpool cachefile used in the RC script is important. -- -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken From nobody Thu Nov 28 15:47:41 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XzggP2xQDz5dtHP for ; Thu, 28 Nov 2024 15:47:45 +0000 (UTC) (envelope-from dclarke@blastwave.org) Received: from mail.oetec.com (mail.oetec.com [108.160.241.186]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "mail.oetec.com", Issuer "E6" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XzggP2MrSz4KF6; Thu, 28 Nov 2024 15:47:45 +0000 (UTC) (envelope-from dclarke@blastwave.org) Authentication-Results: mx1.freebsd.org; none Received: from [172.16.35.3] (pool-99-253-118-250.cpe.net.cable.rogers.com [99.253.118.250]) (authenticated bits=0) by mail.oetec.com (8.17.1/8.17.1) with ESMTPSA id 4ASFlf3M030338 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT); Thu, 28 Nov 2024 10:47:42 -0500 (EST) (envelope-from dclarke@blastwave.org) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=blastwave.org; s=default; t=1732808863; bh=Cb0uK05e0xFhwEZ4997rd2RNx8PNLPt6mW+JZT7ZAbQ=; h=Date:Subject:To:Cc:References:From:In-Reply-To; b=qNFlpelBoEEo8PqaRq8NJ0uH+8oKw8yACBRqH0c9WHXQrPxMFXnJ6vKW8DmtNvq9F vR/UwbJOZrheZqkzlTxsVRX/yu9nQCONWD1L2HIpHnjhvLzr7yfESq9cxea3JlP5YD VxaGvWmt+WP6arzODLvlD5Ys6O0SLTDpsZxCQIjemNDpgBZ199ZpPj+YtWjtmm7gj5 ocgiqGUqmlooqawA4tjcNtBhGkdVP2OuVxYOD4HsHivPZbU7PfbLJo1brCBIrux4Pf nzyGDB1LdYKlfh6y2j32bogN54SsPQgHO/f5AIy0roCWfhJcew9b6Z+dzrVz8NnbPc t+LQ9i+i4q5vA== Message-ID: <722a3644-3af6-4ff9-b1ee-022c32872001@blastwave.org> Date: Thu, 28 Nov 2024 10:47:41 -0500 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: zpools no longer exist after boot Content-Language: en-CA To: Alan Somers Cc: Current FreeBSD References: <5798b0db-bc73-476a-908a-dd1f071bfe43@blastwave.org> <22187e59-b6e9-4f2e-ba9b-f43944d1a37b@blastwave.org> From: Dennis Clarke Organization: GENUNIX In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-oetec-MailScanner-Information: Please contact the ISP for more information X-oetec-MailScanner-ID: 4ASFlf3M030338 X-oetec-MailScanner: Found to be clean X-oetec-MailScanner-From: dclarke@blastwave.org X-Spam-Status: No X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:812, ipnet:108.160.240.0/20, country:CA] X-Rspamd-Queue-Id: 4XzggP2MrSz4KF6 X-Spamd-Bar: ---- On 11/28/24 09:52, Alan Somers wrote: > On Thu, Nov 28, 2024, 8:45 AM Dennis Clarke wrote: > ... > > For "zpool import", the "-c" argument instructs zfs which cachefile to > search for importable pools. "-O", on the other hand, specifies how the > cachefile property should be set after the pool is imported. > I have to wonder what value there is in NOT having the cachefile property set in a zpool ? Certainly given that the zpool RC script really wants to check in a few places and then use those cache files. -- -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken From nobody Thu Nov 28 15:48:48 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xzghz0MhNz5dtSR for ; Thu, 28 Nov 2024 15:49:07 +0000 (UTC) (envelope-from rb@gid.co.uk) Received: from gid2.gid.co.uk (ns0.gid.co.uk [IPv6:2001:470:94de::240]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gid2.gid.co.uk", Issuer "gid2.gid.co.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xzghy06CYz4Kvw for ; Thu, 28 Nov 2024 15:49:05 +0000 (UTC) (envelope-from rb@gid.co.uk) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of rb@gid.co.uk designates 2001:470:94de::240 as permitted sender) smtp.mailfrom=rb@gid.co.uk; dmarc=pass (policy=none) header.from=gid.co.uk Received: from mx0.gid.co.uk (mx0.gid.co.uk [194.32.164.250]) by gid2.gid.co.uk (8.15.2/8.15.2) with ESMTP id 4ASFn3gZ030119; Thu, 28 Nov 2024 15:49:03 GMT (envelope-from rb@gid.co.uk) Received: from smtpclient.apple ([89.248.30.154]) by mx0.gid.co.uk (8.14.2/8.14.2) with ESMTP id 4ASFmwqP045552; Thu, 28 Nov 2024 15:48:58 GMT (envelope-from rb@gid.co.uk) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\)) Subject: Re: RFC: fixing PR#282995 From: rb@gid.co.uk In-Reply-To: Date: Thu, 28 Nov 2024 15:48:48 +0000 Cc: FreeBSD CURRENT , Michael Proto Content-Transfer-Encoding: quoted-printable Message-Id: <4FEA762C-5E0F-4D3F-82D9-DF93CF1BC1F5@gid.co.uk> References: To: Rick Macklem X-Mailer: Apple Mail (2.3776.700.51) X-Spamd-Result: default: False [-3.78 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.98)[-0.981]; DMARC_POLICY_ALLOW(-0.50)[gid.co.uk,none]; R_SPF_ALLOW(-0.20)[+mx:c]; MIME_GOOD(-0.10)[text/plain]; TAGGED_RCPT(0.00)[]; ARC_NA(0.00)[]; FROM_NO_DN(0.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; MIME_TRACE(0.00)[0:+]; APPLE_MAILER_COMMON(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; R_DKIM_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FREEMAIL_TO(0.00)[gmail.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@FreeBSD.org]; RCPT_COUNT_THREE(0.00)[3] X-Rspamd-Queue-Id: 4Xzghy06CYz4Kvw X-Spamd-Bar: --- > On 28 Nov 2024, at 15:04, Rick Macklem wrote: >=20 > On Thu, Nov 28, 2024 at 4:36=E2=80=AFAM Bob Bishop = wrote: >>=20 >> Hi, >>=20 >>> On 27 Nov 2024, at 21:56, Rick Macklem = wrote: >>>=20 >>> Hi, >>>=20 >>> PR#282995 reports that the "-alldirs" export option is broken, >>> since it allows an export where the directory path is not a mount = point. >>>=20 >>> I'll admit I did not recall this semantic for -alldirs and I now see = it is only >>> documented in the "Examples" section of exports(5). >>>=20 >>> Looking at the code, it appears this was broken between releng1 and >>> releng2.0 (about 30years ago) when the call to mount(2) in mountd.c >>> was changed from using the path in the exports line to using = f_mntonname. >>> (The check for "it is a mount point" depended on mount(2) failing = because >>> the path was not a mount point.) >>>=20 >>> I do believe the semantic is a useful one, >>=20 >> Why? > Suppose /cdrom is where a CD is mounted sometimes. > If this is exported when the CD is not mounted, it will export > the "/" file system. > --> This export is probably not what the sysadmin wanted. > mountd does now generate a warning about this, which > was how the exporter spotted the bug. > For example (the line in /etc/exports): > /cdrom -alldirs > will export "/" to "the world" if /cdrom is not mounted. I will agree that is undesirable. > The example in the exports(5) man page claims the export > line will fail, so "/" would not be exported. This seems like > a better semantic to me. It=E2=80=99s certainly safer but will cause client mounts to fail as = well. It would be nicer to export an empty directory. > rick >=20 >>=20 >>> although making it that way >>> after 30years might be construed as a POLA violation? >>>=20 >>> So, what do others think I should do with this? >>> (A) - Patch mountd to enforce the "must be a mount point when = -alldirs >>> is specified, plus update exports(5) to state this semantic = clearly. >>> or >>> (B) - Patch mountd so that it enforces "must be a mount point when = -alldirs >>> is specified, but only enabled via a new mountd command line = option. >>> --> ie. Leave the default as not enforced, but allow = enforcement based >>> on a new mountd option. >>> - Document this in both exports(5) and mountd(8). >>> or >>> ??? >>=20 >> (C) - Patch mountd so that it enforces "must be a mount point when = -alldirs >> is specified, but provide a new mountd command line option to = restore the old behaviour. >> --> ie. Default as enforced, but allow an override based on a = new mountd option. >> - Document this in both exports(5) and mountd(8). >>=20 >> I think that (A) is too POLA-unfriendly. >>=20 >>> Thanks in advance for your comments, rick >>>=20 >>=20 >> -- >> Bob Bishop >> rb@gid.co.uk >>=20 >>=20 >>=20 >>=20 >=20 -- Bob Bishop rb@gid.co.uk From nobody Thu Nov 28 15:55:51 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xzgs10DS0z5dtv5 for ; Thu, 28 Nov 2024 15:56:05 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lj1-f171.google.com (mail-lj1-f171.google.com [209.85.208.171]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xzgs05KrFz4Mlb; Thu, 28 Nov 2024 15:56:04 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-lj1-f171.google.com with SMTP id 38308e7fff4ca-2ff550d37a6so11417981fa.0; Thu, 28 Nov 2024 07:56:04 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732809363; x=1733414163; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=3bWj8X1hGnRwIGLoA2Civ1vke8LiOji56hB224G+PuU=; b=kCCR5KtIt/fIQxSw5mqDGJ47Mi8FyLywL5n55tn0QgwykNs3apINQ/3m7/4Kt9RfkF M5yMQzFGQWaJ3oEUIGArOcieXDsryUs2uO4K36Rf+/kMzbgSiIEOuSyrmhp+H69rirxQ PzPeTXPXoW3r8pGSvdRKlEW8/vZbwYJ55iy2Tov5nCmldMc4LwQmu5SdqXQtOytFdxWi 14+R3jpjzTA8/mC8ma1e+kqQhajIGb06T6FwwcvdIXv6Qn5bUUFa5Y/UzsyeME/PvW4H 3yDsWUrb7ZEWCEI46AYA7pGsDnSbayFF790UN/xMu8PRNszm68o24A1DRvQsjoO7gTQW PV6A== X-Forwarded-Encrypted: i=1; AJvYcCX0bSleXemMxP/7XaGKmCmULR4ZSUr5XQ5QLi7Symh/466BGIvuDfVfONoTXcf0ikFI6Nm7GztGVVCrjaiufBM=@freebsd.org X-Gm-Message-State: AOJu0YyeHaZYNT1NW89BVdF88sZtfIrzI9DE4TbwoXZv0wcYC56pYyoA ID44CPNvc0M/lIyuvNzNZhihUr7DxeDh5hkyA6Q4+/7eOlpaVHYZfK0nONhfXDpq2ZHq3VVW+eq bD6TsxfOrHeYnJsgU5oI/thqnp9TahA== X-Gm-Gg: ASbGnctoUnY/pqFwnaHZSKD409mS/pZ2lDrFlmchVdy70H1+WEwKVhkhdKlyJwy+mLJ mkld3E+PrEkMlN6QPLUhS3VzSNYdEJv9MdDXSIUSW3/A7ZbRj+1oljS5tVV6Evg+v9A== X-Google-Smtp-Source: AGHT+IFR11FDrIAwLZiABntrQD18TWmFjX8Zz3z8DJyD3D41XvlN/TXxjdxEXJlWXtvyFVI1x6npka6HPyn3zIeTxEU= X-Received: by 2002:a05:651c:19a2:b0:2fb:4b40:1e18 with SMTP id 38308e7fff4ca-2ffd605c41cmr41689981fa.13.1732809362568; Thu, 28 Nov 2024 07:56:02 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 References: <5798b0db-bc73-476a-908a-dd1f071bfe43@blastwave.org> <22187e59-b6e9-4f2e-ba9b-f43944d1a37b@blastwave.org> <722a3644-3af6-4ff9-b1ee-022c32872001@blastwave.org> In-Reply-To: <722a3644-3af6-4ff9-b1ee-022c32872001@blastwave.org> From: Alan Somers Date: Thu, 28 Nov 2024 09:55:51 -0600 Message-ID: Subject: Re: zpools no longer exist after boot To: Dennis Clarke Cc: Alan Somers , Current FreeBSD Content-Type: multipart/alternative; boundary="00000000000084ff8b0627fb1e94" X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4Xzgs05KrFz4Mlb X-Spamd-Bar: ---- --00000000000084ff8b0627fb1e94 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Nov 28, 2024, 9:47=E2=80=AFAM Dennis Clarke = wrote: > On 11/28/24 09:52, Alan Somers wrote: > > On Thu, Nov 28, 2024, 8:45=E2=80=AFAM Dennis Clarke > wrote: > > > ... > > > > For "zpool import", the "-c" argument instructs zfs which cachefile to > > search for importable pools. "-O", on the other hand, specifies how the > > cachefile property should be set after the pool is imported. > > > > I have to wonder what value there is in NOT having the cachefile > property set in a zpool ? Certainly given that the zpool RC script > really wants to check in a few places and then use those cache files. > > -- > -- > Dennis Clarke > RISC-V/SPARC/PPC/ARM/CISC > UNIX and Linux spoken > Usually the default value for cachefile is sufficient. In fact, the only time I've ever needed to set cachefile has been when I DON'T want the pool to import at boot. I wonder if there was some other reason, since resolved, why your pools didn't import the first time. And subsequently they didn't import because you set the cachefile to a non default value? > --00000000000084ff8b0627fb1e94 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Thu, Nov 28, 2024, 9:47=E2=80=AFAM Dennis Clarke <dclarke@blastwave.org> wrote:
=
On 11/28/24 09:52, = Alan Somers wrote:
> On Thu, Nov 28, 2024, 8:45=E2=80=AFAM Dennis Clarke <dclarke@bla= stwave.org> wrote:
>
...
>
> For "zpool import", the "-c" argument instructs zf= s which cachefile to
> search for importable pools. "-O", on the other hand, specif= ies how the
> cachefile property should be set after the pool is imported.
>

I have to wonder what value there is in NOT having the cachefile
property set in a zpool ?=C2=A0 Certainly given that the zpool RC script really wants to check in a few places and then use those cache files.

--
--
Dennis Clarke
RISC-V/SPARC/PPC/ARM/CISC
UNIX and Linux spoken

Usually the default value for cachefile is sufficient.= In fact, the only time I've ever needed to set cachefile has been when= I DON'T want the pool to import at boot. I wonder if there was some ot= her reason, since resolved, why your pools didn't import the first time= . And subsequently they didn't import because you set the cachefile to = a non default value?
--00000000000084ff8b0627fb1e94-- From nobody Thu Nov 28 16:31:40 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XzhfP03Vrz5dx2g for ; Thu, 28 Nov 2024 16:31:57 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic303-24.consmr.mail.gq1.yahoo.com (sonic303-24.consmr.mail.gq1.yahoo.com [98.137.64.205]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XzhfN4SJjz4RST for ; Thu, 28 Nov 2024 16:31:56 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732811514; bh=1UMUnY1gL8ZUsndVXLtif9YW5dy0MHnKz+k6gcmJuY4=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=rNFj4TRnKeLm88/YwlTPKQ0sB8OospzN8aSeWcIYTVueFfvzSlS6YYG9os8UIvEO/6gRViAwH2uT1tF4zjb29Sn6edXa/Xhn9KmojIMNocBiRakpY4eTQFqsX51qMa1frVTcRcY+ta23MwKqw4ZkWaeEICKzKCdrH127HoQXWmVhAcAciFIRXCDDRMdg8mEDwYggnQqfQX8XoAcGZPoddycHaUzc9M8y0ZvUSG0lX5Chlqtp1dDTTxwa6c/OIBunlA0Cx7WvRwtv7NLPvGtofHwU+MMlp/bSO9Wi+DyDDxsiGfvX37Mw5k81Miu6bNMdwyQFIYXqa3W2FESKX0aC+g== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732811514; bh=TD+FjDzb5XNOnjjscZa9G/7gbATC3ZXIeqOq1AQuCkd=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=K6JeGL9GYHHJZGPotRCOxDfnWrl1ZSGFmoOXR/KlKPksuuuPD7MxZdo+pkNluis4S9R92qW3EuWJiQSJTXTTHJjLPPvBUOKbU4FvVMKBSyQhGByfjk9zXc3KLTKlB9xmThdLMU24GUfjjz3JfW+xxXID6MGA1HYhrLJiNqMlddMx1GbOkQ7Rxlj210ecXg5RktFHGJse+wEKZfcwYjVqJK6oOWj1CAPV6AKJ5qRJw5q0U++S3vrxMGmXQzKz5vf4iDRCTmzInJLp9U9vJrZi4f+BUS4xiHV3OtuiAHl7rJ6PcGEUUem4jiBs7P14Vq1FCnzMWcbUX/o5OerKgSnQWw== X-YMail-OSG: L39G8QwVM1lWmMPPU0lY.tN1oJrbjKbMUq7rSiH72OeCoT9GFkSb3bpoZe8E9a3 HhTlD3KLNwZb3MRLskopdtgztBZMdjmfb13mNBg6gUcpNGW9S1ddv.3J8g8gSPvsudSHpE9EC5Fq WQAoyLX92KDbVXmsqepGuui0ebt3opkoUPWAWswOXS0K9YRhk_x4nse4tkDMRfRL7qv1MDgMI2Yw vAOLa6CZm0xxcP0CmzZDMv7oOyAlCZLz6Ww3xaM3j8vY95Sn3rMwpS7ZiGfvnynUaTeLsauFM2xY FOG6tList2Dl4p7JCXrT6nfrs9w7IYO.TjEpK7KUUIVkp_P3SUHeSKE8.NOYcV8PUAah4K7YtPFB TP8wInyAPZNBMtdCQweYTMDz9dBLPEAzPy5Dk0ARtuEY34huA4vK1FOnkV6Q2HWZDhRpXYXjJDMf vQD6IJYOpmgcsm8noz77JrSMll7iDHsljTpdAbTi3uOZY7sV34g4NzJAQrVsU9niDM8Qh9zlAB5Z aZW1FkYTgpsWikk4ikHr.AHZR1LOJAHEM6aNbNZA4bdIY.uV9bQHvRRcE1DvMBOonD85wLinfdmQ m.pEpRyhLyBrfTWjjnt3FXxCtHnwq21bPySoujXbXtCCr.RVdI33TRg8_btvX7wwvYuDsqcBKUOQ HlRZkTUD_2psPmsx0ZvMWH9jnbwRZo2bOR5TgqIogZNrzBchd64bFBq1OpCiK1IPEPHW7tNRj24w 9HpwBO2e7qfi1LCk885XFiHdezQbvwW4MmS3INrTX5NqlScAhG4GulgyY9e7whmUplbLPaJ5BB5w jPxDdaKEHJMsr4La2bq4Wbj5nuHKDWzQLM1RA2RjgjxcT1O3PUiNZVCsGltcADCgG.QmvAlBuE52 Rsk4RZFFdjSPVS.X34Qb40KR8OCbD2xMjVslTTBSY3IOWgHCqk4tFA.fTMmwOGnaUzSMSMXtqGmP pwVqRo2ouSYfMUB7qaDIal64d62kwyMqUduzYXXT3MprBDubGQsGRIPEoj4Bw6BjKEtldp8SkPxJ 1Jh8ooFykVG.7KOx7QrbNoHGmogE0h8ABZO3VtornEW3I7cf3zzM3Mz31OSnfBNyEJx2SGLumOai r2kI92oDAlLcARJGTLz_F7K8zgVYc3RXTZ1jQE7C94QS6JP6mfKGeHgYUD6SjtL5k6yNfI4ZC8zI Lc8QXlM96_GJvPRUOf8mjBE.XW0m8qXqOcqmrlAW9kZMKq2I.e0LSaig1Bk16Ru5WSqREmvg0rXn 8BPG5hML.dC1aXL8wds31dbEvi6UE88VGZyqH9AW2R5SiH.z7DTlbbxoSAqLdVEarDWhRHDgUZMT ub9ecp5lDDgmoDZ0zajXit7hWBpxC3kVTJYL0f6vHndsrQD0_fgdgRA9n6FHldAeVuM7d_8PM4xX _y5QxPI05Rm0TOJQaUsyRpRP.3F5GydDbEfx6vPtsX8iKtBQyiSm9m6pXhSSH9DEuvZsq3Pvu7K4 _UHSqAGeOHZV.uZhZ57QFZ5AKQvckmw1D3HeWJjv8hK5pxw58QHaifTbEy6QJkb6POZBDrTWf7A7 sF8OHC81LsuqIdY_lDEyc1o3rrpwJwxq7i4YTeVgBHtL8yB2lmfPBtw2VFn8UwGRczWs4jwYXxxq o9KW8sxLlrGYvNjbVV4r75oKXBOzfT.Yo.btwiIkVxG.d_CClV4HFvcJ7gmEkBeFQgwu3KPtDc1H PVCiKkp1wZzyxS66tgZUx2ZvAxmpAXen87swX1DF8n7KLAL2X69IhIHklRWxKdI6rKcJr9XSecim Qd8kqkfCP_WRAYUPaMuV6htkzQw4nTJ..B2dt5nKQfZT2sQO5W6SoCkuEL0LCwSnzroAuZhhTE12 p2V_r1jesPHQJSk3TimvXVPusuB3EEKYHsbivwEklDQfChXfiFn1M5FyIklfdZ1ra6xxtGufAh0u HhQcP8DGY9.kEsM2OcUb67eW.Yt_7MktFxYVat4dLrAMICN8AEA3JEwv89bvaDyG6J04fiXnxAP. okdhq77RrUUDBhzJUkPfxS0qAmDmxf7b5yS67Xf1iobgXlUbB.SKfLrl9IGHNBszQmxR51IQI14h sL7N3imhTw5pPdNIjCOqu_eYjw8L.iN3dVWlHdh8okZyp5fWEH86SeI5zvW25Ev_pLKOckSuJh79 budj5Ugh5bO4Nq6DAgC91hqGgpv3oITzXt3dNwaiN7ZyAAAZgmzYr6G57TCK7N73yrJt1Quz3_a1 lyf8sQMQ6sYSSSybqTBgUhXwB0apdmyO_TccqDZf6WWDQdk8xfcmipz2aQmn7AFVqH_sT9J.Iedd kSEmT9TvK8g-- X-Sonic-MF: X-Sonic-ID: b80a38da-32e5-4cd3-a385-1bf3baab6d48 Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.gq1.yahoo.com with HTTP; Thu, 28 Nov 2024 16:31:54 +0000 Received: by hermes--production-gq1-5dd4b47f46-bwg5p (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 8980a201acedad9612dc9ced57934661; Thu, 28 Nov 2024 16:31:51 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51.11.1\)) Subject: Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros] From: Mark Millard In-Reply-To: <5e37b8a5-2bd2-49b5-9746-674bd26ad770@FreeBSD.org> Date: Thu, 28 Nov 2024 08:31:40 -0800 Cc: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , Konstantin Belousov , Dimitry Andric , "jah@freebsd.org" , dougm@freebsd.org, Alan Somers , Mark Johnston , FreeBSD Current , Guido Falsi , Yasuhiro Kimura , ports@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <0654A56E-08C7-42CC-A6D8-63C85120C1D8@yahoo.com> References: <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> <0690CFB1-6A6D-4B63-916C-BAB7F6256000@yahoo.com> <3660625A-0EE8-40DA-A248-EC18C734718C@yahoo.com> <865xoa2t6f.fsf@ltc.des.dev> <69A2E921-F5E3-40D2-977D-0964EE27349A@FreeBSD.org> <4AE5B316-D7EB-4290-8D52-7FBF244EA7A4@FreeBSD.org> <33D56E3E-6476-48E8-B115-B906629B8AF5@yahoo.com> <65d47ca6-b0b9-4c03-9e36-d0f2cf6b4937@FreeBSD.org> <86zflj1t6b.fsf@ltc.des.dev> <5e37b8a5-2bd2-49b5-9746-674bd26ad770@FreeBSD.org> To: Andriy Gapon X-Mailer: Apple Mail (2.3776.700.51.11.1) X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4XzhfN4SJjz4RST X-Spamd-Bar: ---- On Nov 28, 2024, at 04:19, Andriy Gapon wrote: > On 28/11/2024 13:42, Dag-Erling Sm=C3=B8rgrav wrote: >> Andriy Gapon writes: >>> FWIW, I am not sure if it's relevant but I am seeing a similar = pattern >>> of corruption on tmpfs although in a different context, on FreeBSD >>> 13.3. >> Not relevant at all. In this case the file is not actually corrupted >> but `install(1)` skips over some of it when copying because = `SEEK_DATA` >> is implemented incorrectly. >=20 > Still could be relevant... > I don't know the "true state" of my corrupted files, I only observe = the consequences. And the files get some post-processing, then they are = uploaded and originals are removed. So, the problem could be not during = the write phase, but during the read phase of post-processing. First an FYI for why I started with 2bed60 instead of a page boundry: 2bed60 was the start of .got.plt, which is what was involved in the program crash. In every case, it seems likely that the whole page containing that start was zero, no matter if it should have been at the page start or not. The page start is just not what I was focused on for reporting. So I expect a "tail of page is all zero but should not be, start of page was a normal not-all-zero" problem would be a distinct problem. Or are you always seeing the problem as a full page of zeros instead of just the tail of that page (that should not be all zero)? In Dag-Erling's wording, "this case" refers to the context I was gathering investigative data for, not your context, as I understand it. [I've referenced: https://lists.freebsd.org/archives/freebsd-fs/2024-November/003855.html ] As for: "The writes are done by appending variable sized records to a file. There are no seeks or overwrites.": Am I to interpret that as: ) New file with just sequential writes that are variable sized? vs. ) Appending to a pre-existing file? (That would involve seeking and typically merging new data with old data from the original last-page-with-data and writing that update back out.) =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Thu Nov 28 18:16:16 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xzkz64RSzz5f4bG; Thu, 28 Nov 2024 18:16:34 +0000 (UTC) (envelope-from scf@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xzkz63gg7z4fMy; Thu, 28 Nov 2024 18:16:34 +0000 (UTC) (envelope-from scf@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732817794; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=DvxOhPNGi/OvPPZ+dkB4Is8Cfmm8yLU56rHq7zAXA5o=; b=KRUFIxmU9Cap4NnzEqd0CqUsYWqSAHn3MsQytr0q9YfIjphH7383HlJ5Phsr3BSWm4jNsC HzXfbJNqvJeqgSympoQiLItJTWgQAyljpXXZXI1aIAM5qijOCTezcl8wY4T+TMS0EcnyQE uBWCy1VraXy2NGcC96IkbQW3tY+dbQXB1oh7HhVulLD6/SlHJgbrcb+bcYp2wuAA0L5wgB SY+hGLc0mSQQOTGbXwlmVTn+0I6U3WV7ugmm4ynilYQcT72wosuErpa2rampnFk7uCQvjE Aum7qLJpH+E2gg0umRcCVXJ0gh8E8FQ5uMRyg4XoT4Dd2PElapGXUkudsHbT8w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732817794; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=DvxOhPNGi/OvPPZ+dkB4Is8Cfmm8yLU56rHq7zAXA5o=; b=V/ep+BfTkCqPP8Spzs4xLGRdAN5hdYKguUtfB+eCpxKvX3nI7EW4R/vyjkY6DUqRlufyOG PLSdWPfdchwpaMBbJk/Radbdhxx2cpNzFgtmL82z8coBeOOePBqrXB/24tMObZcV4BB/U2 nquqdPNYvMp0yVL0QHbjlGl36CqeSkmCx8xnt/9DjJTUal+oP6hCARuPK/5dEfxhhdNNo2 MSmte12T9Ljkp0OEC2Mo6DO8/kHtNzGbs//v+GO0CJoIXDSh3RvZLDLECjC1K+yHbaJVuF Tkx4zuDvba6InOuBAzQC1giCt4EeA32NHnBo68QJtJvZZ6c/QPRk49E+QtcTMQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1732817794; a=rsa-sha256; cv=none; b=ZaYI/2jIgw8J/BSdGNo4HeEJM89syWO179lH2AE5Qp90ZRhjLvSnX6oSBKXSUum+7TtoTb uTa15lR7tqdGz/74YRn/zRPKXs0inPB7vRGZUKPyYhfLVguB6LwhRX0uHP22s3JqN7smzp YG0tye1NoPIKApe4PDUzxHICDj3gQUJwxm+C7BSvqMQfab7UI6OrUZEdKaNvxusg4E5XBG xxBFchKDlKGCb6p54H0eVL68riz9VsWrQnhS86Rqka26NDIzYh2HEz0dtnoqEi5ISLQ/8l u0jWTgSodPXUsmy38LZ+Jennuyxg3PCq2z87tKiJKzGyYvPgE14+0WOHjJR3nA== Received: from thor.farley.org (1609341-v107.1360-static.crmlinaa.metronetinc.net [104.254.222.35]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: scf/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Xzkz62GLSz16GT; Thu, 28 Nov 2024 18:16:34 +0000 (UTC) (envelope-from scf@FreeBSD.org) Date: Thu, 28 Nov 2024 13:16:16 -0500 (EST) From: "Sean C. Farley" To: freebsd-ports@FreeBSD.org, freebsd-current@FreeBSD.org Subject: Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros] In-Reply-To: <0690CFB1-6A6D-4B63-916C-BAB7F6256000@yahoo.com> Message-ID: <813bef1e-0189-27b2-9ee1-8ebb57a82296@FreeBSD.org> References: <7c9c3cf5-bbd1-4642-8d04-33aa07a4db02@madpilot.net> <9df256a8-c6ed-46d9-b955-fc2657c12d36@madpilot.net> <5c502054-7353-4a1e-8350-c403482e9c0d@madpilot.net> <3127C3BA-FC93-4636-ADDB-89518DE9C60D@FreeBSD.org> <86ed2zsp6l.fsf@ltc.des.dev> <5f24a570-26e0-4c0a-817f-591a234fd07b@madpilot.net> <5918C6A1-8FDB-40CA-8C86-EB7B7BE75A2E@yahoo.com> <86ed2zc8r5.fsf@ltc.des.dev> <45098ccf-4dc6-426c-849a-c923805d6723@madpilot.net> <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> <0690CFB1-6A6D-4B63-916C-BAB7F6256000@yahoo.com> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/mixed; BOUNDARY="3279119474-871193679-1732816971=:34935" Content-ID: This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --3279119474-871193679-1732816971=:34935 Content-Type: text/plain; CHARSET=UTF-8; format=flowed Content-Transfer-Encoding: 8BIT Content-ID: On Mon, 25 Nov 2024, Mark Millard wrote: > On Nov 25, 2024, at 18:05, Mark Millard wrote: > >> Top posting going in a different direction that >> established a way to control the behavior in my >> context . . . > > For folks new to the discoveries: the context here > is poudriere bulk builds, for USE_TMPFS=all vs. > USE_TMPFS=no . My test context is amd64 on a > 7950X3D system with 192 GiBytes of RAM. Others have > other contexts, including an Intel system. I have been seeing some odd behavior from Firefox as well as with poudriere builds on my system. Both of which are touching a tmpfs system as I have setup /tmp as tmpfs, which Firefox uses, and USE_TMPFS=all. The system has been an experiment, for me, with undervolting. I have been attributing any flakiness to the undervolting, but I have reduced that a lot while the instability has been consistent as in it has stayed rare. I cannot tell how many times I have run memtest86 on this system. System setup: - FreeBSD 14.2-STABLE - i7-14700K (latest BIOS which *should* fix Intel power-related bugs) - 128 GiB RAM - ZFS (mirrored drives) - 2 encrypted swap partitions (64 GiB each, lightly used) - Lightly undervolted (-0.06 offset to Global Core SVID Voltage) - /tmp is tmpfs - ${HOME}/.cache is tmpfs - Poudriere: - USE_TMPFS=all - ccache - jail version in sync with host - /usr/ports is mounted with nullfs I have wondered if it was swap-related, but recently I noticed a build failure with games/veloren-weekly where swap was available but zero bytes were used. The system was under little load at the time so less chance of undervolting being an issue. Build failure: ----------------------------- portpicker = { path = '/wrkdirs/usr/ports/games/veloren-weekly/work/portpicker-rs-df6b37872f3586ac3b21d08b56c8ec7cd92fb172' } ===> Updating Cargo.lock error: checksum for `windows_x86_64_msvc v0.42.2` changed between lock files this could be indicative of a few possible errors: * the lock file is corrupt * a replacement source in use (e.g., a mirror) returned a different checksum * the source itself may be corrupt in one way or another unable to verify that `windows_x86_64_msvc v0.42.2` is the same as when the lockfile was generated *** Error code 101 ----------------------------- Restarting the build finished successfully. >> I changed USE_TMPFS=all to USE_TMPFS=no : >> >> USE_TMPFS=all gets the failure *snip* >> vs. >> USE_TMPFS=no works just fine >> >> So it is a FreeBSD system error associated with >> use of tmpfs . > > Recent work on tmpfs includes: > > Mon, 09 Sep 2024 > • git: 8fa5e0f21fd1 - main - tmpfs: Account for whiteouts during rename/rmdir Jason A. Harmening > Fri, 04 Oct 2024 > • git: 75734c4360fc - main - tmpfs: check residence in data_locked Doug Moore > Sun, 13 Oct 2024 > • git: ec22e705c266 - main - tmpfs: remove duplicate flags check in tmpfs_rmdir Alan Somers > Thu, 24 Oct 2024 > • git: db08b0b04dec - main - tmpfs_vnops: move swap work to swap_pager Doug Moore > > swap_pager (given the reference to it above): > > Tue, 08 Oct 2024 > • git: d0b225d16418 - main - swap_pager: use iterators in swp_pager_meta_build Doug Moore > Fri, 11 Oct 2024 > • git: 1107834090be - main - swap_pager: swapoff detecting object death Doug Moore > Thu, 24 Oct 2024 > • git: 34951b0b9e78 - main - swap_pager: move scan_all_shadowed, use iterators Doug Moore > • git: 02e85d1c8a41 - main - swap_pager: fix assert in seek_data Doug Moore > • git: faa9356f97d2 - main - swap_pager: fix seek_hole assert Doug Moore > Sat, 26 Oct 2024 > • git: 39f6d1e7f835 - main - swap_pager: iter in haspage, lookup, getpages Doug Moore > Wed, 13 Nov 2024 > • git: d11d407aee48 - main - swap_pager: Ensure that swapoff puts swapped-in pages in page queues Mark Johnston > > I do not know at this time when the corruptions started. The > above is only suggestive. Thank you for listing those. I need to find some time to look over those changes although I am no kernel guru by a long shot. However, I see now that it looks like much more knowledgeable people are already looking on the current mailing list at the issue. Sean -- scf@FreeBSD.org --3279119474-871193679-1732816971=:34935-- From nobody Thu Nov 28 19:55:07 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xzn944ff2z5fBRG for ; Thu, 28 Nov 2024 19:55:20 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com [IPv6:2a00:1450:4864:20::52d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xzn942j8Jz4pHs for ; Thu, 28 Nov 2024 19:55:20 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-x52d.google.com with SMTP id 4fb4d7f45d1cf-5d071f70b51so1388762a12.3 for ; Thu, 28 Nov 2024 11:55:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1732823718; x=1733428518; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=n1yTVFIuvOJbiqVHghLBZ9rLoLkclQneWIpIczwlhbI=; b=M/TmFcxZct+seFjrMOfSEfh7Zs2zPwMVVaazMMFhZ35otcSWCHpsf4noffoCL3Dvwa j+3x4II4lTjsmkkSwmia2oEpmsP2fnCDkZPpsFqM7qhwDtrr39AvOIZ9YNEpAppTKf8K xENsEQFvCdk+fIpDAF9zqH02ysUfY7oNiDrfn1e2RXf5u5txPJYXY/Fk2K9wL86gap4s +k4aL90jsX2+QcLXdjg5mtvApRe9HX1+LiB//XvyBZ8e+Hnvy1v+4FvYdfoFsnZJadEW BIuS38B/yTYnd/qjKlm1rwQ+7lh0bK0saZCrCHsTzZ+lqHqjC38lK5v6TA6cBGM7TTkg 51gw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732823718; x=1733428518; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=n1yTVFIuvOJbiqVHghLBZ9rLoLkclQneWIpIczwlhbI=; b=SzaRpwGyQacD1NuqQaYe29PbS9tbfnIGPPRZWaJQvyNPfWFwtc2PyRsug7w8SgKvcg yJVryikops08NrFGjZdp/p0XGpQDzGaJBZYwS6YRsWQqxgX9vAPnsYedggOwLKp4EhtU NhNfblPl1p5LkH4myJODndYThkg6QgulG7sOkt94umHn5iB3VJOsi0Y0zrrGCuZdIR4h /NpD1+z89mIkXnz2NPEAwFAfkqGvZSWaE344XttnZrNPOrU9rjdWgOc5Wq4tvfa+/yEW mGtXVRNSx77nz3Oyj7IFLQFydVP6+ko82BMYvFP41vbzpCcsUdFlyNT5sclOx3sSqdPT n6Fw== X-Gm-Message-State: AOJu0YywBbPyylCHxE3/BbXa1AYKQIfZghpUoNrOIi3FFaP1BsqPzp8R rZMxgfLlpTVLCM5Y1EQlZ4JyWzgpru5bpjUIt6XVMaoK063W8oYlmQlakXz9lF/bF+Tl5itUw8s ncom8OUVo/L2ATKAEJiA7Ahv8UA== X-Gm-Gg: ASbGnctOpOVLhIDBsyxHLTydccctYp9+h6VpxIMUGmcqmXP2xH/XFRUbJ8gO4jvTacR Uk+TAacfR0Z/H/arjmWQGf2UZKIbX8z9curnM+L3MWjgqk4NX7SHY4vtRCoACBlU= X-Google-Smtp-Source: AGHT+IG3g8pVkBvjim6Q5WvY12sSCi3Ce+/cWgZM3MM5MZW4BVM7+v/k1tuOG0kiHI0gt1fuP1o60Npkpn7Ar8HW5zU= X-Received: by 2002:a05:6402:354f:b0:5cf:f42f:de82 with SMTP id 4fb4d7f45d1cf-5d080b8c885mr7730606a12.7.1732823717387; Thu, 28 Nov 2024 11:55:17 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 References: <4FEA762C-5E0F-4D3F-82D9-DF93CF1BC1F5@gid.co.uk> In-Reply-To: <4FEA762C-5E0F-4D3F-82D9-DF93CF1BC1F5@gid.co.uk> From: Rick Macklem Date: Thu, 28 Nov 2024 11:55:07 -0800 Message-ID: Subject: Re: RFC: fixing PR#282995 To: rb@gid.co.uk Cc: FreeBSD CURRENT , Michael Proto Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4Xzn942j8Jz4pHs X-Spamd-Bar: ---- On Thu, Nov 28, 2024 at 7:49=E2=80=AFAM wrote: > > > > > On 28 Nov 2024, at 15:04, Rick Macklem wrote: > > > > On Thu, Nov 28, 2024 at 4:36=E2=80=AFAM Bob Bishop wrote= : > >> > >> Hi, > >> > >>> On 27 Nov 2024, at 21:56, Rick Macklem wrote= : > >>> > >>> Hi, > >>> > >>> PR#282995 reports that the "-alldirs" export option is broken, > >>> since it allows an export where the directory path is not a mount poi= nt. > >>> > >>> I'll admit I did not recall this semantic for -alldirs and I now see = it is only > >>> documented in the "Examples" section of exports(5). > >>> > >>> Looking at the code, it appears this was broken between releng1 and > >>> releng2.0 (about 30years ago) when the call to mount(2) in mountd.c > >>> was changed from using the path in the exports line to using f_mntonn= ame. > >>> (The check for "it is a mount point" depended on mount(2) failing bec= ause > >>> the path was not a mount point.) > >>> > >>> I do believe the semantic is a useful one, > >> > >> Why? > > Suppose /cdrom is where a CD is mounted sometimes. > > If this is exported when the CD is not mounted, it will export > > the "/" file system. > > --> This export is probably not what the sysadmin wanted. > > mountd does now generate a warning about this, which > > was how the exporter spotted the bug. > > For example (the line in /etc/exports): > > /cdrom -alldirs > > will export "/" to "the world" if /cdrom is not mounted. > > I will agree that is undesirable. > > > The example in the exports(5) man page claims the export > > line will fail, so "/" would not be exported. This seems like > > a better semantic to me. > > It=E2=80=99s certainly safer but will cause client mounts to fail as well= . It would be nicer to export an empty directory. A couple of comments here (mostly for everyone else reading this). 1 - From a security point of view, exports apply to server file systems, not directories. (They are stored in the kernel on file system mount points.) 2 - The "administrative controls" which allow mounts for only certain directories within a server file system is not a significant security tool. They can be easily circumvented and only work for NFSv3 and not NFSv4, since they only affect the NFSv3 sideband Mount protocol. 3 - The whole purpose of exports(5) is to make undesirable NFS client mounts fail. Personally, I wish these "administrative controls" did not exist. They were created way back in the mid-1980s so that BSD (CSRG) could be "Sun compatible". When I got involved in NFS on FreeBSD I tried to convince the "collective" to get rid of them, but there was pushback, due to that being a POLA violation. --> As such, they still exist. They still cause confusion w.r.t. what is exported and I, personally, recommend they be avoided when a exports(5) file is created in order to minimize the risk of exporting some file system that is undesirable from a security perspective. rick ps: Thanks for the comments. I am proceeding with (C). > > > rick > > > >> > >>> although making it that way > >>> after 30years might be construed as a POLA violation? > >>> > >>> So, what do others think I should do with this? > >>> (A) - Patch mountd to enforce the "must be a mount point when -alldir= s > >>> is specified, plus update exports(5) to state this semantic cle= arly. > >>> or > >>> (B) - Patch mountd so that it enforces "must be a mount point when -a= lldirs > >>> is specified, but only enabled via a new mountd command line op= tion. > >>> --> ie. Leave the default as not enforced, but allow enforcemen= t based > >>> on a new mountd option. > >>> - Document this in both exports(5) and mountd(8). > >>> or > >>> ??? > >> > >> (C) - Patch mountd so that it enforces "must be a mount point when -al= ldirs > >> is specified, but provide a new mountd command line option to re= store the old behaviour. > >> --> ie. Default as enforced, but allow an override based on a n= ew mountd option. > >> - Document this in both exports(5) and mountd(8). > >> > >> I think that (A) is too POLA-unfriendly. > >> > >>> Thanks in advance for your comments, rick > >>> > >> > >> -- > >> Bob Bishop > >> rb@gid.co.uk > >> > >> > >> > >> > > > > -- > Bob Bishop > rb@gid.co.uk > > > > From nobody Thu Nov 28 20:29:09 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XznwN65Blz5fDhF for ; Thu, 28 Nov 2024 20:29:24 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic304-25.consmr.mail.gq1.yahoo.com (sonic304-25.consmr.mail.gq1.yahoo.com [98.137.68.206]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XznwM2X1mz4ryN for ; Thu, 28 Nov 2024 20:29:23 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b="WT/rHaK1"; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.206 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732825761; bh=BzM/7JCOnR7iAW1KItZB4zzZ74yUhH29Je1jnF6OG3c=; h=From:Subject:Date:Cc:To:References:From:Subject:Reply-To; b=WT/rHaK1XHDD8O3k3h+zKqD3I7aE3XLT3HWUiU5X2Z8OYYxeCDQm5xkxKARUhDKL8NSMqcp0U/MNwH8qpRDr4fg5VGSAAN8OtggrYbZhgxKl2L0ZaWVUtjB7b39TCcrtl8JnSP03rM4VdiQ4vGQhTdy8w2NU5rra0z6vWZPWJFySVzTx+Kn2oEz5O2oroTpHyFnZ8m5K9T1eEDdfDUBEZhrQsKAGPEaFjOTKEnbblvgT9UeQoxr6kVFZvkP7sTGaDhelx4Z5a+EQpVdbcIepYkcKV0c+uir7Xo559Bu3VwK6+oe6uSXpeLagC1YTlpOTe0OybtwYjGfSlQnfDj4FqQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732825761; bh=uhMNetRXk6WGxyAixx3PW+Ukb2rhD6G9NUSt18v34W9=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=Z3EzVxvB/gD1Ho6HxGizUmmfBQPRkX4bUsq+WpyYqNYoe5rDaOPihTC5Wa4V4EwQVreG32Wn8qD+BcGDI5TCrMv/Q3hyKa3OQahXgMgVuw1W0LsHicTte5TbIo3QJ43SZzN6SB6iuFnSZ6RQOcflnXxk8Wn5oI6bYODziFyymdPKQbTKgeRLEueXeR/zEAwvE3bL69aRQCP6bhNY3PMnzbO3kAtEci78F0QV3rrNxm0Liu6PO9HyXYwYTN7vfSL4X2r93OkyJm9gooHLGI4glp8Y+2Fhd+y/aAkp/1LprtUazmdeNYSu00hfcHOY/ZKGECpVZy+NvcQpJBKLlRWycw== X-YMail-OSG: 0vEZIgYVM1nM.GYTpQpiymv2zlIkc49iYmh6TKC4Q6Zsufup5t3Yv9gmJulPnUa ST7sQUQueukQt3eVwcEPoQyjOJsb2WkFnvi0wGQpVzQt74kpq5meB4dAzp73aThoGlkAG9cEdnAv SNcyOcKyByW0BqVuCOrC5D0Uf2XRxzg99x02mjxHOCv5GAPCrYHzRWptQyQ8AOYYcBjpjvEuE2zS 5uPMJw2IjZBhZMQ8F2YPUgpiP0jyn5SX2cz1791zrVS7501YzNcMykvstnmP4_LTMa834TdA0NMG CUFdqyoec0L4TeIVXrHz_W.mY0BcAhNTlExkgewEFiNd0f1Bpd9NqzO3Mx.fmru_BdCRwaE1obwP WXJvP41VtFCRRP0YQTGjo5BzwdxjzETZHAyxcwWNeA23Xz77_70ELYHSazI0cT2AEqJqpqr8RhgR VoFI_A8JXgs7LZO8.0kw5GztzgKpY2JZBG_nwnbwPA7wzdOjBnMrRC2U2jWAjgdKQrOm6O27WADL 3RjlAD1wLwd5raX8.K6y.eFMJz89f2tJog0hAem_WMiMZPutmMDFmcAJ422oEt4RfKpLNzM9o9XN T6dj.WF_4qnAXHceOv5TBm.75uusnMUHs4btfHP9EV_Vfap7hqltQvFxYTa9UZdlpsg.DXfDnNk. buIvroDqfUrKnQh3_vpetET69Um0TeeVug8XkKNMfxnScv9Fy63iuINDFQbjHA1aQ3drfhvYeu9M 7bB.7ROfbzu4czrQk3MQZFcz06sOHJ.dQF31VjMjnlVIhCytZU_SSceuujjh6q1sNf2NQqrh6vuI IJ2BmL4NOOKetOFkSKn6tHp4txW5xBcKyxbuPlnd7qUjRfzkZq1WVGNCaLoU0PBO9Z9zy23px1R8 H91u3MaXyXAdgXtbJruT6o9nComhbtaefvjGLEbguQ2tev9IK7P.R_JjnR6Ht1gFucx7xOOEpYa0 ds85wqjSAhc9WX6bOpGjEmeDzivbimqWPa0jYVYSWEtPd6uCJ_rGVoq1U8DQKypZ_hyr0okP6f.3 ynJuoDgpLeeX6HjlWyvfzZI_QZO7op2fx1X3_WsXcbOVNIi.h5yyIX0m.49OJCWfYF3Ug3sZ7fzQ Bzj7rn3b9dch7o9tMyqWIchcJFflUOU.4IazDOeEv4pIsP9McEJYTnkwSwiV9IVC42hM_3Uunkj8 OxneeTx4qJXGjEc3Tk2qccJrlOjbuQgwR.ug0naeao2f6g22p4cilgG_uFX8rOQD5GQIE9tXElC9 z4.y6cYrZERYbxYpg3qPmkFcD5iuAz3MqprXufq_YMC4fI9pss6AEpzP2W3ulv07pfGzpZCsfn8F 77vbXhqF_9SpfbE_d7sgjDAuthY3Uh0m_H_FXXgCTdIVrK._PV5LAtW.d7UpAjywETk8zahnpHKt 3Bh2BQBfRI54Kzjg3RKxhR2XqCkSGITlc.6TUJFllG3OaVm4HHYeiYZY6GDWFNGNZEWUH5org_q4 X9ukid4uQShjzJG8ihEwJRikjmWumKJj3n9f95efs4jdKieQJObB9.ZveUqwwYCsAZzKLcv0Aui2 Shm6.p431ueIGNpCdx9McxLWGOZZw4kZ.Xenm88Z0cfEvMWZhClPcY2kn0LJTLPLWNw2.wLxUIXQ YgVLevaPc2u1eOllmzhatwyxdkBVEG8XobIySbjop9Y4ZyJiu6sQq5661SzFI1lS4.ftfJYtrq5e gyUvmD4dxfgNx9bHcZJQqk8u01DS0ITumeCfAkWzmUQVv.zBFhTLA_ksAL81B.Y8cC7FmBY36r8c ZKC7dH624i.h3XnvpM76FamzCU6f9oSs3r38fV0JpCRiCOy1jhY4x8No4EOarDv2ee5A.ZqopGRh XCo_DhiETOJYurT._UI9UXA.dyxfRgTkkrB2CXlvnUF44twT1ri0iyB6RMyvhd11CF0p3YZCIoyR zIu6s8tEi0.8mtHjlKiKROWV3O_jqRxOaMKHPZmIgM4ZKbOt_h55PNiKb_ANLx2qec0p_BZHy6VY K746bEYduGe0KRPZe3358jVtsBUgpyJxHVNd4Few83cGEfdsQMw1XvNOxJPltyVH0KzWJhTr53Gk 9HQDnsZoO4klD5WDWFZUt2AeAXiK.DtVsI4m4_6rbqJk27DjNGAjUJLfEjsyjTpZPhlXBsut68iU t8bf.lGeSBaCiQwz8vo48GD0R2Lk6e3U6J8m5pAlUmKv_GtgAjHRK9c8WR5B7djoYz3ud0giVXTs WaVPeK3l3.I3IbvE2thP4rUO_IPtEU3RSKQl8kIxT49vyfH1ehdG2yIzF.jCAlA9y X-Sonic-MF: X-Sonic-ID: 43578921-9275-41c9-8520-a7661edc6142 Received: from sonic.gate.mail.ne1.yahoo.com by sonic304.consmr.mail.gq1.yahoo.com with HTTP; Thu, 28 Nov 2024 20:29:21 +0000 Received: by hermes--production-gq1-5dd4b47f46-fhdpd (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 661dd44bf463b81a4e885253030570c6; Thu, 28 Nov 2024 20:29:20 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51.11.1\)) Subject: Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros] Message-Id: <4D541C32-7DF4-4CAC-B31E-D4DD17977154@yahoo.com> Date: Thu, 28 Nov 2024 12:29:09 -0800 Cc: FreeBSD Current , FreeBSD Mailing List To: "scf@freebsd.org" X-Mailer: Apple Mail (2.3776.700.51.11.1) References: <4D541C32-7DF4-4CAC-B31E-D4DD17977154.ref@yahoo.com> X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FREEMAIL_FROM(0.00)[yahoo.com]; DKIM_TRACE(0.00)[yahoo.com:+]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.206:from]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.206:from] X-Rspamd-Queue-Id: 4XznwM2X1mz4ryN X-Spamd-Bar: --- Sean C. Farley wrote on Date: Thu, 28 Nov 2024 18:16:16 UTC : > On Mon, 25 Nov 2024, Mark Millard wrote: >=20 > > On Nov 25, 2024, at 18:05, Mark Millard wrote: > > > >> Top posting going in a different direction that > >> established a way to control the behavior in my > >> context . . . > > > > For folks new to the discoveries: the context here > > is poudriere bulk builds, for USE_TMPFS=3Dall vs. > > USE_TMPFS=3Dno . My test context is amd64 on a > > 7950X3D system with 192 GiBytes of RAM. Others have > > other contexts, including an Intel system. >=20 > I have been seeing some odd behavior from Firefox as well as with=20 > poudriere builds on my system. Both of which are touching a tmpfs=20 > system as I have setup /tmp as tmpfs, which Firefox uses, and=20 > USE_TMPFS=3Dall. >=20 > The system has been an experiment, for me, with undervolting. I have=20= > been attributing any flakiness to the undervolting, but I have reduced=20= > that a lot while the instability has been consistent as in it has = stayed=20 > rare. I cannot tell how many times I have run memtest86 on this = system. >=20 > System setup: > - FreeBSD 14.2-STABLE The context that I investigated --and what was fixed by a commit only applies to-- main [so; 15 as stands], not stable/14 . stable/14 has no commits mentioning "tmpfs" after 2024-Jun-04. > - i7-14700K (latest BIOS which *should* fix Intel power-related bugs) > - 128 GiB RAM > - ZFS (mirrored drives) The primary test context was ZFS but no redundancy or such. (Only really used for bectl activity.) But testing on a UFS copy of the live directory tree also got the problem. The actual problem was in tmpfs support. > - 2 encrypted swap partitions (64 GiB each, lightly used) No encryption involved in my context at all. > - Lightly undervolted (-0.06 offset to Global Core SVID Voltage) Nothing analogous in my context. > - /tmp is tmpfs I have no default areas that are tmpfs: so only what poudriere temporarily created during the bulk builds. > - ${HOME}/.cache is tmpfs No use of ccache or the like. > - Poudriere: > - USE_TMPFS=3Dall I also use TMPFS_BLACKLIST . My personal environment causes use of -gline-tables-only as debug information normally. (That option is clang/clang++ specific. gcc* and clang* do not seem to have a common notation for analogous settings on the command line.) > - ccache No use of ccache or the like. > - jail version in sync with host True for my context. But the issue that was fixed was in the kernel code, not the world code. > - /usr/ports is mounted with nullfs Also true for my context. > I have wondered if it was swap-related, but recently I noticed a build=20= > failure with games/veloren-weekly where swap was available but zero=20 > bytes were used. The system was under little load at the time so less=20= > chance of undervolting being an issue. >=20 > Build failure: > ----------------------------- >=20 > portpicker =3D { path =3D = '/wrkdirs/usr/ports/games/veloren-weekly/work/portpicker-rs-df6b37872f3586= ac3b21d08b56c8ec7cd92fb172' } > =3D=3D=3D> Updating Cargo.lock > error: checksum for `windows_x86_64_msvc v0.42.2` changed between lock = files >=20 > this could be indicative of a few possible errors: >=20 > * the lock file is corrupt > * a replacement source in use (e.g., a mirror) returned a different = checksum > * the source itself may be corrupt in one way or another >=20 > unable to verify that `windows_x86_64_msvc v0.42.2` is the same as = when the lockfile was generated >=20 > *** Error code 101 >=20 > ----------------------------- >=20 > Restarting the build finished successfully. >=20 > >> I changed USE_TMPFS=3Dall to USE_TMPFS=3Dno : > >> > >> USE_TMPFS=3Dall gets the failure >=20 > *snip* >=20 > >> vs. > >> USE_TMPFS=3Dno works just fine > >> > >> So it is a FreeBSD system error associated with > >> use of tmpfs . > > > > Recent work on tmpfs includes: None of this is directly stable/14 : all main [so: 15 as stands]. stable/14 has no commits mentioning "tmpfs" after 2024-Jun-04. So none of these changes are involved for stable/14 . > > > > Mon, 09 Sep 2024 > > =C3=A2=E2=82=AC=C2=A2 git: 8fa5e0f21fd1 - main - tmpfs: Account for = whiteouts during rename/rmdir Jason A. Harmening > > Fri, 04 Oct 2024 > > =C3=A2=E2=82=AC=C2=A2 git: 75734c4360fc - main - tmpfs: check = residence in data_locked Doug Moore > > Sun, 13 Oct 2024 > > =C3=A2=E2=82=AC=C2=A2 git: ec22e705c266 - main - tmpfs: remove = duplicate flags check in tmpfs_rmdir Alan Somers > > Thu, 24 Oct 2024 > > =C3=A2=E2=82=AC=C2=A2 git: db08b0b04dec - main - tmpfs_vnops: move = swap work to swap_pager Doug Moore > > > > swap_pager (given the reference to it above): > > > > Tue, 08 Oct 2024 > > =C3=A2=E2=82=AC=C2=A2 git: d0b225d16418 - main - swap_pager: use = iterators in swp_pager_meta_build Doug Moore > > Fri, 11 Oct 2024 > > =C3=A2=E2=82=AC=C2=A2 git: 1107834090be - main - swap_pager: swapoff = detecting object death Doug Moore > > Thu, 24 Oct 2024 > > =C3=A2=E2=82=AC=C2=A2 git: 34951b0b9e78 - main - swap_pager: move = scan_all_shadowed, use iterators Doug Moore > > =C3=A2=E2=82=AC=C2=A2 git: 02e85d1c8a41 - main - swap_pager: fix = assert in seek_data Doug Moore > > =C3=A2=E2=82=AC=C2=A2 git: faa9356f97d2 - main - swap_pager: fix = seek_hole assert Doug Moore > > Sat, 26 Oct 2024 > > =C3=A2=E2=82=AC=C2=A2 git: 39f6d1e7f835 - main - swap_pager: iter in = haspage, lookup, getpages Doug Moore > > Wed, 13 Nov 2024 > > =C3=A2=E2=82=AC=C2=A2 git: d11d407aee48 - main - swap_pager: Ensure = that swapoff puts swapped-in pages in page queues Mark Johnston > > > > I do not know at this time when the corruptions started. The > > above is only suggestive. >=20 > Thank you for listing those. >=20 > I need to find some time to look over those changes although I am no=20= > kernel guru by a long shot. However, I see now that it looks like much=20= > more knowledgeable people are already looking on the current mailing=20= > list at the issue. None of them were applied to stable/14 . =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Fri Nov 29 00:55:09 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xzvq61R5vz5fXM8 for ; Fri, 29 Nov 2024 00:55:14 +0000 (UTC) (envelope-from dclarke@blastwave.org) Received: from mail.oetec.com (mail.oetec.com [108.160.241.186]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "mail.oetec.com", Issuer "E6" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xzvq56Zncz4G9Z; Fri, 29 Nov 2024 00:55:13 +0000 (UTC) (envelope-from dclarke@blastwave.org) Authentication-Results: mx1.freebsd.org; none Received: from [172.16.35.3] (pool-99-253-118-250.cpe.net.cable.rogers.com [99.253.118.250]) (authenticated bits=0) by mail.oetec.com (8.17.1/8.17.1) with ESMTPSA id 4AT0t9uR041837 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT); Thu, 28 Nov 2024 19:55:11 -0500 (EST) (envelope-from dclarke@blastwave.org) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=blastwave.org; s=default; t=1732841711; bh=cJ0btU00+c0NMU76aqS8my8jqsjoeZgb3BZ0HQ4ysGQ=; h=Date:Subject:To:Cc:References:From:In-Reply-To; b=PmSTi4USFV4kiIuswPpew6HGxM1uHL9QTZCzi7F3oOPWgLZfeGqG8E0shqsv+Uw/3 CJtaSzx1twJc0D8f7/VuGneHJIInDZFTT99jdH6h0oNex2rVhbZMMOkAewt8ZmYpZl umrwRRwlEmtEqto80T/ips5XROPLJ5Do9O+cFQLIaT7eIY7LVxu59YTpB7rbS13vhj 6BwwRgyD1uvcsKS8Cu6T63woMulLvxXJ2TirG2yKERx5O0GPKrHNm5Wf8B+bPI8v++ Z4O77x516tvWYWgooDepXj6YCR23obvj8RAsjm00X0Da5iHKWGLPfR4H3/8wRWcTX6 1Ld1X7WRDJY4g== Message-ID: <54808a60-a25c-4a26-915f-1ca69387db2a@blastwave.org> Date: Thu, 28 Nov 2024 19:55:09 -0500 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: zpools no longer exist after boot Content-Language: en-CA To: Alan Somers Cc: Current FreeBSD References: <5798b0db-bc73-476a-908a-dd1f071bfe43@blastwave.org> <22187e59-b6e9-4f2e-ba9b-f43944d1a37b@blastwave.org> <722a3644-3af6-4ff9-b1ee-022c32872001@blastwave.org> From: Dennis Clarke Organization: GENUNIX In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-oetec-MailScanner-Information: Please contact the ISP for more information X-oetec-MailScanner-ID: 4AT0t9uR041837 X-oetec-MailScanner: Found to be clean X-oetec-MailScanner-From: dclarke@blastwave.org X-Spam-Status: No X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:812, ipnet:108.160.240.0/20, country:CA] X-Rspamd-Queue-Id: 4Xzvq56Zncz4G9Z X-Spamd-Bar: ---- On 11/28/24 10:55, Alan Somers wrote: > On Thu, Nov 28, 2024, 9:47 AM Dennis Clarke wrote: > >> On 11/28/24 09:52, Alan Somers wrote: >>> On Thu, Nov 28, 2024, 8:45 AM Dennis Clarke >> wrote: >>> >> ... >>> >>> For "zpool import", the "-c" argument instructs zfs which cachefile to >>> search for importable pools. "-O", on the other hand, specifies how the >>> cachefile property should be set after the pool is imported. >>> >> >> I have to wonder what value there is in NOT having the cachefile >> property set in a zpool ? Certainly given that the zpool RC script >> really wants to check in a few places and then use those cache files. >> >> -- >> -- >> Dennis Clarke >> RISC-V/SPARC/PPC/ARM/CISC >> UNIX and Linux spoken >> > > Usually the default value for cachefile is sufficient. In fact, the only > time I've ever needed to set cachefile has been when I DON'T want the pool > to import at boot. I wonder if there was some other reason, since resolved, > why your pools didn't import the first time. And subsequently they didn't > import because you set the cachefile to a non default value? > I am at a loss. Really. Getting the iSCSI stuff working was a real treat and this just makes things even more strange. I really do not know. -- -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken From nobody Fri Nov 29 02:25:19 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xzxq91GMPz5fdNk for ; Fri, 29 Nov 2024 02:25:25 +0000 (UTC) (envelope-from dclarke@blastwave.org) Received: from mail.oetec.com (mail.oetec.com [108.160.241.186]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "mail.oetec.com", Issuer "E6" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xzxq80gc0z4MKP for ; Fri, 29 Nov 2024 02:25:24 +0000 (UTC) (envelope-from dclarke@blastwave.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=blastwave.org header.s=default header.b="IWW+hn/6"; spf=pass (mx1.freebsd.org: domain of dclarke@blastwave.org designates 108.160.241.186 as permitted sender) smtp.mailfrom=dclarke@blastwave.org; dmarc=pass (policy=quarantine) header.from=blastwave.org Received: from [172.16.35.3] (pool-99-253-118-250.cpe.net.cable.rogers.com [99.253.118.250]) (authenticated bits=0) by mail.oetec.com (8.17.1/8.17.1) with ESMTPSA id 4AT2PJER043121 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT) for ; Thu, 28 Nov 2024 21:25:21 -0500 (EST) (envelope-from dclarke@blastwave.org) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=blastwave.org; s=default; t=1732847121; bh=QOZxBgvcMcMfYl3R789P+/LFcpBkkquWLFdNq6fSSC0=; h=Date:To:From:Subject; b=IWW+hn/6k2Douw2dOhRQ6Pcu8tGPUSvg3fldB9YzGvbeOHsSWOmcW1TJWKErHajjg VaO1luPbBZEpQAoB1MlHfeansVYEJOXtsTxyuJ9iWYzKPyNe2ZGweHZZ6FeV0N5xBd o8M85DyLXt6tXDrFu/30FAklccTIw9rKISnEQCEJcoMiIjwyCGHqgHL3AEU1lQLOr1 DMEFm0VmBQnJzsXdNToL36ZiF3S15BvMuJhxXBjnLIqEUqmAd/os/RMVPIo7bXmA7/ PiGsMF7PIkulxLREsnnrn386fiy9XGjPMcKe/it8degButLIUNFOb9rexu8j7eDO1H Nh2RY5A8J0J2w== Message-ID: <64c81235-7871-4c44-bef2-95210f12bf86@blastwave.org> Date: Thu, 28 Nov 2024 21:25:19 -0500 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: Current FreeBSD Content-Language: en-CA From: Dennis Clarke Subject: top seems confused about memory ? Organization: GENUNIX Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-oetec-MailScanner-Information: Please contact the ISP for more information X-oetec-MailScanner-ID: 4AT2PJER043121 X-oetec-MailScanner: Found to be clean X-oetec-MailScanner-From: dclarke@blastwave.org X-Spam-Status: No X-Spamd-Result: default: False [-3.69 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[blastwave.org,quarantine]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; RCVD_IN_DNSWL_MED(-0.20)[108.160.241.186:from]; R_DKIM_ALLOW(-0.20)[blastwave.org:s=default]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; TO_DN_ALL(0.00)[]; HAS_ORG_HEADER(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[blastwave.org:+]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:812, ipnet:108.160.240.0/20, country:CA]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[] X-Rspamd-Queue-Id: 4Xzxq80gc0z4MKP X-Spamd-Bar: --- On a machine here I see top reports this with " top -CSITa -s 10" last pid: 6680; load averages: 0.29, 0.12, 0 up 0+11:40:46 02:23:01 51 processes: 2 running, 47 sleeping, 2 waiting CPU: 0.6% user, 0.0% nice, 0.2% system, 0.0% interrupt, 99.2% idle Mem: 587M Active, 480G Inact, 1332K Laundry, 7410M Wired, 456M Buf, 11G Free ARC: 3624M Total, 85M MFU, 3359M MRU, 32M Anon, 118M Header, 27M Other 2919M Compressed, 32G Uncompressed, 11.13:1 Ratio Swap: 32G Total, 32G Free THR USERNAME THR PRI NICE SIZE RES STATE C TIME CPU COMMAND 100003 root 40 187 ki31 0B 640K CPU0 0 464.8H 3967.69% [idle] 101142 root 1 48 0 1530M 574M piperd 34 0:27 24.69% /usr/lo 100000 root 731 -16 - 0B 11M parked 18 112:10 3.14% [kernel 102993 root 1 21 0 30M 15M select 26 0:03 2.77% /usr/bi Seems only 11G of memory is free ? That seems impossible. titan# sysctl hw.physmem hw.physmem: 549599244288 titan# titan# titan# sysctl -a | grep 'free' | grep 'mem' vm.uma.vmem.stats.frees: 0 vm.uma.vmem.keg.domain.1.free_slabs: 0 vm.uma.vmem.keg.domain.1.free_items: 0 vm.uma.vmem.keg.domain.0.free_slabs: 0 vm.uma.vmem.keg.domain.0.free_items: 0 vm.uma.vmem_btag.stats.frees: 523236 vm.uma.vmem_btag.keg.domain.1.free_slabs: 0 vm.uma.vmem_btag.keg.domain.1.free_items: 34398 vm.uma.vmem_btag.keg.domain.0.free_slabs: 0 vm.uma.vmem_btag.keg.domain.0.free_items: 34378 vm.kmem_map_free: 528152154112 kstat.zfs.misc.arcstats.memory_free_bytes: 11707904000 titan# I have no idea what "top" is reporting but 11G free on a machine doing nothing seems ... unlikely. -- -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken From nobody Fri Nov 29 03:15:54 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XzyxX1N3Tz5fhHQ for ; Fri, 29 Nov 2024 03:16:00 +0000 (UTC) (envelope-from dclarke@blastwave.org) Received: from mail.oetec.com (mail.oetec.com [108.160.241.186]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "mail.oetec.com", Issuer "E6" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XzyxV6TFQz4RbV for ; Fri, 29 Nov 2024 03:15:58 +0000 (UTC) (envelope-from dclarke@blastwave.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=blastwave.org header.s=default header.b=X3ou8J+m; spf=pass (mx1.freebsd.org: domain of dclarke@blastwave.org designates 108.160.241.186 as permitted sender) smtp.mailfrom=dclarke@blastwave.org; dmarc=pass (policy=quarantine) header.from=blastwave.org Received: from [172.16.35.3] (pool-99-253-118-250.cpe.net.cable.rogers.com [99.253.118.250]) (authenticated bits=0) by mail.oetec.com (8.17.1/8.17.1) with ESMTPSA id 4AT3Fsed043852 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT) for ; Thu, 28 Nov 2024 22:15:56 -0500 (EST) (envelope-from dclarke@blastwave.org) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=blastwave.org; s=default; t=1732850156; bh=EFsuZQr6D7wQWvIzLR5J8U3y+ZXZxt6MwbOSRHqVEeM=; h=Date:Subject:From:To:References:In-Reply-To; b=X3ou8J+m82YobMVda1gl4953U+K8h96BSw2hJI5j/SsqTINehLgmQUMnaK+EBYj0A XyFwVpI6X4GeELHamXIbR2jJTt5TBbJvzyIaRxB3u/mRHSwYdI6CTKRDZMk+hFPghw nXIHJ9wgxMKtVLQdQn67rTg4toyMqH4mNRu25L1pHR0Y4cx/Ta8D/+pUbHXsoOCVSL ffqA5YlGFv4Z1XkVeBRR38Hv/Dj4C4W4zaesHuRbbRX0AMesLLKCTqqovY77vbi9PQ lGSfsifhAYhjPpL0gMy3J/R4hFq6I7jJ2sSLqeXXOqRBV2Sid7S4JIfR48PZ/M12Vc HnWMGnNn0ZAKw== Message-ID: <24f1e839-d312-4ff0-b655-9a6bd185a0b8@blastwave.org> Date: Thu, 28 Nov 2024 22:15:54 -0500 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: top seems confused about memory ? Content-Language: en-CA From: Dennis Clarke To: Current FreeBSD References: <64c81235-7871-4c44-bef2-95210f12bf86@blastwave.org> Organization: GENUNIX In-Reply-To: <64c81235-7871-4c44-bef2-95210f12bf86@blastwave.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-oetec-MailScanner-Information: Please contact the ISP for more information X-oetec-MailScanner-ID: 4AT3Fsed043852 X-oetec-MailScanner: Found to be clean X-oetec-MailScanner-From: dclarke@blastwave.org X-Spam-Status: No X-Spamd-Result: default: False [-3.68 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.994]; DMARC_POLICY_ALLOW(-0.50)[blastwave.org,quarantine]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; RCVD_IN_DNSWL_MED(-0.20)[108.160.241.186:from]; R_DKIM_ALLOW(-0.20)[blastwave.org:s=default]; R_SPF_ALLOW(-0.20)[+mx:c]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; TO_DN_ALL(0.00)[]; HAS_ORG_HEADER(0.00)[]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[blastwave.org:+]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:812, ipnet:108.160.240.0/20, country:CA]; RCPT_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[] X-Rspamd-Queue-Id: 4XzyxV6TFQz4RbV X-Spamd-Bar: --- On 11/28/24 21:25, Dennis Clarke wrote: > > On a machine here I see top reports this with " top -CSITa -s 10" > > > last pid:  6680;  load averages:    0.29,    0.12,    0 up 0+11:40:46 > 02:23:01 > 51 processes:  2 running, 47 sleeping, 2 waiting > CPU:  0.6% user,  0.0% nice,  0.2% system,  0.0% interrupt, 99.2% idle > Mem: 587M Active, 480G Inact, 1332K Laundry, 7410M Wired, 456M Buf, 11G > Free > ARC: 3624M Total, 85M MFU, 3359M MRU, 32M Anon, 118M Header, 27M Other >      2919M Compressed, 32G Uncompressed, 11.13:1 Ratio > Swap: 32G Total, 32G Free > >    THR USERNAME    THR PRI NICE   SIZE    RES STATE    C   TIME     CPU > COMMAND > 100003 root         40 187 ki31     0B   640K CPU0     0 464.8H 3967.69% > [idle] > 101142 root          1  48    0  1530M   574M piperd  34   0:27  24.69% > /usr/lo > 100000 root        731 -16    -     0B    11M parked  18 112:10   3.14% > [kernel > 102993 root          1  21    0    30M    15M select  26   0:03   2.77% > /usr/bi > > Seems only 11G of memory is free ? > > That seems impossible. > > titan# sysctl hw.physmem > hw.physmem: 549599244288 > titan# > > titan# > titan# sysctl -a | grep 'free' | grep 'mem' > vm.uma.vmem.stats.frees: 0 > vm.uma.vmem.keg.domain.1.free_slabs: 0 > vm.uma.vmem.keg.domain.1.free_items: 0 > vm.uma.vmem.keg.domain.0.free_slabs: 0 > vm.uma.vmem.keg.domain.0.free_items: 0 > vm.uma.vmem_btag.stats.frees: 523236 > vm.uma.vmem_btag.keg.domain.1.free_slabs: 0 > vm.uma.vmem_btag.keg.domain.1.free_items: 34398 > vm.uma.vmem_btag.keg.domain.0.free_slabs: 0 > vm.uma.vmem_btag.keg.domain.0.free_items: 34378 > vm.kmem_map_free: 528152154112 > kstat.zfs.misc.arcstats.memory_free_bytes: 11707904000 > titan# > > I have no idea what "top" is reporting but 11G free on a machine doing > nothing seems ... unlikely. > > even worse ... under load it seems to make no sense at all : last pid: 98884; load averages: 32.01, 30.51, 25 up 0+12:33:20 03:15:35 172 processes: 34 running, 136 sleeping, 2 waiting CPU: 78.4% user, 0.0% nice, 1.8% system, 0.0% interrupt, 19.8% idle Mem: 7531M Active, 450G Inact, 9588K Laundry, 27G Wired, 456M Buf, 14G Free ARC: 17G Total, 7543M MFU, 4337M MRU, 37M Anon, 260M Header, 5005M Other 7207M Compressed, 24G Uncompressed, 3.39:1 Ratio Swap: 32G Total, 32G Free THR USERNAME THR PRI NICE SIZE RES STATE C TIME CPU COMMAND 100003 root 40 187 ki31 0B 640K RUN 0 486.9H 792.70% [idle] 103554 root 1 156 i0 786M 632M CPU37 37 0:44 99.82% /usr/bi 101148 root 1 156 i0 1317M 822M CPU2 2 2:20 99.82% /usr/bi -- -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken From nobody Fri Nov 29 03:54:32 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XzzpM16sSz5fkcD; Fri, 29 Nov 2024 03:54:51 +0000 (UTC) (envelope-from scf@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XzzpM0Xdgz4VLD; Fri, 29 Nov 2024 03:54:51 +0000 (UTC) (envelope-from scf@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732852491; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=cPlqcsZueNMhoN/nGCdf9Pq46pvKs+8i1OH/HOMbZPs=; b=Bgjs0rF/0RzlWdazDlPeTbhgNZqyQojm82EWzvNcmMFwoylrHO9vC0hGGh//01r2HHEQzR fa6LletW82IIdUSKFi8CNyhpd7C4RyGFJ1Qgq9hvjmD1PB1R9ln+gqYf6tnE+JhL7OK637 TS+5ZWqE7MzpOJ+leTxsB/2LQoAWrUW3q9LJz+vBeufoD7N2mJkd5JDVi7x0JSn2hwSUA9 JOJEUWW4oHo7aSaSk/I63Sacedeno9FIl9zG3Ca7yqL4HeIva7++mmN476N9FiORJ45z8X bjTBxJYZPJWzhVl9j5OathhZSUkCbJMXqukXuawuMX1dcUFymqYQ3jEiYWHUnQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732852491; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=cPlqcsZueNMhoN/nGCdf9Pq46pvKs+8i1OH/HOMbZPs=; b=LnlHnpUzkqeK15+66Pv93WUx8JpACYna+CuuRZfWNLRD96YIq+0t2GkxWAY0n6KvlQf145 cc5DqotrmOLuCM8r4tKRZyX5lxNlN5/XtUo2C2T17/GijtD6oolXzPLpK/9h5Nc+yMvc2U a/LMwdUHft7S4vPSmx6GLnNb+lW902RkRbyctjyq3FAkvtBiNCOglhgIcaCyFNRXC6sCmt N07CN0CO3i91h0vmQG9+JuDN0sqIHKlLtsA7J8xiDLcg8T51Y0l/chSWT5tGjg0d/MKUuq GP0AOH7t1wQnuhFnQTW64ZdDFN3NHOZ+O+xp+ucv7i151mkky3GWs7FrnC/EsQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1732852491; a=rsa-sha256; cv=none; b=kTultyJsZk71OU0/NTDtbnbS8DwWR1/z/P7JJp9EU8H3ayY6V4ngKlNMcyjts+3Vyrdj7Y JPj9TwUc+Nd1GnaEpVGWdY3/+NhowUh4wuPnhcrIBsHXB+P6Q+y3qItBMEWDx6FihDFbyW 3srOYUQZAzB++27O7TGHjuVCzQ360ILt3+xm86eTqvWLEKL4FwMXwauOgh05qlXeBTmstX gkL4l7Qe6MTx4vd9UVK3sZj+Ng9jxZYwnDrdI44pcEi0zUYlq0lZrAIENZXgaNDQrgsr/l zVbDyqOHeoDsG9MkU/CupdagvjQRxfJsWJ5xlLuUS2r82hofCzxWgAiJGz1rig== Received: from thor.farley.org (1609341-v107.1360-static.crmlinaa.metronetinc.net [104.254.222.35]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: scf/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4XzzpL5mPMz1Hr3; Fri, 29 Nov 2024 03:54:50 +0000 (UTC) (envelope-from scf@FreeBSD.org) Date: Thu, 28 Nov 2024 22:54:32 -0500 (EST) From: "Sean C. Farley" To: Mark Millard cc: FreeBSD Current , FreeBSD Mailing List Subject: Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros] In-Reply-To: <4D541C32-7DF4-4CAC-B31E-D4DD17977154@yahoo.com> Message-ID: <7f4080f5-a371-9940-480b-df13956c76ab@FreeBSD.org> References: <4D541C32-7DF4-4CAC-B31E-D4DD17977154.ref@yahoo.com> <4D541C32-7DF4-4CAC-B31E-D4DD17977154@yahoo.com> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed On Thu, 28 Nov 2024, Mark Millard wrote: > Sean C. Farley wrote on > Date: Thu, 28 Nov 2024 18:16:16 UTC : > >> On Mon, 25 Nov 2024, Mark Millard wrote: >> >>> On Nov 25, 2024, at 18:05, Mark Millard wrote: >>> >>>> Top posting going in a different direction that >>>> established a way to control the behavior in my >>>> context . . . >>> >>> For folks new to the discoveries: the context here >>> is poudriere bulk builds, for USE_TMPFS=all vs. >>> USE_TMPFS=no . My test context is amd64 on a >>> 7950X3D system with 192 GiBytes of RAM. Others have >>> other contexts, including an Intel system. *snip* >> System setup: >> - FreeBSD 14.2-STABLE > > The context that I investigated --and what was fixed by a commit only > applies to-- main [so; 15 as stands], not stable/14 . > > stable/14 has no commits mentioning "tmpfs" after 2024-Jun-04. Thank you. That was my mistake. I will continue searching for an answer. Once I find a way to more consistently trigger it, it will be much easier. I ran all of the tmpfs*.sh tests from HEAD which all pass except for tmpfs24.sh. $ ./all.sh -o tmpfs24.sh 20241128 22:33:38 all: tmpfs24.sh Min hole size is 4096, file size is 524288000. data #1 @ 0, size=4096) hole #2 @ 4096, size=4096 data #3 @ 8192, size=4096) hole #4 @ 12288, size=4096 data #5 @ 16384, size=4096) hole #6 @ 20480, size=524267520 --- /tmp/tmpfs24.exp 2024-11-28 22:33:40.222199000 -0500 +++ /tmp/tmpfs24.log 2024-11-28 22:33:40.225048000 -0500 @@ -5,4 +5,3 @@ hole #4 @ 12288, size=4096 data #5 @ 16384, size=4096) hole #6 @ 20480, size=524267520 -<> FAIL tmpfs24.sh exit code 1 >> - i7-14700K (latest BIOS which *should* fix Intel power-related bugs) >> - 128 GiB RAM >> - ZFS (mirrored drives) > > The primary test context was ZFS but no redundancy or such. (Only > really used for bectl activity.) But testing on a UFS copy of > the live directory tree also got the problem. The actual problem > was in tmpfs support. That was what I thought from what I read, but I wanted to make sure I did not leave out an important detail. >> - 2 encrypted swap partitions (64 GiB each, lightly used) > > No encryption involved in my context at all. > >> - Lightly undervolted (-0.06 offset to Global Core SVID Voltage) > > Nothing analogous in my context. > >> - /tmp is tmpfs > > I have no default areas that are tmpfs: so only what > poudriere temporarily created during the bulk builds. > >> - ${HOME}/.cache is tmpfs > > No use of ccache or the like. > >> - Poudriere: >> - USE_TMPFS=all > > I also use TMPFS_BLACKLIST . > > My personal environment causes use of -gline-tables-only as > debug information normally. (That option is clang/clang++ > specific. gcc* and clang* do not seem to have a common > notation for analogous settings on the command line.) > >> - ccache > > No use of ccache or the like. > >> - jail version in sync with host > > True for my context. But the issue that was fixed was > in the kernel code, not the world code. > >> - /usr/ports is mounted with nullfs > > Also true for my context. I appreciate that information. *snip build failure* > None of this is directly stable/14 : all main > [so: 15 as stands]. > > stable/14 has no commits mentioning "tmpfs" after 2024-Jun-04. So > none of these changes are involved for stable/14 . It was a long shot on my part anyway. :) Sean -- scf@FreeBSD.org From nobody Fri Nov 29 03:56:36 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xzzrl3tg0z5fkk0 for ; Fri, 29 Nov 2024 03:56:55 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic304-23.consmr.mail.gq1.yahoo.com (sonic304-23.consmr.mail.gq1.yahoo.com [98.137.68.204]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xzzrj4FVgz4XZP for ; Fri, 29 Nov 2024 03:56:53 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b="Q2gSbn/Z"; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.204 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732852611; bh=a87JnikN9b+v/1PGfKtiUpohQZ9Kq1NNWxUsSOFC7pU=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=Q2gSbn/ZRaMa4T3br6xW9iN0XRBo71/mH9elWs38nZ4HGJ32u3ZLS9snt40KK77PkHnO6awlUDI943jLIC4AZoLsaSWsWz+R1nLV91yUszx945ljNlGEd9//nL6ubAmrQ1ifh7dHnfgUrExX3J/ioolZgcdjBHNQE2qjg0dNaTKjJVKDGRBmYriykMJ6k06Fq2C/egvAxxJEFgLL0TZ8XVdT8h3KEuNPcUSXWH4cEA7Xpp/cNzDP0I/Ti/h7dqZvpD74f3Eaek3bVdFubr0Oho9C3W/CqFKzShQIJe6a+I6dABGXNjK6kRzkuziUwpv2NNjPBE9tG6VvHDoiAJ8ylw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732852611; bh=5xfDeIu0JJZvQlNRsyX8sdpP/0zkwwlIW65xRr2oU2r=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=ha9CZRT2rxfojJrej5vnDxm0XQMfjq/m6L1gZAabyY2R+1j9/ZEWeZ1dhklxMadF36AYw9r3RxmJBKs95lv5Rd83q2qF3vArDVIHuZAq97vqafrkpMdtl3ahEnrLjdtfYI1gOWtG8B9kqw1unevlFRYMXBtbNm++JML954RKcVXj/RwElCf5CnfBIsh89CobRfK3BT73ckZk63Z4EAp2os9MLEODm0mLB8hA25dzrb3BOmvrkZ87S1LHBuwTAVFXOq11QOX9XJACFVQxug8rWl87vQ8WgI71E3J/67n4Fnhy+bnpn5TW91UyiVTtKCFqbcd4tSGtEhIrNA1tBKo92A== X-YMail-OSG: 6Qa9tnoVM1mZ.7RcrB3SDdXV8UyVxsIKqQk3kffUtiIv09blLyZ0aGzhWfBGRu2 m3iRMB4FjIb6g9kZfpQ0U9bSbT_iJz30CXvmwmzgKBI2IuxjZAS_uPXw_8uzSNF1RmjQRbkqwvhN OzaOP2tGzvBh_TYHwLuOpp6gDtvPTmItlHMCeafKdRcHq7IUGdIlTw1x2CMf1hASFIjJPMORzVSJ .kk_9WJmLehTEs1rmwGi2iU.GzeRt1sENg4Y1M43xff8ozOOzdXMdbq1IYXxSqIEwpUUYLYnlKAh .Sle9LkvOwUuNsh9CN3GbOLba1jhleWvNamHsEODwlmDOJWcFNX24cSlSvYvmasR0FT92bQ5zKDM zsRrB2MWlQZSU6X0U5R1Odn3FWS8e9NH1cHH5bNALhrShmAhvdfvohvdDM1hZ1VpY4EbL8nas_G8 oeQItnu5ibap0sAL0nNqsGU1JTv473Yww36Kzx4gc3Yr6ARNKt_WPSqGw7vIDlMlkzxtgoVZCDMD MelknqOfpaiLfShY2ln.a49tww3d.NHDnTPivfdkbskBiNDWWzvseO3sa43hGBR5IwWSQqWxsWgb k6Fg9IKI8Y2mDpw5BjUNZGGZXmyNIUDBrlrXZIg7bjBvbj0sP7rychGXHHILUGTrHlK1v1MHwsNW 2wNhkzszZHB2oHlhT4x.b8_qp1DA_BaMdT2SSAVzAyj_HQO3w2XOxRyHllT_88gcDTLFeGSTskY4 78iE16oZdAh08QvEIGSrI9dk1.5_Iisj_7yL2QqCLzeYwIT00YxQIBQUQu_nLcvAxmFGJorfcFG7 mpPXpWb9f47EYh8wdQRCPzzxtdTgopPcEj.C9puLll58fe4Y_f4mrCWctxvHJFGmDtBNskl_GD5Z rIND1GNcJYTi36KKC_gvwk_L2cUth_z.a3rRSFtjgyuY03y1M4my.END4rBH2U1NpYiJPKsSa3T3 _lw3Gl_WVN2n6p2fWwVa6Cf6hmz7SWe9.NL24HSR0HEqZpljZNp73lSfp69O__gXYhbzbFlYIwxN 65yTNvcsE3uEE.gShCIHG5i60y9hCjhS5MoFlHR9MQNXqYLn2K2dxjmKAQq78QipvAprBu8H.jQl zwK7mtxRufCT6mzL23.9ky1VOLzXw3EO2.mSJu1aVEWvZl5wG3DAcv4CNao4g67BMC0VcxsQLRWk _unExin_lbncyJ7f1QI6mBShPw96C1Buqz7s64ff1cg3iN8Yp7ggBoXHslBP4qyJucQ6F6EZMeCF ZXKtpUhTmbIqO27_tnE3.L5JTi9AbFXsskt0gqSCRdNl_nTv.ieF9tPUhQTj79EGpxemF_NTijaH dOSY.V8RxFNVjtq7upoB48BaswUfywU_Z45qxJkCqAZUQxcoi5ULw.3xi2_2RzqaVElJVv7vU_Sz JsT9x.5zVqeraHv.CYbHETDBP.nIyIu5v82aEL3gbJTiyYzAQAp9Z0dPZaua.e5Wqfjk1X_W872s rI1A6a2yizIKi7u7NFTK72C_Y9P2C8UgWUy6fAeW4JEX1hVZRZb_xscQRpkKBip6qVC11v05nSDK 9q1zXm9ZqTiojQZLtBAwNeqX3iQdBXbENMj8M_0g5HSLFPkNF0UjLZ77dtbQUI1T4VrgXhAsiPoE TJpY7.48nZFb2Tf7B5wPKWKmDTHit9J4t9RgIyOtobrUlTmrFp_EVwTV0b_wfYitH4NN.H7nqzUc md2JkPbYzVlwlvsbIKnCzOY_ncQnzOiy.H.yyafx5GqzzKXv1G0cDYizp09ESmsEr3qEezF6NS4u LXcFwOGppjFwwFPQFMJDYBDTxcD53AhG4zd8uiTzMWIcyG1tX3SnzzL6vcGj3EBfzr2cW2MDV8Ea jHwqi6Nstszg9mf9OmdoS8g_27UXuM_FCfTR8Jau.DBRoi6KUKeAzhGCFfaKE6FbQkfqrktOsMdb zDCgb7noDeUp.FFooPceA1SESR.2fm.5FHaav_kd8U1zbZqKOE85YhjFKpw797C6nz68889oSDPH 0TNeIvfYoIm8p_vrDqaV5aco3Q6NVGRjWEUgA_gumk92FEsjt4WycXEYG3CnZsnOYbreVTdJVhLZ gEQI6O2MEShUiD.Yu4.9BV7iJVb.Gg4tiycq0tjArbRifNLnDVPLajpZ7jQVanorbdp.4UkUM3cG ubhgs1oVZdbBTmP.Z.g5d1bVwleKPylJkro3Q4ZUDlhVePoNthhZyPS4V2F6KThwrq6X_CIORBBx U5CdNrgLvhVw9VDxNhRTeMgTbceWdBZPNjCYF2b_6RGU9QeSc6kxsX6wb9F4Q2eaW__Q89__n04T shiYzxznE X-Sonic-MF: X-Sonic-ID: 7cf93184-dda8-4169-858b-9dd128cefeff Received: from sonic.gate.mail.ne1.yahoo.com by sonic304.consmr.mail.gq1.yahoo.com with HTTP; Fri, 29 Nov 2024 03:56:51 +0000 Received: by hermes--production-gq1-5dd4b47f46-fhdpd (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 5ab48aa000d3ca84cfd471655f48f5e8; Fri, 29 Nov 2024 03:56:47 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51.11.1\)) Subject: Re: top seems confused about memory ? Message-Id: <8B5B9B8F-2B69-45BB-B711-8A906FD9BE02@yahoo.com> Date: Thu, 28 Nov 2024 19:56:36 -0800 To: Dennis Clarke , FreeBSD Current X-Mailer: Apple Mail (2.3776.700.51.11.1) References: <8B5B9B8F-2B69-45BB-B711-8A906FD9BE02.ref@yahoo.com> X-Spamd-Result: default: False [-2.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_SHORT(-1.00)[-0.997]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MIME_TRACE(0.00)[0:+]; FREEMAIL_FROM(0.00)[yahoo.com]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.204:from]; MID_RHS_MATCH_FROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.204:from] X-Rspamd-Queue-Id: 4Xzzrj4FVgz4XZP X-Spamd-Bar: -- Dennis Clarke wrote on Date: Fri, 29 Nov 2024 03:15:54 UTC : > On 11/28/24 21:25, Dennis Clarke wrote: > > > > On a machine here I see top reports this with " top -CSITa -s 10" > > > > > > last pid: 6680; load averages: 0.29, 0.12, 0 up 0+11:40:46 > > 02:23:01 > > 51 processes: 2 running, 47 sleeping, 2 waiting > > CPU: 0.6% user, 0.0% nice, 0.2% system, 0.0% interrupt, 99.2% idle > > Mem: 587M Active, 480G Inact, 1332K Laundry, 7410M Wired, 456M Buf, 11G > > Free Notice the "480G Inact" (inactive): that tells me that prior activity has caused some mix of clean and dirty pages to be loaded into RAM. The context has no memory pressures to cause freeing any clean pages or to page out any dirty pages to swap. It matters not that the machine became idle after loading all those pages. > > ARC: 3624M Total, 85M MFU, 3359M MRU, 32M Anon, 118M Header, 27M Other > > 2919M Compressed, 32G Uncompressed, 11.13:1 Ratio > > Swap: 32G Total, 32G Free > > > > THR USERNAME THR PRI NICE SIZE RES STATE C TIME CPU > > COMMAND > > 100003 root 40 187 ki31 0B 640K CPU0 0 464.8H 3967.69% > > [idle] > > 101142 root 1 48 0 1530M 574M piperd 34 0:27 24.69% > > /usr/lo > > 100000 root 731 -16 - 0B 11M parked 18 112:10 3.14% > > [kernel > > 102993 root 1 21 0 30M 15M select 26 0:03 2.77% > > /usr/bi > > > > Seems only 11G of memory is free ? > > > > That seems impossible. > > > > titan# sysctl hw.physmem > > hw.physmem: 549599244288 > > titan# > > > > titan# > > titan# sysctl -a | grep 'free' | grep 'mem' > > vm.uma.vmem.stats.frees: 0 > > vm.uma.vmem.keg.domain.1.free_slabs: 0 > > vm.uma.vmem.keg.domain.1.free_items: 0 > > vm.uma.vmem.keg.domain.0.free_slabs: 0 > > vm.uma.vmem.keg.domain.0.free_items: 0 > > vm.uma.vmem_btag.stats.frees: 523236 > > vm.uma.vmem_btag.keg.domain.1.free_slabs: 0 > > vm.uma.vmem_btag.keg.domain.1.free_items: 34398 > > vm.uma.vmem_btag.keg.domain.0.free_slabs: 0 > > vm.uma.vmem_btag.keg.domain.0.free_items: 34378 > > vm.kmem_map_free: 528152154112 > > kstat.zfs.misc.arcstats.memory_free_bytes: 11707904000 > > titan# > > > > I have no idea what "top" is reporting but 11G free on a machine doing > > nothing seems ... unlikely. Seems perfectly normal to me, presuming prior activity caused the 480G of Inact. > > > > > > even worse ... under load it seems to make no sense at all : > > > last pid: 98884; load averages: 32.01, 30.51, 25 up 0+12:33:20 > 03:15:35 > 172 processes: 34 running, 136 sleeping, 2 waiting > CPU: 78.4% user, 0.0% nice, 1.8% system, 0.0% interrupt, 19.8% idle > Mem: 7531M Active, 450G Inact, 9588K Laundry, 27G Wired, 456M Buf, 14G Free Notice the 450G of Inact: smaller by roughly 30G. (Still no swap use.) It make comparison easier . . . Mem: 587M Active, 480G Inact, 1332K Laundry, 7410M Wired, 456M Buf, 11G Free Mem: 7531M Active, 450G Inact, 9588K Laundry, 27G Wired, 456M Buf, 14G Free Mem increased by around 6.8G or so. Wired increased by around 19.?G or so. Free increased by around 3.?G or so. That looks to be the majority of the around 30G. Some Inact clean pages may have been freed that lead to the increase in Free pages. But there seems to not have been enough memory pressured to lead to more clean out of Inact. The system is biased to keep around information in RAM that it might be able to put to use --unless there is sufficient competing activity for RAM use (memory pressure). As for Wired, ARC is stored in Wired and . . . ARC: 3624M Total ARC: 17G Total ARC increased by around 13.?G, making up much of the 19.?G increase in Wired. > ARC: 17G Total, 7543M MFU, 4337M MRU, 37M Anon, 260M Header, 5005M Other > 7207M Compressed, 24G Uncompressed, 3.39:1 Ratio > Swap: 32G Total, 32G Free > > THR USERNAME THR PRI NICE SIZE RES STATE C TIME CPU > COMMAND > 100003 root 40 187 ki31 0B 640K RUN 0 486.9H 792.70% > [idle] > 103554 root 1 156 i0 786M 632M CPU37 37 0:44 99.82% > /usr/bi > 101148 root 1 156 i0 1317M 822M CPU2 2 2:20 99.82% > /usr/bi I do not understand what about the above indicates any problem. May be more context about the prior activity that lead to the above needs to be reported? === Mark Millard marklmi at yahoo.com From nobody Fri Nov 29 04:07:09 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Y004s6B5Tz5fl85 for ; Fri, 29 Nov 2024 04:07:25 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-21.consmr.mail.gq1.yahoo.com (sonic317-21.consmr.mail.gq1.yahoo.com [98.137.66.147]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Y004s3kFPz4Zjy for ; Fri, 29 Nov 2024 04:07:25 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732853244; bh=+LuyE+fOQ9Qk5XkEpMUDfcJ/2P5ziGvNBTD8q0RYiHk=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=ULXixp5TG6N3780y/tZ20aInCIbnXf5nSp8QwvTy/+U0crwU2M6r+viXMx2lXMNCddOWhAjHgm6w0GeFHydh45rt9R3n9b/mZX/8bnTLBjxNUGHzH4fR9Xxj1dJZdwMtLFE454G484e1XHwCdApMwkI3KkYDMSL+PVwo0tgXv6aSK1baSjdrjER1BaemdaTbVFPzbVykqOycByH6+qhjnpuv4duZi8qn9pmtLc2f6PXK/hSywDE0lIKRVif7pSWXbmUn/+8q0mkpGR+1cZZVwJIrkTNgmaioWKj2DTRZUSBmGSpsvv1U+aL3WIKLKnjkXuxziQpQj3q3geSXv1qlQg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732853244; bh=F9CRKHdwzMsj/x+m6k82TJPdoI9Ue+8Bx5llxOqAzt2=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=NT9IdI28qrcjNH0L+vij0+vtPnfN/LyUWsUog9D8Oc8kSqMOfZUDZLD4dG/2qf6lM+nzF9qIMEzY4n0WhGNrqcVJEwQreHb4siiXYG25Usu7yrB/PUVLhC+j75FH0zNcfuZBTcJZOued0QpA/5cIku49Qu3ZBlRp1sq7Qf/EQNNEXHVkSTUulPP1qsN+O3CHPYvqCJm2XXSWQHrAjUneKfsM9TlKmxrUgbJLeNPTMKkksVPj9sYa5syWwBR8DqsP0LWEznhxkhas0bLs4XAuXPT2xXF/dxQDNaTQ7EAAk/1lYkAehzfYtnKWhjmx0lFdagj6yIcK/bBkCRc83Ml/9A== X-YMail-OSG: 83C0Yy0VM1kSpzskDueh9eIMIO1yJ_ZDJt92fMgOOYVW8OjgKTgvqS61Kuwt3ID n6wU8ZtLE002NFhTCS10IEZyCBrZ0pQ38Nw529vmkxuAc0vaCPpv2YcTGHUM7UfGi7.Dd7hpq608 LeOccRAIjjNVBUDgudwVpSUf_eW9OWhUly7l995d_X_kiWVe6hPYXZ5_GuAlq5Eo9HCXd7sCCpBj dl8LOBgolqQoiy9n7Z0C79aH4iXeZENE86bFEeqLDPEZ8mqXpKGp6_.dse7Rg6ThUoo7_WRNPRyn ShlcdAeOI85TRM1SoY8J2OH5_NwYhetdmWAOJrHL9dkbkKHDVt3LV.a_FuB6DuZSivC8a5aDjb5I Y.XNOmhdfQRWQLiCM9v8rnxj.L4IFaVUHvSvIK6.2PvjE9yLi5jgbegHt_0keooC0bwAShs779Uy 7JIP2dIFl1tSmG0sKlpWmsg1LBRuDNua3YPwdoF2b_IuKszcOGPo9IUUxfpK5XAgAqs5OgsG4FF9 JVvUtgHvdwap1vA63YNyZ6yaFJ13haNX0O65QAgGOZgfH_ZHKbSIigvJaIEVDgo7rns0vfGKHTZc lBS8cWsGx5v6mgHTlkkBE0rb195InBi7.15suEoeG462_V8yEUYdQIFTlu28Gb8.AimokMfA0aNU iDW5Md0ilduO5kkMOva5eSf91pgWOGLglVu7UxvzlAICSSHf_m64xi7dLIKgAd6hrxZrjTcuZJYA JnHpi87nbX74GSd7YiRvS8LZHm7F9LU3VI_rf7wy09q1trpQkV8sSKMymFBnLSB.yawEb.V2647D nlsPK4WMBO32PF9PKgY7lsBYaBoVzmacPDjxlub8KwN48ZktUXgZ_WjGFuwNvHYFiwPrIAXD9fWn 0t_eQS3hjLUVv0peoDcU92a_XT.odU4GJymrT0NTfreP6L3TcWWGpGv5bTwBRIV5H2L1ZyH2bEqG H1HBA9cwUtXCmmiuAOjzN3.cRGoZnbaw_bO11tss6Uf2rw01wS7h2DH_4II0vvr7SUU_FoxbHv7s 4Rq5DyOOsVv2MNezaY_CoMXVBhMEGIIDw4GNkeO92atRXIvkPAa2wPcWAT8sYZCF6JLbgbJuJ.Kv zzpb6mGnK1IvOJr.E_ypELb7CHNh.De74GMtWguw00lYhDgSd6_hd.d0rsUT5yVu4JB4QWyp4Ed8 e002F5hjWJp.g1E_ZUNffnDOHrzJePJvhbrXygfa8axE5lM6bf7ZIsk4gW_IwUdaVrDVovhG0wlx rtz3_1dsxnjXctNJwb6VU4MR60fTw9pVkOXYF1Nic5kEIkQOMaVAQJIeRBi7n7Wy89I4.0cuEfl. pclFEehh9e9s643yACbeH9RLa6yekQTjH6TlTV3KKLaEo1zfuU_obTx8iCyH8xWgWmOHeLscKd_W N7tKAoNcZNvjtJ1EapPb287UEtId_hONc4kHGP8ltu7NJIXWbj.akuyKLL2MmKD4BbJaWTQQcOsr c7GoxGQ0HdZ2EunW_TZR5l2EZPeFwQvVBXREOGlS2DdHw5LzcaDRr_awsxVIk1O5.32ZmCpjYrWy 0juDCvnMrqT9yjdk1Dmd5e0OLWBkNl1AhAH3JEpgk.Apgtz3f2egh1Gu1z_Bb7Tth.osu_EAgeal MmIBhXZq8ByrLwwxNOQuK26p86eKDTR6HYe1fIV6iaIQHvTaO87ikKAZx5nLJSEIlbNrmw1q.LMq oWS.n7QuliDT3.VS3hKmvWwVILpco7vKrCu7u19Fx0W0vGTClJFWSKiw0R9a265jp62JJNcHMqA5 r.DT3wzFNZJwRl2P2pgK0oNJz6emPtFsxMWTVBCwfpRVMgItTYKktB9ZB.F3jgAmQ9SRJCQlWU8h wLZcJLpY1A3OFsO35xmutnY_zyCiOEfEtJZSZfpaqs7SyStiuoQNh0xF5smAJCTUW2z23rnuWw5Q P_ksRi55Ms7zxl5PWFAY7qsSf7GvtR.VnO6g3zgN6RjX6Sn3g_AsTOjZVnBhpY4WLiYtlUClNv.3 MeDExpho11viFav2qfYhjBDDRsdIwI59h.HKRISbri4ISLa51bhNizdc5f5JsHSN69ONMRfJPui2 _lLxAwtnvLiDecYHetNPXarP1yW0nZzsHbn6YVfqutNPr5Jo1A7lUHe8rqAIWutSh0BRoCrEmojc FJPEFDxGXSBgHIiS2rs7hdIrtBkELsA614GgJ6IbaxQy3w5EjHCv9guC5rthLreao1gQOEgVd7v6 Th0A2TOXBYrJuHuRwEjFy3kZSsNtkJCE__Es2t5j5yGAx.xEJ2t_uIHJ3GBRvHYsLAGbmncgU1AN swYnw.XW6 X-Sonic-MF: X-Sonic-ID: c222138a-501f-488f-977a-93feb9f7474f Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.gq1.yahoo.com with HTTP; Fri, 29 Nov 2024 04:07:24 +0000 Received: by hermes--production-gq1-5dd4b47f46-5kxd4 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID ab6d3a6d69d754e2c8e415c7960c67b7; Fri, 29 Nov 2024 04:07:20 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51.11.1\)) Subject: Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros] From: Mark Millard In-Reply-To: <7f4080f5-a371-9940-480b-df13956c76ab@FreeBSD.org> Date: Thu, 28 Nov 2024 20:07:09 -0800 Cc: FreeBSD Current , FreeBSD Mailing List Content-Transfer-Encoding: quoted-printable Message-Id: <3AC4C67F-2DA3-4AA0-9B69-066146A5771E@yahoo.com> References: <4D541C32-7DF4-4CAC-B31E-D4DD17977154.ref@yahoo.com> <4D541C32-7DF4-4CAC-B31E-D4DD17977154@yahoo.com> <7f4080f5-a371-9940-480b-df13956c76ab@FreeBSD.org> To: "Sean C. Farley" X-Mailer: Apple Mail (2.3776.700.51.11.1) X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4Y004s3kFPz4Zjy X-Spamd-Bar: ---- On Nov 28, 2024, at 19:54, Sean C. Farley wrote: > On Thu, 28 Nov 2024, Mark Millard wrote: >=20 >> . . . >=20 >>> System setup: >>> - FreeBSD 14.2-STABLE >>=20 >> The context that I investigated --and what was fixed by a commit only >> applies to-- main [so; 15 as stands], not stable/14 . >>=20 >> stable/14 has no commits mentioning "tmpfs" after 2024-Jun-04. >=20 > Thank you. That was my mistake. I will continue searching for an = answer. Once I find a way to more consistently trigger it, it will be = much easier. >=20 > I ran all of the tmpfs*.sh tests from HEAD which all pass except for = tmpfs24.sh. Did you notice: Thu, 28 Nov 2024 . . . =E2=80=A2 git: 2e2699c48a7e - main - stress2: Added new tmpfs test = scenarios Peter Holm that added: stress2: Added new tmpfs test scenarios --- tools/test/stress2/misc/tmpfs26.sh | 179 = +++++++++++++++++++++++++++++++++++++ tools/test/stress2/misc/tmpfs27.sh | 49 ++++++++++ tools/test/stress2/misc/tmpfs28.sh | 61 +++++++++++++ 3 files changed, 289 insertions(+) ? > $ ./all.sh -o tmpfs24.sh > 20241128 22:33:38 all: tmpfs24.sh > Min hole size is 4096, file size is 524288000. > data #1 @ 0, size=3D4096) > hole #2 @ 4096, size=3D4096 > data #3 @ 8192, size=3D4096) > hole #4 @ 12288, size=3D4096 > data #5 @ 16384, size=3D4096) > hole #6 @ 20480, size=3D524267520 > --- /tmp/tmpfs24.exp 2024-11-28 22:33:40.222199000 -0500 > +++ /tmp/tmpfs24.log 2024-11-28 22:33:40.225048000 -0500 > @@ -5,4 +5,3 @@ > hole #4 @ 12288, size=3D4096 > data #5 @ 16384, size=3D4096) > hole #6 @ 20480, size=3D524267520 > -<> > FAIL tmpfs24.sh exit code 1 >=20 >>> . . . =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Fri Nov 29 04:18:18 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Y00LW6cwhz5flkd for ; Fri, 29 Nov 2024 04:19:15 +0000 (UTC) (envelope-from freebsd-current-freebsd-org111@ketas.si.pri.ee) Received: from mail.ketas.si.pri.ee (d004-fea2-0bff-021e-13e8-8437-07d0-2001.dyn.estpak.ee [IPv6:2001:7d0:8437:13e8:21e:bff:fea2:d004]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Y00LV66q2z4clx for ; Fri, 29 Nov 2024 04:19:14 +0000 (UTC) (envelope-from freebsd-current-freebsd-org111@ketas.si.pri.ee) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ketas.si.pri.ee header.s=ketas-si-pri-ee-20240416002854-4096 header.b=plp8EfAm; spf=pass (mx1.freebsd.org: domain of freebsd-current-freebsd-org111@ketas.si.pri.ee designates 2001:7d0:8437:13e8:21e:bff:fea2:d004 as permitted sender) smtp.mailfrom=freebsd-current-freebsd-org111@ketas.si.pri.ee; dmarc=pass (policy=reject) header.from=ketas.si.pri.ee X-Original-To: freebsd-current@freebsd.org DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=ketas.si.pri.ee; s=ketas-si-pri-ee-20240416002854-4096; t=1732853944; bh=Iujmx3hErGpKpcJLrB8LVe84/WPpTBeplxShaJej5cY=; h=Date:From:To:Subject:In-Reply-To:References; b=plp8EfAmL1gsLIw3uBvQvt1PR7i9i64gEAwPo2Wd1XI2scwxf2EH6cKIowyV8Inw/ MGbd2sOSuQsouQMoylx4IsmN2lAedfnjC/s/kNZupT8yyQD08WLO7cz985ooc1JCoT wlGjogXpRf66+y0r5w5PCaFnPypV0bnS3sJNAq9jOO6VSesYWmEcDHHT+XGTeuk7p/ TiweBar5ThKDl+hDovMIXl6mbcqbPTp9Q7cwVjU8ZhZuShL8hIHZxWoAq0McH+Y7Kj X4w9rd/s3doTBGkpH66uK8ssrQPX8ysRC/WWFcB3hnfCYwZqGMCvMuyx94XOGgluml c/NM50OOj4/ir+HOtE7KhQsJuVMftEMQNrUchGylwDq5mcQir+8kYccOPnU06mN2nH K0owF578x2cr7kLdiAWL6IhTBAhKZoV9WesvC3XkI9+5YRPCRZk9w+41MNu7Zh8h61 Qkm2DVHX3FJbxepQa2d1fdleNA6WPDobEAVgUePIBGrURypGAi92sQFTnh5TGgZ/Sm Mov+4/cF7UwUJtPeX2q+TytwAbtAO5XrdpuOfh6yHeYv84IUZp5krUEFt8lFsRheaW MWJuHYqUb45B1I/GzcuaTsAbHUMsh4PPoy5Oe/ghDRlO8p9SZ471kyEoGsULrxUb1v a9hNY0Y/ch5jjGjQ+hwFgioY= Received: from [IPv6:::1] (0114-0000-0000-0000-13c8-8437-07d0-2001.dyn.estpak.ee [IPv6:2001:7d0:8437:13c8::114]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.ketas.si.pri.ee (Postfix) with ESMTPSA id 495055A3A14 for ; Fri, 29 Nov 2024 06:19:04 +0200 (EET) Date: Fri, 29 Nov 2024 06:18:18 +0200 From: Sulev-Madis Silber To: freebsd-current@freebsd.org Subject: Re: RFC: fixing PR#282995 User-Agent: K-9 Mail for Android In-Reply-To: References: <4FEA762C-5E0F-4D3F-82D9-DF93CF1BC1F5@gid.co.uk> Message-ID: List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-0.90 / 15.00]; HFILTER_HOSTNAME_5(3.00)[d004-fea2-0bff-021e-13e8-8437-07d0-2001.dyn.estpak.ee]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; DMARC_POLICY_ALLOW(-0.50)[ketas.si.pri.ee,reject]; R_SPF_ALLOW(-0.20)[+ip6:2001:7d0:8437:1300::/56]; R_DKIM_ALLOW(-0.20)[ketas.si.pri.ee:s=ketas-si-pri-ee-20240416002854-4096]; ONCE_RECEIVED(0.10)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:3249, ipnet:2001:7d0::/32, country:EE]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_ONE(0.00)[1]; RCVD_TLS_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[ketas.si.pri.ee:+] X-Rspamd-Queue-Id: 4Y00LV66q2z4clx X-Spamd-Bar: / sorry to jump in like this but is this the reason the nfs exports file has been confusing the hell ou= t of me since fbsd 4=2E6? i have had no previous experience with nfs before= , nor have i used it on other oses, ever actually whole nfs was like oh god whyyy=2E felt like yeah i'm sure this m= ade sense to some guys who have been long dead since then and i'm battling = with this now, reading this about how exports should work and history of behaviour, = it makes perfect sense why it was as if black magic an manpages didn't hell= in any way either=2E it felt like i'm constantly making some grave securit= y mistakes there=2E it should never be the only method for limiting the acc= ess but nevertheless wtfs per minute has been very high there and i rather = not even touch it so after i read on reason, it partially makes sense to me=2E i wonder if i= t could be fixed somehow? or should just be left like this? or should be le= ft to confuse newcomers until big crunch takes universe away? From nobody Fri Nov 29 07:52:07 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Y054N2DBCz5g03Y; Fri, 29 Nov 2024 07:52:20 +0000 (UTC) (envelope-from mad@madpilot.net) Received: from vogon.madpilot.net (vogon.madpilot.net [159.69.1.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Y054M2Y5tz4wZy; Fri, 29 Nov 2024 07:52:19 +0000 (UTC) (envelope-from mad@madpilot.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=madpilot.net header.s=cyq4qetkgngm header.b="Y a/VU+F"; spf=pass (mx1.freebsd.org: domain of mad@madpilot.net designates 159.69.1.99 as permitted sender) smtp.mailfrom=mad@madpilot.net; dmarc=pass (policy=quarantine) header.from=madpilot.net Received: from mail (mail [IPv6:fd5c:5351:d272::3]) by vogon.madpilot.net (Postfix) with ESMTP id 4Y054D2F5TzL80L; Fri, 29 Nov 2024 08:52:12 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=madpilot.net; h= content-transfer-encoding:content-type:content-type:in-reply-to :from:from:content-language:references:subject:subject:date:date :message-id:received; s=cyq4qetkgngm; t=1732866730; x= 1734681131; bh=xBzEQuPysbtR5cz40VOmBtgSHVqwa+q7lUOD6UeQ+xw=; b=Y a/VU+Fd7tWsyzfvHzHnbhz3qstcdpNvV4iwphrZiRtRIICVy1b6EkZP7x1FiYwBy DiCe8RiLty9a2klJWpEQov75HkZV2Q6tOAQndhzroghGHCNqs+QYfRfsLwYCQXxk Tx918boitwn+wOUm7wWpZiMnaAhfzD0pu3VTMyqBtb9yEmYVYT7TlG+nSFgEG1H9 01ZLmVhPBwWd0ANfFjB2NMNZdlmRtcsgmKpSSjNosLNtlqzB1NRJ6DVuf5odSqCl 4LGXqmIvZOx1EYlY3wvUsyHWHfmRplKkdpNIP6EYhUTNpd/CWjlLYlpiWrujPzmK DuOx4/iHQLlHqe1C7oO4g== Received: from vogon.madpilot.net ([IPv6:fd5c:5351:d272::3]) by mail (vogon.madpilot.net [IPv6:fd5c:5351:d272::3]) (amavisd-new, port 10026) with ESMTP id JgKG8_gsHxop; Fri, 29 Nov 2024 08:52:10 +0100 (CET) Message-ID: <77244885-126e-431e-b514-44407b69695a@madpilot.net> Date: Fri, 29 Nov 2024 08:52:07 +0100 Subject: Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros] To: Doug Moore , Mark Millard , Konstantin Belousov Cc: Dimitry Andric , =?UTF-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , "jah@freebsd.org" , Alan Somers , Mark Johnston , FreeBSD Current , Yasuhiro Kimura , ports@freebsd.org References: <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> <0690CFB1-6A6D-4B63-916C-BAB7F6256000@yahoo.com> <3660625A-0EE8-40DA-A248-EC18C734718C@yahoo.com> <865xoa2t6f.fsf@ltc.des.dev> <69A2E921-F5E3-40D2-977D-0964EE27349A@FreeBSD.org> <4AE5B316-D7EB-4290-8D52-7FBF244EA7A4@FreeBSD.org> <33D56E3E-6476-48E8-B115-B906629B8AF5@yahoo.com> Content-Language: en-US, it From: Guido Falsi Autocrypt: addr=mad@madpilot.net; keydata= xsBNBE+G+l0BCADi/WBQ0aRJfnE7LBPsM0G3m/m3Yx7OPu4iYFvS84xawmRHtCNjWIntsxuX fptkmEo3Rsw816WUrek8dxoUAYdHd+EcpBcnnDzfDH5LW/TZ4gbrFezrHPdRp7wdxi23GN80 qPwHEwXuF0X4Wy5V0OO8B6VT/nA0ADYnBDhXS52HGIJ/GCUjgqJn+phDTdCFLvrSFdmgx4Wl c0W5Z1p5cmDF9l8L/hc959AeyNf7I9dXnjekGM9gVv7UDUYzCifR3U8T0fnfdMmS8NeI9NC+ wuREpRO4lKOkTnj9TtQJRiptlhcHQiAlG1cFqs7EQo57Tqq6cxD1FycZJLuC32bGbgalABEB AAHNHkd1aWRvIEZhbHNpIDxtYWRAbWFkcGlsb3QubmV0PsLAeQQTAQgAIwIbAwIeAQIXgAUL CQgHAwUVCgkICwQWAgMBBQJS79AgAhkBAAoJEBrmhg5Wy9KTc0kH/RO64ORBlTbTHaUaOj8F Je5O5NU2Pt9Cyt5ZWBRvxntr1zPTJGKRPS9ihlIfqT4ZvEngQGp57EUyFbCpI0UWasTerImM tt5WACnGmCzUTB39UXx8Oy4b1EgWeTJQ747e/F1mQLXTNa6ijRBE9fYlTb4gAkPN88/wVV9v 3PZozKLTg16ghBzHM/P7Lk8L7clPEZChX1FTa/6eSt3nvzfCuTMZbBPJF/ph+q1KyPqRgVfh tyhu5dvgMoPz/ni41IfeSrkJTD5RXzdyGR9q4Z1NYeBsLkRjC4LxKAP5KqUsvlOUjKvO1byj ApYdMarol+IGkaSk9e3zVYAJkWKjn/ni8XbOwU0EUxB7QQEQAKFhrDceoPdK/IHDSmoj6SQY isvM7VdhcleS7E9DoEAVt7yMbf6HbbMVTTY6ckvwTWQssywLBXNVqxgc4WLJjzfUhgef+WE7 5M3+WFYlOVQLGZY/zEVgma1raYnOHNAOzeHLDmEXjbZP6vGAeDyBbGfQPpE7qGYZ7ubeT3Xw QO+PklcCrvOPj2ZPcAxGNS2xVU/LzONqCrJqLMJSIcCdsbiSP4G5PnDFHtMokaTY6OEr8OEQ fOAerhcHUa/z7Uu8YtmaqKH+QGkE/WEgaRqSiTnv0JOTD+DxehaqvoKPPZ++2NpCZMHB2i6A /xifmQwEiIjEXtcueBRzkNUQkxhqZyS13SrhocL9ydtaVPBzZatAEjUDDEJmAMLVFs45qfyh MiNapHJo2n3MW/E5omqCvEkDdWX/en3P7CK2TemeaDghMsgkNKax/z0wNo5UZCkOPOz0xpNi UilOVbkuezZZNg65741qee2lfXhQIaZ66yT7hphc/N/z3PIAtLeze4u1VR2EXAuZ2sWAdlKC NTlJMsaU/x70BV11Wd/ypnVzM68dfdQIIAj1iMFAD/lXGlEUmKXg5Ov2VQDlTntQoanCYrAg +8CttPzjrydgLZFq3hrtQmfc0se5yv1WHS69+BsUOG09RvvawUDZxUjW19kyeN9THaNRgow3 kSuArUp6zSmJABEBAAHCwF8EGAEIAAkFAlMQe0ECGwwACgkQGuaGDlbL0pMN5wgA4bCkX/qw EVC06ToeR6C2putmSWQMgpDaqrv65Hubo+QGmg2P4ewTYQQ4g6oYWS03qHxqVVWhKz7FjfrV +dH8qbCLfSgIcvdBha7ayGZVrsiuMLKGbw36fcmkZPpSDOfHcP0XH8Z+u9CWj0xUkTxAlZ/7 i6gYSUpG2JWNtdmE/X8VVEyXusCLwy0K0BI60A/4dRTIX3C4QKrJ3ZbUXegz70ynjHf+lQMZ 9IZKASoRMuS5FozPQh6abvmwZEPdf5I9riUElzvHrqJ8Bx0t3Pujdoth+yNHpnBxrtO8LkQd rQ58P0SwcaIX33T2U9pG8bhu5YVR88FQ8OQ0cEsPBpDncg== In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Result: default: False [-2.00 / 15.00]; MISSING_MIME_VERSION(2.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[madpilot.net,quarantine]; R_DKIM_ALLOW(-0.20)[madpilot.net:s=cyq4qetkgngm]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; MISSING_XM_UA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:159.69.0.0/16, country:DE]; FREEMAIL_TO(0.00)[gmail.com,yahoo.com,freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_SEVEN(0.00)[11]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org,ports@freebsd.org]; DKIM_TRACE(0.00)[madpilot.net:+] X-Rspamd-Queue-Id: 4Y054M2Y5tz4wZy X-Spamd-Bar: - List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org On 26/11/24 18:29, Doug Moore wrote: > I think @kib has found the source of the problem.  I've attached an > attempt to fix it. Thanks for your work! I have noticed this is already in base and upgraded successfully, issue is now solved for me. Sorry for the delay in reporting this. -- Guido Falsi From nobody Fri Nov 29 14:46:29 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Y0GGK67gyz5fGfL for ; Fri, 29 Nov 2024 14:46:33 +0000 (UTC) (envelope-from SRS0=jQ2Y=SY=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int.realworks.nl (smtp-relay-int.realworks.nl [194.109.157.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Y0GGK3shZz4LKk; Fri, 29 Nov 2024 14:46:33 +0000 (UTC) (envelope-from SRS0=jQ2Y=SY=klop.ws=ronald-lists@realworks.nl) Authentication-Results: mx1.freebsd.org; none Date: Fri, 29 Nov 2024 15:46:29 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1732891590; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=AQuMZEhYjAz5balUU53GKEknEABDfOgPjpN6n7X4sPY=; b=bPK9d/TU93AsKSb01U0RcaqJ7j/5i37kaAgrEj1mgwlwKgE9MxisLALanByokDNrbZMwZE IQQd+PCImt9wx64ntoOEsmgPBytcHTA5HXpbLpycptEA++fYXxTQIHLQ4pcieiBjzW/wzf xuEZvuL8zaC/DjmkzOK9CfdkgIGoHQFAHOZKvrTn1C6QvRELzqdoomzLAo10wZw/HkamMw 4NJSHGKjZyiZqFxSb3s209GcifUPVXtXBJJBYIgq2W1KrSEABuor6ZCKoU8G+P3cErzP3n CG/cz/yAS0boTFM0sC3xNhhjMFIeTsDAeafw1YF50plIuZdLOjHIghK0fUQgXA== From: Ronald Klop To: Dennis Clarke Cc: Alan Somers , Current FreeBSD Message-ID: <1565006767.9193.1732891589527@localhost> In-Reply-To: <54808a60-a25c-4a26-915f-1ca69387db2a@blastwave.org> References: <5798b0db-bc73-476a-908a-dd1f071bfe43@blastwave.org> <22187e59-b6e9-4f2e-ba9b-f43944d1a37b@blastwave.org> <722a3644-3af6-4ff9-b1ee-022c32872001@blastwave.org> <54808a60-a25c-4a26-915f-1ca69387db2a@blastwave.org> Subject: Re: zpools no longer exist after boot List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_9192_1958593303.1732891589524" X-Mailer: Realworks (730.93) Importance: Normal X-Priority: 3 (Normal) X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:3265, ipnet:194.109.0.0/16, country:NL] X-Rspamd-Queue-Id: 4Y0GGK3shZz4LKk X-Spamd-Bar: ---- ------=_Part_9192_1958593303.1732891589524 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Van: Dennis Clarke Datum: vrijdag, 29 november 2024 01:55 Aan: Alan Somers CC: Current FreeBSD Onderwerp: Re: zpools no longer exist after boot > > On 11/28/24 10:55, Alan Somers wrote: > > On Thu, Nov 28, 2024, 9:47AM Dennis Clarke wrote: > > > >> On 11/28/24 09:52, Alan Somers wrote: > >>> On Thu, Nov 28, 2024, 8:45AM Dennis Clarke > >> wrote: > >>> > >> ... > >>> > >>> For "zpool import", the "-c" argument instructs zfs which cachefile to > >>> search for importable pools. "-O", on the other hand, specifies how the > >>> cachefile property should be set after the pool is imported. > >>> > >> > >> I have to wonder what value there is in NOT having the cachefile > >> property set in a zpool ? Certainly given that the zpool RC script > >> really wants to check in a few places and then use those cache files. > >> > >> -- > >> -- > >> Dennis Clarke > >> RISC-V/SPARC/PPC/ARM/CISC > >> UNIX and Linux spoken > >> > > > > Usually the default value for cachefile is sufficient. In fact, the only > > time I've ever needed to set cachefile has been when I DON'T want the pool > > to import at boot. I wonder if there was some other reason, since resolved, > > why your pools didn't import the first time. And subsequently they didn't > > import because you set the cachefile to a non default value? > > > > I am at a loss. Really. > > Getting the iSCSI stuff working was a real treat and this just makes > things even more strange. I really do not know. > > > > > -- > -- > Dennis Clarke > RISC-V/SPARC/PPC/ARM/CISC > UNIX and Linux spoken > > > > > Hi, I hope I can clarify a bit for you. See 'man zpoolprops' and search for: cachefile=path|none. The property 'cachefile' is not stored on disk. It is only used at the moment of import or creation of a pool to tell the 'zpool import' command (or kernel) where to store the information about imported zpools. So as the property is not stored on disk, after a reboot the zpool property 'cachefile' is always empty. It is more like a pseudo-property. Maybe it would be more clear if it had its own command line option in zpool import. If you don't pass the zpool cachefile property to zpool import it will default to /etc/zfs/zpool.cache. What happend on boot: 1. the boot loader looks at the boot-disk what pool to start from. This does not use any cachefile, it just looks at the disk blocks (because the cachefile is not available yet, as the FS is not mounted yet). 2. the init process /etc/rc.d/zpool looks in /etc/zfs/zpool.cache for extra pools to import. (as shown in a previous mail) 3. the init process /etc/rc.d/zfs mounts the zfs filesystems. There is not much more to it for local disks. As you are using iscsi also, I think you have a little bit more configuration as the iscsi device appears after /etc/rc.d/zpool & /etc/rc.d/zfs is up and running. I will answer on the iscsi part in another email. NB: I also have systems with multiple pools. I have never changed the cachefile property. If this doesn't clarify the process to you, would you be able to tell more about what you mean with 'I am at a loss'? Regards, Ronald. PS: I might have some details wrong here, other might be more knowledgeable about this, but I think this is the bigger picture. ------=_Part_9192_1958593303.1732891589524 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit

Van: Dennis Clarke <dclarke@blastwave.org>
Datum: vrijdag, 29 november 2024 01:55
Aan: Alan Somers <asomers@freebsd.org>
CC: Current FreeBSD <freebsd-current@freebsd.org>
Onderwerp: Re: zpools no longer exist after boot

On 11/28/24 10:55, Alan Somers wrote:
> On Thu, Nov 28, 2024, 9:47AM Dennis Clarke <dclarke@blastwave.org> wrote:
>
>> On 11/28/24 09:52, Alan Somers wrote:
>>> On Thu, Nov 28, 2024, 8:45AM Dennis Clarke <dclarke@blastwave.org>
>> wrote:
>>>
>> ...
>>>
>>> For "zpool import", the "-c" argument instructs zfs which cachefile to
>>> search for importable pools. "-O", on the other hand, specifies how the
>>> cachefile property should be set after the pool is imported.
>>>
>>
>> I have to wonder what value there is in NOT having the cachefile
>> property set in a zpool ?  Certainly given that the zpool RC script
>> really wants to check in a few places and then use those cache files.
>>
>> --
>> --
>> Dennis Clarke
>> RISC-V/SPARC/PPC/ARM/CISC
>> UNIX and Linux spoken
>>
>
> Usually the default value for cachefile is sufficient. In fact, the only
> time I've ever needed to set cachefile has been when I DON'T want the pool
> to import at boot. I wonder if there was some other reason, since resolved,
> why your pools didn't import the first time. And subsequently they didn't
> import because you set the cachefile to a non default value?
>

I am at a loss.  Really.

Getting the iSCSI stuff working was a real treat and this just makes
things even more strange. I really do not know.




-- 
--
Dennis Clarke
RISC-V/SPARC/PPC/ARM/CISC
UNIX and Linux spoken

 


Hi,

I hope I can clarify a bit for you.

See 'man zpoolprops' and search for: cachefile=path|none.

The property 'cachefile' is not stored on disk. It is only used at the moment of import or creation of a pool to tell the 'zpool import' command (or kernel) where to store the information about imported zpools.
So as the property is not stored on disk, after a reboot the zpool property 'cachefile' is always empty. It is more like a pseudo-property. Maybe it would be more clear if it had its own command line option in zpool import.
If you don't pass the zpool cachefile property to zpool import it will default to /etc/zfs/zpool.cache.

What happend on boot:
1. the boot loader looks at the boot-disk what pool to start from. This does not use any cachefile, it just looks at the disk blocks (because the cachefile is not available yet, as the FS is not mounted yet).
2. the init process /etc/rc.d/zpool looks in /etc/zfs/zpool.cache for extra pools to import. (as shown in a previous mail)
3. the init process /etc/rc.d/zfs mounts the zfs filesystems.

There is not much more to it for local disks. As you are using iscsi also, I think you have a little bit more configuration as the iscsi device appears after /etc/rc.d/zpool & /etc/rc.d/zfs is up and running. I will answer on the iscsi part in another email.

NB: I also have systems with multiple pools. I have never changed the cachefile property.

If this doesn't clarify the process to you, would you be able to tell more about what you mean with 'I am at a loss'?

Regards,
Ronald.

PS: I might have some details wrong here, other might be more knowledgeable about this, but I think this is the bigger picture. ------=_Part_9192_1958593303.1732891589524-- From nobody Fri Nov 29 14:49:27 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Y0GKk4w6mz5fGqq for ; Fri, 29 Nov 2024 14:49:30 +0000 (UTC) (envelope-from SRS0=jQ2Y=SY=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int.realworks.nl (smtp-relay-int.realworks.nl [194.109.157.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Y0GKj1RJDz4MDG; Fri, 29 Nov 2024 14:49:29 +0000 (UTC) (envelope-from SRS0=jQ2Y=SY=klop.ws=ronald-lists@realworks.nl) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=klop.ws header.s=rw2 header.b=JF6oaIP0; spf=pass (mx1.freebsd.org: domain of "SRS0=jQ2Y=SY=klop.ws=ronald-lists@realworks.nl" designates 194.109.157.24 as permitted sender) smtp.mailfrom="SRS0=jQ2Y=SY=klop.ws=ronald-lists@realworks.nl"; dmarc=pass (policy=quarantine) header.from=klop.ws Date: Fri, 29 Nov 2024 15:49:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1732891767; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=pNEEgVKbXj03Ubotxam9QZSc6z+JXtVXvru++o/l3Ok=; b=JF6oaIP0oAnR/S+8GDzGZ+PkN9s4Ym44Qh3All2Bwfu1+gwzekql6q7BIBStOdGpL/ZbYp pJ4BY5RiuyweCscDUYgiQLy7zlgQk79nOZn4d9OQFWzThj0vkljbaRNLxgHWkYjK9zdcog ybRZzvGMb2ZEV+gVJZfX1JHOMphl5Iahfehk4PAssS4KPro+aF2xTG6HlZN5ffAGSnhahx fkMmVjNspmcfmI7SN/RUFPW1t0/uJm94WXr/OiLPQeCkQExwSLW0mKjc9igqyTxKKhaV1a +WdSpetALd7CNVl3lmy7XXMWgogTJIHcj2wbtbysIA5czkdsX/BXkddnEVP93A== From: Ronald Klop To: Dennis Clarke Cc: Alan Somers , Current FreeBSD Message-ID: <754754561.9245.1732891767670@localhost> In-Reply-To: <22187e59-b6e9-4f2e-ba9b-f43944d1a37b@blastwave.org> References: <5798b0db-bc73-476a-908a-dd1f071bfe43@blastwave.org> <22187e59-b6e9-4f2e-ba9b-f43944d1a37b@blastwave.org> Subject: Re: zpools no longer exist after boot List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_9244_611194504.1732891767665" X-Mailer: Realworks (730.93) Importance: Normal X-Priority: 3 (Normal) X-Spamd-Result: default: False [-3.19 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.988]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[klop.ws,quarantine]; FORGED_SENDER(0.30)[ronald-lists@klop.ws,SRS0=jQ2Y=SY=klop.ws=ronald-lists@realworks.nl]; R_DKIM_ALLOW(-0.20)[klop.ws:s=rw2]; R_SPF_ALLOW(-0.20)[+ip4:194.109.157.0/24]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; ARC_NA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[194.109.157.24:from]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:3265, ipnet:194.109.0.0/16, country:NL]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCPT_COUNT_THREE(0.00)[3]; FROM_NEQ_ENVFROM(0.00)[ronald-lists@klop.ws,SRS0=jQ2Y=SY=klop.ws=ronald-lists@realworks.nl]; FROM_HAS_DN(0.00)[]; RCVD_COUNT_ZERO(0.00)[0]; HAS_X_PRIO_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[klop.ws:+] X-Rspamd-Queue-Id: 4Y0GKj1RJDz4MDG X-Spamd-Bar: --- ------=_Part_9244_611194504.1732891767665 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Van: Dennis Clarke Datum: donderdag, 28 november 2024 15:45 Aan: Alan Somers CC: Current FreeBSD Onderwerp: Re: zpools no longer exist after boot > > On 11/28/24 08:52, Alan Somers wrote: > > On Thu, Nov 28, 2024, 7:06AM Dennis Clarke wrote: > > > >> > >> This is a baffling problem wherein two zpools no longer exist after > >> boot. This is : > . > . > . > > Do you have zfs_enable="YES" set in /etc/rc.conf? If not then nothing will > > get imported. > > > > Regarding the cachefile property, it's expected that "zpool import" will > > change it, unless you do "zpool import -O cachefile=whatever". > > > > The rc script seems to do something slightly different with zpool import -c $FOOBAR thus : > > > titan# cat /etc/rc.d/zpool > #!/bin/sh > # > # > > # PROVIDE: zpool > # REQUIRE: hostid disks > # BEFORE: mountcritlocal > # KEYWORD: nojail > > . /etc/rc.subr > > name="zpool" > desc="Import ZPOOLs" > rcvar="zfs_enable" > start_cmd="zpool_start" > required_modules="zfs" > > zpool_start() > { > local cachefile > > for cachefile in /etc/zfs/zpool.cache /boot/zfs/zpool.cache; do > if [ -r $cachefile ]; then > zpool import -c $cachefile -a -N > if [ $? -ne 0 ]; then > echo "Import of zpool cache ${cachefile} failed," \ > "will retry after root mount hold release" > root_hold_wait > zpool import -c $cachefile -a -N > fi > break > fi > done > } > > load_rc_config $name > run_rc_command "$1" > titan# > > > > I may as well nuke the pre-existing cache file and start over : > > > titan# ls -l /etc/zfs/zpool.cache /boot/zfs/zpool.cache > -rw-r--r-- 1 root wheel 1424 Jan 16 2024 /boot/zfs/zpool.cache > -rw-r--r-- 1 root wheel 4960 Nov 28 14:15 /etc/zfs/zpool.cache > titan# > titan# > titan# rm /boot/zfs/zpool.cache > titan# zpool set cachefile="/boot/zfs/zpool.cache" t0 > titan# > titan# ls -l /boot/zfs/zpool.cache > -rw-r--r-- 1 root wheel 1456 Nov 28 14:27 /boot/zfs/zpool.cache > titan# > titan# zpool set cachefile="/boot/zfs/zpool.cache" leaf > titan# > titan# ls -l /boot/zfs/zpool.cache > -rw-r--r-- 1 root wheel 3536 Nov 28 14:28 /boot/zfs/zpool.cache > titan# > titan# zpool set cachefile="/boot/zfs/zpool.cache" proteus > titan# > titan# ls -l /boot/zfs/zpool.cache > -rw-r--r-- 1 root wheel 4960 Nov 28 14:28 /boot/zfs/zpool.cache > titan# > titan# zpool get cachefile t0 > NAME PROPERTY VALUE SOURCE > t0 cachefile /boot/zfs/zpool.cache local > titan# > titan# zpool get cachefile leaf > NAME PROPERTY VALUE SOURCE > leaf cachefile /boot/zfs/zpool.cache local > titan# > titan# zpool get cachefile proteus > NAME PROPERTY VALUE SOURCE > proteus cachefile /boot/zfs/zpool.cache local > titan# > > titan# > titan# reboot > Nov 28 14:34:05 Waiting (max 60 seconds) for system process `vnlru' to stop... done > Waiting (max 60 seconds) for system process `syncer' to stop... > Syncing disks, vnodes remaining... 0 0 0 0 0 0 done > All buffers synced. > Uptime: 2h38m57s > GEOM_MIRROR: Device swap: provider destroyed. > GEOM_MIRROR: Device swap destroyed. > uhub5: detached > uhub1: detached > uhub4: detached > uhub2: detached > uhub3: detached > uhub6: detached > uhub0: detached > ix0: link state changed to DOWN > . > . > . > > Starting iscsid. > Starting iscsictl. > Clearing /tmp. > Updating /var/run/os-release done. > Updating motd:. > Creating and/or trimming log files. > Starting syslogd. > No core dumps found. > Starting local daemons:failed to open cache file: No such file or directory > . > Starting ntpd. > Starting powerd. > Mounting late filesystems:. > Starting cron. > Performing sanity check on sshd configuration. > Starting sshd. > Starting background file system > FreeBSD/amd64 (titan) (ttyu0) > > login: root > Password: > Nov 28 14:36:29 titan login[4162]: ROOT LOGIN (root) ON ttyu0 > Last login: Thu Nov 28 14:33:45 on ttyu0 > FreeBSD 15.0-CURRENT (GENERIC-NODEBUG) #1 main-n273749-4b65481ac68a-dirty: Wed Nov 20 15:08:52 GMT 2024 > > Welcome to FreeBSD! > > Release Notes, Errata: https://www.FreeBSD.org/releases/ > Security Advisories: https://www.FreeBSD.org/security/ > FreeBSD Handbook: https://www.FreeBSD.org/handbook/ > FreeBSD FAQ: https://www.FreeBSD.org/faq/ > Questions List: https://www.FreeBSD.org/lists/questions/ > FreeBSD Forums: https://forums.FreeBSD.org/ > > Documents installed with the system are in the /usr/local/share/doc/freebsd/ > directory, or can be installed later with: pkg install en-freebsd-doc > For other languages, replace "en" with a language code like de or fr. > > Show the version of FreeBSD installed: freebsd-version ; uname -a > Please include that output and any error messages when posting questions. > Introduction to manual pages: man man > FreeBSD directory layout: man hier > > To change this login announcement, see motd(5). > You have new mail. > titan# > titan# zpool list > NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > leaf 18.2T 984K 18.2T - - 0% 0% 1.00x ONLINE - > proteus 1.98T 361G 1.63T - - 1% 17% 1.00x ONLINE - > t0 444G 91.2G 353G - - 27% 20% 1.00x ONLINE - > titan# > > This is progress ... however the cachefile property is wiped out again : > > titan# zpool get cachefile t0 > NAME PROPERTY VALUE SOURCE > t0 cachefile - default > titan# zpool get cachefile leaf > NAME PROPERTY VALUE SOURCE > leaf cachefile - default > titan# zpool get cachefile proteus > NAME PROPERTY VALUE SOURCE > proteus cachefile - default > titan# > > Also, strangely, none of the filesystem in proteus are mounted : > > titan# > titan# zfs list -o name,exec,checksum,canmount,mounted,mountpoint -r proteus > NAME EXEC CHECKSUM CANMOUNT MOUNTED MOUNTPOINT > proteus on sha512 on no none > proteus/bhyve off sha512 on no /bhyve > proteus/bhyve/disk off sha512 on no /bhyve/disk > proteus/bhyve/isos off sha512 on no /bhyve/isos > proteus/obj on sha512 on no /usr/obj > proteus/src on sha512 on no /usr/src > titan# > > If I reboot again without doing anything will the zpools re-appear ? > > > titan# > titan# Nov 28 14:37:08 titan su[4199]: admsys to root on /dev/pts/0 > > titan# reboot > Nov 28 14:40:29 Waiting (max 60 seconds) for system process `vnlru' to stop... done > Waiting (max 60 seconds) for system process `syncer' to stop... > Syncing disks, vnodes remaining... 0 0 0 0 0 done > All buffers synced. > Uptime: 4m50s > GEOM_MIRROR: Device swap: provider destroyed. > GEOM_MIRROR: Device swap destroyed. > uhub4: detached > uhub1: detached > uhub5: detached > uhub0: detached > uhub3: detached > uhub6: detached > uhub2: detached > ix0: link state changed to DOWN > . > . > . > Starting iscsid. > Starting iscsictl. > Clearing /tmp. > Updating /var/run/os-release done. > Updating motd:. > Creating and/or trimming log files. > Starting syslogd. > No core dumps found. > Starting local daemons:failed to open cache file: No such file or directory > . > Starting ntpd. > Starting powerd. > Mounting late filesystems:. > Starting cron. > Performing sanity check on sshd configuration. > Starting sshd. > Starting background file system > FreeBSD/amd64 (titan) (ttyu0) > > login: root > Password: > Nov 28 14:43:01 titan login[4146]: ROOT LOGIN (root) ON ttyu0 > Last login: Thu Nov 28 14:36:29 on ttyu0 > FreeBSD 15.0-CURRENT (GENERIC-NODEBUG) #1 main-n273749-4b65481ac68a-dirty: Wed Nov 20 15:08:52 GMT 2024 > > Welcome to FreeBSD! > > Release Notes, Errata: https://www.FreeBSD.org/releases/ > Security Advisories: https://www.FreeBSD.org/security/ > FreeBSD Handbook: https://www.FreeBSD.org/handbook/ > FreeBSD FAQ: https://www.FreeBSD.org/faq/ > Questions List: https://www.FreeBSD.org/lists/questions/ > FreeBSD Forums: https://forums.FreeBSD.org/ > > Documents installed with the system are in the /usr/local/share/doc/freebsd/ > directory, or can be installed later with: pkg install en-freebsd-doc > For other languages, replace "en" with a language code like de or fr. > > Show the version of FreeBSD installed: freebsd-version ; uname -a > Please include that output and any error messages when posting questions. > Introduction to manual pages: man man > FreeBSD directory layout: man hier > > To change this login announcement, see motd(5). > You have new mail. > titan# > titan# zpool list > NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > leaf 18.2T 1.01M 18.2T - - 0% 0% 1.00x ONLINE - > proteus 1.98T 361G 1.63T - - 1% 17% 1.00x ONLINE - > t0 444G 91.2G 353G - - 27% 20% 1.00x ONLINE - > titan# > titan# zfs list -o name,exec,checksum,canmount,mounted,mountpoint -r proteus > NAME EXEC CHECKSUM CANMOUNT MOUNTED MOUNTPOINT > proteus on sha512 on no none > proteus/bhyve off sha512 on no /bhyve > proteus/bhyve/disk off sha512 on no /bhyve/disk > proteus/bhyve/isos off sha512 on no /bhyve/isos > proteus/obj on sha512 on no /usr/obj > proteus/src on sha512 on no /usr/src > titan# > > OKay so the zpools appear to be back in spite of the strange situation with the cachefile property is empty everywhere. My guess is the zpool > rc script is bring in information during early boot. > > Why the zfs filesystems on proteus do not mount? Well that is a strange problem but at least the zpool can be used. > > -- > -- > Dennis Clarke > RISC-V/SPARC/PPC/ARM/CISC > UNIX and Linux spoken > > > > > Hi, The output you provide contains this line: "Starting local daemons:failed to open cache file: No such file or directory" Where does that output come from? What is in your file /etc/rc.local file? Regards, Ronald. ------=_Part_9244_611194504.1732891767665 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit

Van: Dennis Clarke <dclarke@blastwave.org>
Datum: donderdag, 28 november 2024 15:45
Aan: Alan Somers <asomers@freebsd.org>
CC: Current FreeBSD <freebsd-current@freebsd.org>
Onderwerp: Re: zpools no longer exist after boot

On 11/28/24 08:52, Alan Somers wrote:
> On Thu, Nov 28, 2024, 7:06AM Dennis Clarke <dclarke@blastwave.org> wrote:
>
>>
>> This is a baffling problem wherein two zpools no longer exist after
>> boot. This is :
.
.
.
> Do you have zfs_enable="YES" set in /etc/rc.conf? If not then nothing will
> get imported.
>
> Regarding the cachefile property, it's expected that "zpool import" will
> change it, unless you do "zpool import -O cachefile=whatever".
>

The rc script seems to do something slightly different with zpool import -c $FOOBAR thus :


titan# cat  /etc/rc.d/zpool
#!/bin/sh
#
#

# PROVIDE: zpool
# REQUIRE: hostid disks
# BEFORE: mountcritlocal
# KEYWORD: nojail

. /etc/rc.subr

name="zpool"
desc="Import ZPOOLs"
rcvar="zfs_enable"
start_cmd="zpool_start"
required_modules="zfs"

zpool_start()
{
         local cachefile

         for cachefile in /etc/zfs/zpool.cache /boot/zfs/zpool.cache; do
                 if [ -r $cachefile ]; then
                         zpool import -c $cachefile -a -N
                         if [ $? -ne 0 ]; then
                                 echo "Import of zpool cache ${cachefile} failed," \
                                     "will retry after root mount hold release"
                                 root_hold_wait
                                 zpool import -c $cachefile -a -N
                         fi
                         break
                 fi
         done
}

load_rc_config $name
run_rc_command "$1"
titan#



I may as well nuke the pre-existing cache file and start over :


titan# ls -l /etc/zfs/zpool.cache /boot/zfs/zpool.cache
-rw-r--r--  1 root wheel 1424 Jan 16  2024 /boot/zfs/zpool.cache
-rw-r--r--  1 root wheel 4960 Nov 28 14:15 /etc/zfs/zpool.cache
titan#
titan#
titan# rm /boot/zfs/zpool.cache
titan# zpool set cachefile="/boot/zfs/zpool.cache" t0
titan#
titan# ls -l /boot/zfs/zpool.cache
-rw-r--r--  1 root wheel 1456 Nov 28 14:27 /boot/zfs/zpool.cache
titan#
titan# zpool set cachefile="/boot/zfs/zpool.cache" leaf
titan#
titan# ls -l /boot/zfs/zpool.cache
-rw-r--r--  1 root wheel 3536 Nov 28 14:28 /boot/zfs/zpool.cache
titan#
titan# zpool set cachefile="/boot/zfs/zpool.cache" proteus
titan#
titan# ls -l /boot/zfs/zpool.cache
-rw-r--r--  1 root wheel 4960 Nov 28 14:28 /boot/zfs/zpool.cache
titan#
titan# zpool get cachefile t0
NAME  PROPERTY   VALUE                  SOURCE
t0    cachefile  /boot/zfs/zpool.cache  local
titan#
titan# zpool get cachefile leaf
NAME  PROPERTY   VALUE                  SOURCE
leaf  cachefile  /boot/zfs/zpool.cache  local
titan#
titan# zpool get cachefile proteus
NAME     PROPERTY   VALUE                  SOURCE
proteus  cachefile  /boot/zfs/zpool.cache  local
titan#

titan#
titan# reboot
Nov 28 14:34:05 Waiting (max 60 seconds) for system process `vnlru' to stop... done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining... 0 0 0 0 0 0 done
All buffers synced.
Uptime: 2h38m57s
GEOM_MIRROR: Device swap: provider destroyed.
GEOM_MIRROR: Device swap destroyed.
uhub5: detached
uhub1: detached
uhub4: detached
uhub2: detached
uhub3: detached
uhub6: detached
uhub0: detached
ix0: link state changed to DOWN
.
.
.

Starting iscsid.
Starting iscsictl.
Clearing /tmp.
Updating /var/run/os-release done.
Updating motd:.
Creating and/or trimming log files.
Starting syslogd.
No core dumps found.
Starting local daemons:failed to open cache file: No such file or directory
.
Starting ntpd.
Starting powerd.
Mounting late filesystems:.
Starting cron.
Performing sanity check on sshd configuration.
Starting sshd.
Starting background file system
FreeBSD/amd64 (titan) (ttyu0)

login: root
Password:
Nov 28 14:36:29 titan login[4162]: ROOT LOGIN (root) ON ttyu0
Last login: Thu Nov 28 14:33:45 on ttyu0
FreeBSD 15.0-CURRENT (GENERIC-NODEBUG) #1 main-n273749-4b65481ac68a-dirty: Wed Nov 20 15:08:52 GMT 2024

Welcome to FreeBSD!

Release Notes, Errata: https://www.FreeBSD.org/releases/
Security Advisories:   https://www.FreeBSD.org/security/
FreeBSD Handbook:      https://www.FreeBSD.org/handbook/
FreeBSD FAQ:           https://www.FreeBSD.org/faq/
Questions List:        https://www.FreeBSD.org/lists/questions/
FreeBSD Forums:        https://forums.FreeBSD.org/

Documents installed with the system are in the /usr/local/share/doc/freebsd/
directory, or can be installed later with:  pkg install en-freebsd-doc
For other languages, replace "en" with a language code like de or fr.

Show the version of FreeBSD installed:  freebsd-version ; uname -a
Please include that output and any error messages when posting questions.
Introduction to manual pages:  man man
FreeBSD directory layout:      man hier

To change this login announcement, see motd(5).
You have new mail.
titan#
titan# zpool list
NAME      SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP HEALTH  ALTROOT
leaf     18.2T   984K  18.2T        -         -     0%     0%  1.00x ONLINE  -
proteus  1.98T   361G  1.63T        -         -     1%    17%  1.00x ONLINE  -
t0        444G  91.2G   353G        -         -    27%    20%  1.00x ONLINE  -
titan#

This is progress ... however the cachefile property is wiped out again :

titan# zpool get cachefile t0
NAME  PROPERTY   VALUE      SOURCE
t0    cachefile  -          default
titan# zpool get cachefile leaf
NAME  PROPERTY   VALUE      SOURCE
leaf  cachefile  -          default
titan# zpool get cachefile proteus
NAME     PROPERTY   VALUE      SOURCE
proteus  cachefile  -          default
titan#

Also, strangely, none of the filesystem in proteus are mounted :

titan#
titan# zfs list -o name,exec,checksum,canmount,mounted,mountpoint -r proteus
NAME                EXEC  CHECKSUM   CANMOUNT  MOUNTED  MOUNTPOINT
proteus             on    sha512     on        no       none
proteus/bhyve       off   sha512     on        no       /bhyve
proteus/bhyve/disk  off   sha512     on        no       /bhyve/disk
proteus/bhyve/isos  off   sha512     on        no       /bhyve/isos
proteus/obj         on    sha512     on        no       /usr/obj
proteus/src         on    sha512     on        no       /usr/src
titan#

If I reboot again without doing anything will the zpools re-appear ?


titan#
titan# Nov 28 14:37:08 titan su[4199]: admsys to root on /dev/pts/0

titan# reboot
Nov 28 14:40:29 Waiting (max 60 seconds) for system process `vnlru' to stop... done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining... 0 0 0 0 0 done
All buffers synced.
Uptime: 4m50s
GEOM_MIRROR: Device swap: provider destroyed.
GEOM_MIRROR: Device swap destroyed.
uhub4: detached
uhub1: detached
uhub5: detached
uhub0: detached
uhub3: detached
uhub6: detached
uhub2: detached
ix0: link state changed to DOWN
.
.
.
Starting iscsid.
Starting iscsictl.
Clearing /tmp.
Updating /var/run/os-release done.
Updating motd:.
Creating and/or trimming log files.
Starting syslogd.
No core dumps found.
Starting local daemons:failed to open cache file: No such file or directory
.
Starting ntpd.
Starting powerd.
Mounting late filesystems:.
Starting cron.
Performing sanity check on sshd configuration.
Starting sshd.
Starting background file system
FreeBSD/amd64 (titan) (ttyu0)

login: root
Password:
Nov 28 14:43:01 titan login[4146]: ROOT LOGIN (root) ON ttyu0
Last login: Thu Nov 28 14:36:29 on ttyu0
FreeBSD 15.0-CURRENT (GENERIC-NODEBUG) #1 main-n273749-4b65481ac68a-dirty: Wed Nov 20 15:08:52 GMT 2024

Welcome to FreeBSD!

Release Notes, Errata: https://www.FreeBSD.org/releases/
Security Advisories:   https://www.FreeBSD.org/security/
FreeBSD Handbook:      https://www.FreeBSD.org/handbook/
FreeBSD FAQ:           https://www.FreeBSD.org/faq/
Questions List:        https://www.FreeBSD.org/lists/questions/
FreeBSD Forums:        https://forums.FreeBSD.org/

Documents installed with the system are in the /usr/local/share/doc/freebsd/
directory, or can be installed later with:  pkg install en-freebsd-doc
For other languages, replace "en" with a language code like de or fr.

Show the version of FreeBSD installed:  freebsd-version ; uname -a
Please include that output and any error messages when posting questions.
Introduction to manual pages:  man man
FreeBSD directory layout:      man hier

To change this login announcement, see motd(5).
You have new mail.
titan#
titan# zpool list
NAME      SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP HEALTH  ALTROOT
leaf     18.2T  1.01M  18.2T        -         -     0%     0%  1.00x ONLINE  -
proteus  1.98T   361G  1.63T        -         -     1%    17%  1.00x ONLINE  -
t0        444G  91.2G   353G        -         -    27%    20%  1.00x ONLINE  -
titan#
titan# zfs list -o name,exec,checksum,canmount,mounted,mountpoint -r proteus
NAME                EXEC  CHECKSUM   CANMOUNT  MOUNTED  MOUNTPOINT
proteus             on    sha512     on        no       none
proteus/bhyve       off   sha512     on        no       /bhyve
proteus/bhyve/disk  off   sha512     on        no       /bhyve/disk
proteus/bhyve/isos  off   sha512     on        no       /bhyve/isos
proteus/obj         on    sha512     on        no       /usr/obj
proteus/src         on    sha512     on        no       /usr/src
titan#

OKay so the zpools appear to be back in spite of the strange situation with the cachefile property is empty everywhere.  My guess is the zpool
rc script is bring in information during early boot.

Why the zfs filesystems on proteus do not mount? Well that is a strange problem but at least the zpool can be used.

-- 
--
Dennis Clarke
RISC-V/SPARC/PPC/ARM/CISC
UNIX and Linux spoken

 


Hi,

The output you provide contains this line:
"Starting local daemons:failed to open cache file: No such file or directory"

Where does that output come from? What is in your file /etc/rc.local file?

Regards,
Ronald.
  ------=_Part_9244_611194504.1732891767665-- From nobody Fri Nov 29 15:00:35 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Y0Gb80M2Pz5fGyB; Fri, 29 Nov 2024 15:01:08 +0000 (UTC) (envelope-from thj@freebsd.org) Received: from fout-b1-smtp.messagingengine.com (fout-b1-smtp.messagingengine.com [202.12.124.144]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Y0Gb71ZJ3z4NwV; Fri, 29 Nov 2024 15:01:07 +0000 (UTC) (envelope-from thj@freebsd.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=nJ1JLEuH; spf=softfail (mx1.freebsd.org: 202.12.124.144 is neither permitted nor denied by domain of thj@freebsd.org) smtp.mailfrom=thj@freebsd.org; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=freebsd.org (policy=none) Received: from phl-compute-02.internal (phl-compute-02.phl.internal [10.202.2.42]) by mailfout.stl.internal (Postfix) with ESMTP id 5612411401B3; Fri, 29 Nov 2024 10:01:06 -0500 (EST) Received: from phl-imap-07 ([10.202.2.97]) by phl-compute-02.internal (MEProxy); Fri, 29 Nov 2024 10:01:06 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:message-id:mime-version:reply-to:subject:subject:to :to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1732892466; x=1732978866; bh=3yHdVRkEN7McyPIADxyVinlEdZgSnpNU/Ir B1FrGjac=; b=nJ1JLEuHIGj1pZFSvzy8ZQjS9n1s2+JYLqUZj/TvaF8h9bRF4RV nV2UzqCnBYxoCFwgTR+Q2rO6ZTPNGPKAArtPFZAf6t5hi7rGl/rsIzuFjYlP2r9L eXjF7S0WNAEZ2cpsvk98CE/ZyziBSfvcrn5Hk93TbdrO8QtFGGgSo0t/44zlvSMM JVbnid3TdOhz0h3gxL2qlJhBksWC73piZVZho0F9/fO3+89dxpxj7+7+LEJfU4Bx ko5XKUgC2Pku3nJKhOclmpGIr0yTZktOWQTmHfXS/WUThhTh+OeQ0bmq/S7Lj7H8 BahDpA/oeWFssk0wpc+4Pt5N+f7b1nKg3ug== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefuddrheefgdejtdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdpuffr tefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecunecujfgurhepofggfffhvf fkufgtgfesthejredtredttdenucfhrhhomhepfdfvohhmucflohhnvghsfdcuoehthhhj sehfrhgvvggsshgurdhorhhgqeenucggtffrrghtthgvrhhnpeefvdevhedtheekveffge dtieehhedvgedvteevgffhleehhfefieeuhfeijeetteenucffohhmrghinheprgguvhgv nhhtuhhrihhsthdrmhgvnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrg hilhhfrhhomhepthhhjhesfhhrvggvsghsugdrohhrghdpnhgspghrtghpthhtohepvddp mhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepfhhrvggvsghsugdqtghurhhrvghnth esfhhrvggvsghsugdrohhrghdprhgtphhtthhopehfrhgvvggsshguqdhnvghtsehfrhgv vggsshgurdhorhhg X-ME-Proxy: Feedback-ID: ib75146ab:Fastmail Received: by mailuser.phl.internal (Postfix, from userid 501) id B0DCFBA0073; Fri, 29 Nov 2024 10:01:05 -0500 (EST) X-Mailer: MessagingEngine.com Webmail Interface List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Date: Fri, 29 Nov 2024 15:00:35 +0000 From: "Tom Jones" To: freebsd-current@freebsd.org, freebsd-net@freebsd.org Message-Id: <681118fa-ea1f-4278-ba91-29e76709f612@app.fastmail.com> Subject: FreeBSD Network Status Report for Week 48 2024 Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-3.26 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.97)[-0.969]; R_DKIM_ALLOW(-0.20)[messagingengine.com:s=fm1]; MIME_GOOD(-0.10)[text/plain]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : No valid SPF, DKIM not aligned (relaxed),none]; RCVD_IN_DNSWL_LOW(-0.10)[202.12.124.144:from]; XM_UA_NO_VERSION(0.01)[]; ARC_NA(0.00)[]; ASN(0.00)[asn:151847, ipnet:202.12.124.0/24, country:AU]; MIME_TRACE(0.00)[0:+]; FREEFALL_USER(0.00)[thj]; DWL_DNSWL_NONE(0.00)[messagingengine.com:dkim]; FROM_HAS_DN(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org,freebsd-net@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; R_SPF_SOFTFAIL(0.00)[~all:c]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; TO_DN_NONE(0.00)[]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[messagingengine.com:+] X-Rspamd-Queue-Id: 4Y0Gb71ZJ3z4NwV X-Spamd-Bar: --- Hi hackers, I have written a 10th Network status report, you can find it here: https://adventurist.me/posts/00337 And all previous posts are collected at this url: https://adventurist.me/tag/networkstatus One thing we would like to know is how people are coming across these reports. Is it from these mailing lists? Some sort of social media? RFC 1149 datagrams? I plan to write two more of these in 2024 and then break for the Winter. If you would like to see them continue in 2025 it will help me to know what from the reports gives you the most value and what else you would like to know about. Thanks, - Tom From nobody Fri Nov 29 18:41:16 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Y0MTH2fXLz5fY5N for ; Fri, 29 Nov 2024 18:41:23 +0000 (UTC) (envelope-from dclarke@blastwave.org) Received: from mail.oetec.com (mail.oetec.com [108.160.241.186]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "mail.oetec.com", Issuer "E6" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Y0MTG3FpVz4lcM for ; Fri, 29 Nov 2024 18:41:22 +0000 (UTC) (envelope-from dclarke@blastwave.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=blastwave.org header.s=default header.b=RNdtgZQX; spf=pass (mx1.freebsd.org: domain of dclarke@blastwave.org designates 108.160.241.186 as permitted sender) smtp.mailfrom=dclarke@blastwave.org; dmarc=pass (policy=quarantine) header.from=blastwave.org Received: from [172.16.35.3] (pool-99-253-118-250.cpe.net.cable.rogers.com [99.253.118.250]) (authenticated bits=0) by mail.oetec.com (8.17.1/8.17.1) with ESMTPSA id 4ATIfHAq066071 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT) for ; Fri, 29 Nov 2024 13:41:18 -0500 (EST) (envelope-from dclarke@blastwave.org) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=blastwave.org; s=default; t=1732905678; bh=/pNHqA+mQ/vEvZV2UW0hyIFb49g0ZtFj/FHups9frDw=; h=Date:Subject:To:References:From:In-Reply-To; b=RNdtgZQXuedFUNudF9KbrG0Drv//AmQgUd09+ijwP3jt0jYBO8qUsPllykKq5bg8U SRMDRmxJPgSCjnR5QNwL2TK+TwU8VbZz9Xbg1w6ukFvFqjHqKE2YjqxvikKGPZi9Ft ysYrJlbqrpoq2i9hyuxD+oyFX17955hYXYXtphFuv1h+RTt2rfRswWw/YP3wGCYEWx 26WeYppBb/almyc9RVx320mTC4FkhJNvU56wlqP7bJyfREcX0nbbd2lLFerOQCibGE uUL+cu76lpWywkduVEwo0c7K15/cOinVQDH4BIIaDM3pFLoDzMmCt4w44A4+OO9qmm yzNRVfanWJRJQ== Message-ID: <1d22bbb4-85fc-4817-a0ee-d1b25a55d220@blastwave.org> Date: Fri, 29 Nov 2024 13:41:16 -0500 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: zpools no longer exist after boot Content-Language: en-CA To: freebsd-current@freebsd.org References: <5798b0db-bc73-476a-908a-dd1f071bfe43@blastwave.org> <22187e59-b6e9-4f2e-ba9b-f43944d1a37b@blastwave.org> <754754561.9245.1732891767670@localhost> From: Dennis Clarke Organization: GENUNIX In-Reply-To: <754754561.9245.1732891767670@localhost> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-oetec-MailScanner-Information: Please contact the ISP for more information X-oetec-MailScanner-ID: 4ATIfHAq066071 X-oetec-MailScanner: Found to be clean X-oetec-MailScanner-From: dclarke@blastwave.org X-Spam-Status: No X-Spamd-Result: default: False [-4.69 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[blastwave.org,quarantine]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; R_SPF_ALLOW(-0.20)[+mx]; RCVD_IN_DNSWL_MED(-0.20)[108.160.241.186:from]; R_DKIM_ALLOW(-0.20)[blastwave.org:s=default]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_ONE(0.00)[1]; TO_MATCH_ENVRCPT_ALL(0.00)[]; HAS_ORG_HEADER(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; ARC_NA(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[blastwave.org:+]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:812, ipnet:108.160.240.0/20, country:CA]; FROM_HAS_DN(0.00)[] X-Rspamd-Queue-Id: 4Y0MTG3FpVz4lcM X-Spamd-Bar: ---- On 11/29/24 09:49, Ronald Klop wrote: > Van: Dennis Clarke > Datum: donderdag, 28 november 2024 15:45 > Aan: Alan Somers > CC: Current FreeBSD > Onderwerp: Re: zpools no longer exist after boot >> >> On 11/28/24 08:52, Alan Somers wrote: >> > On Thu, Nov 28, 2024, 7:06AM Dennis Clarke >> wrote: >> > >> >> >> >> This is a baffling problem wherein two zpools no longer exist after >> >> boot. This is : >> . >> . >> . >> > Do you have zfs_enable="YES" set in /etc/rc.conf? If not then >> nothing will >> > get imported. >> > >> > Regarding the cachefile property, it's expected that "zpool import" >> will >> > change it, unless you do "zpool import -O cachefile=whatever". >> > >> >> The rc script seems to do something slightly different with zpool >> import -c $FOOBAR thus : >> >> >> titan# cat /etc/rc.d/zpool >> #!/bin/sh >> # >> # >> >> # PROVIDE: zpool >> # REQUIRE: hostid disks >> # BEFORE: mountcritlocal >> # KEYWORD: nojail >> >> . /etc/rc.subr >> >> name="zpool" >> desc="Import ZPOOLs" >> rcvar="zfs_enable" >> start_cmd="zpool_start" >> required_modules="zfs" >> >> zpool_start() >> { >> local cachefile >> >> for cachefile in /etc/zfs/zpool.cache /boot/zfs/zpool.cache; do >> if [ -r $cachefile ]; then >> zpool import -c $cachefile -a -N >> if [ $? -ne 0 ]; then >> echo "Import of zpool cache >> ${cachefile} failed," \ >> "will retry after root mount hold >> release" >> root_hold_wait >> zpool import -c $cachefile -a -N >> fi >> break >> fi >> done >> } >> >> load_rc_config $name >> run_rc_command "$1" >> titan# >> >> >> >> I may as well nuke the pre-existing cache file and start over : >> >> >> titan# ls -l /etc/zfs/zpool.cache /boot/zfs/zpool.cache >> -rw-r--r-- 1 root wheel 1424 Jan 16 2024 /boot/zfs/zpool.cache >> -rw-r--r-- 1 root wheel 4960 Nov 28 14:15 /etc/zfs/zpool.cache >> titan# >> titan# >> titan# rm /boot/zfs/zpool.cache >> titan# zpool set cachefile="/boot/zfs/zpool.cache" t0 >> titan# >> titan# ls -l /boot/zfs/zpool.cache >> -rw-r--r-- 1 root wheel 1456 Nov 28 14:27 /boot/zfs/zpool.cache >> titan# >> titan# zpool set cachefile="/boot/zfs/zpool.cache" leaf >> titan# >> titan# ls -l /boot/zfs/zpool.cache >> -rw-r--r-- 1 root wheel 3536 Nov 28 14:28 /boot/zfs/zpool.cache >> titan# >> titan# zpool set cachefile="/boot/zfs/zpool.cache" proteus >> titan# >> titan# ls -l /boot/zfs/zpool.cache >> -rw-r--r-- 1 root wheel 4960 Nov 28 14:28 /boot/zfs/zpool.cache >> titan# >> titan# zpool get cachefile t0 >> NAME PROPERTY VALUE SOURCE >> t0 cachefile /boot/zfs/zpool.cache local >> titan# >> titan# zpool get cachefile leaf >> NAME PROPERTY VALUE SOURCE >> leaf cachefile /boot/zfs/zpool.cache local >> titan# >> titan# zpool get cachefile proteus >> NAME PROPERTY VALUE SOURCE >> proteus cachefile /boot/zfs/zpool.cache local >> titan# >> >> titan# >> titan# reboot >> Nov 28 14:34:05 Waiting (max 60 seconds) for system process `vnlru' to >> stop... done >> Waiting (max 60 seconds) for system process `syncer' to stop... >> Syncing disks, vnodes remaining... 0 0 0 0 0 0 done >> All buffers synced. >> Uptime: 2h38m57s >> GEOM_MIRROR: Device swap: provider destroyed. >> GEOM_MIRROR: Device swap destroyed. >> uhub5: detached >> uhub1: detached >> uhub4: detached >> uhub2: detached >> uhub3: detached >> uhub6: detached >> uhub0: detached >> ix0: link state changed to DOWN >> . >> . >> . >> >> Starting iscsid. >> Starting iscsictl. >> Clearing /tmp. >> Updating /var/run/os-release done. >> Updating motd:. >> Creating and/or trimming log files. >> Starting syslogd. >> No core dumps found. >> Starting local daemons:failed to open cache file: No such file or >> directory >> . >> Starting ntpd. >> Starting powerd. >> Mounting late filesystems:. >> Starting cron. >> Performing sanity check on sshd configuration. >> Starting sshd. >> Starting background file system >> FreeBSD/amd64 (titan) (ttyu0) >> >> login: root >> Password: >> Nov 28 14:36:29 titan login[4162]: ROOT LOGIN (root) ON ttyu0 >> Last login: Thu Nov 28 14:33:45 on ttyu0 >> FreeBSD 15.0-CURRENT (GENERIC-NODEBUG) #1 >> main-n273749-4b65481ac68a-dirty: Wed Nov 20 15:08:52 GMT 2024 >> >> Welcome to FreeBSD! >> >> Release Notes, Errata: https://www.FreeBSD.org/releases/ >> Security Advisories: https://www.FreeBSD.org/security/ >> FreeBSD Handbook: https://www.FreeBSD.org/handbook/ >> FreeBSD FAQ: https://www.FreeBSD.org/faq/ >> Questions List: https://www.FreeBSD.org/lists/questions/ >> FreeBSD Forums: https://forums.FreeBSD.org/ >> >> Documents installed with the system are in the >> /usr/local/share/doc/freebsd/ >> directory, or can be installed later with: pkg install en-freebsd-doc >> For other languages, replace "en" with a language code like de or fr. >> >> Show the version of FreeBSD installed: freebsd-version ; uname -a >> Please include that output and any error messages when posting questions. >> Introduction to manual pages: man man >> FreeBSD directory layout: man hier >> >> To change this login announcement, see motd(5). >> You have new mail. >> titan# >> titan# zpool list >> NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP >> HEALTH ALTROOT >> leaf 18.2T 984K 18.2T - - 0% 0% 1.00x >> ONLINE - >> proteus 1.98T 361G 1.63T - - 1% 17% 1.00x >> ONLINE - >> t0 444G 91.2G 353G - - 27% 20% 1.00x >> ONLINE - >> titan# >> >> This is progress ... however the cachefile property is wiped out again : >> >> titan# zpool get cachefile t0 >> NAME PROPERTY VALUE SOURCE >> t0 cachefile - default >> titan# zpool get cachefile leaf >> NAME PROPERTY VALUE SOURCE >> leaf cachefile - default >> titan# zpool get cachefile proteus >> NAME PROPERTY VALUE SOURCE >> proteus cachefile - default >> titan# >> >> Also, strangely, none of the filesystem in proteus are mounted : >> >> titan# >> titan# zfs list -o name,exec,checksum,canmount,mounted,mountpoint -r >> proteus >> NAME EXEC CHECKSUM CANMOUNT MOUNTED MOUNTPOINT >> proteus on sha512 on no none >> proteus/bhyve off sha512 on no /bhyve >> proteus/bhyve/disk off sha512 on no /bhyve/disk >> proteus/bhyve/isos off sha512 on no /bhyve/isos >> proteus/obj on sha512 on no /usr/obj >> proteus/src on sha512 on no /usr/src >> titan# >> >> If I reboot again without doing anything will the zpools re-appear ? >> >> >> titan# >> titan# Nov 28 14:37:08 titan su[4199]: admsys to root on /dev/pts/0 >> >> titan# reboot >> Nov 28 14:40:29 Waiting (max 60 seconds) for system process `vnlru' to >> stop... done >> Waiting (max 60 seconds) for system process `syncer' to stop... >> Syncing disks, vnodes remaining... 0 0 0 0 0 done >> All buffers synced. >> Uptime: 4m50s >> GEOM_MIRROR: Device swap: provider destroyed. >> GEOM_MIRROR: Device swap destroyed. >> uhub4: detached >> uhub1: detached >> uhub5: detached >> uhub0: detached >> uhub3: detached >> uhub6: detached >> uhub2: detached >> ix0: link state changed to DOWN >> . >> . >> . >> Starting iscsid. >> Starting iscsictl. >> Clearing /tmp. >> Updating /var/run/os-release done. >> Updating motd:. >> Creating and/or trimming log files. >> Starting syslogd. >> No core dumps found. >> Starting local daemons:failed to open cache file: No such file or >> directory >> . >> Starting ntpd. >> Starting powerd. >> Mounting late filesystems:. >> Starting cron. >> Performing sanity check on sshd configuration. >> Starting sshd. >> Starting background file system >> FreeBSD/amd64 (titan) (ttyu0) >> >> login: root >> Password: >> Nov 28 14:43:01 titan login[4146]: ROOT LOGIN (root) ON ttyu0 >> Last login: Thu Nov 28 14:36:29 on ttyu0 >> FreeBSD 15.0-CURRENT (GENERIC-NODEBUG) #1 >> main-n273749-4b65481ac68a-dirty: Wed Nov 20 15:08:52 GMT 2024 >> >> Welcome to FreeBSD! >> >> Release Notes, Errata: https://www.FreeBSD.org/releases/ >> Security Advisories: https://www.FreeBSD.org/security/ >> FreeBSD Handbook: https://www.FreeBSD.org/handbook/ >> FreeBSD FAQ: https://www.FreeBSD.org/faq/ >> Questions List: https://www.FreeBSD.org/lists/questions/ >> FreeBSD Forums: https://forums.FreeBSD.org/ >> >> Documents installed with the system are in the >> /usr/local/share/doc/freebsd/ >> directory, or can be installed later with: pkg install en-freebsd-doc >> For other languages, replace "en" with a language code like de or fr. >> >> Show the version of FreeBSD installed: freebsd-version ; uname -a >> Please include that output and any error messages when posting questions. >> Introduction to manual pages: man man >> FreeBSD directory layout: man hier >> >> To change this login announcement, see motd(5). >> You have new mail. >> titan# >> titan# zpool list >> NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP >> HEALTH ALTROOT >> leaf 18.2T 1.01M 18.2T - - 0% 0% 1.00x >> ONLINE - >> proteus 1.98T 361G 1.63T - - 1% 17% 1.00x >> ONLINE - >> t0 444G 91.2G 353G - - 27% 20% 1.00x >> ONLINE - >> titan# >> titan# zfs list -o name,exec,checksum,canmount,mounted,mountpoint -r >> proteus >> NAME EXEC CHECKSUM CANMOUNT MOUNTED MOUNTPOINT >> proteus on sha512 on no none >> proteus/bhyve off sha512 on no /bhyve >> proteus/bhyve/disk off sha512 on no /bhyve/disk >> proteus/bhyve/isos off sha512 on no /bhyve/isos >> proteus/obj on sha512 on no /usr/obj >> proteus/src on sha512 on no /usr/src >> titan# >> >> OKay so the zpools appear to be back in spite of the strange situation >> with the cachefile property is empty everywhere. My guess is the zpool >> rc script is bring in information during early boot. >> >> Why the zfs filesystems on proteus do not mount? Well that is a >> strange problem but at least the zpool can be used. >> >> -- >> -- >> Dennis Clarke >> RISC-V/SPARC/PPC/ARM/CISC >> UNIX and Linux spoken >> >> >> >> >> > > > Hi, > > The output you provide contains this line: > "Starting local daemons:failed to open cache file: No such file or > directory" > > Where does that output come from? What is in your file /etc/rc.local file? > > Regards, > Ronald. > Ah ha ! I should really do better documentation on this machines config. Sure enough there is something there to handle the iSCSI based zpool : titan# ls -la /etc/rc.local -rw-r--r-- 1 root wheel 92 Mar 12 2024 /etc/rc.local titan# titan# cat /etc/rc.local zpool import -a -c /var/cache/iscsi-zpools.cache -o cachefile=/var/cache/iscsi-zpools.cache This seems familiar because the iSCSI based zpool would not be available at boot time. At some point in the past, late 2023 I think, I was trying to get the iSCSI services working and I saw that iSCSI device(s) were not available after boot. It took a bit of wrangling to get that working in an order where at least I can see the zpool and then import it. titan# zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT leaf 18.2T 267G 17.9T - - 0% 1% 1.00x ONLINE - proteus 1.98T 365G 1.63T - - 1% 17% 1.00x ONLINE - t0 444G 152G 292G - - 31% 34% 1.00x ONLINE - titan# zpool get cachefile proteus NAME PROPERTY VALUE SOURCE proteus cachefile - default titan# ls /var/cache/iscsi-zpools.cache ls: /var/cache/iscsi-zpools.cache: No such file or directory Somehow that cache file vanished and I suspect it was when I moved around ccache location also. A mistake on my part. I wanted the ccache location to not have sync=standard and so it made some scary sense to set sync=disable. To me that is a scary idea. However myself and some others felt that it was okay for the ccache location. Therefore I made a new zfs filesystem on the local NVME boot device just for cache operations and then /var/cache/iscsi-zpools.cache must have been lost in the shuffle around. titan# zpool set cachefile=/var/cache/iscsi-zpools.cache proteus titan# ls -l /var/cache/iscsi-zpools.cache -rw-r--r-- 1 root wheel 1440 Nov 29 18:31 /var/cache/iscsi-zpools.cache Perhaps now, at next reboot, all the zpools will exist and the zfs filesystems on the iSCSI based storage will exist and be mounted as required. At the moment the machine has a large poudriere bulk build running and likely will be busy half of today. Thank you for the excellent catch there! -- -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken From nobody Fri Nov 29 19:38:36 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Y0NlZ4Kvsz5fcQT for ; Fri, 29 Nov 2024 19:38:50 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-pl1-x629.google.com (mail-pl1-x629.google.com [IPv6:2607:f8b0:4864:20::629]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Y0NlY4pFRz4sBV for ; Fri, 29 Nov 2024 19:38:49 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20230601.gappssmtp.com header.s=20230601 header.b=Z3k62+Wm; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::629) smtp.mailfrom=wlosh@bsdimp.com; dmarc=none Received: by mail-pl1-x629.google.com with SMTP id d9443c01a7336-21207f0d949so20703365ad.2 for ; Fri, 29 Nov 2024 11:38:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1732909128; x=1733513928; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=o/1P5yDXsulP5MSWHHrHqKjLo61Kpd/sFVyuY1AJUs8=; b=Z3k62+Wm6ELEPTyPU1QJECeSe6YmAr+6VaBuEfohWDl96Au0TSfYcFfcs8/Y7+sLPW RN4VWO4X1e8yH0ODEY6J5oEbNFWNpin9Cdll/ZtWYZ9pK4aKTFOcSn4+Cxg5sZhn6ZOV EBWPjtUPYYCcSD/FRk+4tpn8YGPYIBVXXwCYy0+T8f8Y5ScqdJEhDrzle5rS880Hh/Lj mt34ASRZZfU9/B1TCE+BvNgS7fdnIKJ0E04G291xcc4QGSiHviBDDe+mjpONV4wtHVKi trqHbeCSA3ql8DohaqILExOeUwSjmlexcxyDdrET0we7tnA86jroos8y6IpEh6St0v/E QfvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732909128; x=1733513928; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=o/1P5yDXsulP5MSWHHrHqKjLo61Kpd/sFVyuY1AJUs8=; b=L92UEXLteb1ZzzqijuGjKGCgcw98zSqLeILCfKDExkAletRf2VvGBmBpAW4yfcb38h gP1oe0iS4+0MpOcHeNy4TtIG1+wiC6ECH2OMP0T7X/ZoUH7v0gVmoI0d0CwusZj5rAVd 7aZ21I2o3zojPbGR3ItEsfzXWsOPsMC64A9FsKrZT2Z1yH8w0GEr9Hz7D+gVfbBqJG/X x6YuebgghyOqDSkRlOeKbvMzXbGKz0j6h0AGzXapzM2JXj8Mxi6Oche7oy/8DDyjFzk8 Tk0jDdy6Tu48u2zIJ+PTslaj6ZC1l9IbycEjA8GuFJDqIR9S3jeNTphP1v4UYGndv7vv u7CA== X-Gm-Message-State: AOJu0YyZMSk63Ed5uJ5/Ep13JEQAghtd5IE7gzuNzjFyp8qhdc8ZKRwm Uhrs24jTNyyPIf4GeIWdbRdtyOyqSch7tmmuZVq+DSbDa+8EMNIIzmQFa8cg3gU12Fay1nhBBj+ pV8Cv/1TkGzOcPu/LvL7vXgzVuGla4MPRI3WSFQ== X-Gm-Gg: ASbGncsVd41V6S+AzlZzL3X8VVF+Dy22gXQ0z/rTl4KjeT7/9Vg59o+c1VJ92AaB32t SriWYFr8QdGckcaIH8KjzzZ6Vf2OIhMw= X-Google-Smtp-Source: AGHT+IH99shIBuYnK0YgFwanE6pDvHCQzObR7/EYGy28+uNM36pV+09RYySdiOfWXcQghhjAQX769FaGNk3VuY7AeUo= X-Received: by 2002:a17:902:e810:b0:212:5d38:b45f with SMTP id d9443c01a7336-21501085456mr164379975ad.8.1732909128259; Fri, 29 Nov 2024 11:38:48 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Fri, 29 Nov 2024 12:38:36 -0700 Message-ID: Subject: Re: Long time outdated jemalloc To: cglogic Cc: FreeBSD CURRENT , Minsoo Choo Content-Type: multipart/alternative; boundary="00000000000004b6df06281259ae" X-Spamd-Result: default: False [-3.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20230601.gappssmtp.com:s=20230601]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_ONE(0.00)[1]; R_SPF_NA(0.00)[no SPF record]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ARC_NA(0.00)[]; MISSING_XM_UA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::629:from]; DMARC_NA(0.00)[bsdimp.com]; FREEMAIL_TO(0.00)[protonmail.com]; RCVD_TLS_LAST(0.00)[]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; DKIM_TRACE(0.00)[bsdimp-com.20230601.gappssmtp.com:+] X-Rspamd-Queue-Id: 4Y0NlY4pFRz4sBV X-Spamd-Bar: -- --00000000000004b6df06281259ae Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I've been swamped. we need to bootstrap the vendor branch, and the way prior updates were done isn't so great. Warner On Mon, Nov 25, 2024 at 2:21=E2=80=AFAM cglogic wr= ote: > Hello guys, > > How the update of jemalloc is going? It's November now. > > Thanks. > On Monday, July 22nd, 2024 at 7:02 PM, Minsoo Choo < > minsoochoo0122@proton.me> wrote: > > First, sorry for late response. > > cglogic, thank you for bringing up this issue again since I nearly forgot > that this issue was still open. > > Warner, as I can't access to my FreeBSD instance until the end of August, > but I can still edit and push the code through my Arm Mac. This means tha= t > I can't test the updated code on my machine, but I can join the review > process and listen to change proposals. > > I'll open a Github PR in a few hours. (The phabricator review will stay > opened just in case) > On Monday, July 22nd, 2024 at 5:08 AM, Warner Losh wrote= : > > > > On Sun, Jul 21, 2024 at 2:03=E2=80=AFPM cglogic = wrote: > >> >> On Sunday, July 21st, 2024 at 6:54 AM, Warner Losh >> wrote: >> >> >> >> On Sat, Jul 20, 2024 at 1:59=E2=80=AFAM cglogic = wrote: >> >>> Hello FreeBSD community, >>> >>> After Jason Evans stepped aside from maintaining jemalloc in FreeBSD, >>> it's not updating in time anymore. >>> Version 5.3.0 was released May 6, 2022 and FreeBSD still not imported >>> it into the tree. >>> >>> There is a pending review https://reviews.freebsd.org/D41421 from Aug >>> 11, 2023. >>> I'm successfully running FreeBSD/amd64 system with D41421 applied for 8 >>> months, as well as many other people. >>> >>> Can it be reviewed and committed to CURRENT? >>> Or, if there is no committers willing to do it, can commit bit be given >>> to submitter or another person willing to do this? >>> >>> It's very disappointing when users spend their time to fill such gaps >>> and their efforts just ignored by the developers. >>> Every year FreeBSD Community Survey asking about user experience in >>> contributing to FreeBSD. >>> Here you can see an example of such contributing. >>> >>> >> First, thank you for being persistent and continuing to bring it up. It'= s >> important to do that to make sure this (and your many other) contributio= n >> doesn't fall on the floor. >> >> And to be fair, we're only 3 months since the last update. Still, quite = a >> bit longer than you should have to wait, but not nearly the year the >> original date suggests. >> >> And this is a perfect storm of "how the project is bad at accepting >> contributions": >> (1) The original submission was close to the 14 branch creation time. >> This meant that we weren't well prepared to look at it since it is such = an >> invasive change (at least on its surface). It also slowed the initial >> response... >> (2) There was a number of back and forth requests for changes, which too= k >> time to sort out... >> (3) The size of this is huge, well beyond the capacity of Phabricator to >> review accurately... >> (4) It's a vendor import. That means we can't just drop the Phabricator >> review into the tree... >> (5) It's phabricator: this is a great tool for developers, but we have a >> terrible track record of using it for intake from new contributors. We >> don't have any oversight at all over this tool, at there's at best tepid >> and luke warm attempts to look for drop balls. >> >> All of these things are a terrible experience. I can only apologize. >> These days, we might steer this towards github, but the 'vendor import' >> means you really need someone on the inside, or you need to be on the >> inside to make that work. >> >> So, how to move forward? Well, I'd like to propose the following: >> (1) submit all the other Phabricator reviews you have open (they are >> mostly good, or close to good) to github. Github is being actively manag= ed >> and will make it faster to get things it. It's a much better tool for ne= w >> contributors (and even frequent contributors of smallish things). >> (2) I should do an vendor import of 5.3.0 from github, and do the merge >> to a branch and push that to github. You can then layer on your changes = and >> those can be reviewed more closely as a pull request against the branch = I >> push. I suspect that most of the issues are sorted out already >> (3) I'll land it via that route... >> >> And, if the sum of the other pull requests and this are good (and I >> suspect they will be), then we can talk about commit bits and such. >> >> It's experiences like this which is why I'm trying to stand up github >> pull requests as a reliable way to get things and and the best place to >> send people... >> >> Thanks again for persisting, and also for expressing this criticism that >> we (hopefully) can use to make it better. >> >> Warner >> >> >> Hello. >> >> I'm not the author of D41421. Just applied the patch to test it 8 months >> ago. And recently discovered that it's still not committed. >> I can't copy your message to Phabricator because don't have an account. = Please, >> if you have time, help the author in D41421. >> > > Ah yes. I've been in touch with the author for other things, and somehow > thought it was you.... I'll reach out to him via other means... > > Warner > > > > --00000000000004b6df06281259ae Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I've been swamped. we need to bootstrap the vendor bra= nch, and the way prior updates were done
isn't so great.=C2=A0

Warner

On Mon, Nov 25, 2024= at 2:21=E2=80=AFAM cglogic <c= glogic@protonmail.com> wrote:
Hello guys,

= How the update of jemalloc is going? It's November now.

Thanks.
I'll open a Github P= R in a few hours. (The phabricator review will stay opened just in case)
On Monday, July 22nd, 2024 at 5:08 AM, Warner Losh <imp@bsdimp.com> wrote:


On Sun, Jul 21, 2024 at 2= :03=E2=80=AFPM cglogic <cglogic@protonmail.com= > wrote:

On Sunday, July 21st, 2024 at 6:54 AM, Warner Losh <imp@bsdimp.com> wrote:


On Sat, Jul 20, 2024 at 1= :59=E2=80=AFAM cglogic <cglogic@protonmail.com= > wrote:
Hello FreeBSD community,

After Jason Evans stepped aside from = maintaining jemalloc in FreeBSD, it's not updating in time anymore.
Version 5.3.0 was released = May 6, 2022 and FreeBSD still not imported it into the tree.

There is a pending review https://reviews.freebsd.org/D41421 from Aug 11, 2023.
I'm succes= sfully running FreeBSD/amd64 system with D41421 applied for 8 months, as we= ll as many other people.

= Can it be reviewed and committed to CURRENT?
Or, if there is no committe= rs willing to do it, can commit bit be given to submitter or another person= willing to do this?
=
It's very disappointing when users spend their time to fi= ll such gaps and their efforts just ignored by the developers.
Every year FreeBSD Community Survey asking about user experience in contr= ibuting to FreeBSD.
Here you can see an example of su= ch contributing.


Firs= t, thank you for being persistent and continuing to bring it up. It's i= mportant to do that to make sure this (and your many other) contribution do= esn't fall on the floor.

And to be fair, w= e're only 3 months since the last update. Still, quite a bit longer tha= n you should have to wait, but not nearly the year the original date sugges= ts.

And this is a perfect storm of "how t= he project is bad at accepting contributions":
(1) The origi= nal submission was close to the 14 branch creation time. This meant that we= weren't well prepared to look at it since it is such an invasive chang= e (at least on its surface). It also slowed the initial response...
(2) There was a number of back and forth requests for changes, which= took time to sort out...
(3) The size of this is huge, well beyo= nd the capacity of Phabricator to review accurately...
(4) It'= ;s a vendor import. That means we can't just drop the Phabricator revie= w into the tree...
(5) It's phabricator: this is a great tool= for developers, but we have a terrible track record of using it for intake= from new contributors. We don't have any oversight at all over this to= ol, at there's at best tepid and luke warm attempts to look for drop ba= lls.

All of these things are a terrible experience= . I can only apologize. These days, we might steer this towards github, but= the 'vendor import' means you really need someone on the inside, o= r you need to be on the inside to make that work.

= So, how to move forward? Well, I'd like to propose the following:
=
(1) submit all the other Phabricator reviews you have open (they are m= ostly good, or close to good) to github. Github is being actively managed a= nd will make it faster to get things it. It's a much better tool for ne= w contributors (and even frequent contributors of smallish things).
(2) I should do an vendor import of 5.3.0 from github, and do the merge = to a branch and push that to github. You can then layer on your changes and= those can be reviewed more closely as a pull request against the branch I = push. I suspect that most of the issues are sorted out already
(3) I'll land it via that route...

And, if = the sum of the other pull requests and this are good (and I suspect they wi= ll be), then we can talk about commit bits and such.

It's experiences like this which is why I'm trying to stand up g= ithub pull requests as a reliable way to get things and and the best place = to send people...

Thanks again for persistin= g, and also for expressing this criticism that we (hopefully) can use to ma= ke it better.

Warner

Hello.

I'm not the author of = D41421. Just applied the patch to test it 8 months ago. And recently = discovered that it's still not committed.
I can't copy your message to Phabricator becaus= e don't have an account. Please, if you have time, help the auth= or in D41421.

Ah yes. I've been i= n touch with the author for other things, and somehow thought it was you...= . I'll reach out to him via other means...

Wa= rner


--00000000000004b6df06281259ae-- From nobody Sat Nov 30 10:17:00 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Y0mF04ChHz5fT3S for ; Sat, 30 Nov 2024 10:17:08 +0000 (UTC) (envelope-from freebsd@walstatt-de.de) Received: from smtp052.goneo.de (smtp052.goneo.de [85.220.129.60]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Y0mDz1xPrz468b for ; Sat, 30 Nov 2024 10:17:07 +0000 (UTC) (envelope-from freebsd@walstatt-de.de) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=walstatt-de.de header.s=DKIM001 header.b=FAFEMQNk; spf=pass (mx1.freebsd.org: domain of freebsd@walstatt-de.de designates 85.220.129.60 as permitted sender) smtp.mailfrom=freebsd@walstatt-de.de; dmarc=none Received: from hub1.goneo.de (hub1.goneo.de [85.220.129.52]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by smtp5.goneo.de (Postfix) with ESMTPS id 48A78240EC0 for ; Sat, 30 Nov 2024 11:17:05 +0100 (CET) Received: from hub1.goneo.de (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by hub1.goneo.de (Postfix) with ESMTPS id AF8D6240562 for ; Sat, 30 Nov 2024 11:17:03 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=walstatt-de.de; s=DKIM001; t=1732961823; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=BMKQB3qp23En12NffcAAE5mOluNJ6EeqihBzDfS7QZM=; b=FAFEMQNkbaABqAjzFtHNjiO/59CKVCL6n4f6gd2Plwktp7H/IcYb9NbxfDg9M+/2+nsMvt AeVqx5l6i1xv1GM4ZX/dxScF6jxgGyUgm3WVXoSOfhybU7Mz1qtyRXK5m6zWsiNJ+HM/Pm XZ+acQ3XnPI3vPRsTlvUuUCxEj1dHCYa6kcd7CYuT0KGc0UuIZEFI6RBccPvypaq9n8B4z eI1ea73vlN8S3PATOY3pRkysFym1HmebwdpZXC7DUPKyIwuc8ZBwlfZVmZNIV5QwXT6PUD fLD8MWizTxcInU3oZbszshdit4vlgRcLevsPfsOdZRuAFpHGnfC9mPL8S84YxA== Received: from hermann.intern.walstatt.dynvpn.de (dynamic-078-054-002-023.78.54.pool.telefonica.de [78.54.2.23]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (prime256v1) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by hub1.goneo.de (Postfix) with ESMTPSA id 7D64B2402AC for ; Sat, 30 Nov 2024 11:17:03 +0100 (CET) Date: Sat, 30 Nov 2024 11:17:00 +0100 From: FreeBSD User To: FreeBSD CURRENT Subject: CURRENT: error: RPC failed: curl 56 recv failure: Operation timed out Message-ID: <20241130111700.1489b7b1@hermann.intern.walstatt.dynvpn.de> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-UID: 2ecf80 X-Rspamd-UID: 3437a9 X-Spamd-Result: default: False [-3.49 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.990]; R_SPF_ALLOW(-0.20)[+ip4:85.220.129.0/25]; R_DKIM_ALLOW(-0.20)[walstatt-de.de:s=DKIM001]; MIME_GOOD(-0.10)[text/plain]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:25394, ipnet:85.220.128.0/17, country:DE]; MIME_TRACE(0.00)[0:+]; MISSING_XM_UA(0.00)[]; ARC_NA(0.00)[]; RCVD_TLS_ALL(0.00)[]; DMARC_NA(0.00)[walstatt-de.de]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[85.220.129.60:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[walstatt-de.de:+] X-Rspamd-Queue-Id: 4Y0mDz1xPrz468b X-Spamd-Bar: --- Updating CURRENT (FreeBSD 15.0-CURRENT #61 main-n273942-e8d027be6b84: Sat Nov 30 01:24:07 CET 2024 amd64) left the system in a "starnge" state: most applications accessing the net (firefox, librewold, claws-mail) do have access problems - mostly timeout errors. git -C /usr/src pull fails after a while with error: RPC failed: curl 56 recv failure: Operation timed out fatal: exprected 'acknowledgements' (git remote -v: origin htps://git.freebsd.org/src.git ...) I do not have any problems accessing/pulling ports repository via git. I do not have ANY problem access git repos on an 1 week older CURRENT or FreeBSD 14-STABLE (recently updated, FreeBSD 14.2-STABLE #19 stable/14-n269714-57a7f61b13c3: Sat Nov 30 10:20:15 CET 2024 amd64). Is there anything to be aware of? Thanks in advance, oh From nobody Sat Nov 30 18:42:55 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Y0zSk1NcHz5fSrN for ; Sat, 30 Nov 2024 18:43:02 +0000 (UTC) (envelope-from cglogic@protonmail.com) Received: from mail-40130.protonmail.ch (mail-40130.protonmail.ch [185.70.40.130]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Y0zSj4w6Lz49rh for ; Sat, 30 Nov 2024 18:43:01 +0000 (UTC) (envelope-from cglogic@protonmail.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail3; t=1732992178; x=1733251378; bh=SrlKp3qr+FEvpx4Lwykbi7wFe4vlcAdddz58dZ9Vnbc=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector:List-Unsubscribe:List-Unsubscribe-Post; b=merkA7SmqhszEcA9jzxSaMbDhl6znAgWtZys3qd8vGWVZrdZHe5N4Ve26CaKbFJfH 5L/NGDZsur2d2PUAOG+1BoFcvYQHB74LyfJ9D4i/gNj13VJE/Y136aVUbm6LU+9Tkm nHvKenXacsWQo3C6wqXQ0WbBcsQOrOLOUvZxFh3j7fyx0sdkPB/4yysq6MyARtv/nS Y1LhHVvXG7//VIdO3AShxe+XPN0Qwu98eion67CRmblMbcNw79gaaJjbafrTMP/6qW FV84oMBf8R1ZJ/pp38z0BJemopRrrpneYmz28FsNdLUUpnz2JsXryQnaCL2TnTUuEm IlPMRBFXtV97g== Date: Sat, 30 Nov 2024 18:42:55 +0000 To: Warner Losh From: cglogic Cc: FreeBSD CURRENT , Minsoo Choo Subject: Re: Long time outdated jemalloc Message-ID: In-Reply-To: References: Feedback-ID: 25313618:user:proton X-Pm-Message-ID: b81497804d3f89066edd11bed9a9617cb84c5d8a List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="b1=_VrKTCxXWptU2LwIb5XlmprDDuAaVwa6prnu2JD7IM" X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH] X-Rspamd-Queue-Id: 4Y0zSj4w6Lz49rh X-Spamd-Bar: ---- --b1=_VrKTCxXWptU2LwIb5XlmprDDuAaVwa6prnu2JD7IM Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 SSBzZWUsIGl0IGhhcHBlbnMuCk1heWJlIGFub3RoZXIgY29tbWl0dGVyIHdpbGwgdm9sdW50ZWVy IHRvIGRvIHRoZSB1cGRhdGUuCkkgaG9wZSBpdCB3aWxsIG1ha2UgaXRzIHdheSBpbnRvIDE1LjAg cmVsZWFzZS4KClRoYW5rcy4KT24gRnJpZGF5LCBOb3ZlbWJlciAyOXRoLCAyMDI0IGF0IDk6Mzgg UE0sIFdhcm5lciBMb3NoIDxpbXBAYnNkaW1wLmNvbT4gd3JvdGU6Cgo+IEkndmUgYmVlbiBzd2Ft cGVkLiB3ZSBuZWVkIHRvIGJvb3RzdHJhcCB0aGUgdmVuZG9yIGJyYW5jaCwgYW5kIHRoZSB3YXkg cHJpb3IgdXBkYXRlcyB3ZXJlIGRvbmUKPiBpc24ndCBzbyBncmVhdC4KPgo+IFdhcm5lcgo+Cj4g T24gTW9uLCBOb3YgMjUsIDIwMjQgYXQgMjoyMeKAr0FNIGNnbG9naWMgPGNnbG9naWNAcHJvdG9u bWFpbC5jb20+IHdyb3RlOgo+Cj4+IEhlbGxvIGd1eXMsCj4+Cj4+IEhvdyB0aGUgdXBkYXRlIG9m IGplbWFsbG9jIGlzIGdvaW5nPyBJdCdzIE5vdmVtYmVyIG5vdy4KPj4KPj4gVGhhbmtzLgo+PiBP biBNb25kYXksIEp1bHkgMjJuZCwgMjAyNCBhdCA3OjAyIFBNLCBNaW5zb28gQ2hvbyA8bWluc29v Y2hvbzAxMjJAcHJvdG9uLm1lPiB3cm90ZToKPj4KPj4+IEZpcnN0LCBzb3JyeSBmb3IgbGF0ZSBy ZXNwb25zZS4KPj4+Cj4+PiBjZ2xvZ2ljLCB0aGFuayB5b3UgZm9yIGJyaW5naW5nIHVwIHRoaXMg aXNzdWUgYWdhaW4gc2luY2UgSSBuZWFybHkgZm9yZ290IHRoYXQgdGhpcyBpc3N1ZSB3YXMgc3Rp bGwgb3Blbi4KPj4+Cj4+PiBXYXJuZXIsIGFzIEkgY2FuJ3QgYWNjZXNzIHRvIG15IEZyZWVCU0Qg aW5zdGFuY2UgdW50aWwgdGhlIGVuZCBvZiBBdWd1c3QsIGJ1dCBJIGNhbiBzdGlsbCBlZGl0IGFu ZCBwdXNoIHRoZSBjb2RlIHRocm91Z2ggbXkgQXJtIE1hYy4gVGhpcyBtZWFucyB0aGF0IEkgY2Fu J3QgdGVzdCB0aGUgdXBkYXRlZCBjb2RlIG9uIG15IG1hY2hpbmUsIGJ1dCBJIGNhbiBqb2luIHRo ZSByZXZpZXcgcHJvY2VzcyBhbmQgbGlzdGVuIHRvIGNoYW5nZSBwcm9wb3NhbHMuCj4+Pgo+Pj4g SSdsbCBvcGVuIGEgR2l0aHViIFBSIGluIGEgZmV3IGhvdXJzLiAoVGhlIHBoYWJyaWNhdG9yIHJl dmlldyB3aWxsIHN0YXkgb3BlbmVkIGp1c3QgaW4gY2FzZSkKPj4+IE9uIE1vbmRheSwgSnVseSAy Mm5kLCAyMDI0IGF0IDU6MDggQU0sIFdhcm5lciBMb3NoIDxpbXBAYnNkaW1wLmNvbT4gd3JvdGU6 Cj4+Pgo+Pj4+IE9uIFN1biwgSnVsIDIxLCAyMDI0IGF0IDI6MDPigK9QTSBjZ2xvZ2ljIDxjZ2xv Z2ljQHByb3Rvbm1haWwuY29tPiB3cm90ZToKPj4+Pgo+Pj4+PiBPbiBTdW5kYXksIEp1bHkgMjFz dCwgMjAyNCBhdCA2OjU0IEFNLCBXYXJuZXIgTG9zaCA8aW1wQGJzZGltcC5jb20+IHdyb3RlOgo+ Pj4+Pgo+Pj4+Pj4gT24gU2F0LCBKdWwgMjAsIDIwMjQgYXQgMTo1OeKAr0FNIGNnbG9naWMgPGNn bG9naWNAcHJvdG9ubWFpbC5jb20+IHdyb3RlOgo+Pj4+Pj4KPj4+Pj4+PiBIZWxsbyBGcmVlQlNE IGNvbW11bml0eSwKPj4+Pj4+Pgo+Pj4+Pj4+IEFmdGVyIEphc29uIEV2YW5zIHN0ZXBwZWQgYXNp ZGUgZnJvbSBtYWludGFpbmluZyBqZW1hbGxvYyBpbiBGcmVlQlNELCBpdCdzIG5vdCB1cGRhdGlu ZyBpbiB0aW1lIGFueW1vcmUuCj4+Pj4+Pj4gVmVyc2lvbiA1LjMuMCB3YXMgcmVsZWFzZWQgTWF5 IDYsIDIwMjIgYW5kIEZyZWVCU0Qgc3RpbGwgbm90IGltcG9ydGVkIGl0IGludG8gdGhlIHRyZWUu Cj4+Pj4+Pj4KPj4+Pj4+PiBUaGVyZSBpcyBhIHBlbmRpbmcgcmV2aWV3IGh0dHBzOi8vcmV2aWV3 cy5mcmVlYnNkLm9yZy9ENDE0MjEgZnJvbSBBdWcgMTEsIDIwMjMuCj4+Pj4+Pj4gSSdtIHN1Y2Nl c3NmdWxseSBydW5uaW5nIEZyZWVCU0QvYW1kNjQgc3lzdGVtIHdpdGggRDQxNDIxIGFwcGxpZWQg Zm9yIDggbW9udGhzLCBhcyB3ZWxsIGFzIG1hbnkgb3RoZXIgcGVvcGxlLgo+Pj4+Pj4+Cj4+Pj4+ Pj4gQ2FuIGl0IGJlIHJldmlld2VkIGFuZCBjb21taXR0ZWQgdG8gQ1VSUkVOVD8KPj4+Pj4+PiBP ciwgaWYgdGhlcmUgaXMgbm8gY29tbWl0dGVycyB3aWxsaW5nIHRvIGRvIGl0LCBjYW4gY29tbWl0 IGJpdCBiZSBnaXZlbiB0byBzdWJtaXR0ZXIgb3IgYW5vdGhlciBwZXJzb24gd2lsbGluZyB0byBk byB0aGlzPwo+Pj4+Pj4+Cj4+Pj4+Pj4gSXQncyB2ZXJ5IGRpc2FwcG9pbnRpbmcgd2hlbiB1c2Vy cyBzcGVuZCB0aGVpciB0aW1lIHRvIGZpbGwgc3VjaCBnYXBzIGFuZCB0aGVpciBlZmZvcnRzIGp1 c3QgaWdub3JlZCBieSB0aGUgZGV2ZWxvcGVycy4KPj4+Pj4+Pgo+Pj4+Pj4+IEV2ZXJ5IHllYXIg RnJlZUJTRCBDb21tdW5pdHkgU3VydmV5IGFza2luZyBhYm91dCB1c2VyIGV4cGVyaWVuY2UgaW4g Y29udHJpYnV0aW5nIHRvIEZyZWVCU0QuCj4+Pj4+Pj4KPj4+Pj4+PiBIZXJlIHlvdSBjYW4gc2Vl IGFuIGV4YW1wbGUgb2Ygc3VjaCBjb250cmlidXRpbmcuCj4+Pj4+Pgo+Pj4+Pj4gRmlyc3QsIHRo YW5rIHlvdSBmb3IgYmVpbmcgcGVyc2lzdGVudCBhbmQgY29udGludWluZyB0byBicmluZyBpdCB1 cC4gSXQncyBpbXBvcnRhbnQgdG8gZG8gdGhhdCB0byBtYWtlIHN1cmUgdGhpcyAoYW5kIHlvdXIg bWFueSBvdGhlcikgY29udHJpYnV0aW9uIGRvZXNuJ3QgZmFsbCBvbiB0aGUgZmxvb3IuCj4+Pj4+ Pgo+Pj4+Pj4gQW5kIHRvIGJlIGZhaXIsIHdlJ3JlIG9ubHkgMyBtb250aHMgc2luY2UgdGhlIGxh c3QgdXBkYXRlLiBTdGlsbCwgcXVpdGUgYSBiaXQgbG9uZ2VyIHRoYW4geW91IHNob3VsZCBoYXZl IHRvIHdhaXQsIGJ1dCBub3QgbmVhcmx5IHRoZSB5ZWFyIHRoZSBvcmlnaW5hbCBkYXRlIHN1Z2dl c3RzLgo+Pj4+Pj4KPj4+Pj4+IEFuZCB0aGlzIGlzIGEgcGVyZmVjdCBzdG9ybSBvZiAiaG93IHRo ZSBwcm9qZWN0IGlzIGJhZCBhdCBhY2NlcHRpbmcgY29udHJpYnV0aW9ucyI6Cj4+Pj4+PiAoMSkg VGhlIG9yaWdpbmFsIHN1Ym1pc3Npb24gd2FzIGNsb3NlIHRvIHRoZSAxNCBicmFuY2ggY3JlYXRp b24gdGltZS4gVGhpcyBtZWFudCB0aGF0IHdlIHdlcmVuJ3Qgd2VsbCBwcmVwYXJlZCB0byBsb29r IGF0IGl0IHNpbmNlIGl0IGlzIHN1Y2ggYW4gaW52YXNpdmUgY2hhbmdlIChhdCBsZWFzdCBvbiBp dHMgc3VyZmFjZSkuIEl0IGFsc28gc2xvd2VkIHRoZSBpbml0aWFsIHJlc3BvbnNlLi4uCj4+Pj4+ PiAoMikgVGhlcmUgd2FzIGEgbnVtYmVyIG9mIGJhY2sgYW5kIGZvcnRoIHJlcXVlc3RzIGZvciBj aGFuZ2VzLCB3aGljaCB0b29rIHRpbWUgdG8gc29ydCBvdXQuLi4KPj4+Pj4+ICgzKSBUaGUgc2l6 ZSBvZiB0aGlzIGlzIGh1Z2UsIHdlbGwgYmV5b25kIHRoZSBjYXBhY2l0eSBvZiBQaGFicmljYXRv ciB0byByZXZpZXcgYWNjdXJhdGVseS4uLgo+Pj4+Pj4gKDQpIEl0J3MgYSB2ZW5kb3IgaW1wb3J0 LiBUaGF0IG1lYW5zIHdlIGNhbid0IGp1c3QgZHJvcCB0aGUgUGhhYnJpY2F0b3IgcmV2aWV3IGlu dG8gdGhlIHRyZWUuLi4KPj4+Pj4+ICg1KSBJdCdzIHBoYWJyaWNhdG9yOiB0aGlzIGlzIGEgZ3Jl YXQgdG9vbCBmb3IgZGV2ZWxvcGVycywgYnV0IHdlIGhhdmUgYSB0ZXJyaWJsZSB0cmFjayByZWNv cmQgb2YgdXNpbmcgaXQgZm9yIGludGFrZSBmcm9tIG5ldyBjb250cmlidXRvcnMuIFdlIGRvbid0 IGhhdmUgYW55IG92ZXJzaWdodCBhdCBhbGwgb3ZlciB0aGlzIHRvb2wsIGF0IHRoZXJlJ3MgYXQg YmVzdCB0ZXBpZCBhbmQgbHVrZSB3YXJtIGF0dGVtcHRzIHRvIGxvb2sgZm9yIGRyb3AgYmFsbHMu Cj4+Pj4+Pgo+Pj4+Pj4gQWxsIG9mIHRoZXNlIHRoaW5ncyBhcmUgYSB0ZXJyaWJsZSBleHBlcmll bmNlLiBJIGNhbiBvbmx5IGFwb2xvZ2l6ZS4gVGhlc2UgZGF5cywgd2UgbWlnaHQgc3RlZXIgdGhp cyB0b3dhcmRzIGdpdGh1YiwgYnV0IHRoZSAndmVuZG9yIGltcG9ydCcgbWVhbnMgeW91IHJlYWxs eSBuZWVkIHNvbWVvbmUgb24gdGhlIGluc2lkZSwgb3IgeW91IG5lZWQgdG8gYmUgb24gdGhlIGlu c2lkZSB0byBtYWtlIHRoYXQgd29yay4KPj4+Pj4+Cj4+Pj4+PiBTbywgaG93IHRvIG1vdmUgZm9y d2FyZD8gV2VsbCwgSSdkIGxpa2UgdG8gcHJvcG9zZSB0aGUgZm9sbG93aW5nOgo+Pj4+Pj4gKDEp IHN1Ym1pdCBhbGwgdGhlIG90aGVyIFBoYWJyaWNhdG9yIHJldmlld3MgeW91IGhhdmUgb3BlbiAo dGhleSBhcmUgbW9zdGx5IGdvb2QsIG9yIGNsb3NlIHRvIGdvb2QpIHRvIGdpdGh1Yi4gR2l0aHVi IGlzIGJlaW5nIGFjdGl2ZWx5IG1hbmFnZWQgYW5kIHdpbGwgbWFrZSBpdCBmYXN0ZXIgdG8gZ2V0 IHRoaW5ncyBpdC4gSXQncyBhIG11Y2ggYmV0dGVyIHRvb2wgZm9yIG5ldyBjb250cmlidXRvcnMg KGFuZCBldmVuIGZyZXF1ZW50IGNvbnRyaWJ1dG9ycyBvZiBzbWFsbGlzaCB0aGluZ3MpLgo+Pj4+ Pj4gKDIpIEkgc2hvdWxkIGRvIGFuIHZlbmRvciBpbXBvcnQgb2YgNS4zLjAgZnJvbSBnaXRodWIs IGFuZCBkbyB0aGUgbWVyZ2UgdG8gYSBicmFuY2ggYW5kIHB1c2ggdGhhdCB0byBnaXRodWIuIFlv dSBjYW4gdGhlbiBsYXllciBvbiB5b3VyIGNoYW5nZXMgYW5kIHRob3NlIGNhbiBiZSByZXZpZXdl ZCBtb3JlIGNsb3NlbHkgYXMgYSBwdWxsIHJlcXVlc3QgYWdhaW5zdCB0aGUgYnJhbmNoIEkgcHVz aC4gSSBzdXNwZWN0IHRoYXQgbW9zdCBvZiB0aGUgaXNzdWVzIGFyZSBzb3J0ZWQgb3V0IGFscmVh ZHkKPj4+Pj4+ICgzKSBJJ2xsIGxhbmQgaXQgdmlhIHRoYXQgcm91dGUuLi4KPj4+Pj4+Cj4+Pj4+ PiBBbmQsIGlmIHRoZSBzdW0gb2YgdGhlIG90aGVyIHB1bGwgcmVxdWVzdHMgYW5kIHRoaXMgYXJl IGdvb2QgKGFuZCBJIHN1c3BlY3QgdGhleSB3aWxsIGJlKSwgdGhlbiB3ZSBjYW4gdGFsayBhYm91 dCBjb21taXQgYml0cyBhbmQgc3VjaC4KPj4+Pj4+Cj4+Pj4+PiBJdCdzIGV4cGVyaWVuY2VzIGxp a2UgdGhpcyB3aGljaCBpcyB3aHkgSSdtIHRyeWluZyB0byBzdGFuZCB1cCBnaXRodWIgcHVsbCBy ZXF1ZXN0cyBhcyBhIHJlbGlhYmxlIHdheSB0byBnZXQgdGhpbmdzIGFuZCBhbmQgdGhlIGJlc3Qg cGxhY2UgdG8gc2VuZCBwZW9wbGUuLi4KPj4+Pj4+Cj4+Pj4+PiBUaGFua3MgYWdhaW4gZm9yIHBl cnNpc3RpbmcsIGFuZCBhbHNvIGZvciBleHByZXNzaW5nIHRoaXMgY3JpdGljaXNtIHRoYXQgd2Ug KGhvcGVmdWxseSkgY2FuIHVzZSB0byBtYWtlIGl0IGJldHRlci4KPj4+Pj4+Cj4+Pj4+PiBXYXJu ZXIKPj4+Pj4KPj4+Pj4gSGVsbG8uCj4+Pj4+Cj4+Pj4+IEknbSBub3QgdGhlIGF1dGhvciBvZiBE NDE0MjEuIEp1c3QgYXBwbGllZCB0aGUgcGF0Y2ggdG8gdGVzdCBpdCA4IG1vbnRocyBhZ28uIEFu ZCByZWNlbnRseSBkaXNjb3ZlcmVkIHRoYXQgaXQncyBzdGlsbCBub3QgY29tbWl0dGVkLgo+Pj4+ PiBJIGNhbid0IGNvcHkgeW91ciBtZXNzYWdlIHRvIFBoYWJyaWNhdG9yIGJlY2F1c2UgZG9uJ3Qg aGF2ZSBhbiBhY2NvdW50LiBQbGVhc2UsIGlmIHlvdSBoYXZlIHRpbWUsIGhlbHAgdGhlIGF1dGhv ciBpbiBENDE0MjEuCj4+Pj4KPj4+PiBBaCB5ZXMuIEkndmUgYmVlbiBpbiB0b3VjaCB3aXRoIHRo ZSBhdXRob3IgZm9yIG90aGVyIHRoaW5ncywgYW5kIHNvbWVob3cgdGhvdWdodCBpdCB3YXMgeW91 Li4uLiBJJ2xsIHJlYWNoIG91dCB0byBoaW0gdmlhIG90aGVyIG1lYW5zLi4uCj4+Pj4KPj4+PiBX YXJuZXI= --b1=_VrKTCxXWptU2LwIb5XlmprDDuAaVwa6prnu2JD7IM Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: base64 PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0 cHg7Ij5JIHNlZSwgaXQgaGFwcGVucy48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTogQXJp YWwsIHNhbnMtc2VyaWY7IGZvbnQtc2l6ZTogMTRweDsiPjxzcGFuPk1heWJlIGFub3RoZXIgY29t bWl0dGVyIHdpbGwgdm9sdW50ZWVyIHRvIGRvIHRoZSB1cGRhdGUuPC9zcGFuPjxkaXY+SSBob3Bl IGl0IHdpbGwgbWFrZSBpdHMgd2F5IGludG8gMTUuMCByZWxlYXNlLjwvZGl2PjxkaXY+PGJyPjwv ZGl2PjxkaXY+VGhhbmtzLjwvZGl2PjwvZGl2PjxkaXYgY2xhc3M9InByb3Rvbm1haWxfcXVvdGUi Pg0KICAgICAgICBPbiBGcmlkYXksIE5vdmVtYmVyIDI5dGgsIDIwMjQgYXQgOTozOCBQTSwgV2Fy bmVyIExvc2ggJmx0O2ltcEBic2RpbXAuY29tJmd0OyB3cm90ZTo8YnI+DQogICAgICAgIDxibG9j a3F1b3RlIGNsYXNzPSJwcm90b25tYWlsX3F1b3RlIiB0eXBlPSJjaXRlIj4NCiAgICAgICAgICAg IDxkaXYgZGlyPSJsdHIiPkkndmUgYmVlbiBzd2FtcGVkLiB3ZSBuZWVkIHRvIGJvb3RzdHJhcCB0 aGUgdmVuZG9yIGJyYW5jaCwgYW5kIHRoZSB3YXkgcHJpb3IgdXBkYXRlcyB3ZXJlIGRvbmU8ZGl2 Pmlzbid0IHNvIGdyZWF0LiA8L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2Pldhcm5lcjwvZGl2Pjwv ZGl2Pjxicj48ZGl2IGNsYXNzPSJnbWFpbF9xdW90ZSBnbWFpbF9xdW90ZV9jb250YWluZXIiPjxk aXYgY2xhc3M9ImdtYWlsX2F0dHIiIGRpcj0ibHRyIj5PbiBNb24sIE5vdiAyNSwgMjAyNCBhdCAy OjIx4oCvQU0gY2dsb2dpYyAmbHQ7PGEgaHJlZj0ibWFpbHRvOmNnbG9naWNAcHJvdG9ubWFpbC5j b20iIHJlbD0ibm9yZWZlcnJlciBub2ZvbGxvdyBub29wZW5lciI+Y2dsb2dpY0Bwcm90b25tYWls LmNvbTwvYT4mZ3Q7IHdyb3RlOjxicj48L2Rpdj48YmxvY2txdW90ZSBzdHlsZT0ibWFyZ2luOjBw eCAwcHggMHB4IDAuOGV4O2JvcmRlci1sZWZ0OjFweCBzb2xpZCByZ2IoMjA0LDIwNCwyMDQpO3Bh ZGRpbmctbGVmdDoxZXgiIGNsYXNzPSJnbWFpbF9xdW90ZSI+PGRpdiBzdHlsZT0iZm9udC1mYW1p bHk6QXJpYWwsc2Fucy1zZXJpZjtmb250LXNpemU6MTRweCI+SGVsbG8gZ3V5cyw8L2Rpdj48ZGl2 IHN0eWxlPSJmb250LWZhbWlseTpBcmlhbCxzYW5zLXNlcmlmO2ZvbnQtc2l6ZToxNHB4Ij48YnI+ PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6QXJpYWwsc2Fucy1zZXJpZjtmb250LXNpemU6 MTRweCI+SG93IHRoZSB1cGRhdGUgb2YgamVtYWxsb2MgaXMgZ29pbmc/IEl0J3MgTm92ZW1iZXIg bm93LjwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OkFyaWFsLHNhbnMtc2VyaWY7Zm9udC1z aXplOjE0cHgiPjxicj48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTpBcmlhbCxzYW5zLXNl cmlmO2ZvbnQtc2l6ZToxNHB4Ij5UaGFua3MuPC9kaXY+PGRpdj4NCiAgICAgICAgT24gTW9uZGF5 LCBKdWx5IDIybmQsIDIwMjQgYXQgNzowMiBQTSwgTWluc29vIENob28gJmx0OzxhIHRhcmdldD0i X2JsYW5rIiBocmVmPSJtYWlsdG86bWluc29vY2hvbzAxMjJAcHJvdG9uLm1lIiByZWw9Im5vcmVm ZXJyZXIgbm9mb2xsb3cgbm9vcGVuZXIiPm1pbnNvb2Nob28wMTIyQHByb3Rvbi5tZTwvYT4mZ3Q7 IHdyb3RlOjxicj4NCiAgICAgICAgPGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSI+DQogICAgICAgICAg ICA8ZGl2IHN0eWxlPSJmb250LWZhbWlseTpBcmlhbCxzYW5zLXNlcmlmO2ZvbnQtc2l6ZToxNHB4 Ij5GaXJzdCwgc29ycnkgZm9yIGxhdGUgcmVzcG9uc2UuPC9kaXY+PGRpdiBzdHlsZT0iZm9udC1m YW1pbHk6QXJpYWwsc2Fucy1zZXJpZjtmb250LXNpemU6MTRweCI+PGJyPjwvZGl2PjxkaXYgc3R5 bGU9ImZvbnQtZmFtaWx5OkFyaWFsLHNhbnMtc2VyaWY7Zm9udC1zaXplOjE0cHgiPmNnbG9naWMs IHRoYW5rIHlvdSBmb3IgYnJpbmdpbmcgdXAgdGhpcyBpc3N1ZSBhZ2FpbiBzaW5jZSBJIG5lYXJs eSBmb3Jnb3QgdGhhdCB0aGlzIGlzc3VlIHdhcyBzdGlsbCBvcGVuLjwvZGl2PjxkaXYgc3R5bGU9 ImZvbnQtZmFtaWx5OkFyaWFsLHNhbnMtc2VyaWY7Zm9udC1zaXplOjE0cHgiPjxicj48L2Rpdj48 ZGl2IHN0eWxlPSJmb250LWZhbWlseTpBcmlhbCxzYW5zLXNlcmlmO2ZvbnQtc2l6ZToxNHB4Ij5X YXJuZXIsIGFzIEkgY2FuJ3QgYWNjZXNzIHRvIG15IEZyZWVCU0QgaW5zdGFuY2UgdW50aWwgdGhl IGVuZCBvZiBBdWd1c3QsIGJ1dCBJIGNhbiBzdGlsbCBlZGl0IGFuZCBwdXNoIHRoZSBjb2RlIHRo cm91Z2ggbXkgQXJtIE1hYy4gVGhpcyBtZWFucyB0aGF0IEkgY2FuJ3QgdGVzdCB0aGUgdXBkYXRl ZCBjb2RlIG9uIG15IG1hY2hpbmUsIGJ1dCBJIGNhbiBqb2luIHRoZSByZXZpZXcgcHJvY2VzcyBh bmQgbGlzdGVuIHRvIGNoYW5nZSBwcm9wb3NhbHMuPC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1p bHk6QXJpYWwsc2Fucy1zZXJpZjtmb250LXNpemU6MTRweCI+PGJyPjwvZGl2PjxkaXYgc3R5bGU9 ImZvbnQtZmFtaWx5OkFyaWFsLHNhbnMtc2VyaWY7Zm9udC1zaXplOjE0cHgiPkknbGwgb3BlbiBh IEdpdGh1YiBQUiBpbiBhIGZldyBob3Vycy4gKFRoZSBwaGFicmljYXRvciByZXZpZXcgd2lsbCBz dGF5IG9wZW5lZCBqdXN0IGluIGNhc2UpPC9kaXY+PGRpdj4NCiAgICAgICAgT24gTW9uZGF5LCBK dWx5IDIybmQsIDIwMjQgYXQgNTowOCBBTSwgV2FybmVyIExvc2ggJmx0OzxhIHRhcmdldD0iX2Js YW5rIiBocmVmPSJtYWlsdG86aW1wQGJzZGltcC5jb20iIHJlbD0ibm9yZWZlcnJlciBub2ZvbGxv dyBub29wZW5lciI+aW1wQGJzZGltcC5jb208L2E+Jmd0OyB3cm90ZTo8YnI+DQogICAgICAgIDxi bG9ja3F1b3RlIHR5cGU9ImNpdGUiPg0KICAgICAgICAgICAgPGRpdiBkaXI9Imx0ciI+PGRpdiBk aXI9Imx0ciI+PGJyPjwvZGl2Pjxicj48ZGl2IGNsYXNzPSJnbWFpbF9xdW90ZSI+PGRpdiBjbGFz cz0iZ21haWxfYXR0ciIgZGlyPSJsdHIiPk9uIFN1biwgSnVsIDIxLCAyMDI0IGF0IDI6MDPigK9Q TSBjZ2xvZ2ljICZsdDs8YSB0YXJnZXQ9Il9ibGFuayIgaHJlZj0ibWFpbHRvOmNnbG9naWNAcHJv dG9ubWFpbC5jb20iIHJlbD0ibm9yZWZlcnJlciBub2ZvbGxvdyBub29wZW5lciI+Y2dsb2dpY0Bw cm90b25tYWlsLmNvbTwvYT4mZ3Q7IHdyb3RlOjxicj48L2Rpdj48YmxvY2txdW90ZSBzdHlsZT0i bWFyZ2luOjBweCAwcHggMHB4IDAuOGV4O2JvcmRlci1sZWZ0OjFweCBzb2xpZCByZ2IoMjA0LDIw NCwyMDQpO3BhZGRpbmctbGVmdDoxZXgiIGNsYXNzPSJnbWFpbF9xdW90ZSI+PGRpdiBzdHlsZT0i Zm9udC1mYW1pbHk6QXJpYWwsc2Fucy1zZXJpZjtmb250LXNpemU6MTRweCI+PGJyPjwvZGl2Pjxk aXY+DQogICAgICAgIE9uIFN1bmRheSwgSnVseSAyMXN0LCAyMDI0IGF0IDY6NTQgQU0sIFdhcm5l ciBMb3NoICZsdDs8YSB0YXJnZXQ9Il9ibGFuayIgaHJlZj0ibWFpbHRvOmltcEBic2RpbXAuY29t IiByZWw9Im5vcmVmZXJyZXIgbm9mb2xsb3cgbm9vcGVuZXIiPmltcEBic2RpbXAuY29tPC9hPiZn dDsgd3JvdGU6PGJyPg0KICAgICAgICA8YmxvY2txdW90ZSB0eXBlPSJjaXRlIj4NCiAgICAgICAg ICAgIDxkaXYgZGlyPSJsdHIiPjxkaXYgZGlyPSJsdHIiPjxicj48L2Rpdj48YnI+PGRpdiBjbGFz cz0iZ21haWxfcXVvdGUiPjxkaXYgY2xhc3M9ImdtYWlsX2F0dHIiIGRpcj0ibHRyIj5PbiBTYXQs IEp1bCAyMCwgMjAyNCBhdCAxOjU54oCvQU0gY2dsb2dpYyAmbHQ7PGEgdGFyZ2V0PSJfYmxhbmsi IGhyZWY9Im1haWx0bzpjZ2xvZ2ljQHByb3Rvbm1haWwuY29tIiByZWw9Im5vcmVmZXJyZXIgbm9m b2xsb3cgbm9vcGVuZXIiPmNnbG9naWNAcHJvdG9ubWFpbC5jb208L2E+Jmd0OyB3cm90ZTo8YnI+ PC9kaXY+PGJsb2NrcXVvdGUgc3R5bGU9Im1hcmdpbjowcHggMHB4IDBweCAwLjhleDtib3JkZXIt bGVmdDoxcHggc29saWQgcmdiKDIwNCwyMDQsMjA0KTtwYWRkaW5nLWxlZnQ6MWV4IiBjbGFzcz0i Z21haWxfcXVvdGUiPjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsg Zm9udC1zaXplOiAxNHB4OyBjb2xvcjogcmdiKDAsIDAsIDApOyBiYWNrZ3JvdW5kLWNvbG9yOiBy Z2IoMjU1LCAyNTUsIDI1NSk7Ij5IZWxsbyBGcmVlQlNEIGNvbW11bml0eSw8L2Rpdj48ZGl2IHN0 eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNhbnMtc2VyaWY7IGZvbnQtc2l6ZTogMTRweDsgY29s b3I6IHJnYigwLCAwLCAwKTsgYmFja2dyb3VuZC1jb2xvcjogcmdiKDI1NSwgMjU1LCAyNTUpOyI+ PGJyPjwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9u dC1zaXplOiAxNHB4OyBjb2xvcjogcmdiKDAsIDAsIDApOyBiYWNrZ3JvdW5kLWNvbG9yOiByZ2Io MjU1LCAyNTUsIDI1NSk7Ij48c3BhbiBzdHlsZT0iZGlzcGxheTogaW5saW5lOyBiYWNrZ3JvdW5k LWNvbG9yOiByZ2IoMjU1LCAyNTUsIDI1NSk7Ij5BZnRlciA8L3NwYW4+PHNwYW4gc3R5bGU9ImJh Y2tncm91bmQtY29sb3I6IHJnYigyNTUsIDI1NSwgMjU1KTsiPkphc29uIEV2YW5zIHN0ZXBwZWQg YXNpZGUgZnJvbSBtYWludGFpbmluZyBqZW1hbGxvYyBpbiBGcmVlQlNELCBpdCdzIG5vdCB1cGRh dGluZyBpbiB0aW1lIGFueW1vcmUuPC9zcGFuPjxicj48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZh bWlseTogQXJpYWwsIHNhbnMtc2VyaWY7IGZvbnQtc2l6ZTogMTRweDsgY29sb3I6IHJnYigwLCAw LCAwKTsgYmFja2dyb3VuZC1jb2xvcjogcmdiKDI1NSwgMjU1LCAyNTUpOyI+VmVyc2lvbiA1LjMu MCB3YXMgcmVsZWFzZWQgPHNwYW4+TWF5IDYsIDIwMjIgYW5kIEZyZWVCU0Qgc3RpbGwgbm90IGlt cG9ydGVkIGl0IGludG8gdGhlIHRyZWUuPC9zcGFuPjwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFt aWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxNHB4OyBjb2xvcjogcmdiKDAsIDAs IDApOyBiYWNrZ3JvdW5kLWNvbG9yOiByZ2IoMjU1LCAyNTUsIDI1NSk7Ij48c3Bhbj48YnI+PC9z cGFuPjwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9u dC1zaXplOiAxNHB4OyBjb2xvcjogcmdiKDAsIDAsIDApOyBiYWNrZ3JvdW5kLWNvbG9yOiByZ2Io MjU1LCAyNTUsIDI1NSk7Ij5UaGVyZSBpcyBhIHBlbmRpbmcgcmV2aWV3IDxzcGFuPjxhIHRhcmdl dD0iX2JsYW5rIiBocmVmPSJodHRwczovL3Jldmlld3MuZnJlZWJzZC5vcmcvRDQxNDIxIiByZWw9 Im5vcmVmZXJyZXIgbm9mb2xsb3cgbm9vcGVuZXIiPmh0dHBzOi8vcmV2aWV3cy5mcmVlYnNkLm9y Zy9ENDE0MjE8L2E+IGZyb20gPHNwYW4+QXVnIDExLCAyMDIzLjwvc3Bhbj48L3NwYW4+PC9kaXY+ PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0 cHg7IGNvbG9yOiByZ2IoMCwgMCwgMCk7IGJhY2tncm91bmQtY29sb3I6IHJnYigyNTUsIDI1NSwg MjU1KTsiPjxzcGFuPjxzcGFuPkknbSBzdWNjZXNzZnVsbHkgcnVubmluZyBGcmVlQlNEL2FtZDY0 IHN5c3RlbSB3aXRoIEQ0MTQyMSBhcHBsaWVkIGZvciA4IG1vbnRocywgYXMgd2VsbCBhcyBtYW55 IG90aGVyIHBlb3BsZS48L3NwYW4+PC9zcGFuPjwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5 OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxNHB4OyBjb2xvcjogcmdiKDAsIDAsIDAp OyBiYWNrZ3JvdW5kLWNvbG9yOiByZ2IoMjU1LCAyNTUsIDI1NSk7Ij48c3Bhbj48c3Bhbj48YnI+ PC9zcGFuPjwvc3Bhbj48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNhbnMt c2VyaWY7IGZvbnQtc2l6ZTogMTRweDsgY29sb3I6IHJnYigwLCAwLCAwKTsgYmFja2dyb3VuZC1j b2xvcjogcmdiKDI1NSwgMjU1LCAyNTUpOyI+PHNwYW4+PHNwYW4+Q2FuIGl0IGJlIHJldmlld2Vk IGFuZCBjb21taXR0ZWQgdG8gQ1VSUkVOVD88L3NwYW4+PC9zcGFuPjwvZGl2PjxkaXYgc3R5bGU9 ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxNHB4OyBjb2xvcjog cmdiKDAsIDAsIDApOyBiYWNrZ3JvdW5kLWNvbG9yOiByZ2IoMjU1LCAyNTUsIDI1NSk7Ij48c3Bh bj48c3Bhbj5PciwgaWYgdGhlcmUgaXMgbm8gY29tbWl0dGVycyB3aWxsaW5nIHRvIGRvIGl0LCBj YW4gY29tbWl0IGJpdCBiZSBnaXZlbiB0byBzdWJtaXR0ZXIgb3IgYW5vdGhlciBwZXJzb24gd2ls bGluZyB0byBkbyB0aGlzPzwvc3Bhbj48L3NwYW4+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1p bHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0cHg7IGNvbG9yOiByZ2IoMCwgMCwg MCk7IGJhY2tncm91bmQtY29sb3I6IHJnYigyNTUsIDI1NSwgMjU1KTsiPjxzcGFuPjxzcGFuPjxi cj48L3NwYW4+PC9zcGFuPjwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fu cy1zZXJpZjsgZm9udC1zaXplOiAxNHB4OyBjb2xvcjogcmdiKDAsIDAsIDApOyBiYWNrZ3JvdW5k LWNvbG9yOiByZ2IoMjU1LCAyNTUsIDI1NSk7Ij48c3Bhbj48c3Bhbj48c3Bhbj5JdCdzIHZlcnkg ZGlzYXBwb2ludGluZyB3aGVuIHVzZXJzIHNwZW5kIHRoZWlyIHRpbWUgdG8gZmlsbCBzdWNoIGdh cHMgYW5kIHRoZWlyIGVmZm9ydHMganVzdCBpZ25vcmVkIGJ5IHRoZSBkZXZlbG9wZXJzLjwvc3Bh bj48YnI+PC9zcGFuPjwvc3Bhbj48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTogQXJpYWws IHNhbnMtc2VyaWY7IGZvbnQtc2l6ZTogMTRweDsgY29sb3I6IHJnYigwLCAwLCAwKTsgYmFja2dy b3VuZC1jb2xvcjogcmdiKDI1NSwgMjU1LCAyNTUpOyI+PHNwYW4+PHNwYW4+PHNwYW4+PHNwYW4+ RXZlcnkgeWVhciBGcmVlQlNEIENvbW11bml0eSBTdXJ2ZXkgYXNraW5nIGFib3V0IHVzZXIgZXhw ZXJpZW5jZSBpbiBjb250cmlidXRpbmcgdG8gRnJlZUJTRC4gPC9zcGFuPjxicj48L3NwYW4+PC9z cGFuPjwvc3Bhbj48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNhbnMtc2Vy aWY7IGZvbnQtc2l6ZTogMTRweDsgY29sb3I6IHJnYigwLCAwLCAwKTsgYmFja2dyb3VuZC1jb2xv cjogcmdiKDI1NSwgMjU1LCAyNTUpOyI+PHNwYW4+PHNwYW4+PHNwYW4+PHNwYW4+SGVyZSB5b3Ug Y2FuIHNlZSBhbiBleGFtcGxlIG9mIHN1Y2ggY29udHJpYnV0aW5nLjwvc3Bhbj48L3NwYW4+PC9z cGFuPjwvc3Bhbj48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNhbnMtc2Vy aWY7IGZvbnQtc2l6ZTogMTRweDsgY29sb3I6IHJnYigwLCAwLCAwKTsgYmFja2dyb3VuZC1jb2xv cjogcmdiKDI1NSwgMjU1LCAyNTUpOyI+PHNwYW4+PHNwYW4+PHNwYW4+PC9zcGFuPjwvc3Bhbj48 L3NwYW4+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBm b250LXNpemU6IDE0cHg7IGNvbG9yOiByZ2IoMCwgMCwgMCk7IGJhY2tncm91bmQtY29sb3I6IHJn YigyNTUsIDI1NSwgMjU1KTsiPjxzcGFuPjxicj48L3NwYW4+PC9kaXY+PC9ibG9ja3F1b3RlPjxk aXY+PGJyPjwvZGl2PjxkaXY+Rmlyc3QsIHRoYW5rIHlvdSBmb3IgYmVpbmcgcGVyc2lzdGVudCBh bmQgY29udGludWluZyB0byBicmluZyBpdCB1cC4gSXQncyBpbXBvcnRhbnQgdG8gZG8gdGhhdCB0 byBtYWtlIHN1cmUgdGhpcyAoYW5kIHlvdXIgbWFueSBvdGhlcikgY29udHJpYnV0aW9uIGRvZXNu J3QgZmFsbCBvbiB0aGUgZmxvb3IuPGJyPjwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+QW5kIHRv IGJlIGZhaXIsIHdlJ3JlIG9ubHkgMyBtb250aHMgc2luY2UgdGhlIGxhc3QgdXBkYXRlLiBTdGls bCwgcXVpdGUgYSBiaXQgbG9uZ2VyIHRoYW4geW91IHNob3VsZCBoYXZlIHRvIHdhaXQsIGJ1dCBu b3QgbmVhcmx5IHRoZSB5ZWFyIHRoZSBvcmlnaW5hbCBkYXRlIHN1Z2dlc3RzLjxicj48L2Rpdj48 ZGl2Pjxicj48L2Rpdj48ZGl2PkFuZCB0aGlzIGlzIGEgcGVyZmVjdCBzdG9ybSBvZiAiaG93IHRo ZSBwcm9qZWN0IGlzIGJhZCBhdCBhY2NlcHRpbmcgY29udHJpYnV0aW9ucyI6PC9kaXY+PGRpdj4o MSkgVGhlIG9yaWdpbmFsIHN1Ym1pc3Npb24gd2FzIGNsb3NlIHRvIHRoZSAxNCBicmFuY2ggY3Jl YXRpb24gdGltZS4gVGhpcyBtZWFudCB0aGF0IHdlIHdlcmVuJ3Qgd2VsbCBwcmVwYXJlZCB0byBs b29rIGF0IGl0IHNpbmNlIGl0IGlzIHN1Y2ggYW4gaW52YXNpdmUgY2hhbmdlIChhdCBsZWFzdCBv biBpdHMgc3VyZmFjZSkuIEl0IGFsc28gc2xvd2VkIHRoZSBpbml0aWFsIHJlc3BvbnNlLi4uPGJy PjwvZGl2PjxkaXY+KDIpIFRoZXJlIHdhcyBhIG51bWJlciBvZiBiYWNrIGFuZCBmb3J0aCByZXF1 ZXN0cyBmb3IgY2hhbmdlcywgd2hpY2ggdG9vayB0aW1lIHRvIHNvcnQgb3V0Li4uPC9kaXY+PGRp dj4oMykgVGhlIHNpemUgb2YgdGhpcyBpcyBodWdlLCB3ZWxsIGJleW9uZCB0aGUgY2FwYWNpdHkg b2YgUGhhYnJpY2F0b3IgdG8gcmV2aWV3IGFjY3VyYXRlbHkuLi48L2Rpdj48ZGl2Pig0KSBJdCdz IGEgdmVuZG9yIGltcG9ydC4gVGhhdCBtZWFucyB3ZSBjYW4ndCBqdXN0IGRyb3AgdGhlIFBoYWJy aWNhdG9yIHJldmlldyBpbnRvIHRoZSB0cmVlLi4uPC9kaXY+PGRpdj4oNSkgSXQncyBwaGFicmlj YXRvcjogdGhpcyBpcyBhIGdyZWF0IHRvb2wgZm9yIGRldmVsb3BlcnMsIGJ1dCB3ZSBoYXZlIGEg dGVycmlibGUgdHJhY2sgcmVjb3JkIG9mIHVzaW5nIGl0IGZvciBpbnRha2UgZnJvbSBuZXcgY29u dHJpYnV0b3JzLiBXZSBkb24ndCBoYXZlIGFueSBvdmVyc2lnaHQgYXQgYWxsIG92ZXIgdGhpcyB0 b29sLCBhdCB0aGVyZSdzIGF0IGJlc3QgdGVwaWQgYW5kIGx1a2Ugd2FybSBhdHRlbXB0cyB0byBs b29rIGZvciBkcm9wIGJhbGxzLjwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+QWxsIG9mIHRoZXNl IHRoaW5ncyBhcmUgYSB0ZXJyaWJsZSBleHBlcmllbmNlLiBJIGNhbiBvbmx5IGFwb2xvZ2l6ZS4g VGhlc2UgZGF5cywgd2UgbWlnaHQgc3RlZXIgdGhpcyB0b3dhcmRzIGdpdGh1YiwgYnV0IHRoZSAn dmVuZG9yIGltcG9ydCcgbWVhbnMgeW91IHJlYWxseSBuZWVkIHNvbWVvbmUgb24gdGhlIGluc2lk ZSwgb3IgeW91IG5lZWQgdG8gYmUgb24gdGhlIGluc2lkZSB0byBtYWtlIHRoYXQgd29yay48L2Rp dj48ZGl2Pjxicj48L2Rpdj48ZGl2PlNvLCBob3cgdG8gbW92ZSBmb3J3YXJkPyBXZWxsLCBJJ2Qg bGlrZSB0byBwcm9wb3NlIHRoZSBmb2xsb3dpbmc6PC9kaXY+PGRpdj4oMSkgc3VibWl0IGFsbCB0 aGUgb3RoZXIgUGhhYnJpY2F0b3IgcmV2aWV3cyB5b3UgaGF2ZSBvcGVuICh0aGV5IGFyZSBtb3N0 bHkgZ29vZCwgb3IgY2xvc2UgdG8gZ29vZCkgdG8gZ2l0aHViLiBHaXRodWIgaXMgYmVpbmcgYWN0 aXZlbHkgbWFuYWdlZCBhbmQgd2lsbCBtYWtlIGl0IGZhc3RlciB0byBnZXQgdGhpbmdzIGl0LiBJ dCdzIGEgbXVjaCBiZXR0ZXIgdG9vbCBmb3IgbmV3IGNvbnRyaWJ1dG9ycyAoYW5kIGV2ZW4gZnJl cXVlbnQgY29udHJpYnV0b3JzIG9mIHNtYWxsaXNoIHRoaW5ncykuPC9kaXY+PGRpdj4oMikgSSBz aG91bGQgZG8gYW4gdmVuZG9yIGltcG9ydCBvZiA1LjMuMCBmcm9tIGdpdGh1YiwgYW5kIGRvIHRo ZSBtZXJnZSB0byBhIGJyYW5jaCBhbmQgcHVzaCB0aGF0IHRvIGdpdGh1Yi4gWW91IGNhbiB0aGVu IGxheWVyIG9uIHlvdXIgY2hhbmdlcyBhbmQgdGhvc2UgY2FuIGJlIHJldmlld2VkIG1vcmUgY2xv c2VseSBhcyBhIHB1bGwgcmVxdWVzdCBhZ2FpbnN0IHRoZSBicmFuY2ggSSBwdXNoLiBJIHN1c3Bl Y3QgdGhhdCBtb3N0IG9mIHRoZSBpc3N1ZXMgYXJlIHNvcnRlZCBvdXQgYWxyZWFkeSA8YnI+PC9k aXY+PGRpdj4oMykgSSdsbCBsYW5kIGl0IHZpYSB0aGF0IHJvdXRlLi4uPC9kaXY+PGRpdj48YnI+ PC9kaXY+PGRpdj5BbmQsIGlmIHRoZSBzdW0gb2YgdGhlIG90aGVyIHB1bGwgcmVxdWVzdHMgYW5k IHRoaXMgYXJlIGdvb2QgKGFuZCBJIHN1c3BlY3QgdGhleSB3aWxsIGJlKSwgdGhlbiB3ZSBjYW4g dGFsayBhYm91dCBjb21taXQgYml0cyBhbmQgc3VjaC48L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2 Pkl0J3MgZXhwZXJpZW5jZXMgbGlrZSB0aGlzIHdoaWNoIGlzIHdoeSBJJ20gdHJ5aW5nIHRvIHN0 YW5kIHVwIGdpdGh1YiBwdWxsIHJlcXVlc3RzIGFzIGEgcmVsaWFibGUgd2F5IHRvIGdldCB0aGlu Z3MgYW5kIGFuZCB0aGUgYmVzdCBwbGFjZSB0byBzZW5kIHBlb3BsZS4uLiAgPGJyPjwvZGl2Pjxk aXY+PGJyPjwvZGl2PjxkaXY+VGhhbmtzIGFnYWluIGZvciBwZXJzaXN0aW5nLCBhbmQgYWxzbyBm b3IgZXhwcmVzc2luZyB0aGlzIGNyaXRpY2lzbSB0aGF0IHdlIChob3BlZnVsbHkpIGNhbiB1c2Ug dG8gbWFrZSBpdCBiZXR0ZXIuPGJyPjwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+V2FybmVyPGJy PjwvZGl2PjwvZGl2PjwvZGl2Pg0KDQogICAgICAgIDwvYmxvY2txdW90ZT48L2Rpdj48ZGl2IHN0 eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNhbnMtc2VyaWY7IGZvbnQtc2l6ZTogMTRweDsgY29s b3I6IHJnYigwLCAwLCAwKTsgYmFja2dyb3VuZC1jb2xvcjogcmdiKDI1NSwgMjU1LCAyNTUpOyI+ PGJyPjwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9u dC1zaXplOiAxNHB4OyBjb2xvcjogcmdiKDAsIDAsIDApOyBiYWNrZ3JvdW5kLWNvbG9yOiByZ2Io MjU1LCAyNTUsIDI1NSk7Ij5IZWxsby48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTogQXJp YWwsIHNhbnMtc2VyaWY7IGZvbnQtc2l6ZTogMTRweDsgY29sb3I6IHJnYigwLCAwLCAwKTsgYmFj a2dyb3VuZC1jb2xvcjogcmdiKDI1NSwgMjU1LCAyNTUpOyI+PGJyPjwvZGl2PjxkaXYgc3R5bGU9 ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxNHB4OyBjb2xvcjog cmdiKDAsIDAsIDApOyBiYWNrZ3JvdW5kLWNvbG9yOiByZ2IoMjU1LCAyNTUsIDI1NSk7Ij5JJ20g bm90IHRoZSBhdXRob3Igb2YgPHNwYW4+RDQxNDIxLiBKdXN0IGFwcGxpZWQgdGhlIHBhdGNoIHRv IHRlc3QgaXQgOCBtb250aHMgYWdvLiBBbmQgcmVjZW50bHkgZGlzY292ZXJlZCB0aGF0IGl0J3Mg c3RpbGwgbm90IGNvbW1pdHRlZC48L3NwYW4+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6 IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0cHg7IGNvbG9yOiByZ2IoMCwgMCwgMCk7 IGJhY2tncm91bmQtY29sb3I6IHJnYigyNTUsIDI1NSwgMjU1KTsiPjxzcGFuPkkgY2FuJ3QgY29w eSB5b3VyIG1lc3NhZ2UgdG8gUGhhYnJpY2F0b3IgYmVjYXVzZSBkb24ndCBoYXZlIGFuIGFjY291 bnQuIDwvc3Bhbj5QbGVhc2UsIGlmIHlvdSBoYXZlIHRpbWUsIGhlbHAgdGhlIGF1dGhvciBpbiBE NDE0MjEuPC9kaXY+PC9ibG9ja3F1b3RlPjxkaXY+PGJyPjwvZGl2PjxkaXY+QWggeWVzLiBJJ3Zl IGJlZW4gaW4gdG91Y2ggd2l0aCB0aGUgYXV0aG9yIGZvciBvdGhlciB0aGluZ3MsIGFuZCBzb21l aG93IHRob3VnaHQgaXQgd2FzIHlvdS4uLi4gIEknbGwgcmVhY2ggb3V0IHRvIGhpbSB2aWEgb3Ro ZXIgbWVhbnMuLi48L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2Pldhcm5lcjwvZGl2PjwvZGl2Pjwv ZGl2Pg0KDQogICAgICAgIDwvYmxvY2txdW90ZT48YnI+DQogICAgPC9kaXY+DQogICAgICAgIDwv YmxvY2txdW90ZT48YnI+DQogICAgPC9kaXY+PC9ibG9ja3F1b3RlPjwvZGl2Pg0KDQogICAgICAg IDwvYmxvY2txdW90ZT48YnI+DQogICAgPC9kaXY+ --b1=_VrKTCxXWptU2LwIb5XlmprDDuAaVwa6prnu2JD7IM-- From nobody Sun Dec 1 00:00:41 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Y16WF51Vmz5fsWS; Sun, 01 Dec 2024 00:00:41 +0000 (UTC) (envelope-from salvadore@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Y16WF2KLZz4rS5; Sun, 1 Dec 2024 00:00:41 +0000 (UTC) (envelope-from salvadore@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1733011241; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc; bh=fX7O2+XsLTbdxTTjqISDnAmRPcm8ltigtm5MwSRW3KE=; b=njRFIbW9h71H5HIAtCsJ5tw7GaEK9JepqkucJ2VSQfpv8Zti0i1N1U11v09MG/WAfH9aXV ZXQ9hXEpuBO2jxO3DxKKKnI7BsqKhe5gyFjmg99zjgXBgW15Oz8mG4GZyJFySJYe1ia9u2 F6gO/rbJbv/h5rkaaFY+bpRXVjp4X0gPMo/2tLLEOVqB8vP44GSr7FK2++ZPtLIykxvErW roAViyVKers+O+Xn92OsD47BljZ2xzZzoliUTu2lnSgV58M1B6K/6bD3A++1GQjdAFziWG 2/DIinB3nw7XPe0ZNy/CRc6mZgokGzA8v7cxsm2ajcFVOruwY6UrIcHhS57JTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1733011241; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc; bh=fX7O2+XsLTbdxTTjqISDnAmRPcm8ltigtm5MwSRW3KE=; b=hoDH73vCho3HCU2/XUGB9QyiJoL4IzK21dAmoc/NRRN3MVF/z1mVeWVeBD+01yA1r91CZp AwyrEAJePkCEtIFiQNY3K0LAXi0Yv0DOrwfcsZu4VV+0gdJ0ORpvbp1qlEyu2VunyJSVL7 jgAhK/q/06+NY/tZTZVkki0my4F4OQQYltq4Q3cVPp4XzNi2bB8c3gtTYjaXtRf3j6UuaX cm/o/0HJ15VxIBy86lbARD4HWL9zbHNZbANllDeIGA2zpCAQY1cArfGwD8Y/NNG+WDihGq jaNFJHhT77N+CX8Imo9LYKUW7i7GLZ0Oyqsn71GDlblmn0/dTrIi1QgSgTN/zQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1733011241; a=rsa-sha256; cv=none; b=FVGaaerNPIlVxq0uz52w6xBnN+0TPvM8Dg5cq0ebgVN48LHm6m30bBuumFl/Rbuy3F5Dfc OtZh1Hn3QqMuoICKxZDEc+IY4aMlQTzJpaS/VetjDhOk4SNv6tamaAMCGRtmHVNQdxeqQO 6bzpEBa8syltuBBut8K8iIt9ba5TX1mmnabvbyODetTtKHXtBzUP5BYQpUcwLpeWwOg0Yg 8sl8ub8i6JAcAaC9cqTGkchACxDoQDx9JWre/Zcw7C4jph5W/QXu+JfCZwEJ1GUuO2BFtn Ag67NAhOTKHGPeGkYItG1IqeGWzbydXLIMfzFOBpoQMGrYlOPDZmA4Cwz34Upg== Received: by freefall.freebsd.org (Postfix, from userid 1472) id 2C84BEE90; Sun, 01 Dec 2024 00:00:41 +0000 (UTC) To: freebsd-status-calls@FreeBSD.org Subject: Call for 2024Q4 status reports Cc: freebsd-current@FreeBSD.org,freebsd-hackers@FreeBSD.org,devsummit@FreeBSD.org Message-Id: <20241201000041.2C84BEE90@freefall.freebsd.org> Date: Sun, 01 Dec 2024 00:00:41 +0000 (UTC) From: Lorenzo Salvadore List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Dear FreeBSD Community, The deadline for the next FreeBSD Status Report update is December, 31st 2024 for work done since the last round of quarterly reports: October 2024 - December 2024. I would like to remind you that reports are published on a quarterly basis and are usually collected during the last month of each quarter, You are also welcome to submit them even earlier if you want, and the earlier you submit them, the more time we have for reviewing. Status report submissions do not need to be very long. They may be about anything happening in the FreeBSD project and community, and they provide a great way to inform FreeBSD users and developers about work that is underway or has been completed. Report submissions are not limited to committers; anyone doing anything interesting and FreeBSD related can -- and should -- write one! The following methods are available to submit your reports: * submit a review on Phabricator and add the group "status" to the reviewers list. You should put your reports in the directory doc/website/content/en/status/report-2024-10-2024-12/ (create it if it is missing); * submit a pull request at . You should put your reports in the directory doc/website/content/en/status/report-2024-10-2024-12/ (create it if it is missing); * send an email to status-submissions@FreeBSD.org including your report. An AsciiDoc template is available at . We look forward to seeing your 2024Q4 reports! Thanks, Lorenzo Salvadore (on behalf of status@) From nobody Sun Dec 1 03:26:17 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Y1C4f0Kbnz5g7RY for ; Sun, 01 Dec 2024 03:26:26 +0000 (UTC) (envelope-from minsoochoo0122@proton.me) Received: from mail-4319.protonmail.ch (mail-4319.protonmail.ch [185.70.43.19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Y1C4d1tdPz4017 for ; Sun, 1 Dec 2024 03:26:25 +0000 (UTC) (envelope-from minsoochoo0122@proton.me) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proton.me; s=j7gnfwfmsfgine2sfeuszfasom.protonmail; t=1733023581; x=1733282781; bh=77upd/OWbyr+Rm8gRS61zJMxi1yMDzONqBCXW5xzWJo=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector:List-Unsubscribe:List-Unsubscribe-Post; b=E2KBqC6qQ4Ie1yIb2dUUZCRfbNcFngqc6CTqEKMFN+n6YIuX3sQSPHv9S/R5NJuOw NCg+s04dvFer4dGz2W53yXRazKuR+oVo6fsabIDl3z5lXs8tkc7oPw/NEXJomqR+y6 iYMr219NpnzvEuJ89LLkDS2Jqjth3zJxK7kLikJV2l26xEiqBZkPgj+IczfqQ7FL6D oaYKFkl146SCrIh9sSx0fDftJPsbfskgG7Z8dLnm6oFt49LIDAEJ/6Vf5dYGgNw5UL jfX9pUSb2dzyhps/aB4vggg5NAbz/XFYjhrJxOUsHE0TzGZ+PLjzPZ5t7cwQD+TRHO pzVLvjGEzyobg== Date: Sun, 01 Dec 2024 03:26:17 +0000 To: cglogic From: Minsoo Choo Cc: Warner Losh , FreeBSD CURRENT Subject: Re: Long time outdated jemalloc Message-ID: In-Reply-To: References: Feedback-ID: 45891198:user:proton X-Pm-Message-ID: f2d803cd6d5fd2bbd07d026f85f5bc53b7cc7c5b List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="b1=_63fffS1uc9klJoy0NA5T6JyR8TtrsJ8bYUnVUHbeUbU" X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:62371, ipnet:185.70.43.0/24, country:CH] X-Rspamd-Queue-Id: 4Y1C4d1tdPz4017 X-Spamd-Bar: ---- --b1=_63fffS1uc9klJoy0NA5T6JyR8TtrsJ8bYUnVUHbeUbU Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 SSBoYXZlIGFscmVhZHkgc3VibWl0dGVkIFBSIG9uIGdpdGh1YiAoaHR0cHM6Ly9naXRodWIuY29t L2ZyZWVic2QvZnJlZWJzZC1zcmMvcHVsbC8xMzM3KSBhbmQgcGhhYnJpY2F0b3IgKGh0dHBzOi8v cmV2aWV3cy5mcmVlYnNkLm9yZy9ENDE0MjEpLiBJIGRvbid0IGhhdmUgYWNjZXNzIChjb21taXQg Yml0KSB0byBmcmVlYnNkIGdpdCByZXBvLCBzbyB0aGVyZSBpcyBub3RoaW5nIEkgY2FuIGRvIGF0 IHRoaXMgcG9pbnQgc2luY2UgdmVuZG9yIGltcG9ydCBhbmQgbGFuZGluZyBwYXRjaGVzIHJlcXVp cmVzIGNvbW1pdCBiaXQuCk9uIFNhdHVyZGF5LCBOb3ZlbWJlciAzMHRoLCAyMDI0IGF0IDE6NDIg UE0sIGNnbG9naWMgPGNnbG9naWNAcHJvdG9ubWFpbC5jb20+IHdyb3RlOgoKPiBJIHNlZSwgaXQg aGFwcGVucy4KPiBNYXliZSBhbm90aGVyIGNvbW1pdHRlciB3aWxsIHZvbHVudGVlciB0byBkbyB0 aGUgdXBkYXRlLgo+IEkgaG9wZSBpdCB3aWxsIG1ha2UgaXRzIHdheSBpbnRvIDE1LjAgcmVsZWFz ZS4KPgo+IFRoYW5rcy4KPiBPbiBGcmlkYXksIE5vdmVtYmVyIDI5dGgsIDIwMjQgYXQgOTozOCBQ TSwgV2FybmVyIExvc2ggPGltcEBic2RpbXAuY29tPiB3cm90ZToKPgo+PiBJJ3ZlIGJlZW4gc3dh bXBlZC4gd2UgbmVlZCB0byBib290c3RyYXAgdGhlIHZlbmRvciBicmFuY2gsIGFuZCB0aGUgd2F5 IHByaW9yIHVwZGF0ZXMgd2VyZSBkb25lCj4+IGlzbid0IHNvIGdyZWF0Lgo+Pgo+PiBXYXJuZXIK Pj4KPj4gT24gTW9uLCBOb3YgMjUsIDIwMjQgYXQgMjoyMeKAr0FNIGNnbG9naWMgPGNnbG9naWNA cHJvdG9ubWFpbC5jb20+IHdyb3RlOgo+Pgo+Pj4gSGVsbG8gZ3V5cywKPj4+Cj4+PiBIb3cgdGhl IHVwZGF0ZSBvZiBqZW1hbGxvYyBpcyBnb2luZz8gSXQncyBOb3ZlbWJlciBub3cuCj4+Pgo+Pj4g VGhhbmtzLgo+Pj4gT24gTW9uZGF5LCBKdWx5IDIybmQsIDIwMjQgYXQgNzowMiBQTSwgTWluc29v IENob28gPG1pbnNvb2Nob28wMTIyQHByb3Rvbi5tZT4gd3JvdGU6Cj4+Pgo+Pj4+IEZpcnN0LCBz b3JyeSBmb3IgbGF0ZSByZXNwb25zZS4KPj4+Pgo+Pj4+IGNnbG9naWMsIHRoYW5rIHlvdSBmb3Ig YnJpbmdpbmcgdXAgdGhpcyBpc3N1ZSBhZ2FpbiBzaW5jZSBJIG5lYXJseSBmb3Jnb3QgdGhhdCB0 aGlzIGlzc3VlIHdhcyBzdGlsbCBvcGVuLgo+Pj4+Cj4+Pj4gV2FybmVyLCBhcyBJIGNhbid0IGFj Y2VzcyB0byBteSBGcmVlQlNEIGluc3RhbmNlIHVudGlsIHRoZSBlbmQgb2YgQXVndXN0LCBidXQg SSBjYW4gc3RpbGwgZWRpdCBhbmQgcHVzaCB0aGUgY29kZSB0aHJvdWdoIG15IEFybSBNYWMuIFRo aXMgbWVhbnMgdGhhdCBJIGNhbid0IHRlc3QgdGhlIHVwZGF0ZWQgY29kZSBvbiBteSBtYWNoaW5l LCBidXQgSSBjYW4gam9pbiB0aGUgcmV2aWV3IHByb2Nlc3MgYW5kIGxpc3RlbiB0byBjaGFuZ2Ug cHJvcG9zYWxzLgo+Pj4+Cj4+Pj4gSSdsbCBvcGVuIGEgR2l0aHViIFBSIGluIGEgZmV3IGhvdXJz LiAoVGhlIHBoYWJyaWNhdG9yIHJldmlldyB3aWxsIHN0YXkgb3BlbmVkIGp1c3QgaW4gY2FzZSkK Pj4+PiBPbiBNb25kYXksIEp1bHkgMjJuZCwgMjAyNCBhdCA1OjA4IEFNLCBXYXJuZXIgTG9zaCA8 aW1wQGJzZGltcC5jb20+IHdyb3RlOgo+Pj4+Cj4+Pj4+IE9uIFN1biwgSnVsIDIxLCAyMDI0IGF0 IDI6MDPigK9QTSBjZ2xvZ2ljIDxjZ2xvZ2ljQHByb3Rvbm1haWwuY29tPiB3cm90ZToKPj4+Pj4K Pj4+Pj4+IE9uIFN1bmRheSwgSnVseSAyMXN0LCAyMDI0IGF0IDY6NTQgQU0sIFdhcm5lciBMb3No IDxpbXBAYnNkaW1wLmNvbT4gd3JvdGU6Cj4+Pj4+Pgo+Pj4+Pj4+IE9uIFNhdCwgSnVsIDIwLCAy MDI0IGF0IDE6NTnigK9BTSBjZ2xvZ2ljIDxjZ2xvZ2ljQHByb3Rvbm1haWwuY29tPiB3cm90ZToK Pj4+Pj4+Pgo+Pj4+Pj4+PiBIZWxsbyBGcmVlQlNEIGNvbW11bml0eSwKPj4+Pj4+Pj4KPj4+Pj4+ Pj4gQWZ0ZXIgSmFzb24gRXZhbnMgc3RlcHBlZCBhc2lkZSBmcm9tIG1haW50YWluaW5nIGplbWFs bG9jIGluIEZyZWVCU0QsIGl0J3Mgbm90IHVwZGF0aW5nIGluIHRpbWUgYW55bW9yZS4KPj4+Pj4+ Pj4gVmVyc2lvbiA1LjMuMCB3YXMgcmVsZWFzZWQgTWF5IDYsIDIwMjIgYW5kIEZyZWVCU0Qgc3Rp bGwgbm90IGltcG9ydGVkIGl0IGludG8gdGhlIHRyZWUuCj4+Pj4+Pj4+Cj4+Pj4+Pj4+IFRoZXJl IGlzIGEgcGVuZGluZyByZXZpZXcgaHR0cHM6Ly9yZXZpZXdzLmZyZWVic2Qub3JnL0Q0MTQyMSBm cm9tIEF1ZyAxMSwgMjAyMy4KPj4+Pj4+Pj4gSSdtIHN1Y2Nlc3NmdWxseSBydW5uaW5nIEZyZWVC U0QvYW1kNjQgc3lzdGVtIHdpdGggRDQxNDIxIGFwcGxpZWQgZm9yIDggbW9udGhzLCBhcyB3ZWxs IGFzIG1hbnkgb3RoZXIgcGVvcGxlLgo+Pj4+Pj4+Pgo+Pj4+Pj4+PiBDYW4gaXQgYmUgcmV2aWV3 ZWQgYW5kIGNvbW1pdHRlZCB0byBDVVJSRU5UPwo+Pj4+Pj4+PiBPciwgaWYgdGhlcmUgaXMgbm8g Y29tbWl0dGVycyB3aWxsaW5nIHRvIGRvIGl0LCBjYW4gY29tbWl0IGJpdCBiZSBnaXZlbiB0byBz dWJtaXR0ZXIgb3IgYW5vdGhlciBwZXJzb24gd2lsbGluZyB0byBkbyB0aGlzPwo+Pj4+Pj4+Pgo+ Pj4+Pj4+PiBJdCdzIHZlcnkgZGlzYXBwb2ludGluZyB3aGVuIHVzZXJzIHNwZW5kIHRoZWlyIHRp bWUgdG8gZmlsbCBzdWNoIGdhcHMgYW5kIHRoZWlyIGVmZm9ydHMganVzdCBpZ25vcmVkIGJ5IHRo ZSBkZXZlbG9wZXJzLgo+Pj4+Pj4+Pgo+Pj4+Pj4+PiBFdmVyeSB5ZWFyIEZyZWVCU0QgQ29tbXVu aXR5IFN1cnZleSBhc2tpbmcgYWJvdXQgdXNlciBleHBlcmllbmNlIGluIGNvbnRyaWJ1dGluZyB0 byBGcmVlQlNELgo+Pj4+Pj4+Pgo+Pj4+Pj4+PiBIZXJlIHlvdSBjYW4gc2VlIGFuIGV4YW1wbGUg b2Ygc3VjaCBjb250cmlidXRpbmcuCj4+Pj4+Pj4KPj4+Pj4+PiBGaXJzdCwgdGhhbmsgeW91IGZv ciBiZWluZyBwZXJzaXN0ZW50IGFuZCBjb250aW51aW5nIHRvIGJyaW5nIGl0IHVwLiBJdCdzIGlt cG9ydGFudCB0byBkbyB0aGF0IHRvIG1ha2Ugc3VyZSB0aGlzIChhbmQgeW91ciBtYW55IG90aGVy KSBjb250cmlidXRpb24gZG9lc24ndCBmYWxsIG9uIHRoZSBmbG9vci4KPj4+Pj4+Pgo+Pj4+Pj4+ IEFuZCB0byBiZSBmYWlyLCB3ZSdyZSBvbmx5IDMgbW9udGhzIHNpbmNlIHRoZSBsYXN0IHVwZGF0 ZS4gU3RpbGwsIHF1aXRlIGEgYml0IGxvbmdlciB0aGFuIHlvdSBzaG91bGQgaGF2ZSB0byB3YWl0 LCBidXQgbm90IG5lYXJseSB0aGUgeWVhciB0aGUgb3JpZ2luYWwgZGF0ZSBzdWdnZXN0cy4KPj4+ Pj4+Pgo+Pj4+Pj4+IEFuZCB0aGlzIGlzIGEgcGVyZmVjdCBzdG9ybSBvZiAiaG93IHRoZSBwcm9q ZWN0IGlzIGJhZCBhdCBhY2NlcHRpbmcgY29udHJpYnV0aW9ucyI6Cj4+Pj4+Pj4gKDEpIFRoZSBv cmlnaW5hbCBzdWJtaXNzaW9uIHdhcyBjbG9zZSB0byB0aGUgMTQgYnJhbmNoIGNyZWF0aW9uIHRp bWUuIFRoaXMgbWVhbnQgdGhhdCB3ZSB3ZXJlbid0IHdlbGwgcHJlcGFyZWQgdG8gbG9vayBhdCBp dCBzaW5jZSBpdCBpcyBzdWNoIGFuIGludmFzaXZlIGNoYW5nZSAoYXQgbGVhc3Qgb24gaXRzIHN1 cmZhY2UpLiBJdCBhbHNvIHNsb3dlZCB0aGUgaW5pdGlhbCByZXNwb25zZS4uLgo+Pj4+Pj4+ICgy KSBUaGVyZSB3YXMgYSBudW1iZXIgb2YgYmFjayBhbmQgZm9ydGggcmVxdWVzdHMgZm9yIGNoYW5n ZXMsIHdoaWNoIHRvb2sgdGltZSB0byBzb3J0IG91dC4uLgo+Pj4+Pj4+ICgzKSBUaGUgc2l6ZSBv ZiB0aGlzIGlzIGh1Z2UsIHdlbGwgYmV5b25kIHRoZSBjYXBhY2l0eSBvZiBQaGFicmljYXRvciB0 byByZXZpZXcgYWNjdXJhdGVseS4uLgo+Pj4+Pj4+ICg0KSBJdCdzIGEgdmVuZG9yIGltcG9ydC4g VGhhdCBtZWFucyB3ZSBjYW4ndCBqdXN0IGRyb3AgdGhlIFBoYWJyaWNhdG9yIHJldmlldyBpbnRv IHRoZSB0cmVlLi4uCj4+Pj4+Pj4gKDUpIEl0J3MgcGhhYnJpY2F0b3I6IHRoaXMgaXMgYSBncmVh dCB0b29sIGZvciBkZXZlbG9wZXJzLCBidXQgd2UgaGF2ZSBhIHRlcnJpYmxlIHRyYWNrIHJlY29y ZCBvZiB1c2luZyBpdCBmb3IgaW50YWtlIGZyb20gbmV3IGNvbnRyaWJ1dG9ycy4gV2UgZG9uJ3Qg aGF2ZSBhbnkgb3ZlcnNpZ2h0IGF0IGFsbCBvdmVyIHRoaXMgdG9vbCwgYXQgdGhlcmUncyBhdCBi ZXN0IHRlcGlkIGFuZCBsdWtlIHdhcm0gYXR0ZW1wdHMgdG8gbG9vayBmb3IgZHJvcCBiYWxscy4K Pj4+Pj4+Pgo+Pj4+Pj4+IEFsbCBvZiB0aGVzZSB0aGluZ3MgYXJlIGEgdGVycmlibGUgZXhwZXJp ZW5jZS4gSSBjYW4gb25seSBhcG9sb2dpemUuIFRoZXNlIGRheXMsIHdlIG1pZ2h0IHN0ZWVyIHRo aXMgdG93YXJkcyBnaXRodWIsIGJ1dCB0aGUgJ3ZlbmRvciBpbXBvcnQnIG1lYW5zIHlvdSByZWFs bHkgbmVlZCBzb21lb25lIG9uIHRoZSBpbnNpZGUsIG9yIHlvdSBuZWVkIHRvIGJlIG9uIHRoZSBp bnNpZGUgdG8gbWFrZSB0aGF0IHdvcmsuCj4+Pj4+Pj4KPj4+Pj4+PiBTbywgaG93IHRvIG1vdmUg Zm9yd2FyZD8gV2VsbCwgSSdkIGxpa2UgdG8gcHJvcG9zZSB0aGUgZm9sbG93aW5nOgo+Pj4+Pj4+ ICgxKSBzdWJtaXQgYWxsIHRoZSBvdGhlciBQaGFicmljYXRvciByZXZpZXdzIHlvdSBoYXZlIG9w ZW4gKHRoZXkgYXJlIG1vc3RseSBnb29kLCBvciBjbG9zZSB0byBnb29kKSB0byBnaXRodWIuIEdp dGh1YiBpcyBiZWluZyBhY3RpdmVseSBtYW5hZ2VkIGFuZCB3aWxsIG1ha2UgaXQgZmFzdGVyIHRv IGdldCB0aGluZ3MgaXQuIEl0J3MgYSBtdWNoIGJldHRlciB0b29sIGZvciBuZXcgY29udHJpYnV0 b3JzIChhbmQgZXZlbiBmcmVxdWVudCBjb250cmlidXRvcnMgb2Ygc21hbGxpc2ggdGhpbmdzKS4K Pj4+Pj4+PiAoMikgSSBzaG91bGQgZG8gYW4gdmVuZG9yIGltcG9ydCBvZiA1LjMuMCBmcm9tIGdp dGh1YiwgYW5kIGRvIHRoZSBtZXJnZSB0byBhIGJyYW5jaCBhbmQgcHVzaCB0aGF0IHRvIGdpdGh1 Yi4gWW91IGNhbiB0aGVuIGxheWVyIG9uIHlvdXIgY2hhbmdlcyBhbmQgdGhvc2UgY2FuIGJlIHJl dmlld2VkIG1vcmUgY2xvc2VseSBhcyBhIHB1bGwgcmVxdWVzdCBhZ2FpbnN0IHRoZSBicmFuY2gg SSBwdXNoLiBJIHN1c3BlY3QgdGhhdCBtb3N0IG9mIHRoZSBpc3N1ZXMgYXJlIHNvcnRlZCBvdXQg YWxyZWFkeQo+Pj4+Pj4+ICgzKSBJJ2xsIGxhbmQgaXQgdmlhIHRoYXQgcm91dGUuLi4KPj4+Pj4+ Pgo+Pj4+Pj4+IEFuZCwgaWYgdGhlIHN1bSBvZiB0aGUgb3RoZXIgcHVsbCByZXF1ZXN0cyBhbmQg dGhpcyBhcmUgZ29vZCAoYW5kIEkgc3VzcGVjdCB0aGV5IHdpbGwgYmUpLCB0aGVuIHdlIGNhbiB0 YWxrIGFib3V0IGNvbW1pdCBiaXRzIGFuZCBzdWNoLgo+Pj4+Pj4+Cj4+Pj4+Pj4gSXQncyBleHBl cmllbmNlcyBsaWtlIHRoaXMgd2hpY2ggaXMgd2h5IEknbSB0cnlpbmcgdG8gc3RhbmQgdXAgZ2l0 aHViIHB1bGwgcmVxdWVzdHMgYXMgYSByZWxpYWJsZSB3YXkgdG8gZ2V0IHRoaW5ncyBhbmQgYW5k IHRoZSBiZXN0IHBsYWNlIHRvIHNlbmQgcGVvcGxlLi4uCj4+Pj4+Pj4KPj4+Pj4+PiBUaGFua3Mg YWdhaW4gZm9yIHBlcnNpc3RpbmcsIGFuZCBhbHNvIGZvciBleHByZXNzaW5nIHRoaXMgY3JpdGlj aXNtIHRoYXQgd2UgKGhvcGVmdWxseSkgY2FuIHVzZSB0byBtYWtlIGl0IGJldHRlci4KPj4+Pj4+ Pgo+Pj4+Pj4+IFdhcm5lcgo+Pj4+Pj4KPj4+Pj4+IEhlbGxvLgo+Pj4+Pj4KPj4+Pj4+IEknbSBu b3QgdGhlIGF1dGhvciBvZiBENDE0MjEuIEp1c3QgYXBwbGllZCB0aGUgcGF0Y2ggdG8gdGVzdCBp dCA4IG1vbnRocyBhZ28uIEFuZCByZWNlbnRseSBkaXNjb3ZlcmVkIHRoYXQgaXQncyBzdGlsbCBu b3QgY29tbWl0dGVkLgo+Pj4+Pj4gSSBjYW4ndCBjb3B5IHlvdXIgbWVzc2FnZSB0byBQaGFicmlj YXRvciBiZWNhdXNlIGRvbid0IGhhdmUgYW4gYWNjb3VudC4gUGxlYXNlLCBpZiB5b3UgaGF2ZSB0 aW1lLCBoZWxwIHRoZSBhdXRob3IgaW4gRDQxNDIxLgo+Pj4+Pgo+Pj4+PiBBaCB5ZXMuIEkndmUg YmVlbiBpbiB0b3VjaCB3aXRoIHRoZSBhdXRob3IgZm9yIG90aGVyIHRoaW5ncywgYW5kIHNvbWVo b3cgdGhvdWdodCBpdCB3YXMgeW91Li4uLiBJJ2xsIHJlYWNoIG91dCB0byBoaW0gdmlhIG90aGVy IG1lYW5zLi4uCj4+Pj4+Cj4+Pj4+IFdhcm5lcg== --b1=_63fffS1uc9klJoy0NA5T6JyR8TtrsJ8bYUnVUHbeUbU Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: base64 PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0 cHg7IGNvbG9yOiByZ2IoMCwgMCwgMCk7IGJhY2tncm91bmQtY29sb3I6IHJnYigyNTUsIDI1NSwg MjU1KTsiPkkgaGF2ZSBhbHJlYWR5IHN1Ym1pdHRlZCBQUiBvbiBnaXRodWIgKDxzcGFuPjxhIHRh cmdldD0iX2JsYW5rIiByZWw9Im5vcmVmZXJyZXIgbm9mb2xsb3cgbm9vcGVuZXIiIGhyZWY9Imh0 dHBzOi8vZ2l0aHViLmNvbS9mcmVlYnNkL2ZyZWVic2Qtc3JjL3B1bGwvMTMzNyI+aHR0cHM6Ly9n aXRodWIuY29tL2ZyZWVic2QvZnJlZWJzZC1zcmMvcHVsbC8xMzM3PC9hPjwvc3Bhbj4pIGFuZCBw aGFicmljYXRvciAoPHNwYW4+PGEgdGFyZ2V0PSJfYmxhbmsiIHJlbD0ibm9yZWZlcnJlciBub2Zv bGxvdyBub29wZW5lciIgaHJlZj0iaHR0cHM6Ly9yZXZpZXdzLmZyZWVic2Qub3JnL0Q0MTQyMSI+ aHR0cHM6Ly9yZXZpZXdzLmZyZWVic2Qub3JnL0Q0MTQyMTwvYT48L3NwYW4+KS4gSSBkb24ndCBo YXZlIGFjY2VzcyAoY29tbWl0IGJpdCkgdG8gZnJlZWJzZCBnaXQgcmVwbywgc28gdGhlcmUgaXMg bm90aGluZyBJIGNhbiBkbyBhdCB0aGlzIHBvaW50IHNpbmNlIHZlbmRvciBpbXBvcnQgYW5kIGxh bmRpbmcgcGF0Y2hlcyByZXF1aXJlcyBjb21taXQgYml0Ljxicj48L2Rpdj48ZGl2IGNsYXNzPSJw cm90b25tYWlsX3F1b3RlIj4NCiAgICAgICAgT24gU2F0dXJkYXksIE5vdmVtYmVyIDMwdGgsIDIw MjQgYXQgMTo0MiBQTSwgY2dsb2dpYyAmbHQ7Y2dsb2dpY0Bwcm90b25tYWlsLmNvbSZndDsgd3Jv dGU6PGJyPg0KICAgICAgICA8YmxvY2txdW90ZSBjbGFzcz0icHJvdG9ubWFpbF9xdW90ZSIgdHlw ZT0iY2l0ZSI+DQogICAgICAgICAgICA8ZGl2IHN0eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNh bnMtc2VyaWY7IGZvbnQtc2l6ZTogMTRweDsiPkkgc2VlLCBpdCBoYXBwZW5zLjwvZGl2PjxkaXYg c3R5bGU9ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxNHB4OyI+ PHNwYW4+TWF5YmUgYW5vdGhlciBjb21taXR0ZXIgd2lsbCB2b2x1bnRlZXIgdG8gZG8gdGhlIHVw ZGF0ZS48L3NwYW4+PGRpdj5JIGhvcGUgaXQgd2lsbCBtYWtlIGl0cyB3YXkgaW50byAxNS4wIHJl bGVhc2UuPC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj5UaGFua3MuPC9kaXY+PC9kaXY+PGRpdiBj bGFzcz0icHJvdG9ubWFpbF9xdW90ZSI+DQogICAgICAgIE9uIEZyaWRheSwgTm92ZW1iZXIgMjl0 aCwgMjAyNCBhdCA5OjM4IFBNLCBXYXJuZXIgTG9zaCAmbHQ7aW1wQGJzZGltcC5jb20mZ3Q7IHdy b3RlOjxicj4NCiAgICAgICAgPGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSIgY2xhc3M9InByb3Rvbm1h aWxfcXVvdGUiPg0KICAgICAgICAgICAgPGRpdiBkaXI9Imx0ciI+SSd2ZSBiZWVuIHN3YW1wZWQu IHdlIG5lZWQgdG8gYm9vdHN0cmFwIHRoZSB2ZW5kb3IgYnJhbmNoLCBhbmQgdGhlIHdheSBwcmlv ciB1cGRhdGVzIHdlcmUgZG9uZTxkaXY+aXNuJ3Qgc28gZ3JlYXQuIDwvZGl2PjxkaXY+PGJyPjwv ZGl2PjxkaXY+V2FybmVyPC9kaXY+PC9kaXY+PGJyPjxkaXYgY2xhc3M9ImdtYWlsX3F1b3RlIGdt YWlsX3F1b3RlX2NvbnRhaW5lciI+PGRpdiBkaXI9Imx0ciIgY2xhc3M9ImdtYWlsX2F0dHIiPk9u IE1vbiwgTm92IDI1LCAyMDI0IGF0IDI6MjHigK9BTSBjZ2xvZ2ljICZsdDs8YSByZWw9Im5vcmVm ZXJyZXIgbm9mb2xsb3cgbm9vcGVuZXIiIGhyZWY9Im1haWx0bzpjZ2xvZ2ljQHByb3Rvbm1haWwu Y29tIj5jZ2xvZ2ljQHByb3Rvbm1haWwuY29tPC9hPiZndDsgd3JvdGU6PGJyPjwvZGl2PjxibG9j a3F1b3RlIGNsYXNzPSJnbWFpbF9xdW90ZSIgc3R5bGU9Im1hcmdpbjowcHggMHB4IDBweCAwLjhl eDtib3JkZXItbGVmdDoxcHggc29saWQgcmdiKDIwNCwyMDQsMjA0KTtwYWRkaW5nLWxlZnQ6MWV4 Ij48ZGl2IHN0eWxlPSJmb250LWZhbWlseTpBcmlhbCxzYW5zLXNlcmlmO2ZvbnQtc2l6ZToxNHB4 Ij5IZWxsbyBndXlzLDwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OkFyaWFsLHNhbnMtc2Vy aWY7Zm9udC1zaXplOjE0cHgiPjxicj48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTpBcmlh bCxzYW5zLXNlcmlmO2ZvbnQtc2l6ZToxNHB4Ij5Ib3cgdGhlIHVwZGF0ZSBvZiBqZW1hbGxvYyBp cyBnb2luZz8gSXQncyBOb3ZlbWJlciBub3cuPC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6 QXJpYWwsc2Fucy1zZXJpZjtmb250LXNpemU6MTRweCI+PGJyPjwvZGl2PjxkaXYgc3R5bGU9ImZv bnQtZmFtaWx5OkFyaWFsLHNhbnMtc2VyaWY7Zm9udC1zaXplOjE0cHgiPlRoYW5rcy48L2Rpdj48 ZGl2Pg0KICAgICAgICBPbiBNb25kYXksIEp1bHkgMjJuZCwgMjAyNCBhdCA3OjAyIFBNLCBNaW5z b28gQ2hvbyAmbHQ7PGEgcmVsPSJub3JlZmVycmVyIG5vZm9sbG93IG5vb3BlbmVyIiBocmVmPSJt YWlsdG86bWluc29vY2hvbzAxMjJAcHJvdG9uLm1lIiB0YXJnZXQ9Il9ibGFuayI+bWluc29vY2hv bzAxMjJAcHJvdG9uLm1lPC9hPiZndDsgd3JvdGU6PGJyPg0KICAgICAgICA8YmxvY2txdW90ZSB0 eXBlPSJjaXRlIj4NCiAgICAgICAgICAgIDxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OkFyaWFsLHNh bnMtc2VyaWY7Zm9udC1zaXplOjE0cHgiPkZpcnN0LCBzb3JyeSBmb3IgbGF0ZSByZXNwb25zZS48 L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTpBcmlhbCxzYW5zLXNlcmlmO2ZvbnQtc2l6ZTox NHB4Ij48YnI+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6QXJpYWwsc2Fucy1zZXJpZjtm b250LXNpemU6MTRweCI+Y2dsb2dpYywgdGhhbmsgeW91IGZvciBicmluZ2luZyB1cCB0aGlzIGlz c3VlIGFnYWluIHNpbmNlIEkgbmVhcmx5IGZvcmdvdCB0aGF0IHRoaXMgaXNzdWUgd2FzIHN0aWxs IG9wZW4uPC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6QXJpYWwsc2Fucy1zZXJpZjtmb250 LXNpemU6MTRweCI+PGJyPjwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OkFyaWFsLHNhbnMt c2VyaWY7Zm9udC1zaXplOjE0cHgiPldhcm5lciwgYXMgSSBjYW4ndCBhY2Nlc3MgdG8gbXkgRnJl ZUJTRCBpbnN0YW5jZSB1bnRpbCB0aGUgZW5kIG9mIEF1Z3VzdCwgYnV0IEkgY2FuIHN0aWxsIGVk aXQgYW5kIHB1c2ggdGhlIGNvZGUgdGhyb3VnaCBteSBBcm0gTWFjLiBUaGlzIG1lYW5zIHRoYXQg SSBjYW4ndCB0ZXN0IHRoZSB1cGRhdGVkIGNvZGUgb24gbXkgbWFjaGluZSwgYnV0IEkgY2FuIGpv aW4gdGhlIHJldmlldyBwcm9jZXNzIGFuZCBsaXN0ZW4gdG8gY2hhbmdlIHByb3Bvc2Fscy48L2Rp dj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTpBcmlhbCxzYW5zLXNlcmlmO2ZvbnQtc2l6ZToxNHB4 Ij48YnI+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6QXJpYWwsc2Fucy1zZXJpZjtmb250 LXNpemU6MTRweCI+SSdsbCBvcGVuIGEgR2l0aHViIFBSIGluIGEgZmV3IGhvdXJzLiAoVGhlIHBo YWJyaWNhdG9yIHJldmlldyB3aWxsIHN0YXkgb3BlbmVkIGp1c3QgaW4gY2FzZSk8L2Rpdj48ZGl2 Pg0KICAgICAgICBPbiBNb25kYXksIEp1bHkgMjJuZCwgMjAyNCBhdCA1OjA4IEFNLCBXYXJuZXIg TG9zaCAmbHQ7PGEgcmVsPSJub3JlZmVycmVyIG5vZm9sbG93IG5vb3BlbmVyIiBocmVmPSJtYWls dG86aW1wQGJzZGltcC5jb20iIHRhcmdldD0iX2JsYW5rIj5pbXBAYnNkaW1wLmNvbTwvYT4mZ3Q7 IHdyb3RlOjxicj4NCiAgICAgICAgPGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSI+DQogICAgICAgICAg ICA8ZGl2IGRpcj0ibHRyIj48ZGl2IGRpcj0ibHRyIj48YnI+PC9kaXY+PGJyPjxkaXYgY2xhc3M9 ImdtYWlsX3F1b3RlIj48ZGl2IGRpcj0ibHRyIiBjbGFzcz0iZ21haWxfYXR0ciI+T24gU3VuLCBK dWwgMjEsIDIwMjQgYXQgMjowM+KAr1BNIGNnbG9naWMgJmx0OzxhIHJlbD0ibm9yZWZlcnJlciBu b2ZvbGxvdyBub29wZW5lciIgaHJlZj0ibWFpbHRvOmNnbG9naWNAcHJvdG9ubWFpbC5jb20iIHRh cmdldD0iX2JsYW5rIj5jZ2xvZ2ljQHByb3Rvbm1haWwuY29tPC9hPiZndDsgd3JvdGU6PGJyPjwv ZGl2PjxibG9ja3F1b3RlIGNsYXNzPSJnbWFpbF9xdW90ZSIgc3R5bGU9Im1hcmdpbjowcHggMHB4 IDBweCAwLjhleDtib3JkZXItbGVmdDoxcHggc29saWQgcmdiKDIwNCwyMDQsMjA0KTtwYWRkaW5n LWxlZnQ6MWV4Ij48ZGl2IHN0eWxlPSJmb250LWZhbWlseTpBcmlhbCxzYW5zLXNlcmlmO2ZvbnQt c2l6ZToxNHB4Ij48YnI+PC9kaXY+PGRpdj4NCiAgICAgICAgT24gU3VuZGF5LCBKdWx5IDIxc3Qs IDIwMjQgYXQgNjo1NCBBTSwgV2FybmVyIExvc2ggJmx0OzxhIHJlbD0ibm9yZWZlcnJlciBub2Zv bGxvdyBub29wZW5lciIgaHJlZj0ibWFpbHRvOmltcEBic2RpbXAuY29tIiB0YXJnZXQ9Il9ibGFu ayI+aW1wQGJzZGltcC5jb208L2E+Jmd0OyB3cm90ZTo8YnI+DQogICAgICAgIDxibG9ja3F1b3Rl IHR5cGU9ImNpdGUiPg0KICAgICAgICAgICAgPGRpdiBkaXI9Imx0ciI+PGRpdiBkaXI9Imx0ciI+ PGJyPjwvZGl2Pjxicj48ZGl2IGNsYXNzPSJnbWFpbF9xdW90ZSI+PGRpdiBkaXI9Imx0ciIgY2xh c3M9ImdtYWlsX2F0dHIiPk9uIFNhdCwgSnVsIDIwLCAyMDI0IGF0IDE6NTnigK9BTSBjZ2xvZ2lj ICZsdDs8YSByZWw9Im5vcmVmZXJyZXIgbm9mb2xsb3cgbm9vcGVuZXIiIGhyZWY9Im1haWx0bzpj Z2xvZ2ljQHByb3Rvbm1haWwuY29tIiB0YXJnZXQ9Il9ibGFuayI+Y2dsb2dpY0Bwcm90b25tYWls LmNvbTwvYT4mZ3Q7IHdyb3RlOjxicj48L2Rpdj48YmxvY2txdW90ZSBjbGFzcz0iZ21haWxfcXVv dGUiIHN0eWxlPSJtYXJnaW46MHB4IDBweCAwcHggMC44ZXg7Ym9yZGVyLWxlZnQ6MXB4IHNvbGlk IHJnYigyMDQsMjA0LDIwNCk7cGFkZGluZy1sZWZ0OjFleCI+PGRpdiBzdHlsZT0iZm9udC1mYW1p bHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0cHg7IGNvbG9yOiByZ2IoMCwgMCwg MCk7IGJhY2tncm91bmQtY29sb3I6IHJnYigyNTUsIDI1NSwgMjU1KTsiPkhlbGxvIEZyZWVCU0Qg Y29tbXVuaXR5LDwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJp ZjsgZm9udC1zaXplOiAxNHB4OyBjb2xvcjogcmdiKDAsIDAsIDApOyBiYWNrZ3JvdW5kLWNvbG9y OiByZ2IoMjU1LCAyNTUsIDI1NSk7Ij48YnI+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6 IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0cHg7IGNvbG9yOiByZ2IoMCwgMCwgMCk7 IGJhY2tncm91bmQtY29sb3I6IHJnYigyNTUsIDI1NSwgMjU1KTsiPjxzcGFuIHN0eWxlPSJkaXNw bGF5OiBpbmxpbmU7IGJhY2tncm91bmQtY29sb3I6IHJnYigyNTUsIDI1NSwgMjU1KTsiPkFmdGVy IDwvc3Bhbj48c3BhbiBzdHlsZT0iYmFja2dyb3VuZC1jb2xvcjogcmdiKDI1NSwgMjU1LCAyNTUp OyI+SmFzb24gRXZhbnMgc3RlcHBlZCBhc2lkZSBmcm9tIG1haW50YWluaW5nIGplbWFsbG9jIGlu IEZyZWVCU0QsIGl0J3Mgbm90IHVwZGF0aW5nIGluIHRpbWUgYW55bW9yZS48L3NwYW4+PGJyPjwv ZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9udC1zaXpl OiAxNHB4OyBjb2xvcjogcmdiKDAsIDAsIDApOyBiYWNrZ3JvdW5kLWNvbG9yOiByZ2IoMjU1LCAy NTUsIDI1NSk7Ij5WZXJzaW9uIDUuMy4wIHdhcyByZWxlYXNlZCA8c3Bhbj5NYXkgNiwgMjAyMiBh bmQgRnJlZUJTRCBzdGlsbCBub3QgaW1wb3J0ZWQgaXQgaW50byB0aGUgdHJlZS48L3NwYW4+PC9k aXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6 IDE0cHg7IGNvbG9yOiByZ2IoMCwgMCwgMCk7IGJhY2tncm91bmQtY29sb3I6IHJnYigyNTUsIDI1 NSwgMjU1KTsiPjxzcGFuPjxicj48L3NwYW4+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6 IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0cHg7IGNvbG9yOiByZ2IoMCwgMCwgMCk7 IGJhY2tncm91bmQtY29sb3I6IHJnYigyNTUsIDI1NSwgMjU1KTsiPlRoZXJlIGlzIGEgcGVuZGlu ZyByZXZpZXcgPHNwYW4+PGEgcmVsPSJub3JlZmVycmVyIG5vZm9sbG93IG5vb3BlbmVyIiBocmVm PSJodHRwczovL3Jldmlld3MuZnJlZWJzZC5vcmcvRDQxNDIxIiB0YXJnZXQ9Il9ibGFuayI+aHR0 cHM6Ly9yZXZpZXdzLmZyZWVic2Qub3JnL0Q0MTQyMTwvYT4gZnJvbSA8c3Bhbj5BdWcgMTEsIDIw MjMuPC9zcGFuPjwvc3Bhbj48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNh bnMtc2VyaWY7IGZvbnQtc2l6ZTogMTRweDsgY29sb3I6IHJnYigwLCAwLCAwKTsgYmFja2dyb3Vu ZC1jb2xvcjogcmdiKDI1NSwgMjU1LCAyNTUpOyI+PHNwYW4+PHNwYW4+SSdtIHN1Y2Nlc3NmdWxs eSBydW5uaW5nIEZyZWVCU0QvYW1kNjQgc3lzdGVtIHdpdGggRDQxNDIxIGFwcGxpZWQgZm9yIDgg bW9udGhzLCBhcyB3ZWxsIGFzIG1hbnkgb3RoZXIgcGVvcGxlLjwvc3Bhbj48L3NwYW4+PC9kaXY+ PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0 cHg7IGNvbG9yOiByZ2IoMCwgMCwgMCk7IGJhY2tncm91bmQtY29sb3I6IHJnYigyNTUsIDI1NSwg MjU1KTsiPjxzcGFuPjxzcGFuPjxicj48L3NwYW4+PC9zcGFuPjwvZGl2PjxkaXYgc3R5bGU9ImZv bnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxNHB4OyBjb2xvcjogcmdi KDAsIDAsIDApOyBiYWNrZ3JvdW5kLWNvbG9yOiByZ2IoMjU1LCAyNTUsIDI1NSk7Ij48c3Bhbj48 c3Bhbj5DYW4gaXQgYmUgcmV2aWV3ZWQgYW5kIGNvbW1pdHRlZCB0byBDVVJSRU5UPzwvc3Bhbj48 L3NwYW4+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBm b250LXNpemU6IDE0cHg7IGNvbG9yOiByZ2IoMCwgMCwgMCk7IGJhY2tncm91bmQtY29sb3I6IHJn YigyNTUsIDI1NSwgMjU1KTsiPjxzcGFuPjxzcGFuPk9yLCBpZiB0aGVyZSBpcyBubyBjb21taXR0 ZXJzIHdpbGxpbmcgdG8gZG8gaXQsIGNhbiBjb21taXQgYml0IGJlIGdpdmVuIHRvIHN1Ym1pdHRl ciBvciBhbm90aGVyIHBlcnNvbiB3aWxsaW5nIHRvIGRvIHRoaXM/PC9zcGFuPjwvc3Bhbj48L2Rp dj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNhbnMtc2VyaWY7IGZvbnQtc2l6ZTog MTRweDsgY29sb3I6IHJnYigwLCAwLCAwKTsgYmFja2dyb3VuZC1jb2xvcjogcmdiKDI1NSwgMjU1 LCAyNTUpOyI+PHNwYW4+PHNwYW4+PGJyPjwvc3Bhbj48L3NwYW4+PC9kaXY+PGRpdiBzdHlsZT0i Zm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0cHg7IGNvbG9yOiBy Z2IoMCwgMCwgMCk7IGJhY2tncm91bmQtY29sb3I6IHJnYigyNTUsIDI1NSwgMjU1KTsiPjxzcGFu PjxzcGFuPjxzcGFuPkl0J3MgdmVyeSBkaXNhcHBvaW50aW5nIHdoZW4gdXNlcnMgc3BlbmQgdGhl aXIgdGltZSB0byBmaWxsIHN1Y2ggZ2FwcyBhbmQgdGhlaXIgZWZmb3J0cyBqdXN0IGlnbm9yZWQg YnkgdGhlIGRldmVsb3BlcnMuPC9zcGFuPjxicj48L3NwYW4+PC9zcGFuPjwvZGl2PjxkaXYgc3R5 bGU9ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxNHB4OyBjb2xv cjogcmdiKDAsIDAsIDApOyBiYWNrZ3JvdW5kLWNvbG9yOiByZ2IoMjU1LCAyNTUsIDI1NSk7Ij48 c3Bhbj48c3Bhbj48c3Bhbj48c3Bhbj5FdmVyeSB5ZWFyIEZyZWVCU0QgQ29tbXVuaXR5IFN1cnZl eSBhc2tpbmcgYWJvdXQgdXNlciBleHBlcmllbmNlIGluIGNvbnRyaWJ1dGluZyB0byBGcmVlQlNE LiA8L3NwYW4+PGJyPjwvc3Bhbj48L3NwYW4+PC9zcGFuPjwvZGl2PjxkaXYgc3R5bGU9ImZvbnQt ZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxNHB4OyBjb2xvcjogcmdiKDAs IDAsIDApOyBiYWNrZ3JvdW5kLWNvbG9yOiByZ2IoMjU1LCAyNTUsIDI1NSk7Ij48c3Bhbj48c3Bh bj48c3Bhbj48c3Bhbj5IZXJlIHlvdSBjYW4gc2VlIGFuIGV4YW1wbGUgb2Ygc3VjaCBjb250cmli dXRpbmcuPC9zcGFuPjwvc3Bhbj48L3NwYW4+PC9zcGFuPjwvZGl2PjxkaXYgc3R5bGU9ImZvbnQt ZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxNHB4OyBjb2xvcjogcmdiKDAs IDAsIDApOyBiYWNrZ3JvdW5kLWNvbG9yOiByZ2IoMjU1LCAyNTUsIDI1NSk7Ij48c3Bhbj48c3Bh bj48c3Bhbj48L3NwYW4+PC9zcGFuPjwvc3Bhbj48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWls eTogQXJpYWwsIHNhbnMtc2VyaWY7IGZvbnQtc2l6ZTogMTRweDsgY29sb3I6IHJnYigwLCAwLCAw KTsgYmFja2dyb3VuZC1jb2xvcjogcmdiKDI1NSwgMjU1LCAyNTUpOyI+PHNwYW4+PGJyPjwvc3Bh bj48L2Rpdj48L2Jsb2NrcXVvdGU+PGRpdj48YnI+PC9kaXY+PGRpdj5GaXJzdCwgdGhhbmsgeW91 IGZvciBiZWluZyBwZXJzaXN0ZW50IGFuZCBjb250aW51aW5nIHRvIGJyaW5nIGl0IHVwLiBJdCdz IGltcG9ydGFudCB0byBkbyB0aGF0IHRvIG1ha2Ugc3VyZSB0aGlzIChhbmQgeW91ciBtYW55IG90 aGVyKSBjb250cmlidXRpb24gZG9lc24ndCBmYWxsIG9uIHRoZSBmbG9vci48YnI+PC9kaXY+PGRp dj48YnI+PC9kaXY+PGRpdj5BbmQgdG8gYmUgZmFpciwgd2UncmUgb25seSAzIG1vbnRocyBzaW5j ZSB0aGUgbGFzdCB1cGRhdGUuIFN0aWxsLCBxdWl0ZSBhIGJpdCBsb25nZXIgdGhhbiB5b3Ugc2hv dWxkIGhhdmUgdG8gd2FpdCwgYnV0IG5vdCBuZWFybHkgdGhlIHllYXIgdGhlIG9yaWdpbmFsIGRh dGUgc3VnZ2VzdHMuPGJyPjwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+QW5kIHRoaXMgaXMgYSBw ZXJmZWN0IHN0b3JtIG9mICJob3cgdGhlIHByb2plY3QgaXMgYmFkIGF0IGFjY2VwdGluZyBjb250 cmlidXRpb25zIjo8L2Rpdj48ZGl2PigxKSBUaGUgb3JpZ2luYWwgc3VibWlzc2lvbiB3YXMgY2xv c2UgdG8gdGhlIDE0IGJyYW5jaCBjcmVhdGlvbiB0aW1lLiBUaGlzIG1lYW50IHRoYXQgd2Ugd2Vy ZW4ndCB3ZWxsIHByZXBhcmVkIHRvIGxvb2sgYXQgaXQgc2luY2UgaXQgaXMgc3VjaCBhbiBpbnZh c2l2ZSBjaGFuZ2UgKGF0IGxlYXN0IG9uIGl0cyBzdXJmYWNlKS4gSXQgYWxzbyBzbG93ZWQgdGhl IGluaXRpYWwgcmVzcG9uc2UuLi48YnI+PC9kaXY+PGRpdj4oMikgVGhlcmUgd2FzIGEgbnVtYmVy IG9mIGJhY2sgYW5kIGZvcnRoIHJlcXVlc3RzIGZvciBjaGFuZ2VzLCB3aGljaCB0b29rIHRpbWUg dG8gc29ydCBvdXQuLi48L2Rpdj48ZGl2PigzKSBUaGUgc2l6ZSBvZiB0aGlzIGlzIGh1Z2UsIHdl bGwgYmV5b25kIHRoZSBjYXBhY2l0eSBvZiBQaGFicmljYXRvciB0byByZXZpZXcgYWNjdXJhdGVs eS4uLjwvZGl2PjxkaXY+KDQpIEl0J3MgYSB2ZW5kb3IgaW1wb3J0LiBUaGF0IG1lYW5zIHdlIGNh bid0IGp1c3QgZHJvcCB0aGUgUGhhYnJpY2F0b3IgcmV2aWV3IGludG8gdGhlIHRyZWUuLi48L2Rp dj48ZGl2Pig1KSBJdCdzIHBoYWJyaWNhdG9yOiB0aGlzIGlzIGEgZ3JlYXQgdG9vbCBmb3IgZGV2 ZWxvcGVycywgYnV0IHdlIGhhdmUgYSB0ZXJyaWJsZSB0cmFjayByZWNvcmQgb2YgdXNpbmcgaXQg Zm9yIGludGFrZSBmcm9tIG5ldyBjb250cmlidXRvcnMuIFdlIGRvbid0IGhhdmUgYW55IG92ZXJz aWdodCBhdCBhbGwgb3ZlciB0aGlzIHRvb2wsIGF0IHRoZXJlJ3MgYXQgYmVzdCB0ZXBpZCBhbmQg bHVrZSB3YXJtIGF0dGVtcHRzIHRvIGxvb2sgZm9yIGRyb3AgYmFsbHMuPC9kaXY+PGRpdj48YnI+ PC9kaXY+PGRpdj5BbGwgb2YgdGhlc2UgdGhpbmdzIGFyZSBhIHRlcnJpYmxlIGV4cGVyaWVuY2Uu IEkgY2FuIG9ubHkgYXBvbG9naXplLiBUaGVzZSBkYXlzLCB3ZSBtaWdodCBzdGVlciB0aGlzIHRv d2FyZHMgZ2l0aHViLCBidXQgdGhlICd2ZW5kb3IgaW1wb3J0JyBtZWFucyB5b3UgcmVhbGx5IG5l ZWQgc29tZW9uZSBvbiB0aGUgaW5zaWRlLCBvciB5b3UgbmVlZCB0byBiZSBvbiB0aGUgaW5zaWRl IHRvIG1ha2UgdGhhdCB3b3JrLjwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+U28sIGhvdyB0byBt b3ZlIGZvcndhcmQ/IFdlbGwsIEknZCBsaWtlIHRvIHByb3Bvc2UgdGhlIGZvbGxvd2luZzo8L2Rp dj48ZGl2PigxKSBzdWJtaXQgYWxsIHRoZSBvdGhlciBQaGFicmljYXRvciByZXZpZXdzIHlvdSBo YXZlIG9wZW4gKHRoZXkgYXJlIG1vc3RseSBnb29kLCBvciBjbG9zZSB0byBnb29kKSB0byBnaXRo dWIuIEdpdGh1YiBpcyBiZWluZyBhY3RpdmVseSBtYW5hZ2VkIGFuZCB3aWxsIG1ha2UgaXQgZmFz dGVyIHRvIGdldCB0aGluZ3MgaXQuIEl0J3MgYSBtdWNoIGJldHRlciB0b29sIGZvciBuZXcgY29u dHJpYnV0b3JzIChhbmQgZXZlbiBmcmVxdWVudCBjb250cmlidXRvcnMgb2Ygc21hbGxpc2ggdGhp bmdzKS48L2Rpdj48ZGl2PigyKSBJIHNob3VsZCBkbyBhbiB2ZW5kb3IgaW1wb3J0IG9mIDUuMy4w IGZyb20gZ2l0aHViLCBhbmQgZG8gdGhlIG1lcmdlIHRvIGEgYnJhbmNoIGFuZCBwdXNoIHRoYXQg dG8gZ2l0aHViLiBZb3UgY2FuIHRoZW4gbGF5ZXIgb24geW91ciBjaGFuZ2VzIGFuZCB0aG9zZSBj YW4gYmUgcmV2aWV3ZWQgbW9yZSBjbG9zZWx5IGFzIGEgcHVsbCByZXF1ZXN0IGFnYWluc3QgdGhl IGJyYW5jaCBJIHB1c2guIEkgc3VzcGVjdCB0aGF0IG1vc3Qgb2YgdGhlIGlzc3VlcyBhcmUgc29y dGVkIG91dCBhbHJlYWR5IDxicj48L2Rpdj48ZGl2PigzKSBJJ2xsIGxhbmQgaXQgdmlhIHRoYXQg cm91dGUuLi48L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2PkFuZCwgaWYgdGhlIHN1bSBvZiB0aGUg b3RoZXIgcHVsbCByZXF1ZXN0cyBhbmQgdGhpcyBhcmUgZ29vZCAoYW5kIEkgc3VzcGVjdCB0aGV5 IHdpbGwgYmUpLCB0aGVuIHdlIGNhbiB0YWxrIGFib3V0IGNvbW1pdCBiaXRzIGFuZCBzdWNoLjwv ZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+SXQncyBleHBlcmllbmNlcyBsaWtlIHRoaXMgd2hpY2gg aXMgd2h5IEknbSB0cnlpbmcgdG8gc3RhbmQgdXAgZ2l0aHViIHB1bGwgcmVxdWVzdHMgYXMgYSBy ZWxpYWJsZSB3YXkgdG8gZ2V0IHRoaW5ncyBhbmQgYW5kIHRoZSBiZXN0IHBsYWNlIHRvIHNlbmQg cGVvcGxlLi4uICA8YnI+PC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj5UaGFua3MgYWdhaW4gZm9y IHBlcnNpc3RpbmcsIGFuZCBhbHNvIGZvciBleHByZXNzaW5nIHRoaXMgY3JpdGljaXNtIHRoYXQg d2UgKGhvcGVmdWxseSkgY2FuIHVzZSB0byBtYWtlIGl0IGJldHRlci48YnI+PC9kaXY+PGRpdj48 YnI+PC9kaXY+PGRpdj5XYXJuZXI8YnI+PC9kaXY+PC9kaXY+PC9kaXY+DQoNCiAgICAgICAgPC9i bG9ja3F1b3RlPjwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJp ZjsgZm9udC1zaXplOiAxNHB4OyBjb2xvcjogcmdiKDAsIDAsIDApOyBiYWNrZ3JvdW5kLWNvbG9y OiByZ2IoMjU1LCAyNTUsIDI1NSk7Ij48YnI+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6 IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0cHg7IGNvbG9yOiByZ2IoMCwgMCwgMCk7 IGJhY2tncm91bmQtY29sb3I6IHJnYigyNTUsIDI1NSwgMjU1KTsiPkhlbGxvLjwvZGl2PjxkaXYg c3R5bGU9ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxNHB4OyBj b2xvcjogcmdiKDAsIDAsIDApOyBiYWNrZ3JvdW5kLWNvbG9yOiByZ2IoMjU1LCAyNTUsIDI1NSk7 Ij48YnI+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBm b250LXNpemU6IDE0cHg7IGNvbG9yOiByZ2IoMCwgMCwgMCk7IGJhY2tncm91bmQtY29sb3I6IHJn YigyNTUsIDI1NSwgMjU1KTsiPkknbSBub3QgdGhlIGF1dGhvciBvZiA8c3Bhbj5ENDE0MjEuIEp1 c3QgYXBwbGllZCB0aGUgcGF0Y2ggdG8gdGVzdCBpdCA4IG1vbnRocyBhZ28uIEFuZCByZWNlbnRs eSBkaXNjb3ZlcmVkIHRoYXQgaXQncyBzdGlsbCBub3QgY29tbWl0dGVkLjwvc3Bhbj48L2Rpdj48 ZGl2IHN0eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNhbnMtc2VyaWY7IGZvbnQtc2l6ZTogMTRw eDsgY29sb3I6IHJnYigwLCAwLCAwKTsgYmFja2dyb3VuZC1jb2xvcjogcmdiKDI1NSwgMjU1LCAy NTUpOyI+PHNwYW4+SSBjYW4ndCBjb3B5IHlvdXIgbWVzc2FnZSB0byBQaGFicmljYXRvciBiZWNh dXNlIGRvbid0IGhhdmUgYW4gYWNjb3VudC4gPC9zcGFuPlBsZWFzZSwgaWYgeW91IGhhdmUgdGlt ZSwgaGVscCB0aGUgYXV0aG9yIGluIEQ0MTQyMS48L2Rpdj48L2Jsb2NrcXVvdGU+PGRpdj48YnI+ PC9kaXY+PGRpdj5BaCB5ZXMuIEkndmUgYmVlbiBpbiB0b3VjaCB3aXRoIHRoZSBhdXRob3IgZm9y IG90aGVyIHRoaW5ncywgYW5kIHNvbWVob3cgdGhvdWdodCBpdCB3YXMgeW91Li4uLiAgSSdsbCBy ZWFjaCBvdXQgdG8gaGltIHZpYSBvdGhlciBtZWFucy4uLjwvZGl2PjxkaXY+PGJyPjwvZGl2Pjxk aXY+V2FybmVyPC9kaXY+PC9kaXY+PC9kaXY+DQoNCiAgICAgICAgPC9ibG9ja3F1b3RlPjxicj4N CiAgICA8L2Rpdj4NCiAgICAgICAgPC9ibG9ja3F1b3RlPjxicj4NCiAgICA8L2Rpdj48L2Jsb2Nr cXVvdGU+PC9kaXY+DQoNCiAgICAgICAgPC9ibG9ja3F1b3RlPjxicj4NCiAgICA8L2Rpdj4NCiAg ICAgICAgPC9ibG9ja3F1b3RlPjxicj4NCiAgICA8L2Rpdj4= --b1=_63fffS1uc9klJoy0NA5T6JyR8TtrsJ8bYUnVUHbeUbU-- From nobody Sun Dec 1 04:38:47 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Y1DhM6wvFz5gCHL for ; Sun, 01 Dec 2024 04:38:59 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Y1DhM269Cz464Q for ; Sun, 1 Dec 2024 04:38:59 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pf1-x433.google.com with SMTP id d2e1a72fcca58-724fee568aaso3016351b3a.1 for ; Sat, 30 Nov 2024 20:38:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1733027937; x=1733632737; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=cz/N7jaK/7vzf4dqJad3ugKlG0on7Twj2L+jQ2wAU2U=; b=tQZs6OT5Bd/NKk7QdZZtU5VU3TzEUL3Z/9J7nsP7nGnGK3VAuiPifyVBJLEK0XXuJd 5VwhIDq9jLE50pwEaL3FFILVBNahXUwSzhstQfp4O6YY3+orpZEGaoOLNMOQtPyJTsVN w5uyj4LhEVdhqT9QPt2sfQ8yqphFgBFuSwscpXT47ioF4s+ISwD9BQCiMyOjYb9VDmUE ismX2O/kdD7XvlmBwpDCD8EaBk6eZHKtyz1WcKj9ldwBTUuPBpiZUgO1pln/qBt/V9on d1BkmFgGvrams8gZ6E3Si4ZKMxwov7I7hjTLc6cyCmK3OUAxHCBxfmGzZqSkDguIsw/Q Vu8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733027937; x=1733632737; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=cz/N7jaK/7vzf4dqJad3ugKlG0on7Twj2L+jQ2wAU2U=; b=HlWVsn3NB2dXH8lCI++LUQ++pThwYQW68/dZEI9pBp6etXxod9tLuvK8Z6K1yVl4jd r/tb9Q8asSEl7nlTwH1OFm0bUxIuPy7YHV2QilQt+BIpZ95L+mStvLDmPTg9RUFKvKpD G1VVBPPKt5h1SJJBPMea1eyvxgKbCQVsk36InpraM6W/zMU8M5QIvjBSBB8RTWrUjWZf a7o6EU3dUU7fIfmdIarQxqMIJGyXmZ3DfjJDVoCRb/7NJS4wR1vZ6GwIVNz5IpbRlvE8 roGFZNX0SctdrysNijaIplyxHbRo10c4pw8nmy6OnjR4f0Bzm/xUdh4GsGxzLdsyMBOk OGTA== X-Forwarded-Encrypted: i=1; AJvYcCVcHL4K1Dwj+CkL1ljzxpqlHbnlEAF22JBLu1z+99HRJRsIRL8j8AueAopCGg5QPiVaLn0gDFMKi5e7LEOAy2c=@freebsd.org X-Gm-Message-State: AOJu0YxrEQBnRDqC/01HHHgapxaYBFE30XSnt7iK/v+YprIHBPweTpus 0GJD5v1PS5Ln5WhF90HQ09uXpXiblHjTWz1Ks+gZvJsMSJJ9IWDu03RMCYOIPAwx3svSKCMraEM nzIV7v/7BwtxZ0DYUTdrZqiGmplGti7n+e8mmuA== X-Gm-Gg: ASbGnctJ1MQ9I9k2wUEN/6oF6vD5K+ih5E6Pq9SmhnOsh3ONoVdkBK0fdDqupKq2rlV L1R2y+RS9B5Z8xupr6WRZBmU1x61LVsTWCP7YifNeAiqYen/8TIsEPfXYvkZZQQ== X-Google-Smtp-Source: AGHT+IGVzOHpCFn4aZ9afwV8gXJKyg+FSj58gLoZDESZfX1VW0t3Pd+NZsRPI3A4iNATb0zn36sjKTJMlG1q09PCX9Q= X-Received: by 2002:a17:90b:3a90:b0:2ee:ab11:fab2 with SMTP id 98e67ed59e1d1-2eeab11fca1mr1465631a91.22.1733027937444; Sat, 30 Nov 2024 20:38:57 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Sat, 30 Nov 2024 21:38:47 -0700 Message-ID: Subject: Re: Long time outdated jemalloc To: Minsoo Choo Cc: cglogic , FreeBSD CURRENT Content-Type: multipart/alternative; boundary="00000000000098fea706282e028f" X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4Y1DhM269Cz464Q X-Spamd-Bar: ---- --00000000000098fea706282e028f Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Yea, I need to get a copy of jemalloc 5.3.0 and 5.2.1 to try to 'bootstrap' the vendor branch. Then I need to bootstrap it... I just did the same with edk2 (which had a vendor branch, but hadn't been updated since svn times). However, jemalloc doesn't have a vendor branch yet, so I'll have to create that, but I'll start with the current version rather than doing full history... So I'll start there. I also just did awk and lua, so once I have things bootstrapped, I'll be able to add 5.3.0 and then layer Minsoo's on top of that and then start testing it somehow. Malloc makes me nervous to touch, honestly, but I'll give it a go and test boot on my system and maybe see if we can survive a workload at work w/o regressions... But I can't do a full test with lots of machines until after the first of the year (though I can do a couple for a few days before then). So my next step is to bootstrap the vendor branch... I'll give that a try tonight. Warner On Sat, Nov 30, 2024 at 8:26=E2=80=AFPM Minsoo Choo wrote: > I have already submitted PR on github ( > https://github.com/freebsd/freebsd-src/pull/1337) and phabricator ( > https://reviews.freebsd.org/D41421). I don't have access (commit bit) to > freebsd git repo, so there is nothing I can do at this point since vendor > import and landing patches requires commit bit. > On Saturday, November 30th, 2024 at 1:42 PM, cglogic < > cglogic@protonmail.com> wrote: > > I see, it happens. > Maybe another committer will volunteer to do the update. > I hope it will make its way into 15.0 release. > > Thanks. > On Friday, November 29th, 2024 at 9:38 PM, Warner Losh > wrote: > > I've been swamped. we need to bootstrap the vendor branch, and the way > prior updates were done > isn't so great. > > Warner > > On Mon, Nov 25, 2024 at 2:21=E2=80=AFAM cglogic = wrote: > >> Hello guys, >> >> How the update of jemalloc is going? It's November now. >> >> Thanks. >> On Monday, July 22nd, 2024 at 7:02 PM, Minsoo Choo < >> minsoochoo0122@proton.me> wrote: >> >> First, sorry for late response. >> >> cglogic, thank you for bringing up this issue again since I nearly forgo= t >> that this issue was still open. >> >> Warner, as I can't access to my FreeBSD instance until the end of August= , >> but I can still edit and push the code through my Arm Mac. This means th= at >> I can't test the updated code on my machine, but I can join the review >> process and listen to change proposals. >> >> I'll open a Github PR in a few hours. (The phabricator review will stay >> opened just in case) >> On Monday, July 22nd, 2024 at 5:08 AM, Warner Losh >> wrote: >> >> >> >> On Sun, Jul 21, 2024 at 2:03=E2=80=AFPM cglogic = wrote: >> >>> >>> On Sunday, July 21st, 2024 at 6:54 AM, Warner Losh >>> wrote: >>> >>> >>> >>> On Sat, Jul 20, 2024 at 1:59=E2=80=AFAM cglogic wrote: >>> >>>> Hello FreeBSD community, >>>> >>>> After Jason Evans stepped aside from maintaining jemalloc in FreeBSD, >>>> it's not updating in time anymore. >>>> Version 5.3.0 was released May 6, 2022 and FreeBSD still not imported >>>> it into the tree. >>>> >>>> There is a pending review https://reviews.freebsd.org/D41421 from Aug >>>> 11, 2023. >>>> I'm successfully running FreeBSD/amd64 system with D41421 applied for = 8 >>>> months, as well as many other people. >>>> >>>> Can it be reviewed and committed to CURRENT? >>>> Or, if there is no committers willing to do it, can commit bit be give= n >>>> to submitter or another person willing to do this? >>>> >>>> It's very disappointing when users spend their time to fill such gaps >>>> and their efforts just ignored by the developers. >>>> Every year FreeBSD Community Survey asking about user experience in >>>> contributing to FreeBSD. >>>> Here you can see an example of such contributing. >>>> >>>> >>> First, thank you for being persistent and continuing to bring it up. >>> It's important to do that to make sure this (and your many other) >>> contribution doesn't fall on the floor. >>> >>> And to be fair, we're only 3 months since the last update. Still, quite >>> a bit longer than you should have to wait, but not nearly the year the >>> original date suggests. >>> >>> And this is a perfect storm of "how the project is bad at accepting >>> contributions": >>> (1) The original submission was close to the 14 branch creation time. >>> This meant that we weren't well prepared to look at it since it is such= an >>> invasive change (at least on its surface). It also slowed the initial >>> response... >>> (2) There was a number of back and forth requests for changes, which >>> took time to sort out... >>> (3) The size of this is huge, well beyond the capacity of Phabricator t= o >>> review accurately... >>> (4) It's a vendor import. That means we can't just drop the Phabricator >>> review into the tree... >>> (5) It's phabricator: this is a great tool for developers, but we have = a >>> terrible track record of using it for intake from new contributors. We >>> don't have any oversight at all over this tool, at there's at best tepi= d >>> and luke warm attempts to look for drop balls. >>> >>> All of these things are a terrible experience. I can only apologize. >>> These days, we might steer this towards github, but the 'vendor import' >>> means you really need someone on the inside, or you need to be on the >>> inside to make that work. >>> >>> So, how to move forward? Well, I'd like to propose the following: >>> (1) submit all the other Phabricator reviews you have open (they are >>> mostly good, or close to good) to github. Github is being actively mana= ged >>> and will make it faster to get things it. It's a much better tool for n= ew >>> contributors (and even frequent contributors of smallish things). >>> (2) I should do an vendor import of 5.3.0 from github, and do the merge >>> to a branch and push that to github. You can then layer on your changes= and >>> those can be reviewed more closely as a pull request against the branch= I >>> push. I suspect that most of the issues are sorted out already >>> (3) I'll land it via that route... >>> >>> And, if the sum of the other pull requests and this are good (and I >>> suspect they will be), then we can talk about commit bits and such. >>> >>> It's experiences like this which is why I'm trying to stand up github >>> pull requests as a reliable way to get things and and the best place to >>> send people... >>> >>> Thanks again for persisting, and also for expressing this criticism tha= t >>> we (hopefully) can use to make it better. >>> >>> Warner >>> >>> >>> Hello. >>> >>> I'm not the author of D41421. Just applied the patch to test it 8 >>> months ago. And recently discovered that it's still not committed. >>> I can't copy your message to Phabricator because don't have an account.= Please, >>> if you have time, help the author in D41421. >>> >> >> Ah yes. I've been in touch with the author for other things, and somehow >> thought it was you.... I'll reach out to him via other means... >> >> Warner >> >> >> >> > > --00000000000098fea706282e028f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Yea, I need to get a copy of jemalloc 5.3.0 and 5.2.1 to t= ry to 'bootstrap' the vendor branch.
Then I need to bootstrap i= t...

I just did the same with edk2 (which had a ve= ndor branch, but hadn't been updated since svn times).
Howeve= r, jemalloc doesn't have a vendor branch yet, so I'll have to creat= e that, but I'll start with the
current version rather than d= oing full history...=C2=A0 So I'll start there.
I also just d= id awk and lua, so once I have things bootstrapped, I'll be able to add= 5.3.0 and then layer Minsoo's=C2=A0 on
top of that and then = start testing it somehow.

Malloc makes me nervous = to touch, honestly, but I'll give it a go and test boot on my system an= d
maybe see if we can survive a workload at work w/o regressions.= .. But I can't do a full test with lots
of machines until= after the first of the year (though I can do a couple for a few days befor= e then).

So my next step is to bootstrap the vendo= r branch... I'll give that a try tonight.

Warn= er

On Sat, Nov 30, 2024 at 8:26=E2=80=AFPM Minso= o Choo <minsoochoo0122@proto= n.me> wrote:
I have already submitted PR on githu= b (https://github.com/freebs= d/freebsd-src/pull/1337) and phabricator (https://reviews.freebsd.org/D41421). I don't hav= e access (commit bit) to freebsd git repo, so there is nothing I can do at = this point since vendor import and landing patches requires commit bit.
=
On Friday, November 29th, 2024 at 9:38 PM, Warner Losh <imp@bsdimp.com> wrote:<= br>
I've been swamped. we need to bootstrap th= e vendor branch, and the way prior updates were done
isn't so great= .

Warner

On Mon, Nov 25, 2024 at 2:21=E2=80= =AFAM cglogic <cglogic@protonmail.com> wrot= e:
Hello guys,

How the update of jemalloc is go= ing? It's November now.

Thanks.
On Monday, July 22nd, 2024 at 7:02 PM, Minsoo Choo <minsoochoo0122@proton.me> wrote:
Firs= t, sorry for late response.

cglogic, thank you for bringing up this issue again since I near= ly forgot that this issue was still open.

Warner, as I can't access to my FreeBSD instan= ce until the end of August, but I can still edit and push the code through = my Arm Mac. This means that I can't test the updated code on my machine= , but I can join the review process and listen to change proposals.

I'll open a Github P= R in a few hours. (The phabricator review will stay opened just in case)
On Monday, July 22nd, 2024 at 5:08 AM, Warner Losh <imp@bsdimp.com> wrote:


On Sun, Jul 21, 2024 at 2= :03=E2=80=AFPM cglogic <cglogic@protonmail.com= > wrote:

On Sunday, July 21st, 2024 at 6:54 AM, Warner Losh <imp@bsdimp.com> wrote:


On Sat, Jul 20, 2024 at 1= :59=E2=80=AFAM cglogic <cglogic@protonmail.com= > wrote:
Hello FreeBSD community,

After Jason Evans stepped aside from = maintaining jemalloc in FreeBSD, it's not updating in time anymore.
Version 5.3.0 was released = May 6, 2022 and FreeBSD still not imported it into the tree.

There is a pending review https://reviews.freebsd.org/D41421 from Aug 11, 2023.
I'm succes= sfully running FreeBSD/amd64 system with D41421 applied for 8 months, as we= ll as many other people.

= Can it be reviewed and committed to CURRENT?
Or, if there is no committe= rs willing to do it, can commit bit be given to submitter or another person= willing to do this?
=
It's very disappointing when users spend their time to fi= ll such gaps and their efforts just ignored by the developers.
Every year FreeBSD Community Survey asking about user experience in contr= ibuting to FreeBSD.
Here you can see an example of su= ch contributing.


Firs= t, thank you for being persistent and continuing to bring it up. It's i= mportant to do that to make sure this (and your many other) contribution do= esn't fall on the floor.

And to be fair, w= e're only 3 months since the last update. Still, quite a bit longer tha= n you should have to wait, but not nearly the year the original date sugges= ts.

And this is a perfect storm of "how t= he project is bad at accepting contributions":
(1) The origi= nal submission was close to the 14 branch creation time. This meant that we= weren't well prepared to look at it since it is such an invasive chang= e (at least on its surface). It also slowed the initial response...
(2) There was a number of back and forth requests for changes, which= took time to sort out...
(3) The size of this is huge, well beyo= nd the capacity of Phabricator to review accurately...
(4) It'= ;s a vendor import. That means we can't just drop the Phabricator revie= w into the tree...
(5) It's phabricator: this is a great tool= for developers, but we have a terrible track record of using it for intake= from new contributors. We don't have any oversight at all over this to= ol, at there's at best tepid and luke warm attempts to look for drop ba= lls.

All of these things are a terrible experience= . I can only apologize. These days, we might steer this towards github, but= the 'vendor import' means you really need someone on the inside, o= r you need to be on the inside to make that work.

= So, how to move forward? Well, I'd like to propose the following:
=
(1) submit all the other Phabricator reviews you have open (they are m= ostly good, or close to good) to github. Github is being actively managed a= nd will make it faster to get things it. It's a much better tool for ne= w contributors (and even frequent contributors of smallish things).
(2) I should do an vendor import of 5.3.0 from github, and do the merge = to a branch and push that to github. You can then layer on your changes and= those can be reviewed more closely as a pull request against the branch I = push. I suspect that most of the issues are sorted out already
(3) I'll land it via that route...

And, if = the sum of the other pull requests and this are good (and I suspect they wi= ll be), then we can talk about commit bits and such.

It's experiences like this which is why I'm trying to stand up g= ithub pull requests as a reliable way to get things and and the best place = to send people...

Thanks again for persistin= g, and also for expressing this criticism that we (hopefully) can use to ma= ke it better.

Warner

Hello.

I'm not the author of = D41421. Just applied the patch to test it 8 months ago. And recently = discovered that it's still not committed.
I can't copy your message to Phabricator becaus= e don't have an account. Please, if you have time, help the auth= or in D41421.

Ah yes. I've been i= n touch with the author for other things, and somehow thought it was you...= . I'll reach out to him via other means...

Wa= rner




--00000000000098fea706282e028f-- From nobody Sun Dec 1 07:05:01 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Y1Hx61z0Mz5f991 for ; Sun, 01 Dec 2024 07:05:14 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Y1Hx52ZyYz4d4Q for ; Sun, 1 Dec 2024 07:05:13 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20230601.gappssmtp.com header.s=20230601 header.b=jvzUCVGL; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::1032) smtp.mailfrom=wlosh@bsdimp.com; dmarc=none Received: by mail-pj1-x1032.google.com with SMTP id 98e67ed59e1d1-2ee69fc0507so1327179a91.0 for ; Sat, 30 Nov 2024 23:05:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1733036712; x=1733641512; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=H1az5Ie/BvoA3FcOmxV8amiFs09HuKhYuHHSE+ItXc0=; b=jvzUCVGLDqmvyNj+R/rxESvpnbRNxhywzgHF8fAiEnrpSlhLTLkpHY5j6GvQXQRzBo 78+F1nHcxJlnh4ybiVjXe0UYwd0DbYpNIsIYv+YasgpCx8ixe0Gsz9GkiJkvaGIke8Xn uOTKez1guTcrr1tCdQzVzGAibUhlcgcw3DGSv/quxQAiwGTRrT3sJzcUBlyAjy8g3KDo 5jIkZtsWHi4FVJPWCR0tLgCBsRRfeh2ijbmzo0uaSbUBEHhBSLP/O7VJCioHp1fehmg8 WYeI3qVPJZ9pJvyht502jFKa+i2sNHJ31qHrukKUsewbS8Rm+0A0W2t/+1KbaWXZOpJk qXKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733036712; x=1733641512; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=H1az5Ie/BvoA3FcOmxV8amiFs09HuKhYuHHSE+ItXc0=; b=vj+RXtc0eraPXWIlPZhwT1cXxnswQ16ZhbM8wbjDsleXFn3wTzfo8nby45v7UnTZWE 3O/NzLjyFbcOUi8zTmkhEAU2HiwpeqElCdYR/R2SYZqudZJY1TuQd8TdrKjtP5vvFqQq HHCGWXw2J1lmRpZ7l5IQBF4WNTyBfzVjk65B/Zg8r/0WEbcYCxouDj/M4m1uNc8Aqg06 n458o9/FGQD3w4sTvBS5cLldAq+jcciAjA2wpmlaOygNmsb3Tv7oyyf6NVEEz1kIugyX pCiWUFO0IZWyoQsrVmTb3loZrFWd8a/ZqUSzF/7Q5C/2RP9Xxtzr1tWHTKAbrlDZf8mi hZVw== X-Forwarded-Encrypted: i=1; AJvYcCXLFVm3k3NAiCQ3f02MW2XVI0CXVtI+/S8T+Ky3hKMrlLKesV1AZbSnAp6b+BpnvWyB9rupvMxw1/U/G75ARFY=@freebsd.org X-Gm-Message-State: AOJu0Yz5xCSigRMRWPXWJuwRE0slPbfj1cIn8hz/0u9K9gORbSg3wNkH iaH35lo3iVXjL6RsxHlmUziTeHKmSfpR9MCQnDz0rmihA7XG5m17hZ0xwzGCt1Is3v/+6MkqUMu 7ta+B1zOlyZO6arniwQD01KWclu8wwOwyP5zP1kyf+fM/JYLrEmA= X-Gm-Gg: ASbGncsbRYMda/PVCW/C9w5HWlPBPjTmzg2e7LDfVTnJ3tbOm3U+bQQ5tivPjNq8OaB HLho9HNe7Il0iDezj8gh+iRoU+my6RERIjwQL8rIyufESu++VO2ht1pE5ucwLcg== X-Google-Smtp-Source: AGHT+IE4FMgOXD4KVZb335xDvBFrvJzmCXf8TUSQ6w/pKcKI5EhYZaXMCpeQX0oTbmDlq9tRTcNuEH7GM3V4WXlFZgU= X-Received: by 2002:a17:90b:548f:b0:2ea:b564:4b31 with SMTP id 98e67ed59e1d1-2ee08eb7ca7mr23003563a91.19.1733036711763; Sat, 30 Nov 2024 23:05:11 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Sun, 1 Dec 2024 00:05:01 -0700 Message-ID: Subject: Re: Long time outdated jemalloc To: Minsoo Choo Cc: cglogic , FreeBSD CURRENT Content-Type: multipart/alternative; boundary="000000000000967b870628300dde" X-Spamd-Result: default: False [-3.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20230601.gappssmtp.com:s=20230601]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ARC_NA(0.00)[]; MISSING_XM_UA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::1032:from]; R_SPF_NA(0.00)[no SPF record]; RCVD_TLS_LAST(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; FROM_HAS_DN(0.00)[]; FREEMAIL_CC(0.00)[protonmail.com,freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; DKIM_TRACE(0.00)[bsdimp-com.20230601.gappssmtp.com:+] X-Rspamd-Queue-Id: 4Y1Hx52ZyYz4d4Q X-Spamd-Bar: -- --000000000000967b870628300dde Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable (sorry to follow up to my own email and topposting) I got the vendor branch bootstrapped: I created vendor/jemalloc and tagged vendor/jemalloc/5.2.1, and created a merge commit from that branch to main. I had to tweak the FREEBSD-Xlist a little.I've not updated the other two FREEBSD-* files, but the steps were documented in the commit messages (the vendor branch one is especially long and likely should migrate into the developer handbook). FREEBSD-update is basically a shell script to do the same thing that git subtree merge does, though I'm sure some tweaks could help the next time (if there is a next time, jemalloc upstream seems to have slowed way down of late). Next up,I'll create 5.3.0 import on the vendor branch, do the merge and start testing (it will be Minsoo's pull request, rebased, with any conflicts resolved and merge commit recorded). But that will have to be tomorrow or more likely during the work week. I'm too tired tonight to get it right at the moment. And a special thanks to emaste for giving bz the right recipe for doing the subtree merge w/o git subtree for his work on the linux wifi drivers in the tree. Warner On Sat, Nov 30, 2024 at 9:38=E2=80=AFPM Warner Losh wrote: > Yea, I need to get a copy of jemalloc 5.3.0 and 5.2.1 to try to > 'bootstrap' the vendor branch. > Then I need to bootstrap it... > > I just did the same with edk2 (which had a vendor branch, but hadn't been > updated since svn times). > However, jemalloc doesn't have a vendor branch yet, so I'll have to creat= e > that, but I'll start with the > current version rather than doing full history... So I'll start there. > I also just did awk and lua, so once I have things bootstrapped, I'll be > able to add 5.3.0 and then layer Minsoo's on > top of that and then start testing it somehow. > > Malloc makes me nervous to touch, honestly, but I'll give it a go and tes= t > boot on my system and > maybe see if we can survive a workload at work w/o regressions... But I > can't do a full test with lots > of machines until after the first of the year (though I can do a couple > for a few days before then). > > So my next step is to bootstrap the vendor branch... I'll give that a try > tonight. > > Warner > > On Sat, Nov 30, 2024 at 8:26=E2=80=AFPM Minsoo Choo > wrote: > >> I have already submitted PR on github ( >> https://github.com/freebsd/freebsd-src/pull/1337) and phabricator ( >> https://reviews.freebsd.org/D41421). I don't have access (commit bit) to >> freebsd git repo, so there is nothing I can do at this point since vendo= r >> import and landing patches requires commit bit. >> On Saturday, November 30th, 2024 at 1:42 PM, cglogic < >> cglogic@protonmail.com> wrote: >> >> I see, it happens. >> Maybe another committer will volunteer to do the update. >> I hope it will make its way into 15.0 release. >> >> Thanks. >> On Friday, November 29th, 2024 at 9:38 PM, Warner Losh >> wrote: >> >> I've been swamped. we need to bootstrap the vendor branch, and the way >> prior updates were done >> isn't so great. >> >> Warner >> >> On Mon, Nov 25, 2024 at 2:21=E2=80=AFAM cglogic = wrote: >> >>> Hello guys, >>> >>> How the update of jemalloc is going? It's November now. >>> >>> Thanks. >>> On Monday, July 22nd, 2024 at 7:02 PM, Minsoo Choo < >>> minsoochoo0122@proton.me> wrote: >>> >>> First, sorry for late response. >>> >>> cglogic, thank you for bringing up this issue again since I nearly >>> forgot that this issue was still open. >>> >>> Warner, as I can't access to my FreeBSD instance until the end of >>> August, but I can still edit and push the code through my Arm Mac. This >>> means that I can't test the updated code on my machine, but I can join = the >>> review process and listen to change proposals. >>> >>> I'll open a Github PR in a few hours. (The phabricator review will stay >>> opened just in case) >>> On Monday, July 22nd, 2024 at 5:08 AM, Warner Losh >>> wrote: >>> >>> >>> >>> On Sun, Jul 21, 2024 at 2:03=E2=80=AFPM cglogic wrote: >>> >>>> >>>> On Sunday, July 21st, 2024 at 6:54 AM, Warner Losh >>>> wrote: >>>> >>>> >>>> >>>> On Sat, Jul 20, 2024 at 1:59=E2=80=AFAM cglogic wrote: >>>> >>>>> Hello FreeBSD community, >>>>> >>>>> After Jason Evans stepped aside from maintaining jemalloc in FreeBSD, >>>>> it's not updating in time anymore. >>>>> Version 5.3.0 was released May 6, 2022 and FreeBSD still not imported >>>>> it into the tree. >>>>> >>>>> There is a pending review https://reviews.freebsd.org/D41421 from Aug >>>>> 11, 2023. >>>>> I'm successfully running FreeBSD/amd64 system with D41421 applied for >>>>> 8 months, as well as many other people. >>>>> >>>>> Can it be reviewed and committed to CURRENT? >>>>> Or, if there is no committers willing to do it, can commit bit be >>>>> given to submitter or another person willing to do this? >>>>> >>>>> It's very disappointing when users spend their time to fill such gaps >>>>> and their efforts just ignored by the developers. >>>>> Every year FreeBSD Community Survey asking about user experience in >>>>> contributing to FreeBSD. >>>>> Here you can see an example of such contributing. >>>>> >>>>> >>>> First, thank you for being persistent and continuing to bring it up. >>>> It's important to do that to make sure this (and your many other) >>>> contribution doesn't fall on the floor. >>>> >>>> And to be fair, we're only 3 months since the last update. Still, quit= e >>>> a bit longer than you should have to wait, but not nearly the year the >>>> original date suggests. >>>> >>>> And this is a perfect storm of "how the project is bad at accepting >>>> contributions": >>>> (1) The original submission was close to the 14 branch creation time. >>>> This meant that we weren't well prepared to look at it since it is suc= h an >>>> invasive change (at least on its surface). It also slowed the initial >>>> response... >>>> (2) There was a number of back and forth requests for changes, which >>>> took time to sort out... >>>> (3) The size of this is huge, well beyond the capacity of Phabricator >>>> to review accurately... >>>> (4) It's a vendor import. That means we can't just drop the Phabricato= r >>>> review into the tree... >>>> (5) It's phabricator: this is a great tool for developers, but we have >>>> a terrible track record of using it for intake from new contributors. = We >>>> don't have any oversight at all over this tool, at there's at best tep= id >>>> and luke warm attempts to look for drop balls. >>>> >>>> All of these things are a terrible experience. I can only apologize. >>>> These days, we might steer this towards github, but the 'vendor import= ' >>>> means you really need someone on the inside, or you need to be on the >>>> inside to make that work. >>>> >>>> So, how to move forward? Well, I'd like to propose the following: >>>> (1) submit all the other Phabricator reviews you have open (they are >>>> mostly good, or close to good) to github. Github is being actively man= aged >>>> and will make it faster to get things it. It's a much better tool for = new >>>> contributors (and even frequent contributors of smallish things). >>>> (2) I should do an vendor import of 5.3.0 from github, and do the merg= e >>>> to a branch and push that to github. You can then layer on your change= s and >>>> those can be reviewed more closely as a pull request against the branc= h I >>>> push. I suspect that most of the issues are sorted out already >>>> (3) I'll land it via that route... >>>> >>>> And, if the sum of the other pull requests and this are good (and I >>>> suspect they will be), then we can talk about commit bits and such. >>>> >>>> It's experiences like this which is why I'm trying to stand up github >>>> pull requests as a reliable way to get things and and the best place t= o >>>> send people... >>>> >>>> Thanks again for persisting, and also for expressing this criticism >>>> that we (hopefully) can use to make it better. >>>> >>>> Warner >>>> >>>> >>>> Hello. >>>> >>>> I'm not the author of D41421. Just applied the patch to test it 8 >>>> months ago. And recently discovered that it's still not committed. >>>> I can't copy your message to Phabricator because don't have an account= . Please, >>>> if you have time, help the author in D41421. >>>> >>> >>> Ah yes. I've been in touch with the author for other things, and someho= w >>> thought it was you.... I'll reach out to him via other means... >>> >>> Warner >>> >>> >>> >>> >> >> --000000000000967b870628300dde Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
(sorry to follow up=C2=A0to my own email = and topposting)

I got the vendor branc= h bootstrapped: I created vendor/jemalloc and tagged vendor/jemalloc/5.2.1,=
and created a merge commit from that branch=C2=A0to main. I had = to tweak the
FREEBSD-Xlist a little.I've not updated the othe= r two FREEBSD-* files, but the steps were
documented in the commi= t messages (the vendor branch one is especially long and
likely s= hould migrate into the developer handbook). FREEBSD-update is basically
a shell script to do the same thing that git subtree merge does, tho= ugh I'm sure some
tweaks could help the next time (if there i= s a next time, jemalloc upstream seems to
have slowed way down of= late).

Next up,I'll create 5.3.0 import on th= e vendor branch, do the=C2=A0merge and start testing (it will be
= Minsoo's pull request, rebased, with any conflicts resolved and merge c= ommit recorded).
But that will have to be tomorrow or more likely= during the work week. I'm too tired tonight
to get it right = at the moment.

And a special thanks to emaste for = giving bz the right recipe for doing the subtree merge
w/o git su= btree for his work on the linux wifi drivers in the tree.

Warner

On Sat, Nov 30, 2024 at 9:38=E2=80=AFPM= Warner Losh <imp@bsdimp.com> w= rote:
Yea, I need to get a copy of jemalloc 5.3.0 and 5.2.1 to try to &#= 39;bootstrap' the vendor branch.
Then I need to bootstrap it...

I just did the same with edk2 (which had a vendor bra= nch, but hadn't been updated since svn times).
However, jemal= loc doesn't have a vendor branch yet, so I'll have to create that, = but I'll start with the
current version rather than doing ful= l history...=C2=A0 So I'll start there.
I also just did awk a= nd lua, so once I have things bootstrapped, I'll be able to add 5.3.0 a= nd then layer Minsoo's=C2=A0 on
top of that and then start te= sting it somehow.

Malloc makes me nervous to touch= , honestly, but I'll give it a go and test boot on my system and
<= div>maybe see if we can survive a workload at work w/o regressions... But I= can't do a full test with lots
of machines until after t= he first of the year (though I can do a couple for a few days before then).=

So my next step is to bootstrap the vendor branch= ... I'll give that a try tonight.

Warner
=

= On Sat, Nov 30, 2024 at 8:26=E2=80=AFPM Minsoo Choo <minsoochoo0122@proton.me>= wrote:
I have already submitted PR on github (https://github.com/freebsd/freebsd-sr= c/pull/1337) and phabricator (h= ttps://reviews.freebsd.org/D41421). I don't have access (com= mit bit) to freebsd git repo, so there is nothing I can do at this point si= nce vendor import and landing patches requires commit bit.
On Friday, November 29th, 2024 at 9:38 PM, Warner Losh <imp@bsdimp.com> wrote:<= br>
I've been swamped. we need to bootstrap th= e vendor branch, and the way prior updates were done
isn't so great= .

Warner

On Mon, Nov 25, 2024 at 2:21=E2=80= =AFAM cglogic <cglogic@protonmail.com> wrot= e:
Hello guys,

How the update of jemalloc is go= ing? It's November now.

Thanks.
On Monday, July 22nd, 2024 at 7:02 PM, Minsoo Choo <minsoochoo0122@proton.me> wrote:
Firs= t, sorry for late response.

cglogic, thank you for bringing up this issue again since I near= ly forgot that this issue was still open.

Warner, as I can't access to my FreeBSD instan= ce until the end of August, but I can still edit and push the code through = my Arm Mac. This means that I can't test the updated code on my machine= , but I can join the review process and listen to change proposals.

I'll open a Github P= R in a few hours. (The phabricator review will stay opened just in case)
On Monday, July 22nd, 2024 at 5:08 AM, Warner Losh <imp@bsdimp.com> wrote:


On Sun, Jul 21, 2024 at 2= :03=E2=80=AFPM cglogic <cglogic@protonmail.com= > wrote:

On Sunday, July 21st, 2024 at 6:54 AM, Warner Losh <imp@bsdimp.com> wrote:


On Sat, Jul 20, 2024 at 1= :59=E2=80=AFAM cglogic <cglogic@protonmail.com= > wrote:
Hello FreeBSD community,

After Jason Evans stepped aside from = maintaining jemalloc in FreeBSD, it's not updating in time anymore.
Version 5.3.0 was released = May 6, 2022 and FreeBSD still not imported it into the tree.

There is a pending review https://reviews.freebsd.org/D41421 from Aug 11, 2023.
I'm succes= sfully running FreeBSD/amd64 system with D41421 applied for 8 months, as we= ll as many other people.

= Can it be reviewed and committed to CURRENT?
Or, if there is no committe= rs willing to do it, can commit bit be given to submitter or another person= willing to do this?
=
It's very disappointing when users spend their time to fi= ll such gaps and their efforts just ignored by the developers.
Every year FreeBSD Community Survey asking about user experience in contr= ibuting to FreeBSD.
Here you can see an example of su= ch contributing.


Firs= t, thank you for being persistent and continuing to bring it up. It's i= mportant to do that to make sure this (and your many other) contribution do= esn't fall on the floor.

And to be fair, w= e're only 3 months since the last update. Still, quite a bit longer tha= n you should have to wait, but not nearly the year the original date sugges= ts.

And this is a perfect storm of "how t= he project is bad at accepting contributions":
(1) The origi= nal submission was close to the 14 branch creation time. This meant that we= weren't well prepared to look at it since it is such an invasive chang= e (at least on its surface). It also slowed the initial response...
(2) There was a number of back and forth requests for changes, which= took time to sort out...
(3) The size of this is huge, well beyo= nd the capacity of Phabricator to review accurately...
(4) It'= ;s a vendor import. That means we can't just drop the Phabricator revie= w into the tree...
(5) It's phabricator: this is a great tool= for developers, but we have a terrible track record of using it for intake= from new contributors. We don't have any oversight at all over this to= ol, at there's at best tepid and luke warm attempts to look for drop ba= lls.

All of these things are a terrible experience= . I can only apologize. These days, we might steer this towards github, but= the 'vendor import' means you really need someone on the inside, o= r you need to be on the inside to make that work.

= So, how to move forward? Well, I'd like to propose the following:
=
(1) submit all the other Phabricator reviews you have open (they are m= ostly good, or close to good) to github. Github is being actively managed a= nd will make it faster to get things it. It's a much better tool for ne= w contributors (and even frequent contributors of smallish things).
(2) I should do an vendor import of 5.3.0 from github, and do the merge = to a branch and push that to github. You can then layer on your changes and= those can be reviewed more closely as a pull request against the branch I = push. I suspect that most of the issues are sorted out already
(3) I'll land it via that route...

And, if = the sum of the other pull requests and this are good (and I suspect they wi= ll be), then we can talk about commit bits and such.

It's experiences like this which is why I'm trying to stand up g= ithub pull requests as a reliable way to get things and and the best place = to send people...

Thanks again for persistin= g, and also for expressing this criticism that we (hopefully) can use to ma= ke it better.

Warner

Hello.

I'm not the author of = D41421. Just applied the patch to test it 8 months ago. And recently = discovered that it's still not committed.
I can't copy your message to Phabricator becaus= e don't have an account. Please, if you have time, help the auth= or in D41421.

Ah yes. I've been i= n touch with the author for other things, and somehow thought it was you...= . I'll reach out to him via other means...

Wa= rner




--000000000000967b870628300dde-- From nobody Sun Dec 1 10:28:04 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Y1NXj4b70z5fQTy for ; Sun, 01 Dec 2024 10:32:53 +0000 (UTC) (envelope-from mpysw@vip.163.com) Received: from mail-m16.vip.163.com (mail-m16.vip.163.com [1.95.21.4]) by mx1.freebsd.org (Postfix) with ESMTP id 4Y1NRd1zrYz3wb3 for ; Sun, 1 Dec 2024 10:28:28 +0000 (UTC) (envelope-from mpysw@vip.163.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=vip.163.com header.s=s110527 header.b=W7lDoqVH; spf=pass (mx1.freebsd.org: domain of mpysw@vip.163.com designates 1.95.21.4 as permitted sender) smtp.mailfrom=mpysw@vip.163.com; dmarc=pass (policy=none) header.from=vip.163.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vip.163.com; s=s110527; h=Date:Subject:Message-ID:From: MIME-Version:Content-Type; bh=6VMpmIACsH5wgjZLAoQ0yXV/Y+yJT0/w/Y lWh770LeY=; b=W7lDoqVH093P+tnPMEiNKOWPCl7ARkY90x4NdDhqJDrShALGkP Iime4pIDrYM5zf8AP7C1gvOurHvwJFFR5AvyHX+n+EAZkyp1UgBUvFYTrQpx5gIX wJqwea564QoXbYMckQpVipbXKB54jgEQnhA+TzAFOhZbjKDwt067g8MAg= Received: from [192.168.1.103] (unknown [111.35.220.212]) by gzsmtp1 (Coremail) with SMTP id Ac8vCgBngIk1OkxnhGyZBA--.37578S3; Sun, 01 Dec 2024 18:28:07 +0800 (CST) Date: Sun, 01 Dec 2024 18:28:04 +0800 Subject: =?UTF-8?Q?=E5=9B=9E=E5=A4=8D=EF=BC=9ALong_time_outdated_jemalloc?= X-Priority: 3 Message-ID: <-eqr7h4489qu1ah9r775oe9mrcvzhqjk1nih7ydup1meby0kb-7mugcrg4e32kvderqx-rzyzff-iaj2h9-2r7c0v-mkpvyc9s8cdt47jee7uanbw3-jm6swi-i6uz4a526e8fp2xp7s-1buouzd7zci.1733048884821@email.android.com> References: From: =?UTF-8?B?4oCq4oCq4oCqeXUgc2hhbiB3ZWnigKzigKzigKw=?= To: Warner Losh , Minsoo Choo Cc: cglogic , FreeBSD CURRENT List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: base64 X-CM-TRANSID:Ac8vCgBngIk1OkxnhGyZBA--.37578S3 X-Coremail-Antispam: 1Uf129KBjvJXoWxKw18Xry5Ar13WF15ZryftFb_yoW3ZFWDpF WrKFW3trs5XFsxJwn3Ar1Iqa4Fy395JrW5Xas5t3y0v3s8A34xKr1jkrWF9asrArs3Cw1Y vr4qv3s3G3Z8ZFJanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07jFg4-UUUUU= X-Originating-IP: [111.35.220.212] X-CM-SenderInfo: 5ps124g6yl1hqrwthudrp/1tbiBQqob2dMMg4IXAAAsL X-Spamd-Result: default: False [-3.49 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.990]; DMARC_POLICY_ALLOW(-0.50)[vip.163.com,none]; MIME_HTML_ONLY(0.20)[]; R_DKIM_ALLOW(-0.20)[vip.163.com:s=s110527]; R_SPF_ALLOW(-0.20)[+ip4:1.95.21.0/27]; RCVD_NO_TLS_LAST(0.10)[]; MIME_BASE64_TEXT(0.10)[]; FREEMAIL_FROM(0.00)[vip.163.com]; HAS_X_PRIO_THREE(0.00)[3]; FREEMAIL_CC(0.00)[protonmail.com,freebsd.org]; TO_DN_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[163.com:dkim]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:~]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; HAS_XOIP(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[vip.163.com:+]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:55990, ipnet:1.95.0.0/19, country:CN]; RCVD_IN_DNSWL_NONE(0.00)[1.95.21.4:from]; MISSING_XM_UA(0.00)[]; FREEMAIL_ENVFROM(0.00)[vip.163.com] X-Rspamd-Queue-Id: 4Y1NRd1zrYz3wb3 X-Spamd-Bar: --- PGRpdiBkaXI9ImF1dG8iPuecn+aYr+S4quWlvea2iOaBr++8jOW4jOacm+iDveaXqeeCueWNh+e6 p+WIsDUuMy4wPGJyPjxicj48YnI+PGJyPjxicj48ZGl2IGlkPSJod19zaWduYXR1cmUiPuWPkeiH quaIkeeahOaJi+acujwvZGl2PjwvZGl2PjxkaXYgc3R5bGU9ImxpbmUtaGVpZ2h0OjEuNSI+PGJy Pjxicj4tLS0tLS0tLSDljp/lp4vpgq7ku7YgLS0tLS0tLS08YnI+5Y+R5Lu25Lq677yaIFdhcm5l ciBMb3NoICZsdDtpbXBAYnNkaW1wLmNvbSZndDs8YnI+5pel5pyf77yaIDIwMjTlubQxMuaciDHm l6Xlkajml6UgMTU6MDU8YnI+5pS25Lu25Lq677yaIE1pbnNvbyBDaG9vICZsdDttaW5zb29jaG9v MDEyMkBwcm90b24ubWUmZ3Q7PGJyPuaKhOmAge+8miBjZ2xvZ2ljICZsdDtjZ2xvZ2ljQHByb3Rv bm1haWwuY29tJmd0OywgRnJlZUJTRCBDVVJSRU5UICZsdDtmcmVlYnNkLWN1cnJlbnRAZnJlZWJz ZC5vcmcmZ3Q7PGJyPuS4uyAgICDpopjvvJogUmU6IExvbmcgdGltZSBvdXRkYXRlZCBqZW1hbGxv Yzxicj48YmxvY2txdW90ZT48ZGl2IGRpcj0ibHRyIj48ZGl2IGRpcj0ibHRyIj4oc29ycnkgdG8g Zm9sbG93IHVwwqB0byBteSBvd24gZW1haWwgYW5kIHRvcHBvc3RpbmcpPC9kaXY+PGRpdiBkaXI9 Imx0ciI+PGJyIC8+PC9kaXY+PGRpdj5JIGdvdCB0aGUgdmVuZG9yIGJyYW5jaCBib290c3RyYXBw ZWQ6IEkgY3JlYXRlZCB2ZW5kb3IvamVtYWxsb2MgYW5kIHRhZ2dlZCB2ZW5kb3IvamVtYWxsb2Mv NS4yLjEsPC9kaXY+PGRpdj5hbmQgY3JlYXRlZCBhIG1lcmdlIGNvbW1pdCBmcm9tIHRoYXQgYnJh bmNowqB0byBtYWluLiBJIGhhZCB0byB0d2VhayB0aGU8L2Rpdj48ZGl2PkZSRUVCU0QtWGxpc3Qg YSBsaXR0bGUuSSYjMzk7dmUgbm90IHVwZGF0ZWQgdGhlIG90aGVyIHR3byBGUkVFQlNELSogZmls ZXMsIGJ1dCB0aGUgc3RlcHMgd2VyZTwvZGl2PjxkaXY+ZG9jdW1lbnRlZCBpbiB0aGUgY29tbWl0 IG1lc3NhZ2VzICh0aGUgdmVuZG9yIGJyYW5jaCBvbmUgaXMgZXNwZWNpYWxseSBsb25nIGFuZDwv ZGl2PjxkaXY+bGlrZWx5IHNob3VsZCBtaWdyYXRlIGludG8gdGhlIGRldmVsb3BlciBoYW5kYm9v aykuIEZSRUVCU0QtdXBkYXRlIGlzIGJhc2ljYWxseTwvZGl2PjxkaXY+YSBzaGVsbCBzY3JpcHQg dG8gZG8gdGhlIHNhbWUgdGhpbmcgdGhhdCBnaXQgc3VidHJlZSBtZXJnZSBkb2VzLCB0aG91Z2gg SSYjMzk7bSBzdXJlIHNvbWU8L2Rpdj48ZGl2PnR3ZWFrcyBjb3VsZCBoZWxwIHRoZSBuZXh0IHRp bWUgKGlmIHRoZXJlIGlzIGEgbmV4dCB0aW1lLCBqZW1hbGxvYyB1cHN0cmVhbSBzZWVtcyB0bzwv ZGl2PjxkaXY+aGF2ZSBzbG93ZWQgd2F5IGRvd24gb2YgbGF0ZSkuPC9kaXY+PGRpdj48YnIgLz48 L2Rpdj48ZGl2Pk5leHQgdXAsSSYjMzk7bGwgY3JlYXRlIDUuMy4wIGltcG9ydCBvbiB0aGUgdmVu ZG9yIGJyYW5jaCwgZG8gdGhlwqBtZXJnZSBhbmQgc3RhcnQgdGVzdGluZyAoaXQgd2lsbCBiZTwv ZGl2PjxkaXY+TWluc29vJiMzOTtzIHB1bGwgcmVxdWVzdCwgcmViYXNlZCwgd2l0aCBhbnkgY29u ZmxpY3RzIHJlc29sdmVkIGFuZCBtZXJnZSBjb21taXQgcmVjb3JkZWQpLjwvZGl2PjxkaXY+QnV0 IHRoYXQgd2lsbCBoYXZlIHRvIGJlIHRvbW9ycm93IG9yIG1vcmUgbGlrZWx5IGR1cmluZyB0aGUg d29yayB3ZWVrLiBJJiMzOTttIHRvbyB0aXJlZCB0b25pZ2h0PC9kaXY+PGRpdj50byBnZXQgaXQg cmlnaHQgYXQgdGhlIG1vbWVudC48L2Rpdj48ZGl2PjxiciAvPjwvZGl2PjxkaXY+QW5kIGEgc3Bl Y2lhbCB0aGFua3MgdG8gZW1hc3RlIGZvciBnaXZpbmcgYnogdGhlIHJpZ2h0IHJlY2lwZSBmb3Ig ZG9pbmcgdGhlIHN1YnRyZWUgbWVyZ2U8L2Rpdj48ZGl2PncvbyBnaXQgc3VidHJlZSBmb3IgaGlz IHdvcmsgb24gdGhlIGxpbnV4IHdpZmkgZHJpdmVycyBpbiB0aGUgdHJlZS48L2Rpdj48ZGl2Pjxi ciAvPjwvZGl2PjxkaXY+V2FybmVyPC9kaXY+PGJyIC8+PGRpdiBjbGFzcz0iZ21haWxfcXVvdGUg Z21haWxfcXVvdGVfY29udGFpbmVyIj48ZGl2IGRpcj0ibHRyIiBjbGFzcz0iZ21haWxfYXR0ciI+ T24gU2F0LCBOb3YgMzAsIDIwMjQgYXQgOTozOOKAr1BNIFdhcm5lciBMb3NoICZsdDs8YSBocmVm PSJtYWlsdG86aW1wQGJzZGltcC5jb20iPmltcEBic2RpbXAuY29tPC9hPiZndDsgd3JvdGU6PGJy IC8+PC9kaXY+PGJsb2NrcXVvdGUgc3R5bGU9Im1hcmdpbjowcHggMHB4IDBweCAwLjhleDtib3Jk ZXItbGVmdDoxcHggc29saWQgcmdiKCAyMDQgLCAyMDQgLCAyMDQgKTtwYWRkaW5nLWxlZnQ6MWV4 Ij48ZGl2IGRpcj0ibHRyIj5ZZWEsIEkgbmVlZCB0byBnZXQgYSBjb3B5IG9mIGplbWFsbG9jIDUu My4wIGFuZCA1LjIuMSB0byB0cnkgdG8gJiMzOTtib290c3RyYXAmIzM5OyB0aGUgdmVuZG9yIGJy YW5jaC48ZGl2PlRoZW4gSSBuZWVkIHRvIGJvb3RzdHJhcCBpdC4uLjwvZGl2PjxkaXY+PGJyIC8+ PC9kaXY+PGRpdj5JIGp1c3QgZGlkIHRoZSBzYW1lIHdpdGggZWRrMiAod2hpY2ggaGFkIGEgdmVu ZG9yIGJyYW5jaCwgYnV0IGhhZG4mIzM5O3QgYmVlbiB1cGRhdGVkIHNpbmNlIHN2biB0aW1lcyku PC9kaXY+PGRpdj5Ib3dldmVyLCBqZW1hbGxvYyBkb2VzbiYjMzk7dCBoYXZlIGEgdmVuZG9yIGJy YW5jaCB5ZXQsIHNvIEkmIzM5O2xsIGhhdmUgdG8gY3JlYXRlIHRoYXQsIGJ1dCBJJiMzOTtsbCBz dGFydCB3aXRoIHRoZTwvZGl2PjxkaXY+Y3VycmVudCB2ZXJzaW9uIHJhdGhlciB0aGFuIGRvaW5n IGZ1bGwgaGlzdG9yeS4uLsKgIFNvIEkmIzM5O2xsIHN0YXJ0IHRoZXJlLjwvZGl2PjxkaXY+SSBh bHNvIGp1c3QgZGlkIGF3ayBhbmQgbHVhLCBzbyBvbmNlIEkgaGF2ZSB0aGluZ3MgYm9vdHN0cmFw cGVkLCBJJiMzOTtsbCBiZSBhYmxlIHRvIGFkZCA1LjMuMCBhbmQgdGhlbiBsYXllciBNaW5zb28m IzM5O3PCoCBvbjwvZGl2PjxkaXY+dG9wIG9mIHRoYXQgYW5kIHRoZW4gc3RhcnQgdGVzdGluZyBp dCBzb21laG93LjwvZGl2PjxkaXY+PGJyIC8+PC9kaXY+PGRpdj5NYWxsb2MgbWFrZXMgbWUgbmVy dm91cyB0byB0b3VjaCwgaG9uZXN0bHksIGJ1dCBJJiMzOTtsbCBnaXZlIGl0IGEgZ28gYW5kIHRl c3QgYm9vdCBvbiBteSBzeXN0ZW0gYW5kPC9kaXY+PGRpdj5tYXliZSBzZWUgaWYgd2UgY2FuIHN1 cnZpdmUgYSB3b3JrbG9hZCBhdCB3b3JrIHcvbyByZWdyZXNzaW9ucy4uLiBCdXQgSSBjYW4mIzM5 O3QgZG8gYSBmdWxsIHRlc3Qgd2l0aCBsb3RzPGJyIC8+PC9kaXY+PGRpdj5vZiBtYWNoaW5lcyB1 bnRpbCBhZnRlciB0aGUgZmlyc3Qgb2YgdGhlIHllYXIgKHRob3VnaCBJIGNhbiBkbyBhIGNvdXBs ZSBmb3IgYSBmZXcgZGF5cyBiZWZvcmUgdGhlbikuPC9kaXY+PGRpdj48YnIgLz48L2Rpdj48ZGl2 PlNvIG15IG5leHQgc3RlcCBpcyB0byBib290c3RyYXAgdGhlIHZlbmRvciBicmFuY2guLi4gSSYj Mzk7bGwgZ2l2ZSB0aGF0IGEgdHJ5IHRvbmlnaHQuPC9kaXY+PGRpdj48YnIgLz48L2Rpdj48ZGl2 Pldhcm5lcjwvZGl2PjwvZGl2PjxiciAvPjxkaXYgY2xhc3M9ImdtYWlsX3F1b3RlIGVsaWRlZC10 ZXh0Ij48ZGl2IGRpcj0ibHRyIiBjbGFzcz0iZ21haWxfYXR0ciI+T24gU2F0LCBOb3YgMzAsIDIw MjQgYXQgODoyNuKAr1BNIE1pbnNvbyBDaG9vICZsdDs8YSBocmVmPSJtYWlsdG86bWluc29vY2hv bzAxMjJAcHJvdG9uLm1lIj5taW5zb29jaG9vMDEyMkBwcm90b24ubWU8L2E+Jmd0OyB3cm90ZTo8 YnIgLz48L2Rpdj48YmxvY2txdW90ZSBzdHlsZT0ibWFyZ2luOjBweCAwcHggMHB4IDAuOGV4O2Jv cmRlci1sZWZ0OjFweCBzb2xpZCByZ2IoIDIwNCAsIDIwNCAsIDIwNCApO3BhZGRpbmctbGVmdDox ZXgiPjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiYjMzk7YXJpYWwmIzM5OyAsIHNhbnMtc2VyaWY7 Zm9udC1zaXplOjE0cHg7Y29sb3I6cmdiKCAwICwgMCAsIDAgKTtiYWNrZ3JvdW5kLWNvbG9yOnJn YiggMjU1ICwgMjU1ICwgMjU1ICkiPkkgaGF2ZSBhbHJlYWR5IHN1Ym1pdHRlZCBQUiBvbiBnaXRo dWIgKDxhIGhyZWY9Imh0dHBzOi8vZ2l0aHViLmNvbS9mcmVlYnNkL2ZyZWVic2Qtc3JjL3B1bGwv MTMzNyI+aHR0cHM6Ly9naXRodWIuY29tL2ZyZWVic2QvZnJlZWJzZC1zcmMvcHVsbC8xMzM3PC9h PikgYW5kIHBoYWJyaWNhdG9yICg8YSBocmVmPSJodHRwczovL3Jldmlld3MuZnJlZWJzZC5vcmcv RDQxNDIxIj5odHRwczovL3Jldmlld3MuZnJlZWJzZC5vcmcvRDQxNDIxPC9hPikuIEkgZG9uJiMz OTt0IGhhdmUgYWNjZXNzIChjb21taXQgYml0KSB0byBmcmVlYnNkIGdpdCByZXBvLCBzbyB0aGVy ZSBpcyBub3RoaW5nIEkgY2FuIGRvIGF0IHRoaXMgcG9pbnQgc2luY2UgdmVuZG9yIGltcG9ydCBh bmQgbGFuZGluZyBwYXRjaGVzIHJlcXVpcmVzIGNvbW1pdCBiaXQuPGJyIC8+PC9kaXY+PGRpdj4N CiAgICAgICAgT24gU2F0dXJkYXksIE5vdmVtYmVyIDMwdGgsIDIwMjQgYXQgMTo0MiBQTSwgY2ds b2dpYyAmbHQ7PGEgaHJlZj0ibWFpbHRvOmNnbG9naWNAcHJvdG9ubWFpbC5jb20iPmNnbG9naWNA cHJvdG9ubWFpbC5jb208L2E+Jmd0OyB3cm90ZTo8YnIgLz4NCiAgICAgICAgPGJsb2NrcXVvdGU+ DQogICAgICAgICAgICA8ZGl2IHN0eWxlPSJmb250LWZhbWlseTomIzM5O2FyaWFsJiMzOTsgLCBz YW5zLXNlcmlmO2ZvbnQtc2l6ZToxNHB4Ij5JIHNlZSwgaXQgaGFwcGVucy48L2Rpdj48ZGl2IHN0 eWxlPSJmb250LWZhbWlseTomIzM5O2FyaWFsJiMzOTsgLCBzYW5zLXNlcmlmO2ZvbnQtc2l6ZTox NHB4Ij5NYXliZSBhbm90aGVyIGNvbW1pdHRlciB3aWxsIHZvbHVudGVlciB0byBkbyB0aGUgdXBk YXRlLjxkaXY+SSBob3BlIGl0IHdpbGwgbWFrZSBpdHMgd2F5IGludG8gMTUuMCByZWxlYXNlLjwv ZGl2PjxkaXY+PGJyIC8+PC9kaXY+PGRpdj5UaGFua3MuPC9kaXY+PC9kaXY+PGRpdj4NCiAgICAg ICAgT24gRnJpZGF5LCBOb3ZlbWJlciAyOXRoLCAyMDI0IGF0IDk6MzggUE0sIFdhcm5lciBMb3No ICZsdDs8YSBocmVmPSJtYWlsdG86aW1wQGJzZGltcC5jb20iPmltcEBic2RpbXAuY29tPC9hPiZn dDsgd3JvdGU6PGJyIC8+DQogICAgICAgIDxibG9ja3F1b3RlPg0KICAgICAgICAgICAgPGRpdiBk aXI9Imx0ciI+SSYjMzk7dmUgYmVlbiBzd2FtcGVkLiB3ZSBuZWVkIHRvIGJvb3RzdHJhcCB0aGUg dmVuZG9yIGJyYW5jaCwgYW5kIHRoZSB3YXkgcHJpb3IgdXBkYXRlcyB3ZXJlIGRvbmU8ZGl2Pmlz biYjMzk7dCBzbyBncmVhdC4gPC9kaXY+PGRpdj48YnIgLz48L2Rpdj48ZGl2Pldhcm5lcjwvZGl2 PjwvZGl2PjxiciAvPjxkaXYgY2xhc3M9ImdtYWlsX3F1b3RlIGVsaWRlZC10ZXh0Ij48ZGl2IGRp cj0ibHRyIiBjbGFzcz0iZ21haWxfYXR0ciI+T24gTW9uLCBOb3YgMjUsIDIwMjQgYXQgMjoyMeKA r0FNIGNnbG9naWMgJmx0OzxhIGhyZWY9Im1haWx0bzpjZ2xvZ2ljQHByb3Rvbm1haWwuY29tIj5j Z2xvZ2ljQHByb3Rvbm1haWwuY29tPC9hPiZndDsgd3JvdGU6PGJyIC8+PC9kaXY+PGJsb2NrcXVv dGUgc3R5bGU9Im1hcmdpbjowcHggMHB4IDBweCAwLjhleDtib3JkZXItbGVmdDoxcHggc29saWQg cmdiKCAyMDQgLCAyMDQgLCAyMDQgKTtwYWRkaW5nLWxlZnQ6MWV4Ij48ZGl2IHN0eWxlPSJmb250 LWZhbWlseTomIzM5O2FyaWFsJiMzOTsgLCBzYW5zLXNlcmlmO2ZvbnQtc2l6ZToxNHB4Ij5IZWxs byBndXlzLDwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiYjMzk7YXJpYWwmIzM5OyAsIHNh bnMtc2VyaWY7Zm9udC1zaXplOjE0cHgiPjxiciAvPjwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFt aWx5OiYjMzk7YXJpYWwmIzM5OyAsIHNhbnMtc2VyaWY7Zm9udC1zaXplOjE0cHgiPkhvdyB0aGUg dXBkYXRlIG9mIGplbWFsbG9jIGlzIGdvaW5nPyBJdCYjMzk7cyBOb3ZlbWJlciBub3cuPC9kaXY+ PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6JiMzOTthcmlhbCYjMzk7ICwgc2Fucy1zZXJpZjtmb250 LXNpemU6MTRweCI+PGJyIC8+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6JiMzOTthcmlh bCYjMzk7ICwgc2Fucy1zZXJpZjtmb250LXNpemU6MTRweCI+VGhhbmtzLjwvZGl2PjxkaXY+DQog ICAgICAgIE9uIE1vbmRheSwgSnVseSAyMm5kLCAyMDI0IGF0IDc6MDIgUE0sIE1pbnNvbyBDaG9v ICZsdDs8YSBocmVmPSJtYWlsdG86bWluc29vY2hvbzAxMjJAcHJvdG9uLm1lIj5taW5zb29jaG9v MDEyMkBwcm90b24ubWU8L2E+Jmd0OyB3cm90ZTo8YnIgLz4NCiAgICAgICAgPGJsb2NrcXVvdGU+ DQogICAgICAgICAgICA8ZGl2IHN0eWxlPSJmb250LWZhbWlseTomIzM5O2FyaWFsJiMzOTsgLCBz YW5zLXNlcmlmO2ZvbnQtc2l6ZToxNHB4Ij5GaXJzdCwgc29ycnkgZm9yIGxhdGUgcmVzcG9uc2Uu PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6JiMzOTthcmlhbCYjMzk7ICwgc2Fucy1zZXJp Zjtmb250LXNpemU6MTRweCI+PGJyIC8+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6JiMz OTthcmlhbCYjMzk7ICwgc2Fucy1zZXJpZjtmb250LXNpemU6MTRweCI+Y2dsb2dpYywgdGhhbmsg eW91IGZvciBicmluZ2luZyB1cCB0aGlzIGlzc3VlIGFnYWluIHNpbmNlIEkgbmVhcmx5IGZvcmdv dCB0aGF0IHRoaXMgaXNzdWUgd2FzIHN0aWxsIG9wZW4uPC9kaXY+PGRpdiBzdHlsZT0iZm9udC1m YW1pbHk6JiMzOTthcmlhbCYjMzk7ICwgc2Fucy1zZXJpZjtmb250LXNpemU6MTRweCI+PGJyIC8+ PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6JiMzOTthcmlhbCYjMzk7ICwgc2Fucy1zZXJp Zjtmb250LXNpemU6MTRweCI+V2FybmVyLCBhcyBJIGNhbiYjMzk7dCBhY2Nlc3MgdG8gbXkgRnJl ZUJTRCBpbnN0YW5jZSB1bnRpbCB0aGUgZW5kIG9mIEF1Z3VzdCwgYnV0IEkgY2FuIHN0aWxsIGVk aXQgYW5kIHB1c2ggdGhlIGNvZGUgdGhyb3VnaCBteSBBcm0gTWFjLiBUaGlzIG1lYW5zIHRoYXQg SSBjYW4mIzM5O3QgdGVzdCB0aGUgdXBkYXRlZCBjb2RlIG9uIG15IG1hY2hpbmUsIGJ1dCBJIGNh biBqb2luIHRoZSByZXZpZXcgcHJvY2VzcyBhbmQgbGlzdGVuIHRvIGNoYW5nZSBwcm9wb3NhbHMu PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6JiMzOTthcmlhbCYjMzk7ICwgc2Fucy1zZXJp Zjtmb250LXNpemU6MTRweCI+PGJyIC8+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6JiMz OTthcmlhbCYjMzk7ICwgc2Fucy1zZXJpZjtmb250LXNpemU6MTRweCI+SSYjMzk7bGwgb3BlbiBh IEdpdGh1YiBQUiBpbiBhIGZldyBob3Vycy4gKFRoZSBwaGFicmljYXRvciByZXZpZXcgd2lsbCBz dGF5IG9wZW5lZCBqdXN0IGluIGNhc2UpPC9kaXY+PGRpdj4NCiAgICAgICAgT24gTW9uZGF5LCBK dWx5IDIybmQsIDIwMjQgYXQgNTowOCBBTSwgV2FybmVyIExvc2ggJmx0OzxhIGhyZWY9Im1haWx0 bzppbXBAYnNkaW1wLmNvbSI+aW1wQGJzZGltcC5jb208L2E+Jmd0OyB3cm90ZTo8YnIgLz4NCiAg ICAgICAgPGJsb2NrcXVvdGU+DQogICAgICAgICAgICA8ZGl2IGRpcj0ibHRyIj48ZGl2IGRpcj0i bHRyIj48YnIgLz48L2Rpdj48YnIgLz48ZGl2IGNsYXNzPSJnbWFpbF9xdW90ZSBlbGlkZWQtdGV4 dCI+PGRpdiBkaXI9Imx0ciIgY2xhc3M9ImdtYWlsX2F0dHIiPk9uIFN1biwgSnVsIDIxLCAyMDI0 IGF0IDI6MDPigK9QTSBjZ2xvZ2ljICZsdDs8YSBocmVmPSJtYWlsdG86Y2dsb2dpY0Bwcm90b25t YWlsLmNvbSI+Y2dsb2dpY0Bwcm90b25tYWlsLmNvbTwvYT4mZ3Q7IHdyb3RlOjxiciAvPjwvZGl2 PjxibG9ja3F1b3RlIHN0eWxlPSJtYXJnaW46MHB4IDBweCAwcHggMC44ZXg7Ym9yZGVyLWxlZnQ6 MXB4IHNvbGlkIHJnYiggMjA0ICwgMjA0ICwgMjA0ICk7cGFkZGluZy1sZWZ0OjFleCI+PGRpdiBz dHlsZT0iZm9udC1mYW1pbHk6JiMzOTthcmlhbCYjMzk7ICwgc2Fucy1zZXJpZjtmb250LXNpemU6 MTRweCI+PGJyIC8+PC9kaXY+PGRpdj4NCiAgICAgICAgT24gU3VuZGF5LCBKdWx5IDIxc3QsIDIw MjQgYXQgNjo1NCBBTSwgV2FybmVyIExvc2ggJmx0OzxhIGhyZWY9Im1haWx0bzppbXBAYnNkaW1w LmNvbSI+aW1wQGJzZGltcC5jb208L2E+Jmd0OyB3cm90ZTo8YnIgLz4NCiAgICAgICAgPGJsb2Nr cXVvdGU+DQogICAgICAgICAgICA8ZGl2IGRpcj0ibHRyIj48ZGl2IGRpcj0ibHRyIj48YnIgLz48 L2Rpdj48YnIgLz48ZGl2IGNsYXNzPSJnbWFpbF9xdW90ZSBlbGlkZWQtdGV4dCI+PGRpdiBkaXI9 Imx0ciIgY2xhc3M9ImdtYWlsX2F0dHIiPk9uIFNhdCwgSnVsIDIwLCAyMDI0IGF0IDE6NTnigK9B TSBjZ2xvZ2ljICZsdDs8YSBocmVmPSJtYWlsdG86Y2dsb2dpY0Bwcm90b25tYWlsLmNvbSI+Y2ds b2dpY0Bwcm90b25tYWlsLmNvbTwvYT4mZ3Q7IHdyb3RlOjxiciAvPjwvZGl2PjxibG9ja3F1b3Rl IHN0eWxlPSJtYXJnaW46MHB4IDBweCAwcHggMC44ZXg7Ym9yZGVyLWxlZnQ6MXB4IHNvbGlkIHJn YiggMjA0ICwgMjA0ICwgMjA0ICk7cGFkZGluZy1sZWZ0OjFleCI+PGRpdiBzdHlsZT0iZm9udC1m YW1pbHk6JiMzOTthcmlhbCYjMzk7ICwgc2Fucy1zZXJpZjtmb250LXNpemU6MTRweDtjb2xvcjpy Z2IoIDAgLCAwICwgMCApO2JhY2tncm91bmQtY29sb3I6cmdiKCAyNTUgLCAyNTUgLCAyNTUgKSI+ SGVsbG8gRnJlZUJTRCBjb21tdW5pdHksPC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6JiMz OTthcmlhbCYjMzk7ICwgc2Fucy1zZXJpZjtmb250LXNpemU6MTRweDtjb2xvcjpyZ2IoIDAgLCAw ICwgMCApO2JhY2tncm91bmQtY29sb3I6cmdiKCAyNTUgLCAyNTUgLCAyNTUgKSI+PGJyIC8+PC9k aXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6JiMzOTthcmlhbCYjMzk7ICwgc2Fucy1zZXJpZjtm b250LXNpemU6MTRweDtjb2xvcjpyZ2IoIDAgLCAwICwgMCApO2JhY2tncm91bmQtY29sb3I6cmdi KCAyNTUgLCAyNTUgLCAyNTUgKSI+PHNwYW4gc3R5bGU9ImRpc3BsYXk6aW5saW5lO2JhY2tncm91 bmQtY29sb3I6cmdiKCAyNTUgLCAyNTUgLCAyNTUgKSI+QWZ0ZXIgPC9zcGFuPjxzcGFuIHN0eWxl PSJiYWNrZ3JvdW5kLWNvbG9yOnJnYiggMjU1ICwgMjU1ICwgMjU1ICkiPkphc29uIEV2YW5zIHN0 ZXBwZWQgYXNpZGUgZnJvbSBtYWludGFpbmluZyBqZW1hbGxvYyBpbiBGcmVlQlNELCBpdCYjMzk7 cyBub3QgdXBkYXRpbmcgaW4gdGltZSBhbnltb3JlLjwvc3Bhbj48YnIgLz48L2Rpdj48ZGl2IHN0 eWxlPSJmb250LWZhbWlseTomIzM5O2FyaWFsJiMzOTsgLCBzYW5zLXNlcmlmO2ZvbnQtc2l6ZTox NHB4O2NvbG9yOnJnYiggMCAsIDAgLCAwICk7YmFja2dyb3VuZC1jb2xvcjpyZ2IoIDI1NSAsIDI1 NSAsIDI1NSApIj5WZXJzaW9uIDUuMy4wIHdhcyByZWxlYXNlZCBNYXkgNiwgMjAyMiBhbmQgRnJl ZUJTRCBzdGlsbCBub3QgaW1wb3J0ZWQgaXQgaW50byB0aGUgdHJlZS48L2Rpdj48ZGl2IHN0eWxl PSJmb250LWZhbWlseTomIzM5O2FyaWFsJiMzOTsgLCBzYW5zLXNlcmlmO2ZvbnQtc2l6ZToxNHB4 O2NvbG9yOnJnYiggMCAsIDAgLCAwICk7YmFja2dyb3VuZC1jb2xvcjpyZ2IoIDI1NSAsIDI1NSAs IDI1NSApIj48YnIgLz48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTomIzM5O2FyaWFsJiMz OTsgLCBzYW5zLXNlcmlmO2ZvbnQtc2l6ZToxNHB4O2NvbG9yOnJnYiggMCAsIDAgLCAwICk7YmFj a2dyb3VuZC1jb2xvcjpyZ2IoIDI1NSAsIDI1NSAsIDI1NSApIj5UaGVyZSBpcyBhIHBlbmRpbmcg cmV2aWV3IDxhIGhyZWY9Imh0dHBzOi8vcmV2aWV3cy5mcmVlYnNkLm9yZy9ENDE0MjEiPmh0dHBz Oi8vcmV2aWV3cy5mcmVlYnNkLm9yZy9ENDE0MjE8L2E+IGZyb20gQXVnIDExLCAyMDIzLjwvZGl2 PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiYjMzk7YXJpYWwmIzM5OyAsIHNhbnMtc2VyaWY7Zm9u dC1zaXplOjE0cHg7Y29sb3I6cmdiKCAwICwgMCAsIDAgKTtiYWNrZ3JvdW5kLWNvbG9yOnJnYigg MjU1ICwgMjU1ICwgMjU1ICkiPkkmIzM5O20gc3VjY2Vzc2Z1bGx5IHJ1bm5pbmcgRnJlZUJTRC9h bWQ2NCBzeXN0ZW0gd2l0aCBENDE0MjEgYXBwbGllZCBmb3IgOCBtb250aHMsIGFzIHdlbGwgYXMg bWFueSBvdGhlciBwZW9wbGUuPC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6JiMzOTthcmlh bCYjMzk7ICwgc2Fucy1zZXJpZjtmb250LXNpemU6MTRweDtjb2xvcjpyZ2IoIDAgLCAwICwgMCAp O2JhY2tncm91bmQtY29sb3I6cmdiKCAyNTUgLCAyNTUgLCAyNTUgKSI+PGJyIC8+PC9kaXY+PGRp diBzdHlsZT0iZm9udC1mYW1pbHk6JiMzOTthcmlhbCYjMzk7ICwgc2Fucy1zZXJpZjtmb250LXNp emU6MTRweDtjb2xvcjpyZ2IoIDAgLCAwICwgMCApO2JhY2tncm91bmQtY29sb3I6cmdiKCAyNTUg LCAyNTUgLCAyNTUgKSI+Q2FuIGl0IGJlIHJldmlld2VkIGFuZCBjb21taXR0ZWQgdG8gQ1VSUkVO VD88L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTomIzM5O2FyaWFsJiMzOTsgLCBzYW5zLXNl cmlmO2ZvbnQtc2l6ZToxNHB4O2NvbG9yOnJnYiggMCAsIDAgLCAwICk7YmFja2dyb3VuZC1jb2xv cjpyZ2IoIDI1NSAsIDI1NSAsIDI1NSApIj5PciwgaWYgdGhlcmUgaXMgbm8gY29tbWl0dGVycyB3 aWxsaW5nIHRvIGRvIGl0LCBjYW4gY29tbWl0IGJpdCBiZSBnaXZlbiB0byBzdWJtaXR0ZXIgb3Ig YW5vdGhlciBwZXJzb24gd2lsbGluZyB0byBkbyB0aGlzPzwvZGl2PjxkaXYgc3R5bGU9ImZvbnQt ZmFtaWx5OiYjMzk7YXJpYWwmIzM5OyAsIHNhbnMtc2VyaWY7Zm9udC1zaXplOjE0cHg7Y29sb3I6 cmdiKCAwICwgMCAsIDAgKTtiYWNrZ3JvdW5kLWNvbG9yOnJnYiggMjU1ICwgMjU1ICwgMjU1ICki PjxiciAvPjwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiYjMzk7YXJpYWwmIzM5OyAsIHNh bnMtc2VyaWY7Zm9udC1zaXplOjE0cHg7Y29sb3I6cmdiKCAwICwgMCAsIDAgKTtiYWNrZ3JvdW5k LWNvbG9yOnJnYiggMjU1ICwgMjU1ICwgMjU1ICkiPkl0JiMzOTtzIHZlcnkgZGlzYXBwb2ludGlu ZyB3aGVuIHVzZXJzIHNwZW5kIHRoZWlyIHRpbWUgdG8gZmlsbCBzdWNoIGdhcHMgYW5kIHRoZWly IGVmZm9ydHMganVzdCBpZ25vcmVkIGJ5IHRoZSBkZXZlbG9wZXJzLjxiciAvPjwvZGl2PjxkaXYg c3R5bGU9ImZvbnQtZmFtaWx5OiYjMzk7YXJpYWwmIzM5OyAsIHNhbnMtc2VyaWY7Zm9udC1zaXpl OjE0cHg7Y29sb3I6cmdiKCAwICwgMCAsIDAgKTtiYWNrZ3JvdW5kLWNvbG9yOnJnYiggMjU1ICwg MjU1ICwgMjU1ICkiPkV2ZXJ5IHllYXIgRnJlZUJTRCBDb21tdW5pdHkgU3VydmV5IGFza2luZyBh Ym91dCB1c2VyIGV4cGVyaWVuY2UgaW4gY29udHJpYnV0aW5nIHRvIEZyZWVCU0QuIDxiciAvPjwv ZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiYjMzk7YXJpYWwmIzM5OyAsIHNhbnMtc2VyaWY7 Zm9udC1zaXplOjE0cHg7Y29sb3I6cmdiKCAwICwgMCAsIDAgKTtiYWNrZ3JvdW5kLWNvbG9yOnJn YiggMjU1ICwgMjU1ICwgMjU1ICkiPkhlcmUgeW91IGNhbiBzZWUgYW4gZXhhbXBsZSBvZiBzdWNo IGNvbnRyaWJ1dGluZy48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTomIzM5O2FyaWFsJiMz OTsgLCBzYW5zLXNlcmlmO2ZvbnQtc2l6ZToxNHB4O2NvbG9yOnJnYiggMCAsIDAgLCAwICk7YmFj a2dyb3VuZC1jb2xvcjpyZ2IoIDI1NSAsIDI1NSAsIDI1NSApIj48L2Rpdj48ZGl2IHN0eWxlPSJm b250LWZhbWlseTomIzM5O2FyaWFsJiMzOTsgLCBzYW5zLXNlcmlmO2ZvbnQtc2l6ZToxNHB4O2Nv bG9yOnJnYiggMCAsIDAgLCAwICk7YmFja2dyb3VuZC1jb2xvcjpyZ2IoIDI1NSAsIDI1NSAsIDI1 NSApIj48YnIgLz48L2Rpdj48L2Jsb2NrcXVvdGU+PGRpdj48YnIgLz48L2Rpdj48ZGl2PkZpcnN0 LCB0aGFuayB5b3UgZm9yIGJlaW5nIHBlcnNpc3RlbnQgYW5kIGNvbnRpbnVpbmcgdG8gYnJpbmcg aXQgdXAuIEl0JiMzOTtzIGltcG9ydGFudCB0byBkbyB0aGF0IHRvIG1ha2Ugc3VyZSB0aGlzIChh bmQgeW91ciBtYW55IG90aGVyKSBjb250cmlidXRpb24gZG9lc24mIzM5O3QgZmFsbCBvbiB0aGUg Zmxvb3IuPGJyIC8+PC9kaXY+PGRpdj48YnIgLz48L2Rpdj48ZGl2PkFuZCB0byBiZSBmYWlyLCB3 ZSYjMzk7cmUgb25seSAzIG1vbnRocyBzaW5jZSB0aGUgbGFzdCB1cGRhdGUuIFN0aWxsLCBxdWl0 ZSBhIGJpdCBsb25nZXIgdGhhbiB5b3Ugc2hvdWxkIGhhdmUgdG8gd2FpdCwgYnV0IG5vdCBuZWFy bHkgdGhlIHllYXIgdGhlIG9yaWdpbmFsIGRhdGUgc3VnZ2VzdHMuPGJyIC8+PC9kaXY+PGRpdj48 YnIgLz48L2Rpdj48ZGl2PkFuZCB0aGlzIGlzIGEgcGVyZmVjdCBzdG9ybSBvZiAmIzM0O2hvdyB0 aGUgcHJvamVjdCBpcyBiYWQgYXQgYWNjZXB0aW5nIGNvbnRyaWJ1dGlvbnMmIzM0Ozo8L2Rpdj48 ZGl2PigxKSBUaGUgb3JpZ2luYWwgc3VibWlzc2lvbiB3YXMgY2xvc2UgdG8gdGhlIDE0IGJyYW5j aCBjcmVhdGlvbiB0aW1lLiBUaGlzIG1lYW50IHRoYXQgd2Ugd2VyZW4mIzM5O3Qgd2VsbCBwcmVw YXJlZCB0byBsb29rIGF0IGl0IHNpbmNlIGl0IGlzIHN1Y2ggYW4gaW52YXNpdmUgY2hhbmdlIChh dCBsZWFzdCBvbiBpdHMgc3VyZmFjZSkuIEl0IGFsc28gc2xvd2VkIHRoZSBpbml0aWFsIHJlc3Bv bnNlLi4uPGJyIC8+PC9kaXY+PGRpdj4oMikgVGhlcmUgd2FzIGEgbnVtYmVyIG9mIGJhY2sgYW5k IGZvcnRoIHJlcXVlc3RzIGZvciBjaGFuZ2VzLCB3aGljaCB0b29rIHRpbWUgdG8gc29ydCBvdXQu Li48L2Rpdj48ZGl2PigzKSBUaGUgc2l6ZSBvZiB0aGlzIGlzIGh1Z2UsIHdlbGwgYmV5b25kIHRo ZSBjYXBhY2l0eSBvZiBQaGFicmljYXRvciB0byByZXZpZXcgYWNjdXJhdGVseS4uLjwvZGl2Pjxk aXY+KDQpIEl0JiMzOTtzIGEgdmVuZG9yIGltcG9ydC4gVGhhdCBtZWFucyB3ZSBjYW4mIzM5O3Qg anVzdCBkcm9wIHRoZSBQaGFicmljYXRvciByZXZpZXcgaW50byB0aGUgdHJlZS4uLjwvZGl2Pjxk aXY+KDUpIEl0JiMzOTtzIHBoYWJyaWNhdG9yOiB0aGlzIGlzIGEgZ3JlYXQgdG9vbCBmb3IgZGV2 ZWxvcGVycywgYnV0IHdlIGhhdmUgYSB0ZXJyaWJsZSB0cmFjayByZWNvcmQgb2YgdXNpbmcgaXQg Zm9yIGludGFrZSBmcm9tIG5ldyBjb250cmlidXRvcnMuIFdlIGRvbiYjMzk7dCBoYXZlIGFueSBv dmVyc2lnaHQgYXQgYWxsIG92ZXIgdGhpcyB0b29sLCBhdCB0aGVyZSYjMzk7cyBhdCBiZXN0IHRl cGlkIGFuZCBsdWtlIHdhcm0gYXR0ZW1wdHMgdG8gbG9vayBmb3IgZHJvcCBiYWxscy48L2Rpdj48 ZGl2PjxiciAvPjwvZGl2PjxkaXY+QWxsIG9mIHRoZXNlIHRoaW5ncyBhcmUgYSB0ZXJyaWJsZSBl eHBlcmllbmNlLiBJIGNhbiBvbmx5IGFwb2xvZ2l6ZS4gVGhlc2UgZGF5cywgd2UgbWlnaHQgc3Rl ZXIgdGhpcyB0b3dhcmRzIGdpdGh1YiwgYnV0IHRoZSAmIzM5O3ZlbmRvciBpbXBvcnQmIzM5OyBt ZWFucyB5b3UgcmVhbGx5IG5lZWQgc29tZW9uZSBvbiB0aGUgaW5zaWRlLCBvciB5b3UgbmVlZCB0 byBiZSBvbiB0aGUgaW5zaWRlIHRvIG1ha2UgdGhhdCB3b3JrLjwvZGl2PjxkaXY+PGJyIC8+PC9k aXY+PGRpdj5TbywgaG93IHRvIG1vdmUgZm9yd2FyZD8gV2VsbCwgSSYjMzk7ZCBsaWtlIHRvIHBy b3Bvc2UgdGhlIGZvbGxvd2luZzo8L2Rpdj48ZGl2PigxKSBzdWJtaXQgYWxsIHRoZSBvdGhlciBQ aGFicmljYXRvciByZXZpZXdzIHlvdSBoYXZlIG9wZW4gKHRoZXkgYXJlIG1vc3RseSBnb29kLCBv ciBjbG9zZSB0byBnb29kKSB0byBnaXRodWIuIEdpdGh1YiBpcyBiZWluZyBhY3RpdmVseSBtYW5h Z2VkIGFuZCB3aWxsIG1ha2UgaXQgZmFzdGVyIHRvIGdldCB0aGluZ3MgaXQuIEl0JiMzOTtzIGEg bXVjaCBiZXR0ZXIgdG9vbCBmb3IgbmV3IGNvbnRyaWJ1dG9ycyAoYW5kIGV2ZW4gZnJlcXVlbnQg Y29udHJpYnV0b3JzIG9mIHNtYWxsaXNoIHRoaW5ncykuPC9kaXY+PGRpdj4oMikgSSBzaG91bGQg ZG8gYW4gdmVuZG9yIGltcG9ydCBvZiA1LjMuMCBmcm9tIGdpdGh1YiwgYW5kIGRvIHRoZSBtZXJn ZSB0byBhIGJyYW5jaCBhbmQgcHVzaCB0aGF0IHRvIGdpdGh1Yi4gWW91IGNhbiB0aGVuIGxheWVy IG9uIHlvdXIgY2hhbmdlcyBhbmQgdGhvc2UgY2FuIGJlIHJldmlld2VkIG1vcmUgY2xvc2VseSBh cyBhIHB1bGwgcmVxdWVzdCBhZ2FpbnN0IHRoZSBicmFuY2ggSSBwdXNoLiBJIHN1c3BlY3QgdGhh dCBtb3N0IG9mIHRoZSBpc3N1ZXMgYXJlIHNvcnRlZCBvdXQgYWxyZWFkeSA8YnIgLz48L2Rpdj48 ZGl2PigzKSBJJiMzOTtsbCBsYW5kIGl0IHZpYSB0aGF0IHJvdXRlLi4uPC9kaXY+PGRpdj48YnIg Lz48L2Rpdj48ZGl2PkFuZCwgaWYgdGhlIHN1bSBvZiB0aGUgb3RoZXIgcHVsbCByZXF1ZXN0cyBh bmQgdGhpcyBhcmUgZ29vZCAoYW5kIEkgc3VzcGVjdCB0aGV5IHdpbGwgYmUpLCB0aGVuIHdlIGNh biB0YWxrIGFib3V0IGNvbW1pdCBiaXRzIGFuZCBzdWNoLjwvZGl2PjxkaXY+PGJyIC8+PC9kaXY+ PGRpdj5JdCYjMzk7cyBleHBlcmllbmNlcyBsaWtlIHRoaXMgd2hpY2ggaXMgd2h5IEkmIzM5O20g dHJ5aW5nIHRvIHN0YW5kIHVwIGdpdGh1YiBwdWxsIHJlcXVlc3RzIGFzIGEgcmVsaWFibGUgd2F5 IHRvIGdldCB0aGluZ3MgYW5kIGFuZCB0aGUgYmVzdCBwbGFjZSB0byBzZW5kIHBlb3BsZS4uLiAg PGJyIC8+PC9kaXY+PGRpdj48YnIgLz48L2Rpdj48ZGl2PlRoYW5rcyBhZ2FpbiBmb3IgcGVyc2lz dGluZywgYW5kIGFsc28gZm9yIGV4cHJlc3NpbmcgdGhpcyBjcml0aWNpc20gdGhhdCB3ZSAoaG9w ZWZ1bGx5KSBjYW4gdXNlIHRvIG1ha2UgaXQgYmV0dGVyLjxiciAvPjwvZGl2PjxkaXY+PGJyIC8+ PC9kaXY+PGRpdj5XYXJuZXI8YnIgLz48L2Rpdj48L2Rpdj48L2Rpdj4NCg0KICAgICAgICA8L2Js b2NrcXVvdGU+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6JiMzOTthcmlhbCYjMzk7ICwg c2Fucy1zZXJpZjtmb250LXNpemU6MTRweDtjb2xvcjpyZ2IoIDAgLCAwICwgMCApO2JhY2tncm91 bmQtY29sb3I6cmdiKCAyNTUgLCAyNTUgLCAyNTUgKSI+PGJyIC8+PC9kaXY+PGRpdiBzdHlsZT0i Zm9udC1mYW1pbHk6JiMzOTthcmlhbCYjMzk7ICwgc2Fucy1zZXJpZjtmb250LXNpemU6MTRweDtj b2xvcjpyZ2IoIDAgLCAwICwgMCApO2JhY2tncm91bmQtY29sb3I6cmdiKCAyNTUgLCAyNTUgLCAy NTUgKSI+SGVsbG8uPC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6JiMzOTthcmlhbCYjMzk7 ICwgc2Fucy1zZXJpZjtmb250LXNpemU6MTRweDtjb2xvcjpyZ2IoIDAgLCAwICwgMCApO2JhY2tn cm91bmQtY29sb3I6cmdiKCAyNTUgLCAyNTUgLCAyNTUgKSI+PGJyIC8+PC9kaXY+PGRpdiBzdHls ZT0iZm9udC1mYW1pbHk6JiMzOTthcmlhbCYjMzk7ICwgc2Fucy1zZXJpZjtmb250LXNpemU6MTRw eDtjb2xvcjpyZ2IoIDAgLCAwICwgMCApO2JhY2tncm91bmQtY29sb3I6cmdiKCAyNTUgLCAyNTUg LCAyNTUgKSI+SSYjMzk7bSBub3QgdGhlIGF1dGhvciBvZiBENDE0MjEuIEp1c3QgYXBwbGllZCB0 aGUgcGF0Y2ggdG8gdGVzdCBpdCA4IG1vbnRocyBhZ28uIEFuZCByZWNlbnRseSBkaXNjb3ZlcmVk IHRoYXQgaXQmIzM5O3Mgc3RpbGwgbm90IGNvbW1pdHRlZC48L2Rpdj48ZGl2IHN0eWxlPSJmb250 LWZhbWlseTomIzM5O2FyaWFsJiMzOTsgLCBzYW5zLXNlcmlmO2ZvbnQtc2l6ZToxNHB4O2NvbG9y OnJnYiggMCAsIDAgLCAwICk7YmFja2dyb3VuZC1jb2xvcjpyZ2IoIDI1NSAsIDI1NSAsIDI1NSAp Ij5JIGNhbiYjMzk7dCBjb3B5IHlvdXIgbWVzc2FnZSB0byBQaGFicmljYXRvciBiZWNhdXNlIGRv biYjMzk7dCBoYXZlIGFuIGFjY291bnQuIFBsZWFzZSwgaWYgeW91IGhhdmUgdGltZSwgaGVscCB0 aGUgYXV0aG9yIGluIEQ0MTQyMS48L2Rpdj48L2Jsb2NrcXVvdGU+PGRpdj48YnIgLz48L2Rpdj48 ZGl2PkFoIHllcy4gSSYjMzk7dmUgYmVlbiBpbiB0b3VjaCB3aXRoIHRoZSBhdXRob3IgZm9yIG90 aGVyIHRoaW5ncywgYW5kIHNvbWVob3cgdGhvdWdodCBpdCB3YXMgeW91Li4uLiAgSSYjMzk7bGwg cmVhY2ggb3V0IHRvIGhpbSB2aWEgb3RoZXIgbWVhbnMuLi48L2Rpdj48ZGl2PjxiciAvPjwvZGl2 PjxkaXY+V2FybmVyPC9kaXY+PC9kaXY+PC9kaXY+DQoNCiAgICAgICAgPC9ibG9ja3F1b3RlPjxi ciAvPg0KICAgIDwvZGl2Pg0KICAgICAgICA8L2Jsb2NrcXVvdGU+PGJyIC8+DQogICAgPC9kaXY+ PC9ibG9ja3F1b3RlPjwvZGl2Pg0KDQogICAgICAgIDwvYmxvY2txdW90ZT48YnIgLz4NCiAgICA8 L2Rpdj4NCiAgICAgICAgPC9ibG9ja3F1b3RlPjxiciAvPg0KICAgIDwvZGl2PjwvYmxvY2txdW90 ZT48L2Rpdj4NCjwvYmxvY2txdW90ZT48L2Rpdj48L2Rpdj48L2Jsb2NrcXVvdGU+PC9kaXY+ From nobody Sun Dec 1 23:11:00 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Y1jMt21Lkz5f6cX; Sun, 01 Dec 2024 23:11:22 +0000 (UTC) (envelope-from scf@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Y1jMt1Fk5z4WNs; Sun, 1 Dec 2024 23:11:22 +0000 (UTC) (envelope-from scf@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1733094682; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=7eNf3q/er97jmSNTOL1ITiSlimkX5YiTAyEq0FsCwEA=; b=XbKSbxUm+Ehnk7MeeYz90tgH8RRQ58sfav5e6mAT5bnMctPB2od4HESwrseMZJ2zht75WT SDYdN3NaQHqewJudrxlrqRj0gIzd3HncE8MqG2f9ClfvLdKWzUdO9UM2hEpdwy0wPsKGj3 Y33lWYWYOOcdCPKvvqgdegFBmZRgO5IB4+dRrr3+2sTjOwRzVp/KsANR02O97syqaBfm96 peh9ChjNuwPmezidEDEpDvgsoQj5o9ccFTkWV6jTtAptd/YOEwciZdxJRlirT1D+kDuwGH gFyjRM+B5AqG7WE0ld81gSWnBBp168k6kmzLxyGwXlBcufaF9NLBzvlPVBEA3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1733094682; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=7eNf3q/er97jmSNTOL1ITiSlimkX5YiTAyEq0FsCwEA=; b=sbdFO3MgL/BHjDe0FeO5pSndC96tMCbCekhZDyswdKzfLXREgF/ak+vT/rcRtmnc0i/e6M uThcfzUEnB2QLvp997g4kHBNjWZd9QJbVFAqA+YrV8IcqZb+2W/hZ0oMQrr6N5Dg7Z4NA3 F9RxE2r9bOg8gLRQl4eHyz5XoCZBpEgU369havoeA+hrUdDn9/QwhTzA+IY+MIndXgYMnD Jy+zz5kZq2MY19DcumZK9SKD4F1bbYzcd9dmqtcCFae54jLwe7xaGwrFlqY7INMY/Jcud2 q4TsQXUJhsmE1BVRCp50/CSu5MZoO/Pu7bA0e/UJ0SMXhx8oLbjY/tIM9RWv5g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1733094682; a=rsa-sha256; cv=none; b=o04zzZubjietI9wCB4rrx5CBkOtUpnsvSeRAsEWmyqP31JVGCVRhShYTki/ZHhc9i+YBy1 An3iqg1USK/EpRcBt9V8ewW/eieH25wpjioZGTuUajpKdDbT4P1XDTI3sYs7L8i3FVpVQv lcOAzT8D4D8oGmR3MADBIs66Kc2UQn8OO24a8iaa1S4MAQy7nWsxCSiIHbb2ysL+0LVWr8 sQl25mIwmOCnoJolHgajT8IJAyQrdCzXi57qoilJzo8c83LQ6COIgqh35/AAxo870GE3g2 08x+Q967V6jfX4iCqJMKxa0buc3nU+uqYvm0Xj/Jj2aO2d8q7q6nDlpTN/AVug== Received: from thor.farley.org (1609341-v107.1360-static.crmlinaa.metronetinc.net [104.254.222.35]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: scf/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Y1jMs6PTvzWsx; Sun, 1 Dec 2024 23:11:21 +0000 (UTC) (envelope-from scf@FreeBSD.org) Date: Sun, 1 Dec 2024 18:11:00 -0500 (EST) From: "Sean C. Farley" To: Mark Millard cc: FreeBSD Current , FreeBSD Mailing List Subject: Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros] In-Reply-To: <3AC4C67F-2DA3-4AA0-9B69-066146A5771E@yahoo.com> Message-ID: References: <4D541C32-7DF4-4CAC-B31E-D4DD17977154.ref@yahoo.com> <4D541C32-7DF4-4CAC-B31E-D4DD17977154@yahoo.com> <7f4080f5-a371-9940-480b-df13956c76ab@FreeBSD.org> <3AC4C67F-2DA3-4AA0-9B69-066146A5771E@yahoo.com> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/mixed; BOUNDARY="3279119474-128719847-1733093882=:88209" Content-ID: This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --3279119474-128719847-1733093882=:88209 Content-Type: text/plain; CHARSET=UTF-8; format=flowed Content-Transfer-Encoding: 8BIT Content-ID: On Thu, 28 Nov 2024, Mark Millard wrote: > On Nov 28, 2024, at 19:54, Sean C. Farley wrote: > >> On Thu, 28 Nov 2024, Mark Millard wrote: >> >>> . . . >> >>>> System setup: >>>> - FreeBSD 14.2-STABLE >>> >>> The context that I investigated --and what was fixed by a commit only >>> applies to-- main [so; 15 as stands], not stable/14 . >>> >>> stable/14 has no commits mentioning "tmpfs" after 2024-Jun-04. >> >> Thank you. That was my mistake. I will continue searching for an >> answer. Once I find a way to more consistently trigger it, it will >> be much easier. >> >> I ran all of the tmpfs*.sh tests from HEAD which all pass except for >> tmpfs24.sh. > > Did you notice: > > Thu, 28 Nov 2024 > . . . > • git: 2e2699c48a7e - main - stress2: Added new tmpfs test scenarios Peter Holm > > that added: > > stress2: Added new tmpfs test scenarios > --- > tools/test/stress2/misc/tmpfs26.sh | 179 +++++++++++++++++++++++++++++++++++++ > tools/test/stress2/misc/tmpfs27.sh | 49 ++++++++++ > tools/test/stress2/misc/tmpfs28.sh | 61 +++++++++++++ > 3 files changed, 289 insertions(+) Yes, I had noticed those and tested those too albeit on a 14.2-STABLE. They passed as expected. Only tmpfs24.sh failed. It failed on two 14.2-STABLE systems and one 13.3-RELEASE-p7. I am not thinking it related to my problem, but it is unexpected. >> $ ./all.sh -o tmpfs24.sh >> 20241128 22:33:38 all: tmpfs24.sh >> Min hole size is 4096, file size is 524288000. >> data #1 @ 0, size=4096) >> hole #2 @ 4096, size=4096 >> data #3 @ 8192, size=4096) >> hole #4 @ 12288, size=4096 >> data #5 @ 16384, size=4096) >> hole #6 @ 20480, size=524267520 >> --- /tmp/tmpfs24.exp 2024-11-28 22:33:40.222199000 -0500 >> +++ /tmp/tmpfs24.log 2024-11-28 22:33:40.225048000 -0500 >> @@ -5,4 +5,3 @@ >> hole #4 @ 12288, size=4096 >> data #5 @ 16384, size=4096) >> hole #6 @ 20480, size=524267520 >> -<> >> FAIL tmpfs24.sh exit code 1 Sean -- scf@FreeBSD.org --3279119474-128719847-1733093882=:88209--