From owner-freebsd-stable@freebsd.org Mon May 3 00:27:53 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 46DBB5FF86D for ; Mon, 3 May 2021 00:27:53 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-eopbgr670082.outbound.protection.outlook.com [40.107.67.82]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FYP2J16MFz3kst; Mon, 3 May 2021 00:27:51 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IJRnHnkv8YJkLJk7SdKLJ0+8QSp8WjhhIvkproPqbAtCy4Xk6vLzIzCP5mQWRiIOz8VTm6vQ8bWPyv5dlZ/35Ac72/fyRjzksr051w9Pf5/+coCSjkEzSdj4BwZbHgHUfgPYzjfduq0kvkfL1FWO5G8J1CiibFOTuVqEiGAblBbJ6iPSbRu81b7B9sRGcHlF+vlNTkk67gah5AEZhA3r8LNCQ4Q1w1X2w2MG9rlc9S66HJjDgmzBo4fFCFo5jWHMvvd1JbdulKmMzfx/moLfFuUqYSCiT3D2+GzyjBm65oFUvZNWQLea3H/uJqoFU/aDJcNs77XUpOAbBh0QT5quZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=d2cVHY1xu7/c2xCEjWRj6HtaOsAFhVE5HpeAE+rzT9g=; b=UnMW5X+beOWOANeBWPpuPKup4HcYqfkTlhxK0HxwZfpj+HM8DY4suXIMxP3Xl0ctYu/eEXG+ymNfa/Xt9PlCsbtivqYXZi0bE3Kf/SCpoxrkpDGNFJ4Y2LWgOOiJzGTCa0wTRVrM7Yzg2l66WkyPx3rx5R2UdnAgjfpqUAwDTwCaqGwnRqe/dNyrwzHcda9uqYOJmiMRLiZeOCBaQJ5i2taw2vvytRAnEXFrNZV/4P8meQoP/WjCE28/VNTN0m7zOKjgevVVbH1RYrCIEaQTmbHfilmqYWorQPZWyf+8sL1MbkJb0gD/78y7O+HkqqmKw9/XBcffDF1OtM/EE3g9IA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uoguelph.ca; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=d2cVHY1xu7/c2xCEjWRj6HtaOsAFhVE5HpeAE+rzT9g=; b=cuMEMja96IFz8E3G4tKSDLCwwTl7UW4l9RJUrBAbxuXOlzpYYIoF9lxpoln7s85FOIsFC3ec7Ocyxadvaes0ACnSF/R3IfbG0ANfcTP4s9zerQj0vSxKl91+3ve9eJP1LI4Y6t2CZgv7CKguv4f4hHr3/0IZL8CUItuDmRsuPVGCukbfo5LlPrYAQAzNcIh6dHpWcD9yRcaml2i1oGoG+F9T0a2ripvuwFojCopE7jfLn13hWs5zaWXf72p82iKeElFk9ZEbw1XpwTFz/GR9IdHgjQewZnB3omzhfYLgL51k8GI2BPLfjty2t1owGKq6RTs8FoCLeSvNt4i+K5IVtg== Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:19::29) by YQXPR0101MB1478.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:17::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25; Mon, 3 May 2021 00:27:49 +0000 Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::1c05:585a:132a:f08e]) by YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::1c05:585a:132a:f08e%4]) with mapi id 15.20.4087.043; Mon, 3 May 2021 00:27:42 +0000 From: Rick Macklem To: freebsd-stable CC: Peter Eriksson , Ryan Moeller , Garrett Wollman , Alan Somers , Juraj Lutter Subject: Re: wanna solve the Linux NFSv4 client puzzle? Thread-Topic: wanna solve the Linux NFSv4 client puzzle? Thread-Index: AQHXPEHIsjwmd971wkuLdHVmW5Ho6KrQ2a+E Date: Mon, 3 May 2021 00:27:42 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 811809be-0fa2-4314-1f48-08d90dca4426 x-ms-traffictypediagnostic: YQXPR0101MB1478: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:538; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: eVsvDg0nt631mFI6CKUV4ZoWm9QDWoOfwayJ3ZfH2LbIiDRXfQVGKnlLtG8okPMYNMNpxnqlofFi9E/N0j+dXuUdJAX9aJ7Ov68pgmGSksNZrRFPkjHXuWZCdkV7R2WNDwaHziwJkSwU2j7l02iUXugNMnOljnFD5IW2Md++51m21CGgxkZunRWIVWzMFOdaQwZCNAMbZlC1Fq0mpy8fzoTTNPcGF0uM5MC+/Uk5zvfnIrkxVsXJUw0JGC5qkP4aufQjW7q2LX+tyi5arapny0VCH7W9gbT5aZZT13sdvARLtlGVxTVaPfdI/lxf0+iZqU04+j/5Lt/qdPitdBFup/wiYzOXc2C2iunAJeWBQMRrPe93japTu13RXjfACRmiJw6sCsZ9MhyAnu9FEn8EGI0m8oZJ5L6JnMxNqyEDay2pcJh77j4zULGwlluYBOchhyqfrbIkmiUYeD5o7LidEx8m8upXW2wD/8WUDCMBMXaJEYepSBhRyaR3C0CfDCwys1T4RHYMsCAD4Xa7mBlvJBWSxrZYqT4b3Ir4hUR7+/4/qazYacK7AFLY3Coo/frE/RzNekxAfZDOsRRTin2H7UiMS3Imu1xfC8/LvGs8hp57x8rgzObTO5iihWu67+JbrqstlgxTPURksvDnDxQESgoRjpJa9K5ksjgmDp3GusdkxtYN/ZSQYBGbtsiMtMfy x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(366004)(346002)(396003)(376002)(136003)(39850400004)(2906002)(7696005)(9686003)(66476007)(8676002)(5660300002)(8936002)(6506007)(186003)(316002)(54906003)(786003)(478600001)(55016002)(4326008)(52536014)(86362001)(6916009)(83380400001)(91956017)(66946007)(38100700002)(966005)(122000001)(66446008)(76116006)(33656002)(71200400001)(66556008)(64756008); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?iso-8859-1?Q?kSDIGwOCSEB5bgLs0yHGmuTizjgIE00NL+LTjD8G51pQqz3hgH/VG3RMx5?= =?iso-8859-1?Q?qSsGWl6u64cOEsj6J/iSFzHx3D3bpM085CF9ArwNMdD1cfjnNnGUE69tOc?= =?iso-8859-1?Q?ftTKZ0MU/AEvi3fyR/M0sLAjtnS+W+UK2m5zud35B+3y3Tj6U0bx61wgXr?= =?iso-8859-1?Q?0GY+yX1gVG2VDwi4MThd6hSuwnsBbtXDrwGac4MOYNnQjrtaw8yfHc3bAS?= =?iso-8859-1?Q?lnBmUNNEx6SCzbQawE8gZW9EAvy1deW7MsMimTEvkcsEHMoip02hz25jNg?= =?iso-8859-1?Q?cvyqelbzrmI1oi27rONu3wHTV1l9F4JEMBZumh0fUo5HHtyzbVXNGobjhU?= =?iso-8859-1?Q?2AIxRFsfT4Eix6dpFlvriGU1yIl5jW139Ru9k8HldHG02mEQnzX1H1RBU6?= =?iso-8859-1?Q?qDCkqz8BK3B9z2m+PGyDFrQ/d18hpCc/I+tR67hEO4cQ5rIqp/L46mHMWo?= =?iso-8859-1?Q?Mla3vYKM6CWcxqRFq9aF68tZkb/dn5Fajc0WsXTiE9uvNIkwaRdQ3RTejF?= =?iso-8859-1?Q?+W1i12dJa4CS451s/SXpvzyplvJdwjGMsp+xpb1AsVsN++ebYUaCX48QbP?= =?iso-8859-1?Q?0rJziWbyercLzJHPBGlEa8kdSgvwvsBg0nXKPkq7++G3PGwXBsZwBwECyu?= =?iso-8859-1?Q?SoCTmg1jCne1ywhXvc/zpugpRjScv2MHocyPUh/EEDf3sZfJtg+zEZCp2L?= =?iso-8859-1?Q?62OEGayTCuP7+CPerR5VMTvKMgc4i7QsW1+tR93Yc1r4GdMxdUm6wYr3WY?= =?iso-8859-1?Q?UXO3Fpny1QnGPvJyiYWdElaEPu28Aas6PQABJCKrnGIpG3kCvDOA01U/7r?= =?iso-8859-1?Q?3SzH+4AAfIRanLQX65J+ps8DnAafyeB/920/jVXLmig+0bwj2iRYdBxz0B?= =?iso-8859-1?Q?ROMQVW5w7J2qlje7qfI9UPXPJ0EHyRMuFEpIt1aicwUjNN8jk2Ps9RKGOf?= =?iso-8859-1?Q?7SGIKjko47THJ5jITFvmfrBKuwXFS+DqXLrkRByYuB8hxZMWKAv4TeqHoc?= =?iso-8859-1?Q?yP0+nZlk7a4doE6EPKtvIvKIqb5uPfaP8OxGMgq2/DjHXHL82UCQQ+oRm3?= =?iso-8859-1?Q?bz4TDAEDhFpE8O2MnQOShvrNxEQ/zFw+ghaXySj6z8bO1ciPPwuoNEd0pd?= =?iso-8859-1?Q?0K88VQsfgphFp1yEDmlc28+7Go49HqBCi9NChyfHwMgEpuryYQUvE5r8mV?= =?iso-8859-1?Q?XHddYb4r3IKwoKJqtAgQ2KEMf1vQhV6u6RcyiLInq0khcJu0vi1JmsEOfv?= =?iso-8859-1?Q?XENngiP5NEt+d1vFZp3ebcF9/GoYxFpPnQtIpDuzrnrldcXR6+4smxnuEV?= =?iso-8859-1?Q?rWkU6wKqETm83UVKMb/JkLIXB6suxP0Ox+hzH0mu+3dQeNmEKmvTh+YNrG?= =?iso-8859-1?Q?c5t9CScMRne7E+ddZIE5L9n3sJJ41MFk+HBZOUnleq/2ZAtplg+ZWoljho?= =?iso-8859-1?Q?XAb+SXiQmW6J8oA9?= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: 811809be-0fa2-4314-1f48-08d90dca4426 X-MS-Exchange-CrossTenant-originalarrivaltime: 03 May 2021 00:27:42.6260 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: uPZ6Hzq4nCgDXdx4615BHBapgwtJm4n7kKzrjsbwcXSSIznpMpfqvc3FpjXHPBJ6WaB9vnFYyymzQQ5Mr48tvw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQXPR0101MB1478 X-Rspamd-Queue-Id: 4FYP2J16MFz3kst X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=uoguelph.ca header.s=selector1 header.b=cuMEMja9; arc=pass (microsoft.com:s=arcselector9901:i=1); dmarc=pass (policy=none) header.from=uoguelph.ca; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 40.107.67.82 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-3.00 / 15.00]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; RCPT_COUNT_FIVE(0.00)[6]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[uoguelph.ca:+]; DMARC_POLICY_ALLOW(-0.50)[uoguelph.ca,none]; FROM_EQ_ENVFROM(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[40.107.67.82:from]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:8075, ipnet:40.104.0.0/14, country:US]; NEURAL_HAM_MEDIUM(-1.00)[-0.998]; R_DKIM_ALLOW(-0.20)[uoguelph.ca:s=selector1]; FREEFALL_USER(0.00)[rmacklem]; RCVD_TLS_LAST(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(1.00)[0.999]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[40.107.67.82:from:127.0.2.255]; DWL_DNSWL_LOW(-1.00)[uoguelph.ca:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[40.107.67.82:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[40.107.67.82:from]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 May 2021 00:27:53 -0000 Rick Macklem wrote:=0A= >Hi,=0A= >=0A= >I posted recently that enabling delegations should be avoided at this time= ,=0A= >especially if your FreeBSD NFS server has Linux client mounts...=0A= >=0A= >I thought some of you might be curious why, and I thought it would be=0A= >more fun if you look for yourselves.=0A= >To play the game, you need to download a packet capture:=0A= >fetch https://people.freebsd.org/~rmacklem/twoclientdeleg.pcap=0A= >and then load it into wireshark.=0A= >=0A= >192.168.1.5 - FreeBSD server with all recent patches=0A= >192.168.1.6 - FedoraCore 30 (Linux 5.2 kernel) client=0A= >192.168.1.13 - FreeBSD client=0A= >=0A= >A few hints buried in RFC5661:=0A= >- A fore channel is used for normal client->server RPCs and a back channel= =0A= > is used for server->client callback RPCs.=0A= >- After a new TCP is created, neither the fore nor back channels=0A= > are bound to the connection.=0A= >- Bindings channel(s) to a connection is done by BindConnectionToSession.= =0A= > but an implicit binding for the fore channel is created when the first R= PC=0A= > request with a Sequence operation in it is sent on the new TCP connectio= n.=0A= >- A server->client callback cannot be done until the back channel is bound= =0A= > via BindConnectionToServer.=0A= >=0A= >Ok, so we are ready...=0A= >- Look at packet #s 3518->3605.=0A= > - What is going on here?=0A= Ok, so here's my solution...=0A= packet #3518, 3520 and 3521 are delegation recalls (CB_RECALL)=0A= for 3 different delegations on three different session slots.=0A= time: 137.5=0A= =0A= Expected response from the Linux client--> 3 replies to the CB_RECALLs.=0A= What does it actually do?=0A= --> Creates a new TCP connection using same port#. You can see it send=0A= a FIN (packet# 3523) and a SYN (packet# 3527).=0A= This means that the client is no longer obliged to reply to the CB_RE= CALLs=0A= and the FreeBSD server will probably need to retry them.=0A= --> It also means that no back channel is bound to the session, so th= e=0A= server cannot do callbacks (ie. cannot retry the CB_RECALLs ye= t).=0A= =0A= packet# 3530 is a Setattr RPC, which has a Sequence operation in it.=0A= --> This means the fore channel is implicitly bound to the new TCP=0A= connection, but no back channel, so the server cannot retry the CB_RE= CALLs.=0A= =0A= You will notice a bunch of Setattr RPCs getting NFS4ERR_DELAY replies.=0A= This tells the Linux client to "try again later".=0A= --> It happens because the FreeBSD server cannot perform the Setattr=0A= until the client returns a delegation.=0A= --> That requires a CB_RECALL.=0A= =0A= packet# 3582 is a Setattr RPC reply. If you look in the Sequence operation= =0A= reply, you will see the flag SEQ4_STATUS_CB_PATH_DOWN is set.=0A= --> This is the FreeBSD server telling the Linux client that the callback p= ath=0A= is down (the back channel is not bound to the new TCP connection).= =0A= Time: 137.6 (took about 0.1sec for the server to notice that the callback= =0A= path/back channel is not working).=0A= =0A= packet# 3604 Linux client does a BindConnectionToSession to bind the=0A= back channel.=0A= --> This is not permitted by RFC5661, since it is required to be done on=0A= the new TCP connection before the implicit binding of the fore=0A= channel only, already done by packet# 3530.=0A= packet# 3605 FreeBSD server violates RFC5661 and allows the binding=0A= to be done, so that CB_RECALLs can again be done.=0A= Time: 152.7=0A= =0A= - How long does this take?=0A= 152.7 - 137.5 =3D 15.2seconds=0A= =0A= >--> One more hint. Starting with #3605, things are working again.=0A= --> Things start working again because the FreeBSD server=0A= cheats and allows the BindConnectionToSession to be done.=0A= RFC5661 specifies a reply of NFS4ERR_INVAL for this.=0A= =0A= >There are actually 3 other examples of this in the pack capture.=0A= Every time multiple concurrent callbacks are attempted, the Linux=0A= client "bails out" by creating a new TCP connection.=0A= --> This is said to be fixed in Linux 5.3, but I haven't tested a newer=0A= kernel than 5.2 yet.=0A= =0A= >Btw, one of the weirdnesses is said to be fixed in Linux 5.3 and the other= =0A= >in Linux 5.7, although I have not yet upgraded my kernel and tested this.= =0A= The "do BindConnectionToSession after an implicit binding" is said to be fi= xed=0A= in Linux 5.7, however the fix is not exactly what I would have expected.=0A= --> I would have expected a BindConnectionToSession to be done right=0A= away when a new TCP connection is created.=0A= --> Linux 5.7 and newer is said to still wait (15sec?) to do the=0A= BindConnectionToSession, but fixes the bug by creating yet=0A= another new TCP connection just before doing the=0A= BindConnectionToSession RPC.=0A= --> A SEQ4_STATUS_CB_PATH_DOWN flag set in a Sequence operation=0A= reply is what triggers the BindConnectionToSession and that is = still=0A= required for 5.7 or newer, but I'll need to test to see how lon= g it takes=0A= for newer kernels?=0A= =0A= The old "cheat", which is still in the released server code (recently remov= ed=0A= by a patch in main, stable/12 and stable/13) implicitly bound both the fore= =0A= and back channels. Look for this comment in sys/fs/nfsserver/nfs_nfsdstate.= c=0A= in unpatched code...=0A= /*=0A= * If this session handles the backchannel, save the nd_xprt for this=0A= * RPC, since this is the one being used.=0A= * RFC-5661 specifies that the fore channel will be implicitly=0A= * bound by a Sequence operation. However, since some NFSv4.1 clients=0A= * erroneously assumed that the back channel would be implicitly=0A= * bound as well, do the implicit binding unless a=0A= * BindConnectiontoSession has already been done on the session.=0A= */=0A= =0A= --> This worked fine and avoided most of the above craziness, but...=0A= (A) It violated RFC5661.=0A= and=0A= (B) It broke the Linux client badly when the "nconnects" mount=0A= option (added fairly recently) was used.=0A= --> So I felt I had to get rid of it. (The non-conformance with=0A= RFC5661 was reported by redhat.)=0A= =0A= Bottom line...unless all your Linux clients are kernel version 5.3 or newer= ,=0A= avoid enabling delegations in the FreeBSD NFSv4.1/4.2 server.=0A= --> Even with a completely patched server, you will still get 15second paus= es=0A= every time the server attempts multiple concurrent callbacks.=0A= =0A= >Have fun with it, rick=0A= At least you can now see why I have "fun with it";-) rick=0A= =0A= _______________________________________________=0A= freebsd-stable@freebsd.org mailing list=0A= https://lists.freebsd.org/mailman/listinfo/freebsd-stable=0A= To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"= =0A= From owner-freebsd-stable@freebsd.org Mon May 3 00:35:35 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 44D30621895 for ; Mon, 3 May 2021 00:35:35 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi1-f174.google.com (mail-oi1-f174.google.com [209.85.167.174]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FYPCB3ySrz3ltT; Mon, 3 May 2021 00:35:34 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oi1-f174.google.com with SMTP id d21so1882576oic.11; Sun, 02 May 2021 17:35:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=s6dJPCisXPx3cUnTBwceo7Bxqs4WYlditCTQymbAiqM=; b=tjKFJv38VnPI0a273KdoATrVm9FQBW3wWdclzIxZj5YU/LDe4CwROW8iXfEZ1ZVXOe ximhqduiLplFVy6mRkKHUQLopuXc0tFKbc1wb1WKfD5CZW5Z7J990SJOtw/LVt7PwYoE 3fQ/GYZVeS/DJbEmumJdNQZ3+vZAnulOIE/UKks9FkUREo53XD1xQN49FB+G7dd2d7Lb XZe+vQG1tMDww8Vnfid9ClAp2v06EVtHU78OmC9VHgMPzYOdaT7xH2GgydoOpSrLCxpu NJ5V47zX4z/6M33HNthOuZOWDsn4Z13S7xvNMQXd03FOjxgK34d2Hfgmq71YQ9japMoL xj6A== X-Gm-Message-State: AOAM530TRe9gOD7TzLUyzFoZANk9ta4SRHJnFZbqjf7eeVp2wLBoYmih VF3pD/Mw9m7fj4duXwF0RLzPfPRnBdR5ZiCbiTQYjaqf X-Google-Smtp-Source: ABdhPJyC10VJgSwiVGha2GhmqxNf2IXgZYGLeQxUHUZcOInDaEXVIkVm6F9g9IHUIfqD3DSN575DtV4DG1ODaa2+Yko= X-Received: by 2002:a54:4518:: with SMTP id l24mr12248634oil.73.1620002133365; Sun, 02 May 2021 17:35:33 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Sun, 2 May 2021 18:35:22 -0600 Message-ID: Subject: Re: wanna solve the Linux NFSv4 client puzzle? To: Rick Macklem Cc: freebsd-stable , Peter Eriksson , Ryan Moeller , Garrett Wollman , Juraj Lutter X-Rspamd-Queue-Id: 4FYPCB3ySrz3ltT X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.167.174 as permitted sender) smtp.mailfrom=asomers@gmail.com X-Spamd-Result: default: False [-1.67 / 15.00]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; RCPT_COUNT_FIVE(0.00)[6]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.68)[-0.680]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; SUBJECT_ENDS_QUESTION(1.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[209.85.167.174:from]; R_DKIM_NA(0.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[asomers]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.995]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000]; SPAMHAUS_ZRD(0.00)[209.85.167.174:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[209.85.167.174:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.167.174:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 May 2021 00:35:35 -0000 On Sun, May 2, 2021 at 6:27 PM Rick Macklem wrote: > Rick Macklem wrote: > >Hi, > > > >I posted recently that enabling delegations should be avoided at this > time, > >especially if your FreeBSD NFS server has Linux client mounts... > > > >I thought some of you might be curious why, and I thought it would be > >more fun if you look for yourselves. > >To play the game, you need to download a packet capture: > >fetch https://people.freebsd.org/~rmacklem/twoclientdeleg.pcap > >and then load it into wireshark. > > > >192.168.1.5 - FreeBSD server with all recent patches > >192.168.1.6 - FedoraCore 30 (Linux 5.2 kernel) client > >192.168.1.13 - FreeBSD client > > > >A few hints buried in RFC5661: > >- A fore channel is used for normal client->server RPCs and a back channel > > is used for server->client callback RPCs. > >- After a new TCP is created, neither the fore nor back channels > > are bound to the connection. > >- Bindings channel(s) to a connection is done by BindConnectionToSession. > > but an implicit binding for the fore channel is created when the first > RPC > > request with a Sequence operation in it is sent on the new TCP > connection. > >- A server->client callback cannot be done until the back channel is bound > > via BindConnectionToServer. > > > >Ok, so we are ready... > >- Look at packet #s 3518->3605. > > - What is going on here? > Ok, so here's my solution... > packet #3518, 3520 and 3521 are delegation recalls (CB_RECALL) > for 3 different delegations on three different session slots. > time: 137.5 > > Expected response from the Linux client--> 3 replies to the CB_RECALLs. > What does it actually do? > --> Creates a new TCP connection using same port#. You can see it send > a FIN (packet# 3523) and a SYN (packet# 3527). > This means that the client is no longer obliged to reply to the > CB_RECALLs > and the FreeBSD server will probably need to retry them. > --> It also means that no back channel is bound to the session, so > the > server cannot do callbacks (ie. cannot retry the CB_RECALLs > yet). > > packet# 3530 is a Setattr RPC, which has a Sequence operation in it. > --> This means the fore channel is implicitly bound to the new TCP > connection, but no back channel, so the server cannot retry the > CB_RECALLs. > > You will notice a bunch of Setattr RPCs getting NFS4ERR_DELAY replies. > This tells the Linux client to "try again later". > --> It happens because the FreeBSD server cannot perform the Setattr > until the client returns a delegation. > --> That requires a CB_RECALL. > > packet# 3582 is a Setattr RPC reply. If you look in the Sequence operation > reply, you will see the flag SEQ4_STATUS_CB_PATH_DOWN is set. > --> This is the FreeBSD server telling the Linux client that the callback > path > is down (the back channel is not bound to the new TCP connection). > Time: 137.6 (took about 0.1sec for the server to notice that the callback > path/back channel is not working). > > packet# 3604 Linux client does a BindConnectionToSession to bind the > back channel. > --> This is not permitted by RFC5661, since it is required to be done on > the new TCP connection before the implicit binding of the fore > channel only, already done by packet# 3530. > packet# 3605 FreeBSD server violates RFC5661 and allows the binding > to be done, so that CB_RECALLs can again be done. > Time: 152.7 > > - How long does this take? > 152.7 - 137.5 = 15.2seconds > > >--> One more hint. Starting with #3605, things are working again. > --> Things start working again because the FreeBSD server > cheats and allows the BindConnectionToSession to be done. > RFC5661 specifies a reply of NFS4ERR_INVAL for this. > > >There are actually 3 other examples of this in the pack capture. > Every time multiple concurrent callbacks are attempted, the Linux > client "bails out" by creating a new TCP connection. > --> This is said to be fixed in Linux 5.3, but I haven't tested a newer > kernel than 5.2 yet. > > >Btw, one of the weirdnesses is said to be fixed in Linux 5.3 and the other > >in Linux 5.7, although I have not yet upgraded my kernel and tested this. > The "do BindConnectionToSession after an implicit binding" is said to be > fixed > in Linux 5.7, however the fix is not exactly what I would have expected. > --> I would have expected a BindConnectionToSession to be done right > away when a new TCP connection is created. > --> Linux 5.7 and newer is said to still wait (15sec?) to do the > BindConnectionToSession, but fixes the bug by creating yet > another new TCP connection just before doing the > BindConnectionToSession RPC. > --> A SEQ4_STATUS_CB_PATH_DOWN flag set in a Sequence operation > reply is what triggers the BindConnectionToSession and that is > still > required for 5.7 or newer, but I'll need to test to see how > long it takes > for newer kernels? > > The old "cheat", which is still in the released server code (recently > removed > by a patch in main, stable/12 and stable/13) implicitly bound both the fore > and back channels. Look for this comment in > sys/fs/nfsserver/nfs_nfsdstate.c > in unpatched code... > /* > * If this session handles the backchannel, save the nd_xprt for > this > * RPC, since this is the one being used. > * RFC-5661 specifies that the fore channel will be implicitly > * bound by a Sequence operation. However, since some NFSv4.1 > clients > * erroneously assumed that the back channel would be implicitly > * bound as well, do the implicit binding unless a > * BindConnectiontoSession has already been done on the session. > */ > > --> This worked fine and avoided most of the above craziness, but... > (A) It violated RFC5661. > and > (B) It broke the Linux client badly when the "nconnects" mount > option (added fairly recently) was used. > --> So I felt I had to get rid of it. (The non-conformance with > RFC5661 was reported by redhat.) > > Bottom line...unless all your Linux clients are kernel version 5.3 or > newer, > avoid enabling delegations in the FreeBSD NFSv4.1/4.2 server. > --> Even with a completely patched server, you will still get 15second > pauses > every time the server attempts multiple concurrent callbacks. > > >Have fun with it, rick > At least you can now see why I have "fun with it";-) rick > Ughh. I'm glad you figured it out so I didn't have to. Thanks for all the hard work, Rick. From owner-freebsd-stable@freebsd.org Mon May 3 14:47:55 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 692D56342EE; Mon, 3 May 2021 14:47:55 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-il1-f174.google.com (mail-il1-f174.google.com [209.85.166.174]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FYm6f2NFtz4skf; Mon, 3 May 2021 14:47:53 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-il1-f174.google.com with SMTP id i22so3844652ila.11; Mon, 03 May 2021 07:47:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=14wmHnw9J91Z6E+mgjOMzXrqMVyYtz6PQ9uIY2PBlKI=; b=ijuYfjMH7woVc/rzTmUrEj9PNWeHTJ4LikYgYYrJoIel8j+CLwm+AlRdVFXngmnmVZ GYAGAOF4a3+I6J/IFZVydAJJwKTUc6bW3OxfrKX62g2HgwAalurZt54YVtg04bqrfsYa itu2+qeEpzLOXFmrvbAmEG5pgplSTV7H5PdoOcOy4f8qLu49esfHbi2ErNgOQp5u/OCa YzxFiKxr3/2mJyzUu3yqtim6NZzqNJJnwPz0QKFlN3lIJdRZvaJHJewHzK+sr8CFsYhP 6LtEX5ZXfCNvZxNVIXFbLRSUKD69gH3rbj05A4AE/JgL7GU1T3rR/fkJHSLnRi2ErFih ea7g== X-Gm-Message-State: AOAM532w4D1pNUm+sYpeUdwPG+pNY87z2iDk2IQVnh7acBDb/DRGfD1m LpG60UvOVa6HjpdGJoIQ6t8M9FGZAPLqSODeIiJtwcR2 X-Google-Smtp-Source: ABdhPJyjGQq/Lr7BwLPwH4SrbYHxlOoAXWiCjcofYvvJ2TKvCQmDRXp6NyOotvSlsHxFklTwAcBA3zy0eAwc+TDbjJk= X-Received: by 2002:a05:6e02:92c:: with SMTP id o12mr16581939ilt.256.1620053273039; Mon, 03 May 2021 07:47:53 -0700 (PDT) MIME-Version: 1.0 References: <35482701-95A3-48B2-9A8E-B7E0092119B1.ref@yahoo.com> <35482701-95A3-48B2-9A8E-B7E0092119B1@yahoo.com> In-Reply-To: <35482701-95A3-48B2-9A8E-B7E0092119B1@yahoo.com> From: Ed Maste Date: Mon, 3 May 2021 10:47:02 -0400 Message-ID: Subject: Re: FYI: WITH_REPRODUCIBLE_BUILD= problem for some files? To: Mark Millard Cc: FreeBSD-STABLE Mailing List , freebsd-current Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4FYm6f2NFtz4skf X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of carpeddiem@gmail.com designates 209.85.166.174 as permitted sender) smtp.mailfrom=carpeddiem@gmail.com X-Spamd-Result: default: False [-1.94 / 15.00]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.94)[-0.939]; FREEMAIL_TO(0.00)[yahoo.com]; FORGED_SENDER(0.30)[emaste@freebsd.org,carpeddiem@gmail.com]; MIME_TRACE(0.00)[0:+]; SUBJECT_ENDS_QUESTION(1.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[209.85.166.174:from]; R_DKIM_NA(0.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[carpeddiem]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-1.00)[-0.996]; FROM_NEQ_ENVFROM(0.00)[emaste@freebsd.org,carpeddiem@gmail.com]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000]; SPAMHAUS_ZRD(0.00)[209.85.166.174:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[209.85.166.174:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.166.174:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable,freebsd-current] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 May 2021 14:47:55 -0000 On Thu, 29 Apr 2021 at 02:50, Mark Millard via freebsd-current wrote: > > Files /usr/obj/DESTDIRs/13_0R-CA7-chroot/sbin/ping and /usr/obj/DESTDIRs/13_0R-CA7-poud/sbin/ping differ > Files /usr/obj/DESTDIRs/13_0R-CA7-chroot/sbin/ping6 and /usr/obj/DESTDIRs/13_0R-CA7-poud/sbin/ping6 differ > Files /usr/obj/DESTDIRs/13_0R-CA7-chroot/usr/bin/ntpq and /usr/obj/DESTDIRs/13_0R-CA7-poud/usr/bin/ntpq differ... This is unexpected. Unfortunately I haven't looked at reproducibility in a while, and my work was all on x86. This could be a regression or a longstanding issue with arm64. If you install the diffoscope package (py37-diffoscope) and run it on the two directories / files it should give a more convenient view of the differences. (Or, if you can make a tarball of the differing files I can take a look.) From owner-freebsd-stable@freebsd.org Mon May 3 17:52:00 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0A6726387B7 for ; Mon, 3 May 2021 17:52:00 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic309-21.consmr.mail.gq1.yahoo.com (sonic309-21.consmr.mail.gq1.yahoo.com [98.137.65.147]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FYrC3000qz3J9Y for ; Mon, 3 May 2021 17:51:58 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1620064317; bh=1dgeJS5JPt2OZJx8GDASMil/yPADC1mbzFQ1/NviMwu=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=HJUTCV2AmjTfqwQFNuS2McEK3uiNHV8a6ffQmWn/U1qx1CP3jUyh8+vF0ZyfjAuiqzyenZyNfjFRrTuF684KaOAjrhgd/HQ3lro1tuWhEJnrNy1Jw1xwHiXiIhmk1EsbzoTH/9VNke1dOf/V2TKG6FPNq9QeLaDoC1pQ5q+lX8FOMgDjIrqDV6jnLk4GJzcik2ku6FmqruvFnhbx/Wrd8EIk88Ex13LqvEpCZG9aogqO4qW/BkRyp05EY5eCjWArxxERodbhz+n4z40UDIJZU7kku04yEfCenowzF3uBlbQhpb3VQP7BVkcsUJHYSkC23REDFUIc7an6uJQiFXFETg== X-YMail-OSG: r8XUusAVM1ksaPHQMjYGeoFCyI3C.5F0VYB7V2FHLNQwlGU73IPIjiKe6tSUA6Z yYUe0XoLw5y_8qIiKD_9WOq1Whn0Uzl3grizre_D8f7WyrL20w2qbpL3XirdoTyuVnxdnlckaNyQ 0oGLv8P8SQWnPM2UsodUuULrPyNiQx0q1GoNgCjAzl7TPCrSe_xSOXCZ6_Umenao3BPXVss1Ecq5 qw1oqiN_P7b1vPACVbH23vGr0v_GCKBGvp42YNiwYL1.GKMg1wi.Qs3_4.dsEBu3nnhlmGo.UM6a HTHrCSjxqMcMaOEI5KgtirWHYypYRfAfxAodoe1tJQg4NITCu2wVxgpkKr7TsmHGF7saewU4zi4y h3a5KkTiFw7OFXLXf_WELODpzR6aOPpqhfzwlfW6sQFrK_nxFHN56DhHG47Laa17viHnlCRMmuvd arWXTP_ZKM7EelzQOL758pY76SsPGgv9YaDSwCyFvX4BiT_ALCU00HFAhk4XOn5eoKPrkdQ.nvsu 7f7jswPCw6negCPg5qg9uYOa7dcSZrZuiYkjA5PwT3tp19t7S0.wSBF8WEhIUgtCtuvG72ZGgLpq _SW7y66IW5fSCqjxXBUjqvljYpdPBeG.udHh1dg0JvVVvb9MxYmuRpJfB4NGcmWDSfsW9uMUFqnJ E0KpBsnNo5fPQyS5v.00GANg3JMQhyMtLPJHGWgIfkFs70dLkeL7FL_ToLwtsDcSMos_OwClkB.o fYyhOlS77v7eqk3Qo3kGY.mjA9fwB_KFcQGHKHzvzLLaliF3xtkqRIaf831JWO.lCI3qZbGS2Bxc wNXwvshK1RkmhALkhCG8rsVK2iZVZVjM3bI6rqcOrv8rddHQUTAfeE88EY0LSphrN2j_sOr9MPdY f8I54TddSq7g6Huk1C2mECWAvMhZdKCj3PUBi0FtCMl7GSSaGAnkNnkQ47nmakBPvEh9k55PfyTA Q8pVkytardVU.WyQjwRDBrhnWobXkACgF2gECcjg7YzlkyS5bSL3FEiibc9ZVh4e3lJBSheXBB0Q 8KD0zDN0RCnYJdkn3vgyvcWxg0QlEipSXt2fSUPOEVEInArsH8FDsarK0kCsbU_RmrnTi8LhaLYv FR176runNKAVUSjL_gALzw8eKxs3irZOv3ugjk4bHyCxoAkr9tcOsoEvZAGza.KyH8CPd8LlUm55 sf41wOq5Bs17FbQ49ga5Ss.05KpJPEyJ.MThT.1HGBhPIeYzTyGN.RimYz1VJRCrP0i6a7oeKRPg s3r2FzfkCrdZhLy7ADCMwBpo_Y7ESu_ZuuMDj9gHfCH8LNd5AlAX8pt01FOwoILRjbCzkVekaV1e o53x9Y4e3qbpaeql9Na3NSHzbaaJ5eY7Ek5a6biAk3I7kgYF2e8vrnG7iDKp9eYHjvSxRB8vjInH 3HqUup6TtXTHlFgm3i3dr5N2aKwcJfPu_C29HjKEfCJ2Fq9iaPlaDlezdZAsriVJPMZSM1TfqIDO 1SNjWr7tDXvAm9InrXaZ_.VuI.vhC9XQGeTj9r03d2cJg0Owu6Z.wT.GDM9VExGGKNiXMoMH7BPg xE16oYejIyhWKX2q6asRNnxd3y87x4ppUltNk_Ejd5qm5t9q4MxvFZ7eO78eEq_NrzUvcNQRLSDY DwRBsTE8i9ttQanwvy20Z.Sg_BCdOdZPWuThYu3SonCHgjFYwZ_FdBS0wcmEFZpGt8ZQk3LlR7AJ HX0MqhFpp7hK36LT3LwkxcR_5G8cLeHjgyZ_IN4Y1YVkichP5Jz7mnx1VFr7sBGe9QWV0aPauFU7 d1oRln3b.Az0kwnxFTOQxWr5qpKpowEAWcL7F0V8mvwC3C4dz100jIH1lHWdYfz9M56dR9frgxgE g7L6UJVSj7nU._hhOO7MniUK0NHurTHuIdTs7dINgoFzShE3IcgL7cYyKAPye62hQT56_cDwMu4v fXPCFXz5Z.0VCzpmaCyT_CdVkZtv8dlTjpRV.9k66HIYwnveStzFapMfCLsF66M_4xWyujtfu8DN hnrxhijlcTQJ1GtNPFfffdbB_FYrFpWSMRPNF.I_e2cRXyMdY3nEXJ8E8pnz2Rxf0J4ZtzVAoZiH WsNkAO7wfu8iZlduuFiQ2fvxOFs49VxdQwvbBLnBc_ej_mnohzyqbChU5mqhXIrQxl4.n.RcX4lT X44L7fk_rifKB2xQ7r0Hi58qH0Z56Npqdht9keQNTqcTtvjZeEnFz.hsb1TB5puH3i.mNgVxo1ld a5O107I9YB0MTtpztmlx47HpQ1curbgyc50_d7ewDhfORQndZQJTFQHKxldDjOmo2uNjdoP2K9f5 IwuEU4IK7HazH1g1TQQTjkjZ_qdsyBSFe7I5NAfjejdBgL5zhq6awEAGjiESl0ZwOfTq8Lo_RluU SNUEpOGBmDGkr4CU02CRnNsTZ0KYuWRCPDcVEQkc0LJtbICzkDP1aEdFs0LG.nOHuSHFx.t4OGKn UQqmVnRi17QmTJKHss1iiC4FVms59gx5_xVw3WZ1iVS13dluCI.m.Nv5wYdPCDM.Q6Uzo2eTclWZ D4gUcfWCbQt8COQa9zapjBDrCfzyNqg1isGvbFS0sfEdTGyjAArsm66cxrKSOE52bCMA.yOEO_wr 8bqiSG3sDPeUIhBrrWZ.ZmioJ5U67bk8JgTZq7ksXVBHJF50.mpPBmxdxIf57 X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic309.consmr.mail.gq1.yahoo.com with HTTP; Mon, 3 May 2021 17:51:57 +0000 Received: by kubenode573.mail-prod1.omega.gq1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 8b311bf104a2bbdf9f2ada34ac0923f1; Mon, 03 May 2021 17:51:56 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: Re: FYI: WITH_REPRODUCIBLE_BUILD= problem for some files? From: Mark Millard In-Reply-To: Date: Mon, 3 May 2021 10:51:55 -0700 Cc: FreeBSD-STABLE Mailing List , freebsd-current Content-Transfer-Encoding: quoted-printable Message-Id: <43F20589-A7C7-42FF-9020-09CEE037D1CD@yahoo.com> References: <35482701-95A3-48B2-9A8E-B7E0092119B1.ref@yahoo.com> <35482701-95A3-48B2-9A8E-B7E0092119B1@yahoo.com> To: Ed Maste X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Rspamd-Queue-Id: 4FYrC3000qz3J9Y X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.47 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.97)[-0.971]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; SUBJECT_ENDS_QUESTION(1.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.65.147:from]; SPAMHAUS_ZRD(0.00)[98.137.65.147:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.147:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.147:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 May 2021 17:52:00 -0000 On 2021-May-3, at 07:47, Ed Maste wrote: > On Thu, 29 Apr 2021 at 02:50, Mark Millard via freebsd-current > wrote: >>=20 >> Files /usr/obj/DESTDIRs/13_0R-CA7-chroot/sbin/ping and = /usr/obj/DESTDIRs/13_0R-CA7-poud/sbin/ping differ >> Files /usr/obj/DESTDIRs/13_0R-CA7-chroot/sbin/ping6 and = /usr/obj/DESTDIRs/13_0R-CA7-poud/sbin/ping6 differ >> Files /usr/obj/DESTDIRs/13_0R-CA7-chroot/usr/bin/ntpq and = /usr/obj/DESTDIRs/13_0R-CA7-poud/usr/bin/ntpq differ... >=20 > This is unexpected. Unfortunately I haven't looked at reproducibility > in a while, and my work was all on x86. This could be a regression or > a longstanding issue with arm64. >=20 > If you install the diffoscope package (py37-diffoscope) and run it on > the two directories / files it should give a more convenient view of > the differences. (Or, if you can make a tarball of the differing files > I can take a look.) I no longer have the same content in those directory trees: newer rebuild and the same buildworld used to installworld to both places, instead of 2 different buildworld's. I'm also unsure how reproducible getting differences was. I can eventually do experiments to test multiple separate buildworld's and installworld's, but the machine is busy building ports and the llvm builds involved means it will be some time before I'd switch activities. And the buildworld's involve llvm builds as well and take notable time themselves. So my next comparison will not be any time soon. I'll let you know if I manage to generate another example, this time being sure to keep the data. If I try multiple times without finding any differences, I'll eventually decide "enough is enough" and let you know. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-stable@freebsd.org Mon May 3 20:46:46 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A5EAE63D302 for ; Mon, 3 May 2021 20:46:46 +0000 (UTC) (envelope-from juraj@lutter.sk) Received: from ns2.wilbury.net (ns2.wilbury.net [IPv6:2a01:b200:0:1:f816:3eff:fecd:13e6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "svc.wilbury.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FYw4h68Yfz3jTh for ; Mon, 3 May 2021 20:46:44 +0000 (UTC) (envelope-from juraj@lutter.sk) Received: from smtpclient.apple (gw-upc.owhome.net [188.167.168.254]) (Authenticated sender: juraj@lutter.sk) by svc.wilbury.net (Postfix) with ESMTPSA id 4146145CF21 for ; Mon, 3 May 2021 22:46:35 +0200 (CEST) From: Juraj Lutter Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.80.0.2.43\)) Subject: bhyve and multiple network devices Message-Id: <72FC2AD5-7E59-4C3D-8E4A-19AAC6608304@lutter.sk> Date: Mon, 3 May 2021 22:46:34 +0200 To: freebsd-stable stable X-Mailer: Apple Mail (2.3654.80.0.2.43) X-Rspamd-Queue-Id: 4FYw4h68Yfz3jTh X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of juraj@lutter.sk designates 2a01:b200:0:1:f816:3eff:fecd:13e6 as permitted sender) smtp.mailfrom=juraj@lutter.sk X-Spamd-Result: default: False [-2.80 / 15.00]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a01:b200:0:1:f816:3eff:fecd:13e6:from]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2a01:b200:0:1:f816:3eff:fecd:13e6:from:127.0.2.255]; DMARC_NA(0.00)[lutter.sk]; TO_DN_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; R_SPF_ALLOW(-0.20)[+ip6:2a01:b200:0:1:f816:3eff:fecd:13e6]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:44185, ipnet:2a01:b200::/32, country:SK]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 May 2021 20:46:46 -0000 Hi, my bhyve command line (on 13.0-RELEASE) is: /usr/sbin/bhyve -c 2 -m 4G -H -A -P -u -s 0:0,hostbridge -s 1:0,lpc -s = 2:0,virtio-net,tap100 -s 3:0,virtio-net,priv0 -s = 4:0,virtio-blk,/dev/zvol/zroot/data/minio/host0/root -l = com1,/dev/nmdm100B mr the result is: device emulation initialization error: File exists It works when using original interface name (tap105 in this case). priv0 is renamed tap: priv0: flags=3D8902 metric 0 mtu = 1500 options=3D80000 ether 58:9c:fc:10:2b:0b groups: tap media: Ethernet autoselect status: no carrier nd6 options=3D29 Am I doing something inadequate, is this an expected behavior or is this = a kind of a bug? Thanks. otis =E2=80=94 Juraj Lutter XMPP: juraj (at) lutter.sk GSM: +421907986576 From owner-freebsd-stable@freebsd.org Tue May 4 02:26:13 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6B6816216AF for ; Tue, 4 May 2021 02:26:13 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic312-25.consmr.mail.gq1.yahoo.com (sonic312-25.consmr.mail.gq1.yahoo.com [98.137.69.206]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FZ3cN45VCz4V6P for ; Tue, 4 May 2021 02:26:12 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1620095170; bh=UR3wi6xHhFv9EGqIW8qVhLasqn9Pa8PMArOOi8VXfrz=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=BG1EOffVGlY9Aoub1xWMz+VbR3JuohuDnaEcr932xgNH/8hnAQUxavelwMw9lX31I4Cqk7kDIMJ8gBAy++kUgN0ABwfGXF7zOJQbGkgFErTZJB+xydHDVXX73BfxQjYL8/H1wKiWxBd18N6SVyhxrwNe4dcrD3M+DvUmLw6eKtXY24PlbAvEwNnL/mZphIOgYbxpmjA1yd4niJqTSLkO2edYjkrcxcDKDXPxom+Ggj6XZv18df0QdntSt3d5gjZCcbYKEBI8Pr5JAE4jYGnhwdbplNDPCWZIh0IaebcgbpXkHPXG4oZSwzzaX1RvF2+dgSiVu2BGK7J/F9c+G5lENw== X-YMail-OSG: GBguPP4VM1mPxEdFzDXJ0_R5J9qZIVIvskx80xOQVjR91PVHUbKVHsv7q0ubuqu BhJeUE8WohYJh.o3fKgsjmTBKEa9..alIb4Diij.7eYcExOPK83E6YcT29o9dBxaQVO_qRNqI9zD ngya22e5yZBIuHUCU36uTt2IVSVo6fSuPzVLIfmmqk80IJjsdVe3GuQwh2kW2P9YT9XA0MS.2r0N SSBhB9mE9h51uqoqcQUPk3yMgpLItuR2aAZM6S2QmTCMOFVYpE33n0yPaMUT7iogA7WuCFVct2wP 6v1I_BgUJnSE9L844OfKNzL9.s768kMMbHfaAGej7QFzYoIrfju6Cgai0IUQ.YVKMSC4ro6hxBSF bi4x1T37IZpb_59lsMCz0R.5JlnvbjZ8P4P0fO20eYUctQJkIeCWsz3OrlgnpZpS28dlvQrARDah 3NzVG4sgmfig7dSNcaHQ4T7uH4cp_xU5I.MGvcCOkwSNCmimNrb2VLvylazjHiGLg3QpRvrys3Tl EEh_RzuyiEHPWbdIoJTr5gJ9mdhp7G3NHkrPTS4LhPP1b_f1BqWSYOsG66R2ZBKprXBCZDouhmPd Ac9pUsgK9JruvwFduxkpUydFLxUqKg7BCy2Oa2aUZ7btloWelPSaOhGW_65bas76GJaPpHmjUlpR VlYmNCbXhL_Aig.DywUVWq5h8fGV0s8xmax85pwg6hPGOxqsUQcSXpZg9neN2kswYEwZuEwtujCp _yzHR7wVhr5KZCYM1lyRXR6._IAsKhqsG3kMq9pzIm10niQ_7_mByRlfRjtNw81NlU7abWt9kAp3 DrsZV3D8aHzHsmIWp4m48LBWBRNIAjho8K2P0Vzs7LlVgMDi5SNxeye1Dx3RsrUSW6LoBN270q0w UvR.vryfWhXLwdfhQ_J.RnB1wpC12YnCfMo3LIf.L_3WiNqHT4T7E0thz0Pq3uTu4.oLXu_wi6vD .RNPaj6y_ObJH4ZoL0C44m2U5FxhfGamlJmOrsZ7mhie7fbrJ2wttvIL.V4bzNrOh1D2AV1fTcB_ f9TLAE4DJ.QNNyWSQ1AuUn5El2ZeP07MZhnWL76zVm6flJFMtCbpbDuaFNHPk5Q0i9vz3SxXakM. TuvA5QYSo0hGWHL738niQFJsdJExkiULAackW_y5FNSeDayCqy9C3lZYiiUO40p3WCm7oYs_Bt.D YPwkrOnXxo9byQXW3D4fB87plCC7QyhFfrjhRT7iOEgvrpEdOhK37IxwnY69fLDspjQgOo7Ed7m1 JLj6qQ5SEw8wox7Bugw5hqdzl6fqJMtRVWwVywiscQVneqTMA0qbXgq.PaDDb2eiAIx0rZz9f5OX 71aGvbU8y.CbaCXUCHAQpTMZdusEgsFtZ2.uT3cnRrQGqzFdJRcZysp3rdC6odLTqnuveUHNauBE JlNkriJWkOBx0oOFFE9OzvkjjA9LVpmGGWmgpWQjp66KY29m7vLXyZTVTH0rARSNzj8zv_kufIGy 0cUGANIDNhqFKq25l4_NmuNEsxXquH5gs6rxV0PCqTK02LghT5y3aEzK5dGMFYSRjWC02RJ_JacU GCXbNrVWLZPm8CrwetExaG9qraLb6MosTY3S8Z8jjOsM2PsnPQfDSMBk.9LR4HrT5Dy5H8uc8cCv LalMh9XdIConO0fygF3LxOKIsj7TGQKQch0BIaKavfizIuA7_ZiyZkW0j1p10LyYhVJ4NZj5MOQP 5ux3LdmOTBDxO05VnJQ0866fAkSehSMglLsRvkSqvmbOgC4BwXJGdW8m827XvDBbcikyx1AZ8iQN BuVxDaxsJenBsrpQSCycAWi.vaKgvmJu5TevZZGAHPzLwI93Ewxkdk8d.io8Y2Fs09IGkebwDopy 3ilb7jtRCfVMXXMY57DYGrG2Z1LUSqvb.VxuBk_8KdHpk9tkXftn4t._X7WIBbBtR51lAgY4Jk1C PnYK0dc67XfvJUIjM39gJYG6sZcArf4SUxaKduDJ5tTlgbdQqIu7XLThsbK._9CK3nCizuwVqqzs i6cooCUHm2frxEin_A2vYNa9qTMoiFjo2u2vhWdj1CXljRxdoFvNkEu4HD.bj8qUPZEwYU_PiQ7Q iywiUF6GfvRWNtyOgL_XFN4.YJu3iyShLciLzxib5vRxmW5veroWOiPq4YmpCMIsCEJPDML1snW4 _Nli1gtoYbeK5XW3zxG2OzyMxPYqTmoSwh6H7G9qqluTqZ5aBcSHn4nv25NsXm1h62ydbBcwkwFs yBMBPAEgIhqeT8lI8JhzFJZAdLDOar2TlBdI7Sv.ZtJgEJefLRxXMWotNq03X.UhdMQLz603NhKe GnpEy_qg.xBsy1DSpd37ylOQBjw3VUA41U.zlOHdYdM_t8BYz.r_lMUs.mPeQ_m9xYSpREg65YOu dnkXwqUuF7Nro3lUd_kZMxxMUgeUuxJYDBj9OKqTkqY8jPPzTx11uJeHRUi8B12YPB0TRDQ9Xs06 S_xC_bn3GPUWD6PCBDGd4G5lSAQWyMTM2NBBoFQs0Pp9uB4JyhkPTrS44UtU4rYDJRY0F5ZqXZUa Cj3nTUA89ZAC14VwHA0RxLQxbBCId6sFgTDCuxz7J0ouKB73HMHFnRQ8dtW65RTUKBPLbvz7G125 WeASZSTuDa96OklulBRxSu1e9dYTwJlYl5.DsyzWnhBSsgWkzxVyJe_6CYKR0ur02oGPyWX_zUCX VwOoFuO5vMOOfJpRhqraRdZT_sOE1KCjf0kpbAc9T X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic312.consmr.mail.gq1.yahoo.com with HTTP; Tue, 4 May 2021 02:26:10 +0000 Received: by kubenode507.mail-prod1.omega.ne1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 1427c9896a661f47101f2c9304e55cd3; Tue, 04 May 2021 02:26:06 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: Re: FYI: WITH_REPRODUCIBLE_BUILD= problem for some files? From: Mark Millard In-Reply-To: <43F20589-A7C7-42FF-9020-09CEE037D1CD@yahoo.com> Date: Mon, 3 May 2021 19:26:04 -0700 Cc: FreeBSD-STABLE Mailing List , freebsd-current Content-Transfer-Encoding: quoted-printable Message-Id: <91F820A1-8940-4246-A20A-E62685F50079@yahoo.com> References: <35482701-95A3-48B2-9A8E-B7E0092119B1.ref@yahoo.com> <35482701-95A3-48B2-9A8E-B7E0092119B1@yahoo.com> <43F20589-A7C7-42FF-9020-09CEE037D1CD@yahoo.com> To: Ed Maste X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Rspamd-Queue-Id: 4FZ3cN45VCz4V6P X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.49 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.99)[-0.994]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.69.206:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; SUBJECT_ENDS_QUESTION(1.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[98.137.69.206:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.206:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.206:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 May 2021 02:26:13 -0000 On 2021-May-3, at 10:51, Mark Millard wrote: > On 2021-May-3, at 07:47, Ed Maste wrote: >=20 >> On Thu, 29 Apr 2021 at 02:50, Mark Millard via freebsd-current >> wrote: >>>=20 >>> Files /usr/obj/DESTDIRs/13_0R-CA7-chroot/sbin/ping and = /usr/obj/DESTDIRs/13_0R-CA7-poud/sbin/ping differ >>> Files /usr/obj/DESTDIRs/13_0R-CA7-chroot/sbin/ping6 and = /usr/obj/DESTDIRs/13_0R-CA7-poud/sbin/ping6 differ >>> Files /usr/obj/DESTDIRs/13_0R-CA7-chroot/usr/bin/ntpq and = /usr/obj/DESTDIRs/13_0R-CA7-poud/usr/bin/ntpq differ... >>=20 >> This is unexpected. Unfortunately I haven't looked at reproducibility >> in a while, and my work was all on x86. This could be a regression or >> a longstanding issue with arm64. >>=20 >> If you install the diffoscope package (py37-diffoscope) and run it on >> the two directories / files it should give a more convenient view of >> the differences. (Or, if you can make a tarball of the differing = files >> I can take a look.) >=20 > I no longer have the same content in those directory > trees: newer rebuild and the same buildworld used to > installworld to both places, instead of 2 different > buildworld's. I'm also unsure how reproducible getting > differences was. >=20 > I can eventually do experiments to test multiple separate > buildworld's and installworld's, but the machine is busy > building ports and the llvm builds involved means it > will be some time before I'd switch activities. And the > buildworld's involve llvm builds as well and take notable > time themselves. So my next comparison will not be any > time soon. >=20 > I'll let you know if I manage to generate another example, > this time being sure to keep the data. If I try multiple > times without finding any differences, I'll eventually > decide "enough is enough" and let you know. I've still got a long ways to go to do the first actual comparison of builds. But I'll note that I've built and stalled py37-diffoscope (new to me). A basic quick test showed that it reports: W: diffoscope.main: Fuzzy-matching is currently disabled as the "tlsh" = module is unavailable. As I'm not familiar with the tool, you might need to send notes about how you want me to use the tool to get the output that you would want. (And, so, I get to learn . . .) =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-stable@freebsd.org Tue May 4 04:27:17 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6617E625716 for ; Tue, 4 May 2021 04:27:17 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic309-22.consmr.mail.gq1.yahoo.com (sonic309-22.consmr.mail.gq1.yahoo.com [98.137.65.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FZ6J42tryz4b0T for ; Tue, 4 May 2021 04:27:16 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1620102434; bh=5vn3CYtZvFsTpF5Obu35isw8a6/iG1ZCO6qVLijwEqU=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=paJQRbzYGTyzQCn10x0pQvjOPAT2lRQ85Iflx1Rr25diXcHcKMbpB36KLT4yi103ozz8tIRXbfuSgO7g11DkDogGQpAYh/4wNiUBfZuKZFBPRNXGycqbKo2/VulwlkcyzaZZbWAdgSwuFMK48EgWtcl1t3ra+u0NmjV5f1EQp9Qts44BvxxLfQOSMLubzDi2AmND5Xq5G56HowMzpTp2P1rhsHUFotI23RpiI4O7oU01bNGO2E4QPlme2XF/2+l2Itni7lZ/WljxrqaCk7T3L+DNfuzHN8K2wDw2rw7JRj54IoFSzL4mzJyAHTthPwCHNEWoTzGU9s0F72POxs9H0A== X-YMail-OSG: 4CVIMHkVM1m1Ebnvf8orEQVhXRBTik6nAZl6w9qKXUypiBEuDTXRwDOqnlpxlzB F88wS67sqkcBwP8LDjcoRvy1syyg0A821jOuzQ9zBf1115BNbJSNj2VV5GiOKPX8oHf7_uc4MB.f gWSVgF.BYNW55k4GQTJeuveFX7pIkoUTfVBFsIALf4qZch2EhHF62ZslTsLUXTM8.OKEvFWSqauX tEvKso1W6nf3iiKWPgJ5chqmc2AJN1OX.gULbwNcJjrthCI7pH7VWW8UH_BiwFCFpgluhHiQTQGl jI2OylLV26hh1SRjG09Se49IJdAMtJbHKif3Nm_DvTBxYoMDXGTvX_PECyw6PGzlxRAgtCjA.uLi e2.XhAvzSbi4JnBn1RrFf3_XtTVdMZ9aaVqdOQtRaeXQEcO2FtLdEBnoQ3syWH3wqBo9W9wNT5tk 4.MNzQTRtZBrLIs8dr0Mv.zIPNY3AH200EbZQJfk2bHTaKdKWqKfFo.wnnTuEzGiO_C8m2FxWu60 eT_DPszAPmy1.n_ZFCAnVW_elasMYuZh.bapmioONjWKaPvqJEPKH2r_xJSBnVQ6pOdbriAILCCE scuuTjw_SCz7SVaG64kfqXe245g.Eg1YPme0pw0OUG4n5jX4nFcDH8GKt1p.kPjRaeJyDVx.9x.F KVphon6hVk._lyk2hXW5jo3KHBcD08kCT3fQKyBEH3xQpvvPNdq5c40GNmJzuYZ8E5UBZWBQljK5 MnCctirF7PKq7MadurQOQUL5FuVgLv5iZMpkonlN.oKT_Doux0umADJDwSaVu321vp.W1dcIZYMC AOaKo89BfExl0I2SKNZdzw7yakX4RdJCj.sThfgru0Zu6QCZRAvLbyAq4SVb4tlPWa5gI35OfpER PobYHUxt4T23YMs4p6Bpkv6A7pSDjK_1T438nK5QCo86nRL8zm7B0_kiLcfEs_C3ye8kmD6_4Y75 MZxj5x8Dzqmi5QosYM4gHvjdHK2MIdKzkyAuAyu78Zu.zL.6h.QcliQJXl8qQ7YPyiaNsyZskzxO .uoPGENP3CQ64pWuEfASfYGqDgHs6y0N.VH0XCUcTA1f21Cu.GaAnfnfkOlpaj9G8WgnDCXE59Uw 5kjJq8DvErNihFxU.1E7qTiPTfgHOHQoQne4SM09Q9gONIXxeTUeu6ZXgFALgKI4uqSU.FlbcP8K VbO7p.U8wjw39V76Fzdo_y10xf3nP7Kdrhg1vPCNPcHrzjqWrWvxD4z6Gbf1LdeCWxg6bJT.ZQam ovi2ANbn5mPek9aIJMjkae6ysjRJdENdNYamFVVZJBMF1TNVQK9Keh3krpJZBMEcAQfOnRVEqv4V 4mLTBatFVDxWaCMFEQn2f7bD9or_yLmBwaORk7r4_pVvEPPCE9EhMXfDYDgT66yvhcY2R12Cl7Wn ud0huD1_BMuJYC5RGfQhx_J1DIahCxD9QPC4bzhaYj5VnbgR5SlrZ4.spI2WDV96eWHF.mN7orw9 pMf27qlg6pzU7lCvxRKCAMtUNDetEjhIq2UBQHY4fXDC_Nqf14mOb2k5M.P21obVd0BQh7DOW8vv e8V2wCTxe26JYoTmbSvx4KdRNhCOBKPHPi_vGxGkJvM9lYWx7Whj56u0FUombeIJPQ8GnsOdfxNY gqej6lHzOLFk1u9fShw0kSXVar6A80fk3hJgnG6IDFgesG6UMDo5Wkw_Gk.ajnEj3tv_CIvtXEff JpHVx.5LbxdA2Duj_0wWHlFOrax8MwY4aj1m9TCQAKsyshavtm3k_hGoOitaJO2dBV2iLYqhdK2T 0pu.U0HwygFk.zYjhUPA6x5DrpBlB0EJUEbtlEeFtLmLCmhE3aS_CZyw9jFQDDfJUZ4Jh1Wtpppq a7cpM0Id3IyI3ezw.2_mDElcWBkbV.X7buL3DuvwUP7EZTkRJIzKTGSsq72NS4x59yg.X2zloJlx ua8I.s8cO9QmVR2U.Pr1G1XW0s5dpelsC.fZuJPSv2AKvbb2RF.fnv7iqoSF9qqZ8s4i4u4mw_JJ Qb6UBtU1Vuflm0p7AELMU9oYYuy_gD2mU2gKorZrK07aQwm4IN44bgokPEbVQ48jPYDxTmXlGYY4 yW8CGTSTyruGXzOZ_eSrEOU8gdzGLGKZr2MAdVCd_3i72GW2r.mswQii_MFsYRif3BliohJhB0qr UyX1h7vfglft2yslNWM97zvPj.1QnQ5TZJReqnqqPGu1YjVVLFfi7H4to3YydOvxm9VKhOPMelT6 xihUvRH0Cmp2mgS6mBsAedMW1RhlY08e5ZAguBMSSh2nEQ9DdmQQOt0W.QgVdKpPnAnBlao.lrZJ XgdYed5jO0KZi8mMaAPG1aee56k.26_SPHGr0thL3FdrayHx7VUxrYLZfa48t0VhOV.IWjUNpQei HiCPuoN8gWDYIcxtEu..1wA2LsQw0mjTommJxFk9QOwW8D_62uoSLHi__zxV6V3lXQ3Fo16GW2tS gW9Pn5rmAA5FEmf8Gm6d2QUEAQspyLZ1A_dAmSYMWRlE48sgpIbtTKBm6Amsy_MuLqTfFMMMU_Wo JGrCI0kvy8y5vfRRizJ39beWgUeospZ3BFzpkti4uqYgCO5maIw2WAPhPL.c46mDu8WXNK4UaNiv c.LI9W9KgtZiLt8URRsjVV0KMUnBJ11or7Exa9jZMMClBrX_qKrvyY4DTlMefdpC3bCRQVrAZhHO HCNMK2C9hD_EUsjCkN.8Hi0JvBrQybBeFtBrsoK6dQA-- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic309.consmr.mail.gq1.yahoo.com with HTTP; Tue, 4 May 2021 04:27:14 +0000 Received: by kubenode508.mail-prod1.omega.ne1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID ab1cae79be7857d625e2d1658bf2680a; Tue, 04 May 2021 04:27:09 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: Re: FYI: WITH_REPRODUCIBLE_BUILD= problem for some files? From: Mark Millard In-Reply-To: <91F820A1-8940-4246-A20A-E62685F50079@yahoo.com> Date: Mon, 3 May 2021 21:27:07 -0700 Cc: FreeBSD-STABLE Mailing List , freebsd-current Content-Transfer-Encoding: quoted-printable Message-Id: References: <35482701-95A3-48B2-9A8E-B7E0092119B1.ref@yahoo.com> <35482701-95A3-48B2-9A8E-B7E0092119B1@yahoo.com> <43F20589-A7C7-42FF-9020-09CEE037D1CD@yahoo.com> <91F820A1-8940-4246-A20A-E62685F50079@yahoo.com> To: Ed Maste X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Rspamd-Queue-Id: 4FZ6J42tryz4b0T X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.50 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.65.148:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; SUBJECT_ENDS_QUESTION(1.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[98.137.65.148:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.148:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.148:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 May 2021 04:27:17 -0000 On 2021-May-3, at 19:26, Mark Millard wrote: > On 2021-May-3, at 10:51, Mark Millard wrote: >=20 >> On 2021-May-3, at 07:47, Ed Maste wrote: >>=20 >>> On Thu, 29 Apr 2021 at 02:50, Mark Millard via freebsd-current >>> wrote: >>>>=20 >>>> Files /usr/obj/DESTDIRs/13_0R-CA7-chroot/sbin/ping and = /usr/obj/DESTDIRs/13_0R-CA7-poud/sbin/ping differ >>>> Files /usr/obj/DESTDIRs/13_0R-CA7-chroot/sbin/ping6 and = /usr/obj/DESTDIRs/13_0R-CA7-poud/sbin/ping6 differ >>>> Files /usr/obj/DESTDIRs/13_0R-CA7-chroot/usr/bin/ntpq and = /usr/obj/DESTDIRs/13_0R-CA7-poud/usr/bin/ntpq differ... >>>=20 >>> This is unexpected. Unfortunately I haven't looked at = reproducibility >>> in a while, and my work was all on x86. This could be a regression = or >>> a longstanding issue with arm64. >>>=20 >>> If you install the diffoscope package (py37-diffoscope) and run it = on >>> the two directories / files it should give a more convenient view of >>> the differences. (Or, if you can make a tarball of the differing = files >>> I can take a look.) >>=20 >> I no longer have the same content in those directory >> trees: newer rebuild and the same buildworld used to >> installworld to both places, instead of 2 different >> buildworld's. I'm also unsure how reproducible getting >> differences was. >>=20 >> I can eventually do experiments to test multiple separate >> buildworld's and installworld's, but the machine is busy >> building ports and the llvm builds involved means it >> will be some time before I'd switch activities. And the >> buildworld's involve llvm builds as well and take notable >> time themselves. So my next comparison will not be any >> time soon. >>=20 >> I'll let you know if I manage to generate another example, >> this time being sure to keep the data. If I try multiple >> times without finding any differences, I'll eventually >> decide "enough is enough" and let you know. >=20 > I've still got a long ways to go to do the first > actual comparison of builds. >=20 > But I'll note that I've built and stalled py37-diffoscope > (new to me). A basic quick test showed that it reports: >=20 > W: diffoscope.main: Fuzzy-matching is currently disabled as the "tlsh" = module is unavailable. >=20 > As I'm not familiar with the tool, you might need to send > notes about how you want me to use the tool to get the > output that you would want. (And, so, I get to learn . . .) I've tried another experiment (* in the path matches "28" and "30"): # diffoscope /.zfs/snapshot/2021-04-*-01:40:48-0/bin/sh $<3/>2021-05-03 21:08:48 W: diffoscope.main: Fuzzy-matching is currently = disabled as the "tlsh" module is unavailable. $<3/>Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/diffoscope/main.py", line = 745, in main sys.exit(run_diffoscope(parsed_args)) File "/usr/local/lib/python3.7/site-packages/diffoscope/main.py", line = 677, in run_diffoscope difference =3D load_diff_from_path(path1) File = "/usr/local/lib/python3.7/site-packages/diffoscope/readers/__init__.py", = line 31, in load_diff_from_path return load_diff(codecs.getreader("utf-8")(fp), path) File = "/usr/local/lib/python3.7/site-packages/diffoscope/readers/__init__.py", = line 35, in load_diff return JSONReaderV1().load(fp, path) File = "/usr/local/lib/python3.7/site-packages/diffoscope/readers/json.py", = line 33, in load raw =3D json.load(fp) File "/usr/local/lib/python3.7/json/__init__.py", line 293, in load return loads(fp.read(), File "/usr/local/lib/python3.7/codecs.py", line 504, in read newchars, decodedbytes =3D self.decode(data, self.errors) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb7 in position 18: = invalid start byte The two older snapshots of a Boot Environment have bin/sh files that compare equal. But every program I tried the above sort of thing against on got the same UnicodeDecodeError result from diffoscope, byte value and position matching. These snapshots have more than an installworld in them and so are messy to compare overall. But the installworld (and installkernel) content show similar differences to what I reported before as far as example files with differences go. But this is aarch64, not armv7. It will still be notable time before I have simple installworld tree's to compare. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-stable@freebsd.org Tue May 4 07:10:28 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9821E6287B7 for ; Tue, 4 May 2021 07:10:28 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic310-20.consmr.mail.gq1.yahoo.com (sonic310-20.consmr.mail.gq1.yahoo.com [98.137.69.146]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FZ9wM535Mz4hSr for ; Tue, 4 May 2021 07:10:27 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1620112225; bh=TXn+exF8+ws+Nv/mmJg8+Lo6gi2ok16zp0rVnBFjxig=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=i9y2XJS8ph1MZy9Zq6b2cNKZUjuQiYIHVcluaZgFk6Q1Xroe+3QoAt7XTvJA2vgKTIjlYjoOX1kClGwf55OcHz3k5Lji+55iZod+kND1NAB5hCvnLB5JfRkb90C/SAoCDT9BB3VFZt6/WfXs9XpkiqQXp5WY7deTX9jpECgGtDRBory0efn8vVJ8kctbWR6tlaiG31pE1Tzsa6rPHHJO1/5R2suIvLvFgHfruJMvtJ5BKyno6Pf+dXefmFuGOAzQNWU6++6Wprpb4tCmk8xBc01JYY00gE9D/Ktj1N8eaAiF7YqlAKArWNz6iiAtUenVBAiq1w7B+DZ2kun8MLnTrA== X-YMail-OSG: z_T7vscVM1mPnc_KHLs0GWzwxZdJOWLxX4WBJlJ5UZfPh5XgNcInxZLpbqKdRhZ vnFz6pVxx.bpxNteBtoIeju02HVMUQsgYr.2OkZr8ql80NnT1VGeuQl2SjBvUEQs3iIS5ff4LT1X EeMYUGDbh0wy54byJ4Qm8JicanTqAQ.zUHiw043rzl6HOdswfNFnXNIzBkoQuccoJUgIkZtGm17H 0BqCfxL61u10PUFK4uTPpTAzqbHk5jVlnxHZ89blker.UmYyF3RaWL9e1vNVfv6nWkKqN8giSujw S1E0Mth97JeCTdFO6IAPaW1x2a86zxhAZeKzAloWCmC5i9T.Xs73HqI2jSQQLfqR62a4BKe5Z3nR wHVZ32GdyrY5URij5GkvN7u8Q05GzOSpvLuV4aouHBkBHfKij5ox8vKX8tWAsBnPRseIvT6MaYJ8 b1DAM3YEM0opRxnTTCfLpDG9KVOEEt1dL7H8XFemP7s2iF8ryZliPFLeJ1qWZ61BJ7WBzNFKuVRV lmMkl_Bt_uiY_RNAKjhFvvKsqJsjBt8CxKOQj97nmnQW08aXefEx16VNvPaVVcJVa_1nMYqYxGtb fk9ajJ6pHzHXQtUeIhVadLi9XOl_r.zfQree1EbAdowdy.SasT7GmdNn6Es.eYTtgvLJT0DmJDLo nc9X3gTXxBMjhjArYmkDVoZma3xTHVWWWMpqIPicT9WuriB1oNhnYIiLi6AIQ.2sOefF6drLgB4z nloujHgmlM6Aij6w.Km_XUmpZcd4SGLgvaqTkFP8JStunSiyyaPTgFgPDA7Qimv8sE2KgnR5lPKI 7f_cnkZcZdW0sXZh6r8Jq8CRWPT0DFHYl4NjDPdaVM2Rl1jWOGcGS3c_y0Kt2e51KIxcWX_kXYGI 0OJt9OwGqpugMl4RN8QMgnYXgTkSxrJNBBT6RNo4p7xaDJKGqtQk2ivsMIAZbZsW.kryIadUlAcm sH1zi3dkIjmuRHC2Z38mflIghIe0vQrY5NpgwGJO8yo6wEaluDwOP4w375eX_e_3BDiWy5HS9_gZ fhXPMjGJw0uZbfRvubKUKyjtvgvzVYEoxuwvtss6OuDNbrvjemSFqiU0dD0Z8p9aQjIKWCrLvUaO PneV4szVoxgB1gkcxMGL3oDK2bFlmLwOUT8h03recyit4bFA.d9mfA97IEeHFP94IyJkr8BD6_aG AphVZg.ibSPEKdx8ygCTgoN4HkdVIv7wUzbH5qiXkzMSE98UzY7rSXsoKGNM2ahUU5yxfhuu7ezh X0e0_ZHJ_ofOwZJ09EqouVvbwJtKLMR2WkTMSqc6Dq2eCAoLX91YzlEUMheoNxBS81_oL352JFuX 0.qf5XLoTopgpVQM8E5Q1xwQpbxnrZO6xKH9i5.MRca6IAwqL1NFvuj.MHPFRTMIVxc3ZxID5RRT FrgGSP96woNxdkXBTgah7xwugeTzSCdcLwQ2HnmuF0xNt346Dt8pnOeKCvYBp2YigMrdchBHWXoM ewvDuNsCkpQz9p_TeU.JhlgcrumDQPwy4ndqMb32TJ9mlen0VpHJ27kz0eAiuJ0OLzBLEWeHDaAn GJNCyELAnjrPI11afW10ZKh9ds7KqsqtwJkWmOJIGsmjxKfYrK1GbkpZ_MrFQdIAqfc9p3.PoEfA 8N52SIxdcWBg9Xsa9hZytp6LmKNZPcz1mPnJmL6AUkcbV3zaN1JW3FrF7OQYblYuMvuFCoIF5yOj mpj4awwxuIkQAKdKKgMk3gGw72t2.Opn7wHRkPFXwKmC1kULmePptPCqgT2TLcfmNbf4CjvKQOtl wMgvr9bpdNOCfTZACK4hYfyYSpAqQUQ8GqGNMG5.BVBALsPqMGpy5NY2IPEVYAYlGouWvi.Q7OEN aeYWlNEmVQBlg7bpdcgW7ZXXYR1gyCvADNRYLEUjvt_RZHNAMHBh9bP6slO6ez_ez0H3qs.E6NZz gd9AKc1nq00LKmE2MoESB5BULb0gBvau_tJgaqcAmcCym_8fcUZORlxGg4_4VVY731pdF46q6R3K m9jLZN9TtoELcFAH7Fi1WF5IC2RkQoVWiUGySBBdBd7TLjGDDKfn.0JFz3nPe6VZPEiPaYXRYojy a6nyHElV_ywFuwqeF140cKjd5khhfCvAif9pbwT.JcgktjyU0WCaL.rKSH3PTt9BLyVeIXqqw_kG VMDDSngekaK07u7VvOb7twgUwiBMbnzf.xzH7sJK3NjCMKhsXUuaYKTTv3SNhUGC9Mz2UKpiYIAj GrBxSn1YhKkXggW4O2XBnHKndvCabM0yJr2WONALwRTCG9JSnh0lvupoHP0deTBOVzHMe8Lj1tR9 6Uo1xcMzapTdag705VrKHtUJ4TM3BcvwFY7R5Js6UgFGYy9l7AhjGmuY3BIqnJ5obFJKuCReKBR3 xzk_kLMVpl0nnXRX9q8oCx7cTCBUlJRtB8cPTc8CHHq0WjpL5kHOS1W7UxEV51yeFLCnehZyDqzF QywLERj.bCWUMivUlW8JAuXmJncOCpCf64tK0y0i5VcCiUNUDRMTfz4htqCNshyIyGCLvPHQ_JxE OZxxEm8BhnE8hs9qbHsYx40aS8QH17UPXchNGs.DtsT.L54wKq9NOjz_i9Vl9zdh6eKGHme7k6cW S1PP5xW_kDb5932PFmNece67Y0gLx0HYGLwcWZm9ItyhkZcr5vSJ42Ystk73VEUy0wP7lYZYfyVI nNfMdz_Uu82WaOKkCXgTdg2anvBNOLhOaU.QRQFl. X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.gq1.yahoo.com with HTTP; Tue, 4 May 2021 07:10:25 +0000 Received: by kubenode547.mail-prod1.omega.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID b42c6738fb9278c8f9dc9d01867b8717; Tue, 04 May 2021 07:10:19 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: Re: FYI: WITH_REPRODUCIBLE_BUILD= problem for some files? From: Mark Millard In-Reply-To: Date: Tue, 4 May 2021 00:10:16 -0700 Cc: FreeBSD-STABLE Mailing List , freebsd-current Content-Transfer-Encoding: quoted-printable Message-Id: <7D83A5C8-FDB4-46D6-A148-15011606B3BE@yahoo.com> References: <35482701-95A3-48B2-9A8E-B7E0092119B1.ref@yahoo.com> <35482701-95A3-48B2-9A8E-B7E0092119B1@yahoo.com> <43F20589-A7C7-42FF-9020-09CEE037D1CD@yahoo.com> <91F820A1-8940-4246-A20A-E62685F50079@yahoo.com> To: Ed Maste X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Rspamd-Queue-Id: 4FZ9wM535Mz4hSr X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.50 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; SUBJECT_ENDS_QUESTION(1.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.69.146:from]; SPAMHAUS_ZRD(0.00)[98.137.69.146:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.146:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.146:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 May 2021 07:10:28 -0000 On 2021-May-3, at 21:27, Mark Millard wrote: > On 2021-May-3, at 19:26, Mark Millard wrote: >=20 >> On 2021-May-3, at 10:51, Mark Millard wrote: >>=20 >>> On 2021-May-3, at 07:47, Ed Maste wrote: >>>=20 >>>> On Thu, 29 Apr 2021 at 02:50, Mark Millard via freebsd-current >>>> wrote: >>>>>=20 >>>>> Files /usr/obj/DESTDIRs/13_0R-CA7-chroot/sbin/ping and = /usr/obj/DESTDIRs/13_0R-CA7-poud/sbin/ping differ >>>>> Files /usr/obj/DESTDIRs/13_0R-CA7-chroot/sbin/ping6 and = /usr/obj/DESTDIRs/13_0R-CA7-poud/sbin/ping6 differ >>>>> Files /usr/obj/DESTDIRs/13_0R-CA7-chroot/usr/bin/ntpq and = /usr/obj/DESTDIRs/13_0R-CA7-poud/usr/bin/ntpq differ... >>>>=20 >>>> This is unexpected. Unfortunately I haven't looked at = reproducibility >>>> in a while, and my work was all on x86. This could be a regression = or >>>> a longstanding issue with arm64. >>>>=20 >>>> If you install the diffoscope package (py37-diffoscope) and run it = on >>>> the two directories / files it should give a more convenient view = of >>>> the differences. (Or, if you can make a tarball of the differing = files >>>> I can take a look.) >>>=20 >>> I no longer have the same content in those directory >>> trees: newer rebuild and the same buildworld used to >>> installworld to both places, instead of 2 different >>> buildworld's. I'm also unsure how reproducible getting >>> differences was. >>>=20 >>> I can eventually do experiments to test multiple separate >>> buildworld's and installworld's, but the machine is busy >>> building ports and the llvm builds involved means it >>> will be some time before I'd switch activities. And the >>> buildworld's involve llvm builds as well and take notable >>> time themselves. So my next comparison will not be any >>> time soon. >>>=20 >>> I'll let you know if I manage to generate another example, >>> this time being sure to keep the data. If I try multiple >>> times without finding any differences, I'll eventually >>> decide "enough is enough" and let you know. >>=20 >> I've still got a long ways to go to do the first >> actual comparison of builds. >>=20 >> But I'll note that I've built and stalled py37-diffoscope >> (new to me). A basic quick test showed that it reports: >>=20 >> W: diffoscope.main: Fuzzy-matching is currently disabled as the = "tlsh" module is unavailable. >>=20 >> As I'm not familiar with the tool, you might need to send >> notes about how you want me to use the tool to get the >> output that you would want. (And, so, I get to learn . . .) >=20 > I've tried another experiment (* in the path matches "28" and "30"): >=20 > # diffoscope /.zfs/snapshot/2021-04-*-01:40:48-0/bin/sh > $<3/>2021-05-03 21:08:48 W: diffoscope.main: Fuzzy-matching is = currently disabled as the "tlsh" module is unavailable. > $<3/>Traceback (most recent call last): > File "/usr/local/lib/python3.7/site-packages/diffoscope/main.py", = line 745, in main > sys.exit(run_diffoscope(parsed_args)) > File "/usr/local/lib/python3.7/site-packages/diffoscope/main.py", = line 677, in run_diffoscope > difference =3D load_diff_from_path(path1) > File = "/usr/local/lib/python3.7/site-packages/diffoscope/readers/__init__.py", = line 31, in load_diff_from_path > return load_diff(codecs.getreader("utf-8")(fp), path) > File = "/usr/local/lib/python3.7/site-packages/diffoscope/readers/__init__.py", = line 35, in load_diff > return JSONReaderV1().load(fp, path) > File = "/usr/local/lib/python3.7/site-packages/diffoscope/readers/json.py", = line 33, in load > raw =3D json.load(fp) > File "/usr/local/lib/python3.7/json/__init__.py", line 293, in load > return loads(fp.read(), > File "/usr/local/lib/python3.7/codecs.py", line 504, in read > newchars, decodedbytes =3D self.decode(data, self.errors) > UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb7 in position = 18: invalid start byte >=20 > The two older snapshots of a Boot Environment have > bin/sh files that compare equal. But every program I > tried the above sort of thing against on got the same > UnicodeDecodeError result from diffoscope, byte value > and position matching. >=20 > These snapshots have more than an installworld in them > and so are messy to compare overall. But the > installworld (and installkernel) content show similar > differences to what I reported before as far as > example files with differences go. But this is aarch64, > not armv7. >=20 > It will still be notable time before I have simple > installworld tree's to compare. While waiting for the 2nd buildworld to complete, I used the 1st buildworld's materials to installworld twice (to different, empty directory trees) and then did a diff. The diff reported: # diff -rq /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/ = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm2/ | more diff: /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/usr/tests/local: No = such file or directory diff: = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/usr/tests/sys/pjdfstest/tests/t= ests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tes= ts/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests= /tests/tests/tests/tests/tests/tests/tests: Too many levels of symbolic = links FYI: # ls -Tld /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/usr/tests/local lrwxr-xr-x 1 root wheel 14 May 3 23:18:22 2021 = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/usr/tests/local -> = ../local/tests So this looks like the expected result and suggests that the problem(s) are at the buildworld stage. (No surprise.) I should be sleeping by the time the 2nd buildworld for cortex-a72 (aarch64) is done. (That is what I decided to test initially.) So it will be more hours until I have basic comparison results from 2 distinct buildworlds. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-stable@freebsd.org Tue May 4 13:02:35 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6B2C5631981; Tue, 4 May 2021 13:02:35 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-il1-f170.google.com (mail-il1-f170.google.com [209.85.166.170]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FZKkZ08p4z3GVJ; Tue, 4 May 2021 13:02:29 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-il1-f170.google.com with SMTP id l19so6242470ilk.13; Tue, 04 May 2021 06:02:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=LthTeXlz074DBmzKyJn1728JnXgN4bhkxNT5P3WSM64=; b=b+RnMf4Od/tK7d4zgoAGUEQ3tEIepPCNgO41Nn6hsCCVK4NVDHv1UT91GKbgnGOp1Z OJjQG4vCyj1jm55UTF7G181JmVG8UcdrFNqvWrXklRAwGCRPVVmDcvhTm5703hqe+9vj 88SZuS55X3zJ+Qe8YcYWYA+Jyyy0zQkf06y3LyLMVKLzpwv13zmxMIdmS/p4TlzzRYnZ oJ3M97x1H17ikkS0UTBgyysIPYCxMhfp7UwtkgjzujCTfCyW3EujTaq8iG3LcC5x+FEa co9RmIDk3JYDGF6A6XwaRuBFBJ3y3FOLT6hhzcpvhmLYJUWTsu9HF/KhifgW5V/B0d5d rMPQ== X-Gm-Message-State: AOAM532VG1TwSXw05kynTTkdFZOIGue/c3o+EAFUMULOnGLqZC5ocooS MUvIHC7agVG3MMX5Glqbldbn9Sihd+OvfC82wug= X-Google-Smtp-Source: ABdhPJy8Wi0KbdkuzOOHRI7tyqZlrKN/LDYKcgWnZBdbepJuJ/1JhOXg4fPQGRDtWzSUrv/45rA5gS8wyz3+VG0M958= X-Received: by 2002:a05:6e02:102:: with SMTP id t2mr16143626ilm.182.1620133348597; Tue, 04 May 2021 06:02:28 -0700 (PDT) MIME-Version: 1.0 References: <35482701-95A3-48B2-9A8E-B7E0092119B1.ref@yahoo.com> <35482701-95A3-48B2-9A8E-B7E0092119B1@yahoo.com> <43F20589-A7C7-42FF-9020-09CEE037D1CD@yahoo.com> <91F820A1-8940-4246-A20A-E62685F50079@yahoo.com> In-Reply-To: From: Ed Maste Date: Tue, 4 May 2021 09:01:34 -0400 Message-ID: Subject: Re: FYI: WITH_REPRODUCIBLE_BUILD= problem for some files? To: Mark Millard Cc: FreeBSD-STABLE Mailing List , freebsd-current Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4FZKkZ08p4z3GVJ X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of carpeddiem@gmail.com designates 209.85.166.170 as permitted sender) smtp.mailfrom=carpeddiem@gmail.com X-Spamd-Result: default: False [-2.00 / 15.00]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FREEMAIL_TO(0.00)[yahoo.com]; FORGED_SENDER(0.30)[emaste@freebsd.org,carpeddiem@gmail.com]; R_DKIM_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[209.85.166.170:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[emaste@freebsd.org,carpeddiem@gmail.com]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FREEFALL_USER(0.00)[carpeddiem]; SUBJECT_ENDS_QUESTION(1.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[freebsd.org]; SPAMHAUS_ZRD(0.00)[209.85.166.170:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; BLOCKLISTDE_FAIL(0.00)[209.85.166.170:query timed out]; RCVD_IN_DNSWL_NONE(0.00)[209.85.166.170:from]; NEURAL_HAM_LONG(-1.00)[-1.000]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.166.170:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable,freebsd-current] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 May 2021 13:02:35 -0000 On Mon, 3 May 2021 at 22:26, Mark Millard wrote: > > But I'll note that I've built and stalled py37-diffoscope > (new to me). A basic quick test showed that it reports: > > W: diffoscope.main: Fuzzy-matching is currently disabled as the "tlsh" module is unavailable. I just looked up tlsh - its "A Locality Sensitive Hash"; I presume diffoscope uses it to infer file renames. I believe the warning emitted here should have no impact on the output we're looking for. As far as the utf-8 issues go, diffoscope requires a utf-8 locale and I suspect that is the issue. If you don't have LANG set already, try setting LANG=C.UTF-8 in your environment. From owner-freebsd-stable@freebsd.org Tue May 4 13:23:13 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 31F0D631BFE for ; Tue, 4 May 2021 13:23:13 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mail-qt1-x831.google.com (mail-qt1-x831.google.com [IPv6:2607:f8b0:4864:20::831]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FZLBR0H3Wz3HKX for ; Tue, 4 May 2021 13:23:09 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: by mail-qt1-x831.google.com with SMTP id o12so6231477qtx.8 for ; Tue, 04 May 2021 06:23:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=RVY2aADUypi/PDVJ+VHcHv0fWu1XhS294NQfl8K25L4=; b=aAzM1qQ+977s2e3ywmHPPsLaj8s6a4XF5QjU/Ag1S+4MVhCd9RzqtA9zxWzW36MwMJ QC59i0q7tSCC4SevcYjiDN5ZN6TSo7IhZIqcTBrITu/67zB6fuvVnDdrfjCY+qiY4JgM skXWyJNU3g5L8ruQzFQPWfKWe5YAA7+oFFWJvJtPOLZ5mgfKYRVdfUECAyVMHkJmJ0c4 ETyfKxJqSIXDtpIDQqRS3Qy8sU0JwBmAYn+tjJfW8mkbImTdNwsHVpKPeVqzhF6+cTIb mFPglUkWYBhamyowIzDxTUQF0b3059w7L5uqoYipFq/PGtcFruNLOQJTwvOHqXOt00qq dm8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=RVY2aADUypi/PDVJ+VHcHv0fWu1XhS294NQfl8K25L4=; b=K0D1y4XEctVujeCJNx4Q/gR5UAYBu/HAPuc3AFyun6Se0O9gBf0IYeXtDBB1S+K3b0 ZiPgOWEBlXHXV2yt02lOXfvYfXtnCPtYoJwgbWZakS7rGLRR7B067mFBckQE8iLaBFs3 2c4J6jmvzdGKDl5IgAtgQttpnS3m/ZwIln1W1Qe+NzjJ5LcLxMe9FCZZiMqMj7YrtdL9 Z94O/bFNnwkwHhVwDrvGU5yKSNKkc7Tv7A1Z5vnFg+lcRhP3B6jsGHBA3oEiySphJDsM 8TYSO1kI5iJ5dB7af8IfHt4svPzu9247aSUgwGeD+QJ4ExzLpvhicBsAsP/StzSluvdK y04g== X-Gm-Message-State: AOAM532fgJV+WpqTXtmtPDHBXGlVKmt0xJsDbGM4cmV0I9oAM2vv3aBm xyE0AuTTZDImGAgbrIXl9vcW8dnffQ/mabLm X-Google-Smtp-Source: ABdhPJzkySE6Ys0Ag3v/wyG6O2rQain1CTk7CwGl8X/SaVyWaKAFAalH0RxcSotJHGWbyVCmlqUn5Q== X-Received: by 2002:ac8:7405:: with SMTP id p5mr22869422qtq.231.1620134588661; Tue, 04 May 2021 06:23:08 -0700 (PDT) Received: from mutt-hbsd (pool-100-16-222-53.bltmmd.fios.verizon.net. [100.16.222.53]) by smtp.gmail.com with ESMTPSA id h11sm7497024qkl.108.2021.05.04.06.23.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 May 2021 06:23:07 -0700 (PDT) Date: Tue, 4 May 2021 09:23:07 -0400 From: Shawn Webb To: Juraj Lutter Cc: freebsd-stable stable Subject: Re: bhyve and multiple network devices Message-ID: <20210504132307.3kggcui5t63biqw7@mutt-hbsd> X-Operating-System: FreeBSD mutt-hbsd 14.0-CURRENT-HBSD FreeBSD 14.0-CURRENT-HBSD X-PGP-Key: https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/blob/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc References: <72FC2AD5-7E59-4C3D-8E4A-19AAC6608304@lutter.sk> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="pr5pdjzyeyo67tf4" Content-Disposition: inline In-Reply-To: <72FC2AD5-7E59-4C3D-8E4A-19AAC6608304@lutter.sk> X-Rspamd-Queue-Id: 4FZLBR0H3Wz3HKX X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=hardenedbsd.org header.s=google header.b=aAzM1qQ+; dmarc=none; spf=pass (mx1.freebsd.org: domain of shawn.webb@hardenedbsd.org designates 2607:f8b0:4864:20::831 as permitted sender) smtp.mailfrom=shawn.webb@hardenedbsd.org X-Spamd-Result: default: False [-5.09 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[hardenedbsd.org:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.99)[-0.991]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::831:from]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RECEIVED_SPAMHAUS_PBL(0.00)[100.16.222.53:received]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[hardenedbsd.org:s=google]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[hardenedbsd.org]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::831:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::831:from]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 May 2021 13:23:13 -0000 --pr5pdjzyeyo67tf4 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, May 03, 2021 at 10:46:34PM +0200, Juraj Lutter wrote: > Hi, >=20 > my bhyve command line (on 13.0-RELEASE) is: >=20 > /usr/sbin/bhyve -c 2 -m 4G -H -A -P -u -s 0:0,hostbridge -s 1:0,lpc -s 2:= 0,virtio-net,tap100 -s 3:0,virtio-net,priv0 -s 4:0,virtio-blk,/dev/zvol/zro= ot/data/minio/host0/root -l com1,/dev/nmdm100B mr >=20 > the result is: >=20 > device emulation initialization error: File exists >=20 > It works when using original interface name (tap105 in this case). >=20 > priv0 is renamed tap: > priv0: flags=3D8902 metric 0 mtu 1500 > options=3D80000 > ether 58:9c:fc:10:2b:0b > groups: tap > media: Ethernet autoselect > status: no carrier > nd6 options=3D29 >=20 > Am I doing something inadequate, is this an expected behavior or is this = a kind of a bug? Unfortunately, bhyve doesn't support renamed tap devices. You'll need to keep the original tapN name. What you might want to experiment with is setting a description for the tap device. For example: ifconfig tapN description "private vNIC" Thanks, --=20 Shawn Webb Cofounder / Security Engineer HardenedBSD https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A= 4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc --pr5pdjzyeyo67tf4 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEA6TL67gupaZ9nzhT/y5nonf44foFAmCRSrUACgkQ/y5nonf4 4fp9DBAAgRlpHQohkC1SDVj8wxegprfHCA4945sgGGvZ+CIJVHnYAFsZWvqD4t+Y pViNVY3o3irj0/FiELQ/iuRZr94HiRZtScER9rZW4X42EqBj4T/DDV8cpXhvFMvS HJPIzDn+4RHyrDw7ffzjfcx+VMIFQdXJgnYL6kwj/1sfsqwPaSh/eFHpj83xjdJj GqegEDxoPvi2zoeGImflcYlfOHoiroj+8edrTZzVtbJs/vqf6ArVwJ4poCqRQaJ4 cGdfr9as4JcyHy6FIrGgHM37F8tzhGWJWDPGVvIARAp0F6HF3gNKaczGUGEC1rZf Lssv1IpF5nzHaubPehc6kh5nIgbKptQXr1P9F5Ewq++XoObrxG0oOYbOuDc9FYPo gUSD9IY8NeyfgegXfeArtmY5GJqYfg1ppfthw+wjaizffH9ZW7MOxxiJrbMdeLKH 9GtKfRA7hBEZ+YMLvWJlnyWqhNSBbHaheIro8gbzFZRwhLMYhayWa2fvxauOpsw5 Rfyz7XE4SoxfBvnBaLLf5tyhCs4KA9iHgBq/zG5pVTn5BhEA8hrdSp1SGPHcW8a8 XDrbj3G4nJ3ilHf5CcrTsk6dRbCQFRKnkskX4mWsfm8Us+gfLGEFg8X5AOFt4XMh hwAN432O2obioNEOWl9HatdwP6Kh47FMOuDHW9nOX43rGycHM80= =Iqxv -----END PGP SIGNATURE----- --pr5pdjzyeyo67tf4-- From owner-freebsd-stable@freebsd.org Tue May 4 13:33:02 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1E90F6326A7 for ; Tue, 4 May 2021 13:33:02 +0000 (UTC) (envelope-from juraj@lutter.sk) Received: from ns2.wilbury.net (ns2.wilbury.net [92.60.51.55]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "svc.wilbury.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FZLPn1bk1z3Hqd for ; Tue, 4 May 2021 13:33:00 +0000 (UTC) (envelope-from juraj@lutter.sk) Received: from smtpclient.apple (hq.bonet.sk [92.60.48.52]) (Authenticated sender: juraj@lutter.sk) by svc.wilbury.net (Postfix) with ESMTPSA id 94D1F45D14F; Tue, 4 May 2021 15:32:53 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.80.0.2.43\)) Subject: Re: bhyve and multiple network devices From: Juraj Lutter In-Reply-To: <20210504132307.3kggcui5t63biqw7@mutt-hbsd> Date: Tue, 4 May 2021 15:32:53 +0200 Cc: freebsd-stable stable Content-Transfer-Encoding: quoted-printable Message-Id: <6AFCDC36-E3C0-48A4-9F3C-41B3C70402A4@lutter.sk> References: <72FC2AD5-7E59-4C3D-8E4A-19AAC6608304@lutter.sk> <20210504132307.3kggcui5t63biqw7@mutt-hbsd> To: Shawn Webb X-Mailer: Apple Mail (2.3654.80.0.2.43) X-Rspamd-Queue-Id: 4FZLPn1bk1z3Hqd X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of juraj@lutter.sk designates 92.60.51.55 as permitted sender) smtp.mailfrom=juraj@lutter.sk X-Spamd-Result: default: False [-2.70 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[92.60.51.55:from]; R_SPF_ALLOW(-0.20)[+ip4:92.60.51.55:c]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[lutter.sk]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_RHS_MATCH_FROM(0.00)[]; SPAMHAUS_ZRD(0.00)[92.60.51.55:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.90)[-0.904]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:44185, ipnet:92.60.48.0/22, country:SK]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 May 2021 13:33:02 -0000 > On 4 May 2021, at 15:23, Shawn Webb = wrote: >=20 > Unfortunately, bhyve doesn't support renamed tap devices. You'll need > to keep the original tapN name. >=20 > What you might want to experiment with is setting a description for > the tap device. For example: >=20 > ifconfig tapN description "private vNIC" >=20 Fair enough. Anyway, I can specify the original name while also keeping = the device in ifconfig=E2=80=99s output renamed. It works. I=E2=80=99ve been able to make a lab setup of 4x minio nodes, 1x frr = router, all routed with BGP+ECMP. otis =E2=80=94 Juraj Lutter otis@FreeBSD.org From owner-freebsd-stable@freebsd.org Tue May 4 15:51:58 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BEDFC636E08 for ; Tue, 4 May 2021 15:51:58 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic311-24.consmr.mail.gq1.yahoo.com (sonic311-24.consmr.mail.gq1.yahoo.com [98.137.65.205]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FZPV55fdwz3h53 for ; Tue, 4 May 2021 15:51:57 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1620143516; bh=clcec9wkVDW3poI9b8J7ttC+NTMCfnWQLvLIvG/V57A=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=egZs7NpSYoUcdrjR9nb43q3yUgA5LX395Se7y0CEFviedYOAtHLji/zSPwnYLB34R+BHOYblRUc6lterlCeAUAhkePXSvPKjfbti4tTGG11xMaWqJmsa1/lGqEslB0t1Id60chXBVE9F2SM5fB2EaETH9e/zX8dLS2UOg3/g7YbbDsbjOgIWij6GW646eBmyiDRjokMIeDw5dd98oDD4AyE9dg1CR4uO2mnmi0a/B9MUq2C0JKBXx3VlzXb5e+/RMBSfi0bPp8QVhmrZeuVKukxHQd5FMJxugB+KoCVy0VHZpkTCOaYPwDldp7IfJP91Y2SFGhNH1BP3aCyJB5O/sg== X-YMail-OSG: 0g77.OoVM1mbrLyUD4uLM.ZvuOqIdPLvw88bBFEL5f3ynau0OK45fb04ZraeQ.p KwA77j2C7MN4DbGlIYvujsjalGnAB44btUsntRD3eWeQL3dqmxKiCf304uFJ3E8MU9fK9IGdiyrc yZB6XeULFe9BE2.02zGE.YetIUdFSLkkMJG9IGWdU4Rp.f24HUMhG9h1Pc1NukiWPtKqQmGPAJUi 9hRipjDlUkLWP7MU4y46RCsbIAHxcuLRFOfatm5Dx8NXVRWrhySs5ELma69.TyxM0P66Bbo_S1i1 B9IliDd3S0WgAqqF_ukWJPW..y0KwmCyddsv22A2YkJGuuakPgG0vuyNv25iAAFdc_3cBv.ksa7E oHrH6l7q1gBCEmqpvH3uqxYFgkCJYqxpV75dUIfOW19eU0A3e6ifPHQevUv7ciRBrWRdYlfpafWo u2oOVmfVQc21w5DC.vHx7Wi4qC0_gdl.KiIXAQmHR7OGOSI9a5JmLmQ8dhttED2M5V51hdVsvKgt PNMrZIe3vlm7Hn_MRqqczwdDHbUIZE3t7iAKb.u2mVkkp_4C_IBMkFI12RdGAIPEo7e7MC1ARFPV 0SR6jDH1ecsnjvK189L1XOOpz0v5hLAvbZyQVVj4RW6.GZsfPSzVBcblmMnN2eOhDB6JTir0W42T iQTFq1UnS8vNUCzgShmwCReFuxL5OM_9McjHSr9UPquFu3NVD7lpPO1vCHfcytYds0h1Nj4poaZw C7ASQJG6lHKxK8y4lb9RyiM4dkyN7I1HIlScNnve6H4Ny_5p0WEYmOJuR3FAc732gmPwtdZ_L9OF BgzV4Qf.R9ZZWsm1q683OFA_23tPdJ9OEBKPhYMelJ8rhr_FC_W04SmdicP6AE3_OkcQDU2sTA67 7BsElzS.onI4.YhWwf6FXOoFBDCkVKIENfpE9fajJmUGiaT6QIg7lQweNWJmAbe1tyaaDu5S9Bhb nCNjo3ULOiybgBW7QWc_37E4iBPgKozRMNuxjzHIE2agjfstj4kLh0kHN0zB4ESSDSanPKWMDVHI ZjVZFCFDaRJMAlSIBxpzWue_6g_WuUVBCUMF_c979uAanD1OV2U3kC9S6r5ttsFYhU_JB68H9PEO ZwrDeU9bb5y8tP4prBXBoTNuDuyLoYmCH4Ui3brfMTmeA3NieWUjLmW5TrdmkJIm1f8mdLrBZYtw ej1dAuH1eiBM4E5ikj8Qs2tLh2dtB8AejE9KufXweV09nKQ8WOd3aNv5LEyQcEBwRRz9_ry_2naC i56cMo5mHnfYBgIXNVnnMyo1TA3vF08QFOSUDJvBREZrpv8p2kOOl6z5NY_EXjVIU6GUlS1905CT tUWnmWsS7xObVgCxI3qsKSzyzJADB9.A29HUhpxilnAUvRyfgBnF0.DmEWa.Vt4uJ8Mo6lj1DRxp 4xPr9W562Nv4l9CQSvVbrsbayH5SBXpgrTBKGeaWK_lRITXl9JaT7pvcQVIyueoGoYFMDx.IJUPs 1QD0_7ChdCEWDmNjv6pfK6ln.noID3sHK8dVfUvuFQUSvk5mqggdwvsteI9Edrw90pcHcSsk1z_U YbJX3DkjSamttnZXwQHPi5AB9HHxjqSkXuCaucd2gEKT7UhspjMeh7eMj47VgLzLO_nN0.89o1MQ 987Hu6_UWjD_EV20_h_1AEKsoooQaiV49i2bNAYDtWtDdcJTte5AAK6387IwD58BxtryxKm530Ps HdrBSWrN1jeEHe.cYm8uekPu0RoLhAwZnZjl5bl_Zb67SzHyeRcU6Zru_0xIfeJIC7yOR4Iw9jKL NA7hJscnVRA_4eU_M3VjqmSQLUpMNBwpb26XwPZn3P6RvCTZbmCiloMaVH4BwUBYxL.UkAsvxCI7 wD1oIvvUcF2KSkngwxvn9IetewbEGVlqPXBZ3Q0Q3XMqzAIwiv41noBnBOogYbo8wwxsYUE8ax2J kYgoSuLH95N9qTtQ_afIaUrgFBnlLcgzH2yj3PZuM2sLpgFtVpVt2paV3XgWLmIwZMrBqVHLQw0X 71osrfX39U7tGvNCx6K4YWZZF_7ivNmbK43zAz3pQoXIpdxsuJZm.Enw3Ied2AEe.4ST42ilVBim EdsRHq1pcuSIqjFXQ2jfM0wrOHgmefPLG4WXickk6fbdFaN5Fvg7bLQUYPUa9pqp.RZOGs8hXDPA HJvzKvOas3KCMukd2G41EZOB6NFdUFo5mIr91r0rkBFroTLhTJglOH4B83ThcIVjfZzhfihfgii7 znm.vJg0XIOIGFBJ0bQbW4BaRsIrkjY0wBxl_pfnByg4O_2JgLMpfYSgfSTirUDJVSabxI7HBRYA nvvGlr5CzEVHAc.eypPP.Bie9nqgZd6_shDuI3UEvzd.TeDxiYdUpfrrqKewKAXqwmcmxz6Csabq agf9bZvpFOjN.azRo_Rota.wwZ0zy28B_IsMXYHY_iizAmYy._xI7ai1NR3ZljzCjIoXR891S0Ne ONYlbOKAMCIeT5XzYHNiC2uamIU2sYFfee.y8ETw0F4B7.I_DHKdl2lXTdTjQrhSNtnV5vOG8vEs Qj9uGyF4QZ3UR8p3acxkCn9DAicpsvl7hfzdSjCkOVWUgCXpzsfKh20Ljsi6P6aCnfHGMShNUngj j543tz931nAdeLd8yNvYPz77OcI9lVvhUZURIUO_F86uE9i36nhwq_C14TKnOk7.VbEYxDflHli5 xPbJqbbfPWDbSWnnkPRUMUVjJthn6ujE1lQaeL0Xq X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic311.consmr.mail.gq1.yahoo.com with HTTP; Tue, 4 May 2021 15:51:56 +0000 Received: by kubenode562.mail-prod1.omega.gq1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 248506260a44890ad829ac6f4f1c1881; Tue, 04 May 2021 15:51:51 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: Re: FYI: WITH_REPRODUCIBLE_BUILD= problem for some files? From: Mark Millard In-Reply-To: Date: Tue, 4 May 2021 08:51:50 -0700 Cc: FreeBSD-STABLE Mailing List , freebsd-current Content-Transfer-Encoding: quoted-printable Message-Id: <27B5086A-7C98-4AE6-885A-CC7C7BD2F64B@yahoo.com> References: <35482701-95A3-48B2-9A8E-B7E0092119B1.ref@yahoo.com> <35482701-95A3-48B2-9A8E-B7E0092119B1@yahoo.com> <43F20589-A7C7-42FF-9020-09CEE037D1CD@yahoo.com> <91F820A1-8940-4246-A20A-E62685F50079@yahoo.com> To: Ed Maste X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Rspamd-Queue-Id: 4FZPV55fdwz3h53 X-Spamd-Bar: - X-Spamd-Result: default: False [-1.10 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.65.205:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; SUBJECT_ENDS_QUESTION(1.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; NEURAL_SPAM_SHORT(0.40)[0.399]; SPAMHAUS_ZRD(0.00)[98.137.65.205:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.205:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.205:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 May 2021 15:51:58 -0000 On 2021-May-4, at 06:01, Ed Maste wrote: > On Mon, 3 May 2021 at 22:26, Mark Millard wrote: >>=20 >> But I'll note that I've built and stalled py37-diffoscope >> (new to me). A basic quick test showed that it reports: >>=20 >> W: diffoscope.main: Fuzzy-matching is currently disabled as the = "tlsh" module is unavailable. >=20 > I just looked up tlsh - its "A Locality Sensitive Hash"; I presume > diffoscope uses it to infer file renames. I believe the warning > emitted here should have no impact on the output we're looking for. Okay. > As far as the utf-8 issues go, diffoscope requires a utf-8 locale and > I suspect that is the issue. If you don't have LANG set already, try > setting LANG=3DC.UTF-8 in your environment. That is not the issue for the UnicodeDecodeError: # echo $LANG C.UTF-8 # diffoscope /.zfs/snapshot/2021-04-*-01:40:48-0/bin/sh $<3/>2021-05-04 08:49:21 W: diffoscope.main: Fuzzy-matching is currently = disabled as the "tlsh" module is unavailable. $<3/>Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/diffoscope/main.py", line = 745, in main sys.exit(run_diffoscope(parsed_args)) File "/usr/local/lib/python3.7/site-packages/diffoscope/main.py", line = 677, in run_diffoscope difference =3D load_diff_from_path(path1) File = "/usr/local/lib/python3.7/site-packages/diffoscope/readers/__init__.py", = line 31, in load_diff_from_path return load_diff(codecs.getreader("utf-8")(fp), path) File = "/usr/local/lib/python3.7/site-packages/diffoscope/readers/__init__.py", = line 35, in load_diff return JSONReaderV1().load(fp, path) File = "/usr/local/lib/python3.7/site-packages/diffoscope/readers/json.py", = line 33, in load raw =3D json.load(fp) File "/usr/local/lib/python3.7/json/__init__.py", line 293, in load return loads(fp.read(), File "/usr/local/lib/python3.7/codecs.py", line 504, in read newchars, decodedbytes =3D self.decode(data, self.errors) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb7 in position 18: = invalid start byte =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-stable@freebsd.org Tue May 4 16:56:49 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DB66F5F939A; Tue, 4 May 2021 16:56:49 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-io1-f52.google.com (mail-io1-f52.google.com [209.85.166.52]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FZQww5bW9z3m0X; Tue, 4 May 2021 16:56:48 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-io1-f52.google.com with SMTP id k25so7770079iob.6; Tue, 04 May 2021 09:56:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=yvYZEOLtbQSpF61Px6IhxYagIqcItrcyki0blF33n0w=; b=OdWgi0ve/dosh3c/oeCdA2ZGY10UiW8ItCfNbv7AgqimuBZwgXy7VCVRDJ6l1hRvVB ZLXODyXvagNyyWfcpZbbU2aelU0F6EN2zDw+vEdyj1n79JGXgwznKkaTl7krl8MDsa1w +/EH1aKbyfZKklGE1uIOvI0QFxdX9X0EZqpJ8FYbW57c+N3BjY5WoenwlCmB4PMhf67H 8dMrSUq7Vgwnku2qUpqD1VEueUtx3dSQx+PbGe0u4R0nFz6r6Uh0tlpBYo44dndewGYn /wTDZUhzbVUpG2MSWL/XZcuE88GuXUz6DjkLV7tWv9Iw/QFMj6/D/Lf9EVPIX4bhOF5T Svqg== X-Gm-Message-State: AOAM532b0JpjyEBK6bc7F1V636LFzkOoZy4PFmzWEKZZkoCvzkhBplcF lcEmUiekR9gxoavd2rr6oSYeuI/v6pjq/P1WVLM= X-Google-Smtp-Source: ABdhPJwK2xNBK4OLyYlqmTjSrpckuOd0vteVSsfVvPqY8ujaZmswTH6O4tgR/5xEh6yiQqr9L7nnU0veY61zvMUnIv4= X-Received: by 2002:a05:6638:37a6:: with SMTP id w38mr20980791jal.106.1620147407450; Tue, 04 May 2021 09:56:47 -0700 (PDT) MIME-Version: 1.0 References: <35482701-95A3-48B2-9A8E-B7E0092119B1.ref@yahoo.com> <35482701-95A3-48B2-9A8E-B7E0092119B1@yahoo.com> <43F20589-A7C7-42FF-9020-09CEE037D1CD@yahoo.com> <91F820A1-8940-4246-A20A-E62685F50079@yahoo.com> <27B5086A-7C98-4AE6-885A-CC7C7BD2F64B@yahoo.com> In-Reply-To: <27B5086A-7C98-4AE6-885A-CC7C7BD2F64B@yahoo.com> From: Ed Maste Date: Tue, 4 May 2021 12:55:52 -0400 Message-ID: Subject: Re: FYI: WITH_REPRODUCIBLE_BUILD= problem for some files? To: Mark Millard Cc: FreeBSD-STABLE Mailing List , freebsd-current Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4FZQww5bW9z3m0X X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of carpeddiem@gmail.com designates 209.85.166.52 as permitted sender) smtp.mailfrom=carpeddiem@gmail.com X-Spamd-Result: default: False [-1.99 / 15.00]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FREEMAIL_TO(0.00)[yahoo.com]; FORGED_SENDER(0.30)[emaste@freebsd.org,carpeddiem@gmail.com]; R_DKIM_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[209.85.166.52:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[emaste@freebsd.org,carpeddiem@gmail.com]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.994]; FREEFALL_USER(0.00)[carpeddiem]; SUBJECT_ENDS_QUESTION(1.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[freebsd.org]; SPAMHAUS_ZRD(0.00)[209.85.166.52:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCVD_IN_DNSWL_NONE(0.00)[209.85.166.52:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.166.52:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable,freebsd-current] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 May 2021 16:56:49 -0000 On Tue, 4 May 2021 at 11:52, Mark Millard wrote: > > > As far as the utf-8 issues go, diffoscope requires a utf-8 locale and > > I suspect that is the issue. If you don't have LANG set already, try > > setting LANG=C.UTF-8 in your environment. > > That is not the issue for the UnicodeDecodeError: > > # echo $LANG > C.UTF-8 > > # diffoscope /.zfs/snapshot/2021-04-*-01:40:48-0/bin/sh > [...] > $<3/>2021-05-04 08:49:21 W: diffoscope.main: Fuzzy-matching is currently disabled as the "tlsh" module is unavailable. > UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb7 in position 18: invalid start byte Hmm, interesting - if you don't mind sharing I'd be interested in a copy of /.zfs/snapshot/2021-04-*-01:40:48-0/bin/sh, in order to track down what appears to be a diffoscope issue. To investigate the non-reproducibility though we can just manually run through the same sort of process that Diffoscope uses. I would suggest cmp -x to find the offsets of the difference(s), then use readelf -S to determine which section(s) have differences. From owner-freebsd-stable@freebsd.org Tue May 4 17:01:25 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 54B875FA0BB for ; Tue, 4 May 2021 17:01:25 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic314-19.consmr.mail.gq1.yahoo.com (sonic314-19.consmr.mail.gq1.yahoo.com [98.137.69.82]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FZR2D2xzhz3mQd for ; Tue, 4 May 2021 17:01:24 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1620147682; bh=7BIiPqAM/LhnxKPot5GI2Bflf4HIDLcquhABCiIfFUv=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=s7VMdMiOJqMeJiQ8xlHc0dLIvap39FPGG7XIgeSi3Flh0LsCIoU05zjzJIJ5HmIiOf8JyJNUSsL0cHnjlI9vy6RmvmOuuTikzH6EvyTUul/igqDrAYHJhtLbuGRsfDvX3i2oESWnnfDcaB/9viXlFIxdB/bFLf58KfCA67CevQCCyMbhz5/WQAjTgqTL3T03mKCd4lTS74ZInUxRQnSDB+tT+MerWaxnL5d/99p0KWOAPeBn4dQ0u5ItH+jgGMn+GfMO12gxZwVeyzv+AOllYvWbvcFhHgU/Imo+n+MNLwltbMc4+hcfBZKnFqvJqhcMV6mFe+S9e2UbM9MpxbZNSA== X-YMail-OSG: oslOhY8VM1lsaDBsmMAnZPpJ60kzzIGgsY6UKYYTArp2w2mzorUW9pZcWgOOxpp QUsOkyumH5E5q9GE1A_4B..vweaZMa2c_ZcNb4isKhJExyoDUtD_lThKQewFu5XKodirLPUqgQch REGW0buuMAh1CjiH0Du8z35GUktdvhSZbQ7s9hUD5cJbC2CapO_IJKABSiK6xh2zmtaFAiIsHjMX AoU_J2b2byqS4taqatCAqVVf.VkaFprgcpPgVVykRoF907YtNgD7JE2jjA0Y5Uwl7jU2Vv6Hx6_N ieCX4VXrasBoNRkPUaa7Kg13R7KbAOgZygblBHX1ZxdmIQQFM44sB6pKETTuoaJ2NQHmakG8Kzix 8sihZWjORFNra2qeFxBmRjF_Du28G1rt9.gz38vzSUmyaIICaYaeQJ.xI_QditrmY93NZG3aoteU CsP9iyvyh4H8c6UOOe6T9cguzGMKPusAsA7FpFKMSDGHpIwUZlgz9D5qHa_IFmghsV2mFaTEkrK9 tP.ssaSUKugVwNvu2GmFD.mvxsU1f15WoxjF.rlXFbkSTQFeOKAChQdO.KmrEgwed4XwuaOabBPx qCgIHxl7OIKjocqWiqZhshbGw_cJo0zGfS3Grk56QJFQUh8haZ7Ir_4lme.7eztBTkot0WL_UoOI h6NJ41Kou2IJSS_auBtgtJePFKQgTaZRBytWUJPoJ2bXXp.XK12x0XW6Rr7GVk.3oh6rTi2eHgzm ygR5a0jwtPmCiLpdfzROfdC2fgiKgAuxXXrbgZ.dvyfQIcx3iCIOjrajX8mVATnVCD4VdMd0JHn. na5ilpcomsKx8Zhlm4fL_vmGmFYOc6qwu3apS1Vzu5gWe8vOtHi6OexoHHcv_NCjAJ9kYj_aCSVz eoWFAlmE52j5rKAdkYlz_t6lKGtKg_QfDfAMkU6lQ.itW3X7km2oHFqaMyJicMmKkojt.eYpYhXY hJeNAA4fO_9YeSer9DrJEB2FkT2Df0pVuhyvEF49bHWUW.1bwzG_CCuMScNnxs3xoQ_NWtBBm6xY FdhOpdtI06vDz5pjbk0iTvSjys4zGH4pHwPXzlq_GKJyOKxtTD9a7KpTl_JoGOpe0MawnQhfC_OG zV.t8hG8HkV4huGihHwGP5Wytzj0be073i35_HNrCoFP71DTqH1j_Cd1BZWpnQJW97Nunf2K1G9N OlYCYKL6Fsxhh6oDY4b8vcQAzFN7suNZBu6Hm5TK4Gyq.SWr7yuB2iLqm3ImMCGqP6qhb9eI1mJE M.bcO7haqbbhZDqmamR8eZ2UjApheISJhMvkOHad7C9AI7LufNSGU_w2rdqVlbFjWYFvSGA2d4bY CCgBMAPHtZPgaq_KQ1Fp5EyU0pCijQk6zeUTltQaiB.PkBg0i0qkM25GaYaX7qYimiEntLfOMHyd kTH4.ougDx4IfzKHtoDd7eJsCJA4aZ1RTQxFN1Qojp7yQRpe62S5f.4lPKl2p9G3Ez_pi34QXX6r cnkcLjqjvJ3Af9Aw3wWsZXSDnx_Xmt8Apph1_2dFktM6qvfiaCVqv_BBdWRHe2YtisKVlYqS3xj5 NJOQTVAi5CBaT5FitHF9WHTXbKWeGwWeEcutLcL3Fo2R2eXTSCpjuYwqyPEdCYSx87IEfH9.sDa0 N2ZLW_upJQOk1XFSo06O6_04qDYOC.iflYNNVsv73s6hd9j9Za_Xw59xKFsk.sb6tVd.u9NDLF.c UpiFAxBNx0utVXK6Ak.FcMsz4mIcFz3foT7LFJ9z3qpVW2aDwBJx2YmZ8fq2wQDa9NXmm2Q5DrPz 1SwSJP27V9RwjvuPV_VXY.3npH9ckYARycZSl6Y._LhI.tnpK2Vc6N4N9c6TYmV3AFAhcKo36Xly NYA5fA.DkhugqMeNmqr_gEotJAEBladGsMwkblfg_PsFV8kRdB22YIyYcAUX68jJfCbgseHILhJ4 WTMB0x1xjybfAbTRy.6YCga2dQ5z28ZJ434ce_xfYpBJDje8OKV_rtpnLmeYpYpdaTY8TTvNjA2s LAhIylF_R1989uHiE3z0oIkXzEkmn.HRF8WdWC75JWznepZshGZlDMfQ60a7w2UQZt4FdbuAnPAX 2vShHJ2fmAzdhJmZI8EmEbLeKvjM4I.JRE1biXUR.jmAj34kmFkKPSmdeXigIpnWB2w62o.WGLoj YTuxucLLl1TBnJKQeA2_KqMKRcXZ4z0UOk4THsPltNMDTNrUVe.WAoTkBArNWPrvbFMAg6CpidO6 OkeN.clAh8BMD8rTFu93zC77Q77hSR_ORoeuNxOyGwa8KY.LRPWTOdmOD7KmHDV9ahK2wxLranND Tsdp1fh0zY4PHKHyBGMyPxseydfzUBvaYqraLvx1oq4sKCH2IhpyzIISrfeARiLOybAM6xgA7cti BJyYNMQ4i96u1KTHsc9L5EOgp2HJLyGyFJ3k0eyzrbu5l2vIOqOT3.SdMuoRu_D4gBNQU5WmJIse tOKTwpeQA0Eb8bJQPKYHQokGSPTsxd12CQfJ9aBTA8Fo73MuRBPf3MBhR0aXSTtCMcNTJ9L49uTT l69hRqSI3fwwP4RD2a4rBE0o0knfcUTGMZt9IzG.Ho8a23Z4ZC6GHvyHSajzGhOXqjSMwmynjg6r gChPhQ1byk7uX.1IraMHOy90QNIkV1woal71kUPUi7TRCzDmOBwMB65oIvFgh5lZG7ixEQtH.m9E bPS1iEsVhNfLWnzhNo58wIXH211kSmFm1YltEoWq00fx7_g-- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.gq1.yahoo.com with HTTP; Tue, 4 May 2021 17:01:22 +0000 Received: by kubenode538.mail-prod1.omega.ne1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID c435762641fda1a4169dcdc43b265b30; Tue, 04 May 2021 17:01:17 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: Re: FYI: WITH_REPRODUCIBLE_BUILD= problem for some files? From: Mark Millard In-Reply-To: <27B5086A-7C98-4AE6-885A-CC7C7BD2F64B@yahoo.com> Date: Tue, 4 May 2021 10:01:15 -0700 Cc: FreeBSD-STABLE Mailing List , freebsd-current Content-Transfer-Encoding: quoted-printable Message-Id: <3FC6BCDD-5865-4B5D-8238-3EC38AD4E78C@yahoo.com> References: <35482701-95A3-48B2-9A8E-B7E0092119B1.ref@yahoo.com> <35482701-95A3-48B2-9A8E-B7E0092119B1@yahoo.com> <43F20589-A7C7-42FF-9020-09CEE037D1CD@yahoo.com> <91F820A1-8940-4246-A20A-E62685F50079@yahoo.com> <27B5086A-7C98-4AE6-885A-CC7C7BD2F64B@yahoo.com> To: Ed Maste X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Rspamd-Queue-Id: 4FZR2D2xzhz3mQd X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.50 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.69.82:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; SUBJECT_ENDS_QUESTION(1.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[98.137.69.82:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.82:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.82:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 May 2021 17:01:25 -0000 On 2021-May-4, at 08:51, Mark Millard wrote: > On 2021-May-4, at 06:01, Ed Maste wrote: >=20 >> On Mon, 3 May 2021 at 22:26, Mark Millard wrote: >>>=20 >>> But I'll note that I've built and stalled py37-diffoscope >>> (new to me). A basic quick test showed that it reports: >>>=20 >>> W: diffoscope.main: Fuzzy-matching is currently disabled as the = "tlsh" module is unavailable. >>=20 >> I just looked up tlsh - its "A Locality Sensitive Hash"; I presume >> diffoscope uses it to infer file renames. I believe the warning >> emitted here should have no impact on the output we're looking for. >=20 > Okay. >=20 >> As far as the utf-8 issues go, diffoscope requires a utf-8 locale and >> I suspect that is the issue. If you don't have LANG set already, try >> setting LANG=3DC.UTF-8 in your environment. >=20 > That is not the issue for the UnicodeDecodeError: >=20 > # echo $LANG > C.UTF-8 >=20 > # diffoscope /.zfs/snapshot/2021-04-*-01:40:48-0/bin/sh > $<3/>2021-05-04 08:49:21 W: diffoscope.main: Fuzzy-matching is = currently disabled as the "tlsh" module is unavailable. > $<3/>Traceback (most recent call last): > File "/usr/local/lib/python3.7/site-packages/diffoscope/main.py", = line 745, in main > sys.exit(run_diffoscope(parsed_args)) > File "/usr/local/lib/python3.7/site-packages/diffoscope/main.py", = line 677, in run_diffoscope > difference =3D load_diff_from_path(path1) > File = "/usr/local/lib/python3.7/site-packages/diffoscope/readers/__init__.py", = line 31, in load_diff_from_path > return load_diff(codecs.getreader("utf-8")(fp), path) > File = "/usr/local/lib/python3.7/site-packages/diffoscope/readers/__init__.py", = line 35, in load_diff > return JSONReaderV1().load(fp, path) > File = "/usr/local/lib/python3.7/site-packages/diffoscope/readers/json.py", = line 33, in load > raw =3D json.load(fp) > File "/usr/local/lib/python3.7/json/__init__.py", line 293, in load > return loads(fp.read(), > File "/usr/local/lib/python3.7/codecs.py", line 504, in read > newchars, decodedbytes =3D self.decode(data, self.errors) > UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb7 in position = 18: invalid start byte >=20 Well, the list of differing files is huge. But this seems to be .gnu_debuglink content for the area it is in. I'll note that I did installworld but not the likes of distrib-dirs or distribution this time. This test did buildworld to two distinct directories: zroot/BUILDs/13_0R-CA72-nodbg-clang 5.13G 118G 5.13G = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang zroot/BUILDs/13_0R-CA72-nodbg-clang-alt 4.28G 118G 4.28G = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt and installworld to 2 distinct directories: zroot/DESTDIRs/13_0R-CA72-instwrld-alt 1.44G 118G 1.44G = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt zroot/DESTDIRs/13_0R-CA72-instwrld-norm 1.44G 118G 1.44G = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm Previously (armv7 target) I had built, installed, rebuilt to same directory (after clean-out) and installed to an alternate directory. That had gotten only a few files different but I do not know (yet) if it was the procedural difference that made the difference. Prefix of the list of different files this time: # diff -rq /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/ = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/ | more Files /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/[ and = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/[ differ Files /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/cat and = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/cat differ Files /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/chflags and = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/chflags differ Files /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/chio and = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/chio differ . . . Looking, aarch64 seems to typically get a back-to-back sequence of 4 bytes different in native programs in my builds: # cmp -x /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/cat = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/cat 00003bd4 1d 65 00003bd5 eb a3 00003bd6 bb ca 00003bd7 8e 1a # ls -Tld /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/cat = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/cat -r-xr-xr-x 1 root wheel 18448 May 4 08:55:01 2021 = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/cat -r-xr-xr-x 1 root wheel 18448 May 3 23:16:36 2021 = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/cat Sections: Idx Name Size VMA LMA File off = Algn . . . 25 .gnu_debuglink 00000010 0000000000000000 0000000000000000 = 00003bc8 2**0 CONTENTS, READONLY 00003bd4-00003bc8 =3D=3D 0xC # cmp -x /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/chflags = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/chflags 00002208 88 a1 00002209 e6 40 0000220a 60 94 0000220b bf ce # ls -Tld /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/chflags = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/chflags -r-xr-xr-x 1 root wheel 11440 May 4 08:55:01 2021 = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/chflags -r-xr-xr-x 1 root wheel 11440 May 3 23:16:36 2021 = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/chflags Sections: Idx Name Size VMA LMA File off = Algn . . . 25 .gnu_debuglink 00000014 0000000000000000 0000000000000000 = 000021f8 2**0 CONTENTS, READONLY 00002208-000021f8 =3D=3D 0x10 # cmp -x /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/chio = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/chio 000050c4 6b 3e 000050c5 08 ca 000050c6 7a 2f 000050c7 5d 64 # ls -Tld /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/chio = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/chio -r-xr-xr-x 1 root wheel 23728 May 4 08:55:01 2021 = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/chio -r-xr-xr-x 1 root wheel 23728 May 3 23:16:37 2021 = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/chio Sections: Idx Name Size VMA LMA File off = Algn . . . 25 .gnu_debuglink 00000010 0000000000000000 0000000000000000 = 000050b8 2**0 CONTENTS, READONLY 000050c4-000050b8 =3D=3D 0xC For all I know, some individual byte(s) in the 4 might accidentally match sometimes. The addition offset after .gnu_debuglink's file offset does vary (0xC and 0x10 above). The content of those differences do not look like file path components, for example the 0x08 byte. I do build with: # Avoid stripping but do not control host -g status as well: DEBUG_FLAGS+=3D # WITH_REPRODUCIBLE_BUILD=3D WITH_DEBUG_FILES=3D But that was true for the earlier armv7 target example that I reported that only got a few files with differences. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-stable@freebsd.org Tue May 4 17:17:42 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6EA415FAC30 for ; Tue, 4 May 2021 17:17:42 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic304-24.consmr.mail.gq1.yahoo.com (sonic304-24.consmr.mail.gq1.yahoo.com [98.137.68.205]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FZRNw4BSlz3nFm for ; Tue, 4 May 2021 17:17:36 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1620148654; bh=5hPBqD1eQ4r3oEFTQXB4jZ+ruJJjDNzv3EnxgpC8AxD=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=JDGeFUZmtnZPWcq5PjQ/zWtytZOXvtgjBjnCx8pHb+kxdI9idJQEAsFrEKViietcW0OdKHkhssLQWbgOqOsi9nNprQ1qySXqEc1iyoOKRoHUCRbg1482dkSW+FcVoqvdwoJLs7mhexKSWjuSkAWz4mKEY5Sa658lThZ4FENlxbpqCzVksDFurHdbKRHc437C+OnMQUTyq/PtcAiXOYPNWbctj9MWbRfkF8iC3kl4/yJ2qpi1oLsZYJ5kvK3LqR24820qQhL1Gc3ArHyCywBgW8cfY8oWqJ6pE+ELKBAw7RMA8fpQ1iJuNgRG3FyjsC44hhIPXc9i9yxYHPX22KYyRg== X-YMail-OSG: jab7ix8VM1kUz_1QVWzmsslDaw.he.zwmC1cgpNUdVfFLIGPbOT4GOAyCIo9J8v vfHqo1GN4HqHm2jA4Zsae2XMTmHGMveWZ2lVQ3FQvctlbCOcJQFbi.Ggjlv4y0BMHIZnTNGLbC4t oOfISLjDLHdfRz7S2uK3Ay.Gvd0wLllS1s58aKkzrJ647WZMZkf3cC9YSugPH0JVgnw8xjuhvuKO fEo1P0K._Uwjb.f3HhNHTpKQ1J9vstozA855DU3VpcUJLdc59NGIrVUFdI9mvEUEQhell._lHiMi 9p6_sN0.zzAFMrliE0mKeqpMirndwXFi60IDqZNVb15ysVc_ygxS2OS6KsBqPOSJBWOFgUWKvuB1 nMRJKxmUEO2zp4Wf1hjGr2yBOgReqwErXL07dMKX2i9s.O8.R6m2jwGkwAb9tCyEa0pfLHMyDv5q KeGrc85GZqVHRqlkFQnJ3yCiGlftmQlgDOygnM.c7tGx8WhAoLgKHZ_gA931RkflI6p1Y6aCk3cE boRyh2JxKghqyVhceFV8ulVg1Ws4HErccxE.Uk9wVWpg0O1hc3LK6hDKnThhoW_W5qCDXZcLOZhA UB2S8I7USm.LcAs39jsFegNmtqh2oNUi4sy0jk59KAoQATJ01tkJvbcHt5eYsrdviNTpINIFUXaJ X23LiGK37PrE5I2mE85yVDNEYVRwaBe8hQV1z3g__nYRCHg63MmOUMQvYVX9fsHJt94VeYsiL59S z_x.tqcpxtQL0_xXMX9oFCuq4c4S5o0uBz_9GIKRr7IE6_0qX8uOHaCF6bXh1MjtkFwRbbbJi4cP jQwUNqWIm_MDPoYbyGk2UvWLRL252IFwwHXDNyS3hzWQDR5re3UOWVvWR7tuO0y1SN5EhEuwIqzB VpJpc0SHlrt2OLHOCTxBC9jeHGWd72KwVezR9z28ahTfx2wo0e58ohfCDJdllg2TPXXMHphSAsjg qnwdJM_zLc6eY9hjxtoEyZnsHXh4E7tZjwpGb8NznMxeRQrN1IqiRjzlB2CfAZlYW6dA.gTBmNJL 8jmvmQov0zh8jrgSKChh_fU2tYDLrvLo7iFVjgE0ySrINlltNvmVHs0AiTkW7GtueaHc8fSapd30 2bgy9JVIhtbQ19YhwQMVqWm.bjIrinVifrJV5XzcU7szUkaBs6sXeDjdYpLgYY4A7tSdHK2QKbqr C3zRGuV_0dCV3u45j0dMG.qI1upHqo8PLzDjT5zmzEkHLnz06j5U62MM9QQjuYUgMRVrm20NaD5k BkFSokCsXkYAlzTDhNhOWeaU1IbCVUZBYk.sw5xiqy858RId5yXbWXCsQjRJEUu4va9CalYuQyai p08nKOcl9LHV1Ag2p6noFXYuY9Wnuroffa_qaWYch3y6D3tJ_mM33iRDJGyNXPSgkXHCiEUgRd0P ROsAB4I_dqB1SsdHeYjL8TuA2RRQwS3ul0dSQ4JnJyXT11DvNEGRD6oQXYEo02Ysjn01mTsL71qC Ki82fUXWZ4rkR6TBklKKITouuPXV6ZT1.jqegkV0k3WQYn2tS_gG_UwnKQg4cNoOydUzR0b2K.K6 Pj09JjM8AfI8eym4pmTlLN5h.tfhLs5R.DnZt25f1JDApoNYO8DZsvgu6_x9FW.Dmn4nXWjMIulW Cdtp4ruSTBvli94fI_DSnIrCXXjqIPivMt4MYuhVqoAQXsdNq8S9bFj_AKfw5FIUxtPtrNPcGUTP O6xFCAqP0cz5AX3rziiBxdfaBiz875x09rIRw8mFIRjVGx5Tr9Bsm0d_LNTAyyBvtVpOlo.8USjO YHZYHeZdlkhmNL1dyyGy7BktDNQanCu5nF463V5iZ6ApyrHvW91rShRPPJlGDyqAVJFrknZ6uNjI VLCdKCJbQrQP_KnE_U7cjN79hHz1Lc9qbLuWvAx8k1DhWc1d.KcDKV7J7NgsttN0Bkhtx2sugwBp 4f7Cl6.9Icd9Wh9KKbWZcq6lCWRJIXCe31qRluX8roI_k1fjbS3qsDke9kGJmUDSNVz3HkONkIa5 CNgQ8M4jXEEmrhveILjhz3AFGZQw4na4nnCfgJK_DROFPFe9oVTQm.aNSVbxp3HcU3cO2V6gLMRw xUSOoR0Ghp07T6yq9ZRbJXGLiZREYRH6URsWzUvjUzoWIrbbIPupWdWXL88PbQv4cTcZyGXcjpOd LvFPQW5cdGKUwBipgTqEFSBKHhzDpsTMbSRoC8lK5vzD8abZkj7k4GC4.PAMrmFZb.j1TDDwhhuU _aj5ubZARUjJi3mFjcNp3OTY_0OWvctLIF13eCmzV41EaWXAHekCOzcYreDsjPLb689xpn3qzb0Q p_vypr9uyQmq9gRx.cRrbDU4QEhR11cEAKG6zl6dc41tMD7daJ4FKcpdUb9aGbHgEPNTnl2oK9rT X_GB_UV64r.I4IDacd.KfbkeNhPAXPmVzRC1ehTtlrbMTENialJc48Maag7iW_pnwzGooUu8oJ3Y amGfi182KfS8wD32ZW3D5zUYLZeOR77znfGsn9GEcpBdYFkhCvdtUUlIQKJWr1BKWN66TIPXPTZl LehESH8xxCHeNIWkcebaLK8LtsamsK2bGtteV1tIDBlgdkHUToy_BJO1JP0C3CPsVPsJS6GW0LFl 2TaMrYfGBcaqhzM9muCCTnfM4kj4JDxFGne_Eb3pAPNcybP7WYiEoLfRDcof0eb0MEg-- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic304.consmr.mail.gq1.yahoo.com with HTTP; Tue, 4 May 2021 17:17:34 +0000 Received: by kubenode542.mail-prod1.omega.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 3916381615388a11c064df3a50eecc99; Tue, 04 May 2021 17:17:30 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: Re: FYI: WITH_REPRODUCIBLE_BUILD= problem for some files? From: Mark Millard In-Reply-To: <3FC6BCDD-5865-4B5D-8238-3EC38AD4E78C@yahoo.com> Date: Tue, 4 May 2021 10:17:27 -0700 Cc: FreeBSD-STABLE Mailing List , freebsd-current Content-Transfer-Encoding: quoted-printable Message-Id: References: <35482701-95A3-48B2-9A8E-B7E0092119B1.ref@yahoo.com> <35482701-95A3-48B2-9A8E-B7E0092119B1@yahoo.com> <43F20589-A7C7-42FF-9020-09CEE037D1CD@yahoo.com> <91F820A1-8940-4246-A20A-E62685F50079@yahoo.com> <27B5086A-7C98-4AE6-885A-CC7C7BD2F64B@yahoo.com> <3FC6BCDD-5865-4B5D-8238-3EC38AD4E78C@yahoo.com> To: Ed Maste X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Rspamd-Queue-Id: 4FZRNw4BSlz3nFm X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.50 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.68.205:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; SUBJECT_ENDS_QUESTION(1.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[98.137.68.205:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; BLOCKLISTDE_FAIL(0.00)[98.137.68.205:query timed out]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.205:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.205:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 May 2021 17:17:42 -0000 [Just adding readelf -S info since it seems to show more.] On 2021-May-4, at 10:01, Mark Millard wrote: > On 2021-May-4, at 08:51, Mark Millard wrote: >=20 >> On 2021-May-4, at 06:01, Ed Maste wrote: >>=20 >>> On Mon, 3 May 2021 at 22:26, Mark Millard wrote: >>>>=20 >>>> But I'll note that I've built and stalled py37-diffoscope >>>> (new to me). A basic quick test showed that it reports: >>>>=20 >>>> W: diffoscope.main: Fuzzy-matching is currently disabled as the = "tlsh" module is unavailable. >>>=20 >>> I just looked up tlsh - its "A Locality Sensitive Hash"; I presume >>> diffoscope uses it to infer file renames. I believe the warning >>> emitted here should have no impact on the output we're looking for. >>=20 >> Okay. >>=20 >>> As far as the utf-8 issues go, diffoscope requires a utf-8 locale = and >>> I suspect that is the issue. If you don't have LANG set already, try >>> setting LANG=3DC.UTF-8 in your environment. >>=20 >> That is not the issue for the UnicodeDecodeError: >>=20 >> # echo $LANG >> C.UTF-8 >>=20 >> # diffoscope /.zfs/snapshot/2021-04-*-01:40:48-0/bin/sh >> $<3/>2021-05-04 08:49:21 W: diffoscope.main: Fuzzy-matching is = currently disabled as the "tlsh" module is unavailable. >> $<3/>Traceback (most recent call last): >> File "/usr/local/lib/python3.7/site-packages/diffoscope/main.py", = line 745, in main >> sys.exit(run_diffoscope(parsed_args)) >> File "/usr/local/lib/python3.7/site-packages/diffoscope/main.py", = line 677, in run_diffoscope >> difference =3D load_diff_from_path(path1) >> File = "/usr/local/lib/python3.7/site-packages/diffoscope/readers/__init__.py", = line 31, in load_diff_from_path >> return load_diff(codecs.getreader("utf-8")(fp), path) >> File = "/usr/local/lib/python3.7/site-packages/diffoscope/readers/__init__.py", = line 35, in load_diff >> return JSONReaderV1().load(fp, path) >> File = "/usr/local/lib/python3.7/site-packages/diffoscope/readers/json.py", = line 33, in load >> raw =3D json.load(fp) >> File "/usr/local/lib/python3.7/json/__init__.py", line 293, in load >> return loads(fp.read(), >> File "/usr/local/lib/python3.7/codecs.py", line 504, in read >> newchars, decodedbytes =3D self.decode(data, self.errors) >> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb7 in position = 18: invalid start byte >>=20 >=20 > Well, the list of differing files is huge. But this seems to > be .gnu_debuglink content for the area it is in. Specifically: the last 4 bytes of the .gnu_debuglink section. > I'll note > that I did installworld but not the likes of distrib-dirs > or distribution this time. >=20 > This test did buildworld to two distinct directories: >=20 > zroot/BUILDs/13_0R-CA72-nodbg-clang 5.13G 118G 5.13G = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang > zroot/BUILDs/13_0R-CA72-nodbg-clang-alt 4.28G 118G 4.28G = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt >=20 > and installworld to 2 distinct directories: >=20 > zroot/DESTDIRs/13_0R-CA72-instwrld-alt 1.44G 118G 1.44G = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt > zroot/DESTDIRs/13_0R-CA72-instwrld-norm 1.44G 118G 1.44G = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm >=20 > Previously (armv7 target) I had built, installed, rebuilt > to same directory (after clean-out) and installed to an > alternate directory. That had gotten only a few files > different but I do not know (yet) if it was the procedural > difference that made the difference. >=20 > Prefix of the list of different files this time: >=20 > # diff -rq /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/ = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/ | more > Files /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/[ and = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/[ differ > Files /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/cat and = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/cat differ > Files /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/chflags and = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/chflags differ > Files /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/chio and = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/chio differ > . . . >=20 > Looking, aarch64 seems to typically get a back-to-back > sequence of 4 bytes different in native programs in my > builds: >=20 > # cmp -x /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/cat = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/cat > 00003bd4 1d 65 > 00003bd5 eb a3 > 00003bd6 bb ca > 00003bd7 8e 1a >=20 > # ls -Tld /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/cat = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/cat > -r-xr-xr-x 1 root wheel 18448 May 4 08:55:01 2021 = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/cat > -r-xr-xr-x 1 root wheel 18448 May 3 23:16:36 2021 = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/cat >=20 > Sections: > Idx Name Size VMA LMA File = off Algn > . . . > 25 .gnu_debuglink 00000010 0000000000000000 0000000000000000 = 00003bc8 2**0 > CONTENTS, READONLY Section Headers: [Nr] Name Type Address Offset Size EntSize Flags Link Info Align . . . [25] .comment PROGBITS 0000000000000000 00002c70 00000000000000b2 0000000000000001 MS 0 0 1 [26] .gnu_debuglink PROGBITS 0000000000000000 00003bc8 0000000000000010 0000000000000000 0 0 1 [27] .shstrtab STRTAB 0000000000000000 00003bd8 0000000000000100 0000000000000000 0 0 1 [28] .symtab SYMTAB 0000000000000000 00002d28 0000000000000ea0 0000000000000018 29 96 8 [29] .strtab STRTAB 0000000000000000 00003cd8 00000000000003b3 0000000000000000 0 0 1 > 00003bd4-00003bc8 =3D=3D 0xC Note: 0xC+0x4 =3D=3D 0x10 (the size), so the last 4 bytes of .gnu_debuglink > # cmp -x /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/chflags = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/chflags > 00002208 88 a1 > 00002209 e6 40 > 0000220a 60 94 > 0000220b bf ce >=20 > # ls -Tld /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/chflags = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/chflags > -r-xr-xr-x 1 root wheel 11440 May 4 08:55:01 2021 = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/chflags > -r-xr-xr-x 1 root wheel 11440 May 3 23:16:36 2021 = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/chflags >=20 > Sections: > Idx Name Size VMA LMA File = off Algn > . . . > 25 .gnu_debuglink 00000014 0000000000000000 0000000000000000 = 000021f8 2**0 > CONTENTS, READONLY Section Headers: [Nr] Name Type Address Offset Size EntSize Flags Link Info Align . . . [25] .comment PROGBITS 0000000000000000 000016d8 00000000000000b2 0000000000000001 MS 0 0 1 [26] .gnu_debuglink PROGBITS 0000000000000000 000021f8 0000000000000014 0000000000000000 0 0 1 [27] .shstrtab STRTAB 0000000000000000 0000220c 0000000000000100 0000000000000000 0 0 1 [28] .symtab SYMTAB 0000000000000000 00001790 0000000000000a68 0000000000000018 29 83 8 [29] .strtab STRTAB 0000000000000000 0000230c 000000000000021f 0000000000000000 0 0 1 > 00002208-000021f8 =3D=3D 0x10 Note: 0x10+0x4 =3D=3D 0x14 (the size), so the last 4 bytes of .gnu_debuglink > # cmp -x /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/chio = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/chio > 000050c4 6b 3e > 000050c5 08 ca > 000050c6 7a 2f > 000050c7 5d 64 >=20 > # ls -Tld /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/chio = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/chio > -r-xr-xr-x 1 root wheel 23728 May 4 08:55:01 2021 = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt/bin/chio > -r-xr-xr-x 1 root wheel 23728 May 3 23:16:37 2021 = /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/bin/chio >=20 > Sections: > Idx Name Size VMA LMA File = off Algn > . . . > 25 .gnu_debuglink 00000010 0000000000000000 0000000000000000 = 000050b8 2**0 > CONTENTS, READONLY Section Headers: [Nr] Name Type Address Offset Size EntSize Flags Link Info Align . . . [25] .comment PROGBITS 0000000000000000 00004298 00000000000000b2 0000000000000001 MS 0 0 1 [26] .gnu_debuglink PROGBITS 0000000000000000 000050b8 0000000000000010 0000000000000000 0 0 1 [27] .shstrtab STRTAB 0000000000000000 000050c8 0000000000000100 0000000000000000 0 0 1 [28] .symtab SYMTAB 0000000000000000 00004350 0000000000000d68 0000000000000018 29 100 8 [29] .strtab STRTAB 0000000000000000 000051c8 0000000000000363 0000000000000000 0 0 1 > 000050c4-000050b8 =3D=3D 0xC Note: 0xC+0x4 =3D=3D 0x10 (the size), so the last 4 bytes of .gnu_debuglink > For all I know, some individual byte(s) in the 4 might accidentally > match sometimes. The addition offset after .gnu_debuglink's file > offset does vary (0xC and 0x10 above). Specifically: the last 4 bytes of the .gnu_debuglink section. > The content of those differences do not look like > file path components, for example the 0x08 byte. >=20 > I do build with: >=20 > # Avoid stripping but do not control host -g status as well: > DEBUG_FLAGS+=3D > # > WITH_REPRODUCIBLE_BUILD=3D > WITH_DEBUG_FILES=3D >=20 > But that was true for the earlier armv7 target example > that I reported that only got a few files with > differences. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-stable@freebsd.org Tue May 4 17:32:00 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7D6E05FB748 for ; Tue, 4 May 2021 17:32:00 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic308-55.consmr.mail.gq1.yahoo.com (sonic308-55.consmr.mail.gq1.yahoo.com [98.137.68.31]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FZRjW0LT9z3pL3 for ; Tue, 4 May 2021 17:31:58 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1620149517; bh=ND/hpKdx0OSW/CqhlkX3ybbEYvJsDOrYeKwxZj6OhD+=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=rDOvBDcWbG3NToHu4av64s8KXN4EraiZ1uDsvX1yrCeQXMpLKgdL9+Q7iuvZfMM0AKh8qvwHN1/UungdQ+94Bxu/YUVh5K0nxbuk5Dg7dYzSaXRnPZOs3A83zhghVNFCuxk38C5TNoreElpIlgFJieYuTTKMBjHP7R9f2fC7uLXEk+5M5KK+2PwYVD3rMlkvQqQGarPCvrKHPJ0f1K2XThIRnF+2/9v6eqYXBVN2t4asYq6vtBC0a1xZ9yo1AEeJS9+yoxntyT4hUYG0rioNdXW6uvhobrTeSlLhbtzGxsr7Kzd/CHrEE2c/6pYRSPsEHL0yB+xJgP4nN8rnAy5fsg== X-YMail-OSG: THc9opkVM1mH71BplFVrdO1UNVqrpaPqVhr33S1fdkf8CFSm9TXiC76GJsk26g4 65IYeq69lvwj24SHdC_Gx8MuxrqDgSoC4GKY4XFSnc0g2EFV22d64RwHVoqA2SONcGnBrERtpKDG w1MpsB.0kHFqw4XvXFNMVgiiwQ2gGBTIobrg_BcSIJF9u2r99EyXOASqngr3pmc4mWfhyQbNVlZM VDz.s0RaA9zOK33FAwwhW7qvbK6PIv7F7CxMokzS6zZ5X2FuffWEQx3zinHIGhPOlxxJToUm.kAh IBVgImKDoF7kPnkuCwp5ISyvvIF04OHJbNxghX0eHZNXmt1cLg0FUIgTuNV38gj9pu79vQhETI7v foCpe5z6soxjVl.9LtzdvvTk28VJaNcVi4Xok9bUSV.QTqjaKfORqYKdr5TB_XcTNEX_N2eGlFTA 3EV9Nqn7O7WZR5hZfWHusq7ftbaFuZ68wr7CfEcCEC0zQwgMTjeHD9dJLOjHK9ZMuc6yduGwznuk j3xUtaJilTcolQECcbMWS9tGm7hRLZUlXM832RH1BtLithlBgtK0IKb5N2unWBUYyRE4BhSMenUC J2snLkMhwNFNt811IDsAUZG1vXBhLYSR.sf7JnmJmeOamgVyYcDTZhKb.nnds0p4JXBMbF3baDCH 3Z5VLC_Kpi9rDPIPlPZR1hp6SamzKPTCoFpVa7BZ8rjZTPMzkQ6smvosSZqf7wioxeh7CFtDlLyl pP5RA7qVF92crSZX7k93KijN.QXfSNpzef5e7b0cLaZIzCeMlZTApYwrE.aD5fetycKSvQcWDeLY wQyoKFseiRxFod_L9ML2DPzr1lFB0TrwKtwhZyyWv11wtdMg0Iy_Yp_eYqMi8O9CPLLd1YsFL0ko VSqeVCgnesBh6uDEbc1rhINmHOEB.m1A0Q4.dWxT9LpIFrtGZu3tL_XNUvn0yKvtDEPD_tPt5cas yckl.Tycl2.pjjUgEIUeg1Jn_bpVHmk1KzgpYpQ7wG9DZalIvp53ESa6LFZj4L1t4NGdAwfeNhDK 7O.wSPqaHAdaTw8MQmbCjGarkBNT0D1qwm0RhGLZG401W2OATw5OKgEH2lBpzp6J4.2w9waLGJ8y Xgo2IJ5n.ExDZ.3p0WV1v_diT8B5GPs18q9umPmmsDQBV0K38RrDW5nRo8UAACd2ifzbMBBXSs4l 9H8hangCfdcF5WzJKPhb.iusxZ0fOq2qeho.Szz1pPN8lQxKnj9iTlNEMl.uJZbJFr38STOUUCcl Tk.mMSMxlqZCobBs2L6_TgoSHHGXSAJEnTVpvKy2ZssAwb_aZ5o5U.8CDgMJW6MbMUjU5SBQWK4E IbZyZTzEeCBhBKsFPm.mcI7yTulpuh79PI2RP5OkyDVXAVxopi.7uvqiwhGSRce_AKmtqrL3qkvm w9dyk.3l.z.SGATkoC34uXiiu7S6eCK95PJm0nlut6nl2V_J1nuv7NB04.wAsapKHE9q0cEdcXQ2 1WJGkyI4vVMBdDiZlpmw71fepRrC3cSxOwv9D2vP8JTsH9oV1IdpU4b4DVUvSEmoMsFili5F3Nck 5sY2YyeuPJA46oPTFFAju3hFtRmJB7iSAY31RHcCIPM.jDm1eW.2xDJ8P1mXQVJ7gFdGuvFbhl_w rqaAHspfWt6OTwWYNmtV4FNf6hJt14KPaBEjwOfs5_txPBulIKR0WQ6rDTnYCAYFb8B6ricu0LtE 870S1s7cp2Cazkm1._Q.Fi81CnNUa_HtFw0JCZlvi1ULAGi609oPPBeO7XK0SgfLCDHiUJH6a9V2 fjMrFMBYTGLLqCDfNRfsGzoTzT9HZW1fO_BC6rKv3h7lkwMgHNGfiUhTbfz6LLq2RYWqWm4IhJAW bR4n05LgfunMLK.Qw2hVzOef3fFw9AY__JeUVSjwppAxBu3wE2QwcDE2Kam8t9VI4q7ZmNbInEfN XeT5Mydjs3LiMIARmic6mLGSeEwdR42V4AhVeicX1dJadcuWumWzmIBYT8unRuMTc8jmeUz5GNPS Mv7Zz3SW3sPW4JfHG3ViQcUQP2ZAVHpgkgJ7gYri1Sb0XyEMNTpaQ9E_zv7VOIsT3auS_764L.wM byFVTDI5XoEjmxR6zv90dawXL_KcvDOPbh3ilOCV8MHcF3jJqYsRVIIs_wIR1_RbwA_MOxwf7.hS D4uTIQsBQpzSBWtge0.e_XNyHj0zcSXYWpS_CwWQvgx3K9s26Z6fHLchDb7VDTN6hjcc2RnuOX3F UeDdIS370fNCbmuCjGSbM9ScGblShOpRjL45zSNPiyytYAvYOP0abIRYyt1VJjT7NiJMR8hsDfY6 8jmq2YYStP3TXAqR.GaUz_xpS13OADmwFrn.90jMFt6DPXPATqbcf3BE7yDLk77KK8vUR3WyQYBv _pl6gqF9aPD20DRcEVriGZlFlRGsRhpnbGzZmipSIY3i.Z7WawU5ITdje7BHuiTW.UQwVe1vgtlW TygesMgMaKZXZF4KUiL2Z1hq8TUu.5uwWjolunFK1i8uh0qMQ8R_a.9HrkZtAPQAu64sJFXwy__G MEkF43Th5oCOQ5Y3Eyug_lhcNsdnXGzjFmL5o_XXjH9RxuiHF11OAbCmPCE1O5dEpfIy4RAnRHUa rY0kTQ63oXt23axf1StQdt7bXqxzapcrNwLUBcpTBx44RKnEnErTuQKNnowyK8MkEuuppKN9EElb tTqO8EfF_MI_25gPWjbjtEqzNbyxgEF6pDZ8YIstArsMKJg.G83H5bkL2y0ocqSvvrTpQQh2.hxP 7T1GwEDOpNqp15z_vua970_9STJOWhx.Ohh.DAKafPk14M4ia X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic308.consmr.mail.gq1.yahoo.com with HTTP; Tue, 4 May 2021 17:31:57 +0000 Received: by kubenode548.mail-prod1.omega.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 77048009a4be959ffd146f7875073e75; Tue, 04 May 2021 17:31:55 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: Re: FYI: WITH_REPRODUCIBLE_BUILD= problem for some files? [Ignore recent test: -dirty vs. checked-in usage difference] From: Mark Millard In-Reply-To: Date: Tue, 4 May 2021 10:31:50 -0700 Cc: FreeBSD-STABLE Mailing List , freebsd-current Content-Transfer-Encoding: 7bit Message-Id: <2E86A8E9-9F0E-446E-BF80-5FA3B8ECB197@yahoo.com> References: <35482701-95A3-48B2-9A8E-B7E0092119B1.ref@yahoo.com> <35482701-95A3-48B2-9A8E-B7E0092119B1@yahoo.com> <43F20589-A7C7-42FF-9020-09CEE037D1CD@yahoo.com> <91F820A1-8940-4246-A20A-E62685F50079@yahoo.com> <27B5086A-7C98-4AE6-885A-CC7C7BD2F64B@yahoo.com> <3FC6BCDD-5865-4B5D-8238-3EC38AD4E78C@yahoo.com> To: Ed Maste X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Rspamd-Queue-Id: 4FZRjW0LT9z3pL3 X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.50 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.00)[-0.999]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.68.31:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MID_RHS_MATCH_FROM(0.00)[]; SUBJECT_HAS_QUESTION(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[98.137.68.31:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.31:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.31:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 May 2021 17:32:00 -0000 I probably know why the huge count of differences this time unlike the original report . . . Previously I built based on a checked-in branch as part of my experimenting. This time it was in a -dirty form (not checked in), again as part of my experimental exploration. WITH_REPRODUCIBLE_BUILD= makes a distinction between these if I remember right: (partially?) disabling itself for -dirty style. To reproduce the original style of test I need to create a branch with my few patches checked in and do the buildworlds from that branch. This will, of course, take a while. Sorry for the noise. === Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-stable@freebsd.org Tue May 4 18:14:14 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 97CC95FCAFC for ; Tue, 4 May 2021 18:14:14 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic307-55.consmr.mail.gq1.yahoo.com (sonic307-55.consmr.mail.gq1.yahoo.com [98.137.64.31]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FZSfF514zz3rPs for ; Tue, 4 May 2021 18:14:13 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1620152051; bh=GmPNLOvMfHVGBSRe1VVRcsmnoYcseLJp8+XosYnOH8T=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=ERyzCF6jF5NRqhaW0mzjSPZPRPMlMjzhSE35UOGE/lH0ZLoSsGSUDFHJu0sIemyjBHMnvMthg9Fh1PQlF3eToHJWmWFrYyPHgpPJNUgQeneEOESL010JzDGbDo3dMo8kQ5dugqH/1x85ouePKRXOQqruOSxlunv6QLtKWg6zNkr5Iwo7uejCxINRfRX16e2nwC4sA2sUushtie3hdImkFIyWm7NYMZw2KOYhUKTAMZTwn22RjgGsij3e9w5GwMZve8NThv8wjfDZUomKIT2OvTRFU7JUEK1AZrCEg30yeu7vs8BOKZDm0alsnRqndSaZYPUt9hzYJWZJW60Z8EXzFA== X-YMail-OSG: lGMCIYsVM1kQIXqZ8pRxR_ldKm5mGJEVyl9huDai2XYxoM7laIhznsKPy79GBqT dB7TGhhZUHCv9L0068mGysFagKbMPs1F8o9eo8heU6P1WBm4pyEdPDTeMl7RsZUYfT_zLuiPDvfi .fcjOzz41yBUdP5cNizix2A68de5Dvg_DwPagXp9cKeEJ70T5jslmkz1fu9hkGH12x2QpLCNNeIF ZST9IY73TkTku4UweuJs.NSSpKGkG8Q6PiwNAQ1XHBW1tApcfhWG_EH_uHLfV1mSGA8sOgAzPwS2 DX6bkNcHTdwR59V7iIVAS2ws2LITTXqj6QKydIVqlLiQ3YjEseSwzlzt.aGj2bmCVatKKtL298pr 8xqceTp103qKXpLJ2ntjhnsq4POwIiGlNkexPXCAA1Ewrx8iEhE6My3.TFXlOxL5rcrzrP_oDPO3 KX5Yqz9UHS238RdfCdWBfdhiqYzE6cwtwELiWx3rWw0oUQI2zkKlnwqYuJlkaBHoZmLNaab7UHqm g_yNu27pbE9N46fEat68lRV4_pOdVLfzMVSjjIzaBftwHyj4roz.bUBNGQQrwhWjrVbHFTQPdsdw rQAwtmfQH3ipFIxOPXAUCiIjc3yTKioCYEXTbqu3IpdEB9MxqFk6IzWvQ.eI3ipFKP_0Ce0spCQt kbFeEVpbKtSW08TXqZ.l66a8FreRgI4i6WASRNXRfwR133nck5OEOxEVbLBqai1aKYADz8HsyPFo lAzv3RU_YkhN33siDVpSC2Sj8J0rDhGAV5MaY8WIwqPW7up.wiycE8VHuYWrxSjuKCzn49HhGSgl nHzyqUhAbK_SbzigFbn0r1pA4P4gp5LW3Q95JQisqN6uQAMWykJwZp83H4nRZ_2woQtGWqUTyIVv 0B2QJnoZ5oy1xoi8uh.LOJ9p5ml3X0rEcwQ1fLFgL1wpidu6mi5dsJt_uuex_uFOYUCld6KqnWzU 2vV0nErsn3MUSqretzhperNTfsVpIdPP3cEx.miIGvzAXKTaw.7UHSYK28aA1KUC81v_Av4rtd5I ivHOaze1lUk5ks_tT8GRawjZ3zeHGLzaiBjj824BNpz6uUTovmU742QON04.jZAi8DOiZNXrEVqa OTQr.o7sd5JaUl3MACQTePihGB_E8VMCfHd5BxxG9ZIRaVwUyycTMLGs_7GSq5rexiU6ZG1bu3_K d1GgYDjRMHk0bF01llWnL69qNYl2sh0rL0z3xWoJFATPd.45s0.QylewtB7veAHtCbSlnfI7Lq_M OB.Ce2s7EO14BruWgllvcW8nNjPkec6E_.gXRYlltsGHC8LuBPicPwW7xHy8KdeGkSzSLCTUsPof k.Tjg3t4GxVqfjsdTjAcdDuV_CqNs7SyfMKkL4uchFTvtqmJZVGkuYU1S8K9eeJUzjmaht0zDTSg NMa3BPfehOdiUHLkoMJNXgBvxRpRwIwzx2Q.YQMLLswHd74jfqF.qRnfQLCzjgll6zGqr3y2E1Iy 6zI_sYBWaw8uwu.EO_1dRsAHQg8nmfjOw.QBBTfFikNYuXbx4vodDjs2An.CWVAp5WGC2e3z1jLi kbZJX2X_rJHdeaNVbzzyiSuOBkNyeZAndNfKdj.czAvadti01lIDzNyRNxgSYairkVb8XFpVvcom _j2oPriDhfvHfcRt88saEGm5.okxYAu0XyBL3fHZtB9Y3zL4Hso_YcE379yW7IMDdLYY9Vr2yE5F wh0djQr1YupN4hP7sNmsga1acAr18DwifOl93Oqd.43dRnfLW9WJ9zJh9H9aX9CWfaXJoupQgLzm cEsC62Hk.U3UmqIhpDbdMDO3qE16DeaY3dfLAOPh_9wEJfKvFWuH5orqbbU6IZ0IdlcdVHM7tDU_ UFWYiWBKy64IjxGGugyLVaxM3tpgu_5hf.cnx98zfUNHp_0jpmpD8mD8qruFp4NVEIOd7Zpvfihu X4VGbNid4MDwosu6GX0EL6cBATnzbgZhHwZPNxWLUTlliZOeog8aX_jIwKK7O6fTwSocWRzdTvEg IF1kzPUlR87GxvFlQpVW4SwhExXx.QCD3kAXWU1k9W6odIihZkVZwqYNX9epqhb6uKoRTDjj1wmU P2xnoHz_0Kuh.NSfLhobbhrwMtQhffC1HhKPEfkx.QdVVFISISxGnlkTOoy6ss6LF7IfmmM3Ou4M 8dTU7ELoE5rkoyWFnh4H0IqdB5pgVGJAsZPFblVKF4zpAH_lwc9X9hVD3GM7PU31N9vzn3NuWGEt 9n.078_PgC30Clx9BY6xwFPQHe9hqdR3dLFYdQN0_L8vV9zSlP52PAD.TlXidSNElDEO1qFgna3j 19PiUFgDUTLcLrMbYYKh95iEUyritYhFm_DExomB1.dTAPLx7dBHMuFy1Qf7PalJpkmocT26oBcs eKngfwHJbzPRQ0pUQWI3xuhYmLsV87PT.d0Bmmm98FDBiYJ4X_OcFnXc.t76n6rwsyz3vnVuNtYE 6ucPsf1__nozt0hLx3rwY4m7QRQWvLV99tXB0sdsh56EozevhhIL_PrKP9hPmbnTe66CZ3in9sMp Rq4byqlLx8WBrzOdcErX1DAQQ6va3VpkLk7uWs2ozqRZw0eOKu0ms0q7SYrmWgybC04LChtgDqEX HuHTADmgpgeFMIMeTlOpZnMTl.p5xfRHfm51H3Nuyx1j_ETwyoCsGyoqFmRDwDoeVKN6vHo6T6IU IJo8VBTnHSzuhCb5ETfH0lHcGAxk.8S_j5VE0yz.8cVMkTdWvvfA- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic307.consmr.mail.gq1.yahoo.com with HTTP; Tue, 4 May 2021 18:14:11 +0000 Received: by kubenode544.mail-prod1.omega.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID d3ba890bf511f0744d19eba011966c03; Tue, 04 May 2021 18:14:07 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: diffoscope's odd UnicodeDecodeError error message: reason found Message-Id: <27291038-3538-4A09-A09F-A2202A2D1965@yahoo.com> Date: Tue, 4 May 2021 11:14:02 -0700 Cc: freebsd-current , FreeBSD-STABLE Mailing List To: Ed Maste X-Mailer: Apple Mail (2.3654.60.0.2.21) References: <27291038-3538-4A09-A09F-A2202A2D1965.ref@yahoo.com> X-Rspamd-Queue-Id: 4FZSfF514zz3rPs X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.50 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.64.31:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[98.137.64.31:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.31:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.31:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 May 2021 18:14:14 -0000 I had reported in the reproducable build list messages: > # diffoscope /.zfs/snapshot/2021-04-*-01:40:48-0/bin/sh > [...] > $<3/>2021-05-04 08:49:21 W: diffoscope.main: Fuzzy-matching is = currently disabled as the "tlsh" module is unavailable. > UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb7 in position = 18: invalid start byte Well, it turns out that the file name pattern was incorrect and only matched one file. By contrast: # diffoscope /.zfs/snapshot/2021-04-*/bin/sh $<3/>2021-05-04 11:05:25 W: diffoscope.main: Fuzzy-matching is currently = disabled as the "tlsh" module is unavailable. worked fine. And making the "one file" status obvious: # diffoscope c_tests/a.out $<3/>2021-05-04 11:11:45 W: diffoscope.main: Fuzzy-matching is currently = disabled as the "tlsh" module is unavailable. $<3/>Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/diffoscope/main.py", line = 745, in main sys.exit(run_diffoscope(parsed_args)) File "/usr/local/lib/python3.7/site-packages/diffoscope/main.py", line = 677, in run_diffoscope difference =3D load_diff_from_path(path1) File = "/usr/local/lib/python3.7/site-packages/diffoscope/readers/__init__.py", = line 31, in load_diff_from_path return load_diff(codecs.getreader("utf-8")(fp), path) File = "/usr/local/lib/python3.7/site-packages/diffoscope/readers/__init__.py", = line 35, in load_diff return JSONReaderV1().load(fp, path) File = "/usr/local/lib/python3.7/site-packages/diffoscope/readers/json.py", = line 33, in load raw =3D json.load(fp) File "/usr/local/lib/python3.7/json/__init__.py", line 293, in load return loads(fp.read(), File "/usr/local/lib/python3.7/codecs.py", line 504, in read newchars, decodedbytes =3D self.decode(data, self.errors) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb7 in position 18: = invalid start byte Not exactly an obvious error message for the issue. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-stable@freebsd.org Tue May 4 20:38:32 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 173386225CE for ; Tue, 4 May 2021 20:38:32 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic307-55.consmr.mail.gq1.yahoo.com (sonic307-55.consmr.mail.gq1.yahoo.com [98.137.64.31]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FZWrl0y1Lz4TlW for ; Tue, 4 May 2021 20:38:30 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1620160709; bh=jdk0Xc4yTYdBCBygFrt4DQvYa3zbnB4pXHkJK5/52AC=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=sku3nEYEPhUT08y7Jv6SlIxyV5T48xpPne62eBM9YmIzyg0KFyvMc9ljj7XCJ2OWtlzNcRSETPTCNEtNOeKx81acfxSzM95yOfbeHcIm6Po6G7Y8qoL2z210H9v9Y+kiIp7xTqMpvQbHfDBSKOgLHBy0u286dpiPpJ0b/PygxAHDsJUMliLoaNFpikPErz8MgISVFhoc4HXs9Mmj7YfphOy6sPkohofz2UF3l1GhAVwxmHAne41vL+WrzDYJuELRVFzkQkyyoHF1vkk3ItPYbsYgsYSS68mvUqEYZe39QF6i47ZJGOtyYqAD3ACgbVNCHEAeEQ3feKSi8stOL8mRoQ== X-YMail-OSG: Yw783QUVM1kAtTfsW8jqy.RUd4B6hCrgxp48NoukvA16gWYKTgPeLH9GsCVtIxU _zu_NRRkjh5DEbrf9iECdjZX7N0Z_dKK1yy6SEuPJ2enERyszlJMl2I1ncEP.LUnsoCJlQAd9LRS YfeazartpzyYAg2ZCC92fOfxmvwMzFUtUz3ey9CSyqEZ5ttrZ.AUUnla3HI9FN1q5VQwygstd2tX 3U8IXSkN6oi5FrJdUiPRNCf9ssmARFRSBVJFaH2C1lz9HgF_dmWI0vFsKnHjR3TqyJnqoLA79XTe zCMAyMtuQxIl.5DbdcBORu9tnUQjVlfj8OUHsP61tlwMD6.k2QXWPJKbEBcbO8OJ.PRcLNTzT0Tf b_4FEVoXFOhTnaU53h17jC1tcddfPXqe_NYYzg_Y4qMRyVAv2Celu57Gsi2qyakJoqxLe0nN76Bd L1PVT5W_pwXuUiTdoXAH83aeW7mDFJy6UD33oG3wer8GOcZGh_vw75BzFbKL4iR_7sa2LGbY3qra ww7iuNaquTmFjgRzkG0wsGNslAjqH_EOclh89Is2AI1.JTw8vNE6P7BlRMT4HnVFQRNgPs241ceh O53RRVG6BMn2r2ChxrRgZwNskvjm7qaPxW2LRXtcRDSobZd9N2efocWZs8Lh8.0qTniB0iUScy8V 2C3bFJGKEjFNQaS2zyM9_5upxeBrXoDgS3Y0XfZDAm.D9SXNwnf2_YkXq73Q_YzfLM3jspncJnfp eEYGZwf8Zm4RYD3dJMhZ8yhYL1AlO9zam6SdzG4juFACXB0Gi1jxEiR59ZAwhUDB26SrJvBV4HUB _ZnKeXuwyUpbHD4o1wKjoKnuORVUKd8brn6_PT4pGQG0UXau8H9W9ZIVcu72fg1syxwuT0BMekf1 fqfXghyhpwhIiuLgTXQasaMNL0j_FPC8UdVYP5SAlu69Omw9.wkQU4b3nKlvwhdzWKdTy1o65FmX U2_MgPwe1T_egeXapakshczAccoc9x7jSfFiUjbCXOVA93vY_ndMELbqLoYS7yeS5I6uLe4gIquZ 6Ap3TmXr1Xnuj16Q50yGgn1dt0FVwO_wpe11C34P54UE8WJBC442iHSAYLq8hprrPcjF2CcksAdp L8fyEqHckLE9vPOTIVjlVV5bqyKHsjehx5TmDSBfGZSrskEh7EjSuICciYvJF0fiUhasOBG93GYM TodGBc39STFjUHQ7CEIjE8KXWhXjcNTyZvjqQR_53VzaN7crltkx.rGfBiXfkzv0LkYCm4x2wt1I Ihd5fLZyHtvT9eJMly12a6RXrwdasVIHrHvhHJ_YXL1S9HjHas5wg5eQlR3.S35_xnTm7y.vTe0T 8bNXb29F9HDA6J1_tU.4C4zh0l.zqSqqoNQ3HSkX_4YiiNVlTLlmkOMSvxHjVw_plV6lbDBAl_ra zYDPNMmCGXWvXT.qFcJ7p1nsZ.My21HX_t6eikN7NW_8BfUmqv9sosCSvxTbA5D0g37bFBg1omsK 8xKdsU2MQjR.dcL.5fl0tLDxtn6pk2BF0c9v86dEwjD7L0GPJgwDKlxGcaxcDReKREDd50Wcuaf7 id5FNvZrWRz_P4cHCmS5jq8HpV4WtBNJoyBfS2_7Hvib4e3dUSs1hy7mn_TKYGL37qe31rg8EWHZ F2YvnumP_w94V0Ot4Vkck3OZ8oCfiMjlVo9cN9jV58V6LPTWXZxZ1tmhax6L75hHiibAzmIB0s0f 28v4_MqNJ6DzKrPABGyzV350100giD9pqZUMgIooeP2bdG6T5mrBsM8VZBB_6VhnDZvEfwSMLdnW lS_iSFtiObYCRbWrcFUyzzHdXoufglTFmffFKzQB.ay7mGgG6TTu.syUU0ZqIw9IjaSrY3P5aDEn 3ywRRTJCH1LqvYmHv2498u5d371gIItXoLvCjYoktgl_UMHBSSOZrNPL6WYVsUvAb5cX78GKqEvk EF93qY1Nj2Id6cKPgHpL1i50Y_2avVsT4sFTgtoTPOTguMcBio0opOrBs1zU97KhWeQ4MMY3V9AP hmfeIOALTzYJVm7cxxXEdS9EgO5WVhWddBoZgtimmMdRsP_qh7Sk.KUwnGOGx7LQjweuAjmXmk2s _nCAmntNIDCOhwU5xTg1y_j2Z3kMzsjO2eiLgl3Hdo0f6E0M.suhKUxC0KKuxcBqAQ_eU.ytSABW n_bTfnamUrGT_RbcaRQYcLi5yr_uMlW2vHutbh77htmZ3Tfj58IkX6L_46eJdKZAPNPA3XIMaoja tElCgALvWrs0yGl97uB3ryBaTgwidA5e2XtWbwpkiDggnAESEFknw6W.vod7STd8jbjmucAZzjy3 iAIk2kk2IfZZT_t0791kyE4d2k8EtEqJH5Lfihckbe7WZ7Jm_ndld9uZYc.cNWWb6WG7M_ADjPTU OKXNxjlAWk5f4xWl3oo_aUCf63.VI58i62_P38xA37QM5USw_d0.ZoS4XND42rS7yQhM.lP1YzAR KBuXCBCT2fkqU7xTx5YrdDf7EnklcSxmcEh7kjVcr_sDWA.vVsEJ20d9TC3o9XL0CCO3sRlvHxO2 zQ9UkhG8WeDPZbmQMth6lnAGSS2sDD84LH4N8cmYYzfp.L2E_vPhg0iNOIpV5UnriWOGI3VrpX45 AV6dcjMKU9Ms.1MLNy4GqCoFkMLgMjLQuEa5uN3befBs6MdvZbhN63ZuoT6w_q68NEX2UbqJ4cEU x_UV_ZTaaEhww4x888S7N2biuSK3I_JFdVCSrcskgM0v29lMcif7zIvDqdvOQzTiqIw-- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic307.consmr.mail.gq1.yahoo.com with HTTP; Tue, 4 May 2021 20:38:29 +0000 Received: by kubenode562.mail-prod1.omega.gq1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 1558a91f3260afdf8c8e457506c5aa6d; Tue, 04 May 2021 20:38:24 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: Re: FYI: WITH_REPRODUCIBLE_BUILD= problem for some files? [Ignore recent test: -dirty vs. checked-in usage difference] From: Mark Millard In-Reply-To: <2E86A8E9-9F0E-446E-BF80-5FA3B8ECB197@yahoo.com> Date: Tue, 4 May 2021 13:38:23 -0700 Cc: FreeBSD-STABLE Mailing List , freebsd-current Content-Transfer-Encoding: quoted-printable Message-Id: References: <35482701-95A3-48B2-9A8E-B7E0092119B1.ref@yahoo.com> <35482701-95A3-48B2-9A8E-B7E0092119B1@yahoo.com> <43F20589-A7C7-42FF-9020-09CEE037D1CD@yahoo.com> <91F820A1-8940-4246-A20A-E62685F50079@yahoo.com> <27B5086A-7C98-4AE6-885A-CC7C7BD2F64B@yahoo.com> <3FC6BCDD-5865-4B5D-8238-3EC38AD4E78C@yahoo.com> <2E86A8E9-9F0E-446E-BF80-5FA3B8ECB197@yahoo.com> To: Ed Maste X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Rspamd-Queue-Id: 4FZWrl0y1Lz4TlW X-Spamd-Bar: - X-Spamd-Result: default: False [-1.53 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.64.31:from]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_SPAM_SHORT(0.97)[0.975]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SUBJECT_HAS_QUESTION(0.00)[]; SPAMHAUS_ZRD(0.00)[98.137.64.31:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.31:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.31:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 May 2021 20:38:32 -0000 [The first buidlworld is still in process. So while waiting . . .] On 2021-May-4, at 10:31, Mark Millard wrote: > I probably know why the huge count of differences this time > unlike the original report . . . >=20 > Previously I built based on a checked-in branch as part of > my experimenting. This time it was in a -dirty form (not > checked in), again as part of my experimental exploration. >=20 > WITH_REPRODUCIBLE_BUILD=3D makes a distinction between these > if I remember right: (partially?) disabling itself for > -dirty style. >=20 > To reproduce the original style of test I need to create > a branch with my few patches checked in and do the > buildworlds from that branch. >=20 > This will, of course, take a while. >=20 > Sorry for the noise. >=20 I've confirmed some of the details of the large number of files with difference while waiting for the 1st buildworld : The 4 bytes at the end of the .gnu_debuglink section that are ending up different are the checksum for the .debug file. The .debug files have differences such as: =E2=94=82 - <1a> DW_AT_comp_dir : (indirect string) = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/usr/13_0R-src/arm64.aarch64/lib= /csu/aarch64 =E2=94=82 + <1a> DW_AT_comp_dir : (indirect string) = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang/usr/13_0R-src/arm64.aarch64/lib/csu= /aarch64 So I need to build, snapshot (in case need to reference), install, clean-out, build, install elsewhere, compare. (Or analogous that uses the same build base-path for both installs despite separate buildworld's.) This is separate from any potential -dirty vs. checked-in handling variation by WITH_REPRODUCIBLE_BUILD=3D . My process that produced the original armv7 report happened to do that before I accidentally discovered the presence of the few files with differences. My new experiments were different and I'd not though of needing to vary the procedure to get you the right evidence. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-stable@freebsd.org Tue May 4 22:59:18 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AB1216264D8 for ; Tue, 4 May 2021 22:59:18 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic307-8.consmr.mail.gq1.yahoo.com (sonic307-8.consmr.mail.gq1.yahoo.com [98.137.64.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FZZz95Y44z4bdf for ; Tue, 4 May 2021 22:59:17 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1620169155; bh=88eTLRIICdiwDZq+5RahK8UZqp7o5N2iGr97rbHF/aO=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=jsJ8CXsHROdwyGrgOLlJpojoQtaUSsoKTtGYFfm6ZVBOPiHty/Bqk+vpczrMwSR0hG//UTLx7NCZnqdLGfYZSJfAyLF+hNRKNpoD5Sa+RzWrTMkMBL+ltmCaCdQHHSJ4qc2SxuUaimqSFRDG2p2p4wIV21LLWNwnVimeezvUrz/6bj3Kmvdp4MukeBYeFHmSa/m657A7aunepqhMa4heS0L+VEoFCbFeXEryP3Ft79xr0PBNpy0gfEfflfKBDtj59akfFbybE8eC4hoJ5/FSaj5rBCnmaw6vISfcBe/QGqBxGK4GxqG2C/wFxft6wgG+q3Y9Z2hcTxkeVITyc1+tKg== X-YMail-OSG: HEu8NroVM1nIB711qg6TmAM6CReMB7K6t67Z0GAG17B54.fBtYBcnDIBNfBt6yc 9sZ_eDEm9KSHf9EPwi650f4HnHArdiEEd926pV1uf2bTk1eV8ShkP_HQ0PeP4awEY.byje..oM5P hnpj1BQSiL2Shsr_8gJhl2xNeLO3lDpwyHsHpMUOne6aqi62028H_LvDKuEcbNdaUnuBv6.T1eR. ikfkRQNX4l6YibGwJHTbB0EJkuKzYff_N0w0FrqqSaeR0ZZzu6HM62FnuCfYyt1zXzVZUwp0_f08 Gvey61dyNvA13sQgHH8BYGuubDpdYGbuftOn72X0O.dc5.PiUGbCW0bFeJRQ5KjfLLt76j7A1vKH KFu3OkCuTa257p7KfsnSvlF0S1QdMXJRwpGIO2j9WM4BNLgd8cCLqUW62pP1HHsSgKjNcmQpAPlg Tl_0ZqO9vJ9kjLFKYglkREQg95Cfc6XHkGWEO4A7LERXUdd4xZd3mjcsrpO0ywQPT2u9ZKrclY8E En5.LxkJkBawev6.sDbc.RLiWe1gHKr4lRHCxQcSaw9vNnHDmWqUmxHa2xS_.zd1tCL76b_uPtJB Bur2xxGqWKscFEvDxjIIZBaL_EI9a_QhKBstJr.DIBsDYMtANo.E4NTMCW7w8R4GnruhKc.8r5jF QUH8OiuF0T3xRh6GKI4CtIkO.5RMKHx5w44Kiyjt7K97c7veu5hgadzHrFJIQcPd.a3UPgo.Cqxw O3Rme4wqY0nLL0D4TVOTEbsA8QblzpOCe6A_WCtcdiZDmy8mNhL2voYb2rL1PipNEH_DYD3PIStR 4EyoAzGLx5QP_DJK7WQRBFGhcqTJcWU45qShgJuxiL42vePP9MCXYrA7nJIQkZcemzGT5hf6FsrE RssGuL35GIEw4UvQrIgwq2kLqaya4.wbhG.Dm5wUinz5N0.lYph.H8AZA5fIr7ZTO4gzFfk_ireK TQulM.L5HZ3_pUWnciFQ4jhKvHX0o7XQgOFz2reGsEA.TiXmLdpnPUbM6EWECY_tV9SKYCGnHJrY gsr19uevoB3ZJc6Q0trHk7AzcmniTb1fJq_J0wSvrePpE6scOhBVWP1p6I4KthcCQzDP1pBpyK2E NvmffEkI6m.Bng.PkmjQ5_Ne9sd4zsLMdcOhZYtu2uUPsO7nFx2N7BP2bu6w5subHM3c7z.sXs6L wl6Dky4WihpbI9LyLZDQ63fYJbwvac5RgbKPpzhly1hVIjTmsjgoXSqYanW5jcKBp59.SoA6LsCZ PG7njearvSbYOjHrmsWLn6VBnjr02Nj5Bmeb4sf3NJ9eY6BAUzAY9RlpseqEeZcQLvM54tCZTfZK w8mbT8aEcX4W8Iyh.pDs_ACG2NeFp1jTQm4cimNQ_oO1YPX9hNd_WLcqH2XZMlxVcfJ8pvd26oxN 11n_A1DSPsfVXXvDsHjd3xc59HB_WG4Fx4n1o7cBkj8M8wlpQoFgZ0f4A9g9RGRRVK8CGedb75xG 6wM.67ifm2JIQe277LM.43PJ74ibvc7cGmaOGAqNnvQp.aP5oMX4Ey6pGWEWH3N0tfPkOIJk8rW_ oLYjsEeMIN11zzdNU.TEW_ouR2JWNBds_nT4r6tBI5z5kd3P3FYOZYxGGbDiw8e5qsctWzPQv53l 6.PgvIUo569xwnIOovvpt8fxXxKU7OyD534Ht2jH7ls4JG9vu02ag8vvANzbYTArmYlBdCr7nwOK b.CEmZwJWdXCSQSDVJfws7nMVCp7p6oCJ2WwsUtQM7z_YJpn72jQxQYfHatGpgo4cWopagENcIY1 vE6PYHsh1EVz6L5xLMKzcPUFOpdlzw7_TpB_G4YUS0F66ch_stk_1sbzz0ahqIpNcdJt.eyVEzDT HiQkouLGEyR985UOflB7zV.6eOn6FRFogY2vdynnI01QlZyKQmx3dHR7iM5.EaxFlkQXcau9iXzw IIR5Now3OFKtwJKMz2xmTrTKOc2KP9Km5ZOjFO1tfRXBO1EmO6eHakoU5pGi8KW_4G8Gizo4fn9O iRViAXReBD5CPrX6L8esBcD6n0zzjzUVf6s._UJTIaARmhT2xPcHFlXKRr2v4iWk8TeCvBV6yoS9 DDoS96.xKK4faa5GkwxFxI_dZ1qS5BS0leLpPSq9YtpSjZ6.Ddn4yecaBhhJ0MIrzvT9GduAdl4K uy.1h_6hyeU3FfT5jQ89zS6Pbg9sPV5INq9BxUMu6EhZG0T8As2k5ZuVhPNoqxsk0_RGsGXfYor5 qzoTaqYXBhlRSC9_t0LNuTAySficEZ2NXy3W6n2YlH1rDrCKEIqMkQfaWzZNraQ13BVQqtnH8BO9 bzW7zpfCxfKmEN.ybdg61WW_5loy57jz2T8Mn.KMr8qhlTLoBhJjq2Uy5j.UYTXp1WqgxQw8kmuc HN.K25estKSW.fcKM5v3xMA39x7FRUxF74m2AKS4oFt2xY4ql_nQgDBQPVYasVySkC6DGtT9JAV1 A59V.BIRz1gTeHFnLI0xylFC7NwsaKNnTbv4S96bzESILM7p1vzbav_AGR5YAwBI6GHogth8c86D VMw1GR_SAsuQGJwQlwhn07.sZRXJCipNIcusZ1U3y4qGFQpy9hymDSmga8X4bKM4ffb2oGebkfi2 _Ss33I6deAfrMegu3NKso4w-- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic307.consmr.mail.gq1.yahoo.com with HTTP; Tue, 4 May 2021 22:59:15 +0000 Received: by kubenode557.mail-prod1.omega.gq1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 062cf227df73c11c7a8abcdfc5cffd03; Tue, 04 May 2021 22:59:14 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: ZFS rename with associated snapshot present: odd error message Message-Id: <8335C81D-B83C-42BC-B296-C05FAEAE538A@yahoo.com> Date: Tue, 4 May 2021 15:59:13 -0700 Cc: freebsd-current To: FreeBSD-STABLE Mailing List X-Mailer: Apple Mail (2.3654.60.0.2.21) References: <8335C81D-B83C-42BC-B296-C05FAEAE538A.ref@yahoo.com> X-Rspamd-Queue-Id: 4FZZz95Y44z4bdf X-Spamd-Bar: - X-Spamd-Result: default: False [-1.59 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.64.32:from]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.91)[0.905]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[98.137.64.32:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.32:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.32:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 May 2021 22:59:18 -0000 I had a: # zfs list -tall NAME USED AVAIL REFER = MOUNTPOINT . . . zroot/DESTDIRs/13_0R-CA72-instwrld-norm 1.44G 117G = 96K /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm zroot/DESTDIRs/13_0R-CA72-instwrld-norm@dirty-style 1.44G - = 1.44G -. . . . . . (copied/pasted from somewhat earlier) and then attempted: # zfs rename zroot/DESTDIRs/13_0R-CA72-instwrld-norm = zroot/DESTDIRs/13_0R-CA72-instwrld-alt-0 cannot open 'zroot/DESTDIRs/13_0R-CA72-instwrld-norm@dirty-style': = snapshot delimiter '@' is not expected here Despite the "cannot open" message, the result looks like: # zfs list -tall NAME USED AVAIL = REFER MOUNTPOINT . . . zroot/DESTDIRs/13_0R-CA72-instwrld-alt-0 1.44G 114G = 96K /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt-0 zroot/DESTDIRs/13_0R-CA72-instwrld-alt-0@dirty-style 1.44G - = 1.44G - . . . Still, it leaves me wondering if everything is okay given that internal attempt to use the old name with @dirty-style when it was apparently no longer available under that naming. For reference: # uname -apKU FreeBSD CA72_4c8G_ZFS 13.0-RELEASE FreeBSD 13.0-RELEASE #0 = releng/13.0-n244733-ea31abc261ff-dirty: Thu Apr 29 21:53:20 PDT 2021 = root@CA72_4c8G_ZFS:/usr/obj/BUILDs/13_0R-CA72-nodbg-clang/usr/13_0R-src/ar= m64.aarch64/sys/GENERIC-NODBG-CA72 arm64 aarch64 1300139 1300139 =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-stable@freebsd.org Wed May 5 03:26:41 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B07EE62E0A2 for ; Wed, 5 May 2021 03:26:41 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-55.consmr.mail.gq1.yahoo.com (sonic316-55.consmr.mail.gq1.yahoo.com [98.137.69.31]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FZhvh1Ht4z4pPk for ; Wed, 5 May 2021 03:26:39 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1620185198; bh=KQNG3JV0dMKY6IlgMNeyhPsyT3OGDupsYVUdbcGKpbC=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=hMKOuvoqBzM9C/+TqZKFUOD4rVxr4Xc8qXEMkl2oxvDuoFS9OQ0EBDeNTUSTtd+dIAV8WQSMSHiiTy167fQbeN2r2J3Pm7lDjP1rmFHn+tu194yoVYr4+p7g7Jv8DoHe816SiqQF+BAUVcHtMGoTSN3GyBziMibrQBIz4Dya5w73x8hjZ6GCPFuEJWPliQoECmdM+JYgP2YvGceaiS95dkuPSgO38EJahnTtTZ9YYrScqYICqoaOg6CYoyf5bpN2UYNPkHdlT3EFuK4sfXFhOK9PKcDbZCDkGzLiwk3jINdngOHUv9E30QiT0CRFQS02zyr0deo7MMgUG5KlNiXJEw== X-YMail-OSG: XAoIB1wVM1kf.LFZAQr8RhvlmGyFdMXNZhw2SuFLKEEKDhiVeq_j_gDbk.H1UsE zlHuxURHrn4hDqx7DFzKS6pDa9WZ09mGVvwWXsJcM4E17Ey.IKNs299MfmV0_8ZrJjpbItEMOw1I jE6LN_CVbR4F2gmWGbX1ozU6UrzDj5WBiyG9TkZFkK57TvF8D2yBlbnd5FS9KILrSThk38Zuk.RH nzzYbkH22HMXvR.te_GTuADkmyvaSD3nAWc9e4h3rhzywsB6g8hvM7rjueWOxlYAstmGEi.8AcQ1 MnhDji7mvTwDna60K_j.P.KJ.dnlQIe8KlhuMClbO7eQ4PJzFBOmobFJdznsyLqBDqMN83euLgTw Tq63SS3_Zy5Zla5USTjllgsRt2IMdbCftedzGlrTUqCq8nswE3P7KJ9yoHG0XYSoRNlZ195I7K4s SEEJVEjYU_vUwO6GFGY.lPnZXmDYa8jy8lAKW8VyTVhgNxi7yFX.o9Uc4j40sy7GC4btLTzzucw. UDQERTCQwDhd2cKMXBhE2GpheVC3lmldJEpUdJMib5FHLnnOVcJFs87Hcy_WCQL592hauJU0lxwU BfATXY_VutjbdO4EGdWLTONQDS5ZTmL99CtfO6eG7wz3q8xW2_rxgTUyg1w8bnDvGuwC4BW4xU4E Qg1BLfl8N5y9ljEZNKGQ_42BR1yHeIlmFKqNhkcHqq3mjvaePlPD099Z55t36inzvlGcLaSGn26M wzJz_oJQJGPbiRIJoCkZA7.p8TwMBNOeyEK9qsMbUvBTBmYLjGz4HWItQhDb1WdHY5ba8LCYpaDd 1FDOKq0_ibeL_6Z3T0pxMZ_EIQgIFPG01A_GZ4IybSyjf8sKPLSjRl30ev2HaBgnNB5QGgW.ISa5 yQKfE4tYGzQz64VmRMuZ1l1dDkXvFqj0vAWhLLHhGRTr206Qep17_xufE_C2jGRpHvfZIqzlnL5k DrC2C9ck8PYkJTiKZDOHmZCfZ69kTDQc1btoeDWNnm2V_5pu9y7q7ZFl6CcCX3zWJMrwKeujh1mc mRgkMdwtmEktoi8AQY7Y409gwPAj4IWNBIa8cNgWaidS2pqjvtJfO3ECWJ.LOUGySVmsTlKHXELP Mia4u5Dk5QNvaGTHcgQ.ODBUcwnDB0LCNtAryrHGDvOQSb2sjRnp5r1x2oKeYNGwt.6lUw9YSUyg zzGD9pkT2GusoXMRbeO5d8xcAXtPdkYeC2B_tYlKkXEmM1U3i.NzZsUvoAWMLZ48sc8BwcgGXkp6 Z17pfM6Ys0eoC_HkhhWVv0n9IgZ8u3BQEAL4s7jfAQsF9VO4GBAsUxCm4CJqsJBbp4mPGuLO006o KtTK3.3YJt7L6A9WbfhAvmFWVaFb1FTwwPzvLrFs9AB5smE4an_lyVvxNqbtnORVJLp.mZqLfHkn VA_86pe9JDxRq21ehSy5xspCU0m969Das3qxYoAHarCemPNidbJchhPBHicFLvWC0X3u0iZ80_iJ Wrqy3iFs_2jbBQHIWob5BLxHFdLiFqzJB13choQpM5hrJx.lFq_PaeyPU5EnYn1MDn7M6Izj9W6s W4K9KpA44SOYLTsn1NvIgDGGRl0Bh7uQVJkl_6VpgEYBoO5V3CpU5BsgcP66Fl4K3fX6uVOImhRo YWUsGvd2fxnZrpM.D.By5uD2_pfz65f9p4CBfFgRueqVoqW6BsAt41nDtaTZpiG1TgMbOxoUcf33 y9SO487bC0dwL8kq4W04qn2y4.WSgxNws4HaRQ0_rotJcU3laMT3DrA2G21j104Y0AUzUFKamSe2 QrKR2J3F8VmwdbZpdik2O5v5fdYRxteEXnQ1j_eI9OiMDbuJrC4UDBDdbdnf3e8JhiKzZl9tCSTs vAS7ZYo4MLJWuy8ielWlqJ45W.3dnKpvEvYuiveiKVuU7boDy8hbTRJeAjWelPxIMrL4Z5MKT8zX DxDms7vM9Nib.t6GuEn7zyDZmAnTa2oGkodrK1v_JhmP6ShH6Sl_xhmSjugbnCoAF5y0EVhDawcs FtAPXIDyx3EFTtWFfTHSVA0HrSNPStV59AUL.INWk6CxFvnckeMyLb75vZUknLfuVfqpqZadtDv0 HuJT3XfHsR_ZwGroxYj7DGdCmt_ZlPUpZ.evgushnTMemcQgKCv4F2ieLGB7quFJBSpeD37gkV8k tvJ_1UlkYh24R8GdpQy6oR0Z9yYg_JSjhavqLgS_oC41uq1bhTLsxmrUU8jZaMuFgL0mGs7IiOYb PXlqqVBIbZTdZOWHgphgl.qnv8COHcHp4.ARpzC3I0hx7hzpkMYPclJeKI4sH3Mtmxpx.TsVue6k Y_Nr7OQ2kWIcrkaYFiKONx.oF5R_qV9t0YTlyN3OV2AXKn3KXqUuWJQTkS3_b.2aWlGF3AnMVGUY 7wZzFEjCwU7SoBHw4YRuQEvTnoo8oXIuQ5Ku_YnyfTeadHUmwEoiiDGonDJd60yDCnU8mKh4x460 ltu48oAK5_KPhX8rLPIAtXlIgyzOgBPiVNWUwm1q9C4tolfnV_6inNav86mcIJ4oGAY5.9VnIWIN jpZJ_mD_QS06b743cdZaIASU5JHOU5rLZ1R3By5NnXef_BGGlU4t_YIFlVAXLEUZcwzHzcRnyAoo PJ2yb5xfX5e15pf7XZV6MN2P6pbD61__ZIugTxjr4fm8AcndD_RThubAxLvWeJYZg0foy_xKl4XY g2g5ye4Y_HSgSmVuDpNiSPnaQMmlDMQDSnEHcVPMjYvkPFXh01_.lhjPIBtQRd5dCgjqPMBrvx6. JPIVaHAOXu6f_Y3uCw4YZovgD1ZclCYZF1LixwJdHrg-- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Wed, 5 May 2021 03:26:38 +0000 Received: by kubenode548.mail-prod1.omega.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID b01a7d29a0fa63ba3f53d2a3e60d14b1; Wed, 05 May 2021 03:26:32 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: Re: FYI: WITH_REPRODUCIBLE_BUILD= problem for some files? [aarch64 test did not reproduce the issue] From: Mark Millard In-Reply-To: Date: Tue, 4 May 2021 20:26:30 -0700 Cc: FreeBSD-STABLE Mailing List , freebsd-current Content-Transfer-Encoding: quoted-printable Message-Id: References: <35482701-95A3-48B2-9A8E-B7E0092119B1.ref@yahoo.com> <35482701-95A3-48B2-9A8E-B7E0092119B1@yahoo.com> <43F20589-A7C7-42FF-9020-09CEE037D1CD@yahoo.com> <91F820A1-8940-4246-A20A-E62685F50079@yahoo.com> <27B5086A-7C98-4AE6-885A-CC7C7BD2F64B@yahoo.com> <3FC6BCDD-5865-4B5D-8238-3EC38AD4E78C@yahoo.com> <2E86A8E9-9F0E-446E-BF80-5FA3B8ECB197@yahoo.com> To: Ed Maste X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Rspamd-Queue-Id: 4FZhvh1Ht4z4pPk X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.50 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.00)[-0.999]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.69.31:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MID_RHS_MATCH_FROM(0.00)[]; SUBJECT_HAS_QUESTION(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[98.137.69.31:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.31:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.31:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 May 2021 03:26:41 -0000 On 2021-May-4, at 13:38, Mark Millard wrote: > [The first buidlworld is still in process. So while waiting . . .] >=20 > On 2021-May-4, at 10:31, Mark Millard wrote: >=20 >> I probably know why the huge count of differences this time >> unlike the original report . . . >>=20 >> Previously I built based on a checked-in branch as part of >> my experimenting. This time it was in a -dirty form (not >> checked in), again as part of my experimental exploration. >>=20 >> WITH_REPRODUCIBLE_BUILD=3D makes a distinction between these >> if I remember right: (partially?) disabling itself for >> -dirty style. >>=20 >> To reproduce the original style of test I need to create >> a branch with my few patches checked in and do the >> buildworlds from that branch. >>=20 >> This will, of course, take a while. >>=20 >> Sorry for the noise. >>=20 >=20 > I've confirmed some of the details of the large number of > files with difference while waiting for the 1st buildworld : >=20 > The 4 bytes at the end of the .gnu_debuglink section > that are ending up different are the checksum for the > .debug file. The .debug files have differences such as: >=20 > =E2=94=82 - <1a> DW_AT_comp_dir : (indirect string) = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/usr/13_0R-src/arm64.aarch64/lib= /csu/aarch64 > =E2=94=82 + <1a> DW_AT_comp_dir : (indirect string) = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang/usr/13_0R-src/arm64.aarch64/lib/csu= /aarch64 >=20 > So I need to build, snapshot (in case need > to reference), install, clean-out, build, > install elsewhere, compare. (Or analogous > that uses the same build base-path for both > installs despite separate buildworld's.) > This is separate from any potential -dirty > vs. checked-in handling variation by > WITH_REPRODUCIBLE_BUILD=3D . >=20 > My process that produced the original armv7 > report happened to do that before I accidentally > discovered the presence of the few files with > differences. My new experiments were different > and I'd not though of needing to vary the > procedure to get you the right evidence. >=20 The two aarch64 test installs did not show any differences in a "diff -rq" . Ignoring *.meta files generated during the builds, the build directory tree snapshots showed just the differences: # diff -rq = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-0/= usr = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-1/= usr | grep -v '\.meta' | more Files = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-0/= usr/13_0R-src/arm64.aarch64/stand/ficl/softcore.c and = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-1/= usr/13_0R-src/arm64.aarch64/stand/ficl/softcore.c differ Files = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-0/= usr/13_0R-src/arm64.aarch64/toolchain-metadata.mk and = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-1/= usr/13_0R-src/arm64.aarch64/toolchain-metadata.mk differ # diff -u = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-0/= usr/13_0R-src/arm64.aarch64/stand/ficl/softcore.c = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-1/= usr/13_0R-src/arm64.aarch64/stand/ficl/softcore.c --- = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-0/= usr/13_0R-src/arm64.aarch64/stand/ficl/softcore.c 2021-05-04 = 13:45:14.463351000 -0700 +++ = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-1/= usr/13_0R-src/arm64.aarch64/stand/ficl/softcore.c 2021-05-04 = 19:04:32.338203000 -0700 @@ -4,7 +4,7 @@ ** Words from CORE set written in FICL ** Author: John Sadler (john_sadler@alum.mit.edu) ** Created: 27 December 1997 -** Last update: Tue May 4 13:45:14 PDT 2021 +** Last update: Tue May 4 19:04:32 PDT 2021 *******************************************************************/ /* ** DO NOT EDIT THIS FILE -- it is generated by softwords/softcore.awk # diff -u = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-0/= usr/13_0R-src/arm64.aarch64/toolchain-metadata.mk = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-1/= usr/13_0R-src/arm64.aarch64/toolchain-metadata.mk --- = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-0/= usr/13_0R-src/arm64.aarch64/toolchain-metadata.mk 2021-05-04 = 10:55:26.030179000 -0700 +++ = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-1/= usr/13_0R-src/arm64.aarch64/toolchain-metadata.mk 2021-05-04 = 16:14:24.513346000 -0700 @@ -1,4 +1,4 @@ -.info Using cached toolchain metadata from build at CA72_4c8G_ZFS on = Tue May 4 10:55:26 PDT 2021 +.info Using cached toolchain metadata from build at CA72_4c8G_ZFS on = Tue May 4 16:14:24 PDT 2021 _LOADED_TOOLCHAIN_METADATA=3Dt COMPILER_VERSION=3D110001 X_COMPILER_VERSION=3D110001 I may run a 'target cortex-a7 (so: armv7) from aarch64' test next. That was the context for the original discovery and report. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-stable@freebsd.org Wed May 5 09:47:40 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2D63B638028; Wed, 5 May 2021 09:47:40 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from relay11.mail.gandi.net (relay11.mail.gandi.net [217.70.178.231]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FZsMH4pt7z3Q2Z; Wed, 5 May 2021 09:47:39 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from [192.168.0.88] (unknown [195.64.148.76]) (Authenticated sender: andriy.gapon@uabsd.com) by relay11.mail.gandi.net (Postfix) with ESMTPSA id DF356100012; Wed, 5 May 2021 09:47:36 +0000 (UTC) Subject: Re: ZFS rename with associated snapshot present: odd error message To: Mark Millard , FreeBSD-STABLE Mailing List Cc: freebsd-current References: <8335C81D-B83C-42BC-B296-C05FAEAE538A.ref@yahoo.com> <8335C81D-B83C-42BC-B296-C05FAEAE538A@yahoo.com> From: Andriy Gapon Message-ID: <4ddf96da-e7f7-663d-8539-04f91297389a@FreeBSD.org> Date: Wed, 5 May 2021 12:47:35 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Firefox/78.0 Thunderbird/78.10.0 MIME-Version: 1.0 In-Reply-To: <8335C81D-B83C-42BC-B296-C05FAEAE538A@yahoo.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4FZsMH4pt7z3Q2Z X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; ASN(0.00)[asn:29169, ipnet:217.70.176.0/20, country:FR] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 May 2021 09:47:40 -0000 On 05/05/2021 01:59, Mark Millard via freebsd-current wrote: > I had a: > > # zfs list -tall > NAME USED AVAIL REFER MOUNTPOINT > . . . > zroot/DESTDIRs/13_0R-CA72-instwrld-norm 1.44G 117G 96K /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm > zroot/DESTDIRs/13_0R-CA72-instwrld-norm@dirty-style 1.44G - 1.44G -. . . > . . . > > (copied/pasted from somewhat earlier) and then attempted: > > # zfs rename zroot/DESTDIRs/13_0R-CA72-instwrld-norm zroot/DESTDIRs/13_0R-CA72-instwrld-alt-0 > cannot open 'zroot/DESTDIRs/13_0R-CA72-instwrld-norm@dirty-style': snapshot delimiter '@' is not expected here > > Despite the "cannot open" message, the result looks like: > > # zfs list -tall > NAME USED AVAIL REFER MOUNTPOINT > . . . > zroot/DESTDIRs/13_0R-CA72-instwrld-alt-0 1.44G 114G 96K /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt-0 > zroot/DESTDIRs/13_0R-CA72-instwrld-alt-0@dirty-style 1.44G - 1.44G - > . . . > > Still, it leaves me wondering if everything is okay > given that internal attempt to use the old name with > @dirty-style when it was apparently no longer > available under that naming. > > For reference: > > # uname -apKU > FreeBSD CA72_4c8G_ZFS 13.0-RELEASE FreeBSD 13.0-RELEASE #0 releng/13.0-n244733-ea31abc261ff-dirty: Thu Apr 29 21:53:20 PDT 2021 root@CA72_4c8G_ZFS:/usr/obj/BUILDs/13_0R-CA72-nodbg-clang/usr/13_0R-src/arm64.aarch64/sys/GENERIC-NODBG-CA72 arm64 aarch64 1300139 1300139 Cannot reproduce here (but with much simpler names and on stable/13): zfs create testz/test zfs snapshot testz/test@snap1 zfs rename testz/test testz/test2 All worked. -- Andriy Gapon From owner-freebsd-stable@freebsd.org Wed May 5 12:28:33 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0744263D1E0 for ; Wed, 5 May 2021 12:28:33 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic307-54.consmr.mail.gq1.yahoo.com (sonic307-54.consmr.mail.gq1.yahoo.com [98.137.64.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FZwwv4qpZz3pKf for ; Wed, 5 May 2021 12:28:31 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1620217709; bh=zAok200WkYcdxN0AwMzElDUPvx2fpciESsD8wyu0vTZ=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=goqSdl49ojPfuq6l+qNdrKZZasDG4ZnEjdTFSXpwmRHUCHyJN1DEoId1fvQSuVGGTD19KudnWgyz4qeInEXbN7vgGGmr3b8fJI2eVRxTx6VyMBHbmVNDbr9rotTS5DGE9n+feJ1zbBWfbvllgW+E3aTMeSrdMOn4gJDO2DWSOf4YcqCMxWr7RGaSebdUGBpnkwijqtXpTjYGRWp+TtCWvwck4rETQn1amFi7s0nw/Jkb8W/xF8KMJ47X8lZueDk0gXT4cEsiuCWYAb95PhURszXU4EywArv8T6PeHDYBH4z2tCN+yNkDSoWS2vmArUaG+ma23z5QTh9c9/935jEVTA== X-YMail-OSG: j6dyDwwVM1lFU_GO6px.6lLqJ5_CRH.grP_yyUy9c.a65XDUJPWfsr4D4X6Otpr uWpAQ06fDNGrb1jhNm7dtj476_b_AX0dudgDk1UcMYGj_x9YJAAWR2QeyLWh7cTcHR2n5OgpPTil _qF_nzXAg1wIWdS7TeQtBcDRFe_rGO4aMwQcb0zJ0_8VuIh098Hm5VLOxx1qUpTKpyb8jUW9Pu2u Eo1vR0kZu4jvTUvlSmVFr__fg9m5V_MQIfygjvGbMN_WeIWIF0pXQw0_ExYYt7lkF7G46y3H6Mas QgQPrucZbf8Jf4voDvSxIc2eY8Zx2uAifgXghUZTCsHykWEpcQ2Y8YO2MLYVSOcV8GVknereNZ_j Vfb8U3DY7ZhojgybeeWeWwilyTNvUX8wkIT9xkx9NLQ75vqDR.XIYmpABA4dXsZhsfEDBCbWX099 oQU5rv4WF8taxj0i3DVpZI2Ay264S2YjzwT2aQwhRkXCjn3gHXSfVAPJl5hjpWa5OMJLQJ.yFCCk dNW67vGjbU03PsUVz7TSQbYXfjna.EokZtYTC7BjK1Cgaaur2pX44XGXYaBWCUgZfX5DWBmVCLcD p1311fj51yr38RLSfp.tSHpTtkpk47nrt1dx7W.St7_ij6_5jOn06uXjNbSJCSsRaibdnb7LsvPS nktP1WABKm1_0ZdazjPflBU351uAs89olrc7UmBeUW.UYrZ9OLT4YFbeOZ.ONcCGHO4WZwhxVNDs D4vOdRBvCaO_w2Z1HoX622jN6kLxGD3se1cnKtW5gydSdbbGiHdaZBE6oRt5YZkjY.JaZ4hi8Fzt Hq7kQYhTELbRrt_yo72QYCCVT96gYlGWFqby80oQhFKpKn4eHPd42aiwYVSZIiUNxBgFVHqNGN9D tvWbexbEKkohl0hAdFsjgcUw39rbKFkGysMBS2qvWSaSOwU6Qrwn.Og5AHSGkjoH7zYh8aW6JYjK Qvt4SXXNUj1CQJgJOB0jI0.ZsV5gog7IkhebxLffUNPnXNyeBuHD.S5iUrfGc2OpFeIArLNQdcCv sOfiZehhbO6VHaYed4rXwFi7xvxYqMfGQD4Em.GUJtakVVs1hArmhvV0nLtzuFScX8BFn37m7mvm G3ZiyOllZsK7IQFqy8VZtzD.5dOxwfiyCYgtJtbQx2AGabuSSHubb8fnRP8bqIe3lljrRKicnfG_ MUQRiyFMUxA0hDjc8Xh6xPn89HgZENwM5IsrEfF_lZFnevHufy79AZHCy9OVtCfUs9zc3Oz6q82B hTT15QdHAUiw7glblddIaZafGAgBtV4KQW2nh0SMtEtD211k8xz87JP4jZPvfUHDpdNCjcJ7B05e WzUKzWWnDFnpNsNX_Gdp3cTX.wuFLvvYnUcMeq93lW89J7BLXaxwb7trBGZTtcKVUdyFdLAoDnTG F.6UoBGmX80myXfAOAhx.r0vnDLARhVpwggXpBRi.R5JAPnFrTL87SrY3g5o8H03S88cq0_LpSVk czhNJbKz.g0dIUE9E2wkqk11wK6LiskoDBFNUH6ApFgaBelrsUlc1KcZPIqQ509Z9tyrnGpXMURo ntvIA.CE.I6VHTuU0Jf23009kOKNqVs5K3WI09w.1dunROhbmSVUHMk4QO3ziid8hF._bAo5zuN1 dEKbHe4cnWOV0IS5ecZL45mzyAiqrvvVAL0tqISh0NI7atQdEaWFmttVX9lj.a8UTpOF8XUrMXa9 IEiPnayqvEqO0bPG1ZRsFM0_5yUmwv5vEvAqL8a7r6TTc30sbTNdkZdKNlKyC1gu85PlfI.vWLi0 YVrZPipFnQUf2uCJT_CidNDBR6tJqpnV..Wl10amGpnAIp0sFv73fRPfilZfc2hSJRsKQ4VG6Sqb BkPSm29o1az5O.wS0QGujb8GBD3E9WA810_AXZC0UR2hfGCnjmD3otoYOjs9CnHcA8Kk8ngt5Pmf yfjqiDa9_nXB4wThFk3RWLsbh0M.i7EiYh0JaJAZMLdAXVIoHN2mODxMQVaOQeUkdSbBzo1zdsGd e3a7J4d1_C02xwTA7yOapTPy64c1.F3B2PIbVoW0qixXrHNmk3_uuY93zgKcNiW37JZj3A0VoLoO yZ6DjM6F8LPyJVedeOrZehefpOZsws.FLs2OccIVhcx7M1w6H3nGZ0MLRZpm0tyqQyaAnI127YtY 0YDgXofkOBsxlZEwWmjj_4_tzeGmdmoGuNBRDyfIx8bLBj_Mfprtn7RGNMCs2b1wssLkQbx7pL67 zk5OzXvULXNrCwGCTmajZZCxL0o_oYlPj9aQR3GQ8krUVVjeopW80aCLBLNjxpeLMzah0TLYGwT7 3mdkwnVe4IE9DK7_GqZwYlTqZ8a43SuR5R4Yjjmwm2F8RP8r3j7w_NLf7B7.sJJy4aCvn6t5iiWe tWkhy2CqXVOnGhGjsmh1EYA2vyAS.dUR4TC5lLITH40KOFdAxzECznOc1PtjAj3e5R2hUoIMnZYW G4wRmwPmpswPXr9ezQpm.ALlG2u9a2rrep6Cc5KejtqhlMmnCkFi7.nRM044IV9EvA2QrMZ_Z6Qb OWAJ0Ubt8cefX3jMPVDN2PrLi1c_PN9U8NwK5e5iB0tbBb4sQx7CBwyOuRPyDTEVz4sgKYj4eUZU Do5p4YYxJ4PW2nJK1dOyMuP9KKeOcEAinqRFT9M7XQqY24tRPqcDZcDjM8ITvBVOcqYK7WQ-- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic307.consmr.mail.gq1.yahoo.com with HTTP; Wed, 5 May 2021 12:28:29 +0000 Received: by kubenode572.mail-prod1.omega.gq1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 5db36cac940f6273fa794232fbf1f177; Wed, 05 May 2021 12:28:27 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: Re: ZFS rename with associated snapshot present: odd error message From: Mark Millard In-Reply-To: <4ddf96da-e7f7-663d-8539-04f91297389a@FreeBSD.org> Date: Wed, 5 May 2021 05:28:27 -0700 Cc: FreeBSD-STABLE Mailing List , freebsd-current Content-Transfer-Encoding: quoted-printable Message-Id: <5294FCAA-14C8-45CB-B34A-B4D76F70AA8B@yahoo.com> References: <8335C81D-B83C-42BC-B296-C05FAEAE538A.ref@yahoo.com> <8335C81D-B83C-42BC-B296-C05FAEAE538A@yahoo.com> <4ddf96da-e7f7-663d-8539-04f91297389a@FreeBSD.org> To: Andriy Gapon X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Rspamd-Queue-Id: 4FZwwv4qpZz3pKf X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.50 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.64.30:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[98.137.64.30:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.30:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.30:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 May 2021 12:28:33 -0000 On 2021-May-5, at 02:47, Andriy Gapon wrote: > On 05/05/2021 01:59, Mark Millard via freebsd-current wrote: >> I had a: >> # zfs list -tall >> NAME USED AVAIL REFER = MOUNTPOINT >> . . . >> zroot/DESTDIRs/13_0R-CA72-instwrld-norm 1.44G 117G = 96K /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm >> zroot/DESTDIRs/13_0R-CA72-instwrld-norm@dirty-style 1.44G - = 1.44G -. . . >> . . . >> (copied/pasted from somewhat earlier) and then attempted: >> # zfs rename zroot/DESTDIRs/13_0R-CA72-instwrld-norm = zroot/DESTDIRs/13_0R-CA72-instwrld-alt-0 >> cannot open 'zroot/DESTDIRs/13_0R-CA72-instwrld-norm@dirty-style': = snapshot delimiter '@' is not expected here >> Despite the "cannot open" message, the result looks like: >> # zfs list -tall >> NAME USED = AVAIL REFER MOUNTPOINT >> . . . >> zroot/DESTDIRs/13_0R-CA72-instwrld-alt-0 1.44G = 114G 96K /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt-0 >> zroot/DESTDIRs/13_0R-CA72-instwrld-alt-0@dirty-style 1.44G = - 1.44G - >> . . . >> Still, it leaves me wondering if everything is okay >> given that internal attempt to use the old name with >> @dirty-style when it was apparently no longer >> available under that naming. >> For reference: >> # uname -apKU >> FreeBSD CA72_4c8G_ZFS 13.0-RELEASE FreeBSD 13.0-RELEASE #0 = releng/13.0-n244733-ea31abc261ff-dirty: Thu Apr 29 21:53:20 PDT 2021 = root@CA72_4c8G_ZFS:/usr/obj/BUILDs/13_0R-CA72-nodbg-clang/usr/13_0R-src/ar= m64.aarch64/sys/GENERIC-NODBG-CA72 arm64 aarch64 1300139 1300139 >=20 > Cannot reproduce here (but with much simpler names and on stable/13): > zfs create testz/test > zfs snapshot testz/test@snap1 > zfs rename testz/test testz/test2 >=20 > All worked. >=20 I've noticed that sometimes in my explorations it has been silent instead of complaining. I've no clue at this point what prior activity (or lack of activity) makes the difference for if a message will be generated vs. not. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-stable@freebsd.org Wed May 5 13:10:09 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 203BB63E67A for ; Wed, 5 May 2021 13:10:09 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic307-54.consmr.mail.gq1.yahoo.com (sonic307-54.consmr.mail.gq1.yahoo.com [98.137.64.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FZxrw0Qb0z3qpd for ; Wed, 5 May 2021 13:10:07 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1620220206; bh=ZY4QfGsU7CN9D8nTDPfhWWR8AaL/9Ku86a5h9wMdbEg=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=Zpg79N+CUa7AQ1cGxoJ4VRbkC7mNtajVB/v2w9oTw1rhV8LABrmhktF4UvHDA88yFMt/kacFCIDzlGVhPH/bBhBJL6bBDuFbASbCkvCvt79vuqZ5GlI3SZXDFiN2wgjk5AdHPHI3oeI4Ohg4cJo+YPY0xoIkWH0wUl8zUaEwE4gjQ8+46L6EMY4Ka3FvXIq5FZgrBTdPtRo8OaUvxRn1ITGqhl8b7na+x7kC8uRWXL2QzQBil0xUp24SS5854cDxTtq5HHYXW+34qgQ0HSx/FvnuAqqIkTsONs5rDwIUm/igMColJQiDtKWOEYbcWuRcFhF+flTAXDO4KCHN8X+dHA== X-YMail-OSG: nBUbTQIVM1li.Vc0XIIHhwVFOncw9SNALRSm9eqBkzX_Pu5EcPkhZ65sq9ANjGT M23mejif7C4v4fyBEec.lhIdj3.qm0J5lRH2u3WFJPS3dqANLSrsuvYm6EAZBDJHaI.iA1HozSPY WMW1E9.ulwEWxR_Ai4UVZJcDy0nNhIs49F6L25US0NPP80PigExay2sYw1VR0XDLCphd21h37FvJ R4u8upAp59EB56O1hyF0uTFJfdmT.xXIo.g5DT_NoOiVmQ2hY.4MGCL7xfs_dTCZsivnqduvcsia QGFD10_x97pMbmxP96rvf_3LxrEa.Hi4gulD.XqWlR6nKndr6f6ePiTl25B0MkovGVgmPSA2ZzQb VHSyAfNkuaSJi9vrinYCWbjLBAyySbZfYGQp4Bi_nN.eia6I3Q66ZA6nRXZsuQ2CuPIKAUqjNRaH TWvy1y.TlkYqUgd07Hlg2MhSdM_YgUaZTs51S5DwsKFdTYOl9J.KwNUHXXgr5k1mQlCxyfP5gqJ0 Rh0mibccmXgIzXM.XKHhrK1sP9pOyZNcLsaMJ5k.Ba2jgdzcLna_naaFak8ZgpwiHKGPr5A6jNIs h4dbTXuhI3cYZoHMhAzbNy5no9L6pK3uY_onRm616s4jbrhrRJU6nCzd7S37WbxYj.reXv_huR_Q pkMh6IxoYoEwTAXM2ssfrxbuRgvbcAe96OueIGnTYqwlctPEHv9sES1SZ4m.WkE00G7FyZluJNlH MRysEkL.XMv63xD.dgPQvDj8aCJIt8qCEvgeFmA1NuJdEj8UKB9ehdz9BoVmkJ1Mueqyv0p0.h7t svyro55QLDGlk4cYEl002t5svvleA35.7V4DEzrpFqqkxyMMZu5jXUuaRHmsnE5UeDCz.UDa5Px0 A94Ywj1ZRZG2X3TfzkUwK5AkGLE1gdoQJWMXwdRGtaesfX7my8bndGheyB5j4yLY.DCZj6mLogW9 M_h0ca3zzjx5R_vDvevpELy9embNIMyqlxlSI5S9kvVQdI6s.Y4doEl0FT2VQHO_J_7ccIFy74iE qjNCM52GftmOoukVJjfccXkSArjEqTL4cFDMOkDOmoKJmfInDrdHt7S..aSp1Huywaw9rA1GX3zp mS_EujRvdVhESoiycZO7xyVRhHNU3545hTykvFTdTntZP5In7s7PGioRslOkdKXAfuglx0RMNdbG vApOISsAAieCg13riciSC8Dc2L88Mv3RQR7qisB2eN4OfvANTO.7h9wQjAp91rKniVx5M_CUeH36 bQG61DzNdE80aFWISFy3nidhasp2gG1vCEOtfPZIhT7oGmdaz_p43AtkbOX7XA20q3uO5sgNlrKd p1u0saqiPXWCN5AFHkH9pjujaPOucHZUtR5ScIqIn7lA6AfR7mjNsfT30bHqz5kY9U6YnS6Y8466 QiPuarI4UgmtydzNdjz7CBlQ5hODv8PRh7IqJb4l_6pexN7Wf5jlnUQkPsZSulbY2dI5olv8s8xG 1fa3o1K2WRLqI2sosCJ694pRCqYLZQxNtxhECSuiYq1RuhFEAe3iNNMRpKhlgvGKFKZmDeXBB8Qj PgVfuDmrhj3BpYGLvugqcFnuyjq.vgt07XEQn4dR4AgmJQsE2ehIPVQZhSm0JzlGneDpFEeCKMD5 1MatkIWT1JjMV2AXwHCuif3jYA1P.HiMKKhdkLOVHe5GhAt9VpX7rX_T92x6hHXZ1i6n8.9BsFb4 yjcNMGSlIyg9y9Kma4Ny1PIIYasjdJ.9IET5qi79JNIwEwmvhi7GjVgGb5LX8uL0DpfGlnpvCX4c HlSYTDSss2En3EnI4DUyhNGIand_kPnocDiEhnIsld523RVvQ_qv5tVDkYavtIVkXhmjqXwWoQ1v KojQA3mUQAw0CzSg0ygJzsPGhlu_8MhaTiH8O_ndmejfigOGaPjI2q_Bm7XseKilu3ux_glrTHG7 vEJmlUqMt2l5LKnwpd_ntUbFTbRvxpZ5S3hV4p1GvA5O3P0wzNfBeER3yemzteq9cvyQLxEi5x9j j_YPOtWnrPYWhtRZ8T5CiMVFuRGPMIUKIYDGl6faQItsiJg4hy5jvynMzScL_4woAEN72mcWwyUU _83DdGLbVSAQ5f1OxBX1kqIhRI2rqN5..zRbnooZcC3HuE_V.wFcqiOpN_Y6sP_Bc.rN25aWnITY Y.RaoGF0HXZsjXkTESIbljwHhliamKkrphKMpQkej6WFAbCTkZalQtJPzx.hmAqy6cVbOJleoZfg ZgiUQYsk2iiStt96U5xAXXob1Cb99Nv0vXvqpaxRvae3vbM0GkxTDOAEZ6_X8kff3DXLNFZTMZ2F j0FtiVPpb42PJXaNvC4a2J3F1nscDKz.zcuTD5uaTck06Q0Vyp32ECF9uFp9rCVmXJblN2Fv0rfD Y91xC1UAcMQ2fJIk3V7SuofLPZB.T7XQIDwcpWASWzdlHkhlvoD1Z3Xdd8YKejhpNdiH4tnpF.TL DoWgNAq5Dx4RqpZI.muoW2wX_5rqP99JnntM2hndKCyJrjAQ2C.SSIW74VFshpfs_yPBqfUsuLY6 kg7lnd1AAfkUOshFpA9ZHyl8D.W58rxEmfLZjEFELZqbM03mBHeNyV_Ml3OimYD416EbmDWfujIX cqvF6udbi9cdkhvRsXlzG50Sm.7OaMlbwS_zMt1tQEx6Ip.wTSNZVFYS9vDcd6COw0evr1N2ttn3 As98hZGu7gRq9tZeRrLY6PSRfSvD17wYkryNZkehXel0LySW5nl04JqjkTfXj7NZjKa56ZVLk_sp hE9YGTuzuSplC8loTUoYpmhOZAZuwhq9KUHfNY.7eHQ-- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic307.consmr.mail.gq1.yahoo.com with HTTP; Wed, 5 May 2021 13:10:06 +0000 Received: by kubenode512.mail-prod1.omega.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 8f9d596d715241f5175657af5d33c051; Wed, 05 May 2021 13:10:02 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: Re: FYI: WITH_REPRODUCIBLE_BUILD= problem for some files? [aarch64 test did not reproduce the issue] From: Mark Millard In-Reply-To: Date: Wed, 5 May 2021 06:09:59 -0700 Cc: FreeBSD-STABLE Mailing List , freebsd-current Content-Transfer-Encoding: quoted-printable Message-Id: References: <35482701-95A3-48B2-9A8E-B7E0092119B1.ref@yahoo.com> <35482701-95A3-48B2-9A8E-B7E0092119B1@yahoo.com> <43F20589-A7C7-42FF-9020-09CEE037D1CD@yahoo.com> <91F820A1-8940-4246-A20A-E62685F50079@yahoo.com> <27B5086A-7C98-4AE6-885A-CC7C7BD2F64B@yahoo.com> <3FC6BCDD-5865-4B5D-8238-3EC38AD4E78C@yahoo.com> <2E86A8E9-9F0E-446E-BF80-5FA3B8ECB197@yahoo.com> To: Ed Maste X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Rspamd-Queue-Id: 4FZxrw0Qb0z3qpd X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.11 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.61)[-0.606]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.64.30:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MID_RHS_MATCH_FROM(0.00)[]; SUBJECT_HAS_QUESTION(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[98.137.64.30:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.30:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.30:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 May 2021 13:10:09 -0000 On 2021-May-4, at 20:26, Mark Millard wrote: > On 2021-May-4, at 13:38, Mark Millard wrote: >=20 >> [The first buidlworld is still in process. So while waiting . . .] >>=20 >> On 2021-May-4, at 10:31, Mark Millard wrote: >>=20 >>> I probably know why the huge count of differences this time >>> unlike the original report . . . >>>=20 >>> Previously I built based on a checked-in branch as part of >>> my experimenting. This time it was in a -dirty form (not >>> checked in), again as part of my experimental exploration. >>>=20 >>> WITH_REPRODUCIBLE_BUILD=3D makes a distinction between these >>> if I remember right: (partially?) disabling itself for >>> -dirty style. >>>=20 >>> To reproduce the original style of test I need to create >>> a branch with my few patches checked in and do the >>> buildworlds from that branch. >>>=20 >>> This will, of course, take a while. >>>=20 >>> Sorry for the noise. >>>=20 >>=20 >> I've confirmed some of the details of the large number of >> files with difference while waiting for the 1st buildworld : >>=20 >> The 4 bytes at the end of the .gnu_debuglink section >> that are ending up different are the checksum for the >> .debug file. The .debug files have differences such as: >>=20 >> =E2=94=82 - <1a> DW_AT_comp_dir : (indirect string) = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/usr/13_0R-src/arm64.aarch64/lib= /csu/aarch64 >> =E2=94=82 + <1a> DW_AT_comp_dir : (indirect string) = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang/usr/13_0R-src/arm64.aarch64/lib/csu= /aarch64 >>=20 >> So I need to build, snapshot (in case need >> to reference), install, clean-out, build, >> install elsewhere, compare. (Or analogous >> that uses the same build base-path for both >> installs despite separate buildworld's.) >> This is separate from any potential -dirty >> vs. checked-in handling variation by >> WITH_REPRODUCIBLE_BUILD=3D . >>=20 >> My process that produced the original armv7 >> report happened to do that before I accidentally >> discovered the presence of the few files with >> differences. My new experiments were different >> and I'd not though of needing to vary the >> procedure to get you the right evidence. >>=20 >=20 > The two aarch64 test installs did not show any > differences in a "diff -rq" . Ignoring *.meta > files generated during the builds, the build > directory tree snapshots showed just the > differences: >=20 > # diff -rq = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-0/= usr = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-1/= usr | grep -v '\.meta' | more > Files = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-0/= usr/13_0R-src/arm64.aarch64/stand/ficl/softcore.c and = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-1/= usr/13_0R-src/arm64.aarch64/stand/ficl/softcore.c differ > Files = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-0/= usr/13_0R-src/arm64.aarch64/toolchain-metadata.mk and = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-1/= usr/13_0R-src/arm64.aarch64/toolchain-metadata.mk differ >=20 > # diff -u = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-0/= usr/13_0R-src/arm64.aarch64/stand/ficl/softcore.c = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-1/= usr/13_0R-src/arm64.aarch64/stand/ficl/softcore.c > --- = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-0/= usr/13_0R-src/arm64.aarch64/stand/ficl/softcore.c 2021-05-04 = 13:45:14.463351000 -0700 > +++ = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-1/= usr/13_0R-src/arm64.aarch64/stand/ficl/softcore.c 2021-05-04 = 19:04:32.338203000 -0700 > @@ -4,7 +4,7 @@ > ** Words from CORE set written in FICL > ** Author: John Sadler (john_sadler@alum.mit.edu) > ** Created: 27 December 1997 > -** Last update: Tue May 4 13:45:14 PDT 2021 > +** Last update: Tue May 4 19:04:32 PDT 2021 > *******************************************************************/ > /* > ** DO NOT EDIT THIS FILE -- it is generated by softwords/softcore.awk >=20 > # diff -u = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-0/= usr/13_0R-src/arm64.aarch64/toolchain-metadata.mk = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-1/= usr/13_0R-src/arm64.aarch64/toolchain-metadata.mk > --- = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-0/= usr/13_0R-src/arm64.aarch64/toolchain-metadata.mk 2021-05-04 = 10:55:26.030179000 -0700 > +++ = /usr/obj/BUILDs/13_0R-CA72-nodbg-clang-alt/.zfs/snapshot/commited-style-1/= usr/13_0R-src/arm64.aarch64/toolchain-metadata.mk 2021-05-04 = 16:14:24.513346000 -0700 > @@ -1,4 +1,4 @@ > -.info Using cached toolchain metadata from build at CA72_4c8G_ZFS on = Tue May 4 10:55:26 PDT 2021 > +.info Using cached toolchain metadata from build at CA72_4c8G_ZFS on = Tue May 4 16:14:24 PDT 2021 > _LOADED_TOOLCHAIN_METADATA=3Dt > COMPILER_VERSION=3D110001 > X_COMPILER_VERSION=3D110001 >=20 >=20 >=20 > I may run a 'target cortex-a7 (so: armv7) from aarch64' test > next. That was the context for the original discovery and > report. >=20 >=20 The armv7 builds and installs get the same sort of diff results as the aarch64 ones. So it does not look like I can readily reproduce the problem files that had differences. Given the time it takes to run the experiments, I would not expect to reproduce the problem on any given timescale. I'll stop the reporting as I go along in my activities --unless I end up with a repetition of the problem. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-stable@freebsd.org Wed May 5 18:57:03 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 01B115FF834 for ; Wed, 5 May 2021 18:57:03 +0000 (UTC) (envelope-from gbe@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fb5YB6hbNz4d9m for ; Wed, 5 May 2021 18:57:02 +0000 (UTC) (envelope-from gbe@freebsd.org) Received: from localhost (p200300d5d70db0ef3d2b701b60ea2a15.dip0.t-ipconnect.de [IPv6:2003:d5:d70d:b0ef:3d2b:701b:60ea:2a15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: gbe) by smtp.freebsd.org (Postfix) with ESMTPSA id 7E391212B7 for ; Wed, 5 May 2021 18:57:02 +0000 (UTC) (envelope-from gbe@freebsd.org) Date: Wed, 5 May 2021 20:56:58 +0200 From: Gordon Bergling To: freebsd-stable@freebsd.org Subject: Building an 13-STABLE release for i386 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Url: X-Operating-System: FreeBSD 12.2-STABLE amd64 X-Host-Uptime: 8:48PM up 1 day, 2:18, 4 users, load averages: 0.30, 0.21, 0.18 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 May 2021 18:57:03 -0000 Hi, I am currently try to build a custom 13-STABLE release for i386, build on an amd64 system. According to release(7) the following command should do the trick, but it fails with the following error message. $ doas sh release.sh -c i386/i386.conf ld-elf32.so.1: Shared object "libedit.so.8" not found, required by "sh" Has anyone an idea what could cause this? --Gordon From owner-freebsd-stable@freebsd.org Wed May 5 19:02:45 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 878425FFD85 for ; Wed, 5 May 2021 19:02:45 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fb5gn3RWRz4df7; Wed, 5 May 2021 19:02:45 +0000 (UTC) (envelope-from gjb@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1620241365; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=nAsFHdKrjVtb5BQUOR32xnPi3Lko0CCMD35S+S7hst8=; b=PKBAtm1IxDx08fzNgRwJHmi393Q77JrTInGo0DmKiY0l7XQys07VxjQYYJmcl0f/7h5ZVF sontNdvJHqsnAk9xHIMFfsOe3zFgBmq09zF3ij9Y6ukxFd6x1gLuUpvAq3I691B2S6VcR3 lV91n1xeKECCziifpt38E1bJhYboJAdw3kMZL+vhn2O/Exkw+X/aPkWOXe1Yap+EvVw9GW /bRRuaVJzjgP5ysP6f5cHpkzN2k48kYJarugnKZXhVM4VOxoqw8G4jhPjeyojfLm4cXcls yaJQHpxksbIhJ5Ui+Y7lfXf0UD+UqIpiArAsYuMfLfg+fOtCRPogL6KTbNFwfA== Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id 216461B465; Wed, 5 May 2021 19:02:45 +0000 (UTC) (envelope-from gjb@freebsd.org) Date: Wed, 5 May 2021 19:02:42 +0000 From: Glen Barber To: Gordon Bergling Cc: freebsd-stable@freebsd.org Subject: Re: Building an 13-STABLE release for i386 Message-ID: <20210505190242.GD92026@FreeBSD.org> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="ZFnrwZjhZHc4CUJd" Content-Disposition: inline In-Reply-To: ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1620241365; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=nAsFHdKrjVtb5BQUOR32xnPi3Lko0CCMD35S+S7hst8=; b=wxfqk22c5njNoOks9pagv91X+YOz7QGdmZ0Y0xnmMJ9g3eDJBDtJuNXavulVz5OWjY+hkW YMFJmHIuLkoL5PySo9ESJW9TrKXiHcuw7ven8SwXlZ4aazm39Iuf+y9OKVtRFotg/HuLPe aibYtSrfmS0t08p2XF0RJyON+DPaPqXSMNkE0MjRY0UBU7iPQ5jEyLcJ0KUjHX65KXoh1G qveGfB8+r3ctRzDMJaVwsStz+7lSbSn3TulIIjhHXzRJTn20Fp5FjIIP2oAwZyk3YfAkL9 qcogWFYQJxjCKXDSYDgmoVD9nej8nFC0/ZCfAhR/L60tEocDPdhErHkeyJ1rhw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1620241365; a=rsa-sha256; cv=none; b=YQ45ESHsEaNCxxk4s1G9wbzGAAnF/zBbnDHKZTcFS5rjgdnnX/fblhlAfdpoZAmQQciubc kiCOkxAaSoPN/4SJy+hy4RJxGgPnfaV1TlLrq4Gnm8ZzQcFRVmhIPTdiwsm9ocZc+4GXZ6 ZPsPfquqwLY+C4TczKobizembthQbJZffMxN3cyqJgdRphlCM6CeqJG6a8cXrowO/RsDDR kDflPPpk/xWaNlXPdnndTGvHkfCruTNDRp9BcWBFr/Nr6ly9Inmwsh2xPmZMH8c+C4affh /WlbyKAI1XjFaWpA5XajzSMLAe2zoXbHkh/EGEI4167PnQg7d5o1KwOq3lweqA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 May 2021 19:02:45 -0000 --ZFnrwZjhZHc4CUJd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, May 05, 2021 at 08:56:58PM +0200, Gordon Bergling wrote: > Hi, >=20 > I am currently try to build a custom 13-STABLE release for i386, build on > an amd64 system. According to release(7) the following command should > do the trick, but it fails with the following error message. >=20 > $ doas sh release.sh -c i386/i386.conf=20 > ld-elf32.so.1: Shared object "libedit.so.8" not found, required by "sh" >=20 > Has anyone an idea what could cause this? >=20 Do you have lib32 compatibility installed on the build host? I.e., do you have WITHOUT_LIB32 defined in your src.conf? Glen --ZFnrwZjhZHc4CUJd Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEjRJAPC5sqwhs9k2jAxRYpUeP4pMFAmCS69IACgkQAxRYpUeP 4pOU0A//bDBH68n4Ye/79pt410Pj4KiEEP2JCfrkkIDtZnD4krLkMDbFr3vfOYyE wLIptsiAhEPWRRK4+Zx9+wuTPTFdqXjbKXDrqeKK5ASC8E2QnRx7RnyB97uPYpCY bDvAsTI2WK4DnMldTKKnZD9WtR2uChGk+yriKwACTgPQp8e0pAzvZK5dmi4fnqMQ UHbEx0Nr6u7kwN0OsytdkanrV52ipbm9XA6zFvO50VmJ7VpbYyRK+t6tXmKHxWat ccrktHAN7UbUvkKRcPJeTyiLTsdYOPDVTC4GOlBAhSNO5oeDubDruz9Ll1AqNSpe C/7eXxzK/Muz69EwWHix7Mamt+xawurl/hqYOAkkPSUT4X3B4JNMDdt5haHXDjnM kJnLG0HCvGehpWEHidoDu0iiQ+dWoVZ8jrSVi1yNLXZyRIgO0WldiUu5WtgpFWy1 BuKVQzHHEMaxNZknplNufjnR9lPMWNd3neNL0FEE1AZKSOWxjKPWvD7pLqYvpcLN Hn9mz9Y7P8XoEGJkHLOTxztJUiKlDV+tJSEraT4kIGeWlu1hwgEQfuU9+C5pNfgc y+sGQp6uFGvA3Gyq7BVYGRPNikDFm/UxSqTOBYdhFIuUNwHz10j3ICO1esSIcF4X wxT1UMOoi+bc+5fI3FrZMWFFGlOg11C7nhGdKa8Lq6QhZXfUt9U= =/q1U -----END PGP SIGNATURE----- --ZFnrwZjhZHc4CUJd-- From owner-freebsd-stable@freebsd.org Wed May 5 19:16:22 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3510E6211C1 for ; Wed, 5 May 2021 19:16:22 +0000 (UTC) (envelope-from gbe@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fb5zV12zdz4fJT; Wed, 5 May 2021 19:16:22 +0000 (UTC) (envelope-from gbe@freebsd.org) Received: from localhost (p200300d5d70db0ef3d2b701b60ea2a15.dip0.t-ipconnect.de [IPv6:2003:d5:d70d:b0ef:3d2b:701b:60ea:2a15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: gbe) by smtp.freebsd.org (Postfix) with ESMTPSA id B6D98212C4; Wed, 5 May 2021 19:16:21 +0000 (UTC) (envelope-from gbe@freebsd.org) Date: Wed, 5 May 2021 21:16:19 +0200 From: Gordon Bergling To: Glen Barber Cc: freebsd-stable@freebsd.org Subject: Re: Building an 13-STABLE release for i386 Message-ID: References: <20210505190242.GD92026@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210505190242.GD92026@FreeBSD.org> X-Url: X-Operating-System: FreeBSD 12.2-STABLE amd64 X-Host-Uptime: 9:12PM up 1 day, 2:42, 4 users, load averages: 0.24, 0.24, 0.18 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 May 2021 19:16:22 -0000 On Wed, May 05, 2021 at 07:02:42PM +0000, Glen Barber wrote: > On Wed, May 05, 2021 at 08:56:58PM +0200, Gordon Bergling wrote: > > Hi, > > > > I am currently try to build a custom 13-STABLE release for i386, build on > > an amd64 system. According to release(7) the following command should > > do the trick, but it fails with the following error message. > > > > $ doas sh release.sh -c i386/i386.conf > > ld-elf32.so.1: Shared object "libedit.so.8" not found, required by "sh" > > > > Has anyone an idea what could cause this? > > > > Do you have lib32 compatibility installed on the build host? I.e., do > you have WITHOUT_LIB32 defined in your src.conf? > > Glen lib32 compatibility should be installed, the src.conf of the buildsystem (recent 12-STABLE) only has the following entries defined: WITH_PIE=1 WITH_RETPOLINE=1 --Gordon From owner-freebsd-stable@freebsd.org Wed May 5 19:33:25 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D83BD6219CE for ; Wed, 5 May 2021 19:33:25 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fb6M95qGpz4fr8; Wed, 5 May 2021 19:33:25 +0000 (UTC) (envelope-from gjb@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1620243205; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=a5DEE0s8HkMvIMxB5xe8BH9USs7oTQ+R25CMTRgstyg=; b=v76w5Ml9fMcLHxp1VbKuzkiPXZ/KqzKvKCN+RDyXw4jc5fRE1+JZJE62zgjk+FTTB2R5fz 4oIRXzrJP4K6IZ3Mrnz+CW3buWF1uYTKmHr4U7q45Epcfd7naTIYtG0hPMkb9uYjGUsQLW pR//qJCaKdAXZnfUvZser1j7yQ2+v8+4yXUF4ryofjcDlbcrK5vjgdHbegE6LVEQk8GZWG uw2kgvu8T9vVMalL/MndfGQx3/uso9bjovXSDU3bw/Ufx2P0hTY4tDbnlzeNDJRtRx39aa YdamUVNBp9GVZtGpqsVuRqhDO/kvdsprU/lRF89N/Q2tQe1gsWZ9pdFPxYuobQ== Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id 67C0D1B6FB; Wed, 5 May 2021 19:33:25 +0000 (UTC) (envelope-from gjb@freebsd.org) Date: Wed, 5 May 2021 19:33:23 +0000 From: Glen Barber To: Gordon Bergling Cc: freebsd-stable@freebsd.org Subject: Re: Building an 13-STABLE release for i386 Message-ID: <20210505193323.GE92026@FreeBSD.org> References: <20210505190242.GD92026@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="kK9AL8AnBmMfbzhN" Content-Disposition: inline In-Reply-To: ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1620243205; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=a5DEE0s8HkMvIMxB5xe8BH9USs7oTQ+R25CMTRgstyg=; b=xSCPzw0QcT/ZLmKbNf6esJFIcn4R3vT0lC95qeAB8FHOxeLeniglTReeZ8DKMiXynM19fa DyycPRcZnxRs0xTCOIXk5t99sioccoZjajB6OyOcLagLfz+q+y15H+gRmjL+nzwTiO3ttP D5JF0Y00lIRWHBKhHQSaS92pqIguvq9pviYpwu2SssBcVXLARcLWRwwc1Y4oRTZiIEFLUB Cw/7vJb9GgcFX7GMgse0UNHBldm3wYGWm936Zu5Sf4sT5jLJYHMimlwy4128Du2GghDiP9 xmgAccszmJSg2omQLZJhIf14kByD+jO+kK/15LECETy4OAk5HjZJAP6ar6ifPw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1620243205; a=rsa-sha256; cv=none; b=mKR2OzPMLCWAs8CtymKl7CKAo1j7xpe1ke+W1ajqAaNvxd9oj5T6Lc9M7Q2BvChsYwVuyn QJihpEFhYNLvEb9eRzaGsRNCYFjd10Db98MIZZlzuAGkMmREqT7gMpBBgClaoCtz79GgyE d6ToUrqnE61gG98Av3XPIF4V1Eddy84zCz1RH8Oy+z/roMMdY/81tWgvWYEeBoQv82a/GG BdTxWMUEC+9qtfL0sQilrOu9ficpKCeMeS3Z6kYF8dSGK0CMdJhI6PFpWwHUwgarSMQ1b/ gAUFoFNi/px//1EohBWtgJanHAai0+9pm9MXMoXz7ZT+85ZJ9ixZsXuBynhnFg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 May 2021 19:33:25 -0000 --kK9AL8AnBmMfbzhN Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, May 05, 2021 at 09:16:19PM +0200, Gordon Bergling wrote: > On Wed, May 05, 2021 at 07:02:42PM +0000, Glen Barber wrote: > > On Wed, May 05, 2021 at 08:56:58PM +0200, Gordon Bergling wrote: > > > Hi, > > >=20 > > > I am currently try to build a custom 13-STABLE release for i386, buil= d on > > > an amd64 system. According to release(7) the following command should > > > do the trick, but it fails with the following error message. > > >=20 > > > $ doas sh release.sh -c i386/i386.conf=20 > > > ld-elf32.so.1: Shared object "libedit.so.8" not found, required by "s= h" > > >=20 > > > Has anyone an idea what could cause this? > > >=20 > >=20 > > Do you have lib32 compatibility installed on the build host? I.e., do > > you have WITHOUT_LIB32 defined in your src.conf? > >=20 > > Glen >=20 > lib32 compatibility should be installed, the src.conf of the buildsystem= =20 > (recent 12-STABLE) only has the following entries defined: >=20 > WITH_PIE=3D1 > WITH_RETPOLINE=3D1 >=20 Hmm. I can't see why this would be failing for you then. I routinely do this every week for the development snapshots. Is there a chance your system is somehow mismatched regarding userland and kernel? What version is the build host running? What does 'uname -UK' show? libedit(3) was bumped on Feb 1, 2021 following an update to ncurses(3). My gut tells me these may be related. Glen --kK9AL8AnBmMfbzhN Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEjRJAPC5sqwhs9k2jAxRYpUeP4pMFAmCS8v0ACgkQAxRYpUeP 4pOzbw/9HKvIsMPY35wFLkKLikfPKMfouE0cl5e3p3KIONR1ydw/swpZJWJ3VsHC swGMByPG79TlOEXqBDVMDwcZ+4SYBSZoEHahRIv4SaI3umNpLkjt7QMGb4sEAnoS qGNMR4HwVF7H3WE6+iD3ds9QdgRbxOjjD8j2YbySH4ULLkjmBtkk5W6GTz7lz3yE IIcjpM5HZPlqxRCBpmO/ixDK0vdZjcxPGYP6W3K8usKPeZsrUhbLb2Y3V8QTE6Wb T0xAzc9fKc/dtm/bBG5n8orkV8ygIni6WhMlmUqPe7VhZhOaDGiZ2CLUecqVfIj8 A0B6aqwOXF2L7Njat0KhAGFpoKFW2ca+Iw0KU+6DLr9TJKILswvsmQ3iAB3e/fU8 eDZAWcdM9UrBFD0QUIgVjkz1gA2flLUQgIFijKk2BhtRc4iRZ1BbPh5K/lllx6oo thF6HnoxE685Noie1pTGwc/xxzCfIf0JjdUQT5SY7yjrgU6B8Gsa6IKeE5WwISsg MYIZ/u5I1eH+luSwUoNdDIJVU20Gz2cmCF+6EYar/g/kqr4EGxqq72vb7hQH014h CFf2KBTTsp7d5YcmZnYQlkgOQDcnWiROR8jlLEsaaaUC0uZIESBIQv8IvVvEAZAL NUwjYJAoYPU1tfp+cWgGQwvf4H+tnGgNNmx4v8XoUlmIYqmmnXo= =mxvE -----END PGP SIGNATURE----- --kK9AL8AnBmMfbzhN-- From owner-freebsd-stable@freebsd.org Wed May 5 22:53:52 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 287DF625CB8 for ; Wed, 5 May 2021 22:53:52 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic314-20.consmr.mail.gq1.yahoo.com (sonic314-20.consmr.mail.gq1.yahoo.com [98.137.69.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FbBpR1MVZz4pkx for ; Wed, 5 May 2021 22:53:50 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1620255228; bh=CFPeakQxkhhc43r4iHuYAJdyvxoWmMBxR74Dm+NLCPD=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=UlhrWHjQznMUWDz3O7wtbEwv4LtGBWRN0YZ8QyLzSSw5fg5hX55d00RzcSUbUMHFjfey+UTQNrFA9r1SSWc3JVS/TXIvsOYgecpSrAL2vU6Jlgz0NCrKpXhbrIK1zUQxCYCcRgsDmpOch0yc/kjSCd02sl/7tgmnI2rvekynNEe93a+9jMPOIETC4chp5IJ3rUwvvRqcs0Sjz9lfEDQPUA+Dc+JC4nP3uVjyAfqr3wEQwIpRWmMLnnRfQvzEtdBqQTVpqgOMcm9wYTfBXtUQ6Hl0eaFqTljcJnLMjzQG7qq70tso9+RHzQXT1gBsXt7LRJ+fO7tVO0piCY3QWLt4tw== X-YMail-OSG: hF4mTBgVM1ksfidG9AovH.pRaTzRhg4yhsWu81L_B6ySGHtZXr4wtt8gtTPBACv Cq88SdvEd.vg1qG1GW5ih7BNY9KMR2ttv56q7lOaESuSjHlK_BX9iom._bbUM0JbeJWVcSafR2Gz 28fcJn9Auq6o87GmWTtcZNxjTx0_rJMlkNYicO7NcTc23gATEB9DSHb7X4loifAZ8IClnc8gjKI0 r41OMAkL7Cjbz2K2l74pjQL5NwEaJzRIz1ID1PdxN86krFo3O7jO5qBQ6zzqDr2IsYF9D5nb8Pgf z8mIYctADdNtoWMkksuFo.o2uXcTWLQYhCxb1SDcSalPB2BQI9Gy88zfodfRJL4CyhAPLzcBU7kE HMNVL9zIls4AMESZG2clloHUXajWqTtGoSLV5Rhv0gr7Q5STvd6fGcC6elgZUA_j3iNiX63587c8 OCuKXUxU39_bBYsc2LfKO4MHuQsQjLB6nZq1CBEmNawNJ0w2l02klBwY6wjIvh631s9JdRlvOqhF Oc7JQJY7MjKhMfSgvi3d9iEAxlt70RFY6qSmcxva2qJ4MGQGg5l4vxbgBio5qgifn4ccBKdIHsRs Bij4sghDXtxtwyb3B3NOtxn0dQZBCbVrLF4hEf72jOUu0T2e7y5xcUxeIjfh4CkVghQoNqZ4Yjw4 00oZbaUNKDegUxMA4OiQSRLqopjIZszmp8aKelhW7BUoEuN.Gp0iQ2Ia2yFWz4L7wdcSww2gheXt ZQpFaq7ORrOoPbKi_w59OSe0_S.BW95BlBen8F7L3OxUZW4.BZXPj4w.njjIydXvouIh.qWO4DfY iJQePzM.Kp2JpvzLnquq1z5Pd6OtH2veFwW3h5rFnukoJZkChXbMNrXal3Gznnkj0mAopA1c4bDq KxjifQvYODT3NUGyeg_gvCqnllIhOEAPiH80VxbAt.EddNW4huciKaKpvR.RyLs6.nnF6zZ9t8kE v54u876C4XidT44aRbfP8JOS39qa3GnFULfx.nN3O9YQhA4eHQ2L7IMv7BR6Ki3Bkn7wBKPX2SL0 1CLKl07_IVtuhU9YodWwbtbZMpiiV2SgxP2K8UZiKVZ27IygNM5qn_PWNOdlmDsIva5VcCmcfL1M c0TyBvY87r4d2jYpjiMZ8cHkS1jrEGThJlWfZFJVvnYvq4JH4oL2Fz1GcghUWUJUMrPd9H40HJzO DrGcAkhKezCaiSDQ71STQvt4zI2yEWXWh5GxkcPeUVioJM0QyhGzliH3FLIzJ5tdhgN9_KM8nzYM wed24Ch9tS0DIsHCfCQYg5I6gqJyy9vzJAHJM7fgMLu6tkw7W0dUEoCR2see73XQIRzm5Iz_fF3M WQ5v8VJT8HQMUf7Tseu9.qXbZUQhQl3vYyoRrEyKK4wBeX_fofjA8nZtgoO.pn3IerK1eZW1pcVu SIkMw8n1jhTzJZf7B5SndknMBUNFr4Sf6U3I9xLtPCZCK3kGTqoPrz3mp3kckYtRR1ZAMgeplPys R4yPZnnCMmgzFTcNWolTola8EDg.8OxECPsNRxnJOvSb2aBTcXcgzmThzx0I5yB8QU475IfMIlVE IVjfUrElJN_WP6r3kxF_WiekTYzf0lptGMjrBeUvH5RN5S6867RyltAbaLDb7RSs1V8Z7L4oZcK2 7MIDfQTLVanzmc3IkFPRkoaFTzCaMAQj8qps6Rr48x5nmDL4U7w.yJlWhJ3OQGEe2JdDk7hFC7uN GcrSjfVrPUNfZXrmVSdbFyigZbs8AF0ZbuaiuOd8ePSr8pOB0WAXvBTViLXRHv9WDPVcdKpkJ4nT inKA_VU0t6nAx0b8hCc5a.LfaKSLZFqOo_LI9nF.9tKVqVrTykOWARRPy2gwrViMQYXF5CK4BI1d AYYEwGYmD.hh3VSxyLQlYWc8Zug.Cv9kpvdMOKeYnLw3CIgo2WSFRa92UrR5wv2t3iQU14HC2tEy i5kRcfBR.vKcD0MSbcPnIph3Ln2PZZlcDbE4A3KeQTLDQuRVigj2v46VUuEyNj5kcaJ4U8L4eijo 0gbCEEJDXjWgcvpoBKOy.2i0Nhefj5X6ajpwV4kZuFuQEpJnwlpBwPgGIsfAiYb8p_nooxJVeXUY M2eEczZ.8k4mVFJ5S97P9k2uD_pit2CmuNqx_DPeKLGLoKf5OYuakHxhgpN0eEC4grCYAJm7Rong WK6GKYvCORtIWTmr5kddB2PhpUblzM.mdNSO0X8hBS_MeMAQWorMN0T5bJXpyRTSAru6iXVybn1w J9EBjUXJtsZb6tTEvM9biugvCoun2_ehAXnB7ePhcUv_5mWUt_yHTmNTziHAL9IF6Cy1vrM3Ttld GRyIfQPMwDS2LfNP7HkO1c934qpU6COTIRjcpx71aFZEmYvO3ksT.iH4P4YBuXjdZNT1Axz_3f2p F0Bpm6wVKWNPq6HW0_v8ubTQ14wr0t93Qb6uXT4c.ygAwjlOE5tWlwHmbOS0Kob2Qmcq4CK7dnCY Rv_6Z4zG3IN99TqUrku93sxwq4lL2gCjx6ztfa4wKYaWKMZigvojlPHrkEOsza1dF6MiWy_rO1Up y.lOlBeTOtVQDfQdUM56XOSr7urqdPogHE0_i_efqV31Hluf_4HwNjblK2TlA3h6C7gxdIleM97I qkwCyJlLupq5EiZ9Pr3Zvfevwf4tkKGy9yJGYOhV23PWMQolnyg5Bp7zVYtfiYOKxP5lBeFDF X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.gq1.yahoo.com with HTTP; Wed, 5 May 2021 22:53:48 +0000 Received: by kubenode513.mail-prod1.omega.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 6f17629e21a7be59e78048d241fe079b; Wed, 05 May 2021 22:53:44 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: Re: ZFS rename with associated snapshot present: odd error message From: Mark Millard In-Reply-To: <5294FCAA-14C8-45CB-B34A-B4D76F70AA8B@yahoo.com> Date: Wed, 5 May 2021 15:53:41 -0700 Cc: FreeBSD-STABLE Mailing List , freebsd-current Content-Transfer-Encoding: quoted-printable Message-Id: <3B6DD415-0352-4D5C-88A4-F3D07B082FBE@yahoo.com> References: <8335C81D-B83C-42BC-B296-C05FAEAE538A.ref@yahoo.com> <8335C81D-B83C-42BC-B296-C05FAEAE538A@yahoo.com> <4ddf96da-e7f7-663d-8539-04f91297389a@FreeBSD.org> <5294FCAA-14C8-45CB-B34A-B4D76F70AA8B@yahoo.com> To: Andriy Gapon X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Rspamd-Queue-Id: 4FbBpR1MVZz4pkx X-Spamd-Bar: - X-Spamd-Result: default: False [-1.92 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.69.83:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_SPAM_SHORT(0.58)[0.584]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[98.137.69.83:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.83:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.83:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 May 2021 22:53:52 -0000 On 2021-May-5, at 05:28, Mark Millard wrote: > On 2021-May-5, at 02:47, Andriy Gapon wrote: >=20 >> On 05/05/2021 01:59, Mark Millard via freebsd-current wrote: >>> I had a: >>> # zfs list -tall >>> NAME USED AVAIL REFER = MOUNTPOINT >>> . . . >>> zroot/DESTDIRs/13_0R-CA72-instwrld-norm 1.44G 117G = 96K /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm >>> zroot/DESTDIRs/13_0R-CA72-instwrld-norm@dirty-style 1.44G - = 1.44G -. . . >>> . . . >>> (copied/pasted from somewhat earlier) and then attempted: >>> # zfs rename zroot/DESTDIRs/13_0R-CA72-instwrld-norm = zroot/DESTDIRs/13_0R-CA72-instwrld-alt-0 >>> cannot open 'zroot/DESTDIRs/13_0R-CA72-instwrld-norm@dirty-style': = snapshot delimiter '@' is not expected here >>> Despite the "cannot open" message, the result looks like: >>> # zfs list -tall >>> NAME USED = AVAIL REFER MOUNTPOINT >>> . . . >>> zroot/DESTDIRs/13_0R-CA72-instwrld-alt-0 1.44G = 114G 96K /usr/obj/DESTDIRs/13_0R-CA72-instwrld-alt-0 >>> zroot/DESTDIRs/13_0R-CA72-instwrld-alt-0@dirty-style 1.44G = - 1.44G - >>> . . . >>> Still, it leaves me wondering if everything is okay >>> given that internal attempt to use the old name with >>> @dirty-style when it was apparently no longer >>> available under that naming. >>> For reference: >>> # uname -apKU >>> FreeBSD CA72_4c8G_ZFS 13.0-RELEASE FreeBSD 13.0-RELEASE #0 = releng/13.0-n244733-ea31abc261ff-dirty: Thu Apr 29 21:53:20 PDT 2021 = root@CA72_4c8G_ZFS:/usr/obj/BUILDs/13_0R-CA72-nodbg-clang/usr/13_0R-src/ar= m64.aarch64/sys/GENERIC-NODBG-CA72 arm64 aarch64 1300139 1300139 >>=20 >> Cannot reproduce here (but with much simpler names and on stable/13): >> zfs create testz/test >> zfs snapshot testz/test@snap1 >> zfs rename testz/test testz/test2 >>=20 >> All worked. >>=20 >=20 > I've noticed that sometimes in my explorations it has been > silent instead of complaining. I've no clue at this point > what prior activity (or lack of activity) makes the > difference for if a message will be generated vs. not. One difference in context is that your above sort of sequence generates the after-snapshot context (using some things I have around now): zroot/DESTDIRs/13_0R-CA53-poud 1.45G 127G = 1.45G /usr/obj/DESTDIRs/13_0R-CA53-poud zroot/DESTDIRs/13_0R-CA53-poud@test 0B - = 1.45G - where my example had something more like (hand edited the above just for illustration): zroot/DESTDIRs/13_0R-CA53-poud 1.45G 125G = 96K /usr/obj/DESTDIRs/13_0R-CA53-poud zroot/DESTDIRs/13_0R-CA53-poud@test 1.45G - = 1.45G - before the rename. In other words, I'd updated the original (almost?) completely after the snapshot (as a side effect of my overall activity). It was only later that I tried the rename to track a new purpose/context that I was going to switch to. I'm not claiming that such is sufficient to (always? ever?) reproduce the message. I'm just pointing out that I'd had some significant activity on the writable file system before the rename. Some of my activity has been more like your test and I'd not seen the problems from such. But it is not a very good comparison/contrast context so I'd not infer much. I still can not at-will set up a context to produce the messages. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-stable@freebsd.org Wed May 5 23:40:11 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 93F07626A93 for ; Wed, 5 May 2021 23:40:11 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic305-21.consmr.mail.gq1.yahoo.com (sonic305-21.consmr.mail.gq1.yahoo.com [98.137.64.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FbCqt4SpLz4rKX for ; Wed, 5 May 2021 23:40:10 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1620258008; bh=ZQlwMm+Af+F4MpjpjGpxtAMFZy+B+23qYFBrabkqJFU=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=oEPR03QVjQhCuBdydInQ2yxCw290IkgNj5IHcBaLWAJrhv+X1UGMuFsu/yKcZimHTtCDeJ88vmY5N5In65E0TGSrgjnw2JoX4/Ac37dPKe6nfmLc+5d89w7ThZWp3GIAIk5HtsTOlRblgoFuBjV3JcvW/8ZUPgIJJgyX2CrM50KQ3RzxbOG6WF6nI4Pk73eTO9v9gmTs05kw8nqfCHeYSTOEfh3XQpe36vjHJS30K2BHGq5irw4/prn3+22QVar/if6KOowIvv+Ber/wge5w1hyXX6/wr/XjoC6wRpFmjlgpAInaqIxQg9Le+ud1Yr8AEjKRvfz9qoPgnLAuiyPilA== X-YMail-OSG: WSQIWQUVM1k1Jsg8IOwpWz4IcJThgPJz.PDI8VPYtETUlZf3pDtyona_v38ptMy n9EDt6J3Z7VgJWLlkieuq7_HwsIUvDrtgjFinAtAZ7e0nmra5tFSUbzYF95ENG2ABzlTO47Xq.cc foloxLU9vM5oKvUXhUOn_QSa9I5U5xdKGk5jTknpmAboqDUyeyQQforXEJFrMmIMaoj3b40NQLct NotibOdkeJusoL3IqtLNXFxC5aHuTxFVneEF1ySscIr3nrNlS6eK6XsFYTo2qnu_I_Pzic9vrtDm 73.9_kOkgpcVZqYWRsHnzhfLPAAraVjZ1SOJZt1XewzoOLmf11VlSUSx1LVV3vOlBUiDu5BTIZhL gh7ok13EeQvZcjvrAPpPtH.gC0d_a4teiL6ZVczE6tvjsEHeTc3Na1jg0Hp_BPRK11aAHCmqSBay k.pNRUNrAFJDZTbTSo8r.OIYOvnEo4Q.KRonEn6m0eC21pSxGBb6dsBSLk9dYWZZHST9WeG_ResD .ZSuVmPD5W5YxkJloaZF.82NqmE59Mj.U1v1wVl5XT2Xsw15bWTdj8_s5722U4rl_r9_qigacSCr KaBzz9IFCx.DaKLG1deCDDw1wJ79Zq6xdJ6OEBN_P8okXs723zLGk0kxeJBqM0x4bWfRk060oEvv sbKEcqlfkCofXaX0Rqz33qQQpCnIHSXwgXf_MldURe78FsFa_p06IkJ12YRaaUo4G26hOFb3BgSW 9e8uo8KgBUV86GpY_.JwtSbb8fcpbeLNkMEYI64mDoSlpoDJJh7F29IdFDwETo.AjscJhCrc9q.1 9db44sFQ1AJr02JHB_fkaufgPnx0dtdbR7Yl98ye74J1N4cDXVqz7.Pz9YuAG1KYyXws7s3jWMhe zQFTduAMhMdWoBt7x.NRy7SlYCQ05zMvPsPn6t9H19sjX_nMC1jp7G8M_eaUpgCLuRCR4.uGBemy 3yD0.73GqRdtzrNfaQHsWBtrNhv.5L._E40ZGzCoPozsIcuH5r_e3fe2oV3rNNAiuEW3pNYdBr.b IvYgQRlHPLHoJ..enpO.ZU8tcoXGRLBfX1zcJe4T_.Nulii.VkDJuA3J.Yb1II6cc7Ud77arHHDf hvtaUx4CfnaTprg1ESGOglFeLkMZ.XBU7HMUw9BH1hBhTUg_xXZ8drLyoZ9x7iSHGkQUaMrjgUs6 EvwD8d3Tn0R3qYwkCXPX3_o5X7L4xpu8HhZnUN7QUpACyKGOWUguRxQMDAjfjCzJUqn_wgKv.jc9 hyT1x_87uZOhet5Aclub8fDrOVW74Bq0ff06EYXuY8g99y5.ikF5Zcx8x.1XOAt9wsKglOEZCWVi T..RMf.eOS.CFWfRDoEXouuYQs0hSCmyl4U6YGDdb4CeKfiIOx1MohNT66e00Fv6Ua1htU2Bf4ti 4iG7fy5hE0su0ADOEF0GbPpFE_D6pkMUUGuCQMDVA2f6HAOejaVKvilPke5sAlAzkl6ZfByMnrNH feyGrDFy.vuUP_Hb_6.Z27Gmciw.D5jvgsVd0zuVLdDHKDdcVmP2obf8Hv5LiekqcEREu37jj3HW 5MgmDKk2uL0ZqwAwmTFk8j.qnPvu5FfYM6hs1cxWG5_dVKHcGCDxKD_SVANRi0bsBjPMW3AnlNt5 Tcml5NT25Mxzim3ned.x0obMgWPxSU30IkfEohh84Hpvj_pOL3MMm4knqEcB2nI.ZUpdjVesYHCb ifLAC1kHhdZypY6O4dEHOMKScTgUEwxJayRLpHFtConYLNGE6erddB2B_P6gPgRpZZWgM0GT6XmL 3E7Vb9xBPwo5WrzTEG8c2KcBN_yppSG19r1v1VMu170dJrjMo8xJMjWG.S_qcCHEDGKllhRRR8Vn u_CmCghQykI5QnkKm_DqqodGr7UUfmkT6xorCqlm5mc19Yfmd2suRAi_5ZjX7bC.aMsIiWTQJes_ Zfc.geurwpldWlKggw1TRzXlj5jDt6jVB5dsfhvOQGaQME4rmzqu1UDHOOVBPcSzN3WHKqoUt_p_ b4tgdalqrKeXXbh4VfNGvNK37sT5WW7mI_y4mILH9Z6MNx9vrbxd5070KhB1MQc6zXuDMFHjk9u9 Zzv18QeTSjK8d.LUgSGciVKonyirki54OBfx3lC6pqlf.vKUmFqYq2FXMsuhb237xbipKhR0._qk fJMcoLfnTZ9C3aTmuMK838aeGuGU3rmJIjrHtct3U3MLvQAosl2Siz8DanbtTpWcZ2pt9vpw0aWa FBYQ9kEpZwq7w86BplAGW5V6E1juernCT9o1_TnqIvf.D.7byADFPn6p05DNsdNQ_RwRrC.Wd4Os CajCl6l0a9JaVfWoxC6eE5n.XpZWcGyhG7xcXuc28xK8Yj9MKsvKJsp4orWKVb_b.xfjnOMVjTmU FL1c83rpLijCxYLxRxffgLPyt9S1FdU.MXTpHldX8IHIhNnALh4hEnlF1snRTF9X6d.CbfVXmB6p 1hWKXv6hcNeBjjkH60BF0W21brxP23zLffEKr6U..Y3MH8F09XD2nVcZdeAp1IH_NDwND0aZLbof hUh7N27BZdlnGUepPauyJYw-- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.gq1.yahoo.com with HTTP; Wed, 5 May 2021 23:40:08 +0000 Received: by kubenode502.mail-prod1.omega.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID b4a0212dbefdb6c1e62a35d42010be7c; Wed, 05 May 2021 23:40:03 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: zpool list -p 's FREE vs. zfs list -p's AVAIL ? FREE-AVAIL == 6_675_374_080 (199G zroot pool) Message-Id: Date: Wed, 5 May 2021 16:40:01 -0700 To: freebsd-current , FreeBSD-STABLE Mailing List X-Mailer: Apple Mail (2.3654.60.0.2.21) References: X-Rspamd-Queue-Id: 4FbCqt4SpLz4rKX X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.50 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.64.84:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MID_RHS_MATCH_FROM(0.00)[]; SUBJECT_HAS_QUESTION(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[98.137.64.84:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.84:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.84:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 May 2021 23:40:11 -0000 Context: # gpart show -pl da0 =3D> 40 468862048 da0 GPT (224G) 40 532480 da0p1 efiboot0 (260M) 532520 2008 - free - (1.0M) 534528 25165824 da0p2 swp12a (12G) 25700352 25165824 da0p4 swp12b (12G) 50866176 417994752 da0p3 zfs0 (199G) 468860928 1160 - free - (580K) There is just one pool: zroot and it is on zfs0 above. # zpool list -p NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG = CAP DEDUP HEALTH ALTROOT zroot 213674622976 71075655680 142598967296 - - 28 = 33 1.00 ONLINE - So FREE: 142_598_967_296 (using _ to make it more readable) # zfs list -p zroot=20 NAME USED AVAIL REFER MOUNTPOINT zroot 71073697792 135923593216 98304 /zroot So AVAIL: 135_923_593_216 FREE-AVAIL =3D=3D 6_675_374_080 The questions: Is this sort of unavailable pool-free-space normal? Is this some sort of expected overhead that just is not explicitly reported? Possibly a "FRAG" consequence? For reference: # zpool status pool: zroot state: ONLINE scan: scrub repaired 0B in 00:31:48 with 0 errors on Sun May 2 = 19:52:14 2021 config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 da0p3 ONLINE 0 0 0 errors: No known data errors =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-stable@freebsd.org Wed May 5 23:45:56 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C6A1D627261 for ; Wed, 5 May 2021 23:45:56 +0000 (UTC) (envelope-from debdrup@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4FbCyX0Hm9z4s0W for ; Wed, 5 May 2021 23:45:56 +0000 (UTC) (envelope-from debdrup@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 04DF7627507; Wed, 5 May 2021 23:45:56 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 041F162734B; Wed, 5 May 2021 23:45:56 +0000 (UTC) (envelope-from debdrup@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FbCyW48Z3z4rtV; Wed, 5 May 2021 23:45:55 +0000 (UTC) (envelope-from debdrup@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1620258355; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=rrJ/C5LwQHzaG3LJF6uvUhFGPQ0tjX/3ofQaTQ3CVqs=; b=hb018o/V/uv/88ed98JfF8ar9h5kUbT0P3VpYvg6su6YyuHTnieLVL76rlO5qGWAZI0gqM gUGBdA/jR7AkjixU55f2qzHdUpfF3GarX3VrCkSt/2QckWS4RL+azwYf72x4g8gvkUHcZt 951Vk3YQaXqPIIwHyGiJj2vKmH0iA18Z23uGZYx9+wJ137/BaeksWKmYpX45l/8GRxNkTH 7MJMvZKns+6oQKGiG2s4iOODV9VwLGcG6spVvqaV4A6oNVYneB85zj6GMjIrZ3T+pZa8KM BFfr7IT1sJqHjfT+V4ongiBYArKz/UmsWi0ns1rKs3kAtRX7CkzbG8rhIah0SA== Received: by freefall.freebsd.org (Postfix, from userid 1471) id 827D01C061; Wed, 5 May 2021 23:45:55 +0000 (UTC) Date: Thu, 6 May 2021 01:45:53 +0200 From: Daniel Ebdrup Jensen To: hackers@FreeBSD.org Cc: current@FreeBSD.org, stable@FreeBSD.org Subject: FreeBSD Quarterly Status Report - First Quarter 2021 Message-ID: <20210505234553.br4gjrtkrjewezzs@nerd-thinkpad.local> Mail-Followup-To: Daniel Ebdrup Jensen , hackers@FreeBSD.org, current@FreeBSD.org, stable@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="p2qii6v5xswo6umh" Content-Disposition: inline ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1620258355; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=rrJ/C5LwQHzaG3LJF6uvUhFGPQ0tjX/3ofQaTQ3CVqs=; b=MU47EcyRiUfXLP90bn8A1Wfo7iWT/gxmlkuro27nq8ykDc4XMSFiHkiGrqBe6ItUQMRD60 Pbz5FWTjhrxaC+sLH21G0yznXX3g9+2GWF+L85oWbqEgncM+3zd9tr/hlDd7+h0OSqwgtS x6FISe651Wx6e/6Vl+/ks/t+NqFcmVayujsFPWIcNhLu6gnNm80iWai7Ku372RUFbk3apO x6H4A8gCzBD/tSUWPymU7Axcc4VtFqhBGPE2X6F8glqsC0YWfqKIz4lze1oB1tUREqknJk 7FbPfdSG9nQKEHWhzUGOI7zc8kDgA1Vo9d6eZKGtOrlFRgZA16s/6WcZaHRpDA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1620258355; a=rsa-sha256; cv=none; b=cdwLTE2n5BsWtXU3jDegFsKpakkriJrc8FUrNLh9wDfdx8OYZx0Q2Ew8qi681I9xw8u3Iq 1yD37EEfCd/5BqMyA94hupckDIefHF3vNsZ2jhm10HTd1mXU8qrBX2SVClYDk5lbNGbdzS aZb6tPuG5PiOKoC+jWmsPmqkDRlBakj89C5tG49bkWqKE+E2xPf1iwp64FJD7oennHidxO 8L92Ud+mwCAS2aeQpnxxIexKSVrxt5efA4HzTwDEGJJwjKXyO1ksTqzANZ2yctPn3hZk+L FusqV2V7fRWtqxdJ07jiX3xbaA8SIIZ92xE3ZE39f0xc0luNykX+jAbxAujxUA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 May 2021 23:45:56 -0000 --p2qii6v5xswo6umh Content-Type: text/plain; charset=utf-8 Content-Description: FreeBSD status report, 2020q1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Introduction This report covers FreeBSD related projects for the period between January = and March, and is the first of four planned reports for 2021. The first quarter of 2021 has been very active in both FreeBSD-CURRENT and -STABLE, with 13.0-RELEASE work starting in January and finishing up mid-Ap= ril. It provides lots of new features, and there=E2=80=99s even a good chance th= at some workloads will experience performance improvements. The number of entries is slightly down, and this is probably due to a combination of factors like code slush as well as the ongoing issues with COVID-19, but we naturally hope that things will look up next quarter. This combined with a switch-over to AsciiDoctor and a decision to make full use = of the status report work schedule to avoid stress, means that the report can = now be expected to come out at the end of the first month after the quarter has finished, rather than in the middle. This report in particular includes a number of interesting entries, covering everything from the linuxulator, various mitigation work, long-awaited work= on OpenBSM, work on kernel sanitizers, and many more things that it is hoped y= ou will enjoy reading about. Yours, Daniel Ebdrup Jensen, with a status hat on. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Table of Contents =E2=80=A2 FreeBSD Team Reports =E2=96=A1 FreeBSD Foundation =E2=96=A1 FreeBSD Release Engineering Team =E2=96=A1 Cluster Administration Team =E2=96=A1 Continuous Integration =E2=96=A1 Ports Collection =E2=80=A2 Projects =E2=96=A1 Git Migration Working Group =E2=96=A1 LLDB Debugger Improvements =E2=96=A1 Linux compatibility layer update =E2=96=A1 Vulnerability Mitigations =E2=96=A1 OpenBSM Synchronisation =E2=80=A2 Kernel =E2=96=A1 ENA FreeBSD Driver Update =E2=96=A1 Intel wireless update =E2=96=A1 Kernel Sanitizers =E2=96=A1 Marvell ARM64 SoCs support =E2=96=A1 nv(9)-based audio device enumeration =E2=80=A2 Ports =E2=96=A1 KDE on FreeBSD =E2=96=A1 FreeBSD Office team 2021Q1 status report =E2=96=A1 VirtualBox FreeBSD port =E2=80=A2 Documentation =E2=96=A1 DOCNG on FreeBSD =E2=96=A1 FreeBSD Translations on Weblate =E2=96=A1 WebApps working group =E2=80=A2 Miscellaneous =E2=96=A1 Discord Server & Community Growth =E2=80=A2 Third-Party Projects =E2=96=A1 CBSD Project =E2=96=A1 helloSystem =E2=96=A1 PkgBase.live =E2=96=A1 Potluck & Potman =E2=96=A1 sysctl improvements =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 FreeBSD Team Reports Entries from the various official and semi-official teams, as found in the Administration Page. FreeBSD Foundation Contact: Deb Goodkin The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the FreeBSD Project and community worldwide. Fundi= ng comes from individual and corporate donations and is used to fund and manage software development projects, conferences and developer summits, and provi= de travel grants to FreeBSD contributors. The Foundation purchases and supports hardware to improve and maintain FreeBSD infrastructure and provides resour= ces to improve security, quality assurance, and release engineering efforts; publishes marketing material to promote, educate, and advocate for the Free= BSD Project; facilitates collaboration between commercial vendors and FreeBSD developers; and finally, represents the FreeBSD Project in executing contra= cts, license agreements, and other legal arrangements that require a recognized legal entity. Here are some highlights of what we did to help FreeBSD last quarter: COVID-19 Impact to the Foundation Like most organizations, our team continued to work from home. Our temporary ban on travel for staff members remains in effect, but continues to not aff= ect our output too much, since most conferences are still virtual. We continued supporting the community and Project, even though some of our work and responses may have been delayed because of changes in some of our priorities and the impact of limited childcare for a few of our staff members. Partnerships and Commercial User Support We help facilitate collaboration between commercial users and FreeBSD developers. We also meet with companies to discuss their needs and bring th= at information back to the Project. Not surprisingly, the stay at home orders, combined with our company ban on travel during Q1 made in-person meetings non-existent. However, the team was able to continue meeting with our partn= ers and commercial users virtually. These meetings help us understand some of t= he applications where FreeBSD is used. We were thrilled for the opportunity to work with AMD early on to ensure FreeBSD worked on their recently released third generation EPYC series. You= can read more about that here: https://freebsdfoundation.org/news-and-events/ latest-news/freebsd-well-prepared-for-amd-epyc-7003-series-processors/. Fundraising Efforts First, we=E2=80=99d like to say thank you to everyone who has given us a fi= nancial contribution this year! Last quarter we raised $88,237, which includes donations from organizations like Facebook and Tarsnap, as well as many individuals. We also have a few donation commitments for this quarter. Going forward this quarter, we will be reaching out to FreeBSD commercial u= sers to help support our growing efforts. At the beginning of 2021, we opened two job positions in our software development team, to increase the amount of support we are able to provide in this area. That includes increasing the amount of code reviews and bug fixes we do and adding some major features to FreeBSD, to help keep FreeBSD the innovative, secure, and reliable operating system you rely on. You=E2=80=99ll find out how we used your donations for Q1 in our report, as= well as individual reports throughout this status report. We are excited about our plans for 2021, which include more FreeBSD online advocacy and training, operating system course content, and the software development work mentioned above. While we are still in this pandemic, we= =E2=80=99re working hard to help connect folks within the community with more virtual opportunities. Please consider making a donation to help us continue and increase our supp= ort for FreeBSD in 2021: https://www.freebsdfoundation.org/donate/. We also have the Partnership Program, to provide more benefits for our larg= er commercial donors. Find out more information and share with your companies! https://www.freebsdfoundation.org/FreeBSD-foundation-partnership-program/ OS Improvements Over the quarter a total of 264 base system commits, 63 ports commits, and = 10 doc tree commits were tagged as sponsored by the FreeBSD Foundation. The Foundation also sponsored work that was committed to third-party repositori= es, including 26 commits to LLDB (the LLVM project debugger). This includes work =66rom staff members, interns, and grant recipients. In other quarterly rep= ort entries you can read more about some of these sponsored projects, such as L= LDB and other kernel debugging improvements, and kernel sanitizers. As usual, staff members committed numerous bug fixes, minor improvements, a= nd security patches. Focus areas in the kernel included virtual memory, x86 pm= ap, uma, tmpfs, nullfs, ffs and ufs, and job control improvements. User space work included changes to the libc, libcasper, and libthr librari= es, the run-time linker, as well as the ldd, cmp, diff, makefs, elfctl, growfs,= and bhyve utilities. Foundation staff also participated in many Phabricator code reviews, suppor= ted bug triage, integrated a number of submissions from third parties, and supported the Git transition working group. Foundation staff also supported the promotion of the AArch64 (arm64) architecture to Tier-1 status. Work included additions to freebsd-update, integration of various bug fixes, and test run issue triage. Continuous Integration and Quality Assurance The Foundation provides a full-time staff member and funds projects on improving continuous integration, automated testing, and overall quality assurance efforts for the FreeBSD Project. During the first quarter of 2021, the work was focused on pre-commit tests = and building release artifacts in the CI staging environment. The other main working item is following the VCS migration to change the src source from Subversion to Git and doc changed to AsciiDoc format. See the FreeBSD CI section of this report for completed work items and deta= iled information. Supporting FreeBSD Infrastructure The Foundation provides hardware and support to improve the FreeBSD infrastructure. Last quarter, we continued supporting FreeBSD hardware loca= ted around the world. We coordinated efforts between the new NYI Chicago facili= ty and clusteradm to start working on getting the facility prepared for some of the new FreeBSD hardware we are planning on purchasing. NYI generously prov= ides this for free to the Project. We also worked on connecting with the new own= ers of the NYI Bridgewater site, where most of the FreeBSD infrastructure is located. FreeBSD Advocacy and Education A large part of our efforts are dedicated to advocating for the Project. Th= is includes promoting work being done by others with FreeBSD; producing advoca= cy literature to teach people about FreeBSD and help make the path to starting using FreeBSD or contributing to the Project easier; and attending and gett= ing other FreeBSD contributors to volunteer to run FreeBSD events, staff FreeBSD tables, and give FreeBSD presentations. The FreeBSD Foundation sponsors many conferences, events, and summits around the globe. These events can be BSD-related, open source, or technology even= ts geared towards underrepresented groups. We support the FreeBSD-focused even= ts to help provide a venue for sharing knowledge, to work together on projects, and to facilitate collaboration between developers and commercial users. Th= is all helps provide a healthy ecosystem. We support the non-FreeBSD events to promote and raise awareness of FreeBSD, to increase the use of FreeBSD in different applications, and to recruit more contributors to the Project. Wh= ile we were still unable to attend in-person meetings due to covid-19, we were = able to attend virtual events and began planning for the online Spring FreeBSD Developer Summit. In addition to attending and planning virtual events, we = are continually working on new training initiatives and updating our selection = of how-to guides to facilitate getting more folks to try out FreeBSD. https:// www.freebsdfoundation.org/freebsd/how-to-guides/ Check out some of the advocacy and education work we did last quarter: =E2=80=A2 Presented a workshop at Apricot 2021 =E2=80=A2 Staffed a virtual stand at FOSDEM 2021 and created a what=E2=80= =99s new in 13.0 video to accompany the stand =E2=80=A2 Staffed a virtual booth and was a community sponsor for FOSSASI= A 2021 =E2=80=A2 Participated as an Industry Partner for USENIX FAST =E2=80=9821 =E2=80=A2 Committed to be an Industry Partner for USENIX Annual Tech, USE= NIX OSDI, USENIX Security and USENIX LISA =E2=80=A2 Continued to promote the FreeBSD Office Hours series Videos fro= m the one hour sessions can be found on the Project=E2=80=99s YouTube Channel: ht= tps:// www.youtube.com/c/FreeBSDProject. See the Office Hours section of this report for more information. =E2=80=A2 Worked with the organizing committee to begin planning the Spri= ng FreeBSD Developers Summit. =E2=80=A2 Continued recruiting for the FreeBSD Fridays series. The series= will return in May. =E2=80=A2 Participated in an interview with The Register about FreeBSD 13= =2E0 highlights. https://www.theregister.com/2021/03/10/the_state_of_freebsd/ Keep up to date with our latest work in our newsletters: https:// freebsdfoundation.org/our-work/latest-updates/?filter=3Dnewsletter We help educate the world about FreeBSD by publishing the professionally produced FreeBSD Journal. As we mentioned previously, the FreeBSD Journal is now a free publication. Find out more and access the latest issues at https= :// www.freebsdfoundation.org/journal/. You can find out more about events we attended and upcoming events at https= :// www.freebsdfoundation.org/news-and-events/. Legal/FreeBSD IP The Foundation owns the FreeBSD trademarks, and it is our responsibility to protect them. We also provide legal support for the core team to investigate questions that arise. Go to http://www.freebsdfoundation.org to find out how we support FreeBSD a= nd how we can help you! =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 FreeBSD Release Engineering Team Links: FreeBSD 13.0-RELEASE schedule URL: https://www.freebsd.org/releases/13.0R/ schedule.html FreeBSD development snapshots URL: https://download.freebsd.org/ftp/snapsho= ts/ ISO-IMAGES/ Contact: FreeBSD Release Engineering Team, The FreeBSD Release Engineering Team is responsible for setting and publish= ing release schedules for official project releases of FreeBSD, announcing code freezes and maintaining the respective branches, among other things. During the first quarter of 2021, the Release Engineering Team started work= on the 13.0-RELEASE cycle, the first release from the stable/13 branch. As of = this writing, the release is progressing smoothly, with one additional BETA build and two additional RC builds added to the schedule. The schedule has been updated on the FreeBSD Project website to reflect the updates. Additionally throughout the quarter, several development snapshots builds w= ere released for the head, stable/12, and stable/11 branches. Development snaps= hot builds for stable/13 will be available after the 13.0 release. Thank you to all that have helped test the 13.0 builds up until this point = and have reported issues. As always, we strive for quality over quantity. Sponsor: Rubicon Communications, LLC ("Netgate") Sponsor: The FreeBSD Foundation =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Cluster Administration Team Contact: Cluster Administration Team Links: Cluster Administration Team members URL: https://www.freebsd.org/administra= tion /#t-clusteradm The FreeBSD Cluster Administration Team consists of the people responsible = for administering the machines that the Project relies on for its distributed w= ork and communications to be synchronised. In this quarter, the team has worked= on the following: =E2=80=A2 Installed a new package builder =E2=80=A2 Added Git support to cluster management scripts =E2=80=A2 Put local Git mirrors on the universe machines =E2=80=A2 Replaced disks in the UK mirror =E2=80=A2 Replaced a disk in pointyhat (https://pkg-status.freebsd.org) =E2=80=A2 Recycled some old dead-weight servers eating up rackspace and p= ower at our primary cluster site =E2=80=A2 Upgraded developer reference platforms =E2=96=A1 ref{11,12,13,14}-{amd64,i386} =E2=96=A1 universe* =E2=80=A2 Installed two new aarch64 machines =E2=96=A1 ref12-aarch64, ref13-aarch64, ref14-aarch64 =E2=96=A1 security-officer aarch64 freebsd-update builder =E2=80=A2 Worked with asciidoc project to update https://www.freebsd.org = and https:// docs.freebsd.org =E2=80=A2 Installed a new mirror server in Brazil, sponsored by nic.br =E2=96=A1 gdns points everyone from South America to this mirror =E2=96=A1 complete {download,ftp,pkg}.freebsd.org mirror =E2=80=A2 Helped rmacklem@ participate in this year=E2=80=99s NFS Bakeath= on interoperability testing event by providing a cluster machine to the testing VPN =E2=80=A2 Ongoing day to day cluster management activity =E2=96=A1 Putting out fires =E2=96=A1 Babysitting pkgsync Work in progress: =E2=80=A2 Move pkg-master.nyi to new hardware =E2=80=A2 Fix git fallouts =E2=80=A2 Upgrade cluster hardware =E2=80=A2 Upgrade developer-facing machines to 14-CURRENT =E2=96=A1 Install ref14* machines =E2=80=A2 Improve to the package building infrastructure =E2=80=A2 Research and test migration away from mailman2 =E2=80=A2 Work with Git migration working group for ports tree migration =E2=80=A2 Review the service jails and service administrators operation =E2=80=A2 Improve the web service architecture =E2=80=A2 Improve the cluster backup plan =E2=80=A2 Setup powerpc pkgbuilder/ref/universal machines =E2=80=A2 Prepare for a new mirror site in Australia, to be hosted by IX = Australia =E2=80=A2 Search for more providers that can fit the requirements for a g= eneric mirrored layout or a tiny mirror =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Continuous Integration Links: FreeBSD Jenkins Instance URL: https://ci.FreeBSD.org FreeBSD Hardware Testing Lab URL: https://ci.FreeBSD.org/hwlab FreeBSD CI artifact archive URL: https://artifact.ci.FreeBSD.org FreeBSD CI weekly report URL: https://hackmd.io/@FreeBSD-CI FreeBSD Jenkins wiki URL: https://wiki.freebsd.org/Jenkins Hosted CI wiki URL: https://wiki.freebsd.org/HostedCI 3rd Party Software CI URL: https://wiki.freebsd.org/3rdPartySoftwareCI Tickets related to freebsd-testing@ URL: https://preview.tinyurl.com/y9maau= wg FreeBSD CI Repository URL: https://github.com/freebsd/freebsd-ci Contact: Jenkins Admin Contact: Li-Wen Hsu Contact: freebsd-testing Mailing List Contact: IRC #freebsd-ci channel on EFNet The FreeBSD CI team maintains the continuous integration system of the Free= BSD project. The CI system firstly checks the committed changes can be successf= ully built, then performs various tests and analysis over the newly built result= s. The artifacts from those builds are archived in the artifact server for fur= ther testing and debugging needs. The CI team members examine the failing builds= and unstable tests and work with the experts in that area to fix the code or ad= just test infrastructure. The details of these efforts are available in the week= ly CI reports. During the first quarter of 2021, we continued working with the contributors and developers in the project to fulfil their testing needs and also keep collaborating with external projects and companies to improve their products and FreeBSD. Important changes: =E2=80=A2 All src jobs were changed to use git to follow VCS migration. T= hanks Brandon Bergren (bdragon@) again. =E2=80=A2 Doc job was updated for following the AsciiDoc migration. New jobs added: =E2=80=A2 TCP test suite for main on amd64 =E2=80=A2 GCC 9 build for main on amd64 Work in progress and open tasks: =E2=80=A2 Designing and implementing pre-commit CI building and testing =E2=80=A2 Designing and implementing use of CI cluster to build release a= rtifacts as release engineering does =E2=80=A2 Collecting and sorting CI tasks and ideas here =E2=80=A2 Testing and merging pull requests in the FreeBSD-ci repo =E2=80=A2 Reducing the procedures of CI/test environment setting up for c= ontributors and developers =E2=80=A2 Setting up the CI stage environment and putting the experimenta= l jobs on it =E2=80=A2 Setting up public network access for the VM guest running tests =E2=80=A2 Implementing automatic tests on bare metal hardware =E2=80=A2 Adding drm ports building tests against -CURRENT =E2=80=A2 Planning to run ztest and network stack tests =E2=80=A2 Adding more external toolchain related jobs =E2=80=A2 Improving the hardware lab to be more mature and adding more ha= rdware =E2=80=A2 Helping more software get FreeBSD support in their CI pipeline = Wiki pages: 3rdPartySoftwareCI, HostedCI =E2=80=A2 Working with hosted CI providers to have better FreeBSD support =E2=80=A2 The build and test results will be sent to the dev-ci mailing l= ist soon. Feedback and help with analysis is very appreciated! Please see freebsd-testing@ related tickets for more WIP information, and d= on=E2=80=99t hesitate to join the effort! Sponsor: The FreeBSD Foundation =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Ports Collection Links: About FreeBSD Ports URL: https://www.FreeBSD.org/ports/ Contributing to Ports URL: https://docs.freebsd.org/en/articles/contributin= g/ ports-contributing/ FreeBSD Ports Monitoring URL: http://portsmon.freebsd.org/ Ports Management Team URL: https://www.freebsd.org/portmgr/ Ports Tarball URL: http://ftp.freebsd.org/pub/FreeBSD/ports/ports/ Contact: Ren=C3=A9 Ladan Contact: FreeBSD Ports Management Team The Ports Management Team is responsible for overseeing the overall directi= on of the Ports Tree, building packages, and personnel matters. Below is what happened in the last quarter. As always, first the quarterly dashboard: * we currently have around 43,800 ports (including flavors). * the open PR count for ports is currently 2477,= of which 532 are unassigned. * during the last quarter, 9481 commits were made= by 168 committers on the main branch, and 620 commits by 64 committers on the 2021Q1 branch. Compared to 2020Q4, the number of ports again grew by five percent, the number of open PRs dropped a bit, and the number of commits on= the main branch grew with almost nine percent. During the last quarter, we welcomed Neel Chauhan (nc@), Lewis Cook (lcook@= ), and Nuno Teixeira (eduardo@). Adrian Chadd (adrian@) who is already a src committer got a ports commit bit extension. Tobias Berner (tcberner@) asked= if he could join the portmgr-lurker program and was shortly added afterwards. We sent another mail to the ports@ mailing list outlining further plans for removing Python 2.7 from the Ports Tree. Currently all ports recursively depending on Python 2.7 are marked to expire on 2021-06-23, which unfortuna= tely includes a lot of KDE ports due to the qt5-webengine port. We are evaluating various mitigation strategies. portmgr has been collaborating with the Git Working Group over the last yea= r to prepare the Ports Tree to be converted to Git. Tasks included: * converting various scripts and tools to support Git * attending Git Working Group meet= ings * updating documentation * updating various internal and public third-party services * evaluating numerous test conversion (git-beta) results Regarding the Ports Tree itself, two new USES were introduced: * kodi to ea= se porting of Kodi add-ons * mpi for dependencies of MPICH and OpenMPI A new default version for ImageMagick was added and the default version for Julia= was removed as no Julia port currently exists. pkg was updated to 1.16.3, Firef= ox to 87.0, and Chromium to 89.0.4389.114 The Cluster Administration Team assisted with getting three new package building machines running in the build cluster. Two are for arm64 builds and one is a general builder. antoine@ was again busy with exp-runs, 28 this time, to: * test various por= ts updates * update the clang/LLVM version from 6 to 10 in USES=3Dcompiler * r= educe includes in /usr/include/crypto =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Projects Projects that span multiple categories, from the kernel and userspace to the Ports Collection or external projects. Git Migration Working Group Links: Git transition wiki URL: https://wiki.freebsd.org/git doc git repo URL: https://cgit.FreeBSD.org/doc ports git repo URL: https://cgit.FreeBSD.org/ports src (base system) git repo URL: https://cgit.FreeBSD.org/src Committers guide Git primer URL: https://docs.freebsd.org/en/articles/ committers-guide/#git-primer Handbook Using Git appendix URL: https://docs.freebsd.org/en/books/handbook/ mirrors/#git Game of Trees URL: http://gameoftrees.org/ gitup URL: https://github.com/johnmehr/gitup Contact: Li-Wen Hsu Contact: Warner Losh Contact: Ed Maste Contact: Ulrich Sp=C3=B6rlein Contact: FreeBSD-git mailing list Contact IRC #gitcvt channel on EFnet The doc and src trees were migrated from Subversion to Git at the end of 20= 20, with some additional work extending into the first quarter of 2021. The Git Working Group implemented or updated commit hooks, and prepared for FreeBSD= 13 to be built from Git. We converted draft documentation from Markdown to AsciiDoc and merged it into the committer=E2=80=99s guide and handbook. The ports repository migration to Git started at the end of the quarter, beginning with a final Subversion commit on March 31st to indicate that the conversion started. We are working on portsnap and other ports infrastructu= re and they will be finished before or soon after the migration. The Git Working Group continues to track progress on two permissively-licen= sed git compatible tools: Gitup and Game of Trees. Gitup is a small, dependency-free tool to clone and update git repositories. It is used only = to keep a local tree up-to-date, and has no support for local commits. Game of Trees is a version control client that is compatible with Git repositories. It provides a user interface and workflow that is distinct fr= om that of Git. It is in no way intended to be a drop-in replacement for git, = but can be used to develop software maintained in a Git repository. Gitup and Game of Trees are currently available as ports and packages. Futu= re work will evaluate them as candidates for the base system. In the second quarter of 2021 we expect to complete some minor remaining migration tasks. This will complete the initial phase of the Git migration,= and the working group will wind down. The core team will then begin a new effor= t to investigate and evaluate new workflow changes. Sponsor: The FreeBSD Foundation (in part) =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 LLDB Debugger Improvements Links: Moritz Systems Project Description URL: https://www.moritz.systems/blog/ lldb-freebsd-cpu-target-support-and-userland-debugging-improvements/ Progress Report 1 URL: https://www.moritz.systems/blog/ freebsd-remote-process-plugin-on-non-x86-architectures/ Progress Report 2 URL: https://www.moritz.systems/blog/ freebsd-legacy-process-plugin-removed/ Progress Report 3 URL: https://www.moritz.systems/blog/ lldb-support-for-fork-and-vfork/ Git Repository URL: https://github.com/moritz-systems/llvm-project Contact: Kamil Rytarowski Contact: Micha=C5=82 G=C3=B3rny The LLDB project builds on libraries provided by LLVM and Clang to provide a great modern debugger. It uses the Clang ASTs and the expression parser, LL= VM JIT, LLVM disassembler, etc so that it provides an experience that =E2=80= =9Cjust works=E2=80=9D. It is also blazingly fast and more permissively licensed th= an GDB, the GNU Debugger. FreeBSD includes LLDB in the base system. At present, it has some limitatio= ns in comparison with the GNU GDB debugger, and does not yet provide a complete replacement. This project aimed to finish porting the FreeBSD platform supp= ort in LLDB to the modern client-server model on all architectures originally supported by LLDB on FreeBSD and removing the obsolete plugin. After switching to the new process model, the project focused on implementi= ng support for tracing fork(2) and vfork(2) syscalls. The proposed model is compatible with the follow-fork-mode setting from GDB. On fork, the debugger can either continuing tracing the parent and detach the child, or switch to tracing the child and detach the parent. The new code makes it possible to debug child processes. It also prevents software breakpoints from leaking to child processes and causing them to crash. The introduced changes are expected to be shipped with LLDB 13.0. The overall experience of FreeBSD/LLDB developers and advanced users on this rock solid Operating System reached the state known from other environments. Furthermore, the FreeBSD-focused work also resulted in generic improvements, enhancing the LLDB support for Linux and NetBSD. TODO: we are currently working on adding a ptrace(2) request to create a co= re dump of the stopped program without crashing it. Afterwards, we are plannin= g to improve LLDB test coverage for core dump support and work on any issues we might hit. Sponsor: The FreeBSD Foundation =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Linux compatibility layer update Contact: Edward Tomasz Napierala, Linuxulator improvements have been ongoing for the last two years, with sup= port =66rom the FreeBSD Foundation over a few distinct project grants as well as contributions from the community. The goal of this project is to improve FreeBSD=E2=80=99s ability to execute unmodified Linux binaries. Current sup= port status of specific Linux applications is being tracked at the Linux app status Wiki page. The work this quarter focused on making sure the 13.0-RELEASE ships with Linuxulator in a good shape, and fixing problems reported by users. There a= re some new directories provided by linsysfs(5), the lack of which, through a curious chain of events, broke installation of make(1) in Ubuntu Focal. The getcwd(2) syscall was fixed to no longer return the wrong error value for certain conditions, which was breaking Mono. The getsockopt(2) syscall now supports SO_PEERSEC and SO_PEERGROUPS, which are being used by su(8) and su= do (8). Other fixes include flag handling for 32-bit send(2) syscall, and seve= ral ptrace(2) problems, which were affecting Steam games. The kernel version was bumped to 3.17.0 to unbreak Qt applications from Focal. The sysutils/ debootstrap port, and its corresponding debootstrap package, now correctly handle Ubuntu=E2=80=99s GPG keys. The debootstrap utility now installs the = mremap(2) workaround for apt(8). This reduces the number of steps required to set up Linux chroot or jail. Finally there have been some improvements to the star= tup scripts. Sponsor: The FreeBSD Foundation =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Vulnerability Mitigations Contact: Ed Maste Contact: Konstantin Belousov Contact: Marcin Wojtas Contact: Dawid G=C3=B3recki We added support for enforcing a write XOR execute mapping policy. It is enabled by setting the kern.elf64.allow_wx and/or kern.elf32.allow_wx sysct= ls to 0 (for 64-bit and 32-bit binaries, respectively). Binaries can indicate = that they requre writeable and executable mappings by setting the NT_FREEBSD_FCTL_WXNEEDED ELF feature bit, set via elfctl. In addition, elfctl received a few usability improvements to support use by ports, targeting different FreeBSD base system versions. We added a -i flag= to ignore unknown flags (so that the same elfctl invocation could be used on o= lder FreeBSD versions) as well as the ability to specify features by value. Flags that request opt-out of a mitigation now have a no prefix to make the sense clearer. For example, the flag to indicate that the binary is not compatible with ASLR is now named noaslr. Unprefixed flag names are still supported, for backwards compatibility, but will emit a warning and will be removed in a later version. The next step is to introduce ports infrastructure to support tagging binar= ies in ports that require special flags. Details can be found in PR252629. Another update is that the base system binaries are now built as position-independent executable (PIE) by default, for 64-bit architectures.= PIE executables are used in conjunction with address randomization (ASLR) as a mitigation for certain types of security vulnerabilities. The ASLR feature still remains opt-in, however the described change allows enabling it using only sysctl knobs, without a need to rebuild the image. Enabling PIE result= s in no material performance impact for most workloads. It is also worth mentioning that a certain number of ports inherit the base systems /usr/share/mk infrastructure, and some initially failed to build af= ter toggling the PIE setting. All issues detected by executing the exp-run were addressed. The details can be found in PR253275. The next step is to try enabling ASLR by default for 64-bit architectures. = The patch is under discussion. Sponsor: The FreeBSD Foundation Sponsor: Stormshield =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 OpenBSM Synchronisation Links: TrustedBSD / OpenBSM URL: http://www.trustedbsd.org/openbsm.html OpenBSM Github Sources URL: https://github.com/openbsm/openbsm Synchronisation with macOS Catalina URL: https://github.com/openbsm/openbsm/ commit/54a0c07cf8bac71554130e8f6760ca68e5f36c7f Apple OpenSource URL: https://opensource.apple.com Contact: Gordon Bergling OpenBSM is a crucial part of FreeBSD, which provides auditing features for = the operating system. OpenBSM is incorporated into FreeBSD and macOS. Both Apple and FreeBSD have currently made changes to the OpenBSM framework, which wer= en=E2=80=99t upstreamed. This small project aims to consolidate these changes and upstre= am them to the OpenBSM github repository, so that both development efforts can= be merged to FreeBSD later on. There is currently a pull request pending that synchronizes the FreeBSD sou= rces with OpenBSM. A comparison was made to incorporate Apple=E2=80=99s Catalina= changes. A few weeks ago Apple has also made the source code of Big Sur available. In = the latest comparison against OpenBSM Apple has made overlapping ID changes, wh= ich are making a simple import of the changes impossible. I am currently trying= to work around that issue by making OpenBSM a little vendor specific. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Kernel Updates to kernel subsystems/features, driver support, filesystems, and mor= e. ENA FreeBSD Driver Update Links: ENA README URL: https://github.com/amzn/amzn-drivers/blob/master/kernel/fbs= d/ ena/README Contact: Michal Krawczyk Contact: Artur Rojek Contact: Marcin Wojtas ENA (Elastic Network Adapter) is the smart NIC available in the virtualized environment of Amazon Web Services (AWS). The ENA driver supports multiple transmit and receive queues and can handle up to 100 Gb/s of network traffi= c, depending on the instance type on which it is used. Completed since the last update: =E2=80=A2 Update ENA driver to v2.3.1 =E2=80=A2 Determine location of the MSIx vector table on the device and a= llocate it dynamically - this enables driver usage on instances like c5gn. =E2=80=A2 MFC of the ENA v2.3.1 driver to the FreeBSD 11/12/13-STABLE bra= nches =E2=80=A2 ENA v2.3.1 will be a part of the FreeBSD 13.0-RELEASE Work in progress: =E2=80=A2 Internal review ongoing: =E2=80=A2 Introduce full kernel RSS API support =E2=80=A2 Allow reconfiguration of the RSS indirection table and hash key =E2=80=A2 Adjust iflib framework for the ENA requirements =E2=80=A2 Add DMA width configuration field commit 6dd69f0064f1 =E2=80=A2 Add support for admin completion queues commit 09c3f04ff3be =E2=80=A2 Prototype the driver port to the iflib framework Sponsor: Amazon.com Inc =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Intel wireless update Links: The freebsd-wireless mailing list URL: https://lists.freebsd.org/mailman/ listinfo/freebsd-wireless Contact: Bjoern A. Zeeb Newer Intel Wireless device support The Intel Wireless driver update project aims to bring support for newer chipsets. During the first quarter the driver and firmware were synched from upstream= so that we will have support for all modern cards currently supported in Linux. Some iwlwifi driver changes were also submitted back upstream. Several conflicts with the original implementation of LinuxKPI were or are being resolved and more LinuxKPI code was upstreamed to FreeBSD HEAD. LinuxKPI 802.11 compat code was improved and as of the day of writing we ha= ve data packets going over 11a. The plan for the next weeks is to clean things up, land as much as possible= in HEAD, provide the code for testing and work on stability based on feedback before filling gaps in the LinuxKPI 802.11 compat code to enhance support f= or more standards and features. Sponsor: The FreeBSD Foundation =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Kernel Sanitizers Contact: Mark Johnston Work has been ongoing to port a pair of kernel sanitizers from NetBSD to FreeBSD. Sanitizers are debugging tools which make use of compiler instrumentation to validate memory accesses. They can automatically detect = many classes of C programming bugs, such as use-after-frees and loads of uninitialized variables. When combined with syzkaller or other testing tool= s, they are effective at detecting kernel bugs that may otherwise require a gr= eat deal of manual effort to identify. Two sanitizers are being ported, KASAN (AddressSanitizer) and KMSAN ( MemorySanitizer). KASAN checks the validity of all memory accesses within t= he kernel map and triggers a panic upon an out-of-bounds access or a use-after-free. Various kernel memory allocators annotate regions of memory= to denote whether corresponding accesses are valid. The initial port of KASAN = is complete and is planned to appear in the FreeBSD development branch in mid-April. KMSAN detects uses of uninitialized memory and can detect bugs t= hat result in the contents of kernel memory being leaked to userspace. Both sanitizers incur considerable memory and CPU overhead. They are intended to= be used mainly in conjunction with test frameworks, though it is certainly possible to boot and run sanitizer-enabled kernels in a desktop or laptop environment. Currently this work is amd64-only. It should be possible to po= rt it to arm64 and riscv with relatively little effort. Future work in this area consists of finishing the KMSAN port, fixing bugs found by the combination of KASAN and syzkaller, and making sure that sanitizers can validate accesses to the direct map. This may consist of an option to disable usage of the direct map, or introducing a shadow for the direct map. Sponsor: The FreeBSD Foundation =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Marvell ARM64 SoCs support Contact: Zyta Szpak Contact: Kornel Dul=C4=99ba Contact: Marcin Wojtas The Semihalf team is working on improving the FreeBSD support for the Marve= ll Octeon TX2 CN913x and Armada 7k/8k SoC families. Marvell Armada 7k8k and Octeon TX2 CN913x SoC families are quad-core 64-bit ARMv8 Cortex-A72 processors with high speed peripherals including 10 Gb Ethernet, PCIe 3.0, SATA 3.0 and USB 3.0 for a wide range of networking, storage, security and industrial applications. Although the mentioned SoCs are mostly supported in FreeBSD HEAD, some piec= es required improvements. Applied changes: =E2=80=A2 Add missing frequency modes in ap806_clock driver (commit a86b0= 839d7bf) =E2=80=A2 Multiple fixes in mvebu_gpio driver - in cooperation with mmel = (commit a5dce53b75d8) =E2=80=A2 Fix device tree data parsing in mv\_ap806\_gicp interrupt contr= oller driver (commit 622d17da46eb) =E2=80=A2 Rework the ICU interrupt controller (mv\_cp110\_icu) and its pa= rent (mv\ _ap806\_gicp), so that they no longer rely on the data provided by firmware, which fixes booting the OS from the newer U-Boot/TF-A revisio= ns ( D28803) =E2=80=A2 PCIE Designware driver (pci_dw) fixes: =E2=96=A1 Correct setting of outbound I/O ATU window. =E2=96=A1 Allow mapping ATU windows bigger than 4GB. =E2=80=A2 Generic improvements that enable proper user-space mapping and = access of the PCI BARs TODO: =E2=80=A2 Upstream PCIE improvements. =E2=80=A2 Improve and merge ICU support rework. Sponsor: Marvell =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 nv(9)-based audio device enumeration Links: D26884 Implement sndstat nvlist-based enumeration ioctls. URL: https:// reviews.freebsd.org/D26884 commit c96151d33509655efb7fb26768cb56a041c176f1 URL: https://cgit.freebsd.o= rg/ src/commit/?id=3Dc96151d33509655efb7fb26768cb56a041c176f1 Contact: Ka Ho Ng This work presents a number of ioctl commands on /dev/sndstat using nv(9) to expose all available audio device nodes. nv(9) is used to generate a serial= ized binary stream representation of the information of audio device nodes prese= nted in the running system. The documented nvlist structure in sndstat(4) manual page is stable for programming use. For a long time, enumerating the audio device node interface required parsi= ng content of /dev/sndstat. It is tedious to write such a parser and handle different hw.snd.verbose levels correctly. Using nv(9) eliminates the need = to write a text parser to do audio device nodes enumeration. This work has been committed and is available in FreeBSD 14-CURRENT. Sponsor: The FreeBSD Foundation =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Ports Changes affecting the Ports Collection, whether sweeping changes that touch most of the tree, or individual ports themselves. KDE on FreeBSD Links: KDE FreeBSD URL: https://freebsd.kde.org/ KDE Community FreeBSD URL: https://community.kde.org/FreeBSD Contact: Adriaan de Groot The KDE on FreeBSD project aims to package all of the software produced by = the KDE Community for the FreeBSD ports tree. The software includes a full desk= top environment called KDE Plasma, graphics applications, instant-messengers, a video-editing suite, as well as a tea timer and hundreds of other applicati= ons that can be used on any FreeBSD machine. The KDE team (kde@) is part of desktop@ and x11@ as well, building the soft= ware stack to make FreeBSD beautiful and usable as a daily-driver graphics-based desktop machine. The KDE Frameworks have a monthly release cycle; KDE Plasma and the rest of= KDE software run on a quarterly cycle plus monthly bugfixes. All of those relea= ses landed in ports in a timely manner. Around KDE there are several hundred ot= her applications with their own releases, of which notable or new ones are: =E2=80=A2 deskutils/calindori, deskutils/kongress, net-im/kaidan, deskuti= ls/semantik and graphics/kgeotag =E2=80=A2 net-im/ruqola and net-im/neochat for Rocket and Matrix instant-= messaging, respectively =E2=80=A2 audio/amarok, the one-time favorite KDE music player Infrastructure work improved the way Qt5 ports install- and un-install chan= ges to the global header qconfig-modules.h. CMake releases landed with distress= ing regularity, and various low-level things like devel/libphonenumber and grap= hics /poppler were updated as needed. The big issue in the Qt stack on FreeBSD is Qt5-WebEngine, which is based on Chromium. Like Chromium itself (upstream), it has a tangled mess of a build system based on Python 2.7. The scheduled removal of Python 2.7 and ports t= hat depend on it is a sword looming over a large chunk of the Qt and KDE stack. Some resolution may be forthcoming in the form of WebEngine-less ports, but= the real effort is in trying to get WebEngine to build with Python3. More detailed descriptions of the updates in this quarter are available here (part 1) and here (part 2). =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 FreeBSD Office team 2021Q1 status report Links: The FreeBSD Office project URL: https://wiki.freebsd.org/Office Contact: FreeBSD Office team ML Contact: Dima Panov Contact: Li-Wen Hsu The FreeBSD Office team works on a number of office-related software suites= and tools such as OpenOffice and LibreOffice. Work during this quarter was focused on providing the latest stable release= of LibreOffice suite and companion apps to all FreeBSD users. Latest and quarterly ports branches got a new branch (7.1) of the LibreOffi= ce suite and updated to 7.1.1 release. Meanwhile, our WIP repository got back a working CI instance again, thanks = to Li-Wen Hsu. We are looking for people to help with the open tasks: =E2=80=A2 The open bugs list contains all filed issues which need some at= tention =E2=80=A2 Upstream local patches in ports Patches, comments and objections are always welcome in the mailing list and bugzilla. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 VirtualBox FreeBSD port Links: VirtualBox home page URL: https://www.virtualbox.org/ VirtualBox OSE port on FreshPorts URL: https://www.freshports.org/emulators/ virtualbox-ose Contact: VirtualBox port team The VirtualBox ports have been updated to the upstream 6.1.18 release. This is a new major release with new features and better support, especially for graphics output. This new release has support only for recent amd64 CPUs providing virtualization support in hardware (VT-x, AMD-V bits). The previous versions of the VirtualBox ports have been preserved as the -legacy versions to allow people unable to use the new version to have a virtualization solution. The new additions port at present fails to build on i386 but the old additi= ons do provide basic functionality for that emulation. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Documentation Noteworthy changes in the documentation tree, in manpages, or in external b= ooks /documents. DOCNG on FreeBSD Contact: Sergio Carlavilla The Doc New Generation project is finished. The switch-over date was Saturd= ay, January 23rd. But there=E2=80=99s a list of remaining tasks: =E2=80=A2 Convert the Python scripts to Ruby using the AsciiDoctor API =E2=80=A2 Convert from releases from 4.4 to 9.0 to AsciiDoctor =E2=80=A2 Use rouge in the source sections instead of the CSS hack =E2=80=A2 Split the news page to reduce the total size of the page =E2=80=A2 Split the books =E2=80=A2 Improve the FDP book If you want to reduce the TODO list, give me a ping :) =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 FreeBSD Translations on Weblate Links: Translate FreeBSD on Weblate wiki URL: https://wiki.freebsd.org/ DocTranslationOnWeblate FreeBSD Weblate Instance URL: https://translate-dev.freebsd.org/ Contact: Danilo G. Baio After the doc migration to Hugo/AsciiDoctor the Weblate tool it=E2=80=99s o= pened again. There are three projects on our Weblate: =E2=80=A2 Documentation (Books and Articles) - open =E2=80=A2 FreeBSD Doc (Archived) - former project =E2=80=A2 Website - pending Language teams that were using Weblate before the migration to Hugo/ AsciiDoctor, please see our Translation based on Automatic Suggestions Wiki article for more details. We=E2=80=99ve just started a project for converting the old method of autom= atically translating strings, using the Machine Translation feature, to a new system that will work for the new documentation. This will save time for our translators. There are still pending items: you can check the Status Page; any help is v= ery welcome. The next step for the new quarter is to prepare and release the Website translations through Weblate as well. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 WebApps working group Contact: Sergio Carlavilla The purpose of this working group is to redesign the Website and the Documentation Portal. The work will be divided into 4 phases. Phase 1: =E2=80=A2 Redesign the documentation portal: new design, responsive and g= lobal search. Phase 2: =E2=80=A2 Redesign the manual pages scripts to generate the HTML using ma= ndoc. Phase 3: =E2=80=A2 Redesign the ports scripts to create an applications portal. Phase 4: =E2=80=A2 Redesign the main website: new design, responsive and dark them= e. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Miscellaneous Objects that defy categorization. Discord Server & Community Growth Links: Discord Wiki Page URL: https://wiki.freebsd.org/Discord Discord Invitation Link URL: https://discord.gg/RHprKbvWJN Contact: Lewis Cook Contact: Vincent Milum Jr Contact: Kubilay Kocak The FreeBSD project and community continues to grow, and in the last 2 quarters, we established an official FreeBSD server on the popular Discord platform, as a complementary means for out-reach, support, collaboration and social connection for and between members of the FreeBSD community, new and old. Discord boasts broad accessibility and unique features that the the communi= ty can enjoy without the steeper learning curves associated with other support= and social mediums, that can often deter or overwhelm newcomers. It also provid= es an opportunity to broaden FreeBSD=E2=80=99s reach outside traditional space= s and audiences. We currently have a respectable 480 members, with that number growing daily, and have established baseline community guidelines and moderation processes consistent with and complementary to other support channels and the FreeBSD Code of Conduct. While it=E2=80=99s early days, events have already been successfully hosted= on the platform. In January, Tom Jones (thj@) announced and ran an online Bugathon focusing on issues related to the branching of FreeBSD 13. We=E2=80=99ve al= so created dedicated text and voice channels to facilitate more of these kinds of even= ts in the future. We hope to see more events like these run as examples of how= we can utilize Discord constructively and in ways we haven=E2=80=99t as a comm= unity or project before. With the future in mind, we have plans to: =E2=80=A2 Automatically announce news, updates and advisories in Discord. =E2=80=A2 Verify and enable additional Discord features designed for large communities. =E2=80=A2 Set up bots with unique features, including moderation and inte= ractive features for members. We welcome ideas in this regard. We are keen on project and community members to reach out to talk about how= we can best leverage Discord. Some ideas we=E2=80=99d love people to get invol= ved with include: =E2=80=A2 Brainstorm/Suggest unique and creative ideas or features. =E2=80=A2 Provide bug reports and user experience feedback and suggestion= s. =E2=80=A2 Actively promote Discord in other social media spaces, particul= arly those that maybe new or curious to learn more about FreeBSD. =E2=80=A2 Contribute to the Wiki page and its content. =E2=80=A2 Participate and support other members on Discord. =E2=80=A2 Run a live stream on a FreeBSD-related topic. =E2=80=A2 Hang out in our live audio and video channels if you=E2=80=99re= comfortable doing so. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Third-Party Projects Many projects build upon FreeBSD or incorporate components of FreeBSD into their project. As these projects may be of interest to the broader FreeBSD community, we sometimes include brief updates submitted by these projects in our quarterly report. The FreeBSD project makes no representation as to the accuracy or veracity of any claims in these submissions. CBSD Project Links: CBSD API module URL: https://www.bsdstore.ru/en/cbsd_api_ssi.html Contact: Oleg Ginzburg What is CBSD? CBSD is a management layer written for the FreeBSD jail(8) subsystem, bhyve= (8) and xen(4). CBSD allows users to manage jail/bhyve/xen environments at different levels of abstraction by providing a varied number of unified methods: vagrant-like CBSDfiles, CLI and via dialog(1). CBSD 2021Q1 Status Report A RestAPI service layer was added during last quarter, enabling creation of programmable cloud solutions. In addition, work has been done to support RestAPI through a CBSDfile, allowing for private cloud environments deploym= ent. In such cases the local CBSD layer acts as a thin client. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 helloSystem Links: Documentation URL: https://hellosystem.github.io/docs/ Contact: Simon Peter Contact: #helloSystem on irc.freenode.net, mirrored to #helloSystem:matrix.= org on Matrix What is helloSystem? helloSystem is FreeBSD preconfigured as a desktop operating system with a f= ocus on simplicity, elegance, and usability. Its design follows the =E2=80=9CLes= s, but better=E2=80=9D philosophy. Q1 2021 Status =E2=80=A2 helloSystem and some of the motivations and core ideas behind i= t were presented at the FOSDEM 2021 BSD Devroom in the talk "hello=E2=80=A6=E2= =80=8B again? Simplicity, elegance, and usability for the desktop". Video recordings = of the talk and Q&A session are available WebM/VP9, mp4 =E2=80=A2 Version 0.4.0 of helloSystem was published. Installable Live IS= O images are available. =E2=80=A2 Work has started towards 0.5.0. We are beginning to see contrib= uted features and bugfixes =E2=96=A1 System menu reflects changes made in Applications immediate= ly =E2=96=A1 Filer file manager brings already-open windows to the front= rather than opening multiple windows for the same folder =E2=96=A1 Initial spatial mode option (each folder opens in its own w= indow in the file manager that remembers its on-screen location) Experimental and release builds of the Live ISO are available at https:// github.com/helloSystem/ISO/releases. Contributing Help is wanted in a number of areas, especially in the areas of the FreeBSD core OS and kernel, and Qt/C++. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 PkgBase.live Links: Website URL: https://alpha.pkgbase.live/ Contact: Mina Gali=C4=87 PkgBase.live is an unofficial repository for the FreeBSD PkgBase project. PkgBase packages the FreeBSD base system as ca 330 packages. The PkgBase project gives users the choice of which parts of the system to install. Users can choose which parts of the base system to install, without building their system from source and optionally choose to install -dbg packages when they need them. PkgBase is built with default options. There= =E2=80=99s no need to build WITHOUT_SENDMAIL, when users can just chose not to install it= ! In addition, PkgBase.live builds every usable kernel! This is especially impor= tant for architectures like armv7. As a service, PkgBase.live was inspired by up.bsd.live, which provides freebsd-update(8) for STABLE and CURRENT branches. Despite this inspiration, freebsd-update has been a constant point of frustrations for me, so I was looking for alternatives. PkgBase is not ready for prime time yet, or else the FreeBSD project would = be providing this service. With the call for testing open since 2016, I though= t it was time to offer a public service, so a broader part of the community can = take part in testing, without having to do all the work for themselves. A lot of things already work fine, but more work needs to be done, as can be seen from the TODO list, as well as the "Pending Changes" on the website. Perhaps the most important thing would be to provide ISOs which lets people setup a fresh system with PkgBase from the get-go. Hardware for PkgBase is kindly sponsored by a member of the FreeBSD communi= ty. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Potluck & Potman Links: Potluck Repository & Project URL: https://potluck.honeyguide.net/ Potluck on github URL: https://github.com/hny-gd/potluck Potman on github URL: https://github.com/grembo/potman Pot Project URL https://pot.pizzamig.dev Contact: Stephan Lichtenauer (Potluck) Contact: Michael Gmelin (Potman) Pot is a jail management tool that also supports orchestration through Noma= d. Potluck aims to be to FreeBSD and Pot what Dockerhub is to Linux and Docker= : A repository of Pot flavours and complete images for usage with Pot and in ma= ny cases Nomad. The new Potman project aims to simplify building Pot images with Vagrant and VirtualBox based on the Potluck approach, e.g. as part of a DevOps workflow= for software development and testing. That way, Pot images can more easily be created as part of a Jenkins workflow to be deployed with Nomad and Consul. In the last two quarters, FreeBSD 12.2 Potluck images have been built and t= he 11.4 images have been upgraded with the new packages. Furthermore, new imag= es like Wordpress and an improved flavour script (which squashes a nasty netwo= rk problem) have been created. Future plans include further Potman workflow features, new and improved Pot= luck images and publishing FreeBSD 13-based images. As always, feedback and patches are welcome. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 sysctl improvements Links: sysctlinfo URL: https://gitlab.com/alfix/sysctlinfo sysctlbyname-improved URL: https://gitlab.com/alfix/sysctlbyname-improved BSDCan 2020 - sysctlinfo questions URL: https://git.io/Jm9x7 sysctl-libnv URL:https://gitlab.com/alfix/sysctl-libnv[https://gitlab.com/a= lfix /sysctl-libnv] sysctlmibinfo2 URL:https://gitlab.com/alfix/sysctlmibinfo2[https://gitlab.c= om/ alfix/sysctlmibinfo2] sysctlview URL: https://gitlab.com/alfix/sysctlview nsysctl URL: https://gitlab.com/alfix/nsysctl Contact: Alfonso Sabato Siciliano The sysctl system call and the wrapper sysctl utility can get and set the system state at runtime; the kernel exposes the available parameters as obj= ects of a Management Information Base. After a recent system update with a CURRENT-GENERIC configuration, the MIB has exceeded ten thousand objects (b= oth internal nodes and leaves, most in the vm.uma subtree) on my laptop with a Desktop environment without ZFS. Furthermore I received tips, ideas, PRs and issues about sysctl so some improvement has been fulfilled; finally the suggestions received during BSDCan 2020 have been accomplished. Kernel space sysutils/sysctlinfo has been updated to 20210222 and is an interface to vis= it the MIB and to retrieve info about an object (description, type, format, fl= ags, and so on). It has been refactored so the new version is almost 100% more efficient to explore the MIB and to pass all info about an object to userla= nd. Moreover new features have been implemented: to get more info about an obje= ct, to avoid extra computation in userland, and to improve the compatibilty with the current undocumented interface. sysutils/sysctlbyname-improved has been updated to 20210223 and is an exten= sion of sysctlinfo to handle an object name with some empty string level or exte= nded to pass an input to the handler of a CTLTYPE_NODE; it has been updated to t= ake advantage of the improvements (mainly efficiency) of sysctlinfo. sysctl-libnv is a project that provides an implementation and an example of= how to build a sysctl object with an nvlist value - to learn more about nvlist,= see the libnv(9) manual page. Properly a new sysctl handler has been defined: i= t is enough to create a nvlist and to pass it to a macro; then the system call u= ses the new handler to pass the nvlist to the userland and the nsysctl utility = can manage the object value. The following tools are been updated to give advantages from new kernel features and improvements. Library devel/libsysctlmibinfo2 has been updated to 2.0.1; primarily the sysctlmibi= nfo2 library wraps the low-level interfaces described above; moreover it defines= a struct sysctlmif_object with the properties of an object and provides a convenient API to build data structures of sysctlmif_object (for example: a subtree, a list of a list of a Depth First Traversal, and so on); therefore= it is useful for handling an object correctly and/or for building a sysctl-like utility. Obviously sysctlmibinfo2 benefits from the features of sysctlinfo: handles = OIDs up to CTL_MAXNAME levels, supports the Capability Mode, can seek an object matching its name (avoiding having to explore the MIB just to find the corresponding OID), gets all info about an object at a time, and manages a special name via sysctlbyname-improved. Version 2.0.1 takes advantage from the kernel improvements: improved effici= ency to build a sysctlmif_object and new features to get info about an object: "handler" and "nextbyname". The new functions are: sysctlmif_hashandler() a= nd sysctlmif_hashandlerbyname() to know if an object has a defined kernel hand= ler, sysctlmif_nextnodebyname() and sysctlmif_nextleafbyname() to explore the MI= B, sysctlmif_leaves() and sysctlmif_leavesbyname() to build only-leaf data structures. Documentation The APIs described above (both kernel and userspace) are really easy: "sysc= tl -aN", "sysctl -d kern.ostype", etc., can be implemented in a few lines of c= ode. Nevertheless each project provides a README with Introduction, Getting star= ted, Features, API, Real-world use cases, FAQ, and examples in the Public Domain= to build new projects. Of course the manuals and examples have recently been updated. Utilities deskutils/sysctlview has been updated to 2.1; the first version of sysctlvi= ew was just a graphical representation of the MIB, now it could be considered a GUI version of sysctl. This utility exploits the object serialization of sysctlinfo; indeed it is not feasible to have the kernel to find the same object many times to retrieve all its properties, considering the current M= IB size. Thanks to user feedbacks the new version provides a better UI, for example clicking a column title to sort the entries, moreover the "Handler" entry is been added in the "Object" window, it is useful to know if an obje= ct has a value or if the OID of a CTLTYPE_NODE can be extended. sysutils/nsysctl has been updated to 2.0 and is the CLI version of sysctlvi= ew; the output is explicitly indicated by the options and is printed via libxo = in human- and machine-readable formats; moreover some string value is parsed to display structured output. The options are not mutually exclusive and allow showing the properties of a parameter so nsysctl is useful to know the info= to handle an object without finding its implementation, for example: Is Multiprocessor safe? Is Capability mode available? Is the OID extensible? D= oes the integer represent a kelvin? Does it have a value? What is the label? An= d so on. The new version supports libnv; it is useful to manage a non-primitive = data type and could avoid hardcoding a generic opaque type in the future. Finally the new features of sysctlinfo allow using nsysctl to debug the MIB without= a kernel recompilation with SYSCTL_DEBUG. Note: the project provides a tutori= al to describe every feature. --p2qii6v5xswo6umh Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGTBAABCgB9FiEEDonNJPbg/JLIMoS6Ps5hSHzN87oFAmCTLjFfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDBF ODlDRDI0RjZFMEZDOTJDODMyODRCQTNFQ0U2MTQ4N0NDREYzQkEACgkQPs5hSHzN 87oD7QgAslUF+YnxoSqDL8wJt7RzRfEaw75npm8glTp9I5Jbd9+n4mlxbueFH7KW KsARVZOrtdYW2X/M2VGZvl2Zd9vqPbelThpfquLvYrfwr/EajrjtElbLpFi/JVEY j+XuCTIEgNM+8KA2wLjKIKhxbs2q2bngPLGW6HbYxbkSBTdPCV8QNbRpvCFrz8cy QqbdjdybyptvzzLARO03RkV7y/wUoxErkcnBdRwkz0iSEX9a71DWzAllmLTsM3nQ RoswtfaLOwb7qt9DwNjFcSGmZx2io6/qbGmVy4QEiVLgjwtF+YT1xbnvQQTCM+eY 2T6L0/W6SVFiMjbqIOex17O0Q5iMmg== =tIS8 -----END PGP SIGNATURE----- --p2qii6v5xswo6umh-- From owner-freebsd-stable@freebsd.org Thu May 6 00:01:25 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A174C628911; Thu, 6 May 2021 00:01:25 +0000 (UTC) (envelope-from yuripv@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FbDJP4HQCz4tLf; Thu, 6 May 2021 00:01:25 +0000 (UTC) (envelope-from yuripv@FreeBSD.org) Received: from [192.168.1.12] (unknown [91.240.124.245]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: yuripv) by smtp.freebsd.org (Postfix) with ESMTPSA id E4CCD23539; Thu, 6 May 2021 00:01:24 +0000 (UTC) (envelope-from yuripv@FreeBSD.org) Subject: Re: zpool list -p 's FREE vs. zfs list -p's AVAIL ? FREE-AVAIL == 6_675_374_080 (199G zroot pool) To: Mark Millard , freebsd-current , FreeBSD-STABLE Mailing List References: From: Yuri Pankov Message-ID: <34521f0b-a7a1-1743-244a-4149e72911a0@FreeBSD.org> Date: Thu, 6 May 2021 03:01:07 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 May 2021 00:01:25 -0000 Mark Millard via freebsd-current wrote: > Context: > > # gpart show -pl da0 > => 40 468862048 da0 GPT (224G) > 40 532480 da0p1 efiboot0 (260M) > 532520 2008 - free - (1.0M) > 534528 25165824 da0p2 swp12a (12G) > 25700352 25165824 da0p4 swp12b (12G) > 50866176 417994752 da0p3 zfs0 (199G) > 468860928 1160 - free - (580K) > > There is just one pool: zroot and it is on zfs0 above. > > # zpool list -p > NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > zroot 213674622976 71075655680 142598967296 - - 28 33 1.00 ONLINE - > > So FREE: 142_598_967_296 > (using _ to make it more readable) > > # zfs list -p zroot > NAME USED AVAIL REFER MOUNTPOINT > zroot 71073697792 135923593216 98304 /zroot > > So AVAIL: 135_923_593_216 > > FREE-AVAIL == 6_675_374_080 > > > > The questions: > > Is this sort of unavailable pool-free-space normal? > Is this some sort of expected overhead that just is > not explicitly reported? Possibly a "FRAG" > consequence? >From zpoolprops(8): free The amount of free space available in the pool. By contrast, the zfs(8) available property describes how much new data can be written to ZFS filesystems/volumes. The zpool free property is not generally useful for this purpose, and can be substantially more than the zfs available space. This discrepancy is due to several factors, including raidz parity; zfs reservation, quota, refreservation, and refquota properties; and space set aside by spa_slop_shift (see zfs-module-parameters(5) for more information). From owner-freebsd-stable@freebsd.org Thu May 6 00:18:43 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 739F9629D1D for ; Thu, 6 May 2021 00:18:43 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic308-8.consmr.mail.gq1.yahoo.com (sonic308-8.consmr.mail.gq1.yahoo.com [98.137.68.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FbDhM1C9jz4vZK for ; Thu, 6 May 2021 00:18:42 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1620260320; bh=ZiDWzzUQ5JGaPQAwFI67YRJQ8dSeiC9IuLkpSr3tsF7=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=jOKd7lSuWgi1VnBFjaATg4lI9oi9EQrYz4YNvc40H1vbBw1tVTqVCyNZsD8OplVHI5zzTDZi1TPArQ/zjrV3c89EhCjZ9iVJakzW2cT+UtrYiVvi4ei3NImHCGAkzYFfrlCfUqSDHPQMN5lKMveUYcSvMj+ibIt7Yc+x1GHdZi51dwCmFaLBgbuiDXw74hFDSEzLAKgEILDdk/iEY4GFCL8v/NNJW9ynJCTIWtsUUMJHDnQZPujlSX2Rdfx1XCjTMPXTr+bKAN93mrLz/s+/C3IEavc/KhlLACapYV+MtGl4CnTjcpTHi++oQDKSc4YShRqn5izt7dvoCde2Fskodg== X-YMail-OSG: wbPMNm4VM1kqLZSwpu4yohf_wMm5XPskZkQs5nb24C0Z0998yjaf9GMIzjUWRvs 54RnZcq_RVE0CrUYRMF3otKLO7m7e7imUAQ_MePtZPQOgxHwWI0nLG1H8mNj3eBQ2n4PI0z5tUG. RXCLOCC52YK5ACpVAtJCwEonEIhLbGkvOjQiNZD5E3vXnHZ9l28oo4LhvzR9sSkVXO0IqREaUbu6 dQDvSY9P1wigs8LY62BDiEVE8z5w.UT4pp1Jadv7jegpiropYb0MOAlGP6yU2fvEui7n_lJ21cFI dsm4ERoRn_tu.W2gODeGvikbulWbOCce0EqJTMXtqUtKdHZyam3GE5eUDtG6e2NW1muvmiqq7H7P Ns0_MdzRFmyXp_NSsbpvRz0Qdw_uektYay98Y5jRHqoHRnPeWm7F6YCatolUUAMHnjKp_3EyTAza YQYUxzApPsRdnQoIiUh24HkYK2DpsU9sM6CRtCk1ERLvcwQ5oHcAIa9QewcVNZnSJvBzDobtiwI5 Xk4R.W4dDPwRrmklAokSeDoqFFxTAj0vNiG7ljuVm94wD75XUvj42Bg_PVwO1vkbHQNfkbm9k8ig ARYxzZ8V8nuKCNinWCVuePOvP10KEnBkFCe6mC5FrDycD8NTIReFsx9qM1UnqZ_ZHCHM197jozr7 aSc84zRouj9snZxFiXSA7_EN9dS0N5TSZlOGzjhxSCbxRvkCLwZ5XEv1Kf6ZRjRIdSFVsD0MbZ.5 Vohf3vS11TrtF3ouqMzgJirXdpQkiRfyD30H0BN0LS_py3Vrdt4bWjipS2qUXSNM_O2.uN4OOKyx Aba9g0XQs0Fugccpeqj43lAigEa_MQNpimO2yPtfy0XeziT_CYF2Wh8vj2w5A9E8GYAXWDFxa6ql jALtHokjtjIEr1ivbCqQlH7gH7v3.BAfU4Y6prWvQYpll7ZDmiJUpxo.XxuxN_Ed8bpr1Sc5Z_lZ JGWH0rxuTJ0W5u9LP.98fBIVt4GJLzGSo8K7Pqb3fh3B5qF.V_.KQh2pAvmocsoK4vo0.ciGp8nV 47NfYwx_IpzZlI7va4sHQPBa4e3kHjQQ7RLs3fQGw5sNZ6LM3BpSEh8QsK_uCbc2PXYbs_gqSYGx 5LA0YCY2MnjW9i5_WO91lVCkS1LuH3Fb3i9krQI0GgIHr6Q_0.bhPMmoASip_4YGNLFwPSFbmOds 3.qS5.sGeODuFDSdwzsX7PA43gxAXBmRgGA3.SxB2CdCW97YMcM.TqwWWjLsvb6FBJH2vyzNnxCx dpqQ3EyS15jwqvx.6uhtlT7MPTenNQdlIVGXVIKcNpdDKHmGZkUlcfnoo_KCNxPiUFzOko_5Ot45 O68a_ePUu61kjHVpwxRDOHM7G_VyBg.R6C72OT9WxWK.NBQXoH.bAe4mjWSqnv6RyVgUUAQINQeA j_zm9X.8NUOm0mn0lUDGC0caTPrCLtnPLeM0uJWJyAC0MA3XHJZzVtKLBtuyeuarZCeHmncg35QJ pDZakK44PRYeIDZHq9MeS05D8dReIPIDtOMtcU9zGGPnfzV23ZYp_gpAeYmOQ71V.bbTxDrwTPg9 ZnxlLQ9SbXTGVzfqlOdxo5OsZarfX2L1cwe7bXQxRV2pmvt2CSesRc97KoIC7Lfk1tD4MogbDgb4 teTNheZm9KPyw2oaXegUyFcpG5RRBiZnD7h9GY6_WR_ilsa2bCiZERU9XsiZj7LzIKpvwJVpxcgm KugXpdiZHSseMxILt.5dIi9S3KOFdWRWIV.lR5z3blrGA4yIxrJxHOA0fQajxzLM.0QDPxBPCUdZ xqPu077pccKWKUyRuExFX0rO1VorqfsxbOFJqAWBOv.CfsOvGr8sVMI8XD3C_wxooiy89cNGfD5w V3U0iub2kbn94bEZdTZxkECl9K6MAgxh2LwjAvcbWcqlw2y6jpud2FZvDXB3v2_3E_Zepn91Ppph vRT1i44RIT.Mh0dZ61SiZOs78AmV4bg5F5pOCMfaqya05fLxYFvMwtuKdNhNB.m4M_thBeZBvRJP 8W04yP5y9z49cZRoC4ivQlVo9NEdZN7JO9y4E40qYJ6Z92DgrkiWsCJxtaVsUYzYkBKDmgfDZJdC um1Av24DoY4R4alXUZ82U8Iz31XnzhKKsAi4V3ht9h1jB2rhu25M0e09Vc6JQtITYkU2eBoQG5Xq SfZQPGAPivulETGl1B5WpZT7VAJpwPsPZYqE7m.YQP4J6kj6Lyj9EHP.K7hglWzBKmIxUBYZqRWF hiNvST962t235PEawNTaKlg1pmKytZetNUqj4IspPj0nF3TuGd_38eB1z1Nr7dLOOUvugjMnqHir 7gQQhm54wtJ9ES6mlDICz0RYbdoyFYioQ0w4lWctcE17Nv92xpJq1hCUe19p1M9S.soTJl2gHURj zT_C8yuFK1.vTxvowk0IOx5330bxd8uD0hzCGvw3qgf2Ahq.pDCRZuJfRzGbMxXjLMOLBfrReb6h IW0nfYdpQwvIlwtNtcyuSDC3LJ2JGoof6jv24CeiSrU_A07mim9smprUS8.oCF6Rfxgw1k6JMeZo Yhu.R1sSG1a8BRs9Xus3CrZ5juYKvQfzUJV5vPgW4q_cd1tRHSW63xi.h1jq4MXGXgY34NK4.Zpv 8fMix69_eel9oMx6Dbc5drR7NIX2WJGzd2rTnKcPbIVH82yAcI4fnING1Oapo.i7lvkd_Sjf0b8m y1JF1eATroiT24sDQP5zgS5kCM049QpiXnqshmjUDemnF2Z3G8oAp.59sZW6iD1MkO2at609olR2 mIj3xAgXEkLt34VMiYA-- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic308.consmr.mail.gq1.yahoo.com with HTTP; Thu, 6 May 2021 00:18:40 +0000 Received: by kubenode544.mail-prod1.omega.ne1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID ae539ec798e8ac8d2e9120c77e83a682; Thu, 06 May 2021 00:18:36 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: Re: zpool list -p 's FREE vs. zfs list -p's AVAIL ? FREE-AVAIL == 6_675_374_080 (199G zroot pool) From: Mark Millard In-Reply-To: <34521f0b-a7a1-1743-244a-4149e72911a0@FreeBSD.org> Date: Wed, 5 May 2021 17:18:34 -0700 Cc: freebsd-current , FreeBSD-STABLE Mailing List Content-Transfer-Encoding: quoted-printable Message-Id: <164F9986-FB9C-43A9-ABC0-0D091D2CFF3D@yahoo.com> References: <34521f0b-a7a1-1743-244a-4149e72911a0@FreeBSD.org> To: Yuri Pankov X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Rspamd-Queue-Id: 4FbDhM1C9jz4vZK X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 May 2021 00:18:43 -0000 On 2021-May-5, at 17:01, Yuri Pankov wrote: > Mark Millard via freebsd-current wrote: >> Context: >>=20 >> # gpart show -pl da0 >> =3D> 40 468862048 da0 GPT (224G) >> 40 532480 da0p1 efiboot0 (260M) >> 532520 2008 - free - (1.0M) >> 534528 25165824 da0p2 swp12a (12G) >> 25700352 25165824 da0p4 swp12b (12G) >> 50866176 417994752 da0p3 zfs0 (199G) >> 468860928 1160 - free - (580K) >>=20 >> There is just one pool: zroot and it is on zfs0 above. >>=20 >> # zpool list -p >> NAME SIZE ALLOC FREE CKPOINT EXPANDSZ = FRAG CAP DEDUP HEALTH ALTROOT >> zroot 213674622976 71075655680 142598967296 - - = 28 33 1.00 ONLINE - >>=20 >> So FREE: 142_598_967_296 >> (using _ to make it more readable) >>=20 >> # zfs list -p zroot=20 >> NAME USED AVAIL REFER MOUNTPOINT >> zroot 71073697792 135923593216 98304 /zroot >>=20 >> So AVAIL: 135_923_593_216 >>=20 >> FREE-AVAIL =3D=3D 6_675_374_080 >>=20 >>=20 >>=20 >> The questions: >>=20 >> Is this sort of unavailable pool-free-space normal? >> Is this some sort of expected overhead that just is >> not explicitly reported? Possibly a "FRAG" >> consequence? >=20 > =46rom zpoolprops(8): >=20 > free The amount of free space available in the pool. By contrast, > the zfs(8) available property describes how much new data can = be > written to ZFS filesystems/volumes. The zpool free property is > not generally useful for this purpose, and can be substantially > more than the zfs available space. This discrepancy is due to > several factors, including raidz parity; zfs reservation, = quota, > refreservation, and refquota properties; and space set aside by > spa_slop_shift (see zfs-module-parameters(5) for more > information). Thanks for pointing to the reference material. 6_675_374_080/213_674_622_976 =3Dapprox=3D 0.03124 =3Dapprox=3D 1.0/32.0 and spa_slop_shift's description reports: QUOTE spa_slop_shift (int) Normally, we don't allow the last 3.2% (1/(2^spa_slop_shift)) of space in the pool to be = consumed. This ensures that we don't run the pool completely = out of space, due to unaccounted changes (e.g. to the MOS). = It also limits the worst-case time to allocate space. = If we have less than this amount of free space, most ZPL operations (e.g. write, create) will return ENOSPC. Default value: 5. END QUOTE So in my simple context, apparently not much else contributes and the figures are basically as expected. Thanks again. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-stable@freebsd.org Thu May 6 10:13:59 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 08FAC5FA931 for ; Thu, 6 May 2021 10:13:59 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vtr.rulingia.com (vtr.rulingia.com [IPv6:2001:19f0:5801:ebe:5400:1ff:fe53:30fd]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA512 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "vtr.rulingia.com", Issuer "R3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FbTvB0SwTz4WmP for ; Thu, 6 May 2021 10:13:57 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from server.rulingia.com (ppp239-208.static.internode.on.net [59.167.239.208]) by vtr.rulingia.com (8.16.1/8.15.2) with ESMTPS id 146ADfTt002366 (version=TLSv1.3 cipher=AEAD-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 6 May 2021 20:13:47 +1000 (AEST) (envelope-from peter@rulingia.com) DKIM-Filter: OpenDKIM Filter v2.10.3 vtr.rulingia.com 146ADfTt002366 X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.16.1/8.16.1) with ESMTPS id 146ADa9v081373 (version=TLSv1.3 cipher=AEAD-AES256-GCM-SHA384 bits=256 verify=NO) for ; Thu, 6 May 2021 20:13:36 +1000 (AEST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.16.1/8.16.1/Submit) id 146ADamo081372 for freebsd-stable@freebsd.org; Thu, 6 May 2021 20:13:36 +1000 (AEST) (envelope-from peter) Date: Thu, 6 May 2021 20:13:36 +1000 From: Peter Jeremy To: freebsd-stable@freebsd.org Subject: tail(1) broken in 13-stable Message-ID: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="9L7O9ezs9xowJvzI" Content-Disposition: inline X-PGP-Key: http://www.rulingia.com/keys/peter.pgp X-Rspamd-Queue-Id: 4FbTvB0SwTz4WmP X-Spamd-Bar: ------ X-Spamd-Result: default: False [-6.10 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[rulingia.com:s=default]; FREEFALL_USER(0.00)[peter]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2001:19f0:5801:ebe:5400:1ff:fe53:30fd:from:127.0.2.255]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[rulingia.com:+]; DMARC_POLICY_ALLOW(-0.50)[rulingia.com,quarantine]; NEURAL_HAM_SHORT(-1.00)[-1.000]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:19f0:5801:ebe:5400:1ff:fe53:30fd:from]; ASN(0.00)[asn:20473, ipnet:2001:19f0:5800::/38, country:US]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 May 2021 10:13:59 -0000 --9L7O9ezs9xowJvzI Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Since updating from 12-stable to 13-stable, I've found that tail(1) crashes, reporting: Assertion failed: (procfd > STDERR_FILENO), function service_clean, file /u= sr/src/lib/libcasper/libcasper/service.c, line 394. tail: unable to init casper: Socket is not connected unless all three of stdin, stdout and stderr are open. Whilst it probably doesn't make sense to call tail without stdout open. there's no obvious reason to require that stdin or stderr must be open. --=20 Peter Jeremy --9L7O9ezs9xowJvzI Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEE7rKYbDBnHnTmXCJ+FqWXoOSiCzQFAmCTwUhfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEVF QjI5ODZDMzA2NzFFNzRFNjVDMjI3RTE2QTU5N0EwRTRBMjBCMzQACgkQFqWXoOSi CzT2Jg/9H5vaJlursfq7Ff4UKUW4n1cgQ0Md+DYLak/Sg0Jwa/2WSGpGVP2TSt7a ouX3QxvSWkoYpJWZx9vU2J492fKJwjIq2WulzNVG80kMwehnD9QICbm/D/9scVn9 IPTqzHPQP8C5A5z9duC/5MAbaEcbLMV6EUnRkXaCq9/X7hkDVQXqT4PPR5+M+A9V rRMaAO5+Kg23uvn4MbUQK5URdUb1tI+JskKGCEUI4wDcHzguSNszgBeINEejplw4 F3M2KPh3Uf/R0sFR0LhDV6dgDDScVTZeEaIiAz3/61Yz2pEM7/9XFDQFQBFE+Qyx 8iLpDk2oZ7XTY/qgvovkYb39dpjo0t8n8LrX6hp/tju4OarD3hnYBhpTFVWalLtH uG9ByPVifAOUiEQKuSaaVuj3X0l9nkFro9gYDqcX9qBCiV62QRVeJVVV/vNm6kL9 eYt28WzMEE0DNQhFP5t794GjTvUXbnNboPGz/bnypyg3iv88SUd82ab3VejRpX/x KoCNixRwd5EJO+pjByFtKE3dLafa6onstuTqTKw2iue0oGYv1o3ctHhuQNYErbko AffmC3KLZvh/k2ErChohYv9y/EWVyEeAo94fdO0PuzgW4324CH3W5nEVVl3mJMM6 h4hMQqP3c1/IEOp+NqGpO/Icv4i/fJL+v5GpaON098Bek76z9As= =uBHT -----END PGP SIGNATURE----- --9L7O9ezs9xowJvzI-- From owner-freebsd-stable@freebsd.org Thu May 6 11:00:07 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E86B95FBE0D for ; Thu, 6 May 2021 11:00:07 +0000 (UTC) (envelope-from oshogbo.vx@gmail.com) Received: from mail-lf1-f50.google.com (mail-lf1-f50.google.com [209.85.167.50]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FbVwR1k7hz4YS2 for ; Thu, 6 May 2021 11:00:07 +0000 (UTC) (envelope-from oshogbo.vx@gmail.com) Received: by mail-lf1-f50.google.com with SMTP id z9so7136978lfu.8 for ; Thu, 06 May 2021 04:00:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=mL46t/iyb8gWjzWnNJc4vfO5DzLERNiOhJojNVasAyY=; b=FwgWtxTvzAvA9E0SyXkR+BZjT+H2g825oa1SzrjqpIFtF1ftBgGicdnluVUCa+Rt3h xrRFrTwkU8aNWZ700HY+kIxKwHldgoCrrUM4nOL3gtUn2epz/XqxmhFrLcMgZY7uYPiL PSZwYYYBg3yMKTi/Y5cGLS9SPsyFLlMSQ5c1O9IX/5NHqFWuwutuP5Gx9zbiOVRmzsEc aEKYQgg/YOO9iPcDpSmGErZD1byklLIotFpxKoeR1UVNAqkbes1YJT1IugPniG6Jrt3w VsV5Axi+9eULTTwyTdEUibQVS2du04hcZmZwnvsWY3ooQv7rXD58d7wjrjwFbPtos/hG EYOw== X-Gm-Message-State: AOAM531LSpydoCa6yPj7a/tuuhUVOlkfSw7GdTsadTc5k43nrqUGQm4S 2aBBuboF2DKjB7pRTZYKCco+6x+YTxv0Y3uzOMxUtbOYb20= X-Google-Smtp-Source: ABdhPJzI2+En7gBB3erCCE4jwYaqB1faPUkFCifj5XnRb3p84q+BqRBQkDvyb4JtC0uLWfG2jcMPGghJUqQOB9ZG4U8= X-Received: by 2002:a19:6a06:: with SMTP id u6mr2412346lfu.322.1620298805154; Thu, 06 May 2021 04:00:05 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Mariusz Zaborski Date: Thu, 6 May 2021 12:59:54 +0200 Message-ID: Subject: Re: tail(1) broken in 13-stable To: Peter Jeremy Cc: FreeBSD-Stable ML Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4FbVwR1k7hz4YS2 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of oshogbovx@gmail.com designates 209.85.167.50 as permitted sender) smtp.mailfrom=oshogbovx@gmail.com X-Spamd-Result: default: False [-2.35 / 15.00]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.35)[-0.348]; RCPT_COUNT_TWO(0.00)[2]; FORGED_SENDER(0.30)[oshogbo@freebsd.org,oshogbovx@gmail.com]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RBL_DBL_DONT_QUERY_IPS(0.00)[209.85.167.50:from]; R_DKIM_NA(0.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_NEQ_ENVFROM(0.00)[oshogbo@freebsd.org,oshogbovx@gmail.com]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; SPAMHAUS_ZRD(0.00)[209.85.167.50:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[209.85.167.50:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.167.50:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 May 2021 11:00:08 -0000 Could you provide details how to reproduce this? On Thu, 6 May 2021 at 12:13, Peter Jeremy via freebsd-stable wrote: > > Since updating from 12-stable to 13-stable, I've found that tail(1) > crashes, reporting: > Assertion failed: (procfd > STDERR_FILENO), function service_clean, file /usr/src/lib/libcasper/libcasper/service.c, line 394. > tail: unable to init casper: Socket is not connected > unless all three of stdin, stdout and stderr are open. Whilst it > probably doesn't make sense to call tail without stdout open. there's > no obvious reason to require that stdin or stderr must be open. > > -- > Peter Jeremy From owner-freebsd-stable@freebsd.org Thu May 6 11:49:33 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 17C605FD0EC for ; Thu, 6 May 2021 11:49:33 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vtr.rulingia.com (vtr.rulingia.com [IPv6:2001:19f0:5801:ebe:5400:1ff:fe53:30fd]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA512 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "vtr.rulingia.com", Issuer "R3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FbX1R6vhLz4b8j; Thu, 6 May 2021 11:49:31 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from server.rulingia.com (ppp239-208.static.internode.on.net [59.167.239.208]) by vtr.rulingia.com (8.16.1/8.15.2) with ESMTPS id 146BnLnS002834 (version=TLSv1.3 cipher=AEAD-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 6 May 2021 21:49:27 +1000 (AEST) (envelope-from peter@rulingia.com) DKIM-Filter: OpenDKIM Filter v2.10.3 vtr.rulingia.com 146BnLnS002834 X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.16.1/8.16.1) with ESMTPS id 146BnGDl084588 (version=TLSv1.3 cipher=AEAD-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 6 May 2021 21:49:16 +1000 (AEST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.16.1/8.16.1/Submit) id 146BnGEO084587; Thu, 6 May 2021 21:49:16 +1000 (AEST) (envelope-from peter) Date: Thu, 6 May 2021 21:49:16 +1000 From: Peter Jeremy To: Mariusz Zaborski Cc: FreeBSD-Stable ML Subject: Re: tail(1) broken in 13-stable Message-ID: References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="mG8r4KKZYcdbWuqu" Content-Disposition: inline In-Reply-To: X-PGP-Key: http://www.rulingia.com/keys/peter.pgp X-Rspamd-Queue-Id: 4FbX1R6vhLz4b8j X-Spamd-Bar: ------ X-Spamd-Result: default: False [-6.10 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[rulingia.com:s=default]; FREEFALL_USER(0.00)[peter]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; NEURAL_HAM_LONG(-1.00)[-1.000]; SPAMHAUS_ZRD(0.00)[2001:19f0:5801:ebe:5400:1ff:fe53:30fd:from:127.0.2.255]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[rulingia.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[rulingia.com,quarantine]; NEURAL_HAM_SHORT(-1.00)[-0.998]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:19f0:5801:ebe:5400:1ff:fe53:30fd:from]; ASN(0.00)[asn:20473, ipnet:2001:19f0:5800::/38, country:US]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 May 2021 11:49:33 -0000 --mG8r4KKZYcdbWuqu Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2021-May-06 12:59:54 +0200, Mariusz Zaborski wrote: >Could you provide details how to reproduce this? > >On Thu, 6 May 2021 at 12:13, Peter Jeremy via freebsd-stable > wrote: >> >> Since updating from 12-stable to 13-stable, I've found that tail(1) >> crashes, reporting: >> Assertion failed: (procfd > STDERR_FILENO), function service_clean, file= /usr/src/lib/libcasper/libcasper/service.c, line 394. >> tail: unable to init casper: Socket is not connected >> unless all three of stdin, stdout and stderr are open. Whilst it >> probably doesn't make sense to call tail without stdout open. there's >> no obvious reason to require that stdin or stderr must be open. server% tail /COPYRIGHT <&- Assertion failed: (procfd > STDERR_FILENO), function service_clean, file /u= sr/src/lib/libcasper/libcasper/service.c, line 394. tail: unable to init casper: Socket is not connected --=20 Peter Jeremy --mG8r4KKZYcdbWuqu Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEE7rKYbDBnHnTmXCJ+FqWXoOSiCzQFAmCT17ZfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEVF QjI5ODZDMzA2NzFFNzRFNjVDMjI3RTE2QTU5N0EwRTRBMjBCMzQACgkQFqWXoOSi CzSKLQ/+JbhQdFOX+lehbDTXJ5lZg8mlDSW6yWEu+HKO7t2sowJ9ysbJWFt763rK 7oLhu07D4qq1WnjU2VEXIfz5dEFFZ1xkO35RGbrcb8K0D1mp8unTLQBEzAiNDsqS DGdKw7DsxGdkdJvmOWnmMbMFv+xjoh1QEWOCK6D21XuATHi9RkH1eamr0j1hS6yt 9LKrOmCIBzzKBVwqrnunvvBGkjyliB4U3BsVBa68Ekp5a6yhUlO3zMmx9m9kDTol lcJER22wSTXt4j3rT4r3xXTbJkpqm2WQxs/3QrmJQUePtd5A6ULDxU0wwNpUMzP1 85Zifr+t+myE18AIy9k7EaaI5PZAdRq3YJAXPk4BmyeRNhaBx5Oev1f/yD9tDQUp a8QipapTbDGVhflpME/1HEgetvkwsMuGtoCYxaxoeJbg6lyWGBvOd2pGTljkWffh 4MtB9jY/W757YTbVi4XJQ7XtAb6eoHJsvQRBZd3Fb1ZYZqwfoACiQSpEC1exrrbd noQuru3ob9ElQagO4paKDYjf0tHqEruWu8XWF9YiXXOP3b9acUBLoQmno8+GYSEv TczCut49oSTtKjDLZ3U0hwEVDTGTYaBvPIL6bFMJaOAtYVQFvfI+HjmqWWBB93aJ 0Tpi2BhVDsdyGpIKfqvi4d5O8Ii7jVuTF13YsSrJsPZ5XCezgJY= =nS0S -----END PGP SIGNATURE----- --mG8r4KKZYcdbWuqu-- From owner-freebsd-stable@freebsd.org Thu May 6 21:02:31 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 56BC362DEE2 for ; Thu, 6 May 2021 21:02:31 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic305-21.consmr.mail.gq1.yahoo.com (sonic305-21.consmr.mail.gq1.yahoo.com [98.137.64.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FbmHV2SYwz54GQ for ; Thu, 6 May 2021 21:02:30 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1620334948; bh=9idmAMRoYEBKxRPf4rKd83t/6k/2NGYc86nWV71BYfT=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=RWBMw0aEnXjF4DD+dIswG50HQIWhL/mNSAQEsGHVDyUgKKMBknyeGX2rMIvwkNOed3UJww6Zv19cqITK1bolUYHD1W4IYHn8t1ZY43n3KsZun0VH+uWRwzJyDmxCPfceF73clxDc20i0PtxzdY0lpaACRG4ATDcbIODBerWUHHyoulB7zhA4FphggFySv08AMhA5P2jXp5nO5tPjxd4K0G14Wi1ibtCJWVgI5o+4oFXqIvxLVB726c6nWMDjTBa5xs33kYmaCOat4J8mYL0mdoLI4wMXIGUgLAV9d6UIlm33aN5mHAUo6FCMuytXfYsuy/eJna7fJdNf3Fw4K+sE6w== X-YMail-OSG: loYcaTcVM1n1r_w2jeKy5RbrCdPndcU98Ss5S3nxUyOelko4t_FqgN5ooOsOtMW ElxMY3pgDgIyafQHjoBX.XeEHqXQ3OxC.K4K4peUlVDEvnOi99yiqZlAJLHODZOzatSY9s.wNFzB EcLd8IwkMBB82Nl5LUEVM_WI7qK__WMNE7_Jqwnf31vot5NOKPYxBhyYcHSvqOaS_r9QeQ9AI4yn OX8RpqqluVNBoCPpoHFw0_UrssV22BJde4nU.TpyZo67kqhFz90LGyTvnSTcmr9YME7bd1zv59VZ xb3glcOe3zaAd2p0ZNMCOZ6fQiuBYJJLCA7gGW3Gg0TqoqTPG5JQmve6fVFb5SBZ01RKn8npEYiV WLeG7Zr8BRj9HBveJbcKdaknr5AKpddvfE8Kt7NJDANXIDPRwZqcGjaw5VSM7ptSH1VGttsadI5Z IOCD3hAPHGW6FVancxEIZ1F8D974xV7KXoqoAsAmZ.MjmL_FExUUN0WxjlH6Ey9smqzcEomPc24h jp1msSm1Nv6Y4tTI2_69P7Iv3bqyVMepbQbHAvZO9H2NhGeYq.VbpMCgl2_.wIWaBNDsMHvv_Yqb mCeOECNJcMGbKx5l9VsPVm0D2qMIZzHCvyqCsXHuVKRNVTNTfyelzaLX6ldgFmM.KMMqz1ejhFM_ 7tW5Qa0.f.IEWZsnqFeRYRnE0TAQfWIXXUrckQ.jh.b.WLXOFHz52BD_su9bx.G.mUX5J.LCnHBp vaRp_ZwakbBXpc_R4D_oqXKcA97w.3w.pyrD.EYSg5VnNKZKzV409eMqnN0tJXy6OgVWGNFgFyDS BfGdlgjf4V4DS5JTn86gFPvxjlhvzvBarq.UVuq_.loHHIKVXCGjVzuvLVBOTAe5nMejAN0N42bb R1KDR66jOg4rN8pAJfrHvYEmClVpzApeosFjGt34XwvvQLnRRgPx19dnvTGrAI0_dAAqFGRGQV0H mA1LxLBr6VgaZYdNqUliPuGRPDEwOH18s1yfoQxecoVw5exR1ak4ESsFESbZxl37b36NyCJWNi2L iDTgsaIW6vRD_9zsbfR7x82WQUe0pAOiS3zuwTnYP9puHaRDURVXGNPFrbLlxLu8GSB28x0PEJQg 7.CP21uOlgutDMx2xzBmsZOUmkxusdJuJ_KoljTRDmjBEXqxjz2N.Txdn0QQrlI3xLCPm2lf2pMl fn5MjD5HUi2oF3CchBNkBpuZ.P0fz861TE_1IiH5e7ggoJN6wQkbbxXC5nSX3eXLwDPm4lPlNYOq FQFDzJ3XDIppysxem3ZXu7m7pWXORNU7YGQl8eDAtpZOpQJTfOBP82i4mZ3gBOwPZwqzBfdua19z TcGeEDeDFotZzWs4rirnfzpha0xBidBUBrKrvPRqbq_.pcQ3B0iSEc.aSyRzskvk_KK.urJLmMH4 p_PZU5qEatVaabaSHCYKXx1VijlGNbJqIgp53RbJJImVyiDLed3VH84yKXMjB3LPcVvJQbsx086u Kx2qOdCMBo_yz1bNDNGFEsK17DKy9Q2mnFhlfA.c8WzrVVlznfagM6ep7lwxep5pZi4u0wQJM5xX Lf31uEIbYqCBMrAHn8DYshKEupGVpnqkeAi2YJN55y5SODZM0MBJ9.7zd_1JWgFMBbw4BJq1Ew05 xV_aQQpNHV0NbAoyugG79A9uB8t5LMqpUDgRIeuqN0X1_QhA2p4UB66zmkXpA5rpsL2PUDcDNfXg jUsY87Fq27QWrh7I1twLpyAtCGKk09ELsBZHpxNjTNW2dEzXCaqIOEcvzNMWY5aC1HqvAOepFoTe WpdN.RMHVZTgjl7nN3Hr93c8UM_nQEpRvalStEaDeXK4b_7WN9IdU7WPzUxEevKNcXPyGq1imOUw hduG.ALCVPNPZ4Je_PV6uzMdNSEG66fzWef..NiIS9YXOO4I05SoReV.cp4S.V6JaDwric0x80A1 eaO79FI65RjJPI.zGy8POFl3MHc7a6znMLdXozWu0CgmdyGPHS3y2ZQ7GvkRD_xU_ch7_TYW_Tfo Ts58vhiFS9SHQ47vgMsQY.p7B7DA7LbpbCa2npoodo66v8.BCTTuDZEDlmuoB7vBaTVLqMr7lDH5 .22FUGtpRkofnAnOz3NSYLaRqOQp_32H3.8WWpdcVD8XtM.XVYx3YDmz2GEs7oVDPM8OGJaQjJMo n1yeTheRxAbKDGWnZroJLMhPwFEloXOMsFhVEOpNI7vygnLImVU9ttZE_A7dffY5u2iqvUMUIOXG IPJgFh3M2SRnkXHIJwXCw0gz064QbtiMRc.JrUxFUP2Km5oRFIJoGyYmY4ME.7zm9FTEBwdWI_bG G4eqb905keFh_9hDnpkzLzPlK2wUbVB2Gu8OYjE8z_tEauNw_NIsnjRYN3wSolJPcET9tFFqano6 hPjua6CsO27x2wa85OTK09YenOjmMIx4I6olwknaDmofrN5W7gdTOiRpEkfr_ZEFQVPj0SsFnZqQ x5kMjY6koP51TJL9ZdkJKlL6.riwV._VHxOqPQ_7M_o2QLczSxsvw2bDKJri_LhG_ATHzt_U_Ekv IQMuYD6lnBxqt5EavOFOsz4JjmsmFAmK4iPkshA_NEKhEOF0nK2MbRfNUKzg- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.gq1.yahoo.com with HTTP; Thu, 6 May 2021 21:02:28 +0000 Received: by kubenode575.mail-prod1.omega.gq1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID e2347f0c5b114a50a47b3b2e7d35b70a; Thu, 06 May 2021 21:02:27 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.80.0.2.43\)) Subject: Fresh releng/13.0 release/13.0.0 install: "newsyslog: malformed 'at' value" messages Message-Id: <2206517F-7F5F-458D-9956-70EF71DA9C0F@yahoo.com> Date: Thu, 6 May 2021 14:02:26 -0700 Cc: freebsd-arm To: FreeBSD-STABLE Mailing List X-Mailer: Apple Mail (2.3654.80.0.2.43) References: <2206517F-7F5F-458D-9956-70EF71DA9C0F.ref@yahoo.com> X-Rspamd-Queue-Id: 4FbmHV2SYwz54GQ X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.42 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.95)[-0.954]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.64.84:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-0.97)[-0.967]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[98.137.64.84:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.84:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.84:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 May 2021 21:02:31 -0000 Having used bsdinstall to make a USB3 SSD on a RPi4B (zfs-on-root, GPT parition, RPi4B materials copied copied to msdos file system), booting gets error notices: newsyslog: malformed 'at' value: /var/log/all.log 600 7 * @T00 J newsyslog: malformed 'at' value: /var/log/auth.log 600 7 1000 @0101T JC newsyslog: malformed 'at' value: /var/log/daily.log 640 7 * @T00 JN newsyslog: malformed 'at' value: /var/log/maillog 640 7 * @T00 JC newsyslog: malformed 'at' value: /var/log/messages 644 5 1000 @0101T JC newsyslog: malformed 'at' value: /var/log/utx.log 644 3 * @01T05 B newsyslog: malformed 'at' value: /var/log/daemon.log 644 5 1000 @0101T JC It is apparently complaining about some of the content in: # more /etc/newsyslog.conf # configuration file for newsyslog # $FreeBSD$ # # Entries which do not specify the '/pid_file' field will cause the # syslogd process to be signalled when that log file is rotated. This # action is only appropriate for log files which are written to by the # syslogd process (ie, files listed in /etc/syslog.conf). If there # is no process which needs to be signalled when a given log file is # rotated, then the entry for that file should include the 'N' flag. # # Note: some sites will want to select more restrictive protections than = the # defaults. In particular, it may be desirable to switch many of the = 644 # entries to 640 or 600. For example, some sites will consider the # contents of maillog, messages, and lpd-errs to be confidential. In = the # future, these defaults may change to more conservative ones. # # logfilename [owner:group] mode count size when flags = [/pid_file] [sig_num] /var/log/all.log 600 7 * @T00 J /var/log/auth.log 600 7 1000 @0101T JC /var/log/console.log 600 5 1000 * J /var/log/cron 600 3 1000 * JC /var/log/daily.log 640 7 * @T00 JN /var/log/debug.log 600 7 1000 * JC /var/log/init.log 644 3 1000 * J /var/log/kerberos.log 600 7 1000 * J /var/log/maillog 640 7 * @T00 JC /var/log/messages 644 5 1000 @0101T JC /var/log/monthly.log 640 12 * $M1D0 JN /var/log/devd.log 644 3 1000 * JC /var/log/security 600 10 1000 * JC /var/log/utx.log 644 3 * @01T05 B /var/log/weekly.log 640 5 * $W6D0 JN /var/log/daemon.log 644 5 1000 @0101T JC /etc/newsyslog.conf.d/[!.]*.conf /usr/local/etc/newsyslog.conf.d/[!.]*.conf Specifically, the 7 lines with "@" involved under "when" get the complaints. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-stable@freebsd.org Thu May 6 22:55:27 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6A96C6307FD for ; Thu, 6 May 2021 22:55:27 +0000 (UTC) (envelope-from monochrome@twcny.rr.com) Received: from p-impout008.msg.pkvw.co.charter.net (p-impout008aa.msg.pkvw.co.charter.net [47.43.26.139]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fbpnp2jqHz58pC for ; Thu, 6 May 2021 22:55:25 +0000 (UTC) (envelope-from monochrome@twcny.rr.com) Received: from [192.168.13.11] ([45.47.45.55]) by cmsmtp with ESMTP id emtqlwPmZnDwfemtrlZepZ; Thu, 06 May 2021 22:55:19 +0000 X-Authority-Analysis: v=2.4 cv=I5Kg+Psg c=1 sm=1 tr=0 ts=609473d7 a=oJ4f3sEZGTo/oTi6ieWQlw==:117 a=oJ4f3sEZGTo/oTi6ieWQlw==:17 a=N659UExz7-8A:10 a=Nldk7IltU9k3B4gmDgoA:9 a=pILNOxqGKmIA:10 Subject: Re: tail(1) broken in 13-stable To: freebsd-stable@freebsd.org References: From: monochrome Message-ID: <9d19ca94-e2e7-0300-6d76-4fa5ddf02eec@twcny.rr.com> Date: Thu, 6 May 2021 18:55:18 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.10.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-CMAE-Envelope: MS4xfFW3rpixZaY9U5Pw9WNdLVyyop3X28efB9n5xaKw4EevXfOc1oAtXguyscOMUufcPLDKF7wwqZrr2wnSGL0ayT5V3pVekrznarN9sU5jK4yOKoTOmt5L EqFSk/2XT5e+5q9xo3Y02rOdGC/ogQtmULZ/k0GXTnMwNFUwX39wQx1xT5Q0kFzASJEpb1jNRH7kHTV1nhriqDvh2Znj0u2vS1Y= X-Rspamd-Queue-Id: 4Fbpnp2jqHz58pC X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of monochrome@twcny.rr.com designates 47.43.26.139 as permitted sender) smtp.mailfrom=monochrome@twcny.rr.com X-Spamd-Result: default: False [-3.30 / 15.00]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[45.47.45.55:received]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[47.43.26.139:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:47.43.26.0/24]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[rr.com]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[47.43.26.139:from:127.0.2.255]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; RBL_DBL_DONT_QUERY_IPS(0.00)[47.43.26.139:from]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:40294, ipnet:47.43.24.0/21, country:US]; MIME_TRACE(0.00)[0:+]; MAILMAN_DEST(0.00)[freebsd-stable]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 May 2021 22:55:27 -0000 On 5/6/21 7:49 AM, Peter Jeremy via freebsd-stable wrote: > tail /COPYRIGHT <&- I get a different error on a 13.0-RELEASE machine I converted from 12 to current about a year ago: $ tail /COPYRIGHT <&- tail: can't limit stdio rights: Bad file descriptor From owner-freebsd-stable@freebsd.org Thu May 6 23:07:26 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7D938631171 for ; Thu, 6 May 2021 23:07:26 +0000 (UTC) (envelope-from monochrome@twcny.rr.com) Received: from p-impout003.msg.pkvw.co.charter.net (p-impout003aa.msg.pkvw.co.charter.net [47.43.26.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fbq3d14K9z3BmN for ; Thu, 6 May 2021 23:07:24 +0000 (UTC) (envelope-from monochrome@twcny.rr.com) Received: from [192.168.13.11] ([45.47.45.55]) by cmsmtp with ESMTP id en5XlsRrTYRZ9en5XlFJAm; Thu, 06 May 2021 23:07:23 +0000 X-Authority-Analysis: v=2.4 cv=DtiTREz+ c=1 sm=1 tr=0 ts=609476ab a=oJ4f3sEZGTo/oTi6ieWQlw==:117 a=oJ4f3sEZGTo/oTi6ieWQlw==:17 a=N659UExz7-8A:10 a=6I5d2MoRAAAA:8 a=Y8dqpXDON2tZQHkyz9IA:9 a=pILNOxqGKmIA:10 a=IjZwj45LgO3ly-622nXo:22 Subject: Re: tail(1) broken in 13-stable To: freebsd-stable@freebsd.org References: From: monochrome Message-ID: Date: Thu, 6 May 2021 19:07:23 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.10.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-CMAE-Envelope: MS4xfHkcFIBp1WK10gYn4Oj7ABlLNhEVi6SJd96tUR6yFutBQqh+OSr9bXAJcR6rSwP5lVCPiZijT77TC5lRd9Q9vMt334mLuho+XNdd7Ub0iws3ixLa5jJd u2ZvfiDuN6OdZPiMmhZim+wpeOZJroSrDVFcq15gQbKAzzssC/71uVi7ZVPyrc2KCJWU9PcHfvP4D0beL+VWam2R4HThEetzIz8= X-Rspamd-Queue-Id: 4Fbq3d14K9z3BmN X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of monochrome@twcny.rr.com designates 47.43.26.134 as permitted sender) smtp.mailfrom=monochrome@twcny.rr.com X-Spamd-Result: default: False [-3.22 / 15.00]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[45.47.45.55:received]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[47.43.26.134:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:47.43.26.0/24]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[rr.com]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[47.43.26.134:from:127.0.2.255]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.92)[-0.924]; RCVD_IN_DNSWL_NONE(0.00)[47.43.26.134:from]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:40294, ipnet:47.43.24.0/21, country:US]; MIME_TRACE(0.00)[0:+]; MAILMAN_DEST(0.00)[freebsd-stable]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 May 2021 23:07:26 -0000 On 5/6/21 7:49 AM, Peter Jeremy via freebsd-stable wrote: > On 2021-May-06 12:59:54 +0200, Mariusz Zaborski wrote: >> Could you provide details how to reproduce this? >> >> On Thu, 6 May 2021 at 12:13, Peter Jeremy via freebsd-stable >> wrote: >>> >>> Since updating from 12-stable to 13-stable, I've found that tail(1) >>> crashes, reporting: >>> Assertion failed: (procfd > STDERR_FILENO), function service_clean, file /usr/src/lib/libcasper/libcasper/service.c, line 394. >>> tail: unable to init casper: Socket is not connected >>> unless all three of stdin, stdout and stderr are open. Whilst it >>> probably doesn't make sense to call tail without stdout open. there's >>> no obvious reason to require that stdin or stderr must be open. > > server% tail /COPYRIGHT <&- > Assertion failed: (procfd > STDERR_FILENO), function service_clean, file /usr/src/lib/libcasper/libcasper/service.c, line 394. > tail: unable to init casper: Socket is not connected > (fixed quote with some additional tests) I get a different error on a 13.0-RELEASE machine I converted from 12 to current about a year ago (bash and sh): $ tail /COPYRIGHT <&- tail: can't limit stdio rights: Bad file descriptor works with sudo: $ sudo tail /COPYRIGHT <&- * California, Berkeley and its contributors." etc etc... but not as root, with a different error in sh: # tail /COPYRIGHT <&- Missing name for redirect. as root in bash: # tail /COPYRIGHT <&- tail: can't limit stdio rights: Bad file descriptor works as both user and root in ksh From owner-freebsd-stable@freebsd.org Fri May 7 04:48:17 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EBE556371AF for ; Fri, 7 May 2021 04:48:17 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vtr.rulingia.com (vtr.rulingia.com [IPv6:2001:19f0:5801:ebe:5400:1ff:fe53:30fd]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA512 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "vtr.rulingia.com", Issuer "R3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fbycw4s8Qz3R4G for ; Fri, 7 May 2021 04:48:16 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from server.rulingia.com (ppp239-208.static.internode.on.net [59.167.239.208]) by vtr.rulingia.com (8.16.1/8.15.2) with ESMTPS id 1474m5Av010008 (version=TLSv1.3 cipher=AEAD-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 7 May 2021 14:48:11 +1000 (AEST) (envelope-from peter@rulingia.com) DKIM-Filter: OpenDKIM Filter v2.10.3 vtr.rulingia.com 1474m5Av010008 X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.16.1/8.16.1) with ESMTPS id 1474m07E071924 (version=TLSv1.3 cipher=AEAD-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 7 May 2021 14:48:00 +1000 (AEST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.16.1/8.16.1/Submit) id 1474m0on071923; Fri, 7 May 2021 14:48:00 +1000 (AEST) (envelope-from peter) Date: Fri, 7 May 2021 14:48:00 +1000 From: Peter Jeremy To: monochrome Cc: freebsd-stable@freebsd.org Subject: fileargs_init(3) doesn't work without CAPABILITIES (was: Re: tail(1) broken in 13-stable) Message-ID: References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="ApwmoHD7ukuZUtXS" Content-Disposition: inline In-Reply-To: X-PGP-Key: http://www.rulingia.com/keys/peter.pgp X-Rspamd-Queue-Id: 4Fbycw4s8Qz3R4G X-Spamd-Bar: ------ X-Spamd-Result: default: False [-6.10 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[rulingia.com:s=default]; FREEFALL_USER(0.00)[peter]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; SPAMHAUS_ZRD(0.00)[2001:19f0:5801:ebe:5400:1ff:fe53:30fd:from:127.0.2.255]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[rulingia.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[rulingia.com,quarantine]; NEURAL_HAM_SHORT(-1.00)[-1.000]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:19f0:5801:ebe:5400:1ff:fe53:30fd:from]; ASN(0.00)[asn:20473, ipnet:2001:19f0:5800::/38, country:US]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 May 2021 04:48:18 -0000 --ApwmoHD7ukuZUtXS Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2021-May-06 19:07:23 -0400, monochrome wrote: =2E.. >On 5/6/21 7:49 AM, Peter Jeremy via freebsd-stable wrote: =2E.. >> server% tail /COPYRIGHT <&- >> Assertion failed: (procfd > STDERR_FILENO), function service_clean, file= /usr/src/lib/libcasper/libcasper/service.c, line 394. >> tail: unable to init casper: Socket is not connected >I get a different error on a 13.0-RELEASE machine I converted from 12 to= =20 >current about a year ago (bash and sh): > >$ tail /COPYRIGHT <&- >tail: can't limit stdio rights: Bad file descriptor I've done some more testing across a number of systems and narrowed the difference in behaviour down to the presence of the CAPABILITIES option in the kernel (it looks like I never added it to my kernel config on that system): If CAPABILITIES is present then the cap_rights_limit(2) call for the closed FD fails, generating the "can't limit stdio rights" error. (Whether this behaviour is reasonable is a different issue - it was introduced in r348708, based on https://reviews.freebsd.org/D20393 and the issue of closed file descriptors doesn't seem to have been considered). If CAPABILITIES is not present then the cap_rights_limit() failure is (correctly) ignored but the subsequent fileargs_init(3) call gets upset at opening a FD <=3D 2. This behaviour seems wrong - if CAPABILITIES aren't present in the kernel then the userland behaviour should be the same as if WITHOUT_CASPER is specified. IMO, this is a bug in fileargs_init(3). --=20 Peter Jeremy --ApwmoHD7ukuZUtXS Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEE7rKYbDBnHnTmXCJ+FqWXoOSiCzQFAmCUxnNfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEVF QjI5ODZDMzA2NzFFNzRFNjVDMjI3RTE2QTU5N0EwRTRBMjBCMzQACgkQFqWXoOSi CzQucQ//dIKgwdPp3VbIZFFposFSDzam/5Sv9jFVXKpurBozlQnlHevQczxIbgt9 8jDrJk/N1SgGJOAvFYklnDgyxOF33NDlNcIltnpsvPqa82cLwTdIONKY446rrh33 ZaIYxhmgYt5Lape0iLWuRA1AX6dCJ/5K+z6eFcgB0+EWVMALvonSYfVmTTRb/hqk YxhaEZayh5ugKgs+9d1cmh3wEpBEqm7sTdM5N9ivO2KJK/1T0jrkiyG2vDN3vjYj 5tRqYN4gaWjz46Bt5sPeW9tbMXf9GlyERuWxDcl692QLwbBw2Dm3FTPCBzyL9XlQ mHVx09rF+8Tayw2kKNZkUSLc42EVVCxJnwBzruVkGgBccnlUp9bBUZbTWdQFDgAJ kns1nsCS8+R5p6AVLVtDni8gnyszHPUUzNwetcKERfPVrWtjsN3hsb4ud03BLIRl uCaFEdv4mdewZmi5JMpeatb+T/wx6cVov1p8rOX0mk74d/0lIOD51HFcMTxICMzH o4K6wIlaU7Q2dvzDwSdAoe5l+SGrR2ReVGiIy45i84XLrvvEx23MPzg+tS4tVEfM PT3a9g8L17CEA0/k9j0mTEqTmJMHFQxxWfL+ZDRfpfQOoyELMwMfUj0ApfjP/kl6 QglE+H9e58+XCG2UgE0VMvbhVS7avOgPVgiOEp15/nwrZR6Lldw= =LRic -----END PGP SIGNATURE----- --ApwmoHD7ukuZUtXS-- From owner-freebsd-stable@freebsd.org Fri May 7 12:48:53 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4D62162C3C1 for ; Fri, 7 May 2021 12:48:53 +0000 (UTC) (envelope-from yasu@utahime.org) Received: from maybe.home.utahime.org (gate.home.utahime.org [183.180.29.210]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fc9HS1tcjz4Z5J for ; Fri, 7 May 2021 12:48:51 +0000 (UTC) (envelope-from yasu@utahime.org) Received: from eastasia.home.utahime.org (eastasia.home.utahime.org [192.168.174.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by maybe.home.utahime.org (Postfix) with ESMTPS id 24A4228694 for ; Fri, 7 May 2021 21:48:43 +0900 (JST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=utahime.org; s=maybe2019112701; t=1620391723; bh=U5Bk/cR+beeBRsOwWEqPCwhMR8mrBXtQCKdBWo2Zr80=; h=Date:To:Subject:From; b=Kfq9vLp+0/XG1dprpewsahUFeS4j4l/NlNLpTerwu1KVcRSvxvJcA3VM+o9C/Tqqp SvuHI1FeLLJ+MC+SxMDlL3j/xbuaeq3a3fxUB2AUauTJ8LB3r9ATz+O7ZVA+oC/KP5 k7sfjZHg3NOPOkde9rAOZI9ZSfRfFxtyin9v92oVUPt5IsOcGWTBh9HHD318xox/eg WjDjEYv+FIdlITbTw70JHBtoqw1HPvSrb8hH2LTO3En1keiqqAQs9Wp5tH5lHyyjQ1 /T/ggJyUFijwTFgsKgWiwHPFGb0r2xjg3ZGFQx/M0eHIW9zb7tlFWlUgM9av1uGDU9 zJVhl4YHCyx+Q== Received: from localhost (rolling.home.utahime.org [192.168.174.11]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by eastasia.home.utahime.org (Postfix) with ESMTPSA id 2118E213C9; Fri, 7 May 2021 21:48:42 +0900 (JST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.2 at eastasia.home.utahime.org Date: Fri, 07 May 2021 21:47:59 +0900 (JST) Message-Id: <20210507.214759.1825935389016318351.yasu@utahime.org> To: freebsd-stable@freebsd.org Subject: Install of 13.0-RELEASE i386 with ZFS root hangs up From: Yasuhiro Kimura X-Mailer: Mew version 6.8 on Emacs 27.2 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4Fc9HS1tcjz4Z5J X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=utahime.org header.s=maybe2019112701 header.b=Kfq9vLp+; dmarc=none; spf=pass (mx1.freebsd.org: domain of yasu@utahime.org designates 183.180.29.210 as permitted sender) smtp.mailfrom=yasu@utahime.org X-Spamd-Result: default: False [-0.70 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+a:spf-authorized.utahime.org]; TO_DN_NONE(0.00)[]; HFILTER_HELO_IP_A(1.00)[maybe.home.utahime.org]; HFILTER_HELO_NORES_A_OR_MX(0.30)[maybe.home.utahime.org]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[utahime.org:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[183.180.29.210:from]; ASN(0.00)[asn:2519, ipnet:183.180.0.0/16, country:JP]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[utahime.org:s=maybe2019112701]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[utahime.org]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[183.180.29.210:from:127.0.2.255]; MID_CONTAINS_FROM(1.00)[]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 May 2021 12:48:53 -0000 Hello, Does anyone succeed to install 13.0-RELEASE i386 with ZFS root? I tried this with VirtualBox and VMware Player on Windows with following VM condition. * 4 CPUs * 8GB memory * 100GB disk * Bridge mode NIC But in both cases, VM gets high CPU load and hangs up after I moved to 'YES' at 'ZFS Configuration' menu and type return key. If I select UFS root installation completes successfully. So the problem is specific to ZFS root. --- Yasuhiro Kimura From owner-freebsd-stable@freebsd.org Fri May 7 13:53:39 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 44D6C62E558 for ; Fri, 7 May 2021 13:53:39 +0000 (UTC) (envelope-from yasu@utahime.org) Received: from maybe.home.utahime.org (gate.home.utahime.org [183.180.29.210]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FcBk94cBsz4dLV for ; Fri, 7 May 2021 13:53:37 +0000 (UTC) (envelope-from yasu@utahime.org) Received: from eastasia.home.utahime.org (eastasia.home.utahime.org [192.168.174.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by maybe.home.utahime.org (Postfix) with ESMTPS id 44D8B2862E for ; Fri, 7 May 2021 22:53:32 +0900 (JST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=utahime.org; s=maybe2019112701; t=1620395612; bh=D/B8zFKawI4JGXpNbmQjWA2+/1juRNVAxxuygbE900A=; h=Date:To:Subject:From:In-Reply-To:References; b=O5x5aSB3odGBO3PeQqXj73MQQTUvd9v0GItD0vBZWoGSwc8v6p1QQBEAccp22JI7L 8whnjQCE+WYkJ9v1+kTHYtcfEje+rk6Stl9iF3uxxQdxHiPrgYWcZDUbuNGXrgnXlD qGdV5++CinW7TRwnj3uNUjQI+X6tu/AD6FBPIdd6JpFCQqO7Puu4xgTwAef4lZDmEi GA9zhInXc9rF6DyjQpPbUyBiUrOPx0Ik9+WvO/q5EFmN6wvtdcZeF/Eun+WKqKbzaf 01WtzQpPno8iFHTxBqMq0m/3732uYXXWiSZyyX5vI/r/uj9iXw6gspTI47p6jfCMg3 6NndGPfIGPoFQ== Received: from localhost (rolling.home.utahime.org [192.168.174.11]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by eastasia.home.utahime.org (Postfix) with ESMTPSA id 0E18621514; Fri, 7 May 2021 22:53:30 +0900 (JST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.2 at eastasia.home.utahime.org Date: Fri, 07 May 2021 22:53:10 +0900 (JST) Message-Id: <20210507.225310.1688354962068074616.yasu@utahime.org> To: freebsd-stable@freebsd.org Subject: Re: Install of 13.0-RELEASE i386 with ZFS root hangs up From: Yasuhiro Kimura In-Reply-To: <202105071341.147Dfj4g005701@nuc.oldach.net> References: <20210507.214759.1825935389016318351.yasu@utahime.org> <202105071341.147Dfj4g005701@nuc.oldach.net> X-Mailer: Mew version 6.8 on Emacs 27.2 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4FcBk94cBsz4dLV X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=utahime.org header.s=maybe2019112701 header.b=O5x5aSB3; dmarc=none; spf=pass (mx1.freebsd.org: domain of yasu@utahime.org designates 183.180.29.210 as permitted sender) smtp.mailfrom=yasu@utahime.org X-Spamd-Result: default: False [-0.70 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+a:spf-authorized.utahime.org]; TO_DN_NONE(0.00)[]; HFILTER_HELO_IP_A(1.00)[maybe.home.utahime.org]; HFILTER_HELO_NORES_A_OR_MX(0.30)[maybe.home.utahime.org]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[utahime.org:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[183.180.29.210:from]; ASN(0.00)[asn:2519, ipnet:183.180.0.0/16, country:JP]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[utahime.org:s=maybe2019112701]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[utahime.org]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[183.180.29.210:from:127.0.2.255]; MID_CONTAINS_FROM(1.00)[]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 May 2021 13:53:39 -0000 From: 8zwkhn1@oldach.net (Helge Oldach) Subject: Re: Install of 13.0-RELEASE i386 with ZFS root hangs up Date: Fri, 7 May 2021 15:41:45 +0200 (CEST) > Yasuhiro Kimura wrote on Fri, 07 May 2021 14:47:59 +0200 (CEST): >> Does anyone succeed to install 13.0-RELEASE i386 with ZFS root? > ^^^^ > >> * 8GB memory > > Try with 4GB perhaps? > > Kind regards > Helge Thank you for suggestion. But unfortunately hangup still happens with 4GB memory. --- Yasuhiro Kimura From owner-freebsd-stable@freebsd.org Fri May 7 16:18:50 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 854496341EA for ; Fri, 7 May 2021 16:18:50 +0000 (UTC) (envelope-from gbe@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FcFxk3T0Qz4p4v; Fri, 7 May 2021 16:18:50 +0000 (UTC) (envelope-from gbe@freebsd.org) Received: from localhost (p200300d5d70db088d961eb679867db66.dip0.t-ipconnect.de [IPv6:2003:d5:d70d:b088:d961:eb67:9867:db66]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: gbe) by smtp.freebsd.org (Postfix) with ESMTPSA id 124827082; Fri, 7 May 2021 16:18:49 +0000 (UTC) (envelope-from gbe@freebsd.org) Date: Fri, 7 May 2021 18:18:46 +0200 From: Gordon Bergling To: Glen Barber Cc: freebsd-stable@freebsd.org Subject: Re: Building an 13-STABLE release for i386 Message-ID: References: <20210505190242.GD92026@FreeBSD.org> <20210505193323.GE92026@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210505193323.GE92026@FreeBSD.org> X-Url: X-Operating-System: FreeBSD 12.2-STABLE amd64 X-Host-Uptime: 6:12PM up 5:24, 4 users, load averages: 0.59, 0.50, 0.51 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 May 2021 16:18:50 -0000 On Wed, May 05, 2021 at 07:33:23PM +0000, Glen Barber wrote: > On Wed, May 05, 2021 at 09:16:19PM +0200, Gordon Bergling wrote: > > On Wed, May 05, 2021 at 07:02:42PM +0000, Glen Barber wrote: > > > On Wed, May 05, 2021 at 08:56:58PM +0200, Gordon Bergling wrote: > > > > I am currently try to build a custom 13-STABLE release for i386, build on > > > > an amd64 system. According to release(7) the following command should > > > > do the trick, but it fails with the following error message. > > > > > > > > $ doas sh release.sh -c i386/i386.conf > > > > ld-elf32.so.1: Shared object "libedit.so.8" not found, required by "sh" > > > > > > > > Has anyone an idea what could cause this? > > > > > > > > > > Do you have lib32 compatibility installed on the build host? I.e., do > > > you have WITHOUT_LIB32 defined in your src.conf? > > > > > > Glen > > > > lib32 compatibility should be installed, the src.conf of the buildsystem > > (recent 12-STABLE) only has the following entries defined: > > > > WITH_PIE=1 > > WITH_RETPOLINE=1 > > > > Hmm. I can't see why this would be failing for you then. I routinely > do this every week for the development snapshots. > > Is there a chance your system is somehow mismatched regarding userland > and kernel? What version is the build host running? What does > 'uname -UK' show? > > libedit(3) was bumped on Feb 1, 2021 following an update to ncurses(3). > My gut tells me these may be related. I was somehow mistaken how the release.sh process works. I have a local tree with some modifications to the i386 GENERIC kernel config to hopefully get FreeBSD booted on an old Thinkpad X31. Just starting a release build with the default i386.conf always checks out HEAD, no matter of what branch I started the release build from. --Gordon From owner-freebsd-stable@freebsd.org Fri May 7 16:48:19 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7BD9963511F for ; Fri, 7 May 2021 16:48:19 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-qt1-x836.google.com (mail-qt1-x836.google.com [IPv6:2607:f8b0:4864:20::836]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FcGbk3PRHz4qjc for ; Fri, 7 May 2021 16:48:18 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by mail-qt1-x836.google.com with SMTP id j11so7025165qtn.12 for ; Fri, 07 May 2021 09:48:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=eA9mrRXNWm2iLv4230xyEkIHP6Lf6NoG61kdAjcq39g=; b=V4Q0TwsqUyCPPbF3E0szQsGpQXMY/iXFMsWYo78GFZf4UCXmOxIOFNRdHFVawm5nIv 5RmnSmK27y98GKr/voWhA4MUo/JcHKdNLUOQhLHxB/juetoLh5p0Z5fPIs3fW5ZTsPJS /rj0fAj4lWMQhtKuR8SlLf0kHs2rMoqWv/QmmBbzCKFuudXmlhdr2CNUvi2oMdu5Rbwc oKZlQEpqfHgnogiryrZTEZLhJOblvw1m8WU6wJlHthxAMw0bU2FxOfwpAktDI6NGAnZU 7FMnTJGoD04+obiUR4h7n+WiqVZSj8QUkSvNRyMC0OJ/8GdgcipRdVNYjw4+F0t8DPrG 7lZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=eA9mrRXNWm2iLv4230xyEkIHP6Lf6NoG61kdAjcq39g=; b=CgxFLp+8IJ3++DLbWS3LZ0uNUQArYtWbVVSBFkNB+OECQM5SOEbwhHZggDZHFTv9Xe hB7L/FAh2hKDsTZOD5Wqw3t6ClgF0hT+gFEZ9czX+HXGxxhfIidAKI/qkNFo+CXfkdtZ vTj+lrnbw1DssKSCPnvcfKqixWZizJ7eFlJhqUPZ/NOiZNM4JliQJCTOWvq46uaC9CJa DF28AzYsVPBT6zffMLYhO5Q73VCv0s98tB7UB1rbw1FGqk2MdY52yyyeVblB+VNyxykn jsSB9/doqgg7+LIBxhFV6wubrGX6jjRb3VMLNr9ono5swFtgxqcc9CuLQm38g4aT1+jE 0vaQ== X-Gm-Message-State: AOAM531WGkk/TW/h9vcvz8X9e4ZFTsQGO5/zXJDvGVjm8ZV86LxKYe4J Tfchgqy7/ETkz5hheD4NW9NQak6cUipEbmVqqSJ6gtzi X-Google-Smtp-Source: ABdhPJx0BxZTWyrWnBFSrB2aDCMf/4Y0U0KNpGhhxVn06mFzj39b9fp+XChO6UxsZJhn4qiAGKaviGuQQBA+78yA7o0= X-Received: by 2002:a05:622a:1044:: with SMTP id f4mr10703667qte.224.1620406097829; Fri, 07 May 2021 09:48:17 -0700 (PDT) MIME-Version: 1.0 References: <20210507.214759.1825935389016318351.yasu@utahime.org> In-Reply-To: <20210507.214759.1825935389016318351.yasu@utahime.org> From: Freddie Cash Date: Fri, 7 May 2021 09:48:07 -0700 Message-ID: Subject: Re: Install of 13.0-RELEASE i386 with ZFS root hangs up To: Yasuhiro Kimura Cc: FreeBSD Stable X-Rspamd-Queue-Id: 4FcGbk3PRHz4qjc X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=V4Q0Twsq; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of fjwcash@gmail.com designates 2607:f8b0:4864:20::836 as permitted sender) smtp.mailfrom=fjwcash@gmail.com X-Spamd-Result: default: False [-4.00 / 15.00]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::836:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::836:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::836:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 May 2021 16:48:19 -0000 On Fri, May 7, 2021 at 5:49 AM Yasuhiro Kimura wrote: > Does anyone succeed to install 13.0-RELEASE i386 with ZFS root? > > I tried this with VirtualBox and VMware Player on Windows with > following VM condition. > > * 4 CPUs > * 8GB memory > * 100GB disk > * Bridge mode NIC > > But in both cases, VM gets high CPU load and hangs up after I moved > to 'YES' at 'ZFS Configuration' menu and type return key. > > If I select UFS root installation completes successfully. So the > problem is specific to ZFS root. > Running ZFS on 32-bit OSes is doable (although not recommended) but requires a lot of manual configuration and tweaking, especially around kernel memory and ARC usage. You're limited to 4 GB of memory space, so you need to tune the ARC to use less than that. The auto-tuning has improved a lot over the years, but you still need to limit the ARC size to around 2 GB (or less) to keep the system stable. KVA memory space tuning shouldn't be needed anymore, but you can do research into that, just in case. You can compile a custom kernel to enable PAE support, that will sometimes help with memory issues on i386 (and will allow you to use more than 4 GB of system RAM, although individual processes are still limited to 4 GB). If you really need to, you can make ZFS work on i386. If at all possible, though, you really should run it on amd64 instead. -- Freddie Cash fjwcash@gmail.com From owner-freebsd-stable@freebsd.org Fri May 7 19:53:06 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4B87063A788 for ; Fri, 7 May 2021 19:53:06 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FcLhx20G1z3HWg for ; Fri, 7 May 2021 19:53:04 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 147Jqnhh035922 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 7 May 2021 22:52:52 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 147Jqnhh035922 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 147Jqm9p035921; Fri, 7 May 2021 22:52:48 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 7 May 2021 22:52:48 +0300 From: Konstantin Belousov To: Freddie Cash Cc: Yasuhiro Kimura , FreeBSD Stable Subject: Re: Install of 13.0-RELEASE i386 with ZFS root hangs up Message-ID: References: <20210507.214759.1825935389016318351.yasu@utahime.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=0.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM,FREEMAIL_REPLY, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4FcLhx20G1z3HWg X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com X-Spamd-Result: default: False [-0.90 / 15.00]; RCVD_TLS_ALL(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[gmail.com]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:470:d5e7:1::1:from]; R_SPF_SOFTFAIL(0.00)[~all]; NEURAL_SPAM_SHORT(0.87)[0.870]; SPAMHAUS_ZRD(0.00)[2001:470:d5e7:1::1:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; NEURAL_HAM_MEDIUM(-0.77)[-0.766]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 May 2021 19:53:06 -0000 On Fri, May 07, 2021 at 09:48:07AM -0700, Freddie Cash wrote: > On Fri, May 7, 2021 at 5:49 AM Yasuhiro Kimura wrote: > > > Does anyone succeed to install 13.0-RELEASE i386 with ZFS root? > > > > I tried this with VirtualBox and VMware Player on Windows with > > following VM condition. > > > > * 4 CPUs > > * 8GB memory > > * 100GB disk > > * Bridge mode NIC > > > > But in both cases, VM gets high CPU load and hangs up after I moved > > to 'YES' at 'ZFS Configuration' menu and type return key. > > > > If I select UFS root installation completes successfully. So the > > problem is specific to ZFS root. > > > > Running ZFS on 32-bit OSes is doable (although not recommended) but > requires a lot of manual configuration and tweaking, especially around > kernel memory and ARC usage. > > You're limited to 4 GB of memory space, so you need to tune the ARC to use > less than that. The auto-tuning has improved a lot over the years, but you > still need to limit the ARC size to around 2 GB (or less) to keep the > system stable. KVA memory space tuning shouldn't be needed anymore, but > you can do research into that, just in case. > > You can compile a custom kernel to enable PAE support, that will sometimes > help with memory issues on i386 (and will allow you to use more than 4 GB > of system RAM, although individual processes are still limited to 4 GB). i386 kernel uses memory up to 24G since 13.0. PAE only means that devices that can access full 64bit address are allowed to avoid dma bouncing. > > If you really need to, you can make ZFS work on i386. If at all possible, > though, you really should run it on amd64 instead. > > -- > Freddie Cash > fjwcash@gmail.com > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Fri May 7 22:32:43 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 92A2663EBAF; Fri, 7 May 2021 22:32:43 +0000 (UTC) (envelope-from yasu@utahime.org) Received: from maybe.home.utahime.org (gate.home.utahime.org [183.180.29.210]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FcQF60y1fz3hY6; Fri, 7 May 2021 22:32:41 +0000 (UTC) (envelope-from yasu@utahime.org) Received: from eastasia.home.utahime.org (eastasia.home.utahime.org [192.168.174.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by maybe.home.utahime.org (Postfix) with ESMTPS id D405028881; Sat, 8 May 2021 07:32:37 +0900 (JST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=utahime.org; s=maybe2019112701; t=1620426757; bh=oojlDhKEB8WVuD/nOFc+isqS81ruoi1a5JrMO6yhmq4=; h=Date:To:Subject:From:In-Reply-To:References; b=gl1kdsThXC5nA0g9NQkXzSSQJpkhUSKMZnFHz9oxH7XpikxG4d9muf0hUtPmSlW+h rVTp5o++RbK6UOQB40IwKehRcwZHoWBtjTvrIyv0CBDPvyPRwKt3uIapFRDya9EDeU /bbHRD5g3iI7pv3/824sVxNd4IVe+ZDadFgAkFGEyp2VHEXNNNAmZURDYA5KTomqsy TB13EYlRfpB9LaBdvozBANvadEZfEeszaVs33RB6GV14McU6PZcBJkgqHdFjJ9cKjT DDIRnJl5xEAGYf0LHtmd86qn309dPh2JOojfB/RvBk8nqTam25Nook1s0SeWTQkkRj RxQQ+RHS0pFQw== Received: from localhost (rolling.home.utahime.org [192.168.174.11]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by eastasia.home.utahime.org (Postfix) with ESMTPSA id C66A021870; Sat, 8 May 2021 07:32:36 +0900 (JST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.2 at eastasia.home.utahime.org Date: Sat, 08 May 2021 07:31:47 +0900 (JST) Message-Id: <20210508.073147.1934590966598603586.yasu@utahime.org> To: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Loading zfs module results in hangup on i386 (Re: Install of 13.0-RELEASE i386 with ZFS root hangs up) From: Yasuhiro Kimura In-Reply-To: <20210507.214759.1825935389016318351.yasu@utahime.org> References: <20210507.214759.1825935389016318351.yasu@utahime.org> X-Mailer: Mew version 6.8 on Emacs 27.2 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4FcQF60y1fz3hY6 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=utahime.org header.s=maybe2019112701 header.b=gl1kdsTh; dmarc=none; spf=pass (mx1.freebsd.org: domain of yasu@utahime.org designates 183.180.29.210 as permitted sender) smtp.mailfrom=yasu@utahime.org X-Spamd-Result: default: False [0.23 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+a:spf-authorized.utahime.org]; TO_DN_NONE(0.00)[]; HFILTER_HELO_IP_A(1.00)[maybe.home.utahime.org]; HFILTER_HELO_NORES_A_OR_MX(0.30)[maybe.home.utahime.org]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[utahime.org:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.07)[-0.067]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[183.180.29.210:from]; ASN(0.00)[asn:2519, ipnet:183.180.0.0/16, country:JP]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[utahime.org:s=maybe2019112701]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[utahime.org]; SPAMHAUS_ZRD(0.00)[183.180.29.210:from:127.0.2.255]; MID_CONTAINS_FROM(1.00)[]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 May 2021 22:32:43 -0000 From: Yasuhiro Kimura Subject: Install of 13.0-RELEASE i386 with ZFS root hangs up Date: Fri, 07 May 2021 21:47:59 +0900 (JST) > Hello, > > Does anyone succeed to install 13.0-RELEASE i386 with ZFS root? > > I tried this with VirtualBox and VMware Player on Windows with > following VM condition. > > * 4 CPUs > * 8GB memory > * 100GB disk > * Bridge mode NIC > > But in both cases, VM gets high CPU load and hangs up after I moved > to 'YES' at 'ZFS Configuration' menu and type return key. > > If I select UFS root installation completes successfully. So the > problem is specific to ZFS root. Now I think I know what is the source of problem. After all, on 13.0-RELEASE i386 system simply loading zfs module results in system hang up. The steps to reproduce it are, 1. Boot with install media of 13.0-RELEASE i386 2. At the first menu of FreeBSD installer, select 'Shell'. 3. At the shell prompt, type `kldload zfs` and return key. I confirmed hangup happens with VirtualBox, VMware Player and my bare metal PC environement. So the problem doesn't depend on hardware. And hangup also happens with 13-STABLE and 14-CURRENT. --- Yasuhiro Kimura From owner-freebsd-stable@freebsd.org Fri May 7 22:45:04 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6500163F21E; Fri, 7 May 2021 22:45:04 +0000 (UTC) (envelope-from yasu@utahime.org) Received: from maybe.home.utahime.org (gate.home.utahime.org [183.180.29.210]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FcQWM3ygWz3hsH; Fri, 7 May 2021 22:45:03 +0000 (UTC) (envelope-from yasu@utahime.org) Received: from eastasia.home.utahime.org (eastasia.home.utahime.org [192.168.174.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by maybe.home.utahime.org (Postfix) with ESMTPS id CEE5028882; Sat, 8 May 2021 07:44:59 +0900 (JST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=utahime.org; s=maybe2019112701; t=1620427499; bh=s50A3uNf+CRwNVo9l5w3AwJLgRO9xlx7MeesvluQQ+s=; h=Date:To:Subject:From:In-Reply-To:References; b=idVHz58dBd38puEI5dpVIa+j9Bokl0pvS4y+12tnK7NIXHvKqOQz5i7hBtkX93Jva 2fEQE3X4YylsV5+eP50qjUCAtLmhumEmRP1JJOXGuEGPkqcgd8fyzBKYJ56tNdldIK xVdu54lsYsv/VVnXalgIQdSKJH/oJmCemTaY90sqYx3AJPdg4uspmuxsxoqfeWU1sK 8/Z3kN8hIKktvl/LVtMbaMWZAlZHo9gcIIOaXJSgeLX63wt+h2wWIcNGxtEOl2RWYP scn0i6Jc5tz+t0CatoLEd/UfSQzHUMFdAdJ9AbD7jWPBz75zPyeVeFQedKt8qI2tF0 jjAZC1p875HJg== Received: from localhost (rolling.home.utahime.org [192.168.174.11]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by eastasia.home.utahime.org (Postfix) with ESMTPSA id 76ED721913; Sat, 8 May 2021 07:44:58 +0900 (JST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.2 at eastasia.home.utahime.org Date: Sat, 08 May 2021 07:44:15 +0900 (JST) Message-Id: <20210508.074415.2261369884561178196.yasu@utahime.org> To: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Re: Loading zfs module results in hangup on i386 From: Yasuhiro Kimura In-Reply-To: <20210508.073147.1934590966598603586.yasu@utahime.org> References: <20210507.214759.1825935389016318351.yasu@utahime.org> <20210508.073147.1934590966598603586.yasu@utahime.org> X-Mailer: Mew version 6.8 on Emacs 27.2 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4FcQWM3ygWz3hsH X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=utahime.org header.s=maybe2019112701 header.b=idVHz58d; dmarc=none; spf=pass (mx1.freebsd.org: domain of yasu@utahime.org designates 183.180.29.210 as permitted sender) smtp.mailfrom=yasu@utahime.org X-Spamd-Result: default: False [-0.41 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+a:spf-authorized.utahime.org:c]; MV_CASE(0.50)[]; TO_DN_NONE(0.00)[]; HFILTER_HELO_IP_A(1.00)[maybe.home.utahime.org]; HFILTER_HELO_NORES_A_OR_MX(0.30)[maybe.home.utahime.org]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[utahime.org:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.71)[-0.705]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[183.180.29.210:from]; ASN(0.00)[asn:2519, ipnet:183.180.0.0/16, country:JP]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[utahime.org:s=maybe2019112701]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[utahime.org]; SPAMHAUS_ZRD(0.00)[183.180.29.210:from:127.0.2.255]; MID_CONTAINS_FROM(1.00)[]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 May 2021 22:45:04 -0000 From: Yasuhiro Kimura Subject: Loading zfs module results in hangup on i386 (Re: Install of 13.0-RELEASE i386 with ZFS root hangs up) Date: Sat, 08 May 2021 07:31:47 +0900 (JST) > Now I think I know what is the source of problem. After all, on > 13.0-RELEASE i386 system simply loading zfs module results in system > hang up. > > The steps to reproduce it are, > > 1. Boot with install media of 13.0-RELEASE i386 > 2. At the first menu of FreeBSD installer, select 'Shell'. > 3. At the shell prompt, type `kldload zfs` and return key. > > I confirmed hangup happens with VirtualBox, VMware Player and my bare > metal PC environement. So the problem doesn't depend on hardware. > > And hangup also happens with 13-STABLE and 14-CURRENT. This problem is already reported to Bugzilla. Bug 254177 When ZFS is recognized, An i386 machine with a lot of memory hangs. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254177 --- Yasuhiro Kimura From owner-freebsd-stable@freebsd.org Sat May 8 11:33:38 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 85C9F6322C6 for ; Sat, 8 May 2021 11:33:38 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FclZ91Kgdz4nRv for ; Sat, 8 May 2021 11:33:36 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id 148BXOte014974 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sat, 8 May 2021 11:33:25 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: freebsd-stable@freebsd.org Received: from [10.58.0.10] (dadvw [10.58.0.10]) by eg.sd.rdtc.ru (8.16.1/8.16.1) with ESMTPS id 148BXKi8081630 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Sat, 8 May 2021 18:33:20 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: Install of 13.0-RELEASE i386 with ZFS root hangs up To: Konstantin Belousov References: <20210507.214759.1825935389016318351.yasu@utahime.org> Cc: FreeBSD Stable From: Eugene Grosbein Message-ID: <4bee4d1d-46a7-e0d1-48d9-c3e964a05ef0@grosbein.net> Date: Sat, 8 May 2021 18:33:02 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=2.2 required=5.0 tests=BAYES_00,LOCAL_FROM, NICE_REPLY_A,RDNS_NONE,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record * -0.0 SPF_PASS SPF: sender matches SPF record * 1.9 RDNS_NONE Delivered to internal network by a host with no rDNS * 2.6 LOCAL_FROM From my domains * -0.0 NICE_REPLY_A Looks like a legit reply (A) X-Spam-Level: ** X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 4FclZ91Kgdz4nRv X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=fail (mx1.freebsd.org: domain of eugen@grosbein.net does not designate 2a01:4f8:c2c:26d8::2 as permitted sender) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-0.10 / 15.00]; MID_RHS_MATCH_FROM(0.00)[]; R_SPF_FAIL(1.00)[-all]; FREEFALL_USER(0.00)[eugen]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a01:4f8:c2c:26d8::2:from]; ARC_NA(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[grosbein.net]; NEURAL_SPAM_SHORT(1.00)[0.999]; SPAMHAUS_ZRD(0.00)[2a01:4f8:c2c:26d8::2:from:127.0.2.255]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 May 2021 11:33:38 -0000 08.05.2021 2:52, Konstantin Belousov wrote: > i386 kernel uses memory up to 24G since 13.0. > > PAE only means that devices that can access full 64bit address are allowed > to avoid dma bouncing. Maybe you could tell something on similar topic? There is FreeBSD 12.2-STABLE r369567 Base12 amd64 running with Intel Atom CPU capable of long mode and addressing 8GB RAM, ASRock A330ION motherboard and two memory modules installed: 4G+2GB. Why so small "avail memory"? FreeBSD clang version 10.0.1 (git@github.com:llvm/llvm-project.git llvmorg-10.0.1-0-gef32c611aa2) CPU: Intel(R) Atom(TM) CPU 330 @ 1.60GHz (1600.03-MHz K8-class CPU) Origin="GenuineIntel" Id=0x106c2 Family=0x6 Model=0x1c Stepping=2 Features=0xbfe9fbff Features2=0x40e31d AMD Features=0x20000800 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 6442450944 (6144 MB) Physical memory chunk(s): 0x0000000000010000 - 0x000000000009dfff, 581632 bytes (142 pages) 0x0000000000103000 - 0x00000000001fffff, 1036288 bytes (253 pages) 0x0000000002b00000 - 0x00000000d8709fff, 3586170880 bytes (875530 pages) avail memory = 3571384320 (3405 MB) Also http://www.grosbein.net/freebsd/dmidecode.txt From owner-freebsd-stable@freebsd.org Sat May 8 12:00:41 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 132A46335CB; Sat, 8 May 2021 12:00:41 +0000 (UTC) (envelope-from yasu@utahime.org) Received: from maybe.home.utahime.org (gate.home.utahime.org [183.180.29.210]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fcm9M2mxvz4pVM; Sat, 8 May 2021 12:00:38 +0000 (UTC) (envelope-from yasu@utahime.org) Received: from eastasia.home.utahime.org (eastasia.home.utahime.org [192.168.174.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by maybe.home.utahime.org (Postfix) with ESMTPS id 4BDDF28859; Sat, 8 May 2021 21:00:29 +0900 (JST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=utahime.org; s=maybe2019112701; t=1620475229; bh=IF6Zyh5fS4MnuUV0ma9/ENb+iV6619cs6s+3v4LPb1k=; h=Date:To:Subject:From:In-Reply-To:References; b=LyBJUiby5OQMu4mCVvpczfonSV3KCdsK3mmoiqInUet4QmaIRMh+t/rU6KFMeRkyr YxULNcbC00l+LKXhLb9+hqYdejHtreSkTNKwVsigvDYa9Aqk18EE7GCrs+pnhzSI+k /OrZj4ehcEtYNEHxQCAzvP0lz/zQ2RgpbXRnTWPn0CyHjeuZMOvFWig4CEuoskmWnJ FOxbeBM66tFvQg9dE62R/qhu7i/koG74f9z0amFEKQ/qy7TrY3QHzA033cfkVWm/DU t9lT68H3EjOhKqzyFVbse7Pi/VW4b+K28aVLF4rTNa7UVR4+z2FD7dLBSnypugLlNW aWytHoZauk64w== Received: from localhost (rolling-vm-freebsd5.home.utahime.org [192.168.174.55]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by eastasia.home.utahime.org (Postfix) with ESMTPSA id 92C9121D62; Sat, 8 May 2021 21:00:26 +0900 (JST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.2 at eastasia.home.utahime.org Date: Sat, 08 May 2021 20:59:40 +0900 (JST) Message-Id: <20210508.205940.216790454.yasu@utahime.org> To: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Re: Loading zfs module results in hangup on i386 From: Yasuhiro Kimura In-Reply-To: <20210508.074415.2261369884561178196.yasu@utahime.org> References: <20210507.214759.1825935389016318351.yasu@utahime.org> <20210508.073147.1934590966598603586.yasu@utahime.org> <20210508.074415.2261369884561178196.yasu@utahime.org> X-Mailer: Mew version 6.8 on Emacs 27.2 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4Fcm9M2mxvz4pVM X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=utahime.org header.s=maybe2019112701 header.b=LyBJUiby; dmarc=none; spf=pass (mx1.freebsd.org: domain of yasu@utahime.org designates 183.180.29.210 as permitted sender) smtp.mailfrom=yasu@utahime.org X-Spamd-Result: default: False [-0.70 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+a:spf-authorized.utahime.org]; TO_DN_NONE(0.00)[]; HFILTER_HELO_IP_A(1.00)[maybe.home.utahime.org]; HFILTER_HELO_NORES_A_OR_MX(0.30)[maybe.home.utahime.org]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[utahime.org:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[183.180.29.210:from]; ASN(0.00)[asn:2519, ipnet:183.180.0.0/16, country:JP]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[utahime.org:s=maybe2019112701]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[utahime.org]; SPAMHAUS_ZRD(0.00)[183.180.29.210:from:127.0.2.255]; MID_CONTAINS_FROM(1.00)[]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 May 2021 12:00:41 -0000 From: Yasuhiro Kimura Subject: Re: Loading zfs module results in hangup on i386 Date: Sat, 08 May 2021 07:44:15 +0900 (JST) >> Now I think I know what is the source of problem. After all, on >> 13.0-RELEASE i386 system simply loading zfs module results in system >> hang up. >> >> The steps to reproduce it are, >> >> 1. Boot with install media of 13.0-RELEASE i386 >> 2. At the first menu of FreeBSD installer, select 'Shell'. >> 3. At the shell prompt, type `kldload zfs` and return key. >> >> I confirmed hangup happens with VirtualBox, VMware Player and my bare >> metal PC environement. So the problem doesn't depend on hardware. >> >> And hangup also happens with 13-STABLE and 14-CURRENT. > > This problem is already reported to Bugzilla. > > Bug 254177 When ZFS is recognized, An i386 machine with a lot of memory hangs. > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254177 Referencing the bug report, I applied attached patch to d474440ab33 of main (14-CURRENT). built install image and tried install of ZFS root i386 system with it. Then it completed successfully with 8GB memory. Additionally GENERIC kernel recognizes 8GB of memory. And ZFS root system works fine without any tuning. ---------------------------------------------------------------------- diff --git a/sys/contrib/openzfs/module/zfs/dbuf.c b/sys/contrib/openzfs/module/zfs/dbuf.c index d48dc7943a2..c85500453fb 100644 --- a/sys/contrib/openzfs/module/zfs/dbuf.c +++ b/sys/contrib/openzfs/module/zfs/dbuf.c @@ -796,7 +796,7 @@ dbuf_init(void) * By default, the table will take up * totalmem * sizeof(void*) / 8K (1MB per GB with 8-byte pointers). */ - while (hsize * zfs_arc_average_blocksize < physmem * PAGESIZE) + while (hsize * zfs_arc_average_blocksize < (uint64_t)physmem * PAGESIZE) hsize <<= 1; retry: ---------------------------------------------------------------------- --- Yasuhiro Kimura From owner-freebsd-stable@freebsd.org Sat May 8 14:02:33 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EC495635EC9 for ; Sat, 8 May 2021 14:02:33 +0000 (UTC) (envelope-from rleigh@codelibre.net) Received: from b-painless.mh.aa.net.uk (b-painless.mh.aa.net.uk [IPv6:2001:8b0:0:30::52]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fcpsx17vzz4vCv for ; Sat, 8 May 2021 14:02:29 +0000 (UTC) (envelope-from rleigh@codelibre.net) Received: from 8.2.0.7.b.a.e.f.f.f.7.a.a.e.a.3.d.b.d.d.0.6.8.0.0.b.8.0.1.0.0.2.ip6.arpa ([2001:8b0:860:ddbd:3aea:a7ff:feab:7028] helo=smtpclient.apple) by painless-b.tch.aa.net.uk with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lfNXI-0002e2-Cq for freebsd-stable@freebsd.org; Sat, 08 May 2021 15:02:28 +0100 From: Roger Leigh Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.80.0.2.43\)) Subject: FreeBSD 13 console stops working under VMware Message-Id: <89FD974C-F8A0-489B-B325-C8AABF919C02@codelibre.net> Date: Sat, 8 May 2021 15:02:18 +0100 To: FreeBSD-STABLE Mailing List X-Mailer: Apple Mail (2.3654.80.0.2.43) X-Rspamd-Queue-Id: 4Fcpsx17vzz4vCv X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of rleigh@codelibre.net has no SPF policy when checking 2001:8b0:0:30::52) smtp.mailfrom=rleigh@codelibre.net X-Spamd-Result: default: False [-1.60 / 15.00]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:8b0:0:30::52:from]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[codelibre.net]; ARC_NA(0.00)[]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2001:8b0:0:30::52:from:127.0.2.255]; NEURAL_HAM_MEDIUM(-1.00)[-0.998]; TO_DN_ALL(0.00)[]; BLOCKLISTDE_FAIL(0.00)[2001:8b0:0:30::52:server fail,2001:8b0:860:ddbd:3aea:a7ff:feab:7028:server fail]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:20712, ipnet:2001:8b0::/32, country:GB]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 May 2021 14:02:34 -0000 Hi, This might sound like a bit of an odd one, but I=E2=80=99ll try to = describe it. When I run a FreeBSD 13-RELEASE virtual machine under = VMware, it appears to work correctly, but randomly stops working. If I focus the VMware window, and press Ctrl-G to grab input focus (or = click in the window), I can log into the system using the console. = However, if I press Ctrl-Alt to ungrab the input focus, or click outside = the window, the block cursor on the console vanishes, and it=E2=80=99s = no longer possible to type any input. However=E2=80=A6 if I grab focus again, I can use Alt-Fn to switch to a = different virtual console, log in again and everything is fine=E2=80=A6 = up until I switch focus to something else and the block cursor vanishes = in that virtual console. Repeat until you run out of virtual consoles! I can=E2=80=99t reproduce this with FreeBSD 12. It seems to only happen = with FreeBSD 13. I=E2=80=99ve had it happen reproducibly when losing = focus, but then again sometimes I=E2=80=99ve had a few minutes where it = doesn=E2=80=99t happen, until it starts occurring again. While it seems = that losing focus is the trigger, there might be something else going = on. Has anyone else noticed this or have any suggested workarounds? Kind regards, Roger= From owner-freebsd-stable@freebsd.org Sat May 8 14:20:09 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CABC6636759 for ; Sat, 8 May 2021 14:20:09 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FcqGK5LZcz4vXQ; Sat, 8 May 2021 14:20:09 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "tensor.andric.com", Issuer "R3" (verified OK)) (Authenticated sender: dim) by smtp.freebsd.org (Postfix) with ESMTPSA id 9055E214EA; Sat, 8 May 2021 14:20:09 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtpclient.apple (unknown [IPv6:2001:470:7a58:0:1dca:bba0:f51c:b110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id E90F133B9B; Sat, 8 May 2021 16:20:06 +0200 (CEST) From: Dimitry Andric Message-Id: <0F510E0A-32F9-464E-AAF4-E9F056F17E05@FreeBSD.org> Content-Type: multipart/signed; boundary="Apple-Mail=_41940078-6B8A-4C03-8EDF-3B4F0B41875D"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.80.0.2.43\)) Subject: Re: FreeBSD 13 console stops working under VMware Date: Sat, 8 May 2021 16:20:02 +0200 In-Reply-To: <89FD974C-F8A0-489B-B325-C8AABF919C02@codelibre.net> Cc: FreeBSD-STABLE Mailing List To: Roger Leigh References: <89FD974C-F8A0-489B-B325-C8AABF919C02@codelibre.net> X-Mailer: Apple Mail (2.3654.80.0.2.43) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 May 2021 14:20:09 -0000 --Apple-Mail=_41940078-6B8A-4C03-8EDF-3B4F0B41875D Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 On 8 May 2021, at 16:02, Roger Leigh wrote: >=20 > This might sound like a bit of an odd one, but I=E2=80=99ll try to = describe it. When I run a FreeBSD 13-RELEASE virtual machine under = VMware, it appears to work correctly, but randomly stops working. >=20 > If I focus the VMware window, and press Ctrl-G to grab input focus (or = click in the window), I can log into the system using the console. = However, if I press Ctrl-Alt to ungrab the input focus, or click outside = the window, the block cursor on the console vanishes, and it=E2=80=99s = no longer possible to type any input. >=20 > However=E2=80=A6 if I grab focus again, I can use Alt-Fn to switch to = a different virtual console, log in again and everything is fine=E2=80=A6 = up until I switch focus to something else and the block cursor vanishes = in that virtual console. Repeat until you run out of virtual consoles! >=20 > I can=E2=80=99t reproduce this with FreeBSD 12. It seems to only = happen with FreeBSD 13. I=E2=80=99ve had it happen reproducibly when = losing focus, but then again sometimes I=E2=80=99ve had a few minutes = where it doesn=E2=80=99t happen, until it starts occurring again. While = it seems that losing focus is the trigger, there might be something else = going on. >=20 > Has anyone else noticed this or have any suggested workarounds? Press the Scroll Lock key to 'fix' it, if that is possible for you. This = is some weird interaction between VMware's input focus grabbing method = and our console, which sometimes turns on Scroll Lock accidentally. I = have not been able to put my finger on when it happens exactly, but it = does happen often. For me, it usually occurs when I use Microsoft Remote Desktop to access = a Windows machine running VMware, and switch back and forth between = Remote desktop and another application. Something about losing the focus = is making the VMware GUI inject a Scroll Lock event. It's pretty tricky = to generate Scroll Lock via Remote Desktop though, especially from a = Mac, which doesn't have that key at all. :) -Dimitry PS: Note that Scroll Lock is normally used in FreeBSD's console to = scroll back in the virtual consoles, as opposed to Linux's shift-PageUp = and shift-PageDown. But it is a toggle, not a one-off key. --Apple-Mail=_41940078-6B8A-4C03-8EDF-3B4F0B41875D Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.2 iF0EARECAB0WIQR6tGLSzjX8bUI5T82wXqMKLiCWowUCYJaeEgAKCRCwXqMKLiCW o17tAJsHwFMJcVAkelm7KAwHKGAoS/n/wwCfbqvjcedDCIwGhi91/xh/4o+6Oyk= =EtOM -----END PGP SIGNATURE----- --Apple-Mail=_41940078-6B8A-4C03-8EDF-3B4F0B41875D-- From owner-freebsd-stable@freebsd.org Sat May 8 15:13:45 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DEF0F637858 for ; Sat, 8 May 2021 15:13:45 +0000 (UTC) (envelope-from rleigh@codelibre.net) Received: from authenticated.a-painless.mh.aa.net.uk (painless-a.thn.aa.net.uk [IPv6:2001:8b0:0:62::26]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FcrS961zRz3Dcx; Sat, 8 May 2021 15:13:45 +0000 (UTC) (envelope-from rleigh@codelibre.net) Received: from b.b.5.c.8.b.7.6.9.9.4.6.7.c.5.2.d.b.d.d.0.6.8.0.0.b.8.0.1.0.0.2.ip6.arpa ([2001:8b0:860:ddbd:25c7:6499:67b8:c5bb]) by painless-a.thn.aa.net.uk with esmtpsa (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1lfOeG-0003Oj-Md; Sat, 08 May 2021 16:13:44 +0100 Subject: Re: FreeBSD 13 console stops working under VMware To: Dimitry Andric Cc: FreeBSD-STABLE Mailing List References: <89FD974C-F8A0-489B-B325-C8AABF919C02@codelibre.net> <0F510E0A-32F9-464E-AAF4-E9F056F17E05@FreeBSD.org> From: Roger Leigh Message-ID: Date: Sat, 8 May 2021 16:13:32 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.0 MIME-Version: 1.0 In-Reply-To: <0F510E0A-32F9-464E-AAF4-E9F056F17E05@FreeBSD.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB X-Rspamd-Queue-Id: 4FcrS961zRz3Dcx X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 May 2021 15:13:45 -0000 On 08/05/2021 15:20, Dimitry Andric wrote: > On 8 May 2021, at 16:02, Roger Leigh wrote: >> This might sound like a bit of an odd one, but I’ll try to describe it. When I run a FreeBSD 13-RELEASE virtual machine under VMware, it appears to work correctly, but randomly stops working. >> >> If I focus the VMware window, and press Ctrl-G to grab input focus (or click in the window), I can log into the system using the console. However, if I press Ctrl-Alt to ungrab the input focus, or click outside the window, the block cursor on the console vanishes, and it’s no longer possible to type any input. >> >> However… if I grab focus again, I can use Alt-Fn to switch to a different virtual console, log in again and everything is fine… up until I switch focus to something else and the block cursor vanishes in that virtual console. Repeat until you run out of virtual consoles! >> >> I can’t reproduce this with FreeBSD 12. It seems to only happen with FreeBSD 13. I’ve had it happen reproducibly when losing focus, but then again sometimes I’ve had a few minutes where it doesn’t happen, until it starts occurring again. While it seems that losing focus is the trigger, there might be something else going on. >> >> Has anyone else noticed this or have any suggested workarounds? > Press the Scroll Lock key to 'fix' it, if that is possible for you. This is some weird interaction between VMware's input focus grabbing method and our console, which sometimes turns on Scroll Lock accidentally. I have not been able to put my finger on when it happens exactly, but it does happen often. > > For me, it usually occurs when I use Microsoft Remote Desktop to access a Windows machine running VMware, and switch back and forth between Remote desktop and another application. Something about losing the focus is making the VMware GUI inject a Scroll Lock event. It's pretty tricky to generate Scroll Lock via Remote Desktop though, especially from a Mac, which doesn't have that key at all. :) > > -Dimitry > > PS: Note that Scroll Lock is normally used in FreeBSD's console to scroll back in the virtual consoles, as opposed to Linux's shift-PageUp and shift-PageDown. But it is a toggle, not a one-off key. Thanks Dimitry, that certainly makes some sort of sense!  I am indeed connecting from a Mac to a beefier Windows 10 PC running VMware workstation using Remote Desktop.  Going back to one of the "broken" consoles, I can indeed use PgUp/PgDn to scroll up and down, so it certainly appears as though a Scroll Lock keypress was sent or triggered somehow.  While I do have a regular PC keyboard hooked up, I don't find myself able to send that key event through the Remote Desktop session.  However, if I physically log into the Windows PC, I can unstick each console with the physical Scroll Lock key, so it seems clear that (somehow) Scroll Lock was triggered in the first place to cause the problem. I have tried to trigger various combinations of grab/ungrab/switch to window inside or outside of the Remote Desktop session, but I've not been able to pinpoint the specific trigger. Kind regards, Roger From owner-freebsd-stable@freebsd.org Sat May 8 17:45:00 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0C98A63ADAE for ; Sat, 8 May 2021 17:45:00 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fcvpf5JnLz3LyT for ; Sat, 8 May 2021 17:44:58 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 148HioRT095479 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sat, 8 May 2021 20:44:53 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 148HioRT095479 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 148HioEv095478; Sat, 8 May 2021 20:44:50 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 8 May 2021 20:44:50 +0300 From: Konstantin Belousov To: Eugene Grosbein Cc: FreeBSD Stable Subject: Re: Install of 13.0-RELEASE i386 with ZFS root hangs up Message-ID: References: <20210507.214759.1825935389016318351.yasu@utahime.org> <4bee4d1d-46a7-e0d1-48d9-c3e964a05ef0@grosbein.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4bee4d1d-46a7-e0d1-48d9-c3e964a05ef0@grosbein.net> X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4Fcvpf5JnLz3LyT X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com X-Spamd-Result: default: False [-1.40 / 15.00]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:470:d5e7:1::1:from]; FREEMAIL_FROM(0.00)[gmail.com]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_SPF_SOFTFAIL(0.00)[~all:c]; NEURAL_SPAM_SHORT(0.60)[0.599]; SPAMHAUS_ZRD(0.00)[2001:470:d5e7:1::1:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 May 2021 17:45:00 -0000 On Sat, May 08, 2021 at 06:33:02PM +0700, Eugene Grosbein wrote: > 08.05.2021 2:52, Konstantin Belousov wrote: > > > i386 kernel uses memory up to 24G since 13.0. > > > > PAE only means that devices that can access full 64bit address are allowed > > to avoid dma bouncing. > > Maybe you could tell something on similar topic? > > There is FreeBSD 12.2-STABLE r369567 Base12 amd64 running > with Intel Atom CPU capable of long mode and addressing 8GB RAM, > ASRock A330ION motherboard and two memory modules installed: 4G+2GB. > Why so small "avail memory"? > > FreeBSD clang version 10.0.1 (git@github.com:llvm/llvm-project.git llvmorg-10.0.1-0-gef32c611aa2) > CPU: Intel(R) Atom(TM) CPU 330 @ 1.60GHz (1600.03-MHz K8-class CPU) > Origin="GenuineIntel" Id=0x106c2 Family=0x6 Model=0x1c Stepping=2 > Features=0xbfe9fbff > Features2=0x40e31d > AMD Features=0x20000800 > AMD Features2=0x1 > TSC: P-state invariant, performance statistics > real memory = 6442450944 (6144 MB) > Physical memory chunk(s): > 0x0000000000010000 - 0x000000000009dfff, 581632 bytes (142 pages) > 0x0000000000103000 - 0x00000000001fffff, 1036288 bytes (253 pages) > 0x0000000002b00000 - 0x00000000d8709fff, 3586170880 bytes (875530 pages) > avail memory = 3571384320 (3405 MB) > > Also http://www.grosbein.net/freebsd/dmidecode.txt Some necromancy revealed that this CPU did not have memory controller on-chip, it was a design from the 2008 where MCH handled memory. It is up to the chipset and BIOS to configure and report the memory above 4G to OS. As you clearly see from the SMAP printed above, BIOS does not report anything above 4G. Might be, look at bios settings. No other ideas.