From nobody Mon Jan 8 00:15:47 2024 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T7ZND4PXjz56N8d; Mon, 8 Jan 2024 00:15:56 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [192.147.25.65]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "*.lerctr.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4T7ZND27c7z4cHJ; Mon, 8 Jan 2024 00:15:56 +0000 (UTC) (envelope-from ler@lerctr.org) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=ler2019; h=Content-Transfer-Encoding:Content-Type:Message-ID:References: In-Reply-To:Subject:Cc:To:From:Date:MIME-Version:Sender:Reply-To:Content-ID: Content-Description; bh=g7z0hYleGgP4Nu4An7VBd3Wr1PnrWOWntywRHGB3OFo=; b=myI5H mSmpdQIWt4reJPJ5eVYJU661LQ7RGjz910HiU1sgHBVVoVAe1OSLjSD8jJwwGi0/CyxEI9EM9wzaA PuWU++4eGNq8fUWx8UyKtcg0NDRMk2mTZT8VMhbkIfas/Z7i5TEofgW6sD3Nr+N++QrBZj8JFH080 k5vFhIT6Nik4+CYiN8o7iBb13Py0n3mCeQLwvdyGPxt05NmmByJItLjg1TYDTTtgvzUq2KHB7uAc4 p8OcRWMA820WYeru14a1ozmmpSADfIRqRw7svWmbaqnGhg8bN3yu7VA72LsOa5N4gH7NlkX6S9cSx 3zXuAPlqGfwEUw9ztW72nigJagGFA==; Received-SPF: pass (thebighonker.lerctr.org: domain of lerctr.org designates 192.147.25.65 as permitted sender) client-ip=192.147.25.65; envelope-from=ler@lerctr.org; helo=webmail.lerctr.org; Received: from thebighonker.lerctr.org ([192.147.25.65]:39661 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.97.1 (FreeBSD)) (envelope-from ) id 1rMdIx-00000000BsB-1MKe; Sun, 07 Jan 2024 18:15:47 -0600 Received: from 99-190-128-217.lightspeed.austtx.sbcglobal.net ([99.190.128.217]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Sun, 07 Jan 2024 18:15:47 -0600 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Date: Sun, 07 Jan 2024 18:15:47 -0600 From: Larry Rosenman To: lev@freebsd.org Cc: Warner Losh , freebsd-fs , freebsd-stable Subject: Re: FreeBSD 13.2-STABLE can not boot from damaged mirror AND pool stuck in "resilver" state even without new devices. In-Reply-To: <962b242d-546f-46ce-9eb2-9bd2a10f4608@FreeBSD.org> References: <065f4f5c-f38b-45f4-b7e7-5248f871f7e6@FreeBSD.org> <2f91eeb7-430b-49e2-817b-5acd0f445fe9@FreeBSD.org> <962b242d-546f-46ce-9eb2-9bd2a10f4608@FreeBSD.org> Message-ID: <30315c170f7146a5e1a05e4a2eff3d1b@lerctr.org> X-Sender: ler@lerctr.org Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4T7ZND27c7z4cHJ X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:18474, ipnet:192.147.25.0/24, country:US] On 01/07/2024 3:19 pm, Lev Serebryakov wrote: > On 07.01.2024 22:15, Warner Losh wrote: > >> So, if this is a mirror, then ada0 blank and ada1 with good data, in >> theory >> you should be fine. However, perhaps ZFS is finding that there's an >> error from >> ada1 for real. Does all of ada1 read with a simple dd? > Yep, it is read with dd, I've checked it > >> Not sure about the losing devices you described later on. >> >> > ZFS: i/o error - all block copies unavailable >> > ZFS: can't read MOS of pool zroot >> > >> > >> >   To be honest, I thinks there is something else. Because >> sequence of events were (sorry, too long, but I think, tht every >> detail matters here): >> >> >> Yea. There's something that's failing, which zio_read is woefully >> under reporting for our diagnostic efforts. And/or something is >> getting confused by the blank disk and/or the partially resilvered >> disk. > > My theory, that something is confused when one disk is 512/4096 and > other is 512/512. > > I want to check it on VM, but can not find VM that both (1) allows > CMS boot and (2) allows to configure logical and physical sector of > virtual HDD. > > bhyve could configure sector sizes, but doesn't support BIOS, and > VBox and qemu-system can not emulate sector sizes (or I can not google > proper configuration). When I first saw this, I wonder what ashift is set to on the pool? -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106 From nobody Mon Jan 8 11:15:21 2024 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T7s174xCtz55PPc; Mon, 8 Jan 2024 11:15:23 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4T7s174RCkz50r4; Mon, 8 Jan 2024 11:15:23 +0000 (UTC) (envelope-from lev@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1704712523; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=K8zAIJCc5HIP1p9sowRu93OYK8OXP/FYEhTQxRIsymQ=; b=HBJ3LVu9sJsbLe4y7n66xSsP3LfuPJxkXsy7stS281ceJrZcJjbBCW1mRmxcFiSNh8Z8t7 RbVZ8p/GRjrCsbuqmM6z+7FJXCo6KmL6ePL/exf8rrnRhDCp30fJLfl32vDhLLdPoxGyZ7 qKF+PM7tiADRvYWT5G1BwQ6ZsadOn7LgdwOd+EnlmImLFfYAl6Sx1XpbIlxwtdlCuBA9P7 QSfWkKe5KP4VjLy6lR6PIbm0A9cuj5cyhnhwZhCbLo6PmACD4vYBRHKmHR9dfXJgOKGT9+ 6ItP14Yg7jl3Eb3hyQasDqQAEfEL1qFsHi1YQqnEqtcZPnl6gzxgTKA6RPR+sg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1704712523; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=K8zAIJCc5HIP1p9sowRu93OYK8OXP/FYEhTQxRIsymQ=; b=KzoVsmr2f+QKrAItmhcfVsZqpny6DL6CyaKjPyTSjfiOCTZVCPFX0Ydr0cq/6eZfn9M/rY aMPyyobTrg9S6IPPmJSCny3Xxvv0HedRvzclOJI5wj3KxEXTnv7FuG8YGTjBnYPfq9BEo+ yDVPbYpKzZw/k2GCNXfv4muMc7S/d4RebCIDXMxjVNa0zV0PRyWIIhmveEgFKkFZhIBtEk ESCpoTFcDwCWvvW9g9tDdBnfbxwjugquTNujIdCkMmaKoa26Emrf0DuGBnQcAZmoIpbvfW 0iVavhaZFYhbEaXFX/JzZN7xfbyZVrEoEVogBpcS7Rh3xdrwG15q+MpVtMOKPw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1704712523; a=rsa-sha256; cv=none; b=LY7uSc4tSswc0vfvcXImkihRJHAnyZ8tjOBg/cJUsoK8Nj1IM0PzROMcaSq/okD3plgSVj YnR/96qXHSuhvWnjLE4R+l4vEvYX/bdP2Z0yOB0lpcnUXbXkETQeqSNTFtOALVE5rFsq94 RJcJfbplQveTbK5HsR27USKUMOXWNuIlPP6HWqm4rymRk+wMvIbNEo0SMhI9JSFMNju6G6 IMEEro4qd7jfMxydwWrcqAjRnpEuVe1QDXawVR25CRgWhxuFVvO0VIb7vBTbqE4nKchpxm mzHRqrfFOwnDUkOtKarX/V1F5e+OMuTN60DSM1UbLzexijNq8PayWCJI8ci0QA== Received: from onlyone.not-for.work (onlyone.not-for.work [148.251.9.81]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: lev/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4T7s172pgPz16gj; Mon, 8 Jan 2024 11:15:23 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from [192.168.136.24] (83-84-181-95.cable.dynamic.v4.ziggo.nl [83.84.181.95]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.not-for.work (Postfix) with ESMTPSA id 23D013C2; Mon, 8 Jan 2024 14:15:21 +0300 (MSK) Message-ID: Date: Mon, 8 Jan 2024 12:15:21 +0100 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Reply-To: lev@FreeBSD.org Subject: Re: FreeBSD 13.2-STABLE can not boot from damaged mirror AND pool stuck in "resilver" state even without new devices. To: Larry Rosenman Cc: freebsd-fs , freebsd-stable References: <065f4f5c-f38b-45f4-b7e7-5248f871f7e6@FreeBSD.org> <2f91eeb7-430b-49e2-817b-5acd0f445fe9@FreeBSD.org> <962b242d-546f-46ce-9eb2-9bd2a10f4608@FreeBSD.org> <30315c170f7146a5e1a05e4a2eff3d1b@lerctr.org> Content-Language: en-US From: Lev Serebryakov Organization: FreeBSD In-Reply-To: <30315c170f7146a5e1a05e4a2eff3d1b@lerctr.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 08.01.2024 1:15, Larry Rosenman wrote: >>> So, if this is a mirror, then ada0 blank and ada1 with good data, in theory >>> you should be fine. However, perhaps ZFS is finding that there's an error from >>> ada1 for real. Does all of ada1 read with a simple dd? >>   Yep, it is read with dd, I've checked it >> >>> Not sure about the losing devices you described later on. >>> >>>      > ZFS: i/o error - all block copies unavailable >>>      > ZFS: can't read MOS of pool zroot >>>      > >>>      > >>>      >   To be honest, I thinks there is something else. Because sequence of events were (sorry, too long, but I think, tht every detail matters here): >>> >>> >>> Yea. There's something that's failing, which zio_read is woefully under reporting for our diagnostic efforts. And/or something is >>> getting confused by the blank disk and/or the partially resilvered disk. >> >>   My theory, that something is confused when one disk is 512/4096 and other is 512/512. >> >>   I want to check it on VM, but can not find VM that both (1) allows CMS boot and (2) allows to configure logical and physical sector of virtual HDD. >> >>   bhyve could configure sector sizes, but doesn't support BIOS, and VBox and qemu-system can not emulate sector sizes (or I can not google proper configuration). > > When I first saw this, I wonder what ashift is set to on the pool? old pool was with ashift=9, but new one is with ashift=12. -- // Lev Serebryakov From nobody Mon Jan 8 15:44:12 2024 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T7yzL6yMrz567Nm; Mon, 8 Jan 2024 15:44:14 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [192.147.25.65]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "*.lerctr.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4T7yzL59kSz4RbD; Mon, 8 Jan 2024 15:44:14 +0000 (UTC) (envelope-from ler@lerctr.org) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=ler2019; h=Content-Transfer-Encoding:Content-Type:Message-ID:References: In-Reply-To:Subject:Cc:To:From:Date:MIME-Version:Sender:Reply-To:Content-ID: Content-Description; bh=42LSM3esW806EpSEuqGI3fc26vgwVHNjRaPF4pY8wBI=; b=RFuu8 YglIFqFGhnDPYtAdEkzH7+LI+yur55mevDwgJ9RTrJEjz1eqr58yL/im6W7nhn+jnL2TC8txBHyoF 0BpKceEGjdbL9M8hv/MHm9zAlwp9gcbIcVzdCpZCtCrdP3VXc2LqodqVnZYKxQPBCQArEgFoSLYI9 OcFr4cKF1B8+7Gn169M92zZ+hPSzI3F741jPRzG+a3vaBj0ON+rfP07HI/Kc7d4LSInFRvvbxKmsL 4aqAhVGYP4FKKT+E4q8fg1hPvwRufc0jrPxoL87Qt4APkRU6yxlK/OpIK21/bMPJKLc469KQVUzts YC0iiMJ2GuxZ8VAT7aWl9sXwwr2Ug==; Received-SPF: pass (thebighonker.lerctr.org: domain of lerctr.org designates 192.147.25.65 as permitted sender) client-ip=192.147.25.65; envelope-from=ler@lerctr.org; helo=webmail.lerctr.org; Received: from thebighonker.lerctr.org ([192.147.25.65]:23548 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.97.1 (FreeBSD)) (envelope-from ) id 1rMrnQ-00000000MYb-1IUd; Mon, 08 Jan 2024 09:44:12 -0600 Received: from 99-190-128-217.lightspeed.austtx.sbcglobal.net ([99.190.128.217]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Mon, 08 Jan 2024 09:44:12 -0600 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Date: Mon, 08 Jan 2024 09:44:12 -0600 From: Larry Rosenman To: lev@freebsd.org Cc: freebsd-fs , freebsd-stable Subject: Re: FreeBSD 13.2-STABLE can not boot from damaged mirror AND pool stuck in "resilver" state even without new devices. In-Reply-To: References: <065f4f5c-f38b-45f4-b7e7-5248f871f7e6@FreeBSD.org> <2f91eeb7-430b-49e2-817b-5acd0f445fe9@FreeBSD.org> <962b242d-546f-46ce-9eb2-9bd2a10f4608@FreeBSD.org> <30315c170f7146a5e1a05e4a2eff3d1b@lerctr.org> Message-ID: <262428cf89227e192953e4540875bc41@lerctr.org> X-Sender: ler@lerctr.org Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4T7yzL59kSz4RbD X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:18474, ipnet:192.147.25.0/24, country:US] On 01/08/2024 5:15 am, Lev Serebryakov wrote: > On 08.01.2024 1:15, Larry Rosenman wrote: > >>>> So, if this is a mirror, then ada0 blank and ada1 with good data, in >>>> theory >>>> you should be fine. However, perhaps ZFS is finding that there's an >>>> error from >>>> ada1 for real. Does all of ada1 read with a simple dd? >>>   Yep, it is read with dd, I've checked it >>> >>>> Not sure about the losing devices you described later on. >>>> >>>>      > ZFS: i/o error - all block copies unavailable >>>>      > ZFS: can't read MOS of pool zroot >>>>      > >>>>      > >>>>      >   To be honest, I thinks there is something else. Because >>>> sequence of events were (sorry, too long, but I think, tht every >>>> detail matters here): >>>> >>>> >>>> Yea. There's something that's failing, which zio_read is woefully >>>> under reporting for our diagnostic efforts. And/or something is >>>> getting confused by the blank disk and/or the partially resilvered >>>> disk. >>> >>>   My theory, that something is confused when one disk is 512/4096 and >>> other is 512/512. >>> >>>   I want to check it on VM, but can not find VM that both (1) allows >>> CMS boot and (2) allows to configure logical and physical sector of >>> virtual HDD. >>> >>>   bhyve could configure sector sizes, but doesn't support BIOS, and >>> VBox and qemu-system can not emulate sector sizes (or I can not >>> google proper configuration). >> >> When I first saw this, I wonder what ashift is set to on the pool? > old pool was with ashift=9, but new one is with ashift=12. I wonder if the ashif=9 caused the issue when you added the 4Kn disk? -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106 From nobody Mon Jan 8 17:27:03 2024 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T81G42bwHz56BP1; Mon, 8 Jan 2024 17:27:08 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4T81G427xgz4mDN; Mon, 8 Jan 2024 17:27:08 +0000 (UTC) (envelope-from lev@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1704734828; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kXL8y2YDr559KJJb+uYLCbBer3qc5RPGihp5jvNGx54=; b=mO+8D1KxBrqf8iRX6S3oixqfjugTDbaf2I+uDjczRWsMBxkTyFtA6I/lh0B0YUyDWAW414 d8yZyghw4Uq82jU7cDBRHw3x3qTxaU2Z9Lg9WhaYTcxUhOpPJmDaDosMbAnugY5V+zBORK vpG8ta0AJh/SNDBQm4P+/TMP21mj1GPiFzKbVX6s0loHkad540Q3ouTVVwB+RoMNEJgVZ9 pFkmEuA5Y7uLX6ElvPPdDCtY87EFGu4mMnWyxLM+vZwpiNCFRRrq/wK5PGCz5IIwohfCia i+tXTmHZKM8X2Dbwd8UBm+1Zm+RVyrvwqhjzCBUC7ddWFfbDlPOa0Vb+fDx0GQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1704734828; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kXL8y2YDr559KJJb+uYLCbBer3qc5RPGihp5jvNGx54=; b=cPJg3l+oARnmrPJT7uYgtEbA7hAvFrLf7pgDvpz8zLrJN0626/D56bq1IMYQ//w0ffRsDn u5p7S7JO9IV0ORV9gYkBd66+zQKO38cXHy0J8DHfUj7TspkeFmaYN8guX5Jph+Cljk+Mv0 Uikg+ajD/r9hK9X1c3snZt4N8aHm32fjA7c8/4HUIkLtzRXZbgiLRhSSs2hBhS0PFWWWmO DNnK1uAFSE4vC3W9PHXvVGVxQuImGHCYeX9TglW8ULIhPDOkshOL8UexBzYf3J/lBaRaJx 8bKD40c86NWG73g3bqJpDcMqYmWpN+yVpn9tA6bNysC2RFAPe3q8lc2vExtjLg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1704734828; a=rsa-sha256; cv=none; b=a9JQL7gU80AiEZbvkvOz6hXFN4Tjcv6inoGOF7739VomC3mKFcOgycYnACS0fdpx9ATUN/ R/9VREI9rtm5V2RVEW1mnw9Va5JBEcZu7LPsdhiuIqj6YudJSezmfqg4wfPlmgXlf91Zr/ JIQN+xVggaeRc76FAab8sWPZvYeROX9/126Ugpnqtgn3ip85+zADeKRTyffkViQ5kkEWJf ynYpxxIFbq+unM1Lnf/9pTHKGw5z6AqnsWEbgPtOKxcYpPc/VGiQwEktCiLPjKUpjFU6dw L5socr6paLq3RuapUbbn9r8zar/CW/Qg/jUygmIgqL14qf3f5e2COGCpvBSjYg== Received: from onlyone.not-for.work (onlyone.not-for.work [148.251.9.81]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: lev/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4T81G40SbNz1DRc; Mon, 8 Jan 2024 17:27:08 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from [IPV6:2001:470:923f:1:8967:d924:426d:40bd] (unknown [IPv6:2001:470:923f:1:8967:d924:426d:40bd]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.not-for.work (Postfix) with ESMTPSA id 3D919362; Mon, 8 Jan 2024 20:27:05 +0300 (MSK) Message-ID: Date: Mon, 8 Jan 2024 18:27:03 +0100 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Reply-To: lev@FreeBSD.org Subject: Re: FreeBSD 13.2-STABLE can not boot from damaged mirror AND pool stuck in "resilver" state even without new devices. Content-Language: en-US To: Larry Rosenman Cc: freebsd-fs , freebsd-stable References: <065f4f5c-f38b-45f4-b7e7-5248f871f7e6@FreeBSD.org> <2f91eeb7-430b-49e2-817b-5acd0f445fe9@FreeBSD.org> <962b242d-546f-46ce-9eb2-9bd2a10f4608@FreeBSD.org> <30315c170f7146a5e1a05e4a2eff3d1b@lerctr.org> <262428cf89227e192953e4540875bc41@lerctr.org> From: Lev Serebryakov Organization: FreeBSD In-Reply-To: <262428cf89227e192953e4540875bc41@lerctr.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 08.01.2024 16:44, Larry Rosenman wrote: >>>>> So, if this is a mirror, then ada0 blank and ada1 with good data, in theory >>>>> you should be fine. However, perhaps ZFS is finding that there's an error from >>>>> ada1 for real. Does all of ada1 read with a simple dd? >>>>   Yep, it is read with dd, I've checked it >>>> >>>>> Not sure about the losing devices you described later on. >>>>> >>>>>      > ZFS: i/o error - all block copies unavailable >>>>>      > ZFS: can't read MOS of pool zroot >>>>>      > >>>>>      > >>>>>      >   To be honest, I thinks there is something else. Because sequence of events were (sorry, too long, but I think, tht every detail matters here): >>>>> >>>>> >>>>> Yea. There's something that's failing, which zio_read is woefully under reporting for our diagnostic efforts. And/or something is >>>>> getting confused by the blank disk and/or the partially resilvered disk. >>>> >>>>   My theory, that something is confused when one disk is 512/4096 and other is 512/512. >>>> >>>>   I want to check it on VM, but can not find VM that both (1) allows CMS boot and (2) allows to configure logical and physical sector of virtual HDD. >>>> >>>>   bhyve could configure sector sizes, but doesn't support BIOS, and VBox and qemu-system can not emulate sector sizes (or I can not google proper configuration). >>> >>> When I first saw this, I wonder what ashift is set to on the pool? >>   old pool was with ashift=9, but new one is with ashift=12. > I wonder if the ashif=9 caused the issue when you added the 4Kn disk? I'm looking for ways to check this hypothesis without additional hardware. -- // Lev Serebryakov From nobody Mon Jan 8 17:34:24 2024 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T81QY32q5z56C5c; Mon, 8 Jan 2024 17:34:29 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4T81QY2T66z4qJJ; Mon, 8 Jan 2024 17:34:29 +0000 (UTC) (envelope-from lev@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1704735269; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=stKjwY5hCj/5z9uzkUoFFa7QNkd9PKG4DjkD/pfxHtE=; b=wiPD1kv/YVN+exQY+QdIBgZrI/MQTpAUY4RLxjqqDu1ikMNzDVFAiouLCTX3qcv+oR6CPQ Tu5BPBfb0cnTvLMPSfX3iwhUt1J56Y1zv/kmLHyrGhsD22N1qO7zeTq586DBr+L8AzEVEX ZzjC9v9yTF23mN00qqtcDyyqJJpd+0Qa/2qyK31GDuKdUm8VGB0F3U4Vy2SV6KKAwdxBKm zx23tJmHr6hu9KWRpl8+w+vGzEtHxXvmMX5nnz0qo+m+BVF3JbW9W3EjZejAFiCFP+gYWB KeHlTpJnioiXPH0faSgDbujc5R0XvhtMgBvW29QYBepdYNt0na5II3SmZ1dlNA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1704735269; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=stKjwY5hCj/5z9uzkUoFFa7QNkd9PKG4DjkD/pfxHtE=; b=UPYuVCXRK0F7mN/fl5b1Kui7k9NbZRWm+6miH2J+CQzwQ2YGEsSA9oLUaiveRSYwTIcJEa R9viFWSgH5/Ep5H//oF+hqGU9C+VM7PNQLMHmAf2POHk8Y8pB/7is3c9kuTZCgDHj5lzvV 5XfedF0qvSlnmMbk8EVYa6nSR6frGXfzya2zJ3DVU5zJnVuVo4G+pOc7mkPeCJfmUYJJZ9 T8ljJLMH47ehMg9lFb8jSBEQG9OM7xR8eCtzaSyq8WEK3Egi0yO/DXqhS//PPMzkFO/gDc OgQS6iKm/wzdCAWpByADZV8nzMSY5Oav1JiDssaHxSP1/Sx8ae8XzhsinzgGTQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1704735269; a=rsa-sha256; cv=none; b=Seo8xXhxgsYwgMFTenFVYtolANheajFoTQ8HX7noW/F7vEyGk60QUr274hjXSYGCJpWW5w jzcajEB/72PMZj1HUjFXnnR3FInbvucsILsw51jdcerDWRToVSYttdXMXrBmMA+OObAn3B EMp5NvcTErnNEVeDMJ5MsMryrL5mhfFCbawdOnJEs1v6dEla8R26+kkgfMRGq0WRPQrllF GtjJPDXXYQDBRk/Se9RaLcRHAJHyUYd2IGxkqKgMCos+lEfhx8UNuR4GvqTAQrQ0uOQXNf VXM1zjDuWn+FDI/y6jxmWijKQSU0NSwOALKyRIrmWheqFhFyORSUm2qxgvzRVQ== Received: from onlyone.not-for.work (onlyone.not-for.work [IPv6:2a01:4f8:201:6350::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: lev/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4T81QY0nMDz1Dh1; Mon, 8 Jan 2024 17:34:29 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from [IPV6:2001:470:923f:1:8967:d924:426d:40bd] (unknown [IPv6:2001:470:923f:1:8967:d924:426d:40bd]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.not-for.work (Postfix) with ESMTPSA id DF9442BC; Mon, 8 Jan 2024 20:34:24 +0300 (MSK) Message-ID: <22167ed4-887d-4fe0-b0a6-36312ae38fea@FreeBSD.org> Date: Mon, 8 Jan 2024 18:34:24 +0100 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: freebsd-fs , freebsd-stable Content-Language: en-US From: Lev Serebryakov Reply-To: lev@FreeBSD.org Subject: Crash on adding L2ARC to raidz1 pool Organization: FreeBSD Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hello! System: FreeBSD 13.2-STABLE stable/13-n256849-05c55eed44e5 BLOB amd64 I'm still trying to add NVMe L2ARC to my 5xHDD RADIZ1 pool. Last attempt of adding 640GiB (10x RAM) GPT partition on NVMe (AData Legend 960) leads to instant crash. I have: vfs.zfs.compressed_arc_enabled=0 vfs.zfs.abd_scatter_enabled=0 Fortunately, crashdump was successful. Here is backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0162741a30 vpanic() at vpanic+0x152/frame 0xfffffe0162741a80 panic() at panic+0x43/frame 0xfffffe0162741ae0 vm_fault() at vm_fault+0x11cb/frame 0xfffffe0162741bf0 vm_fault_trap() at vm_fault_trap+0xb0/frame 0xfffffe0162741c40 trap_pfault() at trap_pfault+0x1ee/frame 0xfffffe0162741ca0 calltrap() at calltrap+0x8/frame 0xfffffe0162741ca0 --- trap 0xc, rip = 0xffffffff809350e5, rsp = 0xfffffe0162741d70, rbp = 0xfffffe0162741d70 --- memset_std() at memset_std+0xe5/frame 0xfffffe0162741d70 l2arc_feed_thread() at l2arc_feed_thread+0xe83/frame 0xfffffe0162741ef0 or #9 No locals. #10 memset_std () at /usr/src/sys/amd64/amd64/support.S:670 No locals. #11 0xffffffff81233fb3 in l2arc_apply_transforms (spa=0xfffffe028928b000, hdr=0xfffff8036e5835a0, asize=4096, abd_out=) at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:9372 mac = '\000' tmp = 0xfffffe0f9cbba800 cabd = 0xfffff806842f2e80 eabd = 0x0 to_write = compress = psize = 0 size = 1536 ismd = 131072 dck = 0x0 no_crypt = 0 ret = bswap = out = #12 l2arc_write_buffers (spa=0xfffffe028928b000, dev=0xfffffe0f9cce0000, target_sz=9502720) at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:9606 type = ARC_BUFC_METADATA ret = to_write = 0x0 hash_lock = 0xffffffff814d48a0 psize = 512 asize = 4096 passed_sz = 2041344 mls = pass = 1 cb = 0xfffff8086d0ea200 guid = 16009569622998995335 l2dhdr = 0xfffff8097cfce000 pio = 0xfffff80937081000 write_psize = 578048 write_asize = 1499136 write_lsize = full = 0 head = 0xfffff809b7b37e40 hdr = 0xfffff8036e5835a0 headroom = 19005440 hdr_prev = 0xfffff80dd5a70a50 wzio = pass = passed_sz = mls = to_write = hash_lock = psize = asize = type = ret = _verify3_left = _verify3_right = #13 l2arc_feed_thread (unused=) at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:9830 cpr = {cc_lockp = 0xffffffff814e1568 , cc_events = 0 '\000', cc_id = 0xfffff800175fed00, cc_callb_cv = { cv_description = 0xffffffff8143dbd9 <.L.str.47+3> "cpr)->cc_callb_cv", cv_waiters = 0}, cc_stop_cv = { cv_description = 0xffffffff8143ac72 <.L.str.48+3> "cpr)->cc_stop_cv", cv_waiters = 0}} next = cookie = 0 dev = 0xfffffe0f9cce0000 spa = 0xfffffe028928b000 begin = 1662606050 size = 9502720 wrote = #14 0xffffffff805e90dd in fork_exit ( callout=0xffffffff81233130 , arg=0x0, frame=0xfffffe0162741f40) at /usr/src/sys/kern/kern_fork.c:1151 td = 0xfffffe01623b31e0 p = 0xfffffe0162367560 dtd = #15 -- // Lev Serebryakov From nobody Mon Jan 8 20:40:31 2024 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T85YL1Wgkz56k4v for ; Mon, 8 Jan 2024 20:40:38 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [IPv6:2607:f3e0:0:1::12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smarthost1.sentex.ca", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4T85YK3k7Pz4Wfq; Mon, 8 Jan 2024 20:40:37 +0000 (UTC) (envelope-from mike@sentex.net) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of mike@sentex.net designates 2607:f3e0:0:1::12 as permitted sender) smtp.mailfrom=mike@sentex.net Received: from pyroxene2a.sentex.ca (pyroxene19.sentex.ca [199.212.134.19]) by smarthost1.sentex.ca (8.17.1/8.16.1) with ESMTPS id 408KeVBJ036157 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=FAIL); Mon, 8 Jan 2024 15:40:31 -0500 (EST) (envelope-from mike@sentex.net) Received: from [IPV6:2607:f3e0:0:4:841e:44bf:309d:bc39] ([IPv6:2607:f3e0:0:4:841e:44bf:309d:bc39]) by pyroxene2a.sentex.ca (8.17.1/8.15.2) with ESMTPS id 408KeTvX023811 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Mon, 8 Jan 2024 15:40:30 -0500 (EST) (envelope-from mike@sentex.net) Message-ID: <06381879-c328-4051-bb2f-9c3620d90fa0@sentex.net> Date: Mon, 8 Jan 2024 15:40:31 -0500 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: FreeBSD-STABLE Mailing List From: mike tancsa Subject: clang 17 and ports fallout Autocrypt: addr=mike@sentex.net; keydata= xsBNBFywzOMBCACoNFpwi5MeyEREiCeHtbm6pZJI/HnO+wXdCAWtZkS49weOoVyUj5BEXRZP xflV2ib2hflX4nXqhenaNiia4iaZ9ft3I1ebd7GEbGnsWCvAnob5MvDZyStDAuRxPJK1ya/s +6rOvr+eQiXYNVvfBhrCfrtR/esSkitBGxhUkBjOti8QwzD71JVF5YaOjBAs7jZUKyLGj0kW yDg4jUndudWU7G2yc9GwpHJ9aRSUN8e/mWdIogK0v+QBHfv/dsI6zVB7YuxCC9Fx8WPwfhDH VZC4kdYCQWKXrm7yb4TiVdBh5kgvlO9q3js1yYdfR1x8mjK2bH2RSv4bV3zkNmsDCIxjABEB AAHNHW1pa2UgdGFuY3NhIDxtaWtlQHNlbnRleC5uZXQ+wsCOBBMBCAA4FiEEmuvCXT0aY6hs 4SbWeVOEFl5WrMgFAl+pQfkCGwMFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQeVOEFl5W rMiN6ggAk3H5vk8QnbvGbb4sinxZt/wDetgk0AOR9NRmtTnPaW+sIJEfGBOz47Xih+f7uWJS j+uvc9Ewn2Z7n8z3ZHJlLAByLVLtcNXGoRIGJ27tevfOaNqgJHBPbFOcXCBBFTx4MYMM4iAZ cDT5vsBTSaM36JZFtHZBKkuFEItbA/N8ZQSHKdTYMIA7A3OCLGbJBqloQ8SlW4MkTzKX4u7R yefAYQ0h20x9IqC5Ju8IsYRFacVZconT16KS81IBceO42vXTN0VexbVF2rZIx3v/NT75r6Vw 0FlXVB1lXOHKydRA2NeleS4NEG2vWqy/9Boj0itMfNDlOhkrA/0DcCurMpnpbM7ATQRcsMzk AQgA1Dpo/xWS66MaOJLwA28sKNMwkEk1Yjs+okOXDOu1F+0qvgE8sVmrOOPvvWr4axtKRSG1 t2QUiZ/ZkW/x/+t0nrM39EANV1VncuQZ1ceIiwTJFqGZQ8kb0+BNkwuNVFHRgXm1qzAJweEt RdsCMohB+H7BL5LGCVG5JaU0lqFU9pFP40HxEbyzxjsZgSE8LwkI6wcu0BLv6K6cLm0EiHPO l5G8kgRi38PS7/6s3R8QDsEtbGsYy6O82k3zSLIjuDBwA9GRaeigGppTxzAHVjf5o9KKu4O7 gC2KKVHPegbXS+GK7DU0fjzX57H5bZ6komE5eY4p3oWT/CwVPSGfPs8jOwARAQABwsB2BBgB CAAgFiEEmuvCXT0aY6hs4SbWeVOEFl5WrMgFAl+pQfkCGwwACgkQeVOEFl5WrMiVqwf9GwU8 c6cylknZX8QwlsVudTC8xr/L17JA84wf03k3d4wxP7bqy5AYy7jboZMbgWXngAE/HPQU95NM aukysSnknzoIpC96XZJ0okLBXVS6Y0ylZQ+HrbIhMpuQPoDweoF5F9wKrsHRoDaUK1VR706X rwm4HUzh7Jk+auuMYfuCh0FVlFBEuiJWMLhg/5WCmcRfiuB6F59ZcUQrwLEZeNhF2XJV4KwB Tlg7HCWO/sy1foE5noaMyACjAtAQE9p5kGYaj+DuRhPdWUTsHNuqrhikzIZd2rrcMid+ktb0 NvtvswzMO059z1YGMtGSqQ4srCArju+XHIdTFdiIYbd7+jeehg== Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.84 on 64.7.153.18 X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.36 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.97)[-0.970]; R_SPF_ALLOW(-0.20)[+ip6:2607:f3e0::/32]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[199.212.134.19:received]; XM_UA_NO_VERSION(0.01)[]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA]; FREEFALL_USER(0.00)[mike]; RCPT_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; MID_RHS_MATCH_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; ARC_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; R_DKIM_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_DN_ALL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[sentex.net]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4T85YK3k7Pz4Wfq After today's MFC of clang17,  I am seeing some fallout from a few ports that build with clang16 on RELENG_13 but now fail.  Any ideas what might be going on ?   I have nothing in /etc/make.conf nor /etc/src.conf Specifically devel/ivykis databases/rrdtool --- iv_thread_posix.lo --- mv -f .deps/iv_thread_posix.Tpo .deps/iv_thread_posix.Plo --- iv_time_posix.lo --- mv -f .deps/iv_time_posix.Tpo .deps/iv_time_posix.Plo --- iv_wait.lo --- libtool: compile:  cc -DHAVE_CONFIG_H -I. -I.. -D_GNU_SOURCE -I../src/include -I../src/include -O2 -pipe -fstack-protector-strong -fno-strict-aliasing -Wall -MT iv_wait.lo -MD -MP -MF .deps/iv_wait.Tpo -c iv_wait.c -o iv_wait.o >/dev/null 2>&1 --- iv_fd_kqueue.lo --- libtool: compile:  cc -DHAVE_CONFIG_H -I. -I.. -D_GNU_SOURCE -I../src/include -I../src/include -O2 -pipe -fstack-protector-strong -fno-strict-aliasing -Wall -MT iv_fd_kqueue.lo -MD -MP -MF .deps/iv_fd_kqueue.Tpo -c iv_fd_kqueue.c -o iv_fd_kqueue.o >/dev/null 2>&1 --- iv_signal.lo --- mv -f .deps/iv_signal.Tpo .deps/iv_signal.Plo --- iv_wait.lo --- mv -f .deps/iv_wait.Tpo .deps/iv_wait.Plo --- iv_fd_kqueue.lo --- mv -f .deps/iv_fd_kqueue.Tpo .deps/iv_fd_kqueue.Plo --- libivykis.la --- /bin/sh ../libtool  --tag=CC    --mode=link cc  -O2 -pipe -fstack-protector-strong -fno-strict-aliasing  -Wall  -version-info 5:6:5 -Wl,--version-script,../libivykis.posix.ver -fstack-protector-strong -o libivykis.la -rpath /usr/local/lib iv_avl.lo iv_event.lo iv_fatal.lo iv_task.lo  iv_timer.lo iv_tls.lo iv_work.lo iv_event_raw_posix.lo iv_fd.lo  iv_fd_poll.lo iv_fd_pump.lo iv_main_posix.lo  iv_popen.lo iv_signal.lo iv_thread_posix.lo  iv_tid_posix.lo iv_time_posix.lo iv_wait.lo iv_fd_kqueue.lo libtool: link: cc -shared  -fPIC -DPIC  .libs/iv_avl.o .libs/iv_event.o .libs/iv_fatal.o .libs/iv_task.o .libs/iv_timer.o .libs/iv_tls.o .libs/iv_work.o .libs/iv_event_raw_posix.o .libs/iv_fd.o .libs/iv_fd_poll.o .libs/iv_fd_pump.o .libs/iv_main_posix.o .libs/iv_popen.o .libs/iv_signal.o .libs/iv_thread_posix.o .libs/iv_tid_posix.o .libs/iv_time_posix.o .libs/iv_wait.o .libs/iv_fd_kqueue.o    -O2 -fstack-protector-strong -Wl,--version-script -Wl,../libivykis.posix.ver -fstack-protector-strong   -Wl,-soname -Wl,libivykis.so.0 -o .libs/libivykis.so.0.5.6 ld: error: version script assignment of 'IVYKIS_0.29' to symbol 'iv_inotify_register' failed: symbol not defined ld: error: version script assignment of 'IVYKIS_0.29' to symbol 'iv_inotify_unregister' failed: symbol not defined ld: error: version script assignment of 'IVYKIS_0.29' to symbol 'iv_inotify_watch_register' failed: symbol not defined ld: error: version script assignment of 'IVYKIS_0.29' to symbol 'iv_inotify_watch_unregister' failed: symbol not defined cc: error: linker command failed with exit code 1 (use -v to see invocation) *** [libivykis.la] Error code 1 make[3]: stopped in /usr/ports/devel/ivykis/work/ivykis-0.42.4/src 1 error make[3]: stopped in /usr/ports/devel/ivykis/work/ivykis-0.42.4/src make[2]: stopped in /usr/ports/devel/ivykis/work/ivykis-0.42.4 make[1]: stopped in /usr/ports/devel/ivykis/work/ivykis-0.42.4 ===> Compilation failed unexpectedly. Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to the maintainer. *** Error code 1 gmake[3]: Leaving directory '/usr/ports/databases/rrdtool/work/rrdtool-1.8.0/po' Making all in src gmake[3]: Entering directory '/usr/ports/databases/rrdtool/work/rrdtool-1.8.0/src' gmake  all-am gmake[4]: Entering directory '/usr/ports/databases/rrdtool/work/rrdtool-1.8.0/src' /bin/sh ../libtool  --tag=CC   --mode=link cc -O2 -pipe -fstack-protector-strong -fno-strict-aliasing  -D_GNU_SOURCE -fno-strict-aliasing -Wall -std=gnu99 -pedantic -Wundef -Wshadow -Wpointer-arith -Wcast-align -Wmissing-prototypes -Wmissing-declarations -Wnested-externs -Winline -Wold-style-definition -W -I.. -D_THREAD_SAFE -pthread -O2 -pipe -fstack-protector-strong -fno-strict-aliasing  -D_GNU_SOURCE -fno-strict-aliasing -Wall -std=gnu99 -pedantic -Wundef -Wshadow -Wpointer-arith -Wcast-align -Wmissing-prototypes -Wmissing-declarations -Wnested-externs -Winline -Wold-style-definition -W  -version-info 11:0:3 -export-symbols ./librrd.sym  -fstack-protector-strong  -L/usr/local/lib -L/usr/local/lib  -o librrd.la -rpath /usr/local/lib librrd_la-rrd_version.lo librrd_la-rrd_last.lo librrd_la-rrd_lastupdate.lo librrd_la-rrd_first.lo librrd_la-rrd_dump.lo librrd_la-rrd_flushcached.lo librrd_la-rrd_fetch.lo librrd_la-rrd_fetch_cb.lo librrd_la-rrd_resize.lo librrd_la-rrd_tune.lo librrd_la-rrd_list.lo  librrd_la-rrd_restore.lo   librrdupd.la -lglib-2.0 -lintl -lm -lwrap -lxml2 libtool: link: rm -fr  .libs/librrd.so.8.3.0-ver libtool: link: echo "{ global:" > .libs/librrd.so.8.3.0-ver libtool: link:           sed -e "s|$|;|" < ./librrd.sym >> .libs/librrd.so.8.3.0-ver libtool: link:   echo "local: *; };" >> .libs/librrd.so.8.3.0-ver libtool: link: cc -shared  -fPIC -DPIC .libs/librrd_la-rrd_version.o .libs/librrd_la-rrd_last.o .libs/librrd_la-rrd_lastupdate.o .libs/librrd_la-rrd_first.o .libs/librrd_la-rrd_dump.o .libs/librrd_la-rrd_flushcached.o .libs/librrd_la-rrd_fetch.o .libs/librrd_la-rrd_fetch_cb.o .libs/librrd_la-rrd_resize.o .libs/librrd_la-rrd_tune.o .libs/librrd_la-rrd_list.o .libs/librrd_la-rrd_restore.o -Wl,--whole-archive ./.libs/librrdupd.a -Wl,--no-whole-archive -L/usr/local/lib -lglib-2.0 -lintl -lm -lwrap -lxml2  -O2 -fstack-protector-strong -pthread -O2 -fstack-protector-strong -fstack-protector-strong   -pthread -Wl,-soname -Wl,librrd.so.8 -Wl,-version-script -Wl,.libs/librrd.so.8.3.0-ver -o .libs/librrd.so.8.3.0 ld: error: version script assignment of 'global' to symbol 'rrd_graph' failed: symbol not defined ld: error: version script assignment of 'global' to symbol 'rrd_graph_v' failed: symbol not defined ld: error: version script assignment of 'global' to symbol 'rrd_lcd' failed: symbol not defined ld: error: version script assignment of 'global' to symbol 'rrd_reduce_data' failed: symbol not defined ld: error: version script assignment of 'global' to symbol 'rrd_xport' failed: symbol not defined cc: error: linker command failed with exit code 1 (use -v to see invocation) gmake[4]: *** [Makefile:773: librrd.la] Error 1 gmake[4]: Leaving directory '/usr/ports/databases/rrdtool/work/rrdtool-1.8.0/src' gmake[3]: *** [Makefile:618: all] Error 2 gmake[3]: Leaving directory '/usr/ports/databases/rrdtool/work/rrdtool-1.8.0/src' gmake[2]: *** [Makefile:504: all-recursive] Error 1 gmake[2]: Leaving directory '/usr/ports/databases/rrdtool/work/rrdtool-1.8.0' ===> Compilation failed unexpectedly. Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to the maintainer. *** Error code 1 Stop. make[1]: stopped in /usr/ports/databases/rrdtool *** Error code 1 Stop. make: stopped in /usr/ports/databases/rrdtool From nobody Mon Jan 8 20:57:10 2024 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T85wP51N6z56mJ6 for ; Mon, 8 Jan 2024 20:57:09 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [IPv6:2607:f3e0:0:1::12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smarthost1.sentex.ca", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4T85wP22Qsz4bXX; Mon, 8 Jan 2024 20:57:09 +0000 (UTC) (envelope-from mike@sentex.net) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of mike@sentex.net designates 2607:f3e0:0:1::12 as permitted sender) smtp.mailfrom=mike@sentex.net Received: from pyroxene2a.sentex.ca (pyroxene19.sentex.ca [199.212.134.19]) by smarthost1.sentex.ca (8.17.1/8.16.1) with ESMTPS id 408Kv9s1037757 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=FAIL); Mon, 8 Jan 2024 15:57:09 -0500 (EST) (envelope-from mike@sentex.net) Received: from [IPV6:2607:f3e0:0:4:841e:44bf:309d:bc39] ([IPv6:2607:f3e0:0:4:841e:44bf:309d:bc39]) by pyroxene2a.sentex.ca (8.17.1/8.15.2) with ESMTPS id 408Kv8i5025330 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Mon, 8 Jan 2024 15:57:08 -0500 (EST) (envelope-from mike@sentex.net) Message-ID: <4a0db68d-bb13-4238-a251-74d02ff14a17@sentex.net> Date: Mon, 8 Jan 2024 15:57:10 -0500 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: clang 17 and ports fallout Content-Language: en-US From: mike tancsa To: FreeBSD-STABLE Mailing List References: <06381879-c328-4051-bb2f-9c3620d90fa0@sentex.net> Autocrypt: addr=mike@sentex.net; keydata= xsBNBFywzOMBCACoNFpwi5MeyEREiCeHtbm6pZJI/HnO+wXdCAWtZkS49weOoVyUj5BEXRZP xflV2ib2hflX4nXqhenaNiia4iaZ9ft3I1ebd7GEbGnsWCvAnob5MvDZyStDAuRxPJK1ya/s +6rOvr+eQiXYNVvfBhrCfrtR/esSkitBGxhUkBjOti8QwzD71JVF5YaOjBAs7jZUKyLGj0kW yDg4jUndudWU7G2yc9GwpHJ9aRSUN8e/mWdIogK0v+QBHfv/dsI6zVB7YuxCC9Fx8WPwfhDH VZC4kdYCQWKXrm7yb4TiVdBh5kgvlO9q3js1yYdfR1x8mjK2bH2RSv4bV3zkNmsDCIxjABEB AAHNHW1pa2UgdGFuY3NhIDxtaWtlQHNlbnRleC5uZXQ+wsCOBBMBCAA4FiEEmuvCXT0aY6hs 4SbWeVOEFl5WrMgFAl+pQfkCGwMFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQeVOEFl5W rMiN6ggAk3H5vk8QnbvGbb4sinxZt/wDetgk0AOR9NRmtTnPaW+sIJEfGBOz47Xih+f7uWJS j+uvc9Ewn2Z7n8z3ZHJlLAByLVLtcNXGoRIGJ27tevfOaNqgJHBPbFOcXCBBFTx4MYMM4iAZ cDT5vsBTSaM36JZFtHZBKkuFEItbA/N8ZQSHKdTYMIA7A3OCLGbJBqloQ8SlW4MkTzKX4u7R yefAYQ0h20x9IqC5Ju8IsYRFacVZconT16KS81IBceO42vXTN0VexbVF2rZIx3v/NT75r6Vw 0FlXVB1lXOHKydRA2NeleS4NEG2vWqy/9Boj0itMfNDlOhkrA/0DcCurMpnpbM7ATQRcsMzk AQgA1Dpo/xWS66MaOJLwA28sKNMwkEk1Yjs+okOXDOu1F+0qvgE8sVmrOOPvvWr4axtKRSG1 t2QUiZ/ZkW/x/+t0nrM39EANV1VncuQZ1ceIiwTJFqGZQ8kb0+BNkwuNVFHRgXm1qzAJweEt RdsCMohB+H7BL5LGCVG5JaU0lqFU9pFP40HxEbyzxjsZgSE8LwkI6wcu0BLv6K6cLm0EiHPO l5G8kgRi38PS7/6s3R8QDsEtbGsYy6O82k3zSLIjuDBwA9GRaeigGppTxzAHVjf5o9KKu4O7 gC2KKVHPegbXS+GK7DU0fjzX57H5bZ6komE5eY4p3oWT/CwVPSGfPs8jOwARAQABwsB2BBgB CAAgFiEEmuvCXT0aY6hs4SbWeVOEFl5WrMgFAl+pQfkCGwwACgkQeVOEFl5WrMiVqwf9GwU8 c6cylknZX8QwlsVudTC8xr/L17JA84wf03k3d4wxP7bqy5AYy7jboZMbgWXngAE/HPQU95NM aukysSnknzoIpC96XZJ0okLBXVS6Y0ylZQ+HrbIhMpuQPoDweoF5F9wKrsHRoDaUK1VR706X rwm4HUzh7Jk+auuMYfuCh0FVlFBEuiJWMLhg/5WCmcRfiuB6F59ZcUQrwLEZeNhF2XJV4KwB Tlg7HCWO/sy1foE5noaMyACjAtAQE9p5kGYaj+DuRhPdWUTsHNuqrhikzIZd2rrcMid+ktb0 NvtvswzMO059z1YGMtGSqQ4srCArju+XHIdTFdiIYbd7+jeehg== In-Reply-To: <06381879-c328-4051-bb2f-9c3620d90fa0@sentex.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.84 on 64.7.153.18 X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.36 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.97)[-0.970]; R_SPF_ALLOW(-0.20)[+ip6:2607:f3e0::/32]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[199.212.134.19:received]; XM_UA_NO_VERSION(0.01)[]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA]; FREEFALL_USER(0.00)[mike]; RCPT_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; MID_RHS_MATCH_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; ARC_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; R_DKIM_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_DN_ALL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[sentex.net]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4T85wP22Qsz4bXX On 1/8/2024 3:40 PM, mike tancsa wrote: > After today's MFC of clang17,  I am seeing some fallout from a few > ports that build with clang16 on RELENG_13 but now fail.  Any ideas > what might be going on ?   I have nothing in /etc/make.conf nor > /etc/src.conf > These build on RELENG_14, so perhaps something still needs to be MFC'd to RELENG_13 ? I have tried both with and without poudriere. Same error in poudriere d_la-rrd_update.lo librrdupd_la-rrd_modify.lo librrdupd_la-quicksort.lo librrdupd_la-rrd_thread_safe.lo  -lm -lwrap -lglib-2.0 -lintl libtool: link: ar cr .libs/librrdupd.a .libs/librrdupd_la-mutex.o .libs/librrdupd_la-optparse.o .libs/librrdupd_la-rrd_strtod.o .libs/librrdupd_la-rrd_create.o .libs/librrdupd_la-hash_32.o .libs/librrdupd_la-rrd_parsetime.o .libs/librrdupd_la-rrd_hw.o .libs/librrdupd_la-rrd_hw_math.o .libs/librrdupd_la-rrd_hw_update.o .libs/librrdupd_la-rrd_diff.o .libs/librrdupd_la-rrd_format.o .libs/librrdupd_la-rrd_info.o .libs/librrdupd_la-rrd_error.o .libs/librrdupd_la-rrd_open.o .libs/librrdupd_la-rrd_client.o .libs/librrdupd_la-rrd_nan_inf.o .libs/librrdupd_la-rrd_rpncalc.o .libs/librrdupd_la-rrd_utils.o .libs/librrdupd_la-rrd_snprintf.o .libs/librrdupd_la-rrd_update.o .libs/librrdupd_la-rrd_modify.o .libs/librrdupd_la-quicksort.o .libs/librrdupd_la-rrd_thread_safe.o libtool: link: ranlib .libs/librrdupd.a libtool: link: ( cd ".libs" && rm -f "librrdupd.la" && ln -s "../librrdupd.la" "librrdupd.la" ) /bin/sh ../libtool  --tag=CC   --mode=link cc -O2 -pipe -fstack-protector-strong -fno-strict-aliasing  -D_GNU_SOURCE -fno-strict-aliasing -Wall -std=gnu99 -pedantic -Wundef -Wshadow -Wpointer-arith -Wcast-align -Wmissing-prototypes -Wmissing-declarations -Wnested-externs -Winline -Wold-style-definition -W -I.. -D_THREAD_SAFE -pthread -O2 -pipe -fstack-protector-strong -fno-strict-aliasing  -D_GNU_SOURCE -fno-strict-aliasing -Wall -std=gnu99 -pedantic -Wundef -Wshadow -Wpointer-arith -Wcast-align -Wmissing-prototypes -Wmissing-declarations -Wnested-externs -Winline -Wold-style-definition -W  -version-info 11:0:3 -export-symbols ./librrd.sym  -fstack-protector-strong  -L/usr/local/lib -L/usr/local/lib  -o librrd.la -rpath /usr/local/lib librrd_la-rrd_version.lo librrd_la-rrd_last.lo librrd_la-rrd_lastupdate.lo librrd_la-rrd_first.lo librrd_la-rrd_dump.lo librrd_la-rrd_flushcached.lo librrd_la-rrd_fetch.lo librrd_la-rrd_fetch_cb.lo librrd_la-rrd_resize.lo librrd_la-rrd_tune.lo librrd_la-rrd_list.lo  librrd_la-rrd_restore.lo   librrdupd.la -lglib-2.0 -lintl -lm -lwrap -lxml2 libtool: link: echo "{ global:" > .libs/librrd.so.8.3.0-ver libtool: link:           sed -e "s|$|;|" < ./librrd.sym >> .libs/librrd.so.8.3.0-ver libtool: link:   echo "local: *; };" >> .libs/librrd.so.8.3.0-ver libtool: link: cc -shared  -fPIC -DPIC .libs/librrd_la-rrd_version.o .libs/librrd_la-rrd_last.o .libs/librrd_la-rrd_lastupdate.o .libs/librrd_la-rrd_first.o .libs/librrd_la-rrd_dump.o .libs/librrd_la-rrd_flushcached.o .libs/librrd_la-rrd_fetch.o .libs/librrd_la-rrd_fetch_cb.o .libs/librrd_la-rrd_resize.o .libs/librrd_la-rrd_tune.o .libs/librrd_la-rrd_list.o .libs/librrd_la-rrd_restore.o -Wl,--whole-archive ./.libs/librrdupd.a -Wl,--no-whole-archive -L/usr/local/lib -lglib-2.0 -lintl -lm -lwrap -lxml2  -O2 -fstack-protector-strong -pthread -O2 -fstack-protector-strong -fstack-protector-strong   -pthread -Wl,-soname -Wl,librrd.so.8 -Wl,-version-script -Wl,.libs/librrd.so.8.3.0-ver -o .libs/librrd.so.8.3.0 ld: error: version script assignment of 'global' to symbol 'rrd_graph' failed: symbol not defined ld: error: version script assignment of 'global' to symbol 'rrd_graph_v' failed: symbol not defined ld: error: version script assignment of 'global' to symbol 'rrd_lcd' failed: symbol not defined ld: error: version script assignment of 'global' to symbol 'rrd_reduce_data' failed: symbol not defined ld: error: version script assignment of 'global' to symbol 'rrd_xport' failed: symbol not defined cc: error: linker command failed with exit code 1 (use -v to see invocation) gmake[3]: *** [Makefile:773: librrd.la] Error 1 gmake[3]: Leaving directory '/wrkdirs/usr/ports/databases/rrdtool/work/rrdtool-1.8.0/src' gmake[2]: *** [Makefile:618: all] Error 2 gmake[2]: Leaving directory '/wrkdirs/usr/ports/databases/rrdtool/work/rrdtool-1.8.0/src' gmake[1]: *** [Makefile:504: all-recursive] Error 1 gmake[1]: Leaving directory '/wrkdirs/usr/ports/databases/rrdtool/work/rrdtool-1.8.0' *** Error code 1 Stop. make: stopped in /usr/ports/databases/rrdtool =>> Cleaning up wrkdir ===>  Cleaning for rrdtool-1.8.0_2 build of databases/rrdtool | rrdtool-1.8.0_2 ended at Mon Jan  8 15:44:27 EST 2024 build time: 00:01:04 !!! build failure encountered !!!     ---Mike From eugen@grosbein.net Mon Jan 8 20:57:28 2024 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T85x03kJbz56mTV for ; Mon, 8 Jan 2024 20:57:40 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4T85wz1m95z4cKT; Mon, 8 Jan 2024 20:57:39 +0000 (UTC) (envelope-from eugen@grosbein.net) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=grosbein.net (policy=none); spf=fail (mx1.freebsd.org: domain of eugen@grosbein.net does not designate 2a01:4f8:c2c:26d8::2 as permitted sender) smtp.mailfrom=eugen@grosbein.net Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.17.1/8.17.1) with ESMTPS id 408KvU78015243 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 8 Jan 2024 20:57:30 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: freebsd-stable@freebsd.org Received: from [10.58.0.10] (dadvw [10.58.0.10]) by eg.sd.rdtc.ru (8.17.1/8.17.1) with ESMTPS id 408KvSAu076184 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 9 Jan 2024 03:57:28 +0700 (+07) (envelope-from eugen@grosbein.net) To: FreeBSD stable Cc: FreeBSD Release Engineering Team From: Eugene Grosbein Subject: kern.version and uname -v Message-ID: Date: Tue, 9 Jan 2024 03:57:28 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,SHORTCIRCUIT autolearn=disabled version=3.4.6 X-Spam-Report: * -0.0 SHORTCIRCUIT No description available. * -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on hz.grosbein.net X-Spamd-Bar: - X-Spamd-Result: default: False [-1.97 / 15.00]; R_SPF_FAIL(1.00)[-all]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.97)[-0.971]; DMARC_POLICY_SOFTFAIL(0.10)[grosbein.net : No valid SPF, No valid DKIM,none]; MIME_GOOD(-0.10)[text/plain]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE]; ARC_NA(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; FREEFALL_USER(0.00)[eugen]; MID_RHS_MATCH_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; R_DKIM_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_DN_ALL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4T85wz1m95z4cKT Hi! For ages, "uname -v" output (sligtly polished sysctl kern.version) had the following format: $ uname -v FreeBSD 13.2-STABLE 36a037f15 KERNELIDENT Where KERNELIDENT is GENERIC for x86 distribution media. But now: # uname -v FreeBSD 14.0-RELEASE #0 releng/14.0-n265380-f9716eee8ab4: Fri Nov 10 05:51:26 UTC 2023 root@releng1.nyi.freebsd.org:/usr/obj/usr/src/i386.i386/sys/GENERIC Do we really need to break the format and include these into "uname -v" output for release and stable branches? Eugene Grosbein From nobody Mon Jan 8 21:16:43 2024 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T86MD1X6Pz56prQ for ; Mon, 8 Jan 2024 21:16:56 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [IPv6:2a00:1450:4864:20::62e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4T86MC6y7Gz4hxs for ; Mon, 8 Jan 2024 21:16:55 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ej1-x62e.google.com with SMTP id a640c23a62f3a-a29a4f610b1so239012866b.3 for ; Mon, 08 Jan 2024 13:16:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1704748614; x=1705353414; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=TK3kXYFsP4+pxpCsOpSW1T2bYgg865Ia8FOVsN817GM=; b=WbzkxaNA3Ut8F7dOPqcOU5r90QbEbmv0Sf39i/KDll8oxql4c+HUkkJug8zsycICnD 84mQ+vi6ANftWPXtg5CuthNMlhVOFkIn62myHie6A5riwX1EpvrXo/jfGCyoNh8+oQcD YVvPjjxARwxcg8RUYSS7IBYbyzVTE8c6Z4ZGZJjxfAIDogn5Jn4fsR/3zGjVHsX4Y3rA rXPasOx4RfP9UsIXRv1OEFleE4vZV+6AMPhWQbEWIGABj8FKU5fWRh5IIUoBSRIkxZ33 8Vqff1rnkz5FGCCjRFm1PlPFCfgObUQJbbinqa9Xw+/36ggwKHYoGgI8BuJQEgTN4zqC LtKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704748614; x=1705353414; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=TK3kXYFsP4+pxpCsOpSW1T2bYgg865Ia8FOVsN817GM=; b=BSYHml+fIUlT5XPqNOyvoasRjRwMtj/pm4FjtJF/S5lSpZiQ1FD5T2S7xu0+5PFuli WryRmzE9nqLl+PwOB7EjRAn6B0Klcd4m37ZXjCcenK+qOGpg0M6S+MZjDwM5Ydgmh8ut agbjve79Tz0/6fli1vGr5SE7kT1FJ4VckCRMfIcqtuazSU3fAK/uskuvdActv2h8326M hwnshXylALviq6m/nMa/+XDzpn3gMEoMtnhKPoHgV4hfj5/epwB62gdHqLVa7Ja1cPaM dSScw8acbCfpWZTU7QMn1U2FJ6bmjjQ51iPPGPuTdUEajJxbGLRtUOCz2kw4CRJ1IlBa U6rA== X-Gm-Message-State: AOJu0YxPRyjE7CE5b+L/5nUjhOoq6XG9QRj191FS3VWIr50yxS/WdrqR OLOXsI3cErtvwBoJjFfIVo3sBCLOXzEtP1keOLu8cjWtqVWKCQ== X-Google-Smtp-Source: AGHT+IHa3FAVHPwn6XasvEI/o63ANjS43hBrtnspoeXvWn/a1AOiqxvq7mP2gHTf7p5WsbLWTk9cR2NFnKFYl9wDRzY= X-Received: by 2002:a17:907:72c6:b0:a27:a64e:34bc with SMTP id du6-20020a17090772c600b00a27a64e34bcmr34239ejc.34.1704748614422; Mon, 08 Jan 2024 13:16:54 -0800 (PST) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Mon, 8 Jan 2024 14:16:43 -0700 Message-ID: Subject: Re: kern.version and uname -v To: Eugene Grosbein Cc: FreeBSD stable , FreeBSD Release Engineering Team Content-Type: multipart/alternative; boundary="000000000000982328060e75b704" X-Rspamd-Queue-Id: 4T86MC6y7Gz4hxs X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] --000000000000982328060e75b704 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, Jan 8, 2024 at 1:58=E2=80=AFPM Eugene Grosbein = wrote: > Hi! > > For ages, "uname -v" output (sligtly polished sysctl kern.version) had th= e > following format: > $ uname -v > FreeBSD 13.2-STABLE 36a037f15 KERNELIDENT > This is the reproducible format: only include data that is identical from build to build. It's relatively recent (FreeBSD 11 maybe) > Where KERNELIDENT is GENERIC for x86 distribution media. > > But now: > > # uname -v > FreeBSD 14.0-RELEASE #0 releng/14.0-n265380-f9716eee8ab4: Fri Nov 10 > 05:51:26 UTC 2023 root@releng1.nyi.freebsd.org: > /usr/obj/usr/src/i386.i386/sys/GENERIC > This is the old, non-reproducible format. We've had both formats for several major releases, and this format, with various tweaks as we went from CVS -> svn -> git. Maybe the problem here is that in the run up to 14.0 we didn't turn on reproducible builds? > Do we really need to break the format and include these into "uname -v" > output > for release and stable branches? > I'd argue it is not broken. uname -v format is not specified nor guaranteed to produce specific results. Changed maybe, but not broken. It changes at major releases. Warner --000000000000982328060e75b704 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Mon, Jan 8, 2024 at 1:58=E2=80=AFP= M Eugene Grosbein <eugen@grosbein.= net> wrote:
Hi!

For ages, "uname -v" output (sligtly polished sysctl kern.version= ) had the following format:
$ uname -v
FreeBSD 13.2-STABLE 36a037f15 KERNELIDENT

This is the reproducible format: only include data that is identical fro= m build to build. It's relatively recent (FreeBSD 11 maybe)
= =C2=A0
Where KERNELIDENT is GENERIC for x86 distribution media.

But now:

# uname -v
FreeBSD 14.0-RELEASE #0 releng/14.0-n265380-f9716eee8ab4: Fri Nov 10 05:51:= 26 UTC 2023=C2=A0 =C2=A0 =C2=A0root@releng1.nyi.freebsd.org:/usr/obj/usr/sr= c/i386.i386/sys/GENERIC

This is the old= , non-reproducible format. We've had both formats for several major rel= eases, and this format, with various tweaks as we went from CVS -> svn -= > git.

Maybe the problem here is that in the ru= n up to 14.0 we didn't turn on reproducible builds?
=C2=A0
Do we really need to break the format and include these into "uname -v= " output
for release and stable branches?

I'= d argue it is not broken. uname -v format is not specified nor guaranteed t= o produce specific results. Changed maybe, but not broken. It changes at ma= jor releases.

Warner
--000000000000982328060e75b704-- From nobody Mon Jan 8 21:26:00 2024 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T86Z21CQHz56rDv for ; Mon, 8 Jan 2024 21:26:18 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4T86Z16ZtFz4kZK; Mon, 8 Jan 2024 21:26:17 +0000 (UTC) (envelope-from otis@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1704749177; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=b0oQAlOTioPt3v+HZyAtVvdPtcPytCxnvqCDbNRyLi4=; b=CNA6WZcs7OYuVEUAclVfLtTNSNnahRM32C0mq2tdfXYn13LL3Xu8UKqAXdH6/jLzp5LEbz v3NEaq5VPYZspe7uO/F7rH+mLUU7LR3h8RNw58zI2dctifmcwTvY3U03ITNzOKENJJu9sg r8PmVZa9OLcE5/zSLsiGGDRwhIbN2W1Dvu3JQQvS0SSsqC/PYRlRweKHTEpaZgWJ9batwo M6b0dvqJL9QGQZ8ioShjrYKIdPSVN3O/14+0nJzCwMj3UaVbp1itHzzhnqEoK8tPBR2OFZ deBRHtlNzzaPx30pO69RCStLu2e22+hE03WbgjIYhe9D/CpIzGe/CdYYPeazng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1704749177; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=b0oQAlOTioPt3v+HZyAtVvdPtcPytCxnvqCDbNRyLi4=; b=cKL8ueADBsbWg39PewofsQtQxG7XtSoBI3fpa4BIec4BS08V5dpAwFEiyhZqNy0mfn6Dxl NupNy+2WoEF0iNbepT/DYY/ozgRrRxrLULpwQTNjxmgMbYk2Pf52R3JeXqB37d3iwLjMNc PHTAMKPlyr0SMKLaah0cWgPoa51D3AeHC7BxZELgpBhQMM4MoSgsCumJBsNRes8IzUWeDP 9oUldjKJmfNZzeV5CYPNyCF+stYHTJmfRGmtR+HEZZv/E4kuuiUZdEILO8fhOtc/vVZyqN RrFis1KZnvLO7zP1xW6ue24Wr9zKzZnvEIAyOjRmQeZsZ8EucMW9mRLpnSAoHw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1704749177; a=rsa-sha256; cv=none; b=yyAx8pDAWyTYLy4s3paZxQm1juJPCqgspIZe3ziG2AT91lGEMOmAlxWNUlf92pfNkIp8fF S8XQWmj3HnvjjGp+9PK6tH1Oo/RsjirzQL+nED2G8RCwI88NpqutQQkqLdAbwxcruRP6Ie eZYoSa2Dr82szJTViWclwYgJBQ+SKVJYm/vXHA7iwTsPJP4KlmSLbxzY6I+a6Ttg/sMZYJ Pi/NCHgoUUNboiRQV0l5f94ieEQOobAKqGMyn/VE06OkPfjAj5MbtnwKEG07dOtBAUladz Z8r+4Ni7m1GnVKRVyyjHs53DsMetfYUzxpdkU6Uygu6JV4vxve8WijRbX1OsGA== Received: from ns2.wilbury.net (ns2.wilbury.net [IPv6:2a01:b200:0:1:f816:3eff:fecd:13e6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "svc.wilbury.net", Issuer "R3" (verified OK)) (Authenticated sender: otis) by smtp.freebsd.org (Postfix) with ESMTPSA id 4T86Z14mL2z1HwV; Mon, 8 Jan 2024 21:26:17 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtpclient.apple (gw-upc.owhome.net [188.167.168.254]) (Authenticated sender: juraj@lutter.sk) by svc.wilbury.net (Postfix) with ESMTPSA id 9AD0361FAD; Mon, 8 Jan 2024 22:26:10 +0100 (CET) From: Juraj Lutter Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_DF6E925E-A5C5-4DFE-92BA-8D8D71244191" List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.300.61.1.2\)) Subject: Re: kern.version and uname -v Date: Mon, 8 Jan 2024 22:26:00 +0100 In-Reply-To: Cc: Eugene Grosbein , FreeBSD stable , FreeBSD Release Engineering Team To: Warner Losh References: X-Mailer: Apple Mail (2.3774.300.61.1.2) --Apple-Mail=_DF6E925E-A5C5-4DFE-92BA-8D8D71244191 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi, > On 8 Jan 2024, at 22:16, Warner Losh wrote: >=20 > This is the old, non-reproducible format. We've had both formats for = several major releases, and this format, with various tweaks as we went = from CVS -> svn -> git. >=20 > Maybe the problem here is that in the run up to 14.0 we didn't turn on = reproducible builds? I=E2=80=99ve reported this on IRC already that 14.0-RELEASE was not = built with reproducible build turned on. =E2=80=94 Juraj Lutter otis@FreeBSD.org --Apple-Mail=_DF6E925E-A5C5-4DFE-92BA-8D8D71244191 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Hi,

On 8 Jan 2024, at 22:16, Warner Losh = <imp@bsdimp.com> wrote:

This is the old, = non-reproducible format. We've had both formats for several major = releases, and this format, with various tweaks as we went from CVS -> = svn -> git.

Maybe the problem here is that = in the run up to 14.0 we didn't turn on reproducible = builds?

I=E2=80=99ve = reported this on IRC already that 14.0-RELEASE was not built with = reproducible build turned on.


=E2=80=94
Juraj = Lutter
otis@FreeBSD.org

= --Apple-Mail=_DF6E925E-A5C5-4DFE-92BA-8D8D71244191-- From nobody Mon Jan 8 22:50:31 2024 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T88RG3BTpz5746X for ; Mon, 8 Jan 2024 22:50:34 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4T88RG2N2yz4vnr; Mon, 8 Jan 2024 22:50:34 +0000 (UTC) (envelope-from dim@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1704754234; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Xk9GE8Bir7BRgmBJeKo/jms/zc8KJShIZAZJaHpMkEo=; b=geKO2ppy+bSybIZJ7ak7mmI/EEonX6/3Xjr+l8M862aFYIkPkRz8Dyp7vMouH9mHYDwzHc 31M7shMlOclWdDgrJQjivR6JTQPdv/MapweVxysjM8AWbtnp2wZ+gU/W5h5BgTZMaqjsDG AWswzLg+oAAhEpcNDdS4e7B4UYX7xAzE8t9FZl0FkcT2nvmc+6+4LT3Z+7fJvnI5dnlHl7 pxElJSC3T899ZHfpVYbxoDat8vMGrIa/AHoIkQPeHV7fEzyB2OEQB+G3BR7KbMraUqeN6K hUdG0akU2CrpY30dno3ynmyJuIEh45PR9+2BNkBPxqKc0pLfn2VBSRW2xmXkTA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1704754234; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Xk9GE8Bir7BRgmBJeKo/jms/zc8KJShIZAZJaHpMkEo=; b=Lb1ajAAR8QjyC2WDN91PjdyYLy/M0V2VlT2aU9U1AN96TiC6WPzEKmVG8iIyQFp+NJV+pm fihDSSxayTIwj6RKSgjYssVq4QrggxNiXCxmjkzmNWiRmkwiCIW7UrG/Hg9UAF1aeKulgX vyELMI1rdGrihMy8Yyz2/wlkD0dGVyp7oyAo9lnd2IrwHsgOpmmgyDsm6k+t5fVjtL+1xz /Z9zqGO84D7wGOmB90woOrK5h261jFLZFgisMeQdbVj2oVYL5LFRR/VFnGwYR+6HaX7W1d n78XnnQXqDi0C2uKHMpeVWtzP7oLLVL8mDOEObAO2385Z5JEolBU/q4n3hjbTg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1704754234; a=rsa-sha256; cv=none; b=fD7q3ayI2t9MpALNCOVkLqeT+X7hQVXuqxfz2AOFZueIgcRVdsooNxAC0Dns+YfyiuF+n2 HVHPvKNx9V/ZLpLXg/b2BWcVR0qTgYT5wWjT7gFVhgbM0EqPOozJUFPD4G262apzBsVa2f LMWkVJ8huW1FYIoi+o/nPMHrwuG5F2FZRblb1mOX5aO0F0ozSXn2wmDed7xq6l2v7hX/48 /3NbRTMpEBNHvOW5wZm+ibb+NjCHcPdeEaiJNQVQ2f7VH/r1caKRA7g8CcVCf3PoRMAQ5H lkE626H+INebweZz50vkiOI1AFCiBBmFvkNKgN9eroZN90e3Z8pXRaUF6wgYrQ== Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "tensor.andric.com", Issuer "R3" (verified OK)) (Authenticated sender: dim) by smtp.freebsd.org (Postfix) with ESMTPSA id 4T88RG09pRz1Kt6; Mon, 8 Jan 2024 22:50:34 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtpclient.apple (longrow.home.andric.com [192.168.0.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 273AD54CE8; Mon, 8 Jan 2024 23:50:32 +0100 (CET) Content-Type: text/plain; charset=us-ascii List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: clang 17 and ports fallout From: Dimitry Andric In-Reply-To: <4a0db68d-bb13-4238-a251-74d02ff14a17@sentex.net> Date: Mon, 8 Jan 2024 23:50:31 +0100 Cc: FreeBSD-STABLE Mailing List Content-Transfer-Encoding: quoted-printable Message-Id: References: <06381879-c328-4051-bb2f-9c3620d90fa0@sentex.net> <4a0db68d-bb13-4238-a251-74d02ff14a17@sentex.net> To: mike tancsa X-Mailer: Apple Mail (2.3731.700.6) On 8 Jan 2024, at 21:57, mike tancsa wrote: >=20 > On 1/8/2024 3:40 PM, mike tancsa wrote: >> After today's MFC of clang17, I am seeing some fallout from a few = ports that build with clang16 on RELENG_13 but now fail. Any ideas what = might be going on ? I have nothing in /etc/make.conf nor /etc/src.conf >>=20 > These build on RELENG_14, so perhaps something still needs to be MFC'd = to RELENG_13 ? >=20 > I have tried both with and without poudriere. Same error in poudriere >=20 > d_la-rrd_update.lo librrdupd_la-rrd_modify.lo = librrdupd_la-quicksort.lo librrdupd_la-rrd_thread_safe.lo -lm -lwrap = -lglib-2.0 -lintl > libtool: link: ar cr .libs/librrdupd.a .libs/librrdupd_la-mutex.o = libs/librrdupd_la-optparse.o .libs/librrdupd_la-rrd_strtod.o = libs/librrdupd_la-rrd_create.o .libs/librrdupd_la-hash_32.o = libs/librrdupd_la-rrd_parsetime.o .libs/librrdupd_la-rrd_hw.o = libs/librrdupd_la-rrd_hw_math.o .libs/librrdupd_la-rrd_hw_update.o = libs/librrdupd_la-rrd_diff.o .libs/librrdupd_la-rrd_format.o = libs/librrdupd_la-rrd_info.o .libs/librrdupd_la-rrd_error.o = libs/librrdupd_la-rrd_open.o .libs/librrdupd_la-rrd_client.o = libs/librrdupd_la-rrd_nan_inf.o .libs/librrdupd_la-rrd_rpncalc.o = libs/librrdupd_la-rrd_utils.o .libs/librrdupd_la-rrd_snprintf.o = libs/librrdupd_la-rrd_update.o .libs/librrdupd_la-rrd_modify.o = libs/librrdupd_la-quicksort.o .libs/librrdupd_la-rrd_thread_safe.o > libtool: link: ranlib .libs/librrdupd.a > libtool: link: ( cd ".libs" && rm -f "librrdupd.la" && ln -s = "../librrdupd.la" "librrdupd.la" ) > /bin/sh ../libtool --tag=3DCC --mode=3Dlink cc -O2 -pipe = -fstack-protector-strong -fno-strict-aliasing -D_GNU_SOURCE = -fno-strict-aliasing -Wall -std=3Dgnu99 -pedantic -Wundef -Wshadow = -Wpointer-arith -Wcast-align -Wmissing-prototypes -Wmissing-declarations = -Wnested-externs -Winline -Wold-style-definition -W -I.. -D_THREAD_SAFE = -pthread -O2 -pipe -fstack-protector-strong -fno-strict-aliasing = -D_GNU_SOURCE -fno-strict-aliasing -Wall -std=3Dgnu99 -pedantic -Wundef = -Wshadow -Wpointer-arith -Wcast-align -Wmissing-prototypes = -Wmissing-declarations -Wnested-externs -Winline -Wold-style-definition = -W -version-info 11:0:3 -export-symbols ./librrd.sym = -fstack-protector-strong -L/usr/local/lib -L/usr/local/lib -o = librrd.la -rpath /usr/local/lib librrd_la-rrd_version.lo = librrd_la-rrd_last.lo librrd_la-rrd_lastupdate.lo librrd_la-rrd_first.lo = librrd_la-rrd_dump.lo librrd_la-rrd_flushcached.lo = librrd_la-rrd_fetch.lo librrd_la-rrd_fetch_cb.lo librrd_la-rrd_resize.lo = librrd_la-rrd_tune.lo librrd_la-rrd_list.lo librrd_la-rrd_restore.lo = librrdupd.la -lglib-2.0 -lintl -lm -lwrap -lxml2 > libtool: link: echo "{ global:" > .libs/librrd.so.8.3.0-ver > libtool: link: sed -e "s|$|;|" < ./librrd.sym >> = libs/librrd.so.8.3.0-ver > libtool: link: echo "local: *; };" >> .libs/librrd.so.8.3.0-ver > libtool: link: cc -shared -fPIC -DPIC .libs/librrd_la-rrd_version.o = libs/librrd_la-rrd_last.o .libs/librrd_la-rrd_lastupdate.o = libs/librrd_la-rrd_first.o .libs/librrd_la-rrd_dump.o = libs/librrd_la-rrd_flushcached.o .libs/librrd_la-rrd_fetch.o = libs/librrd_la-rrd_fetch_cb.o .libs/librrd_la-rrd_resize.o = libs/librrd_la-rrd_tune.o .libs/librrd_la-rrd_list.o = libs/librrd_la-rrd_restore.o -Wl,--whole-archive ./.libs/librrdupd.a = -Wl,--no-whole-archive -L/usr/local/lib -lglib-2.0 -lintl -lm -lwrap = -lxml2 -O2 -fstack-protector-strong -pthread -O2 = -fstack-protector-strong -fstack-protector-strong -pthread -Wl,-soname = -Wl,librrd.so.8 -Wl,-version-script -Wl,.libs/librrd.so.8.3.0-ver -o = libs/librrd.so.8.3.0 > ld: error: version script assignment of 'global' to symbol 'rrd_graph' = failed: symbol not defined > ld: error: version script assignment of 'global' to symbol = 'rrd_graph_v' failed: symbol not defined > ld: error: version script assignment of 'global' to symbol 'rrd_lcd' = failed: symbol not defined > ld: error: version script assignment of 'global' to symbol = 'rrd_reduce_data' failed: symbol not defined > ld: error: version script assignment of 'global' to symbol 'rrd_xport' = failed: symbol not defined > cc: error: linker command failed with exit code 1 (use -v to see = invocation) > gmake[3]: *** [Makefile:773: librrd.la] Error 1 > gmake[3]: Leaving directory = '/wrkdirs/usr/ports/databases/rrdtool/work/rrdtool-1.8.0/src' > gmake[2]: *** [Makefile:618: all] Error 2 > gmake[2]: Leaving directory = '/wrkdirs/usr/ports/databases/rrdtool/work/rrdtool-1.8.0/src' > gmake[1]: *** [Makefile:504: all-recursive] Error 1 > gmake[1]: Leaving directory = '/wrkdirs/usr/ports/databases/rrdtool/work/rrdtool-1.8.0' > *** Error code 1 I fixed a lot of ports in the run-up to merging llvm-17 in 15-CURRENT, but I could not get them all. The preferred way is fixing the port by removing the undefined symbols from the linker version script in the port, but if that is not possible or difficult, add -Wl,--undefined-version to the linker flags suppresses the error. E.g. in the port Makefile: LDFLAGS+=3D -Wl,--undefined-version For an example, see: = https://github.com/freebsd/freebsd-ports/commit/37790b26cbda11cd4bb6f237b8= 6cd94739c4059c -Dimitry From eugen@grosbein.net Tue Jan 9 02:17:27 2024 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T8F2550l3z55ftT for ; Tue, 9 Jan 2024 02:17:33 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4T8F25249Fz4TDT; Tue, 9 Jan 2024 02:17:33 +0000 (UTC) (envelope-from eugen@grosbein.net) Authentication-Results: mx1.freebsd.org; none Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.17.1/8.17.1) with ESMTPS id 4092HSrl018468 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Tue, 9 Jan 2024 02:17:29 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: imp@bsdimp.com Received: from [10.58.0.10] (dadvw [10.58.0.10]) by eg.sd.rdtc.ru (8.17.1/8.17.1) with ESMTPS id 4092HRHK081430 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 9 Jan 2024 09:17:28 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: kern.version and uname -v To: Warner Losh References: Cc: FreeBSD stable , FreeBSD Release Engineering Team From: Eugene Grosbein Message-ID: Date: Tue, 9 Jan 2024 09:17:27 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,SHORTCIRCUIT autolearn=disabled version=3.4.6 X-Spam-Report: * -0.0 SHORTCIRCUIT No description available. * -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on hz.grosbein.net X-Rspamd-Queue-Id: 4T8F25249Fz4TDT X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE] 09.01.2024 4:16, Warner Losh wrote: > On Mon, Jan 8, 2024 at 1:58 PM Eugene Grosbein > wrote: > For ages, "uname -v" output (sligtly polished sysctl kern.version) had the following format: > $ uname -v > FreeBSD 13.2-STABLE 36a037f15 KERNELIDENT > > This is the reproducible format: only include data that is identical from build to build. It's relatively recent (FreeBSD 11 maybe) > > Where KERNELIDENT is GENERIC for x86 distribution media. > > But now: > > # uname -v > FreeBSD 14.0-RELEASE #0 releng/14.0-n265380-f9716eee8ab4: Fri Nov 10 05:51:26 UTC 2023 root@releng1.nyi.freebsd.org:/usr/obj/usr/src/i386.i386/sys/GENERIC > > This is the old, non-reproducible format. We've had both formats for several major releases, and this format, with various tweaks as we went from CVS -> svn -> git. > > Maybe the problem here is that in the run up to 14.0 we didn't turn on reproducible builds? You meant opposite, did you? Eugene From nobody Tue Jan 9 03:29:23 2024 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T8GdC0djfz55tTg for ; Tue, 9 Jan 2024 03:29:35 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4T8GdB45VLz4dhj for ; Tue, 9 Jan 2024 03:29:34 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-wr1-x432.google.com with SMTP id ffacd0b85a97d-3373a30af67so2411537f8f.0 for ; Mon, 08 Jan 2024 19:29:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1704770971; x=1705375771; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=l6KjdrT5vkhIjAuTWZ9bOpQWoSj/Gjyt19LFUOr1/N4=; b=AXTZRLLvzXgEyj0M17eEiry/Ord8Et5zKArfQylY+fytmgfdiuDuDV/bX1qYwd4AQS DGabPw0+zPc8vchVfdQ4wkFDomtqmA5gu+vgERYBe1U7HSB593zI31iWrF4ygZeK7Dia fllujwV+mIJijePSBOnjKx62WFr/EF5atLcS3oesu1EP3XZGbDpTl0086yB4vgYMzhIj duQn3TYO2ez4GKfHmELogioOv7GbM33N84eNIBxD+gXgA+Nq2K9Xz2Y2d1HSuDqxyb4g AM88UWvNzuvgflFxG1zZGmhNAH7aB7dfuxOQTQ1xflR9nn3YtdiG5Fb4gsMryce75zcl B+5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704770971; x=1705375771; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=l6KjdrT5vkhIjAuTWZ9bOpQWoSj/Gjyt19LFUOr1/N4=; b=xLVX2+ypJSlnun8NqE4lrysIMXqjuMMxS6hDgRVda48+/t6gutcgTugjrY/gCnoCHS 9WFTuXt+UgVOYW99E0Qb0jgpXx2yMahq+PCApc3y4BzuhFUKAm2JatUbYtU5+SoiqXJG W42ptaZCyeDyzCgSuk3XGJL2GmDsgVZPGoTRXCTOLL+3seudg9TQ3EGEAvzB2t9zUWm0 UpZvs4OVYl5R1r+c1S2KgZJLGmRyWBnfk26SbXTVG9LQgeR/ns5NBU381wnsWjL0d9D8 kfwVyUTxs1W9BDPGztHWoE8Q2Y3fJlzAVkDsjT1Cfue1PCu/MzoqwYdnvxYr+RnJdglD gOQA== X-Gm-Message-State: AOJu0YzvuIIVrx3CDgu9uPE/dS+Fz2rigjrJQWEyLlDe1770bPOASJHk Z4Ld/g968VJxihkJ9jlq4TdZFQ3l9I6dbfCugMOXB3Anox3YhA== X-Google-Smtp-Source: AGHT+IEwgLUjGZu/VTQ3XjkrzlpXjlHeqDoPeH9Y7J5f4JyYkow/Fd/c4mDwSuu6beRQtTBC4+bSHCkDDZFh6OpL90A= X-Received: by 2002:a05:600c:138c:b0:40d:8fcd:319e with SMTP id u12-20020a05600c138c00b0040d8fcd319emr1574409wmf.240.1704770971150; Mon, 08 Jan 2024 19:29:31 -0800 (PST) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Mon, 8 Jan 2024 20:29:23 -0700 Message-ID: Subject: Re: kern.version and uname -v To: Eugene Grosbein Cc: FreeBSD stable , FreeBSD Release Engineering Team Content-Type: multipart/alternative; boundary="00000000000028be68060e7aecb0" X-Rspamd-Queue-Id: 4T8GdB45VLz4dhj X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] --00000000000028be68060e7aecb0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, Jan 8, 2024 at 7:17=E2=80=AFPM Eugene Grosbein = wrote: > 09.01.2024 4:16, Warner Losh wrote: > > > On Mon, Jan 8, 2024 at 1:58=E2=80=AFPM Eugene Grosbein > wrote: > > For ages, "uname -v" output (sligtly polished sysctl kern.version) > had the following format: > > $ uname -v > > FreeBSD 13.2-STABLE 36a037f15 KERNELIDENT > > > > This is the reproducible format: only include data that is identical > from build to build. It's relatively recent (FreeBSD 11 maybe) > > > > Where KERNELIDENT is GENERIC for x86 distribution media. > > > > But now: > > > > # uname -v > > FreeBSD 14.0-RELEASE #0 releng/14.0-n265380-f9716eee8ab4: Fri Nov 1= 0 > 05:51:26 UTC 2023 root@releng1.nyi.freebsd.org: > /usr/obj/usr/src/i386.i386/sys/GENERIC > > > > This is the old, non-reproducible format. We've had both formats for > several major releases, and this format, with various tweaks as we went > from CVS -> svn -> git. > > > > Maybe the problem here is that in the run up to 14.0 we didn't turn on > reproducible builds? > > You meant opposite, did you? > I don't think so. The quoted value is the old WITHOUT_REPRODUCIBLE_BUILDS format. We forgot to turn on WITH_REPRODUCIBLE_BUILDS in the branch before the release. Warner --00000000000028be68060e7aecb0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Mon, Jan 8, 2024 at 7:17=E2=80=AFP= M Eugene Grosbein <eugen@grosbein.= net> wrote:
09.01.2024 4:16, Warner Losh wrote:

> On Mon, Jan 8, 2024 at 1:58=E2=80=AFPM Eugene Grosbein <eugen@grosbein.net <mai= lto:eugen@grosbein.= net>> wrote:
>=C2=A0 =C2=A0 =C2=A0For ages, "uname -v" output (sligtly poli= shed sysctl kern.version) had the following format:
>=C2=A0 =C2=A0 =C2=A0$ uname -v
>=C2=A0 =C2=A0 =C2=A0FreeBSD 13.2-STABLE 36a037f15 KERNELIDENT
>
> This is the reproducible format: only include data that is identical f= rom build to build. It's relatively recent (FreeBSD 11 maybe)
>
>=C2=A0 =C2=A0 =C2=A0Where KERNELIDENT is GENERIC for x86 distribution m= edia.
>
>=C2=A0 =C2=A0 =C2=A0But now:
>
>=C2=A0 =C2=A0 =C2=A0# uname -v
>=C2=A0 =C2=A0 =C2=A0FreeBSD 14.0-RELEASE #0 releng/14.0-n265380-f9716ee= e8ab4: Fri Nov 10 05:51:26 UTC 2023=C2=A0 =C2=A0 =C2=A0root@releng1.nyi.fre= ebsd.org:/usr/obj/usr/src/i386.i386/sys/GENERIC
>
> This is the old, non-reproducible format. We've had both formats f= or several major releases, and this format, with various tweaks as we went = from CVS -> svn -> git.
>
> Maybe the problem here is that in the run up to 14.0 we didn't tur= n on reproducible builds?

You meant opposite, did you?

I don'= t think so. The quoted value is the old WITHOUT_REPRODUCIBLE_BUILDS format.= We forgot to turn on WITH_REPRODUCIBLE_BUILDS in the branch before the rel= ease.

Warner
--00000000000028be68060e7aecb0-- From nobody Tue Jan 9 13:18:46 2024 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T8WjF5h2Rz57SJl for ; Tue, 9 Jan 2024 13:18:57 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [IPv6:2607:f3e0:0:1::12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smarthost1.sentex.ca", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4T8WjF3hHjz4yh7; Tue, 9 Jan 2024 13:18:57 +0000 (UTC) (envelope-from mike@sentex.net) Authentication-Results: mx1.freebsd.org; none Received: from pyroxene2a.sentex.ca (pyroxene19.sentex.ca [199.212.134.19]) by smarthost1.sentex.ca (8.17.1/8.16.1) with ESMTPS id 409DIu0T039186 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=FAIL); Tue, 9 Jan 2024 08:18:56 -0500 (EST) (envelope-from mike@sentex.net) Received: from [IPV6:2607:f3e0:0:4:c7:b59d:3c25:d80a] ([IPv6:2607:f3e0:0:4:c7:b59d:3c25:d80a]) by pyroxene2a.sentex.ca (8.17.1/8.15.2) with ESMTPS id 409DIjm8015288 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Tue, 9 Jan 2024 08:18:55 -0500 (EST) (envelope-from mike@sentex.net) Content-Type: multipart/alternative; boundary="------------93SDc0twK8siL5B3tOvjiI6a" Message-ID: <04837a47-a39c-40c5-a045-2d27fb3893ac@sentex.net> Date: Tue, 9 Jan 2024 08:18:46 -0500 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: clang 17 and ports fallout Content-Language: en-US To: Dimitry Andric Cc: FreeBSD-STABLE Mailing List References: <06381879-c328-4051-bb2f-9c3620d90fa0@sentex.net> <4a0db68d-bb13-4238-a251-74d02ff14a17@sentex.net> From: mike tancsa Autocrypt: addr=mike@sentex.net; keydata= xsBNBFywzOMBCACoNFpwi5MeyEREiCeHtbm6pZJI/HnO+wXdCAWtZkS49weOoVyUj5BEXRZP xflV2ib2hflX4nXqhenaNiia4iaZ9ft3I1ebd7GEbGnsWCvAnob5MvDZyStDAuRxPJK1ya/s +6rOvr+eQiXYNVvfBhrCfrtR/esSkitBGxhUkBjOti8QwzD71JVF5YaOjBAs7jZUKyLGj0kW yDg4jUndudWU7G2yc9GwpHJ9aRSUN8e/mWdIogK0v+QBHfv/dsI6zVB7YuxCC9Fx8WPwfhDH VZC4kdYCQWKXrm7yb4TiVdBh5kgvlO9q3js1yYdfR1x8mjK2bH2RSv4bV3zkNmsDCIxjABEB AAHNHW1pa2UgdGFuY3NhIDxtaWtlQHNlbnRleC5uZXQ+wsCOBBMBCAA4FiEEmuvCXT0aY6hs 4SbWeVOEFl5WrMgFAl+pQfkCGwMFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQeVOEFl5W rMiN6ggAk3H5vk8QnbvGbb4sinxZt/wDetgk0AOR9NRmtTnPaW+sIJEfGBOz47Xih+f7uWJS j+uvc9Ewn2Z7n8z3ZHJlLAByLVLtcNXGoRIGJ27tevfOaNqgJHBPbFOcXCBBFTx4MYMM4iAZ cDT5vsBTSaM36JZFtHZBKkuFEItbA/N8ZQSHKdTYMIA7A3OCLGbJBqloQ8SlW4MkTzKX4u7R yefAYQ0h20x9IqC5Ju8IsYRFacVZconT16KS81IBceO42vXTN0VexbVF2rZIx3v/NT75r6Vw 0FlXVB1lXOHKydRA2NeleS4NEG2vWqy/9Boj0itMfNDlOhkrA/0DcCurMpnpbM7ATQRcsMzk AQgA1Dpo/xWS66MaOJLwA28sKNMwkEk1Yjs+okOXDOu1F+0qvgE8sVmrOOPvvWr4axtKRSG1 t2QUiZ/ZkW/x/+t0nrM39EANV1VncuQZ1ceIiwTJFqGZQ8kb0+BNkwuNVFHRgXm1qzAJweEt RdsCMohB+H7BL5LGCVG5JaU0lqFU9pFP40HxEbyzxjsZgSE8LwkI6wcu0BLv6K6cLm0EiHPO l5G8kgRi38PS7/6s3R8QDsEtbGsYy6O82k3zSLIjuDBwA9GRaeigGppTxzAHVjf5o9KKu4O7 gC2KKVHPegbXS+GK7DU0fjzX57H5bZ6komE5eY4p3oWT/CwVPSGfPs8jOwARAQABwsB2BBgB CAAgFiEEmuvCXT0aY6hs4SbWeVOEFl5WrMgFAl+pQfkCGwwACgkQeVOEFl5WrMiVqwf9GwU8 c6cylknZX8QwlsVudTC8xr/L17JA84wf03k3d4wxP7bqy5AYy7jboZMbgWXngAE/HPQU95NM aukysSnknzoIpC96XZJ0okLBXVS6Y0ylZQ+HrbIhMpuQPoDweoF5F9wKrsHRoDaUK1VR706X rwm4HUzh7Jk+auuMYfuCh0FVlFBEuiJWMLhg/5WCmcRfiuB6F59ZcUQrwLEZeNhF2XJV4KwB Tlg7HCWO/sy1foE5noaMyACjAtAQE9p5kGYaj+DuRhPdWUTsHNuqrhikzIZd2rrcMid+ktb0 NvtvswzMO059z1YGMtGSqQ4srCArju+XHIdTFdiIYbd7+jeehg== In-Reply-To: X-Scanned-By: MIMEDefang 2.84 on 64.7.153.18 X-Rspamd-Queue-Id: 4T8WjF3hHjz4yh7 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA] This is a multi-part message in MIME format. --------------93SDc0twK8siL5B3tOvjiI6a Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 1/8/2024 5:50 PM, Dimitry Andric wrote: > I fixed a lot of ports in the run-up to merging llvm-17 in 15-CURRENT, > but I could not get them all. > > The preferred way is fixing the port by removing the undefined symbols > from the linker version script in the port, but if that is not possible > or difficult, add -Wl,--undefined-version to the linker flags suppresses > the error. E.g. in the port Makefile: > > LDFLAGS+= -Wl,--undefined-version > > For an example, see: > > https://github.com/freebsd/freebsd-ports/commit/37790b26cbda11cd4bb6f237b86cd94739c4059c Thanks very much!  That did indeed fix databases/rrdtool and and sysutils/flashrom builds.   What is the best way to flag any such issues ? Just open a PR for each individual port ?     ---Mike --------------93SDc0twK8siL5B3tOvjiI6a Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit
On 1/8/2024 5:50 PM, Dimitry Andric wrote:
I fixed a lot of ports in the run-up to merging llvm-17 in 15-CURRENT,
but I could not get them all.

The preferred way is fixing the port by removing the undefined symbols
from the linker version script in the port, but if that is not possible
or difficult, add -Wl,--undefined-version to the linker flags suppresses
the error. E.g. in the port Makefile:

LDFLAGS+=	-Wl,--undefined-version

For an example, see:

https://github.com/freebsd/freebsd-ports/commit/37790b26cbda11cd4bb6f237b86cd94739c4059c

Thanks very much!  That did indeed fix databases/rrdtool and and sysutils/flashrom builds.   What is the best way to flag any such issues ? Just open a PR for each individual port ?

    ---Mike



    

  


--------------93SDc0twK8siL5B3tOvjiI6a--

From nobody Tue Jan  9 17:29:35 2024
X-Original-To: stable@mlmmj.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
	by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T8dGj19F5z560JW
	for ; Tue,  9 Jan 2024 17:29:49 +0000 (UTC)
	(envelope-from greg.bal4@gmail.com)
Received: from mail-ua1-x92f.google.com (mail-ua1-x92f.google.com [IPv6:2607:f8b0:4864:20::92f])
	(using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256
	 client-signature RSA-PSS (2048 bits) client-digest SHA256)
	(Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK))
	by mx1.freebsd.org (Postfix) with ESMTPS id 4T8dGh3G2Jz4K7K
	for ; Tue,  9 Jan 2024 17:29:48 +0000 (UTC)
	(envelope-from greg.bal4@gmail.com)
Authentication-Results: mx1.freebsd.org;
	dkim=pass header.d=gmail.com header.s=20230601 header.b=Ps1PTnwO;
	dmarc=pass (policy=none) header.from=gmail.com;
	spf=pass (mx1.freebsd.org: domain of greg.bal4@gmail.com designates 2607:f8b0:4864:20::92f as permitted sender) smtp.mailfrom=greg.bal4@gmail.com
Received: by mail-ua1-x92f.google.com with SMTP id a1e0cc1a2514c-7ce363b3772so393165241.2
        for ; Tue, 09 Jan 2024 09:29:48 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1704821387; x=1705426187; darn=freebsd.org;
        h=to:subject:message-id:date:from:mime-version:from:to:cc:subject
         :date:message-id:reply-to;
        bh=p1LfWB2wwdyFphsByMaVpQII6ZR+vbpR4fElbThEJfE=;
        b=Ps1PTnwOX8J11VPW2uSL+E6hoNbcjIn2GuUPkeKFI2/nriQX+IdCZVmGyjUGf7Bmxo
         MYW+/7twfhuDKEJTfu4k/GNKJngysJgBCimQpJPvlVwBiiyja+SI8yBtWuNohcWHEBGO
         eGdD9/j5yzciBuqu1o74AFw5JMLyuMOy0Dq83zmp2Xymdy45qgw3CWjryMbKL0u0WP6r
         cVakJK60LZtyhze/v/RQCOBMmNiu5cNRIUoDatQWBzZ6IWpzbUr6LY+yMuKqqxSua4+o
         G2hMmHarNnyO3FX8/Q13z9yWSBO/lX5E0Qq8d8Q4Nmq7/DEvJhSaZxBGdbhG58DYU1zl
         hikQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1704821387; x=1705426187;
        h=to:subject:message-id:date:from:mime-version:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=p1LfWB2wwdyFphsByMaVpQII6ZR+vbpR4fElbThEJfE=;
        b=JrvF3PolR8GiGeMx0d/id8NxAPHDqXBPXkLE4iS6Qimn/5sqfjidZlViD2afDcTzIe
         InuF6DpQ4RToCtF7o6FNHxqa9D75dOM7FScpVopXX1D+mkgqoxSKIoxXeXBV9qfV4FbP
         0SakkGTzS83EKw3Qotj+xZp/05yg31WbuLju8MEBAS/TpiA0gb7dOV8zg5PTzoWDxMiW
         0fQ2VjPzdqU+3zTlOEewpLVCXIhrAMty6Ey2lvHjNXJdqJVpyfHoos2JO51ncX+tnHKi
         KjVO6QW1y52/M+qmITmZf0AdtrO89P1XiGzx9UO+DDso6SVQ5sQQ/XPwQQqVCbeU2TC6
         PqJg==
X-Gm-Message-State: AOJu0Yx5N6ImmrznvLvHPXlyUL3y+GqsO3LRKEdak3oskpc7+QcVQqez
	SSPucfIUlEggB4naVc5JnKMl84/CM2vJ+hSHR9w+wlorUho=
X-Google-Smtp-Source: AGHT+IHjiNu1w+9Lzcg1hW8hDr7i5w8Nn9ckAhcjgFKYNoBGJQM0v3BdNOvPo66KioNhLLPRIqdnRxquSi6GxXgMtXY=
X-Received: by 2002:a05:6122:4585:b0:4b7:3eee:3a4a with SMTP id
 de5-20020a056122458500b004b73eee3a4amr2079302vkb.28.1704821386907; Tue, 09
 Jan 2024 09:29:46 -0800 (PST)
List-Id: Production branch of FreeBSD source code 
List-Archive: https://lists.freebsd.org/archives/freebsd-stable
List-Help: 
List-Post: 
List-Subscribe: 
List-Unsubscribe: 
Sender: owner-freebsd-stable@freebsd.org
X-BeenThere: freebsd-stable@freebsd.org
MIME-Version: 1.0
From: Greg Balfour 
Date: Tue, 9 Jan 2024 11:29:35 -0600
Message-ID: 
Subject: LinuxKPI fix for 13.3
To: stable@freebsd.org
Content-Type: multipart/alternative; boundary="0000000000002c1823060e86a956"
X-Spamd-Bar: ---
X-Spamd-Result: default: False [-3.94 / 15.00];
	NEURAL_HAM_MEDIUM(-1.00)[-1.000];
	NEURAL_HAM_SHORT(-0.98)[-0.978];
	NEURAL_HAM_LONG(-0.96)[-0.959];
	DMARC_POLICY_ALLOW(-0.50)[gmail.com,none];
	R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36];
	R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601];
	MIME_GOOD(-0.10)[multipart/alternative,text/plain];
	RCVD_TLS_LAST(0.00)[];
	TO_MATCH_ENVRCPT_ALL(0.00)[];
	ARC_NA(0.00)[];
	RCPT_COUNT_ONE(0.00)[1];
	DWL_DNSWL_NONE(0.00)[gmail.com:dkim];
	TAGGED_FROM(0.00)[];
	FREEMAIL_FROM(0.00)[gmail.com];
	MIME_TRACE(0.00)[0:+,1:+,2:~];
	FROM_HAS_DN(0.00)[];
	MISSING_XM_UA(0.00)[];
	FREEMAIL_ENVFROM(0.00)[gmail.com];
	PREVIOUSLY_DELIVERED(0.00)[stable@freebsd.org];
	TO_DN_NONE(0.00)[];
	FROM_EQ_ENVFROM(0.00)[];
	DKIM_TRACE(0.00)[gmail.com:+];
	MID_RHS_MATCH_FROMTLD(0.00)[];
	ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US];
	MLMMJ_DEST(0.00)[stable@freebsd.org];
	RCVD_COUNT_ONE(0.00)[1];
	RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::92f:from]
X-Rspamd-Queue-Id: 4T8dGh3G2Jz4K7K

--0000000000002c1823060e86a956
Content-Type: text/plain; charset="UTF-8"

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271333

Can this be merged to 13 for the upcoming 13.3 release?  I manually
applied it to my 13.2 system a few weeks ago and haven't seen a
crash since.  Was seeing the occasional crash without it.

--0000000000002c1823060e86a956
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable


https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D271333=


Can this be merged to 13 for the upcoming 13.3 release?=C2=A0 I man=
ually
applied it to my 13.2 system a few weeks ago and haven't seen =
a
crash since.=C2=A0 Was seeing the occasional crash without it.


--0000000000002c1823060e86a956--

From nobody Tue Jan  9 18:00:18 2024
X-Original-To: stable@mlmmj.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
	by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T8dy10h8Sz565Q4
	for ; Tue,  9 Jan 2024 18:00:25 +0000 (UTC)
	(envelope-from markjdb@gmail.com)
Received: from mail-yb1-xb29.google.com (mail-yb1-xb29.google.com [IPv6:2607:f8b0:4864:20::b29])
	(using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256
	 client-signature RSA-PSS (2048 bits) client-digest SHA256)
	(Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK))
	by mx1.freebsd.org (Postfix) with ESMTPS id 4T8dxy74Slz4Q1L
	for ; Tue,  9 Jan 2024 18:00:22 +0000 (UTC)
	(envelope-from markjdb@gmail.com)
Authentication-Results: mx1.freebsd.org;
	dkim=pass header.d=gmail.com header.s=20230601 header.b=iuPx96eo;
	dmarc=none;
	spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates 2607:f8b0:4864:20::b29 as permitted sender) smtp.mailfrom=markjdb@gmail.com
Received: by mail-yb1-xb29.google.com with SMTP id 3f1490d57ef6-dbeff495c16so1811398276.3
        for ; Tue, 09 Jan 2024 10:00:22 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1704823222; x=1705428022; darn=freebsd.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:sender:from:to:cc:subject:date:message-id
         :reply-to;
        bh=mA2Vkrkxx0iegTQJEsx6ldQRACSYhAVJLdJLujeb8pg=;
        b=iuPx96eom0wn6bKxMrH9cBAYKAeE+wQNympKVSjV2Ts6bCwnmPwhea68E6/gVglUKd
         /VulonMoWqOKXRciWyuCg4RISEI99M1Brjhs67DZIMyj9YTbQeQg6nuoa9EYD9IjvDbs
         rT7Zw85DnaQO6UgMrjuz/Idsy0VFXKDtomGTqrrpRWeajaW4ESiEhJd3I0I55DV7g9xq
         4vxu2mSH3AduU0hwfW/fm9DdaNkLIF5HvZf0vs+7GjtvK9FXJK7j9//6y9fVj/+kCv2d
         vDnr+/0bU+hwitTZ4PxPA18d/W5Hs9KxrO90Dj4PGKhIGg3MslX1lNq2jt0DTI7fFruV
         nFwg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1704823222; x=1705428022;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:sender:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=mA2Vkrkxx0iegTQJEsx6ldQRACSYhAVJLdJLujeb8pg=;
        b=CO8t4Bv6SA4wp/C50Cu9BDjC+/XntnH4yPrG/wBjaUBmDmDO64VHJ27v2YcoqftGLN
         mPuxrlEdjU3wpZc3D1sjKa4Jd0VkpjFsriZGhtMeCCqLrejQsZRe6KIvnKlxv5zwcHsX
         IttBoNhPvc5IQTSo2bEQmb0Tg3sU/fntr44dimhUkby656V+ks/k9DUTJSCB7/a9ApSK
         IAJ+SeLJRpYZyobZXDSASxJ5Ov61B5TYlFJ2yYE20yEr0/VZAa/B1w1fWp+WKeD39CRk
         dGO4SbPkNXNc+HruUz5BymWHukiVBCx4h8/YU18hrGwTXLHJnrxfguIcFsZysWJbH/Te
         ZyAA==
X-Gm-Message-State: AOJu0YwahQMMwpHV1KmUEwXKT+tRNyD/un1qYOZIBjmVauNK+Kg1rz2A
	BKEVakqiVUIRuryu6dSXgFY=
X-Google-Smtp-Source: AGHT+IG3G8EPCg60rI0OHrkX6V4XCzCfjohxgyy2nIHDVh2SqmNCLvsK1NZ75KOWI04itS13zwdUUg==
X-Received: by 2002:a5b:888:0:b0:dbd:bd00:809c with SMTP id e8-20020a5b0888000000b00dbdbd00809cmr3421101ybq.6.1704823221117;
        Tue, 09 Jan 2024 10:00:21 -0800 (PST)
Received: from nuc (192-0-220-237.cpe.teksavvy.com. [192.0.220.237])
        by smtp.gmail.com with ESMTPSA id z4-20020ac875c4000000b00427f8c50f31sm1060590qtq.46.2024.01.09.10.00.20
        (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
        Tue, 09 Jan 2024 10:00:20 -0800 (PST)
Date: Tue, 9 Jan 2024 13:00:18 -0500
From: Mark Johnston 
To: Greg Balfour 
Cc: stable@freebsd.org
Subject: Re: LinuxKPI fix for 13.3
Message-ID: 
References: 
List-Id: Production branch of FreeBSD source code 
List-Archive: https://lists.freebsd.org/archives/freebsd-stable
List-Help: 
List-Post: 
List-Subscribe: 
List-Unsubscribe: 
Sender: owner-freebsd-stable@freebsd.org
X-BeenThere: freebsd-stable@freebsd.org
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
X-Spamd-Bar: --
X-Spamd-Result: default: False [-2.70 / 15.00];
	NEURAL_HAM_MEDIUM(-1.00)[-1.000];
	NEURAL_HAM_LONG(-1.00)[-1.000];
	NEURAL_HAM_SHORT(-1.00)[-1.000];
	MID_RHS_NOT_FQDN(0.50)[];
	FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com];
	R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36];
	R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601];
	MIME_GOOD(-0.10)[text/plain];
	RCPT_COUNT_TWO(0.00)[2];
	DMARC_NA(0.00)[freebsd.org];
	MIME_TRACE(0.00)[0:+];
	ARC_NA(0.00)[];
	FREEMAIL_ENVFROM(0.00)[gmail.com];
	FREEMAIL_TO(0.00)[gmail.com];
	TO_DN_SOME(0.00)[];
	RCVD_TLS_LAST(0.00)[];
	FROM_HAS_DN(0.00)[];
	DWL_DNSWL_NONE(0.00)[gmail.com:dkim];
	TO_MATCH_ENVRCPT_SOME(0.00)[];
	PREVIOUSLY_DELIVERED(0.00)[stable@freebsd.org];
	RCVD_COUNT_TWO(0.00)[2];
	FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com];
	DKIM_TRACE(0.00)[gmail.com:+];
	MLMMJ_DEST(0.00)[stable@freebsd.org];
	RCVD_VIA_SMTP_AUTH(0.00)[];
	TAGGED_RCPT(0.00)[];
	MISSING_XM_UA(0.00)[];
	ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US];
	RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::b29:from]
X-Rspamd-Queue-Id: 4T8dxy74Slz4Q1L

On Tue, Jan 09, 2024 at 11:29:35AM -0600, Greg Balfour wrote:
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271333
> 
> Can this be merged to 13 for the upcoming 13.3 release?  I manually
> applied it to my 13.2 system a few weeks ago and haven't seen a
> crash since.  Was seeing the occasional crash without it.

Certainly.  The patch is now in stable/13, so it'll appear in 13.3.

From nobody Tue Jan  9 20:23:36 2024
X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
	by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T8j7G3cM2z56VMt
	for ; Tue,  9 Jan 2024 20:23:38 +0000 (UTC)
	(envelope-from henrichhartzer@tuta.io)
Received: from w1.tutanota.de (w1.tutanota.de [81.3.6.162])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits)
	 client-signature RSA-PSS (2048 bits))
	(Client CN "mail.tutanota.de", Issuer "Sectigo RSA Domain Validation Secure Server CA" (verified OK))
	by mx1.freebsd.org (Postfix) with ESMTPS id 4T8j7G0z7Wz4pdJ;
	Tue,  9 Jan 2024 20:23:38 +0000 (UTC)
	(envelope-from henrichhartzer@tuta.io)
Authentication-Results: mx1.freebsd.org;
	none
Received: from tutadb.w10.tutanota.de (unknown [192.168.1.10])
	by w1.tutanota.de (Postfix) with ESMTP id E4921FBFB43;
	Tue,  9 Jan 2024 20:23:36 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1704831816;
	s=s1; d=tuta.io;
	h=From:From:To:To:Subject:Subject:Content-Description:Content-ID:Content-Type:Content-Type:Content-Transfer-Encoding:Content-Transfer-Encoding:Cc:Cc:Date:Date:In-Reply-To:In-Reply-To:MIME-Version:MIME-Version:Message-ID:Message-ID:Reply-To:References:References:Sender;
	bh=paScRL0DLie7Ea7HAxcIB3cyYmHB1+c28hbthKqU9S0=;
	b=Lsyscwi/Gk9xYFAPDEPmw/6Bt0ZqU/D4Imz1qk7HKFw3KG90xnqPjqoTNwCpKn0j
	RE1tQ49LoJD7VRHuy9N8WMjDQirSeXPxEWVAohdpXmJUxM9AzTnC2qXc+tXQpWQantt
	CH5DkXaA0eccRlXCg+OAyW1rcgCrJtf0lyxF1hG3Wpy+r0Qfd2tSzbskcx+v8Et1z3c
	u6MzGsLtDBWx56392aYoBeYJIl0/SDs3nN7hntzrGv4qtNCB3H96ZnzX4Q+hKE5hluA
	vSSMyi/X0zN2CmIi9vxSJlZ7l08rvgJy3cvlViMDSw/YEZtb4U3xHRTkEZV+d/4IQbw
	OHuBXIoezw==
Date: Tue, 9 Jan 2024 21:23:36 +0100 (CET)
From: henrichhartzer@tuta.io
To: Warner Losh 
Cc: Eugene Grosbein ,
	FreeBSD stable ,
	FreeBSD Release Engineering Team ,
	Freebsd Reproducibility 
Message-ID: 
In-Reply-To: 
References:    
Subject: Re: kern.version and uname -v
List-Id: Production branch of FreeBSD source code 
List-Archive: https://lists.freebsd.org/archives/freebsd-stable
List-Help: 
List-Post: 
List-Subscribe: 
List-Unsubscribe: 
Sender: owner-freebsd-stable@freebsd.org
X-BeenThere: freebsd-stable@freebsd.org
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Rspamd-Queue-Id: 4T8j7G0z7Wz4pdJ
X-Spamd-Bar: ----
X-Rspamd-Pre-Result: action=no action;
	module=replies;
	Message is reply to one we originated
X-Spamd-Result: default: False [-4.00 / 15.00];
	REPLY(-4.00)[];
	ASN(0.00)[asn:24679, ipnet:81.3.0.0/18, country:DE]

I personally feel that building releases reproducibly is quite important. H=
ow can we trust the current 14.0-RELEASE images? Would be nice to be able t=
o build them locally and know they are the same.

What do we need to do to make sure that 14.1-RELEASE is built reproducibly?

Thanks!

PS: It looks like freebsd-reproducibility@ may have some issues?


Jan 9, 2024, 03:29 by imp@bsdimp.com:

>
>
> On Mon, Jan 8, 2024 at 7:17=E2=80=AFPM Eugene Grosbein <> eugen@grosbein.=
net> > wrote:
>
>> 09.01.2024 4:16, Warner Losh wrote:
>>
>> > On Mon, Jan 8, 2024 at 1:58=E2=80=AFPM Eugene Grosbein <>> eugen@grosb=
ein.net>>  > eugen@grosbein.net>> >> wrote:
>> >=C2=A0 =C2=A0 =C2=A0For ages, "uname -v" output (sligtly polished sysct=
l kern.version) had the following format:
>> >=C2=A0 =C2=A0 =C2=A0$ uname -v
>> >=C2=A0 =C2=A0 =C2=A0FreeBSD 13.2-STABLE 36a037f15 KERNELIDENT
>> >
>> > This is the reproducible format: only include data that is identical f=
rom build to build. It's relatively recent (FreeBSD 11 maybe)
>> >
>> >=C2=A0 =C2=A0 =C2=A0Where KERNELIDENT is GENERIC for x86 distribution m=
edia.
>> >
>> >=C2=A0 =C2=A0 =C2=A0But now:
>> >
>> >=C2=A0 =C2=A0 =C2=A0# uname -v
>> >=C2=A0 =C2=A0 =C2=A0FreeBSD 14.0-RELEASE #0 releng/14.0-n265380-f9716ee=
e8ab4: Fri Nov 10 05:51:26 UTC 2023=C2=A0 =C2=A0 =C2=A0root@releng1.nyi.fre=
ebsd.org:/usr/obj/usr/src/i386.i386/sys/GENERIC
>> >
>> > This is the old, non-reproducible format. We've had both formats for s=
everal major releases, and this format, with various tweaks as we went from=
 CVS -> svn -> git.
>> >
>> > Maybe the problem here is that in the run up to 14.0 we didn't turn on=
 reproducible builds?
>>
>> You meant opposite, did you?
>>
>
> I don't think so. The quoted value is the old WITHOUT_REPRODUCIBLE_BUILDS=
 format. We forgot to turn on WITH_REPRODUCIBLE_BUILDS in the branch before=
 the release.
>
> Warner
>


From nobody Wed Jan 10 07:53:38 2024
X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
	by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T90Rj2pd7z56LZs
	for ; Wed, 10 Jan 2024 07:53:53 +0000 (UTC)
	(envelope-from antoine.brodin.freebsd@gmail.com)
Received: from mail-vk1-f169.google.com (mail-vk1-f169.google.com [209.85.221.169])
	(using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256
	 client-signature RSA-PSS (2048 bits) client-digest SHA256)
	(Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK))
	by mx1.freebsd.org (Postfix) with ESMTPS id 4T90Rh5pB8z4sZG;
	Wed, 10 Jan 2024 07:53:52 +0000 (UTC)
	(envelope-from antoine.brodin.freebsd@gmail.com)
Authentication-Results: mx1.freebsd.org;
	none
Received: by mail-vk1-f169.google.com with SMTP id 71dfb90a1353d-4b743ca0597so826107e0c.1;
        Tue, 09 Jan 2024 23:53:52 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1704873231; x=1705478031;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=oigOSO14k3tH3P1XQB4vgM6LK+1h7g4Aav4rFW0gS6U=;
        b=ca6scBGL+SZilktvBj40VDPc05I81VDzsSZ5LMpbG9dZiT0LhuO5UIkQgw9LPthUGR
         vieB4XaS21hGruNyWSr5FCnq208C+T5+lu0cQl4oU4V1RZdIsSCvhRx34lXzX8bGUVdV
         lWXC6lwYxfXs2+w0jOU+wAgZehErdJ1qNfdNreXQ/sgXCHxYqUXlEfOXceEEJcF5CF/m
         9DWY1fp7eeDNUSXPYpGWqiaUc7hInuzgnyGa27pj4c8qVYFqC8VAgSV+ZPvI/jpxEylj
         GyXliyf7fDqO/ZRqaYNKqgPSSGFzcG+jW8EAg+kAG6akHmWALdz0DO3+m6qII3v7fbkR
         xNnQ==
X-Gm-Message-State: AOJu0YzDblOe/oTMOMnfF9DjlF3kVY4qlyTprpKXlVHoczWBuPdSfg/E
	n0yWMskHrgrikAgK23tl+VTH+j0dm/m1ruSDuQc=
X-Google-Smtp-Source: AGHT+IHcTo5t9C28qu5/HfYMB9n7YBb2QgFZH7oHjdBhDv1QVcUvxnZO/D/OHqH3w2mNJ+V1HnbO8YAsWZmuFkDFXAI=
X-Received: by 2002:a05:6122:305:b0:4b6:d9d4:7516 with SMTP id
 c5-20020a056122030500b004b6d9d47516mr283363vko.17.1704873231441; Tue, 09 Jan
 2024 23:53:51 -0800 (PST)
List-Id: Production branch of FreeBSD source code 
List-Archive: https://lists.freebsd.org/archives/freebsd-stable
List-Help: 
List-Post: 
List-Subscribe: 
List-Unsubscribe: 
Sender: owner-freebsd-stable@freebsd.org
X-BeenThere: freebsd-stable@freebsd.org
MIME-Version: 1.0
References: <06381879-c328-4051-bb2f-9c3620d90fa0@sentex.net>
 <4a0db68d-bb13-4238-a251-74d02ff14a17@sentex.net> 
 <04837a47-a39c-40c5-a045-2d27fb3893ac@sentex.net>
In-Reply-To: <04837a47-a39c-40c5-a045-2d27fb3893ac@sentex.net>
From: Antoine Brodin 
Date: Wed, 10 Jan 2024 08:53:38 +0100
Message-ID: 
Subject: Re: clang 17 and ports fallout
To: mike tancsa 
Cc: Dimitry Andric , 
	FreeBSD-STABLE Mailing List 
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Rspamd-Queue-Id: 4T90Rh5pB8z4sZG
X-Spamd-Bar: ----
X-Rspamd-Pre-Result: action=no action;
	module=replies;
	Message is reply to one we originated
X-Spamd-Result: default: False [-4.00 / 15.00];
	REPLY(-4.00)[];
	TAGGED_FROM(0.00)[];
	ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]

On Tue, Jan 9, 2024 at 2:19=E2=80=AFPM mike tancsa  wrote:
>
> On 1/8/2024 5:50 PM, Dimitry Andric wrote:
>
> I fixed a lot of ports in the run-up to merging llvm-17 in 15-CURRENT,
> but I could not get them all.
>
> The preferred way is fixing the port by removing the undefined symbols
> from the linker version script in the port, but if that is not possible
> or difficult, add -Wl,--undefined-version to the linker flags suppresses
> the error. E.g. in the port Makefile:
>
> LDFLAGS+=3D -Wl,--undefined-version
>
> For an example, see:
>
> https://github.com/freebsd/freebsd-ports/commit/37790b26cbda11cd4bb6f237b=
86cd94739c4059c
>
> Thanks very much!  That did indeed fix databases/rrdtool and and sysutils=
/flashrom builds.   What is the best way to flag any such issues ? Just ope=
n a PR for each individual port ?

Hello,

databases/rrdtool builds fine on stable/13 here:
https://pkg-status.freebsd.org/gohan04/data/stable13amd64-default-foo/2024-=
01-09_21h18m24s/logs/rrdtool-1.8.0_2.log

Antoine

From nobody Wed Jan 10 12:16:51 2024
X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
	by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T96HL3Txyz56pPY
	for ; Wed, 10 Jan 2024 12:17:02 +0000 (UTC)
	(envelope-from mike@sentex.net)
Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [IPv6:2607:f3e0:0:1::12])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256
	 client-signature RSA-PSS (2048 bits) client-digest SHA256)
	(Client CN "smarthost1.sentex.ca", Issuer "R3" (verified OK))
	by mx1.freebsd.org (Postfix) with ESMTPS id 4T96HK6d9sz4WXR;
	Wed, 10 Jan 2024 12:17:01 +0000 (UTC)
	(envelope-from mike@sentex.net)
Authentication-Results: mx1.freebsd.org;
	none
Received: from pyroxene2a.sentex.ca (pyroxene19.sentex.ca [199.212.134.19])
	by smarthost1.sentex.ca (8.17.1/8.16.1) with ESMTPS id 40ACGsmi083813
	(version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=FAIL);
	Wed, 10 Jan 2024 07:16:54 -0500 (EST)
	(envelope-from mike@sentex.net)
Received: from [IPV6:2607:f3e0:0:4:d1e0:fd54:1ff2:1fb7] ([IPv6:2607:f3e0:0:4:d1e0:fd54:1ff2:1fb7])
	by pyroxene2a.sentex.ca (8.17.1/8.15.2) with ESMTPS id 40ACGpAN041668
	(version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO);
	Wed, 10 Jan 2024 07:16:53 -0500 (EST)
	(envelope-from mike@sentex.net)
Content-Type: multipart/alternative;
 boundary="------------TmukDo5Aa3rYMdeA1hJHI5Da"
Message-ID: <20ceec23-fe90-41de-a3d8-82eada0d56e7@sentex.net>
Date: Wed, 10 Jan 2024 07:16:51 -0500
List-Id: Production branch of FreeBSD source code 
List-Archive: https://lists.freebsd.org/archives/freebsd-stable
List-Help: 
List-Post: 
List-Subscribe: 
List-Unsubscribe: 
Sender: owner-freebsd-stable@freebsd.org
X-BeenThere: freebsd-stable@freebsd.org
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: clang 17 and ports fallout
To: Antoine Brodin 
Cc: Dimitry Andric ,
        FreeBSD-STABLE Mailing List 
References: <06381879-c328-4051-bb2f-9c3620d90fa0@sentex.net>
 <4a0db68d-bb13-4238-a251-74d02ff14a17@sentex.net>
 
 <04837a47-a39c-40c5-a045-2d27fb3893ac@sentex.net>
 
Content-Language: en-US
From: mike tancsa 
Autocrypt: addr=mike@sentex.net; keydata=
 xsBNBFywzOMBCACoNFpwi5MeyEREiCeHtbm6pZJI/HnO+wXdCAWtZkS49weOoVyUj5BEXRZP
 xflV2ib2hflX4nXqhenaNiia4iaZ9ft3I1ebd7GEbGnsWCvAnob5MvDZyStDAuRxPJK1ya/s
 +6rOvr+eQiXYNVvfBhrCfrtR/esSkitBGxhUkBjOti8QwzD71JVF5YaOjBAs7jZUKyLGj0kW
 yDg4jUndudWU7G2yc9GwpHJ9aRSUN8e/mWdIogK0v+QBHfv/dsI6zVB7YuxCC9Fx8WPwfhDH
 VZC4kdYCQWKXrm7yb4TiVdBh5kgvlO9q3js1yYdfR1x8mjK2bH2RSv4bV3zkNmsDCIxjABEB
 AAHNHW1pa2UgdGFuY3NhIDxtaWtlQHNlbnRleC5uZXQ+wsCOBBMBCAA4FiEEmuvCXT0aY6hs
 4SbWeVOEFl5WrMgFAl+pQfkCGwMFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQeVOEFl5W
 rMiN6ggAk3H5vk8QnbvGbb4sinxZt/wDetgk0AOR9NRmtTnPaW+sIJEfGBOz47Xih+f7uWJS
 j+uvc9Ewn2Z7n8z3ZHJlLAByLVLtcNXGoRIGJ27tevfOaNqgJHBPbFOcXCBBFTx4MYMM4iAZ
 cDT5vsBTSaM36JZFtHZBKkuFEItbA/N8ZQSHKdTYMIA7A3OCLGbJBqloQ8SlW4MkTzKX4u7R
 yefAYQ0h20x9IqC5Ju8IsYRFacVZconT16KS81IBceO42vXTN0VexbVF2rZIx3v/NT75r6Vw
 0FlXVB1lXOHKydRA2NeleS4NEG2vWqy/9Boj0itMfNDlOhkrA/0DcCurMpnpbM7ATQRcsMzk
 AQgA1Dpo/xWS66MaOJLwA28sKNMwkEk1Yjs+okOXDOu1F+0qvgE8sVmrOOPvvWr4axtKRSG1
 t2QUiZ/ZkW/x/+t0nrM39EANV1VncuQZ1ceIiwTJFqGZQ8kb0+BNkwuNVFHRgXm1qzAJweEt
 RdsCMohB+H7BL5LGCVG5JaU0lqFU9pFP40HxEbyzxjsZgSE8LwkI6wcu0BLv6K6cLm0EiHPO
 l5G8kgRi38PS7/6s3R8QDsEtbGsYy6O82k3zSLIjuDBwA9GRaeigGppTxzAHVjf5o9KKu4O7
 gC2KKVHPegbXS+GK7DU0fjzX57H5bZ6komE5eY4p3oWT/CwVPSGfPs8jOwARAQABwsB2BBgB
 CAAgFiEEmuvCXT0aY6hs4SbWeVOEFl5WrMgFAl+pQfkCGwwACgkQeVOEFl5WrMiVqwf9GwU8
 c6cylknZX8QwlsVudTC8xr/L17JA84wf03k3d4wxP7bqy5AYy7jboZMbgWXngAE/HPQU95NM
 aukysSnknzoIpC96XZJ0okLBXVS6Y0ylZQ+HrbIhMpuQPoDweoF5F9wKrsHRoDaUK1VR706X
 rwm4HUzh7Jk+auuMYfuCh0FVlFBEuiJWMLhg/5WCmcRfiuB6F59ZcUQrwLEZeNhF2XJV4KwB
 Tlg7HCWO/sy1foE5noaMyACjAtAQE9p5kGYaj+DuRhPdWUTsHNuqrhikzIZd2rrcMid+ktb0
 NvtvswzMO059z1YGMtGSqQ4srCArju+XHIdTFdiIYbd7+jeehg==
In-Reply-To: 
X-Scanned-By: MIMEDefang 2.84 on 64.7.153.18
X-Rspamd-Queue-Id: 4T96HK6d9sz4WXR
X-Spamd-Bar: ----
X-Rspamd-Pre-Result: action=no action;
	module=replies;
	Message is reply to one we originated
X-Spamd-Result: default: False [-4.00 / 15.00];
	REPLY(-4.00)[];
	ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA]

This is a multi-part message in MIME format.
--------------TmukDo5Aa3rYMdeA1hJHI5Da
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 1/10/2024 2:53 AM, Antoine Brodin wrote:
> The preferred way is fixing the port by removing the undefined symbols
>> from the linker version script in the port, but if that is not possible
>> or difficult, add -Wl,--undefined-version to the linker flags suppresses
>> the error. E.g. in the port Makefile:
>>
>> LDFLAGS+= -Wl,--undefined-version
>>
>> For an example, see:
>>
>> https://github.com/freebsd/freebsd-ports/commit/37790b26cbda11cd4bb6f237b86cd94739c4059c
>>
>> Thanks very much!  That did indeed fix databases/rrdtool and and sysutils/flashrom builds.   What is the best way to flag any such issues ? Just open a PR for each individual port ?
> Hello,
>
> databases/rrdtool builds fine on stable/13 here:
> https://pkg-status.freebsd.org/gohan04/data/stable13amd64-default-foo/2024-01-09_21h18m24s/logs/rrdtool-1.8.0_2.log
>
I am not sure why it fails for me both in poudrier and outside it. But 
the same type of error as

https://pkg-status.freebsd.org/gohan04/data/stable13amd64-default-foo/2024-01-09_21h18m24s/logs/errors/ivykis-0.42.4.log

and adding

LDFLAGS+= -Wl,--undefined-version

fixes it for me

I just checked on releng_14 that also has clang 17 MFCd and I get the 
same error on the port. I am building it with less features than the 
default so not sure if that makes it fail in my environment. However the 
same options work with clang16

/usr/local/lib  -L/usr/local/lib  -o rrdupdate rrdupdate.o librrdupd.la
libtool: link: cc -O2 -pipe -fstack-protector-strong -isystem 
/usr/local/include -fno-strict-aliasing -D_GNU_SOURCE 
-fno-strict-aliasing -Wall -std=gnu99 -pedantic -Wundef -Wshadow 
-Wpointer-arith -Wcast-align -Wmissing-prototypes -Wmissing-declarations 
-Wnested-externs -Winline -Wold-style-definition -W -I.. -O2 -pipe 
-fstack-protector-strong -isystem /usr/local/include 
-fno-strict-aliasing -D_GNU_SOURCE -fno-strict-aliasing -Wall -std=gnu99 
-pedantic -Wundef -Wshadow -Wpointer-arith -Wcast-align 
-Wmissing-prototypes -Wmissing-declarations -Wnested-externs -Winline 
-Wold-style-definition -W -fstack-protector-strong -o rrdupdate 
rrdupdate.o  -L/usr/local/lib ./.libs/librrdupd.a -lm -lwrap -lglib-2.0 
-lintl -pthread
libtool: link: echo "{ global:" > .libs/librrd.so.8.3.0-ver
libtool: link:           sed -e "s|$|;|" < ./librrd.sym >> 
.libs/librrd.so.8.3.0-ver
libtool: link:   echo "local: *; };" >> .libs/librrd.so.8.3.0-ver
libtool: link: cc -shared  -fPIC -DPIC .libs/librrd_la-rrd_version.o 
.libs/librrd_la-rrd_last.o .libs/librrd_la-rrd_lastupdate.o 
.libs/librrd_la-rrd_first.o .libs/librrd_la-rrd_dump.o 
.libs/librrd_la-rrd_flushcached.o .libs/librrd_la-rrd_fetch.o 
.libs/librrd_la-rrd_fetch_cb.o .libs/librrd_la-rrd_resize.o 
.libs/librrd_la-rrd_tune.o .libs/librrd_la-rrd_list.o 
.libs/librrd_la-rrd_restore.o -Wl,--whole-archive ./.libs/librrdupd.a 
-Wl,--no-whole-archive -L/usr/local/lib -lglib-2.0 -lintl -lm -lwrap 
-lxml2  -O2 -fstack-protector-strong -pthread -O2 
-fstack-protector-strong -fstack-protector-strong -Wl,-rpath 
-Wl,/usr/local/lib   -pthread -Wl,-soname -Wl,librrd.so.8 
-Wl,-version-script -Wl,.libs/librrd.so.8.3.0-ver -o .libs/librrd.so.8.3.0
ld: error: version script assignment of 'global' to symbol 'rrd_graph' 
failed: symbol not defined
ld: error: version script assignment of 'global' to symbol 'rrd_graph_v' 
failed: symbol not defined
ld: error: version script assignment of 'global' to symbol 'rrd_lcd' 
failed: symbol not defined
ld: error: version script assignment of 'global' to symbol 
'rrd_reduce_data' failed: symbol not defined
ld: error: version script assignment of 'global' to symbol 'rrd_xport' 
failed: symbol not defined
cc: error: linker command failed with exit code 1 (use -v to see invocation)
gmake[4]: *** [Makefile:773: librrd.la] Error 1
gmake[4]: Leaving directory 
'/usr/ports/databases/rrdtool/work/rrdtool-1.8.0/src'
gmake[3]: *** [Makefile:618: all] Error 2
gmake[3]: Leaving directory 
'/usr/ports/databases/rrdtool/work/rrdtool-1.8.0/src'
gmake[2]: *** [Makefile:504: all-recursive] Error 1
gmake[2]: Leaving directory 
'/usr/ports/databases/rrdtool/work/rrdtool-1.8.0'
===> Compilation failed unexpectedly.
Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to
the maintainer.
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/databases/rrdtool
*** Error code 1

Stop.
make: stopped in /usr/ports/databases/rrdtool
- 0 root@build14:/usr/ports/databases/rrdtool # uname -aK
FreeBSD build14.sentex.ca 14.0-STABLE FreeBSD 14.0-STABLE #0 
stable/14-009c8a3d2: Tue Jan  9 10:06:28 UTC 2024 
root@build14.sentex.ca:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 
1400504
- 0 root@build14:/usr/ports/databases/rrdtool #

     ---Mike

--------------TmukDo5Aa3rYMdeA1hJHI5Da
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit



  
    
  
  
    
On 1/10/2024 2:53 AM, Antoine Brodin
      wrote:

    

    
The preferred way is fixing the port by removing the undefined symbols
      

        
from the linker version script in the port, but if that is not possible
or difficult, add -Wl,--undefined-version to the linker flags suppresses
the error. E.g. in the port Makefile:

LDFLAGS+= -Wl,--undefined-version

For an example, see:

https://github.com/freebsd/freebsd-ports/commit/37790b26cbda11cd4bb6f237b86cd94739c4059c

Thanks very much!  That did indeed fix databases/rrdtool and and sysutils/flashrom builds.   What is the best way to flag any such issues ? Just open a PR for each individual port ?


      

      
Hello,

databases/rrdtool builds fine on stable/13 here:
https://pkg-status.freebsd.org/gohan04/data/stable13amd64-default-foo/2024-01-09_21h18m24s/logs/rrdtool-1.8.0_2.log



    

    
I am not sure why it fails for me both in poudrier and outside
      it. But the same type of error as 

    

    
https://pkg-status.freebsd.org/gohan04/data/stable13amd64-default-foo/2024-01-09_21h18m24s/logs/errors/ivykis-0.42.4.log

    
and adding 

    

    
LDFLAGS+= -Wl,--undefined-version



    
fixes it for me

    
I just checked on releng_14 that also has clang 17 MFCd and I get
      the same error on the port. I am building it with less features
      than the default so not sure if that makes it fail in my
      environment. However the same options work with clang16

    

    
/usr/local/lib  -L/usr/local/lib  -o rrdupdate rrdupdate.o
      librrdupd.la 

      libtool: link: cc -O2 -pipe -fstack-protector-strong -isystem
      /usr/local/include -fno-strict-aliasing -D_GNU_SOURCE
      -fno-strict-aliasing -Wall -std=gnu99 -pedantic -Wundef -Wshadow
      -Wpointer-arith -Wcast-align -Wmissing-prototypes
      -Wmissing-declarations -Wnested-externs -Winline
      -Wold-style-definition -W -I.. -O2 -pipe -fstack-protector-strong
      -isystem /usr/local/include -fno-strict-aliasing -D_GNU_SOURCE
      -fno-strict-aliasing -Wall -std=gnu99 -pedantic -Wundef -Wshadow
      -Wpointer-arith -Wcast-align -Wmissing-prototypes
      -Wmissing-declarations -Wnested-externs -Winline
      -Wold-style-definition -W -fstack-protector-strong -o rrdupdate
      rrdupdate.o  -L/usr/local/lib ./.libs/librrdupd.a -lm -lwrap
      -lglib-2.0 -lintl -pthread

      libtool: link: echo "{ global:" > .libs/librrd.so.8.3.0-ver

      libtool: link:           sed -e "s|$|;|" < ./librrd.sym
      >> .libs/librrd.so.8.3.0-ver

      libtool: link:   echo "local: *; };" >>
      .libs/librrd.so.8.3.0-ver

      libtool: link: cc -shared  -fPIC -DPIC 
      .libs/librrd_la-rrd_version.o .libs/librrd_la-rrd_last.o
      .libs/librrd_la-rrd_lastupdate.o .libs/librrd_la-rrd_first.o
      .libs/librrd_la-rrd_dump.o .libs/librrd_la-rrd_flushcached.o
      .libs/librrd_la-rrd_fetch.o .libs/librrd_la-rrd_fetch_cb.o
      .libs/librrd_la-rrd_resize.o .libs/librrd_la-rrd_tune.o
      .libs/librrd_la-rrd_list.o .libs/librrd_la-rrd_restore.o 
      -Wl,--whole-archive ./.libs/librrdupd.a -Wl,--no-whole-archive 
      -L/usr/local/lib -lglib-2.0 -lintl -lm -lwrap -lxml2  -O2
      -fstack-protector-strong -pthread -O2 -fstack-protector-strong
      -fstack-protector-strong -Wl,-rpath -Wl,/usr/local/lib   -pthread
      -Wl,-soname -Wl,librrd.so.8 -Wl,-version-script
      -Wl,.libs/librrd.so.8.3.0-ver -o .libs/librrd.so.8.3.0

      ld: error: version script assignment of 'global' to symbol
      'rrd_graph' failed: symbol not defined

      ld: error: version script assignment of 'global' to symbol
      'rrd_graph_v' failed: symbol not defined

      ld: error: version script assignment of 'global' to symbol
      'rrd_lcd' failed: symbol not defined

      ld: error: version script assignment of 'global' to symbol
      'rrd_reduce_data' failed: symbol not defined

      ld: error: version script assignment of 'global' to symbol
      'rrd_xport' failed: symbol not defined

      cc: error: linker command failed with exit code 1 (use -v to see
      invocation)

      gmake[4]: *** [Makefile:773: librrd.la] Error 1

      gmake[4]: Leaving directory
      '/usr/ports/databases/rrdtool/work/rrdtool-1.8.0/src'

      gmake[3]: *** [Makefile:618: all] Error 2

      gmake[3]: Leaving directory
      '/usr/ports/databases/rrdtool/work/rrdtool-1.8.0/src'

      gmake[2]: *** [Makefile:504: all-recursive] Error 1

      gmake[2]: Leaving directory
      '/usr/ports/databases/rrdtool/work/rrdtool-1.8.0'

      ===> Compilation failed unexpectedly.

      Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the
      failure to

      the maintainer.

      *** Error code 1

      

      Stop.

      make[1]: stopped in /usr/ports/databases/rrdtool

      *** Error code 1

      

      Stop.

      make: stopped in /usr/ports/databases/rrdtool

      - 0 root@build14:/usr/ports/databases/rrdtool # uname -aK

      FreeBSD build14.sentex.ca 14.0-STABLE FreeBSD 14.0-STABLE #0
      stable/14-009c8a3d2: Tue Jan  9 10:06:28 UTC 2024    
      root@build14.sentex.ca:/usr/obj/usr/src/amd64.amd64/sys/GENERIC
      amd64 1400504

      - 0 root@build14:/usr/ports/databases/rrdtool # 

    

    
    ---Mike

    

    

      

    

  


--------------TmukDo5Aa3rYMdeA1hJHI5Da--

From nobody Wed Jan 10 16:22:10 2024
X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
	by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T9CkG6jv4z57HbL
	for ; Wed, 10 Jan 2024 16:22:14 +0000 (UTC)
	(envelope-from dim@FreeBSD.org)
Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256
	 client-signature RSA-PSS (4096 bits) client-digest SHA256)
	(Client CN "smtp.freebsd.org", Issuer "R3" (verified OK))
	by mx1.freebsd.org (Postfix) with ESMTPS id 4T9CkG68qGz56Kk;
	Wed, 10 Jan 2024 16:22:14 +0000 (UTC)
	(envelope-from dim@FreeBSD.org)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim;
	t=1704903734;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=w1zRj/xh0oRbzd1Z1KcpuTqlICBU+bKNZU5dZtHpxGs=;
	b=hzsRv+dAK552RDdhQBylt2lWR0gmr6qDYNYbIjNB6LuQPey78mvoncFnteOn1xgKFXy85d
	Ltd3FFkgdekmB5WC8RKmAb9M1NszmYgmhsW5A7fYw6ZYz2otGIsSpeHy0/wYZCoESKQd7q
	6x9gdpBbajSmxAKwvjiy3RI4X3n3FxXpV4UwmPa80w/kLxJd1j0oLI8/LosyLIW3nLYmpS
	iAe8uE7l4A/kSrASrpeYAZVZYZXIHmrkfDrBbpUGlP/2KEb+XWm3MZ49JRFol7wD+c92xC
	rhyPvoZMLe38bvkCjHebbwXM3oiLM1cJgYBZcLZ/UYNBTCYxmALlp0h+NHcevw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org;
	s=dkim; t=1704903734;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=w1zRj/xh0oRbzd1Z1KcpuTqlICBU+bKNZU5dZtHpxGs=;
	b=N+6QFg0ePyYi12BeIBULeacJ0jdmOl/sfbZ4562uKB9Jm1xga27r1V3TxeCij/KiGinxjY
	rN8Dn1fMm7N2EbIcLMTtDf03g5PuSxH3JNMdlsPuDO3VQurNpzKIuLyp9AFcJu8UrDpWUv
	KoUV53i4wxqtdrC/n7w4gjB2b+uoAzf9gZRi8/ft9w36hFIlXPZFiBK9dnpu6ch7+m/PjQ
	87u0Yg/WUrwK3ZW9spOTBxuriNv0cs48JX5G/wIhrTF5UfZu6eoLzRjkljHQr0m3RfuewE
	me+pCIvDp6bpVlYxRPlQcgji2ZQVrUfkGfm1nCyyMyIX7pxKYrwypf0GuiXeOw==
ARC-Authentication-Results: i=1;
	mx1.freebsd.org;
	none
ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1704903734; a=rsa-sha256; cv=none;
	b=KCz3f/lBB5FJ4eORGgqDcRLyF/2eelp2f08npAKEMd86XLtqobrH1xvMFH55/nfjZeEhWJ
	9wYxUmu2A8GSL1aJL3zjWKftiYYpAfcI3QCAuvEHoFx2CIb1BF9EBsWTEUMP54iEBVjzuW
	iGiLhmHoo7w28NFsBkQ4r5fI+kH3QTUoMyGaUZvfe7/AiQUeN1QAaJMo8hGOD+80Hxeu9k
	u6mmDTTlEcKjufFUhh7RajtinzQpH/aR7c/vm6KJCr1/35nETOHVkwcdbAsFTfKgTplFV3
	jLt/UsTtxAA84hHE3qosaGd3tHg7zMJNwGIgPEOWgwWkhRkmA9suFc8MR4vZhA==
Received: from tensor.andric.com (tensor.andric.com [87.251.56.140])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256
	 client-signature RSA-PSS (2048 bits) client-digest SHA256)
	(Client CN "tensor.andric.com", Issuer "R3" (verified OK))
	(Authenticated sender: dim)
	by smtp.freebsd.org (Postfix) with ESMTPSA id 4T9CkG42hBz197j;
	Wed, 10 Jan 2024 16:22:14 +0000 (UTC)
	(envelope-from dim@FreeBSD.org)
Received: from smtpclient.apple (longrow.home.andric.com [192.168.0.17])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(No client certificate requested)
	by tensor.andric.com (Postfix) with ESMTPSA id 2D02D571C2;
	Wed, 10 Jan 2024 17:22:11 +0100 (CET)
Content-Type: text/plain;
	charset=us-ascii
List-Id: Production branch of FreeBSD source code 
List-Archive: https://lists.freebsd.org/archives/freebsd-stable
List-Help: 
List-Post: 
List-Subscribe: 
List-Unsubscribe: 
Sender: owner-freebsd-stable@freebsd.org
X-BeenThere: freebsd-stable@freebsd.org
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\))
Subject: Re: clang 17 and ports fallout
From: Dimitry Andric 
In-Reply-To: <20ceec23-fe90-41de-a3d8-82eada0d56e7@sentex.net>
Date: Wed, 10 Jan 2024 17:22:10 +0100
Cc: Antoine Brodin ,
 FreeBSD-STABLE Mailing List 
Content-Transfer-Encoding: quoted-printable
Message-Id: 
References: <06381879-c328-4051-bb2f-9c3620d90fa0@sentex.net>
 <4a0db68d-bb13-4238-a251-74d02ff14a17@sentex.net>
 
 <04837a47-a39c-40c5-a045-2d27fb3893ac@sentex.net>
 
 <20ceec23-fe90-41de-a3d8-82eada0d56e7@sentex.net>
To: mike tancsa 
X-Mailer: Apple Mail (2.3731.700.6)

On 10 Jan 2024, at 13:16, mike tancsa  wrote:
>=20
> On 1/10/2024 2:53 AM, Antoine Brodin wrote:
>> The preferred way is fixing the port by removing the undefined =
symbols
>>>=20
>>> from the linker version script in the port, but if that is not =
possible
>>> or difficult, add -Wl,--undefined-version to the linker flags =
suppresses
>>> the error. E.g. in the port Makefile:
>>>=20
>>> LDFLAGS+=3D -Wl,--undefined-version
>>>=20
>>> For an example, see:
>>>=20
>>> =
https://github.com/freebsd/freebsd-ports/commit/37790b26cbda11cd4bb6f237b8=
6cd94739c4059c
>>>=20
>>> Thanks very much! That did indeed fix databases/rrdtool and and =
sysutils/flashrom builds. What is the best way to flag any such issues ? =
Just open a PR for each individual port ?
>>>=20
>> Hello,
>>=20
>> databases/rrdtool builds fine on stable/13 here:
>> =
https://pkg-status.freebsd.org/gohan04/data/stable13amd64-default-foo/2024=
-01-09_21h18m24s/logs/rrdtool-1.8.0_2.log
>>=20
>>=20
> I am not sure why it fails for me both in poudrier and outside it. But =
the same type of error as=20
> =
https://pkg-status.freebsd.org/gohan04/data/stable13amd64-default-foo/2024=
-01-09_21h18m24s/logs/errors/ivykis-0.42.4.log
> and adding=20
> LDFLAGS+=3D -Wl,--undefined-version
>=20
>=20
> fixes it for me
> I just checked on releng_14 that also has clang 17 MFCd and I get the =
same error on the port. I am building it with less features than the =
default so not sure if that makes it fail in my environment. However the =
same options work with clang16
> /usr/local/lib  -L/usr/local/lib  -o rrdupdate rrdupdate.o =
librrdupd.la=20
> libtool: link: cc -O2 -pipe -fstack-protector-strong -isystem =
/usr/local/include -fno-strict-aliasing -D_GNU_SOURCE =
-fno-strict-aliasing -Wall -std=3Dgnu99 -pedantic -Wundef -Wshadow =
-Wpointer-arith -Wcast-align -Wmissing-prototypes -Wmissing-declarations =
-Wnested-externs -Winline -Wold-style-definition -W -I.. -O2 -pipe =
-fstack-protector-strong -isystem /usr/local/include =
-fno-strict-aliasing -D_GNU_SOURCE -fno-strict-aliasing -Wall -std=3Dgnu99=
 -pedantic -Wundef -Wshadow -Wpointer-arith -Wcast-align =
-Wmissing-prototypes -Wmissing-declarations -Wnested-externs -Winline =
-Wold-style-definition -W -fstack-protector-strong -o rrdupdate =
rrdupdate.o  -L/usr/local/lib ./.libs/librrdupd.a -lm -lwrap -lglib-2.0 =
-lintl -pthread
> libtool: link: echo "{ global:" > .libs/librrd.so.8.3.0-ver
> libtool: link:           sed -e "s|$|;|" < ./librrd.sym >> =
.libs/librrd.so.8.3.0-ver
> libtool: link:   echo "local: *; };" >> .libs/librrd.so.8.3.0-ver
> libtool: link: cc -shared  -fPIC -DPIC  .libs/librrd_la-rrd_version.o =
.libs/librrd_la-rrd_last.o .libs/librrd_la-rrd_lastupdate.o =
.libs/librrd_la-rrd_first.o .libs/librrd_la-rrd_dump.o =
.libs/librrd_la-rrd_flushcached.o .libs/librrd_la-rrd_fetch.o =
.libs/librrd_la-rrd_fetch_cb.o .libs/librrd_la-rrd_resize.o =
.libs/librrd_la-rrd_tune.o .libs/librrd_la-rrd_list.o =
.libs/librrd_la-rrd_restore.o  -Wl,--whole-archive ./.libs/librrdupd.a =
-Wl,--no-whole-archive  -L/usr/local/lib -lglib-2.0 -lintl -lm -lwrap =
-lxml2  -O2 -fstack-protector-strong -pthread -O2 =
-fstack-protector-strong -fstack-protector-strong -Wl,-rpath =
-Wl,/usr/local/lib   -pthread -Wl,-soname -Wl,librrd.so.8 =
-Wl,-version-script -Wl,.libs/librrd.so.8.3.0-ver -o =
.libs/librrd.so.8.3.0
> ld: error: version script assignment of 'global' to symbol 'rrd_graph' =
failed: symbol not defined
> ld: error: version script assignment of 'global' to symbol =
'rrd_graph_v' failed: symbol not defined
> ld: error: version script assignment of 'global' to symbol 'rrd_lcd' =
failed: symbol not defined
> ld: error: version script assignment of 'global' to symbol =
'rrd_reduce_data' failed: symbol not defined
> ld: error: version script assignment of 'global' to symbol 'rrd_xport' =
failed: symbol not defined
> cc: error: linker command failed with exit code 1 (use -v to see =
invocation)
> gmake[4]: *** [Makefile:773: librrd.la] Error 1
> gmake[4]: Leaving directory =
'/usr/ports/databases/rrdtool/work/rrdtool-1.8.0/src'
> gmake[3]: *** [Makefile:618: all] Error 2
> gmake[3]: Leaving directory =
'/usr/ports/databases/rrdtool/work/rrdtool-1.8.0/src'
> gmake[2]: *** [Makefile:504: all-recursive] Error 1
> gmake[2]: Leaving directory =
'/usr/ports/databases/rrdtool/work/rrdtool-1.8.0'
> =3D=3D=3D> Compilation failed unexpectedly.
> Try to set MAKE_JOBS_UNSAFE=3Dyes and rebuild before reporting the =
failure to

I built the port and it worked just fine, but apparently this happens =
only when you turn off the GRAPH option.

I committed a fix in =
https://cgit.freebsd.org/ports/commit/?id=3D2ed094adef32cc683a9a077f1c8eb2=
241754068a .

-Dimitry


From nobody Wed Jan 10 17:59:05 2024
X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
	by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T9Ft32cg8z55W7f
	for ; Wed, 10 Jan 2024 17:59:07 +0000 (UTC)
	(envelope-from dim@FreeBSD.org)
Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256
	 client-signature RSA-PSS (4096 bits) client-digest SHA256)
	(Client CN "smtp.freebsd.org", Issuer "R3" (verified OK))
	by mx1.freebsd.org (Postfix) with ESMTPS id 4T9Ft30x43z49GD;
	Wed, 10 Jan 2024 17:59:07 +0000 (UTC)
	(envelope-from dim@FreeBSD.org)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim;
	t=1704909547;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=3ER+2lslSkZcwH5OsbRB2CkBRxjZRP3nUYVO2pYqMXg=;
	b=xrY+d88ofsgkK8U7NZjL/HCZnAgV6dbP7gnvaORK7iRQqtOj4m4PcjNJyS35oTqi1zUo7H
	l4MKVzNduhPKRu37oWMD5t9H9WZAvtO9NASPi3KeQ1a39lxLAoyl4+8fr0DuqIt4IpaudB
	c8pDFNLiqxfj9iG2DbCf5hU0TQzH8TAM0iZIiMCNw0dQreHDgQVVPlL9FJ6ds1Q1YcaMXP
	tsD4A7WQLi9BUr2fvpJy7LipVuY47922jFnWjzVqBUzuYjGURYQIsOSXgzYox/VlvKrwZR
	2qYyVyBGAI9P9BTDgT+uACjf1uwfCddpa/pDH0xSNbCLiuxf6S7cD3wEDhwx1A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org;
	s=dkim; t=1704909547;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=3ER+2lslSkZcwH5OsbRB2CkBRxjZRP3nUYVO2pYqMXg=;
	b=aU0zbTB/1gWEPgoBYAMtsduot3aoDiH6PCCVa9Sw0Bp64ToRqm5NMXIEF8iQnGaefMsS2L
	4RmYg8XfnmCUYhys2GAmB8ENCunhEetOE5hscTOIXyAKLXIjs42DMFAGF+4n7otnHOpoa7
	yzoVOLQ3vcIzLuhZzX8tvBcSKyUYp+9kyLPEoQOpYJVOHuvkXO/R0J5hLSSUsiIGBplufo
	6DjqDG4rK5ZJ4KOPgM1WJWDCIFeAPzHPNGEhsnlZKnaDQAlkw8twlXfHugdl0XtczeBKMf
	/n8Db5PqMrAkUUm+UwYlGN7xitRl6h0rhdgSvohFX8Q48Xal2ACAL5fj361NwQ==
ARC-Authentication-Results: i=1;
	mx1.freebsd.org;
	none
ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1704909547; a=rsa-sha256; cv=none;
	b=A8fuWwcrRg+zLyHxFWJ/vh0Bj3q8tpG4x5gWG51V7Qyn3Kq6JfBEEh6zA5BMDheGPzwgk4
	+OYHiPVH/0S0OVamTOhohV/x3QHph9LmZjUUpKaZgyVDKpXlG/8vUcpph1n4PkOsQGchXb
	Frui3MagU+rcpID+Uciv+kaXHaA1OzvxjhaT3bF8mJ6ncguxJ8jITmwc9J07azpGsZ/Bwj
	7DPCHO+6Z3jOkI+meR1sS8xR9YqSmQlstnq7YitOLpEcCLwvJkeJOoUqX1unhn7vVRNNe5
	2EHA8de0gP4+bwPciPI8ggWH6Kqfnl6qFwxNQ8UDql7Ey49oVEtHa/kRKz7z8A==
Received: from tensor.andric.com (tensor.andric.com [87.251.56.140])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256
	 client-signature RSA-PSS (2048 bits) client-digest SHA256)
	(Client CN "tensor.andric.com", Issuer "R3" (verified OK))
	(Authenticated sender: dim)
	by smtp.freebsd.org (Postfix) with ESMTPSA id 4T9Ft25kW9z1BsC;
	Wed, 10 Jan 2024 17:59:06 +0000 (UTC)
	(envelope-from dim@FreeBSD.org)
Received: from smtpclient.apple (longrow.home.andric.com [192.168.0.17])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(No client certificate requested)
	by tensor.andric.com (Postfix) with ESMTPSA id A76B3572D3;
	Wed, 10 Jan 2024 18:59:05 +0100 (CET)
Content-Type: text/plain;
	charset=us-ascii
List-Id: Production branch of FreeBSD source code 
List-Archive: https://lists.freebsd.org/archives/freebsd-stable
List-Help: 
List-Post: 
List-Subscribe: 
List-Unsubscribe: 
Sender: owner-freebsd-stable@freebsd.org
X-BeenThere: freebsd-stable@freebsd.org
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\))
Subject: Re: clang 17 and ports fallout
From: Dimitry Andric 
In-Reply-To: <06381879-c328-4051-bb2f-9c3620d90fa0@sentex.net>
Date: Wed, 10 Jan 2024 18:59:05 +0100
Cc: FreeBSD-STABLE Mailing List 
Content-Transfer-Encoding: quoted-printable
Message-Id: <04CD8DE7-E87A-4423-A3DB-77D1399C3BA1@FreeBSD.org>
References: <06381879-c328-4051-bb2f-9c3620d90fa0@sentex.net>
To: mike tancsa 
X-Mailer: Apple Mail (2.3731.700.6)

On 8 Jan 2024, at 21:40, mike tancsa  wrote:
>=20
> After today's MFC of clang17,  I am seeing some fallout from a few =
ports that build with clang16 on RELENG_13 but now fail.  Any ideas what =
might be going on ?   I have nothing in /etc/make.conf nor /etc/src.conf
...
> ld: error: version script assignment of 'IVYKIS_0.29' to symbol =
'iv_inotify_register' failed: symbol not defined
> ld: error: version script assignment of 'IVYKIS_0.29' to symbol =
'iv_inotify_unregister' failed: symbol not defined
> ld: error: version script assignment of 'IVYKIS_0.29' to symbol =
'iv_inotify_watch_register' failed: symbol not defined
> ld: error: version script assignment of 'IVYKIS_0.29' to symbol =
'iv_inotify_watch_unregister' failed: symbol not defined

I pushed a fix for this in:
=
https://cgit.freebsd.org/ports/commit/?id=3D05917d340dee6d2bf02463af53372f=
2da2ecab00

-Dimitry


From nobody Thu Jan 11 03:04:51 2024
X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
	by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T9Tzy27xqz55vFD
	for ; Thu, 11 Jan 2024 03:05:02 +0000 (UTC)
	(envelope-from wollman@hergotha.csail.mit.edu)
Received: from hergotha.csail.mit.edu (tunnel82308-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256
	 client-signature RSA-PSS (2048 bits) client-digest SHA256)
	(Client CN "garrett.wollman.name", Issuer "R3" (verified OK))
	by mx1.freebsd.org (Postfix) with ESMTPS id 4T9Tzx2L1Pz4jTT
	for ; Thu, 11 Jan 2024 03:05:01 +0000 (UTC)
	(envelope-from wollman@hergotha.csail.mit.edu)
Authentication-Results: mx1.freebsd.org;
	dkim=none;
	dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=bimajority.org (policy=none);
	spf=pass (mx1.freebsd.org: domain of wollman@hergotha.csail.mit.edu designates 2001:470:1f06:ccb::2 as permitted sender) smtp.mailfrom=wollman@hergotha.csail.mit.edu
Received: from hergotha.csail.mit.edu (localhost [127.0.0.1])
	by hergotha.csail.mit.edu (8.17.1/8.17.1) with ESMTPS id 40B34rst003632
	(version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO);
	Wed, 10 Jan 2024 22:04:53 -0500 (EST)
	(envelope-from wollman@hergotha.csail.mit.edu)
Received: (from wollman@localhost)
	by hergotha.csail.mit.edu (8.17.1/8.17.1/Submit) id 40B34qfH003631;
	Wed, 10 Jan 2024 22:04:52 -0500 (EST)
	(envelope-from wollman)
List-Id: Production branch of FreeBSD source code 
List-Archive: https://lists.freebsd.org/archives/freebsd-stable
List-Help: 
List-Post: 
List-Subscribe: 
List-Unsubscribe: 
Sender: owner-freebsd-stable@freebsd.org
X-BeenThere: freebsd-stable@freebsd.org
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <26015.23251.994177.437150@hergotha.csail.mit.edu>
Date: Wed, 10 Jan 2024 22:04:51 -0500
From: Garrett Wollman 
To: "Patrick M. Hausen" 
Cc: FreeBSD-STABLE Mailing List 
Subject: Re: Odd values for various memory metrics via SNMP
In-Reply-To: 
References: <1EDAF2AC-3A2B-43BE-B66B-E095F5A80C2C@punkt.de>
	
	<84B7C7D7-BC06-4944-A7E0-5AFC47B6BC0E@punkt.de>
	<8f4cf72e-8320-4bfd-a4d9-3db34db4580d@aetern.org>
	
	<25996.34932.339497.605798@hergotha.csail.mit.edu>
	
X-Mailer: VM 8.2.0b under 29.1 (amd64-portbld-freebsd13.2)
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.4 (hergotha.csail.mit.edu [0.0.0.0]); Wed, 10 Jan 2024 22:04:53 -0500 (EST)
X-Spam-Status: No, score=-4.5 required=5.0 tests=ALL_TRUSTED,
	HEADER_FROM_DIFFERENT_DOMAINS,NICE_REPLY_A,T_SCC_BODY_TEXT_LINE
	autolearn=disabled version=4.0.0
X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on
	hergotha.csail.mit.edu
X-Spamd-Bar: --
X-Spamd-Result: default: False [-2.89 / 15.00];
	NEURAL_HAM_LONG(-1.00)[-1.000];
	NEURAL_HAM_MEDIUM(-1.00)[-1.000];
	NEURAL_HAM_SHORT(-0.99)[-0.992];
	FORGED_SENDER(0.30)[wollman@bimajority.org,wollman@hergotha.csail.mit.edu];
	R_SPF_ALLOW(-0.20)[+ip6:2001:470:1f06:ccb::2];
	DMARC_POLICY_SOFTFAIL(0.10)[bimajority.org : SPF not aligned (relaxed), No valid DKIM,none];
	MIME_GOOD(-0.10)[text/plain];
	FROM_HAS_DN(0.00)[];
	FREEFALL_USER(0.00)[wollman];
	ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US];
	MIME_TRACE(0.00)[0:+];
	RCPT_COUNT_TWO(0.00)[2];
	MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org];
	RCVD_TLS_LAST(0.00)[];
	FROM_NEQ_ENVFROM(0.00)[wollman@bimajority.org,wollman@hergotha.csail.mit.edu];
	R_DKIM_NA(0.00)[];
	RCVD_COUNT_TWO(0.00)[2];
	TO_DN_ALL(0.00)[];
	TO_MATCH_ENVRCPT_SOME(0.00)[];
	ARC_NA(0.00)[]
X-Rspamd-Queue-Id: 4T9Tzx2L1Pz4jTT

< said:

>> [I wrote:]
>> This is computed by the function vmtotal() in sys/vm/vm_meter.c.  It
>> walks all VM objects in the system, skipping those that are
>> unreferenced, and adds up all of their sizes.
>> [...]

> Why does FreeBSD calculate the values this way while MWL's book clearly
> states differently? Who's wrong here?

I cannot speak for Michael Lucas.  FreeBSD has ~always calculated VM
statistics in this way; the same algorithm was in 4.4BSD, thirty years
ago.

-GAWollman


From nobody Fri Jan 12 17:57:30 2024
X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
	by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TBTlT6MNCz57GDR
	for ; Fri, 12 Jan 2024 17:57:41 +0000 (UTC)
	(envelope-from dfr@rabson.org)
Received: from mail-yb1-xb31.google.com (mail-yb1-xb31.google.com [IPv6:2607:f8b0:4864:20::b31])
	(using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256
	 client-signature RSA-PSS (2048 bits) client-digest SHA256)
	(Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK))
	by mx1.freebsd.org (Postfix) with ESMTPS id 4TBTlT1NhNz4m50
	for ; Fri, 12 Jan 2024 17:57:41 +0000 (UTC)
	(envelope-from dfr@rabson.org)
Authentication-Results: mx1.freebsd.org;
	dkim=pass header.d=rabson-org.20230601.gappssmtp.com header.s=20230601 header.b="QAjxWG/M";
	dmarc=none;
	spf=pass (mx1.freebsd.org: domain of dfr@rabson.org designates 2607:f8b0:4864:20::b31 as permitted sender) smtp.mailfrom=dfr@rabson.org
Received: by mail-yb1-xb31.google.com with SMTP id 3f1490d57ef6-dbed71baa91so5518814276.0
        for ; Fri, 12 Jan 2024 09:57:41 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=rabson-org.20230601.gappssmtp.com; s=20230601; t=1705082260; x=1705687060; darn=freebsd.org;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:from:to:cc:subject:date:message-id:reply-to;
        bh=qM4tWqQDfZoLSufw8tZE+YKA3Igv9zj1c8H834BTqlQ=;
        b=QAjxWG/Me1hJv05m/XakLXy7ctrZckQZepYo7duS2LB6RxJjRhU/WB54RIbpObbBmT
         C2ykBV7fxFkoY0DV0C91QRtSPlY9C9cBW1oqp38+XrL/7uipb62mRlXk+9hNtPHVe/0s
         jMusY3myShbjcyw5EYrm8Xnic72MQodObtBAc5HAj5sxeclNHnaKK+4BAP4ZaSizJBHc
         vNoe7WZDRdqZVlj15E3/0FuAx9yeutwt8MrDDzTyGfxVJBLdgrA663eG9iVvNZGfFpjs
         42KPh69JSDTPCQ0uVGlj0Tht4N7OIaIRsNGthDh5tIwGk7Y8nryMVQy2fR8rl+7gm3gc
         UQHA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1705082260; x=1705687060;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=qM4tWqQDfZoLSufw8tZE+YKA3Igv9zj1c8H834BTqlQ=;
        b=J8SrakxulNr0/fZtDbXwy0uAI9gz90ic9RUUw/18nC9MoShLPT3PDckzgOcTFmJt6Y
         2bW0SPYSUiQnI12syfM3mqmPG4g2GWPwbzda/bHNRurrWMWFR3w3WH1c1/6qyPrmYqSc
         6ATDMsIxWilGLpb90Awat33P2QMeFPtMLbqHv+WFPsU+VEQg9UvRF42Z+voYRStbmjBg
         OHUEfZgbN3S10YvixQEEjsHaSTGceFF1ih5gfV03HP3UhfyXPb4XMlFyi243cpuv7RJa
         91Zepz+whEN8KXDDWjVnYPMMGYvLCiiZ9oobtO97yYoB4vb3GC5V/Rj5fzG3414r5XJX
         woAw==
X-Gm-Message-State: AOJu0YyDD0IZidvMNoGwjRKNvG4emPu4QC71rRsfPPO3rHafwVxzNCql
	3xYsP0JFKfSSJPuFe0tZWj0zUP+cq1iSYK+fw4lDdhnO0oG15A==
X-Google-Smtp-Source: AGHT+IHvML+kf0qsvmuK4n9CWqZPIGtlANuO82waX+AGnx1t5x6eJSzSOj68wtbMUFj4yJCo1d0uHbK+qmK3xfb1FUI=
X-Received: by 2002:a25:ae48:0:b0:dbd:ac49:223 with SMTP id
 g8-20020a25ae48000000b00dbdac490223mr871043ybe.90.1705082259965; Fri, 12 Jan
 2024 09:57:39 -0800 (PST)
List-Id: Production branch of FreeBSD source code 
List-Archive: https://lists.freebsd.org/archives/freebsd-stable
List-Help: 
List-Post: 
List-Subscribe: 
List-Unsubscribe: 
Sender: owner-freebsd-stable@freebsd.org
X-BeenThere: freebsd-stable@freebsd.org
MIME-Version: 1.0
References: <3CB904C2-D983-4EF7-84D3-6BDED0700B08.ref@yahoo.com> <3CB904C2-D983-4EF7-84D3-6BDED0700B08@yahoo.com>
In-Reply-To: <3CB904C2-D983-4EF7-84D3-6BDED0700B08@yahoo.com>
From: Doug Rabson 
Date: Fri, 12 Jan 2024 17:57:30 +0000
Message-ID: 
Subject: Re: 15 & 14: ram_attach vs. its using regions_to_avail vs.
 "bus_alloc_resource" can lead to: panic("ram_attach: resource %d failed to
 attach", rid)
To: Mark Millard 
Cc: Current FreeBSD , 
	FreeBSD-STABLE Mailing List 
Content-Type: multipart/alternative; boundary="0000000000006b1e9f060ec36675"
X-Spamd-Bar: ---
X-Spamd-Result: default: False [-3.50 / 15.00];
	NEURAL_HAM_MEDIUM(-1.00)[-1.000];
	NEURAL_HAM_LONG(-1.00)[-1.000];
	NEURAL_HAM_SHORT(-1.00)[-0.999];
	R_DKIM_ALLOW(-0.20)[rabson-org.20230601.gappssmtp.com:s=20230601];
	R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36];
	MIME_GOOD(-0.10)[multipart/alternative,text/plain];
	FREEFALL_USER(0.00)[dfr];
	ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US];
	RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::b31:from];
	MIME_TRACE(0.00)[0:+,1:+,2:~];
	MISSING_XM_UA(0.00)[];
	ARC_NA(0.00)[];
	RCVD_COUNT_ONE(0.00)[1];
	FROM_HAS_DN(0.00)[];
	RCPT_COUNT_THREE(0.00)[3];
	PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org];
	TO_MATCH_ENVRCPT_SOME(0.00)[];
	FROM_EQ_ENVFROM(0.00)[];
	DMARC_NA(0.00)[rabson.org];
	TO_DN_ALL(0.00)[];
	RCVD_TLS_LAST(0.00)[];
	MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org];
	FREEMAIL_TO(0.00)[yahoo.com];
	DKIM_TRACE(0.00)[rabson-org.20230601.gappssmtp.com:+]
X-Rspamd-Queue-Id: 4TBTlT1NhNz4m50

--0000000000006b1e9f060ec36675
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sat, 30 Sept 2023 at 08:47, Mark Millard  wrote:

> ram_attach is based on regions_to_avail but that is a problem for
> its later bus_alloc_resource use --and that can lead to:
>
> panic("ram_attach: resource %d failed to attach", rid);
>
> Unfortunately, the known example is use of EDK2 on RPi4B
> class systems, not what is considered the supported way.
> The panic happens for main [so: 15] and will happen once
> the cortex-a72 handling in 14.0-* is in a build fixed by:
>
>     =E2=80=A2 git: 906bcc44641d - releng/14.0 - arm64: Fix errata workaro=
unds that
> depend on smccc Andrew Turner
>
> The lack of the fix leads to an earlier panic as stands.
>
>
> sys/kern/subr_physmem.c 's regions_to_avail is based on ignoring
> phys_avail and using only hwregions and exregions. In other words,
> in part:
>
>  * Initially dump_avail and phys_avail are identical.  Boot time memory
>  * allocations remove extents from phys_avail that may still be included
>  * in dumps.
>
> This means that early, dedicated memory allocations are treated
> as available for general use by regions_to_avail . The distinction
> is visible in the  boot -v output in that:
>
> real memory  =3D 3138154496 (2992 MB)
> Physical memory chunk(s):
> 0x00000000200000 - 0x0000002b7fffff, 727711744 bytes (177664 pages)
> 0x0000002ce3a000 - 0x0000003385ffff, 111304704 bytes (27174 pages)
> 0x000000338c0000 - 0x000000338c6fff, 28672 bytes (7 pages)
> 0x00000033a30000 - 0x00000036efffff, 55377920 bytes (13520 pages)
> 0x000000372e0000 - 0x0000003b2fffff, 67239936 bytes (16416 pages)
> 0x00000040000000 - 0x000000bb3dcfff, 2067648512 bytes (504797 pages)
> avail memory =3D 3027378176 (2887 MB)
>
> does not list the wider:
>
> 0x00000040000000 - 0x000000bfffffff
>
> because of phys_avail . But the earlier dump based on hwregions and
> exregions shows:
>
> Physical memory chunk(s):
>   0x001d0000 - 0x001effff,     0 MB (     32 pages)
>   0x00200000 - 0x338c6fff,   822 MB ( 210631 pages)
>   0x33920000 - 0x3b2fffff,   121 MB (  31200 pages)
>   0x40000000 - 0xbfffffff,  2048 MB ( 524288 pages)
> Excluded memory regions:
>   0x001d0000 - 0x001effff,     0 MB (     32 pages) NoAlloc
>   0x2b800000 - 0x2ce39fff,    22 MB (   5690 pages) NoAlloc
>   0x33860000 - 0x338bffff,     0 MB (     96 pages) NoAlloc
>   0x33920000 - 0x33a2ffff,     1 MB (    272 pages) NoAlloc
>   0x36f00000 - 0x372dffff,     3 MB (    992 pages) NoAlloc
>
> which indicates:
>
>   0x40000000 - 0xbfffffff
>
> is available as far as it is concerned.
>
> (Note some code works/displays in terms of: 0x40000000 - 0xc0000000
> instead.)
>
> For aarch64 , sys/arm64/arm64/nexus.c has a nexus_alloc_resource
> that is used as bus_alloc_resource . It ends up rejecting the
> RPi4B boot via using the result of the call in ram_attach:
>
>                 if (bus_alloc_resource(dev, SYS_RES_MEMORY, &rid, start,
> end,
>                     end - start, 0) =3D=3D NULL)
>                         panic("ram_attach: resource %d failed to attach",
> rid);
>
> as shown by the just-prior start/end pair sequence messages:
>
> ram0: reserving memory region:   200000-2b800000
> ram0: reserving memory region:   2ce3a000-33860000
> ram0: reserving memory region:   338c0000-338c7000
> ram0: reserving memory region:   33a30000-36f00000
> ram0: reserving memory region:   372e0000-3b300000
> ram0: reserving memory region:   40000000-c0000000
> panic: ram_attach: resource 5 failed to attach
>
> I do not see anything about this that looks inherently RPi*
> specific for possibly ending up with an analogous panic. So
> I expect the example is sufficient context to identify a
> problem is present, despite EDK2 use not being normal for
> RPi4B's and the like as far as FreeBSD is concerned.
>

I'm not quite clear why phys_avail changes and why that is triggered by the
906bcc44641d commit. I'm wondering if it makes sense to arrange for
ram_attach to happen before acpi, e.g. using BUS_PASS_ORDER_FIRST?

Doug.

--0000000000006b1e9f060ec36675
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable




=

On Sat, 30 Sept 2023 at 08:47, Mark M=
illard <marklmi@yahoo.com> w=
rote:
ram_attach is based on regions_to_avail but=
 that is a problem for

its later bus_alloc_resource use --and that can lead to:



panic("ram_attach: resource %d failed to attach", rid);



Unfortunately, the known example is use of EDK2 on RPi4B

class systems, not what is considered the supported way.

The panic happens for main [so: 15] and will happen once

the cortex-a72 handling in 14.0-* is in a build fixed by:



=C2=A0 =C2=A0 =E2=80=A2 git: 906bcc44641d - releng/14.0 - arm64: Fix errata=
 workarounds that depend on smccc Andrew Turner



The lack of the fix leads to an earlier panic as stands.





sys/kern/subr_physmem.c 's regions_to_avail is based on ignoring

phys_avail and using only hwregions and exregions. In other words,

in part:



=C2=A0* Initially dump_avail and phys_avail are identical.=C2=A0 Boot time =
memory

=C2=A0* allocations remove extents from phys_avail that may still be includ=
ed

=C2=A0* in dumps.



This means that early, dedicated memory allocations are treated

as available for general use by regions_to_avail . The distinction

is visible in the=C2=A0 boot -v output in that:



real memory=C2=A0 =3D 3138154496 (2992 MB)

Physical memory chunk(s):

0x00000000200000 - 0x0000002b7fffff, 727711744 bytes (177664 pages)

0x0000002ce3a000 - 0x0000003385ffff, 111304704 bytes (27174 pages)

0x000000338c0000 - 0x000000338c6fff, 28672 bytes (7 pages)

0x00000033a30000 - 0x00000036efffff, 55377920 bytes (13520 pages)

0x000000372e0000 - 0x0000003b2fffff, 67239936 bytes (16416 pages)

0x00000040000000 - 0x000000bb3dcfff, 2067648512 bytes (504797 pages)

avail memory =3D 3027378176 (2887 MB)



does not list the wider:



0x00000040000000 - 0x000000bfffffff



because of phys_avail . But the earlier dump based on hwregions and

exregions shows:



Physical memory chunk(s):

=C2=A0 0x001d0000 - 0x001effff,=C2=A0 =C2=A0 =C2=A00 MB (=C2=A0 =C2=A0 =C2=
=A032 pages)

=C2=A0 0x00200000 - 0x338c6fff,=C2=A0 =C2=A0822 MB ( 210631 pages)

=C2=A0 0x33920000 - 0x3b2fffff,=C2=A0 =C2=A0121 MB (=C2=A0 31200 pages)

=C2=A0 0x40000000 - 0xbfffffff,=C2=A0 2048 MB ( 524288 pages)

Excluded memory regions:

=C2=A0 0x001d0000 - 0x001effff,=C2=A0 =C2=A0 =C2=A00 MB (=C2=A0 =C2=A0 =C2=
=A032 pages) NoAlloc 

=C2=A0 0x2b800000 - 0x2ce39fff,=C2=A0 =C2=A0 22 MB (=C2=A0 =C2=A05690 pages=
) NoAlloc 

=C2=A0 0x33860000 - 0x338bffff,=C2=A0 =C2=A0 =C2=A00 MB (=C2=A0 =C2=A0 =C2=
=A096 pages) NoAlloc 

=C2=A0 0x33920000 - 0x33a2ffff,=C2=A0 =C2=A0 =C2=A01 MB (=C2=A0 =C2=A0 272 =
pages) NoAlloc 

=C2=A0 0x36f00000 - 0x372dffff,=C2=A0 =C2=A0 =C2=A03 MB (=C2=A0 =C2=A0 992 =
pages) NoAlloc 



which indicates:



=C2=A0 0x40000000 - 0xbfffffff



is available as far as it is concerned.



(Note some code works/displays in terms of: 0x40000000 - 0xc0000000

instead.)



For aarch64 , sys/arm64/arm64/nexus.c has a nexus_alloc_resource

that is used as bus_alloc_resource . It ends up rejecting the

RPi4B boot via using the result of the call in ram_attach:



=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (bus_alloc_resou=
rce(dev, SYS_RES_MEMORY, &rid, start, end,

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 end -=
 start, 0) =3D=3D NULL)

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 panic("ram_attach: resource %d failed to attach", rid)=
;



as shown by the just-prior start/end pair sequence messages:



ram0: reserving memory region:=C2=A0 =C2=A0200000-2b800000

ram0: reserving memory region:=C2=A0 =C2=A02ce3a000-33860000

ram0: reserving memory region:=C2=A0 =C2=A0338c0000-338c7000

ram0: reserving memory region:=C2=A0 =C2=A033a30000-36f00000

ram0: reserving memory region:=C2=A0 =C2=A0372e0000-3b300000

ram0: reserving memory region:=C2=A0 =C2=A040000000-c0000000

panic: ram_attach: resource 5 failed to attach



I do not see anything about this that looks inherently RPi*

specific for possibly ending up with an analogous panic. So

I expect the example is sufficient context to identify a

problem is present, despite EDK2 use not being normal for

RPi4B's and the like as far as FreeBSD is concerned.

I'm not quite clear why phys_avail changes and why th=
at is triggered by the 906bcc44641d commit. I'm wondering if it makes s=
ense to arrange for ram_attach to happen before acpi, e.g. using BUS_PASS_O=
RDER_FIRST?

Doug.
=C2=A0


--0000000000006b1e9f060ec36675--

From nobody Fri Jan 12 21:53:15 2024
X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
	by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TBZzf5v1mz56SYK
	for ; Fri, 12 Jan 2024 21:53:34 +0000 (UTC)
	(envelope-from marklmi@yahoo.com)
Received: from sonic303-25.consmr.mail.gq1.yahoo.com (sonic303-25.consmr.mail.gq1.yahoo.com [98.137.64.206])
	(using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
	(Client did not present a certificate)
	by mx1.freebsd.org (Postfix) with ESMTPS id 4TBZzf3QFdz4cyR
	for ; Fri, 12 Jan 2024 21:53:34 +0000 (UTC)
	(envelope-from marklmi@yahoo.com)
Authentication-Results: mx1.freebsd.org;
	none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1705096411; bh=BRrGmWDSgk3+66fQMJlET9RSeFG8z6+hvUTU+ANvlBI=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=KnF96TyrgMXbkwghHZtocVoXzS1atwQbhZICdHcHs4/zjSAFlkfuY667QuIIQjVWjGlCLi+5OogfOQNt5zz9hYuDHafuzqSCL0vqbArPdZeoXfdcSVmPUrgiy21K45JrWo0+8ecIrM/wyFbEGxIjNfVdwYsULxDRoiCKYZ33ytISkT7jxRVeWE+dbbVSKGhjdS0WW0xP67GMVoDpKsKa0KQi4xjG4zQTQq1TX2LtklZmBdtyqBex5uZHU6XO3rRpVMEzrtNt0oR2LyKY2XxcEasQ/Mz/XougjFi9TJmqdd7zOkmFx8rCpz/sFW4r78VEnyOKeYYYnwI9IDUkTJcs6A==
X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1705096411; bh=QG7iZq3eG/eiUt0mE7OZLYDtLtmrqk9SJTPgyWKt/aQ=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=UAXsIJQoMflkqhtKmEPVwAj2isNgZNkvPf9fOIjYEU88Ka0JYEpSyo4rhFSt6AJ9tvH6cEE4PzacTeA/2u+pv/LEf3/kiXR/OLckyixOiAB1w8mbhNCedy33wa43XjyjIi8z9IPeXTeLdj0fK9WB/Wx6DmSz/qfbxvOOdo5RZT8hDW7HO/GnwhsYsAiKskP2FPE9k1gKf1jrxeBKYSsnlsvFewHlOSndHhRFn+LJZ1kwQ5vABqesFjkStVwHCSxF5NqSQ3lk3tXss5QXwdnYuhUzKzdz2BTLmzZZ+b0dxHDfMMxT/W2qJ2G6GXdhKpEj1tPhZrPZLazTkW4rOy5pwg==
X-YMail-OSG: vTj8hDcVM1nQxat8nwNeY.odmaZKAnRbMDd7eOmrn6MJWiE_iXLIL6E3V81aK74
 X7SRGDZSrXemjrdpFMgR9unKOQtHZuHAPEKgay36ONI.zhEQDNvsAq5gHXmesEK0y8lzvRFlbxrg
 owv435NNiU2bDsu2NdIdi5lHU2AiAlLL_XE2zIFUJuoMB4jTCXTiOgm73Bk8t021zb3Lhv8aYdm5
 WE_.7EFE.KFUnUvu3hwk8Yz9tlYSUx1yO18Gnskasb1rJ7Y6ssnlAM_7G1cCqam16lKRPhknus9Y
 2UfVNTu0Lehjf_kzjzsT3.zLtLMxlYbtYvkkDVLmtHjOolky4OhA3KY0gMCwUOgrO885J_0OQhJz
 5R1dGTxR98p_Dq6.EAby05oH7FlqlhOAbcu_4ct8XPhp1crWtJ5q1KZfwNEXbSl0MLifxvKBmh.1
 O_yfPYFA7Q1WpS7U3iKk2e238FQLF7VmvI4TQ9U.w2jVXuC53372_rO3zt6nMLbNwKH_h9tGlYsB
 Ot4wpmdpmOCPmT6TxdxpfPrIrgLNQDx0l2wZGKTpZITSCEHhu1Uqdto3SMtrLbGTUujqx3fax_Rl
 cHmdcQyYS55ivMYdKxMPIjmMHTuPu0k4XlhSBCu19j7zhtAanxY2GYLQtAu6JF1MQbqct7oYTF1z
 KMgbfm0tRaUZoxdZNyq7xRPmkfsKVwMpLKFbbaHTrVyMZFCzXfTG.ptN5YIw6famDKo9aBPN3Uyg
 DEvYRj6Kfj97nsmnY7euwFZxEC6Bcz_FJqqU7cS8L9O2LVPxgbSvE9eKBFzs0XHS1AxThoRo8CV1
 767Mffp1sDQSZTvNpFqmHJqSWiXLp7Rt.zz5zC4cUL6RcFY46u75AgNtK9Wxru.VnDo_ndTSU_LB
 wWrbQzZ1u6bJ68FDcRtL6Cl8zfUi_ySbbwZPCq_y7HaVnMjKHst687lxH3dwV0hspy3hbfzKtlwo
 FQOMQrX7XJPYVdLCjWlw_XVX9DxTiwOWkAUT4YwZE4_F9HE.Iwk_eooJIZCYf_EEFwamaq0kw2Bo
 ucovqmqKX8PgaVYTICjK6BwDcSGrR7bJlf1UbnvBVtwNEPDkoFd7FOC.LN.TUJ5UsDNlvXuqUMXf
 PPO5OrbKhaEsAzY3fL8yOtFAdbf9MFpSMbWUiIXv492sOK_hTohMX.5TFb3F1AR.JyoXhcQzd3Pe
 3QH7E0zCsjc2E.GotOd6eWIG2L6Kkx0AbhXLAyQfnodvTiGHmuJM11tVIsB2Q_nlyh43Z9JPxIDj
 Q3X4DEzlffyUprthwAaC1EKFB27UuPSIPyvuG0hQZIJlRdmdbqZQP6zuxw9cj1mR4Hw_s7jaxxxu
 JBwJe6kKBcvG1WE_1jdP0YoV7z_CbB6jEN_lnR5kVcx8CitnuA4hHbKWzbbFnwMcZgBAoQXYXGpW
 CsCP7kR3cNtxV6lxoJ4BX9IAYKNKJlXGL7Ok7U6YtrjTUSmL73FYgcPLMiylKYtgIzRm6QG1Ihsw
 PijtMLN3767B5i3CCoFhoFGfhK1Q7ZT_LQcfopkjxdAcd1GYmcP6lJVe209IREF_dgVY1PcHKP24
 Uuap8aIoEgcScsfvKk4TM5h8PcF4P1dabTzfRuoh4b3gjHjdgesCq5Nm3n8yl._rHPuR6HGb89JV
 .ASV0hYqJ54WxiubFUsw2L7H0IY3JwTYxxohEu1zRTAkgpTdOvCgbmgHEYmVGP4a6NFUeYBbhrPY
 8Ctbd7W.squbO3fa6I5GtBikVmONFUyXBG.KHtFC0Bp7FpV7B9n63G7m0QzFUhppn74UpUXswKqp
 BgKgKmzG..4u.Os7hMznLyir7cCayMt1Jbp7sy3kR_OU4WY_0Vp7hmPu34sfWI_1mDiRg1Kw6b8x
 61Jpoipzh__mhA5kzOmE_k7k5KHV9UEHX4F7SAJVD_BTWDSkJm4oDjG_u6_gRqcHBDfy1OqQEj4i
 1oJEqVF24tQN6kL68LkNWOsWOfF6t3trj4kbQMsXNaHN3c40l2XbOxTTqUHqE5sJdBG.Kl3_m4dH
 r7aBZdJrvLttowZMslpjBv7gyQpQ7WA2B5nRsn.9ufufP6QYvIvkgRTjBqksBouaw.r2yD98vBZK
 hyfmWJ4qgeIvGjRetkoVG283cGJMM4dT51ZmO0UZq6Ccead2AnVEo3oFsCVdTcYbZ4vl8hrPRsCE
 ULlwCvbdi8WcHkrDfqfVNcR1YfGCh_8u2ajf5xgK49tLz5IRE1O.SAyesZeZtXtaI_ida1UKliMP
 bxxBTITTQkxqyPmcq29EQK8aCJyEkD2qcd4VeiBH5TENFpdc1LuBuGWVENsRteLRf0wl9NRXlkyF
 W3w--
X-Sonic-MF: 
X-Sonic-ID: f35c313c-4174-4ff6-9b5a-658b8f3e69e8
Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.gq1.yahoo.com with HTTP; Fri, 12 Jan 2024 21:53:31 +0000
Received: by hermes--production-gq1-78d49cd6df-4xktb (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 2054c6487691fb4b5323ace5ced448ff;
          Fri, 12 Jan 2024 21:53:26 +0000 (UTC)
Content-Type: text/plain;
	charset=utf-8
List-Id: Production branch of FreeBSD source code 
List-Archive: https://lists.freebsd.org/archives/freebsd-stable
List-Help: 
List-Post: 
List-Subscribe: 
List-Unsubscribe: 
Sender: owner-freebsd-stable@freebsd.org
X-BeenThere: freebsd-stable@freebsd.org
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.300.61.1.2\))
Subject: Re: 15 & 14: ram_attach vs. its using regions_to_avail vs.
 "bus_alloc_resource" can lead to: panic("ram_attach: resource %d failed to
 attach", rid)
From: Mark Millard 
In-Reply-To: 
Date: Fri, 12 Jan 2024 13:53:15 -0800
Cc: Current FreeBSD ,
 FreeBSD-STABLE Mailing List 
Content-Transfer-Encoding: quoted-printable
Message-Id: 
References: <3CB904C2-D983-4EF7-84D3-6BDED0700B08.ref@yahoo.com>
 <3CB904C2-D983-4EF7-84D3-6BDED0700B08@yahoo.com>
 
To: Doug Rabson 
X-Mailer: Apple Mail (2.3774.300.61.1.2)
X-Rspamd-Queue-Id: 4TBZzf3QFdz4cyR
X-Spamd-Bar: ----
X-Rspamd-Pre-Result: action=no action;
	module=replies;
	Message is reply to one we originated
X-Spamd-Result: default: False [-4.00 / 15.00];
	REPLY(-4.00)[];
	ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]

On Jan 12, 2024, at 09:57, Doug Rabson  wrote:

> On Sat, 30 Sept 2023 at 08:47, Mark Millard  wrote:
> ram_attach is based on regions_to_avail but that is a problem for
> its later bus_alloc_resource use --and that can lead to:
>=20
> panic("ram_attach: resource %d failed to attach", rid);
>=20
> Unfortunately, the known example is use of EDK2 on RPi4B
> class systems, not what is considered the supported way.
> The panic happens for main [so: 15] and will happen once
> the cortex-a72 handling in 14.0-* is in a build fixed by:
>=20
>     =E2=80=A2 git: 906bcc44641d - releng/14.0 - arm64: Fix errata =
workarounds that depend on smccc Andrew Turner
>=20
> The lack of the fix leads to an earlier panic as stands.
>=20
>=20
> sys/kern/subr_physmem.c 's regions_to_avail is based on ignoring
> phys_avail and using only hwregions and exregions. In other words,
> in part:
>=20
>  * Initially dump_avail and phys_avail are identical.  Boot time =
memory
>  * allocations remove extents from phys_avail that may still be =
included
>  * in dumps.
>=20
> This means that early, dedicated memory allocations are treated
> as available for general use by regions_to_avail . The distinction
> is visible in the  boot -v output in that:
>=20
> real memory  =3D 3138154496 (2992 MB)
> Physical memory chunk(s):
> 0x00000000200000 - 0x0000002b7fffff, 727711744 bytes (177664 pages)
> 0x0000002ce3a000 - 0x0000003385ffff, 111304704 bytes (27174 pages)
> 0x000000338c0000 - 0x000000338c6fff, 28672 bytes (7 pages)
> 0x00000033a30000 - 0x00000036efffff, 55377920 bytes (13520 pages)
> 0x000000372e0000 - 0x0000003b2fffff, 67239936 bytes (16416 pages)
> 0x00000040000000 - 0x000000bb3dcfff, 2067648512 bytes (504797 pages)
> avail memory =3D 3027378176 (2887 MB)
>=20
> does not list the wider:
>=20
> 0x00000040000000 - 0x000000bfffffff
>=20
> because of phys_avail . But the earlier dump based on hwregions and
> exregions shows:
>=20
> Physical memory chunk(s):
>   0x001d0000 - 0x001effff,     0 MB (     32 pages)
>   0x00200000 - 0x338c6fff,   822 MB ( 210631 pages)
>   0x33920000 - 0x3b2fffff,   121 MB (  31200 pages)
>   0x40000000 - 0xbfffffff,  2048 MB ( 524288 pages)
> Excluded memory regions:
>   0x001d0000 - 0x001effff,     0 MB (     32 pages) NoAlloc=20
>   0x2b800000 - 0x2ce39fff,    22 MB (   5690 pages) NoAlloc=20
>   0x33860000 - 0x338bffff,     0 MB (     96 pages) NoAlloc=20
>   0x33920000 - 0x33a2ffff,     1 MB (    272 pages) NoAlloc=20
>   0x36f00000 - 0x372dffff,     3 MB (    992 pages) NoAlloc=20
>=20
> which indicates:
>=20
>   0x40000000 - 0xbfffffff
>=20
> is available as far as it is concerned.
>=20
> (Note some code works/displays in terms of: 0x40000000 - 0xc0000000
> instead.)
>=20
> For aarch64 , sys/arm64/arm64/nexus.c has a nexus_alloc_resource
> that is used as bus_alloc_resource . It ends up rejecting the
> RPi4B boot via using the result of the call in ram_attach:
>=20
>                 if (bus_alloc_resource(dev, SYS_RES_MEMORY, &rid, =
start, end,
>                     end - start, 0) =3D=3D NULL)
>                         panic("ram_attach: resource %d failed to =
attach", rid);
>=20
> as shown by the just-prior start/end pair sequence messages:
>=20
> ram0: reserving memory region:   200000-2b800000
> ram0: reserving memory region:   2ce3a000-33860000
> ram0: reserving memory region:   338c0000-338c7000
> ram0: reserving memory region:   33a30000-36f00000
> ram0: reserving memory region:   372e0000-3b300000
> ram0: reserving memory region:   40000000-c0000000
> panic: ram_attach: resource 5 failed to attach
>=20
> I do not see anything about this that looks inherently RPi*
> specific for possibly ending up with an analogous panic. So
> I expect the example is sufficient context to identify a
> problem is present, despite EDK2 use not being normal for
> RPi4B's and the like as far as FreeBSD is concerned.
>=20
> I'm not quite clear why phys_avail changes

Do not be confused by common labeling to distinct
data: Note the "phys_avail" vs. "hwregions" despite
the label "Physical memory chunk(s):" :

static void
cpu_startup(void *dummy)
{
        vm_paddr_t size;
        int i;
           printf("real memory  =3D %ju (%ju MB)\n", =
ptoa((uintmax_t)realmem),
            ptoa((uintmax_t)realmem) / 1024 / 1024);

        if (bootverbose) {
                printf("Physical memory chunk(s):\n");
                for (i =3D 0; phys_avail[i + 1] !=3D 0; i +=3D 2) {
                        size =3D phys_avail[i + 1] - phys_avail[i];
                        printf("%#016jx - %#016jx, %ju bytes (%ju =
pages)\n",
                            (uintmax_t)phys_avail[i],
                            (uintmax_t)phys_avail[i + 1] - 1,
                            (uintmax_t)size, (uintmax_t)size / =
PAGE_SIZE);
                }
        }
. . .

vs.

physmem_dump_tables(int (*prfunc)(const char *, ...) __printflike(1, 2))
{
        size_t i;
        int flags;
        uintmax_t addr, size;
        const unsigned int mbyte =3D 1024 * 1024;

        prfunc("Physical memory chunk(s):\n");
        for (i =3D 0; i < hwcnt; ++i) {
                addr =3D hwregions[i].addr;
                size =3D hwregions[i].size;
                prfunc("  0x%08jx - 0x%08jx, %5ju MB (%7ju pages)\n", =
addr,
                    addr + size - 1, size / mbyte, size / PAGE_SIZE);
        }
        prfunc("Excluded memory regions:\n");
        for (i =3D 0; i < excnt; ++i) {
                addr  =3D exregions[i].addr;
                size  =3D exregions[i].size;
                flags =3D exregions[i].flags;
                prfunc("  0x%08jx - 0x%08jx, %5ju MB (%7ju pages) %s =
%s\n",
                    addr, addr + size - 1, size / mbyte, size / =
PAGE_SIZE,
                    (flags & EXFLAG_NOALLOC) ? "NoAlloc" : "",
                    (flags & EXFLAG_NODUMP)  ? "NoDump" : "");
        }
. . .

In other words, phys_avail does not change: It is just that phys_avail
and hwregions are for different purposes and can get distinct values but
ultimately both are involved overall and a net-result has to be
generated from them.

> and why that is triggered by the 906bcc44641d commit. I'm wondering if =
it makes sense to arrange for ram_attach to happen before acpi, e.g. =
using BUS_PASS_ORDER_FIRST?

=3D=3D=3D
Mark Millard
marklmi at yahoo.com


From nobody Sat Jan 13 11:26:36 2024
X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
	by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TBx1s0b0kz57NM0;
	Sat, 13 Jan 2024 11:26:41 +0000 (UTC)
	(envelope-from lev@FreeBSD.org)
Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256
	 client-signature RSA-PSS (4096 bits) client-digest SHA256)
	(Client CN "smtp.freebsd.org", Issuer "R3" (verified OK))
	by mx1.freebsd.org (Postfix) with ESMTPS id 4TBx1r6nngz416M;
	Sat, 13 Jan 2024 11:26:40 +0000 (UTC)
	(envelope-from lev@FreeBSD.org)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim;
	t=1705145201; h=from:from:reply-to:reply-to:subject:subject:date:date:
	 message-id:message-id:to:to:cc:mime-version:mime-version:
	 content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=0Fdr4uzjf7C7zyO5OUqhOQezm/awGUebrNwjg9w/yv4=;
	b=E0DbIk/kJFtBS8Dgix8ZZSrbtP+X6EbpRLN+OUfZ8cjKK7a4DsyUJe3wJoR86zP3cZ62N9
	R+cAf/D6iFawSBK+T91DHQsRWgnsW7oixLCsa1x8KArO7tNd17QRX1sBLZjt2bMFre+/8K
	tGhGczuaMoAzK0WmrFkG8xATh3yB4yH0pDOCfZN0bZOd8fKkculrYhA78WLKM1xrqUsEzL
	Q9WFY8WIl6aCRNRXTcpme4t4GfOO//tzlqsKLZf/V5c+GafAngG/0+KOpPUzL9OQ+RYTEh
	aS38ldngoUceo9Mu6u9jMCbERBKUoutlwahqSLZFaihiEuD6XZ6RBj6Yxof/6Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org;
	s=dkim; t=1705145200; h=from:from:reply-to:reply-to:subject:subject:date:date:
	 message-id:message-id:to:to:cc:mime-version:mime-version:
	 content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=0Fdr4uzjf7C7zyO5OUqhOQezm/awGUebrNwjg9w/yv4=;
	b=h2X5rxIf2vdQel3ijBwMxu776U4o0Zs97oVVdY/cl9gHYt3MsaiJezCaTCrRaFP+kiQ7or
	mLAEEZEXbsYN8XuAmfFUSazQq+mGreZiwtwR0Byd3EErBu16M0chEWasFWgNPyN95vmwKW
	9dijG9EovtecU9imw+p+u51G0484V3xJ97G8tyOmWr16ub0HR/3LdWW0dx5ZL8tKM1wovs
	2lLBkmfLZztVMBwnWiqPK8p5yS59xexNpw3vhrPFTedwEAE8+E30iwADeuBBwovxIZpDaE
	d6IFoWNDdDX2W6+1/ogFJd1pGoEXjFgq+LX2JqW8sWj7nVwPn9PeyT0DP2nRzg==
ARC-Authentication-Results: i=1;
	mx1.freebsd.org;
	none
ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1705145201; a=rsa-sha256; cv=none;
	b=KLeQRfQrrRUlge7tbTic6gri6lwfP89IEHR/KrijJW/cYU9zEunHOJU4ER4a/bVTj7V9yT
	p1toMgTL02J7yORaCvTZweBnpQDOYtLfUY87DaPWRQsYoz+VxWlp5KWvBc9OlItuMG8vM4
	UiZyi6f2XbW3p6kziRa6qPiGvUqctaJl3c040l3qrRpdfnoRpk7e/Uj7PcJDIeyHg6h4d3
	lGCzv3GeLpXKQ2+E6QblFxuiJ65PYG73CHmslDb1/gmQMQ4g8Ei1RnEoAvq+c1X1H6tpsm
	FWjxfO7ayIndMPWtDII3L7nrbJizLwyTzlS4WtNGvXYKoRB6W1jWnG59of6QcA==
Received: from onlyone.not-for.work (onlyone.not-for.work [IPv6:2a01:4f8:201:6350::2])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
	(Client did not present a certificate)
	(Authenticated sender: lev/mail)
	by smtp.freebsd.org (Postfix) with ESMTPSA id 4TBx1r561LzGTn;
	Sat, 13 Jan 2024 11:26:40 +0000 (UTC)
	(envelope-from lev@FreeBSD.org)
Received: from [IPV6:2001:470:923f:3:4829:e77e:ddf2:a093] (unknown [IPv6:2001:470:923f:3:4829:e77e:ddf2:a093])
	(Authenticated sender: lev@serebryakov.spb.ru)
	by onlyone.not-for.work (Postfix) with ESMTPSA id 9B3C3642;
	Sat, 13 Jan 2024 14:26:36 +0300 (MSK)
Message-ID: 
Date: Sat, 13 Jan 2024 12:26:36 +0100
List-Id: Production branch of FreeBSD source code 
List-Archive: https://lists.freebsd.org/archives/freebsd-stable
List-Help: 
List-Post: 
List-Subscribe: 
List-Unsubscribe: 
Sender: owner-freebsd-stable@freebsd.org
X-BeenThere: freebsd-stable@freebsd.org
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: vfs.zfs.compressed_arc_enabled=0 is INCOMPATIBLE with L2ARC at least
 in FreeBSD 13 (Was: Crash on adding L2ARC to raidz1 pool)
From: Lev Serebryakov 
To: freebsd-fs ,
 freebsd-stable , Warner Losh ,
 Dimitry Andric 
Reply-To: lev@FreeBSD.org
References: <22167ed4-887d-4fe0-b0a6-36312ae38fea@FreeBSD.org>
Content-Language: en-US
Organization: FreeBSD
In-Reply-To: <22167ed4-887d-4fe0-b0a6-36312ae38fea@FreeBSD.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 08.01.2024 18:34, Lev Serebryakov wrote:

   I've found that all my L2ARC problems (live-locks and crashes) are result of OpenZFS bug which can not support L2ARC with un-compressed ARC (vfs.zfs.compressed_arc_enabled=0).

   It is NOT hardware-depended (and my NVMe is perfectly Ok and healthy) and could be easily reproduced under VM with all-virtual disks.

   I've opened the ticket in OpenZFS project (https://github.com/openzfs/zfs/issues/15764).

   Maybe, FreeBSD need ERRATA entry?


   Previous threads:

    [1] ZFS pool hangs (live-locks?) after adding L2ARC
    [2] Crash on adding L2ARC to raidz1 pool

-- 
// Lev Serebryakov

From nobody Sat Jan 13 17:11:18 2024
X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
	by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TC4gk0jn0z56D4G
	for ; Sat, 13 Jan 2024 17:11:30 +0000 (UTC)
	(envelope-from alex@alexburke.ca)
Received: from out-176.mta1.migadu.com (out-176.mta1.migadu.com [IPv6:2001:41d0:203:375::b0])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
	(Client did not present a certificate)
	by mx1.freebsd.org (Postfix) with ESMTPS id 4TC4gj5DYrz4kQX
	for ; Sat, 13 Jan 2024 17:11:29 +0000 (UTC)
	(envelope-from alex@alexburke.ca)
Authentication-Results: mx1.freebsd.org;
	none
Date: Sat, 13 Jan 2024 18:11:18 +0100 (GMT+01:00)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alexburke.ca;
	s=key1; t=1705165881;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=l9Q10cE/DhOa2xJsS8tDgjk5hMXRe/wD36V2UwPLpvs=;
	b=IlN008EawjA1KjwhB5z/qvsc2pQ8C97+qj+9XLmrVnXwlmhTVKYN1fD9kdYZnkJFUyFyqC
	OGXsQCT5kIZUoBZuMBri2SYCVEljsBN8CUJyS5sc+leoggHVpFAdZwh4Ys2hvgfYbne+1G
	47vFG2cl/miCDDGdfPqtaLVj0fEdRi0=
X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers.
From: Alexander Burke 
To: Lev Serebryakov 
Cc: freebsd-fs ,
	freebsd-stable ,
	Warner Losh , Dimitry Andric 
Message-ID: 
In-Reply-To: 
References: <22167ed4-887d-4fe0-b0a6-36312ae38fea@FreeBSD.org> 
Subject: Re: vfs.zfs.compressed_arc_enabled=0 is INCOMPATIBLE with L2ARC at
 least in FreeBSD 13 (Was: Crash on adding L2ARC to raidz1 pool)
List-Id: Production branch of FreeBSD source code 
List-Archive: https://lists.freebsd.org/archives/freebsd-stable
List-Help: 
List-Post: 
List-Subscribe: 
List-Unsubscribe: 
Sender: owner-freebsd-stable@freebsd.org
X-BeenThere: freebsd-stable@freebsd.org
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Correlation-ID: 
X-Migadu-Flow: FLOW_OUT
X-Rspamd-Queue-Id: 4TC4gj5DYrz4kQX
X-Spamd-Bar: ----
X-Rspamd-Pre-Result: action=no action;
	module=replies;
	Message is reply to one we originated
X-Spamd-Result: default: False [-4.00 / 15.00];
	REPLY(-4.00)[];
	ASN(0.00)[asn:16276, ipnet:2001:41d0::/32, country:FR]

Hello,

It looks like the issue is fixed in OpenZFS 2.2 (and thus in FreeBSD 14-REL=
EASE):

https://github.com/openzfs/zfs/issues/15764#issuecomment-1890491789

Cheers,
Alex
----------------------------------------

Jan 13, 2024 12:26:50 Lev Serebryakov :

> On 08.01.2024 18:34, Lev Serebryakov wrote:
>=20
> =C2=A0=C2=A0 I've found that all my L2ARC problems (live-locks and crashe=
s) are result of OpenZFS bug which can not support L2ARC with un-compressed=
 ARC (vfs.zfs.compressed_arc_enabled=3D0).
>=20
> =C2=A0=C2=A0 It is NOT hardware-depended (and my NVMe is perfectly Ok and=
 healthy) and could be easily reproduced under VM with all-virtual disks.
>=20
> =C2=A0=C2=A0 I've opened the ticket in OpenZFS project (https://github.co=
m/openzfs/zfs/issues/15764).
>=20
> =C2=A0=C2=A0 Maybe, FreeBSD need ERRATA entry?
>=20
>=20
> =C2=A0=C2=A0 Previous threads:
>=20
> =C2=A0=C2=A0=C2=A0 [1] ZFS pool hangs (live-locks?) after adding L2ARC
> =C2=A0=C2=A0=C2=A0 [2] Crash on adding L2ARC to raidz1 pool
>=20
> --=20
> // Lev Serebryakov

From nobody Sat Jan 13 17:35:46 2024
X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
	by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TC5Cq0n8Cz56Fkt;
	Sat, 13 Jan 2024 17:35:51 +0000 (UTC)
	(envelope-from lev@FreeBSD.org)
Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256
	 client-signature RSA-PSS (4096 bits) client-digest SHA256)
	(Client CN "smtp.freebsd.org", Issuer "R3" (verified OK))
	by mx1.freebsd.org (Postfix) with ESMTPS id 4TC5Cq0HXjz4n3J;
	Sat, 13 Jan 2024 17:35:51 +0000 (UTC)
	(envelope-from lev@FreeBSD.org)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim;
	t=1705167351; h=from:from:reply-to:reply-to:subject:subject:date:date:
	 message-id:message-id:to:to:cc:cc:mime-version:mime-version:
	 content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=EiPS2lCOwQauDnqZ3RAN4X+1jzsYbMp29dqLVVY2+R0=;
	b=Xhrl/zAtSR6/H9LDmCkhpECNeonYIbpY9HmIcSIzPXDWtT+G0Gp3l6+N9vgn9K8QGXhXSu
	HKpTGObtRbU4GCLbWgDIFSDyQ11qGfULdtOs7P+vdgFQMXMtJcCKRgwRAPcvwDxSu3U0VK
	jPI9/aiivLoIHyV5PDvjqnkXBRxchzaNjvCuXObpeSvdH++wkEDzL8rYaxBqS5kQS+WPgp
	LUp4Ebwgbte4F8YYbZi3+FXpOilCg11DmuqH4+lJ3FIbxCs6pYrpOgH8y7vKwfRRKYKBT+
	eWcxLwCFM3lwF5KKVS9WHwlhGEpMs1cNZrGZpAP04XjDGZy7aWEvTQLuPETv/w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org;
	s=dkim; t=1705167351; h=from:from:reply-to:reply-to:subject:subject:date:date:
	 message-id:message-id:to:to:cc:cc:mime-version:mime-version:
	 content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=EiPS2lCOwQauDnqZ3RAN4X+1jzsYbMp29dqLVVY2+R0=;
	b=wxVlDzEXKNPxqwXx7NOr5xmJekxk3FYSf24ExC2BlDTmLhj5MEAIJmMU1Pt/zawJlf3Zbt
	MaIfNLL6sJ7Q7/sAzlnS+PioUBLPkjQxkPdR+7iTMa/Ti80GF/xQmCBqCzRsnIBqykX/uR
	7mYNJ4W7xsDIPH5XCW4+cDYdCvoHi2pQc5mhmdJZkrnCNwdJNOV+05oO+5Ccifm4gs35CO
	VWKJsVBOvizqVDB/AQpAUWE4ZUIx8nu8RzIWYqlJ/LukhZNEAb4pYpiZyGREibt5Sw3mH2
	8vjlFduwbNI7MlMF+/yB49S9bEF8oKCXB5GobK+X/r3CvETZqNpysTI0ytHmJA==
ARC-Authentication-Results: i=1;
	mx1.freebsd.org;
	none
ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1705167351; a=rsa-sha256; cv=none;
	b=q+NTV4D1LLj/nISy7M42cgklnmKe5WjwYchremhzNaEyCAoiUF7ng2H6gAsDZ3Cgk02DWo
	uBmgFwxAZP+5HJjPHImYV9iNJxUgmLGhCsz36+uz15m+W9d1Gu8dOdb2OjXCRw4f5/dis1
	+bT7UIjzj/2nzaRD0Q94/XJiF4ZykmPANKyWBzl4D6/mfUlXrpxerFr65ZNDv5eWXWak+u
	esUBUHDRuV968BIa0zXlCyWDO1MjNckhXw7K9YPfG3HxF0HxJ6vOkPZWqSllwUFnIc9Fqs
	UgYBv4pggYtI6xH+jAHyq2O2zfrRDJtvngf1PvHZJdonI/SKHDFgKwVL7HyA4A==
Received: from onlyone.not-for.work (onlyone.not-for.work [IPv6:2a01:4f8:201:6350::2])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
	(Client did not present a certificate)
	(Authenticated sender: lev/mail)
	by smtp.freebsd.org (Postfix) with ESMTPSA id 4TC5Cp5jhKzNm6;
	Sat, 13 Jan 2024 17:35:50 +0000 (UTC)
	(envelope-from lev@FreeBSD.org)
Received: from [IPV6:2001:470:923f:3:4829:e77e:ddf2:a093] (unknown [IPv6:2001:470:923f:3:4829:e77e:ddf2:a093])
	(Authenticated sender: lev@serebryakov.spb.ru)
	by onlyone.not-for.work (Postfix) with ESMTPSA id F21508DE;
	Sat, 13 Jan 2024 20:35:46 +0300 (MSK)
Message-ID: <2757ebae-b543-47a3-9a77-b181a2e8bfd7@FreeBSD.org>
Date: Sat, 13 Jan 2024 18:35:46 +0100
List-Id: Production branch of FreeBSD source code 
List-Archive: https://lists.freebsd.org/archives/freebsd-stable
List-Help: 
List-Post: 
List-Subscribe: 
List-Unsubscribe: 
Sender: owner-freebsd-stable@freebsd.org
X-BeenThere: freebsd-stable@freebsd.org
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Reply-To: lev@FreeBSD.org
Subject: Re: vfs.zfs.compressed_arc_enabled=0 is INCOMPATIBLE with L2ARC at
 least in FreeBSD 13 (Was: Crash on adding L2ARC to raidz1 pool)
Content-Language: en-US
To: Alexander Burke 
Cc: freebsd-fs ,
 freebsd-stable 
References: <22167ed4-887d-4fe0-b0a6-36312ae38fea@FreeBSD.org>
 
 
From: Lev Serebryakov 
Organization: FreeBSD
In-Reply-To: 
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 13.01.2024 18:11, Alexander Burke wrote:
> Hello,
> 
> It looks like the issue is fixed in OpenZFS 2.2 (and thus in FreeBSD 14-RELEASE):
> 
> https://github.com/openzfs/zfs/issues/15764#issuecomment-1890491789
   It is my issue and my comment. GitHub's blacklion is me :)

   Problem is, such smoke tests can not prove absence of bugs. I prefer to see some comments from active OpenZFS developers, that something was fixed (or not).

   There is problem with "zfs send" when arc_compression_enabled=0, for example (see https://github.com/openzfs/zfs/discussions/15720).

-- 
// Lev Serebryakov


From nobody Sat Jan 13 22:16:38 2024
X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
	by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TCCTq1wMdz56nvr
	for ; Sat, 13 Jan 2024 22:18:23 +0000 (UTC)
	(envelope-from li-fbsd@citylink.dinoex.sub.org)
Received: from uucp.dinoex.org (uucp.dinoex.org [IPv6:2a0b:f840::12])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256
	 client-signature RSA-PSS (2048 bits) client-digest SHA256)
	(Client CN "uucp.dinoex.sub.de", Issuer "R3" (verified OK))
	by mx1.freebsd.org (Postfix) with ESMTPS id 4TCCTn3wmNz4WJy
	for ; Sat, 13 Jan 2024 22:18:21 +0000 (UTC)
	(envelope-from li-fbsd@citylink.dinoex.sub.org)
Authentication-Results: mx1.freebsd.org;
	dkim=none;
	dmarc=none;
	spf=pass (mx1.freebsd.org: domain of li-fbsd@citylink.dinoex.sub.org designates 2a0b:f840::12 as permitted sender) smtp.mailfrom=li-fbsd@citylink.dinoex.sub.org;
	arc=pass ("uucp.dinoex.org:s=M20221114:i=1")
Received: from uucp.dinoex.org (uucp.dinoex.org [IPv6:2a0b:f840:0:0:0:0:0:12])
	by uucp.dinoex.org (8.18.0.2/8.17.2) with ESMTPS id 40DMI6SC064068
	(version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO)
	for ; Sat, 13 Jan 2024 23:18:06 +0100 (CET)
	(envelope-from li-fbsd@citylink.dinoex.sub.org)
ARC-Seal: i=1; a=rsa-sha256; d=uucp.dinoex.org; s=M20221114; t=1705184289;
	cv=none; b=SmFMlEB+CDXFRLRiJUr2/Ns4e8A4UoAKrHLLuT0hR1dkWjKamS7krgAWZ8vo0Z8V8hDiVqAtJZTWzwjZXdWww1kORN4MCIm6tNwnQdWeHGgVJBMcA0b2SXUf6NIA4aY84DDIoZWm4ClEgQVDgCuZqAl/hedPd+cZInoFydh4CBE=
ARC-Message-Signature: i=1; a=rsa-sha256; d=uucp.dinoex.org; s=M20221114;
	t=1705184289; c=relaxed/simple;
	bh=y4zmz3Ahy9c43FHPrspoLkvyUMX6owRAlitSBngI3UU=;
	h=Received:Received:Received:X-Authentication-Warning:From:
	 X-Newsgroups:Subject:Date:Message-ID:References:MIME-Version:
	 Content-Type:Content-Transfer-Encoding:Injection-Date:
	 Injection-Info:User-Agent:To:X-Milter:X-Greylist; b=IDM1KsC+XZJqLbz0k4ltZd7bldBPH/L+naxBHhtxz5N5J8mKof2QLp/OcivzS+H03/6jnL7fBdNJqS65X5rKbWiI3SD96vrrGoYxnxtijmxI1aEha89Jgaap494cAd3A1d4gH3r9JWfHCZzlYaoV4BIcNqy6LYGvFCNDxoYNUVY=
ARC-Authentication-Results: i=1; uucp.dinoex.org
X-MDaemon-Deliver-To: 
Received: (from uucp@localhost)
	by uucp.dinoex.org (8.18.0.2/8.17.2/Submit) with UUCP id 40DMI6vg064067
	for freebsd-stable@freebsd.org; Sat, 13 Jan 2024 23:18:06 +0100 (CET)
	(envelope-from li-fbsd@citylink.dinoex.sub.org)
Received: from admn.intra.daemon.contact (localhost [127.0.0.1])
	by admn.intra.daemon.contact (8.17.1/8.17.1) with ESMTPS id 40DMGkMN055437
	(version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO)
	for ; Sat, 13 Jan 2024 23:16:47 +0100 (CET)
	(envelope-from li-fbsd@citylink.dinoex.sub.org)
Received: from intra.daemon.contact (news@localhost)
	by admn.intra.daemon.contact (8.17.1/8.17.1/Submit) with NNTP id 40DMGco5055369
	for freebsd-stable@freebsd.org; Sat, 13 Jan 2024 23:16:38 +0100 (CET)
	(envelope-from li-fbsd@citylink.dinoex.sub.org)
X-Authentication-Warning: admn.intra.daemon.contact: news set sender to li-fbsd@citylink.dinoex.sub.org using -f
From: "Peter 'PMc' Much" 
X-Newsgroups: m2n.fbsd.stable
Subject: Re: vfs.zfs.compressed_arc_enabled=0 is INCOMPATIBLE with L2ARC at
 least in FreeBSD 13 (Was: Crash on adding L2ARC to raidz1 pool)
Date: Sat, 13 Jan 2024 22:16:38 -0000 (UTC)
Message-ID: 
References: <22167ed4-887d-4fe0-b0a6-36312ae38fea@FreeBSD.org>
 
 
List-Id: Production branch of FreeBSD source code 
List-Archive: https://lists.freebsd.org/archives/freebsd-stable
List-Help: 
List-Post: 
List-Subscribe: 
List-Unsubscribe: 
Sender: owner-freebsd-stable@freebsd.org
X-BeenThere: freebsd-stable@freebsd.org
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Injection-Date: Sat, 13 Jan 2024 22:16:38 -0000 (UTC)
Injection-Info: admn.intra.daemon.contact;
	logging-data="48704"; mail-complaints-to="usenet@citylink.dinoex.sub.org"
User-Agent: slrn/1.0.3 (FreeBSD)
To: freebsd-stable@freebsd.org
X-Milter: Spamilter (Reciever: uucp.dinoex.org; Sender-ip: 0:0:2a0b:f840::; Sender-helo: uucp.dinoex.org;)
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (uucp.dinoex.org [IPv6:2a0b:f840:0:0:0:0:0:12]); Sat, 13 Jan 2024 23:18:09 +0100 (CET)
X-Spamd-Bar: ---
X-Spamd-Result: default: False [-4.00 / 15.00];
	ARC_ALLOW(-1.00)[uucp.dinoex.org:s=M20221114:i=1];
	NEURAL_HAM_LONG(-1.00)[-1.000];
	NEURAL_HAM_MEDIUM(-1.00)[-1.000];
	NEURAL_HAM_SHORT(-1.00)[-1.000];
	FORGED_SENDER(0.30)[pmc@citylink.dinoex.sub.org,li-fbsd@citylink.dinoex.sub.org];
	R_SPF_ALLOW(-0.20)[+mx];
	MIME_GOOD(-0.10)[text/plain];
	FROM_HAS_DN(0.00)[];
	R_DKIM_NA(0.00)[];
	RCPT_COUNT_ONE(0.00)[1];
	HAS_XAW(0.00)[];
	ASN(0.00)[asn:205376, ipnet:2a0b:f840::/32, country:DE];
	FROM_NEQ_ENVFROM(0.00)[pmc@citylink.dinoex.sub.org,li-fbsd@citylink.dinoex.sub.org];
	MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org];
	MIME_TRACE(0.00)[0:+];
	TO_MATCH_ENVRCPT_ALL(0.00)[];
	RCVD_COUNT_THREE(0.00)[4];
	PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org];
	DMARC_NA(0.00)[sub.org];
	RCVD_TLS_LAST(0.00)[];
	TO_DN_NONE(0.00)[]
X-Rspamd-Queue-Id: 4TCCTn3wmNz4WJy

On 2024-01-13, Alexander Burke  wrote:
> Hello,
>
> It looks like the issue is fixed in OpenZFS 2.2 (and thus in FreeBSD 14-RELEASE):
>
> https://github.com/openzfs/zfs/issues/15764#issuecomment-1890491789
>
> Cheers,
> Alex
> ----------------------------------------
>
> Jan 13, 2024 12:26:50 Lev Serebryakov :
>
>> On 08.01.2024 18:34, Lev Serebryakov wrote:
>> 
>>    I've found that all my L2ARC problems (live-locks and crashes) are result of OpenZFS bug which can not support L2ARC with un-compressed ARC (vfs.zfs.compressed_arc_enabled=0).
>> 
>>    It is NOT hardware-depended (and my NVMe is perfectly Ok and healthy) and could be easily reproduced under VM with all-virtual disks.
>> 
>>    I've opened the ticket in OpenZFS project (https://github.com/openzfs/zfs/issues/15764).
>> 
>>    Maybe, FreeBSD need ERRATA entry?
>> 
>> 
>>    Previous threads:
>> 
>>     [1] ZFS pool hangs (live-locks?) after adding L2ARC
>>     [2] Crash on adding L2ARC to raidz1 pool
>> 
>> -- 
>> // Lev Serebryakov
>
>

Just for the records, there is a note in my loader.conf:

> vfs.zfs.compressed_arc_enabled="1" # 27.7.17: since R11.1 l2_cksum_errs if 0

Apparently I didn't bother to open a ticket, since the general stance
was that one shouldn't mess with the defaults.

The more interesting question might be how we managed to improve from
checksum-errors (that were otherwise harmless) to "live-locks and
crashes" ;)