From nobody Sat Mar 19 18:06:51 2022 X-Original-To: dev-commits-ports-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 41C461A2C7A1; Sat, 19 Mar 2022 18:06:51 +0000 (UTC) (envelope-from rene@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4KLTNW1Jpsz4gXt; Sat, 19 Mar 2022 18:06:51 +0000 (UTC) (envelope-from rene@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1647713211; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Mspww2HPiWN44iLHjeb+XA1CkhowsYr9Lr70irqDc4I=; b=Px18zMQThmzazYmUrYPdYj/Fl1MhliqPcCliygu8zlT/VkwZwN7CR3PmZniCI4naKjXWBI OMHetfMvFE/oFaNTybChNMGWzO+qcHlKq0Csb5CMPktPy1yTNpk4UMNx6WN+zhsN9oDgzT X2ugcwLhnRyBe21w7/U+VA8+l4m+hm1IIYsOnnUpYLGrJyEoLxw/6LjpF+/hUP6f2/+0Ak XRirQRozT9FhMtipenRhQGnGUGc1DuZ2M36p4R5w522aWFL6Lxy6fHDGbMKhgLwTWiFlp4 +s/HL1ivlhBxk+jl+dR/oE97pqhYY2+9K3hq5yjFYAuVO2LsVtjs84yb2cCoIw== Received: by freefall.freebsd.org (Postfix, from userid 1185) id 1447B13B65; Sat, 19 Mar 2022 18:06:51 +0000 (UTC) Date: Sat, 19 Mar 2022 18:06:51 +0000 From: Rene Ladan To: Mark Johnston Cc: Thomas Zander , Matthias Fechner , Christoph Moench-Tegeder , Bernard Spil , "ports-committers@FreeBSD.org" , dev-commits-ports-all@freebsd.org, dev-commits-ports-main@freebsd.org, secteam@freebsd.org Subject: Re: git: 43741377b143 - main - security/openssl: Security update to 1.1.1n Message-ID: References: <579f562b-8add-d3f7-77c9-1a6126bd282b@FreeBSD.org> List-Id: Commit messages for all branches of the ports repository List-Archive: https://lists.freebsd.org/archives/dev-commits-ports-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-ports-all@freebsd.org X-BeenThere: dev-commits-ports-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1647713211; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Mspww2HPiWN44iLHjeb+XA1CkhowsYr9Lr70irqDc4I=; b=khZ8NNXtNAn/8XWpOkzQ2IYsJ/WA4jIlQZWYmidLsq5903qwDCQD0IidyN8Rqh9B3m22KS cvHi4JSbfRQNco5pTJiuStPYQkg4yOBlVDOONqlVsk+8MXtmsRNECpr61MtoWjUt7UIPXc DgJaC0sUlebD7+BK07DhvQquV2z1+38iOAaOHTLfPajCh0eus41ufr9uKmGOJuwaIAI3BB KFNbGeGEtxsl9l1Lky1DsTa8N2iCMYcz/16hyq6DMwXwyUGLyrjrOINKdZLittJSw5+Ejm n68o39DzrLdPzCSrJbr2fmndfqt7aCqyWKrxbkIycN6IX/4uAQD2frVwJPqtSg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1647713211; a=rsa-sha256; cv=none; b=FEfu0ly/JGUeYnX8hhYIaGaqoWhyTFcXnlMl3LfEni4xG17U9+MAnor4EpkYJ2UqAGT0sg jBZKvqzapI0Vet/gDhzmX99N6jSlQ0lSdp605QR93Nd6lqiJwiI7jfsIsMBfKLioDG+DGw GS60/jFv1Gk6CaSehlZ2M6cMu883GiWP389HWvdkd5fR1SnnDMpQYNeKucQUX6IUQDGCPF jHv23GFovosvL+ayBMF/Z0ygQIyaYfxFl5z4itjHfX9ZlNZWN3GA0z4xLo7OHNcOXDol7I 7QPed0Xr+QFZ6/c0spFzH84TBNmjGFE7+nVDAen7MlH0CCia5vNbjXtV/Tu/mQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N On Sat, Mar 19, 2022 at 12:42:59PM -0400, Mark Johnston wrote: > On Sat, Mar 19, 2022 at 12:40:45PM +0100, Thomas Zander wrote: > > On Sat, 19 Mar 2022 at 12:11, Rene Ladan wrote: > > > > > > On Sat, Mar 19, 2022 at 11:04:58AM +0100, Thomas Zander wrote: > > > > On Sat, 19 Mar 2022 at 09:00, Matthias Fechner wrote: > > > > > > > > > I can confirm now, the problem is definitely related to the -p8 update. > > > > > I rolled back now to -p7 using `freebsd-update rollback`. > > > > > [...] > > > > > System is now up and running again. > > > > > This all works even if poudriere jail is using -p8. No need to downgrade the jail/base version poudriere is using. > > > > > It is caused by the kernel so the ZFS patch seems to be broken and -p8 should maybe not rolled out to not break more systems of users. > > > > > > > > On top of "stop rollout", there is the question how to identify the > > > > broken files for the users who have already upgraded to -p8. A `zpool > > > > scrub` presumably won't help. > > > > > > I think it also applies to 13.1-BETA2 ? > > > > > > Should we involve/CC some src committers? > > > > I have just rolled back to -p7 and run a number of test builds in > > poudriere (the jails still have the -p8 user land). I see the same as > > Matthias and Christoph, the rollback to the -p7 kernel/zfs resolved > > the build problems, there are no NUL byte files generated anymore. > > Adding markj@ to the discussion. Mark, the TLDR so far: > > - One of the zfs patches in -p8 seems to cause erroneous writes. > > - We noticed because of many build failures with poudriere (presumably > > highly io-loaded during build). > > - Symptom: Production of files with large runs of NUL-bytes. > I'm not sure if it is the same problem or something that popped up at the same time, but since on a 13.1-BETA2-amd64 host updated with freebsd-update and a poudriere 13.1-BETA2-armv6x jail built from git+https building lang/python38 reliably hangs in the stage phase until poudriere decides to kill it: rene@e17:~ % ps axww | grep python38 52652 8 S+ 0:00.00 grep python38 12503 12 I 0:00.03 sh: poudriere[13_1-armv6x-quarterly][01]: build_pkg (python38-3.8.12_1) (sh) 42562 12 I 0:00.00 sh: poudriere[13_1-armv6x-quarterly][01]: build_pkg (python38-3.8.12_1) (sh) 42563 12 IJ 0:00.02 /usr/bin/make -C /usr/ports/lang/python38 stage 42622 12 IJ 0:00.02 /usr/bin/make -f Makefile INSTALL_SHARED=install -s -m 0644 DESTDIR=/wrkdirs/usr/ports/lang/python38/work/stage altinstall 46066 12 IJ 0:05.91 /usr/local/bin/qemu-arm-static ./python -E -Wi -OO /wrkdirs/usr/ports/lang/python38/work/stage/usr/local/lib/python3.8/compileall.py -j0 -d /usr/local/lib/python3.8 -f -x bad_coding|badsyntax|site-packages|lib2to3/tests/data /wrkdirs/usr/ports/lang/python38/work/stage/usr/local/lib/python3.8 46101 12 IJ 0:04.07 /usr/local/bin/qemu-arm-static ./python -E -Wi -OO /wrkdirs/usr/ports/lang/python38/work/stage/usr/local/lib/python3.8/compileall.py -j0 -d /usr/local/lib/python3.8 -f -x bad_coding|badsyntax|site-packages|lib2to3/tests/data /wrkdirs/usr/ports/lang/python38/work/stage/usr/local/lib/python3.8 46102 12 IJ 0:04.09 /usr/local/bin/qemu-arm-static ./python -E -Wi -OO /wrkdirs/usr/ports/lang/python38/work/stage/usr/local/lib/python3.8/compileall.py -j0 -d /usr/local/lib/python3.8 -f -x bad_coding|badsyntax|site-packages|lib2to3/tests/data /wrkdirs/usr/ports/lang/python38/work/stage/usr/local/lib/python3.8 46104 12 IJ 0:04.28 /usr/local/bin/qemu-arm-static ./python -E -Wi -OO /wrkdirs/usr/ports/lang/python38/work/stage/usr/local/lib/python3.8/compileall.py -j0 -d /usr/local/lib/python3.8 -f -x bad_coding|badsyntax|site-packages|lib2to3/tests/data /wrkdirs/usr/ports/lang/python38/work/stage/usr/local/lib/python3.8 rene@e17:~ % > I've had zero luck reproducing this locally. I built several hundred > ports, including textproc/py-pystemmer mentioned elsewhere in the > thread, without any failures or instances of zero-filled files. Another > member of secteam also hasn't been able to trigger any build failures on > -p8. Any hints on a reproducer would be useful. > I haven't tested anything on 13.0-p8 yet. > We can simply push a -p9 which reverts EN-22:10 and :11, but of course > it would be preferable to precisely identify the problem. Perhaps there are some tests/commands I can run? Regards, René