Date: Sat, 19 Mar 2022 16:18:03 -0700 From: Mark Millard <marklmi@yahoo.com> To: Mark Johnston <markj@FreeBSD.org> Cc: Thomas Zander <riggs@freebsd.org>, freebsd-ports@freebsd.org, FreeBSD-STABLE Mailing List <freebsd-stable@freebsd.org>, dev-commits-ports-main@freebsd.org Subject: Re: git: 43741377b143 - main - security/openssl: Security update to 1.1.1n Message-ID: <46C5E80C-3038-4AF4-9E37-103586E3A318@yahoo.com> In-Reply-To: <FCD8C1A3-CEEF-4406-977F-C42306683F04@yahoo.com> References: <2E632408-4D4E-426C-8DC7-A1024A1D328D.ref@yahoo.com> <2E632408-4D4E-426C-8DC7-A1024A1D328D@yahoo.com> <CAFU734xvJB3a=0TE=bQvjp7wN=8iKP1AbuEqrP1MF8vOp1fb7Q@mail.gmail.com> <DFE73A51-BABA-4138-93CA-60245C16EA86@yahoo.com> <YjY03OHBQlSDqh94@framework> <4370EEAF-A332-4DEB-8E21-B24689857463@yahoo.com> <FCD8C1A3-CEEF-4406-977F-C42306683F04@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2022-Mar-19, at 14:34, Mark Millard <marklmi@yahoo.com> wrote:
> On 2022-Mar-19, at 14:24, Mark Millard <marklmi@yahoo.com> wrote:
>=20
>> On 2022-Mar-19, at 12:54, Mark Johnston <markj@FreeBSD.org> wrote:
>>=20
>>> On Sat, Mar 19, 2022 at 12:00:20PM -0700, Mark Millard wrote:
>>>> On 2022-Mar-19, at 11:07, Thomas Zander <riggs@freebsd.org> wrote:
>>>>=20
>>>>> On Sat, 19 Mar 2022 at 18:32, Mark Millard <marklmi@yahoo.com> =
wrote:
>>>>>> May be report to Mark J. how to run the same test builds
>>>>>> that failed for -p8 but worked for -p7?
>>>>>=20
>>>>> Sure, good point.
>>>>> A build that reliably causes broken packages on p8 but not on p7 =
for
>>>>> me is running:
>>>>>=20
>>>>> poudriere testport -o multimedia/mplayer -j <13.0-amd64-jail here>
>>>>>=20
>>>>> This caused the broken png and python packages when they were =
built as
>>>>> dependencies.
>>>>> In poudriere.conf I set this:
>>>>> DISTFILES_CACHE=3D/vcache/distfiles
>>>>> CCACHE_DIR=3D/vcache/ccache
>>>>> ALLOW_MAKE_JOBS=3Dyes
>>>>>=20
>>>>> The ALLOW_MAKE_JOBS should increase the number of parallel IO
>>>>> operations in-flight on the pool, maybe this increases the =
likelihood
>>>>> of triggering the issue?
>>>>> The DISTFILES_CACHE and CCACHE_DIR are in the same zfs pool as
>>>>> /poudriere, not sure if this is relevant.
>>>>> The zfs pool is a single disk, no raid, mirror or anything fancy.
>>>>=20
>>>> On a ThreadRipper 1950X, PCIe Optane storage, 128 GiBytes of
>>>> RAM, I've used bectl to boot the 13.0_RELEASE-p8 environment
>>>> and have started:
>>>>=20
>>>> poudriere testport -o multimedia/mplayer -j13_0R-amd64-bulk_a
>>>>=20
>>>> where the jail had nothing built in it at the start. So:
>>>>=20
>>>> [00:00:08] Building 271 packages using up to 32 builders
>>>>=20
>>>> The primary difference is that I've never used ccache and
>>>> did not try to do so here. The "zfs pool is a single disk,
>>>> no raid, mirror or anything fancy" is accurate, as is the
>>>> use of ALLOW_MAKE_JOBS=3D .
>>>>=20
>>>> That did not take long . . .
>>>>=20
>>>> It proves that ccache is not required. Also some files
>>>> seem to get only small blocks of zero-bytes, others
>>>> large ones. But I've not checked for the null characters
>>>> being at the end instead of earlier in the file.
>>>=20
>>> I still am not able to reproduce it. I think it's indeed a =
concurrency
>>> problem, and I found a possible culprit. Mark or Thomas, if you're =
able
>>> to build a new kernel from the releng/13.0 branch and test it, could =
you
>>> please try this patch?
>>>=20
>>=20
>> Sure. (I build ports in a way that allows large load
>> averages relative to the hardware-thread count. I also
>> have a lot of swap configured. I avoid significant use
>> of tmpfs.)
>>=20
>>> diff --git a/sys/contrib/openzfs/module/zfs/dnode.c =
b/sys/contrib/openzfs/module/zfs/dnode.c
>>> index 8592c5f8c3a9..b69ba68ec780 100644
>>> --- a/sys/contrib/openzfs/module/zfs/dnode.c
>>> +++ b/sys/contrib/openzfs/module/zfs/dnode.c
>>> @@ -1661,7 +1661,7 @@ dnode_is_dirty(dnode_t *dn)
>>> mutex_enter(&dn->dn_mtx);
>>>=20
>>> for (int i =3D 0; i < TXG_SIZE; i++) {
>>> - if (list_head(&dn->dn_dirty_records[i]) !=3D NULL) {
>>> + if (multilist_link_active(&dn->dn_dirty_link[i])) {
>>> mutex_exit(&dn->dn_mtx);
>>> return (B_TRUE);
>>> }
>>>=20
>>=20
>> Change made.
>> Rebuilt.
>> Reinstalled.
>> Rebooted into the 13_0R-amd64 be area.
>> Bulk build started.
>> Bulk build completed.
>> (Took longer because I let it run to completion.)
>>=20
>> No explicit reports of null characters. The same 2 ports that
>> failed before, not reporting zero-byte issues, failed again.
>> Likely independent issues:
>>=20
>> [00:28:28] Failed ports: security/libgcrypt:build =
print/freetype2:package
>=20
> These are what happens for WITH_DEBUG=3D style builds. Turns
> out that the *make.conf files from my last bulk -a experiment
> were still in place and were causing WITH_DEBUG=3D builds. (Not
> my normal context.)
>=20
> I'll disable that and rerun the bulk from scratch.
Done. No failures.
>> Overall it skipped something like 54 ports.
>>=20
>> libgcrypt-1.9.4.log . . .
>>=20
>> --- basic.o ---
>> basic.c:315:16: error: inline assembly requires more registers than =
available
>> asm volatile("movdqu %[data0], %%xmm0\n"
>> ^
>> basic.c:315:16: error: inline assembly requires more registers than =
available
>> basic.c:315:16: error: inline assembly requires more registers than =
available
>> basic.c:315:16: error: inline assembly requires more registers than =
available
>> basic.c:315:16: error: inline assembly requires more registers than =
available
>> basic.c:315:16: error: inline assembly requires more registers than =
available
>> basic.c:315:16: error: inline assembly requires more registers than =
available
>> basic.c:315:16: error: inline assembly requires more registers than =
available
>> basic.c:315:16: error: inline assembly requires more registers than =
available
>> basic.c:315:16: error: inline assembly requires more registers than =
available
>> basic.c:315:16: error: inline assembly requires more registers than =
available
>> basic.c:315:16: error: inline assembly requires more registers than =
available
>> --- mpitests ---
>> . . .
>> --- basic.o ---
>> basic.c:315:16: error: inline assembly requires more registers than =
available
>> basic.c:315:16: error: inline assembly requires more registers than =
available
>> basic.c:315:16: error: inline assembly requires more registers than =
available
>> basic.c:315:16: error: inline assembly requires more registers than =
available
>> basic.c:315:16: error: inline assembly requires more registers than =
available
>> basic.c:315:16: error: inline assembly requires more registers than =
available
>> basic.c:315:16: error: inline assembly requires more registers than =
available
>> fatal error: too many errors emitted, stopping now [-ferror-limit=3D]
>>=20
>>=20
>> freetype2-2.11.1.log . . .
>>=20
>> =3D=3D=3D> Building package for freetype2-2.11.1
>> pkg-static: Unable to access file =
/wrkdirs/usr/ports/print/freetype2/work/stage/usr/local/lib/libfreetype.so=
:No such file or directory
>> pkg-static: Unable to access file =
/wrkdirs/usr/ports/print/freetype2/work/stage/usr/local/lib/libfreetype.so=
.6:No such file or directory
>> pkg-static: Unable to access file =
/wrkdirs/usr/ports/print/freetype2/work/stage/usr/local/lib/libfreetype.so=
.6.18.1:No such file or directory
>> *** Error code 1
>>=20
>>=20
>> I think this indicates that your fix works.
>>=20
>>=20
>>=20
>> For reference:
>>=20
>> # ~/fbsd-based-on-what-commit.sh -C /usr/ports/
>> branch: main
>> merge-base: 0c4fe38bd1b9e3ede128751db17a8a2d1bdffe6f
>> merge-base: CommitDate: 2022-03-15 16:15:33 +0000
>> 0c4fe38bd1b9 (HEAD -> main, freebsd/main, freebsd/HEAD) misc/urh: =
Update 2.9.2 -> 2.9.3
>> n577366 (--first-parent --count for merge-base)
>=20
Avoiding WITH_DEBUG=3D use, everything built, no error logs.
SET PORTS JAIL BUILD STATUS =
QUEUE BUILT FAIL SKIP IGNORE FETCH REMAIN TIME LOGS
. . .
- default 13_0R-amd64-bulk_a 2022-03-19_14h35m20s done =
272 272 0 0 0 0 0 01:33:16 =
/usr/local/poudriere/data/logs/bulk/13_0R-amd64-bulk_a-default/2022-03-19_=
14h35m20s
. . .
SIDE NOTE for multimedia/mplayer :
=3D=3D=3D=3D> Running Q/A tests (stage-qa)
Error: /usr/local/bin/mplayer is linked to /usr/local/lib/libjpeg.so.8 =
from graphics/jpeg-turbo but it is not declared as a dependency
Warning: you need USES+=3Djpeg
Error: /usr/local/bin/mplayer is linked to =
/usr/local/lib/libdvdread.so.8 from multimedia/libdvdread but it is not =
declared as a dependency
Warning: you need LIB_DEPENDS+=3Dlibdvdread.so:multimedia/libdvdread
Error: /usr/local/bin/mplayer is linked to /usr/local/lib/libcairo.so.2 =
from graphics/cairo but it is not declared as a dependency
Warning: you need USE_GNOME+=3Dcairo
Error: /usr/local/bin/mplayer is linked to =
/usr/local/lib/libgdk_pixbuf-2.0.so.0 from graphics/gdk-pixbuf2 but it =
is not declared as a dependency
Warning: you need USE_GNOME+=3Dgdkpixbuf2
Error: /usr/local/bin/mplayer is linked to =
/usr/local/lib/libharfbuzz.so.0 from print/harfbuzz but it is not =
declared as a dependency
Warning: you need LIB_DEPENDS+=3Dlibharfbuzz.so:print/harfbuzz
Warning: Possible REINPLACE_CMD issues:
- - REINPLACE_CMD ran, but did not modify file contents: =
debian/daily-build.sh
- - REINPLACE_CMD ran, but did not modify file contents: version.sh
- - REINPLACE_CMD ran, but did not modify file contents: =
DOCS/tech/release.sh
- - REINPLACE_CMD ran, but did not modify file contents: =
tests/faterun.sh
- - REINPLACE_CMD ran, but did not modify file contents: =
tests/refupdate.sh
- - REINPLACE_CMD ran, but did not modify file contents: =
help/help_create.sh
- - REINPLACE_CMD ran, but did not modify file contents: =
help/help_check.sh
- - REINPLACE_CMD ran, but did not modify file contents: =
vidix/dhahelperwin/NMAKEFILE
- - REINPLACE_CMD ran, but did not modify file contents: =
TOOLS/subsearch.sh
- - REINPLACE_CMD ran, but did not modify file contents: =
TOOLS/checktree.sh
- - REINPLACE_CMD ran, but did not modify file contents: =
TOOLS/divx2svcd.sh
- - REINPLACE_CMD ran, but did not modify file contents: =
TOOLS/mencvcd.sh
- - REINPLACE_CMD ran, but did not modify file contents: =
TOOLS/coverity.sh
- - REINPLACE_CMD ran, but did not modify file contents: =
TOOLS/qepdvcd.sh
- - REINPLACE_CMD ran, but did not modify file contents: =
TOOLS/mpconsole.sh
- - REINPLACE_CMD ran, but did not modify file contents: =
TOOLS/midentify.sh
- - REINPLACE_CMD ran, but did not modify file contents: =
TOOLS/binary_codecs.sh
- - REINPLACE_CMD ran, but did not modify file contents: =
TOOLS/psnr-video.sh
- - REINPLACE_CMD ran, but did not modify file contents: =
TOOLS/aconvert.sh
- - REINPLACE_CMD ran, but did not modify file contents: =
TOOLS/mplmult.sh
=3D=3D=3D
Mark Millard
marklmi at yahoo.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?46C5E80C-3038-4AF4-9E37-103586E3A318>