Date: Fri, 27 Oct 2023 23:00:54 -0700 From: Mark Millard <marklmi@yahoo.com> To: Glen Barber <gjb@FreeBSD.org> Cc: Colin Percival <cperciva@tarsnap.com>, freebsd-arm <freebsd-arm@freebsd.org> Subject: Re: 15-aarch64-RPI-snap Message-ID: <3B40F89C-7E5E-427F-A7A1-2D37CCC06A6F@yahoo.com> In-Reply-To: <13B64416-4334-4070-8588-71F7D938350B@yahoo.com> References: <0100018b6a9d257c-b35e4157-ba97-4aa7-988c-aba797c6d2ca-000000@email.amazonses.com> <ACBCBC83-DD61-4E0A-89DC-9DDD1B71B8DE@freebsd.org> <13B64416-4334-4070-8588-71F7D938350B@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Oct 27, 2023, at 22:24, Mark Millard <marklmi@yahoo.com> wrote:
> On Oct 27, 2023, at 21:34, Glen Barber <gjb@FreeBSD.org> wrote:
>=20
>> Relevant output from the failure follows.
>>=20
>> Glen
>> Sent from my phone.
>> Please excuse my brevity and/or typos.
>>=20
>>> On Oct 26, 2023, at 2:12 AM, Colin Percival <cperciva@tarsnap.com> =
wrote:
>>>=20
>>> =EF=BB=BFErr... that's an interesting failure...
>>>=20
>>> Disk corruption?
>>>=20
>>> Colin Percival
>>>=20
>>>> On 10/25/23 23:05, root@FreeBSD.org wrote:
>>>> =
^
>>>> ./offset.inc:16:15: error: null character ignored =
[-Werror,-Wnull-character]
>>>> =
<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U=
+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0=
000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+000=
0><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>=
<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U=
+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0=
000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+000=
0><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>=
<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U=
+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0=
000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+000=
0><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>=
<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U=
+0000><U+0000><U+0000><U+00
>>>> 00><U+0000>#undef _SA
>>>> =
^
>>>> ./offset.inc:16:16: error: null character ignored =
[-Werror,-Wnull-character]
>>>> =
<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U=
+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0=
000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+000=
0><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>=
<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U=
+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0=
000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+000=
0><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>=
<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U=
+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0=
000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+000=
0><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>=
<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U=
+0000><U+0000><U+0000><U+00
>>>> 00><U+0000>#undef _SA
>>>> =
^
>>>> ./offset.inc:16:17: error: null character ignored =
[-Werror,-Wnull-character]
>>>> =
<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U=
+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0=
000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+000=
0><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>=
<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U=
+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0=
000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+000=
0><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>=
<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U=
+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0=
000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+000=
0><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>=
<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U=
+0000><U+0000><U+0000><U+00
>>>> 00><U+0000>#undef _SA
>>>> =
^
>>>> ./offset.inc:16:18: error: null character ignored =
[-Werror,-Wnull-character]
>>>> =
<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U=
+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0=
000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+000=
0><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>=
<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U=
+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0=
000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+000=
0><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>=
<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U=
+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0=
000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+000=
0><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>=
<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U=
+0000><U+0000><U+0000><U+00
>>>> 00><U+0000>#undef _SA
>>>> =
^
>>>> ./offset.inc:16:19: error: null character ignored =
[-Werror,-Wnull-character]
>>>> =
<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U=
+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0=
000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+000=
0><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>=
<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U=
+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0=
000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+000=
0><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>=
<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U=
+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0=
000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+000=
0><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>=
<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U=
+0000><U+0000><U+0000><U+00
>>>> 00><U+0000>#undef _SA
>>>> =
=
^
>=20
> Are the above from a ZFS file system? UFS? Something else?
>=20
> Back in 2021-Nov (15..21) I had problems where ZFS was leading
> to blocks of such on aarch64, not specifically RPi*'s, various
> files but not the same ones from test to test. When I updated
> past some zfs updates on the 23rd the problem stopped.
>=20
> I also have notes from 2022-Mar (19..22) about replicating
> another example problem someone was having with files ending
> up with such blocks of bytes but the testing was on the
> ThreadRipper 1950X. (The replication showed that ccache did
> not need to be involved since I've never used it.) Again
> ZFS was part of the environment that got the replication.
> Mark Johnson fixed sys/contrib/openzfs/module/zfs/dnode.c
> during this and my ability to replicate the issue then
> stopped when I tested the patch.
>=20
> Which ever file system it is that holds the bad bytes, some
> attempted testing for repeatability of the problem could
> be of interest, some of that being on aarch64 but not on
> RPi*'s, some of it not on aarch64 at all. But it might take
> information about the context to know better what/how to
> test. That could include information about both the host and
> the jail OS versions if such is involved.
Those last notes are likely too generic, in that normally
official buildworld buildkernel activity is done on amd64
for all target platforms (last I knew). (Not that running
such builds on other platforms would be a bad problem-scope
isolation test.)
Any notes that help delimit what sort of test context
would be a reasonable partial replication of the original
context could prove useful.
>>>> fatal error: too many errors emitted, stopping now [-ferror-limit=3D]=
>>>> 20 errors generated.
>>>> *** Error code 1
>>>> Stop.
>>>> make[5]: stopped in /usr/src/sys/modules/cxgbe/if_cxgbe
>>>> *** Error code 1
>>>> Stop.
>>>> make[4]: stopped in /usr/src/sys/modules/cxgbe
>>>> *** Error code 1
>>>> Stop.
>>>> make[3]: stopped in /usr/src/sys/modules
>>>> *** Error code 1
>>>> Stop.
>>>> make[2]: stopped in /usr/obj/usr/src/arm64.aarch64/sys/GENERIC
>>>> 1115.66 real 961.57 user 60.54 sys
>>>> *** Error code 1
>>>> Stop.
>>>> make[1]: stopped in /usr/src
>>>> *** Error code 1
>>>> Stop.
>>>> make: stopped in /usr/src
>>>> rm -rf /R/ftp-stage
>>>> sending incremental file list
>>>> rsync: [sender] change_dir =
"/releng/15-aarch64-RPI-snap/R/ftp-stage/snapshots" failed: No such file =
or directory (2)
>>>> sent 19 bytes received 12 bytes 62.00 bytes/sec
>>>> total size is 0 speedup is 0.00
>>>> rsync error: some files/attrs were not transferred (see previous =
errors) (code 23) at main.c(1336) [sender=3D3.2.7]
>>>=20
>>> --=20
>>> Colin Percival
>>> FreeBSD Deputy Release Engineer & EC2 platform maintainer
>>> Founder, Tarsnap | www.tarsnap.com | Online backups for the truly =
paranoid
>>=20
>=20
=3D=3D=3D
Mark Millard
marklmi at yahoo.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3B40F89C-7E5E-427F-A7A1-2D37CCC06A6F>