Date: Sat, 2 Oct 2021 17:31:18 -0700 From: Mark Millard via freebsd-ports <freebsd-ports@freebsd.org> To: freebsd-ports@freebsd.org Subject: Re: poudriere-devel bulk -a python-related www/chromium build failures (aarch64 context) Message-ID: <7C864817-7C45-439A-9DD0-5C75CA5455D7@yahoo.com> In-Reply-To: <497EAE44-E8D0-4193-B50B-27048D4357FF@yahoo.com> References: <497EAE44-E8D0-4193-B50B-27048D4357FF@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2021-Sep-23, at 15:07, Mark Millard <marklmi@yahoo.com> wrote: > So far my attempts at an aarch64 native bulk -a have had > www/chromium fail with: >=20 > . . . > python3 ../../build/util/python2_action.py = ../../third_party/blink/renderer/bindings/scripts/generate_bindings.py = --web_idl_database = gen/third_party/blink/renderer/bindings/web_idl_database.pickle --r > oot_src_dir ../../ --root_gen_dir gen --output_core_reldir = third_party/blink/renderer/bindings/core/v8/ --output_modules_reldir = third_party/blink/renderer/bindings/modules/v8/ enumeration callback_fun > ction callback_interface interface namespace typedef union > Traceback (most recent call last): > File = "/wrkdirs/usr/ports/www/chromium/work/chromium-92.0.4515.159/buildtools/fr= eebsd/clang-format", line 34, in <module> > stdout, stderr =3D proc.communicate(input=3Dcontents) > File "/usr/local/lib/python3.8/subprocess.py", line 1028, in = communicate > stdout, stderr =3D self._communicate(input, endtime, timeout) > File "/usr/local/lib/python3.8/subprocess.py", line 1848, in = _communicate > input_view =3D memoryview(self._input) > TypeError: memoryview: a bytes-like object is required, not 'str' > Traceback (most recent call last): > File = "/wrkdirs/usr/ports/www/chromium/work/chromium-92.0.4515.159/buildtools/fr= eebsd/clang-format", line 34, in <module> > stdout, stderr =3D proc.communicate(input=3Dcontents) > File "/usr/local/lib/python3.8/subprocess.py", line 1028, in = communicate > stdout, stderr =3D self._communicate(input, endtime, timeout) > File "/usr/local/lib/python3.8/subprocess.py", line 1848, in = _communicate > input_view =3D memoryview(self._input) > TypeError: memoryview: a bytes-like object is required, not 'str' > . . . (more such tracebacks) . . . >=20 > I do not see such a failure in the likes of: >=20 > = http://ampere2.nyi.freebsd.org/data/main-arm64-default/p5f5c0fe00a09_s3fcb= de5e88/logs/chromium-92.0.4515.159_2.log >=20 > I've yet to figure out why my build attempts are different. >=20 > For reference: >=20 > The deafult lvm is llvm12. >=20 > # pwd > /usr/ports > # ~/fbsd-based-on-what-commit.sh=20 > branch: main > merge-base: a59b4fa83271ce1a7610973b2a5f3f0362e4bfba > merge-base: CommitDate: 2021-09-21 01:12:16 +0000 > a59b4fa83271 (HEAD -> main, freebsd/main, freebsd/HEAD) = security/py-scramp: Update to 1.4.1 > n558469 (--first-parent --count for merge-base) >=20 > # uname -apKU > FreeBSD CA72_16Gp_ZFS 14.0-CURRENT FreeBSD 14.0-CURRENT #12 = main-n249019-0637070b5bca-dirty: Tue Aug 31 02:24:20 PDT 2021 = root@CA72_16Gp_ZFS:/usr/obj/BUILDs/main-CA72-nodbg-clang/usr/main-src/arm6= 4.aarch64/sys/GENERIC-NODBG-CA72 arm64 aarch64 1400032 1400032 >=20 > I do not normally build chromium or do bulk -a runs. > My activity is exploratory to see what problems I'd > run into and if I'd need adjustments to do bulk -a > runs successfully. (It has lead to the (root) file > system being made larger, for example. I also disabled > lang/ratfor because the port does not deal with > platforms that have char as unsigned but the port is > coded requiring a form of signed char. This avoided > unbounded looping and a timeout.) When I look at: = /wrkdirs/usr/ports/www/chromium/work/chromium-92.0.4515.159/buildtools/fre= ebsd/clang-format I see (in part): . . . contents =3D '' if '-' in args or not inputfiles: contents =3D sys.stdin.read() . . . proc =3D subprocess.Popen( ['clang-format'] + args, stdin=3Dsubprocess.PIPE, stdout=3Dsubprocess.PIPE, stderr=3Dsubprocess.PIPE, env=3Denv) stdout, stderr =3D proc.communicate(input=3Dcontents) . . . Which looks to me to be passing a str, not a bytes or bytearray and not None. Also Popen.stdin is documented with: QUOTE Popen.stdin If the stdin argument was PIPE, this attribute is a writeable stream = object as returned by open(). If the encoding or errors arguments were = specified or the universal_newlines argument was True, the stream is a text = stream, otherwise it is a byte stream. If the stdin argument was not PIPE, this attribute is None. END QUOTE So, if I gather correctly, stdin ends up being a byte stream acording to the documenation. proc.communicate is documented with "If streams were opened in text mode, input must be a string. Otherwise, it must be bytes." as well. And that in turn involves: /usr/local/lib/python3.8/subprocess.py has: def communicate(self, input=3DNone, timeout=3DNone): . . . stdout, stderr =3D self._communicate(input, endtime, = timeout) . . . So inpput still seems to be a str for the following: def _communicate(self, input, endtime, orig_timeout): . . . self._save_input(input) =20 if self._input: input_view =3D memoryview(self._input) . . . but self._input need not be. This is where: def _save_input(self, input): # This method is called from the _communicate_with_*() = methods # so that if we time out while communicating, we can = continue # sending input if we retry. if self.stdin and self._input is None: self._input_offset =3D 0 self._input =3D input if input is not None and self.text_mode: self._input =3D = self._input.encode(self.stdin.encoding, self.stdin.errors) The implication of the error would seem to be that the encode did not happen and, so, either self.stdin and self._input is None was false or: self.text_mode was false (and it appears to be the later). This fits with the stdin being a byte stream but input being a str --and so the error report from memoryview. But I still have no clue why the FreeBSD server builds of chromium do not end up with such an error report. Note: I've duplicated the failure in my amd64 context. The issue is not aarch64 specific in my environment. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7C864817-7C45-439A-9DD0-5C75CA5455D7>