From owner-freebsd-current@freebsd.org Thu Dec 20 20:29:00 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6BEE71353037 for ; Thu, 20 Dec 2018 20:29:00 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic306-3.consmr.mail.bf2.yahoo.com (sonic306-3.consmr.mail.bf2.yahoo.com [74.6.132.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4EC786BFE8 for ; Thu, 20 Dec 2018 20:28:59 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: 2JFnqj0VM1kJS.8j46j4nt211ocIG31sDXNqAFsvSSEh.j3dRZRf_t7pwSOimSO 73emricSDphZ2IjDSaNhuaoO9M_w6FO4rUZ6XVO2R.onw5IfFqbtItn3whk8pLcJYTqhsXaBhRRe 4s5MKiGD2u1zTkH.XDWjl4lGDPc7Y4vDW4Z5a.9XtneDu16Pw3fQl_HGnBRRVDZdfDsjOWGuRZuT HXEAwx11SeLiPYoK0LHXfPym.901YVT86O67xAKS2sDl_g410FGwG4tb2J5EjUUQr42PxS6b6UZi sa.SZnwDSilh5W0ezB.QKwBF0sbNoxmfTGcySuuEewRxp8yLWMN19Oe4bM2d5DKdD6.Jx6zx7sRH 6LGI8fW1oedbbnTz35Xl9aDPWV8nGScSsmRBylPLmighLuUPp1pE32C308wLzq5pAxAFjuoIQhNn DFlNqu4DYeqMn4w1VtK9s4rpE5I8krKKJzwjyXX5Gxf4jDp7SwNErvjIYaUYHgg2ZIG3lDA2LHVn ObNuq8_57zB9afW_6lauaSTOaey4zkOdSQf.FlOft6ZCVgVzQ7WQ.vHTZf7fyH.kBkKHJcM68e1Q eLM2NohoO7wXL7dfpsXCHHjoVwSRWflWLSFRe.uZH1t09P4qjsXjkVO0tFSgWDNlUvrUZAgAeavD FcPYreQVl_4Oxx7_IG9ERmXgCKt6Nla3YIrUHwOWSO_u7fI4gkFs4oRgxQ9wSw6EzHViFCgIrt27 wqN.2zUztHtw9wHhqI4q_wzACiyVpCQJT.5xx5N39eRWJn4wrtnHDz7mdcBgwojy3VTnLVSf93kg lBoKkH9E.V.XBiBCCZXR6flNRGjjW6wRaD_kNkw0uwie8FZGEPzVXRE9GBhIIoXOIApJiPkRQmPG HAl46JKZsZawkWH2X7HRb51jKIZSoah5iiO.OPiYwmD4UVT4ChmK.nDBskbHi4DWvfHx9NzSOezE OF_Bs8S52fA8jUL5SZWnSmDDk5fALfrAzbtMBpNy0lWy0KE_tk_RFojM59gtugyTRyeRJio4wq2Q uGGZKX1zVeAa8Emg2YauBL1m72VaKMbqh6XehaBDDD2zHZl52wL0hw3yQS2tF_1OjmA_Htw-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.bf2.yahoo.com with HTTP; Thu, 20 Dec 2018 20:28:57 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.109]) ([67.170.167.181]) by smtp408.mail.bf1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 0d16d82eec8ee89c06abd20fccc717bb; Thu, 20 Dec 2018 20:28:54 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: Re: FreeBSD head -r341836 amd64->aarch64 cross-build of -r484783 ports via poudriere: devel/qt5-testlib hung-up during "Checking for POSIX monotonic clock" Date: Thu, 20 Dec 2018 12:28:51 -0800 References: <190D54A2-D29A-4B62-B8FF-99514DBE92A0@yahoo.com> <8A63BA63-FBCB-40DA-B031-4C7F4B1F5118@yahoo.com> To: freebsd-arm , FreeBSD Current , ports-list freebsd In-Reply-To: <8A63BA63-FBCB-40DA-B031-4C7F4B1F5118@yahoo.com> Message-Id: X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: 4EC786BFE8 X-Spamd-Bar: / X-Spamd-Result: default: False [0.95 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.00)[-0.002,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:26101, ipnet:74.6.128.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.01)[-0.008,0]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-0.17)[-0.171,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(1.64)[ip: (5.27), ipnet: 74.6.128.0/21(1.67), asn: 26101(1.34), country: US(-0.08)]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[42.132.6.74.list.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Dec 2018 20:29:00 -0000 [A amd64->armv7 cross build shows interesting hang-up behavior as well, apparently highly repeatable for my current context.] On 2018-Dec-19, at 16:21, Mark Millard wrote: > [I attached to the hung-up process with gdb and looked around a = little.] >=20 > On 2018-Dec-19, at 13:58, Mark Millard wrote: >=20 >> [Looks like a race or some such for devel/qt5-testlib: retry of = poudreire-devel >> did not hang. The other hang-up seems to be repeating and I give some = details.] >>=20 >> On 2018-Dec-19, at 12:20, Mark Millard wrote: >>=20 >>> FYI: Based on FreeBSD head -r341836 (host and target) and ports = -r484783 . This >>> was a rebuild based on going from perl5.26 to perl5.28 without = updating the ports >>> tree and from system clang 6 for the prior FreeBSD-head context used = to clang 7 >>> this time. (I'm not attributing causes here.) poudriere was using = amd64-native >>> tools for speeding up the cross-build. >>>=20 >>> # grep -r =3Dperl5=3D /etc/ ~/src.configs/ /usr/local/etc/ >>> /etc/make.conf:DEFAULT_VERSIONS+=3Dperl5=3D5.28 gcc=3D8 >>> /usr/local/etc/poudriere.d/make.conf:DEFAULT_VERSIONS+=3Dperl5=3D5.28 = gcc=3D8 >>>=20 >>> There was also a "print/texinfo:configure/runaway" but I've not = looked into >>> it at all yet and it may be a while before I do. The other ports = attempted >>> built fine as far as I can tell so far. >>>=20 >>>=20 >>> The devel/qt5-testlib failure looks like: >>>=20 >>> [00:00:13] Building 123 packages using 28 builders >>> . . . >>> [00:49:30] [10] [00:00:00] Building devel/qt5-testlib | = qt5-testlib-5.11.2 >>> . . . >>> [07:31:31] [10] [06:42:01] Saved devel/qt5-testlib | = qt5-testlib-5.11.2 wrkdir to: = /usr/local/poudriere/data/wrkdirs/FBSDFSSDjailCortexA57-default/default/qt= 5-testlib-5.11.2.tar >>> [07:31:32] [10] [06:42:02] Finished devel/qt5-testlib | = qt5-testlib-5.11.2: Failed: configure/runaway >>>=20 >>> With logs/errors/qt5-testlib-5.11.2.log showing: >>>=20 >>> Checking for POSIX monotonic clock...=20 >>> + cd = /wrkdirs/usr/ports/devel/qt5-testlib/work/qtbase-everywhere-src-5.11.2/con= fig.tests/clock-monotonic && = /wrkdirs/usr/ports/devel/qt5-testlib/work/qtbase-everywhere-src-5.11.2/bin= /qmake "CONFIG -=3D qt debug_and_release app_bundle lib_bundle" "CONFIG = +=3D shared warn_off console single_arch" = /wrkdirs/usr/ports/devel/qt5-testlib/work/qtbase-everywhere-src-5.11.2/con= fig.tests/clock-monotonic >>> + cd = /wrkdirs/usr/ports/devel/qt5-testlib/work/qtbase-everywhere-src-5.11.2/con= fig.tests/clock-monotonic && MAKEFLAGS=3D make >>> =3D>> Killing runaway build after 21600 seconds with no output >>> =3D>> Cleaning up wrkdir >>> =3D=3D=3D> Cleaning for qt5-testlib-5.11.2 >>> Killed >>> build of devel/qt5-testlib | qt5-testlib-5.11.2 ended at Wed Dec 19 = 06:45:42 PST 2018 >>> build time: 06:41:46 >>> !!! build failure encountered !!! >>>=20 >>>=20 >>> # less = /wrkdirs/usr/ports/devel/qt5-testlib/work/qtbase-everywhere-src-5.11.2/con= fig.log >>> . . . >>> test config.qtbase_corelib.libraries.librt succeeded >>> executing config test clock-monotonic >>> + cd = /wrkdirs/usr/ports/devel/qt5-testlib/work/qtbase-everywhere-src-5.11.2/con= fig.tests/clock-monotonic && = /wrkdirs/usr/ports/devel/qt5-testlib/work/qtbase-everywhere-src-5.11.2/bin= /qmake "CONFIG -=3D qt debug_and_release app_bundle lib_bundle" "CONFIG = +=3D shared warn_off console single_arch" = /wrkdirs/usr/ports/devel/qt5-testlib/work/qtbase-everywhere-src-5.11.2/con= fig.tests/clock-monotonic >>> + cd = /wrkdirs/usr/ports/devel/qt5-testlib/work/qtbase-everywhere-src-5.11.2/con= fig.tests/clock-monotonic && MAKEFLAGS=3D make >>>=20 >>>=20 >>> Some supporting details of context: >>>=20 >>> # uname -apKU >>> FreeBSD FBSDFSSD 13.0-CURRENT FreeBSD 13.0-CURRENT #5 r341836M: Tue = Dec 11 16:37:42 PST 2018 = markmi@FBSDFSSD:/usr/obj/amd64_clang/amd64.amd64/usr/src/amd64.amd64/sys/G= ENERIC-NODBG amd64 amd64 1300005 1300005 >>>=20 >>> # svnlite info /usr/ports/ | grep "Re[plv]" >>> Relative URL: ^/head >>> Repository Root: svn://svn.freebsd.org/ports >>> Repository UUID: 35697150-7ecd-e111-bb59-0022644237b5 >>> Revision: 484783 >>> Last Changed Rev: 484783 >>>=20 >>=20 >> I started poudriere up again with just the 2 needing to be rebuilt = (plus >> what depends on the 2). devel/qt5-testlib quickly completed just = fine: >>=20 >> [00:02:16] [02] [00:00:00] Building devel/qt5-testlib | = qt5-testlib-5.11.2 >> [00:04:54] [02] [00:02:38] Finished devel/qt5-testlib | = qt5-testlib-5.11.2: Success >>=20 >>=20 >> In the prior build that had the hang-ups I looked and dor = print/texinfo : >>=20 >> /wrkdirs/usr/ports/print/texinfo/work/texinfo-6.5/config.log shows = for its >> hang-up: >>=20 >> . . . >> configure:6639: checking for alloca >> configure:6676: /nxb-bin/usr/bin/cc -o conftest -O2 -pipe = -mcpu=3Dcortex-a57 -DLIBICONV_PLUG -g -fno-strict-aliasing = -mcpu=3Dcortex-a57 -DLIBICONV_PLUG -D_THREAD_SAFE conftest.c >&5 >> configure:6676: $? =3D 0 >> configure:6684: result: yes >> configure:6794: checking for C/C++ restrict keyword >> configure:6821: /nxb-bin/usr/bin/cc -c -O2 -pipe -mcpu=3Dcortex-a57 = -DLIBICONV_PLUG -g -fno-strict-aliasing -mcpu=3Dcortex-a57 = -DLIBICONV_PLUG -D_THREAD_SAFE conftest.c >&5 >> configure:6821: $? =3D 0 >> configure:6829: result: __restrict >> configure:6844: checking whether // is distinct from / >>=20 >>=20 >> In the poudriere re-run print/texinfo seems to be not progressing: >>=20 >> root 87913 0.0 0.0 12920 3668 0 I 13:29 = 0:00.06 | | `-- sh: = poudriere[FBSDFSSDjailCortexA57-default][01]: build_pkg = (texinfo-6.5_1,1) (sh) >> root 88869 0.0 0.0 12920 3660 0 I 13:29 = 0:00.00 | | `-- sh: = poudriere[FBSDFSSDjailCortexA57-default][01]: build_pkg = (texinfo-6.5_1,1) (sh) >> root 88870 0.0 0.0 10412 1848 0 IJ 13:29 = 0:00.01 | | `-- /usr/bin/make -C /usr/ports/print/texinfo = configure >> root 88974 0.0 0.0 10272 1812 0 IJ 13:30 = 0:00.00 | | `-- /bin/sh -e -c (cd = /wrkdirs/usr/ports/print/texinfo/work/texinfo-6.5 && = _LATE_CONFIGURE_ARGS=3D"" ; if [=20 >> root 89283 0.0 0.0 11160 2108 0 IJ 13:30 = 0:00.10 | | `-- /bin/sh ./configure --enable-nls = --prefix=3D/usr/local --localstatedir=3D/var --mandir=3D/usr/local/man = --di >> root 89692 0.0 0.0 227368 14504 0 IJ 13:30 = 0:00.03 | | `-- /usr/local/bin/qemu-aarch64-static = wc //dev/null >> root 89694 0.0 0.0 227424 14596 0 IJ 13:30 = 0:00.01 | | `-- /usr/local/bin/qemu-aarch64-static = wc //dev/null >> root 89695 0.0 0.0 227584 14720 0 IJ 13:30 = 0:00.01 | | `-- wc: zygote (qemu-aarch64-static) >>=20 >>=20 >> So it appears that: >>=20 >> /usr/local/bin/qemu-aarch64-static wc //dev/null >>=20 >> is hanging-up (again). >>=20 >>=20 >> Given that these are hangups I'll note that this is a Ryzen >> Threadripper 1950X context and is running under Hyper-V from >> Windows 10's 1809 update. I gave it 28 logical processors and >> have it to have the virtual NUMA topology match the topology of >> the physical hardware: "Use Hardware Topology". (Processors >> 28, NUMA nodes 2, Sockets 1, Hardware threads per core 2.) >=20 > Attaching to the stuck process via gdb and looking at the backtrace > shows: >=20 > (gdb) attach 89695 > Attaching to program: /usr/local/bin/qemu-aarch64-static, process = 89695 > [New LWP 101548 of process 89695] > [Switching to LWP 100804 of process 89695] > _pselect () at _pselect.S:3 > 3 PSEUDO(pselect) >=20 > (gdb) bt > #0 _pselect () at _pselect.S:3 > #1 0x00000000601da57f in __thr_pselect (count=3D12, = rfds=3D0x7ffffffe3650, wfds=3D0x0, efds=3D0x0, timo=3D0x0, = mask=3D0x7ffffffe3600) at /usr/src/lib/libthr/thread/thr_syscalls.c:378 > #2 0x000000006004928d in do_freebsd_select (env=3D0x860edfb18, = n=3D, rfd_addr=3D140736934698744, wfd_addr=3D, efd_addr=3D0, target_tv_addr=3D0) > at = /wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-4ef7d07/b= sd-user/freebsd/os-time.h:468 > #3 do_freebsd_syscall (cpu_env=3D0x860edfb18, num=3D93, arg1=3D12, = arg2=3D140736934698744, arg3=3D0, arg4=3D0, arg5=3D0, arg6=3D274914043516,= arg7=3D274913946564, arg8=3D6579811) > at = /wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-4ef7d07/b= sd-user/syscall.c:1106 > #4 0x000000006003903c in target_cpu_loop (env=3D0x860edfb18) at = /wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-4ef7d07/b= sd-user/aarch64/target_arch_cpu.h:100 > #5 0x0000000060038e09 in cpu_loop (env=3D0xc) at = /wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-4ef7d07/b= sd-user/main.c:121 > #6 0x0000000060039ecb in main (argc=3D, = argv=3D0x7fffffffd360) at = /wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-4ef7d07/b= sd-user/main.c:513 >=20 > (gdb) up > #1 0x00000000601da57f in __thr_pselect (count=3D12, = rfds=3D0x7ffffffe3650, wfds=3D0x0, efds=3D0x0, timo=3D0x0, = mask=3D0x7ffffffe3600) at /usr/src/lib/libthr/thread/thr_syscalls.c:378 > 378 ret =3D __sys_pselect(count, rfds, wfds, efds, timo, = mask); > (gdb) print *rfds > $1 =3D {__fds_bits =3D {2048, 0 }} >=20 > (gdb) info threads > Id Target Id Frame=20 > * 1 LWP 100804 of process 89695 _pselect () at _pselect.S:3 > 2 LWP 101548 of process 89695 _umtx_op_err () at = /usr/src/lib/libthr/arch/amd64/amd64/_umtx_op_err.S:37 >=20 > (gdb) thread 2=20 > [Switching to thread 2 (LWP 101548 of process 89695)] > #0 _umtx_op_err () at = /usr/src/lib/libthr/arch/amd64/amd64/_umtx_op_err.S:37 > 37 RSYSCALL_ERR(_umtx_op) >=20 > (gdb) bt > #0 _umtx_op_err () at = /usr/src/lib/libthr/arch/amd64/amd64/_umtx_op_err.S:37 > #1 0x00000000601d2ec0 in _thr_umtx_timedwait_uint (mtx=3D0x861027008, = id=3D, clockid=3D, abstime=3D, shared=3D) > at /usr/src/lib/libthr/thread/thr_umtx.c:236 > #2 0x00000000601dc6f8 in cond_wait_user (cvp=3D, = mp=3D0x860515b00, abstime=3D0x0, cancel=3D1) at = /usr/src/lib/libthr/thread/thr_cond.c:307 > #3 cond_wait_common (cond=3D, mutex=3D, = abstime=3D0x0, cancel=3D1) at /usr/src/lib/libthr/thread/thr_cond.c:367 > #4 0x00000000601438bc in qemu_futex_wait (ev=3D, = val=3D4294967295) at util/qemu-thread-posix.c:350 > #5 qemu_event_wait (ev=3D0x62735d10 ) at = util/qemu-thread-posix.c:445 > #6 0x000000006014a92a in call_rcu_thread (opaque=3D) = at util/rcu.c:255 > #7 0x00000000601dc376 in thread_start (curthread=3D0x860518e00) at = /usr/src/lib/libthr/thread/thr_create.c:291 > #8 0x0000000000000000 in ?? () > Backtrace stopped: Cannot access memory at address 0x7fffdfdfc000 >=20 In a separate rebuild of 124 ports print/texinfo repeated its problem. In a rebuild targeting armv7, multimedia/gstreamer1-qt hung-up and timed out. Another poudriere run also hung-up: root 33719 0.0 0.0 12920 3528 0 I 11:40 0:00.03 | = | `-- sh: poudriere[FBSDFSSDjailArmV7-default][02]: build_pkg = (gstreamer1-qt5-1.2.0_14) (sh) root 41551 0.0 0.0 12920 3520 0 I 11:43 0:00.00 | = | `-- sh: poudriere[FBSDFSSDjailArmV7-default][02]: = build_pkg (gstreamer1-qt5-1.2.0_14) (sh) root 41552 0.0 0.0 10340 1744 0 IJ 11:43 0:00.01 | = | `-- /usr/bin/make -C /usr/ports/multimedia/gstreamer1-qt = FLAVOR=3Dqt5 build root 41566 0.0 0.0 10236 1796 0 IJ 11:43 0:00.00 | = | `-- /bin/sh -e -c (cd = /wrkdirs/usr/ports/multimedia/gstreamer1-qt/work-qt5/.build; if ! = /usr/bin/env QT_SELE root 41567 0.0 0.0 89976 12896 0 IJ 11:43 0:00.07 | = | `-- /usr/local/bin/qemu-arm-static ninja -j28 -v all root 41585 0.0 0.0 102848 25056 0 IJ 11:43 0:00.10 | = | |-- /usr/local/bin/qemu-arm-static = /usr/local/bin/cmake -E cmake_autogen /wrkdirs/usr/ports/multimedia/g root 41586 0.0 0.0 102852 25072 0 IJ 11:43 0:00.11 | = | `-- /usr/local/bin/qemu-arm-static = /usr/local/bin/cmake -E cmake_autogen /wrkdirs/usr/ports/multimedia/g or as top showed it: 41552 root 1 52 0 10M 1744K 0 wait 15 0:00 = 0.00% /usr/bin/make -C /usr/ports/multimedia/gstreamer1-qt FLAVOR=3Dqt5 = build 41566 root 1 52 0 10M 1796K 0 wait 1 0:00 = 0.00% /bin/sh -e -c (cd = /wrkdirs/usr/ports/multimedia/gstreamer1-qt/work-qt5/.build; if ! = /usr/bin/env QT_SELECT=3Dqt5 QMAKEMODULES 41567 root 2 52 0 88M 13M 0 select 4 0:00 = 0.00% /usr/local/bin/qemu-arm-static ninja -j28 -v all 41585 root 2 52 0 100M 24M 0 kqread 8 0:00 = 0.00% /usr/local/bin/qemu-arm-static /usr/local/bin/cmake -E = cmake_autogen /wrkdirs/usr/ports/multimedia/gstreamer1-qt/work-qt5/. 41586 root 2 52 0 100M 24M 0 kqread 22 0:00 = 0.00% /usr/local/bin/qemu-arm-static /usr/local/bin/cmake -E = cmake_autogen /wrkdirs/usr/ports/multimedia/gstreamer1-qt/work-qt5/. So: waiting in kqread. Repeated tries have gotten the same result so far. If this keeps up, later I may be able to try a native FreeBSD boot instead of Hyper-V use on the same machine. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)