From owner-freebsd-hackers@freebsd.org Sat Apr 28 20:11:51 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4CAF6FB8EE2 for ; Sat, 28 Apr 2018 20:11:51 +0000 (UTC) (envelope-from marklmi26-fbsd@yahoo.com) Received: from sonic311-14.consmr.mail.bf2.yahoo.com (sonic311-14.consmr.mail.bf2.yahoo.com [74.6.131.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8ED5476D12 for ; Sat, 28 Apr 2018 20:11:50 +0000 (UTC) (envelope-from marklmi26-fbsd@yahoo.com) X-YMail-OSG: j2.v8dQVM1kqMRdAPs50y.kYJtc.a8xajQd1QNs_0ObeiIaqlUlb0tctzS3VCjQ BVdrFEEvXSsjxciJlksOoc0NP55a6z_ipHdlHhEqXp88V6ZJAZLgU95Xx67Wfr1hEekmb2.mIGou jrVG0n9uf2i99uUOamkpDO01NcXadyGnQvKXNcm0bNo8.VWxWk2socRbf0vaR77UmtnE8hw_OaU1 6Q.zpsrHfh.ANSR8AFuWyyAvuf.7tnJCAZGJLqI3QMxPQZCgoYKYsHGpfh0BQSlyy1if5JIxbnAf ZF8g0pYviPnLpPa6QA.RGdmCUaf2sCZ42IumUeq8PCn1mS4.om8Wlkj0CTkPIo49m7J0bQf6O7pU Uox7s8PLPAbYWYRsbPESFDM59YrwwRN.TCpzrj5NL8FVPU.pvY9iqIwae39vnab.X87o5PLPulOD mMR0SOpd1QlPwDmAzhYib8glRqxuQph5KknS1dYClqA9JW9ULZRc1cR3jbcsky1a3HG_Xb7dMPWY TmEmKFfaSDru.OjA.U.R8.uIPm4lSWRz0p_Kk6dis7Mc.e_qoty_WImvxLzvgdypRIerkbemz90S zlrdNeoLPN0NKS5lYjEDX4i9wV56R7hJ1OLLMeE4shwoappIQ.hYdCXPh7ZXnNLgaEjQbSCec Received: from sonic.gate.mail.ne1.yahoo.com by sonic311.consmr.mail.bf2.yahoo.com with HTTP; Sat, 28 Apr 2018 20:11:44 +0000 Received: from c-76-115-7-162.hsd1.or.comcast.net (EHLO [192.168.1.25]) ([76.115.7.162]) by smtp417.mail.bf1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID f8e257e151d9ef51f74ae6dda64803c7; Sat, 28 Apr 2018 20:11:41 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 11.3 \(3445.6.18\)) Subject: Interesting process hang on head -r333079 aarch64 (Pine64+ 2GB)? Message-Id: <77D17217-FC4C-40E6-BD1B-2E6ABE0458D8@yahoo.com> Date: Sat, 28 Apr 2018 13:11:39 -0700 To: freebsd-arm@freebsd.org, "freebsd-hackers@freebsd.org" X-Mailer: Apple Mail (2.3445.6.18) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Apr 2018 20:11:51 -0000 Using a new instance of top to watch and older one, the old one shows (no longer updating): up 0+13:47:14 09:58:51 and the new one shows for it: PID USERNAME THR PRI NICE SIZE RES SWAP STATE C TIME = CPU COMMAND . . . 834 root 1 20 0 13652K 1972K 0K select 3 2:21 = 0.00% top -CawSores =20 when the new one shows: up 0+16:56:14 13:07:51 So, over 3 hours later. I'll note that about 2 hours earlier it showed a larger RES figure, the = same figure the old top display shows: PID USERNAME THR PRI NICE SIZE RES SWAP STATE C TIME = CPU COMMAND . . . 834 root 1 20 0 13652K 2576K 0K select 3 2:21 = 0.00% top -CawSores (gstat -pd also tends to stop updating, but I use top as the example here.) For reference the old (non-updating) top instance shows itself as: PID USERNAME THR PRI NICE SIZE RES SWAP STATE C TIME = CPU COMMAND . . . 834 root 1 20 0 13652K 2576K 0K CPU3 3 2:21 = 0.33% top -CawSores The context is a Pine64+ 2GB building devel/llvm60 over the time, via poudriere-devel, 1 builder but allowing it to use all 4 cores. UFS context, not ZFS. devel/llvm60 started building 3:55:35 into the poudriere run and has been in process for many hours. The instances of top are via ssh logins over Ethernet. poudriere is running on the console. (No non-poudriere console messages during the poudriere run.) Typing Control-L to the ssh session for old instance of top will cause it to start updating again. (That is what I normally do when I notice the lack of updates.) # uname -apKU FreeBSD pine64 12.0-CURRENT FreeBSD 12.0-CURRENT r333079M arm64 = aarch64 1200062 1200062 This is built as a non-debug build with symbols and explicitly causing -mcpu=3Dcortex-a53 : XCFLAGS+=3D -mcpu=3Dcortex-a53 XCXXFLAGS+=3D -mcpu=3Dcortex-a53 # There is no XCPPFLAGS but XCPP gets XCFLAGS content. ACFLAGS.arm64cpuid.S+=3D -mcpu=3Dcortex-a53+crypto ACFLAGS.aesv8-armx.S+=3D -mcpu=3Dcortex-a53+crypto ACFLAGS.ghashv8-armx.S+=3D -mcpu=3Dcortex-a53+crypto As for what vintage I'm building for /usr/ports : # svnlite info /usr/ports/ | grep "Re[plv]" Relative URL: ^/head Repository Root: svn://svn.freebsd.org/ports Repository UUID: 35697150-7ecd-e111-bb59-0022644237b5 Revision: 468505 Last Changed Rev: 468505 But it is a large /usr/ports jump, from something like early January if I remember right. (FreeBSD was also from back then before I started this update sequence.) Context details (if you care): # svnlite status /usr/src | sort ? /usr/src/local_iperf3.sh ? /usr/src/sys/amd64/conf/GENERIC-DBG ? /usr/src/sys/amd64/conf/GENERIC-NODBG ? /usr/src/sys/arm/conf/GENERIC-DBG ? /usr/src/sys/arm/conf/GENERIC-NODBG ? /usr/src/sys/arm64/conf/GENERIC-DBG ? /usr/src/sys/arm64/conf/GENERIC-NODBG ? /usr/src/sys/powerpc/conf/GENERIC64vtsc-DBG ? /usr/src/sys/powerpc/conf/GENERIC64vtsc-NODBG ? /usr/src/sys/powerpc/conf/GENERICvtsc-DBG ? /usr/src/sys/powerpc/conf/GENERICvtsc-NODBG M /usr/src/contrib/llvm/lib/Target/PowerPC/PPCFrameLowering.cpp M /usr/src/contrib/llvm/tools/lld/ELF/Arch/PPC64.cpp M /usr/src/crypto/openssl/crypto/armcap.c M /usr/src/lib/libkvm/kvm_powerpc.c M /usr/src/lib/libkvm/kvm_private.c M /usr/src/release/Makefile.vm M /usr/src/release/scripts/mk-vmimage.sh M /usr/src/release/tools/vmimage.subr M /usr/src/secure/lib/libcrypto/Makefile M /usr/src/stand/defs.mk M /usr/src/stand/powerpc/boot1.chrp/Makefile M /usr/src/stand/powerpc/kboot/Makefile M /usr/src/sys/arm64/arm64/identcpu.c M /usr/src/sys/conf/kmod.mk M /usr/src/sys/conf/ldscript.powerpc M /usr/src/sys/kern/subr_pcpu.c M /usr/src/sys/powerpc/aim/mmu_oea64.c M /usr/src/sys/powerpc/ofw/ofw_machdep.c M /usr/src/sys/powerpc/powerpc/interrupt.c M /usr/src/sys/powerpc/powerpc/mp_machdep.c M /usr/src/sys/powerpc/powerpc/trap.c M /usr/src/sys/vm/uma_core.c M /usr/src/usr.bin/top/machine.c Most of the "M"s are non-aarch64 material. My GENERIC-*DBG's include the matching GENERIC and then override various debug-status points. Only powerpc family ones do somewhat more. # svnlite diff /usr/src/crypto/openssl/crypto/armcap.c Index: /usr/src/crypto/openssl/crypto/armcap.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/crypto/openssl/crypto/armcap.c (revision 333079) +++ /usr/src/crypto/openssl/crypto/armcap.c (working copy) @@ -137,7 +137,9 @@ } } else if (sigsetjmp(ill_jmp, 1) =3D=3D 0) { _armv7_neon_probe(); +#ifndef DISABLE_HACK_THAT_AVOIDS_NEON OPENSSL_armcap_P |=3D ARMV7_NEON; +#endif if (sigsetjmp(ill_jmp, 1) =3D=3D 0) { _armv8_pmull_probe(); OPENSSL_armcap_P |=3D ARMV8_PMULL | ARMV8_AES; # svnlite diff /usr/src/sys/arm64/arm64/identcpu.c Index: /usr/src/sys/arm64/arm64/identcpu.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/sys/arm64/arm64/identcpu.c (revision 333079) +++ /usr/src/sys/arm64/arm64/identcpu.c (working copy) @@ -1109,6 +1109,9 @@ =20 /* Wake up the other CPUs */ atomic_store_rel_int(&ident_lock, 0); - __asm __volatile("sev" ::: "memory"); + __asm __volatile( + "dsb ish \n" + "sev \n" + ::: "memory"); } } # svnlite diff /usr/src/sys/kern/subr_pcpu.c Index: /usr/src/sys/kern/subr_pcpu.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/sys/kern/subr_pcpu.c (revision 333079) +++ /usr/src/sys/kern/subr_pcpu.c (working copy) @@ -279,6 +279,8 @@ struct pcpu * pcpu_find(u_int cpuid) { +if (mp_ncpus =3D=3D 0 && cpuid =3D=3D 0) {} // HACK: This combination = is used in the late bootstrap +//else if (mp_ncpus <=3D cpuid) { panic("pcpu_find: cpuid too large"); = } // HACK =20 return (cpuid_to_pcpu[cpuid]); } (The above I've sometimes uncommented the else for powerpc family experiments.) # svnlite diff /usr/src/sys/vm/uma_core.c Index: /usr/src/sys/vm/uma_core.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/sys/vm/uma_core.c (revision 333079) +++ /usr/src/sys/vm/uma_core.c (working copy) @@ -3441,7 +3441,7 @@ void uma_reclaim_wakeup(void) { - +printf("limit %lX, total %lX, needed %d\n", uma_kmem_limit, = uma_kmem_total, uma_reclaim_needed); if (atomic_fetchadd_int(&uma_reclaim_needed, 1) =3D=3D 0) wakeup(uma_reclaim); } I've seen the behavior with an unmodified top but I add a "K MaxObsUsed, " for swap normally: # svnlite diff /usr/src/usr.bin/top/machine.c Index: /usr/src/usr.bin/top/machine.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/usr.bin/top/machine.c (revision 333079) +++ /usr/src/usr.bin/top/machine.c (working copy) @@ -194,9 +194,9 @@ NULL }; =20 -int swap_stats[7]; +int swap_stats[8]; char *swapnames[] =3D { - "K Total, ", "K Used, ", "K Free, ", "% Inuse, ", "K In, ", "K = Out", + "K Total, ", "K Used, ", "K MaxObsUsed, ", "K Free, ", "% Inuse, = ", "K In, ", "K Out", NULL }; =20 @@ -550,14 +550,14 @@ =20 /* first interval */ if (swappgsin < 0) { - swap_stats[4] =3D 0; - swap_stats[5] =3D 0; + swap_stats[4+1] =3D 0; + swap_stats[5+1] =3D 0; } =20 /* compute differences between old and new swap = statistic */ else { - swap_stats[4] =3D pagetok(((nspgsin - = swappgsin))); - swap_stats[5] =3D pagetok(((nspgsout - = swappgsout))); + swap_stats[4+1] =3D pagetok(((nspgsin - = swappgsin))); + swap_stats[5+1] =3D pagetok(((nspgsout - = swappgsout))); } =20 swappgsin =3D nspgsin; @@ -564,14 +564,16 @@ swappgsout =3D nspgsout; =20 /* call CPU heavy swapmode() only for changes */ - if (swap_stats[4] > 0 || swap_stats[5] > 0 || swap_delay = =3D=3D 0) { - swap_stats[3] =3D swapmode(&swapavail, = &swapfree); + if (swap_stats[4+1] > 0 || swap_stats[5+1] > 0 || = swap_delay =3D=3D 0) { + swap_stats[3+1] =3D swapmode(&swapavail, = &swapfree); swap_stats[0] =3D swapavail; swap_stats[1] =3D swapavail - swapfree; - swap_stats[2] =3D swapfree; + if (swap_stats[2+0]