From owner-freebsd-hackers@freebsd.org Fri Jun 12 00:30:29 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BC2C8346AD4 for ; Fri, 12 Jun 2020 00:30:29 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic303-25.consmr.mail.gq1.yahoo.com (sonic303-25.consmr.mail.gq1.yahoo.com [98.137.64.206]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 49jhTJ1F8Qz4dC2 for ; Fri, 12 Jun 2020 00:30:27 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: YRjKgZ8VM1nz1gmprGN5LHFr7c2ibc7p8GI71NzCZGXTHdzsyZR0vQwHLwPa7AB bYghn03LipAom3UDNW_Q2cQuWn9XgM5DnTGlgCTXl2Wa3Z9HFd3DmIOY2_ysSmEQ.Oeh8ar4wjSt H2ECt..1MXRVRKV5aAQGLl_a.JINY9SQfnYzg5WLKFewIALgETnCy5290ACMFykqHsY8Ti0GxePq PcfnZEM7kMNRE.yR_K6TY6tiqEh2NHuCrLSguIly.fag_GtIkgVTOQ2eRHGUhSEbAtT9.6GSZ6i_ X00wrvE3HjBUUTPCT8BuBemZVigBjsSx33dX98JsihM_WtJLiNYwUgwaxBQknJ2Z_8iugFOrM2OH Irqz9jLGoV4xO_loLjzF4gaAmzcVNXHePPHLnIEdflC4EPSU7.S15xl8E0_q.J4urQjWiqswcxi_ UZ62T_qN2IURrmbceorIo8QZVBJOwehjwrgFKK4V7dXoWDkPySCWqvXxVXl5n3k_SSljrYAxYumf 9GYlPpS7OXSMhdQ2.wHhU4WrmMSfudpJpEv5xgzv.aS2eNVpKSacc21_AjFVz6YBSjLBLbq1swN7 P4sepPNBuDUJ28VuwHcsiWGZK27cl7KLQMdUv1ZHx88SscE1lffueNxy.WHwktmECdU3iqJEFif9 9X1226AOzW5ZJfBtwPm9dqqWtPE7r3qD4_k4Ux_R99OJiAoHjg8IRFsvIP3ic_pJGpmmX3SlgrMA 6rRANDypWVdvWy9h_RMqg20fdyRyQHBh8S5QHF0Q8ZIkRq4aaEn3k8auG4U7lf3_qEIXDLLYw5Zv PMoDFlv5x77KhYdcn7B7ptwLgGg.kXiKxE0CqotkRJS5GAJEFZZwROi0lAGUE9125jdCUMVQptkt mfM9iDyhBnN1tI3pRo3.O6kAA5jtPVQ0Z2M3DYU2qtVFbnloS8JNT1vCpRwbBVZ_C2RK7F8he6oU O.2y9n0Kgm_I5LSOhpG_kvURahhRNtNuPV.Ly0XTd4pZMjcgMVxbOmOAxjdUzMdPCIpOnM1kS8bA FD980GhZofJlwaWY.e7lI.WfOrOHckJovI8suv3y7FUCInGlUshHE8c5q.P72NLynSb0mfBEPjCE j4jZmFFatRjR.iBK.j5ccgpv10lj6fz66U.iKRDvIXS.KmErysrhKzCMT1qpq2p996qCpKcRLk4s 9jlwaUo_7_mspyx_2Hv01lcg1NIBS4fCgtPsNMi3OH7px84.i6QtDFLzQlQ0HAr_AGIiu4oqNXl8 ZBlRXjH0HLu2EnYM_KYkF_A6lIomSaIJpmoVG8Lge8Ub82MeNtvZQWVj860vD0a2JzYKEWZA6REQ Tr_vabjuSf.kKWZQAGGBJc8WLl1q0CjfIVr23Ffus4_7KsuEpDGW3NHvyRljmCp7ZK._23uKIrmr dWSew1eFYyurdVlR32tWiWaXDTAPeIIHBYzIDTB7X4uz8MDzwQHiUPByJ9Q-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.gq1.yahoo.com with HTTP; Fri, 12 Jun 2020 00:30:26 +0000 Received: by smtp428.mail.gq1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 1f421ce3a9c01e05f91777c49cc0466a; Fri, 12 Jun 2020 00:30:25 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.80.23.2.2\)) Subject: Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311 From: Mark Millard In-Reply-To: Date: Thu, 11 Jun 2020 17:30:24 -0700 Cc: "vangyzen@freebsd.org" , svn-src-head@freebsd.org, FreeBSD Current , FreeBSD Hackers , FreeBSD PowerPC ML , Brandon Bergren Content-Transfer-Encoding: quoted-printable Message-Id: References: <8479DD58-44F6-446A-9CA5-D01F0F7C1B38@yahoo.com> <17ACDA02-D7EF-4F26-874A-BB3E935CD072@yahoo.com> <695E6836-F860-4557-B7DE-CC1EDB347F18@yahoo.com> <121B9B09-141B-4DC3-918B-1E7CFB99E779@yahoo.com> <8AAB0462-3FA8-490C-8D8D-7C15B1C9E2DE@yahoo.com> <18E62746-80DB-4195-977D-4FF32D0129EE@yahoo.com> <9562EEE4-62EF-4164-91C0-948CC0432984@yahoo.com> <9B68839B-AEC8-43EE-B3B6-B696A4A57DAE@yahoo.com> <359C9C7D-4106-42B5-AAB5-08EF995B8100@yahoo.com> <20200513105632.06db9e21@titan.knownspace> <20200611155545.55526f7c@ralga.knownspace> <5542B85D-1C3A-41D8-98CE-3C02E990C3EB@yahoo.com> <20200611164216.47f82775@ralga.knownspace> To: Justin Hibbits X-Mailer: Apple Mail (2.3608.80.23.2.2) X-Rspamd-Queue-Id: 49jhTJ1F8Qz4dC2 X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.12 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; RCPT_COUNT_SEVEN(0.00)[7]; NEURAL_HAM_SHORT(-0.61)[-0.614]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.01)[-1.014]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-0.99)[-0.994]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.206:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.206:from]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Jun 2020 00:30:29 -0000 On 2020-Jun-11, at 16:49, Mark Millard wrote: > On 2020-Jun-11, at 14:42, Justin Hibbits = wrote: >=20 > On Thu, 11 Jun 2020 14:36:37 -0700 > Mark Millard wrote: >=20 >> On 2020-Jun-11, at 13:55, Justin Hibbits >> wrote: >>=20 >>> On Wed, 10 Jun 2020 18:56:57 -0700 >>> Mark Millard wrote: > . . . >>=20 >>=20 >>> That said, the attached patch effectively copies >>> what's done in OEA6464 into OEA pmap. Can you test it? =20 >>=20 >> I'll try it once I get a chance, probably later >> today. >> . . . >=20 > No luck at the change being a fix, I'm afraid. >=20 > I verified that the build ended up with >=20 > 00926cb0 bl 008e8dc8 > 00926cb4 mr r27,r3 > 00926cb8 addi r3,r3,36 > 00926cbc hwsync > 00926cc0 lwarx r25,0,r3 > 00926cc4 li r4,0 > 00926cc8 stwcx. r4,0,r3 > 00926ccc bne- 00926cc0 > 00926cd0 andi. r3,r25,128 > 00926cd4 beq 00926ce0 > 00926cd8 mr r3,r27 > 00926cdc bl 008e9874 >=20 > in the installed kernel. So I doubt a > mis-build would be involved. It is a > head -r360311 based context still. World is > without MALLOC_PRODUCTION so that jemalloc > code executes its asserts, catching more > and earlier than otherwise. >=20 > First test . . . >=20 > The only thing that the witness kernel reported was: >=20 > Jun 11 15:58:16 FBSDG4S2 kernel: lock order reversal: > Jun 11 15:58:16 FBSDG4S2 kernel: 1st 0x216fb00 Mountpoints (UMA zone) = @ /usr/src/sys/vm/uma_core.c:4387 > Jun 11 15:58:16 FBSDG4S2 kernel: 2nd 0x1192d2c kernelpmap = (kernelpmap) @ /usr/src/sys/powerpc/aim/mmu_oea.c:1524 > Jun 11 15:58:16 FBSDG4S2 kernel: stack backtrace: > Jun 11 15:58:16 FBSDG4S2 kernel: #0 0x5ec164 at witness_debugger+0x94 > Jun 11 15:58:16 FBSDG4S2 kernel: #1 0x5ebe3c at = witness_checkorder+0xb50 > Jun 11 15:58:16 FBSDG4S2 kernel: #2 0x536d5c at __mtx_lock_flags+0xcc > Jun 11 15:58:16 FBSDG4S2 kernel: #3 0x92636c at moea_kextract+0x5c > Jun 11 15:58:16 FBSDG4S2 kernel: #4 0x965d30 at pmap_kextract+0x98 > Jun 11 15:58:16 FBSDG4S2 kernel: #5 0x8bfdbc at zone_release+0xf0 > Jun 11 15:58:16 FBSDG4S2 kernel: #6 0x8c7854 at bucket_drain+0x2f0 > Jun 11 15:58:16 FBSDG4S2 kernel: #7 0x8c728c at bucket_free+0x54 > Jun 11 15:58:16 FBSDG4S2 kernel: #8 0x8c74fc at = bucket_cache_reclaim+0x1bc > Jun 11 15:58:16 FBSDG4S2 kernel: #9 0x8c7004 at zone_reclaim+0x128 > Jun 11 15:58:16 FBSDG4S2 kernel: #10 0x8c3a40 at uma_reclaim+0x170 > Jun 11 15:58:16 FBSDG4S2 kernel: #11 0x8c3f70 at = uma_reclaim_worker+0x68 > Jun 11 15:58:16 FBSDG4S2 kernel: #12 0x50fbac at fork_exit+0xb0 > Jun 11 15:58:16 FBSDG4S2 kernel: #13 0x9684ac at fork_trampoline+0xc >=20 > The processes that were hit were listed as: >=20 > Jun 11 15:59:11 FBSDG4S2 kernel: pid 971 (cron), jid 0, uid 0: exited = on signal 11 (core dumped) > Jun 11 16:02:59 FBSDG4S2 kernel: pid 1111 (stress), jid 0, uid 0: = exited on signal 6 (core dumped) > Jun 11 16:03:27 FBSDG4S2 kernel: pid 871 (mountd), jid 0, uid 0: = exited on signal 6 (core dumped) > Jun 11 16:03:40 FBSDG4S2 kernel: pid 1065 (su), jid 0, uid 0: exited = on signal 6 > Jun 11 16:04:13 FBSDG4S2 kernel: pid 1088 (su), jid 0, uid 0: exited = on signal 6 > Jun 11 16:04:28 FBSDG4S2 kernel: pid 968 (sshd), jid 0, uid 0: exited = on signal 6 >=20 > Jun 11 16:05:42 FBSDG4S2 kernel: pid 1028 (login), jid 0, uid 0: = exited on signal 6 >=20 > Jun 11 16:05:46 FBSDG4S2 kernel: pid 873 (nfsd), jid 0, uid 0: exited = on signal 6 (core dumped) >=20 >=20 > Rebooting and rerunning and showing the stress output and such > (I did not capture copies during the first test, but the first > test had similar messages at the same sort of points): >=20 > Second test . . . >=20 > # stress -m 2 --vm-bytes 1700M > stress: info: [1166] dispatching hogs: 0 cpu, 0 io, 2 vm, 0 hdd > : = /usr/src/contrib/jemalloc/include/jemalloc/internal/arena_inlines_b.h:258:= Failed assertion: "slab =3D=3D extent_slab_get(extent)" > : = /usr/src/contrib/jemalloc/include/jemalloc/internal/arena_inlines_b.h:258:= Failed assertion: "slab =3D=3D extent_slab_get(extent)" > ^C >=20 > # exit > : = /usr/src/contrib/jemalloc/include/jemalloc/internal/sz.h:200: Failed = assertion: "ret =3D=3D sz_index2size_compute(index)" > Abort trap >=20 > The other stuff was similar to to first test, not repeated here. The updated code looks odd to me for how "m" is handled (part of a egrep to ensure I show all the usage of m): moea_protect(mmu_t mmu, pmap_t pm, vm_offset_t sva, vm_offset_t eva, vm_page_t m; if (pm !=3D kernel_pmap && m !=3D NULL && (m->a.flags & PGA_EXECUTABLE) =3D=3D 0 && if ((m->oflags & VPO_UNMANAGED) =3D=3D = 0) vm_page_aflag_set(m, = PGA_EXECUTABLE); m =3D PHYS_TO_VM_PAGE(old_pte.pte_lo & = PTE_RPGN); refchg =3D = atomic_readandclear_32(&m->md.mdpg_attrs); vm_page_dirty(m); vm_page_aflag_set(m, = PGA_REFERENCED); Or more completely, with notes mixed in: void=20 moea_protect(mmu_t mmu, pmap_t pm, vm_offset_t sva, vm_offset_t eva, vm_prot_t prot) { . . . vm_page_t m; . . . for (pvo =3D RB_NFIND(pvo_tree, &pm->pmap_pvo, &key); pvo !=3D NULL && PVO_VADDR(pvo) < eva; pvo =3D tpvo) { . . . if (pt !=3D NULL) { . . . if (pm !=3D kernel_pmap && m !=3D NULL && NOTE: m seems to be uninitialized but tested for being NULL above. (m->a.flags & PGA_EXECUTABLE) =3D=3D 0 && Note: This looks to potentially be using a random, non-NULL value for m during evaluation of m->a.flags . . . . if ((pvo->pvo_vaddr & PVO_MANAGED) && (pvo->pvo_pte.prot & VM_PROT_WRITE)) { m =3D PHYS_TO_VM_PAGE(old_pte.pte_lo & = PTE_RPGN); Note: m finally is potentially initialized(/set). refchg =3D = atomic_readandclear_32(&m->md.mdpg_attrs); if (refchg & PTE_CHG) vm_page_dirty(m); if (refchg & PTE_REF) vm_page_aflag_set(m, = PGA_REFERENCED); . . . Note: So, if m is set above, then the next loop iteration(s) would use this then-old value instead of an initialized value. It looks to me like at least one assignment to m is missing. moea64_pvo_protect has pg that seems analogous to m and has: pg =3D PHYS_TO_VM_PAGE(pvo->pvo_pte.pa & LPTE_RPGN); . . . if (pm !=3D kernel_pmap && pg !=3D NULL && (pg->a.flags & PGA_EXECUTABLE) =3D=3D 0 && (pvo->pvo_pte.pa & (LPTE_I | LPTE_G | LPTE_NOEXEC)) =3D=3D = 0) { if ((pg->oflags & VPO_UNMANAGED) =3D=3D 0) vm_page_aflag_set(pg, PGA_EXECUTABLE); . . . if (pg !=3D NULL && (pvo->pvo_vaddr & PVO_MANAGED) && (oldprot & VM_PROT_WRITE)) { refchg |=3D atomic_readandclear_32(&pg->md.mdpg_attrs); if (refchg & LPTE_CHG) vm_page_dirty(pg); if (refchg & LPTE_REF) vm_page_aflag_set(pg, PGA_REFERENCED); This might suggest some about what is missing. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)