From owner-freebsd-ppc@freebsd.org Mon Feb 20 23:10:53 2017 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 65842CE633E for ; Mon, 20 Feb 2017 23:10:53 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-76.reflexion.net [208.70.210.76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 206C3C48 for ; Mon, 20 Feb 2017 23:10:52 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 22102 invoked from network); 20 Feb 2017 23:10:46 -0000 Received: from unknown (HELO rtc-sm-01.app.dca.reflexion.local) (10.81.150.1) by 0 (rfx-qmail) with SMTP; 20 Feb 2017 23:10:46 -0000 Received: by rtc-sm-01.app.dca.reflexion.local (Reflexion email security v8.30.0) with SMTP; Mon, 20 Feb 2017 18:10:46 -0500 (EST) Received: (qmail 14277 invoked from network); 20 Feb 2017 23:10:46 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 20 Feb 2017 23:10:46 -0000 Received: from [192.168.1.111] (c-67-170-167-181.hsd1.or.comcast.net [67.170.167.181]) by iron2.pdx.net (Postfix) with ESMTPSA id 8551FEC7B46; Mon, 20 Feb 2017 15:10:45 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\)) Subject: Re: svn commit: r313268 - head/sys/kern [through -r313271 for atomic_fcmpset use and later: fails on PowerMac G5 "Quad Core"; -r313266 works] From: Mark Millard In-Reply-To: <83428304-87BE-413C-BAB9-8FF218E7661C@dsl-only.net> Date: Mon, 20 Feb 2017 15:10:44 -0800 Cc: Justin Hibbits , mjg@freebsd.org, FreeBSD Current , svn-src-head@freebsd.org, FreeBSD PowerPC ML Content-Transfer-Encoding: quoted-printable Message-Id: References: <2FD12B8F-2255-470A-98D4-2DCE9C7495F5@dsl-only.net> <20170220191044.GA8526@dft-labs.eu> <83428304-87BE-413C-BAB9-8FF218E7661C@dsl-only.net> To: Mateusz Guzik X-Mailer: Apple Mail (2.3259) X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Feb 2017 23:10:53 -0000 On 2017-Feb-20, at 2:58 PM, Mark Millard wrote: > On 2017-Feb-20, at 11:10 AM, Mateusz Guzik = wrote: >=20 >> On Sat, Feb 18, 2017 at 04:18:05AM -0800, Mark Millard wrote: >>> [Note: I experiment with clang based powerpc64 builds, >>> reporting problems that I find. Justin is familiar >>> with this, as is Nathan.] >>>=20 >>> I tried to update the PowerMac G5 (a so-called "Quad Core") >>> that I have access to from head -r312761 to -r313864 and >>> ended up with random panics and hang ups in fairly short >>> order after booting. >>>=20 >>> Some approximate bisecting for the kernel lead to: >>> (sometimes getting part way into a buildkernel attempt >>> for a different version before a failure happens) >>>=20 >>> -r313266: works (just before use of atomic_fcmpset) >>> vs. >>> -r313271: fails (last of the "use atomic_fcmpset" check-ins) >>>=20 >>> (I did not try -r313268 through -r313270 as the use was >>> gradually added.) >>>=20 >>> So I'm currently running a -r313864 world with a -r313266 >>> kernel. >>>=20 >>> No kernel that I tried that was from before -r313266 had the >>> problems. >>>=20 >>> Any kernel that I tried that was from after -r313271 had the >>> problems. >>>=20 >>> Of course I did not try them all in other direction. :) >>>=20 >>=20 >> I found that spin mutexes were not properly handling this, fixed in >> r313996. >>=20 >> Locally I added a if (cpu_tick() % 2) return (0); snipped to amd64 >> fcmpset to simulate failures. Everything works, while it would easily >> fail without the patch. >>=20 >> That said, I hope this concludes the 'missing check for not-reread = value >> of failed fcmpset' saga. >>=20 >> --=20 >> Mateusz Guzik >=20 > I tried to update from -r313864 to -r313999 in my amd64 context > (a VirtualBox machine under macOS) but it now crashes late in > the boot sequence (after it processes a dump if I make one but > before I can log in). >=20 > This update was via my usual explicit svnlite update; buildworld > buildkernel; etc. production style build of world and kernel, > including use of MALLOC_PRODUCTION. >=20 > The window shows: >=20 > _vm_map_lock+0xf > vm_map_wire+0x32 > rtROMemObjNativeLockInMap+0x8c > rtROMemObjNativeLockUser+0x51 > RTR0MemObjLockUserTag+0x231 > vbglR0HGCMInternalPreprocessCall+0x65d > vbglR0HGCMInternalCall+0x17c > vgdrvIoCtl_HGCMCall+0x43f > VGDrvCommonIoCtl+0x261 > vgdrvFreeBSDIOCtl+0x2cd > devfs_ioctl+0xae > VOP_IOCTL_APV+0x88 > vn_ioctl+0x161 > devfs_ioctl_f+0x1f > kern_ioctl+0x280 > sys_ioctl+0x13f > amd64_syscall+0x397 > Xfast_syscall+0xfb More detail from booting with the -r313864 kernel.old and using kgdb on what the dump produced: # kgdb kernel.debug /var/crash/vmcore. /var/crash/vmcore.0 /var/crash/vmcore.last # kgdb kernel.debug /var/crash/vmcore.0 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you = are welcome to change it and/or distribute copies of it under certain = conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for = details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: <118>Starting vboxservice. <118>VBoxService 5.1.14 r112924 (verbosity: 0) freebsd.amd64 (Jan 20 = 2017 18:37:45) release log <118>00:00:00.000120 main Log opened 2017-02-20T22:38:46.348080000Z <118>00:00:00.000162 main OS Product: FreeBSD <118>00:00:00.000171 main OS Release: 12.0-CURRENT <118>00:00:00.000180 main OS Version: FreeBSD 12.0-CURRENT r313999M <118>00:00:00.000192 main Executable: /usr/local/sbin/VBoxService <118>00:00:00.000194 main Process ID: 609 <118>00:00:00.000196 main Package type: BSD_64BITS_GENERIC (OSE) Fatal trap 12: page fault while in kernel mode cpuid =3D 2; apic id =3D 02 fault virtual address =3D 0xd6 fault code =3D supervisor read data, page not present instruction pointer =3D 0x20:0xffffffff80d4ebaf stack pointer =3D 0x28:0xfffffe0122e2bef0 frame pointer =3D 0x28:0xfffffe0122e2bf00 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 609 (VBoxService) Reading symbols from /boot/kernel/zfs.ko...Reading symbols from = /usr/lib/debug//boot/kernel/zfs.ko.debug...done. done. Loaded symbols for /boot/kernel/zfs.ko Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from = /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. done. Loaded symbols for /boot/kernel/opensolaris.ko Reading symbols from /boot/modules/vboxguest.ko...done. Loaded symbols for /boot/modules/vboxguest.ko #0 doadump (textdump=3D0) at pcpu.h:232 232 __asm("movq %%gs:%1,%0" : "=3Dr" (td) (kgdb) bt #0 doadump (textdump=3D0) at pcpu.h:232 #1 0xffffffff8039dd0b in db_dump (dummy=3D, = dummy2=3D, dummy3=3D, = dummy4=3D) at /usr/src/sys/ddb/db_command.c:546 #2 0xffffffff8039db0f in db_command (cmd_table=3D) = at /usr/src/sys/ddb/db_command.c:453 #3 0xffffffff8039d884 in db_command_loop () at = /usr/src/sys/ddb/db_command.c:506 #4 0xffffffff803a0814 in db_trap (type=3D, = code=3D) at /usr/src/sys/ddb/db_main.c:254 #5 0xffffffff80a9c0c3 in kdb_trap (type=3D, = code=3D, tf=3D) at = /usr/src/sys/kern/subr_kdb.c:654 #6 0xffffffff80ed25d2 in trap_fatal (frame=3D0xfffffe0122e2be30, = eva=3D214) at /usr/src/sys/amd64/amd64/trap.c:796 #7 0xffffffff80ed27dc in trap_pfault (frame=3D0xfffffe0122e2be30, = usermode=3D0) at /usr/src/sys/amd64/amd64/trap.c:658 #8 0xffffffff80ed1e90 in trap (frame=3D0xfffffe0122e2be30) at = /usr/src/sys/amd64/amd64/trap.c:421 #9 0xffffffff80eb6be1 in calltrap () at = /usr/src/sys/amd64/amd64/exception.S:236 #10 0xffffffff80d4ebaf in _vm_map_lock (map=3D0x1, file=3D0x0, line=3D0) = at /usr/src/sys/vm/vm_map.c:501 #11 0xffffffff80d51ea2 in vm_map_wire (map=3D, = start=3D4534272, end=3D4538368, flags=3D1) at = /usr/src/sys/vm/vm_map.c:2534 #12 0xffffffff8265291c in rtR0MemObjNativeLockInMap () from = /boot/modules/vboxguest.ko #13 0xffffffff82652881 in rtR0MemObjNativeLockUser () from = /boot/modules/vboxguest.ko #14 0xffffffff8264ec01 in RTR0MemObjLockUserTag () from = /boot/modules/vboxguest.ko #15 0xffffffff82624afd in vbglR0HGCMInternalPreprocessCall () from = /boot/modules/vboxguest.ko #16 0xffffffff8262411a in VbglR0HGCMInternalCall () from = /boot/modules/vboxguest.ko #17 0xffffffff8261ec4f in vgdrvIoCtl_HGCMCall () from = /boot/modules/vboxguest.ko #18 0xffffffff8261d221 in VGDrvCommonIoCtl () from = /boot/modules/vboxguest.ko #19 0xffffffff8262327d in vgdrvFreeBSDIOCtl () from = /boot/modules/vboxguest.ko #20 0xffffffff8092976e in devfs_ioctl (ap=3D) at = /usr/src/sys/fs/devfs/devfs_vnops.c:805 #21 0xffffffff8103ef58 in VOP_IOCTL_APV (vop=3D, = a=3D) at vnode_if.c:1067 #22 0xffffffff80b29431 in vn_ioctl (fp=3D0xfffff80006d37730, com=3D, data=3D0xfffffe0122e2c870, = active_cred=3D0xfffff80006495a00, td=3D) at = vnode_if.h:448 #23 0xffffffff80929d5f in devfs_ioctl_f (fp=3D, = com=3D, data=3D, cred=3D, td=3D0xfffff8001504e000) at = /usr/src/sys/fs/devfs/devfs_vnops.c:763 #24 0xffffffff80ab8bf0 in kern_ioctl (td=3D, fd=3D3, = com=3D, data=3D0xfffffe0122e2c870 "\031\002R\031P") = at file.h:322 #25 0xffffffff80ab88bf in sys_ioctl (td=3D, = uap=3D0xfffffe0122e2ca30) at /usr/src/sys/kern/sys_generic.c:743 #26 0xffffffff80ed2e27 in amd64_syscall (td=3D0xfffff8001504e000, = traced=3D0) at subr_syscall.c:135 #27 0xffffffff80eb6ecb in Xfast_syscall () at = /usr/src/sys/amd64/amd64/exception.S:396 #28 0x0000000800c5317a in ?? () Previous frame inner to this frame (corrupt stack?) Current language: auto; currently minimal =3D=3D=3D Mark Millard markmi at dsl-only.net