Date: Tue, 7 Mar 2017 00:10:31 -0800 From: Mark Millard <markmi@dsl-only.net> To: Mark Johnston <markj@FreeBSD.org> Cc: Justin Hibbits <chmeeedalf@gmail.com>, FreeBSD PowerPC ML <freebsd-ppc@freebsd.org> Subject: Re: powerpc64 head -r314687 (PowerMac G5 so-called "Quad Core", clang based): CAM status: Command timeout (always?) Message-ID: <49DE6E23-948B-4DC1-87A0-047F8A2CE966@dsl-only.net> In-Reply-To: <2FA8AC16-8108-4FC7-B1E6-788CBD32F372@dsl-only.net> References: <98A62E0D-C2A0-40B1-AE6D-5810906208AE@dsl-only.net> <4C78F6AA-5ABD-4445-B5EF-4E6778CE36FE@dsl-only.net> <20170306164341.GA83069@wkstn-mjohnston.west.isilon.com> <466C25ED-0A70-4988-9BB1-3B43BD031E5E@dsl-only.net> <E67A6606-941D-4F00-993D-4347C2A1D332@dsl-only.net> <20170307010204.GA3611@wkstn-mjohnston.west.isilon.com> <2FA8AC16-8108-4FC7-B1E6-788CBD32F372@dsl-only.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2017-Mar-6, at 8:03 PM, Mark Millard <markmi at dsl-only.net> wrote: > I'm no longer able to reproduce the problem, not even with an > "svnlite update -r314687" based build where "svnlite status > /usr/src/" does not list ether of the files. This was after > trying the patch sequence, which had no failures at any stage. >=20 > This suggests some sort of intermittent problem someplace. >=20 > At least it fits with your not finding a way for your code > update to cause the results that I got. >=20 > But finding such an intermittent problem is a pain. I've > no clue if/when I'll even see an example again, much less > find a way to investigate it if I do. (PowerMac's do not > take ddb input early.) >=20 > There is the possibility that the recent atomic_fcmpset based > locking changes still has some sort of problem, just not seen > often. Not easy to find if true. >=20 > Anyway I'm now running -r314687 with: >=20 > # svnlite status /usr/src/ | sort > ? /usr/src/sys/amd64/conf/GENERIC-DBG > ? /usr/src/sys/amd64/conf/GENERIC-NODBG > ? /usr/src/sys/arm/conf/BPIM3-DBG > ? /usr/src/sys/arm/conf/BPIM3-NODBG > ? /usr/src/sys/arm/conf/RPI2-DBG > ? /usr/src/sys/arm/conf/RPI2-NODBG > ? /usr/src/sys/arm64/conf/GENERIC-DBG > ? /usr/src/sys/arm64/conf/GENERIC-NODBG > ? /usr/src/sys/powerpc/conf/GENERIC64vtsc-DBG > ? /usr/src/sys/powerpc/conf/GENERIC64vtsc-NODBG > ? /usr/src/sys/powerpc/conf/GENERICvtsc-DBG > ? /usr/src/sys/powerpc/conf/GENERICvtsc-NODBG > M /usr/src/bin/sh/jobs.c > M /usr/src/bin/sh/miscbltin.c > M /usr/src/contrib/llvm/tools/lld/ELF/Target.cpp > M /usr/src/crypto/openssl/crypto/armcap.c > M /usr/src/lib/csu/powerpc64/Makefile > M /usr/src/libexec/rtld-elf/Makefile > M /usr/src/sys/arm/arm/gic.c > M /usr/src/sys/boot/ofw/Makefile.inc > M /usr/src/sys/boot/powerpc/Makefile.inc > M /usr/src/sys/boot/powerpc/kboot/Makefile > M /usr/src/sys/boot/uboot/Makefile.inc > M /usr/src/sys/conf/kmod.mk > M /usr/src/sys/ddb/db_main.c > M /usr/src/sys/ddb/db_script.c > M /usr/src/sys/powerpc/ofw/ofw_machdep.c >=20 > (which are long standing in my environment). >=20 > I'll build and try a debug kernel but I'm not hopeful > for it finding anything. The debug kernel has not failed so far but does report sleeping with a non-sleepable lock held (in addition to a lor): SMP: AP CPU #3 launched SMP: AP CPU #2 launched SMP: AP CPU #1 launched WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Trying to mount root from ufs:/dev/ufs/FBSDG5Lrootfs [rw,noatime]... ugen0.2: <Mitsumi Electric Hub in Apple Extended USB Keyboard> at usbus0 uhub3 on uhub0 uhub3: <Mitsumi Electric Hub in Apple Extended USB Keyboard, class 9/0, = rev 1.10/4.20, addr 2> on usbus0 Expensive timeout(9) function: 0xe84630(0xdc4a10) 0.026154120 s Sleeping on "smu" with the following non-sleepable locks held: exclusive sleep mutex tod2rl (tod2rl) r =3D 0 (0x122ef50) locked @ = /usr/src/sys/kern/subr_rtc.c:135 stack backtrace: #0 0x60e3bc at .get_spr+0x50d4dc #1 0x59a200 at .get_spr+0x499320 #2 0x91b4cc at .rtascall+0x11dac #3 0x91a76c at .rtascall+0x1104c #4 0x5f0680 at .get_spr+0x4ef7a0 #5 0x68729c at .get_spr+0x5863bc #6 0x512af4 at .get_spr+0x411c14 #7 0x546600 at .get_spr+0x445720 #8 0x935f70 at .fork_trampoline+0x10 #9 0x935e44 at blocked_loop+0x38 lock order reversal: (Giant after non-sleepable) 1st 0x122ef50 tod2rl (tod2rl) @ /usr/src/sys/kern/subr_rtc.c:135 2nd 0x1200b00 Giant (Giant) @ /usr/src/sys/kern/kern_synch.c:230 stack backtrace: #0 0x60c9e4 at .get_spr+0x50bb04 #1 0x56c504 at .get_spr+0x46b624 #2 0x59a5d0 at .get_spr+0x4996f0 #3 0x91b4cc at .rtascall+0x11dac #4 0x91a76c at .rtascall+0x1104c #5 0x5f0680 at .get_spr+0x4ef7a0 #6 0x68729c at .get_spr+0x5863bc #7 0x512af4 at .get_spr+0x411c14 #8 0x546600 at .get_spr+0x445720 #9 0x935f70 at .fork_trampoline+0x10 #10 0x935e44 at blocked_loop+0x38 uhub3: 3 ports with 2 removable, bus powered Setting hostuuid: 11fe4cb1-3cac-11e4-a5a8-001451685a44. Setting hostid: 0x489b688f. Starting file system checks: /dev/ufs/FBSDG5Lrootfs: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ufs/FBSDG5Lrootfs: clean, 82126801 free (1058921 frags, 10133485 = blocks, 1.1% fragmentation) Mounting local filesystems:. (The symbolic routine names it shows are not of much use. I could use /usr/local/bin/objdump to look things up.) Later there is another lor: No core dumps found. Clearing /tmp (X related). NFSv4 is disabled lock order reversal: 1st 0xa046b78 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2600 2nd 0xc0000000b6c90ba0 bufwait (bufwait) @ = /usr/src/sys/ufs/ffs/ffs_vnops.c:280 3rd 0xb22e5f0 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2600 stack backtrace: #0 0x60c9e4 at .get_spr+0x50bb04 #1 0x55fc9c at .get_spr+0x45edbc #2 0x88649c at .get_spr+0x7855bc #3 0x945a34 at .phyp_pft_hcall+0xd1e4 #4 0x6a2288 at .get_spr+0x5a13a8 #5 0x68d4e4 at .get_spr+0x58c604 #6 0x67bd3c at .get_spr+0x57ae5c #7 0x88172c at .get_spr+0x78084c #8 0x876518 at .get_spr+0x775638 #9 0x8873e8 at .get_spr+0x786508 #10 0x859098 at .get_spr+0x7581b8 #11 0x88fc94 at .get_spr+0x78edb4 #12 0x89a920 at .get_spr+0x799a40 #13 0x895da0 at .get_spr+0x794ec0 #14 0x941370 at .phyp_pft_hcall+0x8b20 #15 0x6a19a8 at .get_spr+0x5a0ac8 #16 0x6a1630 at .get_spr+0x5a0750 #17 0x698290 at .get_spr+0x5973b0 Starting mountd. The rest looks normal. =3D=3D=3D Mark Millard markmi at dsl-only.net
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?49DE6E23-948B-4DC1-87A0-047F8A2CE966>