Date: Tue, 15 Nov 2016 13:42:40 -0800 From: Mark Millard <markmi@dsl-only.net> To: Jukka Ukkonen <jau789@gmail.com> Cc: Nathan Whitehorn <nwhitehorn@freebsd.org>, Justin Hibbits <chmeeedalf@gmail.com>, FreeBSD PowerPC ML <freebsd-ppc@freebsd.org> Subject: Re: svn commit: r306065 - in head/sys vs. PowerMacs: Nathan's trail patch included but inappropriate? [My hack vs. Apple G4's.] Message-ID: <5F29E512-A5F0-452F-B816-FA5907DF875A@dsl-only.net> In-Reply-To: <7E73DEEA-AA99-48CD-A441-5596A6F0D8E8@dsl-only.net> References: <C48933C3-DB22-4D94-B22D-B51822197E4E@dsl-only.net> <917EFF5A-D054-4424-9D7D-4E4BEF6072EF@gmail.com> <4bb1046a-225d-66b2-7b00-067f0d6f6c60@gmail.com> <465041D5-C1A2-48F4-9CA7-DD03B094FAE4@dsl-only.net> <01cfa4e9-954f-3e86-c8d7-36ec8523dde0@freebsd.org> <B28E3336-2F85-4395-8DDF-CF09CA9D41E1@dsl-only.net> <5C253E59-265B-4F5F-A4C5-E4FB7EEBF084@gmail.com> <7E73DEEA-AA99-48CD-A441-5596A6F0D8E8@dsl-only.net>
next in thread | previous in thread | raw e-mail | index | archive | help
[Top post of an experiment with loading iicsmb.ko from the loader = prompt.] I stopped a PowerMac G5 "Quad Core" at the loader prompt (not using /boot/loader.conf or other such) and did a: load iicsmb boot (smbus.ko also loads --and so for my earlier "in kernel" suggestion "device smbus" should also be listed in the KERNCONF file.) It booted fine. Afterwards kldstat reported: # kldstat Id Refs Address Size Name 1 6 0x100000 16901b0 kernel 2 1 0x1792000 14598 iicsmb.ko 3 2 0x17a7000 13f80 smbus.ko Those are not the addresses that were reported at the loader prompt for iicsmb and smbus: .text for iccsmb was listed as at 0x28e0 if I remember right .text for smbus was listed as at 0x2800 if I remember right The .data for each were listed at the loader prompt as each starting in the 0x6xxx range as I remember. I'm not sure that what the loader prompt context listed was actually the load addresses at that time. I do not know if you get similar results or not. If the loader prompt loads always work and the loader.conf loads do not that might also be interesting evidence about the problem(s) involved. I do not know how to tell if iicsmb and smbus are working or not. It may be that explicit loads from the loader prompt are another workaround. I have not yet tried: unload load iccsmb boot Context details (with my boot-hack for PowerMac G5's and such): # uname -apKU FreeBSD FBSDG5L 12.0-CURRENT FreeBSD 12.0-CURRENT #0 r308247M: Fri Nov = 4 03:18:34 PDT 2016 = markmi@FreeBSDx64:/usr/obj/powerpc64vtsc_xtoolchain/powerpc.powerpc64/usr/= src/sys/GENERIC64vtsc-NODBG powerpc powerpc64 1200014 1200014 # svnlite status /usr/src/ M = /usr/src/contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp M /usr/src/lib/csu/powerpc64/Makefile ? /usr/src/sys/arm/conf/BPIM3-DBG ? /usr/src/sys/arm/conf/BPIM3-NODBG ? /usr/src/sys/arm/conf/RPI2-DBG ? /usr/src/sys/arm/conf/RPI2-NODBG ? /usr/src/sys/arm64/conf/GENERIC-DBG ? /usr/src/sys/arm64/conf/GENERIC-NODBG M /usr/src/sys/boot/ofw/Makefile.inc M /usr/src/sys/boot/powerpc/Makefile.inc M /usr/src/sys/boot/powerpc/kboot/Makefile M /usr/src/sys/boot/uboot/Makefile.inc M /usr/src/sys/conf/Makefile.powerpc M /usr/src/sys/conf/kern.mk M /usr/src/sys/conf/kmod.mk M /usr/src/sys/ddb/db_main.c M /usr/src/sys/ddb/db_script.c ? /usr/src/sys/powerpc/conf/GENERIC64vtsc-DBG ? /usr/src/sys/powerpc/conf/GENERIC64vtsc-NODBG ? /usr/src/sys/powerpc/conf/GENERICvtsc-DBG ? /usr/src/sys/powerpc/conf/GENERICvtsc-NODBG M /usr/src/sys/powerpc/ofw/ofw_machdep.c M /usr/src/sys/powerpc/powerpc/exec_machdep.c (exec_machdep.c has my "red-zone for signals" hack for 32-bit powerpc to deal with clang's stack-handling ABI violations in the powerpc 32-bit code that it generates. That allowed me to continue and find other clang code generation problems for the context.) =3D=3D=3D Mark Millard markmi at dsl-only.net On 2016-Nov-15, at 10:52 AM, Mark Millard <markmi@dsl-only.net> wrote: On 2016-Nov-15, at 4:04 AM, Jukka Ukkonen <jau789 at gmail.com> wrote: > The bad news is that with this modification my G5 is > still prone to panic during the very early phases of booting > depending on which kernel modules are preloaded by > loader. Some preloaded modules do not cause a panic, > but they do not work either until they are unloaded and > the loaded again when the kernel is already in control > of everything. Preloaded modules end up in relatively > low addresses. The modules loaded under the control of > the kernel itself end up in very high addresses. > I assume there is some sort of memory corruption which > tends to make a mess in the lower addresses and some > modules are very sensitive to such corruption causing > panics while others are less sensitive and simply refuse > to operate normally. > I guess this might be related to the initialization of sprg0. > IBM's documentation says sprg0 must point to a separate > buffer area for each CPU, but I have failed to convince > myself that sprg0 actually refers a separate CPU specific > buffer space for each CPU at all times when ofw is called. > The old values of the sprg# registers and something else > get apparently stored in the buffer pointed by sprg0. >=20 > --jau >=20 Ignoring for the moment why explicit run-time loads might not work. . . You might be able to create and use your own kernel configuration that pre-builds the modules that you want into the kernel. This might avoid the problems. I do not touch the standard FreeBSD KERNCONF's but include them in mine. Below is what I do that adds sc and and that turns off INVARIANTS, WITNESS, and the like even if the original GENERIC64 has them turned on. I normally do not worry about duplicating something from the included KERNCONF but instead force what I'm interested in at the time. # more /usr/src/sys/powerpc/conf/GENERIC64vtsc-NODBG=20 # # GENERIC -- Custom configuration for the powerpc/powerpc64 # include "GENERIC64" ident GENERIC64vtsc-NODGB makeoptions DEBUG=3D-g # Build kernel with gdb(1) = debug symbols nooptions PS3 # Sony Playstation 3 = HACK!!! to allow sc options KDB # Enable kernel debugger support # For minimum debugger support (stable branch) use: options KDB_TRACE # Print a stack trace for a = panic options DDB # Enable the kernel debugger options GDB # HACK!!! ... # Extra stuff: #options VERBOSE_SYSINIT # Enable verbose sysinit = messages #options BOOTVERBOSE=3D1 #options BOOTHOWTO=3DRB_VERBOSE #options KTR #options KTR_MASK=3DKTR_TRAP ##options KTR_CPUMASK=3D0xF #options KTR_VERBOSE # HACK!!! to allow sc for 2560x1440 display on Radeon X1950 that vt = historically mishandled during booting device sc #device kbdmux # HACK: already listed by vt options SC_OFWFB # OFW frame buffer options SC_DFLT_FONT # compile font in makeoptions SC_DFLT_FONT=3Dcp437 # Disable any extra checking for. . . nooptions DEADLKRES # Enable the deadlock resolver nooptions INVARIANTS # Enable calls of extra sanity = checking nooptions INVARIANT_SUPPORT # Extra sanity checks of = internal structures, required by INVARIANTS nooptions WITNESS # Enable checks to detect = deadlocks and cycles nooptions WITNESS_SKIPSPIN # Don't run witness on spinlocks = for speed nooptions DIAGNOSTIC nooptions MALLOC_DEBUG_MAXZONES # Separate malloc(9) zones You might not want to enable DDB (and/or GDB). In the early boot things are not ready for user input to ddb. I actually modify ddb to execute an internal script automatically before it gets to the prompt. DDB input does work for problems that happen late enough. (GDB may well be irrelevant. It is carry over from my first experiments and I've never disabled it but also have never used it.) I specify KERNCONF=3DGENERIC64vtsc-NODBG in my kernel building activities for powerpc64 in order to use the above. It the past you have listed the following as problems: tmpfs mqueuefs iicsmb Have you tried putting these into the kernel itself? options TMPFS options P1003_1B_MQUEUE device iicsmb If being in the kernel gets rid of the problems: A) You then have a workaround. B) It is evidence that might eventually contribute to understanding the failing case (compare and contrast with runtime loading). I do not know if run-time loaded modules are supposed to stay in low memory or not (vs. being relocated). That would be a question for Nathan I'd guess. iicsmb depends on iicbus, and iicbus is a device in the kernel = (GENERIC64). So iicbus relocates if/when the kernel is relocated. That would mean = that iicsmb would have to track such relocations if they happen after iicsmb = is loaded. (I do not know the detailed ordering of things. I'd have to go analyze the source code to figure it out.) I will note that so far I've not seen anything that even suggests if SPRG<?> registers are involved in your problems or not. My memory of = what I saw in the kernel code back when I was finding the hack matches up = with what Justin said about OFW calls on the PowerMacs: only one CPU/core is active at the time. Openfirmware calls interfere with concurrency in = order to protect the system: they have rather high overhead in the system. (Of course if openfirmware itself ever started a CPU/core there likely would be uncontrolled consequences.) It is too bad that "set usefdt=3D1" failed when I tried it on the quad = core. May be I should try it again now that I'm running a more recent version = of head instead of -r302214. What I got under -r302214 was: Ok set usefdt=3D1 Ok boot Booting... Error -2 adding node /ht@0,f2000000/pci@8/macio@7/i2c@18000/i2c-bus@0 = (i2c-bus@0), skipping kernel entry at 0x100120 Invalid memory access at %SRR0: 00000000.00100120 %SRR1: = 10000000.00083030 It then reported the Apple model and firmware version and and some other Apple text and got stuck. (Power switch time.) Without the "set usefdt=3D1" it booted normally. I have access to the powerpc's and powerpc64's again (at least for now). So technically it is possible for me to do experiments again. Because of my bias to testing and reporting clang/clang++ issues for powerpc64 and powerpc I run head or the clang import project on the powerpc*'s. I'm waiting on a couple of llvm powerpc64 and powerpc fixes being = applied to projects/clang390-import before switching to it so currently I'm running head (-r308247 at the moment). >> On 15 Nov 2016, at 7.53, Mark Millard <markmi@dsl-only.net> wrote: >>=20 >> On 2016-Sep-21, at 3:01 PM, Peter Grehan <grehan at freebsd.org> = wrote: >>=20 >>>> I've never had my hands on an eMac. I'm not sure if it has the same >>>> SPRGx requirements as other G4's or not. >>>=20 >>> That's the machine that caused the sprg save/restore. >>> https://svnweb.freebsd.org/base?view=3Drevision&revision=3D151891 >>>=20 >>> later, >>>=20 >>> Peter. >>=20 >> My patch/hack as I have always had it for booting PowerMac G5's does = not >> change the content for G4's (such as eMac G4's). I made a separate = case >> for __powerpc64__ --and only for AIM-capable as well. My code has = been: >>=20 >> # svnlite diff /usr/src/sys/powerpc/ofw/ofw_machdep.c=20 >> Index: /usr/src/sys/powerpc/ofw/ofw_machdep.c >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> --- /usr/src/sys/powerpc/ofw/ofw_machdep.c (revision 308247) >> +++ /usr/src/sys/powerpc/ofw/ofw_machdep.c (working copy) >> @@ -111,6 +111,24 @@ >> * Assume that interrupt are disabled at this point, or >> * SPRG1-3 could be trashed >> */ >> +#if defined(AIM) && defined(__powerpc64__) >> +/* HACK: PowerMac G5 specific code to avoid demonstrated hangs in >> + * the early boot time frame: avoid mtsprg0 use. >> + * This would need a live test for PowerMac vs. not in order >> + * to remove HACK status --but without calling into >> + * OpenFirmware or the problem would be recreated. >> + */ >> + if (1) >> + __asm __volatile("mfsprg0 %0\n\t" >> + "mtsprg1 %1\n\t" >> + "mtsprg2 %2\n\t" >> + "mtsprg3 %3\n\t" >> + : "=3D&r"(ofw_sprg0_save) >> + : "r"(ofmsr[2]), >> + "r"(ofmsr[3]), >> + "r"(ofmsr[4])); >> + else >> +#endif >> __asm __volatile("mfsprg0 %0\n\t" >> "mtsprg0 %1\n\t" >> "mtsprg1 %2\n\t" >>=20 >> (The comment does not reflect the modern discussion of why the >> change works: the code has been around a long time.) >>=20 >> I do not know if there are any AIM-capable __powerpc64__ >> contexts for which the change is inappropriate. It would be >> nice if it was always appropriate to that kind of context >> and could be added to head and stable/11 and stable/10 . >> (I've no experience before stable/10: so untested prior to >> that.) >>=20 >> Jukka did experiments with eliminating more --and not just for >> AIM-capable __powerpc64__ but some other PowerMac contexts as >> well. >>=20 >> But I kept to the minimal generated-code change which appeared >> to fix the problematical behavior. >>=20 >> =3D=3D=3D >> Mark Millard >> markmi at dsl-only.net =3D=3D=3D Mark Millard markmi at dsl-only.net
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5F29E512-A5F0-452F-B816-FA5907DF875A>