Date: Tue, 28 Jan 2020 08:51:21 -0800 From: Mark Millard <marklmi@yahoo.com> To: Justin Hibbits <chmeeedalf@gmail.com> Cc: freebsd-ppc <freebsd-ppc@freebsd.org> Subject: Re: I have submitted https://reviews.freebsd.org/D23376 for avoiding the PowerMac threads-stuck-sleeping issue Message-ID: <5175A179-4B5B-4D8D-B954-1E1BFDC9467E@yahoo.com> In-Reply-To: <20200128085747.7699601e@titan.knownspace> References: <1EDABAB4-87F5-4C8E-A7B9-9143D520BDE0.ref@yahoo.com> <1EDABAB4-87F5-4C8E-A7B9-9143D520BDE0@yahoo.com> <20200128085747.7699601e@titan.knownspace>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2020-Jan-28, at 06:57, Justin Hibbits <chmeeedalf@gmail.com> wrote:
> On Sun, 26 Jan 2020 15:28:41 -0800
> Mark Millard via freebsd-ppc <freebsd-ppc@freebsd.org> wrote:
>
>> I've described my first ever reviews.freebsd.org/D?????
>> submittal with:
>>
>> QUOTE
>> REVISION SUMMARY
>> Explicitly initialize the bsp's TB value early on. For PowerMacs
>> with multiple sockets or cores only, have machdep_ap_bootstrap for
>> each ap make a round trip with the bsp to get a current bsp's TB
>> value and then shift the ap's time so that the midpoint of the
>> shifted round tip interval approximately matches the TB value the bsp
>> supplied. The round trips and data transfers use a memory protocol.
>>
>> For non-PowerMacs, do the old way for the ap's.
>>
>> TEST PLAN
>> I've been running based on the patches being invovled since late
>> 2019-May or so: PowerMac11,2 with 2 sockets, 2 cores each, and
>> PowerMac3,6 with 2 sockets, 1 core each. No examples of 1 socket with
>> 2 cores, unfortunately. No examples of powerpc64 or powerpc
>> non-PowerMacs either. I do use the same source tree to build for
>> amd64, aarch64, and armv7 and they have been working too. But the
>> patch material is extracted from a source tree that has more local
>> changes: the testing has not been with only these changes.
>>
>> I've not had any examples of the stuck-sleeping issue since adopting
>> these changes. Nor have I had new problems from the changes. END QUOTE
>>
>>
>> A summary of my changes is:
>>
>>
>> Part 0:
>> Avoid interrupts messing up mttb use:
>>
>> sys/powerpc/include/cpufunc.h
>>
>>
>> Part 1:
>> Enable the "alternate_timebase_sync_style" indicator
>> only for PowerMacs:
>>
>> sys/powerpc/powermac/platform_powermac.c
>>
>>
>> Part 2:
>> The code supporting machdep_ap_bootstrap's part of the
>> hard work, including the changes to machdep_ap_bootstrap:
>>
>> sys/powerpc/powerpc/mp_machdep.c
>>
>>
>> Part 3:
>> Making sure that the bsp follows the rule:
>> "The TB is a volatile resource and must be initialized during reset."
>>
>> sys/powerpc/powerpc/machdep.c
>>
>>
>>
>>
>> I'll note that I'm not style(9) literate/familiar and at the time
>> my capacity went to investigating the problem and potential solutions,
>> not to style(9). So, once a technique is classified as okay, how
>> it is written in the source would need its own round of work,
>> presuming my submittal does not inspire a separate fix by someone.
>
> Hi Mark,
>
> Thanks for taking this on. I'm glad someone can do it, since I don't
> have a FreeBSD G5 anymore. However, your approach only gets partway to
> the correct solution. Please see the solution I made for the mpc85xx,
> in mpc85xx/platform_mpc85xx.c; tl;dr being that you need to stop the
> timebase on all cores, align them, then restart the timebase. On the
> SoCs it's easy, since it's just a simple register. On PowerMac
> hardware it's still doable, but requires some extra steps. Some
> platforms require twiddling a GPIO ('timebase-enable' node, using a
> 'platform function' which you'd either need to write an interpreter
> for, or manually interpret it into C), while others (PowerMac7,2,
> PowerMac7,3, and RackMac3,1 according to Linux) require twiddling on
> i2c.
One of my test cases is a PowerMac7,2 and my code has avoided the
threads-stuck-sleeping problem there whenever I've tested it. But
I've not run that context as much as the 2 socket, 2 cores each G5
context. I could set the 7,2 up to be running on its own rather
than as an alternative. That would get more testing time in.
I expect that your suggestions would make the times more accurate
relative to each other, but I also expect that the degree of
accuracy is not required for the limited purpose of avoiding the
threads-stuck-sleeping problem. That is all I was trying for.
All the multi-socket or mult-core powerpcs that I have access to
are examples of:
PowerMac11,2 2.5 GHz PowerPC 970MP 2 sockets, 2 cores each
PowerMac7,2 2.0 GHz PowerPC 970 2 sockets, 1 core each
PowerMac3,6 1.42 GHz PowerPC 7455 2 sockets, 1 core each
As stands, the 2 socket G4 (7455) has the worst time matching.
But it has still been accurate enough to avoid the
threads-stuck-sleeping problem in my testing. (The code did
improve the time relationships observed compared to not having
it.)
I'd love it if more example multi-core/multi-socket PowerMacs
types could be tested. I'd like to know of any PowerMac
threads-stuck-sleeping problems observed with my code in place.
Until I have a test context that I know shows a behavioral
difference for threads-stuck-sleeping for my existing technique,
I'm not likely to try more involved, complicated code or code
that needs more cases involved (e.g., GPIO vs. i2c involved).
(Sort of like your not having a G5 to work on any more stops
your doing these sorts of things for such contexts.)
(Note: Having GPIO or i2c involved would be a research subject
for me, not familiar material. But the above would apply even
if I was familiar with involving these.)
I'm not sure if your wording "some platforms require" extends
outside PowerMacs or is a PowerMac variation reference. Are
there non-PowerMacs that have observed the threads-stuck-sleeping
problem? What are they (if any)? I certainly did not explicitly
attempt to handle any such. In fact, the patched code would
operate in the old way for such.
> Using the disable-sync-enable method with rendezvous would provide the
> best sync option, and would avoid messing with the platform independent
> code.
>
===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5175A179-4B5B-4D8D-B954-1E1BFDC9467E>