Date: Tue, 24 Dec 1996 14:00:46 +0800 From: Peter Wemm <peter@spinner.dialix.com> To: Erich Boleyn <erich@uruk.org> Cc: smp@freebsd.org, haertel@ichips.intel.com, wscott@ichips.intel.com Subject: Re: I think we have the culprit!! (was -> Re: Eureka (maybe...) (was -> Re: P6 problem idea ) ) Message-ID: <199612240600.OAA19697@spinner.DIALix.COM> In-Reply-To: Your message of "Mon, 23 Dec 1996 22:21:49 PST." <E0vcQFJ-0003Y6-00@uruk.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Erich Boleyn wrote: > Erich Boleyn <erich@uruk.org> writes: > > I tried shutting off the page global stuff, and while I don't have > > a difinitively long run yet, it has run through 3 full kernel compiles > > with no crash yet. I'll run it for the next 1 1/2 hours and see if > > it lives through that. If so, I think we have our main culprit (I'll > > also post the (small) code change which synchronizes the CPUs on TLB > > shootdown before letting the sender continue). > > Well, after 2 hours of kernel builds, and now a few sets of 4 parallel > kernel builds later, the system is still running great. > > I think we have our culprit... the Page Global stuff (plus adding the > TLB shootdown synchronization may be helping a little with stability, but > it's absence doesn't appear to be the major cause). Hmm.. Interesting... I have a theory. On the standard kernel when on a cpu_class >= PPro (not Pentium) we set the PG_G bits. We also have an invltlb() function call as well as the page level invlpg() and invl2pg() calls. (invl2pg just does two invlpg's in a single function call to lower the function call overheads). On the SMP kernel, all three of these functions cause an "global invalidate" broadcast. If the initiating cpu is actually trying to modify a PG_G page, this will screw up since the per-page invalidate gets converted to a global invalidate on the other cpu's, and hence they don't flush their PG_G page. Does that sound like a plausable explanation? If so, we need to refine the implementation of TLB shootdowns more so that we can initiate a per-page flush as well as a global flush.. This will require syncronisation, so if you can send your code you can save some reinvention.. :-) > > All that said, I'm very surprised that this *isn't* also a serious > > problem on the Pentium (the Pentium has the Page Global stuff as > > well... I didn't look to see if it is used for the Pentium as well > > as the Pentium Pro). > > I might be confused here, but as mentioned in the above comment, I > thought this was implemented in the Pentium as well. Can someone > who remembers better (or has the "Appendix H" equivalent released > documentation) comment? Don't know about the Pentium, but we definately don't enable it on anything smaller than a PPro. Cheers, -Peter
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199612240600.OAA19697>