From owner-freebsd-smp  Sat Dec 14 17:22:09 1996
Return-Path: <owner-smp>
Received: (from root@localhost)
          by freefall.freebsd.org (8.8.4/8.8.4) id RAA10071
          for smp-outgoing; Sat, 14 Dec 1996 17:22:09 -0800 (PST)
Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67])
          by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id RAA10063;
          Sat, 14 Dec 1996 17:22:02 -0800 (PST)
Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1])
          by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id JAA12763;
          Sun, 15 Dec 1996 09:21:56 +0800 (WST)
Message-Id: <199612150121.JAA12763@spinner.DIALix.COM>
To: dyson@freebsd.org
cc: smp@freebsd.org, haertel@ichips.intel.com
Subject: Re: some questions concerning TLB shootdowns in FreeBSD 
In-reply-to: Your message of "Sat, 14 Dec 1996 13:38:47 EST."
             <199612141838.NAA00208@dyson.iquest.net> 
Date: Sun, 15 Dec 1996 09:21:55 +0800
From: Peter Wemm <peter@spinner.dialix.com>
Sender: owner-smp@freebsd.org
X-Loop: FreeBSD.org
Precedence: bulk

"John S. Dyson" wrote:
> > 
> > I'm still digesting it,  I am almost worried that we might (shudder!)
> > be forced into doing an IPI to stop all the cpu's *before* the
> > current cpu changes the page tables, then letting them do the tlb
> > flush and letting them proceed.  If this actually is a real problem
> > this means a much bigger code impact.
> > 
> The way that I see it, is that the current pmap code is highly optimized
> for single processor operation.  If I was you, I would try to just
> try to get something working correctly algorithmically -- almost ignoring
> performance issues.  Of course, when performance is easy -- go for that
> also.
> 
> Alot of things like single page invalidates inside of loops appear that
> they could be evil for multi-processor applications (imagine an inter-
> processor interrupt for every loop!?!?.)  I think that you (we or us),
> will have to look at the performance for the SMP direction, and it
> might even entail large differences in pmap eventually.  Hopefully,
> we will all be able to isolate the differences for the maintenance of
> sanity :-).
> 
> John

Originally, I wondered if the CMAP/CADDR and APTD stuff might need
to be per-cpu but couldn't think of a good reason given our presently
99.8% non-reentrant kernel (the IPI code is reentrant).  Perhaps
this is one of them...  I don't recall how much code walks through
the page tables and how much uses CADDR/APTD.

When dealing with the user space of the currently active process
context, remote TLB locking/flushing is not needed as long as other
cpu's cannot get to the same space via their APTD (which should be
valid as long as we have a global lock) for the high level stuff.

However, the shared address space code that I was working on in
-current (for kernel assisted threading in the smp kernel) means
that a single vmspace/pmap/etc can be shared among multiple processes
and this changes the above picture since two cpu's can be using
the user mode parts of the same page tables at once, one in executing
in user mode, one in the kernel.

Cheers,
-Peter