From owner-freebsd-smp  Sun Dec 15 02:50:00 1996
Return-Path: <owner-smp>
Received: (from root@localhost)
          by freefall.freebsd.org (8.8.4/8.8.4) id CAA17093
          for smp-outgoing; Sun, 15 Dec 1996 02:50:00 -0800 (PST)
Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67])
          by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id CAA17068;
          Sun, 15 Dec 1996 02:49:48 -0800 (PST)
Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1])
          by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id SAA19672;
          Sun, 15 Dec 1996 18:49:29 +0800 (WST)
Message-Id: <199612151049.SAA19672@spinner.DIALix.COM>
To: Poul-Henning Kamp <phk@critter.tfs.com>
cc: dyson@freebsd.org, smp@freebsd.org, haertel@ichips.intel.com
Subject: Re: some questions concerning TLB shootdowns in FreeBSD 
In-reply-to: Your message of "Sun, 15 Dec 1996 11:17:14 +0100."
             <9319.850645034@critter.tfs.com> 
Date: Sun, 15 Dec 1996 18:49:28 +0800
From: Peter Wemm <peter@spinner.dialix.com>
Sender: owner-smp@freebsd.org
X-Loop: FreeBSD.org
Precedence: bulk

Poul-Henning Kamp wrote:
> In message <199612151003.SAA14741@spinner.DIALix.COM>, Peter Wemm writes:
> >Poul-Henning Kamp wrote:
> >> In message <199612150121.JAA12763@spinner.DIALix.COM>, Peter Wemm writes:
> >> 
> >> >However, the shared address space code that I was working on in
> >> >-current (for kernel assisted threading in the smp kernel) means
> >> >that a single vmspace/pmap/etc can be shared among multiple processes
> >> >and this changes the above picture since two cpu's can be using
> >> >the user mode parts of the same page tables at once, one in executing
> >> >in user mode, one in the kernel.
> >> 
> >> But we could still have a per-cpu flags:
> >> 	"I'm not in a shared address-space"
> >> 
> >> Ie, this would only be set if the CPU was in userland in a non-threaded
> >> process.
> >
> >eg, something like: 
> >if (is_userland && curproc->p_vmspace->vm_refcnt > 1)
> >  send_tlb_invalidate();
> 
> Well more like:
> 
> 	for (i=0;i<NCPU;i++)
> 		if (is_userland(i) && curproc[i]->p_vmspace->vm_refcnt > 1)
> 			send_tlb_invalidate(i);

I'm not sure that this is needed..  After all, we're deciding whether we
have to flush the TLB of remote processors when fiddling with the user
level components of the local current process.  If the local process is
not shared, we can ignore what the other cpu's are doing.

Anyway, it's probably more correct to say:
  if (!is_userland || curproc....vm_refcnt > 1)
    smp_invltlb();
since as a general rule we'd always want to arrange a tlb flush for kernel
accesses.  I simplified the comment originally because for the kernel
accesses, we can simply set a bit to cause the other cpu's to invalidate
their tlb on entry to the kernel when they go for the mutex lock.

But, for accesses to non-shared process user spaces that are not curproc
on the local cpu, we must do something like this:
  for (i = 0; i < NCPU; i++) {
    if (i != cpnumber() && cpu_present[i] && SMPcurproc[i] == p)
      flush_bits |= 1 << i;
  }
  if (flush_bits)
    freeze_cpu_mask(flush_bits);
  fiddle_non_current_process_PTE()
  if (flush_bits)
    wake_cpu_mask(flush_bits);

etc.  I can tell that John is going to have a heart attack over this.. :-)

Cheers,
-Peter