From owner-freebsd-smp  Thu Dec  5 11:18:29 1996
Return-Path: <owner-smp>
Received: (from root@localhost)
          by freefall.freebsd.org (8.8.4/8.8.4) id LAA25498
          for smp-outgoing; Thu, 5 Dec 1996 11:18:29 -0800 (PST)
Received: from uruk.org (root@ns.uruk.org [198.145.95.253])
          by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id LAA25488
          for <smp@freebsd.org>; Thu, 5 Dec 1996 11:18:24 -0800 (PST)
Received: from uruk.org [127.0.0.1] (erich)
	by uruk.org with esmtp (Exim 0.53 #1)
	id E0vVjN4-0005HN-00; Thu, 5 Dec 1996 11:22:10 -0800
To: Steve Passe <smp@csn.net>
cc: Terje Normann Marthinussen <Terje.N.Marthinussen@cc.uit.no>,
        smp@freebsd.org
Subject: Re: >2 CPU SMP test 
In-reply-to: Your message of "Thu, 05 Dec 1996 09:51:49 MST."
             <199612051651.JAA13340@clem.systemsix.com> 
Date: Thu, 05 Dec 1996 11:22:10 -0800
From: Erich Boleyn <erich@uruk.org>
Message-Id: <E0vVjN4-0005HN-00@uruk.org>
Sender: owner-smp@freebsd.org
X-Loop: FreeBSD.org
Precedence: bulk


Steve Passe <smp@csn.net> writes:

...

> some possible reasons:
> 
>   the bootstrap code, bootMP & MPentry, gets corrupted by being run the 1st
>   time.
> 
>   the code in startAP() somehow leaves its APIC registers in a state where
>   a second run of startAP() doesn't function the same.
> 
>   the first run of startAP() somehow affects all APs, leaving them in a state
>   where they can't respond properly. (partially INITed, or ?)
> 
>   ???
> 
>   one good test of several of the above would be to modify startAllAPs()"
> 
>     /* install the AP 1st level boot code */
>     installApTramp( bootAddr );
> 
>     /* save the current value of the warm-start vector */
>     mpbioswarmvec = *((u_long*)WARMBOOT_OFF);
>     outb( CMOS_REG, BIOS_RESET );
>     mpbiosreason = inb( CMOS_DATA );
> 
>     /* setup a vector to our boot code */
>     *((volatile u_short *)WARMBOOT_OFF) = WARMBOOT_TARGET;
>     *((volatile u_short *)WARMBOOT_SEG) = (bootAddr >> 4);
>     outb( CMOS_REG, BIOS_RESET );
>     outb( CMOS_DATA, BIOS_WARM );	/* write 'warm-start' */
> 
>     /* start each AP */
>     for ( x = 1; x <= mp_naps; ++x ) {
> 	startAP( CPU_TO_ID( x ), (bootAddr >> 12) & 0xff );
> 	cpuApicVersions[ x ] = cpuApicVersions[ 0 ];
>     }
> 
> to:

...

Your problem here is definitely that the warm reset code and vector
are not being set for each CPU.  The *entire* setup sequence for
AP CPU bootup must be done for each CPU (except maybe a memcpy
that isn't overwritten by anything the AP CPU does).

In particular, on an XXPRESS-based Pentium system, the STARTUP IPI is
ignored, and it needs the warm reset code and vector.

The problem here is that the reset code is changed to indicate what
happened to the CPU after it was reset.

For a Pentium Pro-based system, your code would work correctly,
since all of them that I've ever heard of use the integrated APIC
stuff correctly (and therefore use the STARTUP IPI).

--
  Erich Stefan Boleyn                 \_ E-mail (preferred):  <erich@uruk.org>
Mad Genius wanna-be, CyberMuffin        \__      (finger me for other stats)
Web:  http://www.uruk.org/~erich/     Motto: "I'll live forever or die trying"