Skip site navigation (1)Skip section navigation (2)
Date:      Sat, 12 Jul 1997 16:28:08 -0600
From:      Steve Passe <smp@csn.net>
To:        smp@freebsd.org
Subject:   APIC saga continues
Message-ID:  <199707122228.QAA16759@Ilsa.StevesCafe.com>

next in thread | raw e-mail | index | archive | help
Hi,

I have been beating my head on the wall the last 2 weeks trying to get APICs
working for syncing CPUs.  I think I finally see light at the end of the
tunnel.  But getting there I discovered more ugly 'features' of the APIC.

Till now we have been using "mixed mode programming" where the 8259 PIC's
INT output is routed thru the APIC and handled as an 'ExtInt' on those
machines that don't route the 8254 to the APIC.  This includes both machines
that require "options SMP_TIMER_NC" and those which lack a line like:

I/O Ints:       Type    Polarity    Trigger     Bus ID   IRQ    APIC ID INT#
                INT      conforms    conforms        0     0          2    2

in their mptable.

It turns out that when you do this the 8254 timer INTs (or any INT from the
8259) are NOT reflected in the APIC IRR/ISR registers, and CANNOT be masked
via the APIC TPR register/process priority scheme.  This is unacceptable as
we need this facility to direct INTs to those CPUs capable of handling them
(and avoiding those that can't).  As an example, when a CPU is blocked
on a lock when in an INTerrupt routine, the IE flag is clear (INTs masked)
so the CPU can't receive Xinvltlb and other important IPIs.  By setting its
TPR to a level that blocks hardware INTs, but allows IPIs, and then doing an
STI, it can then receive the IPIs while ignoring the hardware INTs.  BUT
this all falls apart for the reasons described above concerning the failire
of ExtInts to be restricted by IRR/ISR bits.

---
Intel has a solution in:

Intel486(TM) MicroProcessors and related Products, order #241731-002 
      pp 4-220 thru 4-302: 82489DX Advanced Programmable Interrupt Controller

specifically see page 4-292: 824889DX/8259A DUAL MODE CONNECTION

their solution for truly symmetric IO in this situation is:

 - program the 8259 to pass ONLY the DMA chaining INT
 - program the IO APIC to handle this as a regular (ie not 'ExtInt') input
 - abandon the 8254 timer, instead using the APIC's internal timer.

---
When you abandon the 'ExtInt' mode, you cannot handle more than i 8259 INT
as it no longer provides the vector, the APIC does.
I believe that we don't want to abandon the 8254, but instead should
abandon the DMA chaining INTs (who uses these anyways???)  Then we
can program in a similar way, but instead pass the 8254 INT thru as
a regular INT.  I have written test code that does this, and it seems
to run nicely.  I specifically trapped on the first timer INT and showed
thru inspection of the APIC ISRs that the old 'ExtInt' method does indeed
bypass the ISR bits, and that the Intel recommended 'non-ExtInt' method
sets the ISR bit properly.

SO I have alot of work ahead of me getting all my experiments cleaned up
and into the main code tree, and the CPU sync code that I started 2 weeks
ago in working shape now that I know what was biting me.

But in the meantime we need to make a policy decision as to whether we can say
bye-bye to the DMA chaining INTs.  If I don't get thoughtful feedback on this
I will just nmake an arbitrary desision (which I suspect to be axing the DMA 
INTs).  I think the only situation needing them is non-busmaster EISA hardware
that does DMA via the motherboard chipset DMA registers.  Please correct me
if I am wrong on this point.

--
Steve Passe	| powered by
smp@csn.net	|            Symmetric MultiProcessor FreeBSD





Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199707122228.QAA16759>