From owner-freebsd-smp Sat Jul 12 15:28:14 1997 Return-Path: Received: (from root@localhost) by hub.freebsd.org (8.8.5/8.8.5) id PAA22738 for smp-outgoing; Sat, 12 Jul 1997 15:28:14 -0700 (PDT) Received: from Ilsa.StevesCafe.com (Ilsa.StevesCafe.com [205.168.119.129]) by hub.freebsd.org (8.8.5/8.8.5) with ESMTP id PAA22732 for ; Sat, 12 Jul 1997 15:28:10 -0700 (PDT) Received: from Ilsa.StevesCafe.com (localhost [127.0.0.1]) by Ilsa.StevesCafe.com (8.8.5/8.8.5) with ESMTP id QAA16759 for ; Sat, 12 Jul 1997 16:28:08 -0600 (MDT) Message-Id: <199707122228.QAA16759@Ilsa.StevesCafe.com> X-Mailer: exmh version 2.0gamma 1/27/96 From: Steve Passe To: smp@freebsd.org Subject: APIC saga continues Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sat, 12 Jul 1997 16:28:08 -0600 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, I have been beating my head on the wall the last 2 weeks trying to get APICs working for syncing CPUs. I think I finally see light at the end of the tunnel. But getting there I discovered more ugly 'features' of the APIC. Till now we have been using "mixed mode programming" where the 8259 PIC's INT output is routed thru the APIC and handled as an 'ExtInt' on those machines that don't route the 8254 to the APIC. This includes both machines that require "options SMP_TIMER_NC" and those which lack a line like: I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID INT# INT conforms conforms 0 0 2 2 in their mptable. It turns out that when you do this the 8254 timer INTs (or any INT from the 8259) are NOT reflected in the APIC IRR/ISR registers, and CANNOT be masked via the APIC TPR register/process priority scheme. This is unacceptable as we need this facility to direct INTs to those CPUs capable of handling them (and avoiding those that can't). As an example, when a CPU is blocked on a lock when in an INTerrupt routine, the IE flag is clear (INTs masked) so the CPU can't receive Xinvltlb and other important IPIs. By setting its TPR to a level that blocks hardware INTs, but allows IPIs, and then doing an STI, it can then receive the IPIs while ignoring the hardware INTs. BUT this all falls apart for the reasons described above concerning the failire of ExtInts to be restricted by IRR/ISR bits. --- Intel has a solution in: Intel486(TM) MicroProcessors and related Products, order #241731-002 pp 4-220 thru 4-302: 82489DX Advanced Programmable Interrupt Controller specifically see page 4-292: 824889DX/8259A DUAL MODE CONNECTION their solution for truly symmetric IO in this situation is: - program the 8259 to pass ONLY the DMA chaining INT - program the IO APIC to handle this as a regular (ie not 'ExtInt') input - abandon the 8254 timer, instead using the APIC's internal timer. --- When you abandon the 'ExtInt' mode, you cannot handle more than i 8259 INT as it no longer provides the vector, the APIC does. I believe that we don't want to abandon the 8254, but instead should abandon the DMA chaining INTs (who uses these anyways???) Then we can program in a similar way, but instead pass the 8254 INT thru as a regular INT. I have written test code that does this, and it seems to run nicely. I specifically trapped on the first timer INT and showed thru inspection of the APIC ISRs that the old 'ExtInt' method does indeed bypass the ISR bits, and that the Intel recommended 'non-ExtInt' method sets the ISR bit properly. SO I have alot of work ahead of me getting all my experiments cleaned up and into the main code tree, and the CPU sync code that I started 2 weeks ago in working shape now that I know what was biting me. But in the meantime we need to make a policy decision as to whether we can say bye-bye to the DMA chaining INTs. If I don't get thoughtful feedback on this I will just nmake an arbitrary desision (which I suspect to be axing the DMA INTs). I think the only situation needing them is non-busmaster EISA hardware that does DMA via the motherboard chipset DMA registers. Please correct me if I am wrong on this point. -- Steve Passe | powered by smp@csn.net | Symmetric MultiProcessor FreeBSD