Date: Thu, 2 Feb 2017 00:37:56 -0800 From: Mark Millard <markmigm@gmail.com> To: Tom Vijlbrief <tvijlbrief@gmail.com>, freebsd-arm <freebsd-arm@freebsd.org> Subject: Re: Arm64 stack issues (was Re: FreeBSD status for/on ODroid-C2?) Message-ID: <A95CC1DC-36C4-4FC3-A8D4-BDBE6FCB136B@gmail.com> In-Reply-To: <F6C3286F-46DF-4819-BDD2-10904018E70C@dsl-only.net> References: <CAOQrpVfK-Dw_rSo_YVY5MT1wbc6Ah-Pj%2BWv8UGjeiUQ1b3%2B-mg@mail.gmail.com> <20170124191357.0ec0abfd@zapp> <20170128010138.iublazyrhhqycn37@mutt-hardenedbsd> <20170128010223.tjivldnh7pyenbg6@mutt-hardenedbsd> <CAOQrpVfxKvSR5PoahnqEsYspHhjjOGJ8iCBUetKxRV57oX_aUg@mail.gmail.com> <009857E3-35BB-4DE4-B3BB-5EC5DDBB5B06@dsl-only.net> <CAOQrpVdKyP2T0V77sfpuKbNP3ARoD1EcwtH6E9o7p5KF%2B=A56A@mail.gmail.com> <CB36F13F-85E9-41D2-A7F3-DA183BE5985A@dsl-only.net> <890B7D8A-27FF-41AC-8291-1858393EC7B1@gmail.com> <54642E5C-D5D6-45B7-BB74-2407CFB351C2@dsl-only.net> <EB1D79C2-CF5E-4C21-BA1B-EC9F34BB737E@gmail.com> <F6C3286F-46DF-4819-BDD2-10904018E70C@dsl-only.net>
next in thread | previous in thread | raw e-mail | index | archive | help
I'm out of my element here but I will note one difference between what I read in the likes of: = http://infocenter.arm.com/help/topic/com.arm.doc.den0024a/DEN0024A_v8_arch= itecture_PG.pdf and what I see in /usr/src/sys/arm/arm/gic.c 's arm_gic_intr . The written description says (quoting 10.6 The Generic Interrupt Controller and 10.6.3 Interrupt Handling): > Interrupts can either be edge-triggered (considered to be asserted = when the GIC detects a rising edge on the relevant input, and to remain = asserted until cleared) or level-sensitive (considered to be asserted = only when the relevant input to the GIC is HIGH). >=20 > . . . >=20 > The priority and list of cores to which an interrupt can be delivered = to are all configured in the Distributor. An interrupt asserted to the = Distributor by a peripheral is in the Pending state (or Active and = Pending if it was already Active). The Distributor determines the = highest priority pending interrupt that can be delivered to a core and = forwards that to the CPU interface of the core. At the CPU interface, = the interrupt is in turn signaled to the core, at which point the core = takes the FIQ or IRQ exception. >=20 > The core executes the exception handler in response. The handler must = query the interrupt ID from a CPU interface register and begin servicing = the interrupt source. When finished, the handler must write to a CPU = interface register to report the end of processing. >=20 > =E2=80=A2 For a given interrupt the typical sequence is: >=20 > =E2=80=A2 Inactive -> Pending > When the interrupt is asserted by the peripheral. >=20 > =E2=80=A2 Pending -> Active > When the handler acknowledges the interrupt. >=20 > =E2=80=A2 Active -> Inactive > When the handle[r] has finished dealing with the interrupt. >=20 > . . . >=20 > The top-level interrupt handler reads the Interrupt Acknowledge = Register from the CPU Interface block to obtain the interrupt ID. >=20 > As well as returning the interrupt ID, the read causes the interrupt = to be marked as active in the Distributor. Once the interrupt ID is = known (identifying the interrupt source), the top-level handler can now = dispatch a device-specific handler to service the interrupt. >=20 > When the device-specific handler finishes execution, the top-level = handler writes the same interrupt ID to the End of Interrupt (EoI) = register in the CPU Interface block, indicating the end of interrupt = processing. So that wording indicates that the write to GICC_EOIR should be after the dispatched activity (after "servicing"), not before. I did not find anything indicating that edge-triggered vs. level triggered would be different for this, for example. (But being unfamiliar I could have missed something.) In two cases below the code has the write to the GICC_EOIR before the dispatch (so before the servicing activity), possibly allowing another interrupt during or even before the dispatched activity (say if the state for the irq is active-and-pending at the time of the GICC_EOIR write or if there is a lower priority interrupt pending at that time): > if (irq <=3D GIC_LAST_SGI) { > #ifdef SMP > /* Call EOI for all IPI before dispatch. */ > gic_c_write_4(sc, GICC_EOIR, irq_active_reg); > intr_ipi_dispatch(sgi_to_ipi[gi->gi_irq], tf); > goto next_irq; > #else > . . . > #endif > } >=20 > . . . > if ((gi->gi_flags & GI_FLAG_EARLY_EOI) =3D=3D = GI_FLAG_EARLY_EOI) > gic_c_write_4(sc, GICC_EOIR, irq_active_reg); >=20 > if (intr_isrc_dispatch(&gi->gi_isrc, tf) !=3D 0) { > gic_irq_mask(sc, irq); > if ((gi->gi_flags & GI_FLAG_EARLY_EOI) !=3D = GI_FLAG_EARLY_EOI) > gic_c_write_4(sc, GICC_EOIR, irq_active_reg); > device_printf(sc->gic_dev, "Stray irq %u disabled\n", = irq); > } >=20 > next_irq: . . . Note: GI_FLAG_EARLY_EOI was set for edge triggered: > /* For MSI/MSI-X we should have already configured these */ > if ((gi->gi_flags & GI_FLAG_MSI) =3D=3D 0) { > if (pol =3D=3D INTR_POLARITY_CONFORM) > pol =3D INTR_POLARITY_LOW; /* just pick = some */ > if (trig =3D=3D INTR_TRIGGER_CONFORM) > trig =3D INTR_TRIGGER_EDGE; /* just pick = some */ >=20 > gi->gi_pol =3D pol; > gi->gi_trig =3D trig; >=20 > /* Edge triggered interrupts need an early EOI sent */ > if (gi->gi_pol =3D=3D INTR_TRIGGER_EDGE) > gi->gi_flags |=3D GI_FLAG_EARLY_EOI; > } =3D=3D=3D Mark Millard markmi at dsl-only.net On 2017-Feb-1, at 7:07 PM, Mark Millard <markmi at dsl-only.net> wrote: > I temporarily modified the Spurious-interrupt-detected notice to also = report > irq and sc->nirqs : >=20 >=20 > . . . > #define gic_c_read_4(_sc, _reg) \ > bus_space_read_4((_sc)->gic_c_bst, (_sc)->gic_c_bsh, (_reg)) >=20 > . . . > int > arm_gic_intr(void *arg) > { > struct arm_gic_softc *sc =3D arg; > struct gic_irqsrc *gi; > uint32_t irq_active_reg, irq; > struct trapframe *tf; >=20 > irq_active_reg =3D gic_c_read_4(sc, GICC_IAR); > irq =3D irq_active_reg & 0x3FF; >=20 > /* > . . . > */ >=20 > if (irq >=3D sc->nirqs) { > #ifdef GIC_DEBUG_SPURIOUS > device_printf(sc->gic_dev, > "Spurious interrupt %d detected of %d: last irq: %d = on CPU%d\n", > irq, sc->nirqs, > sc->last_irq[PCPU_GET(cpuid)], PCPU_GET(cpuid)); > #endif > return (FILTER_HANDLED); > } > . . . >=20 >=20 > The result was irq=3D=3D1023 and sc->nirqs=3D=3D224 in every message > that I've seen so far. 1023=3D=3D0x3FF . >=20 > Looking around I found in: >=20 > = http://www.cl.cam.ac.uk/research/srg/han/ACS-P35/zynq/arm_gic_architecture= _specification.pdf >=20 > the following on various reasons why 1023 would show up (quoting): >=20 >=20 >=20 > =E2=80=A2 A processor reads the GICC_IAR and obtains the = interrupt ID 1023, indicating a spurious interrupt. The processor can = return from its interrupt service routine without writing to its = GICC_EOIR. >=20 > The spurious interrupt ID indicates that the original interrupt is no = longer pending, typically because another target processor is handling = it. >=20 > . . . >=20 > The GIC architecture reserves interrupt ID numbers 1020-1023 for = special purposes. In a GICv1 implementation that does not implement the = GIC Security Extensions, the only one of these used is ID 1023. This = value is returned to a processor, in response to an interrupt = acknowledge, if there is no pending interrupt with sufficient priority = for it to be signaled to the processor. It is described as a response to = a spurious interrupt. >=20 > Note >=20 > A race condition can cause a spurious interrupt. For example, a = spurious interrupt can occur if a processor writes a 1 to a field in an = GICD_ICENABLERn that corresponds to a pending interrupt after the CPU = interface has signaled the interrupt to the processor and the processor = has recognized the interrupt, but before the processor has read from the = GICC_IAR. >=20 > . . . >=20 > =E2=80=A2 If a read of the GICC_IAR does not match the security = of the interrupt, the GICC_IAR read does not acknowledge any interrupt = and returns the value: >=20 > =E2=80=A2 1022 for a Secure read when the highest = priority interrupt is Non-secure >=20 > =E2=80=A2 1023 for a Non-secure read when the highest = priority interrupt is Secure. > . . . >=20 > A read of the GICC_IAR returns the interrupt ID of the highest = priority pending interrupt for the CPU interface. The read returns a = spurious interrupt ID of 1023 if any of the following apply: >=20 > =E2=80=A2 forwarding of interrupts by the Distributor to the CPU = interface is disabled >=20 > =E2=80=A2 signaling of interrupts by the CPU interface to the = connected processor is disabled >=20 > =E2=80=A2 no pending interrupt on the CPU interface has = sufficient priority for the interface to signal it to the processor. >=20 >=20 > =E2=80=A2 The following sequence of events is an example of when = the GIC returns an interrupt ID of 1023, and shows how reads of the = GICC_IAR can be timing critical: >=20 > 1. A peripheral asserts a level-sensitive interrupt. >=20 > 2. The interrupt has sufficient priority and therefore the GIC signals = it to a targeted processor. >=20 > 3. The peripheral deasserts the interrupt. Because there is no other = pending interrupt of sufficient priority, the GIC deasserts the = interrupt request to the processor. >=20 > 4. Before it has recognized the deassertion of the interrupt request = from stage 3, the targeted processor reads the GICC_IAR. Because there = is no interrupt with sufficient priority to signal to the processor, the = GIC returns the spurious ID value of 1023. >=20 >=20 > The determination of the returned interrupt ID is more complex if the = GIC supports interrupt grouping >=20 > . . . >=20 >=20 > Interrupt signaling of the required interrupt group by CPU interface = disabled >=20 >=20 >=20 > =3D=3D=3D > Mark Millard > markmi at dsl-only.net _______________________________________________ freebsd-arm@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-arm To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?A95CC1DC-36C4-4FC3-A8D4-BDBE6FCB136B>