From owner-freebsd-arm@freebsd.org Thu Feb 2 08:37:59 2017 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 261BDCCCE33 for ; Thu, 2 Feb 2017 08:37:59 +0000 (UTC) (envelope-from markmigm@gmail.com) Received: from mail-pf0-x230.google.com (mail-pf0-x230.google.com [IPv6:2607:f8b0:400e:c00::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DA0CA1BE1 for ; Thu, 2 Feb 2017 08:37:58 +0000 (UTC) (envelope-from markmigm@gmail.com) Received: by mail-pf0-x230.google.com with SMTP id 189so3733515pfu.3 for ; Thu, 02 Feb 2017 00:37:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:content-transfer-encoding:mime-version:subject:date:references :to:in-reply-to:message-id; bh=sTFCr8uxPOlnouhJRRxnmSKGUd0nm6UOvCQsEEilJTk=; b=FTBICr16EzgiDZRmG/OEixk2Hzcasx1ekFZUqSfFV9VWNd3FfnkHNxqjB0jX1X9TeM A+NZEqd8ztAPFg130J9pFP0dpHONqew6RV5aWVCYHo5gydi3lmvdeZGCRPFztMFiQxox x1sZgsIkeNpMrzL8INs4+Gw42nu/FoUUZ4IrTQl5w7BxElIJ3uaLHUBKRBIsNMlg0ZT5 qiDtrPT6hwIemiPFvfr/D4Qehcuxgnf5ibVrIpc7ZDUlSTCZqAWrRf88/ia8w3xWbiv6 1XGmqx5aFUQM7F3TDrPMahohuUVR6jo+BU45NxnoZHlDywCUwTjmtauCZ4vZMkG4w9jI zRcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:content-transfer-encoding:mime-version :subject:date:references:to:in-reply-to:message-id; bh=sTFCr8uxPOlnouhJRRxnmSKGUd0nm6UOvCQsEEilJTk=; b=LH0DPGyahsJ9ZNdz8cw7GFdIHv365hvWVRr95PPCDxzpz65VKuTrGm4ruxVL6S+IvD 3uqcht0P1lRGnU+qfRxurYQV9X9EU6r+muc8HUXc9zu28qXlhF1wY3NKy/QX1QOtgG+t W7qQAymXZ3gVlxJ4LEXofMBO/5rnQHUwMTahJGvUVEu34zKthSnNu+FmIZhOZewjnVc9 GiX6PRWYGJ8uBvu9sDFEiF+MkCAjRzjeBuqYCoxBiLx9oNY+CsbM6uSeNjox0QhjYvQe H5JvevzlFF7GUC7CYGaP0tT+I2peH519timd+CObga/AcX5bCL9PteZfduWEMcwHk0Fe NpBQ== X-Gm-Message-State: AIkVDXJpEPJKjsHdvQpWRAlZKYVeCEosQvl5JYbaGmEjKfvlQpyXRgCZmRRQVg2Nmem2/Q== X-Received: by 10.98.192.72 with SMTP id x69mr9106511pff.129.1486024678338; Thu, 02 Feb 2017 00:37:58 -0800 (PST) Received: from ?IPv6:2601:1c0:ce01:a5b7:6505:c813:e6a8:9eda? ([2601:1c0:ce01:a5b7:6505:c813:e6a8:9eda]) by smtp.gmail.com with ESMTPSA id t87sm55967688pfe.59.2017.02.02.00.37.57 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 02 Feb 2017 00:37:57 -0800 (PST) From: Mark Millard Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\)) Subject: Re: Arm64 stack issues (was Re: FreeBSD status for/on ODroid-C2?) Date: Thu, 2 Feb 2017 00:37:56 -0800 References: <20170124191357.0ec0abfd@zapp> <20170128010138.iublazyrhhqycn37@mutt-hardenedbsd> <20170128010223.tjivldnh7pyenbg6@mutt-hardenedbsd> <009857E3-35BB-4DE4-B3BB-5EC5DDBB5B06@dsl-only.net> <890B7D8A-27FF-41AC-8291-1858393EC7B1@gmail.com> <54642E5C-D5D6-45B7-BB74-2407CFB351C2@dsl-only.net> To: Tom Vijlbrief , freebsd-arm In-Reply-To: Message-Id: X-Mailer: Apple Mail (2.3259) X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Feb 2017 08:37:59 -0000 I'm out of my element here but I will note one difference between what I read in the likes of: = http://infocenter.arm.com/help/topic/com.arm.doc.den0024a/DEN0024A_v8_arch= itecture_PG.pdf and what I see in /usr/src/sys/arm/arm/gic.c 's arm_gic_intr . The written description says (quoting 10.6 The Generic Interrupt Controller and 10.6.3 Interrupt Handling): > Interrupts can either be edge-triggered (considered to be asserted = when the GIC detects a rising edge on the relevant input, and to remain = asserted until cleared) or level-sensitive (considered to be asserted = only when the relevant input to the GIC is HIGH). >=20 > . . . >=20 > The priority and list of cores to which an interrupt can be delivered = to are all configured in the Distributor. An interrupt asserted to the = Distributor by a peripheral is in the Pending state (or Active and = Pending if it was already Active). The Distributor determines the = highest priority pending interrupt that can be delivered to a core and = forwards that to the CPU interface of the core. At the CPU interface, = the interrupt is in turn signaled to the core, at which point the core = takes the FIQ or IRQ exception. >=20 > The core executes the exception handler in response. The handler must = query the interrupt ID from a CPU interface register and begin servicing = the interrupt source. When finished, the handler must write to a CPU = interface register to report the end of processing. >=20 > =E2=80=A2 For a given interrupt the typical sequence is: >=20 > =E2=80=A2 Inactive -> Pending > When the interrupt is asserted by the peripheral. >=20 > =E2=80=A2 Pending -> Active > When the handler acknowledges the interrupt. >=20 > =E2=80=A2 Active -> Inactive > When the handle[r] has finished dealing with the interrupt. >=20 > . . . >=20 > The top-level interrupt handler reads the Interrupt Acknowledge = Register from the CPU Interface block to obtain the interrupt ID. >=20 > As well as returning the interrupt ID, the read causes the interrupt = to be marked as active in the Distributor. Once the interrupt ID is = known (identifying the interrupt source), the top-level handler can now = dispatch a device-specific handler to service the interrupt. >=20 > When the device-specific handler finishes execution, the top-level = handler writes the same interrupt ID to the End of Interrupt (EoI) = register in the CPU Interface block, indicating the end of interrupt = processing. So that wording indicates that the write to GICC_EOIR should be after the dispatched activity (after "servicing"), not before. I did not find anything indicating that edge-triggered vs. level triggered would be different for this, for example. (But being unfamiliar I could have missed something.) In two cases below the code has the write to the GICC_EOIR before the dispatch (so before the servicing activity), possibly allowing another interrupt during or even before the dispatched activity (say if the state for the irq is active-and-pending at the time of the GICC_EOIR write or if there is a lower priority interrupt pending at that time): > if (irq <=3D GIC_LAST_SGI) { > #ifdef SMP > /* Call EOI for all IPI before dispatch. */ > gic_c_write_4(sc, GICC_EOIR, irq_active_reg); > intr_ipi_dispatch(sgi_to_ipi[gi->gi_irq], tf); > goto next_irq; > #else > . . . > #endif > } >=20 > . . . > if ((gi->gi_flags & GI_FLAG_EARLY_EOI) =3D=3D = GI_FLAG_EARLY_EOI) > gic_c_write_4(sc, GICC_EOIR, irq_active_reg); >=20 > if (intr_isrc_dispatch(&gi->gi_isrc, tf) !=3D 0) { > gic_irq_mask(sc, irq); > if ((gi->gi_flags & GI_FLAG_EARLY_EOI) !=3D = GI_FLAG_EARLY_EOI) > gic_c_write_4(sc, GICC_EOIR, irq_active_reg); > device_printf(sc->gic_dev, "Stray irq %u disabled\n", = irq); > } >=20 > next_irq: . . . Note: GI_FLAG_EARLY_EOI was set for edge triggered: > /* For MSI/MSI-X we should have already configured these */ > if ((gi->gi_flags & GI_FLAG_MSI) =3D=3D 0) { > if (pol =3D=3D INTR_POLARITY_CONFORM) > pol =3D INTR_POLARITY_LOW; /* just pick = some */ > if (trig =3D=3D INTR_TRIGGER_CONFORM) > trig =3D INTR_TRIGGER_EDGE; /* just pick = some */ >=20 > gi->gi_pol =3D pol; > gi->gi_trig =3D trig; >=20 > /* Edge triggered interrupts need an early EOI sent */ > if (gi->gi_pol =3D=3D INTR_TRIGGER_EDGE) > gi->gi_flags |=3D GI_FLAG_EARLY_EOI; > } =3D=3D=3D Mark Millard markmi at dsl-only.net On 2017-Feb-1, at 7:07 PM, Mark Millard wrote: > I temporarily modified the Spurious-interrupt-detected notice to also = report > irq and sc->nirqs : >=20 >=20 > . . . > #define gic_c_read_4(_sc, _reg) \ > bus_space_read_4((_sc)->gic_c_bst, (_sc)->gic_c_bsh, (_reg)) >=20 > . . . > int > arm_gic_intr(void *arg) > { > struct arm_gic_softc *sc =3D arg; > struct gic_irqsrc *gi; > uint32_t irq_active_reg, irq; > struct trapframe *tf; >=20 > irq_active_reg =3D gic_c_read_4(sc, GICC_IAR); > irq =3D irq_active_reg & 0x3FF; >=20 > /* > . . . > */ >=20 > if (irq >=3D sc->nirqs) { > #ifdef GIC_DEBUG_SPURIOUS > device_printf(sc->gic_dev, > "Spurious interrupt %d detected of %d: last irq: %d = on CPU%d\n", > irq, sc->nirqs, > sc->last_irq[PCPU_GET(cpuid)], PCPU_GET(cpuid)); > #endif > return (FILTER_HANDLED); > } > . . . >=20 >=20 > The result was irq=3D=3D1023 and sc->nirqs=3D=3D224 in every message > that I've seen so far. 1023=3D=3D0x3FF . >=20 > Looking around I found in: >=20 > = http://www.cl.cam.ac.uk/research/srg/han/ACS-P35/zynq/arm_gic_architecture= _specification.pdf >=20 > the following on various reasons why 1023 would show up (quoting): >=20 >=20 >=20 > =E2=80=A2 A processor reads the GICC_IAR and obtains the = interrupt ID 1023, indicating a spurious interrupt. The processor can = return from its interrupt service routine without writing to its = GICC_EOIR. >=20 > The spurious interrupt ID indicates that the original interrupt is no = longer pending, typically because another target processor is handling = it. >=20 > . . . >=20 > The GIC architecture reserves interrupt ID numbers 1020-1023 for = special purposes. In a GICv1 implementation that does not implement the = GIC Security Extensions, the only one of these used is ID 1023. This = value is returned to a processor, in response to an interrupt = acknowledge, if there is no pending interrupt with sufficient priority = for it to be signaled to the processor. It is described as a response to = a spurious interrupt. >=20 > Note >=20 > A race condition can cause a spurious interrupt. For example, a = spurious interrupt can occur if a processor writes a 1 to a field in an = GICD_ICENABLERn that corresponds to a pending interrupt after the CPU = interface has signaled the interrupt to the processor and the processor = has recognized the interrupt, but before the processor has read from the = GICC_IAR. >=20 > . . . >=20 > =E2=80=A2 If a read of the GICC_IAR does not match the security = of the interrupt, the GICC_IAR read does not acknowledge any interrupt = and returns the value: >=20 > =E2=80=A2 1022 for a Secure read when the highest = priority interrupt is Non-secure >=20 > =E2=80=A2 1023 for a Non-secure read when the highest = priority interrupt is Secure. > . . . >=20 > A read of the GICC_IAR returns the interrupt ID of the highest = priority pending interrupt for the CPU interface. The read returns a = spurious interrupt ID of 1023 if any of the following apply: >=20 > =E2=80=A2 forwarding of interrupts by the Distributor to the CPU = interface is disabled >=20 > =E2=80=A2 signaling of interrupts by the CPU interface to the = connected processor is disabled >=20 > =E2=80=A2 no pending interrupt on the CPU interface has = sufficient priority for the interface to signal it to the processor. >=20 >=20 > =E2=80=A2 The following sequence of events is an example of when = the GIC returns an interrupt ID of 1023, and shows how reads of the = GICC_IAR can be timing critical: >=20 > 1. A peripheral asserts a level-sensitive interrupt. >=20 > 2. The interrupt has sufficient priority and therefore the GIC signals = it to a targeted processor. >=20 > 3. The peripheral deasserts the interrupt. Because there is no other = pending interrupt of sufficient priority, the GIC deasserts the = interrupt request to the processor. >=20 > 4. Before it has recognized the deassertion of the interrupt request = from stage 3, the targeted processor reads the GICC_IAR. Because there = is no interrupt with sufficient priority to signal to the processor, the = GIC returns the spurious ID value of 1023. >=20 >=20 > The determination of the returned interrupt ID is more complex if the = GIC supports interrupt grouping >=20 > . . . >=20 >=20 > Interrupt signaling of the required interrupt group by CPU interface = disabled >=20 >=20 >=20 > =3D=3D=3D > Mark Millard > markmi at dsl-only.net _______________________________________________ freebsd-arm@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-arm To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org"