From owner-svn-src-all@freebsd.org Fri Dec 18 05:44:03 2015 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A44FA4B24B; Fri, 18 Dec 2015 05:44:03 +0000 (UTC) (envelope-from adrian@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A880B12CF; Fri, 18 Dec 2015 05:44:02 +0000 (UTC) (envelope-from adrian@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id tBI5i1xY054218; Fri, 18 Dec 2015 05:44:01 GMT (envelope-from adrian@FreeBSD.org) Received: (from adrian@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id tBI5i0Zp054199; Fri, 18 Dec 2015 05:44:00 GMT (envelope-from adrian@FreeBSD.org) Message-Id: <201512180544.tBI5i0Zp054199@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: adrian set sender to adrian@FreeBSD.org using -f From: Adrian Chadd Date: Fri, 18 Dec 2015 05:44:00 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r292426 - in head/sys: arm/allwinner/a20 arm/altera/socfpga arm/amlogic/aml8726 arm/annapurna/alpine arm/arm arm/freescale/imx arm/include arm/qemu arm/rockchip arm/samsung/exynos arm/t... X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Dec 2015 05:44:03 -0000 Author: adrian Date: Fri Dec 18 05:43:59 2015 New Revision: 292426 URL: https://svnweb.freebsd.org/changeset/base/292426 Log: [intrng] Migrate the intrng code from sys/arm/arm to sys/kern/subr_intr.c. The ci20 port (by kan@) is going to reuse almost all of the intrng code since the SoC in question looks suspiciously like someone took an ARM SoC design and replaced the ARM core with a MIPS core. * migrate out the code; * rename ARM_ -> INTR_; * rename arm_ -> intr_; * move the interrupt flush routine from intr.c / intrng.c into arm/machdep_intr.c - removing the code duplication and removing the ARM specific bits from here. Thanks to the Star Wars: The Force Awakens premiere line for allowing me a couple hours of quiet time to finish the universe builds. Tested: * make universe TODO: * The structure definitions in subr_intr.c still includes machine/intr.h which requires one duplicates all of the intrng definitions in the platform code (which kan has done, and I think we don't have to.) Instead I should break out the generic things (function declarations, common intr structures, etc) into a separate header. * Kan has requested I make the PIC based IPI stuff optional. Added: head/sys/arm/arm/machdep_intr.c (contents, props changed) head/sys/kern/pic_if.m - copied, changed from r292394, head/sys/arm/arm/pic_if.m head/sys/kern/subr_intr.c - copied, changed from r292394, head/sys/arm/arm/intrng.c Deleted: head/sys/arm/arm/intrng.c head/sys/arm/arm/pic_if.m Modified: head/sys/arm/allwinner/a20/a20_mp.c head/sys/arm/altera/socfpga/socfpga_mp.c head/sys/arm/amlogic/aml8726/aml8726_mp.c head/sys/arm/annapurna/alpine/alpine_machdep_mp.c head/sys/arm/arm/exception.S head/sys/arm/arm/gic.c head/sys/arm/arm/intr.c head/sys/arm/arm/mp_machdep.c head/sys/arm/arm/nexus.c head/sys/arm/freescale/imx/imx6_mp.c head/sys/arm/freescale/imx/imx_gpio.c head/sys/arm/include/intr.h head/sys/arm/include/smp.h head/sys/arm/qemu/virt_mp.c head/sys/arm/rockchip/rk30xx_mp.c head/sys/arm/samsung/exynos/exynos5_mp.c head/sys/arm/ti/omap4/omap4_mp.c head/sys/arm/xilinx/zy7_mp.c head/sys/conf/files.arm Modified: head/sys/arm/allwinner/a20/a20_mp.c ============================================================================== --- head/sys/arm/allwinner/a20/a20_mp.c Fri Dec 18 05:39:25 2015 (r292425) +++ head/sys/arm/allwinner/a20/a20_mp.c Fri Dec 18 05:43:59 2015 (r292426) @@ -61,7 +61,7 @@ void platform_mp_init_secondary(void) { - arm_pic_init_secondary(); + intr_pic_init_secondary(); } void Modified: head/sys/arm/altera/socfpga/socfpga_mp.c ============================================================================== --- head/sys/arm/altera/socfpga/socfpga_mp.c Fri Dec 18 05:39:25 2015 (r292425) +++ head/sys/arm/altera/socfpga/socfpga_mp.c Fri Dec 18 05:43:59 2015 (r292426) @@ -87,7 +87,7 @@ void platform_mp_init_secondary(void) { - arm_pic_init_secondary(); + intr_pic_init_secondary(); } void Modified: head/sys/arm/amlogic/aml8726/aml8726_mp.c ============================================================================== --- head/sys/arm/amlogic/aml8726/aml8726_mp.c Fri Dec 18 05:39:25 2015 (r292425) +++ head/sys/arm/amlogic/aml8726/aml8726_mp.c Fri Dec 18 05:43:59 2015 (r292426) @@ -351,7 +351,7 @@ platform_mp_init_secondary(void) * each AP. */ - arm_pic_init_secondary(); + intr_pic_init_secondary(); } Modified: head/sys/arm/annapurna/alpine/alpine_machdep_mp.c ============================================================================== --- head/sys/arm/annapurna/alpine/alpine_machdep_mp.c Fri Dec 18 05:39:25 2015 (r292425) +++ head/sys/arm/annapurna/alpine/alpine_machdep_mp.c Fri Dec 18 05:43:59 2015 (r292426) @@ -122,7 +122,7 @@ void platform_mp_init_secondary(void) { - arm_pic_init_secondary(); + intr_pic_init_secondary(); } void Modified: head/sys/arm/arm/exception.S ============================================================================== --- head/sys/arm/arm/exception.S Fri Dec 18 05:39:25 2015 (r292425) +++ head/sys/arm/arm/exception.S Fri Dec 18 05:43:59 2015 (r292426) @@ -410,7 +410,7 @@ ASENTRY_NP(irq_entry) PUSHFRAMEINSVC /* mode stack, build trapframe there. */ adr lr, exception_exit /* Return from handler via standard */ mov r0, sp /* exception exit routine. Pass the */ - b _C_LABEL(arm_irq_handler)/* trapframe to the handler. */ + b _C_LABEL(intr_irq_handler)/* trapframe to the handler. */ END(irq_entry) /* Modified: head/sys/arm/arm/gic.c ============================================================================== --- head/sys/arm/arm/gic.c Fri Dec 18 05:39:25 2015 (r292425) +++ head/sys/arm/arm/gic.c Fri Dec 18 05:43:59 2015 (r292426) @@ -120,14 +120,14 @@ __FBSDID("$FreeBSD$"); #ifdef ARM_INTRNG static u_int gic_irq_cpu; static int arm_gic_intr(void *); -static int arm_gic_bind(device_t dev, struct arm_irqsrc *isrc); +static int arm_gic_bind(device_t dev, struct intr_irqsrc *isrc); #endif struct arm_gic_softc { device_t gic_dev; #ifdef ARM_INTRNG void * gic_intrhand; - struct arm_irqsrc ** gic_irqs; + struct intr_irqsrc ** gic_irqs; #endif struct resource * gic_res[3]; bus_space_tag_t gic_c_bst; @@ -216,7 +216,7 @@ static void arm_gic_init_secondary(device_t dev) { struct arm_gic_softc *sc = device_get_softc(dev); - struct arm_irqsrc *isrc; + struct intr_irqsrc *isrc; u_int irq; for (irq = 0; irq < sc->nirqs; irq += 4) @@ -250,7 +250,7 @@ arm_gic_init_secondary(device_t dev) isrc = sc->gic_irqs[irq]; if (isrc == NULL || isrc->isrc_handlers == 0) continue; - if (isrc->isrc_flags & ARM_ISRCF_BOUND) { + if (isrc->isrc_flags & INTR_ISRCF_BOUND) { if (CPU_ISSET(PCPU_GET(cpuid), &isrc->isrc_cpu)) gic_irq_unmask(sc, irq); } else { @@ -456,7 +456,7 @@ arm_gic_attach(device_t dev) * Now, when everything is initialized, it's right time to * register interrupt controller to interrupt framefork. */ - if (arm_pic_register(dev, xref) != 0) { + if (intr_pic_register(dev, xref) != 0) { device_printf(dev, "could not register PIC\n"); goto cleanup; } @@ -468,23 +468,23 @@ arm_gic_attach(device_t dev) */ pxref = ofw_bus_find_iparent(ofw_bus_get_node(dev)); if (pxref == 0 || xref == pxref) { - if (arm_pic_claim_root(dev, xref, arm_gic_intr, sc, + if (intr_pic_claim_root(dev, xref, arm_gic_intr, sc, GIC_LAST_SGI - GIC_FIRST_SGI + 1) != 0) { device_printf(dev, "could not set PIC as a root\n"); - arm_pic_unregister(dev, xref); + intr_pic_unregister(dev, xref); goto cleanup; } } else { if (sc->gic_res[2] == NULL) { device_printf(dev, "not root PIC must have defined interrupt\n"); - arm_pic_unregister(dev, xref); + intr_pic_unregister(dev, xref); goto cleanup; } if (bus_setup_intr(dev, sc->gic_res[2], INTR_TYPE_CLK, arm_gic_intr, NULL, sc, &sc->gic_intrhand)) { device_printf(dev, "could not setup irq handler\n"); - arm_pic_unregister(dev, xref); + intr_pic_unregister(dev, xref); goto cleanup; } } @@ -508,7 +508,7 @@ static int arm_gic_intr(void *arg) { struct arm_gic_softc *sc = arg; - struct arm_irqsrc *isrc; + struct intr_irqsrc *isrc; uint32_t irq_active_reg, irq; struct trapframe *tf; @@ -560,7 +560,7 @@ dispatch_irq: #ifdef SMP /* Call EOI for all IPI before dispatch. */ gic_c_write_4(sc, GICC_EOIR, irq_active_reg); - arm_ipi_dispatch(isrc, tf); + intr_ipi_dispatch(isrc, tf); goto next_irq; #else device_printf(sc->gic_dev, "SGI %u on UP system detected\n", @@ -576,7 +576,7 @@ dispatch_irq: if (isrc->isrc_trig == INTR_TRIGGER_EDGE) gic_c_write_4(sc, GICC_EOIR, irq_active_reg); - arm_irq_dispatch(isrc, tf); + intr_irq_dispatch(isrc, tf); next_irq: arm_irq_memory_barrier(irq); @@ -589,7 +589,7 @@ next_irq: } static int -gic_attach_isrc(struct arm_gic_softc *sc, struct arm_irqsrc *isrc, u_int irq) +gic_attach_isrc(struct arm_gic_softc *sc, struct intr_irqsrc *isrc, u_int irq) { const char *name; @@ -609,16 +609,16 @@ gic_attach_isrc(struct arm_gic_softc *sc name = device_get_nameunit(sc->gic_dev); if (irq <= GIC_LAST_SGI) - arm_irq_set_name(isrc, "%s,i%u", name, irq - GIC_FIRST_SGI); + intr_irq_set_name(isrc, "%s,i%u", name, irq - GIC_FIRST_SGI); else if (irq <= GIC_LAST_PPI) - arm_irq_set_name(isrc, "%s,p%u", name, irq - GIC_FIRST_PPI); + intr_irq_set_name(isrc, "%s,p%u", name, irq - GIC_FIRST_PPI); else - arm_irq_set_name(isrc, "%s,s%u", name, irq - GIC_FIRST_SPI); + intr_irq_set_name(isrc, "%s,s%u", name, irq - GIC_FIRST_SPI); return (0); } static int -gic_detach_isrc(struct arm_gic_softc *sc, struct arm_irqsrc *isrc, u_int irq) +gic_detach_isrc(struct arm_gic_softc *sc, struct intr_irqsrc *isrc, u_int irq) { mtx_lock_spin(&sc->mutex); @@ -630,7 +630,7 @@ gic_detach_isrc(struct arm_gic_softc *sc isrc->isrc_data = 0; mtx_unlock_spin(&sc->mutex); - arm_irq_set_name(isrc, ""); + intr_irq_set_name(isrc, ""); return (0); } @@ -696,15 +696,15 @@ gic_irq_from_nspc(struct arm_gic_softc * { switch (type) { - case ARM_IRQ_NSPC_PLAIN: + case INTR_IRQ_NSPC_PLAIN: *irqp = num; return (*irqp < sc->nirqs ? 0 : EINVAL); - case ARM_IRQ_NSPC_IRQ: + case INTR_IRQ_NSPC_IRQ: *irqp = num + GIC_FIRST_PPI; return (*irqp < sc->nirqs ? 0 : EINVAL); - case ARM_IRQ_NSPC_IPI: + case INTR_IRQ_NSPC_IPI: *irqp = num + GIC_FIRST_SGI; return (*irqp < GIC_LAST_SGI ? 0 : EINVAL); @@ -714,7 +714,7 @@ gic_irq_from_nspc(struct arm_gic_softc * } static int -gic_map_nspc(struct arm_gic_softc *sc, struct arm_irqsrc *isrc, u_int *irqp) +gic_map_nspc(struct arm_gic_softc *sc, struct intr_irqsrc *isrc, u_int *irqp) { int error; @@ -727,7 +727,7 @@ gic_map_nspc(struct arm_gic_softc *sc, s #ifdef FDT static int -gic_map_fdt(struct arm_gic_softc *sc, struct arm_irqsrc *isrc, u_int *irqp) +gic_map_fdt(struct arm_gic_softc *sc, struct intr_irqsrc *isrc, u_int *irqp) { u_int irq, tripol; enum intr_trigger trig; @@ -773,7 +773,7 @@ gic_map_fdt(struct arm_gic_softc *sc, st if (error != 0) return (error); - isrc->isrc_nspc_type = ARM_IRQ_NSPC_PLAIN; + isrc->isrc_nspc_type = INTR_IRQ_NSPC_PLAIN; isrc->isrc_nspc_num = irq; isrc->isrc_trig = trig; isrc->isrc_pol = pol; @@ -784,16 +784,16 @@ gic_map_fdt(struct arm_gic_softc *sc, st #endif static int -arm_gic_register(device_t dev, struct arm_irqsrc *isrc, boolean_t *is_percpu) +arm_gic_register(device_t dev, struct intr_irqsrc *isrc, boolean_t *is_percpu) { struct arm_gic_softc *sc = device_get_softc(dev); u_int irq; int error; - if (isrc->isrc_type == ARM_ISRCT_NAMESPACE) + if (isrc->isrc_type == INTR_ISRCT_NAMESPACE) error = gic_map_nspc(sc, isrc, &irq); #ifdef FDT - else if (isrc->isrc_type == ARM_ISRCT_FDT) + else if (isrc->isrc_type == INTR_ISRCT_FDT) error = gic_map_fdt(sc, isrc, &irq); #endif else @@ -805,7 +805,7 @@ arm_gic_register(device_t dev, struct ar } static void -arm_gic_enable_intr(device_t dev, struct arm_irqsrc *isrc) +arm_gic_enable_intr(device_t dev, struct intr_irqsrc *isrc) { struct arm_gic_softc *sc = device_get_softc(dev); u_int irq = isrc->isrc_data; @@ -820,7 +820,7 @@ arm_gic_enable_intr(device_t dev, struct * pic_enable_source() and pic_disable_source() should act on * per CPU basis only. Thus, it should be solved here somehow. */ - if (isrc->isrc_flags & ARM_ISRCF_PERCPU) + if (isrc->isrc_flags & INTR_ISRCF_PERCPU) CPU_SET(PCPU_GET(cpuid), &isrc->isrc_cpu); gic_config(sc, irq, isrc->isrc_trig, isrc->isrc_pol); @@ -828,7 +828,7 @@ arm_gic_enable_intr(device_t dev, struct } static void -arm_gic_enable_source(device_t dev, struct arm_irqsrc *isrc) +arm_gic_enable_source(device_t dev, struct intr_irqsrc *isrc) { struct arm_gic_softc *sc = device_get_softc(dev); u_int irq = isrc->isrc_data; @@ -838,7 +838,7 @@ arm_gic_enable_source(device_t dev, stru } static void -arm_gic_disable_source(device_t dev, struct arm_irqsrc *isrc) +arm_gic_disable_source(device_t dev, struct intr_irqsrc *isrc) { struct arm_gic_softc *sc = device_get_softc(dev); u_int irq = isrc->isrc_data; @@ -847,7 +847,7 @@ arm_gic_disable_source(device_t dev, str } static int -arm_gic_unregister(device_t dev, struct arm_irqsrc *isrc) +arm_gic_unregister(device_t dev, struct intr_irqsrc *isrc) { struct arm_gic_softc *sc = device_get_softc(dev); u_int irq = isrc->isrc_data; @@ -856,7 +856,7 @@ arm_gic_unregister(device_t dev, struct } static void -arm_gic_pre_ithread(device_t dev, struct arm_irqsrc *isrc) +arm_gic_pre_ithread(device_t dev, struct intr_irqsrc *isrc) { struct arm_gic_softc *sc = device_get_softc(dev); @@ -865,7 +865,7 @@ arm_gic_pre_ithread(device_t dev, struct } static void -arm_gic_post_ithread(device_t dev, struct arm_irqsrc *isrc) +arm_gic_post_ithread(device_t dev, struct intr_irqsrc *isrc) { arm_irq_memory_barrier(0); @@ -873,7 +873,7 @@ arm_gic_post_ithread(device_t dev, struc } static void -arm_gic_post_filter(device_t dev, struct arm_irqsrc *isrc) +arm_gic_post_filter(device_t dev, struct intr_irqsrc *isrc) { struct arm_gic_softc *sc = device_get_softc(dev); @@ -886,7 +886,7 @@ arm_gic_post_filter(device_t dev, struct } static int -arm_gic_bind(device_t dev, struct arm_irqsrc *isrc) +arm_gic_bind(device_t dev, struct intr_irqsrc *isrc) { struct arm_gic_softc *sc = device_get_softc(dev); uint32_t irq = isrc->isrc_data; @@ -895,7 +895,7 @@ arm_gic_bind(device_t dev, struct arm_ir return (EINVAL); if (CPU_EMPTY(&isrc->isrc_cpu)) { - gic_irq_cpu = arm_irq_next_cpu(gic_irq_cpu, &all_cpus); + gic_irq_cpu = intr_irq_next_cpu(gic_irq_cpu, &all_cpus); CPU_SETOF(gic_irq_cpu, &isrc->isrc_cpu); } return (gic_bind(sc, irq, &isrc->isrc_cpu)); @@ -903,7 +903,7 @@ arm_gic_bind(device_t dev, struct arm_ir #ifdef SMP static void -arm_gic_ipi_send(device_t dev, struct arm_irqsrc *isrc, cpuset_t cpus) +arm_gic_ipi_send(device_t dev, struct intr_irqsrc *isrc, cpuset_t cpus) { struct arm_gic_softc *sc = device_get_softc(dev); uint32_t irq, val = 0, i; @@ -1097,7 +1097,7 @@ arm_get_next_irq(int last_irq) #ifdef SMP void -arm_pic_init_secondary(void) +intr_pic_init_secondary(void) { arm_gic_init_secondary(gic_sc->gic_dev); Modified: head/sys/arm/arm/intr.c ============================================================================== --- head/sys/arm/arm/intr.c Fri Dec 18 05:39:25 2015 (r292425) +++ head/sys/arm/arm/intr.c Fri Dec 18 05:43:59 2015 (r292426) @@ -70,7 +70,7 @@ typedef void (*mask_fn)(void *); static struct intr_event *intr_events[NIRQ]; -void arm_irq_handler(struct trapframe *); +void intr_irq_handler(struct trapframe *); void (*arm_post_filter)(void *) = NULL; int (*arm_config_irq)(int irq, enum intr_trigger trig, @@ -104,7 +104,7 @@ SYSINIT(intr_init, SI_SUB_INTR, SI_ORDER #ifdef FDT int -arm_fdt_map_irq(phandle_t iparent, pcell_t *intr, int icells) +intr_fdt_map_irq(phandle_t iparent, pcell_t *intr, int icells) { fdt_pic_decode_t intr_decode; phandle_t intr_parent; @@ -179,7 +179,7 @@ dosoftints(void) } void -arm_irq_handler(struct trapframe *frame) +intr_irq_handler(struct trapframe *frame) { struct intr_event *event; int i; @@ -199,67 +199,3 @@ arm_irq_handler(struct trapframe *frame) pmc_hook(PCPU_GET(curthread), PMC_FN_USER_CALLCHAIN, frame); #endif } - -/* - * arm_irq_memory_barrier() - * - * Ensure all writes to device memory have reached devices before proceeding. - * - * This is intended to be called from the post-filter and post-thread routines - * of an interrupt controller implementation. A peripheral device driver should - * use bus_space_barrier() if it needs to ensure a write has reached the - * hardware for some reason other than clearing interrupt conditions. - * - * The need for this function arises from the ARM weak memory ordering model. - * Writes to locations mapped with the Device attribute bypass any caches, but - * are buffered. Multiple writes to the same device will be observed by that - * device in the order issued by the cpu. Writes to different devices may - * appear at those devices in a different order than issued by the cpu. That - * is, if the cpu writes to device A then device B, the write to device B could - * complete before the write to device A. - * - * Consider a typical device interrupt handler which services the interrupt and - * writes to a device status-acknowledge register to clear the interrupt before - * returning. That write is posted to the L2 controller which "immediately" - * places it in a store buffer and automatically drains that buffer. This can - * be less immediate than you'd think... There may be no free slots in the store - * buffers, so an existing buffer has to be drained first to make room. The - * target bus may be busy with other traffic (such as DMA for various devices), - * delaying the drain of the store buffer for some indeterminate time. While - * all this delay is happening, execution proceeds on the CPU, unwinding its way - * out of the interrupt call stack to the point where the interrupt driver code - * is ready to EOI and unmask the interrupt. The interrupt controller may be - * accessed via a faster bus than the hardware whose handler just ran; the write - * to unmask and EOI the interrupt may complete quickly while the device write - * to ack and clear the interrupt source is still lingering in a store buffer - * waiting for access to a slower bus. With the interrupt unmasked at the - * interrupt controller but still active at the device, as soon as interrupts - * are enabled on the core the device re-interrupts immediately: now you've got - * a spurious interrupt on your hands. - * - * The right way to fix this problem is for every device driver to use the - * proper bus_space_barrier() calls in its interrupt handler. For ARM a single - * barrier call at the end of the handler would work. This would have to be - * done to every driver in the system, not just arm-specific drivers. - * - * Another potential fix is to map all device memory as Strongly-Ordered rather - * than Device memory, which takes the store buffers out of the picture. This - * has a pretty big impact on overall system performance, because each strongly - * ordered memory access causes all L2 store buffers to be drained. - * - * A compromise solution is to have the interrupt controller implementation call - * this function to establish a barrier between writes to the interrupt-source - * device and writes to the interrupt controller device. - * - * This takes the interrupt number as an argument, and currently doesn't use it. - * The plan is that maybe some day there is a way to flag certain interrupts as - * "memory barrier safe" and we can avoid this overhead with them. - */ -void -arm_irq_memory_barrier(uintptr_t irq) -{ - - dsb(); - cpu_l2cache_drain_writebuf(); -} - Added: head/sys/arm/arm/machdep_intr.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/arm/arm/machdep_intr.c Fri Dec 18 05:43:59 2015 (r292426) @@ -0,0 +1,123 @@ +/* $NetBSD: intr.c,v 1.12 2003/07/15 00:24:41 lukem Exp $ */ + +/*- + * Copyright (c) 2004 Olivier Houchard. + * Copyright (c) 1994-1998 Mark Brinicombe. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. All advertising materials mentioning features or use of this software + * must display the following acknowledgement: + * This product includes software developed by Mark Brinicombe + * for the NetBSD Project. + * 4. The name of the company nor the name of the author may be used to + * endorse or promote products derived from this software without specific + * prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES + * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR + * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * Soft interrupt and other generic interrupt functions. + */ + +#include "opt_platform.h" + +#include +__FBSDID("$FreeBSD$"); + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +/* + * arm_irq_memory_barrier() + * + * Ensure all writes to device memory have reached devices before proceeding. + * + * This is intended to be called from the post-filter and post-thread routines + * of an interrupt controller implementation. A peripheral device driver should + * use bus_space_barrier() if it needs to ensure a write has reached the + * hardware for some reason other than clearing interrupt conditions. + * + * The need for this function arises from the ARM weak memory ordering model. + * Writes to locations mapped with the Device attribute bypass any caches, but + * are buffered. Multiple writes to the same device will be observed by that + * device in the order issued by the cpu. Writes to different devices may + * appear at those devices in a different order than issued by the cpu. That + * is, if the cpu writes to device A then device B, the write to device B could + * complete before the write to device A. + * + * Consider a typical device interrupt handler which services the interrupt and + * writes to a device status-acknowledge register to clear the interrupt before + * returning. That write is posted to the L2 controller which "immediately" + * places it in a store buffer and automatically drains that buffer. This can + * be less immediate than you'd think... There may be no free slots in the store + * buffers, so an existing buffer has to be drained first to make room. The + * target bus may be busy with other traffic (such as DMA for various devices), + * delaying the drain of the store buffer for some indeterminate time. While + * all this delay is happening, execution proceeds on the CPU, unwinding its way + * out of the interrupt call stack to the point where the interrupt driver code + * is ready to EOI and unmask the interrupt. The interrupt controller may be + * accessed via a faster bus than the hardware whose handler just ran; the write + * to unmask and EOI the interrupt may complete quickly while the device write + * to ack and clear the interrupt source is still lingering in a store buffer + * waiting for access to a slower bus. With the interrupt unmasked at the + * interrupt controller but still active at the device, as soon as interrupts + * are enabled on the core the device re-interrupts immediately: now you've got + * a spurious interrupt on your hands. + * + * The right way to fix this problem is for every device driver to use the + * proper bus_space_barrier() calls in its interrupt handler. For ARM a single + * barrier call at the end of the handler would work. This would have to be + * done to every driver in the system, not just arm-specific drivers. + * + * Another potential fix is to map all device memory as Strongly-Ordered rather + * than Device memory, which takes the store buffers out of the picture. This + * has a pretty big impact on overall system performance, because each strongly + * ordered memory access causes all L2 store buffers to be drained. + * + * A compromise solution is to have the interrupt controller implementation call + * this function to establish a barrier between writes to the interrupt-source + * device and writes to the interrupt controller device. + * + * This takes the interrupt number as an argument, and currently doesn't use it. + * The plan is that maybe some day there is a way to flag certain interrupts as + * "memory barrier safe" and we can avoid this overhead with them. + */ +void +arm_irq_memory_barrier(uintptr_t irq) +{ + + dsb(); + cpu_l2cache_drain_writebuf(); +} + Modified: head/sys/arm/arm/mp_machdep.c ============================================================================== --- head/sys/arm/arm/mp_machdep.c Fri Dec 18 05:39:25 2015 (r292425) +++ head/sys/arm/arm/mp_machdep.c Fri Dec 18 05:43:59 2015 (r292426) @@ -443,12 +443,12 @@ release_aps(void *dummy __unused) return; #ifdef ARM_INTRNG - arm_ipi_set_handler(IPI_RENDEZVOUS, "rendezvous", ipi_rendezvous, NULL, 0); - arm_ipi_set_handler(IPI_AST, "ast", ipi_ast, NULL, 0); - arm_ipi_set_handler(IPI_STOP, "stop", ipi_stop, NULL, 0); - arm_ipi_set_handler(IPI_PREEMPT, "preempt", ipi_preempt, NULL, 0); - arm_ipi_set_handler(IPI_HARDCLOCK, "hardclock", ipi_hardclock, NULL, 0); - arm_ipi_set_handler(IPI_TLB, "tlb", ipi_tlb, NULL, 0); + intr_ipi_set_handler(IPI_RENDEZVOUS, "rendezvous", ipi_rendezvous, NULL, 0); + intr_ipi_set_handler(IPI_AST, "ast", ipi_ast, NULL, 0); + intr_ipi_set_handler(IPI_STOP, "stop", ipi_stop, NULL, 0); + intr_ipi_set_handler(IPI_PREEMPT, "preempt", ipi_preempt, NULL, 0); + intr_ipi_set_handler(IPI_HARDCLOCK, "hardclock", ipi_hardclock, NULL, 0); + intr_ipi_set_handler(IPI_TLB, "tlb", ipi_tlb, NULL, 0); #else #ifdef IPI_IRQ_START Modified: head/sys/arm/arm/nexus.c ============================================================================== --- head/sys/arm/arm/nexus.c Fri Dec 18 05:39:25 2015 (r292425) +++ head/sys/arm/arm/nexus.c Fri Dec 18 05:43:59 2015 (r292426) @@ -267,7 +267,7 @@ nexus_config_intr(device_t dev, int irq, int ret = ENODEV; #ifdef ARM_INTRNG - ret = arm_irq_config(irq, trig, pol); + ret = intr_irq_config(irq, trig, pol); #else if (arm_config_irq) ret = (*arm_config_irq)(irq, trig, pol); @@ -286,7 +286,7 @@ nexus_setup_intr(device_t dev, device_t for (irq = rman_get_start(res); irq <= rman_get_end(res); irq++) { #ifdef ARM_INTRNG - arm_irq_add_handler(child, filt, intr, arg, irq, flags, + intr_irq_add_handler(child, filt, intr, arg, irq, flags, cookiep); #else arm_setup_irqhandler(device_get_nameunit(child), @@ -302,7 +302,7 @@ nexus_teardown_intr(device_t dev, device { #ifdef ARM_INTRNG - return (arm_irq_remove_handler(child, rman_get_start(r), ih)); + return (intr_irq_remove_handler(child, rman_get_start(r), ih)); #else return (arm_remove_irqhandler(rman_get_start(r), ih)); #endif @@ -314,7 +314,7 @@ nexus_describe_intr(device_t dev, device void *cookie, const char *descr) { - return (arm_irq_describe(rman_get_start(irq), cookie, descr)); + return (intr_irq_describe(rman_get_start(irq), cookie, descr)); } #ifdef SMP @@ -322,7 +322,7 @@ static int nexus_bind_intr(device_t dev, device_t child, struct resource *irq, int cpu) { - return (arm_irq_bind(rman_get_start(irq), cpu)); + return (intr_irq_bind(rman_get_start(irq), cpu)); } #endif #endif @@ -396,6 +396,6 @@ nexus_ofw_map_intr(device_t dev, device_ pcell_t *intr) { - return (arm_fdt_map_irq(iparent, intr, icells)); + return (intr_fdt_map_irq(iparent, intr, icells)); } #endif Modified: head/sys/arm/freescale/imx/imx6_mp.c ============================================================================== --- head/sys/arm/freescale/imx/imx6_mp.c Fri Dec 18 05:39:25 2015 (r292425) +++ head/sys/arm/freescale/imx/imx6_mp.c Fri Dec 18 05:43:59 2015 (r292426) @@ -69,7 +69,7 @@ void platform_mp_init_secondary(void) { - arm_pic_init_secondary(); + intr_pic_init_secondary(); } void Modified: head/sys/arm/freescale/imx/imx_gpio.c ============================================================================== --- head/sys/arm/freescale/imx/imx_gpio.c Fri Dec 18 05:39:25 2015 (r292425) +++ head/sys/arm/freescale/imx/imx_gpio.c Fri Dec 18 05:43:59 2015 (r292426) @@ -101,7 +101,7 @@ struct imx51_gpio_softc { bus_space_handle_t sc_ioh; int gpio_npins; struct gpio_pin gpio_pins[NGPIO]; - struct arm_irqsrc *gpio_pic_irqsrc[NGPIO]; + struct intr_irqsrc *gpio_pic_irqsrc[NGPIO]; }; static struct ofw_compat_data compat_data[] = { @@ -149,7 +149,7 @@ static int imx51_gpio_pin_toggle(device_ * this is teardown_intr */ static void -gpio_pic_disable_intr(device_t dev, struct arm_irqsrc *isrc) +gpio_pic_disable_intr(device_t dev, struct intr_irqsrc *isrc) { struct imx51_gpio_softc *sc; u_int irq; @@ -168,7 +168,7 @@ gpio_pic_disable_intr(device_t dev, stru * this is mask_intr */ static void -gpio_pic_disable_source(device_t dev, struct arm_irqsrc *isrc) +gpio_pic_disable_source(device_t dev, struct intr_irqsrc *isrc) { struct imx51_gpio_softc *sc; @@ -183,7 +183,7 @@ gpio_pic_disable_source(device_t dev, st * this is setup_intr */ static void -gpio_pic_enable_intr(device_t dev, struct arm_irqsrc *isrc) +gpio_pic_enable_intr(device_t dev, struct intr_irqsrc *isrc) { struct imx51_gpio_softc *sc; int icfg; @@ -226,7 +226,7 @@ gpio_pic_enable_intr(device_t dev, struc * this is unmask_intr */ static void -gpio_pic_enable_source(device_t dev, struct arm_irqsrc *isrc) +gpio_pic_enable_source(device_t dev, struct intr_irqsrc *isrc) { struct imx51_gpio_softc *sc; @@ -238,7 +238,7 @@ gpio_pic_enable_source(device_t dev, str } static void -gpio_pic_post_filter(device_t dev, struct arm_irqsrc *isrc) +gpio_pic_post_filter(device_t dev, struct intr_irqsrc *isrc) { struct imx51_gpio_softc *sc; @@ -250,7 +250,7 @@ gpio_pic_post_filter(device_t dev, struc } static void -gpio_pic_post_ithread(device_t dev, struct arm_irqsrc *isrc) +gpio_pic_post_ithread(device_t dev, struct intr_irqsrc *isrc) { arm_irq_memory_barrier(0); @@ -258,7 +258,7 @@ gpio_pic_post_ithread(device_t dev, stru } static void -gpio_pic_pre_ithread(device_t dev, struct arm_irqsrc *isrc) +gpio_pic_pre_ithread(device_t dev, struct intr_irqsrc *isrc) { gpio_pic_disable_source(dev, isrc); @@ -268,7 +268,7 @@ gpio_pic_pre_ithread(device_t dev, struc * intrng calls this to make a new isrc known to us. */ static int -gpio_pic_register(device_t dev, struct arm_irqsrc *isrc, boolean_t *is_percpu) +gpio_pic_register(device_t dev, struct intr_irqsrc *isrc, boolean_t *is_percpu) { struct imx51_gpio_softc *sc; u_int irq, tripol; @@ -320,7 +320,7 @@ gpio_pic_register(device_t dev, struct a tripol); return (ENOTSUP); } - isrc->isrc_nspc_type = ARM_IRQ_NSPC_PLAIN; + isrc->isrc_nspc_type = INTR_IRQ_NSPC_PLAIN; isrc->isrc_nspc_num = irq; /* @@ -337,12 +337,12 @@ gpio_pic_register(device_t dev, struct a isrc->isrc_data = irq; mtx_unlock_spin(&sc->sc_mtx); - arm_irq_set_name(isrc, "%s,%u", device_get_nameunit(sc->dev), irq); + intr_irq_set_name(isrc, "%s,%u", device_get_nameunit(sc->dev), irq); return (0); } static int -gpio_pic_unregister(device_t dev, struct arm_irqsrc *isrc) +gpio_pic_unregister(device_t dev, struct intr_irqsrc *isrc) { struct imx51_gpio_softc *sc; u_int irq; @@ -359,7 +359,7 @@ gpio_pic_unregister(device_t dev, struct isrc->isrc_data = 0; mtx_unlock_spin(&sc->sc_mtx); - arm_irq_set_name(isrc, ""); + intr_irq_set_name(isrc, ""); return (0); } @@ -378,7 +378,7 @@ gpio_pic_filter(void *arg) if ((interrupts & 0x1) == 0) continue; if (sc->gpio_pic_irqsrc[i]) - arm_irq_dispatch(sc->gpio_pic_irqsrc[i], curthread->td_intr_frame); + intr_irq_dispatch(sc->gpio_pic_irqsrc[i], curthread->td_intr_frame); else device_printf(sc->dev, "spurious interrupt %d\n", i); } @@ -656,7 +656,7 @@ imx51_gpio_attach(device_t dev) } #ifdef ARM_INTRNG - arm_pic_register(dev, OF_xref_from_node(ofw_bus_get_node(dev))); + intr_pic_register(dev, OF_xref_from_node(ofw_bus_get_node(dev))); #endif sc->sc_busdev = gpiobus_attach_bus(dev); Modified: head/sys/arm/include/intr.h ============================================================================== --- head/sys/arm/include/intr.h Fri Dec 18 05:39:25 2015 (r292425) +++ head/sys/arm/include/intr.h Fri Dec 18 05:43:59 2015 (r292426) @@ -51,33 +51,33 @@ #ifdef notyet #define INTR_SOLO INTR_MD1 -typedef int arm_irq_filter_t(void *arg, struct trapframe *tf); +typedef int intr_irq_filter_t(void *arg, struct trapframe *tf); #else -typedef int arm_irq_filter_t(void *arg); +typedef int intr_irq_filter_t(void *arg); #endif -#define ARM_ISRC_NAMELEN (MAXCOMLEN + 1) +#define INTR_ISRC_NAMELEN (MAXCOMLEN + 1) -typedef void arm_ipi_filter_t(void *arg); +typedef void intr_ipi_filter_t(void *arg); -enum arm_isrc_type { - ARM_ISRCT_NAMESPACE, - ARM_ISRCT_FDT +enum intr_isrc_type { + INTR_ISRCT_NAMESPACE, + INTR_ISRCT_FDT }; -#define ARM_ISRCF_REGISTERED 0x01 /* registered in a controller */ -#define ARM_ISRCF_PERCPU 0x02 /* per CPU interrupt */ -#define ARM_ISRCF_BOUND 0x04 /* bound to a CPU */ +#define INTR_ISRCF_REGISTERED 0x01 /* registered in a controller */ +#define INTR_ISRCF_PERCPU 0x02 /* per CPU interrupt */ +#define INTR_ISRCF_BOUND 0x04 /* bound to a CPU */ /* Interrupt source definition. */ -struct arm_irqsrc { +struct intr_irqsrc { device_t isrc_dev; /* where isrc is mapped */ intptr_t isrc_xref; /* device reference key */ uintptr_t isrc_data; /* device data for isrc */ u_int isrc_irq; /* unique identificator */ - enum arm_isrc_type isrc_type; /* how is isrc decribed */ + enum intr_isrc_type isrc_type; /* how is isrc decribed */ u_int isrc_flags; - char isrc_name[ARM_ISRC_NAMELEN]; + char isrc_name[INTR_ISRC_NAMELEN]; uint16_t isrc_nspc_type; uint16_t isrc_nspc_num; enum intr_trigger isrc_trig; @@ -87,8 +87,8 @@ struct arm_irqsrc { u_long * isrc_count; u_int isrc_handlers; struct intr_event * isrc_event; - arm_irq_filter_t * isrc_filter; - arm_ipi_filter_t * isrc_ipifilter; + intr_irq_filter_t * isrc_filter; + intr_ipi_filter_t * isrc_ipifilter; void * isrc_arg; #ifdef FDT u_int isrc_ncells; @@ -96,45 +96,45 @@ struct arm_irqsrc { #endif }; -void arm_irq_set_name(struct arm_irqsrc *isrc, const char *fmt, ...) +void intr_irq_set_name(struct intr_irqsrc *isrc, const char *fmt, ...) __printflike(2, 3); -void arm_irq_dispatch(struct arm_irqsrc *isrc, struct trapframe *tf); +void intr_irq_dispatch(struct intr_irqsrc *isrc, struct trapframe *tf); -#define ARM_IRQ_NSPC_NONE 0 -#define ARM_IRQ_NSPC_PLAIN 1 -#define ARM_IRQ_NSPC_IRQ 2 -#define ARM_IRQ_NSPC_IPI 3 +#define INTR_IRQ_NSPC_NONE 0 +#define INTR_IRQ_NSPC_PLAIN 1 +#define INTR_IRQ_NSPC_IRQ 2 +#define INTR_IRQ_NSPC_IPI 3 -u_int arm_namespace_map_irq(device_t dev, uint16_t type, uint16_t num); +u_int intr_namespace_map_irq(device_t dev, uint16_t type, uint16_t num); #ifdef FDT -u_int arm_fdt_map_irq(phandle_t, pcell_t *, u_int); +u_int intr_fdt_map_irq(phandle_t, pcell_t *, u_int); #endif -int arm_pic_register(device_t dev, intptr_t xref); -int arm_pic_unregister(device_t dev, intptr_t xref); -int arm_pic_claim_root(device_t dev, intptr_t xref, arm_irq_filter_t *filter, +int intr_pic_register(device_t dev, intptr_t xref); +int intr_pic_unregister(device_t dev, intptr_t xref); +int intr_pic_claim_root(device_t dev, intptr_t xref, intr_irq_filter_t *filter, void *arg, u_int ipicount); -int arm_irq_add_handler(device_t dev, driver_filter_t, driver_intr_t, void *, +int intr_irq_add_handler(device_t dev, driver_filter_t, driver_intr_t, void *, u_int, int, void **); -int arm_irq_remove_handler(device_t dev, u_int, void *); -int arm_irq_config(u_int, enum intr_trigger, enum intr_polarity); -int arm_irq_describe(u_int, void *, const char *); +int intr_irq_remove_handler(device_t dev, u_int, void *); +int intr_irq_config(u_int, enum intr_trigger, enum intr_polarity); +int intr_irq_describe(u_int, void *, const char *); -u_int arm_irq_next_cpu(u_int current_cpu, cpuset_t *cpumask); +u_int intr_irq_next_cpu(u_int current_cpu, cpuset_t *cpumask); #ifdef SMP -int arm_irq_bind(u_int, int); +int intr_irq_bind(u_int, int); -void arm_ipi_dispatch(struct arm_irqsrc *isrc, struct trapframe *tf); +void intr_ipi_dispatch(struct intr_irqsrc *isrc, struct trapframe *tf); #define AISHF_NOALLOC 0x0001 -int arm_ipi_set_handler(u_int ipi, const char *name, arm_ipi_filter_t *filter, +int intr_ipi_set_handler(u_int ipi, const char *name, intr_ipi_filter_t *filter, void *arg, u_int flags); -void arm_pic_init_secondary(void); +void intr_pic_init_secondary(void); #endif #else /* ARM_INTRNG */ @@ -178,11 +178,11 @@ extern void (*arm_post_filter)(void *); extern int (*arm_config_irq)(int irq, enum intr_trigger trig, enum intr_polarity pol); -void arm_pic_init_secondary(void); +void intr_pic_init_secondary(void); #ifdef FDT int gic_decode_fdt(phandle_t, pcell_t *, int *, int *, int *); -int arm_fdt_map_irq(phandle_t, pcell_t *, int); +int intr_fdt_map_irq(phandle_t, pcell_t *, int); #endif #endif /* ARM_INTRNG */ Modified: head/sys/arm/include/smp.h ============================================================================== --- head/sys/arm/include/smp.h Fri Dec 18 05:39:25 2015 (r292425) +++ head/sys/arm/include/smp.h Fri Dec 18 05:43:59 2015 (r292426) @@ -16,7 +16,7 @@ enum { IPI_HARDCLOCK, IPI_TLB, IPI_CACHE, - ARM_IPI_COUNT + INTR_IPI_COUNT }; #else #define IPI_AST 0 Modified: head/sys/arm/qemu/virt_mp.c ============================================================================== --- head/sys/arm/qemu/virt_mp.c Fri Dec 18 05:39:25 2015 (r292425) +++ head/sys/arm/qemu/virt_mp.c Fri Dec 18 05:43:59 2015 (r292426) @@ -102,7 +102,7 @@ void platform_mp_init_secondary(void) { - arm_pic_init_secondary(); + intr_pic_init_secondary(); } void Modified: head/sys/arm/rockchip/rk30xx_mp.c ============================================================================== --- head/sys/arm/rockchip/rk30xx_mp.c Fri Dec 18 05:39:25 2015 (r292425) +++ head/sys/arm/rockchip/rk30xx_mp.c Fri Dec 18 05:43:59 2015 (r292426) @@ -82,7 +82,7 @@ void platform_mp_init_secondary(void) { - arm_pic_init_secondary(); + intr_pic_init_secondary(); } void Modified: head/sys/arm/samsung/exynos/exynos5_mp.c ============================================================================== --- head/sys/arm/samsung/exynos/exynos5_mp.c Fri Dec 18 05:39:25 2015 (r292425) +++ head/sys/arm/samsung/exynos/exynos5_mp.c Fri Dec 18 05:43:59 2015 (r292426) @@ -73,7 +73,7 @@ void platform_mp_init_secondary(void) { - arm_pic_init_secondary(); + intr_pic_init_secondary(); } void Modified: head/sys/arm/ti/omap4/omap4_mp.c ============================================================================== --- head/sys/arm/ti/omap4/omap4_mp.c Fri Dec 18 05:39:25 2015 (r292425) +++ head/sys/arm/ti/omap4/omap4_mp.c Fri Dec 18 05:43:59 2015 (r292426) @@ -44,7 +44,7 @@ __FBSDID("$FreeBSD$"); *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***