From owner-freebsd-net@FreeBSD.ORG Wed Jul 16 17:34:35 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BC6461065674 for ; Wed, 16 Jul 2008 17:34:35 +0000 (UTC) (envelope-from davidch@broadcom.com) Received: from mms1.broadcom.com (mms1.broadcom.com [216.31.210.17]) by mx1.freebsd.org (Postfix) with ESMTP id 860378FC0C for ; Wed, 16 Jul 2008 17:34:35 +0000 (UTC) (envelope-from davidch@broadcom.com) Received: from [10.11.16.99] by mms1.broadcom.com with ESMTP (Broadcom SMTP Relay (Email Firewall v6.3.2)); Wed, 16 Jul 2008 10:34:25 -0700 X-Server-Uuid: 02CED230-5797-4B57-9875-D5D2FEE4708A Received: by mail-irva-10.broadcom.com (Postfix, from userid 47) id 113072B1; Wed, 16 Jul 2008 10:34:25 -0700 (PDT) Received: from mail-irva-8.broadcom.com (mail-irva-8 [10.11.18.52]) by mail-irva-10.broadcom.com (Postfix) with ESMTP id F25432B0 for ; Wed, 16 Jul 2008 10:34:24 -0700 (PDT) Received: from mail-irva-13.broadcom.com (mail-irva-13.broadcom.com [10.11.16.103]) by mail-irva-8.broadcom.com (MOS 3.7.5a-GA) with ESMTP id GZX93239; Wed, 16 Jul 2008 10:34:24 -0700 (PDT) Received: from NT-IRVA-0751.brcm.ad.broadcom.com (nt-irva-0751 [10.8.194.65]) by mail-irva-13.broadcom.com (Postfix) with ESMTP id C192574CFE for ; Wed, 16 Jul 2008 10:34:24 -0700 (PDT) Received: from IRVEXCHHUB01.corp.ad.broadcom.com ([10.9.200.131]) by NT-IRVA-0751.brcm.ad.broadcom.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 16 Jul 2008 10:34:24 -0700 Received: from IRVEXCHCCR01.corp.ad.broadcom.com ([10.252.49.30]) by IRVEXCHHUB01.corp.ad.broadcom.com ([10.9.200.131]) with mapi; Wed, 16 Jul 2008 10:34:24 -0700 From: "David Christensen" To: "freebsd-net@freebsd.org" Date: Wed, 16 Jul 2008 10:35:13 -0700 Thread-Topic: Enabling MSI-X on -CURRENT for New Network Driver Thread-Index: AcjnakrJ6d8VEBUpSmyKmnlDSl7HyQ== Message-ID: <5D267A3F22FD854F8F48B3D2B52381932677F1A0C3@IRVEXCHCCR01.corp.ad.broadcom.com> Accept-Language: en-US Content-Language: en-US acceptlanguage: en-US MIME-Version: 1.0 X-OriginalArrivalTime: 16 Jul 2008 17:34:24.0687 (UTC) FILETIME=[3088A3F0:01C8E76A] X-WSS-ID: 6460EEAB4E084755343-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Enabling MSI-X on -CURRENT for New Network Driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2008 17:34:35 -0000 I'm working on adding MSI-X support for a new network driver and having some difficulty in actually getting an interrupt. Does this look right? /* Select and configure the IRQ. */ sc->bxe_msix_count =3D pci_msix_count(dev); rid =3D 1; /* Try allocating MSI-X interrupts. */ if ((sc->bxe_cap_flags & BXE_MSIX_CAPABLE_FLAG) && (bxe_msi_enable >=3D 2) && (sc->bxe_msix_count > 0)) { int msix_needed =3D sc->bxe_msix_count; if (pci_alloc_msix(dev, &sc->bxe_msix_count) =3D=3D 0) { if (sc->bxe_msix_count =3D=3D msix_needed) { DBPRINT(sc, BXE_INFO_LOAD, "%s(): Using %d = MSI-X " "vector(s).\n", __FUNCTION__, sc->b= xe_msix_count); sc->bxe_flags |=3D BXE_USING_MSIX_FLAG; } else { pci_release_msi(dev); sc->bxe_flags &=3D ~BXE_USING_MSIX_FLAG; sc->bxe_msix_count =3D 0; } } } /* Try allocating MSI interrupts if we didn't get MSI-X. */ ... /* Try legacy interrupt. */ ... /* Allocate the interrupt and report any errors. */ sc->bxe_res_irq =3D bus_alloc_resource_any(dev, SYS_RES_IRQ, &rid, RF_ACTIVE); /* Report any IRQ allocation errors. */ if (sc->bxe_res_irq =3D=3D NULL) { BXE_PRINTF("%s(%d): PCI map interrupt failed!\n", __FILE__, __LINE__); rc =3D ENXIO; goto bxe_attach_fail; } sc->bxe_irq_rid =3D rid; sc->bxe_intr =3D bxe_intr; The allocation doesn't fail and I usually see an IRQ allocated to the driver using "vmstat -i" (though not always): =3D=3D=3D[root] /usr/src/sys/modules/bxe # vmstat -i interrupt total rate irq1: atkbd0 1 0 irq4: sio0 46432 6 irq6: fdc0 10 0 irq14: ata0 58 0 irq17: atapci1 42684 5 cpu0: timer 15331063 1999 irq256: em0 917 0 cpu3: timer 15330811 1999 cpu1: timer 15330808 1999 cpu2: timer 15330811 1999 cpu5: timer 15330811 1999 cpu6: timer 15330810 1999 cpu4: timer 15330806 1999 cpu7: timer 15330811 1999 irq258: bxe0 2 0 Total 122736835 16010 But my interrupt handler doesn't seem to be called. The goal is to get a single interrupt working first, multiple queue support comes next. Any ideas? Dave