From owner-freebsd-drivers@FreeBSD.ORG Thu Apr 21 11:51:34 2011 Return-Path: Delivered-To: freebsd-drivers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 031E81065672 for ; Thu, 21 Apr 2011 11:51:34 +0000 (UTC) (envelope-from bcketchum@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 856AA8FC0C for ; Thu, 21 Apr 2011 11:51:33 +0000 (UTC) Received: by bwz12 with SMTP id 12so1873017bwz.13 for ; Thu, 21 Apr 2011 04:51:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=Uj5/OmfaqXwq76iugxqexCNjWaVexlzChicpOXbeUQU=; b=Cic4RGy5dKSKLlOltzVZ86oUYKPvegL72WDBMEtprxz6XEZekAgeTMAGuEAmpK4udp vXhJHJXFL40AJVgGc84Zo48+md0aCybvWFz4uduUhn4ANZIzUxR7+efprCPmiEadVw7+ F2UgI8ym2WU1XG0xU4VW+VWsXwpdHx4RA/Rok= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=NfPoaAc4L+18/gUgOKu5A3+4M/pBDoEYRiyC3oCGWcrvL9RUlEkPm3vlnhr1HVgCeC h23bFN8aZhlXIVxhzqVYb9qqQJfR64wj1xhts8OABy27swJk3/XOq36Ru1al1YiDuVJl WzFznAczR4a5fDUW4hgRRddHm2TT8iEuyZuKo= MIME-Version: 1.0 Received: by 10.204.8.141 with SMTP id h13mr1105272bkh.64.1303386692272; Thu, 21 Apr 2011 04:51:32 -0700 (PDT) Received: by 10.204.36.77 with HTTP; Thu, 21 Apr 2011 04:51:32 -0700 (PDT) In-Reply-To: References: Date: Thu, 21 Apr 2011 06:51:32 -0500 Message-ID: From: Bret Ketchum To: freebsd-drivers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: MSI interrupts. X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Apr 2011 11:51:34 -0000 As an experiment I made align equal to count in apic_alloc_vectors to mimic the way MSIX vectors are assigned and all my issue went away.. On Tue, Apr 12, 2011 at 8:16 AM, Bret Ketchum wrote: > > I've a roll-your-own driver for FreeBSD 8.x that uses MSI interrupts > for PCI-E HBAs where one or more will be installed in a off-the-shelf amd64 > pizza box. The driver is using bus_setup_intr() and depending upon the slots > the HBAs are install I see log messages from apic_alloc_vectors(), for > example: > > Apr 12 06:44:15 mfsbsd kernel: xxxpci10: attempting to allocate 1 MSI > vectors (16 supported) > Apr 12 06:44:15 mfsbsd kernel: APIC: Couldn't find APIC vectors for 1 IRQs > Apr 12 06:44:15 mfsbsd kernel: ioapic1: routing intpin 13 (PCI IRQ 37) to > lapic 0 vector 59 > > Using vmstat -ia: > > interrupt total rate > irq37: xxxpci10 74 0 > > The problem appears to be that HBA interrupts are not being delivered > to the driver. If I swap cards around in slots I can eliminate the message > and: > > Apr 12 06:44:15 mfsbsd kernel: msi: routing MSI IRQ 266 to local APIC 0 > vector 80 > > And interrupts appear to be delivered properly. Before I dive in, can > anyone explain this behavior? > > Thanks in advance. > > Dr. >