From owner-freebsd-drivers@FreeBSD.ORG Tue Apr 12 13:43:45 2011 Return-Path: Delivered-To: freebsd-drivers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2BB1710656D6 for ; Tue, 12 Apr 2011 13:43:45 +0000 (UTC) (envelope-from bcketchum@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id B5EEE8FC16 for ; Tue, 12 Apr 2011 13:43:44 +0000 (UTC) Received: by bwz12 with SMTP id 12so6943176bwz.13 for ; Tue, 12 Apr 2011 06:43:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:date:message-id:subject:from:to :content-type; bh=DHJsltmkdNXI5PS5B+V3wGGOWcELoNEA780zBlN+aSo=; b=iR1Au+npOFejYS2hY16jE7Qx0buOVSd1e8bLmD4qd2Kr/MIOOoKVu9ELiFRzASu8X3 kQiuTUXV/KE0+re6Qe8bKjHEGjnHqkePyx1jZYtJxYy5ox7sE+fyD1z6XPDxWOhK6ZjT oDjVwbOBOY3q7YXuoJwhdl0D8d6tlVhgzVjuw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=q3rAJg21z4MEOs2s/NYQQQjK9Na3dVdp0Um+uHB/+pRgW6GwGnNYAXJgymha6s9NHI +0Udx8jN8vcBBGAXjROkXmaXsaIgfQkqlMIC5cjmPHm1u+dZmC4BAgXEmRcsABtwy3Or Z/RkgUvnPtasNaTPYqvdy5Ak4OrytWt9QKOJM= MIME-Version: 1.0 Received: by 10.204.84.203 with SMTP id k11mr255501bkl.10.1302614176744; Tue, 12 Apr 2011 06:16:16 -0700 (PDT) Received: by 10.204.36.77 with HTTP; Tue, 12 Apr 2011 06:16:16 -0700 (PDT) Date: Tue, 12 Apr 2011 08:16:16 -0500 Message-ID: From: Bret Ketchum To: freebsd-drivers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: MSI interrupts. X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Apr 2011 13:43:45 -0000 I've a roll-your-own driver for FreeBSD 8.x that uses MSI interrupts for PCI-E HBAs where one or more will be installed in a off-the-shelf amd64 pizza box. The driver is using bus_setup_intr() and depending upon the slots the HBAs are install I see log messages from apic_alloc_vectors(), for example: Apr 12 06:44:15 mfsbsd kernel: xxxpci10: attempting to allocate 1 MSI vectors (16 supported) Apr 12 06:44:15 mfsbsd kernel: APIC: Couldn't find APIC vectors for 1 IRQs Apr 12 06:44:15 mfsbsd kernel: ioapic1: routing intpin 13 (PCI IRQ 37) to lapic 0 vector 59 Using vmstat -ia: interrupt total rate irq37: xxxpci10 74 0 The problem appears to be that HBA interrupts are not being delivered to the driver. If I swap cards around in slots I can eliminate the message and: Apr 12 06:44:15 mfsbsd kernel: msi: routing MSI IRQ 266 to local APIC 0 vector 80 And interrupts appear to be delivered properly. Before I dive in, can anyone explain this behavior? Thanks in advance. Dr.