From owner-freebsd-current@FreeBSD.ORG Wed May 4 15:34:53 2011 Return-Path: Delivered-To: Current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B6A721065676 for ; Wed, 4 May 2011 15:34:53 +0000 (UTC) (envelope-from Daan@vehosting.nl) Received: from VM01.VEHosting.nl (VM016.VEHosting.nl [IPv6:2001:1af8:2100:b020::140]) by mx1.freebsd.org (Postfix) with ESMTP id 501118FC15 for ; Wed, 4 May 2011 15:34:53 +0000 (UTC) Received: from [192.168.72.11] (124-54.bbned.dsl.internl.net [92.254.54.124]) (authenticated bits=0) by VM01.VEHosting.nl (8.14.3/8.13.8) with ESMTP id p44FYsSl088707; Wed, 4 May 2011 17:34:54 +0200 (CEST) (envelope-from Daan@vehosting.nl) From: Daan Vreeken Organization: http://VEHosting.nl/ To: Current@FreeBSD.org Date: Wed, 4 May 2011 17:34:50 +0200 User-Agent: KMail/1.9.10 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <201105041734.50738.Daan@vehosting.nl> x-ve-auth-version: mi-1.1.5 2011-02-07 - Copyright (c) 2008, 2011 - Daan Vreeken - VEHosting x-ve-auth: authenticated as 'pa4dan' on VM01.VEHosting.nl X-Mailman-Approved-At: Wed, 04 May 2011 15:39:33 +0000 Cc: Jack Vogel Subject: Interrupt storm with MSI in combination with em1 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 May 2011 15:34:53 -0000 Hi All, I've just updated a machine to -current (r221321) and since then I'm seeing an interrupt storm on irq 16. The storm goes away when I disable MSI/MSIX with the following lines in "loader.conf" : hw.pci.enable_msix="0" hw.pci.enable_msi="0" According to "dmesg" the following devices share IRQ 16 : pcib1: irq 16 at device 1.0 on pci0 em0: port 0xcc00-0xcc1f mem 0xf7de0000-0xf7dfffff,0xf7d00000-0xf7d7ffff,0xf7ddc000-0xf7ddffff irq 16 at device 0.0 on pci1 vgapci0: port 0xbc00-0xbc07 mem 0xf7800000-0xf7bfffff,0xe0000000-0xefffffff irq 16 at device 2.0 on pci0 ehci0: mem 0xf7cfa000-0xf7cfa3ff irq 16 at device 26.0 on pci0 em1: port 0xec00-0xec1f mem 0xf7fe0000-0xf7ffffff,0xf7f00000-0xf7f7ffff,0xf7fdc000-0xf7fdffff irq 16 at device 0.0 on pci4 pcib4: irq 16 at device 28.5 on pci0 During a storm "vmstat -i" shows a rate of about 220.000 interrupts/sec. MSI interrupt delivery to both 'em0' and 'em1' seems to work correctly during a storm, as I see their counters increase normally in the "vmstat -i" output. As only 'em0' and 'em1' seem to be using MSI interrupts, my guess is that the e1000 driver is causing this problem. Could it be that the driver forgets to clear/mask legacy interrupts when attaching the MSI interrupts perhaps? Any tips on how to debug and/or fix this? The full output of "dmesg" can be found here : http://vehosting.nl/pub_diffs/dmesg_plantje2_2011_05_04.txt And the full output of "pciconf -lv" is here : http://vehosting.nl/pub_diffs/pciconf_plantje2_2011_05_04.txt Regards, -- Daan Vreeken VEHosting http://VEHosting.nl tel: +31-(0)40-7113050 / +31-(0)6-46210825 KvK nr: 17174380