From owner-freebsd-current@FreeBSD.ORG Wed May 4 18:47:34 2011 Return-Path: Delivered-To: Current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8AA95106566C for ; Wed, 4 May 2011 18:47:34 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 3E0EF8FC0A for ; Wed, 4 May 2011 18:47:33 +0000 (UTC) Received: by qwc9 with SMTP id 9so1217692qwc.13 for ; Wed, 04 May 2011 11:47:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=/t8+u4lhgY5DJH8WAQ78IQnw3EacvW0o+bvnYALLqyE=; b=nemjJ8nFn2a2FzPT5PuGZJdsGjarKlXenUfImuK6zxI7Z5AioBXGBkrmb5Q0AxU4ZO nPNIpA1r9nAa2qkZNpTofAH0jbrZEqNCFPQMUhZrMsUPGxhyUkNloUH+SKsu38PK3Nq6 OXovG/0NmnQeeSzwwUqWoJuxTy2CmzFNt4CJ4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=J5BGDLCaios337/7TI2e9BO/zZbzLOJCBUuk6XyPgzPhkwowllY9m++SosEWULMJFL PcxRPQJnJxtu8NdKjLrDx/HLyE5rHn93AUSinw9dFGBpVNhOTIamP9YA+gNeLfNqxU2e GV3IW41gR3HbERGCRTr28pt21mkFu/7p4i/CE= MIME-Version: 1.0 Received: by 10.52.76.193 with SMTP id m1mr1845165vdw.204.1304534852692; Wed, 04 May 2011 11:47:32 -0700 (PDT) Received: by 10.52.184.169 with HTTP; Wed, 4 May 2011 11:47:32 -0700 (PDT) In-Reply-To: <201105042019.23899.Daan@vehosting.nl> References: <201105041734.50738.Daan@vehosting.nl> <201105042019.23899.Daan@vehosting.nl> Date: Wed, 4 May 2011 11:47:32 -0700 Message-ID: From: Jack Vogel To: Daan Vreeken Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Current@freebsd.org Subject: Re: Interrupt storm with MSI in combination with em1 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 May 2011 18:47:34 -0000 Will you please set it back to a default and then boot and capture the message for me? Thank you, Jack On Wed, May 4, 2011 at 11:19 AM, Daan Vreeken wrote: > Hi Jack, > > Wednesday 04 May 2011 19:46:05 Jack Vogel wrote: > > Who makes your motherboard? The problem you are having is that MSIX AND > > MSI are both failing as em0 comes up, so it falls back to Legacy > interrupt > > mode, > > and must be having some issue with sharing the line, causing the storm. > > The motherboard is an Asus "P7H55-M". > Sorry, I should have mentioned that the dmesg output is from booting with : > > > > hw.pci.enable_msix="0" > > > hw.pci.enable_msi="0" > > .. in "loader.conf". > > With those lines in "loader.conf", MSI and MSIX is disabled, both cards > work > like they should and there is no interrupt storm. > > With MSI/MSIX enabled, both cards work like they should and I see the > counters > of the MSI interrupts increase (in small amounts, like they should), but at > boot-time an interrupt storm starts on 'legacy' IRQ 16. > > Because the only difference between disabling/enabling MSI/MSIX seems to be > in > the way em0/em1 are used, and because 'em1' shares IRQ 16 according to the > dmesg, I'm suspecting 'em1' is causing the storm. > (But please correct me if I'm wrong :) > > What can I do to help track this problem down? > > > > > > > According to "dmesg" the following devices share IRQ 16 : > > > > > > pcib1: irq 16 at device 1.0 on pci0 > > > em0: port > > > 0xcc00-0xcc1f mem > > > 0xf7de0000-0xf7dfffff,0xf7d00000-0xf7d7ffff,0xf7ddc000-0xf7ddffff > > > irq 16 at device 0.0 on pci1 > > > vgapci0: port 0xbc00-0xbc07 > > > mem 0xf7800000-0xf7bfffff,0xe0000000-0xefffffff irq 16 at > > > device 2.0 on > > > pci0 > > > ehci0: mem > > > 0xf7cfa000-0xf7cfa3ff > > > irq 16 at device 26.0 on pci0 > > > em1: port > > > 0xec00-0xec1f mem > > > 0xf7fe0000-0xf7ffffff,0xf7f00000-0xf7f7ffff,0xf7fdc000-0xf7fdffff > > > irq 16 at device 0.0 on pci4 > > > pcib4: irq 16 at device 28.5 on pci0 > > > > > > During a storm "vmstat -i" shows a rate of about 220.000 > interrupts/sec. > > > MSI > > > interrupt delivery to both 'em0' and 'em1' seems to work correctly > during > > > a storm, as I see their counters increase normally in the "vmstat -i" > > > output. > > > > > > As only 'em0' and 'em1' seem to be using MSI interrupts, my guess is > that > > > the > > > e1000 driver is causing this problem. Could it be that the driver > forgets > > > to > > > clear/mask legacy interrupts when attaching the MSI interrupts perhaps? > > > > > > Any tips on how to debug and/or fix this? > > > > > > > > > The full output of "dmesg" can be found here : > > > http://vehosting.nl/pub_diffs/dmesg_plantje2_2011_05_04.txt > > > > > > And the full output of "pciconf -lv" is here : > > > http://vehosting.nl/pub_diffs/pciconf_plantje2_2011_05_04.txt > > > > > > Regards, > -- > Daan Vreeken > VEHosting > http://VEHosting.nl > tel: +31-(0)40-7113050 / +31-(0)6-46210825 > KvK nr: 17174380 >