From owner-freebsd-amd64@FreeBSD.ORG Tue Jan 20 19:44:01 2009 Return-Path: Delivered-To: freebsd-amd64@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8D287106571D; Tue, 20 Jan 2009 19:44:00 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id E39CA8FC14; Tue, 20 Jan 2009 19:43:59 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (pool-98-109-39-197.nwrknj.fios.verizon.net [98.109.39.197]) by cyrus.watson.org (Postfix) with ESMTPSA id 782BF46B39; Tue, 20 Jan 2009 14:43:59 -0500 (EST) Received: from localhost (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.14.3/8.14.3) with ESMTP id n0KJhaUt026295; Tue, 20 Jan 2009 14:43:53 -0500 (EST) (envelope-from jhb@freebsd.org) From: John Baldwin To: freebsd-amd64@freebsd.org Date: Tue, 20 Jan 2009 11:57:57 -0500 User-Agent: KMail/1.9.7 References: <200901130502.n0D5275X095567@www.freebsd.org> In-Reply-To: <200901130502.n0D5275X095567@www.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200901201157.57956.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Tue, 20 Jan 2009 14:43:53 -0500 (EST) X-Virus-Scanned: ClamAV 0.94.2/8881/Tue Jan 20 09:48:51 2009 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: freebsd-gnats-submit@freebsd.org, Xiuchao Wu Subject: Re: amd64/130483: MSI must be disabled when Myricom 10Gbps Card is used on Dell PowerEdge T300 Server X-BeenThere: freebsd-amd64@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting FreeBSD to the AMD64 platform List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jan 2009 19:44:02 -0000 On Tuesday 13 January 2009 12:02:07 am Xiuchao Wu wrote: > > >Number: 130483 > >Category: amd64 > >Synopsis: MSI must be disabled when Myricom 10Gbps Card is used on Dell PowerEdge T300 Server > >Confidential: no > >Severity: serious > >Priority: medium > >Responsible: freebsd-amd64 > >State: open > >Quarter: > >Keywords: > >Date-Required: > >Class: sw-bug > >Submitter-Id: current-users > >Arrival-Date: Tue Jan 13 05:10:01 UTC 2009 > >Closed-Date: > >Last-Modified: > >Originator: Xiuchao Wu > >Release: FreeBSD 7.0 > >Organization: > National University of Singapore > >Environment: > FreeBSD 7.0/7.1 Release > >Description: > Computer: Dell PowerEdge T300 Server > NICs: Myricom 10G-PCIE-8AL-C > OS: FreeBSD 7.0/7.1 Release > > After I rebuild the kernel (to include NIC driver "mxge") and reboot, kernel panic occurs and the computer is automatically rebooted. After contacting with engineers of Myricom, they suggest to disable message signaled interrupt (MSI) in /boot/loader.conf and these NICs can work now. > " > hw.pci.enable_msix=0 > hw.pci.enable_msi=0 > " > > However, MSI is really very important for high speed data transmission. Huge number of packets generate many interruptions. Considering that Fedora 9 works well on the same computer, it may be a bug of FreeBSD. Below is the screen when kernel panic occurs. > > > ...... > p4tcc3: on cpu3 > pcib0: port 0xcf8-0xcff on acpi0 > pci0: on pcib0 > pcib1: at device 2.0 on pci0 > pci3: on pcib1 > pcib2: at device 3.0 on pci0 > pci4: on pcib2 > pcib3: at device 4.0 on pci0 > pci5: on pcib3 > mxge0: mem 0xd8000000-0xd8ffffff, 0xdfa00000-0xdfafffff irq 16 at device 0.0 on pci5 > panic: nexus_add_irq: failed > ..... How many MSI messages is mxge0 attempting to add and how many other devices in your system are using MSI? Getting a full copy of a verbose dmesg would be most helpful. 'pciconf -lc' might also help answer the first question. -- John Baldwin