From owner-freebsd-net@FreeBSD.ORG Fri Oct 7 20:12:40 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3F209106566B for ; Fri, 7 Oct 2011 20:12:40 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1-6.sentex.ca [IPv6:2607:f3e0:0:1::12]) by mx1.freebsd.org (Postfix) with ESMTP id DAA198FC12 for ; Fri, 7 Oct 2011 20:12:39 +0000 (UTC) Received: from [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a] (saphire3.sentex.ca [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a]) by smarthost1.sentex.ca (8.14.5/8.14.4) with ESMTP id p97KCaMG071927; Fri, 7 Oct 2011 16:12:36 -0400 (EDT) (envelope-from mike@sentex.net) Message-ID: <4E8F5D28.2050109@sentex.net> Date: Fri, 07 Oct 2011 16:12:24 -0400 From: Mike Tancsa Organization: Sentex Communications User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: Arnaud Lacombe References: <4E8F157A.40702@sentex.net> <4E8F51D4.1060509@sentex.net> In-Reply-To: X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.71 on IPv6:2607:f3e0:0:1::12 Cc: freebsd-net@freebsd.org, Jason Wolfe Subject: Re: Intel 82574L interface wedging on em 7.1.9/7.2.3 when MSIX enabled X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2011 20:12:40 -0000 On 10/7/2011 3:55 PM, Arnaud Lacombe wrote: >> When you disable MSI-X, you mean via hw.pci.enable_msix=0 across the >> board, or you disable multi-queue for the NIC, so it uses just one >> interrupt, rather than separate ones for xmit and recv ? >> > em(4)'s multiqueue is misleading. By default, with MSI-X enabled, > before AFAIK, April 2010 it used 2 (RX+TX) queue + 1, ie. 5 MSI-X > vectors[0]. After April 2010, it uses 1 * (RX+TX) queue + 1, ie. 3 > MSI-X vectors. There is no logic for the driver to use 1 vector with > MSI-X enabled. Hi, Very poor choice of wording on my part. I meant to say separate interrupts for the xmit and recv, as opposed to multiqueue which is quite different. On the server I used to see the issue with, it has 2 onboard NICs em0: port 0x4040-0x405f mem 0xb4500000-0xb451ffff,0xb4525000-0xb4525fff irq 16 at device 25.0 on pci0 em0: Using an MSI interrupt em0: [FILTER] em0: Ethernet address: 00:15:17:ed:68:a5 em1: port 0x2000-0x201f mem 0xb4100000-0xb411ffff,0xb4120000-0xb4123fff irq 16 at device 0.0 on pci11 em1: Using MSIX interrupts with 3 vectors em1: [ITHREAD] em1: [ITHREAD] em1: [ITHREAD] em1: Ethernet address: 00:15:17:ed:68:a4 irq257: em0 506496667 813 irq258: em1:rx 0 296438098 476 irq259: em1:tx 0 252033695 404 Actually, I see it in Jason's email now that I re-read it. hw.em.enable_msix="0" However, with 7.1.9 this setting did not help me. ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/