From owner-freebsd-net@FreeBSD.ORG Fri Dec 24 23:07:06 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D34E11065670; Fri, 24 Dec 2010 23:07:06 +0000 (UTC) (envelope-from jan.koum@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 5A2928FC19; Fri, 24 Dec 2010 23:07:06 +0000 (UTC) Received: by qyk36 with SMTP id 36so7081293qyk.13 for ; Fri, 24 Dec 2010 15:07:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=G1X9KC6+eWILcy89mcqm160Y48i5NdZMuAMIZADu/iE=; b=haekRCHNecGttakH3E6lVHx9lsnLOMLwE4KACToQ85ZxaNN6YycnAjgpoPhzxKTavW hR8/yC3GZGu1xgyUKTwijUj4DBtJADj0H+XBk/yeubrX9RHsMs8I2HQWjNq9Q2wcJem6 hteBNSf3pEFq3B1nFn58jsgox/nvnpd+mEQV8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=F0kX7Kw+m7+VmlegUp+dH4Zgx0FMy6Hro9VZMRugOluJZYaEXsGqrDri/ISXex/zfJ KDkySJarvdrqNQmvBEtpw/fyK7ztFDERNZpyI6LRyJ6UrJS0t/3lq/yuikDzXxnyWLG+ oauSzNKGUeKAKuiTz4omH48RoGb3dP4UHqI/o= MIME-Version: 1.0 Received: by 10.229.187.199 with SMTP id cx7mr8506328qcb.132.1293230671009; Fri, 24 Dec 2010 14:44:31 -0800 (PST) Sender: jan.koum@gmail.com Received: by 10.229.241.211 with HTTP; Fri, 24 Dec 2010 14:44:30 -0800 (PST) In-Reply-To: References: <1290533941.3173.50.camel@home-yahoo> <4CEC0548.1080801@sentex.net> Date: Fri, 24 Dec 2010 14:44:30 -0800 X-Google-Sender-Auth: 38AsrF8lzP7uUU_DiMZZxLWiSKo Message-ID: From: Jan Koum To: Ivan Voras , Mike Tancsa Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "freebsd-net@freebsd.org" , Jack Vogel , Sean Bruno , "freebsd-hardware@freebsd.org" Subject: Re: em driver, 82574L chip, and possibly ASPM X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Dec 2010 23:07:06 -0000 hi Ivan and Mike, wanted to follow up and see if you found a solid long-term solution to this bug. we are still seeing this problem in our 8.2 environment with ASPM already disabled. here is what we have: 1. motherboard is SuperMicro X8SIE-LN4F Intel Xeon: em0@pci0:3:0:0: class=0x020000 card=0x040d15d9 chip=0x10d38086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel 82574L Gigabit Ethernet Controller (82574L)' class = network subclass = ethernet em1@pci0:4:0:0: class=0x020000 card=0x040d15d9 chip=0x10d38086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel 82574L Gigabit Ethernet Controller (82574L)' class = network subclass = ethernet em2@pci0:5:0:0: class=0x020000 card=0x040d15d9 chip=0x10d38086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel 82574L Gigabit Ethernet Controller (82574L)' class = network subclass = ethernet em3@pci0:6:0:0: class=0x020000 card=0x040d15d9 chip=0x10d38086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel 82574L Gigabit Ethernet Controller (82574L)' class = network subclass = ethernet 2. ASPM is already disabled in the BIOS 3. when em1 interface locks up, sysctl debug says: Interface is NOT RUNNING and INACTIVE em1: hw tdh = 0, hw tdt = 0 em1: hw rdh = 0, hw rdt = 0 em1: Tx Queue Status = 0 em1: TX descriptors avail = 110 em1: Tx Descriptors avail failure = 319 em1: RX discarded packets = 0 em1: RX Next to Check = 80 em1: RX Next to Refresh = 80 4. doing "ifconfig em1 down; sleep1; ifconfig em1 up" resolves the issue and removes OACTIVE flag from em1. 5. we are running 8.2-PRERELEASE from December 19th: % grep '$FreeBSD' /usr/src/sys/dev/e1000/if_em.c /*$FreeBSD: src/sys/dev/e1000/if_em.c,v 1.21.2.18 2010/12/14 19:59:39 jfv Exp $*/ dmesg output is: em1: port 0xcc00-0xcc1f mem 0xfb4e0000-0xfb4fffff,0xfb4dc000-0xfb4dffff irq 17 at device 0.0 on pci4 em1: Reserved 0x20000 bytes for rid 0x10 type 3 at 0xfb4e0000 em1: Reserved 0x4000 bytes for rid 0x1c type 3 at 0xfb4dc000 em1: attempting to allocate 3 MSI-X vectors (5 supported) msi: routing MSI-X IRQ 259 to local APIC 0 vector 53 msi: routing MSI-X IRQ 260 to local APIC 0 vector 54 msi: routing MSI-X IRQ 261 to local APIC 0 vector 55 em1: using IRQs 259-261 for MSI-X em1: Using MSIX interrupts with 3 vectors em1: [MPSAFE] em1: [ITHREAD] em1: [MPSAFE] em1: [ITHREAD] em1: [MPSAFE] em1: [ITHREAD] em1: bpf attached em1: Ethernet address: 00:25:90:0e:25:e9 aside from running cronjob every minute to check for dead interface and reset it, is there anything else we can try? thanks. On Tue, Nov 23, 2010 at 10:36 AM, Jack Vogel wrote: > 82574 is supposed to be em, not igb :) Its always had this kind of > 'in-between' > status, it was targeted as a 'client' or consumer part, but it has MSIX > which > make it almost like 8257[56]. > > Mike, there are some further 82574 changes to shared code that I'm looking > into today. > > Jack > > > On Tue, Nov 23, 2010 at 10:17 AM, Mike Tancsa wrote: > > > On 11/23/2010 12:39 PM, Sean Bruno wrote: > > > On Tue, 2010-11-23 at 04:47 -0800, Ivan Voras wrote: > > >> It looks like I'm unfortunate enough to have to deploy on a machine > > >> which has the 82574L Intel NIC chip on a Supermicro X8SIE-F board, > which > > > igb0@pci0:5:0:0: class=0x020000 card=0x8975152d chip=0x10c98086 > > > > Strange, the 82574 attaches as em for me, not igb > > > > em1@pci0:10:0:0: class=0x020000 card=0x34ec8086 chip=0x10d38086 > > rev=0x00 hdr=0x00 > > vendor = 'Intel Corporation' > > device = 'Intel 82574L Gigabit Ethernet Controller (82574L)' > > class = network > > subclass = ethernet > > cap 01[c8] = powerspec 2 supports D0 D3 current D0 > > cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message > > cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) > > cap 11[a0] = MSI-X supports 5 messages in map 0x1c > > ecap 0001[100] = AER 1 0 fatal 0 non-fatal 0 corrected > > ecap 0003[140] = Serial 1 001517ffffed68a4 > > > > Normally, its msix, but I had disabled that hoping it would fix the > problem > > > > em1: port 0x2000-0x201f mem > > 0xb4100000-0xb411ffff,0xb4120000-0xb4123fff irq 16 at dev > > ice 0.0 on pci10 > > em1: Using an MSI interrupt > > em1: [FILTER] > > em1: Ethernet address: 00:15:17:ed:68:a4 > > > > > > ---Mike > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-hardware@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hardware > To unsubscribe, send any mail to "freebsd-hardware-unsubscribe@freebsd.org > " >