From owner-freebsd-net@FreeBSD.ORG Sun Feb 6 23:54:04 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C6AC01065670; Sun, 6 Feb 2011 23:54:04 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1-6.sentex.ca [IPv6:2607:f3e0:0:1::12]) by mx1.freebsd.org (Postfix) with ESMTP id 637C18FC15; Sun, 6 Feb 2011 23:54:04 +0000 (UTC) Received: from [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a] (saphire3.sentex.ca [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a]) by smarthost1.sentex.ca (8.14.4/8.14.4) with ESMTP id p16Ns060002096 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sun, 6 Feb 2011 18:54:01 -0500 (EST) (envelope-from mike@sentex.net) Message-ID: <4D4F3497.6050505@sentex.net> Date: Sun, 06 Feb 2011 18:53:59 -0500 From: Mike Tancsa Organization: Sentex Communications User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: Jack Vogel References: <1290533941.3173.50.camel@home-yahoo> <4CEC0548.1080801@sentex.net> <4D2C636B.5040003@sentex.net> <4D3C4795.40205@sentex.net> <4D42EA74.4090807@sentex.net> <1296590190.2326.6.camel@hitfishpass-lx.corp.yahoo.com> <1296591565.2326.7.camel@hitfishpass-lx.corp.yahoo.com> <1296597827.2326.12.camel@hitfishpass-lx.corp.yahoo.com> <4D48C973.7080503@sentex.net> <4D49A26B.5050803@sentex.net> <1296842996.2233.0.camel@hitfishpass-lx.corp.yahoo.com> In-Reply-To: X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on IPv6:2607:f3e0:0:1::12 Cc: "freebsd-net@freebsd.org" , "freebsd-hardware@freebsd.org" , Sean Bruno , Jan Koum , Ivan Voras Subject: Re: em driver, 82574L chip, and possibly ASPM X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Feb 2011 23:54:05 -0000 So far so good. I would often get a hang on the level zero dumps to my backup server Sunday AM, and it made it through! So a good sign, but not a definitive sign. I have a PCIe em card that has this chipset as well and was showing the same sort of problem in a customer's RELENG_7 box. I will see if I can get the customer to try the card in their box with the patch for RELENG_7 as it would show this issue at least once a day until I pulled the card for an older version ---Mike On 2/4/2011 1:12 PM, Jack Vogel wrote: > Was curious too, but being more patient than you :) > > Jack > > > On Fri, Feb 4, 2011 at 10:09 AM, Sean Bruno wrote: > >> Any more data on this problem or do we have to wait a while? >> >> Sean >> >> >> On Wed, 2011-02-02 at 10:28 -0800, Mike Tancsa wrote: >>> On 2/2/2011 12:37 PM, Jack Vogel wrote: >>>> So has everyone that wanted to get something testing been able to do >> so? >>> >>> I have been testing in the back and will deploy to my production box >>> this afternoon. As I am not able to reproduce it easily, it will be a >>> bit before I can say the issue is gone. Jan however, was able to >>> trigger it with greater ease ? >>> >>> ---Mike >>> >>>> >>>> Jack >>>> >>>> >>>> On Tue, Feb 1, 2011 at 7:03 PM, Mike Tancsa wrote: >>>> >>>>> On 2/1/2011 5:03 PM, Sean Bruno wrote: >>>>>> On Tue, 2011-02-01 at 13:43 -0800, Jack Vogel wrote: >>>>>>> To those who are going to test, here is the if_em.c, based on head, >>>>>>> with my >>>>>>> changes, I have to leave for the afternoon, and have not had a >> chance >>>>>>> to build >>>>>>> this, but it should work. I will check back in the later evening. >>>>>>> >>>>>>> Any blatant problems Sean, feel free to fix them :) >>>>>>> >>>>>>> Jack >>>>>>> >>>>>> >>>>>> >>>>>> I suspect that line 1490 should be: >>>>>> if (more_rx || (ifp->if_drv_flags & IFF_DRV_OACTIVE)) { >>>>>> >>>>> >>>>> >>>>> I have hacked up a RELENG_8 version which I think is correct including >>>>> the above change >>>>> >>>>> http://www.tancsa.com/if_em-8.c >>>>> >>>>> >>>>> >>>>> --- if_em.c.orig 2011-02-01 21:47:14.000000000 -0500 >>>>> +++ if_em.c 2011-02-01 21:47:19.000000000 -0500 >>>>> @@ -30,7 +30,7 @@ >>>>> POSSIBILITY OF SUCH DAMAGE. >>>>> >>>>> >>>>> >> ******************************************************************************/ >>>>> -/*$FreeBSD: src/sys/dev/e1000/if_em.c,v 1.21.2.20 2011/01/22 01:37:53 >>>>> jfv Exp $*/ >>>>> +/*$FreeBSD$*/ >>>>> >>>>> #ifdef HAVE_KERNEL_OPTION_HEADERS >>>>> #include "opt_device_polling.h" >>>>> @@ -93,7 +93,7 @@ >>>>> >> /********************************************************************* >>>>> * Driver version: >>>>> >> *********************************************************************/ >>>>> -char em_driver_version[] = "7.1.9"; >>>>> +char em_driver_version[] = "7.1.9-test"; >>>>> >>>>> >> /********************************************************************* >>>>> * PCI Device ID Table >>>>> @@ -927,11 +927,10 @@ >>>>> if (!adapter->link_active) >>>>> return; >>>>> >>>>> - /* Call cleanup if number of TX descriptors low */ >>>>> - if (txr->tx_avail <= EM_TX_CLEANUP_THRESHOLD) >>>>> - em_txeof(txr); >>>>> - >>>>> while (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) { >>>>> + /* First cleanup if TX descriptors low */ >>>>> + if (txr->tx_avail <= EM_TX_CLEANUP_THRESHOLD) >>>>> + em_txeof(txr); >>>>> if (txr->tx_avail < EM_MAX_SCATTER) { >>>>> ifp->if_drv_flags |= IFF_DRV_OACTIVE; >>>>> break; >>>>> @@ -1411,8 +1410,7 @@ >>>>> if (!drbr_empty(ifp, txr->br)) >>>>> em_mq_start_locked(ifp, txr, NULL); >>>>> #else >>>>> - if (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) >>>>> - em_start_locked(ifp, txr); >>>>> + em_start_locked(ifp, txr); >>>>> #endif >>>>> EM_TX_UNLOCK(txr); >>>>> >>>>> @@ -1475,11 +1473,10 @@ >>>>> struct ifnet *ifp = adapter->ifp; >>>>> struct tx_ring *txr = adapter->tx_rings; >>>>> struct rx_ring *rxr = adapter->rx_rings; >>>>> - bool more; >>>>> - >>>>> >>>>> if (ifp->if_drv_flags & IFF_DRV_RUNNING) { >>>>> - more = em_rxeof(rxr, adapter->rx_process_limit, NULL); >>>>> + bool more_rx; >>>>> + more_rx = em_rxeof(rxr, adapter->rx_process_limit, >> NULL); >>>>> >>>>> EM_TX_LOCK(txr); >>>>> em_txeof(txr); >>>>> @@ -1487,12 +1484,10 @@ >>>>> if (!drbr_empty(ifp, txr->br)) >>>>> em_mq_start_locked(ifp, txr, NULL); >>>>> #else >>>>> - if (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) >>>>> - em_start_locked(ifp, txr); >>>>> + em_start_locked(ifp, txr); >>>>> #endif >>>>> - em_txeof(txr); >>>>> EM_TX_UNLOCK(txr); >>>>> - if (more) { >>>>> + if (more_rx || (ifp->if_drv_flags & IFF_DRV_OACTIVE)) >> { >>>>> taskqueue_enqueue(adapter->tq, >> &adapter->que_task); >>>>> return; >>>>> } >>>>> @@ -1604,7 +1599,6 @@ >>>>> if (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) >>>>> em_start_locked(ifp, txr); >>>>> #endif >>>>> - em_txeof(txr); >>>>> E1000_WRITE_REG(&adapter->hw, E1000_IMS, txr->ims); >>>>> EM_TX_UNLOCK(txr); >>>>> } >>>>> @@ -3730,17 +3724,17 @@ >>>>> txr->queue_status = EM_QUEUE_HUNG; >>>>> >>>>> /* >>>>> - * If we have enough room, clear IFF_DRV_OACTIVE >>>>> + * If we have a minimum free, clear IFF_DRV_OACTIVE >>>>> * to tell the stack that it is OK to send packets. >>>>> */ >>>>> - if (txr->tx_avail > EM_TX_CLEANUP_THRESHOLD) { >>>>> + if (txr->tx_avail > EM_MAX_SCATTER) >>>>> ifp->if_drv_flags &= ~IFF_DRV_OACTIVE; >>>>> - /* Disable watchdog if all clean */ >>>>> - if (txr->tx_avail == adapter->num_tx_desc) { >>>>> - txr->queue_status = EM_QUEUE_IDLE; >>>>> - return (FALSE); >>>>> - } >>>>> - } >>>>> + >>>>> + /* Disable watchdog if all clean */ >>>>> + if (txr->tx_avail == adapter->num_tx_desc) { >>>>> + txr->queue_status = EM_QUEUE_IDLE; >>>>> + return (FALSE); >>>>> + } >>>>> >>>>> return (TRUE); >>>>> } >>>>> @@ -5064,8 +5058,8 @@ >>>>> char namebuf[QUEUE_NAME_LEN]; >>>>> >>>>> /* Driver Statistics */ >>>>> - SYSCTL_ADD_UINT(ctx, child, OID_AUTO, "link_irq", >>>>> - CTLFLAG_RD, &adapter->link_irq, 0, >>>>> + SYSCTL_ADD_UINT(ctx, child, OID_AUTO, "link_irq", >>>>> + CTLFLAG_RD, &adapter->link_irq,0, >>>>> "Link MSIX IRQ Handled"); >>>>> SYSCTL_ADD_ULONG(ctx, child, OID_AUTO, "mbuf_alloc_fail", >>>>> CTLFLAG_RD, &adapter->mbuf_alloc_failed, >>>>> @@ -5108,11 +5102,13 @@ >>>>> queue_list = SYSCTL_CHILDREN(queue_node); >>>>> >>>>> SYSCTL_ADD_PROC(ctx, queue_list, OID_AUTO, "txd_head", >>>>> - CTLFLAG_RD, adapter, >> E1000_TDH(txr->me), >>>>> + CTLFLAG_RD, adapter, >>>>> + E1000_TDH(txr->me), >>>>> em_sysctl_reg_handler, "IU", >>>>> "Transmit Descriptor Head"); >>>>> SYSCTL_ADD_PROC(ctx, queue_list, OID_AUTO, "txd_tail", >>>>> - CTLFLAG_RD, adapter, >> E1000_TDT(txr->me), >>>>> + CTLFLAG_RD, adapter, >>>>> + E1000_TDT(txr->me), >>>>> em_sysctl_reg_handler, "IU", >>>>> "Transmit Descriptor Tail"); >>>>> SYSCTL_ADD_ULONG(ctx, queue_list, OID_AUTO, "tx_irq", >>>>> @@ -5123,11 +5119,13 @@ >>>>> "Queue No Descriptor Available"); >>>>> >>>>> SYSCTL_ADD_PROC(ctx, queue_list, OID_AUTO, "rxd_head", >>>>> - CTLFLAG_RD, adapter, >> E1000_RDH(rxr->me), >>>>> + CTLFLAG_RD, adapter, >>>>> + E1000_RDH(rxr->me), >>>>> em_sysctl_reg_handler, "IU", >>>>> "Receive Descriptor Head"); >>>>> SYSCTL_ADD_PROC(ctx, queue_list, OID_AUTO, "rxd_tail", >>>>> - CTLFLAG_RD, adapter, >> E1000_RDT(rxr->me), >>>>> + CTLFLAG_RD, adapter, >>>>> + E1000_RDT(rxr->me), >>>>> em_sysctl_reg_handler, "IU", >>>>> "Receive Descriptor Tail"); >>>>> SYSCTL_ADD_ULONG(ctx, queue_list, OID_AUTO, "rx_irq", >>>>> @@ -5141,19 +5139,19 @@ >>>>> CTLFLAG_RD, NULL, "Statistics"); >>>>> stat_list = SYSCTL_CHILDREN(stat_node); >>>>> >>>>> - SYSCTL_ADD_QUAD(ctx, stat_list, OID_AUTO, "excess_coll", >>>>> + SYSCTL_ADD_QUAD(ctx, stat_list, OID_AUTO, "excess_coll", >>>>> CTLFLAG_RD, &stats->ecol, >>>>> "Excessive collisions"); >>>>> - SYSCTL_ADD_QUAD(ctx, stat_list, OID_AUTO, "single_coll", >>>>> + SYSCTL_ADD_QUAD(ctx, stat_list, OID_AUTO, "single_coll", >>>>> CTLFLAG_RD, &stats->scc, >>>>> "Single collisions"); >>>>> - SYSCTL_ADD_QUAD(ctx, stat_list, OID_AUTO, "multiple_coll", >>>>> + SYSCTL_ADD_QUAD(ctx, stat_list, OID_AUTO, "multiple_coll", >>>>> CTLFLAG_RD, &stats->mcc, >>>>> "Multiple collisions"); >>>>> - SYSCTL_ADD_QUAD(ctx, stat_list, OID_AUTO, "late_coll", >>>>> + SYSCTL_ADD_QUAD(ctx, stat_list, OID_AUTO, "late_coll", >>>>> CTLFLAG_RD, &stats->latecol, >>>>> "Late collisions"); >>>>> - SYSCTL_ADD_QUAD(ctx, stat_list, OID_AUTO, "collision_count", >>>>> + SYSCTL_ADD_QUAD(ctx, stat_list, OID_AUTO, "collision_count", >>>>> CTLFLAG_RD, &stats->colc, >>>>> "Collision Count"); >>>>> SYSCTL_ADD_QUAD(ctx, stat_list, OID_AUTO, "symbol_errors", >>>>> @@ -5240,12 +5238,12 @@ >>>>> SYSCTL_ADD_QUAD(ctx, stat_list, OID_AUTO, >> "rx_frames_1024_1522", >>>>> CTLFLAG_RD, &adapter->stats.prc1522, >>>>> "1023-1522 byte frames received"); >>>>> - SYSCTL_ADD_QUAD(ctx, stat_list, OID_AUTO, "good_octets_recvd", >>>>> + SYSCTL_ADD_QUAD(ctx, stat_list, OID_AUTO, "good_octets_recvd", >>>>> CTLFLAG_RD, &adapter->stats.gorc, >>>>> "Good Octets Received"); >>>>> >>>>> /* Packet Transmission Stats */ >>>>> - SYSCTL_ADD_QUAD(ctx, stat_list, OID_AUTO, "good_octets_txd", >>>>> + SYSCTL_ADD_QUAD(ctx, stat_list, OID_AUTO, "good_octets_txd", >>>>> CTLFLAG_RD, &adapter->stats.gotc, >>>>> "Good Octets Transmitted"); >>>>> SYSCTL_ADD_QUAD(ctx, stat_list, OID_AUTO, "total_pkts_txd", >>>>> >>>>> -- >>>>> ------------------- >>>>> Mike Tancsa, tel +1 519 651 3400 >>>>> Sentex Communications, mike@sentex.net >>>>> Providing Internet services since 1994 www.sentex.net >>>>> Cambridge, Ontario Canada http://www.tancsa.com/ >>>>> >>>> >>> >>> >>> -- >>> ------------------- >>> Mike Tancsa, tel +1 519 651 3400 >>> Sentex Communications, mike@sentex.net >>> Providing Internet services since 1994 www.sentex.net >>> Cambridge, Ontario Canada http://www.tancsa.com/ >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> >> > -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/