From owner-freebsd-net@FreeBSD.ORG Mon Feb 7 22:33:48 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AF22B1065672 for ; Mon, 7 Feb 2011 22:33:48 +0000 (UTC) (envelope-from fodillemlinkarim@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 7B7748FC16 for ; Mon, 7 Feb 2011 22:33:48 +0000 (UTC) Received: by pwi10 with SMTP id 10so1210929pwi.13 for ; Mon, 07 Feb 2011 14:33:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=q9syjohT/nbpzreBfvNlZlCAxSWOfFomp6ScMizhlno=; b=E6+0eqZZgwa+OiS+MpmvEDl8z51oZbZn/JzEpmOgbvGF7jBMJr15PaLVKAa1+VGFWa fBTos5RosjSKS7FO2ZDnGPToy4EH0wPwesF7BJhJN3s4DQRucEQZmMy090CjPrRTG2EO yfouMyQ22VqxlsR7PzBOusnI7ak09EO7/azIY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=g9W8YWXRw41jTr6Ee5DaubFkyvFh/7N2xovlIslvLiMuuaMZd+XmA3q2p3hia1P+MG YqEPvo45tBYimdejRjLh7kkuazc39D1NFpB92DQe4Ijb6a3MLNSAc+3XajQCaCbk7wUd 9pTK0tSmDARI5u+xywJlQlmVpMyaZQEvm6K5M= MIME-Version: 1.0 Received: by 10.142.48.17 with SMTP id v17mr16049072wfv.314.1297118027261; Mon, 07 Feb 2011 14:33:47 -0800 (PST) Received: by 10.142.246.3 with HTTP; Mon, 7 Feb 2011 14:33:47 -0800 (PST) In-Reply-To: References: <10510673199.20110207203507@serebryakov.spb.ru> Date: Mon, 7 Feb 2011 17:33:47 -0500 Message-ID: From: Karim Fodil-Lemelin To: freebsd-net@freebsd.org Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Fwd: igb driver tx hangs when out of mbuf clusters X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Feb 2011 22:33:48 -0000 Subject: Re: igb driver tx hangs when out of mbuf clusters > To: Lev Serebryakov > Cc: freebsd-net@freebsd.org > > > 2011/2/7 Lev Serebryakov > > Hello, Karim. >> You wrote 7 =C6=C5=D7=D2=C1=CC=D1 2011 =C7., 19:58:04: >> >> >> > The issue is with the igb driver from 7.4 RC3 r218406. If the driver >> runs >> > out of mbuf clusters it simply stops receiving even after the clusters >> have >> > been freed. >> It looks like my problems with em0 (see thread "em0 hangs without >> any messages like "Watchdog timeout", only down/up reset it.")... >> Codebase for em and igb is somewhat common... >> >> -- >> // Black Lion AKA Lev Serebryakov >> >> I agree. > > Do you get missed packets in mac_stats (sysctl dev.em | grep missed)? > > I might not have mentioned but I can also 'fix' the problem by doing > ifconfig igb0 down/up. > > I will try using POLLING to 'automatize' the reset as you mentioned in yo= ur > thread. > > Karim. > > Follow up on tests with POLLING: The problem is still occurring although it takes more time ... Outputs of sysctl dev.igb0 and netstat -m will follow: 9219/99426/108645 mbufs in use (current/cache/total) 9217/90783/100000/100000 mbuf clusters in use (current/cache/total/max) 0/640 mbuf+clusters out of packet secondary zone in use (current/cache) 0/12800/12800/12800 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) 20738K/257622K/278361K bytes allocated to network (current/cache/total) 0/291/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/5/6656 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines dev.igb.0.%desc: Intel(R) PRO/1000 Network Connection version - 2.0.7 dev.igb.0.%driver: igb dev.igb.0.%location: slot=3D0 function=3D0 dev.igb.0.%pnpinfo: vendor=3D0x8086 device=3D0x10a7 subvendor=3D0x8086 subdevice=3D0x0000 class=3D0x020000 dev.igb.0.%parent: pci7 dev.igb.0.nvm: -1 dev.igb.0.flow_control: 3 dev.igb.0.enable_aim: 1 dev.igb.0.rx_processing_limit: 100 dev.igb.0.link_irq: 4 dev.igb.0.dropped: 0 dev.igb.0.tx_dma_fail: 0 dev.igb.0.rx_overruns: 464 dev.igb.0.watchdog_timeouts: 0 dev.igb.0.device_control: 1490027073 dev.igb.0.rx_control: 67141658 dev.igb.0.interrupt_mask: 0 dev.igb.0.extended_int_mask: 0 dev.igb.0.tx_buf_alloc: 14 dev.igb.0.rx_buf_alloc: 34 dev.igb.0.fc_high_water: 29488 dev.igb.0.fc_low_water: 29480 dev.igb.0.queue0.interrupt_rate: 111111 dev.igb.0.queue0.txd_head: 877 dev.igb.0.queue0.txd_tail: 877 dev.igb.0.queue0.no_desc_avail: 0 dev.igb.0.queue0.tx_packets: 92013 dev.igb.0.queue0.rxd_head: 570 dev.igb.0.queue0.rxd_tail: 570 dev.igb.0.queue0.rx_packets: 163386 dev.igb.0.queue0.rx_bytes: 240260310 dev.igb.0.queue0.lro_queued: 0 dev.igb.0.queue0.lro_flushed: 0 dev.igb.0.mac_stats.excess_coll: 0 dev.igb.0.mac_stats.single_coll: 0 dev.igb.0.mac_stats.multiple_coll: 0 dev.igb.0.mac_stats.late_coll: 0 dev.igb.0.mac_stats.collision_count: 0 dev.igb.0.mac_stats.symbol_errors: 0 dev.igb.0.mac_stats.sequence_errors: 0 dev.igb.0.mac_stats.defer_count: 0 dev.igb.0.mac_stats.missed_packets: 3104 dev.igb.0.mac_stats.recv_no_buff: 4016 dev.igb.0.mac_stats.recv_undersize: 0 dev.igb.0.mac_stats.recv_fragmented: 0 dev.igb.0.mac_stats.recv_oversize: 0 dev.igb.0.mac_stats.recv_jabber: 0 dev.igb.0.mac_stats.recv_errs: 0 dev.igb.0.mac_stats.crc_errs: 0 dev.igb.0.mac_stats.alignment_errs: 0 dev.igb.0.mac_stats.coll_ext_errs: 0 dev.igb.0.mac_stats.xon_recvd: 0 dev.igb.0.mac_stats.xon_txd: 346 dev.igb.0.mac_stats.xoff_recvd: 0 dev.igb.0.mac_stats.xoff_txd: 3450 dev.igb.0.mac_stats.total_pkts_recvd: 166515 dev.igb.0.mac_stats.good_pkts_recvd: 163411 dev.igb.0.mac_stats.bcast_pkts_recvd: 0 dev.igb.0.mac_stats.mcast_pkts_recvd: 51 dev.igb.0.mac_stats.rx_frames_64: 10 dev.igb.0.mac_stats.rx_frames_65_127: 1601 dev.igb.0.mac_stats.rx_frames_128_255: 53 dev.igb.0.mac_stats.rx_frames_256_511: 42 dev.igb.0.mac_stats.rx_frames_512_1023: 18 dev.igb.0.mac_stats.rx_frames_1024_1522: 161687 dev.igb.0.mac_stats.good_octets_recvd: 240948229 dev.igb.0.mac_stats.good_octets_txd: 5947150 dev.igb.0.mac_stats.total_pkts_txd: 95809 dev.igb.0.mac_stats.good_pkts_txd: 92013 dev.igb.0.mac_stats.bcast_pkts_txd: 1516 dev.igb.0.mac_stats.mcast_pkts_txd: 1817 dev.igb.0.mac_stats.tx_frames_64: 90302 dev.igb.0.mac_stats.tx_frames_65_127: 1711 dev.igb.0.mac_stats.tx_frames_128_255: 0 dev.igb.0.mac_stats.tx_frames_256_511: 0 dev.igb.0.mac_stats.tx_frames_512_1023: 0 dev.igb.0.mac_stats.tx_frames_1024_1522: 0 dev.igb.0.mac_stats.tso_txd: 0 dev.igb.0.mac_stats.tso_ctx_fail: 0 dev.igb.0.interrupts.asserts: 5584 dev.igb.0.interrupts.rx_pkt_timer: 163411 dev.igb.0.interrupts.rx_abs_timer: 163386 dev.igb.0.interrupts.tx_pkt_timer: 92013 dev.igb.0.interrupts.tx_abs_timer: 0 dev.igb.0.interrupts.tx_queue_empty: 92013 dev.igb.0.interrupts.tx_queue_min_thresh: 0 dev.igb.0.interrupts.rx_desc_min_thresh: 19 dev.igb.0.interrupts.rx_overrun: 0 dev.igb.0.host.breaker_tx_pkt: 0 dev.igb.0.host.host_tx_pkt_discard: 0 dev.igb.0.host.rx_pkt: 0 dev.igb.0.host.breaker_rx_pkts: 0 dev.igb.0.host.breaker_rx_pkt_drop: 0 dev.igb.0.host.tx_good_pkt: 0 dev.igb.0.host.breaker_tx_pkt_drop: 0 dev.igb.0.host.rx_good_bytes: 240948229 dev.igb.0.host.tx_good_bytes: 5947150 dev.igb.0.host.length_errors: 0 dev.igb.0.host.serdes_violation_pkt: 0 dev.igb.0.host.header_redir_missed: 0