Date: Mon, 7 Feb 2011 17:33:47 -0500 From: Karim Fodil-Lemelin <fodillemlinkarim@gmail.com> To: freebsd-net@freebsd.org Subject: Fwd: igb driver tx hangs when out of mbuf clusters Message-ID: <AANLkTi=AT6tVHbG-jz01275-8pMzVFfHiU9Hz5BzF4yZ@mail.gmail.com> In-Reply-To: <AANLkTinuLMVif-tNE3s%2B%2BtJccm%2B-_3qvM=Z4BtTF0%2B1q@mail.gmail.com> References: <AANLkTim=OYB5cC1H86N_-tDW1w_ipR5-gZjZnT6k%2BMv5@mail.gmail.com> <10510673199.20110207203507@serebryakov.spb.ru> <AANLkTing_L5eLe09xzwgmtF4cp4qO8n8mdRsp4d4ZAxY@mail.gmail.com> <AANLkTikPEK_69if-gZ0nygdLBOTtrUZmPNTnyhHtJr6K@mail.gmail.com> <AANLkTinuLMVif-tNE3s%2B%2BtJccm%2B-_3qvM=Z4BtTF0%2B1q@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Subject: Re: igb driver tx hangs when out of mbuf clusters > To: Lev Serebryakov <lev@serebryakov.spb.ru> > Cc: freebsd-net@freebsd.org > > > 2011/2/7 Lev Serebryakov <lev@serebryakov.spb.ru> > > Hello, Karim. >> You wrote 7 =C6=C5=D7=D2=C1=CC=D1 2011 =C7., 19:58:04: >> >> >> > The issue is with the igb driver from 7.4 RC3 r218406. If the driver >> runs >> > out of mbuf clusters it simply stops receiving even after the clusters >> have >> > been freed. >> It looks like my problems with em0 (see thread "em0 hangs without >> any messages like "Watchdog timeout", only down/up reset it.")... >> Codebase for em and igb is somewhat common... >> >> -- >> // Black Lion AKA Lev Serebryakov <lev@serebryakov.spb.ru> >> >> I agree. > > Do you get missed packets in mac_stats (sysctl dev.em | grep missed)? > > I might not have mentioned but I can also 'fix' the problem by doing > ifconfig igb0 down/up. > > I will try using POLLING to 'automatize' the reset as you mentioned in yo= ur > thread. > > Karim. > > Follow up on tests with POLLING: The problem is still occurring although it takes more time ... Outputs of sysctl dev.igb0 and netstat -m will follow: 9219/99426/108645 mbufs in use (current/cache/total) 9217/90783/100000/100000 mbuf clusters in use (current/cache/total/max) 0/640 mbuf+clusters out of packet secondary zone in use (current/cache) 0/12800/12800/12800 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) 20738K/257622K/278361K bytes allocated to network (current/cache/total) 0/291/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/5/6656 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines dev.igb.0.%desc: Intel(R) PRO/1000 Network Connection version - 2.0.7 dev.igb.0.%driver: igb dev.igb.0.%location: slot=3D0 function=3D0 dev.igb.0.%pnpinfo: vendor=3D0x8086 device=3D0x10a7 subvendor=3D0x8086 subdevice=3D0x0000 class=3D0x020000 dev.igb.0.%parent: pci7 dev.igb.0.nvm: -1 dev.igb.0.flow_control: 3 dev.igb.0.enable_aim: 1 dev.igb.0.rx_processing_limit: 100 dev.igb.0.link_irq: 4 dev.igb.0.dropped: 0 dev.igb.0.tx_dma_fail: 0 dev.igb.0.rx_overruns: 464 dev.igb.0.watchdog_timeouts: 0 dev.igb.0.device_control: 1490027073 dev.igb.0.rx_control: 67141658 dev.igb.0.interrupt_mask: 0 dev.igb.0.extended_int_mask: 0 dev.igb.0.tx_buf_alloc: 14 dev.igb.0.rx_buf_alloc: 34 dev.igb.0.fc_high_water: 29488 dev.igb.0.fc_low_water: 29480 dev.igb.0.queue0.interrupt_rate: 111111 dev.igb.0.queue0.txd_head: 877 dev.igb.0.queue0.txd_tail: 877 dev.igb.0.queue0.no_desc_avail: 0 dev.igb.0.queue0.tx_packets: 92013 dev.igb.0.queue0.rxd_head: 570 dev.igb.0.queue0.rxd_tail: 570 dev.igb.0.queue0.rx_packets: 163386 dev.igb.0.queue0.rx_bytes: 240260310 dev.igb.0.queue0.lro_queued: 0 dev.igb.0.queue0.lro_flushed: 0 dev.igb.0.mac_stats.excess_coll: 0 dev.igb.0.mac_stats.single_coll: 0 dev.igb.0.mac_stats.multiple_coll: 0 dev.igb.0.mac_stats.late_coll: 0 dev.igb.0.mac_stats.collision_count: 0 dev.igb.0.mac_stats.symbol_errors: 0 dev.igb.0.mac_stats.sequence_errors: 0 dev.igb.0.mac_stats.defer_count: 0 dev.igb.0.mac_stats.missed_packets: 3104 dev.igb.0.mac_stats.recv_no_buff: 4016 dev.igb.0.mac_stats.recv_undersize: 0 dev.igb.0.mac_stats.recv_fragmented: 0 dev.igb.0.mac_stats.recv_oversize: 0 dev.igb.0.mac_stats.recv_jabber: 0 dev.igb.0.mac_stats.recv_errs: 0 dev.igb.0.mac_stats.crc_errs: 0 dev.igb.0.mac_stats.alignment_errs: 0 dev.igb.0.mac_stats.coll_ext_errs: 0 dev.igb.0.mac_stats.xon_recvd: 0 dev.igb.0.mac_stats.xon_txd: 346 dev.igb.0.mac_stats.xoff_recvd: 0 dev.igb.0.mac_stats.xoff_txd: 3450 dev.igb.0.mac_stats.total_pkts_recvd: 166515 dev.igb.0.mac_stats.good_pkts_recvd: 163411 dev.igb.0.mac_stats.bcast_pkts_recvd: 0 dev.igb.0.mac_stats.mcast_pkts_recvd: 51 dev.igb.0.mac_stats.rx_frames_64: 10 dev.igb.0.mac_stats.rx_frames_65_127: 1601 dev.igb.0.mac_stats.rx_frames_128_255: 53 dev.igb.0.mac_stats.rx_frames_256_511: 42 dev.igb.0.mac_stats.rx_frames_512_1023: 18 dev.igb.0.mac_stats.rx_frames_1024_1522: 161687 dev.igb.0.mac_stats.good_octets_recvd: 240948229 dev.igb.0.mac_stats.good_octets_txd: 5947150 dev.igb.0.mac_stats.total_pkts_txd: 95809 dev.igb.0.mac_stats.good_pkts_txd: 92013 dev.igb.0.mac_stats.bcast_pkts_txd: 1516 dev.igb.0.mac_stats.mcast_pkts_txd: 1817 dev.igb.0.mac_stats.tx_frames_64: 90302 dev.igb.0.mac_stats.tx_frames_65_127: 1711 dev.igb.0.mac_stats.tx_frames_128_255: 0 dev.igb.0.mac_stats.tx_frames_256_511: 0 dev.igb.0.mac_stats.tx_frames_512_1023: 0 dev.igb.0.mac_stats.tx_frames_1024_1522: 0 dev.igb.0.mac_stats.tso_txd: 0 dev.igb.0.mac_stats.tso_ctx_fail: 0 dev.igb.0.interrupts.asserts: 5584 dev.igb.0.interrupts.rx_pkt_timer: 163411 dev.igb.0.interrupts.rx_abs_timer: 163386 dev.igb.0.interrupts.tx_pkt_timer: 92013 dev.igb.0.interrupts.tx_abs_timer: 0 dev.igb.0.interrupts.tx_queue_empty: 92013 dev.igb.0.interrupts.tx_queue_min_thresh: 0 dev.igb.0.interrupts.rx_desc_min_thresh: 19 dev.igb.0.interrupts.rx_overrun: 0 dev.igb.0.host.breaker_tx_pkt: 0 dev.igb.0.host.host_tx_pkt_discard: 0 dev.igb.0.host.rx_pkt: 0 dev.igb.0.host.breaker_rx_pkts: 0 dev.igb.0.host.breaker_rx_pkt_drop: 0 dev.igb.0.host.tx_good_pkt: 0 dev.igb.0.host.breaker_tx_pkt_drop: 0 dev.igb.0.host.rx_good_bytes: 240948229 dev.igb.0.host.tx_good_bytes: 5947150 dev.igb.0.host.length_errors: 0 dev.igb.0.host.serdes_violation_pkt: 0 dev.igb.0.host.header_redir_missed: 0
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AANLkTi=AT6tVHbG-jz01275-8pMzVFfHiU9Hz5BzF4yZ>