Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 7 Feb 2017 20:39:30 +0300
From:      Alexandre Snarskii <snar@snar.spb.ru>
To:        freebsd-net@freebsd.org
Subject:   igb (82575EB) not passing traffic out.
Message-ID:  <20170207173930.GA64097@staff.retn.net>

next in thread | raw e-mail | index | archive | help

Hi!

After upgrading 12-stable from early January 2017 to February 2017
igb card (onboard dual-port 82575EB) stopped passing outbound traffic. 
It can be seen in tcpdump, but switch reports "no packets received" and 
sysctl counters reports zeros for tx.

Details:

dmesg:

FreeBSD 12.0-CURRENT #0 r313342: Tue Feb  7 17:56:16 MSK 2017

igb0: <Intel(R) PRO/1000 PCI-Express Network Driver> port 0x2020-0x203f mem 0xb1f20000-0xb1f3ffff,0xb1f44000-0xb1f47fff irq 40 at device 0.0 on pci1
igb0: attach_pre capping queues at 4
igb0: using 1024 tx descriptors and 1024 rx descriptors
igb0: msix_init qsets capped at 4
igb0: pxm cpus: 4 queue msgs: 9 admincnt: 1
igb0: using 4 rx queues 4 tx queues 
igb0: Using MSIX interrupts with 5 vectors
igb0: allocated for 4 tx_queues
igb0: allocated for 4 rx_queues
igb0: Ethernet address: 00:15:17:b0:e6:f8
igb0: netmap queues/slots: TX 4/1024, RX 4/1024

pciconf -lv:

igb0@pci0:1:0:0:	class=0x020000 card=0x34de8086 chip=0x10a78086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82575EB Gigabit Network Connection'
    class      = network
    subclass   = ethernet

ifconfig:

igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=6525bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
	ether 00:15:17:b0:e6:f8
	inet 87.245.206.15 netmask 0xffffffc0 broadcast 87.245.206.63 
	inet6 fe80::215:17ff:feb0:e6f8%igb0 prefixlen 64 scopeid 0x1 
	inet6 2a02:2d8:0:20ff:232a::3 prefixlen 64 
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active

(tried to disable all features like lro/tso/csum, no changes).

sysctl dev.igb.0: note mac_stats.tx_frames_* of zero:

dev.igb.0.wake: 0
dev.igb.0.interrupts.rx_overrun: 0
dev.igb.0.interrupts.rx_desc_min_thresh: 0
dev.igb.0.interrupts.tx_queue_min_thresh: 0
dev.igb.0.interrupts.tx_queue_empty: 0
dev.igb.0.interrupts.tx_abs_timer: 0
dev.igb.0.interrupts.tx_pkt_timer: 8
dev.igb.0.interrupts.rx_abs_timer: 9786
dev.igb.0.interrupts.rx_pkt_timer: 9786
dev.igb.0.interrupts.asserts: 9682
dev.igb.0.mac_stats.tso_ctx_fail: 0
dev.igb.0.mac_stats.tso_txd: 0
dev.igb.0.mac_stats.tx_frames_1024_1522: 0
dev.igb.0.mac_stats.tx_frames_512_1023: 0
dev.igb.0.mac_stats.tx_frames_256_511: 0
dev.igb.0.mac_stats.tx_frames_128_255: 0
dev.igb.0.mac_stats.tx_frames_65_127: 0
dev.igb.0.mac_stats.tx_frames_64: 0
dev.igb.0.mac_stats.mcast_pkts_txd: 0
dev.igb.0.mac_stats.bcast_pkts_txd: 0
dev.igb.0.mac_stats.good_pkts_txd: 0
dev.igb.0.mac_stats.total_pkts_txd: 0
dev.igb.0.mac_stats.good_octets_txd: 0
dev.igb.0.mac_stats.good_octets_recvd: 907217
dev.igb.0.mac_stats.rx_frames_1024_1522: 0
dev.igb.0.mac_stats.rx_frames_512_1023: 304
dev.igb.0.mac_stats.rx_frames_256_511: 258
dev.igb.0.mac_stats.rx_frames_128_255: 191
dev.igb.0.mac_stats.rx_frames_65_127: 2817
dev.igb.0.mac_stats.rx_frames_64: 6216
dev.igb.0.mac_stats.mcast_pkts_recvd: 2409
dev.igb.0.mac_stats.bcast_pkts_recvd: 7369
dev.igb.0.mac_stats.good_pkts_recvd: 9786
dev.igb.0.mac_stats.total_pkts_recvd: 15480
dev.igb.0.mac_stats.xoff_txd: 0
dev.igb.0.mac_stats.xoff_recvd: 0
dev.igb.0.mac_stats.xon_txd: 0
dev.igb.0.mac_stats.xon_recvd: 0
dev.igb.0.mac_stats.coll_ext_errs: 0
dev.igb.0.mac_stats.alignment_errs: 0
dev.igb.0.mac_stats.crc_errs: 0
dev.igb.0.mac_stats.recv_errs: 0
dev.igb.0.mac_stats.recv_jabber: 0
dev.igb.0.mac_stats.recv_oversize: 0
dev.igb.0.mac_stats.recv_fragmented: 0
dev.igb.0.mac_stats.recv_undersize: 0
dev.igb.0.mac_stats.recv_no_buff: 0
dev.igb.0.mac_stats.missed_packets: 0
dev.igb.0.mac_stats.defer_count: 0
dev.igb.0.mac_stats.sequence_errors: 0
dev.igb.0.mac_stats.symbol_errors: 0
dev.igb.0.mac_stats.collision_count: 0
dev.igb.0.mac_stats.late_coll: 0
dev.igb.0.mac_stats.multiple_coll: 0
dev.igb.0.mac_stats.single_coll: 0
dev.igb.0.mac_stats.excess_coll: 0
dev.igb.0.queue_rx_3.rx_irq: 0
dev.igb.0.queue_rx_3.rxd_tail: 44
dev.igb.0.queue_rx_3.rxd_head: 46
dev.igb.0.queue_rx_2.rx_irq: 0
dev.igb.0.queue_rx_2.rxd_tail: 158
dev.igb.0.queue_rx_2.rxd_head: 160
dev.igb.0.queue_rx_1.rx_irq: 0
dev.igb.0.queue_rx_1.rxd_tail: 60
dev.igb.0.queue_rx_1.rxd_head: 62
dev.igb.0.queue_rx_0.rx_irq: 0
dev.igb.0.queue_rx_0.rxd_tail: 300
dev.igb.0.queue_rx_0.rxd_head: 302
dev.igb.0.queue_tx_3.no_desc_avail: 0
dev.igb.0.queue_tx_3.tx_irq: 0
dev.igb.0.queue_tx_3.txd_tail: 0
dev.igb.0.queue_tx_3.txd_head: 0
dev.igb.0.queue_tx_2.no_desc_avail: 0
dev.igb.0.queue_tx_2.tx_irq: 0
dev.igb.0.queue_tx_2.txd_tail: 0
dev.igb.0.queue_tx_2.txd_head: 0
dev.igb.0.queue_tx_1.no_desc_avail: 0
dev.igb.0.queue_tx_1.tx_irq: 0
dev.igb.0.queue_tx_1.txd_tail: 0
dev.igb.0.queue_tx_1.txd_head: 0
dev.igb.0.queue_tx_0.no_desc_avail: 0
dev.igb.0.queue_tx_0.tx_irq: 0
dev.igb.0.queue_tx_0.txd_tail: 457
dev.igb.0.queue_tx_0.txd_head: 1
dev.igb.0.fc_low_water: 45604
dev.igb.0.fc_high_water: 47104
dev.igb.0.rx_control: 67403806
dev.igb.0.device_control: 1490027073
dev.igb.0.watchdog_timeouts: 0
dev.igb.0.rx_overruns: 0
dev.igb.0.tx_dma_fail: 0
dev.igb.0.mbuf_defrag_fail: 0
dev.igb.0.link_irq: 4
dev.igb.0.dropped: 0
dev.igb.0.eee_control: 1
dev.igb.0.rx_processing_limit: 100
dev.igb.0.itr: 488
dev.igb.0.tx_abs_int_delay: 66
dev.igb.0.rx_abs_int_delay: 66
dev.igb.0.tx_int_delay: 66
dev.igb.0.rx_int_delay: 0
dev.igb.0.reg_dump: General Registers
	CTRL	 58d00241
	STATUS	 00080787
	CTRL_EXIT	 91180c01

Interrupt Registers
	ICR	 00000000

RX Registers
	RCTL	 0404801e
	RDLEN	 00004000
	RDH	 0000012e
	RDT	 0000012c
	RXDCTL	 02040810
	RDBAL	 02ec1000
	RDBAH	 00000000

TX Registers
	TCTL	 b50400fa
	TDBAL	 02eb1000
	TDBAH	 00000000
	TDLEN	 00004000
	TDH	 00000001
	TDT	 000001c9
	TXDCTL	 0241011f
	TDFH	 80001800
	TDFT	 00001800
	TDFHS	 00001800
	TDFPC	 00000000


dev.igb.0.fc: 0
dev.igb.0.debug: -1
dev.igb.0.nvm: -1
dev.igb.0.iflib.rxq3.rxq_fl0.credits: 1023
dev.igb.0.iflib.rxq3.rxq_fl0.cidx: 46
dev.igb.0.iflib.rxq3.rxq_fl0.pidx: 45
dev.igb.0.iflib.rxq2.rxq_fl0.credits: 1023
dev.igb.0.iflib.rxq2.rxq_fl0.cidx: 160
dev.igb.0.iflib.rxq2.rxq_fl0.pidx: 159
dev.igb.0.iflib.rxq1.rxq_fl0.credits: 1023
dev.igb.0.iflib.rxq1.rxq_fl0.cidx: 62
dev.igb.0.iflib.rxq1.rxq_fl0.pidx: 61
dev.igb.0.iflib.rxq0.rxq_fl0.credits: 1023
dev.igb.0.iflib.rxq0.rxq_fl0.cidx: 302
dev.igb.0.iflib.rxq0.rxq_fl0.pidx: 301
dev.igb.0.iflib.txq3.r_abdications: 0
dev.igb.0.iflib.txq3.r_restarts: 0
dev.igb.0.iflib.txq3.r_stalls: 0
dev.igb.0.iflib.txq3.r_starts: 0
dev.igb.0.iflib.txq3.r_drops: 0
dev.igb.0.iflib.txq3.r_enqueues: 0
dev.igb.0.iflib.txq3.ring_state: pidx_head: 0000 pidx_tail: 0000 cidx: 0000 state: IDLE
dev.igb.0.iflib.txq3.txq_cleaned: 0
dev.igb.0.iflib.txq3.txq_processed: 0
dev.igb.0.iflib.txq3.txq_in_use: 0
dev.igb.0.iflib.txq3.txq_cidx_processed: 0
dev.igb.0.iflib.txq3.txq_cidx: 0
dev.igb.0.iflib.txq3.txq_pidx: 0
dev.igb.0.iflib.txq3.no_tx_dma_setup: 0
dev.igb.0.iflib.txq3.txd_encap_efbig: 0
dev.igb.0.iflib.txq3.tx_map_failed: 0
dev.igb.0.iflib.txq3.no_desc_avail: 0
dev.igb.0.iflib.txq3.mbuf_defrag_failed: 0
dev.igb.0.iflib.txq3.m_pullups: 0
dev.igb.0.iflib.txq3.mbuf_defrag: 0
dev.igb.0.iflib.txq2.r_abdications: 0
dev.igb.0.iflib.txq2.r_restarts: 0
dev.igb.0.iflib.txq2.r_stalls: 0
dev.igb.0.iflib.txq2.r_starts: 0
dev.igb.0.iflib.txq2.r_drops: 0
dev.igb.0.iflib.txq2.r_enqueues: 0
dev.igb.0.iflib.txq2.ring_state: pidx_head: 0000 pidx_tail: 0000 cidx: 0000 state: IDLE
dev.igb.0.iflib.txq2.txq_cleaned: 0
dev.igb.0.iflib.txq2.txq_processed: 0
dev.igb.0.iflib.txq2.txq_in_use: 0
dev.igb.0.iflib.txq2.txq_cidx_processed: 0
dev.igb.0.iflib.txq2.txq_cidx: 0
dev.igb.0.iflib.txq2.txq_pidx: 0
dev.igb.0.iflib.txq2.no_tx_dma_setup: 0
dev.igb.0.iflib.txq2.txd_encap_efbig: 0
dev.igb.0.iflib.txq2.tx_map_failed: 0
dev.igb.0.iflib.txq2.no_desc_avail: 0
dev.igb.0.iflib.txq2.mbuf_defrag_failed: 0
dev.igb.0.iflib.txq2.m_pullups: 0
dev.igb.0.iflib.txq2.mbuf_defrag: 0
dev.igb.0.iflib.txq1.r_abdications: 0
dev.igb.0.iflib.txq1.r_restarts: 0
dev.igb.0.iflib.txq1.r_stalls: 0
dev.igb.0.iflib.txq1.r_starts: 0
dev.igb.0.iflib.txq1.r_drops: 0
dev.igb.0.iflib.txq1.r_enqueues: 0
dev.igb.0.iflib.txq1.ring_state: pidx_head: 0000 pidx_tail: 0000 cidx: 0000 state: IDLE
dev.igb.0.iflib.txq1.txq_cleaned: 0
dev.igb.0.iflib.txq1.txq_processed: 0
dev.igb.0.iflib.txq1.txq_in_use: 0
dev.igb.0.iflib.txq1.txq_cidx_processed: 0
dev.igb.0.iflib.txq1.txq_cidx: 0
dev.igb.0.iflib.txq1.txq_pidx: 0
dev.igb.0.iflib.txq1.no_tx_dma_setup: 0
dev.igb.0.iflib.txq1.txd_encap_efbig: 0
dev.igb.0.iflib.txq1.tx_map_failed: 0
dev.igb.0.iflib.txq1.no_desc_avail: 0
dev.igb.0.iflib.txq1.mbuf_defrag_failed: 0
dev.igb.0.iflib.txq1.m_pullups: 0
dev.igb.0.iflib.txq1.mbuf_defrag: 0
dev.igb.0.iflib.txq0.r_abdications: 0
dev.igb.0.iflib.txq0.r_restarts: 0
dev.igb.0.iflib.txq0.r_stalls: 0
dev.igb.0.iflib.txq0.r_starts: 359
dev.igb.0.iflib.txq0.r_drops: 0
dev.igb.0.iflib.txq0.r_enqueues: 359
dev.igb.0.iflib.txq0.ring_state: pidx_head: 0359 pidx_tail: 0359 cidx: 0359 state: IDLE
dev.igb.0.iflib.txq0.txq_cleaned: 0
dev.igb.0.iflib.txq0.txq_processed: 1
dev.igb.0.iflib.txq0.txq_in_use: 457
dev.igb.0.iflib.txq0.txq_cidx_processed: 1
dev.igb.0.iflib.txq0.txq_cidx: 0
dev.igb.0.iflib.txq0.txq_pidx: 457
dev.igb.0.iflib.txq0.no_tx_dma_setup: 0
dev.igb.0.iflib.txq0.txd_encap_efbig: 0
dev.igb.0.iflib.txq0.tx_map_failed: 0
dev.igb.0.iflib.txq0.no_desc_avail: 0
dev.igb.0.iflib.txq0.mbuf_defrag_failed: 0
dev.igb.0.iflib.txq0.m_pullups: 0
dev.igb.0.iflib.txq0.mbuf_defrag: 0
dev.igb.0.iflib.override_nrxds: 0
dev.igb.0.iflib.override_ntxds: 0
dev.igb.0.iflib.override_qs_enable: 0
dev.igb.0.iflib.override_nrxqs: 0
dev.igb.0.iflib.override_ntxqs: 0
dev.igb.0.iflib.driver_version: 7.6.1-k
dev.igb.0.%parent: pci1
dev.igb.0.%pnpinfo: vendor=0x8086 device=0x10a7 subvendor=0x8086 subdevice=0x34de class=0x020000
dev.igb.0.%location: slot=0 function=0 dbsf=pci0:1:0:0 handle=\_SB_.PCI0.MRP1.ZOR1
dev.igb.0.%driver: igb
dev.igb.0.%desc: Intel(R) PRO/1000 PCI-Express Network Driver

tcpdump with arp replies (demonstrating that input traffic reaches kernel ok):

tcpdump: listening on igb0, link-type EN10MB (Ethernet), capture size 262144 bytes
18:33:06.929553 00:15:17:b0:e6:f8 > b0:a8:6e:7b:47:c8, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Reply 87.245.206.15 is-at 00:15:17:b0:e6:f8, length 28
18:33:07.369398 00:15:17:b0:e6:f8 > 5c:5e:ab:0f:2a:80, ethertype 802.1Q (0x8100), length 46: vlan 490, p 0, ethertype ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.31.0.2 is-at 00:15:17:b0:e6:f8, length 28

switch output demonstrating that there were no packets from host
in last four hours:

  Last flapped   : 2017-02-07 18:09:50 GMT-3 (00:24:09 ago)
  Statistics last cleared: 2017-02-07 13:54:55 GMT-3 (04:39:04 ago)
  Traffic statistics:
   Input  bytes  :                    0                    0 bps
   Output bytes  :             30559996                16160 bps
   Input  packets:                    0                    0 pps
   Output packets:               273220                   20 pps




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20170207173930.GA64097>