From owner-freebsd-net@FreeBSD.ORG Thu Oct 27 06:30:21 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 91C19106564A for ; Thu, 27 Oct 2011 06:30:21 +0000 (UTC) (envelope-from gpm@hotplug.ru) Received: from gate.pikinvest.ru (gate.pikinvest.ru [87.245.155.170]) by mx1.freebsd.org (Postfix) with ESMTP id 280478FC1B for ; Thu, 27 Oct 2011 06:30:20 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailgate.pik.ru (Postfix) with ESMTP id 2F6551C084A; Thu, 27 Oct 2011 10:30:18 +0400 (MSD) Received: from EX03PIK.PICompany.ru (unknown [192.168.156.52]) by mailgate.pik.ru (Postfix) with ESMTP id 673681C0840; Thu, 27 Oct 2011 10:30:15 +0400 (MSD) Received: from [192.168.148.9] ([192.168.148.9]) by EX03PIK.PICompany.ru with Microsoft SMTPSVC(6.0.3790.4675); Thu, 27 Oct 2011 10:29:39 +0400 Message-ID: <4EA8FA40.7010504@hotplug.ru> Date: Thu, 27 Oct 2011 10:29:20 +0400 From: Emil Muratov User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.23) Gecko/20110922 Thunderbird/3.1.15 MIME-Version: 1.0 To: Hooman Fazaeli References: <4E8F157A.40702@sentex.net> <4E8F51D4.1060509@sentex.net> <4EA7E203.3020306@sepehrs.com> <4EA80818.3030504@sentex.net> <4EA80F88.4000400@hotplug.ru> <4EA82715.2000404@gmail.com> In-Reply-To: <4EA82715.2000404@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 27 Oct 2011 06:29:39.0035 (UTC) FILETIME=[CDB5A2B0:01CC9471] Cc: freebsd-net@freebsd.org, Jason Wolfe Subject: Re: Intel 82574L interface wedging on em 7.1.9/7.2.3 when MSIX enabled X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Oct 2011 06:30:21 -0000 > Hi, > > Can yan you pls post the output of these command _when_ the problem > happens? > > uname -a > sysctl dev.em > netstat -ind > ifconfig > Hi Hooman Here is what I've got when the script triggered just in time when the interface was locked 11.10.26-23:39:10 ... interface em0 is down... FreeBSD ion.hotplug.ru 8.2-STABLE FreeBSD 8.2-STABLE #0: Thu Oct 20 20:20:25 MSD 2011 root@epia.home .lan:/usr/obj/usr/src/sys/ION6debug amd64 11:39PM up 1:12, 2 users, load averages: 0.26, 0.48, 0.58 == vmstat -i == interrupt total rate irq22: nfe0 16644480 3865 cpu0: timer 8610122 1999 irq256: ahci0 606705 140 irq257: em0:rx 0 3896622 904 irq258: em0:tx 0 2762957 641 irq259: em0:link 620 0 cpu3: timer 8609499 1999 cpu1: timer 8609499 1999 cpu2: timer 8609499 1999 Total 58350003 13550 == netstat -ind == Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll Drop usbus 0 0 0 0 0 0 0 0 usbus 0 0 0 0 0 0 0 0 nfe0 1500 00:25:22:21:86:89 7157140 0 0 12266747 0 0 0 nfe0 1500 fe80::225:22f fe80::225:22ff:fe 0 - - 85 - - - nfe0 1500 10.16.128.0/1 10.16.189.71 0 - - 48135 - - - em0 9000 00:1b:21:ab:bf:4a 5465087 623 0 2862028 0 0 113 em0 9000 192.168.168.0 192.168.168.1 764085 - - 1005078 - - - em0 9000 fe80::21b:21f fe80::21b:21ff:fe 45 - - 252 - - - em0 9000 2002:d58d:871 2002:d58d:8715:1: 73 - - 38 - - - wifi 1500 00:1b:21:ab:bf:4a 347 0 0 350 0 0 0 wifi 1500 192.168.168.6 192.168.168.65 0 - - 0 - - - wifi 1500 fe80::225:x fe80::225:x:x 0 - - 349 - - - wifi 1500 2002:x:x 2002:x:x:2: 0 - - 0 - - - wifio 1500 00:1b:21:ab:bf:4a 59559 0 0 114639 0 0 0 wifio 1500 192.168.168.8 192.168.168.81 0 - - 160 - - - wifio 1500 fe80::225:x fe80::225:x:x 0 - - 0 - - - stf0 1280 5725 0 0 6125 420 0 0 stf0 1280 2002:x:x 2002:x:x::1 1878 - - 1121 - - - ng0* 1500 0 0 0 0 0 0 0 ng1* 1500 0 0 0 0 0 0 0 ng2 1492 7143733 0 0 12234436 0 0 0 ng2 1492 213.141.x.x 213.141.x.x 4735932 - - 8480089 - - - ng2 1492 fe80::x:x fe80::x:x:x 0 - - 1 - - - tun0 1455 350 0 0 172 0 0 0 tun0 1455 fe80::225:x fe80::225:x:x 0 - - 2 - - - tun0 1455 192.168.169.1 192.168.169.1 117 - - 167 - - - Oct 26 23:39:11 ion kernel: em0: hw tdh = 975, hw tdt = 944 Oct 26 23:39:11 ion kernel: em0: hw rdh = 960, hw rdt = 959 Oct 26 23:39:11 ion kernel: em0: Tx Queue Status = 1 Oct 26 23:39:11 ion kernel: em0: TX descriptors avail = 31 Oct 26 23:39:11 ion kernel: em0: Tx Descriptors avail failure = 0 Oct 26 23:39:11 ion kernel: em0: RX discarded packets = 0 Oct 26 23:39:11 ion kernel: em0: RX Next to Check = 960 Oct 26 23:39:11 ion kernel: em0: RX Next to Refresh = 959 net.inet.ip.intr_queue_maxlen: 4096 net.inet.ip.intr_queue_drops: 0 dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.2.3 dev.em.0.%driver: em dev.em.0.%location: slot=0 function=0 dev.em.0.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0xa01f class=0x020000 dev.em.0.%parent: pci2 dev.em.0.nvm: -1 dev.em.0.debug: -1 dev.em.0.rx_int_delay: 200 dev.em.0.tx_int_delay: 200 dev.em.0.rx_abs_int_delay: 4096 dev.em.0.tx_abs_int_delay: 4096 dev.em.0.rx_processing_limit: 100 dev.em.0.flow_control: 3 dev.em.0.eee_control: 0 dev.em.0.link_irq: 648 dev.em.0.mbuf_alloc_fail: 0 dev.em.0.cluster_alloc_fail: 0 dev.em.0.dropped: 0 dev.em.0.tx_dma_fail: 0 dev.em.0.rx_overruns: 0 dev.em.0.watchdog_timeouts: 0 dev.em.0.device_control: 1477444168 dev.em.0.rx_control: 100827170 dev.em.0.fc_high_water: 11264 dev.em.0.fc_low_water: 9764 dev.em.0.queue0.txd_head: 975 dev.em.0.queue0.txd_tail: 944 dev.em.0.queue0.tx_irq: 2762762 dev.em.0.queue0.no_desc_avail: 0 dev.em.0.queue0.rxd_head: 960 dev.em.0.queue0.rxd_tail: 959 dev.em.0.queue0.rx_irq: 3895860 dev.em.0.mac_stats.excess_coll: 0 dev.em.0.mac_stats.single_coll: 0 dev.em.0.mac_stats.multiple_coll: 0 dev.em.0.mac_stats.late_coll: 0 dev.em.0.mac_stats.collision_count: 0 dev.em.0.mac_stats.symbol_errors: 0 dev.em.0.mac_stats.sequence_errors: 0 dev.em.0.mac_stats.defer_count: 0 dev.em.0.mac_stats.missed_packets: 647 dev.em.0.mac_stats.recv_no_buff: 0 dev.em.0.mac_stats.recv_undersize: 0 dev.em.0.mac_stats.recv_fragmented: 0 dev.em.0.mac_stats.recv_fragmented: 0 dev.em.0.mac_stats.recv_oversize: 0 dev.em.0.mac_stats.recv_jabber: 0 dev.em.0.mac_stats.recv_errs: 0 dev.em.0.mac_stats.crc_errs: 0 dev.em.0.mac_stats.alignment_errs: 0 dev.em.0.mac_stats.coll_ext_errs: 0 dev.em.0.mac_stats.xon_recvd: 438789 dev.em.0.mac_stats.xon_txd: 366 dev.em.0.mac_stats.xoff_recvd: 438789 dev.em.0.mac_stats.xoff_txd: 1013 dev.em.0.mac_stats.total_pkts_recvd: 5465524 dev.em.0.mac_stats.good_pkts_recvd: 4587299 dev.em.0.mac_stats.bcast_pkts_recvd: 1102 dev.em.0.mac_stats.mcast_pkts_recvd: 162 dev.em.0.mac_stats.rx_frames_64: 325765 dev.em.0.mac_stats.rx_frames_65_127: 1029229 dev.em.0.mac_stats.rx_frames_128_255: 118432 dev.em.0.mac_stats.rx_frames_256_511: 11360 dev.em.0.mac_stats.rx_frames_512_1023: 100708 dev.em.0.mac_stats.rx_frames_1024_1522: 3001805 dev.em.0.mac_stats.good_octets_recvd: 4648591681 dev.em.0.mac_stats.good_octets_txd: 2203060494 dev.em.0.mac_stats.total_pkts_txd: 3780652 dev.em.0.mac_stats.good_pkts_txd: 3779273 dev.em.0.mac_stats.bcast_pkts_txd: 89 dev.em.0.mac_stats.mcast_pkts_txd: 534 dev.em.0.mac_stats.tx_frames_64: 1323163 dev.em.0.mac_stats.tx_frames_65_127: 850801 dev.em.0.mac_stats.tx_frames_128_255: 193136 dev.em.0.mac_stats.tx_frames_256_511: 64088 dev.em.0.mac_stats.tx_frames_512_1023: 47149 dev.em.0.mac_stats.tx_frames_1024_1522: 1300936 dev.em.0.mac_stats.tso_txd: 429804 dev.em.0.mac_stats.tso_ctx_fail: 0 dev.em.0.interrupts.asserts: 44 dev.em.0.interrupts.rx_pkt_timer: 0 dev.em.0.interrupts.rx_abs_timer: 0 dev.em.0.interrupts.tx_pkt_timer: 0 dev.em.0.interrupts.tx_abs_timer: 0 dev.em.0.interrupts.tx_queue_empty: 0 dev.em.0.interrupts.tx_queue_min_thresh: 0 dev.em.0.interrupts.rx_desc_min_thresh: 0 dev.em.0.interrupts.rx_overrun: 0 ifconfig em0 em0: flags=8c43 metric 0 mtu 9000 description: LAN options=219b ether 00:1b:21:ab:bf:4a inet 192.168.168.1 netmask 0xffffffc0 broadcast 192.168.168.63 inet6 fe80::21b:21ff:feab:bf4a%em0 prefixlen 64 scopeid 0x4 inet6 2002:x:x:1::1 prefixlen 64 nd6 options=1 media: Ethernet autoselect (1000baseT ) status: active > >> I've got almost the same problem with intel 82574L based nic. My >> platform is nvidia ion running Atom 1.6 and nic is an external >> PCI-express adapter. Unlike Jason's case mine is always stuck in >> receiving traffic, it's Ierrs increasing while Ipkts not. Thanks to >> Jason's script I can see those locks and interface flapping every >> several hours. My system is not a heavy loaded server but just a home >> nas/router, usually routing at 100 mbps or less. Nither disabling >> MSIX nor tuning txd rxd doesn't help me. >>