Date: Mon, 31 Oct 2011 12:13:48 +0400 From: Emil Muratov <gpm@hotplug.ru> To: Hooman Fazaeli <hoomanfazaeli@gmail.com> Cc: freebsd-net@freebsd.org, Jack Vogel <jfvogel@gmail.com>, Jason Wolfe <nitroboost@gmail.com> Subject: Re: Intel 82574L interface wedging on em 7.1.9/7.2.3 when MSIX enabled Message-ID: <4EAE58BC.2090000@hotplug.ru> In-Reply-To: <4EA91836.2040508@gmail.com> References: <CAAAm0r0RXEJo4UiKS=Ui0e5OQTg6sg-xcYf3mYB5%2Bvk8i8557w@mail.gmail.com> <4E8F157A.40702@sentex.net> <CAAAm0r2JH43Rct7UxQK2duH1p43Nepnj5mpb6bXo==DPayhJLg@mail.gmail.com> <4E8F51D4.1060509@sentex.net> <CACqU3MVwLaepFymZJkaVk6p=SpykGhqs=VYFjLh9fP9S=AxDhg@mail.gmail.com> <CAAAm0r1DKvoL9=Ket9up=4%2B5xiCzTTZJK99FhF9jcCA28B0M%2BA@mail.gmail.com> <CAAAm0r3XdsMHZh%2BP_NF-txZasdExzwZ8ymmGQgGhJQds0fOiBQ@mail.gmail.com> <CAAAm0r1iS3z-7CBJ=xYDf%2BJOA1Q2nU0O54Twbyb7FjvgWHjKVw@mail.gmail.com> <4EA7E203.3020306@sepehrs.com> <CAAAm0r3Nr2t8cCetPkFnLQ-3KwqHw_0SpqbtvYPRUkSP=9n8CA@mail.gmail.com> <4EA80818.3030504@sentex.net> <4EA80F88.4000400@hotplug.ru> <4EA82715.2000404@gmail.com> <4EA8FA40.7010504@hotplug.ru> <4EA91836.2040508@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
>
> You may try these settings and see if they help:
>
> - hw.em.fc_setting=0 (in /boot/loader.conf)
> - hw.em.rxd="4096" (in /boot/loader.conf)
> - hw.em.txd="4096" (in /boot/loader.conf)
> - Fix speed and duplex at both link sides. After doing that, confirm
> on the freebsd
> box (with ifconfig) and the other device (with whatever command it
> provides) that
> the same speed and duplex is used by both devices.
>
> you also have high values for dev.em.0.rx/tx_[abs]_int_delay. If you
> have set them manually, remove them or replace them with these in
> loader.conf:
>
> hw.em.rx_int_delay=0
> hw.em.tx_int_delay=66
> hw.em.tx_abs_int_delay=66
> hw.em.rx_abs_int_delay=66
>
> these may be set via corresponding sysctls too.
>
Still no luck with the above settings, I've got another lockups a couple
of times. Here is the recent details
=================================
11.10.30-23:43:06 ... interface em0 is down...
we have Ierrs and no ingoing packets for 5 secs, interface em0 must be
toggled
11:43PM up 1 day, 3:01, 2 users, load averages: 0.76, 0.64, 0.70
== vmstat -i ==
interrupt total rate
irq18: ehci0 1145540 11
irq22: nfe0 473895599 4872
cpu0: timer 195004026 2005
irq256: ahci0 12832958 131
irq257: em0:rx 0 95571051 982
irq258: em0:tx 0 88777545 912
irq259: em0:link 946 0
cpu3: timer 195003397 2005
cpu1: timer 195003398 2005
cpu2: timer 195003399 2005
Total 1452237859 14932
== netstat -m ==
5424/1701/7125 mbufs in use (current/cache/total)
719/1185/1904/51200 mbuf clusters in use (current/cache/total/max)
719/582 mbuf+clusters out of packet secondary zone in use (current/cache)
329/583/912/12800 4k (page size) jumbo clusters in use
(current/cache/total/max)
4095/342/4437/12800 9k jumbo clusters in use (current/cache/total/max)
0/0/0/3200 16k jumbo clusters in use (current/cache/total/max)
40978K/8205K/49183K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/6663503/0 requests for jumbo clusters denied (4k/9k/16k)
0/0/0 sfbufs in use (current/peak/max)
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile
0 calls to protocol drain routines
== netstat -ind ==
Name Mtu Network Address Ipkts Ierrs Idrop
Opkts Oerrs Coll Drop
usbus 0 <Link#1> 0 0 0
0 0 0 0
usbus 0 <Link#2> 0 0 0
0 0 0 0
nfe0 1500 <Link#3> 00:25:22:21:86:89 196018201 0 0
350650768 0 0 664
nfe0 1500 fe80::225:22f fe80::225:22ff:fe 0 - -
0 - - -
nfe0 1500 10.16.128.0/1 10.16.189.71 6 - -
29787707 - - -
em0 9000 <Link#4> 00:1b:21:ab:bf:4a 175676617 949 0
101627139 0 0 0
em0 9000 192.168.168.0 192.168.168.1 7628423 - -
13654747 - - -
em0 9000 fe80::21b:21f fe80::21b:21ff:fe 45 - -
5747 - - -
em0 9000 2002:d5xx:xxx 2002:d5xx:xxxx:x: 153 - -
159 - - -
Oct 30 23:43:06 ion kernel: Interface is RUNNING and INACTIVE
Oct 30 23:43:07 ion kernel: em0: hw tdh = 2656, hw tdt = 3271
Oct 30 23:43:07 ion kernel: em0: hw rdh = 2112, hw rdt = 2111
Oct 30 23:43:07 ion kernel: em0: Tx Queue Status = 1
Oct 30 23:43:07 ion kernel: em0: TX descriptors avail = 3481
Oct 30 23:43:07 ion kernel: em0: Tx Descriptors avail failure = 0
Oct 30 23:43:07 ion kernel: em0: RX discarded packets = 0
Oct 30 23:43:07 ion kernel: em0: RX Next to Check = 2112
Oct 30 23:43:07 ion kernel: em0: RX Next to Refresh = 2111
net.inet.ip.intr_queue_maxlen: 4096
net.inet.ip.intr_queue_drops: 0
dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.2.3
dev.em.0.%driver: em
dev.em.0.%location: slot=0 function=0
dev.em.0.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086
subdevice=0xa01f class=0x020000
dev.em.0.%parent: pci2
dev.em.0.nvm: -1
dev.em.0.debug: -1
dev.em.0.rx_int_delay: 0
dev.em.0.tx_int_delay: 66
dev.em.0.rx_abs_int_delay: 66
dev.em.0.tx_abs_int_delay: 66
dev.em.0.rx_processing_limit: 100
dev.em.0.flow_control: 0
dev.em.0.eee_control: 0
dev.em.0.link_irq: 956
dev.em.0.mbuf_alloc_fail: 0
dev.em.0.cluster_alloc_fail: 0
dev.em.0.dropped: 0
dev.em.0.tx_dma_fail: 1
dev.em.0.rx_overruns: 0
dev.em.0.watchdog_timeouts: 0
dev.em.0.device_control: 1074790984
dev.em.0.rx_control: 100827170
dev.em.0.fc_high_water: 11264
dev.em.0.fc_low_water: 9764
dev.em.0.queue0.txd_head: 2656
dev.em.0.queue0.txd_tail: 3274
dev.em.0.queue0.tx_irq: 88769608
dev.em.0.queue0.no_desc_avail: 0
dev.em.0.queue0.rxd_head: 2112
dev.em.0.queue0.rxd_tail: 2111
dev.em.0.queue0.rx_irq: 95554873
dev.em.0.mac_stats.excess_coll: 0
dev.em.0.mac_stats.single_coll: 0
dev.em.0.mac_stats.multiple_coll: 0
dev.em.0.mac_stats.late_coll: 0
dev.em.0.mac_stats.collision_count: 0
dev.em.0.mac_stats.symbol_errors: 0
dev.em.0.mac_stats.sequence_errors: 0
dev.em.0.mac_stats.defer_count: 0
dev.em.0.mac_stats.missed_packets: 959
dev.em.0.mac_stats.recv_no_buff: 0
dev.em.0.mac_stats.recv_undersize: 0
dev.em.0.mac_stats.recv_fragmented: 0
dev.em.0.mac_stats.recv_oversize: 0
dev.em.0.mac_stats.recv_jabber: 0
dev.em.0.mac_stats.recv_errs: 0
dev.em.0.mac_stats.crc_errs: 0
dev.em.0.mac_stats.alignment_errs: 0
dev.em.0.mac_stats.coll_ext_errs: 0
dev.em.0.mac_stats.xon_recvd: 0
dev.em.0.mac_stats.xon_txd: 0
dev.em.0.mac_stats.xoff_recvd: 0
dev.em.0.mac_stats.xoff_txd: 0
dev.em.0.mac_stats.total_pkts_recvd: 175670049
dev.em.0.mac_stats.good_pkts_recvd: 175669090
dev.em.0.mac_stats.bcast_pkts_recvd: 2928
dev.em.0.mac_stats.mcast_pkts_recvd: 1082
dev.em.0.mac_stats.rx_frames_64: 4539426
dev.em.0.mac_stats.rx_frames_65_127: 10242159
dev.em.0.mac_stats.rx_frames_128_255: 576238
dev.em.0.mac_stats.rx_frames_256_511: 372297
dev.em.0.mac_stats.rx_frames_512_1023: 2208708
dev.em.0.mac_stats.rx_frames_1024_1522: 157730262
dev.em.0.mac_stats.good_octets_recvd: 238712986216
dev.em.0.mac_stats.good_octets_txd: 40512955499
dev.em.0.mac_stats.total_pkts_txd: 108590895
dev.em.0.mac_stats.good_pkts_txd: 108590895
dev.em.0.mac_stats.bcast_pkts_txd: 4604
dev.em.0.mac_stats.mcast_pkts_txd: 13969
dev.em.0.mac_stats.tx_frames_64: 13905445
dev.em.0.mac_stats.tx_frames_65_127: 69702633
dev.em.0.mac_stats.tx_frames_128_255: 1442225
dev.em.0.mac_stats.tx_frames_256_511: 755348
dev.em.0.mac_stats.tx_frames_512_1023: 375982
dev.em.0.mac_stats.tx_frames_1024_1522: 22409262
dev.em.0.mac_stats.tso_txd: 4695644
dev.em.0.mac_stats.tso_ctx_fail: 0
dev.em.0.interrupts.asserts: 36
dev.em.0.interrupts.rx_pkt_timer: 0
dev.em.0.interrupts.rx_abs_timer: 0
dev.em.0.interrupts.tx_pkt_timer: 0
dev.em.0.interrupts.tx_abs_timer: 0
dev.em.0.interrupts.tx_queue_empty: 0
dev.em.0.interrupts.tx_queue_min_thresh: 0
dev.em.0.interrupts.rx_desc_min_thresh: 0
dev.em.0.interrupts.rx_overrun: 0
== ifconfig em0 ==
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
description: LAN
options=219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC>
ether 00:1b:21:ab:bf:4a
inet 192.168.168.1 netmask 0xffffffc0 broadcast 192.168.168.63
inet6 fe80::21b:21ff:feab:bf4a%em0 prefixlen 64 scopeid 0x4
inet6 2002:d5xx:xxxx:1::1 prefixlen 64
nd6 options=1<PERFORMNUD>
media: Ethernet 1000baseT (1000baseT <full-duplex>)
status: active
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4EAE58BC.2090000>