Date: Mon, 8 Sep 2014 22:05:42 +0200 From: "=?iso-8859-2?Q?Ing._Ale=B9_Nechv=E1tal?=" <alesnechvatal@videon-znojmo.cz> To: "'Marcelo Gondim'" <gondim@bsdinfo.com.br>, "'Eric Joyner'" <ricera10@gmail.com> Cc: 'Jack F Vogel' <jfv@freebsd.org>, 'FreeBSD Net' <freebsd-net@freebsd.org>, 'Adrian Chadd' <adrian@freebsd.org> Subject: RE: ixgbe CRITICAL: ECC ERROR!! Please Reboot!! Message-ID: <002001cfcba0$4812c7f0$d83857d0$@videon-znojmo.cz> In-Reply-To: <540DFAE3.8070001@bsdinfo.com.br> References: <5408F23C.2030309@bsdinfo.com.br> <CAJ-VmomOECsnRt98ULijSkj9JmwnaLEnZ1ny7fqazsM3WJDNvA@mail.gmail.com> <54091607.9010100@bsdinfo.com.br> <5409CA44.8070203@bsdinfo.com.br> <CAJ-VmomnzTfHzw2i0C01kmfg1rKLePcTHhZqbPWsAezg%2Be9q0g@mail.gmail.com> <540A1A3E.2040306@bsdinfo.com.br> <540A8200.5010404@bsdinfo.com.br> <CA%2Bb0zg9%2Bhhz6sCzrTAHjhc5VzTbDWdnQ%2BBcfdMPsyTGsPeo3gQ@mail.gmail.com> <540DFAE3.8070001@bsdinfo.com.br>
next in thread | previous in thread | raw e-mail | index | archive | help
I was experiencing similar behaviour on a machine acting as a gateway. It has one X520-SR2, both SPF+ ports used to connet to another machines with the same NICs. One of the NICs suddenly stopped to pass data through and the the ifconfing down up cycle was needed to make it work again, that in better case. In the worse case, the machine rebooted without leaving any message in syslog (it's on a remote site). This happened in intervals of 5 mins. up to 1 hour (I am not sure about the exact correlation between traffic going through and the number of these events, but during traffic peaks, it seems to have happened more often). First I checked the mbufs, but didn't see any problem. Then I tried to disable TSO (on both sides of the links), but without observing any improvement. I was playing with different versions of the kernel, the newer the kernel (I went up to 9.3 STABLE, didn't try 10 or HEAD) the more instability. It seemed, that the most stable was 9.1 PRERELEASE with 2.4.8 driver (at least I didn't see reboots, a cron script could keep the gateway passing data by shutting down and bringing up the interfaces). Now I am running 9.3 RELEASE r271227M with the 2.5.25 driver from Intel's website. This configuration was not stable either, until I discovered high interrupt rate on each queue (the sum on all queues reached over 400k in some moments). I turned off the limit of interrupts by setting hw.intr_storm_threshold=0 (previously set to 150000). Since the following reboot (involuntary, btw.) the gateway keeps running smoothly for 7 hours now with peak throughput of approx. 1.6 Gbit/s. The sum of interrupt rates on all your queues looks quite high too, so there could be some similarity and disabling the mentioned limit could help. My sysctl.conf: kern.ipc.somaxconn=1024 net.inet.icmp.icmplim=100 net.inet.tcp.blackhole=2 net.inet.udp.blackhole=1 net.inet.tcp.sendspace=262144 net.inet.tcp.recvspace=262144 net.inet.ip.fastforwarding=1 kern.ipc.maxsockbuf=16777216 net.inet.tcp.sendbuf_max=16777216 net.inet.tcp.recvbuf_max=16777216 hw.intr_storm_threshold=0 #sysctl -a dev.ix.1 dev.ix.1.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 2.5.25 dev.ix.1.%driver: ix dev.ix.1.%location: slot=0 function=1 dev.ix.1.%pnpinfo: vendor=0x8086 device=0x10fb subvendor=0x8086 subdevice=0x7a11 class=0x020000 dev.ix.1.%parent: pci1 dev.ix.1.fc: 3 dev.ix.1.enable_aim: 1 dev.ix.1.advertise_speed: 0 dev.ix.1.dropped: 0 dev.ix.1.mbuf_defrag_failed: 0 dev.ix.1.watchdog_events: 0 dev.ix.1.link_irq: 2 dev.ix.1.queue0.interrupt_rate: 31250 dev.ix.1.queue0.irqs: 152182132 dev.ix.1.queue0.txd_head: 526 dev.ix.1.queue0.txd_tail: 526 dev.ix.1.queue0.tso_tx: 2 dev.ix.1.queue0.no_tx_dma_setup: 0 dev.ix.1.queue0.no_desc_avail: 119373 dev.ix.1.queue0.tx_packets: 274418209 dev.ix.1.queue0.rxd_head: 441 dev.ix.1.queue0.rxd_tail: 440 dev.ix.1.queue0.rx_packets: 103852222 dev.ix.1.queue0.rx_bytes: 25483398737 dev.ix.1.queue0.rx_copies: 87237192 dev.ix.1.queue0.lro_queued: 0 dev.ix.1.queue0.lro_flushed: 0 dev.ix.1.queue1.interrupt_rate: 31250 dev.ix.1.queue1.irqs: 152973172 dev.ix.1.queue1.txd_head: 1282 dev.ix.1.queue1.txd_tail: 1282 dev.ix.1.queue1.tso_tx: 50 dev.ix.1.queue1.no_tx_dma_setup: 0 dev.ix.1.queue1.no_desc_avail: 114840 dev.ix.1.queue1.tx_packets: 271827311 dev.ix.1.queue1.rxd_head: 495 dev.ix.1.queue1.rxd_tail: 494 dev.ix.1.queue1.rx_packets: 103292003 dev.ix.1.queue1.rx_bytes: 20681070454 dev.ix.1.queue1.rx_copies: 89490795 dev.ix.1.queue1.lro_queued: 0 dev.ix.1.queue1.lro_flushed: 0 dev.ix.1.queue2.interrupt_rate: 21739 dev.ix.1.queue2.irqs: 81502157 dev.ix.1.queue2.txd_head: 313 dev.ix.1.queue2.txd_tail: 313 dev.ix.1.queue2.tso_tx: 50 dev.ix.1.queue2.no_tx_dma_setup: 0 dev.ix.1.queue2.no_desc_avail: 111923 dev.ix.1.queue2.tx_packets: 23189156 dev.ix.1.queue2.rxd_head: 349 dev.ix.1.queue2.rxd_tail: 348 dev.ix.1.queue2.rx_packets: 94457797 dev.ix.1.queue2.rx_bytes: 18053240855 dev.ix.1.queue2.rx_copies: 82430878 dev.ix.1.queue2.lro_queued: 0 dev.ix.1.queue2.lro_flushed: 0 dev.ix.1.queue3.interrupt_rate: 83333 dev.ix.1.queue3.irqs: 81678068 dev.ix.1.queue3.txd_head: 1990 dev.ix.1.queue3.txd_tail: 1990 dev.ix.1.queue3.tso_tx: 3 dev.ix.1.queue3.no_tx_dma_setup: 0 dev.ix.1.queue3.no_desc_avail: 110545 dev.ix.1.queue3.tx_packets: 21258249 dev.ix.1.queue3.rxd_head: 1544 dev.ix.1.queue3.rxd_tail: 1543 dev.ix.1.queue3.rx_packets: 99140619 dev.ix.1.queue3.rx_bytes: 19476001717 dev.ix.1.queue3.rx_copies: 86228489 dev.ix.1.queue3.lro_queued: 0 dev.ix.1.queue3.lro_flushed: 0 dev.ix.1.mac_stats.crc_errs: 0 dev.ix.1.mac_stats.ill_errs: 0 dev.ix.1.mac_stats.byte_errs: 0 dev.ix.1.mac_stats.short_discards: 0 dev.ix.1.mac_stats.local_faults: 0 dev.ix.1.mac_stats.remote_faults: 2 dev.ix.1.mac_stats.rec_len_errs: 0 dev.ix.1.mac_stats.xon_txd: 0 dev.ix.1.mac_stats.xon_recvd: 0 dev.ix.1.mac_stats.xoff_txd: 0 dev.ix.1.mac_stats.xoff_recvd: 0 dev.ix.1.mac_stats.total_octets_rcvd: 85345731961 dev.ix.1.mac_stats.good_octets_rcvd: 85345731961 dev.ix.1.mac_stats.total_pkts_rcvd: 400734734 dev.ix.1.mac_stats.good_pkts_rcvd: 400734734 dev.ix.1.mac_stats.mcast_pkts_rcvd: 0 dev.ix.1.mac_stats.bcast_pkts_rcvd: 0 dev.ix.1.mac_stats.rx_frames_64: 190985269 dev.ix.1.mac_stats.rx_frames_65_127: 142879679 dev.ix.1.mac_stats.rx_frames_128_255: 18114498 dev.ix.1.mac_stats.rx_frames_256_511: 5882972 dev.ix.1.mac_stats.rx_frames_512_1023: 7407080 dev.ix.1.mac_stats.rx_frames_1024_1522: 35465236 dev.ix.1.mac_stats.recv_undersized: 0 dev.ix.1.mac_stats.recv_fragmented: 0 dev.ix.1.mac_stats.recv_oversized: 0 dev.ix.1.mac_stats.recv_jabberd: 0 dev.ix.1.mac_stats.management_pkts_rcvd: 0 dev.ix.1.mac_stats.management_pkts_drpd: 0 dev.ix.1.mac_stats.checksum_errs: 462621 dev.ix.1.mac_stats.good_octets_txd: 705061464268 dev.ix.1.mac_stats.total_pkts_txd: 590658400 dev.ix.1.mac_stats.good_pkts_txd: 590658400 dev.ix.1.mac_stats.bcast_pkts_txd: 2907 dev.ix.1.mac_stats.mcast_pkts_txd: 0 dev.ix.1.mac_stats.management_pkts_txd: 0 dev.ix.1.mac_stats.tx_frames_64: 24533923 dev.ix.1.mac_stats.tx_frames_65_127: 65221733 dev.ix.1.mac_stats.tx_frames_128_255: 24274012 dev.ix.1.mac_stats.tx_frames_256_511: 12092320 dev.ix.1.mac_stats.tx_frames_512_1023: 10358523 dev.ix.1.mac_stats.tx_frames_1024_1522: 454177889 > -----Original Message----- > From: owner-freebsd-net@freebsd.org [mailto:owner-freebsd- > net@freebsd.org] On Behalf Of Marcelo Gondim > Sent: Monday, September 08, 2014 8:52 PM > To: Eric Joyner > Cc: Jack F Vogel; FreeBSD Net; Adrian Chadd > Subject: Re: ixgbe CRITICAL: ECC ERROR!! Please Reboot!! > > On 08/09/2014 15:38, Eric Joyner wrote: > > Getting local / remote faults is strange; are these stats from after you > > rebooted? > > When the crash happens I just do: > > # ifconfig ix0 down; ifconfig ix0 up; ifconfig ix1 down; ifconfig up ixi1 > > After that the two interface ports return to work. > > In no time I had to restart the server. > > # uptime > 3:51PM up 11 days, 15:44, 2 users, load averages: 5.36, 5.68, 5.69 > > Tomorrow will be going to the Datacenter to check on the temperature of > the network interface. As the crashes occur when traffic is high, I > believe that the temperature increases because processing is high. > > > > > > --- > > - Eric Joyner > > > > On Fri, Sep 5, 2014 at 8:39 PM, Marcelo Gondim <gondim@bsdinfo.com.br> > > wrote: > > > >> On 05/09/2014 17:17, Marcelo Gondim wrote: > >> > >>> On 05/09/2014 16:49, Adrian Chadd wrote: > >>> > >>>> Hi, > >>>> > >>>> But is the airflow in the unit sufficient? > >>>> > >>>> I had this problem at a previous job - the box was running fine, the > >>>> room was very cold, but the internal fans in the server were set to > >>>> "be very quiet". It wasn't enough to keep the ixgbe NICs happy. I had > >>>> to change the fan settings to "just always run full speed". > >>>> > >>>> The fan temperature feedback loop was based on sensors on the CPU, > >>>> _not_ on the peripherals. > >>>> > >>> Hi Adrian, > >>> > >>> Ummm. I'll check it and improve internal cooling. :) > >>> She is not happy and I'm also not. rsrsrsr > >>> > >>> Cheers, > >>> > >> Besides the problem of heating of the network interface, I am putting > some > >> information here. Could you tell me if there is something strange or is it > >> normal? > >> > >> dev.ix.0.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version > - > >> 2.5.15 > >> dev.ix.0.%driver: ix > >> dev.ix.0.%location: slot=0 function=0 handle=\_SB_.PCI1.BR48.S3F0 > >> dev.ix.0.%pnpinfo: vendor=0x8086 device=0x154d subvendor=0x8086 > >> subdevice=0x7b11 class=0x020000 > >> dev.ix.0.%parent: pci131 > >> dev.ix.0.fc: 3 > >> dev.ix.0.enable_aim: 1 > >> dev.ix.0.advertise_speed: 0 > >> dev.ix.0.dropped: 0 > >> dev.ix.0.mbuf_defrag_failed: 0 > >> dev.ix.0.watchdog_events: 0 > >> dev.ix.0.link_irq: 121769 > >> dev.ix.0.queue0.interrupt_rate: 5319 > >> dev.ix.0.queue0.irqs: 7900830877 > >> dev.ix.0.queue0.txd_head: 1037 > >> dev.ix.0.queue0.txd_tail: 1037 > >> dev.ix.0.queue0.tso_tx: 142 > >> dev.ix.0.queue0.no_tx_dma_setup: 0 > >> dev.ix.0.queue0.no_desc_avail: 0 > >> dev.ix.0.queue0.tx_packets: 9725701450 > >> dev.ix.0.queue0.rxd_head: 1175 > >> dev.ix.0.queue0.rxd_tail: 1174 > >> dev.ix.0.queue0.rx_packets: 13069276955 > >> dev.ix.0.queue0.rx_bytes: 3391061018 > >> dev.ix.0.queue0.rx_copies: 8574407 > >> dev.ix.0.queue0.lro_queued: 0 > >> dev.ix.0.queue0.lro_flushed: 0 > >> dev.ix.0.queue1.interrupt_rate: 41666 > >> dev.ix.0.queue1.irqs: 7681141208 > >> dev.ix.0.queue1.txd_head: 219 > >> dev.ix.0.queue1.txd_tail: 221 > >> dev.ix.0.queue1.tso_tx: 57 > >> dev.ix.0.queue1.no_tx_dma_setup: 0 > >> dev.ix.0.queue1.no_desc_avail: 44334 > >> dev.ix.0.queue1.tx_packets: 10196891433 > >> dev.ix.0.queue1.rxd_head: 1988 > >> dev.ix.0.queue1.rxd_tail: 1987 > >> dev.ix.0.queue1.rx_packets: 13210132242 > >> dev.ix.0.queue1.rx_bytes: 4317357059 > >> dev.ix.0.queue1.rx_copies: 8131936 > >> dev.ix.0.queue1.lro_queued: 0 > >> dev.ix.0.queue1.lro_flushed: 0 > >> dev.ix.0.queue2.interrupt_rate: 5319 > >> dev.ix.0.queue2.irqs: 7647486080 > >> dev.ix.0.queue2.txd_head: 761 > >> dev.ix.0.queue2.txd_tail: 761 > >> dev.ix.0.queue2.tso_tx: 409 > >> dev.ix.0.queue2.no_tx_dma_setup: 0 > >> dev.ix.0.queue2.no_desc_avail: 54207 > >> dev.ix.0.queue2.tx_packets: 10161246425 > >> dev.ix.0.queue2.rxd_head: 1874 > >> dev.ix.0.queue2.rxd_tail: 1872 > >> dev.ix.0.queue2.rx_packets: 13175551880 > >> dev.ix.0.queue2.rx_bytes: 4472798418 > >> dev.ix.0.queue2.rx_copies: 7488876 > >> dev.ix.0.queue2.lro_queued: 0 > >> dev.ix.0.queue2.lro_flushed: 0 > >> dev.ix.0.queue3.interrupt_rate: 500000 > >> dev.ix.0.queue3.irqs: 7641129521 > >> dev.ix.0.queue3.txd_head: 2039 > >> dev.ix.0.queue3.txd_tail: 2039 > >> dev.ix.0.queue3.tso_tx: 9 > >> dev.ix.0.queue3.no_tx_dma_setup: 0 > >> dev.ix.0.queue3.no_desc_avail: 150346 > >> dev.ix.0.queue3.tx_packets: 10619971896 > >> dev.ix.0.queue3.rxd_head: 1055 > >> dev.ix.0.queue3.rxd_tail: 1054 > >> dev.ix.0.queue3.rx_packets: 13137835529 > >> dev.ix.0.queue3.rx_bytes: 4063197306 > >> dev.ix.0.queue3.rx_copies: 8188713 > >> dev.ix.0.queue3.lro_queued: 0 > >> dev.ix.0.queue3.lro_flushed: 0 > >> dev.ix.0.queue4.interrupt_rate: 5319 > >> dev.ix.0.queue4.irqs: 7439824996 > >> dev.ix.0.queue4.txd_head: 26 > >> dev.ix.0.queue4.txd_tail: 26 > >> dev.ix.0.queue4.tso_tx: 553912 > >> dev.ix.0.queue4.no_tx_dma_setup: 0 > >> dev.ix.0.queue4.no_desc_avail: 0 > >> dev.ix.0.queue4.tx_packets: 10658683718 > >> dev.ix.0.queue4.rxd_head: 684 > >> dev.ix.0.queue4.rxd_tail: 681 > >> dev.ix.0.queue4.rx_packets: 13204786830 > >> dev.ix.0.queue4.rx_bytes: 3700845239 > >> dev.ix.0.queue4.rx_copies: 8193379 > >> dev.ix.0.queue4.lro_queued: 0 > >> dev.ix.0.queue4.lro_flushed: 0 > >> dev.ix.0.queue5.interrupt_rate: 15151 > >> dev.ix.0.queue5.irqs: 7456613396 > >> dev.ix.0.queue5.txd_head: 603 > >> dev.ix.0.queue5.txd_tail: 603 > >> dev.ix.0.queue5.tso_tx: 17 > >> dev.ix.0.queue5.no_tx_dma_setup: 0 > >> dev.ix.0.queue5.no_desc_avail: 0 > >> dev.ix.0.queue5.tx_packets: 10639139790 > >> dev.ix.0.queue5.rxd_head: 404 > >> dev.ix.0.queue5.rxd_tail: 403 > >> dev.ix.0.queue5.rx_packets: 13144301293 > >> dev.ix.0.queue5.rx_bytes: 3986784766 > >> dev.ix.0.queue5.rx_copies: 8256195 > >> dev.ix.0.queue5.lro_queued: 0 > >> dev.ix.0.queue5.lro_flushed: 0 > >> dev.ix.0.queue6.interrupt_rate: 125000 > >> dev.ix.0.queue6.irqs: 7466940576 > >> dev.ix.0.queue6.txd_head: 1784 > >> dev.ix.0.queue6.txd_tail: 1784 > >> dev.ix.0.queue6.tso_tx: 2001 > >> dev.ix.0.queue6.no_tx_dma_setup: 0 > >> dev.ix.0.queue6.no_desc_avail: 0 > >> dev.ix.0.queue6.tx_packets: 9784312967 > >> dev.ix.0.queue6.rxd_head: 395 > >> dev.ix.0.queue6.rxd_tail: 394 > >> dev.ix.0.queue6.rx_packets: 13103079970 > >> dev.ix.0.queue6.rx_bytes: 3581485264 > >> dev.ix.0.queue6.rx_copies: 7336569 > >> dev.ix.0.queue6.lro_queued: 0 > >> dev.ix.0.queue6.lro_flushed: 0 > >> dev.ix.0.queue7.interrupt_rate: 5319 > >> dev.ix.0.queue7.irqs: 7486391989 > >> dev.ix.0.queue7.txd_head: 1549 > >> dev.ix.0.queue7.txd_tail: 1549 > >> dev.ix.0.queue7.tso_tx: 2052 > >> dev.ix.0.queue7.no_tx_dma_setup: 0 > >> dev.ix.0.queue7.no_desc_avail: 0 > >> dev.ix.0.queue7.tx_packets: 9758335509 > >> dev.ix.0.queue7.rxd_head: 1990 > >> dev.ix.0.queue7.rxd_tail: 1986 > >> dev.ix.0.queue7.rx_packets: 13141436559 > >> dev.ix.0.queue7.rx_bytes: 3877881546 > >> dev.ix.0.queue7.rx_copies: 8505244 > >> dev.ix.0.queue7.lro_queued: 0 > >> dev.ix.0.queue7.lro_flushed: 0 > >> dev.ix.0.mac_stats.crc_errs: 159 > >> dev.ix.0.mac_stats.ill_errs: 5 > >> dev.ix.0.mac_stats.byte_errs: 10 > >> dev.ix.0.mac_stats.short_discards: 0 > >> dev.ix.0.mac_stats.local_faults: 213 > >> dev.ix.0.mac_stats.remote_faults: 37 > >> dev.ix.0.mac_stats.rec_len_errs: 0 > >> dev.ix.0.mac_stats.xon_txd: 33623392288 > >> dev.ix.0.mac_stats.xon_recvd: 0 > >> dev.ix.0.mac_stats.xoff_txd: 110161 > >> dev.ix.0.mac_stats.xoff_recvd: 0 > >> dev.ix.0.mac_stats.total_octets_rcvd: 37967032023930 > >> dev.ix.0.mac_stats.good_octets_rcvd: 37965112076546 > >> dev.ix.0.mac_stats.total_pkts_rcvd: 105201314224 > >> dev.ix.0.mac_stats.good_pkts_rcvd: 18446743122750685300 > >> dev.ix.0.mac_stats.mcast_pkts_rcvd: 43835 > >> dev.ix.0.mac_stats.bcast_pkts_rcvd: 960566 > >> dev.ix.0.mac_stats.rx_frames_64: 454067 > >> dev.ix.0.mac_stats.rx_frames_65_127: 75350664570 > >> dev.ix.0.mac_stats.rx_frames_128_255: 4726321485 > >> dev.ix.0.mac_stats.rx_frames_256_511: 2282382994 > >> dev.ix.0.mac_stats.rx_frames_512_1023: 3293330145 > >> dev.ix.0.mac_stats.rx_frames_1024_1522: 19541190234 > >> dev.ix.0.mac_stats.recv_undersized: 0 > >> dev.ix.0.mac_stats.recv_fragmented: 0 > >> dev.ix.0.mac_stats.recv_oversized: 0 > >> dev.ix.0.mac_stats.recv_jabberd: 4 > >> dev.ix.0.mac_stats.management_pkts_rcvd: 0 > >> dev.ix.0.mac_stats.management_pkts_drpd: 0 > >> dev.ix.0.mac_stats.checksum_errs: 527716650 > >> dev.ix.0.mac_stats.good_octets_txd: 81355567084088 > >> dev.ix.0.mac_stats.total_pkts_txd: 81545370171 > >> dev.ix.0.mac_stats.good_pkts_txd: 47921867720 > >> dev.ix.0.mac_stats.bcast_pkts_txd: 298646 > >> dev.ix.0.mac_stats.mcast_pkts_txd: 18446744040086169062 > >> dev.ix.0.mac_stats.management_pkts_txd: 0 > >> dev.ix.0.mac_stats.tx_frames_64: 18446744045326220993 > >> dev.ix.0.mac_stats.tx_frames_65_127: 15629697019 > >> dev.ix.0.mac_stats.tx_frames_128_255: 4141756940 > >> dev.ix.0.mac_stats.tx_frames_256_511: 2195917491 > >> dev.ix.0.mac_stats.tx_frames_512_1023: 2717769904 > >> dev.ix.0.mac_stats.tx_frames_1024_1522: 51620056990 > >> > >> dev.ix.1.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version > - > >> 2.5.15 > >> dev.ix.1.%driver: ix > >> dev.ix.1.%location: slot=0 function=1 handle=\_SB_.PCI1.BR48.S3F1 > >> dev.ix.1.%pnpinfo: vendor=0x8086 device=0x154d subvendor=0x8086 > >> subdevice=0x7b11 class=0x020000 > >> dev.ix.1.%parent: pci131 > >> dev.ix.1.fc: 3 > >> dev.ix.1.enable_aim: 1 > >> dev.ix.1.advertise_speed: 0 > >> dev.ix.1.dropped: 0 > >> dev.ix.1.mbuf_defrag_failed: 0 > >> dev.ix.1.watchdog_events: 0 > >> dev.ix.1.link_irq: 46692 > >> dev.ix.1.queue0.interrupt_rate: 15151 > >> dev.ix.1.queue0.irqs: 2634341620 > >> dev.ix.1.queue0.txd_head: 1101 > >> dev.ix.1.queue0.txd_tail: 1101 > >> dev.ix.1.queue0.tso_tx: 0 > >> dev.ix.1.queue0.no_tx_dma_setup: 0 > >> dev.ix.1.queue0.no_desc_avail: 4142 > >> dev.ix.1.queue0.tx_packets: 4749431326 > >> dev.ix.1.queue0.rxd_head: 812 > >> dev.ix.1.queue0.rxd_tail: 811 > >> dev.ix.1.queue0.rx_packets: 21135516 > >> dev.ix.1.queue0.rx_bytes: 33229682 > >> dev.ix.1.queue0.rx_copies: 119434 > >> dev.ix.1.queue0.lro_queued: 0 > >> dev.ix.1.queue0.lro_flushed: 0 > >> dev.ix.1.queue1.interrupt_rate: 15151 > >> dev.ix.1.queue1.irqs: 2616347302 > >> dev.ix.1.queue1.txd_head: 797 > >> dev.ix.1.queue1.txd_tail: 797 > >> dev.ix.1.queue1.tso_tx: 0 > >> dev.ix.1.queue1.no_tx_dma_setup: 0 > >> dev.ix.1.queue1.no_desc_avail: 4129 > >> dev.ix.1.queue1.tx_packets: 4730356550 > >> dev.ix.1.queue1.rxd_head: 1345 > >> dev.ix.1.queue1.rxd_tail: 1344 > >> dev.ix.1.queue1.rx_packets: 20844457 > >> dev.ix.1.queue1.rx_bytes: 35014827 > >> dev.ix.1.queue1.rx_copies: 125167 > >> dev.ix.1.queue1.lro_queued: 0 > >> dev.ix.1.queue1.lro_flushed: 0 > >> dev.ix.1.queue2.interrupt_rate: 5555 > >> dev.ix.1.queue2.irqs: 2603166062 > >> dev.ix.1.queue2.txd_head: 1428 > >> dev.ix.1.queue2.txd_tail: 1432 > >> dev.ix.1.queue2.tso_tx: 0 > >> dev.ix.1.queue2.no_tx_dma_setup: 0 > >> dev.ix.1.queue2.no_desc_avail: 34368 > >> dev.ix.1.queue2.tx_packets: 4681339909 > >> dev.ix.1.queue2.rxd_head: 1153 > >> dev.ix.1.queue2.rxd_tail: 1152 > >> dev.ix.1.queue2.rx_packets: 21263862 > >> dev.ix.1.queue2.rx_bytes: 46251419 > >> dev.ix.1.queue2.rx_copies: 125156 > >> dev.ix.1.queue2.lro_queued: 0 > >> dev.ix.1.queue2.lro_flushed: 0 > >> dev.ix.1.queue3.interrupt_rate: 5319 > >> dev.ix.1.queue3.irqs: 2658483423 > >> dev.ix.1.queue3.txd_head: 179 > >> dev.ix.1.queue3.txd_tail: 179 > >> dev.ix.1.queue3.tso_tx: 0 > >> dev.ix.1.queue3.no_tx_dma_setup: 0 > >> dev.ix.1.queue3.no_desc_avail: 4189 > >> dev.ix.1.queue3.tx_packets: 4811705668 > >> dev.ix.1.queue3.rxd_head: 583 > >> dev.ix.1.queue3.rxd_tail: 582 > >> dev.ix.1.queue3.rx_packets: 20472051 > >> dev.ix.1.queue3.rx_bytes: 29823943 > >> dev.ix.1.queue3.rx_copies: 129450 > >> dev.ix.1.queue3.lro_queued: 0 > >> dev.ix.1.queue3.lro_flushed: 0 > >> dev.ix.1.queue4.interrupt_rate: 5319 > >> dev.ix.1.queue4.irqs: 2629295642 > >> dev.ix.1.queue4.txd_head: 592 > >> dev.ix.1.queue4.txd_tail: 594 > >> dev.ix.1.queue4.tso_tx: 0 > >> dev.ix.1.queue4.no_tx_dma_setup: 0 > >> dev.ix.1.queue4.no_desc_avail: 4106 > >> dev.ix.1.queue4.tx_packets: 4774679355 > >> dev.ix.1.queue4.rxd_head: 259 > >> dev.ix.1.queue4.rxd_tail: 258 > >> dev.ix.1.queue4.rx_packets: 22561633 > >> dev.ix.1.queue4.rx_bytes: 40196457 > >> dev.ix.1.queue4.rx_copies: 135966 > >> dev.ix.1.queue4.lro_queued: 0 > >> dev.ix.1.queue4.lro_flushed: 0 > >> dev.ix.1.queue5.interrupt_rate: 18518 > >> dev.ix.1.queue5.irqs: 2628129098 > >> dev.ix.1.queue5.txd_head: 1070 > >> dev.ix.1.queue5.txd_tail: 1072 > >> dev.ix.1.queue5.tso_tx: 0 > >> dev.ix.1.queue5.no_tx_dma_setup: 0 > >> dev.ix.1.queue5.no_desc_avail: 4127 > >> dev.ix.1.queue5.tx_packets: 4750613588 > >> dev.ix.1.queue5.rxd_head: 595 > >> dev.ix.1.queue5.rxd_tail: 594 > >> dev.ix.1.queue5.rx_packets: 21309940 > >> dev.ix.1.queue5.rx_bytes: 29264004 > >> dev.ix.1.queue5.rx_copies: 134128 > >> dev.ix.1.queue5.lro_queued: 0 > >> dev.ix.1.queue5.lro_flushed: 0 > >> dev.ix.1.queue6.interrupt_rate: 125000 > >> dev.ix.1.queue6.irqs: 2615444123 > >> dev.ix.1.queue6.txd_head: 1008 > >> dev.ix.1.queue6.txd_tail: 1008 > >> dev.ix.1.queue6.tso_tx: 0 > >> dev.ix.1.queue6.no_tx_dma_setup: 0 > >> dev.ix.1.queue6.no_desc_avail: 4106 > >> dev.ix.1.queue6.tx_packets: 4703519800 > >> dev.ix.1.queue6.rxd_head: 1209 > >> dev.ix.1.queue6.rxd_tail: 1208 > >> dev.ix.1.queue6.rx_packets: 22783319 > >> dev.ix.1.queue6.rx_bytes: 28091721 > >> dev.ix.1.queue6.rx_copies: 131653 > >> dev.ix.1.queue6.lro_queued: 0 > >> dev.ix.1.queue6.lro_flushed: 0 > >> dev.ix.1.queue7.interrupt_rate: 6024 > >> dev.ix.1.queue7.irqs: 2644640254 > >> dev.ix.1.queue7.txd_head: 1313 > >> dev.ix.1.queue7.txd_tail: 1313 > >> dev.ix.1.queue7.tso_tx: 0 > >> dev.ix.1.queue7.no_tx_dma_setup: 0 > >> dev.ix.1.queue7.no_desc_avail: 4106 > >> dev.ix.1.queue7.tx_packets: 4824584039 > >> dev.ix.1.queue7.rxd_head: 1463 > >> dev.ix.1.queue7.rxd_tail: 1462 > >> dev.ix.1.queue7.rx_packets: 21464418 > >> dev.ix.1.queue7.rx_bytes: 38133966 > >> dev.ix.1.queue7.rx_copies: 163180 > >> dev.ix.1.queue7.lro_queued: 0 > >> dev.ix.1.queue7.lro_flushed: 0 > >> dev.ix.1.mac_stats.crc_errs: 0 > >> dev.ix.1.mac_stats.ill_errs: 0 > >> dev.ix.1.mac_stats.byte_errs: 0 > >> dev.ix.1.mac_stats.short_discards: 0 > >> dev.ix.1.mac_stats.local_faults: 0 > >> dev.ix.1.mac_stats.remote_faults: 2 > >> dev.ix.1.mac_stats.rec_len_errs: 0 > >> dev.ix.1.mac_stats.xon_txd: 11207820571 > >> dev.ix.1.mac_stats.xon_recvd: 0 > >> dev.ix.1.mac_stats.xoff_txd: 3735958948 > >> dev.ix.1.mac_stats.xoff_recvd: 0 > >> dev.ix.1.mac_stats.total_octets_rcvd: 47828274751 > >> dev.ix.1.mac_stats.good_octets_rcvd: 47225376551 > >> dev.ix.1.mac_stats.total_pkts_rcvd: 177931572 > >> dev.ix.1.mac_stats.good_pkts_rcvd: 18446743954331576567 > >> dev.ix.1.mac_stats.mcast_pkts_rcvd: 7410 > >> dev.ix.1.mac_stats.bcast_pkts_rcvd: 8907 > >> dev.ix.1.mac_stats.rx_frames_64: 0 > >> dev.ix.1.mac_stats.rx_frames_65_127: 134619798 > >> dev.ix.1.mac_stats.rx_frames_128_255: 7996096 > >> dev.ix.1.mac_stats.rx_frames_256_511: 3996945 > >> dev.ix.1.mac_stats.rx_frames_512_1023: 5145938 > >> dev.ix.1.mac_stats.rx_frames_1024_1522: 21331154 > >> dev.ix.1.mac_stats.recv_undersized: 0 > >> dev.ix.1.mac_stats.recv_fragmented: 0 > >> dev.ix.1.mac_stats.recv_oversized: 0 > >> dev.ix.1.mac_stats.recv_jabberd: 0 > >> dev.ix.1.mac_stats.management_pkts_rcvd: 0 > >> dev.ix.1.mac_stats.management_pkts_drpd: 0 > >> dev.ix.1.mac_stats.checksum_errs: 219681 > >> dev.ix.1.mac_stats.good_octets_txd: 38874487735713 > >> dev.ix.1.mac_stats.total_pkts_txd: 38026147390 > >> dev.ix.1.mac_stats.good_pkts_txd: 27377335167 > >> dev.ix.1.mac_stats.bcast_pkts_txd: 67411 > >> dev.ix.1.mac_stats.mcast_pkts_txd: 18446744063060778699 > >> dev.ix.1.mac_stats.management_pkts_txd: 0 > >> dev.ix.1.mac_stats.tx_frames_64: 18446744065500352065 > >> dev.ix.1.mac_stats.tx_frames_65_127: 6755049420 > >> dev.ix.1.mac_stats.tx_frames_128_255: 1914523761 > >> dev.ix.1.mac_stats.tx_frames_256_511: 1038553757 > >> dev.ix.1.mac_stats.tx_frames_512_1023: 1240890942 > >> dev.ix.1.mac_stats.tx_frames_1024_1522: 24637516838 > >> > >> > >> Cheers, > >> > >> > >>>> -a > >>>> > >>>> > >>>> On 5 September 2014 07:35, Marcelo Gondim > <gondim@bsdinfo.com.br> wrote: > >>>> > >>>>> Hi Adrian, > >>>>> > >>>>> I confirmed with the support staff of the room where the server is, > >>>>> that the > >>>>> ambient temperature was normal. > >>>>> > >>>>> > >>>>> On 04/09/2014 22:46, Marcelo Gondim wrote: > >>>>> > >>>>>> On 04/09/2014 20:48, Adrian Chadd wrote: > >>>>>> > >>>>>>> Hi, > >>>>>>> > >>>>>>> The only time this has happened to me is because the card > overheated. > >>>>>>> Can you check that? > >>>>>>> > >>>>>> Hi Adrian, > >>>>>> > >>>>>> The room where the equipment is located is very cold but I'll check it > >>>>>> out. > >>>>>> Also seen at the time of the problem, a lot of dropped packets. > >>>>>> > >>>>>> # netstat -idn > >>>>>> ... > >>>>>> ix0 1500 <Link#9> a0:36:9f:2a:6d:ac 18446743423829095869 159 > >>>>>> 750924631703 53285910688 0 0 0 > >>>>>> ix0 - fe80::a236:9f fe80::a236:9fff:f 0 - - 2 > >>>>>> - - - > >>>>>> ix1 1500 <Link#10> a0:36:9f:2a:6d:ae 18446743954328745465 0 > >>>>>> 119550050209 20178077451 0 0 0 > >>>>>> ix1 - fe80::a236:9f fe80::a236:9fff:f 0 - - 1 > >>>>>> - - - > >>>>>> ... > >>>>>> > >>>>>> 119550050209 droped packets on ix1 and 750924631703 droped on ix0 > >>>>>> > >>>>>> Could be interesting I upgrade to10.1-PRERELEASE? > >>>>>> Could there be a problem with the driver? > >>>>>> > >>>>>> Traffic on ix0: 1.4Gbps output / 600Mbps input > >>>>>> Traffic on ix1: 1.2Gbps output > >>>>>> > >>>>>> PPS on ix0: 163Kpps output / 215Kpps input > >>>>>> PPS on ix1: 131Kpps output > >>>>>> > >>>>>> Thanks for your help. > >>>>>> > >>>>>>> > >>>>>>> -a > >>>>>>> > >>>>>>> > >>>>>>> On 4 September 2014 16:14, Marcelo Gondim > <gondim@bsdinfo.com.br> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> Hi All, > >>>>>>>> > >>>>>>>> I have an Intel X520-SR2and today was working when all traffic > >>>>>>>> stopped. > >>>>>>>> I looked in the logs and found this message: > >>>>>>>> > >>>>>>>> Sep 4 18:29:53 rt01 kernel: ix1: > >>>>>>>> Sep 4 18:29:53 rt01 kernel: CRITICAL: ECC ERROR!! Please Reboot!! > >>>>>>>> > >>>>>>>> # uname -a > >>>>>>>> FreeBSD rt01.xxxxx.com.br 10.0-STABLE FreeBSD 10.0-STABLE #10 > >>>>>>>> r267839: > >>>>>>>> Thu > >>>>>>>> Jul 10 15:35:04 BRT 2014 > >>>>>>>> root@rt01.xxxxx.com.br:/usr/obj/usr/src/sys/GONDIM10 amd64 > >>>>>>>> > >>>>>>>> # netstat -m > >>>>>>>> 98324/53476/151800 mbufs in use (current/cache/total) > >>>>>>>> 98301/44951/143252/1014370 mbuf clusters in use > >>>>>>>> (current/cache/total/max) > >>>>>>>> 98301/44897 mbuf+clusters out of packet secondary zone in use > >>>>>>>> (current/cache) > >>>>>>>> 0/421/421/507184 4k (page size) jumbo clusters in use > >>>>>>>> (current/cache/total/max) > >>>>>>>> 0/0/0/150276 9k jumbo clusters in use (current/cache/total/max) > >>>>>>>> 0/0/0/84530 16k jumbo clusters in use (current/cache/total/max) > >>>>>>>> 221183K/104955K/326138K bytes allocated to network > >>>>>>>> (current/cache/total) > >>>>>>>> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > >>>>>>>> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > >>>>>>>> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > >>>>>>>> 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > >>>>>>>> 0 requests for sfbufs denied > >>>>>>>> 0 requests for sfbufs delayed > >>>>>>>> 0 requests for I/O initiated by sendfile > >>>>>>>> > >>>>>>>> Best regards, > >>>>>>>> > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?002001cfcba0$4812c7f0$d83857d0$>