Date: Fri, 2 May 2014 13:49:11 -0700 From: Nick Rogers <ncrogers@gmail.com> To: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: 9/STABLE Panic at netisr_dispatch_src w/ em(4) + PF Message-ID: <CAKOb=YaRz_S5VwF1_UYMz-3HPae-Hg6Sw%2BNKNsKG2DL-JkcOqA@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Hello,
I am hoping someone can help me debug a kernel panic I have been experiencing
on one of my production systems. The system is a PF+ALTQ firewall/gateway with
about 1k simultaneous devices behind it at any given time, pushing no more
than 100Mb/s of traffic. I have obtained a crash dump and tried to debug it
with kgdb, but am still at a loss.
I have completely replaced the hardware to rule out issues with
disk/memory/etc, and it appears to be a kernel issue, likely with e1000 driver
and/or PF.
The panic seems to happen during times of heavier use, but the frequency is
not very predictable (anywhere from a few times a day to a once a week), so I
feel like its some kind of strange traffic pattern that I am unable to
pinpoint.
I am using a slightly custom kernel based on GENERIC, mainly to bring in ALTQ
and some other options.
It may be worth noting that I also have IGB_LEGACY_TX set for the e1000
driver, although I do not believe that should affect em(4).
Hoping someone can be of assistance in debugging this problem. I am willing to
test patches and pull in the e1000 driver from HEAD or newer 9/STABLE if that
is advisable. Unfortunately I cannot try 10-STABLE.
Thanks,
-Nick Rogers
uname -v
FreeBSD 9.2-STABLE #16 r264331M: Thu Apr 10 21:23:18 EDT 2014
root@fbsd_91_amd64_builder.rgnets.com:/usr/obj/usr/src/sys/RGNETS
Here is the kernel conf...
..............................................................................
include GENERIC
ident RGNETS
makeoptions MODULES_OVERRIDE=""
options DEVICE_POLLING
device crypto
device cryptodev
options VLAN_ARRAY
device amdtemp
# PF
device pf
device pflog
options ALTQ
options ALTQ_CBQ # Class Bases Queuing (CBQ)
options ALTQ_RED # Random Early Detection (RED)
options ALTQ_RIO # RED In/Out
options ALTQ_HFSC # Hierarchical Packet Scheduler (HFSC)
options ALTQ_PRIQ # Priority Queuing (PRIQ)
options ALTQ_NOPCC # Required for SMP build
# PPPoE
options NETGRAPH
options NETGRAPH_ETHER
options NETGRAPH_PPPOE
options NETGRAPH_SOCKET
# IPsec
device enc
options IPSEC
options IPSEC_FILTERTUNNEL
options IPSEC_NAT_T
..............................................................................
The crash dump....
..............................................................................
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...
Unread portion of the kernel message buffer:
Fatal trap 12: page fault while in kernel mode
cpuid = 5; apic id = 05
fault virtual address = 0x10
fault code = supervisor read data, page not present
instruction pointer = 0x20:0xffffffff8033d350
stack pointer = 0x28:0xffffff83545384b0
frame pointer = 0x28:0xffffff83545384c0
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 12 (irq262: em2:rx 0)
trap number = 12
panic: page fault
cpuid = 5
KDB: stack backtrace:
#0 0xffffffff80956836 at kdb_backtrace+0x66
#1 0xffffffff8091c40e at panic+0x1ce
#2 0xffffffff80d31e70 at trap_fatal+0x290
#3 0xffffffff80d321d1 at trap_pfault+0x211
#4 0xffffffff80d327d3 at trap+0x363
#5 0xffffffff80d1b9d3 at calltrap+0x8
#6 0xffffffff8034872d at pf_test_rule+0x17ed
#7 0xffffffff8034ba12 at pf_test+0x1032
#8 0xffffffff8035112b at pf_check_in+0x2b
#9 0xffffffff809e952e at pfil_run_hooks+0x9e
#10 0xffffffff80a5286a at ip_input+0x2ea
#11 0xffffffff809e8858 at netisr_dispatch_src+0x218
#12 0xffffffff809df93d at ether_demux+0x14d
#13 0xffffffff809dfc1e at ether_nh_input+0x1fe
#14 0xffffffff809e8858 at netisr_dispatch_src+0x218
#15 0xffffffff809df85f at ether_demux+0x6f
#16 0xffffffff809dfc1e at ether_nh_input+0x1fe
#17 0xffffffff809e8858 at netisr_dispatch_src+0x218
Uptime: 17d7h20m59s
Dumping 2932 out of 12256 MB: (CTRL-C to abort)
..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%
Reading symbols from /boot/kernel/aio.ko...Reading symbols from
/boot/kernel/aio.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/aio.ko
Reading symbols from /boot/kernel/coretemp.ko...Reading symbols from
/boot/kernel/coretemp.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/coretemp.ko
Reading symbols from /boot/kernel/cc_htcp.ko...Reading symbols from
/boot/kernel/cc_htcp.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/cc_htcp.ko
#0 doadump (textdump=Variable "textdump" is not available.
) at pcpu.h:234
234 pcpu.h: No such file or directory.
in pcpu.h
(kgdb) list *0xffffffff8033d350
0xffffffff8033d350 is in pf_addrcpy (/usr/src/sys/contrib/pf/net/pf.c:512).
507 pf_addrcpy(struct pf_addr *dst, struct pf_addr *src, sa_family_t af)
508 {
509 switch (af) {
510 #ifdef INET
511 case AF_INET:
512 dst->addr32[0] = src->addr32[0];
513 break;
514 #endif /* INET */
515 case AF_INET6:
516 dst->addr32[0] = src->addr32[0];
(kgdb) backtrace
#0 doadump (textdump=Variable "textdump" is not available.
) at pcpu.h:234
#1 0xffffffff8091bee6 in kern_reboot (howto=260) at
/usr/src/sys/kern/kern_shutdown.c:454
#2 0xffffffff8091c3e7 in panic (fmt=0x1 <Address 0x1 out of bounds>)
at /usr/src/sys/kern/kern_shutdown.c:642
#3 0xffffffff80d31e70 in trap_fatal (frame=0xc, eva=Variable "eva" is
not available.
) at /usr/src/sys/amd64/amd64/trap.c:878
#4 0xffffffff80d321d1 in trap_pfault (frame=0xffffff8354538400,
usermode=0) at /usr/src/sys/amd64/amd64/trap.c:794
#5 0xffffffff80d327d3 in trap (frame=0xffffff8354538400) at
/usr/src/sys/amd64/amd64/trap.c:456
#6 0xffffffff80d1b9d3 in calltrap () at
/usr/src/sys/amd64/amd64/exception.S:232
#7 0xffffffff8033d350 in pf_addrcpy (dst=0xfffffe010c6416b8,
src=0x10, af=2 '\002') at /usr/src/sys/contrib/pf/net/pf.c:522
#8 0xffffffff8034872d in pf_test_rule (rm=0xffffff8354538788,
sm=0xffffff8354538780, direction=1, kif=0xfffffe0007d08100,
m=0xfffffe0030555d00, off=20, h=0xfffffe0030bad00e,
pd=0xffffff83545386c0, am=0xffffff8354538790,
rsm=0xffffff8354538778, ifq=0x0, inp=0x0) at
/usr/src/sys/contrib/pf/net/pf.c:3900
#9 0xffffffff8034ba12 in pf_test (dir=1, ifp=Variable "ifp" is not available.
) at /usr/src/sys/contrib/pf/net/pf.c:6776
#10 0xffffffff8035112b in pf_check_in (arg=Variable "arg" is not available.
) at /usr/src/sys/contrib/pf/net/pf_ioctl.c:4131
#11 0xffffffff809e952e in pfil_run_hooks (ph=Variable "ph" is not available.
) at /usr/src/sys/net/pfil.c:82
#12 0xffffffff80a5286a in ip_input (m=0xfffffe0030555d00) at
/usr/src/sys/netinet/ip_input.c:510
#13 0xffffffff809e8858 in netisr_dispatch_src (proto=1,
source=Variable "source" is not available.
) at /usr/src/sys/net/netisr.c:1013
#14 0xffffffff809df93d in ether_demux (ifp=0xfffffe0030239000,
m=0xfffffe0030555d00) at /usr/src/sys/net/if_ethersubr.c:943
#15 0xffffffff809dfc1e in ether_nh_input (m=Variable "m" is not available.
) at /usr/src/sys/net/if_ethersubr.c:762
#16 0xffffffff809e8858 in netisr_dispatch_src (proto=9,
source=Variable "source" is not available.
) at /usr/src/sys/net/netisr.c:1013
#17 0xffffffff809df85f in ether_demux (ifp=0xfffffe0003f0c800,
m=0xfffffe0030555d00) at /usr/src/sys/net/if_ethersubr.c:852
#18 0xffffffff809dfc1e in ether_nh_input (m=Variable "m" is not available.
) at /usr/src/sys/net/if_ethersubr.c:762
#19 0xffffffff809e8858 in netisr_dispatch_src (proto=9,
source=Variable "source" is not available.
) at /usr/src/sys/net/netisr.c:1013
#20 0xffffffff804ccb58 in em_rxeof (rxr=0xfffffe0007308200, count=-2,
done=0x0) at /usr/src/sys/dev/e1000/if_em.c:4525
#21 0xffffffff804cceb6 in em_msix_rx (arg=Variable "arg" is not available.
) at /usr/src/sys/dev/e1000/if_em.c:1593
#22 0xffffffff808ecb1d in intr_event_execute_handlers (p=Variable "p"
is not available.
) at /usr/src/sys/kern/kern_intr.c:1272
#23 0xffffffff808ee30d in ithread_loop (arg=0xfffffe000730c300) at
/usr/src/sys/kern/kern_intr.c:1285
#24 0xffffffff808e951f in fork_exit (callout=0xffffffff808ee270
<ithread_loop>, arg=0xfffffe000730c300, frame=0xffffff8354538c40) at
/usr/src/sys/kern/kern_fork.c:996
#25 0xffffffff80d1befe in fork_trampoline () at
/usr/src/sys/amd64/amd64/exception.S:606
#26 0x0000000000000000 in ?? ()
#27 0x0000000000000000 in ?? ()
#28 0x0000000000000001 in ?? ()
#29 0x0000000000000000 in ?? ()
#30 0x0000000000000000 in ?? ()
#31 0x0000000000000000 in ?? ()
#32 0x0000000000000000 in ?? ()
#33 0x0000000000000000 in ?? ()
#34 0x0000000000000000 in ?? ()
#35 0x0000000000000000 in ?? ()
#36 0x0000000000000000 in ?? ()
#37 0x0000000000000000 in ?? ()
#38 0x0000000000000000 in ?? ()
#39 0x0000000000000000 in ?? ()
#40 0x0000000000000000 in ?? ()
#41 0x0000000000000000 in ?? ()
#42 0x0000000000000000 in ?? ()
#43 0x0000000000000000 in ?? ()
#44 0x0000000000000000 in ?? ()
#45 0x0000000000000000 in ?? ()
#46 0x0000000000000000 in ?? ()
#47 0x0000000000000000 in ?? ()
#48 0x0000000000000000 in ?? ()
---Type <return> to continue, or q <return> to quit---
#49 0x0000000000000000 in ?? ()
#50 0x0000000000000000 in ?? ()
#51 0xfffffe0007304920 in ?? ()
#52 0xfffffe0003afd000 in ?? ()
#53 0xfffffe0007304920 in ?? ()
#54 0xffffff8354538b40 in ?? ()
#55 0xffffff8354538ae8 in ?? ()
#56 0xfffffe0003b00920 in ?? ()
#57 0xffffffff80948646 in sched_switch (td=0xffffffff808ee270,
newtd=0xfffffe000730c300, flags=Variable "flags" is not available.
) at /usr/src/sys/kern/sched_ule.c:1898
Previous frame inner to this frame (corrupt stack?)
..............................................................................
Relevant pciconf output...
..............................................................................
em0@pci0:1:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 hdr=0x00
vendor = 'Intel Corporation'
device = '82574L Gigabit Network Connection'
class = network
subclass = ethernet
cap 01[c8] = powerspec 2 supports D0 D3 current D0
cap 05[d0] = MSI supports 1 message, 64 bit
cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1)
cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled
ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected
em1@pci0:2:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 hdr=0x00
vendor = 'Intel Corporation'
device = '82574L Gigabit Network Connection'
class = network
subclass = ethernet
cap 01[c8] = powerspec 2 supports D0 D3 current D0
cap 05[d0] = MSI supports 1 message, 64 bit
cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1)
cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled
ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected
em2@pci0:7:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 hdr=0x00
vendor = 'Intel Corporation'
device = '82574L Gigabit Network Connection'
class = network
subclass = ethernet
cap 01[c8] = powerspec 2 supports D0 D3 current D0
cap 05[d0] = MSI supports 1 message, 64 bit
cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1)
cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled
ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected
em3@pci0:8:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 hdr=0x00
vendor = 'Intel Corporation'
device = '82574L Gigabit Network Connection'
class = network
subclass = ethernet
cap 01[c8] = powerspec 2 supports D0 D3 current D0
cap 05[d0] = MSI supports 1 message, 64 bit
cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1)
cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled
..............................................................................
dev.em sysctl....
..............................................................................
dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
dev.em.0.%driver: em
dev.em.0.%location: slot=0 function=0
dev.em.0.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9
subdevice=0x10d3 class=0x020000
dev.em.0.%parent: pci1
dev.em.0.nvm: -1
dev.em.0.debug: -1
dev.em.0.fc: 3
dev.em.0.rx_int_delay: 0
dev.em.0.tx_int_delay: 66
dev.em.0.rx_abs_int_delay: 66
dev.em.0.tx_abs_int_delay: 66
dev.em.0.itr: 488
dev.em.0.rx_processing_limit: -1
dev.em.0.eee_control: 1
dev.em.0.link_irq: 2
dev.em.0.mbuf_alloc_fail: 0
dev.em.0.cluster_alloc_fail: 0
dev.em.0.dropped: 0
dev.em.0.tx_dma_fail: 0
dev.em.0.rx_overruns: 0
dev.em.0.watchdog_timeouts: 0
dev.em.0.device_control: 1477444168
dev.em.0.rx_control: 67141634
dev.em.0.fc_high_water: 18432
dev.em.0.fc_low_water: 16932
dev.em.0.queue0.txd_head: 3265
dev.em.0.queue0.txd_tail: 3265
dev.em.0.queue0.tx_irq: 81153071
dev.em.0.queue0.no_desc_avail: 0
dev.em.0.queue0.rxd_head: 388
dev.em.0.queue0.rxd_tail: 387
dev.em.0.queue0.rx_irq: 79015024
dev.em.0.mac_stats.excess_coll: 0
dev.em.0.mac_stats.single_coll: 0
dev.em.0.mac_stats.multiple_coll: 0
dev.em.0.mac_stats.late_coll: 0
dev.em.0.mac_stats.collision_count: 0
dev.em.0.mac_stats.symbol_errors: 0
dev.em.0.mac_stats.sequence_errors: 0
dev.em.0.mac_stats.defer_count: 0
dev.em.0.mac_stats.missed_packets: 0
dev.em.0.mac_stats.recv_no_buff: 0
dev.em.0.mac_stats.recv_undersize: 0
dev.em.0.mac_stats.recv_fragmented: 0
dev.em.0.mac_stats.recv_oversize: 0
dev.em.0.mac_stats.recv_jabber: 0
dev.em.0.mac_stats.recv_errs: 0
dev.em.0.mac_stats.crc_errs: 0
dev.em.0.mac_stats.alignment_errs: 0
dev.em.0.mac_stats.coll_ext_errs: 0
dev.em.0.mac_stats.xon_recvd: 6
dev.em.0.mac_stats.xon_txd: 0
dev.em.0.mac_stats.xoff_recvd: 6
dev.em.0.mac_stats.xoff_txd: 0
dev.em.0.mac_stats.total_pkts_recvd: 122072630
dev.em.0.mac_stats.good_pkts_recvd: 122072618
dev.em.0.mac_stats.bcast_pkts_recvd: 257
dev.em.0.mac_stats.mcast_pkts_recvd: 0
dev.em.0.mac_stats.rx_frames_64: 8634
dev.em.0.mac_stats.rx_frames_65_127: 67656673
dev.em.0.mac_stats.rx_frames_128_255: 714152
dev.em.0.mac_stats.rx_frames_256_511: 609615
dev.em.0.mac_stats.rx_frames_512_1023: 8646536
dev.em.0.mac_stats.rx_frames_1024_1522: 44437008
dev.em.0.mac_stats.good_octets_recvd: 82940411216
dev.em.0.mac_stats.good_octets_txd: 25718335997
dev.em.0.mac_stats.total_pkts_txd: 99833592
dev.em.0.mac_stats.good_pkts_txd: 99833592
dev.em.0.mac_stats.bcast_pkts_txd: 13
dev.em.0.mac_stats.mcast_pkts_txd: 0
dev.em.0.mac_stats.tx_frames_64: 2193
dev.em.0.mac_stats.tx_frames_65_127: 29089783
dev.em.0.mac_stats.tx_frames_128_255: 54412030
dev.em.0.mac_stats.tx_frames_256_511: 9565246
dev.em.0.mac_stats.tx_frames_512_1023: 1080398
dev.em.0.mac_stats.tx_frames_1024_1522: 5683942
dev.em.0.mac_stats.tso_txd: 1468623
dev.em.0.mac_stats.tso_ctx_fail: 0
dev.em.0.interrupts.asserts: 2
dev.em.0.interrupts.rx_pkt_timer: 0
dev.em.0.interrupts.rx_abs_timer: 0
dev.em.0.interrupts.tx_pkt_timer: 0
dev.em.0.interrupts.tx_abs_timer: 0
dev.em.0.interrupts.tx_queue_empty: 0
dev.em.0.interrupts.tx_queue_min_thresh: 0
dev.em.0.interrupts.rx_desc_min_thresh: 0
dev.em.0.interrupts.rx_overrun: 0
dev.em.1.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
dev.em.1.%driver: em
dev.em.1.%location: slot=0 function=0
dev.em.1.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9
subdevice=0x10d3 class=0x020000
dev.em.1.%parent: pci2
dev.em.1.nvm: -1
dev.em.1.debug: -1
dev.em.1.fc: 3
dev.em.1.rx_int_delay: 0
dev.em.1.tx_int_delay: 66
dev.em.1.rx_abs_int_delay: 66
dev.em.1.tx_abs_int_delay: 66
dev.em.1.itr: 488
dev.em.1.rx_processing_limit: -1
dev.em.1.eee_control: 1
dev.em.1.link_irq: 2
dev.em.1.mbuf_alloc_fail: 0
dev.em.1.cluster_alloc_fail: 0
dev.em.1.dropped: 0
dev.em.1.tx_dma_fail: 1
dev.em.1.rx_overruns: 0
dev.em.1.watchdog_timeouts: 0
dev.em.1.device_control: 1477444168
dev.em.1.rx_control: 67141634
dev.em.1.fc_high_water: 18432
dev.em.1.fc_low_water: 16932
dev.em.1.queue0.txd_head: 2451
dev.em.1.queue0.txd_tail: 2453
dev.em.1.queue0.tx_irq: 143904807
dev.em.1.queue0.no_desc_avail: 0
dev.em.1.queue0.rxd_head: 342
dev.em.1.queue0.rxd_tail: 341
dev.em.1.queue0.rx_irq: 159303310
dev.em.1.mac_stats.excess_coll: 0
dev.em.1.mac_stats.single_coll: 0
dev.em.1.mac_stats.multiple_coll: 0
dev.em.1.mac_stats.late_coll: 0
dev.em.1.mac_stats.collision_count: 0
dev.em.1.mac_stats.symbol_errors: 0
dev.em.1.mac_stats.sequence_errors: 0
dev.em.1.mac_stats.defer_count: 0
dev.em.1.mac_stats.missed_packets: 0
dev.em.1.mac_stats.recv_no_buff: 0
dev.em.1.mac_stats.recv_undersize: 0
dev.em.1.mac_stats.recv_fragmented: 0
dev.em.1.mac_stats.recv_oversize: 0
dev.em.1.mac_stats.recv_jabber: 0
dev.em.1.mac_stats.recv_errs: 0
dev.em.1.mac_stats.crc_errs: 0
dev.em.1.mac_stats.alignment_errs: 0
dev.em.1.mac_stats.coll_ext_errs: 0
dev.em.1.mac_stats.xon_recvd: 1
dev.em.1.mac_stats.xon_txd: 0
dev.em.1.mac_stats.xoff_recvd: 1
dev.em.1.mac_stats.xoff_txd: 0
dev.em.1.mac_stats.total_pkts_recvd: 331901758
dev.em.1.mac_stats.good_pkts_recvd: 331901756
dev.em.1.mac_stats.bcast_pkts_recvd: 13467
dev.em.1.mac_stats.mcast_pkts_recvd: 0
dev.em.1.mac_stats.rx_frames_64: 13905035
dev.em.1.mac_stats.rx_frames_65_127: 22315178
dev.em.1.mac_stats.rx_frames_128_255: 8343368
dev.em.1.mac_stats.rx_frames_256_511: 8602323
dev.em.1.mac_stats.rx_frames_512_1023: 8170288
dev.em.1.mac_stats.rx_frames_1024_1522: 270565564
dev.em.1.mac_stats.good_octets_recvd: 420794715670
dev.em.1.mac_stats.good_octets_txd: 45361473880
dev.em.1.mac_stats.total_pkts_txd: 217852588
dev.em.1.mac_stats.good_pkts_txd: 217852588
dev.em.1.mac_stats.bcast_pkts_txd: 6
dev.em.1.mac_stats.mcast_pkts_txd: 0
dev.em.1.mac_stats.tx_frames_64: 64102191
dev.em.1.mac_stats.tx_frames_65_127: 120705475
dev.em.1.mac_stats.tx_frames_128_255: 6009336
dev.em.1.mac_stats.tx_frames_256_511: 4593595
dev.em.1.mac_stats.tx_frames_512_1023: 4295623
dev.em.1.mac_stats.tx_frames_1024_1522: 18146368
dev.em.1.mac_stats.tso_txd: 291134
dev.em.1.mac_stats.tso_ctx_fail: 0
dev.em.1.interrupts.asserts: 2
dev.em.1.interrupts.rx_pkt_timer: 0
dev.em.1.interrupts.rx_abs_timer: 0
dev.em.1.interrupts.tx_pkt_timer: 0
dev.em.1.interrupts.tx_abs_timer: 0
dev.em.1.interrupts.tx_queue_empty: 0
dev.em.1.interrupts.tx_queue_min_thresh: 0
dev.em.1.interrupts.rx_desc_min_thresh: 0
dev.em.1.interrupts.rx_overrun: 0
dev.em.2.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
dev.em.2.%driver: em
dev.em.2.%location: slot=0 function=0
dev.em.2.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9
subdevice=0x10d3 class=0x020000
dev.em.2.%parent: pci7
dev.em.2.nvm: -1
dev.em.2.debug: -1
dev.em.2.fc: 3
dev.em.2.rx_int_delay: 0
dev.em.2.tx_int_delay: 66
dev.em.2.rx_abs_int_delay: 66
dev.em.2.tx_abs_int_delay: 66
dev.em.2.itr: 488
dev.em.2.rx_processing_limit: -1
dev.em.2.eee_control: 1
dev.em.2.link_irq: 1
dev.em.2.mbuf_alloc_fail: 0
dev.em.2.cluster_alloc_fail: 0
dev.em.2.dropped: 0
dev.em.2.tx_dma_fail: 6823
dev.em.2.rx_overruns: 0
dev.em.2.watchdog_timeouts: 0
dev.em.2.device_control: 1477444168
dev.em.2.rx_control: 67141634
dev.em.2.fc_high_water: 18432
dev.em.2.fc_low_water: 16932
dev.em.2.queue0.txd_head: 3977
dev.em.2.queue0.txd_tail: 3977
dev.em.2.queue0.tx_irq: 220950699
dev.em.2.queue0.no_desc_avail: 0
dev.em.2.queue0.rxd_head: 83
dev.em.2.queue0.rxd_tail: 82
dev.em.2.queue0.rx_irq: 125920607
dev.em.2.mac_stats.excess_coll: 0
dev.em.2.mac_stats.single_coll: 0
dev.em.2.mac_stats.multiple_coll: 0
dev.em.2.mac_stats.late_coll: 0
dev.em.2.mac_stats.collision_count: 0
dev.em.2.mac_stats.symbol_errors: 0
dev.em.2.mac_stats.sequence_errors: 0
dev.em.2.mac_stats.defer_count: 0
dev.em.2.mac_stats.missed_packets: 0
dev.em.2.mac_stats.recv_no_buff: 0
dev.em.2.mac_stats.recv_undersize: 0
dev.em.2.mac_stats.recv_fragmented: 0
dev.em.2.mac_stats.recv_oversize: 0
dev.em.2.mac_stats.recv_jabber: 0
dev.em.2.mac_stats.recv_errs: 0
dev.em.2.mac_stats.crc_errs: 0
dev.em.2.mac_stats.alignment_errs: 0
dev.em.2.mac_stats.coll_ext_errs: 0
dev.em.2.mac_stats.xon_recvd: 14123
dev.em.2.mac_stats.xon_txd: 1
dev.em.2.mac_stats.xoff_recvd: 14127
dev.em.2.mac_stats.xoff_txd: 1
dev.em.2.mac_stats.total_pkts_recvd: 229919303
dev.em.2.mac_stats.good_pkts_recvd: 229891053
dev.em.2.mac_stats.bcast_pkts_recvd: 909450
dev.em.2.mac_stats.mcast_pkts_recvd: 19452
dev.em.2.mac_stats.rx_frames_64: 1477808
dev.em.2.mac_stats.rx_frames_65_127: 195114744
dev.em.2.mac_stats.rx_frames_128_255: 6579690
dev.em.2.mac_stats.rx_frames_256_511: 5137387
dev.em.2.mac_stats.rx_frames_512_1023: 4223090
dev.em.2.mac_stats.rx_frames_1024_1522: 17358334
dev.em.2.mac_stats.good_octets_recvd: 46129102134
dev.em.2.mac_stats.good_octets_txd: 419293159496
dev.em.2.mac_stats.total_pkts_txd: 332661584
dev.em.2.mac_stats.good_pkts_txd: 332661582
dev.em.2.mac_stats.bcast_pkts_txd: 48506
dev.em.2.mac_stats.mcast_pkts_txd: 78
dev.em.2.mac_stats.tx_frames_64: 14598198
dev.em.2.mac_stats.tx_frames_65_127: 22287108
dev.em.2.mac_stats.tx_frames_128_255: 8897511
dev.em.2.mac_stats.tx_frames_256_511: 9623000
dev.em.2.mac_stats.tx_frames_512_1023: 8325033
dev.em.2.mac_stats.tx_frames_1024_1522: 268930732
dev.em.2.mac_stats.tso_txd: 24357891
dev.em.2.mac_stats.tso_ctx_fail: 0
dev.em.2.interrupts.asserts: 2
dev.em.2.interrupts.rx_pkt_timer: 0
dev.em.2.interrupts.rx_abs_timer: 0
dev.em.2.interrupts.tx_pkt_timer: 0
dev.em.2.interrupts.tx_abs_timer: 0
dev.em.2.interrupts.tx_queue_empty: 0
dev.em.2.interrupts.tx_queue_min_thresh: 0
dev.em.2.interrupts.rx_desc_min_thresh: 0
dev.em.2.interrupts.rx_overrun: 0
dev.em.3.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
dev.em.3.%driver: em
dev.em.3.%location: slot=0 function=0
dev.em.3.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9
subdevice=0x10d3 class=0x020000
dev.em.3.%parent: pci8
dev.em.3.nvm: -1
dev.em.3.debug: -1
dev.em.3.fc: 3
dev.em.3.rx_int_delay: 0
dev.em.3.tx_int_delay: 66
dev.em.3.rx_abs_int_delay: 66
dev.em.3.tx_abs_int_delay: 66
dev.em.3.itr: 488
dev.em.3.rx_processing_limit: -1
dev.em.3.eee_control: 1
dev.em.3.link_irq: 0
dev.em.3.mbuf_alloc_fail: 0
dev.em.3.cluster_alloc_fail: 0
dev.em.3.dropped: 0
dev.em.3.tx_dma_fail: 0
dev.em.3.rx_overruns: 0
dev.em.3.watchdog_timeouts: 0
dev.em.3.device_control: 1074790984
dev.em.3.rx_control: 67141634
dev.em.3.fc_high_water: 18432
dev.em.3.fc_low_water: 16932
dev.em.3.queue0.txd_head: 0
dev.em.3.queue0.txd_tail: 0
dev.em.3.queue0.tx_irq: 0
dev.em.3.queue0.no_desc_avail: 0
dev.em.3.queue0.rxd_head: 0
dev.em.3.queue0.rxd_tail: 4095
dev.em.3.queue0.rx_irq: 0
dev.em.3.mac_stats.excess_coll: 0
dev.em.3.mac_stats.single_coll: 0
dev.em.3.mac_stats.multiple_coll: 0
dev.em.3.mac_stats.late_coll: 0
dev.em.3.mac_stats.collision_count: 0
dev.em.3.mac_stats.symbol_errors: 0
dev.em.3.mac_stats.sequence_errors: 0
dev.em.3.mac_stats.defer_count: 0
dev.em.3.mac_stats.missed_packets: 0
dev.em.3.mac_stats.recv_no_buff: 0
dev.em.3.mac_stats.recv_undersize: 0
dev.em.3.mac_stats.recv_fragmented: 0
dev.em.3.mac_stats.recv_oversize: 0
dev.em.3.mac_stats.recv_jabber: 0
dev.em.3.mac_stats.recv_errs: 0
dev.em.3.mac_stats.crc_errs: 0
dev.em.3.mac_stats.alignment_errs: 0
dev.em.3.mac_stats.coll_ext_errs: 0
dev.em.3.mac_stats.xon_recvd: 0
dev.em.3.mac_stats.xon_txd: 0
dev.em.3.mac_stats.xoff_recvd: 0
dev.em.3.mac_stats.xoff_txd: 0
dev.em.3.mac_stats.total_pkts_recvd: 0
dev.em.3.mac_stats.good_pkts_recvd: 0
dev.em.3.mac_stats.bcast_pkts_recvd: 0
dev.em.3.mac_stats.mcast_pkts_recvd: 0
dev.em.3.mac_stats.rx_frames_64: 0
dev.em.3.mac_stats.rx_frames_65_127: 0
dev.em.3.mac_stats.rx_frames_128_255: 0
dev.em.3.mac_stats.rx_frames_256_511: 0
dev.em.3.mac_stats.rx_frames_512_1023: 0
dev.em.3.mac_stats.rx_frames_1024_1522: 0
dev.em.3.mac_stats.good_octets_recvd: 0
dev.em.3.mac_stats.good_octets_txd: 0
dev.em.3.mac_stats.total_pkts_txd: 0
dev.em.3.mac_stats.good_pkts_txd: 0
dev.em.3.mac_stats.bcast_pkts_txd: 0
dev.em.3.mac_stats.mcast_pkts_txd: 0
dev.em.3.mac_stats.tx_frames_64: 0
dev.em.3.mac_stats.tx_frames_65_127: 0
dev.em.3.mac_stats.tx_frames_128_255: 0
dev.em.3.mac_stats.tx_frames_256_511: 0
dev.em.3.mac_stats.tx_frames_512_1023: 0
dev.em.3.mac_stats.tx_frames_1024_1522: 0
dev.em.3.mac_stats.tso_txd: 0
dev.em.3.mac_stats.tso_ctx_fail: 0
dev.em.3.interrupts.asserts: 0
dev.em.3.interrupts.rx_pkt_timer: 0
dev.em.3.interrupts.rx_abs_timer: 0
dev.em.3.interrupts.tx_pkt_timer: 0
dev.em.3.interrupts.tx_abs_timer: 0
dev.em.3.interrupts.tx_queue_empty: 0
dev.em.3.interrupts.tx_queue_min_thresh: 0
dev.em.3.interrupts.rx_desc_min_thresh: 0
dev.em.3.interrupts.rx_overrun: 0
..............................................................................
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAKOb=YaRz_S5VwF1_UYMz-3HPae-Hg6Sw%2BNKNsKG2DL-JkcOqA>