Date: Tue, 1 Feb 2011 21:46:18 +0300 From: Lev Serebryakov <lev@FreeBSD.org> To: "Vogel, Jack" <jack.vogel@intel.com> Cc: freebsd-stable@freebsd.org Subject: em0 lock up / hangs (WAS: em0: Watchdog timeout -- resetting) Message-ID: <1775331865.20110201214618@serebryakov.spb.ru> In-Reply-To: <1364200185.20110201153833@serebryakov.spb.ru> References: <1481093142.20110201102416@serebryakov.spb.ru> <1522400637.20110201105842@serebryakov.spb.ru> <4D47C2FB.4090803@rdtc.ru> <1364200185.20110201153833@serebryakov.spb.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --]
Hello, Eugene.
You wrote 1 февраля 2011 г., 15:38:33:
> Eugene wrote:
>> You could give a try to netisr parallelism of RELENG_8 instead of POLLING
>> (and tune interrupt throttling) if your box does not have lots of dynamic
>> interfaces like when using mpd.
> Jack wrote:
>> I don't test POLLING, sounds like its broken, I don't understand
>> why you think you need you need it? This hardware supports
>> MSI why not use it?
> I send one answer to two messages, because data is the same.
> Here it is snapshot of "top -S" with "H" pressed when server sends
> 1Gbit/s via SMB with polling (Windows'7 client copies 8GiB sparse file to very
> fast local disk):
> the same without polling, with net.isr settings:
> # sysctl net.isr
> net.isr.direct: 0
> net.isr.direct_force: 0
After these settings server lost connection. It works locally, no
panic, but "ping gateway" shows "No buffer space available", and any
other "network activity" shows the same message.
Up-down of interface helps.
I attached outputs of:
vmstat -m
netstat -m
sysctl dev.em0
BEFORE interface reset
No polling, net.isr.direct=0, net.isr.direct_force=0
--
// Black Lion AKA Lev Serebryakov <lev@FreeBSD.org>
[-- Attachment #2 --]
dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.1.9
dev.em.0.%driver: em
dev.em.0.%location: slot=25 function=0 handle=\_SB_.PCI0.GBEC
dev.em.0.%pnpinfo: vendor=0x8086 device=0x10bd subvendor=0x1043 subdevice=0x8268 class=0x020000
dev.em.0.%parent: pci0
dev.em.0.nvm: -1
dev.em.0.debug: -1
dev.em.0.rx_int_delay: 200
dev.em.0.tx_int_delay: 200
dev.em.0.rx_abs_int_delay: 4000
dev.em.0.tx_abs_int_delay: 4000
dev.em.0.rx_processing_limit: 4096
dev.em.0.flow_control: 3
dev.em.0.link_irq: 0
dev.em.0.mbuf_alloc_fail: 0
dev.em.0.cluster_alloc_fail: 0
dev.em.0.dropped: 0
dev.em.0.tx_dma_fail: 0
dev.em.0.rx_overruns: 0
dev.em.0.watchdog_timeouts: 3
dev.em.0.device_control: 1074790976
dev.em.0.rx_control: 67141634
dev.em.0.fc_high_water: 8192
dev.em.0.fc_low_water: 6692
dev.em.0.queue0.txd_head: 391
dev.em.0.queue0.txd_tail: 354
dev.em.0.queue0.tx_irq: 0
dev.em.0.queue0.no_desc_avail: 0
dev.em.0.queue0.rxd_head: 1704
dev.em.0.queue0.rxd_tail: 1703
dev.em.0.queue0.rx_irq: 0
dev.em.0.mac_stats.excess_coll: 0
dev.em.0.mac_stats.single_coll: 0
dev.em.0.mac_stats.multiple_coll: 0
dev.em.0.mac_stats.late_coll: 0
dev.em.0.mac_stats.collision_count: 0
dev.em.0.mac_stats.symbol_errors: 0
dev.em.0.mac_stats.sequence_errors: 0
dev.em.0.mac_stats.defer_count: 1617
dev.em.0.mac_stats.missed_packets: 9429
dev.em.0.mac_stats.recv_no_buff: 0
dev.em.0.mac_stats.recv_undersize: 0
dev.em.0.mac_stats.recv_fragmented: 2
dev.em.0.mac_stats.recv_oversize: 0
dev.em.0.mac_stats.recv_jabber: 0
dev.em.0.mac_stats.recv_errs: 66
dev.em.0.mac_stats.crc_errs: 65
dev.em.0.mac_stats.alignment_errs: 0
dev.em.0.mac_stats.coll_ext_errs: 0
dev.em.0.mac_stats.xon_recvd: 1739
dev.em.0.mac_stats.xon_txd: 0
dev.em.0.mac_stats.xoff_recvd: 2934
dev.em.0.mac_stats.xoff_txd: 0
dev.em.0.mac_stats.total_pkts_recvd: 279846442
dev.em.0.mac_stats.good_pkts_recvd: 279832266
dev.em.0.mac_stats.bcast_pkts_recvd: 11094
dev.em.0.mac_stats.mcast_pkts_recvd: 1743
dev.em.0.mac_stats.rx_frames_64: 0
dev.em.0.mac_stats.rx_frames_65_127: 0
dev.em.0.mac_stats.rx_frames_128_255: 0
dev.em.0.mac_stats.rx_frames_256_511: 0
dev.em.0.mac_stats.rx_frames_512_1023: 0
dev.em.0.mac_stats.rx_frames_1024_1522: 0
dev.em.0.mac_stats.good_octets_recvd: 26804649175
dev.em.0.mac_stats.good_octets_txd: 706747214491
dev.em.0.mac_stats.total_pkts_txd: 543878097
dev.em.0.mac_stats.good_pkts_txd: 543878097
dev.em.0.mac_stats.bcast_pkts_txd: 2232
dev.em.0.mac_stats.mcast_pkts_txd: 11471
dev.em.0.mac_stats.tx_frames_64: 0
dev.em.0.mac_stats.tx_frames_65_127: 0
dev.em.0.mac_stats.tx_frames_128_255: 0
dev.em.0.mac_stats.tx_frames_256_511: 0
dev.em.0.mac_stats.tx_frames_512_1023: 0
dev.em.0.mac_stats.tx_frames_1024_1522: 0
dev.em.0.mac_stats.tso_txd: 169064286
dev.em.0.mac_stats.tso_ctx_fail: 0
dev.em.0.interrupts.asserts: 3780022
dev.em.0.interrupts.rx_pkt_timer: 0
dev.em.0.interrupts.rx_abs_timer: 0
dev.em.0.interrupts.tx_pkt_timer: 0
dev.em.0.interrupts.tx_abs_timer: 0
dev.em.0.interrupts.tx_queue_empty: 0
dev.em.0.interrupts.tx_queue_min_thresh: 0
dev.em.0.interrupts.rx_desc_min_thresh: 0
dev.em.0.interrupts.rx_overrun: 0
dev.em.0.wake: 0
[-- Attachment #3 --]
Type InUse MemUse HighUse Requests Size(s)
module 152 19K - 152 128
USB 76 66K - 80 16,32,64,128,256,1024,2048,4096
mtx_pool 2 16K - 2
CAM periph 22 6K - 44 16,32,64,256
pci_link 16 2K - 16 64,128
acpisem 19 3K - 19 128
subproc 398 367K - 37271 512,4096
proc 2 16K - 2
session 23 3K - 2396 128
pgrp 25 4K - 2542 128
cred 58 10K - 4992860 64,256
uidinfo 8 3K - 850726 128,2048
plimit 10 3K - 30679 256
acpi_perf 2 1K - 2 128
CAM XPT 293 425K - 414 16,32,64,128,256,1024,2048
DEVFS1 146 73K - 157 512
sysctltmp 0 0K - 872 16,32,64,128,256
sysctloid 3535 175K - 3628 16,32,64,128
sysctl 0 0K - 24476 16,32,64
callout 1 512K - 1
umtx 426 54K - 426 128
p1003.1b 1 1K - 1 16
SWAP 2 549K - 2 64
DEVFS3 172 43K - 184 256
bus-sc 71 413K - 1251 16,32,64,128,256,512,2048,4096
bus 640 65K - 3676 16,32,64,128,256,512,1024
devstat 14 29K - 14 32,4096
eventhandler 67 6K - 67 64,128
kobj 93 372K - 115 4096
Per-cpu 1 1K - 1 32
DEVFS 22 1K - 23 16,128
rman 202 25K - 623 16,32,128
DEVFSP 1 1K - 1 64
sbuf 0 0K - 1292 16,32,64,128,256,512,1024,2048,4096
pfs_nodes 21 6K - 21 256
stack 0 0K - 2 256
taskqueue 15 2K - 15 16,32,128
Unitno 10 1K - 64 32,64
iov 0 0K - 708072 16,32,64,128,256,512
select 121 16K - 5854972140 128,512,1024
ioctlops 0 0K - 129300073 16,32,64,128,256,512,1024,2048,4096
msg 4 30K - 4 2048,4096
sem 4 11K - 4 512,1024
shm 1 20K - 1
tty 20 20K - 27 1024,2048
pts 0 0K - 5 256
mbuf_tag 0 0K - 30 32
shmfd 1 8K - 1
GEOM 175 38K - 741 16,32,64,128,256,512,1024
pcb 266 21K - 3514794 16,32,1024,2048,4096
soname 6 1K - 13515355 16,32,128
acl 0 0K - 2514 4096
biobuf 0 0K - 62 2048
vfscache 1 1024K - 1
cl_savebuf 0 0K - 24873 64,128
export_host 2 1K - 2 256
vfs_hash 1 512K - 1
vnodes 2 1K - 2 256
vnodemarker 0 0K - 149689 512
mount 104 6K - 304 16,32,64,128,256,512
BPF 7 9K - 7 128,512,4096
ether_multi 12 1K - 26 16,64
ifaddr 14 7K - 15 32,512,4096
ifnet 3 5K - 3 128,2048
clone 2 8K - 2 4096
arpcom 1 1K - 1 16
lltable 3 2K - 41 256,512
fw_xfer 0 0K - 1 256
firewire 11 35K - 14 64,128,512,1024,2048,4096
scsi_da 0 0K - 16 16
kbdmux 6 10K - 6 16,512,1024,2048,4096
LED 1 1K - 1 128
isadev 5 1K - 5 128
routetbl 14 4K - 86811 32,64,128,256,512
igmp 2 1K - 2 256
CAM dev queue 8 1K - 8 128
CAM queue 43 3K - 148 16,32,64,256
CAM SIM 8 2K - 8 256
ip_moptions 4 1K - 8 64,256
in_multi 3 1K - 5 256
in_mfilter 2 2K - 4 1024
hostcache 1 28K - 1
syncache 1 96K - 1
NFS FHA 1 2K - 177 64,2048
rpc 186 98K - 590 32,64,128,256,512,2048
audit_evclass 172 6K - 211 32
savedino 0 0K - 18680 256
newdirblk 0 0K - 35 64
dirrem 0 0K - 106481 64
mkdir 0 0K - 140 64
diradd 2 1K - 106292 64
freefile 1 1K - 51395 64
freeblks 1 1K - 51511 256
freefrag 0 0K - 71920 64
allocindir 0 0K - 254181 128
indirdep 0 0K - 2975 64
allocdirect 1 1K - 195293 256
bmsafemap 2 1K - 46947 128
newblk 1 1K - 449475 64,512
inodedep 4 513K - 115184 256
pagedep 3 129K - 12942 128
ufs_dirhash 592 114K - 258043 16,32,64,128,256,512,1024
ufs_mount 15 127K - 15 512,2048,4096
UMAHash 3 11K - 10 512,1024,2048,4096
ddb_capture 1 48K - 1
acpica 3837 393K - 84439 16,32,64,128,256,512,1024,2048
cdev 8 2K - 8 256
vm_pgdata 2 129K - 2 128
acpidev 78 5K - 78 64
sigio 1 1K - 1 64
filedesc 60 77K - 57191 16,32,64,128,256,512,1024,2048,4096
kenv 78 11K - 82 16,32,64,128
io_apic 1 2K - 1 2048
kqueue 2 13K - 128290 256,2048,4096
memdesc 1 4K - 1 4096
acpitask 1 2K - 1 2048
proc-args 26 2K - 102303 16,32,64,128,256
atkbddev 2 1K - 2 64
ithread 76 12K - 76 32,128,256
entropy 1024 64K - 1024 64
UART 3 2K - 3 16,512,1024
KTRACE 100 13K - 100 128
linker 57 6K - 63 16,32,64,128,512
lockf 53 6K - 3115030 64,128
temp 22 401K - 854005 16,32,64,128,256,512,1024,2048,4096
devbuf 20230 35933K - 20329 16,32,64,128,256,512,1024,2048,4096
USBdev 47 12K - 47 64,128,1024
nexusdev 3 1K - 3 16
raid5_data 9 1800K - 121465729 16,64,512,4096
[-- Attachment #4 --]
12498/5307/17805 mbufs in use (current/cache/total)
4228/5062/9290/204800 mbuf clusters in use (current/cache/total/max)
4218/5037 mbuf+clusters out of packet secondary zone in use (current/cache)
0/219/219/192000 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/6400 9k jumbo clusters in use (current/cache/total/max)
0/0/0/3200 16k jumbo clusters in use (current/cache/total/max)
11580K/12326K/23907K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0/0/0 sfbufs in use (current/peak/max)
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile
0 calls to protocol drain routines
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1775331865.20110201214618>