Date: Fri, 25 May 2012 22:36:45 +0200 (CEST) From: Wojciech Puchar <wojtek@wojtek.tensor.gdynia.pl> To: Eugen Konkov <kes-kes@yandex.ru> Cc: freebsd-questions@freebsd.org Subject: Re: re0 take 100% CPU Message-ID: <alpine.BSF.2.00.1205252235530.31165@wojtek.tensor.gdynia.pl> In-Reply-To: <1634481831.20120525211544@yandex.ru> References: <1634481831.20120525211544@yandex.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
broken device or motherboard, or conflict. check if irq is not shared.
re0 are not great network cards but really not that bad. something is
badly broken with hardware
On Fri, 25 May 2012, Eugen Konkov wrote:
> How to look what re0 is doing?? And how to debug that?
>
> last pid: 74164; load averages: 17.97, 13.81, 14.40 up 47+09:31:54 21:50:58
> 110 processes: 9 running, 87 sleeping, 14 waiting
> CPU: 0.5% user, 0.0% nice, 1.6% system, 97.9% interrupt, 0.0% idle
> Mem: 194M Active, 449M Inact, 204M Wired, 20K Cache, 112M Buf, 1143M Free
> Swap: 512M Total, 512M Free
>
> PID USERNAME PRI NICE SIZE RES STATE TIME WCPU COMMAND
> 12 root -76 - 0K 112K WAIT 65.3H 100.00% {irq256: re0}
> 65795 root 39 0 13420K 5804K select 19:58 0.63% snmpd
> 2664 root 36 0 14608K 6812K RUN 75:49 0.54% bgpd
> 0 root -76 0 0K 80K - 31.6H 0.44% {dummynet}
> 7758 bind 36 0 75268K 62880K RUN 178:21 0.24% {named}
> 12 root -44 - 0K 112K WAIT 29.4H 0.10% {swi4: clock}
>
> # systat -v
> 2 users Load 13.61 12.24 13.96 May 24 21:49
>
> Mem:KB REAL VIRTUAL VN PAGER SWAP PAGER
> Tot Share Tot Share Free in out in out
> Act 256100 6368 626872 8116 1171172 count
> All 353960 9500 2792812 26356 pages
> Proc: Interrupts
> r p d s w Csw Trp Sys Int Sof Flt 3 cow 4404 total
> 12 32 2207 106 4216 4404 3983 48 28 zfod 1 ata0 14
> ozfod 4119 hpet0 20
> 4.1%Sys 93.4%Intr 2.6%User 0.0%Nice 0.0%Idle %ozfod uhci0 ehci
> | | | | | | | | | | | daefr 284 re0 256
> ==+++++++++++++++++++++++++++++++++++++++++++++++> 7 prcfr
> 40 dtbuf 44 totfr
> Namei Name-cache Dir-cache 111725 desvn react
> Calls hits % hits % 64097 numvn pdwak
> 3177 3177 100 27926 frevn pdpgs
> intrn
> Disks ad1 da0 pass0 209000 wire
> KB/t 14.85 0.00 0.00 198616 act
> tps 1 0 0 459200 inact
> MB/s 0.01 0.00 0.00 20 cache
> %busy 2 0 0 1169956 free
>
> # vmstat -i
> interrupt total rate
> irq14: ata0 5946124 1
> irq20: hpet0 3979660990 971
> irq23: uhci0 ehci0 85 0
> irq256: re0 3847013663 939
> Total 7832620862 1912
> # uptime
> 9:44PM up 47 days, 9:25, 2 users, load averages: 9.05, 14.17, 15.61
>
> #bwm-ng -I re0
> bwm-ng v0.6 (probing every 0.500s), press 'h' for help
> input: getifaddrs type: rate
> - iface Rx Tx Total
> ==============================================================================
> re0: 13.32 Mb/s 12.42 Mb/s 25.74 Mb/s
> ------------------------------------------------------------------------------
> total: 14.95 Mb/s 13.94 Mb/s 28.89 Mb/s
> # vmstat -z
> ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP
>
> UMA Kegs: 128, 0, 92, 28, 92, 0, 0
> UMA Zones: 224, 0, 92, 10, 92, 0, 0
> UMA Slabs: 284, 0, 1158, 4, 29780, 0, 0
> UMA RCntSlabs: 544, 0, 919, 5, 919, 0, 0
> UMA Hash: 128, 0, 2, 28, 3, 0, 0
> 16 Bucket: 76, 0, 36, 14, 36, 0, 0
> 32 Bucket: 140, 0, 43, 13, 44, 0, 0
> 64 Bucket: 268, 0, 59, 11, 67, 90, 0
> 128 Bucket: 524, 0, 895, 1, 27120, 585, 0
> VM OBJECT: 136, 0, 37976, 130,40520105, 0, 0
> MAP: 140, 0, 7, 49, 7, 0, 0
> KMAP ENTRY: 72, 57505, 32, 127, 79899, 0, 0
> MAP ENTRY: 72, 0, 2094, 1192,100480394, 0, 0
> DP fakepg: 72, 0, 0, 0, 0, 0, 0
> SG fakepg: 72, 0, 0, 0, 0, 0, 0
> mt_zone: 2060, 0, 268, 11, 268, 0, 0
> 16: 16, 0, 5106, 375,251057215, 0, 0
> 32: 32, 0, 26924, 196,1931894354, 0, 0
> 64: 64, 0, 6372, 413,172309547, 0, 0
> 128: 128, 0, 23931, 41019,195195229, 0, 0
> 256: 256, 0, 2786, 289,2302309652, 0, 0
> 512: 512, 0, 638, 58, 4189195, 0, 0
> 1024: 1024, 0, 533, 15, 1586826, 0, 0
> 2048: 2048, 0, 251, 59, 17903, 0, 0
> 4096: 4096, 0, 154, 29, 2738963, 0, 0
> Files: 56, 0, 180, 289,224690270, 0, 0
> TURNSTILE: 72, 0, 156, 24, 156, 0, 0
> umtx pi: 52, 0, 0, 0, 0, 0, 0
> MAC labels: 20, 0, 0, 0, 0, 0, 0
> PROC: 688, 0, 57, 23, 1272718, 0, 0
> THREAD: 728, 0, 124, 31, 206859, 0, 0
> SLEEPQUEUE: 44, 0, 156, 80, 156, 0, 0
> VMSPACE: 232, 0, 38, 30, 1272743, 0, 0
> cpuset: 40, 0, 2, 182, 2, 0, 0
> audit_record: 816, 0, 0, 0, 0, 0, 0
> mbuf_packet: 256, 0, 256, 1536,5400971809, 0, 0
> mbuf: 256, 0, 6, 1412,9019029846, 0, 0
> mbuf_cluster: 2048, 25600, 1792, 6, 1792, 0, 0
> mbuf_jumbo_page: 4096, 12800, 0, 20, 828696, 0, 0
> mbuf_jumbo_9k: 9216, 6400, 0, 0, 0, 0, 0
> mbuf_jumbo_16k: 16384, 3200, 0, 0, 0, 0, 0
> mbuf_ext_refcnt: 4, 0, 0, 0, 0, 0, 0
> NetGraph items: 36, 4130, 2, 175,1535407748, 0, 0
> NetGraph data items: 36, 531, 0, 531,3322859915,26160, 0
> g_bio: 140, 0, 0, 4144,23787066, 0, 0
> ttyinq: 152, 0, 300, 12, 675, 0, 0
> ttyoutq: 256, 0, 157, 8, 353, 0, 0
> ata_request: 204, 0, 0, 1045, 5946685, 0, 0
> ata_composite: 180, 0, 0, 0, 0, 0, 0
> VNODE: 272, 0, 64112, 134, 7873012, 0, 0
> VNODEPOLL: 60, 0, 0, 0, 0, 0, 0
> S VFS Cache: 72, 0, 66367, 466, 8051562, 0, 0
> L VFS Cache: 292, 0, 595, 198, 109305, 0, 0
> NAMEI: 1024, 0, 0, 12,171046456, 0, 0
> NFSMOUNT: 532, 0, 0, 0, 0, 0, 0
> NFSNODE: 460, 0, 0, 0, 0, 0, 0
> DIRHASH: 1024, 0, 485, 11, 497, 0, 0
> pipe: 392, 0, 10, 30, 784482, 0, 0
> Mountpoints: 648, 0, 6, 6, 6, 0, 0
> ksiginfo: 80, 0, 66, 990, 210783, 0, 0
> itimer: 220, 0, 0, 0, 0, 0, 0
> KNOTE: 72, 0, 8, 310,92381338, 0, 0
> socket: 416, 25605, 73, 242,131092823, 0, 0
> unpcb: 172, 25622, 26, 43, 3331350, 0, 0
> ipq: 32, 904, 0, 226, 8756, 0, 0
> udp_inpcb: 220, 25614, 7, 245,127020097, 0, 0
> udpcb: 8, 25781, 7, 399,127020097, 0, 0
> tcp_inpcb: 220, 25614, 33, 57, 644727, 0, 0
> tcpcb: 692, 25600, 26, 39, 644727, 0, 0
> tcptw: 52, 5184, 7, 137, 92058, 0, 0
> syncache: 112, 15365, 0, 70, 360404, 0, 0
> hostcache: 76, 15400, 30, 70, 3515, 0, 0
> tcpreass: 20, 1690, 0, 169, 1927, 0, 0
> sackhole: 20, 0, 0, 169, 620, 0, 0
> sctp_ep: 880, 25600, 0, 0, 0, 0, 0
> sctp_asoc: 1524, 40000, 0, 0, 0, 0, 0
> sctp_laddr: 24, 80040, 0, 145, 45, 0, 0
> sctp_raddr: 464, 80000, 0, 0, 0, 0, 0
> sctp_chunk: 96, 400000, 0, 0, 0, 0, 0
> sctp_readq: 76, 400000, 0, 0, 0, 0, 0
> sctp_stream_msg_out: 72, 400044, 0, 0, 0, 0, 0
> sctp_asconf: 24, 400055, 0, 0, 0, 0, 0
> sctp_asconf_ack: 24, 400055, 0, 0, 0, 0, 0
> ripcb: 220, 25614, 6, 48, 78019, 0, 0
> rtentry: 108, 0, 24066, 18, 288480, 0, 0
> IPFW dynamic rule: 108, 0, 0, 0, 0, 0, 0
> divcb: 220, 25614, 0, 0, 0, 0, 0
> selfd: 28, 0, 158, 223,1786944834, 0, 0
> ip4flow: 40, 25668, 680, 24160,36376221, 0, 0
> ip6flow: 64, 25636, 0, 174, 20, 0, 0
> SWAPMETA: 276, 121576, 0, 0, 0, 0, 0
> FFS inode: 116, 0, 64073, 46, 7872685, 0, 0
> FFS1 dinode: 128, 0, 0, 0, 0, 0, 0
> FFS2 dinode: 256, 0, 64073, 22, 7872685, 0, 0
>
> # uname -a
> FreeBSD bor 9.0-CURRENT FreeBSD 9.0-CURRENT #1: Wed Feb 23 09:39:22 UTC 2011 @:/usr/obj/usr/src/sys/KES_KERN_v9 i386
>
> #tail /var/log/messages
> May 24 21:58:57 bor kernel: Limiting icmp unreach response from 273 to 200 packets/sec
> May 24 21:58:58 bor kernel: Limiting icmp unreach response from 223 to 200 packets/sec
> May 24 21:58:59 bor kernel: Limiting icmp unreach response from 235 to 200 packets/sec
> May 24 21:59:00 bor kernel: Limiting icmp unreach response from 278 to 200 packets/sec
> May 24 21:59:01 bor kernel: Limiting icmp unreach response from 283 to 200 packets/sec
> May 24 21:59:02 bor kernel: Limiting icmp unreach response from 255 to 200 packets/sec
> May 24 21:59:03 bor kernel: Limiting icmp unreach response from 269 to 200 packets/sec
> May 24 21:59:04 bor kernel: Limiting icmp unreach response from 286 to 200 packets/sec
> May 24 21:59:05 bor kernel: Limiting icmp unreach response from 225 to 200 packets/sec
> May 24 21:59:06 bor kernel: Limiting icmp unreach response from 328 to 200 packets/sec
> May 24 21:59:07 bor kernel: Limiting icmp unreach response from 285 to 200 packets/sec
> May 24 21:59:08 bor kernel: Limiting icmp unreach response from 286 to 200 packets/sec
> May 24 21:59:09 bor kernel: Limiting icmp unreach response from 304 to 200 packets/sec
> May 24 21:59:10 bor kernel: Limiting icmp unreach response from 304 to 200 packets/sec
> May 24 21:59:11 bor kernel: Limiting icmp unreach response from 311 to 200 packets/sec
>
> Thank you.
>
>
> _______________________________________________
> freebsd-questions@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.org"
>
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?alpine.BSF.2.00.1205252235530.31165>