From owner-freebsd-mips@freebsd.org Sat Oct 31 19:21:08 2015 Return-Path: Delivered-To: freebsd-mips@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5A4F1A1BF16 for ; Sat, 31 Oct 2015 19:21:08 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id AD45C1B91 for ; Sat, 31 Oct 2015 19:21:07 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221]) by hz.grosbein.net (8.14.9/8.14.9) with ESMTP id t9VJKpno051392 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sat, 31 Oct 2015 20:20:51 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: adrian.chadd@gmail.com Received: from eg.sd.rdtc.ru (eugen@localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTP id t9VJKhL2051994; Sun, 1 Nov 2015 02:20:44 +0700 (KRAT) (envelope-from eugen@grosbein.net) Subject: Re: CPU underload To: Adrian Chadd References: <56348063.3090508@grosbein.net> <56348239.3050701@grosbein.net> <563500FC.8020201@grosbein.net> Cc: "freebsd-mips@freebsd.org" From: Eugene Grosbein X-Enigmail-Draft-Status: N1110 Message-ID: <5635148B.2070307@grosbein.net> Date: Sun, 1 Nov 2015 02:20:43 +0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM autolearn=no version=3.3.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hz.grosbein.net X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Oct 2015 19:21:08 -0000 On 01.11.2015 01:11, Adrian Chadd wrote: > Hi, > > I don't see any DELAY in my testing, nor MipsUserIntr, so yeah, I must > be doing something different. I had compiled the system using CFLAGS+=-Os trying to minimize code size. It seems, that was not wise. Now I rebuilt it without -Os and got better numbers: 60MByte/s under same inter-vlan briding test. However, this time "interrupt storm" stroke again but that was not same bug but just really higher numbers of interrupts on arge0 due to higher speed. I raised hw.intr_storm_threshold from default 1000 upto 10000 and "storm" is no more. "Unaligned Load Word" messages in dmesg I've mentioned before are generated by running pmcstat. If I do not run it, dmesg is not flooded and its last line is: "Nov 1 01:43:06 tl-wdr3600 login: ROOT LOGIN (root) ON ttyu0". No port flapping. CPU utilization has raised too, but idle part is still pretty large and it is very unstable: # vmstat 1 procs memory page disks faults cpu r b w avm fre flt re pi po fr sr fl0 md0 in sy cs us sy id 0 0 0 58M 93M 297 0 3 0 217 2 0 0 1826 8680 184 5 63 32 0 0 0 58M 93M 0 0 0 0 0 2 0 0 3773 105 138 0 78 22 0 0 0 58M 93M 0 0 0 0 0 2 0 0 2316 105 116 0 62 38 0 0 0 58M 93M 0 0 0 0 0 5 0 0 5786 105 167 0 80 20 0 0 0 58M 93M 0 0 0 0 0 2 0 0 4177 105 95 0 79 21 0 0 0 58M 93M 0 0 0 0 0 2 0 0 3032 105 85 0 81 19 0 0 0 58M 93M 0 0 0 0 0 2 0 0 2831 105 152 0 66 34 0 0 0 58M 93M 0 0 0 0 0 1 0 0 4436 105 147 0 75 25 0 0 0 58M 93M 0 0 0 0 0 2 0 0 2943 105 84 0 75 25 0 0 0 58M 93M 0 0 0 0 0 2 0 0 2886 105 141 0 73 27 0 0 0 58M 93M 0 0 0 0 0 3 0 0 5727 105 50 0 95 5 0 0 0 58M 93M 0 0 0 0 0 2 0 0 3021 105 141 0 64 36 0 0 0 58M 93M 0 0 0 0 0 2 0 0 2824 105 127 0 76 24 0 0 0 58M 93M 0 0 0 0 0 2 0 0 2168 105 153 0 54 46 0 0 0 58M 93M 0 0 0 0 0 2 0 0 4879 105 116 0 83 17 0 0 0 58M 93M 0 0 0 0 0 2 0 0 4341 105 210 0 81 19 0 0 0 58M 93M 0 0 0 0 0 2 0 0 2294 105 127 0 54 46 0 0 0 58M 93M 0 0 0 0 0 2 0 0 4178 109 126 1 79 20 0 0 0 58M 93M 0 0 0 0 0 5 0 0 5048 105 106 0 72 28 pmcstat still shows some DELAY samples (but much less now) and MipsUserIntr is still here: PMC: [INSTR_EXECUTED] Samples: 14535 (100.0%) , 0 unresolved %SAMP IMAGE FUNCTION CALLERS 9.1 kernel MipsUserIntr 6.1 kernel if_inc_counter vlan_transmit:1.1 bridge_forward:1.1 ... 3.8 kernel ether_nh_input netisr_dispatch_src 3.7 kernel bridge_input ether_nh_input 3.5 kernel _bus_dmamap_sync arge_start_locked:1.7 arge_intr:1.0 ... 3.3 kernel _bus_dmamap_load_buf _bus_dmamap_load_mbuf_sg 3.1 kernel critical_enter 3.1 kernel arge_intr intr_event_execute_handlers 2.9 kernel bridge_rtnode_lookup bridge_rtlookup:1.5 ... 2.9 kernel memcpy 2.8 kernel arge_start_locked arge_start 2.6 libc.so.7 bsearch pmcstat_symbol_search 2.2 kernel critical_exit if_inc_counter 2.1 kernel bridge_forward bridge_input 1.9 kernel bridge_rtnode_addr_c 1.9 kernel bus_dmamap_sync_buf _bus_dmamap_sync 1.9 kernel DELAY 1.8 kernel arge_newbuf arge_intr 1.7 kernel netisr_dispatch_src netisr_dispatch 1.6 kernel mipsNN_pdcache_inv_r CONVERSION STATISTICS:g_read pmcstat_process_log #exec/elf 2 #samples/total 261959 #samples/unknown-function 8887