Date: Wed, 27 Aug 2014 02:21:32 +0400 From: Slawa Olhovchenkov <slw@zxy.spb.ru> To: Adrian Chadd <adrian@freebsd.org> Cc: "svn-src-head@freebsd.org" <svn-src-head@freebsd.org>, "svn-src-all@freebsd.org" <svn-src-all@freebsd.org>, "src-committers@freebsd.org" <src-committers@freebsd.org> Subject: Re: svn commit: r265792 - head/sys/kern Message-ID: <20140826222131.GG2075@zxy.spb.ru> In-Reply-To: <CAJ-Vmo=pJfH3s%2BTM3t3yeQBnoaJ4fVq6Jf0g-ZkiWpY36N0Gbg@mail.gmail.com> References: <201405100053.s4A0rbF9080571@svn.freebsd.org> <20140511083114.GA53503@zxy.spb.ru> <CAJ-Vmo=x-JkHSVDT4REWQ_g=vb45T%2BTfrpZ8bc28NNbPd-ZYsg@mail.gmail.com> <20140520154113.GA23318@zxy.spb.ru> <CAJ-Vmonby1MVuuJnWoXcPh1cTE1q0SQEQhfL_ESzoDEWHtKcCw@mail.gmail.com> <20140826185407.GE2075@zxy.spb.ru> <CAJ-Vmo=pJfH3s%2BTM3t3yeQBnoaJ4fVq6Jf0g-ZkiWpY36N0Gbg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Aug 26, 2014 at 03:05:41PM -0700, Adrian Chadd wrote: > This to me reads like "we need to fix pmc's locking so it's not so > terrible on multi-multi-core-machines." :) I am may be wrong, but I don't see in pmc interrupt path. May be this is need fix too? > On 26 August 2014 11:54, Slawa Olhovchenkov <slw@zxy.spb.ru> wrote: > > On Tue, May 20, 2014 at 09:04:25AM -0700, Adrian Chadd wrote: > > > >> On 20 May 2014 08:41, Slawa Olhovchenkov <slw@zxy.spb.ru> wrote: > >> > >> >> (But if you try it on 10.0 and it changes things, by all means let me know.) > >> > > >> > I am try on 10.0, but not sure about significant improvement (may be > >> > 10%). > >> > > >> > For current CPU (E5-2650 v2 @ 2.60GHz) hwpmc don't working (1. after > >> > collect some data `pmcstat -R sample.out -G out.txt` don't decode any; > >> > 2. kldunload hwpmc do kernel crash) and I can't collect detailed > >> > profile information. > >> > >> Yup. I'm starting to get really ticked off at how pmc logging on > >> multi-core devices just "stops" after a while. I'll talk with other > >> pmc people and see if we can figure out what the heck is going on. :( > > > > Now I can test you work on CPU w/ working pmc. > > Current traffic 16.8 Gbit/s. > > CPU: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz (2000.04-MHz K8-class CPU) > > > > last pid: 22677; load averages: 9.12, 9.07, 8.90 up 0+04:06:06 22:52:57 > > 47 processes: 3 running, 44 sleeping > > CPU 0: 11.8% user, 0.0% nice, 43.1% system, 1.6% interrupt, 43.5% idle > > CPU 1: 11.4% user, 0.0% nice, 51.8% system, 0.0% interrupt, 36.9% idle > > CPU 2: 10.6% user, 0.0% nice, 46.3% system, 0.8% interrupt, 42.4% idle > > CPU 3: 11.8% user, 0.0% nice, 45.1% system, 0.4% interrupt, 42.7% idle > > CPU 4: 13.7% user, 0.0% nice, 43.1% system, 0.0% interrupt, 43.1% idle > > CPU 5: 14.5% user, 0.0% nice, 45.9% system, 0.4% interrupt, 39.2% idle > > CPU 6: 0.0% user, 0.0% nice, 5.5% system, 68.6% interrupt, 25.9% idle > > CPU 7: 0.0% user, 0.0% nice, 4.3% system, 70.2% interrupt, 25.5% idle > > CPU 8: 0.0% user, 0.0% nice, 2.7% system, 69.4% interrupt, 27.8% idle > > CPU 9: 0.0% user, 0.0% nice, 4.7% system, 67.1% interrupt, 28.2% idle > > CPU 10: 0.0% user, 0.0% nice, 3.1% system, 76.9% interrupt, 20.0% idle > > CPU 11: 0.0% user, 0.0% nice, 5.1% system, 58.8% interrupt, 36.1% idle > > Mem: 322M Active, 15G Inact, 96G Wired, 956K Cache, 13G Free > > ARC: 90G Total, 84G MFU, 5690M MRU, 45M Anon, 394M Header, 98M Other > > Swap: > > > > > > > > @ CPU_CLK_UNHALTED_CORE [241440 samples] > > > > 10.59% [25561] _mtx_lock_spin_cookie @ /boot/kernel/kernel > > 94.25% [24092] pmclog_reserve @ /boot/kernel/hwpmc.ko > > 100.0% [24092] pmclog_process_callchain > > 100.0% [24092] pmc_process_samples > > 100.0% [24092] pmc_hook_handler > > 100.0% [24092] hardclock_cnt @ /boot/kernel/kernel > > 100.0% [24092] handleevents > > 99.62% [24001] timercb > > 100.0% [24001] lapic_handle_timer > > 00.38% [91] cpu_activeclock > > 100.0% [91] cpu_idle > > 100.0% [91] sched_idletd > > 100.0% [91] fork_exit > > 03.04% [777] callout_lock > > 91.63% [712] callout_reset_sbt_on > > 98.60% [702] tcp_timer_activate > > 94.87% [666] tcp_do_segment > > 100.0% [666] tcp_input > > 100.0% [666] ip_input > > 100.0% [666] netisr_dispatch_src > > 100.0% [666] ether_demux > > 100.0% [666] ether_nh_input > > 100.0% [666] netisr_dispatch_src > > 98.05% [653] ixgbe_rxeof @ /boot/kernel/if_ixgbe.ko > > 87.44% [571] ixgbe_msix_que > > 100.0% [571] intr_event_execute_handlers @ /boot/kernel/kernel > > 100.0% [571] ithread_loop > > 100.0% [571] fork_exit > > 12.56% [82] ixgbe_handle_que @ /boot/kernel/if_ixgbe.ko > > 100.0% [82] taskqueue_run_locked @ /boot/kernel/kernel > > 100.0% [82] taskqueue_thread_loop > > 100.0% [82] fork_exit > > 01.95% [13] tcp_lro_flush > > 92.31% [12] ixgbe_rxeof @ /boot/kernel/if_ixgbe.ko > > 100.0% [12] ixgbe_msix_que > > 100.0% [12] intr_event_execute_handlers @ /boot/kernel/kernel > > 100.0% [12] ithread_loop > > 07.69% [1] tcp_lro_rx > > 100.0% [1] ixgbe_rxeof @ /boot/kernel/if_ixgbe.ko > > 100.0% [1] ixgbe_msix_que > > 100.0% [1] intr_event_execute_handlers @ /boot/kernel/kernel > > 05.13% [36] tcp_output > > 100.0% [36] tcp_do_segment > > 100.0% [36] tcp_input > > 100.0% [36] ip_input > > 100.0% [36] netisr_dispatch_src > > 100.0% [36] ether_demux > > 100.0% [36] ether_nh_input > > 100.0% [36] netisr_dispatch_src > > 100.0% [36] ixgbe_rxeof @ /boot/kernel/if_ixgbe.ko > > 91.67% [33] ixgbe_msix_que > > 100.0% [33] intr_event_execute_handlers @ /boot/kernel/kernel > > 100.0% [33] ithread_loop > > 08.33% [3] ixgbe_handle_que @ /boot/kernel/if_ixgbe.ko > > 100.0% [3] taskqueue_run_locked @ /boot/kernel/kernel > > 100.0% [3] taskqueue_thread_loop
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20140826222131.GG2075>