Date: Thu, 11 Jun 2020 01:30:14 -0700 From: Shrikanth Kamath <shrikanth07@gmail.com> To: freebsd-hackers@freebsd.org Subject: Xentimer/Lapic differences when used as a eventtimer source Message-ID: <CAEOAkMWxj9aSx-rchsYJivCUDTPqZ5LnDfjgKQMbxR7%2BPJ4hDg@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Trying to understand the implementation differences for xentimer v/s lapic
when used as eventtimer sources. I have 2 setups where there is FreeBSD
11.0 based Junos running in a virtual environment one on a xen based
hypervisor and another kvm based. The xen version has xentimer as the
eventtimer source v/s lapic for the kvm version. For an application that is
doing nanosleep with same resolution of 100 ns and running on these two
systems the rate of timer interrupts is order of magnitude different and
hogs the cpu in case of lapic but for xentimer it is not as significant.
Below I have listed some readings from dtrace probes I ran to understand,
For xen setup using xentimer as eventtimer
# sysctl kern.eventtimer
kern.eventtimer.periodic: 0
kern.eventtimer.timer: XENTIMER
kern.eventtimer.idletick: 0
kern.eventtimer.singlemul: 2
kern.eventtimer.choice: XENTIMER(950) LAPIC(100)
i8254(100) RTC(0)
kern.eventtimer.et.XENTIMER.quality: 950
kern.eventtimer.et.XENTIMER.frequency: 1000000000
kern.eventtimer.et.XENTIMER.flags: 6
For kvm setup using lapic as eventtimer
# sysctl kern.eventtimer
kern.eventtimer.periodic: 0
kern.eventtimer.timer: LAPIC
kern.eventtimer.idletick: 0
kern.eventtimer.singlemul: 2
kern.eventtimer.choice: LAPIC(600) RTC(0)
kern.eventtimer.et.LAPIC.quality: 600
kern.eventtimer.et.LAPIC.frequency: 3000050812
kern.eventtimer.et.LAPIC.flags: 7
Running fbt::et_start:entry with conditional for execname to be the
application and using an aggregation to count() calls, shows ~6K calls per
second for xentimer v/s ~95K calls for lapic
Trying to count number of timer interrupts on each with
dtrace -n 'fbt:kernel-xen:xentimer_intr:entry /cpu != 0/ { @cnt[cpu] =
count();} tick-1s { printa(@cnt); clear(@cnt);}' gives --> 6083 interrupts
per second
v/s
dtrace -n 'fbt:kernel:lapic_handle_timer:entry /cpu != 0/ { @cnt[cpu] =
count();} tick-1s { printa(@cnt); clear(@cnt);}' gives --> 99063 interrupts
per second
For xentimer
#vmstat -i | grep :xen
interrupt total rate
cpu1:xen 3642296440 6084
v/s lapic
# vmstat -i
interrupt total rate
cpu1:timer 61458150612 102715
Also does lapic source have additional overhead of VM entry/VM exit?
--
Shrikanth R K
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAEOAkMWxj9aSx-rchsYJivCUDTPqZ5LnDfjgKQMbxR7%2BPJ4hDg>