From owner-freebsd-hackers@freebsd.org Thu Dec 15 15:55:52 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2E7A0C813D8 for ; Thu, 15 Dec 2016 15:55:52 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1a.eu.mailhop.org (outbound1a.eu.mailhop.org [52.58.109.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BE3B0F9F for ; Thu, 15 Dec 2016 15:55:51 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: f36fe890-c2de-11e6-9673-39b5816e8152 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 73.78.92.27 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [73.78.92.27]) by outbound1.eu.mailhop.org (Halon) with ESMTPSA id f36fe890-c2de-11e6-9673-39b5816e8152; Thu, 15 Dec 2016 15:55:51 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id uBFFthQZ001324; Thu, 15 Dec 2016 08:55:43 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1481817343.1889.467.camel@freebsd.org> Subject: Re: How to use sem_timedwait? From: Ian Lepore To: Konstantin Belousov Cc: Goran Meki? , freebsd-hackers@freebsd.org Date: Thu, 15 Dec 2016 08:55:43 -0700 In-Reply-To: <20161215112050.GO94325@kib.kiev.ua> References: <20161214074228.zh6q5zya2gszw4g6@hal9000.meka.no-ip.org> <1481748960.1889.398.camel@freebsd.org> <20161215002906.mllorgvvuovbdtze@hal9000.meka.no-ip.org> <1481775511.1889.450.camel@freebsd.org> <1481776532.1889.461.camel@freebsd.org> <20161215112050.GO94325@kib.kiev.ua> Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2016 15:55:52 -0000 On Thu, 2016-12-15 at 13:20 +0200, Konstantin Belousov wrote: > On Wed, Dec 14, 2016 at 09:35:32PM -0700, Ian Lepore wrote: > > > > On Wed, 2016-12-14 at 21:18 -0700, Ian Lepore wrote: > > > > > > On Thu, 2016-12-15 at 01:29 +0100, Goran Meki? wrote: > > > > > > > > > > > > On Wed, Dec 14, 2016 at 01:56:00PM -0700, Ian Lepore wrote: > > > > > > > > > > [...] > > >   > > > Making a guess here:  Is your actual goal to wake up periodically > > > with the period between wakeups as accurate as possible?  If so, > > > a > > > better mechanism might be to use kqueue(2)/kevent(2) > > > with EVFILT_TIMER events.  With EVFILT_TIMER events, the wakeups > > > are > > > scheduled as if aligned to a grid -- even if one wakeup is a bit > > > late > > > due to system workload, the next wakeup after it will still be > > > properly aligned to the original grid.  For example, if you ask > > > for a > > > wakeup once per millisecond and some wake happens after 1.2mS, > > > the > > > next wakeup will be .8mS after that; the phase of the wakeups > > > doesn't > > > shift over time. > Well, this can be alternatively explained as the statement that the > timeouts are scheduled to execute at the absolute times, instead of > specifying offsets from the current moment. With such formulation, it > is > obvious that userspace may do the same, calculating the next absolute > time based on previous desired moment + offset, instead of current > time > + offset. sem_timedwait(2) requests timeouts in CLOCK_REALTIME base. > > Of course, kqueue(2) timers resets are less expensive since all this > happens > in the callout callback in kernel. > > > > > > > I just dug up some old source code for testing kevent timers, > > included > > below.  Good news and bad news...  The good news is that it works > > perfectly on an arm system running -current: > > > >     root@imx6    :~ # ./kevent_evfilt_timer > >     nsec = 10000058742 for 10000 loops of 1000000 nsec > > > > The extra 58uS is fixed overhead for getting the ending time; if > > the > > loop runs for 100 seconds instead of 10 the extra time is still 55- > > 60uS. > > > > The bad news is that it doesn't work right on amd64 running 10- > > stable: > > > >     revolution > ./kevent_evfilt_timer  > >     nsec = 9313236220 for 10000 loops of 1000000 nsec > > > > I don't have any other x86 systems handy to test it on right now, > > but > > it's disturbing that 10 seconds worth of 1mS sleeps takes less than > > 10 > > seconds.  A strange thing here is that the *ratio* of the > > undersleeping > > is fixed, running the loop for 100 seconds instead of 10 gives: > > > >     revolution > ./kevent_evfilt_timer  > >     nsec = 93132267106 for 100000 loops of 1000000 nsec > What are timecounter and eventcounter used on this machine ? > Also please show first 20 lines of the dmesg, where CPU > identification > is performed. > > I checked on two machines, one running 11 and using HPET for events: > nsec = 10000091075 for 10000 loops of 1000000 nsec > another running HEAD and using LAPIC deadline: > nsec = 10000129629 for 10000 loops of 1000000 nsec It makes me feel a bit better to hear that it's only this machine, which is pretty old.  I wonder if it has something to do with overclocking the cpu?  I never have any other kind of timing trouble, though.  (Given that I work in the field of precision timing, I'm a bit sensitive to such things.) revolution > sysctl kern.timecounter kern.eventtimer kern.timecounter.tsc_shift: 1 kern.timecounter.smp_tsc_adjust: 0 kern.timecounter.smp_tsc: 1 kern.timecounter.invariant_tsc: 1 kern.timecounter.fast_gettime: 1 kern.timecounter.tick: 1 kern.timecounter.choice: TSC-low(1000) ACPI-fast(900) HPET(950) i8254(0) dummy(-1000000) kern.timecounter.hardware: TSC-low kern.timecounter.alloweddeviation: 0 kern.timecounter.stepwarnings: 0 kern.timecounter.tc.TSC-low.quality: 1000 kern.timecounter.tc.TSC-low.frequency: 2125044382 kern.timecounter.tc.TSC-low.counter: 1611837909 kern.timecounter.tc.TSC-low.mask: 4294967295 kern.timecounter.tc.ACPI-fast.quality: 900 kern.timecounter.tc.ACPI-fast.frequency: 3579545 kern.timecounter.tc.ACPI-fast.counter: 6548232 kern.timecounter.tc.ACPI-fast.mask: 16777215 kern.timecounter.tc.HPET.quality: 950 kern.timecounter.tc.HPET.frequency: 14318180 kern.timecounter.tc.HPET.counter: 3623382209 kern.timecounter.tc.HPET.mask: 4294967295 kern.timecounter.tc.i8254.quality: 0 kern.timecounter.tc.i8254.frequency: 1193182 kern.timecounter.tc.i8254.counter: 18218 kern.timecounter.tc.i8254.mask: 65535 kern.eventtimer.periodic: 0 kern.eventtimer.timer: LAPIC kern.eventtimer.idletick: 0 kern.eventtimer.singlemul: 2 kern.eventtimer.choice: LAPIC(600) HPET(350) HPET1(340) HPET2(340) HPET3(340) i8254(100) RTC(0) kern.eventtimer.et.HPET3.quality: 340 kern.eventtimer.et.HPET3.frequency: 14318180 kern.eventtimer.et.HPET3.flags: 3 kern.eventtimer.et.HPET2.quality: 340 kern.eventtimer.et.HPET2.frequency: 14318180 kern.eventtimer.et.HPET2.flags: 3 kern.eventtimer.et.HPET1.quality: 340 kern.eventtimer.et.HPET1.frequency: 14318180 kern.eventtimer.et.HPET1.flags: 3 kern.eventtimer.et.HPET.quality: 350 kern.eventtimer.et.HPET.frequency: 14318180 kern.eventtimer.et.HPET.flags: 3 kern.eventtimer.et.RTC.quality: 0 kern.eventtimer.et.RTC.frequency: 32768 kern.eventtimer.et.RTC.flags: 17 kern.eventtimer.et.i8254.quality: 100 kern.eventtimer.et.i8254.frequency: 1193182 kern.eventtimer.et.i8254.flags: 1 kern.eventtimer.et.LAPIC.quality: 600 kern.eventtimer.et.LAPIC.frequency: 85001804 kern.eventtimer.et.LAPIC.flags: 7 FreeBSD 10.3-STABLE #5 r308807: Fri Nov 18 09:51:39 MST 2016     ilepore@revolution.hippie.lan:/b/staging/machines/revolution10/obj/b/staging/machines/revolution10/src/sys/REVOLUTION10 amd64 FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 VT(vga): text 80x25 CPU: Intel(R) Xeon(R) CPU           W3680  @ 3.33GHz (4250.09-MHz K8-class CPU)   Origin="GenuineIntel"  Id=0x206c2  Family=0x6  Model=0x2c  Stepping=2   Features=0xbfebfbff   Features2=0x29ee3ff   AMD Features=0x2c100800   AMD Features2=0x1   VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID   TSC: P-state invariant, performance statistics real memory  = 12884901888 (12288 MB) avail memory = 12425519104 (11849 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: <092410 APIC1941> FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs FreeBSD/SMP: 1 package(s) x 6 core(s) x 2 SMT threads -- Ian