From owner-svn-src-head@freebsd.org Thu Nov 29 03:44:03 2018 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BC4C8114C36D; Thu, 29 Nov 2018 03:44:03 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 266BA70C99; Thu, 29 Nov 2018 03:44:03 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E197314107; Thu, 29 Nov 2018 03:44:02 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id wAT3i2R3032662; Thu, 29 Nov 2018 03:44:02 GMT (envelope-from mjg@FreeBSD.org) Received: (from mjg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id wAT3i2Ai032661; Thu, 29 Nov 2018 03:44:02 GMT (envelope-from mjg@FreeBSD.org) Message-Id: <201811290344.wAT3i2Ai032661@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mjg set sender to mjg@FreeBSD.org using -f From: Mateusz Guzik Date: Thu, 29 Nov 2018 03:44:02 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r341178 - head/sys/kern X-SVN-Group: head X-SVN-Commit-Author: mjg X-SVN-Commit-Paths: head/sys/kern X-SVN-Commit-Revision: 341178 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 266BA70C99 X-Spamd-Result: default: False [1.32 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_SPAM_SHORT(0.50)[0.500,0]; NEURAL_SPAM_MEDIUM(0.31)[0.308,0]; NEURAL_SPAM_LONG(0.51)[0.511,0] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Nov 2018 03:44:04 -0000 Author: mjg Date: Thu Nov 29 03:44:02 2018 New Revision: 341178 URL: https://svnweb.freebsd.org/changeset/base/341178 Log: Tidy up hardclock. - use fcmpset for updating ticks - move (rarely used) itimer handling to a dedicated function Sponsored by: The FreeBSD Foundation Modified: head/sys/kern/kern_clock.c Modified: head/sys/kern/kern_clock.c ============================================================================== --- head/sys/kern/kern_clock.c Thu Nov 29 03:39:11 2018 (r341177) +++ head/sys/kern/kern_clock.c Thu Nov 29 03:44:02 2018 (r341178) @@ -421,6 +421,36 @@ initclocks(void *dummy) #endif } +static __noinline void +hardclock_itimer(struct thread *td, struct pstats *pstats, int cnt, int usermode) +{ + struct proc *p; + int flags; + + flags = 0; + p = td->td_proc; + if (usermode && + timevalisset(&pstats->p_timer[ITIMER_VIRTUAL].it_value)) { + PROC_ITIMLOCK(p); + if (itimerdecr(&pstats->p_timer[ITIMER_VIRTUAL], + tick * cnt) == 0) + flags |= TDF_ALRMPEND | TDF_ASTPENDING; + PROC_ITIMUNLOCK(p); + } + if (timevalisset(&pstats->p_timer[ITIMER_PROF].it_value)) { + PROC_ITIMLOCK(p); + if (itimerdecr(&pstats->p_timer[ITIMER_PROF], + tick * cnt) == 0) + flags |= TDF_PROFPEND | TDF_ASTPENDING; + PROC_ITIMUNLOCK(p); + } + if (flags != 0) { + thread_lock(td); + td->td_flags |= flags; + thread_unlock(td); + } +} + void hardclock(int cnt, int usermode) { @@ -428,15 +458,14 @@ hardclock(int cnt, int usermode) struct thread *td = curthread; struct proc *p = td->td_proc; int *t = DPCPU_PTR(pcputicks); - int flags, global, newticks; - int i; + int global, i, newticks; /* * Update per-CPU and possibly global ticks values. */ *t += cnt; + global = ticks; do { - global = ticks; newticks = *t - global; if (newticks <= 0) { if (newticks < -1) @@ -444,33 +473,16 @@ hardclock(int cnt, int usermode) newticks = 0; break; } - } while (!atomic_cmpset_int(&ticks, global, *t)); + } while (!atomic_fcmpset_int(&ticks, &global, *t)); /* * Run current process's virtual and profile time, as needed. */ pstats = p->p_stats; - flags = 0; - if (usermode && - timevalisset(&pstats->p_timer[ITIMER_VIRTUAL].it_value)) { - PROC_ITIMLOCK(p); - if (itimerdecr(&pstats->p_timer[ITIMER_VIRTUAL], - tick * cnt) == 0) - flags |= TDF_ALRMPEND | TDF_ASTPENDING; - PROC_ITIMUNLOCK(p); - } - if (timevalisset(&pstats->p_timer[ITIMER_PROF].it_value)) { - PROC_ITIMLOCK(p); - if (itimerdecr(&pstats->p_timer[ITIMER_PROF], - tick * cnt) == 0) - flags |= TDF_PROFPEND | TDF_ASTPENDING; - PROC_ITIMUNLOCK(p); - } - if (flags != 0) { - thread_lock(td); - td->td_flags |= flags; - thread_unlock(td); - } + if (__predict_false( + timevalisset(&pstats->p_timer[ITIMER_VIRTUAL].it_value) || + timevalisset(&pstats->p_timer[ITIMER_PROF].it_value))) + hardclock_itimer(td, pstats, cnt, usermode); #ifdef HWPMC_HOOKS if (PMC_CPU_HAS_SAMPLES(PCPU_GET(cpuid)))