From owner-svn-src-projects@FreeBSD.ORG Mon Dec 17 20:36:58 2012 Return-Path: Delivered-To: svn-src-projects@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BE6E5D72; Mon, 17 Dec 2012 20:36:58 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) by mx1.freebsd.org (Postfix) with ESMTP id A36328FC0A; Mon, 17 Dec 2012 20:36:58 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.5/8.14.5) with ESMTP id qBHKawWr064983; Mon, 17 Dec 2012 20:36:58 GMT (envelope-from mav@svn.freebsd.org) Received: (from mav@localhost) by svn.freebsd.org (8.14.5/8.14.5/Submit) id qBHKavP2064975; Mon, 17 Dec 2012 20:36:57 GMT (envelope-from mav@svn.freebsd.org) Message-Id: <201212172036.qBHKavP2064975@svn.freebsd.org> From: Alexander Motin Date: Mon, 17 Dec 2012 20:36:57 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r244364 - in projects/calloutng/sys: kern sys X-SVN-Group: projects MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Dec 2012 20:36:58 -0000 Author: mav Date: Mon Dec 17 20:36:56 2012 New Revision: 244364 URL: http://svnweb.freebsd.org/changeset/base/244364 Log: Experiments with dummynet exposed number of problems in code supporting legacy tick-based callouts. "callou_reset(... , 1, ...)", used by dummynet, effectively means "call me on the next hardclock tick after now". But our new world order had no concept of the "next tick", and concept of "now" was also complicated by using imprecise getbinuptime(). Different attempts to handle that gave either low resolution, or lack of event aggregation. In all cases resolution was limited by 1ms of getbinuptime(), that made at least useless setting hz above 1000. To fix that, new callout code was made to get the time of the hardclock() call directly from the kern_eventtimer.c, where it is already present with full precision. That fixed all above problems at the same time: all legacy callouts are now sychronized and so will aggregate with hardclock events, and because of having precise time values hz above 1000 are again usable. In addition to that, create and use new global variable tc_tick_bt, representing duration of the timecounter tick, that is bigger then duration of hz tick (tick_bt), when hz is set above 1000; Reviewed by: davide Modified: projects/calloutng/sys/kern/kern_clocksource.c projects/calloutng/sys/kern/kern_tc.c projects/calloutng/sys/kern/kern_time.c projects/calloutng/sys/kern/kern_timeout.c projects/calloutng/sys/kern/subr_param.c projects/calloutng/sys/kern/sys_generic.c projects/calloutng/sys/sys/time.h Modified: projects/calloutng/sys/kern/kern_clocksource.c ============================================================================== --- projects/calloutng/sys/kern/kern_clocksource.c Mon Dec 17 19:34:27 2012 (r244363) +++ projects/calloutng/sys/kern/kern_clocksource.c Mon Dec 17 20:36:56 2012 (r244364) @@ -96,7 +96,6 @@ static struct mtx et_hw_mtx; static struct eventtimer *timer = NULL; static struct bintime timerperiod; /* Timer period for periodic mode. */ -static struct bintime hardperiod; /* hardclock() events period. */ static struct bintime statperiod; /* statclock() events period. */ static struct bintime profperiod; /* profclock() events period. */ static struct bintime nexttick; /* Next global timer tick time. */ @@ -146,6 +145,7 @@ struct pcpu_state { }; static DPCPU_DEFINE(struct pcpu_state, timerstate); +DPCPU_DEFINE(struct bintime, hardclocktime); /* * Timer broadcast IPI handler. @@ -174,7 +174,7 @@ hardclockintr(void) static int handleevents(struct bintime *now, int fake) { - struct bintime t; + struct bintime t, *hct; struct trapframe *frame; struct pcpu_state *state; uintfptr_t pc; @@ -199,10 +199,13 @@ handleevents(struct bintime *now, int fa runs = 0; while (bintime_cmp(now, &state->nexthard, >=)) { - bintime_addx(&state->nexthard, hardperiod.frac); + bintime_addx(&state->nexthard, tick_bt.frac); runs++; } if (runs) { + hct = DPCPU_PTR(hardclocktime); + *hct = state->nexthard; + bintime_sub(hct, &tick_bt); if ((timer->et_flags & ET_FLAGS_PERCPU) == 0 && bintime_cmp(&state->nexthard, &nexthard, >)) nexthard = state->nexthard; @@ -282,7 +285,7 @@ getnextcpuevent(struct bintime *event, i if (curcpu == CPU_FIRST() && tc_min_ticktock_freq > hardfreq) hardfreq = tc_min_ticktock_freq; if (hz > hardfreq) { - tmp = hardperiod; + tmp = tick_bt; bintime_mul(&tmp, hz / hardfreq - 1); bintime_add(event, &tmp); } @@ -698,7 +701,7 @@ cpu_initclocks_bsp(void) profhz = round_freq(timer, stathz * 64); } tick = 1000000 / hz; - FREQ2BT(hz, &hardperiod); + FREQ2BT(hz, &tick_bt); FREQ2BT(stathz, &statperiod); FREQ2BT(profhz, &profperiod); ET_LOCK(); Modified: projects/calloutng/sys/kern/kern_tc.c ============================================================================== --- projects/calloutng/sys/kern/kern_tc.c Mon Dec 17 19:34:27 2012 (r244363) +++ projects/calloutng/sys/kern/kern_tc.c Mon Dec 17 20:36:56 2012 (r244364) @@ -121,7 +121,7 @@ SYSCTL_INT(_kern_timecounter, OID_AUTO, ×tepwarnings, 0, "Log time steps"); struct bintime bt_timethreshold; -struct bintime tick_bt; +struct bintime tc_tick_bt; int tc_timeexp; int tc_timepercentage = TC_DEFAULTPERC; TUNABLE_INT("kern.timecounter.alloweddeviation", &tc_timepercentage); @@ -1772,8 +1772,9 @@ inittimecounter(void *dummy) else tc_tick = 1; tc_adjprecision(); + FREQ2BT(hz, &tick_bt); tick_rate = hz / tc_tick; - FREQ2BT(tick_rate, &tick_bt); + FREQ2BT(tick_rate, &tc_tick_bt); p = (tc_tick * 1000000) / hz; printf("Timecounters tick every %d.%03u msec\n", p / 1000, p % 1000); Modified: projects/calloutng/sys/kern/kern_time.c ============================================================================== --- projects/calloutng/sys/kern/kern_time.c Mon Dec 17 19:34:27 2012 (r244363) +++ projects/calloutng/sys/kern/kern_time.c Mon Dec 17 20:36:56 2012 (r244364) @@ -494,7 +494,7 @@ kern_nanosleep(struct thread *td, struct bt_prec = tmp; bintime_divpow2(&bt_prec, tc_timeexp); if (TIMESEL(&bt, &tmp)) - bintime_add(&bt, &tick_bt); + bintime_add(&bt, &tc_tick_bt); bintime_add(&bt, &tmp); error = tsleep_bt(&nanowait, PWAIT | PCATCH, "nanslp", &bt, &bt_prec); TIMESEL(&btt, &tmp); Modified: projects/calloutng/sys/kern/kern_timeout.c ============================================================================== --- projects/calloutng/sys/kern/kern_timeout.c Mon Dec 17 19:34:27 2012 (r244363) +++ projects/calloutng/sys/kern/kern_timeout.c Mon Dec 17 20:36:56 2012 (r244364) @@ -39,6 +39,9 @@ __FBSDID("$FreeBSD$"); #include "opt_callout_profiling.h" #include "opt_kdtrace.h" +#if defined(__arm__) +#include "opt_timer.h" +#endif #include #include @@ -872,6 +875,10 @@ callout_handle_init(struct callout_handl handle->callout = NULL; } +#ifndef NO_EVENTTIMERS +DPCPU_DECLARE(struct bintime, hardclocktime); +#endif + /* * New interface; clients allocate their own callout structures. * @@ -893,26 +900,34 @@ _callout_reset_on(struct callout *c, str struct bintime *precision, int to_ticks, void (*ftn)(void *), void *arg, int cpu, int flags) { - struct bintime now, to_bt, pr; + struct bintime to_bt, pr; struct callout_cpu *cc; int bucket, cancelled, direct; cancelled = 0; if (bt == NULL) { - pr = to_bt = tick_bt; - getbinuptime(&now); +#ifdef NO_EVENTTIMERS + getbinuptime(&to_bt); + /* Add safety belt for the case of hz > 1000. */ + bintime_addx(&to_bt, tc_tick_bt.frac - tick_bt.frac); +#else + /* + * Obtain the time of the last hardclock() call on this CPU + * directly from the kern_clocksource.c. This value is + * per-CPU, but it is equal for all active ones. + */ + spinlock_enter(); + to_bt = DPCPU_GET(hardclocktime); + spinlock_exit(); +#endif + pr = tick_bt; if (to_ticks > 1) - bintime_mul(&to_bt, to_ticks); - bintime_add(&to_bt, &now); - if (C_PRELGET(flags) < 0) { - pr = tick_bt; - } else { - to_ticks >>= C_PRELGET(flags); - if (to_ticks == 0) - pr = tick_bt; - else - bintime_mul(&pr, to_ticks); - } + bintime_mul(&pr, to_ticks); + bintime_add(&to_bt, &pr); + if (C_PRELGET(flags) < 0) + bintime_clear(&pr); + else + bintime_divpow2(&pr, C_PRELGET(flags)); } else { to_bt = *bt; if (precision != NULL) Modified: projects/calloutng/sys/kern/subr_param.c ============================================================================== --- projects/calloutng/sys/kern/subr_param.c Mon Dec 17 19:34:27 2012 (r244363) +++ projects/calloutng/sys/kern/subr_param.c Mon Dec 17 20:36:56 2012 (r244364) @@ -81,8 +81,9 @@ __FBSDID("$FreeBSD$"); static int sysctl_kern_vm_guest(SYSCTL_HANDLER_ARGS); -int hz; -int tick; +int hz; /* system clock's frequency */ +int tick; /* usec per tick (1000000 / hz) */ +struct bintime tick_bt; /* bintime per tick (1s / hz) */ int maxusers; /* base tunable */ int maxproc; /* maximum # of processes */ int maxprocperuid; /* max # of procs per user */ @@ -219,6 +220,7 @@ init_param1(void) if (hz == -1) hz = vm_guest > VM_GUEST_NO ? HZ_VM : HZ; tick = 1000000 / hz; + FREQ2BT(hz, &tick_bt); #ifdef VM_SWZONE_SIZE_MAX maxswzone = VM_SWZONE_SIZE_MAX; Modified: projects/calloutng/sys/kern/sys_generic.c ============================================================================== --- projects/calloutng/sys/kern/sys_generic.c Mon Dec 17 19:34:27 2012 (r244363) +++ projects/calloutng/sys/kern/sys_generic.c Mon Dec 17 20:36:56 2012 (r244364) @@ -1008,7 +1008,7 @@ kern_select(struct thread *td, int nd, f precision = abt; bintime_divpow2(&precision, tc_timeexp); if (TIMESEL(&rbt, &abt)) - bintime_add(&abt, &tick_bt); + bintime_add(&abt, &tc_tick_bt); bintime_add(&abt, &rbt); } else { abt.sec = 0; @@ -1290,7 +1290,7 @@ sys_poll(td, uap) precision = abt; bintime_divpow2(&precision, tc_timeexp); if (TIMESEL(&rbt, &abt)) - bintime_add(&abt, &tick_bt); + bintime_add(&abt, &tc_tick_bt); bintime_add(&abt, &rbt); } else { abt.sec = 0; Modified: projects/calloutng/sys/sys/time.h ============================================================================== --- projects/calloutng/sys/sys/time.h Mon Dec 17 19:34:27 2012 (r244363) +++ projects/calloutng/sys/sys/time.h Mon Dec 17 20:36:56 2012 (r244364) @@ -300,6 +300,7 @@ extern time_t time_second; extern time_t time_uptime; extern struct bintime boottimebin; extern struct bintime tick_bt; +extern struct bintime tc_tick_bt; extern struct timeval boottime; extern int tc_timeexp; extern int tc_timepercentage;