From owner-svn-src-projects@FreeBSD.ORG Fri Dec 14 21:22:24 2012 Return-Path: Delivered-To: svn-src-projects@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 274A9872; Fri, 14 Dec 2012 21:22:24 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) by mx1.freebsd.org (Postfix) with ESMTP id 0DACD8FC13; Fri, 14 Dec 2012 21:22:24 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.5/8.14.5) with ESMTP id qBELMNj8048724; Fri, 14 Dec 2012 21:22:23 GMT (envelope-from mav@svn.freebsd.org) Received: (from mav@localhost) by svn.freebsd.org (8.14.5/8.14.5/Submit) id qBELMNJV048723; Fri, 14 Dec 2012 21:22:23 GMT (envelope-from mav@svn.freebsd.org) Message-Id: <201212142122.qBELMNJV048723@svn.freebsd.org> From: Alexander Motin Date: Fri, 14 Dec 2012 21:22:23 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r244225 - projects/calloutng/sys/kern X-SVN-Group: projects MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Dec 2012 21:22:24 -0000 Author: mav Date: Fri Dec 14 21:22:23 2012 New Revision: 244225 URL: http://svnweb.freebsd.org/changeset/base/244225 Log: Add some simple adaptiveness to the maximal next even lookup depth. If callout_process() was called two times within the same millisecond, limit lookup by 1/16s instead of default 1/2s. On the amd64 test system it reduces usleep(1) time from 8us to 6us, handling about 180K callouts and thread wakeups per second in one stream with not full CPU load. That is the only price of staying with the callwheel data structure. Unless at some point we want to sleep/lookup for minutes, it doesn't look high. Modified: projects/calloutng/sys/kern/kern_timeout.c Modified: projects/calloutng/sys/kern/kern_timeout.c ============================================================================== --- projects/calloutng/sys/kern/kern_timeout.c Fri Dec 14 18:30:01 2012 (r244224) +++ projects/calloutng/sys/kern/kern_timeout.c Fri Dec 14 21:22:23 2012 (r244225) @@ -387,7 +387,8 @@ callout_process(struct bintime *now) struct callout *tmp; struct callout_cpu *cc; struct callout_tailq *sc; - int cpu, depth_dir, firstb, mpcalls_dir, lastb, nowb, lockcalls_dir, + uint64_t lookahead; + int depth_dir, firstb, mpcalls_dir, lastb, nowb, lockcalls_dir, need_softclock, exit_allowed, exit_wanted; need_softclock = 0; @@ -396,16 +397,21 @@ callout_process(struct bintime *now) lockcalls_dir = 0; cc = CC_SELF(); mtx_lock_spin_flags(&cc->cc_lock, MTX_QUIET); - cpu = curcpu; /* Compute the buckets of the last scan and present times. */ firstb = callout_hash(&cc->cc_lastscan); nowb = callout_hash(now); /* Compute the last bucket and minimum time of the bucket after it. */ + if (nowb == firstb) + lookahead = 1LLU << 60; /* 1/16s */ + else if (nowb - firstb == 1) + lookahead = 1LLU << 61; /* 1/8s */ + else + lookahead = 1LLU << 63; /* 1/2s */ first = last = *now; - bintime_addx(&first, (uint64_t)3 << (64 - 3)); /* 0.37s */ - bintime_addx(&last, (uint64_t)3 << (64 - 2)); /* 0.75s */ + bintime_addx(&first, lookahead / 2); + bintime_addx(&last, lookahead); last.frac &= (0xffffffffffffffffLLU << (64 - CC_HASH_SHIFT)); lastb = callout_hash(&last) - 1; max = last; @@ -429,9 +435,6 @@ callout_process(struct bintime *now) sc = &cc->cc_callwheel[firstb]; tmp = TAILQ_FIRST(sc); while (tmp != NULL) { - /* Compute allowed time range for the event */ - tmp_max = tmp->c_time; - bintime_add(&tmp_max, &tmp->c_precision); /* Run the callout if present time within allowed. */ if (bintime_cmp(&tmp->c_time, now, <=)) { /* @@ -471,6 +474,8 @@ callout_process(struct bintime *now) /* Update first and last time, respecting this event. */ if (bintime_cmp(&tmp->c_time, &first, <)) first = tmp->c_time; + tmp_max = tmp->c_time; + bintime_add(&tmp_max, &tmp->c_precision); if (bintime_cmp(&tmp_max, &last, <)) last = tmp_max; next: @@ -492,7 +497,7 @@ next: } cc->cc_exec_next_dir = NULL; if (callout_new_inserted != NULL) - (*callout_new_inserted)(cpu, last, first); + (*callout_new_inserted)(curcpu, last, first); cc->cc_firstevent = last; cc->cc_lastscan = *now; #ifdef CALLOUT_PROFILING