From owner-svn-src-head@freebsd.org Fri May 11 08:16:57 2018 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 15139FC823D; Fri, 11 May 2018 08:16:57 +0000 (UTC) (envelope-from mmacy@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BBA157C4A0; Fri, 11 May 2018 08:16:56 +0000 (UTC) (envelope-from mmacy@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9CCE64E70; Fri, 11 May 2018 08:16:56 +0000 (UTC) (envelope-from mmacy@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w4B8Guti072679; Fri, 11 May 2018 08:16:56 GMT (envelope-from mmacy@FreeBSD.org) Received: (from mmacy@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w4B8Guw4072678; Fri, 11 May 2018 08:16:56 GMT (envelope-from mmacy@FreeBSD.org) Message-Id: <201805110816.w4B8Guw4072678@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mmacy set sender to mmacy@FreeBSD.org using -f From: Matt Macy Date: Fri, 11 May 2018 08:16:56 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r333485 - head/sys/kern X-SVN-Group: head X-SVN-Commit-Author: mmacy X-SVN-Commit-Paths: head/sys/kern X-SVN-Commit-Revision: 333485 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 May 2018 08:16:57 -0000 Author: mmacy Date: Fri May 11 08:16:56 2018 New Revision: 333485 URL: https://svnweb.freebsd.org/changeset/base/333485 Log: epoch(9): callback task fixes - initialize the pcpu STAILQ in the NUMA case - don't enqueue the callback task if there isn't sufficient work to be done Reported by: pho@ Approved by: sbruno@ Modified: head/sys/kern/subr_epoch.c Modified: head/sys/kern/subr_epoch.c ============================================================================== --- head/sys/kern/subr_epoch.c Fri May 11 07:04:57 2018 (r333484) +++ head/sys/kern/subr_epoch.c Fri May 11 08:16:56 2018 (r333485) @@ -108,6 +108,8 @@ struct epoch { int e_flags; /* make sure that immutable data doesn't overlap with the gtask, callout, and mutex*/ struct epoch_pcpu_state *e_pcpu_dom[MAXMEMDOM] __aligned(EPOCH_ALIGN); + counter_u64_t e_frees; + uint64_t e_free_last; struct epoch_pcpu_state *e_pcpu[0]; }; @@ -172,6 +174,7 @@ epoch_init_numa(epoch_t epoch) for (int i = 0; i < domcount[domain]; i++, eps++) { epoch->e_pcpu[cpu_offset + i] = eps; er = &eps->eps_record; + STAILQ_INIT(&eps->eps_cblist); ck_epoch_register(&epoch->e_epoch, &er->er_record, NULL); TAILQ_INIT((struct threadlist *)(uintptr_t)&er->er_tdlist); er->er_cpuid = cpu_offset + i; @@ -201,9 +204,15 @@ static void epoch_callout(void *arg) { epoch_t epoch; + uint64_t frees; epoch = arg; - GROUPTASK_ENQUEUE(&epoch->e_gtask); + frees = counter_u64_fetch(epoch->e_frees); + /* pick some better value */ + if (frees - epoch->e_free_last > 10) { + GROUPTASK_ENQUEUE(&epoch->e_gtask); + epoch->e_free_last = frees; + } if ((epoch->e_flags & EPOCH_EXITING) == 0) callout_reset(&epoch->e_timer, poll_intvl, epoch_callout, epoch); } @@ -218,6 +227,7 @@ epoch_alloc(void) epoch = malloc(sizeof(struct epoch) + mp_ncpus*sizeof(void*), M_EPOCH, M_ZERO|M_WAITOK); ck_epoch_init(&epoch->e_epoch); + epoch->e_frees = counter_u64_alloc(M_WAITOK); mtx_init(&epoch->e_lock, "epoch callout", NULL, MTX_DEF); callout_init_mtx(&epoch->e_timer, &epoch->e_lock, 0); taskqgroup_config_gtask_init(epoch, &epoch->e_gtask, epoch_call_task, "epoch call task"); @@ -252,6 +262,7 @@ epoch_free(epoch_t epoch) gtaskqueue_drain(epoch->e_gtask.gt_taskqueue, &epoch->e_gtask.gt_task); callout_drain(&epoch->e_timer); mtx_destroy(&epoch->e_lock); + counter_u64_free(epoch->e_frees); taskqgroup_config_gtask_deinit(&epoch->e_gtask); if (usedomains) for (domain = 0; domain < vm_ndomains; domain++) @@ -534,6 +545,7 @@ epoch_call(epoch_t epoch, epoch_context_t ctx, void (* MPASS(epoch); MPASS(callback); cb->ec_callback = callback; + counter_u64_add(epoch->e_frees, 1); critical_enter(); eps = epoch->e_pcpu[curcpu]; STAILQ_INSERT_HEAD(&eps->eps_cblist, cb, ec_link);