From owner-svn-src-stable-12@freebsd.org Wed Aug 19 13:07:06 2020 Return-Path: Delivered-To: svn-src-stable-12@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1FE3B3BE2D1; Wed, 19 Aug 2020 13:07:06 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BWp2y01mJz3X6R; Wed, 19 Aug 2020 13:07:06 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D7C8323511; Wed, 19 Aug 2020 13:07:05 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 07JD754t046813; Wed, 19 Aug 2020 13:07:05 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 07JD74oO046808; Wed, 19 Aug 2020 13:07:04 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <202008191307.07JD74oO046808@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Wed, 19 Aug 2020 13:07:04 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-12@freebsd.org Subject: svn commit: r364386 - in stable/12/sys: dev/hwpmc kern netinet netinet6 sys X-SVN-Group: stable-12 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: in stable/12/sys: dev/hwpmc kern netinet netinet6 sys X-SVN-Commit-Revision: 364386 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-stable-12@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: SVN commit messages for only the 12-stable src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2020 13:07:06 -0000 Author: hselasky Date: Wed Aug 19 13:07:04 2020 New Revision: 364386 URL: https://svnweb.freebsd.org/changeset/base/364386 Log: MFC r359438 and r359477: Remove the "config" taskqgroup and its KPIs. Equivalent functionality is already provided by taskqueue(9), just use that instead. Interfaces may be detached from a taskqueue_thread task, for example by prison_complete(), so after r359438, when draining the queue we may end up deadlocking. Modified: stable/12/sys/dev/hwpmc/hwpmc_mod.c stable/12/sys/kern/subr_gtaskqueue.c stable/12/sys/netinet/in_mcast.c stable/12/sys/netinet6/in6_mcast.c stable/12/sys/sys/gtaskqueue.h Directory Properties: stable/12/ (props changed) Modified: stable/12/sys/dev/hwpmc/hwpmc_mod.c ============================================================================== --- stable/12/sys/dev/hwpmc/hwpmc_mod.c Wed Aug 19 12:51:13 2020 (r364385) +++ stable/12/sys/dev/hwpmc/hwpmc_mod.c Wed Aug 19 13:07:04 2020 (r364386) @@ -36,9 +36,9 @@ __FBSDID("$FreeBSD$"); #include +#include #include #include -#include #include #include #include @@ -63,7 +63,7 @@ __FBSDID("$FreeBSD$"); #include #include #include -#include +#include #include #include /* needs to be after */ @@ -187,7 +187,7 @@ static int pmc_threadfreelist_entries=0; /* * Task to free thread descriptors */ -static struct grouptask free_gtask; +static struct task free_task; /* * A map of row indices to classdep structures. @@ -2411,15 +2411,15 @@ pmc_thread_descriptor_pool_free(struct pmc_thread *pt) LIST_INSERT_HEAD(&pmc_threadfreelist, pt, pt_next); pmc_threadfreelist_entries++; if (pmc_threadfreelist_entries > pmc_threadfreelist_max) - GROUPTASK_ENQUEUE(&free_gtask); + taskqueue_enqueue(taskqueue_fast, &free_task); mtx_unlock_spin(&pmc_threadfreelist_mtx); } /* - * A callout to manage the free list. + * An asynchronous task to manage the free list. */ static void -pmc_thread_descriptor_pool_free_task(void *arg __unused) +pmc_thread_descriptor_pool_free_task(void *arg __unused, int pending __unused) { struct pmc_thread *pt; LIST_HEAD(, pmc_thread) tmplist; @@ -5715,11 +5715,8 @@ pmc_initialize(void) mtx_init(&pmc_threadfreelist_mtx, "pmc-threadfreelist", "pmc-leaf", MTX_SPIN); - /* - * Initialize the callout to monitor the thread free list. - * This callout will also handle the initial population of the list. - */ - taskqgroup_config_gtask_init(NULL, &free_gtask, pmc_thread_descriptor_pool_free_task, "thread descriptor pool free task"); + /* Initialize the task to prune the thread free list. */ + TASK_INIT(&free_task, 0, pmc_thread_descriptor_pool_free_task, NULL); /* register process {exit,fork,exec} handlers */ pmc_exit_tag = EVENTHANDLER_REGISTER(process_exit, @@ -5818,6 +5815,7 @@ pmc_cleanup(void) } /* reclaim allocated data structures */ + taskqueue_drain(taskqueue_fast, &free_task); mtx_destroy(&pmc_threadfreelist_mtx); pmc_thread_descriptor_pool_drain(); @@ -5825,7 +5823,6 @@ pmc_cleanup(void) mtx_pool_destroy(&pmc_mtxpool); mtx_destroy(&pmc_processhash_mtx); - taskqgroup_config_gtask_deinit(&free_gtask); if (pmc_processhash) { #ifdef HWPMC_DEBUG struct pmc_process *pp; Modified: stable/12/sys/kern/subr_gtaskqueue.c ============================================================================== --- stable/12/sys/kern/subr_gtaskqueue.c Wed Aug 19 12:51:13 2020 (r364385) +++ stable/12/sys/kern/subr_gtaskqueue.c Wed Aug 19 13:07:04 2020 (r364386) @@ -55,7 +55,6 @@ static int task_is_running(struct gtaskqueue *queue, s static void gtaskqueue_drain_locked(struct gtaskqueue *queue, struct gtask *gtask); TASKQGROUP_DEFINE(softirq, mp_ncpus, 1); -TASKQGROUP_DEFINE(config, 1, 1); struct gtaskqueue_busy { struct gtask *tb_running; @@ -891,24 +890,6 @@ taskqgroup_bind(struct taskqgroup *qgroup) } } -static void -taskqgroup_config_init(void *arg) -{ - struct taskqgroup *qgroup = qgroup_config; - LIST_HEAD(, grouptask) gtask_head = LIST_HEAD_INITIALIZER(NULL); - - LIST_SWAP(>ask_head, &qgroup->tqg_queue[0].tgc_tasks, - grouptask, gt_list); - qgroup->tqg_queue[0].tgc_cnt = 0; - taskqgroup_cpu_create(qgroup, 0, 0); - - qgroup->tqg_cnt = 1; - qgroup->tqg_stride = 1; -} - -SYSINIT(taskqgroup_config_init, SI_SUB_TASKQ, SI_ORDER_SECOND, - taskqgroup_config_init, NULL); - static int _taskqgroup_adjust(struct taskqgroup *qgroup, int cnt, int stride) { @@ -1028,21 +1009,5 @@ taskqgroup_create(const char *name) void taskqgroup_destroy(struct taskqgroup *qgroup) { - } -void -taskqgroup_config_gtask_init(void *ctx, struct grouptask *gtask, gtask_fn_t *fn, - const char *name) -{ - - GROUPTASK_INIT(gtask, 0, fn, ctx); - taskqgroup_attach(qgroup_config, gtask, gtask, -1, name); -} - -void -taskqgroup_config_gtask_deinit(struct grouptask *gtask) -{ - - taskqgroup_detach(qgroup_config, gtask); -} Modified: stable/12/sys/netinet/in_mcast.c ============================================================================== --- stable/12/sys/netinet/in_mcast.c Wed Aug 19 12:51:13 2020 (r364385) +++ stable/12/sys/netinet/in_mcast.c Wed Aug 19 13:07:04 2020 (r364386) @@ -51,7 +51,6 @@ __FBSDID("$FreeBSD$"); #include #include #include -#include #include #include @@ -223,24 +222,17 @@ inm_is_ifp_detached(const struct in_multi *inm) } #endif -static struct grouptask free_gtask; -static struct in_multi_head inm_free_list; -static void inm_release_task(void *arg __unused); -static void inm_init(void) +static struct task free_task; +static struct in_multi_head inm_free_list = SLIST_HEAD_INITIALIZER(); +static void inm_release_task(void *arg __unused, int pending __unused); + +static void +inm_init(void) { - SLIST_INIT(&inm_free_list); - taskqgroup_config_gtask_init(NULL, &free_gtask, inm_release_task, "inm release task"); + TASK_INIT(&free_task, 0, inm_release_task, NULL); } +SYSINIT(inm_init, SI_SUB_TASKQ, SI_ORDER_ANY, inm_init, NULL); -#ifdef EARLY_AP_STARTUP -SYSINIT(inm_init, SI_SUB_SMP + 1, SI_ORDER_FIRST, - inm_init, NULL); -#else -SYSINIT(inm_init, SI_SUB_ROOT_CONF - 1, SI_ORDER_FIRST, - inm_init, NULL); -#endif - - void inm_release_list_deferred(struct in_multi_head *inmh) { @@ -250,7 +242,7 @@ inm_release_list_deferred(struct in_multi_head *inmh) mtx_lock(&in_multi_free_mtx); SLIST_CONCAT(&inm_free_list, inmh, in_multi, inm_nrele); mtx_unlock(&in_multi_free_mtx); - GROUPTASK_ENQUEUE(&free_gtask); + taskqueue_enqueue(taskqueue_thread, &free_task); } void @@ -303,7 +295,7 @@ inm_release_deferred(struct in_multi *inm) } static void -inm_release_task(void *arg __unused) +inm_release_task(void *arg __unused, int pending __unused) { struct in_multi_head inm_free_tmp; struct in_multi *inm, *tinm; Modified: stable/12/sys/netinet6/in6_mcast.c ============================================================================== --- stable/12/sys/netinet6/in6_mcast.c Wed Aug 19 12:51:13 2020 (r364385) +++ stable/12/sys/netinet6/in6_mcast.c Wed Aug 19 13:07:04 2020 (r364386) @@ -41,8 +41,8 @@ __FBSDID("$FreeBSD$"); #include #include -#include #include +#include #include #include #include @@ -50,7 +50,7 @@ __FBSDID("$FreeBSD$"); #include #include #include -#include +#include #include #include @@ -59,7 +59,6 @@ __FBSDID("$FreeBSD$"); #include #include - #include #include #include @@ -510,24 +509,22 @@ in6m_release(struct in6_multi *inm) } } -static struct grouptask free_gtask; -static struct in6_multi_head in6m_free_list; -static void in6m_release_task(void *arg __unused); -static void in6m_init(void) +/* + * Interface detach can happen in a taskqueue thread context, so we must use a + * dedicated thread to avoid deadlocks when draining in6m_release tasks. + */ +TASKQUEUE_DEFINE_THREAD(in6m_free); +static struct task in6m_free_task; +static struct in6_multi_head in6m_free_list = SLIST_HEAD_INITIALIZER(); +static void in6m_release_task(void *arg __unused, int pending __unused); + +static void +in6m_init(void) { - SLIST_INIT(&in6m_free_list); - taskqgroup_config_gtask_init(NULL, &free_gtask, in6m_release_task, "in6m release task"); + TASK_INIT(&in6m_free_task, 0, in6m_release_task, NULL); } +SYSINIT(in6m_init, SI_SUB_TASKQ, SI_ORDER_ANY, in6m_init, NULL); -#ifdef EARLY_AP_STARTUP -SYSINIT(in6m_init, SI_SUB_SMP + 1, SI_ORDER_FIRST, - in6m_init, NULL); -#else -SYSINIT(in6m_init, SI_SUB_ROOT_CONF - 1, SI_ORDER_SECOND, - in6m_init, NULL); -#endif - - void in6m_release_list_deferred(struct in6_multi_head *inmh) { @@ -536,15 +533,13 @@ in6m_release_list_deferred(struct in6_multi_head *inmh mtx_lock(&in6_multi_free_mtx); SLIST_CONCAT(&in6m_free_list, inmh, in6_multi, in6m_nrele); mtx_unlock(&in6_multi_free_mtx); - GROUPTASK_ENQUEUE(&free_gtask); + taskqueue_enqueue(taskqueue_in6m_free, &in6m_free_task); } void in6m_release_wait(void) { - - /* Wait for all jobs to complete. */ - gtaskqueue_drain_all(free_gtask.gt_taskqueue); + taskqueue_drain_all(taskqueue_in6m_free); } void @@ -604,7 +599,7 @@ in6m_disconnect_locked(struct in6_multi_head *inmh, st } static void -in6m_release_task(void *arg __unused) +in6m_release_task(void *arg __unused, int pending __unused) { struct in6_multi_head in6m_free_tmp; struct in6_multi *inm, *tinm; Modified: stable/12/sys/sys/gtaskqueue.h ============================================================================== --- stable/12/sys/sys/gtaskqueue.h Wed Aug 19 12:51:13 2020 (r364385) +++ stable/12/sys/sys/gtaskqueue.h Wed Aug 19 13:07:04 2020 (r364386) @@ -64,9 +64,6 @@ void taskqgroup_detach(struct taskqgroup *qgroup, stru struct taskqgroup *taskqgroup_create(const char *name); void taskqgroup_destroy(struct taskqgroup *qgroup); int taskqgroup_adjust(struct taskqgroup *qgroup, int cnt, int stride); -void taskqgroup_config_gtask_init(void *ctx, struct grouptask *gtask, - gtask_fn_t *fn, const char *name); -void taskqgroup_config_gtask_deinit(struct grouptask *gtask); #define TASK_ENQUEUED 0x1 #define TASK_SKIP_WAKEUP 0x2