From owner-svn-src-head@freebsd.org Wed Feb 22 09:23:40 2017 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 70028CE8326; Wed, 22 Feb 2017 09:23:40 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [88.99.82.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1EAA11529; Wed, 22 Feb 2017 09:23:39 +0000 (UTC) (envelope-from hps@selasky.org) Received: from hps2016.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 4D1531FE086; Wed, 22 Feb 2017 10:23:29 +0100 (CET) Subject: Re: svn commit: r314043 - in head/sys: compat/linuxkpi/common/include/linux compat/linuxkpi/common/src conf modules/linuxkpi To: Adrian Chadd References: <201702211323.v1LDNrJw087417@repo.freebsd.org> Cc: "src-committers@freebsd.org" , "svn-src-all@freebsd.org" , "svn-src-head@freebsd.org" From: Hans Petter Selasky Message-ID: <4ffe93f6-f08a-070c-918d-c0dad443aa1f@selasky.org> Date: Wed, 22 Feb 2017 10:22:54 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/mixed; boundary="------------82C3837B9CA671DC6CB3822A" X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Feb 2017 09:23:40 -0000 This is a multi-part message in MIME format. --------------82C3837B9CA671DC6CB3822A Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit On 02/22/17 00:18, Adrian Chadd wrote: > Hiya, > > My understanding of tasklets is that they run on the CPU that they > were scheduled on, rather than there being a single tasklet thread? > > Is that the direction you're thinking of heading in, or? > Hi Adrian, The plan is to use grouptaskqueue in the end. From what I can see all the pieces are in place for that. Then the tasklets will run on the same CPU that they were scheduled on. Some patches needs to go into the grouptaskqueue first: 1) grouptaskqueue needs to support LinuxKPI module unload. There is current missing/unimplemented drain logic in grouptaskqueue APIs. 2) Needs to expose internal gtaskqueues, so that we don't create unneccesary threads to handle a single IRQ, which is the case currently. I've already CC'ed a few people on this, but not action yet. If you can help, would be great. BTW: TASKQGROUP_DECLARE(net); in sys/gtaskqueue.h seems non-existing and should be removed. --HPS --------------82C3837B9CA671DC6CB3822A Content-Type: text/x-patch; name="qgroup_if_io_tqg.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="qgroup_if_io_tqg.diff" commit 0211693a723ce87f184d950d5a3ac5c2b306d418 Author: Matt Macy Date: Wed Aug 10 18:20:58 2016 -0700 make per-cpu grouptaskqueue threads globally visible diff --git a/sys/kern/subr_gtaskqueue.c b/sys/kern/subr_gtaskqueue.c index 2d655bd..3aecff3 100644 --- a/sys/kern/subr_gtaskqueue.c +++ b/sys/kern/subr_gtaskqueue.c @@ -52,6 +52,7 @@ static MALLOC_DEFINE(M_GTASKQUEUE, "taskqueue", "Task Queues"); static void gtaskqueue_thread_enqueue(void *); static void gtaskqueue_thread_loop(void *arg); +TASKQGROUP_DEFINE(softirq, mp_ncpus, 1); struct gtaskqueue_busy { struct gtask *tb_running; diff --git a/sys/net/iflib.c b/sys/net/iflib.c index 5938aca..7cb00c8 100644 --- a/sys/net/iflib.c +++ b/sys/net/iflib.c @@ -93,7 +93,6 @@ __FBSDID("$FreeBSD$"); #include #endif - /* * enable accounting of every mbuf as it comes in to and goes out of iflib's software descriptor references */ @@ -480,7 +479,6 @@ MODULE_VERSION(iflib, 1); MODULE_DEPEND(iflib, pci, 1, 1, 1); MODULE_DEPEND(iflib, ether, 1, 1, 1); -TASKQGROUP_DEFINE(if_io_tqg, mp_ncpus, 1); TASKQGROUP_DEFINE(if_config_tqg, 1, 1); #ifndef IFLIB_DEBUG_COUNTERS @@ -3706,7 +3704,7 @@ iflib_device_deregister(if_ctx_t ctx) if (ctx->ifc_led_dev != NULL) led_destroy(ctx->ifc_led_dev); /* XXX drain any dependent tasks */ - tqg = qgroup_if_io_tqg; + tqg = qgroup_softirq; for (txq = ctx->ifc_txqs, i = 0; i < NTXQSETS(ctx); i++, txq++) { callout_drain(&txq->ift_timer); callout_drain(&txq->ift_db_check); @@ -4328,7 +4326,7 @@ iflib_irq_alloc_generic(if_ctx_t ctx, if_irq_t irq, int rid, q = &ctx->ifc_txqs[qid]; info = &ctx->ifc_txqs[qid].ift_filter_info; gtask = &ctx->ifc_txqs[qid].ift_task; - tqg = qgroup_if_io_tqg; + tqg = qgroup_softirq; tqrid = irq->ii_rid; fn = _task_fn_tx; break; @@ -4336,7 +4334,7 @@ iflib_irq_alloc_generic(if_ctx_t ctx, if_irq_t irq, int rid, q = &ctx->ifc_rxqs[qid]; info = &ctx->ifc_rxqs[qid].ifr_filter_info; gtask = &ctx->ifc_rxqs[qid].ifr_task; - tqg = qgroup_if_io_tqg; + tqg = qgroup_softirq; tqrid = irq->ii_rid; fn = _task_fn_rx; break; @@ -4384,13 +4382,13 @@ iflib_softirq_alloc_generic(if_ctx_t ctx, int rid, iflib_intr_type_t type, void case IFLIB_INTR_TX: q = &ctx->ifc_txqs[qid]; gtask = &ctx->ifc_txqs[qid].ift_task; - tqg = qgroup_if_io_tqg; + tqg = qgroup_softirq; fn = _task_fn_tx; break; case IFLIB_INTR_RX: q = &ctx->ifc_rxqs[qid]; gtask = &ctx->ifc_rxqs[qid].ifr_task; - tqg = qgroup_if_io_tqg; + tqg = qgroup_softirq; fn = _task_fn_rx; break; case IFLIB_INTR_ADMIN: @@ -4441,7 +4439,7 @@ iflib_legacy_setup(if_ctx_t ctx, driver_filter_t filter, void *filter_arg, int * q = &ctx->ifc_rxqs[0]; info = &rxq[0].ifr_filter_info; gtask = &rxq[0].ifr_task; - tqg = qgroup_if_io_tqg; + tqg = qgroup_softirq; tqrid = irq->ii_rid = *rid; fn = _task_fn_rx; @@ -4457,7 +4455,7 @@ iflib_legacy_setup(if_ctx_t ctx, driver_filter_t filter, void *filter_arg, int * taskqgroup_attach(tqg, gtask, q, tqrid, name); GROUPTASK_INIT(&txq->ift_task, 0, _task_fn_tx, txq); - taskqgroup_attach(qgroup_if_io_tqg, &txq->ift_task, txq, tqrid, "tx"); + taskqgroup_attach(qgroup_softirq, &txq->ift_task, txq, tqrid, "tx"); GROUPTASK_INIT(&ctx->ifc_admin_task, 0, _task_fn_admin, ctx); taskqgroup_attach(qgroup_if_config_tqg, &ctx->ifc_admin_task, ctx, -1, "admin/link"); @@ -4504,7 +4502,7 @@ void iflib_io_tqg_attach(struct grouptask *gt, void *uniq, int cpu, char *name) { - taskqgroup_attach_cpu(qgroup_if_io_tqg, gt, uniq, cpu, -1, name); + taskqgroup_attach_cpu(qgroup_softirq, gt, uniq, cpu, -1, name); } void diff --git a/sys/sys/gtaskqueue.h b/sys/sys/gtaskqueue.h index 88d4b54..cd0f774 100644 --- a/sys/sys/gtaskqueue.h +++ b/sys/sys/gtaskqueue.h @@ -121,5 +121,6 @@ SYSINIT(taskqgroup_adj_##name, SI_SUB_SMP, SI_ORDER_ANY, \ struct __hack #endif TASKQGROUP_DECLARE(net); +TASKQGROUP_DECLARE(softirq); #endif /* !_SYS_GTASKQUEUE_H_ */ --------------82C3837B9CA671DC6CB3822A--