From owner-svn-src-head@freebsd.org Tue Mar 7 13:51:15 2017 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 68E4AD014EF; Tue, 7 Mar 2017 13:51:15 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 439C61195; Tue, 7 Mar 2017 13:51:15 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v27DpE59074900; Tue, 7 Mar 2017 13:51:14 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v27DpEpT074899; Tue, 7 Mar 2017 13:51:14 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201703071351.v27DpEpT074899@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Tue, 7 Mar 2017 13:51:14 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r314853 - head/sys/compat/linuxkpi/common/src X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Mar 2017 13:51:15 -0000 Author: hselasky Date: Tue Mar 7 13:51:14 2017 New Revision: 314853 URL: https://svnweb.freebsd.org/changeset/base/314853 Log: Use grouptaskqueue for tasklets in the LinuxKPI. This avoids creating own per-CPU threads and also ensures the tasklet execution happens on the same CPU core invoking the tasklet. MFC after: 1 week Sponsored by: Mellanox Technologies Modified: head/sys/compat/linuxkpi/common/src/linux_tasklet.c Modified: head/sys/compat/linuxkpi/common/src/linux_tasklet.c ============================================================================== --- head/sys/compat/linuxkpi/common/src/linux_tasklet.c Tue Mar 7 13:41:01 2017 (r314852) +++ head/sys/compat/linuxkpi/common/src/linux_tasklet.c Tue Mar 7 13:51:14 2017 (r314853) @@ -29,12 +29,11 @@ __FBSDID("$FreeBSD$"); #include #include -#include +#include #include #include #include -#include #include #define TASKLET_ST_IDLE 0 @@ -54,17 +53,16 @@ __FBSDID("$FreeBSD$"); struct tasklet_worker { struct mtx mtx; TAILQ_HEAD(, tasklet_struct) head; - struct taskqueue *taskqueue; - struct task task; + struct grouptask gtask; } __aligned(CACHE_LINE_SIZE); #define TASKLET_WORKER_LOCK(tw) mtx_lock(&(tw)->mtx) #define TASKLET_WORKER_UNLOCK(tw) mtx_unlock(&(tw)->mtx) -static struct tasklet_worker tasklet_worker; +static DPCPU_DEFINE(struct tasklet_worker, tasklet_worker); static void -tasklet_handler(void *arg, int pending) +tasklet_handler(void *arg) { struct tasklet_worker *tw = (struct tasklet_worker *)arg; struct tasklet_struct *ts; @@ -72,7 +70,6 @@ tasklet_handler(void *arg, int pending) linux_set_current(curthread); TASKLET_WORKER_LOCK(tw); - local_bh_disable(); /* pin thread to CPU */ while (1) { ts = TAILQ_FIRST(&tw->head); if (ts == NULL) @@ -89,32 +86,47 @@ tasklet_handler(void *arg, int pending) } while (TASKLET_ST_CMPSET(ts, TASKLET_ST_EXEC, TASKLET_ST_IDLE) == 0); TASKLET_WORKER_LOCK(tw); } - local_bh_enable(); /* unpin thread from CPU */ TASKLET_WORKER_UNLOCK(tw); } static void tasklet_subsystem_init(void *arg __unused) { - struct tasklet_worker *tw = &tasklet_worker; - - tw->taskqueue = taskqueue_create("tasklet", M_WAITOK, - taskqueue_thread_enqueue, &tw->taskqueue); - mtx_init(&tw->mtx, "linux_tasklet", NULL, MTX_DEF); - TAILQ_INIT(&tw->head); - TASK_INIT(&tw->task, 0, tasklet_handler, tw); - taskqueue_start_threads(&tw->taskqueue, 1, PI_NET, "tasklet"); + struct tasklet_worker *tw; + char buf[32]; + int i; + + CPU_FOREACH(i) { + if (CPU_ABSENT(i)) + continue; + + tw = DPCPU_ID_PTR(i, tasklet_worker); + + mtx_init(&tw->mtx, "linux_tasklet", NULL, MTX_DEF); + TAILQ_INIT(&tw->head); + GROUPTASK_INIT(&tw->gtask, 0, tasklet_handler, tw); + snprintf(buf, sizeof(buf), "softirq%d", i); + taskqgroup_attach_cpu(qgroup_softirq, &tw->gtask, + "tasklet", i, -1, buf); + } } SYSINIT(linux_tasklet, SI_SUB_INIT_IF, SI_ORDER_THIRD, tasklet_subsystem_init, NULL); static void tasklet_subsystem_uninit(void *arg __unused) { - struct tasklet_worker *tw = &tasklet_worker; + struct tasklet_worker *tw; + int i; + + CPU_FOREACH(i) { + if (CPU_ABSENT(i)) + continue; + + tw = DPCPU_ID_PTR(i, tasklet_worker); - taskqueue_free(tw->taskqueue); - tw->taskqueue = NULL; - mtx_destroy(&tw->mtx); + taskqgroup_detach(qgroup_softirq, &tw->gtask); + mtx_destroy(&tw->mtx); + } } SYSUNINIT(linux_tasklet, SI_SUB_INIT_IF, SI_ORDER_THIRD, tasklet_subsystem_uninit, NULL); @@ -147,14 +159,16 @@ tasklet_schedule(struct tasklet_struct * if (TASKLET_ST_CMPSET(ts, TASKLET_ST_EXEC, TASKLET_ST_LOOP)) { /* tasklet_handler() will loop */ } else if (TASKLET_ST_CMPSET(ts, TASKLET_ST_IDLE, TASKLET_ST_BUSY)) { - struct tasklet_worker *tw = &tasklet_worker; + struct tasklet_worker *tw; + + tw = &DPCPU_GET(tasklet_worker); /* tasklet_handler() was not queued */ TASKLET_WORKER_LOCK(tw); /* enqueue tasklet */ TAILQ_INSERT_TAIL(&tw->head, ts, entry); /* schedule worker */ - taskqueue_enqueue(tw->taskqueue, &tw->task); + GROUPTASK_ENQUEUE(&tw->gtask); TASKLET_WORKER_UNLOCK(tw); } else { /*