From owner-svn-src-all@freebsd.org Thu Aug 9 05:18:29 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EDA451055339; Thu, 9 Aug 2018 05:18:28 +0000 (UTC) (envelope-from mmacy@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9E06780764; Thu, 9 Aug 2018 05:18:28 +0000 (UTC) (envelope-from mmacy@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 747F415017; Thu, 9 Aug 2018 05:18:28 +0000 (UTC) (envelope-from mmacy@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w795IS5l071452; Thu, 9 Aug 2018 05:18:28 GMT (envelope-from mmacy@FreeBSD.org) Received: (from mmacy@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w795ISxD071451; Thu, 9 Aug 2018 05:18:28 GMT (envelope-from mmacy@FreeBSD.org) Message-Id: <201808090518.w795ISxD071451@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mmacy set sender to mmacy@FreeBSD.org using -f From: Matt Macy Date: Thu, 9 Aug 2018 05:18:28 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337525 - head/sys/kern X-SVN-Group: head X-SVN-Commit-Author: mmacy X-SVN-Commit-Paths: head/sys/kern X-SVN-Commit-Revision: 337525 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Aug 2018 05:18:29 -0000 Author: mmacy Date: Thu Aug 9 05:18:27 2018 New Revision: 337525 URL: https://svnweb.freebsd.org/changeset/base/337525 Log: epoch_block_wait: don't check TD_RUNNING struct epoch_thread is not type safe (stack allocated) and thus cannot be dereferenced from another CPU Reported by: novel@ Modified: head/sys/kern/subr_epoch.c Modified: head/sys/kern/subr_epoch.c ============================================================================== --- head/sys/kern/subr_epoch.c Thu Aug 9 03:45:47 2018 (r337524) +++ head/sys/kern/subr_epoch.c Thu Aug 9 05:18:27 2018 (r337525) @@ -56,7 +56,7 @@ __FBSDID("$FreeBSD$"); static MALLOC_DEFINE(M_EPOCH, "epoch", "epoch based reclamation"); /* arbitrary --- needs benchmarking */ -#define MAX_ADAPTIVE_SPIN 1000 +#define MAX_ADAPTIVE_SPIN 100 #define MAX_EPOCHS 64 CTASSERT(sizeof(ck_epoch_entry_t) == sizeof(struct epoch_context)); @@ -240,24 +240,38 @@ epoch_block_handler_preempt(struct ck_epoch *global __ locksheld = td->td_locks; spincount = 0; counter_u64_add(block_count, 1); + /* + * We lost a race and there's no longer any threads + * on the CPU in an epoch section. + */ + if (TAILQ_EMPTY(&record->er_tdlist)) + return; + if (record->er_cpuid != curcpu) { /* * If the head of the list is running, we can wait for it * to remove itself from the list and thus save us the * overhead of a migration */ - if ((tdwait = TAILQ_FIRST(&record->er_tdlist)) != NULL && - TD_IS_RUNNING(tdwait->et_td)) { - gen = record->er_gen; - thread_unlock(td); - do { - cpu_spinwait(); - } while (tdwait == TAILQ_FIRST(&record->er_tdlist) && - gen == record->er_gen && TD_IS_RUNNING(tdwait->et_td) && - spincount++ < MAX_ADAPTIVE_SPIN); - thread_lock(td); + gen = record->er_gen; + thread_unlock(td); + /* + * We can't actually check if the waiting thread is running + * so we simply poll for it to exit before giving up and + * migrating. + */ + do { + cpu_spinwait(); + } while (!TAILQ_EMPTY(&record->er_tdlist) && + gen == record->er_gen && + spincount++ < MAX_ADAPTIVE_SPIN); + thread_lock(td); + /* + * If the generation has changed we can poll again + * otherwise we need to migrate. + */ + if (gen != record->er_gen) return; - } /* * Being on the same CPU as that of the record on which * we need to wait allows us access to the thread