From owner-svn-src-vendor@freebsd.org Tue Oct 16 00:48:19 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3C0EB10CE2C8; Tue, 16 Oct 2018 00:48:19 +0000 (UTC) (envelope-from jtl@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E518F8CCCA; Tue, 16 Oct 2018 00:48:18 +0000 (UTC) (envelope-from jtl@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DFFFE14C0F; Tue, 16 Oct 2018 00:48:18 +0000 (UTC) (envelope-from jtl@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w9G0mI4S013712; Tue, 16 Oct 2018 00:48:18 GMT (envelope-from jtl@FreeBSD.org) Received: (from jtl@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w9G0mIwF013711; Tue, 16 Oct 2018 00:48:18 GMT (envelope-from jtl@FreeBSD.org) Message-Id: <201810160048.w9G0mIwF013711@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jtl set sender to jtl@FreeBSD.org using -f From: "Jonathan T. Looney" Date: Tue, 16 Oct 2018 00:48:18 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r339373 - vendor-sys/ck/dist/src X-SVN-Group: vendor-sys X-SVN-Commit-Author: jtl X-SVN-Commit-Paths: vendor-sys/ck/dist/src X-SVN-Commit-Revision: 339373 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Oct 2018 00:48:19 -0000 Author: jtl Date: Tue Oct 16 00:48:18 2018 New Revision: 339373 URL: https://svnweb.freebsd.org/changeset/base/339373 Log: Import CK as of commit 5221ae2f3722a78c7fc41e47069ad94983d3bccb. This fixes two problems, one where epoch calls could occur before all the readers had exited the epoch section, and one where the epoch calls could be unnecessarily delayed. Modified: vendor-sys/ck/dist/src/ck_epoch.c Modified: vendor-sys/ck/dist/src/ck_epoch.c ============================================================================== --- vendor-sys/ck/dist/src/ck_epoch.c Mon Oct 15 21:59:24 2018 (r339372) +++ vendor-sys/ck/dist/src/ck_epoch.c Tue Oct 16 00:48:18 2018 (r339373) @@ -127,6 +127,14 @@ */ #define CK_EPOCH_GRACE 3U +/* + * CK_EPOCH_LENGTH must be a power-of-2 (because (CK_EPOCH_LENGTH - 1) is used + * as a mask, and it must be at least 3 (see comments above). + */ +#if (CK_EPOCH_LENGTH < 3 || (CK_EPOCH_LENGTH & (CK_EPOCH_LENGTH - 1)) != 0) +#error "CK_EPOCH_LENGTH must be a power of 2 and >= 3" +#endif + enum { CK_EPOCH_STATE_USED = 0, CK_EPOCH_STATE_FREE = 1 @@ -348,7 +356,7 @@ ck_epoch_scan(struct ck_epoch *global, return NULL; } -static void +static unsigned int ck_epoch_dispatch(struct ck_epoch_record *record, unsigned int e, ck_stack_t *deferred) { unsigned int epoch = e & (CK_EPOCH_LENGTH - 1); @@ -366,6 +374,7 @@ ck_epoch_dispatch(struct ck_epoch_record *record, unsi ck_stack_push_spnc(deferred, &entry->stack_entry); else entry->function(entry); + i++; } @@ -381,7 +390,7 @@ ck_epoch_dispatch(struct ck_epoch_record *record, unsi ck_pr_sub_uint(&record->n_pending, i); } - return; + return i; } /* @@ -560,16 +569,28 @@ ck_epoch_poll_deferred(struct ck_epoch_record *record, unsigned int epoch; struct ck_epoch_record *cr = NULL; struct ck_epoch *global = record->global; + unsigned int n_dispatch; epoch = ck_pr_load_uint(&global->epoch); /* Serialize epoch snapshots with respect to global epoch. */ ck_pr_fence_memory(); + + /* + * At this point, epoch is the current global epoch value. + * There may or may not be active threads which observed epoch - 1. + * (ck_epoch_scan() will tell us that). However, there should be + * no active threads which observed epoch - 2. + * + * Note that checking epoch - 2 is necessary, as race conditions can + * allow another thread to increment the global epoch before this + * thread runs. + */ + n_dispatch = ck_epoch_dispatch(record, epoch - 2, deferred); + cr = ck_epoch_scan(global, cr, epoch, &active); - if (cr != NULL) { - record->epoch = epoch; - return false; - } + if (cr != NULL) + return (n_dispatch > 0); /* We are at a grace period if all threads are inactive. */ if (active == false) { @@ -580,10 +601,17 @@ ck_epoch_poll_deferred(struct ck_epoch_record *record, return true; } - /* If an active thread exists, rely on epoch observation. */ + /* + * If an active thread exists, rely on epoch observation. + * + * All the active threads entered the epoch section during + * the current epoch. Therefore, we can now run the handlers + * for the immediately preceding epoch and attempt to + * advance the epoch if it hasn't been already. + */ (void)ck_pr_cas_uint(&global->epoch, epoch, epoch + 1); - ck_epoch_dispatch(record, epoch + 1, deferred); + ck_epoch_dispatch(record, epoch - 1, deferred); return true; }