From owner-svn-src-all@freebsd.org Tue Nov 13 23:20:58 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 95B111136FAA; Tue, 13 Nov 2018 23:20:58 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 187B57F774; Tue, 13 Nov 2018 23:20:58 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 79A7317034; Tue, 13 Nov 2018 23:20:56 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id wADNKu8i074414; Tue, 13 Nov 2018 23:20:56 GMT (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id wADNKuGo074412; Tue, 13 Nov 2018 23:20:56 GMT (envelope-from glebius@FreeBSD.org) Message-Id: <201811132320.wADNKuGo074412@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: glebius set sender to glebius@FreeBSD.org using -f From: Gleb Smirnoff Date: Tue, 13 Nov 2018 23:20:56 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r340415 - in head/sys: kern sys X-SVN-Group: head X-SVN-Commit-Author: glebius X-SVN-Commit-Paths: in head/sys: kern sys X-SVN-Commit-Revision: 340415 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 187B57F774 X-Spamd-Result: default: False [-106.88 / 200.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; ALLOW_DOMAIN_WHITELIST(-100.00)[FreeBSD.org]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; HAS_XAW(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all]; DMARC_NA(0.00)[FreeBSD.org]; RCVD_COUNT_THREE(0.00)[4]; MX_GOOD(-0.01)[cached: mx1.FreeBSD.org]; NEURAL_HAM_SHORT(-1.00)[-1.000,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; IP_SCORE(-3.77)[ip: (-9.91), ipnet: 2610:1c1:1::/48(-4.93), asn: 11403(-3.91), country: US(-0.09)] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Nov 2018 23:20:58 -0000 Author: glebius Date: Tue Nov 13 23:20:55 2018 New Revision: 340415 URL: https://svnweb.freebsd.org/changeset/base/340415 Log: The dualism between epoch_tracker and epoch_thread is fragile and unnecessary. So, expose CK types to kernel and use a single normal structure for epoch_tracker. Reviewed by: jtl, gallatin Modified: head/sys/kern/subr_epoch.c head/sys/sys/epoch.h Modified: head/sys/kern/subr_epoch.c ============================================================================== --- head/sys/kern/subr_epoch.c Tue Nov 13 23:08:46 2018 (r340414) +++ head/sys/kern/subr_epoch.c Tue Nov 13 23:20:55 2018 (r340415) @@ -55,25 +55,13 @@ __FBSDID("$FreeBSD$"); static MALLOC_DEFINE(M_EPOCH, "epoch", "epoch based reclamation"); -typedef struct epoch_thread { -#ifdef EPOCH_TRACKER_DEBUG - uint64_t et_magic_pre; -#endif - TAILQ_ENTRY(epoch_thread) et_link; /* Epoch queue. */ - struct thread *et_td; /* pointer to thread in section */ - ck_epoch_section_t et_section; /* epoch section object */ -#ifdef EPOCH_TRACKER_DEBUG - uint64_t et_magic_post; -#endif -} *epoch_thread_t; -TAILQ_HEAD (epoch_tdlist, epoch_thread); - #ifdef __amd64__ #define EPOCH_ALIGN CACHE_LINE_SIZE*2 #else #define EPOCH_ALIGN CACHE_LINE_SIZE #endif +TAILQ_HEAD (epoch_tdlist, epoch_tracker); typedef struct epoch_record { ck_epoch_record_t er_read_record; ck_epoch_record_t er_write_record; @@ -252,27 +240,25 @@ void epoch_enter_preempt(epoch_t epoch, epoch_tracker_t et) { struct epoch_record *er; - struct epoch_thread *etd; struct thread_lite *td; MPASS(cold || epoch != NULL); INIT_CHECK(epoch); - etd = (void *)et; MPASS(epoch->e_flags & EPOCH_PREEMPT); #ifdef EPOCH_TRACKER_DEBUG - etd->et_magic_pre = EPOCH_MAGIC0; - etd->et_magic_post = EPOCH_MAGIC1; + et->et_magic_pre = EPOCH_MAGIC0; + et->et_magic_post = EPOCH_MAGIC1; #endif td = (struct thread_lite *)curthread; - etd->et_td = (void*)td; + et->et_td = (void*)td; td->td_epochnest++; critical_enter(); sched_pin_lite(td); td->td_pre_epoch_prio = td->td_priority; er = epoch_currecord(epoch); - TAILQ_INSERT_TAIL(&er->er_tdlist, etd, et_link); - ck_epoch_begin(&er->er_read_record, (ck_epoch_section_t *)&etd->et_section); + TAILQ_INSERT_TAIL(&er->er_tdlist, et, et_link); + ck_epoch_begin(&er->er_read_record, &et->et_section); critical_exit(); } @@ -296,7 +282,6 @@ void epoch_exit_preempt(epoch_t epoch, epoch_tracker_t et) { struct epoch_record *er; - struct epoch_thread *etd; struct thread_lite *td; INIT_CHECK(epoch); @@ -307,19 +292,19 @@ epoch_exit_preempt(epoch_t epoch, epoch_tracker_t et) td->td_epochnest--; er = epoch_currecord(epoch); MPASS(epoch->e_flags & EPOCH_PREEMPT); - etd = (void *)et; - MPASS(etd != NULL); - MPASS(etd->et_td == (struct thread *)td); + MPASS(et != NULL); + MPASS(et->et_td == (struct thread *)td); #ifdef EPOCH_TRACKER_DEBUG - MPASS(etd->et_magic_pre == EPOCH_MAGIC0); - MPASS(etd->et_magic_post == EPOCH_MAGIC1); - etd->et_magic_pre = 0; - etd->et_magic_post = 0; + MPASS(et->et_magic_pre == EPOCH_MAGIC0); + MPASS(et->et_magic_post == EPOCH_MAGIC1); + et->et_magic_pre = 0; + et->et_magic_post = 0; #endif - etd->et_td = (void*)0xDEADBEEF; - ck_epoch_end(&er->er_read_record, - (ck_epoch_section_t *)&etd->et_section); - TAILQ_REMOVE(&er->er_tdlist, etd, et_link); +#ifdef INVARIANTS + et->et_td = (void*)0xDEADBEEF; +#endif + ck_epoch_end(&er->er_read_record, &et->et_section); + TAILQ_REMOVE(&er->er_tdlist, et, et_link); er->er_gen++; if (__predict_false(td->td_pre_epoch_prio != td->td_priority)) epoch_adjust_prio((struct thread *)td, td->td_pre_epoch_prio); @@ -351,7 +336,7 @@ epoch_block_handler_preempt(struct ck_epoch *global __ { epoch_record_t record; struct thread *td, *owner, *curwaittd; - struct epoch_thread *tdwait; + struct epoch_tracker *tdwait; struct turnstile *ts; struct lock_object *lock; int spincount, gen; @@ -633,7 +618,7 @@ epoch_call_task(void *arg __unused) int in_epoch_verbose(epoch_t epoch, int dump_onfail) { - struct epoch_thread *tdwait; + struct epoch_tracker *tdwait; struct thread *td; epoch_record_t er; Modified: head/sys/sys/epoch.h ============================================================================== --- head/sys/sys/epoch.h Tue Nov 13 23:08:46 2018 (r340414) +++ head/sys/sys/epoch.h Tue Nov 13 23:20:55 2018 (r340415) @@ -30,31 +30,16 @@ #ifndef _SYS_EPOCH_H_ #define _SYS_EPOCH_H_ -/* - * XXXGL: temporarily keep epoch_tracker exposed to userland until - * we remove trackers embedded into network structs. - */ - struct epoch_context { void *data[2]; } __aligned(sizeof(void *)); typedef struct epoch_context *epoch_context_t; -struct epoch_tracker { - void *datap[3]; -#ifdef EPOCH_TRACKER_DEBUG - int datai[5]; -#else - int datai[1]; -#endif -} __aligned(sizeof(void *)); - -typedef struct epoch_tracker *epoch_tracker_t; - #ifdef _KERNEL #include #include +#include struct epoch; typedef struct epoch *epoch_t; @@ -64,6 +49,21 @@ typedef struct epoch *epoch_t; extern epoch_t global_epoch; extern epoch_t global_epoch_preempt; + +struct epoch_tracker { +#ifdef EPOCH_TRACKER_DEBUG +#define EPOCH_MAGIC0 0xFADECAFEF00DD00D +#define EPOCH_MAGIC1 0xBADDBABEDEEDFEED + uint64_t et_magic_pre; +#endif + TAILQ_ENTRY(epoch_tracker) et_link; + struct thread *et_td; + ck_epoch_section_t et_section; +#ifdef EPOCH_TRACKER_DEBUG + uint64_t et_magic_post; +#endif +} __aligned(sizeof(void *)); +typedef struct epoch_tracker *epoch_tracker_t; epoch_t epoch_alloc(int flags); void epoch_free(epoch_t epoch);