From owner-dev-commits-src-all@freebsd.org Thu Aug 12 17:02:29 2021 Return-Path: Delivered-To: dev-commits-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1DF5966F801; Thu, 12 Aug 2021 17:02:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GltKK0Pgsz3JN2; Thu, 12 Aug 2021 17:02:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DFE4526B05; Thu, 12 Aug 2021 17:02:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 17CH2SUE017741; Thu, 12 Aug 2021 17:02:28 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 17CH2SxC017740; Thu, 12 Aug 2021 17:02:28 GMT (envelope-from git) Date: Thu, 12 Aug 2021 17:02:28 GMT Message-Id: <202108121702.17CH2SxC017740@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: John Baldwin Subject: git: 909459c2eda4 - stable/13 - bhyve: Register new kevents synchronously. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 909459c2eda4a6bc45e1fe5ff3da992e9614392b Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for all branches of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Aug 2021 17:02:29 -0000 The branch stable/13 has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=909459c2eda4a6bc45e1fe5ff3da992e9614392b commit 909459c2eda4a6bc45e1fe5ff3da992e9614392b Author: John Baldwin AuthorDate: 2021-06-12 00:58:54 +0000 Commit: John Baldwin CommitDate: 2021-08-12 15:56:13 +0000 bhyve: Register new kevents synchronously. Change mevent_add*() to synchronously add the new kevent. This permits reporting event registration failures to the caller and avoids failing the registration of other, unrelated events queued up in the same batch. Reviewed by: grehan, markj MFC after: 1 month Differential Revision: https://reviews.freebsd.org/D30502 (cherry picked from commit e8424e2947dd8a0e1d3f08295432941b5c5aafb9) --- usr.sbin/bhyve/mevent.c | 96 ++++++++++++++++++++++++++++++++----------------- 1 file changed, 63 insertions(+), 33 deletions(-) diff --git a/usr.sbin/bhyve/mevent.c b/usr.sbin/bhyve/mevent.c index 2e9059837011..a394b7c5d69f 100644 --- a/usr.sbin/bhyve/mevent.c +++ b/usr.sbin/bhyve/mevent.c @@ -64,8 +64,10 @@ __FBSDID("$FreeBSD$"); #define MEVENT_MAX 64 static pthread_t mevent_tid; +static pthread_once_t mevent_once = PTHREAD_ONCE_INIT; static int mevent_timid = 43; static int mevent_pipefd[2]; +static int mfd; static pthread_mutex_t mevent_lmutex = PTHREAD_MUTEX_INITIALIZER; struct mevent { @@ -124,6 +126,26 @@ mevent_notify(void) } } +static void +mevent_init(void) +{ +#ifndef WITHOUT_CAPSICUM + cap_rights_t rights; +#endif + + mfd = kqueue(); + assert(mfd > 0); + +#ifndef WITHOUT_CAPSICUM + cap_rights_init(&rights, CAP_KQUEUE); + if (caph_rights_limit(mfd, &rights) == -1) + errx(EX_OSERR, "Unable to apply rights for sandbox"); +#endif + + LIST_INIT(&change_head); + LIST_INIT(&global_head); +} + static int mevent_kq_filter(struct mevent *mevp) { @@ -159,8 +181,24 @@ mevent_kq_fflags(struct mevent *mevp) return (0); } +static void +mevent_populate(struct mevent *mevp, struct kevent *kev) +{ + if (mevp->me_type == EVF_TIMER) { + kev->ident = mevp->me_timid; + kev->data = mevp->me_msecs; + } else { + kev->ident = mevp->me_fd; + kev->data = 0; + } + kev->filter = mevent_kq_filter(mevp); + kev->flags = mevent_kq_flags(mevp); + kev->fflags = mevent_kq_fflags(mevp); + kev->udata = mevp; +} + static int -mevent_build(int mfd, struct kevent *kev) +mevent_build(struct kevent *kev) { struct mevent *mevp, *tmpp; int i; @@ -177,17 +215,8 @@ mevent_build(int mfd, struct kevent *kev) */ close(mevp->me_fd); } else { - if (mevp->me_type == EVF_TIMER) { - kev[i].ident = mevp->me_timid; - kev[i].data = mevp->me_msecs; - } else { - kev[i].ident = mevp->me_fd; - kev[i].data = 0; - } - kev[i].filter = mevent_kq_filter(mevp); - kev[i].flags = mevent_kq_flags(mevp); - kev[i].fflags = mevent_kq_fflags(mevp); - kev[i].udata = mevp; + assert((mevp->me_state & EV_ADD) == 0); + mevent_populate(mevp, &kev[i]); i++; } @@ -197,12 +226,6 @@ mevent_build(int mfd, struct kevent *kev) if (mevp->me_state & EV_DELETE) { free(mevp); } else { - /* - * We need to add the event only once, so we can - * reset the EV_ADD bit after it has been propagated - * to the kevent() arguments the first time. - */ - mevp->me_state &= ~EV_ADD; LIST_INSERT_HEAD(&global_head, mevp, me_list); } @@ -234,7 +257,9 @@ mevent_add_state(int tfd, enum ev_type type, void (*func)(int, enum ev_type, void *), void *param, int state) { + struct kevent kev; struct mevent *lp, *mevp; + int ret; if (tfd < 0 || func == NULL) { return (NULL); @@ -242,6 +267,8 @@ mevent_add_state(int tfd, enum ev_type type, mevp = NULL; + pthread_once(&mevent_once, mevent_init); + mevent_qlock(); /* @@ -262,7 +289,7 @@ mevent_add_state(int tfd, enum ev_type type, } /* - * Allocate an entry, populate it, and add it to the change list. + * Allocate an entry and populate it. */ mevp = calloc(1, sizeof(struct mevent)); if (mevp == NULL) { @@ -277,11 +304,22 @@ mevent_add_state(int tfd, enum ev_type type, mevp->me_type = type; mevp->me_func = func; mevp->me_param = param; - - LIST_INSERT_HEAD(&change_head, mevp, me_list); - mevp->me_cq = 1; mevp->me_state = state; - mevent_notify(); + + /* + * Try to add the event. If this fails, report the failure to + * the caller. + */ + mevent_populate(mevp, &kev); + ret = kevent(mfd, &kev, 1, NULL, 0, NULL); + if (ret == -1) { + free(mevp); + mevp = NULL; + goto exit; + } + + mevp->me_state &= ~EV_ADD; + LIST_INSERT_HEAD(&global_head, mevp, me_list); exit: mevent_qunlock(); @@ -415,7 +453,6 @@ mevent_dispatch(void) struct kevent changelist[MEVENT_MAX]; struct kevent eventlist[MEVENT_MAX]; struct mevent *pipev; - int mfd; int numev; int ret; #ifndef WITHOUT_CAPSICUM @@ -425,14 +462,7 @@ mevent_dispatch(void) mevent_tid = pthread_self(); mevent_set_name(); - mfd = kqueue(); - assert(mfd > 0); - -#ifndef WITHOUT_CAPSICUM - cap_rights_init(&rights, CAP_KQUEUE); - if (caph_rights_limit(mfd, &rights) == -1) - errx(EX_OSERR, "Unable to apply rights for sandbox"); -#endif + pthread_once(&mevent_once, mevent_init); /* * Open the pipe that will be used for other threads to force @@ -466,7 +496,7 @@ mevent_dispatch(void) * to eliminate the extra syscall. Currently better for * debug. */ - numev = mevent_build(mfd, changelist); + numev = mevent_build(changelist); if (numev) { ret = kevent(mfd, changelist, numev, NULL, 0, NULL); if (ret == -1) {