From owner-svn-src-head@freebsd.org Tue Nov 12 21:07:51 2019 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CACB91BAFB3; Tue, 12 Nov 2019 21:07:51 +0000 (UTC) (envelope-from vmaffione@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47CL1M55QFz46Kg; Tue, 12 Nov 2019 21:07:51 +0000 (UTC) (envelope-from vmaffione@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 938655AC3; Tue, 12 Nov 2019 21:07:51 +0000 (UTC) (envelope-from vmaffione@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id xACL7pRq093829; Tue, 12 Nov 2019 21:07:51 GMT (envelope-from vmaffione@FreeBSD.org) Received: (from vmaffione@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id xACL7pjd093828; Tue, 12 Nov 2019 21:07:51 GMT (envelope-from vmaffione@FreeBSD.org) Message-Id: <201911122107.xACL7pjd093828@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: vmaffione set sender to vmaffione@FreeBSD.org using -f From: Vincenzo Maffione Date: Tue, 12 Nov 2019 21:07:51 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r354659 - head/usr.sbin/bhyve X-SVN-Group: head X-SVN-Commit-Author: vmaffione X-SVN-Commit-Paths: head/usr.sbin/bhyve X-SVN-Commit-Revision: 354659 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Nov 2019 21:07:51 -0000 Author: vmaffione Date: Tue Nov 12 21:07:51 2019 New Revision: 354659 URL: https://svnweb.freebsd.org/changeset/base/354659 Log: bhyve: rework mevent processing to fix a race condition At the end of both mevent_add() and mevent_update(), mevent_notify() is called to wakeup the I/O thread, that will call kevent(changelist) to update the kernel. A race condition is possible where the client calls mevent_add() and mevent_update(EV_ENABLE) before the I/O thread has the chance to wake up and call mevent_build()+kevent(changelist) in response to mevent_add(). The mevent_add() is therefore ignored by the I/O thread, and kevent(fd, EV_ENABLE) is called before kevent(fd, EV_ADD), resuliting in a failure of the kevent(fd, EV_ENABLE) call. PR: 241808 Reviewed by: jhb, markj MFC with: r354288 Differential Revision: https://reviews.freebsd.org/D22286 Modified: head/usr.sbin/bhyve/mevent.c Modified: head/usr.sbin/bhyve/mevent.c ============================================================================== --- head/usr.sbin/bhyve/mevent.c Tue Nov 12 19:35:46 2019 (r354658) +++ head/usr.sbin/bhyve/mevent.c Tue Nov 12 21:07:51 2019 (r354659) @@ -42,6 +42,7 @@ __FBSDID("$FreeBSD$"); #endif #include #include +#include #include #include #include @@ -62,12 +63,6 @@ __FBSDID("$FreeBSD$"); #define MEVENT_MAX 64 -#define MEV_ADD 1 -#define MEV_ENABLE 2 -#define MEV_DISABLE 3 -#define MEV_DEL_PENDING 4 -#define MEV_ADD_DISABLED 5 - extern char *vmname; static pthread_t mevent_tid; @@ -83,7 +78,7 @@ struct mevent { enum ev_type me_type; void *me_param; int me_cq; - int me_state; + int me_state; /* Desired kevent flags. */ int me_closefd; LIST_ENTRY(mevent) me_list; }; @@ -156,30 +151,7 @@ mevent_kq_filter(struct mevent *mevp) static int mevent_kq_flags(struct mevent *mevp) { - int ret; - - switch (mevp->me_state) { - case MEV_ADD: - ret = EV_ADD; /* implicitly enabled */ - break; - case MEV_ADD_DISABLED: - ret = EV_ADD | EV_DISABLE; - break; - case MEV_ENABLE: - ret = EV_ENABLE; - break; - case MEV_DISABLE: - ret = EV_DISABLE; - break; - case MEV_DEL_PENDING: - ret = EV_DELETE; - break; - default: - assert(0); - break; - } - - return (ret); + return (mevp->me_state); } static int @@ -224,9 +196,15 @@ mevent_build(int mfd, struct kevent *kev) mevp->me_cq = 0; LIST_REMOVE(mevp, me_list); - if (mevp->me_state == MEV_DEL_PENDING) { + if (mevp->me_state & EV_DELETE) { free(mevp); } else { + /* + * We need to add the event only once, so we can + * reset the EV_ADD bit after it has been propagated + * to the kevent() arguments the first time. + */ + mevp->me_state &= ~EV_ADD; LIST_INSERT_HEAD(&global_head, mevp, me_list); } @@ -318,7 +296,7 @@ mevent_add(int tfd, enum ev_type type, void (*func)(int, enum ev_type, void *), void *param) { - return mevent_add_state(tfd, type, func, param, MEV_ADD); + return (mevent_add_state(tfd, type, func, param, EV_ADD)); } struct mevent * @@ -326,36 +304,46 @@ mevent_add_disabled(int tfd, enum ev_type type, void (*func)(int, enum ev_type, void *), void *param) { - return mevent_add_state(tfd, type, func, param, MEV_ADD_DISABLED); + return (mevent_add_state(tfd, type, func, param, EV_ADD | EV_DISABLE)); } static int -mevent_update(struct mevent *evp, int newstate) +mevent_update(struct mevent *evp, bool enable) { + int newstate; + + mevent_qlock(); + /* * It's not possible to enable/disable a deleted event */ - if (evp->me_state == MEV_DEL_PENDING) - return (EINVAL); + assert((evp->me_state & EV_DELETE) == 0); + newstate = evp->me_state; + if (enable) { + newstate |= EV_ENABLE; + newstate &= ~EV_DISABLE; + } else { + newstate |= EV_DISABLE; + newstate &= ~EV_ENABLE; + } + /* * No update needed if state isn't changing */ - if (evp->me_state == newstate) - return (0); - - mevent_qlock(); + if (evp->me_state != newstate) { + evp->me_state = newstate; - evp->me_state = newstate; - - /* - * Place the entry onto the changed list if not already there. - */ - if (evp->me_cq == 0) { - evp->me_cq = 1; - LIST_REMOVE(evp, me_list); - LIST_INSERT_HEAD(&change_head, evp, me_list); - mevent_notify(); + /* + * Place the entry onto the changed list if not + * already there. + */ + if (evp->me_cq == 0) { + evp->me_cq = 1; + LIST_REMOVE(evp, me_list); + LIST_INSERT_HEAD(&change_head, evp, me_list); + mevent_notify(); + } } mevent_qunlock(); @@ -367,14 +355,14 @@ int mevent_enable(struct mevent *evp) { - return (mevent_update(evp, MEV_ENABLE)); + return (mevent_update(evp, true)); } int mevent_disable(struct mevent *evp) { - return (mevent_update(evp, MEV_DISABLE)); + return (mevent_update(evp, false)); } static int @@ -392,7 +380,7 @@ mevent_delete_event(struct mevent *evp, int closefd) LIST_INSERT_HEAD(&change_head, evp, me_list); mevent_notify(); } - evp->me_state = MEV_DEL_PENDING; + evp->me_state = EV_DELETE; if (closefd) evp->me_closefd = 1;