From owner-svn-src-all@freebsd.org Wed Jul 3 20:05:26 2019 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A8C1115DFB4D; Wed, 3 Jul 2019 20:05:26 +0000 (UTC) (envelope-from vmaffione@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4A09F8EF81; Wed, 3 Jul 2019 20:05:26 +0000 (UTC) (envelope-from vmaffione@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2286F18A88; Wed, 3 Jul 2019 20:05:26 +0000 (UTC) (envelope-from vmaffione@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x63K5QZC085757; Wed, 3 Jul 2019 20:05:26 GMT (envelope-from vmaffione@FreeBSD.org) Received: (from vmaffione@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x63K5QNu085756; Wed, 3 Jul 2019 20:05:26 GMT (envelope-from vmaffione@FreeBSD.org) Message-Id: <201907032005.x63K5QNu085756@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: vmaffione set sender to vmaffione@FreeBSD.org using -f From: Vincenzo Maffione Date: Wed, 3 Jul 2019 20:05:26 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r349698 - stable/11/usr.sbin/bhyve X-SVN-Group: stable-11 X-SVN-Commit-Author: vmaffione X-SVN-Commit-Paths: stable/11/usr.sbin/bhyve X-SVN-Commit-Revision: 349698 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4A09F8EF81 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_SHORT(-0.98)[-0.978,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jul 2019 20:05:26 -0000 Author: vmaffione Date: Wed Jul 3 20:05:25 2019 New Revision: 349698 URL: https://svnweb.freebsd.org/changeset/base/349698 Log: MFC r348834 bhyve: vtnet: simplify thread synchronization On vtnet device reset it is necessary to wait for threads to stop TX and RX processing. However, the rx_in_progress variable (used for to wait for RX processing to stop) is actually useless, and can be removed. Acquiring and releasing the RX lock is enough to synchronize correctly. Moreover, it is possible to reset the device while holding both TX and RX locks, so that the "resetting" variable becomes unnecessary for the RX thread, and can be protected by the TX lock (instead of being volatile). Reviewed by: jhb, markj Differential Revision: https://reviews.freebsd.org/D20543 Modified: stable/11/usr.sbin/bhyve/pci_virtio_net.c Directory Properties: stable/11/ (props changed) Modified: stable/11/usr.sbin/bhyve/pci_virtio_net.c ============================================================================== --- stable/11/usr.sbin/bhyve/pci_virtio_net.c Wed Jul 3 19:59:56 2019 (r349697) +++ stable/11/usr.sbin/bhyve/pci_virtio_net.c Wed Jul 3 20:05:25 2019 (r349698) @@ -147,14 +147,13 @@ struct pci_vtnet_softc { struct nm_desc *vsc_nmd; int vsc_rx_ready; - volatile int resetting; /* set and checked outside lock */ + int resetting; /* protected by tx_mtx */ uint64_t vsc_features; /* negotiated features */ struct virtio_net_config vsc_config; pthread_mutex_t rx_mtx; - int rx_in_progress; int rx_vhdrlen; int rx_merge; /* merged rx bufs in use */ @@ -186,62 +185,39 @@ static struct virtio_consts vtnet_vi_consts = { VTNET_S_HOSTCAPS, /* our capabilities */ }; -/* - * If the transmit thread is active then stall until it is done. - */ static void -pci_vtnet_txwait(struct pci_vtnet_softc *sc) +pci_vtnet_reset(void *vsc) { + struct pci_vtnet_softc *sc = vsc; + DPRINTF(("vtnet: device reset requested !\n")); + + /* Acquire the RX lock to block RX processing. */ + pthread_mutex_lock(&sc->rx_mtx); + + /* Set sc->resetting and give a chance to the TX thread to stop. */ pthread_mutex_lock(&sc->tx_mtx); + sc->resetting = 1; while (sc->tx_in_progress) { pthread_mutex_unlock(&sc->tx_mtx); usleep(10000); pthread_mutex_lock(&sc->tx_mtx); } - pthread_mutex_unlock(&sc->tx_mtx); -} -/* - * If the receive thread is active then stall until it is done. - */ -static void -pci_vtnet_rxwait(struct pci_vtnet_softc *sc) -{ - - pthread_mutex_lock(&sc->rx_mtx); - while (sc->rx_in_progress) { - pthread_mutex_unlock(&sc->rx_mtx); - usleep(10000); - pthread_mutex_lock(&sc->rx_mtx); - } - pthread_mutex_unlock(&sc->rx_mtx); -} - -static void -pci_vtnet_reset(void *vsc) -{ - struct pci_vtnet_softc *sc = vsc; - - DPRINTF(("vtnet: device reset requested !\n")); - - sc->resetting = 1; - - /* - * Wait for the transmit and receive threads to finish their - * processing. - */ - pci_vtnet_txwait(sc); - pci_vtnet_rxwait(sc); - sc->vsc_rx_ready = 0; sc->rx_merge = 1; sc->rx_vhdrlen = sizeof(struct virtio_net_rxhdr); - /* now reset rings, MSI-X vectors, and negotiated capabilities */ + /* + * Now reset rings, MSI-X vectors, and negotiated capabilities. + * Do that with the TX lock held, since we need to reset + * sc->resetting. + */ vi_reset_dev(&sc->vsc_vs); sc->resetting = 0; + pthread_mutex_unlock(&sc->tx_mtx); + pthread_mutex_unlock(&sc->rx_mtx); } /* @@ -315,9 +291,9 @@ pci_vtnet_tap_rx(struct pci_vtnet_softc *sc) /* * But, will be called when the rx ring hasn't yet - * been set up or the guest is resetting the device. + * been set up. */ - if (!sc->vsc_rx_ready || sc->resetting) { + if (!sc->vsc_rx_ready) { /* * Drop the packet and try later. */ @@ -512,9 +488,9 @@ pci_vtnet_netmap_rx(struct pci_vtnet_softc *sc) /* * But, will be called when the rx ring hasn't yet - * been set up or the guest is resetting the device. + * been set up. */ - if (!sc->vsc_rx_ready || sc->resetting) { + if (!sc->vsc_rx_ready) { /* * Drop the packet and try later. */ @@ -591,9 +567,7 @@ pci_vtnet_rx_callback(int fd, enum ev_type type, void struct pci_vtnet_softc *sc = param; pthread_mutex_lock(&sc->rx_mtx); - sc->rx_in_progress = 1; sc->pci_vtnet_rx(sc); - sc->rx_in_progress = 0; pthread_mutex_unlock(&sc->rx_mtx); } @@ -918,7 +892,6 @@ pci_vtnet_init(struct vmctx *ctx, struct pci_devinst * sc->rx_merge = 1; sc->rx_vhdrlen = sizeof(struct virtio_net_rxhdr); - sc->rx_in_progress = 0; pthread_mutex_init(&sc->rx_mtx, NULL); /*