From nobody Fri Dec 10 12:33:03 2021 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 08C4D18D2E39; Fri, 10 Dec 2021 12:33:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4J9Vg42HF5z3tlL; Fri, 10 Dec 2021 12:33:04 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 146161A906; Fri, 10 Dec 2021 12:33:04 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1BACX3He069670; Fri, 10 Dec 2021 12:33:03 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1BACX3AQ069669; Fri, 10 Dec 2021 12:33:03 GMT (envelope-from git) Date: Fri, 10 Dec 2021 12:33:03 GMT Message-Id: <202112101233.1BACX3AQ069669@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Andriy Gapon Subject: git: 5f24d2a82c1f - stable/13 - iflib_stop: drain rx tasks to prevent any data races List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: avg X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 5f24d2a82c1fa6555e550dcda99d0c1aae1e44b3 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1639139584; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=PUL2vk97VApGZNz1VyoVZ6D6scA1hzYaUsOS839dZgw=; b=tQqYmTShAmDZ2+7yKcMX3YjkqLvX3RfrW8z5sRZUHM9Ju7N4ffdQR7u6fWokLN1rQhdeFi C5JhiQJ412gg2/S1/+SNCrfnJRhX2DU39Bp6b20XfmNXn8UYyemfJKwDS2wRJYoypH0aX7 EYSU2qPYRc0j+GuaOXkH2mLUN1yGylkiCwfhzyCuLflzack9S9ITGEyGF8sUU1qqsQApz6 fuTKBbz5LK1vc6rcf9RIbrI/zlh3egQu3UUijZ5ETLUPbETMlY411zw6o5O8KrmQQ1QaEj K1TgZbYBBQam8kyBpwuE/391EBtUqePIx9Z/NUUK8Gq8gS/pcZQ7iIl22YqzoQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1639139584; a=rsa-sha256; cv=none; b=oxdCGPOQGtmM2FijhRUYd6zCWrZzWme/0NYbC64tWfLDQB3L7TB4AtW0sE75uC5CID7wZo VL0aUi15l+Tp0CSapIHCWOF1lHTzo+bWJdcRoEXRsYLZb5ujtGq8PMaegkaNYro/4HK4JA 1cMqQ1kLTCN/E7tdzXNSbMtX3fH2rsD1AGdcQfZNY12ICB+UwzR25lFI+CsDDr7CoDfG7M N2zlvjSUe2ZoaA5n+jUtBA2C6RpoHwftb2Ja5xaS3SZnK3PE6ZHmHwMAatpRGDVkkwPqYR d7XplDX73cYdInggZOnPNDD5fvsWxcCMIoibnC5/uGWRSIRyuoCEmTsAqQnYkw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by avg: URL: https://cgit.FreeBSD.org/src/commit/?id=5f24d2a82c1fa6555e550dcda99d0c1aae1e44b3 commit 5f24d2a82c1fa6555e550dcda99d0c1aae1e44b3 Author: Andriy Gapon AuthorDate: 2021-11-19 07:56:30 +0000 Commit: Andriy Gapon CommitDate: 2021-12-10 12:32:37 +0000 iflib_stop: drain rx tasks to prevent any data races iflib_stop modifies iflib data structures that are used by _task_fn_rx, most prominently the free lists. So, iflib_stop has to ensure that the rx task threads are not active. This should help to fix a crash seen when iflib_if_ioctl (e.g., SIOCSIFCAP) is called while there is already traffic flowing. The crash has been seen on VMWare guests with vmxnet3 driver. My guess is that on physical hardware the couple of 1ms delays that iflib_stop has after disabling interrupts are enough for the queued work to be completed before any iflib state is touched. But on busy hypervisors the guests might not get enough CPU time to complete the work, thus there can be a race between the taskqueue threads and the work done to handle an ioctl, specifically in iflib_stop and iflib_init_locked. PR: 259458 (cherry picked from commit 1bfdb812c786ac2607a82633f9c84a5d16f54079) --- sys/net/iflib.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sys/net/iflib.c b/sys/net/iflib.c index 55ba1895a059..85da3e77f037 100644 --- a/sys/net/iflib.c +++ b/sys/net/iflib.c @@ -2612,7 +2612,8 @@ iflib_stop(if_ctx_t ctx) bzero((void *)di->idi_vaddr, di->idi_size); } for (i = 0; i < scctx->isc_nrxqsets; i++, rxq++) { - /* make sure all transmitters have completed before proceeding XXX */ + gtaskqueue_drain(rxq->ifr_task.gt_taskqueue, + &rxq->ifr_task.gt_task); rxq->ifr_cq_cidx = 0; for (j = 0, di = rxq->ifr_ifdi; j < sctx->isc_nrxqs; j++, di++)