From nobody Fri Aug 7 13:10:44 2026 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4hGkzY5s8tz6n437 for ; Fri, 07 Aug 2026 13:10:49 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR1" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4hGkzY4F5Lz3Bqg for ; Fri, 07 Aug 2026 13:10:49 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1786108249; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=VyICU8bpL/mirhUbOzotqEqcOuYwzmyzWc9AqzYGfLw=; b=tF9oU8mks2WF/4YqDfQmvSF9eHbze68qepXrtht0yj8xc8C3YCrGr4MrEDnU08qXlI0LlH FUoJaPPkC43l0a/6P+kodL00ToyUwmXpe/ViZoTMp2kCTzpbe4KOe9O5i9x+VkqE9TcYYd pae1N1eLhHTdfs5V0LJGA7V6vwnC8Opd+JwkUb+sGfUQm+GnbbuPpCO4zQ4+lhMob5Jqbc t695lquSW4SALT7i+LeJmFfl9ttQeQY3Qx2AbVtOBFb49jUU1DQ6O5AvSs1q6gId3LA11o H5phdBS3Flf9abBPGSKM4Iot1JxG5a5AfMM01tJFHYNco6GYuCeLF5QYspWdpw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1786108249; a=rsa-sha256; cv=none; b=PpJnH4iJhEnXgV2VZ70C0SjXaRpScllZvBTaTrkNKV6VM10/JA2O7kvOR0SlGepO2jmb5i a77bN+XFE/Julyw+ezPk3NRMcex2FtwZdhLrbBBClI4FjVVlAdmim2JCGJlTH3jjAk8dPo jpolFSSDwjQBHt+qfPJsvRItfcg/maZ3+jNsQk2arb1vqUO8fcZCBWyXGvBiPxAXmQ0SPu kXuEJkmEJjCp8mC9fMjlKhxkca1NFk/sH+DPAbeeTMwdEIkN1Ll8109DtQ5Zro+gwWGKyB T30pmDYdwmwuLWtqA3JQ0zU/RHW0F1op3Rnrbe+GyedVrvR7QUel+Ql0GWeqyg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1786108249; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=VyICU8bpL/mirhUbOzotqEqcOuYwzmyzWc9AqzYGfLw=; b=lPyt/64azZSl23um0SHK7uiC7zCTpjHOWsdU/f4dYUI+CwKdiC9g8LWlj9KyOl9URfczkR Ve6w0cRZuVDi5wnyuiTaBNY9o3wglv/YI4qgigqVb/IHnPgVfu3S20YwlBJJpZ7XPyRXC7 2RRj4vnsMEW0cGnywEfuUvZ9qlcPnrgyXzIM2Q4l95+QY+0kAEcWyK294mgt08ffcwIWZU AIQKCZljB33gMSEVQ0EmNq7HQ0X0ByA5DrU+dekBwkumNbVlVv7SwDkPySpfTmhEEIqzYp L+2h6RUbaZ5ZDV03JbvGHOXGd2uL4GyhitjAV1vKX+ygzWIUdF4OAFo7W/k1zQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4hGkzY3BLcz17TX for ; Fri, 07 Aug 2026 13:10:49 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 3e55a by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Fri, 07 Aug 2026 13:10:44 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Cc: Sujithra Periasamy From: Mark Johnston Subject: git: 36d57489ca07 - main - gve: Implement AQ batching for queue creation and destruction List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 36d57489ca07642dec31390e90c25b2a0ca9313e Auto-Submitted: auto-generated Date: Fri, 07 Aug 2026 13:10:44 +0000 Message-Id: <6a75d954.3e55a.58c1ac30@gitrepo.freebsd.org> The branch main has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=36d57489ca07642dec31390e90c25b2a0ca9313e commit 36d57489ca07642dec31390e90c25b2a0ca9313e Author: Sujithra Periasamy AuthorDate: 2026-08-07 13:07:20 +0000 Commit: Mark Johnston CommitDate: 2026-08-07 13:10:24 +0000 gve: Implement AQ batching for queue creation and destruction Currently, the FreeBSD driver configures and destroys queues sequentially by issuing individual Admin Queue (AQ) commands. During queue teardown (e.g., interface reset), disabling queues one by one leaves the device in a partially configured state. Because the device does not yet know that the driver is in the process of fully unconfiguring all queues, this intermediate state can trigger transient error logs (such as when queue 0 is disabled while other queues are still active). Modify the driver to use Admin Queue batching for both the creation and destruction of TX and RX queues. Commands are now queued and kicked together, ensuring the queue configuration changes are applied atomically and preventing transient errors from being logged. Signed-off-by: Sujithra Periasamy Reviewed by: markj MFC after: 1 week Sponsored by: Google Differential Revision: https://reviews.freebsd.org/D58696 --- sys/dev/gve/gve_adminq.c | 47 +++++++++++++++++++++++++++++++++++++---------- 1 file changed, 37 insertions(+), 10 deletions(-) diff --git a/sys/dev/gve/gve_adminq.c b/sys/dev/gve/gve_adminq.c index 9b59570a2af4..43952308098b 100644 --- a/sys/dev/gve/gve_adminq.c +++ b/sys/dev/gve/gve_adminq.c @@ -214,6 +214,9 @@ gve_process_device_options(struct gve_priv *priv, return (0); } +static int gve_adminq_issue_cmd(struct gve_priv *priv, + struct gve_adminq_command *cmd); +static int gve_adminq_kick_and_wait(struct gve_priv *priv); static int gve_adminq_execute_cmd(struct gve_priv *priv, struct gve_adminq_command *cmd); @@ -225,7 +228,7 @@ gve_adminq_destroy_tx_queue(struct gve_priv *priv, uint32_t id) cmd.opcode = htobe32(GVE_ADMINQ_DESTROY_TX_QUEUE); cmd.destroy_tx_queue.queue_id = htobe32(id); - return (gve_adminq_execute_cmd(priv, &cmd)); + return (gve_adminq_issue_cmd(priv, &cmd)); } static int @@ -236,7 +239,7 @@ gve_adminq_destroy_rx_queue(struct gve_priv *priv, uint32_t id) cmd.opcode = htobe32(GVE_ADMINQ_DESTROY_RX_QUEUE); cmd.destroy_rx_queue.queue_id = htobe32(id); - return (gve_adminq_execute_cmd(priv, &cmd)); + return (gve_adminq_issue_cmd(priv, &cmd)); } int @@ -248,13 +251,18 @@ gve_adminq_destroy_rx_queues(struct gve_priv *priv, uint32_t num_queues) for (i = 0; i < num_queues; i++) { err = gve_adminq_destroy_rx_queue(priv, i); if (err != 0) { - device_printf(priv->dev, "Failed to destroy rxq %d, err: %d\n", + device_printf(priv->dev, "Failed to issue destroy rxq %d, err: %d\n", i, err); + return (err); } } - if (err != 0) + err = gve_adminq_kick_and_wait(priv); + if (err != 0) { + device_printf(priv->dev, "Failed to batch destroy rx queues, err: %d\n", + err); return (err); + } device_printf(priv->dev, "Destroyed %d rx queues\n", num_queues); return (0); @@ -269,13 +277,18 @@ gve_adminq_destroy_tx_queues(struct gve_priv *priv, uint32_t num_queues) for (i = 0; i < num_queues; i++) { err = gve_adminq_destroy_tx_queue(priv, i); if (err != 0) { - device_printf(priv->dev, "Failed to destroy txq %d, err: %d\n", + device_printf(priv->dev, "Failed to issue destroy txq %d, err: %d\n", i, err); + return (err); } } - if (err != 0) + err = gve_adminq_kick_and_wait(priv); + if (err != 0) { + device_printf(priv->dev, "Failed to batch destroy tx queues, err: %d\n", + err); return (err); + } device_printf(priv->dev, "Destroyed %d tx queues\n", num_queues); return (0); @@ -325,7 +338,7 @@ gve_adminq_create_rx_queue(struct gve_priv *priv, uint32_t queue_index) htobe16(priv->rx_buf_size_dqo); } - return (gve_adminq_execute_cmd(priv, &cmd)); + return (gve_adminq_issue_cmd(priv, &cmd)); } int @@ -337,12 +350,19 @@ gve_adminq_create_rx_queues(struct gve_priv *priv, uint32_t num_queues) for (i = 0; i < num_queues; i++) { err = gve_adminq_create_rx_queue(priv, i); if (err != 0) { - device_printf(priv->dev, "Failed to create rxq %d, err: %d\n", + device_printf(priv->dev, "Failed to issue create rxq %d, err: %d\n", i, err); goto abort; } } + err = gve_adminq_kick_and_wait(priv); + if (err != 0) { + device_printf(priv->dev, "Failed to batch create rx queues, err: %d\n", + err); + goto abort; + } + if (bootverbose) device_printf(priv->dev, "Created %d rx queues\n", num_queues); return (0); @@ -381,7 +401,7 @@ gve_adminq_create_tx_queue(struct gve_priv *priv, uint32_t queue_index) cmd.create_tx_queue.tx_comp_ring_size = htobe16(priv->tx_desc_cnt); } - return (gve_adminq_execute_cmd(priv, &cmd)); + return (gve_adminq_issue_cmd(priv, &cmd)); } int @@ -393,12 +413,19 @@ gve_adminq_create_tx_queues(struct gve_priv *priv, uint32_t num_queues) for (i = 0; i < num_queues; i++) { err = gve_adminq_create_tx_queue(priv, i); if (err != 0) { - device_printf(priv->dev, "Failed to create txq %d, err: %d\n", + device_printf(priv->dev, "Failed to issue create txq %d, err: %d\n", i, err); goto abort; } } + err = gve_adminq_kick_and_wait(priv); + if (err != 0) { + device_printf(priv->dev, "Failed to batch create tx queues, err: %d\n", + err); + goto abort; + } + if (bootverbose) device_printf(priv->dev, "Created %d tx queues\n", num_queues); return (0);