From nobody Mon Sep 29 15:22:38 2025 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4cb4gf4qdBz68P7N; Mon, 29 Sep 2025 15:22:38 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R12" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4cb4gf2ycQz46p8; Mon, 29 Sep 2025 15:22:38 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1759159358; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=yf+px/mCQ2g5jxxhsMXKWZ4SLDd1yBGBksVQaPnxs7w=; b=jBs0NF4I4+iLwdpWa3kFCCA9jzOLJuMGrcmGJY1rYdXGMjojufBUlX8+nex37TkSsgp8rO MD2hkTyi65/JvVOkkzUqBYsNLujuHHyXSOJtVqhJT0clHyFo1576uXH5PVxd2GpjhcgMBd pEz6n6cqgMKJ87wK6UUm4XyIfOtTCymtBqK+tUx2tO3Dt6QbVY4stPUu2sV5YXMps+1FbW yVvzeY/XC+WjvPcfwjkWVGvOdaoCpuki4u/9ExExJZ30bjjQ+yVAeAi54g3zcedKGkKR8Z ytqpK/jrk+ZuM+rldbC1VWy6dM23bIC1ibkXHfw/PnbfsllSZDCMwaD9LWltZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1759159358; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=yf+px/mCQ2g5jxxhsMXKWZ4SLDd1yBGBksVQaPnxs7w=; b=UW/NVI7BLUBmSkScmKUQECEJh0qnJrmkNaokFUYFG1/5K3xt6vqlWLmxPk8/vq2VDpLFUP rahvy3omejNdkoCqfke22h8vmqI9qjiA89irgINu+W6y1AEjXy5Bnr9DGD9Vs7WpdrLfjf QNKvrGFAIqgdvvS5BQiPzqueFIfy1GHMEloXrM4h3bcAT+BuFvQsykKYnCMqryAoaxULUE aso2u13SORCNTWJhLoctWOutQIcacxs7HX1yvDA+ff0QveALO9Vdw8GSFbBX0ERR/PUza8 hfX/LnM+OlVTd5ARiyOPmf8onnZU+KeRcb4p4TMsozIKNVSkucXblS6yj6yJ0A== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1759159358; a=rsa-sha256; cv=none; b=n2Pa9DyjVyYKx0Rf1oLOO6JFaGZXXkWtrPKUuJpvWYg9ics80Y3kqRHltAphkPfWCbo73E c2XiJ1BWSZ9u/m/8djvgNg6P07818RTKOxncWeZ/TnKTO03dWR8NF1FDFLgZ1vO348Q6aK 5IAKKdI5dUMjLobDQZiN7EifZN99LY3MhKyFFbyZcWVnfgYj5LfOWyNtOI02tSjOmCSfaG t6tCA6aw4j3asrjhoIMEL3X3Ze4L031LDyshxUZ7bY2/XT3UUoNtP+X+U6bSMiUCo03Rkn RKQwrIntmbjLDHlHpQFBEPfG06t+xk33ygo7qQrpbw1DDpAH17evVh+eGZkMDw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4cb4gf2WhHz19xh; Mon, 29 Sep 2025 15:22:38 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 58TFMcBI012122; Mon, 29 Sep 2025 15:22:38 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 58TFMcQ2012119; Mon, 29 Sep 2025 15:22:38 GMT (envelope-from git) Date: Mon, 29 Sep 2025 15:22:38 GMT Message-Id: <202509291522.58TFMcQ2012119@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Navdeep Parhar Subject: git: 96054b97b418 - main - cxgbe mp_ring: Add mp_ring_enqueue_only List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: np X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 96054b97b418d1ff2b781e9e23cc86a79d91909a Auto-Submitted: auto-generated The branch main has been updated by np: URL: https://cgit.FreeBSD.org/src/commit/?id=96054b97b418d1ff2b781e9e23cc86a79d91909a commit 96054b97b418d1ff2b781e9e23cc86a79d91909a Author: John Baldwin AuthorDate: 2025-09-29 15:10:42 +0000 Commit: Navdeep Parhar CommitDate: 2025-09-29 15:19:13 +0000 cxgbe mp_ring: Add mp_ring_enqueue_only Unlike mp_ring_enqueue, this function is meant to be called from within a drain routine to enqueue more packets to the queue currently being drained. To avoid recursion, it only enqueues additional items without becoming a consumer. MFC after: 3 days Sponsored by: Chelsio Communications --- sys/dev/cxgbe/t4_mp_ring.c | 81 +++++++++++++++++++++++++++++++++++++++++++++- sys/dev/cxgbe/t4_mp_ring.h | 1 + 2 files changed, 81 insertions(+), 1 deletion(-) diff --git a/sys/dev/cxgbe/t4_mp_ring.c b/sys/dev/cxgbe/t4_mp_ring.c index 531fd356728e..916c363a0c2a 100644 --- a/sys/dev/cxgbe/t4_mp_ring.c +++ b/sys/dev/cxgbe/t4_mp_ring.c @@ -305,7 +305,6 @@ failed: } void - mp_ring_free(struct mp_ring *r) { int i; @@ -472,6 +471,86 @@ mp_ring_enqueue(struct mp_ring *r, void **items, int n, int budget) return (0); } +/* + * Enqueue n items but never drain the ring. Can be called + * to enqueue new items while draining the ring. + * + * Returns an errno. + */ +int +mp_ring_enqueue_only(struct mp_ring *r, void **items, int n) +{ + union ring_state os, ns; + uint16_t pidx_start, pidx_stop; + int i; + + MPASS(items != NULL); + MPASS(n > 0); + + /* + * Reserve room for the new items. Our reservation, if successful, is + * from 'pidx_start' to 'pidx_stop'. + */ + os.state = atomic_load_64(&r->state); + + /* Should only be used from the drain callback. */ + MPASS(os.flags == BUSY || os.flags == TOO_BUSY || + os.flags == TAKING_OVER); + + for (;;) { + if (__predict_false(space_available(r, os) < n)) { + /* Not enough room in the ring. */ + counter_u64_add(r->dropped, n); + return (ENOBUFS); + } + + /* There is room in the ring. */ + + ns.state = os.state; + ns.pidx_head = increment_idx(r, os.pidx_head, n); + critical_enter(); + if (atomic_fcmpset_64(&r->state, &os.state, ns.state)) + break; + critical_exit(); + cpu_spinwait(); + }; + + pidx_start = os.pidx_head; + pidx_stop = ns.pidx_head; + + /* + * Wait for other producers who got in ahead of us to enqueue their + * items, one producer at a time. It is our turn when the ring's + * pidx_tail reaches the beginning of our reservation (pidx_start). + */ + while (ns.pidx_tail != pidx_start) { + cpu_spinwait(); + ns.state = atomic_load_64(&r->state); + } + + /* Now it is our turn to fill up the area we reserved earlier. */ + i = pidx_start; + do { + r->items[i] = *items++; + if (__predict_false(++i == r->size)) + i = 0; + } while (i != pidx_stop); + + /* + * Update the ring's pidx_tail. The release style atomic guarantees + * that the items are visible to any thread that sees the updated pidx. + */ + os.state = atomic_load_64(&r->state); + do { + ns.state = os.state; + ns.pidx_tail = pidx_stop; + } while (atomic_fcmpset_rel_64(&r->state, &os.state, ns.state) == 0); + critical_exit(); + + counter_u64_add(r->not_consumer, 1); + return (0); +} + void mp_ring_check_drainage(struct mp_ring *r, int budget) { diff --git a/sys/dev/cxgbe/t4_mp_ring.h b/sys/dev/cxgbe/t4_mp_ring.h index 949174b9056d..07b15906cd43 100644 --- a/sys/dev/cxgbe/t4_mp_ring.h +++ b/sys/dev/cxgbe/t4_mp_ring.h @@ -62,6 +62,7 @@ int mp_ring_alloc(struct mp_ring **, int, void *, ring_drain_t, ring_can_drain_t, struct malloc_type *, struct mtx *, int); void mp_ring_free(struct mp_ring *); int mp_ring_enqueue(struct mp_ring *, void **, int, int); +int mp_ring_enqueue_only(struct mp_ring *, void **, int); void mp_ring_check_drainage(struct mp_ring *, int); void mp_ring_reset_stats(struct mp_ring *); bool mp_ring_is_idle(struct mp_ring *);