From nobody Fri Apr 24 22:30:36 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4g2SMw5ZnWz6bPpy for ; Fri, 24 Apr 2026 22:30:36 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R13" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4g2SMw3TNYz3kyH for ; Fri, 24 Apr 2026 22:30:36 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1777069836; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=8q+Edl5B0BoHOtnJ5aXvLyrL7WOndyD1BtPRaSy7gwY=; b=GxGihfW5CCtbeSABDK6ya87FY92Z29SgZ7RZ9xn3+DL+JfY7560Yt6o3PtaAt2CSOYIs/C AZ+5nT8Q2mYyHxZtL9XyIj52646g6kCSSATM6w+jUPKw/NnM9Fx+jmkj9+nqNNNMrXoBMF 8RfIXEBnGUOVLHAWvDXKqBSoO5y97X5f2I0QZ0fdMEZKKay33hPNMNZ4Md6+Kg2WEmK5hy bYtcvFQA7xisVhrU54HB5wB4n2fdRmj72SIspWZvOmBGOVVPY3nQ7D1oZH7SCT0fxhDI/X 7BDPX45L3JiZ/4oW7h65Ryjd9dWYwyJbkCN6NveQ4HKCNCoBHoaRfS9fWgTbEg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1777069836; a=rsa-sha256; cv=none; b=B/hyXTjRKh1QUD/b8iH4W/y0De1JYrJpqkMUDhmATgsaYmKMVI2BkVi+qJb+0AkI/WG8mk vajBY9BRJiIPhDLMTPShWTTBS/TOhzhmlaB9+Vd978M+YQsH/W7uAI91ej42ixBZWNCGL7 d5MmJag2mwilo8w6bJiHq5Sf3sjuntfoTpPTU/jYQQ4YOvzqg5R6C+xkz+DVwRhMo8PUiO uGa5qCZeugoN5IzuBCWiuE+ipagXp7K5bd86YsOUFHRWbyH7Tq8zQ2Ya9wzsTgq2g1TMHF Q2We5Zu7t2GdnTwI37tGm60IpKGnoA6lUcehxnsYJIpb0/ulhvjKfGPky6Epog== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1777069836; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=8q+Edl5B0BoHOtnJ5aXvLyrL7WOndyD1BtPRaSy7gwY=; b=I2qO417ZiDP9lJEesBX1dR7keWaitWgBPpzqNRNrYc/2+DnqPlTU9UTNww0ryKTPJC3nMY CuyUhnRUZe0VugiWP2QoajdwlxYQDUwuk8WKDMhWHGbIxWbnlmbXYVRW2PZuZzW3sVQxjT B343TGR8up3v8iw08vtLedhEw8YOlAyQKSCAekFrNTKFO9imiyg77palB+XQG2H4WmheEW t03NV5f1wOIQmpHKczlOBUn2i0qaufUvtGuveO0dQSjGhxFgx2AtVyMINnXywJo8fqmD/N oH0LtGWQSmXTxffonL4lV0juYvOGx2kcDzd8onNgHIDFMLVFojzetDgGDpaSwg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4g2SMw17RRz1Km3 for ; Fri, 24 Apr 2026 22:30:36 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 428ac by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Fri, 24 Apr 2026 22:30:36 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Colin Percival Subject: git: 0f7b8f79f67b - main - ena: Budget rx descriptors, not packets List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: cperciva X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 0f7b8f79f67b25cb0727c7b7d604eb1eec91fef1 Auto-Submitted: auto-generated Date: Fri, 24 Apr 2026 22:30:36 +0000 Message-Id: <69ebef0c.428ac.36a81d7c@gitrepo.freebsd.org> The branch main has been updated by cperciva: URL: https://cgit.FreeBSD.org/src/commit/?id=0f7b8f79f67b25cb0727c7b7d604eb1eec91fef1 commit 0f7b8f79f67b25cb0727c7b7d604eb1eec91fef1 Author: Colin Percival AuthorDate: 2026-04-17 17:40:00 +0000 Commit: Colin Percival CommitDate: 2026-04-24 22:30:13 +0000 ena: Budget rx descriptors, not packets We had ENA_RX_BUDGET = 256 in order to allow up to 256 received packets to be processed before we do other cleanups (handling tx packets and, critically, refilling the rx buffer ring). Since the ring holds 1024 buffers by default, this was fine for normal packets: We refill the ring when it falls below 7/8 full, and even with a large burst of incoming packets allowing it to fall by another 1/4 before we consider refilling the ring still leaves it at 7/8 - 1/4 = 5/8 full. With jumbos, the story is different: A 9k jumbo (as is used by default within the EC2 network) consumes 3 descriptors, so a single rx cleanup pass can consume 3/4 of the default-sized rx ring; if the rx buffer ring wasn't completely full before a packet burst arrives, this puts us perilously close to running out of rx buffers. This precise failure mode has been observed on some EC2 instance types within a Cluster Placement Group, resulting in the nominal 10 Gbps single-flow throughput between instances dropping to ~100 Mbps as a result of repeated rx overruns causing packet loss and ultimately retransmission timeouts. To correct this, switch from processing up to ENA_RX_BUDGET (256) packets to processing up to ENA_RX_DESC_BUDGET (256) descriptors (or slightly more, if we hit the limit in the middle of a packet). This ensures that, even with jumbos, we refill the ring before processing most of a ring worth of descriptors, and returns the throughput to expected levels. Note that theoretically up to ENA_PKT_MAX_BUFS (19) descriptors can be used for a single packet, in which case even 54 packets would exhaust the default rx buffer ring; it's not clear if this ever occurs in practice, but this fix will address that case as well. Reviewed by: akiyano Sponsored by: Amazon MFC after: 6 days Differential Revision: https://reviews.freebsd.org/D56479 --- sys/dev/ena/ena.h | 4 ++-- sys/dev/ena/ena_datapath.c | 13 ++++++++++--- 2 files changed, 12 insertions(+), 5 deletions(-) diff --git a/sys/dev/ena/ena.h b/sys/dev/ena/ena.h index f67c7002327d..b2156437f847 100644 --- a/sys/dev/ena/ena.h +++ b/sys/dev/ena/ena.h @@ -99,8 +99,8 @@ * of TCP retransmissions. */ #define ENA_TX_BUDGET 128 -/* RX cleanup budget. -1 stands for infinity. */ -#define ENA_RX_BUDGET 256 +/* RX cleanup budget, in descriptors. -1 stands for infinity. */ +#define ENA_RX_DESC_BUDGET 256 /* * How many times we can repeat cleanup in the io irq handling routine if the * RX or TX budget was depleted. diff --git a/sys/dev/ena/ena_datapath.c b/sys/dev/ena/ena_datapath.c index 57148d8ef81f..91e3e3b6e4cd 100644 --- a/sys/dev/ena/ena_datapath.c +++ b/sys/dev/ena/ena_datapath.c @@ -571,7 +571,7 @@ ena_rx_cleanup(struct ena_ring *rx_ring) uint32_t do_if_input = 0; unsigned int qid; int rc, i; - int budget = ENA_RX_BUDGET; + int budget = (ENA_RX_DESC_BUDGET == -1) ? INT_MAX : ENA_RX_DESC_BUDGET; #ifdef DEV_NETMAP int done; #endif /* DEV_NETMAP */ @@ -680,7 +680,14 @@ ena_rx_cleanup(struct ena_ring *rx_ring) counter_u64_add_protected(rx_ring->rx_stats.cnt, 1); counter_u64_add_protected(adapter->hw_stats.rx_packets, 1); counter_exit(); - } while (--budget); + + /* + * Adjust our budget; note that we count descriptors, not + * packets, since we need to ensure we don't run out of rx + * buffers when receiving jumbos. + */ + budget -= ena_rx_ctx.descs; + } while (budget > 0); rx_ring->next_to_clean = next_to_clean; @@ -695,7 +702,7 @@ ena_rx_cleanup(struct ena_ring *rx_ring) tcp_lro_flush_all(&rx_ring->lro); - return (budget == 0); + return (budget <= 0); } static void