From owner-svn-src-all@FreeBSD.ORG Wed Mar 25 13:41:28 2015 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1F8D4E78; Wed, 25 Mar 2015 13:41:28 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F2F91358; Wed, 25 Mar 2015 13:41:27 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.9/8.14.9) with ESMTP id t2PDfRST084424; Wed, 25 Mar 2015 13:41:27 GMT (envelope-from arybchik@FreeBSD.org) Received: (from arybchik@localhost) by svn.freebsd.org (8.14.9/8.14.9/Submit) id t2PDfRm8084140; Wed, 25 Mar 2015 13:41:27 GMT (envelope-from arybchik@FreeBSD.org) Message-Id: <201503251341.t2PDfRm8084140@svn.freebsd.org> X-Authentication-Warning: svn.freebsd.org: arybchik set sender to arybchik@FreeBSD.org using -f From: Andrew Rybchenko Date: Wed, 25 Mar 2015 13:41:27 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r280599 - in stable/10: share/man/man4 sys/dev/sfxge X-SVN-Group: stable-10 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2015 13:41:28 -0000 Author: arybchik Date: Wed Mar 25 13:41:26 2015 New Revision: 280599 URL: https://svnweb.freebsd.org/changeset/base/280599 Log: MFC: 280160 sfxge: add tunables to control LRO parameters on driver load time Sponsored by: Solarflare Communications, Inc. Approved by: gnn (mentor) Modified: stable/10/share/man/man4/sfxge.4 stable/10/sys/dev/sfxge/sfxge_rx.c Directory Properties: stable/10/ (props changed) Modified: stable/10/share/man/man4/sfxge.4 ============================================================================== --- stable/10/share/man/man4/sfxge.4 Wed Mar 25 13:32:27 2015 (r280598) +++ stable/10/share/man/man4/sfxge.4 Wed Mar 25 13:41:26 2015 (r280599) @@ -120,6 +120,27 @@ The value must be greater than or equal The maximum number of allocated RSS channels for the Nth adapter. If set to 0 or unset, the number of channels is determined by the number of CPU cores. +.It Va hw.sfxge.lro.table_size +Size of the LRO hash table. +Must be a power of 2. +A larger table means we can accelerate a larger number of streams. +.It Va hw.sfxge.lro.chain_max +The maximum length of a hash chain. +If chains get too long then the lookup time increases and may exceed +the benefit of LRO. +.It Va hw.sfxge.lro.idle_ticks +The maximum time (in ticks) that a connection can be idle before it's LRO +state is discarded. +.It Va hw.sfxge.lro.slow_start_packets +Number of packets with payload that must arrive in-order before a connection +is eligible for LRO. +The idea is we should avoid coalescing segments when the sender is in +slow-start because reducing the ACK rate can damage performance. +.It Va hw.sfxge.lro.loss_packets +Number of packets with payload that must arrive in-order following loss +before a connection is eligible for LRO. +The idea is we should avoid coalescing segments when the sender is recovering +from loss, because reducing the ACK rate can damage performance. .El .Sh SUPPORT For general information and support, Modified: stable/10/sys/dev/sfxge/sfxge_rx.c ============================================================================== --- stable/10/sys/dev/sfxge/sfxge_rx.c Wed Mar 25 13:32:27 2015 (r280598) +++ stable/10/sys/dev/sfxge/sfxge_rx.c Wed Mar 25 13:41:26 2015 (r280599) @@ -36,6 +36,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include @@ -56,20 +57,38 @@ __FBSDID("$FreeBSD$"); #define RX_REFILL_THRESHOLD(_entries) (EFX_RXQ_LIMIT(_entries) * 9 / 10) +SYSCTL_NODE(_hw_sfxge, OID_AUTO, lro, CTLFLAG_RD, NULL, + "Large receive offload (LRO) parameters"); + +#define SFXGE_LRO_PARAM(_param) SFXGE_PARAM(lro._param) + /* Size of the LRO hash table. Must be a power of 2. A larger table * means we can accelerate a larger number of streams. */ static unsigned lro_table_size = 128; +TUNABLE_INT(SFXGE_LRO_PARAM(table_size), &lro_table_size); +SYSCTL_UINT(_hw_sfxge_lro, OID_AUTO, table_size, CTLFLAG_RDTUN, + &lro_table_size, 0, + "Size of the LRO hash table (must be a power of 2)"); /* Maximum length of a hash chain. If chains get too long then the lookup * time increases and may exceed the benefit of LRO. */ static unsigned lro_chain_max = 20; +TUNABLE_INT(SFXGE_LRO_PARAM(chain_max), &lro_chain_max); +SYSCTL_UINT(_hw_sfxge_lro, OID_AUTO, chain_max, CTLFLAG_RDTUN, + &lro_chain_max, 0, + "The maximum length of a hash chain"); /* Maximum time (in ticks) that a connection can be idle before it's LRO * state is discarded. */ static unsigned lro_idle_ticks; /* initialised in sfxge_rx_init() */ +TUNABLE_INT(SFXGE_LRO_PARAM(idle_ticks), &lro_idle_ticks); +SYSCTL_UINT(_hw_sfxge_lro, OID_AUTO, idle_ticks, CTLFLAG_RDTUN, + &lro_idle_ticks, 0, + "The maximum time (in ticks) that a connection can be idle " + "before it's LRO state is discarded"); /* Number of packets with payload that must arrive in-order before a * connection is eligible for LRO. The idea is we should avoid coalescing @@ -77,6 +96,11 @@ static unsigned lro_idle_ticks; /* initi * can damage performance. */ static int lro_slow_start_packets = 2000; +TUNABLE_INT(SFXGE_LRO_PARAM(slow_start_packets), &lro_slow_start_packets); +SYSCTL_UINT(_hw_sfxge_lro, OID_AUTO, slow_start_packets, CTLFLAG_RDTUN, + &lro_slow_start_packets, 0, + "Number of packets with payload that must arrive in-order before " + "a connection is eligible for LRO"); /* Number of packets with payload that must arrive in-order following loss * before a connection is eligible for LRO. The idea is we should avoid @@ -84,6 +108,11 @@ static int lro_slow_start_packets = 2000 * reducing the ACK rate can damage performance. */ static int lro_loss_packets = 20; +TUNABLE_INT(SFXGE_LRO_PARAM(loss_packets), &lro_loss_packets); +SYSCTL_UINT(_hw_sfxge_lro, OID_AUTO, loss_packets, CTLFLAG_RDTUN, + &lro_loss_packets, 0, + "Number of packets with payload that must arrive in-order " + "following loss before a connection is eligible for LRO"); /* Flags for sfxge_lro_conn::l2_id; must not collide with EVL_VLID_MASK */ #define SFXGE_LRO_L2_ID_VLAN 0x4000 @@ -1200,6 +1229,13 @@ sfxge_rx_init(struct sfxge_softc *sc) int index; int rc; + if (!ISP2(lro_table_size)) { + log(LOG_ERR, "%s=%u must be power of 2", + SFXGE_LRO_PARAM(table_size), lro_table_size); + rc = EINVAL; + goto fail_lro_table_size; + } + if (lro_idle_ticks == 0) lro_idle_ticks = hz / 10 + 1; /* 100 ms */ @@ -1226,5 +1262,7 @@ fail: sfxge_rx_qfini(sc, index); sc->rxq_count = 0; + +fail_lro_table_size: return (rc); }