From owner-svn-src-head@FreeBSD.ORG Sat Mar 22 18:24:45 2014 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8C5D2A40; Sat, 22 Mar 2014 18:24:45 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5FB2B996; Sat, 22 Mar 2014 18:24:45 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s2MIOjVF042541; Sat, 22 Mar 2014 18:24:45 GMT (envelope-from glebius@svn.freebsd.org) Received: (from glebius@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s2MIOjq8042539; Sat, 22 Mar 2014 18:24:45 GMT (envelope-from glebius@svn.freebsd.org) Message-Id: <201403221824.s2MIOjq8042539@svn.freebsd.org> From: Gleb Smirnoff Date: Sat, 22 Mar 2014 18:24:45 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r263649 - head/sys/dev/sfxge X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Mar 2014 18:24:45 -0000 Author: glebius Date: Sat Mar 22 18:24:44 2014 New Revision: 263649 URL: http://svnweb.freebsd.org/changeset/base/263649 Log: sfxge: limit software Tx queue size. Previous implementation limits put queue size only (when Tx lock can't be acquired), but get queue may grow unboundedly which results in mbuf pools exhaustion and latency growth. Submitted by: Andrew Rybchenko Sponsored by: Solarflare Communications, Inc. Modified: head/sys/dev/sfxge/sfxge_tx.c head/sys/dev/sfxge/sfxge_tx.h Modified: head/sys/dev/sfxge/sfxge_tx.c ============================================================================== --- head/sys/dev/sfxge/sfxge_tx.c Sat Mar 22 17:28:14 2014 (r263648) +++ head/sys/dev/sfxge/sfxge_tx.c Sat Mar 22 18:24:44 2014 (r263649) @@ -461,6 +461,9 @@ sfxge_tx_qdpl_put(struct sfxge_txq *txq, sfxge_tx_qdpl_swizzle(txq); + if (stdp->std_count >= SFXGE_TX_DPL_GET_PKT_LIMIT_DEFAULT) + return (ENOBUFS); + *(stdp->std_getp) = mbuf; stdp->std_getp = &mbuf->m_nextpkt; stdp->std_count++; @@ -480,8 +483,8 @@ sfxge_tx_qdpl_put(struct sfxge_txq *txq, old_len = mp->m_pkthdr.csum_data; } else old_len = 0; - if (old_len >= SFXGE_TX_MAX_DEFERRED) - return ENOBUFS; + if (old_len >= SFXGE_TX_DPL_PUT_PKT_LIMIT_DEFAULT) + return (ENOBUFS); mbuf->m_pkthdr.csum_data = old_len + 1; mbuf->m_nextpkt = (void *)old; } while (atomic_cmpset_ptr(putp, old, new) == 0); @@ -512,12 +515,9 @@ sfxge_tx_packet_add(struct sfxge_txq *tx */ locked = mtx_trylock(&txq->lock); - /* - * Can only fail if we weren't able to get the lock. - */ if (sfxge_tx_qdpl_put(txq, m, locked) != 0) { - KASSERT(!locked, - ("sfxge_tx_qdpl_put() failed locked")); + if (locked) + mtx_unlock(&txq->lock); rc = ENOBUFS; goto fail; } Modified: head/sys/dev/sfxge/sfxge_tx.h ============================================================================== --- head/sys/dev/sfxge/sfxge_tx.h Sat Mar 22 17:28:14 2014 (r263648) +++ head/sys/dev/sfxge/sfxge_tx.h Sat Mar 22 18:24:44 2014 (r263649) @@ -75,7 +75,8 @@ struct sfxge_tx_mapping { enum sfxge_tx_buf_flags flags; }; -#define SFXGE_TX_MAX_DEFERRED 64 +#define SFXGE_TX_DPL_GET_PKT_LIMIT_DEFAULT 64 +#define SFXGE_TX_DPL_PUT_PKT_LIMIT_DEFAULT 64 /* * Deferred packet list.