From owner-svn-src-stable-10@FreeBSD.ORG Thu Jan 2 01:51:55 2014 Return-Path: Delivered-To: svn-src-stable-10@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 47435CA8; Thu, 2 Jan 2014 01:51:55 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 32377174C; Thu, 2 Jan 2014 01:51:55 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id s021ptNT044257; Thu, 2 Jan 2014 01:51:55 GMT (envelope-from scottl@svn.freebsd.org) Received: (from scottl@localhost) by svn.freebsd.org (8.14.7/8.14.7/Submit) id s021psf4044254; Thu, 2 Jan 2014 01:51:54 GMT (envelope-from scottl@svn.freebsd.org) Message-Id: <201401020151.s021psf4044254@svn.freebsd.org> From: Scott Long Date: Thu, 2 Jan 2014 01:51:54 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r260179 - stable/10/sys/net X-SVN-Group: stable-10 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-stable-10@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: SVN commit messages for only the 10-stable src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Jan 2014 01:51:55 -0000 Author: scottl Date: Thu Jan 2 01:51:54 2014 New Revision: 260179 URL: http://svnweb.freebsd.org/changeset/base/260179 Log: MFC r260070 Multi-queue NIC drivers and multi-port lagg tend to use the same lower bits of the flowid as each other, resulting in a poor distribution of packets among queues in certain cases. Work around this by adding a set of sysctls for controlling a bit-shift on the flowid when doing multi-port aggrigation in lagg and lacp. By default, lagg/lacp will now use bits 16 and higher instead of 0 and higher. Obtained from: Netflix Modified: stable/10/sys/net/ieee8023ad_lacp.c stable/10/sys/net/if_lagg.c stable/10/sys/net/if_lagg.h Directory Properties: stable/10/ (props changed) Modified: stable/10/sys/net/ieee8023ad_lacp.c ============================================================================== --- stable/10/sys/net/ieee8023ad_lacp.c Thu Jan 2 01:44:14 2014 (r260178) +++ stable/10/sys/net/ieee8023ad_lacp.c Thu Jan 2 01:51:54 2014 (r260179) @@ -874,7 +874,7 @@ lacp_select_tx_port(struct lagg_softc *s } if (sc->use_flowid && (m->m_flags & M_FLOWID)) - hash = m->m_pkthdr.flowid; + hash = m->m_pkthdr.flowid >> sc->flowid_shift; else hash = lagg_hashmbuf(sc, m, lsc->lsc_hashkey); hash %= pm->pm_count; Modified: stable/10/sys/net/if_lagg.c ============================================================================== --- stable/10/sys/net/if_lagg.c Thu Jan 2 01:44:14 2014 (r260178) +++ stable/10/sys/net/if_lagg.c Thu Jan 2 01:51:54 2014 (r260179) @@ -184,6 +184,11 @@ TUNABLE_INT("net.link.lagg.default_use_f SYSCTL_INT(_net_link_lagg, OID_AUTO, default_use_flowid, CTLFLAG_RW, &def_use_flowid, 0, "Default setting for using flow id for load sharing"); +static int def_flowid_shift = 16; /* Default value for using M_FLOWID */ +TUNABLE_INT("net.link.lagg.default_flowid_shift", &def_flowid_shift); +SYSCTL_INT(_net_link_lagg, OID_AUTO, default_flowid_shift, CTLFLAG_RW, + &def_flowid_shift, 0, + "Default setting for flowid shift for load sharing"); static int lagg_modevent(module_t mod, int type, void *data) @@ -293,12 +298,17 @@ lagg_clone_create(struct if_clone *ifc, sysctl_ctx_init(&sc->ctx); snprintf(num, sizeof(num), "%u", unit); sc->use_flowid = def_use_flowid; + sc->flowid_shift = def_flowid_shift; sc->sc_oid = oid = SYSCTL_ADD_NODE(&sc->ctx, &SYSCTL_NODE_CHILDREN(_net_link, lagg), OID_AUTO, num, CTLFLAG_RD, NULL, ""); SYSCTL_ADD_INT(&sc->ctx, SYSCTL_CHILDREN(oid), OID_AUTO, - "use_flowid", CTLTYPE_INT|CTLFLAG_RW, &sc->use_flowid, sc->use_flowid, - "Use flow id for load sharing"); + "use_flowid", CTLTYPE_INT|CTLFLAG_RW, &sc->use_flowid, + sc->use_flowid, "Use flow id for load sharing"); + SYSCTL_ADD_INT(&sc->ctx, SYSCTL_CHILDREN(oid), OID_AUTO, + "flowid_shift", CTLTYPE_INT|CTLFLAG_RW, &sc->flowid_shift, + sc->flowid_shift, + "Shift flowid bits to prevent multiqueue collisions"); SYSCTL_ADD_INT(&sc->ctx, SYSCTL_CHILDREN(oid), OID_AUTO, "count", CTLTYPE_INT|CTLFLAG_RD, &sc->sc_count, sc->sc_count, "Total number of ports"); @@ -1850,7 +1860,7 @@ lagg_lb_start(struct lagg_softc *sc, str uint32_t p = 0; if (sc->use_flowid && (m->m_flags & M_FLOWID)) - p = m->m_pkthdr.flowid; + p = m->m_pkthdr.flowid >> sc->flowid_shift; else p = lagg_hashmbuf(sc, m, lb->lb_key); p %= sc->sc_count; Modified: stable/10/sys/net/if_lagg.h ============================================================================== --- stable/10/sys/net/if_lagg.h Thu Jan 2 01:44:14 2014 (r260178) +++ stable/10/sys/net/if_lagg.h Thu Jan 2 01:51:54 2014 (r260179) @@ -231,6 +231,7 @@ struct lagg_softc { struct sysctl_ctx_list ctx; /* sysctl variables */ struct sysctl_oid *sc_oid; /* sysctl tree oid */ int use_flowid; /* use M_FLOWID */ + int flowid_shift; /* shift the flowid */ }; struct lagg_port {