From owner-svn-src-all@FreeBSD.ORG Mon Sep 15 21:09:19 2014 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D07AB5DB; Mon, 15 Sep 2014 21:09:19 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B2AD7F4B; Mon, 15 Sep 2014 21:09:19 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.9/8.14.9) with ESMTP id s8FL9JIM009512; Mon, 15 Sep 2014 21:09:19 GMT (envelope-from adrian@FreeBSD.org) Received: (from adrian@localhost) by svn.freebsd.org (8.14.9/8.14.9/Submit) id s8FL9JMp009510; Mon, 15 Sep 2014 21:09:19 GMT (envelope-from adrian@FreeBSD.org) Message-Id: <201409152109.s8FL9JMp009510@svn.freebsd.org> X-Authentication-Warning: svn.freebsd.org: adrian set sender to adrian@FreeBSD.org using -f From: Adrian Chadd Date: Mon, 15 Sep 2014 21:09:19 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r271649 - in head/sys: conf modules/ixgbe X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Sep 2014 21:09:20 -0000 Author: adrian Date: Mon Sep 15 21:09:19 2014 New Revision: 271649 URL: http://svnweb.freebsd.org/changeset/base/271649 Log: Disable flow-director support until it's been debugged and verified. The flowdirector feature shares on-chip memory with other things such as the RX buffers. In theory it should be configured in a way that doesn't interfere with the rest of operation. In practice, the RX buffer calculation didn't take the flow-director allocation into account and there'd be overlap. This lead to various garbage frames being received containing what looks like internal NIC state. What _I_ saw was traffic ending up in the wrong RX queues. If I was doing a UDP traffic test with only one NIC ring receiving traffic, everything is fine. If I fired up a second UDP stream which came in on another ring, there'd be a few percent of traffic from both rings ending up in the wrong ring. Ie, the RSS hash would indicate it was supposed to come in ring X, but it'd come in ring Y. However, when the allocation was fixed up, the developers at Verisign still saw traffic stalls. The flowdirector feature ends up fiddling with the NIC to do various attempts at load balancing connections by populating flow table rules based on sampled traffic. It's likely that all of that has to be carefully reviewed and made less "magic". So for now the flow director feature is disabled (which fixes both what I was seeing and what they were seeing) until it's all much more debugged and verified. Tested: * (me) 82599EB 2x10G NIC, RSS UDP testing. * (verisign) not sure on the NIC (but likely 82599), 100k-200k/sec TCP transaction tests. Submitted by: Marc De La Gueronniere MFC after: 1 week Sponsored by: Verisign, Inc. Modified: head/sys/conf/files head/sys/modules/ixgbe/Makefile Modified: head/sys/conf/files ============================================================================== --- head/sys/conf/files Mon Sep 15 20:54:12 2014 (r271648) +++ head/sys/conf/files Mon Sep 15 21:09:19 2014 (r271649) @@ -1779,7 +1779,7 @@ dev/ixgb/if_ixgb.c optional ixgb dev/ixgb/ixgb_ee.c optional ixgb dev/ixgb/ixgb_hw.c optional ixgb dev/ixgbe/ixgbe.c optional ixgbe inet \ - compile-with "${NORMAL_C} -I$S/dev/ixgbe -DSMP -DIXGBE_FDIR" + compile-with "${NORMAL_C} -I$S/dev/ixgbe -DSMP" dev/ixgbe/ixv.c optional ixgbe inet \ compile-with "${NORMAL_C} -I$S/dev/ixgbe" dev/ixgbe/ixgbe_phy.c optional ixgbe inet \ Modified: head/sys/modules/ixgbe/Makefile ============================================================================== --- head/sys/modules/ixgbe/Makefile Mon Sep 15 20:54:12 2014 (r271648) +++ head/sys/modules/ixgbe/Makefile Mon Sep 15 21:09:19 2014 (r271649) @@ -10,6 +10,6 @@ SRCS += ixgbe.c ixv.c SRCS += ixgbe_common.c ixgbe_api.c ixgbe_phy.c ixgbe_mbx.c ixgbe_vf.c SRCS += ixgbe_dcb.c ixgbe_dcb_82598.c ixgbe_dcb_82599.c SRCS += ixgbe_82599.c ixgbe_82598.c ixgbe_x540.c -CFLAGS+= -I${.CURDIR}/../../dev/ixgbe -DSMP -DIXGBE_FDIR +CFLAGS+= -I${.CURDIR}/../../dev/ixgbe -DSMP .include