From owner-freebsd-net@FreeBSD.ORG Sun Sep 1 08:11:40 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 34986827; Sun, 1 Sep 2013 08:11:40 +0000 (UTC) (envelope-from melifaro@ipfw.ru) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id ECF3F2D0F; Sun, 1 Sep 2013 08:11:39 +0000 (UTC) Received: from secured.by.ipfw.ru ([95.143.220.47] helo=ws.su29.net) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1VFz0D-000EXb-52; Sun, 01 Sep 2013 08:11:05 +0400 Message-ID: <5222F661.1090601@ipfw.ru> Date: Sun, 01 Sep 2013 12:10:09 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130728 Thunderbird/17.0.7 MIME-Version: 1.0 To: jfv@FreeBSD.org Subject: Re: Flow ID, LACP, and igb References: In-Reply-To: X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "Justin T. Gibbs" , Alan Somers , net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Sep 2013 08:11:40 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 26.08.2013 21:18, Justin T. Gibbs wrote: > Hi Net, > > I'm an infrequent traveler through the networking code and would > appreciate some feedback on some proposed solutions to issues > Spectra has seen with outbound LACP traffic. > > lacp_select_tx_port() uses the flow ID if it is available in the > outbound mbuf to select the outbound port. The igb driver uses the > msix queue of the inbound packet to set a packet's flow ID. This > doesn't provide enough Not sure about igb, but ixgbe (according to advanced RX descriptor format, 7.1.6.2 @ 82599 datasheet) can provide 'real' RSS value which can be used in m_flowid instead of NIC queue id. (And, by the way, another RSS-related problem: there are cases when setting flowid does more harm, for example - PPPoE frames always being received at Q0. Partially this can be solved by analyzing RSS type from the same RX descriptor format (e.g. don't set flowid for RSS type 0x0), but there are other cases like GRE tunneling (where you probably want to perform deeper inspection in SW). So, can we have some kind of per-NIC sysctl disabling setting flowid on given port? (Yes, this should be in some kind of `ethtool` binary but we still don't have it..) ) Jack, what do you think? > bits of information to yield a high quality flow ID. If, for > example, the switch controlling inbound packet distribution does a > poor job, the outbound packet distribution will also be poorly > distributed. > > The majority of the adapters supported by this driver will compute > the Toeplitz RSS hash. Using this data seems to work quite well in > our tests (3 member LAGG group). Is there any reason we shouldn't > use the RSS hash for flow ID? > > We also tried disabling the use of flow ID and doing the hash > directly in the driver. Unfortunately, the current hash is pretty > weak. It multiplies by 33, which yield very poor distributions if > you need to mod the result by 3 (e.g. LAGG group with 3 members). > Alan modified the driver to use the FNV hash, which is already in > the kernel, and this yielded much better results. He is still > benchmarking the impact of this change. Assuming we can get decent > flow ID data, this should only impact outbound UDP, since the stack > doesn't provide a flow ID in this case. > > Are there other checksums we should be looking at in addition to > FNV? > > Thanks, Justin > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net To > unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" > -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (FreeBSD) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlIi9mEACgkQwcJ4iSZ1q2meFQCfQ9QO+y/9ArTXQBAB9RDCGVY2 SpEAnAgZ0vRYuJ0HMamCnpd8q8/yxsLE =RlDE -----END PGP SIGNATURE-----