From owner-freebsd-net@FreeBSD.ORG Mon Mar 23 23:58:45 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1D3B523E; Mon, 23 Mar 2015 23:58:45 +0000 (UTC) Received: from mail-wi0-x235.google.com (mail-wi0-x235.google.com [IPv6:2a00:1450:400c:c05::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AB262243; Mon, 23 Mar 2015 23:58:44 +0000 (UTC) Received: by wibg7 with SMTP id g7so61084465wib.1; Mon, 23 Mar 2015 16:58:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=vaRcF9K4bngPk0xeYn5ncvP6IB1axgdfVBezM5p41wI=; b=zUOAItMbZExsyLCLnI5AtqY9ACwJswVbU+GWDF38fsewpGrsMmQhahLdMooLYMsfmb hOiSR39nNSrEAiHl5mla4Cgp2yxH0f/mg+2arT1w/t3N7k0kFWin1u7bmLYjK+eB3Vpp FEI8L08I6e5q2iw76w9gAIbWt/hO6XaCaYUJdT5YWDxxa+A/WnRKXSm6/H6FVQINTKEh BdhohbgIyxr2NOFjPlHuZqhGaLCnI6m560FYTMsaVYD9gboCu7Jj0qNZGl+E/C+JbDEG 84eRiVwlaFt//wyxpoJEtU7pwBUy7hl15q7G7T/TSCrzzZ0YEqsO7BMHnZ0l7aHra3+k CuWw== MIME-Version: 1.0 X-Received: by 10.180.14.66 with SMTP id n2mr20299167wic.64.1427155123235; Mon, 23 Mar 2015 16:58:43 -0700 (PDT) Received: by 10.194.162.103 with HTTP; Mon, 23 Mar 2015 16:58:43 -0700 (PDT) In-Reply-To: <20150323234214.GU53237@strugglingcoder.info> References: <20150323233908.GT53237@strugglingcoder.info> <20150323234214.GU53237@strugglingcoder.info> Date: Mon, 23 Mar 2015 16:58:43 -0700 Message-ID: Subject: Re: Full 32bit flowid from igb(4) From: Jack Vogel To: hiren panchasara Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: Jack F Vogel , FreeBSD Net , Adrian Chadd , erj@freebsd.org, Jason Wolfe X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Mar 2015 23:58:45 -0000 I think that line was there at one time outside the RSS define, I'd have to take a look at the code a bit more, or maybe Eric will.... Jack On Mon, Mar 23, 2015 at 4:42 PM, hiren panchasara < hiren@strugglingcoder.info> wrote: > Correcting Eric's email and subject line. > > On 03/23/15 at 04:39P, hiren panchasara wrote: > > scottl@freebsd.org > > Bcc: > > Subject: Full 32bit flowid from igb(4) [was: Re: Unbalanced LACP link] > > Reply-To: > > In-Reply-To: <20150319175145.GH53237@strugglingcoder.info> > > > > On 03/19/15 at 10:51P, hiren panchasara wrote: > > > On 03/17/15 at 12:34P, Adrian Chadd wrote: > > > > On 17 March 2015 at 11:33, Jason Wolfe wrote: > > > > > On Mon, Mar 16, 2015 at 2:43 AM, Hans Petter Selasky < > hps@selasky.org> wrote: > > > > >> On 03/16/15 10:37, Vitalii Duk wrote: > > > > >>> > > > > >>> I've changed use_flowid to 0 and it helped! But isn't it setting > > > > >>> significant? In a description it says "Shift flowid bits to > prevent > > > > >>> multiqueue collisions". > > > > >> > > > > >> > > > > >> Hi, > > > > >> > > > > >> Maybe your ethernet hardware is not properly setting the m_flowid > ... > > > > >> > > > > >> --HPS > > > > >> > > > > > > > > > > Flip use_flowid back to 1 and try setting > > > > > net.link.lagg.default_flowid_shift / net.link.lagg.X.flowid_shift > to 0 > > > > > as Hiren suggested. r260179 added this shift, which has caused us > > > > > balancing issues with the i350/igb. > > > > > > > > > > https://svnweb.freebsd.org/base?view=revision&revision=260179 > > > > > > > > > > Based on Adrian's comment about igb/ixgbe not setting the 'full > > > > > flowid' under normal conditions, does that mean this shift should > be 0 > > > > > by default to ensure we don't break balancing for devices that only > > > > > set the CPU/MSIX queue? > > > > > > > > Or we can just see if there's anything wrong with putting the full 32 > > > > bit RSS flowid in received packets that have them. > > > > > > It'd be nice to have but for now I am proposing following to fix a > known > > > broken case because of an optimization: > > > https://reviews.freebsd.org/D2098 > > > > Turns out, setting flowid_shift to 0 is not the correct solution. Please > > look at the review above for more details. > > > > Looking at the code, we have a way to get full flowid but it's hidden > > under "ifdef RSS": > > rxr->fmp->m_pkthdr.flowid = le32toh(cur->wb.lower.hi_dword.rss); > > > > Using just this line gives me good hash on I350 igb(4) chipset that we > > have. > > > > Is it possible to expose this outside RSS? Would this break other > > things/chips? > > > > Cheers, > > Hiren > >