From owner-svn-src-head@freebsd.org Fri Jan 5 17:46:01 2018 Return-Path: <owner-svn-src-head@freebsd.org> Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AA90AEB65F7; Fri, 5 Jan 2018 17:46:01 +0000 (UTC) (envelope-from matt.joras@gmail.com) Received: from mail-wm0-f51.google.com (mail-wm0-f51.google.com [74.125.82.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4254474BE1; Fri, 5 Jan 2018 17:46:00 +0000 (UTC) (envelope-from matt.joras@gmail.com) Received: by mail-wm0-f51.google.com with SMTP id g130so7308600wme.0; Fri, 05 Jan 2018 09:46:00 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=VBaZN1zZukBaqI8JtPzcAuRatvJsMnrTtymFy7YE5xw=; b=ZogbTJcmhF+DmkHTIMC72Y1/3BYq+iqvK7COVzYVNwCkN4gP7oWSQzYBj4Oi+9n5Tg lWnJCg6Dfct/ZNjGwGh6m5Kypu8OuvWH/DKGG9yj99r/4MKRb6K64eUl7BEBLZIx4RFZ Oq05NKqfglni0CAikpvAi3IRAfrHBbxoPg+NAN2Pz3exO63x92xuo4I858O6XThWwzhd gRQGiV6zPVcrHVziiD00zm9UfAOgyqyVQJzPxnoplknyZmf1CD5x36sh9YZ6fhQaHo/W 7kUNkC8fIqb6rvUR5npKGnciCrZKhT/br9ROPdIxvxEo34PgpG9tadlwjNE8yTf3CnNt GHaA== X-Gm-Message-State: AKGB3mIfmyxtwWCrbHeouh73JydP6fPgXYgmBu7nV0TL6F/igBTqMnN2 x9aHutndnJO8SyB53ezI6JbjnBTH X-Google-Smtp-Source: ACJfBou19D+2ZxmiVtxfclRhWLs1kynM9HtRcsPnKlY3QUQOyIV8srxoWvBp0tdJzMcY5q75SCfhGg== X-Received: by 10.80.192.11 with SMTP id r11mr5152528edb.184.1515173956535; Fri, 05 Jan 2018 09:39:16 -0800 (PST) Received: from mail-wm0-f41.google.com (mail-wm0-f41.google.com. [74.125.82.41]) by smtp.gmail.com with ESMTPSA id q55sm4167468eda.43.2018.01.05.09.39.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 Jan 2018 09:39:16 -0800 (PST) Received: by mail-wm0-f41.google.com with SMTP id a79so3801952wma.0; Fri, 05 Jan 2018 09:39:16 -0800 (PST) X-Received: by 10.28.7.144 with SMTP id 138mr2618795wmh.128.1515173956048; Fri, 05 Jan 2018 09:39:16 -0800 (PST) MIME-Version: 1.0 Received: by 10.223.160.170 with HTTP; Fri, 5 Jan 2018 09:39:15 -0800 (PST) In-Reply-To: <5A4FB6BC.6090506@grosbein.net> References: <201801042005.w04K5liB049411@repo.freebsd.org> <5A4E9397.9000308@grosbein.net> <f133b587-1f7e-4594-31d1-974775ad55be@freebsd.org> <5A4EDC62.50508@grosbein.net> <c42bfb9e-e583-aca3-bf0d-4d92c0153d2f@multiplay.co.uk> <5A4F824C.1060405@grosbein.net> <97d173fb-4f47-609d-8319-07282a283473@multiplay.co.uk> <CADdTf+i47-X1HVb=JBFi53wOgbX1x1euHr53-YyqNDDFNF9PLw@mail.gmail.com> <5A4FB6BC.6090506@grosbein.net> From: Matt Joras <mjoras@freebsd.org> Date: Fri, 5 Jan 2018 09:39:15 -0800 X-Gmail-Original-Message-ID: <CADdTf+jJVn_X6UNywnis-KXq6Dua7B8F=2kuND_N6bJ1ZH0D_A@mail.gmail.com> Message-ID: <CADdTf+jJVn_X6UNywnis-KXq6Dua7B8F=2kuND_N6bJ1ZH0D_A@mail.gmail.com> Subject: Re: svn commit: r327559 - in head: . sys/net To: Eugene Grosbein <eugen@grosbein.net> Cc: Steven Hartland <steven@multiplay.co.uk>, src-committers <src-committers@freebsd.org>, svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the src tree for head/-current <svn-src-head.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/svn-src-head>, <mailto:svn-src-head-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/svn-src-head/> List-Post: <mailto:svn-src-head@freebsd.org> List-Help: <mailto:svn-src-head-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/svn-src-head>, <mailto:svn-src-head-request@freebsd.org?subject=subscribe> X-List-Received-Date: Fri, 05 Jan 2018 17:46:01 -0000 On Fri, Jan 5, 2018 at 9:32 AM, Eugene Grosbein <eugen@grosbein.net> wrote: > 06.01.2018 0:28, Matt Joras wrote: > >> For what it's worth, this was the conclusion I came to, and at Isilon >> we've made the same change being discussed here. For the case of >> drivers that end up using a queue index for the flowid, you end up >> with pathological behavior on the lagg; the flowid ends up getting >> right shifted by (default) 16. So in the case of e.g. two bxe(4) >> interfaces with 4 queues, you always end up choosing the interface in >> the lagg at index 0. > > Not all drivers have this bug. These are drivers that needs to be fixed to not shift by 16, not lagg. > I don't follow. It is if_lagg that does the shifting. For loadbalance it is done directly in lagg_snd_tag_alloc, and for LACP it is done in a separate fucntion, lacp_select_tx_port_by_hash. For both it shifts the flowid by the flowid_shift set on the lagg sc, which defaults to 16. You could make the argument that we should fix every driver that sets a queue index to instead use an RSS hash, but that seems like more work than simply disabling the use of flowid in if_lagg by default. For cases where this has an appreciable impact on forwarding performance the sysctl can be flipped back. That seems more reasonable to me than making laggs effectively useless for anyone using any one of a random set of drivers that set the flowid to a queue index (grep for "flowid =" and you can see which drivers do this). Matt