From owner-svn-src-all@freebsd.org Fri Jan 5 21:01:11 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8363AEC05F7 for ; Fri, 5 Jan 2018 21:01:11 +0000 (UTC) (envelope-from steven@multiplay.co.uk) Received: from mail-wm0-x22e.google.com (mail-wm0-x22e.google.com [IPv6:2a00:1450:400c:c09::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0C8717E39B for ; Fri, 5 Jan 2018 21:01:11 +0000 (UTC) (envelope-from steven@multiplay.co.uk) Received: by mail-wm0-x22e.google.com with SMTP id f206so4637991wmf.5 for ; Fri, 05 Jan 2018 13:01:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=from:subject:to:cc:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=bSI5yhqw+EP0Vk9ilPPK/niRexyAe4eOwCk+9dxTgyM=; b=F+OXOKZRT6tmWn5AZVGRNmFFTNyTIAnDI59hqGkGfe70nLZGxVvLgsAjn2zMJLOMaE Q7KZIFiSlAI52fjBUznCHCwoZlGrxA5sH/J0/P3dC6S594UdO3K+EjRFubXGLYNhzBCV tIRUxsieSbIClfnVH7l7I6l2tGbq1NvbhfJoAzziECYLFYk9WLg2YrmF8sEjLjDFcLUo Lc3nYF5H8HKAE8A/8yqwMb+h9nK/eqZaPyC7VUgSP6UgeR+u33aMzALiihYZNKMedvqa MnPuaHoQUOw8h3QgY1I+Kb9pLmI8Ly/1BeSlvdUEb43a8GvPNccsTjyRIJPYE8dD2iB/ BKfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:subject:to:cc:references:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=bSI5yhqw+EP0Vk9ilPPK/niRexyAe4eOwCk+9dxTgyM=; b=AqUOtENN88pDA9ISOUlIwjkOKgaqtjjzsEZkfqdExNAryubanyGMq8CEFg+aIfG3l0 7NrRLXdH7yGw6afBVX/qUnsUu9erRNh78xvvo+ZeqJww7I3IPQ3+xh8Iv7xb0cHzL39G NcRz2ZnUS+Sldr7d2rx0ojrfaz6Re5fTdoSxm7oS5MT586usDV7/KD6OQUSsY+6nTmd1 bs25GZNDX82stPlYj7OWWfhDvyj62RyozoC+HHrdGV4iPoGSVXJQ1TC2WjulQ1KRIyNS Z5/jh8TXmgpGLeJfHonlCwR+AyjfjDB7EhC7C6C0zCA9Ddfyt8QWv6qXGVNP1m318osg jTfQ== X-Gm-Message-State: AKGB3mJhUMIWwB0QP9oZxKNLkiMvCYDLhMfB5UGwayOI88i++D8ceQnO fqkz2ZXvWsJ+JJOpP+Po9V8URw== X-Google-Smtp-Source: ACJfBosFkbsNW57fFlshbbSQkqqGwcNtj7CeJaWKFqi4ZUbKU5OJVF6QHd/lAxLraXkWoTLii3neLQ== X-Received: by 10.80.240.10 with SMTP id r10mr5808583edl.91.1515186068887; Fri, 05 Jan 2018 13:01:08 -0800 (PST) Received: from [10.10.1.111] ([185.97.61.1]) by smtp.gmail.com with ESMTPSA id j27sm5466316eda.59.2018.01.05.13.01.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 Jan 2018 13:01:07 -0800 (PST) From: Steven Hartland Subject: Re: svn commit: r327559 - in head: . sys/net To: Matt Joras , Eugene Grosbein Cc: src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org References: <201801042005.w04K5liB049411@repo.freebsd.org> <5A4E9397.9000308@grosbein.net> <5A4EDC62.50508@grosbein.net> <5A4F824C.1060405@grosbein.net> <97d173fb-4f47-609d-8319-07282a283473@multiplay.co.uk> <5A4FB6BC.6090506@grosbein.net> Message-ID: Date: Fri, 5 Jan 2018 21:01:09 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 21:01:11 -0000 On 05/01/2018 17:39, Matt Joras wrote: > On Fri, Jan 5, 2018 at 9:32 AM, Eugene Grosbein wrote: >> 06.01.2018 0:28, Matt Joras wrote: >> >>> For what it's worth, this was the conclusion I came to, and at Isilon >>> we've made the same change being discussed here. For the case of >>> drivers that end up using a queue index for the flowid, you end up >>> with pathological behavior on the lagg; the flowid ends up getting >>> right shifted by (default) 16. So in the case of e.g. two bxe(4) >>> interfaces with 4 queues, you always end up choosing the interface in >>> the lagg at index 0. >> Not all drivers have this bug. These are drivers that needs to be fixed to not shift by 16, not lagg. >> > I don't follow. It is if_lagg that does the shifting. For loadbalance > it is done directly in lagg_snd_tag_alloc, and for LACP it is done in > a separate fucntion, lacp_select_tx_port_by_hash. For both it shifts > the flowid by the flowid_shift set on the lagg sc, which defaults to > 16. For reference lacp_select_tx_port is the normal method, lacp_select_tx_port_by_hash is only used if RATELIMIT is enabled. They both do the same shift though, so ... > You could make the argument that we should fix every driver that sets > a queue index to instead use an RSS hash, but that seems like more > work than simply disabling the use of flowid in if_lagg by default. > For cases where this has an appreciable impact on forwarding > performance the sysctl can be flipped back. That seems more reasonable > to me than making laggs effectively useless for anyone using any one > of a random set of drivers that set the flowid to a queue index (grep > for "flowid =" and you can see which drivers do this). > > Matt