Date: Fri, 5 Jan 2018 21:01:09 +0000 From: Steven Hartland <steven@multiplay.co.uk> To: Matt Joras <mjoras@freebsd.org>, Eugene Grosbein <eugen@grosbein.net> Cc: src-committers <src-committers@freebsd.org>, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r327559 - in head: . sys/net Message-ID: <cfd47cb5-bb55-f729-d359-e32c65d77b0c@multiplay.co.uk> In-Reply-To: <CADdTf%2BjJVn_X6UNywnis-KXq6Dua7B8F=2kuND_N6bJ1ZH0D_A@mail.gmail.com> References: <201801042005.w04K5liB049411@repo.freebsd.org> <5A4E9397.9000308@grosbein.net> <f133b587-1f7e-4594-31d1-974775ad55be@freebsd.org> <5A4EDC62.50508@grosbein.net> <c42bfb9e-e583-aca3-bf0d-4d92c0153d2f@multiplay.co.uk> <5A4F824C.1060405@grosbein.net> <97d173fb-4f47-609d-8319-07282a283473@multiplay.co.uk> <CADdTf%2Bi47-X1HVb=JBFi53wOgbX1x1euHr53-YyqNDDFNF9PLw@mail.gmail.com> <5A4FB6BC.6090506@grosbein.net> <CADdTf%2BjJVn_X6UNywnis-KXq6Dua7B8F=2kuND_N6bJ1ZH0D_A@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 05/01/2018 17:39, Matt Joras wrote: > On Fri, Jan 5, 2018 at 9:32 AM, Eugene Grosbein<eugen@grosbein.net> wrote: >> 06.01.2018 0:28, Matt Joras wrote: >> >>> For what it's worth, this was the conclusion I came to, and at Isilon >>> we've made the same change being discussed here. For the case of >>> drivers that end up using a queue index for the flowid, you end up >>> with pathological behavior on the lagg; the flowid ends up getting >>> right shifted by (default) 16. So in the case of e.g. two bxe(4) >>> interfaces with 4 queues, you always end up choosing the interface in >>> the lagg at index 0. >> Not all drivers have this bug. These are drivers that needs to be fixed to not shift by 16, not lagg. >> > I don't follow. It is if_lagg that does the shifting. For loadbalance > it is done directly in lagg_snd_tag_alloc, and for LACP it is done in > a separate fucntion, lacp_select_tx_port_by_hash. For both it shifts > the flowid by the flowid_shift set on the lagg sc, which defaults to > 16. For reference lacp_select_tx_port is the normal method, lacp_select_tx_port_by_hash is only used if RATELIMIT is enabled. They both do the same shift though, so ... > You could make the argument that we should fix every driver that sets > a queue index to instead use an RSS hash, but that seems like more > work than simply disabling the use of flowid in if_lagg by default. > For cases where this has an appreciable impact on forwarding > performance the sysctl can be flipped back. That seems more reasonable > to me than making laggs effectively useless for anyone using any one > of a random set of drivers that set the flowid to a queue index (grep > for "flowid =" and you can see which drivers do this). > > Matt
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?cfd47cb5-bb55-f729-d359-e32c65d77b0c>