Date: Fri, 5 Jan 2018 09:39:15 -0800 From: Matt Joras <mjoras@freebsd.org> To: Eugene Grosbein <eugen@grosbein.net> Cc: Steven Hartland <steven@multiplay.co.uk>, src-committers <src-committers@freebsd.org>, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r327559 - in head: . sys/net Message-ID: <CADdTf%2BjJVn_X6UNywnis-KXq6Dua7B8F=2kuND_N6bJ1ZH0D_A@mail.gmail.com> In-Reply-To: <5A4FB6BC.6090506@grosbein.net> References: <201801042005.w04K5liB049411@repo.freebsd.org> <5A4E9397.9000308@grosbein.net> <f133b587-1f7e-4594-31d1-974775ad55be@freebsd.org> <5A4EDC62.50508@grosbein.net> <c42bfb9e-e583-aca3-bf0d-4d92c0153d2f@multiplay.co.uk> <5A4F824C.1060405@grosbein.net> <97d173fb-4f47-609d-8319-07282a283473@multiplay.co.uk> <CADdTf%2Bi47-X1HVb=JBFi53wOgbX1x1euHr53-YyqNDDFNF9PLw@mail.gmail.com> <5A4FB6BC.6090506@grosbein.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Jan 5, 2018 at 9:32 AM, Eugene Grosbein <eugen@grosbein.net> wrote: > 06.01.2018 0:28, Matt Joras wrote: > >> For what it's worth, this was the conclusion I came to, and at Isilon >> we've made the same change being discussed here. For the case of >> drivers that end up using a queue index for the flowid, you end up >> with pathological behavior on the lagg; the flowid ends up getting >> right shifted by (default) 16. So in the case of e.g. two bxe(4) >> interfaces with 4 queues, you always end up choosing the interface in >> the lagg at index 0. > > Not all drivers have this bug. These are drivers that needs to be fixed to not shift by 16, not lagg. > I don't follow. It is if_lagg that does the shifting. For loadbalance it is done directly in lagg_snd_tag_alloc, and for LACP it is done in a separate fucntion, lacp_select_tx_port_by_hash. For both it shifts the flowid by the flowid_shift set on the lagg sc, which defaults to 16. You could make the argument that we should fix every driver that sets a queue index to instead use an RSS hash, but that seems like more work than simply disabling the use of flowid in if_lagg by default. For cases where this has an appreciable impact on forwarding performance the sysctl can be flipped back. That seems more reasonable to me than making laggs effectively useless for anyone using any one of a random set of drivers that set the flowid to a queue index (grep for "flowid =" and you can see which drivers do this). Matt
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CADdTf%2BjJVn_X6UNywnis-KXq6Dua7B8F=2kuND_N6bJ1ZH0D_A>