Date: Thu, 3 Aug 2023 14:41:23 -0700 From: Alan Somers <asomers@freebsd.org> To: Eugene Grosbein <eugen@grosbein.net> Cc: FreeBSD <freebsd-stable@freebsd.org> Subject: Re: Best practices for cxgbei and link aggregation? Message-ID: <CAOtMX2h41y84L_T-T1WhiA9DC93y3dQEZW=M3kj4JrAQGZTUpA@mail.gmail.com> In-Reply-To: <708494cd-9c88-974e-da5f-1e8c1f270950@grosbein.net> References: <CAOtMX2jkwFXrjby7W-xyLSDUXhAwRKyrpBFTBMbD1pqQz9Z76g@mail.gmail.com> <708494cd-9c88-974e-da5f-1e8c1f270950@grosbein.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Aug 3, 2023 at 2:14=E2=80=AFPM Eugene Grosbein <eugen@grosbein.net>= wrote: > > 04.08.2023 0:56, Alan Somers wrote: > > > I'm trying to build a high-speed iSCSI server. I have two Chelsio T6 > > cards providing 4x 25GbE ports. I have a requirement for > > high-availability networking, and I also need multiple ports' worth of > > bandwidth. What's the best way to use them? > > > > First I tried LACP, of course. That works. But it doesn't work in > > combination with cxgbei iSCSI offload. The clients can't connect. > > This makes sense, because the offload engine probably requires all > > packets from a single iSCSI session to enter and leave through the > > same network port. With LACP, that won't be the case. > > This is a common misunderstanding. In fact, LACP has nothing to do with p= er-port > traffic distribution. Used kind of hashing function is not a part of LACP= . > You just need to use link partners capable of good hashing. > > For example, FreeBSD lagg(4) hashing function is capable of using L2 (MAC= ), L3 (IP) and L4 (TCP/UDP ports) > headers and any kind of combination of such headers to make sure that pac= kets if single flow go out > using same port. Take a look at ifconfig.8 manual page for "lagghash". For iSCSI, any combination of those hash settings should result in a flow that is always tied to a single port. But for cxgbei to work, I need to ensure that the switch also hashes flows in the same way, to the _exact_same_port_. Even if I had access to the switch's LACP configuration, which I do not (but perhaps could acquire), is there any guarantee that it would hash things the same way as FreeBSD? Is the LACP hashing algorithm standardized?
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOtMX2h41y84L_T-T1WhiA9DC93y3dQEZW=M3kj4JrAQGZTUpA>