Date: Sat, 12 Oct 2024 15:51:57 -0600 From: John Nielsen <lists@jnielsen.net> To: Navdeep Parhar <np@freebsd.org> Cc: freebsd-net@freebsd.org Subject: Re: Traffic between cxgbe VFs and/or PF on a host Message-ID: <024F07FC-EF22-4259-9678-0F4D4AF2725E@jnielsen.net> In-Reply-To: <CAPFoGT8zWe0-6C7V5wBmHuBpUHceXVsg%2B3a1r6Yhh3LiNDZxMw@mail.gmail.com> References: <CAPFoGT8zWe0-6C7V5wBmHuBpUHceXVsg%2B3a1r6Yhh3LiNDZxMw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail-EE54B904-B54D-4006-91E9-414A7E93D260 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable <html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D= utf-8"></head><body dir=3D"auto"><div dir=3D"ltr"></div><div dir=3D"ltr"><bl= ockquote type=3D"cite">On Oct 11, 2024, at 6:21=E2=80=AFPM, Navdeep Parhar &= lt;np@freebsd.org> wrote:<br></blockquote></div><blockquote type=3D"cite"= ><div dir=3D"ltr">=EF=BB=BF<div dir=3D"ltr"><div dir=3D"ltr">On Fri, Oct 11,= 2024 at 3:56=E2=80=AFPM John Nielsen <<a href=3D"mailto:lists@jnielsen.n= et">lists@jnielsen.net</a>> wrote:</div><div class=3D"gmail_quote"><block= quote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1p= x solid rgb(204,204,204);padding-left:1ex"><br> I=E2=80=99m running a FreeBSD 14-STABLE host with a Chelstio T520. I have a b= hyve VM (also running 14-STABLE) to which I have assigned a VF of the NIC. T= hat is all working as expected; the host can pass traffic using the PF cxl0 a= nd the guest can pass traffic using the VF cxlv0. However the host cannot co= mmunicate with the guest. I am looking in to the possibility of enabling 802= .1qbg / VEPA / reflective relay on the switch port but I=E2=80=99d like to k= now if the T5 can do that switching itself without sending the packets over t= he wire. The marketing material says the card "integrates a high performance= packet switch=E2=80=9D but I don=E2=80=99t know how to configure that funct= ionality on FreeBSD or if this use case is supported. Can anyone shed some l= ight on that?<br></blockquote><div><br></div><div>The PF driver's tx bypasse= s the internal switch by default and is not visible to the VFs because of th= at. Set this knob to force it go through the switch.<br></div><div><br= ></div><div> hw.cxgbe.tx_vm_wr<br> &n= bsp; Setting this to 1 instructs the driver to use VM wo= rk requests to transmit data.<br> &= nbsp;This lets PF interfaces transmit frames to VF interfaces over the inter= nal switch in<br> the ASIC.&n= bsp; Note that the cxgbev(4) VF driver always uses VM work requests and is n= ot<br> affected by this tunab= le. The default value is 0 and should be changed only if PF<br> &= nbsp; and VF interfaces need to communicat= e with each other. Different interfaces can be<br> = assigned different values using the dev.<port= >.X.tx_vm_wr sysctl when the interface<br> &nb= sp; is administratively down.</div></div></div> </div></blockquote><br><div>Thank you! That indeed works as described. I did= scan the man page before posting but obviously could have benefited from cl= oser study. I appreciate you pointing me in the right direction.</div><div><= br></div><div>Out of curiosity is there any practical difference (performanc= e; security, etc) between setting this knob and just using an additional VF f= or the host rather than the PF (aside from being one less VF available for u= se elsewhere)?</div><div><br></div><div>JN</div></body></html>= --Apple-Mail-EE54B904-B54D-4006-91E9-414A7E93D260--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?024F07FC-EF22-4259-9678-0F4D4AF2725E>