From nobody Fri Aug 12 00:48:04 2022 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4M3lRG1GYzz4Z71C for ; Fri, 12 Aug 2022 00:48:42 +0000 (UTC) (envelope-from ricera10@gmail.com) Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4M3lRF3xd2z3y2f for ; Fri, 12 Aug 2022 00:48:41 +0000 (UTC) (envelope-from ricera10@gmail.com) Received: by mail-pl1-x636.google.com with SMTP id p18so18323645plr.8 for ; Thu, 11 Aug 2022 17:48:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc; bh=Sl/mmrLCdfq4jNsTuOykIyKUIRaFej41puTe4t+FTDA=; b=ifUVJRo8Rghe0TAEcm3B7DsZN4bmS+biyC85P1xSwEYrRGBPtjvJCzPbIIlbW6tOtG 8nruM+US7b4BGydVAPLNHN8ZTzGb0ljyjRRdaoWPWqMmD84O3kelFXOhPP/13XMqY0Ci xeIPOQt44q3NqVjV0mZPfrz7rtyUf0MlcgNEkuNyiRox9N5Ck5NYpfaILK2X1YU31wuv JXhPUkbWSfKAebUIlZLF2RfZ8m0PIlYTnn2yTjiw1VGHKYAzJaUusZNC96Y1UxG6vVXB iAQqFfG7gpRHxbUnMhX6ahBqkPooXgg2ybFNaGcVNDg4ymnwzQi84JPG4xy4wVxqPXkc OGkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc; bh=Sl/mmrLCdfq4jNsTuOykIyKUIRaFej41puTe4t+FTDA=; b=z06Xss7W0R8DaRGg6ev5iRGtaFyBUyLqOAZ7/48eEelzxzsM+qnVuU/xoQP/elWmqQ K/QaaMf6p8DgMvqJ2tiGrvLgGjFsrezvIU/zT2oqhJv/ym9U3oqVF3sEAmL0rbx5L+X7 RBota0Sph9Kz9VFpeR0bmXT+/vPCRKKg92/+nTKnAfrY9kJ+KT71gzsGJfT5ScLTv2r8 qp4FXNnfnYqNxJDHofsBN+FOhvVcw8rgDqcDvflbzktYNyYGibu4iXURbvYjy6LdGC82 e3D8g/Qy5p/e7US407fUKg/jKLI7jqhEF6kh7qlzJgGWT1xOagsLnkhmjQrg2930PVcX mDUg== X-Gm-Message-State: ACgBeo3bpAJKWNewSwVIQFZL9hCfU2uOwfeqqOiD++pBPVwGjOjyKh7/ GuTcsqitHeMt1wkRi0V0WtILTPoYKruVpTdh0R7RB2q8QK4= X-Google-Smtp-Source: AA6agR4X59ZgAuOwrUxADZWCqtqGisHt7yWRBtXcuS8QqrBMn6HZheU0A9dIEBqQKR1LfqgDMRYiRQwgW9Earnk6X60= X-Received: by 2002:a17:902:ce83:b0:16d:d667:d4df with SMTP id f3-20020a170902ce8300b0016dd667d4dfmr1639434plg.159.1660265320321; Thu, 11 Aug 2022 17:48:40 -0700 (PDT) List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 References: <2f362689-0feb-bd41-93b2-afb46b4a4a08@codenetworks.net> In-Reply-To: <2f362689-0feb-bd41-93b2-afb46b4a4a08@codenetworks.net> From: Eric Joyner Date: Thu, 11 Aug 2022 17:48:04 -0700 Message-ID: Subject: Re: 25/100 G performance on freebsd To: Santiago Martinez Cc: Benoit Chesneau , "freebsd-net@FreeBSD.org" , Michael Dexter Content-Type: multipart/alternative; boundary="000000000000a69a6305e600a412" X-Rspamd-Queue-Id: 4M3lRF3xd2z3y2f X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=ifUVJRo8; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of ricera10@gmail.com designates 2607:f8b0:4864:20::636 as permitted sender) smtp.mailfrom=ricera10@gmail.com X-Spamd-Result: default: False [-3.95 / 15.00]; NEURAL_HAM_SHORT(-0.99)[-0.994]; NEURAL_HAM_LONG(-0.99)[-0.994]; NEURAL_HAM_MEDIUM(-0.96)[-0.964]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-net@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::636:from]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-net@freebsd.org]; MID_RHS_MATCH_FROMTLD(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; DKIM_TRACE(0.00)[gmail.com:+]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_COUNT_TWO(0.00)[2] X-ThisMailContainsUnwantedMimeParts: N --000000000000a69a6305e600a412 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I think Benoit may be referring to this bug? https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D265207 >From that, it looks like his cards are QLogic ones that use the qlnxe driver. As for who would know what to do about that ioctl issue, I don't know. I'm not sure if the driver has an active maintainer. - Eric On Wed, Aug 10, 2022 at 2:32 PM Santiago Martinez wrote: > Hi Benoit, sorry to hear that the SR-IOV still not working on your HW. > > Have you tested the last patch from Intel? > > Regarding Bhyve, you can use Vale switches (based on netmap). > On my machines, i get around ~33Gbps between VM (same local machine), > sometimes going towards 40Gbps... ( These are basic tests with iperf3 and > TSO/LRO enabled). > > @Michael Dexter is working on a document that contains configuration > examples and test results for the different network backend available in > bhyve. > > If you need help, let me know and we can set up a call. > Take care. > Santi > > On 8/8/22 08:57, Benoit Chesneau wrote: > > For some reasons. I can=E2=80=99t use SR-IOV on my freebsd machines (HPE= DL160 > gen10) with latest 25G HPE branded cards. I opened tickets for that but > since then no move happened. > > So I wonder id there is a good setup to use these cards with the > virtualization. Which kind of performance should I expect using if_bridge= ? > What if i am doing L3 routing instead using epair or tap (for bhyve). Wou= ld > it work better? > > Any hint is welcome, > > Beno=C3=AEt > > --000000000000a69a6305e600a412 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I think Benoit may be referring to this bug?=C2=A0https://bug= s.freebsd.org/bugzilla/show_bug.cgi?id=3D265207

From= that, it looks like his cards are QLogic ones that use the qlnxe driver. A= s for who would know what to do about that ioctl issue, I don't know. I= 'm not sure if the driver has an active maintainer.

- Eric

On Wed, Aug 10, 2022 at 2:32 PM Santiago Martinez <sm@codenetworks.net> wrote:
<= /div>
=20 =20 =20
Hi Benoit, sorry to hear that the SR-IOV still not working on your HW.

Have you tested the last patch from Intel?

Regarding Bhyve, you can use Vale switches (based on netmap).
On my machines, i get around ~33Gbps between VM (same local machine), sometimes going towards 40Gbps... ( These are basic tests with iperf3 and TSO/LRO enabled).

@Michael Dexter is working on a document that contains configuration examples and test results for the different network backend available in bhyve.

If you need help, let me know and we can set up a call.
Take care.
Santi

On 8/8/22 08:57, Benoit Chesneau wrote:
=20
For some reasons.=C2=A0I can=E2=80=99t use SR-IOV = on my freebsd machines =C2=A0(HPE DL160 gen10)=C2=A0with latest 25G HPE b= randed cards. I opened tickets for that but since then no move happened.

So I wonder id there is a good setup to use these cards with the virtualization. Which kind of performance should I expect using if_bridge? What if i am doing L3 routing instead using epair or tap (for bhyve). Would it work better?

Any hint is welcome,

Beno=C3=AEt

--000000000000a69a6305e600a412--