From nobody Wed Aug 10 21:31:49 2022 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4M336m5kZvz4YbHN for ; Wed, 10 Aug 2022 21:32:00 +0000 (UTC) (envelope-from SRS0=nFie6+=YO=codenetworks.net=sm@eigbox.net) Received: from bosmailout08.eigbox.net (bosmailout08.eigbox.net [66.96.185.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4M336m0kpGz45G3 for ; Wed, 10 Aug 2022 21:32:00 +0000 (UTC) (envelope-from SRS0=nFie6+=YO=codenetworks.net=sm@eigbox.net) Received: from bosmailscan09.eigbox.net ([10.20.15.9]) by bosmailout08.eigbox.net with esmtp (Exim) id 1oLtJ0-0006WA-Sb for freebsd-net@freebsd.org; Wed, 10 Aug 2022 17:31:58 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=codenetworks.net; s=dkim; h=Sender:In-Reply-To:Subject:From:References:To: MIME-Version:Date:Message-ID:Content-Type:Reply-To:Cc: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=L50NN/MGgVEPbZHbjedIQCY4uuVR4qq0KfpbH78rG3U=; b=QEufEYCW+sIESBKWLOO9aHboT dSwpFENrDPn1wWKYTtAvOVoXDEL91pctBH+X9tI/UMxNVYi8yl1PThGUayxSQJaarHQN1iH2Lq5bG 6/VJGvx264YT/zuUoaUU+S68qu3T44GMmE3utjaMaWD6MfzAh5vVyiNhFuA6/rmypdYmsVKfLCH6G cj98kAg3lmZjN70mIzcVV5Z5P1NeTUA6TiIpVydVxQbtIzDqQt9U8Y9zSlnp29+BImNeBvhpHsNcp TJ1vfTWQtwqKDVo/c9kHH+dDO1BngN87qsJPn+dE1AM6k1Vjj+kgJaLJJfh/xuTlBVBRf/LFgwY1F 4n9kfwoxw==; Received: from [10.115.3.33] (helo=bosimpout13) by bosmailscan09.eigbox.net with esmtp (Exim) id 1oLtIz-0005nV-DM for freebsd-net@freebsd.org; Wed, 10 Aug 2022 17:31:57 -0400 Received: from bosauthsmtp06.yourhostingaccount.com ([10.20.18.6]) by bosimpout13 with id 5xXt2800B07rX7u01xXwg4; Wed, 10 Aug 2022 17:31:57 -0400 X-Authority-Analysis: v=2.3 cv=H7JAP9Qi c=1 sm=1 tr=0 a=fIc3/5IyPUehxkj7BpkQ7Q==:117 a=Ek/qOh1uPkKSHvd30yk7rg==:17 a=biHskzXt2R4A:10 a=-Yl_685HdVUA:10 a=r77TgQKjGQsHNAKrUKIA:9 a=sP1lWflLZG9e9DAKRUwA:9 a=QEXdDO2ut3YA:10 a=Y2PtTiLEAAAA:8 a=ksiMXDkchsBHMlfuNaYA:9 a=sYiuuOGV3x0dP1oD:21 a=_W_S_7VecoQA:10 a=7OkdhSbzDfCo1JLT1PZq:22 Received: from cm-81-9-194-73.telecable.es ([81.9.194.73]:26973 helo=[192.168.3.100]) by bosauthsmtp06.eigbox.net with esmtpa (Exim) id 1oLtIv-0004ni-68; Wed, 10 Aug 2022 17:31:53 -0400 Content-Type: multipart/alternative; boundary="------------IvnK3XSODuPD2kDJF5WJSL5G" Message-ID: <2f362689-0feb-bd41-93b2-afb46b4a4a08@codenetworks.net> Date: Wed, 10 Aug 2022 23:31:49 +0200 List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:91.0) Gecko/20100101 Thunderbird/91.12.0 Content-Language: en-US To: Benoit Chesneau , "freebsd-net@FreeBSD.org" , Michael Dexter References: From: Santiago Martinez Subject: Re: 25/100 G performance on freebsd In-Reply-To: X-EN-UserInfo: d3bdfab0736480cedf04ed92aaea2ef5:931c98230c6409dcc37fa7e93b490c27 X-EN-AuthUser: sm@codenetworks.net X-EN-OrigIP: 81.9.194.73 X-EN-OrigHost: cm-81-9-194-73.telecable.es X-Rspamd-Queue-Id: 4M336m0kpGz45G3 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none ("invalid DKIM record") header.d=codenetworks.net header.s=dkim header.b=QEufEYCW; dmarc=none; spf=pass (mx1.freebsd.org: domain of "SRS0=nFie6+=YO=codenetworks.net=sm@eigbox.net" designates 66.96.185.8 as permitted sender) smtp.mailfrom="SRS0=nFie6+=YO=codenetworks.net=sm@eigbox.net" X-Spamd-Result: default: False [-3.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; NEURAL_HAM_MEDIUM(-1.00)[-0.998]; FORGED_SENDER(0.30)[sm@codenetworks.net,SRS0=nFie6@eigbox.net]; R_SPF_ALLOW(-0.20)[+ip4:66.96.128.0/18]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCPT_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DMARC_NA(0.00)[codenetworks.net: no valid DMARC record]; R_DKIM_PERMFAIL(0.00)[codenetworks.net:s=dkim]; RCVD_IN_DNSWL_NONE(0.00)[66.96.185.8:from]; RCVD_COUNT_FIVE(0.00)[5]; FROM_NEQ_ENVFROM(0.00)[sm@codenetworks.net,SRS0=nFie6@eigbox.net]; MLMMJ_DEST(0.00)[freebsd-net@freebsd.org]; DKIM_TRACE(0.00)[codenetworks.net:~]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:29873, ipnet:66.96.128.0/18, country:US]; RCVD_TLS_LAST(0.00)[]; TAGGED_FROM(0.00)[=YO=codenetworks.net=sm]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[66.96.185.8:from] X-ThisMailContainsUnwantedMimeParts: N This is a multi-part message in MIME format. --------------IvnK3XSODuPD2kDJF5WJSL5G Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Hi Benoit, sorry to hear that the SR-IOV still not working on your HW. Have you tested the last patch from Intel? Regarding Bhyve, you can use Vale switches (based on netmap). On my machines, i get around ~33Gbps between VM (same local machine), sometimes going towards 40Gbps... ( These are basic tests with iperf3 and TSO/LRO enabled). @Michael Dexter is working on a document that contains configuration examples and test results for the different network backend available in bhyve. If you need help, let me know and we can set up a call. Take care. Santi On 8/8/22 08:57, Benoit Chesneau wrote: > For some reasons. I can’t use SR-IOV on my freebsd machines  (HPE > DL160 gen10) with latest 25G HPE branded cards. I opened tickets for > that but since then no move happened. > > So I wonder id there is a good setup to use these cards with the > virtualization. Which kind of performance should I expect using > if_bridge? What if i am doing L3 routing instead using epair or tap > (for bhyve). Would it work better? > > Any hint is welcome, > > Benoît > --------------IvnK3XSODuPD2kDJF5WJSL5G Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit Hi Benoit, sorry to hear that the SR-IOV still not working on your HW.

Have you tested the last patch from Intel?

Regarding Bhyve, you can use Vale switches (based on netmap).
On my machines, i get around ~33Gbps between VM (same local machine), sometimes going towards 40Gbps... ( These are basic tests with iperf3 and TSO/LRO enabled).

@Michael Dexter is working on a document that contains configuration examples and test results for the different network backend available in bhyve.

If you need help, let me know and we can set up a call.
Take care.
Santi

On 8/8/22 08:57, Benoit Chesneau wrote:
For some reasons. I can’t use SR-IOV on my freebsd machines  (HPE DL160 gen10) with latest 25G HPE branded cards. I opened tickets for that but since then no move happened.

So I wonder id there is a good setup to use these cards with the virtualization. Which kind of performance should I expect using if_bridge? What if i am doing L3 routing instead using epair or tap (for bhyve). Would it work better?

Any hint is welcome,

Benoît

--------------IvnK3XSODuPD2kDJF5WJSL5G--