From nobody Wed May 24 03:17:29 2023 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QQxFY3BqCz4TPpD for ; Wed, 24 May 2023 03:17:37 +0000 (UTC) (envelope-from zlei@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QQxFY2Vtvz3MfM; Wed, 24 May 2023 03:17:37 +0000 (UTC) (envelope-from zlei@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1684898257; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VR9euGQrH+u1SMNe+Gglh1IhHyqp9905anN2ggncYOM=; b=WAIYIzfW69kus++mUT4lcXG5eKfSINPtv6rlF3b7cCGirhLrZOBVvF2jsbYOkopMrLbqs8 8Z780/uMlfpFAhUk1ew47Ya/30blLVPjGX2y2uz8auGoDmCPMW9WEtjgwrPbW4uuD4VK2C WVvh0eV0m9QXJMdv9Ivq20xbjXM1utseBMPly8SCZyp3Xf5Hj7xNz31tZAWOIL9BwqTcp2 JHvXKeMNXB3M0DQC+eD3j13FmqmPoiVHNevrJnl0ZsEbizelBZzH1cYYcCawxQK7oFm6J9 FDFQW5t+f4xjoEdT3z8Ocl1W4/Ptnx4iZD5hxbkNiZ3jpr4Yt3A6DzBD3SgKpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1684898257; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VR9euGQrH+u1SMNe+Gglh1IhHyqp9905anN2ggncYOM=; b=WSmM+wnOp6mvsjesFKdFt10n50lbSN6wJqAUNmUWYFV2PR5OjmzEXdjd4pTYYMEOnDYAgY V7myNBdmqRQiILY06sNNHOO1ZZuZYLTzRCohu8QX9JOMtTll0c/5ilJshMPr8Mh5lzI6gG OK4JPinSsHik32E1Zxt/tnkf0k39Kgxpd0Sd4x4HEckCk9wN7zMoD1arV37COAVmv4AFQE d5Yqn1Pb86m1XoT7P0Y2+ZSx83/kq8IoksxXBYYjwIr1jO9gULQAQvQzB/DS40sBSW8xee 4JNY47qPmKAeE7kcD8XXvNK7h8WldIIMWASySxxqjLYGrgtvi97icl/v5qmNRw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1684898257; a=rsa-sha256; cv=none; b=eDV/HzoY7D31Vhf4t4b5S4w9quPBP/yUwkv3PPnEBemkEWAhuzqBobi+vaf5pMWiEXjqxX x265sQVZp6SgS4G+0H+LP81nJHfGhAM9UaQfkcvbH8jRHZQlkKJxZ1bptVolzEyitNrwdl hCcd6ToyBw0EjHr7Q45xui5hbJdFLQ4mgKznVyQgaPqMe3z4EKfskSgCrfkkSTz+/kfaiN ITwE71IC8OALXDflCgk9zQ5zAVu+S0AEM73WkNgGQThw1HtcM0k1zMio6cLAYG0HRIoRVt 1ugxdMECaa0SH3fSH+FzfYEjRezX44wOlox9nMj83RRXJVjXDBmVVFlO0uxsug== Received: from smtpclient.apple (unknown [IPv6:2001:19f0:6001:9db:98f0:9fe0:3545:10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: zlei/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4QQxFW70KWznf3; Wed, 24 May 2023 03:17:35 +0000 (UTC) (envelope-from zlei@FreeBSD.org) Content-Type: text/plain; charset=us-ascii List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\)) Subject: Re: how to increase the vnet speed? From: Zhenlei Huang In-Reply-To: <20230523231533.32c1d022@x25> Date: Wed, 24 May 2023 11:17:29 +0800 Cc: Benoit Chesneau , "freebsd-net@FreeBSD.org" Content-Transfer-Encoding: 7bit Message-Id: <46D05994-43DB-4BEC-A3E3-C8B15C220155@FreeBSD.org> References: <20230523231533.32c1d022@x25> To: Marko Zec X-Mailer: Apple Mail (2.3696.120.41.1.2) X-ThisMailContainsUnwantedMimeParts: N > On May 24, 2023, at 5:15 AM, Marko Zec wrote: > > On Tue, 23 May 2023 19:58:07 +0000 > Benoit Chesneau wrote: > >> Hi all, >> >> I've created a jail using bastille and setup network. The mainin >> terface is a 25Gbps nic and between hosts I get 24.6 Gbits/sec : > > [...] > >> But between one host and the jail I only get 3.96 Gbits/sec > > [...] > >> Is there a way to increase the performance of the of the jail? The >> nice is a mellannox ConnectX-4 Lx, mce(4) . > > Modern NICs offload a lot of the protocol stack processing (checksum, > segmentation, and / or reassembly) from the CPU to dedicated silicon, > whereas inter-vnet traffic needs to be handled completely in software, > that's where the difference comes from. > > Perhaps we could gain some speed by abusing mbuf flags to skip RXCSUM > for epair traffic, maybe even skip and fake TXCSUM... For epair I think it is doable. Other hardware offload features such as VLAN_HWTAGGING can be also taken into account. > > Marko > Best regards, Zhenlei