Date: Thu, 29 Jun 2023 14:52:44 +0200 From: tuexen@freebsd.org To: Murali Krishnamurthy <muralik1@vmware.com> Cc: "freebsd-transport@FreeBSD.org" <freebsd-transport@FreeBSD.org> Subject: Re: FreeBSD TCP (with iperf3) comparison with Linux Message-ID: <19F174D3-F0DB-4CD0-B43C-7A3BD8F47AFF@freebsd.org> In-Reply-To: <PH0PR05MB10064288C78F8F3A0116D023DFB469@PH0PR05MB10064.namprd05.prod.outlook.com> References: <PH0PR05MB10064288C78F8F3A0116D023DFB469@PH0PR05MB10064.namprd05.prod.outlook.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> On 29. Jun 2023, at 11:51, Murali Krishnamurthy <muralik1@vmware.com> = wrote: >=20 > Hello FreeBSD Transport experts, > We are evaluating performance of FreeBSD 13 VM on ESX hypervisor in = long RTT setup and happened to compare the performance with Linux VM = with same hypervisor. > We see a substantially better performance with Linux getting close to = the BDP limit, whereas BSD 13 not filling up the pipe enough. > We are trying to figure out what could lead to such a huge difference = and feel we could be missing something here. > Could you please help us to know if there is a way to make it perform = better? > Setup details: > We have 2 ESX hypervisors where 2 VMs (one FreeBSD 13 and one Ubuntu = 23.04/Linux kernel 6.2) were launched on each hypervisor. > Then we ran iperf between, > =E2=80=A2 BSD 13 <-> BSD 13 > =E2=80=A2 Ubuntu <-> Ubuntu > Even though the network environment were same in both cases, we see = Ubuntu performing much better. > Below are connection parameters: > Socket buffer: 16MB > TCP CC Algo: Cubic. We used this as this is suitable for Long Fat = Networks. > Ping RTT: 100 ms between the two end points. > We kept all other parameters to default on both Linux and BSD. > BDP for 16MB Socket buffer: 16 MB * (1000 ms * 100ms latency) * 8 = bits/ 1024 =3D 1.25 Gbps > Ubuntu consistently hits around 1 Gbps Bitrate almost reaching the BDP = limit. > FreeBSD 13 shows a Bit rate between the range of 300-600 Mbps only. So = it seems to be doing half as good as Linux. > For lower socket buffer of 4MB, both FreeBSD and Linux seem to do same = and able to meet BDP of 300 Mbps consistently. > Larger socket buffer seems to have an issue. > Please let us know if there are ways to fine tune the system = parameters to make BSD perform better. > Or any other suggestions/queries welcome. > Regards > Murali Hi Murali, I'll bring this up on the biweekly transport call. Best regards Michael
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?19F174D3-F0DB-4CD0-B43C-7A3BD8F47AFF>