Date: Mon, 28 Aug 2023 10:03:02 +0000 From: Wei Hu <weh@microsoft.com> To: Mikhail Zakharov <zmey20000@yahoo.com>, "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org> Subject: RE: Very slow scp performance comparing to Linux Message-ID: <SI2P153MB04412D004F9930203B04A032BBE0A@SI2P153MB0441.APCP153.PROD.OUTLOOK.COM> In-Reply-To: <6952b7fd-f90f-2677-1997-d0c708cb559e@yahoo.com> References: <SI2P153MB0441858EC3FE24D5B7A6B34DBBE0A@SI2P153MB0441.APCP153.PROD.OUTLOOK.COM> <6952b7fd-f90f-2677-1997-d0c708cb559e@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Mike, All VMs are with same type and they are in Azure. If the copy on Linux is b= eing cached on the RX side, so is FreeBSD? Thanks, Wei > -----Original Message----- > From: owner-freebsd-hackers@freebsd.org <owner-freebsd- > hackers@freebsd.org> On Behalf Of Mikhail Zakharov > Sent: Monday, August 28, 2023 4:28 PM > To: freebsd-hackers@freebsd.org > Subject: Re: Very slow scp performance comparing to Linux >=20 > [You don't often get email from zmey20000@yahoo.com. Learn why this is > important at https://aka.ms/LearnAboutSenderIdentification ] >=20 > Hi, >=20 > What is the drive back-end of your machines? I mean, what is seem to be f= ast > copied could still be cached in filesystem/storage buffers but not actual= ly > written on the disk. >=20 > Best, > Mike >=20 >=20 > On 8/28/2023 9:32 AM, Wei Hu wrote: > > Hi, > > > > When I was testing a new NIC, I found the single stream scp performance > was almost 8 time slower than Linux on the RX side. Initially I thought i= t might > be something with the NIC. But when I switched to sending the file on > localhost, the numbers stay the same. > > > > Here I was sending a 2GB file from sender to receiver using scp. FreeB= SD is a > recent NON-DEBUG build from CURRENT. The Ubuntu Linux kernel is 6.2.0. > Both run in HyperV VMs on the same type of hardware. The FreeBSD VM has > 16 vcpus, while Ubuntu VM has 4 vcpu. > > > > Sender Receiver throughput > > Linux FreeBSD 70 MB/s > > Linux Linux 550 MB/s > > FreeBSD FreeBSD 70 MB/s > > FreeBSD Linux 350 MB/s > > FreeBSD localhost 70 MB/s > > Linux localhost 550 MB/s > > > > From theses test, it seems I can rule out the issue on NIC and its dri= ver. > Looks the FreeBSD kernel network stack is much slower than Linux on singl= e > stream TCP, or there are some problem with scp? > > > > I also tried turning on following kernel parameters on FreeBSD kernel. = But it > makes no difference, neither do the other tcp cc algorithms such as htcp = and > newreno. > > > > net.inet.tcp.soreceive_stream=3D"1" > > net.isr.maxthreads=3D"-1" > > net.isr.bindthreads=3D"1" > > > > net.inet.ip.intr_queue_maxlen=3D2048 > > net.inet.tcp.recvbuf_max=3D16777216 > > net.inet.tcp.recvspace=3D419430 > > net.inet.tcp.sendbuf_max=3D16777216 > > net.inet.tcp.sendspace=3D209715 > > kern.ipc.maxsockbuf=3D16777216 > > > > Any ideas? > > > > Thanks, > > Wei > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?SI2P153MB04412D004F9930203B04A032BBE0A>