From nobody Tue Aug 29 08:28:57 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RZgZ42SZsz4rX9L for ; Tue, 29 Aug 2023 08:29:00 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-oo1-xc30.google.com (mail-oo1-xc30.google.com [IPv6:2607:f8b0:4864:20::c30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RZgZ35lZtz4Shm for ; Tue, 29 Aug 2023 08:28:59 +0000 (UTC) (envelope-from mjguzik@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-oo1-xc30.google.com with SMTP id 006d021491bc7-5738949f62cso1094723eaf.0 for ; Tue, 29 Aug 2023 01:28:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693297738; x=1693902538; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=QOmSrM029mBkkII1KnqOlbK+0gZWcp05A0JMXroBRaE=; b=iNQCCb3MqStptGPVOTqOk6nu+wN3RxBbLT2Sd8rneWYVkm79S1piQ/uTFI5WfBAwoA Nh6Z4sdcmCdcha35wdMs5latp/ZSF8nia2dy6CyJ2OuPp2IL2hbvoDcQmHV/C5a4m2Nk it2Q7O2O+EcHqFBX51Uw7AUJo7SyNV34HGQtFuaf4arclFVWjMvLjpJXDxWujXPklE2u /P4vL4tKZTSt/lb4vf5lZC7B+l5MUBXJYwapZZceHSihCVOqfvdQ9FLKiXO2bBgir9S3 9XVEFCRLm9acA/J47AnH8B4lUtE9GOTao2bR+YY0XJMO123n2b09rjDrk50MGSLoqXRk xzfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693297738; x=1693902538; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=QOmSrM029mBkkII1KnqOlbK+0gZWcp05A0JMXroBRaE=; b=hG3D2HUJxnkTXuH5xYXjHa9ZnkPuAfnUe89cifq5S0voJb1fhsrv4gB1f9PVrsQaVP OJNHl9Ws9sAtbqHaS9jmqclcAcyGnoLXitYH6Y1jetMQmwVB4BZurxfy+spvz11QXLP9 5c6mCgy5gcr4IX8oxcFR3jdYbKxG7V3eiJz997mmX++7dULh/Oo5vnqK6/C2OkEPEYOJ EH27+TVEHeoo0o4G9Vo8eNWrwqkZ++aRk3FAaqPFf+lIze2LpeqcjxbJhgNO4k6/AgMB v7/Qt6KReo9ZHztFti/Jguac9OGTfGHiIX4SmhZCeENU/jfe4sVhmOCY3hdLfDdiC6PU k28g== X-Gm-Message-State: AOJu0YzZ5faNvCjuZqPdyC+pVpMSFzCFBi2Zjom0mUwIKYbQd+X3sklo nYKKOceqA4yaH/dDX9uQt+i3Ei6TzP9GN352exk= X-Google-Smtp-Source: AGHT+IGMo+327KGp+WFGsOVqizeQHf4RoDPuBO+S6Z8yafjPH081lTyuyPhSOzmU3LOrdzJUSqTyEE/XFrWCTsxth3E= X-Received: by 2002:a4a:3014:0:b0:573:55f4:5175 with SMTP id q20-20020a4a3014000000b0057355f45175mr7932883oof.0.1693297738125; Tue, 29 Aug 2023 01:28:58 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Received: by 2002:a8a:1141:0:b0:4f0:1250:dd51 with HTTP; Tue, 29 Aug 2023 01:28:57 -0700 (PDT) In-Reply-To: References: <6952b7fd-f90f-2677-1997-d0c708cb559e@yahoo.com> <455dc69a-dd15-18d9-8e93-91bbff3997e4@madpilot.net> From: Mateusz Guzik Date: Tue, 29 Aug 2023 10:28:57 +0200 Message-ID: Subject: Re: Very slow scp performance comparing to Linux To: Wei Hu Cc: Guido Falsi , Mikhail Zakharov , "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4RZgZ35lZtz4Shm On 8/29/23, Wei Hu wrote: > > >> -----Original Message----- >> From: Guido Falsi >> Sent: Monday, August 28, 2023 6:49 PM >> To: Wei Hu ; Mikhail Zakharov >> ; freebsd-hackers@freebsd.org >> Subject: Re: Very slow scp performance comparing to Linux >> >> On 28/08/23 12:03, Wei Hu wrote: >> > Hi Mike, >> > >> > All VMs are with same type and they are in Azure. If the copy on Linux >> > is >> being cached on the RX side, so is FreeBSD? >> > >> >> You could perform the test using files in memory filesystems (tmpfs or >> the >> like). This would factor out disk performance whatever the backend. >> > Thanks, Guido. I tried tmpfs on the RX side with both FreeBSD and Linux VMs. > > The throughput on FreeBSD went up significantly from 50 MB/s to 630 MB/s > with > NIC interface. Linux went up modestly form 550 MB/s to 660 MB/s. > > So, looks in the non-tmpfs case, the Linux ext4 filesystem does cache large > amounts > of data in memory, much larger than FreeBSD ufs. > > Many thanks, Mike for bringing this up and Guido, for the suggestion to try > tmpfs. > There is probably something funky going on here. Here is a hack which will collect basic profiling info, works in a vm: dtrace -w -n 'profile:::profile-4999 { @[sym(arg0)] = count(); } tick-10s { system("clear"); trunc(@, 40); printa("%40a %@16d\n", @); clear(@); }' so start scp, start dtrace and see what happens -- Mateusz Guzik