Date: Thu, 23 Feb 2023 08:17:01 +1000 From: George Michaelson <ggm@algebras.org> To: freebsd-fs <freebsd-fs@freebsd.org> Subject: Re: speeding up zfs send | recv (update) Message-ID: <CAKr6gn0URKzqcx8Y7aYSmTQLS0%2BjrNdsc07Ao14oNCjScG6kcQ@mail.gmail.com> In-Reply-To: <CAOjFWZ7k7ANwcGyNCoYMg%2BLUBzAz2VyNfxQo5rKcrYj8XFgG3Q@mail.gmail.com> References: <866d6937-a4e8-bec3-d61b-07df3065fca9@sentex.net> <CAOtMX2gifUmgqwSKpRGcfzCm_=BX_szNF1AF8WTMfAmbrJ5UWA@mail.gmail.com> <f6ea3387-faf8-4c63-d1e7-906fa397b00b@sentex.net> <a38578c6-b633-249d-90f0-0652377d76c0@quip.cz> <c229a502-fb76-ec6a-a56b-934d3b56e474@sentex.net> <1031e2b0-b245-1dc6-a499-8f4da3796543@quip.cz> <46455168-d7f1-6ca9-ad2f-9bcd3359e0f3@sentex.net> <78c78aec-a34b-f188-ef96-8ced9a1eda35@quip.cz> <CAOjFWZ7k7ANwcGyNCoYMg%2BLUBzAz2VyNfxQo5rKcrYj8XFgG3Q@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
I use mbuffer for this purpose. I'm only on protected "inside" links even if cross-router (ie not just a switching fabric) which may make my use-case a poor fit for your need. mbuffer has several strategies around how much buffer to use, what rate limits to apply, I believe can do tweaks to use scatter-gather models (the necessary ioctls to turn things on &c but these are almost certainly tuned to linux) you also need to look at your ethernet card offload. Normally beneficial. it can interact badly with in-kernel models of end-to-end flow, because it's performing its own work "on your behalf" I sort of miss having a "null" cipher in SSH. I didn't entirely understand why it got stripped out: the 'who are you" initialisation authorisation is beneficial, even if the datastream is (deliberately) unprotected. the RC2 fallback didn't seem to impose much cost, but thats gone too now. -G On Thu, Feb 23, 2023 at 7:43 AM Freddie Cash <fjwcash@gmail.com> wrote: > > [Sorry for top part, GMail sucks for replies.] > > If this is a LAN or private WAN where you trust the network, piping the send stream through netcat will remove ssh from the equation. > > That's what we switched to using once it became almost impossible to get the "none" cipher working with ssh on FreeBSD. > > We use ssh to connect to the remote server and enable a netcat listener on port X, then pipe the send through netcat to the remote system on port X. That way it's logged and uses ssh for authentication. > > We easily saturate gigabit links between our ZFS systems using netcat. > > > > Cheers, > Freddie > > Typos due to smartphone keyboard. > > On Wed., Feb. 22, 2023, 1:31 p.m. Miroslav Lachman, <000.fbsd@quip.cz> wrote: >> >> On 22/02/2023 22:08, mike tancsa wrote: >> > On 2/22/2023 4:03 PM, Miroslav Lachman wrote: >> >> Interresting numbers. I think I am the only one who get best speed >> >> with chacha20-poly1305@openssh.com >> >> >> >> >> >> It seems the speed of SSH is limited by single core performance which >> >> is very poor on this machine (Intel(R) Pentium(R) Dual CPU E2160). >> >> Even if CPU has 50% idle, ssh runs on 99.8% of single core. >> > >> > The CPU I have has >> > aesni0: <AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS> on motherboard >> > >> > which probably helps. >> >> That explains it >> aesni0: No AES or SHA support. >> >> >> I know there were some HPN patches to ssh, beside that is there any >> >> option I can try to use less CPU? >> >> >> >> I will play with cpuset to pin ssh on one core and everything else on >> >> the other core. >> > >> > It looks like you are running into a CPU bottleneck TBH >> >> Yes. Pinning on cores with cpuset helps a bit (about +3MiB/s) but >> without some tweaks on ssh I will not gain more speed :( >> >> Thank you for your help! >> >> Miroslav Lachman >> >>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAKr6gn0URKzqcx8Y7aYSmTQLS0%2BjrNdsc07Ao14oNCjScG6kcQ>