Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 13 May 2021 11:50:14 -0400
From:      mike tancsa <mike@sentex.net>
To:        Alan Somers <asomers@freebsd.org>
Cc:        freebsd-fs <freebsd-fs@freebsd.org>
Subject:   Re: speeding up zfs send | recv
Message-ID:  <f390a350-eff2-8d7d-63dd-9e908d39f50f@sentex.net>
In-Reply-To: <CAOtMX2gifUmgqwSKpRGcfzCm_=BX_szNF1AF8WTMfAmbrJ5UWA@mail.gmail.com>
References:  <866d6937-a4e8-bec3-d61b-07df3065fca9@sentex.net> <CAOtMX2gifUmgqwSKpRGcfzCm_=BX_szNF1AF8WTMfAmbrJ5UWA@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
On 5/13/2021 11:37 AM, Alan Somers wrote:
> On Thu, May 13, 2021 at 8:45 AM mike tancsa <mike@sentex.net
> <mailto:mike@sentex.net>> wrote:
>
>     For offsite storage, I have been doing a zfs send across a 10G
>     link and
>     <trim>
>     Why would the mail spool send be so slow compared to the sends wher=
e
>     datasets only have a few large files ?
>
>
> Is this a high latency link?=C2=A0 ZFS send streams can be bursty.=C2=A0=
 Piping
> the stream through mbuffer helps with that.=C2=A0 Just google "zfs send=

> mbuffer" for some examples.=C2=A0 And be aware that your speed may be
> limited by the sender.=C2=A0 Especially if those small files are random=
ly
> spread across the platter, your sending server's disks may be the
> limiting factor.=C2=A0 Use gstat to check.
> -Alan

Thanks for the mbuffer suggestion, I will give it a try!=C2=A0 The fiber =
is
just over to the next building and connected via layer 2 switch so very
low latency.

5 packets transmitted, 5 packets received, 0.0% packet loss
round-trip min/avg/max/stddev =3D 0.042/0.057/0.087/0.015 ms

zfs is all "black box" to me, but I don't understand why the contents of
the dataset would make a difference ?=C2=A0 I am sending from my backup
server to my offsite backup server. i.e. the mail server sends its
incremental snaphots to the backup server. I then once a week focus on
the latest snapshot on the backup server and send it to my offsite
server.=C2=A0 Would not that zfs send just be sending blocks of data from=
 the
zfs dataset ? and wouldn't all contents have an equal chance of being
spread across the platters on the backup server ?

=C2=A0=C2=A0=C2=A0 ---Mike






Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?f390a350-eff2-8d7d-63dd-9e908d39f50f>