Date: Thu, 13 May 2021 11:50:14 -0400 From: mike tancsa <mike@sentex.net> To: Alan Somers <asomers@freebsd.org> Cc: freebsd-fs <freebsd-fs@freebsd.org> Subject: Re: speeding up zfs send | recv Message-ID: <f390a350-eff2-8d7d-63dd-9e908d39f50f@sentex.net> In-Reply-To: <CAOtMX2gifUmgqwSKpRGcfzCm_=BX_szNF1AF8WTMfAmbrJ5UWA@mail.gmail.com> References: <866d6937-a4e8-bec3-d61b-07df3065fca9@sentex.net> <CAOtMX2gifUmgqwSKpRGcfzCm_=BX_szNF1AF8WTMfAmbrJ5UWA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 5/13/2021 11:37 AM, Alan Somers wrote: > On Thu, May 13, 2021 at 8:45 AM mike tancsa <mike@sentex.net > <mailto:mike@sentex.net>> wrote: > > For offsite storage, I have been doing a zfs send across a 10G > link and > <trim> > Why would the mail spool send be so slow compared to the sends wher= e > datasets only have a few large files ? > > > Is this a high latency link?=C2=A0 ZFS send streams can be bursty.=C2=A0= Piping > the stream through mbuffer helps with that.=C2=A0 Just google "zfs send= > mbuffer" for some examples.=C2=A0 And be aware that your speed may be > limited by the sender.=C2=A0 Especially if those small files are random= ly > spread across the platter, your sending server's disks may be the > limiting factor.=C2=A0 Use gstat to check. > -Alan Thanks for the mbuffer suggestion, I will give it a try!=C2=A0 The fiber = is just over to the next building and connected via layer 2 switch so very low latency. 5 packets transmitted, 5 packets received, 0.0% packet loss round-trip min/avg/max/stddev =3D 0.042/0.057/0.087/0.015 ms zfs is all "black box" to me, but I don't understand why the contents of the dataset would make a difference ?=C2=A0 I am sending from my backup server to my offsite backup server. i.e. the mail server sends its incremental snaphots to the backup server. I then once a week focus on the latest snapshot on the backup server and send it to my offsite server.=C2=A0 Would not that zfs send just be sending blocks of data from= the zfs dataset ? and wouldn't all contents have an equal chance of being spread across the platters on the backup server ? =C2=A0=C2=A0=C2=A0 ---Mike
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?f390a350-eff2-8d7d-63dd-9e908d39f50f>