Date: Mon, 22 Jul 2002 08:16:14 +1000 From: Peter Jeremy <peter.jeremy@alcatel.com.au> To: Andreas Koch <koch@eis.cs.tu-bs.de> Cc: freebsd-stable@FreeBSD.ORG Subject: Re: 4.6-RC: Glacial speed of dump backups Message-ID: <20020722081614.E367@gsmx07.alcatel.com.au> In-Reply-To: <20020606204948.GA4540@ultra4.eis.cs.tu-bs.de>; from koch@eis.cs.tu-bs.de on Thu, Jun 06, 2002 at 10:49:48PM %2B0200 References: <20020606204948.GA4540@ultra4.eis.cs.tu-bs.de>
next in thread | previous in thread | raw e-mail | index | archive | help
[This is ancient, but I don't recall seeing anyone else comment]
On Thu, Jun 06, 2002 at 10:49:48PM +0200, Andreas Koch wrote:
>The following applies to 4.6-RC cvsup'ped on May 23.
It's actually a long-standing problem with dump. I first noticed it
in November 1998 (a troll through the freebsd-hackers archives should
find it).
The problem is that dump only reads enough data to satisfy it's
immediate requirements - it makes no effort to aggregate reads and can
re-read the same block of inodes multiple times. This leads to the
"lots of small reads" problem that you've seen. I believe that
NetBSD have made some changes to allieviate this (I haven't tried it).
At this stage, all I can suggest is:
- Try the dump from NetBSD.
- Embed a buffering facility (similar to team) into dump.
- Enhance team (or another buffering tool) to handle multiple volumes.
(The downside is that dump will see the entire backup as one volume
so you won't have checkpoints or the ability to skip tapes on
partial restores).
One difficulty with aggregating reads is that whilst the data out of
two adjacent blocks may be needed, it may need to be written to tape
in a different order and FreeBSD does not support scatter-gather
aggregation[1] - some other OSs have an ioctl to control aggregation.
>Especially when considering that dump internally appears to perform
>some double buffering itself (three processes: one reading, one for
>slack, one writing).
Actually, those slack processes are for checkpointing: dump takes
a checkpoint by forking at the end of each tape and (I think) between
each phase. The former allows dump to restart a volume following
an I/O error.
[1] At least last time I looked, each iovec passed to (eg) writev(2)
gets passed independently to the driver.
Peter
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-stable" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020722081614.E367>