From owner-freebsd-stable Sun Jul 21 15:16:26 2002 Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CC35037B400 for ; Sun, 21 Jul 2002 15:16:24 -0700 (PDT) Received: from alcanet.com.au (mail2.alcanet.com.au [203.62.196.17]) by mx1.FreeBSD.org (Postfix) with ESMTP id DAB5B43E58 for ; Sun, 21 Jul 2002 15:16:23 -0700 (PDT) (envelope-from peter.jeremy@alcatel.com.au) Received: from gsmx07.alcatel.com.au (IDENT:root@localhost.localdomain [127.0.0.1]) by alcanet.com.au (8.12.4/8.12.4/Alcanet1.3) with ESMTP id g6LMGEiv031102; Mon, 22 Jul 2002 08:16:17 +1000 Received: from gsmx07.alcatel.com.au (localhost [127.0.0.1]) by gsmx07.alcatel.com.au (8.12.3/8.12.3) with ESMTP id g6LMGE2h032810; Mon, 22 Jul 2002 08:16:14 +1000 (EST) (envelope-from peter.jeremy@alcatel.com.au) Received: (from jeremyp@localhost) by gsmx07.alcatel.com.au (8.12.3/8.12.3/Submit) id g6LMGEgP032809; Mon, 22 Jul 2002 08:16:14 +1000 (EST) Date: Mon, 22 Jul 2002 08:16:14 +1000 From: Peter Jeremy To: Andreas Koch Cc: freebsd-stable@FreeBSD.ORG Subject: Re: 4.6-RC: Glacial speed of dump backups Message-ID: <20020722081614.E367@gsmx07.alcatel.com.au> Mail-Followup-To: Andreas Koch , freebsd-stable@FreeBSD.ORG References: <20020606204948.GA4540@ultra4.eis.cs.tu-bs.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <20020606204948.GA4540@ultra4.eis.cs.tu-bs.de>; from koch@eis.cs.tu-bs.de on Thu, Jun 06, 2002 at 10:49:48PM +0200 Sender: owner-freebsd-stable@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG [This is ancient, but I don't recall seeing anyone else comment] On Thu, Jun 06, 2002 at 10:49:48PM +0200, Andreas Koch wrote: >The following applies to 4.6-RC cvsup'ped on May 23. It's actually a long-standing problem with dump. I first noticed it in November 1998 (a troll through the freebsd-hackers archives should find it). The problem is that dump only reads enough data to satisfy it's immediate requirements - it makes no effort to aggregate reads and can re-read the same block of inodes multiple times. This leads to the "lots of small reads" problem that you've seen. I believe that NetBSD have made some changes to allieviate this (I haven't tried it). At this stage, all I can suggest is: - Try the dump from NetBSD. - Embed a buffering facility (similar to team) into dump. - Enhance team (or another buffering tool) to handle multiple volumes. (The downside is that dump will see the entire backup as one volume so you won't have checkpoints or the ability to skip tapes on partial restores). One difficulty with aggregating reads is that whilst the data out of two adjacent blocks may be needed, it may need to be written to tape in a different order and FreeBSD does not support scatter-gather aggregation[1] - some other OSs have an ioctl to control aggregation. >Especially when considering that dump internally appears to perform >some double buffering itself (three processes: one reading, one for >slack, one writing). Actually, those slack processes are for checkpointing: dump takes a checkpoint by forking at the end of each tape and (I think) between each phase. The former allows dump to restart a volume following an I/O error. [1] At least last time I looked, each iovec passed to (eg) writev(2) gets passed independently to the driver. Peter To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message