From owner-freebsd-hackers@FreeBSD.ORG Sun Jan 11 05:05:23 2009 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CB676106564A for ; Sun, 11 Jan 2009 05:05:23 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 8C3D08FC0A for ; Sun, 11 Jan 2009 05:05:23 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.2/8.14.1) with ESMTP id n0B532l3034526; Sat, 10 Jan 2009 22:03:02 -0700 (MST) (envelope-from imp@bsdimp.com) Date: Sat, 10 Jan 2009 22:03:22 -0700 (MST) Message-Id: <20090110.220322.2008390173.imp@bsdimp.com> To: peterjeremy@optushome.com.au From: "M. Warner Losh" In-Reply-To: <20090111041710.GB5661@server.vk2pj.dyndns.org> References: <20090109.095027.-1672857892.imp@bsdimp.com> <20090111041710.GB5661@server.vk2pj.dyndns.org> X-Mailer: Mew version 5.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: hackers@FreeBSD.org Subject: Re: 3x read to write ratio on dump/restore X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Jan 2009 05:05:24 -0000 In message: <20090111041710.GB5661@server.vk2pj.dyndns.org> Peter Jeremy writes: : On 2009-Jan-09 09:50:27 -0700, "M. Warner Losh" wrote: : >The read kBps was 3x the write kBps. : ... : >Any ideas what gives? I observed this with 16MB cache and with 32MB : >cache, fwiw. : : I've seen this as well. AFAIK, this is a side-effect of dump's caching. : : My top-of-head explanation is that each dump process has its own cache : but actual I/O is round-robined on a (roughly) block scale so a large : contiguous file will wind up in each 'slave' process's cache. : : The most obvious (and easiest) fixes are to either implement a shared : cache (though this means another level of inter-process communication) : or only use a single 'slave' process when caching is enabled. I'll have to look into this... Most of the files I was backing up were large contiguous files (26 10GiB .dv files from my camera). : The cache algorithm could probably be enhanced as well - apart from : inode blocks, any block will only be accessed once so once a block has : been accessed, it can be purged from the cache (which is completely : opposite to a "normal" cache). Yes, read everything, purge once it is touched. Warner