From owner-freebsd-fs@FreeBSD.ORG  Wed May  2 05:22:43 2007
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
X-Original-To: freebsd-fs@freebsd.org
Delivered-To: freebsd-fs@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id 9583016A400
	for <freebsd-fs@freebsd.org>; Wed,  2 May 2007 05:22:43 +0000 (UTC)
	(envelope-from bakul@bitblocks.com)
Received: from mail.bitblocks.com (ns1.bitblocks.com [64.142.15.60])
	by mx1.freebsd.org (Postfix) with ESMTP id 8602513C45B
	for <freebsd-fs@freebsd.org>; Wed,  2 May 2007 05:22:43 +0000 (UTC)
	(envelope-from bakul@bitblocks.com)
Received: from bitblocks.com (localhost.bitblocks.com [127.0.0.1])
	by mail.bitblocks.com (Postfix) with ESMTP id 485FE5B51
	for <freebsd-fs@freebsd.org>; Tue,  1 May 2007 22:22:43 -0700 (PDT)
To: freebsd-fs@freebsd.org
Date: Tue, 01 May 2007 22:22:43 -0700
From: Bakul Shah <bakul@bitblocks.com>
Message-Id: <20070502052243.485FE5B51@mail.bitblocks.com>
Subject: ZFS vs UFS2 overhead and may be a bug?
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 02 May 2007 05:22:43 -0000

Here is a surprising result for ZFS.

I ran the following script on both ZFS and UF2 filesystems.

$ dd </dev/zero bs=1m count=10240 >SPACY# 10G zero bytes allocated
$ truncate -s 10G HOLEY			# no space allocated

$ time dd <SPACY >/dev/null bs=1m	# A1
$ time dd <HOLEY >/dev/null bs=1m	# A2
$ time cat SPACY >/dev/null		# B1
$ time cat HOLEY >/dev/null		# B2
$ time md5 SPACY			# C1
$ time md5 HOLEY			# C2

I have summarized the results below.

		      ZFS	     UFS2
		Elapsed System	Elapsed	System	       Test
dd SPACY bs=1m  110.26   22.52	340.38	 19.11		A1
dd HOLEY bs=1m   22.44   22.41	 24.24	 24.13		A2

cat SPACY	119.64   33.04	342.77	 17.30		B1
cat HOLEY	222.85  222.08	 22.91	 22.41		B2

md5 SPACY	210.01	 77.46	337.51	 25.54		C1	
md5 HOLEY	856.39	801.21	 82.11	 28.31		C2


A1, A2:
Numbers are more or less as expected.  When doing large
reads, reading from "holes" takes far less time than from a
real disk.  We also see that UFS2 disk is about 3 times
slower for sequential reads.

B1, B2:
UFS2 numbers are as expected but ZFS numbers for the HOLEY
file are much too high.  Why should *not* going to a real
disk cost more?  We also see that UFS2 handles holey files 10
times more efficiently than ZFS!

C1, C2:
Again UFS2 numbers and C1 numbers for ZFS are as expected.
but C2 numbers for ZFS are very high.  md5 uses BLKSIZ (==
1k) size reads and does hardly any other system calls.  For
ZFS each syscall takes 76.4 microseconds while UFS2 syscalls
are 2.7 us each!  zpool iostat shows there is no IO to the
real disk so this implies that for the HOLEY case zfs read
calls have a significantly higher overhead or there is a bug.

Basically C tests just confirm what we find in B tests.