From owner-freebsd-fs@FreeBSD.ORG Sat Feb 6 14:51:16 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 83407106566B for ; Sat, 6 Feb 2010 14:51:16 +0000 (UTC) (envelope-from urmas.lett@eenet.ee) Received: from muheleja.eenet.ee (muheleja.eenet.ee [193.40.0.132]) by mx1.freebsd.org (Postfix) with ESMTP id 3780A8FC12 for ; Sat, 6 Feb 2010 14:51:15 +0000 (UTC) Received: from muheleja.eenet.ee (localhost [127.0.0.1]) by localhost.eenet.ee (Postfix) with ESMTP id E79FF1CC27 for ; Sat, 6 Feb 2010 16:51:13 +0200 (EET) X-Virus-Scanned: amavisd-new at eenet.ee Received: from muheleja.eenet.ee ([127.0.0.1]) by muheleja.eenet.ee (muheleja.eenet.ee [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id D1RPRx20yDBF for ; Sat, 6 Feb 2010 16:51:11 +0200 (EET) Received: from [193.40.0.223] (poriseja.eenet.ee [193.40.0.223]) by muheleja.eenet.ee (Postfix) with ESMTP id 628851CC1F for ; Sat, 6 Feb 2010 16:51:11 +0200 (EET) Message-ID: <4B6D81DB.3090100@eenet.ee> Date: Sat, 06 Feb 2010 16:51:07 +0200 From: Urmas Lett Organization: EENet User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <4B619485.8080106@eenet.ee> <4B63088F.9030407@comcast.net> In-Reply-To: <4B63088F.9030407@comcast.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: Reading from ZFS mirror 2x slower than expected? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Feb 2010 14:51:16 -0000 Steve Polyack wrote: > I can also vouch that the mirror performance is less than stellar. > Writes are also about 50% slower than single-disk performance, although > this may be expected due to checksums / other ZFS overhead. I can try > to provide more details when I get time. I'm luckily observing only 50% slowness when reading from mirror. Writing seems to be OK. dd if=/dev/zero of=test.bin bs=1m count=8192 8589934592 bytes transferred in 143.145040 secs (60008608 bytes/sec) systat -vm shows 400+ tps per disk dd if=test.bin of=/dev/null bs=1m 8589934592 bytes transferred in 155.068721 secs (55394373 bytes/sec) Disks ada0 ada1 ada2 pass0 pass1 pass2 KB/t 0.00 128 128 0.00 0.00 0.00 tps 0 226 224 0 0 0 MB/s 0.00 28.30 28.06 0.00 0.00 0.00 %busy 0 54 64 0 0 0 systat -vm shows only 200+ tps per disk After detaching one disk from mirror: dd if=test.bin of=/dev/null bs=1m 8589934592 bytes transferred in 149.991512 secs (57269471 bytes/sec) Disks ada0 ada1 ada2 pass0 pass1 pass2 KB/t 0.00 128 0.00 0.00 0.00 0.00 tps 0 444 0 0 0 0 MB/s 0.00 55.55 0.00 0.00 0.00 0.00 %busy 0 98 0 0 0 0 systat -vm shows 400+ tps Now I did simple test on the same hardware with Belenix 7.1 liveCD: jack@belenix:/zpool0# dd if=/dev/zero of=8gb.test bs=1024k count=8192 8192+0 records in 8192+0 records out 8589934592 bytes (8.6 GB) copied, 167.512 s, 51.3 MB/s jack@belenix:/zpool0# dd if=8gb.test of=/dev/null bs=1024k 8192+0 records in 8192+0 records out 8589934592 bytes (8.6 GB) copied, 82.4957 s, 104 MB/s read speed from mirror is 2x better and Solaris iostat shows 400+ tps and 50+ MB/s per mirrored disk. -- Urmas Lett Tel: +(372) 7 302 110 Fax: +(372) 7 302 111 E-Mail: urmas.lett@eenet.ee