From owner-freebsd-fs@FreeBSD.ORG Mon Aug 27 08:28:29 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3EFBF1065673 for ; Mon, 27 Aug 2012 08:28:29 +0000 (UTC) (envelope-from mailinglists@nobaq.net) Received: from mail.nobaq.net (mail.nobaq.net [IPv6:2001:7b8:3cd:3::172]) by mx1.freebsd.org (Postfix) with ESMTP id 3A82E8FC29 for ; Mon, 27 Aug 2012 08:28:24 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=mail.nobaq.net) by mail.nobaq.net with esmtp (Exim 4.72) (envelope-from ) id 1T5ug9-0000qa-Pf; Mon, 27 Aug 2012 10:28:13 +0200 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=nobaq.net; h=message-id :date:from:mime-version:to:cc:references:in-reply-to :content-type:content-transfer-encoding:subject; s=n200908; bh=E QVG1WbAEXXpx54aroQ9iVl8zjs=; b=AKksz0we9Oi021h3Hb3jt8SqZegyWAK5F xHKWF/kEPveIvAgFbSdnW/xt/aeEh85Raq8hQzI5h3JA92MyD6w1KGl1Ogl0GJDh fZNjyhyHsbetxWR0fRBzPK7aB/O86YzScRQyi07Nde5jBOLNXzCQT/LURqMQZGCk ytnn2Gg1T0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=nobaq.net; h=message-id:date :from:mime-version:to:cc:references:in-reply-to:content-type :content-transfer-encoding:subject; q=dns; s=n200908; b=AqPZQrlQ +O6k1RsZQ8RWrFkm1Hh45Bll6VjdYziSYW0ISesPBfeBpH+vZ2Yy/cegwfGgOCC3 mm1ZkCYwQDrTo0Q2xiOxWIwfLAJVC3ZvCviQfKRdfYaPZq290LX9E2Lxkt6HCnou AG5Y+EinnSzBFnL7MuhqxSogGppOcs0yik4= Received: from 1360029737.d-dsl.at ([81.16.104.41] helo=[10.69.11.204]) by mail.nobaq.net with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.72) (envelope-from ) id 1T5ufd-0000q0-B4; Mon, 27 Aug 2012 10:28:13 +0200 Message-ID: <503B2F78.4070800@nobaq.net> Date: Mon, 27 Aug 2012 10:27:36 +0200 From: Niki Hammler User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1.24) Gecko/20100228 Thunderbird/2.0.0.24 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Freddie Cash References: <503A6F9F.7070801@nobaq.net> In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.nobaq.net X-Spam-Level: X-Spam-Status: No, score=-1.8 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.3.1 X-SA-Exim-Connect-IP: 127.0.0.1 X-SA-Exim-Mail-From: mailinglists@nobaq.net X-SA-Exim-Scanned: No (on mail.nobaq.net); SAEximRunCond expanded to false Cc: freebsd-fs@freebsd.org Subject: Re: zvol + raidz issue? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2012 08:28:29 -0000 Hi, According to [1], I want to add some debug info: http://www.nobaq.net/~niki/raidz-problem/zfs_debug_info.txt * iostat while writing to zvol on raidz1: http://www.nobaq.net/~niki/raidz-problem/zpool_iostat_raidz1.txt -> Often only one disk accessed, only few MB, sometimes even kb/s! * iostat while writing 1 GB directly to a dataset (same zpool!): http://www.nobaq.net/~niki/raidz-proble/zpool_iostat_raidz1_dataset2.txt -> No problem * iostat while writing to zvol on mirror (same disks): http://www.nobaq.net/~niki/raidz-proble/zpool_iostat_mirror.txt -> Also no problem Furthermore, I did some more tests and I think this must be a bug: The problem is not related with hardware and disks and it ONLY occurs when I create a zvol on the RAIDz1: Copying linear 1GB file onto a fresh 500g zvol on these zpools: ada2 ada3 (mirror) 105 MB/s ada1 ada3 (mirror) 128 MB/s ada1 ada2 (mirror) 117 MB/s ada1 ada2 ada3 (raidz) 2,5 MB/s <--- !!!!!!!!!!!!! ada1 ada2 ada3 (mirror) 112 MB/s Since 512 and 4096 drives are mixed, the alignment is set correctly: # zdb | grep ashift ashift=12 ashift=12 What other tests/debugging can I do and where could I file a bug report? Regards, Niki [1] http://wiki.freebsd.org/ZFS/ProblemReporting Am 26.08.2012 22:13, schrieb Freddie Cash: > On Aug 26, 2012 11:50 AM, "Niki Hammler" > wrote: > > Hi, > > Given: new HP Proliant Microserver N40L (4 GB RAM) and 3x2TB SATA drives > (SAMSUNG HD204UI, ST32000542AS, WDC WD20EARX-00PASB0). > > Goal: RAIDz1 containg datasets and zvols to be exported via iSCSI. > > Issue: When I create a zvol on a RAIDz1 I get horrible performance (few > MB/s or less). > > First test: 500G zvol on a mirror (freshly created): > > # zpool list > NAME SIZE USED AVAIL CAP HEALTH ALTROOT > plvl1i0 1.81T 1.97G 1.81T 0% ONLINE /mnt > # zfs list > NAME USED AVAIL REFER MOUNTPOINT > plvl1i0 500G 1.30T 112K /mnt/plvl1i0 > plvl1i0/zvtest 500G 1.78T 1.97G - > # dd if=/dev/zero of=/dev/zvol/plvl1i0/zvtest bs=2048k count=1000 > 1000+0 records in > 1000+0 records out > 2097152000 bytes transferred in 17.318348 secs (121094230 bytes/sec) > # > > Corresponds to 115,48 MB/s which is good (similar results for a single > drive). > > Second test: 500G zvol on the 3x2TB raidz1 (freshly created): > > # dd if=/dev/zero of=/dev/zvol/plvl5i0/zvtest bs=2048k count=1000 > > 1000+0 records in > 1000+0 records out > 2097152000 bytes transferred in 700.126725 secs (2995389 bytes/sec) > # > > which is only 2,85 MB/s. > > Remark: Both pools are created with the force 4096 alignment option > (since I have 512 and 4096 drives mixed). > > Now is the point where you might say the problem is related to the > raidz1. But it is not: I created a 500G dataset in the same RAIDz pool > and copied about 100G data onto it with rsync+ssh. Result: about 28MB/s > end2end performance which is reasonable. > > Are there any issues with zvol + raidz1? Google resulted in empty result > set. > > I run a minimal FreeBSD 8.2 (FreeNAS): > > # uname -a > FreeBSD zetta 8.2-RELEASE-p9 FreeBSD 8.2-RELEASE-p9 #0: Thu Jul 19 > 12:39:10 PDT 2012 > root@build.ixsystems.com:/build/home/jpaetzel/8.2.0/os-base/amd64/build/home/jpaetzel/8.2.0/FreeBSD/src/sys/FREENAS.amd64 > amd64 > > Regards, > Niki > > > PS: This is also posted on > http://forums.freenas.org/showthread.php?p=35590 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org > " >