From owner-freebsd-fs@FreeBSD.ORG Tue Jul 16 11:53:12 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 6778B4AC for ; Tue, 16 Jul 2013 11:53:12 +0000 (UTC) (envelope-from feld@freebsd.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by mx1.freebsd.org (Postfix) with ESMTP id 4266AC0 for ; Tue, 16 Jul 2013 11:53:11 +0000 (UTC) Received: from compute6.internal (compute6.nyi.mail.srv.osa [10.202.2.46]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 50E0D20F78 for ; Tue, 16 Jul 2013 07:53:10 -0400 (EDT) Received: from frontend2 ([10.202.2.161]) by compute6.internal (MEProxy); Tue, 16 Jul 2013 07:53:10 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=date:from:to:subject:message-id :references:mime-version:content-type:in-reply-to; s=smtpout; bh=qX1EwoWbVQPKDn700pWCAjjGrYI=; b=n4RZQU1oA4t8qeZJs+MjzNdKAT8a NLe/Et3/QfSZ9wzhRxObj9bsIetrhSBiidSc1Mk7M7xUr5NMyux6ikKYGSn90Yv4 YzdNQ+offHF08VrOQL3d7dR9q9LDutAhX2+efWCJALQL05BVqB0C9df6mzbEvEGb rQrzb0vIK7c9IpA= X-Sasl-enc: +Ml65F9anGLoJbKs+tdj46YkjSfG8VO+dH+mPLkaSHS3 1373975590 Received: from mwi1.coffeenet.org (unknown [66.170.3.2]) by mail.messagingengine.com (Postfix) with ESMTPA id 0F0A56800AE for ; Tue, 16 Jul 2013 07:53:10 -0400 (EDT) Date: Tue, 16 Jul 2013 06:53:05 -0500 From: Mark Felder To: freebsd-fs@freebsd.org Subject: Re: ZFS vdev I/O questions Message-ID: <20130716115305.GA40918@mwi1.coffeenet.org> References: <51E5316B.9070201@digsys.bg> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <51E5316B.9070201@digsys.bg> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jul 2013 11:53:12 -0000 On Tue, Jul 16, 2013 at 02:41:31PM +0300, Daniel Kalchev wrote: > I am observing some "strange" behaviour with I/O spread on ZFS vdevs and > thought I might ask if someone has observed it too. > --SNIP-- > Drives da0-da5 were Hitachi Deskstar 7K3000 (Hitachi HDS723030ALA640, > firmware MKAOA3B0) -- these are 512 byte sector drives, but da0 has been > replaced by Seagate Barracuda 7200.14 (AF) (ST3000DM001-1CH166, firmware > CC24) -- this is an 4k sector drive of a new generation (notice the > relatively 'old' firmware, that can't be upgraded). --SNIP-- > The other observation I have is with the first vdev: the 512b drives do > a lot of I/O fast, complete first and then sit idle, while da0 continues > to write for many more seconds. They consistently show many more IOPS > than the other drives for this type of activity -- on streaming writes > all drives behave more or less the same. It is only on this un-dedup > scenario where the difference is so much pronounced. > > All the vdevs in the pool are with ashift=12 so the theory that ZFS > actually issues 512b writes to these drives can't be true, can it? > > Another worry is this Seagate Barracuda 7200.14 (AF) > (ST3000DM001-1CH166, firmware CC24) drive. It seems constantly > under-performing. Does anyone know if it is so different from the > ST3000DM001-9YN166 drives? Might be, I should just replace it? > A lot of information here. Those Hitachis are great drives. The addition of the Barracuda with different performance characteristics could be part of the problem. I'm glad you pointed out that the pool ashift=12 so we can try to rule that out. I'd be quite interested in knowing if some or perhaps even all of your issues go away simply by replacing that drive with another Hitachi.