From owner-freebsd-fs@FreeBSD.ORG Wed Jun 12 12:04:01 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id DAD43E71 for ; Wed, 12 Jun 2013 12:04:01 +0000 (UTC) (envelope-from feld@feld.me) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) by mx1.freebsd.org (Postfix) with ESMTP id AFDB4169D for ; Wed, 12 Jun 2013 12:04:01 +0000 (UTC) Received: from compute4.internal (compute4.nyi.mail.srv.osa [10.202.2.44]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 7061C20E1D; Wed, 12 Jun 2013 08:03:59 -0400 (EDT) Received: from frontend2.nyi.mail.srv.osa ([10.202.2.161]) by compute4.internal (MEProxy); Wed, 12 Jun 2013 08:04:00 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=feld.me; h= content-type:to:cc:subject:references:date:mime-version :content-transfer-encoding:from:message-id:in-reply-to; s= mesmtp; bh=H/qxRRnrMV/D1SIGS6r2tLHnGlw=; b=Cirh4FoyepQSMn+zAtWSs c0SvqAaMh4QVL07s3XnKEmKbzpreUOe3UFPnVgfwXBOlufqfyKSgIkybjDhfjZlt Gdhnoj6hvzajR33Hvo+/bbJ+bseUPuMRC++6Q8xcsgSahe6XN0JAZZPfE+oZr/Nk YFzzt9lqqEQAHDVo91y/Js= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=content-type:to:cc:subject:references :date:mime-version:content-transfer-encoding:from:message-id :in-reply-to; s=smtpout; bh=H/qxRRnrMV/D1SIGS6r2tLHnGlw=; b=JyaT jcphGy4OyitIwiq6ndaGeRrgwY+uCePpxjKtR3KqfRZRTu5Xqvn37rBpC/K+oMOV nbKpAddQ8zuh3uqM+78QOp700UgBpunySYbCBH5j9YbN/39SJiGtqDo29EOnXB1x 0PIsHxjrUsnE5St7AilZ0KFmRZwnzx21BEVBmik= X-Sasl-enc: 2QnJhI0Rp23FCEfkf4xV6ybxSRXJBZxjnOgnLX6JljYw 1371038639 Received: from markf.office.supranet.net (unknown [66.170.8.18]) by mail.messagingengine.com (Postfix) with ESMTPA id 5A6236801F3; Wed, 12 Jun 2013 08:03:59 -0400 (EDT) Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes To: "Jeremy Chadwick" Subject: Re: An order of magnitude higher IOPS needed with ZFS than UFS References: <51B79023.5020109@fsn.hu> <20130612114937.GA13688@icarus.home.lan> Date: Wed, 12 Jun 2013 07:03:58 -0500 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: "Mark Felder" Message-ID: In-Reply-To: <20130612114937.GA13688@icarus.home.lan> User-Agent: Opera Mail/12.15 (FreeBSD) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Jun 2013 12:04:01 -0000 On Wed, 12 Jun 2013 06:49:37 -0500, Jeremy Chadwick wrote: > > Mark, do you have any references for this? I'd love to learn/read more > about this engineering/design aspect (I won't say flaw, I'll just say > aspect) to ZFS, as it's the first I've heard of it. Firsthand experience on a couple servers, and some old Sun docs that I can't find anymore since Oracle broke the links. If you start googling for "ZFS performance 80%" you should come across similar reports. The recommendation was always that when you hit about 80% you need to add a new vdev or you'll be in serious trouble. I'd always believed that it has to do with the way the ZFS COW algorithm works. If my suspicion is correct I'd guess it probably stalls trying to find an ideal place to write -- maybe some cost calculation? I'm reaching for straws now because I don't know anything about the code itself. I'd love to hear from people who have actually touched the code and can give a more definitive answer because this does border on "urban legend" territory, but I've read it and experienced it a few times so I'm just passing it on.