From owner-freebsd-questions@freebsd.org Thu Sep 24 18:46:14 2015 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 35118A08DF9 for ; Thu, 24 Sep 2015 18:46:14 +0000 (UTC) (envelope-from paul@kraus-haus.org) Received: from mail-qg0-f52.google.com (mail-qg0-f52.google.com [209.85.192.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E959B1E72 for ; Thu, 24 Sep 2015 18:46:12 +0000 (UTC) (envelope-from paul@kraus-haus.org) Received: by qgx61 with SMTP id 61so51935278qgx.3 for ; Thu, 24 Sep 2015 11:46:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:mime-version:content-type:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=OVpPrICU5R+Oq0hbzRFdwMaRI2EEQKKzEYb2eJOIQVM=; b=LjPdTtuKXGYewKlY5lbhlms0QL7faHhcZwLGQnzPOrxoY7+Z6i8YD/b1NOUKx2Eq+k YdviU8EeArHDjXz8f48/mGJJMMcGABxZG8/N+q8Ggk5Eg+XSnFBi0oASsFsA7GXox15m JvcRzT7Wg187I8d4fIYvR8tcycRpzhH21MDP1JUN+fCu9f98nwwNPivtrKpyvXSn5L0j /EhPoOJk77EPTc0/r11JSB0KjyPlx1IfEcXj/MFel6Wc68PJWENwsaLAiHgsZOclcj07 ap1mORVknOj72nxp5YZSEV+etu956AbkZNdC7K9USXyiywzh9pg9BYvuPVk8npkv2R1/ g53A== X-Gm-Message-State: ALoCoQl78+5eu7woRgVj0+Hshw/m6V9JaiHjmwNl9bEhl9B0y6qjh01oC0qdGSgMgGTWRvw3/XmU X-Received: by 10.140.101.119 with SMTP id t110mr1556676qge.71.1443119884003; Thu, 24 Sep 2015 11:38:04 -0700 (PDT) Received: from mbp-1.thecreativeadvantage.com (mail.thecreativeadvantage.com. [96.236.20.34]) by smtp.gmail.com with ESMTPSA id 139sm4949302qhh.43.2015.09.24.11.38.01 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 24 Sep 2015 11:38:01 -0700 (PDT) Subject: Re: zfs performance degradation Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Content-Type: text/plain; charset=windows-1252 From: Paul Kraus In-Reply-To: <56042209.8040903@dim.lv> Date: Thu, 24 Sep 2015 14:37:56 -0400 Cc: FreeBSD Questions Content-Transfer-Encoding: quoted-printable Message-Id: <2008181C-F0B5-4581-9D15-11911A1DE41B@kraus-haus.org> References: <56019211.2050307@dim.lv> <37A37E9D-9D65-4553-BBA2-C5B032163499@kraus-haus.org> <56038054.5060906@dim.lv> <782C9CEF-BE07-4E05-83ED-133B7DA96780@kraus-haus.org> <56040150.90403@dim.lv> <60BF2FC3-0342-46C9-A718-52492303522F@kraus-haus.org> <560412B2.9070905@dim.lv> <8D1FF55C-7068-4AB6-8C0E-B4E64C1BB5FA@kraus-haus.org> <56042209.8040903@dim.lv> To: Dmitrijs X-Mailer: Apple Mail (2.1878.6) X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Sep 2015 18:46:14 -0000 On Sep 24, 2015, at 12:17, Dmitrijs wrote: > I also get about 60-70MB/sec via CIFS or ftp, but my aim is to be = limited by network, so 100MB is wanted.=20 It=92s nice to want things. But be prepared to pay for the things you = want... > Or, to understand why it is not possible on my config :) > But simple dd of=3D/dev/null in the console shows me 110MB/sec=85 dd of anything, but especially of /dev/null is a very poor way of = measuring anything. Turn on compression and and do that test again. > iozone gives me the same 100+Mb/sec both on read and write. What size blocks ? Files ? Random or sequential I/O ? All that matters. > 2x HGST HDN724040ALE640, 4Tb, 64Mb, 7200. Consumer NAS drives=85 I have not purchased anything but an Enterprise = drive for close to 10 years now. The small additional cost is well worth = the longer (5 year) warranty and better build quality=85 there _is_ a = difference. Even looking at the specs, the uncorrectable error spec is a = very good indicator of build quality and these drives are typical 1 in = 10^14 consumer drives. Enterprise drives are typically an order of = magnitude better, 1 in 10^15. In your original post you mentioned WD Green drives, also consumer = grade. In my experience I have seen better performance from WD than = HGST, with Seagate at the bottom of the ladder. I was comparing all = Enterprise drives, and even among those offerings there are differences=85= the WD RE are noticeable faster than the SE. I look at svc_t as the = primary metric for _comparing_ drives. I create a simple striped zpool, = reboot the system to clear counters, then do _lots_ of (typically) = random I/O, then look at iostat -x and compare svc_t, lower numbers are = better. I generally don=92t buy matched drives for mirrors, but different makes = and models if I can. That way of there is a bad production run I don=92t = lose all my drives at once. When a drive fails I RMA it under warranty = and buy 2 more of the same type and capacity, one goes into the server = and the other sits on the shelf. Eventually I have enough drives to grow = the zpool and move on. > For example, yesterday I explored QNAP TS-451 > official site: = https://www.qnap.com/i/en/product/model.php?II=3D143&event=3D2 (Intel=AE = Celeron=AE 2.41GHz dual-core processor, 1GB DDR3L, etc) > and review: http://www.storagereview.com/qnap_ts451_nas_review > 473euro They _might_ be fine products, But I don=92t trust my data to = appliances. _I_ want to control the redundancy. > Promised performance of the models is about 100Mb/sec, even up to = 200Mb/sec but ok, it's marketing and pretty diagrams ;) And all the tests were probably done on empty (to start) volumes. You = can achieve similar numbers with ZFS with similar hardware and LOTS of = parallel clients. -- Paul Kraus paul@kraus-haus.org