From owner-freebsd-questions@freebsd.org Thu Sep 24 13:17:08 2015 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 069B4A07FD2 for ; Thu, 24 Sep 2015 13:17:08 +0000 (UTC) (envelope-from paul@kraus-haus.org) Received: from mail-qk0-f172.google.com (mail-qk0-f172.google.com [209.85.220.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BEBAC1276 for ; Thu, 24 Sep 2015 13:17:07 +0000 (UTC) (envelope-from paul@kraus-haus.org) Received: by qkdw123 with SMTP id w123so29294998qkd.0 for ; Thu, 24 Sep 2015 06:17:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:content-transfer-encoding:message-id:references:to; bh=ZcSzKDTGCsjez3uskXiMSKo3PE47Owc2XlF13o6oGAE=; b=it3l1oeHy4zRy4iGW3DqfBnZSwhBDbbs0ATRx65KDMM2jBFL+Ojov3KGn+mxKzCrs4 u5bLusyAzbvadr0JI3/9L9W4S1HAaO5x52i8idk2D/H9nAaH5Q+PuAG/lLJmkodOav6D tT2AMJa+ofX4WxlDmsaRsl9mhuTFuC0Qyn/iWF3AyKRJybNj44DxEsDNY81RIEV2uNT0 C0hVoPOeRnTEyIyR9NqnVRVJb0ByZvyR8PiqUgJA+LUWTm56vgoAbc9e5f+RpSldshsD e0zKpUWqk2N45m7aUPCL1dNsxw+vRdC1oM9Jq6lWHpQS20OnwhEAqtS3AWqqk2flR546 ybxA== X-Gm-Message-State: ALoCoQkTe/K2rwSvrs2ms9UOdMHNUpnMiTg16t1VmI9RVdMvnaFs+dN3S6g3D86y+Q6Nlnq0knuR X-Received: by 10.55.23.161 with SMTP id 33mr1788610qkx.90.1443100626128; Thu, 24 Sep 2015 06:17:06 -0700 (PDT) Received: from mbp-1.thecreativeadvantage.com (mail.thecreativeadvantage.com. [96.236.20.34]) by smtp.gmail.com with ESMTPSA id b104sm3424769qga.7.2015.09.24.06.17.03 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 24 Sep 2015 06:17:04 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: zfs performance degradation From: Paul Kraus In-Reply-To: <56038054.5060906@dim.lv> Date: Thu, 24 Sep 2015 09:17:02 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <782C9CEF-BE07-4E05-83ED-133B7DA96780@kraus-haus.org> References: <56019211.2050307@dim.lv> <37A37E9D-9D65-4553-BBA2-C5B032163499@kraus-haus.org> <56038054.5060906@dim.lv> To: Dmitrijs , FreeBSD Questions X-Mailer: Apple Mail (2.1878.6) X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Sep 2015 13:17:08 -0000 On Sep 24, 2015, at 0:47, Dmitrijs wrote: > 2015.09.23. 23:08, Paul Kraus =D0=BF=D0=B8=D1=88=D0=B5=D1=82: >> On Sep 22, 2015, at 13:38, Dmitrijs wrote: >>=20 >>> I've encountered strange ZFS behavior - serious performance = degradation over few days. >>>=20 >>> Could it happen because of pool being 78% full? So I cannot fill = puls full? >>> Can anyone please advice how could I fix the situation - or is it = normal? >>=20 >> So the short answer (way too late for that) is that you can, in fact, = not use all of the capacity of a zpool unless the data is written once, = never modified, and you do not have any snapshots, clones, or the like. > Thank you very much for explanation. Am I getting it right - it will = not work faster even if I add +4Gb RAM to be 8Gb in total? I am not = using DeDuplication and compression, neither planing using them. If you are seeing the performance degrade due to the zpool being over = some capacity threshold, then adding RAM will make little difference. If = you are seeing general performance issues, then adding RAM (increasing = ARC) _may_ improve the performance. > So if I plan to work with data a lot, get decent performance and still = be sure I'm on the safe side with mirror-raid1, should I choose another = filesystem? Especially, if i do not really need snapshots, clones, etc. What is your definition of =E2=80=9Cdecent=E2=80=9D performance ? What = does your _real_ workload look like ? Did you have performance issues doing real work which caused you to try = to find the cause -or- were you benchmarking before trying to use the = system for real work ? > Or is it not possible at all, and I should put something like raid0 = for work and tolerate slow backup on raid1 at nights? There are many places in ZFS where you can run into performance = bottlenecks. Remember, ZFS was designed for data integrity (end to end = checksums), data reliability (lots of ways to get redundancy), and = scalability. Performance was secondary from the very beginning. There = are lots of other filesystems with much better performance, there are = few (if any) with more protection for your data. Do not get me wrong, = the performance of ZFS _can_ be very good, but you need to understand = your workload and layout the zpool to accommodate that workload. For example, one of my critical workloads is NFS with sync writes. My = zpool layout is many vdevs of 3-way mirrors with a separate ZIL device = (SLOG). I have not been able to go production with this server yet = because I am waiting on backordered SSDs for the SLOG. The original SSDs = I used just did not have the small block write performance I needed. Another example is one of my _backup_ servers, which has a 6 drive = RAIDz2 zpool layout. In this case I am not terribly concerned about = performance as I am limited by the 1 Gbps network connection. Also note that in general, the _best_ performance you can expect of any = zpool layout is equivalent to _1_ drives worth of I/O per _vdev_. So my = 6 drive RAIDz2 has performance equivalent to _one_ of the drives that = make up that vdev. Which is fine for _my_ workload. The rule of thumb = for performance that I received over on the OpenZFS mailing list a while = back was to assume you can get 100 MB/sec and 100 random I/Ops from a = consumer SATA hard disk drive. I have seen nothing, even using = =E2=80=9Centerprise=E2=80=9D grade HDDs, to convince me that is a bad = rule of thumb. If your workload is strictly sequential you _may_ get = more. So a zpool made up of one single vdev, no matter how many drives, will = average the performance of one of those drives. It does not really = matter if it is a 2-way mirror vdev, a 3-way mirror vdev, a RAIDz2 vdev, = a RAIDz3 vdev, etc. This is more true for write operations that read = (mirrors can achieve higher performance by reading from multiple copies = at once). -- Paul Kraus paul@kraus-haus.org