From owner-freebsd-stable@freebsd.org Tue Feb 9 16:35:26 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E4FE6AA2685 for ; Tue, 9 Feb 2016 16:35:25 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi0-x229.google.com (mail-oi0-x229.google.com [IPv6:2607:f8b0:4003:c06::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AC4B713B2 for ; Tue, 9 Feb 2016 16:35:25 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oi0-x229.google.com with SMTP id m82so18259915oif.2 for ; Tue, 09 Feb 2016 08:35:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=1/wxarL/jvmnm/fTbrr0ssfOFQvpsaxTazG6rNwo630=; b=dj8Vl9AhSEZL3f0yny1dcbgDeOVMonkIfcethsJ2QjJYMD6jWEn8/RluWT5s+Pflza dKfZQP+63CmQqa3tip8/Orz3eqnRpDkajER/VYf99XaNuPhizgw1dh1i+y3vNV9DjSH4 P3+M+UAfsx4mIixRxmfq7izYAvFj4Yd9z6tpUuhk8QMl0WUbYcXZxDuMEokXUIXI9yQm OBF5xLABJJ1AJ2lU4jKzLwEjN5SeremGj9CkKN7Ig8Gy1NtumVauX5hWBS6nrVm+u7uu 51aylKSNgf3KsqvxQ/g1GEq6BKExd8yRAYkPTB6GZhN9gFfq70zvQtLjWabKC35XWWct vp4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=1/wxarL/jvmnm/fTbrr0ssfOFQvpsaxTazG6rNwo630=; b=AqDy6Mbf/vtExG4gNrmxQrkvjb7LANeb3mFvKJWPdIUQdv39Xd34OPittCpVqoRQhn zdz2XZaglyX56b+qAgwThLesosCpYJzGVZU3HGTiuTo6ROzCgNC9gpOT/5Du/aipYh2x 9i3FVtlDE9kR5v3LgdTTMu7Z75lEnNhhe41W6KL57mcKw8bnMJzcv1+jzxlyKBEsYzM8 cnVIVbRjipDmj4zhySthnUfGvQ539X53y302z6e9ojSV6L7+xSbMz/Yh1wJSRofjztk8 koiRKMe+NETsRm8b/HnUxl4umnQJBeYRJsUgSuy9zxuKXw2G5z4t8M753Hj+FLUHelkF Y3Jg== X-Gm-Message-State: AG10YOTFnu5qvfpYL5Bn0ti3piRIyOV0kDJe3ph+06AYYTFecF0MJvO1/+y65IGFq++yeZTUOiEw329LfS62ww== MIME-Version: 1.0 X-Received: by 10.202.199.214 with SMTP id x205mr4305569oif.107.1455035535227; Tue, 09 Feb 2016 08:32:15 -0800 (PST) Sender: asomers@gmail.com Received: by 10.202.210.79 with HTTP; Tue, 9 Feb 2016 08:32:15 -0800 (PST) In-Reply-To: <2D296837-3B06-4E72-B8B0-A33AE6CE48AE@punkt.de> References: <2D296837-3B06-4E72-B8B0-A33AE6CE48AE@punkt.de> Date: Tue, 9 Feb 2016 09:32:15 -0700 X-Google-Sender-Auth: epMGRjZrZJPJg7r6mJZl_wDvXdM Message-ID: Subject: Re: Best practices for ZFS setup for a strictly SSD based system? From: Alan Somers To: "Patrick M. Hausen" Cc: freebsd-stable Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Feb 2016 16:35:26 -0000 On Tue, Feb 9, 2016 at 8:54 AM, Patrick M. Hausen wrote: > Hi, all, > > while there is quite a bit of documentation on how to improve ZFS performance > by using a combination of rotating disks and SSDs, I have not found much about > an SSD only setup. > > We are planning to try a hosting server with 8 SATA SSDs with ZFS. Things I am > not at all sure about: > > * Does the recommended limit of 6 disks for a RAIDZ2 still > hold? 2x 4 disks is quite a bit of overhead, could I use all 8 > in one vdev and get away with it? > (The maximum of 6 recommendation is in some old Sun doc) Nah, you can go much higher. This post describes the RAIDZ overhead. The main penalty to larger stripes is lower IOPs. Your RAIDZ array will have the same read IOPs as a single SSD, no matter how large it is. So, for example, a pool of two RAIDZ stripes each containing 4+2 disks will have twice the IOPS as a pool containing one RAIDZ stripe with 8+2 disks, and about the same storage overhead. http://blog.delphix.com/matt/2014/06/06/zfs-stripe-width/ > > * Will e.g. MySQL still profit from residing on a mirror > instead of a RAIDZ2, even if all disks are SSDs? Yes, because a mirrored vdev has as many read IOPs as all of its disks combined. So a RAID10 of SSDs will have many read IOPs indeed. > > * Does a separate ZIL and/or ARC cache device still > make sense? Usually no. But it might make a difference if the ZIL or L2ARC devices have different characteristics from the regular devices. For example, you might use medium speed MLC flash for your regular vdevs and a very fast, small SLC device for the ZIL. But I wouldn't do it unless you thoroughly test it with your workload. > > Any pointers or direct help greatly appreciated. Or should I take this to freebsd-fs@? Will MySQL access its files in fixed-size records? If so, you can set the recsize filesystem property accordingly. If not, you should probably leave recsize at the default. If you profile MySQL's disk accesses and determine that there is a dominant recordsize, then go ahead and set ZFS's recsize to the next highest power of two. As usual, disable atime. > > Thanks and best regards, > Patrick > -- -Alan