From owner-freebsd-performance@FreeBSD.ORG Thu Nov 17 18:49:40 2011 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 977231065674 for ; Thu, 17 Nov 2011 18:49:40 +0000 (UTC) (envelope-from list@sprymed.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 337BE8FC1D for ; Thu, 17 Nov 2011 18:49:39 +0000 (UTC) Received: by bkbzs8 with SMTP id zs8so3251250bkb.13 for ; Thu, 17 Nov 2011 10:49:39 -0800 (PST) MIME-Version: 1.0 Received: by 10.204.10.81 with SMTP id o17mr34648582bko.65.1321554409205; Thu, 17 Nov 2011 10:26:49 -0800 (PST) Received: by 10.205.83.206 with HTTP; Thu, 17 Nov 2011 10:26:49 -0800 (PST) Date: Thu, 17 Nov 2011 13:26:49 -0500 Message-ID: From: "list, mailing" To: freebsd-performance@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: ZFS Few Questions X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Nov 2011 18:49:40 -0000 Hello everyone I just had a few questions about ZFS. I normally use Hardware RAID 5. Question 1: With the ZFS snapshots what is the lost in drive space? Hard drives I have: 4 x 500 GB = 1.5T on RAID 5 I have see lots of videos like: http://www.youtube.com/watch?v=CN6iDzesEs0 Question 2: FreeBSD 9.0 installable on ZFS root? Question 3: Anyone Recommend for MySQL server? (Performance) Question 4: fsck used with when Server just turns off? (Fast or slower filesystem check when compared to UFS on HW RAID 5 ) Taking note of: "FreeBSD 9.0 adds support for lightweight journaling on top of softupdates(SU+J), which greatly reduces and need for background fsck, and uses NFS-style ACLs by default." Thanks for all your input on setting up a new FAMP box. -- Ben Adams http://www.SpryMed.com/ From owner-freebsd-performance@FreeBSD.ORG Thu Nov 17 19:04:55 2011 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 057C7106564A for ; Thu, 17 Nov 2011 19:04:55 +0000 (UTC) (envelope-from feld@feld.me) Received: from mwi1.coffeenet.org (unknown [IPv6:2607:f4e0:100:300::2]) by mx1.freebsd.org (Postfix) with ESMTP id AF9178FC08 for ; Thu, 17 Nov 2011 19:04:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=feld.me; s=blargle; h=In-Reply-To:Message-Id:From:Mime-Version:Date:References:Subject:To:Content-Type; bh=jubh7kzh4Srrp4RX0Dd2u+UUvUdM0NW7/VkpmzBpD7U=; b=CdjJZ7nfj1BFFrL/OLRbJa7Sd4GWom7KW4kIOyFCE3RRN4TDRb1P+5kEvtP+EL8V5JBhYcHi8u8XSByooRRlVab6KkhwkyagAt/iy4LJFDJTKvKJL2Oq+ZEqfZEHTZW9; Received: from localhost ([127.0.0.1] helo=mwi1.coffeenet.org) by mwi1.coffeenet.org with esmtp (Exim 4.77 (FreeBSD)) (envelope-from ) id 1RR7GX-000PaT-IO for freebsd-performance@freebsd.org; Thu, 17 Nov 2011 13:04:54 -0600 Received: from feld@feld.me by mwi1.coffeenet.org (Archiveopteryx 3.1.4) with esmtpsa id 1321556687-1840-1839/5/22; Thu, 17 Nov 2011 19:04:47 +0000 Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: freebsd-performance@freebsd.org References: Date: Thu, 17 Nov 2011 13:04:46 -0600 Mime-Version: 1.0 From: Mark Felder Message-Id: In-Reply-To: User-Agent: Opera Mail/11.60 (FreeBSD) X-SA-Score: -1.0 Subject: Re: ZFS Few Questions X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Nov 2011 19:04:55 -0000 On Thu, 17 Nov 2011 12:26:49 -0600, list, mailing wrote: > Question 1: > With the ZFS snapshots what is the lost in drive space? ZFS tracks the blocks that change. If you change 60GB of data, you'll lose 60GB of space. (roughly... there is probably some slight overhead but you get the point) > Question 2: > FreeBSD 9.0 installable on ZFS root? Not in the bsdinstaller. You can do it manually though. > Question 3: > Anyone Recommend for MySQL server? (Performance) No idea; I haven't run any SQL servers on ZFS > Question 4: > fsck used with when Server just turns off? ZFS doesn't fsck. From owner-freebsd-performance@FreeBSD.ORG Thu Nov 17 19:28:37 2011 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 22713106566B for ; Thu, 17 Nov 2011 19:28:37 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id C65BC8FC14 for ; Thu, 17 Nov 2011 19:28:36 +0000 (UTC) Received: by vcbfy13 with SMTP id fy13so2286323vcb.13 for ; Thu, 17 Nov 2011 11:28:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=POzRw1UBfoBBZWXzvd1oVHdfJeAiUS6BbMIFLq6CQ1o=; b=pKCzfGoL1MOhJxatV88VhofKgH4Lnb2oGsSIgeaIeScTYN5dBnD74pKDR+0G5H03F0 GloWj1RH0W7daFYXJ4z7VXTlCBX15s+QkMgaUU1hcI8YC9qOU0bVFT4faxTpWoDohzwv 0JSi7gcu8t8ZkYsRAQ8JTY1mVgQKvzTpuZupY= MIME-Version: 1.0 Received: by 10.220.153.81 with SMTP id j17mr155203vcw.30.1321556312347; Thu, 17 Nov 2011 10:58:32 -0800 (PST) Received: by 10.220.190.71 with HTTP; Thu, 17 Nov 2011 10:58:32 -0800 (PST) In-Reply-To: References: Date: Thu, 17 Nov 2011 10:58:32 -0800 Message-ID: From: Freddie Cash To: "list, mailing" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-performance@freebsd.org Subject: Re: ZFS Few Questions X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Nov 2011 19:28:37 -0000 On Thu, Nov 17, 2011 at 10:26 AM, list, mailing wrote: > Hello everyone I just had a few questions about ZFS. > I normally use Hardware RAID 5. > > Question 1: > > With the ZFS snapshots what is the lost in drive space? > > If no data changes after you create the snapshots, then no disk space is used by the snapshots. If data changes after you create the snapshot, then the snapshot holds the original data. For example, if you have 100 GB in the filesystem, create a snapshot, then modify 10 GB of data, the snapshot will hold 10 GB of data (the original, unchanged data), and the total disk usage (filesystem + snapshot) is 110 GB. > Hard drives I have: > 4 x 500 GB = 1.5T on RAID 5 > > I have see lots of videos like: http://www.youtube.com/watch?v=CN6iDzesEs0 With ZFS, you decide how much disk space you want to use for redundancy. With 4 harddrives, you have the following options: 2x mirror vdev = 1.0 TB of usable space; best performance, can lose 2 drives before losing data 1x raidz1 vdev = 1.5 TB of usable space; decent performance, can lose 1 drive before losing data 1x raidz2 vdev = 1.0 TB of usable space; ok performance,, can lose 2 drives before losing data raidz1 is similar to RAID5. raidz2 is similar to RAID6. > > Question 2: > > FreeBSD 9.0 installable on ZFS root? > Yes. > Question 3: > > Anyone Recommend for MySQL server? (Performance) > Read through the ZFS Admin Guide for recommendations for running databases on top of ZFS. > Question 4: > > fsck used with when Server just turns off? (Fast or slower filesystem check > when compared to UFS on HW RAID 5 ) > Taking note of: "FreeBSD 9.0 adds support for lightweight journaling on top > of softupdates(SU+J), which greatly reduces and need for background fsck, > and uses NFS-style ACLs by default." > > ZFS does not have a separate "fsck" tool. It does not need it. If the box crashes, ZFS will just come back online, possibly losing 5-10 seconds worth of uncommitted data. If ZFS is unable to come back online automatically, you can manually roll it back a transaction group or two. You'll lose a bit of data, but the filesystems will be coherent and intact and operational. -- Freddie Cash fjwcash@gmail.com From owner-freebsd-performance@FreeBSD.ORG Thu Nov 17 19:52:00 2011 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 09B9A1065693 for ; Thu, 17 Nov 2011 19:52:00 +0000 (UTC) (envelope-from feld@feld.me) Received: from mwi1.coffeenet.org (unknown [IPv6:2607:f4e0:100:300::2]) by mx1.freebsd.org (Postfix) with ESMTP id CB8058FC0C for ; Thu, 17 Nov 2011 19:51:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=feld.me; s=blargle; h=In-Reply-To:Message-Id:From:Mime-Version:Date:References:Subject:To:Content-Type; bh=k7bhLXCOety69mZh5FtuzYiyFT5RhOYALVzvUXP+apE=; b=WhhA4K9uoDO7Cbr+tG4PF00PAH7FZFOeAA8ezRFoV/K6ljHmx3V6IaLyNJDag8qRTI9yUNqLdQY/+yam71TxPYdID+dJ49Y1BIhobQrqyjRT8b0+EBso6a8+dwtU8o0K; Received: from localhost ([127.0.0.1] helo=mwi1.coffeenet.org) by mwi1.coffeenet.org with esmtp (Exim 4.77 (FreeBSD)) (envelope-from ) id 1RR806-00011P-QJ for freebsd-performance@freebsd.org; Thu, 17 Nov 2011 13:51:59 -0600 Received: from feld@feld.me by mwi1.coffeenet.org (Archiveopteryx 3.1.4) with esmtpsa id 1321559512-1840-1839/5/23; Thu, 17 Nov 2011 19:51:52 +0000 Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: freebsd-performance@freebsd.org References: Date: Thu, 17 Nov 2011 13:51:51 -0600 Mime-Version: 1.0 From: Mark Felder Message-Id: In-Reply-To: User-Agent: Opera Mail/11.60 (FreeBSD) X-SA-Score: -1.0 Subject: Re: ZFS Few Questions X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Nov 2011 19:52:00 -0000 On Thu, 17 Nov 2011 12:58:32 -0600, Freddie Cash wrote: >> >> Question 2: >> >> FreeBSD 9.0 installable on ZFS root? >> > Yes. This is now an option in the installer? It wasn't last time I checked, but that was a while ago and last I heard it wasn't going to make it in... I'll have to fire up the latest RC and look. From owner-freebsd-performance@FreeBSD.ORG Thu Nov 17 20:07:46 2011 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 228B1106566C for ; Thu, 17 Nov 2011 20:07:46 +0000 (UTC) (envelope-from list@sprymed.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 9FF888FC0A for ; Thu, 17 Nov 2011 20:07:45 +0000 (UTC) Received: by bkbzs8 with SMTP id zs8so3376934bkb.13 for ; Thu, 17 Nov 2011 12:07:44 -0800 (PST) MIME-Version: 1.0 Received: by 10.204.10.81 with SMTP id o17mr86566bko.65.1321560464227; Thu, 17 Nov 2011 12:07:44 -0800 (PST) Received: by 10.205.83.206 with HTTP; Thu, 17 Nov 2011 12:07:44 -0800 (PST) In-Reply-To: References: Date: Thu, 17 Nov 2011 15:07:44 -0500 Message-ID: From: "list, mailing" To: Mark Felder Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "freebsd-performance@freebsd.org" Subject: Re: ZFS Few Questions X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Nov 2011 20:07:46 -0000 Thanks for all the help. On Thursday, November 17, 2011, Mark Felder wrote: > On Thu, 17 Nov 2011 12:58:32 -0600, Freddie Cash wrote: > >>> >>> Question 2: >>> >>> FreeBSD 9.0 installable on ZFS root? >>> >> Yes. > > > This is now an option in the installer? It wasn't last time I checked, but that was a while ago and last I heard it wasn't going to make it in... I'll have to fire up the latest RC and look. > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to " freebsd-performance-unsubscribe@freebsd.org" > -- Ben Adams http://www.SpryMed.com/ From owner-freebsd-performance@FreeBSD.ORG Thu Nov 17 20:12:18 2011 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 36FAD1065672 for ; Thu, 17 Nov 2011 20:12:18 +0000 (UTC) (envelope-from list@sprymed.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id C1B618FC12 for ; Thu, 17 Nov 2011 20:12:17 +0000 (UTC) Received: by bkbzs8 with SMTP id zs8so3383996bkb.13 for ; Thu, 17 Nov 2011 12:12:16 -0800 (PST) MIME-Version: 1.0 Received: by 10.205.131.3 with SMTP id ho3mr164872bkc.11.1321560736426; Thu, 17 Nov 2011 12:12:16 -0800 (PST) Received: by 10.205.83.206 with HTTP; Thu, 17 Nov 2011 12:12:16 -0800 (PST) In-Reply-To: References: Date: Thu, 17 Nov 2011 15:12:16 -0500 Message-ID: From: "list, mailing" To: Freddie Cash Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "freebsd-performance@freebsd.org" Subject: Re: ZFS Few Questions X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Nov 2011 20:12:18 -0000 Wow. Great reply. Going to do some practice with mirror type and 9 RC 2. Thanks again On Thursday, November 17, 2011, Freddie Cash wrote: > On Thu, Nov 17, 2011 at 10:26 AM, list, mailing wrote: >> >> Hello everyone I just had a few questions about ZFS. >> I normally use Hardware RAID 5. >> >> Question 1: >> >> With the ZFS snapshots what is the lost in drive spacoe? >> > > If no data changes after you create the snapshots, then no disk space is used by the snapshots. > If data changes after you create the snapshot, then the snapshot holds the original data. For example, if you have 100 GB in the filesystem, create a snapshot, then modify 10 GB of data, the snapshot will hold 10 GB of data (the original, unchanged data), and the total disk usage (filesystem + snapshot) is 110 GB. > >> >> Hard drives I have: >> 4 x 500 GB = 1.5T on RAID 5 >> >> I have see lots of videos like: ZFS is Smashing Baby > > With ZFS, you decide how much disk space you want to use for redundancy. With 4 harddrives, you have the following options: > 2x mirror vdev = 1.0 TB of usable space; best performance, can lose 2 drives before losing data > 1x raidz1 vdev = 1.5 TB of usable space; decent performance, can lose 1 drive before losing data > 1x raidz2 vdev = 1.0 TB of usable space; ok performance,, can lose 2 drives before losing data > raidz1 is similar to RAID5. raidz2 is similar to RAID6. > >> >> Question 2: >> >> FreeBSD 9.0 installable on ZFS root? > > Yes. > >> >> Question 3: >> >> Anyone Recommend for MySQL server? (Performance) > > Read through the ZFS Admin Guide for recommendations for running databases on top of ZFS. > >> >> Question 4: >> >> fsck used with when Server just turns off? (Fast or slower filesystem check >> when compared to UFS on HW RAID 5 ) >> Taking note of: "FreeBSD 9.0 adds support for lightweight journaling on top >> of softupdates(SU+J), which greatly reduces and need for background fsck, >> and uses NFS-style ACLs by default." >> > ZFS does not have a separate "fsck" tool. It does not need it. If the box crashes, ZFS will just come back online, possibly losing 5-10 seconds worth of uncommitted data. If ZFS is unable to come back online automatically, you can manually roll it back a transaction group or two. You'll lose a bit of data, but the filesystems will be coherent and intact and operational. > -- > Freddie Cash > fjwcash@gmail.com > -- Ben Adams http://www.SpryMed.com/ From owner-freebsd-performance@FreeBSD.ORG Thu Nov 17 20:37:31 2011 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D3BAB106566C for ; Thu, 17 Nov 2011 20:37:31 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 502FF8FC08 for ; Thu, 17 Nov 2011 20:37:31 +0000 (UTC) Received: by vbbfa15 with SMTP id fa15so26285vbb.13 for ; Thu, 17 Nov 2011 12:37:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Z1AWkAJnlKPXPoUhIkxChZjNk6Iy2vUK21E7KDppcdY=; b=qq2fUDaDYmL/61QBOuGtCwXIv6k2VxSZzqT7/bs3sZr2Qjm/MUtiJP6JOPjgS+9lIU OaPAqhYMiqWKNmxvKT+rrsKfiHxK+uE7zkmN+DfXnB9JEO/GORZ3cGPVG8DTEYLkX0Mb KwNj9TrOU2KMT5ekpbiidgE2LWLeG2SzkM8IA= MIME-Version: 1.0 Received: by 10.52.22.226 with SMTP id h2mr237858vdf.46.1321562250486; Thu, 17 Nov 2011 12:37:30 -0800 (PST) Received: by 10.220.190.71 with HTTP; Thu, 17 Nov 2011 12:37:30 -0800 (PST) In-Reply-To: References: Date: Thu, 17 Nov 2011 12:37:30 -0800 Message-ID: From: Freddie Cash To: Mark Felder Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-performance@freebsd.org Subject: Re: ZFS Few Questions X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Nov 2011 20:37:32 -0000 On Thu, Nov 17, 2011 at 11:51 AM, Mark Felder wrote: > On Thu, 17 Nov 2011 12:58:32 -0600, Freddie Cash > wrote: > > Question 2: >>> >>> FreeBSD 9.0 installable on ZFS root? >>> >>> Yes. >> > > This is now an option in the installer? It wasn't last time I checked, > but that was a while ago and last I heard it wasn't going to make it in... > I'll have to fire up the latest RC and look. > I don't know if it's an option in the installer's disk formatter or not (I don't use ZFS-on-root). But the install CD can be used to create a ZFS-on-root installation (or any other disk setup you want). You just drop to the shell in the disk formatter, create the pool, create the filesystems, mount them to where the installer tells you (there's a message at the top when you drop to the shell), then exit the shell and carry on. Or something along those lines. I've never done a ZFS-on-root install; I prefer to keep the / and /usr filesystems on separate gmirror'd disks. Then put the user data (/home, /usr/local and a few other things like /usr/obj, /usr/ports) onto the storage pool. I find that setup gives your better tools for diagnosing problems with the pool and booting and whatnot. Once Boot Environment-type support is added to ZFS on FreeBSD, that may change. :) -- Freddie Cash fjwcash@gmail.com From owner-freebsd-performance@FreeBSD.ORG Thu Nov 17 21:36:53 2011 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 58F67106566C for ; Thu, 17 Nov 2011 21:36:53 +0000 (UTC) (envelope-from m.seaman@infracaninophile.co.uk) Received: from smtp.infracaninophile.co.uk (smtp6.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3fd3:cd67:fafa:3d78]) by mx1.freebsd.org (Postfix) with ESMTP id B781F8FC16 for ; Thu, 17 Nov 2011 21:36:52 +0000 (UTC) Received: from seedling.black-earth.co.uk (seedling.black-earth.co.uk [81.187.76.163]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.14.5/8.14.5) with ESMTP id pAHLahPS035302 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Thu, 17 Nov 2011 21:36:44 GMT (envelope-from m.seaman@infracaninophile.co.uk) X-DKIM: OpenDKIM Filter v2.4.1 smtp.infracaninophile.co.uk pAHLahPS035302 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=infracaninophile.co.uk; s=201001-infracaninophile; t=1321565804; bh=0I24T5EtCq4VScKEBf8pwkGcWe3wAwoxIcHXArCmajA=; h=Message-ID:Date:From:MIME-Version:To:Subject:References: In-Reply-To:Content-Type:Cc; b=p9ry2+cX/gdi5nuR2Qlc1fH1a1sDOU2mJ4BEder+rtDso9Gf8tc/3DhhnUaigd8uM rkiPWYdFIONe8UWR4Q7lv0RsKKQXqPg6Fz3/mRCsuhvG952NJAnALzUaV8Z9LSz0xS fayo5hJ0jyLv6PDH3IuATJE4LImZp+n8n4Tcu1qg= Message-ID: <4EC57E62.9000007@infracaninophile.co.uk> Date: Thu, 17 Nov 2011 21:36:34 +0000 From: Matthew Seaman User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 To: freebsd-performance@freebsd.org References: In-Reply-To: X-Enigmail-Version: 1.3.3 OpenPGP: id=60AE908C Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig41691E48561C4FB5B2B4712F" X-Virus-Scanned: clamav-milter 0.97.3 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-0.6 required=5.0 tests=BAYES_05,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,SPF_FAIL autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lucid-nonsense.infracaninophile.co.uk Subject: Re: ZFS Few Questions X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Nov 2011 21:36:53 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig41691E48561C4FB5B2B4712F Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 17/11/2011 19:04, Mark Felder wrote: >> Question 3: >> Anyone Recommend for MySQL server? (Performance) >=20 > No idea; I haven't run any SQL servers on ZFS The sort of randomly located small IOs that RDBMSes do is the hardest sort of IO pattern for ZFS (or any filesystem for that matter) to manage. ZFS has a particular problem in that its default storage unit is a 128kB block -- and the copy-on-write semantics mean that the filesystem layer can in principle end up doing a 128kB read, altering a few bytes, then doing a 128kB write to get that data back on disk. You can get pretty reasonable DB performance on ZFS, but it takes quite a bit of tuning. * ZFS needs plenty of RAM. The DB needs plenty of RAM. Exactly what the balance should be is hard to predict -- dependent on specific workloads -- so expect to spend some time benchmarking and experimenting with different settings. * Putting the ARC (Adaptive Replacement Cache) on a separate, fast device will make a big difference to performance. SSD cards are popular for this purpose. (Be aware though that SSDs have a limited lifetime, and tend to fail suddenly and completely when they do wear out. You will need multiple layers of resilience and very good backups...) While SSD cards are intrinsically faster than individual rotating magnetic media, they are no match for a large disk array that can spread the IO over lots of spindles. But that costs a very great deal of money... * Reducing the ZFS block size (the recordsize property when creating a zfs) to match the IO size of your DB system can help a lot. Do this before creating the database. * Separating the DB's data and transaction logging onto separate ZFS pools helps. See http://www.solarisinternals.com/wiki/index.php/ZFS_for_Databases for more details. Just about everything on that page applies equally to FreeBSD as it does to Solaris. Cheers, Matthew --=20 Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate JID: matthew@infracaninophile.co.uk Kent, CT11 9PW --------------enig41691E48561C4FB5B2B4712F Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.16 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk7FfmsACgkQ8Mjk52CukIwIHQCeIMrBAvwSsPKGgtdkFBB54VyM d40AniX0/CDfvD3T1CR1/IFMOJNH9S3E =5nYt -----END PGP SIGNATURE----- --------------enig41691E48561C4FB5B2B4712F-- From owner-freebsd-performance@FreeBSD.ORG Fri Nov 18 07:54:51 2011 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 412621065672 for ; Fri, 18 Nov 2011 07:54:51 +0000 (UTC) (envelope-from nec556@retena.com) Received: from resmaa13.ono.com (smtp13.ono.com [62.42.230.16]) by mx1.freebsd.org (Postfix) with ESMTP id 05B078FC0A for ; Fri, 18 Nov 2011 07:54:49 +0000 (UTC) Received: from GogPortatil.retena.com (95.20.141.54) by resmaa13.ono.com (8.5.113) (authenticated as nec556@retena.com) id 4EB89924002646DC; Fri, 18 Nov 2011 08:54:48 +0100 Message-ID: <4EB89924002646DC@> (added by postmaster@resmaa13.ono.com) X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Fri, 18 Nov 2011 08:54:45 +0100 To: "list, mailing" From: Eduardo Morras In-Reply-To: References: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Antivirus: AVG for E-mail 10.0.1411 [2092/4022] Cc: freebsd-performance@freebsd.org Subject: Re: ZFS Few Questions X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Nov 2011 07:54:51 -0000 At 19:26 17/11/2011, you wrote: >Hello everyone I just had a few questions about ZFS. > >Question 3: > >Anyone Recommend for MySQL server? (Performance) A short comment about databases. With ZFS and similar file systems, when you make a snapshot the file system is recoverable/congruent/ok, but the database files may be in inconsistent state. Nothing saves you from a backup. From owner-freebsd-performance@FreeBSD.ORG Fri Nov 18 10:40:56 2011 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2CC2C106564A for ; Fri, 18 Nov 2011 10:40:56 +0000 (UTC) (envelope-from nec556@retena.com) Received: from resmaa12.ono.com (smtp12.ono.com [62.42.230.20]) by mx1.freebsd.org (Postfix) with ESMTP id B154D8FC17 for ; Fri, 18 Nov 2011 10:40:55 +0000 (UTC) Received: from GogPortatil.retena.com (95.20.141.54) by resmaa12.ono.com (8.5.113) (authenticated as nec556@retena.com) id 4EB899200026A45B for freebsd-performance@freebsd.org; Fri, 18 Nov 2011 11:40:53 +0100 Message-ID: <4EB899200026A45B@> (added by postmaster@resmaa12.ono.com) X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Fri, 18 Nov 2011 11:40:49 +0100 To: freebsd-performance@freebsd.org From: Eduardo Morras In-Reply-To: References: <4ec60f6b.893a440a.4ce1.ffffa30dSMTPIN_ADDED@mx.google.com> Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1"; format=flowed Content-Transfer-Encoding: quoted-printable X-Antivirus: AVG for E-mail 10.0.1411 [2092/4022] Subject: Re: ZFS Few Questions X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Nov 2011 10:40:56 -0000 At 11:33 18/11/2011, Istv=C3=A1n wrote: >No problem for the most of the MySQL DBAs=20 >because consistency is not in their dictionary anyway :) I mean that surely db will be corrupted and=20 nothing could be recovered. I know postgresql and=20 there you have a begin snapshot - end snapshot=20 for this topic, data changes are stored in=20 temporal archives and main db files are=20 consistent, allowing you to make a filesystem snapshot. For mysql don't= know.=20 From owner-freebsd-performance@FreeBSD.ORG Fri Nov 18 10:55:26 2011 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E867C106566B for ; Fri, 18 Nov 2011 10:55:25 +0000 (UTC) (envelope-from leccine@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id A596D8FC12 for ; Fri, 18 Nov 2011 10:55:25 +0000 (UTC) Received: by vcbfl11 with SMTP id fl11so304559vcb.13 for ; Fri, 18 Nov 2011 02:55:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=qxCLwJFrNo5GTOy0vIZ2OkyxuilOscOwMmz4RjefXsQ=; b=K2e5GizOgkN/Ul/+g0fm9iK0qrUkUDHPywS8dEdi05nGZVBTlDJX3365F8lTgYNfsd VTf6rTtIp0r3MjFgS2S+A8buFf/mbfq1R/sf2/ji20cPOCQh7K1Wze7tiKRVxFZ4U2du waRVXKRmvS1KHIUkfMztxnJG3qy2teIStj/a8= MIME-Version: 1.0 Received: by 10.220.189.137 with SMTP id de9mr173371vcb.171.1321612411621; Fri, 18 Nov 2011 02:33:31 -0800 (PST) Received: by 10.52.29.164 with HTTP; Fri, 18 Nov 2011 02:33:31 -0800 (PST) In-Reply-To: <4ec60f6b.893a440a.4ce1.ffffa30dSMTPIN_ADDED@mx.google.com> References: <4ec60f6b.893a440a.4ce1.ffffa30dSMTPIN_ADDED@mx.google.com> Date: Fri, 18 Nov 2011 10:33:31 +0000 Message-ID: From: =?UTF-8?Q?Istv=C3=A1n?= To: Eduardo Morras Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "list, mailing" , freebsd-performance@freebsd.org Subject: Re: ZFS Few Questions X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Nov 2011 10:55:26 -0000 No problem for the most of the MySQL DBAs because consistency is not in their dictionary anyway :) On Fri, Nov 18, 2011 at 7:54 AM, Eduardo Morras wrote: > At 19:26 17/11/2011, you wrote: > >> Hello everyone I just had a few questions about ZFS. >> >> Question 3: >> >> Anyone Recommend for MySQL server? (Performance) >> > > A short comment about databases. With ZFS and similar file systems, when > you make a snapshot the file system is recoverable/congruent/ok, but the > database files may be in inconsistent state. Nothing saves you from a > backup. > > ______________________________**_________________ > freebsd-performance@freebsd.**org mailing list > http://lists.freebsd.org/**mailman/listinfo/freebsd-**performance > To unsubscribe, send any mail to "freebsd-performance-** > unsubscribe@freebsd.org " > -- the sun shines for all From owner-freebsd-performance@FreeBSD.ORG Fri Nov 18 12:19:09 2011 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DCCCE106564A for ; Fri, 18 Nov 2011 12:19:08 +0000 (UTC) (envelope-from tevans.uk@googlemail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 95A808FC16 for ; Fri, 18 Nov 2011 12:19:08 +0000 (UTC) Received: by vcbfl11 with SMTP id fl11so433843vcb.13 for ; Fri, 18 Nov 2011 04:19:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=Zdr2xwV7BvFzj0ne6mbvsA+aVC5YAjZt+c+F8JlwWPk=; b=qwZ8GadiE9nNEXU91IlP6n7ChITDhjIzPhDCIaQ32zJm2PJHRIlpZbUdcrxbbeV5S1 zVFj+0cckmBRgh7JevrkQ/f/ChRJiWylrZaiPMA4GIEmJ5GZvM7HGipqo9iGO9xZrXKn RkbhMXVCb3QdkjMFE+HsJm3Z+CzbakkYD55Xg= MIME-Version: 1.0 Received: by 10.52.93.146 with SMTP id cu18mr3159069vdb.56.1321618747834; Fri, 18 Nov 2011 04:19:07 -0800 (PST) Received: by 10.52.182.40 with HTTP; Fri, 18 Nov 2011 04:19:07 -0800 (PST) In-Reply-To: <4EC57E62.9000007@infracaninophile.co.uk> References: <4EC57E62.9000007@infracaninophile.co.uk> Date: Fri, 18 Nov 2011 12:19:07 +0000 Message-ID: From: Tom Evans To: Matthew Seaman Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-performance@freebsd.org Subject: Re: ZFS Few Questions X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Nov 2011 12:19:09 -0000 On Thu, Nov 17, 2011 at 9:36 PM, Matthew Seaman wrote: > On 17/11/2011 19:04, Mark Felder wrote: >>> Question 3: >>> Anyone Recommend for MySQL server? (Performance) >> >> No idea; I haven't run any SQL servers on ZFS > > The sort of randomly located small IOs that RDBMSes do is the hardest > sort of IO pattern for ZFS (or any filesystem for that matter) to > manage. =C2=A0ZFS has a particular problem in that its default storage un= it > is a 128kB block -- and the copy-on-write semantics mean that the > filesystem layer can in principle end up doing a 128kB read, altering a > few bytes, then doing a 128kB write to get that data back on disk. > > You can get pretty reasonable DB performance on ZFS, but it takes quite > a bit of tuning. > > =C2=A0 * ZFS needs plenty of RAM. =C2=A0The DB needs plenty of RAM. =C2= =A0Exactly > =C2=A0 =C2=A0 what the balance should be is hard to predict -- dependent = on > =C2=A0 =C2=A0 specific workloads -- so expect to spend some time benchmar= king > =C2=A0 =C2=A0 and experimenting with different settings. > > =C2=A0 * Putting the ARC (Adaptive Replacement Cache) on a separate, fast > =C2=A0 =C2=A0 device will make a big difference to performance. =C2=A0SSD= cards are > =C2=A0 =C2=A0 popular for this purpose. =C2=A0(Be aware though that SSDs = have a > =C2=A0 =C2=A0 limited lifetime, and tend to fail suddenly and completely = when > =C2=A0 =C2=A0 they do wear out. =C2=A0You will need multiple layers of re= silience and > =C2=A0 =C2=A0 very good backups...) =C2=A0While SSD cards are intrinsical= ly faster > =C2=A0 =C2=A0 than individual rotating magnetic media, they are no match = for a > =C2=A0 =C2=A0 large disk array that can spread the IO over lots of spindl= es. > =C2=A0 =C2=A0 But that costs a very great deal of money... > > =C2=A0 * Reducing the ZFS block size (the recordsize property when creati= ng > =C2=A0 =C2=A0 a zfs) to match the IO size of your DB system can help a lo= t. =C2=A0Do > =C2=A0 =C2=A0 this before creating the database. > > =C2=A0 * Separating the DB's data and transaction logging onto separate Z= FS > =C2=A0 =C2=A0 pools helps. > > See http://www.solarisinternals.com/wiki/index.php/ZFS_for_Databases for > more details. =C2=A0Just about everything on that page applies equally to > FreeBSD as it does to Solaris. > > =C2=A0 =C2=A0 =C2=A0 =C2=A0Cheers, > > =C2=A0 =C2=A0 =C2=A0 =C2=A0Matthew > If you are running a write heavy database, in addition to what Matthew has said, you will definitely want a separate ZIL from your pool. To speed up reads, you will want to allocate as much to ARC as you can spare from your applications. L1 ARC is RAM; set vfs.zfs.arc_max in loader.conf to control the maximum amount of RAM you want to use. L2 ARC is optional, to add it you need to add cache devices to your pool. You can lose the L2 ARC from the pool without side effects, so just add some ssds like so: zpool tank add cache ada0 ada2 To speed up synchronous writes, you need to add a dedicated ZFS Intent Log (ZIL). If you don't specify a separate ZIL, then part of the pool is used to be the ZIL. Some versions of ZFS would complain loudly (panic) if the ZIL disappeared, I think in 9.0 it does not, but you should use a mirror anyway: zpool tank add log mirror ada1 ada3 Rather than adding extra drives, you can use PCIe SSD plugin cards, which are super fast. The ones we use present two drives per device rather than one, we put two cards in each machine, and we use one drive on each device for L2ARC and ZIL. It's only in testing so far - we're waiting for 8.3 to be released - but it works nicely. Cheers Tom From owner-freebsd-performance@FreeBSD.ORG Fri Nov 18 12:22:21 2011 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F330E1065670 for ; Fri, 18 Nov 2011 12:22:21 +0000 (UTC) (envelope-from tevans.uk@googlemail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id B22318FC12 for ; Fri, 18 Nov 2011 12:22:21 +0000 (UTC) Received: by vbbfa15 with SMTP id fa15so209373vbb.13 for ; Fri, 18 Nov 2011 04:22:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=x0IG38v6p1YOQwQUZ4sowN56bQ4CuHvZjLbQrCi8gpk=; b=rKEZFO6eISuMWQ4aC1NKodPvUrAK5ho2rbrUxv/K8WgUKufqWGaSs1+cIHwJ8fCX8Z 6Zj73+0rqppXJ102jFX9fg6yuJkF5KzVkQPRwnTTggH5ua3O0QjJSBNx41VXdGz+L0bH 7hUbcptEHygb241stQxaU+ZlWcg9Llp6HYCKU= MIME-Version: 1.0 Received: by 10.52.89.206 with SMTP id bq14mr3186083vdb.39.1321618940916; Fri, 18 Nov 2011 04:22:20 -0800 (PST) Received: by 10.52.182.40 with HTTP; Fri, 18 Nov 2011 04:22:20 -0800 (PST) In-Reply-To: <4ec63674.03d5960a.7701.58bfSMTPIN_ADDED@mx.google.com> References: <4ec60f6b.893a440a.4ce1.ffffa30dSMTPIN_ADDED@mx.google.com> <4ec63674.03d5960a.7701.58bfSMTPIN_ADDED@mx.google.com> Date: Fri, 18 Nov 2011 12:22:20 +0000 Message-ID: From: Tom Evans To: Eduardo Morras Content-Type: text/plain; charset=UTF-8 Cc: freebsd-performance@freebsd.org Subject: Re: ZFS Few Questions X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Nov 2011 12:22:22 -0000 On Fri, Nov 18, 2011 at 10:40 AM, Eduardo Morras wrote: > I mean that surely db will be corrupted and nothing could be recovered. I > know postgresql and there you have a begin snapshot - end snapshot for this > topic, data changes are stored in temporal archives and main db files are > consistent, allowing you to make a filesystem snapshot. For mysql don't > know. You can do similar thing with MySQL - 'FLUSH TABLES WITH READ LOCK', zfs snapshot, release lock, job done. It's not as good as postgresql, which doesn't require the tables to be locked for writes. I would imagine as ZFS and MySQL are both owned by Oracle the situation will improve. Cheers Tom