From owner-freebsd-questions@freebsd.org Thu Sep 24 13:31:48 2015 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8CC7EA0884A for ; Thu, 24 Sep 2015 13:31:48 +0000 (UTC) (envelope-from paul@kraus-haus.org) Received: from mail-qk0-f175.google.com (mail-qk0-f175.google.com [209.85.220.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 50FB91BFA for ; Thu, 24 Sep 2015 13:31:47 +0000 (UTC) (envelope-from paul@kraus-haus.org) Received: by qkap81 with SMTP id p81so29588833qka.2 for ; Thu, 24 Sep 2015 06:31:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:content-transfer-encoding:message-id:references:to; bh=n0gFCbe+UfZXH0KM1LgS6CrQYJM8q1MV7O8kFccgCcg=; b=MYh116kL/tpqkFrozrrOzCP9vvLBCzJUtfdyqspjmjEcl5/+M+o6JMAO9UAIz0eD/b cYwdj3q/d+GRZTyOkCkaRHXbcVjQhQG9qFkGMsw2LDQjz+SRJNDv87uzUh55K5DYkKgn K5EIIeAgzvXfu9QZ5LnjUGMSVnihF1axdCO9QDzDLLCMXATpfElxMmX/TrljhlWmNo0T QmiP1vzljDeeOwSfxBayhiBCOqOq6rtqrgLp7OesczLeY6NblHFbmD4gUyCbkFho0a3m 7mmit/7MrsoElKda50bh7L5ocvhXXEZkPdIFk4lt3bfcJR28HYwJUyUDnslVvPZJCMQa UztQ== X-Gm-Message-State: ALoCoQk01C37FcXIcJM+poYznETXvsIJGHiQhY7gcsePQoQhO6mU1wW9pmL4OgFjLqaG9c4Fma+n X-Received: by 10.55.40.207 with SMTP id o76mr43285900qko.106.1443101506674; Thu, 24 Sep 2015 06:31:46 -0700 (PDT) Received: from mbp-1.thecreativeadvantage.com (mail.thecreativeadvantage.com. [96.236.20.34]) by smtp.gmail.com with ESMTPSA id e63sm2667735qka.5.2015.09.24.06.31.44 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 24 Sep 2015 06:31:45 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Restructure a ZFS Pool From: Paul Kraus In-Reply-To: <480627999.9462316.1443098561442.JavaMail.zimbra@logitravel.com> Date: Thu, 24 Sep 2015 09:31:42 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <9EE24D9C-260A-408A-A7B5-14BACB12DDA9@kraus-haus.org> References: <480627999.9462316.1443098561442.JavaMail.zimbra@logitravel.com> To: Raimund Sacherer , FreeBSD Questions X-Mailer: Apple Mail (2.1878.6) X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Sep 2015 13:31:48 -0000 On Sep 24, 2015, at 8:42, Raimund Sacherer = wrote: > I had the pool fill up to over 80%, then I got it back to about = 50-60%, but it feels more sluggish. I use a lot of NFS and we use it to = backup some 5 million files in lots of sub-directorys (a/b/c/d/abcd...), = besides other big files (SQL dump backups, bacula, etc.) >=20 > I said above sluggish because I do not have empirical data and I do = not know exactly how to test the system correctly, but I read a lot and = there seem to be suggestions that if you have NFS etc. that a = independent ZIL helps with copy-on-write fragmentation.=20 A SLOG (Separate Log Device) will not remove existing fragmentation, but = it will help prevent future fragmentation _iff_ (if and only if) the = write operations are synchronous. NFS is not, by itself, sync, but the = write calls on the client _may_ be sync. > What I would like to know is if I can eliminate one Spare disk from = the pool, and add it as a ZIL again, without having to shutdown/reboot = the server? Yes, but unless you can stand loosing data in flight (writes that the = system says have been committed but have only made it to the SLOG), you = really want your SLOG vdev to be a mirror (at least 2 drives). > I am also thinking about swapping the spare 4TB disk for a small SSD, = but that's immaterial to whether I can perform the change.=20 I assume you want to swap instead of just add due to lack of open drive = slots / ports. In a zpool of this size, especially a RAIDz zpool, you really want a = hot spare and a notification mechanism so you can replace a failed drive = ASAP. The resilver time (to replace afield drive) will be limited by the = performance of a _single_ drive for _random_ I/O. See this post = http://pk1048.com/zfs-resilver-observations/ for one of my resilver = operations and the performance of such. > Also I would appreciate it if someone has some pointers on how to test = correctly so I see if there are real benefits before/after this = operation. I use a combination of iozone and filebench to test, but first I = characterize my workload. Once I know what my workload looks like I can = adjust the test parameters to match the workload. If the test results do = not agree with observed behavior, then I tune them until they do. = Recently I needed to test a server before going live. I knew the = workload was NFS for storing VM images. So I ran iozone with 8-64 GB = files and 4 KB to 1 MB blocks, and sync writes (the -o option). The = measurements matched very closely to the observations, so I knew could = trust them and any changes I made would give me valid results. -- Paul Kraus paul@kraus-haus.org