From owner-freebsd-fs@FreeBSD.ORG Mon Mar 10 17:40:55 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AFA25112 for ; Mon, 10 Mar 2014 17:40:55 +0000 (UTC) Received: from r2-d2.netlabs.org (r2-d2.netlabs.org [213.238.45.90]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 296948A9 for ; Mon, 10 Mar 2014 17:40:54 +0000 (UTC) Received: (qmail 7103 invoked by uid 89); 10 Mar 2014 17:40:53 -0000 Received: from unknown (HELO eternal.metropolis.netlabs.org) (ml-ktk@netlabs.org@213.144.156.18) by 0 with ESMTPA; 10 Mar 2014 17:40:53 -0000 Message-ID: <531DF924.5030109@netlabs.org> Date: Mon, 10 Mar 2014 18:40:52 +0100 From: Adrian Gschwend User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: Steven Hartland Subject: Re: Reoccuring ZFS performance problems References: <531DF0DD.8070809@netlabs.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Mar 2014 17:40:55 -0000 On 10.03.14 18:31, Steven Hartland wrote: > Looks like you may be out of IOP/s but just incase, are you using TRIM > at all? > sysctl -a |grep trim vfs.zfs.vdev.trim_on_init: 1 vfs.zfs.vdev.trim_max_pending: 64 vfs.zfs.vdev.trim_max_bytes: 2147483648 vfs.zfs.trim.enabled: 1 vfs.zfs.trim.max_interval: 1 vfs.zfs.trim.timeout: 30 vfs.zfs.trim.txg_delay: 32 kstat.zfs.misc.zio_trim.bytes: 0 kstat.zfs.misc.zio_trim.success: 0 kstat.zfs.misc.zio_trim.unsupported: 115 kstat.zfs.misc.zio_trim.failed: 0 so looks like trim is enabled > If you are what does "gstat -d" show? It looks like finally my MySQL process finished and now the system is back to completely fine: dT: 1.010s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d %busy Name 10 203 0 0 0.0 192 1674 38.8 0 0 0.0 95.2| vtbd0 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| vtbd0p1 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| vtbd0p2 10 203 0 0 0.0 192 1674 39.0 0 0 0.0 95.5| vtbd0p3 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| cd0 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| gptid/e402ecce-89ca-11e2-a867-3264262b9894 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| gptid/e4112d88-89ca-11e2-a867-3264262b9894 I restarted MySQL now, curious how long it will take. > You also mention your using mysql, have you applied the standard tuning > for mysql on ZFS? At first I didn't so anything special with MySQL, during the process I redid the MySQL ZFS with a new record size: # zfs get recordsize tank/storage/data/db/data NAME PROPERTY VALUE SOURCE tank/storage/data/db/data recordsize 16K local regards Adrian