From owner-freebsd-stable@FreeBSD.ORG  Sun Jan  3 20:05:50 2010
Return-Path: <owner-freebsd-stable@FreeBSD.ORG>
Delivered-To: freebsd-stable@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id CAECB1065697
	for <freebsd-stable@freebsd.org>; Sun,  3 Jan 2010 20:05:50 +0000 (UTC)
	(envelope-from rmtodd@ichotolot.servalan.com)
Received: from mx1.synetsystems.com (mx1.synetsystems.com [76.10.206.14])
	by mx1.freebsd.org (Postfix) with ESMTP id A35B38FC1B
	for <freebsd-stable@freebsd.org>; Sun,  3 Jan 2010 20:05:50 +0000 (UTC)
Received: by mx1.synetsystems.com (Postfix, from userid 66)
	id 19715E41; Sun,  3 Jan 2010 14:45:16 -0500 (EST)
Received: from rmtodd by servalan.servalan.com with local (Exim 4.71 (FreeBSD))
	(envelope-from <rmtodd@ichotolot.servalan.com>) id 1NRVSJ-000CLZ-99
	for freebsd-stable@freebsd.org; Sun, 03 Jan 2010 12:45:35 -0600
To: freebsd-stable@freebsd.org
References: <7346c5c61001030842r7dc76199y51e4c1c90a3eea6e@mail.gmail.com>
From: Richard Todd <rmtodd@ichotolot.servalan.com>
Date: Sun, 03 Jan 2010 12:45:34 -0600
In-Reply-To: <servalan.mailinglist.fbsd-stable/7346c5c61001030842r7dc76199y51e4c1c90a3eea6e@mail.gmail.com>
	(Garrett Moore's message of "Sun, 3 Jan 2010 11:42:14 -0500")
Message-ID: <x7eim7owmp.fsf@ichotolot.servalan.com>
User-Agent: Gnus/5.1008 (Gnus v5.10.8) XEmacs/21.5-b28 (berkeley-unix)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Subject: Re: ZFS performance degradation over time
X-BeenThere: freebsd-stable@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Production branch of FreeBSD source code <freebsd-stable.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-stable>, 
	<mailto:freebsd-stable-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-stable>
List-Post: <mailto:freebsd-stable@freebsd.org>
List-Help: <mailto:freebsd-stable-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-stable>,
	<mailto:freebsd-stable-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sun, 03 Jan 2010 20:05:50 -0000

Garrett Moore <garrettmoore@gmail.com> writes:

> I'm having problems with ZFS performance. When my system comes up,
> read/write speeds are excellent (testing with dd if=/dev/zero
> of=/tank/bigfile and dd if=/tank/bigfile of=/dev/null); I get at least
> 100MB/s on both reads and writes, and I'm happy with that.
>
> The longer the system is up, the worse my performance gets. Currently my
> system has been up for 4 days, and read/write performance is down to about
> 10MB/s at best.
>
> The system is only accessed by 3 clients: myself, my roommate, and our HTPC.
> Usually, only one client will be doing anything at a time, so it is not
> under heavy load or anything.

This could be due to the amount of memory available for ZFS caching declining
as time goes on (for reasons that are not entirely clear, though I suspect
it may be due to increasing fragmentation in the kernel memory).  You might
try doing "sysctl kstat.zfs.misc.arcstats.size" repeatedly to monitor the
amount of memory the ZFS cache is taking up as your system uptime increases.