From owner-freebsd-fs@FreeBSD.ORG Sat Feb 27 08:23:19 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5527C1065673 for ; Sat, 27 Feb 2010 08:23:19 +0000 (UTC) (envelope-from h.schmalzbauer@omnilan.de) Received: from host.omnilan.net (host.omnilan.net [62.245.232.135]) by mx1.freebsd.org (Postfix) with ESMTP id B227B8FC18 for ; Sat, 27 Feb 2010 08:23:18 +0000 (UTC) Received: from titan.flintsbach.schmalzbauer.de (titan.flintsbach.schmalzbauer.de [172.21.1.150]) (authenticated bits=0) by host.omnilan.net (8.13.8/8.13.8) with ESMTP id o1R8NGsm031895 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 27 Feb 2010 09:23:16 +0100 (CET) (envelope-from h.schmalzbauer@omnilan.de) Message-ID: <4B88D66E.9040609@omnilan.de> Date: Sat, 27 Feb 2010 09:23:10 +0100 From: Harald Schmalzbauer Organization: OmniLAN User-Agent: Thunderbird 2.0.0.23 (X11/20090906) MIME-Version: 1.0 To: "Sam Fourman Jr." References: <4B8649F4.4070303@omnilan.de> <1C136B0A-E98F-4F93-8480-7FACD3FE4E4B@gmail.com> <11167f521002250358i46a09fb6j82aaddbd8e0b8334@mail.gmail.com> In-Reply-To: <11167f521002250358i46a09fb6j82aaddbd8e0b8334@mail.gmail.com> X-Enigmail-Version: 0.95.6 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig654D26BA0DFD027D19D9F936" Cc: freebsd-fs@freebsd.org Subject: Re: ZFS write stalls (starving reads) and tuning zfs_write_limit_override X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 08:23:19 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig654D26BA0DFD027D19D9F936 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable Sam Fourman Jr. schrieb am 25.02.2010 12:58 (localtime): >> Hi, >> >> What I posted was not a patch, but just the lines need to be added to = dsl_pool.c and it should work for either 8 or CURRENT. >> Here is a patch for 8-STABLE that I just generated, can you try that o= ne (pasting in the email in case the email eats the attachment >> ): >> >> --- .zfs/snapshot/orig/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/= dsl_pool.c 2009-08-24 07:30:23.677549074 +0300 >> +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_pool.c 2010-0= 2-19 09:17:50.058020997 +0200 >> @@ -47,6 +47,11 @@ >> uint64_t zfs_write_limit_override =3D 0; >> extern uint64_t zfs_write_limit_min; >> >> +SYSCTL_DECL(_vfs_zfs); >> +TUNABLE_ULONG("vfs.zfs.write_limit_override", &zfs_write_limit_overri= de); >> +SYSCTL_ULONG(_vfs_zfs, OID_AUTO, zfs_write_limit_override, CTLFLAG_RW= , &zfs_write_limit_override, 0, >> + "Override maximum TXG size"); >> + >> kmutex_t zfs_write_limit_lock; >> >> static pgcnt_t old_physmem =3D 0; >> >=20 >=20 > what setting have you found to be the best for zfs_write_limit_override= Unfortunately I couldn't find a sensible value. 250M was the best=20 compromise. But this reduced my avarage rsync rate to 30MB/s. Without zfs_write_limit_override and vfs.zfs.txg.timeout=3D1 I can get=20 65MB/s average, even with the mentioned pauses (peak is almost 90MB/s). Today I'm aware that 5400rpm spindles are not well suitable for ZFS, but = for big backup servers I'd take them again. If we could just find a way=20 to throttle ZFS _a bit_. Three 5400rpms should surely manage more than=20 30MB/s. Eventually I can transfer 65MB/s. Thanks, -Harry --------------enig654D26BA0DFD027D19D9F936 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.13 (FreeBSD) iEYEARECAAYFAkuI1nQACgkQLDqVQ9VXb8j/YACgmIa6Wu8eG7Y+k/StFfoSFGJ6 XVkAnRFzLQP6yghaibBznwK4+gNE5WPb =7vuk -----END PGP SIGNATURE----- --------------enig654D26BA0DFD027D19D9F936--