From owner-freebsd-fs@FreeBSD.ORG Thu Feb 25 12:02:00 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4913510656A4 for ; Thu, 25 Feb 2010 12:02:00 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from mail-fx0-f223.google.com (mail-fx0-f223.google.com [209.85.220.223]) by mx1.freebsd.org (Postfix) with ESMTP id C88D38FC1F for ; Thu, 25 Feb 2010 12:01:59 +0000 (UTC) Received: by fxm23 with SMTP id 23so45402fxm.3 for ; Thu, 25 Feb 2010 04:01:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:subject:mime-version :content-type:from:in-reply-to:date:cc:content-transfer-encoding :message-id:references:to:x-mailer; bh=ODBUrvaTjdlo/87EtwBduDCeLL+AhxL0ytnYg+2r864=; b=sqUezwvNgpN8Rzv+MllDQVULr597+3K18niawJi9AHKLDGGnVfFMB4OW+LPzXCkq6A 7HnNNVwfaCk4lpk5dRjQEHLkLypwrW4vlhupZCyV+1DrFtYZAVcPFW+e13S2HXLOcEwO 31oyRFI0i0J6wmkhRQi3AU/jK9kb9y+173Se0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; b=QMHX53JVKu68jO0eIpbFphiHCizuEzYgYQBvt7XUZug3whvv8WKoooc6R7X5F0r7nG vWMhNJLbeGyd0FktMPkZER4d5ma/EtUJ6uWSLOTrciRA8gkeVeg1ze065cc8CHZ3Qwo0 X9ASP7XvVya4znhbLmoNMdpj7Nygl60ZTsDmc= Received: by 10.102.196.33 with SMTP id t33mr808861muf.2.1267099294664; Thu, 25 Feb 2010 04:01:34 -0800 (PST) Received: from ?10.32.23.105? ([195.34.111.178]) by mx.google.com with ESMTPS id w5sm13943072mue.22.2010.02.25.04.01.33 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 25 Feb 2010 04:01:33 -0800 (PST) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Nikolay Denev In-Reply-To: <11167f521002250358i46a09fb6j82aaddbd8e0b8334@mail.gmail.com> Date: Thu, 25 Feb 2010 14:01:32 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <72D31CA4-110E-4A85-AA19-BD74AFFC94B1@gmail.com> References: <4B8649F4.4070303@omnilan.de> <1C136B0A-E98F-4F93-8480-7FACD3FE4E4B@gmail.com> <11167f521002250358i46a09fb6j82aaddbd8e0b8334@mail.gmail.com> To: Sam Fourman Jr. X-Mailer: Apple Mail (2.1077) Cc: freebsd-fs@freebsd.org Subject: Re: ZFS write stalls (starving reads) and tuning zfs_write_limit_override X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 12:02:00 -0000 On Feb 25, 2010, at 1:58 PM, Sam Fourman Jr. wrote: >> Hi, >>=20 >> What I posted was not a patch, but just the lines need to be added to = dsl_pool.c and it should work for either 8 or CURRENT. >> Here is a patch for 8-STABLE that I just generated, can you try that = one (pasting in the email in case the email eats the attachment >> ): >>=20 >> --- = .zfs/snapshot/orig/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_pool= .c 2009-08-24 07:30:23.677549074 +0300 >> +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_pool.c = 2010-02-19 09:17:50.058020997 +0200 >> @@ -47,6 +47,11 @@ >> uint64_t zfs_write_limit_override =3D 0; >> extern uint64_t zfs_write_limit_min; >>=20 >> +SYSCTL_DECL(_vfs_zfs); >> +TUNABLE_ULONG("vfs.zfs.write_limit_override", = &zfs_write_limit_override); >> +SYSCTL_ULONG(_vfs_zfs, OID_AUTO, zfs_write_limit_override, = CTLFLAG_RW, &zfs_write_limit_override, 0, >> + "Override maximum TXG size"); >> + >> kmutex_t zfs_write_limit_lock; >>=20 >> static pgcnt_t old_physmem =3D 0; >>=20 >=20 >=20 > what setting have you found to be the best for = zfs_write_limit_override ? >=20 > Sam Fourman Jr. > Fourman Networks Hi, I think you should experiment. Here I have 2G of ram, and have found = 256M for the zfs_write_limit_override to provide smooth writes without the stalls.=20 I started one console with zpool iostat -v 1 and started writing to one = iSCSI exported zvol, and played with the sysctl until the writes were steady. Regards, Niki Denev