From owner-freebsd-fs@FreeBSD.ORG Tue Jul 22 22:22:24 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0C046106564A for ; Tue, 22 Jul 2008 22:22:24 +0000 (UTC) (envelope-from matt@corp.spry.com) Received: from wf-out-1314.google.com (wf-out-1314.google.com [209.85.200.169]) by mx1.freebsd.org (Postfix) with ESMTP id EA4BC8FC18 for ; Tue, 22 Jul 2008 22:22:23 +0000 (UTC) (envelope-from matt@corp.spry.com) Received: by wf-out-1314.google.com with SMTP id 24so1452883wfg.7 for ; Tue, 22 Jul 2008 15:22:23 -0700 (PDT) Received: by 10.142.49.4 with SMTP id w4mr2052907wfw.201.1216765343102; Tue, 22 Jul 2008 15:22:23 -0700 (PDT) Received: from matts.spry.com ( [207.178.4.6]) by mx.google.com with ESMTPS id 24sm6763863wff.17.2008.07.22.15.22.21 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 22 Jul 2008 15:22:22 -0700 (PDT) Message-Id: From: Matt Simerson To: freebsd-fs@freebsd.org In-Reply-To: <48865A68.1010504@quip.cz> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v928.1) Date: Tue, 22 Jul 2008 15:22:17 -0700 References: <5E8D64DE-EC9B-4B11-BCB4-17BA63650BB7@corp.spry.com> <48865A68.1010504@quip.cz> X-Mailer: Apple Mail (2.928.1) Subject: Re: ZFS hang issue and prefetch_disable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jul 2008 22:22:24 -0000 On Jul 22, 2008, at 3:08 PM, Miroslav Lachman wrote: > Matt Simerson wrote: >> Symptoms >> Deadlocks under heavy IO load on the ZFS file system with >> prefetch_disable=0. Setting vfs.zfs.prefetch_disable=1 results in >> a stable system. > > [...] > >> With vfs.zfs.prefetch_disable=1, a hang will occur within a few >> hours (no more than a day). If I keep the i/o load (measured via >> iostat) down to a low level (< 200 iops) then I still get hangs >> but less frequently (1-6 days). The only way I have found to >> prevent the hangs is by setting vfs.zfs.prefetch_disable=1. > > "With vfs.zfs.prefetch_disable=1, a hang will occur within...", did > you realy mean prefetch_disable=1 in this sentence? Your whole e- > mail seems that prefetch_disable=1 is good workaround, so I expect > you have prefetch_disable=0 previously which causes hangs... Aye. That is exactly what I meant. With vfs.zfs.prefetch_disable=1, I get a stable system. With vfs.zfs.prefetch_disable=0 (the default) I have frequent deadlocks. Matt Rant: I really wish that variable wasn't named in the negative, creating a double negative when prefetch_disable=0. IE, it should be named vfs.zfs.prefetch_enable instead. It's much easier to express in English that prefetch_enable=1 means ON and prefetch_enable=0 means OFF. There's also the matter than in some languages, a double or triple negative still means the negative case. %-\. I'd rather not have to guess what prefetch_disable=1 means.