From owner-freebsd-questions@FreeBSD.ORG Wed Oct 5 22:37:19 2011 Return-Path: Delivered-To: questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4E9B7106564A; Wed, 5 Oct 2011 22:37:19 +0000 (UTC) (envelope-from syshackmin@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id B41FC8FC1A; Wed, 5 Oct 2011 22:37:18 +0000 (UTC) Received: by wyj26 with SMTP id 26so3003258wyj.13 for ; Wed, 05 Oct 2011 15:37:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=s0DbMPy1RBojdkNBPHxPxzrv5lKM7//GFFazhCMQo24=; b=L0AzshxtbCF5hMtOr3Sy76HYNY/MFeZYx4ivjhMJpLrBuVwRr8HtcrkqrymFTYEAEJ imvu4kihlGijebmXmQSOugn8XAifYQf/OjohnYOp3jFwr9aykIYWn9BYQu8dC4uUihgE hfgT/4H/8bvAIpGyS0FQS22mSvwuMieYOyYsw= MIME-Version: 1.0 Received: by 10.216.137.223 with SMTP id y73mr38421wei.6.1317854237513; Wed, 05 Oct 2011 15:37:17 -0700 (PDT) Received: by 10.216.53.21 with HTTP; Wed, 5 Oct 2011 15:37:17 -0700 (PDT) In-Reply-To: References: Date: Wed, 5 Oct 2011 18:37:17 -0400 Message-ID: From: Dave Cundiff To: Daniel Staal Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, questions@freebsd.org Subject: Re: ZFS Write Lockup X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 22:37:19 -0000 On Wed, Oct 5, 2011 at 5:52 PM, Daniel Staal wrote: > --As of October 4, 2011 2:43:45 AM -0400, Dave Cundiff is alleged to have > said: > >> I don't know what triggers the problem but I know how to fix it. If I >> perform a couple snapshot deletes the IO will come back in line every >> single time. Fortunately I have LOTS of snapshots to delete. >> >> [root@san2 ~]# zfs list -r -t snapshot | wc -l >> =A0 =A05236 >> [root@san2 ~]# zfs list -r -t volume | wc -l >> =A0 =A0 =A017 > > --As for the rest, it is mine. > > I have no good advice, but I have a thought. =A0;) > > The thought is: Why so many snapshots? =A0And: How many other people have= that > many snapshots? =A0I know that ZFS is supposed to be able to handle huge > numbers of snapshots (far more than a few thousand, from my understanding= ), > but if it hasn't been used much in that config, there may be bugs lurking= . > > You might try weeding through and figuring out if you can drop a good amo= unt > of those snapshots. =A0Also, try the filesystems list. =A0They may have b= etter > thoughts. > > Daniel T. Staal > Its for a backup service I've been working on. It takes a snapshot hourly of all 17 zvols. I was planning on keeping them for a month. I had the same thought about the snapshots and deleted them all yesterday. It appears there is some issue with keeping that many. I removed them all and the zvols are now functioning correctly. Its strange that the large number didn't cause incremental slowdown. While the snapshots were still there the IO was normal when it wasn't acting up. Just it would have spurts of almost total lockup until I performed a snapshot removal operation or 2. Thanks, --=20 Dave Cundiff System Administrator A2Hosting, Inc http://www.a2hosting.com