From owner-freebsd-fs@FreeBSD.ORG Tue May 8 19:48:51 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 266B51065672 for ; Tue, 8 May 2012 19:48:51 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-qa0-f47.google.com (mail-qa0-f47.google.com [209.85.216.47]) by mx1.freebsd.org (Postfix) with ESMTP id CFE7B8FC1C for ; Tue, 8 May 2012 19:48:50 +0000 (UTC) Received: by qabg1 with SMTP id g1so1039569qab.13 for ; Tue, 08 May 2012 12:48:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; bh=KUrRoDLZSrl13CIwO7rk7VD7Z6w0lDhdLW1hcxxlLk8=; b=FjxacbXySHdP+5tvSEk0m5EdZj2JW7HXwsrP7MLyQlwCZxJTjVSW8Xmg29wvFEQdHz AK68g54ecPunp7FKALlhASNXHFGdADzFjifoXoU+65YuLIG+zR7jO1nNh5gczMeHqMiM QPSCyxvLf/Sqm3EsnlFjASdA5xBdoIAuuV6m4eMaUz+T1GbTktMNSx88H31VadjH8JAa 8HCd8WwH7ruBYT1Pm4C/s+JVwBT4FF5YGzils6PZ4P9B6doBBd8bei/rN3FSIKivOZZi jbuKzGihV/S52wWJXtEP07FGi6Hr2E/Q7/U6MfZLXjyLIoVvAChw/Ay0UiikyRYkCQzo IsYw== MIME-Version: 1.0 Received: by 10.224.73.1 with SMTP id o1mr619844qaj.43.1336506529316; Tue, 08 May 2012 12:48:49 -0700 (PDT) Received: by 10.229.224.147 with HTTP; Tue, 8 May 2012 12:48:49 -0700 (PDT) In-Reply-To: References: Date: Tue, 8 May 2012 12:48:49 -0700 Message-ID: From: Freddie Cash To: FreeBSD Filesystems Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: Broken ZFS filesystem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 May 2012 19:48:51 -0000 On Tue, May 8, 2012 at 10:23 AM, Freddie Cash wrote: > I have an interesting issue with one single ZFS filesystem in a pool. > All the other filesystems are fine, and can be mounted, snapshoted, > destroyed, etc. =C2=A0But this one filesystem, if I try to do any operati= on > on it (zfs mount, zfs snapshot, zfs destroy, zfs set ), it > spins the system until all RAM is used up (wired), and then hangs the > box. =C2=A0The zfs process sits in tx -> tx_sync_done_cv state until the > box locks up. =C2=A0CTRL+T of the process only ever shows this: > =C2=A0 =C2=A0load: 0.46 =C2=A0cmd: zfs 3115 [tx->tx_sync_done_cv)] 36.63r= 0.00u 0.00s 0% 2440k > > Anyone come across anything similar? =C2=A0And found a way to fix it, or = to > destroy the filesystem? =C2=A0Any suggestions on how to go about debuggin= g > this? =C2=A0Any magical zdb commands to use? > > The filesystem only has 5 MB of data in it (log files), compressed via > LZJB for a compressratio of ~6x. =C2=A0There are no snapshots for this > filesystem. > > Dedupe is enabled on the pool and all filesystems. After more fiddling, testing, and experimenting, it all came down to not enough RAM in the box to mount the 5 MB filesystem. After installing an extra 8 GB of RAM (32 GB total), everything mounted correctly. Took 27 GB of wired kernel memory (guessing ARC space) to do it. Unmount, mount, export, import, change properties all completed successfully. And the box is running correctly with 24 GB of RAM again. We'll be ordering more RAM for our ZFS boxes, now. :) --=20 Freddie Cash fjwcash@gmail.com