From owner-freebsd-fs@FreeBSD.ORG Wed Oct 10 16:59:06 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 18230A55 for ; Wed, 10 Oct 2012 16:59:06 +0000 (UTC) (envelope-from david@wimsey.us) Received: from mail-yh0-f54.google.com (mail-yh0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id BBC018FC14 for ; Wed, 10 Oct 2012 16:59:05 +0000 (UTC) Received: by mail-yh0-f54.google.com with SMTP id s35so196632yhf.13 for ; Wed, 10 Oct 2012 09:58:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=U3+is0vy/s+XGEn7QMA4IwdMMyd5CXUcv9lvysjmqz4=; b=Mwhcycg3NzLJjheFRrjDoUrqr+ro8qe6o76ThR0E2w0CtlTfB5sv+r+oxRpsHJlXh2 zR7uP75bOslsuLKwEmkx0uaQ5mNfQPpbST0niMRvHSPbuLCVSXvWz4xriESluY1tyEq3 DVrUMGnPGLbolQosqfUJdMtVZcP5esIMEvKFPLJL8FkSDZmRR7hfqnG7faQggnOxeyde cxceA6/xJT+3w3O9KhjxR77GNnWUIksufNue5wkk5FoH7/dehJa2D5pe4F9Ry+7xpHYV mIhLV81a4k6Dr9iHucFDuyXQFyVqTRH9rTp3GgxP+WBx2bRO1OxSKej4lZdl3blL0uCl J4hg== Received: by 10.236.125.133 with SMTP id z5mr23663962yhh.121.1349888339181; Wed, 10 Oct 2012 09:58:59 -0700 (PDT) Received: from [10.27.1.242] (cpe-107-015-155-065.nc.res.rr.com. [107.15.155.65]) by mx.google.com with ESMTPS id o66sm1643298yhi.19.2012.10.10.09.58.58 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 10 Oct 2012 09:58:58 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: Deadlock on zfs import From: David Wimsey In-Reply-To: Date: Wed, 10 Oct 2012 12:58:57 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <99965AED-8B44-49F7-9EF0-2CD16DB2EEC9@wimsey.us> References: <074F3CC1-E29F-4552-840F-A38FDDCC7E76@wimsey.us> To: Freddie Cash X-Mailer: Apple Mail (2.1499) X-Gm-Message-State: ALoCoQn2HPD86xD2JyTDn376u5pa5oHadNjTX99/GvxEDaEhCgIn5gS9GdODpn78nLl5hu5qu/rw Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Oct 2012 16:59:06 -0000 I should have followed up on my original message. I presumed it was a = memory issue as I know everyone says to use more RAM so I went ahead and = put another 4GB in the machine and imported the pool. The first import = took a few minutes as it chewed through the disks for some reason but = it did import and since then I've exported and reimported with normal = response times and the machine functions just fine now. After all the = reading I did I came to the conclusion that my setup was just not ZFS = worthy with only 4GB since I calculated that I needed about 2GB just for = DDT. In the one instance I had top running it didn't end up at 100% in wired, = but it came fairly close before the consoles stopped updating. If = anyone cares enough about the issue to look into it deeper I can pull = the extra RAM out and do it again to be sure but I'm considering the = problem resolved as an ID10T error on the admin's part (me). The only change to the whole thing I would like to see is some sort of = console indication of the problem. I realize the difficulty in doing = things when the kernel itself is out of memory but it seems like it'd be = possible to reserve a little space for outputting a 'Don't do that = dummy' type of message for people like me. Thanks David Wimsey On Oct 10, 2012, at 12:45 PM, Freddie Cash wrote: > If you boot to single-user mode, disable the auto-import of pools > (zfs_enable=3Dno in rc.conf), boot to multi-user, start top in one > terminal, and then manually import the pool in another window, and > watch top out, does it run out of RAM (putting everything into wired)? >=20 > You're running dedupe with only 4 GB of RAM, with pools over 90% full. > My guess is that it's running out of ARC space trying to import the > pool and load the DDT (dedupe table). The above test will show that > to be the case (wired memory goes to 100% and system locks up). >=20 > If that's the case, the only solution I've found is to stick more RAM > into the box. >=20 > If the cause of the "can't import due to running out of ARC" is that > you were destroying a ZFS filesystem that had dedupe enabled, then you > can try upgrading to 9-STABLE and add the "zfs features" patch that > was posted here last week. That will run the "zfs destroy" process in > the background and allow the pool to import. It worked for us ("zfs > destroy" of a 1 TB filesystem locked up our pool; but the background > destroy feature let us work around it for importing). >=20 > --=20 > Freddie Cash > fjwcash@gmail.com