From owner-freebsd-fs@FreeBSD.ORG Thu Jan 16 11:58:22 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1CB2AA13 for ; Thu, 16 Jan 2014 11:58:22 +0000 (UTC) Received: from mail-la0-x22e.google.com (mail-la0-x22e.google.com [IPv6:2a00:1450:4010:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 91C3D1060 for ; Thu, 16 Jan 2014 11:58:21 +0000 (UTC) Received: by mail-la0-f46.google.com with SMTP id b8so2447556lan.5 for ; Thu, 16 Jan 2014 03:58:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=TB+VcmG0aHtl5AXUYbMJZxjpLtYZoSUhh+2kTWWD7cY=; b=GtfIsyoGR234czWfzpt4dkAZ3S3FGZ9dAqjgJrgfiB03+zZotcnE3yXJe6EAQvHhw7 iFEtv+EszkWOlTO/x0bJqVvgOC69t1S5/lGKDA6Nn7VW77fSUXaOBAfE7CrPRGEGeUC/ ckQxifIlOa9Y5FWE13opObdvc8DjVq4b+EDhdcKtTeUNEZQLt+vUtgOZnlhAeGYsbX3d r1eeoYQF6tRmDraCHeRl7ZgJ1oiql8+pxR/DQT1BqpzegWg89Z6hQBJ03kI2I/RfH/L5 oHWJ1WEvdpg+4IPTqZxV/JlsYKYgEiUFoOjpHVMMirYCEaDoPVUCoNjJmjtAiIGZeD4N ogOQ== X-Received: by 10.152.182.235 with SMTP id eh11mr60887lac.73.1389873499570; Thu, 16 Jan 2014 03:58:19 -0800 (PST) Received: from [192.168.1.129] (mau.donbass.com. [92.242.127.250]) by mx.google.com with ESMTPSA id rb4sm4348976lbb.1.2014.01.16.03.58.18 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 16 Jan 2014 03:58:18 -0800 (PST) Message-ID: <52D7C959.9080305@gmail.com> Date: Thu, 16 Jan 2014 13:58:17 +0200 From: Volodymyr Kostyrko User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Ulysse 31 , Freddie Cash Subject: Re: zpool import taking weeks ... References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Filesystems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Jan 2014 11:58:22 -0000 15.01.2014 10:29, Ulysse 31 wrote: > Thanks for the answer. > > I was beginning to ... well ... have a little start of panic ^^' > The import still running but no real activity on disks : sometimes, > maybe something like each hour, there is some really brief led > activity on the hdd, but nothing more. > one thing that would be great is to have an overall percentage or just > counting of what is done or what is left ... for now i don't even know > if it's gonna end on the next 5 mins or on the next 5 weeks :s > So now since i have no real activity on the actual zpool import (the > last import started a week ago, it do not have go out of swap since i > added lot of swap (260G HDD swap)), should i just let it run, or make > a reboot and start over ? > Also once imported, and since i don't care about the data actually on > the incriminated dataset (the one that crashed with the > recv/rollback), I was planning to simply delete that dataset and make > a new full sync, what do you think about it ? > Thanks again for your help. You odds are grim. I had the same problem with crashed ZFS when dropping some snapshots (deduplicated ones). The situation looks like: 1. Boot. Import start. 2. Memory depletes. 3. When memory is over the process is stuck or machine crashes. In your case no crash means you are running a lil bit newer code. All of this comes in something like 15 minutes. My machine was at 4G of memory back then. I added 4G more and process successfully completed in near 20 minutes eating ~6G of mem. The culprit was deleting deduplicated snapshot worth 14G of disk space. I personally think that previous versions of ZFS tend to interpret all snapshot operations as atomic ones meaning all changes to fs should happen simultaneously thus pushing to memory: - all old dedup data that still can be accessed, - all new dedup data that should be active after the change, - all metadata of filesystem in question. I'm not a pro in this and haven't retested this situation after the recent changes with snapshots (like background deleting). But this was a final grave reason for me to stop thinking about dedup. Or at least in ZFS as I still like the way HAMMER does it. -- Sphinx of black quartz, judge my vow.