From owner-freebsd-fs@FreeBSD.ORG Mon Jan 20 10:23:36 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 90C86D4A; Mon, 20 Jan 2014 10:23:36 +0000 (UTC) Received: from mail-pa0-x22f.google.com (mail-pa0-x22f.google.com [IPv6:2607:f8b0:400e:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 59EEB18BE; Mon, 20 Jan 2014 10:23:36 +0000 (UTC) Received: by mail-pa0-f47.google.com with SMTP id kp14so6850808pab.20 for ; Mon, 20 Jan 2014 02:23:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=ofluTke3s4GLaL28lC3lW5vNswbxnk7MjstR08/9STQ=; b=yb82nU9thX8j2qsCFXVGkQpJvrhbyFsxNTKVYjpG64O3bVfigUnZiHHWkMXZ74RBIp KjS1MJpDES5ndM3GnqO4YZ+Wui3DoqzVNh9tP5eyxFWbV7tPx186dMfHvSjyPYu9dRSa yrm2j5z/o5rGQexGiko33Ua8YzbAYOFbKQ7uexd5Xj8YQtWWu+Oekw7ws9xFn1HLhXLD CoGsykHAGFSIBvidW7XLsRpsCpnK6czpXiY5K1Rqqx2FozhT8ZXazHczdSuxEJMiVojF tfqoI+Ywu1gV6GUY8VbkEF2ymV0Sw/1f3YpV2/SW/X+BAXOpHN+Zkn7ZoqdshBHGdjkP 4yhA== MIME-Version: 1.0 X-Received: by 10.68.164.4 with SMTP id ym4mr17954057pbb.53.1390213415982; Mon, 20 Jan 2014 02:23:35 -0800 (PST) Received: by 10.68.0.73 with HTTP; Mon, 20 Jan 2014 02:23:35 -0800 (PST) In-Reply-To: References: <52D71421.3030103@delphij.net> <52D82178.6030903@freebsd.org> <879A72E5-3F29-4AC4-A4B4-E65430800A7C@gmail.com> Date: Mon, 20 Jan 2014 11:23:35 +0100 Message-ID: Subject: Re: zpool import taking weeks ... From: Ulysse 31 To: Freddie Cash Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Filesystems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jan 2014 10:23:36 -0000 Hi all, Finally solved this problem ^^, hope that my answer would help others have similar problems in the future: After read some more threads, have confirmed that DDT is stored only on ARC, and since ARC is only in RAM (not swappable), the import process on the live cd was starting slowly filling ram and pushing userland to swap ... The result was that the system still running without out of swap message space but zfs simply "stalls" without having enought ram to continue import ... I then remember that i had 2 pre-prod servers with 64Gb RAM on each, scavenged one and added 64Gb on the zfs server (which ended so with 128Gb) Booted the server normally (native Freebsd) and the import run then correctly within approximately 12h, taking 90Gb for ARC (not checked but suppose that it was mainly DDT). Once booted, destroyed the remotely sync'ed dataset that caused crash, that took another 30Gb on RAM, loading used mem to 120Gb and something like 3.5Gb free during about a hour ^^' (finger crossed). Once dataset was destroy, lot of RAM was free'ed . Rebooted the server and now it boots normally without wait on import. Thanks all for the help. Cheers, 2014/1/16 Freddie Cash : > On Thu, Jan 16, 2014 at 11:18 AM, Gomes do Vale Victor > wrote: >> >> >> Le 16 janv. 2014 =E0 19:14, Stefan Esser a =E9crit : >> >> > Am 16.01.2014 08:00, schrieb Gomes do Vale Victor: >> >>> Thanks a lot for this info ! I have read about the "-T" option >> >>> on import with tgx but it was with ZFS on Linux ... Don't knew it >> >>> is available on newer implementation under FreeBSD. Will so wait >> >>> that it ends correctly and will think on upgrading on the system >> >>> (the rootfs is ZFS also ...) >> > >> > Just for the record, since your reply seems to imply that the -T >> > option is only available on Linux: >> > >> > The -T option of zpool import has been present in FreeBSD for years, >> > but is undocumented (same as in Solaris/OpenIndiana). It has been >> > mentioned in some ZFS recovery guides (most written for Solaris) and >> > it actually works quite well (I used it to recover an inconsistent poo= l >> > and helped somebody else who has lost access to his ZFS file-systems). >> > >> > Regards, STefan >> >> Sorry if you understood that. It was not what I meant. I wanted to say >> that I only found one thread talking about that option (the import with = the >> -T option) and that thread was about ZFS on Linux, and since I ain't fou= nd >> no mention of it on the actual man of my FreeBSD version I have here, I >> tough it was only available on newer ZFS versions (like v5000) under >> FreeBSD. >> Thanks for the info, so do you think I could use this method to recover = my >> pool that is on ZFS v28 ? >> Also, the import is now running without crash for 8 days now, and still >> does not have finished, and I do not see any revelant activity on disks,= nor >> any load under CTRL+T on the zpool process (sometimes goes to 0.16/0.08 = but >> quickly returns to 0.00 and stays there a while) do you think I should >> interrupt it, reboot and restart again ? Or directly test with the -T op= tion >> ? > > > I'd try a reboot to restart the process. Let it run for a day. If you > still aren't seeing any disk activity, then try the -T and/or the -F impo= rt > options. > > -- > Freddie Cash > fjwcash@gmail.com --=20 Ulysse31