From owner-freebsd-stable@freebsd.org Tue Apr 30 09:50:59 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AAFB4158AD77; Tue, 30 Apr 2019 09:50:59 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-io1-xd2c.google.com (mail-io1-xd2c.google.com [IPv6:2607:f8b0:4864:20::d2c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D713F967CB; Tue, 30 Apr 2019 09:50:58 +0000 (UTC) (envelope-from delphij@gmail.com) Received: by mail-io1-xd2c.google.com with SMTP id u12so11685174iop.0; Tue, 30 Apr 2019 02:50:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=rL0uEn9GIJsVfF474I0qV3xC8yy/pTExmK8Xhy+g5Jc=; b=NMs9EaolCWZVWbVkDP2qpQ1m/2qYQFUgibcuM/1qWgXT7sA26qQX/3Moym1imaemsl smxbfDi8H3c/Hl7qlWIi1m8bQrQQHcuS7qJs0BSlSdzk6ptxW0Rc5d6383XJcMOGE4Yz HpawlT5dz+5yOKMHVbNh44hVbZH+E9W1U9Wt5DJaCnEH0DHPA/nbnHHTI8sP/Wgqslns aBRF7C9RPZfnmkMcVitOSHhCrKDFOA9VdA9ACn8vah/b+3wcsKppb6Ljjs3KCPJURNs/ 78tmZHWE773DpiWihlqz2eDTJegZo+drD16GrVdpvCqGDUav3m/K5GGMN7M9NepXT3en uGKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=rL0uEn9GIJsVfF474I0qV3xC8yy/pTExmK8Xhy+g5Jc=; b=mOINMek+G40pucncVcZ8csrwJ2tgtczb4C/hO6E594vbWykWW4HaiHi1clARKfvRDx Wpigjoa6xY02mG6lrJGBmnjCPpaCrPCPyVupq0/x5imuVRpLVUv6D8N3lwlmcBGTVZe9 DxLGre0GXnlXcL5D8T0F3KYN1RtiEfqHbJCyDy0UHHn0mhwvqJOKfJfnUy7gUIRNLGbZ SShIcGEPxtJXQIgXXa7tfAuSwlXz42k6OauN09cbQt19bMXFY9JrE/laV0NxT64q08M3 f/zX4oI7glhQmhZz/Yic4SflFI/fb7/wbrqc+tsk3jq8WKl3rXiqtucgkBmDalyHveHa GJ2g== X-Gm-Message-State: APjAAAXHQZGBkV9W7lnNBuI2ELrBWJEXUIMXsXClCDuKIkstAbC4zZCL yRhRNAvtJNbn+Uzy7wHIGq26IOPRRxEIhZKWbNDIp/Tl X-Google-Smtp-Source: APXvYqwHMSHx6G3Dol+ayowyBt0nlwUVmk5HdrdAMKfiweIF4CiTJHMUt3/+0Lp/3+9lQTFvEP1ANRQASiPH2eMy+5U= X-Received: by 2002:a5d:8e19:: with SMTP id e25mr26907698iod.139.1556617857959; Tue, 30 Apr 2019 02:50:57 -0700 (PDT) MIME-Version: 1.0 References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> In-Reply-To: <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> From: Xin LI Date: Tue, 30 Apr 2019 17:50:40 +0800 Message-ID: Subject: Re: ZFS... To: Michelle Sullivan Cc: rainer@ultra-secure.de, owner-freebsd-stable@freebsd.org, freebsd-stable , Andrea Venturoli X-Rspamd-Queue-Id: D713F967CB X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=NMs9Eaol; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of delphij@gmail.com designates 2607:f8b0:4864:20::d2c as permitted sender) smtp.mailfrom=delphij@gmail.com X-Spamd-Result: default: False [-6.42 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCVD_TLS_LAST(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCPT_COUNT_FIVE(0.00)[5]; NEURAL_HAM_SHORT(-0.96)[-0.963,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[c.2.d.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-2.90)[ip: (-9.04), ipnet: 2607:f8b0::/32(-3.16), asn: 15169(-2.24), country: US(-0.06)]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 09:51:00 -0000 On Tue, Apr 30, 2019 at 5:08 PM Michelle Sullivan wrote: > but in my recent experience 2 issues colliding at the same time results in > disaster > Do we know exactly what kind of corruption happen to your pool? If you see it twice in a row, it might suggest a software bug that should be investigated. Note that ZFS stores multiple copies of its essential metadata, and in my experience with my old, consumer grade crappy hardware (non-ECC RAM, with several faulty, single hard drive pool: bad enough to crash almost monthly and damages my data from time to time), I've never seen a corruption this bad and I was always able to recover the pool. At previous employer, the only case that we had the pool corrupted enough to the point that mount was not allowed was because two host nodes happen to import the pool at the same time, which is a situation that can be avoided with SCSI reservation; their hardware was of much better quality, though. Speaking for a tool like 'fsck': I think I'm mostly convinced that it's *not* necessary, because at the point ZFS says the metadata is corrupted, it means that these metadata was really corrupted beyond repair (all replicas were corrupted; otherwise it would recover by finding out the right block and rewrite the bad ones). An interactive tool may be useful (e.g. "I saw data structure version 1, 2, 3 available, and all with bad checksum, choose which one you would want to try"), but I think they wouldn't be very practical for use with large data pools -- unlike traditional filesystems, ZFS uses copy-on-write and heavily depends on the metadata to find where the data is, and a regular "scan" is not really useful. I'd agree that you need a full backup anyway, regardless what storage system is used, though.