From owner-freebsd-current@freebsd.org Fri Jun 21 21:54:23 2019 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 387AB15C0D5E for ; Fri, 21 Jun 2019 21:54:23 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lf1-f50.google.com (mail-lf1-f50.google.com [209.85.167.50]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 41061830A8; Fri, 21 Jun 2019 21:54:22 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lf1-f50.google.com with SMTP id y17so6080900lfe.0; Fri, 21 Jun 2019 14:54:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=bAOhQ1vEtJM5eBzE8Xl+MA9P/5/d5tHNwg87yKvDdr8=; b=tD6hr3homXIBvXpePxL251M0BdOwt1+EqQFqbqcFvo1C8VGubK8JZqAl+sg9BiLlC7 X5zR4r57OZMk8b4wQRU4PAqnzQwWihyZOvysBHxuOqcuPG/fc+keHD8O4kx+8OZvE57C hCCzGsSLkqkdCSF5SJrZZ1Z+6jHabDRxwYktI1jOV5ql4u56wD/ScaTs7j313m8zV3vM rDhzOnSWDkKOUEYB92AEM9sroJeRMzrggZjN53C4z5R8M8tKFcjlReIaog36q9kiumQs OeeEByL96dBS4H6K016X+MdQYyo4gwt+NZ8aqYc58N57t9se0E10REbjlNi8tf6Op7jX HbWQ== X-Gm-Message-State: APjAAAUe8ARxEUe1A/GA/eaaMhjr7hsPFW+BbQj0NySJDGUBBuTI7D0O /+gYtQ3C+4NdTl+pBnEo0bYqC2HQsavQQWasWDk= X-Google-Smtp-Source: APXvYqwoVvzoOdzXV9HtSAOas9OQWPxBKobHFPtkdIAFHFpHoMaygUlZxEUHZ8sSEPa+lLOLyBkmWBFcuCZmR/1BiHA= X-Received: by 2002:a19:7905:: with SMTP id u5mr69219437lfc.117.1561154054776; Fri, 21 Jun 2019 14:54:14 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Fri, 21 Jun 2019 15:54:03 -0600 Message-ID: Subject: Re: Reducing UFS corruption from unclean shutdowns? To: Scott Long Cc: Don Lewis , Chuck Silvers , Kirk McKusick , FreeBSD CURRENT Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 41061830A8 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.167.50 as permitted sender) smtp.mailfrom=asomers@gmail.com X-Spamd-Result: default: False [-3.20 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; IP_SCORE(-1.28)[ip: (-0.60), ipnet: 209.85.128.0/17(-3.44), asn: 15169(-2.33), country: US(-0.06)]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[freebsd.org]; RCPT_COUNT_FIVE(0.00)[5]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.91)[-0.905,0]; RCVD_IN_DNSWL_NONE(0.00)[50.167.85.209.list.dnswl.org : 127.0.5.0]; SUBJECT_ENDS_QUESTION(1.00)[]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jun 2019 21:54:23 -0000 On Fri, Jun 21, 2019 at 3:51 PM Scott Long wrote: > > > > > On Jun 21, 2019, at 3:49 PM, Don Lewis wrote: > > > > On 21 Jun, Scott Long wrote: > >> > >>> On Jun 21, 2019, at 2:09 PM, Alan Somers wrote: > >>> > >>> On Fri, Jun 21, 2019 at 1:56 PM Scott Long wrote: > >>>> > >>>> > >>>> > >>>>> On Jun 21, 2019, at 1:49 PM, Alan Somers wrot= e: > >>>>> > >>>>> I panic my development VM regularly. Each time, I need to fsck the > >>>>> file system. Even if I had run sync(8) just before the panic, I > >>>>> frequently find corruption. What should I change to make sync(8) > >>>>> work, or at least to make corruption rare? It looks like my root f= ile > >>>>> system is using soft-updates+journal. Should I disable those? > >>>>> > >>>> > >>>> What corruption do you regularly see? > >>>> > >>>> Scott > >>> > >>> fsck reports various types of errors, all repairable, like "INODE > >>> CHECK-HASH FAILED", "FREE BLK COUNT(S) WRONG IN SUPERBLK", "SUMMARY > >>> INFORMATION BAD", "BLK(S) MISSING IN BIT MAPS", and "UNREF FILE". If > >>> I don't run fsck, then I get errors when I try to access files. Like > >>> "inode XXX: check-hash failed" and "such and such is marked as an > >>> executable file but could not be run by the operating system". > >>> -Alan > >> > >> The freeblk count and summary information messages are normal and expe= cted. I > >> don=E2=80=99t think that the blks missing message is expected, and the= unref file message is > >> definitely a red flag of something that should have been handed with j= ournal > >> recovery. Kirk and Chuck, do you have any insight here? > > > > Blks missing is to be expected. The free block bitmap isn't updated > > until after the pointers to them in the inode are cleared. Likewise th= e > > unref file warning is also to be expected because the reference to the > > inode in the parent directory is cleared before the inode is cleared. > > These aren't a fatal problem, just a resource leak until fsck is run. > > > > I would not expect the inode check-hash error. I would expect the hash > > update to happen at the same time as any other bits of the inode are > > changed, but this is a new feature that went in after I last looked at > > the code. > > > > I thought that unref=E2=80=99d files were also supposed to be cleaned up = on journal recovery, > different from plain SU recovery/preening. It=E2=80=99s been so long, ma= ybe I don=E2=80=99t remember > correctly anymore. > > Scott I would've thought that immediately following a sync(8), the filesystem would be consistent. Why do I still see errors after a panic in files that were written before I sync()ed? -Alan