From owner-freebsd-current@freebsd.org Thu Dec 13 10:39:29 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7A3791316458 for ; Thu, 13 Dec 2018 10:39:29 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: from mail-wr1-x442.google.com (mail-wr1-x442.google.com [IPv6:2a00:1450:4864:20::442]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9300F81954 for ; Thu, 13 Dec 2018 10:39:28 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: by mail-wr1-x442.google.com with SMTP id p4so1451438wrt.7 for ; Thu, 13 Dec 2018 02:39:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:subject:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=ke5xqhevFAIVmwTLi/pAsKgtWC17HkIXJ+X846jK1EI=; b=OXLqULil8Na4P+nbDHKisS35ZKKRi43gN7teX5DqjjHzC4mZmJEfA0OxLE/T9A5Yiu 0ShoBDMKoSsigw88K2UWNk6B7N+olH1pQeYoBWjaj4TF5bIDoiEtMZbnKj8vi1hBWnmR ypCFlbhC9vxS3ig0DQWAkaM5hxAU3J86oD04P/HK3rU70mSCu7w7TYh9N8ns2Ogn97HU gRqTAlRr/u1Qui1ejv2LpvWTibBZITvbit2XeVID+rc4qvpA5ZszGzBcnP0tBelXEPgQ doh8X3K8bt6Bc/p5M8GyVwQNshg+V31r3xnCDBqJ4YsVuYq1M+RdTmt9NBdE6itRFpxt sGZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:subject:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=ke5xqhevFAIVmwTLi/pAsKgtWC17HkIXJ+X846jK1EI=; b=HEmuAkw/FEVi6cdJt3mBtMsQAdxuLWaTQN7GwRssM4vMKP/xVE6rkTWV4/VlTuhWhb Sj8McCIk8b2c0HjVH2Bu0T/Pk+m7oMK6lF48EqViKjO6AYf5/QAlZ46AbGetmD1gKElW 3UOIdzEjYdD5CMnFMb/vHwvCAPMxUt38NBhgsPufjU0vw7gnPIrIqFwfvKlS0T7vdLY5 tuHbTt9Y+w0CBVBOTQvSG5LM1/zmYGfJ+wxJyOblgbQXNPQdK1nBftboeHnUyHDhoDiA 6GoUSh8uRCdJTlUERAC+POMdh32lgx0hB3SqszgAnKoloiWZtCVsWR2r4pH5OhjUMNHO +dMw== X-Gm-Message-State: AA+aEWbzY4xKSFhyYLT14RIdMCQwpFUeaAPTqKt7XNubGm35smTCHnNY TJPAVqf+eFd/jVt5q2Wb8zueAjnG X-Google-Smtp-Source: AFSGD/XC8XB5wjhClDJl+dqnw3Ch1xUmpZSC8KniSiSt+Eyg82+pgyJ8hWKm5Djuq1nvVDm0EJhRgQ== X-Received: by 2002:adf:e9d1:: with SMTP id l17mr19234924wrn.73.1544697567377; Thu, 13 Dec 2018 02:39:27 -0800 (PST) Received: from ernst.home (p5B02380D.dip0.t-ipconnect.de. [91.2.56.13]) by smtp.gmail.com with ESMTPSA id a12sm1291330wro.18.2018.12.13.02.39.26 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 13 Dec 2018 02:39:26 -0800 (PST) Date: Thu, 13 Dec 2018 11:39:25 +0100 From: Gary Jennejohn To: freebsd-current@freebsd.org Subject: Re: enabling inode hashes results in kernel panics Message-ID: <20181213113925.5f100b5e@ernst.home> In-Reply-To: <20181213104713.02a14912@ernst.home> References: <20181213104713.02a14912@ernst.home> Reply-To: gljennjohn@gmail.com X-Mailer: Claws Mail 3.16.0 (GTK+ 2.24.32; amd64-portbld-freebsd12.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 9300F81954 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=OXLqULil; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of gljennjohn@gmail.com designates 2a00:1450:4864:20::442 as permitted sender) smtp.mailfrom=gljennjohn@gmail.com X-Spamd-Result: default: False [-3.82 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_REPLYTO(0.00)[gljennjohn@gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_NONE(0.00)[]; REPLYTO_ADDR_EQ_FROM(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.94)[-0.939,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_TLS_LAST(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; FREEMAIL_REPLYTO(0.00)[gmail.com]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-0.88)[ip: (-1.49), ipnet: 2a00:1450::/32(-1.50), asn: 15169(-1.30), country: US(-0.09)]; RCVD_IN_DNSWL_NONE(0.00)[2.4.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2018 10:39:29 -0000 On Thu, 13 Dec 2018 10:47:13 +0100 Gary Jennejohn wrote: > I just had two panics as a result of enabling inode hashes on a > file system yesterday when I had to run fsck on it. > > The filesystem showed no problems at all yesterday, the problem > didn't appear until I accessed it today. > > NOTE that my fsck was installed from r341840 yesterday morning, > so it should be up-to-date. My kernel is also at r341840. > > I have run fsck on this filesystem six times. Although fsck > claims to have repaired the inode-check hash every time, it in > reality has not repaired anything. I know this because, after > the first two fsck runs, I mounted the filesystem. Accessing it > resulted in a immediate panic (the second of the two). > > The file system is located on a SSD with trim enabled. Whether > that is relevant I cannot say, but this is the ONLY filesystem > with inode hashes enabled. > > The inodes are all contiguous, from 122229888 thorugh 122229904. > Strangely, MTIME=Sep 6 23:21 2002 on every one of them, but the > SSD itslef is only a month old and the files on it only a few > weeks old. > > The panic message is > Inode 122229888: check-hash failed > panic: softdep_update_inodeblock: bad link count > > It almost appears like the softdep code in the kernel is not > aware of inode hashes and gets confused. > > I have the crash dumps and the core.txt files. > > It doesn't appear that I can disable the hashes using tunefs, so > my only remedy will be to run fsdb and clear these inodes and lose > quite a few rather large files. > I ran fsdb on every affected inode and allowed it to correct the inode hashes. After that, fsck no longer found any errors and I was able to mount and access the filesystem without a kernel panic. So, it would appear that fsdb really does correct inode hash errors, whereas fsck does not. -- Gary Jennejohn