From owner-freebsd-stable@freebsd.org Tue Aug 11 00:43:32 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 541879987A0 for ; Tue, 11 Aug 2015 00:43:32 +0000 (UTC) (envelope-from parv@bitter-almonds.com) Received: from nisba.pair.com (nisba.pair.com [209.68.5.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2D0BA660 for ; Tue, 11 Aug 2015 00:43:31 +0000 (UTC) (envelope-from parv@bitter-almonds.com) Received: from [192.168.11.4] (cpe-66-91-233-235.hawaii.res.rr.com [66.91.233.235]) by nisba.pair.com (Postfix) with ESMTPSA id E2072114234; Mon, 10 Aug 2015 20:43:29 -0400 (EDT) In-Reply-To: References: <1438783814.70393.138.camel@freebsd.org> <119E4C6D-ABFD-4D82-B353-D3829088233B@bitter-almonds.com> <8d35eae569c2ef115207bb8a1ac97b53@ultimatedns.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Subject: Re: 8-stable crashing recently in ufsdirhash From: parv Date: Mon, 10 Aug 2015 14:43:07 -1000 To: Kevin Oberman ,Chris H CC: FreeBSD-STABLE Mailing List Message-ID: <2C865403-726E-42D0-85FB-27880F3EAC45@bitter-almonds.com> X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Aug 2015 00:43:32 -0000 On August 5, 2015 12:20:15 PM HST, Kevin O wrote: >On Wed, Aug 5, 2015 at 1:15 PM, Chris H wrote: > >> On Wed, 05 Aug 2015 09:30:28 -1000 parv >wrote >> >> > On August 5, 2015 4:10:14 AM HST, Ian L >> > wrote: >> > >On Tue, 2015-08-04 at 23:54 -1000, parv wrote: >> > >> Please CC me as I cannot properly use my laptop, Thinkpad X200 >> > >> (i386). >> > >> >> > >> 8-stable has been crashing a lot since source update of Jul >2015[0]. >> > >> After building debug kernel, kgdb shows lock reversal order & in >> > >> ufsdirhash. File systems (/, var, misc) are all UFS, with var & misc >> > >> having soft updates enabled. ... >> > >> Most recent crash ... >> > >> >> > >> http://imagebin.ca/v/2B50NARvIHsH ... >> > >When you say you built a debug kernel, does that include option >> > >WITNESS_KDB? If so, remove that so you can find the real error. LORs >> > >related to ufs_dirhash have been reported for years, and nobody with >> > >the >> > >appropriate skills seems to be interested in fixing them; they just get >> > >declared to be harmless. ... >> > I will try that as soon as I can. Currently after every little fs >> > operation, >> > I am thrown in debugger-reboot-fsck cycle. As such I cannot do anything. ... >> > I tried booting >> > /boot/kernel.old/kernel from boot prompt but that presented debugger soon >> > after boot. >> > >> > BTW is it possible to set kernel.old to boot next time at boot prompt? ... >> It might be somewhat easier to boot from the boot-only/install >> CD/DVD, and then choose rescue mode. >> After you've gotten there. Simply mount "/" in read/write, them open >> it's /boot/loader.conf, and add the following: >> kernel="kernel.old" >> boot_single="YES" >> >> and save it. You can then remove the CD/DVD, and reboot which will >land >> you in single-user mode, from your kernel.old/kernel. >> Assuming it booted to that kernel OK, you can run fsck -f ... Chris, after spending my mail I was thinking on the same lines. But I delayed due to laziness, till ... >You're working too hard. > >You can drop into the loader prompt and enter "boot kernel.old". This >will >reload both the kernel and modules from /boot/kernel.old. Thanks Kevin. It was the loader prompt, not the boot prompt, that I should have tried. After booting old kernel from loader prompt, functionality was restored (to some workable state). > Also handy and >virtually unknown is that you can build test kernels and not blow away >the >old, working kernel by using "make reinstallkernel" instead of >installkernel. This will keep the existing kernel.old and just replace >the >currently running kernel. Thanks again. >N.B. There is a bug that will cause a failure to re-install >PORTS_MODULES. ... At this point I am not worried about kernel modules built from ports. I just want a working version that won't seemingly randomly crash, leaving no core dump or any other relevant messages. The only thing pointing to a crash are entries in wtmp, which lastlog reports as "crash". Nothing. Else. --