From owner-freebsd-arm@FreeBSD.ORG Tue Sep 30 14:29:30 2014 Return-Path: Delivered-To: freebsd-arm@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 10E0FB89; Tue, 30 Sep 2014 14:29:30 +0000 (UTC) Received: from mail-la0-x234.google.com (mail-la0-x234.google.com [IPv6:2a00:1450:4010:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 52A20299; Tue, 30 Sep 2014 14:29:29 +0000 (UTC) Received: by mail-la0-f52.google.com with SMTP id hz20so3691543lab.25 for ; Tue, 30 Sep 2014 07:29:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=vjB/FGCxYS9ucM0ep36SlNY+qBFS0ycWWYTcot8CQQQ=; b=gg4bFQCNHcVE6EndT8NlaCJlplkN70x9CbQ5NaFbfsLKtrzFI4CVI5f5LLS9oUulHA y9Rgd7rjs2pHdVDLzvijIPC8PW0/BMYZv6MMdGes3C2HPOQyFu6HxVPi/ukFgA/FvwvX uQ7GPK4Cj8ZpP2Qsz9tsINblaSh2K1t/Okkfg80qL6BoHRA+YkKYr9PvhkRrW9DBBIIg o1GI4y8NfcneQ2pEMnc0wpb/Q7PneLYVZ+2euE2paNbPNamE4wutKfM0krfkPsRFtV9N ZBAqyoQAryPxqYdT0V/X0C6b6kKpV8JdhaOaIQM3iJEWmBwXem9NNyMINcKcyD6vK/5m JGXA== X-Received: by 10.152.5.100 with SMTP id r4mr25570887lar.41.1412087367351; Tue, 30 Sep 2014 07:29:27 -0700 (PDT) Received: from ?IPv6:2001:1620:ff0:c51:65fd:7630:d87f:4cbe? ([2001:1620:ff0:c51:65fd:7630:d87f:4cbe]) by mx.google.com with ESMTPSA id kw10sm6129942lbc.17.2014.09.30.07.29.26 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 30 Sep 2014 07:29:26 -0700 (PDT) Message-ID: <542ABE45.3020402@gmail.com> Date: Tue, 30 Sep 2014 16:29:25 +0200 From: Mattia Rossi User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Ian Lepore Subject: Re: Random Kernel Panic on Dreamplug (FS related) References: <542559BC.7090100@gmail.com> <20140929040126.GG43300@funkthat.com> <54291B74.5010307@gmail.com> <20140930112937.GU43300@funkthat.com> <542A9EA4.70109@gmail.com> <20140930123010.GZ43300@funkthat.com> <542AB897.3020309@gmail.com> <1412086795.66615.363.camel@revolution.hippie.lan> In-Reply-To: <1412086795.66615.363.camel@revolution.hippie.lan> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-arm X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Sep 2014 14:29:30 -0000 Am 30.09.2014 16:19, schrieb Ian Lepore: > On Tue, 2014-09-30 at 16:05 +0200, Mattia Rossi wrote: >> Am 30.09.2014 14:30, schrieb John-Mark Gurney: >>> Mattia Rossi wrote this message on Tue, Sep 30, 2014 at 14:14 +0200: >>>> Am 30.09.2014 13:29, schrieb John-Mark Gurney: >>>>> Mattia Rossi wrote this message on Mon, Sep 29, 2014 at 10:42 +0200: >>>>>> Am 29.09.2014 06:01, schrieb John-Mark Gurney: >>>>>>> Mattia Rossi wrote this message on Fri, Sep 26, 2014 at 14:19 +0200: >>>>>>>> This might be part of the weird FFS issues the Dreamplug has and no-one >>>>>>>> knows why they're happening. >>>>>>> Are you running w/ FFS journaling? If so, try turning it off, but >>>>>>> keeping softupdates on.. >>>>>> No journaling, no softupdates. I'll try enabling softupdates next time. >>>>>> don't know if it will panic though >>>>>>>> data_abort_handler() at data_abort_handler+0x5c0 >>>>>>>> pc = 0xc0de7a28 lr = 0xc0dd711c (exception_exit) >>>>>>>> sp = 0xde019898 fp = 0xde019a20 >>>>>>>> r4 = 0xffffffff r5 = 0xffff1004 >>>>>>>> r6 = 0xc3f3f6c0 r7 = 0x00001000 >>>>>>>> r8 = 0xc443e880 r9 = 0x00000000 >>>>>>>> r10 = 0xc3d69000 >>>>>>>> exception_exit() at exception_exit >>>>>>>> pc = 0xc0dd711c lr = 0xc0d53828 (ffs_truncate+0xaa8) >>>>>>>> sp = 0xde0198e8 fp = 0xde019a20 >>>>>>>> r0 = 0xd0238120 r1 = 0x00000e60 >>>>>>>> r2 = 0x00000000 r3 = 0x00000000 >>>>>>>> r4 = 0x00000120 r5 = 0x00000000 >>>>>>>> r6 = 0xc3f3f6c0 r7 = 0x00001000 >>>>>>>> r8 = 0xc443e880 r9 = 0x00000000 >>>>>>>> r10 = 0xc3d69000 r12 = 0xd0238120 >>>>>>>> memset() at memset+0x48 >>>>>>>> pc = 0xc0de521c lr = 0xc0d53828 (ffs_truncate+0xaa8) >>>>>>>> sp = 0xde0198e8 fp = 0xde019a20 >>>>>>>> Unwind failure (no registers changed) >>>>>>> No more beyond this? If you could run addr2line on 0xc0d53828 so >>>>>>> that we know where in ffs_truncate it's failing, that'd be very >>>>>>> nice... >>>>>> So I was trying to save the coredump in order to reboot and run >>>>>> addr2line, but that failed: >>>>>> >>>>>> Physical memory: 504 MB >>>>>> Dumping 67 MB:(da0:umass-sim0:0:0:0): WRITE(10). CDB: 2a 00 01 d5 1f20 >>>>>> 00 00 01 00 >>>>>> (da0:umass-sim0:0:0:0): CAM status: Resource Unavailable >>>>>> (da0:umass-sim0:0:0:0): Error 5, Retries exhausted >>>>>> Aborting dump due to I/O error. >>>>>> >>>>>> ** DUMP FAILED (ERROR 5) ** >>>>>> >>>>>> So I guess this error is related to the CAM errors I'm getting from time >>>>>> to time. I was hoping that those errors were related to the INVARIANTS >>>>>> option that slowed down the system and thus might have triggered CAM >>>>>> errors, but obviously the SD Card seems to be the real issue here. >>>>>> So no crashdump for further analysis. >>>>> That's fine.. w/ the addr2line we have some lines to explore... >>>>> >>>>>> Interestingly the CAM errors didn't show up on the terminal as other >>>>>> times, the kernel just panicked straight away. >>>>> Hmm.. that is odd.. someone who knows the SD card layer should look >>>>> at this part... It could be that the SD card driver doesn't handle >>>>> dumping (there is this global flag that gets set) properly and the driver >>>>> needs to behave differently when it's set... >>>> I also need to grab a new SD card, just to make sure it's really not the >>>> card. >>>> >>>>>> But I've got the addr2line output, even though I'm not sure it makes any >>>>>> difference: >>>>>> >>>>>> addr2line -f -e /mnt/kernel.debug 0xc0d53828 >>>>>> >>>>>> ffs_truncate >>>>>> /usr/devel/dreamplug/sys/ufs/ffs/ffs_inode.c:321 >>>>> can you give me the contents of the line? and a few lines of context >>>>> around it? In HEAD's source, this is DOINGASYNC, and there is no call >>>>> to memset, nor a variable assignment that would result in memset being >>>>> called... >>>> Same here.. The file hasn't been changed in a while (Fri, 31 May 2013): >>>> >>>> ip->i_size = length; >>>> DIP_SET(ip, i_size, length); >>>> if (bp->b_bufsize == fs->fs_bsize) >>>> bp->b_flags |= B_CLUSTEROK; >>>> if (flags & IO_SYNC) >>>> bwrite(bp); >>>> 321: else if (DOINGASYNC(vp)) >>>> bdwrite(bp); >>>> else >>>> bawrite(bp); >>>> ip->i_flag |= IN_CHANGE | IN_UPDATE; >>>> return (ffs_update(vp, !DOINGASYNC(vp))); >>>> >>>> No idea what's going on. >>> ok, could you send me the output of objdump -dSl, but you only need >>> to include the part from XXXXX : to the next XXX: >>> line... probably off list as it'll be quite long... >> I'm sorry, but given that I just broke all my working worlds using fsck, >> I'm not going to be able to do that until I'm back from holidays.... >> currently working on the stuff remotely and after today's work day, I'm >> not going to be able to get my hands on the dreamplug. >> >> > BTW, for anyone playing with this problem, step one is to edit > your /etc/fstab and set the fsck pass number to 0 for all filesystems. > There's a risk of filesystem corruption after a crash, but it's smaller > than the 100% corruption rate of letting fsck run. :) > Of course! Great idea :-) Sometimes just can't think of the right tweak to save a lot of pain... Anyhow, I just found out, that I was rebooting the dreamplug from the sd card instead of the usb stick the whole time, and the usb stick hasn't been damaged enough by fsck, so it actually booted :-) I'll send the objdump soon.