Date: Tue, 30 Sep 2014 08:17:41 -0600 From: Ian Lepore <ian@FreeBSD.org> To: Mattia Rossi <mattia.rossi.mailinglists@gmail.com> Cc: freebsd-arm <freebsd-arm@freebsd.org> Subject: Re: Random Kernel Panic on Dreamplug (FS related) Message-ID: <1412086661.66615.361.camel@revolution.hippie.lan> In-Reply-To: <542A9EA4.70109@gmail.com> References: <542559BC.7090100@gmail.com> <20140929040126.GG43300@funkthat.com> <54291B74.5010307@gmail.com> <20140930112937.GU43300@funkthat.com> <542A9EA4.70109@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 2014-09-30 at 14:14 +0200, Mattia Rossi wrote: > Am 30.09.2014 13:29, schrieb John-Mark Gurney: > > Mattia Rossi wrote this message on Mon, Sep 29, 2014 at 10:42 +0200: > >> Am 29.09.2014 06:01, schrieb John-Mark Gurney: > >>> Mattia Rossi wrote this message on Fri, Sep 26, 2014 at 14:19 +0200: > >>>> This might be part of the weird FFS issues the Dreamplug has and no-one > >>>> knows why they're happening. > >>> Are you running w/ FFS journaling? If so, try turning it off, but > >>> keeping softupdates on.. > >> No journaling, no softupdates. I'll try enabling softupdates next time. > >> don't know if it will panic though > >>>> data_abort_handler() at data_abort_handler+0x5c0 > >>>> pc = 0xc0de7a28 lr = 0xc0dd711c (exception_exit) > >>>> sp = 0xde019898 fp = 0xde019a20 > >>>> r4 = 0xffffffff r5 = 0xffff1004 > >>>> r6 = 0xc3f3f6c0 r7 = 0x00001000 > >>>> r8 = 0xc443e880 r9 = 0x00000000 > >>>> r10 = 0xc3d69000 > >>>> exception_exit() at exception_exit > >>>> pc = 0xc0dd711c lr = 0xc0d53828 (ffs_truncate+0xaa8) > >>>> sp = 0xde0198e8 fp = 0xde019a20 > >>>> r0 = 0xd0238120 r1 = 0x00000e60 > >>>> r2 = 0x00000000 r3 = 0x00000000 > >>>> r4 = 0x00000120 r5 = 0x00000000 > >>>> r6 = 0xc3f3f6c0 r7 = 0x00001000 > >>>> r8 = 0xc443e880 r9 = 0x00000000 > >>>> r10 = 0xc3d69000 r12 = 0xd0238120 > >>>> memset() at memset+0x48 > >>>> pc = 0xc0de521c lr = 0xc0d53828 (ffs_truncate+0xaa8) > >>>> sp = 0xde0198e8 fp = 0xde019a20 > >>>> Unwind failure (no registers changed) > >>> No more beyond this? If you could run addr2line on 0xc0d53828 so > >>> that we know where in ffs_truncate it's failing, that'd be very > >>> nice... > >> So I was trying to save the coredump in order to reboot and run > >> addr2line, but that failed: > >> > >> Physical memory: 504 MB > >> Dumping 67 MB:(da0:umass-sim0:0:0:0): WRITE(10). CDB: 2a 00 01 d5 1f20 > >> 00 00 01 00 <sip:2000000100> > >> (da0:umass-sim0:0:0:0): CAM status: Resource Unavailable > >> (da0:umass-sim0:0:0:0): Error 5, Retries exhausted > >> Aborting dump due to I/O error. > >> > >> ** DUMP FAILED (ERROR 5) ** > >> > >> So I guess this error is related to the CAM errors I'm getting from time > >> to time. I was hoping that those errors were related to the INVARIANTS > >> option that slowed down the system and thus might have triggered CAM > >> errors, but obviously the SD Card seems to be the real issue here. > >> So no crashdump for further analysis. > > That's fine.. w/ the addr2line we have some lines to explore... > > > >> Interestingly the CAM errors didn't show up on the terminal as other > >> times, the kernel just panicked straight away. > > Hmm.. that is odd.. someone who knows the SD card layer should look > > at this part... It could be that the SD card driver doesn't handle > > dumping (there is this global flag that gets set) properly and the driver > > needs to behave differently when it's set... > > I also need to grab a new SD card, just to make sure it's really not the > card. > > > > >> But I've got the addr2line output, even though I'm not sure it makes any > >> difference: > >> > >> addr2line -f -e /mnt/kernel.debug 0xc0d53828 > >> > >> ffs_truncate > >> /usr/devel/dreamplug/sys/ufs/ffs/ffs_inode.c:321 > > can you give me the contents of the line? and a few lines of context > > around it? In HEAD's source, this is DOINGASYNC, and there is no call > > to memset, nor a variable assignment that would result in memset being > > called... > > Same here.. The file hasn't been changed in a while (Fri, 31 May 2013): > > ip->i_size = length; > DIP_SET(ip, i_size, length); > if (bp->b_bufsize == fs->fs_bsize) > bp->b_flags |= B_CLUSTEROK; > if (flags & IO_SYNC) > bwrite(bp); > 321: else if (DOINGASYNC(vp)) > bdwrite(bp); > else > bawrite(bp); > ip->i_flag |= IN_CHANGE | IN_UPDATE; > return (ffs_update(vp, !DOINGASYNC(vp))); > > No idea what's going on. > > >>>> The sad thing is, that with fsck broken for the dreamplug, I have to > >>>> re-format the disk, reinstall everything and recreate the config files > >>>> which I didn't manage to copy to a safe place beforehand :-( > >> Didn't get around yet on checking whether the fsck issue persists if > >> everything is compiled with gcc. Will take a bit, as I'm going to be on > >> holidays for the next one and a half weeks. Fact is though, fsck is > >> broken on the Dreamplug (ARMv5TE), at least when run on EVERY device > >> that is attached via USB and if compiled with CLANG. > >>>> Before I do that I'll leave the system in debugging mode for a few days, > >>>> in case someone can help and needs some more information. > >> Currently I've run fsck on all the disks/cards that had a working world > >> for the dreamplug on it, so they're all gone. Can't do eny debugging > >> atm. I'll let you know hoe the gcc build goes once I'm back from > >> holidays though. > > Hmmm. this is also worth investigating as it could be that clang is > > producing bad code somewhere... > > I'm trying to get kernel and world compiled with gcc atm. It fails > though, as somehow the armv7 code is currently broken... and the > toolchain build is not happy. > > In file included from > /usr/devel/dreamplug/sys/arm/arm/cpufunc_asm_armv7.S:36: > ./machine/sysreg.h:97:5: error: "__ARM_ARCH" is not defined > > (followed by a few more of those) > > My build command: > > make TARGET=arm TARGET_ARCH=arm DESTDIR=/usr/devel/dreamplug-dist > KERNCONF=DREAMPLUG-1001 -DWITHOUT_SHAREDOCS -DWITHOUT_EXAMPLES > -DWITHOUT_HTML -DWITHOUT_MAN -DWITHOUT_GAMES toolchain buildworld kernel > installworld The build was broken briefly yesterday, you must have grabbed it during that window. But... I did a gcc build and got the same results as with clang. -- Ian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1412086661.66615.361.camel>