Date: Fri, 11 Oct 2013 13:10:10 -0600 From: Ian Lepore <ian@FreeBSD.org> To: George Mitchell <george+freebsd@m5p.com> Cc: "freebsd-arm@freebsd.org" <freebsd-arm@FreeBSD.org> Subject: Re: Raspberry Pi prefetch aborts Message-ID: <1381518610.1130.155.camel@revolution.hippie.lan> In-Reply-To: <5254AC6C.9070400@m5p.com> References: <5254AC6C.9070400@m5p.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 2013-10-08 at 21:07 -0400, George Mitchell wrote: > Am I the only one seeing these? > > Fatal kernel mode prefetch abort at 0x4278f502 > trapframe: 0xdd2eafa0, spsr=200000f3 > r0 =00000001, r1 =00000001, r2 =00000001, r3 =c058dbd0 > r4 =c2400000, r5 =00000004, r6 =c05826a0, r7 =c2400308 > r8 =c058d8c8, r9 =c25fd300, r10=c29aa9e4, r11=0004240c > r12=00000000, ssp=dd2eaff0, slr=c046079c, pc =4278f502 > > [ thread pid 27951 tid 100090 ] > Stopped at 0x4278f502: address 0x4278f502 is invalid > *** error reading from address 4278f502 *** > db> t > Tracing pid 27951 tid 100090 td 0xc2799c80 > db_trace_self() at db_trace_self > pc = 0xc045d3b0 lr = 0xc012c5e8 (db_stack_trace+0xf4) > sp = 0xdd2eacc8 fp = 0xdd2eace0 > r10 = 0xc0542420 > db_stack_trace() at db_stack_trace+0xf4 > pc = 0xc012c5e8 lr = 0xc012bf54 (db_command+0x264) > sp = 0xdd2eace8 fp = 0xdd2ead88 > r4 = 0x00000000 r5 = 0x00000000 > r6 = 0xc04b9e26 > db_command() at db_command+0x264 > pc = 0xc012bf54 lr = 0xc012bcc4 (db_command_loop+0x60) > sp = 0xdd2ead90 fp = 0xdd2eada0 > r4 = 0xc0493b6b r5 = 0xc04ab7ec > r6 = 0xc058bf7c r7 = 0xdd2eafa0 > r8 = 0xdd2eafa0 r9 = 0xc0582b24 > r10 = 0xc0542690 > db_command_loop() at db_command_loop+0x60 > pc = 0xc012bcc4 lr = 0xc012e6c4 (db_trap+0xdc) > sp = 0xdd2eada8 fp = 0xdd2eaec8 > r4 = 0x00000000 r5 = 0xdd2eadb0 > r6 = 0xc0582b50 > db_trap() at db_trap+0xdc > pc = 0xc012e6c4 lr = 0xc0267be0 (kdb_trap+0xd4) > sp = 0xdd2eaed0 fp = 0xdd2eaef0 > r4 = 0x00000000 r5 = 0x00000000 > r6 = 0xc0582b50 r7 = 0xdd2eafa0 > kdb_trap() at kdb_trap+0xd4 > pc = 0xc0267be0 lr = 0xc046e824 (dab_fatal+0x174) > sp = 0xdd2eaef8 fp = 0xdd2eaf10 > r4 = 0xdd2eafa0 r5 = 0x200001d3 > r6 = 0x4278f502 r7 = 0x00000000 > r8 = 0xdd2eafa0 r9 = 0xc25fd300 > r10 = 0xc2981960 > dab_fatal() at dab_fatal+0x174 > pc = 0xc046e824 lr = 0xc046eb10 ($d) > sp = 0xdd2eaf18 fp = 0xdd2eaf98 > r4 = 0xc2400000 r5 = 0xc2799c80 > r6 = 0xc05826a0 r7 = 0xc2400308 > $d() at $d > pc = 0xc046eb10 lr = 0xc045eb60 (exception_exit) > sp = 0xdd2eafa0 fp = 0x0004240c > r4 = 0xc2400000 r5 = 0x00000004 > r6 = 0xc05826a0 r7 = 0xc2400308 > r8 = 0xc058d8c8 r9 = 0xc25fd300 > r10 = 0xc29aa9e4 > exception_exit() at exception_exit > pc = 0xc045eb60 lr = 0xc046079c (spinlock_exit+0x18) > sp = 0xdd2eaff8 fp = 0x0004240c > r0 = 0x00000001 r1 = 0x00000001 > r2 = 0x00000001 r3 = 0xc058dbd0 > r4 = 0xc2400000 r5 = 0x00000004 > r6 = 0xc05826a0 r7 = 0xc2400308 > r8 = 0xc058d8c8 r9 = 0xc25fd300 > r10 = 0xc29aa9e4 r12 = 0x00000000 > Unable to unwind into user mode > db> > > The numbers vary, but the list of addresses is always the same. I do a > portmaster run, and sometimes it goes all the way through, and sometimes > I get crashes every five minutes. The crashes seem somewhat more likely > to happen during portmaster's dependency checking than at other times, > and they almost never happen while configuring, compiling, linking, > or installing is in progress. > > SVN revision r256073M (RPI-B config file modified for serial console). > > Suggestions, please? -- George Are you the only one seeing these? Yes, or no, or maybe. I think you're the only one seeing it happen on an RPi, from userland. I've been working for weeks on a prefetch abort that happens in the kernel with some similarities in the crash info. For me it happens before init is even launched. I spent much of last week doing ports builds on my rpi, just to try to get a panic or other trouble. I had a few ports fail to build for mundane reasons (complaints about gcc 4.6 and other ports-glitch stuff). I had other ports build and install just fine. I haven't tried portmaster (I wouldn't know how, I've only got a vague idea what it even is). I couldn't get a system crash even once (in fact, this may be the longest my rpi has ever stayed up and running, it's been a couple weeks now). I think the crash you're seeing may be happening in the kernel as it tries to return to userland after an interrupt, in that the backtrace leads to spinlock_exit, and probably to the line of code in that routine that re-enables interrupts. That's part of the signature for the panics I'm seeing too. So if this is the same problem I'm chasing, then I'm working on it when I'm not busy doing $work. Soon enough, fixing it will become an issue for $work. :) In the future, no need to post a whole backtrace, because it's always the same. (Well, if you ever saw one where the exception_exit frame didn't backlink to spinlock_exit, that would be interesting.) The crash info that is most useful for this problem is this register dump block: Fatal kernel mode prefetch abort at 0x4278f502 > trapframe: 0xdd2eafa0, spsr=200000f3 > r0 =00000001, r1 =00000001, r2 =00000001, r3 =c058dbd0 > r4 =c2400000, r5 =00000004, r6 =c05826a0, r7 =c2400308 > r8 =c058d8c8, r9 =c25fd300, r10=c29aa9e4, r11=0004240c > r12=00000000, ssp=dd2eaff0, slr=c046079c, pc =4278f502 > > [ thread pid 27951 tid 100090 ] > Stopped at 0x4278f502: address 0x4278f502 is invalid Also, next time it happens it might be good to see the output of these commands: show all procs show proc PID show thread TID where PID and TID are the pid and tid numbers from the end of the register dump block. -- Ian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1381518610.1130.155.camel>