Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 2 Sep 2015 17:10:37 -0700
From:      bob prohaska <fbsd@www.zefox.net>
To:        freebsd-arm@freebsd.org
Subject:   Reproducible crashes on RPI2 under 11-CURRENT using stress2
Message-ID:  <20150903001037.GC33831@www.zefox.net>

next in thread | raw e-mail | index | archive | help


It seems possible to cause relatively reproducible crashes on RPI2 under
11-CURRENT using Peter Holm's stress2 suite. Below are console output
and backtrace of four crashes produced in the space of one day. 

To my surprise, all four seem related to virtual memory. How, or if,
that's related to the crashes seen during build of the OS is unclear,
since those seemed related to writing to the microSD card.

FreeBSD 11.0-CURRENT (RPI2) #48 r287318M: Mon Aug 31 20:02:27 PDT 2015

Crash #1

Triggered by stress2, invocation
sh ./run.sh -a
stress2 started at 11:57:20, last output lines were

14:51:14 Loop #1
    swap: run time  0+00:00:15, incarnations  30, load  80, verbose 1
 syscall: run time  0+00:00:15, incarnations  18, load 100, verbose 1

The smsc0 warnings began a few minutes after stress2 was started.

Console output and backtrace:
smsc0: warning: MII is busy
smsc0: warning: Failed to write register 0x114
smsc0: warning: Failed to read register 0x114
smsc0: warning: MII is busy
panic: vm_fault: fault on nofault entry, addr: d1a2f000
cpuid = 0
KDB: enter: panic
[ thread pid 5372 tid 100149 ]
Stopped at      $d.7:   ldrb    r15, [r15, r15, ror r15]!
db> bt
Tracing pid 5372 tid 100149 td 0xc4a8b360
db_trace_self() at db_trace_self
         pc = 0xc0543a6c  lr = 0xc014103c (db_stack_trace+0x108)
         sp = 0xed605730  fp = 0xed605748
        r10 = 0xc07885f8
db_stack_trace() at db_stack_trace+0x108
         pc = 0xc014103c  lr = 0xc0140a88 (db_command+0x388)
         sp = 0xed605750  fp = 0xed6057f0
         r4 = 0x00000000  r5 = 0x00000000
         r6 = 0x00000000 r10 = 0xc07885f8
db_command() at db_command+0x388
         pc = 0xc0140a88  lr = 0xc01406f0 (db_command_loop+0x74)
         sp = 0xed6057f8  fp = 0xed605808
         r4 = 0xc05aa9d8  r5 = 0xc05cbaa6
         r6 = 0xc07885e4  r7 = 0xed6059d8
         r8 = 0xc077d620  r9 = 0xc0693fa4
        r10 = 0xc077d624
db_command_loop() at db_command_loop+0x74
         pc = 0xc01406f0  lr = 0xc0143220 (db_trap+0x108)
         sp = 0xed605810  fp = 0xed605928
         r4 = 0x00000000  r5 = 0xc07885f0
         r6 = 0xc077d648 r10 = 0xc077d624
db_trap() at db_trap+0x108
         pc = 0xc0143220  lr = 0xc02eb6a0 (kdb_trap+0x184)
         sp = 0xed605930  fp = 0xed605958
         r4 = 0x00000000  r5 = 0x00000001
         r6 = 0xc077d648  r7 = 0xed6059d8
kdb_trap() at kdb_trap+0x184
         pc = 0xc02eb6a0  lr = 0xc055bb1c (undefinedinstruction+0x344)
         sp = 0xed605960  fp = 0xed6059d0
         r4 = 0x00000000  r5 = 0x00000000
         r6 = 0xc055b728  r7 = 0xe7ffffff
         r8 = 0xc4a8b360  r9 = 0xc02eadf8
        r10 = 0xed6059d8
undefinedinstruction() at undefinedinstruction+0x344
         pc = 0xc055bb1c  lr = 0xc05450f4 (exception_exit)
         sp = 0xed6059d8  fp = 0xed605a70
         r4 = 0xc05cbafb  r5 = 0x00000001
         r6 = 0xc076e0e0  r7 = 0xc076e278
         r8 = 0xed605ab4  r9 = 0xc078a3a0
        r10 = 0xc4a8b360
exception_exit() at exception_exit
         pc = 0xc05450f4  lr = 0xc02eade8 (kdb_enter+0x48)
         sp = 0xed605a68  fp = 0xed605a70
         r0 = 0xc077d634  r1 = 0x00000000
         r2 = 0xed60599c  r3 = 0xc05cfbc0
         r4 = 0xc05cbafb  r5 = 0x00000001
         r6 = 0xc076e0e0  r7 = 0xc076e278
         r8 = 0xed605ab4  r9 = 0xc078a3a0
        r10 = 0xc4a8b360 r12 = 0xc06aed58
$a.8() at $a.8
         pc = 0xc02eadfc  lr = 0xc02adf98 (vpanic+0x164)
         sp = 0xed605a78  fp = 0xed605a98
         r4 = 0x00000100 r10 = 0xc4a8b360
vpanic() at vpanic+0x164
         pc = 0xc02adf98  lr = 0xc02adfe4 (kproc_shutdown)
         sp = 0xed605aa0  fp = 0xed605aa8
         r4 = 0xc097e000  r5 = 0xed605bd4
         r6 = 0xed605bc8  r7 = 0xc05f7991
         r8 = 0xd1a2f000  r9 = 0x00000000
        r10 = 0x000007c0
kproc_shutdown() at kproc_shutdown
         pc = 0xc02adfe4  lr = 0xc0516afc (vm_fault_dirty)
         sp = 0xed605ab0  fp = 0xed605c28
         r4 = 0xed605bd4  r5 = 0xed605ab4
vm_fault_dirty() at vm_fault_dirty
         pc = 0xc0516afc  lr = 0xc0514dec (vm_fault+0x88)
         sp = 0xed605c30  fp = 0xed605c50
         r4 = 0x00000002  r5 = 0xc4a8b360
         r6 = 0xd1a2f000  r7 = 0x00000000
         r8 = 0x00000001  r9 = 0xc078e1f8
vm_fault() at vm_fault+0x88
         pc = 0xc0514dec  lr = 0xc055af04 (abort_handler+0x400)
         sp = 0xed605c58  fp = 0xed605cf8
         r4 = 0xed605d00  r5 = 0x00000000
         r6 = 0x00000007  r7 = 0x00000007
         r8 = 0xd1a2fb00  r9 = 0xc4a8b360
        r10 = 0x00000013
abort_handler() at abort_handler+0x400
         pc = 0xc055af04  lr = 0xc05450f4 (exception_exit)
         sp = 0xed605d00  fp = 0x00000000
         r4 = 0xc4a8b360  r5 = 0xed605e08
         r6 = 0xed605dc8  r7 = 0x00000000
         r8 = 0xed605e00  r9 = 0xc078a010
        r10 = 0xed605ea8
exception_exit() at exception_exit
         pc = 0xc05450f4  lr = 0xc053e3e0 (copyin+0x80)
         sp = 0xed605d94  fp = 0x00000000
         r0 = 0xd1a2fb00  r1 = 0xed605dc8
         r2 = 0x00000010  r3 = 0x00000001
         r4 = 0xc4a8b360  r5 = 0xed605e08
         r6 = 0xed605dc8  r7 = 0x00000000
         r8 = 0xed605e00  r9 = 0xc078a010
        r10 = 0xed605ea8 r12 = 0x00000001
copyin() at copyin+0x2e4
         pc = 0xc053e644  lr = 0xc053e3e0 (copyin+0x80)
         sp = 0xed605d94  fp = 0x00000000
copyin() at copyin+0x80
         pc = 0xc053e3e0  lr = 0xc053e3e0 (copyin+0x80)
         sp = 0xed605d94  fp = 0x00000000
Unwind failure (no registers changed)
db> 

Crash #2

Stress2 started as above.
20150901 15:14:56 all.cfg, elapsed 00:00:01
Last words from stress2 
17:18:09 Loop #1
    swap: run time  0+00:00:15, incarnations  23, load  80, verbose 1
 syscall: run time  0+00:00:15, incarnations  17, load 100, verbose 1

Console and backtrace follow:

smsc0: warning: MII is busy
smsc0: warning: Failed to write register 0x114
smsc0: warning: Failed to write register 0x114
smsc0: warning: Failed to write register 0x114
smsc0: warning: Failed to write register 0x114
smsc0: warning: Failed to write register 0x114
panic: vm_fault: fault on nofault entry, addr: c48ee000
cpuid = 3
KDB: enter: panic
[ thread pid 4860 tid 100234 ]
Stopped at      $d.7:   ldrb    r15, [r15, r15, ror r15]!
db> bt
Tracing pid 4860 tid 100234 td 0xc4c11a20
db_trace_self() at db_trace_self
         pc = 0xc0543a6c  lr = 0xc014103c (db_stack_trace+0x108)
         sp = 0xed7bef40  fp = 0xed7bef58
        r10 = 0xc07885f8
db_stack_trace() at db_stack_trace+0x108
         pc = 0xc014103c  lr = 0xc0140a88 (db_command+0x388)
         sp = 0xed7bef60  fp = 0xed7bf000
         r4 = 0x00000000  r5 = 0x00000000
         r6 = 0x00000000 r10 = 0xc07885f8
db_command() at db_command+0x388
         pc = 0xc0140a88  lr = 0xc01406f0 (db_command_loop+0x74)
         sp = 0xed7bf008  fp = 0xed7bf018
         r4 = 0xc05aa9d8  r5 = 0xc05cbaa6
         r6 = 0xc07885e4  r7 = 0xed7bf1e8
         r8 = 0xc077d620  r9 = 0xc0693fa4
        r10 = 0xc077d624
db_command_loop() at db_command_loop+0x74
         pc = 0xc01406f0  lr = 0xc0143220 (db_trap+0x108)
         sp = 0xed7bf020  fp = 0xed7bf138
         r4 = 0x00000000  r5 = 0xc07885f0
         r6 = 0xc077d648 r10 = 0xc077d624
db_trap() at db_trap+0x108
         pc = 0xc0143220  lr = 0xc02eb6a0 (kdb_trap+0x184)
         sp = 0xed7bf140  fp = 0xed7bf168
         r4 = 0x00000000  r5 = 0x00000001
         r6 = 0xc077d648  r7 = 0xed7bf1e8
kdb_trap() at kdb_trap+0x184
         pc = 0xc02eb6a0  lr = 0xc055bb1c (undefinedinstruction+0x344)
         sp = 0xed7bf170  fp = 0xed7bf1e0
         r4 = 0x00000000  r5 = 0x00000000
         r6 = 0xc055b728  r7 = 0xe7ffffff
         r8 = 0xc4c11a20  r9 = 0xc02eadf8
        r10 = 0xed7bf1e8
undefinedinstruction() at undefinedinstruction+0x344
         pc = 0xc055bb1c  lr = 0xc05450f4 (exception_exit)
         sp = 0xed7bf1e8  fp = 0xed7bf280
         r4 = 0xc05cbafb  r5 = 0x00000001
         r6 = 0xc076e0e0  r7 = 0xc076e278
         r8 = 0xed7bf2c4  r9 = 0xc078a3a0
        r10 = 0xc4c11a20
exception_exit() at exception_exit
         pc = 0xc05450f4  lr = 0xc02eade8 (kdb_enter+0x48)
         sp = 0xed7bf278  fp = 0xed7bf280
         r0 = 0xc077d634  r1 = 0x00000000
         r2 = 0xed7bf1ac  r3 = 0xc05cfbc0
         r4 = 0xc05cbafb  r5 = 0x00000001
         r6 = 0xc076e0e0  r7 = 0xc076e278
         r8 = 0xed7bf2c4  r9 = 0xc078a3a0
        r10 = 0xc4c11a20 r12 = 0xc06aed58
$a.8() at $a.8
         pc = 0xc02eadfc  lr = 0xc02adf98 (vpanic+0x164)
         sp = 0xed7bf288  fp = 0xed7bf2a8
         r4 = 0x00000100 r10 = 0xc4c11a20
vpanic() at vpanic+0x164
         pc = 0xc02adf98  lr = 0xc02adfe4 (kproc_shutdown)
         sp = 0xed7bf2b0  fp = 0xed7bf2b8
         r4 = 0xc097e000  r5 = 0xed7bf3e4
         r6 = 0xed7bf3d8  r7 = 0xc05f7991
         r8 = 0xc48ee000  r9 = 0x00000000
        r10 = 0x000007c0
kproc_shutdown() at kproc_shutdown
         pc = 0xc02adfe4  lr = 0xc0516afc (vm_fault_dirty)
         sp = 0xed7bf2c0  fp = 0xed7bf438
         r4 = 0xed7bf3e4  r5 = 0xed7bf2c4
vm_fault_dirty() at vm_fault_dirty
         pc = 0xc0516afc  lr = 0xc0514dec (vm_fault+0x88)
         sp = 0xed7bf440  fp = 0xed7bf460
         r4 = 0x00000002  r5 = 0xc4c11a20
         r6 = 0xc48ee000  r7 = 0x00000000
         r8 = 0x00000001  r9 = 0xc078e1f8
vm_fault() at vm_fault+0x88
         pc = 0xc0514dec  lr = 0xc055af04 (abort_handler+0x400)
         sp = 0xed7bf468  fp = 0xed7bf508
         r4 = 0xed7bf510  r5 = 0x00000000
         r6 = 0x0000000f  r7 = 0x0000000f
         r8 = 0xc48ee4fc  r9 = 0xc4c11a20
        r10 = 0x00000013
abort_handler() at abort_handler+0x400
         pc = 0xc055af04  lr = 0xc05450f4 (exception_exit)
         sp = 0xed7bf510  fp = 0x00000000
         r4 = 0xed7bfe08  r5 = 0xc4c11a20
         r6 = 0x00000000  r7 = 0x00000000
         r8 = 0xed7bfe00  r9 = 0xc078a010
        r10 = 0xed7bfea8
exception_exit() at exception_exit
         pc = 0xc05450f4  lr = 0xc053e3e0 (copyin+0x80)
         sp = 0xed7bf5a4  fp = 0x00000000
         r0 = 0xc48ee4fc  r1 = 0xed7bf5c0
         r2 = 0x00000004  r3 = 0x00000001
         r4 = 0xed7bfe08  r5 = 0xc4c11a20
         r6 = 0x00000000  r7 = 0x00000000
         r8 = 0xed7bfe00  r9 = 0xc078a010
        r10 = 0xed7bfea8 r12 = 0x00000001
copyin() at copyin+0x2e4
         pc = 0xc053e644  lr = 0xc053e3e0 (copyin+0x80)
         sp = 0xed7bf5a4  fp = 0x00000000
copyin() at copyin+0x80
         pc = 0xc053e3e0  lr = 0xc053e3e0 (copyin+0x80)
         sp = 0xed7bf5a4  fp = 0x00000000
Unwind failure (no registers changed)
db> 

Crash #3
Setup as before: Reboot, fsck-fy until filesystem was clean, reboot to multi-user
Stress2 invoked with sh ./run.shl-a, initial terminal output was
20150901 17:37:06 all.cfg, elapsed 00:00:00
     run: run time  0+00:05:00, incarnations   1, load 100, verbose 1
17:37:06 Loop #1
     udp: run time  0+00:02:00, incarnations  16, load  20, verbose 1
   mkdir: run time  0+00:02:00, incarnations   1, load  80, verbose 1
  mkfifo: run time  0+00:02:00, incarnations   8, load  20, verbose 1
 symlink: run time  0+00:02:00, incarnations   7, load  20, verbose 1
    swap: run time  0+00:02:00, incarnations  38, load  80, verbose 1
    mmap: run time  0+00:02:00, incarnations   2, load  20, verbose 1
    thr1: run time  0+00:02:00, incarnations  12, load  20, verbose 1

An intermediate output which looks odd:

19:08:25 Loop #1
   mkdir: run time  0+00:02:00, incarnations   5, load  80, verbose 1
     tcp: run time  0+00:02:00, incarnations  12, load  20, verbose 1
    thr1: run time  0+00:02:00, incarnations  16, load  20, verbose 1
      rw: run time  0+00:02:00, incarnations   6, load  70, verbose 1
   creat: run time  0+00:02:00, incarnations  12, load  80, verbose 1
tcp: write(3), tcp.c:146: Connection reset by peer



20150901 19:14:45 pty.cfg, elapsed 01:37:39
     run: run time  0+00:05:00, incarnations   1, load 100, verbose 1

Last words were:

20150901 19:30:31 syscall.cfg, elapsed 01:53:24
     run: run time  0+00:05:00, incarnations   1, load 100, verbose 1
19:30:36 Loop #1

Console output and backtrace:
smsc0: warning: MII is busy
smsc0: warning: Failed to write register 0x114
smsc0: warning: Failed to write register 0x114
panic: vm_fault: fault on nofault entry, addr: c8ace000
cpuid = 2
KDB: enter: panic
[ thread pid 3146 tid 100747 ]
Stopped at      $d.7:   ldrb    r15, [r15, r15, ror r15]!
db> bt
Tracing pid 3146 tid 100747 td 0xc4fe3360
db_trace_self() at db_trace_self
         pc = 0xc0543a6c  lr = 0xc014103c (db_stack_trace+0x108)
         sp = 0xeda676b0  fp = 0xeda676c8
        r10 = 0xc07885f8
db_stack_trace() at db_stack_trace+0x108
         pc = 0xc014103c  lr = 0xc0140a88 (db_command+0x388)
         sp = 0xeda676d0  fp = 0xeda67770
         r4 = 0x00000000  r5 = 0x00000000
         r6 = 0x00000000 r10 = 0xc07885f8
db_command() at db_command+0x388
         pc = 0xc0140a88  lr = 0xc01406f0 (db_command_loop+0x74)
         sp = 0xeda67778  fp = 0xeda67788
         r4 = 0xc05aa9d8  r5 = 0xc05cbaa6
         r6 = 0xc07885e4  r7 = 0xeda67958
         r8 = 0xc077d620  r9 = 0xc0693fa4
        r10 = 0xc077d624
db_command_loop() at db_command_loop+0x74
         pc = 0xc01406f0  lr = 0xc0143220 (db_trap+0x108)
         sp = 0xeda67790  fp = 0xeda678a8
         r4 = 0x00000000  r5 = 0xc07885f0
         r6 = 0xc077d648 r10 = 0xc077d624
db_trap() at db_trap+0x108
         pc = 0xc0143220  lr = 0xc02eb6a0 (kdb_trap+0x184)
         sp = 0xeda678b0  fp = 0xeda678d8
         r4 = 0x00000000  r5 = 0x00000001
         r6 = 0xc077d648  r7 = 0xeda67958
kdb_trap() at kdb_trap+0x184
         pc = 0xc02eb6a0  lr = 0xc055bb1c (undefinedinstruction+0x344)
         sp = 0xeda678e0  fp = 0xeda67950
         r4 = 0x00000000  r5 = 0x00000000
         r6 = 0xc055b728  r7 = 0xe7ffffff
         r8 = 0xc4fe3360  r9 = 0xc02eadf8
        r10 = 0xeda67958
undefinedinstruction() at undefinedinstruction+0x344
         pc = 0xc055bb1c  lr = 0xc05450f4 (exception_exit)
         sp = 0xeda67958  fp = 0xeda679f0
         r4 = 0xc05cbafb  r5 = 0x00000001
         r6 = 0xc076e0e0  r7 = 0xc076e278
         r8 = 0xeda67a34  r9 = 0xc078a3a0
        r10 = 0xc4fe3360
exception_exit() at exception_exit
         pc = 0xc05450f4  lr = 0xc02eade8 (kdb_enter+0x48)
         sp = 0xeda679e8  fp = 0xeda679f0
         r0 = 0xc077d634  r1 = 0x00000000
         r2 = 0xeda6791c  r3 = 0xc05cfbc0
         r4 = 0xc05cbafb  r5 = 0x00000001
         r6 = 0xc076e0e0  r7 = 0xc076e278
         r8 = 0xeda67a34  r9 = 0xc078a3a0
        r10 = 0xc4fe3360 r12 = 0xc06aed58
$a.8() at $a.8
         pc = 0xc02eadfc  lr = 0xc02adf98 (vpanic+0x164)
         sp = 0xeda679f8  fp = 0xeda67a18
         r4 = 0x00000100 r10 = 0xc4fe3360
vpanic() at vpanic+0x164
         pc = 0xc02adf98  lr = 0xc02adfe4 (kproc_shutdown)
         sp = 0xeda67a20  fp = 0xeda67a28
         r4 = 0xc097e000  r5 = 0xeda67b54
         r6 = 0xeda67b48  r7 = 0xc05f7991
         r8 = 0xc8ace000  r9 = 0x00000000
        r10 = 0x000007c0
kproc_shutdown() at kproc_shutdown
         pc = 0xc02adfe4  lr = 0xc0516afc (vm_fault_dirty)
         sp = 0xeda67a30  fp = 0xeda67ba8
         r4 = 0xeda67b54  r5 = 0xeda67a34
vm_fault_dirty() at vm_fault_dirty
         pc = 0xc0516afc  lr = 0xc0514dec (vm_fault+0x88)
         sp = 0xeda67bb0  fp = 0xeda67bd0
         r4 = 0x00000002  r5 = 0xc4fe3360
         r6 = 0xc8ace000  r7 = 0x00000000
         r8 = 0x00000001  r9 = 0xc078e1f8
vm_fault() at vm_fault+0x88
         pc = 0xc0514dec  lr = 0xc055af04 (abort_handler+0x400)
         sp = 0xeda67bd8  fp = 0xeda67c78
         r4 = 0xeda67c80  r5 = 0x00000000
         r6 = 0x00000007  r7 = 0x00000007
         r8 = 0xc8ace554  r9 = 0xc4fe3360
        r10 = 0x00000013
abort_handler() at abort_handler+0x400
         pc = 0xc055af04  lr = 0xc05450f4 (exception_exit)
         sp = 0xeda67c80  fp = 0x00000000
         r4 = 0xeda67e08  r5 = 0xc4fe3360
         r6 = 0x00000000  r7 = 0x00000000
         r8 = 0xeda67e00  r9 = 0xc078a010
        r10 = 0xeda67ea8
exception_exit() at exception_exit
         pc = 0xc05450f4  lr = 0xc053e3e0 (copyin+0x80)
         sp = 0xeda67d14  fp = 0x00000000
         r0 = 0xc8ace554  r1 = 0xeda67d44
         r2 = 0x0000001c  r3 = 0x00000001
         r4 = 0xeda67e08  r5 = 0xc4fe3360
         r6 = 0x00000000  r7 = 0x00000000
         r8 = 0xeda67e00  r9 = 0xc078a010
        r10 = 0xeda67ea8 r12 = 0x00000001
copyin() at copyin+0x2e4
         pc = 0xc053e644  lr = 0xc053e3e0 (copyin+0x80)
         sp = 0xeda67d14  fp = 0x00000000
copyin() at copyin+0x80
         pc = 0xc053e3e0  lr = 0xc053e3e0 (copyin+0x80)
         sp = 0xeda67d14  fp = 0x00000000
Unwind failure (no registers change)
db> 
Crash #4

Setup as before, start was
bob@www:~/stress2 % sh ./run.sh -a
20150901 20:14:22 all.cfg, elapsed 00:00:00
     run: run time  0+00:05:00, incarnations   1, load 100, verbose 1
20:14:23 Loop #1
     udp: run time  0+00:02:00, incarnations  20, load  20, verbose 1
 badcode: run time  0+00:02:00, incarnations  18, load  20, verbose 1
   creat: run time  0+00:02:00, incarnations  16, load  80, verbose 1
    mmap: run time  0+00:02:00, incarnations   8, load  20, verbose 1
    swap: run time  0+00:02:00, incarnations  10, load  80, verbose 1
   mkdir: run time  0+00:02:00, incarnations   5, load  80, verbose 1
  rename: run time  0+00:02:00, incarnations  10, load  20, verbose 1

Curious development at:
20:36:22 Loop #2
      rw: run time  0+00:02:00, incarnations   4, load 100, verbose 1
   mkdir: run time  0+00:02:00, incarnations   3, load  80, verbose 1

/tmp: write failed, filesystem is full
rw: write(p01317), rw.c:152: No space left on device

Last words were:
23:16:39 Loop #1
    swap: run time  0+00:00:15, incarnations  16, load  80, verbose 1
 syscall: run time  0+00:00:15, incarnations  14, load 100, verbose 1

Console and backtrace follow.
 warning: MII is busy
smsc0: warning: Failed to write register 0x114
smsc0: warning: Failed to read register 0x114
smsc0: warning: MII read timeout
panic: vm_fault: fault on nofault entry, addr: c42c6000
cpuid = 0
KDB: enter: panic
[ thread pid 3456 tid 100172 ]
Stopped at      $d.7:   ldrb    r15, [r15, r15, ror r15]!
db> bt
Tracing pid 3456 tid 100172 td 0xc4a64360
db_trace_self() at db_trace_self
         pc = 0xc0543a6c  lr = 0xc014103c (db_stack_trace+0x108)
         sp = 0xed5da638  fp = 0xed5da650
        r10 = 0xc07885f8
db_stack_trace() at db_stack_trace+0x108
         pc = 0xc014103c  lr = 0xc0140a88 (db_command+0x388)
         sp = 0xed5da658  fp = 0xed5da6f8
         r4 = 0x00000000  r5 = 0x00000000
         r6 = 0x00000000 r10 = 0xc07885f8
db_command() at db_command+0x388
         pc = 0xc0140a88  lr = 0xc01406f0 (db_command_loop+0x74)
         sp = 0xed5da700  fp = 0xed5da710
         r4 = 0xc05aa9d8  r5 = 0xc05cbaa6
         r6 = 0xc07885e4  r7 = 0xed5da8e0
         r8 = 0xc077d620  r9 = 0xc0693fa4
        r10 = 0xc077d624
db_command_loop() at db_command_loop+0x74
         pc = 0xc01406f0  lr = 0xc0143220 (db_trap+0x108)
         sp = 0xed5da718  fp = 0xed5da830
         r4 = 0x00000000  r5 = 0xc07885f0
         r6 = 0xc077d648 r10 = 0xc077d624
db_trap() at db_trap+0x108
         pc = 0xc0143220  lr = 0xc02eb6a0 (kdb_trap+0x184)
         sp = 0xed5da838  fp = 0xed5da860
         r4 = 0x00000000  r5 = 0x00000001
         r6 = 0xc077d648  r7 = 0xed5da8e0
kdb_trap() at kdb_trap+0x184
         pc = 0xc02eb6a0  lr = 0xc055bb1c (undefinedinstruction+0x344)
         sp = 0xed5da868  fp = 0xed5da8d8
         r4 = 0x00000000  r5 = 0x00000000
         r6 = 0xc055b728  r7 = 0xe7ffffff
         r8 = 0xc4a64360  r9 = 0xc02eadf8
        r10 = 0xed5da8e0
undefinedinstruction() at undefinedinstruction+0x344
         pc = 0xc055bb1c  lr = 0xc05450f4 (exception_exit)
         sp = 0xed5da8e0  fp = 0xed5da978
         r4 = 0xc05cbafb  r5 = 0x00000001
         r6 = 0xc076e0e0  r7 = 0xc076e278
         r8 = 0xed5da9bc  r9 = 0xc078a3a0
        r10 = 0xc4a64360
exception_exit() at exception_exit
         pc = 0xc05450f4  lr = 0xc02eade8 (kdb_enter+0x48)
         sp = 0xed5da970  fp = 0xed5da978
         r0 = 0xc077d634  r1 = 0x00000000
         r2 = 0xed5da8a4  r3 = 0xc05cfbc0
         r4 = 0xc05cbafb  r5 = 0x00000001
         r6 = 0xc076e0e0  r7 = 0xc076e278
         r8 = 0xed5da9bc  r9 = 0xc078a3a0
        r10 = 0xc4a64360 r12 = 0xc06aed58
$a.8() at $a.8
         pc = 0xc02eadfc  lr = 0xc02adf98 (vpanic+0x164)
         sp = 0xed5da980  fp = 0xed5da9a0
         r4 = 0x00000100 r10 = 0xc4a64360
vpanic() at vpanic+0x164
         pc = 0xc02adf98  lr = 0xc02adfe4 (kproc_shutdown)
         sp = 0xed5da9a8  fp = 0xed5da9b0
         r4 = 0xc097e000  r5 = 0xed5daadc
         r6 = 0xed5daad0  r7 = 0xc05f7991
         r8 = 0xc42c6000  r9 = 0x00000000
        r10 = 0x000007c0
kproc_shutdown() at kproc_shutdown
         pc = 0xc02adfe4  lr = 0xc0516afc (vm_fault_dirty)
         sp = 0xed5da9b8  fp = 0xed5dab30
         r4 = 0xed5daadc  r5 = 0xed5da9bc
vm_fault_dirty() at vm_fault_dirty
         pc = 0xc0516afc  lr = 0xc0514dec (vm_fault+0x88)
         sp = 0xed5dab38  fp = 0xed5dab58
         r4 = 0x00000002  r5 = 0xc4a64360
         r6 = 0xc42c6000  r7 = 0x00000000
         r8 = 0x00000001  r9 = 0xc078e1f8
vm_fault() at vm_fault+0x88
         pc = 0xc0514dec  lr = 0xc055af04 (abort_handler+0x400)
         sp = 0xed5dab60  fp = 0xed5dac00
         r4 = 0xed5dac08  r5 = 0x00000000
         r6 = 0x0000000f  r7 = 0x0000000f
         r8 = 0xc42c6065  r9 = 0xc4a64360
        r10 = 0x00000013
abort_handler() at abort_handler+0x400
         pc = 0xc055af04  lr = 0xc05450f4 (exception_exit)
         sp = 0xed5dac08  fp = 0xed5dad20
         r4 = 0xed5daea8  r5 = 0xc0541b90
         r6 = 0x00000000  r7 = 0x00000000
         r8 = 0xc4a64360  r9 = 0xc078a010
        r10 = 0xed5dad98
exception_exit() at exception_exit
         pc = 0xc05450f4  lr = 0xc034a7d4 (namei+0x108)
         sp = 0xed5dac9c  fp = 0xed5dad20
         r0 = 0xc42c6065  r1 = 0xc5372400
         r2 = 0x00000400  r3 = 0xed5dad98
         r4 = 0xed5daea8  r5 = 0xc0541b90
         r6 = 0x00000000  r7 = 0x00000000
         r8 = 0xc4a64360  r9 = 0xc078a010
        r10 = 0xed5dad98 r12 = 0x00000001
copyinstr() at copyinstr+0x28
         pc = 0xc0541af0  lr = 0xc034a7d4 (namei+0x108)
         sp = 0xed5dac9c  fp = 0xed5dad20
namei() at namei+0x108
         pc = 0xc034a7d4  lr = 0xc0338d84 (sys___acl_aclcheck_file+0x50)
         sp = 0xed5dad28  fp = 0xed5dade8
         r4 = 0xc4a64360  r5 = 0xed5dae08
         r6 = 0xed5dad40  r7 = 0x00000000
         r8 = 0xed5dae00  r9 = 0xc078a010
        r10 = 0xbfbffac0
sys___acl_aclcheck_file() at sys___acl_aclcheck_file+0x50
         pc = 0xc0338d84  lr = 0xc055a760 (swi_handler+0x2c8)
         sp = 0xed5dadf0  fp = 0xed5dae50
         r4 = 0xc4a64360  r5 = 0xc4a74700
         r6 = 0x00000000 r10 = 0xbfbffac0
swi_handler() at swi_handler+0x2c8
         pc = 0xc055a760  lr = 0xc0545084 (swi_exit)
         sp = 0xed5dae58  fp = 0xbfbff940
         r4 = 0x4faeba54  r5 = 0x46ba5d64
         r6 = 0xbfbff8f0  r7 = 0x00000000
         r8 = 0xbfbffac8  r9 = 0x00000000
        r10 = 0xbfbffac0
swi_exit() at swi_exit
         pc = 0xc0545084  lr = 0xc0545084 (swi_exit)
         sp = 0xed5dae58  fp = 0xbfbff940
db> 

One possible experiment is to separate the stress tests, to see if the swap
exercises alone can generate the crash. Are there more fruitful things to try?

Thanks for reading,

bob prohaska




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20150903001037.GC33831>