From owner-freebsd-arm@FreeBSD.ORG Sat Oct 10 18:16:48 2009 Return-Path: Delivered-To: arm@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E7B15106568B for ; Sat, 10 Oct 2009 18:16:48 +0000 (UTC) (envelope-from xcllnt@mac.com) Received: from asmtpout028.mac.com (asmtpout028.mac.com [17.148.16.103]) by mx1.freebsd.org (Postfix) with ESMTP id D74458FC0C for ; Sat, 10 Oct 2009 18:16:48 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; charset=us-ascii; format=flowed; delsp=yes Received: from macbook-pro.lan.xcllnt.net (mail.xcllnt.net [75.101.29.67]) by asmtp028.mac.com (Sun Java(tm) System Messaging Server 6.3-8.01 (built Dec 16 2008; 32bit)) with ESMTPSA id <0KRB00ECY9FZ2Y60@asmtp028.mac.com> for arm@freebsd.org; Sat, 10 Oct 2009 11:16:48 -0700 (PDT) From: Marcel Moolenaar Date: Sat, 10 Oct 2009 11:16:47 -0700 Message-id: To: arm@freebsd.org X-Mailer: Apple Mail (2.1076) Cc: Subject: panic: _mtx_lock_sleep: recursed on non-recursive mutex pmap X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting FreeBSD to the StrongARM Processor List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Oct 2009 18:16:49 -0000 Hardware: o Marvell DB-78XXX o 500GB USB disk (da0, mounted /) o 500GB SATA disk (ad0, mounted /mnt) o 1Gb ethernet (NFS mounted /nfs) While running rsync from da0p1 to ad0p1 without crossing file systems (i.e. NFS not involved) I got the following panic: # sudo rsync -alHv --delete --one-file-system / /mnt/ sending incremental file list ./ .cshrc .profile ... usr/local/man/man1/pod2html.1.gz usr/local/man/man1/pod2latex.1.gz usr/local/man/man1/pod2man.1.gz panic: _mtx_lock_sleep: recursed on non-recursive mutex pmap @ / zmirror/nfs/freebsd/base/head/sys/arm/arm/pmap.c:1916 KDB: enter: panic [thread pid 857 tid 100051 ] Stopped at kdb_enter+0x48: ldrb r15, [r15, r15, ror r15]! db> It looks like cron is the culprit: db> show alllocks Process 857 (cron) thread 0xc3984d80 (100051) exclusive sleep mutex pmap (pmap) r = 0 (0xc3a330b0) locked @ /zmirror/ nfs/freebsd/base/head/sys/arm/arm/pmap.c:3254 exclusive sleep mutex vm page queue mutex (vm page queue mutex) r = 1 (0xc0d897c0) locked @ /zmirror/nfs/freebsd/base/head/sys/arm/arm/ pmap.c:1915 exclusive sx user map (user map) r = 0 (0xc3a7752c) locked @ /zmirror/ nfs/freebsd/base/head/sys/vm/vm_map.c:2981 exclusive sx user map (user map) r = 0 (0xc3a33048) locked @ /zmirror/ nfs/freebsd/base/head/sys/vm/vm_map.c:2973 db> bt Tracing pid 857 tid 100051 td 0xc3984d80 db_trace_thread() at db_trace_thread+0x10 scp=0xc0b47420 rlv=0xc0919f8c (db_command_init+0x4d4) rsp=0xd1c6b83c rfp=0xd1c6b85c r10=0x00000001 r9=0xc0c290b0 r8=0xc0c20e4c r7=0xc0c20620 r6=0x00000010 r5=0x00000000 r4=0xc3984d80 db_command_init() at db_command_init+0x454 scp=0xc0919f0c rlv=0xc0919738 (db_skip_to_eol+0x38c) rsp=0xd1c6b860 rfp=0xd1c6b904 r6=0x00000002 r5=0x00000000 r4=0xc0bfb5c8 db_skip_to_eol() at db_skip_to_eol+0x1d0 scp=0xc091957c rlv=0xc0919954 (db_command_loop+0x50) rsp=0xd1c6b908 rfp=0xd1c6b918 r10=0x00000001 r8=0x00000000 r7=0xd1c6bae4 r6=0xc0c290ac r5=0x000000c0 r4=0xc0c2061c db_command_loop() at db_command_loop+0x18 scp=0xc091991c rlv=0xc091bb9c (X_db_sym_numargs+0xa0) rsp=0xd1c6b91c rfp=0xd1c6ba38 r4=0xd1c6b920 X_db_sym_numargs() at X_db_sym_numargs+0x18 scp=0xc091bb14 rlv=0xc09fe9a8 (kdb_trap+0xb0) rsp=0xd1c6ba3c rfp=0xd1c6ba64 r4=0x000000c0 kdb_trap() at kdb_trap+0x10 scp=0xc09fe908 rlv=0xc0b57ed0 (undefinedinstruction+0x124) rsp=0xd1c6ba68 rfp=0xd1c6bae0 r10=0xc3984d80 r9=0xc0d94e80 r8=0xc09fe69c r7=0xe7ffffff r6=0xd1c6bae4 r5=0x00000000 r4=0x00000000 undefinedinstruction() at undefinedinstruction+0x10 scp=0xc0b57dbc rlv=0xc0b48ea8 (address_exception_entry+0x50) rsp=0xd1c6bae4 rfp=0xd1c6bb44 r10=0xc0d94e80 r9=0x00000001 r8=0x0000077c r7=0xc3984d80 r6=0xc0c252f8 r5=0xffff1004 r4=0x00000000 kdb_enter() at kdb_enter+0x14 scp=0xc09fe668 rlv=0xc09d2304 (panic+0xa0) rsp=0xd1c6bb48 rfp=0xd1c6bb5c r5=0xc0b936f8 r4=0x00000100 panic() at panic+0x1c scp=0xc09d2280 rlv=0xc09c410c (_mtx_lock_sleep+0x130) rsp=0xd1c6bb70 rfp=0xd1c6bb88 _mtx_lock_sleep() at _mtx_lock_sleep+0x10 scp=0xc09c3fec rlv=0xc09c41ec (_mtx_lock_flags+0xd4) rsp=0xd1c6bb8c rfp=0xd1c6bbb4 r6=0x00000000 r5=0xc3a330b0 r4=0x00000000 _mtx_lock_flags() at _mtx_lock_flags+0x10 scp=0xc09c4128 rlv=0xc0b4dfe4 (pmap_fault_fixup+0x48) rsp=0xd1c6bbb8 rfp=0xd1c6bbec r10=0xd1c6bc94 r8=0x00000001 r7=0xc3984d80 r6=0x00019000 r5=0xc3a330b0 r4=0x00000000 pmap_fault_fixup() at pmap_fault_fixup+0x10 scp=0xc0b4dfac rlv=0xc0b571a8 (data_abort_handler+0x16c) rsp=0xd1c6bbf0 rfp=0xd1c6bc90 r10=0xd1c6bc94 r9=0xd1c6bef8 r8=0x00000001 r7=0xc3984d80 r6=0x000e7000 r5=0x00019000 r4=0x00000007 data_abort_handler() at data_abort_handler+0x10 scp=0xc0b5704c rlv=0xc0b48ea8 (address_exception_entry+0x50) rsp=0xd1c6bc94 rfp=0xd1c6bd20 r10=0xc3a330b0 r9=0x00100000 r8=0x000e9000 r7=0x00019000 r6=0x000e7000 r5=0xffff1004 r4=0x00017000 pmap_protect() at pmap_protect+0x10 scp=0xc0b50970 rlv=0xc0b2f460 (vmspace_fork+0x420) rsp=0xd1c6bd24 rfp=0xd1c6bd60 r10=0xc3a33000 r9=0x00008000 r8=0xc3a774e4 r7=0x00000001 r6=0xc3a1c168 r5=0xc3a5cca8 r4=0x00000000 vmspace_fork() at vmspace_fork+0x10 scp=0xc0b2f050 rlv=0xc09ad6cc (fork1+0x100) rsp=0xd1c6bd64 rfp=0xd1c6bdbc r10=0xc397f834 r9=0xc3984d80 r8=0x00000014 r7=0xd1c6beac r6=0xc3a30af0 r5=0x00000002 r4=0x00000000 fork1() at fork1+0x10 scp=0xc09ad5dc rlv=0xc09ae848 (fork+0x24) rsp=0xd1c6bdc0 rfp=0xd1c6bdd8 r10=0xc397f834 r9=0x00000000 r8=0xc0c05360 r7=0xd1c6beac r6=0x00000002 r5=0xc3984d80 r4=0xc3984d80 fork() at fork+0x10 scp=0xc09ae834 rlv=0xc0b57674 (swi_handler+0x128) rsp=0xd1c6bddc rfp=0xd1c6bea8 r4=0x00000000 swi_handler() at swi_handler+0x10 scp=0xc0b5755c rlv=0xc0b48c80 (swi_entry+0x40) rsp=0xd1c6beac rfp=0xbfffed20 r10=0x0000000b r9=0x00000000 r8=0x00000000 r7=0x000187c8 r6=0x00000000 r5=0x20215028 r4=0x20216040 db> ps pid ppid pgrp uid state wmesg wchan cmd 970 969 968 0 R+ rsync 969 968 968 0 S+ select 0xc3a21324 rsync 968 907 968 0 R+ rsync 907 906 907 501 S+ pause 0xc38d388c tcsh 906 1 906 0 Ss+ wait 0xc3a302bc login 884 1 884 0 Ss select 0xc38f8be4 inetd 857 1 857 0 Rs CPU 0 cron 851 1 851 25 Ss pause 0xc3a30058 sendmail 847 1 847 0 Ss select 0xc3a203a4 sendmail 842 1 842 0 Ss select 0xc3a201a4 sshd 811 1 811 1 Ss sbwait 0xc3a22254 rwhod 781 1 781 0 Ss select 0xc38c7764 ntpd 717 1 717 0 Ss rpcsvc 0xc38ec910 NLM: master 711 1 711 0 Ss select 0xc38dfae4 rpc.statd 636 1 636 0 Ss select 0xc38ec624 ypbind 618 1 618 0 Ss select 0xc38f8064 rpcbind 598 1 598 0 Ss select 0xc38ecb64 syslogd 482 1 482 0 Ss select 0xc37589e4 devd 481 1 481 65 Ss select 0xc386c6e4 dhclient 441 1 441 0 Ss select 0xc38ecd24 dhclient 16 0 0 0 SL - 0xc0c25554 [schedcpu] 15 0 0 0 SL syncer 0xc0d83f38 [syncer] 14 0 0 0 SL vlruwt 0xc38d3000 [vnlru] 9 0 0 0 SL psleep 0xc0d83cb4 [bufdaemon] 8 0 0 0 SL pgzero 0xc0d89d4c [pagezero] 7 0 0 0 SL pollid 0xc0c24de4 [idlepoll] 6 0 0 0 SL psleep 0xc0d89848 [pagedaemon] 13 0 0 0 SL (threaded) [usb] 100032 D - 0xc3679d0c [usbus2] 100031 D - 0xc3679cdc [usbus2] 100030 D - 0xc3679cac [usbus2] 100029 D - 0xc3679c7c [usbus2] 100027 D - 0xc3657d0c [usbus1] 100026 D - 0xc3657cdc [usbus1] 100025 D - 0xc3657cac [usbus1] 100024 D - 0xc3657c7c [usbus1] 100022 D - 0xc3634d0c [usbus0] 100021 D - 0xc3634cdc [usbus0] 100020 D - 0xc3634cac [usbus0] 100019 D - 0xc3634c7c [usbus0] 5 0 0 0 SL ccb_scan 0xc0c1fa24 [xpt_thrd] 12 0 0 0 SL - 0xc0c25554 [yarrow] 4 0 0 0 SL - 0xc0c22de8 [g_down] 3 0 0 0 SL - 0xc0c22de4 [g_up] 2 0 0 0 SL - 0xc0c22ddc [g_event] 11 0 0 0 WL (threaded) [intr] 100037 I [intr26: sata0] 100036 I [intr46: mge1] 100035 I [intr45: mge1] 100034 I [intr42: mge0] 100033 I [intr41: mge0] 100028 I [intr18: ehci2] 100023 I [intr17: ehci1] 100018 I [intr16: ehci0] 100017 I [swi0: uart uart] 100015 I [swi2: cambio] 100013 I [swi6: task queue] 100012 I [swi6: Giant taskq] 100010 I [swi5: +] 100005 I [swi4: clock] 100004 I [swi3: vm] 100003 I [swi1: netisr 0] 10 0 0 0 RL [idle] 1 0 1 0 SLs wait 0xc35bf000 [init] 0 0 0 0 SLs (threaded) [kernel] 100016 D - 0xc361cd80 [kqueue taskq] 100011 D - 0xc361d140 [thread taskq] 100000 D sched 0xc0c22e38 [swapper] db> show intrcnt irq8: timer0 90926 irq12: uart0 37904 irq16: ehci0 75948 irq41: mge0 679 irq42: mge0 660 irq26: sata0 265343 FYI, -- Marcel Moolenaar xcllnt@mac.com