From owner-freebsd-fs@freebsd.org Tue Aug 2 05:37:37 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ABC82BAC9FB for ; Tue, 2 Aug 2016 05:37:37 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-vk0-x22a.google.com (mail-vk0-x22a.google.com [IPv6:2607:f8b0:400c:c05::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6DEDF1E63 for ; Tue, 2 Aug 2016 05:37:37 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: by mail-vk0-x22a.google.com with SMTP id w127so114954845vkh.2 for ; Mon, 01 Aug 2016 22:37:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=r2qXWqCsO2BfEl7dMJ3K/cFdMkKMMB2dV3U3PQg+CNQ=; b=bX6YEf+qrkO4qmROS12rdsXnjV1H4dAmnBadwvAeGv/8lWV6AHPUyCv18JmoWQEhOI WcZK6JFdz6voCvudT5wkR5fv1lYUKx62SlyMyTM8RRt1wq7aZpzIUXvhA7D6fS/U4b+c vzJh2TofwKtEeV1yB2k9S3ZChBMRSiFZR/38eF6ae2yAachxE4TgQMTRUorAJgblw8zV C2hDkRp6Nlo2o9wxHOl4Afqcx2TyOdo+1kExUBS16HAFJYChxVii1S3fle3G5FN+KMvj Qlbk22bgSErYBDaufU/3fO/rKcsR6x7HwRSCDgl5cLGu6zb2ui6gLwnMaUxuzhtxZDbl C/6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=r2qXWqCsO2BfEl7dMJ3K/cFdMkKMMB2dV3U3PQg+CNQ=; b=LI8WqzgKFwTpoGXT+64iUiK5e1C8cI/Jf0am5veL9dHnPFTj//hfDjASLK2mnemXrz XNArryZutASaNsECekUr7jWaGXP03rtI9Cm8gzY1krgFxN7AnLBE9czLKbkEqTivHtbp 7c6KJ1BK1CVDyUBG6L3yflPi4ghCyujhHcfmrfe6Qmpa9PGvDP8Y2WPHUNg0CrJ90IO2 hzB0bcH6IqllfsoMfEQ5RbLZVG6KPEzFfjIvHnrDBa6908m1VSpPSY5YtYmgly12yzsv OWRVh71qpZbSh9Z97L8VWG1NsEkKEL/cKK1OJjqP1ZlqYsxHJF/QC6YcRgpQYqjLHdn+ JJBg== X-Gm-Message-State: AEkooutdnzM+y9evionsvxS13iolui/4HjXkdGXwiy+oEzayn21gcQ7DjamqG3B641ztKMdGP1LqfNqDRTxKrw== X-Received: by 10.31.47.143 with SMTP id v137mr25040427vkv.58.1470116256231; Mon, 01 Aug 2016 22:37:36 -0700 (PDT) MIME-Version: 1.0 Received: by 10.176.6.5 with HTTP; Mon, 1 Aug 2016 22:37:35 -0700 (PDT) From: Zaphod Beeblebrox Date: Tue, 2 Aug 2016 01:37:35 -0400 Message-ID: Subject: Crashes (with dumps) in zap_leaf.c To: freebsd-fs Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2016 05:37:37 -0000 I'm getting multiple crashes that look rather like: #0 0xffffffff8098e390 at kdb_backtrace+0x60 #1 0xffffffff80951066 at vpanic+0x126 #2 0xffffffff80950f33 at panic+0x43 #3 0xffffffff80bcfa4c at vm_fault_hold+0x1b2c #4 0xffffffff80bcded7 at vm_fault+0x77 #5 0xffffffff80d5612c at trap_pfault+0x19c #6 0xffffffff80d558fa at trap+0x47a #7 0xffffffff80d3b8d2 at calltrap+0x8 #8 0xffffffff81a49a5a at zap_entry_create+0x27a #9 0xffffffff81a45eee at fzap_add_cd+0xde #10 0xffffffff81a4c051 at zap_add+0x101 #11 0xffffffff81a6bfb5 at zfs_link_create+0x415 #12 0xffffffff81a87a34 at zfs_freebsd_rename+0xac4 #13 0xffffffff80e81e1b at VOP_RENAME_APV+0xab #14 0xffffffff80a03476 at kern_renameat+0x4a6 #15 0xffffffff80d5694f at amd64_syscall+0x40f #16 0xffffffff80d3bbbb at Xfast_syscall+0xfb Analyzing one of those crashes: #0 doadump (textdump=) at pcpu.h:219 219 pcpu.h: No such file or directory. in pcpu.h (kgdb) bt #0 doadump (textdump=) at pcpu.h:219 #1 0xffffffff80950cc2 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:486 #2 0xffffffff809510a5 in vpanic (fmt=, ap=) at /usr/src/sys/kern/kern_shutdown.c:889 #3 0xffffffff80950f33 in panic (fmt=0x0) at /usr/src/sys/kern/kern_shutdown.c:818 #4 0xffffffff80bcfa4c in vm_fault_hold (map=, vaddr=, fault_type=, fault_flags=, m_hold=) at /usr/src/sys/vm/vm_fault.c:329 #5 0xffffffff80bcded7 in vm_fault (map=0xfffff80002000000, vaddr=, fault_type=1 '\001', fault_flags=0) at /usr/src/sys/vm/vm_fault.c:273 #6 0xffffffff80d5612c in trap_pfault (frame=0xfffffe0c56854320, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:757 #7 0xffffffff80d558fa in trap (frame=0xfffffe0c56854320) at /usr/src/sys/amd64/amd64/trap.c:447 #8 0xffffffff80d3b8d2 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:236 #9 0xffffffff81a494c5 in zap_leaf_array_create (l=0xfffff8008b3a3b00, buf=, integer_size=1, num_integers=) ---Type to continue, or q to quit--- at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zap_leaf.c:198 #10 0xffffffff81a49a5a in zap_entry_create (l=0xfffff8008b3a3b00, zn=0xfffff8008bf19e00, cd=Cannot access memory at address 0x0 ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zap_leaf.c:644 #11 0xffffffff81a45eee in fzap_add_cd (zn=0xfffff8008bf19e00, integer_size=, num_integers=1, val=0xfffffe0c568546d0, cd=4294967295, tx=0xfffff80787758c00) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c:814 #12 0xffffffff81a4c051 in zap_add (os=, zapobj=, key=, integer_size=, num_integers=, val=, tx=) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zap_micro.c:1014 #13 0xffffffff81a6bfb5 in zfs_link_create (dl=0xfffff800966cce00, zp=, tx=0xfffff80787758c00, flag=) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_dir.c:767 #14 0xffffffff81a87a34 in zfs_freebsd_rename (ap=) ---Type to continue, or q to quit--- at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:4067 #15 0xffffffff80e81e1b in VOP_RENAME_APV (vop=, a=) at vnode_if.c:1546 #16 0xffffffff80a03476 in kern_renameat (td=, oldfd=, old=, newfd=, new=, pathseg=) at vnode_if.h:636 #17 0xffffffff80d5694f in amd64_syscall (td=0xfffff8048f7fd000, traced=0) at subr_syscall.c:141 #18 0xffffffff80d3bbbb in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:396 #19 0x000000080381fbaa in ?? () Previous frame inner to this frame (corrupt stack?) Current language: auto; currently minimal (kgdb) frame 9 #9 0xffffffff81a494c5 in zap_leaf_array_create (l=0xfffff8008b3a3b00, buf=, integer_size=1, num_integers=) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zap_leaf.c:198 198 ZAP_LEAF_CHUNK(l, chunk).l_free.lf_next; (kgdb) p chunk No symbol "chunk" in current context. ... now I don't debug kernels too often, but I don't know why I can't see chunk. It's declared as an int in this function. Does this give anyone insight?