Date: Tue, 30 Jun 2020 20:33:39 +0000 From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 247668] Page fault in zfsctl_snapdir_getattr Message-ID: <bug-247668-227@https.bugs.freebsd.org/bugzilla/>
index | next in thread | raw e-mail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=247668 Bug ID: 247668 Summary: Page fault in zfsctl_snapdir_getattr Product: Base System Version: 12.1-STABLE Hardware: Any OS: Any Status: New Severity: Affects Many People Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: asomers@FreeBSD.org On a very heavily loaded server I observed the following kernel-mode page fault. The offending process was a "procstat -af", which did VOP_GETATTR on every open file descriptor on the whole system, including the .zfs/snapshot directories. On one of those, it called dsl_dataset_phys, which tried to dereference a null pointer. There were also 5 "zfs destroy" processes, and dozens of "zfs list" and "zfs recv" running concurrently. I suspect that zfsctl_snapdir_getattr is missing some lock when it checks dsl_dataset_phys, while trying to calculate the directory's nlink attribute. But it's not clear what lock it ought to hold. It's worth noting that ZoL doesn't have this problem because it doesn't even try to calculate nlink; instead it always returns "2". Sadly, I haven't been able to reproduce the issue on any non-production machine. The server in question is running 12-STABLE at svn r346022. #1 doadump (textdump=<optimized out>) at /usr/src/sys/kern/kern_shutdown.c:371 #2 0xffffffff80bbe655 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:451 #3 0xffffffff80bbea96 in vpanic (fmt=<optimized out>, ap=<optimized out>) at /usr/src/sys/kern/kern_shutdown.c:880 #4 0xffffffff80bbe8b3 in panic (fmt=<unavailable>) at /usr/src/sys/kern/kern_shutdown.c:807 #5 0xffffffff81090310 in trap_fatal (frame=0xfffffe04b95c08a0, eva=24) at /usr/src/sys/amd64/amd64/trap.c:925 #6 0xffffffff8109035f in trap_pfault (frame=0xfffffe04b95c08a0, usermode=<optimized out>, signo=<optimized out>, ucode=<optimized out>) at /usr/src/sys/amd64/amd64/trap.c:743 #7 0xffffffff8108f9b8 in trap (frame=0xfffffe04b95c08a0) at /usr/src/sys/amd64/amd64/trap.c:407 #8 <signal handler called> #9 0xffffffff825f4cbc in dsl_dataset_phys (ds=0xfffff86821e72e10) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dsl_dataset.h:257 #10 zfsctl_snapdir_getattr (ap=<optimized out>) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ctldir.c:1133 #11 0xffffffff81211315 in VOP_GETATTR_APV ( vop=0xffffffff826be060 <zfsctl_ops_snapdir>, a=0xfffffe04b95c0a98) at vnode_if.c:733 #12 0xffffffff80c7bd29 in VOP_GETATTR (vp=0x1, vap=<optimized out>, cred=0xfffff88e58a45700) at ./vnode_if.h:309 #13 vop_stdvptocnp (ap=<optimized out>) at /usr/src/sys/kern/vfs_default.c:743 #14 0xffffffff8121495b in VOP_VPTOCNP_APV ( vop=0xffffffff81b281b8 <default_vnodeops>, a=0xfffffe04b95c0d90) at vnode_if.c:3718 #15 0xffffffff80c78304 in VOP_VPTOCNP (vp=0x0, vpp=<optimized out>, cred=0xfffff88e58a45700, buf=0xfffff86ed5d7d400 "", buflen=0xfffffe04b95c0e34) at ./vnode_if.h:1599 #16 vn_vptocnp (vp=0xfffffe04b95c0e28, cred=<optimized out>, buf=<optimized out>, buflen=<optimized out>) at /usr/src/sys/kern/vfs_cache.c:2296 #17 0xffffffff80c77db7 in vn_fullpath1 (td=0xfffff865848d7000, vp=0xfffff80e4a8a53c0, rdir=0xfffff860440f0b40, buf=0xfffff86ed5d7d400 "", retbuf=0xfffffe04b95c0fa8, buflen=1023) at /usr/src/sys/kern/vfs_cache.c:2392 #18 0xffffffff80c780f8 in vn_fullpath (td=0xfffff865848d7000, vn=0xfffff80e4a8a53c0, retbuf=0xfffff865848d75a0, freebuf=0xfffffe04b95c0fb0) at /usr/src/sys/kern/vfs_cache.c:2221 #19 0xffffffff80ca0635 in vn_fill_kinfo_vnode (vp=0xfffff80e4a8a53c0, kif=0xfffff831bcf5e818) at /usr/src/sys/kern/vfs_vnops.c:2352 #20 0xffffffff80c9d3f6 in vn_fill_kinfo (fp=<optimized out>, kif=0xfffff831bcf5e818, fdp=<optimized out>) at /usr/src/sys/kern/vfs_vnops.c:2318 #21 0xffffffff80b6ca25 in fo_fill_kinfo (fp=<optimized out>, kif=<optimized out>, fdp=<optimized out>) at /usr/src/sys/sys/file.h:407 #22 export_file_to_kinfo (fp=<optimized out>, fd=<optimized out>, rightsp=<optimized out>, kif=<optimized out>, fdp=0xfffff86618252450, flags=1) at /usr/src/sys/kern/kern_descrip.c:3494 #23 export_file_to_sb (fp=0xfffff8210a788460, fd=4, rightsp=<optimized out>, efbuf=<optimized out>) at /usr/src/sys/kern/kern_descrip.c:3560 #24 kern_proc_filedesc_out (p=<optimized out>, sb=<optimized out>, maxlen=<optimized out>, flags=-1124734960) at /usr/src/sys/kern/kern_descrip.c:3667 #25 0xffffffff80b6dbbd in sysctl_kern_proc_filedesc (oidp=<optimized out>, arg1=0xfffffe04b95c12bc, arg2=<optimized out>, req=<optimized out>) at /usr/src/sys/kern/kern_descrip.c:3701 #26 0xffffffff80bcd639 in sysctl_root_handler_locked ( oid=0xffffffff81b0a760 <sysctl___kern_proc_filedesc>, arg1=0xfffffe04b95c12bc, arg2=1, req=0xfffffe04b95c11f0, tracker=0xfffffe04b95c1168) at /usr/src/sys/kern/kern_sysctl.c:166 #27 0xffffffff80bcccf9 in sysctl_root (oidp=<optimized out>, arg1=0xfffffe04b95c12bc, arg2=1, req=0xfffffe04b95c11f0) at /usr/src/sys/kern/kern_sysctl.c:2062 #28 0xffffffff80bcd368 in userland_sysctl (td=0xfffff865848d7000, name=0xfffffe04b95c12b0, namelen=4, old=<optimized out>, oldlenp=<optimized out>, inkernel=<optimized out>, new=0x0, newlen=0, retval=0xfffffe04b95c1318, flags=0) at /usr/src/sys/kern/kern_sysctl.c:2157 #29 0xffffffff80bcd1af in sys___sysctl (td=0xfffff865848d7000, uap=0xfffff865848d73c0) at /usr/src/sys/kern/kern_sysctl.c:2092 #30 0xffffffff81090e87 in syscallenter (td=0xfffff865848d7000) at /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:135 #31 amd64_syscall (td=0xfffff865848d7000, traced=0) at /usr/src/sys/amd64/amd64/trap.c:1168 #32 <signal handler called> #33 0x000000080045789a in ?? () -- You are receiving this mail because: You are the assignee for the bug.help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-247668-227>