Date: Fri, 20 Dec 2024 17:34:48 +0000 From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 283448] [fusefs] use after free on NFS-exported file fuse systems Message-ID: <bug-283448-227@https.bugs.freebsd.org/bugzilla/>
next in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D283448 Bug ID: 283448 Summary: [fusefs] use after free on NFS-exported file fuse systems Product: Base System Version: 15.0-CURRENT Hardware: Any OS: Any Status: New Severity: Affects Only Me Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: asomers@FreeBSD.org NFS is weird. It's stateless, so it never calls VOP_OPEN or VOP_CLOSE. The fuse protocol, however, usually requires FUSE_OPEN and FUSE_CLOSE commands = to be sent to the server. So fusefs(4) has some weird hacks in it to make that work. But it's possible to trigger a use-after-free bug in those weird hac= ks.=20 On a debug kernel, it will panic. The problem is that VOP_READ will open a fuse_filehandle itself, if necessa= ry.=20 It should only be necessary when the read is coming from the nfs server. T= hen, VOP_READ will close the fuse_filehandle. But in the meantime another thread may be borrowing that fuse_filehandle, and the close isn't properly synchronized. To reproduce this bug, you must have a fuse file system that is suitable for export over NFS. That is, it does not need FUSE_OPEN or FUSE_OPENDIR, correctly handles FUSE_LOOKUP for ".", and only uses 32-bit (or less) generation numbers. Not all fuse file systems satisfy these requirements. = In particular, anything that links to libfuse2 will not. In this example, I u= se bfffs. Steps to Reproduce =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 0. Create a suitable fusefs file system, and mount it to /mnt 1. export that file system over NFS, in /etc/exports, and start rpcbind and nfsd 2. Mount it on a client: sudo mount -t nfs -o nfsv4,minorversion=3D2 192.168.0.27:/mnt /mnt 3. Install fsx pkg install -y fsx 4. Create an fsx.toml file with these contents: flen =3D 1048576 nomsyncafterwrite =3D false nosizechecks =3D false blockmode =3D false [opsize] max =3D 262144 min =3D 0 align =3D 1 [weights] close_open =3D 1 read =3D 10 write =3D 10 mapread =3D 10 mapwrite =3D 10 invalidate =3D 1 truncate =3D 1 fsync =3D 1 fdatasync =3D 1 posix_fallocate =3D 0 punch_hole =3D 1 sendfile =3D 1 posix_fadvise =3D 0 copy_file_range =3D 1 5. Run fsx on the NFS mount. I had to run it 4 times to trigger the bug sudo fsx -N 1000 -f /tmp/bfffs-on-nfs.toml /mnt/fsx.bin Stack Trace =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D panic: page fault cpuid =3D 12 time =3D 1734714504 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0150adf= 350 vpanic() at vpanic+0x136/frame 0xfffffe0150adf480 panic() at panic+0x43/frame 0xfffffe0150adf4e0 trap_fatal() at trap_fatal+0x40b/frame 0xfffffe0150adf540 trap_pfault() at trap_pfault+0xa0/frame 0xfffffe0150adf5b0 calltrap() at calltrap+0x8/frame 0xfffffe0150adf5b0 --- trap 0xc, rip =3D 0xffffffff82813b3e, rsp =3D 0xfffffe0150adf680, rbp = =3D 0xfffffe0150adf690 --- fuse_ticket_drop() at fuse_ticket_drop+0xe/frame 0xfffffe0150adf690 fuse_internal_fsync() at fuse_internal_fsync+0xf8/frame 0xfffffe0150adf730 nfsvno_fsync() at nfsvno_fsync+0x5f/frame 0xfffffe0150adf7a0 nfsrvd_commit() at nfsrvd_commit+0xf0/frame 0xfffffe0150adf9a0 nfsrvd_dorpc() at nfsrvd_dorpc+0x167e/frame 0xfffffe0150adfbb0 nfssvc_program() at nfssvc_program+0x808/frame 0xfffffe0150adfdb0 svc_run_internal() at svc_run_internal+0xa09/frame 0xfffffe0150adfee0 svc_thread_start() at svc_thread_start+0xb/frame 0xfffffe0150adfef0 fork_exit() at fork_exit+0x82/frame 0xfffffe0150adff30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0150adff30 --- trap 0xc, rip =3D 0x107988c0c1fa, rsp =3D 0x107986884028, rbp =3D 0x107= 9868842c0 --- KDB: enter: panic Analysis =3D=3D=3D=3D=3D=3D=3D=3D kgdb shows that within fdisp_destroy, ftick->tick was NULL. That indicates= a double-free. Most likely, one thread was running fuse_internal_fsync while another fuse_vnop_read. The first thread accessed a fuse_filehandle struct= ure while the second was freeing it. --=20 You are receiving this mail because: You are the assignee for the bug.=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-283448-227>