From nobody Fri Dec 20 17:34:48 2024 X-Original-To: bugs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4YFF0m6c9yz5X2Hk for ; Fri, 20 Dec 2024 17:34:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4YFF0m4f5Nz4CSb for ; Fri, 20 Dec 2024 17:34:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1734716088; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=btJWVSFtNJfn1XnnOQcpyewg8FXmwYZFX03Ij3oZAZ4=; b=eN4OkfaAf+In3vLQCudaXuifIYFtRIic6sO9xha2FMxDJw9ZclsrZ2hJbSSkU73eKuirCW C2PfFiPTZsUnxGTjbn75Pvdhq286WpyhmsYR8vKvMrS8pbk6JIqlCvIN8GpNu524cgISfY WoS/wV4VmXWJ/jsHSiSXGjwjlfs6dOYusuNofxAFRgLWnsKbm4zN0G1RRUJ6sg0mYzx5n/ dm/MeW9NY7pZtNeEFcIYRB/T+rgzCJqJCTPBWj0UKKwhA339m3GyeXSJVQAkA2KRcUtNq4 hS1anZSetejcRpaOagUUw7twxrpqw3Fg3MDyrxYhs877uCJhJ/eWNLdQxf3iCA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1734716088; a=rsa-sha256; cv=none; b=fa33AJ5KtBFsc9u1jgMaxo9Wnk8fD9cva6m2C+TaDBlpOW2x0SQ0aEHHCU6nCA4EW7X+wU 31RikosSt1J62aPclh07omKX0lvkqMYshqyzNL5mAjIsqdZarm2PMXVAhanIzKf2gOdbw7 5EVZOfb2FQVywzQR1d0duiJ6hFN6LACLglwI7m6vdMLVzrh1YWViauCBYvGR6ncDwZk4vt sIBTS4u0mVanRYDPONremCbV49E3NfW6F+2aGo4S2+LXYD9LmIrTPVgaPCw72hlUKIKcNC gbvx+13cbtOn+b6xtFgKQ+B1MaDMbC06IOrwlNnX8Nqfp0qF4E+XgHrkhftl7g== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4YFF0m3zcKznk7 for ; Fri, 20 Dec 2024 17:34:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 4BKHYmE6034442 for ; Fri, 20 Dec 2024 17:34:48 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 4BKHYmB6034441 for bugs@FreeBSD.org; Fri, 20 Dec 2024 17:34:48 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 283448] [fusefs] use after free on NFS-exported file fuse systems Date: Fri, 20 Dec 2024 17:34:48 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 15.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: asomers@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Bug reports List-Archive: https://lists.freebsd.org/archives/freebsd-bugs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-bugs@FreeBSD.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D283448 Bug ID: 283448 Summary: [fusefs] use after free on NFS-exported file fuse systems Product: Base System Version: 15.0-CURRENT Hardware: Any OS: Any Status: New Severity: Affects Only Me Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: asomers@FreeBSD.org NFS is weird. It's stateless, so it never calls VOP_OPEN or VOP_CLOSE. The fuse protocol, however, usually requires FUSE_OPEN and FUSE_CLOSE commands = to be sent to the server. So fusefs(4) has some weird hacks in it to make that work. But it's possible to trigger a use-after-free bug in those weird hac= ks.=20 On a debug kernel, it will panic. The problem is that VOP_READ will open a fuse_filehandle itself, if necessa= ry.=20 It should only be necessary when the read is coming from the nfs server. T= hen, VOP_READ will close the fuse_filehandle. But in the meantime another thread may be borrowing that fuse_filehandle, and the close isn't properly synchronized. To reproduce this bug, you must have a fuse file system that is suitable for export over NFS. That is, it does not need FUSE_OPEN or FUSE_OPENDIR, correctly handles FUSE_LOOKUP for ".", and only uses 32-bit (or less) generation numbers. Not all fuse file systems satisfy these requirements. = In particular, anything that links to libfuse2 will not. In this example, I u= se bfffs. Steps to Reproduce =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 0. Create a suitable fusefs file system, and mount it to /mnt 1. export that file system over NFS, in /etc/exports, and start rpcbind and nfsd 2. Mount it on a client: sudo mount -t nfs -o nfsv4,minorversion=3D2 192.168.0.27:/mnt /mnt 3. Install fsx pkg install -y fsx 4. Create an fsx.toml file with these contents: flen =3D 1048576 nomsyncafterwrite =3D false nosizechecks =3D false blockmode =3D false [opsize] max =3D 262144 min =3D 0 align =3D 1 [weights] close_open =3D 1 read =3D 10 write =3D 10 mapread =3D 10 mapwrite =3D 10 invalidate =3D 1 truncate =3D 1 fsync =3D 1 fdatasync =3D 1 posix_fallocate =3D 0 punch_hole =3D 1 sendfile =3D 1 posix_fadvise =3D 0 copy_file_range =3D 1 5. Run fsx on the NFS mount. I had to run it 4 times to trigger the bug sudo fsx -N 1000 -f /tmp/bfffs-on-nfs.toml /mnt/fsx.bin Stack Trace =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D panic: page fault cpuid =3D 12 time =3D 1734714504 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0150adf= 350 vpanic() at vpanic+0x136/frame 0xfffffe0150adf480 panic() at panic+0x43/frame 0xfffffe0150adf4e0 trap_fatal() at trap_fatal+0x40b/frame 0xfffffe0150adf540 trap_pfault() at trap_pfault+0xa0/frame 0xfffffe0150adf5b0 calltrap() at calltrap+0x8/frame 0xfffffe0150adf5b0 --- trap 0xc, rip =3D 0xffffffff82813b3e, rsp =3D 0xfffffe0150adf680, rbp = =3D 0xfffffe0150adf690 --- fuse_ticket_drop() at fuse_ticket_drop+0xe/frame 0xfffffe0150adf690 fuse_internal_fsync() at fuse_internal_fsync+0xf8/frame 0xfffffe0150adf730 nfsvno_fsync() at nfsvno_fsync+0x5f/frame 0xfffffe0150adf7a0 nfsrvd_commit() at nfsrvd_commit+0xf0/frame 0xfffffe0150adf9a0 nfsrvd_dorpc() at nfsrvd_dorpc+0x167e/frame 0xfffffe0150adfbb0 nfssvc_program() at nfssvc_program+0x808/frame 0xfffffe0150adfdb0 svc_run_internal() at svc_run_internal+0xa09/frame 0xfffffe0150adfee0 svc_thread_start() at svc_thread_start+0xb/frame 0xfffffe0150adfef0 fork_exit() at fork_exit+0x82/frame 0xfffffe0150adff30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0150adff30 --- trap 0xc, rip =3D 0x107988c0c1fa, rsp =3D 0x107986884028, rbp =3D 0x107= 9868842c0 --- KDB: enter: panic Analysis =3D=3D=3D=3D=3D=3D=3D=3D kgdb shows that within fdisp_destroy, ftick->tick was NULL. That indicates= a double-free. Most likely, one thread was running fuse_internal_fsync while another fuse_vnop_read. The first thread accessed a fuse_filehandle struct= ure while the second was freeing it. --=20 You are receiving this mail because: You are the assignee for the bug.=