From owner-freebsd-bugs@freebsd.org Sat Nov 28 03:48:07 2020 Return-Path: Delivered-To: freebsd-bugs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9D311474975 for ; Sat, 28 Nov 2020 03:48:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4CjcsM3sQSz4rvd for ; Sat, 28 Nov 2020 03:48:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 846AE474CA7; Sat, 28 Nov 2020 03:48:07 +0000 (UTC) Delivered-To: bugs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 84238474E00 for ; Sat, 28 Nov 2020 03:48:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CjcsM3CFhz4rvc for ; Sat, 28 Nov 2020 03:48:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5E66B20AC1 for ; Sat, 28 Nov 2020 03:48:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 0AS3m7jh091547 for ; Sat, 28 Nov 2020 03:48:07 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 0AS3m7eT091546 for bugs@FreeBSD.org; Sat, 28 Nov 2020 03:48:07 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 251363] use unionfs as a disk-cache for NFS [feature] Date: Sat, 28 Nov 2020 03:48:07 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: raj@gusw.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Nov 2020 03:48:07 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D251363 --- Comment #12 from Gunther Schadow --- While it works in principle, there is a little problem. Here is on a system which actually has /usr mounted via NFS. Root sets up t= he unionfs cache: # mkdir /var/cache/ # mount -t unionfs -o copypolicy=3Dalways /var/cache/usr /usr # df Filesystem 1K-blocks Us= ed=20=20 Avail Capacity Mounted on /dev/gpt/rootfs 507280 2458= 44=20=20 220856 53% / devfs 1 = 1=20=20 0 100% /dev /dev/gpt/varfs 507536 1581= 92=20=20 308744 34% /var fdescfs 1 = 1=20=20 0 100% /dev/fd procfs 4 = 4=20=20 0 100% /proc /usr 507280 2458= 44=20=20 220856 53% /.usr fs-ca0cf63f.efs.us-east-1.amazonaws.com:/ 9007199254739968 66395= 76 9007199248100392 0% /usr :/var/cache/usr -9007199254234480 -90071992545838= 24=20=20 308744 100% /usr Now another user just uses which(1) $ which java bash: /usr/bin/which: Permission denied $ ls -l /usr/bin/which -r-xr-xr-x 1 root wheel 7816 Oct 23 06:48 /usr/bin/which $ ls -l /var/cache/usr/bin total 0 oh, so the copying of the file did not work? Why? $ find /var/cache/ -ls 16394 8 drwxr-xr-x 3 root wheel 512 Nov 26 23:12 /var/cache/ 16384 0 lrwxr-xr-x 1 root wheel 18 Oct 31 18:16 /var/cache/pk= g -> /usr/var/cache/pkg 16407 8 drwxr-xr-x 8 root wheel 512 Nov 28 03:10 /var/cache/usr 16387 8 drwxr-xr-x 5 root wheel 512 Nov 28 03:07 /var/cache/usr/local 16410 8 drwxr-xr-x 2 root wheel 512 Oct 31 03:59 /var/cache/usr/local/etc 16422 8 drwxr-xr-x 2 root wheel 512 Oct 31 03:59 /var/cache/usr/local/share 16423 8 drwxr-xr-x 2 root wheel 512 Oct 31 03:59 /var/cache/usr/local/lib 16416 8 drwxr-xr-x 2 root wheel 512 Oct 23 06:50 /var/cache/usr/sbin 16417 8 drwxr-xr-x 2 root wheel 512 Oct 31 18:20 /var/cache/usr/bin 16418 8 drwxr-xr-x 4 root wheel 512 Nov 28 03:07 /var/cache/usr/share 16420 8 drwxr-xr-x 3 root wheel 512 Nov 28 03:07 /var/cache/usr/share/nls 16421 8 drwxr-xr-x 2 root wheel 512 Oct 23 06:49 /var/cache/usr/share/nls/C 16431 8 drwxr-xr-x 2 root wheel 512 Oct 23 06:48 /var/cache/usr/share/zoneinfo 16424 8 drwxr-xr-x 2 root wheel 512 Nov 28 03:10 /var/cache/usr/lib 16440 104 -r--r--r-- 1 root wheel 53048 Oct 23 06:45 /var/cache/usr/lib/libpam.so.6 16434 8 drwxr-xr-x 2 root wheel 512 Nov 28 03:10 /var/cache/usr/libexec 16439 40 -r-xr-xr-x 1 root wheel 16576 Oct 23 06:45 /var/cache/usr/libexec/atrun Could it be that the problem comes when it's not the root user who is tryin= g to access the unionfs system? Yes, that could be! Because the expectation of unionfs_copyfile(...) is that the user actually had write permission! Prove that, as toot: # which java /usr/local/bin/java and now over for the normal user: $ which java /usr/local/bin/java working too now, and indeed it was copied; $ ls -l /var/cache/usr/bin/which -r-xr-xr-x 1 root wheel 7816 Oct 23 06:48 /var/cache/usr/bin/which So, to fix this, what I need is a way to run unionfs_copyfile(...) under ro= ot permissions, it being inside the kernel I thought that's what it would do anyway. But apparently it doesn't and apparently this issue doesn't get to = be seen. But what about standard unionfs use as copy on write. What if the lower lay= er has a file with user root, with group write permission, and now I come as t= he non-root user, but in the same group, and wanting to write this file. Let's try. # mkdir test-lower # mkdir test-upper # ls -l test-* drwxr-xr-x 2 root wheel 512 Nov 28 03:34 test-lower drwxr-xr-x 2 root wheel 512 Nov 28 03:34 test-upper # echo "I am root" > test-lower/data # chmod 664 test-lower/data # mount -t unionfs test-upper test-lower # df Filesystem 1K-blocks Used=20=20=20= =20=20=20=20=20=20=20=20 Avail Capacity Mounted on ... :/tmp/test-upper 1014560 753136=20=20=20= =20=20=20=20=20=20=20 220844 77% /tmp/test-lower $ cat test-lower/data I am root $ echo "But I'm not root" > test-lower/data bash: test-lower/data: Permission denied Look at that! But without unionfs I was able to do that! Let's unmount that unionfs # umount test-lower and try again: $ echo "But I'm not root" > test-lower/data $ cat test-lower/data But I'm not root So, as we can see, even your normal unionfs is not really transparent! The = copy should have been made with the exact same modes and user:group. The man page says something that is not actually happening: copymode =3D traditional | transparent | masquerade Specifies the way to create a file or a directory in t= he upper layer automatically when needed. The traditional mode uses the same way as the old unionfs for backward compatibility, and transparent duplicates the file and directory mode bits and the ownership in the lower lay= er to the created file in the upper layer. For behavior = of the masquerade mode, see MASQUERADE MODE below. "transparent duplicates the file and directory mode bits and the ownership = in the lower layer to the created file in the upper layer." And that is totally not what's happening! Looks like a bug in unionfs to me now. But one that I'm determined to squash. --=20 You are receiving this mail because: You are the assignee for the bug.=