From owner-freebsd-bugs@freebsd.org Thu Nov 26 01:01:49 2020 Return-Path: Delivered-To: freebsd-bugs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 92801479698 for ; Thu, 26 Nov 2020 01:01:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4ChKGP3YBLz4hpF for ; Thu, 26 Nov 2020 01:01:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 79CCA479697; Thu, 26 Nov 2020 01:01:49 +0000 (UTC) Delivered-To: bugs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7988247945E for ; Thu, 26 Nov 2020 01:01:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ChKGP2y4wz4j77 for ; Thu, 26 Nov 2020 01:01:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 55D4F1A7D7 for ; Thu, 26 Nov 2020 01:01:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 0AQ11nJK027534 for ; Thu, 26 Nov 2020 01:01:49 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 0AQ11nhG027533 for bugs@FreeBSD.org; Thu, 26 Nov 2020 01:01:49 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 251363] use unionfs as a disk-cache for NFS [feature] Date: Thu, 26 Nov 2020 01:01:49 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: Unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: raj@gusw.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Nov 2020 01:01:49 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D251363 --- Comment #6 from Gunther Schadow --- Success! Here is my full diff in /usr/src/sys/fs/unionfs # svnlite diff -x-w Index: union.h =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- union.h (revision 368012) +++ union.h (working copy) @@ -40,6 +40,12 @@ #ifdef _KERNEL +/* copy policy from lower to upper layer */ +typedef enum _unionfs_copypolicy { + UNIONFS_COPY_ON_WRITE =3D 0, + UNIONFS_COPY_ALWAYS +} unionfs_copypolicy; + /* copy method of attr from lower to upper */ typedef enum _unionfs_copymode { UNIONFS_TRADITIONAL =3D 0, @@ -57,6 +63,7 @@ struct vnode *um_lowervp; /* VREFed once */ struct vnode *um_uppervp; /* VREFed once */ struct vnode *um_rootvp; /* ROOT vnode */ + unionfs_copypolicy um_copypolicy; unionfs_copymode um_copymode; unionfs_whitemode um_whitemode; uid_t um_uid; Index: union_vfsops.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- union_vfsops.c (revision 368012) +++ union_vfsops.c (working copy) @@ -89,6 +89,7 @@ gid_t gid; u_short udir; u_short ufile; + unionfs_copypolicy copypolicy; unionfs_copymode copymode; unionfs_whitemode whitemode; struct nameidata nd, *ndp; @@ -102,6 +103,7 @@ gid =3D 0; udir =3D 0; ufile =3D 0; + copypolicy =3D UNIONFS_COPY_ON_WRITE; copymode =3D UNIONFS_TRANSPARENT; /* default */ whitemode =3D UNIONFS_WHITE_ALWAYS; ndp =3D &nd; @@ -190,6 +192,20 @@ return (EINVAL); } } + if (vfs_getopt(mp->mnt_optnew, "copypolicy", (void **)&tmp, + NULL) =3D=3D 0) { + if (tmp =3D=3D NULL) { + vfs_mount_error(mp, "Invalid copy policy"); + return (EINVAL); + } else if (strcasecmp(tmp, "always") =3D=3D 0) + copypolicy =3D UNIONFS_COPY_ALWAYS; + else if (strcasecmp(tmp, "onwrite") =3D=3D 0) + copypolicy =3D UNIONFS_COPY_ON_WRITE; + else { + vfs_mount_error(mp, "Invalid copy policy"); + return (EINVAL); + } + } if (vfs_getopt(mp->mnt_optnew, "copymode", (void **)&tmp, NULL) =3D=3D 0) { if (tmp =3D=3D NULL) { @@ -229,7 +245,7 @@ UNIONFSDEBUG("unionfs_mount: uid=3D%d, gid=3D%d\n", uid, gid); UNIONFSDEBUG("unionfs_mount: udir=3D0%03o, ufile=3D0%03o\n", udir, = ufile); - UNIONFSDEBUG("unionfs_mount: copymode=3D%d\n", copymode); + UNIONFSDEBUG("unionfs_mount: copypolicy=3D%d, copymode=3D%d, whitemode=3D%d\n", copypolicy, copymode, whitemode); /* * Find upper node @@ -265,6 +281,7 @@ ump->um_gid =3D gid; ump->um_udir =3D udir; ump->um_ufile =3D ufile; + ump->um_copypolicy =3D copypolicy; ump->um_copymode =3D copymode; ump->um_whitemode =3D whitemode; Index: union_vnops.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- union_vnops.c (revision 368012) +++ union_vnops.c (working copy) @@ -464,6 +464,7 @@ { int error; struct unionfs_node *unp; + struct unionfs_mount *ump; struct unionfs_node_status *unsp; struct vnode *uvp; struct vnode *lvp; @@ -477,6 +478,7 @@ error =3D 0; unp =3D VTOUNIONFS(ap->a_vp); + ump =3D MOUNTTOUNIONFSMOUNT(ap->a_vp->v_mount); uvp =3D unp->un_uppervp; lvp =3D unp->un_lowervp; targetvp =3D NULLVP; @@ -498,7 +500,7 @@ } if (targetvp =3D=3D NULLVP) { if (uvp =3D=3D NULLVP) { - if ((ap->a_mode & FWRITE) && lvp->v_type =3D=3D VRE= G) { + if (((ap->a_mode & FWRITE) || (ump->um_copypolicy = =3D=3D UNIONFS_COPY_ALWAYS)) && lvp->v_type =3D=3D VREG) { error =3D unionfs_copyfile(unp, !(ap->a_mode & O_TRUNC), cred, td); if (error !=3D 0) r Testing: # mdconfig -a -t vnode /data/cachfs md0 newfs /dev/md0 /dev/md0: 1024.0MB (2097152 sectors) block size 32768, fragment size 4096 using 4 cylinder groups of 256.03MB, 8193 blks, 32896 inodes. super-block backups (for fsck_ffs -b #) at: 192, 524544, 1048896, 1573248 # mkdir /cache # mount /dev/md0 /cache # mount_unionfs -o copypolicy=3Dalways /cache /worker-efs # /efs/bin/less ... # /efs/local/bin/python3.7 ... # find /cache cache cache/.snap cache/bin cache/bin/less cache/sbin cache/local cache/local/bin cache/local/bin/python3.7 cache/local/lib cache/local/lib/python3.7 cache/local/lib/python3.7/lib-dynload cache/local/lib/python3.7/lib-dynload/readline.so cache/local/lib/python3.7/encodings cache/local/lib/python3.7/encodings/__pycache__ cache/local/lib/python3.7/encodings/__pycache__/__init__.cpython-37.pyc cache/local/lib/python3.7/encodings/__pycache__/aliases.cpython-37.pyc cache/local/lib/python3.7/encodings/__pycache__/utf_8.cpython-37.pyc cache/local/lib/python3.7/encodings/__pycache__/latin_1.cpython-37.pyc cache/local/lib/python3.7/__pycache__ cache/local/lib/python3.7/__pycache__/codecs.cpython-37.pyc cache/local/lib/python3.7/__pycache__/io.cpython-37.pyc cache/local/lib/python3.7/__pycache__/abc.cpython-37.pyc cache/local/lib/python3.7/__pycache__/site.cpython-37.pyc cache/local/lib/python3.7/__pycache__/os.cpython-37.pyc cache/local/lib/python3.7/__pycache__/stat.cpython-37.pyc cache/local/lib/python3.7/__pycache__/_collections_abc.cpython-37.pyc cache/local/lib/python3.7/__pycache__/posixpath.cpython-37.pyc cache/local/lib/python3.7/__pycache__/genericpath.cpython-37.pyc cache/local/lib/python3.7/__pycache__/_sitebuiltins.cpython-37.pyc cache/local/lib/python3.7/__pycache__/_bootlocale.cpython-37.pyc cache/local/lib/python3.7/__pycache__/rlcompleter.cpython-37.pyc cache/local/lib/python3.7/site-packages cache/local/lib/python3.7/site-packages/easy-install.pth ... So this is really cool, I can even evict the cache by just writing to the unionfs! This is nice, so I might not even need that UNIONFS_WIHTE_NEVER. --=20 You are receiving this mail because: You are the assignee for the bug.=