From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 13 06:41:12 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3F6D3117 for ; Thu, 13 Feb 2014 06:41:12 +0000 (UTC) Received: from mail.bitblocks.com (ns1.bitblocks.com [173.228.5.8]) by mx1.freebsd.org (Postfix) with ESMTP id 1B59A105F for ; Thu, 13 Feb 2014 06:41:11 +0000 (UTC) Received: from altos.bitblocks.com (altos.bitblocks.com [192.168.125.3]) by mail.bitblocks.com (Postfix) with ESMTP id 2EB7CB827; Wed, 12 Feb 2014 22:35:03 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: Thoughts on Multi-Symlink Concept From: Bakul Shah In-Reply-To: Date: Wed, 12 Feb 2014 22:35:01 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Jordan Hubbard X-Mailer: Apple Mail (2.1827) Cc: freebsd-filesystems@freebsd.org, "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Feb 2014 06:41:12 -0000 On Feb 12, 2014, at 9:08 PM, Jordan Hubbard = wrote: >=20 > On Feb 11, 2014, at 9:09 PM, Jason Hellenthal = wrote: >=20 >> Instead of: cat /path/to/files* >> ln -sm /path/to/files* ./my_concat_list >> cat ./my_concat_list >>=20 >> Or >>=20 >> ln -sm /path/to/file1 /path/to/file2 ./my_concat_filters >> pfctl -v -f ./my_concat_filters >=20 > Globbing is done in user land (by the shell) - you wouldn=92t want to = push that down into the kernel, which is either what you=92d have to do = or you=92d need a user land daemon which did round-trips with the kernel = to do the translation, which would also need to make sure to get all of = the process permission stuff right since the user id / gid / $CWD would = all potentially affect the expansion of the =93symlink=94. >=20 > There are other problems. Since =93my_concat_list=94 now expands to = multiple files, you have to pre-expand its contents before actually = execing cat(1), and since filenames can have spaces and other magic = characters (to the shell) in them, you would need to be able to deal = with escaping all of that. Just like=85 the shell does! >=20 > Even worse still, you=92d be essentially creating a new file type with = very non-file like semantics. Even variant symlinks (/bin -> = /${ARCH}/bin), which can expand differently depending on the user = context, have clearly understandable semantics - you know that the = symlink is going to expand to exactly one file no matter what ARCH is = set to. How, conversely, would you expect =93cat < my_concat_filters=94 = or =93dd if=3Dmy_concat_filters of=3D=85=94 to work? You=92ve = essentially created a construct that can cause truly weird behaviors = unless you use it in exactly one way and one way only (as a file = argument list), the user having to explicitly know the difference when = they consume the symlink. That creates a whole raft of layering = violations, and is very =93not Unix=94, to be honest. May be a similar idea can be used to implement file striping & mirroring = on top of existing filesystems -- modern distributed FS seem to reinvent = a lot stuff. Instead one can just build a simple thin layer and use = local and remote simple filesystems to create a distributed FS. I = wouldn't use symlinks but indicate additional semantics via some = metadata. > If you want to get creative along these lines without breaking Unix, I = think creating some new type of artificial directory that doesn=92t = actually physically contain files but can flexibly reference a = collection of files based on some sort of query - that might be = interesting. Lot of such variations are possible using fusefs