From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 26 17:08:34 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7024C53D; Tue, 26 Nov 2013 17:08:34 +0000 (UTC) Received: from mail-pb0-x22d.google.com (mail-pb0-x22d.google.com [IPv6:2607:f8b0:400e:c01::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 41C712DEA; Tue, 26 Nov 2013 17:08:34 +0000 (UTC) Received: by mail-pb0-f45.google.com with SMTP id rp16so8415921pbb.18 for ; Tue, 26 Nov 2013 09:08:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=zcuDVBL9+HVR+z2AVOM08PtBBo7uwAPFg+8gdOLugu8=; b=MNIfHjdyeiRunG67qBN8k4vewGBVDeJHCSe4cwhUNoCvdIR5OcIitEy986cc5ZGT+Y jLx5eJ3TvYIVK0wtH6q0vsgY7qzrNbhDzIstf6uenIbonyu9+zcx1q+8MJltkTFTpiXY GNg6x8I8v0iUy/nG+z4p7NYcFCsCIJTTeSkMwxOGkVM/03dx8LR5ummXFG7LlLcDxdAq zZTsaf1dU49VJZ6MieWxi33YiA3lRKrrvUh5WjzIwfBILowI0WftSK1jydD6kbwvRiDL FMb0n6r2hFbQd0BpLtGw+1wk/miQHmOg796ILThNt64VaFpAAviRklb6a5jAOaKQHNQx QwdA== X-Received: by 10.68.232.37 with SMTP id tl5mr402218pbc.86.1385485713262; Tue, 26 Nov 2013 09:08:33 -0800 (PST) Received: from [10.20.30.117] (45.sub-70-197-14.myvzw.com. [70.197.14.45]) by mx.google.com with ESMTPSA id sg1sm81909879pbb.16.2013.11.26.09.08.30 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 26 Nov 2013 09:08:32 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.0 \(1812\)) Subject: Re: Alternate Data Stream Support in FreeBSD (was Re: O_XATTR support in FreeBSD?) From: Jordan Hubbard In-Reply-To: Date: Tue, 26 Nov 2013 09:08:29 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <1934C2F2-C380-4689-A811-61BB0009BA6B@gmail.com> References: To: Lionel Cons X-Mailer: Apple Mail (2.1812) Cc: Freebsd hackers list , Richard Yao , Pedro Giffuni , Cedric Blancher , Rick Macklem X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Nov 2013 17:08:34 -0000 On Nov 26, 2013, at 4:27 AM, Lionel Cons = wrote: >> I don=92t know if I=92d go so far as to say =93you do not need more = syscalls=94; >> there are additional functions for manipulating EAs that go well = beyond >> the Solaris extensions to the directory and file I/O functions. = Assuming you >> want to be able to get/set as well as enumerate or remove EAs, then >> you might just as well add getxattr(2), listxattr(2), removexattr(2), = setxattr(2) >> too and follow the herd (Linux and OS X, so far). >=20 > You mean 'follow the lemmings down into the abyss'? :) Well, I don=92t know that it=92s an =93abyss=94 - EAs may or may not be = useful, depending on how you employ them! In the first version of OS X to support them, in fact, I believe they = were limited in size to 4883 bytes (don=92t ask me why that number) and = they were still used to apply various =93tags=94 to files (Finder = metadata, some index values into the search database, etc). General = pressure to use them for more things eventually got this size bumped up = to 128K, and now it=92s actually 2GB(!) = (http://support.apple.com/kb/HT5983) so I think it=92s fair to say that = EAs in OS X are now essentially equivalent to forked files, more or = less. > Could we first agree what we are talking about, please? I'm a bit new > to this thread, but AFAIK we are talking about the Windows Alternate > Data Streams as they appear in networked filesystem like NFSv4 and > CIFS and physical filesystems like NTFS, ZFS and Solaris UFS, right? > ACLs have no direct relation to those streams. Actually, I didn=92t think we were talking about alternate data streams = myself. Conceptually they=92re equivalent, I guess, but I=92ve always = through they were somewhat overkill and I=92ve yet to encounter an = application that seriously uses them. I=92m sure they=92re out there = somewhere, but even back in the days when EAs were limited to just over = 4K, we found them very useful for what was essentially their original = purpose - an extension to the file attribute data that Unix already = proves. The only reason that ACLs crept into the discussion is because = of where they=92re stored. I don=92t know about Linux, but Apple has = chosen to store ACLs in EAs, which is pretty useful because this gives = you an easy way of serializing the ACLs too - you just serialize them = from a suitably privileged process. The main point I was trying to make is that if you=92re going to have = EAs at all, you need to commit fully. The various Unix tools need to = support them (we=92ve already talked about the archivers and = compressors) and tools like ls(1) need to be able to show them on files. = You need a way of dealing with them on foreign filesystems that don=92t = support EAs. Most folks just cram EAs into the filesystem, add a few = decorations to existing system calls and then shout =93done!=94 and do a = victory dance. Then when nobody actually uses EAs, they go =93See? I = always told you EAs were crap! Terrible idea! Never should have added = them!=94 This is tantamount to building a car with an engine but no wheels, = dashboard or steering wheel and then declaring that the world just isn=92t= ready for cars since they=92re not buying yours. I know you cite Solaris=92 integration as an example of such a =93full = solution, and maybe the =93~@=93 syntax was awesome in practice, I = dunno, all I can say is that the only namespace trick we needed to pull = at Apple were the AppleDouble (._) sidecar files. There was an earlier = filename/..rfork/ syntax for addressing resource forks, which predated = EAs in OS X, and some folks used it quite a bit, but it was eventually = deprecated in favor of the single sidecar file. I never found a need to = =93cd=94 into the namespace of a file=92s EAs when I had the xattr(1) = command so handy for deleting / changing them, and ls=92s -@ argument = would also display them for me. I suppose it is all a matter of taste. = If someone wanted to do the namespace thing in FreeBSD, I wouldn=92t = argue against it. I also wouldn=92t argue against fully parallel =93forks=94 being a = superset of EAs, since I guess at CERN where folks are routinely looking = at Petabytes of data from a detector like ATLAS or CMS, anything that = puts size constraints on their data is just the devil, but again, that = wasn=92t actually the point I was trying to make. I was simply trying = to say that NFSv4 or ZFS =93native EA support=94 is the easy part. The = harder part is in making sure that the EAs don=92t get stripped out in = transit or during routine file manipulations, and this requires that = everything from cp(1) to rsync(8) becomes EA-aware. Most of the = implementations I=92ve seen don=92t bother to do that last mile of = integration, and as a result EAs are just basically untrustworthy beasts = that users shy away from. - Jordan