From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 26 17:48:35 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D12B48EC; Tue, 26 Nov 2013 17:48:35 +0000 (UTC) Received: from mail-ie0-x232.google.com (mail-ie0-x232.google.com [IPv6:2607:f8b0:4001:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 98F0F2070; Tue, 26 Nov 2013 17:48:35 +0000 (UTC) Received: by mail-ie0-f178.google.com with SMTP id lx4so9785633iec.37 for ; Tue, 26 Nov 2013 09:48:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=0bTbCpF8CajlSaVamxi+I+cGZ2ZnQtG2Koku1Az4KUw=; b=SAKZfs/EWfi0BX6B1lSNrqmZFhJbWbvo4iUdas6+NhnEsQ1hSYEkIiy6AainfDkYTU /xh4ir//z0ZZRJAZAHbx30OdF5Vfy0ilbKpItfuOMBhTIO8Q7CQpNgVaw5TvK5zVx2hc 8tntrPTYU9hTdXncGNQkaQeSCtIKVAzeUGypDaCoQH96xuM3UHYS2F0hPXczNu8oQqya cdeBlmVJhg24fNOmiEWAxdC2seRruHqEqce0kxoix0Dp5bgWrBntfsngMDV/qkvRI464 wC8Nc9eJeNypnwAmHAiUj4DxngzCfqG+3kina0xc+BZDaHHF/ZYRrpRAM6NVqkZt5A4E xIKg== MIME-Version: 1.0 X-Received: by 10.50.238.196 with SMTP id vm4mr17944968igc.43.1385488106991; Tue, 26 Nov 2013 09:48:26 -0800 (PST) Received: by 10.50.225.70 with HTTP; Tue, 26 Nov 2013 09:48:26 -0800 (PST) In-Reply-To: References: Date: Tue, 26 Nov 2013 18:48:26 +0100 Message-ID: Subject: Re: Alternate Data Stream Support in FreeBSD (was Re: O_XATTR support in FreeBSD?) From: Cedric Blancher To: Lionel Cons Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Cc: Freebsd hackers list , Richard Yao , Jordan Hubbard , Pedro Giffuni , Rick Macklem X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Nov 2013 17:48:35 -0000 On 26 November 2013 13:27, Lionel Cons wrote: > On 26 November 2013 11:19, Jordan Hubbard wrot= e: >> >> On Nov 26, 2013, at 1:51 AM, Cedric Blancher = wrote: >> >>> 1. You do not need more syscalls. Solaris uses the plain openat() >>> syscall for this, with the O_XATTR flag passed to the normal >>> open()/openat() flags to open a named attribute. Likewise read(), >>> write(), mmap() etc work, too. >> >> I don=92t know if I=92d go so far as to say =93you do not need more sysc= alls=94; >> there are additional functions for manipulating EAs that go well beyond >> the Solaris extensions to the directory and file I/O functions. Assumin= g you >> want to be able to get/set as well as enumerate or remove EAs, then >> you might just as well add getxattr(2), listxattr(2), removexattr(2), se= txattr(2) >> too and follow the herd (Linux and OS X, so far). > > You mean 'follow the lemmings down into the abyss'? :) > >> We=92re also glossing over ACLs and where they get to live. I don=92t k= now if Robert >> and friends have stuck them in a separate namespace on FreeBSD or if the= y=92re >> in system-protected EAs, as they are in OS X, but ACL preservation acros= s >> serialization / deserialization is just as important as it is for EAs. > > Could we first agree what we are talking about, please? I'm a bit new > to this thread, but AFAIK we are talking about the Windows Alternate > Data Streams as they appear in networked filesystem like NFSv4 and > CIFS and physical filesystems like NTFS, ZFS and Solaris UFS, right? > ACLs have no direct relation to those streams. > > The attributes support from Linux has been proven (at least from CERNs > viewpoint) as pretty useless because of their size constrains and > crappy API (i.e. no mmap(), no sparse support, no normal tools can > access them, ...) so IMHO the herd to follow is the herd which > implements at least the following requirements: > 1) A proper implementation, which includes access using the normal > system utilities (in Solaris there is the runat(1) utility to access > the hidden directory containing the attribute files, and bash4.3 and > ksh have cd -@ to cwd into the hidden directories containing the > attribute files. From that point on (inside the hidden directory) > ls(1) and even chown(1) and chmod(1) work as usual. You can even stick > ZFS and NFSv4 ACLs on the files in the hidden directory containing the > attribute files) > 2) read(), write() and mmap() access, i.e. the normal POSIX API (of > course with the minor extension to flag an access to an alternate data > stream or the directory containing the alternate data streams) > 2) Support in networked filesystems (i.e. NFSv4, CIFS) > 3) No size restrictions (just to explain, at CERN the alternate data > streams are often precompiled caches or index files of the main file's > contents, and can easily in the TB range) > 4) Support for sparse data (i.e. SEEK_HOLE and SEEK_DATA) > 5) More than one implementation available > > AFAIK Solaris, Nexenta, Illumos (NFSv4, ZFS, UFS) and Windows > Alternate Data Streams (CIFS, NTFS) fit these requirements. +1 Other argument pro-Alternate Data Streams: Alternate Data Streams are a superset of the Linux extended attributes (and can thus be used to emulate them in libc), have all their strengths but none of their weaknesses (like the hideously duplicated vfs apis and the lack of support in POSIX utilities). IMHO the Solaris/Illumos/Nexenta solution of O_XATTR provides a better integration into the Unix filesystem philosophy (everything is a file) and already reached such a common market penetration that common of the shelf shells like bash and ksh integrated support for them. Ced --=20 Cedric Blancher Institute Pasteur