From nobody Thu Oct 2 15:49:41 2025 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ccx8K4TM1z69PWx for ; Thu, 02 Oct 2025 15:50:25 +0000 (UTC) (envelope-from aurelien.couderc2002@gmail.com) Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com [IPv6:2a00:1450:4864:20::529]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ccx8J57Fcz3Tvq for ; Thu, 02 Oct 2025 15:50:24 +0000 (UTC) (envelope-from aurelien.couderc2002@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=fy9NXAT7; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of aurelien.couderc2002@gmail.com designates 2a00:1450:4864:20::529 as permitted sender) smtp.mailfrom=aurelien.couderc2002@gmail.com Received: by mail-ed1-x529.google.com with SMTP id 4fb4d7f45d1cf-62105d21297so2473374a12.0 for ; Thu, 02 Oct 2025 08:50:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759420218; x=1760025018; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=nKRPl/KLhF3538wIW1orkWm9x45NJk1O8gZBLYZPRZc=; b=fy9NXAT7JjuJGxxb9/gJ7TJsBxBj5YW26ekcPRZ+hrpAbBZMbFsZQmEafxL+22wVmI +j+3fGENx8aVSRQiUTb4E++HR2JZatkUYs0JBvCrjB/TsvRoCJNmJaFCMFbzhys3f6LY pivvWIptpMRJ17Qy0OS4dR9ZgqNsuscDXuCOwnjDx5GIDhHHQQ3OBVeHqstSgAVHjNyo zcKuZZCZgyjb7WhItzKnHd7Y3oy9nG1evd6YNQM/UW6Y8/2bqlHmSKfNQPE0rYGwQ3QR vuoWbHER1LMCcORrd6Ep+vlfEdaessBBRiV/Lzw/lCFZiRJeS3CTemS6wFHyilV2LIVh 1pvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759420218; x=1760025018; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nKRPl/KLhF3538wIW1orkWm9x45NJk1O8gZBLYZPRZc=; b=Oha4mCR8SgPrdkh+o9Y/X1mFhh73/QZLgQefSDwFXH7GbYKXmrHSKMZtwBv3s1UcGc Yt7eUYWH45yFDPxMNp5flEpiu4X3+n7sJNHEvfB8odW3DrSpozq/YzFl6gjkMQgdg9U1 h7mfJ7AbA/MjxvSMs6QrcBWJEY+nQVzosx1zJjKkCp/FAkOmgl39SDQnY1emQXeKOURM AWUwUFmI+n4dE6wKDtsc6dOqRpQng16UQSZtb+IBWyTS5Sl9M9njH3MCStcx15gl+8Xl sLHF7YXXAedGGKZCf9XkQYgyZ3LCZqptPGx5GBbzkmWUPLoSwmNN4xyZpEIIX5bqXBMV 9m2w== X-Gm-Message-State: AOJu0Yw7pBHevtG5ExU9zhCWiAqjC8iJ08MCHligbu5GGYS+xiuBAFkP bEq3QBr+7MIdDaqamaiPlNgPYIKw2YSYPrd1H8qRh28VWPVOBTVHC6YpKjQDmpZXjBaCRMwEQ65 PH+RGRoZFi96ImCpBDREhD0bYz/b5k9mQH7Kv X-Gm-Gg: ASbGncvF8zjcpIN9V20EepHAh5Id09BL4Yl1UTDPHBafxTxtnxNcXPU3bYTjlokSb1i qKGLlsIKnMGEV7lBfD7GYViP0/gt/xckticSUhGKGnHJ5ziTiMkFPsA7od6kLtQ6ikQVyCXylmm OVL5RtnBqYQKKsZMJI88RrAxFdsf6oXKFDwbvGWl6wUDFDZn4GS8U3W5MMPi8eRl915mnPxF6yG aJJb2yZiSCMLY+No8EF2BfQebNfP9Y= X-Google-Smtp-Source: AGHT+IF+vMEOcRwHA334krg+5F4qoO5JzjukhQII9hAdTtD64wUuDdXcsm1UlM3y5gLv0emiYcgPMDQe4LmEzdhxEiA= X-Received: by 2002:aa7:d687:0:b0:632:16ad:d70b with SMTP id 4fb4d7f45d1cf-63678c78dddmr6287886a12.29.1759420217466; Thu, 02 Oct 2025 08:50:17 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: <20250913220457.76606b75@nuclight.lan> In-Reply-To: From: =?UTF-8?Q?Aur=C3=A9lien_Couderc?= Date: Thu, 2 Oct 2025 17:49:41 +0200 X-Gm-Features: AS18NWByu1Yp3gPiNJoUopp6SgehmGxvP4AocCAfIysX-m761Dozsu79IWERCh0 Message-ID: Subject: Re: Renaming O_NAMEDATTR to O_NAMEDSTREAMS Re: Size of "alternate data streams"/"resource forks" / O_NAMEDATTR Re: FreeBSD Status Report - Second Quarter 2025 To: Rick Macklem Cc: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.97 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.97)[-0.973]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_TWO(0.00)[2]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_SOME(0.00)[]; TAGGED_FROM(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; MISSING_XM_UA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::529:from]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MID_RHS_MATCH_FROMTLD(0.00)[]; TAGGED_RCPT(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-Rspamd-Queue-Id: 4ccx8J57Fcz3Tvq On Thu, Oct 2, 2025 at 4:34=E2=80=AFPM Rick Macklem wrote: > > On Thu, Oct 2, 2025 at 4:55=E2=80=AFAM Aur=C3=A9lien Couderc > wrote: > > > > On Sat, Sep 13, 2025 at 9:05=E2=80=AFPM Vadim Goncharov wrote: > > > > > > On Wed, 10 Sep 2025 15:06:47 -0700 > > > Rick Macklem wrote: > > > > > > > > > > > > > 1. There are no size constraints. The main problem is d= ifferent: > > > > > > > > > > SUN's original name is both a misnomer, and was hijacke= d by > > > > > > > > > > Linux to stuff in their broken clone of Windows extende= d > > > > > > > > > > attributes. > > > > > > > > > > > > > > > > > > > > The O_NAMEDATTR/O_XATTR files are in fact "alternate da= ta > > > > > > > > > > streams", like on Windows, macOS (which calls them reso= urce > > > > > > > > > > forks), and mainframe operating systems. > > > > > > > > > > They can have unlimited size (like the "main" data stre= am), and > > > > > > > > > > can even be sparse (SEEK_HOLE&friends). These are very = different > > > > > > > > > > beats from "extended attributes". > > > > > > > > > > > > > > > > > > > > Background: > > > > > > > > > > Early versions of Windows started with EA (extended att= ributes), > > > > > > > > > > and then Windows NT4 adopted alternate data streams as = a > > > > > > > > > > superior super set of the EA. Windows just couldn't get= rid of > > > > > > > > > > the EAs, so they (and their evil Linux xattr clone) and= their > > > > > > > > > > limitations keep haunting us. > > > > > > > > > > > > > > > > > > > > Funny is, EAs in Windows are nowadays just emulated via > > > > > > > > > > alternate data streams (stream "$EA" is the index, stre= am > > > > > > > > > > "$EA_INFORMATION" has the raw data). > > > > > > > > > > > > > > > > > > > > 2. Look at Solaris tar, which can handle unlimited size= of > > > > > > > > > > O_XATTR streams > > > > > > > > > It might be possible to put Solaris tar (from OpenSolaris= ) in > > > > > > > > > ports. I'll put it on my (currently rather long) todo lis= t, unless > > > > > > > > > someone else is inspired > > > > > > > > > to look at it? > > > > > > > > > > > > > > > > > > As for libarchive, there are two problems: > > > > > > > > > - The way it is structured, it generates a linked list of= a file's > > > > > > > > > extended attributes. > > > > > > > > > As such, it is possible for very large ones to run into= malloc() > > > > > > > > > failures (aka ENOMEM). > > > > > > > > > To fix this would be a major re-write of libarchive, so= I do not > > > > > > > > > see that happenning. > > > > > > > > > - The other is that it currently uses > > > > > > > > > extattr_get_file/extattr_set_file, which copies the > > > > > > > > > entire attribute in one syscall. This actually works fo= r ZFS > > > > > > > > > locally, but will not work > > > > > > > > > for NFSv4.2 because there is a limit (currently a littl= e over > > > > > > > > > 1Mbyte) on RPC message > > > > > > > > > size. > > > > > > > > > --> This should be fixable via a patch that replaces th= e above > > > > > > > > > syscalls with loops > > > > > > > > > on read/write for file systems that support named > > > > > > > > > attributes. This is already on my todo list. > > > > > > > > > > > > > > > > I think the problem here is that people think O_XATTR/O_NAM= EDATTRS > > > > > > > > are "attributes". Which implies a short amount of data, whi= ch is > > > > > > > > ABSOLUTELY NOT THE CASE!! > > > > > > > > > > > > > > > > Maybe O_NAMEDATTR and O_XATTR should be renamed to O_NAMEDS= TREAMS > > > > > > > > (plural!), as a clear and present warning that this stuff c= an store > > > > > > > > lots of data? > > > > > > > > > > > > > > I like the idea, because it fixes the problem that the > > > > > > > |O_ATTR|/|O_NAMEDATTR| are treated like attributes, while SUN= 's > > > > > > > intention was to implement "Alternate Data Streams" (which is= a > > > > > > > superset of "extended attributes" as seen in Win32 EA and Lin= ux XATTR, > > > > > > > but with none of their limitations and problems) for Windows,= MacOS, > > > > > > > VMS and mainframe OS support. > > > > > > > > > > > > > > Re Plural: NO, please read openat(2): You open ONE named (att= ribute) > > > > > > > stream by passing this flag and the name, or the index (dir) = by > > > > > > > passing "." as name. One attribute =3D=3D Singular. > > > > > > > > > > > > Please also update the related #defines: > > > > > > 1. for openat() > > > > > > O_NAMEDSTREAM > > > > > > > > > > > > 2. for pathconf() > > > > > > _PC_NAMEDSTREAM_ENABLED > > > > > > _PC_NAMEDSTREAM_EXISTS > > > > > > > > > > > > That is actually a naming convention better than XATTR. > > > > Well, I've been trying to ignore this, but I guess I have to respon= d... > > > > (Here's some rather random comments.) > > > > - The bike shed is already painted green and the paint is dry. > > > > I do not see any reason to repaint it blue. > > > > - As others have noted, every vendor has used a different name > > > > for this mechanism. > > > > The only place I am aware of where a standards document > > > > describes them is the NFSv4 RFCs and these documents > > > > call them "named attribute". > > > > - I'm not a Windows guy and when I read "stream" I think of > > > > the SVR4 mechanism for handling network connections, > > > > which makes me shy of "namedstream". > > > > > > Fortunately, SVR4 were dead already in previous century. We may safel= y > > > forget them. While being non-Windows guy, the NTFS is very good desig= ned and > > > deserves knowledge about it's format at least in tutorial level (Reis= erFS > > > tried to achieve same flexibility). They still call everything pertai= ning to > > > file as "attributes", though - so those are just several $DATA attrib= utes (in > > > addition to other attributes, such as ACLs), default one (vast majori= ty of > > > regular files) just has empty name. > > > > > > > - I chose a name other than "extended attribute" (which is > > > > what Solaris calls it) because FreeBSD already uses > > > > extended attribute for another mechanism (the one that > > > > I think originated in SGI Irix and is in Linux). > > > > > > Yes, in NTFS attribute $EA could be one of them. > > > > > > > - If you think Oracle is going to change O_XATTR to some > > > > other name, well I'd be really surprised if they did. > > > > (I thought Solaris was on life support at this time.) > > > > > > > > FreeBSD is now in "slush" for the FreeBSD15 release, > > > > which means commits are supposed to be focused on > > > > bugfixes and I do not see renaming these as bugfixes. > > > > > > > > If I see a groundswell of support for the name change > > > > by what I will refer to as FreeBSD users that post > > > > comments regularly, I will consider doing it after > > > > the FreeBSD15 release goes out. > > > > (Sorry, but from what I have seen, the list of people posting > > > > to this thread all appear to be somehow related to > > > > the Windows NFSv4.1 client and not people who > > > > have been using/contributing to FreeBSD for years. > > > > This doesn't meant your opinions are not valid, but > > > > I'd like to see support for this name change include > > > > others outside your group, since I think it is basically > > > > silly. If someone wants to use this mechanism they > > > > will use it, whatever it happens to be called.) > > > > > > I think we are discussing adding an additional name as alias, not a t= rue > > > renaming? > > > > I just had a bloody run-in with the Linux camp, who still AGGRESSIVELY > > tries to sell Linux XATTR as a solution for everything, and - of > > course - supersedes O_NAMEDATTR!! Which left me with a particularly > > bitter taste about that religious "all hail Linux XATTR" zealotism. > > Zero knowledge about ADS/O_XATTR on their side, zero logical > > arguments, but as soon as something has the letters 'ATTR' inside it > > MUST BE superseded, killed and buried by Linux XATTR!! All who oppose > > this must be slaughtered!!!! > Just to be clear, there are technical advantages to the Linux style > extended attributes (I think it was SGI Irix that first shipped a system > supporting this mechanism, but I could be incorrect w.r.t. this). > "man extattr" on FreeBSD > > On FreeBSD, the zfs property xattr is set to "sa" by default. > This setting means that extended attributes are stored in a block > which handled ACLs and small extended attributes. > This is more efficient, but does not allow what you call > "named stream" to work. > --> So, if a site only wants to support ACLs and small extended > attributes that are set atomically to a value, this is works well. > > If you do want what you call "named stream", then you need to > set "xattr=3Ddir". This makes all extended attributes, including ACLs > and small ones be stored as files in a directory. Less efficient, > but more general, in that large values (as in number of bytes) can > be stored and manipulated like any file. Wait. My goal was not to compare apples and oranges. "xttrs" can stay in ZFS xattr land, I am interested ONLY in "alternate data streams", and do not want to mingle them with "xattr" / "XATTR". Different use cases. That is also the reason why I want a clear and clean separation by name - one side is "xattr" (attributes!), the other side is "named streams", byte streams with unlimited size, and unlimited potential. > It's a technical tradeoff which happens to be easy to do for > OpenZFS because it has the Solaris underpinnings in it > for "xattr=3Ddir". Solaris UFS supports Solaris O_XATTR too. > In summary, FreeBSD is a small niche market and use of > "named stream" on FreeBSD will be a small niche of that > small niche. > > The simple fact that Linux has chosen to not support what > you call "named stream" guarantees that most open source > software will not use it. 1. I think the Linux issue is, like RichACLs, mostly a religious one. Look what they did to ReiserFS (which had alternate data streams), just because the authors wanted a richer interface. 2. I disagree about "most open source software will not use it". Linux open source software, yes, because their religion forbids "named stream" by the punishment of permanent attacks of their zealots. But there are OTHERS, z/OS in my case, OSX too. JAVA is doing work on named streams too. FreeBSD (thank you!!). Aur=C3=A9lien --=20 Aur=C3=A9lien Couderc Big Data/Data mining expert, chess enthusiast