From nobody Sat Mar 23 16:06:35 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V23wZ6VyCz5FKKD for ; Sat, 23 Mar 2024 16:06:38 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mail-io1-xd31.google.com (mail-io1-xd31.google.com [IPv6:2607:f8b0:4864:20::d31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4V23wZ3Xk0z4lQN for ; Sat, 23 Mar 2024 16:06:38 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=hardenedbsd.org header.s=google header.b=FwQd0Lmm; dmarc=none; spf=pass (mx1.freebsd.org: domain of shawn.webb@hardenedbsd.org designates 2607:f8b0:4864:20::d31 as permitted sender) smtp.mailfrom=shawn.webb@hardenedbsd.org Received: by mail-io1-xd31.google.com with SMTP id ca18e2360f4ac-7cf092f6bb7so126951739f.3 for ; Sat, 23 Mar 2024 09:06:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd.org; s=google; t=1711209997; x=1711814797; darn=freebsd.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=m2Z6Mi5nVDIOLpL6RGxY1mU6rn5mGvTWpdUgWIT5E+U=; b=FwQd0Lmm6N0OnbSWDIPPDkQ/V8vXCivNvkE8WMIKhvfSVRGzS6M4FJJpG7z0KoDD7u 1HvUmMTRUTFK3mxiwzXWJ7DE1ArE57tyM+NkGH1kvK9eOt3fPZUq7Z7ida75uUn/E0QZ iMo9WtN8SxohWGV5mseNPjmdxia2xsrbgMRmydQCvk1sekqfPx9DeJjo1Dkx4fhE+Wqb i4C0bZJ+r+glFoIRO2gVsHGN94Yf73kamTVS13/UHGsXFzQ3w1M+QItMG0oUcceGke9f uuH27JtaVgm27TQh3Pnym0Jw/4GZN93G801tXm+Q8/LHvdZ8LdrCRd6SqJIzuuqspmyo Yrvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711209997; x=1711814797; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=m2Z6Mi5nVDIOLpL6RGxY1mU6rn5mGvTWpdUgWIT5E+U=; b=lPv4fLazD9ax2/xok+//jOxC/rJ627BDKLxiIiKGzjbSjiXAbxDCdo/2BkHUUXRyM9 dYDHCUbYlhJiHDtXOYxOxlKiJEHs8G5C8ihDh/kV9gBYPeu6l7os4KZd/6HoZGXcziJ/ PxjUZxokM5gtSQ8EyeCCSSChBogG9NoSGqezqqEBl8Ki+z/mOhlZnDl/F2+a4JRKt1dc DM6/N6Nv5olCSMEkNrYOA/sRBHpo8L978X6j/PbSiiZhXa2MNf1dTPOHWAIFYFuBEXBy 1oepVcZfZMHd4y3PHF6wnbCPVxch4GQMeVMkCzPEA2LtbsD1P6Sd6Pu7qk6ng3qTxDJI dZ1w== X-Forwarded-Encrypted: i=1; AJvYcCW7mAEQbzJEeKp4Iv525hBNwCTqE02kEV12oCWaQhOxiy1ICLw2tCxepDPWVLmpcVSQZEcdoDjOKowF20G/8jOBbCYDiBfg0bNKYxU= X-Gm-Message-State: AOJu0Yw3QmHYci4bh8yQ7MqbxYdk8BTgXlGMLrULxtjDeIrMeYs9Dwzb Soi1cigHdF60LUED3bPsElsyugEu5fidZszWlYCRqjJW1cAmDkA9DOkpDgfMiwvOUITu4cqZnNb 0 X-Google-Smtp-Source: AGHT+IEj2byfFQf4iKeM7F8LyrIp2NghrpGbTe4CmcjZIDxaZNYsSt8y6twmvwfND406IVTGf59gzA== X-Received: by 2002:a5e:de49:0:b0:7d0:2d83:1519 with SMTP id e9-20020a5ede49000000b007d02d831519mr2967845ioq.4.1711209997142; Sat, 23 Mar 2024 09:06:37 -0700 (PDT) Received: from mutt-hbsd (174-24-72-211.clsp.qwest.net. [174.24.72.211]) by smtp.gmail.com with ESMTPSA id g8-20020a02b708000000b0047469b04c35sm1142741jam.65.2024.03.23.09.06.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 Mar 2024 09:06:36 -0700 (PDT) Date: Sat, 23 Mar 2024 16:06:35 +0000 From: Shawn Webb To: alan somers Cc: Alan Somers , freebsd-hackers@freebsd.org Subject: Re: Filesystem extended attributes and Capsicum Message-ID: X-Operating-System: FreeBSD mutt-hbsd 15.0-CURRENT-HBSD FreeBSD 15.0-CURRENT-HBSD X-PGP-Key: https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/blob/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc References: <56n3xzefvyd3s3b6p3ipggf5hshawrts4nezkw2oud3haihdhj@besj4txt7yar> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="qpnblpzeya3764pj" Content-Disposition: inline In-Reply-To: <56n3xzefvyd3s3b6p3ipggf5hshawrts4nezkw2oud3haihdhj@besj4txt7yar> X-Spamd-Bar: ----- X-Spamd-Result: default: False [-5.10 / 15.00]; SIGNED_PGP(-2.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; R_DKIM_ALLOW(-0.20)[hardenedbsd.org:s=google]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[hardenedbsd.org:+]; FROM_HAS_DN(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; DMARC_NA(0.00)[hardenedbsd.org]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ARC_NA(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TO_DN_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; MISSING_XM_UA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::d31:from] X-Rspamd-Queue-Id: 4V23wZ3Xk0z4lQN --qpnblpzeya3764pj Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Mar 23, 2024 at 04:00:42PM +0000, Shawn Webb wrote: > On Sat, Mar 23, 2024 at 02:54:22PM +0000, Shawn Webb wrote: > > On Sat, Mar 23, 2024 at 02:25:26PM +0000, Shawn Webb wrote: > > > On Sat, Mar 23, 2024 at 07:58:37AM -0600, alan somers wrote: > > > > On Fri, Mar 22, 2024 at 9:52=E2=80=AFPM Shawn Webb wrote: > > > > > > > > > > On Fri, Mar 22, 2024 at 08:07:17PM -0600, Alan Somers wrote: > > > > > > On Fri, Mar 22, 2024 at 6:56=E2=80=AFPM Shawn Webb wrote: > > > > > > > > > > > > > > On Fri, Mar 22, 2024 at 06:20:48PM -0600, Alan Somers wrote: > > > > > > > > On Fri, Mar 22, 2024 at 5:38=E2=80=AFPM Shawn Webb wrote: > > > > > > > > > > > > > > > > > > Hey all, > > > > > > > > > > > > > > > > > > I'm writing an application in which I hope to enable Caps= icum. I'm > > > > > > > > > experiencing an issue whereby extattr_get_fd fails with a= file > > > > > > > > > descriptor that has all the extended attribute capabiliti= es enabled > > > > > > > > > (CAP_EXTATTR_DELETE, CAP_EXTATTR_GET, CAP_EXTATTR_LIST, a= nd > > > > > > > > > CAP_EXTATTR_SET). > > > > > > > > > > > > > > > > > > Looking at the kernel source (sys/kern/vfs_extattr.c) tel= ls me that > > > > > > > > > kern_extattr_get_fd only requires CAP_EXTATTR_GET. > > > > > > > > > > > > > > > > > > So I'm a bit puzzled as to why my call to extattr_get_fd(= 2) is > > > > > > > > > failing. Am I doing something wrong or are filesystem ext= ended > > > > > > > > > attributes not supported in a Capabilities-enabled proces= s? > > > > > > > > > > > > > > > > > > Here's how I'm creating the file descriptor (before calli= ng > > > > > > > > > cap_enter(2)): > > > > > > > > > > > > > > > > > > =3D=3D=3D=3D BEGIN CODE =3D=3D=3D=3D > > > > > > > > > static int > > > > > > > > > open_file(const char *path) > > > > > > > > > { > > > > > > > > > cap_rights_t rights; > > > > > > > > > int fd; > > > > > > > > > > > > > > > > > > fd =3D open(path, O_PATH | O_CLOEXEC); > > > > > > > > > if (fd =3D=3D -1) { > > > > > > > > > return (-1); > > > > > > > > > } > > > > > > > > > > > > > > > > > > memset(&rights, 0, sizeof(rights)); > > > > > > > > > cap_rights_init(&rights, CAP_EXTATTR_DELETE, CAP_= EXTATTR_GET, > > > > > > > > > CAP_EXTATTR_LIST, CAP_EXTATTR_SET); > > > > > > > > > cap_rights_limit(fd, &rights); > > > > > > > > > > > > > > > > > > return (fd); > > > > > > > > > } > > > > > > > > > =3D=3D=3D=3D END CODE =3D=3D=3D=3D > > > > > > > > > > > > > > > > > > Eventually, after calling cap_enter(2), the following cod= e is called: > > > > > > > > > > > > > > > > > > =3D=3D=3D=3D BEGIN CODE =3D=3D=3D=3D > > > > > > > > > #define ATTRNAME_ENABLED "hbsd.pax.aslr" > > > > > > > > > sz =3D extattr_get_fd(fd, ctx->hc_namespace, ATTR= NAME_ENABLED, NULL, 0); > > > > > > > > > if (sz <=3D 0) { > > > > > > > > > if (errno =3D=3D ENOATTR) { > > > > > > > > > /* > > > > > > > > > * This is okay, it just means tha= t nothing has been set. > > > > > > > > > * No error condition here. > > > > > > > > > */ > > > > > > > > > return (RES_SUCCESS); > > > > > > > > > } > > > > > > > > > return (RES_FAIL); > > > > > > > > > } > > > > > > > > > =3D=3D=3D=3D END CODE =3D=3D=3D=3D > > > > > > > > > > > > > > > > > > For reference, the program's code is here: > > > > > > > > > https://git.hardenedbsd.org/shawn.webb/hbsdctrl/-/tree/ma= in?ref_type=3Dheads > > > > > > > > > > > > > > > > > > The library code, which is what's responsible for calling= the > > > > > > > > > filesystem extended attribute related syscalls is here: > > > > > > > > > > > > > > > > > > https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/tre= e/hardened/current/hbsdcontrol-v2/lib/libhbsdcontrol?ref_type=3Dheads > > > > > > > > > > > > > > > > > > From the rights(4) manual page, I'm instructed all I need= are to apply > > > > > > > > > those capabilities to that file descriptor: > > > > > > > > > > > > > > > > > > =3D=3D=3D=3D BEGIN PASTE =3D=3D=3D=3D > > > > > > > > > CAP_EXTATTR_DELETE Permit extattr_delete_fd(2). > > > > > > > > > > > > > > > > > > CAP_EXTATTR_GET Permit extattr_get_fd(2). > > > > > > > > > > > > > > > > > > CAP_EXTATTR_LIST Permit extattr_list_fd(2). > > > > > > > > > > > > > > > > > > CAP_EXTATTR_SET Permit extattr_set_fd(2). > > > > > > > > > =3D=3D=3D=3D END PASTE =3D=3D=3D=3D > > > > > > > > > > > > > > > > > > So I'm a bit unsure if I'm doing something wrong. > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Shawn Webb > > > > > > > > > Cofounder / Security Engineer > > > > > > > > > HardenedBSD > > > > > > > > > > > > > > > > > > Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50 > > > > > > > > > https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/mas= ter/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc > > > > > > > > > > > > > > > > What error code does it fail with? If it's ENOTCAPABLE, th= en I > > > > > > > > suggest using dtrace to find the reason why it fails. Do s= omething > > > > > > > > like this: > > > > > > > > > > > > > > > > dtrace -i 'fbt:kernel::return /arg1 =3D=3D 93 && pid =3D=3D= $target/ > > > > > > > > {trace(".");}' -c ./my_application > > > > > > > > > > > > > > > > That will print the name of every non-inlined kernel functi= on that > > > > > > > > returns ENOTCAPABLE during your process. But it will also = print the > > > > > > > > names of any other kernel functions that return an integer = value of > > > > > > > > 93. From there, guess which function is the real source of= the error. > > > > > > > > Then you can do > > > > > > > > > > > > > > DTrace is unavailable on this particular system. > > > > > > > > > > > > > > It does indeed fail with ENOTCAPABLE. I have the kern.trap_en= otcap sysctl > > > > > > > set to 1 so that I can know at exactly what point we're faili= ng, and > > > > > > > it's indeed at extattr_get_fd. > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > -- > > > > > > > Shawn Webb > > > > > > > Cofounder / Security Engineer > > > > > > > HardenedBSD > > > > > > > > > > > > > > Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50 > > > > > > > https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/= Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc > > > > > > > > > > > > Without dtrace, you've got your work cut out for you. I suggest > > > > > > simply adding all capabilities, verifying that extattr_get_fd w= orks, > > > > > > and then removing capabilities until it fails. Or, run your pr= ogram > > > > > > on vanilla FreeBSD with dtrace. > > > > > > > > > > HardenedBSD doesn't have any modifications that would affect Caps= icum > > > > > in this manner. Regardless, I reproduced the problem successfully= on > > > > > FreeBSD 14.0-RELEASE without any code changes. I tried running yo= ur > > > > > DTrace script on FreeBSD 14.0-RELEASE and got this output: > > > > > > > > > > =3D=3D=3D=3D BEGIN OUTPUT =3D=3D=3D=3D > > > > > $ sudo dtrace -i 'fbt:kernel::return /arg1 =3D=3D 93 && pid =3D= =3D $target/ {trace(".");}' -c "obj/hbsdctrl pax list /bin/ls" > > > > > dtrace: description 'fbt:kernel::return ' matched 31396 probes > > > > > aslr: sysdef > > > > > mprotect: sysdef > > > > > pageexec: sysdef > > > > > segvguard: sysdef > > > > > dtrace: pid 29270 has exited > > > > > CPU ID FUNCTION:NAME > > > > > 3 47780 foffset_unlock_uio:return . > > > > > 3 50605 foffset_lock:return . > > > > > 3 47778 foffset_lock_uio:return . > > > > > =3D=3D=3D=3D END OUTPUT =3D=3D=3D=3D > > > > > > > > > > But I'm still unsure what I'm missing, if anything. > > > >=20 > > > > That's red herring. Those functions return void, but dtrace doesn't > > > > know it. So the "93" is just register garbage. I also notice that > > > > kern_extattr_get_fd isn't listed. Are you sure that your program is > > > > really failing with ENOTCAPABLE? You can also try running it with > > > > ktrace. kdump will show you exactly what capabilities you limited = the > > > > file descriptor to. That can help you verify if you applied the > > > > limits correctly. > > >=20 > > > I'm pretty sure it's failing with ENOTCAPABLE, since the > > > kern.trap_enotcap sysctl logic is being hit at the point of the > > > extattr_get_fd syscall. I'll see what I can do > > > with ktrace. > >=20 > > ktrace output captured here: > > https://hardenedbsd.org/~shawn/2024-03-23_kdump-01.txt > >=20 > > The relevant portion is pasted here: > >=20 > > =3D=3D=3D=3D BEGIN ktrace OUTPUT =3D=3D=3D=3D > > 41878 hbsdctrl CALL openat(AT_FDCWD,0x6894a36d4f86,0x500000) > > 41878 hbsdctrl NAMI "/scratch/tmp/ls" > > 41878 hbsdctrl RET openat 3 > > 41878 hbsdctrl CALL cap_rights_limit(0x3,0x6894a36d4bc0) > > 41878 hbsdctrl STRU cap_rights_t CAP_SEEK,CAP_FSTATAT,CAP_EXTATTR_DEL= ETE,CAP_EXTATTR_GET,CAP_EXTATTR_LIST,CAP_EXTATTR_SET,CAP_ACL_GET > > 41878 hbsdctrl RET cap_rights_limit 0 > > 41878 hbsdctrl CALL cap_enter > > 41878 hbsdctrl RET cap_enter 0 > > 41878 hbsdctrl CALL extattr_get_fd(0x3,0x2,0x33ff63afbaf,0,0) > > 41878 hbsdctrl NAMI "freebsd:system:hbsd.pax.segvguard" > > 41878 hbsdctrl CAP restricted VFS lookup > > 41878 hbsdctrl RET extattr_get_fd -1 errno 94 Not permitted in capab= ility mode > > 41878 hbsdctrl PSIG SIGTRAP SIG_DFL code=3DTRAP_CAP > > =3D=3D=3D=3D END ktrace OUTPUT =3D=3D=3D=3D > >=20 > > As you can see, I'm adding a bunch more capabilities to the file > > descriptor. The point of failure is still extattr_get_fd returning > > ENOTCAPABLE. > >=20 > > I'm beginning to suspect that while extattr_get_fd is documented to be > > allowed with a file descriptor with the CAP_EXTATTR_GET capability, > > this functionality is broken in FreeBSD. > >=20 > > The thing that really confuses me is that the extattr_get_fd syscall > > entry in syscalls.master has the CAPENABLED flag set: > >=20 > > =3D=3D=3D=3D BEGIN syscalls.master =3D=3D=3D=3D > > 372 AUE_EXTATTR_GET_FD STD|CAPENABLED { > > ssize_t extattr_get_fd( > > int fd, > > int attrnamespace, > > _In_z_ const char *attrname, > > _Out_writes_bytes_(nbytes) void *data, > > size_t nbytes > > ); > > } > > =3D=3D=3D=3D END syscalls.master =3D=3D=3D=3D > >=20 > > I think it's now time for me to file a bug report in Bugzilla. >=20 > Bug report submitted: > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D277908 Last message, sorry to bother. ZFS is indeed the culprit here: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D277908#c1 Thanks, --=20 Shawn Webb Cofounder / Security Engineer HardenedBSD Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50 https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A= 4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc --qpnblpzeya3764pj Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEA6TL67gupaZ9nzhT/y5nonf44foFAmX+/gsACgkQ/y5nonf4 4folIw//W8aSWovfB3l7W3jW93QXFinDE9qmXzMbxMzs2cTVoi8l7GwENxaJIuNb jDC74U4UeH/XBg6Ws+qVjpgj9ncqguS/dcRl/vdmBTAUegkb3gJzjsQO14MnwlnJ 8PCuGVRXrogSZRsQGhuyKbh7SXiCct/077uxO+iggD9r0GPyETWa8wHHZKtgOUNL esy4cIJD77C2wYQEm7dRpPhWuS5Do6bvpApNthZ9XtDFoiKlbmvLLfVft+5NDMXL XpLGSoz0h+9O1xNi72OfMwST0P3cpmEsz1x7NrHqHwde2JOk1g5g/wqSEexTi5Mb U5Bh6BNSnItroF6YDbXMrr0JrSkI/0XlRHwJuFgC/4i4nMVpjsXKPRdv8xgH7mle /ZCyDdwiaNgmU0+hFi/z8Uqqom161p4w/2WWOtp6AeMwVD7yYkL7D7w1TjoSt7iy WKBFy5kkH2HFPm6QSFN/K68zBaZ+cnl2FQBg70F6exN6/blkByo6E3y3uEGatksJ J8E5u/bzEBDA/9s+rYiW1wJQpBQ7RVDE94yFms5fbWeW1Ze3g4cx/McRv1aLAunR tFxnXWpAb5/45DY+RW0S0OEM9GxoPZgPqG/AuGAWfOeRE+TlxVQpdoZnDb6mT5uS 51Gr8zgw/WWF/VYEHPE/VPZqV86HzTodfg0qYUDSMGJ9ih4wNkw= =sJBp -----END PGP SIGNATURE----- --qpnblpzeya3764pj--