Date: Sat, 23 Mar 2024 14:54:22 +0000 From: Shawn Webb <shawn.webb@hardenedbsd.org> To: alan somers <asomers@gmail.com> Cc: Alan Somers <asomers@freebsd.org>, freebsd-hackers@freebsd.org Subject: Re: Filesystem extended attributes and Capsicum Message-ID: <om7536duc4lo6zsu2rmwgtghya6uxouwvuxec6czxp3lp4y7ey@pov3ksihtthl> In-Reply-To: <tnbavez4e42ovbvyjrzguf6xhydrgrn3olfcwhuouxwhzh57c6@jplqitihfove> References: <bjeg3z2aa5owo7uur75olwuw6le2g3whzxwfgef4ozyy6w6kog@33b426ax6inf> <CAOtMX2gC3iZ6X6HotpgxgfAfPN-8C7rg3ZCP%2Bpr5e8KddasUsQ@mail.gmail.com> <ae4isfrqtyc7l3rwqd2rz6efmuztwqxw5h67xqaqts6ra6br3l@ly347trhgdoq> <CAOtMX2hiWRPQM5mbrhTifnycgroQ2Y_MeCHN8bZ9mfypQYR_GA@mail.gmail.com> <gmrellzwibhxqazp3vt22tqk2flrsyklrcwrufr4mgnhjnj3cn@rufmfxktmop5> <CAOtMX2hoXr6-6rJHRcmistroZLqyZZLeCByU_amPNPVikOnG_A@mail.gmail.com> <tnbavez4e42ovbvyjrzguf6xhydrgrn3olfcwhuouxwhzh57c6@jplqitihfove>
next in thread | previous in thread | raw e-mail | index | archive | help
--zogso432rrk4m4bp Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Mar 23, 2024 at 02:25:26PM +0000, Shawn Webb wrote: > On Sat, Mar 23, 2024 at 07:58:37AM -0600, alan somers wrote: > > On Fri, Mar 22, 2024 at 9:52=E2=80=AFPM Shawn Webb <shawn.webb@hardened= bsd.org> wrote: > > > > > > On Fri, Mar 22, 2024 at 08:07:17PM -0600, Alan Somers wrote: > > > > On Fri, Mar 22, 2024 at 6:56=E2=80=AFPM Shawn Webb <shawn.webb@hard= enedbsd.org> wrote: > > > > > > > > > > On Fri, Mar 22, 2024 at 06:20:48PM -0600, Alan Somers wrote: > > > > > > On Fri, Mar 22, 2024 at 5:38=E2=80=AFPM Shawn Webb <shawn.webb@= hardenedbsd.org> wrote: > > > > > > > > > > > > > > Hey all, > > > > > > > > > > > > > > I'm writing an application in which I hope to enable Capsicum= =2E I'm > > > > > > > experiencing an issue whereby extattr_get_fd fails with a file > > > > > > > descriptor that has all the extended attribute capabilities e= nabled > > > > > > > (CAP_EXTATTR_DELETE, CAP_EXTATTR_GET, CAP_EXTATTR_LIST, and > > > > > > > CAP_EXTATTR_SET). > > > > > > > > > > > > > > Looking at the kernel source (sys/kern/vfs_extattr.c) tells m= e that > > > > > > > kern_extattr_get_fd only requires CAP_EXTATTR_GET. > > > > > > > > > > > > > > So I'm a bit puzzled as to why my call to extattr_get_fd(2) is > > > > > > > failing. Am I doing something wrong or are filesystem extended > > > > > > > attributes not supported in a Capabilities-enabled process? > > > > > > > > > > > > > > Here's how I'm creating the file descriptor (before calling > > > > > > > cap_enter(2)): > > > > > > > > > > > > > > =3D=3D=3D=3D BEGIN CODE =3D=3D=3D=3D > > > > > > > static int > > > > > > > open_file(const char *path) > > > > > > > { > > > > > > > cap_rights_t rights; > > > > > > > int fd; > > > > > > > > > > > > > > fd =3D open(path, O_PATH | O_CLOEXEC); > > > > > > > if (fd =3D=3D -1) { > > > > > > > return (-1); > > > > > > > } > > > > > > > > > > > > > > memset(&rights, 0, sizeof(rights)); > > > > > > > cap_rights_init(&rights, CAP_EXTATTR_DELETE, CAP_EXTA= TTR_GET, > > > > > > > CAP_EXTATTR_LIST, CAP_EXTATTR_SET); > > > > > > > cap_rights_limit(fd, &rights); > > > > > > > > > > > > > > return (fd); > > > > > > > } > > > > > > > =3D=3D=3D=3D END CODE =3D=3D=3D=3D > > > > > > > > > > > > > > Eventually, after calling cap_enter(2), the following code is= called: > > > > > > > > > > > > > > =3D=3D=3D=3D BEGIN CODE =3D=3D=3D=3D > > > > > > > #define ATTRNAME_ENABLED "hbsd.pax.aslr" > > > > > > > sz =3D extattr_get_fd(fd, ctx->hc_namespace, ATTRNAME= _ENABLED, NULL, 0); > > > > > > > if (sz <=3D 0) { > > > > > > > if (errno =3D=3D ENOATTR) { > > > > > > > /* > > > > > > > * This is okay, it just means that no= thing has been set. > > > > > > > * No error condition here. > > > > > > > */ > > > > > > > return (RES_SUCCESS); > > > > > > > } > > > > > > > return (RES_FAIL); > > > > > > > } > > > > > > > =3D=3D=3D=3D END CODE =3D=3D=3D=3D > > > > > > > > > > > > > > For reference, the program's code is here: > > > > > > > https://git.hardenedbsd.org/shawn.webb/hbsdctrl/-/tree/main?r= ef_type=3Dheads > > > > > > > > > > > > > > The library code, which is what's responsible for calling the > > > > > > > filesystem extended attribute related syscalls is here: > > > > > > > > > > > > > > https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/tree/ha= rdened/current/hbsdcontrol-v2/lib/libhbsdcontrol?ref_type=3Dheads > > > > > > > > > > > > > > From the rights(4) manual page, I'm instructed all I need are= to apply > > > > > > > those capabilities to that file descriptor: > > > > > > > > > > > > > > =3D=3D=3D=3D BEGIN PASTE =3D=3D=3D=3D > > > > > > > CAP_EXTATTR_DELETE Permit extattr_delete_fd(2). > > > > > > > > > > > > > > CAP_EXTATTR_GET Permit extattr_get_fd(2). > > > > > > > > > > > > > > CAP_EXTATTR_LIST Permit extattr_list_fd(2). > > > > > > > > > > > > > > CAP_EXTATTR_SET Permit extattr_set_fd(2). > > > > > > > =3D=3D=3D=3D END PASTE =3D=3D=3D=3D > > > > > > > > > > > > > > So I'm a bit unsure if I'm doing something wrong. > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > -- > > > > > > > Shawn Webb > > > > > > > Cofounder / Security Engineer > > > > > > > HardenedBSD > > > > > > > > > > > > > > Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50 > > > > > > > https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/= Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc > > > > > > > > > > > > What error code does it fail with? If it's ENOTCAPABLE, then I > > > > > > suggest using dtrace to find the reason why it fails. Do somet= hing > > > > > > like this: > > > > > > > > > > > > dtrace -i 'fbt:kernel::return /arg1 =3D=3D 93 && pid =3D=3D $ta= rget/ > > > > > > {trace(".");}' -c ./my_application > > > > > > > > > > > > That will print the name of every non-inlined kernel function t= hat > > > > > > returns ENOTCAPABLE during your process. But it will also prin= t the > > > > > > names of any other kernel functions that return an integer valu= e of > > > > > > 93. From there, guess which function is the real source of the= error. > > > > > > Then you can do > > > > > > > > > > DTrace is unavailable on this particular system. > > > > > > > > > > It does indeed fail with ENOTCAPABLE. I have the kern.trap_enotca= p sysctl > > > > > set to 1 so that I can know at exactly what point we're failing, = and > > > > > it's indeed at extattr_get_fd. > > > > > > > > > > Thanks, > > > > > > > > > > -- > > > > > Shawn Webb > > > > > Cofounder / Security Engineer > > > > > HardenedBSD > > > > > > > > > > Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50 > > > > > https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shaw= n_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc > > > > > > > > Without dtrace, you've got your work cut out for you. I suggest > > > > simply adding all capabilities, verifying that extattr_get_fd works, > > > > and then removing capabilities until it fails. Or, run your program > > > > on vanilla FreeBSD with dtrace. > > > > > > HardenedBSD doesn't have any modifications that would affect Capsicum > > > in this manner. Regardless, I reproduced the problem successfully on > > > FreeBSD 14.0-RELEASE without any code changes. I tried running your > > > DTrace script on FreeBSD 14.0-RELEASE and got this output: > > > > > > =3D=3D=3D=3D BEGIN OUTPUT =3D=3D=3D=3D > > > $ sudo dtrace -i 'fbt:kernel::return /arg1 =3D=3D 93 && pid =3D=3D $t= arget/ {trace(".");}' -c "obj/hbsdctrl pax list /bin/ls" > > > dtrace: description 'fbt:kernel::return ' matched 31396 probes > > > aslr: sysdef > > > mprotect: sysdef > > > pageexec: sysdef > > > segvguard: sysdef > > > dtrace: pid 29270 has exited > > > CPU ID FUNCTION:NAME > > > 3 47780 foffset_unlock_uio:return . > > > 3 50605 foffset_lock:return . > > > 3 47778 foffset_lock_uio:return . > > > =3D=3D=3D=3D END OUTPUT =3D=3D=3D=3D > > > > > > But I'm still unsure what I'm missing, if anything. > >=20 > > That's red herring. Those functions return void, but dtrace doesn't > > know it. So the "93" is just register garbage. I also notice that > > kern_extattr_get_fd isn't listed. Are you sure that your program is > > really failing with ENOTCAPABLE? You can also try running it with > > ktrace. kdump will show you exactly what capabilities you limited the > > file descriptor to. That can help you verify if you applied the > > limits correctly. >=20 > I'm pretty sure it's failing with ENOTCAPABLE, since the > kern.trap_enotcap sysctl logic is being hit at the point of the > extattr_get_fd syscall. I'll see what I can do > with ktrace. ktrace output captured here: https://hardenedbsd.org/~shawn/2024-03-23_kdump-01.txt The relevant portion is pasted here: =3D=3D=3D=3D BEGIN ktrace OUTPUT =3D=3D=3D=3D 41878 hbsdctrl CALL openat(AT_FDCWD,0x6894a36d4f86,0x500000<O_RDONLY|O_CL= OEXEC|O_PATH>) 41878 hbsdctrl NAMI "/scratch/tmp/ls" 41878 hbsdctrl RET openat 3 41878 hbsdctrl CALL cap_rights_limit(0x3,0x6894a36d4bc0) 41878 hbsdctrl STRU cap_rights_t CAP_SEEK,CAP_FSTATAT,CAP_EXTATTR_DELETE,= CAP_EXTATTR_GET,CAP_EXTATTR_LIST,CAP_EXTATTR_SET,CAP_ACL_GET 41878 hbsdctrl RET cap_rights_limit 0 41878 hbsdctrl CALL cap_enter 41878 hbsdctrl RET cap_enter 0 41878 hbsdctrl CALL extattr_get_fd(0x3,0x2,0x33ff63afbaf,0,0) 41878 hbsdctrl NAMI "freebsd:system:hbsd.pax.segvguard" 41878 hbsdctrl CAP restricted VFS lookup 41878 hbsdctrl RET extattr_get_fd -1 errno 94 Not permitted in capabilit= y mode 41878 hbsdctrl PSIG SIGTRAP SIG_DFL code=3DTRAP_CAP =3D=3D=3D=3D END ktrace OUTPUT =3D=3D=3D=3D As you can see, I'm adding a bunch more capabilities to the file descriptor. The point of failure is still extattr_get_fd returning ENOTCAPABLE. I'm beginning to suspect that while extattr_get_fd is documented to be allowed with a file descriptor with the CAP_EXTATTR_GET capability, this functionality is broken in FreeBSD. The thing that really confuses me is that the extattr_get_fd syscall entry in syscalls.master has the CAPENABLED flag set: =3D=3D=3D=3D BEGIN syscalls.master =3D=3D=3D=3D 372 AUE_EXTATTR_GET_FD STD|CAPENABLED { ssize_t extattr_get_fd( int fd, int attrnamespace, _In_z_ const char *attrname, _Out_writes_bytes_(nbytes) void *data, size_t nbytes ); } =3D=3D=3D=3D END syscalls.master =3D=3D=3D=3D I think it's now time for me to file a bug report in Bugzilla. Thanks, --=20 Shawn Webb Cofounder / Security Engineer HardenedBSD Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50 https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A= 4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc --zogso432rrk4m4bp Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEA6TL67gupaZ9nzhT/y5nonf44foFAmX+7RcACgkQ/y5nonf4 4froBA/9FQFIMjo0iYo8XvF/hWv2iKRoNZo18Sqe84P4h402cW6ornsPwpJ9MI9C 72wYkXPto3WcO4MyFfs9kYoZrzHZqs9EduCmoEgFV10PrXhiAiVMRW98VWay8981 gr9mGNejgbbe5LFWtrV3qIWoxSEhxDGlI55WZhysxcxBrHzmE/WbohPZInN0IaPt SbXYsWeYfriPvMU4A4E6sik7ZhhKG3B7m91z5cKguEwNciTk5LUDQJZiA/oCIALt fHXCeYyNdf7QCLiwq+mYzJ4tLguQX+b02tZDkbGKADD9BofBy1Le1+prnL7TlMOh GWThGhkk2qLYyw3B+++LlpeWB20+Aj2wCO5WK4m78VKh/PRql+Vi4mBu0SV7Gw1C DX/PSMJJeTdDszxP0N9F2i/yymakf0bcXu/yLoOPWPTF46RHEfA18XO4YDRqJ2AC 9qN+F48+FF08YY+l7kXVH3ujheLoVkj6A7jm3wJJMrWapEq5mAc2iR3hnmaSMe2t 6zIc95jNYli1xyRsEczz0OSsLgEz8CiyzIlbhvHARSlgtmXNpjrZai16xMURMtgj LHiadzQ9wP8P4xHAMpxxl9RaTaQBsJLv3H4C9lNxPyFVZDrFOccYniUwRWJHVG9E cjRgI1q8SChIRK9AYaBb/zjyenwdxN7sXphRdO5uuVK1nWaE5SE= =hCeC -----END PGP SIGNATURE----- --zogso432rrk4m4bp--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?om7536duc4lo6zsu2rmwgtghya6uxouwvuxec6czxp3lp4y7ey>