From owner-freebsd-pkg@freebsd.org Sat Jul 4 14:14:18 2020 Return-Path: Delivered-To: freebsd-pkg@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 59A0E34F47F for ; Sat, 4 Jul 2020 14:14:18 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 49zYjk0qfPz3Vpw for ; Sat, 4 Jul 2020 14:14:18 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 1A49134ECE6; Sat, 4 Jul 2020 14:14:18 +0000 (UTC) Delivered-To: pkg@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1A05734F7DD for ; Sat, 4 Jul 2020 14:14:18 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mail-qt1-x843.google.com (mail-qt1-x843.google.com [IPv6:2607:f8b0:4864:20::843]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49zYjh6pRpz3VkL for ; Sat, 4 Jul 2020 14:14:16 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: by mail-qt1-x843.google.com with SMTP id z2so25832830qts.5 for ; Sat, 04 Jul 2020 07:14:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd.org; s=google; h=date:from:to:cc:subject:message-id:mime-version:content-disposition; bh=Emk/auWKS3Es8pUyHFYPSsv+b37IwlY4KMcnv27okJQ=; b=jT4RJtzxQooCSyl7x+5vABwIgibbowOcuRy3tTZVsh7CMNvwBbGfPf2j0vty4vzfZB hAA+XA+ecupBJ6pzjAWg2XEqWjSUPODnECe7qKcrbV4QtEJnxo4c+KxlZZR66Hf8/Rqj qqt42EW7insTG4JHysYoderrSuYb9ree0pMHlRjqsx07dn0WZGv1mVH7ICp9WO8iNcb8 +ZEa80pPr+zbNZ77pzUPluAcrXVPjat4poBrK8nyXceLv65qE1ZNYAqMyLn++de958+S dy/NGiEi60J/1dIQXK1Zs4m4mAGG1xTVD8q6XkLxddr1z26x8Ik4D215Wmtu/cHdzWMR NZgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:mime-version :content-disposition; bh=Emk/auWKS3Es8pUyHFYPSsv+b37IwlY4KMcnv27okJQ=; b=hOwdQWDfcGUZOwkeDzK9V/g0ivLVufBrZtpgwTYjSGB82Ww2PZC4PpjeYeCqSTYfI2 4txLSF48+OAzOCvYxkKmyT4wp8ITSvZLFArV8PA+fzIul/QFMnIczAeLeNsVGX/un91F irmbah7jDb/t5Pjag3j/IP7y3pr9D2kuhyWlwXFcIidgf1O6fcJx7dwIPe1lXnb4S/KO QMIFwe5z/Iq/pLDWrH9qvkCiaIdbeVTmJSoC7vJFgKlLrQpUSAlVkVgeNi65GHL4BL0z mUkUSTg+sljooH5/AOdlZLDZkKLhEy14kzFsKAZMBrFTdgokuHKOKIVn7Hh5pKj1bkiW 7FMg== X-Gm-Message-State: AOAM53288AZ9hZony/6tdLVm8JM901s5fkLgW4gngU3C05mB0P28ijg+ rha+GcmqdYHTGIiZsGW0GQDhkX01rnIjF867xnj15jBXocuGgX/8nmJDk1IVj4KdSFPipinJccm LXHaFdA9G8Mks0sPQ0bNZAQSTJc6rNpOtqGMNj5Y5RJq3ufKvrB6sub6gkaRnOW5OPTv8kw== X-Google-Smtp-Source: ABdhPJxpDfN8hKqYxGUNGLpdQ3S6fcIhUX54ehbUvWQrr9fT17Ccg0MzsU1uirT2h3De8f/EOGOrJA== X-Received: by 2002:ac8:22cd:: with SMTP id g13mr42183614qta.100.1593872055432; Sat, 04 Jul 2020 07:14:15 -0700 (PDT) Received: from mutt-hbsd (pool-100-16-231-224.bltmmd.fios.verizon.net. [100.16.231.224]) by smtp.gmail.com with ESMTPSA id v10sm14355962qtk.14.2020.07.04.07.14.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 04 Jul 2020 07:14:14 -0700 (PDT) Date: Sat, 4 Jul 2020 10:13:45 -0400 From: Shawn Webb To: pkg@freebsd.org Cc: dev@hardenedbsd.org Subject: Filesystem extended attributes support Message-ID: <20200704141345.xwdf2ckxak2hfpkh@mutt-hbsd> X-Operating-System: FreeBSD mutt-hbsd 13.0-CURRENT-HBSD FreeBSD 13.0-CURRENT-HBSD X-PGP-Key: http://pgp.mit.edu/pks/lookup?op=vindex&search=0xFF2E67A277F8E1FA MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="mmsvrmd4pasf3ghv" Content-Disposition: inline X-Rspamd-Queue-Id: 49zYjh6pRpz3VkL X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=hardenedbsd.org header.s=google header.b=jT4RJtzx; dmarc=none; spf=pass (mx1.freebsd.org: domain of shawn.webb@hardenedbsd.org designates 2607:f8b0:4864:20::843 as permitted sender) smtp.mailfrom=shawn.webb@hardenedbsd.org X-Spamd-Result: default: False [-5.06 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[hardenedbsd.org:s=google]; NEURAL_HAM_MEDIUM(-0.99)[-0.991]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.001]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[pkg@freebsd.org]; TO_DN_NONE(0.00)[]; HAS_ATTACHMENT(0.00)[]; DMARC_NA(0.00)[hardenedbsd.org]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[hardenedbsd.org:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::843:from]; NEURAL_HAM_SHORT(-0.97)[-0.971]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~]; MID_RHS_NOT_FQDN(0.50)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_TLS_ALL(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[100.16.231.224:received] X-BeenThere: freebsd-pkg@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Binary package management and package tools discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Jul 2020 14:14:18 -0000 --mmsvrmd4pasf3ghv Content-Type: multipart/mixed; boundary="bbelxb2uxmln7nal" Content-Disposition: inline --bbelxb2uxmln7nal Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hey FreeBSD pkg(8) developers, Attached is a patch that implements filesystem extended attributes support. Only the system namespace is supported. In case the patch gets scrubbed from this email, I've posted it here: https://gist.github.com/d0b4653bc5942dbcdcd1db877d37c2dc Anyone who wants to write unit tests is welcomed to do so. This patch to pkg does depend on a separate patch to libarchive: https://github.com/libarchive/libarchive/pull/1409 HardenedBSD has a separate patch to tmpfs that enables incredibly basic extended attribute support. The tmpfs patch is only needed for those who use tmpfs with poudriere. And finally, another patch to the jails subsystem that allows a privileged user within a jail to set system namespace filesystem extended attributes (disabled by default) is needed for poudriere users. The patch to tmpfs and jails is not included here as they are tangential. Thanks, --=20 Shawn Webb Cofounder / Security Engineer HardenedBSD GPG Key ID: 0xFF2E67A277F8E1FA GPG Key Fingerprint: D206 BB45 15E0 9C49 0CF9 3633 C85B 0AF8 AB23 0FB2 https://git-01.md.hardenedbsd.org/HardenedBSD/pubkeys/src/branch/master/Sha= wn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc --bbelxb2uxmln7nal Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="pkg.patch.txt" Content-Transfer-Encoding: quoted-printable diff --git a/libpkg/Makefile.autosetup b/libpkg/Makefile.autosetup index ae722976..77854f16 100644 --- a/libpkg/Makefile.autosetup +++ b/libpkg/Makefile.autosetup @@ -42,7 +42,8 @@ SRCS=3D backup.c \ pkg_repo_create.c \ pkg_version.c \ rcscripts.c \ - flags.c + flags.c \ + extattr.c =20 LOCAL_CFLAGS=3D -I$(top_srcdir)/compat \ -I$(top_srcdir)/external/blake2 \ diff --git a/libpkg/extattr.c b/libpkg/extattr.c new file mode 100644 index 00000000..00618787 --- /dev/null +++ b/libpkg/extattr.c @@ -0,0 +1,119 @@ +/*- + * Copyright (c) 2020 Shawn Webb + *=20 + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer + * in this position and unchanged. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + *=20 + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTI= ES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF US= E, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include "pkg.h" +#include "private/event.h" +#include "private/pkg.h" + +int +pkg_archive_extattrs(int fd, struct archive_entry *entry) +{ + const char *nameprefix =3D "system."; + char *endp, *name, *names, *namep; + ssize_t datasize, listsize; + int err, namespace; + uint8_t namesize; + void *attrdata; + + err =3D EPKG_OK; + attrdata =3D NULL; + names =3D NULL; + + namespace =3D EXTATTR_NAMESPACE_SYSTEM; + + listsize =3D extattr_list_fd(fd, namespace, NULL, 0); + if (listsize < 0) { + return (EPKG_OK); + } + + if (listsize =3D=3D 0) { + return (EPKG_OK); + } + + names =3D calloc(listsize, 1); + if (names =3D=3D NULL) { + return (EPKG_OK); + } + + if (extattr_list_fd(fd, namespace, names, listsize) !=3D + listsize) { + goto end; + } + endp =3D names + (size_t)listsize; + for (namep =3D names; namep < endp; namep +=3D namesize) { + namesize =3D *((uint8_t *)(namep)); + name =3D calloc(strlen(nameprefix) + namesize+1, 1); + if (name =3D=3D NULL) { + goto end; + } + namep +=3D sizeof(uint8_t); + strcpy(name, nameprefix); + strncat(name, namep, namesize); + + datasize =3D extattr_get_fd(fd, namespace, name+strlen(nameprefix), NULL= , 0); + if (datasize < 0) { + free(name); + continue; + } + + attrdata =3D calloc(1, (size_t)datasize); + if (attrdata =3D=3D NULL) { + free(name); + goto end; + } + + if (extattr_get_fd(fd, namespace, name+strlen(nameprefix), attrdata, + datasize) !=3D datasize) { + perror("extattr_get_fd"); + free(name); + free(attrdata); + goto end; + } + + archive_entry_xattr_add_entry(entry, name, attrdata, + datasize); + + free(name); + free(attrdata); + } + +end: + free(names); + return (EPKG_OK); +} diff --git a/libpkg/packing.c b/libpkg/packing.c index 1e0e35b1..7424aa11 100644 --- a/libpkg/packing.c +++ b/libpkg/packing.c @@ -232,17 +232,21 @@ packing_append_file_attr(struct packing *pack, const = char *filepath, if (sparse_entry !=3D NULL && entry =3D=3D NULL) entry =3D sparse_entry; =20 - archive_write_header(pack->awrite, entry); - - if (archive_entry_size(entry) <=3D 0) - goto cleanup; - if ((fd =3D open(filepath, O_RDONLY)) < 0) { pkg_emit_errno("open", filepath); retcode =3D EPKG_FATAL; goto cleanup; } =20 + pkg_archive_extattrs(fd, entry); + + archive_write_header(pack->awrite, entry); + + if (archive_entry_size(entry) <=3D 0) { + close(fd); + goto cleanup; + } + while ((len =3D read(fd, buf, sizeof(buf))) > 0) { if (archive_write_data(pack->awrite, buf, len) =3D=3D -1) { pkg_emit_errno("archive_write_data", "archive write error"); diff --git a/libpkg/pkg_add.c b/libpkg/pkg_add.c index a2170bdc..dbf04f1e 100644 --- a/libpkg/pkg_add.c +++ b/libpkg/pkg_add.c @@ -43,6 +43,9 @@ #include #include =20 +#include +#include + #include "pkg.h" #include "private/event.h" #include "private/utils.h" @@ -526,6 +529,9 @@ create_regfile(struct pkg *pkg, struct pkg_file *f, str= uct archive *a, bool tried_mkdir =3D false; size_t len; char buf[32768]; + const char *attrname; + void *attrval; + size_t attrsz; =20 pkg_hidden_tempfile(f->temppath, sizeof(f->temppath), f->path); =20 @@ -572,6 +578,25 @@ retry: } } =20 + if (archive_entry_xattr_reset(ae)) { + attrname =3D NULL; + attrval =3D NULL; + attrsz =3D 0; + + while (archive_entry_xattr_next(ae, &attrname, + &attrval, &attrsz) =3D=3D ARCHIVE_OK) { + assert(attrname !=3D NULL); + assert(attrval !=3D NULL); + assert(attrsz > 0); + + if (!strncmp(attrname, "system.", 7)) { + extattr_set_fd(fd, + EXTATTR_NAMESPACE_SYSTEM, + attrname+7, attrval, attrsz); + } + } + } + if (!f->config && archive_read_data_into_fd(a, fd) !=3D ARCHIVE_OK) { pkg_emit_error("Fail to extract %s from package: %s", f->path, archive_error_string(a)); diff --git a/libpkg/private/pkg.h b/libpkg/private/pkg.h index e9021ed8..0826c515 100644 --- a/libpkg/private/pkg.h +++ b/libpkg/private/pkg.h @@ -881,4 +881,7 @@ struct pkg_dep* pkg_adddep_chain(struct pkg_dep *chain, void backup_library(struct pkgdb *, struct pkg *, const char *); int suggest_arch(struct pkg *, bool); =20 +/* Filesystem extended attribute support */ +int pkg_archive_extattrs(int, struct archive_entry *); + #endif --bbelxb2uxmln7nal-- --mmsvrmd4pasf3ghv Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEA6TL67gupaZ9nzhT/y5nonf44foFAl8AjpMACgkQ/y5nonf4 4foXtBAApcSQPTsPc9UFKt/OghfIfAgIRjluJ5hwv/kpRthlklnpK99DQbStqiPD 8mhd/3hP9EqMsV7DhW8RY4+aM1fg5qKYr6aIHUjRGyICbj9EHKYjGQVcA5Nm4qEw bsyzwNoY+E/4d6c3WAx1Wqd86gWX0H6hu6R9q2du7Xt8j9fZ91Dw3AKFaw7WpbHH gIPq/P9f71JO/wdOkLcW7D9cCcg5VM+0RWTEAmZvAq1xZ7GhXupBfYMPsaKuyIEa wNVp+WupgTOobtFH2EvAaBFggDTJiQHqbV/sodw8frGRCd4d1GNN032O117tM0h5 PSfw15GH2MQ1jJGq9AfS391CdlZJZOnsEt+jLV4zt6lWX9WzeA/gkV48Ls4Yg+BV U4U16jszrYlNZ+PgkhY0eqWt7IcQX7WUOn8FsJg0K1ajintHJwRqUr9JByscankb Jtd8+7Fjku/vdyp51NLDJ3kXyr5fqHOhtlhAOb3IzdEqMtmaqMgS9pvwyDRiHCAu 0l1/DE9zmjtxWbijG3J8e7XPZ0Hc5XJAlTzetjWP2X6wllQ5EbKbaGfJ6ai4USFD SKReYNQSbtaGNpFroNE990mCd0XKySz7W3NWyktGEi0PNJjUZ/hphF4YeztIClAE R6ZUaHY4tQr+GE+SIumF2WAkDIyH9flV8S4wAP9gtfNIJnu87AA= =G+lu -----END PGP SIGNATURE----- --mmsvrmd4pasf3ghv--