Date: Tue, 5 Dec 2017 09:17:42 -0800 From: Conrad Meyer <cem@freebsd.org> To: Eitan Adler <eadler@freebsd.org> Cc: src-committers <src-committers@freebsd.org>, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r326554 - in head: . usr.bin/sponge usr.bin/sponge/tests usr.bin/tee Message-ID: <CAG6CVpURfvTWR2ZF4hyQFk=3THOTOn21Gthm%2BVPOUY6RHzUg5g@mail.gmail.com> In-Reply-To: <201712050355.vB53tApl074041@repo.freebsd.org> References: <201712050355.vB53tApl074041@repo.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Please revert this change, along with r326555 and r326557. I am not opposed to the idea of a version of this utility in base. I am opposed to breaking the build, and functionally broken programs. Thanks, Conrad On Mon, Dec 4, 2017 at 7:55 PM, Eitan Adler <eadler@freebsd.org> wrote: > Author: eadler > Date: Tue Dec 5 03:55:10 2017 > New Revision: 326554 > URL: https://svnweb.freebsd.org/changeset/base/326554 > > Log: > sponge(1): initial commit > > sponge(1) is a utility that reads input until > complete, then opens the output file, then > writes to it. This makes it useful in pipelines > that read and write to the same file. > > Reviewed by: wblock, jilles, imp, cem, danfe (all: various iterations) > Inspired by: https://joeyh.name/code/moreutils/ > > Added: > head/usr.bin/sponge/ > head/usr.bin/sponge/Makefile (contents, props changed) > head/usr.bin/sponge/sponge.1 (contents, props changed) > head/usr.bin/sponge/sponge.c (contents, props changed) > head/usr.bin/sponge/tests/ > head/usr.bin/sponge/tests/Makefile (contents, props changed) > head/usr.bin/sponge/tests/Makefile.depend (contents, props changed) > head/usr.bin/sponge/tests/sponge_test.sh (contents, props changed) > Modified: > head/.arclint > head/usr.bin/tee/tee.1 > > Modified: head/.arclint > ============================================================================== > --- head/.arclint Tue Dec 5 02:23:36 2017 (r326553) > +++ head/.arclint Tue Dec 5 03:55:10 2017 (r326554) > @@ -9,7 +9,8 @@ > "type": "spelling" > }, > "chmod": { > - "type": "chmod" > + "type": "chmod", > + "exclude": "(/tests/)" > }, > "merge-conflict": { > "type": "merge-conflict" > > Added: head/usr.bin/sponge/Makefile > ============================================================================== > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ head/usr.bin/sponge/Makefile Tue Dec 5 03:55:10 2017 (r326554) > @@ -0,0 +1,8 @@ > +# $FreeBSD$ > + > +PROG= sponge > + > +HAS_TESTS= > +SUBDIR.${MK_TESTS}+= tests > + > +.include <bsd.prog.mk> > > Added: head/usr.bin/sponge/sponge.1 > ============================================================================== > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ head/usr.bin/sponge/sponge.1 Tue Dec 5 03:55:10 2017 (r326554) > @@ -0,0 +1,75 @@ > +.\" Eitan Adler. All rights reserved. > +.\" > +.\" Redistribution and use in source and binary forms, with or without > +.\" modification, are permitted provided that the following conditions > +.\" are met: > +.\" 1. Redistributions of source code must retain the above copyright > +.\" notice, this list of conditions and the following disclaimer. > +.\" 2. Redistributions in binary form must reproduce the above copyright > +.\" notice, this list of conditions and the following disclaimer in the > +.\" documentation and/or other materials provided with the distribution. > +.\" > +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND > +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE > +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE > +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE > +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL > +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS > +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT > +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY > +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF > +.\" SUCH DAMAGE. > +.\" > +.\" $FreeBSD$ > +.\" > +.Dd November 1, 2017 > +.Dt SPONGE 1 > +.Os > +.Sh NAME > +.Nm sponge > +.Nd buffer stdin and write to stdout > +.Sh SYNOPSIS > +.Nm > +.Op Fl a > +.Ar filename > +.Sh DESCRIPTION > +The > +.Nm > +utility reads standard in until complete, then opens > +the output file and writes to it. > +This makes it useful in pipelines that read a file and then write to it. > +These options are available: > +.Bl -tag -width indent > +.It Fl a > +Open > +.Ar filename > +in append mode. > +.El > +.Pp > +If an attempt to allocate memory fails, > +.Nm > +fails without output. > +The file is written even if earlier components > +of the pipeline failed. > +.Sh SEE ALSO > +.Xr builtin 1 , > +.Xr csh 1 , > +.Xr getrusage 2 , > +.Xr tee 1 , > +.Xr wait 2 > +.Sh EXIT STATUS > +.Ex -std > +.Sh EXAMPLES > +A > +.Pa file > +can be be sorted "in place" by executing > +.Cm sort file | sponge file > +.Sh HISTORY > +The > +.Nm > +utility was written by > +.An Eitan Adler Aq Mt eadler@FreeBSD.org > +and first appeared > +in > +.Fx 12.0 . > > Added: head/usr.bin/sponge/sponge.c > ============================================================================== > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ head/usr.bin/sponge/sponge.c Tue Dec 5 03:55:10 2017 (r326554) > @@ -0,0 +1,189 @@ > +/*- > + * SPDX-License-Identifier: BSD-2-Clause > + * > + * Copyright (c) 2017 Eitan Adler > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * 1. Redistributions of source code must retain the above copyright > + * notice, this list of conditions and the following disclaimer. > + * 2. Redistributions in binary form must reproduce the above copyright > + * notice, this list of conditions and the following disclaimer in the > + * documentation and/or other materials provided with the distribution. > + * > + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND > + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE > + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE > + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE > + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL > + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS > + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT > + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY > + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF > + * SUCH DAMAGE. > + * > + * $FreeBSD$ > + */ > + > +#include <err.h> > +#include <fcntl.h> > +#include <limits.h> > +#include <stdbool.h> > +#include <stdio.h> > +#include <stdlib.h> > +#include <getopt.h> > +#include <unistd.h> > +#include <sys/uio.h> > + > +#define DEFAULT_BUF_SIZE 16384 > +#define DEFAULT_BUF_CNT 12 > + > +static int flag_append = 0; > + > +static void usage(void); > +static void *safe_malloc(size_t size); > +static void *safe_calloc(size_t count, size_t size); > +static void *safe_reallocf(void *ptr, size_t size); > + > +static void * > +safe_malloc(size_t size) > +{ > + void *ret; > + > + ret = malloc(size); > + if (ret == NULL) { > + err(1, "malloc failed"); > + } > + return (ret); > +} > + > +static void * > +safe_calloc(size_t count, size_t size) > +{ > + void *ret; > + > + ret = calloc(count, size); > + if (ret == NULL) { > + err(1, "calloc failed"); > + } > + return (ret); > +} > + > +static void * > +safe_reallocf(void *ptr, size_t size) > +{ > + void *ret; > + > + ret = reallocf(ptr, size); > + if (ret == NULL) { > + err(1, "reallocf failed"); > + } > + return (ret); > +} > + > +static void > +usage(void) > +{ > + fprintf(stderr, "usage: sponge [-a] filename\n"); > +} > + > +int > +main(int argc, char* argv[]) > +{ > + struct iovec *iov; > + char *buf; > + char *outfile; > + ssize_t i; > + size_t bufcnt; > + size_t whichbuf; > + size_t bufremain; > + long maxiovec; > + int fd; > + int openflags = O_WRONLY; > + int opt; > + > + while ((opt = getopt(argc, argv, "ah")) != -1) { > + switch (opt) { > + case 'a': > + flag_append = 1; > + break; > + case 'h': > + usage(); > + exit(0); > + case '?': > + default: > + usage(); > + exit(1); > + } > + } > + > + if (optind < argc) { > + outfile = argv[optind]; > + } > + > + > + bufcnt = DEFAULT_BUF_CNT; > + whichbuf = 0; > + iov = safe_calloc(bufcnt, sizeof(*iov)); > + > + for (;;) { > + buf = safe_malloc(DEFAULT_BUF_SIZE); > + i = read(STDIN_FILENO, buf, DEFAULT_BUF_SIZE); > + if (whichbuf == bufcnt) { > + bufcnt *= 2; > + iov = safe_reallocf(iov, bufcnt * sizeof(*iov)); > + } > + if (i < 0) { > + err(1, "read failed"); > + } > + if (i == 0) { > + free(buf); > + break; > + } > + iov[whichbuf].iov_base = buf; > + iov[whichbuf].iov_len = i; > + whichbuf++; > + } > + > + if (outfile) { > + if (flag_append) { > + openflags |= O_APPEND; > + } else { > + openflags |= O_TRUNC; > + } > + fd = open(outfile, openflags); > + } > + else { > + fd = STDOUT_FILENO; > + } > + > + if (fd < 0) { > + err(1, "failed to open"); > + } > + > + maxiovec = sysconf(_SC_IOV_MAX); > + if (maxiovec == -1) { > + maxiovec = _XOPEN_IOV_MAX; > + } > + bufcnt = whichbuf; > + bufremain = bufcnt; > + > + while (bufremain > 0) { > + whichbuf = (bufremain < maxiovec) ? bufremain : maxiovec; > + bufremain -= whichbuf; > + > + i = writev(fd, iov, whichbuf); > + if (i < 0) { > + err(1, "failed to write"); > + } > + } > + > + if (outfile) { > + i = close(fd); > + if (i < 0) { > + err(1, "failed to close"); > + } > + } > +} > > Added: head/usr.bin/sponge/tests/Makefile > ============================================================================== > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ head/usr.bin/sponge/tests/Makefile Tue Dec 5 03:55:10 2017 (r326554) > @@ -0,0 +1,9 @@ > +# $FreeBSD$ > + > +PACKAGE= ${TESTBASE}/usr.bin/sponge > + > +TESTSDIR= ${TESTSBASE}/usr.bin/sponge/tests > + > +ATF_TESTS_SH= cp_test > + > +.include <bsd.test.mk> > > Added: head/usr.bin/sponge/tests/Makefile.depend > ============================================================================== > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ head/usr.bin/sponge/tests/Makefile.depend Tue Dec 5 03:55:10 2017 (r326554) > @@ -0,0 +1,18 @@ > +# $FreeBSD$ > +# Autogenerated - do NOT edit! > + > +DIRDEPS = \ > + gnu/lib/csu \ > + include \ > + include/xlocale \ > + lib/${CSU_DIR} \ > + lib/atf/libatf-c \ > + lib/libc \ > + lib/libcompiler_rt \ > + > + > +.include <dirdeps.mk> > + > +.if ${DEP_RELDIR} == ${_DEP_RELDIR} > +# local dependencies - needed for -jN in clean tree > +.endif > > Added: head/usr.bin/sponge/tests/sponge_test.sh > ============================================================================== > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ head/usr.bin/sponge/tests/sponge_test.sh Tue Dec 5 03:55:10 2017 (r326554) > @@ -0,0 +1,48 @@ > +# $FreeBSD$ > +# > +# SPDX-License-Identifier: BSD-2-Clause > +# > +# Copyright 2017 Eitan Adler > +# All rights reserved. > +# > +# Redistribution and use in source and binary forms, with or without > +# modification, are permitted provided that the following conditions are > +# met: > +# > +# * Redistributions of source code must retain the above copyright > +# notice, this list of conditions and the following disclaimer. > +# * Redistributions in binary form must reproduce the above copyright > +# notice, this list of conditions and the following disclaimer in the > +# documentation and/or other materials provided with the distribution. > +# > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS > +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT > +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR > +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT > +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, > +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT > +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, > +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY > +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT > +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE > +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > + > +atf_test_case simple > +simple_body() { > + printf 'd\ne\na\n' >| x > + printf 'a\nd\ne\n' >| y > + sort x | sponge x > + > + atf_check -s exit:0 -o empty -e empty 'sort x | sponge x' > + > + if ! cmp -s x y; then > + echo "x and y differ, but they should be equal" > + diff -u x y > + atf_fail "Original and copy do not match" > + fi > + > +} > + > +atf_init_test_cases() { > + atf_add_test_case simple > +} > > Modified: head/usr.bin/tee/tee.1 > ============================================================================== > --- head/usr.bin/tee/tee.1 Tue Dec 5 02:23:36 2017 (r326553) > +++ head/usr.bin/tee/tee.1 Tue Dec 5 03:55:10 2017 (r326554) > @@ -72,6 +72,8 @@ utility takes the default action for all signals, > except in the event of the > .Fl i > option. > +.Sh SEE ALSO > +.Xr sponge 1 > .Sh EXIT STATUS > .Ex -std > .Sh STANDARDS >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAG6CVpURfvTWR2ZF4hyQFk=3THOTOn21Gthm%2BVPOUY6RHzUg5g>