From owner-svn-src-all@freebsd.org Tue Dec 5 17:23:28 2017 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A6D6E71BCA; Tue, 5 Dec 2017 17:23:28 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-it0-f52.google.com (mail-it0-f52.google.com [209.85.214.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AC0CA6429A; Tue, 5 Dec 2017 17:23:27 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-it0-f52.google.com with SMTP id m11so17654781iti.1; Tue, 05 Dec 2017 09:23:27 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to:cc; bh=vZU/ms8NeTAe/5GLLeYBE2k6BwEXO0BAFPKm3ueo1F0=; b=oMi9ofeCZ9sZXuGrgAvStedTbK/HPWqmhiW8RP3g/tmKrTA8d2YYqZ7bVxQ81v2PjL DILhwMgEKvKFeOruTVw+vRGrOdn2bUm+ZJt4b84Dc3UTlMuPSfMc9+Ph4ZN0UfJUmAJO sdjjLk+V/Qgmfxmf9Ixxz8tv51rjOw7e4XmAW6fQ4TOnB/eJpyGcioGBUfpQZ77r3z0N lGj2rKCu2eBamuP7tLnOh08QeRrjHq7iNpFRcJ0NW77D6+QJrZfezC4ym4JP1O+gOkNS LXKWXzzMS2MyGSOy1FW+sBern87MPyNSwTFhXgFlZzjjDFWu+sNy2lpl7Z3YQmaZNd9L QLyg== X-Gm-Message-State: AKGB3mL1UB+Uk/BxiX4YnHSha3fXreyw6kvPC295chwjrG3QYSEpNCW2 SZsOfMdpNCi1qsZ8j7tpwVsXn5rM X-Google-Smtp-Source: AGs4zMaLxiIAfTLkPDgwke+5xW9e3EdbnGKhYoNrIL39/k7l7SsQFKfOZariXEy4Pr68tBnSSnT8pQ== X-Received: by 10.36.14.86 with SMTP id 83mr17253118ite.8.1512494264135; Tue, 05 Dec 2017 09:17:44 -0800 (PST) Received: from mail-it0-f48.google.com (mail-it0-f48.google.com. [209.85.214.48]) by smtp.gmail.com with ESMTPSA id u94sm236962ioi.67.2017.12.05.09.17.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 05 Dec 2017 09:17:43 -0800 (PST) Received: by mail-it0-f48.google.com with SMTP id z6so3298678iti.4; Tue, 05 Dec 2017 09:17:43 -0800 (PST) X-Received: by 10.36.192.2 with SMTP id u2mr18451271itf.119.1512494263458; Tue, 05 Dec 2017 09:17:43 -0800 (PST) MIME-Version: 1.0 Reply-To: cem@freebsd.org Received: by 10.2.165.150 with HTTP; Tue, 5 Dec 2017 09:17:42 -0800 (PST) In-Reply-To: <201712050355.vB53tApl074041@repo.freebsd.org> References: <201712050355.vB53tApl074041@repo.freebsd.org> From: Conrad Meyer Date: Tue, 5 Dec 2017 09:17:42 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: svn commit: r326554 - in head: . usr.bin/sponge usr.bin/sponge/tests usr.bin/tee To: Eitan Adler Cc: src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Dec 2017 17:23:28 -0000 Please revert this change, along with r326555 and r326557. I am not opposed to the idea of a version of this utility in base. I am opposed to breaking the build, and functionally broken programs. Thanks, Conrad On Mon, Dec 4, 2017 at 7:55 PM, Eitan Adler wrote: > Author: eadler > Date: Tue Dec 5 03:55:10 2017 > New Revision: 326554 > URL: https://svnweb.freebsd.org/changeset/base/326554 > > Log: > sponge(1): initial commit > > sponge(1) is a utility that reads input until > complete, then opens the output file, then > writes to it. This makes it useful in pipelines > that read and write to the same file. > > Reviewed by: wblock, jilles, imp, cem, danfe (all: various iterations) > Inspired by: https://joeyh.name/code/moreutils/ > > Added: > head/usr.bin/sponge/ > head/usr.bin/sponge/Makefile (contents, props changed) > head/usr.bin/sponge/sponge.1 (contents, props changed) > head/usr.bin/sponge/sponge.c (contents, props changed) > head/usr.bin/sponge/tests/ > head/usr.bin/sponge/tests/Makefile (contents, props changed) > head/usr.bin/sponge/tests/Makefile.depend (contents, props changed) > head/usr.bin/sponge/tests/sponge_test.sh (contents, props changed) > Modified: > head/.arclint > head/usr.bin/tee/tee.1 > > Modified: head/.arclint > ============================================================================== > --- head/.arclint Tue Dec 5 02:23:36 2017 (r326553) > +++ head/.arclint Tue Dec 5 03:55:10 2017 (r326554) > @@ -9,7 +9,8 @@ > "type": "spelling" > }, > "chmod": { > - "type": "chmod" > + "type": "chmod", > + "exclude": "(/tests/)" > }, > "merge-conflict": { > "type": "merge-conflict" > > Added: head/usr.bin/sponge/Makefile > ============================================================================== > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ head/usr.bin/sponge/Makefile Tue Dec 5 03:55:10 2017 (r326554) > @@ -0,0 +1,8 @@ > +# $FreeBSD$ > + > +PROG= sponge > + > +HAS_TESTS= > +SUBDIR.${MK_TESTS}+= tests > + > +.include > > Added: head/usr.bin/sponge/sponge.1 > ============================================================================== > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ head/usr.bin/sponge/sponge.1 Tue Dec 5 03:55:10 2017 (r326554) > @@ -0,0 +1,75 @@ > +.\" Eitan Adler. All rights reserved. > +.\" > +.\" Redistribution and use in source and binary forms, with or without > +.\" modification, are permitted provided that the following conditions > +.\" are met: > +.\" 1. Redistributions of source code must retain the above copyright > +.\" notice, this list of conditions and the following disclaimer. > +.\" 2. Redistributions in binary form must reproduce the above copyright > +.\" notice, this list of conditions and the following disclaimer in the > +.\" documentation and/or other materials provided with the distribution. > +.\" > +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND > +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE > +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE > +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE > +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL > +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS > +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT > +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY > +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF > +.\" SUCH DAMAGE. > +.\" > +.\" $FreeBSD$ > +.\" > +.Dd November 1, 2017 > +.Dt SPONGE 1 > +.Os > +.Sh NAME > +.Nm sponge > +.Nd buffer stdin and write to stdout > +.Sh SYNOPSIS > +.Nm > +.Op Fl a > +.Ar filename > +.Sh DESCRIPTION > +The > +.Nm > +utility reads standard in until complete, then opens > +the output file and writes to it. > +This makes it useful in pipelines that read a file and then write to it. > +These options are available: > +.Bl -tag -width indent > +.It Fl a > +Open > +.Ar filename > +in append mode. > +.El > +.Pp > +If an attempt to allocate memory fails, > +.Nm > +fails without output. > +The file is written even if earlier components > +of the pipeline failed. > +.Sh SEE ALSO > +.Xr builtin 1 , > +.Xr csh 1 , > +.Xr getrusage 2 , > +.Xr tee 1 , > +.Xr wait 2 > +.Sh EXIT STATUS > +.Ex -std > +.Sh EXAMPLES > +A > +.Pa file > +can be be sorted "in place" by executing > +.Cm sort file | sponge file > +.Sh HISTORY > +The > +.Nm > +utility was written by > +.An Eitan Adler Aq Mt eadler@FreeBSD.org > +and first appeared > +in > +.Fx 12.0 . > > Added: head/usr.bin/sponge/sponge.c > ============================================================================== > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ head/usr.bin/sponge/sponge.c Tue Dec 5 03:55:10 2017 (r326554) > @@ -0,0 +1,189 @@ > +/*- > + * SPDX-License-Identifier: BSD-2-Clause > + * > + * Copyright (c) 2017 Eitan Adler > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * 1. Redistributions of source code must retain the above copyright > + * notice, this list of conditions and the following disclaimer. > + * 2. Redistributions in binary form must reproduce the above copyright > + * notice, this list of conditions and the following disclaimer in the > + * documentation and/or other materials provided with the distribution. > + * > + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND > + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE > + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE > + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE > + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL > + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS > + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT > + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY > + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF > + * SUCH DAMAGE. > + * > + * $FreeBSD$ > + */ > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#define DEFAULT_BUF_SIZE 16384 > +#define DEFAULT_BUF_CNT 12 > + > +static int flag_append = 0; > + > +static void usage(void); > +static void *safe_malloc(size_t size); > +static void *safe_calloc(size_t count, size_t size); > +static void *safe_reallocf(void *ptr, size_t size); > + > +static void * > +safe_malloc(size_t size) > +{ > + void *ret; > + > + ret = malloc(size); > + if (ret == NULL) { > + err(1, "malloc failed"); > + } > + return (ret); > +} > + > +static void * > +safe_calloc(size_t count, size_t size) > +{ > + void *ret; > + > + ret = calloc(count, size); > + if (ret == NULL) { > + err(1, "calloc failed"); > + } > + return (ret); > +} > + > +static void * > +safe_reallocf(void *ptr, size_t size) > +{ > + void *ret; > + > + ret = reallocf(ptr, size); > + if (ret == NULL) { > + err(1, "reallocf failed"); > + } > + return (ret); > +} > + > +static void > +usage(void) > +{ > + fprintf(stderr, "usage: sponge [-a] filename\n"); > +} > + > +int > +main(int argc, char* argv[]) > +{ > + struct iovec *iov; > + char *buf; > + char *outfile; > + ssize_t i; > + size_t bufcnt; > + size_t whichbuf; > + size_t bufremain; > + long maxiovec; > + int fd; > + int openflags = O_WRONLY; > + int opt; > + > + while ((opt = getopt(argc, argv, "ah")) != -1) { > + switch (opt) { > + case 'a': > + flag_append = 1; > + break; > + case 'h': > + usage(); > + exit(0); > + case '?': > + default: > + usage(); > + exit(1); > + } > + } > + > + if (optind < argc) { > + outfile = argv[optind]; > + } > + > + > + bufcnt = DEFAULT_BUF_CNT; > + whichbuf = 0; > + iov = safe_calloc(bufcnt, sizeof(*iov)); > + > + for (;;) { > + buf = safe_malloc(DEFAULT_BUF_SIZE); > + i = read(STDIN_FILENO, buf, DEFAULT_BUF_SIZE); > + if (whichbuf == bufcnt) { > + bufcnt *= 2; > + iov = safe_reallocf(iov, bufcnt * sizeof(*iov)); > + } > + if (i < 0) { > + err(1, "read failed"); > + } > + if (i == 0) { > + free(buf); > + break; > + } > + iov[whichbuf].iov_base = buf; > + iov[whichbuf].iov_len = i; > + whichbuf++; > + } > + > + if (outfile) { > + if (flag_append) { > + openflags |= O_APPEND; > + } else { > + openflags |= O_TRUNC; > + } > + fd = open(outfile, openflags); > + } > + else { > + fd = STDOUT_FILENO; > + } > + > + if (fd < 0) { > + err(1, "failed to open"); > + } > + > + maxiovec = sysconf(_SC_IOV_MAX); > + if (maxiovec == -1) { > + maxiovec = _XOPEN_IOV_MAX; > + } > + bufcnt = whichbuf; > + bufremain = bufcnt; > + > + while (bufremain > 0) { > + whichbuf = (bufremain < maxiovec) ? bufremain : maxiovec; > + bufremain -= whichbuf; > + > + i = writev(fd, iov, whichbuf); > + if (i < 0) { > + err(1, "failed to write"); > + } > + } > + > + if (outfile) { > + i = close(fd); > + if (i < 0) { > + err(1, "failed to close"); > + } > + } > +} > > Added: head/usr.bin/sponge/tests/Makefile > ============================================================================== > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ head/usr.bin/sponge/tests/Makefile Tue Dec 5 03:55:10 2017 (r326554) > @@ -0,0 +1,9 @@ > +# $FreeBSD$ > + > +PACKAGE= ${TESTBASE}/usr.bin/sponge > + > +TESTSDIR= ${TESTSBASE}/usr.bin/sponge/tests > + > +ATF_TESTS_SH= cp_test > + > +.include > > Added: head/usr.bin/sponge/tests/Makefile.depend > ============================================================================== > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ head/usr.bin/sponge/tests/Makefile.depend Tue Dec 5 03:55:10 2017 (r326554) > @@ -0,0 +1,18 @@ > +# $FreeBSD$ > +# Autogenerated - do NOT edit! > + > +DIRDEPS = \ > + gnu/lib/csu \ > + include \ > + include/xlocale \ > + lib/${CSU_DIR} \ > + lib/atf/libatf-c \ > + lib/libc \ > + lib/libcompiler_rt \ > + > + > +.include > + > +.if ${DEP_RELDIR} == ${_DEP_RELDIR} > +# local dependencies - needed for -jN in clean tree > +.endif > > Added: head/usr.bin/sponge/tests/sponge_test.sh > ============================================================================== > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ head/usr.bin/sponge/tests/sponge_test.sh Tue Dec 5 03:55:10 2017 (r326554) > @@ -0,0 +1,48 @@ > +# $FreeBSD$ > +# > +# SPDX-License-Identifier: BSD-2-Clause > +# > +# Copyright 2017 Eitan Adler > +# All rights reserved. > +# > +# Redistribution and use in source and binary forms, with or without > +# modification, are permitted provided that the following conditions are > +# met: > +# > +# * Redistributions of source code must retain the above copyright > +# notice, this list of conditions and the following disclaimer. > +# * Redistributions in binary form must reproduce the above copyright > +# notice, this list of conditions and the following disclaimer in the > +# documentation and/or other materials provided with the distribution. > +# > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS > +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT > +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR > +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT > +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, > +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT > +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, > +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY > +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT > +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE > +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > + > +atf_test_case simple > +simple_body() { > + printf 'd\ne\na\n' >| x > + printf 'a\nd\ne\n' >| y > + sort x | sponge x > + > + atf_check -s exit:0 -o empty -e empty 'sort x | sponge x' > + > + if ! cmp -s x y; then > + echo "x and y differ, but they should be equal" > + diff -u x y > + atf_fail "Original and copy do not match" > + fi > + > +} > + > +atf_init_test_cases() { > + atf_add_test_case simple > +} > > Modified: head/usr.bin/tee/tee.1 > ============================================================================== > --- head/usr.bin/tee/tee.1 Tue Dec 5 02:23:36 2017 (r326553) > +++ head/usr.bin/tee/tee.1 Tue Dec 5 03:55:10 2017 (r326554) > @@ -72,6 +72,8 @@ utility takes the default action for all signals, > except in the event of the > .Fl i > option. > +.Sh SEE ALSO > +.Xr sponge 1 > .Sh EXIT STATUS > .Ex -std > .Sh STANDARDS >