From owner-svn-src-all@freebsd.org Tue Dec 5 08:43:52 2017 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C17CEDFCF58; Tue, 5 Dec 2017 08:43:52 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4E250712C7; Tue, 5 Dec 2017 08:43:52 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id vB58hjTj049702 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 5 Dec 2017 10:43:46 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua vB58hjTj049702 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id vB58hj9c049701; Tue, 5 Dec 2017 10:43:45 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 5 Dec 2017 10:43:45 +0200 From: Konstantin Belousov To: Eitan Adler Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r326554 - in head: . usr.bin/sponge usr.bin/sponge/tests usr.bin/tee Message-ID: <20171205084345.GR2272@kib.kiev.ua> References: <201712050355.vB53tApl074041@repo.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201712050355.vB53tApl074041@repo.freebsd.org> User-Agent: Mutt/1.9.1 (2017-09-22) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Dec 2017 08:43:52 -0000 On Tue, Dec 05, 2017 at 03:55:10AM +0000, Eitan Adler wrote: > Author: eadler > Date: Tue Dec 5 03:55:10 2017 > New Revision: 326554 > URL: https://svnweb.freebsd.org/changeset/base/326554 > > Log: > sponge(1): initial commit > > sponge(1) is a utility that reads input until > complete, then opens the output file, then > writes to it. This makes it useful in pipelines > that read and write to the same file. > > Reviewed by: wblock, jilles, imp, cem, danfe (all: various iterations) > Inspired by: https://joeyh.name/code/moreutils/ > > Added: > head/usr.bin/sponge/ > head/usr.bin/sponge/Makefile (contents, props changed) > head/usr.bin/sponge/sponge.1 (contents, props changed) > head/usr.bin/sponge/sponge.c (contents, props changed) > head/usr.bin/sponge/tests/ > head/usr.bin/sponge/tests/Makefile (contents, props changed) > head/usr.bin/sponge/tests/Makefile.depend (contents, props changed) > head/usr.bin/sponge/tests/sponge_test.sh (contents, props changed) > Modified: > head/.arclint > head/usr.bin/tee/tee.1 > > Modified: head/.arclint > ============================================================================== > --- head/.arclint Tue Dec 5 02:23:36 2017 (r326553) > +++ head/.arclint Tue Dec 5 03:55:10 2017 (r326554) > @@ -9,7 +9,8 @@ > "type": "spelling" > }, > "chmod": { > - "type": "chmod" > + "type": "chmod", > + "exclude": "(/tests/)" > }, > "merge-conflict": { > "type": "merge-conflict" > > Added: head/usr.bin/sponge/Makefile > ============================================================================== > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ head/usr.bin/sponge/Makefile Tue Dec 5 03:55:10 2017 (r326554) > @@ -0,0 +1,8 @@ > +# $FreeBSD$ > + > +PROG= sponge > + > +HAS_TESTS= > +SUBDIR.${MK_TESTS}+= tests > + > +.include > > Added: head/usr.bin/sponge/sponge.1 > ============================================================================== > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ head/usr.bin/sponge/sponge.1 Tue Dec 5 03:55:10 2017 (r326554) > @@ -0,0 +1,75 @@ > +.\" Eitan Adler. All rights reserved. > +.\" > +.\" Redistribution and use in source and binary forms, with or without > +.\" modification, are permitted provided that the following conditions > +.\" are met: > +.\" 1. Redistributions of source code must retain the above copyright > +.\" notice, this list of conditions and the following disclaimer. > +.\" 2. Redistributions in binary form must reproduce the above copyright > +.\" notice, this list of conditions and the following disclaimer in the > +.\" documentation and/or other materials provided with the distribution. > +.\" > +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND > +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE > +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE > +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE > +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL > +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS > +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT > +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY > +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF > +.\" SUCH DAMAGE. > +.\" > +.\" $FreeBSD$ > +.\" > +.Dd November 1, 2017 > +.Dt SPONGE 1 > +.Os > +.Sh NAME > +.Nm sponge > +.Nd buffer stdin and write to stdout > +.Sh SYNOPSIS > +.Nm > +.Op Fl a > +.Ar filename > +.Sh DESCRIPTION > +The > +.Nm > +utility reads standard in until complete, then opens > +the output file and writes to it. > +This makes it useful in pipelines that read a file and then write to it. > +These options are available: > +.Bl -tag -width indent > +.It Fl a > +Open > +.Ar filename > +in append mode. > +.El > +.Pp > +If an attempt to allocate memory fails, > +.Nm > +fails without output. > +The file is written even if earlier components > +of the pipeline failed. > +.Sh SEE ALSO > +.Xr builtin 1 , > +.Xr csh 1 , > +.Xr getrusage 2 , > +.Xr tee 1 , > +.Xr wait 2 > +.Sh EXIT STATUS > +.Ex -std > +.Sh EXAMPLES > +A > +.Pa file > +can be be sorted "in place" by executing > +.Cm sort file | sponge file > +.Sh HISTORY > +The > +.Nm > +utility was written by > +.An Eitan Adler Aq Mt eadler@FreeBSD.org > +and first appeared > +in > +.Fx 12.0 . > > Added: head/usr.bin/sponge/sponge.c > ============================================================================== > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ head/usr.bin/sponge/sponge.c Tue Dec 5 03:55:10 2017 (r326554) > @@ -0,0 +1,189 @@ > +/*- > + * SPDX-License-Identifier: BSD-2-Clause > + * > + * Copyright (c) 2017 Eitan Adler > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * 1. Redistributions of source code must retain the above copyright > + * notice, this list of conditions and the following disclaimer. > + * 2. Redistributions in binary form must reproduce the above copyright > + * notice, this list of conditions and the following disclaimer in the > + * documentation and/or other materials provided with the distribution. > + * > + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND > + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE > + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE > + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE > + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL > + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS > + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT > + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY > + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF > + * SUCH DAMAGE. > + * > + * $FreeBSD$ > + */ > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#define DEFAULT_BUF_SIZE 16384 > +#define DEFAULT_BUF_CNT 12 > + > +static int flag_append = 0; > + > +static void usage(void); > +static void *safe_malloc(size_t size); > +static void *safe_calloc(size_t count, size_t size); > +static void *safe_reallocf(void *ptr, size_t size); > + > +static void * > +safe_malloc(size_t size) > +{ > + void *ret; > + > + ret = malloc(size); > + if (ret == NULL) { > + err(1, "malloc failed"); > + } > + return (ret); > +} > + > +static void * > +safe_calloc(size_t count, size_t size) > +{ > + void *ret; > + > + ret = calloc(count, size); > + if (ret == NULL) { > + err(1, "calloc failed"); > + } > + return (ret); > +} > + > +static void * > +safe_reallocf(void *ptr, size_t size) > +{ > + void *ret; > + > + ret = reallocf(ptr, size); > + if (ret == NULL) { > + err(1, "reallocf failed"); > + } > + return (ret); > +} > + > +static void > +usage(void) > +{ > + fprintf(stderr, "usage: sponge [-a] filename\n"); > +} > + > +int > +main(int argc, char* argv[]) > +{ > + struct iovec *iov; > + char *buf; > + char *outfile; > + ssize_t i; > + size_t bufcnt; > + size_t whichbuf; > + size_t bufremain; > + long maxiovec; > + int fd; > + int openflags = O_WRONLY; > + int opt; > + > + while ((opt = getopt(argc, argv, "ah")) != -1) { > + switch (opt) { > + case 'a': > + flag_append = 1; > + break; > + case 'h': > + usage(); > + exit(0); > + case '?': > + default: > + usage(); > + exit(1); > + } > + } > + > + if (optind < argc) { > + outfile = argv[optind]; > + } > + > + > + bufcnt = DEFAULT_BUF_CNT; > + whichbuf = 0; > + iov = safe_calloc(bufcnt, sizeof(*iov)); > + > + for (;;) { > + buf = safe_malloc(DEFAULT_BUF_SIZE); > + i = read(STDIN_FILENO, buf, DEFAULT_BUF_SIZE); > + if (whichbuf == bufcnt) { > + bufcnt *= 2; > + iov = safe_reallocf(iov, bufcnt * sizeof(*iov)); > + } > + if (i < 0) { > + err(1, "read failed"); > + } > + if (i == 0) { > + free(buf); > + break; > + } > + iov[whichbuf].iov_base = buf; > + iov[whichbuf].iov_len = i; > + whichbuf++; > + } > + > + if (outfile) { > + if (flag_append) { > + openflags |= O_APPEND; > + } else { > + openflags |= O_TRUNC; > + } > + fd = open(outfile, openflags); > + } > + else { > + fd = STDOUT_FILENO; > + } > + > + if (fd < 0) { > + err(1, "failed to open"); > + } > + > + maxiovec = sysconf(_SC_IOV_MAX); > + if (maxiovec == -1) { > + maxiovec = _XOPEN_IOV_MAX; > + } > + bufcnt = whichbuf; > + bufremain = bufcnt; > + > + while (bufremain > 0) { > + whichbuf = (bufremain < maxiovec) ? bufremain : maxiovec; > + bufremain -= whichbuf; > + > + i = writev(fd, iov, whichbuf); > + if (i < 0) { > + err(1, "failed to write"); > + } > + } This loop is nonsensical.