From owner-svn-src-head@freebsd.org Tue Dec 5 15:19:20 2017 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 836D3E6D412; Tue, 5 Dec 2017 15:19:20 +0000 (UTC) (envelope-from cy.schubert@komquats.com) Received: from smtp-out-no.shaw.ca (smtp-out-no.shaw.ca [64.59.134.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id BCACD7E84B; Tue, 5 Dec 2017 15:19:19 +0000 (UTC) (envelope-from cy.schubert@komquats.com) Received: from spqr.komquats.com ([96.50.22.10]) by shaw.ca with SMTP id MElUeHSBBp2osMElVetLTF; Tue, 05 Dec 2017 08:04:13 -0700 X-Authority-Analysis: v=2.2 cv=KLEqNBNo c=1 sm=1 tr=0 a=jvE2nwUzI0ECrNeyr98KWA==:117 a=jvE2nwUzI0ECrNeyr98KWA==:17 a=kj9zAlcOel0A:10 a=ocR9PWop10UA:10 a=6I5d2MoRAAAA:8 a=vJ3PJytkAAAA:8 a=YxBL1-UpAAAA:8 a=Ug2Wxoev8HxC__jtvqgA:9 a=IW8E0xEPpk4UX4_Z:21 a=CjuIK1q_8ugA:10 a=-FEs8UIgK8oA:10 a=NWVoK91CQyQA:10 a=IjZwj45LgO3ly-622nXo:22 a=l9wNFzH4Rd0V1LYgFcnv:22 a=Ia-lj3WSrqcvXOmTRaiG:22 Received: from slippy.cwsent.com (slippy [10.1.1.91]) by spqr.komquats.com (Postfix) with ESMTPS id E7F45A1; Tue, 5 Dec 2017 07:04:07 -0800 (PST) Received: from slippy.cwsent.com (localhost [127.0.0.1]) by slippy.cwsent.com (8.15.2/8.15.2) with ESMTP id vB5F46VC002900; Tue, 5 Dec 2017 07:04:06 -0800 (PST) (envelope-from Cy.Schubert@cschubert.com) Received: from slippy (cy@localhost) by slippy.cwsent.com (8.15.2/8.15.2/Submit) with ESMTP id vB5F458w002872; Tue, 5 Dec 2017 07:04:06 -0800 (PST) (envelope-from Cy.Schubert@cschubert.com) Message-Id: <201712051504.vB5F458w002872@slippy.cwsent.com> X-Authentication-Warning: slippy.cwsent.com: cy owned process doing -bs X-Mailer: exmh version 2.8.0 04/21/2012 with nmh-1.7 Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: Eitan Adler cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r326554 - in head: . usr.bin/sponge usr.bin/sponge/tests usr.bin/tee In-Reply-To: Message from Eitan Adler of "Tue, 05 Dec 2017 03:55:10 +0000." <201712050355.vB53tApl074041@repo.freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 05 Dec 2017 07:04:04 -0800 X-CMAE-Envelope: MS4wfF/esH6kYj6nRvMlLWv6iEXn+VYwJ8tPwWUfePiuwJAWoDPBPM+yknWKMnr7V/PGfo5EHZMPEJp0wzL7A25M8Rq0VtvNcMztrOJhfz58Y9QMCvT9H1Eu m3tTzPcKvU0l6yL3ur8f60lpromXEuV25Ho6e++Y3WYDUANjw56sfjLBskSOOOVnXiEPDXhSOc0cNEZnww0XWhKQyBAD6hMzWAIvFb7yoS/jvnpXO+y3kgzF d9/cGsvIU5e2YpfCY+R+gf6llNLYxCaUCuT4n80D0+IeGgOxL+y5nCcQrtf0tqE2 X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Dec 2017 15:19:20 -0000 In message <201712050355.vB53tApl074041@repo.freebsd.org>, Eitan Adler writes: > Author: eadler > Date: Tue Dec 5 03:55:10 2017 > New Revision: 326554 > URL: https://svnweb.freebsd.org/changeset/base/326554 > > Log: > sponge(1): initial commit > > sponge(1) is a utility that reads input until > complete, then opens the output file, then > writes to it. This makes it useful in pipelines > that read and write to the same file. > > Reviewed by: wblock, jilles, imp, cem, danfe (all: various iteration > s) > Inspired by: https://joeyh.name/code/moreutils/ > > Added: > head/usr.bin/sponge/ > head/usr.bin/sponge/Makefile (contents, props changed) > head/usr.bin/sponge/sponge.1 (contents, props changed) > head/usr.bin/sponge/sponge.c (contents, props changed) > head/usr.bin/sponge/tests/ > head/usr.bin/sponge/tests/Makefile (contents, props changed) > head/usr.bin/sponge/tests/Makefile.depend (contents, props changed) > head/usr.bin/sponge/tests/sponge_test.sh (contents, props changed) > Modified: > head/.arclint > head/usr.bin/tee/tee.1 > > Modified: head/.arclint > ============================================================================= > = > --- head/.arclint Tue Dec 5 02:23:36 2017 (r326553) > +++ head/.arclint Tue Dec 5 03:55:10 2017 (r326554) > @@ -9,7 +9,8 @@ > "type": "spelling" > }, > "chmod": { > - "type": "chmod" > + "type": "chmod", > + "exclude": "(/tests/)" > }, > "merge-conflict": { > "type": "merge-conflict" > > Added: head/usr.bin/sponge/Makefile > ============================================================================= > = > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ head/usr.bin/sponge/Makefile Tue Dec 5 03:55:10 2017 (r32655 > 4) > @@ -0,0 +1,8 @@ > +# $FreeBSD$ > + > +PROG= sponge > + > +HAS_TESTS= > +SUBDIR.${MK_TESTS}+= tests > + > +.include > > Added: head/usr.bin/sponge/sponge.1 > ============================================================================= > = > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ head/usr.bin/sponge/sponge.1 Tue Dec 5 03:55:10 2017 (r32655 > 4) > @@ -0,0 +1,75 @@ > +.\" Eitan Adler. All rights reserved. > +.\" > +.\" Redistribution and use in source and binary forms, with or without > +.\" modification, are permitted provided that the following conditions > +.\" are met: > +.\" 1. Redistributions of source code must retain the above copyright > +.\" notice, this list of conditions and the following disclaimer. > +.\" 2. Redistributions in binary form must reproduce the above copyright > +.\" notice, this list of conditions and the following disclaimer in the > +.\" documentation and/or other materials provided with the distribution. > +.\" > +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND > +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE > +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPO > SE > +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE > +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTI > AL > +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS > +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRI > CT > +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WA > Y > +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF > +.\" SUCH DAMAGE. > +.\" > +.\" $FreeBSD$ > +.\" > +.Dd November 1, 2017 > +.Dt SPONGE 1 > +.Os > +.Sh NAME > +.Nm sponge > +.Nd buffer stdin and write to stdout > +.Sh SYNOPSIS > +.Nm > +.Op Fl a > +.Ar filename > +.Sh DESCRIPTION > +The > +.Nm > +utility reads standard in until complete, then opens > +the output file and writes to it. > +This makes it useful in pipelines that read a file and then write to it. > +These options are available: > +.Bl -tag -width indent > +.It Fl a > +Open > +.Ar filename > +in append mode. > +.El > +.Pp > +If an attempt to allocate memory fails, > +.Nm > +fails without output. > +The file is written even if earlier components > +of the pipeline failed. > +.Sh SEE ALSO > +.Xr builtin 1 , > +.Xr csh 1 , > +.Xr getrusage 2 , > +.Xr tee 1 , > +.Xr wait 2 > +.Sh EXIT STATUS > +.Ex -std > +.Sh EXAMPLES > +A > +.Pa file > +can be be sorted "in place" by executing > +.Cm sort file | sponge file > +.Sh HISTORY > +The > +.Nm > +utility was written by > +.An Eitan Adler Aq Mt eadler@FreeBSD.org > +and first appeared > +in > +.Fx 12.0 . > > Added: head/usr.bin/sponge/sponge.c > ============================================================================= > = > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ head/usr.bin/sponge/sponge.c Tue Dec 5 03:55:10 2017 (r32655 > 4) > @@ -0,0 +1,189 @@ > +/*- > + * SPDX-License-Identifier: BSD-2-Clause > + * > + * Copyright (c) 2017 Eitan Adler > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * 1. Redistributions of source code must retain the above copyright > + * notice, this list of conditions and the following disclaimer. > + * 2. Redistributions in binary form must reproduce the above copyright > + * notice, this list of conditions and the following disclaimer in the > + * documentation and/or other materials provided with the distribution. > + * > + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND > + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE > + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOS > E > + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE > + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIA > L > + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS > + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRIC > T > + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY > + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF > + * SUCH DAMAGE. > + * > + * $FreeBSD$ > + */ > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#define DEFAULT_BUF_SIZE 16384 > +#define DEFAULT_BUF_CNT 12 > + > +static int flag_append = 0; > + > +static void usage(void); > +static void *safe_malloc(size_t size); > +static void *safe_calloc(size_t count, size_t size); > +static void *safe_reallocf(void *ptr, size_t size); > + > +static void * > +safe_malloc(size_t size) > +{ > + void *ret; > + > + ret = malloc(size); > + if (ret == NULL) { > + err(1, "malloc failed"); > + } > + return (ret); > +} > + > +static void * > +safe_calloc(size_t count, size_t size) > +{ > + void *ret; > + > + ret = calloc(count, size); > + if (ret == NULL) { > + err(1, "calloc failed"); > + } > + return (ret); > +} > + > +static void * > +safe_reallocf(void *ptr, size_t size) > +{ > + void *ret; > + > + ret = reallocf(ptr, size); > + if (ret == NULL) { > + err(1, "reallocf failed"); > + } > + return (ret); > +} > + > +static void > +usage(void) > +{ > + fprintf(stderr, "usage: sponge [-a] filename\n"); > +} > + > +int > +main(int argc, char* argv[]) > +{ > + struct iovec *iov; > + char *buf; > + char *outfile; > + ssize_t i; > + size_t bufcnt; > + size_t whichbuf; > + size_t bufremain; > + long maxiovec; > + int fd; > + int openflags = O_WRONLY; > + int opt; > + > + while ((opt = getopt(argc, argv, "ah")) != -1) { > + switch (opt) { > + case 'a': > + flag_append = 1; > + break; > + case 'h': > + usage(); > + exit(0); > + case '?': > + default: > + usage(); > + exit(1); > + } > + } > + > + if (optind < argc) { > + outfile = argv[optind]; > + } > + > + > + bufcnt = DEFAULT_BUF_CNT; > + whichbuf = 0; > + iov = safe_calloc(bufcnt, sizeof(*iov)); > + > + for (;;) { > + buf = safe_malloc(DEFAULT_BUF_SIZE); > + i = read(STDIN_FILENO, buf, DEFAULT_BUF_SIZE); > + if (whichbuf == bufcnt) { > + bufcnt *= 2; > + iov = safe_reallocf(iov, bufcnt * sizeof(*iov)); > + } > + if (i < 0) { > + err(1, "read failed"); > + } > + if (i == 0) { > + free(buf); > + break; > + } > + iov[whichbuf].iov_base = buf; > + iov[whichbuf].iov_len = i; > + whichbuf++; > + } > + > + if (outfile) { > + if (flag_append) { > + openflags |= O_APPEND; > + } else { > + openflags |= O_TRUNC; > + } > + fd = open(outfile, openflags); > + } > + else { > + fd = STDOUT_FILENO; > + } > + > + if (fd < 0) { > + err(1, "failed to open"); > + } > + > + maxiovec = sysconf(_SC_IOV_MAX); > + if (maxiovec == -1) { > + maxiovec = _XOPEN_IOV_MAX; > + } > + bufcnt = whichbuf; > + bufremain = bufcnt; > + > + while (bufremain > 0) { > + whichbuf = (bufremain < maxiovec) ? bufremain : maxiovec; Buildworld stops here. Should this not be (unsigned long) maxiovec? > + bufremain -= whichbuf; > + > [...] -- Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org The need of the many outweighs the greed of the few.