From owner-svn-src-stable@freebsd.org Tue Mar 26 22:34:10 2019 Return-Path: Delivered-To: svn-src-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B8F6C15620AB; Tue, 26 Mar 2019 22:34:10 +0000 (UTC) (envelope-from jilles@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 584E289FA3; Tue, 26 Mar 2019 22:34:10 +0000 (UTC) (envelope-from jilles@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 30AC526322; Tue, 26 Mar 2019 22:34:10 +0000 (UTC) (envelope-from jilles@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x2QMY9BQ016351; Tue, 26 Mar 2019 22:34:09 GMT (envelope-from jilles@FreeBSD.org) Received: (from jilles@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x2QMY7iG016340; Tue, 26 Mar 2019 22:34:07 GMT (envelope-from jilles@FreeBSD.org) Message-Id: <201903262234.x2QMY7iG016340@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jilles set sender to jilles@FreeBSD.org using -f From: Jilles Tjoelker Date: Tue, 26 Mar 2019 22:34:07 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r345561 - in stable/11/bin/sh: . tests/execution X-SVN-Group: stable-11 X-SVN-Commit-Author: jilles X-SVN-Commit-Paths: in stable/11/bin/sh: . tests/execution X-SVN-Commit-Revision: 345561 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 584E289FA3 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-0.99)[-0.989,0]; NEURAL_HAM_SHORT(-0.98)[-0.977,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for all the -stable branches of the src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Mar 2019 22:34:11 -0000 Author: jilles Date: Tue Mar 26 22:34:07 2019 New Revision: 345561 URL: https://svnweb.freebsd.org/changeset/base/345561 Log: MFC r344502: sh: Add set -o pipefail The pipefail option allows checking the exit status of all commands in a pipeline more easily, at a limited cost of complexity in sh itself. It works similarly to the option in bash, ksh93 and mksh. Like ksh93 and unlike bash and mksh, the state of the option is saved when a pipeline is started. Therefore, even in the case of commands like A | B & a later change of the option does not change the exit status, the same way (A | B) & works. Since SIGPIPE is not handled specially, more work in the script is required for a proper exit status for pipelines containing commands such as head that may terminate successfully without reading all input. This can be something like ( cmd1 r=$? if [ "$r" -gt 128 ] && [ "$(kill -l "$r")" = PIPE ]; then exit 0 else exit "$r" fi ) | head PR: 224270 Relnotes: yes Added: stable/11/bin/sh/tests/execution/pipefail1.0 - copied unchanged from r344502, head/bin/sh/tests/execution/pipefail1.0 stable/11/bin/sh/tests/execution/pipefail2.42 - copied unchanged from r344502, head/bin/sh/tests/execution/pipefail2.42 stable/11/bin/sh/tests/execution/pipefail3.42 - copied unchanged from r344502, head/bin/sh/tests/execution/pipefail3.42 stable/11/bin/sh/tests/execution/pipefail4.42 - copied unchanged from r344502, head/bin/sh/tests/execution/pipefail4.42 stable/11/bin/sh/tests/execution/pipefail5.42 - copied unchanged from r344502, head/bin/sh/tests/execution/pipefail5.42 stable/11/bin/sh/tests/execution/pipefail6.42 - copied unchanged from r344502, head/bin/sh/tests/execution/pipefail6.42 stable/11/bin/sh/tests/execution/pipefail7.0 - copied unchanged from r344502, head/bin/sh/tests/execution/pipefail7.0 Modified: stable/11/bin/sh/jobs.c stable/11/bin/sh/options.h stable/11/bin/sh/sh.1 stable/11/bin/sh/tests/execution/Makefile Directory Properties: stable/11/ (props changed) Modified: stable/11/bin/sh/jobs.c ============================================================================== --- stable/11/bin/sh/jobs.c Tue Mar 26 22:14:50 2019 (r345560) +++ stable/11/bin/sh/jobs.c Tue Mar 26 22:34:07 2019 (r345561) @@ -104,6 +104,7 @@ struct job { char changed; /* true if status has changed */ char foreground; /* true if running in the foreground */ char remembered; /* true if $! referenced */ + char pipefail; /* pass any non-zero status */ #if JOBS char jobctl; /* job running under job control */ struct job *next; /* job used after this one */ @@ -143,6 +144,7 @@ static void setcurjob(struct job *); static void deljob(struct job *); static struct job *getcurjob(struct job *); #endif +static int getjobstatus(const struct job *); static void printjobcmd(struct job *); static void showjob(struct job *, int); @@ -340,6 +342,20 @@ jobscmd(int argc __unused, char *argv[] __unused) return (0); } +static int getjobstatus(const struct job *jp) +{ + int i, status; + + if (!jp->pipefail) + return (jp->ps[jp->nprocs - 1].status); + for (i = jp->nprocs - 1; i >= 0; i--) { + status = jp->ps[i].status; + if (status != 0) + return (status); + } + return (0); +} + static void printjobcmd(struct job *jp) { @@ -376,7 +392,7 @@ showjob(struct job *jp, int mode) } #endif coredump = ""; - status = jp->ps[jp->nprocs - 1].status; + status = getjobstatus(jp); if (jp->state == 0) { statestr = "Running"; #if JOBS @@ -555,7 +571,7 @@ waitcmdloop(struct job *job) do { if (job != NULL) { if (job->state == JOBDONE) { - status = job->ps[job->nprocs - 1].status; + status = getjobstatus(job); if (WIFEXITED(status)) retval = WEXITSTATUS(status); else @@ -780,6 +796,7 @@ makejob(union node *node __unused, int nprocs) jp->nprocs = 0; jp->foreground = 0; jp->remembered = 0; + jp->pipefail = pipefailflag; #if JOBS jp->jobctl = jobctl; jp->next = NULL; @@ -1075,7 +1092,7 @@ waitforjob(struct job *jp, int *origstatus) if (jp->state == JOBSTOPPED) setcurjob(jp); #endif - status = jp->ps[jp->nprocs - 1].status; + status = getjobstatus(jp); if (origstatus != NULL) *origstatus = status; /* convert to 8 bits */ Modified: stable/11/bin/sh/options.h ============================================================================== --- stable/11/bin/sh/options.h Tue Mar 26 22:14:50 2019 (r345560) +++ stable/11/bin/sh/options.h Tue Mar 26 22:34:07 2019 (r345561) @@ -65,9 +65,10 @@ struct shparam { #define Pflag optval[17] #define hflag optval[18] #define nologflag optval[19] +#define pipefailflag optval[20] #define NSHORTOPTS 19 -#define NOPTS 20 +#define NOPTS 21 extern char optval[NOPTS]; extern const char optletter[NSHORTOPTS]; @@ -95,6 +96,7 @@ static const unsigned char optname[] = "\010physical" "\010trackall" "\005nolog" + "\010pipefail" ; #endif Modified: stable/11/bin/sh/sh.1 ============================================================================== --- stable/11/bin/sh/sh.1 Tue Mar 26 22:14:50 2019 (r345560) +++ stable/11/bin/sh/sh.1 Tue Mar 26 22:34:07 2019 (r345561) @@ -32,7 +32,7 @@ .\" from: @(#)sh.1 8.6 (Berkeley) 5/4/95 .\" $FreeBSD$ .\" -.Dd May 30, 2016 +.Dd February 24, 2019 .Dt SH 1 .Os .Sh NAME @@ -348,6 +348,18 @@ Another do-nothing option for .Tn POSIX compliance. It only has a long name. +.It Li pipefail +Change the exit status of a pipeline to the last non-zero exit status of +any command in the pipeline, if any. +Since an exit due to +.Dv SIGPIPE +counts as a non-zero exit status, +this option may cause non-zero exit status for successful pipelines +if a command such as +.Xr head 1 +in the pipeline terminates with status 0 without reading its +input completely. +This option only has a long name. .El .Pp The @@ -865,12 +877,15 @@ If the keyword .Ic !\& does not precede the pipeline, the exit status is the exit status of the last command specified -in the pipeline. +in the pipeline if the +.Cm pipefail +option is not set or all commands returned zero, +or the last non-zero exit status of any command in the pipeline otherwise. Otherwise, the exit status is the logical -NOT of the exit status of the last command. +NOT of that exit status. That is, if -the last command returns zero, the exit status is 1; if -the last command returns greater than zero, the exit status +that status is zero, the exit status is 1; if +that status is greater than zero, the exit status is zero. .Pp Because pipeline assignment of standard input or standard Modified: stable/11/bin/sh/tests/execution/Makefile ============================================================================== --- stable/11/bin/sh/tests/execution/Makefile Tue Mar 26 22:14:50 2019 (r345560) +++ stable/11/bin/sh/tests/execution/Makefile Tue Mar 26 22:34:07 2019 (r345561) @@ -30,6 +30,13 @@ ${PACKAGE}FILES+= killed2.0 ${PACKAGE}FILES+= not1.0 ${PACKAGE}FILES+= not2.0 ${PACKAGE}FILES+= path1.0 +${PACKAGE}FILES+= pipefail1.0 +${PACKAGE}FILES+= pipefail2.42 +${PACKAGE}FILES+= pipefail3.42 +${PACKAGE}FILES+= pipefail4.42 +${PACKAGE}FILES+= pipefail5.42 +${PACKAGE}FILES+= pipefail6.42 +${PACKAGE}FILES+= pipefail7.0 ${PACKAGE}FILES+= redir1.0 ${PACKAGE}FILES+= redir2.0 ${PACKAGE}FILES+= redir3.0 Copied: stable/11/bin/sh/tests/execution/pipefail1.0 (from r344502, head/bin/sh/tests/execution/pipefail1.0) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ stable/11/bin/sh/tests/execution/pipefail1.0 Tue Mar 26 22:34:07 2019 (r345561, copy of r344502, head/bin/sh/tests/execution/pipefail1.0) @@ -0,0 +1,4 @@ +# $FreeBSD$ + +set -o pipefail +: && : | : && : | : | : && : | : | : | : Copied: stable/11/bin/sh/tests/execution/pipefail2.42 (from r344502, head/bin/sh/tests/execution/pipefail2.42) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ stable/11/bin/sh/tests/execution/pipefail2.42 Tue Mar 26 22:34:07 2019 (r345561, copy of r344502, head/bin/sh/tests/execution/pipefail2.42) @@ -0,0 +1,4 @@ +# $FreeBSD$ + +set -o pipefail +(exit 42) | : Copied: stable/11/bin/sh/tests/execution/pipefail3.42 (from r344502, head/bin/sh/tests/execution/pipefail3.42) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ stable/11/bin/sh/tests/execution/pipefail3.42 Tue Mar 26 22:34:07 2019 (r345561, copy of r344502, head/bin/sh/tests/execution/pipefail3.42) @@ -0,0 +1,4 @@ +# $FreeBSD$ + +set -o pipefail +: | (exit 42) Copied: stable/11/bin/sh/tests/execution/pipefail4.42 (from r344502, head/bin/sh/tests/execution/pipefail4.42) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ stable/11/bin/sh/tests/execution/pipefail4.42 Tue Mar 26 22:34:07 2019 (r345561, copy of r344502, head/bin/sh/tests/execution/pipefail4.42) @@ -0,0 +1,4 @@ +# $FreeBSD$ + +set -o pipefail +(exit 43) | (exit 42) Copied: stable/11/bin/sh/tests/execution/pipefail5.42 (from r344502, head/bin/sh/tests/execution/pipefail5.42) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ stable/11/bin/sh/tests/execution/pipefail5.42 Tue Mar 26 22:34:07 2019 (r345561, copy of r344502, head/bin/sh/tests/execution/pipefail5.42) @@ -0,0 +1,5 @@ +# $FreeBSD$ + +set -o pipefail +(exit 42) | : & +wait %+ Copied: stable/11/bin/sh/tests/execution/pipefail6.42 (from r344502, head/bin/sh/tests/execution/pipefail6.42) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ stable/11/bin/sh/tests/execution/pipefail6.42 Tue Mar 26 22:34:07 2019 (r345561, copy of r344502, head/bin/sh/tests/execution/pipefail6.42) @@ -0,0 +1,6 @@ +# $FreeBSD$ + +set -o pipefail +(exit 42) | : & +set +o pipefail +wait %+ Copied: stable/11/bin/sh/tests/execution/pipefail7.0 (from r344502, head/bin/sh/tests/execution/pipefail7.0) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ stable/11/bin/sh/tests/execution/pipefail7.0 Tue Mar 26 22:34:07 2019 (r345561, copy of r344502, head/bin/sh/tests/execution/pipefail7.0) @@ -0,0 +1,5 @@ +# $FreeBSD$ + +(exit 42) | : & +set -o pipefail +wait %+