Date: Thu, 16 Feb 2012 19:09:26 -0500 From: Matthew Story <matthewstory@gmail.com> To: freebsd-arch@freebsd.org Subject: Change to xargs to avoid orphaning of utility processes on signal|exit 255 from child Message-ID: <CAB%2B9ogetrht1Ttf53vSbq7L5Fe9DbB5igSynKM3=tjZh0z0_ew@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Apologies if this is the wrong list, I would like to submit a patch that
changes the behavior of xargs(1) on signal to child utility process or
child utility process exiting 255. The patch(es) is|are available here:
http://axe0.blackskyresearch.net/patches/matt/xargs.no_orphan.patch.txt--
this version will apply to current xargs, and adds diagnostic
information for exit 255|signal to utility, as required by POSIX (see
PR165155).
http://axe0.blackskyresearch.net/patches/matt/xargs.no_orphan.PR165155.patch.txt--
this version will apply on top of the patch in PR165155, as the errx
calls in that patch need to be modified to warnx calls.
I will happily provide a third option without diagnostics if that is useful
independent of PR165155. Currently, if a child exits 255, or is terminated
via signal, the xargs process exits immediately with exit status 1, with
maxprocs > 1, this orphans all outstanding child processes:
$ jot - 1 10 | time xargs -P2 -n1 sh -c 'echo "$1: $$"; sleep $1; [ $1 -eq
1 ] && kill $$; echo "$1: $$:done";' worker
1: 69752
2: 69753
1.00 real 0.00 user 0.00 sys
$ 2: 69753:done
$ # or ...
$ jot - 1 10 | time xargs -P2 -n1 sh -c 'echo "$1: $$"; sleep $1; [ $1 -eq
1 ] && exit 255; echo "$1: $$:done";' worker
1: 70379
2: 70380
1.00 real 0.00 user 0.00 sys
$ 2: 70380:done
This behavior is expected per xargs(1):
The xargs utility exits immediately (without processing any further
input) if a command line cannot be assembled, utility cannot be
invoked,
an invocation of utility is terminated by a signal, or an invocation of
utility exits with a value of 255.
Per the POSIX specification, xargs is only required to stop processing
input, and to exit 1-125, it is not exit immediately (utility here is the
utility invoked by xargs, not xargs itself):
If a command line meeting the specified requirements cannot be assembled,
the utility cannot be invoked, an invocation of the utility is terminated
by a signal, or an invocation of the utility exits with exit status 255,
the *xargs* utility shall write a diagnostic message and exit without
processing any remaining input.
The patch preserves orphaning when the xargs process itself is terminated
by signal, but augments the behavior when a child utility process is
terminated by signal or exits 255 to wait for other existing child
utilities until exiting 1. My reasoning for this (beyond orphaning
nastiness) is that I always want to fail as soon as I know an operation is
fatal, and then clean-up. By orphaning children, there is no reliable way
to clean-up following use of the 255 exit code (or signal termination):
$ # a contrived example forcing a race-condition, with a clean-up function.
$ mkdir -p foo; jot - 1 10 | xargs -P5 -n1 sh -c 'sleep $1; touch foo/$1;
exit 255;' worker || find foo -type f -delete
$ # demonstration that cleanup is not possible
$ sleep 5 && ls -l foo
total 2
-rw-r--r-- 1 matt matt 0 Feb 16 19:01 2
-rw-r--r-- 1 matt matt 0 Feb 16 19:01 3
-rw-r--r-- 1 matt matt 0 Feb 16 19:01 4
-rw-r--r-- 1 matt matt 0 Feb 16 19:01 5
Following the patch, we get a nice and reliable cleanup, as we have no
orphans:
$ mkdir -p foo; jot - 1 10 | usr.bin/xargs/xargs -P5 -n1 sh -c 'sleep $1;
touch foo/$1; exit 255;' worker || find foo -type f -delete
xargs: sh: exited with status 255, aborting
xargs: sh: exited with status 255, aborting
xargs: sh: exited with status 255, aborting
xargs: sh: exited with status 255, aborting
xargs: sh: exited with status 255, aborting
$ ls -l foo/
total 0
Please let me know what you think, I would very much like to see this patch
make it's way into xargs as I find this short-circuit behavior nearly very
usable, and it's only failing is that it orphans children unnecessarily,
resulting in unpredictable behavior.
--
regards,
matt
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAB%2B9ogetrht1Ttf53vSbq7L5Fe9DbB5igSynKM3=tjZh0z0_ew>