From owner-freebsd-arch@FreeBSD.ORG Tue Feb 21 17:50:21 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E5F551065670 for ; Tue, 21 Feb 2012 17:50:20 +0000 (UTC) (envelope-from matthewstory@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 911C68FC16 for ; Tue, 21 Feb 2012 17:50:20 +0000 (UTC) Received: by vbbfa15 with SMTP id fa15so6320601vbb.13 for ; Tue, 21 Feb 2012 09:50:19 -0800 (PST) Received-SPF: pass (google.com: domain of matthewstory@gmail.com designates 10.52.76.196 as permitted sender) client-ip=10.52.76.196; Authentication-Results: mr.google.com; spf=pass (google.com: domain of matthewstory@gmail.com designates 10.52.76.196 as permitted sender) smtp.mail=matthewstory@gmail.com; dkim=pass header.i=matthewstory@gmail.com Received: from mr.google.com ([10.52.76.196]) by 10.52.76.196 with SMTP id m4mr12448935vdw.112.1329846619647 (num_hops = 1); Tue, 21 Feb 2012 09:50:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=bAgCxWAU6mFpzn2icorH+LLd96Px6DGjdD1vAZBWWtE=; b=yHwhEX0WSEVozNdHohZtjzBa2WaMbJj1SPab8AwuomqUGPDLoZIEtzcSvjhXLRnIke n7wgnIfVp1rfotXZlI11Dz/8pDdlk9sX3O7HV8mPBIsxiPMH5rt2HPr1VuC6QAg1imim 0Y8AgV44RCNnSPgNvWXNyZNtw2RfWCv1rAqH8= MIME-Version: 1.0 Received: by 10.52.76.196 with SMTP id m4mr10106767vdw.112.1329846619584; Tue, 21 Feb 2012 09:50:19 -0800 (PST) Received: by 10.52.21.84 with HTTP; Tue, 21 Feb 2012 09:50:19 -0800 (PST) In-Reply-To: References: Date: Tue, 21 Feb 2012 12:50:19 -0500 Message-ID: From: Matthew Story To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: Change to xargs to avoid orphaning of utility processes on signal|exit 255 from child X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Feb 2012 17:50:21 -0000 On Thu, Feb 16, 2012 at 7:09 PM, Matthew Story wrote: > Apologies if this is the wrong list, I would like to submit a patch that > changes the behavior of xargs(1) on signal to child utility process or > child utility process exiting 255. The patch(es) is|are available here: > > > http://axe0.blackskyresearch.net/patches/matt/xargs.no_orphan.patch.txt-- this version will apply to current xargs, and adds diagnostic > information for exit 255|signal to utility, as required by POSIX (see > PR165155). > > http://axe0.blackskyresearch.net/patches/matt/xargs.no_orphan.PR165155.patch.txt-- this version will apply on top of the patch in PR165155, as the errx > calls in that patch need to be modified to warnx calls. > I have updated these 2 patches above to branch correctly (the change from errx to warnx requires else'ing ... may have to purge browser catch to pick up the change), wondering if this patch should be expanded to include all of the cases mentioned in the ``Consequences of Errors'' section of the POSIX specification: > [... snip] > If a command line meeting the specified requirements cannot be assembled, > the utility cannot be invoked, an invocation of the utility is terminated > by a signal, or an invocation of the utility exits with exit status 255, > the *xargs* utility shall write a diagnostic message and exit without > processing any remaining input. > This would cause xargs to wait on children if the command line cannot be assembled, or utility cannot be invoked, in addition to the 2 cases covered by the patch. This should leave termination via signal to the xargs process itself as the only outstanding case where xargs orphans utilities, which is congruent with sh(1) behavior, and still allows for signaling all processes via signal to the process group, if you actually desire to signal all utility processes, along with xargs itself. Thoughts? > > The patch preserves orphaning when the xargs process itself is terminated > by signal, but augments the behavior when a child utility process is > terminated by signal or exits 255 to wait for other existing child > utilities until exiting 1. My reasoning for this (beyond orphaning > nastiness) is that I always want to fail as soon as I know an operation is > fatal, and then clean-up. By orphaning children, there is no reliable way > to clean-up following use of the 255 exit code (or signal termination): > > $ # a contrived example forcing a race-condition, with a clean-up function. > $ mkdir -p foo; jot - 1 10 | xargs -P5 -n1 sh -c 'sleep $1; touch foo/$1; > exit 255;' worker || find foo -type f -delete > $ # demonstration that cleanup is not possible > $ sleep 5 && ls -l foo > total 2 > -rw-r--r-- 1 matt matt 0 Feb 16 19:01 2 > -rw-r--r-- 1 matt matt 0 Feb 16 19:01 3 > -rw-r--r-- 1 matt matt 0 Feb 16 19:01 4 > -rw-r--r-- 1 matt matt 0 Feb 16 19:01 5 > > Following the patch, we get a nice and reliable cleanup, as we have no > orphans: > $ mkdir -p foo; jot - 1 10 | usr.bin/xargs/xargs -P5 -n1 sh -c 'sleep $1; > touch foo/$1; exit 255;' worker || find foo -type f -delete > xargs: sh: exited with status 255, aborting > xargs: sh: exited with status 255, aborting > xargs: sh: exited with status 255, aborting > xargs: sh: exited with status 255, aborting > xargs: sh: exited with status 255, aborting > $ ls -l foo/ > total 0 > > Please let me know what you think, I would very much like to see this > patch make it's way into xargs as I find this short-circuit behavior nearly > very usable, and it's only failing is that it orphans children > unnecessarily, resulting in unpredictable behavior. > > -- > regards, > matt > -- regards, matt