From owner-freebsd-arch@FreeBSD.ORG Fri Feb 24 18:04:31 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BAF5E106564A for ; Fri, 24 Feb 2012 18:04:31 +0000 (UTC) (envelope-from matthewstory@gmail.com) Received: from mail-qw0-f47.google.com (mail-qw0-f47.google.com [209.85.216.47]) by mx1.freebsd.org (Postfix) with ESMTP id 65D1F8FC0C for ; Fri, 24 Feb 2012 18:04:31 +0000 (UTC) Received: by qadz30 with SMTP id z30so116124qad.13 for ; Fri, 24 Feb 2012 10:04:30 -0800 (PST) Received-SPF: pass (google.com: domain of matthewstory@gmail.com designates 10.229.102.79 as permitted sender) client-ip=10.229.102.79; Authentication-Results: mr.google.com; spf=pass (google.com: domain of matthewstory@gmail.com designates 10.229.102.79 as permitted sender) smtp.mail=matthewstory@gmail.com; dkim=pass header.i=matthewstory@gmail.com Received: from mr.google.com ([10.229.102.79]) by 10.229.102.79 with SMTP id f15mr2620217qco.94.1330106670824 (num_hops = 1); Fri, 24 Feb 2012 10:04:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=BJtX81+ZyoT7o/iWRZQ/TBEEHJ6RztkKM1dEvcH0X2w=; b=REXmK80D5U9KgHuExZhqmaMYgpavB5ojg/Qf7k3JvAxByZkf3vUgRoVuNf5RL1T46P WBUlu5Fpwtj/hGzy6Bo52D/vw0niwlf/hRAACno9TSaQKimNVhrAzQsWLgxgYjfmOADl M3T5YKHBEGdd5m6JYWHL92SPdIZCLU6jCZbzw= MIME-Version: 1.0 Received: by 10.229.102.79 with SMTP id f15mr2164583qco.94.1330106670664; Fri, 24 Feb 2012 10:04:30 -0800 (PST) Received: by 10.229.95.74 with HTTP; Fri, 24 Feb 2012 10:04:22 -0800 (PST) In-Reply-To: References: Date: Fri, 24 Feb 2012 13:04:22 -0500 Message-ID: From: Matthew Story To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: Change to xargs to avoid orphaning of utility processes on signal|exit 255 from child X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Feb 2012 18:04:31 -0000 On Tue, Feb 21, 2012 at 12:50 PM, Matthew Story wrote: > On Thu, Feb 16, 2012 at 7:09 PM, Matthew Story wrote: > >> Apologies if this is the wrong list, I would like to submit a patch that >> changes the behavior of xargs(1) on signal to child utility process or >> child utility process exiting 255. The patch(es) is|are available here: >> >> >> http://axe0.blackskyresearch.net/patches/matt/xargs.no_orphan.patch.txt-- this version will apply to current xargs, and adds diagnostic >> information for exit 255|signal to utility, as required by POSIX (see >> PR165155). >> >> http://axe0.blackskyresearch.net/patches/matt/xargs.no_orphan.PR165155.patch.txt-- this version will apply on top of the patch in PR165155, as the errx >> calls in that patch need to be modified to warnx calls. >> > My HTTP logs are indicating that these links are being hit with trailing --'s, so just cleaning these up ... time to switch back to a real mail client, i guess ... http://axe0.blackskyresearch.net/patches/matt/xargs.no_orphan.patch.txt http://axe0.blackskyresearch.net/patches/matt/xargs.no_orphan.PR165155.patch.txt Sorry for any confusion. > I have updated these 2 patches above to branch correctly (the change from > errx to warnx requires else'ing ... may have to purge browser catch to pick > up the change), wondering if this patch should be expanded to include all > of the cases mentioned in the ``Consequences of Errors'' section of the > POSIX specification: > > >> [... snip] >> If a command line meeting the specified requirements cannot be assembled, >> the utility cannot be invoked, an invocation of the utility is terminated >> by a signal, or an invocation of the utility exits with exit status 255, >> the *xargs* utility shall write a diagnostic message and exit without >> processing any remaining input. >> > > This would cause xargs to wait on children if the command line cannot be > assembled, or utility cannot be invoked, in addition to the 2 cases covered > by the patch. This should leave termination via signal to the xargs > process itself as the only outstanding case where xargs orphans utilities, > which is congruent with sh(1) behavior, and still allows for signaling all > processes via signal to the process group, if you actually desire to signal > all utility processes, along with xargs itself. Thoughts? > > >> >> The patch preserves orphaning when the xargs process itself is terminated >> by signal, but augments the behavior when a child utility process is >> terminated by signal or exits 255 to wait for other existing child >> utilities until exiting 1. My reasoning for this (beyond orphaning >> nastiness) is that I always want to fail as soon as I know an operation is >> fatal, and then clean-up. By orphaning children, there is no reliable way >> to clean-up following use of the 255 exit code (or signal termination): >> >> $ # a contrived example forcing a race-condition, with a clean-up >> function. >> $ mkdir -p foo; jot - 1 10 | xargs -P5 -n1 sh -c 'sleep $1; touch foo/$1; >> exit 255;' worker || find foo -type f -delete >> $ # demonstration that cleanup is not possible >> $ sleep 5 && ls -l foo >> total 2 >> -rw-r--r-- 1 matt matt 0 Feb 16 19:01 2 >> -rw-r--r-- 1 matt matt 0 Feb 16 19:01 3 >> -rw-r--r-- 1 matt matt 0 Feb 16 19:01 4 >> -rw-r--r-- 1 matt matt 0 Feb 16 19:01 5 >> >> Following the patch, we get a nice and reliable cleanup, as we have no >> orphans: >> $ mkdir -p foo; jot - 1 10 | usr.bin/xargs/xargs -P5 -n1 sh -c 'sleep $1; >> touch foo/$1; exit 255;' worker || find foo -type f -delete >> xargs: sh: exited with status 255, aborting >> xargs: sh: exited with status 255, aborting >> xargs: sh: exited with status 255, aborting >> xargs: sh: exited with status 255, aborting >> xargs: sh: exited with status 255, aborting >> $ ls -l foo/ >> total 0 >> >> Please let me know what you think, I would very much like to see this >> patch make it's way into xargs as I find this short-circuit behavior nearly >> very usable, and it's only failing is that it orphans children >> unnecessarily, resulting in unpredictable behavior. >> >> -- >> regards, >> matt >> > > > > -- > regards, > matt > -- regards, matt