From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 14 19:18:29 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D8B431065675 for ; Tue, 14 Feb 2012 19:18:29 +0000 (UTC) (envelope-from matthewstory@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 94F828FC08 for ; Tue, 14 Feb 2012 19:18:29 +0000 (UTC) Received: by vbbfa15 with SMTP id fa15so330709vbb.13 for ; Tue, 14 Feb 2012 11:18:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=cu8kkM9WxjZr07VRvDuHqeqUDVOogaWtvfbr1n6xvwI=; b=xuC1vO+7sPV1VupNZqEuAh9wR9R1LxXs91H81+0caRXhM4CwTsi2N71BHZR7z0fWNI mNMtMruOSMnkKyabQuqPOhkiSSYzewUldP04+Svc/xSer5EwLgYxFVCIEW+sjzgEoKfd OT+5dGyQEg1MXuHWlaFRrrBz9MoLDuzmjIoqo= MIME-Version: 1.0 Received: by 10.52.73.169 with SMTP id m9mr9660181vdv.95.1329247108903; Tue, 14 Feb 2012 11:18:28 -0800 (PST) Received: by 10.52.21.84 with HTTP; Tue, 14 Feb 2012 11:18:28 -0800 (PST) In-Reply-To: <093b01cceb4b$9a956ba0$cfc042e0$@fisglobal.com> References: <093b01cceb4b$9a956ba0$cfc042e0$@fisglobal.com> Date: Tue, 14 Feb 2012 14:18:28 -0500 Message-ID: From: Matthew Story To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: xargs short-circuit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Feb 2012 19:18:29 -0000 On Tue, Feb 14, 2012 at 2:05 PM, Devin Teske wrote: > > > > -----Original Message----- > > From: owner-freebsd-hackers@freebsd.org [mailto:owner-freebsd- > > hackers@freebsd.org] On Behalf Of Matthew Story > > Sent: Tuesday, February 14, 2012 10:35 AM > > To: freebsd-hackers@freebsd.org > > Subject: xargs short-circuit > > > > After reading the man-page, and browsing around the internet for a > minute, > > I was just wondering if there is an option in (any) xargs to > short-circuit > > on first failure of [utility [arguments]]. > > > > e.g. > > > > $ jot - 1 10 | xargs -e -n1 sh -c 'echo "$*"; echo exit 1' worker || > echo $? > > 1 > > 1 > > > > such that any non-0 exit code in a child process would cause xargs to > stop > > processing. seems like this would be a nice feature to have. > > > > You can achieve this quite easily with a sub-shell: > > As a bourne-shell script: > > #!/bin/sh > jot - 1 10 | ( while read ARG1 REST; do > sh -c 'echo "$*"; exit 1' worker $ARG1 || exit $? > shift 1 > done ) > read is often not sufficient for a variety of reasons, the most notable of them is that new-lines are valid in file names on most file systems. While some shells do support a variety of options, POSIX only supports -r (raw, treat backslashes as literal, not escape). find . -print0 | xargs -0 -e ... Is vastly nicer in most cases. Additionally, xargs provides the possibility of concurrency, via -P ... while you can spoof this with trailing & and wait(1) in sh, this is both vastly more complicated than xargs -P, and not as efficient in spawning jobs, it would be nice to be able to have xargs stop spawning new jobs on first failure in this case, and exit at last reap of existing child processes if the short-circuit flag is sent: find . -print0 | xargs -n1 -0 -e -P4 ... My use-case is a CPU-bound operation running with concurrency and many more jobs than concurrency, on failure xargs continues to work until finished to report failure, which is a large number of wasted cycles, and box load. Would be nice to bail as early as possible in situations where any failure is fatal to the larger operation. > > Or interactively in sh/bash: > > $ jot - 1 10 | ( while read ARG1 REST; do sh -c 'echo "$*"; exit 1' worker > $ARG1 > || exit $?; shift 1;done ) > > Or interactively in csh/tcsh: > > % jot - 1 10 | /bin/sh -c 'while read ARG1 REST; do sh -c '\''echo "$*"; > exit > 1'\'' worker $ARG1 || exit $?; shift 1; done' > > -- > Devin > > _____________ > The information contained in this message is proprietary and/or > confidential. If you are not the intended recipient, please: (i) delete the > message and all copies; (ii) do not disclose, distribute or use the message > in any manner; and (iii) notify the sender immediately. In addition, please > be aware that any message addressed to our domain is subject to archiving > and review by persons other than the intended recipient. Thank you. > -- regards, matt