From owner-svn-src-head@freebsd.org Tue Dec 5 17:18:29 2017 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DC859E71922; Tue, 5 Dec 2017 17:18:29 +0000 (UTC) (envelope-from devin@shxd.cx) Received: from shxd.cx (mail.shxd.cx [64.201.244.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C8A0663CE1; Tue, 5 Dec 2017 17:18:29 +0000 (UTC) (envelope-from devin@shxd.cx) Received: from [64.201.244.132] (port=54936 helo=[10.0.0.109]) by shxd.cx with esmtps (TLSv1:AES256-SHA:256) (Exim 4.77 (FreeBSD)) (envelope-from ) id 1eMGrU-000NA0-G5; Tue, 05 Dec 2017 17:18:28 +0000 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: svn commit: r326554 - in head: . usr.bin/sponge usr.bin/sponge/tests usr.bin/tee From: Devin Teske X-Mailer: iPhone Mail (13G36) In-Reply-To: Date: Tue, 5 Dec 2017 09:18:28 -0800 Cc: Hans Petter Selasky , rgrimes@freebsd.org, cem@freebsd.org, Eitan Adler , src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <2E8767A0-D427-47D1-B294-B30330AD970D@shxd.cx> References: <201712051258.vB5CwjQN051356@pdx.rh.CN85.dnsmgr.net> <22918eec-4c98-01e4-4c63-e145fbc6eab9@selasky.org> <16BDAEF5-87B3-482F-B5AB-372E128D0514@shxd.cx> To: Matt Joras X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Dec 2017 17:18:30 -0000 >> On Dec 5, 2017, at 8:29 AM, Matt Joras wrote: >>=20 >> On Tue, Dec 5, 2017 at 8:06 AM, Devin Teske wrote: >>=20 >>=20 >> The problems I have are: >>=20 >> 1. Should be in ports >>=20 >> Not pre-installed on Linux, why should we have it in base? > "Pre-installed on Linux" is meaningless. The closest analog to our > base is arguably GNU coreutils. It is indeed not part of GNU coreutils > but then again there are several things in our base that are not in > coreutils, and vice versa. Why should we have anything in base? If > people find it useful and it doens't have a high cost of > maintainership then why not have it? Hand-waving and baseless conjecture is not going to change the fact that I h= ave administered tens-of-thousands of Linux machines and: 1. None have ever had moreutils installed 2. Before today I have never heard of sponge or moreutils You point to Linux not having a "base" as it were, but this is patently fals= e. In the kickstart scripts for anaconda, you can specify ^@minimal or @base or= @whatever to get a classification of packages that have been glommed togeth= er. When installing Linux (be it CentOS, RedHat, Ubuntu, or whatever) there a= re package collections. No collection that I have ever installef anywhere has ever installed moreuti= ls. Ever. Contrast that with the facts that: All linux distros regardless of what package collection you choose gives you= awk, sed, tr, etc. So while you point to the notion that "Linux has no base" and "everything is= a package anyway" (my paraphrasing), this is not the case. It is not arguing "slippery slope" to not want sponge in base, but rather th= at it literally does not get installed with any package collection in Linux a= nd therefore no Linux base has it (the "base" in Linux is based on the packa= ge collection you choose, and thus since you cannot choose a package collect= ion that contains it, you cannot build a Linux system that has sponge on fir= st-boot unless you explicitly mention sponge as a package in the packages se= ction of a kickstart/preseed or manually select during install). >=20 >> If in base, people will target it thinking it solves a need that can't >> otherwise be solved and thus end up creating a script that is less portab= le >> because it is encumbered with dependencies specific to our base. > It's not even a homegrown idea though... As we've already covered this > is a tool that exists in the broader OSS ecosystem. Tens of thousands of machines across multiple companies tells me otherwise. > As long as it is > compatible with the more common implementation I don't see the issue. > Anything one writes using it is just "encumbered" with a dependency on > sponge. >> 2. Teaches bad practice >>=20 >> sed ... somefile | sponge somefile >>=20 >> Ignores if there is a sed error and indiscriminately zeroes somefile. >=20 > Calling this unequivocally bad practice is silly. sh(1) manually calls it bad practice > There are plenty > uses of sponge that aren't bad practice. I have a git commit hook that > utilizes sponge to do the same "auto-culling" that our svn patches do. Ignoring errors in a pipeline as was pointed out. > I like the sponge version better than creating temporary files myself: You. Do not. Need. Temporary files. > sed '/^$/d' $(git config commit.message) | awk 'NR=3D=3DFNR{a[$0];next} > !($0 in a)' /dev/fd/0 "$1" | sponge "$1" No temp files AND proper error checking. set -e # All errors fatal data=3D$( sed '/^$/d' $(git config commit.message) ) uncommon_lines=3D$( echo "$data" | awk 'NR=3D=3DFNR{a[$0];next} !($0 in a)' /= dev/fd/0 "$1" ) echo "$uncommon_lines" > "$1" If you don't want the "set -e" just throw "|| exit" at the end of the first t= wo lines. The important bit is: 1. If the path produced by $( git config commit.message ) makes sed throw an= error (e.g., ENOENT) resulting in error status, you don't blindly go on to o= verwrite "$1" 2. If awk is unable to open and read "$1" on the second line, you do not bli= ndly forge ahead and overwrtie "$1" 3. Only if you successfully ran sed, successfully ran awk, should "$1" be up= dated. The sponge approach is just plain bad practice because: A. An error on the commit message results in "$1" being truncated to zero by= tes B. An error on /dev/fd/0 results in "$1" being truncated to zero bytes C. An error by awk on "$1" results in "$1" being truncated to zero bytes (e.= g., a flaky NFS connection wherein an awk read() fails but the sponge write(= ) succeeds). >=20 >> 3. Solution in search of a problem > Again, stating this unequivocally is silly. I discovered sponge years > ago when I was searching how best to handle something where I wanted > to write output back to the same file in a shell pipeline. Something that is explicitly warned against in sh(1). > I was > literally someone with a problem in search of a solution, and that > solution was and still is sponge. Sure, if you fly in the face of warnings abound. > Since then I have seen it > recommended numerous times in passing. Ubiquitous bad practice is still bad practice. --=20 Devin=