Date: Tue, 5 Dec 2017 09:18:28 -0800 From: Devin Teske <devin@shxd.cx> To: Matt Joras <matt.joras@gmail.com> Cc: Hans Petter Selasky <hps@selasky.org>, rgrimes@freebsd.org, cem@freebsd.org, Eitan Adler <eadler@freebsd.org>, src-committers <src-committers@freebsd.org>, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r326554 - in head: . usr.bin/sponge usr.bin/sponge/tests usr.bin/tee Message-ID: <2E8767A0-D427-47D1-B294-B30330AD970D@shxd.cx> In-Reply-To: <CADdTf%2Bhzcn2S6DjKMY5JBSjjYioQ5qSkVPPYG=wH4YrHBRmRDg@mail.gmail.com> References: <201712051258.vB5CwjQN051356@pdx.rh.CN85.dnsmgr.net> <22918eec-4c98-01e4-4c63-e145fbc6eab9@selasky.org> <AE022237-A2D5-49E1-A6DA-DD65A460B266@shxd.cx> <CADdTf%2Bh=Tqb635r7e8U8HdZQtwSdNtzcoC4hXuhRk7ir=TXviw@mail.gmail.com> <CADdTf%2BiWp149DifcSA7K-6DzzOzyHpbsGDC2FNvnFGdoe0Yrfg@mail.gmail.com> <CADdTf%2BhLaLUb=XF9jAqPOx7TgFXrWyf3PJywP-KsaDrvu3xs4w@mail.gmail.com> <16BDAEF5-87B3-482F-B5AB-372E128D0514@shxd.cx> <CADdTf%2Bhzcn2S6DjKMY5JBSjjYioQ5qSkVPPYG=wH4YrHBRmRDg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
>> On Dec 5, 2017, at 8:29 AM, Matt Joras <matt.joras@gmail.com> wrote:
>>=20
>> On Tue, Dec 5, 2017 at 8:06 AM, Devin Teske <devin@shxd.cx> wrote:
>>=20
>>=20
>> The problems I have are:
>>=20
>> 1. Should be in ports
>>=20
>> Not pre-installed on Linux, why should we have it in base?
> "Pre-installed on Linux" is meaningless. The closest analog to our
> base is arguably GNU coreutils. It is indeed not part of GNU coreutils
> but then again there are several things in our base that are not in
> coreutils, and vice versa. Why should we have anything in base? If
> people find it useful and it doens't have a high cost of
> maintainership then why not have it?
Hand-waving and baseless conjecture is not going to change the fact that I h=
ave administered tens-of-thousands of Linux machines and:
1. None have ever had moreutils installed
2. Before today I have never heard of sponge or moreutils
You point to Linux not having a "base" as it were, but this is patently fals=
e.
In the kickstart scripts for anaconda, you can specify ^@minimal or @base or=
@whatever to get a classification of packages that have been glommed togeth=
er. When installing Linux (be it CentOS, RedHat, Ubuntu, or whatever) there a=
re package collections.
No collection that I have ever installef anywhere has ever installed moreuti=
ls. Ever.
Contrast that with the facts that:
All linux distros regardless of what package collection you choose gives you=
awk, sed, tr, etc.
So while you point to the notion that "Linux has no base" and "everything is=
a package anyway" (my paraphrasing), this is not the case.
It is not arguing "slippery slope" to not want sponge in base, but rather th=
at it literally does not get installed with any package collection in Linux a=
nd therefore no Linux base has it (the "base" in Linux is based on the packa=
ge collection you choose, and thus since you cannot choose a package collect=
ion that contains it, you cannot build a Linux system that has sponge on fir=
st-boot unless you explicitly mention sponge as a package in the packages se=
ction of a kickstart/preseed or manually select during install).
>=20
>> If in base, people will target it thinking it solves a need that can't
>> otherwise be solved and thus end up creating a script that is less portab=
le
>> because it is encumbered with dependencies specific to our base.
> It's not even a homegrown idea though... As we've already covered this
> is a tool that exists in the broader OSS ecosystem.
Tens of thousands of machines across multiple companies tells me otherwise.
> As long as it is
> compatible with the more common implementation I don't see the issue.
> Anything one writes using it is just "encumbered" with a dependency on
> sponge.
>> 2. Teaches bad practice
>>=20
>> sed ... somefile | sponge somefile
>>=20
>> Ignores if there is a sed error and indiscriminately zeroes somefile.
>=20
> Calling this unequivocally bad practice is silly.
sh(1) manually calls it bad practice
> There are plenty
> uses of sponge that aren't bad practice. I have a git commit hook that
> utilizes sponge to do the same "auto-culling" that our svn patches do.
Ignoring errors in a pipeline as was pointed out.
> I like the sponge version better than creating temporary files myself:
You.
Do not.
Need.
Temporary files.
> sed '/^$/d' $(git config commit.message) | awk 'NR=3D=3DFNR{a[$0];next}
> !($0 in a)' /dev/fd/0 "$1" | sponge "$1"
No temp files AND proper error checking.
set -e # All errors fatal
data=3D$( sed '/^$/d' $(git config commit.message) )
uncommon_lines=3D$( echo "$data" | awk 'NR=3D=3DFNR{a[$0];next} !($0 in a)' /=
dev/fd/0 "$1" )
echo "$uncommon_lines" > "$1"
If you don't want the "set -e" just throw "|| exit" at the end of the first t=
wo lines. The important bit is:
1. If the path produced by $( git config commit.message ) makes sed throw an=
error (e.g., ENOENT) resulting in error status, you don't blindly go on to o=
verwrite "$1"
2. If awk is unable to open and read "$1" on the second line, you do not bli=
ndly forge ahead and overwrtie "$1"
3. Only if you successfully ran sed, successfully ran awk, should "$1" be up=
dated.
The sponge approach is just plain bad practice because:
A. An error on the commit message results in "$1" being truncated to zero by=
tes
B. An error on /dev/fd/0 results in "$1" being truncated to zero bytes
C. An error by awk on "$1" results in "$1" being truncated to zero bytes (e.=
g., a flaky NFS connection wherein an awk read() fails but the sponge write(=
) succeeds).
>=20
>> 3. Solution in search of a problem
> Again, stating this unequivocally is silly. I discovered sponge years
> ago when I was searching how best to handle something where I wanted
> to write output back to the same file in a shell pipeline.
Something that is explicitly warned against in sh(1).
> I was
> literally someone with a problem in search of a solution, and that
> solution was and still is sponge.
Sure, if you fly in the face of warnings abound.
> Since then I have seen it
> recommended numerous times in passing.
Ubiquitous bad practice is still bad practice.
--=20
Devin=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2E8767A0-D427-47D1-B294-B30330AD970D>