Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 5 Dec 2017 09:17:42 -0800
From:      Conrad Meyer <cem@freebsd.org>
To:        Eitan Adler <eadler@freebsd.org>
Cc:        src-committers <src-committers@freebsd.org>, svn-src-all@freebsd.org,  svn-src-head@freebsd.org
Subject:   Re: svn commit: r326554 - in head: . usr.bin/sponge usr.bin/sponge/tests usr.bin/tee
Message-ID:  <CAG6CVpURfvTWR2ZF4hyQFk=3THOTOn21Gthm%2BVPOUY6RHzUg5g@mail.gmail.com>
In-Reply-To: <201712050355.vB53tApl074041@repo.freebsd.org>
References:  <201712050355.vB53tApl074041@repo.freebsd.org>

next in thread | previous in thread | raw e-mail | index | archive | help
Please revert this change, along with r326555 and r326557.

I am not opposed to the idea of a version of this utility in base.

I am opposed to breaking the build, and functionally broken programs.

Thanks,
Conrad

On Mon, Dec 4, 2017 at 7:55 PM, Eitan Adler <eadler@freebsd.org> wrote:
> Author: eadler
> Date: Tue Dec  5 03:55:10 2017
> New Revision: 326554
> URL: https://svnweb.freebsd.org/changeset/base/326554
>
> Log:
>   sponge(1): initial commit
>
>   sponge(1) is a utility that reads input until
>   complete, then opens the output file, then
>   writes to it. This makes it useful in pipelines
>   that read and write to the same file.
>
>   Reviewed by:  wblock, jilles, imp, cem, danfe (all: various iterations)
>   Inspired by:  https://joeyh.name/code/moreutils/
>
> Added:
>   head/usr.bin/sponge/
>   head/usr.bin/sponge/Makefile   (contents, props changed)
>   head/usr.bin/sponge/sponge.1   (contents, props changed)
>   head/usr.bin/sponge/sponge.c   (contents, props changed)
>   head/usr.bin/sponge/tests/
>   head/usr.bin/sponge/tests/Makefile   (contents, props changed)
>   head/usr.bin/sponge/tests/Makefile.depend   (contents, props changed)
>   head/usr.bin/sponge/tests/sponge_test.sh   (contents, props changed)
> Modified:
>   head/.arclint
>   head/usr.bin/tee/tee.1
>
> Modified: head/.arclint
> ==============================================================================
> --- head/.arclint       Tue Dec  5 02:23:36 2017        (r326553)
> +++ head/.arclint       Tue Dec  5 03:55:10 2017        (r326554)
> @@ -9,7 +9,8 @@
>        "type": "spelling"
>      },
>      "chmod": {
> -      "type": "chmod"
> +      "type": "chmod",
> +      "exclude": "(/tests/)"
>      },
>      "merge-conflict": {
>        "type": "merge-conflict"
>
> Added: head/usr.bin/sponge/Makefile
> ==============================================================================
> --- /dev/null   00:00:00 1970   (empty, because file is newly added)
> +++ head/usr.bin/sponge/Makefile        Tue Dec  5 03:55:10 2017        (r326554)
> @@ -0,0 +1,8 @@
> +# $FreeBSD$
> +
> +PROG=  sponge
> +
> +HAS_TESTS=
> +SUBDIR.${MK_TESTS}+= tests
> +
> +.include <bsd.prog.mk>
>
> Added: head/usr.bin/sponge/sponge.1
> ==============================================================================
> --- /dev/null   00:00:00 1970   (empty, because file is newly added)
> +++ head/usr.bin/sponge/sponge.1        Tue Dec  5 03:55:10 2017        (r326554)
> @@ -0,0 +1,75 @@
> +.\"    Eitan Adler.  All rights reserved.
> +.\"
> +.\" Redistribution and use in source and binary forms, with or without
> +.\" modification, are permitted provided that the following conditions
> +.\" are met:
> +.\" 1. Redistributions of source code must retain the above copyright
> +.\"    notice, this list of conditions and the following disclaimer.
> +.\" 2. Redistributions in binary form must reproduce the above copyright
> +.\"    notice, this list of conditions and the following disclaimer in the
> +.\"    documentation and/or other materials provided with the distribution.
> +.\"
> +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
> +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
> +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
> +.\" ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
> +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
> +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
> +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
> +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
> +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
> +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
> +.\" SUCH DAMAGE.
> +.\"
> +.\" $FreeBSD$
> +.\"
> +.Dd November 1, 2017
> +.Dt SPONGE 1
> +.Os
> +.Sh NAME
> +.Nm sponge
> +.Nd buffer stdin and write to stdout
> +.Sh SYNOPSIS
> +.Nm
> +.Op Fl a
> +.Ar filename
> +.Sh DESCRIPTION
> +The
> +.Nm
> +utility reads standard in until complete, then opens
> +the output file and writes to it.
> +This makes it useful in pipelines that read a file and then write to it.
> +These options are available:
> +.Bl -tag -width indent
> +.It Fl a
> +Open
> +.Ar filename
> +in append mode.
> +.El
> +.Pp
> +If an attempt to allocate memory fails,
> +.Nm
> +fails without output.
> +The file is written even if earlier components
> +of the pipeline failed.
> +.Sh SEE ALSO
> +.Xr builtin 1 ,
> +.Xr csh 1 ,
> +.Xr getrusage 2 ,
> +.Xr tee 1 ,
> +.Xr wait 2
> +.Sh EXIT STATUS
> +.Ex -std
> +.Sh EXAMPLES
> +A
> +.Pa file
> +can be be sorted "in place" by executing
> +.Cm sort file | sponge file
> +.Sh HISTORY
> +The
> +.Nm
> +utility was written by
> +.An Eitan Adler Aq Mt eadler@FreeBSD.org
> +and first appeared
> +in
> +.Fx 12.0 .
>
> Added: head/usr.bin/sponge/sponge.c
> ==============================================================================
> --- /dev/null   00:00:00 1970   (empty, because file is newly added)
> +++ head/usr.bin/sponge/sponge.c        Tue Dec  5 03:55:10 2017        (r326554)
> @@ -0,0 +1,189 @@
> +/*-
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Copyright (c) 2017 Eitan Adler
> + *
> + * Redistribution and use in source and binary forms, with or without
> + * modification, are permitted provided that the following conditions
> + * are met:
> + * 1. Redistributions of source code must retain the above copyright
> + *    notice, this list of conditions and the following disclaimer.
> + * 2. Redistributions in binary form must reproduce the above copyright
> + *    notice, this list of conditions and the following disclaimer in the
> + *    documentation and/or other materials provided with the distribution.
> + *
> + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
> + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
> + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
> + * ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
> + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
> + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
> + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
> + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
> + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
> + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
> + * SUCH DAMAGE.
> + *
> + * $FreeBSD$
> + */
> +
> +#include <err.h>
> +#include <fcntl.h>
> +#include <limits.h>
> +#include <stdbool.h>
> +#include <stdio.h>
> +#include <stdlib.h>
> +#include <getopt.h>
> +#include <unistd.h>
> +#include <sys/uio.h>
> +
> +#define DEFAULT_BUF_SIZE 16384
> +#define DEFAULT_BUF_CNT 12
> +
> +static int flag_append = 0;
> +
> +static void usage(void);
> +static void *safe_malloc(size_t size);
> +static void *safe_calloc(size_t count, size_t size);
> +static void *safe_reallocf(void *ptr, size_t size);
> +
> +static void *
> +safe_malloc(size_t size)
> +{
> +       void *ret;
> +
> +       ret = malloc(size);
> +       if (ret == NULL) {
> +               err(1, "malloc failed");
> +       }
> +       return (ret);
> +}
> +
> +static void *
> +safe_calloc(size_t count, size_t size)
> +{
> +       void *ret;
> +
> +       ret = calloc(count, size);
> +       if (ret == NULL) {
> +               err(1, "calloc failed");
> +       }
> +       return (ret);
> +}
> +
> +static void *
> +safe_reallocf(void *ptr, size_t size)
> +{
> +       void *ret;
> +
> +       ret = reallocf(ptr, size);
> +       if (ret == NULL) {
> +               err(1, "reallocf failed");
> +       }
> +       return (ret);
> +}
> +
> +static void
> +usage(void)
> +{
> +       fprintf(stderr, "usage: sponge [-a] filename\n");
> +}
> +
> +int
> +main(int argc, char* argv[])
> +{
> +       struct iovec *iov;
> +       char *buf;
> +       char *outfile;
> +       ssize_t i;
> +       size_t bufcnt;
> +       size_t whichbuf;
> +       size_t bufremain;
> +       long maxiovec;
> +       int fd;
> +       int openflags = O_WRONLY;
> +       int opt;
> +
> +       while ((opt = getopt(argc, argv, "ah")) != -1) {
> +               switch (opt) {
> +               case 'a':
> +                       flag_append = 1;
> +                       break;
> +               case 'h':
> +                       usage();
> +                       exit(0);
> +               case '?':
> +               default:
> +                       usage();
> +                       exit(1);
> +               }
> +       }
> +
> +       if (optind < argc) {
> +                outfile = argv[optind];
> +       }
> +
> +
> +       bufcnt = DEFAULT_BUF_CNT;
> +       whichbuf = 0;
> +       iov = safe_calloc(bufcnt, sizeof(*iov));
> +
> +       for (;;) {
> +               buf = safe_malloc(DEFAULT_BUF_SIZE);
> +               i = read(STDIN_FILENO, buf, DEFAULT_BUF_SIZE);
> +               if (whichbuf == bufcnt) {
> +                       bufcnt *= 2;
> +                       iov = safe_reallocf(iov, bufcnt * sizeof(*iov));
> +               }
> +               if (i < 0) {
> +                       err(1, "read failed");
> +               }
> +               if (i == 0) {
> +                       free(buf);
> +                       break;
> +               }
> +               iov[whichbuf].iov_base = buf;
> +               iov[whichbuf].iov_len = i;
> +               whichbuf++;
> +       }
> +
> +       if (outfile) {
> +               if (flag_append) {
> +                       openflags |= O_APPEND;
> +               } else {
> +                       openflags |= O_TRUNC;
> +               }
> +               fd = open(outfile, openflags);
> +       }
> +       else {
> +               fd = STDOUT_FILENO;
> +       }
> +
> +       if (fd < 0) {
> +               err(1, "failed to open");
> +       }
> +
> +       maxiovec = sysconf(_SC_IOV_MAX);
> +       if (maxiovec == -1) {
> +               maxiovec =  _XOPEN_IOV_MAX;
> +       }
> +       bufcnt = whichbuf;
> +       bufremain = bufcnt;
> +
> +       while (bufremain > 0) {
> +               whichbuf = (bufremain < maxiovec) ? bufremain : maxiovec;
> +               bufremain -= whichbuf;
> +
> +               i = writev(fd, iov, whichbuf);
> +               if (i < 0) {
> +                       err(1, "failed to write");
> +               }
> +       }
> +
> +       if (outfile) {
> +               i = close(fd);
> +               if (i < 0) {
> +                       err(1, "failed to close");
> +               }
> +       }
> +}
>
> Added: head/usr.bin/sponge/tests/Makefile
> ==============================================================================
> --- /dev/null   00:00:00 1970   (empty, because file is newly added)
> +++ head/usr.bin/sponge/tests/Makefile  Tue Dec  5 03:55:10 2017        (r326554)
> @@ -0,0 +1,9 @@
> +# $FreeBSD$
> +
> +PACKAGE=       ${TESTBASE}/usr.bin/sponge
> +
> +TESTSDIR=      ${TESTSBASE}/usr.bin/sponge/tests
> +
> +ATF_TESTS_SH=  cp_test
> +
> +.include <bsd.test.mk>
>
> Added: head/usr.bin/sponge/tests/Makefile.depend
> ==============================================================================
> --- /dev/null   00:00:00 1970   (empty, because file is newly added)
> +++ head/usr.bin/sponge/tests/Makefile.depend   Tue Dec  5 03:55:10 2017        (r326554)
> @@ -0,0 +1,18 @@
> +# $FreeBSD$
> +# Autogenerated - do NOT edit!
> +
> +DIRDEPS = \
> +       gnu/lib/csu \
> +       include \
> +       include/xlocale \
> +       lib/${CSU_DIR} \
> +       lib/atf/libatf-c \
> +       lib/libc \
> +       lib/libcompiler_rt \
> +
> +
> +.include <dirdeps.mk>
> +
> +.if ${DEP_RELDIR} == ${_DEP_RELDIR}
> +# local dependencies - needed for -jN in clean tree
> +.endif
>
> Added: head/usr.bin/sponge/tests/sponge_test.sh
> ==============================================================================
> --- /dev/null   00:00:00 1970   (empty, because file is newly added)
> +++ head/usr.bin/sponge/tests/sponge_test.sh    Tue Dec  5 03:55:10 2017        (r326554)
> @@ -0,0 +1,48 @@
> +# $FreeBSD$
> +#
> +# SPDX-License-Identifier: BSD-2-Clause
> +#
> +# Copyright 2017 Eitan Adler
> +# All rights reserved.
> +#
> +# Redistribution and use in source and binary forms, with or without
> +# modification, are permitted provided that the following conditions are
> +# met:
> +#
> +# * Redistributions of source code must retain the above copyright
> +#   notice, this list of conditions and the following disclaimer.
> +# * Redistributions in binary form must reproduce the above copyright
> +#   notice, this list of conditions and the following disclaimer in the
> +#   documentation and/or other materials provided with the distribution.
> +#
> +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> +
> +atf_test_case simple
> +simple_body() {
> +  printf 'd\ne\na\n' >| x
> +  printf 'a\nd\ne\n' >| y
> +  sort x | sponge x
> +
> +  atf_check -s exit:0 -o empty -e empty 'sort x | sponge x'
> +
> +  if ! cmp -s x y; then
> +    echo "x and y differ, but they should be equal"
> +    diff -u x y
> +    atf_fail "Original and copy do not match"
> +  fi
> +
> +}
> +
> +atf_init_test_cases() {
> +  atf_add_test_case simple
> +}
>
> Modified: head/usr.bin/tee/tee.1
> ==============================================================================
> --- head/usr.bin/tee/tee.1      Tue Dec  5 02:23:36 2017        (r326553)
> +++ head/usr.bin/tee/tee.1      Tue Dec  5 03:55:10 2017        (r326554)
> @@ -72,6 +72,8 @@ utility takes the default action for all signals,
>  except in the event of the
>  .Fl i
>  option.
> +.Sh SEE ALSO
> +.Xr sponge 1
>  .Sh EXIT STATUS
>  .Ex -std
>  .Sh STANDARDS
>



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAG6CVpURfvTWR2ZF4hyQFk=3THOTOn21Gthm%2BVPOUY6RHzUg5g>