Date: Tue, 05 Dec 2017 07:04:04 -0800 From: Cy Schubert <Cy.Schubert@komquats.com> To: Eitan Adler <eadler@FreeBSD.org> Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r326554 - in head: . usr.bin/sponge usr.bin/sponge/tests usr.bin/tee Message-ID: <201712051504.vB5F458w002872@slippy.cwsent.com> In-Reply-To: Message from Eitan Adler <eadler@FreeBSD.org> of "Tue, 05 Dec 2017 03:55:10 %2B0000." <201712050355.vB53tApl074041@repo.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
In message <201712050355.vB53tApl074041@repo.freebsd.org>, Eitan Adler writes: > Author: eadler > Date: Tue Dec 5 03:55:10 2017 > New Revision: 326554 > URL: https://svnweb.freebsd.org/changeset/base/326554 > > Log: > sponge(1): initial commit > > sponge(1) is a utility that reads input until > complete, then opens the output file, then > writes to it. This makes it useful in pipelines > that read and write to the same file. > > Reviewed by: wblock, jilles, imp, cem, danfe (all: various iteration > s) > Inspired by: https://joeyh.name/code/moreutils/ > > Added: > head/usr.bin/sponge/ > head/usr.bin/sponge/Makefile (contents, props changed) > head/usr.bin/sponge/sponge.1 (contents, props changed) > head/usr.bin/sponge/sponge.c (contents, props changed) > head/usr.bin/sponge/tests/ > head/usr.bin/sponge/tests/Makefile (contents, props changed) > head/usr.bin/sponge/tests/Makefile.depend (contents, props changed) > head/usr.bin/sponge/tests/sponge_test.sh (contents, props changed) > Modified: > head/.arclint > head/usr.bin/tee/tee.1 > > Modified: head/.arclint > ============================================================================= > = > --- head/.arclint Tue Dec 5 02:23:36 2017 (r326553) > +++ head/.arclint Tue Dec 5 03:55:10 2017 (r326554) > @@ -9,7 +9,8 @@ > "type": "spelling" > }, > "chmod": { > - "type": "chmod" > + "type": "chmod", > + "exclude": "(/tests/)" > }, > "merge-conflict": { > "type": "merge-conflict" > > Added: head/usr.bin/sponge/Makefile > ============================================================================= > = > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ head/usr.bin/sponge/Makefile Tue Dec 5 03:55:10 2017 (r32655 > 4) > @@ -0,0 +1,8 @@ > +# $FreeBSD$ > + > +PROG= sponge > + > +HAS_TESTS= > +SUBDIR.${MK_TESTS}+= tests > + > +.include <bsd.prog.mk> > > Added: head/usr.bin/sponge/sponge.1 > ============================================================================= > = > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ head/usr.bin/sponge/sponge.1 Tue Dec 5 03:55:10 2017 (r32655 > 4) > @@ -0,0 +1,75 @@ > +.\" Eitan Adler. All rights reserved. > +.\" > +.\" Redistribution and use in source and binary forms, with or without > +.\" modification, are permitted provided that the following conditions > +.\" are met: > +.\" 1. Redistributions of source code must retain the above copyright > +.\" notice, this list of conditions and the following disclaimer. > +.\" 2. Redistributions in binary form must reproduce the above copyright > +.\" notice, this list of conditions and the following disclaimer in the > +.\" documentation and/or other materials provided with the distribution. > +.\" > +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND > +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE > +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPO > SE > +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE > +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTI > AL > +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS > +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRI > CT > +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WA > Y > +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF > +.\" SUCH DAMAGE. > +.\" > +.\" $FreeBSD$ > +.\" > +.Dd November 1, 2017 > +.Dt SPONGE 1 > +.Os > +.Sh NAME > +.Nm sponge > +.Nd buffer stdin and write to stdout > +.Sh SYNOPSIS > +.Nm > +.Op Fl a > +.Ar filename > +.Sh DESCRIPTION > +The > +.Nm > +utility reads standard in until complete, then opens > +the output file and writes to it. > +This makes it useful in pipelines that read a file and then write to it. > +These options are available: > +.Bl -tag -width indent > +.It Fl a > +Open > +.Ar filename > +in append mode. > +.El > +.Pp > +If an attempt to allocate memory fails, > +.Nm > +fails without output. > +The file is written even if earlier components > +of the pipeline failed. > +.Sh SEE ALSO > +.Xr builtin 1 , > +.Xr csh 1 , > +.Xr getrusage 2 , > +.Xr tee 1 , > +.Xr wait 2 > +.Sh EXIT STATUS > +.Ex -std > +.Sh EXAMPLES > +A > +.Pa file > +can be be sorted "in place" by executing > +.Cm sort file | sponge file > +.Sh HISTORY > +The > +.Nm > +utility was written by > +.An Eitan Adler Aq Mt eadler@FreeBSD.org > +and first appeared > +in > +.Fx 12.0 . > > Added: head/usr.bin/sponge/sponge.c > ============================================================================= > = > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ head/usr.bin/sponge/sponge.c Tue Dec 5 03:55:10 2017 (r32655 > 4) > @@ -0,0 +1,189 @@ > +/*- > + * SPDX-License-Identifier: BSD-2-Clause > + * > + * Copyright (c) 2017 Eitan Adler > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * 1. Redistributions of source code must retain the above copyright > + * notice, this list of conditions and the following disclaimer. > + * 2. Redistributions in binary form must reproduce the above copyright > + * notice, this list of conditions and the following disclaimer in the > + * documentation and/or other materials provided with the distribution. > + * > + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND > + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE > + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOS > E > + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE > + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIA > L > + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS > + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRIC > T > + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY > + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF > + * SUCH DAMAGE. > + * > + * $FreeBSD$ > + */ > + > +#include <err.h> > +#include <fcntl.h> > +#include <limits.h> > +#include <stdbool.h> > +#include <stdio.h> > +#include <stdlib.h> > +#include <getopt.h> > +#include <unistd.h> > +#include <sys/uio.h> > + > +#define DEFAULT_BUF_SIZE 16384 > +#define DEFAULT_BUF_CNT 12 > + > +static int flag_append = 0; > + > +static void usage(void); > +static void *safe_malloc(size_t size); > +static void *safe_calloc(size_t count, size_t size); > +static void *safe_reallocf(void *ptr, size_t size); > + > +static void * > +safe_malloc(size_t size) > +{ > + void *ret; > + > + ret = malloc(size); > + if (ret == NULL) { > + err(1, "malloc failed"); > + } > + return (ret); > +} > + > +static void * > +safe_calloc(size_t count, size_t size) > +{ > + void *ret; > + > + ret = calloc(count, size); > + if (ret == NULL) { > + err(1, "calloc failed"); > + } > + return (ret); > +} > + > +static void * > +safe_reallocf(void *ptr, size_t size) > +{ > + void *ret; > + > + ret = reallocf(ptr, size); > + if (ret == NULL) { > + err(1, "reallocf failed"); > + } > + return (ret); > +} > + > +static void > +usage(void) > +{ > + fprintf(stderr, "usage: sponge [-a] filename\n"); > +} > + > +int > +main(int argc, char* argv[]) > +{ > + struct iovec *iov; > + char *buf; > + char *outfile; > + ssize_t i; > + size_t bufcnt; > + size_t whichbuf; > + size_t bufremain; > + long maxiovec; > + int fd; > + int openflags = O_WRONLY; > + int opt; > + > + while ((opt = getopt(argc, argv, "ah")) != -1) { > + switch (opt) { > + case 'a': > + flag_append = 1; > + break; > + case 'h': > + usage(); > + exit(0); > + case '?': > + default: > + usage(); > + exit(1); > + } > + } > + > + if (optind < argc) { > + outfile = argv[optind]; > + } > + > + > + bufcnt = DEFAULT_BUF_CNT; > + whichbuf = 0; > + iov = safe_calloc(bufcnt, sizeof(*iov)); > + > + for (;;) { > + buf = safe_malloc(DEFAULT_BUF_SIZE); > + i = read(STDIN_FILENO, buf, DEFAULT_BUF_SIZE); > + if (whichbuf == bufcnt) { > + bufcnt *= 2; > + iov = safe_reallocf(iov, bufcnt * sizeof(*iov)); > + } > + if (i < 0) { > + err(1, "read failed"); > + } > + if (i == 0) { > + free(buf); > + break; > + } > + iov[whichbuf].iov_base = buf; > + iov[whichbuf].iov_len = i; > + whichbuf++; > + } > + > + if (outfile) { > + if (flag_append) { > + openflags |= O_APPEND; > + } else { > + openflags |= O_TRUNC; > + } > + fd = open(outfile, openflags); > + } > + else { > + fd = STDOUT_FILENO; > + } > + > + if (fd < 0) { > + err(1, "failed to open"); > + } > + > + maxiovec = sysconf(_SC_IOV_MAX); > + if (maxiovec == -1) { > + maxiovec = _XOPEN_IOV_MAX; > + } > + bufcnt = whichbuf; > + bufremain = bufcnt; > + > + while (bufremain > 0) { > + whichbuf = (bufremain < maxiovec) ? bufremain : maxiovec; Buildworld stops here. Should this not be (unsigned long) maxiovec? > + bufremain -= whichbuf; > + > [...] -- Cheers, Cy Schubert <Cy.Schubert@cschubert.com> FreeBSD UNIX: <cy@FreeBSD.org> Web: http://www.FreeBSD.org The need of the many outweighs the greed of the few.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201712051504.vB5F458w002872>