Date: Fri, 10 Feb 2012 19:20:12 GMT From: Martin Cracauer <cracauer@cons.org> To: freebsd-bugs@FreeBSD.org Subject: Re: bin/164947: tee looses data when writing to non-blocking file descriptors Message-ID: <201202101920.q1AJKCtb081889@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
The following reply was made to PR bin/164947; it has been noted by GNATS.
From: Martin Cracauer <cracauer@cons.org>
To: Diomidis Spinellis <dds@aueb.gr>
Cc: freebsd-gnats-submit@freebsd.org
Subject: Re: bin/164947: tee looses data when writing to non-blocking file descriptors
Date: Fri, 10 Feb 2012 14:17:36 -0500
Diomidis Spinellis wrote on Fri, Feb 10, 2012 at 07:04:41AM +0000:
>
> >Number: 164947
> >Category: bin
> >Synopsis: tee looses data when writing to non-blocking file descriptors
> >Confidential: no
> >Severity: serious
> >Priority: medium
> >Responsible: freebsd-bugs
> >State: open
> >Quarter:
> >Keywords:
> >Date-Required:
> >Class: sw-bug
> >Submitter-Id: current-users
> >Arrival-Date: Fri Feb 10 07:10:09 UTC 2012
> >Closed-Date:
> >Last-Modified:
> >Originator: Diomidis Spinellis
> >Release: 8.1
> >Organization:
> AUEB
> >Environment:
> FreeBSD istlab.dmst.aueb.gr 8.1-RELEASE-p6 FreeBSD 8.1-RELEASE-p6 #0: Tue Nov 1 15:16:34 EET 2011 dds@istlab.dmst.aueb.gr:/usr/obj/usr/src/sys/ISTLAB i386
> You have new mail in /var/mail/dds
>
> >Description:
> When tee(1) tries to write to a file descriptor that has been set to non-blocking mode the write(2) call may fail with EAGAIN. Instead of retrying the operation, tee will throw that chunk of data away.
> >How-To-Repeat:
> Run the following:
> #!/usr/local/bin/bash
> # bash needed for the >(...) functionality
> # ssh apparently sets O_NONBLOCK
> # Remove the 2>/dev/null to see tee complaining
> dd count=100000 if=/dev/zero |
> tee >(ssh localhost dd of=/dev/null) 2>/dev/null |
> (ssh localhost dd of=/dev/null)
I don't think it is ssh that is causing this. If you use a named pipe
explicitly and hook ssh up to that the error doesn't appear. Seems to
be something that bash is doing there.
That doesn't mean I am opposed to handling EAGAIN.
The way I normally do it is a simple retry loop, not using select.
I'm aware of the tradeoffs, so far I was always better off not
investing a second system call into every retry.
Martin
> 100000+0 records in
> 100000+0 records out
> 51200000 bytes transferred in 9.224390 secs (5550503 bytes/sec)
> 100000+0 records in
> 100000+0 records out
> 51200000 bytes transferred in 9.061471 secs (5650297 bytes/sec)
> 92080+0 records in
> 92080+0 records out
> 47144960 bytes transferred in 9.101738 secs (5179776 bytes/sec)
>
> >Fix:
> I attach a patch that fixes the problem.
>
> Patch attached with submission follows:
>
> --- tee.c 2012/02/08 14:50:10 1.1
> +++ tee.c 2012/02/08 14:59:10
> @@ -46,8 +46,10 @@
> #endif /* not lint */
>
> #include <sys/types.h>
> +#include <sys/select.h>
> #include <sys/stat.h>
> #include <err.h>
> +#include <errno.h>
> #include <fcntl.h>
> #include <signal.h>
> #include <stdio.h>
> @@ -64,6 +66,7 @@
>
> void add(int, const char *);
> static void usage(void);
> +static void waitfor(int fd);
>
> int
> main(int argc, char *argv[])
> @@ -110,9 +113,14 @@
> bp = buf;
> do {
> if ((wval = write(p->fd, bp, n)) == -1) {
> - warn("%s", p->name);
> - exitval = 1;
> - break;
> + if (errno == EAGAIN) {
> + waitfor(p->fd);
> + wval = 0;
> + } else {
> + warn("%s", p->name);
> + exitval = 1;
> + break;
> + }
> }
> bp += wval;
> } while (n -= wval);
> @@ -141,3 +149,15 @@
> p->next = head;
> head = p;
> }
> +
> +/* Wait for the specified fd to be ready for writing */
> +static void
> +waitfor(int fd)
> +{
> + fd_set writefds;
> +
> + FD_ZERO(&writefds);
> + FD_SET(fd, &writefds);
> + if (select(fd + 1, NULL, &writefds, NULL, NULL) == -1)
> + err(1, "select");
> +}
>
>
> >Release-Note:
> >Audit-Trail:
> >Unformatted:
> _______________________________________________
> freebsd-bugs@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-bugs
> To unsubscribe, send any mail to "freebsd-bugs-unsubscribe@freebsd.org"
--
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Martin Cracauer <cracauer@cons.org> http://www.cons.org/cracauer/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201202101920.q1AJKCtb081889>