Date: Mon, 10 Oct 2011 15:23:17 +0200 From: Jeremie Le Hen <jeremie@le-hen.org> To: freebsd-hackers@FreeBSD.org Cc: jhb@FreeBSD.org Subject: Re: [RFC] stdbuf: Force stdio's streams initial buffering mode (patch) Message-ID: <20111010132317.GA96131@felucia.tataz.chchile.org> In-Reply-To: <20110804171928.GB96031@felucia.tataz.chchile.org> References: <20110219185043.GA6573@felucia.tataz.chchile.org> <20110804171928.GB96031@felucia.tataz.chchile.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi, On Thu, Aug 04, 2011 at 07:19:28PM +0200, Jeremie Le Hen wrote: > On Sat, Feb 19, 2011 at 07:50:43PM +0100, Jeremie Le Hen wrote: > > I've been annoyed multiple time when running a command such like > > iostat -x 1 | grep -v ad10 | cat -n > > > > The problem stems from two factors: > > - grep's stdio sees that its stdout is not a terminal, so stdout is > > full buffered and not line-buffered; > > - iostat produces output too slowly so the aforementioned buffer takes > > numerous seconds to be filled and flushed to the last command. > > > > This problems is not specific to FreeBSD, it is actually a consequence > > of POSIX specification. I've checked this on Solaris and Linux. > > > > I've attached a small patch for stdio, so if the environment variable > > STDIO_IOLBF is set, the output streams will be line-oriented by default. > > iostat -x 1 | env STDIO_IOLBF=1 grep -v ad10 | cat -n > > I improved the whole picture. Now there is a shared library > libstdbuf.so which can be loaded with LD_PRELOAD and configured through > a environment variables. It is able to initial control buffering for > stdin, stdout and stderr (no buffering, line buffering, block > buffering). There is also an utility named stdbuf(1) which can be used > to run a command with the appropriate environment variables. Those are > named after a similar tool in recent versions of GNU coreutils; of > course, I also borrowed the interface for POLA. > > Here is how to use it (example taken from the manpage): > > vmstat 1 | stdbuf -o L awk '$2 > 1 || $3 > 1' | cat -n > > I think that using a preloaded shared library is better performance-wise > because libc doesn't bother looking up configuration variables in the > environment upon each execve(2), especially for something which is to be > rarely used. > > Manpages for both stdbuf(1) and libstdbuf(3) are provided. There is > surely room for improvement, so feel free to propose corrections to > them. > > The patch can be applied as-is I think, although I've developped it > against an old -CURRENT source tree (about 6 months ago). > > I'm looking for some feedback and review and hopefully this feature will > be committed for FreeBSD 9.0-RELEASE. It's now too late for 9.0-RELEASE :). But I'm still willing to see this feature committed eventually! While talking about this implementation with a friend, he told me that having a separate shared library sounds a little bit hackish and two or three getenv(3) in the libc startup path shouldn't eat too much CPU cycles. Any opinion on this? Anyone willing to commit this? Thanks. Regards, -- Jeremie Le Hen Men are born free and equal. Later on, they're on their own. Jean Yanne
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20111010132317.GA96131>