Skip site navigation (1)Skip section navigation (2) 
Date:      Sat, 18 Oct 2008 23:16:44 +0200
From:      Ivan Voras <ivoras@freebsd.org>
To:        freebsd-hackers@freebsd.org
Subject:   Pipes, cat buffer size
Message-ID:  <gddjoj$apg$1@ger.gmane.org>
next in thread | raw e-mail | index | archive | help 
This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enigB92BE62FD97B29136A533951
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi,

I'm working on a program that's intended to be used as a "filter", as in
"something | myprogram > file". I'm trying it with cat and I'm seeing my
read()s return small blocks, 64 kB in size. I suppose this is because
cat writes in 64 kB blocks. So:

a) Is there a way to programatically, per-process, set the pipe buffer
size? The program in question is a compressor and it's particularly
inefficient when given small blocks and I'm wondering if the system can
buffer enough data for it.
b) Is there any objection to the following patch to cat:

Index: cat.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- cat.c       (revision 184033)
+++ cat.c       (working copy)
@@ -247,7 +247,16 @@
        if (buf =3D=3D NULL) {
                if (fstat(wfd, &sbuf))
                        err(1, "%s", filename);
-               bsize =3D MAX(sbuf.st_blksize, 1024);
+               if (S_ISREG(sbuf.st_mode)) {
+                       /* If there's plenty of RAM, use a 1 MB
+                        * copy buffer, else use a 128 kB buffer */
+                       if (sysconf(_SC_PHYS_PAGES) > 32768)
+                               bsize =3D 1*1024*1024;
+                       else
+                               bsize =3D 128*1024;
+               } else
+                       bsize =3D MAX(sbuf.st_blksize,
+                                       (blksize_t)sysconf(_SC_PAGESIZE))=
;
                if ((buf =3D malloc(bsize)) =3D=3D NULL)
                        err(1, "buffer");
        }

?


--------------enigB92BE62FD97B29136A533951
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkj6UkMACgkQldnAQVacBcimNQCfQKS3VLR5jRkGzBU8AuojtE6J
cHUAnilJd7PunT9jB7qjdn6j7OiE282f
=kCTP
-----END PGP SIGNATURE-----

--------------enigB92BE62FD97B29136A533951--

Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?gddjoj$apg$1>