From owner-freebsd-hackers@FreeBSD.ORG Sat Oct 18 21:17:19 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9E242106568C for ; Sat, 18 Oct 2008 21:17:19 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 23B128FC16 for ; Sat, 18 Oct 2008 21:17:18 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1KrJAi-0002mO-Ke for freebsd-hackers@freebsd.org; Sat, 18 Oct 2008 21:17:16 +0000 Received: from 89-172-60-70.adsl.net.t-com.hr ([89.172.60.70]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 18 Oct 2008 21:17:16 +0000 Received: from ivoras by 89-172-60-70.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 18 Oct 2008 21:17:16 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Ivan Voras Date: Sat, 18 Oct 2008 23:16:44 +0200 Lines: 62 Message-ID: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigB92BE62FD97B29136A533951" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 89-172-60-70.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.17 (Windows/20080914) X-Enigmail-Version: 0.95.7 Sender: news Subject: Pipes, cat buffer size X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Oct 2008 21:17:19 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigB92BE62FD97B29136A533951 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi, I'm working on a program that's intended to be used as a "filter", as in "something | myprogram > file". I'm trying it with cat and I'm seeing my read()s return small blocks, 64 kB in size. I suppose this is because cat writes in 64 kB blocks. So: a) Is there a way to programatically, per-process, set the pipe buffer size? The program in question is a compressor and it's particularly inefficient when given small blocks and I'm wondering if the system can buffer enough data for it. b) Is there any objection to the following patch to cat: Index: cat.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- cat.c (revision 184033) +++ cat.c (working copy) @@ -247,7 +247,16 @@ if (buf =3D=3D NULL) { if (fstat(wfd, &sbuf)) err(1, "%s", filename); - bsize =3D MAX(sbuf.st_blksize, 1024); + if (S_ISREG(sbuf.st_mode)) { + /* If there's plenty of RAM, use a 1 MB + * copy buffer, else use a 128 kB buffer */ + if (sysconf(_SC_PHYS_PAGES) > 32768) + bsize =3D 1*1024*1024; + else + bsize =3D 128*1024; + } else + bsize =3D MAX(sbuf.st_blksize, + (blksize_t)sysconf(_SC_PAGESIZE))= ; if ((buf =3D malloc(bsize)) =3D=3D NULL) err(1, "buffer"); } ? --------------enigB92BE62FD97B29136A533951 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkj6UkMACgkQldnAQVacBcimNQCfQKS3VLR5jRkGzBU8AuojtE6J cHUAnilJd7PunT9jB7qjdn6j7OiE282f =kCTP -----END PGP SIGNATURE----- --------------enigB92BE62FD97B29136A533951--