From owner-freebsd-current@FreeBSD.ORG Mon Feb 2 14:06:39 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9A82216A4CE for ; Mon, 2 Feb 2004 14:06:39 -0800 (PST) Received: from mail3.speakeasy.net (mail3.speakeasy.net [216.254.0.203]) by mx1.FreeBSD.org (Postfix) with ESMTP id EDAB843D1F for ; Mon, 2 Feb 2004 14:06:10 -0800 (PST) (envelope-from jmg@hydrogen.funkthat.com) Received: (qmail 30480 invoked from network); 2 Feb 2004 22:06:10 -0000 Received: from dsl017-045-168.spk4.dsl.speakeasy.net (HELO hydrogen.funkthat.com) ([69.17.45.168]) (envelope-sender ) by mail3.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 2 Feb 2004 22:06:10 -0000 Received: from hydrogen.funkthat.com (qdovsj@localhost.funkthat.com [127.0.0.1])i12M667Y039631; Mon, 2 Feb 2004 14:06:08 -0800 (PST) (envelope-from jmg@hydrogen.funkthat.com) Received: (from jmg@localhost) by hydrogen.funkthat.com (8.12.10/8.12.10/Submit) id i12M64T6039630; Mon, 2 Feb 2004 14:06:04 -0800 (PST) Date: Mon, 2 Feb 2004 14:06:04 -0800 From: John-Mark Gurney To: Dan Nelson Message-ID: <20040202220604.GE572@funkthat.com> Mail-Followup-To: Dan Nelson , Ian Freislich , Poul-Henning Kamp , current@freebsd.org References: <98907.1074546817@critter.freebsd.dk> <20040122180918.GA94901@dan.emsphone.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040122180918.GA94901@dan.emsphone.com> User-Agent: Mutt/1.4.1i X-Operating-System: FreeBSD 4.2-RELEASE i386 X-PGP-Fingerprint: B7 EC EF F8 AE ED A7 31 96 7A 22 B3 D8 56 36 F4 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html cc: Ian Freislich cc: Poul-Henning Kamp cc: current@freebsd.org Subject: Re: Is BUFSIZ too small ? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: John-Mark Gurney List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Feb 2004 22:06:39 -0000 Dan Nelson wrote this message on Thu, Jan 22, 2004 at 12:09 -0600: > > > I share many of your doubts, but I would still like to see some > > > benchmarks :-) > > > > Perhaps ftp is one of those things that uses BUFSIZ for the actual > > I/O ops. All of it's reads and writes if you truss it are 1024 bytes > > which impacts its performance (here at least). > > Yeah, it's not so much stdio's use of BUFSIZ, it's other applications > using it for their preferred I/O size. I upped the buffer size in ftpd > locally because of this. There are a lot of references to BUFSIZ in > the base system's code, but they're mainly just for reading in a config > file, for example, or misused as sizing a filename buffer. ftpd and > lpr jumped out as really wanting larger I/O sizes. I had a k5/90 or so that was seriously limited by lpr's 1k buffer size.. I did some experiments with dd on obtaining an ideal block size, and I came up with about 16kb last time I ran it.. This is the standard dd if=/dev/zero of=/dev/null bs=xK style testing... gen,ttyp4,~,505$time dd if=/dev/zero of=/dev/null bs=2k count=500000 1024000000 bytes transferred in 1.778601 secs (575733345 bytes/sec) 1.78 real 0.13 user 1.64 sys hydrogen,ttyp4,~,507$time dd if=/dev/zero of=/dev/null bs=8k count=125000 1024000000 bytes transferred in 0.673048 secs (1521437302 bytes/sec) 0.67 real 0.02 user 0.65 sys hydrogen,ttyp4,~,508$time dd if=/dev/zero of=/dev/null bs=16k count=62500 1024000000 bytes transferred in 0.498403 secs (2054562956 bytes/sec) 0.50 real 0.00 user 0.49 sys hydrogen,ttyp4,~,509$time dd if=/dev/zero of=/dev/null bs=32k count=31250 1024000000 bytes transferred in 0.408496 secs (2506757087 bytes/sec) 0.41 real 0.00 user 0.40 sys hydrogen,ttyp4,~,510$time dd if=/dev/zero of=/dev/null bs=64k count=$((31250/2)) 1024000000 bytes transferred in 0.425369 secs (2407321506 bytes/sec) 0.42 real 0.00 user 0.41 sys Looks like it's now 32k in size.. Sure this isn't very scientific, but it does show how the overhead of syscalls effect performance... This was done on a: CPU: AMD Duron(tm) Processor (1211.96-MHz 686-class CPU) so, this might just be l1/l2 cache being stressed, but it does show that read/writes of smaller block sizes significantly impact performance... It's hard to choose a value, since larger values impact memory usage for apps that may not benifit as much, but also in the days of cheap memory, this isn't such a bad thing... I'd say 16k is a pretty good number, but it is very arbitrary.. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."