From owner-freebsd-current@FreeBSD.ORG Wed Aug 3 14:16:52 2005 Return-Path: X-Original-To: current@freebsd.org Delivered-To: freebsd-current@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3A1CA16A41F for ; Wed, 3 Aug 2005 14:16:52 +0000 (GMT) (envelope-from rizzo@icir.org) Received: from xorpc.icir.org (xorpc.icir.org [192.150.187.68]) by mx1.FreeBSD.org (Postfix) with ESMTP id E790943D5E for ; Wed, 3 Aug 2005 14:16:51 +0000 (GMT) (envelope-from rizzo@icir.org) Received: from xorpc.icir.org (localhost [127.0.0.1]) by xorpc.icir.org (8.12.11/8.12.11) with ESMTP id j73EGpWH084621; Wed, 3 Aug 2005 07:16:51 -0700 (PDT) (envelope-from rizzo@xorpc.icir.org) Received: (from rizzo@localhost) by xorpc.icir.org (8.12.11/8.12.3/Submit) id j73EGpYS084620; Wed, 3 Aug 2005 07:16:51 -0700 (PDT) (envelope-from rizzo) Date: Wed, 3 Aug 2005 07:16:51 -0700 From: Luigi Rizzo To: Maxim.Sobolev@portaone.com Message-ID: <20050803071651.A84608@xorpc.icir.org> References: <42F0CCD5.9090200@portaone.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <42F0CCD5.9090200@portaone.com>; from sobomax@portaone.com on Wed, Aug 03, 2005 at 04:55:33PM +0300 Cc: "current@freebsd.org" Subject: Re: Sub-optimal libc's read-ahead buffering behaviour X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Aug 2005 14:16:52 -0000 On Wed, Aug 03, 2005 at 04:55:33PM +0300, Maxim Sobolev wrote: > Hi, > > I have found the scenario in which our libc behaves utterly > suboptimally. Consider the following piece of code reads and processes > every other 512-bytes block in a file (error handling intentionally apparently the issue is in fseek(), you should try to track why fseeko() it goes to the 'dumb:' label where it simply discards the buffer and does the seek... cheers luigi > omitted): > > FILE *f; > int i; > char buf[512]; > > f = fopen(...); > for (i = 0; feof(f) == 0; i++) { > fread(buf, sizeof(buf), 1, f); > do_process(buf); > fseek(f, i * 2 * sizeof(buf), SEEK_SET); > } > > What I have discovered in this case is that libc reads 4096 bytes from > the file for *each* fread(3) call, despite the fact that it can only do > one actual read(2) for every fourth fread(3) and satisfy the rest from > the internal buffer (4096 bytes). However, if I replace fseek(3) with > just another dummy fread(3) everything works as expected - libc does > only one read for every 8 fread(3) calls (4 dummy and 4 real). > > Is it something which should be fixed or are there some subtle reasons > for the current behaviour? > > Following is piece of code which illustrates the problem: > > #include > #include > > int > main(int argc, char **argv) > { > FILE *f; > int i; > char buf[512]; > > f = fopen("/dev/zero", "r"); > for (i = 0; i < 16; i++) { > fread(buf, sizeof(buf), 1, f); > if (argc == 1) > fread(buf, sizeof(buf), 1, f); > else > fseek(f, i * 2 * sizeof(buf), SEEK_SET); > } > exit(0); > } > > When run with zero arguments relevant truss output looks like: > > open("/dev/zero",0x0,0666) = 3 (0x3) > fstat(3,0xbfbfe900) = 0 (0x0) > readlink("/etc/malloc.conf",0xbfbfe8c0,63) ERR#2 'No such file or > directory' > issetugid() = 0 (0x0) > mmap(0x0,4096,(0x3)PROT_READ|PROT_WRITE,(0x1002)MAP_ANON|MAP_PRIVATE,-1,0x0) > = 1209335808 (0x48150000) > break(0x804b000) = 0 (0x0) > break(0x804c000) = 0 (0x0) > ioctl(3,TIOCGETA,0xbfbfe940) ERR#19 'Operation not > supported by device' > read(0x3,0x804b000,0x1000) = 4096 (0x1000) > read(0x3,0x804b000,0x1000) = 4096 (0x1000) > read(0x3,0x804b000,0x1000) = 4096 (0x1000) > read(0x3,0x804b000,0x1000) = 4096 (0x1000) > exit(0x0) > > While when I am specifying some argument it becomes: > > open("/dev/zero",0x0,0666) = 3 (0x3) > fstat(3,0xbfbfe900) = 0 (0x0) > readlink("/etc/malloc.conf",0xbfbfe8c0,63) ERR#2 'No such file or > directory' > issetugid() = 0 (0x0) > mmap(0x0,4096,(0x3)PROT_READ|PROT_WRITE,(0x1002)MAP_ANON|MAP_PRIVATE,-1,0x0) > = 1209335808 (0x48150000) > break(0x804b000) = 0 (0x0) > break(0x804c000) = 0 (0x0) > ioctl(3,TIOCGETA,0xbfbfe940) ERR#19 'Operation not > supported by device' > read(0x3,0x804b000,0x1000) = 4096 (0x1000) > lseek(3,0x0,SEEK_SET) = 0 (0x0) > read(0x3,0x804b000,0x1000) = 4096 (0x1000) > lseek(3,0x400,SEEK_SET) = 1024 (0x400) > read(0x3,0x804b000,0x1000) = 4096 (0x1000) > lseek(3,0x800,SEEK_SET) = 2048 (0x800) > read(0x3,0x804b000,0x1000) = 4096 (0x1000) > lseek(3,0xc00,SEEK_SET) = 3072 (0xc00) > read(0x3,0x804b000,0x1000) = 4096 (0x1000) > lseek(3,0x1000,SEEK_SET) = 4096 (0x1000) > read(0x3,0x804b000,0x1000) = 4096 (0x1000) > lseek(3,0x1400,SEEK_SET) = 5120 (0x1400) > read(0x3,0x804b000,0x1000) = 4096 (0x1000) > lseek(3,0x1800,SEEK_SET) = 6144 (0x1800) > read(0x3,0x804b000,0x1000) = 4096 (0x1000) > lseek(3,0x1c00,SEEK_SET) = 7168 (0x1c00) > read(0x3,0x804b000,0x1000) = 4096 (0x1000) > lseek(3,0x2000,SEEK_SET) = 8192 (0x2000) > read(0x3,0x804b000,0x1000) = 4096 (0x1000) > lseek(3,0x2400,SEEK_SET) = 9216 (0x2400) > read(0x3,0x804b000,0x1000) = 4096 (0x1000) > lseek(3,0x2800,SEEK_SET) = 10240 (0x2800) > read(0x3,0x804b000,0x1000) = 4096 (0x1000) > lseek(3,0x2c00,SEEK_SET) = 11264 (0x2c00) > read(0x3,0x804b000,0x1000) = 4096 (0x1000) > lseek(3,0x3000,SEEK_SET) = 12288 (0x3000) > read(0x3,0x804b000,0x1000) = 4096 (0x1000) > lseek(3,0x3400,SEEK_SET) = 13312 (0x3400) > read(0x3,0x804b000,0x1000) = 4096 (0x1000) > lseek(3,0x3800,SEEK_SET) = 14336 (0x3800) > read(0x3,0x804b000,0x1000) = 4096 (0x1000) > lseek(3,0x3c00,SEEK_SET) = 15360 (0x3c00) > exit(0x0) > > The output speaks for itself (32 syscalls instead of 4)! > > -Maxim > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"