From owner-freebsd-questions@FreeBSD.ORG Fri Feb 20 21:39:38 2009 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 73474106566B for ; Fri, 20 Feb 2009 21:39:38 +0000 (UTC) (envelope-from pieter@degoeje.nl) Received: from mx.utwente.nl (mx3.utsp.utwente.nl [130.89.2.14]) by mx1.freebsd.org (Postfix) with ESMTP id E8F348FC20 for ; Fri, 20 Feb 2009 21:39:37 +0000 (UTC) (envelope-from pieter@degoeje.nl) Received: from nox.student.utwente.nl (nox.student.utwente.nl [130.89.165.91]) by mx.utwente.nl (8.12.10/SuSE Linux 0.7) with ESMTP id n1KL9CIS032002; Fri, 20 Feb 2009 22:09:12 +0100 From: Pieter de Goeje To: freebsd-questions@freebsd.org Date: Fri, 20 Feb 2009 22:09:12 +0100 User-Agent: KMail/1.9.10 References: <7873ac110902201207t61038469o6a94d81bcd392941@mail.gmail.com> In-Reply-To: <7873ac110902201207t61038469o6a94d81bcd392941@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200902202209.12765.pieter@degoeje.nl> X-UTwente-MailScanner-Information: Scanned by MailScanner. Contact servicedesk@icts.utwente.nl for more information. X-UTwente-MailScanner: Found to be clean X-UTwente-MailScanner-SpamScore: s X-UTwente-MailScanner-From: pieter@degoeje.nl X-Spam-Status: No Cc: Junsuk Shin Subject: Re: read() vs fread() X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Feb 2009 21:39:38 -0000 On Friday 20 February 2009 21:07:57 Junsuk Shin wrote: > Hi BSD guys, > > While I was doing simple file read test, I found that there is a huge > difference in file read performance between read() and fread(). I'm > wondering if I'm doing something wrong or if someone has experienced > similar things. > > Here is what I did, > > For the specific application, I need to bypass cache (I read only > once, and that's all) > The test file is 700Mbytes dummy file. > Test app just reads the whole file. > > Test is done on FreeBSD 7.1 amd 64, Celeron E1200, WD Caviar SE16 SATA 7200 > RPM > > For test 1, > > fd = open(name, O_RDONLY | O_DIRECT); > while(...) { > cnt = read(....); > .... > } > > for test 2, > > fd = open(name, O_RDONLY | O_DIRECT); > file = fdopen(fd,"r"); > while(...) { > cnt = fread(....); > .... > } > > test 1 takes about 11.64 seconds (63 MBytes/s), and test 2 takes about > 51.53 seconds (14 MBytes/s) > > If I use the pair of fopen() and fread(), it will have cache effect, > so the result doesn't say much of hdd performance. > > Personally, I don't think the overhead of fread() (wrapper in libc) is > that huge. What would be the reason for this? The reason is that by default a FILE has a really small internal buffer. Take a look at gstat(8) while running the test: you can clearly see an insane amount of I/O requests being done (almost 5000 reads per second on my HDD). To solve this call setvbuf(3): setvbuf(file, buf, _IOFBF, bufsize); A bufsize of 16k or bigger should help a lot. After this modification, I see about 900 reads per second (using bufsize = 64k) and the read speed is equal to the read(2) case. Regards, Pieter de Goeje