Date: Fri, 20 Feb 2009 22:09:12 +0100 From: Pieter de Goeje <pieter@degoeje.nl> To: freebsd-questions@freebsd.org Cc: Junsuk Shin <junsukshin@gmail.com> Subject: Re: read() vs fread() Message-ID: <200902202209.12765.pieter@degoeje.nl> In-Reply-To: <7873ac110902201207t61038469o6a94d81bcd392941@mail.gmail.com> References: <7873ac110902201207t61038469o6a94d81bcd392941@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Friday 20 February 2009 21:07:57 Junsuk Shin wrote:
> Hi BSD guys,
>
> While I was doing simple file read test, I found that there is a huge
> difference in file read performance between read() and fread(). I'm
> wondering if I'm doing something wrong or if someone has experienced
> similar things.
>
> Here is what I did,
>
> For the specific application, I need to bypass cache (I read only
> once, and that's all)
> The test file is 700Mbytes dummy file.
> Test app just reads the whole file.
>
> Test is done on FreeBSD 7.1 amd 64, Celeron E1200, WD Caviar SE16 SATA 7200
> RPM
>
> For test 1,
>
> fd = open(name, O_RDONLY | O_DIRECT);
> while(...) {
> cnt = read(....);
> ....
> }
>
> for test 2,
>
> fd = open(name, O_RDONLY | O_DIRECT);
> file = fdopen(fd,"r");
> while(...) {
> cnt = fread(....);
> ....
> }
>
> test 1 takes about 11.64 seconds (63 MBytes/s), and test 2 takes about
> 51.53 seconds (14 MBytes/s)
>
> If I use the pair of fopen() and fread(), it will have cache effect,
> so the result doesn't say much of hdd performance.
>
> Personally, I don't think the overhead of fread() (wrapper in libc) is
> that huge. What would be the reason for this?
The reason is that by default a FILE has a really small internal buffer. Take
a look at gstat(8) while running the test: you can clearly see an insane
amount of I/O requests being done (almost 5000 reads per second on my HDD).
To solve this call setvbuf(3):
setvbuf(file, buf, _IOFBF, bufsize);
A bufsize of 16k or bigger should help a lot. After this modification, I see
about 900 reads per second (using bufsize = 64k) and the read speed is equal
to the read(2) case.
Regards,
Pieter de Goeje
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200902202209.12765.pieter>