From owner-freebsd-questions@FreeBSD.ORG Fri Feb 20 23:01:53 2009 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D1F1F106564A for ; Fri, 20 Feb 2009 23:01:53 +0000 (UTC) (envelope-from junsukshin@gmail.com) Received: from mail-gx0-f176.google.com (mail-gx0-f176.google.com [209.85.217.176]) by mx1.freebsd.org (Postfix) with ESMTP id 8B4AB8FC14 for ; Fri, 20 Feb 2009 23:01:53 +0000 (UTC) (envelope-from junsukshin@gmail.com) Received: by mail-gx0-f176.google.com with SMTP id 24so3941603gxk.19 for ; Fri, 20 Feb 2009 15:01:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=zqSOEUjD1162nN7jF4AzCRJl8fSE9Ge6vfFP9pVLV28=; b=MDpPr//sj2KcDxeNt9jgb06FieeENJbebf331X6M3yo07WpFQ8wp9onrODN3/uO/Et gcNwVa83YE7vaN8nfIHXnWKpjYxTj/IxGC5rOat9xHAWUEV11hE3+IgvUh7OTxrdKLe5 xVvLcfHoeXR834vWlWC1QDrgY4E7cQWWWB8oU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=SDl1WGf6f8fF+DDKvq1Vz0+sc8svwt/BT9IKYl7mkaCXJ3N2hSUDoB6uHU96qKuPHM IpGxzxLASfdQh6eGLQPy9q7vRCRL5l/F2E8B9M+DzF/vZ0yPAFQK3yUeJ1+FDmbybTn0 xgAJgnJrDvTwYK+//7mDC4pmK40ZuL0earIQg= MIME-Version: 1.0 Received: by 10.231.19.72 with SMTP id z8mr1659901iba.6.1235170913084; Fri, 20 Feb 2009 15:01:53 -0800 (PST) In-Reply-To: <200902202209.12765.pieter@degoeje.nl> References: <7873ac110902201207t61038469o6a94d81bcd392941@mail.gmail.com> <200902202209.12765.pieter@degoeje.nl> Date: Fri, 20 Feb 2009 18:01:53 -0500 Message-ID: <7873ac110902201501l74a79ffyfca138331386c317@mail.gmail.com> From: Junsuk Shin To: Pieter de Goeje , freebsd-questions@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: Re: read() vs fread() X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Feb 2009 23:01:54 -0000 setvbuf(file, buf, _IOFBF, bufsize) solved the problem perfectly. Thanks a lot. On Fri, Feb 20, 2009 at 4:09 PM, Pieter de Goeje wrote: > > On Friday 20 February 2009 21:07:57 Junsuk Shin wrote: > > Hi BSD guys, > > > > While I was doing simple file read test, I found that there is a huge > > difference in file read performance between read() and fread(). I'm > > wondering if I'm doing something wrong or if someone has experienced > > similar things. > > > > Here is what I did, > > > > For the specific application, I need to bypass cache (I read only > > once, and that's all) > > The test file is 700Mbytes dummy file. > > Test app just reads the whole file. > > > > Test is done on FreeBSD 7.1 amd 64, Celeron E1200, WD Caviar SE16 SATA 7200 > > RPM > > > > For test 1, > > > > fd = open(name, O_RDONLY | O_DIRECT); > > while(...) { > > cnt = read(....); > > .... > > } > > > > for test 2, > > > > fd = open(name, O_RDONLY | O_DIRECT); > > file = fdopen(fd,"r"); > > while(...) { > > cnt = fread(....); > > .... > > } > > > > test 1 takes about 11.64 seconds (63 MBytes/s), and test 2 takes about > > 51.53 seconds (14 MBytes/s) > > > > If I use the pair of fopen() and fread(), it will have cache effect, > > so the result doesn't say much of hdd performance. > > > > Personally, I don't think the overhead of fread() (wrapper in libc) is > > that huge. What would be the reason for this? > > The reason is that by default a FILE has a really small internal buffer. Take > a look at gstat(8) while running the test: you can clearly see an insane > amount of I/O requests being done (almost 5000 reads per second on my HDD). > To solve this call setvbuf(3): > > setvbuf(file, buf, _IOFBF, bufsize); > > A bufsize of 16k or bigger should help a lot. After this modification, I see > about 900 reads per second (using bufsize = 64k) and the read speed is equal > to the read(2) case. > > Regards, > > Pieter de Goeje -- Junsuk