From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 8 22:33:00 2007 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B834F16A41B for ; Thu, 8 Nov 2007 22:33:00 +0000 (UTC) (envelope-from randyhyde@earthlink.net) Received: from elasmtp-dupuy.atl.sa.earthlink.net (elasmtp-dupuy.atl.sa.earthlink.net [209.86.89.62]) by mx1.freebsd.org (Postfix) with ESMTP id 7727813C4A8 for ; Thu, 8 Nov 2007 22:32:57 +0000 (UTC) (envelope-from randyhyde@earthlink.net) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=dk20050327; d=earthlink.net; b=fUilTAh+Ih9U5eJeeYvFWnxSSS9LMD/tnKFAzfe08XfmkKzyF80d1ud38e0HSITs; h=Received:Message-ID:From:To:Subject:Date:MIME-Version:Content-Type:Content-Transfer-Encoding:X-Priority:X-MSMail-Priority:X-Mailer:X-MimeOLE:X-ELNK-Trace:X-Originating-IP; Received: from [66.215.252.78] (helo=pentiv) by elasmtp-dupuy.atl.sa.earthlink.net with asmtp (Exim 4.34) id 1IqFWr-0006gA-Ox for freebsd-hackers@freebsd.org; Thu, 08 Nov 2007 17:07:13 -0500 Message-ID: <000701c82253$b3a8c030$6302a8c0@pentiv> From: "Randall Hyde" To: Date: Thu, 8 Nov 2007 14:07:06 -0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1807 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1896 X-ELNK-Trace: eba5e0c9192a36dcd6dd28457998182d7e972de0d01da9408dd5772cef89dbdcea4ff6cb890bde0f350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c X-Originating-IP: 66.215.252.78 Subject: Some FreeBSD performance Issues X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2007 22:33:00 -0000 Hi All, I recently ported my HLA (High Level Assembler) compiler to FreeBSD and, along with it, the HLA Standard Library. I have a performance-related question concerning file I/O. It appears that character-at-a-time file I/O is *exceptionally* slow. Yes, I realize that when processing large files I really ought to be doing block/buffered I/O to get the best performance, but for certain library routines I've written it's been far more convenient to do character-at-a-time I/O rather than deal with all the buffering issues. In the past, while slower, this character-at-a-time paradigm has provided reasonable, though not stellar, performance under Windows and Linux. However, with the port to FreeBSD I'm seeing a three-orders-of-magnitude performance loss. Here's my little test program: program t; #include( "stdlib.hhf" ) //#include( "bsd.hhf" ) static f :dword; buffer :char[64*1024]; begin t; fileio.open( "socket.h", fileio.r ); mov( eax, f ); #if( false ) // Windows: 0.25 seconds // BSD: 5.2 seconds while( !fileio.eof( f )) do fileio.getc( f ); //stdout.put( (type char al )); endwhile; #elseif( false ) // Windows: 0.0 seconds (below 1ms threshold) // BSD: 5.2 seconds forever fileio.read( f, buffer, 1 ); breakif( eax <> 1 ); //stdout.putc( buffer[0] ); endfor; #elseif( false ) // BSD: 5.1 seconds forever bsd.read( f, buffer, 1 ); breakif( @c ); breakif( eax <> 1 ); //stdout.putc( buffer[0] ); endfor; #else // BSD: 0.016 seconds bsd.read( f, buffer, 64*1024 ); //stdout.write( buffer, eax ); #endif fileio.close( f ); end t; (I selectively set one of the conditionals to true to run a different test; yeah, this is HLA assembly code, but I suspect that most people who can read C can *mostly* figure out what's going on here). The "fileio.open" call is basically a bsd.open( "socket.h", bsd.O_RDONLY ); API call. The socket.h file is about 19K long (it's from the FreeBSD include file set). In particular, I would draw your attention to the first two tests that do character-at-a-time I/O. The difference in performance between Windows and FreeBSD is dramatic (note: Linux numbers are comparable to Windows). Just to make sure that the library code wasn't doing something incredibly stupid, the third test makes a direct FreeBSD API call to read the data a byte at a time -- the results are comparable to the first two tests. Finally, I read the whole file at once, just to make sure the problem was character-at-a-time I/O (which obviously is the problem). Naturally, at one point I'd uncommented all the output statements to verify that I was reading the entire file -- no problem there. Is this really the performance I can expect from FreeBSD when doing character I/O this way? Is is there some tuning parameter I can set to change internal buffering or something? From this numbers, if I had to guess, I'd suspect that FreeBSD was re-reading the entire 4K (or whatever) block from the file cache everytime I read a single character. Can anyone explain what's going on here? I'm loathe to change my fileio module to add buffering as that will create some subtle semantic differences that could break existing code (I do have an object-oriented file I/O class that I'm going to use to implement buffered I/O, I would prefer to leave the fileio module unbuffered, if possible). And a more general question: if this is the way FreeBSD works, should something be done about it? Thanks, Randy Hyde