From owner-freebsd-stable@FreeBSD.ORG Fri Jan 15 02:56:56 2010 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 984AA106568D for ; Fri, 15 Jan 2010 02:56:56 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.freebsd.org (Postfix) with ESMTP id 5DBE28FC0A for ; Fri, 15 Jan 2010 02:56:55 +0000 (UTC) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.14.2/8.14.1) with ESMTP id o0F2uhR9080844; Thu, 14 Jan 2010 18:56:44 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.14.2/8.13.4/Submit) id o0F2ucOx080837; Thu, 14 Jan 2010 18:56:38 -0800 (PST) Date: Thu, 14 Jan 2010 18:56:38 -0800 (PST) From: Matthew Dillon Message-Id: <201001150256.o0F2ucOx080837@apollo.backplane.com> To: "Mikhail T." References: <200603232352.k2NNqPS8018729@gate.bitblocks.com> <200603241518.01027.mi+mx@aldan.algebra.com> <20060325103927.GE703@turion.vk2pj.dyndns.org> <200603250920.14208@aldan> <20060325190333.GD7001@funkthat.com> <4B4F7CF5.4040307@aldan.algebra.com> Cc: alc@freebsd.org, Peter Jeremy , stable@freebsd.org Subject: Re: An old gripe: Reading via mmap stinks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jan 2010 02:56:56 -0000 : mmap: 43.400u 9.439s 2:35.19 34.0% 16+184k 0+0io 106994pf+0w : read: 41.358u 23.799s 2:12.04 49.3% 16+177k 67677+0io 0pf+0w : :Observe, that even though read-ing is quite taxing on the kernel (high :sys-time), the mmap-ing loses overall -- at least, on an otherwise idle :system -- because read gets the full throughput of the drive (systat -vm :shows 100% disk utilization), while pagefaulting gets only about 69%. : :When I last brought this up in 2006, it was "revealed", that read(2) :uses heuristics to perform a read-ahead. Why can't the pagefaulting-in :implementation use the same or similar "trickery" was never explained... Well, the VM system does do read-ahead, but clearly the pipelining is not working properly because if it were then either the cpu or the disk would be pegged, and neither is. It's broken in DFly too. Both FreeBSD and DragonFly use vnode_pager_generic_getpages() (UFS's ffs_getpages() just calls the generic) which means (typically) the whole thing devolves into a UIO_NOCOPY VOP_READ(). The VOP_READ should be doing read-ahead based on the sequential access heuristic but I already see issues in both implementations of vnode_pager_generic_getpages() where it finds a valid page from an earlier read-ahead and stops (failing to issue any new read-aheads because it fails to issue a new UIO_NOCOPY VOP_READ... doh!). This would explain why the performance is not as bad as linux but is not as good as a properly pipelined case. I'll play with it some in DFly and I'm sure the FreeBSD folks can fix it in FreeBSD. -Matt