From owner-freebsd-current@FreeBSD.ORG Thu Aug 19 16:16:58 2010 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CA39F1065693 for ; Thu, 19 Aug 2010 16:16:58 +0000 (UTC) (envelope-from alc@cs.rice.edu) Received: from mail.cs.rice.edu (mail.cs.rice.edu [128.42.1.31]) by mx1.freebsd.org (Postfix) with ESMTP id 8FF838FC2A for ; Thu, 19 Aug 2010 16:16:58 +0000 (UTC) Received: from mail.cs.rice.edu (localhost.localdomain [127.0.0.1]) by mail.cs.rice.edu (Postfix) with ESMTP id 3FAE62C2B58; Thu, 19 Aug 2010 10:51:40 -0500 (CDT) X-Virus-Scanned: by amavis-2.4.0 at mail.cs.rice.edu Received: from mail.cs.rice.edu ([127.0.0.1]) by mail.cs.rice.edu (mail.cs.rice.edu [127.0.0.1]) (amavisd-new, port 10024) with LMTP id JXL+S8p4evpY; Thu, 19 Aug 2010 10:51:32 -0500 (CDT) Received: from adsl-216-63-78-18.dsl.hstntx.swbell.net (adsl-216-63-78-18.dsl.hstntx.swbell.net [216.63.78.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.cs.rice.edu (Postfix) with ESMTP id 183CB2C2A81; Thu, 19 Aug 2010 10:51:32 -0500 (CDT) Message-ID: <4C6D5302.4030602@cs.rice.edu> Date: Thu, 19 Aug 2010 10:51:30 -0500 From: Alan Cox User-Agent: Thunderbird 2.0.0.24 (X11/20100725) MIME-Version: 1.0 To: Dimitry Andric References: <4C6505A4.9060203@FreeBSD.org> <20100813085235.GA16268@freebsd.org> <4C66C010.3040308@FreeBSD.org> <4C673F02.8000805@FreeBSD.org> <20100815013438.GA8958@troutmask.apl.washington.edu> <4C67492C.5020206@FreeBSD.org> <8639ufd78w.fsf@ds4.des.no> <4C6844D8.5070602@andric.com> <86sk2faqdl.fsf@ds4.des.no> <4C6AAA88.5080606@andric.com> <4C6AF13A.1080606@andric.com> <4C6D3BBB.7030104@andric.com> In-Reply-To: <4C6D3BBB.7030104@andric.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: alc@freebsd.org, current@freebsd.org Subject: Re: Official request: Please make GNU grep the default X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Aug 2010 16:16:58 -0000 Dimitry Andric wrote: > On 2010-08-17 23:24, Alan Cox wrote: > >>> So normal mmap is ~3% slower, and prefault mmap does not seem to make >>> any measurable difference. I guess the added complexity is not really >>> worth it, for now. >>> >> Do you know what fraction of this time is being spent in the kernel? >> > > I ran 100 trials again, but now using "time -a -o logfile", so I could > run ministat over the accumulated results. This gives: > > x gnugrep > + bsdgrep-r210927 (the initial version that started this thread) > * bsdgrep-r211490 (current version) > % bsdgrep-r211490-mmap-plain > # bsdgrep-r211490-mmap-prefault > > Real time: > N Min Max Median Avg Stddev > x 100 1.15 1.98 1.18 1.2122 0.11159613 > + 100 8.57 14.26 8.79 9.1823 1.0496126 > * 100 2.81 6.57 2.91 3.0189 0.4304259 > % 100 2.34 4.03 2.99 3.0022 0.12635992 > # 100 2.85 3.49 2.88 2.8981 0.075232904 > > User time: > N Min Max Median Avg Stddev > x 100 0 0.07 0.03 0.0239 0.015627934 > + 100 1.6 3.33 1.9 1.976 0.30264824 > * 100 0.29 1 0.39 0.4004 0.08696824 > % 100 1.8 3.56 2.73 2.7274 0.13260117 > # 100 2.78 3.04 2.81 2.8238 0.04039652 > > System time: > N Min Max Median Avg Stddev > x 100 1.08 1.91 1.15 1.1809 0.10953617 > + 100 6.55 10.9 6.94 7.1905 0.77911809 > * 100 2.38 5.5 2.53 2.6061 0.35068445 > % 100 0.18 0.53 0.25 0.2645 0.053586049 > # 100 0.03 0.54 0.06 0.0668 0.052259647 > > E.g. it looks like bsdgrep with 'plain' mmap performs almost the same > as the regular bsdgrep (both around 3.0s average), but with mmap much > more of the time is spent in user mode. > > That makes sense to me. With traditional I/O, such as read(2), the copyout to user space fills the processor's data cache with the data to be processed. Grep's core algorithm in user space shouldn't be experiencing cache misses to obtain the data. By and large, the cache misses will have occurred in the kernel. However, once you switch to mmap(2), the kernel never touches the data, and all cache misses occur in user space. You ought to be able to confirm this with pmcstat's sampling mode set to sample L2 cache misses. Here is what actually puzzles me about these results. With traditional I/O, even after the optimizations to bsdgrep, the system time for gnugrep is still less than half that of the optimized bsdgrep. I haven't looked at the changes, but I would have thought the system time for gnugrep and bsdgrep would be almost the same. > And it seems prefaulting does help now! I guess it also makes sense to > add madvise(..., MADV_SEQUENTIAL)? > > This won't matter as long as you are working with memory resident files. With a memory resident file, it would only be a waste of cycles. > >> Does >> the value of "sysctl vm.pmap.pde.mappings" increase as a result of your >> test? If not, there is still room for improvement in the results with >> mmap(). >> > > It always stays at 0, I have never seen any other value. > Addressing this issue would mostly affect the system time, which is already tiny for mmap-prefault, so I wouldn't be concerned about this (yet). Did you ever describe your test machine? If so, I'm sorry, but I missed that. Is it running an amd64 or i386 kernel? Can you briefly describe what kind of processor and memory it has? Regards, Alan