Date: Tue, 10 Jan 2012 01:43:28 +1100 (EST) From: Bruce Evans <brde@optusnet.com.au> To: Bruce Evans <brde@optusnet.com.au> Cc: Marc Olzheim <marcolz@stack.nl>, Garrett Cooper <yanegomi@gmail.com>, freebsd-performance@freebsd.org, Dieter BSD <dieterbsd@engineer.com> Subject: Re: cmp(1) has a bottleneck, but where? Message-ID: <20120110013455.D2530@besplex.bde.org> In-Reply-To: <20120104000111.K6684@besplex.bde.org> References: <20120103073736.218240@gmx.com> <CAGH67wQXuMasyc9BE8M9fHsQv6d2zdRxDQ2ekX4whjHJFyqZyg@mail.gmail.com> <20120103083454.GA22673@zlo.nu> <20120104000111.K6684@besplex.bde.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, 4 Jan 2012, Bruce Evans wrote:
> On Tue, 3 Jan 2012, Marc Olzheim wrote:
>
>> On Tue, Jan 03, 2012 at 12:21:10AM -0800, Garrett Cooper wrote:
>>> The file is 3.0GB in size. Look at all those page faults though!
>>> Thanks!
>>> -Garrett
>>
>> From usr.bin/cmp/c_regular.c:
>>
>> #define MMAP_CHUNK (8*1024*1024)
>> ...
>> for (..) {
>> mmap() chunk of size MMAP_CHUNK.
>> compare
>> munmap()k
>> }
>>
>> That 8 MB chunk size sounds like a bad plan to me. I can imagine
>> something needed to be done to compare files larger than X GB on a 32bit
>> system, but 8MB is pretty small...
>
> 8MB is more than large enough. It works at disk speed in my tests. cp
> still uses this value. Old versions of cmp used the bogus value of
> ...
> In my tests, using "-" for one of the files mainly takes lots more user
> time. It only reduces the real time by 25%. This is on a core2. On
> a system with a slow CPU, it is easy for getc() to be much slower than
> the disk.
More careful tests showed serious slowness when the combined file sizes
exceeded the cache size. cmp takes an enormous amount of CPU (see another
reply), and this seems to be done mostly in series with i/o, so the total
time increases too much. A smaller mmap() size or not using mmap() at
all might improve paralellism.
Bruce
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20120110013455.D2530>